Fear not - mine is the same. What you are hearing is the "Tilt Switch". When you remove the back cover, you will see a spherical bulb that contains a ball bearing. When the ball rolls off the bottom of the sphere, the system senses it's been tilted and cuts off the propane. I was concerned at first as well... but it's just a safety switch.
I actually used this to my advantage. I wired in a switch from the tilt switch wire terminals so I can choose when I want hot water or cold water. By flipping the switch, I trick the system into thinking it's been tilted. In this case, the pump still runs, but no propane is flowing and no ignitions occurs which gives me cold water. Turning the switch "off" returns the unit to it's factory setting.
-JM