Yes we do - that's the way it was designed.

You have to "freeze" your karma (karma vacation) to prevent it from falling.

I always forget to do that.