I never really noticed it on mine until you brought it up, but I think I can remember mine doing that a little while driving around town. It wouldn't blow warm or anything, just not as cold every now and then. It definitely does not do it on the hwy though.
What year is your Tacoma?