I had the same problem on my 04 Taco and on my 99 4runner.
My solution wasn't pretty but it worked.
Two screws, one on either side of the ashtray...
..and a rubber band across the screws.
As we used to say in the Army, "If it's stupid but it works, it isn't stupid." If you're reluctant to drill holes in your dashboard, you could probably make a similar arrangement using velcro or double-stick tape.
Other option would be to just remove the ashtray.