Without bench testing it, it would be hard to guess what is going on. If the assembly is clean and not binding due to dry parts or fine grit but still does not respond, there is perhaps an electrical fault. I wouldn't expect such a fault to be cured by banging on the unit though.
Heat can also play a part in starter failure. If it fails when really hot, but works when cool, try to isolate the starter from the heat source, with a blanket or shield. Next time you take it apart, look at the tolerances between moving parts in the engagement device. If there are close tolerances between relatively heavy metal parts, you could be getting binding when parts grow due to heat, which might be mitigated by adding clearance where needed. This is hard to guess at without one in front of me, but take a look.
For reference, is this starter the one on your Nissan? If so, do you have the 4.0?