I would not even think of running any 5/16" line on a 12000 lb winch as the line would pull into itself under the load for sure.
My hydro will pull my 3/8" line into itself on heavy pulls, using a smaller diameter line would only make it worse. I have often though of making a move to 7/16" line just to improve the wind charachteristics under load. I have even thought of dropping back to like 60' of 1/2" for the same reason.
If you look at cranes they use much larger diameter line then they need because the larger diameter line when under load will wind onto the spool perfect.
My hydro will pull my 3/8" line into itself on heavy pulls, using a smaller diameter line would only make it worse. I have often though of making a move to 7/16" line just to improve the wind charachteristics under load. I have even thought of dropping back to like 60' of 1/2" for the same reason.
If you look at cranes they use much larger diameter line then they need because the larger diameter line when under load will wind onto the spool perfect.