First off...why do you need access to a GPS and how often do you need to see it? Perhaps Jonathan Hansen will chime in since he's the authority on skinny boats.
Personally, I have used a GPS unit on my kayaks off and on. In Alaska it was helpful for longer crossings in foggy weather or on days with aggressive tide flows. I would use it to gauge speed and possible drift on those days. In Baja, I used it to help locate pre-selected beach sites. Outside of that? It was just another widget to keep me occupied.
For me, I usually just kept my GPS in my PFD pocket on a lanyard. I could pull it out, set in on my spray skirt and get a brief reading of speed and direction. On days navigating in thick fog in Alaska, my unit was attached to my deck bag with a peice of waterproof velcro. I even tried a similar approach with mounting my unit to my paddle. That actually worked pretty well when on approach to beach targets.
If this is for a true sea kayak, be cautious of extra deck mounted do-dads. I wouldn't put a Ram mount on a sea kayak. Just one more thing to get snagged.
For 75% off all kayaking scenarios, navigating by visual landmarks and a simple map is easiest.