I have an emotion stealth paddle kayak. I am from Oklahoma and it is often breezy so I use drift socks ( sea anchors) on anchor trolleys to drift the shorelines of lakes for bass, catfish and pan fish. I use a 30" or an 18" drift sock and when the wind nears 20 mph i use them both at once on the same side to slow the kayak down enough to fish well. I have the paddle crossways in my lap and make a stroke or two occasionally to move around a tree or point. Adjusting the anchor trolley allows me to keep the right distance and angle to shore. I have drift fished for 5 hrs without making 2 dozen strokes. Generally I have the larger drift sock rigged on the main wind side and the smaller sock rigged on the port side for protected bays. The kayak weighs 44 lbs so that works for my old bones. If I had to paddle for 4 or more miles before I fished this would not be the right kayak for the situation.
With greater distances to cover to get to fishing I would think that one of the peddle yaks may be the way to go. They are stable enough to fish and fast enough to negate the distance. You just have to make provision for their weight. There is probably not one kayak that suits all needs, which is why some people have their own fleet.