I'm not at all familiar with your waters, but I'd say fly selection has something to do with it. Try swinging big, heavy streamers down past the big guys.