Solving the riddle is part of the fun! And sometimes the fish are just in that mood.... Try it again. In some waters, especially clear waters, the fish tend to feed in the low light hours rather than during middle of the day.
With bass on the patrol, panfish seem to know that if they let down their guard, the bass may charge in and nab them.
I'd suggest trying to solve the bass issue first. Catch a few, and while they sulk after the release, you may find the panfish will relax more. For the bass, try large streamers that resemble panfish and fish them fairly fast, like they are trying to escape. And try large topwaters that create a good disturbance....like a blockhead popper.
Once the panfish relax, try #12 or smaller nymphs or leech patterns with SHORT tails. That way, you may catch some nippers.
Fishing faster gives the fish less time to carefully scrutinize a fly.