Have you bought one of those rods by now? If not, then I'd say they're both pretty good choices.
Normally I'm a big fan of lighter weight fly rods, even ultralight rods, if the quarry is panfish. But that's easy for me to say; I do most of my fishing out of a canoe. (Fishing from a boat lets me move in and dislodge snagged flies, whereas a bank fisherman has to break it off.)
For you a 5-wt. would be a good choice. For one thing, farm ponds have a way of harboring some really big fish and it helps to use a rod with enough spine to fight them. Also, if you hook a fish of any species or size then chances are that fish will wrap your leader around some kind of submerged cover (weeds, limbs, grass). To better extract the fish from that cover, it helps to have a rod with enough spine that you can apply pulling pressure. A 5-wt rod accomplishes this extraction better than an ultralight.
But if you haven't bought a rod yet, I would suggest that you stick with the 5-wt rod but buy one that is 9 feet long instead of 8 feet. That extra foot of length lets you do roll casts easier and farther (for the many times when standing vegetation or trees behind you interfere with normal false casting). And when you're standing in spots where the area behind you is open, the added lenth of a 9-foot rod lets you make longer casts with less effort.
You didn't specify what your handicap or disability is. And I'm not asking, but apparently it isn't something so incapacitating that it prevents you from fishing from the banks of farm ponds? If it is some sort of ambulatory issue, your "handicap" might actually give you a powerful tactical advantage. Do you "naturally" move about slowly and quietly? If so you'll enjoy better luck than many non-handicapped fishers do, who stomp around hurridly with their foot vibrations spooking off the fish before the angler gets close enough to cast to them.
"Better small than not at all."