I used to think there were more high income fly anglers than others as well. That was until I got the batch of orders for custom baitcast and spinning rods that has come in over the last couple of months. These are pricing out right up there with the fly rods.
That said, when I got into fly fishing I discovered it could be every bit as expensive as golf if not more so. I am very glad I chose fishing over golf, though. Rods are a lot more fun to build than putters.