I would be willing to bet the difference was not worms vs. flies. I'd bet he was using split shot to get the bait down and you weren't using split shot to get your flies down. Try a beadhead nymph or small beadhead bugger next time, and I think your results will be better.