If fish are swimming up to the fly, but not biting it, I change. If it is coming undone (mine never do), I change. If it is just missing a leg, or similar, and fish are still biting it, I stay with that fly. If I go 15 or more casts with nothing, and I know there are fish there, I change flies. Sometimes it is just a matter of color, or size.