Sound in air travels at approximately 1130 feet/second.
Sound in water travels at approximately 1500 yards/second with variances dependant on temperature, salinity and pressure.

Normal conversation above the water will normally not affect fish.
Any activity that causes soundwaves IN THE WATER can significantly affect fish.
For example, a pocket radio in your pocket may not have any effect. The same radio placed on the metal floor of a boat may have a significant effect.