8T,
If the students complain too much about you missing class to go fishing, pile on a few extra projects, ask some unanswerable questions for the final and fail them all. Hopefully, they'll take the class from another proff next semester.
One of the proffs when I was in college was also a fisherman. Everyone warned to not take his class in the summer term, he was always in a bad mood.
Jeff