Remove them same as you did the ferrule...I have taken a few old ones off...hot water should do i t....the glue is usually pretty brittle around the handles/cork.

You'll probably have to strip the cork off with a pliers, just grab and twist after heating, it'll crumble off leaving the epoxyy residue behind.