It can harm the earphones. It can demagnetize. or increase the magnets in them. Most small magnets carried are of the neo type with are usually a lot stronger.
Not a wise thing to allow such near any electronics. It can have adverse effects upon any ferromagnetic type material in such.