Personally I don't.
Battery life isn't a factor with me, as I use Duracell rechargeables and they go strong for a week or more between recharges.
The reason I don't wear them is the obvious one - I don't care for the extra clutter,tangles and uncomfort headphones entail.
As I start hitting open fields with little trash I'm sure I'll wear them from time to time to try to increase the odds of finding deeper targets, but generally no the reasons stated here by everyone aren't enough to get me to wear them all the time no way.