River Rat is correct, it is a Musical Band uniform button. Yours is exactly like the ones on my high school band uniform back in the 1960s.
Although the Scovill Manufacturing Co. was in business under that name since 1850, it continued to manufacture buttons until 1962. Your button's backmark seems to be written in 20th-Century style lettering.
I think if you cleaning it a bit more, it would say "WATERBURY CONN", not just WATERBURY CO. (I've never seen one with an abbreviation "CO" for Connecticut.)