I live in Beaufort and am famillar with New Bern. Check on Lawson's Creek Park and Union park. My parents live in Kinston and I thought about stopping at one of these parks to check it out on the way to visit them.
Yes there is. There is the Coastal Artifact Recovery of North Carolina Club (CARe of NC).
They meet on the 2nd Wednesday of each month at 6:30pm at Moores BBQ on Hwy 17S, New Bern. https://www.facebook.com/events/225413690935157/