I couldn't say from just that description.
If you are in an area where you find fossils you'll notice that they take on different color depending on the sediment they spent most of their time in.
Just being grainy and dark wouldn't necessarily make something identifiable as a fossil.
You should see evidence of some latticework of bone.. It will also help to check a geological map of the area you hunt to see what type of fossils are there.. Also look for examples of fossils found in your area and compare