Cave laws for any state are Archaeologically strict and removing anything from a cave can come with severe penalties.
Here is Ky law, and Indiana law is the same as is most states. Read the whole thing because at first it looks like you only need written permission from the land owner to remove objects from the cave. however, that is only the first requirement to gaining an archaeological and scientific permit from the state to legally remove any artifacts or minerals or formations.
*Note
I would not admit to removing any artifact or mineral or formation from any cave anywhere on a public forum, because of risk of prosecution. Just some friendly and wise advice.