I've forgotten about the holes in the ground, but as far as the holes in the wall saying you're too big, there is a relic or artifact you can get that will allow you to shrink so you can enter. But that is when you get to the castle in the southeast and go into the bunker.
First of all, which holes in the ground are you talking about?
And second, those little holes in the wall that says that it's too small to fit in, you can only get the items from those places by having the relic/artfact Midget Oil, which you'll find a bit later. Maybe the Snake Heart is in one of the holes in the wall.