The beach is one screen down from the first forest. There's a fisherman there who gives Celica a fish for the cat, if I'm not mistaken (in exchange for a stone tablet you get from Mr Fuse (Mage guild guy, dark coat)).
For the cave, you'll need to "fuse" a light crystal and a lantern. You buy the crystal, I think, but not sure where the lantern is. Maybe the thread already has the answer.
Sounds like you need to progress the main plot to get to fusing, so make sure to visit the Guild and talk to Mr Muscle (Bert?) and exhaust dialogue. If that doesn't trigger it, just talk to everyone in town - and as always, the bartender may be able to help if you ask him for a hint.