I always get "cottage looks abandoned/decrepit".
Yes, this is the message you will get if you try going to the cottage before receiving a gem from the suspicious man. Here is a step-by-step:
1. Enter Silajam
2. Go to Town Square (during the day)
3. Listen to the town crier. He'll mention Mount Xiuh and a creature residing there.
4. Go back to Town Square, but eavesdrop this time. You should encounter 'suspicious man'
5. Satisfy him however you please. I'm relatively certain you'll receive the item you need even if you fail the check.
6. Return to witch cottage (You don't actually need to go to it before getting the item. As long as you've spoken with the suspicious man, you should be able to access it)
Ta-dah, ya did it.