If you are talking about climbing ladders, there is no mechanic in the game for that yet. Any ladders you see are just for show.
If you are talking about stairs and stair cases, you should be able to just walk up the stairs. There are probably old ruined buildings that have 'stairs' but they are not always accessible, meaning that some stair cases you see are just for show.
Tutorial's bugged, craft spear 1/1 no further objectives
Edit: Minions are also bugged. They attack you when you summon them.
I'm somewhat surprised there's so much variety in what you can explore and the types of enemies and materials and yet the basic game mechanics don't work. Maybe work on making something playable, and then start adding more things.
Are you running on the newest hotfix version (v0.8.5.3.1 Hotfix)? If you are not, download the latest version, that problem was fixed in the Hotfix. There are other bugs still in the game after that, but the issues with minions/servants and the fisherman quests were resolved.
If you are running on the newest hotfix version, delete the wickedisland folder in your appdata\local folder. Pretty much any time the game is updated, it changes parts of the code and old saves and old settings files are broken.