Found a way to bypass this bug.
1. You need a quest item Horse.
2. You need RpgMakerSaveEdit (i've been using 0.69 version)
In the gnoll camp use Horse item, this should teleport you to castle, create new save file. (If you'll try to leave castle, you end up in gnoll camp again)
Open created save file in RpgMakerSaveEdit. Advanced > Edit Switches and Variables.
In Switches you need to set HorseActive switch to not selected.
Save changes in program, load edited save file and leave castle, you should be in oakshire.