6F is where you first meet the Adventurer, right?
So, firstly you need to enter center, meet the Adventurer, get boomerang from her and toggle a lever.
Then you go to left side to the end, find and toggle a lever there.
Then you go to right side, where you will eventually get to center down with two big enemies/trolls. Defeat one of them to open the door, and behind that door will be the stairs down and Shady Man who offers to make a bridge in center, allowing to go directly from entrance to exit.