It's rather annoying (one of the worst lockpicking minigames I've personally seen). There is an in game hint (mouse over the lower left corner of the screen), but it's still annoying and (from what I can tell) random. Basically, as the hint says, you have to listen to the sounds the lock makes as you turn each ring. The outer ring is a deep kchunk noise, the middle ring a lighter thunk, and the inner is a click. You need to click the left and right arrows to rotate a ring until it makes the correct sound, repeat for each ring (using the up and down arrows, remember which direction you clicked since there's no indicator which ring you're turning and you don't want to mess up one you already set). Unfortunately, I don't think it's a specific spot on the ring, but a random chance each single rotation will give the correct sound and holding down the rotation buttons or clicking too fast will jump over the correct one.
After you reach rank hunting 2 quests a new NPC appears near the quest board, and she asks you to rescue a priestess from the slavers, the door is part of that at their camp by the coast.