Well, both girls have shown up to sleep in my room. Didn't unlock it. And I have bought all outfits, and given them where I could. Let me just try unlocking the dungeon and see if that helps... Okay, unlocked it and did the first scene with Lin (this gave me friendship level 17) and she shows up in my room again to sleep with me. Got friendship level 18 next morning. And it unlocked the sleep with Lin/Andi option.
Guess the dungeon isn't 100% optional since it gave friendship levels. Actually thought I had maxed Lin's friendship.
Edit: And now that high elf shows up. Guess there's more story to do