Door keys are always in a chest (the sprite not the icon tile) somewhere on that floor. Use overview to look for them.
Nameless Ruins is an outlier in this regard. There is not obvious chest to get to in order to acquire it. I ended up using the mtool to get it, but I believe it's a high-rng thing with the green ? squares--believe as in I could be wrong--because there is a stupid small chance you'll get an event where you can reach into a space and try to grab whatever's in it on a dice roll. You have to roll well, and I only got two common items from a poor roll; hence my use of mtool. Sorry, but, when your entire puzzle is layers of rng, with each layer being a bit overtuned, I'm not going to play along as intended.
So feel free to confirm this about the green ? spaces, if someone can. Be ready to roll for it.