I wouldn't think there'd be any giantesses who can grow to Croixelle's size who'd be petty enough to destroy a mountain because some people are living there- and the ones there are would be too busy doing other things.
Half-orcs and mages would make excavation a much quicker process, just like how quickly they're able to rebuild neutral towns that get sacked by a bored giantess looking for some easy amusement.
You could bring food to the cave, hell, if the subterranean home doubled as a mine you could simply trade for food, which could be preserved easily by drying it, smoking it, salting it, or just using magic- there *are* normal-sized mages, magic isn't solely the purview of giantesses.
As for why, the same reason people choose to live in neutral territory- to live apart from giantesses.