All in all knocking out Dave will make mostly change dialogue and most likely open up for marriage with Frisha down the line and potentially some other scenes but at the moment if you want the most content then let her stay with Dave though this runs the risk of her marrying Dave down the line.
Frisha marrying Dave is a possibility I hadn't considered; but if Liandra is at the Camp and Giron success at marrying Julia and the MC (making the MC his successor), that could happen.
If Frisha ends up in a relationship with Grant, even if she could dump Dave, Giron wouldn't let her go.
Even if Grant knows Frisha is fucking the Baron, I don't see him making the effort of liberating her from Giron. He can't extort anything from Giron and ending up at odds with the lord of Arenfield is not worth it for him.