Combat isn't broken. This is how it's supposed to be. Getting the bridge is tough, but after you can upgrade your dice so your rolling 3d6 and then it's just a matter of making the right choices on where to put the dice.
I'd argue combat
is at least partially broken- not bad, just fundamentally flawed- when there's zero leniency on the dice rolls. In DnD or Baldur's Gate or any other dice-based game, a single roll isn't going to make or break things most of the time, and when it does it usually means change of plans or healer panic instead of outright combat loss. Despite that, BG3 still includes an option to have dice rolls always err towards middle-ground and DnD always has the option of GM fudging the numbers.
Meanwhile in this game I resorted to save-scumming to save time retrying fights because the D4s consistently rolled higher than the D6s, and my D8 straight up never rolled higher than a 3, while the bosses seemed to land at least one perfectly-rolled dice almost every single round, so the good rolls I did get had to go to defending even on the defensive bosses, and I had to pray for a second high-roll dice. That's the sort of behaviour I'd expect and be fine from games that are built into that identity, not one that trails off the end of a sandbox VN and then demands you watch or skip through the loss dialogue before you can reload your save, which is time-wasting but still faster than going back to the mansion to skip time and nap.
Personally, I'd make the dice upgrades act as extra dice rather than upgrading existing dice. Going for the reinforcement option early on would effectively be like rolling with advantage, as you'd roll 3 D4 and a D6, then pick any three rolls and apply them. At the end of the current game, you could max out your dice and pick that same option to roll
seven dice, but you'd still be picking only three of them and it'd effectively be more akin to RNG mitigation when you're facing bosses that have triple-D6 of their own.