I suspect that we actually agree. It _is_ extremely hard to write an internally-consistent and interesting narrative. But player agency should have little to no impact on the narrative... if it's done the right way.
The mistake that many Devs make is that they assume that player agency = branching plot. Those branches can absolutely kill the integrity of the story, exhaust the writer(s), and create hopeless complexity. They can kill a game.
But there are plenty of ways to grant a player agency without spawning a hopelessly complicated series of plot branches. _Planescape: Torment_, which for my money is still the best-written game of all time, gave the player a massive feeling of agency with almost no real plot branches. Instead, a player's decisions -- including things as simple as asking certain questions -- changed the way the PC saw himself, the NPCs, and the world. These changes were profound... but they hardly affected the plot at all. The player goes on the same journey, and has the same confrontations in the same order... but the meaning of those confrontations for the PC changed profoundly... because of the player's decisions.