It's still not worth it.
For every person like you, there's someone like me. I intensely dislike a lot of VA. The only company that ever got it right was the one that did Xenoblade and FFXIV for Monolith and Square Enix.
When I was playing the Trails series, when I moved on from Sky, and onto Trails From Zero, I pulled that disgusted lip curl/snarl in the opening scene when I heard them speaking, and immediately switched to Japanese language so I could read the dialogue in peace.
Everyone is happy with the game as it is (although perhaps this dev cycle is making a few disgruntled with the time being taken) - there's no incentive to make a change when the audience has already accepted the game as it is.