It's not you, it's the game. You have to be earlier than you think you should be. Even after that if you screw up you're punished instead of just not getting any points.
Yeah that's what makes these minigames even more annoying, most of the time they're utterly unresponsive and very poorly implemented. Professional game studios stopped using them a long time ago, but they at least had them working properly. Even then, a studio like Telltales died from making that their main gameplay tool, ignoring people's criticisms. Some of the devs that have games on this site are even worse, just appears they never played any games or playtest it themselves, ignoring the whole point of games which is for them to be fun.