Well right off I noticed you don't have run like dodo on but that doesn't matter for this. Are you buffing your luck? I tried a lot of things like weapon element, insanity, his buffs, etc. After stalling the fight for a long time just using omnibuff, phalanx, and guarding and dodging appropriately, I broke him everytime (about 4 or 5 attempts in the same fight), even his mirrors. I tried again this time only buffing my defense and mdefense with items and using phalanx and couldn't break him. You could also upgrade the mock turtle's ladle, it gives +50% luck at +5, same as stacking two luck buffs in case you don't want to spend two turns on omnibuff.
In other words, the fight is literally luck based.