Been attempting this myself. Nothing more annoying than fighting your way through four swarms, two boats and a whale only to find yourself waking up on the beach again.
For the sake of my sanity i'll disable swarms and tentacles (apparently whales come with two of them now) next time i'm trying to reach the island.
Update: Made it to the island, not really sure what influenced the roll but passing out during or right after an encounter seemed to have a higher chance than swiming back and forth until my stamina and stress were full, so maybe pain related? Also, had bestiality turned off and still got attacked by dolphinboys occasionally, so it may be encounter related. Keep in mind it's a very small test sample though, so i could be (and most likely am) wrong.
Still need to get back since the save i went there on was my older one that i maxed out before Housekeeping was a thing, and a number of camp upgrades require Housekeeping (including a feat for something requiring S-rank) which i did not have. Still, got a ton of feats done and even the high Sadism one from fighting off a ton of encounters.
My advice to people that need to visit the island? Make sure you also have high Skullduggery (avoid encounters) and, unless you feel like going crazy trying to look for walnuts in order to not starve, the Trait that allows you to drink your own breastmilk for sustenance. Not sure if it counts, but if it does it's absolutely useful and pretty much everything uses aphrodisiac-tipped arrows so your Arousal will rarely ever drop below 50% anyway. Or, as mentioned, high Housekeeping and prioritizing the bird feeder so there is at least some form of food income going on.
Productive couple of weeks, got some 50 Vrelcoins added. As an aside, does anyone know what the six-armed statue does in the Temple and how to fix it? I can't find it on the wiki and the text implies i need to be doing something about it since it's getting progressively worse despite me constantly training whips, Batons and upping Grace.