Strange problem indeed, seems like PC restarts because GPU is overheating, which is might be (or might not) caused by unlimited FPS.
If it were memory leaks, the game would crash, but not PC i think.
But that is kinda weird, GTX 1080 is a fairly powerful card, but i think it's not enough to make game run at very high FPS because this game have very poor performance.
If i were in your situation, i would reinstal nvidia drivers, reset all nvidia control panel settings to default, and made sure that no third party program runs in background. Then reset all game setting to default, and tried it on different graphics presets.
And ofcourse monitored temperature of GPU, both when idle and in game. If none of it work, i would give up
But yeah, even though this game is stable in general, it's very poor optimized. Some locations like a park area is complete disaster.