it's a typical problem of the GC managing of C#
To an extent.
I'm browsing through the code right now, and I kid you not, I replaced some lines labelled "unload" with destroy(the thing to destroy); and now, when the game loads up for the first time, instead of taking 1.7 gigs to itself, it takes 133 megabytes.
I also found out that in scenarios where it gets rid of textures, it didn't get rid of .TIFF textures because it didn't even look for them, so I added that in.
I'm going to have a look if I can find the hard reset function and make it destroy textures instead of keeping them as well, see if that nets me any results.
As it is, Hard Resetting an empty scene makes VaM take up MORE memory! It's insane!
...I'm realizing it looks like it doesn't destroy unused objects upon loading a new scene.
Which would be why the memory keeps climbing up and up on every scene loaded.
Seeing as once you're loaded the memory usage remains the same, the issue is squarely in the loading and hard-resetting of scenes.
Have a look in the code with DnSpy at one point, it'll give you a good idea of what's going on.