I sometimes switch between computers when playing VNs, so I've gotten into the habit to keep backups on a series of small USB drives. When I reach the end of an update, I zip a copy of that VN's save folder to one of the drives, then zip a copy of the entire main game folder to a separate drive. The latter in case I want to replay part or all of the last release at some point.
I also keep a zipped copy of VNs that have been completed in this drive system, in case I want to replay any of them later. All zipped copies of entire releases would also include any mods I've installed on them.