Ren'Py The Headmaster [Mykock's Manor Ch.3] [Altos and Herdone]

[_Kiyoshi_]

very complicated game

 

[raven54]

Could it be that all affected users are using Windows with Japanese or Korean settings? I just learned that the Windows path separator \ (backslash) is replaced by ¥ (Yen) for these versions. For Westerners this is totally unexpected, so I can easily see it cause bugs in a variety of ways.

The path separator might not be the biggest problem. Software is supposed to run unicode today, but windows used to use codepages, like

You must be registered to see the links

and

You must be registered to see the links

. It offers a subset of the unicode characterset, but it uses less memory, which used to be important. For instance you can expect Japanese text written in codepage 932 to use 50% more memory if converted to utf-8.

What might happen here is if the zip file is encoded in some codepage and is read with another codepage, or encoded in unicode, but the reader assumes it's a codepage or similar, then it's possible that there is a byte combination, which simply doesn't exist in that codepage, which would result in an error. It's also possible that some text just ends up as garbage, which results in unreadable text.

Example: Let's say a sentence ends with "á.". Western Europe uses codepage 1252, which makes it E1 2E. Reading this in Japanese, it becomes codepage 932, where E1 indicates that it's the first byte in a two byte character. However the second character will never be below 3F meaning the charater E1 2E doesn't exist in that encoding, hence throwing an encoding error.

I will say that I have encountered encoding errors in zip files, so it's plausible that this has something to do with it, through worst case I have encountered has been incorrectly named files. Luckily the 128 first characters are the same in all encodings, so if all characters are in ASCII as in plain English (like a instead of á), then filenames should be correct even if decoded with the wrong encoding.

This is also why some games requires Japanese locale. The games assumes all file paths are encoded in codepage 932, but doesn't actually explicitly tell windows that, so windows falls back to using the locale set in windows for applications, which doesn't tell which locale to use. It's one line of code to make a game tell windows to always use Japanese locale regardless of the settings in windows, but this is rarely used and since it has to be added before compiling, it's something developers should do, not end users.

For quite a number of years the recommendation has been to not use codepages as that will avoid the problems. Microsoft is even working on making it harder to use them to push developers to not keep using them, but that's not the same as developers will stop doing things like they did it 30 years ago, hence why this is still a problem.


I will add that sometimes the problem is not even with the filename, but the path, so if your username has some character, which doesn't work with the encoding in question, then that can make it fail. However if you unzip in something like C:\my_folder, then that will get around that issue. The precise location isn't important, just that it only has plain ASCII and not any extended characters in it.

It's probably not an issue here, but while we are with encoding, some applications have problems with spaces in the path, which is why some foreign developed programs doesn't work inside "program files". This is rare, but

 

[torpedogoat]

The path separator might not be the biggest problem. Software is supposed to run unicode today, but windows used to use codepages, like

You must be registered to see the links

and

You must be registered to see the links

. It offers a subset of the unicode characterset, but it uses less memory, which used to be important. For instance you can expect Japanese text written in codepage 932 to use 50% more memory if converted to utf-8.

What might happen here is if the zip file is encoded in some codepage and is read with another codepage, or encoded in unicode, but the reader assumes it's a codepage or similar, then it's possible that there is a byte combination, which simply doesn't exist in that codepage, which would result in an error. It's also possible that some text just ends up as garbage, which results in unreadable text.

Example: Let's say a sentence ends with "á.". Western Europe uses codepage 1252, which makes it E1 2E. Reading this in Japanese, it becomes codepage 932, where E1 indicates that it's the first byte in a two byte character. However the second character will never be below 3F meaning the charater E1 2E doesn't exist in that encoding, hence throwing an encoding error.

I will say that I have encountered encoding errors in zip files, so it's plausible that this has something to do with it, through worst case I have encountered has been incorrectly named files. Luckily the 128 first characters are the same in all encodings, so if all characters are in ASCII as in plain English (like a instead of á), then filenames should be correct even if decoded with the wrong encoding.

This is also why some games requires Japanese locale. The games assumes all file paths are encoded in codepage 932, but doesn't actually explicitly tell windows that, so windows falls back to using the locale set in windows for applications, which doesn't tell which locale to use. It's one line of code to make a game tell windows to always use Japanese locale regardless of the settings in windows, but this is rarely used and since it has to be added before compiling, it's something developers should do, not end users.

For quite a number of years the recommendation has been to not use codepages as that will avoid the problems. Microsoft is even working on making it harder to use them to push developers to not keep using them, but that's not the same as developers will stop doing things like they did it 30 years ago, hence why this is still a problem.


I will add that sometimes the problem is not even with the filename, but the path, so if your username has some character, which doesn't work with the encoding in question, then that can make it fail. However if you unzip in something like C:\my_folder, then that will get around that issue. The precise location isn't important, just that it only has plain ASCII and not any extended characters in it.

It's probably not an issue here, but while we are with encoding, some applications have problems with spaces in the path, which is why some foreign developed programs doesn't work inside "program files". This is rare, but

I think you missed that we already found out (mostly) what's going on. The locale thing was a red herring.

There is a broken Python library for creating ZIP files, which is used by Ren'Py. The normal Ren'Py deployment process creates ZIP files which are missing the "end-of-central-directory", whatever that is. It's clearly not needed to extract the ZIP file, but some extractors refuse to do so when they detect this irregularity, or if they do, warn about it. There is still some confusion about why 7-Zip seems to sometimes warn and sometimes not, but otherwise everything seems to be clear now.

 

[carl199319]

Help, I passed all the other missions but I can't find Maxine in the dorms.

 

[Sploooogerrific]

On my M1 MBP, running Sonoma, it works fine.

What steps did you take to make the download executable? chmod +x, sudo xattr?

 

[raven54]

The fact that Ren'py doesn't follow the zip standard might not be a problem in itself. It's bad and it causes warnings, but it seems to work anyway.

You must be registered to see the links

doesn't follow the HTML standard as it skips a bunch of mandatory, yet for this specific page not needed information. This is done to reduce bandwidth usage as this page is loaded a whole lot of times every second. Google got away with this because nobody will release a browser where this page doesn't work. In other words browsers work even on pages, which doesn't follow the standard, so it's plausible that zip programs work on files not following the standard.

Having said that, generally speaking not following a standard 100% will usually result in problems eventually, so most of the time it's a bad idea.

This is a problem, which should be fixed in Ren'py as repacking has a tendency to mess with the executable flags, particularly on mac, through according to this thread also on linux.

linux users can always use proton as a workaround . i can confirm this game runs just fine with it on steamos .

If it works, it works, but it seems a bit silly as Ren'py officially supports linux. When getting Ren'py to build a release zip, there is an option to make a windows+linux zip. Obviously that goes back to the issue of Ren'py apparently not making proper zip files.

I'm a on Mac, and I also get this executable error thingy all the time, but I never knew the reason (not saying I understand all that you're saying, tech-wise, but I kind of get it, in an elusive, just-out-of-my-reach kind of way)

It should be fixable by opening the terminal and using chmod on the executable (not the application bundle), so something like

chmod +755 "the headmaster.app/Contents/MacOS/renpy"

, through I don't have the mac version here right now and I don't feel like downloading 8 GB just to test this command, so it's somewhat based on memory and might not be completely accurate.

I think you missed that we already found out (mostly) what's going on. The locale thing was a red herring.

It's worse than that. Somehow I missed an entire page of this thread. It did pop up with new posts and when I clicked it, it behaved a bit different from how I expected, but I didn't catch that it skipped an entire page of posts. I should have paid more attention.

 

[PBS666]

What steps did you take to make the download executable? chmod +x, sudo xattr?

In my experience, Chmod is lately rarely necessary, but you can always check the executable in the package to see if its kind is Unix executable.
Quarantine on the other hand is almost always imposed, so I routinely do xattr -dr com.apple.quarantine on each app downloaded from here, and on the sdk's. Sudo should not be necessary for this.
I always use The Unarchiver for unpacking the archives.

 

[BigCosyStoryFan]

There is a broken Python library for creating ZIP files, which is used by Ren'Py. The normal Ren'Py deployment process creates ZIP files which are missing the "end-of-central-directory", whatever that is. It's clearly not needed to extract the ZIP file, but some extractors refuse to do so when they detect this irregularity, or if they do, warn about it. There is still some confusion about why 7-Zip seems to sometimes warn and sometimes not, but otherwise everything seems to be clear now.

I see, so it is something with how Ren'Py does the zipping? That makes some sense. I will ask around about that. Thanks.

Because this is apparently a widespread issue, I was going to look into it to patch it upstream, but apparently that isn't needed, because Ren'Py isn't the problem here.

The bottom line here is that the ZIP files in question aren't malformed. They meet the requirements of the ZIP specification (such as it is), and nobody has to push an upstream patch to Ren'Py unless the package format is going to be changed. The library and packaging are both fine. If the package used by Ren'Py were generating invalid files, given the widespread use of Python, and the tools linked to it, we'd have seem them flagged in a bunch of places well before Ren'Py users started having issues.

Technical details in the spoiler to keep the vertical space down.

Note: I did not test 32-bit Linux distributions. They are being phased out at the distribution level, but they could experience a failure case for archived files over 4.3GB if support for larger files isn't compiled into their tools, or the platform does not have certain extensions/modules/drivers enabled.

Spoiler: Technical breakdown

You don't have permission to view the spoiler content. Log in or register now.

tl;dr: From the perspective of the specifications of the archive format, the files for this game are fine. If there's an issue, you need to address it on your system's end.

Edited a bit for some glaring spelling issues

 

[TheDevian]

Because this is apparently a widespread issue, I was going to look into it to patch it upstream, but apparently that isn't needed, because Ren'Py isn't the problem here.

The bottom line here is that the ZIP files in question aren't malformed. They meet the requirements of the ZIP specification (such as it is), and nobody has to push an upstream patch to Ren'Py unless the package format is going to be changed. The library and packaging are both fine. If the pakage used by Ren'Py were generating invalid files, given the widespread use of Python, and the tools linked to it, we'd have seem them flagged in a bunch of places well before Ren'Py users started having issues.

Technical details in the spoiler to keep the vertical space down.

Note: I did not test 32-bit Linux distributions. They are being phased out at the distribution level, but they could experience a failure case for archived files over 4.3GB if support for larger files isn't compiled into their tools, or the platform does not have certain extensions/modules/drivers enabled.

Spoiler: Technical breakdown

You don't have permission to view the spoiler content. Log in or register now.

tl;dr: From the perspective of the specifications of the archive format, the files for this game are fine. If there's an issue, you need to address it on your system's end.

That was similar to my initial thoughts, I have also tested these files without issue, but the fix is what I can't track down. It is quite difficult to find a solution, when I can't find someone suffering the issue to test things.

Thanks for that.

 

[raven54]

Spoiler: Technical breakdown

You don't have permission to view the spoiler content. Log in or register now.

Great work. Now that the problem is understood, hopefully we can completely end all talk of malware. While we can't fix the problem itself, I wonder if a valid workaround is to split up the rpa files to ensure none of them exceeds 4 GB. This will have the added benefit of allowing the game to work even on file systems, which doesn't allow files to exceed 4.2 GB (through people really shouldn't use those anymore for other reasons).

 

[c3p0]

That was similar to my initial thoughts, I have also tested these files without issue, but the fix is what I can't track down. It is quite difficult to find a solution, when I can't find someone suffering the issue to test things.

Thanks for that.

Because this is apparently a widespread issue, I was going to look into it to patch it upstream, but apparently that isn't needed, because Ren'Py isn't the problem here.

The bottom line here is that the ZIP files in question aren't malformed. They meet the requirements of the ZIP specification (such as it is), and nobody has to push an upstream patch to Ren'Py unless the package format is going to be changed. The library and packaging are both fine. If the package used by Ren'Py were generating invalid files, given the widespread use of Python, and the tools linked to it, we'd have seem them flagged in a bunch of places well before Ren'Py users started having issues.

Technical details in the spoiler to keep the vertical space down.

Note: I did not test 32-bit Linux distributions. They are being phased out at the distribution level, but they could experience a failure case for archived files over 4.3GB if support for larger files isn't compiled into their tools, or the platform does not have certain extensions/modules/drivers enabled.

Spoiler: Technical breakdown

You don't have permission to view the spoiler content. Log in or register now.

tl;dr: From the perspective of the specifications of the archive format, the files for this game are fine. If there's an issue, you need to address it on your system's end.

Edited a bit for some glaring spelling issues

I have tested the around 3 800 archives I have on my PC.

Spoiler: More detailed

You don't have permission to view the spoiler content. Log in or register now.

 

[TheDevian]

I'm currently testing the 3 800 archives I have on my PC. Only one I've encountered until know (minus one incomplete archive that is corrupt it is a positive case, is one 3.36 GB Zip file with header error - below the 32bit limit).

Spoiler: More detailed

You don't have permission to view the spoiler content. Log in or register now.

I have been tracking this error on several games. This is part of how we know it's not the game itself.

 

[torpedogoat]

Here is what I know:

• A lot of Ren'Py ZIP files are slightly broken. Most ZIP tools decompress them just fine, but 7-Zip sometimes(???) does so with a warning "Headers Error", which is why this discussion started. I have personally reproduced this warning with the 7-Zip console client on Linux after another user posted a screenshot of the warning in the graphical 7-Zip client on Windows.
• Only few ZIP tools completely refuse to work on these files. This includes unzipada, which, coming from the Ada world, is particularly pedantic and seems to have the best ZIP diagnostics available. Unzipada complains: "Bad (or no) end-of-central-directory".
• It appears that only ZIP files direct from the developer are affected. Repackaged ZIP files such as compressed versions never seem to have this problem.
• Based on a screenshot above, the person contacted on The Headmaster's Discord seemed to suggest that it is a problem with how Ren'Py creates ZIP files. I don't know to what extent that was prior knowledge or the result of a quick test, and to what extent it was conjecture. In any case it is consistent with the above results.
• I checked Ren'Py 8.1.3 running on Linux. It does not seem to have this problem. I deployed the tutorial project as a ZIP file, and it was fine.
• Checking on the major operating systems is critical because Python's included libraries always use native code libraries for this kind of thing as it would be too slow otherwise. It seems likely that only the one for Windows, or only the one for MacOS, is broken.
• The most likely operating system affected is Windows. This is because on Linux and MacOS programs and shell scripts are made executable by setting the file's executable bit. In Windows file systems this bit does not exist, and so the Ren'Py SDK must use some trickery as a work-around. This could be the source of the bug.

To get definitive results, a Windows user would need to install the Ren'Py SDK and deploy the tutorial project as a ZIP file. That's almost as easy as playing a Ren'Py game: Start renpy.sh in the unzipped Ren'Py SDK. Click "Build Distributions". Click "Build". This creates two ZIP files (Windows/Linux and MacOS) and stores them in a subdirectory of the Ren'Py SDK's root directory.
To check the ZIP files, download zipada56-bin-win32.zip (the latest binary release) from

You must be registered to see the links

, extract, and use unzipada.exe -t <name of ZIP file>. If you get a long list of lines looking like "...1805-pc/tutorial_7.sh Deflate 679 : 41% of 1663 [........]", the file is fine. If it is broken, unzipada doesn't even look at the individual files and gives you a very long error message instead.
(I would do this myself, but I have no working Windows since a regular Windows update in 2020 made my Toughbook unbootable.)

On MacOS the second step (checking the ZIP file) is trickier. You could try to see if 7-Zip warns. For a definitive result if not you would have to check the file on Windows, use Windows emulation to run unzipada.exe (like I do on Linux), or compile the Ada code of unzipada. That's why I suggest a Windows user should try first.

 

[c3p0]

To get definitive results, a Windows user would need to install the Ren'Py SDK and deploy the tutorial project as a ZIP file. That's almost as easy as playing a Ren'Py game: Start renpy.sh in the unzipped Ren'Py SDK. Click "Build Distributions". Click "Build". This creates two ZIP files (Windows/Linux and MacOS) and stores them in a subdirectory of the Ren'Py SDK's root directory.

On Windows you would launch the renpy.exe file not the renpy.sh file.
Also the tutorial won't build:

But "the question" did and according to 7Zip, nothing to be found there (for what is worth:

You must be registered to see the links

).

 

[torpedogoat]

On Windows you would launch the renpy.exe file not the renpy.sh file.
Also the tutorial won't build:
View attachment 4750446
But "the question" did and according to 7Zip, nothing to be found there (for what is worth: Pixeldrain).

Thanks for trying this. (Yes, of course you need to run the .exe on Windows. I hope I didn't confuse anyone.)
I can confirm that unzipada is happy with all three ZIP files. Clearly we are still not there.

There is still a slight chance that the bug is on MacOS only. Otherwise it's either version dependent or the developer is probably re-zipping with a non-standard tool, after all.

 

[PBS666]

There is stilla slight chance that the bug is on MacOS only. Otherwise it's either version dependent or the developer is probably re-zipping with a non-standard tool, after all.

No bugs on MacOS for me, just the annoying quarantine. But that's not a bug but a feature.

 

[LucT]

Surely there is already a lot of material to work on and I don't remember if it has already been proposed, but I thought it could be a good idea for some students to ask for a course in sexual education, not to improve their knowledge but to improve their skills, seduce, charm, use body and sex as a weapon.

 

[torpedogoat]

Surely there is already a lot of material to work on and I don't remember if it has already been proposed, but I thought it could be a good idea for some students to ask for a course in sexual education, not to improve their knowledge but to improve their skills, seduce, charm, use body and sex as a weapon.

One girl is already getting individual lessons on this in a way that maintains the imbalance of power characteristic for this game. An entire course might be difficult, as it is basically about a form of female empowerment that seems contrary to the goals of the school's owners in some respects, despite agreeing with it in others. This school isn't about improving the girls' lives; it's about exploiting them right now and making them easy objects of exploitation by others in the future. Your proposed lessons further this goal only with the most stupid of the girls. With the others it would be counterproductive.

 

[TheDevian]

Surely there is already a lot of material to work on and I don't remember if it has already been proposed, but I thought it could be a good idea for some students to ask for a course in sexual education, not to improve their knowledge but to improve their skills, seduce, charm, use body and sex as a weapon.

They are already talking about doing that as part of Liz's future events.

 

[MaxRichard]

Fortunately, as a nudist, she won’t mind being naked in front of all her classmates and teachers.

LOL this dev is just too good