Any interest in a source code control how to?

[toolkitxx]

To actually answer your question from the OP:

I strongly urge you to just create the thread you mentioned - dont wait for users to give you a green light. F95 is a big collection of people ranging from all kinds of backgrounds and you wont see the replies you are looking for. To elaborate this:

The devs that believe they have it all under control wont answer you to begin with as they ofc believe its all under control
The ones that understand where you are coming from dont really need it but wont stop you from creating it either.
The ones that struggle but dont know why and with what have no clue you might hold the answer to their problems until they actually read about it in simple and understandable terms.

Many a dev here on F95 used to be a consumer of the stuff before they thought they could do better than dev xyz. So many come with a background of being self-taught or non-professional development. General project administration and version control are just buzz words for them until they actually read something that simplifies that down to a level that peeks their interest.

So in other words: Write that stuff and dont wait for approval. Believe me when i tell you today that it will be read and will be helpful for those who like to improve themselves. The rest couldnt be bothered anyways or will just discuss it needlessly and try to discourage you from doing it. Many guides, walk-throughs and tools here on F95 started by someone just doing it - the audience and appreciation came on its own afterwards. If you would have asked before creation if it would be needed the majority of answers would have stated 'no'

 

[Marzepain]

There is no reason to use github at all. You can create and use a local git repository with a couple of clicks of a mouse. Hosting locally is trivial as well.

You are right. I think to much as a developer and not as an artist.
The lone artists here are into there personal labor of love and not so much into success.

My background in game development pushes me to think that making a game is a highly complex team effort.
Even 25 years ago making demo's on an Amiga was a team effort of coders, gfx artists and musicans sharing there stuff by disks in the snail mail and modems hooked up to telephone lines.
It used to be all so hard, but that all changed. Sorry, I'm just old.

You do have to agree that, having some team support in place from the start, does allow a project to scale.

 

[Deleted member 487]

Sorry for the late and somewhat long response, but I think source control is kind of important.

I've been a lurker too and I too wrestle with the level of professionalism this community has attained. I'm also an ancient dev and amazes me how some people are making life difficult for themselves.

Being a solo developer has nothing to do with source control. Having Undo (ctrl-z) in your application is also handy for a solo developer, so having an undo option on specific files is useful and faster then creating a backup. Source control is something to make your live easier as you can see your changes (diff) when you last edited your files and you never lose anything anymore. The later being important if somebody else wants to finish the work or do a gamemaker to renpy rebuild.
Alternatively this might convince non coders.

You must be registered to see the links

I pity the talent being wasted here for lack of knowledge and fear of complexity. The 3D tools are far more complex too handle than a simple source control system. Sometimes the level of low self esteem on display here is perplexing. Why some people who have proven themselves smart, simply don't have the courage to try something that makes difficult tasks easier is beyond me.

I might even be inclined to fix a few bugs or type a bug report if the project was on Github or Gitlab, but less so if it was private.
Another case is that I've got quite a hefty development machine and I might be able to help out with rendering. The main customers of heavy duty machines seem developers, researchers, gamers and artists. Most of them get what they need, unfortunately lots of starving artists don't.
By the way, normally things that can be build from other sources in source control like executable don't get checked in, as they hog space and are easily recreate. In the case of rendered images that took hours, I'll make an exception.

Maybe the simplest way to get people here to start using source control is to start using an Integrated Development Enviroment (IDE) like Visual Studio Code. In there it's just a few setting and away you go. Code does Python and C# so your set for renpy, Unreal and Unity.

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

By the way, if you use an IDE you'll probably get a build number added to your exe when you build and install/deploy packages for get them probably too. IDE's are intended to take care of this sort of drudge work.

If you use Github you get the possibility to download a tarbal (.tar.gz) or zipbal (.zip) to download the project as a whole. This might be more stable than dropping it on certain file sharing sites. There is also the possibility to get an older version if you like. Check out this discussion on Stack Overflow.

You must be registered to see the links

A pro tip is having multiple repositories. One, possibly private repository, with your development stuff like models, scenes and code you wont publish. Another one with the the thing that can be installed by a user like rendered images and build exe.
Then a user doesn't have to download things they don't need. You could even make that 2 users on Github 1 with the name of your "publishing house" having the download repository of the official versions of the game and the other one for your personal, possibly private, development repository.
There are 2 tiny improvement for me that would be great as I don't have to dowload the whole backup every new version, but can simply do update on the repo and only get the stuff that changed. Secondly the repo's are neatly put in folders as my download dir is currently a huge mess.
There may be a better way to do it. If anyone has any better idea I would love to hear it.
There is however a caveat and that are quota's. (LFS stands for large file structures)
Github support repositories up to 100Gb, but pommises to whine when you go over 1Gb.
Gitlab supports 10Gb and may do something similar.

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

That is per repository, so if you have multiple repositories your fine.
You can have sub repositories in your repository. These subrepositories have there own quota, but do get downloaded when you update the repository that uses them

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

It is commonly used on Unreal projects even with large assets.

You must be registered to see the links

And also on Unity

You must be registered to see the links

You must be registered to see the links

You must be registered to see the links

For Godot

You must be registered to see the links

For Gamemaker

You must be registered to see the links

It may be telling that there is no Renpy video for github, but renpy itself is on Github.

You must be registered to see the links

If you know the project gets large, planning ahead and having a repo per instalment/DLC/Plugin or any other way a massive asset folder can be divided, may help.

In conclusion, working professionally is not extra work, it is having different habits and less trouble.

One thing that is some extra work is adding some descriptive text to your check in's. I as seasoned programmer know that my brain can't hold all the stuff I've done. Things just fall out of it at the most inappropriate times. It's very helpful if you can look back in your history and recognize what you have been working on. It's also quite motivating if you see progress.
Another thing that a little work is setting tags (in git or labels in other source control systems) for versions. That way you know what changes where done since the last version. Handy for making a change log.

You must be registered to see the links

You must be registered to see the links

I personally used to the Microsoft way of Mayor.Minor.Patch.Build system for exe's and Dll's, but make it just Mayor and Minor for the tag so specific patches can go on a minor version branch.

You must be registered to see the links

Python seems to do it's own thing in this case. Quite unusual.


On thing that is bothering me is that I can't seem to find any statement about erotica by Github or Gitlab. There seems to be some text adventure porn and some ero programmer humor, but not much more. There is the internet rule 34

You must be registered to see the links

so they must have thought about it.
Does anybody know if they allow it?

I dunno if this is an option for a lot of users but I think another option is a self hosted git service. It might cost you the price of the instance running, but I think it should not be too much, cloud instances are somewhat cheap, a git service instance would be cheap cause of the price required. However this might be tough to do for most amateur devs here without a good guide.

 

[Marzepain]

I dunno if this is an option for a lot of users but I think another option is a self hosted git service. It might cost you the price of the instance running, but I think it should not be too much, cloud instances are somewhat cheap, a git service instance would be cheap cause of the price required. However this might be tough to do for most amateur devs here without a good guide.

Your post kind of fell through the cracks, but here is my response.

The nice thing about Open Source is that you can always host it yourself and tinker with it. GitHub, GitLab and BitBucket do offer some extra tooling and services that can not be copied, but may be of benefit, like project management, issue tracking, wiki and Continues Integration/Continues Development (CI/CD). Although all those can all be done on a private server too, it's a lot of work and quite difficult to get right.

I'm currently looking into GitOps combining Git with CI/CD on Kubernetes. As most of the world uses Git already and seems to move to Kubernetes as it's generic cloud solution. Although Serverless solutions like Azure Functions seem to gain traction among the Early Adopters/Edge Junkies. Combined with a Blazor frontend, it greatly reduces the moving parts you have to monitor and configure.

As always the question is how much are you willing to spend and how much does your time cost.

 

[hkennereth]

As someone tinkering with the idea of making a game in the near-ish future, I had a question about this subject: I'm already familiar with version control since I'm a web designer/developer, so I definitely understand the needs and benefits. I however don't have any previous experience with projects that had the kind of files something like a RenPy game would have, namely hundreds or thousands of high resolution pictures, videos, and perhaps even project files in Daz3D, Blender, etc.

How do you usually address the need to add these files to something like Git, while having a remote backup on GitHub or GitLab that hopefully doesn't require a paid account because of huge repository file sizes?

Would you even have the big files within your repo, or just leave those on a separate folder on Google Drive to ensure you have remote backups. Would you add your project files to the repo, or only final assets?

 

[Tompte]

Git LFS is one solution (Git Large File Storage). Basically it works by committing only a small reference file to your main repository and uploads the actual file to a separate location that's more optimized for large files, while making it work seamlessly through git.

A good topic for the how-to, I think.

 

[AgentWhackjob]

Binary files in version control is always a question. The likely nature of images for a game like this (high resolution, minor tweaks are likely to change a lot of data in the image).. I would not put them under version control even LFS.

As far as backing up, whatever you have access to. You can backup a local git repository to google drive by having it in a folder synced to google drive.. or copying it to google drive. Just make sure you copy the .git folder and are not in the middle of manipulating the repo when copying.

 

[Tompte]

Binary files in version control is always a question. The likely nature of images for a game like this (high resolution, minor tweaks are likely to change a lot of data in the image).. I would not put them under version control even LFS.

I don't think it matters much for small to mid-sized PNGs or JPEGs. In most places I've worked, the artists tend to put their work files either on a NAS or a cloud service like Dropbox or Google Drive. But once the asset is done and put in to the game it gets pushed on the repo (with LFS) alongside everything else. Selectively putting some files under source control while leaving out others sounds like a recipe for tears.

But I don't know. Most devs on here seem to do games with CG renders and I presume there's a lot of iteration involved. Not sure what the best solution would be, but it's a special case I think.

 

[toolkitxx]

As someone tinkering with the idea of making a game in the near-ish future, I had a question about this subject: I'm already familiar with version control since I'm a web designer/developer, so I definitely understand the needs and benefits. I however don't have any previous experience with projects that had the kind of files something like a RenPy game would have, namely hundreds or thousands of high resolution pictures, videos, and perhaps even project files in Daz3D, Blender, etc.

How do you usually address the need to add these files to something like Git, while having a remote backup on GitHub or GitLab that hopefully doesn't require a paid account because of huge repository file sizes?

Would you even have the big files within your repo, or just leave those on a separate folder on Google Drive to ensure you have remote backups. Would you add your project files to the repo, or only final assets?

The base of this is related to the overall need for any version control: Do you expect many changes to existing material or not?
A web page is probably more dynamic in terms of changes to content than assets for a game. So the basic need here is probably smaller but you can always split your project in 2 parts. Use a local git-version for binary assets and just use remote as actual backup. Any git-system is usually not build for large binaries to begin with but they can handle them if needed - with some form of headache.
For your binary assets a simple version of open source content management/digital asset management system beside your code repo is one of the best solutions.

 

[hkennereth]

You can backup a local git repository to google drive by having it in a folder synced to google drive.. or copying it to google drive. Just make sure you copy the .git folder and are not in the middle of manipulating the repo when copying.

The idea of having the repo folder directly synced to Google Drive was one of the ideas I had, but I do have concerns with the two-way sync that will inevitably happen. I don't think that Drive will push anything to the folder that wasn't there to begin with, but... it feels kinda dirty, you know. Manual backups might be less troublesome, or maybe automate copying late at night somehow, but then again that leaves some chance for things to go wrong before the backup happens.

I'm a bit paranoid with backups. Shit always happens.

But once the asset is done and put in to the game it gets pushed on the repo (with LFS) alongside everything else. Selectively putting some files under source control while leaving out others sounds like a recipe for tears.

I agree, that's mostly why I had the question to begin with. I'm somewhat confident that images should be mostly fine even without LFS, but if I decide to add videos into the mix that would seriously complicate things making LFS obligatory. But then my concern is the limitations of storage and bandwidth of GitHub and GitLab, which even for a single developer handling big files they could be too small.

The base of this is related to the overall need for any version control: Do you expect many changes to existing material or not?

My experience with version control is that it's like health insurance. You use it your whole life hoping you will never need it, but the moment you do you will be glad you have because things really, really suck if you don't. I had some seemingly small projects get some major failure down the line, and the only way to solve was to roll back to a previous version. When that happens, you better have access to a previous version, because the alternative might be starting from scratch. Honestly I don't do any coding without version control no matter how simple the project, which is why I'm asking this probably months before I even start working on a game project lol

Regarding if I think these assets will change much or not, that's not really my main concern, but how I must organize them for creating the game. I suppose that for something like a RenPy game I will need to create some folders with all my files, be them ascii or binary, for development and then packing the final executable so if I create a repo for one the others will kinda go along for the ride. So the concern is how to address these larger files so they don't make a repo bloat and become unusable.

 

[toolkitxx]

The idea of having the repo folder directly synced to Google Drive was one of the ideas I had, but I do have concerns with the two-way sync that will inevitably happen. I don't think that Drive will push anything to the folder that wasn't there to begin with, but... it feels kinda dirty, you know. Manual backups might be less troublesome, or maybe automate copying late at night somehow, but then again that leaves some chance for things to go wrong before the backup happens.

I'm a bit paranoid with backups. Shit always happens.


I agree, that's mostly why I had the question to begin with. I'm somewhat confident that images should be mostly fine even without LFS, but if I decide to add videos into the mix that would seriously complicate things making LFS obligatory. But then my concern is the limitations of storage and bandwidth of GitHub and GitLab, which even for a single developer handling big files they could be too small.


My experience with version control is that it's like health insurance. You use it your whole life hoping you will never need it, but the moment you do you will be glad you have because things really, really suck if you don't. I had some seemingly small projects get some major failure down the line, and the only way to solve was to roll back to a previous version. When that happens, you better have access to a previous version, because the alternative might be starting from scratch. Honestly I don't do any coding without version control no matter how simple the project, which is why I'm asking this probably months before I even start working on a game project lol

Regarding if I think these assets will change much or not, that's not really my main concern, but how I must organize them for creating the game. I suppose that for something like a RenPy game I will need to create some folders with all my files, be them ascii or binary, for development and then packing the final executable so if I create a repo for one the others will kinda go along for the ride. So the concern is how to address these larger files so they don't make a repo bloat and become unusable.

You did read my entire post, right? Version/source control systems like git are not build to handle large binaries - that's it. It will bloat your git if you decide to take them 'on the ride' of your code. Especially since you addressed the costs earlier there isnt much choice. Either run your own local/hosted git-version and put everything including binaries into it or split stuff in 2 major parts.

 

[dspeed]

People should listen to toolkit. Do NOT put binaries into source control systems, it significantly impacts performance.

Also, source control systems are not a backup utility. They are a versioning control utility. Backing up of binary files is a different issue and not one that Git et al is designed to solve nor can solve in any meaningful way. And when GitHub/Lab offers the LFS service, I'm not sure that they were thinking about hosting 5GB of incest pornography for you and they have traffic limits.

There's numerous ways for version control for binaries that are used in industry. Helix Core has it built in, there's Nexus and Artifactory too but these are all overkill for a project that will end up on here. Buy a $10 USB stick and use it as a backup drive. Job done.

You must be registered to see the links

(and never forget to adhere to the God that is

You must be registered to see the links

)

 

[hkennereth]

You did read my entire post, right? Version/source control systems like git are not build to handle large binaries - that's it. It will bloat your git if you decide to take them 'on the ride' of your code. Especially since you addressed the costs earlier there isnt much choice. Either run your own local/hosted git-version and put everything including binaries into it or split stuff in 2 major parts.

People should listen to toolkit. Do NOT put binaries into source control systems, it significantly impacts performance.

You both misunderstood my point. What I meant was that when setting up a project you will need to organize all files related to that project into a single folder because that's how the build process works. You will have images folder containing all your pictures and videos inside your main project folder, the one you will likely have as the root of your repo. Sure, you can set that folder to be ignored because you don't want those large binaries in your git repo, but then you now have a synchronicity issue, where your code might be pointing to files that you don't necessarily have in the folder because you didn't backed them up properly. This is exact issue that Git LFS is meant to solve, storing pointers to large files that are being backed up in a separate place outside of the repo but also in an organized manner so that they don't affect performance, but still keep your repo as the single source of truth.

But as I pointed out, I noticed that might still cause issues with using GitHub or GitLab due to restrictions of storage and bandwidth. I posted my original question to know if anyone had had the chance to deal with these types of issues before, not because I think that you should just put all files, including binaries, into your standard repo.

Also, source control systems are not a backup utility. They are a versioning control utility.

I am aware that source control is not a backup. Having it cloned to a remote location, however, is a backup of your repository, which is what I was asking about. If all you care is having version control a local git repo is surely enough, and will avoid all these issues described above.

 

[toolkitxx]

You both misunderstood my point. What I meant was that when setting up a project you will need to organize all files related to that project into a single folder because that's how the build process works. You will have images folder containing all your pictures and videos inside your main project folder, the one you will likely have as the root of your repo. Sure, you can set that folder to be ignored because you don't want those large binaries in your git repo, but then you now have a synchronicity issue, where your code might be pointing to files that you don't necessarily have in the folder because you didn't backed them up properly. This is exact issue that Git LFS is meant to solve, storing pointers to large files that are being backed up in a separate place outside of the repo but also in an organized manner so that they don't affect performance, but still keep your repo as the single source of truth.

But as I pointed out, I noticed that might still cause issues with using GitHub or GitLab due to restrictions of storage and bandwidth. I posted my original question to know if anyone had had the chance to deal with these types of issues before, not because I think that you should just put all files, including binaries, into your standard repo.


I am aware that source control is not a backup. Having it cloned to a remote location, however, is a backup of your repository, which is what I was asking about. If all you care is having version control a local git repo is surely enough, and will avoid all these issues described above.

You are mixing a lot of things into one topic here. You are starting the process backwards here.

Yes - an application requires a build process. But that step is the very last step before actual publishing it. Every organisation or dev has its own setup depending on various factors. It all starts with the actual development of code and content. That's the step where source/version control is essential unless you love to live dangerous and risky.

Content usually gets replaced by placeholders for a very long time since it doesnt matter for code if the graphic or animation is perfect and right. That is also the reason why it is normally not a concern for source control systems to begin with as almost every half-serious workflow will not work with the actual media. In workflows where it does so it is usually a push-flow triggered by the content creators. So they replace the placeholders with the actual stuff. The 'when' again depends on the workflow decided on. I have seen all forms of it by now. From no real stuff until the very last second before the build and versions that worked with the right stuff all the way through. Since the content creators are in control of the media they also dont use the same system as the coders.

Once the coders are done and it is build time the most common step is to run a build-script that collects all stuff into the right positions and packs and compiles the output.

So even if you are the lonely dev trying to do everything by yourself you still have to think with different hats on in this process. And that means: media simply doesnt belong into your source system as it doesnt support what you need for media to begin with. For media one uses stuff like

You must be registered to see the links

for example and pushes all media into the build directory when it is needed. Outside of the workflow you of course also need a file security flow as in backup etc. But that is a general aspect for all computer activity and nothing special in regards to source systems. Backups are not roll-backs nor do they keep control of different versions of the same file.

 

[hkennereth]

I wasn't familiar with this Phraseanet, I will take a look at it. Thanks for the suggestions and feedback.