Wrapping a game engine

[Diconica]

Originally I was considering embedding python in my C++ game engine.
Though my game engine is multi-threaded cpython well isn't.

So I was thinking why not make a modified version of the game engine that is contained in a DLL.
Then use python an initializer and setup system.

Most of you probably are thinking what benefit would that give you over SDL or pygame.
Well pygame and SDL are libraries that the primary execution runs in python.
In this case I am talking about building the primary loop inside the DLL. (Yes, that is possible.)
As I stated above pythons job would be to initialize it and setup.

Those of you that are moderately good at programming are probably thinking that you have no control over the loop and how to handle any of that.
Not really a flow model can be used sort of like a script system just it doesn't need interpretation.

So what's the benefit?
SDL2 and opengl access. I can add other like vulkan and maybe one day I'll give two hoots about mac and see what they need.
Plus the performance of C/C++ so 10+ times faster.
You don't need to learn C/C++.

The other benefit is it isn't just python it can be used with given it is a shared library.

If you are wondering how realistic it is well I spent the better part of today making sure it was by coding and testing.
Before it is fully usable there is a lot to do. All I made at present was a very basic system to prove it was possible. That way I would have an idea how much work it will be.

For those of you with an advanced understanding of C++ who are curious how flow control is handled for a variety of games:
A manager class: I have 2 options here.
1. I can have it register classes with it that have a single function for designating the game operational path.
2. I can build the operational path using multiple functions that are registered in the order desired.

If functions A,B,C are registered with the manager the manager then runs the functions in order A,B,C.​

This functionality can be used from low order systems to higher order systems so the general flow of the game to what happens if player A does X to another player.​

The problem with using this method is while it is fairly fast the more you add to it the slower it becomes. You are still having to iterate through a loop of some type as overhead and so on. So using pre-designed methods for each type of game would result improved performance. Either method though allows for multi-threading.​

​

Update:
I figured I would update this to give a better understanding of what actually can be done this way.
If you read below a few response it might answer some question but this might give you a bit more of an idea.
There are technically more options in which I could choose to solve the problem. I chose one that gives good performance but isn't the most amount of work. There is still a lot of work just not as much as there could be.

1. I could write an inline compiler. By that I mean it would be an actual compiler to generate machine code when the game is initialized then run that code as functions. The way GNU's licensing is it can't be used and nor can any proprietary compiler. That said LLVM could be used for this method.​

That would give the greatest performance and highest flexibility.​

a. This does have one interesting benefit. You could write a python parser with it so that you could compile it. Thus the person writing code would be able to modify the internal game code using python and suffer no performance hit doing so.​

​

2. I could write an interpreter. So that they could pass in scripts. Moderately flexible poor performance.​

 

[Saki_Sliz]

Yes, I suppose it would be a challenge trying to figure out how to manage loop control when the main engine is handled in a sort of dll extension, but most of the game and application is in python. While I'm sure there are other projects that exists for python game engines, other than run speed, I imagine the main benifit forr you is since you are making the extension/engine, its not a black box, so you can go in and play with it as you see fit.

I don't know how easy/hard this will be to implement with the engine in a dll like form limiting how 'connected' the main game code can be, but when I was making my own engine using C# and the monogame framework, I found that I could steal the same idea that unity uses, defining a base class that all game objects have to use, with that base class having virtual functions ready for the user to take advantage of, such as start and update. for my implementation, I had developed a philosophy between hard and soft code, as the engine needed to monitor what it was doing, stop tasks to focus on other tasks (multithreading is limited by operating systems and is annoying. if I had money I'd love to use some of teh code made by havok because they spent a lot of time researching the quirks of multi thread issues that come up that are syntax issues, especially for running hundreds of AI like what I wanted to do. Even with C# which has fancy libraries to make the problem easier, it can still lock itself up a few hundred threads in). But I was dealing with the possibility that some game objects load and operate separately from the rest of the game (think of star citizens dynamic object container system, on MMO's where menu's have loading screen because you are basically loading a webpage in game, waiting for a server, etc.), and again with C# I had the benifite that the langauge and library was built with this in mind, no need for third party code or to figure things out on my own. But using a similar, root class, I don't know if its possible unless you limit all classes being passed and processed by the engine are based on a root class.

 

[Diconica]

Yes, I suppose it would be a challenge trying to figure out how to manage loop control when the main engine is handled in a sort of dll extension, but most of the game and application is in python. While I'm sure there are other projects that exists for python game engines, other than run speed, I imagine the main benifit forr you is since you are making the extension/engine, its not a black box, so you can go in and play with it as you see fit.

I don't know how easy/hard this will be to implement with the engine in a dll like form limiting how 'connected' the main game code can be, but when I was making my own engine using C# and the monogame framework, I found that I could steal the same idea that unity uses, defining a base class that all game objects have to use, with that base class having virtual functions ready for the user to take advantage of, such as start and update. for my implementation, I had developed a philosophy between hard and soft code, as the engine needed to monitor what it was doing, stop tasks to focus on other tasks (multithreading is limited by operating systems and is annoying. if I had money I'd love to use some of teh code made by havok because they spent a lot of time researching the quirks of multi thread issues that come up that are syntax issues, especially for running hundreds of AI like what I wanted to do. Even with C# which has fancy libraries to make the problem easier, it can still lock itself up a few hundred threads in). But I was dealing with the possibility that some game objects load and operate separately from the rest of the game (think of star citizens dynamic object container system, on MMO's where menu's have loading screen because you are basically loading a webpage in game, waiting for a server, etc.), and again with C# I had the benifite that the langauge and library was built with this in mind, no need for third party code or to figure things out on my own. But using a similar, root class, I don't know if its possible unless you limit all classes being passed and processed by the engine are based on a root class.

It's not so much hard as it is just work. You have to look at it with a different mindset from normal.
Every game shares some features with each other. The extent of how much is shared depends on the type of game and so on.
The hardest part was figuring a way to make those features accessible without having the game developer actually needing to program in them.

It becomes more of a selection process. So lets say you have a character you want to move around.
In the OP I mentioned having a manager you could register functions with to create code flow.
If you want to have it move on screen you would register a function that modifies the screen position.
Maybe, you don't want the characters actual screen position moving but the world position. So you modify that then when it updates it moves the render position of the objects around the character.
You could attach an AI to a character.
Attach a event system with dialog options and menus... So you can create a vendor... or just someone to talk to and get part of the story...
Or you could be making an FPS or RPG...

So basically all the programming is about making the choice for setting up stuff. Once the engine initializes the job of python is done until it becomes time to shut down.

Game loop:
My game engine is effectively a dual layered state system for the managing thread and the rest of the threads are workers.
Whatever gets dumped in their Queue they do. If you are using SDL2 2D your ability to use multi-threading is limited partly.
I have a nice email between me and the developer were he warns me of it. Not an issue if you are using opengl or vulkan.
In SDL 2D they are using batch rendering they are using 2 threads already. Which run on different cores. If you try to multithread and go beyond the number of open cores it will collide with them and you will end up getting worse performance not better.
So if you have 6 cores and 12 threads you will limited to using 4 threads will in SDL2 2D with out doing more work.

the various states:
initialization, event(KB mouse sys handling), update, render, cleanup
Then inside those there are other states for things like intro screen, the various windows and so on.
They are duplicated between update and render.
The prior features/functions are used in the screen they are going to be visible in.

In truth if it is something like physics, AI updating or whatever it will be tasked off to whatever method or thread will be dealing with it.

As I said not so much that it is hard work just a lot of it. Basically you have to think of all the stuff other people will want to do then provide them a way to do it without them having to have access to the C/C++ and still get good performance.

Multithreading can be some what difficult to get used to once you understand it fairly well it gets easier.
I think it clicked for me because of watching Sean Parents video on youtube.
Even then that doesn't give you everything. You still need to have a grasp of issues around cache collision and how much it helps to fit stuff in L1 cache. Combine those aspect together then you get performance.

 

[Tompte]

I'm not sure if I fully understand what you're saying. It sounds like you're talking about a Python front-end to a C++ back-end, but then you say it's only for initialization and setup. I admit, I don't really follow.

But if you're talking about creating a kind of user-friendly framework that lets users write simple python scripts that translates to relatively advanced things like moving characters around and displaying menus then, yeah, you could do that. Like, more of a scripting environment than an full blown, open ended game engine?

Do you have any example of what a user would actually type when they interact with this thing, in Python?

 

[Diconica]

I'm not sure if I fully understand what you're saying. It sounds like you're talking about a Python front-end to a C++ back-end, but then you say it's only for initialization and setup. I admit, I don't really follow.

But if you're talking about creating a kind of user-friendly framework that lets users write simple python scripts that translates to relatively advanced things like moving characters around and displaying menus then, yeah, you could do that. Like, more of a scripting environment than an full blown, open ended game engine?

Do you have any example of what a user would actually type when they interact with this thing, in Python?

You know how there are math libraries you can use in python to do intensive calculations.
You use python to setup the data and pass the data to the library. After that the main loop is run in the library till it is done.
Which then passes the results back to python.
Python is used to just do the setup and close. All the work is actually done by the C/C++ library.
That's how they get the performance from them.

Well I am doing the same thing, just with a lot more features.
Basically:
You put all your assets in a folder
You load the engine object
You then write a python script to load them into their perspective objects.
You choose the type of screens you will have.
You select the actions for each object and bind them to the IO and screens
After all that is done you start the engine
***
From this point everything is run in the C/C++ library
***
When they stop playing it passes it back to python to shut down.

 

[_retorik_]

If I understood your idea is to create a thin wrapper in python to bootstrap your engine in c++ to do the hardwork.

The point I can see that can be a problem is that, if I understood correctly, the entry point is the script. I would avoid that because it requires python to be installed in Userland.

 

[Diconica]

If I understood your idea is to create a thin wrapper in python to bootstrap your engine in c++ to do the hardwork.

The point I can see that can be a problem is that, if I understood correctly, the entry point is the script. I would avoid that because it requires python to be installed in Userland.

If you mean that the developer has to install python to create a game yes.
If you mean the game after it is created will need python to run again yes.
Renpy has python packaged with it. Hasn't been a problem for them.
Also you can covert the finished product to an executable.

If you mean that python would need to be installed in the game engine itself to allow for scripting. No.
Not required in the least bit.

 

[_retorik_]

I meant by the final user of the game (userland).

I had all sort of problems trying to package python scripts as binaries ... for just one platform.

Not even considering all the hassle that you will have to go through if you want to support another platform. But since you ve chosen c++ I imagine you already count on that.

 

[Diconica]

I meant by the final user of the game (userland).

I had all sort of problems trying to package python scripts as binaries ... for just one platform.

Not even considering all the hassle that you will have to go through if you want to support another platform. But since you ve chosen c++ I imagine you already count on that.

Yea, with C/C++ I already have to create a shared library for each of the different systems it can be used on to start with.
I guess I could choose just to support windows.

I just updated the post above before while you were apparently posting this.

I guess I could do this another way. I could just publish a fully executable game engine in C/C++ and build in an inline compiler.
I could then add in the ability to write different lexical parsers for it and so on. That way people could select the language they like and it compiles it into native machine code prior to running. Hell it could even save that code to a file so the next time it runs it only need load the file and not compile it. Then they could just ship that.

Just looked LLVM already has a python front end written for it. Hmmm. I'm wondering how hard it would be to embed llvm.
They literally added an exception to their license so people can do this.
Well looks like not to hard they even provide a tutorial.

You must be registered to see the links

Hmmm. Now it has me rethinking this.