Procedural based games?

[InfiniTales]

Hi all,

Long time lurker here, but new account created for this. It might fizzle out quickly, or it might go quite far.

In short... for years I've had this idea that tools exist to create games with a lot of procedural content. I'm thinking characters, animations., textures.. Unity game engine tools: Daz models with blendshapes, FinalIK/Puppetmaster, Substances,...

I've always wondered why I haven't seen it implemented much. Have people tried but ran into too many problems?

Would love to hear if anyone has tried this and what their experiences were!

That's the short of it, here's more detail for those interested:

I started out a few weeks ago, and here's where I got:

- I have a Daz based character rigged and working in Unity, complete with a large set of blendshapes. I have a 'randomize' script that applies quite a few blendshapes to change body and face.
- Using Substances, I generate random skin colors based on mixing a few preset colours, random eyebrows,...
- the thus randomly sculpted character has a working Mixamo dance animation.

I'm AMAZED I got this far in about 2 weeks!

My background:
- virtually zero experience in c#, but advanced programming knowledge of PHP and Javascript. I figure the switch to C# will give me least problems.
- Basic to intermediate but mostly outdated knowledge of 3D modeling, texturing, rigging. Some dabbling in animation.
- Beginner but again outdated knowledge of Unity. Some years ago I fooled around with it for quite a bit, but it was mostly checking out what the engine and the various assets were capable of.

First step was to get a rigged Daz model into Unity with working blendshapes. I started by first going from Daz to Blender using Diffeomorphic, and then from Blender to Unity. I got some good results quite quickly but found the process elaborate to repeat with different trials. So I gave Daz To unity Bridge a try, was very impressed at first but then hit a snag with unsupported geograft morphs. So currently I'm back to the Diffeomorphic path.
Second step was texturing and I've had quite some fun creating Substances in Substance Designer (Pre Adobe version). I've exposed plenty of parameters in designer that can be accessed and changed at will in Unity at runtime, using script.
I still have to do hair, create a set of different prefabs, create substances, etc, but aside from time to invest into it I don't really see any major problems there.
Next would be animations and playing around with FinalIK and Puppeteer.

Let me know what you think!

 

[Winterfire]

Main issue is that you probably lack the license to use DAZ3D assets in Unity (different license than just rendering images).

Last but not least, procedural generation of itself is bland and boring... A well crafted character (both personality and looks) will always be superior. Once you play with a few of those 100 characters, they will all start to feel the same... It is the classic quantity vs quality.

 

[InfiniTales]

Thanks for the feedback. =)
I forgot to mention that this is and always will be a hobby project. I have no intention whatsoever to go in any way commercial/profitable with this. That includes no Patreon. This is an experiment mostly to see how far gaming software has advanced over the years. Sooo... Even IF (and that's a big if) I'd go a route where I get profit out of it and thus the Licensing would become an urgent issue, from what I read it's not an uncommon and silently tolerated practice to start buy all the required licenses before going in any way commercial . I'm well aware of the licensing, that's why I'm posting here. =)

Now... the danger of quickly turning into something boring is a very valid point, and I'm well aware of it. In order for this concept to work, it would need a very solid and elaborate base in scripted storytelling! But that's where I got my inspiration: Rimworld, story based gaming. Just imagine a Rimworld foundation enhanced with procedural graphics like characters and animation.

 

[anne O'nymous]

I've always wondered why I haven't seen it implemented much.

Because you aren't looking at the right place and the right epoch. Procedural based games were a trend in 00's and 10's flash games. And there, they proved their limits, that are mostly what Winterfire already said.

Passing to realtime 3D could possibly make those games a little more interesting, but I'm not sure that the efforts/interest ratio effectively worth it.
You can't just put a 3D figures then randomly apply morphs to it. Soon or later, you'll encounter meshes collisions, because the morphs goes too far, or a weird look because it don't goes far enough. Without counting the problems with the animations. Therefore you've to define different bases, that will then receive some randomization limited to a small range.
In those condition, why add complexity to your game since you already have a bunch of pre-created characters ? What would it effectively add to the interest of the game ?

 

[InfiniTales]

True... I quickly found that bodytype morphs like "heavy", "pear figure", "voluptuous", etc... seriously mess up the skinning when dialed too high. When limiting the randomisation of those morphs to smaller values I still get quite some variation, tho. But it's true I would have to create several base rigs for different weight categories.
As for problems with animations, that's why I want to see how far I can take the FinalIK and Puppetmaster assets. Even Unity itself has built in (optional to add) "Animation Rigging" now.

The trick would be either to script placement of markers on the mesh after morphing into a randomised character and use those as (IK) constraints. I looked into it and it seems there are ways to even get the world position of vertices in a mesh. That's for Unity's "animation rigging".
Or, with the Puppetmaster asset route, after morphing the mesh, scriptwise adjust the rig colliders and go full ragdoll puppetmaster powered animation.

 

[Winterfire]

Well it is not like procedural generation was dropped as a whole, it was and still is used to this day.
Point is that a fully procedural generation game would get bland quite fast, but a game with procedural generation used in the right places would be better and even save you quite some time.

To make a small example: It would be like a Skyrim bandit vs a Skyrim named npc vs an important npc in a storytelling game.
The latter would leave more of an impression.

Likewise, both Oblivion and Skyrim have radiant quests, and again, they do not hold a candle to actual written quests.

 

[InfiniTales]

Yeah, The radiant quests of Elder Scrolls are a fine example of the pitfalls in procedural game content. How quickly they get boring. The way I see it, they kept it far too basic. The only procedural elements were a few basic elements like, "get item/ bounty<procedural> at location <procedural> from questgiver <procedural>, and that was mostly it.

I'm again looking at Rimworld, where each npc has LOT of attributes and stats that each influence and drive how they behave and react to events.

But even sticking completely to classic storytelling, adding procedural variation in graphics and animation should be pretty amazing. Think "The Sims"... You see the npc's going through the same animations all the time, right? Now imagine being able to take something like the sleep animation, and then go scripted procedural on that animation. Move one leg a bit this way, a hand that way, by scripts. Have colliders and physics in runtime changing that animation, Puppetmaster and Animation rigging constraints can do that.

That's just a very basic example.

 

[InfiniTales]

Take a common concept in many games here: There's a set of locations that you can visit at diferent morning/noon/evening times a day. Depending on the day and time, location X shows image/scene Y. And when the storyline progresses, that scene or image will have changed into a new stage.

While playing and finding things out, you (or at least I) very often go and check a place and time, see that the image hasn't changed, and I instantly click away and move on. Now Daz images take plenty of time to set up and render, so it's not surprising there are very few or none at all variations.

But that's just it. In Unity it would take about the same time to set up the scene, but once you have established a procedural animation (or even static pose) framework you can make the scene to some degree different every single time with hardly any additional work.

Granted, that by itself would not make that game fantastic, it would still need a good foundation like all the other games.

 

[Winterfire]

I am sorry to say this, but I do not think your idea is going anywhere (at least in the form you are dreaming about).
However, that should not mean you should give up... Go for it, as it will be a good learning experience and give you a better understanding of why procedural generation is good, but not by itself.

 

[InfiniTales]

I am sorry to say this, but I do not think your idea is going anywhere (at least in the form you are dreaming about).
However, that should not mean you should give up... Go for it, as it will be a good learning experience and give you a better understanding of why procedural generation is good, but not by itself.

Again, thank you for your feedback. =-)
Given I have zero experience in game development other than fiddling and trying things out, it's more than likely you'll prove yourself to be right.

But I will definitely keep going for now and see what I find out. I've already had a lot of fun simply playing with Substance Designer to create materials for a Daz mesh and seeing it change in runtime in Unity. PBR materials are and look amazing!

 

[Egglock]

The concept of procedural is everywhere in video games, but most of it is used at the development level of the game more than the runtime. The few games I can think of that uses procedural elements are, Binding of Isaac, Enter the Gungeon, Slay The Spire and a lot of the rogue like games out there.

I believe procedural animations has always been there, it's just that we as the end user will never see it because it's done at the development phases. The other thing is that it might have some performance issue. Animation already have determined data that are stored and can be retrieved easily, when procedural gets introduced the hardware will have to recalculate and trash those generated data. Sure an argument can be made that with our current hardware it's not that big of a deal, but think about what happens when the game starts to have more than 4+ procedural objects being render on screen, it'll only eat into the performance budget.

I don't think a "purely" procedural game is a good thing. There needs to be some control to what and how things happen. Though it may not be pure procedural, having control over certain variables and creating a procedural algorithm around it will yield the best results. Dungeons, environments, non-essential NPC's are good use of procedural elements.

 

[JustAnotherHuman7]

From a person that has nearly 0 knowledge, I believe that an interesting fully procedural game can only be created in conjunction with a powerful AI, which we do not yet have and which we will have in the future. This is because AI can compensate for the defect that procedural games have, such as No Man's Sky, that is, the fact that after a few hours you have seen everything. This is because a very powerful AI, which I repeat we do not have yet, but which we will have in the future, could create totally new scenarios and characters, and also new missions, giving new life to roleplay games for example. Think for example of sites like

You must be registered to see the links

or

You must be registered to see the links

, or text-generation games like NovelAI or what it was of AID, although they are a starting point and are quite basic for now, they portend a promising future for artificial intelligence use on videogames.

 

[InfiniTales]

The concept of procedural is everywhere in video games, but most of it is used at the development level of the game more than the runtime. The few games I can think of that uses procedural elements are, Binding of Isaac, Enter the Gungeon, Slay The Spire and a lot of the rogue like games out there.

I believe procedural animations has always been there, it's just that we as the end user will never see it because it's done at the development phases. The other thing is that it might have some performance issue. Animation already have determined data that are stored and can be retrieved easily, when procedural gets introduced the hardware will have to recalculate and trash those generated data. Sure an argument can be made that with our current hardware it's not that big of a deal, but think about what happens when the game starts to have more than 4+ procedural objects being render on screen, it'll only eat into the performance budget.

I don't think a "purely" procedural game is a good thing. There needs to be some control to what and how things happen. Though it may not be pure procedural, having control over certain variables and creating a procedural algorithm around it will yield the best results. Dungeons, environments, non-essential NPC's are good use of procedural elements.

You make very good points, and I agree with all of them. Performance will definitely become an issue sooner than later. At this point I'm still mostly thinking more tech demo than anything else, but if I ever do get far enough to build some sort of game around it, I'd definitely have to be creative in staying within performance limitations. No mass scenes, preferably no more than two characters, like "something unlimited", for example. No elaborate wardrobes, as limited as possible , hence an environment with maybe same uniforms, etc...

I do still think that hardware advances are taking us closer to one day have in most games what I'm talking about. I'm old enough to remember where it took 1 hour to physics simulate a straight cloth dropping on a ball, and be utterly amazed at the result. On a ridiculously expensive state of the art machine.
Now something like a Unity engine does that same thing in realtime at 400 fps on a 6 year old desktop.

Anyway, I have already given more than a little thought on a concept that would allow me to keep performance requirements at least limited a bit.

As proof, here's a very rough draft of what the introduction text could be like: =D

*Warning beeps*
- _pc_, your attention, please.
- Yes, _ai_, what is it?
- Sensors have intercepted a vessel from the _fed_.
- All right, full scan, let's see what we have here.
- scanning...
- Medium transport cruiser, _x_ class...
- Public records indicate a civilian vessel. Crew complement mostly terran. 1 captain and 1 science officer, the others are Science Academy graduates serving as temporary crew.
- Threat assessment: NONE
- Strategic interest: NONE
- Further investigation required: NONE
- You mean that's it? This is the first vessel that's been passing through the sector in months! Can't we have a closer look, at least?
- That would be a waste of strategic resources and equipment. Not allowed under Section 268, Rule 2.658.
- Ah, come on! Override rule 2658!
- I require your personal code for that...
- Here!
- Rule 2.658 override accepted and logged. Scanning vessel further, please stand by...
- ...
- Vessel does not have military protocols, minimal civilian security only. Bypassing...
- Accessing computer mainframe... done. Full access.
- That was easy!
- A vessel of this nature does not need advanced security, they pose no threat nor interest. Only certain... bored elements would ever bother to use military grade equipment to take a closer look.
- Yea, yea. So, what have you got?
- It's transporting Academy Graduates to _planet_name_. They've just established a new route through this sector. More ships like this will be passing through here from now on. The _Academy name_ is a terran organisation from the _faction_name_ faction. Only women are allowed on _planet_name_. The Academy only accept young girls, usually around age 10. The girls remain on premises for the duration of their training, which is typically 10 years.
- You mean... this entire crew is mostly young women that haven't seen a man in ten years!?
- Correct.
- Whoa! There must have been some major girl-on-girl action in those dorms!!
- If you are referring to sexual activity, there would have been none at all. A little known custom in all Terran military and civilian organisations is that all active personnel is administered daily medication. Most of it are components to boost the biological system, but it also contains sexual suppressants. Sexual activity is considered to be a major distraction and have a negative impact on performances.
- Noooo way!
- Surely you have observed terran _fed_name_ crew missions before? Have you never noticed a complete lack of sexual interests or endeavours? There have only been very few exceptions. Captain Biterious is a notable example, he was one of the only ones to be allowed to skip the suppressants.
- That's... barbaric!
- I believe the Terrans are the ones to consider others to be barbaric, distracted and driven mostly by primal urges.
- So... a vessel full of young women completely oblivious of carnal pleasures. What a waste! All right... you have full mainframe access, can we see any internal sensor feeds?
- Affirmative. Accessing main bridge view...
- Whoa! Who's that next to the captain!?
- That would be Grk' A Trc'h Khoum.
- Gesundheid.
- ... ... I mean... That is her name. She's a _complicatedracename_.
- A what, now?
- ... ... Meaning, someone from planet _complicatedplanetname_. I would have expected someone with your... particular interests to be very familiar with it's existence.
- Yeayea, whatever, would you look at the view she's giving him, bent over like that! And he's not even sneaking a peek! What's wrong with this guy?!
- There is nothing wrong with him. He is simply acting as one under the influence of the daily supressants.
- Everything is wrong with him, believe me! We have to do something! How are those suppressants administered?
- Most of the ship is completely automated and managed by the main computer. That includes monitoring the health of the crew. The Terran sleep pods subtly alter brain waves during the night to accomplish the desired effect.
- ... Sooo... you have full mainframe access, right? That means... we could... adjust those sleep pods a bit? Hmmm?
- In theory, yes. But that would result in major changes in the crew's behaviour. I don't see the point? And besides, it would certainly be noticed by the crew and they'd start investigating.
- Hmm... right... you have a point. Shame, shame!
- ...
- ... Hang on! What if... captain and crew were to receive a communication, orders..., to notify them they are part of a social experiment. A secret experiment! Yes... And to have them expect certain ...unspecified changes in behaviour and it's all part of the experiment... to ... yes... evaluate their ability to adapt to new situations?
- That could avoid any intial suspicion amongst the crew, I admit.
- Right! Do it! Make it so! And show me the crew manifest!

 

[InfiniTales]

From a person that has nearly 0 knowledge, I believe that an interesting fully procedural game can only be created in conjunction with a powerful AI, which we do not yet have and which we will have in the future. This is because AI can compensate for the defect that procedural games have, such as No Man's Sky, that is, the fact that after a few hours you have seen everything. This is because a very powerful AI, which I repeat we do not have yet, but which we will have in the future, could create totally new scenarios and characters, and also new missions, giving new life to roleplay games for example. Think for example of sites like

You must be registered to see the links

or

You must be registered to see the links

, or text-generation games like NovelAI or what it was of AID, although they are a starting point and are quite basic for now, they portend a promising future for artificial intelligence use on videogames.

Yea, I agree that trying to have main story elements created by an AI, procedural generator, is far too soon. That's why I'm mostly thinking only procedurally generated variations on scenes, graphically, not story. Or at least very limited storywise, like in Rimworld or Degrees of Lewdity.

 

[Jofur]

There are a few games that do this, most notably LifePlay. It's interesting, but troubled, and seems to have fallen into many of the pit traps that make randomly generated content boring. But it's probably the most extensive project out there when it comes to the subject.
You also have modded The Sims 4, or even Crusader Kings 3, not to mention a lot of text based games, which does similar things.

The issues with mixing hand crafted content and randomly generated stuff is that you need to either have a ridiculous amount of possible variation when it comes to dialogue and scenes to fit every custom personality and character(it's always annoying and disappointing when randomly generated characters repeat themselves or all act the same as each other), or you need focus entirely on AI + cause and effect, which is sort of what Sims and Crusader Kings does.
While not randomly generated, Artifical Academy 2 did pretty well merging both AI and hand written dialog and scenes I thought, but being made by Illusion it was abandoned before it ever reached it's full potential.

I've played around with it a bit myself, and it's hard. In the end it might just be best to give up and be okay with the fact that characters and sex scenes will never be as intense as hand crafted content, and instead hope the gameplay will make up for it.

 

[JustAnotherHuman7]

I've played around with it a bit myself, and it's hard. In the end it might just be best to give up and be okay with the fact that characters and sex scenes will never be as intense as hand crafted content, and instead hope the gameplay will make up for it.

I would not say "will never be as intense as hand crafted content", like I said before, in a more or less distant future, depending on the speed of technological progress, this will be probably possible. Think of the progress AI made in the last decade, the thought that sites like the ones I listed before could have been created in a decade was impossible even for people as optimist as Elon Musk, and we all know the constant meme of his overly optimistic predictions. Considering this, the most rational thing to say is not that it will never be possible, but that it might be possible in the future.

 

[Jofur]

I would not say "will never be as intense as hand crafted content"

Well, never with the current technology. I mean there is already stuff like AI Dungeon which has some surprisingly tasty(if sporadic) porn capabilities. I suppose the biggest hurdle is that most people who work a lot with cutting edge AI aren't especially interested in the porn applications, and it's gonna be even longer until all these AI tools like GPT-3 are easy enough to work with for your average game developer. At least without first doing a deep dive into AI research. However I think AI driven design tools is gonna be a huge deal moving forwards. Things like machine learning have already done amazing job in upscaling old games like Resident Evil or Final Fantasy 7, and it's only gonna get better.

 

[InfiniTales]

The focus on "procedural" seems to be going mostly in full AI storydriven direction. That's okay, but not entirely/everything that I have in mind.

An example: I just finished a 'procedural' substance for the very complex Daz eye meshes. That leaves only a few left to do, eyelashes and whatever 'tear' is exactly. Now I could've easily used a few different image texture maps and saved a lot of time. Instead I spent many hours into creating procedural versions of the different maps.

Advantage? I'm working with 2k maps. That is, 2k not as 2k resolution, but 2kb filesizes! (mostly svg vector images for the more complex masks.
This gives me a filesize footprint a fraction of what bitmap images would take me, AND almost infinite customisation during runtime. I had so much fun that I went a bit overboard and even have 2 sliders now for pubic hair gradually growing. I can select a trim from a few options (svg masks), and with a slider have it grow out into a full bush. =D
And again in runtime I can set up to have all texture maps go as high as 8k if I wanted to, with hardly any loss of detail. Procedural.
Performance in Unity is excellent so far, but I'll have to see what happens once I try modifying multiple substances at once.

Now, I am also thinking of possibilities to enhance story elements as well, don't get me wrong. But as many have pointed out, there's a lot of pitfalls and examples of "not quite" out there, like No Man's Sky and Elder Scrolls' radiant quests.

To really add something extra without failing, it would take several times as much work in storytelling, and then have a randomizer select from different possibilities and chain things together without failing. Each and every possibility would have to be given equal detail. A LOT of work.

So for now I just keep fiddling. Soon I'll tackle animations and see what's possible there these days.

Interesting discussion, all!

 

[Jofur]

So for now I just keep fiddling. Soon I'll tackle animations and see what's possible there these days.

Procedural animations is what gets me really excited. Not just in making animations themselves easier to make and nicer(less clipping issues and what not), but also less repetitive. That is a major issue with most games, procedural or hand crafted. I can bang 30 different characters in Artificial Academy 2, but no matter what the doogy style animation will always be the same. Even if the character has a completely different personality or build.

 

[InfiniTales]

Yepppp, that's exactly my reasoning! I haven't really tried it all yet, but even years ago I looked into features like Unity's animation layer blending. Something like creating a set of animations for different facial expression reactions, containing keyframes only for the head bones.
That can then be layered and blended on top of another animation that drives the body, etc.
So... in theory, you could set up a number of partial animations, and depending on parameters select one among them to run on top of a main animation. And at the very least it's not even required any more to keyframe anymations for certain things. Lookat IK constraints automatically having an npc/pc look at exactly a certain (moving) point by script are common now. Placing a hand exactly against a wall, picking up an item with the hand exactly in the right place... Those hand placements are already script driven instead of truly animated.

In years past that's something that's really evolved, judging by demo videos.