Procedurally generated 3d characters

[Dould255]

Here's a crazy idea:
A game that, given a suitable character description, is able to generate an image of a 3D model and show to the user.

The way I see it working is one of two ways: either by a

You must be registered to see the links

system with prerendered images, or, the much cooler way: by calling a subprocess which takes the parameters as input, and renders an image on the fly.
The options for what to use as such a subprocess is harder, though. There are a few things to consider, in order of importance:

1. It needs to be a 3d program capable of generating chacters from parameters.
2. It needs to be able to output the rendered image in a suitable way, perhaps as just a bitmap file in a specific place.
   
3. It needs a headless mode, with an API of some sort, to allow the game to pass the parameters. We can't have a GUI pop up whenever a character needs to be generated.
   
4. It should be reasonably fast and light on resources, as well as a small install size, to permit running in the background, even on lower-end harware.
5. It should have a repository of asset which allow for redistribution (i.e. creative commons), since assets would have to be distributed with the game.
6. It should be cross-platform, and support at least the three major PC OSes.
   
7. It should look pretty.

The options I have considered so far are:

• You must be registered to see the links
  : Fulfills conditions 1, 2, and 7, but fails 3, 4, 5, and 6.
• You must be registered to see the links
  : Fulfills 1, 2, 3, 4, 5, 6, but in my opinion fails 7.
• You must be registered to see the links
  with
  You must be registered to see the links
  Fulfills: 1, 2, 3, 6, and 7, but kind of fail 4 and 6.

I hear Blender 2.8 would fullfill the speed requirement of 4, through the

You must be registered to see the links

, and technically, ManuelBastioni's characters can be made to wear any clothes that are in the .obj file format, including those in the MakeHuman repo, but that doesn't seem ideal to me.

Do you guys think this is a good idea? Am I on the right track so far? Is there anything I'm missing? Does anyone know of a good repository for CC-licensed hairstyles, clothing etc? Preferably pre-fitted for ManuelBastioni models. And lastly, does anyone know if there is a way to download a version of Blender (preferably a single release for all OSes) without the components you don't need, such as the GUI in my case?

 

[The Architect]

You can run blender as a backgroung proccess and start a python script in a command line:

blender --background --python generate-character.py gender=female age=32 hair=blonde23

Or even try to embbed Blender as a module in any python environment

You must be registered to see the links

 

[Dould255]

As I said, it fullfills condition 3, the problems I have with blender is that as of right now, the options are either to use 2.7x with ManuelBastioniLAB, which renders slowly and is generally resource heavy due to only using ray-tracing, or to use a 2.8 nightly build which has Eevee, but then ManuelBastioniLAB doesn't work. I guess I could just wait for 2.8 to be officially released, but the expected timeline says it's set for beta in July, so that doesn't seem great at the moment.

 

[Rich]

This is the kind of thing you could do with Unity. Unity handles 3D meshes, textures, etc., perfectly well, since that's what it's designed to do. And you can modify textures on the fly. It doesn't necessarily have as hard-core a rendering engine as Daz or Blender, since it's designed to work in real time, but it's pretty capable.

1. Check
2. You have pretty much complete control on this - you can render to a RenderTexture, then save as a PNG or JPG.
3. Not sure what you're going for here - if the game was coded in Unity, this wouldn't be an issue.
4. Not sure this is significant if you code in Unity.
5. The Unity store has a lot of pre-done assets you can obtain. Also, Unity will take in lots of other formats, so you could build base meshes in, say, MakeHuman.
6. Check
7. "Eye of the beholder"

 

[anne O'nymous]

As I said, it fullfills condition 3, the problems I have with blender is that as of right now, the options are either to use 2.7x with ManuelBastioniLAB, which renders slowly and is generally resource heavy due to only using ray-tracing, or to use a 2.8 nightly build which has Eevee, but then ManuelBastioniLAB doesn't work. I guess I could just wait for 2.8 to be officially released, but the expected timeline says it's set for beta in July, so that doesn't seem great at the moment.

I don't remember the name right now, but there's at least one attempt of real time 3D rendering with a pure Python engine. It didn't had the quality you can find with Unity or Unreal, but it worked and both the rendering and speed were good enough for an indie game.
But obviously, even if you use existing Python modules, you'll still need a strong background in codding, and a strong knowledge on 3D and 3D codding, to achieve this. It can be interesting to add realtime 3D support to Ren'py using this way, but in the same times, Unity do better for less efforts, so it don't really worth it.

 

[captainJugs]

This is the kind of thing you could do with Unity. Unity handles 3D meshes, textures, etc., perfectly well, since that's what it's designed to do. And you can modify textures on the fly. It doesn't necessarily have as hard-core a rendering engine as Daz or Blender, since it's designed to work in real time, but it's pretty capable.

1. Check
2. You have pretty much complete control on this - you can render to a RenderTexture, then save as a PNG or JPG.
3. Not sure what you're going for here - if the game was coded in Unity, this wouldn't be an issue.
4. Not sure this is significant if you code in Unity.
5. The Unity store has a lot of pre-done assets you can obtain. Also, Unity will take in lots of other formats, so you could build base meshes in, say, MakeHuman.
6. Check
7. "Eye of the beholder"

I second this.
You could put a daz model in Unity with all the blend shapes (morphs) and change the character "on the fly".

 

[NandabaCanti]

Blender has multiple render engines, Cycles is really the only slow one in the bunch. If you used the Blender Internal renderer, BGE, or even an external renderer like LuxRender then you can save a ton of time on renders. Though they would require you to have some knowledge of setting up proper studio lighting for a biased rendering environment. You'd also have to use different materials.

 

[Droid Productions]

I guess one option would be a cloud renderer of sorts, if all you need is a couple of headshots. Push the data to a script, run DAZ or Blender on the server (so you don't ned to package 4GB worth of DAZ software + models with your game), then download the PNG files to the client. I guess with a bit of work that could be a generic RenPy module that given some parameters can just return images for ready use.

 

[caLTD]

Umm, if you are going to use RenPY or non 3d game framework (Like unity). Paperdoll was your best bet.
Even if you use blender or same kind of engine perfectly you had tons of animation or pose problems.

Making customizable characters in game was tough job.

AND

Making customizing characters interact each other and environment plus clothes was bit tall order.

 

[lancelotdulak]

There are "games' that do this now. Not EXACTLY what youre thinking but Very close. If you mean "an engine that does this on the fly for a game to use to create new characters without input".. that would require a specifically designed API/Engine and it would bulk the hell out of yoru game (it would end up being a dll the program called). The actual problem other than it adding massive bloat to a program for little payoff vs just a large number of prebuilt characters is youd end up with some bizarre characters occasionally and with no human input before theyre showin in your game people would be screaming about it being a 'bug'

 

[deepglugs]

What would be the headless command for Makehuman and can it apply a preset pose? For my game(s), I am creating an AI model that takes a pose image and descriptive tags and outputs a "rendered" scene. It's pretty fast on CPU and can generate several images per second (and even faster on GPU). The hard part is getting the data to train the model. I've been manually creating scenes in blender, setting poses, rendering, changing attributes/colors/clothing, rerendering... you get the point. I'm up to nearly 30k images (in over a month), but to make a full featured game I'd need hundreds of thousands of training images.

I know it's not quite the topic here, but if I could script a character into a scene with a series of poses, in different scenes and animate the camera around I could build the dataset much more quickly.

With DAZ it seems really easy to apply a pose to a model, but it's not headless process and doesn't run on Linux so it's kind of a deal breaker. I agree makehuman models are kinda ugly. *sigh*