The latest version of avatar mod features a new AI-gen UI allowing you to easily generated portraits in-game, as soon as you've set up a backend. Here is how it works.


Install the avatar mod. Optionally, test that the pixel-art avatars correctly show up in-game.
https://gtm.steamproxy.vip/sharedfiles/filedetails/?id=3111373293

Set up a local backend for AI-gen. This step depends on what hardware setup you have. If you have no clue what this is about, you should probably go to 2.B.

Update. If you have a local Stable Diffusion instance, @John Jacob Jingleheimer Wick has made the following configurable python script https://github.com/deadmanIsARabbit/RimworldAvatarToStableDiffusion/
You can check out the instructions there.

If you have a fancy GPU setup or a remote AI-gen service, you can set up an executable script which has the following repaint syntax (you can name it whatever you like)
It accepts the name of an image and some prompts, and repaint and replace the image with AI-gen. The actual script content will be depending on the software you are using (if you don't know how, google how to use the software via commandline). It's recommended to use img2img to use the input image as reference, but you can also just ignore it and do txt2img instead and overwrite the image.

For AUTOMATIC1111 here are two example scripts kindly shared by @Saiphe and @Pat https://pastebin.com/PXy0NHn9 https://pastebin.com/Tp9jXXt7 You can also check out their comments below for more detailed instructions (scroll back to comments from 14-16 Apr, 2024).

If you don't have a fancy GPU, you can use my script which makes use of the api of the site https://pixai.art. Note that for now you'll need python installed on your computer. I also do not guarantee this will always work in the future.

• Register an account on https://pixai.art (you can use fake addresses if you want)
  
• Open DevTools in your browser and go to Storage -> Local Storage -> api.pixai.art:token (for Chrome it's under the Application tab)
  
• Copy the value of the token, which should be a long sequence of letters and numbers

• Install python if it's not yet available on your computer. Once done, install the dependency via the following command
  
  python -m pip install requests pillow
• Download the files from this repo https://github.com/bolphen/pixaiAPI
  
• Open the "repaint.py" file as a text file and replace the token with the one you just got
  
• Change "high_priority = False" to "high_priority = True". Save the changes you've made.
  
• Make sure the python script is executable. Follow the instructions for your OS if you don't know how: Windows[stackoverflow.com] Linux[stackoverflow.com] macOS[stackoverflow.com]
  To check that the script is executable, you should be able to run directly
  
  repaint.py "path/to/image.png" "portrait, beautiful"
  without "python" at the beginning (also note that "path/to/image.png" is a placeholder; replace it with the path of an actual image!).

Now you can start the game and go into the settings of avatar. Put the path of the repaint executable (the full path of "repaint.py" if you are using mine) in the last text field. And now you are good to go!

When in game, you can right click on the avatar and choose the "Generate portrait" option.

You'll see a new window with the reference image and some basic prompts.

You can choose whether to draw the headgear / clothes, and you can review and freely change the prompts as you find fit. Once you clicked "OK", the image and the prompts will be sent to the repaint script for AI-gen. The image will automatically show up once the generation has finished.
Enjoy!

If you are having trouble setting the mod up, make sure the script is correctly working outside the game. The instructions in the StackOverflow links should normally work. Alternatives are either set up a .bat script to run the python script and provide the mod with the .bat, or use "auto-py-to-exe" to convert the python script into an actual .exe.

If you have successfully set up the mod but it somehow stopped working, here are a few common pitfalls (if you are using pixai.art)

• You are simply out of credits...
  
• Someone noticed that the api token might change after a certain amount of time. You can try to get the token again and update it in the script.

• For each pawn, the mod tries to read the static portrait and displays it in-game. The static portraits are stored in the avatar folder, which should be
  
  • Windows: %USERPROFILE%\Appdata\LocalLow\Ludeon Studios\RimWorld by Ludeon Studios\avatar
    
  • Linux: ~/.config/unity3d/Ludeon Studios/RimWorld by Ludeon Studios/avatar/
    
  • macOS: ~/Library/Application Support/RimWorld/avatar/
  The portraits are named like "Tynan_Sylvester_123.png". Generating portrait in-game will create a png with this name. You can further retouch / modify the AI-gen portraits any way you like, or even replace it with any other image.
  
• The default settings generate portraits with white or gray backgrounds. You can set up the repaint script to auto-remove the background (for example, using https://github.com/danielgatis/rembg ). Tho it shouldn't be too much of a hassle to remove them by hand (magic-wand!).
  
• If you are using the pixai api script, the cost is 200 credits per generation, so the daily free quota allows you to generate 50 portraits.
  
• In fact, the script is doing 4x batch generations, so if you want, you can go on the website and choose the best one yourself (the script currently simply picks the first one) and manually replace the static portrait with it.
  
• You can also use other models, for example realistic ones: just replace the model id in "repaint.py".
  
• You can add your own AIGenPromptDef: see here[github.com]

• The auto-generated prompts may not be optimized. For example, if a pawn is wearing both a T-shirt and marine armor, both keywords will show up, but you should remove "T-shirt" since it's just noise and might end up making the armor look "soft-textile-like".
  
• Headgear are usually tricky to draw, so they are disabled by default.
  
• Xenotypes are also tricky to get right, and I don't think there's any easy all-in-one solution. Just play with the prompts until you're happy with the results. Maybe someone will train a model for RimWorld one day, who knows :)

Thanks to Tiago, Red007Master, and SirOctogiraffe for comments and feedbacks!

I've set up a discord server, feel free to join https://discord.gg/3FAVpagmK9
The idea is that people who've already set up the script may help others troubleshooting. You can also share your AI-gen portraits and your workflows.