Caves of Qud: Godot POC
On September 12, 2023, Unity Technologies—the developers of the Unity game engine—announced that a new Unity Runtime Fee would be introduced starting on January 1, 2024.
Amidst the resulting uproar from game developers over the changes, Brian Bucklew—cofounder and technical lead at Freehold Games, creators of roguelike epic Caves of Qud—took to Twitter to live-tweet his journey of doing a proof-of-concept port of Qud from Unity to Godot.
The following is a transcription of that tweet thread.
time to fuck around and find out (and record my hours as records of damages inflicted)
i should probably build some simple projects to figure it out but instead I'm just gonna try to port qud into with absolutely know engine knowledge because honestly im too old for this shit it cant be worse than ATL/MFC
So I really have 3 tasks here, which I can approach in basically any order
1 I've gotta figure out how to get my assets (first just my tiles) into the game
2 I've gotta lift out the core game assembly and the engine-side game manager and port them over
3 Setup a rendering rig
I'm gonna scream "Hey Alex" (my oldest kid, now an adult) and ask her about each of these 3, because she's been buildng a sonic fangame in Godot 3 and 4 for like 4 years and knows probably as much as anyone out there.
So I've created a godot "mobile" project.
Alex says that I can just copy Qud's texture content into the root project folder and Godot will import it. So I'm going to do that.
goodbye meta files
cool
ok it didn't like the ancient .bmp files, so I'll have to convert them to pngs en-masse
ok that did it for the ascii tiles for now
Ok converted them all to png. Godot loads them when I switch back but it takes like a full 30 seconds for the inport progress to pop up, but it does work.
okey dokey
Ok, I'm gonna go ahead and copy "XRL Application" which is the main non-unity game engine over. Alex says theres a "weird hidden menu option" to generate the cs proj after I do this.
Ok cool, all of the Caves of Qud game is in here. It tried to convert a random CSV to a translation file for some reason but ok, I'll just delete that.
cool an msbuild tab showed up down here. and I have a csproj & sln
5641 errors. Not too bad actually? This step will take me several hours but even on first blush it looks very tractable. Gotta go do some housekeeping.
I'm just gonna have to start pulling stuff over that wasn't part of XRL Applicaiton and probably creating a greenfield "UnityEngine" assembly that just has the same interface; might take me a little bit, but here I go.
ok, for the core shared glue "Console" library, a few core Unity classes like Color were being used.
switching using UnityEngine; to unity Godot; fixed most of these types of warnings, though the interfaces are a little different for Color, they are pretty close
(before the reference change, by comparison; this is simple stuff, but maybe it'll help lightbulb someone that's early career)
Starting with my core "XRL Application", I'm starting to copy over additional support libraries as I work through errors/missing references; thinking carefully about each one.
ConsoleLib and Genkit are some of my general purpose libraries, they come over wholesale...
"Kobold" on the other hand is a very old sprite & atlasing solution I built from scratch before Unity had a sprite and atlasing solution. I'd like to evict it, so I'm going to only pull over files one by one and think about each one and if I want to take the moment to ditch it
KoboldJSON is just a simple json serialization library though, so I'm going to pull it over
I have a bunch of files where Color is being used, and using UnityEngine is not working, so I'm gonna start with a global search replace and just swap all 85 occurrences
That gets me down to 4700 errors; already killed like 15% of them. The last like 5% are going to be hell, but it's a good start.
Looking at my error list, my main errors are simple unity audio and video surfaces that I probably strictly incorrectly added to the core game.
AH I realize I didn't pull over my "GeneratedCode" folder (we have a lot of in Caves of Qud, we codegen classes for each kind of event with a lot of boilerplate, that wins us performance and nice strongly typed event surfaces)
I just killed 70% of my errors! Down to 1557!
I'm mad with power now.
In porting these kinds of systems, these kind of errors will generate new errors of different types as I close them, so once I fix a "I can't find this class" by swapping it out I'll have to deal with "this class interface is different" but it's progress!
Embark Builder/chargen has a ton of unity UI stuff in it. Gonna nuke it for now. We never wrote a console version for it, so chargen will be missing from the initial ascii POC, but that's ok. We can do console versions, and I can test it by loading a game or randomly embarking.
I'll discuss a decision point here. "Game" is mostly our unity->game glue. However some of these folders like "CodeGeneration" are really not glue. I think I'm going to port a few of these folders into a different root folder, when they aren't really Glue. Maybe \Platform?
I'm down to 944 errors.
I've pulled over the above mention core libraries, and now starting to portion out a few internal namespaces into a Platform folder.
Feeling pretty good about this right now, 15 years of architectural choices about game/glue separation are paying off.
Ported over our 'Language' and 'HistoryKit' libraries, and I'm down to 454 errors.
Our Game/Glue split isn't perfect, there will be plenty of webbing I'll have to cut, but even having an imperfect one, and a little work making sure libraries arent deeply engine nested is paying
My next big block of errors is that we use "Color32" in a bunch of places. Which uses bytes instead of floats. I think what I'll do is just write a drop-in replacement from scratch real quick.
greenfielding this based on errors without reference to unitys docs or anything.
getting a lot of grim satisfaction out of creating the "UnityEngineReplacer" folder, gonna be honest.
doing some wide search and replaces...
We use newtonsoft json, so now I'm reading about how to install nuget packages in godot...
holy shit I literally just added the nuget package from visual studio hahahaha. god...
I am down to 402 errors, mostly unity glue...
This is few enough that I can fully take stock of remaining errors.
- CodeDom/Roslyn
- Playfab
- Harmony
- I have no idea some compiler bullshit
That's it! Everything else is UnityEngine surfaces that leaked out of the glue layer into the game layer.
VERY tractable
Having gone from "complete unknown" to "some work but tractable and fairly well scoped" for the POC port in 2 hours, I'm going to get a snack and stretch (important coding activities, remember to stretch I should have done it once already)
Ahhh
[The Offspring intermission music playing]
A snack
[Intermission music continues]
Ahhh.. ok thats better. Back to it.
Frisky found me walking and came with.
The war room (disheveled)
Honestly I don't know what's going on with the IsNullOrEmpty being missing, is it a .net platform vesion issue? was that from Unity? I don't know off the top of my head, and I don't really care atm, so I'm just going to define the extension method in a new /Platform/Extensions.cs
My process at this point is scrolling up and down my error list, looking for the next easiest thing to knock out. I'm down to 364 errors, having just now knocked out the bulk of the remaining "Color" and "Color32" conflicts, mostly adding 'usings'
I need a replacement for a bunch of straight Debug.Log's being used. The drop-in replacement shim is trivial.
Ok, I found out we had written our own IsNullOrEmpty extensions, and I pulled over that Extension library, haha. It's a really nice extension library with hundreds of helpers that make a functional programming style much nicer.
Porting some "Math" references, using the using ='s format. sigh at PI vs Pi; wishing for some C++ macros right now.
Ok, I'm down to 380 errors. I'm going to create a stub GameObject class.
This will convert every "I don't know what the fuck a GameObject is" error into a more helpful error with the missing fields or methods, showing me the interface surface that's actually being used.
This is fruitful, there's actually a pretty slim surface area being used. (There's a few more than this like GetComponent, which I'll start to stub out in the same way, and after I unroll this out I'll have the full port surface area for these objects)
Currently I'm continuing to build out the stub surface area. Errors slightly higher because I pulled in even more support libraries, but this is likely the peak; the actual game is mostly ported, and I'm working on whittling the game/glue interface down to something buildable now
Here's a good example; I'm building out the stub AudioSource replacement interface. You can see the list of members that are accessed by the engine in the error list. Pretty short list, all told, I'll go through and add a stub member for each one.
Ok, 0 AudioSource interface errors, and here is the full AudioSource stub interface used by the game (note this is the 100% port interface, not even just a POC).
100% without reference to Unity docs or code. Simply looking at compile errors for missing fields and methos.
lol Godot uses capital X and Y instead of lowercase x and y. Why u gotta be like that.
Just commenting out a bunch of PlayFab stuff; taking a note that this should probably be bumped into a glue module.
taking a git snapshot because I'm about to try to do whatever the hell it is I need to do to get CodeDom/Roslyn in here (I have no idea what)
Oh it was"add the CodeDom nuget package", easy enough.
Continuing to peel away stuff that's really in the unity presentation layer not the core engine. Down to 232 errors.
187 errors, none of these are blockers or risky; now it's just some elbow grease
Up to 190, but that's because it got low enough that I went ahead and started importing the whole input system >:3
Gonna take a break, time to stretch and relax.
I got distracted and time to hit the sack, but I'll be back on the bullshit to finish this up tomorrow!
with carnitas and caffeine fueling my morning, I'm down to 155 errors and dropping fast. I'll need a nap a bit later, but maybe I can get the qud core sub assembly to 0 before I have nappy time
Just going through and fixing fallout from some of my little quick port shims. Like a handful of errors popping up because I lost implicit Color/Color32 conversion and I can't be assed to write that, quicker to just swap in some simple replacements.
I did write an equals though cause whatever its probably wrong at 255=1 but I don't care right now zzz
We do a big amount of actual async programming, hopping async calls between ui and game thread contexts. This is a surface area that I can imagine breaking in Godot contexst just for nobody ever doing it before (but hopefully not?)
My IDE is helpfully importing "System.Drawing" for Color and "System.Numerics" for vector3s.
Which, really, if you think about it, is not that helpful.
AI am I right?
93 errors. Nothing specific or noteworthy. Just carving out/tweaking old UnityEngine references, and carving away things that really weren't part of the game core and belonged in the glue side instead.
Unlike Unity, Godot has a kind of sane set of namespace names, which means there's a good amount of overlap with Qud's kind of sane set of namespace schemes, so I'm doing a bit of
using Popup = XRL.UI.Popup
etc
73 errors [checking my watch and being a little bored while my brain & fingers automatically fix errors]
37 errors. Time to stretch!
quick and dirty Screen replacer. 28 errors left.
I am down to 11 errors, all of which are in the dynamic C# compilation of mod management. This isn't actually mandatory, but it's also a little bit of pain in the ass to just carve out because it's complex, so it's been waiting for last.
ok, that actually got it to the next pass of linking. another quick batch of errors, these are all just gonna be missing fields on my stubs...
There we go!
Something like half a million lines of .cs building.
Next step is to build a little renderer and input harness and boot this core assembly.
Once that's working it should get the game booting and playable in ascii+tiles mode, without any modern VFX or UI.
That's my goal for this technical proof of concept.
ok so godot seems happy enough with it. Now I have to figure out... literally anything about godot rigging. Gimme a few.
Ok, Alex is helping, I created an empty scene, with a basic node type. I'm going to hit the add script button on the node, and then browse to my "GameManager.cs" file. Which I'll probably have to change to inherit from Node.
inheriting from Node, and Alex says it has to be a partial code because it codegens behind
Saved the scene as "game.tscn" and I'm right clicking to set it as the "main scene" which is the default run scene
Hit F5, it builds and runs. Empty scene cool. Let's get started.
Ok, Alex says _Ready is Awake and _Process is Update. Adding those to GameManager.cs
Let's get the game thread booting!
Cool it's booting, got some preflight initialization I need get ported over before it's fully happy.
Progressing through startup initialization, this is mostly just porting over the initialization steps and making sure my load paths are right.
Mod management initialized as well! Making progress on full boot. Gonna take a lunch break, back later!
ok looks like the remaining big blocker is our refleciton type resolver isn't finding types by name, but it's almost certainly a naming issue of some kind from the new godot harness, so I'll see if I can fix it. Haven't figured out how to attach visual studio to it so printf it
easy fix we were eliding dynamic assemblies from our main assembly check, because our mod assemblies are dynamic; but in godot our main game assembly is dynamic at least in editor so it wasn't being probed for types.
full boot
game, set, match; nothing but net
This is a true full boot of the 500kloc game core, its feeding frames and waiting for input.
The rest is "just work"; though there is a substantial amount of work in the VFX/sound/UI rigging, the derisk is complete.