Stellaris modding
Introduction
I love Grand Strategy games. They are one of my favourite genres and within these I think Stellaris is my most favourite. Because I've been playing it for so long I also like to mod it.
The issues
If you spend any amount of time modding Stellaris you probably know that modding it gets really annoying once you surpass a pretty low amount of mods total. This is because the modding scene is not particularly mature in terms of tooling.
Versioning
Mods by and large don't have good versioning: It is (usually, not always) unclear exactly what versions of mods they're meant to patch if they're patching one. Most mod authors also don't publish good changelogs, if they publish them with any consistency to begin with. Very often they're empty, even on mods that otherwise have pretty solid quality.
Infighting
Because I've spent a pretty significant amount of time patching mod conflicts (more on that later), I've had the displeasure of seeing a distressingly large amount of what I'm going to term mod infighting: Mods will overwrite files that specific other mods make that aren't in vanilla (so it is not an accident). These replacement files are generally empty, so they just end up breaking whatever mod they're overwriting.
This isn't accidental due to the file naming that happens, isn't documented in the files themselves and isn't documented on the mod pages. It is bad form to ascribe malice to unknown motivation (because it's usually indifference), but this type of behaviour is so purposeful that it is hard to see what possible reason it could have.
Sorting
There are sorting rules for Stellaris mods, especially when it comes to patching them. But there are no tools to automatically sort mods properly or at least not ones that are widely used and recommended. Instead the somewhat more mature mods have sorting suggestions in a forum somewhere that mean you have to manually sort everything.
Aside from manually sorting being very sensitive to user error, there's also the issue that most of these mods have different suggestions for what you should do and it is not immediately clear which ones matter and which ones do not. Unless you truly understand how Stellaris mod loading works, you're going to need to gamble, study or go to a random discord and hope you actually get useful advice. All of these are bad options.
For sorting, I did find a tool that seems to get it. That bodes well, but I haven't tried it yet.
Conflicts
Due to the generally wonky way that the game loads mods it's pretty sensitive to super weird mod conflicts which are currently mostly resolved by merging mod changes by hand. This means it's reserved for people who really understand what's going on or more likely people who are very enthusiastic and mostly understand what's going on.
This problem is made significantly more difficult due to a lack of a semantic diff tool and a lack of consistent formatting.
Diff
Currently when comparing two mods, a naive line-based diffing tool is used which doesn't work very well in the best of times and this is not one of them.
Even in software engineering such diffing tools are usually used even though we have much better ones that compare the structure instead which is what we should be moving towards.
Format
The lack of a semantic diff tool is made more egregious by the lack of consistent formatting: The parser Stellaris uses is very permissive, does not use a consistent published grammar and as a result of this most people have a very inconsistent style.
There are no opinionated tools that remedy this. Because of this, line-based diffing tools are even less effective than they otherwise might be.
The good stuff
Irony mod manager
There is a community made mod manager that takes a stab at making the workflow surrounding identifying conflicts and creating merge patches for them less painful:
It is great that it exists and I mostly enjoy using it.
The barrier to entry is sadly very high. The conflict resolution flow, which you will spend most of your time using it in, is pretty far from intuitive (try to figure out how copying a single line on the left side works - without overwriting what is on the right side). Still, that it exists at all and works even close to how well that it does is I think a testament to the fairly small modding community.
cwtools
This is a tool meant to provide the usual accoutrements that an IDE user wants to have when they're editing what is essentially source code. It also provides parsing, which is used by other tools to dynamically generate f.e. in-game tech trees (don't even get me started on that... The situation before these tools can only be called dire) for you.
Future
I wrote this for my own benefit. When I have more energy I'll no doubt return to this subject to write or patch some tools (most likely a formatter, it would solve a lot). It helps to not have to do all this research again.
I think given a formatter, a semantic diff / merge tool, a good sorter and support for judicious rules for automatic patching (add-only is usually okay, empty overwrite files must be discarded and the mod should be moved to the lowest priority for overwrite possible, etc) the situation for Stellaris modding could be improved in a way that lowers the barrier to entry for most people.
Once we are in that situation, a scenario could be imagined where a semantic merge tool has access to two versions of vanilla and a mod for the older version and is able to update it automatically. Perhaps a lot of mods could even be expressed as a change set over base game concept (i.e. "a new civic", "a change to this existing civic", etc). Then a more consistent set of rules and more sophisticated merge engine than the game currently offers can be used to generate a mod set that works. Deterministically, with some resilience against small patches. We can dream.