PDXScript tree-sitter
Preamble
A couple of days ago I said a bunch of stuff about Stellaris after what I realised later was maybe a bit too little research. Most of my improvement ideas have been proposed in cwtools or the irony mod manager github.
I did it anyway
That said, I wanted to learn tree-sitter and it was a nice way to get mergiraf to maybe work on pdxscript files. That way we'll get nice semantic merging without anyone having to maintain yet another tool. I have a bunch of things that I want from my modlist (not least of which is a way to estimate how horrifying it would be to merge a mod to work with others before I add it). Step one in that process is making merges less painful.
Cool things I came across
I came across a few cool things that would come in handy while procrastinating falling asleep yesterday.
tree-sitter
If I didn't lose you right in the preamble you likely already have a pretty good idea what this is, but very briefly put it's a set of tools to make writing parsers more convenient. The big added value for most people is probably syntax highlighting in vscode or (neo)vim. I've written quite a lot of parsers so I was primarily interested in how this might be less painful. And it was actually very ergonomic!
mergiraf
This is a merge tool that allows you to do a syntax-aware merge given a treesitter parser and a little bespoke configuration. When that's done right, you get a merge tool that ignores when the order of expressions has been changed or there are whitespace changes (for example). Those kinds of conflicts (together with "this only added code") constitute what is probably 90% of the merge work that comes with working with a large modlist in Irony.
jujutsu
I also found jujutsu, which is not terribly related but promised to have a nicer paradigm for working with source code while using git under the hood. For me the big draw was the file management (add by default), it's conceptualisation of commits (the current state of the file system is also the latest commit), it's conflict model (you can ignore them until you're ready to deal with them. It doesn't stop you making commits) and the fact that it's still git.
A nice convenience is that it allows you to sign all of your commits before you push, all in one go. This saves me having to unlock my gpg key before I can do anything.
I tried it out just for this, and I enjoyed using it. I'll probably keep doing so.
The code
I put what I made up on my sourcehut. It's not quite as nice as it needs to be to take all of my issues with merging away: it doesn't currently see keywords and I think I'll need to specify types of expressions a bit more. That said, given how incredibly not order sensitive pdxscript is I'll probably be able to make a basic mergiraf config for it and apply myself towards zipping together the 50 or so mods I'm using.
Future
mergiraf
I need to make a config for this and test if it does what I need it to do and it works how I assume it works from the documentation. The examples give me hope :)
mod lists for stellaris
My primary motivator was how painful it is to make a nice modlist that is relatively coherent (nothing with the wrong text, or that doesn't really work, etc) for Stellaris. After the mergiraf config I need to make a little tool that turns this into a palatable workflow.
I'm thinking
- select the mods you want
- have the tool pull them with steamcmd
- do a first-pass analysis for things that are especially script-heavy
- these are usually merge mods or mods that are very baroque. Unless you're really colouring within the lines they're probably not going to work well
- allow you to select one or more mods to mergiraf together (for example ascension perk mods, tradition tree mods, civic mods, those types of categories are liable to interact within themselves)
pie in the sky
Eventually I'd like to automate a lot of the gruntwork here. Automerge what can be automerged, have a database somewhere of problem children (mods that won't play nice), a set of rules about changesets that can always be ignored (empty file overwrites, always=no type compatibility stuff from certain mods) and using the steam api to diff stellaris versions and tell you which mods have touched things that changed in a way that can't be autoresolved.
A lot of these could be integrated into a mod manager for this game that would really improve how many people can use mods without problems. Then I won't have to read so many bug reports that don't make sense in the comments (and presumably the mod authors won't either). A little server that tracks the results of the stellaris version diff analysis as applied to mods can be used to provide people with an extra layer of security about whether a mod still works.