10 users online: Ade2995,  Alex, bucketofwetsocks, buggy789,  Burning Loaf, Dzing, Green, MANGOMILK, monkey03297, Swanz - Guests: 176 - Bots: 133
Users: 59,942 (2,182 active)
Latest user: yuri2138

Version controlling hacks: Lunar Helper, Lunar Monitor and git

Super Mario WorldToolResource Release

Would you be interested in a text guide (with screenshots!) on how to set these tools up and use them with git?

94.1% (16 votes)
5.9% (1 vote)

If you want to use these tools with an existing project, please keep backups around! These tools are still quite new and only tested on a few systems so I cannot guarantee that there are no bugs present that could screw up your ROM and/or project folder!



Some people I would like to give credit to right off the bat:

- Maddy Thorson for creating Lunar Helper in the first place
- Atari2.0 for adding logging functionality to Lunar Monitor and looking over my code
- AmperSam for first making me aware of Lunar Helper

So, what's all this about?

Hi, today I'm breaking my ~1.5 posting hiatus to bring you a bundle of two tools I have found incredibly useful, as well as a full text guide on how to set up an example project using them later if people are interested (though readmes are of course already included alongside the tools!). Please let me know in the poll if you would like to see such a guide!

I was made aware of Lunar Helper a few months ago and since then the approach I've been using for hacking has changed quite significantly. I've never hacked as often as in the past year and I believe this new approach contributed a lot to this productivity spike, so I decided it was something worth sharing right now (despite me also wanting to save my 2000th post for a different project).

If you've been hacking for a while, you've probably felt the pain of somehow introducing a bug into your ROM that seemingly came out of nowhere, being unable to port your ROM to a clean one or maybe you've even lost a ton of work because you somehow lost access to your hack and had no backups anywhere.

I've always felt like hacking was a quite precarious activity, like the state of my project was always in a state of flux where something could go terribly wrong at any moment. Yes, there was always Lunar Magic's recovery system, but since it only actually recovers the state of your ROM at a certain point and is pretty unwieldy in general (in my opinion) I have never found it very useful personally.

I became familiar with git a few years ago because I was learning how to program and its a widely used tool in those circles. Essentially, it allows you to save snapshots of your files whenever you want and to then restore them to the state they were in at the time. Git has a lot of other very useful features, but that is its most core functionality.

One day, I asked in the SMWC discord, whether anyone had experience using git for SMW hacking, since I had previously used it for programming projects and was eager to see if I could apply the organizational benefits it brought to those projects to hacking as well.

AmperSam pointed me towards Lunar Helper, a build system for SMW ROMs created by Maddy Thorson, which could take a bunch of resources (level files, patches, sprites, blocks, map16, graphics, etc.) and insert them all into a clean ROM to produce a hacked ROM with just the press of a button.

It could also extract the needed resources (MWL files, PAL files, MAP16 files, etc.) from a ROM just as easily.

Obviously, this is a great fit for git, we can save snapshots of all these extracted files, our blocks, sprites, etc. and then build our ROM from them whenever we want! We don't even need to store snapshots of our ROM, because we can just build it from the files we extracted from it. Not needing a ROM is really nice, because that means we can share our snapshots with other people without having to share actual ROMs (which would of course be illegal) and they can just build the ROM from the same files using Lunar Helper!

I gave this a try for a bit. I would edit things in Lunar Magic, extract all the resources from my ROM using Lunar Helper's Save function and then use git to create snapshots of the extracted files. But I noticed some small but annoying problems with this approach. Git was detecting changes in level files, even though I had not edited those levels since my last snapshot, what was going on?

Turns out Lunar Helper extracts all levels as MWL files whenever you use its Save function, because of course it cannot actually tell which levels you edited. Unfortunately, Lunar Magic will apparently sometimes shuffle the data in levels slightly, despite them not actually having changed. As a result, the extracted MWL file for a level may be different from the previous one, even though the underlying level has not been touched at all!

This was not necessarily a huge problem, I just had to tell git to discard those "changes" (keeping the changes would make it harder to tell what changes were made between two snapshots and also waste space) by hand each time, but at some point I realized that having to keep track of which levels I changed and which were just superficially shuffled by Lunar Magic was actually quite a nuisance and would take up an annoying amount of time and just make the overall workflow less comfortable and more error prone.

I started thinking about how I could make sure only actually relevant changes would result in resources being exported from the ROM. After attempting fruitlessly to figure out which changes to keep by altering Lunar Helper itself, I realized I had to go to the source, where all these changes stem from, Lunar Magic. If Lunar Magic could just extract these resources whenever we save them to our ROM, that would ensure that only changes we actually made ever make it to git. We would also no longer need to use Lunar Helper's Save function to extract resources, it would all just happen automatically!

But how could I make Lunar Magic do this? It's a well-known fact that Lunar Magic is closed-source, so customizing its code was out of the question.

Or was it?

Researching this topic, I learned about a technique called DLL injection, which essentially allows you to hijack a running program's code and alter it while it's running.

Of course I had to give this a try immediately. I wrote a program to inject a DLL into Lunar Magic, found out where its functions for saving levels, map16, shared palettes, etc. are stored in the program and hijacked them to export all these resources immediately after they are saved to the ROM, as long as the save was successful.

Surprisingly, it totally worked. Now, given that hijacking an already compiled program is a little tricky, there were some small issues, Lunar Magic would no longer display error message boxes when it encountered an error while saving some resources to the ROM, but with some pretty hacky workarounds I managed to make those error messages pop up again and had a fully functional altered version of Lunar Magic.

Once I was satisfied with my whole project setup, I got to actually hacking. I was using a dropbox folder before, since they sort of keep backups of your files too (but only for 30 days and in an unorganized manner) and I would never go back to that workflow again. Since August I've taken roughly 330 snapshots of my project at various points, meaning that I could restore 330 versions of my entire project and re-build the state of the ROM at that point whenever I want to. I've also pushed these snapshots to GitHub in a private repository, meaning that even if my PC were to die and all my local data to be lost, I could still restore all of these 330 snapshots of my project and rebuild my ROM from them.

These 330 snapshots are currently 34.3 MB in size, which is 3.43% of GitHub's 1 GB repository size limit, meaning I could keep working at this pace for about 10 years until I reach their limit. Note that GitLab purportedly has a 10 GB limit, so if you're planning on working on your hack for ~100 years or you're a little wasteful with your space, consider using GitLab instead.


Having said all that, I am now happy to present you a bundle of both Lunar Helper and Lunar Monitor (as well as a standalone version of Lunar Helper) which I submitted to the tools section earlier! The Lunar Helper version bundled with Lunar Monitor has its Save function removed but is otherwise identical to the standalone version.

Please note that Lunar Monitor is currently only compatible with Lunar Magic 3.30 and 3.31, so if you're using an older version you will have to stick with using the standalone Lunar Halper tool and extracting resources via its Save function!

Bundle Download
Bundle Dropbox Mirror
Lunar Helper Only

The source code for both included versions of Lunar Helper can be found on my GitHub, the original version can be found on Maddy Thorson's GitHub. The DLL part of Lunar Monitor can be found here and the program that injects the DLL here.

There is a lot of text to read through in the included readme files. Setting all this up is actually not as much work as they make it look, so there is a poll at the top of this thread asking you if you would like me to post a little guide you can follow along to get an idea of how this setup may actually work in practice (and maybe to also get a rough idea of how to use git if you haven't used it before, though I won't go into incredible detail or explain all its features since there already resources available for learning basic git functionality!).

Final thoughts

One thing that currently still irks me about this approach is that map16 data is saved in one large all.map16 file. This makes it somewhat hard to work with map16 changes, because git will not be able to tell what exactly the difference between two all.map16 files is, i.e. which tiles changed between them. This means that we have to manually and somewhat painstakingly resolve conflicts between map16 files whenever they occur. Ideally, all.map16 and all other resources Lunar Magic exports would be in some text-based format (JSON for example), which git handles a lot better than binary files. Alternatively, it would be nice if Lunar Magic let you export every map16 tile as its own map16 file, though it's probably unlikely that either of these will ever be implemented by FuSoYa.

I've thought about implementing either of these myself, after all I can hijack Lunar Magic's code, but the amount of effort that would be required to do so would make the effort I had to put into Lunar Monitor pale in comparison.

I'm not sure if enough people are masochistic enough to attempt similar hijacking of Lunar Magic, but if so maybe it would be cool to create a sort of plugin system that would make it easier to inject DLLs for various functionality into Lunar Magic. Just something to think about.

If you've made it this far, that's it from me for now, thank you for reading, I hope you enjoy the tools as much as I do!

If you think there is a problem with the tools or you have questions feel free to leave a post and I'll try to respond in a timely manner!

See you later for the full text guide (maybe!) and have a nice C3!
I know we've talked about this privately but I love that this exists because it is exactly what I need for my overstuffed hack. I'm very excited to use this.
Whoa! This looks like a really helpful tool for SMW hacking! Now I'll never have to worry about losing progress on my hack...not that I've been working on one. >_>

Also, congrats on 2k posts!
You're a real hero, Underway. Now that I want to set up my hack project even more flawless than ever before. Better try this one out! (And hey, I would also remind you that Maddy Thorson is YMM on this site, right?)

Also, 'grats on 2k!!
Expect to have more intel about The Hacking of PuyoPuyo coming soon. 『いけいけ団長、頑張れ頑張れ団長!』
Credits to Nenilein (a.k.a. @PinkGeekNeni on Twitter) for the Pride avatar!
Help us raise funds for the Armed Forces of Ukraine. #ДопомагаємоРазом / #HelpTogether
“Even if you personally are so dissatisfied with life that you want the world to end, surely the cruel reality is that it will continue on, unchanging. All the better for someone perfectly content, like me.”
Aya Shameimaru, Touhou Suzunaan ~ Forbidden Scrollery
ive been avoiding using git in smw hacks because i didn't know how beneficial it would be with how complicated and cluttered folders of hack content can be.

that said, this is insane, and i will be using it in all future projects. nice work.
can this be used to yoink stuff from protected roms
Thanks for all the nice comments everyone! Given that several people seem interested, I'll probably be posting the text guide I mentioned either tomorrow or the day after. I'm a little busy the next few days and want to make sure that I'm including all the stuff I think is really important so please bear with me a little.

Originally posted by Klug
(And hey, I would also remind you that Maddy Thorson is YMM on this site, right?)

Thanks for reminding me, I remembered having known her account at some point, but I couldn't remember the actual name when I was submitting, will update that asap!

Originally posted by aaa
can this be used to yoink stuff from protected roms

I'm not particularly familiar with what locking a ROM actually does, but I would assume no. All Lunar Helper and Lunar Monitor really do is use Lunar Magic's command line arguments. So if using the command line arguments can't extract protected stuff then neither will Lunar Helper or Lunar Monitor.
I can see this being a helpful tool for hacking later on very nice

You are challenged by Champion Daisy!

its funny. i recently got into skyrim modding, and one of the major modding tools there (mo2) lets you organize the load order of your mods and resolve conflicts without touching the skyrim folder or executable, and wondered why we havent done something similar for smw: build out the rom without touching it

this is a wonderful step into that direction. thank you for your time and effort on this (as well as to maddy and atari)
Very cool updates, you and I have already talked about the Lunar Helper, it and some of these other tools you've made here might break me of my habits and I may try and switch to this sort of workflow for future projects. :D
It's something SMW hacking needed for quite some time, when collabs started to become more popular. However I have read the process to make the base patch (thanks FuSoYa) and ... would it be possible for Lunar Monitor to do it automatically for you?
Originally posted by Disk Poppy
However I have read the process to make the base patch (thanks FuSoYa) and ... would it be possible for Lunar Monitor to do it automatically for you?

Creating the base patch is relatively cumbersome right now and I've been thinking about how to make this process less annoying. I was thinking of just including a base patch with Lunar Helper, but the problem is that depending on your needs, you may require a different base patch setup. One thing I probably could do is including a few common base patches (FastROM, SA-1, different compression settings (maybe?)) and explaining how to choose the one that fits your needs.

You could still create your own custom base patch for more specific needs, but I think a few base patches would cover most people's requirements.

I'll probably look into what kinds of base patches would be useful to include and update the submissions at some point.

(Text guide still in progress btw, might post it later today!)
Alright everyone, here's the guide as promised! It turned out a little long, but I think as a result it should be detailed enough to actually convey basically everything I myself currently know and think of this workflow. Please take your time and let me know if anything is wrong or you run into problems, thank you!

Firstly, please make sure you are familiar with using Lunar Magic, GPS, PIXI, AddMusicK, UberASMTool, FLIPS, Asar and just hacking in general at least a little bit. If you aren't, I'm sure you can find plenty of useful resources in the tutorials section. I would really only recommend this sort of setup if you are already pretty familiar with the basic tools and tasks involved in hacking.

Brief intro to git
This will be a short introduction to what I think are the most important parts of git to get you started, but you should feel free to check out other resources on git, since there are plenty out there (here's just one for example and a cheatsheet Atari recommended) and probably a lot that are better and more thorough. If you're already familiar with git or have consulted some other source for the basics you can safely skip this section, I will talk about using git specifically for hacking later on!
As previously stated, git in its essence is a program that allows you to save snapshots of your files. It's actually a little smarter than that even, instead of saving an entire backup of your work, it will only save the changes you made since the last snapshot. If you go back to a previous snapshot, it merely reverts the changes between your current snapshot and the one you want to revert to. In this way, you can think of these "snapshots" not as separate things, but little checkpoints that connect to form a path from the start of your project towards the current state of your project, each containing little changes that together add up to the entirety of your work!
Here are some terms that are pretty essential to git's workflow:
commit: What I previously referred to as "snapshot" or "checkpoint", it's a set of changes you've made to files since your last commit. You can add a title and description to your commits to state and explain the changes you made, which makes it very easy to tell when and why what was changed[1].
repository (often abbreviated to repo): A structure git creates which contains the history of all changes you ever make to your project, basically a collection of all your commits.
push: A repository can be "pushed", meaning essentially "uploaded" to some remote location, usually a website that specializes in hosting git repositories like GitHub, GitLab, Bitbucket, etc., from which it can later be restored.
pull: You can "pull" or "download" changes made to your "remote" repository (the one hosted on GitHub or a similiar site) to integrate them into your "local" repository (the one you have on your own computer).
clone: You can "clone" a remote repository to create a copy of it (a local repository) on your own computer. The difference between pull and clone is basically that clone creates the repository again from the ground up (for example if you somehow lost your local repository), while pull only applies new changes it finds in the remote repository to your local repository (for example if someone you're working with made some changes since you last pushed).
staging area: The staging area is where you add your changes to before you commit them, its use is basically to let you split your changes into multiple smaller commits and to make it easier for you to tell what you are actually committing in the first place. Changes that are currently in the staging area and not yet committed are often said to be "staged".
branch: I previously explained how your commits form a path from the start of your project towards the current state of the project, this was a bit of a simplification, with git you can actually have multiple such "paths" or branches, that can split, merge or end at any point where there is a commit, forming something similar to an actual road network. Branches are useful if you want to work on different aspects of your project in isolation or if you're working together with others, but there are some pitfalls specific to using branches while hacking that you should be aware of that I will explain later! As you may expect, you must always have at least one such branch, or your project would not even have a path to walk along, this is called the "main"[2] branch.
conflict: A (merge) conflict occurs when two or more conflicting changes are made to the same file. In a solo project you should normally never run into a conflict if you're only using one (the main) branch. Resolving such conflicts is a relatively simple affair with text files, you simply pick the lines you want to keep or discard from each set of changes, but unfortunately basically all the files Lunar Magic exports are binary files, where "lines" to keep or discard don't really exist, which complicates matters. I will discuss specifics about merge conflicts and the problems with binary files in the context of hacking later.
.gitignore file: An incredibly important feature of git, .gitignore is a file you can create that contains specifications about which files or folders you want git to ignore completely. Any changes you make to files specified in .gitignore will be ignored by git, it will basically fully disregard their existance and never add them to the repository. Obviously this is very important in our context and I will talk about which files we probably want to ignore later on (spoiler: ROMs!!).

Some basic git commands

I will now list some important git commands, you can find a list of every command git has here.
git clone <url>
Command used to create a local copy of a remote repository. You can obtain the <url> to the specificy repository you want to clone from the website the remote repository is hosted on.

git init
Command used to create a new repository in the current folder.

git add <path>
The command we use to add changes we made to the staging area. The <path> specifies the file(s) we would like to add. You can specify the path to a folder instead of single files too and git will add all changes to files it can find in that folder or the folders it contains to the staging area.
For example, if I'm in the root of my project and I do 'git add levels' git will stage all changes it can find in the "levels" folder. 'git add .' is very useful (note the dot), it will add all changes it can find in the current directory (denoted by the dot) or any folders inside it. Using this we can quickly add all changes in our whole project to the staging area if we want to!
Note that you can use git add as often as you want to, you don't have to add all your changes in one go.

git status
This command will show you which changes git currently sees and which ones are already in the staging area.

git commit
The command we use to commit the changes that are currently in the staging area. By default, if you use just "git commit" you will be prompted via some text editor to enter a commit message, if you want to just write a quick little note instead, you can do 'git commit -m <your commit message here>' for example 'git commit -m "Remove a Goomba"' will commit your changes with the message "Remove a Goomba". I would generally recommend using the text editor though, since you ideally want to be as descriptive as possible for the sake of organization.

git log
This command will show you the history of your repository, meaning your commits and commit messages as well as which branch they were created in, when they were created and by whom, as well as their hash (basically an ID that can be used to refer to the commit). The default git log format is not the prettiest so see [3] if you want to learn how to make it a little easier to look at.

git branch <name>
This command lets you create a new branch at the current position with the given <name>. For example, I could create a new branch called "rework_world_1" via 'git branch rework_world_1'.

git checkout <branch or commit hash>
This command lets you "check out" or "switch" the state of your project to the state it is/was in in the specified branch or commit. To checkout a commit, get its commit hash using git log and then do 'git checkout <hash here>' for example, if I wanted to check out a commit with the hash "283a94f" I would write 'git checkout 283a94f' and to check out a branch with the name "rework_world_1" I would write 'git checkout rework_world_1'.
The latest commit in a branch has a special name called "HEAD". To switch from a previous commit back to the latest state of your branch, let's say it's the "main" branch you can thus do either 'git checkout HEAD' or 'git checkout main'.
Note that if you're in a previous commit git will tell you that you're in a "detached head", this is nothing to worry about as long as you do not make any changes until you return to the latest (aka HEAD) commit as described earlier, see [4] for more info on "detached head".

git restore <path>
Inverse of git add, lets you remove files from the staging area if you accidentally added them.

git push
Command used to push changes to a remote repository.

git pull
Command used to pull changes from a remote repository. This command can cause merge conflicts under some circumstances.

git merge <branch>
Command used to merge <branch> into the current branch. Basically means git will try to automatically integrate the changes from the <branch> into your current branch. This command can cause merge conflicts under some circumstances.

Here are a few more useful commands which I use sometimes but will not explain in detail here (you can find plenty of resources on them elsewhere if you're interested):
- git stash
- git rebase
- git tag
- git reset
- git bisect

The actual setup
I hope I didn't scare you off with all the git commands and such. They may look scary at first but in my experience you get used to the workflow pretty quickly, especially because to start out with you don't actually need that many commands.
Let's now actually set up our example project as promised!

Required resources
Now, I'm going to list a bunch of stuff you will need to follow this guide from start to finish, you may either download these all right now and keep them handy for later in the guide, or you can come back and download them as soon as we actually need them, up to you:
- git (I would recommend installing it immediately if you don't have it yet)
- Lunar Magic 3.31 or 3.30
- AddMusicK
- Asar
- UberASMTool
- FLIPS (you may get the version from this comment for now, the version SMWC is hosting is outdated by 6 years and has a bug that makes Lunar Helper not work, note that I cannot guarantee that the version in this comment is safe so you'll have to download and use it at your own risk, wait for SMWC to update FLIPS or compile an up-to-date version yourself!)
- Lunar Helper + Lunar Monitor
- initial_patch.bps (will be explained later!)
- GitKraken (optional), offers a nice GUI for git but is only free for public repositories (or if you're a student), so I will be sticking to using git via the command line for this guide

Folder structure
For the purposes of this guide, I will create a new folder and call it "example_project". I will then create two empty folders in it and call one "my_hack" and the other "tools". "my_hack" will be our actual project folder which we will version control with git and "tools" will contain all our tools. It's also totally fine to keep your tools in your project folder and to version control them too, I'm just leaving them out of the project for simplicity. I will talk about pros and cons of keeping your tools inside/outside your project a bit at the end of this guide.
This is what your "example_project" folder should look like now:

Now I'm going to open up my "tools" folder and just dump all the tools I need in there. It should look like this afterwards:

Here are links to screenshots of the contents of each folder just for completeness (nothing exciting, just extracted the files from the links above):
- AddmuscK_1.0.8
- asar181
- GPS (V1.4.21)
- Lunar Magic 3.31
- pixi_v1.32
- UberASMTool14
Note that I did add a (for now) empty "list.txt" file in PIXI's folder and you should do so too, because that's the sprite list file Lunar Helper will tell PIXI to use later!
At this point, open the Lunar Helper + Lunar Monitor bundle's "lunar-monitor-v0.3.1" folder. Take the "lunar-monitor-injector.exe", "usertoolbar.txt" and one of the two DLLs in the "LM3.30" or "LM3.31" folders (depends on which Lunar Magic version you're using, I'm using 3.31 for this guide) and copy them into your Lunar Magic folder. Afterwards, it should look like this:

While you're here, also take the "lunar-monitor-config.txt" file and copy it to your "my_hack" folder.
Let's make a "Lunar Helper" folder inside "my_hack", then take the "LunarHelper.exe", "config_project.txt" and "config_user.txt" files from the Lunar Helper + Lunar Monitor bundle's "lunar-helper-v1.0.0-LMC" folder and put them in this folder.
Afterwards, your "Lunar Helper" folder inside your "my_hack" folder should look like this:

and your "my_hack" folder like this:

Let's add a few more folders, a "Levels" folder and a "Other" folder which will contain our extracted resources later, as well as a "Patches" folder for our patches to go:

While we're at it, let's also add a "clean.smc" and "my_hack.smc" like this ("clean.smc" should obviously be a clean ROM, "my_hack.smc" will also be a clean ROM for the purposes of this tutorial but if you were porting an existing project this should be your hacked ROM):

To finish our folder setup, please take the "initial_patch.bps" file I linked above and copy it to the "Other" folder.
And that's it for our folder setup, next we will be configuring Lunar Helper and Lunar Monitor!

Let's start with the "config_user.txt" file in your "Lunar Helper" folder. Open it in a text editor of your choice and you should see something like this:

Most of this should be pretty self-explanatory, just change the paths to the paths of the tools we set up (most of them should already fit mostly), you can delete the retroarch variables for the purposes of this tutorial, or, if you have retroarch and know where it is, you can change those paths too! (Note if you're not that familiar with relative paths: ".." denotes the parent folder of the current folder, so in our case, ".." would refer to our "example_project" folder, since our "dir" is pointing to our "my_hack" folder, which is inside our "example_project" folder, so this way we can refer to our "tools" folder, which is not actually inside our "my_hack" folder, using "../tools"!)
This is what my "config_user.txt" looks like after I edit it, your "dir" will very likely be different, because the path on your local system to your project folder is going to be different:

Let's now switch over to "config_project.txt". There are a few more configuration variables here but most should be pretty self-explanatory again, just change the paths to match our folder structure. I'm going to remove the "title_moves", "test_level" and "test_level_dest" lines for the sake of simplicity. Since we don't have any patches yet, the "patches" list will be empty for now (I removed some comments from the file for brevity, but you can keep them if you want):

Note that the initial_patch variable specifies the path to a .bps patch that will be applied during the build process before anything else. This is so you can apply Lunar Magic hijacks that other patches and tools might rely on before those are applied. There is a full guide on creating such an initial_patch in the readme, I only included one here to make the guide more convenient to follow. Depending on what you need (FastROM, SA-1, compression options, etc.) you may want your initial_patch to be different, please consult the readme for details on this when you start a real project!
If you have a keen eye, you might have noticed that the "gps_path", "pixi_path", "addmusick_path" and "uberasm_path" are now pointing to folders outside our project folder, you can tell this is the case because of the ".." referring to the parent directory as described earlier! This should cause you to wonder if you should be specifying these paths in the "config_project.txt" file, since it's the config file of the project, but we're referring to something outside of our project. Generally, I would highly recommend moving any variables that refer to paths outside your project from the "config_project.txt" file to the "config_user.txt" file. We will later make git ignore the "config_user.txt" file, since we do not want to push configuration files that depend on folders that are not a part of the project (imagine if you were working together with someone and they had their tools installed somewhere else, the paths wouldn't match up with the ones in "config_user.txt" because they match your folder structure but not theirs!).
After I move those variables from "config_project.txt" to "config_user.txt", mine look like this:


That's it for Lunar Helper's configuration, let's switch over to Lunar Monitor's now.
Open the "lunar-monitor-config.txt" file we copied over to our "my_hack" folder earlier, you should see something like this:

There's some obvious similarities between this and Lunar Helper's configuration files, just adjust the paths to match those we had for Lunar Helper (all paths are relative to the folder the config file is in!). I have this afterwards:

The last two variables, "log_level" and "log_path" specify where Lunar Monitor will log what it does to, so you can figure out what's wrong if it's not working and see what it exported when. The way they're set up is fine for our purposes here, if you want more attention-grabbing errors or a different log location you can check out the readme for additional details.
That should be it as far as configuration goes!

Finalizing and verifying our setup
It's finally time, open your Lunar Magic from your "tools" folder. First things first, let's turn of restore points (we don't need those were we're going). Under Options -> Restore Point Options... just uncheck all options, note that I believe this might apply to all Lunar Magic executables, so if you actually want to keep Lunar Magic creating restore points for different hacks that might be kind of a problem (not much I can do about that right this second, I'll think about it though, sorry!).
Now open the "my_hack.smc" ROM inside Lunar Magic. In good ol' level 105, grab something, move it and move it back, just so we can save the level real quick without actually altering it. If saving takes a second and you have a "level 105.mwl" in your "Levels" directory afterwards, Lunar Monitor is working:

Next, open the palette editor, uncheck "Auto-Enable custom palette on edit", click on a color and just click OK (you don't need to alter the color). This will let you save shared palettes using the little save icon in the top right of the palette editor:

Click it, saving should take a second again and you should have a "shared.pal" file in your "Other" folder afterwards!
Now open the map16 editor, move stuff around and return it to its original position (just to be allowed to save, again).
Press the save icon on the map16 editor and you should have an "all.map16" file in your "Other" folder.
Finally, open the overworld editor. Again, just move stuff around and return it and click the save icon in the overworld editor. You should have a "global_data.bps" file in your "Other" folder afterwards.
Before closing Lunar Magic, make sure to export GFX and ExGFX, and then close it for now. Your "Other" folder should now look like this:

You can open "lunar-monitor-log.txt" to see what Lunar Monitor did, if there were any errors you should also see them here and be able to fix them (hopefully!).
Go ahead and open up "LunarHelper.exe" from your "Lunar Helper" folder, you should be greeted by a window containing this friendly menu:

Let's try building our ROM using the "B - Build" option, just press "B" on your keyboard and Lunar Helper will try to build our ROM from the (vanilla) resources we just extracted with Lunar Monitor.
You should see Lunar Helper applying all of the tools we specified and inserting our admittedly currently pretty uninteresting exported data.
If Lunar Helper tells you "ROM patched successfully to 'my_hack.smc'!", everything went well and our setup works!
Git repository setup
Now it's finally time to set up our git repository, exciting!
Open a command prompt (or PowerShell) in your "my_hack" folder. I believe on newer Windows versions you can do so by holding shift and right-clicking in/on the folder and selecting "Open command window here", if that does not work for you here are a bunch of other ways to do it.
In this command prompt, simply type "git init" to create a new git repository in your project folder. If successful, you should get something like this (your folder path may differ, that's normal!):

If you get an error about "git" not being found or something, make sure you have git installed!
If you enter "git status", you'll get a nice little list of what files git is currently aware of:

But what's that, there are ROMs in there! Not good, git by default will try to track all files it can find in the whole project, but we don't want it to track our ROMs ("clean.smc" and "my_hack.smc") as well as any other formats similar to the .smc format that may contain ROM data. This is where .gitignore comes to the rescue! Create a ".gitignore" file in your "my_hack" folder, like this:

In your ".gitignore" file, add the following content:

"*" simply means that the files we want to ignore could have any name, we only care about their types (generally, "*" is a wildcard for any text/path). We're ignoring the Lunar Monitor config file, because it's not really actually part of our project, remember that our Lunar Magic executable is in the "tools" folder and so is Lunar Monitor. Since those aren't a part of our project, neither should the config file. I would always recommend ignoring Lunar Monitor's log file, because it's not something we want to track (after all, what's the point in restoring a log file?). We're also ignoring the user specific configuration for Lunar Helper since it's referring to stuff outside our "my_hack" folder as well.
Do "git status" again and voila, as you can see, no more ROMs, Lunar Monitor config or log files:

Go ahead and add use "git add ." to add all of your changes (in this case, all these new "untracked" files) to the staging area.
You can see all files that are in the staging area if you do "git status" again, it's a very useful command!
Now that they're staged, you can use "git commit" to create a commit for these changes like so:

Do "git status" again and you can see that there are no current changes. We just created the starting point for our project, give yourself a pat on the back!
Pushing our repository
To actually push our repository, I will create a new remote repository on GitHub (you can also use GitLab, Bitbucket, etc. instead of course). An account is required to create repositories, but it's totally free! I would recommend making a private repository (like this). Afterwards, you will be redirected to your new remote repo with instructions on how to push to it. We have an existing local repository, so we are interested in this:

Perform these steps in your command prompt (with your own repo URL of course!):

If you see something like this, it worked. Refresh your remote repository's page and you should see your repository was uploaded:

Working on our hack
That's all fine and good, but we haven't even actually worked on our hack yet. Let's do something about that!
Open up the Lunar Magic from the "tools" folder again and open "my_hack.smc" in it. In level 105, I never liked that first ledge very much, let's get rid of it (and the Koopa):

Save the level and Lunar Monitor will automatically export your changes.
If we do "git status" now, we will see a new change, that's the level file Lunar Monitor exported because we changed it:

We could make more changes and keep saving them and Lunar Monitor would keep exporting them, but let's commit our changes for now (as a rule of thumb, you should make commits whenever it feels logical to you, but I will talk about upsides and downsides of many/few commits later). Do "git add ." and "git commit":

If we want, we can also immediately push our change to the remote repository, so that it's stored there too:

For illustrative purposes, let's try "accidentally" deleting our ROM. Just go ahead delete your "my_hack.smc". Let's now rebuild it with Lunar Helper, just do what we did earlier and use its "B - Build" option. If everything goes well, you should get your ROM back exactly in the state we had it in. Open it up, the ledge is still gone, our changes were saved:

Right now you might be thinking "So what? Lunar Magic could have restored a ROM to this state again too", which is true, so let's kick it up a notch.
Make a copy of the "lunar-monitor-config.txt" and "config_user.txt" files somewhere outside your project folder (not necessary, but typing those up again would be slightly repetitive so we're going to avoid that, even though it would obviously be very easy). Now delete your entire "my_hack" folder (only do this if you actually pushed your repository or it'll be gone for real!).
We will now simply restore our entire project, including our git repository, like it's nothing. Copy your repository's URL from GitHub from the box that pops up when you click on "Code":

Now open a command prompt (or PowerShell) in your "example_project" folder and type "git clone <url>" (replace <url> with the URL you just copied) in my case:

Rename the cloned folder to what it was called before, in this case, since I called the remote repository "example_project" but the folder was called "my_hack", I had to rename it from "example_project" to "my_hack". Open the newly cloned folder and you should see most files we had earlier with the exception of the ROMs and the config files, which we specifically told git to ignore.
Take the config files we backed up earlier and move them back to their original locations (or recreate them, if you feel like it, it's no harder than first setting them up).
Afterwards, we need a clean ROM again, so create a "clean.smc" in "my_hack" again.
Now just run Lunar Helper's "B - Build" option again like earlier and you'll have your entire project, not just your ROM, back in the exact state it was in before we "lost" it, pretty sick, right?
This is the basic idea behind this workflow. Make changes, commit them and push them at some point. Not that hard really, is it?

Advanced workflow concepts

GitHub issues
Many repository hosting sites have a very handy feature called "issues". They're kind of similar to threads in forums, you can use issues to keep track of "issues" in your project (bugs, features you want to implement, etc.). Let's create one for our example project:

We can do a bunch of things with issues, you can assign them to people, you can label them, make them part of a milestone, comment on them, etc., very useful! Note also that every issue is assigned a number, you can see it next to its title, in our case it's #1, since it's our very first. Once the bug/feature/whatever that's associated with an issue is taken care of (i.e. there is a commit or a set of commits that addressed it) an issue is typically closed, a button for closing an issue can be seen on the comment box when you have an issue open.
Let's actually address our issue now, 105 needs to be diamond-dirtified and quick!

That's better, beautiful even, some might say.
Save your changes, "git status" will once again show us that level 105 is modified. Let's commit our changes, but let's learn a really cool feature we can use to close an issue via commit message!
We always used "git commit -m <commit message here>" before, let's use "git commit" alone this time, so git will let us create a commit message in a text editor instead, by default this text editor is vim, which is not very nice to work with (at least imo), so see this for how to make git use common text editors instead, if you want to.
I have it set up to open Visual Studio Code, so after "git add ." and "git commit":

git pops up Visual Studio Code for me:

and I can type my commit message (it's convention to type a subject with at most 50 characters on one line and then a body explaining in more detail below):

Note the "Closes #1" at the end, this is the secret sauce that will close the issue we made on GitHub once we push our changes.
After I close the commit text file in Visual Studio Code, git finishes the commit and adds it to our history.
If we do "git status", we can see that our branch is "ahead of origin/main by 1 commit". Meaning we once again have changes we can push to our repository, let's do a "git push" again to do so:

and now let's check on our issue:

It's closed as expected, pretty cool, no? This workflow of opening issues and closing them like this feels very organized and intuitive for many, I've been really enjoying using it for hacking personally!
Note that you can also use phrases like "close #1", "closed #1", "fixes #1", "fix #1", "resolve #1", "resolves #1" instead, for a full list see here.

Branches and conflicts
As I explained briefly at the start, branches are essentially sequences of commits in their own little separate location. Believe it or not, you've already used a branch, the main branch! You can make a branch for any reason, but usually a branch serves some specific purpose. For example, you may have a branch where you work on a specific feature and only once you're confident it's fully implemented and working you merge it back into your main branch. Likewise, some projects use their main branch only when they release their work to the public, using a development branch in the meantime and only merging it back into the main branch once it's ready to be released. In team projects, sometimes people prefer having their own branch to work on stuff. There are a lot more uses for branches, these are just some common ones I've encountered myself!
There are currently some pitfalls when using branches in our context, but that doesn't necessarily mean you can't use them, you just have to be aware of these pitfalls and know how to deal with them!
Let's create a branch so we can see what can go wrong (and right!). Let's say it's a "rework_level_106" branch, we can just create it with "git branch rework_level_106" and then check it out with "git checkout rework_level_106" or we can also do "git checkout -b rework_level_106", which does both in one command:

Let's also create a new issue for this on GitHub (you can have cool little checklists like this in issues too!):

As you can see, I already got the first step covered:

I'll save the level and commit my changes like usual:

Now let's add the diamond dirt, I'll just create a map16 tile for it and save it like this:

And the sprinkle a bit of it across the level:

And commit my changes again (note that "Other/all.map16" was modified because we added the diamond dirt tile to the map16!):

All the tasks in our little list are done! We could now try to merge this branch back into our main branch with "git checkout main" and then "git merge rework_level_106", but let's take advantage of another GitHub feature: pull requests!
First, we will have to push our branch, you cannot just "git push" on branches you create locally, because they do not yet exist in the remote repository, but luckily if we try to, git will tell us exactly how to do what we wanted to do in the first place:

After we take this advice:

We're even told about how to create a pull request, how nice!
Note that you will only have to do this "--set-upstream" stuff once for a new branch, afterwards you can just do "git push", since the branch has now been set up in the remote repository.
If we go to our GitHub page, it will also suggest that our branch had recent pushes and gives us the option to create a pull request:

But what is a pull request actually? It's really not that special, basically, instead of just merging our changes into the main branch immediately (i.e. copying our commits from the rework_level_106 into it), we first submit a request to do so. This is useful if you want someone else to verify that whatever you were working on actually works before merging it into a branch you want to ensure stays bug-free, for example. You can also set your project up so that only certain people can approve pull requests and nobody can just go ahead and merge stuff on their own, there are a lot of options when it comes controlling what gets merged where!
Let's open such a pull request by clicking on "Compare & pull request". We can write a little comment explaining what's contained in the pull request if we want to, I'll just write "Closes #2" so that when our pull request is approved and merged, it will close the issue we created earlier (just like we did with the commit message earlier!):

Note also the "Able to merge." message. This is a great sign, it means our branches have no conflicts and can be merged without us having to do anything special!
Click on "Create pull request" and you will see a page created for it on GitHub. Here you can request reviews, add labels, etc. In this case, we could simply click "Merge pull request" and have our changes be added to the main branch and it would be totally fine, but I want to show you something instead.
Let's go back and do a "git checkout main", we can totally work on two different branches if we want to! First thing you should do after checking out a new branch is running Lunar Helper's build option, this will make your ROM actually match the state of the project in the branch you newly checked out. If you don't do this, things could get a little wonky [6]! You should also reload the ROM in Lunar Magic if you had it open during the build (just to make it refresh map16 and such).
Now, let's make a change in this branch. Go back to our trusty level 105 and remove the first Yoshi coin, it's kinda hard to reach now after all:

Save the level and commit your changes again, then push them:

Go back to your pull request page on GitHub, you'll see that it can still be merged automatically, even though we made changes in both branches. Nice, right?
Well, let's now create a cool new (totally unique) block:

And add it to level 105:

We were even smart about it and created the new map16 tile one tile to the right of the one in the other branch, I'm sure that will totally make it so there's no conflict (famous last words).
Let's commit and push these changes:

Refresh your pull request page again, uh oh:

Why did this happen? Well, unfortunately "all.map16" is a binary file, the way git handles these is simply that it keeps a full copy of each new version it encounters in the repository so that it can later be restored. Since we made changes to this file in both our branches, git has two conflicting copies that do not share a history, i.e. one is not a descendant of the other, they are on separate paths in our repository's "roadmap" that are about to collide.
For regular text files (and code), resolving conflicts is simple, git shows you which lines differ between the files and tells you to figure out which ones to keep or (re)move. But for binary files, git doesn't know what it is that changed, it doesn't understand the file format!
When this happens we have to really get our hands dirty and handle the merge even more manually than if two text files collided. We essentially have to merge the two different map16 versions by hand. It's "lucky" for me that I documented which map16 tile I changed in our most recent commit, otherwise, in an actual project, I may have to sift through dozens of pages and compare them by eye, not great!
GitHub also will no longer give us a nice button to merge automatically (how could it, there are conflicts!) so we will have to use the command prompt (fine by me).
Let's try to merge our branch, we're already in the main branch so it's just "git merge rework_level_106":

As we expected, git cannot merge our binary files for us, "git status" will give us:

Since we need to resolve this by hand, let's stop our merge for now with "git merge --abort", now let's switch back to our other branch with "git checkout rework_level_106". Remember, first thing after doing a checkout, always re-build and reload your ROM!
Opening our ROM again, we will be greeted by our good old tiny diamond dirt in the map16 editor, we need this fella for our merge, so we need to somehow grab a hold of him. We could try copying it with Ctrl+V, but I'm not sure if that actually works, so I will export the tile in a map16 file (do not export all map16 pages, that is exactly the problem we're addressing!) so I can later import it manually during our merge. I'll just put the file somewhere outside our repository so git won't try to track it.
Now switch back to the main branch with "git checkout main", re-build your ROM and try "git merge rework_level_106" again. Obviously you'll get the same error, after all, we didn't change anything in either branch! But we do now have the tile(s) we need from the branch we're trying to merge and we're currently merging, so let's now import the map16 file we saved of our beautiful diamond dirt:

Look at the two, so cute, what a power couple! Let's save map16 and then "git add ." to stage our resulting "all.map16" file (git tells you to stage conflicting files to mark the conflict as resolved):

Git is giving us the go ahead, "All conflicts fixed but you are still merging. (use "git commit" to conclude merge)", let's do so (your text editor will pop up to let you change the commit message, the default of "Merge branch 'rework_level_106'" is totally fine for our purposes so you can just leave it unchanged):

"git status" will now tell us that our branch is ahead of 'origin/main' by 3 commits, two of those are our commits from the rework_level_106 branch, the third is the merge commit we just finished, let's push:

Look at your pull request page:

Yay, we did it, you can even see our issue was closed (on the right!).
As GitHub tells you, you can now delete the branch if you want, since all the commits are integrated into the main branch and will not be lost.
As you can see, resolving binary file merge conflicts can be a real pain in the ass, which is why you'll generally want to avoid them as much as possible. Given that we're stuck with "all.map16" and its binary format for now, avoiding conflicts like this is relatively difficult if you're using multiple branches or working with other people (the same conflict could have happened if someone had changed "all.map16" while we were working on our own version of it in the same branch and they pushed it to GitHub before us!).
Note also that conflicts like this could happen the same way with shared palettes, levels, global data, even title screen moves, and they would be just as annoying to resolve, because those are also all stored in a binary format. You'd have to use a similar approach like we did, somehow get your hands on the conflicting versions from each branch and merge them by hand (i.e. getting conflicting MWL files and merging them side-by-side in two Lunar Magic windows, opening two OWs side-by-side and merging them, etc.). Sadly, that's currently the state of things, but we can mitigate it somewhat by writing clear and descriptive commit message (i.e. me documenting that I only altered tile 201 in our main branch's map16 commit so I could find that tile manually later, just in case).
That's pretty much it for branches and conflicts. Personally, I stick to using branches for highly self-contained parts of hacks, so that I can avoid conflicts as much as possible. If I can tell there might be a conflict at some point (map16 is often a culprit because it changes often and is not split up at all), I write very detailed commit messages, so I can find the actual tiles I need to merge easily later.

Considerations and common problems

Tools in or outside of repository?
In our example project we kept our tools outside of our repository for simplicity. The reason it is simpler this way is that these tools are very noise and some of them produce a seemingly endless amount of irrelevant (for our purposes) files that will clog up your commit history with completely useless changes if you don't ignore all of them via your .gitignore file.
So if you want to keep your tools in your repository, make sure to build your ROM with Lunar Helper and then take note of all the irrelevant files git spots and tries to track. Make sure to add any that are just build artifacts and not necessary to be kept as backup to your .gitignore or you will waste time and space on them.
Another, but negligible, downside is that your repository will have to contain these tools, taking up a slight amount of space (not a big deal at all, git is good at compression).
The upside of keeping tools in your repo is pretty simple, you can ensure when you check out your project at a certain point, it will be built with exactly the tools and versions of them you were using at that point.
Make sure to follow the general idea I mentioned when configuring Lunar Helper of keeping paths referring to tools that are part of the project in "config_project.txt" and paths to tools that are outside of the project folder in "config_user.txt" to keep your project portable!

Many/few large/small commits?
In our case, in theory, fewer commits with more changes would save space, since git has to store our binary files as full copies instead of just storing the difference between them and the less commits we make with them, the less copies it will have to store. However, we really don't need to worry that much about space, the binary files actually take up surprisingly little space in the repository (thanks git compression!), and being able to restore our work at important points if we need to is much more important, so in my opinion you should be striving to make your commits as small and commit as often as you find sensible without worrying about space too much!

I tried to push my commit, but someone pushed to the same branch in the meantime, what can I do?
This often occurs, you're working with other people and they push changes in the same branch as you before you pushed yours, usually you'd just "git pull" to add their changes to your local repository, but you have work committed already, what now? Luckily there is "git pull --rebase", this command will first pull the changes and then add your commits after them, very convenient! Note that this will only work automatically if they did not change the same files as you did. In that case, you'd have a merge conflict to resolve (good luck!).

Can I make my repository public?
DISCLAIMER: I'm not a lawyer!
If there are no ROMs in there, you might think it would be okay to make your repository public. Maybe it would be, but in theory it could still be illegal, but for different reasons. If you have any sort of resource (text, code, graphics, tools, etc.) in your repository that you did not create (pretty much a 100% chance), you in theory have to respect that resource's license (if it even has one!) or re-distributing it (which is what you're doing by making your repository public) could still be technically illegal! Is anybody going to care? I don't know, I wish nobody did and we could all just have a good time, but unfortunately I cannot guarantee that that's the case. I would thus recommend keeping your repository private and only inviting collaborators to it, if you want to work in a team. Might still be illegal (I do not know), but at least almost nobody will know about it or be able to prove it!

Final thoughts
If you've made it this far, I do not believe you read the whole thing (if you actually did, I'm very impressed). This turned out a lot longer and more detailed than I thought it would, I hope it's actually useful. I did not want to cut too much, since there are a lot of specifics with this system.
One thing I will mention is that git to me is very "learn as you go". I still encounter situations I don't entirely know how to handle sometimes, but luckily so many people use git that you can almost always find advice on your specific situation online, no matter what's going on.
That's pretty much it from me, thanks again for the kind words everyone, I'm glad people actually enjoy this release, thanks again to everyone involved (especially Maddy, of course)! If you think anything is missing from this guide or just plain wrong, please let me know and I'll look into it!
See you around.

[1] Here is a guide on writing "good" commit messages if that is something you're interested in
[2] Historically and problematically also sometimes still called the "master" branch, thankfully most important tools will either use "main" by default now or at least allow you to switch the default name from "master" to "main" easily
[3] Here is a short guide on how to create your own "command" that will let you log changes in a much prettier format
[6] Very advanced concept: see this for "Git Hooks" which let you run programs whenever you perform an action in git, you could theoretically create a post-checkout hook which could run Lunar Helper's build option for you!

Super Mario WorldToolResource Release