(i know this looks long and that's because it is long but most of you are interested only in the middle part, i.e. the third list)
For those of you on Discord, I have hinted this for a while. It is time it finally went public.
In August 2017, the sprite remoderation was started. Let me state this right now: I do not think it was a mistake, but the choice of PIXI, a brand new tool, was definitely rushed. As it stands right now, the section is almost completely converted to PIXI, which turned out to be a very unstable tool. As a result, the community has become quite split over which sprite tool is the best. We should settle this once and for all.
My proposal for the definitive standard sprite tool is boldowa’s GIEPY. Like PIXI, it is new, but I hope that shouldn’t be a problem if everyone thoroughly tests it beforehand.
I will first cover my reasoning behind this whole proposal. If you’re not interested in that, you can skip to the middle of the post which outlines my plan for the standardization and how you can help with it.
Let me show you why I think GIEPY is a good choice by giving you a list of things it does better than SpriteTool and/or PIXI:
- User friendliness. GIEPY has a command line interface (called GIEPY) and an easy-to-use GUI (called Piee). Unlike other tools (GPS with Grilled Pancake Salad, AddmusicK with AMKGUI), the GUI is not an unstable wrapper around the tool. Instead, both GIEPY and Piee use a library called Mewthree which handles sprite insertion.
- Documentation. Unlike PIXI (and, to some extent, SpriteTool), GIEPY has detailed documentation which explains everything you need to know about using the tool, as well as its internals.
- Support. boldowa is very active and responds quickly to issues on GitHub and messages in his Discord guild. Additionally, GIEPY’s code is quite clean, so it would be easy for someone else to manage it. Compare this to SpriteTool and PIXI, whose authors are inactive and whose source code is a mess.
- Extra bits. GIEPY completely removes the concept of the extra bit. In SpriteTool and PIXI, the second extra bit was used to determine if a sprite is standard or custom, while the first extra bit was left free. GIEPY uses both extra bits as a single number called the “group”. Every group has a separate sprite list, so sprite 04 from group 2 and sprite 04 from group 3 can be completely different sprites. This allows it to override vanilla sprites and insert a total of 1024 sprites.
- Shooters and generators. In SpriteTool and PIXI, a custom sprite’s type is determined based on its number. As we all know, C0-CF are shooters and D0-DF are generators. This meant you could have no more than 16 shooters and 16 generators in one ROM, which greatly limited their usefulness (so much, in fact, that most sprite remoderators decided it was better to completely drop support for generators). With GIEPY, the type of the sprite is specified in the CFG file, so you can have as many shooters and generators as you want.
- Initializers and scrollers. GIEPY supports two new sprite types, initializer sprites (you might have heard them called run-once sprites) and scroller sprites (autoscroll generators).
- Extra bytes. GIEPY handles extra bytes a lot better than PIXI did. The most important detail is that changing the number of extra bytes an inserted sprite has will no longer corrupt level sprite data. Additionally, generators, shooters, initializers, and scrollers all support extra bytes (up to 251 for scrollers).
- Shared subroutines. GIEPY inserts shared subroutines (libraries) in a completely different way from GPS and PIXI. This makes them more stable and fixes the sublabel bug.
- Assemblers. GIEPY supports inserting sprites with both Asar and TRASM. Additionally, it automatically detects a sprite’s intended assembler. In spite of this, I don’t think it’s wise to allow TRASM sprites into the section once again.
- Compatibility. GIEPY is almost fully backward compatible with lists and sprites from both SpriteTool and PIXI.
- Stability. GIEPY has strict assertion on every change it makes, so its system patch will refuse to assemble if it will break something. It also has the option of being completely uninstalled from a ROM.
Of course, there’s no way I won’t list disadvantages of GIEPY. Most of these can be worked around, however, and my plan for it can be found after this list.
- SA-1. GIEPY has the same problem which PIXI does on SA-1: having more than 84 sprites in a level will crash the game, even though the SA-1 pack can supposedly handle up to 256. I’m currently debugging the issue and will hopefully manage to fix it soon.
- Conversion. Unlike PIXI (which will attempt to remove SpriteTool when used), GIEPY will not remove any sprite tools previously used on a ROM. Instead it, it will simply refuse to install its system patch.
- CFG editor. SpriteTool and PIXI both have their own CFG editor. GIEPY, on the hand, doesn’t. Additionally, its CFG format is different, so you have to edit CFG/JSON files manually.
- Extra bits. As you may have guessed from the above list, GIEPY’s new group system breaks the concept every hacker is familiar with: the difference between a “clear” and a “set” extra bit. Naturally, this complicates sprites with different behavior based on the extra bit.
- Spawning. Because of the group system, the procedure for spawning a new sprite is different. It involves swapping two instructions but does break every single resource which spawns sprites, including other sprites, blocks, UberASM code, etc.
- Overworld sprites. GIEPY does not support inserting overworld sprites, but neither does PIXI. I’m still uncertain about boldowa’s plans for this, but there is no guarantee they will be added.
Now for the part which most of you will be interested in. This is my detailed plan for what should be done during the standardization, how I can deal with it, and how you can help with it. The first list documents what I think should be standardized. I ask every person to read this list and discuss the items on it in this thread.
- Testing. GIEPY is a relatively new tool. I’ve already tested quite a bit with it, but I encourage people to test it themselves. I am not going to start any work on the standardization until everyone who tests it declares it bug-free.
- Sprite configuration. GIEPY supports using both CFG and JSON files for sprites (YAML and other formats are also planned). Unlike PIXI (where only JSON supports specifying Lunar Magic display data), all supported formats can specify the same information, just laid out in a different format. To avoid confusion, we should standardize a format and convert all sprites to it (this can be done automatically with a script).
- Extra bits. We have to decide on a standard for these. Up till now, sprites could easily have different behavior based on the first extra bit. My proposal is as follows: Convert all sprites which use the first extra bit and let them use either their group number or their extra bytes. If you insert the sprite with no extra bytes defined, it will use its separate behavior if it’s in group (extra bits field in Lunar Magic) 1 or 3. If you insert the sprite with at least one extra byte, it will instead use its first (or, if that’s in use, the next available) extra byte to determine its behavior. With this system, your old levels will still be compatible (because your sprites would use their group number) and usage of the “extra bit” will slowly fade away over time.
- Sprite spawning. Unlike SpriteTool and PIXI, the sprite behind a number also depends on the group (extra bits field in Lunar Magic). If a sprite needs to be spawned, the group number should also be specified. My proposal is as follows: If a sprite uses defines in its ASM file to determine the sprite number to spawn, add another define for the group. If it uses its extra bytes, squeeze the group in them. If it uses its extra property bytes, use GIEPY’s new “unique number” CFG file (it is an 8-bit number which is loaded into A when a sprite is called). A lot of this could be automatically done with a script. In the case of GPS and other tools, modify the routines (and hosted resources) to accommodate the group.
- Routines. GIEPY has a new library specification. I have two proposals here. One, we could ship GIEPY with PIXI’s routines (carefully tested and fixed) and require users to insert it with PIXI compatibility mode. Two, we could standardize a set of native GIEPY libraries (with
GetDrawInfo
,SubOffScreen
, etc.) and convert every hosted sprite to use them instead (this can be automatically done with a script). I would rather implement the second proposal, but the first one requires less manual work. - Generators. GIEPY makes generators useful once again by dropping the hard limit. While they still are limited when compared to UberASM, they are easier to use. I propose to bring back the generators which were firebombed during the remoderation but also host them as UberASM code.
The items below require, at very least, ASM knowledge. The tools require higher-level programming languages. Those who want to help should reach out to me on Discord, preferably by DM-ing me (though you could also ping me in any channel). My tag is Telinc1#2670.
- CFG editor. This is the first big problem. The creation of a new CFG editor is essential. Being the perfectionist I am, it would also be nice to make it an all-in-one tool which can also edit vanilla sprites (a la Tweaker). I am willing to create this editor myself but the best language I could program it with is Java since I don’t know any language which would be potentially more suitable. While this does mean it will be cross-platform, running it will require you to have the JRE installed (aside from all of Java’s quirks like a long start-up time). If anyone else is willing to create this editor in a language which is more suitable for the job (like C# or Qt with C++), please do say.
- CFG file converter. GIEPY works with SpriteTool and PIXI configuration files, but it throws warnings for them. Additionally, generators and shooters need to be converted in order to be properly inserted. A good solution for this is a tool which can convert a whole directory of CFG/JSON files into the format we decide to standardize. If I build the CFG editor, I will make this a built-in feature of it.
- Sprite tool remover. GIEPY does not (and will never) remove old sprite tools. The usability of PIXI on old ROMs was a big problem with the sprite remoderation, but it’s an even bigger problem now. I do not want to require anyone to port their hacks just because the staff team is pushing another standard, which is why I want a tool which can remove any sprite tool (obviously SpriteTool and GIEPY, but also exotic cases like Tessera, GEMS, Daiyousei, Cluster SpriteTool, manual insertion patches, and others). This ion cannon could very well be an Asar patch, but I’m not sure if SpriteTool can be removed through Asar’s limited looping functionality. I haven’t looked into this yet, but I might need external help.
- Sprite remoderators. The standardization of GIEPY will require us to do a lot of manual work in bulk. I would like every current sprite remoderator to say if they would be interested in helping with this. Additionally, I’m looking for new helpers to take care of the GIEPY situation.
That should be about it. The project will start when we have decided what to make standard. If anyone is opposed to this, please state your reasoning. Any other comments or additions to the plan are welcome, too.