PIXI v1.2: Some Sprite Insertion Tool of Sorts - Resource & Tool Releases

I'm not sure what to think about the new format, but if it makes things easier to handle both for devs and people, I'm all in.

Originally posted by Hamtaro126

If you are including support for more sprite types, why not include almost ALL of the sprite types,

Of course "almost" means this excludes "Overworld Sprites" (for now), also the final result requires more external patches to be applied.

I'd love to see Minor Extended Sprites here, but there are more useful sprites types and focusing on them first is a wise move to see if they work correctly on this tool before adding other Sprite types.

Originally posted by Rykon-V73

Does this also mean the inclusion of score sprites?

Yes... This is possible.

Originally posted by LX5

I'd love to see Minor Extended Sprites here, but there are more useful sprites types and focusing on them first is a wise move to see if they work correctly on this tool before adding other Sprite types.

I agree, since priority is key!

A.K.A. SmilyMZX,

Also on NESDEV, and on various other forums, Find me on other places if I'm there.

Look for me also on YT...

Beware of false lookalikes!

Overworld sprites aren't included yet because I have something bigger planned with them and as it turns out will take a lot more time, so I scrapped it for now rather than delaying this any further than needs be.

As for minor extended and score sprites... honestly I could add them and any other sprite type too for that matter, since they are about 99% the same code as cluster and extended.

The reason why I didn't do it yet is because I honestly saw no reason for it. I don't even have any sprites I could use for testing. It would just cluster up the directory with a bunch of empty unused folders.
In terms of priority, I wanna get OW sprites done next, cause they actually are more of a big thing and after that, maybe run-once sprites.

Lastly, Alcaro brought up some good points against binary format (namely people editing the filename with a text editor and endians + string encoding) so I'm gonna stick with the text format... for now.

Anime statistic on MyAnimeList:
400 animes completed ✓
6000 episodes completed ✓
100 Days completed ✓
... what even am I doing with my life?

I have a few Minor Extended Sprites, so I can give them to you at any time. I'm sure that I'm the person who does most Minor Extended Sprites on this site, lel. If you want help with them, tell me.

I'm actually more excited for a tool that has extended sprites support, though.

If you've got some extended sprites, I'd like those too for testing.

Anime statistic on MyAnimeList:
400 animes completed ✓
6000 episodes completed ✓
100 Days completed ✓
... what even am I doing with my life?

I have some, but most of them are Sonikku's.

----------------

I'm working on a hack! Check it out here. Progress: 64/95 levels.

Version 1.1 is finally done. Yay I guess.
Knowing me though, I probably forgot to test and overlooked some basic issues again. Anyway, in addition to bugfixes this is the new stuff:

v1.1

Cluster and Extended Sprites
Now supports the insertion of cluster and extended sprites, done so, via the same list file. Just use the header "CLUSTER:" or "EXTENDED:" to switch modes. (or "SPRITE:" but that's default).
Cluster and extended sprites are capped at 0x80 slots. They also get inserted in addition to their original lists, not like custom sprites with their own lists. Which basically just means that, when inserting a cluster sprite in slot 0, you actually have to spawn number 0x09 (and 0x13 for extended).
I realize this might be slightly stupid, but in the end, it's up to the coder to just add the offsets. Better than leaving it up the the enduser to start with slot 9/13. Also, sa1def.asm provides defines:
Code
```
!ClusterOffset  = $09
!ExtendedOffset = $13
```
Also, the way they are inserted now, as oppose to previous tools, they have to end in an RTL, not RTS! The entry point is also no longer controlled by labels, instead it is tagged just like other sprites with:
Code
```
print "MAIN ",pc
```
Up to 4 Extra Bytes
Controlled via the cfg file and supported by Lunar Magic, you can set up to 4 bytes of custom properties when placing the sprite in the level. Which means you now have 1024 times as many options per sprite as with just the extra bit.
I'd however advice against abusing this too much. For A) sprites still work in 8bit mode and can thus only support 255 bytes of sprite data. Randomly enabling more custom bytes just handicaps yourself. Use it for variable number settings (like how many projectiles a piranha plant shoots).
On the other hand, sprites B0-BF (the per level sprites) always have all 4 custom bytes enabled.
Also, be EXTREMELY CAUTIONS with this feature. Changing the number of extra bytes for an already inserted sprite will corrupt the level data, as Lunar Magic will expect the sprite to be a different size than it is. So make sure to delete all instances of a sprite before changing it's number.
Again, sa1def.asm provides defines to access these bytes within your code:
Code
```
!extra_byte_1
!extra_byte_2
!extra_byte_3
!extra_byte_4
```
New Command-Line Options
In addition to the new options for the cluster and extended directory, I changed some of the option names for using custom directories as command line option, so do lookout for that if you had some scripts setup or something.
Updated the CFG Editor
Other than fixing bugs and implementing the new extra bytes, it can now also function as Tweaker for ROMs. So basically, you can open your ROM with it and use it like the Tweaker tool, except this one has all the bits named :3

In case you actually care about my GitHub repo, I want to clean up the structure a bit more before merging into the master branch.

Plans for the future:
- Add OW sprites using Lui's patch
- Add LM display information. (Given the per level sprites and extra bytes, I doubt SpriteTip is gonna cut it)
- other stuff?

Anime statistic on MyAnimeList:
400 animes completed ✓
6000 episodes completed ✓
100 Days completed ✓
... what even am I doing with my life?

Bug Report:

%ChangeMap16() does not work well with SA-1. Apparently, some addresses were not properly converted (lines 149 and 163, according to GitHub).

Dream team (feed them, please):

I'm finding an error when inserting more than 115 (dec) sprites.

It's just crashing without giving any error (segfault, probably). I've traced it to somewhere in this code in sprite.cpp:

Code

	patch(TEMP_SPR_FILE, rom, TEMP_SPR_FILE);
		
	int print_count = 0;
	const char * const * prints = asar_getprints(&print_count);
	int addr = 0xFFFFFF;
	char buf[5];

Based on the contents of spr_temp.asm and the fact that it doesn't print out the sprite I'm expecting it to go to. Without compiling myself, that's as far as I can trace it, and I can't see anything obvious in there that would break it, but that's what I've got.

If I remove the sprite, the next sprite also causes a crash, and if I remove sprites before the sprite that crashes, the sprite that crashes assembles just fine and the next one crashes.

Also, unrelated, but a request. Would it be possible to un-per levelify sprites? Like, if I added

Code

all:B0 my_sprite.cfg

It would be great if it would just ignore the per-levelness of it. I'm dealing with a code base that uses those sprites, and has no need for per-level sprites.

I tested the new version of PIXI and i found a problem:

When i insert the per level sprites, the normal sprites will disappear in the next screens.

Okay, I think I have a bug report. PIXI doesn't seem to actually insert the sprite size table even if you enable extra bytes for a sprite; I just tried, and the sprite still showed up in the level with no extra bytes.

Edit: Wait, I guess it does insert the size table; it just ignores the extra byte count set by the .cfg file, apparently.

Edit 2: And hex-editing it to have the proper number of extra bytes apparently causes the ROM to crash, even when both the init and main routine just return.

Edit 3: Ignore all of this, because apparently I am derp.

----------------

I'm working on a hack! Check it out here. Progress: 64/95 levels.

Originally posted by HuFlungDu

I'm finding an error when inserting more than 115 (dec) sprites.

Given the code area you pointed at, I'm very tempted to blame this on the asar api, as pixi has no real reason for failing randomly. (neither does asar but I know for a fact it does at times so...)

Originally posted by HuFlungDu

Also, unrelated, but a request. Would it be possible to un-per levelify sprites? Like, if I added

Code

all:B0 my_sprite.cfg

It would be great if it would just ignore the per-levelness of it. I'm dealing with a code base that uses those sprites, and has no need for per-level sprites.

I though about this myself, the thing is, the per-level tables are fairly huge. One bank per quater level space (eg 000-07F). So just nonchalanty filling the entire table with the same sprite doesn't really sit well with me.
I though about looking into a way to disable per-level sprites it just... what kind of insane code/rom do you have that needs more than 176 sprites. If it's just about the slots B0-BF, then I can't imagine fixing your code base being more work than updating pixi's patch.

Anime statistic on MyAnimeList:
400 animes completed ✓
6000 episodes completed ✓
100 Days completed ✓
... what even am I doing with my life?

So, I've been working on implementing the lunar magic custom display data into the CFG files as to generate the .ssc, .mwt and .mw2 files.

This is what the CFG editor looks like atm:

The example sprite being a venus firetrap, upside down if extra bit is set and shooting a variing number of fireballs depending on extra byte 1.
So the custom list in lunar magic would have 6 entries with the plant in normal and upside down state shooting between 0 and 2 fireballs.

This is one of the more fancy (aka big) examples I could come up with.
Now to the problem, the cfg file is growing rather large with all this infomartion cramped in. This is (prototype) what it'd look like for this sprite:

Code

01
36
81 14 08 00 10 20
00 00
piranha_sideways.asm
01:01
C0 10 D0 10 C1 10 D1 10 D0 90 C0 90 D1 90 C1 90
2
0020 0,0,1F4 0,-16,400 -24,0,119
Venus Firetrap shooting a variing number of fireballs depending on the extra property byte 1.
0030 0,0,1B 0,16,401 -24,8,119
Upside down Venus Firetrap shooting a variing number of fireballs depending on the extra property byte 1.
6
0 00 00 00 00 Venus Firetrap
0 01 00 00 00 Venus Firetrap shooting once
0 02 00 00 00 Venus Firetrap shooting twice
1 00 00 00 00 Upside down Venus Firetrap shooting twice
1 01 00 00 00 Upside down Venus Firetrap shooting twice
1 02 00 00 00 Upside down Venus Firetrap shooting twice

Here line 7 is the custom map16 data. Two tiles in this case. I need 8 digits per tile so this line could get really long if you need a lot of custom map16 "blocks".
Line 8 is the number of custom appearances. They always appear in pairs of 2, description and tile arrangment. So here I have 2 (eg 4 lines) because of normal and extra bit appearance.
Line 13 (variing) is the number of lines for the custom collection.

So, any oppinions on this? I don't really know if I can make it any simpler.

mod edit: are you the one stretching the tables?

I wouldn't really consider ~700 bytes "big" in this day and age. Like, even with longer descriptions. Even with, let's say, a 1000 byte cfg file, and with, let's say, 100 custom sprites. That still only makes 100 KB of data. Who cares about 100 KB of data nowadays? It's not even like that data will go into the ROM, it's just for development purposes, so yeah, it's not even really worth investing the time to make it smaller. I mean, I guess you could argue that the more stuff is in there, the less "human readable" it becomes, but cfg files aren't really meant to be human readable, anyways, so that shouldn't matter. After all, that's why the CFG Editor exists in the first place.

If you wanted to make it smaller, one thing you could of course do would be to separate the cfg file from the description (since the CFG really contains "logic" for how the actual sprite will behave, whereas everything else is, again, just for development purposes). Two ideas for this are:
a) Make the cfg reference an external file with the description data.
b) Require the description file to always have the same name and be in the same location as the cfg file. For example: a roto_disc.cfg should have a roto_disc.desc file in the same location. PIXI/the CFG Editor then checks if the file exists. If it does, it generates the description data for it upon inserting the sprite list. If it doesn't, the CFG Editor offers you the ability to directly create the desc file (or simply ignores it upon generating description data).

That's the simplest I can come up with, but as I said, even putting it inside the cfg file should be fine as long as it's optional and doesn't make anything crash when it's missing.

Feel free to visit my website/blog - it's updated rarely, but it looks pretty cool!

hmm...

Because the amount of data is large, I think that it's better to shift to a format like xml...
It can flexibly deal with, and it is serializable, deserializable and convenient.
---
Post footer:
Also known as signature. This will be appended to your posts. HTML is allowed.

@RPG Hacker: I though about a seperate file but I feel like it's just gonna be a mess for both the CFG editor and PIXI if I have to look out for more files than I really want to.
Plus it's not that well manageable with pixi's current code.

@6646: There are a lot of better ways than text format. I also considered pure binary. The main reason I decided against it is because I don't wanna implement multiple different handlers for different cfg-formats.
As PIXI is now, I simple have to add new "line handlers". Which is mostly why the format looks the way I presented it. As in, it's the easiest to implement. Especially since I don't trust my abilities in C++ enough to include an XML parser. :P

Anyway, as it is now, dare I say the format is still managble even by hand. The only thing that's a bit of a mess is the custom map16 stuff but no real way to easen that up. (Ok, I could, but I really don't want to).

After all the lunar magic description lines are ultimately the same as .ssc files. And the custom collection is just the extra bit, followed by 4 extra property bytes and then the name.

Anime statistic on MyAnimeList:
400 animes completed ✓
6000 episodes completed ✓
100 Days completed ✓
... what even am I doing with my life?

Please avoid XML. The specification (https://www.w3.org/TR/REC-xml/) looks complicated and trying to implement a comforming parser therefore equally hard. Furthermore it looks a bit ugly in my opinion.

This week I more or less implemented byuu's BML in two days. Since it is already used in some aspects of emulation, you could use this if you want a more structured markup language.

mod edit: good link. fixed

Your layout has been removed.

Originally posted by JackTheSpades

Originally posted by HuFlungDu

I'm finding an error when inserting more than 115 (dec) sprites.

Given the code area you pointed at, I'm very tempted to blame this on the asar api, as pixi has no real reason for failing randomly. (neither does asar but I know for a fact it does at times so...)

That's what I figured, but I still don't now why. Like I said, I'd have to run it through a debugger to know for sure.

Originally posted by JackTheSpades

Originally posted by HuFlungDu

Also, unrelated, but a request. Would it be possible to un-per levelify sprites? Like, if I added

Code

all:B0 my_sprite.cfg

It would be great if it would just ignore the per-levelness of it. I'm dealing with a code base that uses those sprites, and has no need for per-level sprites.

I though about this myself, the thing is, the per-level tables are fairly huge. One bank per quater level space (eg 000-07F). So just nonchalanty filling the entire table with the same sprite doesn't really sit well with me.
I though about looking into a way to disable per-level sprites it just... what kind of insane code/rom do you have that needs more than 176 sprites. If it's just about the slots B0-BF, then I can't imagine fixing your code base being more work than updating pixi's patch.

It's SMWCP2.

I assume you are already using pointers to the level tables, so my suggestion would be to use a magic number at the beginning of a table that means "any level". Probably FFFF would work fine, since that's not a level. If you implemented it as I think you did, that should be easy, plus it would let you say "this sprite in this level, other sprite in other level, final sprite in all other levels", which could also be cool.

Edit: I've become a big fan of JSON for configuration files. It's a good balance between human readable and machine readable (yaml it is easily human readable, but kinda hard to parse), and it should have all the features you need.

Edit2: This library looks awesome

JSON, YAML, BML. Machine friendly, human friendly, anyone knows what it is. Pick two. (XML fails both machine and human friendliness.)

Writing an YML parser is just about impossible, the spec contains so many options. Sure, you could import a YML parser, but there's a fair chance that'd be an extra DLL, make the build process more complex (iirc one of our tools decided to use Boost, making it a huge pain to compile? AddmusicK, I think? Let's not reinvent that), contain security holes or infinite loops (malware in cfg files, yay, because malware in zsnes roms isn't fun enough), and so on.

JSON is kinda human friendly, but all the quotes and backslashes irritate me, especially the quotes on key names. They also tend to be quite unreadable if not indented properly. YML and BML don't need any of that fluff. (In fact, BML looks a bit like a subset of YML?)

My favorite is BML, but it's your choice.

<blm> zsnes users are the flatearthers of emulation

Looking at an old tweet where byuu says bml is basically a subset of yaml, though not a proper one.

I'm looking for a description of the bml standard, but I can't find one. Is there a published standard anywhere?