Asar: Under new management - Resource & Tool Releases

Originally posted by WhiteYoshiEgg

And while I'm at it, @Alcaro: you once mentioned you had discontinued Asar and dislike the way you coded it back then. Just out of curiosity, would you mind explaining what you'd do differently today?

As RPG Hacker said, it's a fucking mess.

To make a good program, you need to either know how the finished product should look before starting (and act on that, rather than taking a dozen 'good enough for now' shortcuts), or be willing to rewrite half of it. Naturally, I did neither, so it became a wall of crap.

A good architecture for this thing would be multiple passes, where each pass simplifies the internal representation. Replacing freecode with org, evaluating defines/if/incsrc/etc, replacing sublabels with non-namespaced ones, etc. (Not in that order, of course.)

Instead, Asar parses the entire thing thrice, with each pass gathering only the minimal information needed by the next pass; the first pass finds label freespace IDs and approximate freespace sizes, the second pass assigns freespaces and finds label locations, and only the third pass is kept.

Naturally, I've had about 999999999999999999999 different bugs where the passes don't assemble things the same way. For example:
- if read3(random address that's also written by the patch)
- labels are treated as 0 in the first pass; 'if label' will take different branches between passes
- not resetting defines between passes, so ?= takes the other branch
- letting pass-1 labels leak, like this guy
- to ensure errors aren't ignored nor repeated, they all know which pass they should be in, and are silent in other passes - but still set the 'errors happened, not writing rom' flag. Guess what happened when I divided by a label? (No, I don't remember what I was doing when I found that.)
- that's not the only phantom error I've fixed, I've had so many I had to give them a name
- check my changelogs in this thread if you need more

(In my defense, xkas did it exactly the same way, I just copied its architecture. Along with about forty other braindead things. Like the macro/define split - they're both text replacement, they should be replaced with something that can represent both features. Instead, I created a third kind!)

Originally posted by RPG Hacker

I'll take a stab it at later and then probably bump the version to 1.41.

Perhaps jump straight to 1.42, so we don't get two tools (Sprite Tool) at 1.41. It could be slightly confusing.

<blm> zsnes users are the flatearthers of emulation

In the end, it does work, though, so there's that. You know, own of my many problems in programming is that I often think so much about a good structure from the beginning that I get overwhelmed and don't even get started for a while. I respect people who can just sit down and get stuff done, even if the end results sometimes turn out a bit messy. Especially in short terms, a quick and dirty solution is sometimes better than slow and structured solution that never gets done.

Originally posted by Alcaro

Perhaps jump straight to 1.42, so we don't get two tools (Sprite Tool) at 1.41. It could be slightly confusing.

Oh yeah, I see what you mean. Might be a good idea.

Feel free to visit my website/blog - it's updated rarely, but it looks pretty cool!

pls add a way to incsrc/incbin (the contents of) a folder

something like

org $008000
incsrc junk
warnpc $00FFC0

and in junk we have:
init.asm
nmi.asm
main.asm
lewd.asm
something.bin (maybe check file extension so its not parsed as code)

optionally auto add labels (so to access something.bin we do LDA.l something,x)

also replace the viking helmet icon with a loli muncher. i can help with this if needed

Originally posted by Ladida

pls add a way to incsrc/incbin (the contents of) a folder

something like

org $008000
incsrc junk
warnpc $00FFC0

and in junk we have:
init.asm
nmi.asm
main.asm
lewd.asm
something.bin (maybe check file extension so its not parsed as code)

I guess that should technically be doable, although it could require some work depending on how good the C interfaces for directory parsing are and on how easy it is to restructure incsrc and incbin in Asar so that they run in a loop. Though if I did something like that, I'd probably want it to support some kind of regex-like (or at least Ruby-like) functionality with support for wildcards, so you could so something like

incsrc junk/*.asm

to incsrc all .asm files in junk/ or

incsrc junk/**/*.asm

to include all .asm files in junk/ or any subfolder (to any depth) of junk/

This would be even more work, though, and might currently be out of scope for me.

Do you have a lot of .asm files that need to be included so that it would be worth the effort?

Originally posted by Ladida

optionally auto add labels (so to access something.bin we do LDA.l something,x)

That should be easy to do, I think, but I'm not sure if I should, since that could break compatibility with existing patches if they happen to already have labels that share their name with some file that is incbin'd.

Originally posted by Ladida

also replace the viking helmet icon with a loli muncher. i can help with this if needed

I'll consider it!

Feel free to visit my website/blog - it's updated rarely, but it looks pretty cool!

Originally posted by RPG Hacker

In the end, it does work, though, so there's that. [...] Especially in short terms, a quick and dirty solution is sometimes better than slow and structured solution that never gets done.

The problem with releasing something half-working is I get bug reports for years after dropping the project. The last one was about two months ago.

Quote

how good the C interfaces for directory parsing are

The C and C++ standards don't know what a directory is, so you'll need either an ifdef or some cross-platform wrapper.

Quote

how easy it is to restructure incsrc and incbin in Asar so that they run in a loop

Probably pretty easy, incsrc is recursive.

But Asar is a bigger mess than Kim Jong Un's mind, so there's probably a trap lurking somewhere...

Quote

incsrc junk/*.asm
incsrc junk/**/*.asm

Remember that you'll have to deal with weirder cases like
incsrc junk/mario*.asm
incsrc ../**/*.asm
incsrc junk/**/mario/**/luigi.asm
incsrc c:/**/*.asm ; have fun parsing the entire filesystem
incsrc junk/**/../candy.asm ; what does this even mean?
incsrc junk/m**o/luigi.asm ; or this?

It's fine to reject some of them, but you have to do something.

Quote

also replace the viking helmet icon with a loli muncher. i can help with this if needed

As much as Munchers are a good idea, changing the icon would confuse people about whether it's the same tool. I must insist that either the icon stays, or the name is changed as well.

<blm> zsnes users are the flatearthers of emulation

You're right, that's a lot of hurdles to watch out for. Kinda makes me wish I could just let Ruby handle everything, but oh well. It should somehow be doable nevertheless (assuming I actually find the time and motivation).

Feel free to visit my website/blog - it's updated rarely, but it looks pretty cool!

Originally posted by Alcaro

As much as Munchers are a good idea, changing the icon would confuse people about whether it's the same tool. I must insist that either the icon stays, or the name is changed as well.

If anyone ends up making a new assembler, I reckon a good name would be "munch".

Or, if it's by you, perhaps "piranha" (patching improved radically: alcaro's new hacking assembler).

Honestly, if I ever created a new assembler from scratch, it would probably be called "Eduard LAsar".

Feel free to visit my website/blog - it's updated rarely, but it looks pretty cool!

Originally posted by WhiteYoshiEgg

Originally posted by Alcaro

As much as Munchers are a good idea, changing the icon would confuse people about whether it's the same tool. I must insist that either the icon stays, or the name is changed as well.

If anyone ends up making a new assembler, I reckon a good name would be "munch".

Or, if it's by you, perhaps "piranha" (patching improved radically: alcaro's new hacking assembler).

The only good name is Quasar.

Or something ridiculous: 14851, ughhhhhhhhhh, ☢, tool, etc. It wouldn't be the first nonsense name.

<blm> zsnes users are the flatearthers of emulation

Random proposal, think it would be possible to expand the library functions to include something that returns a list/array or inserted blocks? Like, start address and number of bytes?

Random example:

Code

org $123456
	LDA #$01
	STA $19

freecode
	db "Hi there"
	RTL

Resulting in:

Code

codeblock[0].address = 0x123456
codeblock[0].size = 4
codeblock[1].adddress = 0x128000
codeblock[1].size = 13

Anime statistic on MyAnimeList:
400 animes completed ✓
6000 episodes completed ✓
100 Days completed ✓
... what even am I doing with my life?

Not sure. Depends on whether Asar keeps track of all of that information already. I guess in the case of freecode/freedata, it might, not sure about general org. Not sure how complicated it would be to build into Asar otherwise. Maybe it's as simple as "take the difference between the current address and the last address", but maybe it's also more complicated than that.

Do you need this information to do actual math with it, or just to pass it to the print command? If it's just the latter, I think it's fairly easy to simulate by just using macros.

Feel free to visit my website/blog - it's updated rarely, but it looks pretty cool!

He wants it in the library API, as in the DLL. There's about five million circular dependencies going on if you try to let the patch read it.

It's not a bad idea, but you have to determine the order if pushpc and incbin-> are nearby. And in the case of org $008004 : nop : skip -5 : nop, where's the start and end?

(no, I don't know why that's allowed at all)

<blm> zsnes users are the flatearthers of emulation

I dunno how exactly asar writes to the rom, like if it's a stream that auto increases the position and stuff like skip and org just seek relative or absolute, but otherwise, you could just track every time you write to the rom and get a list of addresses like that, which can then be grouped. Not the best way to do it but again, I've no idea how asar is even handling this.

Anime statistic on MyAnimeList:
400 animes completed ✓
6000 episodes completed ✓
100 Days completed ✓
... what even am I doing with my life?

Oh, you're right, Alcaro, I missed the library part.

Yeah, that sounds like it would be even more complicated than allowing wildcards in incsrc/incbin. It's probably out of scope as far as my current plans with Asar go, although I agree that it would be useful. An easy but unrealiable way of doing this would probably be to do a simple byte compare between "original ROM data" and "new ROM data". Unrealiable, because it wouldn't detect if a byte was overwritten, but remained unchanged. Alternatively, I guess I could search the source code for all places where "romdata" is written to and then modify the source code here to call some function instead which then keeps track of all the changes done to romdata. Effectively, this would probably accomplish what you need (although this wouldn't tell you which line exactly caused the ROM data to change, but I'm not sure if that is even needed in the first place?). I would have to implement a rock solid solution for this, though, or otherwise it could greatly influence Asar's performance in compilation.

Ninja'd by Jack. Yeah, we basically had a similar idea.

EDIT:
One thing that could be tough about this is getting a good format to save the address of a change in. It would be fairly simple if just saving a PC offset, but if you want an SNES address, this would cause some trouble, since the mapping mode can change at runtime, I think, making it hard to reliably group data blocks together.

Feel free to visit my website/blog - it's updated rarely, but it looks pretty cool!

Originally posted by RPG Hacker

An easy but unrealiable way of doing this would probably be to do a simple byte compare between "original ROM data" and "new ROM data"

This is what I've been doing so far, and because it's unreliable, I figured I'd ask you xD

Originally posted by RPG Hacker

One thing that could be tough about this is getting a good format to save the address of a change in. It would be fairly simple if just saving a PC offset, but if you want an SNES address, this would cause some trouble, since the mapping mode can change at runtime, I think, making it hard to reliably group data blocks together.

While pc addresses would also work, how many patches change their mapping mid runtime?

Anime statistic on MyAnimeList:
400 animes completed ✓
6000 episodes completed ✓
100 Days completed ✓
... what even am I doing with my life?

I'm not that experienced with the SNES, to be honest, 99% of my SNES experience comes from SMW hacking, so I'm not sure which things can affect the mapping mode. Is it only lorom/hirom/sa1rom or can other things (like switching architectures) affect the mapping mode as well? If it's only those commands, then I guess it is reasonable to assume the mapping mode won't just switch out of nowhere, or can switching the mode in the middle of a ROM make any sense? Like, is there a scenario where that could ever be useful? Otherwise I could just make Asar throw an error in that case to begin with. Since I'm not too familiar, I really just don't know if switching mapping modes is a valid thing to do mid-ROM.

Also should I group these code blocks by banks? Like, if some code or data goes from $01E000 to $01FFFF and some other code or data goes from $020000 to $021FFF, should those be two separate code blocks or just a single one? In the case of two code blocks, can I assume that a SNES address is always 24-bit and that the high byte always denotes the bank, no matter the current mapping mode (e.g. is there no mapping mode that uses, let's say, 32-bit or 16-bit addressing)? If all those things are given, I guess it shouldn't be too problematic to group the blocks by SNES address and also by bank.

This also reminds me that I didn't update the DLL at all when uploading Asar 1.40, so I should do that for 1.42.

So the current list of planned features for 1.42:

Add structs from p4plus2's branch
Generate a list of data blocks modified by Asar
Functionality to incsrc/incbin multiple files at once (very maybe, only if I can find a satisfying solution)
Actually build the DLL this time (duh!)

Anything else?

Feel free to visit my website/blog - it's updated rarely, but it looks pretty cool!

Mapping mode just tells us how asar will translate a SNES address to a PC address (and where to look for freespace I guess). Either way, the snes address will always be 24 bit.
Also, there is also norom, exhirom and exlorom, but the later two aren't in asar anyway.
Architecture, from my understanding, is just how code is interpreted and what syntax it uses.

As for grouping beyond banks, I'd say leave it as two. In the end, any enduser can still split or merge them themselves, but merging is a lot easier than splitting.

Lastly, since this came up a couple of times now, maybe also add an api for telling the mapping mode? Just an int with 0=norom, 1=lorom, 2=hirom, 3=sa1rom or something.

Lastly²
Not really all that important, but asar has a bug where it treats forward jumps as +1 byte when calculating the RATS size. Not really that important, but I figured I'd point it out anyway.

Anime statistic on MyAnimeList:
400 animes completed ✓
6000 episodes completed ✓
100 Days completed ✓
... what even am I doing with my life?

Originally posted by JackTheSpades

Mapping mode just tells us how asar will translate a SNES address to a PC address (and where to look for freespace I guess). Either way, the snes address will always be 24 bit.
Also, there is also norom, exhirom and exlorom, but the later two aren't in asar anyway.
Architecture, from my understanding, is just how code is interpreted and what syntax it uses.

So basically, what I'm getting from this is that changing the mapping mode multiple times or mid-ROM doesn't make much sense and can safely be treated as an error, right?

Originally posted by JackTheSpades

Lastly, since this came up a couple of times now, maybe also add an api for telling the mapping mode? Just an int with 0=norom, 1=lorom, 2=hirom, 3=sa1rom or something.

I guess that shouldn't be too hard, Asar most likely already stores that somewhere, anyways.

Originally posted by JackTheSpades

Lastly²
Not really all that important, but asar has a bug where it treats forward jumps as +1 byte when calculating the RATS size. Not really that important, but I figured I'd point it out anyway.

I'm not quite sure if I'm understanding the bug correctly. Can you give an example of some code that causes the bug, together with the result (and the expected result)? That would help.

Feel free to visit my website/blog - it's updated rarely, but it looks pretty cool!

Originally posted by RPG Hacker

So basically, what I'm getting from this is that changing the mapping mode multiple times or mid-ROM doesn't make much sense and can safely be treated as an error, right?

I'd think so. Only concern I have is that asar by default assumes stuff to be in lorom, and sa1rom has the same "low" mapping as lorom.
But even then, I guess you could treat it as an error if one changes the mapping AFTER actually writing code (not stuff like read1 which we use to check for sa-1 roms)

Originally posted by RPG Hacker

I'm not quite sure if I'm understanding the bug correctly. Can you give an example of some code that causes the bug, together with the result (and the expected result)? That would help.

Been a while since I've encountered it, but I think it was something like this:

Code

org $xxxxxx
	autoclean JML free

freecode
	LDA $00		;2 byte
	BEQ +		;2 byte
	JML ret1	;4 byte
+	JML ret2	;4 byte

Usually, you'd expect the freecode to be inserted with a RATS tag protecting 12 byte, but it seems that it counts the positive (forwards) branch as a byte too, so it actually ends up protecting 1 additional byte per branch. You can kinda see this if you open a rom with multiple patches or blocks installed and look for "STAR" that most protected codeblocks end with a bunch of zeros more before the next "STAR" (RATS tag) begins.

Anime statistic on MyAnimeList:
400 animes completed ✓
6000 episodes completed ✓
100 Days completed ✓
... what even am I doing with my life?

Understood. I'll go looking for the bug. Seems a bit tricky, though, so I probably won't invest too much time into it and only fix it if it ends up being something rather simple or obvious.

Feel free to visit my website/blog - it's updated rarely, but it looks pretty cool!