Banner
Views: 924,485,303
Time:
14 users online: 4Fs, ageVerrly, ASSATAKKU,  BeeKaay, HammerBrother, Hayashi Neru, Ice Man, KevinM, ocked, Oviludo, Paperdomo101, SJandCharlieTheCat, Triple P, yoshi9429 - Guests: 64 - Bots: 112 Users: 51,700 (2,052 active)
Latest: ZeroToonsX
Tip: Don't just modify the original SMW overworld; start from scratch.
Not logged in.
AddMusicKFF Summer 2021 C3 Update Thread + AddmusicK 1.0.9 Release Candidate
Forum Index - Sunken Ghost Ship - C3 Museum - Summer 2021 - AddMusicKFF Summer 2021 C3 Update Thread + AddmusicK 1.0.9 Release Candidate
Tags:
Pages: « 1 » Link
Hello! Welcome to my Summer C3 progress update thread for AddmusicKFF!

The main thread is here. You are looking at an abbreviated version that summarizes some of the material.

What is AddMusicKFF?


AddMusicKFF is a fork of AddMusicK by Kipernal. I'm not bumping the version number unlike the others due to me having seen two past cases of doing so, and me admittedly realizing that I don't want to join the conflict myself. Plus, I'm intending on maintaining backward compatibility with the older Addmusics.
This actually originated when I created a slowdown fix for AddMusicK for Super Mario All-Stars + Super Mario World that got officially implemented in the standalone version on Super Mario World, debuting on 1.0.7. I also implemented fades to any volume for compatibility for Super Mario All-Stars and the original ID is backward compatible with Super Mario World (that hasn't been ported over yet partially because on the 65816 end, I introduce a conflict past 128 songs, and partially because that code can be improved somewhat by utilizing unused memory locations).

Now... you may have noticed that I have added something about an AddMusicK 1.0.9 release candidate. That's because I feel the master branch is finally ready to be merged into the real deal, which in turn means that I finally feel comfortable about preparing to bump the version number up. However, I'm not bumping it to 1.1.0 or 2.0. For 1.1.0, it's because there's an extant beta without C++-side source by Codec and 6646 on the forums. Although I'm not completely interested in all of the things in this beta, I have decided that this makes a good benchmark of at least some of the things that could be in future versions. The other one is 2.0 by StackDino, which doesn't have a public version (and Kipernal himself was planning out a 2.0 version prior to his disappearance from the internet sometime in early 2015 by my own estimate).

This version adds code containing either direct contributions or adaptations from  Atari2.0, KevinM, Akaginite and HertzDevil.

What have you done since Winter C3 2021?


Ready for 1.0.9


  • A bunch of bugfixes, mostly to the sound driver itself, though the C++-side also got a few fixes here and there. I consider the C++-side bugfixes to be far more critical than the sound driver bugfixes. That means...
    • The most critical bugfix that I implemented in my opinion has to do with getting compiled SPCs to better represent the compiled ROMs with their sample selections: I factored in user expectations based mainly on the compiled ROM results to perform this particular repair.
    • Another big bug, also on the C++ side, deals with .BNK files causing errors when mixed in with BRR files from another song when sample identification is occurring by matching by path. This bug is a deal breaker with using Addmusic405 songs raw in a compiled ROM, and I accidentally ran into this in the process of fixing the above bug when it was reported to me after I accidentally created a regression.

  • All C++-side code can now be compiled with a GCC compiler rather than requiring Visual Studio. Initially I only did the main conversion utensil, but I extended this to AM405Remover, AMMBatch and AM4Batch after getting a compiling error with AM405Remover (which gets embedded into AddmusicK) after testing it again and deciding that enough was enough. All of these were adapted from HertzDevil's fork of AddmusicK, just like how the makefile was adapted from the same place.
  • SFX priority is in! Now you can replicate vanilla SMW's priority system if you like, or modify the raw data and shuffle things around!
  • P-Switch SFX has been enhanced and made functional! The SFX can now be completely stopped on the fly, removing the uninterruptibility of one of the channels.
  • I made an .asm file called UserDefines.asm and moved all of the user-definable defines there. This allows both the SNES side and SPC side to share defines, and makes things a lot easier from a code filtering standpoint if some 65816 code becomes redundant because the SPC700 code is also gone.
  • I've made an ARAM map spanning from Vanilla SMW to today, thanks to the contributions of Masterlink and SimFan96 for giving me some archival Addmusic versions so that I can document the utilization of the memory locations accordingly.
  • If you want the list of changes between 1.0.8 and 1.0.9, I compiled the entire list right in the changelog, in addition to copying over 1.0.7-1.0.8's, because they weren't there before.

Bleeding Edge exclusive for now


  • A few new VCMDs here and there, though I haven't been focusing on these as much. These are bleeding-edge exclusive for now until I implement (manual) code filtering to avoid bloated filesizes on the sound driver.
  • A new remote code event type! It's mostly identical to -1, but I have it ignore arpeggio-triggered notes. This is bleeding-edge exclusive for now until I implement (manual) code filtering to avoid bloated filesizes on the sound driver.
  • AddmusicM's Special Pulse Wave is back!!! Probably the biggest missing feature I can think of that didn't make its way over to AddmusicK. This is bleeding-edge exclusive for now until I implement (manual) code filtering to avoid bloated filesizes on the sound driver.


Anything to merge that might require some remoderation?


So far I haven't done so within the context of the master branch, but I have four of them. All four are not too major, and I have recorded the method to detect whether the bug has been exploited for all four.
  • Have readahead look inside subroutines and loop sections. This ensure that long chains of subroutines don't cause a long series of bytes being read before finally finding a note or tie command that is well past what was expected to be read, and may or may not key off the note as intended.
    For detection of utilization for this bug...
    The check is split into two parts. You're looking for mismatches between notes, ties and rests at two different points.
    For labels, loops and recalls, you have to jump over every single one to emulate the original readahead behavior. That is the first point. The second point is the start of the loop. The end marker for the loop section will count towards a mismatch if either remote code type 3 events are turned on or if there is a tie past the loop end marker. The counter should also be read: if it only happens once, then you don't need to go back to the beginning and check that out.
    For superloops, you merely have to look at the start and end points of each section, as well as just beyond the end marker.
  • Having an arpeggio play while a rest command is supposed to be active felt a bit unnatural to me, so I added in some code so that there can be a gap. Interestingly, this was technically noted in the commentary. The arpeggio will also start right back up when you play the next note.
    For detection of utilization for this bug...
    Check for $FB xx usage per channel, then check for either a $C7-$CF or a r command per channel while the arpeggio is running.
  • Pitch slide to note commands (referring to the $DD command) failed to account for per-channel transposition.
    For detection of utilization for this bug...
    First, you need to make a list of all channels that use $FA $02 xx. Recalled labels that use this VCMD will also count for the channel.
    Do not look for h: that doesn't generate a $FA $02 xx VCMD.
    Then take a look at hits for the $DD xx yy zz VCMD and/or the & command on a per-channel basis.
    If both $FA $02 xx and $DD xx yy zz are used in a channel, then we need to go into finer detail. Go down each channel that uses it, and track the transposition, including on looparound if it doesn't reset itself immediately. When $DD xx yy zz is hit, check the transposition at its current state. If it is non-zero, then the bug exploit is confirmed.
  • Write GAIN before ADSR when updating instrument data. The ADSR/GAIN commands (referring to VCMDs $ED and $FA $01) have an unexpected side effect due to the ordering of the writes of DSP registers. This mainly involves having the previous gain value being a fixed volume set value, but there are other side effects.
    For detection of utilization for this bug...
    This one requires tracking DSP register writes through a specific method. The bug is triggered when the instrument setup code is invoked and GAIN mode is being activated from ADSR.
    Thus, you'll be looking for cases when the GAIN register was previously set to a non-stop volume envelope value (not $00-$80, $A0, $C0 or $E0) and ADSR is not set to gain mode (ADSR1 is $80 or greater). This setup can happen with...
    • $DA xx
    • $ED xx yy
    • $F6 xx yy (only with xx being $x5 or $x7, where x is the voice ID)

    The bug is triggered either by $FA $01 xx or $ED $80 xx when the preceding conditions are met.


What has been done since Summer C3 2021 started?


  • NEW 7/4/21: Fixed a bug with the $F3 command not zeroing out the pitch base fractional multiplier, which in turn caused problems for songs ported over from AddmusicM not accounting for pitch base fractional multipliers being in play. (Thanks Masterlink for reporting this bug!)
  • NEW 7/5/21: Fixed a regression caused by the noise SFX frequency conflict resolution code (specifically caused by the music noise frequency being restored) where the FLG DSP register was being overwritten by ModifyNoise, causing the mute to fail.
  • NEW 7/5/21: The $FF SFX VCMD no longer rewinds in order to fetch note data. Instead, it now does exactly what it is supposed to do: execute itself endlessly every time the note length elapses using the parameters that were last used (note length, pitch and volume for example). By removing fetching any note data from prior, we remove ambiguity done by trying to scan backwards for a valid note.


What haven't you done yet since Winter C3 2021?


  • Accounting for pitch bends when extending the pitch range for notes. The only thing I did was in the pitch code itself: the rest of it requires that I check what's going on.
  • Music/song groups, as requested by  imamelia. I'm focusing mostly on getting the sound driver itself in tip-top shape and getting the C++ side in its current state all nice and touched up, and this is currently in the conceptual stage on implementation (with the details having been recorded). Thankfully I've done real SNES game analysis on this kind of stuff, and I have a few examples that I could theoretically refer to for a start... that being Sculptured Software (not the earliest ones, though) and Opus/DBOOT... though ultimately I'm thinking of something a little more custom.
  • Phrase support. Similar case to music/song groups, including having details recorded for MML implementation.


What are your future plans?


  • Enhance backwards compatibility with Addmusic405. I unintentionally restored the original functionality of most of these VCMDs by simply adding them on in the first place without thinking twice, then realizing that they had previously existed. For some fine print...
    • ARAM writes are going to be limited to the stable ARAM write locations ($0000-$04FF pre-conversion), and my ARAM map will do that job accordingly of tracking relocations as needed. This is because I consider everywhere else to technically be unstable and subject to revisions, unless it's a vanilla SMW build memory location. Plus, Super Mario World itself has one build variant in Super Mario All Stars + Super Mario World, which desynchronizes everything at $097D. That's because...
      • The initial cause is code swap of checking the $1DFB/CPUIO2 output value with executing the loading routine ($1DF9/CPUIO1). The code resynchronizes memory location-wise at $0ACD.
      • There is a new $1DFA/CPUIO1 command: $F0, which does some cleanup, then activates and jumps to the IPL boot ROM at $FFC0. This causes the memory locations to permanently desynchronize at $09E9, initially because the previous check for $FF uses a JMP opcode, but then this ID check seals the deal. Past these two checks, the memory locations are desynchronized by $A bytes. Then the new code itself makes an appearance in the SMAS+SMW build at $0B4A, imposing an additional $2A byte desynchronization for a grand total of $2A+$A=$34 bytes.
      • The loading routine is relocated to $FE00, and thus is not found at $1326 in the SMAS+SMW build. Interestingly, there are no further desynchronizations: the music, SFX and sample data are not affected, so the desynchronization is only in the code.
    • Custom assembly is trickier for me. I have conjured up a concept of a new version of this VCMD, but I determined it has to account for readahead in order to work properly, since the user may want to put in custom parameters beyond that point. The complexity increases with subroutines in the readahead code when it comes to validation, so I'll probably have to come up with a custom end marker to deal with this circumstance rather than try and auto-detect this case through code emulation. I also probably need some examples from Addmusic405 for proper porting, if it was ever used.



What bugs do you know about and are planning to fix?


See the Github issues list.

Do you have compiled binaries?


You can find the latest compiled binary for Windows here. (Thanks  Atari2.0 for setting this up and hosting the binary!)
Three files are available:
  • AMKFF_latest.zip - Contains the latest version containing all merged changes from the master branch.
  • AMKFF_latestbeta.zip - Contains the majority, if not all unmerged changes, for those that want a more bleeding edge experience with the program (and if you want to catch some bugs in the process caused by my changes). The branch that contains all of these changes that I have merged into this package is called bleeding-edge.
  • AddmusicK1.0.9_latestRC.zip is the latest release candidate build for AddmusicK. A special AddmusicK1.0.9 branch, branched off of the master branch, was created especially for this occasion. Any bugfixes needed to be applied during the C3 period will go in here, and the versioning has been updated accordingly here (except it is marked as a release candidate: a final version without this fine print will end up on SMWCentral once I submit it).
  • AddmusicK1.0.9_latestRCbeta.zip is like the above build, but it includes all four of the changes that might require some remoderation (and thus is stored in the AddmusicK1.0.9bleeding-edge branch). It otherwise does not contain other bleeding-edge branch changes: these are only changes that I feel can make the cut for 1.0.9 under the condition that the bugs are detected accordingly in the music archives.
  • All builds come with a copy of asar 1.81, as I intend on keeping asar up to date.


View the project here on Github

Very cool what you've been doing with AddmusicK over the past year. All the bug fixes and new features I've seen sound very useful and something that will definitely help out porters and hackers alike. Excited to see that you're planning to release an official 1.0.9 update, which sounds like it already has a lot of new cool stuff. The SFX priority seems like the best new feature in the planned update, but also the fact that you can compile it with gcc makes me hope that this can be compiled without hassle on Linux or Mac, which would be pretty great.
I've tried running a test song with 1.0.9, and was surprised to see that the overall insert size seems to be actually lower than 1.0.8. I guess this won't be the case anymore once the bleeding edge updates are factored in, which makes me worry if this will break existing ports and what would be the best way to deal with that. I've seen you mention manual code filtering, what would that imply?
About the bugs you mentioned that could require a remoderation, they sound niche enough that I doubt anyone has exploited them before, but you never know. I'd suggest talking so a music mod/manager to see what the best course of action for that would be.
Could you explain a little bit more what SFX priority is? Does that mean I can make a certain sound effect, like the "time is running out" SFX when a P switch is about to end, and make it impossible to interrupt with another SFX or note event? Also, how exactly does the vanilla SFX priority work? Do you mean how the jump and grinder sounds are on channel 5?

--------------------
NewPointless
Originally posted by KevinM
Very cool what you've been doing with AddmusicK over the past year. All the bug fixes and new features I've seen sound very useful and something that will definitely help out porters and hackers alike. Excited to see that you're planning to release an official 1.0.9 update, which sounds like it already has a lot of new cool stuff. The SFX priority seems like the best new feature in the planned update, but also the fact that you can compile it with gcc makes me hope that this can be compiled without hassle on Linux or Mac, which would be pretty great.

Everything except for AMKGUI can be compiled with gcc... however, by my own admission, I didn't create makefiles for AMMBatch and AM4Batch.

Originally posted by KevinM
I've tried running a test song with 1.0.9, and was surprised to see that the overall insert size seems to be actually lower than 1.0.8. I guess this won't be the case anymore once the bleeding edge updates are factored in, which makes me worry if this will break existing ports and what would be the best way to deal with that.

Yes, I actually managed the keep the overall default size lower. The main reason it's lower in overall filesize is because of the SFX data (done by removing redundant data and also some data optimizations, plus implementing an actual rest VCMD, with its overall usage offsetting the increase in filesize on the sound driver), rather than the sound driver build itself: the sound driver build itself is about the same size as the original. Three out of the four bugfixes that might affect existing ports barely affect the filesize: the only one that noticeably adds something to the filesize (specifically on the sound driver code side, rather than the SFX data) is the readahead modifications I made to account for $E9 and $E6 VCMDs.

Originally posted by KevinM
I've seen you mention manual code filtering, what would that imply?

See my planned implementation details for VCMD filtering. I recorded my intentions there for a start. The idea is that I would create true/false defines for UserDefines.asm for the user to select VCMDs to filter out. The biggest thing on my mind is safety: NOPs need to be safely implemented when VCMDs are encountered when they're not supposed to, and there are a few cases where if I implement filtering, I have to have a way to force them on if they're ever used (this specifically affects jumps, custom ASM, subroutines ($E9) and repeat sections ($E6)).

Originally posted by KevinM
About the bugs you mentioned that could require a remoderation, they sound niche enough that I doubt anyone has exploited them before, but you never know. I'd suggest talking so a music mod/manager to see what the best course of action for that would be.

One of them actually has an example that I was notified of by  Anorakun, and it utilizes the $DD VCMD while the transposition is non-zero. This is why I added some text on detecting bug exploitation.

Originally posted by NewPointless
Could you explain a little bit more what SFX priority is?

SFX priority within the context of this sound driver refers to two channels that conflict with each other due to one of them wanting to play while the other SFX is playing on that channel. Most of the time, the default behavior is for the incoming SFX to overwrite the previous SFX playing. If the SFX currently playing has some kind of priority assigned to it, then the incoming SFX is not allowed to play.

I have made a special VCMD ID for SFX priority, $E0, that, if defined at the very beginning of the SFX, assigns a priority to the SFX and gives it a value to compare to, with higher values having more priority. If this VCMD is absent, the default value is zero, meaning any SFX can overwrite this one. The pause SFX have the maximum possible, $FF, meaning they can only overwrite other SFX with a priority of $FF. Although they're not on by default due to AMK's default channel allocation scheme, I have priority values for all other SFX that replicate vanilla SMW's priority system (when they were used in the first place).

Originally posted by NewPointless
Does that mean I can make a certain sound effect, like the "time is running out" SFX when a P switch is about to end, and make it impossible to interrupt with another SFX or note event?

Yes, if it is the only SFX with that high of a priority. I programmed the pause SFX to have the absolute highest priorities, and ties in priority are coded to overwrite each other: thus, only the Pause SFX can interrupt each other.

Originally posted by NewPointless
Also, how exactly does the vanilla SFX priority work?

OK, this is going to take a bit for me to explain... because the original priority system is hardcoded. AMK 1.0.8's SFX priority system is also hardcoded, and it is what I replaced.

First, this is the ASM for what I replicated in a more soft-coded fashion (courtesy of loveemu's disassembly of the sound driver):
$1DF9 (APU port 0/$2140/$F4)
Code
06ae: 78 12 00  cmp   $00,#$12
06b1: f0 0f     beq   $06c2
06b3: 78 11 00  cmp   $00,#$11
06b6: f0 0a     beq   $06c2
06b8: 78 11 04  cmp   $04,#$11
06bb: f0 0b     beq   $06c8
06bd: 78 1d 04  cmp   $04,#$1d
06c0: f0 06     beq   $06c8

Memory location $00 refers to input from the SNES's $1DF9, cleared at a rate of (256/56)*16 timer 0 ticks (or 1 SFX tempo tick).
Memory location $04 refers to the SFX ID currently playing from the SNES's $1DF9 (within the vanilla sound driver's context: behavior has changed somewhat on AddmusicK).
Memory location $06c2 refers to a target memory location that automatically goes to read the input data from the SNES.
Memory location $06c8 refers to a target memory location that bypasses reading input data from the SNES.

$1DFA (APU port 1/$2141/$F5)
Code
09e5: e4 01     mov   a,$01
09e7: 68 ff     cmp   a,#$ff
09e9: f0 b1     beq   $099c
09eb: 68 02     cmp   a,#$02
09ed: f0 8e     beq   $097d
09ef: 68 03     cmp   a,#$03
09f1: f0 a2     beq   $0995
09f3: 68 01     cmp   a,#$01
09f5: f0 1d     beq   $0a14
09f7: e4 05     mov   a,$05
09f9: 68 01     cmp   a,#$01
09fb: f0 06     beq   $0a03
09fd: e4 01     mov   a,$01
09ff: 68 04     cmp   a,#$04
0a01: f0 0b     beq   $0a0e             ; (jmp $0ace)
;
0a03: e4 05     mov   a,$05
0a05: 68 01     cmp   a,#$01
0a07: f0 48     beq   $0a51
0a09: 68 04     cmp   a,#$04
0a0b: f0 04     beq   $0a11             ; (jmp $0b08)
0a0d: 6f        ret
0a0e: 5f ce 0a  jmp   $0ace
0a11: 5f 08 0b  jmp   $0b08

Memory location $01 has an identical purpose to Memory location $00, except with a different APU port.
Same case for Memory location $05 as compared to Memory location $04. (within the vanilla sound driver's context: behavior has changed somewhat on AddmusicK, but not as much as the previous APU port mirror, so to speak)
Memory location $0a03 refers to a target memory location that checks what SFX is currently playing.
Memory location $0a0e refers to a target memory location that initializes the girder/grinder SFX (after jumping to $0ace).
Memory location $0a14 refers to a target memory location that processes the next tick for the girder/grinder SFX (after jumping to $0b08).
Memory location $0a51 refers to a target memory location that processes the next tick for the jump SFX.

$1DFC (APU port 3/$2143/$F7)
Code
0816: 78 24 07  cmp   $07,#$24
0819: f0 13     beq   $082e
081b: 78 24 03  cmp   $03,#$24
081e: f0 0a     beq   $082a
0820: 78 1d 07  cmp   $07,#$1d
0823: f0 09     beq   $082e
0825: 78 05 07  cmp   $07,#$05
0828: f0 04     beq   $082e

Memory location $03 has an identical purpose to Memory location $00, except with a different APU port.
Same case for Memory location $07 as compared to Memory location $04 (within the vanilla sound driver's context: behavior has changed somewhat on AddmusicK).
Memory location $082a refers to a target memory location that automatically goes to read the input data from the SNES.
Memory location $082e refers to a target memory location refers to a target memory location that bypasses reading input data from the SNES.

In all three cases, priority is implemented via checking hardcoded SFX IDs for each port based off of both incoming data from the SNES and the ID currently playing.

Currently in AddmusicK 1.0.8, the only SFX that have any priority coded for it are the "time is running out" SFX (in both ProcessAPU0Input and ProcessAPU3Input in main.asm as ID $1d) and the pause SFXs (albeit not including the brand new unpause silent SFX, and only done in ProcessAPU0Input in main.asm as IDs $11 and $12). $1DFA also has its SFX coded, but the coded is currently poorly adapted to account for outside interferences and does not restore properly, particularly with the girder/grinder SFX.

Originally posted by NewPointless
Do you mean how the jump and grinder sounds are on channel 5?

Nope, those two SFX were always on the last channel, even in vanilla. (channel 8 if we start from 1). $1DF9's SFX is what was in that location in vanilla SMW's.
The work you are doing for this program is absolutely amazing! It's wonderful to see all of the features you've been adding over the time you've been working on this. And finally we can have more AddmusicM ports compatible too, it's huge to have that pulse wave command restored along with many other features added! It's awesome to see all of these bugs ironed out too!

Love seeing all of the stuff that you're doing and looking forward to seeing what else you do! Keep up the amazing work! #tb{:j}
I'll make sure that code filtering is among the things I implement for the next version after finalizing 1.0.9 (and implement the actual conversion, which is among my last steps remaining).
Pages: « 1 » Link
Forum Index - Sunken Ghost Ship - C3 Museum - Summer 2021 - AddMusicKFF Summer 2021 C3 Update Thread + AddmusicK 1.0.9 Release Candidate

The purpose of this site is not to distribute copyrighted material, but to honor one of our favourite games.

Copyright © 2005 - 2021 - SMW Central
Legal Information - Privacy Policy - Link To Us


Menu

Follow Us On

  • YouTube
  • Twitch
  • Twitter

Affiliates

  • Super Mario Bros. X Community
  • ROMhacking.net
  • Mario Fan Games Galaxy
  • sm64romhacks