Language…
9 users online: Batata Douce, crocodileman94, DimeR, DixyNL, eltiolavara9, matthwsan, Sweetdude, temsuper1, TheBourgyman - Guests: 258 - Bots: 291
Users: 64,795 (2,377 active)
Latest user: mathew

Posts by Near

Near's Profile → Posts

Quote
My only real concern about this is that there seem to be an ungodly ammount of ways MSU-1 can implemented in a hack


It took a while to perfect how to implement game folders, so unfortunately there was a lot of variance right at the beginning (specifically in the fine tuning of the manifest syntax.)

The problem is that a lot of people decided to stick with bsnes v073 (or forks of it), which was released in 2011 or so.

I can't do anything about people clinging to six-year old emulator builds :(

...

We do have an official, set-in-stone standard at this point:

For game folders (higan, sd2snes*), it's:

Mario.sfc/
* program.rom
* save.ram
* msu1.rom
* track-#.pcm

(* sd2snes game folder support was a patch by qwertymodo; unclear if ikari will accept it.)

For game files (higan, Snes9X, sd2snes), it's:

SNES ROMs/
* Mario.sfc
* Mario.srm
* Mario.msu
* Mario-#.pcm

Note how higan is in the latter list. So you can produce an MSU1 package that will run in all emulators and on real hardware by going with the latter.

Also, note how there's no longer any XML or BML manifests required. You *can* optionally create one, and then give all your songs whatever filenames you like, eg "02 - Boss Battle.pcm"; however that is optional, and will break compatibility with everything else, so it's not really recommended.

Quote
tbh MSU-1 support would be more viable if byuu just stuck to one format.


I only changed the format from XML to BML once. That change was also made at least 4-5 years ago.

You can still use BML, but now I've made that optional so that neither are needed.
Quote
where patchinfo.xml is an XML document that's compliant with BPS's metadata format. Eg.:


There is no official metadata format. Long ago, I was defining it as XML, but that's no longer the case.

And yes, I realize that's a problem, but ... no matter what I did, I would limit certain use cases, so I just don't define it anymore.

The only thing that's absolutely forbidden, is to use metadata to extend the patching format itself. That will break compatibility with other patchers that don't parse the metadata.

My recommendation these days is that metadata is in a human-readable format: XML/JSON works, but YAML/BML is much nicer. Plaintext is also okay. Markdown is nice, too. Not needing a special tool to read it is convenient and lets *any* patcher potentially display it.

Even if we tried to define an official format, it'd just be a mess. Look at MP3s. There's ID3v1/v2, APEv1/v2, Lyricsv3, etc.

Quote
The manifests are indeed not arbitrary text. Per the BPS specification, it's an XML document; in practice, it tells bsnes if it's HiROM, LoROM, or something else.


Noooooooooooooooooooo please don't ;_;

I know BPS is an agnostic format, and my own (lousy) patcher doesn't support it yet, but ... SNES/PCE/etc game BPS patches should always be made -sans- header. That's also the only way Snes9X and some versions of bsnes/higan will soft-patch them.

Quote
The BPS format doesn't contain any filename for the metadata, we can't do any better.


It wasn't meant to be a file you extract. I was afraid people would get cheeky and embed ZIP files in there and ... ugh. So that's why I didn't include a manifest filename field.

Quote
Yes, byuu is crazy.


Can confirm.

Quote
That entire metadata field is silly. Without a proper specification, it's only confusing. Even if there is a bsnes manifest, it's better off being a separate file beside the bps.


The idea of embedding a higan manifest was certainly a thing long ago, but is *very* much not the case today.

I have tried and failed to make some kind of universal standard format for describing the 100+ SNES PCB forms in existence, and I no longer try.

I generate manifests dynamically now instead of requiring them from users. So yeah, please don't embed bsnes/higan manifests in BPS patches ... the format is not stable and has changed not only details, but file formats (XML->BML).

Quote
Yeah, it disappeared off the internet.


I redid my site, and all the old documents were in raw HTML form. Now I am using a markdown-like syntax, but instead of converting all the old stuff, I just deleted it. Because I am terrible.

Haven't had the motivation to recreate the document since then.

Quote
UPS support


Sigh, I'm very sorry for inflicting that upon the world :(

Why can we never truly eradicate *anything* deprecated? ;_;

UPS is a trash garbage format that needs to die.

I mean, as long as it's just UPS patch application and not patch creation, I suppose it's pragmatic, which is Alcaro's cup of tea, so that's okay.

Quote
the bps-delta algorithm is both faster than bps/beat AND results in smaller files


Yeah, my algorithm has quadratic overhead, and to make it faster, I cut corners on the match comparison length, resulting in larger files.

beat's delta mode is awful. Flips' is vastly superior.

Quote
But Flips is six times slower than Xdelta, and patches are about the same size. I have no idea what it does. Maybe it's cheating by not looking at the entire source file. Maybe I should rip apart its source code and steal its secrets.


On its raw speed, no ideas.

But as an unrelated tangent on its patch sizes ... BPS vs Xdelta is tough.

BPS beats plain Xdelta, but not Xdelta -9. But Xdelta -9 is technically using compression. BPS often beats Xdelta -9 when you put the BPS file inside a 7za -9 archive. But if you put an Xdelta -9 patch inside a 7za -9 archive, then it tends to win on size once again.

But where BPS always wins is ... the BPS file format is 100x simpler than Xdelta's. Even if the patches are slightly larger, it's absolutely worth it.

Consider there are *zero* implementations of Xdelta other than the original author's. Then again, his default implementaiton doesn't suck, unlike the one from the BPS author.
Quote
Should've known you couldn't restrain yourself.


Of course not.

Quote
Don't allow encoding -0 in Source/TargetCopy


Wish you were around during the entire year I asked for suggestions or improvements as then went from being called blip through to release as BPS.

That's far and away my biggest regret. I didn't even notice it for a few years after the format was finalized, but it sure is glaringly obvious now ;_;

The simple change would be here:

https://gitlab.com/higan/higan/blob/master/nall/beat/patch.hpp#L186

> - if(negative) offset = -offset;
> + if(negative) offset = ~offset;

It would also make the patch sizes infinitesimally smaller. But fixing this would require a version bump, and that just isn't worth it because we'll have to still support BPSv1, so I'm not going to do it.

Quote
Ditch metadata, it goes in a separate file (and it's already used in nonstandard ways, spec says XML but bsnes disagrees)


I hope you don't judge over people running ancient versions of bsnes like v073 (six years old) or v094 (three years old.) Current higan/bsnes doesn't even support BPS >_> But in a future release it will, but not with metadata parsing.

Quote
Invert 0x80 bit in the encoded numbers, set means continue; it would simplify the decoder


I believe I tried it both ways, and there was no real difference in compression ratio.

Quote
Make the checksums mandatory


The values are mandatory. I'm not going to require it upon applying patches, or we break multi-patching, and that's a big deal to some. Although RHDN still pretends multi-patching isn't possible with BPS, even though it is.

People -can- encode checksums of 0x00000000, but that's not valid per the spec. It'll be treated as though that's the checksum of the file.

Quote
(1) (2) (3)


Sorry, no idea what any of that means.

Quote
I have no idea how defining the canonical representation for the bpsview output would limit anything.


We could just say "whoo XML!" or "whoo, 'foo: bar' line feeds!", but if we tried to name specific fields (author, version, description, license, ...) we would -invariably- never have all the fields we wanted, and would have a lot of fields people didn't want.

If you're serious about standardizing this, I'm willing to work with you on that. Personally, I would *much* prefer this:

Code
version: 1.0
author: byuu
website: https://byuu.org/
description
  :this is a dumb patch.
  :don't use it.


To this:
Code
<?xml version="1.0" encoding="UTF-8"?>
<metadata>
  <author>byuu</author>
  <website>https://byuu.org/</website>
  <description>this is a dumb patch.[br]don't use it.</description>
</metadata>


We could drop multi-line support and parsing it would be two lines of code:
Code
for(string item : metadata.split("\n").trim("\r")) {  //fuck carriage returns ;_;
  auto parts = item.split(":", 1L).strip();  //fuck white space ;_;
  //parts[0] = name, parts[1] = value
}


Quote
What on earth does that have to do with anything either of us said


Nothing, I'm just making it clear.

Quote
But while you're here, maybe you could convince this guy that headers are evil


The link to beat has been dead for over a year now because I'm a super lazy fuck, anyway.

Quote
As much as I want to remove that ugly special case from Flips, I can't and won't until we can declare headers dead (at least within SMW), aka when *that guy* stops being a blueberry.


I did my part with Header Magic. If you guys want to join in, I've been calling them "FuSoYa headers" instead of "copier headers", as this is really the last major known tool I'm aware of that uses/enforces headers in 2017. I don't think it'll result in him supporting headerless ROMs, but at least we can call them what they are. Nobody's using floppy-disk based SNES copiers anymore.

Quote
I see your facts and I raise you some alternative facts


Isn't Xdelta its own extension on VCDIFF? I wasn't aware you could use any of those patchers to make .xdelta files.

Quote
A desktop app written in PHP. Just ... why.


It was all Derrick knew at the time. But I share your concern all the same.

Quote
byuu! Nice to see you. I didn't realize you still frequented these forums.


I don't. When you mention my name three times ... it shows up in Google search results.

Quote
Maybe beat is finished, there is a point where there are no more plausible features to add.


I need to add SA-IS, but I'm too ... obsessive? to add a library I don't understand. I want to implement the algorithm myself first. And ... I don't have the requisite seven years of free time to understand that voodoo ;_;

I do also want to define and finalize the BPM (multi-patch) and possibly BPA (strict archival) formats, though. Not that I'd expect Flips to support that, but those are useful.

Quote
i'm more important


You kind of are. You've done more for the BPS file format's success than I have.

The only thing really holding the format back at this point is RHDN being unreceptive to encouraging BPS to replace IPS. I suspect part of the reason why is because the format was created by me.
Unexpected end tag (</author>) at 3297, expected </pre>
Unexpected end tag (</website>) at 3337, expected </pre>
Unexpected end tag (</description>) at 3406, expected </pre>
Unexpected end tag (</metadata>) at 3419, expected </pre>
Quote
But I can live with another +1.


Yeah, same. It isn't worth breaking all existing patches; or having two versions.

There's still some people that don't trust me over UPS now.

But, you know how I am about perfection. It really gets to me >_<

Quote
Yes, of course patch sizes remain identical if you flip that bit.


I meant after compressing it via 7za -9. It was a wash either way.

Quote
Which is why that one mentions decoder simplicity, not patch size. In particular, these XORs and 1<<7s.


Well I mean it's too late now anyway, but ... if it would've resulted in less code, then that's another unfortunate miss.

Quote
And I believe said some are cave dwellers from the era when our feature patches used IPS.


Personally, I don't like multi-patching at all.

UPS had the problem that if two patches changed the same byte, the end result would be invalid no matter what. Both IPS and BPS have the identical problem that the byte will be the result of the second patch.

Both are a big problem. But if I *had* to support it, the ideal way would be, you pick the *original* ROM, then you pick all the patches you want at the same time, which all have the original-file checksum as it should be, and then when the patcher applies them all ... it's an -error- if two files change the same byte of the source file. But that's just me.

You really should be using some other kind of system, like a GUI version that loads assembler patches that can relocate code so all the patches apply safely.

Really ... I think they were trying to justify reasons to stick with IPS instead of using something newer and better. And that's likely why many keep acting like BPS can't multi-patch, even though it can. I don't believe that most of the people even bringing up that point actually care that much about multi-patching -- it's a red herring.

Quote
Most of what would fit in this metadata works at least as well in a readme.


It's kind of a choice thing. The shitty part is how difficult it is to distribute folders on the internet. You have to distribute a ZIP archive or something, and people have to extract it.

Of course, this is undermined by 7za -9 making BPS patches even smaller.

But yeah, people like nitro use it. And it's very simple to ignore the metadata if you don't care about it. It's one offset += readVariableLength();

Quote
Information directly usable to the patcher, like filenames, could be useful there.


I didn't want to embed filenames into BPS. But if you find that important, BPM embeds filenames. And there's no reason you can't have a single-file BPM patch.

But BPM's main justification is for patching PC-based games that have lots of files. Could also be a smarter way to patch CD-based games than a big BPS patch against an ISO. (of course, the real fun would be applying a BPM patch *against* an ISO ... fun!)

Quote
[br], good xml.


This damn forum software was parsing the HTML variant into a newline, so I had to edit it to that. Technically it should be [br/] but, same problem.

Quote
And yes, XML sucks. It's common, but that's its only redeeming quality. Bloated garbage with way too many features, half of which are only useful for XHTML and the other half are security holes. Even without DTDs, why on earth does it have both attributes and inline text, and why do closing tags need to repeat the tag name?


The biggest hell with XML is being able to put elements *inside* of text.

How do you even parse that shit into a node list, I mean seriously?

But yeah, "foo: bar" is so much nicer, both to read and to parse.

Quote
I'm afraid I must veto that one. Either we use full BML, or we use something that doesn't look like BML (INI, for example). If we use almost-but-not-quite BML, someone will miss the memo and use BML's full power, and then all other patchers are sad.


Yeah, but I remember you not liking BML's choices either.

I wasn't crazy about them either. Especially multi-line text. But oh well.

Quote
You've got dependencies on all kinds of unknown stuff already. You don't understand your kernel's scheduler and file system, libc's condition variables, GTK's styling functions, or whatever OpenGL does, do you? Why would another piece of unknown matter?


So the key goal is to compile out of the box on Windows with a fresh MinGW64 install. Be it TDM, msys2, etc. And ... it does.

Thus, unlike most software projects, you don't have to go fetch zlib, libpng, libcurses, etc.

You're right that on Linux there are more dependencies, like GTK+, because there's just no choice there. I'm not writing my own raw GUI toolkit from scratch (yet...)

I could drop the entire library into beat, but if that library ever gets made into a libsais or something, Linux maintainers will lose their shit over including my own instead of using libsais from the repository.

It's doable, yes. But I'd rather understand SA-IS first.

Quote
...wait, you're crazy. Can't really argue with that.


Well, at least I haven't written my own operating system to talk to god (yet...)

Quote
I'll consider it if it becomes more common than IPS/UPS.


Almost nobody fan translates PC games. Those that do just release the whole complete pre-patched game. Most likely because there's not really a good folder patching solution out there.

I still need to finalize the formats, too. I should probably post threads about them. The main discussion is around which metadata to track (file timestamps? Unix permissions? etc.)

Quote
stop agreeing when i'm trying to pick a fight


We can always argue about bsnes-mercury instead, if you'd rather :P

I'm a lot less annoyed now, knowing that the defaults have the hacks disabled, but I'm still really bummed about your unrelenting pragmatism re: DSP firmware. I'm *so* close to getting CV Reynolds to ship the firmware with the individual games ... but I need more impetus.

Maybe I can talk the Snes9X devs into merging a patch from me to replace their HLE with LLE too? Probably not ...

Quote
Easy to veto if you've got the staff team on your side


Specifically, the site admin. But yeah, exactly.
Quote
Except whatever evil voodoo shenanigans you did to the source code.


I am like 95% sure I released the source to Header Magic separately ...

Shit. I definitely don't have it anymore.

Quote
BPS has the additional problem that if two patches change non-overlapping bytes, but the second one reads from a byte changed by the second, you're gonna have a bad time.


But that's not going to happen. Each patch is applied in a separate context. It's 100% equivalent to IPS.

Now if you were to insert/delete bytes from one patch ... that will definitely cause issues. But it'd be the same thing if you tried with IPS, only the IPS patches would be gigantic since they don't support delta patching.

Quote
But the "what the hell is that good for" issue remains, so I'm still lazy.


Yeah, I'm not doing it either. And I'm sure someone will find a way to craft two patches that, even without overlapping bytes, breaks horifically if both are applied together, but both work when applied individually.

Quote
You are telling them to use Flips, right?


Yes. https://byuu.org/tool/beat/

Quote
Note: currently, beat v03 uses a quadratic function for creating delta-based BPS patches. This can be quite slow, especially on files over 32MB in size. As such, for the time being I'd strongly recommend using Floating IPS by Alcaro instead. You can download it here.


I know beat is terrible.

Quote
It's more about ensuring the new option is 100% undisputably better.


I used to think that was it, but ... nowadays, I think it's that the new one has to be better in a way people care about.

I really thought, "you don't have to guess whether you need to add or delete a header, and your patch success rate goes from 50% to 100%" would be the killer feature. BUT I GUESS NOT.

But yeah, look at nitro. He used the format because of metadata. It being better than IPS in every respect alone wasn't enough.

That said, I'm also opposed to throwing in the kitchen sink and trying to please everyone. I tried that with bsnes and it almost broke me as a person.

Quote
didn't we just agree I'm not doing bpm?


Yeah, and sure. Flips is better suited for patching ROM files anyway. Was just mentioning it.

Quote
Or maybe we should yell at whoever wrote it. Those code tags are stupid.


That they are. HTML should not be parsed inside of code tags.

Quote
I implemented text nodes as just another child. Something like

<foo>bar<baz/>quux</foo>
->
<foo><text:bar/><baz/><text:quux/></foo>


... damn. That's really clever. If I still used XML at all, I'd be updating my parser to do just that.

Quote
The only part I remember not liking is foo: bar. According to you, it's key="foo" value=" bar".


Correct. The key thing though is that wen you say node.text(), it strips whitespace off both ends. So it's not a problem in practice. The concern is thus:

Code
author: byuu
version: 1.0
description: blargh


vs:

Code
author:      byuu
version:     1.0
description: blargh


In the latter, should it be key="author" value="byuu"? If so, just use node.text() and get the same thing. And what if you actually -wanted- a space at the start?

If not, then should it be value=" byuu"? Because if so, gross! And what would "author:byuu" be if we required it to be ": " for the separator? An error?

Quote
And here's another unclear part: We both support comments. But you use //, while I use #.


Yeah, that's just a style choice. I was matching comments in bass, where I also use //.
But then you have xkas/Asar and its ; comments.
I like consistency a lot, as you've probably surmised.

But really, // vs # wasn't a big deal to me. If you had asked back when I was standardizing the spec, I'd have probably said okay.

Comments are rather limited in BML anyway. You can't use them after you use : on any line, so they're not very friendly. I don't think I even support them after -any- non-whitespace character appears.

Quote
Windows or Linux, there's still a condvar implementation in your process. If you don't want libdivsufsort in your process, why isn't libc/kernel32 a problem?


Everyone running higan has libc. Everyone running higan/Windows has kernel32.

libdivsufsort is a separate library. It's just me being picky and NIH. Feel free to ignore me. Or do you want competition for Flips that badly? ;)

Quote
and I used SA-IS for a while before finding LDSS


... you moved on from SA-IS? ;_;

Quote
Gladly, I love proving people wrong. To start with, is bsnes-mercury the performance core or something else?


Ask this idiot. ;)
Unexpected end tag (</foo>) at 3151, expected </div>
Unexpected end tag (</foo>) at 3197, expected </div>
Quote
But the tricky parts - nall::launch, nall::detours - remain, I believe the rest was pretty straightforward.


The trick is that almost every Win32 API has the same entry point. I think something like "push ebp; mov ebp,esp;" ? Forget exactly. So you don't really need detours if you were to code your own.

Quote
But with any existing software, it breaks.


Unsure what you're talking about. Right now you'd apply one patch to a ROM. Then apply another. The result will be identical whether it's IPS or BPS.

Quote
Fun fact: We had a couple of IPSes made from unheadered source and headered target ROMs. They were huge, but still worked.


>_____>

Quote
Your favorite feature is the deltaing


Not sure I really have a favorite feature ... but I really like the delta patching, yeah.

I'd like the format to go beyond just the SNES.

Quote
...unfortunately, RHDN seems to not like any of those.


I'm really not trying to be insufferable, but I sincerely believe the only thing they don't like about BPS is me being the format's author. Again, not everyone. But mainly the most influential that could affect the site's policies are looking more at the person instead of the format. But hey, maybe I'm wrong. Would love to see any arguments that aren't an appeal to tradition.

Quote
I think it did break you. Either that or you were broken already, you've been acting quite strangely.


I've always been strange. Before then, at that time, and especially now.

I will admit I've been more strange lately. Especially on Twitter. I'm a lot less defensive and more open these days. Just tired of being so serious and caring so much about what other people think.

Quote
...what, you didn't think of that yourself? Sometimes I wonder how you managed to figure out the SNES if you struggle with this kind of stuff...


No. But to be fair, I didn't really have a pressing need to do it, so I didn't spend much time on it.

But I'm not that smart. I solve problems by putting in way more time than smart people put in. Most extreme example ... I spent about a week working out these three lookup tables to compute the DMA<>CPU delays. And I emulated it, and it worked perfectly. I posted my findings, and two hours later, anomie posted and said, "oh, it's just X - 8 + Y before; and then TotalCycles % LastCycleCount after" ... and sure enough, he was right.

Sure, my tables were why he was able to figure it out. But he figured out what I didn't see, and extremely quickly.

Quote
Again, that screws up foo=" bar ", which is quite clearly not intended to be equivalent to foo="bar".


How often do you really want to start or end strings with spaces? If you do, then have the parser look at .data() instead.

Quote
Same as author="byuu", author: byuu and author: byuu. The separator is colon plus zero or more spaces.


So what if you want author=" byuu" with : syntax? That's just not possible now?

Quote
The results are nearly always the same; we both agree that whitespace should (almost always) be trimmed from 'foo: bar', we just do it at different points.


Agreed :)

One of the formats took it to an extreme and tried to design data formats for every value. Defining strings, booleans, integers, floats, arrays, etc. Way too complicated and limits the flexibility. Just let the person parsing it decide how to do that.

Quote
You didn't answer my questions, so I'll just make up my own answers:


I only support comments before whitespace. Once a non-space/tab appears, no more comments for that line.

Quote
You're good at dodging questions.


I assumed it was the performance profile because it was focused on speed and hacks. Apparently I was wrong. Sorry.

Contrary to popular belief, I'm not *always* right ;)

I still don't like your fork. I still don't like 14 of the 18 forks (that I know about) of my emulator. At times I really wish I didn't move to the GPL. But I did, so what you're doing is allowed, but you still have to put up with me being bitchy about it, sorry.
Quote
Though I guess it helps that I have a proper BML writer, which automatically switches to multiline if needed.


I have a function that can write out a BML node (and all of its children) to a string that you can save into a text file. But the formatting is shitty on it. So if you import a document, and then save it right away, it loses all the unique formatting, and compacts things like foo:bar without aligning the values vertically. So instead I usually just use file print statements to write out BML data.

Quote
'b #b' is error, understood. I'd prefer allowing it, but if you say so.


I mean, I'd like to support it, but I don't currently.

The tricky part is, once you hit :, I don't want to forbid the use of # in the value. And so you get:

Code
foo bar=hello #comment
foo bar:hello #not a comment


And while that's perfectly parseable, it's deceptive to someone that doesn't know the ins and outs of the file format.

Quote
The biggest speedhack it does is keeping pointers to ROM/RAM around, rather than calling that function pointer table for every access to $00:8000 or $7E:0000.


That table is a living hell that eats CPU cache for breakfast. higan's SNES manifest format is way too fine grained, allowing for pathological cases that make shortcutting harder.

Without manifests, one could combine the speed() function with identifying what was actually being addressed (into MMIO, WRAM, CART, etc.) and not need two decoding steps.

One of these days I'd like to write an Snes9X successor. Something more accurate, but optimized for speed, and allowing more creature comforts that I don't want to add to higan (movie recording, layer toggling, SPC ripping, etc.)

Quote
Maybe I should've said no when Squarepusher told me to put those chips back. I'm a weak-minded coward.


I suspect his decision driving process is more, "what would annoy byuu?" than "what is best for the emulation community?"

I know it's more convenient if you don't have the firmware. But the problem with convenience is, it keeps people lazy and things never improve. And then we deal with the false bug reports caused in games like Suzuka 8 Hours because the HLE has no timing emulation. Yes, it would be painful for a short while as people had to fetch firmware, but this would lead to pressure on No-Intro to include the -whole- ROM contents with their games instead of separating the firmware out, and then everyone would have better emulation. Pragmatism is the nicer option only in the present.

Quote
On a totally related note, wonder how much faster bsnes would be with C++17 coroutines, libco plus branch predictor equals false. And lack of valgrind isn't really my favorite feature.


Aside from them being scrapped from C++17 (as well as UFCS, sigh ... fucking design by committees), they would also be required to have a stack frame, and I am not sure C++17 coroutines did.

In that case, I am still unsure if C++17's could beat out libco. Only real possible way I can see is if the compiler was smart enough to transform simple cases (like my SNES DSP core) into a state machine transparently.
Quote
My writer expects to be told which style to use for each node.


Yeah, I need to extend mine to store the type of the original node when read in, and only override if the value changes to a form that can no longer be used. Eg "foo=bar" but now you changed it to "foo=bar\r\nbaz", so it'd have to become a multi-line node.

Quote
Could always allow parsing it, but document it as not recommended


I can do that, yeah. I treat "\r\n" and the use of tabs the same in BML now. Forbidding them made the format a nightmare to work with. If you want people hand-editing these documents, you want them to be as permissive as possible ... what is that old saying? Flexible on inputs, strict on outputs? Yeah.

Quote
Even without my use-pointer-directly optimization, reducing cache pressure would help a bit. If you don't bother with pointers, you won't need much extra code, either; I'd estimate about 30 lines for a 2-3% speed boost (Mercury gets about 5%, but its direct pointers take more than 30 lines).


Yeah, so this is a lot of the contention between our philosophies.

I used to have a tile cache that decided bitplane to bitmap. It was updated on VRAM writes automatically for 2bpp, 4bpp, 8bpp modes. And in this way, grabbing pixels to blit out was a lot simpler. The result was about 40 lines of code and a 2% speedup.

Then I had this bug where a soft reset would flush the tile cache but not the VRAM. One game, Super Buster Bros, detected a reset and would not reload the VRAM. So I started rendering empty tiles and the game predictably did not look right at all. Amusingly, ZSNES has the same bug.

So I just see these kinds of performance speedups as ... adding to the complexity of the program as a whole. And I'm not smart enough to handle what I've already created. So anything I can do to simplify it, is worth doing. So there are many instances I give up those 2% - 3% speedups. And yeah, after a while, they all add up. I could probably have my emulator 20% faster without an extreme amount of effort if I were willing to add more code like this.

Quote
For your own sake, don't create a GUI, it'll just drive you crazy again. Leave it to libretro or something.


Yeah, that's the main thing I want someone else to make. And ... a lot of people don't enjoy the libretro UI. Myself included; I prefer regular UIs that match other windows on my desktop. Fullscreen one is fine if I'm in fullscreen mode, but often I want other windows open, like a chat window or a web browser.

But, yeah. My thought was if this were to happen, it'd be a team thing. And I'd try to design the core parts. Have someone else help with the optimizations and caching and GUI parts. Or multiple people helping. I don't care.

Quote
I can't get an improved version past Squarepusher.


That right there's the hardest thing about me working in a group. Especially with someone else as a leader instead of an equal. If someone tells me no, I immediately want to make my own software instead :P

Quote
He wouldn't say to no a chance to spite you, but he doesn't spend any real effort on that either.


I'm really hoping the person that keeps making brand new Reddit accounts to attack me and defend Retroarch every single week isn't him.

I'm not so vain as to believe I can't possibly have anyone else that dislikes me to that extent. But, whoever it is ... they're clearly a superfan of RA and aren't doing the project any good by being so overzealous about it. They're also a coward for hiding their identity through new accounts each time.

But ... meh. I hope you're right. As long as he doesn't go around bugging me, then I don't have any beef with him either.

Quote
Instead, I'd summarize his goals as "what's best for RetroArch's users immediately right now", ignoring everything else


Yeah, that's a great summary and fits perfectly with the SNES coprocessor HLE. It's the best option to make users happy *today*, but it's the worst possible option in the long-term.

But like I said, Snes9X is the real domino here. We get LLE into there, and No-Intro will fall in line and this'll be a solved issue in six months. But ugh, I really don't feel like doing it, and waiting another 3-6 years for the next Snes9X official release to be put out again.

Quote
Isn't that the entire point of those coroutines? Microsoft's C# compiler does exactly that already.


Only if the coroutines also have a stateful mode that can capture yielding from inside subroutines. In which case, yes. That sounds amazing. And if it works well, and isn't more than say 10% slower than what I have now, I'd be more than happy to switch to them and drop libco.

Quote
Pretty sure stackful can be emulated on stackless with (the equivalent of) 'yield from EverySingleFunctionCall()', and compiler probably inlines that... maybe...


You certainly can, but ... keep in mind each function will also have to capture all variables prior to the yield instruction, and restore them every invocation. It can get *way* more expensive than a one-time stack pointer change.

The only thing that makes libco so bad is ... processors throw a big fit when you change the stack pointer and start throwing out lots of pipelining optimizations on you in retaliation. State machines clearly win on a single level, and possibly even two levels deep. But get to three or more levels of depth and libco stomps them into the ground.
Quote
Most of the time, we're perfectly happy with each others' choices.


I only really minded Mercury, and I hopefully explained why. You don't have to agree, and you're welcome to disagree with me on it. You have the right to do it under the GPLv3 no matter how much I complain.

I gave some grief about Asar, but honestly I don't really care that much about cross assembler syntax. It's all the same after assembly. I just feel a bit bad people are stuck with the ugly syntax I came up with, but if they like it then who am I to judge?

Quote
But I believe you said bsnes is intended as a reference document more than being playable


That's why I left out even features I'd like, such as layer toggling, movie recording, music ripping (I naturally came up with my own format for that too, of course), etc. And why I'm struggling so much to create a debugger. Something I've wanted since day one of starting on bsnes.

Quote
libretro is no more than Emulator::Interface.


True enough. And both interfaces stem from libsnes initially, but were made to support more systems.

Theoretically, it should be quite possible to make a bridge between the two APIs. Though it's probably not very desirable (as you'd only want the SNES core), and is unlikely to get approval to be merged.

It would be amusing to run libretro cores in higan, and Emulator::Interface cores in RA, though.

Quote
and otherwise changing his mind fifty times a day.


Yeah, you should've seen him on Reddit today. "byuu, let's bury the hatchet and be friends!" Next post, "I hate you and you're awful and here's 70 paragraphs explaining why!" Next post, "We really need to stop fighting." Next post, "Here's a hundred things that I hate about byuu:" >_>

Quote
Once minir is a bit more finished


On that note, the main change since v094 to Emulator::Interface was redoing the file handling to make it much easier to virtualize filesystem access, thus allowing you to load plain merged ROMs ala other emulators, and have path selectiosn for different filetypes and such. In fact, I used it in last year's April Fools' joke of bws to do exactly that.

Quote
He said he's way too busy with RetroArch to have any time for that.


Is he really? Count how many words he's said in this thread:

https://www.reddit.com/r/emulation/comments/6cnm1e/does_anyone_else_think_that_all_libretro_cores/

He really doesn't seem all that busy to me >_>

Quote
Or maybe 500 random guys with one account each, none knowing about the others.


I believe that at least 500 people hate me; that's totally fair and justified given my actions. But I can't believe there are others that hate me to this insane of a level.

I mean look at you: I've driven you up the wall and the worst you did was post a page of quotes about me.

Quote
LLE is about 5x slower for no real end user benefit, I doubt they want that.


No perceived user benefit. Plenty of actual benefit. HLE acts like a quantum computer that returns the answers to any advanced math problem instantaneously to the games. Zero timing emulation involved. Games run incorrectly due to this. Preservation is also a very important reason here. If you consider improved accuracy no benefit, then why not just drop bsnes-mercuy and have people use Snes9X?

Further, I do still believe static recompilation is possible thanks to strict Harvard architecture being used by these chips. It would probably only be marginally slower after that.

Quote
And wrong direction, I expect properly implemented compiler-supported coroutines to be at least 50% faster. You guessed wrong on Flips' output size too.


In that case, I hope I'm wrong, and I'll happily jump on board with these. For a 50% performance increase, I'll be okay requiring a bleeding edge version of Clang to compile my emulator.

Quote
...what, why would that even be needed, just use the state right out of the coroutine data block, just like a hand-written state machine.


Won't that be slower than potentially shuffling things into registers or being right next to the stack pointer for the processor?

Quote
On the other hand, nested state machines would have to waste a LOT of time to be slower than libco's dismissal of the branch predictor


When I implemented libco into bsnes' CPU/SMP, where I had one state machine to select the current opcode, another to select the current cycle of said opcode, and finally one more for the half of the bus hold delay time (for reads), the speedup of libco was about 10% over the entire emulation core. Obviously there was all the other stuff (PPU took about 50% of program time alone). Another instance was the state machine in the DSP core. That's just a single switch. But to make it consistent with all other processors, I used libcco on it. That caused about an 8% total performance loss.

That's just two random anecdotal accounts, though. Narrow timing test cases are junk and not realistic. Like the one that comes with libco itself. If you can find a way to properly benchmark this, I'd be quite interested in having more than a single test case.

Quote
And even at infinite depth, they still win at portability, exception safety, and not annoying Valgrind.


And being very problematic for implementing save states, debuggers, and chips that can lock waiting on another chip to release it (eg Z80 stall for 68K CPU access on the Mega Drive.)

libco has a whole lot of downsides, I don't deny them.
Quote
bsnes-libretro translates Emulator::Interface to libretro, does that count?


So......... why is Retroarch still using v094? ;_;

Even if we're stuck with v094-mercury, I'd at least be happy to know people can use v102+. Then it's not so different than people just running old desktop versions. Can't really be helped.

Quote
I don't remember how it handles the SPC IPL ROM.


Well, Emulator::Interface will send a load() request for it pointed at folder 0 (system path). You can just hardcode it if you want and return the 64-byte array from RAM. Doesn't have to exist on disk.

Quote
I think I'm better off not seeing that.


Yeah, that's for the best >_>

It's locked now anyway due to the drama.

Quote
How many accounts does he have by now? He's got /u/squarepusher6 and the five prior, did he hit a dozen yet?


I've honestly lost count. He's going by Italian poet names currently.

Quote
But I do agree that an official bsnes-libretro port would get a bunch of users off the existing ones.


I just really don't want to maintain it. But as long as I can work with someone other than SP, I'd be happy to collaborate on a project to make an updated port. I won't merge it into my official releases, but we can maintain the bridge as a separate project on Gitlab or something.

Quote
Or fork the project and move on, but Mercury wasn't really because you were annoying me.


Yeah, even Header Magic was just me showing off some old (now useless) Win32 hacking skills. Don't really care all that much about SMW hacking. I support the copier headers anyway and will probably always have to :/

Quote
Or maybe I'm wrong.


Well, they've been receptive to it since the second generation of devs (funkyass, anomie). The third ones are too (BearOso, OV2, zones). So, good possibility if the patches are submitted, they'll accept them.

Quote
I don't think %rsp-relative addressing is faster than %rdi-relative.


Hmm, I'd guess it would be but ... maybe not if it gets into L1 cache either way.

I admit that I'm out of my depth on how modern CPUs work. Pure voodoo >_<

Quote
Doubt compiler-generated coroutines would help much with those, they're not gonna be any less opaque than the libco stacks.


Correct. All of the downsides apply to coroutines, too. Except *maybe* portability. But portability really isn't a big deal with libco. It takes like 15 minutes to make a new platform port, and I already support everything anyone sane would be using (I even run on Loongson MIPS systems. Not at 60fps, but it runs.)
Quote
And there were no real improvements to the SNES core since then


I really don't like other people deciding that for everyone else.

> There are also several improvements to SNES emulation. Most significantly is support for mid-scanline changes to the background mode in the accuracy profile.

SFC: added MSU1 audio resume support
SFC: added new expansion port device (eBoot); simulation of SNES-Boot hardware
SFC: expansion port device can now be selected from system menu
SFC: updated handling of open bus (thanks to Exophase for the design idea)

SFC: expansion port devices can now be changed during gameplay (atlhough you shouldn't)
SFC: fixed bug in SharpRTC leap year calculations
SFC: emulated new research findings for the S-DD1 coprocessor
SFC: fixed CPU emulation-mode wrapping bug with pei, [dp], [dp]+y instructions [AWJ]
SFC: fixed Super Game Boy bug that caused the bottom tile-row to flicker in games

SFC: 8% speedup over the previous release thanks to PPU optimizations
SFC: USART developer controller removed; superseded by 21fx
SFC: Super Multitap option removed from controller port 1; ports renamed 2-5
SFC: hidden option to experiment with 128KB VRAM (strictly for novelty)

new scheduler for all emulator cores (precision of 2^-127)

These may not be important to you, but ask 93143 how he feels about the dot-based PPU rendering improvements, with all the interesting demos he's working on for his new homebrew game idea.

... and I only really document probably half of the actual changes I make. I'm really bad with documentation. Screwtape's Gitlab repo has per-WIP changelogs.

Quote
(oh, and we ifdeffed out your cheat engine, just having them there gave a ~3% slowdown.)


Yeah, that's a conditional test in the hottest area of code, unfortunately.

Snes9X has a much neater system where it changes the memory once per frame instead. It's not perfect, but it works for 99%+ of cheats. Unsure how it handles disabling cheats, but I assume it caches the last value there when enabled.

Quote
And then whoever you're working with leaves or gets bored, and we're stuck on v123.


Well, that's better than v094 at least ...

Quote
And/or someone submits a pull request you don't like, but Squarepusher does, and then it's a fork.


Correct.

Look, I keep saying it, but I'll say it again ... I understand this is open source. You're fully in the right to change my code. But I'm still entitled to have my opinion. I gave out the source, so that people would hopefully be able to make use of it without changing my code. But I know full well they can and do change it. That's how FOSS works. Yes, I get it. But I don't like it, sorry. I would really like it if the emulator core weren't modified, if people were using the latest version of my codebase, and I could then accept bug reports from RA users. No, I don't have to like it. Yes, you can tell me to fuck off. But I'm going to keep complaining about it, because it bothers me.

Quote
Not replacing %rsp means compiler-supported coroutines play nice with Valgrind


If that is indeed confirmed how it works, then yes, that's a good point.

Quote
...it only takes you 15 minutes to learn a new assembly language?


The PowerPC backend written by blargg is massive and looks extremely intimidating.

But yes, it only took me about 10-15 minutes to make the ARM32 port for my Raspberry Pi. It was literally just two instructions. You really only have to learn how to load and store whatever the non-volatile registers are for the given platform. Sometimes finding the ABI documentation is difficult, too. I was glad I didn't have to write MIPS or SPARC assembly implementations, since setjmp worked well enough there.
Quote
the other half is worth caring about


And although it's my own fault for not documenting them in changelogs, that's probably only half of the actual fixes that were made.

Quote
And we'd have to find some way to deal with the removal of the balance core.


This is the main thing holding me back from just trying to make an Snes9X-like GUI as an alternative to higan, and just have it be the SNES core, and call it bsnes again. Make it load regular ROMs, have the "recently used" file list, screensaver suppression, softpatching, and other creature comforts.

The only balanced PPU core was by -far- the oldest, untouched code in higan. I tried to make the performance PPU to replace it, but it ended up being just as slow once I added mosaic support in. I still don't fully grok why mosaic support obliterated its performance speedup.

But I would've even been okay with that ... -however-, the balanced PPU emulated certain edge cases better, so it would've been a bad idea to replace the old scanline PPU with the new scanline PPU.

Maybe one day I'll try again at a fast PPU.

I still think an entirely new emulator is the best course of action, though.

Quote
...no, it's not. Neither of them is the latest version.


I'm not really an all or nothing person. Two years out of date is better than five years out of date.

Quote
That's more like it. Make walls of text great again!


You have the right to use old versions and hack them as you want.

But I also have the right to complain about it :P

Quote
Now make an Itanium port.


>__>

Point taken. Some ports are much easier than others.
Quote
We'd keep bal, but 098 > 094


Was talking to hex_usr, it looks like he's maintaining the balanced core in his nSide fork, so that'll get you v102+ with balanced. He dropped performance, but honestly you're better off using Snes9X instead of that.

Can't say he'll maintain it forever, but the changes slow down more and more over time anyway.

Quote
Pushing a git commit after every change could serve as a changelog ... You've rejected public source repos a couple of times


Screwtape's been nice enough to help with that for quite a while now.

https://gitlab.com/higan/higan/tree/master/

Each WIP commit has changes and notes:

https://gitlab.com/higan/higan/commits/master

So it's about 90% of the way to using Git myself; the only real loss is that you can't do pull requests or whatever that's called. Have to send me patches.

Quote
libretro is stuck on v094 until someone makes a new port, and I'm not aware of anyone who's interested.


Yeah, same =(

Quote
Who are you and where did you hide the real byuu?


Why, are you a bad enough dude to rescue the byuu?

But anyway, the trick to getting the old me back out again is to anger me a whole lot. But it's not recommended ;) </MajinByuu>
Quote
Isn't perf about 2% faster than bal and 2% more accurate than snes9x?


Performance is about 80% faster than balanced, and 80% slower than Snes9X v1.54.

The main limitation of Snes9X is that the main CPU is opcode-based and not cycle-based. They have a few hacks for the most important games that require cycle accuracy (well, if you don't hack them, that is.) Thus, this also applies to the SA-1. But all SA-1 emulation is really bad right now, including mine.

Quote
Let's just hope someone does check that no unexpected changes try to sneak past...


I trust Screwtape. Known him a long time.

Quote
I'm sending patches until you get on Github, I don't want 999 accounts everywhere.


When they stop threatening to ban people for using words they don't like in their source code repositories, I might consider doing something with the account Matthew Callis made for me there.

Quote
Like this?


Hi, so I know most of you prefer xkas or asar, and that's cool!

But I wanted to say for anyone using my cross-assembler, bass, that the next release is going to have some breaking changes to the old syntax.

Everything possible in older versions of bass should be possible in v15, it'll just require a few changes to keywords. For instance, renaming "scope foo {" to "namespace foo {" and "scope foo: {" to "function foo {".

My goal for v15 is to stop making these breaking changes. I can't say I will be 100% successful, but I'm going to try as hard as possible to maintain backward compatibility with v15 in all future releases. Obviously, new features can be added after, but no renaming/reworking of existing features should occur.

What that obviously means is that the final syntax choices for v15 are going to be very important.

So for anyone who uses bass, please pay attention to my v14 WIP thread on my board, and offer feedback if there's something you are unhappy with or feel will limit you in some way compared to past releases. Best to proide feedback on my board if you can, as that's where most of the discussion is happening and I'd prefer not to fragment it.

Once v15 ships, if bass users are unhappy with the syntax, it'll be a real problem, so ... speak now or forever hold your peace and all that ^-^;

Forum link: https://board.byuu.org/viewtopic.php?p=45003#p45003

Thanks in advance!
> You haven't been paying attention, have you?

I didn't come here for this.

> It is indeed kinda out of place here too

Oh, sorry about that. Saw the other mentions of assemblers so I thought this was right. Please feel free to move the topic wherever you see fit.

> Why is everything so verbose?

Personal preference. I spend five times as long planning my code as I do typing it.

> Would you rather type 'integer', 'boolean' ... all over your C++?

... you probably shouldn't look at my C++ code.

> I can't keep track of how >^< map to .b .w .l.

You can use .b .w .l; or make defines to say {byte}=<, {word}=>, {long}=^. It actually works kinda neat with the defines.

> Why do I need to know about lorom/hirom (or copypaste a macro written by someone who knows about lorom/hirom) to write a proper org statement?

Because there's more than lorom and hirom, and bass supports more than just the SNES (there's a fork which has about 20 different CPU tables for it.)

> To us, Asar's freespace finder is completely invaluable.

Different use cases.

I think that's a really cool feature you added to Asar, though.
Quote
So how do I know if the SA-1 support actually works?


I have much of the same problem. Hard to test my SA-1 code against hardware. There's some RAM on some cartridges that may work, but I need more time. Should not have tried to emulate 12 systems, but I did, so ...

Quote
You'll have to wait for the only one that's programmable (the SD2SNES) to have a firmware update and boy that's going to be in a long time, unfortunately.


It will also be an emulator even if it does manage to add support, so you still haven't answered if it will work on real hardware or not. Only method there is reproduction carts that replace mask ROMs from official SA1 games.

Quote
see Ladida's recent troubles with HDMA and the VWF Level Intros patch


I did fix that, at least. Hopefully fixed it. So far no regressions reported.

Quote
The fact that the SA-1 is full of undocumented features (or so I've heard) doesn't help, either.


The hard part is it has nearly a dozen features that not a single licensed game ever uses. Emulation of those features is totally untested right now, and I'm pretty sure only bsnes/higan even tries to emulate them.

Quote
we can only hope that emulators such as bsnes are close enough to the real deal


I really, really wouldn't count on it in this case, sorry.

The SA-1 is the least accurately emulated of everything.
I added an option to my emulator to use shadow echo RAM, which prevents the ear-torture and crashing of the earlier bugged Addmusic versions and allows one to run previously ZSNES-only ROM hacks.

The downside is that people have to know to enable the option, and it's not good to just always leave it enabled as it may interfere with other games.

I would like it to be automatic, so I wanted to pose a question / challenge to this board: is there a pattern of bytes, say from the Addmusic ASM code, that will only appear in bugged versions of Addmusic, but not in corrected versions?

If such a pattern exists, then I can automatically enable the shadow echo RAM only for the older Addmusic hacks, and leave it off everywhere else.

Barring this, the only other option is likely a database of SHA256 ROM hashes for games to opt-in to the echo shadow RAM, but that seems like a substantially more difficult challenge, given there's probably hundreds of these that would need to be cataloged and indexed. But presuming that turns out to be the only way, would anyone here be up for making such a list? Just need it to be a CSV file in the format of "SHA256,Hack Name", and I can take it from there.

Thanks!