 |
|
 |
|
| SMW/ROMhacking tool related code library |
|
Forum Index - Hobbies - Computers & Technology - Programming - SMW/ROMhacking tool related code library |
|
Pages:  1 2  |
|
|
|
| Posted on 2009-04-25 09:33:07 PM |
Link | Quote |
|
This is a thread used for the archiving of programming code that can be used in the development of SMW tools.
Unfortunately, the only language I know is C#.net, so I can only contribute C#.net code.
I'll update this post with any new code that anyone posts here.
Also, there's nothing I can do about the table stretch... D:
If such a thing exists for the programming forum, I'd like to request for sticky.
------------C#.NET---------------
1. Getting ROM data in to a byte array
This code actually allows you to get the ROM data so that you can change it.
Variables:
ROMname: A string that contains the filepath of the ROM.
Codebyte[] _rombuffer = File.ReadAllBytes(ROMname);
Thanks to Morton for the simpler version.
2. Writing data to a ROM file
This code will take a byte array and write it to a ROM file.
Variables:
ROMname: A string that contains the filepath of the ROM.
ROMbyte: A byte array containing the contents of the ROM.
CodeFile.WriteAllBytes(ROMname, ROMbyte);
3. Patching an IPS patch
This code patches an IPS patch.
Variables:
romname: A string containing the filepath of the ROM.
patchname: A string containing the filepath of the patch.
Code FileStream romstream = new FileStream(romname, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite);
FileStream ipsstream = new FileStream(patchname, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite);
int lint = (int)ipsstream.Length;
byte[] ipsbyte = new byte[ipsstream.Length];
byte[] rombyte = new byte[romstream.Length];
IAsyncResult romresult;
IAsyncResult ipsresult = ipsstream.BeginRead(ipsbyte, 0, lint, null, null);
ipsstream.EndRead(ipsresult);
int ipson = 5;
int totalrepeats = 0;
int offset = 0;
bool keepgoing = true;
//////////////////End Init code
//////////////////Start main code
while (keepgoing == true)
{
offset = ipsbyte[ipson] * 0x10000 + ipsbyte[ipson + 1] * 0x100 + ipsbyte[ipson + 2];
ipson++;
ipson++;
ipson++;
/////////////split between repeating byte mode and standard mode
if (ipsbyte[ipson] * 256 + ipsbyte[ipson + 1] == 0)
{
////////////repeating byte mode
ipson++;
ipson++;
totalrepeats = ipsbyte[ipson] * 256 + ipsbyte[ipson + 1];
ipson++;
ipson++;
byte[] repeatbyte = new byte[totalrepeats];
for (int ontime = 0; ontime < totalrepeats; ontime++)
repeatbyte[ontime] = ipsbyte[ipson];
romstream.Seek(offset, SeekOrigin.Begin);
romresult = romstream.BeginWrite(repeatbyte, 0, totalrepeats, null, null);
romstream.EndWrite(romresult);
ipson++;
}
else
{
////////////standard mode
totalrepeats = ipsbyte[ipson] * 256 + ipsbyte[ipson + 1];
ipson++;
ipson++;
romstream.Seek(offset, SeekOrigin.Begin);
romresult = romstream.BeginWrite(ipsbyte, ipson, totalrepeats, null, null);
romstream.EndWrite(romresult);
ipson = ipson + totalrepeats;
}
/////////////Test For "EOF"
if (ipsbyte[ipson] == 69 && ipsbyte[ipson + 1] == 79 && ipsbyte[ipson + 2] == 70)
keepgoing = false;
}
romstream.Close();
ipsstream.Close();
-----------Ruby 1.9-------------
Linky
|
| Last edited on 2009-08-17 11:48:34 AM by Ersanio. |
|
| Posted on 2009-04-26 05:56:28 PM |
Link | Quote |
|
Not to sound like a smartass or something, but there is an easier way to load a ROM into a byte array. It's just 1 line of code. Assume that path is the filepath to the ROM.
Codebyte[] _rombuffer = File.ReadAllBytes(path);
|
|
| Posted on 2009-04-26 07:39:47 PM |
Link | Quote |
|
Originally posted by MortonNot to sound like a smartass or something, but there is an easier way to load a ROM into a byte array. It's just 1 line of code. Assume that path is the filepath to the ROM.
Codebyte[] _rombuffer = File.ReadAllBytes(path);
Oh. I had no idea about that, but thanks! Updated that method and also added File.WriteAllBytes to the list.
I'm also working on the code to create an IPS patch, if anyone cares.
|
|
| Posted on 2009-04-27 01:08:55 AM |
Link | Quote |
|
|
Reading/writing the entire file at once is not a good method for most programs. It's slow (lots of unnecessary disk I/O), uses more RAM than it needs to, and if you have the file also open in another program (e.g. hex editor) and change something completely unrelated to what you're working on, the change gets wiped out.
|
|
| Posted on 2009-05-03 11:03:42 PM |
Link | Quote |
|
I have edited this post from the original version. Revision log follows.- (2 July 2009) Added Makefile section with code to run xkas to transform .s files into .smc files.
- (5 June 2009) Show output from newer snes-rom.rb that includes version; link to wiki page about SNES header.
- (3 June 2009) Add find-runs.rb to find free space, or runs of repeated bytes.
- (31 May 2009) Added a public domain dedication; added Subversion notice; added ruby-ips-2.rb and snes-rom.rb to the post; removed the chat about async I/O from the top of the post; replaced the bottom of the post with a short note below Ruby example 2.
- (4 May 2009) Posted the original version.
Dedication to the public domain: I intend to allow the general public to use my contributions to this SMW-tool-related code library. I place the source code, that I write and embed into this post, into the public domain. Therefore, you do not need a copyright license from me to use this source code.
Subversion: My SVN repository http://opensvn.csie.org/kernigh/trunk/examples/ruby-snes/ holds some files that I link from this post. I may replace these files with newer revisions.
------------Ruby 1.9---------------
Ruby language is not popular like C#, but I use Ruby with SMW. Kaizo and Ersanio have three pieces in C#, I can redo them in Ruby 1.9.
1. Getting ROM data in to a byte array
This is the code:
Coderombuffer = open(path, "rb") { |f| f.read }
This is an example to use my code at the interactive Ruby prompt:
Codeirb(main):001:0> path = "smw-clean.smc"
=> "smw-clean.smc"
irb(main):002:0> rombuffer = open(path, "rb") { |f| f.read }; 0
=> 0
irb(main):003:0> rombuffer.length
=> 524800
irb(main):004:0> rombuffer.index "SUPER"
=> 33216
irb(main):005:0> rombuffer[33216, 21]
=> "SUPER MARIOWORLD "
2. Writing data to a ROM file
This is the code:
Codeopen(path, "wb") { |f| f.write(rombuffer) }
This continues my example:
Codeirb(main):006:0> rombuffer[33216, 21] = "Super Ruby 1.9 World "
=> "Super Ruby 1.9 World "
irb(main):007:0> path = "smw-ruby.smc"
=> "smw-ruby.smc"
irb(main):008:0> open(path, "wb") { |f| f.write(rombuffer) }
=> 524800
Now if emulator plays smw-ruby.smc, then the title of ROM is "Super Ruby 1.9 World " (and the checksum is bad).
(31 May) This example was slow and wasteful because I used code that reads and writes the entire file, but I only needed to seek and write 21 bytes. HyperHacker (in the post before this post) explained, "Reading/writing the entire file at once is not a good method for most programs."
3. Patching an IPS patch (ruby-ips-2.rb)
Old version: ruby-ips.rb
(31 May 2009) New version: ruby-ips-2.rb (svn)
Around March 2009, I wrote ruby-ips.rb and began to use it as my normal IPS patcher before I play SMW hack. This program has two known bugs: the error messages at line 115 does not work, and Emacs does not like "-*- encoding: US-ASCII -*-" at line 2.
Around C3 of May 2009, I wrote ruby-ips-2.rb and began to use it as my normal IPS patcher. This second version fixes the known bugs and also adds -o option to copy clean ROM before apply patch. Now I do not need to copy my clean SMW ROM before I start patcher.
From command line, to apply patch.ips to rom.sfc, modifying rom.sfc:
Code$ ./ruby-ips-2.rb -f rom.sfc patch.ips
To copy smw-clean.smc to smtko-demo1.smc then apply smtko_demo_1.ips:
Code$ ./ruby-ips-2.rb -o smtko-demo1.smc -f smw-clean.smc smtko_demo_1.ips
This is the main loop to process the patch, where patch is the IO for the patch, file is the IO for the ROM, pread.call(3) is a lambda to read 3 bytes from patch but raise an error if there are less than 3 bytes, unpack2 and unpack3 are methods to convert bytes to numbers, hex is method to convert number to ASCII.
Code # process each record
while true
offset = pread.call(3)
# "EOF" marks the end of an IPS patch, but check if there are
# bytes in this patch after the "EOF".
#
if offset == "EOF"
truncation = patch.read(4)
if truncation == nil
puts "\tend of patch" if list_records
elsif truncation.length == 3
# truncate the file, as does Lunar IPS
truncation = unpack3(truncation)
if list_records
puts "\tend of patch, truncation at #{hex(truncation)}"
end
file.truncate(unpack3(truncation)) if filename
else
raise "unexpected data after \"EOF\" in patch\n" +
"problem with offset \"EOF\" #{hex(0x454f46)}?"
end
# break from 'while true' loop
break
end
offset = unpack3(offset)
length = unpack2(pread.call(2))
case length
when 0
# this patch record uses run-length encoding (RLE)
length = unpack2(pread.call(2))
byte = pread.call(1)
if list_records
puts "\toffset #{hex(offset)}, length #{hex(length)}, " +
"RLE #{hex(byte.ord)}"
end
if filename
file.pos = offset
file.write(byte * length)
end
else
# this is a normal patch record
if list_records
puts "\toffset #{hex(offset)}, length #{hex(length)}"
end
if filename
file.pos = offset
file.write pread.call(length)
else
patch.pos += length
end
end # case length
end # while true
4. ROM expansion with clean ROM check
I have some code to expand a ROM and compute the checksum, but I cut and derived the code from a longer Ruby program that I wrote for my SMW hack, so some variable names might be strange.
This code makes an expanded copy of a ROM and also checks that the original ROM is clean Super Mario World. The clean ROM must be headerless or have a valid SMC header. CAVEAT: the expanded ROM will be headerless. (I am not sure how to make the header, because I noticed that SMW hacks have invalid SMC headers. I need to learn more about SMC headers.) Also, the expanded ROM will have a bad checksum.
This code might not be ready for the code library, because I would need to give SMC header to expanded ROM, and to separate the ROM expansion and the clean ROM check into two pieces of code.
Change clean_rom to path to clean ROM. Change expanded_rom to path to expanded ROM. Change Expanded_banks to 32 for 1 MB, 64 for 2 MB, or 128 for 4 MB. Other numbers from 17 through 127 will play in snes9x, but the ROM size in the Super NES header will wrong, because it will be the next power of 2.
Coderequire 'digest/sha1'
clean_rom = "smw-ruby.smc"
expanded_rom = "expanded.sfc"
# each lorom bank has 0x8000 bytes
SMW_banks = 16
Expanded_banks = 128
# SHA1 of the clean SMW ROM (all 16 banks, without SMC header)
SMW_clean = "6b47bb75d16514b6a476aa0c73a683a2a4c18765"
smw = nil
rom = nil
begin
# open SMW ROM image for reading; seek past any SMC header
smw = open(clean_rom, "rb")
size = smw.stat.size
goal = SMW_banks * 0x8000
if size == goal
# headerless ROM
elsif size >= (goal + 512)
# check SMC header
size = smw.read(2).unpack("v")[0] * 0x2000
if size == goal
# seek past header
smw.pos = 512
else
raise "#{clean_rom}: wrong size in SMC header"
end
else
raise "#{clean_rom}: wrong size"
end
rom = open(expanded_rom, "wb")
# copy banks
check = Digest::SHA1.new
SMW_banks.times do
bank = smw.read(0x8000)
check.update(bank)
rom.write(bank)
end
smw.close
unless check.hexdigest == SMW_clean
raise "${clean_rom}: not a clean SMW ROM"
end
# expand ROM
bank = "\xff" * 0x8000
(Expanded_banks - SMW_banks).times do
rom.write(bank)
end
# Write ROM size to Super NES header. If the number of
# banks is not a power of 2, then round upward.
#
byte = (Math.log2(Expanded_banks) + 5).ceil.to_i
rom.pos = 0x7fd7
rom.write([byte].pack "C")
rom.close
rescue Exception => e
smw.close if smw and not smw.closed?
rom.close if rom and not rom.closed?
File.delete expanded_rom # delete bad file
raise e
end
5. Recompute checksum in Super NES header
This is a good programming exercise. This code recomputes the checksum, but my version only works with LoROM (like Super Mario World) and only if the ROM contains a whole number of banks. If the number of banks is not a power of 2, then my code will mirror the banks, so that the checksum equals what snes9x would compute.
(Lunar Magic seems to adjust some bytes so that the ROM has the same checksum as before, instead of recomputing the checksum.)
Change target_rom to path to ROM.
Codetarget_rom = "expanded.sfc"
io = nil
begin
io = open(target_rom, "rb+")
size = io.stat.size
bank_count = size / 0x8000
case size % 0x8000
when 0
base = 0 # no SMC header
when 512
base = 512 # SMC header
else
raise "#{rom}: wrong size"
end
# Compute checksum in Super NES header. Use the formula
# from wlalink/compute.c of WLA DX, which is the same
# as that from memmap.c of Snes9x ("from NSRT").
#
io.pos = base + 0x7fdc
io.write([0xffff, 0x0000].pack "vv")
# compute checksum of each bank
banksum = []
io.pos = base
bank_count.times do |bank|
banksum[bank] = io.read(0x8000).sum(16)
end
# compute checksum of all banks
checksum = 0
(2 ** Math.log2(bank_count).ceil).times do |bank|
# handle the mirror banks
bank >>= 1 while bank >= banksum.length
checksum += banksum[bank]
end
checksum &= 0xffff
# write checksum
io.pos = base + 0x7fdc
io.write([checksum ^ 0xffff, checksum].pack "vv")
ensure
io.close
end
6. Examine the Super NES header (snes-rom.rb)
(31 May 2009) Full source code: snes-rom.rb (svn)
My snes-rom.rb is a program to print the information from the SNES header.
Code$ ./snes-rom.rb
Usage: snes-rom [options] rom...
-l List all candidate headers
The Super NES header consists of 64 bytes inside the ROM at SNES address 0x00ffc0. The SNES header contains information like the checksum and the interrupt vectors. (5 June 2009) For more information, see a wiki page about the SNES header.
My snes-rom.rb tries to pick the best SNES header from four possible locations, and print the information from this header. (My scoring system to pick a header seems to work for my ROM images, but is more primitive than the scoring system in bsnes, mess or snes9x.)
Here is the info from when I run snes-rom.rb with my Super Mario World ROM.
Code$ ./snes-rom.rb smw-clean.smc
smw-clean.smc:
SMC header: yes
SNES header: offset 0x81c0, score: 9/9
name: "SUPER MARIOWORLD " (21 bytes)
status: 0x20 (slow LoROM)
cartridge type: 0x02 (ROM and save-RAM)
ROM size: 0x09 (512 kilobytes)
RAM size: 0x01 (2 kilobytes)
country: 0x01 (NTSC)
licensee: 0x01 (Nintendo)
version: 0x00
complement/checksum: 0x5f25/0xa0da
interrupt handlers (native mode):
COP BRK ABORT NMI UNUSED IRQ
0x82c3 0xffff 0x82c3 0x816a ------ 0x8374
interrupt handlers (emulation mode):
COP UNUSED ABORT NMI RESET IRQBRK
0x82c3 ------ 0x82c3 0x82c3 0x8000 0x82c3
Here is the part of the code that unpacks the header.
Codeclass SnesHeader
attr_reader :name, :status, :cartridge_type, :rom_size, :ram_size
attr_reader :country, :licensee, :version, :ckcom, :cksum
attr_reader :nvector, :evector
class Vector
attr_reader :cop, :brk, :abort, :nmi, :reset, :irq
alias_method :irqbrk, :irq
def initialize(array)
@cop, @brk, @abort, @nmi, @reset, @irq = array
end
end
def initialize(string)
a = string.unpack "a21CCCCCCCvva4vvvvvva4vvvvvv"
@name = a[0]
@status = a[1]
@cartridge_type = a[2]
@rom_size = a[3]
@ram_size = a[4]
@country = a[5]
@licensee = a[6]
@version = a[7]
@ckcom = a[8]
@cksum = a[9]
@nvector = Vector.new a[11..16]
@evector = Vector.new a[18..23]
end
end
Here is an example after I pasted the above code into the interactive Ruby prompt:
Codeirb(main):032:0> s64 = open("smw-clean.smc") { |f| f.pos = 0x81c0; f.read(64) }
=> "SUPER MARIOWORLD \x02\t\x01\x01\x01\x00%_\xDA\xA0\xFF\xFF\xFF\xFF\xC3\x
82\xFF\xFF\xC3\x82j\x81\x00\x80t\x83\xFF\xFF\xFF\xFF\xC3\x82\xC3\x82\xC3\x82\xC3
\x82\x00\x80\xC3\x82"
irb(main):033:0> h = SnesHeader.new s64
=> #<SnesHeader:0x2b9f5270 @name="SUPER MARIOWORLD ", @status=32, @cartridge
_type=2, @rom_size=9, @ram_size=1, @country=1, @licensee=1, @version=0, @ckcom=2
4357, @cksum=41178, @nvector=#<SnesHeader::Vector:0x2b9f51b0 @cop=33475, @brk=65
535, @abort=33475, @nmi=33130, @reset=32768, @irq=33652>, @evector=#<SnesHeader:
:Vector:0x2b9f5180 @cop=33475, @brk=33475, @abort=33475, @nmi=33475, @reset=3276
8, @irq=33475>>
irb(main):034:0> printf "complement/checksum: 0x%x/0x%x\n", h.ckcom, h.cksum
complement/checksum: 0x5f25/0xa0da
=> nil
irb(main):035:0> printf "reset handler: 0x%06x\n", h.evector.reset
reset handler: 0x008000
=> nil
In the above example, I looked at the SNES header of Super Mario World. I found that h.ckcom is 0x5f25 and h.cksum is 0xa0da, which matches what I see in the emulator. The reset handler is at SNES address 0x008000.
7. Find free space, or runs of repeated bytes (find-runs.rb)
(3 June 2009) Full source code: find-runs.rb (svn)
To find free space in Super Mario World, one must look for runs of repeated bytes 0xff. There might be other tools to look for free space, but I wrote a new tool.
Code$ ./find-runs.rb
Usage: find-runs [options] file...
-b HEX_BYTE Look for HEX_BYTE (default ff)
-s Sort by length of run
-t THRESHOLD Require THRESHOLD bytes per run
For a clean headered SMW ROM, here are areas of free space of at least 500 bytes, sorted by length:
Code$ find-runs -st 500 smw-clean.smc
smw-clean.smc:
offset 0x7f190: run of 4208 bytes
offset 0x772f0: run of 3856 bytes
offset 0x37738: run of 2760 bytes
offset 0x3e96e: run of 2194 bytes
offset 0x34b63: run of 1693 bytes
offset 0x1bc02: run of 1534 bytes
offset 0x3a378: run of 1160 bytes
offset 0x2de46: run of 954 bytes
offset 0x1e25c: run of 932 bytes
offset 0x3fe90: run of 880 bytes
offset 0x2713e: run of 834 bytes
offset 0x5ff0c: run of 756 bytes
offset 0x6f28a: run of 630 bytes
offset 0x223b6: run of 586 bytes
offset 0x1ffe0: run of 544 bytes
My code might be too slow, because it requires 2 to 3 seconds of my computer time to look through a file of only 512 kilobytes (like Super Mario World). The first version required 8 to 10 seconds, before I changed file.read(1) to file.getc. This change caused Ruby to fill a buffer, instead of doing one system call per byte.
Here is the code to find the runs, where file is opened to read in binary mode, free_byte is the byte to look for, threshold is the minimum length to report.
Code while char = file.getc
offset += 1
if char.ord == free_byte
# save the offset of this first byte
first_offset = offset
run_length = 1
# look for more bytes
catch(:end_of_run) do
while char = file.getc
offset += 1
if char.ord == free_byte
run_length += 1
else
throw :end_of_run
end
end
end
if run_length >= threshold
# ... code to report this run ...
end
end
end
------------Makefile---------------
This section is for the make(1) tool from BSD, GNU/Linux and Unix systems.
1001. Run xkas to transform .s files into .smc files
I have some xkas patches with the .s suffix. I want to test each patch with a clean ROM of Super Mario World. I want a different ROM for each patch. I also want to use the save-RAM with "all exits (Star 96)" from SMWC's Official SRAM/Save State Archive, so that I can test any level.
This Makefile copies the clean ROM, runs xkas to apply to patch, and copies the save-RAM. Here is the entire Makefile.
CodeROM=../roms/smw-clean.smc
SRM=../distfiles/clean_smw.SRM
XKAS=xkas
all:
@for i in *.s; do \
if test -e "$${i}"; then \
j="$${i%.s}.smc"; \
make "$${j}"; \
fi; \
done
.SUFFIXES: .smc .s
.s.smc:
@rm -f "$@"
@cat < "${ROM}" > "$@"
xkas "$@" "$<"
@i="$@"; j="$${i%.smc}.srm"; cat < "${SRM}" > "$${j}"
clean:
@for i in *.smc; do \
if test -e "$${i}"; then \
j="$${i%.smc}.srm"; \
echo rm -f \""$${i}"\" \""$${j}"\"; \
rm -f "$${i}" "$${j}"; \
fi; \
done
Attention! I run xkas v0.12, so I need to use ''xkas "$@" "$<"''. Some other SMW hackers run xkas v0.06, which reverses the order of the arguments, so they would need to use ''xkas "$<" "$@"''.
If I run make force-balloon-up.smc, then this Makefile copies the clean ROM to force-balloon-up.smc, applies the patch force-balloon-up.s to the new ROM, and also copies the "all exits (Star 96)" save-RAM to force-balloon-up.srm.
If I edit force-balloon-up.s and run again make force-balloon-up.smc, then this Makefile discards the ROM and the save-RAM, starts again from the clean ROM, and creates again the ROM and the save-RAM.
If I run make clean, then this Makefile deletes every .smc file and every matching .srm file in this directory. (I must put the clean ROM in a different directory.) If I run make, then this Makefile creates the ROM and the save-RAM for every .s patch in this directory.
|
| Last edited on 2009-07-02 05:08:24 PM by Kernigh. |
|
| Posted on 2009-05-04 07:33:49 PM |
Link | Quote |
|
I thank you for the giant hunk of code, Kernigh.
Linked to it in the first post.
Originally posted by HyperHackerReading/writing the entire file at once is not a good method for most programs. It's slow (lots of unnecessary disk I/O), uses more RAM than it needs to, and if you have the file also open in another program (e.g. hex editor) and change something completely unrelated to what you're working on, the change gets wiped out.
If that's a problem, then you can use this code:
Variables:
My_Rom_Path: a string containing the filepath of the ROM.
My_Offset: an integer containing the offset at the start of the read/write.
My_Length: an integer containing the number of bytes to read/write from My_Offset.
Code FileStream MyStream = new FileStream(My_Rom_Path, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite);
byte[] Rom_array = new byte[My_Length];
MyStream.Read(Rom_array, My_Offset, My_Length);
//Insert code that edits the ROM here...
MyStream.Write(Rom_array, My_Offset, My_Length);
MyStream.Close();
|
|
| Posted on 2009-05-05 06:31:13 PM |
Link | Quote |
|
Originally posted by KaizoI thank you for the giant hunk of code, Kernigh.
Linked to it in the first post.
Because you linked the first post to my post, I will edit my post when I add more code.
My next code is not ready. I wrote some code that would read *.bin/*.pal files like ExGFX. To check my code, I saved the graphics in PPM format, a simple image format. Then I can open the PPM image in Krita. If I want to post the image here, then Krita can scale by 300% and save as PNG for upload to TinyPic.
Using my current code, here is the conversion for ExGFXFFB.bin and smw4.pal from SMB3 Overworld Stuff:
The colors are wrong! This is actually my second image, after the first image was too dark. I like this image because I can already see number signs, a toad house, a fortress, a destroyed castle, a pipe. My problem is my wrong use of the palette data. I might soon have a fix.
|
|
| Posted on 2009-05-17 07:55:04 PM |
Link | Quote |
|
grr... I've been working on code to create an IPS patch for a long time now, and I STILL CAN'T GET IT!
What happens is that it takes about a minute and a half to make the patch, and then when I try to patch it, LIPS gives me an error.
Here's the current code:
Variables:
Rom: A byte array with the un-clean ROM in it.
Crom: A byte array with the clean ROM, the same length as the un-clean ROM.
Code byte[] finpatch =
{
0x50, 0x41, 0x54, 0x43, 0x48
};
int repeats = 0;
int offset = 0;
byte[] vars = new byte[5];
for (int onbyte = 0; onbyte < crom.Length; onbyte++)
{
if (rom[onbyte] != crom[onbyte])
{ //If the two bytes are different, go into long thing.
offset = onbyte;
for (repeats = 0; (rom[onbyte + repeats] != crom[onbyte + repeats]); repeats++)
{
//find out how many repeats there are
}
//Can't be more than FFFF bytes long!
if (repeats > 0xFFFF)
repeats = 0xFFFF;
byte[] temp = new byte[repeats];
//For loop assemballing temp array
for (int myint = 0; myint < repeats; myint++)
{
temp[myint] = rom[onbyte];
onbyte++;
}
//Prepare vars variables
string repeats_s = string.Format("{0:X}", repeats);
while (repeats_s.Length < 4)
repeats_s = "0" + repeats_s;
string offset_s = string.Format("{0:X}", offset);
while (offset_s.Length < 6)
offset_s = "0" + offset_s;
//Assemble Vars
if (bytesame(temp) && temp.Length > 4)
{
//repeating byte mode
vars = new byte[7];
vars[0] = (byte)((stringtohex(offset_s.Substring(0, 1)) * 0x10) + stringtohex(offset_s.Substring(1, 1)));
vars[1] = (byte)((stringtohex(offset_s.Substring(2, 1)) * 0x10) + stringtohex(offset_s.Substring(3, 1)));
vars[2] = (byte)((stringtohex(offset_s.Substring(4, 1)) * 0x10) + stringtohex(offset_s.Substring(5, 1)));
vars[3] = 0;
vars[4] = 0;
vars[5] = (byte)((stringtohex(repeats_s.Substring(0, 1)) * 0x10) + stringtohex(repeats_s.Substring(1, 1)));
vars[6] = (byte)((stringtohex(repeats_s.Substring(2, 1)) * 0x10) + stringtohex(repeats_s.Substring(3, 1)));
byte thebyte = temp[0];
temp = new byte[1];
temp[0] = thebyte;
}
else
{
//standard mode
vars = new byte[5];
vars[0] = (byte)((stringtohex(offset_s.Substring(0, 1)) * 0x10) + stringtohex(offset_s.Substring(1, 1)));
vars[1] = (byte)((stringtohex(offset_s.Substring(2, 1)) * 0x10) + stringtohex(offset_s.Substring(3, 1)));
vars[2] = (byte)((stringtohex(offset_s.Substring(4, 1)) * 0x10) + stringtohex(offset_s.Substring(5, 1)));
vars[3] = (byte)((stringtohex(repeats_s.Substring(0, 1)) * 0x10) + stringtohex(repeats_s.Substring(1, 1)));
vars[4] = (byte)((stringtohex(repeats_s.Substring(2, 1)) * 0x10) + stringtohex(repeats_s.Substring(3, 1)));
}
//put it all together
finpatch = appendarray(appendarray(finpatch, vars), temp);
}
}
byte[] EOF =
{
0x45, 0x4f, 0x46
};
appendarray(finpatch, EOF);
Here's some of the functions I call in that code.
Code private int stringtohex(string input)
{
switch (input)
{
case "0":
return 0;
case "1":
return 1;
case "2":
return 2;
case "3":
return 3;
case "4":
return 4;
case "5":
return 5;
case "6":
return 6;
case "7":
return 7;
case "8":
return 8;
case "9":
return 9;
case "A":
return 0xA;
case "B":
return 0xB;
case "C":
return 0xC;
case "D":
return 0xD;
case "E":
return 0xE;
case "F":
return 0xF;
}
return 0x10;
}
private byte[] appendarray(byte[] partone, byte[] parttwo)
{
MemoryStream appendstream = new MemoryStream();
appendstream.Write(partone, 0, partone.Length);
appendstream.Write(parttwo, 0, parttwo.Length);
byte[] endbyte = new byte[partone.Length + parttwo.Length];
appendstream.Seek(0, SeekOrigin.Begin);
appendstream.Read(endbyte, 0, endbyte.Length);
appendstream.Close();
return endbyte;
}
private bool bytesame(byte[] bytearraytocheck)
{
for (int myint = 1; myint < bytearraytocheck.Length; myint++)
{
if (bytearraytocheck[myint] != bytearraytocheck[myint - 1])
return false;
}
return true;
}
|
|
| Posted on 2009-05-23 07:54:17 PM |
Link | Quote |
|
Originally posted by KaizoWhat happens is that it takes about a minute and a half to make the patch...
Your code is much too slow! You might want more speed before you fix the other problems with your code. So I tried to check your code for slowness.
Your function appendarray might be the major source of slowness, because it always copies the array into a new byte[...]. For example, smtko_demo_1.ips (from Super Mario TKO (demo 1)) has a size of about 994K. If your code would make this patch, then finpatch would contain 497K on average. To make each patch record (in repeating-byte mode or standard mode), you twice call finpatch. This TKO patch has about 2500 records, so you would 5000 times call finpatch. You would copy about 497K times 5000 equals about 2427 MB. I conclude that your code would copy about 2.37 GB between memory locations to generate a 994K patch for a 4 MB ROM.
You would need to replace byte[] finpatch with a smart array that keeps extra capacity and can grow larger without making 5000 extra copies. I am no C# hacker, but I wandered around Microsoft's documentation, and I guess that System.Collections.Generic.List<byte> is a smart array. The method AddRange might append another array to a smart array, like so:
Code// This might not be valid C# code...
System.Collections.Generic.List<byte> finpatch = ...;
byte[] stuffToAppend = ...;
finpatch.AddRange(
new System.Collections.Generic.List<byte>(stuffToAppend));
Instead of System.Collections.Generic.List, I think that MemoryStream might also work, because Microsoft says that MemoryStream is resizable. You already use MemoryStream, but I mean to use MemoryStream instead of byte[] for finpatch, like so:
Code// This might not be valid C# code...
MemoryStream finpatch = new MemoryStream();
byte[] stuffToAppend = ...;
finpatch.Write(stuffToAppend, 0, stuffToAppend.Length);
Use something like System.Collections.Generic.List or MemoryStream instead of your slow function appendarray, and I hope that your code will not be slow.
I tried to check the remainder of your code, but (possibly because I know not C#) I am not sure why it makes an incorrect IPS patch. I only found one possible problem, in here:
Code// Kaizo's code
for (repeats = 0; (rom[onbyte + repeats] != crom[onbyte + repeats]); repeats++)
{
//find out how many repeats there are
}
//Can't be more than FFFF bytes long!
if (repeats > 0xFFFF)
repeats = 0xFFFF;
So this code loops until rom[onbyte + repeats] == crom[onbyte + repeats]. If the unequal bytes repeat to the end of the array, then I suppose that the array accesses will become out of bounds. You would need to add a check that onbyte + repeats is less than the length of the arrays (unless you already have this check).
Other than this, I failed to find your problem. Maybe another user (perhaps someone who knows C#) can find the problem.
If you upload an IPS patch made from your code, then I might run the IPS patch through my patcher ruby-ips-2.rb and try to investigate why the patch is bad.
|
|
| Posted on 2009-05-27 07:38:38 PM |
Link | Quote |
|
*FacePalms*
How could I possibly forget about MemoryStream?
...
So, I took your advice and used a MemoryStream instead of appendarray, and you were right! That was the source of the slowdown! The entire code runs in less that a split second.
Originally posted by KernighI tried to check the remainder of your code, but (possibly because I know not C#) I am not sure why it makes an incorrect IPS patch. I only found one possible problem, in here:
Code
// Kaizo's code
for (repeats = 0; (rom[onbyte + repeats] != crom[onbyte + repeats]); repeats++)
{
//find out how many repeats there are
}
//Can't be more than FFFF bytes long!
if (repeats > 0xFFFF)
repeats = 0xFFFF;
So this code loops until rom[onbyte + repeats] == crom[onbyte + repeats]. If the unequal bytes repeat to the end of the array, then I suppose that the array accesses will become out of bounds. You would need to add a check that onbyte + repeats is less than the length of the arrays (unless you already have this check).
I already considered that possibility; I have implemented a check that breaks the loop if onbyte + repeats is greater than or equal to the length of the arrays.
Also, before, the patch wouldn't patch by LIPS, and now, LIPS patches it, it runs in an emulator, AND it opens in LM. The only problem is that when patched, there is a bad checksum, as where the one made by LIPS doesn't.
So, thanks!
|
|
| Posted on 2009-05-30 10:31:37 PM |
Link | Quote |
|
Originally posted by KaizoAlso, before, the patch wouldn't patch by LIPS, and now, LIPS patches it, it runs in an emulator, AND it opens in LM. The only problem is that when patched, there is a bad checksum, as where the one made by LIPS doesn't.
The two patched ROM images would must be equal at every byte. You might want to find which bytes are different, because I think that the different checksum implies that part of the ROM is corrupt.
Meanwhile, I made some progress with ppm-from-snes.rb, but I am not ready to post this program, which would convert graphics from SNES 4bpp (*.bin/*.pal/*.tpl) to PPM (*.ppm) format. My program is not ready because I want to add code for MAP16. I guess that there is some way to use MAP16 to match tiles with palettes.
I have readied two other programs, and edited my earlier post to include them. The first program is ruby-ips-2.rb (svn), a second version of my program to apply IPS patches. I made this version during C3 of May 2009. (I did not present the code to C3.) I added a feature to copy clean ROM before apply patch, so that I need not copy my clean SMW ROM before I run patcher. This helps because I obtained 19 different IPS patches at C3.
The second program is snes-rom.rb (svn), to print information about the internal SNES header in each ROM, like so:
Codesmw-clean.smc:
SMC header: yes
SNES header: offset 0x81c0, score: 9/9
name: "SUPER MARIOWORLD " (21 bytes)
status: 0x20 (slow LoROM)
cartridge type: 0x02 (ROM and save-RAM)
ROM size: 0x09 (512 kilobytes)
RAM size: 0x01 (2 kilobytes)
country: 0x01 (NTSC)
licensee: 0x01 (Nintendo)
complement/checksum: 0x5f25/0xa0da
interrupt handlers (native mode):
COP BRK ABORT NMI UNUSED IRQ
0x82c3 0xffff 0x82c3 0x816a ------ 0x8374
interrupt handlers (emulation mode):
COP UNUSED ABORT NMI RESET IRQBRK
0x82c3 ------ 0x82c3 0x82c3 0x8000 0x82c3
smtko-demo1.smc:
SMC header: yes
SNES header: offset 0x81c0, score: 9/9
name: "SUPER MARIOWORLD " (21 bytes)
status: 0x20 (slow LoROM)
cartridge type: 0x02 (ROM and save-RAM)
ROM size: 0x0c (4 megabytes)
RAM size: 0x03 (8 kilobytes)
country: 0x01 (NTSC)
licensee: 0x01 (Nintendo)
complement/checksum: 0x5f25/0xa0da
interrupt handlers (native mode):
COP BRK ABORT NMI UNUSED IRQ
0x82c3 0xffff 0x82c3 0x816a ------ 0x8374
interrupt handlers (emulation mode):
COP UNUSED ABORT NMI RESET IRQBRK
0x82c3 ------ 0x82c3 0x82c3 0x8000 0x82c3
mario-paint.sfc:
SMC header: no
SNES header: offset 0x7fc0, score: 7/9
name: "MARIOPAINT " (21 bytes)
status: 0x20 (slow LoROM)
cartridge type: 0x02 (ROM and save-RAM)
ROM size: 0x0a (1 megabyte)
RAM size: 0x05 (32 kilobytes)
country: 0x00 (NTSC)
licensee: 0x01 (Nintendo)
complement/checksum: 0xb461/0x4b9e
interrupt handlers (native mode):
COP BRK ABORT NMI UNUSED IRQ
0x80ae 0x0000 0x0000 0x80d4 ------ 0x80af
interrupt handlers (emulation mode):
COP UNUSED ABORT NMI RESET IRQBRK
0x0000 ------ 0x0000 0x0000 0x8000 0x0000
n-warp-daisakusen-1.0.sfc:
SMC header: no
SNES header: offset 0xffc0, score: 9/9
name: "N-Warp Daisakusen " (21 bytes)
status: 0x31 (fast HiROM)
cartridge type: 0x00 (only ROM)
ROM size: 0x0b (2 megabytes)
RAM size: 0x00 (zero)
country: 0x01 (NTSC)
licensee: 0x33 (id "\\\xAAO\xC0")
complement/checksum: 0x79de/0x8621
interrupt handlers (native mode):
COP BRK ABORT NMI UNUSED IRQ
0xffa0 0xffa0 0xffa0 0xffb6 ------ 0xffba
interrupt handlers (emulation mode):
COP UNUSED ABORT NMI RESET IRQBRK
0xffa0 ------ 0xffa0 0xffa0 0xffa1 0xffa0
I can use snes-rom.rb to see how SMW hacks increase the ROM size. The SMW hack smtko-demo1.smc has more ROM and more save-RAM than the clean ROM smw-clean.smc. My snes-rom.rb can also handle ROMs like Mario Paint and N-Warp Daisakusen, but it might have trouble with ROMs that have unusual layout or incomplete SNES header.
My computer does not run GoodTools nor NSRT, so snes-rom.rb provides my first easy way to examine a ROM, without starting an emulator, and to check if a ROM is LoROM or HiROM and if a ROM has a SMC header. I find that some ROMs have no SMC header but come with *.smc suffix, so I rename such ROMs to *.sfc to not imply a header.
|
|
| Posted on 2009-06-01 09:59:36 AM |
Link | Quote |
|
Originally posted by KerneghI have readied two other programs, and edited my earlier post to include them. The first program is ruby-ips-2.rb (svn), a second version of my program to apply IPS patches. I made this version during C3 of May 2009. (I did not present the code to C3.) I added a feature to copy clean ROM before apply patch, so that I need not copy my clean SMW ROM before I run patcher. This helps because I obtained 19 different IPS patches at C3.
Wait, you did that too?
I just finished this, a simple console app that does auto clean rom copy, ips patching, and runs the rom in an emulator, as long as the cfg is set up correctly. Source code is included.
|
|
| Posted on 2009-08-15 03:16:28 PM |
Link | Quote |
|
|
I actually felt like stickying this for good measure.
|
|
| Posted on 2009-08-15 10:39:28 PM |
Link | Quote |
|
Originally posted by ErsanioI actually felt like stickying this for good measure.
...aaaand to add some content to this post, here's some code that will convert between PC and SNES RGB values. (from after my embarrassing incident with the data repository)
Code private int convSnesRBG(int SnesValue)
{
int fin = 0;
fin += ((SnesValue & 31) * 8) << 16;
fin += (((SnesValue >> 5) & 31) * 8) << 8;
fin += ((SnesValue >> 10) & 31) * 8;
return fin;
}
Code private int convPcRGB(int PcValue)
{
int fin = 0;
// make 5-bit color values
// blue
int blue = (PcValue & 0xF8) >> 3;
// green
int green = (PcValue & 0xF800) >> 11;
// red
int red = (PcValue & 0xF80000) >> 19;
// make a bgr color by shifting
int bgr = blue << 10;
bgr += green << 5;
bgr += red;
return fin;
}
The second one was a good deal harder than the first one to make, so I broke it down and added comments for good measure.
|
|
| Posted on 2009-08-17 12:42:10 PM |
Link | Quote |
|
Here is a very small code in C# to devide something (panel, picturebox, whatever you want) in a 16x16 raster. I used this with the MouseClick event:
Codeint location = (e.X >> 4) + ((e.Y >> 4) * 0x10);
e.X being the event argument mouse X position and guess what e.Y is.
This is useful for when making a palette editor. Here is a graphical example of how the raster coordinates are laid out:
(Yes I converted the values to hex manually and that blue grid isn't related to this.)
|
|
| Posted on 2009-09-16 10:52:37 PM |
Link | Quote |
|
Computation of SNES checksum can be much simpler than my code from 4 May 2009. For a headerless ROM image of Mario Paint (JU) or Super Mario World (U), said computation requires one and only one line of Ruby code.
open(image, "rb") { |rom| rom.read }.sum(16)
Codeirb(main):001:0> image = "SMW/super-mario-world.sfc"
=> "SMW/super-mario-world.sfc"
irb(main):002:0> open(image, "rb") { |rom| rom.read }.sum(16).to_s 16
=> "a0da"
irb(main):003:0> image = "mario-paint.sfc"
=> "mario-paint.sfc"
irb(main):004:0> open(image, "rb") { |rom| rom.read }.sum(16).to_s 16
=> "4b9e"
Checksum of SMW is xa0da. Checksum of Mario Paint is x49be.
If the size of the ROM is not a power of 2, then you might to mirror part of the ROM, including the mirror in the checksum. I am not sure how to do the mirror, because the formulas in Snes9x and in WLA DX seem to disagree. My code from 4 May 2009 tries to mirror some banks, but this attempt might be wrong.
Also, my dump-tms.rb script, which dumps transmusical strings in Mario Paint, also seems to work with Super Mario World. The script needs a headerless ROM. I might later post the script.
Code$ mpdisc/dump-tms.rb -a 0e8000,0f8000 \
> roms/SMW/super-mario-world.sfc
string $0e8000:
$0500..$133d: from $0e8004..$0e8e42, download 3646 bytes
$5570..$5fda: from $0e8e46..$0e98b1, download 2667 bytes
$1360..$297c: from $0e98b5..$0eaed2, download 5661 bytes
$0500: jump (string ends at $0eaed6)
string $0f8000:
$8000..$804f: from $0f8004..$0f8054, download 80 bytes
$8100..$f01f: from $0f8058..$0fef78, download 28448 bytes
$0500: jump (string ends at $0fef7c)
|
| Last edited on 2009-09-16 11:02:45 PM by Kernigh. |
|
| Posted on 2009-09-17 09:02:24 AM |
Link | Quote |
|
Heh, does this mean I accidentally made a checksum recomputer? I just needed an AND 0xFFFF, change the format from long to ushort, and my tool was all set. lulz
Code//Code to compute checksum in C#
//Note: this does NOT deal with ROM headers. It could go ahead and sum the header bytes.
//Note2: I know the code is not that good but atleast it does the wanted function
Array.Clear(_rombuffer, 0, _rombuffer.Length); //clear stuff from previous run
result = 0; //same as above
_rombuffer = File.ReadAllBytes(ofd.FileName);
for (int x = 0; x < _rombuffer.Length; x++)
{
result += (ushort)_rombuffer[x]; //sum the bytes one by one.
}
result &= 0xFFFF; //filter unnecessary digits
Where _rombuffer is a byte array (byte[]), which holds the bytes of the ROM, and result being an 16-bit unsigned number (ushort), holding the result of the calculation.
I'll PERHAPS post a faster code later.
|
|
| Posted on 2009-09-19 12:01:34 AM |
Link | Quote |
|
Originally posted by ErsanioCodeArray.Clear(_rombuffer, 0, _rombuffer.Length);
//clear stuff from previous run
result = 0; //same as above
_rombuffer = File.ReadAllBytes(ofd.FileName);
The Array.Clear might not be necessary, because (if C# is like Java, Perl, Ruby) I guess that File.ReadAllBytes returns a new array.
Originally posted by Ersanio//Note: this does NOT deal with ROM headers. It could go ahead and sum the header bytes.
You can always use a headerless ROM image, because the SMC header is an annoyance. Or in this case, with an open file handle, you can seek past the first $200 bytes and read everything else. The BSNES method of detect header is to simply check if ROM size modulo $8000 equals $200.
Here is Ruby program to print checksum of headered or headerless image.
Code#!/usr/bin/env ruby
ARGV.each do |filename|
open(filename, "rb") do |rom|
header = rom.stat.size & 0x7fff == 0x200
rom.pos = 0x200 if header
checksum = rom.read.sum 16
printf "%s: checksum is $%04x\n", filename, checksum
end
end
Here is same program in Perl.
Code#!/usr/bin/env perl
use strict;
use warnings;
use Fcntl qw(SEEK_SET);
use List::Util qw(sum);
for my $filename (@ARGV) {
open my $rom, '<', $filename or die "$filename: $!";
my $header = ((stat $rom)[7] & 0x7fff) == 0x200;
seek $rom, 0x200, SEEK_SET if $header;
my $checksum = sum(unpack "C*", do { local $/; <$rom>}) & 0xffff;
printf "%s: checksum is \$%04x\n", $filename, $checksum;
}
I run both programs.
Code~/park/snes $ ./sum.rb roms/SMW/{super-mario-world.sfc,smw-clean.smc}
roms/SMW/super-mario-world.sfc: checksum is $a0da
roms/SMW/smw-clean.smc: checksum is $a0da
~/park/snes $ ./sum.pl roms/SMW/{super-mario-world.sfc,smw-clean.smc}
roms/SMW/super-mario-world.sfc: checksum is $a0da
roms/SMW/smw-clean.smc: checksum is $a0da
The Perl version is slower, I guess that unpack is slow? Ruby has String#sum but I found no equivalent in Perl.
Ruby => Perl
rom.read => do { local $/; <$rom> }
string.sum 16 => no equivalent
string.unpack("C*").reduce(:+) => reduce { $a + $b } unpack "C*", string
no equivalent => sum unpack "C*", string
This simple formula is valid if the ROM size is a power 2 (or equivalently, if the ROM size matches a possible value for SNES address $00ffd7 in the SNES header).
If the ROM size is not a power of 2, then Snes9X, WLA DX and uCON64 disagree the checksum. They use three different formulas. I checked! Our simple formula cannot be more wrong than any of difficult formulas in Snes9x, WLA DX and uCON64. We can keep our simple formula. Any correct ROM size is a power of 2, and all four formulas agree when the ROM size is a power of 2.
I checked the formulas by reading the source code, not by running the programs. I checked the formulas at
* CMemory::Checksum_Calculate in memmap.cpp of Snes9x-GTK 74,
* compute_snes_checksum in wlalink/compute.c of WLA DX 9.5a,
* snes_chksum in src/console/snes.c of uCON64 2.0.0.
|
|
| Posted on 2009-12-11 01:11:16 PM |
Link | Quote |
|
|
Sorry for the bump, but what is the code for patch a ips patch in VB2008?
|
|
| Posted on 2009-12-11 03:58:06 PM |
Link | Quote |
|
|
Uh, we don't have one unless someone feels like contributing it here. Just wait and you'll see (or not).
|
|
|
| Pages: 1 2 |
|
|
|
|
Forum Index - Hobbies - Computers & Technology - Programming - SMW/ROMhacking tool related code library |
|
|
 |
|
 |
The purpose of this site is not to distribute copyrighted material, but to honor one of our favourite games.
Copyright © 2005 - 2010 - SMW Central
Legal Information - Link To UsTotal queries: 24
|
|
|
|
