Saturday, May 12, 2007

IWRAM code vs. recompiler

(NOTE: this post pertains to any ports of gpSP using dynarec, that means the PSP and GP2X versions)

There is an issue with certain GBA games that has plagued me since I first started work on gpSP. I've tried some hacks to get around it, and they've helped, but they were very ugly. Some months ago I came up with a new idea to try to improve the situation. This idea was a bit complex and had some serious complications and I basically abandoned it when I thought of something simpler.

This is the issue: GBA can execute code from one of two places, ROM (the cartridge and the BIOS) and RAM (that includes IWRAM, EWRAM, and rarely, VRAM. gpSP doesn't support executing code from VRAM right now so that's out already). ROM can't be overwritten, so when code is encountered from ROM it's known that it'll always be the same. RAM is another story.

Most games will load a bit of code into RAM to run it quickly. Usually this code is just left alone after being loaded, so it's like ROM but faster. Sometimes it's reloaded when the game changes significantly (like entering a new area, or going from the title screen to ingame), but usually it stays the same for a long time. These modifications to code do have to be checked, because it means that the code that has been dynamically recompiled has to be flushed (since it is no longer valid). Normally all of the code that was in RAM is flushed, since the buffers aren't setup in a way that allow you to flush part of it, and doing so would make any other blocks that branch into the modified block invalid. The operation to scan and update all of RAM would probably be as slow or slower than just recompiling all of it.

This works fine for a majority of games. There's some overhead checking if writes modify code, but it's not a lot.

The problem comes when games modify code in RAM a LOT. Some games only change a few bytes, but constantly. This is known as self-modifying code. It's a pretty old technique that's shunned these days because it doesn't get along well with caching. The GBA doesn't have cache, so some games still do this. It may be the case that some games also load in huge chunks of code to swap it in and out of RAM constantly, which isn't quite the same as self-modifying code but appears the same. This usually happens over DMA. From what I've seen not a lot of code in IWRAM is modified per-frame, even when DMA is used. Some games are very sloppy though, like those written by Camelot. They will DMA the same thing into IWRAM over and over again.

Anyway, even if only a few bytes are changed, with the way things work now it'll cause everything that has been compiled to be flushed. So I came up with a solution to help change this. Now I have two buffers for RAM code, a "static" one and a "dynamic" one. There are counters for code regions associated with self modifying code writes. When something has been written to enough times the region changes from static to dynamic. Now when code in that region is recompiled it will go to the dynamic buffer instead of the static one. The consequence of this is that when this code is modified in the future only the dynamic buffer needs to be flushed. If the amount of code being rewritten is relatively small then this dynamic region will be minimized.

I haven't done any real world speed tests on this yet, but I have determined that this method is causing far fewer bytes to be recompiled per frame. There's still quite a bit of overhead in flushing, but this can be alleviated tremendously and cheaply by using region tables. I haven't actually looked at numbers from the original implementation, actually, I was comparing it with dynamic + static buffers but with both being flushed on SMC writes. This would probably still perform better than the original because the "translation gate" custom optimization was effectively being performed dynamically.

Translation gates are a hack I added to try to prevent things from "overcompiling." The issue is that the recompiler is very deep, and will compile as much as it can. If a region of code is changed then the recompilation buffers will be flushed. What happens next is that to resume CPU execution a pointer to the currently translated code has to be found. If the code was running in ROM then this is fine, it'll still be there. But if it was running in RAM (and Camelot games like to modify code that is dangerously close to the code that's currently running) then it'll have to recompile it again. If the deep recompilation ends up catching the code that was modified then it'll recompile it over and over again, every time it's modified. If the code is modifying a huge block in a loop then this means every iteration of the loop will cause a recompilation of this code. What we really want is to make it so this code isn't recompiled until it's actually used (and thus the writing is long done). The translation gates were like breakpoints that I determined by looking at the code in a debugger. They told the recompiler to stop compiling when they're hit, and generate an indirect branch that's resolved at runtime, to get to the next portion.

In short, the static/dynamic analysis finds these breakpoints automatically, and it does a better job than I was doing with the translation gates (although it does add overhead of more indirect branches at runtime). But the selective flushing adds a huge gain on top of this.

I don't remember the exact numbers, but I do know that Golden Sun recompiles far less per frame (I think it was about 500-3000 bytes now compared to over 10000 bytes before). However, this improvement is nothing compared to the one in Duke Nukem Advance and Doom 2. These were extremely pathological in how they hit the recompiler, apparently they were compiling several dozens of thousands of bytes per frame, even well over 100,000. Now they typically recompile under 1000. I expect this to make a huge difference in performance, although the GP2X version will need more optimized video writing code to see good performance in these games (the PSP version doesn't have this problem).

There are perhaps still some things I can do to make things better. I can segment the dynamic cache into several regions, so this way if a number of places are being modified they can each be flushed independently (in practice these locations probably don't jump between each other, but who knows?). I also had some really crazy ideas about aliasing different RAM frames that get flipped when DMA occurs, but I don't think this will ever work out because the modifications to code are just too irregular.. I expected they'd follow a cyclic pattern, and they just don't, at all.

I need to fix and refine the code a bit more (some games are broken now?), then I'm going to do some tests on the GP2X to see how the current version fares against this one for problem games.


Sevenz said...
This comment has been removed by the author.
Sevenz said...

Wow... That's really a deep explanation O_O
Wish I could understand all of it, which I`m only know part of it, hopefully.

Just wish you all the best while trying every possibilities ^_____^



DarkPacMan77 said...

Hi there Exophase. I represent I'm attempting to contact you so that I could perhaps take a few minutes of your time and interview you. You can contact me via Private Message at or through my email address at I look forward to hearing from you and about the direction you plan to take with your famed gpSP emulator.

David said...

You are doing awesome work, Exophase.

Jeff Wilcox said...

I for one am interested in seeing how much the dynarec envelope can be pushed on the PSP... and what better platform to emulate than the GBA!! There's a phenominal library of VERY good games out there for this platform rather than just a few titles.

Keep up the good work Exophase, between your work and StrmnNrmn's work on Daedalus, I'm sure a large amount of wisdom on implementing dynamic recompilation on the PSP has been gained.
Maybe you guys could put together paper on it. I for one would read it.

Unknown said...

Hi Exophase. I know you are very busy with debuging gpSP, we all really appreciate it.
I just want to request you support codebreaker in the cheat section. Most of the useful cheat database is base on codebreaker, and when I try converse to GameShark those code didn't work. I like codebreaker because it support both condense and raw code.
If you can improve the cheat function more on the next release, I won't be able to express my happiness for your help.

XG917 said...

hey exo. i love the work you have put into gpSP. this is an awsome emulator and i use it everyday! :)
are you still continuing this project? i really hope so
it would be awsome if you can add wifi connection for multiplayer action. lol maybe someday.. well i have to go now. i wish i was a coder so i understand every thing you say. hehe. well, bye


Maxi Power said...

Keep it up dude, i love ur emulator more than anything else on the Homebrew side of the PSP. if and when u sit back down to it could ya focus on golden sun the lost age.


Unknown said...

Eeeeeh... That includes Shining Soul I and Shining Soul II? I never got to play these games on gpSP :(

Punkonjunk said...

I stumbled onto this blog looking to see if there are any updates past .9 for Gpsp.
And jesus christ, I'd just like to say... I'm impressed. I thought Gpsp was (in my horrible uninformed-ness) just a port of some PC GBA emu, like most other emus are for PSP right now.

Read this entry and a bit of the few others really brought to light that this is something that alot of work went into.

As a huge homebrew enthusiast, I'm sure you don't get this kind of thing enough.

Thanks. What you're doing is breathing life into a somewhat old system. It's not quite obsolete, but it's definately wonderful to have it all in one place.

From me and I'm sure a lot more homebrew enthusiasts, thank you. Keep up the good work. Here's to 1.0!

Riddoct said...

you said that psp doesnt have this issue with Camalot games yet i was playing Golden Sun 1 and it crashed on me, the screen went black, and the sound just crackled and screeched and popped. haha maybe its the rom.

regardless, thank you for making gpSP for all of us and im glad i get to play my gba games whereever i go

Unknown said...

I notice things are quiet lately. I really have enjoyed gpSP so far, and I wanted to express my appreciation for your work.

Also, after reading what you've done since 0.9, I have to say I'm pretty impressed. I look forward to your future releases.

I just hope everybody realizes that v1.0 is a *huge* step, and as such it takes a lot of time and work to make everything satisfactory enough to be out of the "beta" stage. Until that time comes, good luck making gpSP everything you want it to be :)

Jinkson Fox said...

Just stopping in, and I also want to say that I really appreciate your work and hope to see future releases!

Unknown said...

Could you please make a very short post just to assure us you are still around?

Chris Murphy said...

Exophase, whatever you are doing (existing, thinking), keep doing it, because what you are doing is wesome. I just want you to know that I appreciate the work you are doing here. It is truly a great accomplishment. Again, thank you.

Also, you are doing something that a lot of people can't. People complain. They whine. Say your programs suck. But they themselves possess not even a portion of your talent. Do not listen to them. For they can't appreciate what you do.

I just want you to know you are appreciated. You are doing something that is to be respected. I support you all the way ^_^

Anonymous said...

would it ever be possible to include ad hoc play emulating the gba's wireless adhoc link adapter

picture here

then you could link between a GBA and a PSP or even a DS and GBA using the adhoc link on the DS playing a GBA game.

that would be awesome and would shock everyone.

AustinISO said...

visit my blog for GBA roms N64 roms and PSP ISOs

Unknown said...

hi Exop:

great work you did....

B said...

I love you.

Sardaukar said...

Hello Exophase.

I would like to try and port GPSP to the Nokia N900 (Arm Cortex A8). IS there a public source-code tracker?



I just want to say that gpSP is the best emulator I've ever used. I've only really tried it with Mario Kart, but it works perfectly, even the sound. Even NES emulators on a big powerful PC tend to be imperfect one way or another. The menu is great too - simple and functional, thumbnail of the game, screenshot function, cheat function, temporary save state, rapid-fire, full key remapping. It's absolutely fantastic. :D

Froy said...

I have a problem with gpSP running Golden Sun. It seems when every time the main character talks to the guy who is pulling is stuff it crashes horribly. Sometimes the game randomly crashes!! Is there new versions for the Wiz to repair this problem? Thanks in advance.

zaphod said...

SO is there any chance of Mario Golf Advance Tour getting fixed?

AS it stands, the only way to make it even sorta playable is to turn off the ingame music, and take multiple practice swings until the meter stabilizes, and it still goes to zero frames per second for the swing impact.

A friend of mine REALLY wants to play this game on his PSP. At this point i'd accept a special version of the emu that only works with this game, or a rom patch that modifies the game's code to be not so wonky.

zaphod said...
This comment has been removed by the author.