I finally got Super Mario to work! (well, at least the title screen shows up)
Both my NMI and RTI routines were broken, so it's no wonder it crashed horribly before..

Also had some problems with the flags, because I'd been looking at the source code of VICE when implementing CMP, CPX and CPY. VICE uses unsigned ints for temp vars, while there is no such thing in Java. Subtraction does of course not work the same then.. I don't know how I could have overlooked a problem that obvious :-/