It's good that there's plenty of support but there's not much organization yet. As I see it, there's 3 major branches of work to be done:

1) NES documentation
2) The emulator itself
3) Development tools ([dis]assembler, debugger, etc)

They shouldn't necessarily have to be worked on in any particular order, it would be best (IMHO) to establish procedures and structure things in a way such that people can immediately start contributing in areas they are most experienced in and not be waiting on someone else's work. If we don't see any results and people don't feel like they're contributing, they will lose interest quickly.

Documentation is probably the easiest part to get rolling. Make a decision on the format and work up a basic outline that we can start filling in with details, and it should take off from there.

The emulator.. I have mixed feelings about this. I'm not sure I agree about the original plan of starting completely from scratch. A lot of very good work has already been done between all the various emulators out there and shouldn't be wasted. Maybe pick one as a base and work from there?

Dev tools can be done individually at this point. Decide what tools we need and keep track of who's working on what, and leave things open so they can be integrated at some point in the future.

There's my $.02. Thoughts? Disagreements? Or just tell me to shut up and go away :)

I was wondering the exact same thing myself. I think the first big decision needs to be how to organize? Is this a "just let everyone do what they want when they want", or are we going to have structured organization, with subprojects, and leaders and hierarchy and decision-making policies, or what?

-- Theodore Reed (rizen/bancus)

Shut up and go away!

No, on the contrary, I'm glad you posted this! I was getting worried.
Infact, I was getting so worried I was actually thinking of posting something like this myself, but I would have done a worse job of it.

When it comes to the emulator, I guess the biggest shortcut would be using an existing 6502 core, and why not? There are several excellent 6502-emulators out there, and if we get sick of them, there should be no problem replacing it with one of our own, right?

Getting a homepage up and running as soon as possible might be a good idea, and I guess creating a Sourceforge account would take care of that, so let's decide on a name already! I don't really care what it is as long as it's not Rape... :)

I'm off to drink lots of Japanese beer. Don't let me come back here just to find the board completely untouched! Get posting, people!

Here is how I propose the project work:

a) NES Documentation

This part should be downloadable in various formats, and should also be online in .html. This should be no problem to do by using a single common base file format for all documentation, and then doing batch conversions to various other formats (PDF? .HTML, Etc), and allow the individual to choose their favorite.

The NES Documentation is the section we can start work on immediately, once a common file format is decided on (Suggestions?).

b) Emulator
c) Debugger

There should be (at least) two releases, a release with the Emulator and Debugger compiled together, and a release with just the Emulator (By compiling the Emulator without all the debugger extensions, it will gain speed and shrink in size).

d) Assembler
e) Disassembler

The debugger should interface with an Assembler and Disassembler, possibly even in such a way that you can modify the source of a game while you are debugging it.

As you can see, with this plan, the project is already divided into 5 components: Documentation, Emulation, Debugger, Assembler, Disassembler. I think all components should be developed in such a way to allow further components (such as Sprite editors, and other development tools) to be added.

Peace,
Caustik

I agree. And if we use the PeNES concept of an emulation engine and several interface layers, the release of an emu and an emu/debugger would be pretty easy. And as long as we write in nicely objectified code, allowing later expansion shouldn't be that hard.

Not the kind of organization I was talking about, tho. I meant personal organization. Who takes charge of which subprojects, who works where, or is it just a, everyone can contribute what they want to the projects they want. The disorganized method is fine until you start getting more than say 20 people involved. At that point, some hierarchical system becomes nessecary(sp).

-- Theodore Reed (rizen/bancus)

I agree completely. I really think we should have "owners" of the five major submodules. These owners would be nominated, given a chance to explain why they would be good for the job, and an election could be held. I'm not suggesting that the owners be any more important than anyone else, just that they be responsible for their module.

Here are my suggestions. First off I suggest using FCE Ultra as the starting point for the emulator. You don't have to use all of FCE Ultra, but it is opensourced under the GPL, and it is the most accurate NES emulator to date. At least you could use the CPU core. Also, here are my tools suggestions...


Tools:
1. Macro Assembler: an assembler for the NES's version of the 6502 that supports macros and does basic static error checking. Furthermore, the output should be a UNIF rom file. It should also allow binary files such for tiles and sounds to be included in the build process.

2. Common Macro Library Routines: there are many different snippets of asm code that are found in just about every NES game. Why not bring those into a collection of nicely documented and easy to use macro routines? This would allow future NES game development to make use of these common snippets of code in an elegent manner.

3. Macro Disassembler: This should take a UNIF file and spit out binary files for tiles and sounds, in addition to text files for the code. The disassembler should make an attempt to identify "common" blocks of code, which can then be disassembled into a macro. Of course, you can't get macro names back, but such structuring the disassembly as much as possible makes reverse engineering easier.

4. Trace Debugger: takes an asm project and steps through the code showing the memory values after each instruction is executed. Also shows the screen and lets the user make inputs using the gamepad, etc...

You claim that FCE Ultra is the most accurate emulator to date.
I'm assuming you've never seen my NES emu...

FCE Ultra may be one of the fastest accurate emulators, but it is not the most accurate one (unless he recently added proper emulation of mid-scanline PPU writes).

--
Quietust
P.S. If you don't get this note, let me know and I'll write you another.

It may be my lack of knowledge of disassembly programming but the disassembly part of the suggestions sound more difficult to do than the other 3 and if so add a lot more time to this project

Interesting... I will check it out (your emu). I prefer absolute accuracy of emulation over netplay, quantity of mapper support (quality is more important), and lame graphics filters such as those many different interpolation techniques.

A good NES emu should be indistinquishable from the real thing.

A good NES emu should be indistinquishable from the real thing.

So will we emulate the lockout chip found on front-loading consoles?

What about emulating the dirt on the connectors?

Or plugging homemade devices into the controller ports and expansion ports (possibly through DLL or .so)?

Point: I find a precise definition of the scope of what will be emulated (and what won't) important.

I would be quite happy when it's impossible for a program to detect that it runs in an emulator instead of the "real thing".

Amen! Yes! That is the best definition of perfect emulation.
Btw, I like your NES Stress Testing rom. I hope you continue to turn it into a complete compatability testing suite for NES clones and NES emulators.

Btw, I assume that you run the stress tester on a real NES, and the real NES passes the test ;-)

The problem is I don't have any hardware to test with, ofcourse I have a NES but I have no copier/backup/testing thing for the NES. But I would be realy interested if could get hold of one. The CPU _instruction_ test should all be OK, the first page with PPU test should be OK, the sprite test doesn't work, the last PPU test is OK.

/FluBBa

As a compiler/assembler type (who wouldn't mind handling that side of things for CNES, actually), some comments:

Assembler:

We should be drawing on the finest of the assembler traditions when we produce the CNES assembler. Based on the feedback that I got for P65, those would be TASM and DASM. TASM is payware, and DASM was a 1988 Amiga program, so we'd be filling a very-much needed role with this. That means the assembler needs:

- Parameterized macros - not merely lists of instructions, the arguments to those instructions should be seperately settable. First-class macros would rule (macros that take macros as arguments).
- A module system, including anonymous and local labels. This is absolutely necessary for projects of any size to keep the namespace from blowing up.
- Assemble-time expressions. LSB, MSB, addition, subtruction, and multiplication, at minimum, with parenthesization. (To resolve ambiguities between parentheses and indirect instructions, DASM used [] for arithmetic parenthesizing.)
- Automatic instruction selection - NESASM's requirement that YOU tell it where the zero page instructions are is unacceptable.
- Multi-file support, both source and binary. Given this and the macro system, a "UNIF output" and "iNES output" set of macros could be very easily defined. Hell, an "Apple IIe" output would be just as easily definable, as could special names for the memory registers.
- Memory segmentation simulation, with overlay support. No extant assembler currently has both (though many have support for one or the other, or let you fake them).

Libraries:

Not a lot to say about libraries other than yeah, we need them, and a combination of macro definitions and code libraries as includable source would handle it nicely (stdsprite.h anyone?)

Disassembler:

The disassembler should be able to take arbitrary binary data, not just UNIF. A lot of logic/math code written for the Commodore 64 and Apple II will also run on the NES if it doesn't hinge on Decimal mode - gotta be able to get at that, too.

Basic table-driven disassembly is trivial. We can get a *simple* disassembler out the door in no time flat, as soon as somebody cares enough to make it.

Automatic macro detection is extremely expensive (naive algorithms are n^4) but not conceptually difficult, and hey, 64k^4 instructions doesn't take THAT long on a 500MHz Celeron, now does it? A larger problem is that a lot of "macros" aren't - they're just code that's superficially similar but that does entirely different things. A way around that would be to scan for predefined macros (which is linear time).

More useful than automatic breaking into macros would be breaking things into basic blocks and procedures: marking the targets of JSR statements, and splitting code up into chunks that always run in sequence (that is, nothing in the code ever jumps into that location). That gets trickier to do right in the presence of switchable code blocks. A "first iteration" would stick to code like Gradius and SMB that only have one PRG-ROM block - otherwise, you'd need code-scanners to figure out which PRG-ROM chunks were swapped in when you made the jump, which could wind up being arbitrarily complex, maybe even undecidable.

Debugger:

Trace debugger should really be part of the emulator, not the assembler, though (if we break up the modules right) they should all be easily able to talk to one another. Possibly some way of mapping memory locations/mapper status to source code file/line to jump to the right source line as well as the right instruction.

When a assembler is done which is specified towards unif as well as .nes then perhaps a fully clear tutorial describing making a unif rom could be done which would help the inexperienced or newbie nes programmer start off the way people want (coding unif roms) - just a thought

It's on my list. However, I'd like a basic tutorial to cover at least NROM, UNROM, and CNROM, and I'm not sure how the multiple PRG and CHR section maps to the UNROM and CNROM. Would Zanac (a 128kb UNROM) have two 64kb PRGs, or one 128kb one? If it matters to the reader, where does this info come from?

With UNROM it should probably be a 128KB one, since there is one 128KB ROM on the cartridge. Or it could be 16KB PRGs, since that is the bank size the mapper works with.

Also, I don't think an assembler should necessarily have the task of making a UNIF rom. If a commandline-based UNIF builder was made it would allow programmers to use whichever assembler they are most familiar with, just by passing the cart config info to the program in the batch file that assembles the program. Or maybe use a configuration file.

Replace "assembler" with "devkit" here, then. Even as a linker, the source-writer needs to be kinda aware of it, since he's either producing one file for the chip with a bunch of .org $8000s in it at the end of each block, or he's producing two-to-sixteen files, one for each "cartridge page".

would't this have to be a proper prog (gui based , command lines are pain in neck) to extract the code from rom such as the 1kbytes in the programming competition from the 15kbytes (padding) the assembler produces (at least nesasm does) to fit in the ines format else the rom would be as large or larger than the .nes rom

This is among the reasons that NESasm honks me off :) Producing iNES formats should also be done in a "linking" stage by a separate program, or hand-coded in if not. Memblers is claiming that the PRG-ROM (or, possibly, each mappable page, which would vary by mapper type/board type) should be produced as a separate source file, then stitched together by the linker.

In NESASM at least, suppose you were making an NROM game; 32K PRG and an 8K CHR-ROM stored as a .chr file, and that they were named "game.asm" and "game.chr".

1. Remove the .incbin "game.chr" part of the source code (or comment it out)

2. At the DOS prompt, type "nesasm -raw game.asm" - which will assemble the PRG-ROM into game.nes, without an iNES header and without the .chr attached. rename game.nes to game.prg or something.

3. either by hand or using some utility, create a new UNIF file using game.prg as the PRG-ROM and game.chr as the CHR-ROM.

4. voilà! the same could be done for almost any other game type (of any size), provided you know the correct board ID for the game you're making.