That depends on the content of those images. For photographs or other continuous tone pictures, JPEG is best. For line art like schematics, PNG is good. GIF can also be good for that, especially if the diagram is black+white, and you tell the software to only use gray levels in the GIF.
As for pixel dimensions, I would make them as small as they can be to clearly show what you want to show. Larger pictures cause trouble or are inconvenient in some viewers. It's best if the picture doesn't need to be scrolled or the viewer resize it to fit into whatever window the user chose.
I would stay to 800 wide or less if at all possible. 512 or less is pretty much safe and you don't have to worry about it. If you really need more resolution (this is actually quite rare), then you can make the 512-wide picture clickable so that it brings up the full resolution version.