Gavare's eXperimental Emulator: Introduction

Back to the index

## Introduction

- Overview
- Is GXemul Free software?
- How to compile/build the emulator
- How to run the emulator
- Which processor architectures does GXemul emulate?
- Which host architectures are supported?
- What kind of translation does GXemul use?
- Emulation accuracy
- Which machines does GXemul emulate?

NetBSD/pmax 1.6.2 with X11
running in GXemul

### Overview:

GXemul is an experimental instruction-level machine emulator. Several emulation modes are available. In some modes, processors and surrounding hardware components are emulated well enough to let unmodified operating systems (e.g. NetBSD) run as if they were running on a real machine.

Devices and processors (ARM, MIPS, PowerPC) are not simulated with 100% accuracy. They are only ``faked'' well enough to allow guest operating systems run without complaining too much. Still, the emulator could be of interest for academic research and experiments, such as when learning how to write operating system code.

The emulator is written in C, does not depend on third-party libraries, and should compile and run on most 64-bit and 32-bit Unix-like systems.

The emulator contains code which tries to emulate the workings of CPUs and surrounding hardware found in real machines, but it does not contain any ROM code. You will need some form of program (in binary form) to run in the emulator. For many emulation modes, PROM calls are handled by the emulator itself, so you do not need to use any ROM image at all.

You can use pre-compiled kernels (for example NetBSD kernels, or Linux), or other programs that are in binary format, and in some cases even actual ROM images. A couple of different file formats are supported (ELF, a.out, ECOFF, SREC, and raw binaries).

If you do not have a kernel as a separate file, but you have a bootable disk image, then it is sometimes possible to boot directly from that image. (This works for example with DECstation emulation, or when booting from ISO9660 CDROM images.)

### Is GXemul Free software?

Yes. I have released GXemul under a Free license. The code in GXemul is Copyrighted software, it is not public domain. (If this is confusing to you, you might want to read up on the definitions of the four freedoms associated with Free software, http://www.gnu.org/philosophy/free-sw.html.)

The code I have written is released under a 3-clause BSD-style license (or "revised BSD-style" if one wants to use GNU jargon). Apart from the code I have written, some files are copied from other sources such as NetBSD, for example header files containing symbolic names of bitfields in device registers. They are also covered by similar licenses, but with some additional clauses. The main point, however, is that the licenses require that the original Copyright and license terms are included when you make a copy or modification.

If you plan to redistribute GXemul without supplying the source code, then you need to comply with each individual source file some other way, for example by writing additional documentation containing copyright notes. I have not done this, since I do not plan on making distributions without source code. You need to check all individual files for details. The "easiest way out" if you plan to redistribute code from GXemul is, of course, to let it remain open source and simply supply the source code.

In case you want to reuse parts of GXemul, but you need to do that under a different license (e.g. the GPL), then contact me and I might re-license/dual-license files on a case-by-case basis.

### How to compile/build the emulator:

Uncompress the .tar.gz distribution file, and run

    	$ ./configure
    	$ make

This should work on most Unix-like systems. GXemul does not require any specific libraries to build, however, if you build on a system which does not have X11 libraries installed, some functionality will be lost.

The emulator's performance is highly dependent on both runtime settings and on compiler settings, so you might want to experiment with different CC and CFLAGS environment variable values. For example, on an AMD Athlon host, you might want to try setting `CFLAGS` to `-march=athlon` before running `configure`.

### How to run the emulator:

Once you have built GXemul, running it should be rather straight-forward. Running `gxemul` without arguments (or with the `-h` or `-H` command line options) will display a help message.

To get some ideas about what is possible to run in the emulator, please read the section about installing "guest" operating systems. If you are interested in using the emulator to develop code on your own, then you should also read the section about Hello World.

To exit the emulator, type CTRL-C to enter the single-step debugger, and then type `quit`.

If you are starting an emulation by entering settings directly on the command line, and you are not using the `-x` option, then all terminal input and output will go to the main controlling terminal. CTRL-C is used to break into the debugger, so in order to send CTRL-C to the running (emulated) program, you may use CTRL-B. (This should be a reasonable compromise to allow the emulator to be usable even on systems without X Windows.)

There is no way to send an actual CTRL-B to the emulated program, when typing in the main controlling terminal window. The solution is to either use configuration files, or use `-x`. Both these solutions cause new xterms to be opened for each emulated serial port that is written to. CTRL-B and CTRL-C both have their original meaning in those xterm windows.

### Which processor architectures does GXemul emulate?

The architectures that are emulated well enough to let at least one guest operating system run (per architecture) are ARM, MIPS, and PowerPC.

### Which host architectures are supported?

As of release 0.4.0 of GXemul, the old binary translation subsystem, which was used for emulation of MIPS processors on Alpha and i386 hosts, has been removed. The current dynamic translation subsystem should work on any host.

### What kind of translation does GXemul use?

Static vs. dynamic:

In order to support guest operating systems, which can overwrite old code pages in memory with new code, it is necessary to translate code dynamically. It is not possible to do a "one-pass" (static) translation. Self-modifying code and Just-in-Time compilers running inside the emulator are other things that would not work with a static translator. GXemul is a dynamic translator. However, it does not necessarily translate into native code, like many other emulators.

"Runnable" Intermediate Representation:

Dynamic translators usually translate from the emulated architecture (e.g. MIPS) into a kind of intermediate representation (IR), and then to native code (e.g. AMD64 or x86 code). Since one of my main goals for GXemul is to keep everything as portable as possible, I have tried to make sure that the IR is something which can be executed regardless of whether the final step (translation from IR to native code) has been implemented or not.

The IR in GXemul consists of arrays of pointers to functions, and a few arguments which are passed along to those functions. The functions are implemented in either manually hand-coded C, or automatically generated C. In any case, this is all statically linked into the GXemul binary at link time.

Here is a simplified diagram of how these arrays work.

There is one instruction call slot for every possible program counter location. In the MIPS case, instruction words are 32 bits in length, and pages are (usually) 4 KB large, resulting in 1024 instruction call slots. After the last of these instruction calls, there is an additional call to a special "end of page" function (which doesn't count as an executed instruction). This function switches to the first instruction on the next virtual page (which might cause exceptions, etc).

The complexity of individual instructions vary. A simple example of what an instruction can look like is the MIPS `addiu` instruction:

    	X(addiu)
    	{
    	        reg(ic->arg[1]) = (int32_t)
    	            ((int32_t)reg(ic->arg[0]) + (int32_t)ic->arg[2]);
    	}

It stores the result of a 32-bit addition of the register at arg[0] with the immediate value arg[2] (treating both as signed 32-bit integers) into register arg[1]. If the emulated CPU is a 64-bit CPU, then this will store a correctly sign-extended value into arg[1]. If it is a 32-bit CPU, then only the lowest 32 bits will be stored, and the high part ignored. `X(addiu)` is expanded to `mips_instr_addiu` in the 64-bit case, and `mips32_instr_addiu` in the 32-bit case. Both are compiled into the GXemul executable; no code is created during run-time.

Here are examples of what the `addiu` instruction actually looks like when it is compiled, on various host architectures:

GCC 4.0.1 on Alpha: |  |

    mips_instr_addiu:
         ldq     t1,8(a1)
         ldq     t2,24(a1)
         ldq     t3,16(a1)
         ldq     t0,0(t1)
         addl    t0,t2,t0
         stq     t0,0(t3)
         ret

    mips32_instr_addiu:
         ldq     t2,8(a1)
         ldq     t0,24(a1)
         ldq     t3,16(a1)
         ldl     t1,0(t2)
         addq    t0,t1,t0
         stl     t0,0(t3)
         ret

GCC 3.4.4 on AMD64:

    mips_instr_addiu:
         mov    0x8(%rsi),%rdx
         mov    0x18(%rsi),%rax
         mov    0x10(%rsi),%rcx
         add    (%rdx),%eax
         cltq
         mov    %rax,(%rcx)
         retq

    mips32_instr_addiu:
         mov    0x8(%rsi),%rcx
         mov    0x10(%rsi),%rdx
         mov    (%rcx),%eax
         add    0x18(%rsi),%eax
         mov    %eax,(%rdx)
         retq

GCC 4.0.1 on i386:

    mips_instr_addiu:
         mov    0x8(%esp),%eax
         mov    0x8(%eax),%ecx
         mov    0x4(%eax),%edx
         mov    0xc(%eax),%eax
         add    (%edx),%eax
         mov    %eax,(%ecx)
         cltd
         mov    %edx,0x4(%ecx)
         ret

    mips32_instr_addiu:
         mov    0x8(%esp),%eax
         mov    0x8(%eax),%ecx
         mov    0x4(%eax),%edx
         mov    0xc(%eax),%eax
         add    (%edx),%eax
         mov    %eax,(%ecx)
         ret

On 64-bit hosts, there is not much difference, but on 32-bit hosts (and to some extent on AMD64), the difference is enough to make it worthwhile.

Performance:

The performance of using this kind of runnable IR is obviously lower than what can be achieved by emulators using native code generation, but can be significantly higher than using a naive fetch-decode-execute interpretation loop. In my opinion, using a runnable IR is an interesting compromise.

The overhead per emulated instruction is usually around or below approximately 10 host instructions. This is very much dependent on your host architecture and what compiler and compiler switches you are using. Added to this instruction count is (of course) also the C code used to implement each specific instruction.

Instruction Combinations:

Short, common instruction sequences can sometimes be replaced by a "compound" instruction. An example could be a compare instruction followed by a conditional branch instruction. The advantages of instruction combinations are that

- the amortized overhead per instruction is slightly reduced, and

- the host's compiler can make a good job at optimizing the common instruction sequence.

The special cases where instruction combinations give the most gain are in the cores of string/memory manipulation functions such as `memset()` or `strlen()`. The core loop can then (at least to some extent) be replaced by a native call to the equivalent function.

The implementations of compound instructions still keep track of the number of executed instructions, etc. When single-stepping, these translations are invalidated, and replaced by normal instruction calls (one per emulated instruction).

Native Code Back-ends: (not in this release)

In theory, it will be possible to implement native code generation (similar to what is used in high-performance emulators such as QEMU), as long as that generated code abides to the C ABI on the host, but for now I wanted to make sure that GXemul works without such native code back-ends. For this reason, as of release 0.4.0, GXemul is completely free of native code back-ends.

### Emulation accuracy:

GXemul is an instruction-level emulator; things that would happen in several steps within a real CPU are not taken into account (e.g. pipe-line stalls or out-of-order execution). Still, instruction-level accuracy seems to be enough to be able to run complete guest operating systems inside the emulator.

The existance of instruction and data caches is "faked" to let operating systems think that they are there, but for all practical purposes, these caches are non-working.

The emulator is not timing-accurate. It can be run in a "deterministic" mode, `-D`. The meaning of deterministic is simply that running two emulations with the same settings will result in identical runs. Obviously, this requires that no user interaction is taking place, and that clock speeds are fixed with the `-I` option. (Deterministic in this case does not mean that the emulation will be identical to some actual real-world machine.)

(Note that user interaction means both input to the emulated program/OS, and interaction with the emulator's debugger. Breaking into the debugger and then continuing execution may affect when/how interrupts occur.)

### Which machines does GXemul emulate?

A few different machine types are emulated. The following machine types are emulated well enough to run at least one "guest OS":

- ARM
  - CATS (NetBSD/cats, OpenBSD/cats)
  - IQ80321 (NetBSD/evbarm)
  - NetWinder (NetBSD/netwinder)

- MIPS
  - DECstation 5000/200 (NetBSD/pmax, OpenBSD/pmax, Ultrix, Linux/DECstation, Sprite)
  - Acer Pica-61 (NetBSD/arc)
  - NEC MobilePro 770, 780, 800, and 880 (NetBSD/hpcmips)
  - Cobalt (NetBSD/cobalt)
  - Malta (NetBSD/evbmips)
  - Algorithmics P5064 (NetBSD/algor)
  - SGI O2 (aka IP32) (<super>*</super>) (NetBSD/sgi)

- PowerPC
  - IBM 6050/6070 (PReP, PowerPC Reference Platform) (NetBSD/prep)

(<super>*</super>) = Enough for root-on-nfs, but not for disk boot.)

There is code in GXemul for emulation of many other machine types; the degree to which these work range from almost being able to run a complete OS, to almost completely unsupported (perhaps just enough support to output a few boot messages via serial console).

In addition to emulating real machines, there is also a "test-machine". A test-machine consists of one or more CPUs and a few experimental devices such as:

- a console I/O device (putchar() and getchar()...)
- an inter-processor communication device, for SMP experiments
- a very simple linear framebuffer device (for graphics output)
- a simple SCSI disk controller
- a simple ethernet controller

This mode is useful if you wish to run experimental code, but do not wish to target any specific real-world machine type, for example for educational purposes.

You can read more about these experimental devices here.