Overview Of SBCL, How It Works And Where It Came From
Prev | Chapter 1. Introduction | Next

## Overview Of SBCL, How It Works And Where It Came From

You can work productively with SBCL without knowing anything understanding anything about where it came from, how it is implemented, or how it extends the ANSI Common Lisp standard. However, a little knowledge can be helpful in order to understand error messages, to troubleshoot problems, to understand why some parts of the system are better debugged than others, and to anticipate which known bugs, known performance problems, and missing extensions are likely to be fixed, tuned, or added.

SBCL is descended from CMU CL, which is itself descended from Spice Lisp, including early implementations for the Mach operating system on the IBM RT, back in the 1980s. Design decisions from that time are still reflected in the current implementation:

- The system expects to be loaded into a fixed-at-compile-time location in virtual memory, and also expects the location of all of its heap storage to be specified at compile time.

- The system overcommits memory, allocating large amounts of address space from the system (often more than the amount of virtual memory available) and then failing if ends up using too much of the allocated storage.

- A word is a 32-bit quantity. The system has been ported to many processor architectures without altering this basic principle. Some hacks allow the system to run on the Alpha chip (a 64-bit architecture) but even there 32-bit words are used. The assumption that a word is 32 bits wide is implicit in hundreds of places in the system.

- The system is implemented as a C program which is responsible for supplying low-level services and loading a Lisp ‘.core’ file.

SBCL also inherited some newer architectural features from CMU CL. The most important is that it has a generational garbage collector (“GC”), which has various implications (mostly good) for performance. These are discussed in another chapter.

SBCL has diverged from CMU CL in that SBCL is now essentially a “compiler-only implementation” of Common Lisp. A Common Lisp implementation is permitted to implement both a compiler and an interpreter, and there's some special support in the standard (e.g. the distinction between `functionp` and `compiled-function-p`) to help support that. But SBCL has only a vestigial, rudimentary true interpreter. In SBCL, the `eval` function only truly “interprets” a few special classes of forms, such as symbols which are `boundp`. More complicated forms are evaluated by calling `compile` and then calling `funcall` on the returned result.

The direct ancestor of SBCL is the X86 port of CMU CL. This port was in some ways the most cobbled-together of all the CMU CL ports, since a number of strange changes had to be made to support the register-poor X86 architecture. Some things (like tracing and debugging) do not work particularly well there. SBCL should be able to improve in these areas (and has already improved in some other areas), but it takes a while.

On the x86, SBCL like the X86 port of CMU CL, uses a conservative GC. This means that it doesn't maintain a strict separation between tagged and untagged data, instead treating some untagged data (e.g. raw floating point numbers) as possibly-tagged data and so not collecting any Lisp objects that they point to. This has some negative consequences for average time efficiency (though possibly no worse than the negative consequences of trying to implement an exact GC on a processor architecture as register-poor as the X86) and also has potentially unlimited consequences for worst-case memory efficiency. In practice, conservative garbage collectors work reasonably well, not getting anywhere near the worst case. But they can occasionally cause odd patterns of memory usage.

The fork from CMU CL was based on a major rewrite of the system bootstrap process. CMU CL has for many years tolerated a very unusual “build” procedure which doesn't actually build the complete system from scratch, but instead progressively overwrites parts of a running system with new versions. This quasi-build procedure can cause various bizarre bootstrapping hangups, especially when a major change is made to the system. It also makes the connection between the current source code and the current executable more tenuous than in other software systems -- it's easy to accidentally “build” a CMU CL system containing characteristics not reflected in the current version of the source code.

Other major changes since the fork from CMU CL include

- SBCL has dropped support for many CMU CL extensions, (e.g. IP networking, remote procedure call, Unix system interface, and X11 interface). Some of these are now available as contributed or third-party modules.

- SBCL has deleted or deprecated some nonstandard features and code complexity which helped efficiency at the price of maintainability. For example, the SBCL compiler no longer implements memory pooling internally (and so is simpler and more maintainable, but generates more garbage and runs more slowly), and various block-compilation efficiency-increasing extensions to the language have been deleted or are no longer used in the implementation of SBCL itself.

Prev | Up | Next
Where To Go For More Information About SBCL | Home | Chapter 2. The Compiler