### From The `sfront` Reference Manual by John Lazzaro and John Wawrzynek.

# Part II/2: Sfront File Rendering

## Sections

- Introduction.
- MP4 File Decoding. Rendering binary MP4-SA files to a file.
- SAOL, SASL and MIDI Decoding. Rendering MP4 component files.
- Audio Input. How to connect files to the input_bus.
- MP4 File Encoding. Creating MP4-SA files.
- Incompatibilities. Sfront deviations from the MP4-SA specification.

## Introduction

This chapter describes how to use `sfront` for off-line applications, where audio input and output are sent to files. These rendering applications work on all supported platforms.

We begin with decoding applications, including simple MP4 file rendering, and more complex jobs that use multiple file types (ASCII SAOL and SASL files, and binary MIDI Files).

We also describe how to use sfront to encode binary MP4 files. Finally, we describe the ways that sfront is not MP4-SA compliant.

##

## MP4 File Decoding

In Chapter I/1, we describe the set of example files included with sfront for testing purposes. These examples use sfront to create an audio file in an off-line fashion. We describe this sfront application as file rendering.

To render a binary MP4 file, simply run sfront and specify the binary file name using the `-bit` option (see the right panel for syntax). Running this command creates the file `sa.c`, that when compiled and executed, renders the audio performance in the WAV file `output.wav`.

The `-bit` option reads both the configuration block and the streaming blocks from the MP4 file, which is the proper behavior for most applications. The alternative `-bitc` option reads only the configuration block data, and ignores the streaming blocks.

Note that sfront uses the MP4 extension for binary files, to match the naming convention of the MPEG reference decoder saolc. Sfront MP4 files are compatible with saolc MP4 files.

However, these MP4 files do not use MPEG 4 Systems technology, and are incompatible with Structured Audio files that do use MPEG 4 Systems.

#### C filename option

The `-o` command-line option changes the C output filename from the default `sa.c`. See the right panel for examples.

#### Audio output options

The `-aout` command-line option overrides the default option of writing a WAV file names `output.wav`. If the string following the `-aout` option contains a period, sfront assumes the string is a filename, and uses the extension after the period to determine the filetype. See the right panel for examples.

At this time, sfront recognizes four extensions:

- .wav -- Microsoft WAV files
- .aif -- Electronic Arts AIF(F) files
- .raw -- a raw binary file
- .dat -- used by the Chipmunk tool view.

By default, WAV and AIFF files use 16-bit samples. 24-bit and 8-bit linear samples are also supported for WAV and AIFF files, via the syntax shown on the right panel.

If a raw file is selected, sfront writes a C file that generates a file containing channel-interleaved, 16-bit signed integers, in the native byte order of the machine. A companion ASCII file, fname.raw.info, is also written, which has three lines: line one is the sampling rate, line two is the number of channels, line three is the total number of samples in the file.

If a dat file is selected, sfront writes a C file that creates ASCII .dat files that are readable by the curve viewing and manipulation program view. View is a part of the Chipmunk tools, which are licensed under the GNU Public License. These tools run on most UNIX systems, but do not run under Microsoft Windows.

In addition to audio files names, the `-aout` command-line option recognizes the string `std`. If this option is used, channel-interleaved, 16-bit signed integers, in the native byte order of the machine, are streamed to standard output (i.e. `stdout`).

## File Rendering

`

    sfront -bit min.mp4

    creates sa.c file, that when
    compiled, renders a WAV file
    into output.wav, based on the
    binary MP4-SA file min.mp4.
    compiles both configuration
    and streaming data into the
    sa.c file.

    sfront -bitc min.mp4

    creates sa.c file that only
    includes information from
    the configuration block of
    the MP4-SA file min.mp4. Use
    for network streaming
    or for decoding mp4 files
    when the streaming block of
    the mp4 file is very large.

`

## C File Output

`

    sfront -bit min.mp4 -o min.c

    creates min.c file, that when
    compiled, renders a WAV file
    into output.wav

`

## Audio File Output

`

    sfront -bit min.mp4 -aout new.wav

    Creates an sa.c file that creates
    the WAV file new.wav (16-bit samples).

    sfront -bit min.mp4 -aout new.wav-24
    sfront -bit min.mp4 -aout new.wav-8

    Syntax to specify 24-bit and 8-bit
    samples (created file is new.wav).

    sfront -bit min.mp4 -aout new.aif

    Creates an sa.c file that creates
    the AIF file new.aif (16-bit samples).

    sfront -bit min.mp4 -aout new.aif-24
    sfront -bit min.mp4 -aout new.aif-8

    Syntax to specify 24-bit and 8-bit
    samples (created file is new.aif).

    sfront -bit min.mp4 -aout new.dat

    Creates an sa.c file that creates
    the view file new.dat for SAOL
    programs with outchannels = 1,
    and the files new1.dat, new2.dat,
    new3.dat ... for SAOL programs
    with outchannels > 1.

    sfront -bit min.mp4 -aout new.raw

    Creates an sa.c file that creates
    the files new.raw and new.raw.info,
    in the format described on the
    left panel.

    sfront -bit min.mp4 -aout std

    Creates an sa.c file that sends
    the audio stream to stdout, using
    the format described on the left
    panel.

    sfront -bit min.mp4 -aout null

    Creates an sa.c file that discards
    the audio stream. Use for testing.

`

## SAOL, SASL and MIDI File Decoding

The last section described how to use sfront to decode MP4 binary files. Sfront can also decode ASCII SAOL and SASL files and binary MIDI files directly. Note that if these component file options are used, the MP4 file `-bit` option may not be used.

The right panel shows how to use the `-orc` option to specify one or more SAOL files, the `-sco` option to specify a single SASL file, and the `-midi` option to specify a single binary MIDI file. Note that, per the MP4-SA specification, only Format 0 and Format 1 MIDI files are processed by sfront.

Sfront also uses component options to specify score data for encoding MP4 files. Files specified using `-sco` and `-midi` options are encoded as configuration data at the start of the MP4 file. In addition, the options `-sstr` and `-mstr` can be used to encode SASL and MIDI files as streaming events at the end of the MP4 file.

#### MIDI Verbose Mode

A common use for sfront is to normatively orchestrate MIDI files. Since MIDI files are binary, it is difficult to figure out the extended-channel numbers and preset numbers to use in SAOL instruments.

The `-mv` option generates an ASCII listing of the MIDI file specified by the `-midi` option. This listing highlights the information the SAOL programmer requires to orchestrate a MIDI file.

## ASCII File Decoding

`

    sfront -orc min.saol -sco min.sasl

    Decodes the MP4-SA performance
    represented by the ASCII SAOL
    file min.saol and the ASCII
    SASL file min.sasl.

    sfront -orc t1.saol t2.saol

    Decodes the MP4-SA performance
    represented by the ASCII SAOL
    files t1.saol and t2.saol.

`

## MIDI File Decoding

`

    sfront -orc p.saol -midi p.mid
    sfront -mv -orc p.saol -midi p.mid

    The first line decodes the MP4-SA
    performance represented by the ASCII
    SAOL file perc.saol and the binary
    MIDI file perc.mid. The second line
    also prints out information about
    the MIDI file, useful for SAOL coding.

`

## Audio Input

SAOL programs can process external audio input placed on the input_bus. The sfront command-line option `-ain` sets the audio input source file for the C program created by sfront.

The right panel shows the file arguments for the `-ain` option. Like the `-aout` option, the file type is deduced from the file name extension.

Sfront follows the rules set forth in the MPEG standard to determine the input_bus width and audio sampling rate of programs that use the input_bus.

Specifically, if the global parameters inchannels or srate are set, their values become the input_bus width and audio sampling rate of the program.

If either of these parameters are not set, sfront queries the audio input device for a suitable parameter. For example, if audio input is a file, sfront opens the file and checks its sampling rate and width.

Note that these checks occur when sfront runs, not when the C program sfront creates runs.

## Audio File Input

`

     option       behavior
    --------------------------
    -ain fn.wav   Reads from the
                  WAV file fn.wav
                  (the default option,
                  reading input.wav).

    -ain fn.aif   Reads from the
                  AIF file fn.aif

    -ain fn.dat   Reads from the
                  .dat file fn.dat.
                  Only reads the first
                  curve, data sent
                  to all inchannels.

    -ain fn.raw   Reads the raw binary
                  file fn.raw, as
                  channel-interleaved,
                  16-bit signed integers.

    -ain std      Reads from standard
                  input (stdin) as
                  channel-interleaved
                  16-bit signed integers.

    -ain null     Reads silence.

`

## MP4 File Encoding

The right panel shows how to use `-bitout` option of sfront to generate a binary MP4 file. The binary file encodes the SAOL, SASL, and MIDI files specified on the command line, using the syntax shown in a previous section.

The MP4 file contains a binary encoding of the program files, as well as the sample data specified by any sample core wavetable generators used in the SAOL and SASL files.

The MP4 file format has two ways to store SASL and MIDI information: as configuration data at the start of the file, or as streaming events at the end of the file. The right panel shows an example of both types of encoding: the `-sco` and `-midi` options specify score data for the configuration section, and the `-sstr` and `-mstr` options specify score data for the streaming section.

If score data is encoded as streaming events, the sfront networking system can stream it to remote clients. However, this encoding does result in larger mp4 files, compared to encoding score data in the configuration block.

#### Regeneration

When decoding a binary MP4 file, sfront can recreate the SAOL, SASL, and MIDI files that make up the file. See the right panel for an example. Note that sfront always creates WAV files for any sound samples in an MP4 file, which the C program reads when executed.

By default, when encoding an MP4, the MP4 file does not include a symbol table. As a result, SAOL and SASL files regenerated from these MP4 files have randomly-generated variable and instrument names.

To encode MP4 files with the original variable names, use the `-symtab` option to include a symbol table. Note that this produces significantly larger MP4 files.

## MP4 File Encoding

`

    This command creates the MP4
    file x.mp4:

    sfront -bitout x.mp4 -orc s.saol
           -sco l.sasl

    Both SAOL and SASL information
    is encoded in the configuration
    section at the start of x.mp4.

    To extract the SAOL and SASL
    components from x.mp4, use:

    sfront -bit x.mp4 -orcout s2.saol
           -scoout l2.sasl

    The logic of s.saol and s2.saol
    is identical, but the SAOL
    variable names may be different.
    Add the -symtab option to the
    encoding line to preserve names.

    ---

    This command creates the MP4
    file y.mp4:

    sfront -bitout y.mp4 -orc s.saol
           -sstr l.sasl

    In y.mp4, the SAOL information
    is encoded in the configuration
    section at the start of the file,
    and the SASL information is encoded
    as a list of streaming events at
    the end of the file.

    Sfront networking can stream
    MIDI score data encoded as
    streaming events, using the -mstr
    option.

    ---

    Note that streaming (-sstr, -mstr)
    and configuration (-sco, -midi)
    options may be used together, to
    distribute score data in both parts
    of the MP4 file.

`

## Incompatibilities

Sfront translates MPEG4 Structured Audio Object 3 bitstreams (plus MIDI functionality from Object 4).

It does not handle Object 1 (equivalent to MIDI files), Object 2 (sasbf synthesis driven by MIDI files), or Object 4 bitstreams (Objects 1-3, plus sasbf and MIDI file integration into SAOL).

In addition, sfront is currently non-compliant in the following major ways:

1. If the SAOL global parameter interp set to 1, sfront implements band-limited interpolation for most core opcodes that use interpolation. However, the grain() opcode uses linear interpolation for envelope and wave tables.
2. AudioFX-related core opcodes are implemented, since sfront isn't integrated into an MPEG 4 system. However, AudioFX related standard names are available to the control driver interface for reading and writing, and SAOL programs can see these changes.
3. Array, oparray, and tablemap indexes are not checked for illegal array index values. This omission has security ramifications for sfront networking users.

Next section: Part I/3: Real-Time Operation

Copyright 1999 John Lazzaro and John Wawrzynek.