    FORMULA:  Syntax for arithmetic formulas.

    Arithmetic formulas are used to specify a distribution (see dist.doc),
    and by the calculator program (see calc.doc).  These formulas are
    written in pretty much the usual way, with the following conventions:

      1) A variable name is either a single lower-case letter (a-z) or a
         lower-case letter followed by a single digit (0-9).  Note,
         however, that when a formula is used to specify a distribution,
         the allowed state variables are restricted (see dist.doc).

      2) Function names start with an upper-case letter (eg, Sin or Sqrt).

      3) As usual, addition and subtraction (+ and -) have lowest
         precedence, with multiplication and division (* and /) having
         higher precedence, and exponentiation (^) having highest
         precedence.  Evaluation is otherwise left to right.

      4) The multiplication operator can be omitted.  Note, however, that
         x6 is a variable name, not x*6 (but x 6 and 6x are products).

      5) Only positive integer constants are allowed as exponents (eg,
         x^2 is allowed, but not x^y or x^0.5).

      6) Three kinds of brackets can be used: (), [], and {}.  There's no
         difference in their effects, but they must be properly matched.

      7) Scientific notation uses "E", not "e" - eg, 2.1E-10, not 2.1e-10.

    The constant Pi=3.14159... is defined.  Use Exp(1), defined below, to
    get the value of e=2.71828...

    A number of functions are built in; others can be defined by external
    programs, as described below.  The following univariate functions are
    built in:

      Sin   Cos   Tan   Log   Exp     Abs   Theta  Delta
      Sinh  Cosh  Tanh  Sqrt  LGamma  Frac  Sign

    These compute the obvious things, except as follows:

      LGamma  Computes the log of the Gamma function.

      Frac    Computes the fractional part of a number (eg, Frac(2.7) is
              0.7, Frac(3.0) is 0, and Frac(-0.1) is 0.9).

      Sign    Finds the sign of its argument, -1 or +1, or 0.

      Theta   Has the value 1 for non-negative arguments, 0 for negative
              arguments.

      Delta   Has the value 1 for an argument of zero, 0 for non-zero
              arguments.

    A special function LogSumExp, which takes an indefinite number of
    arguments, is also provided.  It computes the log of the sum of the
    exponentials of its arguments.  This is done in such a way as to avoid
    overflow whenever the final result is within range.  For example
    LogSumExp(1000,2000,1500) evaluates to 2000 (or very close to it),
    without overflow occurring.

    The following functions return minus the log of the probability
    density for a quantity with respect to some distribution:

      Normal  Gaussian  ExpGamma  ExpGamma2

    The Normal (equivalently, Gaussian) function computes minus the log of
    the probability density for the normal distribution with the given
    mean and variance.  That is, Normal(x,m,v) is equivalent to

      Log(2*Pi*v)/2 + (x-m)^2/(2*v)

    The ExpGamma function computes minus the log of the probability
    density for a quantity whose exponential has the Gamma distribution
    with the given shape and mean.  ExpGamma(x,s,b) is equivalent to

      - s*Log(b) + LGamma(s) + Exp(x)*b - s*x

    ExpGamma2(x,a,m) is an alternative parameterization, defined by

      ExpGamma2 (x, a, m) = ExpGamma (x, a/2, (a/2)/m)

    This is closer to the way priors are specified for neural network and
    other models.  For example, the log of a hyperparameter whose prior is
    specified by "5:3" has the ExpGamma2(3,1/5^2) distribution.

    If the first parameter of one of these density functions is a
    variable, it can be written as follows instead:

      x ~ Normal(m,v)     is equivalent to   Normal(x,m,v)
      x ~ Gaussian(m,v)   is equivalent to   Gaussian(x,m,v)
      x ~ ExpGamma(s,b)   is equivalent to   ExpGamma(x,s,b)
      x ~ ExpGamma2(a,m)  is equivalent to   ExpGamma2(x,a,m)

    Use of this form can allow random number generation by some programs.

    Errors in computations (eg, division by zero) are handled in whatever
    way C handles them.  Multiplications and divisions in which the first
    operand evaluates to zero evaluate to zero without evaluating the
    second operand being evaluated.  For example, 0*Log(0) will evaluate
    to zero, whereas Log(0)*0 might generate an error.  (However, this
    shortcut does not apply when derivatives are also being computed.)

    When formulas are given as arguments to commands, they must often be
    quoted, to prevent the shell from interpreting the special characters
    they contain.

    The detailed syntax of formulas is as follows, with | indicating
    alternatives, [] indicating optional parts, and {} indicating parts
    that can occur zero or more times).

     <formula>   ::= [ <term> ] { <plusminus> <term> }
     <plusminus> ::= "+" | "-"

     <term>      ::=  <factor> { <timesdiv> <factor> }
     <timesdiv>  ::= [ "*" ] | "/"

     <factor>    ::= <prefactor> [ "^" <integer> ]
     <prefactor> ::= "(" <formula> ")" | "[" <formula> "]" | "{" <formula> "}"
                   | <variable> | <constant> | <number>
                   | <ufunction> <prefactor> | <dfunction> <arglist>
                   | <variable> ~ <dfunction> <arglist>
                   | "LogSumExp" <arglist>
     <arglist>   ::= "(" <formula> { "," <formula> } ")"
                   | "[" <formula> { "," <formula> } "]"
                   | "{" <formula> { "," <formula> } "}"

     <ufunction> ::= <one of the univariate functions built in or defined externally>
     <dfunction> ::= <one of the log density functions built in or defined externally>
     <variable>  ::= <lower-case-letter> [ <digit> ]
     <constant>  ::= Pi
     <number>    ::= <integer> [ "." { <digit> } ] [ <exponent> ]
                   | "." <digit> { <digit> } [ <exponent> ]
     <exponent>  ::= "E" [ <plusminus> ] <integer>
     <integer>   ::= <digit> { <digit> }

    Spaces may be inserted anywhere except inside variable names, function
    names, or numbers.

    See calc.doc for some examples of formulas.

    Defining new functions.

    A new function can be defined by writing a program for computing its
    value, typically in C.  New distributions can also be defined in this
    way.  However, at present, gradients for these functions are not
    handled, and random generation from distributions is also not
    implemented.

    Such an externally-defined function or distribution must have a name
    consisting of an upper-case letter followed by zero or more upper-case
    or lower-case letters or digits.  A program with this same name must
    exist in the current directory.  When the value of the function is
    first needed, this program will be started as a subprocess connected
    via pipes replacing standard input and output.  The program should
    consist of a loop that reads a request to evaluate the function for
    specific values of its arguments from standard input, and then writes
    the corresponding function value to standard output.  The program
    should exit when it encounters EOF on standard input.

    Each function evaluation request will start with the following header,
    defined in extfunc.h in the util directory:

      typedef struct
      { enum { Value, Value_and_gradient, Random_variate} want;
        int n_args;
      } ext_header;

    This should be read from standard input using fread.  The 'want' field
    says what type of operation has been requested.  Currently, only
    'Value' is used.  The 'n_args' field says how many arguments were
    passed to the function.  After the header has been read, this many
    double-precision floating-point arguments should be read from standard
    input using fread.  The value of the function with these arguments
    should then be written to standard output as a double-precision
    floating-point number using fwrite, and the output forced out using
    fflush.

    Note that this procedure makes use of Unix-specific facilities.

                Copyright (c) 1995-2003 by Radford M. Neal