This is ../info/lispref.info, produced by makeinfo version 4.7 from
lispref/lispref.texi.

INFO-DIR-SECTION XEmacs Editor
START-INFO-DIR-ENTRY
* Lispref: (lispref).		XEmacs Lisp Reference Manual.
END-INFO-DIR-ENTRY

   Edition History:

   GNU Emacs Lisp Reference Manual Second Edition (v2.01), May 1993 GNU
Emacs Lisp Reference Manual Further Revised (v2.02), August 1993 Lucid
Emacs Lisp Reference Manual (for 19.10) First Edition, March 1994
XEmacs Lisp Programmer's Manual (for 19.12) Second Edition, April 1995
GNU Emacs Lisp Reference Manual v2.4, June 1995 XEmacs Lisp
Programmer's Manual (for 19.13) Third Edition, July 1995 XEmacs Lisp
Reference Manual (for 19.14 and 20.0) v3.1, March 1996 XEmacs Lisp
Reference Manual (for 19.15 and 20.1, 20.2, 20.3) v3.2, April, May,
November 1997 XEmacs Lisp Reference Manual (for 21.0) v3.3, April 1998

   Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995 Free Software
Foundation, Inc.  Copyright (C) 1994, 1995 Sun Microsystems, Inc.
Copyright (C) 1995, 1996 Ben Wing.

   Permission is granted to make and distribute verbatim copies of this
manual provided the copyright notice and this permission notice are
preserved on all copies.

   Permission is granted to copy and distribute modified versions of
this manual under the conditions for verbatim copying, provided that the
entire resulting derived work is distributed under the terms of a
permission notice identical to this one.

   Permission is granted to copy and distribute translations of this
manual into another language, under the above conditions for modified
versions, except that this permission notice may be stated in a
translation approved by the Foundation.

   Permission is granted to copy and distribute modified versions of
this manual under the conditions for verbatim copying, provided also
that the section entitled "GNU General Public License" is included
exactly as in the original, and provided that the entire resulting
derived work is distributed under the terms of a permission notice
identical to this one.

   Permission is granted to copy and distribute translations of this
manual into another language, under the above conditions for modified
versions, except that the section entitled "GNU General Public License"
may be included in a translation approved by the Free Software
Foundation instead of in the original English.


File: lispref.info,  Node: Network,  Prev: Transaction Queues,  Up: Processes

56.13 Network Connections
=========================

XEmacs Lisp programs can open TCP network connections to other
processes on the same machine or other machines.  A network connection
is handled by Lisp much like a subprocess, and is represented by a
process object.  However, the process you are communicating with is not
a child of the XEmacs process, so you can't kill it or send it signals.
All you can do is send and receive data.  `delete-process' closes the
connection, but does not kill the process at the other end; that
process must decide what to do about closure of the connection.

   You can distinguish process objects representing network connections
from those representing subprocesses with the `process-status'
function.  It always returns either `open' or `closed' for a network
connection, and it never returns either of those values for a real
subprocess.  *Note Process Information::.

 -- Function: open-network-stream name buffer-or-name host service
          &optional protocol
     This function opens a TCP connection for a service to a host.  It
     returns a process object to represent the connection.

     Input and output work as for other process objects.
     `delete-process' closes the connection.

     The NAME argument specifies the name for the process object.  It
     is modified as necessary to make it unique.

     The BUFFER-OR-NAME argument is the buffer to associate with the
     connection.  It can be a buffer or the name of one.  Output from
     the connection is inserted in the buffer, unless you specify a
     filter function to handle the output.  If BUFFER-OR-NAME is `nil',
     it means that the connection is not associated with any buffer.

     The arguments HOST and SERVICE specify where to connect to; HOST
     is the host name or IP address (a string), and SERVICE is the name
     of a defined network service (a string) or a port number (an
     integer).

     Optional fifth arg PROTOCOL is the network protocol to use.
     Currently only `tcp' (Transmission Control Protocol) and `udp'
     (User Datagram Protocol) are supported.  When omitted, `tcp' is
     assumed.

     Output via `process-send-string' and input via buffer or filter
     (see `set-process-filter') are stream-oriented.  That means UDP
     datagrams are not guaranteed to be sent and received in discrete
     packets. (But small datagrams around 500 bytes that are not
     truncated by `process-send-string' are usually fine.)  Note further
     that the UDP protocol does not guard against lost packets.


File: lispref.info,  Node: System Interface,  Next: X-Windows,  Prev: Processes,  Up: Top

57 Operating System Interface
*****************************

This chapter is about starting and getting out of Emacs, access to
values in the operating system environment, and terminal input, output,
and flow control.

   *Note Building XEmacs::, for related information.  See also *Note
Display::, for additional operating system status information
pertaining to the terminal and the screen.

* Menu:

* Starting Up::         Customizing XEmacs start-up processing.
* Getting Out::         How exiting works (permanent or temporary).
* System Environment::  Distinguish the name and kind of system.
* User Identification:: Finding the name and user id of the user.
* Time of Day::		Getting the current time.
* Time Conversion::     Converting a time from numeric form to a string, or
                          to calendrical data (or vice versa).
* Timers::		Setting a timer to call a function at a certain time.
* Terminal Input::      Recording terminal input for debugging.
* Terminal Output::     Recording terminal output for debugging.
* Flow Control::        How to turn output flow control on or off.
* Batch Mode::          Running XEmacs without terminal interaction.


File: lispref.info,  Node: Starting Up,  Next: Getting Out,  Up: System Interface

57.1 Starting Up XEmacs
=======================

This section describes what XEmacs does when it is started, and how you
can customize these actions.

* Menu:

* Start-up Summary::        Sequence of actions XEmacs performs at start-up.
* Init File::               Details on reading the init file (`.emacs').
* Terminal-Specific::       How the terminal-specific Lisp file is read.
* Command Line Arguments::  How command line arguments are processed,
                              and how you can customize them.


File: lispref.info,  Node: Start-up Summary,  Next: Init File,  Up: Starting Up

57.1.1 Summary: Sequence of Actions at Start Up
-----------------------------------------------

The order of operations performed (in `startup.el') by XEmacs when it
is started up is as follows:

  1. It loads the initialization library for the window system, if you
     are using a window system.  This library's name is
     `term/WINDOWSYSTEM-win.el'.

  2. It processes the initial options.  (Some of them are handled even
     earlier than this.)

  3. It initializes the X window frame and faces, if appropriate.

  4. It runs the normal hook `before-init-hook'.

  5. It loads the library `site-start', unless the option
     `-no-site-file' was specified.  The library's file name is usually
     `site-start.el'.  

  6. It loads the file `~/.emacs' unless `-q' was specified on the
     command line.  (This is not done in `-batch' mode.)  The `-u'
     option can specify the user name whose home directory should be
     used instead of `~'.

  7. It loads the library `default' unless `inhibit-default-init' is
     non-`nil'.  (This is not done in `-batch' mode or if `-q' was
     specified on the command line.)  The library's file name is
     usually `default.el'.  

  8. It runs the normal hook `after-init-hook'.

  9. It sets the major mode according to `initial-major-mode', provided
     the buffer `*scratch*' is still current and still in Fundamental
     mode.

 10. It loads the terminal-specific Lisp file, if any, except when in
     batch mode or using a window system.

 11. It displays the initial echo area message, unless you have
     suppressed that with `inhibit-startup-echo-area-message'.

 12. It processes the action arguments from the command line.

 13. It runs `term-setup-hook'.

 14. It calls `frame-notice-user-settings', which modifies the
     parameters of the selected frame according to whatever the init
     files specify.

 15. It runs `window-setup-hook'.  *Note Terminal-Specific::.

 16. It displays copyleft, nonwarranty, and basic use information,
     provided there were no remaining command line arguments (a few
     steps above) and the value of `inhibit-startup-message' is `nil'.

 -- User Option: inhibit-startup-message
     This variable inhibits the initial startup messages (the
     nonwarranty, etc.).  If it is non-`nil', then the messages are not
     printed.

     This variable exists so you can set it in your personal init file,
     once you are familiar with the contents of the startup message.
     Do not set this variable in the init file of a new user, or in a
     way that affects more than one user, because that would prevent
     new users from receiving the information they are supposed to see.

 -- User Option: inhibit-startup-echo-area-message
     This variable controls the display of the startup echo area
     message.  You can suppress the startup echo area message by adding
     text with this form to your `.emacs' file:

          (setq inhibit-startup-echo-area-message
                "YOUR-LOGIN-NAME")

     Simply setting `inhibit-startup-echo-area-message' to your login
     name is not sufficient to inhibit the message; Emacs explicitly
     checks whether `.emacs' contains an expression as shown above.
     Your login name must appear in the expression as a Lisp string
     constant.

     This way, you can easily inhibit the message for yourself if you
     wish, but thoughtless copying of your `.emacs' file will not
     inhibit the message for someone else.


File: lispref.info,  Node: Init File,  Next: Terminal-Specific,  Prev: Start-up Summary,  Up: Starting Up

57.1.2 The Init File: `.emacs'
------------------------------

When you start XEmacs, it normally attempts to load the file `.emacs'
from your home directory.  This file, if it exists, must contain Lisp
code.  It is called your "init file".  The command line switches `-q'
and `-u' affect the use of the init file; `-q' says not to load an init
file, and `-u' says to load a specified user's init file instead of
yours.  *Note Entering XEmacs: (xemacs)Entering XEmacs.

   A site may have a "default init file", which is the library named
`default.el'.  XEmacs finds the `default.el' file through the standard
search path for libraries (*note How Programs Do Loading::).  The
XEmacs distribution does not come with this file; sites may provide one
for local customizations.  If the default init file exists, it is
loaded whenever you start Emacs, except in batch mode or if `-q' is
specified.  But your own personal init file, if any, is loaded first; if
it sets `inhibit-default-init' to a non-`nil' value, then XEmacs does
not subsequently load the `default.el' file.

   Another file for site-customization is `site-start.el'.  Emacs loads
this _before_ the user's init file.  You can inhibit the loading of
this file with the option `-no-site-file'.

 -- Variable: site-run-file
     This variable specifies the site-customization file to load before
     the user's init file.  Its normal value is `"site-start"'.

   If there is a great deal of code in your `.emacs' file, you should
move it into another file named `SOMETHING.el', byte-compile it (*note
Byte Compilation::), and make your `.emacs' file load the other file
using `load' (*note Loading::).

   *Note Init File Examples: (xemacs)Init File Examples, for examples
of how to make various commonly desired customizations in your `.emacs'
file.

 -- User Option: inhibit-default-init
     This variable prevents XEmacs from loading the default
     initialization library file for your session of XEmacs.  If its
     value is non-`nil', then the default library is not loaded.  The
     default value is `nil'.

 -- Variable: before-init-hook
 -- Variable: after-init-hook
     These two normal hooks are run just before, and just after,
     loading of the user's init file, `default.el', and/or
     `site-start.el'.


File: lispref.info,  Node: Terminal-Specific,  Next: Command Line Arguments,  Prev: Init File,  Up: Starting Up

57.1.3 Terminal-Specific Initialization
---------------------------------------

Each terminal type can have its own Lisp library that XEmacs loads when
run on that type of terminal.  For a terminal type named TERMTYPE, the
library is called `term/TERMTYPE'.  XEmacs finds the file by searching
the `load-path' directories as it does for other files, and trying the
`.elc' and `.el' suffixes.  Normally, terminal-specific Lisp library is
located in `emacs/lisp/term', a subdirectory of the `emacs/lisp'
directory in which most XEmacs Lisp libraries are kept.

   The library's name is constructed by concatenating the value of the
variable `term-file-prefix' and the terminal type.  Normally,
`term-file-prefix' has the value `"term/"'; changing this is not
recommended.

   The usual function of a terminal-specific library is to enable
special keys to send sequences that XEmacs can recognize.  It may also
need to set or add to `function-key-map' if the Termcap entry does not
specify all the terminal's function keys.  *Note Terminal Input::.

   When the name of the terminal type contains a hyphen, only the part
of the name before the first hyphen is significant in choosing the
library name.  Thus, terminal types `aaa-48' and `aaa-30-rv' both use
the `term/aaa' library.  If necessary, the library can evaluate
`(getenv "TERM")' to find the full name of the terminal type.

   Your `.emacs' file can prevent the loading of the terminal-specific
library by setting the variable `term-file-prefix' to `nil'.  This
feature is useful when experimenting with your own peculiar
customizations.

   You can also arrange to override some of the actions of the
terminal-specific library by setting the variable `term-setup-hook'.
This is a normal hook which XEmacs runs using `run-hooks' at the end of
XEmacs initialization, after loading both your `.emacs' file and any
terminal-specific libraries.  You can use this variable to define
initializations for terminals that do not have their own libraries.
*Note Hooks::.

 -- Variable: term-file-prefix
     If the `term-file-prefix' variable is non-`nil', XEmacs loads a
     terminal-specific initialization file as follows:

          (load (concat term-file-prefix (getenv "TERM")))

     You may set the `term-file-prefix' variable to `nil' in your
     `.emacs' file if you do not wish to load the
     terminal-initialization file.  To do this, put the following in
     your `.emacs' file: `(setq term-file-prefix nil)'.

 -- Variable: term-setup-hook
     This variable is a normal hook that XEmacs runs after loading your
     `.emacs' file, the default initialization file (if any) and the
     terminal-specific Lisp file.

     You can use `term-setup-hook' to override the definitions made by a
     terminal-specific file.

 -- Variable: window-setup-hook
     This variable is a normal hook which XEmacs runs after loading your
     `.emacs' file and the default initialization file (if any), after
     loading terminal-specific Lisp code, and after running the hook
     `term-setup-hook'.


File: lispref.info,  Node: Command Line Arguments,  Prev: Terminal-Specific,  Up: Starting Up

57.1.4 Command Line Arguments
-----------------------------

You can use command line arguments to request various actions when you
start XEmacs.  Since you do not need to start XEmacs more than once per
day, and will often leave your XEmacs session running longer than that,
command line arguments are hardly ever used.  As a practical matter, it
is best to avoid making the habit of using them, since this habit would
encourage you to kill and restart XEmacs unnecessarily often.  These
options exist for two reasons: to be compatible with other editors (for
invocation by other programs) and to enable shell scripts to run
specific Lisp programs.

   This section describes how Emacs processes command line arguments,
and how you can customize them.

 -- Function: command-line
     This function parses the command line that XEmacs was called with,
     processes it, loads the user's `.emacs' file and displays the
     startup messages.

 -- Variable: command-line-processed
     The value of this variable is `t' once the command line has been
     processed.

     If you redump XEmacs by calling `dump-emacs', you may wish to set
     this variable to `nil' first in order to cause the new dumped
     XEmacs to process its new command line arguments.

 -- Variable: command-switch-alist
     The value of this variable is an alist of user-defined command-line
     options and associated handler functions.  This variable exists so
     you can add elements to it.

     A "command line option" is an argument on the command line of the
     form:

          -OPTION

     The elements of the `command-switch-alist' look like this:

          (OPTION . HANDLER-FUNCTION)

     The HANDLER-FUNCTION is called to handle OPTION and receives the
     option name as its sole argument.

     In some cases, the option is followed in the command line by an
     argument.  In these cases, the HANDLER-FUNCTION can find all the
     remaining command-line arguments in the variable
     `command-line-args-left'.  (The entire list of command-line
     arguments is in `command-line-args'.)

     The command line arguments are parsed by the `command-line-1'
     function in the `startup.el' file.  See also *Note Command Line
     Switches and Arguments: (xemacs)Command Switches.

 -- Variable: command-line-args
     The value of this variable is the list of command line arguments
     passed to XEmacs.

 -- Variable: command-line-functions
     This variable's value is a list of functions for handling an
     unrecognized command-line argument.  Each time the next argument
     to be processed has no special meaning, the functions in this list
     are called, in order of appearance, until one of them returns a
     non-`nil' value.

     These functions are called with no arguments.  They can access the
     command-line argument under consideration through the variable
     `argi'.  The remaining arguments (not including the current one)
     are in the variable `command-line-args-left'.

     When a function recognizes and processes the argument in `argi', it
     should return a non-`nil' value to say it has dealt with that
     argument.  If it has also dealt with some of the following
     arguments, it can indicate that by deleting them from
     `command-line-args-left'.

     If all of these functions return `nil', then the argument is used
     as a file name to visit.


File: lispref.info,  Node: Getting Out,  Next: System Environment,  Prev: Starting Up,  Up: System Interface

57.2 Getting out of XEmacs
==========================

There are two ways to get out of XEmacs: you can kill the XEmacs job,
which exits permanently, or you can suspend it, which permits you to
reenter the XEmacs process later.  As a practical matter, you seldom
kill XEmacs--only when you are about to log out.  Suspending is much
more common.

* Menu:

* Killing XEmacs::        Exiting XEmacs irreversibly.
* Suspending XEmacs::     Exiting XEmacs reversibly.


File: lispref.info,  Node: Killing XEmacs,  Next: Suspending XEmacs,  Up: Getting Out

57.2.1 Killing XEmacs
---------------------

Killing XEmacs means ending the execution of the XEmacs process.  The
parent process normally resumes control.  The low-level primitive for
killing XEmacs is `kill-emacs'.

 -- Command: kill-emacs &optional exit-data
     This function exits the XEmacs process and kills it.

     If EXIT-DATA is an integer, then it is used as the exit status of
     the XEmacs process.  (This is useful primarily in batch operation;
     see *Note Batch Mode::.)

     If EXIT-DATA is a string, its contents are stuffed into the
     terminal input buffer so that the shell (or whatever program next
     reads input) can read them.

   All the information in the XEmacs process, aside from files that have
been saved, is lost when the XEmacs is killed.  Because killing XEmacs
inadvertently can lose a lot of work, XEmacs queries for confirmation
before actually terminating if you have buffers that need saving or
subprocesses that are running.  This is done in the function
`save-buffers-kill-emacs'.

 -- Variable: kill-emacs-query-functions
     After asking the standard questions, `save-buffers-kill-emacs'
     calls the functions in the list `kill-buffer-query-functions', in
     order of appearance, with no arguments.  These functions can ask
     for additional confirmation from the user.  If any of them returns
     non-`nil', XEmacs is not killed.

 -- Variable: kill-emacs-hook
     This variable is a normal hook; once `save-buffers-kill-emacs' is
     finished with all file saving and confirmation, it runs the
     functions in this hook.


File: lispref.info,  Node: Suspending XEmacs,  Prev: Killing XEmacs,  Up: Getting Out

57.2.2 Suspending XEmacs
------------------------

"Suspending XEmacs" means stopping XEmacs temporarily and returning
control to its superior process, which is usually the shell.  This
allows you to resume editing later in the same XEmacs process, with the
same buffers, the same kill ring, the same undo history, and so on.  To
resume XEmacs, use the appropriate command in the parent shell--most
likely `fg'.

   Some operating systems do not support suspension of jobs; on these
systems, "suspension" actually creates a new shell temporarily as a
subprocess of XEmacs.  Then you would exit the shell to return to
XEmacs.

   Suspension is not useful with window systems such as X, because the
XEmacs job may not have a parent that can resume it again, and in any
case you can give input to some other job such as a shell merely by
moving to a different window.  Therefore, suspending is not allowed
when XEmacs is an X client.

 -- Command: suspend-emacs &optional stuffstring
     This function stops XEmacs and returns control to the superior
     process.  If and when the superior process resumes XEmacs,
     `suspend-emacs' returns `nil' to its caller in Lisp.

     If optional arg STUFFSTRING is non-`nil', its characters are sent
     to be read as terminal input by XEmacs's superior shell.  The
     characters in STUFFSTRING are not echoed by the superior shell;
     only the results appear.

     Before suspending, `suspend-emacs' runs the normal hook
     `suspend-hook'.  In Emacs version 18, `suspend-hook' was not a
     normal hook; its value was a single function, and if its value was
     non-`nil', then `suspend-emacs' returned immediately without
     actually suspending anything.

     After the user resumes XEmacs, `suspend-emacs' runs the normal hook
     `suspend-resume-hook'.  *Note Hooks::.

     The next redisplay after resumption will redraw the entire screen,
     unless the variable `no-redraw-on-reenter' is non-`nil' (*note
     Refresh Screen::).

     In the following example, note that `pwd' is not echoed after
     XEmacs is suspended.  But it is read and executed by the shell.

          (suspend-emacs)
               => nil

          (add-hook 'suspend-hook
                    (function (lambda ()
                                (or (y-or-n-p
                                      "Really suspend? ")
                                    (error "Suspend cancelled")))))
               => (lambda nil
                    (or (y-or-n-p "Really suspend? ")
                        (error "Suspend cancelled")))
          (add-hook 'suspend-resume-hook
                    (function (lambda () (message "Resumed!"))))
               => (lambda nil (message "Resumed!"))
          (suspend-emacs "pwd")
               => nil
          ---------- Buffer: Minibuffer ----------
          Really suspend? y
          ---------- Buffer: Minibuffer ----------

          ---------- Parent Shell ----------
          lewis@slug[23] % /user/lewis/manual
          lewis@slug[24] % fg

          ---------- Echo Area ----------
          Resumed!

 -- Variable: suspend-hook
     This variable is a normal hook run before suspending.

 -- Variable: suspend-resume-hook
     This variable is a normal hook run after suspending.


File: lispref.info,  Node: System Environment,  Next: User Identification,  Prev: Getting Out,  Up: System Interface

57.3 Operating System Environment
=================================

XEmacs provides access to variables in the operating system environment
through various functions.  These variables include the name of the
system, the user's UID, and so on.

 -- Variable: system-type
     The value of this variable is a symbol indicating the type of
     operating system XEmacs is operating on.  Here is a table of the
     possible values:

    `aix-v3'
          AIX.

    `berkeley-unix'
          Berkeley BSD.

    `dgux'
          Data General DGUX operating system.

    `gnu'
          A GNU system using the GNU HURD and Mach.

    `hpux'
          Hewlett-Packard HPUX operating system.

    `irix'
          Silicon Graphics Irix system.

    `linux'
          A GNU system using the Linux kernel.

    `ms-dos'
          Microsoft MS-DOS "operating system."

    `next-mach'
          NeXT Mach-based system.

    `rtu'
          Masscomp RTU, UCB universe.

    `unisoft-unix'
          UniSoft UniPlus.

    `usg-unix-v'
          AT&T System V.

    `windows-nt'
          Microsoft windows NT.

    `xenix'
          SCO Xenix 386.

     We do not wish to add new symbols to make finer distinctions
     unless it is absolutely necessary!  In fact, we hope to eliminate
     some of these alternatives in the future.  We recommend using
     `system-configuration' to distinguish between different operating
     systems.

 -- Variable: system-configuration
     This variable holds the three-part configuration name for the
     hardware/software configuration of your system, as a string.  The
     convenient way to test parts of this string is with `string-match'.

 -- Function: system-name
     This function returns the name of the machine you are running on.
          (system-name)
               => "prep.ai.mit.edu"

   The symbol `system-name' is a variable as well as a function.  In
fact, the function returns whatever value the variable `system-name'
currently holds.  Thus, you can set the variable `system-name' in case
Emacs is confused about the name of your system.  The variable is also
useful for constructing frame titles (*note Frame Titles::).

 -- Variable: mail-host-address
     If this variable is non-`nil', it is used instead of `system-name'
     for purposes of generating email addresses.  For example, it is
     used when constructing the default value of `user-mail-address'.
     *Note User Identification::.  (Since this is done when XEmacs
     starts up, the value actually used is the one saved when XEmacs
     was dumped.  *Note Building XEmacs::.)

 -- Command: getenv var &optional interactivep
     This function returns the value of the environment variable VAR,
     as a string.  Within XEmacs, the environment variable values are
     kept in the Lisp variable `process-environment'.

     When invoked interactively, `getenv' prints the value in the echo
     area.

          (getenv "USER")
               => "lewis"

          lewis@slug[10] % printenv
          PATH=.:/user/lewis/bin:/usr/bin:/usr/local/bin
          USER=lewis
          TERM=ibmapa16
          SHELL=/bin/csh
          HOME=/user/lewis

 -- Command: setenv variable &optional value unset
     This command sets the value of the environment variable named
     VARIABLE to VALUE.  Both arguments should be strings.  This
     function works by modifying `process-environment'; binding that
     variable with `let' is also reasonable practice.

 -- Variable: process-environment
     This variable is a list of strings, each describing one environment
     variable.  The functions `getenv' and `setenv' work by
     manipulating this variable.

          process-environment
          => ("l=/usr/stanford/lib/gnuemacs/lisp"
              "PATH=.:/user/lewis/bin:/usr/class:/nfsusr/local/bin"
              "USER=lewis"
              "TERM=ibmapa16"
              "SHELL=/bin/csh"
              "HOME=/user/lewis")

 -- Variable: path-separator
     This variable holds a string which says which character separates
     directories in a search path (as found in an environment
     variable).  Its value is `":"' for Unix and GNU systems, and `";"'
     for MS-DOS and Windows NT.

 -- Variable: invocation-name
     This variable holds the program name under which Emacs was
     invoked.  The value is a string, and does not include a directory
     name.

 -- Variable: invocation-directory
     This variable holds the directory from which the Emacs executable
     was invoked, or perhaps `nil' if that directory cannot be
     determined.

 -- Variable: installation-directory
     If non-`nil', this is a directory within which to look for the
     `lib-src' and `etc' subdirectories.  This is non-`nil' when Emacs
     can't find those directories in their standard installed
     locations, but can find them in a directory related somehow to the
     one containing the Emacs executable.

 -- Function: load-average &optional use-floats
     This function returns a list of the current 1-minute, 5-minute and
     15-minute load averages.  The values are integers that are 100
     times the system load averages.  (The load averages indicate the
     number of processes trying to run.)

     When USE-FLOATS is non-`nil', floats will be returned instead of
     integers.  These floats are not multiplied by 100.

          (load-average)
               => (169 158 164)
          (load-average t)
               => (1.69921875 1.58984375 1.640625)

          lewis@rocky[5] % uptime
            8:06pm  up 16 day(s), 21:57,  40 users,
           load average: 1.68, 1.59, 1.64

     If the 5-minute or 15-minute load averages are not available,
     return a shortened list, containing only those averages which are
     available.

     On some systems, this function may require special privileges to
     run, or it may be unimplemented for the particular system type.
     In that case, the function will signal an error.

 -- Function: emacs-pid
     This function returns the process ID of the Emacs process.


File: lispref.info,  Node: User Identification,  Next: Time of Day,  Prev: System Environment,  Up: System Interface

57.4 User Identification
========================

 -- Variable: user-mail-address
     This holds the nominal email address of the user who is using
     Emacs.  When Emacs starts up, it computes a default value that is
     usually right, but users often set this themselves when the
     default value is not right.

 -- Function: user-login-name &optional uid
     If you don't specify UID, this function returns the name under
     which the user is logged in.  If the environment variable `LOGNAME'
     is set, that value is used.  Otherwise, if the environment variable
     `USER' is set, that value is used.  Otherwise, the value is based
     on the effective UID, not the real UID.

     If you specify UID, the value is the user name that corresponds to
     UID (which should be an integer).

          (user-login-name)
               => "lewis"

 -- Function: user-real-login-name
     This function returns the user name corresponding to Emacs's real
     UID.  This ignores the effective UID and ignores the environment
     variables `LOGNAME' and `USER'.

 -- Variable: user-full-name
     This variable holds the name of the user running this Emacs.  It is
     initialized at startup time from the value of `NAME' environment
     variable.  You can change the value of this variable to alter the
     result of the `user-full-name' function.

 -- Function: user-full-name &optional user
     This function returns the full name of USER.  If USER is `nil', it
     defaults to the user running this Emacs.  In that case, the value
     of `user-full-name' variable, if non-`nil', will be used.

     If USER is specified explicitly, `user-full-name' variable is
     ignored.

          (user-full-name)
               => "Hrvoje Niksic"
          (setq user-full-name "Hrvoje \"Niksa\" Niksic")
          (user-full-name)
               => "Hrvoje \"Niksa\" Niksic"
          (user-full-name "hniksic")
               => "Hrvoje Niksic"

   The symbols `user-login-name', `user-real-login-name' and
`user-full-name' are variables as well as functions.  The functions
return the same values that the variables hold.  These variables allow
you to "fake out" Emacs by telling the functions what to return.  The
variables are also useful for constructing frame titles (*note Frame
Titles::).

 -- Function: user-real-uid
     This function returns the real UID of the user.

          (user-real-uid)
               => 19

 -- Function: user-uid
     This function returns the effective UID of the user.

 -- Function: user-home-directory
     This function returns the "`HOME'" directory of the user, and is
     intended to replace occurrences of "`(getenv "HOME")'".  Under
     Unix systems, the following is done:

       1. Return the value of "`(getenv "HOME")'", if set.

       2. Return "/", as a fallback, but issue a warning.  (Future
          versions of XEmacs will also attempt to lookup the `HOME'
          directory via `getpwent()', but this has not yet been
          implemented.)

     Under MS Windows, this is done:

       1. Return the value of "`(getenv "HOME")'", if set.

       2. If the environment variables `HOMEDRIVE' and `HOMEDIR' are
          both set, return the concatenation (the following description
          uses MS Windows environment variable substitution syntax):
          `%HOMEDRIVE%%HOMEDIR%'.

       3. Return "C:\", as a fallback, but issue a warning.


File: lispref.info,  Node: Time of Day,  Next: Time Conversion,  Prev: User Identification,  Up: System Interface

57.5 Time of Day
================

This section explains how to determine the current time and the time
zone.

 -- Function: current-time-string &optional time-value
     This function returns the current time and date as a
     humanly-readable string.  The format of the string is unvarying;
     the number of characters used for each part is always the same, so
     you can reliably use `substring' to extract pieces of it.  It is
     wise to count the characters from the beginning of the string
     rather than from the end, as additional information may be added
     at the end.

     The argument TIME-VALUE, if given, specifies a time to format
     instead of the current time.  The argument should be a list whose
     first two elements are integers.  Thus, you can use times obtained
     from `current-time' (see below) and from `file-attributes' (*note
     File Attributes::).

          (current-time-string)
               => "Wed Oct 14 22:21:05 1987"

 -- Function: current-time
     This function returns the system's time value as a list of three
     integers: `(HIGH LOW MICROSEC)'.  The integers HIGH and LOW
     combine to give the number of seconds since 0:00 January 1, 1970,
     which is HIGH * 2**16 + LOW.

     The third element, MICROSEC, gives the microseconds since the
     start of the current second (or 0 for systems that return time
     only on the resolution of a second).

     The first two elements can be compared with file time values such
     as you get with the function `file-attributes'.  *Note File
     Attributes::.

 -- Function: current-time-zone &optional time-value
     This function returns a list describing the time zone that the
     user is in.

     The value has the form `(OFFSET NAME)'.  Here OFFSET is an integer
     giving the number of seconds ahead of UTC (east of Greenwich).  A
     negative value means west of Greenwich.  The second element, NAME
     is a string giving the name of the time zone.  Both elements
     change when daylight savings time begins or ends; if the user has
     specified a time zone that does not use a seasonal time
     adjustment, then the value is constant through time.

     If the operating system doesn't supply all the information
     necessary to compute the value, both elements of the list are
     `nil'.

     The argument TIME-VALUE, if given, specifies a time to analyze
     instead of the current time.  The argument should be a cons cell
     containing two integers, or a list whose first two elements are
     integers.  Thus, you can use times obtained from `current-time'
     (see above) and from `file-attributes' (*note File Attributes::).


File: lispref.info,  Node: Time Conversion,  Next: Timers,  Prev: Time of Day,  Up: System Interface

57.6 Time Conversion
====================

These functions convert time values (lists of two or three integers) to
strings or to calendrical information.  There is also a function to
convert calendrical information to a time value.  You can get time
values from the functions `current-time' (*note Time of Day::) and
`file-attributes' (*note File Attributes::).

 -- Function: format-time-string format-string &optional time
     This function converts TIME to a string according to
     FORMAT-STRING.  If TIME is omitted, it defaults to the current
     time.  The argument FORMAT-STRING may contain `%'-sequences which
     say to substitute parts of the time.  Here is a table of what the
     `%'-sequences mean:

    `%a'
          This stands for the abbreviated name of the day of week.

    `%A'
          This stands for the full name of the day of week.

    `%b'
          This stands for the abbreviated name of the month.

    `%B'
          This stands for the full name of the month.

    `%c'
          This is a synonym for `%x %X'.

    `%C'
          This has a locale-specific meaning.  In the default locale
          (named C), it is equivalent to `%A, %B %e, %Y'.

    `%d'
          This stands for the day of month, zero-padded.

    `%D'
          This is a synonym for `%m/%d/%y'.

    `%e'
          This stands for the day of month, blank-padded.

    `%h'
          This is a synonym for `%b'.

    `%H'
          This stands for the hour (00-23).

    `%I'
          This stands for the hour (00-12).

    `%j'
          This stands for the day of the year (001-366).

    `%k'
          This stands for the hour (0-23), blank padded.

    `%l'
          This stands for the hour (1-12), blank padded.

    `%m'
          This stands for the month (01-12).

    `%M'
          This stands for the minute (00-59).

    `%n'
          This stands for a newline.

    `%p'
          This stands for `AM' or `PM', as appropriate.

    `%r'
          This is a synonym for `%I:%M:%S %p'.

    `%R'
          This is a synonym for `%H:%M'.

    `%S'
          This stands for the seconds (00-60).

    `%t'
          This stands for a tab character.

    `%T'
          This is a synonym for `%H:%M:%S'.

    `%U'
          This stands for the week of the year (01-52), assuming that
          weeks start on Sunday.

    `%w'
          This stands for the numeric day of week (0-6).  Sunday is day
          0.

    `%W'
          This stands for the week of the year (01-52), assuming that
          weeks start on Monday.

    `%x'
          This has a locale-specific meaning.  In the default locale
          (named C), it is equivalent to `%D'.

    `%X'
          This has a locale-specific meaning.  In the default locale
          (named C), it is equivalent to `%T'.

    `%y'
          This stands for the year without century (00-99).

    `%Y'
          This stands for the year with century.

    `%Z'
          This stands for the time zone abbreviation.

 -- Function: decode-time &optional specified-time
     This function converts a time value into calendrical information.
     The optional SPECIFIED-TIME should be a list of (HIGH LOW .
     IGNORED) or (HIGH . LOW), as from `current-time' and
     `file-attributes', or `nil' to use the current time.

     The return value is a list of nine elements, as follows:

          (SECONDS MINUTES HOUR DAY MONTH YEAR DOW DST ZONE)

     Here is what the elements mean:

    SEC
          The number of seconds past the minute, as an integer between
          0 and 59.

    MINUTE
          The number of minutes past the hour, as an integer between 0
          and 59.

    HOUR
          The hour of the day, as an integer between 0 and 23.

    DAY
          The day of the month, as an integer between 1 and 31.

    MONTH
          The month of the year, as an integer between 1 and 12.

    YEAR
          The year, an integer typically greater than 1900.

    DOW
          The day of week, as an integer between 0 and 6, where 0
          stands for Sunday.

    DST
          `t' if daylight savings time is effect, otherwise `nil'.

    ZONE
          An integer indicating the time zone, as the number of seconds
          east of Greenwich.

     Note that Common Lisp has different meanings for DOW and ZONE.

 -- Function: encode-time seconds minutes hour day month year &optional
          zone
     This function is the inverse of `decode-time'.  It converts seven
     items of calendrical data into a time value.  For the meanings of
     the arguments, see the table above under `decode-time'.

     Year numbers less than 100 are treated just like other year
     numbers.  If you want them to stand for years above 1900, you must
     alter them yourself before you call `encode-time'.

     The optional argument ZONE defaults to the current time zone and
     its daylight savings time rules.  If specified, it can be either a
     list (as you would get from `current-time-zone') or an integer (as
     you would get from `decode-time').  The specified zone is used
     without any further alteration for daylight savings time.


File: lispref.info,  Node: Timers,  Next: Terminal Input,  Prev: Time Conversion,  Up: System Interface

57.7 Timers for Delayed Execution
=================================

You can set up a timer to call a function at a specified future time.

 -- Function: add-timeout secs function object &optional resignal
     This function adds a timeout, to be signaled after the timeout
     period has elapsed.  SECS is a number of seconds, expressed as an
     integer or a float.  FUNCTION will be called after that many
     seconds have elapsed, with one argument, the given OBJECT.  If the
     optional RESIGNAL argument is provided, then after this timeout
     expires, `add-timeout' will automatically be called again with
     RESIGNAL as the first argument.

     This function returns an object which is the "id" of this
     particular timeout.  You can pass that object to `disable-timeout'
     to turn off the timeout before it has been signalled.

     The number of seconds may be expressed as a floating-point number,
     in which case some fractional part of a second will be used.
     Caveat: the usable timeout granularity will vary from system to
     system.

     Adding a timeout causes a timeout event to be returned by
     `next-event', and the function will be invoked by
     `dispatch-event', so if XEmacs is in a tight loop, the function
     will not be invoked until the next call to sit-for or until the
     return to top-level (the same is true of process filters).

     WARNING: if you are thinking of calling add-timeout from inside of
     a callback function as a way of resignalling a timeout, think
     again.  There is a race condition.  That's why the RESIGNAL
     argument exists.

     (NOTE: In FSF Emacs, this function is called `run-at-time' and has
     different semantics.)

 -- Function: disable-timeout id
     Cancel the requested action for ID, which should be a value
     previously returned by `add-timeout'.  This cancels the effect of
     that call to `add-timeout'; the arrival of the specified time will
     not cause anything special to happen.  (NOTE: In FSF Emacs, this
     function is called `cancel-timer'.)


File: lispref.info,  Node: Terminal Input,  Next: Terminal Output,  Prev: Timers,  Up: System Interface

57.8 Terminal Input
===================

This section describes functions and variables for recording or
manipulating terminal input.  See *Note Display::, for related
functions.

* Menu:

* Input Modes::		Options for how input is processed.
* Translating Input::   Low level conversion of some characters or events
			  into others.
* Recording Input::	Saving histories of recent or all input events.


File: lispref.info,  Node: Input Modes,  Next: Translating Input,  Up: Terminal Input

57.8.1 Input Modes
------------------

 -- Function: set-input-mode interrupt flow meta &optional quit-char
          console
     This function sets the mode for reading keyboard input.  If
     INTERRUPT is non-null, then XEmacs uses input interrupts.  If it is
     `nil', then it uses CBREAK mode.  When XEmacs communicates
     directly with X, it ignores this argument and uses interrupts if
     that is the way it knows how to communicate.

     If FLOW is non-`nil', then XEmacs uses XON/XOFF (`C-q', `C-s')
     flow control for output to the terminal.  This has no effect except
     in CBREAK mode.  *Note Flow Control::.

     The default setting is system dependent.  Some systems always use
     CBREAK mode regardless of what is specified.

     The argument META controls support for input character codes above
     127.  If META is `t', XEmacs converts characters with the 8th bit
     set into Meta characters.  If META is `nil', XEmacs disregards the
     8th bit; this is necessary when the terminal uses it as a parity
     bit.  If META is neither `t' nor `nil', XEmacs uses all 8 bits of
     input unchanged.  This is good for terminals using European 8-bit
     character sets.

     If QUIT-CHAR is non-`nil', it specifies the character to use for
     quitting.  Normally this character is `C-g'.  *Note Quitting::.

   The `current-input-mode' function returns the input mode settings
XEmacs is currently using.

 -- Function: current-input-mode &optional console
     This function returns current mode for reading keyboard input.  It
     returns a list, corresponding to the arguments of `set-input-mode',
     of the form `(INTERRUPT FLOW META QUIT)' in which:
    INTERRUPT
          is non-`nil' when XEmacs is using interrupt-driven input.  If
          `nil', Emacs is using CBREAK mode.

    FLOW
          is non-`nil' if XEmacs uses XON/XOFF (`C-q', `C-s') flow
          control for output to the terminal.  This value has no effect
          unless INTERRUPT is non-`nil'.

    META
          is `t' if XEmacs treats the eighth bit of input characters as
          the meta bit; `nil' means XEmacs clears the eighth bit of
          every input character; any other value means XEmacs uses all
          eight bits as the basic character code.

    QUIT
          is the character XEmacs currently uses for quitting, usually
          `C-g'.


File: lispref.info,  Node: Translating Input,  Next: Recording Input,  Prev: Input Modes,  Up: Terminal Input

57.8.2 Translating Input Events
-------------------------------

This section describes features for translating input events into other
input events before they become part of key sequences.

 -- Variable: function-key-map
     This variable holds a keymap that describes the character sequences
     sent by function keys on an ordinary character terminal.  This
     keymap uses the same data structure as other keymaps, but is used
     differently: it specifies translations to make while reading
     events.

     If `function-key-map' "binds" a key sequence K to a vector V, then
     when K appears as a subsequence _anywhere_ in a key sequence, it
     is replaced with the events in V.

     For example, VT100 terminals send `<ESC> O P' when the keypad PF1
     key is pressed.  Therefore, we want XEmacs to translate that
     sequence of events into the single event `pf1'.  We accomplish
     this by "binding" `<ESC> O P' to `[pf1]' in `function-key-map',
     when using a VT100.

     Thus, typing `C-c <PF1>' sends the character sequence `C-c <ESC> O
     P'; later the function `read-key-sequence' translates this back
     into `C-c <PF1>', which it returns as the vector `[?\C-c pf1]'.

     Entries in `function-key-map' are ignored if they conflict with
     bindings made in the minor mode, local, or global keymaps.  The
     intent is that the character sequences that function keys send
     should not have command bindings in their own right.

     The value of `function-key-map' is usually set up automatically
     according to the terminal's Terminfo or Termcap entry, but
     sometimes those need help from terminal-specific Lisp files.
     XEmacs comes with terminal-specific files for many common
     terminals; their main purpose is to make entries in
     `function-key-map' beyond those that can be deduced from Termcap
     and Terminfo.  *Note Terminal-Specific::.

     Emacs versions 18 and earlier used totally different means of
     detecting the character sequences that represent function keys.

 -- Variable: key-translation-map
     This variable is another keymap used just like `function-key-map'
     to translate input events into other events.  It differs from
     `function-key-map' in two ways:

        * `key-translation-map' goes to work after `function-key-map' is
          finished; it receives the results of translation by
          `function-key-map'.

        * `key-translation-map' overrides actual key bindings.

     The intent of `key-translation-map' is for users to map one
     character set to another, including ordinary characters normally
     bound to `self-insert-command'.

   You can use `function-key-map' or `key-translation-map' for more
than simple aliases, by using a function, instead of a key sequence, as
the "translation" of a key.  Then this function is called to compute
the translation of that key.

   The key translation function receives one argument, which is the
prompt that was specified in `read-key-sequence'--or `nil' if the key
sequence is being read by the editor command loop.  In most cases you
can ignore the prompt value.

   If the function reads input itself, it can have the effect of
altering the event that follows.  For example, here's how to define
`C-c h' to turn the character that follows into a Hyper character:

     (defun hyperify (prompt)
       (let ((e (read-event)))
         (vector (if (numberp e)
                     (logior (lsh 1 20) e)
                   (if (memq 'hyper (event-modifiers e))
                       e
                     (add-event-modifier "H-" e))))))

     (defun add-event-modifier (string e)
       (let ((symbol (if (symbolp e) e (car e))))
         (setq symbol (intern (concat string
                                      (symbol-name symbol))))
         (if (symbolp e)
             symbol
           (cons symbol (cdr e)))))

     (define-key function-key-map "\C-ch" 'hyperify)

   The `iso-transl' library uses this feature to provide a way of
inputting non-ASCII Latin-1 characters.


File: lispref.info,  Node: Recording Input,  Prev: Translating Input,  Up: Terminal Input

57.8.3 Recording Input
----------------------

 -- Function: recent-keys &optional number
     This function returns a vector containing recent input events from
     the keyboard or mouse.  By default, 100 events are recorded, which
     is how many `recent-keys' returns.

     All input events are included, whether or not they were used as
     parts of key sequences.  Thus, you always get the last 100 inputs,
     not counting keyboard macros.  (Events from keyboard macros are
     excluded because they are less interesting for debugging; it
     should be enough to see the events that invoked the macros.)

     If NUMBER is specified, not more than NUMBER events will be
     returned.  You may change the number of stored events using
     `set-recent-keys-ring-size'.

 -- Function: recent-keys-ring-size
     This function returns the number of recent events stored
     internally.  This is also the maximum number of events
     `recent-keys' can return.  By default, 100 events are stored.

 -- Function: set-recent-keys-ring-size size
     This function changes the number of events stored by XEmacs and
     returned by `recent-keys'.

     For example, `(set-recent-keys-ring-size 250)' will make XEmacs
     remember last 250 events and will make `recent-keys' return last
     250 events by default.

 -- Command: open-dribble-file filename
     This function opens a "dribble file" named FILENAME.  When a
     dribble file is open, each input event from the keyboard or mouse
     (but not those from keyboard macros) is written in that file.  A
     non-character event is expressed using its printed representation
     surrounded by `<...>'.

     You close the dribble file by calling this function with an
     argument of `nil'.

     This function is normally used to record the input necessary to
     trigger an XEmacs bug, for the sake of a bug report.

          (open-dribble-file "~/dribble")
               => nil

   See also the `open-termscript' function (*note Terminal Output::).


File: lispref.info,  Node: Terminal Output,  Next: Flow Control,  Prev: Terminal Input,  Up: System Interface

57.9 Terminal Output
====================

The terminal output functions send output to the terminal or keep track
of output sent to the terminal.  The function `device-baud-rate' tells
you what XEmacs thinks is the output speed of the terminal.

 -- Function: device-baud-rate &optional device
     This function's value is the output speed of the terminal
     associated with DEVICE, as far as XEmacs knows.  DEVICE defaults
     to the selected device (usually the only device) if omitted.
     Changing this value does not change the speed of actual data
     transmission, but the value is used for calculations such as
     padding.  This value has no effect for window-system devices.
     (This is different in FSF Emacs, where the baud rate also affects
     decisions about whether to scroll part of the screen or repaint,
     even when using a window system.)

     The value is measured in bits per second.

   XEmacs attempts to automatically initialize the baud rate by querying
the terminal.  If you are running across a network, however, and
different parts of the network work are at different baud rates, the
value returned by XEmacs may be different from the value used by your
local terminal.  Some network protocols communicate the local terminal
speed to the remote machine, so that XEmacs and other programs can get
the proper value, but others do not.  If XEmacs has the wrong value, it
makes decisions that are less than optimal.  To fix the problem, use
`set-device-baud-rate'.

 -- Function: set-device-baud-rate device baud-rate
     This function sets the output speed of DEVICE.  See
     `device-baud-rate'.  DEVICE defaults to the selected device
     (usually the only device) if `nil'.

 -- Function: send-string-to-terminal char-or-string &optional stdout-p
          device
     This function sends CHAR-OR-STRING to the terminal without
     alteration.  Control characters in CHAR-OR-STRING have
     terminal-dependent effects.

     If DEVICE is `nil', this function writes to XEmacs's stderr, or to
     stdout if STDOUT-P is non-`nil'.  Otherwise, DEVICE should be a
     tty or stream device, and the function writes to the device's
     normal or error output, according to STDOUT-P.

     One use of this function is to define function keys on terminals
     that have downloadable function key definitions.  For example,
     this is how on certain terminals to define function key 4 to move
     forward four characters (by transmitting the characters `C-u C-f'
     to the computer):

          (send-string-to-terminal "\eF4\^U\^F")
               => nil

 -- Command: open-termscript filename
     This function is used to open a "termscript file" that will record
     all the characters sent by XEmacs to the terminal. (If there are
     multiple tty or stream devices, all characters sent to all such
     devices are recorded.) The function returns `nil'.  Termscript
     files are useful for investigating problems where XEmacs garbles
     the screen, problems that are due to incorrect Termcap entries or
     to undesirable settings of terminal options more often than to
     actual XEmacs bugs.  Once you are certain which characters were
     actually output, you can determine reliably whether they
     correspond to the Termcap specifications in use.

     A `nil' value for FILENAME stops recording terminal output.

     See also `open-dribble-file' in *Note Terminal Input::.

          (open-termscript "../junk/termscript")
               => nil


File: lispref.info,  Node: Flow Control,  Next: Batch Mode,  Prev: Terminal Output,  Up: System Interface

57.10 Flow Control
==================

This section attempts to answer the question "Why does XEmacs choose to
use flow-control characters in its command character set?"  For a
second view on this issue, read the comments on flow control in the
`emacs/INSTALL' file from the distribution; for help with Termcap
entries and DEC terminal concentrators, see `emacs/etc/TERMS'.

   At one time, most terminals did not need flow control, and none used
`C-s' and `C-q' for flow control.  Therefore, the choice of `C-s' and
`C-q' as command characters was uncontroversial.  XEmacs, for economy
of keystrokes and portability, used nearly all the ASCII control
characters, with mnemonic meanings when possible; thus, `C-s' for
search and `C-q' for quote.

   Later, some terminals were introduced which required these characters
for flow control.  They were not very good terminals for full-screen
editing, so XEmacs maintainers did not pay attention.  In later years,
flow control with `C-s' and `C-q' became widespread among terminals,
but by this time it was usually an option.  And the majority of users,
who can turn flow control off, were unwilling to switch to less
mnemonic key bindings for the sake of flow control.

   So which usage is "right", XEmacs's or that of some terminal and
concentrator manufacturers?  This question has no simple answer.

   One reason why we are reluctant to cater to the problems caused by
`C-s' and `C-q' is that they are gratuitous.  There are other
techniques (albeit less common in practice) for flow control that
preserve transparency of the character stream.  Note also that their use
for flow control is not an official standard.  Interestingly, on the
model 33 teletype with a paper tape punch (which is very old), `C-s'
and `C-q' were sent by the computer to turn the punch on and off!

   As X servers and other window systems replace character-only
terminals, this problem is gradually being cured.  For the mean time,
XEmacs provides a convenient way of enabling flow control if you want
it: call the function `enable-flow-control'.

 -- Command: enable-flow-control &optional argument
     This function enables use of `C-s' and `C-q' for output flow
     control, and provides the characters `C-\' and `C-^' as aliases
     for them using `keyboard-translate-table' (*note Translating
     Input::).

     With optional argument ARGUMENT (interactively the prefix
     argument), enable flow control mode if ARGUMENT is positive; else
     disable it.

   You can use the function `enable-flow-control-on' in your `.emacs'
file to enable flow control automatically on certain terminal types.

 -- Function: enable-flow-control-on &rest termtypes
     This function enables flow control, and the aliases `C-\' and
     `C-^', if the terminal type is one of TERMTYPES.  For example:

          (enable-flow-control-on "vt200" "vt300" "vt101" "vt131")

   Here is how `enable-flow-control' does its job:

  1. It sets CBREAK mode for terminal input, and tells the operating
     system to handle flow control, with `(set-input-mode nil t)'.

  2. It sets up `keyboard-translate-table' to translate `C-\' and `C-^'
     into `C-s' and `C-q'.  Except at its very lowest level, XEmacs
     never knows that the characters typed were anything but `C-s' and
     `C-q', so you can in effect type them as `C-\' and `C-^' even when
     they are input for other commands.  *Note Translating Input::.

   If the terminal is the source of the flow control characters, then
once you enable kernel flow control handling, you probably can make do
with less padding than normal for that terminal.  You can reduce the
amount of padding by customizing the Termcap entry.  You can also
reduce it by setting `baud-rate' to a smaller value so that XEmacs uses
a smaller speed when calculating the padding needed.  *Note Terminal
Output::.


File: lispref.info,  Node: Batch Mode,  Prev: Flow Control,  Up: System Interface

57.11 Batch Mode
================

The command line option `-batch' causes XEmacs to run noninteractively.
In this mode, XEmacs does not read commands from the terminal, it does
not alter the terminal modes, and it does not expect to be outputting
to an erasable screen.  The idea is that you specify Lisp programs to
run; when they are finished, XEmacs should exit.  The way to specify
the programs to run is with `-l FILE', which loads the library named
FILE, and `-f FUNCTION', which calls FUNCTION with no arguments.

   Any Lisp program output that would normally go to the echo area,
either using `message' or using `prin1', etc., with `t' as the stream,
goes instead to XEmacs's standard error descriptor when in batch mode.
Thus, XEmacs behaves much like a noninteractive application program.
(The echo area output that XEmacs itself normally generates, such as
command echoing, is suppressed entirely.)

 -- Function: noninteractive
     This function returns non-`nil' when XEmacs is running in batch
     mode.

 -- Variable: noninteractive
     This variable is non-`nil' when XEmacs is running in batch mode.
     Setting this variable to `nil', however, will not change whether
     XEmacs is running in batch mode, and will not change the return
     value of the `noninteractive' function.


File: lispref.info,  Node: X-Windows,  Next: ToolTalk Support,  Prev: System Interface,  Up: Top

58 Functions Specific to the X Window System
********************************************

XEmacs provides the concept of "devices", which generalizes connections
to an X server, a TTY device, etc.  Most information about an X server
that XEmacs is connected to can be determined through general console
and device functions.  *Note Consoles and Devices::.  However, there
are some features of the X Window System that do not generalize well,
and they are covered specially here.

* Menu:

* X Selections::                Transferring text to and from other X clients.
* X Server::                    Information about the X server connected to
                                  a particular device.
* X Miscellaneous::             Other X-specific functions and variables.


File: lispref.info,  Node: X Selections,  Next: X Server,  Up: X-Windows

58.1 X Selections
=================

The X server records a set of "selections" which permit transfer of
data between application programs.  The various selections are
distinguished by "selection types", represented in XEmacs by symbols.
X clients including XEmacs can read or set the selection for any given
type.

 -- Function: x-own-selection data &optional type
     This function sets a "selection" in the X server.  It takes two
     arguments: a value, DATA, and the selection type TYPE to assign it
     to.  DATA may be a string, a cons of two markers, or an extent.
     In the latter cases, the selection is considered to be the text
     between the markers, or between the extent's endpoints.

     Each possible TYPE has its own selection value, which changes
     independently.  The usual values of TYPE are `PRIMARY' and
     `SECONDARY'; these are symbols with upper-case names, in accord
     with X Windows conventions.  The default is `PRIMARY'.

     (In FSF Emacs, this function is called `x-set-selection' and takes
     different arguments.)

 -- Function: x-get-selection
     This function accesses selections set up by XEmacs or by other X
     clients.  It returns the value of the current primary selection.

 -- Function: x-disown-selection &optional secondary-p
     Assuming we own the selection, this function disowns it.  If
     SECONDARY-P is non-`nil', the secondary selection instead of the
     primary selection is discarded.

   The X server also has a set of numbered "cut buffers" which can
store text or other data being moved between applications.  Cut buffers
are considered obsolete, but XEmacs supports them for the sake of X
clients that still use them.

 -- Function: x-get-cutbuffer &optional n
     This function returns the contents of cut buffer number N. (This
     function is called `x-get-cut-buffer' in FSF Emacs.)

 -- Function: x-store-cutbuffer string &optional push
     This function stores STRING into the first cut buffer (cut buffer
     0).

     Normally, the contents of the first cut buffer are simply replaced
     by STRING.  However, if optional argument PUSH is non-`nil', the
     cut buffers are rotated.  This means that the previous value of
     the first cut buffer moves to the second cut buffer, and the
     second to the third, and so on, moving the other values down
     through the series of cut buffers, kill-ring-style.  There are 8
     cut buffers altogether.

     Cut buffers are considered obsolete; you should use selections
     instead.

     This function has no effect if support for cut buffers was not
     compiled in.

     This function is called `x-set-cut-buffer' in FSF Emacs.


File: lispref.info,  Node: X Server,  Next: X Miscellaneous,  Prev: X Selections,  Up: X-Windows

58.2 X Server
=============

This section describes how to access and change the overall status of
the X server XEmacs is using.

* Menu:

* Resources::                   Getting resource values from the server.
* Server Data::                 Getting info about the X server.
* Grabs::                       Restricting access to the server by other apps.


File: lispref.info,  Node: Resources,  Next: Server Data,  Up: X Server

58.2.1 Resources
----------------

 -- Function: default-x-device
     This function return the default X device for resourcing.  This is
     the first-created X device that still exists.

 -- Function: x-get-resource name class type &optional locale device
          noerror
     This function retrieves a resource value from the X resource
     manager.

        * The first arg is the name of the resource to retrieve, such as
          `"font"'.

        * The second arg is the class of the resource to retrieve, like
          `"Font"'.

        * The third arg should be one of the symbols `string',
          `integer', `natnum', or `boolean', specifying the type of
          object that the database is searched for.

        * The fourth arg is the locale to search for the resources on,
          and can currently be a buffer, a frame, a device, or the
          symbol `global'.  If omitted, it defaults to `global'.

        * The fifth arg is the device to search for the resources on.
          (The resource database for a particular device is constructed
          by combining non-device- specific resources such any
          command-line resources specified and any app-defaults files
          found [or the fallback resources supplied by XEmacs, if no
          app-defaults file is found] with device-specific resources
          such as those supplied using `xrdb'.) If omitted, it defaults
          to the device of LOCALE, if a device can be derived (i.e. if
          LOCALE is a frame or device), and otherwise defaults to the
          value of `default-x-device'.

        * The sixth arg NOERROR, if non-`nil', means do not signal an
          error if a bogus resource specification was retrieved (e.g.
          if a non-integer was given when an integer was requested).
          In this case, a warning is issued instead.

     The resource names passed to this function are looked up relative
     to the locale.

     If you want to search for a subresource, you just need to specify
     the resource levels in NAME and CLASS.  For example, NAME could be
     `"modeline.attributeFont"', and CLASS `"Face.AttributeFont"'.

     Specifically,

       1. If LOCALE is a buffer, a call

                   `(x-get-resource "foreground" "Foreground" 'string SOME-BUFFER)'

          is an interface to a C call something like

                   `XrmGetResource (db, "xemacs.buffer.BUFFER-NAME.foreground",
                                       "Emacs.EmacsLocaleType.EmacsBuffer.Foreground",
                                       "String");'

       2. If LOCALE is a frame, a call

                   `(x-get-resource "foreground" "Foreground" 'string SOME-FRAME)'

          is an interface to a C call something like

                   `XrmGetResource (db, "xemacs.frame.FRAME-NAME.foreground",
                                       "Emacs.EmacsLocaleType.EmacsFrame.Foreground",
                                       "String");'

       3. If LOCALE is a device, a call

                   `(x-get-resource "foreground" "Foreground" 'string SOME-DEVICE)'

          is an interface to a C call something like

                   `XrmGetResource (db, "xemacs.device.DEVICE-NAME.foreground",
                                       "Emacs.EmacsLocaleType.EmacsDevice.Foreground",
                                       "String");'

       4. If LOCALE is the symbol `global', a call

                   `(x-get-resource "foreground" "Foreground" 'string 'global)'

          is an interface to a C call something like

                   `XrmGetResource (db, "xemacs.foreground",
                                       "Emacs.Foreground",
                                       "String");'

     Note that for `global', no prefix is added other than that of the
     application itself; thus, you can use this locale to retrieve
     arbitrary application resources, if you really want to.

     The returned value of this function is `nil' if the queried
     resource is not found.  If TYPE is `string', a string is returned,
     and if it is `integer', an integer is returned.  If TYPE is
     `boolean', then the returned value is the list `(t)' for true,
     `(nil)' for false, and is `nil' to mean "unspecified".

 -- Function: x-put-resource resource-line &optional device
     This function adds a resource to the resource database for DEVICE.
     RESOURCE-LINE specifies the resource to add and should be a
     standard resource specification.

 -- Variable: x-emacs-application-class
     This variable holds The X application class of the XEmacs process.
     This controls, among other things, the name of the "app-defaults"
     file that XEmacs will use.  For changes to this variable to take
     effect, they must be made before the connection to the X server is
     initialized, that is, this variable may only be changed before
     XEmacs is dumped, or by setting it in the file
     `lisp/term/x-win.el'.

     By default, this variable is `nil' at startup.  When the connection
     to the X server is first initialized, the X resource database will
     be consulted and the value will be set according to whether any
     resources are found for the application class "XEmacs".


File: lispref.info,  Node: Server Data,  Next: Grabs,  Prev: Resources,  Up: X Server

58.2.2 Data about the X Server
------------------------------

This section describes functions and a variable that you can use to get
information about the capabilities and origin of the X server
corresponding to a particular device.  The device argument is generally
optional and defaults to the selected device.

 -- Function: x-server-version &optional device
     This function returns the list of version numbers of the X server
     DEVICE is on.  The returned value is a list of three integers: the
     major and minor version numbers of the X protocol in use, and the
     vendor-specific release number.

 -- Function: x-server-vendor &optional device
     This function returns the vendor supporting the X server DEVICE is
     on.

 -- Function: x-display-visual-class &optional device
     This function returns the visual class of the display DEVICE is
     on.  The value is one of the symbols `static-gray', `gray-scale',
     `static-color', `pseudo-color', `true-color', and `direct-color'.
     (Note that this is different from previous versions of XEmacs,
     which returned `StaticGray', `GrayScale', etc.)


File: lispref.info,  Node: Grabs,  Prev: Server Data,  Up: X Server

58.2.3 Restricting Access to the Server by Other Apps
-----------------------------------------------------

 -- Function: x-grab-keyboard &optional device
     This function grabs the keyboard on the given device (defaulting
     to the selected one).  So long as the keyboard is grabbed, all
     keyboard events will be delivered to XEmacs--it is not possible
     for other X clients to eavesdrop on them.  Ungrab the keyboard
     with `x-ungrab-keyboard' (use an `unwind-protect').  Returns `t'
     if the grab was successful; `nil' otherwise.

 -- Function: x-ungrab-keyboard &optional device
     This function releases a keyboard grab made with `x-grab-keyboard'.

 -- Function: x-grab-pointer &optional device cursor ignore-keyboard
     This function grabs the pointer and restricts it to its current
     window.  If optional DEVICE argument is `nil', the selected device
     will be used.  If optional CURSOR argument is non-`nil', change
     the pointer shape to that until `x-ungrab-pointer' is called (it
     should be an object returned by the `make-cursor' function).  If
     the second optional argument IGNORE-KEYBOARD is non-`nil', ignore
     all keyboard events during the grab.  Returns `t' if the grab is
     successful, `nil' otherwise.

 -- Function: x-ungrab-pointer &optional device
     This function releases a pointer grab made with `x-grab-pointer'.
     If optional first arg DEVICE is `nil' the selected device is used.
     If it is `t' the pointer will be released on all X devices.


File: lispref.info,  Node: X Miscellaneous,  Prev: X Server,  Up: X-Windows

58.3 Miscellaneous X Functions and Variables
============================================

 -- Variable: x-bitmap-file-path
     This variable holds a list of the directories in which X bitmap
     files may be found.  If `nil', this is initialized from the
     `"*bitmapFilePath"' resource.  This is used by the
     `make-image-instance' function (however, note that if the
     environment variable `XBMLANGPATH' is set, it is consulted first).

 -- Variable: x-library-search-path
     This variable holds the search path used by `read-color' to find
     `rgb.txt'.

 -- Function: x-valid-keysym-name-p keysym
     This function returns true if KEYSYM names a keysym that the X
     library knows about.  Valid keysyms are listed in the files
     `/usr/include/X11/keysymdef.h' and in `/usr/lib/X11/XKeysymDB', or
     whatever the equivalents are on your system.

 -- Function: x-window-id &optional frame
     This function returns the ID of the X11 window.  This gives us a
     chance to manipulate the Emacs window from within a different
     program.  Since the ID is an unsigned long, we return it as a
     string.

 -- Variable: x-allow-sendevents
     If non-`nil', synthetic events are allowed.  `nil' means they are
     ignored.  Beware: allowing XEmacs to process SendEvents opens a
     big security hole.

 -- Function: x-debug-mode arg &optional device
     With a true arg, make the connection to the X server synchronous.
     With false, make it asynchronous.  Synchronous connections are
     much slower, but are useful for debugging. (If you get X errors,
     make the connection synchronous, and use a debugger to set a
     breakpoint on `x_error_handler'.  Your backtrace of the C stack
     will now be useful.  In asynchronous mode, the stack above
     `x_error_handler' isn't helpful because of buffering.)  If DEVICE
     is not specified, the selected device is assumed.

     Calling this function is the same as calling the C function
     `XSynchronize', or starting the program with the `-sync' command
     line argument.

 -- Variable: x-debug-events
     If non-zero, debug information about events that XEmacs sees is
     displayed.  Information is displayed on stderr.  Currently defined
     values are:

        * 1 == non-verbose output

        * 2 == verbose output


File: lispref.info,  Node: ToolTalk Support,  Next: LDAP Support,  Prev: X-Windows,  Up: Top

59 ToolTalk Support
*******************

* Menu:

* XEmacs ToolTalk API Summary::
* Sending Messages::
* Receiving Messages::


File: lispref.info,  Node: XEmacs ToolTalk API Summary,  Next: Sending Messages,  Up: ToolTalk Support

59.1 XEmacs ToolTalk API Summary
================================

The XEmacs Lisp interface to ToolTalk is similar, at least in spirit,
to the standard C ToolTalk API.  Only the message and pattern parts of
the API are supported at present; more of the API could be added if
needed.  The Lisp interface departs from the C API in a few ways:

   * ToolTalk is initialized automatically at XEmacs startup-time.
     Messages can only be sent other ToolTalk applications connected to
     the same X11 server that XEmacs is running on.

   * There are fewer entry points; polymorphic functions with keyword
     arguments are used instead.

   * The callback interface is simpler and marginally less functional.
     A single callback may be associated with a message or a pattern;
     the callback is specified with a Lisp symbol (the symbol should
     have a function binding).

   * The session attribute for messages and patterns is always
     initialized to the default session.

   * Anywhere a ToolTalk enum constant, e.g. `TT_SESSION', is valid, one
     can substitute the corresponding symbol, e.g. `'TT_SESSION'.  This
     simplifies building lists that represent messages and patterns.


File: lispref.info,  Node: Sending Messages,  Next: Receiving Messages,  Prev: XEmacs ToolTalk API Summary,  Up: ToolTalk Support

59.2 Sending Messages
=====================

* Menu:

* Example of Sending Messages::
* Elisp Interface for Sending Messages::


File: lispref.info,  Node: Example of Sending Messages,  Next: Elisp Interface for Sending Messages,  Up: Sending Messages

59.2.1 Example of Sending Messages
----------------------------------

Here's a simple example that sends a query to another application and
then displays its reply.  Both the query and the reply are stored in
the first argument of the message.

     (defun tooltalk-random-query-handler (msg)
       (let ((state (get-tooltalk-message-attribute msg 'state)))
         (cond
           ((eq state 'TT_HANDLED)
            (message (get-tooltalk-message-attribute msg arg_val 0)))
           ((memq state '(TT_FAILED TT_REJECTED))
            (message "Random query turns up nothing")))))

     (defvar random-query-message
       '(   class TT_REQUEST
            scope TT_SESSION
          address TT_PROCEDURE
               op "random-query"
             args '((TT_INOUT "?" "string"))
         callback tooltalk-random-query-handler))

     (let ((m (make-tooltalk-message random-query-message)))
       (send-tooltalk-message m))


File: lispref.info,  Node: Elisp Interface for Sending Messages,  Prev: Example of Sending Messages,  Up: Sending Messages

59.2.2 Elisp Interface for Sending Messages
-------------------------------------------

 -- Function: make-tooltalk-message attributes
     Create a ToolTalk message and initialize its attributes.  The
     value of ATTRIBUTES must be a list of alternating keyword/values,
     where keywords are symbols that name valid message attributes.
     For example:

            (make-tooltalk-message
              '(class TT_NOTICE
                scope TT_SESSION
                address TT_PROCEDURE
                op "do-something"
                args ("arg1" 12345 (TT_INOUT "arg3" "string"))))

     Values must always be strings, integers, or symbols that represent
     ToolTalk constants.  Attribute names are the same as those
     supported by `set-tooltalk-message-attribute', plus `args'.

     The value of `args' should be a list of message arguments where
     each message argument has the following form:

             `(mode [value [type]])' or just `value'

     Where MODE is one of `TT_IN', `TT_OUT', or `TT_INOUT' and TYPE is
     a string.  If TYPE isn't specified then `int' is used if VALUE is
     a number; otherwise `string' is used.  If TYPE is `string' then
     VALUE is converted to a string (if it isn't a string already) with
     `prin1-to-string'.  If only a value is specified then MODE
     defaults to `TT_IN'.  If MODE is `TT_OUT' then VALUE and TYPE
     don't need to be specified.  You can find out more about the
     semantics and uses of ToolTalk message arguments in chapter 4 of
     the `ToolTalk Programmer's Guide'.


 -- Function: send-tooltalk-message msg
     Send the message on its way.  Once the message has been sent it's
     almost always a good idea to get rid of it with
     `destroy-tooltalk-message'.


 -- Function: return-tooltalk-message msg &optional mode
     Send a reply to this message.  The second argument can be `reply',
     `reject' or `fail'; the default is `reply'.  Before sending a
     reply, all message arguments whose mode is `TT_INOUT' or `TT_OUT'
     should have been filled in--see `set-tooltalk-message-attribute'.


 -- Function: get-tooltalk-message-attribute msg attribute &optional
          argn
     Returns the indicated ToolTalk message attribute.  Attributes are
     identified by symbols with the same name (underscores and all) as
     the suffix of the ToolTalk `tt_message_<attribute>' function that
     extracts the value.  String attribute values are copied and
     enumerated type values (except disposition) are converted to
     symbols; e.g. `TT_HANDLER' is `'TT_HANDLER', `uid' and `gid' are
     represented by fixnums (small integers), `opnum' is converted to a
     string, and `disposition' is converted to a fixnum.  We convert
     `opnum' (a C int) to a string (e.g. `123' => `"123"') because
     there's no guarantee that opnums will fit within the range of
     XEmacs Lisp integers.

     [TBD] Use the `plist' attribute instead of C API `user' attribute
     for user-defined message data.  To retrieve the value of a message
     property, specify the indicator for ARGN.  For example, to get the
     value of a property called `rflag', use

             (get-tooltalk-message-attribute msg 'plist 'rflag)

     To get the value of a message argument use one of the `arg_val'
     (strings), `arg_ival' (integers), or `arg_bval' (strings with
     embedded nulls), attributes.  For example, to get the integer
     value of the third argument:

             (get-tooltalk-message-attribute msg 'arg_ival 2)

     As you can see, argument numbers are zero-based.  The type of each
     arguments can be retrieved with the `arg_type' attribute; however
     ToolTalk doesn't define any semantics for the string value of
     `arg_type'.  Conventionally `string' is used for strings and `int'
     for 32 bit integers.  Note that XEmacs Lisp stores the lengths of
     strings explicitly (unlike C) so treating the value returned by
     `arg_bval' like a string is fine.


 -- Function: set-tooltalk-message-attribute value msg attribute
          &optional argn
     Initialize one ToolTalk message attribute.

     Attribute names and values are the same as for
     `get-tooltalk-message-attribute'.  A property list is provided for
     user data (instead of the `user' message attribute); see
     `get-tooltalk-message-attribute'.

     Callbacks are handled slightly differently than in the C ToolTalk
     API.  The value of CALLBACK should be the name of a function of one
     argument.  It will be called each time the state of the message
     changes.  This is usually used to notice when the message's state
     has changed to `TT_HANDLED' (or `TT_FAILED'), so that reply
     argument values can be used.

     If one of the argument attributes is specified as `arg_val',
     `arg_ival', or `arg_bval', then ARGN must be the number of an
     already created argument.  Arguments can be added to a message
     with `add-tooltalk-message-arg'.


 -- Function: add-tooltalk-message-arg msg mode type &optional value
     Append one new argument to the message.  MODE must be one of
     `TT_IN', `TT_INOUT', or `TT_OUT', TYPE must be a string, and VALUE
     can be a string or an integer.  ToolTalk doesn't define any
     semantics for TYPE, so only the participants in the protocol
     you're using need to agree what types mean (if anything).
     Conventionally `string' is used for strings and `int' for 32 bit
     integers.  Arguments can initialized by providing a value or with
     `set-tooltalk-message-attribute'; the latter is necessary if you
     want to initialize the argument with a string that can contain
     embedded nulls (use `arg_bval').


 -- Function: create-tooltalk-message &optional no-callback
     Create a new ToolTalk message.  The message's session attribute is
     initialized to the default session.  Other attributes can be
     initialized with `set-tooltalk-message-attribute'.
     `make-tooltalk-message' is the preferred way to create and
     initialize a message.

     Optional arg NO-CALLBACK says don't add a C-level callback at all.
     Normally don't do that; just don't specify the Lisp callback when
     calling `make-tooltalk-message'.


 -- Function: destroy-tooltalk-message msg
     Apply `tt_message_destroy' to the message.  It's not necessary to
     destroy messages after they've been processed by a message or
     pattern callback, the Lisp/ToolTalk callback machinery does this
     for you.


File: lispref.info,  Node: Receiving Messages,  Prev: Sending Messages,  Up: ToolTalk Support

59.3 Receiving Messages
=======================

* Menu:

* Example of Receiving Messages::
* Elisp Interface for Receiving Messages::


File: lispref.info,  Node: Example of Receiving Messages,  Next: Elisp Interface for Receiving Messages,  Up: Receiving Messages

59.3.1 Example of Receiving Messages
------------------------------------

Here's a simple example of a handler for a message that tells XEmacs to
display a string in the mini-buffer area.  The message operation is
called `emacs-display-string'.  Its first (0th) argument is the string
to display.

     (defun tooltalk-display-string-handler (msg)
       (message (get-tooltalk-message-attribute msg 'arg_val 0)))

     (defvar display-string-pattern
       '(category TT_HANDLE
            scope TT_SESSION
               op "emacs-display-string"
         callback tooltalk-display-string-handler))

     (let ((p (make-tooltalk-pattern display-string-pattern)))
       (register-tooltalk-pattern p))


File: lispref.info,  Node: Elisp Interface for Receiving Messages,  Prev: Example of Receiving Messages,  Up: Receiving Messages

59.3.2 Elisp Interface for Receiving Messages
---------------------------------------------

 -- Function: make-tooltalk-pattern attributes
     Create a ToolTalk pattern and initialize its attributes.  The
     value of attributes must be a list of alternating keyword/values,
     where keywords are symbols that name valid pattern attributes or
     lists of valid attributes.  For example:

            (make-tooltalk-pattern
              '(category TT_OBSERVE
                   scope TT_SESSION
                      op ("operation1" "operation2")
                    args ("arg1" 12345 (TT_INOUT "arg3" "string"))))

     Attribute names are the same as those supported by
     `add-tooltalk-pattern-attribute', plus `'args'.

     Values must always be strings, integers, or symbols that represent
     ToolTalk constants or lists of same.  When a list of values is
     provided all of the list elements are added to the attribute.  In
     the example above, messages whose `op' attribute is `"operation1"'
     or `"operation2"' would match the pattern.

     The value of ARGS should be a list of pattern arguments where each
     pattern argument has the following form:

             `(mode [value [type]])' or just `value'

     Where MODE is one of `TT_IN', `TT_OUT', or `TT_INOUT' and TYPE is
     a string.  If TYPE isn't specified then `int' is used if VALUE is
     a number; otherwise `string' is used.  If TYPE is `string' then
     VALUE is converted to a string (if it isn't a string already) with
     `prin1-to-string'.  If only a value is specified then MODE
     defaults to `TT_IN'.  If MODE is `TT_OUT' then VALUE and TYPE
     don't need to be specified.  You can find out more about the
     semantics and uses of ToolTalk pattern arguments in chapter 3 of
     the `ToolTalk Programmer's Guide'.


 -- Function: register-tooltalk-pattern pattern
     XEmacs will begin receiving messages that match this pattern.

 -- Function: unregister-tooltalk-pattern pattern
     XEmacs will stop receiving messages that match this pattern.

 -- Function: add-tooltalk-pattern-attribute value pattern indicator
     Add one value to the indicated pattern attribute. The names of
     attributes are the same as the ToolTalk accessors used to set them
     less the `tooltalk_pattern_' prefix and the `_add' suffix.  For
     example, the name of the attribute for the
     `tt_pattern_disposition_add' attribute is `disposition'.  The
     `category' attribute is handled specially, since a pattern can only
     be a member of one category (`TT_OBSERVE' or `TT_HANDLE').

     Callbacks are handled slightly differently than in the C ToolTalk
     API.  The value of CALLBACK should be the name of a function of one
     argument.  It will be called each time the pattern matches an
     incoming message.

 -- Function: add-tooltalk-pattern-arg pattern mode vtype &optional
          value
     Add one fully-specified argument to a ToolTalk pattern.  MODE must
     be one of `TT_IN', `TT_INOUT', or `TT_OUT'.  VTYPE must be a
     string.  VALUE can be an integer, string or `nil'.  If VALUE is an
     integer then an integer argument (`tt_pattern_iarg_add') is added;
     otherwise a string argument is added.  At present there's no way
     to add a binary data argument.


 -- Function: create-tooltalk-pattern
     Create a new ToolTalk pattern and initialize its session attribute
     to be the default session.

 -- Function: destroy-tooltalk-pattern pattern
     Apply `tt_pattern_destroy' to the pattern.  This effectively
     unregisters the pattern.

 -- Function: describe-tooltalk-message msg &optional stream
     Print the message's attributes and arguments to STREAM.  This is
     often useful for debugging.


File: lispref.info,  Node: LDAP Support,  Next: PostgreSQL Support,  Prev: ToolTalk Support,  Up: Top

60 LDAP Support
***************

XEmacs can be linked with a LDAP client library to provide Elisp
primitives to access directory servers using the Lightweight Directory
Access Protocol.

* Menu:

* Building XEmacs with LDAP support::  How to add LDAP support to XEmacs
* XEmacs LDAP API::             Lisp access to LDAP functions
* Syntax of Search Filters::    A brief summary of RFC 1558


File: lispref.info,  Node: Building XEmacs with LDAP support,  Next: XEmacs LDAP API,  Prev: LDAP Support,  Up: LDAP Support

60.1 Building XEmacs with LDAP support
======================================

LDAP support must be added to XEmacs at build time since it requires
linking to an external LDAP client library.  As of 21.2, XEmacs has been
successfully built and tested with

   * OpenLDAP 1.2 (`http://www.openldap.org/')

   * University of Michigan's LDAP 3.3
     (`http://www.umich.edu/~dirsvcs/ldap/')

   * LDAP SDK 1.0 from Netscape Corp. (`http://developer.netscape.com/')

   Other libraries conforming to RFC 1823 will probably work also but
may require some minor tweaking at C level.

   The standard XEmacs configure script auto-detects an installed LDAP
library provided the library itself and the corresponding header files
can be found in the library and include paths.  A successful detection
will be signalled in the final output of the configure script.


File: lispref.info,  Node: XEmacs LDAP API,  Next: Syntax of Search Filters,  Prev: Building XEmacs with LDAP support,  Up: LDAP Support

60.2 XEmacs LDAP API
====================

XEmacs LDAP API consists of two layers:  a low-level layer which tries
to stay as close as possible to the C API (where practical) and a
higher-level layer which provides more convenient primitives to
effectively use LDAP.

   The low-level API should be used directly for very specific purposes
(such as multiple operations on a connection) only.  The higher-level
functions provide a more convenient way to access LDAP directories
hiding the subtleties of handling the connection, translating arguments
and ensuring compliance with LDAP internationalization rules and formats
(currently partly implemented only).

* Menu:

* LDAP Variables::              Lisp variables related to LDAP
* The High-Level LDAP API::     High-level LDAP lisp functions
* The Low-Level LDAP API::      Low-level LDAP lisp primitives
* LDAP Internationalization::   I18n variables and functions


File: lispref.info,  Node: LDAP Variables,  Next: The High-Level LDAP API,  Prev: XEmacs LDAP API,  Up: XEmacs LDAP API

60.2.1 LDAP Variables
---------------------

 -- Variable: ldap-default-host
     The default LDAP server hostname.  A TCP port number can be
     appended to that name using a colon as a separator.

 -- Variable: ldap-default-port
     Default TCP port for LDAP connections.  Initialized from the LDAP
     library. Default value is 389.

 -- Variable: ldap-default-base
     Default base for LDAP searches.  This is a string using the syntax
     of RFC 1779.  For instance, "o=ACME, c=US" limits the search to the
     Acme organization in the United States.

 -- Variable: ldap-host-parameters-alist
     An alist of per host options for LDAP transactions.  The list
     elements look like `(HOST PROP1 VAL1 PROP2 VAL2 ...)' HOST is the
     name of an LDAP server. A TCP port number can be appended to that
     name using a colon as a separator.  PROPN and VALN are
     property/value pairs describing parameters for the server.  Valid
     properties:
    `binddn'
          The distinguished name of the user to bind as.  This may look
          like `cn=Babs Jensen,o=ACME,c=US', see RFC 1779 for details.

    `passwd'
          The password to use for authentication.

    `auth'
          The authentication method to use, possible values depend on
          the LDAP library XEmacs was compiled with, they may include
          `simple', `krbv41' and `krbv42'.

    `base'
          The base for the search. This may look like `cÿ, o¬me', see
          RFC 1779 for syntax details.

    `scope'
          One of the symbols `base', `onelevel' or `subtree' indicating
          the scope of the search limited to a base object, to a single
          level or to the whole subtree.

    `deref'
          The dereference policy is one of the symbols `never',
          `always', `search' or `find' and defines how aliases are
          dereferenced.
         `never'
               Aliases are never dereferenced

         `always'
               Aliases are always dereferenced

         `search'
               Aliases are dereferenced when searching

         `find'
               Aliases are dereferenced when locating the base object
               for the search

    `timelimit'
          The timeout limit for the connection in seconds.

    `sizelimit'
          The maximum number of matches to return for searches
          performed on this connection.

 -- Variable: ldap-verbose
     If non-`nil', LDAP operations will echo progress messages.
     Defaults to `nil'.


File: lispref.info,  Node: The High-Level LDAP API,  Next: The Low-Level LDAP API,  Prev: LDAP Variables,  Up: XEmacs LDAP API

60.2.2 The High-Level LDAP API
------------------------------

The following functions provide the most convenient interface to perform
LDAP operations.  All of them open a connection to a host, perform an
operation (add/search/modify/delete) on one or several entries and
cleanly close the connection thus insulating the user from all the
details of the low-level interface such as LDAP Lisp objects *note The
Low-Level LDAP API::.

   Note that `ldap-search' which used to be the name of the high-level
search function in XEmacs 21.1 is now obsolete.  For consistency  in the
naming as well as backward compatibility, that function now acts as a
wrapper that calls either `ldap-search-basic' (low-level search
function) or `ldap-search-entries' (high-level search function)
according to the actual parameters.  A direct call to one of these two
functions is preferred since it is faster and unambiguous.

 -- Command: ldap-search-entries filter &optional host attributes
          attrsonly withdn
     Perform an LDAP search.  FILTER is the search filter *note Syntax
     of Search Filters:: HOST is the LDAP host on which to perform the
     search.  ATTRIBUTES is the specific attributes to retrieve, `nil'
     means retrieve all.  ATTRSONLY if non-`nil' retrieves the
     attributes only without their associated values.  If WITHDN is
     non-`nil' each entry in the result will be prepended with its
     distinguished name DN.  Additional search parameters can be
     specified through `ldap-host-parameters-alist'.  The function
     returns a list of matching entries.  Each entry is itself an alist
     of attribute/value pairs optionally preceded by the DN of the
     entry according to the value of WITHDN.

 -- Function: ldap-add-entries entries &optional host binddn passwd
     Add entries to an LDAP directory.  ENTRIES is a list of entry
     specifications of the form `(DN (ATTR . VALUE) (ATTR . VALUE) ...)'
     where DN the distinguished name of an entry to add, the following
     are cons cells containing attribute/value string pairs.  HOST is
     the LDAP host, defaulting to `ldap-default-host'.  BINDDN is the
     DN to bind as to the server.  PASSWD is the corresponding password.

 -- Function: ldap-modify-entries entry-mods &optional host binddn
          passwd
     Modify entries of an LDAP directory.  ENTRY_MODS is a list of
     entry modifications of the form `(DN MOD-SPEC1 MOD-SPEC2 ...)'
     where DN is the distinguished name of the entry to modify, the
     following are modification specifications.  A modification
     specification is itself a list of the form `(MOD-OP ATTR VALUE1
     VALUE2 ...)' MOD-OP and ATTR are mandatory, VALUES are optional
     depending on MOD-OP.  MOD-OP is the type of modification, one of
     the symbols `add', `delete' or `replace'. ATTR is the LDAP
     attribute type to modify.  HOST is the LDAP host, defaulting to
     `ldap-default-host'.  BINDDN is the DN to bind as to the server.
     PASSWD is the corresponding password.

 -- Function: ldap-delete-entries dn &optional host binddn passwd
     Delete an entry from an LDAP directory.  DN is the distinguished
     name of an entry to delete or a list of those.  HOST is the LDAP
     host, defaulting to `ldap-default-host'.  BINDDN is the DN to bind
     as to the server.  PASSWD is the corresponding password.


File: lispref.info,  Node: The Low-Level LDAP API,  Next: LDAP Internationalization,  Prev: The High-Level LDAP API,  Up: XEmacs LDAP API

60.2.3 The Low-Level LDAP API
-----------------------------

The low-level API should be used directly for very specific purposes
(such as multiple operations on a connection) only.  The higher-level
functions provide a more convenient way to access LDAP directories
hiding the subtleties of handling the connection, translating arguments
and ensuring compliance with LDAP internationalization rules and formats
(currently partly implemented only). See *note The High-Level LDAP API::

   Note that the former functions `ldap-*-internal' functions have been
renamed in XEmacs 21.2

* Menu:

* The LDAP Lisp Object::
* Opening and Closing a LDAP Connection::
* Low-level Operations on a LDAP Server::


File: lispref.info,  Node: The LDAP Lisp Object,  Next: Opening and Closing a LDAP Connection,  Prev: The Low-Level LDAP API,  Up: The Low-Level LDAP API

60.2.3.1 The LDAP Lisp Object
.............................

An internal built-in `ldap' lisp object represents a LDAP connection.

 -- Function: ldapp object
     This function returns non-`nil' if OBJECT is a `ldap' object.

 -- Function: ldap-host ldap
     Return the server host of the connection represented by LDAP.

 -- Function: ldap-live-p ldap
     Return non-`nil' if LDAP is an active LDAP connection.


File: lispref.info,  Node: Opening and Closing a LDAP Connection,  Next: Low-level Operations on a LDAP Server,  Prev: The LDAP Lisp Object,  Up: The Low-Level LDAP API

60.2.3.2 Opening and Closing a LDAP Connection
..............................................

 -- Function: ldap-open host &optional plist
     Open a LDAP connection to HOST.  PLIST is a property list
     containing additional parameters for the connection.  Valid keys
     in that list are:
    `port'
          The TCP port to use for the connection if different from
          `ldap-default-port' or the library builtin value

    `auth'
          The authentication method to use, possible values depend on
          the LDAP library XEmacs was compiled with, they may include
          `simple', `krbv41' and `krbv42'.

    `binddn'
          The distinguished name of the user to bind as.  This may look
          like `c=com, o=Acme, cn=Babs Jensen', see RFC 1779 for
          details.

    `passwd'
          The password to use for authentication.

    `deref'
          The dereference policy is one of the symbols `never',
          `always', `search' or `find' and defines how aliases are
          dereferenced.
         `never'
               Aliases are never dereferenced.

         `always'
               Aliases are always dereferenced.

         `search'
               Aliases are dereferenced when searching.

         `find'
               Aliases are dereferenced when locating the base object
               for the search.
          The default is `never'.

    `timelimit'
          The timeout limit for the connection in seconds.

    `sizelimit'
          The maximum number of matches to return for searches
          performed on this connection.

 -- Function: ldap-close ldap
     Close the connection represented by LDAP.


File: lispref.info,  Node: Low-level Operations on a LDAP Server,  Prev: Opening and Closing a LDAP Connection,  Up: The Low-Level LDAP API

60.2.3.3 Low-level Operations on a LDAP Server
..............................................

`ldap-search-basic' is the low-level primitive to perform a search on a
LDAP server.  It works directly on an open LDAP connection thus
requiring a preliminary call to `ldap-open'.  Multiple searches can be
made on the same connection, then the session must be closed with
`ldap-close'.

 -- Function: ldap-search-basic ldap filter &optional base scope attrs
          attrsonly withdn verbose
     Perform a search on an open connection LDAP created with
     `ldap-open'.  FILTER is a filter string for the search *note
     Syntax of Search Filters:: BASE is the distinguished name at which
     to start the search.  SCOPE is one of the symbols `base',
     `onelevel' or `subtree' indicating the scope of the search limited
     to a base object, to a single level or to the whole subtree.  The
     default is `subtree'.  ATTRS is a list of strings indicating which
     attributes to retrieve for each matching entry. If `nil' all
     available attributes are returned.  If ATTRSONLY is non-`nil' then
     only the attributes are retrieved, not their associated values.
     If WITHDN is non-`nil' then each entry in the result is prepended
     with its distinguished name DN.  If VERBOSE is non-`nil' then
     progress messages are echoed The function returns a list of
     matching entries.  Each entry  is itself an alist of
     attribute/value pairs optionally preceded by the DN of the entry
     according to the value of WITHDN.

 -- Function: ldap-add ldap dn entry
     Add ENTRY to a LDAP directory which a connection LDAP has been
     opened to with `ldap-open'.  DN is the distinguished name of the
     entry to add.  ENTRY is an entry specification, i.e., a list of
     cons cells containing attribute/value string pairs.

 -- Function: ldap-modify ldap dn mods
     Modify an entry in an LDAP directory.  LDAP is an LDAP connection
     object created with `ldap-open'.  DN is the distinguished name of
     the entry to modify.  MODS is a list of modifications to apply.  A
     modification is a list of the form `(MOD-OP ATTR VALUE1 VALUE2
     ...)'  MOD-OP and ATTR are mandatory, VALUES are optional
     depending on MOD-OP.  MOD-OP is the type of modification, one of
     the symbols `add', `delete' or `replace'. ATTR is the LDAP
     attribute type to modify.

 -- Function: ldap-delete ldap dn
     Delete an entry to an LDAP directory.  LDAP is an LDAP connection
     object created with `ldap-open'.  DN is the distinguished name of
     the entry to delete.


File: lispref.info,  Node: LDAP Internationalization,  Prev: The Low-Level LDAP API,  Up: XEmacs LDAP API

60.2.4 LDAP Internationalization
--------------------------------

The XEmacs LDAP API provides basic internationalization features based
on the LDAP v3 specification (essentially RFC2252 on "LDAP v3 Attribute
Syntax Definitions").  Unfortunately since there is currently no free
LDAP v3 server software, this part has not received much testing and
should be considered experimental.  The framework is in place though.

 -- Function: ldap-decode-attribute attr
     Decode the attribute/value pair ATTR according to LDAP rules.  The
     attribute name is looked up in `ldap-attribute-syntaxes-alist' and
     the corresponding decoder is then retrieved from
     `ldap-attribute-syntax-decoders'' and applied on the value(s).

* Menu:

* LDAP Internationalization Variables::
* Encoder/Decoder Functions::


File: lispref.info,  Node: LDAP Internationalization Variables,  Next: Encoder/Decoder Functions,  Prev: LDAP Internationalization,  Up: LDAP Internationalization

60.2.4.1 LDAP Internationalization Variables
............................................

 -- Variable: ldap-ignore-attribute-codings
     If non-`nil', no encoding/decoding will be performed LDAP
     attribute values

 -- Variable: ldap-coding-system
     Coding system of LDAP string values.  LDAP v3 specifies the coding
     system of strings to be UTF-8.  You need an XEmacs with Mule
     support for this.

 -- Variable: ldap-default-attribute-decoder
     Decoder function to use for attributes whose syntax is unknown.
     Such a function receives an encoded attribute value as a string
     and should return the decoded value as a string.

 -- Variable: ldap-attribute-syntax-encoders
     A vector of functions used to encode LDAP attribute values.  The
     sequence of functions corresponds to the sequence of LDAP
     attribute syntax object identifiers of the form
     1.3.6.1.4.1.1466.1115.121.1.* as defined in RFC2252 section 4.3.2.
     As of this writing, only a few encoder functions are available.

 -- Variable: ldap-attribute-syntax-decoders
     A vector of functions used to decode LDAP attribute values.  The
     sequence of functions corresponds to the sequence of LDAP
     attribute syntax object identifiers of the form
     1.3.6.1.4.1.1466.1115.121.1.* as defined in RFC2252 section 4.3.2.
     As of this writing, only a few decoder functions are available.

 -- Variable: ldap-attribute-syntaxes-alist
     A map of LDAP attribute names to their type object id minor number.
     This table is built from RFC2252 Section 5 and RFC2256 Section 5.


File: lispref.info,  Node: Encoder/Decoder Functions,  Prev: LDAP Internationalization Variables,  Up: LDAP Internationalization

60.2.4.2 Encoder/Decoder Functions
..................................

 -- Function: ldap-encode-boolean bool
     A function that encodes an elisp boolean BOOL into a LDAP boolean
     string representation.

 -- Function: ldap-decode-boolean str
     A function that decodes a LDAP boolean string representation STR
     into an elisp boolean.

 -- Function: ldap-decode-string str
     Decode a string STR according to `ldap-coding-system'.

 -- Function: ldap-encode-string str
     Encode a string STR according to `ldap-coding-system'.

 -- Function: ldap-decode-address str
     Decode an address STR according to `ldap-coding-system' and
     replacing $ signs with newlines as specified by LDAP encoding
     rules for addresses.

 -- Function: ldap-encode-address str
     Encode an address STR according to `ldap-coding-system' and
     replacing newlines with $ signs as specified by LDAP encoding
     rules for addresses.


File: lispref.info,  Node: Syntax of Search Filters,  Prev: XEmacs LDAP API,  Up: LDAP Support

60.3 Syntax of Search Filters
=============================

LDAP search functions use RFC1558 syntax to describe the search filter.
In that syntax simple filters have the form:

     (<attr> <filtertype> <value>)

   `<attr>' is an attribute name such as `cn' for Common Name, `o' for
Organization, etc...

   `<value>' is the corresponding value.  This is generally an exact
string but may also contain `*' characters as wildcards

   `filtertype' is one `=' `~=', `<=', `>=' which respectively describe
equality, approximate equality, inferiority and superiority.

   Thus `(cn=John Smith)' matches all records having a canonical name
equal to John Smith.

   A special case is the presence filter `(<attr>=*' which matches
records containing a particular attribute.  For instance `(mail=*)'
matches all records containing a `mail' attribute.

   Simple filters can be connected together with the logical operators
`&', `|' and `!' which stand for the usual and, or and not operators.

   `(&(objectClass=Person)(mail=*)(|(sn=Smith)(givenname=John)))'
matches records of class `Person' containing a `mail' attribute and
corresponding to people whose last name is `Smith' or whose first name
is `John'.


File: lispref.info,  Node: PostgreSQL Support,  Next: Internationalization,  Prev: LDAP Support,  Up: Top

61 PostgreSQL Support
*********************

XEmacs can be linked with PostgreSQL libpq run-time support to provide
relational database access from Emacs Lisp code.

* Menu:

* Building XEmacs with PostgreSQL support::
* XEmacs PostgreSQL libpq API::
* XEmacs PostgreSQL libpq Examples::


File: lispref.info,  Node: Building XEmacs with PostgreSQL support,  Next: XEmacs PostgreSQL libpq API,  Up: PostgreSQL Support

61.1 Building XEmacs with PostgreSQL support
============================================

XEmacs PostgreSQL support requires linking to the PostgreSQL libpq
library.  Describing how to build and install PostgreSQL is beyond the
scope of this document.  See the PostgreSQL manual for details.

   If you have installed XEmacs from one of the binary kits on
(`ftp://ftp.xemacs.org/'), or are using an XEmacs binary from a CD ROM,
you may have XEmacs PostgreSQL support by default.  `M-x
describe-installation' will tell you if you do.

   If you are building XEmacs from source, you need to install
PostgreSQL first.  On some systems, PostgreSQL will come pre-installed
in /usr.  In this case, it should be autodetected when you run
configure.  If PostgreSQL is installed into its default location,
`/usr/local/pgsql', you must specify `--site-prefixes=/usr/local/pgsql'
when you run configure.  If PostgreSQL is installed into another
location, use that instead of `/usr/local/pgsql' when specifying
`--site-prefixes'.

   As of XEmacs 21.2, PostgreSQL versions 6.5.3 and 7.0 are supported.
XEmacs Lisp support for V7.0 is somewhat more extensive than support for
V6.5.  In particular, asynchronous queries are supported.


File: lispref.info,  Node: XEmacs PostgreSQL libpq API,  Next: XEmacs PostgreSQL libpq Examples,  Prev: Building XEmacs with PostgreSQL support,  Up: PostgreSQL Support

61.2 XEmacs PostgreSQL libpq API
================================

The XEmacs PostgreSQL API is intended to be a policy-free, low-level
binding to libpq.  The intent is to provide all the basic functionality
and then let high level Lisp code decide its own policies.

   This documentation assumes that the reader has knowledge of SQL, but
requires no prior knowledge of libpq.

   There are many examples in this manual and some setup will be
required.  In order to run most of the following examples, the
following code needs to be executed.  In addition to the data is in
this table, nearly all of the examples will assume that the free
variable `P' refers to this database connection.  The examples in the
original edition of this manual were run against Postgres 7.0beta1.

     (progn
       (setq P (pq-connectdb ""))
       ;; id is the primary key, shikona is a Japanese word that
       ;; means `the professional name of a Sumo wrestler', and
       ;; rank is the Sumo rank name.
       (pq-exec P (concat "CREATE TABLE xemacs_test"
                          " (id int, shikona text, rank text);"))
       (pq-exec P "COPY xemacs_test FROM stdin;")
       (pq-put-line P "1\tMusashimaru\tYokuzuna\n")
       (pq-put-line P "2\tDejima\tOozeki\n")
       (pq-put-line P "3\tMusoyama\tSekiwake\n")
       (pq-put-line P "4\tMiyabiyama\tSekiwake\n")
       (pq-put-line P "5\tWakanoyama\tMaegashira\n")
       (pq-put-line P "\\.\n")
       (pq-end-copy P))
          => nil

* Menu:

* libpq Lisp Variables::
* libpq Lisp Symbols and DataTypes::
* Synchronous Interface Functions::
* Asynchronous Interface Functions::
* Large Object Support::
* Other libpq Functions::
* Unimplemented libpq Functions::


File: lispref.info,  Node: libpq Lisp Variables,  Next: libpq Lisp Symbols and DataTypes,  Prev: XEmacs PostgreSQL libpq API,  Up: XEmacs PostgreSQL libpq API

61.2.1 libpq Lisp Variables
---------------------------

Various Unix environment variables are used by libpq to provide defaults
to the many different parameters.  In the XEmacs Lisp API, these
environment variables are bound to Lisp variables to provide more
convenient access to Lisp Code.  These variables are passed to the
backend database server during the establishment of a database
connection and when the `pq-setenv' call is made.

 -- Variable: pg:host
     Initialized from the `PGHOST' environment variable.  The default
     host to connect to.

 -- Variable: pg:user
     Initialized from the `PGUSER' environment variable.  The default
     database user name.

 -- Variable: pg:options
     Initialized from the `PGOPTIONS' environment variable.  Default
     additional server options.

 -- Variable: pg:port
     Initialized from the `PGPORT' environment variable.  The default
     TCP port to connect to.

 -- Variable: pg:tty
     Initialized from the `PGTTY' environment variable.  The default
     debugging TTY.

     Compatibility note:  Debugging TTYs are turned off in the XEmacs
     Lisp binding.

 -- Variable: pg:database
     Initialized from the `PGDATABASE' environment variable.  The
     default database to connect to.

 -- Variable: pg:realm
     Initialized from the `PGREALM' environment variable.  The default
     Kerberos realm.

 -- Variable: pg:client-encoding
     Initialized from the `PGCLIENTENCODING' environment variable.  The
     default client encoding.

     Compatibility note:  This variable is not present in non-Mule
     XEmacsen.  This variable is not present in versions of libpq prior
     to 7.0.  In the current implementation, client encoding is
     equivalent to the `file-name-coding-system' format.

 -- Variable: pg:authtype
     Initialized from the `PGAUTHTYPE' environment variable.  The
     default authentication scheme used.

     Compatibility note:  This variable is unused in versions of libpq
     after 6.5.  It is not implemented at all in the XEmacs Lisp
     binding.

 -- Variable: pg:geqo
     Initialized from the `PGGEQO' environment variable.  Genetic
     optimizer options.

 -- Variable: pg:cost-index
     Initialized from the `PGCOSTINDEX' environment variable.  Cost
     index options.

 -- Variable: pg:cost-heap
     Initialized from the `PGCOSTHEAP' environment variable.  Cost heap
     options.

 -- Variable: pg:tz
     Initialized from the `PGTZ' environment variable.  Default
     timezone.

 -- Variable: pg:date-style
     Initialized from the `PGDATESTYLE' environment variable.  Default
     date style in returned date objects.

 -- Variable: pg-coding-system
     This is a variable controlling which coding system is used to
     encode non-ASCII strings sent to the database.

     Compatibility Note: This variable is not present in InfoDock.


File: lispref.info,  Node: libpq Lisp Symbols and DataTypes,  Next: Synchronous Interface Functions,  Prev: libpq Lisp Variables,  Up: XEmacs PostgreSQL libpq API

61.2.2 libpq Lisp Symbols and Datatypes
---------------------------------------

The following set of symbols are used to represent the intermediate
states involved in the asynchronous interface.

 -- Symbol: pgres::polling-failed
     Undocumented.  A fatal error has occurred during processing of an
     asynchronous operation.

 -- Symbol: pgres::polling-reading
     An intermediate status return during an asynchronous operation.  It
     indicates that one may use `select' before polling again.

 -- Symbol: pgres::polling-writing
     An intermediate status return during an asynchronous operation.  It
     indicates that one may use `select' before polling again.

 -- Symbol: pgres::polling-ok
     An asynchronous operation has successfully completed.

 -- Symbol: pgres::polling-active
     An intermediate status return during an asynchronous operation.
     One can call the poll function again immediately.

 -- Function: pq-pgconn conn field
     CONN A database connection object.  FIELD A symbol indicating
     which field of PGconn to fetch.  Possible values are shown in the
     following table.
    `pq::db'
          Database name

    `pq::user'
          Database user name

    `pq::pass'
          Database user's password

    `pq::host'
          Hostname database server is running on

    `pq::port'
          TCP port number used in the connection

    `pq::tty'
          Debugging TTY

          Compatibility note:  Debugging TTYs are not used in the
          XEmacs Lisp API.

    `pq::options'
          Additional server options

    `pq::status'
          Connection status.  Possible return values are shown in the
          following table.
         `pg::connection-ok'
               The normal, connected status.

         `pg::connection-bad'
               The connection is not open and the PGconn object needs
               to be deleted by `pq-finish'.

         `pg::connection-started'
               An asynchronous connection has been started, but is not
               yet complete.

         `pg::connection-made'
               An asynchronous connect has been made, and there is data
               waiting to be sent.

         `pg::connection-awaiting-response'
               Awaiting data from the backend during an asynchronous
               connection.

         `pg::connection-auth-ok'
               Received authentication, waiting for the backend to
               start up.

         `pg::connection-setenv'
               Negotiating environment during an asynchronous
               connection.

    `pq::error-message'
          The last error message that was delivered to this connection.

    `pq::backend-pid'
          The process ID of the backend database server.

   The `PGresult' object is used by libpq to encapsulate the results of
queries.  The printed representation takes on four forms.  When the
PGresult object contains tuples from an SQL `SELECT' it will look like:

     (setq R (pq-exec P "SELECT * FROM xemacs_test;"))
          => #<PGresult PGRES_TUPLES_OK[5] - SELECT>

   The number in brackets indicates how many rows of data are available.
When the PGresult object is the result of a command query that doesn't
return anything, it will look like:

     (pq-exec P "CREATE TABLE a_new_table (i int);")
          => #<PGresult PGRES_COMMAND_OK - CREATE>

   When either the query is a command-type query that can affect a
number of different rows, but doesn't return any of them it will look
like:

     (progn
       (pq-exec P "INSERT INTO a_new_table VALUES (1);")
       (pq-exec P "INSERT INTO a_new_table VALUES (2);")
       (pq-exec P "INSERT INTO a_new_table VALUES (3);")
       (setq R (pq-exec P "DELETE FROM a_new_table;")))
          => #<PGresult PGRES_COMMAND_OK[3] - DELETE 3>

   Lastly, when the underlying PGresult object has been deallocated
directly by `pq-clear' the printed representation will look like:

     (progn
       (setq R (pq-exec P "SELECT * FROM xemacs_test;"))
       (pq-clear R)
       R)
          => #<PGresult DEAD>

   The following set of functions are accessors to various data in the
PGresult object.

 -- Function: pq-result-status result
     Return status of a query result.  RESULT is a PGresult object.
     The return value is one of the symbols in the following table.
    `pgres::empty-query'
          A query contained no text.  This is usually the result of a
          recoverable error, or a minor programming error.

    `pgres::command-ok'
          A query command that doesn't return anything was executed
          properly by the backend.

    `pgres::tuples-ok'
          A query command that returns tuples was executed properly by
          the backend.

    `pgres::copy-out'
          Copy Out data transfer is in progress.

    `pgres::copy-in'
          Copy In data transfer is in progress.

    `pgres::bad-response'
          An unexpected response was received from the backend.

    `pgres::nonfatal-error'
          Undocumented.  This value is returned when the libpq function
          `PQresultStatus' is called with a `NULL' pointer.

    `pgres::fatal-error'
          Undocumented.  An error has occurred in processing the query
          and the operation was not completed.

 -- Function: pq-res-status result
     Return the query result status as a string, not a symbol.  RESULT
     is a PGresult object.

          (setq R (pq-exec P "SELECT * FROM xemacs_test;"))
               => #<PGresult PGRES_TUPLES_OK[5] - SELECT>
          (pq-res-status R)
               => "PGRES_TUPLES_OK"

 -- Function: pq-result-error-message result
     Return an error message generated by the query, if any.  RESULT is
     a PGresult object.

          (setq R (pq-exec P "SELECT * FROM xemacs-test;"))
               => <A fatal error is signaled in the echo area>
          (pq-result-error-message R)
               => "ERROR:  parser: parse error at or near \"-\"
          "

 -- Function: pq-ntuples result
     Return the number of tuples in the query result.  RESULT is a
     PGresult object.

          (setq R (pq-exec P "SELECT * FROM xemacs_test;"))
               => #<PGresult PGRES_TUPLES_OK[5] - SELECT>
          (pq-ntuples R)
               => 5

 -- Function: pq-nfields result
     Return the number of fields in each tuple of the query result.
     RESULT is a PGresult object.

          (setq R (pq-exec P "SELECT * FROM xemacs_test;"))
               => #<PGresult PGRES_TUPLES_OK[5] - SELECT>
          (pq-nfields R)
               => 3

 -- Function: pq-binary-tuples result
     Returns t if binary tuples are present in the results, nil
     otherwise.  RESULT is a PGresult object.

          (setq R (pq-exec P "SELECT * FROM xemacs_test;"))
               => #<PGresult PGRES_TUPLES_OK[5] - SELECT>
          (pq-binary-tuples R)
               => nil

 -- Function: pq-fname result field-index
     Returns the name of a specific field.  RESULT is a PGresult object.
     FIELD-INDEX is the number of the column to select from.  The first
     column is number zero.

          (let (i l)
            (setq R (pq-exec P "SELECT * FROM xemacs_test;"))
            (setq i (pq-nfields R))
            (while (>= (decf i) 0)
              (push (pq-fname R i) l))
            l)
               => ("id" "shikona" "rank")

 -- Function: pq-fnumber result field-name
     Return the field number corresponding to the given field name.  -1
     is returned on a bad field name.  RESULT is a PGresult object.
     FIELD-NAME is a string representing the field name to find.
          (setq R (pq-exec P "SELECT * FROM xemacs_test;"))
               => #<PGresult PGRES_TUPLES_OK[5] - SELECT>
          (pq-fnumber R "id")
               => 0
          (pq-fnumber R "Not a field")
               => -1

 -- Function: pq-ftype result field-num
     Return an integer code representing the data type of the specified
     column.  RESULT is a PGresult object.  FIELD-NUM is the field
     number.

     The return value of this function is the Object ID (Oid) in the
     database of the type.  Further queries need to be made to various
     system tables in order to convert this value into something useful.

 -- Function: pq-fmod result field-num
     Return the type modifier code associated with a field.  Field
     numbers start at zero.  RESULT is a PGresult object.  FIELD-INDEX
     selects which field to use.

 -- Function: pq-fsize result field-index
     Return size of the given field.  RESULT is a PGresult object.
     FIELD-INDEX selects which field to use.

          (let (i l)
            (setq R (pq-exec P "SELECT * FROM xemacs_test;"))
            (setq i (pq-nfields R))
            (while (>= (decf i) 0)
              (push (list (pq-ftype R i) (pq-fsize R i)) l))
            l)
               => ((23 23) (25 25) (25 25))

 -- Function: pq-get-value result tup-num field-num
     Retrieve a return value.  RESULT is a PGresult object.  TUP-NUM
     selects which tuple to fetch from.  FIELD-NUM selects which field
     to fetch from.

     Both tuples and fields are numbered from zero.

          (setq R (pq-exec P "SELECT * FROM xemacs_test;"))
               => #<PGresult PGRES_TUPLES_OK[5] - SELECT>
          (pq-get-value R 0 1)
               => "Musashimaru"
          (pq-get-value R 1 1)
               => "Dejima"
          (pq-get-value R 2 1)
               => "Musoyama"

 -- Function: pq-get-length result tup-num field-num
     Return the length of a specific value.  RESULT is a PGresult
     object.  TUP-NUM selects which tuple to fetch from.  FIELD-NUM
     selects which field to fetch from.

          (setq R (pq-exec P "SELECT * FROM xemacs_test;"))
               => #<PGresult PGRES_TUPLES_OK[5] - SELECT>
          (pq-get-length R 0 1)
               => 11
          (pq-get-length R 1 1)
               => 6
          (pq-get-length R 2 1)
               => 8

 -- Function: pq-get-is-null result tup-num field-num
     Return t if the specific value is the SQL `NULL'.  RESULT is a
     PGresult object.  TUP-NUM selects which tuple to fetch from.
     FIELD-NUM selects which field to fetch from.

 -- Function: pq-cmd-status result
     Return a summary string from the query.  RESULT is a PGresult
     object.
          (setq R (pq-exec P "INSERT INTO xemacs_test
                             VALUES (6, 'Wakanohana', 'Yokozuna');"))
               => #<PGresult PGRES_COMMAND_OK[1] - INSERT 542086 1>
          (pq-cmd-status R)
               => "INSERT 542086 1"
          (setq R (pq-exec P "UPDATE xemacs_test SET rank='retired'
                              WHERE shikona='Wakanohana';"))
               => #<PGresult PGRES_COMMAND_OK[1] - UPDATE 1>
          (pq-cmd-status R)
               => "UPDATE 1"

     Note that the first number returned from an insertion, like in the
     example, is an object ID number and will almost certainly vary from
     system to system since object ID numbers in Postgres must be unique
     across all databases.

 -- Function: pq-cmd-tuples result
     Return the number of tuples if the last command was an
     INSERT/UPDATE/DELETE.  If the last command was something else, the
     empty string is returned.  RESULT is a PGresult object.

          (setq R (pq-exec P "INSERT INTO xemacs_test VALUES
                              (7, 'Takanohana', 'Yokuzuna');"))
               => #<PGresult PGRES_COMMAND_OK[1] - INSERT 38688 1>
          (pq-cmd-tuples R)
               => "1"
          (setq R (pq-exec P "SELECT * from xemacs_test;"))
               => #<PGresult PGRES_TUPLES_OK[7] - SELECT>
          (pq-cmd-tuples R)
               => ""
          (setq R (pq-exec P "DELETE FROM xemacs_test
                              WHERE shikona LIKE '%hana';"))
               => #<PGresult PGRES_COMMAND_OK[2] - DELETE 2>
          (pq-cmd-tuples R)
               => "2"

 -- Function: pq-oid-value result
     Return the object id of the insertion if the last command was an
     INSERT.  0 is returned if the last command was not an insertion.
     RESULT is a PGresult object.

     In the first example, the numbers you will see on your local
     system will almost certainly be different, however the second
     number from the right in the unprintable PGresult object and the
     number returned by `pq-oid-value' should match.
          (setq R (pq-exec P "INSERT INTO xemacs_test VALUES
                              (8, 'Terao', 'Maegashira');"))
               => #<PGresult PGRES_COMMAND_OK[1] - INSERT 542089 1>
          (pq-oid-value R)
               => 542089
          (setq R (pq-exec P "SELECT shikona FROM xemacs_test
                              WHERE rank='Maegashira';"))
               => #<PGresult PGRES_TUPLES_OK[2] - SELECT>
          (pq-oid-value R)
               => 0

 -- Function: pq-make-empty-pgresult conn status
     Create an empty pgresult with the given status.  CONN a database
     connection object STATUS a value that can be returned by
     `pq-result-status'.

     The caller is responsible for making sure the return value gets
     properly freed.


File: lispref.info,  Node: Synchronous Interface Functions,  Next: Asynchronous Interface Functions,  Prev: libpq Lisp Symbols and DataTypes,  Up: XEmacs PostgreSQL libpq API

61.2.3 Synchronous Interface Functions
--------------------------------------

 -- Function: pq-connectdb conninfo
     Establish a (synchronous) database connection.  CONNINFO A string
     of blank separated options.  Options are of the form "OPTION =
     VALUE".  If VALUE contains blanks, it must be single quoted.
     Blanks around the equal sign are optional.  Multiple option
     assignments are blank separated.
          (pq-connectdb "dbname=japanese port = 25432")
               => #<PGconn localhost:25432 steve/japanese>
     The printed representation of a database connection object has four
     fields.  The first field is the hostname where the database server
     is running (in this case localhost), the second field is the port
     number, the third field is the database user name, and the fourth
     field is the name of the database.

     Database connection objects which have been disconnected and will
     generate an immediate error if they are used look like:
            #<PGconn BAD>
     Bad connections can be reestablished with `pq-reset', or deleted
     entirely with `pq-finish'.

     A database connection object that has been deleted looks like:
          (let ((P1 (pq-connectdb "")))
            (pq-finish P1)
            P1)
               => #<PGconn DEAD>

     Note that database connection objects are the most heavy weight
     objects in XEmacs Lisp at this writing, usually representing as
     much as several megabytes of virtual memory on the machine the
     database server is running on.  It is wisest to explicitly delete
     them when you are finished with them, rather than letting garbage
     collection do it.  An example idiom is:

          (let ((P (pq-connectiondb "")))
            (unwind-protect
                (progn
          	(...)) ; access database here
              (pq-finish P)))

     The following options are available in the options string:
    `authtype'
          Authentication type.  Same as `PGAUTHTYPE'.  This is no
          longer used.

    `user'
          Database user name.  Same as `PGUSER'.

    `password'
          Database password.

    `dbname'
          Database name.  Same as `PGDATABASE'

    `host'
          Symbolic hostname.  Same as `PGHOST'.

    `hostaddr'
          Host address as four octets (eg. like 192.168.1.1).

    `port'
          TCP port to connect to.  Same as `PGPORT'.

    `tty'
          Debugging TTY.  Same as `PGTTY'.  This value is suppressed in
          the XEmacs Lisp API.

    `options'
          Extra backend database options.  Same as `PGOPTIONS'.
     A database connection object is returned regardless of whether a
     connection was established or not.

 -- Function: pq-reset conn
     Reestablish database connection.  CONN A database connection
     object.

     This function reestablishes a database connection using the
     original connection parameters.  This is useful if something has
     happened to the TCP link and it has become broken.

 -- Function: pq-exec conn query
     Make a synchronous database query.  CONN A database connection
     object.  QUERY A string containing an SQL query.  A PGresult
     object is returned, which in turn may be queried by its many
     accessor functions to retrieve state out of it.  If the query
     string contains multiple SQL commands, only results from the final
     command are returned.

          (setq R (pq-exec P "SELECT * FROM xemacs_test;
          DELETE FROM xemacs_test WHERE id=8;"))
               => #<PGresult PGRES_COMMAND_OK[1] - DELETE 1>

 -- Function: pq-notifies conn
     Return the latest async notification that has not yet been handled.
     CONN A database connection object.  If there has been a
     notification, then a list of two elements will be returned.  The
     first element contains the relation name being notified, the second
     element contains the backend process ID number.  nil is returned
     if there aren't any notifications to process.

 -- Function: PQsetenv conn
     Synchronous transfer of environment variables to a backend CONN A
     database connection object.

     Environment variable transfer is done as a normal part of database
     connection.

     Compatibility note: This function was present but not documented
     in versions of libpq prior to 7.0.


File: lispref.info,  Node: Asynchronous Interface Functions,  Next: Large Object Support,  Prev: Synchronous Interface Functions,  Up: XEmacs PostgreSQL libpq API

61.2.4 Asynchronous Interface Functions
---------------------------------------

Making command by command examples is too complex with the asynchronous
interface functions.  See the examples section for complete calling
sequences.

 -- Function: pq-connect-start conninfo
     Begin establishing an asynchronous database connection.  CONNINFO
     A string containing the connection options.  See the documentation
     of `pq-connectdb' for a listing of all the available flags.

 -- Function: pq-connect-poll conn
     An intermediate function to be called during an asynchronous
     database connection.  CONN A database connection object.  The
     result codes are documented in a previous section.

 -- Function: pq-is-busy conn
     Returns t if `pq-get-result' would block waiting for input.  CONN
     A database connection object.

 -- Function: pq-consume-input conn
     Consume any available input from the backend.  CONN A database
     connection object.

     Nil is returned if anything bad happens.

 -- Function: pq-reset-start conn
     Reset connection to the backend asynchronously.  CONN A database
     connection object.

 -- Function: pq-reset-poll conn
     Poll an asynchronous reset for completion CONN A database
     connection object.

 -- Function: pq-reset-cancel conn
     Attempt to request cancellation of the current operation.  CONN A
     database connection object.

     The return value is t if the cancel request was successfully
     dispatched, nil if not (in which case conn->errorMessage is set).
     Note: successful dispatch is no guarantee that there will be any
     effect at the backend.  The application must read the operation
     result as usual.

 -- Function: pq-send-query conn query
     Submit a query to Postgres and don't wait for the result.  CONN A
     database connection object.  Returns: t if successfully submitted
            nil if error (conn->errorMessage is set)

 -- Function: pq-get-result conn
     Retrieve an asynchronous result from a query.  CONN A database
     connection object.

     `nil' is returned when no more query work remains.

 -- Function: pq-set-nonblocking conn arg
     Sets the PGconn's database connection non-blocking if the arg is
     TRUE or makes it non-blocking if the arg is FALSE, this will not
     protect you from PQexec(), you'll only be safe when using the
     non-blocking API.  CONN A database connection object.

 -- Function: pq-is-nonblocking conn
     Return the blocking status of the database connection CONN A
     database connection object.

 -- Function: pq-flush conn
     Force the write buffer to be written (or at least try) CONN A
     database connection object.

 -- Function: PQsetenvStart conn
     Start asynchronously passing environment variables to a backend.
     CONN A database connection object.

     Compatibility note: this function is only available with libpq-7.0.

 -- Function: PQsetenvPoll conn
     Check an asynchronous environment variables transfer for
     completion.  CONN A database connection object.

     Compatibility note: this function is only available with libpq-7.0.

 -- Function: PQsetenvAbort conn
     Attempt to terminate an asynchronous environment variables
     transfer.  CONN A database connection object.

     Compatibility note: this function is only available with libpq-7.0.


File: lispref.info,  Node: Large Object Support,  Next: Other libpq Functions,  Prev: Asynchronous Interface Functions,  Up: XEmacs PostgreSQL libpq API

61.2.5 Large Object Support
---------------------------

 -- Function: pq-lo-import conn filename
     Import a file as a large object into the database.  CONN a
     database connection object FILENAME filename to import

     On success, the object id is returned.

 -- Function: pq-lo-export conn oid filename
     Copy a large object in the database into a file.  CONN a database
     connection object.  OID object id number of a large object.
     FILENAME filename to export to.


File: lispref.info,  Node: Other libpq Functions,  Next: Unimplemented libpq Functions,  Prev: Large Object Support,  Up: XEmacs PostgreSQL libpq API

61.2.6 Other libpq Functions
----------------------------

 -- Function: pq-finish conn
     Destroy a database connection object by calling free on it.  CONN
     a database connection object

     It is possible to not call this routine because the usual XEmacs
     garbage collection mechanism will call the underlying libpq
     routine whenever it is releasing stale `PGconn' objects.  However,
     this routine is useful in `unwind-protect' clauses to make
     connections go away quickly when unrecoverable errors have
     occurred.

     After calling this routine, the printed representation of the
     XEmacs wrapper object will contain the string "DEAD".

 -- Function: pq-client-encoding conn
     Return the client encoding as an integer code.  CONN a database
     connection object

          (pq-client-encoding P)
               => 1

     Compatibility note: This function did not exist prior to libpq-7.0
     and does not exist in a non-Mule XEmacs.

 -- Function: pq-set-client-encoding conn encoding
     Set client coding system.  CONN a database connection object
     ENCODING a string representing the desired coding system

          (pq-set-client-encoding P "EUC_JP")
               => 0

     The current idiom for ensuring proper coding system conversion is
     the following (illustrated for EUC Japanese encoding):
          (setq P (pq-connectdb "..."))
          (let ((file-name-coding-system 'euc-jp)
                (pg-coding-system 'euc-jp))
            (pq-set-client-encoding "EUC_JP")
            ...)
          (pq-finish P)
     Compatibility note: This function did not exist prior to libpq-7.0
     and does not exist in a non-Mule XEmacs.

 -- Function: pq-env-2-encoding
     Return the integer code representing the coding system in
     `PGCLIENTENCODING'.

          (pq-env-2-encoding)
               => 0
     Compatibility note: This function did not exist prior to libpq-7.0
     and does not exist in a non-Mule XEmacs.

 -- Function: pq-clear res
     Destroy a query result object by calling free() on it.  RES a
     query result object

     Note:  The memory allocation systems of libpq and XEmacs are
     different.  The XEmacs representation of a query result object
     will have both the XEmacs version and the libpq version freed at
     the next garbage collection when the object is no longer being
     referenced.  Calling this function does not release the XEmacs
     object, it is still subject to the usual rules for Lisp objects.
     The printed representation of the XEmacs object will contain the
     string "DEAD" after this routine is called indicating that it is no
     longer useful for anything.

 -- Function: pq-conn-defaults
     Return a data structure that represents the connection defaults.
     The data is returned as a list of lists, where each sublist
     contains info regarding a single option.


File: lispref.info,  Node: Unimplemented libpq Functions,  Prev: Other libpq Functions,  Up: XEmacs PostgreSQL libpq API

61.2.7 Unimplemented libpq Functions
------------------------------------

 -- Unimplemented Function: PGconn *PQsetdbLogin (char *pghost, char
          *pgport, char *pgoptions, char *pgtty, char *dbName, char
          *login, char *pwd)
     Synchronous database connection.  PGHOST is the hostname of the
     PostgreSQL backend to connect to.  PGPORT is the TCP port number
     to use.  PGOPTIONS specifies other backend options.  PGTTY
     specifies the debugging tty to use.  DBNAME specifies the database
     name to use.  LOGIN specifies the database user name.  PWD
     specifies the database user's password.

     This routine is deprecated as of libpq-7.0, and its functionality
     can be replaced by external Lisp code if needed.

 -- Unimplemented Function: PGconn *PQsetdb (char *pghost, char
          *pgport, char *pgoptions, char *pgtty, char *dbName)
     Synchronous database connection.  PGHOST is the hostname of the
     PostgreSQL backend to connect to.  PGPORT is the TCP port number
     to use.  PGOPTIONS specifies other backend options.  PGTTY
     specifies the debugging tty to use.  DBNAME specifies the database
     name to use.

     This routine was deprecated in libpq-6.5.

 -- Unimplemented Function: int PQsocket (PGconn *conn)
     Return socket file descriptor to a backend database process.  CONN
     database connection object.

 -- Unimplemented Function: void PQprint (FILE *fout, PGresult *res,
          PGprintOpt *ps)
     Print out the results of a query to a designated C stream.  FOUT C
     stream to print to RES the query result object to print PS the
     print options structure.

     This routine is deprecated as of libpq-7.0 and cannot be sensibly
     exported to XEmacs Lisp.

 -- Unimplemented Function: void PQdisplayTuples (PGresult *res, FILE
          *fp, int fillAlign, char *fieldSep, int printHeader, int
          quiet)
     RES query result object to print FP C stream to print to FILLALIGN
     pad the fields with spaces FIELDSEP field separator PRINTHEADER
     display headers?  QUIET

     This routine was deprecated in libpq-6.5.

 -- Unimplemented Function: void PQprintTuples (PGresult *res, FILE
          *fout, int printAttName, int terseOutput, int width)
     RES query result object to print FOUT C stream to print to
     PRINTATTNAME print attribute names TERSEOUTPUT delimiter bars
     WIDTH width of column, if 0, use variable width

     This routine was deprecated in libpq-6.5.

 -- Unimplemented Function: int PQmblen (char *s, int encoding)
     Determine length of a multibyte encoded char at `*s'.  S encoded
     string ENCODING type of encoding

     Compatibility note:  This function was introduced in libpq-7.0.

 -- Unimplemented Function: void PQtrace (PGconn *conn, FILE
          *debug_port)
     Enable tracing on `debug_port'.  CONN database connection object.
     DEBUG_PORT C output stream to use.

 -- Unimplemented Function: void PQuntrace (PGconn *conn)
     Disable tracing.  CONN database connection object.

 -- Unimplemented Function: char *PQoidStatus (PGconn *conn)
     Return the object id as a string of the last tuple inserted.  CONN
     database connection object.

     Compatibility note: This function is deprecated in libpq-7.0,
     however it is used internally by the XEmacs binding code when
     linked against versions prior to 7.0.

 -- Unimplemented Function: PGresult *PQfn (PGconn *conn, int fnid, int
          *result_buf, int *result_len, int result_is_int, PQArgBlock
          *args, int nargs)
     "Fast path" interface -- not really recommended for application use
     CONN A database connection object.  FNID RESULT_BUF RESULT_LEN
     RESULT_IS_INT ARGS NARGS

   The following set of very low level large object functions aren't
appropriate to be exported to Lisp.

 -- Unimplemented Function: int pq-lo-open (PGconn *conn, int lobjid,
          int mode)
     CONN a database connection object.  LOBJID a large object ID.
     MODE opening modes.

 -- Unimplemented Function: int pq-lo-close (PGconn *conn, int fd)
     CONN a database connection object.  FD a large object file
     descriptor

 -- Unimplemented Function: int pq-lo-read (PGconn *conn, int fd, char
          *buf, int len)
     CONN a database connection object.  FD a large object file
     descriptor.  BUF buffer to read into.  LEN size of buffer.

 -- Unimplemented Function: int pq-lo-write (PGconn *conn, int fd, char
          *buf, size_t len)
     CONN a database connection object.  FD a large object file
     descriptor.  BUF buffer to write from.  LEN size of buffer.

 -- Unimplemented Function: int pq-lo-lseek (PGconn *conn, int fd, int
          offset, int whence)
     CONN a database connection object.  FD a large object file
     descriptor.  OFFSET WHENCE

 -- Unimplemented Function: int pq-lo-creat (PGconn *conn, int mode)
     CONN a database connection object.  MODE opening modes.

 -- Unimplemented Function: int pq-lo-tell (PGconn *conn, int fd)
     CONN a database connection object.  FD a large object file
     descriptor.

 -- Unimplemented Function: int pq-lo-unlink (PGconn *conn, int lobjid)
     CONN a database connection object.  LBOJID a large object ID.


File: lispref.info,  Node: XEmacs PostgreSQL libpq Examples,  Prev: XEmacs PostgreSQL libpq API,  Up: PostgreSQL Support

61.3 XEmacs PostgreSQL libpq Examples
=====================================

This is an example of one method of establishing an asynchronous
connection.

     (defun database-poller (P)
       (message "%S before poll" (pq-pgconn P 'pq::status))
       (pq-connect-poll P)
       (message "%S after poll" (pq-pgconn P 'pq::status))
       (if (eq (pq-pgconn P 'pq::status) 'pg::connection-ok)
           (message "Done!")
         (add-timeout .1 'database-poller P)))
          => database-poller
     (progn
       (setq P (pq-connect-start ""))
       (add-timeout .1 'database-poller P))
          => pg::connection-started before poll
          => pg::connection-made after poll
          => pg::connection-made before poll
          => pg::connection-awaiting-response after poll
          => pg::connection-awaiting-response before poll
          => pg::connection-auth-ok after poll
          => pg::connection-auth-ok before poll
          => pg::connection-setenv after poll
          => pg::connection-setenv before poll
          => pg::connection-ok after poll
          => Done!
     P
          => #<PGconn localhost:25432 steve/steve>

   Here is an example of one method of doing an asynchronous reset.

     (defun database-poller (P)
       (let (PS)
         (message "%S before poll" (pq-pgconn P 'pq::status))
         (setq PS (pq-reset-poll P))
         (message "%S after poll [%S]" (pq-pgconn P 'pq::status) PS)
         (if (eq (pq-pgconn P 'pq::status) 'pg::connection-ok)
     	(message "Done!")
           (add-timeout .1 'database-poller P))))
          => database-poller
     (progn
       (pq-reset-start P)
       (add-timeout .1 'database-poller P))
          => pg::connection-started before poll
          => pg::connection-made after poll [pgres::polling-writing]
          => pg::connection-made before poll
          => pg::connection-awaiting-response after poll [pgres::polling-reading]
          => pg::connection-awaiting-response before poll
          => pg::connection-setenv after poll [pgres::polling-reading]
          => pg::connection-setenv before poll
          => pg::connection-ok after poll [pgres::polling-ok]
          => Done!
     P
          => #<PGconn localhost:25432 steve/steve>

   And finally, an asynchronous query.

     (defun database-poller (P)
       (let (R)
         (pq-consume-input P)
         (if (pq-is-busy P)
     	(add-timeout .1 'database-poller P)
           (setq R (pq-get-result P))
           (if R
     	  (progn
     	    (push R result-list)
     	    (add-timeout .1 'database-poller P))))))
          => database-poller
     (when (pq-send-query P "SELECT * FROM xemacs_test;")
       (setq result-list nil)
       (add-timeout .1 'database-poller P))
          => 885
     ;; wait a moment
     result-list
          => (#<PGresult PGRES_TUPLES_OK - SELECT>)

   Here is an example showing how multiple SQL statements in a single
query can have all their results collected.
     ;; Using the same `database-poller' function from the previous example
     (when (pq-send-query P "SELECT * FROM xemacs_test;
     SELECT * FROM pg_database;
     SELECT * FROM pg_user;")
       (setq result-list nil)
       (add-timeout .1 'database-poller P))
          => 1782
     ;; wait a moment
     result-list
          => (#<PGresult PGRES_TUPLES_OK - SELECT> #<PGresult PGRES_TUPLES_OK - SELECT> #<PGresult PGRES_TUPLES_OK - SELECT>)

   Here is an example which illustrates collecting all data from a
query, including the field names.

     (defun pg-util-query-results (results)
       "Retrieve results of last SQL query into a list structure."
       (let ((i (1- (pq-ntuples R)))
     	j l1 l2)
         (while (>= i 0)
           (setq j (1- (pq-nfields R)))
           (setq l2 nil)
           (while (>= j 0)
     	(push (pq-get-value R i j) l2)
     	(decf j))
           (push l2 l1)
           (decf i))
         (setq j (1- (pq-nfields R)))
         (setq l2 nil)
         (while (>= j 0)
           (push (pq-fname R j) l2)
           (decf j))
         (push l2 l1)
         l1))
          => pg-util-query-results
     (setq R (pq-exec P "SELECT * FROM xemacs_test ORDER BY field2 DESC;"))
          => #<PGresult PGRES_TUPLES_OK - SELECT>
     (pg-util-query-results R)
          => (("f1" "field2") ("a" "97") ("b" "97") ("stuff" "42") ("a string" "12") ("foo" "10") ("string" "2") ("text" "1"))

   Here is an example of a query that uses a database cursor.

     (let (data R)
       (setq R (pq-exec P "BEGIN;"))
       (setq R (pq-exec P "DECLARE k_cursor CURSOR FOR SELECT * FROM xemacs_test ORDER BY f1 DESC;"))

       (setq R (pq-exec P "FETCH k_cursor;"))
       (while (eq (pq-ntuples R) 1)
         (push (list (pq-get-value R 0 0) (pq-get-value R 0 1)) data)
         (setq R (pq-exec P "FETCH k_cursor;")))
       (setq R (pq-exec P "END;"))
       data)
          => (("a" "97") ("a string" "12") ("b" "97") ("foo" "10") ("string" "2") ("stuff" "42") ("text" "1"))

   Here's another example of cursors, this time with a Lisp macro to
implement a mapping function over a table.

     (defmacro map-db (P table condition callout)
       `(let (R)
          (pq-exec ,P "BEGIN;")
          (pq-exec ,P (concat "DECLARE k_cursor CURSOR FOR SELECT * FROM "
     			 ,table
     			 " "
     			 ,condition
     			 " ORDER BY f1 DESC;"))
          (setq R (pq-exec P "FETCH k_cursor;"))
          (while (eq (pq-ntuples R) 1)
            (,callout (pq-get-value R 0 0) (pq-get-value R 0 1))
            (setq R (pq-exec P "FETCH k_cursor;")))
          (pq-exec P "END;")))
          => map-db
     (defun callback (arg1 arg2)
       (message "arg1 = %s, arg2 = %s" arg1 arg2))
          => callback
     (map-db P "xemacs_test" "WHERE field2 > 10" callback)
          => arg1 = stuff, arg2 = 42
          => arg1 = b, arg2 = 97
          => arg1 = a string, arg2 = 12
          => arg1 = a, arg2 = 97
          => #<PGresult PGRES_COMMAND_OK - COMMIT>


File: lispref.info,  Node: Internationalization,  Next: MULE,  Prev: PostgreSQL Support,  Up: Top

62 Internationalization
***********************

* Menu:

* I18N Levels 1 and 2:: Support for different time, date, and currency formats.
* I18N Level 3::        Support for localized messages.
* I18N Level 4::        Support for Asian languages.


File: lispref.info,  Node: I18N Levels 1 and 2,  Next: I18N Level 3,  Up: Internationalization

62.1 I18N Levels 1 and 2
========================

XEmacs is now compliant with I18N levels 1 and 2.  Specifically, this
means that it is 8-bit clean and correctly handles time and date
functions.  XEmacs will correctly display the entire ISO-Latin 1
character set.

   The compose key may now be used to create any character in the
ISO-Latin 1 character set not directly available via the keyboard..  In
order for the compose key to work it is necessary to load the file
`x-compose.el'.  At any time while composing a character, `C-h' will
display all valid completions and the character which would be produced.


File: lispref.info,  Node: I18N Level 3,  Next: I18N Level 4,  Prev: I18N Levels 1 and 2,  Up: Internationalization

62.2 I18N Level 3
=================

* Menu:

* Level 3 Basics::
* Level 3 Primitives::
* Dynamic Messaging::
* Domain Specification::
* Documentation String Extraction::


File: lispref.info,  Node: Level 3 Basics,  Next: Level 3 Primitives,  Up: I18N Level 3

62.2.1 Level 3 Basics
---------------------

XEmacs now provides alpha-level functionality for I18N Level 3.  This
means that everything necessary for full messaging is available, but
not every file has been converted.

   The two message files which have been created are `src/emacs.po' and
`lisp/packages/mh-e.po'.  Both files need to be converted using
`msgfmt', and the resulting `.mo' files placed in some locale's
`LC_MESSAGES' directory.  The test "translations" in these files are
the original messages prefixed by `TRNSLT_'.

   The domain for a variable is stored on the variable's property list
under the property name VARIABLE-DOMAIN.  The function
`documentation-property' uses this information when translating a
variable's documentation.


File: lispref.info,  Node: Level 3 Primitives,  Next: Dynamic Messaging,  Prev: Level 3 Basics,  Up: I18N Level 3

62.2.2 Level 3 Primitives
-------------------------

 -- Function: gettext string
     This function looks up STRING in the default message domain and
     returns its translation.  If `I18N3' was not enabled when XEmacs
     was compiled, it just returns STRING.

 -- Function: dgettext domain string
     This function looks up STRING in the specified message domain and
     returns its translation.  If `I18N3' was not enabled when XEmacs
     was compiled, it just returns STRING.

 -- Function: bind-text-domain domain pathname
     This function associates a pathname with a message domain.  Here's
     how the path to message file is constructed under SunOS 5.x:

          `{pathname}/{LANG}/LC_MESSAGES/{domain}.mo'

     If `I18N3' was not enabled when XEmacs was compiled, this function
     does nothing.

 -- Special Form: domain string
     This function specifies the text domain used for translating
     documentation strings and interactive prompts of a function.  For
     example, write:

          (defun foo (arg) "Doc string" (domain "emacs-foo") ...)

     to specify `emacs-foo' as the text domain of the function `foo'.
     The "call" to `domain' is actually a declaration rather than a
     function; when actually called, `domain' just returns `nil'.

 -- Function: domain-of function
     This function returns the text domain of FUNCTION; it returns
     `nil' if it is the default domain.  If `I18N3' was not enabled
     when XEmacs was compiled, it always returns `nil'.


File: lispref.info,  Node: Dynamic Messaging,  Next: Domain Specification,  Prev: Level 3 Primitives,  Up: I18N Level 3

62.2.3 Dynamic Messaging
------------------------

The `format' function has been extended to permit you to change the
order of parameter insertion.  For example, the conversion format
`%1$s' inserts parameter one as a string, while `%2$s' inserts
parameter two.  This is useful when creating translations which require
you to change the word order.


File: lispref.info,  Node: Domain Specification,  Next: Documentation String Extraction,  Prev: Dynamic Messaging,  Up: I18N Level 3

62.2.4 Domain Specification
---------------------------

The default message domain of XEmacs is `emacs'.  For add-on packages,
it is best to use a different domain.  For example, let us say we want
to convert the "gorilla" package to use the domain `emacs-gorilla'.  To
translate the message "What gorilla?", use `dgettext' as follows:

     (dgettext "emacs-gorilla" "What gorilla?")

   A function (or macro) which has a documentation string or an
interactive prompt needs to be associated with the domain in order for
the documentation or prompt to be translated.  This is done with the
`domain' special form as follows:

     (defun scratch (location)
       "Scratch the specified location."
       (domain "emacs-gorilla")
       (interactive "sScratch: ")
       ... )

   It is most efficient to specify the domain in the first line of the
function body, before the `interactive' form.

   For variables and constants which have documentation strings,
specify the domain after the documentation.

 -- Special Form: defvar symbol [value [doc-string [domain]]]
     Example:
          (defvar weight 250 "Weight of gorilla, in pounds." "emacs-gorilla")

 -- Special Form: defconst symbol [value [doc-string [domain]]]
     Example:
          (defconst limbs 4 "Number of limbs" "emacs-gorilla")

 -- Function: autoload function filename &optional docstring
          interactive type
     This function defines FUNCTION to autoload from FILENAME Example:
          (autoload 'explore "jungle" "Explore the jungle." nil nil "emacs-gorilla")


File: lispref.info,  Node: Documentation String Extraction,  Prev: Domain Specification,  Up: I18N Level 3

62.2.5 Documentation String Extraction
--------------------------------------

The utility `etc/make-po' scans the file `DOC' to extract documentation
strings and creates a message file `doc.po'.  This file may then be
inserted within `emacs.po'.

   Currently, `make-po' is hard-coded to read from `DOC' and write to
`doc.po'.  In order to extract documentation strings from an add-on
package, first run `make-docfile' on the package to produce the `DOC'
file.  Then run `make-po -p' with the `-p' argument to indicate that we
are extracting documentation for an add-on package.

   (The `-p' argument is a kludge to make up for a subtle difference
between pre-loaded documentation and add-on documentation:  For add-on
packages, the final carriage returns in the strings produced by
`make-docfile' must be ignored.)


File: lispref.info,  Node: I18N Level 4,  Prev: I18N Level 3,  Up: Internationalization

62.3 I18N Level 4
=================

The Asian-language support in XEmacs is called "MULE".  *Note MULE::.


File: lispref.info,  Node: MULE,  Next: Tips,  Prev: Internationalization,  Up: Top

63 MULE
*******

"MULE" is the name originally given to the version of GNU Emacs
extended for multi-lingual (and in particular Asian-language) support.
"MULE" is short for "MUlti-Lingual Emacs".  It is an extension and
complete rewrite of Nemacs ("Nihon Emacs" where "Nihon" is the Japanese
word for "Japan"), which only provided support for Japanese.  XEmacs
refers to its multi-lingual support as "MULE support" since it is based
on "MULE".

* Menu:

* Internationalization Terminology::
                        Definition of various internationalization terms.
* Charsets::            Sets of related characters.
* MULE Characters::     Working with characters in XEmacs/MULE.
* Composite Characters:: Making new characters by overstriking other ones.
* Coding Systems::      Ways of representing a string of chars using integers.
* CCL::                 A special language for writing fast converters.
* Category Tables::     Subdividing charsets into groups.
* Unicode Support::     The universal coded character set.
* Charset Unification:: Handling overlapping character sets.
* Charsets and Coding Systems:: Tables and reference information.


File: lispref.info,  Node: Internationalization Terminology,  Next: Charsets,  Up: MULE

63.1 Internationalization Terminology
=====================================

In internationalization terminology, a string of text is divided up
into "characters", which are the printable units that make up the text.
A single character is (for example) a capital `A', the number `2', a
Katakana character, a Hangul character, a Kanji ideograph (an
"ideograph" is a "picture" character, such as is used in Japanese
Kanji, Chinese Hanzi, and Korean Hanja; typically there are thousands
of such ideographs in each language), etc.  The basic property of a
character is that it is the smallest unit of text with semantic
significance in text processing--i.e., characters are abstract units
defined by their meaning, not by their exact appearance.

   Human beings normally process text visually, so to a first
approximation a character may be identified with its shape.  Note that
the same character may be drawn by two different people (or in two
different fonts) in slightly different ways, although the "basic shape"
will be the same.  But consider the works of Scott Kim; human beings
can recognize hugely variant shapes as the "same" character.
Sometimes, especially where characters are extremely complicated to
write, completely different shapes may be defined as the "same"
character in national standards.  The Taiwanese variant of Hanzi is
generally the most complicated; over the centuries, the Japanese,
Koreans, and the People's Republic of China have adopted
simplifications of the shape, but the line of descent from the original
shape is recorded, and the meanings and pronunciation of different
forms of the same character are considered to be identical within each
language.  (Of course, it may take a specialist to recognize the
related form; the point is that the relations are standardized, despite
the differing shapes.)

   In some cases, the differences will be significant enough that it is
actually possible to identify two or more distinct shapes that both
represent the same character.  For example, the lowercase letters `a'
and `g' each have two distinct possible shapes--the `a' can optionally
have a curved tail projecting off the top, and the `g' can be formed
either of two loops, or of one loop and a tail hanging off the bottom.
Such distinct possible shapes of a character are called "glyphs".  The
important characteristic of two glyphs making up the same character is
that the choice between one or the other is purely stylistic and has no
linguistic effect on a word (this is the reason why a capital `A' and
lowercase `a' are different characters rather than different
glyphs--e.g.  `Aspen' is a city while `aspen' is a kind of tree).

   Note that "character" and "glyph" are used differently here than
elsewhere in XEmacs.

   A "character set" is essentially a set of related characters.  ASCII,
for example, is a set of 94 characters (or 128, if you count
non-printing characters).  Other character sets are ISO8859-1 (ASCII
plus various accented characters and other international symbols), JIS
X 0201 (ASCII, more or less, plus half-width Katakana), JIS X 0208
(Japanese Kanji), JIS X 0212 (a second set of less-used Japanese Kanji),
GB2312 (Mainland Chinese Hanzi), etc.

   The definition of a character set will implicitly or explicitly give
it an "ordering", a way of assigning a number to each character in the
set.  For many character sets, there is a natural ordering, for example
the "ABC" ordering of the Roman letters.  But it is not clear whether
digits should come before or after the letters, and in fact different
European languages treat the ordering of accented characters
differently.  It is useful to use the natural order where available, of
course.  The number assigned to any particular character is called the
character's "code point".  (Within a given character set, each
character has a unique code point.  Thus the word "set" is ill-chosen;
different orderings of the same characters are different character sets.
Identifying characters is simple enough for alphabetic character sets,
but the difference in ordering can cause great headaches when the same
thousands of characters are used by different cultures as in the Hanzi.)

   It's important to understand that a character is defined not by any
number attached to it, but by its meaning.  For example, ASCII and
EBCDIC are two charsets containing exactly the same characters
(lowercase and uppercase letters, numbers 0 through 9, particular
punctuation marks) but with different numberings.  The `comma'
character in ASCII and EBCDIC, for instance, is the same character
despite having a different numbering.  Conversely, when comparing ASCII
and JIS-Roman, which look the same except that the latter has a yen sign
substituted for the backslash, we would say that the backslash and yen
sign are _not_ the same characters, despite having the same number (95)
and despite the fact that all other characters are present in both
charsets, with the same numbering.  ASCII and JIS-Roman, then, do _not_
have exactly the same characters in them (ASCII has a backslash
character but no yen-sign character, and vice-versa for JIS-Roman),
unlike ASCII and EBCDIC, even though the numberings in ASCII and
JIS-Roman are closer.

   Sometimes, a code point is not a single number, but instead a group
of numbers, called "position codes".  In such cases, the number of
position codes required to index a particular character in a character
set is called the "dimension" of the character set.  Character sets
indexed by more than one position code typically use byte-sized position
codes.  Small character sets, e.g. ASCII, invariably use a single
position code, but for larger character sets, the choice of whether to
use multiple position codes or a single large (16-bit or 32-bit) number
is arbitrary.  Unicode typically uses a single large number, but
language-specific or "national" character sets often use multiple
(usually two) position codes.  For example, JIS X 0208, i.e. Japanese
Kanji, has thousands of characters, and is of dimension two - every
character is indexed by two position codes, each in the range 1 through
94.  (This number "94" is not a coincidence; it is the same as the
number of printable characters in ASCII, and was chosen so that JIS
characters could be directly encoded using two printable ASCII
characters.)  Note that the choice of the range here is somewhat
arbitrary - it could just as easily be 0 through 93, 2 through 95, etc.
In fact, the range for JIS position codes (and for other character sets
modeled after it) is often given as range 33 through 126, so as to
directly match ASCII printing characters.

   An "encoding" is a way of numerically representing characters from
one or more character sets into a stream of like-sized numerical values
called "words" - typically 8-bit bytes, but sometimes 16-bit or 32-bit
quantities.  It's very important to clearly distinguish between
charsets and encodings.  For a simple charset like ASCII, there is only
one encoding normally used - each character is represented by a single
byte, with the same value as its code point.  For more complicated
charsets, however, or when a single encoding needs to represent more
than charset, things are not so obvious.  Unicode version 2, for
example, is a large charset with thousands of characters, each indexed
by a 16-bit number, often represented in hex, e.g. 0x05D0 for the Hebrew
letter "aleph".  One obvious encoding (actually two encodings, depending
on which of the two possible byte orderings is chosen) simply uses two
bytes per character.  This encoding is convenient for internal
processing of Unicode text; however, it's incompatible with ASCII, and
thus external text (files, e-mail, etc.) that is encoded this way is
completely uninterpretable by programs lacking Unicode support.  For
this reason, a different, ASCII-compatible encoding, e.g. UTF-8, is
usually used for external text.  UTF-8 represents Unicode characters
with one to three bytes (often extended to six bytes to handle
characters with up to 31-bit indices).  Unicode characters 00 to 7F
(identical with ASCII) are directly represented with one byte, and other
characters with two or more bytes, each in the range 80 to FF.
Applications that don't understand Unicode will still be able to process
ASCII characters represented in UTF-8-encoded text, and will typically
ignore (and hopefully preserve) the high-bit characters.

   Naive use of code points is also not possible if more than one
character set is to be used in the encoding.  For example, printed
Japanese text typically requires characters from multiple character sets
- ASCII, JIS X 0208, and JIS X 0212, to be specific.  Each of these is
indexed using one or more position codes in the range 1 through 94 (or
33 through 126), so the position codes could not be used directly or
there would be no way to tell which character was meant.  Different
Japanese encodings handle this differently - JIS uses special escape
characters to denote different character sets; EUC sets the high bit of
the position codes for JIS X 0208 and JIS X 0212, and puts a special
extra byte before each JIS X 0212 character; etc.

   The encodings described above are all 7-bit or 8-bit encodings.  The
fixed-width Unicode encoding previous described, however, is sometimes
considered to be a 16-bit encoding, in which case the issue of byte
ordering does not come up. (Imagine, for example, that the text is
represented as an array of shorts.) Similarly, Unicode version 3 (which
has characters with indices above 0xFFFF), and other very large
character sets, may be represented internally as 32-bit encodings, i.e.
arrays of ints.  However, it does not make too much sense to talk about
16-bit or 32-bit encodings for external data, since nowadays 8-bit data
is a universal standard - the closest you can get is fixed-width
encodings using two or four bytes to encode 16-bit or 32-bit values. (A
"7-bit" encoding is used when it cannot be guaranteed that the high bit
of 8-bit data will be correctly preserved.  Some e-mail gateways, for
example, strip the high bit of text passing through them.  These same
gateways often handle non-printable characters incorrectly, and so 7-bit
encodings usually avoid using bytes with such values.)

   A general method of handling text using multiple character sets
(whether for multilingual text, or simply text in an extremely
complicated single language like Japanese) is defined in the
international standard ISO 2022.  ISO 2022 will be discussed in more
detail later (*note ISO 2022::), but for now suffice it to say that text
needs control functions (at least spacing), and if escape sequences are
to be used, an escape sequence introducer.  It was decided to make all
text streams compatible with ASCII in the sense that the codes 0-31
(and 128-159) would always be control codes, never graphic characters,
and where defined by the character set the `SPC' character would be
assigned code 32, and `DEL' would be assigned 127.  Thus there are 94
code points remaining if 7 bits are used.  This is the reason that most
character sets are defined using position codes in the range 1 through
94.  Then ISO 2022 compatible encodings are produced by shifting the
position codes 1 to 94 into character codes 33 to 126, or (if 8 bit
codes are available) into character codes 161 to 254.

   Encodings are classified as either "modal" or "non-modal".  In a
"modal encoding", there are multiple states that the encoding can be
in, and the interpretation of the values in the stream depends on the
current global state of the encoding.  Special values in the encoding,
called "escape sequences", are used to change the global state.  JIS,
for example, is a modal encoding.  The bytes `ESC $ B' indicate that,
from then on, bytes are to be interpreted as position codes for JIS X
0208, rather than as ASCII.  This effect is cancelled using the bytes
`ESC ( B', which mean "switch from whatever the current state is to
ASCII".  To switch to JIS X 0212, the escape sequence `ESC $ ( D'.
(Note that here, as is common, the escape sequences do in fact begin
with `ESC'.  This is not necessarily the case, however.  Some encodings
use control characters called "locking shifts" (effect persists until
cancelled) to switch character sets.)

   A "non-modal encoding" has no global state that extends past the
character currently being interpreted.  EUC, for example, is a
non-modal encoding.  Characters in JIS X 0208 are encoded by setting
the high bit of the position codes, and characters in JIS X 0212 are
encoded by doing the same but also prefixing the character with the
byte 0x8F.

   The advantage of a modal encoding is that it is generally more
space-efficient, and is easily extendible because there are essentially
an arbitrary number of escape sequences that can be created.  The
disadvantage, however, is that it is much more difficult to work with
if it is not being processed in a sequential manner.  In the non-modal
EUC encoding, for example, the byte 0x41 always refers to the letter
`A'; whereas in JIS, it could either be the letter `A', or one of the
two position codes in a JIS X 0208 character, or one of the two
position codes in a JIS X 0212 character.  Determining exactly which
one is meant could be difficult and time-consuming if the previous
bytes in the string have not already been processed, or impossible if
they are drawn from an external stream that cannot be rewound.

   Non-modal encodings are further divided into "fixed-width" and
"variable-width" formats.  A fixed-width encoding always uses the same
number of words per character, whereas a variable-width encoding does
not.  EUC is a good example of a variable-width encoding: one to three
bytes are used per character, depending on the character set.  16-bit
and 32-bit encodings are nearly always fixed-width, and this is in fact
one of the main reasons for using an encoding with a larger word size.
The advantages of fixed-width encodings should be obvious.  The
advantages of variable-width encodings are that they are generally more
space-efficient and allow for compatibility with existing 8-bit
encodings such as ASCII.  (For example, in Unicode ASCII characters are
simply promoted to a 16-bit representation.  That means that every
ASCII character contains a `NUL' byte; evidently all of the standard
string manipulation functions will lose badly in a fixed-width Unicode
environment.)

   The bytes in an 8-bit encoding are often referred to as "octets"
rather than simply as bytes.  This terminology dates back to the days
before 8-bit bytes were universal, when some computers had 9-bit bytes,
others had 10-bit bytes, etc.


File: lispref.info,  Node: Charsets,  Next: MULE Characters,  Prev: Internationalization Terminology,  Up: MULE

63.2 Charsets
=============

A "charset" in MULE is an object that encapsulates a particular
character set as well as an ordering of those characters.  Charsets are
permanent objects and are named using symbols, like faces.

 -- Function: charsetp object
     This function returns non-`nil' if OBJECT is a charset.

* Menu:

* Charset Properties::          Properties of a charset.
* Basic Charset Functions::     Functions for working with charsets.
* Charset Property Functions::  Functions for accessing charset properties.
* Predefined Charsets::         Predefined charset objects.


File: lispref.info,  Node: Charset Properties,  Next: Basic Charset Functions,  Up: Charsets

63.2.1 Charset Properties
-------------------------

Charsets have the following properties:

`name'
     A symbol naming the charset.  Every charset must have a different
     name; this allows a charset to be referred to using its name
     rather than the actual charset object.

`doc-string'
     A documentation string describing the charset.

`registry'
     A regular expression matching the font registry field for this
     character set.  For example, both the `ascii' and `latin-iso8859-1'
     charsets use the registry `"ISO8859-1"'.  This field is used to
     choose an appropriate font when the user gives a general font
     specification such as `-*-courier-medium-r-*-140-*', i.e. a
     14-point upright medium-weight Courier font.

`dimension'
     Number of position codes used to index a character in the
     character set.  XEmacs/MULE can only handle character sets of
     dimension 1 or 2.  This property defaults to 1.

`chars'
     Number of characters in each dimension.  In XEmacs/MULE, the only
     allowed values are 94 or 96. (There are a couple of pre-defined
     character sets, such as ASCII, that do not follow this, but you
     cannot define new ones like this.) Defaults to 94.  Note that if
     the dimension is 2, the character set thus described is 94x94 or
     96x96.

`columns'
     Number of columns used to display a character in this charset.
     Only used in TTY mode. (Under X, the actual width of a character
     can be derived from the font used to display the characters.)  If
     unspecified, defaults to the dimension. (This is almost always the
     correct value, because character sets with dimension 2 are usually
     ideograph character sets, which need two columns to display the
     intricate ideographs.)

`direction'
     A symbol, either `l2r' (left-to-right) or `r2l' (right-to-left).
     Defaults to `l2r'.  This specifies the direction that the text
     should be displayed in, and will be left-to-right for most
     charsets but right-to-left for Hebrew and Arabic. (Right-to-left
     display is not currently implemented.)

`final'
     Final byte of the standard ISO 2022 escape sequence designating
     this charset.  Must be supplied.  Each combination of (DIMENSION,
     CHARS) defines a separate namespace for final bytes, and each
     charset within a particular namespace must have a different final
     byte.  Note that ISO 2022 restricts the final byte to the range
     0x30 - 0x7E if dimension == 1, and 0x30 - 0x5F if dimension == 2.
     Note also that final bytes in the range 0x30 - 0x3F are reserved
     for user-defined (not official) character sets.  For more
     information on ISO 2022, see *Note Coding Systems::.

`graphic'
     0 (use left half of font on output) or 1 (use right half of font on
     output).  Defaults to 0.  This specifies how to convert the
     position codes that index a character in a character set into an
     index into the font used to display the character set.  With
     `graphic' set to 0, position codes 33 through 126 map to font
     indices 33 through 126; with it set to 1, position codes 33
     through 126 map to font indices 161 through 254 (i.e. the same
     number but with the high bit set).  For example, for a font whose
     registry is ISO8859-1, the left half of the font (octets 0x20 -
     0x7F) is the `ascii' charset, while the right half (octets 0xA0 -
     0xFF) is the `latin-iso8859-1' charset.

`ccl-program'
     A compiled CCL program used to convert a character in this charset
     into an index into the font.  This is in addition to the `graphic'
     property.  If a CCL program is defined, the position codes of a
     character will first be processed according to `graphic' and then
     passed through the CCL program, with the resulting values used to
     index the font.

     This is used, for example, in the Big5 character set (used in
     Taiwan).  This character set is not ISO-2022-compliant, and its
     size (94x157) does not fit within the maximum 96x96 size of
     ISO-2022-compliant character sets.  As a result, XEmacs/MULE
     splits it (in a rather complex fashion, so as to group the most
     commonly used characters together) into two charset objects
     (`big5-1' and `big5-2'), each of size 94x94, and each charset
     object uses a CCL program to convert the modified position codes
     back into standard Big5 indices to retrieve a character from a
     Big5 font.

   Most of the above properties can only be set when the charset is
initialized, and cannot be changed later.  *Note Charset Property
Functions::.


File: lispref.info,  Node: Basic Charset Functions,  Next: Charset Property Functions,  Prev: Charset Properties,  Up: Charsets

63.2.2 Basic Charset Functions
------------------------------

 -- Function: find-charset charset-or-name
     This function retrieves the charset of the given name.  If
     CHARSET-OR-NAME is a charset object, it is simply returned.
     Otherwise, CHARSET-OR-NAME should be a symbol.  If there is no
     such charset, `nil' is returned.  Otherwise the associated charset
     object is returned.

 -- Function: get-charset name
     This function retrieves the charset of the given name.  Same as
     `find-charset' except an error is signalled if there is no such
     charset instead of returning `nil'.

 -- Function: charset-list
     This function returns a list of the names of all defined charsets.

 -- Function: make-charset name doc-string props
     This function defines a new character set.  This function is for
     use with MULE support.  NAME is a symbol, the name by which the
     character set is normally referred.  DOC-STRING is a string
     describing the character set.  PROPS is a property list,
     describing the specific nature of the character set.  The
     recognized properties are `registry', `dimension', `columns',
     `chars', `final', `graphic', `direction', and `ccl-program', as
     previously described.

 -- Function: make-reverse-direction-charset charset new-name
     This function makes a charset equivalent to CHARSET but which goes
     in the opposite direction.  NEW-NAME is the name of the new
     charset.  The new charset is returned.

 -- Function: charset-from-attributes dimension chars final &optional
          direction
     This function returns a charset with the given DIMENSION, CHARS,
     FINAL, and DIRECTION.  If DIRECTION is omitted, both directions
     will be checked (left-to-right will be returned if character sets
     exist for both directions).

 -- Function: charset-reverse-direction-charset charset
     This function returns the charset (if any) with the same dimension,
     number of characters, and final byte as CHARSET, but which is
     displayed in the opposite direction.


File: lispref.info,  Node: Charset Property Functions,  Next: Predefined Charsets,  Prev: Basic Charset Functions,  Up: Charsets

63.2.3 Charset Property Functions
---------------------------------

All of these functions accept either a charset name or charset object.

 -- Function: charset-property charset prop
     This function returns property PROP of CHARSET.  *Note Charset
     Properties::.

   Convenience functions are also provided for retrieving individual
properties of a charset.

 -- Function: charset-name charset
     This function returns the name of CHARSET.  This will be a symbol.

 -- Function: charset-description charset
     This function returns the documentation string of CHARSET.

 -- Function: charset-registry charset
     This function returns the registry of CHARSET.

 -- Function: charset-dimension charset
     This function returns the dimension of CHARSET.

 -- Function: charset-chars charset
     This function returns the number of characters per dimension of
     CHARSET.

 -- Function: charset-width charset
     This function returns the number of display columns per character
     (in TTY mode) of CHARSET.

 -- Function: charset-direction charset
     This function returns the display direction of CHARSET--either
     `l2r' or `r2l'.

 -- Function: charset-iso-final-char charset
     This function returns the final byte of the ISO 2022 escape
     sequence designating CHARSET.

 -- Function: charset-iso-graphic-plane charset
     This function returns either 0 or 1, depending on whether the
     position codes of characters in CHARSET map to the left or right
     half of their font, respectively.

 -- Function: charset-ccl-program charset
     This function returns the CCL program, if any, for converting
     position codes of characters in CHARSET into font indices.

   The two properties of a charset that can currently be set after the
charset has been created are the CCL program and the font registry.

 -- Function: set-charset-ccl-program charset ccl-program
     This function sets the `ccl-program' property of CHARSET to
     CCL-PROGRAM.

 -- Function: set-charset-registry charset registry
     This function sets the `registry' property of CHARSET to REGISTRY.


File: lispref.info,  Node: Predefined Charsets,  Prev: Charset Property Functions,  Up: Charsets

63.2.4 Predefined Charsets
--------------------------

The following charsets are predefined in the C code.

     Name                    Type  Fi Gr Dir Registry
     --------------------------------------------------------------
     ascii                    94    B  0  l2r ISO8859-1
     control-1                94       0  l2r ---
     latin-iso8859-1          94    A  1  l2r ISO8859-1
     latin-iso8859-2          96    B  1  l2r ISO8859-2
     latin-iso8859-3          96    C  1  l2r ISO8859-3
     latin-iso8859-4          96    D  1  l2r ISO8859-4
     cyrillic-iso8859-5       96    L  1  l2r ISO8859-5
     arabic-iso8859-6         96    G  1  r2l ISO8859-6
     greek-iso8859-7          96    F  1  l2r ISO8859-7
     hebrew-iso8859-8         96    H  1  r2l ISO8859-8
     latin-iso8859-9          96    M  1  l2r ISO8859-9
     thai-tis620              96    T  1  l2r TIS620
     katakana-jisx0201        94    I  1  l2r JISX0201.1976
     latin-jisx0201           94    J  0  l2r JISX0201.1976
     japanese-jisx0208-1978   94x94 @  0  l2r JISX0208.1978
     japanese-jisx0208        94x94 B  0  l2r JISX0208.19(83|90)
     japanese-jisx0212        94x94 D  0  l2r JISX0212
     chinese-gb2312           94x94 A  0  l2r GB2312
     chinese-cns11643-1       94x94 G  0  l2r CNS11643.1
     chinese-cns11643-2       94x94 H  0  l2r CNS11643.2
     chinese-big5-1           94x94 0  0  l2r Big5
     chinese-big5-2           94x94 1  0  l2r Big5
     korean-ksc5601           94x94 C  0  l2r KSC5601
     composite                96x96    0  l2r ---

   The following charsets are predefined in the Lisp code.

     Name                     Type  Fi Gr Dir Registry
     --------------------------------------------------------------
     arabic-digit             94    2  0  l2r MuleArabic-0
     arabic-1-column          94    3  0  r2l MuleArabic-1
     arabic-2-column          94    4  0  r2l MuleArabic-2
     sisheng                  94    0  0  l2r sisheng_cwnn\|OMRON_UDC_ZH
     chinese-cns11643-3       94x94 I  0  l2r CNS11643.1
     chinese-cns11643-4       94x94 J  0  l2r CNS11643.1
     chinese-cns11643-5       94x94 K  0  l2r CNS11643.1
     chinese-cns11643-6       94x94 L  0  l2r CNS11643.1
     chinese-cns11643-7       94x94 M  0  l2r CNS11643.1
     ethiopic                 94x94 2  0  l2r Ethio
     ascii-r2l                94    B  0  r2l ISO8859-1
     ipa                      96    0  1  l2r MuleIPA
     vietnamese-viscii-lower  96    1  1  l2r VISCII1.1
     vietnamese-viscii-upper  96    2  1  l2r VISCII1.1

   For all of the above charsets, the dimension and number of columns
are the same.

   Note that ASCII, Control-1, and Composite are handled specially.
This is why some of the fields are blank; and some of the filled-in
fields (e.g. the type) are not really accurate.


File: lispref.info,  Node: MULE Characters,  Next: Composite Characters,  Prev: Charsets,  Up: MULE

63.3 MULE Characters
====================

 -- Function: make-char charset arg1 &optional arg2
     This function makes a multi-byte character from CHARSET and octets
     ARG1 and ARG2.

 -- Function: char-charset character
     This function returns the character set of char CHARACTER.

 -- Function: char-octet character &optional n
     This function returns the octet (i.e. position code) numbered N
     (should be 0 or 1) of char CHARACTER.  N defaults to 0 if omitted.

 -- Function: find-charset-region start end &optional buffer
     This function returns a list of the charsets in the region between
     START and END.  BUFFER defaults to the current buffer if omitted.

 -- Function: find-charset-string string
     This function returns a list of the charsets in STRING.


File: lispref.info,  Node: Composite Characters,  Next: Coding Systems,  Prev: MULE Characters,  Up: MULE

63.4 Composite Characters
=========================

Composite characters are not yet completely implemented.

 -- Function: make-composite-char string
     This function converts a string into a single composite character.
     The character is the result of overstriking all the characters in
     the string.

 -- Function: composite-char-string character
     This function returns a string of the characters comprising a
     composite character.

 -- Function: compose-region start end &optional buffer
     This function composes the characters in the region from START to
     END in BUFFER into one composite character.  The composite
     character replaces the composed characters.  BUFFER defaults to
     the current buffer if omitted.

 -- Function: decompose-region start end &optional buffer
     This function decomposes any composite characters in the region
     from START to END in BUFFER.  This converts each composite
     character into one or more characters, the individual characters
     out of which the composite character was formed.  Non-composite
     characters are left as-is.  BUFFER defaults to the current buffer
     if omitted.


File: lispref.info,  Node: Coding Systems,  Next: CCL,  Prev: Composite Characters,  Up: MULE

63.5 Coding Systems
===================

A coding system is an object that defines how text containing multiple
character sets is encoded into a stream of (typically 8-bit) bytes.  The
coding system is used to decode the stream into a series of characters
(which may be from multiple charsets) when the text is read from a file
or process, and is used to encode the text back into the same format
when it is written out to a file or process.

   For example, many ISO-2022-compliant coding systems (such as Compound
Text, which is used for inter-client data under the X Window System) use
escape sequences to switch between different charsets - Japanese Kanji,
for example, is invoked with `ESC $ ( B'; ASCII is invoked with `ESC (
B'; and Cyrillic is invoked with `ESC - L'.  See `make-coding-system'
for more information.

   Coding systems are normally identified using a symbol, and the
symbol is accepted in place of the actual coding system object whenever
a coding system is called for. (This is similar to how faces and
charsets work.)

 -- Function: coding-system-p object
     This function returns non-`nil' if OBJECT is a coding system.

* Menu:

* Coding System Types::               Classifying coding systems.
* ISO 2022::                          An international standard for
                                        charsets and encodings.
* EOL Conversion::                    Dealing with different ways of denoting
                                        the end of a line.
* Coding System Properties::          Properties of a coding system.
* Basic Coding System Functions::     Working with coding systems.
* Coding System Property Functions::  Retrieving a coding system's properties.
* Encoding and Decoding Text::        Encoding and decoding text.
* Detection of Textual Encoding::     Determining how text is encoded.
* Big5 and Shift-JIS Functions::      Special functions for these non-standard
                                        encodings.
* Predefined Coding Systems::         Coding systems implemented by MULE.


File: lispref.info,  Node: Coding System Types,  Next: ISO 2022,  Up: Coding Systems

63.5.1 Coding System Types
--------------------------

The coding system type determines the basic algorithm XEmacs will use to
decode or encode a data stream.  Character encodings will be converted
to the MULE encoding, escape sequences processed, and newline sequences
converted to XEmacs's internal representation.  There are three basic
classes of coding system type: no-conversion, ISO-2022, and special.

   No conversion allows you to look at the file's internal
representation.  Since XEmacs is basically a text editor, "no
conversion" does convert newline conventions by default.  (Use the
'binary coding-system if this is not desired.)

   ISO 2022 (*note ISO 2022::) is the basic international standard
regulating use of "coded character sets for the exchange of data", ie,
text streams.  ISO 2022 contains functions that make it possible to
encode text streams to comply with restrictions of the Internet mail
system and de facto restrictions of most file systems (eg, use of the
separator character in file names).  Coding systems which are not ISO
2022 conformant can be difficult to handle.  Perhaps more important,
they are not adaptable to multilingual information interchange, with
the obvious exception of ISO 10646 (Unicode).  (Unicode is partially
supported by XEmacs with the addition of the Lisp package ucs-conv.)

   The special class of coding systems includes automatic detection,
CCL (a "little language" embedded as an interpreter, useful for
translating between variants of a single character set),
non-ISO-2022-conformant encodings like Unicode, Shift JIS, and Big5,
and MULE internal coding.  (NB: this list is based on XEmacs 21.2.
Terminology may vary slightly for other versions of XEmacs and for GNU
Emacs 20.)

`no-conversion'
     No conversion, for binary files, and a few special cases of
     non-ISO-2022 coding systems where conversion is done by hook
     functions (usually implemented in CCL).  On output, graphic
     characters that are not in ASCII or Latin-1 will be replaced by a
     `?'. (For a no-conversion-encoded buffer, these characters will
     only be present if you explicitly insert them.)

`iso2022'
     Any ISO-2022-compliant encoding.  Among others, this includes JIS
     (the Japanese encoding commonly used for e-mail), national
     variants of EUC (the standard Unix encoding for Japanese and other
     languages), and Compound Text (an encoding used in X11).  You can
     specify more specific information about the conversion with the
     FLAGS argument.

`ucs-4'
     ISO 10646 UCS-4 encoding.  A 31-bit fixed-width superset of
     Unicode.

`utf-8'
     ISO 10646 UTF-8 encoding.  A "file system safe" transformation
     format that can be used with both UCS-4 and Unicode.

`undecided'
     Automatic conversion.  XEmacs attempts to detect the coding system
     used in the file.

`shift-jis'
     Shift-JIS (a Japanese encoding commonly used in PC operating
     systems).

`big5'
     Big5 (the encoding commonly used for Taiwanese).

`ccl'
     The conversion is performed using a user-written pseudo-code
     program.  CCL (Code Conversion Language) is the name of this
     pseudo-code.  For example, CCL is used to map KOI8-R characters
     (an encoding for Russian Cyrillic) to ISO8859-5 (the form used
     internally by MULE).

`internal'
     Write out or read in the raw contents of the memory representing
     the buffer's text.  This is primarily useful for debugging
     purposes, and is only enabled when XEmacs has been compiled with
     `DEBUG_XEMACS' set (the `--debug' configure option).  *Warning*:
     Reading in a file using `internal' conversion can result in an
     internal inconsistency in the memory representing a buffer's text,
     which will produce unpredictable results and may cause XEmacs to
     crash.  Under normal circumstances you should never use `internal'
     conversion.


File: lispref.info,  Node: ISO 2022,  Next: EOL Conversion,  Prev: Coding System Types,  Up: Coding Systems

63.6 ISO 2022
=============

This section briefly describes the ISO 2022 encoding standard.  A more
thorough treatment is available in the original document of ISO 2022 as
well as various national standards (such as JIS X 0202).

   Character sets ("charsets") are classified into the following four
categories, according to the number of characters in the charset:
94-charset, 96-charset, 94x94-charset, and 96x96-charset.  This means
that although an ISO 2022 coding system may have variable width
characters, each charset used is fixed-width (in contrast to the MULE
character set and UTF-8, for example).

   ISO 2022 provides for switching between character sets via escape
sequences.  This switching is somewhat complicated, because ISO 2022
provides for both legacy applications like Internet mail that accept
only 7 significant bits in some contexts (RFC 822 headers, for example),
and more modern "8-bit clean" applications.  It also provides for
compact and transparent representation of languages like Japanese which
mix ASCII and a national script (even outside of computer programs).

   First, ISO 2022 codified prevailing practice by dividing the code
space into "control" and "graphic" regions.  The code points 0x00-0x1F
and 0x80-0x9F are reserved for "control characters", while "graphic
characters" must be assigned to code points in the regions 0x20-0x7F and
0xA0-0xFF.  The positions 0x20 and 0x7F are special, and under some
circumstances must be assigned the graphic character "ASCII SPACE" and
the control character "ASCII DEL" respectively.

   The various regions are given the name C0 (0x00-0x1F), GL
(0x20-0x7F), C1 (0x80-0x9F), and GR (0xA0-0xFF).  GL and GR stand for
"graphic left" and "graphic right", respectively, because of the
standard method of displaying graphic character sets in tables with the
high byte indexing columns and the low byte indexing rows.  I don't
find it very intuitive, but these are called "registers".

   An ISO 2022-conformant encoding for a graphic character set must use
a fixed number of bytes per character, and the values must fit into a
single register; that is, each byte must range over either 0x20-0x7F, or
0xA0-0xFF.  It is not allowed to extend the range of the repertoire of a
character set by using both ranges at the same.  This is why a standard
character set such as ISO 8859-1 is actually considered by ISO 2022 to
be an aggregation of two character sets, ASCII and LATIN-1, and why it
is technically incorrect to refer to ISO 8859-1 as "Latin 1".  Also, a
single character's bytes must all be drawn from the same register; this
is why Shift JIS (for Japanese) and Big 5 (for Chinese) are not ISO
2022-compatible encodings.

   The reason for this restriction becomes clear when you attempt to
define an efficient, robust encoding for a language like Japanese.
Like ISO 8859, Japanese encodings are aggregations of several character
sets.  In practice, the vast majority of characters are drawn from the
"JIS Roman" character set (a derivative of ASCII; it won't hurt to
think of it as ASCII) and the JIS X 0208 standard "basic Japanese"
character set including not only ideographic characters ("kanji") but
syllabic Japanese characters ("kana"), a wide variety of symbols, and
many alphabetic characters (Roman, Greek, and Cyrillic) as well.
Although JIS X 0208 includes the whole Roman alphabet, as a 2-byte code
it is not suited to programming; thus the inclusion of ASCII in the
standard Japanese encodings.

   For normal Japanese text such as in newspapers, a broad repertoire of
approximately 3000 characters is used.  Evidently this won't fit into
one byte; two must be used.  But much of the text processed by Japanese
computers is computer source code, nearly all of which is ASCII.  A not
insignificant portion of ordinary text is English (as such or as
borrowed Japanese vocabulary) or other languages which can represented
at least approximately in ASCII, as well.  It seems reasonable then to
represent ASCII in one byte, and JIS X 0208 in two.  And this is exactly
what the Extended Unix Code for Japanese (EUC-JP) does.  ASCII is
invoked to the GL register, and JIS X 0208 is invoked to the GR
register.  Thus, each byte can be tested for its character set by
looking at the high bit; if set, it is Japanese, if clear, it is ASCII.
Furthermore, since control characters like newline can never be part of
a graphic character, even in the case of corruption in transmission the
stream will be resynchronized at every line break, on the order of 60-80
bytes.  This coding system requires no escape sequences or special
control codes to represent 99.9% of all Japanese text.

   Note carefully the distinction between the character sets (ASCII and
JIS X 0208), the encoding (EUC-JP), and the coding system (ISO 2022).
The JIS X 0208 character set is used in three different encodings for
Japanese, but in ISO-2022-JP it is invoked into GL (so the high bit is
always clear), in EUC-JP it is invoked into GR (setting the high bit in
the process), and in Shift JIS the high bit may be set or reset, and the
significant bits are shifted within the 16-bit character so that the two
main character sets can coexist with a third (the "halfwidth katakana"
of JIS X 0201).  As the name implies, the ISO-2022-JP encoding is also a
version of the ISO-2022 coding system.

   In order to systematically treat subsidiary character sets (like the
"halfwidth katakana" already mentioned, and the "supplementary kanji" of
JIS X 0212), four further registers are defined: G0, G1, G2, and G3.
Unlike GL and GR, they are not logically distinguished by internal
format.  Instead, the process of "invocation" mentioned earlier is
broken into two steps: first, a character set is "designated" to one of
the registers G0-G3 by use of an "escape sequence" of the form:

             ESC [I] I F

   where I is an intermediate character or characters in the range 0x20
- 0x3F, and F, from the range 0x30-0x7Fm is the final character
identifying this charset.  (Final characters in the range 0x30-0x3F are
reserved for private use and will never have a publicly registered
meaning.)

   Then that register is "invoked" to either GL or GR, either
automatically (designations to G0 normally involve invocation to GL as
well), or by use of shifting (affecting only the following character in
the data stream) or locking (effective until the next designation or
locking) control sequences.  An encoding conformant to ISO 2022 is
typically defined by designating the initial contents of the G0-G3
registers, specifying a 7 or 8 bit environment, and specifying whether
further designations will be recognized.

   Some examples of character sets and the registered final characters
F used to designate them:

94-charset
     ASCII (B), left (J) and right (I) half of JIS X 0201, ...

96-charset
     Latin-1 (A), Latin-2 (B), Latin-3 (C), ...

94x94-charset
     GB2312 (A), JIS X 0208 (B), KSC5601 (C), ...

96x96-charset
     none for the moment

   The meanings of the various characters in these sequences, where not
specified by the ISO 2022 standard (such as the ESC character), are
assigned by "ECMA", the European Computer Manufacturers Association.

   The meaning of intermediate characters are:

             $ [0x24]: indicate charset of dimension 2 (94x94 or 96x96).
             ( [0x28]: designate to G0 a 94-charset whose final byte is F.
             ) [0x29]: designate to G1 a 94-charset whose final byte is F.
             * [0x2A]: designate to G2 a 94-charset whose final byte is F.
             + [0x2B]: designate to G3 a 94-charset whose final byte is F.
             , [0x2C]: designate to G0 a 96-charset whose final byte is F.
             - [0x2D]: designate to G1 a 96-charset whose final byte is F.
             . [0x2E]: designate to G2 a 96-charset whose final byte is F.
             / [0x2F]: designate to G3 a 96-charset whose final byte is F.

   The comma may be used in files read and written only by MULE, as a
MULE extension, but this is illegal in ISO 2022.  (The reason is that
in ISO 2022 G0 must be a 94-member character set, with 0x20 assigned
the value SPACE, and 0x7F assigned the value DEL.)

   Here are examples of designations:

             ESC ( B :              designate to G0 ASCII
             ESC - A :              designate to G1 Latin-1
             ESC $ ( A or ESC $ A : designate to G0 GB2312
             ESC $ ( B or ESC $ B : designate to G0 JISX0208
             ESC $ ) C :            designate to G1 KSC5601

   (The short forms used to designate GB2312 and JIS X 0208 are for
backwards compatibility; the long forms are preferred.)

   To use a charset designated to G2 or G3, and to use a charset
designated to G1 in a 7-bit environment, you must explicitly invoke G1,
G2, or G3 into GL.  There are two types of invocation, Locking Shift
(forever) and Single Shift (one character only).

   Locking Shift is done as follows:

             LS0 or SI (0x0F): invoke G0 into GL
             LS1 or SO (0x0E): invoke G1 into GL
             LS2:  invoke G2 into GL
             LS3:  invoke G3 into GL
             LS1R: invoke G1 into GR
             LS2R: invoke G2 into GR
             LS3R: invoke G3 into GR

   Single Shift is done as follows:

             SS2 or ESC N: invoke G2 into GL
             SS3 or ESC O: invoke G3 into GL

   The shift functions (such as LS1R and SS3) are represented by control
characters (from C1) in 8 bit environments and by escape sequences in 7
bit environments.

   (#### Ben says: I think the above is slightly incorrect.  It appears
that SS2 invokes G2 into GR and SS3 invokes G3 into GR, whereas ESC N
and ESC O behave as indicated.  The above definitions will not parse
EUC-encoded text correctly, and it looks like the code in mule-coding.c
has similar problems.)

   Evidently there are a lot of ISO-2022-compliant ways of encoding
multilingual text.  Now, in the world, there exist many coding systems
such as X11's Compound Text, Japanese JUNET code, and so-called EUC
(Extended UNIX Code); all of these are variants of ISO 2022.

   In MULE, we characterize a version of ISO 2022 by the following
attributes:

  1. The character sets initially designated to G0 thru G3.

  2. Whether short form designations are allowed for Japanese and
     Chinese.

  3. Whether ASCII should be designated to G0 before control characters.

  4. Whether ASCII should be designated to G0 at the end of line.

  5. 7-bit environment or 8-bit environment.

  6. Whether Locking Shifts are used or not.

  7. Whether to use ASCII or the variant JIS X 0201-1976-Roman.

  8. Whether to use JIS X 0208-1983 or the older version JIS X
     0208-1976.

   (The last two are only for Japanese.)

   By specifying these attributes, you can create any variant of ISO
2022.

   Here are several examples:

     ISO-2022-JP -- Coding system used in Japanese email (RFC 1463 #### check).
             1. G0 <- ASCII, G1..3 <- never used
             2. Yes.
             3. Yes.
             4. Yes.
             5. 7-bit environment
             6. No.
             7. Use ASCII
             8. Use JIS X 0208-1983

     ctext -- X11 Compound Text
             1. G0 <- ASCII, G1 <- Latin-1, G2,3 <- never used.
             2. No.
             3. No.
             4. Yes.
             5. 8-bit environment.
             6. No.
             7. Use ASCII.
             8. Use JIS X 0208-1983.

     euc-china -- Chinese EUC.  Often called the "GB encoding", but that is
     technically incorrect.
             1. G0 <- ASCII, G1 <- GB 2312, G2,3 <- never used.
             2. No.
             3. Yes.
             4. Yes.
             5. 8-bit environment.
             6. No.
             7. Use ASCII.
             8. Use JIS X 0208-1983.

     ISO-2022-KR -- Coding system used in Korean email.
             1. G0 <- ASCII, G1 <- KSC 5601, G2,3 <- never used.
             2. No.
             3. Yes.
             4. Yes.
             5. 7-bit environment.
             6. Yes.
             7. Use ASCII.
             8. Use JIS X 0208-1983.

   MULE creates all of these coding systems by default.


File: lispref.info,  Node: EOL Conversion,  Next: Coding System Properties,  Prev: ISO 2022,  Up: Coding Systems

63.6.1 EOL Conversion
---------------------

`nil'
     Automatically detect the end-of-line type (LF, CRLF, or CR).  Also
     generate subsidiary coding systems named `NAME-unix', `NAME-dos',
     and `NAME-mac', that are identical to this coding system but have
     an EOL-TYPE value of `lf', `crlf', and `cr', respectively.

`lf'
     The end of a line is marked externally using ASCII LF.  Since this
     is also the way that XEmacs represents an end-of-line internally,
     specifying this option results in no end-of-line conversion.  This
     is the standard format for Unix text files.

`crlf'
     The end of a line is marked externally using ASCII CRLF.  This is
     the standard format for MS-DOS text files.

`cr'
     The end of a line is marked externally using ASCII CR.  This is the
     standard format for Macintosh text files.

`t'
     Automatically detect the end-of-line type but do not generate
     subsidiary coding systems.  (This value is converted to `nil' when
     stored internally, and `coding-system-property' will return `nil'.)


File: lispref.info,  Node: Coding System Properties,  Next: Basic Coding System Functions,  Prev: EOL Conversion,  Up: Coding Systems

63.6.2 Coding System Properties
-------------------------------

`mnemonic'
     String to be displayed in the modeline when this coding system is
     active.

`eol-type'
     End-of-line conversion to be used.  It should be one of the types
     listed in *Note EOL Conversion::.

`eol-lf'
     The coding system which is the same as this one, except that it
     uses the Unix line-breaking convention.

`eol-crlf'
     The coding system which is the same as this one, except that it
     uses the DOS line-breaking convention.

`eol-cr'
     The coding system which is the same as this one, except that it
     uses the Macintosh line-breaking convention.

`post-read-conversion'
     Function called after a file has been read in, to perform the
     decoding.  Called with two arguments, START and END, denoting a
     region of the current buffer to be decoded.

`pre-write-conversion'
     Function called before a file is written out, to perform the
     encoding.  Called with two arguments, START and END, denoting a
     region of the current buffer to be encoded.

   The following additional properties are recognized if TYPE is
`iso2022':

`charset-g0'
`charset-g1'
`charset-g2'
`charset-g3'
     The character set initially designated to the G0 - G3 registers.
     The value should be one of

        * A charset object (designate that character set)

        * `nil' (do not ever use this register)

        * `t' (no character set is initially designated to the
          register, but may be later on; this automatically sets the
          corresponding `force-g*-on-output' property)

`force-g0-on-output'
`force-g1-on-output'
`force-g2-on-output'
`force-g3-on-output'
     If non-`nil', send an explicit designation sequence on output
     before using the specified register.

`short'
     If non-`nil', use the short forms `ESC $ @', `ESC $ A', and `ESC $
     B' on output in place of the full designation sequences `ESC $ (
     @', `ESC $ ( A', and `ESC $ ( B'.

`no-ascii-eol'
     If non-`nil', don't designate ASCII to G0 at each end of line on
     output.  Setting this to non-`nil' also suppresses other
     state-resetting that normally happens at the end of a line.

`no-ascii-cntl'
     If non-`nil', don't designate ASCII to G0 before control chars on
     output.

`seven'
     If non-`nil', use 7-bit environment on output.  Otherwise, use
     8-bit environment.

`lock-shift'
     If non-`nil', use locking-shift (SO/SI) instead of single-shift or
     designation by escape sequence.

`no-iso6429'
     If non-`nil', don't use ISO6429's direction specification.

`escape-quoted'
     If non-`nil', literal control characters that are the same as the
     beginning of a recognized ISO 2022 or ISO 6429 escape sequence (in
     particular, ESC (0x1B), SO (0x0E), SI (0x0F), SS2 (0x8E), SS3
     (0x8F), and CSI (0x9B)) are "quoted" with an escape character so
     that they can be properly distinguished from an escape sequence.
     (Note that doing this results in a non-portable encoding.) This
     encoding flag is used for byte-compiled files.  Note that ESC is a
     good choice for a quoting character because there are no escape
     sequences whose second byte is a character from the Control-0 or
     Control-1 character sets; this is explicitly disallowed by the ISO
     2022 standard.

`input-charset-conversion'
     A list of conversion specifications, specifying conversion of
     characters in one charset to another when decoding is performed.
     Each specification is a list of two elements: the source charset,
     and the destination charset.

`output-charset-conversion'
     A list of conversion specifications, specifying conversion of
     characters in one charset to another when encoding is performed.
     The form of each specification is the same as for
     `input-charset-conversion'.

   The following additional properties are recognized (and required) if
TYPE is `ccl':

`decode'
     CCL program used for decoding (converting to internal format).

`encode'
     CCL program used for encoding (converting to external format).

   The following properties are used internally:  EOL-CR, EOL-CRLF,
EOL-LF, and BASE.


File: lispref.info,  Node: Basic Coding System Functions,  Next: Coding System Property Functions,  Prev: Coding System Properties,  Up: Coding Systems

63.6.3 Basic Coding System Functions
------------------------------------

 -- Function: find-coding-system coding-system-or-name
     This function retrieves the coding system of the given name.

     If CODING-SYSTEM-OR-NAME is a coding-system object, it is simply
     returned.  Otherwise, CODING-SYSTEM-OR-NAME should be a symbol.
     If there is no such coding system, `nil' is returned.  Otherwise
     the associated coding system object is returned.

 -- Function: get-coding-system name
     This function retrieves the coding system of the given name.  Same
     as `find-coding-system' except an error is signalled if there is no
     such coding system instead of returning `nil'.

 -- Function: coding-system-list
     This function returns a list of the names of all defined coding
     systems.

 -- Function: coding-system-name coding-system
     This function returns the name of the given coding system.

 -- Function: coding-system-base coding-system
     Returns the base coding system (undecided EOL convention) coding
     system.

 -- Function: make-coding-system name type &optional doc-string props
     This function registers symbol NAME as a coding system.

     TYPE describes the conversion method used and should be one of the
     types listed in *Note Coding System Types::.

     DOC-STRING is a string describing the coding system.

     PROPS is a property list, describing the specific nature of the
     character set.  Recognized properties are as in *Note Coding
     System Properties::.

 -- Function: copy-coding-system old-coding-system new-name
     This function copies OLD-CODING-SYSTEM to NEW-NAME.  If NEW-NAME
     does not name an existing coding system, a new one will be created.

 -- Function: subsidiary-coding-system coding-system eol-type
     This function returns the subsidiary coding system of
     CODING-SYSTEM with eol type EOL-TYPE.


File: lispref.info,  Node: Coding System Property Functions,  Next: Encoding and Decoding Text,  Prev: Basic Coding System Functions,  Up: Coding Systems

63.6.4 Coding System Property Functions
---------------------------------------

 -- Function: coding-system-doc-string coding-system
     This function returns the doc string for CODING-SYSTEM.

 -- Function: coding-system-type coding-system
     This function returns the type of CODING-SYSTEM.

 -- Function: coding-system-property coding-system prop
     This function returns the PROP property of CODING-SYSTEM.


File: lispref.info,  Node: Encoding and Decoding Text,  Next: Detection of Textual Encoding,  Prev: Coding System Property Functions,  Up: Coding Systems

63.6.5 Encoding and Decoding Text
---------------------------------

 -- Function: decode-coding-region start end coding-system &optional
          buffer
     This function decodes the text between START and END which is
     encoded in CODING-SYSTEM.  This is useful if you've read in
     encoded text from a file without decoding it (e.g. you read in a
     JIS-formatted file but used the `binary' or `no-conversion' coding
     system, so that it shows up as `^[$B!<!+^[(B').  The length of the
     encoded text is returned.  BUFFER defaults to the current buffer
     if unspecified.

 -- Function: encode-coding-region start end coding-system &optional
          buffer
     This function encodes the text between START and END using
     CODING-SYSTEM.  This will, for example, convert Japanese
     characters into stuff such as `^[$B!<!+^[(B' if you use the JIS
     encoding.  The length of the encoded text is returned.  BUFFER
     defaults to the current buffer if unspecified.


File: lispref.info,  Node: Detection of Textual Encoding,  Next: Big5 and Shift-JIS Functions,  Prev: Encoding and Decoding Text,  Up: Coding Systems

63.6.6 Detection of Textual Encoding
------------------------------------

 -- Function: coding-category-list
     This function returns a list of all recognized coding categories.

 -- Function: set-coding-priority-list list
     This function changes the priority order of the coding categories.
     LIST should be a list of coding categories, in descending order of
     priority.  Unspecified coding categories will be lower in priority
     than all specified ones, in the same relative order they were in
     previously.

 -- Function: coding-priority-list
     This function returns a list of coding categories in descending
     order of priority.

 -- Function: set-coding-category-system coding-category coding-system
     This function changes the coding system associated with a coding
     category.

 -- Function: coding-category-system coding-category
     This function returns the coding system associated with a coding
     category.

 -- Function: detect-coding-region start end &optional buffer
     This function detects coding system of the text in the region
     between START and END.  Returned value is a list of possible coding
     systems ordered by priority.  If only ASCII characters are found,
     it returns `autodetect' or one of its subsidiary coding systems
     according to a detected end-of-line type.  Optional arg BUFFER
     defaults to the current buffer.


File: lispref.info,  Node: Big5 and Shift-JIS Functions,  Next: Predefined Coding Systems,  Prev: Detection of Textual Encoding,  Up: Coding Systems

63.6.7 Big5 and Shift-JIS Functions
-----------------------------------

These are special functions for working with the non-standard Shift-JIS
and Big5 encodings.

 -- Function: decode-shift-jis-char code
     This function decodes a JIS X 0208 character of Shift-JIS
     coding-system.  CODE is the character code in Shift-JIS as a cons
     of type bytes.  The corresponding character is returned.

 -- Function: encode-shift-jis-char character
     This function encodes a JIS X 0208 character CHARACTER to
     SHIFT-JIS coding-system.  The corresponding character code in
     SHIFT-JIS is returned as a cons of two bytes.

 -- Function: decode-big5-char code
     This function decodes a Big5 character CODE of BIG5 coding-system.
     CODE is the character code in BIG5.  The corresponding character
     is returned.

 -- Function: encode-big5-char character
     This function encodes the Big5 character CHARACTER to BIG5
     coding-system.  The corresponding character code in Big5 is
     returned.


File: lispref.info,  Node: Predefined Coding Systems,  Prev: Big5 and Shift-JIS Functions,  Up: Coding Systems

63.6.8 Coding Systems Implemented
---------------------------------

MULE initializes most of the commonly used coding systems at XEmacs's
startup.  A few others are initialized only when the relevant language
environment is selected and support libraries are loaded.  (NB: The
following list is based on XEmacs 21.2.19, the development branch at the
time of writing.  The list may be somewhat different for other
versions.  Recent versions of GNU Emacs 20 implement a few more rare
coding systems; work is being done to port these to XEmacs.)

   Unfortunately, there is not a consistent naming convention for
character sets, and for practical purposes coding systems often take
their name from their principal character sets (ASCII, KOI8-R, Shift
JIS).  Others take their names from the coding system (ISO-2022-JP,
EUC-KR), and a few from their non-text usages (internal, binary).  To
provide for this, and for the fact that many coding systems have
several common names, an aliasing system is provided.  Finally, some
effort has been made to use names that are registered as MIME charsets
(this is why the name 'shift_jis contains that un-Lisp-y underscore).

   There is a systematic naming convention regarding end-of-line (EOL)
conventions for different systems.  A coding system whose name ends in
"-unix" forces the assumptions that lines are broken by newlines (0x0A).
A coding system whose name ends in "-mac" forces the assumptions that
lines are broken by ASCII CRs (0x0D).  A coding system whose name ends
in "-dos" forces the assumptions that lines are broken by CRLF sequences
(0x0D 0x0A).  These subsidiary coding systems are automatically derived
from a base coding system.  Use of the base coding system implies
autodetection of the text file convention.  (The fact that the -unix,
-mac, and -dos are derived from a base system results in them showing up
as "aliases" in `list-coding-systems'.)  These subsidiaries have a
consistent modeline indicator as well.  "-dos" coding systems have ":T"
appended to their modeline indicator, while "-mac" coding systems have
":t" appended (eg, "ISO8:t" for iso-2022-8-mac).

   In the following table, each coding system is given with its mode
line indicator in parentheses.  Non-textual coding systems are listed
first, followed by textual coding systems and their aliases. (The
coding system subsidiary modeline indicators ":T" and ":t" will be
omitted from the table of coding systems.)

   ### SJT 1999-08-23 Maybe should order these by language?  Definitely
need language usage for the ISO-8859 family.

   Note that although true coding system aliases have been implemented
for XEmacs 21.2, the coding system initialization has not yet been
converted as of 21.2.19.  So coding systems described as aliases have
the same properties as the aliased coding system, but will not be equal
as Lisp objects.

`automatic-conversion'
`undecided'
`undecided-dos'
`undecided-mac'
`undecided-unix'
     Modeline indicator: `Auto'.  A type `undecided' coding system.
     Attempts to determine an appropriate coding system from file
     contents or the environment.

`raw-text'
`no-conversion'
`raw-text-dos'
`raw-text-mac'
`raw-text-unix'
`no-conversion-dos'
`no-conversion-mac'
`no-conversion-unix'
     Modeline indicator: `Raw'.  A type `no-conversion' coding system,
     which converts only line-break-codes.  An implementation quirk
     means that this coding system is also used for ISO8859-1.

`binary'
     Modeline indicator: `Binary'.  A type `no-conversion' coding
     system which does no character coding or EOL conversions.  An
     alias for `raw-text-unix'.

`alternativnyj'
`alternativnyj-dos'
`alternativnyj-mac'
`alternativnyj-unix'
     Modeline indicator: `Cy.Alt'.  A type `ccl' coding system used for
     Alternativnyj, an encoding of the Cyrillic alphabet.

`big5'
`big5-dos'
`big5-mac'
`big5-unix'
     Modeline indicator: `Zh/Big5'.  A type `big5' coding system used
     for BIG5, the most common encoding of traditional Chinese as used
     in Taiwan.

`cn-gb-2312'
`cn-gb-2312-dos'
`cn-gb-2312-mac'
`cn-gb-2312-unix'
     Modeline indicator: `Zh-GB/EUC'.  A type `iso2022' coding system
     used for simplified Chinese (as used in the People's Republic of
     China), with the `ascii' (G0), `chinese-gb2312' (G1), and `sisheng'
     (G2) character sets initially designated.  Chinese EUC (Extended
     Unix Code).

`ctext-hebrew'
`ctext-hebrew-dos'
`ctext-hebrew-mac'
`ctext-hebrew-unix'
     Modeline indicator: `CText/Hbrw'.  A type `iso2022' coding system
     with the `ascii' (G0) and `hebrew-iso8859-8' (G1) character sets
     initially designated for Hebrew.

`ctext'
`ctext-dos'
`ctext-mac'
`ctext-unix'
     Modeline indicator: `CText'.  A type `iso2022' 8-bit coding system
     with the `ascii' (G0) and `latin-iso8859-1' (G1) character sets
     initially designated.  X11 Compound Text Encoding.  Often
     mistakenly recognized instead of EUC encodings; usual cause is
     inappropriate setting of `coding-priority-list'.

`escape-quoted'
     Modeline indicator: `ESC/Quot'.  A type `iso2022' 8-bit coding
     system with the `ascii' (G0) and `latin-iso8859-1' (G1) character
     sets initially designated and escape quoting.  Unix EOL conversion
     (ie, no conversion).  It is used for .ELC files.

`euc-jp'
`euc-jp-dos'
`euc-jp-mac'
`euc-jp-unix'
     Modeline indicator: `Ja/EUC'.  A type `iso2022' 8-bit coding system
     with `ascii' (G0), `japanese-jisx0208' (G1), `katakana-jisx0201'
     (G2), and `japanese-jisx0212' (G3) initially designated.  Japanese
     EUC (Extended Unix Code).

`euc-kr'
`euc-kr-dos'
`euc-kr-mac'
`euc-kr-unix'
     Modeline indicator: `ko/EUC'.  A type `iso2022' 8-bit coding system
     with `ascii' (G0) and `korean-ksc5601' (G1) initially designated.
     Korean EUC (Extended Unix Code).

`hz-gb-2312'
     Modeline indicator: `Zh-GB/Hz'.  A type `no-conversion' coding
     system with Unix EOL convention (ie, no conversion) using
     post-read-decode and pre-write-encode functions to translate the
     Hz/ZW coding system used for Chinese.

`iso-2022-7bit'
`iso-2022-7bit-unix'
`iso-2022-7bit-dos'
`iso-2022-7bit-mac'
`iso-2022-7'
     Modeline indicator: `ISO7'.  A type `iso2022' 7-bit coding system
     with `ascii' (G0) initially designated.  Other character sets must
     be explicitly designated to be used.

`iso-2022-7bit-ss2'
`iso-2022-7bit-ss2-dos'
`iso-2022-7bit-ss2-mac'
`iso-2022-7bit-ss2-unix'
     Modeline indicator: `ISO7/SS'.  A type `iso2022' 7-bit coding
     system with `ascii' (G0) initially designated.  Other character
     sets must be explicitly designated to be used.  SS2 is used to
     invoke a 96-charset, one character at a time.

`iso-2022-8'
`iso-2022-8-dos'
`iso-2022-8-mac'
`iso-2022-8-unix'
     Modeline indicator: `ISO8'.  A type `iso2022' 8-bit coding system
     with `ascii' (G0) and `latin-iso8859-1' (G1) initially designated.
     Other character sets must be explicitly designated to be used.
     No single-shift or locking-shift.

`iso-2022-8bit-ss2'
`iso-2022-8bit-ss2-dos'
`iso-2022-8bit-ss2-mac'
`iso-2022-8bit-ss2-unix'
     Modeline indicator: `ISO8/SS'.  A type `iso2022' 8-bit coding
     system with `ascii' (G0) and `latin-iso8859-1' (G1) initially
     designated.  Other character sets must be explicitly designated to
     be used.  SS2 is used to invoke a 96-charset, one character at a
     time.

`iso-2022-int-1'
`iso-2022-int-1-dos'
`iso-2022-int-1-mac'
`iso-2022-int-1-unix'
     Modeline indicator: `INT-1'.  A type `iso2022' 7-bit coding system
     with `ascii' (G0) and `korean-ksc5601' (G1) initially designated.
     ISO-2022-INT-1.

`iso-2022-jp-1978-irv'
`iso-2022-jp-1978-irv-dos'
`iso-2022-jp-1978-irv-mac'
`iso-2022-jp-1978-irv-unix'
     Modeline indicator: `Ja-78/7bit'.  A type `iso2022' 7-bit coding
     system.  For compatibility with old Japanese terminals; if you
     need to know, look at the source.

`iso-2022-jp'
`iso-2022-jp-2 (ISO7/SS)'
`iso-2022-jp-dos'
`iso-2022-jp-mac'
`iso-2022-jp-unix'
`iso-2022-jp-2-dos'
`iso-2022-jp-2-mac'
`iso-2022-jp-2-unix'
     Modeline indicator: `MULE/7bit'.  A type `iso2022' 7-bit coding
     system with `ascii' (G0) initially designated, and complex
     specifications to insure backward compatibility with old Japanese
     systems.  Used for communication with mail and news in Japan.  The
     "-2" versions also use SS2 to invoke a 96-charset one character at
     a time.

`iso-2022-kr'
     Modeline indicator: `Ko/7bit'  A type `iso2022' 7-bit coding
     system with `ascii' (G0) and `korean-ksc5601' (G1) initially
     designated.  Used for e-mail in Korea.

`iso-2022-lock'
`iso-2022-lock-dos'
`iso-2022-lock-mac'
`iso-2022-lock-unix'
     Modeline indicator: `ISO7/Lock'.  A type `iso2022' 7-bit coding
     system with `ascii' (G0) initially designated, using Locking-Shift
     to invoke a 96-charset.

`iso-8859-1'
`iso-8859-1-dos'
`iso-8859-1-mac'
`iso-8859-1-unix'
     Due to implementation, this is not a type `iso2022' coding system,
     but rather an alias for the `raw-text' coding system.

`iso-8859-2'
`iso-8859-2-dos'
`iso-8859-2-mac'
`iso-8859-2-unix'
     Modeline indicator: `MIME/Ltn-2'.  A type `iso2022' coding system
     with `ascii' (G0) and `latin-iso8859-2' (G1) initially invoked.

`iso-8859-3'
`iso-8859-3-dos'
`iso-8859-3-mac'
`iso-8859-3-unix'
     Modeline indicator: `MIME/Ltn-3'.  A type `iso2022' coding system
     with `ascii' (G0) and `latin-iso8859-3' (G1) initially invoked.

`iso-8859-4'
`iso-8859-4-dos'
`iso-8859-4-mac'
`iso-8859-4-unix'
     Modeline indicator: `MIME/Ltn-4'.  A type `iso2022' coding system
     with `ascii' (G0) and `latin-iso8859-4' (G1) initially invoked.

`iso-8859-5'
`iso-8859-5-dos'
`iso-8859-5-mac'
`iso-8859-5-unix'
     Modeline indicator: `ISO8/Cyr'.  A type `iso2022' coding system
     with `ascii' (G0) and `cyrillic-iso8859-5' (G1) initially invoked.

`iso-8859-7'
`iso-8859-7-dos'
`iso-8859-7-mac'
`iso-8859-7-unix'
     Modeline indicator: `Grk'.  A type `iso2022' coding system with
     `ascii' (G0) and `greek-iso8859-7' (G1) initially invoked.

`iso-8859-8'
`iso-8859-8-dos'
`iso-8859-8-mac'
`iso-8859-8-unix'
     Modeline indicator: `MIME/Hbrw'.  A type `iso2022' coding system
     with `ascii' (G0) and `hebrew-iso8859-8' (G1) initially invoked.

`iso-8859-9'
`iso-8859-9-dos'
`iso-8859-9-mac'
`iso-8859-9-unix'
     Modeline indicator: `MIME/Ltn-5'.  A type `iso2022' coding system
     with `ascii' (G0) and `latin-iso8859-9' (G1) initially invoked.

`koi8-r'
`koi8-r-dos'
`koi8-r-mac'
`koi8-r-unix'
     Modeline indicator: `KOI8'.  A type `ccl' coding-system used for
     KOI8-R, an encoding of the Cyrillic alphabet.

`shift_jis'
`shift_jis-dos'
`shift_jis-mac'
`shift_jis-unix'
     Modeline indicator: `Ja/SJIS'.  A type `shift-jis' coding-system
     implementing the Shift-JIS encoding for Japanese.  The underscore
     is to conform to the MIME charset implementing this encoding.

`tis-620'
`tis-620-dos'
`tis-620-mac'
`tis-620-unix'
     Modeline indicator: `TIS620'.  A type `ccl' encoding for Thai.  The
     external encoding is defined by TIS620, the internal encoding is
     peculiar to MULE, and called `thai-xtis'.

`viqr'
     Modeline indicator: `VIQR'.  A type `no-conversion' coding system
     with Unix EOL convention (ie, no conversion) using
     post-read-decode and pre-write-encode functions to translate the
     VIQR coding system for Vietnamese.

`viscii'
`viscii-dos'
`viscii-mac'
`viscii-unix'
     Modeline indicator: `VISCII'.  A type `ccl' coding-system used for
     VISCII 1.1 for Vietnamese.  Differs slightly from VSCII; VISCII is
     given priority by XEmacs.

`vscii'
`vscii-dos'
`vscii-mac'
`vscii-unix'
     Modeline indicator: `VSCII'.  A type `ccl' coding-system used for
     VSCII 1.1 for Vietnamese.  Differs slightly from VISCII, which is
     given priority by XEmacs.  Use `(prefer-coding-system
     'vietnamese-vscii)' to give priority to VSCII.



File: lispref.info,  Node: CCL,  Next: Category Tables,  Prev: Coding Systems,  Up: MULE

63.7 CCL
========

CCL (Code Conversion Language) is a simple structured programming
language designed for character coding conversions.  A CCL program is
compiled to CCL code (represented by a vector of integers) and executed
by the CCL interpreter embedded in Emacs.  The CCL interpreter
implements a virtual machine with 8 registers called `r0', ..., `r7', a
number of control structures, and some I/O operators.  Take care when
using registers `r0' (used in implicit "set" statements) and especially
`r7' (used internally by several statements and operations, especially
for multiple return values and I/O operations).

   CCL is used for code conversion during process I/O and file I/O for
non-ISO2022 coding systems.  (It is the only way for a user to specify a
code conversion function.)  It is also used for calculating the code
point of an X11 font from a character code.  However, since CCL is
designed as a powerful programming language, it can be used for more
generic calculation where efficiency is demanded.  A combination of
three or more arithmetic operations can be calculated faster by CCL than
by Emacs Lisp.

   *Warning:*  The code in `src/mule-ccl.c' and
`$packages/lisp/mule-base/mule-ccl.el' is the definitive description of
CCL's semantics.  The previous version of this section contained
several typos and obsolete names left from earlier versions of MULE,
and many may remain.  (I am not an experienced CCL programmer; the few
who know CCL well find writing English painful.)

   A CCL program transforms an input data stream into an output data
stream.  The input stream, held in a buffer of constant bytes, is left
unchanged.  The buffer may be filled by an external input operation,
taken from an Emacs buffer, or taken from a Lisp string.  The output
buffer is a dynamic array of bytes, which can be written by an external
output operation, inserted into an Emacs buffer, or returned as a Lisp
string.

   A CCL program is a (Lisp) list containing two or three members.  The
first member is the "buffer magnification", which indicates the
required minimum size of the output buffer as a multiple of the input
buffer.  It is followed by the "main block" which executes while there
is input remaining, and an optional "EOF block" which is executed when
the input is exhausted.  Both the main block and the EOF block are CCL
blocks.

   A "CCL block" is either a CCL statement or list of CCL statements.
A "CCL statement" is either a "set statement" (either an integer or an
"assignment", which is a list of a register to receive the assignment,
an assignment operator, and an expression) or a "control statement" (a
list starting with a keyword, whose allowable syntax depends on the
keyword).

* Menu:

* CCL Syntax::          CCL program syntax in BNF notation.
* CCL Statements::      Semantics of CCL statements.
* CCL Expressions::     Operators and expressions in CCL.
* Calling CCL::         Running CCL programs.
* CCL Example::         A trivial program to transform the Web's URL encoding.


File: lispref.info,  Node: CCL Syntax,  Next: CCL Statements,  Up: CCL

63.7.1 CCL Syntax
-----------------

The full syntax of a CCL program in BNF notation:

CCL_PROGRAM :=
        (BUFFER_MAGNIFICATION
         CCL_MAIN_BLOCK
         [ CCL_EOF_BLOCK ])

BUFFER_MAGNIFICATION := integer
CCL_MAIN_BLOCK := CCL_BLOCK
CCL_EOF_BLOCK := CCL_BLOCK

CCL_BLOCK :=
        STATEMENT | (STATEMENT [STATEMENT ...])
STATEMENT :=
        SET | IF | BRANCH | LOOP | REPEAT | BREAK | READ | WRITE  | CALL
	| TRANSLATE | MAP | END

SET :=
        (REG = EXPRESSION)
        | (REG ASSIGNMENT_OPERATOR EXPRESSION)
        | INT-OR-CHAR

EXPRESSION := ARG | (EXPRESSION OPERATOR ARG)

IF := (if EXPRESSION CCL_BLOCK [CCL_BLOCK])
BRANCH := (branch EXPRESSION CCL_BLOCK [CCL_BLOCK ...])
LOOP := (loop STATEMENT [STATEMENT ...])
BREAK := (break)
REPEAT :=
        (repeat)
        | (write-repeat [REG | INT-OR-CHAR | string])
        | (write-read-repeat REG [INT-OR-CHAR | ARRAY])
READ :=
        (read REG ...)
        | (read-if (REG OPERATOR ARG) CCL_BLOCK [CCL_BLOCK])
        | (read-branch REG CCL_BLOCK [CCL_BLOCK ...])
WRITE :=
        (write REG ...)
        | (write EXPRESSION)
        | (write INT-OR-CHAR) | (write string) | (write REG ARRAY)
        | string
CALL := (call ccl-program-name)
END := (end)

REG := r0 | r1 | r2 | r3 | r4 | r5 | r6 | r7
ARG := REG | INT-OR-CHAR
OPERATOR :=
        + | - | * | / | % | & | '|' | ^ | << | >> | <8 | >8 | //
        | < | > | == | <= | >= | != | de-sjis | en-sjis
ASSIGNMENT_OPERATOR :=
        += | -= | *= | /= | %= | &= | '|=' | ^= | <<= | >>=
ARRAY := '[' INT-OR-CHAR ... ']'
INT-OR-CHAR := integer | character


File: lispref.info,  Node: CCL Statements,  Next: CCL Expressions,  Prev: CCL Syntax,  Up: CCL

63.7.2 CCL Statements
---------------------

The Emacs Code Conversion Language provides the following statement
types: "set", "if", "branch", "loop", "repeat", "break", "read",
"write", "call", and "end".

Set statement:
==============

The "set" statement has three variants with the syntaxes `(REG =
EXPRESSION)', `(REG ASSIGNMENT_OPERATOR EXPRESSION)', and `INTEGER'.
The assignment operator variation of the "set" statement works the same
way as the corresponding C expression statement does.  The assignment
operators are `+=', `-=', `*=', `/=', `%=', `&=', `|=', `^=', `<<=',
and `>>=', and they have the same meanings as in C.  A "naked integer"
INTEGER is equivalent to a SET statement of the form `(r0 = INTEGER)'.

I/O statements:
===============

The "read" statement takes one or more registers as arguments.  It
reads one byte (a C char) from the input into each register in turn.

   The "write" takes several forms.  In the form `(write REG ...)' it
takes one or more registers as arguments and writes each in turn to the
output.  The integer in a register (interpreted as an Ichar) is encoded
to multibyte form (ie, Ibytes) and written to the current output
buffer.  If it is less than 256, it is written as is.  The forms
`(write EXPRESSION)' and `(write INTEGER)' are treated analogously.
The form `(write STRING)' writes the constant string to the output.  A
"naked string" `STRING' is equivalent to the statement `(write
STRING)'.  The form `(write REG ARRAY)' writes the REGth element of the
ARRAY to the output.

Conditional statements:
=======================

The "if" statement takes an EXPRESSION, a CCL BLOCK, and an optional
SECOND CCL BLOCK as arguments.  If the EXPRESSION evaluates to
non-zero, the first CCL BLOCK is executed.  Otherwise, if there is a
SECOND CCL BLOCK, it is executed.

   The "read-if" variant of the "if" statement takes an EXPRESSION, a
CCL BLOCK, and an optional SECOND CCL BLOCK as arguments.  The
EXPRESSION must have the form `(REG OPERATOR OPERAND)' (where OPERAND is
a register or an integer).  The `read-if' statement first reads from
the input into the first register operand in the EXPRESSION, then
conditionally executes a CCL block just as the `if' statement does.

   The "branch" statement takes an EXPRESSION and one or more CCL
blocks as arguments.  The CCL blocks are treated as a zero-indexed
array, and the `branch' statement uses the EXPRESSION as the index of
the CCL block to execute.  Null CCL blocks may be used as no-ops,
continuing execution with the statement following the `branch'
statement in the containing CCL block.  Out-of-range values for the
EXPRESSION are also treated as no-ops.

   The "read-branch" variant of the "branch" statement takes an
REGISTER, a CCL BLOCK, and an optional SECOND CCL BLOCK as arguments.
The `read-branch' statement first reads from the input into the
REGISTER, then conditionally executes a CCL block just as the `branch'
statement does.

Loop control statements:
========================

The "loop" statement creates a block with an implied jump from the end
of the block back to its head.  The loop is exited on a `break'
statement, and continued without executing the tail by a `repeat'
statement.

   The "break" statement, written `(break)', terminates the current
loop and continues with the next statement in the current block.

   The "repeat" statement has three variants, `repeat', `write-repeat',
and `write-read-repeat'.  Each continues the current loop from its
head, possibly after performing I/O.  `repeat' takes no arguments and
does no I/O before jumping.  `write-repeat' takes a single argument (a
register, an integer, or a string), writes it to the output, then jumps.
`write-read-repeat' takes one or two arguments.  The first must be a
register.  The second may be an integer or an array; if absent, it is
implicitly set to the first (register) argument.  `write-read-repeat'
writes its second argument to the output, then reads from the input
into the register, and finally jumps.  See the `write' and `read'
statements for the semantics of the I/O operations for each type of
argument.

Other control statements:
=========================

The "call" statement, written `(call CCL-PROGRAM-NAME)', executes a CCL
program as a subroutine.  It does not return a value to the caller, but
can modify the register status.

   The "end" statement, written `(end)', terminates the CCL program
successfully, and returns to caller (which may be a CCL program).  It
does not alter the status of the registers.


File: lispref.info,  Node: CCL Expressions,  Next: Calling CCL,  Prev: CCL Statements,  Up: CCL

63.7.3 CCL Expressions
----------------------

CCL, unlike Lisp, uses infix expressions.  The simplest CCL expressions
consist of a single OPERAND, either a register (one of `r0', ..., `r0')
or an integer.  Complex expressions are lists of the form `( EXPRESSION
OPERATOR OPERAND )'.  Unlike C, assignments are not expressions.

   In the following table, X is the target resister for a "set".  In
subexpressions, this is implicitly `r7'.  This means that `>8', `//',
`de-sjis', and `en-sjis' cannot be used freely in subexpressions, since
they return parts of their values in `r7'.  Y may be an expression,
register, or integer, while Z must be a register or an integer.

Name             Operator   Code   C-like Description
CCL_PLUS         `+'        0x00   X = Y + Z
CCL_MINUS        `-'        0x01   X = Y - Z
CCL_MUL          `*'        0x02   X = Y * Z
CCL_DIV          `/'        0x03   X = Y / Z
CCL_MOD          `%'        0x04   X = Y % Z
CCL_AND          `&'        0x05   X = Y & Z
CCL_OR           `|'        0x06   X = Y | Z
CCL_XOR          `^'        0x07   X = Y ^ Z
CCL_LSH          `<<'       0x08   X = Y << Z
CCL_RSH          `>>'       0x09   X = Y >> Z
CCL_LSH8         `<8'       0x0A   X = (Y << 8) | Z
CCL_RSH8         `>8'       0x0B   X = Y >> 8, r[7] = Y & 0xFF
CCL_DIVMOD       `//'       0x0C   X = Y / Z, r[7] = Y % Z
CCL_LS           `<'        0x10   X = (X < Y)
CCL_GT           `>'        0x11   X = (X > Y)
CCL_EQ           `=='       0x12   X = (X == Y)
CCL_LE           `<='       0x13   X = (X <= Y)
CCL_GE           `>='       0x14   X = (X >= Y)
CCL_NE           `!='       0x15   X = (X != Y)
CCL_ENCODE_SJIS  `en-sjis'  0x16   X = HIGHER_BYTE (SJIS (Y, Z))
                                   r[7] = LOWER_BYTE (SJIS (Y, Z)
CCL_DECODE_SJIS  `de-sjis'  0x17   X = HIGHER_BYTE (DE-SJIS (Y, Z))
                                   r[7] = LOWER_BYTE (DE-SJIS (Y, Z))

   The CCL operators are as in C, with the addition of CCL_LSH8,
CCL_RSH8, CCL_DIVMOD, CCL_ENCODE_SJIS, and CCL_DECODE_SJIS.  The
CCL_ENCODE_SJIS and CCL_DECODE_SJIS treat their first and second bytes
as the high and low bytes of a two-byte character code.  (SJIS stands
for Shift JIS, an encoding of Japanese characters used by Microsoft.
CCL_ENCODE_SJIS is a complicated transformation of the Japanese
standard JIS encoding to Shift JIS.  CCL_DECODE_SJIS is its inverse.)
It is somewhat odd to represent the SJIS operations in infix form.


File: lispref.info,  Node: Calling CCL,  Next: CCL Example,  Prev: CCL Expressions,  Up: CCL

63.7.4 Calling CCL
------------------

CCL programs are called automatically during Emacs buffer I/O when the
external representation has a coding system type of `shift-jis',
`big5', or `ccl'.  The program is specified by the coding system (*note
Coding Systems::).  You can also call CCL programs from other CCL
programs, and from Lisp using these functions:

 -- Function: ccl-execute ccl-program status
     Execute CCL-PROGRAM with registers initialized by STATUS.
     CCL-PROGRAM is a vector of compiled CCL code created by
     `ccl-compile'.  It is an error for the program to try to execute a
     CCL I/O command.  STATUS must be a vector of nine values,
     specifying the initial value for the R0, R1 .. R7 registers and
     for the instruction counter IC.  A `nil' value for a register
     initializer causes the register to be set to 0.  A `nil' value for
     the IC initializer causes execution to start at the beginning of
     the program.  When the program is done, STATUS is modified (by
     side-effect) to contain the ending values for the corresponding
     registers and IC.

 -- Function: ccl-execute-on-string ccl-program status string &optional
          continue
     Execute CCL-PROGRAM with initial STATUS on STRING.  CCL-PROGRAM is
     a vector of compiled CCL code created by `ccl-compile'.  STATUS
     must be a vector of nine values, specifying the initial value for
     the R0, R1 .. R7 registers and for the instruction counter IC.  A
     `nil' value for a register initializer causes the register to be
     set to 0.  A `nil' value for the IC initializer causes execution
     to start at the beginning of the program.  An optional fourth
     argument CONTINUE, if non-`nil', causes the IC to remain on the
     unsatisfied read operation if the program terminates due to
     exhaustion of the input buffer.  Otherwise the IC is set to the end
     of the program.  When the program is done, STATUS is modified (by
     side-effect) to contain the ending values for the corresponding
     registers and IC.  Returns the resulting string.

   To call a CCL program from another CCL program, it must first be
registered:

 -- Function: register-ccl-program name ccl-program
     Register NAME for CCL program CCL-PROGRAM in `ccl-program-table'.
     CCL-PROGRAM should be the compiled form of a CCL program, or
     `nil'.  Return index number of the registered CCL program.

   Information about the processor time used by the CCL interpreter can
be obtained using these functions:

 -- Function: ccl-elapsed-time
     Returns the elapsed processor time of the CCL interpreter as cons
     of user and system time, as floating point numbers measured in
     seconds.  If only one overall value can be determined, the return
     value will be a cons of that value and 0.

 -- Function: ccl-reset-elapsed-time
     Resets the CCL interpreter's internal elapsed time registers.


File: lispref.info,  Node: CCL Example,  Prev: Calling CCL,  Up: CCL

63.7.5 CCL Example
------------------

In this section, we describe the implementation of a trivial coding
system to transform from the Web's URL encoding to XEmacs' internal
coding.  Many people will have been first exposed to URL encoding when
they saw "%20" where they expected a space in a file's name on their
local hard disk; this can happen when a browser saves a file from the
web and doesn't encode the name, as passed from the server, properly.

   URL encoding itself is underspecified with regard to encodings beyond
ASCII.  The relevant document, RFC 1738, explicitly doesn't give any
information on how to encode non-ASCII characters, and the "obvious"
way--use the %xx values for the octets of the eight bit MIME character
set in which the page was served--breaks when a user types a character
outside that character set.  Best practice for web development is to
serve all pages as UTF-8 and treat incoming form data as using that
coding system.  (Oh, and gamble that your clients won't ever want to
type anything outside Unicode.  But that's not so much of a gamble with
today's client operating systems.)  We don't treat non-ASCII in this
example, as dealing with `(read-multibyte-character ...)' and errors
therewith would make it much harder to understand.

   Since CCL isn't a very rich language, we move much of the logic that
would ordinarily be computed from operations like `(member ..)', `(and
...)' and `(or ...)' into tables, from which register values are read
and written, and on which `if' statements are predicated.  Much more of
the implementation of this coding system is occupied with constructing
these tables--in normal Emacs Lisp--than it is with actual CCL code.

   All the `defvar' statements we deal with in the next few sections
are surrounded by a `(eval-and-compile ...)', which means that the
logic which initializes these variables executes at compile time, and if
XEmacs loads the compiled version of the file, these variables are
initialized as constants.

* Menu:

* Four bits to ASCII::  Two tables used for getting hex digits from ASCII.
* URI Encoding constants::  Useful predefined characters.
* Numeric to ASCII-hexadecimal conversion:: Trivial in Lisp, not so in CCL.
* Characters to be preserved:: No transformation needed for these characters.
* The program to decode to internal format:: .
* The program to encode from internal format:: .
* The actual coding system:: .


File: lispref.info,  Node: Four bits to ASCII,  Next: URI Encoding constants,  Up: CCL Example

63.7.5.1 Four bits to ASCII
...........................

The first `defvar' is for `url-coding-high-order-nybble-as-ascii', a
256-entry table that maps from an octet's value to the ASCII encoding
for the hex value of its most significant four bits.  That might sound
complex, but it isn't; for decimal 65, hex value `#x41', the entry in
the table is the ASCII encoding of `4'.  For decimal 122, ASCII `z',
hex value `#x7a', `(elt url-coding-high-order-nybble-as-ascii #x7a)'
after this file is loaded gives the ASCII encoding of 7.

     (defvar url-coding-high-order-nybble-as-ascii
       (let ((val (make-vector 256 0))
     	(i 0))
         (while (< i (length val))
           (aset val i (char-to-int (aref (format "%02X" i) 0)))
           (setq i (1+ i)))
         val)
       "Table to find an ASCII version of an octet's most significant 4 bits.")

   The next table, `url-coding-low-order-nybble-as-ascii' is almost the
same thing, but this time it has a map for the hex encoding of the
low-order four bits.  So the sixty-fifth entry (offset `#x41') is the
ASCII encoding of `1', the hundred-and-twenty-second (offset `#x7a') is
the ASCII encoding of `A'.

     (defvar url-coding-low-order-nybble-as-ascii
       (let ((val (make-vector 256 0))
     	(i 0))
         (while (< i (length val))
           (aset val i (char-to-int (aref (format "%02X" i) 1)))
           (setq i (1+ i)))
         val)
       "Table to find an ASCII version of an octet's least significant 4 bits.")


File: lispref.info,  Node: URI Encoding constants,  Next: Numeric to ASCII-hexadecimal conversion,  Prev: Four bits to ASCII,  Up: CCL Example

63.7.5.2 URI Encoding constants
...............................

Next, we have a couple of variables that make the CCL code more
readable.  The first is the ASCII encoding of the percentage sign; this
character is used as an escape code, to start the encoding of a
non-printable character.  For historical reasons, URL encoding allows
the space character to be encoded as a plus sign-it does make typing
URLs like `http://google.com/search?q=XEmacs+home+page' easier-and as
such, we have to check when decoding for this value, and map it to the
space character.  When doing this in CCL, we use the
`url-coding-escaped-space-code' variable.

     (defvar url-coding-escape-character-code (char-to-int ?%)
       "The code point for the percentage sign, in ASCII.")

     (defvar url-coding-escaped-space-code (char-to-int ?+)
       "The URL-encoded value of the space character, that is, +.")


File: lispref.info,  Node: Numeric to ASCII-hexadecimal conversion,  Next: Characters to be preserved,  Prev: URI Encoding constants,  Up: CCL Example

63.7.5.3 Numeric to ASCII-hexadecimal conversion
................................................

Now, we have a couple of utility tables that wouldn't be necessary in a
more expressive programming language than is CCL. The first is sixteen
in length, and maps a hexadecimal number to the ASCII encoding of that
number; so zero maps to ASCII `0', ten maps to ASCII `A.' The second
does the reverse; that is, it maps an ASCII character to its value when
interpreted as a hexadecimal digit. ('A' => 10, 'c' => 12, '2' => 2, as
a few examples.)

     (defvar url-coding-hex-digit-table
       (let ((i 0)
     	(val (make-vector 16 0)))
         (while (< i 16)
           (aset val i (char-to-int (aref (format "%X" i) 0)))
           (setq i (1+ i)))
         val)
       "A map from a hexadecimal digit's numeric value to its encoding in ASCII.")

     (defvar url-coding-latin-1-as-hex-table
       (let ((val (make-vector 256 0))
     	(i 0))
         (while (< i (length val))
           ;; Get a hex val for this ASCII character.
           (aset val i (string-to-int (format "%c" i) 16))
           (setq i (1+ i)))
         val)
       "A map from Latin 1 code points to their values as hexadecimal digits.")


File: lispref.info,  Node: Characters to be preserved,  Next: The program to decode to internal format,  Prev: Numeric to ASCII-hexadecimal conversion,  Up: CCL Example

63.7.5.4 Characters to be preserved
...................................

And finally, the last of these tables.  URL encoding says that
alphanumeric characters, the underscore, hyphen and the full stop (1)
retain their ASCII encoding, and don't undergo transformation.
`url-coding-should-preserve-table' is an array in which the entries are
one if the corresponding ASCII character should be left as-is, and zero
if they should be transformed.  So the entries for all the control and
most of the punctuation charcters are zero.  Lisp programmers will
observe that this initialization is particularly inefficient, but
they'll also be aware that this is a long way from an inner loop where
every nanosecond counts.

     (defvar url-coding-should-preserve-table
       (let ((preserve
     	 (list ?- ?_ ?. ?a ?b ?c ?d ?e ?f ?g ?h ?i ?j ?k ?l ?m ?n ?o
     	       ?p ?q ?r ?s ?t ?u ?v ?w ?x ?y ?z ?A ?B ?C ?D ?E ?F ?G
     	       ?H ?I ?J ?K ?L ?M ?N ?O ?P ?Q ?R ?S ?T ?U ?V ?W ?X ?Y
     	       ?Z ?0 ?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8 ?9))
     	(i 0)
     	(res (make-vector 256 0)))
         (while (< i 256)
           (when (member (int-char i) preserve)
     	(aset res i 1))
           (setq i (1+ i)))
         res)
       "A 256-entry array of flags, indicating whether or not to preserve an
     octet as its ASCII encoding.")

   ---------- Footnotes ----------

   (1) That's what the standards call it, though my North American
readers will be more familiar with it as the period character.


File: lispref.info,  Node: The program to decode to internal format,  Next: The program to encode from internal format,  Prev: Characters to be preserved,  Up: CCL Example

63.7.5.5 The program to decode to internal format
.................................................

After the almost interminable tables, we get to the CCL.  The first CCL
program, `ccl-decode-urlcoding' decodes from the URL coding to our
internal format; since this version of CCL doesn't have support for
error checking on the input, we don't do any verification on it.

   The buffer magnification-approximate ratio of the size of the output
buffer to the size of the input buffer-is declared as one, because
fractional values aren't allowed. (Since all those %20's will map to `
', the length of the output text will be less than that of the input
text.)

   So, first we read an octet from the input buffer into register `r0',
to set up the loop.  Next, we start the loop, with a `(loop ...)'
statement, and we check if the value in `r0' is a percentage sign.
(Note the comma before `url-coding-escape-character-code'; since CCL is
a Lisp macro language, we can break out of the macro evaluation with a
comman, and as such, "`,url-coding-escape-character-code'" will be
evaluated as a literal `37.')

   If it is a percentage sign, we read the next two octets into `r2'
and `r3', and convert them into their hexadecimal numeric values, using
the `url-coding-latin-1-as-hex-table' array declared above.  (But
again, it'll be interpreted as a literal array.)  We then left shift
the first by four bits, mask the two together, and write the result to
the output buffer.

   If it isn't a percentage sign, and it is a `+' sign, we write a
space-hexadecimal 20-to the output buffer.

   If none of those things are true, we pass the octet to the output
buffer untransformed.  (This could be a place to put error checking, in
a more expressive language.)  We then read one more octet from the input
buffer, and move to the next iteration of the loop.

     (define-ccl-program ccl-decode-urlcoding
       `(1
         ((read r0)
          (loop
            (if (r0 == ,url-coding-escape-character-code)
     	   ((read r2 r3)
     	    ;; Assign the value at offset r2 in the url-coding-hex-digit-table
     	    ;; to r3.
     	    (r2 = r2 ,url-coding-latin-1-as-hex-table)
     	    (r3 = r3 ,url-coding-latin-1-as-hex-table)
     	    (r2 <<= 4)
     	    (r3 |= r2)
     	    (write r3))
     	 (if (r0 == ,url-coding-escaped-space-code)
     	     (write #x20)
     	   (write r0)))
            (read r0)
            (repeat))))
       "CCL program to take URI-encoded ASCII text and transform it to our
     internal encoding. ")


File: lispref.info,  Node: The program to encode from internal format,  Next: The actual coding system,  Prev: The program to decode to internal format,  Up: CCL Example

63.7.5.6 The program to encode from internal format
...................................................

Next, we see the CCL program to encode ASCII text as URL coded text.
Here, the buffer magnification is specified as three, to account for ` '
mapping to %20, etc.  As before, we read an octet from the input into
`r0', and move into the body of the loop.  Next, we check if we should
preserve the value of this octet, by reading from offset `r0' in the
`url-coding-should-preserve-table' into `r1'.  Then we have an `if'
statement predicated on the value in `r1'; for the true branch, we
write the input octet directly.  For the false branch, we write a
percentage sign, the ASCII encoding of the high four bits in hex, and
then the ASCII encoding of the low four bits in hex.

   We then read an octet from the input into `r0', and repeat the loop.

     (define-ccl-program ccl-encode-urlcoding
       `(3
         ((read r0)
          (loop
            (r1 = r0 ,url-coding-should-preserve-table)
            ;; If we should preserve the value, just write the octet directly.
            (if r1
     	   (write r0)
     	 ;; else, write a percentage sign, and the hex value of the octet, in
     	 ;; an ASCII-friendly format.
     	 ((write ,url-coding-escape-character-code)
     	  (write r0 ,url-coding-high-order-nybble-as-ascii)
     	  (write r0 ,url-coding-low-order-nybble-as-ascii)))
            (read r0)
            (repeat))))
       "CCL program to encode octets (almost) according to RFC 1738")


File: lispref.info,  Node: The actual coding system,  Prev: The program to encode from internal format,  Up: CCL Example

63.7.5.7 The actual coding system
.................................

To actually create the coding system, we call `make-coding-system'.
The first argument is the symbol that is to be the name of the coding
system, in our case `url-coding'. The second specifies that the coding
system is to be of type `ccl'--there are several other coding system
types available, including, see the documentation for
`make-coding-system' for the full list. Then there's a documentation
string describing the wherefore and caveats of the coding system, and
the final argument is a property list giving information about the CCL
programs and the coding system's mnemonic.

     (make-coding-system
      'url-coding 'ccl
      "The coding used by application/x-www-form-urlencoded HTTP applications.
     This coding form doesn't specify anything about non-ASCII characters, so
     make sure you've transformed to a seven-bit coding system first."
      '(decode ccl-decode-urlcoding
        encode ccl-encode-urlcoding
        mnemonic "URLenc"))

   If you're lucky, the `url-coding' coding system describe here should
be available in the XEmacs package system. Otherwise, downloading it
from `http://www.parhasard.net/url-coding.el' should work for the
foreseeable future.


File: lispref.info,  Node: Category Tables,  Next: Unicode Support,  Prev: CCL,  Up: MULE

63.8 Category Tables
====================

A category table is a type of char table used for keeping track of
categories.  Categories are used for classifying characters for use in
regexps--you can refer to a category rather than having to use a
complicated [] expression (and category lookups are significantly
faster).

   There are 95 different categories available, one for each printable
character (including space) in the ASCII charset.  Each category is
designated by one such character, called a "category designator".  They
are specified in a regexp using the syntax `\cX', where X is a category
designator. (This is not yet implemented.)

   A category table specifies, for each character, the categories that
the character is in.  Note that a character can be in more than one
category.  More specifically, a category table maps from a character to
either the value `nil' (meaning the character is in no categories) or a
95-element bit vector, specifying for each of the 95 categories whether
the character is in that category.

   Special Lisp functions are provided that abstract this, so you do not
have to directly manipulate bit vectors.

 -- Function: category-table-p object
     This function returns `t' if OBJECT is a category table.

 -- Function: category-table &optional buffer
     This function returns the current category table.  This is the one
     specified by the current buffer, or by BUFFER if it is non-`nil'.

 -- Function: standard-category-table
     This function returns the standard category table.  This is the
     one used for new buffers.

 -- Function: copy-category-table &optional category-table
     This function returns a new category table which is a copy of
     CATEGORY-TABLE, which defaults to the standard category table.

 -- Function: set-category-table category-table &optional buffer
     This function selects CATEGORY-TABLE as the new category table for
     BUFFER.  BUFFER defaults to the current buffer if omitted.

 -- Function: category-designator-p object
     This function returns `t' if OBJECT is a category designator (a
     char in the range `' '' to `'~'').

 -- Function: category-table-value-p object
     This function returns `t' if OBJECT is a category table value.
     Valid values are `nil' or a bit vector of size 95.


File: lispref.info,  Node: Unicode Support,  Next: Charset Unification,  Prev: Category Tables,  Up: MULE

63.9 Unicode Support
====================

Unicode support was added by Ben Wing to XEmacs 21.5.6.

 -- Function: set-language-unicode-precedence-list list
     Set the language-specific precedence list used for Unicode
     decoding.  This is a list of charsets, which are consulted in
     order for a translation matching a given Unicode character.  If no
     matches are found, the charsets in the default precedence list (see
     `set-default-unicode-precedence-list') are consulted, and then all
     remaining charsets, in some arbitrary order.

     The language-specific precedence list is meant to be set as part
     of the language environment initialization; the default precedence
     list is meant to be set by the user.

 -- Function: language-unicode-precedence-list
     Return the language-specific precedence list used for Unicode
     decoding.  See `set-language-unicode-precedence-list' for more
     information.

 -- Function: set-default-unicode-precedence-list list
     Set the default precedence list used for Unicode decoding.  This
     is meant to be set by the user.  See
     `set-language-unicode-precedence-list' for more information.

 -- Function: default-unicode-precedence-list
     Return the default precedence list used for Unicode decoding.  See
     `set-language-unicode-precedence-list' for more information.

 -- Function: set-unicode-conversion character code
     Add conversion information between Unicode codepoints and
     characters.  CHARACTER is one of the following:

     - A character (in which case CODE must be a non-negative integer)
     - A vector of characters (in which case CODE must be a vector of
     non-negative integers of the same length)

     Values of CODE above 2^20 - 1 are allowed for the purpose of
     specifying private characters, but will cause errors when
     converted to UTF-16 or UTF-32.  UCS-4 and UTF-8 can handle values
     to 2^31 - 1, but XEmacs Lisp integers top out at 2^30 - 1.

 -- Function: character-to-unicode character
     Convert CHARACTER to Unicode codepoint.  When there is no
     international support (i.e. MULE is not defined), this function
     simply does `char-to-int'.

 -- Function: unicode-to-character code [charsets]
     Convert Unicode codepoint CODE to character.  CODE should be a
     non-negative integer.  If CHARSETS is given, it should be a list
     of charsets, and only those charsets will be consulted, in the
     given order, for a translation.  Otherwise, the default ordering
     of all charsets will be given (see
     `set-unicode-charset-precedence').

     When there is no international support (i.e. MULE is not defined),
     this function simply does `int-to-char' and ignores the CHARSETS
     argument.

 -- Function: parse-unicode-translation-table filename charset start
          end offset flags
     Parse Unicode translation data in FILENAME for MULE CHARSET.  Data
     is text, in the form of one translation per line - charset
     codepoint followed by Unicode codepoint.  Numbers are decimal or
     hex \(preceded by 0x).  Comments are marked with a #.  Charset
     codepoints for two-dimensional charsets should have the first
     octet stored in the high 8 bits of the hex number and the second
     in the low 8 bits.

     If START and END are given, only charset codepoints within the
     given range will be processed.  If OFFSET is given, that value
     will be added to all charset codepoints in the file to obtain the
     internal charset codepoint.  START and END apply to the codepoints
     in the file, before OFFSET is applied.

     (Note that, as usual, we assume that octets are in the range 32 to
     127 or 33 to 126.  If you have a table in kuten form, with octets
     in the range 1 to 94, you will have to use an offset of 5140, i.e.
     0x2020.)

     FLAGS, if specified, control further how the tables are interpreted
     and are used to special-case certain known table weirdnesses in the
     Unicode tables:

    `ignore-first-column''
          Exactly as it sounds.  The JIS X 0208 tables have 3 columns
          of data instead of 2; the first is the Shift-JIS codepoint.

    `big5'
          The charset codepoint is a Big Five codepoint; convert it to
          the proper hacked-up codepoint in `chinese-big5-1' or
          `chinese-big5-2'.


File: lispref.info,  Node: Charset Unification,  Next: Charsets and Coding Systems,  Prev: Unicode Support,  Up: MULE

63.10 Character Set Unification
===============================

Mule suffers from a design defect that causes it to consider the ISO
Latin character sets to be disjoint.  This results in oddities such as
files containing both ISO 8859/1 and ISO 8859/15 codes, and using ISO
2022 control sequences to switch between them, as well as more plausible
but often unnecessary combinations like ISO 8859/1 with ISO 8859/2.
This can be very annoying when sending messages or even in simple
editing on a single host.  Unification works around the problem by
converting as many characters as possible to use a single Latin coded
character set before saving the buffer.

   This node and its children were ripp'd untimely from
`latin-unity.texi', and have been quickly converted for use here.
However as APIs are likely to diverge, beware of inaccuracies.  Please
report any you discover with `M-x report-xemacs-bug RET', as well as
any ambiguities or downright unintelligible passages.

   A lot of the stuff here doesn't belong here; it belongs in the *Note
Top: (xemacs)Top.  Report those as bugs, too, preferably with patches.

* Menu:

* Overview::                    Unification history and general information.
* Usage::                       An overview of the operation of Unification.
* Configuration::               Configuring Unification for use.
* Theory of Operation::         How Unification works.
* What Unification Cannot Do for You::  Inherent problems of 8-bit charsets.
* Charsets and Coding Systems:: Reference lists with annotations.
* Unification Internals::       Utilities and implementation details.


File: lispref.info,  Node: Overview,  Next: Usage,  Prev: Charset Unification,  Up: Charset Unification

63.10.1 An Overview of Unification
----------------------------------

Mule suffers from a design defect that causes it to consider the ISO
Latin character sets to be disjoint.  This manifests itself when a user
enters characters using input methods associated with different coded
character sets into a single buffer.

   A very important example involves email.  Many sites, especially in
the U.S., default to use of the ISO 8859/1 coded character set (also
called "Latin 1," though these are somewhat different concepts).
However, ISO 8859/1 provides a generic CURRENCY SIGN character.  Now
that the Euro has become the official currency of most countries in
Europe, this is unsatisfactory (and in practice, useless).  So
Europeans generally use ISO 8859/15, which is nearly identical to ISO
8859/1 for most languages, except that it substitutes EURO SIGN for
CURRENCY SIGN.

   Suppose a European user yanks text from a post encoded in ISO 8859/1
into a message composition buffer, and enters some text including the
Euro sign.  Then Mule will consider the buffer to contain both ISO
8859/1 and ISO 8859/15 text, and MUAs such as Gnus will (if naively
programmed) send the message as a multipart mixed MIME body!

   This is clearly stupid.  What is not as obvious is that, just as any
European can include American English in their text because ASCII is a
subset of ISO 8859/15, most European languages which use Latin
characters (eg, German and Polish) can typically be mixed while using
only one Latin coded character set (in this case, ISO 8859/2).  However,
this often depends on exactly what text is to be encoded.

   Unification works around the problem by converting as many
characters as possible to use a single Latin coded character set before
saving the buffer.


File: lispref.info,  Node: Usage,  Next: Configuration,  Prev: Overview,  Up: Charset Unification

63.10.2 Operation of Unification
--------------------------------

Normally, Unification works in the background by installing
`unity-sanity-check' on `write-region-pre-hook'.  This is done by
default for the ISO 8859 Latin family of character sets.  The user
activates this functionality for other character set families by
invoking `enable-unification', either interactively or in her init
file.  *Note Init File: (xemacs)Init File.  Unification can be
deactivated by invoking `disable-unification'.

   Unification also provides a few functions for remapping or recoding
the buffer by hand.  To "remap" a character means to change the buffer
representation of the character by using another coded character set.
Remapping never changes the identity of the character, but may involve
altering the code point of the character.  To "recode" a character
means to simply change the coded character set.  Recoding never alters
the code point of the character, but may change the identity of the
character.  *Note Theory of Operation::.

   There are a few variables which determine which coding systems are
always acceptable to Unification:  `unity-ucs-list',
`unity-preferred-coding-system-list', and
`unity-preapproved-coding-system-list'.  The latter two default to
`()', and should probably be avoided because they short-circuit the
sanity check.  If you find you need to use them, consider reporting it
as a bug or request for enhancement.  Because they seem unsafe, the
recommended interface is likely to change.

* Menu:

* Basic Functionality::            User interface and customization.
* Interactive Usage::              Treating text by hand.
                                   Also documents the hook function(s).


File: lispref.info,  Node: Basic Functionality,  Next: Interactive Usage,  Up: Usage

63.11 Basic Functionality
=========================

These functions and user options initialize and configure Unification.
In normal use, none of these should be needed.

   *These APIs are certain to change.*

 -- Function: enable-unification
     Set up hooks and initialize variables for latin-unity.

     There are no arguments.

     This function is idempotent.  It will reinitialize any hooks or
     variables that are not in initial state.

 -- Function: disable-unification
     There are no arguments.

     Clean up hooks and void variables used by latin-unity.

 -- User Option: unity-ucs-list
     List of coding systems considered to be universal.

     The default value is `'(utf-8 iso-2022-7 ctext escape-quoted)'.

     Order matters; coding systems earlier in the list will be
     preferred when recommending a coding system.  These coding systems
     will not be used without querying the user (unless they are also
     present in `unity-preapproved-coding-system-list'), and follow the
     `unity-preferred-coding-system-list' in the list of suggested
     coding systems.

     If none of the preferred coding systems are feasible, the first in
     this list will be the default.

     Notes on certain coding systems:  `escape-quoted' is a special
     coding system used for autosaves and compiled Lisp in Mule.  You
     should never delete this, although it is rare that a user would
     want to use it directly.  Unification does not try to be \"smart\"
     about other general ISO 2022 coding systems, such as ISO-2022-JP.
     (They are not recognized as equivalent to `iso-2022-7'.)  If your
     preferred coding system is one of these, you may consider adding
     it to `unity-ucs-list'.  However, this will typically have the
     side effect that (eg) ISO 8859/1 files will be saved in 7-bit form
     with ISO 2022 escape sequences.

   Coding systems which are not Latin and not in `unity-ucs-list' are
handled by short circuiting checks of coding system against the next
two variables.

 -- User Option: unity-preapproved-coding-system-list
     List of coding systems used without querying the user if feasible.

     The default value is `(buffer-default preferred)'.

     The first feasible coding system in this list is used.  The
     special values `preferred' and `buffer-default' may be present:

    `buffer-default'
          Use the coding system used by `write-region', if feasible.

    `preferred'
          Use the coding system specified by `prefer-coding-system' if
          feasible.

     "Feasible" means that all characters in the buffer can be
     represented by the coding system.  Coding systems in
     `unity-ucs-list' are always considered feasible.  Other feasible
     coding systems are computed by
     `unity-representations-feasible-region'.

     Note that the first universal coding system in this list shadows
     all other coding systems.  In particular, if your preferred coding
     system is a universal coding system, and `preferred' is a member
     of this list, unification will blithely convert all your files to
     that coding system.  This is considered a feature, but it may
     surprise most users.  Users who don't like this behavior should
     put `preferred' in `unity-preferred-coding-system-list'.

 -- User Option: unity-preferred-coding-system-list
     List of coding systems suggested to the user if feasible.

     The default value is `(iso-8859-1 iso-8859-15 iso-8859-2 iso-8859-3
     iso-8859-4 iso-8859-9)'.

     If none of the coding systems in
     `unity-preapproved-coding-system-list' are feasible, this list
     will be recommended to the user, followed by the `unity-ucs-list'.
     The first coding system in this list is default.  The special
     values `preferred' and `buffer-default' may be present:

    `buffer-default'
          Use the coding system used by `write-region', if feasible.

    `preferred'
          Use the coding system specified by `prefer-coding-system' if
          feasible.

     "Feasible" means that all characters in the buffer can be
     represented by the coding system.  Coding systems in
     `unity-ucs-list' are always considered feasible.  Other feasible
     coding systems are computed by
     `unity-representations-feasible-region'.

 -- Variable: unity-iso-8859-1-aliases
     List of coding systems to be treated as aliases of ISO 8859/1.

     The default value is '(iso-8859-1).

     This is not a user variable; to customize input of coding systems
     or charsets, `unity-coding-system-alias-alist' or
     `unity-charset-alias-alist'.


File: lispref.info,  Node: Interactive Usage,  Prev: Basic Functionality,  Up: Usage

63.12 Interactive Usage
=======================

First, the hook function `unity-sanity-check' is documented.  (It is
placed here because it is not an interactive function, and there is not
yet a programmer's section of the manual.)

   These functions provide access to internal functionality (such as the
remapping function) and to extra functionality (the recoding functions
and the test function).

 -- Function: unity-sanity-check begin end filename append visit
          lockname &optional coding-system
     Check if CODING-SYSTEM can represent all characters between BEGIN
     and END.

     For compatibility with old broken versions of `write-region',
     CODING-SYSTEM defaults to `buffer-file-coding-system'.  FILENAME,
     APPEND, VISIT, and LOCKNAME are ignored.

     Return nil if buffer-file-coding-system is not
     (ISO-2022-compatible) Latin.  If `buffer-file-coding-system' is
     safe for the charsets actually present in the buffer, return it.
     Otherwise, ask the user to choose a coding system, and return that.

     This function does _not_ do the safe thing when
     `buffer-file-coding-system' is nil (aka no-conversion).  It
     considers that "non-Latin," and passes it on to the Mule detection
     mechanism.

     This function is intended for use as a `write-region-pre-hook'.  It
     does nothing except return CODING-SYSTEM if `write-region'
     handlers are inhibited.

 -- Function: unity-buffer-representations-feasible
     There are no arguments.

     Apply unity-region-representations-feasible to the current buffer.

 -- Function: unity-region-representations-feasible begin end &optional
          buf
     Return character sets that can represent the text from BEGIN to
     END in BUF.

     BUF defaults to the current buffer.  Called interactively, will be
     applied to the region.  Function assumes BEGIN <= END.

     The return value is a cons.  The car is the list of character sets
     that can individually represent all of the non-ASCII portion of the
     buffer, and the cdr is the list of character sets that can
     individually represent all of the ASCII portion.

     The following is taken from a comment in the source.  Please refer
     to the source to be sure of an accurate description.

     The basic algorithm is to map over the region, compute the set of
     charsets that can represent each character (the "feasible
     charset"), and take the intersection of those sets.

     The current implementation takes advantage of the fact that ASCII
     characters are common and cannot change asciisets.  Then using
     skip-chars-forward makes motion over ASCII subregions very fast.

     This same strategy could be applied generally by precomputing
     classes of characters equivalent according to their effect on
     latinsets, and adding a whole class to the skip-chars-forward
     string once a member is found.

     Probably efficiency is a function of the number of characters
     matched, or maybe the length of the match string?  With
     `skip-category-forward' over a precomputed category table it
     should be really fast.  In practice for Latin character sets there
     are only 29 classes.

 -- Function: unity-remap-region begin end character-set &optional
          coding-system
     Remap characters between BEGIN and END to equivalents in
     CHARACTER-SET.  Optional argument CODING-SYSTEM may be a coding
     system name (a symbol) or nil.  Characters with no equivalent are
     left as-is.

     When called interactively, BEGIN and END are set to the beginning
     and end, respectively, of the active region, and the function
     prompts for CHARACTER-SET.  The function does completion, knows
     how to guess a character set name from a coding system name, and
     also provides some common aliases.  See `unity-guess-charset'.
     There is no way to specify CODING-SYSTEM, as it has no useful
     function interactively.

     Return CODING-SYSTEM if CODING-SYSTEM can encode all characters in
     the region, t if CODING-SYSTEM is nil and the coding system with
     G0 = 'ascii and G1 = CHARACTER-SET can encode all characters, and
     otherwise nil.  Note that a non-null return does _not_ mean it is
     safe to write the file, only the specified region.  (This behavior
     is useful for multipart MIME encoding and the like.)

     Note:  by default this function is quite fascist about universal
     coding systems.  It only admits `utf-8', `iso-2022-7', and
     `ctext'.  Customize `unity-approved-ucs-list' to change this.

     This function remaps characters that are artificially
     distinguished by Mule internal code.  It may change the code point
     as well as the character set.  To recode characters that were
     decoded in the wrong coding system, use `unity-recode-region'.

 -- Function: unity-recode-region begin end wrong-cs right-cs
     Recode characters between BEGIN and END from WRONG-CS to RIGHT-CS.

     WRONG-CS and RIGHT-CS are character sets.  Characters retain the
     same code point but the character set is changed.  Only characters
     from WRONG-CS are changed to RIGHT-CS.  The identity of the
     character may change.  Note that this could be dangerous, if
     characters whose identities you do not want changed are included
     in the region.  This function cannot guess which characters you
     want changed, and which should be left alone.

     When called interactively, BEGIN and END are set to the beginning
     and end, respectively, of the active region, and the function
     prompts for WRONG-CS and RIGHT-CS.  The function does completion,
     knows how to guess a character set name from a coding system name,
     and also provides some common aliases.  See `unity-guess-charset'.

     Another way to accomplish this, but using coding systems rather
     than character sets to specify the desired recoding, is
     `unity-recode-coding-region'.  That function may be faster but is
     somewhat more dangerous, because it may recode more than one
     character set.

     To change from one Mule representation to another without changing
     identity of any characters, use `unity-remap-region'.

 -- Function: unity-recode-coding-region begin end wrong-cs right-cs
     Recode text between BEGIN and END from WRONG-CS to RIGHT-CS.

     WRONG-CS and RIGHT-CS are coding systems.  Characters retain the
     same code point but the character set is changed.  The identity of
     characters may change.  This is an inherently dangerous function;
     multilingual text may be recoded in unexpected ways.  #### It's
     also dangerous because the coding systems are not sanity-checked
     in the current implementation.

     When called interactively, BEGIN and END are set to the beginning
     and end, respectively, of the active region, and the function
     prompts for WRONG-CS and RIGHT-CS.  The function does completion,
     knows how to guess a coding system name from a character set name,
     and also provides some common aliases.  See
     `unity-guess-coding-system'.

     Another, safer, way to accomplish this, using character sets rather
     than coding systems to specify the desired recoding, is to use
     `unity-recode-region'.

     To change from one Mule representation to another without changing
     identity of any characters, use `unity-remap-region'.

   Helper functions for input of coding system and character set names.

 -- Function: unity-guess-charset candidate
     Guess a charset based on the symbol CANDIDATE.

     CANDIDATE itself is not tried as the value.

     Uses the natural mapping in `unity-cset-codesys-alist', and the
     values in `unity-charset-alias-alist'."

 -- Function: unity-guess-coding-system candidate
     Guess a coding system based on the symbol CANDIDATE.

     CANDIDATE itself is not tried as the value.

     Uses the natural mapping in `unity-cset-codesys-alist', and the
     values in `unity-coding-system-alias-alist'."

 -- Function: unity-example
     A cheesy example for Unification.

     At present it just makes a multilingual buffer.  To test, setq
     buffer-file-coding-system to some value, make the buffer dirty (eg
     with RET BackSpace), and save.


File: lispref.info,  Node: Configuration,  Next: Theory of Operation,  Prev: Usage,  Up: Charset Unification

63.12.1 Configuring Unification for Use
---------------------------------------

If you want Unification to be automatically initialized, invoke
`enable-unification' with no arguments in your init file.  *Note Init
File: (xemacs)Init File.  If you are using GNU Emacs or an XEmacs
earlier than 21.1, you should also load `auto-autoloads' using the full
path (_never_ `require' `auto-autoloads' libraries).

   You may wish to define aliases for commonly used character sets and
coding systems for convenience in input.

 -- User Option: unity-charset-alias-alist
     Alist mapping aliases to Mule charset names (symbols)."

     The default value is
             ((latin-1 . latin-iso8859-1)
              (latin-2 . latin-iso8859-2)
              (latin-3 . latin-iso8859-3)
              (latin-4 . latin-iso8859-4)
              (latin-5 . latin-iso8859-9)
              (latin-9 . latin-iso8859-15)
              (latin-10 . latin-iso8859-16))

     If a charset does not exist on your system, it will not complete
     and you will not be able to enter it in response to prompts.  A
     real charset with the same name as an alias in this list will
     shadow the alias.

 -- User Option: unity-coding-system-alias-alist nil
     Alist mapping aliases to Mule coding system names (symbols).

     The default value is `nil'.


File: lispref.info,  Node: Theory of Operation,  Next: What Unification Cannot Do for You,  Prev: Configuration,  Up: Charset Unification

63.12.2 Theory of Operation
---------------------------

Standard encodings suffer from the design defect that they do not
provide a reliable way to recognize which coded character sets in use.
*Note What Unification Cannot Do for You::.  There are scores of
character sets which can be represented by a single octet (8-bit byte),
whose union contains many hundreds of characters.  Obviously this
results in great confusion, since you can't tell the players without a
scorecard, and there is no scorecard.

   There are two ways to solve this problem.  The first is to create a
universal coded character set.  This is the concept behind Unicode.
However, there have been satisfactory (nearly) universal character sets
for several decades, but even today many Westerners resist using Unicode
because they consider its space requirements excessive.  On the other
hand, Asians dislike Unicode because they consider it to be incomplete.
(This is partly, but not entirely, political.)

   In any case, Unicode only solves the internal representation problem.
Many data sets will contain files in "legacy" encodings, and Unicode
does not help distinguish among them.

   The second approach is to embed information about the encodings used
in a document in its text.  This approach is taken by the ISO 2022
standard.  This would solve the problem completely from the users' of
view, except that ISO 2022 is basically not implemented at all, in the
sense that few applications or systems implement more than a small
subset of ISO 2022 functionality.  This is due to the fact that
mono-literate users object to the presence of escape sequences in their
texts (which they, with some justification, consider data corruption).
Programmers are more than willing to cater to these users, since
implementing ISO 2022 is a painstaking task.

   In fact, Emacs/Mule adopts both of these approaches.  Internally it
uses a universal character set, "Mule code".  Externally it uses ISO
2022 techniques both to save files in forms robust to encoding issues,
and as hints when attempting to "guess" an unknown encoding.  However,
Mule suffers from a design defect, namely it embeds the character set
information that ISO 2022 attaches to runs of characters by introducing
them with a control sequence in each character.  That causes Mule to
consider the ISO Latin character sets to be disjoint.  This manifests
itself when a user enters characters using input methods associated with
different coded character sets into a single buffer.

   There are two problems stemming from this design.  First, Mule
represents the same character in different ways.  Abstractly, 'ó'
(LATIN SMALL LETTER O WITH ACUTE) can get represented as
[latin-iso8859-1 #x73] or as [latin-iso8859-2 #x73].  So what looks like
'óó' in the display might actually be represented [latin-iso8859-1
#x73][latin-iso8859-2 #x73] in the buffer, and saved as [#xF3 ESC - B
#xF3 ESC - A] in the file.  In some cases this treatment would be
appropriate (consider HYPHEN, MINUS SIGN, EN DASH, EM DASH, and U+4E00
(the CJK ideographic character meaning "one")), and although arguably
incorrect it is convenient when mixing the CJK scripts.  But in the case
of the Latin scripts this is wrong.

   Worse yet, it is very likely to occur when mixing "different"
encodings (such as ISO 8859/1 and ISO 8859/15) that differ only in a
few code points that are almost never used.  A very important example
involves email.  Many sites, especially in the U.S., default to use of
the ISO 8859/1 coded character set (also called "Latin 1," though these
are somewhat different concepts).  However, ISO 8859/1 provides a
generic CURRENCY SIGN character.  Now that the Euro has become the
official currency of most countries in Europe, this is unsatisfactory
(and in practice, useless).  So Europeans generally use ISO 8859/15,
which is nearly identical to ISO 8859/1 for most languages, except that
it substitutes EURO SIGN for CURRENCY SIGN.

   Suppose a European user yanks text from a post encoded in ISO 8859/1
into a message composition buffer, and enters some text including the
Euro sign.  Then Mule will consider the buffer to contain both ISO
8859/1 and ISO 8859/15 text, and MUAs such as Gnus will (if naively
programmed) send the message as a multipart mixed MIME body!

   This is clearly stupid.  What is not as obvious is that, just as any
European can include American English in their text because ASCII is a
subset of ISO 8859/15, most European languages which use Latin
characters (eg, German and Polish) can typically be mixed while using
only one Latin coded character set (in the case of German and Polish,
ISO 8859/2).  However, this often depends on exactly what text is to be
encoded (even for the same pair of languages).

   Unification works around the problem by converting as many
characters as possible to use a single Latin coded character set before
saving the buffer.

   Because the problem is rarely noticable in editing a buffer, but
tends to manifest when that buffer is exported to a file or process, the
Unification package uses the strategy of examining the buffer prior to
export.  If use of multiple Latin coded character sets is detected,
Unification attempts to unify them by finding a single coded character
set which contains all of the Latin characters in the buffer.

   The primary purpose of Unification is to fix the problem by giving
the user the choice to change the representation of all characters to
one character set and give sensible recommendations based on context.
In the 'ó' example, either ISO 8859/1 or ISO 8859/2 is satisfactory, and
both will be suggested.  In the EURO SIGN example, only ISO 8859/15
makes sense, and that is what will be recommended.  In both cases, the
user will be reminded that there are universal encodings available.

   I call this "remapping" (from the universal character set to a
particular ISO 8859 coded character set).  It is mere accident that this
letter has the same code point in both character sets.  (Not entirely,
but there are many examples of Latin characters that have different code
points in different Latin-X sets.)

   Note that, in the 'ó' example, that treating the buffer in this way
will result in a representation such as [latin-iso8859-2
#x73][latin-iso8859-2 #x73], and the file will be saved as [#xF3 #xF3].
This is guaranteed to occasionally result in the second problem you
observed, to which we now turn.

   This problem is that, although the file is intended to be an
ISO-8859/2-encoded file, in an ISO 8859/1 locale Mule (and every POSIX
compliant program--this is required by the standard, obvious if you
think a bit, *note What Unification Cannot Do for You::) will read that
file as [latin-iso8859-1 #x73] [latin-iso8859-1 #x73].  Of course this
is no problem if all of the characters in the file are contained in ISO
8859/1, but suppose there are some which are not, but are contained in
the (intended) ISO 8859/2.

   You now want to fix this, but not by finding the same character in
another set.  Instead, you want to simply change the character set that
Mule associates with that buffer position without changing the code.
(This is conceptually somewhat distinct from the first problem, and
logically ought to be handled in the code that defines coding systems.
However, unification is not an unreasonable place for it.)  Unification
provides two functions (one fast and dangerous, the other slow and
careful) to handle this.  I call this "recoding", because the
transformation actually involves _encoding_ the buffer to file
representation, then _decoding_ it to buffer representation (in a
different character set).  This cannot be done automatically because
Mule can have no idea what the correct encoding is--after all, it
already gave you its best guess.  *Note What Unification Cannot Do for
You::.  So these functions must be invoked by the user.  *Note
Interactive Usage::.

