README file for the Heirloom Toolchest
======================================

For a description what the Heirloom Toolchest is, see the file
intro.txt, the online introduction, or, if the tools are already
installed, the corresponding manual page intro(1). Note that the
path names mentioned in these documents may differ from those you
select when building the tools as described below. It is assumed
that you know the contents of the introduction as you read on.

For licensing terms, see the files in the LICENSE subdirectory.

How to install
==============

The Heirloom Toolchest principally compiles on any system that provides
the functionality of The Open Group's Single Unix Specification, Version
3, 2001, Base Definitions and Headers, and System Interfaces, (i.e. IEEE
Std. 1003.1-2001 with XSI extensions), and additionally provides some
common System V functions such as memalign() or getdents(). Its build
system needs just basic Unix functionality, a make command that is IEEE
Std 1003.2-1992 (POSIX.2) compliant, flex and any yacc implementation.

System environments that are known to support the Heirloom Toolchest are
Linux with glibc 2.1 and above and GNU make (the development is mainly
done with glibc 2.3), Solaris 8 and 9, and Open UNIX 8 (both the native
and the LKP environment), and FreeBSD 5.2.1. The make command of Solaris
doesn't get the dependencies right and the native make commands of
Open UNIX and FreeBSD apparently lack support for '+' at the beginning
of shell command lines, even though this feature is in POSIX.2, so you
might want to use GNU make on these systems, too.

It is strongly recommended that you use the rpm command to build and
install the Heirloom Toolchest on Linux. Simply execute

        rpmbuild -tb heirloom-<version>.tar.bz2

wait a few minutes and install the resulting rpm that is found in
/usr/src/redhat/RPMS/i386/heirloom-<version>-1.i386.rpm (or wherever
your vendor's setup puts such files). Then access the tools by prefixing
your PATH with /usr/5bin and skip the rest of this section. (With older
versions of rpm you need to use 'rpm' instead of 'rpmbuild'.)

On Solaris and Open UNIX, you can build a package for pkgadd from
the sources. Extract them, change to the heirloom-<version> directory,
and execute

        gmake pkg

This will create a file heirloom.pkg which you can then give to pkgadd.
On Solaris, you will have to delete the existing symbolic link /usr/5bin
to install it.

If your system lacks rpm or pkgadd or if you want to make changes to
the Heirloom Toolchest, you have to build it manually. The first thing
to understand is the build system. This is actually quite simple: Every
directory contains a file named Makefile.mk that includes the directory-
specific make instructions. To generate the real Makefile, configuration
settings are prepended to the directory-specific file. You have to edit
these configuration settings before you start compiling; they are
located in the file build/mk.config which is also in make syntax.
Follow the descriptions in this file and select appropriate values
for your system.

Be very careful at this step! If you overwrite your system utilities
or put the Heirloom Toolchest at an early place in your system's PATH,
some of your shell scripts or of those of your vendor may cease to
work. This is because utilities from the Heirloom Toolchest may behave
differently than the ones supplied by your vendor. You may wish to
correct this later, but the first build is definitively not the right
time. - If you want to stay with the default /usr/5bin path on Solaris,
be sure to remove the existing symbolic link to /usr/bin.

If you don't know what to do here, get a competent person to assist
you. I won't assist at this point and do, as described in the license
terms, not warrant for any problems caused by anyone. You have been
warned.

After finishing the configuration, type 'make' and 'make install',
then use the tools found at the location you selected before.

The Heirloom Toolchest can also be compiled using dietlibc
<http://www.fefe.de/dietlibc/>. This will result in relatively small
statically linked executables; the tools will not have multibyte
functionality, though. Note that you have to compile ncurses, optionally
also zlib and libbz2, and to install the libraries in an appropriate
location for 'diet' before you can start to build the Heirloom tools.

Porting the Heirloom Toolchest
==============================

The Heirloom Toolchest definitively needs much of the functionality
described in SUSv3, most prominently a lot of advanced wide-character
routines. If your system lacks these, don't simply use the preprocessor
to deactivate them. Instead, write emulation calls, as it has been done
in 'libwchar' and parts of 'libcommon'.

If your system supports SUSv3 and the tools just require a few minor
changes to work, send me your patches; if they are well programmed and
unobtrusive, I will very likely accept them. The same goes for support
for inherently system-dependent things such as /proc.

But note that I will not accept any changes for non-Unix environments
such as MacOS X with its case-insensitive default file system, or for
even worse Unix emulations on Windows or the like.

Hacking the Heirloom Toolchest
==============================

One of the considerations the Heirloom Toolchest is designed along
is accessibility for other developers. Each utility was made largely
independent from other ones, to the extent of duplicate functionality
at some places. There are just three libraries you have to know about:

1. The 'common' library, found in libcommon, provides the following
   functionality:

   - Traditional ('simple') regular expressions, available as full
     source to #include in the old-style ed-hack regexp.h and as normal
     library functions in regexpr.h. You should definitively use the
     latter if you want to use simple REs in new applications.

   - Traditional file name matching; manually include the prototype
     'extern int gmatch(const char *, const char *);' if you want
     to use it.

   - SVID4-style error reporting; #include <pfmt.h>.

   - Locale-independent character class functions in <asciitype.h>.
     These are necessary e.g. to recognize programming language syntax
     white space if LC_CTYPE is used otherwise (if you do not want to
     handle no-break-space characters etc. as word delimiters).

   - Functions for reading directory entries with a file descriptor
     obtained from open(), in <getdir.h>. These are only useful if
     you descend a directory hierarchy with fchdir() and have to care
     about the limit of open file descriptors, as the 'file' and 'rm'
     tools; otherwise, stay with the standard readdir().

   - A getopt() that has the traditional functionality, and just the
     traditional functionality. This is here mainly because GNU getopt()
     is so annoyingly non-standard.

   - Functions for buffered input and output in <iblok.h> and <oblok.h>,
     respectively. They can be used as a replacement for stdio in simple
     cases; the main reason for their existence is the desire to read
     lines of arbitrary length without the slow-down caused by getc().

2. A modified version of Caldera's 'UNIX(R) Regular Expression Library'
   in the libuxre directory. This is used for (mostly) POSIX.2-compatible
   regular expression matching. It provides a syntax without too much
   extension fuss, and good performance, since it includes both a DFA
   and a NFA style matcher and uses either of them depending on the
   expression and the need for subexpression locations.

3. The widechar portability library in the libwchar directory. This is
   only used for system environments that do not provide wide character
   functionality in their own libc. It currently just does simple
   copying between bytes and wide characters and thus does not handle
   anything than plain ASCII well. If other wide character functions
   than those currently present are used in new code, it will be
   necessary to add those function to libwchar as well.

I make no premises about keeping the interfaces of these libraries
compatible in upcoming releases, so if you want to use them in other
projects, just fork off your own version and don't rely on mine.

To go on, you now only need to know about the sources in the directory
of the utility you want to hack. This should make it not too difficult,
even though I've generally stayed with good old Unix programming style
(few comments, short variable names - but not too much to read either).

Extending the Heirloom Toolchest
================================

If you want to add an extension to a utility, just go on and do it - as
you've read in the section above, accessibility for hacking is intended
to be easy. If you want to have it included in my version, however, read
on here. The Heirloom Toolchest provides the traditional Unix utilities,
but not much beyond. And it will remain this way. While I've included a
few extensions that are in really wide use today and are justified since
it would be difficult to get their functionality by other means (such as
diff -u), I will not add extensions just for the convenience of people.
For example, the Heirloom Toolchest 'grep' command will not support the
'-r' option of GNU grep, because one can have the same result by using
'find' in addition and can put that in a small script if one deems it
too much to type. In general, chances are quite low that I will accept
your extension for my version. But that doesn't mean that you should
not do it for yourself.

Reporting bugs
==============

Before you report behavior that you suspect to be a bug, check at least
the following to be sure not to report a feature:

- Does it leave a core dump? If yes, it's certainly a bug (unless you
  sent a signal, of course).

- Read the manual page and the source for the utility carefully. Did
  you use the utility properly?

- If the utility is used within a shell script or with another user's
  environment, try how it behaves if used standalone.

- If the behavior disturbs you just because your vendor's utilities do
  something different, that alone is no reason to report it. The Heirloom
  Toolchest is oriented on System V behavior. Verify that real System V
  derivatives behave differently before reporting.

- If you are using the POSIX.2 personality and the specification says
  something different, it is likely to be a bug. But be careful to read
  the specification itself; there are quite a few incompatibilities to
  the wording of POSIX.2 even in certified systems, so you cannot rely
  on a vendor's implementation or documentation here.

If you went through these statements and you're still sure that it is
a bug, report it.

Gunnar Ritter
Freiburg i. Br.
Germany
<http://heirloom.sourceforge.net>
<Gunnar.Ritter@pluto.uni-freiburg.de>				5/2/04
