RWCAT(1)

NAME
rwcat - Concatenate SiLK Flow files into single stream

SYNOPSIS
  rwcat [--output-path=FILE] [--note-add=TEXT] [--note-file-add=FILE]
        [--print-filenames] [--byte-order={big | little | native}]
        [--ipv4-output] [--compression-method=COMP_METHOD]
        [--site-config-file=FILENAME]
        {[--xargs] | [--xargs=FILENAME] | [ input-files ... ]}

  rwcat --help

  rwcat --version

DESCRIPTION
rwcat reads SiLK Flow records from the specified input files and writes the records in the standard binary SiLK format to the specified output-path; rwcat will write the records to the standard output when stdout is not the terminal and --output-path is not provided.

When the --xargs switch is provided, rwcat will read the names of the files to process from the named text file, or from the standard input if no file name argument is provided to the switch. The input should contain one filename per line.

If the input file names end in ".gz", they will be uncompressed as they are read. When "stdin" is provided as an input file name, rwcat will read records from the standard input.

OPTIONS
Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

--output-path=FILE
    Write the SiLK Flow records to FILE, which must not exist. If the switch is not provided or if FILE is "stdout", flows are written to the standard output. If the name ends in ".gz", the output will be compressed using gzip(1).

--note-add=TEXT
    Add the specified TEXT to the header of the output file as an annotation. This switch may be repeated to add multiple annotations to a file. To view the annotations, use the rwfileinfo(1) tool.

--note-file-add=FILENAME
    Open FILENAME and add the contents of that file to the header of the output file as an annotation. This switch may be repeated to add multiple annotations. Currently the application makes no effort to ensure that FILENAME contains text; be careful that you do not attempt to add a SiLK data file as an annotation.

--byte-order=ENDIAN
    Set the byte order for the output SiLK Flow records. The argument is one of the following:

    "native"
        Use the byte order of the machine where rwcat is running. This is the default.

    "big"
        Use network byte order (big endian) for the output.

    "little"
        Write the output in little endian format.

--ipv4-output
    Force the output to contain only IPv4 addresses. When this switch is specified, IPv6 addresses are ignored unless the IPv6 address is an encapsulation of an IPv4 address, in which case the IPv4 address will be written to the output. By default, rwcat writes IP addresses in the same format as the input file. When SiLK has not been compiled with IPv6 support, this switch has no effect.

--compression-method=COMP_METHOD
    Set the compression method of the output to COMP_METHOD. Some SiLK tools can use an external library to compress their binary output. The list of available compression methods and the default method are set when SiLK is compiled (the --help and --version switches print the available and default compression methods) and depend on which supported libraries are found. SiLK can support:

    none
        Do not compress the output using an external library

    zlib
        Use the zlib(3) library for compressing the output

    lzo1x
        Use the lzo1x algorithm from the LZO real time compression library for compression

    best
        Use whichever available method gives the "best" compression in general, though not necessarily the "best" for this particular output.

--print-filenames
    Print the names of input files and the number of records each file contains as the files are read.

--site-config-file=FILENAME
    Read the SiLK site configuration from the named file FILENAME. When this switch is not provided, the location specified by the SILK_CONFIG_FILE environment variable is used if that variable is not empty. The value of SILK_CONFIG_FILE should include the name of the file. Otherwise, the application looks for a file named silk.conf in the following directories: the directory specified in the SILK_DATA_ROOTDIR environment variable; the data root directory that is compiled into SiLK (use the --version switch to view this value); the directories $SILK_PATH/share/silk/ and $SILK_PATH/share/; and the share/silk/ and share/ directories parallel to the application's directory.

--xargs
--xargs=FILENAME
    Causes rwcat to read file names from FILENAME or from the standard input if FILENAME is not provided. The input should have one file name per line. rwcat will open each file in turn and read records from it, as if the files had been listed on the command line.

--help
    Print the available options and exit.

--version
    Print the version number and information about how SiLK was configured, then exit the application.

EXAMPLES
To combine the results of several rwfilter runs---stored in the files run1.rwf, run2.rwf, ... runN.rwf---together, you can use:

  rwcat --output=combined.dat  *.rwf

If the shell complains about too many arguments, you can use the UNIX find(1) function and pipe its output to rwcat:

  find . -name '*.rwf' -print | \
        rwcat --xargs --output=combined.dat

ENVIRONMENT

SILK_CONFIG_FILE
    This environment variable is used as the value for the --site-config-file when that switch is not provided.

SILK_DATA_ROOTDIR
    When the --site-config-file switch is not provided and the SILK_CONFIG_FILE environment variable is not set, rwcat looks for the site configuration file in $SILK_DATA_ROOTDIR/silk.conf.

SILK_PATH
    This environment variable gives the root of the install tree. As part of its search for the SiLK site configuration file, rwcat checks for a file named silk.conf in the directories $SILK_PATH/share/silk and $SILK_PATH/share.

SEE ALSO
rwfilter(1), gzip(1), find(1)

BUGS
Although rwcat will read from the standard input, this feature should be used with caution. rwcat will treat the standard input as a single file, as it has no way to know when one file ends and the next begins. The following will not work:

  cat run1.rwf run2.rwf | rwcat --output=combined.dat  # WRONG!

The header of run2.rwf will be treated as data of run1.rwf, resulting in corrupt output.