dklibs(3)

NAME
dktok - Tokenizer

SYNOPSIS
  #include <dktok.h>

  int fct(void *d, void *tpv, char *s, int *ec);

  dk_tokenizer_t *
  dktok_new(
    size_t b, char **q, char *s, char *w, char *n, char c,
    dk_fct_tokenizer *f, void *d
  );

  void dktok_start(dk_tokenizer_t *tp);

  int dktok_add(dk_tokenizer_t *tp, char c);

  int dktok_stop(dk_tokenizer_t *tp);

  int dktok_get_error_code(dk_tokenizier_t *tp);

  unsigned long dktok_get_lineno(dk_tokenizer_t *tp);

  void dktok_reset_error_code(dk_tokenizer_t *tp);

DESCRIPTION
The dktok module splits incoming data from a stream into tokens and allows to process the tokens by using a processing function.

The fct() function is an example for a processing function. The d parameter is a pointer to additional data used for processing, tpv is a pointer to a tokenizer, s is a pointer to the buffer containing the token, ec is a pointer to a variable which can be set if an error occures. The function must return 1 on success, 0 on errors. If the function returns 0 input processing is aborted.

The dktok_new() function creates a new tokenizer. The parameter b specifies the length of the largest token. The q parameter points to an array containing strings. Each string contains a bracket pair (opening and closing bracket). The array must be finished by a NULL pointer. The s parameter is a string containing all one-character-tokens (i.e. mathematical operators). The w parameter is a string containing all white-space characters. The processing function is specified as parameter f, additional data as specified in d is passed as parameter d to the processing callback function. The function returns a valid pointer on success, NULL on error. On success the tokenizer structure must be released after usage by the dktok_delete() function.

The dktok_start() function must be invoked before feeding data into the tokenizer.

The dktok_add() function feeds data into the tokenizer. If the return value is positive (>0) we can continue feeding the tokenizer.

The dktok_stop() function must be run after providing the last input to process the final token and to flush all buffers.

The dktok_get_error_code() retrieves the error code established by the processing callback function.

The dktok_get_lineno() function returns the current line number. This can be used to generate error messages.

The dktok_reset_error_code() function resets the internal error code.

AUTHOR
Dirk Krause

COPYRIGHT AND LICENSE
Copyright (c) 2001-2008, Dirk Krause All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

 * Redistributions of source code must retain the above
   copyright notice, this list of conditions and the
   following disclaimer.

 * Redistributions in binary form must reproduce the above
   copyright notice, this list of conditions and the following
   disclaimer in the documentation and/or other materials
   provided with the distribution.

 * Neither the name of the Dirk Krause nor the names of
   contributors may be used to endorse or promote products
   derived from this software without specific prior written
   permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.