Utf(3)

NAME
Tcl_UniChar, Tcl_UniCharToUtf, Tcl_UtfToUniChar, Tcl_UtfCharComplete, Tcl_NumUtfChars, Tcl_UtfFindFirst, Tcl_UtfFindLast, Tcl_UtfNext, Tcl_UtfPrev, Tcl_UniCharAtIndex, Tcl_UtfAtIndex, Tcl_UtfBackslash - routines for manipulating UTF-8 strings.

SYNOPSIS
#include <tcl.h>

typedef ... Tcl_UniChar;

int
Tcl_UniCharToUtf(ch, buf)

int
Tcl_UtfToUniChar(src, chPtr)

int
Tcl_UtfCharComplete(src, len)

int
Tcl_NumUtfChars(src, len)

char *
Tcl_UtfFindFirst(src, ch)

char *
Tcl_UtfFindLast(src, ch)

char *
Tcl_UtfNext(src)

char *
Tcl_UtfPrev(src, start)

Tcl_UniChar
Tcl_UniCharAtIndex(src, index)

char *
Tcl_UtfAtIndex(src, index)

int
Tcl_UtfBackslash(src, readPtr, dst)

ARGUMENTS
Buffer in which the UTF-8 representation of the Tcl_UniChar is stored. At most TCL_UTF_MAX bytes are stored in the buffer. The Tcl_UniChar to be converted or examined. Filled with the Tcl_UniChar represented by the head of the UTF-8 string. Pointer to a UTF-8 string. The length of the UTF-8 string in bytes (not UTF-8 characters). If negative, all bytes up to the first null byte are used. Pointer to the beginning of a UTF-8 string. The index of a character (not byte) in the UTF-8 string. If non-NULL, filled with the number of bytes in the backslash sequence, including the backslash character. Buffer in which the bytes represented by the backslash sequence are stored. At most TCL_UTF_MAX bytes are stored in the buffer.

DESCRIPTION

These routines convert between UTF-8 strings and Tcl_UniChars. A Tcl_UniChar is a Unicode character represented as an unsigned, fixed-size quantity. A UTF-8 character is a Unicode character represented as a varying-length sequence of up to TCL_UTF_MAX bytes. A multibyte UTF-8 sequence consists of a lead byte followed by some number of trail bytes.

TCL_UTF_MAX is the maximum number of bytes that it takes to represent one Unicode character in the UTF-8 representation.

Tcl_UniCharToUtf stores the Tcl_UniChar ch as a UTF-8 string in starting at buf. The return value is the number of bytes stored in buf.

Tcl_UtfToUniChar reads one UTF-8 character starting at src and stores it as a Tcl_UniChar in *chPtr. The return value is the number of bytes read from src.. The caller must ensure that the source buffer is long enough such that this routine does not run off the end and dereference non-existent or random memory; if the source buffer is known to be null terminated, this will not happen. If the input is not in proper UTF-8 format, Tcl_UtfToUniChar will store the first byte of src in *chPtr as a Tcl_UniChar between 0x0000 and 0x00ff and return 1.

Tcl_UtfCharComplete returns 1 if the source UTF-8 string src of length len bytes is long enough to be decoded by Tcl_UtfToUniChar, or 0 otherwise. This function does not guarantee that the UTF-8 string is properly formed. This routine is used by procedures that are operating on a byte at a time and need to know if a full Tcl_UniChar has been seen.

Tcl_NumUtfChars corresponds to strlen for UTF-8 strings. It returns the number of Tcl_UniChars that are represented by the UTF-8 string src. The length of the source string is len bytes. If the length is negative, all bytes up to the first NULL byte are used.

Tcl_UtfFindFirst corresponds to strchr for UTF-8 strings. It returns a pointer to the first occurance of the Tcl_UniChar ch in the NULL-terminated UTF-8 string src. The NULL terminator is considered part of the UTF-8 string.

Tcl_UtfFindLast corresponds to strrchr for UTF-8 strings. It returns a pointer to the last occurance of the Tcl_UniChar ch in the NULL terminated UTF-8 string src. The NULL terminator is considered part of the UTF-8 string.

Given src, a pointer to some location in a UTF-8 string, Tcl_UtfNext returns a pointer to the next UTF-8 character in the string. The caller must not ask for the next character after the last character in the string.

Given src, a pointer to some location in a UTF-8 string, Tcl_UtfPrev returns a pointer to the previous UTF-8 character in the string. This function will not back up to a position before start, the start of the UTF-8 string. If src was already at start, the return value will be start.

Tcl_UniCharAtIndex corresponds to a C string array dereference or the Pascal Ord() function. It returns the Tcl_UniChar represented at the specified character (not byte) index in the UTF-8 string src. The source string must contain at least index characters.

Tcl_UtfAtIndex returns a pointer to the specified character (not byte) index in the UTF-8 string src. The source string must contain at least index characters. This is equivalent to calling Tcl_UtfNext index times.

Tcl_UtfBackslash is a utility procedure used by several of the Tcl commands. It parses a backslash sequence and stores the properly formed UTF-8 character represented by the backslash sequence in the output buffer dst. At most TCL_UTF_MAX bytes are stored in the buffer. Tcl_UtfBackslash modifies *readPtr to contain the number of bytes in the backslash sequence, including the backslash character. The return value is the number of bytes stored in the output buffer.

See the Tcl manual entry for information on the valid backslash sequences. All of the sequences described in the Tcl manual entry are supported by Tcl_UtfBackslash.

KEYWORDS
utf, unicode, backslash