bytestring-0.9.1.10: Fast, packed, strict and lazy byte arrays with a list interface

Portability | portable
Stability | experimental
Maintainer | dons@galois.com

Data.ByteString.Lazy

Contents

- The `ByteString` type
- Introducing and eliminating `ByteString`s
- Basic interface
- Transforming ByteStrings
- Reducing `ByteString`s (folds)
  - Special folds
- Building ByteStrings
  - Scans
  - Accumulating maps
  - Infinite ByteStrings
  - Unfolding ByteStrings
- Substrings
  - Breaking strings
  - Breaking into many substrings
- Predicates
  - Search for arbitrary substrings
- Searching ByteStrings
  - Searching by equality
  - Searching with a predicate
- Indexing ByteStrings
- Zipping and unzipping ByteStrings
- Ordered ByteStrings
- Low level conversions
  - Copying ByteStrings
- I/O with `ByteString`s
  - Standard input and output
  - Files
  - I/O with Handles

Description

A time and space-efficient implementation of lazy byte vectors using lists of packed `Word8` arrays, suitable for high performance use, both in terms of large data quantities, or high speed requirements. Byte vectors are encoded as lazy lists of strict `Word8` arrays of bytes. They provide a means to manipulate large byte vectors without requiring the entire vector be resident in memory.

Some operations, such as concat, append, reverse and cons, have better complexity than their Data.ByteString equivalents, due to optimisations resulting from the list spine structure. And for other operations lazy ByteStrings are usually within a few percent of strict ones, but with better heap usage. For data larger than the available memory, or if you have tight memory constraints, this module will be the only option. The default chunk size is 64k, which should be good in most circumstances. For people with large L2 caches, you may want to increase this to fit your cache.

This module is intended to be imported `qualified`, to avoid name clashes with Prelude functions. eg.

     import qualified Data.ByteString.Lazy as B

Original GHC implementation by Bryan O'Sullivan. Rewritten to use `Data.Array.Unboxed.UArray` by Simon Marlow. Rewritten to support slices and use `Foreign.ForeignPtr.ForeignPtr` by David Roundy. Polished and extended by Don Stewart. Lazy variant by Duncan Coutts and Don Stewart.

Synopsis

- data ByteString
- empty :: ByteString
- singleton :: Word8 -> ByteString
- pack :: [Word8] -> ByteString
- unpack :: ByteString -> [Word8]
- fromChunks :: [ByteString] -> ByteString
- toChunks :: ByteString -> [ByteString]
- cons :: Word8 -> ByteString -> ByteString
- cons' :: Word8 -> ByteString -> ByteString
- snoc :: ByteString -> Word8 -> ByteString
- append :: ByteString -> ByteString -> ByteString
- head :: ByteString -> Word8
- uncons :: ByteString -> Maybe (Word8, ByteString)
- last :: ByteString -> Word8
- tail :: ByteString -> ByteString
- init :: ByteString -> ByteString
- null :: ByteString -> Bool
- length :: ByteString -> Int64
- map :: (Word8 -> Word8) -> ByteString -> ByteString
- reverse :: ByteString -> ByteString
- intersperse :: Word8 -> ByteString -> ByteString
- intercalate :: ByteString -> [ByteString] -> ByteString
- transpose :: [ByteString] -> [ByteString]
- foldl :: (a -> Word8 -> a) -> a -> ByteString -> a
- foldl' :: (a -> Word8 -> a) -> a -> ByteString -> a
- foldl1 :: (Word8 -> Word8 -> Word8) -> ByteString -> Word8
- foldl1' :: (Word8 -> Word8 -> Word8) -> ByteString -> Word8
- foldr :: (Word8 -> a -> a) -> a -> ByteString -> a
- foldr1 :: (Word8 -> Word8 -> Word8) -> ByteString -> Word8
- concat :: [ByteString] -> ByteString
- concatMap :: (Word8 -> ByteString) -> ByteString -> ByteString
- any :: (Word8 -> Bool) -> ByteString -> Bool
- all :: (Word8 -> Bool) -> ByteString -> Bool
- maximum :: ByteString -> Word8
- minimum :: ByteString -> Word8
- scanl :: (Word8 -> Word8 -> Word8) -> Word8 -> ByteString -> ByteString
- mapAccumL :: (acc -> Word8 -> (acc, Word8)) -> acc -> ByteString -> (acc, ByteString)
- mapAccumR :: (acc -> Word8 -> (acc, Word8)) -> acc -> ByteString -> (acc, ByteString)
- repeat :: Word8 -> ByteString
- replicate :: Int64 -> Word8 -> ByteString
- cycle :: ByteString -> ByteString
- iterate :: (Word8 -> Word8) -> Word8 -> ByteString
- unfoldr :: (a -> Maybe (Word8, a)) -> a -> ByteString
- take :: Int64 -> ByteString -> ByteString
- drop :: Int64 -> ByteString -> ByteString
- splitAt :: Int64 -> ByteString -> (ByteString, ByteString)
- takeWhile :: (Word8 -> Bool) -> ByteString -> ByteString
- dropWhile :: (Word8 -> Bool) -> ByteString -> ByteString
- span :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)
- break :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)
- group :: ByteString -> [ByteString]
- groupBy :: (Word8 -> Word8 -> Bool) -> ByteString -> [ByteString]
- inits :: ByteString -> [ByteString]
- tails :: ByteString -> [ByteString]
- split :: Word8 -> ByteString -> [ByteString]
- splitWith :: (Word8 -> Bool) -> ByteString -> [ByteString]
- isPrefixOf :: ByteString -> ByteString -> Bool
- isSuffixOf :: ByteString -> ByteString -> Bool
- elem :: Word8 -> ByteString -> Bool
- notElem :: Word8 -> ByteString -> Bool
- find :: (Word8 -> Bool) -> ByteString -> Maybe Word8
- filter :: (Word8 -> Bool) -> ByteString -> ByteString
- partition :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)
- index :: ByteString -> Int64 -> Word8
- elemIndex :: Word8 -> ByteString -> Maybe Int64
- elemIndices :: Word8 -> ByteString -> [Int64]
- findIndex :: (Word8 -> Bool) -> ByteString -> Maybe Int64
- findIndices :: (Word8 -> Bool) -> ByteString -> [Int64]
- count :: Word8 -> ByteString -> Int64
- zip :: ByteString -> ByteString -> [(Word8, Word8)]
- zipWith :: (Word8 -> Word8 -> a) -> ByteString -> ByteString -> [a]
- unzip :: [(Word8, Word8)] -> (ByteString, ByteString)
- copy :: ByteString -> ByteString
- getContents :: IO ByteString
- putStr :: ByteString -> IO ()
- putStrLn :: ByteString -> IO ()
- interact :: (ByteString -> ByteString) -> IO ()
- readFile :: FilePath -> IO ByteString
- writeFile :: FilePath -> ByteString -> IO ()
- appendFile :: FilePath -> ByteString -> IO ()
- hGetContents :: Handle -> IO ByteString
- hGet :: Handle -> Int -> IO ByteString
- hGetNonBlocking :: Handle -> Int -> IO ByteString
- hPut :: Handle -> ByteString -> IO ()
- hPutStr :: Handle -> ByteString -> IO ()

# The `ByteString` type

data ByteString Source

A space-efficient representation of a Word8 vector, supporting many efficient operations. A `ByteString` contains 8-bit characters only.

Instances of Eq, Ord, Read, Show, Data, Typeable

Instances

Eq ByteString |
Data ByteString |
Ord ByteString |
Read ByteString |
Show ByteString |
Typeable ByteString |
IsString ByteString |
Monoid ByteString |

# Introducing and eliminating `ByteString`s

empty :: ByteString Source

O(1) The empty `ByteString`

singleton :: Word8 -> ByteString Source

O(1) Convert a `Word8` into a `ByteString`

pack :: [Word8] -> ByteString Source

O(n) Convert a '[Word8]' into a `ByteString`.

unpack :: ByteString -> [Word8]Source

O(n) Converts a `ByteString` to a '[Word8]'.

fromChunks :: [ByteString] -> ByteString Source

O(c) Convert a list of strict `ByteString` into a lazy `ByteString`

toChunks :: ByteString -> [ByteString]Source

O(n) Convert a lazy `ByteString` into a list of strict `ByteString`

# Basic interface

cons :: Word8 -> ByteString -> ByteString Source

O(1) `cons` is analogous to '(:)' for lists.

cons' :: Word8 -> ByteString -> ByteString Source

O(1) Unlike `cons`, 'cons\'' is strict in the ByteString that we are consing onto. More precisely, it forces the head and the first chunk. It does this because, for space efficiency, it may coalesce the new byte onto the first 'chunk' rather than starting a new 'chunk'.

So that means you can't use a lazy recursive contruction like this:

     let xs = cons\' c xs in xs

You can however use `cons`, as well as `repeat` and `cycle`, to build infinite lazy ByteStrings.

snoc :: ByteString -> Word8 -> ByteString Source

O(n\c)/ Append a byte to the end of a `ByteString`

append :: ByteString -> ByteString -> ByteString Source

O(n\c)/ Append two ByteStrings

head :: ByteString -> Word8 Source

O(1) Extract the first element of a ByteString, which must be non-empty.

uncons :: ByteString -> Maybe (Word8, ByteString)Source

O(1) Extract the head and tail of a ByteString, returning Nothing if it is empty.

last :: ByteString -> Word8 Source

O(n\c)/ Extract the last element of a ByteString, which must be finite and non-empty.

tail :: ByteString -> ByteString Source

O(1) Extract the elements after the head of a ByteString, which must be non-empty.

init :: ByteString -> ByteString Source

O(n\c)/ Return all the elements of a `ByteString` except the last one.

null :: ByteString -> Bool Source

O(1) Test whether a ByteString is empty.

length :: ByteString -> Int64 Source

O(n\c)/ `length` returns the length of a ByteString as an `Int64`

# Transforming ByteStrings

map :: (Word8 -> Word8) -> ByteString -> ByteString Source

O(n) `map` `f xs` is the ByteString obtained by applying `f` to each element of `xs`.

reverse :: ByteString -> ByteString Source

O(n) `reverse` `xs` returns the elements of `xs` in reverse order.

intersperse :: Word8 -> ByteString -> ByteString Source

The `intersperse` function takes a `Word8` and a `ByteString` and `intersperses' that byte between the elements of the `ByteString`. It is analogous to the intersperse function on Lists.

intercalate :: ByteString -> [ByteString] -> ByteString Source

O(n) The `intercalate` function takes a `ByteString` and a list of `ByteString`s and concatenates the list after interspersing the first argument between each element of the list.

transpose :: [ByteString] -> [ByteString]Source

The `transpose` function transposes the rows and columns of its `ByteString` argument.

# Reducing `ByteString`s (folds)

foldl :: (a -> Word8 -> a) -> a -> ByteString -> aSource

`foldl`, applied to a binary operator, a starting value (typically the left-identity of the operator), and a ByteString, reduces the ByteString using the binary operator, from left to right.

foldl' :: (a -> Word8 -> a) -> a -> ByteString -> aSource

'foldl\'' is like `foldl`, but strict in the accumulator.

foldl1 :: (Word8 -> Word8 -> Word8) -> ByteString -> Word8 Source

`foldl1` is a variant of `foldl` that has no starting value argument, and thus must be applied to non-empty `ByteStrings`. This function is subject to array fusion.

foldl1' :: (Word8 -> Word8 -> Word8) -> ByteString -> Word8 Source

'foldl1\'' is like `foldl1`, but strict in the accumulator.

foldr :: (Word8 -> a -> a) -> a -> ByteString -> aSource

`foldr`, applied to a binary operator, a starting value (typically the right-identity of the operator), and a ByteString, reduces the ByteString using the binary operator, from right to left.

foldr1 :: (Word8 -> Word8 -> Word8) -> ByteString -> Word8 Source

`foldr1` is a variant of `foldr` that has no starting value argument, and thus must be applied to non-empty `ByteString`s

## Special folds

concat :: [ByteString] -> ByteString Source

O(n) Concatenate a list of ByteStrings.

concatMap :: (Word8 -> ByteString) -> ByteString -> ByteString Source

Map a function over a `ByteString` and concatenate the results

any :: (Word8 -> Bool) -> ByteString -> Bool Source

O(n) Applied to a predicate and a ByteString, `any` determines if any element of the `ByteString` satisfies the predicate.

all :: (Word8 -> Bool) -> ByteString -> Bool Source

O(n) Applied to a predicate and a `ByteString`, `all` determines if all elements of the `ByteString` satisfy the predicate.

maximum :: ByteString -> Word8 Source

O(n) `maximum` returns the maximum value from a `ByteString`

minimum :: ByteString -> Word8 Source

O(n) `minimum` returns the minimum value from a `ByteString`

# Building ByteStrings

## Scans

scanl :: (Word8 -> Word8 -> Word8) -> Word8 -> ByteString -> ByteString Source

`scanl` is similar to `foldl`, but returns a list of successive reduced values from the left. This function will fuse.

     scanl f z [x1, x2, ...] == [z, z `f` x1, (z `f` x1) `f` x2, ...]

Note that

     last (scanl f z xs) == foldl f z xs.

## Accumulating maps

mapAccumL :: (acc -> Word8 -> (acc, Word8)) -> acc -> ByteString -> (acc, ByteString)Source

The `mapAccumL` function behaves like a combination of `map` and `foldl`; it applies a function to each element of a ByteString, passing an accumulating parameter from left to right, and returning a final value of this accumulator together with the new ByteString.

mapAccumR :: (acc -> Word8 -> (acc, Word8)) -> acc -> ByteString -> (acc, ByteString)Source

The `mapAccumR` function behaves like a combination of `map` and `foldr`; it applies a function to each element of a ByteString, passing an accumulating parameter from right to left, and returning a final value of this accumulator together with the new ByteString.

## Infinite ByteStrings

repeat :: Word8 -> ByteString Source

``repeat` x` is an infinite ByteString, with `x` the value of every element.

replicate :: Int64 -> Word8 -> ByteString Source

O(n) ``replicate` n x` is a ByteString of length `n` with `x` the value of every element.

cycle :: ByteString -> ByteString Source

`cycle` ties a finite ByteString into a circular one, or equivalently, the infinite repetition of the original ByteString.

iterate :: (Word8 -> Word8) -> Word8 -> ByteString Source

``iterate` f x` returns an infinite ByteString of repeated applications of `f` to `x`:

     iterate f x == [x, f x, f (f x), ...]

## Unfolding ByteStrings

unfoldr :: (a -> Maybe (Word8, a)) -> a -> ByteString Source

O(n) The `unfoldr` function is analogous to the List 'unfoldr'. `unfoldr` builds a ByteString from a seed value. The function takes the element and returns `Nothing` if it is done producing the ByteString or returns `Just` `(a,b)`, in which case, `a` is a prepending to the ByteString and `b` is used as the next element in a recursive call.

# Substrings

## Breaking strings

take :: Int64 -> ByteString -> ByteString Source

O(n\c)/ `take` `n`, applied to a ByteString `xs`, returns the prefix of `xs` of length `n`, or `xs` itself if `n > `length` xs`.

drop :: Int64 -> ByteString -> ByteString Source

O(n\c)/ `drop` `n xs` returns the suffix of `xs` after the first `n` elements, or `[]` if `n > `length` xs`.

splitAt :: Int64 -> ByteString -> (ByteString, ByteString)Source

O(n\c)/ `splitAt` `n xs` is equivalent to `(`take` n xs, `drop` n xs)`.

takeWhile :: (Word8 -> Bool) -> ByteString -> ByteString Source

`takeWhile`, applied to a predicate `p` and a ByteString `xs`, returns the longest prefix (possibly empty) of `xs` of elements that satisfy `p`.

dropWhile :: (Word8 -> Bool) -> ByteString -> ByteString Source

`dropWhile` `p xs` returns the suffix remaining after `takeWhile` `p xs`.

span :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)Source

`span` `p xs` breaks the ByteString into two segments. It is equivalent to `(`takeWhile` p xs, `dropWhile` p xs)`

break :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)Source

`break` `p` is equivalent to ``span` (`not` . p)`.

group :: ByteString -> [ByteString]Source

The `group` function takes a ByteString and returns a list of ByteStrings such that the concatenation of the result is equal to the argument. Moreover, each sublist in the result contains only equal elements. For example,

     group "Mississippi" = ["M","i","ss","i","ss","i","pp","i"]

It is a special case of `groupBy`, which allows the programmer to supply their own equality test.

groupBy :: (Word8 -> Word8 -> Bool) -> ByteString -> [ByteString]Source

The `groupBy` function is the non-overloaded version of `group`.

inits :: ByteString -> [ByteString]Source

O(n) Return all initial segments of the given `ByteString`, shortest first.

tails :: ByteString -> [ByteString]Source

O(n) Return all final segments of the given `ByteString`, longest first.

## Breaking into many substrings

split :: Word8 -> ByteString -> [ByteString]Source

O(n) Break a `ByteString` into pieces separated by the byte argument, consuming the delimiter. I.e.

     split '\n' "a\nb\nd\ne" == ["a","b","d","e"]
     split 'a'  "aXaXaXa"    == ["","X","X","X",""]
     split 'x'  "x"          == ["",""]

and

     intercalate [c] . split c == id
     split == splitWith . (==)

As for all splitting functions in this library, this function does not copy the substrings, it just constructs new `ByteStrings` that are slices of the original.

splitWith :: (Word8 -> Bool) -> ByteString -> [ByteString]Source

O(n) Splits a `ByteString` into components delimited by separators, where the predicate returns True for a separator element. The resulting components do not contain the separators. Two adjacent separators result in an empty component in the output. eg.

     splitWith (=='a') "aabbaca" == ["","","bb","c",""]
     splitWith (=='a') []        == []

# Predicates

isPrefixOf :: ByteString -> ByteString -> Bool Source

O(n) The `isPrefixOf` function takes two ByteStrings and returns `True` iff the first is a prefix of the second.

isSuffixOf :: ByteString -> ByteString -> Bool Source

O(n) The `isSuffixOf` function takes two ByteStrings and returns `True` iff the first is a suffix of the second.

The following holds:

     isSuffixOf x y == reverse x `isPrefixOf` reverse y

## Search for arbitrary substrings

# Searching ByteStrings

## Searching by equality

elem :: Word8 -> ByteString -> Bool Source

O(n) `elem` is the `ByteString` membership predicate.

notElem :: Word8 -> ByteString -> Bool Source

O(n) `notElem` is the inverse of `elem`

## Searching with a predicate

find :: (Word8 -> Bool) -> ByteString -> Maybe Word8 Source

O(n) The `find` function takes a predicate and a ByteString, and returns the first element in matching the predicate, or `Nothing` if there is no such element.

     find f p = case findIndex f p of Just n -> Just (p ! n) ; _ -> Nothing

filter :: (Word8 -> Bool) -> ByteString -> ByteString Source

O(n) `filter`, applied to a predicate and a ByteString, returns a ByteString containing those characters that satisfy the predicate.

partition :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)Source

O(n) The `partition` function takes a predicate a ByteString and returns the pair of ByteStrings with elements which do and do not satisfy the predicate, respectively; i.e.,

     partition p bs == (filter p xs, filter (not . p) xs)

# Indexing ByteStrings

index :: ByteString -> Int64 -> Word8 Source

O(c) `ByteString` index (subscript) operator, starting from 0.

elemIndex :: Word8 -> ByteString -> Maybe Int64 Source

O(n) The `elemIndex` function returns the index of the first element in the given `ByteString` which is equal to the query element, or `Nothing` if there is no such element. This implementation uses memchr(3).

elemIndices :: Word8 -> ByteString -> [Int64]Source

O(n) The `elemIndices` function extends `elemIndex`, by returning the indices of all elements equal to the query element, in ascending order. This implementation uses memchr(3).

findIndex :: (Word8 -> Bool) -> ByteString -> Maybe Int64 Source

The `findIndex` function takes a predicate and a `ByteString` and returns the index of the first element in the ByteString satisfying the predicate.

findIndices :: (Word8 -> Bool) -> ByteString -> [Int64]Source

The `findIndices` function extends `findIndex`, by returning the indices of all elements satisfying the predicate, in ascending order.

count :: Word8 -> ByteString -> Int64 Source

count returns the number of times its argument appears in the ByteString

     count = length . elemIndices

But more efficiently than using length on the intermediate list.

# Zipping and unzipping ByteStrings

zip :: ByteString -> ByteString -> [(Word8, Word8)]Source

O(n) `zip` takes two ByteStrings and returns a list of corresponding pairs of bytes. If one input ByteString is short, excess elements of the longer ByteString are discarded. This is equivalent to a pair of `unpack` operations.

zipWith :: (Word8 -> Word8 -> a) -> ByteString -> ByteString -> [a]Source

`zipWith` generalises `zip` by zipping with the function given as the first argument, instead of a tupling function. For example, ``zipWith` (+)` is applied to two ByteStrings to produce the list of corresponding sums.

unzip :: [(Word8, Word8)] -> (ByteString, ByteString)Source

O(n) `unzip` transforms a list of pairs of bytes into a pair of ByteStrings. Note that this performs two `pack` operations.

# Ordered ByteStrings

# Low level conversions

## Copying ByteStrings

copy :: ByteString -> ByteString Source

O(n) Make a copy of the `ByteString` with its own storage. This is mainly useful to allow the rest of the data pointed to by the `ByteString` to be garbage collected, for example if a large string has been read in, and only a small part of it is needed in the rest of the program.

# I/O with `ByteString`s

## Standard input and output

getContents :: IO ByteString Source

getContents. Equivalent to hGetContents stdin. Will read lazily

putStr :: ByteString -> IO () Source

Write a ByteString to stdout

putStrLn :: ByteString -> IO () Source

Write a ByteString to stdout, appending a newline byte

interact :: (ByteString -> ByteString) -> IO () Source

The interact function takes a function of type `ByteString -> ByteString` as its argument. The entire input from the standard input device is passed to this function as its argument, and the resulting string is output on the standard output device.

## Files

readFile :: FilePath -> IO ByteString Source

Read an entire file lazily into a `ByteString`. The Handle will be held open until EOF is encountered.

writeFile :: FilePath -> ByteString -> IO () Source

Write a `ByteString` to a file.

appendFile :: FilePath -> ByteString -> IO () Source

Append a `ByteString` to a file.

## I/O with Handles

hGetContents :: Handle -> IO ByteString Source

Read entire handle contents lazily into a `ByteString`. Chunks are read on demand, using the default chunk size.

Once EOF is encountered, the Handle is closed.

Note: the `Handle` should be placed in binary mode with `System.IO.hSetBinaryMode` for `hGetContents` to work correctly.

hGet :: Handle -> Int -> IO ByteString Source

Read `n` bytes into a `ByteString`, directly from the specified `Handle`.

hGetNonBlocking :: Handle -> Int -> IO ByteString Source

hGetNonBlocking is similar to `hGet`, except that it will never block waiting for data to become available, instead it returns only whatever data is available.

hPut :: Handle -> ByteString -> IO () Source

Outputs a `ByteString` to the specified `Handle`.

hPutStr :: Handle -> ByteString -> IO () Source

A synonym for `hPut`, for compatibility