# Using Threads in Twisted

## Introduction

Before you start using threads, make sure you do at the start of your program:

    from twisted.python import threadable
    threadable.init()

This will make certain parts of Twisted thread-safe so you can use them safely. However, note that most parts of Twisted are not thread-safe.

## Running code in a thread-safe manner

Most code in Twisted is not thread-safe. For example, writing data to a transport from a protocol is not thread-safe. Therefore, we want a way to schedule methods to be run in the main event loop. This can be done using the function `twisted.internet.interfaces.IReactorCore.callFromThread`:

    from twisted.internet import reactor
    from twisted.python import threadable
    threadable.init(1)

    def notThreadSafe(x):
         """do something that isn't thread-safe"""
         # ...

    def threadSafeScheduler():
        """Run in thread-safe manner."""
        reactor.callFromThread(notThreadSafe, 3) # will run 'notThreadSafe(3)' in the event loop

## Running code in threads

Sometimes we may want to run methods in threads - for example, in order to access blocking APIs. Twisted provides methods for doing so using the IReactorThreads API (`twisted.internet.interfaces.IReactorThreads`). Additional utility functions are provided in `twisted.internet.threads`. Basically, these methods allow us to queue methods to be run by a thread pool.

For example, to run a method in a thread we can do:

    from twisted.internet import reactor

    def aSillyBlockingMethod(x):
        import time
        time.sleep(2)
        print x

    # run method in thread
    reactor.callInThread(aSillyBlockingMethod, "2 seconds have passed")

## Utility Methods

The utility methods are not part of the `twisted.internet.reactor` APIs, but are implemented in `twisted.internet.threads`.

If we have multiple methods to run sequentially within a thread, we can do:

    from twisted.internet import threads

    def aSillyBlockingMethodOne(x):
        import time
        time.sleep(2)
        print x

    def aSillyBlockingMethodTwo(x):
        print x

    # run both methods sequentially in a thread
    commands = [(aSillyBlockingMethodOne, ["Calling First"], {})]
    commands.append((aSillyBlockingMethodTwo, ["And the second"], {}))
    threads.callMultipleInThread(commands)

For functions whose results we wish to get, we can have the result returned as a Deferred:

    from twisted.internet import threads

    def doLongCalculation():
        # .... do long calculation here ...
        return 3

    def printResult(x):
        print x

    # run method in thread and get result as defer.Deferred
    d = threads.deferToThread(doLongCalculation)
    d.addCallback(printResult)

## Managing the Thread Pool

The thread pool is implemented by `twisted.python.threadpool.ThreadPool`.

We may want to modify the size of the threadpool, increasing or decreasing the number of threads in use. We can do this do this quite easily:

    from twisted.internet import reactor

    reactor.suggestThreadPoolSize(20)

The size of the thread pool defaults to a maximum of 10 threads. Be careful that you understand threads and their resource usage before drastically altering the thread pool sizes.

Go Home