Invalidations while connecting
==============================

As soon as a client registers with a server, it will recieve
invalidations from the server.  The client must be careful to queue
these invalidations until it is ready to deal with them.  At the time
of the writing of this test, clients weren't careful enogh about
queing invalidations.  This led to cache corruption in the form of
both low-level file corruption as well as out-of-date records marked
as current.

This tests tries to provoke this bug by:

- starting a server

    >>> import ZEO.tests.testZEO, ZEO.tests.forker
    >>> addr = 'localhost', ZEO.tests.testZEO.get_port()
    >>> zconf = ZEO.tests.forker.ZEOConfig(addr)
    >>> sconf = '<filestorage 1>\npath Data.fs\n</filestorage>\n'
    >>> _, adminaddr, pid, conf_path = ZEO.tests.forker.start_zeo_server(
    ...     sconf, zconf, addr[1])

- opening a client to the server that writes some objects, filling
  it's cache at the same time,

    >>> import ZEO.ClientStorage, ZODB.tests.MinPO, transaction
    >>> db = ZODB.DB(ZEO.ClientStorage.ClientStorage(addr, client='x'))
    >>> conn = db.open()
    >>> nobs = 1000
    >>> for i in range(nobs):
    ...     conn.root()[i] = ZODB.tests.MinPO.MinPO(0)
    >>> transaction.commit()

- disconnecting the first client (closing it with a persistent cache),

    >>> db.close()

- starting a second client that writes objects more or less
  constantly,

    >>> import random, threading, time
    >>> stop = False
    >>> db2 = ZODB.DB(ZEO.ClientStorage.ClientStorage(addr))
    >>> tm = transaction.TransactionManager()
    >>> conn2 = db2.open(transaction_manager=tm)
    >>> random = random.Random(0)
    >>> lock = threading.Lock()
    >>> def run():
    ...     while 1:
    ...         i = random.randint(0, nobs-1)
    ...         if stop:
    ...             return
    ...         lock.acquire()
    ...         try:
    ...             conn2.root()[i].value += 1
    ...             tm.commit()
    ...         finally:
    ...             lock.release()
    ...             time.sleep(0)
    >>> thread = threading.Thread(target=run)
    >>> thread.start()

- restarting the first client, and 
- testing for cache validity.

    >>> import zope.testing.loggingsupport, logging
    >>> handler = zope.testing.loggingsupport.InstalledHandler(
    ...    'ZEO', level=logging.ERROR)

    >>> for c in range(10):
    ...    time.sleep(.1)
    ...    db = ZODB.DB(ZEO.ClientStorage.ClientStorage(addr, client='x'))
    ...    _ = lock.acquire()
    ...    try:
    ...      time.sleep(.1)
    ...      assert (db.storage.lastTransaction()
    ...              == db.storage._server.lastTransaction()), (
    ...                  db.storage.lastTransaction(),
    ...                  db.storage._server.lastTransactiion())
    ...      conn = db.open()
    ...      for i in range(1000):
    ...        if conn.root()[i].value != conn2.root()[i].value:
    ...            print 'bad', c, i, conn.root()[i].value,
    ...            print  conn2.root()[i].value
    ...    finally:
    ...      _ = lock.release()
    ...    db.close()

    >>> stop = True
    >>> thread.join(10)
    >>> thread.isAlive()
    False

    >>> for record in handler.records:
    ...     print record.name, record.levelname
    ...     print handler.format(record)

    >>> handler.uninstall()

    >>> db.close()
    >>> db2.close()
    >>> ZEO.tests.forker.shutdown_zeo_server(adminaddr)

