# 16 Other Language Changes

Here are all of the changes that Python 2.3 makes to the core Python language.

- The `yield` statement is now always a keyword, as described in section 2 of this document.

- A new built-in function `enumerate()` was added, as described in section 7 of this document.

- Two new constants, `True` and `False` were added along with the built-in `bool` type, as described in section 9 of this document.

- The `int()` type constructor will now return a long integer instead of raising an `OverflowError` when a string or floating-point number is too large to fit into an integer. This can lead to the paradoxical result that `isinstance(int(expression), int)` is false, but that seems unlikely to cause problems in practice.

- Built-in types now support the extended slicing syntax, as described in section 15 of this document.

- A new built-in function, `sum(iterable, start=0)`, adds up the numeric items in the iterable object and returns their sum. `sum()` only accepts numbers, meaning that you can't use it to concatenate a bunch of strings. (Contributed by Alex Martelli.)

- `list.insert(pos, value)` used to insert value at the front of the list when pos was negative. The behaviour has now been changed to be consistent with slice indexing, so when pos is -1 the value will be inserted before the last element, and so forth.

- `list.index(value)`, which searches for value within the list and returns its index, now takes optional start and stop arguments to limit the search to only part of the list.

- Dictionaries have a new method, `pop(key[, default])`, that returns the value corresponding to key and removes that key/value pair from the dictionary. If the requested key isn't present in the dictionary, default is returned if it's specified and `KeyError` raised if it isn't.

      >>> d = {1:2}
      >>> d
      {1: 2}
      >>> d.pop(4)
      Traceback (most recent call last):
        File "stdin", line 1, in ?
      KeyError: 4
      >>> d.pop(1)
      2
      >>> d.pop(1)
      Traceback (most recent call last):
        File "stdin", line 1, in ?
      KeyError: 'pop(): dictionary is empty'
      >>> d
      {}
      >>>

  There's also a new class method, `dict.fromkeys(iterable, value)`, that creates a dictionary with keys taken from the supplied iterator iterable and all values set to value, defaulting to `None`.

  (Patches contributed by Raymond Hettinger.)

  Also, the `dict()` constructor now accepts keyword arguments to simplify creating small dictionaries:

      >>> dict(red=1, blue=2, green=3, black=4)
      {'blue': 2, 'black': 4, 'green': 3, 'red': 1}

  (Contributed by Just van Rossum.)

- The `assert` statement no longer checks the `__debug__` flag, so you can no longer disable assertions by assigning to `__debug__`. Running Python with the -O switch will still generate code that doesn't execute any assertions.

- Most type objects are now callable, so you can use them to create new objects such as functions, classes, and modules. (This means that the `new` module can be deprecated in a future Python version, because you can now use the type objects available in the `types` module.) For example, you can create a new module object with the following code:

      >>> import types
      >>> m = types.ModuleType('abc','docstring')
      >>> m
      <module 'abc' (built-in)>
      >>> m.__doc__
      'docstring'

- A new warning, `PendingDeprecationWarning` was added to indicate features which are in the process of being deprecated. The warning will not be printed by default. To check for use of features that will be deprecated in the future, supply -Walways::PendingDeprecationWarning:: on the command line or use `warnings.filterwarnings()`.

- The process of deprecating string-based exceptions, as in `raise "Error occurred"`, has begun. Raising a string will now trigger `PendingDeprecationWarning`.

- Using `None` as a variable name will now result in a `SyntaxWarning` warning. In a future version of Python, `None` may finally become a keyword.

- The `xreadlines()` method of file objects, introduced in Python 2.1, is no longer necessary because files now behave as their own iterator. `xreadlines()` was originally introduced as a faster way to loop over all the lines in a file, but now you can simply write `for line in file_obj`. File objects also have a new read-only `encoding` attribute that gives the encoding used by the file; Unicode strings written to the file will be automatically converted to bytes using the given encoding.

- The method resolution order used by new-style classes has changed, though you'll only notice the difference if you have a really complicated inheritance hierarchy. Classic classes are unaffected by this change. Python 2.2 originally used a topological sort of a class's ancestors, but 2.3 now uses the C3 algorithm as described in the paper ``A Monotonic Superclass Linearization for Dylan''. To understand the motivation for this change, read Michele Simionato's article ``Python 2.3 Method Resolution Order'', or read the thread on python-dev starting with the message at http://mail.python.org/pipermail/python-dev/2002-October/029035.html. Samuele Pedroni first pointed out the problem and also implemented the fix by coding the C3 algorithm.

- Python runs multithreaded programs by switching between threads after executing N bytecodes. The default value for N has been increased from 10 to 100 bytecodes, speeding up single-threaded applications by reducing the switching overhead. Some multithreaded applications may suffer slower response time, but that's easily fixed by setting the limit back to a lower number using `sys.setcheckinterval(N)`. The limit can be retrieved with the new `sys.getcheckinterval()` function.

- One minor but far-reaching change is that the names of extension types defined by the modules included with Python now contain the module and a "`.`" in front of the type name. For example, in Python 2.2, if you created a socket and printed its `__class__`, you'd get this output:

      >>> s = socket.socket()
      >>> s.__class__
      <type 'socket'>

  In 2.3, you get this:

      >>> s.__class__
      <type '_socket.socket'>

- One of the noted incompatibilities between old- and new-style classes has been removed: you can now assign to the `__name__` and `__bases__` attributes of new-style classes. There are some restrictions on what can be assigned to `__bases__` along the lines of those relating to assigning to an instance's `__class__` attribute.

## 16.1 String Changes

- The `in` operator now works differently for strings. Previously, when evaluating `X in Y` where X and Y are strings, X could only be a single character. That's now changed; X can be a string of any length, and `X in Y` will return `True` if X is a substring of Y. If X is the empty string, the result is always `True`.

      >>> 'ab' in 'abcd'
      True
      >>> 'ad' in 'abcd'
      False
      >>> '' in 'abcd'
      True

  Note that this doesn't tell you where the substring starts; if you need that information, use the `find()` string method.

- The `strip()`, `lstrip()`, and `rstrip()` string methods now have an optional argument for specifying the characters to strip. The default is still to remove all whitespace characters:

      >>> '   abc '.strip()
      'abc'
      >>> '><><abc<><><>'.strip('<>')
      'abc'
      >>> '><><abc<><><>\n'.strip('<>')
      'abc<><><>\n'
      >>> u'\u4000\u4001abc\u4000'.strip(u'\u4000')
      u'\u4001abc'
      >>>

  (Suggested by Simon Brunning and implemented by Walter Dörwald.)

- The `startswith()` and `endswith()` string methods now accept negative numbers for the start and end parameters.

- Another new string method is `zfill()`, originally a function in the `string` module. `zfill()` pads a numeric string with zeros on the left until it's the specified width. Note that the `%` operator is still more flexible and powerful than `zfill()`.

      >>> '45'.zfill(4)
      '0045'
      >>> '12345'.zfill(4)
      '12345'
      >>> 'goofy'.zfill(6)
      '0goofy'

  (Contributed by Walter Dörwald.)

- A new type object, `basestring`, has been added. Both 8-bit strings and Unicode strings inherit from this type, so `isinstance(obj, basestring)` will return `True` for either kind of string. It's a completely abstract type, so you can't create `basestring` instances.

- Interned strings are no longer immortal and will now be garbage-collected in the usual way when the only reference to them is from the internal dictionary of interned strings. (Implemented by Oren Tirosh.)

## 16.2 Optimizations

- The creation of new-style class instances has been made much faster; they're now faster than classic classes!

- The `sort()` method of list objects has been extensively rewritten by Tim Peters, and the implementation is significantly faster.

- Multiplication of large long integers is now much faster thanks to an implementation of Karatsuba multiplication, an algorithm that scales better than the O(n*n) required for the grade-school multiplication algorithm. (Original patch by Christopher A. Craig, and significantly reworked by Tim Peters.)

- The `SET_LINENO` opcode is now gone. This may provide a small speed increase, depending on your compiler's idiosyncrasies. See section 20 for a longer explanation. (Removed by Michael Hudson.)

- `xrange()` objects now have their own iterator, making `for i in xrange(n)` slightly faster than `for i in range(n)`. (Patch by Raymond Hettinger.)

- A number of small rearrangements have been made in various hotspots to improve performance, such as inlining a function or removing some code. (Implemented mostly by GvR, but lots of people have contributed single changes.)

The net result of the 2.3 optimizations is that Python 2.3 runs the pystone benchmark around 25% faster than Python 2.2.