\section{\module{collections} ---
         High-performance container datatypes}

\declaremodule{standard}{collections}
\modulesynopsis{High-performance datatypes}
\moduleauthor{Raymond Hettinger}{python@rcn.com}
\sectionauthor{Raymond Hettinger}{python@rcn.com}
\versionadded{2.4}


This module implements high-performance container datatypes.  Currently, the
only datatype is a deque.  Future additions may include B-trees
and Fibonacci heaps.

\begin{funcdesc}{deque}{\optional{iterable}}
  Returns a new deque objected initialized left-to-right (using
  \method{append()}) with data from \var{iterable}.  If \var{iterable}
  is not specified, the new deque is empty.

  Deques are a generalization of stacks and queues (the name is pronounced
  ``deck'' and is short for ``double-ended queue'').  Deques support
  thread-safe, memory efficient appends and pops from either side of the deque
  with approximately the same \code{O(1)} performance in either direction.

  Though \class{list} objects support similar operations, they are optimized
  for fast fixed-length operations and incur \code{O(n)} memory movement costs
  for \samp{pop(0)} and \samp{insert(0, v)} operations which change both the
  size and position of the underlying data representation.
  \versionadded{2.4}
\end{funcdesc}

Deque objects support the following methods:

\begin{methoddesc}{append}{x}
   Add \var{x} to the right side of the deque.
\end{methoddesc}

\begin{methoddesc}{appendleft}{x}
   Add \var{x} to the left side of the deque.
\end{methoddesc}

\begin{methoddesc}{clear}{}
   Remove all elements from the deque leaving it with length 0.
\end{methoddesc}

\begin{methoddesc}{extend}{iterable}
   Extend the right side of the deque by appending elements from
   the iterable argument.
\end{methoddesc}

\begin{methoddesc}{extendleft}{iterable}
   Extend the left side of the deque by appending elements from
   \var{iterable}.  Note, the series of left appends results in
   reversing the order of elements in the iterable argument.
\end{methoddesc}

\begin{methoddesc}{pop}{}
   Remove and return an element from the right side of the deque.
   If no elements are present, raises a \exception{IndexError}.
\end{methoddesc}

\begin{methoddesc}{popleft}{}
   Remove and return an element from the left side of the deque.
   If no elements are present, raises a \exception{IndexError}.   
\end{methoddesc}

\begin{methoddesc}{rotate}{n}
   Rotate the deque \var{n} steps to the right.  If \var{n} is
   negative, rotate to the left.  Rotating one step to the right
   is equivalent to:  \samp{d.appendleft(d.pop())}. 
\end{methoddesc}

In addition to the above, deques support iteration, pickling, \samp{len(d)},
\samp{reversed(d)}, \samp{copy.copy(d)}, \samp{copy.deepcopy(d)},
membership testing with the \keyword{in} operator, and subscript references
such as \samp{d[-1]}.

Example:

\begin{verbatim}
>>> from collections import deque
>>> d = deque('ghi')                 # make a new deque with three items
>>> for elem in d:                   # iterate over the deque's elements
...     print elem.upper()	
G
H
I

>>> d.append('j')                    # add a new entry to the right side
>>> d.appendleft('f')                # add a new entry to the left side
>>> d                                # show the representation of the deque
deque(['f', 'g', 'h', 'i', 'j'])

>>> d.pop()                          # return and remove the rightmost item
'j'
>>> d.popleft()                      # return and remove the leftmost item
'f'
>>> list(d)                          # list the contents of the deque
['g', 'h', 'i']
>>> d[0]                             # peek at leftmost item
'g'
>>> d[-1]                            # peek at rightmost item
'i'

>>> list(reversed(d))                # list the contents of a deque in reverse
['i', 'h', 'g']
>>> 'h' in d                         # search the deque
True
>>> d.extend('jkl')                  # add multiple elements at once
>>> d
deque(['g', 'h', 'i', 'j', 'k', 'l'])
>>> d.rotate(1)                      # right rotation
>>> d
deque(['l', 'g', 'h', 'i', 'j', 'k'])
>>> d.rotate(-1)                     # left rotation
>>> d
deque(['g', 'h', 'i', 'j', 'k', 'l'])

>>> deque(reversed(d))               # make a new deque in reverse order
deque(['l', 'k', 'j', 'i', 'h', 'g'])
>>> d.clear()                        # empty the deque
>>> d.pop()                          # cannot pop from an empty deque
Traceback (most recent call last):
  File "<pyshell#6>", line 1, in -toplevel-
    d.pop()
IndexError: pop from an empty deque

>>> d.extendleft('abc')              # extendleft() reverses the input order
>>> d
deque(['c', 'b', 'a'])
\end{verbatim}

\subsection{Recipes \label{deque-recipes}}

This section shows various approaches to working with deques.

The \method{rotate()} method provides a way to implement \class{deque}
slicing and deletion:

\begin{verbatim}
def delete_nth(d, n):
    "del d[n]"
    d.rotate(-n)
    d.popleft()
    d.rotate(n)

>>> d = deque('abcdef')
>>> delete_nth(d, 2)   # remove the entry at d[2]
>>> d
deque(['a', 'b', 'd', 'e', 'f'])

\end{verbatim}

For slicing, the idea is the same.  Use \method{rotate()} to bring a target
element to the left side of the deque.  Remove old entries with
\method{popleft()}, add new entries with \method{extend()}, and then
reverse the rotation.

With minor variations on that approach, it is easy to implement Forth style
stack manipulations such as \code{dup}, \code{drop}, \code{swap}, \code{over},
\code{pick}, \code{rot}, and \code{roll}.

A roundrobin task server can be built from a \class{deque} using
\method{popleft()} to select the current task and \method{append()}
to add it back to the tasklist if the input stream is not exhausted:

\begin{verbatim}
def roundrobin(*iterables):
    pending = deque(iter(i) for i in iterables)
    while pending:
        task = pending.popleft()
        try:
            yield task.next()
        except StopIteration:
            continue
        pending.append(task)

>>> for value in roundrobin('abc', 'd', 'efgh'):
...     print value

a
d
e
b
f
c
g
h

\end{verbatim}


Multi-pass data reduction algorithms can be succinctly expressed and
efficiently coded by extracting elements using multiple calls to
\method{popleft()}, applying the reduction function, and using
\method{append()} for adding the result back to the queue.

For example, building a balanced binary tree of nested lists entails
reducing two adjacent nodes into one by grouping them in a list:

\begin{verbatim}
def maketree(iterable):
    d = deque(iterable)
    while len(d) > 1:
        pair = [d.popleft(), d.popleft()]
        d.append(pair)
    return list(d)

>>> print maketree('abcdefgh')
[[[['a', 'b'], ['c', 'd']], [['e', 'f'], ['g', 'h']]]]

\end{verbatim}