From 0e3f591aeeef9ed715f8770320f4c4c7332a8794 Mon Sep 17 00:00:00 2001 From: Thomas Wouters Date: Fri, 11 Aug 2006 14:57:12 +0000 Subject: Merged revisions 46753-51188 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ........ r46755 | brett.cannon | 2006-06-08 18:23:04 +0200 (Thu, 08 Jun 2006) | 4 lines Make binascii.hexlify() use s# for its arguments instead of t# to actually match its documentation stating it accepts any read-only buffer. ........ r46757 | brett.cannon | 2006-06-08 19:00:45 +0200 (Thu, 08 Jun 2006) | 8 lines Buffer objects would return the read or write buffer for a wrapped object when the char buffer was requested. Now it actually returns the char buffer if available or raises a TypeError if it isn't (as is raised for the other buffer types if they are not present but requested). Not a backport candidate since it does change semantics of the buffer object (although it could be argued this is enough of a bug to bother backporting). ........ r46760 | andrew.kuchling | 2006-06-09 03:10:17 +0200 (Fri, 09 Jun 2006) | 1 line Update functools section ........ r46762 | tim.peters | 2006-06-09 04:11:02 +0200 (Fri, 09 Jun 2006) | 6 lines Whitespace normalization. Since test_file is implicated in mysterious test failures when followed by test_optparse, if I had any brains I'd look at the checkin that last changed test_file ;-) ........ r46763 | tim.peters | 2006-06-09 05:09:42 +0200 (Fri, 09 Jun 2006) | 5 lines To boost morale :-), force test_optparse to run immediately after test_file until we can figure out how to fix it. (See python-dev; at the moment we don't even know which checkin caused the problem.) ........ r46764 | tim.peters | 2006-06-09 05:51:41 +0200 (Fri, 09 Jun 2006) | 6 lines AutoFileTests.tearDown(): Removed mysterious undocumented try/except. Remove TESTFN. Throughout: used open() instead of file(), and wrapped long lines. ........ r46765 | tim.peters | 2006-06-09 06:02:06 +0200 (Fri, 09 Jun 2006) | 8 lines testUnicodeOpen(): I have no idea why, but making this test clean up after itself appears to fix the test failures when test_optparse follows test_file. test_main(): Get rid of TESTFN no matter what. That's also enough to fix the mystery failures. Doesn't hurt to fix them twice :-) ........ r46766 | tim.peters | 2006-06-09 07:12:40 +0200 (Fri, 09 Jun 2006) | 6 lines Remove the temporary hack to force test_optparse to run immediately after test_file. At least 8 buildbot boxes passed since the underlying problem got fixed, and they all failed before the fix, so there's no point to this anymore. ........ r46767 | neal.norwitz | 2006-06-09 07:54:18 +0200 (Fri, 09 Jun 2006) | 1 line Fix grammar and reflow ........ r46769 | andrew.kuchling | 2006-06-09 12:22:35 +0200 (Fri, 09 Jun 2006) | 1 line Markup fix ........ r46773 | andrew.kuchling | 2006-06-09 15:15:57 +0200 (Fri, 09 Jun 2006) | 1 line [Bug #1472827] Make saxutils.XMLGenerator handle \r\n\t in attribute values by escaping them properly. 2.4 bugfix candidate. ........ r46778 | kristjan.jonsson | 2006-06-09 18:28:01 +0200 (Fri, 09 Jun 2006) | 2 lines Turn off warning about deprecated CRT functions on for VisualStudio .NET 2005. Make the definition #ARRAYSIZE conditional. VisualStudio .NET 2005 already has it defined using a better gimmick. ........ r46779 | phillip.eby | 2006-06-09 18:40:18 +0200 (Fri, 09 Jun 2006) | 2 lines Import wsgiref into the stdlib, as of the external version 0.1-r2181. ........ r46783 | andrew.kuchling | 2006-06-09 18:44:40 +0200 (Fri, 09 Jun 2006) | 1 line Add note about XMLGenerator bugfix ........ r46784 | andrew.kuchling | 2006-06-09 18:46:51 +0200 (Fri, 09 Jun 2006) | 1 line Add note about wsgiref ........ r46785 | brett.cannon | 2006-06-09 19:05:48 +0200 (Fri, 09 Jun 2006) | 2 lines Fix inconsistency in naming within an enum. ........ r46787 | tim.peters | 2006-06-09 19:47:00 +0200 (Fri, 09 Jun 2006) | 2 lines Whitespace normalization. ........ r46792 | georg.brandl | 2006-06-09 20:29:52 +0200 (Fri, 09 Jun 2006) | 3 lines Test file.__exit__. ........ r46794 | brett.cannon | 2006-06-09 20:40:46 +0200 (Fri, 09 Jun 2006) | 2 lines svn:ignore .pyc and .pyo files. ........ r46795 | georg.brandl | 2006-06-09 20:45:48 +0200 (Fri, 09 Jun 2006) | 3 lines RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. ........ r46798 | andrew.kuchling | 2006-06-09 21:03:16 +0200 (Fri, 09 Jun 2006) | 1 line Describe startswith()/endswiith() change; add reminder about wsgiref ........ r46799 | tim.peters | 2006-06-09 21:24:44 +0200 (Fri, 09 Jun 2006) | 11 lines Implementing a happy idea from Georg Brandl: make runtest() try to clean up files and directories the tests often leave behind by mistake. This is the first time in history I don't have a bogus "db_home" directory after running the tests ;-) Also worked on runtest's docstring, to say something about all the arguments, and to document the non-obvious return values. New functions runtest_inner() and cleanup_test_droppings() in support of the above. ........ r46800 | andrew.kuchling | 2006-06-09 21:43:25 +0200 (Fri, 09 Jun 2006) | 1 line Remove unused variable ........ r46801 | andrew.kuchling | 2006-06-09 21:56:05 +0200 (Fri, 09 Jun 2006) | 1 line Add some wsgiref text ........ r46803 | thomas.heller | 2006-06-09 21:59:11 +0200 (Fri, 09 Jun 2006) | 1 line set eol-style svn property ........ r46804 | thomas.heller | 2006-06-09 22:01:01 +0200 (Fri, 09 Jun 2006) | 1 line set eol-style svn property ........ r46805 | georg.brandl | 2006-06-09 22:43:48 +0200 (Fri, 09 Jun 2006) | 3 lines Make use of new str.startswith/endswith semantics. Occurences in email and compiler were ignored due to backwards compat requirements. ........ r46806 | brett.cannon | 2006-06-10 00:31:23 +0200 (Sat, 10 Jun 2006) | 4 lines An object with __call__ as an attribute, when called, will have that attribute checked for __call__ itself, and will continue to look until it finds an object without the attribute. This can lead to an infinite recursion. Closes bug #532646, again. Will be backported. ........ r46808 | brett.cannon | 2006-06-10 00:45:54 +0200 (Sat, 10 Jun 2006) | 2 lines Fix bug introduced in rev. 46806 by not having variable declaration at the top of a block. ........ r46812 | georg.brandl | 2006-06-10 08:40:50 +0200 (Sat, 10 Jun 2006) | 4 lines Apply perky's fix for #1503157: "/".join([u"", u""]) raising OverflowError. Also improve error message on overflow. ........ r46817 | martin.v.loewis | 2006-06-10 10:14:03 +0200 (Sat, 10 Jun 2006) | 2 lines Port cygwin kill_python changes from 2.4 branch. ........ r46818 | armin.rigo | 2006-06-10 12:57:40 +0200 (Sat, 10 Jun 2006) | 4 lines SF bug #1503294. PyThreadState_GET() complains if the tstate is NULL, but only in debug mode. ........ r46819 | martin.v.loewis | 2006-06-10 14:23:46 +0200 (Sat, 10 Jun 2006) | 4 lines Patch #1495999: Part two of Windows CE changes. - update header checks, using autoconf - provide dummies for getenv, environ, and GetVersion - adjust MSC_VER check in socketmodule.c ........ r46820 | skip.montanaro | 2006-06-10 16:09:11 +0200 (Sat, 10 Jun 2006) | 1 line document the class, not its initializer ........ r46821 | greg.ward | 2006-06-10 18:40:01 +0200 (Sat, 10 Jun 2006) | 4 lines Sync with Optik docs (rev 518): * restore "Extending optparse" section * document ALWAYS_TYPED_ACTIONS (SF #1449311) ........ r46824 | thomas.heller | 2006-06-10 21:51:46 +0200 (Sat, 10 Jun 2006) | 8 lines Upgrade to ctypes version 0.9.9.7. Summary of changes: - support for 'variable sized' data - support for anonymous structure/union fields - fix severe bug with certain arrays or structures containing more than 256 fields ........ r46825 | thomas.heller | 2006-06-10 21:55:36 +0200 (Sat, 10 Jun 2006) | 8 lines Upgrade to ctypes version 0.9.9.7. Summary of changes: - support for 'variable sized' data - support for anonymous structure/union fields - fix severe bug with certain arrays or structures containing more than 256 fields ........ r46826 | fred.drake | 2006-06-10 22:01:34 +0200 (Sat, 10 Jun 2006) | 4 lines SF patch #1303595: improve description of __builtins__, explaining how it varies between __main__ and other modules, and strongly suggest not touching it but using __builtin__ if absolutely necessary ........ r46827 | fred.drake | 2006-06-10 22:02:58 +0200 (Sat, 10 Jun 2006) | 1 line credit for SF patch #1303595 ........ r46831 | thomas.heller | 2006-06-10 22:29:34 +0200 (Sat, 10 Jun 2006) | 2 lines New docs for ctypes. ........ r46834 | thomas.heller | 2006-06-10 23:07:19 +0200 (Sat, 10 Jun 2006) | 1 line Fix a wrong printf format. ........ r46835 | thomas.heller | 2006-06-10 23:17:58 +0200 (Sat, 10 Jun 2006) | 1 line Fix the second occurrence of the problematic printf format. ........ r46837 | thomas.heller | 2006-06-10 23:56:03 +0200 (Sat, 10 Jun 2006) | 1 line Don't use C++ comment. ........ r46838 | thomas.heller | 2006-06-11 00:01:50 +0200 (Sun, 11 Jun 2006) | 1 line Handle failure of PyMem_Realloc. ........ r46839 | skip.montanaro | 2006-06-11 00:38:13 +0200 (Sun, 11 Jun 2006) | 2 lines Suppress warning on MacOSX about possible use before set of proc. ........ r46840 | tim.peters | 2006-06-11 00:51:45 +0200 (Sun, 11 Jun 2006) | 8 lines shuffle() doscstring: Removed warning about sequence length versus generator period. While this was a real weakness of the older WH generator for lists with just a few dozen elements, and so could potentially bite the naive ;-), the Twister should show excellent behavior up to at least 600 elements. Module docstring: reflowed some jarringly short lines. ........ r46844 | greg.ward | 2006-06-11 02:40:49 +0200 (Sun, 11 Jun 2006) | 4 lines Bug #1361643: fix textwrap.dedent() so it handles tabs appropriately, i.e. do *not* expand tabs, but treat them as whitespace that is not equivalent to spaces. Add a couple of test cases. Clarify docs. ........ r46850 | neal.norwitz | 2006-06-11 07:44:18 +0200 (Sun, 11 Jun 2006) | 5 lines Fix Coverity # 146. newDBSequenceObject would deref dbobj, so it can't be NULL. We know it's not NULL from the ParseTuple and DbObject_Check will verify it's not NULL. ........ r46851 | neal.norwitz | 2006-06-11 07:45:25 +0200 (Sun, 11 Jun 2006) | 4 lines Wrap some long lines Top/Bottom factor out some common expressions Add a XXX comment about widing offset. ........ r46852 | neal.norwitz | 2006-06-11 07:45:47 +0200 (Sun, 11 Jun 2006) | 1 line Add versionadded to doc ........ r46853 | neal.norwitz | 2006-06-11 07:47:14 +0200 (Sun, 11 Jun 2006) | 3 lines Update doc to make it agree with code. Bottom factor out some common code. ........ r46854 | neal.norwitz | 2006-06-11 07:48:14 +0200 (Sun, 11 Jun 2006) | 3 lines f_code can't be NULL based on Frame_New and other code that derefs it. So there doesn't seem to be much point to checking here. ........ r46855 | neal.norwitz | 2006-06-11 09:26:27 +0200 (Sun, 11 Jun 2006) | 1 line Fix errors found by pychecker ........ r46856 | neal.norwitz | 2006-06-11 09:26:50 +0200 (Sun, 11 Jun 2006) | 1 line warnings was imported at module scope, no need to import again ........ r46857 | neal.norwitz | 2006-06-11 09:27:56 +0200 (Sun, 11 Jun 2006) | 5 lines Fix errors found by pychecker. I think these changes are correct, but I'm not sure. Could someone who knows how this module works test it? It can at least start on the cmd line. ........ r46858 | neal.norwitz | 2006-06-11 10:35:14 +0200 (Sun, 11 Jun 2006) | 1 line Fix errors found by pychecker ........ r46859 | ronald.oussoren | 2006-06-11 16:33:36 +0200 (Sun, 11 Jun 2006) | 4 lines This patch improves the L&F of IDLE on OSX. The changes are conditionalized on being in an IDLE.app bundle on darwin. This does a slight reorganisation of the menus and adds support for file-open events. ........ r46860 | greg.ward | 2006-06-11 16:42:41 +0200 (Sun, 11 Jun 2006) | 1 line SF #1366250: optparse docs: fix inconsistency in variable name; minor tweaks. ........ r46861 | greg.ward | 2006-06-11 18:24:11 +0200 (Sun, 11 Jun 2006) | 3 lines Bug #1498146: fix optparse to handle Unicode strings in option help, description, and epilog. ........ r46862 | thomas.heller | 2006-06-11 19:04:22 +0200 (Sun, 11 Jun 2006) | 2 lines Release the GIL during COM method calls, to avoid deadlocks in Python coded COM objects. ........ r46863 | tim.peters | 2006-06-11 21:42:51 +0200 (Sun, 11 Jun 2006) | 2 lines Whitespace normalization. ........ r46864 | tim.peters | 2006-06-11 21:43:49 +0200 (Sun, 11 Jun 2006) | 2 lines Add missing svn:eol-style property to text files. ........ r46865 | ronald.oussoren | 2006-06-11 21:45:57 +0200 (Sun, 11 Jun 2006) | 2 lines Remove message about using make frameworkinstall, that's no longer necesssary ........ r46866 | ronald.oussoren | 2006-06-11 22:23:29 +0200 (Sun, 11 Jun 2006) | 2 lines Use configure to substitute the correct prefix instead of hardcoding ........ r46867 | ronald.oussoren | 2006-06-11 22:24:45 +0200 (Sun, 11 Jun 2006) | 4 lines - Change fixapplepython23.py to ensure that it will run with /usr/bin/python on intel macs. - Fix some minor problems in the installer for OSX ........ r46868 | neal.norwitz | 2006-06-11 22:25:56 +0200 (Sun, 11 Jun 2006) | 5 lines Try to fix several networking tests. The problem is that if hosts have a search path setup, some of these hosts resolve to the wrong address. By appending a period to the hostname, the hostname should only resolve to what we want it to resolve to. Hopefully this doesn't break different bots. ........ r46869 | neal.norwitz | 2006-06-11 22:42:02 +0200 (Sun, 11 Jun 2006) | 7 lines Try to fix another networking test. The problem is that if hosts have a search path setup, some of these hosts resolve to the wrong address. By appending a period to the hostname, the hostname should only resolve to what we want it to resolve to. Hopefully this doesn't break different bots. Also add more info to failure message to aid debugging test failure. ........ r46870 | neal.norwitz | 2006-06-11 22:46:46 +0200 (Sun, 11 Jun 2006) | 4 lines Fix test on PPC64 buildbot. It raised an IOError (really an URLError which derives from an IOError). That seems valid. Env Error includes both OSError and IOError, so this seems like a reasonable fix. ........ r46871 | tim.peters | 2006-06-11 22:52:59 +0200 (Sun, 11 Jun 2006) | 10 lines compare_generic_iter(): Fixed the failure of test_wsgiref's testFileWrapper when running with -O. test_simple_validation_error still fails under -O. That appears to be because wsgiref's validate.py uses `assert` statements all over the place to check arguments for sanity. That should all be changed (it's not a logical error in the software if a user passes bogus arguments, so this isn't a reasonable use for `assert` -- checking external preconditions should generally raise ValueError or TypeError instead, as appropriate). ........ r46872 | neal.norwitz | 2006-06-11 23:38:38 +0200 (Sun, 11 Jun 2006) | 1 line Get test to pass on S/390. Shout if you think this change is incorrect. ........ r46873 | neal.norwitz | 2006-06-12 04:05:55 +0200 (Mon, 12 Jun 2006) | 1 line Cleanup Py_ssize_t a little (get rid of second #ifdef) ........ r46874 | neal.norwitz | 2006-06-12 04:06:17 +0200 (Mon, 12 Jun 2006) | 1 line Fix some Py_ssize_t issues ........ r46875 | neal.norwitz | 2006-06-12 04:06:42 +0200 (Mon, 12 Jun 2006) | 1 line Fix some Py_ssize_t issues ........ r46876 | neal.norwitz | 2006-06-12 04:07:24 +0200 (Mon, 12 Jun 2006) | 2 lines Cleanup: Remove import of types to get StringTypes, we can just use basestring. ........ r46877 | neal.norwitz | 2006-06-12 04:07:57 +0200 (Mon, 12 Jun 2006) | 1 line Don't truncate if size_t is bigger than uint ........ r46878 | neal.norwitz | 2006-06-12 04:08:41 +0200 (Mon, 12 Jun 2006) | 1 line Don't leak the list object if there's an error allocating the item storage. Backport candidate ........ r46879 | neal.norwitz | 2006-06-12 04:09:03 +0200 (Mon, 12 Jun 2006) | 1 line Fix typo. Backport if anyone cares. :-) ........ r46880 | neal.norwitz | 2006-06-12 04:09:34 +0200 (Mon, 12 Jun 2006) | 1 line Fix indentation of case and a Py_ssize_t issue. ........ r46881 | neal.norwitz | 2006-06-12 04:11:18 +0200 (Mon, 12 Jun 2006) | 3 lines Get rid of f_restricted too. Doc the other 4 ints that were already removed at the NeedForSpeed sprint. ........ r46882 | neal.norwitz | 2006-06-12 04:13:21 +0200 (Mon, 12 Jun 2006) | 1 line Fix the socket tests so they can be run concurrently. Backport candidate ........ r46883 | neal.norwitz | 2006-06-12 04:16:10 +0200 (Mon, 12 Jun 2006) | 1 line i and j are initialized below when used. No need to do it twice ........ r46884 | neal.norwitz | 2006-06-12 05:05:03 +0200 (Mon, 12 Jun 2006) | 1 line Remove unused import ........ r46885 | neal.norwitz | 2006-06-12 05:05:40 +0200 (Mon, 12 Jun 2006) | 1 line Impl ssize_t ........ r46886 | neal.norwitz | 2006-06-12 05:33:09 +0200 (Mon, 12 Jun 2006) | 6 lines Patch #1503046, Conditional compilation of zlib.(de)compressobj.copy copy is only in newer versions of zlib. This should allow zlibmodule to work with older versions like the Tru64 buildbot. ........ r46887 | phillip.eby | 2006-06-12 06:04:32 +0200 (Mon, 12 Jun 2006) | 2 lines Sync w/external release 0.1.2. Please see PEP 360 before making changes to external packages. ........ r46888 | martin.v.loewis | 2006-06-12 06:26:31 +0200 (Mon, 12 Jun 2006) | 2 lines Get rid of function pointer cast. ........ r46889 | thomas.heller | 2006-06-12 08:05:57 +0200 (Mon, 12 Jun 2006) | 3 lines I don't know how that happend, but the entire file contents was duplicated. Thanks to Simon Percivall for the heads up. ........ r46890 | nick.coghlan | 2006-06-12 10:19:37 +0200 (Mon, 12 Jun 2006) | 1 line Fix site module docstring to match the code ........ r46891 | nick.coghlan | 2006-06-12 10:23:02 +0200 (Mon, 12 Jun 2006) | 1 line Fix site module docstring to match the code for Mac OSX, too ........ r46892 | nick.coghlan | 2006-06-12 10:27:13 +0200 (Mon, 12 Jun 2006) | 1 line The site module documentation also described the Windows behaviour incorrectly. ........ r46893 | nick.coghlan | 2006-06-12 12:17:11 +0200 (Mon, 12 Jun 2006) | 1 line Make the -m switch conform to the documentation of sys.path by behaving like the -c switch ........ r46894 | kristjan.jonsson | 2006-06-12 17:45:12 +0200 (Mon, 12 Jun 2006) | 2 lines Fix the CRT argument error handling for VisualStudio .NET 2005. Install a CRT error handler and disable the assertion for debug builds. This causes CRT to set errno to EINVAL. This update fixes crash cases in the test suite where the default CRT error handler would cause process exit. ........ r46899 | thomas.heller | 2006-06-12 22:56:48 +0200 (Mon, 12 Jun 2006) | 1 line Add pep-291 compatibility markers. ........ r46901 | ka-ping.yee | 2006-06-13 01:47:52 +0200 (Tue, 13 Jun 2006) | 5 lines Add the uuid module. This module has been tested so far on Windows XP (Python 2.4 and 2.5a2), Mac OS X (Python 2.3, 2.4, and 2.5a2), and Linux (Python 2.4 and 2.5a2). ........ r46902 | tim.peters | 2006-06-13 02:30:01 +0200 (Tue, 13 Jun 2006) | 2 lines Whitespace normalization. ........ r46903 | tim.peters | 2006-06-13 02:30:50 +0200 (Tue, 13 Jun 2006) | 2 lines Added missing svn:eol-style property to text files. ........ r46905 | tim.peters | 2006-06-13 05:30:07 +0200 (Tue, 13 Jun 2006) | 5 lines get_matching_blocks(): rewrote code & comments so they match; added more comments about why it's this way at all; and removed what looked like needless expense (sorting (i, j, k) triples directly should give exactly the same order as sorting (i, (i, j, k)) pairs). ........ r46906 | neal.norwitz | 2006-06-13 06:08:53 +0200 (Tue, 13 Jun 2006) | 1 line Don't fail if another process is listening on our port. ........ r46908 | neal.norwitz | 2006-06-13 10:28:19 +0200 (Tue, 13 Jun 2006) | 2 lines Initialize the type object so pychecker can't crash the interpreter. ........ r46909 | neal.norwitz | 2006-06-13 10:41:06 +0200 (Tue, 13 Jun 2006) | 1 line Verify the crash due to EncodingMap not initialized does not return ........ r46910 | thomas.heller | 2006-06-13 10:56:14 +0200 (Tue, 13 Jun 2006) | 3 lines Add some windows datatypes that were missing from this file, and add the aliases defined in windows header files for the structures. ........ r46911 | thomas.heller | 2006-06-13 11:40:14 +0200 (Tue, 13 Jun 2006) | 3 lines Add back WCHAR, UINT, DOUBLE, _LARGE_INTEGER, _ULARGE_INTEGER. VARIANT_BOOL is a special _ctypes data type, not c_short. ........ r46912 | ronald.oussoren | 2006-06-13 13:19:56 +0200 (Tue, 13 Jun 2006) | 4 lines Linecache contains support for PEP302 loaders, but fails to deal with loaders that return None to indicate that the module is valid but no source is available. This patch fixes that. ........ r46913 | andrew.kuchling | 2006-06-13 13:57:04 +0200 (Tue, 13 Jun 2006) | 1 line Mention uuid module ........ r46915 | walter.doerwald | 2006-06-13 14:02:12 +0200 (Tue, 13 Jun 2006) | 2 lines Fix passing errors to the encoder and decoder functions. ........ r46917 | walter.doerwald | 2006-06-13 14:04:43 +0200 (Tue, 13 Jun 2006) | 3 lines errors is an attribute in the incremental decoder not an argument. ........ r46919 | andrew.macintyre | 2006-06-13 17:04:24 +0200 (Tue, 13 Jun 2006) | 11 lines Patch #1454481: Make thread stack size runtime tunable. Heavily revised, comprising revisions: 46640 - original trunk revision (backed out in r46655) 46647 - markup fix (backed out in r46655) 46692:46918 merged from branch aimacintyre-sf1454481 branch tested on buildbots (Windows buildbots had problems not related to these changes). ........ r46920 | brett.cannon | 2006-06-13 18:06:55 +0200 (Tue, 13 Jun 2006) | 2 lines Remove unused variable. ........ r46921 | andrew.kuchling | 2006-06-13 18:41:41 +0200 (Tue, 13 Jun 2006) | 1 line Add ability to set stack size ........ r46923 | marc-andre.lemburg | 2006-06-13 19:04:26 +0200 (Tue, 13 Jun 2006) | 2 lines Update pybench to version 2.0. ........ r46924 | marc-andre.lemburg | 2006-06-13 19:07:14 +0200 (Tue, 13 Jun 2006) | 2 lines Revert wrong svn copy. ........ r46925 | andrew.macintyre | 2006-06-13 19:14:36 +0200 (Tue, 13 Jun 2006) | 2 lines fix exception usage ........ r46927 | tim.peters | 2006-06-13 20:37:07 +0200 (Tue, 13 Jun 2006) | 2 lines Whitespace normalization. ........ r46928 | marc-andre.lemburg | 2006-06-13 20:56:56 +0200 (Tue, 13 Jun 2006) | 9 lines Updated to pybench 2.0. See svn.python.org/external/pybench-2.0 for the original import of that version. Note that platform.py was not copied over from pybench-2.0 since it is already part of Python 2.5. ........ r46929 | andrew.macintyre | 2006-06-13 21:02:35 +0200 (Tue, 13 Jun 2006) | 5 lines Increase the small thread stack size to get the test to pass reliably on the one buildbot that insists on more than 32kB of thread stack. ........ r46930 | marc-andre.lemburg | 2006-06-13 21:20:07 +0200 (Tue, 13 Jun 2006) | 2 lines Whitespace normalization. ........ r46931 | thomas.heller | 2006-06-13 22:18:43 +0200 (Tue, 13 Jun 2006) | 2 lines More docs for ctypes. ........ r46932 | brett.cannon | 2006-06-13 23:34:24 +0200 (Tue, 13 Jun 2006) | 2 lines Ignore .pyc and .pyo files in Pybench. ........ r46933 | brett.cannon | 2006-06-13 23:46:41 +0200 (Tue, 13 Jun 2006) | 7 lines If a classic class defined a __coerce__() method that just returned its two arguments in reverse, the interpreter would infinitely recourse trying to get a coercion that worked. So put in a recursion check after a coercion is made and the next call to attempt to use the coerced values. Fixes bug #992017 and closes crashers/coerce.py . ........ r46936 | gerhard.haering | 2006-06-14 00:24:47 +0200 (Wed, 14 Jun 2006) | 3 lines Merged changes from external pysqlite 2.3.0 release. Documentation updates will follow in a few hours at the latest. Then we should be ready for beta1. ........ r46937 | brett.cannon | 2006-06-14 00:26:13 +0200 (Wed, 14 Jun 2006) | 2 lines Missed test for rev. 46933; infinite recursion from __coerce__() returning its arguments reversed. ........ r46938 | gerhard.haering | 2006-06-14 00:53:48 +0200 (Wed, 14 Jun 2006) | 2 lines Updated documentation for pysqlite 2.3.0 API. ........ r46939 | tim.peters | 2006-06-14 06:09:25 +0200 (Wed, 14 Jun 2006) | 10 lines SequenceMatcher.get_matching_blocks(): This now guarantees that adjacent triples in the result list describe non-adjacent matching blocks. That's _nice_ to have, and Guido said he wanted it. Not a bugfix candidate: Guido or not ;-), this changes visible endcase semantics (note that some tests had to change), and nothing about this was documented before. Since it was working as designed, and behavior was consistent with the docs, it wasn't "a bug". ........ r46940 | tim.peters | 2006-06-14 06:13:00 +0200 (Wed, 14 Jun 2006) | 2 lines Repaired typo in new comment. ........ r46941 | tim.peters | 2006-06-14 06:15:27 +0200 (Wed, 14 Jun 2006) | 2 lines Whitespace normalization. ........ r46942 | fred.drake | 2006-06-14 06:25:02 +0200 (Wed, 14 Jun 2006) | 3 lines - make some disabled tests run what they intend when enabled - remove some over-zealous triple-quoting ........ r46943 | fred.drake | 2006-06-14 07:04:47 +0200 (Wed, 14 Jun 2006) | 3 lines add tests for two cases that are handled correctly in the current code, but that SF patch 1504676 as written mis-handles ........ r46944 | fred.drake | 2006-06-14 07:15:51 +0200 (Wed, 14 Jun 2006) | 1 line explain an XXX in more detail ........ r46945 | martin.v.loewis | 2006-06-14 07:21:04 +0200 (Wed, 14 Jun 2006) | 1 line Patch #1455898: Incremental mode for "mbcs" codec. ........ r46946 | georg.brandl | 2006-06-14 08:08:31 +0200 (Wed, 14 Jun 2006) | 3 lines Bug #1339007: Shelf objects now don't raise an exception in their __del__ method when initialization failed. ........ r46948 | thomas.heller | 2006-06-14 08:18:15 +0200 (Wed, 14 Jun 2006) | 1 line Fix docstring. ........ r46949 | georg.brandl | 2006-06-14 08:29:07 +0200 (Wed, 14 Jun 2006) | 2 lines Bug #1501122: mention __gt__ &co in description of comparison order. ........ r46951 | thomas.heller | 2006-06-14 09:08:38 +0200 (Wed, 14 Jun 2006) | 1 line Write more docs. ........ r46952 | georg.brandl | 2006-06-14 10:31:39 +0200 (Wed, 14 Jun 2006) | 3 lines Bug #1153163: describe __add__ vs __radd__ behavior when adding objects of same type/of subclasses of the other. ........ r46954 | georg.brandl | 2006-06-14 10:42:11 +0200 (Wed, 14 Jun 2006) | 3 lines Bug #1202018: add some common mime.types locations. ........ r46955 | georg.brandl | 2006-06-14 10:50:03 +0200 (Wed, 14 Jun 2006) | 3 lines Bug #1117556: SimpleHTTPServer now tries to find and use the system's mime.types file for determining MIME types. ........ r46957 | thomas.heller | 2006-06-14 11:09:08 +0200 (Wed, 14 Jun 2006) | 1 line Document paramflags. ........ r46958 | thomas.heller | 2006-06-14 11:20:11 +0200 (Wed, 14 Jun 2006) | 1 line Add an __all__ list, since this module does 'from ctypes import *'. ........ r46959 | andrew.kuchling | 2006-06-14 15:59:15 +0200 (Wed, 14 Jun 2006) | 1 line Add item ........ r46961 | georg.brandl | 2006-06-14 18:46:43 +0200 (Wed, 14 Jun 2006) | 3 lines Bug #805015: doc error in PyUnicode_FromEncodedObject. ........ r46962 | gerhard.haering | 2006-06-15 00:28:37 +0200 (Thu, 15 Jun 2006) | 10 lines - Added version checks in C code to make sure we don't trigger bugs in older SQLite versions. - Added version checks in test suite so that we don't execute tests that we know will fail with older (buggy) SQLite versions. Now, all tests should run against all SQLite versions from 3.0.8 until 3.3.6 (latest one now). The sqlite3 module can be built against all these SQLite versions and the sqlite3 module does its best to not trigger bugs in SQLite, but using SQLite 3.3.3 or later is recommended. ........ r46963 | tim.peters | 2006-06-15 00:38:13 +0200 (Thu, 15 Jun 2006) | 2 lines Whitespace normalization. ........ r46964 | neal.norwitz | 2006-06-15 06:54:29 +0200 (Thu, 15 Jun 2006) | 9 lines Speculative checkin (requires approval of Gerhard Haering) This backs out the test changes in 46962 which prevented crashes by not running the tests via a version check. All the version checks added in that rev were removed from the tests. Code was added to the error handler in connection.c that seems to work with older versions of sqlite including 3.1.3. ........ r46965 | neal.norwitz | 2006-06-15 07:55:49 +0200 (Thu, 15 Jun 2006) | 1 line Try to narrow window of failure on slow/busy boxes (ppc64 buildbot) ........ r46966 | martin.v.loewis | 2006-06-15 08:45:05 +0200 (Thu, 15 Jun 2006) | 2 lines Make import/lookup of mbcs fail on non-Windows systems. ........ r46967 | ronald.oussoren | 2006-06-15 10:14:18 +0200 (Thu, 15 Jun 2006) | 2 lines Patch #1446489 (zipfile: support for ZIP64) ........ r46968 | neal.norwitz | 2006-06-15 10:16:44 +0200 (Thu, 15 Jun 2006) | 6 lines Re-revert this change. Install the version check and don't run the test until Gerhard has time to fully debug the issue. This affects versions before 3.2.1 (possibly only versions earlier than 3.1.3). Based on discussion on python-checkins. ........ r46969 | gregory.p.smith | 2006-06-15 10:52:32 +0200 (Thu, 15 Jun 2006) | 6 lines - bsddb: multithreaded DB access using the simple bsddb module interface now works reliably. It has been updated to use automatic BerkeleyDB deadlock detection and the bsddb.dbutils.DeadlockWrap wrapper to retry database calls that would previously deadlock. [SF python bug #775414] ........ r46970 | gregory.p.smith | 2006-06-15 11:23:52 +0200 (Thu, 15 Jun 2006) | 2 lines minor documentation cleanup. mention the bsddb.db interface explicitly by name. ........ r46971 | neal.norwitz | 2006-06-15 11:57:03 +0200 (Thu, 15 Jun 2006) | 5 lines Steal the trick from test_compiler to print out a slow msg. This will hopefully get the buildbots to pass. Not sure this test will be feasible or even work. But everything is red now, so it can't get much worse. ........ r46972 | neal.norwitz | 2006-06-15 12:24:49 +0200 (Thu, 15 Jun 2006) | 1 line Print some more info to get an idea of how much longer the test will last ........ r46981 | tim.peters | 2006-06-15 20:04:40 +0200 (Thu, 15 Jun 2006) | 6 lines Try to reduce the extreme peak memory and disk-space use of this test. It probably still requires more disk space than most buildbots have, and in any case is still so intrusive that if we don't find another way to test this I'm taking my buildbot offline permanently ;-) ........ r46982 | tim.peters | 2006-06-15 20:06:29 +0200 (Thu, 15 Jun 2006) | 2 lines Whitespace normalization. ........ r46983 | tim.peters | 2006-06-15 20:07:28 +0200 (Thu, 15 Jun 2006) | 2 lines Add missing svn:eol-style property to text files. ........ r46984 | tim.peters | 2006-06-15 20:38:19 +0200 (Thu, 15 Jun 2006) | 2 lines Oops -- I introduced an off-by-6436159488 error. ........ r46990 | neal.norwitz | 2006-06-16 06:30:34 +0200 (Fri, 16 Jun 2006) | 1 line Disable this test until we can determine what to do about it ........ r46991 | neal.norwitz | 2006-06-16 06:31:06 +0200 (Fri, 16 Jun 2006) | 1 line Param name is dir, not directory. Update docstring. Backport candidate ........ r46992 | neal.norwitz | 2006-06-16 06:31:28 +0200 (Fri, 16 Jun 2006) | 1 line Add missing period in comment. ........ r46993 | neal.norwitz | 2006-06-16 06:32:43 +0200 (Fri, 16 Jun 2006) | 1 line Fix whitespace, there are memory leaks in this module. ........ r46995 | fred.drake | 2006-06-17 01:45:06 +0200 (Sat, 17 Jun 2006) | 3 lines SF patch 1504676: Make sgmllib char and entity references pluggable (implementation/tests contributed by Sam Ruby) ........ r46996 | fred.drake | 2006-06-17 03:07:54 +0200 (Sat, 17 Jun 2006) | 1 line fix change that broke the htmllib tests ........ r46998 | martin.v.loewis | 2006-06-17 11:15:14 +0200 (Sat, 17 Jun 2006) | 3 lines Patch #763580: Add name and value arguments to Tkinter variable classes. ........ r46999 | martin.v.loewis | 2006-06-17 11:20:41 +0200 (Sat, 17 Jun 2006) | 2 lines Patch #1096231: Add default argument to wm_iconbitmap. ........ r47000 | martin.v.loewis | 2006-06-17 11:25:15 +0200 (Sat, 17 Jun 2006) | 2 lines Patch #1494750: Destroy master after deleting children. ........ r47003 | george.yoshida | 2006-06-17 18:31:52 +0200 (Sat, 17 Jun 2006) | 2 lines markup fix ........ r47005 | george.yoshida | 2006-06-17 18:39:13 +0200 (Sat, 17 Jun 2006) | 4 lines Update url. Old url returned status code:301 Moved permanently. ........ r47007 | martin.v.loewis | 2006-06-17 20:44:27 +0200 (Sat, 17 Jun 2006) | 2 lines Patch #812986: Update the canvas even if not tracing. ........ r47008 | martin.v.loewis | 2006-06-17 21:03:26 +0200 (Sat, 17 Jun 2006) | 2 lines Patch #815924: Restore ability to pass type= and icon= ........ r47009 | neal.norwitz | 2006-06-18 00:37:45 +0200 (Sun, 18 Jun 2006) | 1 line Fix typo in docstring ........ r47010 | neal.norwitz | 2006-06-18 00:38:15 +0200 (Sun, 18 Jun 2006) | 1 line Fix memory leak reported by valgrind while running test_subprocess ........ r47011 | fred.drake | 2006-06-18 04:57:35 +0200 (Sun, 18 Jun 2006) | 1 line remove unnecessary markup ........ r47013 | neal.norwitz | 2006-06-18 21:35:01 +0200 (Sun, 18 Jun 2006) | 7 lines Prevent spurious leaks when running regrtest.py -R. There may be more issues that crop up from time to time, but this change seems to have been pretty stable (no spurious warnings) for about a week. Other modules which use threads may require similar use of threading_setup/threading_cleanup from test_support. ........ r47014 | neal.norwitz | 2006-06-18 21:37:40 +0200 (Sun, 18 Jun 2006) | 9 lines The hppa ubuntu box sometimes hangs forever in these tests. My guess is that the wait is failing for some reason. Use WNOHANG, so we won't wait until the buildbot kills the test suite. I haven't been able to reproduce the failure, so I'm not sure if this will help or not. Hopefully, this change will cause the test to fail, rather than hang. That will be better since we will get the rest of the test results. It may also help us debug the real problem. ........ r47015 | neal.norwitz | 2006-06-18 22:10:24 +0200 (Sun, 18 Jun 2006) | 1 line Revert 47014 until it is more robust ........ r47016 | thomas.heller | 2006-06-18 23:27:04 +0200 (Sun, 18 Jun 2006) | 6 lines Fix typos. Fix doctest example. Mention in the tutorial that 'errcheck' is explained in the ref manual. Use better wording in some places. Remoce code examples that shouldn't be in the tutorial. Remove some XXX notices. ........ r47017 | georg.brandl | 2006-06-19 00:17:29 +0200 (Mon, 19 Jun 2006) | 3 lines Patch #1507676: improve exception messages in abstract.c, object.c and typeobject.c. ........ r47018 | neal.norwitz | 2006-06-19 07:40:44 +0200 (Mon, 19 Jun 2006) | 1 line Use Py_ssize_t ........ r47019 | georg.brandl | 2006-06-19 08:35:54 +0200 (Mon, 19 Jun 2006) | 3 lines Add news entry about error msg improvement. ........ r47020 | thomas.heller | 2006-06-19 09:07:49 +0200 (Mon, 19 Jun 2006) | 2 lines Try to repair the failing test on the OpenBSD buildbot. Trial and error... ........ r47021 | tim.peters | 2006-06-19 09:45:16 +0200 (Mon, 19 Jun 2006) | 2 lines Whitespace normalization. ........ r47022 | walter.doerwald | 2006-06-19 10:07:50 +0200 (Mon, 19 Jun 2006) | 4 lines Patch #1506645: add Python wrappers for the curses functions is_term_resized, resize_term and resizeterm. This uses three separate configure checks (one for each function). ........ r47023 | walter.doerwald | 2006-06-19 10:14:09 +0200 (Mon, 19 Jun 2006) | 2 lines Make check order match in configure and configure.in. ........ r47024 | tim.peters | 2006-06-19 10:14:28 +0200 (Mon, 19 Jun 2006) | 3 lines Repair KeyError when running test_threaded_import under -R, as reported by Neal on python-dev. ........ r47025 | thomas.heller | 2006-06-19 10:32:46 +0200 (Mon, 19 Jun 2006) | 3 lines Next try to fix the OpenBSD buildbot tests: Use ctypes.util.find_library to locate the C runtime library on platforms where is returns useful results. ........ r47026 | tim.peters | 2006-06-19 11:09:44 +0200 (Mon, 19 Jun 2006) | 13 lines TestHelp.make_parser(): This was making a permanent change to os.environ (setting envar COLUMNS), which at least caused test_float_default() to fail if the tests were run more than once. This repairs the test_optparse -R failures Neal reported on python-dev. It also explains some seemingly bizarre test_optparse failures we saw a couple weeks ago on the buildbots, when test_optparse failed due to test_file failing to clean up after itself, and then test_optparse failed in an entirely different way when regrtest's -w option ran test_optparse a second time. It's now obvious that make_parser() permanently changing os.environ was responsible for the second half of that. ........ r47027 | anthony.baxter | 2006-06-19 14:04:15 +0200 (Mon, 19 Jun 2006) | 2 lines Preparing for 2.5b1. ........ r47029 | fred.drake | 2006-06-19 19:31:16 +0200 (Mon, 19 Jun 2006) | 1 line remove non-working document formats from edist ........ r47030 | gerhard.haering | 2006-06-19 23:17:35 +0200 (Mon, 19 Jun 2006) | 5 lines Fixed a memory leak that was introduced with incorrect usage of the Python weak reference API in pysqlite 2.2.1. Bumbed pysqlite version number to upcoming pysqlite 2.3.1 release. ........ r47032 | ka-ping.yee | 2006-06-20 00:49:36 +0200 (Tue, 20 Jun 2006) | 2 lines Remove Python 2.3 compatibility comment. ........ r47033 | trent.mick | 2006-06-20 01:21:25 +0200 (Tue, 20 Jun 2006) | 2 lines Upgrade pyexpat to expat 2.0.0 (http://python.org/sf/1462338). ........ r47034 | trent.mick | 2006-06-20 01:57:41 +0200 (Tue, 20 Jun 2006) | 3 lines [ 1295808 ] expat symbols should be namespaced in pyexpat (http://python.org/sf/1295808) ........ r47039 | andrew.kuchling | 2006-06-20 13:52:16 +0200 (Tue, 20 Jun 2006) | 1 line Uncomment wsgiref section ........ r47040 | andrew.kuchling | 2006-06-20 14:15:09 +0200 (Tue, 20 Jun 2006) | 1 line Add four library items ........ r47041 | andrew.kuchling | 2006-06-20 14:19:54 +0200 (Tue, 20 Jun 2006) | 1 line Terminology and typography fixes ........ r47042 | andrew.kuchling | 2006-06-20 15:05:12 +0200 (Tue, 20 Jun 2006) | 1 line Add introductory paragraphs summarizing the release; minor edits ........ r47043 | andrew.kuchling | 2006-06-20 15:11:29 +0200 (Tue, 20 Jun 2006) | 1 line Minor edits and rearrangements; markup fix ........ r47044 | andrew.kuchling | 2006-06-20 15:20:30 +0200 (Tue, 20 Jun 2006) | 1 line [Bug #1504456] Mention xml -> xmlcore change ........ r47047 | brett.cannon | 2006-06-20 19:30:26 +0200 (Tue, 20 Jun 2006) | 2 lines Raise TestSkipped when the test socket connection is refused. ........ r47049 | brett.cannon | 2006-06-20 21:20:17 +0200 (Tue, 20 Jun 2006) | 2 lines Fix typo of exception name. ........ r47053 | brett.cannon | 2006-06-21 18:57:57 +0200 (Wed, 21 Jun 2006) | 5 lines At the C level, tuple arguments are passed in directly to the exception constructor, meaning it is treated as *args, not as a single argument. This means using the 'message' attribute won't work (until Py3K comes around), and so one must grab from 'arg' to get the error number. ........ r47054 | andrew.kuchling | 2006-06-21 19:10:18 +0200 (Wed, 21 Jun 2006) | 1 line Link to LibRef module documentation ........ r47055 | andrew.kuchling | 2006-06-21 19:17:10 +0200 (Wed, 21 Jun 2006) | 1 line Note some of Barry's work ........ r47056 | andrew.kuchling | 2006-06-21 19:17:28 +0200 (Wed, 21 Jun 2006) | 1 line Bump version ........ r47057 | georg.brandl | 2006-06-21 19:45:17 +0200 (Wed, 21 Jun 2006) | 3 lines fix [ 1509132 ] compiler module builds incorrect AST for TryExceptFinally ........ r47058 | georg.brandl | 2006-06-21 19:52:36 +0200 (Wed, 21 Jun 2006) | 3 lines Make test_fcntl aware of netbsd3. ........ r47059 | georg.brandl | 2006-06-21 19:53:17 +0200 (Wed, 21 Jun 2006) | 3 lines Patch #1509001: expected skips for netbsd3. ........ r47060 | gerhard.haering | 2006-06-21 22:55:04 +0200 (Wed, 21 Jun 2006) | 2 lines Removed call to enable_callback_tracebacks that slipped in by accident. ........ r47061 | armin.rigo | 2006-06-21 23:58:50 +0200 (Wed, 21 Jun 2006) | 13 lines Fix for an obscure bug introduced by revs 46806 and 46808, with a test. The problem of checking too eagerly for recursive calls is the following: if a RuntimeError is caused by recursion, and if code needs to normalize it immediately (as in the 2nd test), then PyErr_NormalizeException() needs a call to the RuntimeError class to instantiate it, and this hits the recursion limit again... causing PyErr_NormalizeException() to never finish. Moved this particular recursion check to slot_tp_call(), which is not involved in instantiating built-in exceptions. Backport candidate. ........ r47064 | neal.norwitz | 2006-06-22 08:30:50 +0200 (Thu, 22 Jun 2006) | 3 lines Copy the wsgiref package during make install. ........ r47065 | neal.norwitz | 2006-06-22 08:35:30 +0200 (Thu, 22 Jun 2006) | 1 line Reset the doc date to today for the automatic doc builds ........ r47067 | andrew.kuchling | 2006-06-22 15:10:23 +0200 (Thu, 22 Jun 2006) | 1 line Mention how to suppress warnings ........ r47069 | georg.brandl | 2006-06-22 16:46:17 +0200 (Thu, 22 Jun 2006) | 3 lines Set lineno correctly on list, tuple and dict literals. ........ r47070 | georg.brandl | 2006-06-22 16:46:46 +0200 (Thu, 22 Jun 2006) | 4 lines Test for correct compilation of try-except-finally stmt. Test for correct lineno on list, tuple, dict literals. ........ r47071 | fred.drake | 2006-06-22 17:50:08 +0200 (Thu, 22 Jun 2006) | 1 line fix markup nit ........ r47072 | brett.cannon | 2006-06-22 18:49:14 +0200 (Thu, 22 Jun 2006) | 6 lines 'warning's was improperly requiring that a command-line Warning category be both a subclass of Warning and a subclass of types.ClassType. The latter is no longer true thanks to new-style exceptions. Closes bug #1510580. Thanks to AMK for the test. ........ r47073 | ronald.oussoren | 2006-06-22 20:33:54 +0200 (Thu, 22 Jun 2006) | 3 lines MacOSX: Add a message to the first screen of the installer that tells users how to avoid updates to their shell profile. ........ r47074 | georg.brandl | 2006-06-22 21:02:18 +0200 (Thu, 22 Jun 2006) | 3 lines Fix my name ;) ........ r47075 | thomas.heller | 2006-06-22 21:07:36 +0200 (Thu, 22 Jun 2006) | 2 lines Small fixes, mostly in the markup. ........ r47076 | peter.astrand | 2006-06-22 22:06:46 +0200 (Thu, 22 Jun 2006) | 1 line Make it possible to run test_subprocess.py on Python 2.2, which lacks test_support.is_resource_enabled. ........ r47077 | peter.astrand | 2006-06-22 22:21:26 +0200 (Thu, 22 Jun 2006) | 1 line Applied patch #1506758: Prevent MemoryErrors with large MAXFD. ........ r47079 | neal.norwitz | 2006-06-23 05:32:44 +0200 (Fri, 23 Jun 2006) | 1 line Fix refleak ........ r47080 | fred.drake | 2006-06-23 08:03:45 +0200 (Fri, 23 Jun 2006) | 9 lines - SF bug #853506: IP6 address parsing in sgmllib ('[' and ']' were not accepted in unquoted attribute values) - cleaned up tests of character and entity reference decoding so the tests cover the documented relationships among handle_charref, handle_entityref, convert_charref, convert_codepoint, and convert_entityref, without bringing up Unicode issues that sgmllib cannot be involved in ........ r47085 | andrew.kuchling | 2006-06-23 21:23:40 +0200 (Fri, 23 Jun 2006) | 11 lines Fit Makefile for the Python doc environment better; this is a step toward including the howtos in the build process. * Put LaTeX output in ../paper-/. * Put HTML output in ../html/ * Explain some of the Makefile variables * Remove some cruft dating to my environment (e.g. the 'web' target) This makefile isn't currently invoked by the documentation build process, so these changes won't destabilize anything. ........ r47086 | hyeshik.chang | 2006-06-23 23:16:18 +0200 (Fri, 23 Jun 2006) | 5 lines Bug #1511381: codec_getstreamcodec() in codec.c is corrected to omit a default "error" argument for NULL pointer. This allows the parser to take a codec from cjkcodecs again. (Reported by Taewook Kang and reviewed by Walter Doerwald) ........ r47091 | ronald.oussoren | 2006-06-25 22:44:16 +0200 (Sun, 25 Jun 2006) | 6 lines Workaround for bug #1512124 Without this patch IDLE will get unresponsive when you open the debugger window on OSX. This is both using the system Tcl/Tk on Tiger as the latest universal download from tk-components.sf.net. ........ r47092 | ronald.oussoren | 2006-06-25 23:14:19 +0200 (Sun, 25 Jun 2006) | 3 lines Drop the calldll demo's for macos, calldll isn't present anymore, no need to keep the demo's around. ........ r47093 | ronald.oussoren | 2006-06-25 23:15:58 +0200 (Sun, 25 Jun 2006) | 3 lines Use a path without a double slash to compile the .py files after installation (macosx, binary installer). This fixes bug #1508369 for python 2.5. ........ r47094 | ronald.oussoren | 2006-06-25 23:19:06 +0200 (Sun, 25 Jun 2006) | 3 lines Also install the .egg-info files in Lib. This will cause wsgiref.egg-info to be installed. ........ r47097 | andrew.kuchling | 2006-06-26 14:40:02 +0200 (Mon, 26 Jun 2006) | 1 line [Bug #1511998] Various comments from Nick Coghlan; thanks! ........ r47098 | andrew.kuchling | 2006-06-26 14:43:43 +0200 (Mon, 26 Jun 2006) | 1 line Describe workaround for PyRange_New()'s removal ........ r47099 | andrew.kuchling | 2006-06-26 15:08:24 +0200 (Mon, 26 Jun 2006) | 5 lines [Bug #1512163] Fix typo. This change will probably break tests on FreeBSD buildbots, but I'll check in a fix for that next. ........ r47100 | andrew.kuchling | 2006-06-26 15:12:16 +0200 (Mon, 26 Jun 2006) | 9 lines [Bug #1512163] Use one set of locking methods, lockf(); remove the flock() calls. On FreeBSD, the two methods lockf() and flock() end up using the same mechanism and the second one fails. A Linux man page claims that the two methods are orthogonal (so locks acquired one way don't interact with locks acquired the other way) but that clearly must be false. ........ r47101 | andrew.kuchling | 2006-06-26 15:23:10 +0200 (Mon, 26 Jun 2006) | 5 lines Add a test for a conflicting lock. On slow machines, maybe the time intervals (2 sec, 0.5 sec) will be too tight. I'll see how the buildbots like it. ........ r47103 | andrew.kuchling | 2006-06-26 16:33:24 +0200 (Mon, 26 Jun 2006) | 1 line Windows doesn't have os.fork(). I'll just disable this test for now ........ r47106 | andrew.kuchling | 2006-06-26 19:00:35 +0200 (Mon, 26 Jun 2006) | 9 lines Attempt to fix build failure on OS X and Debian alpha; the symptom is consistent with os.wait() returning immediately because some other subprocess had previously exited; the test suite then immediately tries to lock the mailbox and gets an error saying it's already locked. To fix this, do a waitpid() so the test suite only continues once the intended child process has exited. ........ r47113 | neal.norwitz | 2006-06-27 06:06:46 +0200 (Tue, 27 Jun 2006) | 1 line Ignore some more warnings in the dynamic linker on an older gentoo ........ r47114 | neal.norwitz | 2006-06-27 06:09:13 +0200 (Tue, 27 Jun 2006) | 6 lines Instead of doing a make test, run the regression tests out of the installed copy. This will hopefully catch problems where directories are added under Lib/ but not to Makefile.pre.in. This breaks out the 2 runs of the test suite with and without -O which is also nicer. ........ r47115 | neal.norwitz | 2006-06-27 06:12:58 +0200 (Tue, 27 Jun 2006) | 5 lines Fix SF bug #1513032, 'make install' failure on FreeBSD 5.3. No need to install lib-old, it's empty in 2.5. ........ r47116 | neal.norwitz | 2006-06-27 06:23:06 +0200 (Tue, 27 Jun 2006) | 1 line Test unimportant change to verify buildbot does not try to build ........ r47117 | neal.norwitz | 2006-06-27 06:26:30 +0200 (Tue, 27 Jun 2006) | 1 line Try again: test unimportant change to verify buildbot does not try to build ........ r47118 | neal.norwitz | 2006-06-27 06:28:56 +0200 (Tue, 27 Jun 2006) | 1 line Verify buildbot picks up these changes (really needs testing after last change to Makefile.pre.in) ........ r47121 | vinay.sajip | 2006-06-27 09:34:37 +0200 (Tue, 27 Jun 2006) | 1 line Removed buggy exception handling in doRollover of rotating file handlers. Exceptions now propagate to caller. ........ r47123 | ronald.oussoren | 2006-06-27 12:08:25 +0200 (Tue, 27 Jun 2006) | 3 lines MacOSX: fix rather dumb buglet that made it impossible to create extensions on OSX 10.3 when using a binary distribution build on 10.4. ........ r47125 | tim.peters | 2006-06-27 13:52:49 +0200 (Tue, 27 Jun 2006) | 2 lines Whitespace normalization. ........ r47128 | ronald.oussoren | 2006-06-27 14:53:52 +0200 (Tue, 27 Jun 2006) | 8 lines Use staticly build copies of zlib and bzip2 to build the OSX installer, that way the resulting binaries have a better change of running on 10.3. This patch also updates the search logic for sleepycat db3/4, without this patch you cannot use a sleepycat build with a non-standard prefix; with this you can (at least on OSX) if you add the prefix to CPPFLAGS/LDFLAGS at configure-time. This change is needed to build the binary installer for OSX. ........ r47131 | ronald.oussoren | 2006-06-27 17:45:32 +0200 (Tue, 27 Jun 2006) | 5 lines macosx: Install a libpython2.5.a inside the framework as a symlink to the actual dylib at the root of the framework, that way tools that expect a unix-like install (python-config, but more importantly external products like mod_python) work correctly. ........ r47137 | neal.norwitz | 2006-06-28 07:03:22 +0200 (Wed, 28 Jun 2006) | 4 lines According to the man pages on Gentoo Linux and Tru64, EACCES or EAGAIN can be returned if fcntl (lockf) fails. This fixes the test failure on Tru64 by checking for either error rather than just EAGAIN. ........ r47139 | neal.norwitz | 2006-06-28 08:28:31 +0200 (Wed, 28 Jun 2006) | 5 lines Fix bug #1512695: cPickle.loads could crash if it was interrupted with a KeyboardInterrupt since PyTuple_Pack was passed a NULL. Will backport. ........ r47142 | nick.coghlan | 2006-06-28 12:41:47 +0200 (Wed, 28 Jun 2006) | 1 line Make full module name available as __module_name__ even when __name__ is set to something else (like '__main__') ........ r47143 | armin.rigo | 2006-06-28 12:49:51 +0200 (Wed, 28 Jun 2006) | 2 lines A couple of crashers of the "won't fix" kind. ........ r47147 | andrew.kuchling | 2006-06-28 16:25:20 +0200 (Wed, 28 Jun 2006) | 1 line [Bug #1508766] Add docs for uuid module; docs written by George Yoshida, with minor rearrangements by me. ........ r47148 | andrew.kuchling | 2006-06-28 16:27:21 +0200 (Wed, 28 Jun 2006) | 1 line [Bug #1508766] Add docs for uuid module; this puts the module in the 'Internet Protocols' section. Arguably this module could also have gone in the chapters on strings or encodings, maybe even the crypto chapter. Fred, please move if you see fit. ........ r47151 | georg.brandl | 2006-06-28 22:23:25 +0200 (Wed, 28 Jun 2006) | 3 lines Fix end_fill(). ........ r47153 | trent.mick | 2006-06-28 22:30:41 +0200 (Wed, 28 Jun 2006) | 2 lines Mention the expat upgrade and pyexpat fix I put in 2.5b1. ........ r47154 | fred.drake | 2006-06-29 02:51:53 +0200 (Thu, 29 Jun 2006) | 6 lines SF bug #1504333: sgmlib should allow angle brackets in quoted values (modified patch by Sam Ruby; changed to use separate REs for start and end tags to reduce matching cost for end tags; extended tests; updated to avoid breaking previous changes to support IPv6 addresses in unquoted attribute values) ........ r47156 | fred.drake | 2006-06-29 04:57:48 +0200 (Thu, 29 Jun 2006) | 1 line document recent bugfixes in sgmllib ........ r47158 | neal.norwitz | 2006-06-29 06:10:08 +0200 (Thu, 29 Jun 2006) | 10 lines Add new utility function, reap_children(), to test_support. This should be called at the end of each test that spawns children (perhaps it should be called from regrtest instead?). This will hopefully prevent some of the unexplained failures in the buildbots (hppa and alpha) during tests that spawn children. The problems were not reproducible. There were many zombies that remained at the end of several tests. In the worst case, this shouldn't cause any more problems, though it may not help either. Time will tell. ........ r47159 | neal.norwitz | 2006-06-29 07:48:14 +0200 (Thu, 29 Jun 2006) | 5 lines This should fix the buildbot failure on s/390 which can't connect to gmail.org. It makes the error message consistent and always sends to stderr. It would be much better for all the networking tests to hit only python.org. ........ r47161 | thomas.heller | 2006-06-29 20:34:15 +0200 (Thu, 29 Jun 2006) | 3 lines Protect the thread api calls in the _ctypes extension module within #ifdef WITH_THREADS/#endif blocks. Found by Sam Rushing. ........ r47162 | martin.v.loewis | 2006-06-29 20:58:44 +0200 (Thu, 29 Jun 2006) | 2 lines Patch #1509163: MS Toolkit Compiler no longer available ........ r47163 | skip.montanaro | 2006-06-29 21:20:09 +0200 (Thu, 29 Jun 2006) | 1 line add string methods to index ........ r47164 | vinay.sajip | 2006-06-30 02:13:08 +0200 (Fri, 30 Jun 2006) | 1 line Fixed bug in fileConfig() which failed to clear logging._handlerList ........ r47166 | tim.peters | 2006-06-30 08:18:39 +0200 (Fri, 30 Jun 2006) | 2 lines Whitespace normalization. ........ r47170 | neal.norwitz | 2006-06-30 09:32:16 +0200 (Fri, 30 Jun 2006) | 1 line Silence compiler warning ........ r47171 | neal.norwitz | 2006-06-30 09:32:46 +0200 (Fri, 30 Jun 2006) | 1 line Another problem reported by Coverity. Backport candidate. ........ r47175 | thomas.heller | 2006-06-30 19:44:54 +0200 (Fri, 30 Jun 2006) | 2 lines Revert the use of PY_FORMAT_SIZE_T in PyErr_Format. ........ r47176 | tim.peters | 2006-06-30 20:34:51 +0200 (Fri, 30 Jun 2006) | 2 lines Remove now-unused fidding with PY_FORMAT_SIZE_T. ........ r47177 | georg.brandl | 2006-06-30 20:47:56 +0200 (Fri, 30 Jun 2006) | 3 lines Document decorator usage of property. ........ r47181 | fred.drake | 2006-06-30 21:29:25 +0200 (Fri, 30 Jun 2006) | 4 lines - consistency nit: always include "()" in \function and \method (*should* be done by the presentation, but that requires changes all over) - avoid spreading the __name meme ........ r47188 | vinay.sajip | 2006-07-01 12:45:20 +0200 (Sat, 01 Jul 2006) | 1 line Added entry for fileConfig() bugfix. ........ r47189 | vinay.sajip | 2006-07-01 12:47:20 +0200 (Sat, 01 Jul 2006) | 1 line Added duplicate call to fileConfig() to ensure that it cleans up after itself correctly. ........ r47190 | martin.v.loewis | 2006-07-01 17:33:37 +0200 (Sat, 01 Jul 2006) | 2 lines Release all forwarded functions in .close. Fixes #1513223. ........ r47191 | fred.drake | 2006-07-01 18:28:20 +0200 (Sat, 01 Jul 2006) | 7 lines SF bug #1296433 (Expat bug #1515266): Unchecked calls to character data handler would cause a segfault. This merges in Expat's lib/xmlparse.c revisions 1.154 and 1.155, which fix this and a closely related problem (the later does not affect Python). Moved the crasher test to the tests for xml.parsers.expat. ........ r47197 | gerhard.haering | 2006-07-02 19:48:30 +0200 (Sun, 02 Jul 2006) | 4 lines The sqlite3 module did cut off data from the SQLite database at the first null character before sending it to a custom converter. This has been fixed now. ........ r47198 | martin.v.loewis | 2006-07-02 20:44:00 +0200 (Sun, 02 Jul 2006) | 1 line Correct arithmetic in access on Win32. Fixes #1513646. ........ r47203 | thomas.heller | 2006-07-03 09:58:09 +0200 (Mon, 03 Jul 2006) | 1 line Cleanup: Remove commented out code. ........ r47204 | thomas.heller | 2006-07-03 09:59:50 +0200 (Mon, 03 Jul 2006) | 1 line Don't run the doctests with Python 2.3 because it doesn't have the ELLIPSIS flag. ........ r47205 | thomas.heller | 2006-07-03 10:04:05 +0200 (Mon, 03 Jul 2006) | 7 lines Fixes so that _ctypes can be compiled with the MingW compiler. It seems that the definition of '__attribute__(x)' was responsible for the compiler ignoring the '__fastcall' attribute on the ffi_closure_SYSV function in libffi_msvc/ffi.c, took me quite some time to figure this out. ........ r47206 | thomas.heller | 2006-07-03 10:08:14 +0200 (Mon, 03 Jul 2006) | 11 lines Add a new function uses_seh() to the _ctypes extension module. This will return True if Windows Structured Exception handling (SEH) is used when calling functions, False otherwise. Currently, only MSVC supports SEH. Fix the test so that it doesn't crash when run with MingW compiled _ctypes. Note that two tests are still failing when mingw is used, I suspect structure layout differences and function calling conventions between MSVC and MingW. ........ r47207 | tim.peters | 2006-07-03 10:23:19 +0200 (Mon, 03 Jul 2006) | 2 lines Whitespace normalization. ........ r47208 | martin.v.loewis | 2006-07-03 11:44:00 +0200 (Mon, 03 Jul 2006) | 3 lines Only setup canvas when it is first created. Fixes #1514703 ........ r47209 | martin.v.loewis | 2006-07-03 12:05:30 +0200 (Mon, 03 Jul 2006) | 3 lines Reimplement turtle.circle using a polyline, to allow correct filling of arcs. Also fixes #1514693. ........ r47210 | martin.v.loewis | 2006-07-03 12:19:49 +0200 (Mon, 03 Jul 2006) | 3 lines Bug #1514693: Update turtle's heading when switching between degrees and radians. ........ r47211 | martin.v.loewis | 2006-07-03 13:12:06 +0200 (Mon, 03 Jul 2006) | 2 lines Document functions added in 2.3 and 2.5. ........ r47212 | martin.v.loewis | 2006-07-03 14:19:50 +0200 (Mon, 03 Jul 2006) | 3 lines Bug #1417699: Reject locale-specific decimal point in float() and atof(). ........ r47213 | martin.v.loewis | 2006-07-03 14:28:58 +0200 (Mon, 03 Jul 2006) | 3 lines Bug #1267547: Put proper recursive setup.py call into the spec file generated by bdist_rpm. ........ r47215 | martin.v.loewis | 2006-07-03 15:01:35 +0200 (Mon, 03 Jul 2006) | 3 lines Patch #825417: Fix timeout processing in expect, read_until. Will backport to 2.4. ........ r47218 | martin.v.loewis | 2006-07-03 15:47:40 +0200 (Mon, 03 Jul 2006) | 2 lines Put method-wrappers into trashcan. Fixes #927248. ........ r47219 | andrew.kuchling | 2006-07-03 16:07:30 +0200 (Mon, 03 Jul 2006) | 1 line [Bug #1515932] Clarify description of slice assignment ........ r47220 | andrew.kuchling | 2006-07-03 16:16:09 +0200 (Mon, 03 Jul 2006) | 4 lines [Bug #1511911] Clarify description of optional arguments to sorted() by improving the xref to the section on lists, and by copying the explanations of the arguments (with a slight modification). ........ r47223 | kristjan.jonsson | 2006-07-03 16:59:05 +0200 (Mon, 03 Jul 2006) | 1 line Fix build problems with the platform SDK on windows. It is not sufficient to test for the C compiler version when determining if we have the secure CRT from microsoft. Must test with an undocumented macro, __STDC_SECURE_LIB__ too. ........ r47224 | ronald.oussoren | 2006-07-04 14:30:22 +0200 (Tue, 04 Jul 2006) | 7 lines Sync the darwin/x86 port libffi with the copy in PyObjC. This fixes a number of bugs in that port. The most annoying ones were due to some subtle differences between the document ABI and the actual implementation :-( (there are no python unittests that fail without this patch, but without it some of libffi's unittests fail). ........ r47234 | georg.brandl | 2006-07-05 10:21:00 +0200 (Wed, 05 Jul 2006) | 3 lines Remove remaining references to OverflowWarning. ........ r47236 | thomas.heller | 2006-07-05 11:13:56 +0200 (Wed, 05 Jul 2006) | 3 lines Fix the bitfield test when _ctypes is compiled with MingW. Structures containing bitfields may have different layout on MSVC and MingW . ........ r47237 | thomas.wouters | 2006-07-05 13:03:49 +0200 (Wed, 05 Jul 2006) | 15 lines Fix bug in passing tuples to string.Template. All other values (with working str() or repr()) would work, just not multi-value tuples. Probably not a backport candidate, since it changes the behaviour of passing a single-element tuple: >>> string.Template("$foo").substitute(dict(foo=(1,))) '(1,)' versus '1' ........ r47241 | georg.brandl | 2006-07-05 16:18:45 +0200 (Wed, 05 Jul 2006) | 2 lines Patch #1517490: fix glitches in filter() docs. ........ r47244 | georg.brandl | 2006-07-05 17:50:05 +0200 (Wed, 05 Jul 2006) | 2 lines no need to elaborate "string". ........ r47251 | neal.norwitz | 2006-07-06 06:28:59 +0200 (Thu, 06 Jul 2006) | 3 lines Fix refleaks reported by Shane Hathaway in SF patch #1515361. This change contains only the changes related to leaking the copy variable. ........ r47253 | fred.drake | 2006-07-06 07:13:22 +0200 (Thu, 06 Jul 2006) | 4 lines - back out Expat change; the final fix to Expat will be different - change the pyexpat wrapper to not be so sensitive to this detail of the Expat implementation (the ex-crasher test still passes) ........ r47257 | neal.norwitz | 2006-07-06 08:45:08 +0200 (Thu, 06 Jul 2006) | 1 line Add a NEWS entry for a recent pyexpat fix ........ r47258 | martin.v.loewis | 2006-07-06 08:55:58 +0200 (Thu, 06 Jul 2006) | 2 lines Add sqlite3.dll to the DLLs component, not to the TkDLLs component. Fixes #1517388. ........ r47259 | martin.v.loewis | 2006-07-06 09:05:21 +0200 (Thu, 06 Jul 2006) | 1 line Properly quote compileall and Lib paths in case TARGETDIR has a space. ........ r47260 | thomas.heller | 2006-07-06 09:50:18 +0200 (Thu, 06 Jul 2006) | 5 lines Revert the change done in svn revision 47206: Add a new function uses_seh() to the _ctypes extension module. This will return True if Windows Structured Exception handling (SEH) is used when calling functions, False otherwise. ........ r47261 | armin.rigo | 2006-07-06 09:58:18 +0200 (Thu, 06 Jul 2006) | 3 lines A couple of examples about how to attack the fact that _PyType_Lookup() returns a borrowed ref. Many of the calls are open to attack. ........ r47262 | thomas.heller | 2006-07-06 10:28:14 +0200 (Thu, 06 Jul 2006) | 2 lines The test that calls a function with invalid arguments and catches the resulting Windows access violation will not be run by default. ........ r47263 | thomas.heller | 2006-07-06 10:48:35 +0200 (Thu, 06 Jul 2006) | 5 lines Patch #1517790: It is now possible to use custom objects in the ctypes foreign function argtypes sequence as long as they provide a from_param method, no longer is it required that the object is a ctypes type. ........ r47264 | thomas.heller | 2006-07-06 10:58:40 +0200 (Thu, 06 Jul 2006) | 2 lines Document the Struture and Union constructors. ........ r47265 | thomas.heller | 2006-07-06 11:11:22 +0200 (Thu, 06 Jul 2006) | 2 lines Document the changes in svn revision 47263, from patch #1517790. ........ r47267 | ronald.oussoren | 2006-07-06 12:13:35 +0200 (Thu, 06 Jul 2006) | 7 lines This patch solves the problem Skip was seeing with zlib, this patch ensures that configure uses similar compiler flags as setup.py when doing the zlib test. Without this patch configure would use the first shared library on the linker path, with this patch it uses the first shared or static library on that path just like setup.py. ........ r47268 | thomas.wouters | 2006-07-06 12:48:28 +0200 (Thu, 06 Jul 2006) | 4 lines NEWS entry for r47267: fixing configure's zlib probing. ........ r47269 | fredrik.lundh | 2006-07-06 14:29:24 +0200 (Thu, 06 Jul 2006) | 3 lines added XMLParser alias for cElementTree compatibility ........ r47271 | nick.coghlan | 2006-07-06 14:53:04 +0200 (Thu, 06 Jul 2006) | 1 line Revert the __module_name__ changes made in rev 47142. We'll revisit this in Python 2.6 ........ r47272 | nick.coghlan | 2006-07-06 15:04:56 +0200 (Thu, 06 Jul 2006) | 1 line Update the tutorial section on relative imports ........ r47273 | nick.coghlan | 2006-07-06 15:35:27 +0200 (Thu, 06 Jul 2006) | 1 line Ignore ImportWarning by default ........ r47274 | nick.coghlan | 2006-07-06 15:41:34 +0200 (Thu, 06 Jul 2006) | 1 line Cover ImportWarning, PendingDeprecationWarning and simplefilter() in the warnings module docs ........ r47275 | nick.coghlan | 2006-07-06 15:47:18 +0200 (Thu, 06 Jul 2006) | 1 line Add NEWS entries for the ImportWarning change and documentation update ........ r47276 | andrew.kuchling | 2006-07-06 15:57:28 +0200 (Thu, 06 Jul 2006) | 1 line ImportWarning is now silent by default ........ r47277 | thomas.heller | 2006-07-06 17:06:05 +0200 (Thu, 06 Jul 2006) | 2 lines Document the correct return type of PyLong_AsUnsignedLongLongMask. ........ r47278 | hyeshik.chang | 2006-07-06 17:21:52 +0200 (Thu, 06 Jul 2006) | 2 lines Add a testcase for r47086 which fixed a bug in codec_getstreamcodec(). ........ r47279 | hyeshik.chang | 2006-07-06 17:39:24 +0200 (Thu, 06 Jul 2006) | 3 lines Test using all CJK encodings for the testcases which don't require specific encodings. ........ r47280 | martin.v.loewis | 2006-07-06 21:28:03 +0200 (Thu, 06 Jul 2006) | 2 lines Properly generate logical file ids. Fixes #1515998. Also correct typo in Control.mapping. ........ r47287 | neal.norwitz | 2006-07-07 08:03:15 +0200 (Fri, 07 Jul 2006) | 17 lines Restore rev 47014: The hppa ubuntu box sometimes hangs forever in these tests. My guess is that the wait is failing for some reason. Use WNOHANG, so we won't wait until the buildbot kills the test suite. I haven't been able to reproduce the failure, so I'm not sure if this will help or not. Hopefully, this change will cause the test to fail, rather than hang. That will be better since we will get the rest of the test results. It may also help us debug the real problem. *** The reason this originally failed was because there were many zombie children outstanding before rev 47158 cleaned them up. There are still hangs in test_subprocess that need to be addressed, but that will take more work. This should close some holes. ........ r47289 | georg.brandl | 2006-07-07 10:15:12 +0200 (Fri, 07 Jul 2006) | 3 lines Fix RFC number. ........ r50489 | neal.norwitz | 2006-07-08 07:31:37 +0200 (Sat, 08 Jul 2006) | 1 line Fix SF bug #1519018: 'as' is now validated properly in import statements ........ r50490 | georg.brandl | 2006-07-08 14:15:27 +0200 (Sat, 08 Jul 2006) | 3 lines Add an additional test for bug #1519018. ........ r50491 | tim.peters | 2006-07-08 21:55:05 +0200 (Sat, 08 Jul 2006) | 2 lines Whitespace normalization. ........ r50493 | neil.schemenauer | 2006-07-09 18:16:34 +0200 (Sun, 09 Jul 2006) | 2 lines Fix AST compiler bug #1501934: incorrect LOAD/STORE_GLOBAL generation. ........ r50495 | neil.schemenauer | 2006-07-09 23:19:29 +0200 (Sun, 09 Jul 2006) | 2 lines Fix SF bug 1441486: bad unary minus folding in compiler. ........ r50497 | neal.norwitz | 2006-07-10 00:14:42 +0200 (Mon, 10 Jul 2006) | 4 lines On 64 bit systems, int literals that use less than 64 bits are now ints rather than longs. This also fixes the test for eval(-sys.maxint - 1). ........ r50500 | neal.norwitz | 2006-07-10 02:04:44 +0200 (Mon, 10 Jul 2006) | 4 lines Bug #1512814, Fix incorrect lineno's when code at module scope started after line 256. ........ r50501 | neal.norwitz | 2006-07-10 02:05:34 +0200 (Mon, 10 Jul 2006) | 1 line Fix doco. Backport candidate. ........ r50503 | neal.norwitz | 2006-07-10 02:23:17 +0200 (Mon, 10 Jul 2006) | 5 lines Part of SF patch #1484695. This removes dead code. The chksum was already verified in .frombuf() on the lines above. If there was a problem an exception is raised, so there was no way this condition could have been true. ........ r50504 | neal.norwitz | 2006-07-10 03:18:57 +0200 (Mon, 10 Jul 2006) | 3 lines Patch #1516912: improve Modules support for OpenVMS. ........ r50506 | neal.norwitz | 2006-07-10 04:36:41 +0200 (Mon, 10 Jul 2006) | 7 lines Patch #1504046: Add documentation for xml.etree. /F wrote the text docs, Englebert Gruber massaged it to latex and I did some more massaging to try and improve the consistency and fix some name mismatches between the declaration and text. ........ r50509 | martin.v.loewis | 2006-07-10 09:23:48 +0200 (Mon, 10 Jul 2006) | 2 lines Introduce DISTUTILS_USE_SDK as a flag to determine whether the SDK environment should be used. Fixes #1508010. ........ r50510 | martin.v.loewis | 2006-07-10 09:26:41 +0200 (Mon, 10 Jul 2006) | 1 line Change error message to indicate that VS2003 is necessary to build extension modules, not the .NET SDK. ........ r50511 | martin.v.loewis | 2006-07-10 09:29:41 +0200 (Mon, 10 Jul 2006) | 1 line Add svn:ignore. ........ r50512 | anthony.baxter | 2006-07-10 09:41:04 +0200 (Mon, 10 Jul 2006) | 1 line preparing for 2.5b2 ........ r50513 | thomas.heller | 2006-07-10 11:10:28 +0200 (Mon, 10 Jul 2006) | 2 lines Fix bug #1518190: accept any integer or long value in the ctypes.c_void_p constructor. ........ r50514 | thomas.heller | 2006-07-10 11:31:06 +0200 (Mon, 10 Jul 2006) | 3 lines Fixed a segfault when ctypes.wintypes were imported on non-Windows machines. ........ r50516 | thomas.heller | 2006-07-10 13:11:10 +0200 (Mon, 10 Jul 2006) | 3 lines Assigning None to pointer type structure fields possible overwrote wrong fields. ........ r50517 | thomas.heller | 2006-07-10 13:17:37 +0200 (Mon, 10 Jul 2006) | 5 lines Moved the ctypes news entries from the 'Library' section into the 'Extension Modules' section where they belong, probably. This destroyes the original order of the news entries, don't know if that is important or not. ........ r50526 | phillip.eby | 2006-07-10 21:03:29 +0200 (Mon, 10 Jul 2006) | 2 lines Fix SF#1516184 and add a test to prevent regression. ........ r50528 | phillip.eby | 2006-07-10 21:18:35 +0200 (Mon, 10 Jul 2006) | 2 lines Fix SF#1457312: bad socket error handling in distutils "upload" command. ........ r50537 | peter.astrand | 2006-07-10 22:39:49 +0200 (Mon, 10 Jul 2006) | 1 line Make it possible to run test_subprocess.py with Python 2.2, which lacks test_support.reap_children(). ........ r50541 | tim.peters | 2006-07-10 23:08:24 +0200 (Mon, 10 Jul 2006) | 5 lines After approval from Anthony, merge the tim-current_frames branch into the trunk. This adds a new sys._current_frames() function, which returns a dict mapping thread id to topmost thread stack frame. ........ r50542 | tim.peters | 2006-07-10 23:11:49 +0200 (Mon, 10 Jul 2006) | 2 lines Whitespace normalization. ........ r50553 | martin.v.loewis | 2006-07-11 00:11:28 +0200 (Tue, 11 Jul 2006) | 4 lines Patch #1519566: Remove unused _tofill member. Make begin_fill idempotent. Update demo2 to demonstrate filling of concave shapes. ........ r50567 | anthony.baxter | 2006-07-11 04:04:09 +0200 (Tue, 11 Jul 2006) | 4 lines #1494314: Fix a regression with high-numbered sockets in 2.4.3. This means that select() on sockets > FD_SETSIZE (typically 1024) work again. The patch makes sockets use poll() internally where available. ........ r50568 | tim.peters | 2006-07-11 04:17:48 +0200 (Tue, 11 Jul 2006) | 2 lines Whitespace normalization. ........ r50575 | thomas.heller | 2006-07-11 18:42:05 +0200 (Tue, 11 Jul 2006) | 1 line Add missing Py_DECREF. ........ r50576 | thomas.heller | 2006-07-11 18:44:25 +0200 (Tue, 11 Jul 2006) | 1 line Add missing Py_DECREFs. ........ r50579 | andrew.kuchling | 2006-07-11 19:20:16 +0200 (Tue, 11 Jul 2006) | 1 line Bump version number; add sys._current_frames ........ r50582 | thomas.heller | 2006-07-11 20:28:35 +0200 (Tue, 11 Jul 2006) | 3 lines When a foreign function is retrived by calling __getitem__ on a ctypes library instance, do not set it as attribute. ........ r50583 | thomas.heller | 2006-07-11 20:40:50 +0200 (Tue, 11 Jul 2006) | 2 lines Change the ctypes version number to 1.0.0. ........ r50597 | neal.norwitz | 2006-07-12 07:26:17 +0200 (Wed, 12 Jul 2006) | 3 lines Bug #1520864: unpacking singleton tuples in for loop (for x, in) work again. ........ r50598 | neal.norwitz | 2006-07-12 07:26:35 +0200 (Wed, 12 Jul 2006) | 1 line Fix function name in error msg ........ r50599 | neal.norwitz | 2006-07-12 07:27:46 +0200 (Wed, 12 Jul 2006) | 4 lines Fix uninitialized memory read reported by Valgrind when running doctest. This could happen if size == 0. ........ r50600 | neal.norwitz | 2006-07-12 09:28:29 +0200 (Wed, 12 Jul 2006) | 1 line Actually change the MAGIC #. Create a new section for 2.5c1 and mention the impact of changing the MAGIC #. ........ r50601 | thomas.heller | 2006-07-12 10:43:47 +0200 (Wed, 12 Jul 2006) | 3 lines Fix #1467450: ctypes now uses RTLD_GLOBAL by default on OSX 10.3 to load shared libraries. ........ r50604 | thomas.heller | 2006-07-12 16:25:18 +0200 (Wed, 12 Jul 2006) | 3 lines Fix the wrong description of LibraryLoader.LoadLibrary, and document the DEFAULT_MODE constant. ........ r50607 | georg.brandl | 2006-07-12 17:31:17 +0200 (Wed, 12 Jul 2006) | 3 lines Accept long options "--help" and "--version". ........ r50617 | thomas.heller | 2006-07-13 11:53:47 +0200 (Thu, 13 Jul 2006) | 3 lines A misspelled preprocessor symbol caused ctypes to be always compiled without thread support. Replaced WITH_THREADS with WITH_THREAD. ........ r50619 | thomas.heller | 2006-07-13 19:01:14 +0200 (Thu, 13 Jul 2006) | 3 lines Fix #1521375. When running with root priviledges, 'gcc -o /dev/null' did overwrite /dev/null. Use a temporary file instead of /dev/null. ........ r50620 | thomas.heller | 2006-07-13 19:05:13 +0200 (Thu, 13 Jul 2006) | 2 lines Fix misleading words. ........ r50622 | andrew.kuchling | 2006-07-13 19:37:26 +0200 (Thu, 13 Jul 2006) | 1 line Typo fix ........ r50629 | georg.brandl | 2006-07-14 09:12:54 +0200 (Fri, 14 Jul 2006) | 3 lines Patch #1521874: grammar errors in doanddont.tex. ........ r50630 | neal.norwitz | 2006-07-14 09:20:04 +0200 (Fri, 14 Jul 2006) | 1 line Try to improve grammar further. ........ r50631 | martin.v.loewis | 2006-07-14 11:58:55 +0200 (Fri, 14 Jul 2006) | 1 line Extend build_ssl to Win64, using VSExtComp. ........ r50632 | martin.v.loewis | 2006-07-14 14:10:09 +0200 (Fri, 14 Jul 2006) | 1 line Add debug output to analyse buildbot failure. ........ r50633 | martin.v.loewis | 2006-07-14 14:31:05 +0200 (Fri, 14 Jul 2006) | 1 line Fix Debug build of _ssl. ........ r50636 | andrew.kuchling | 2006-07-14 15:32:38 +0200 (Fri, 14 Jul 2006) | 1 line Mention new options ........ r50638 | peter.astrand | 2006-07-14 16:04:45 +0200 (Fri, 14 Jul 2006) | 1 line Bug #1223937: CalledProcessError.errno -> CalledProcessError.returncode. ........ r50640 | thomas.heller | 2006-07-14 17:01:05 +0200 (Fri, 14 Jul 2006) | 4 lines Make the prototypes of our private PyUnicode_FromWideChar and PyUnicode_AsWideChar replacement functions compatible to the official functions by using Py_ssize_t instead of int. ........ r50643 | thomas.heller | 2006-07-14 19:51:14 +0200 (Fri, 14 Jul 2006) | 3 lines Patch #1521817: The index range checking on ctypes arrays containing exactly one element is enabled again. ........ r50647 | thomas.heller | 2006-07-14 20:22:50 +0200 (Fri, 14 Jul 2006) | 2 lines Updates for the ctypes documentation. ........ r50655 | fredrik.lundh | 2006-07-14 23:45:48 +0200 (Fri, 14 Jul 2006) | 3 lines typo ........ r50664 | george.yoshida | 2006-07-15 18:03:49 +0200 (Sat, 15 Jul 2006) | 2 lines Bug #15187702 : ext/win-cookbook.html has a broken link to distutils ........ r50667 | bob.ippolito | 2006-07-15 18:53:15 +0200 (Sat, 15 Jul 2006) | 1 line Patch #1220874: Update the binhex module for Mach-O. ........ r50671 | fred.drake | 2006-07-16 03:21:20 +0200 (Sun, 16 Jul 2006) | 1 line clean up some link markup ........ r50673 | neal.norwitz | 2006-07-16 03:50:38 +0200 (Sun, 16 Jul 2006) | 4 lines Bug #1512814, Fix incorrect lineno's when code within a function had more than 255 blank lines. Byte codes need to go first, line #s second. ........ r50674 | neal.norwitz | 2006-07-16 04:00:32 +0200 (Sun, 16 Jul 2006) | 5 lines a & b were dereffed above, so they are known to be valid pointers. z is known to be NULL, nothing to DECREF. Reported by Klockwork, #107. ........ r50675 | neal.norwitz | 2006-07-16 04:02:57 +0200 (Sun, 16 Jul 2006) | 5 lines self is dereffed (and passed as first arg), so it's known to be good. func is returned from PyArg_ParseTuple and also dereffed. Reported by Klocwork, #30 (self one at least). ........ r50676 | neal.norwitz | 2006-07-16 04:05:35 +0200 (Sun, 16 Jul 2006) | 4 lines proto was dereffed above and is known to be good. No need for X. Reported by Klocwork, #39. ........ r50677 | neal.norwitz | 2006-07-16 04:15:27 +0200 (Sun, 16 Jul 2006) | 5 lines Fix memory leaks in some conditions. Reported by Klocwork #152. ........ r50678 | neal.norwitz | 2006-07-16 04:17:36 +0200 (Sun, 16 Jul 2006) | 4 lines Fix memory leak under some conditions. Reported by Klocwork, #98. ........ r50679 | neal.norwitz | 2006-07-16 04:22:30 +0200 (Sun, 16 Jul 2006) | 8 lines Use sizeof(buffer) instead of duplicating the constants to ensure they won't be wrong. The real change is to pass (bufsz - 1) to PyOS_ascii_formatd and 1 to strncat. strncat copies n+1 bytes from src (not dest). Reported by Klocwork #58. ........ r50680 | neal.norwitz | 2006-07-16 04:32:03 +0200 (Sun, 16 Jul 2006) | 5 lines Handle a NULL name properly. Reported by Klocwork #67 ........ r50681 | neal.norwitz | 2006-07-16 04:35:47 +0200 (Sun, 16 Jul 2006) | 6 lines PyFunction_SetDefaults() is documented as taking None or a tuple. A NULL would crash the PyTuple_Check(). Now make NULL return a SystemError. Reported by Klocwork #73. ........ r50683 | neal.norwitz | 2006-07-17 02:55:45 +0200 (Mon, 17 Jul 2006) | 5 lines Stop INCREFing name, then checking if it's NULL. name (f_name) should never be NULL so assert it. Fix one place where we could have passed NULL. Reported by Klocwork #66. ........ r50684 | neal.norwitz | 2006-07-17 02:57:15 +0200 (Mon, 17 Jul 2006) | 5 lines otherset is known to be non-NULL based on checks before and DECREF after. DECREF otherset rather than XDECREF in error conditions too. Reported by Klockwork #154. ........ r50685 | neal.norwitz | 2006-07-17 02:59:04 +0200 (Mon, 17 Jul 2006) | 7 lines Reported by Klocwork #151. v2 can be NULL if exception2 is NULL. I don't think that condition can happen, but I'm not sure it can't either. Now the code will protect against either being NULL. ........ r50686 | neal.norwitz | 2006-07-17 03:00:16 +0200 (Mon, 17 Jul 2006) | 1 line Add NEWS entry for a bunch of fixes due to warnings produced by Klocworks static analysis tool. ........ r50687 | fred.drake | 2006-07-17 07:47:52 +0200 (Mon, 17 Jul 2006) | 3 lines document xmlcore (still minimal; needs mention in each of the xml.* modules) SF bug #1504456 (partial) ........ r50688 | georg.brandl | 2006-07-17 15:23:46 +0200 (Mon, 17 Jul 2006) | 3 lines Remove usage of sets module (patch #1500609). ........ r50689 | georg.brandl | 2006-07-17 15:26:33 +0200 (Mon, 17 Jul 2006) | 3 lines Add missing NEWS item (#1522771) ........ r50690 | andrew.kuchling | 2006-07-17 18:47:54 +0200 (Mon, 17 Jul 2006) | 1 line Attribute more features ........ r50692 | kurt.kaiser | 2006-07-17 23:59:27 +0200 (Mon, 17 Jul 2006) | 8 lines Patch 1479219 - Tal Einat 1. 'as' highlighted as builtin in comment string on import line 2. Comments such as "#False identity" which start with a keyword immediately after the '#' character aren't colored as comments. 3. u or U beginning unicode string not correctly highlighted Closes bug 1325071 ........ r50693 | barry.warsaw | 2006-07-18 01:07:51 +0200 (Tue, 18 Jul 2006) | 16 lines decode_rfc2231(): Be more robust against buggy RFC 2231 encodings. Specifically, instead of raising a ValueError when there is a single tick in the parameter, simply return that the entire string unquoted, with None for both the charset and the language. Also, if there are more than 2 ticks in the parameter, interpret the first three parts as the standard RFC 2231 parts, then the rest of the parts as the encoded string. Test cases added. Original fewer-than-3-parts fix by Tokio Kikuchi. Resolves SF bug # 1218081. I will back port the fix and tests to Python 2.4 (email 3.0) and Python 2.3 (email 2.5). Also, bump the version number to email 4.0.1, removing the 'alpha' moniker. ........ r50695 | kurt.kaiser | 2006-07-18 06:03:16 +0200 (Tue, 18 Jul 2006) | 2 lines Rebinding Tab key was inserting 'tab' instead of 'Tab'. Bug 1179168. ........ r50696 | brett.cannon | 2006-07-18 06:41:36 +0200 (Tue, 18 Jul 2006) | 6 lines Fix bug #1520914. Starting in 2.4, time.strftime() began to check the bounds of values in the time tuple passed in. Unfortunately people came to rely on undocumented behaviour of setting unneeded values to 0, regardless of if it was within the valid range. Now those values force the value internally to the minimum value when 0 is passed in. ........ r50697 | facundo.batista | 2006-07-18 14:16:13 +0200 (Tue, 18 Jul 2006) | 1 line Comments and docs cleanups, and some little fixes, provided by Santiágo Peresón ........ r50704 | martin.v.loewis | 2006-07-18 19:46:31 +0200 (Tue, 18 Jul 2006) | 2 lines Patch #1524429: Use repr instead of backticks again. ........ r50706 | tim.peters | 2006-07-18 23:55:15 +0200 (Tue, 18 Jul 2006) | 2 lines Whitespace normalization. ........ r50708 | tim.peters | 2006-07-19 02:03:19 +0200 (Wed, 19 Jul 2006) | 18 lines SF bug 1524317: configure --without-threads fails to build Moved the code for _PyThread_CurrentFrames() up, so it's no longer in a huge "#ifdef WITH_THREAD" block (I didn't realize it /was/ in one). Changed test_sys's test_current_frames() so it passes with or without thread supported compiled in. Note that test_sys fails when Python is compiled without threads, but for an unrelated reason (the old test_exit() fails with an indirect ImportError on the `thread` module). There are also other unrelated compilation failures without threads, in extension modules (like ctypes); at least the core compiles again. Do we really support --without-threads? If so, there are several problems remaining. ........ r50713 | thomas.heller | 2006-07-19 11:09:32 +0200 (Wed, 19 Jul 2006) | 4 lines Make sure the _ctypes extension can be compiled when WITH_THREAD is not defined on Windows, even if that configuration is probably not supported at all. ........ r50715 | martin.v.loewis | 2006-07-19 19:18:32 +0200 (Wed, 19 Jul 2006) | 4 lines Revert r50706 (Whitespace normalization) and r50697: Comments and docs cleanups, and some little fixes per recommendation from Raymond Hettinger. ........ r50719 | phillip.eby | 2006-07-20 17:54:16 +0200 (Thu, 20 Jul 2006) | 4 lines Fix SF#1516184 (again) and add a test to prevent regression. (There was a problem with empty filenames still causing recursion) ........ r50720 | georg.brandl | 2006-07-20 18:28:39 +0200 (Thu, 20 Jul 2006) | 3 lines Guard for _active being None in __del__ method. ........ r50721 | vinay.sajip | 2006-07-20 18:28:39 +0200 (Thu, 20 Jul 2006) | 1 line Updated documentation for TimedRotatingFileHandler relating to how rollover files are named. The previous documentation was wrongly the same as for RotatingFileHandler. ........ r50731 | fred.drake | 2006-07-20 22:11:57 +0200 (Thu, 20 Jul 2006) | 1 line markup fix ........ r50739 | kurt.kaiser | 2006-07-21 00:22:52 +0200 (Fri, 21 Jul 2006) | 7 lines Avoid occasional failure to detect closing paren properly. Patch 1407280 Tal Einat M ParenMatch.py M NEWS.txt M CREDITS.txt ........ r50740 | vinay.sajip | 2006-07-21 01:20:12 +0200 (Fri, 21 Jul 2006) | 1 line Addressed SF#1524081 by using a dictionary to map level names to syslog priority names, rather than a string.lower(). ........ r50741 | neal.norwitz | 2006-07-21 07:29:58 +0200 (Fri, 21 Jul 2006) | 1 line Add some asserts that we got good params passed ........ r50742 | neal.norwitz | 2006-07-21 07:31:02 +0200 (Fri, 21 Jul 2006) | 5 lines Move the initialization of some pointers earlier. The problem is that if we call Py_DECREF(frame) like we do if allocating locals fails, frame_dealloc() will try to use these bogus values and crash. ........ r50743 | neal.norwitz | 2006-07-21 07:32:28 +0200 (Fri, 21 Jul 2006) | 4 lines Handle allocation failures gracefully. Found with failmalloc. Many (all?) of these could be backported. ........ r50745 | neal.norwitz | 2006-07-21 09:59:02 +0200 (Fri, 21 Jul 2006) | 1 line Speel initialise write. Tanks Anthony. ........ r50746 | neal.norwitz | 2006-07-21 09:59:47 +0200 (Fri, 21 Jul 2006) | 2 lines Handle more memory allocation failures without crashing. ........ r50754 | barry.warsaw | 2006-07-21 16:51:07 +0200 (Fri, 21 Jul 2006) | 23 lines More RFC 2231 improvements for the email 4.0 package. As Mark Sapiro rightly points out there are really two types of continued headers defined in this RFC (i.e. "encoded" parameters with the form "name*0*=" and unencoded parameters with the form "name*0="), but we were were handling them both the same way and that isn't correct. This patch should be much more RFC compliant in that only encoded params are %-decoded and the charset/language information is only extract if there are any encoded params in the segments. If there are no encoded params then the RFC says that there will be no charset/language parts. Note however that this will change the return value for Message.get_param() in some cases. For example, whereas before if you had all unencoded param continuations you would have still gotten a 3-tuple back from this method (with charset and language == None), you will now get just a string. I don't believe this is a backward incompatible change though because the documentation for this method already indicates that either return value is possible and that you must do an isinstance(val, tuple) check to discriminate between the two. (Yeah that API kind of sucks but we can't change /that/ without breaking code.) Test cases, some documentation updates, and a NEWS item accompany this patch. ........ r50759 | georg.brandl | 2006-07-21 19:36:31 +0200 (Fri, 21 Jul 2006) | 3 lines Fix check for empty list (vs. None). ........ r50771 | brett.cannon | 2006-07-22 00:44:07 +0200 (Sat, 22 Jul 2006) | 2 lines Remove an XXX marker in a comment. ........ r50773 | neal.norwitz | 2006-07-22 18:20:49 +0200 (Sat, 22 Jul 2006) | 1 line Fix more memory allocation issues found with failmalloc. ........ r50774 | neal.norwitz | 2006-07-22 19:00:57 +0200 (Sat, 22 Jul 2006) | 1 line Don't fail if the directory already exists ........ r50775 | greg.ward | 2006-07-23 04:25:53 +0200 (Sun, 23 Jul 2006) | 6 lines Be a lot smarter about whether this test passes: instead of assuming that a 2.93 sec audio file will always take 3.1 sec (as it did on the hardware I had when I first wrote the test), expect that it will take 2.93 sec +/- 10%, and only fail if it's outside of that range. Compute the expected ........ r50776 | kurt.kaiser | 2006-07-23 06:19:49 +0200 (Sun, 23 Jul 2006) | 2 lines Tooltips failed on new-syle class __init__ args. Bug 1027566 Loren Guthrie ........ r50777 | neal.norwitz | 2006-07-23 09:50:36 +0200 (Sun, 23 Jul 2006) | 1 line Handle more mem alloc issues found with failmalloc ........ r50778 | neal.norwitz | 2006-07-23 09:51:58 +0200 (Sun, 23 Jul 2006) | 5 lines If the for loop isn't entered, entryblock will be NULL. If passed to stackdepth_walk it will be dereffed. Not sure if I found with failmalloc or Klockwork #55. ........ r50779 | neal.norwitz | 2006-07-23 09:53:14 +0200 (Sun, 23 Jul 2006) | 4 lines Move the initialization of size_a down below the check for a being NULL. Reported by Klocwork #106 ........ r50780 | neal.norwitz | 2006-07-23 09:55:55 +0200 (Sun, 23 Jul 2006) | 9 lines Check the allocation of b_objects and return if there was a failure. Also fix a few memory leaks in other failure scenarios. It seems that if b_objects == Py_None, we will have an extra ref to b_objects. Add XXX comment so hopefully someone documents why the else isn't necessary or adds it in. Reported by Klocwork #20 ........ r50781 | neal.norwitz | 2006-07-23 09:57:11 +0200 (Sun, 23 Jul 2006) | 2 lines Fix memory leaks spotted by Klocwork #37. ........ r50782 | neal.norwitz | 2006-07-23 09:59:00 +0200 (Sun, 23 Jul 2006) | 5 lines nextlink can be NULL if teedataobject_new fails, so use XINCREF. Ensure that dataobj is never NULL. Reported by Klocwork #102 ........ r50783 | neal.norwitz | 2006-07-23 10:01:43 +0200 (Sun, 23 Jul 2006) | 8 lines Ensure we don't write beyond errText. I think I got this right, but it definitely could use some review to ensure I'm not off by one and there's no possible overflow/wrap-around of bytes_left. Reported by Klocwork #1. Fix a problem if there is a failure allocating self->db. Found with failmalloc. ........ r50784 | ronald.oussoren | 2006-07-23 11:41:09 +0200 (Sun, 23 Jul 2006) | 3 lines Without this patch CMD-W won't close EditorWindows on MacOS X. This solves part of bug #1517990. ........ r50785 | ronald.oussoren | 2006-07-23 11:46:11 +0200 (Sun, 23 Jul 2006) | 5 lines Fix for bug #1517996: Class and Path browsers show Tk menu This patch replaces the menubar that is used by AquaTk for windows without a menubar of their own by one that is more appropriate for IDLE. ........ r50786 | andrew.macintyre | 2006-07-23 14:57:02 +0200 (Sun, 23 Jul 2006) | 2 lines Build updates for OS/2 EMX port ........ r50787 | andrew.macintyre | 2006-07-23 15:00:04 +0200 (Sun, 23 Jul 2006) | 3 lines bugfix: PyThread_start_new_thread() returns the thread ID, not a flag; will backport. ........ r50789 | andrew.macintyre | 2006-07-23 15:04:00 +0200 (Sun, 23 Jul 2006) | 2 lines Get mailbox module working on OS/2 EMX port. ........ r50791 | greg.ward | 2006-07-23 18:05:51 +0200 (Sun, 23 Jul 2006) | 1 line Resync optparse with Optik 1.5.3: minor tweaks for/to tests. ........ r50794 | martin.v.loewis | 2006-07-24 07:05:22 +0200 (Mon, 24 Jul 2006) | 2 lines Update list of unsupported systems. Fixes #1510853. ........ r50795 | martin.v.loewis | 2006-07-24 12:26:33 +0200 (Mon, 24 Jul 2006) | 1 line Patch #1448199: Release GIL around ConnectRegistry. ........ r50796 | martin.v.loewis | 2006-07-24 13:54:53 +0200 (Mon, 24 Jul 2006) | 3 lines Patch #1232023: Don't include empty path component from registry, so that the current directory does not get added to sys.path. Also fixes #1526785. ........ r50797 | martin.v.loewis | 2006-07-24 14:54:17 +0200 (Mon, 24 Jul 2006) | 3 lines Bug #1524310: Properly report errors from FindNextFile in os.listdir. Will backport to 2.4. ........ r50800 | georg.brandl | 2006-07-24 15:28:57 +0200 (Mon, 24 Jul 2006) | 7 lines Patch #1523356: fix determining include dirs in python-config. Also don't install "python-config" when doing altinstall, but always install "python-config2.x" and make a link to it like with the main executable. ........ r50802 | georg.brandl | 2006-07-24 15:46:47 +0200 (Mon, 24 Jul 2006) | 3 lines Patch #1527744: right order of includes in order to have HAVE_CONIO_H defined properly. ........ r50803 | georg.brandl | 2006-07-24 16:09:56 +0200 (Mon, 24 Jul 2006) | 3 lines Patch #1515343: Fix printing of deprecated string exceptions with a value in the traceback module. ........ r50804 | kurt.kaiser | 2006-07-24 19:13:23 +0200 (Mon, 24 Jul 2006) | 7 lines EditorWindow failed when used stand-alone if sys.ps1 not set. Bug 1010370 Dave Florek M EditorWindow.py M PyShell.py M NEWS.txt ........ r50805 | kurt.kaiser | 2006-07-24 20:05:51 +0200 (Mon, 24 Jul 2006) | 6 lines - EditorWindow.test() was failing. Bug 1417598 M EditorWindow.py M ScriptBinding.py M NEWS.txt ........ r50808 | georg.brandl | 2006-07-24 22:11:35 +0200 (Mon, 24 Jul 2006) | 3 lines Repair accidental NameError. ........ r50809 | tim.peters | 2006-07-24 23:02:15 +0200 (Mon, 24 Jul 2006) | 2 lines Whitespace normalization. ........ r50810 | greg.ward | 2006-07-25 04:11:12 +0200 (Tue, 25 Jul 2006) | 3 lines Don't use standard assert: want tests to fail even when run with -O. Delete cruft. ........ r50811 | tim.peters | 2006-07-25 06:07:22 +0200 (Tue, 25 Jul 2006) | 10 lines current_frames_with_threads(): There's actually no way to guess /which/ line the spawned thread is in at the time sys._current_frames() is called: we know it finished enter_g.set(), but can't know whether the instruction counter has advanced to the following leave_g.wait(). The latter is overwhelming most likely, but not guaranteed, and I see that the "x86 Ubuntu dapper (icc) trunk" buildbot found it on the other line once. Changed the test so it passes in either case. ........ r50815 | martin.v.loewis | 2006-07-25 11:53:12 +0200 (Tue, 25 Jul 2006) | 2 lines Bug #1525817: Don't truncate short lines in IDLE's tool tips. ........ r50816 | martin.v.loewis | 2006-07-25 12:05:47 +0200 (Tue, 25 Jul 2006) | 3 lines Bug #978833: Really close underlying socket in _socketobject.close. Will backport to 2.4. ........ r50817 | martin.v.loewis | 2006-07-25 12:11:14 +0200 (Tue, 25 Jul 2006) | 1 line Revert incomplete checkin. ........ r50819 | georg.brandl | 2006-07-25 12:22:34 +0200 (Tue, 25 Jul 2006) | 4 lines Patch #1525766: correctly pass onerror arg to recursive calls of pkg.walk_packages. Also improve the docstrings. ........ r50825 | brett.cannon | 2006-07-25 19:32:20 +0200 (Tue, 25 Jul 2006) | 2 lines Add comment for changes to test_ossaudiodev. ........ r50826 | brett.cannon | 2006-07-25 19:34:36 +0200 (Tue, 25 Jul 2006) | 3 lines Fix a bug in the messages for an assert failure where not enough arguments to a string were being converted in the format. ........ r50828 | armin.rigo | 2006-07-25 20:09:57 +0200 (Tue, 25 Jul 2006) | 2 lines Document why is and is not a good way to fix the gc_inspection crasher. ........ r50829 | armin.rigo | 2006-07-25 20:11:07 +0200 (Tue, 25 Jul 2006) | 5 lines Added another crasher, which hit me today (I was not intentionally writing such code, of course, but it took some gdb time to figure out what my bug was). ........ r50830 | armin.rigo | 2006-07-25 20:38:39 +0200 (Tue, 25 Jul 2006) | 3 lines Document the crashers that will not go away soon as "won't fix", and explain why. ........ r50831 | ronald.oussoren | 2006-07-25 21:13:35 +0200 (Tue, 25 Jul 2006) | 3 lines Install the compatibility symlink to libpython.a on OSX using 'ln -sf' instead of 'ln -s', this avoid problems when reinstalling python. ........ r50832 | ronald.oussoren | 2006-07-25 21:20:54 +0200 (Tue, 25 Jul 2006) | 7 lines Fix for bug #1525447 (renaming to MacOSmodule.c would also work, but not without causing problems for anyone that is on a case-insensitive filesystem). Setup.py tries to compile the MacOS extension from MacOSmodule.c, while the actual file is named macosmodule.c. This is no problem on the (default) case-insensitive filesystem, but doesn't work on case-sensitive filesystems. ........ r50833 | ronald.oussoren | 2006-07-25 22:28:55 +0200 (Tue, 25 Jul 2006) | 7 lines Fix bug #1517990: IDLE keybindings on OSX This adds a new key definition for OSX, which is slightly different from the classic mac definition. Also add NEWS item for a couple of bugfixes I added recently. ........ r50834 | tim.peters | 2006-07-26 00:30:24 +0200 (Wed, 26 Jul 2006) | 2 lines Whitespace normalization. ........ r50839 | neal.norwitz | 2006-07-26 06:00:18 +0200 (Wed, 26 Jul 2006) | 1 line Hmm, only python2.x is installed, not plain python. Did that change recently? ........ r50840 | barry.warsaw | 2006-07-26 07:54:46 +0200 (Wed, 26 Jul 2006) | 6 lines Forward port some fixes that were in email 2.5 but for some reason didn't make it into email 4.0. Specifically, in Message.get_content_charset(), handle RFC 2231 headers that contain an encoding not known to Python, or a character in the data that isn't in the charset encoding. Also forward port the appropriate unit tests. ........ r50841 | georg.brandl | 2006-07-26 09:23:32 +0200 (Wed, 26 Jul 2006) | 3 lines NEWS entry for #1525766. ........ r50842 | georg.brandl | 2006-07-26 09:40:17 +0200 (Wed, 26 Jul 2006) | 3 lines Bug #1459963: properly capitalize HTTP header names. ........ r50843 | georg.brandl | 2006-07-26 10:03:10 +0200 (Wed, 26 Jul 2006) | 6 lines Part of bug #1523610: fix miscalculation of buffer length. Also add a guard against NULL in converttuple and add a test case (that previously would have crashed). ........ r50844 | martin.v.loewis | 2006-07-26 14:12:56 +0200 (Wed, 26 Jul 2006) | 3 lines Bug #978833: Really close underlying socket in _socketobject.close. Fix httplib.HTTPConnection.getresponse to not close the socket if it is still needed for the response. ........ r50845 | andrew.kuchling | 2006-07-26 19:16:52 +0200 (Wed, 26 Jul 2006) | 1 line [Bug #1471938] Fix build problem on Solaris 8 by conditionalizing the use of mvwgetnstr(); it was conditionalized a few lines below. Fix from Paul Eggert. I also tried out the STRICT_SYSV_CURSES case and am therefore removing the 'untested' comment. ........ r50846 | andrew.kuchling | 2006-07-26 19:18:01 +0200 (Wed, 26 Jul 2006) | 1 line Correct error message ........ r50847 | andrew.kuchling | 2006-07-26 19:19:39 +0200 (Wed, 26 Jul 2006) | 1 line Minor grammar fix ........ r50848 | andrew.kuchling | 2006-07-26 19:22:21 +0200 (Wed, 26 Jul 2006) | 1 line Put news item in right section ........ r50850 | andrew.kuchling | 2006-07-26 20:03:12 +0200 (Wed, 26 Jul 2006) | 1 line Use sys.exc_info() ........ r50851 | andrew.kuchling | 2006-07-26 20:15:45 +0200 (Wed, 26 Jul 2006) | 1 line Use sys.exc_info() ........ r50852 | phillip.eby | 2006-07-26 21:48:27 +0200 (Wed, 26 Jul 2006) | 4 lines Allow the 'onerror' argument to walk_packages() to catch any Exception, not just ImportError. This allows documentation tools to better skip unimportable packages. ........ r50854 | tim.peters | 2006-07-27 01:23:15 +0200 (Thu, 27 Jul 2006) | 2 lines Whitespace normalization. ........ r50855 | tim.peters | 2006-07-27 03:14:53 +0200 (Thu, 27 Jul 2006) | 21 lines Bug #1521947: possible bug in mystrtol.c with recent gcc. In general, C doesn't define anything about what happens when an operation on a signed integral type overflows, and PyOS_strtol() did several formally undefined things of that nature on signed longs. Some version of gcc apparently tries to exploit that now, and PyOS_strtol() could fail to detect overflow then. Tried to repair all that, although it seems at least as likely to me that we'll get screwed by bad platform definitions for LONG_MIN and/or LONG_MAX now. For that reason, I don't recommend backporting this. Note that I have no box on which this makes a lick of difference -- can't really test it, except to note that it didn't break anything on my boxes. Silent change: PyOS_strtol() used to return the hard-coded 0x7fffffff in case of overflow. Now it returns LONG_MAX. They're the same only on 32-bit boxes (although C doesn't guarantee that either ...). ........ r50856 | neal.norwitz | 2006-07-27 05:51:58 +0200 (Thu, 27 Jul 2006) | 6 lines Don't kill a normal instance of python running on windows when checking to kill a cygwin instance. build\\python.exe was matching a normal windows instance. Prefix that with a \\ to ensure build is a directory and not PCbuild. As discussed on python-dev. ........ r50857 | neal.norwitz | 2006-07-27 05:55:39 +0200 (Thu, 27 Jul 2006) | 5 lines Closure can't be NULL at this point since we know it's a tuple. Reported by Klocwork # 74. ........ r50858 | neal.norwitz | 2006-07-27 06:04:50 +0200 (Thu, 27 Jul 2006) | 1 line No functional change. Add comment and assert to describe why there cannot be overflow which was reported by Klocwork. Discussed on python-dev ........ r50859 | martin.v.loewis | 2006-07-27 08:38:16 +0200 (Thu, 27 Jul 2006) | 3 lines Bump distutils version to 2.5, as several new features have been introduced since 2.4. ........ r50860 | andrew.kuchling | 2006-07-27 14:18:20 +0200 (Thu, 27 Jul 2006) | 1 line Reformat docstring; fix typo ........ r50861 | georg.brandl | 2006-07-27 17:05:36 +0200 (Thu, 27 Jul 2006) | 6 lines Add test_main() methods. These three tests were never run by regrtest.py. We really need a simpler testing framework. ........ r50862 | tim.peters | 2006-07-27 17:09:20 +0200 (Thu, 27 Jul 2006) | 2 lines News for patch #1529686. ........ r50863 | tim.peters | 2006-07-27 17:11:00 +0200 (Thu, 27 Jul 2006) | 2 lines Whitespace normalization. ........ r50864 | georg.brandl | 2006-07-27 17:38:33 +0200 (Thu, 27 Jul 2006) | 3 lines Amend news entry. ........ r50865 | georg.brandl | 2006-07-27 18:08:15 +0200 (Thu, 27 Jul 2006) | 3 lines Make uuid test suite pass on this box by requesting output with LC_ALL=C. ........ r50866 | andrew.kuchling | 2006-07-27 20:37:33 +0200 (Thu, 27 Jul 2006) | 1 line Add example ........ r50867 | thomas.heller | 2006-07-27 20:39:55 +0200 (Thu, 27 Jul 2006) | 9 lines Remove code that is no longer used (ctypes.com). Fix the DllGetClassObject and DllCanUnloadNow so that they forward the call to the comtypes.server.inprocserver module. The latter was never documented, never used by published code, and didn't work anyway, so I think it does not deserve a NEWS entry (but I might be wrong). ........ r50868 | andrew.kuchling | 2006-07-27 20:41:21 +0200 (Thu, 27 Jul 2006) | 1 line Typo fix ('publically' is rare, poss. non-standard) ........ r50869 | andrew.kuchling | 2006-07-27 20:42:41 +0200 (Thu, 27 Jul 2006) | 1 line Add missing word ........ r50870 | andrew.kuchling | 2006-07-27 20:44:10 +0200 (Thu, 27 Jul 2006) | 1 line Repair typos ........ r50872 | andrew.kuchling | 2006-07-27 20:53:33 +0200 (Thu, 27 Jul 2006) | 1 line Update URL; add example ........ r50873 | andrew.kuchling | 2006-07-27 21:07:29 +0200 (Thu, 27 Jul 2006) | 1 line Add punctuation mark; add some examples ........ r50874 | andrew.kuchling | 2006-07-27 21:11:07 +0200 (Thu, 27 Jul 2006) | 1 line Mention base64 module; rewrite last sentence to be more positive ........ r50875 | andrew.kuchling | 2006-07-27 21:12:49 +0200 (Thu, 27 Jul 2006) | 1 line If binhex is higher-level than binascii, it should come first in the chapter ........ r50876 | tim.peters | 2006-07-27 22:47:24 +0200 (Thu, 27 Jul 2006) | 28 lines check_node(): stop spraying mystery output to stderr. When a node number disagrees, keep track of all sources & the node numbers they reported, and stick all that in the error message. Changed all callers to supply a non-empty "source" argument; made the "source" argument non-optional. On my box, test_uuid still fails, but with the less confusing output: AssertionError: different sources disagree on node: from source 'getnode1', node was 00038a000015 from source 'getnode2', node was 00038a000015 from source 'ipconfig', node was 001111b2b7bf Only the last one appears to be correct; e.g., C:\Code\python\PCbuild>getmac Physical Address Transport Name =================== ========================================================== 00-11-11-B2-B7-BF \Device\Tcpip_{190FB163-5AFD-4483-86A1-2FE16AC61FF1} 62-A1-AC-6C-FD-BE \Device\Tcpip_{8F77DF5A-EA3D-4F1D-975E-D472CEE6438A} E2-1F-01-C6-5D-88 \Device\Tcpip_{CD18F76B-2EF3-409F-9B8A-6481EE70A1E4} I can't find anything on my box with MAC 00-03-8a-00-00-15, and am not clear on where that comes from. ........ r50878 | andrew.kuchling | 2006-07-28 00:40:05 +0200 (Fri, 28 Jul 2006) | 1 line Reword paragraph ........ r50879 | andrew.kuchling | 2006-07-28 00:49:38 +0200 (Fri, 28 Jul 2006) | 1 line Add example ........ r50880 | andrew.kuchling | 2006-07-28 00:49:54 +0200 (Fri, 28 Jul 2006) | 1 line Add example ........ r50881 | barry.warsaw | 2006-07-28 01:43:15 +0200 (Fri, 28 Jul 2006) | 27 lines Patch #1520294: Support for getset and member descriptors in types.py, inspect.py, and pydoc.py. Specifically, this allows for querying the type of an object against these built-in C types and more importantly, for getting their docstrings printed in the interactive interpreter's help() function. This patch includes a new built-in module called _types which provides definitions of getset and member descriptors for use by the types.py module. These types are exposed as types.GetSetDescriptorType and types.MemberDescriptorType. Query functions are provided as inspect.isgetsetdescriptor() and inspect.ismemberdescriptor(). The implementations of these are robust enough to work with Python implementations other than CPython, which may not have these fundamental types. The patch also includes documentation and test suite updates. I commit these changes now under these guiding principles: 1. Silence is assent. The release manager has not said "no", and of the few people that cared enough to respond to the thread, the worst vote was "0". 2. It's easier to ask for forgiveness than permission. 3. It's so dang easy to revert stuff in svn, that you could view this as a forcing function. :) Windows build patches will follow. ........ r50882 | tim.peters | 2006-07-28 01:44:37 +0200 (Fri, 28 Jul 2006) | 4 lines Bug #1529297: The rewrite of doctest for Python 2.4 unintentionally lost that tests are sorted by name before being run. ``DocTestFinder`` has been changed to sort the list of tests it returns. ........ r50883 | tim.peters | 2006-07-28 01:45:48 +0200 (Fri, 28 Jul 2006) | 2 lines Whitespace normalization. ........ r50884 | tim.peters | 2006-07-28 01:46:36 +0200 (Fri, 28 Jul 2006) | 2 lines Add missing svn:eol-style property to text files. ........ r50885 | barry.warsaw | 2006-07-28 01:50:40 +0200 (Fri, 28 Jul 2006) | 4 lines Enable the building of the _types module on Windows. Note that this has only been tested for VS 2003 since that's all I have. ........ r50887 | tim.peters | 2006-07-28 02:23:15 +0200 (Fri, 28 Jul 2006) | 7 lines defdict_reduce(): Plug leaks. We didn't notice these before because test_defaultdict didn't actually do anything before Georg fixed that earlier today. Neal's next refleak run then showed test_defaultdict leaking 9 references on each run. That's repaired by this checkin. ........ r50888 | tim.peters | 2006-07-28 02:30:00 +0200 (Fri, 28 Jul 2006) | 2 lines News about the repaired memory leak in defaultdict. ........ r50889 | gregory.p.smith | 2006-07-28 03:35:25 +0200 (Fri, 28 Jul 2006) | 7 lines - pybsddb Bug #1527939: bsddb module DBEnv dbremove and dbrename methods now allow their database parameter to be None as the sleepycat API allows. Also adds an appropriate test case for DBEnv.dbrename and dbremove. ........ r50895 | neal.norwitz | 2006-07-28 06:22:34 +0200 (Fri, 28 Jul 2006) | 1 line Ensure the actual number matches the expected count ........ r50896 | tim.peters | 2006-07-28 06:51:59 +0200 (Fri, 28 Jul 2006) | 6 lines Live with that "the hardware address" is an ill-defined concept, and that different ways of trying to find "the hardware address" may return different results. Certainly true on both of my Windows boxes, and in different ways (see whining on python-dev). ........ r50897 | neal.norwitz | 2006-07-28 09:21:27 +0200 (Fri, 28 Jul 2006) | 3 lines Try to find the MAC addr on various flavours of Unix. This seems hopeless. The reduces the test_uuid failures, but there's still another method failing. ........ r50898 | martin.v.loewis | 2006-07-28 09:45:49 +0200 (Fri, 28 Jul 2006) | 2 lines Add UUID for upcoming 2.5b3. ........ r50899 | matt.fleming | 2006-07-28 13:27:27 +0200 (Fri, 28 Jul 2006) | 3 lines Allow socketmodule to compile on NetBSD -current, whose bluetooth API differs from both Linux and FreeBSD. Accepted by Neal Norwitz. ........ r50900 | andrew.kuchling | 2006-07-28 14:07:12 +0200 (Fri, 28 Jul 2006) | 1 line [Patch #1529811] Correction to description of r|* mode ........ r50901 | andrew.kuchling | 2006-07-28 14:18:22 +0200 (Fri, 28 Jul 2006) | 1 line Typo fix ........ r50902 | andrew.kuchling | 2006-07-28 14:32:43 +0200 (Fri, 28 Jul 2006) | 1 line Add example ........ r50903 | andrew.kuchling | 2006-07-28 14:33:19 +0200 (Fri, 28 Jul 2006) | 1 line Add example ........ r50904 | andrew.kuchling | 2006-07-28 14:45:55 +0200 (Fri, 28 Jul 2006) | 1 line Don't overwrite built-in name; add some blank lines for readability ........ r50905 | andrew.kuchling | 2006-07-28 14:48:07 +0200 (Fri, 28 Jul 2006) | 1 line Add example. Should I propagate this example to all the other DBM-ish modules, too? ........ r50912 | georg.brandl | 2006-07-28 20:31:39 +0200 (Fri, 28 Jul 2006) | 3 lines Patch #1529686: also run test_email_codecs with regrtest.py. ........ r50913 | georg.brandl | 2006-07-28 20:36:01 +0200 (Fri, 28 Jul 2006) | 3 lines Fix spelling. ........ r50915 | thomas.heller | 2006-07-28 21:42:40 +0200 (Fri, 28 Jul 2006) | 3 lines Remove a useless XXX comment. Cosmetic changes to the code so that the #ifdef _UNICODE block doesn't mess emacs code formatting. ........ r50916 | phillip.eby | 2006-07-28 23:12:07 +0200 (Fri, 28 Jul 2006) | 5 lines Bug #1529871: The speed enhancement patch #921466 broke Python's compliance with PEP 302. This was fixed by adding an ``imp.NullImporter`` type that is used in ``sys.path_importer_cache`` to cache non-directory paths and avoid excessive filesystem operations during imports. ........ r50917 | phillip.eby | 2006-07-28 23:31:54 +0200 (Fri, 28 Jul 2006) | 2 lines Fix svn merge spew. ........ r50918 | thomas.heller | 2006-07-28 23:43:20 +0200 (Fri, 28 Jul 2006) | 4 lines Patch #1529514: More openbsd platforms for ctypes. Regenerated Modules/_ctypes/libffi/configure with autoconf 2.59. Approved by Neal. ........ r50922 | georg.brandl | 2006-07-29 10:51:21 +0200 (Sat, 29 Jul 2006) | 2 lines Bug #835255: The "closure" argument to new.function() is now documented. ........ r50924 | georg.brandl | 2006-07-29 11:33:26 +0200 (Sat, 29 Jul 2006) | 3 lines Bug #1441397: The compiler module now recognizes module and function docstrings correctly as it did in Python 2.4. ........ r50925 | georg.brandl | 2006-07-29 12:25:46 +0200 (Sat, 29 Jul 2006) | 4 lines Revert rev 42617, it was introduced to work around bug #1441397. test_compiler now passes again. ........ r50926 | fred.drake | 2006-07-29 15:22:49 +0200 (Sat, 29 Jul 2006) | 1 line update target version number ........ r50927 | andrew.kuchling | 2006-07-29 15:56:48 +0200 (Sat, 29 Jul 2006) | 1 line Add example ........ r50928 | andrew.kuchling | 2006-07-29 16:04:47 +0200 (Sat, 29 Jul 2006) | 1 line Update URL ........ r50930 | andrew.kuchling | 2006-07-29 16:08:15 +0200 (Sat, 29 Jul 2006) | 1 line Reword paragraph to match the order of the subsequent sections ........ r50931 | andrew.kuchling | 2006-07-29 16:21:15 +0200 (Sat, 29 Jul 2006) | 1 line [Bug #1529157] Mention raw_input() and input(); while I'm at it, reword the description a bit ........ r50932 | andrew.kuchling | 2006-07-29 16:42:48 +0200 (Sat, 29 Jul 2006) | 1 line [Bug #1519571] Document some missing functions: setup(), title(), done() ........ r50933 | andrew.kuchling | 2006-07-29 16:43:55 +0200 (Sat, 29 Jul 2006) | 1 line Fix docstring punctuation ........ r50934 | andrew.kuchling | 2006-07-29 17:10:32 +0200 (Sat, 29 Jul 2006) | 1 line [Bug #1414697] Change docstring of set/frozenset types to specify that the contents are unique. Raymond, please feel free to edit or revert. ........ r50935 | andrew.kuchling | 2006-07-29 17:35:21 +0200 (Sat, 29 Jul 2006) | 1 line [Bug #1530382] Document SSL.server(), .issuer() methods ........ r50936 | andrew.kuchling | 2006-07-29 17:42:46 +0200 (Sat, 29 Jul 2006) | 1 line Typo fix ........ r50937 | andrew.kuchling | 2006-07-29 17:43:13 +0200 (Sat, 29 Jul 2006) | 1 line Tweak wording ........ r50938 | matt.fleming | 2006-07-29 17:55:30 +0200 (Sat, 29 Jul 2006) | 2 lines Fix typo ........ r50939 | andrew.kuchling | 2006-07-29 17:57:08 +0200 (Sat, 29 Jul 2006) | 6 lines [Bug #1528258] Mention that the 'data' argument can be None. The constructor docs referred the reader to the add_data() method's docs, but they weren't very helpful. I've simply copied an earlier explanation of 'data' that's more useful. ........ r50940 | andrew.kuchling | 2006-07-29 18:08:40 +0200 (Sat, 29 Jul 2006) | 1 line Set bug/patch count. Take a bow, everyone! ........ r50941 | fred.drake | 2006-07-29 18:56:15 +0200 (Sat, 29 Jul 2006) | 18 lines expunge the xmlcore changes: 41667, 41668 - initial switch to xmlcore 47044 - mention of xmlcore in What's New 50687 - mention of xmlcore in the library reference re-apply xmlcore changes to xml: 41674 - line ending changes (re-applied manually), directory props 41677 - add cElementTree wrapper 41678 - PSF licensing for etree 41812 - whitespace normalization 42724 - fix svn:eol-style settings 43681, 43682 - remove Python version-compatibility cruft from minidom 46773 - fix encoding of \r\n\t in attr values in saxutils 47269 - added XMLParser alias for cElementTree compatibility additional tests were added in Lib/test/test_sax.py that failed with the xmlcore changes; these relate to SF bugs #1511497, #1513611 ........ r50942 | andrew.kuchling | 2006-07-29 20:14:07 +0200 (Sat, 29 Jul 2006) | 17 lines Reorganize the docs for 'file' and 'open()' after some discussion with Fred. We want to encourage users to write open() when opening a file, but open() was described with a single paragraph and 'file' had lots of explanation of the mode and bufsize arguments. I've shrunk the description of 'file' to cross-reference to the 'File objects' section, and to open() for an explanation of the arguments. open() now has all the paragraphs about the mode string. The bufsize argument was moved up so that it isn't buried at the end; now there's 1 paragraph on mode, 1 on bufsize, and then 3 more on mode. Various other edits and rearrangements were made in the process. It's probably best to read the final text and not to try to make sense of the diffs. ........ r50943 | fred.drake | 2006-07-29 20:19:19 +0200 (Sat, 29 Jul 2006) | 1 line restore test un-intentionally removed in the xmlcore purge (revision 50941) ........ r50944 | fred.drake | 2006-07-29 20:33:29 +0200 (Sat, 29 Jul 2006) | 3 lines make the reference to older versions of the documentation a link to the right page on python.org ........ r50945 | fred.drake | 2006-07-29 21:09:01 +0200 (Sat, 29 Jul 2006) | 1 line document the footnote usage pattern ........ r50947 | fred.drake | 2006-07-29 21:14:10 +0200 (Sat, 29 Jul 2006) | 1 line emphasize and oddball nuance of LaTeX comment syntax ........ r50948 | andrew.kuchling | 2006-07-29 21:24:04 +0200 (Sat, 29 Jul 2006) | 1 line [Patch #1490989 from Skip Montanaro] Mention debugging builds in the API documentation. I've changed Skip's patch to point to Misc/SpecialBuilds and fiddled with the markup a bit. ........ r50949 | neal.norwitz | 2006-07-29 21:29:35 +0200 (Sat, 29 Jul 2006) | 6 lines Disable these tests until they are reliable across platforms. These problems may mask more important, real problems. One or both methods are known to fail on: Solaris, OpenBSD, Debian, Ubuntu. They pass on Windows and some Linux boxes. ........ r50950 | andrew.kuchling | 2006-07-29 21:50:37 +0200 (Sat, 29 Jul 2006) | 1 line [Patch #1068277] Clarify that os.path.exists() can return False depending on permissions. Fred approved committing this patch in December 2004! ........ r50952 | fred.drake | 2006-07-29 22:04:42 +0200 (Sat, 29 Jul 2006) | 6 lines SF bug #1193966: Weakref types documentation misplaced The information about supporting weakrefs with types defined in C extensions is moved to the Extending & Embedding manual. Py_TPFLAGS_HAVE_WEAKREFS is no longer mentioned since it is part of Py_TPFLAGS_DEFAULT. ........ r50953 | skip.montanaro | 2006-07-29 22:06:05 +0200 (Sat, 29 Jul 2006) | 4 lines Add a comment to the csv reader documentation that explains why the treatment of newlines changed in 2.5. Pulled almost verbatim from a comment by Andrew McNamara in . ........ r50954 | neal.norwitz | 2006-07-29 22:20:52 +0200 (Sat, 29 Jul 2006) | 3 lines If the executable doesn't exist, there's no reason to try to start it. This prevents garbage about command not found being printed on Solaris. ........ r50955 | fred.drake | 2006-07-29 22:21:25 +0200 (Sat, 29 Jul 2006) | 1 line fix minor markup error that introduced extra punctuation ........ r50957 | neal.norwitz | 2006-07-29 22:37:08 +0200 (Sat, 29 Jul 2006) | 3 lines Disable test_getnode too, since this is also unreliable. ........ r50958 | andrew.kuchling | 2006-07-29 23:27:12 +0200 (Sat, 29 Jul 2006) | 1 line Follow TeX's conventions for hyphens ........ r50959 | andrew.kuchling | 2006-07-29 23:30:21 +0200 (Sat, 29 Jul 2006) | 1 line Fix case for 'Unix' ........ r50960 | fred.drake | 2006-07-30 01:34:57 +0200 (Sun, 30 Jul 2006) | 1 line markup cleanups ........ r50961 | andrew.kuchling | 2006-07-30 02:27:34 +0200 (Sun, 30 Jul 2006) | 1 line Minor typo fixes ........ r50962 | andrew.kuchling | 2006-07-30 02:37:56 +0200 (Sun, 30 Jul 2006) | 1 line [Bug #793553] Correct description of keyword arguments for SSL authentication ........ r50963 | tim.peters | 2006-07-30 02:58:15 +0200 (Sun, 30 Jul 2006) | 2 lines Whitespace normalization. ........ r50964 | fred.drake | 2006-07-30 05:03:43 +0200 (Sun, 30 Jul 2006) | 1 line lots of markup nits, most commonly Unix/unix --> \UNIX ........ r50965 | fred.drake | 2006-07-30 07:41:28 +0200 (Sun, 30 Jul 2006) | 1 line update information on wxPython, from Robin Dunn ........ r50966 | fred.drake | 2006-07-30 07:49:49 +0200 (Sun, 30 Jul 2006) | 4 lines remove possibly-outdated comment on what GUI toolkit is most commonly used; it is hard to know whether this is right, and it does not add valuable reference information at any rate ........ r50967 | fred.drake | 2006-07-30 07:55:39 +0200 (Sun, 30 Jul 2006) | 3 lines - remove yet another reference to how commonly Tkinter is (thought to be) used - fix an internal section reference ........ r50968 | neal.norwitz | 2006-07-30 08:53:31 +0200 (Sun, 30 Jul 2006) | 4 lines Patch #1531113: Fix augmented assignment with yield expressions. Also fix a SystemError when trying to assign to yield expressions. ........ r50969 | neal.norwitz | 2006-07-30 08:55:48 +0200 (Sun, 30 Jul 2006) | 5 lines Add PyErr_WarnEx() so C code can pass the stacklevel to warnings.warn(). This provides the proper warning for struct.pack(). PyErr_Warn() is now deprecated in favor of PyErr_WarnEx(). As mentioned by Tim Peters on python-dev. ........ r50970 | neal.norwitz | 2006-07-30 08:57:04 +0200 (Sun, 30 Jul 2006) | 3 lines Bug #1515471: string.replace() accepts character buffers again. Pass the char* and size around rather than PyObject's. ........ r50971 | neal.norwitz | 2006-07-30 08:59:13 +0200 (Sun, 30 Jul 2006) | 1 line Whitespace normalization ........ r50973 | georg.brandl | 2006-07-30 12:53:32 +0200 (Sun, 30 Jul 2006) | 3 lines Clarify that __op__ methods must return NotImplemented if they don't support the operation. ........ r50974 | georg.brandl | 2006-07-30 13:07:23 +0200 (Sun, 30 Jul 2006) | 3 lines Bug #1002398: The documentation for os.path.sameopenfile now correctly refers to file descriptors, not file objects. ........ r50977 | martin.v.loewis | 2006-07-30 15:00:31 +0200 (Sun, 30 Jul 2006) | 3 lines Don't copy directory stat times in shutil.copytree on Windows Fixes #1525866. ........ r50978 | martin.v.loewis | 2006-07-30 15:14:05 +0200 (Sun, 30 Jul 2006) | 3 lines Base __version__ on sys.version_info, as distutils is no longer maintained separatedly. ........ r50979 | martin.v.loewis | 2006-07-30 15:27:31 +0200 (Sun, 30 Jul 2006) | 3 lines Mention Cygwin in distutils error message about a missing VS 2003. Fixes #1257728. ........ r50982 | martin.v.loewis | 2006-07-30 16:09:47 +0200 (Sun, 30 Jul 2006) | 5 lines Drop usage of test -e in configure as it is not portable. Fixes #1439538 Will backport to 2.4 Also regenerate pyconfig.h.in. ........ r50984 | georg.brandl | 2006-07-30 18:20:10 +0200 (Sun, 30 Jul 2006) | 3 lines Fix makefile changes for python-config. ........ r50985 | george.yoshida | 2006-07-30 18:37:37 +0200 (Sun, 30 Jul 2006) | 2 lines Rename struct.pack_to to struct.pack_into as changed in revision 46642. ........ r50986 | george.yoshida | 2006-07-30 18:41:30 +0200 (Sun, 30 Jul 2006) | 2 lines Typo fix ........ r50987 | neal.norwitz | 2006-07-30 21:18:13 +0200 (Sun, 30 Jul 2006) | 1 line Add some asserts and update comments ........ r50988 | neal.norwitz | 2006-07-30 21:18:38 +0200 (Sun, 30 Jul 2006) | 1 line Verify that the signal handlers were really called ........ r50989 | neal.norwitz | 2006-07-30 21:20:42 +0200 (Sun, 30 Jul 2006) | 3 lines Try to prevent hangs on Tru64/Alpha buildbot. I'm not certain this will help and may need to be reverted if it causes problems. ........ r50990 | georg.brandl | 2006-07-30 22:18:51 +0200 (Sun, 30 Jul 2006) | 2 lines Bug #1531349: right <-> left glitch in __rop__ description. ........ r50992 | tim.peters | 2006-07-31 03:46:03 +0200 (Mon, 31 Jul 2006) | 2 lines Whitespace normalization. ........ r50993 | andrew.mcnamara | 2006-07-31 04:27:48 +0200 (Mon, 31 Jul 2006) | 2 lines Redo the comment about the 2.5 change in quoted-newline handling. ........ r50994 | tim.peters | 2006-07-31 04:40:23 +0200 (Mon, 31 Jul 2006) | 10 lines ZipFile.close(): Killed one of the struct.pack deprecation warnings on Win32. Also added an XXX about the line: pos3 = self.fp.tell() `pos3` is never referenced, and I have no idea what the code intended to do instead. ........ r50996 | tim.peters | 2006-07-31 04:53:03 +0200 (Mon, 31 Jul 2006) | 8 lines ZipFile.close(): Kill the other struct.pack deprecation warning on Windows. Afraid I can't detect a pattern to when the pack formats decide to use a signed or unsigned format code -- appears nearly arbitrary to my eyes. So I left all the pack formats alone and changed the special-case data values instead. ........ r50997 | skip.montanaro | 2006-07-31 05:09:45 +0200 (Mon, 31 Jul 2006) | 1 line minor tweaks ........ r50998 | skip.montanaro | 2006-07-31 05:11:11 +0200 (Mon, 31 Jul 2006) | 1 line minor tweaks ........ r50999 | andrew.kuchling | 2006-07-31 14:20:24 +0200 (Mon, 31 Jul 2006) | 1 line Add refcounts for PyErr_WarnEx ........ r51000 | andrew.kuchling | 2006-07-31 14:39:05 +0200 (Mon, 31 Jul 2006) | 9 lines Document PyErr_WarnEx. (Bad Neal! No biscuit!) Is the explanation of the 'stacklevel' parameter clear? Please feel free to edit it. I don't have LaTeX installed on this machine, so haven't verified that the markup is correct. Will check tonight, or maybe the automatic doc build will tell me. ........ r51001 | andrew.kuchling | 2006-07-31 14:52:26 +0200 (Mon, 31 Jul 2006) | 1 line Add PyErr_WarnEx() ........ r51002 | andrew.kuchling | 2006-07-31 15:18:27 +0200 (Mon, 31 Jul 2006) | 1 line Mention csv newline changes ........ r51003 | andrew.kuchling | 2006-07-31 17:22:58 +0200 (Mon, 31 Jul 2006) | 1 line Typo fix ........ r51004 | andrew.kuchling | 2006-07-31 17:23:43 +0200 (Mon, 31 Jul 2006) | 1 line Remove reference to notation ........ r51005 | georg.brandl | 2006-07-31 18:00:34 +0200 (Mon, 31 Jul 2006) | 3 lines Fix function name. ........ r51006 | andrew.kuchling | 2006-07-31 18:10:24 +0200 (Mon, 31 Jul 2006) | 1 line [Bug #1514540] Instead of putting the standard types in a section, put them in a chapter of their own. This means string methods will now show up in the ToC. (Should the types come before or after the functions+exceptions+constants chapter? I've put them after, for now.) ........ r51007 | andrew.kuchling | 2006-07-31 18:22:05 +0200 (Mon, 31 Jul 2006) | 1 line [Bug #848556] Remove \d* from second alternative to avoid exponential case when repeating match ........ r51008 | andrew.kuchling | 2006-07-31 18:27:57 +0200 (Mon, 31 Jul 2006) | 1 line Update list of files; fix a typo ........ r51013 | andrew.kuchling | 2006-08-01 18:24:30 +0200 (Tue, 01 Aug 2006) | 1 line typo fix ........ r51018 | thomas.heller | 2006-08-01 18:54:43 +0200 (Tue, 01 Aug 2006) | 2 lines Fix a potential segfault and various potentail refcount leaks in the cast() function. ........ r51020 | thomas.heller | 2006-08-01 19:46:10 +0200 (Tue, 01 Aug 2006) | 1 line Minimal useful docstring for CopyComPointer. ........ r51021 | andrew.kuchling | 2006-08-01 20:16:15 +0200 (Tue, 01 Aug 2006) | 8 lines [Patch #1520905] Attempt to suppress core file created by test_subprocess.py. Patch by Douglas Greiman. The test_run_abort() testcase produces a core file on Unix systems, even though the test is successful. This can be confusing or alarming to someone who runs 'make test' and then finds that the Python interpreter apparently crashed. ........ r51023 | georg.brandl | 2006-08-01 20:49:24 +0200 (Tue, 01 Aug 2006) | 3 lines os.urandom no longer masks unrelated exceptions like SystemExit or KeyboardInterrupt. ........ r51025 | thomas.heller | 2006-08-01 21:14:15 +0200 (Tue, 01 Aug 2006) | 2 lines Speed up PyType_stgdict and PyObject_stgdict. ........ r51027 | ronald.oussoren | 2006-08-01 22:30:31 +0200 (Tue, 01 Aug 2006) | 3 lines Make sure the postinstall action that optionally updates the user's profile on MacOS X actually works correctly in all cases. ........ r51028 | ronald.oussoren | 2006-08-01 23:00:57 +0200 (Tue, 01 Aug 2006) | 4 lines This fixes bug #1527397: PythonLauncher runs scripts with the wrong working directory. It also fixes a bug where PythonLauncher failed to launch scripts when the scriptname (or the path to the script) contains quotes. ........ r51031 | tim.peters | 2006-08-02 05:27:46 +0200 (Wed, 02 Aug 2006) | 2 lines Whitespace normalization. ........ r51032 | tim.peters | 2006-08-02 06:12:36 +0200 (Wed, 02 Aug 2006) | 19 lines Try to squash struct.pack warnings on the "amd64 gentoo trunk" buildbot (& possibly other 64-bit boxes) during test_gzip. The native zlib crc32 function returns an unsigned 32-bit integer, which the Python wrapper implicitly casts to C long. Therefore the same crc can "look negative" on a 32-bit box but "look positive" on a 64-bit box. This patch papers over that platform difference when writing the crc to file. It may be better to change the Python wrapper, either to make the result "look positive" on all platforms (which means it may have to return a Python long at times on a 32-bit box), or to keep the sign the same across boxes. But that would be a visible change in what users see, while the current hack changes no visible behavior (well, apart from stopping the struct deprecation warning). Note that the module-level write32() function is no longer used. ........ r51033 | neal.norwitz | 2006-08-02 06:27:11 +0200 (Wed, 02 Aug 2006) | 4 lines Prevent memory leak on error. Reported by Klocwork #36 ........ r51034 | tim.peters | 2006-08-02 07:20:08 +0200 (Wed, 02 Aug 2006) | 9 lines _Stream.close(): Try to kill struct.pack() warnings when writing the crc to file on the "PPC64 Debian trunk" buildbot when running test_tarfile. This is again a case where the native zlib crc is an unsigned 32-bit int, but the Python wrapper implicitly casts it to signed C long, so that "the sign bit looks different" on different platforms. ........ r51035 | ronald.oussoren | 2006-08-02 08:10:10 +0200 (Wed, 02 Aug 2006) | 2 lines Updated documentation for the script that builds the OSX installer. ........ r51036 | neal.norwitz | 2006-08-02 08:14:22 +0200 (Wed, 02 Aug 2006) | 2 lines _PyWeakref_GetWeakrefCount() now returns a Py_ssize_t instead of long. ........ r51037 | neal.norwitz | 2006-08-02 08:15:10 +0200 (Wed, 02 Aug 2006) | 1 line v is already checked for NULL, so just DECREF it ........ r51038 | neal.norwitz | 2006-08-02 08:19:19 +0200 (Wed, 02 Aug 2006) | 1 line Let us know when there was a problem and the child had to kill the parent ........ r51039 | neal.norwitz | 2006-08-02 08:46:21 +0200 (Wed, 02 Aug 2006) | 5 lines Patch #1519025 and bug #926423: If a KeyboardInterrupt occurs during a socket operation on a socket with a timeout, the exception will be caught correctly. Previously, the exception was not caught. ........ r51040 | neal.norwitz | 2006-08-02 09:09:32 +0200 (Wed, 02 Aug 2006) | 1 line Add some explanation about Klocwork and Coverity static analysis ........ r51041 | anthony.baxter | 2006-08-02 09:43:09 +0200 (Wed, 02 Aug 2006) | 1 line pre-release machinations ........ r51043 | thomas.heller | 2006-08-02 13:35:31 +0200 (Wed, 02 Aug 2006) | 4 lines A few nore words about what ctypes does. Document that using the wrong calling convention can also raise 'ValueError: Procedure called with the wrong number of arguments'. ........ r51045 | thomas.heller | 2006-08-02 14:00:13 +0200 (Wed, 02 Aug 2006) | 1 line Fix a mistake. ........ r51046 | martin.v.loewis | 2006-08-02 15:53:55 +0200 (Wed, 02 Aug 2006) | 3 lines Correction of patch #1455898: In the mbcs decoder, set final=False for stream decoder, but final=True for the decode function. ........ r51049 | tim.peters | 2006-08-02 20:19:35 +0200 (Wed, 02 Aug 2006) | 2 lines Add missing svn:eol-style property to text files. ........ r51079 | neal.norwitz | 2006-08-04 06:50:21 +0200 (Fri, 04 Aug 2006) | 3 lines Bug #1531405, format_exception no longer raises an exception if str(exception) raised an exception. ........ r51080 | neal.norwitz | 2006-08-04 06:58:47 +0200 (Fri, 04 Aug 2006) | 11 lines Bug #1191458: tracing over for loops now produces a line event on each iteration. I'm not positive this is the best way to handle this. I'm also not sure that there aren't other cases where the lnotab is generated incorrectly. It would be great if people that use pdb or tracing could test heavily. Also: * Remove dead/duplicated code that wasn't used/necessary because we already handled the docstring prior to entering the loop. * add some debugging code into the compiler (#if 0'd out). ........ r51081 | neal.norwitz | 2006-08-04 07:09:28 +0200 (Fri, 04 Aug 2006) | 4 lines Bug #1333982: string/number constants were inappropriately stored in the byte code and co_consts even if they were not used, ie immediately popped off the stack. ........ r51082 | neal.norwitz | 2006-08-04 07:12:19 +0200 (Fri, 04 Aug 2006) | 1 line There were really two issues ........ r51084 | fred.drake | 2006-08-04 07:17:21 +0200 (Fri, 04 Aug 2006) | 1 line SF patch #1534048 (bug #1531003): fix typo in error message ........ r51085 | gregory.p.smith | 2006-08-04 07:17:47 +0200 (Fri, 04 Aug 2006) | 3 lines fix typos ........ r51087 | georg.brandl | 2006-08-04 08:03:53 +0200 (Fri, 04 Aug 2006) | 3 lines Fix bug caused by first decrefing, then increfing. ........ r51109 | neil.schemenauer | 2006-08-04 18:20:30 +0200 (Fri, 04 Aug 2006) | 5 lines Fix the 'compiler' package to generate correct code for MAKE_CLOSURE. In the 2.5 development cycle, MAKE_CLOSURE as changed to take free variables as a tuple rather than as individual items on the stack. Closes patch #1534084. ........ r51110 | georg.brandl | 2006-08-04 20:03:37 +0200 (Fri, 04 Aug 2006) | 3 lines Change fix for segfaulting property(), add a NEWS entry and a test. ........ r51111 | georg.brandl | 2006-08-04 20:07:34 +0200 (Fri, 04 Aug 2006) | 3 lines Better fix for bug #1531405, not executing str(value) twice. ........ r51112 | thomas.heller | 2006-08-04 20:17:40 +0200 (Fri, 04 Aug 2006) | 1 line On Windows, make PyErr_Warn an exported function again. ........ r51113 | thomas.heller | 2006-08-04 20:57:34 +0200 (Fri, 04 Aug 2006) | 4 lines Fix #1530448 - fix ctypes build failure on solaris 10. The '-mimpure-text' linker flag is required when linking _ctypes.so. ........ r51114 | thomas.heller | 2006-08-04 21:49:31 +0200 (Fri, 04 Aug 2006) | 3 lines Fix #1534738: win32 debug version of _msi must be _msi_d.pyd, not _msi.pyd. Fix the name of the pdb file as well. ........ r51115 | andrew.kuchling | 2006-08-04 22:37:43 +0200 (Fri, 04 Aug 2006) | 1 line Typo fixes ........ r51116 | andrew.kuchling | 2006-08-04 23:10:03 +0200 (Fri, 04 Aug 2006) | 1 line Fix mangled sentence ........ r51118 | tim.peters | 2006-08-05 00:00:35 +0200 (Sat, 05 Aug 2006) | 2 lines Whitespace normalization. ........ r51119 | bob.ippolito | 2006-08-05 01:59:21 +0200 (Sat, 05 Aug 2006) | 5 lines Fix #1530559, struct.pack raises TypeError where it used to convert. Passing float arguments to struct.pack when integers are expected now triggers a DeprecationWarning. ........ r51123 | georg.brandl | 2006-08-05 08:10:54 +0200 (Sat, 05 Aug 2006) | 3 lines Patch #1534922: correct and enhance unittest docs. ........ r51126 | georg.brandl | 2006-08-06 09:06:33 +0200 (Sun, 06 Aug 2006) | 2 lines Bug #1535182: really test the xreadlines() method of bz2 objects. ........ r51128 | georg.brandl | 2006-08-06 09:26:21 +0200 (Sun, 06 Aug 2006) | 4 lines Bug #1535081: A leading underscore has been added to the names of the md5 and sha modules, so add it in Modules/Setup.dist too. ........ r51129 | georg.brandl | 2006-08-06 10:23:54 +0200 (Sun, 06 Aug 2006) | 3 lines Bug #1535165: fixed a segfault in input() and raw_input() when sys.stdin is closed. ........ r51131 | georg.brandl | 2006-08-06 11:17:16 +0200 (Sun, 06 Aug 2006) | 2 lines Don't produce output in test_builtin. ........ r51133 | andrew.macintyre | 2006-08-06 14:37:03 +0200 (Sun, 06 Aug 2006) | 4 lines test_threading now skips testing alternate thread stack sizes on platforms that don't support changing thread stack size. ........ r51134 | andrew.kuchling | 2006-08-07 00:07:04 +0200 (Mon, 07 Aug 2006) | 2 lines [Patch #1464056] Ensure that we use the panelw library when linking with ncursesw. Once I see how the buildbots react, I'll backport this to 2.4. ........ r51137 | georg.brandl | 2006-08-08 13:52:34 +0200 (Tue, 08 Aug 2006) | 3 lines webbrowser: Silence stderr output if no gconftool or gnome browser found ........ r51138 | georg.brandl | 2006-08-08 13:56:21 +0200 (Tue, 08 Aug 2006) | 7 lines Remove "non-mapping" and "non-sequence" from TypeErrors raised by PyMapping_Size and PySequence_Size. Because len() tries first sequence, then mapping size, it will always raise a "non-mapping object has no len" error which is confusing. ........ r51139 | thomas.heller | 2006-08-08 19:37:00 +0200 (Tue, 08 Aug 2006) | 3 lines memcmp() can return values other than -1, 0, and +1 but tp_compare must not. ........ r51140 | thomas.heller | 2006-08-08 19:39:20 +0200 (Tue, 08 Aug 2006) | 1 line Remove accidently committed, duplicated test. ........ r51147 | andrew.kuchling | 2006-08-08 20:50:14 +0200 (Tue, 08 Aug 2006) | 1 line Reword paragraph to clarify ........ r51148 | andrew.kuchling | 2006-08-08 20:56:08 +0200 (Tue, 08 Aug 2006) | 1 line Move obmalloc item into C API section ........ r51149 | andrew.kuchling | 2006-08-08 21:00:14 +0200 (Tue, 08 Aug 2006) | 1 line 'Other changes' section now has only one item; move the item elsewhere and remove the section ........ r51150 | andrew.kuchling | 2006-08-08 21:00:34 +0200 (Tue, 08 Aug 2006) | 1 line Bump version number ........ r51151 | georg.brandl | 2006-08-08 22:11:22 +0200 (Tue, 08 Aug 2006) | 2 lines Bug #1536828: typo: TypeType should have been StringType. ........ r51153 | georg.brandl | 2006-08-08 22:13:13 +0200 (Tue, 08 Aug 2006) | 2 lines Bug #1536660: separate two words. ........ r51155 | georg.brandl | 2006-08-08 22:48:10 +0200 (Tue, 08 Aug 2006) | 3 lines ``str`` is now the same object as ``types.StringType``. ........ r51156 | tim.peters | 2006-08-09 02:52:26 +0200 (Wed, 09 Aug 2006) | 2 lines Whitespace normalization. ........ r51158 | georg.brandl | 2006-08-09 09:03:22 +0200 (Wed, 09 Aug 2006) | 4 lines Introduce an upper bound on tuple nesting depth in C argument format strings; fixes rest of #1523610. ........ r51160 | martin.v.loewis | 2006-08-09 09:57:39 +0200 (Wed, 09 Aug 2006) | 4 lines __hash__ may now return long int; the final hash value is obtained by invoking hash on the long int. Fixes #1536021. ........ r51168 | andrew.kuchling | 2006-08-09 15:03:41 +0200 (Wed, 09 Aug 2006) | 1 line [Bug #1536021] Mention __hash__ change ........ r51169 | andrew.kuchling | 2006-08-09 15:57:05 +0200 (Wed, 09 Aug 2006) | 1 line [Patch #1534027] Add notes on locale module changes ........ r51170 | andrew.kuchling | 2006-08-09 16:05:35 +0200 (Wed, 09 Aug 2006) | 1 line Add missing 'self' parameters ........ r51171 | andrew.kuchling | 2006-08-09 16:06:19 +0200 (Wed, 09 Aug 2006) | 1 line Reindent code ........ r51172 | armin.rigo | 2006-08-09 16:55:26 +0200 (Wed, 09 Aug 2006) | 2 lines Fix and test for an infinite C recursion. ........ r51173 | ronald.oussoren | 2006-08-09 16:56:33 +0200 (Wed, 09 Aug 2006) | 2 lines It's unlikely that future versions will require _POSIX_C_SOURCE ........ r51178 | armin.rigo | 2006-08-09 17:37:26 +0200 (Wed, 09 Aug 2006) | 2 lines Concatenation on a long string breaks (SF #1526585). ........ r51180 | kurt.kaiser | 2006-08-09 18:46:15 +0200 (Wed, 09 Aug 2006) | 8 lines 1. When used w/o subprocess, all exceptions were preceeded by an error message claiming they were IDLE internal errors (since 1.2a1). 2. Add Ronald Oussoren to CREDITS M NEWS.txt M PyShell.py M CREDITS.txt ........ r51181 | kurt.kaiser | 2006-08-09 19:47:15 +0200 (Wed, 09 Aug 2006) | 4 lines As a slight enhancement to the previous checkin, improve the internal error reporting by moving message to IDLE console. ........ r51182 | andrew.kuchling | 2006-08-09 20:23:14 +0200 (Wed, 09 Aug 2006) | 1 line Typo fix ........ r51183 | kurt.kaiser | 2006-08-09 22:34:46 +0200 (Wed, 09 Aug 2006) | 2 lines ToggleTab dialog was setting indent to 8 even if cancelled (since 1.2a1). ........ r51184 | martin.v.loewis | 2006-08-10 01:42:18 +0200 (Thu, 10 Aug 2006) | 2 lines Add some commentary on -mimpure-text. ........ r51185 | tim.peters | 2006-08-10 02:58:49 +0200 (Thu, 10 Aug 2006) | 2 lines Add missing svn:eol-style property to text files. ........ r51186 | kurt.kaiser | 2006-08-10 03:41:17 +0200 (Thu, 10 Aug 2006) | 2 lines Changing tokenize (39046) to detect dedent broke tabnanny check (since 1.2a1) ........ r51187 | tim.peters | 2006-08-10 05:01:26 +0200 (Thu, 10 Aug 2006) | 13 lines test_copytree_simple(): This was leaving behind two new temp directories each time it ran, at least on Windows. Several changes: explicitly closed all files; wrapped long lines; stopped suppressing errors when removing a file or directory fails (removing /shouldn't/ fail!); and changed what appeared to be incorrect usage of os.removedirs() (that doesn't remove empty directories at and /under/ the given path, instead it must be given an empty leaf directory and then deletes empty directories moving /up/ the path -- could be that the conceptually simpler shutil.rmtree() was really actually intended here). ........ --- Doc/ACKS | 1 + Doc/Makefile | 2 +- Doc/Makefile.deps | 1 + Doc/api/api.tex | 5 - Doc/api/concrete.tex | 76 +- Doc/api/exceptions.tex | 18 +- Doc/api/intro.tex | 56 + Doc/api/refcounts.dat | 5 + Doc/commontex/boilerplate.tex | 2 +- Doc/dist/dist.tex | 27 +- Doc/doc/doc.tex | 22 +- Doc/ext/newtypes.tex | 90 +- Doc/ext/windows.tex | 12 +- Doc/howto/Makefile | 120 +- Doc/howto/doanddont.tex | 5 +- Doc/howto/sockets.tex | 61 +- Doc/inst/inst.tex | 2 +- Doc/lib/email.tex | 11 +- Doc/lib/emailgenerator.tex | 4 +- Doc/lib/lib.tex | 10 +- Doc/lib/libanydbm.tex | 26 + Doc/lib/libbase64.tex | 12 + Doc/lib/libbinascii.tex | 9 +- Doc/lib/libbsddb.tex | 24 +- Doc/lib/libcompileall.tex | 13 + Doc/lib/libcookielib.tex | 2 +- Doc/lib/libcsv.tex | 16 +- Doc/lib/libctypes.tex | 1465 ++++++++++++++-- Doc/lib/libctypesref.tex | 457 ----- Doc/lib/libdifflib.tex | 10 + Doc/lib/libetree.tex | 367 ++++ Doc/lib/libfuncs.tex | 194 ++- Doc/lib/libgettext.tex | 6 +- Doc/lib/libimp.tex | 20 +- Doc/lib/libinspect.tex | 35 +- Doc/lib/liblinecache.tex | 2 +- Doc/lib/liblogging.tex | 14 +- Doc/lib/libmailbox.tex | 5 +- Doc/lib/libmimetypes.tex | 14 + Doc/lib/libnew.tex | 7 +- Doc/lib/liboptparse.tex | 231 ++- Doc/lib/libossaudiodev.tex | 2 +- Doc/lib/libpickle.tex | 45 +- Doc/lib/libpkgutil.tex | 2 +- Doc/lib/libposixpath.tex | 12 +- Doc/lib/librandom.tex | 26 +- Doc/lib/libre.tex | 2 +- Doc/lib/libreadline.tex | 11 +- Doc/lib/libsgmllib.tex | 54 +- Doc/lib/libshelve.tex | 4 +- Doc/lib/libsite.tex | 13 +- Doc/lib/libsocket.tex | 52 + Doc/lib/libsocksvr.tex | 6 +- Doc/lib/libsqlite3.tex | 49 +- Doc/lib/libstdtypes.tex | 76 +- Doc/lib/libstringio.tex | 37 + Doc/lib/libsubprocess.tex | 9 +- Doc/lib/libsys.tex | 43 +- Doc/lib/libtextwrap.tex | 15 +- Doc/lib/libthread.tex | 20 + Doc/lib/libthreading.tex | 20 + Doc/lib/libtime.tex | 4 +- Doc/lib/libturtle.tex | 123 +- Doc/lib/libtypes.tex | 24 + Doc/lib/libundoc.tex | 2 +- Doc/lib/libunicodedata.tex | 25 +- Doc/lib/libunittest.tex | 316 ++-- Doc/lib/liburllib.tex | 8 +- Doc/lib/liburllib2.tex | 16 +- Doc/lib/libuuid.tex | 234 +++ Doc/lib/libwarnings.tex | 38 +- Doc/lib/libweakref.tex | 92 +- Doc/lib/libwebbrowser.tex | 12 + Doc/lib/libwsgiref.tex | 781 +++++++++ Doc/lib/libzipfile.tex | 40 +- Doc/lib/libzipimport.tex | 6 +- Doc/lib/sqlite3/complete_statement.py | 2 +- Doc/lib/tkinter.tex | 62 +- Doc/mac/libmacfs.tex | 6 +- Doc/mac/libmacos.tex | 2 +- Doc/mac/using.tex | 2 +- Doc/ref/ref2.tex | 2 +- Doc/ref/ref3.tex | 31 +- Doc/ref/ref4.tex | 18 +- Doc/ref/ref5.tex | 8 +- Doc/ref/ref8.tex | 2 +- Doc/tut/tut.tex | 33 +- Doc/whatsnew/whatsnew20.tex | 14 +- Doc/whatsnew/whatsnew21.tex | 4 +- Doc/whatsnew/whatsnew23.tex | 4 +- Doc/whatsnew/whatsnew24.tex | 2 +- Doc/whatsnew/whatsnew25.tex | 482 ++++-- Include/Python.h | 2 +- Include/frameobject.h | 4 +- Include/pyerrors.h | 6 +- Include/pyexpat.h | 4 +- Include/pyport.h | 8 +- Include/pystate.h | 5 + Include/pythread.h | 3 + Include/setobject.h | 6 +- Include/unicodeobject.h | 7 + Include/weakrefobject.h | 2 +- Lib/Queue.py | 8 +- Lib/SimpleHTTPServer.py | 2 + Lib/UserString.py | 11 +- Lib/_MozillaCookieJar.py | 3 +- Lib/binhex.py | 25 +- Lib/bsddb/__init__.py | 64 +- Lib/bsddb/dbrecio.py | 6 +- Lib/bsddb/dbtables.py | 6 + Lib/bsddb/dbutils.py | 6 +- Lib/bsddb/test/test_basics.py | 37 +- Lib/compiler/future.py | 7 - Lib/compiler/pycodegen.py | 39 +- Lib/compiler/symbols.py | 2 +- Lib/compiler/transformer.py | 52 +- Lib/ctypes/__init__.py | 87 +- Lib/ctypes/_endian.py | 3 + Lib/ctypes/macholib/__init__.py | 3 + Lib/ctypes/macholib/dyld.py | 3 + Lib/ctypes/macholib/dylib.py | 3 + Lib/ctypes/macholib/framework.py | 3 + Lib/ctypes/test/test_anon.py | 60 + Lib/ctypes/test/test_cast.py | 37 +- Lib/ctypes/test/test_keeprefs.py | 9 +- Lib/ctypes/test/test_loading.py | 10 +- Lib/ctypes/test/test_objects.py | 70 + Lib/ctypes/test/test_parameters.py | 35 + Lib/ctypes/test/test_pointers.py | 17 + Lib/ctypes/test/test_slicing.py | 26 +- Lib/ctypes/test/test_structures.py | 22 +- Lib/ctypes/test/test_varsize_struct.py | 50 + Lib/ctypes/test/test_win32.py | 14 +- Lib/ctypes/util.py | 16 +- Lib/ctypes/wintypes.py | 127 +- Lib/difflib.py | 65 +- Lib/distutils/__init__.py | 4 +- Lib/distutils/command/bdist_rpm.py | 7 +- Lib/distutils/command/upload.py | 2 +- Lib/distutils/msvccompiler.py | 8 +- Lib/distutils/sysconfig.py | 2 +- Lib/distutils/unixccompiler.py | 2 +- Lib/doctest.py | 7 +- Lib/dummy_thread.py | 7 + Lib/email/__init__.py | 2 +- Lib/email/message.py | 13 +- Lib/email/test/test_email.py | 189 ++- Lib/email/test/test_email_renamed.py | 189 ++- Lib/email/utils.py | 61 +- Lib/encodings/mbcs.py | 35 +- Lib/encodings/punycode.py | 6 +- Lib/encodings/utf_8_sig.py | 4 +- Lib/encodings/uu_codec.py | 4 +- Lib/gzip.py | 8 +- Lib/httplib.py | 8 +- Lib/idlelib/Bindings.py | 26 + Lib/idlelib/CREDITS.txt | 15 +- Lib/idlelib/CallTipWindow.py | 6 +- Lib/idlelib/CallTips.py | 2 +- Lib/idlelib/CodeContext.py | 3 +- Lib/idlelib/ColorDelegator.py | 30 +- Lib/idlelib/Debugger.py | 9 +- Lib/idlelib/EditorWindow.py | 49 +- Lib/idlelib/NEWS.txt | 43 + Lib/idlelib/ParenMatch.py | 7 +- Lib/idlelib/PyShell.py | 31 +- Lib/idlelib/ScriptBinding.py | 5 +- Lib/idlelib/ZoomHeight.py | 9 + Lib/idlelib/buildapp.py | 17 - Lib/idlelib/config-keys.def | 53 + Lib/idlelib/configHandler.py | 16 +- Lib/idlelib/configHelpSourceEdit.py | 5 +- Lib/idlelib/idlever.py | 2 +- Lib/idlelib/keybindingDialog.py | 4 +- Lib/idlelib/macosxSupport.py | 112 ++ Lib/inspect.py | 58 +- Lib/lib-tk/Tkinter.py | 99 +- Lib/lib-tk/tkMessageBox.py | 7 +- Lib/lib-tk/turtle.py | 109 +- Lib/linecache.py | 4 + Lib/logging/config.py | 1 + Lib/logging/handlers.py | 42 +- Lib/mailbox.py | 30 +- Lib/mimetypes.py | 4 + Lib/msilib/__init__.py | 10 +- Lib/optparse.py | 18 +- Lib/os.py | 2 +- Lib/pdb.py | 3 +- Lib/pkgutil.py | 53 +- Lib/popen2.py | 4 +- Lib/pstats.py | 9 +- Lib/pydoc.py | 22 +- Lib/random.py | 18 +- Lib/sgmllib.py | 99 +- Lib/shelve.py | 3 + Lib/shutil.py | 8 +- Lib/site.py | 9 +- Lib/socket.py | 29 +- Lib/sqlite3/test/hooks.py | 2 + Lib/sqlite3/test/regression.py | 8 + Lib/sqlite3/test/types.py | 52 +- Lib/sqlite3/test/userfunctions.py | 107 +- Lib/string.py | 6 +- Lib/struct.py | 2 +- Lib/subprocess.py | 27 +- Lib/tarfile.py | 15 +- Lib/telnetlib.py | 16 +- Lib/tempfile.py | 2 +- Lib/test/crashers/bogus_code_obj.py | 19 + Lib/test/crashers/borrowed_ref_1.py | 29 + Lib/test/crashers/borrowed_ref_2.py | 38 + Lib/test/crashers/coerce.py | 9 - Lib/test/crashers/gc_inspection.py | 32 + Lib/test/crashers/infinite_rec_3.py | 9 - Lib/test/crashers/recursion_limit_too_high.py | 16 + Lib/test/crashers/recursive_call.py | 5 + Lib/test/crashers/xml_parsers.py | 56 - Lib/test/fork_wait.py | 9 +- Lib/test/output/test_ossaudiodev | 3 +- Lib/test/output/test_thread | 12 + Lib/test/regrtest.py | 86 +- Lib/test/string_tests.py | 43 +- Lib/test/test__locale.py | 3 + Lib/test/test_ast.py | 2 +- Lib/test/test_asynchat.py | 3 +- Lib/test/test_bigaddrspace.py | 46 + Lib/test/test_bigmem.py | 2 +- Lib/test/test_bsddb.py | 3 +- Lib/test/test_builtin.py | 30 +- Lib/test/test_bz2.py | 3 +- Lib/test/test_cmd_line.py | 1 + Lib/test/test_code.py | 17 + Lib/test/test_codecs.py | 6 + Lib/test/test_commands.py | 3 +- Lib/test/test_compile.py | 35 + Lib/test/test_compiler.py | 38 +- Lib/test/test_curses.py | 7 + Lib/test/test_defaultdict.py | 6 +- Lib/test/test_descr.py | 34 + Lib/test/test_dis.py | 24 + Lib/test/test_doctest.py | 7 - Lib/test/test_email_codecs.py | 12 +- Lib/test/test_exceptions.py | 12 + Lib/test/test_fcntl.py | 7 +- Lib/test/test_file.py | 78 +- Lib/test/test_filecmp.py | 3 +- Lib/test/test_fork1.py | 13 +- Lib/test/test_generators.py | 39 +- Lib/test/test_genexps.py | 2 +- Lib/test/test_getargs2.py | 19 +- Lib/test/test_grammar.py | 5 + Lib/test/test_inspect.py | 32 +- Lib/test/test_iterlen.py | 7 +- Lib/test/test_logging.py | 2 + Lib/test/test_mailbox.py | 29 +- Lib/test/test_mimetools.py | 4 +- Lib/test/test_mimetypes.py | 5 +- Lib/test/test_minidom.py | 92 +- Lib/test/test_multibytecodec.py | 38 +- Lib/test/test_optparse.py | 49 +- Lib/test/test_os.py | 14 + Lib/test/test_ossaudiodev.py | 37 +- Lib/test/test_pep292.py | 7 + Lib/test/test_popen.py | 3 +- Lib/test/test_popen2.py | 3 +- Lib/test/test_pyexpat.py | 21 + Lib/test/test_sax.py | 82 +- Lib/test/test_scope.py | 11 + Lib/test/test_select.py | 3 +- Lib/test/test_sgmllib.py | 111 +- Lib/test/test_shutil.py | 47 + Lib/test/test_signal.py | 51 +- Lib/test/test_socket.py | 67 +- Lib/test/test_socket_ssl.py | 26 +- Lib/test/test_socketserver.py | 30 +- Lib/test/test_struct.py | 95 +- Lib/test/test_subprocess.py | 49 +- Lib/test/test_support.py | 82 +- Lib/test/test_sys.py | 84 + Lib/test/test_tcl.py | 6 +- Lib/test/test_textwrap.py | 61 +- Lib/test/test_thread.py | 43 + Lib/test/test_threaded_import.py | 7 +- Lib/test/test_threadedtempfile.py | 4 +- Lib/test/test_threading.py | 26 + Lib/test/test_time.py | 44 +- Lib/test/test_timeout.py | 2 +- Lib/test/test_trace.py | 4 +- Lib/test/test_traceback.py | 45 +- Lib/test/test_types.py | 8 + Lib/test/test_urllib2.py | 14 +- Lib/test/test_urllib2net.py | 13 +- Lib/test/test_urllibnet.py | 2 +- Lib/test/test_uuid.py | 434 +++++ Lib/test/test_wait3.py | 12 +- Lib/test/test_wait4.py | 12 +- Lib/test/test_warnings.py | 13 + Lib/test/test_winreg.py | 3 + Lib/test/test_wsgiref.py | 615 +++++++ Lib/test/test_xml_etree.py | 44 +- Lib/test/test_xml_etree_c.py | 6 +- Lib/test/test_zipfile.py | 252 ++- Lib/test/test_zipfile64.py | 101 ++ Lib/test/test_zlib.py | 116 +- Lib/textwrap.py | 75 +- Lib/threading.py | 4 +- Lib/trace.py | 2 +- Lib/traceback.py | 85 +- Lib/types.py | 14 +- Lib/urllib.py | 12 +- Lib/urllib2.py | 26 +- Lib/uuid.py | 515 ++++++ Lib/warnings.py | 6 +- Lib/webbrowser.py | 7 +- Lib/wsgiref.egg-info | 8 + Lib/wsgiref/__init__.py | 23 + Lib/wsgiref/handlers.py | 492 ++++++ Lib/wsgiref/headers.py | 205 +++ Lib/wsgiref/simple_server.py | 205 +++ Lib/wsgiref/util.py | 205 +++ Lib/wsgiref/validate.py | 432 +++++ Lib/xml.py | 47 - Lib/xml/__init__.py | 47 + Lib/xml/dom/NodeFilter.py | 27 + Lib/xml/dom/__init__.py | 139 ++ Lib/xml/dom/domreg.py | 99 ++ Lib/xml/dom/expatbuilder.py | 983 +++++++++++ Lib/xml/dom/minicompat.py | 110 ++ Lib/xml/dom/minidom.py | 1936 ++++++++++++++++++++++ Lib/xml/dom/pulldom.py | 351 ++++ Lib/xml/dom/xmlbuilder.py | 386 +++++ Lib/xml/etree/ElementInclude.py | 143 ++ Lib/xml/etree/ElementPath.py | 198 +++ Lib/xml/etree/ElementTree.py | 1260 ++++++++++++++ Lib/xml/etree/__init__.py | 33 + Lib/xml/etree/cElementTree.py | 3 + Lib/xml/parsers/__init__.py | 8 + Lib/xml/parsers/expat.py | 4 + Lib/xml/sax/__init__.py | 108 ++ Lib/xml/sax/_exceptions.py | 131 ++ Lib/xml/sax/expatreader.py | 414 +++++ Lib/xml/sax/handler.py | 342 ++++ Lib/xml/sax/saxutils.py | 299 ++++ Lib/xml/sax/xmlreader.py | 381 +++++ Lib/xmlcore/__init__.py | 20 - Lib/xmlcore/dom/NodeFilter.py | 27 - Lib/xmlcore/dom/__init__.py | 139 -- Lib/xmlcore/dom/domreg.py | 99 -- Lib/xmlcore/dom/expatbuilder.py | 983 ----------- Lib/xmlcore/dom/minicompat.py | 109 -- Lib/xmlcore/dom/minidom.py | 1936 ---------------------- Lib/xmlcore/dom/pulldom.py | 351 ---- Lib/xmlcore/dom/xmlbuilder.py | 386 ----- Lib/xmlcore/etree/ElementInclude.py | 143 -- Lib/xmlcore/etree/ElementPath.py | 198 --- Lib/xmlcore/etree/ElementTree.py | 1257 -------------- Lib/xmlcore/etree/__init__.py | 33 - Lib/xmlcore/etree/cElementTree.py | 3 - Lib/xmlcore/parsers/__init__.py | 8 - Lib/xmlcore/parsers/expat.py | 4 - Lib/xmlcore/sax/__init__.py | 108 -- Lib/xmlcore/sax/_exceptions.py | 131 -- Lib/xmlcore/sax/expatreader.py | 414 ----- Lib/xmlcore/sax/handler.py | 342 ---- Lib/xmlcore/sax/saxutils.py | 297 ---- Lib/xmlcore/sax/xmlreader.py | 381 ----- Lib/zipfile.py | 385 ++++- Mac/BuildScript/README.txt | 44 +- Mac/BuildScript/build-installer.py | 120 +- Mac/BuildScript/resources/Welcome.rtf | 7 +- Mac/BuildScript/scripts/postflight.framework | 2 +- Mac/BuildScript/scripts/postflight.patch-profile | 28 +- Mac/Demo/applescript.html | 6 +- Mac/Demo/calldll/readme | 48 - Mac/Demo/calldll/samplecalldll.py | 24 - Mac/Demo/calldll/testcalldll.py | 132 -- Mac/Demo/index.html | 18 +- Mac/IDLE/config-main.def | 2 +- Mac/Makefile.in | 2 +- Mac/Modules/MacOS.c | 644 +++++++ Mac/Modules/ae/_AEmodule.c | 39 +- Mac/Modules/macosmodule.c | 643 ------- Mac/PythonLauncher/FileSettings.m | 21 +- Mac/Tools/fixapplepython23.py | 10 + Makefile.pre.in | 28 +- Misc/ACKS | 6 + Misc/README.coverity | 22 + Misc/README.klocwork | 26 + Misc/RPM/python-2.5.spec | 2 +- Misc/Vim/python.vim | 2 +- Misc/build.sh | 28 +- Misc/cheatsheet | 1 - Misc/python-config.in | 7 +- Misc/valgrind-python.supp | 9 + Modules/Setup.dist | 13 +- Modules/_bsddb.c | 54 +- Modules/_codecsmodule.c | 15 +- Modules/_ctypes/_ctypes.c | 172 +- Modules/_ctypes/_ctypes_test.c | 5 + Modules/_ctypes/callbacks.c | 21 +- Modules/_ctypes/callproc.c | 109 +- Modules/_ctypes/cfield.c | 37 +- Modules/_ctypes/ctypes.h | 11 +- Modules/_ctypes/ctypes_dlfcn.h | 4 +- Modules/_ctypes/libffi/configure | 136 +- Modules/_ctypes/libffi/configure.ac | 6 + Modules/_ctypes/libffi/src/x86/darwin.S | 56 +- Modules/_ctypes/libffi/src/x86/ffi_darwin.c | 144 +- Modules/_ctypes/libffi_msvc/ffi.c | 23 +- Modules/_ctypes/libffi_msvc/fficonfig.h | 4 +- Modules/_ctypes/libffi_msvc/ffitarget.h | 6 +- Modules/_ctypes/libffi_msvc/mingwin32.S | 228 --- Modules/_ctypes/malloc_closure.c | 4 + Modules/_ctypes/stgdict.c | 154 +- Modules/_cursesmodule.c | 76 +- Modules/_elementtree.c | 15 +- Modules/_hotshot.c | 4 + Modules/_localemodule.c | 2 +- Modules/_sqlite/connection.c | 134 +- Modules/_sqlite/cursor.c | 56 +- Modules/_sqlite/module.c | 150 +- Modules/_sqlite/module.h | 4 +- Modules/_sqlite/util.c | 2 +- Modules/_sqlite/util.h | 2 +- Modules/_sre.c | 191 +-- Modules/_ssl.c | 29 +- Modules/_struct.c | 95 +- Modules/_testcapimodule.c | 20 + Modules/_tkinter.c | 9 +- Modules/_typesmodule.c | 94 ++ Modules/_weakref.c | 6 +- Modules/arraymodule.c | 5 +- Modules/binascii.c | 2 +- Modules/bz2module.c | 42 +- Modules/cPickle.c | 27 +- Modules/collectionsmodule.c | 21 +- Modules/config.c.in | 4 + Modules/cryptmodule.c | 5 + Modules/dlmodule.c | 26 +- Modules/expat/Makefile.in | 158 -- Modules/expat/amigaconfig.h | 96 ++ Modules/expat/expat.h | 35 +- Modules/expat/expat_external.h | 27 + Modules/expat/pyexpatns.h | 124 ++ Modules/expat/xmlparse.c | 333 ++-- Modules/expat/xmlrole.c | 4 +- Modules/expat/xmltok.c | 6 +- Modules/expat/xmltok.h | 4 +- Modules/expat/xmltok_impl.c | 6 +- Modules/expat/xmltok_ns.c | 2 +- Modules/fcntlmodule.c | 4 +- Modules/fpectlmodule.c | 25 + Modules/getpath.c | 10 +- Modules/itertoolsmodule.c | 10 +- Modules/main.c | 12 +- Modules/mmapmodule.c | 3 + Modules/posixmodule.c | 73 +- Modules/pyexpat.c | 32 +- Modules/readline.c | 20 +- Modules/selectmodule.c | 20 +- Modules/socketmodule.c | 224 ++- Modules/spwdmodule.c | 2 +- Modules/sre.h | 24 +- Modules/threadmodule.c | 57 + Modules/timemodule.c | 42 +- Modules/unicodedata.c | 11 +- Modules/zlibmodule.c | 6 + Objects/abstract.c | 82 +- Objects/bufferobject.c | 89 +- Objects/cellobject.c | 2 + Objects/classobject.c | 11 +- Objects/codeobject.c | 1 + Objects/complexobject.c | 14 +- Objects/descrobject.c | 25 +- Objects/dictnotes.txt | 2 +- Objects/dictobject.c | 7 +- Objects/exceptions.c | 35 + Objects/fileobject.c | 23 +- Objects/frameobject.c | 15 +- Objects/funcobject.c | 6 +- Objects/listobject.c | 4 +- Objects/listsort.txt | 2 +- Objects/longobject.c | 8 +- Objects/object.c | 30 +- Objects/setobject.c | 52 +- Objects/stringobject.c | 335 ++-- Objects/typeobject.c | 53 +- Objects/unicodeobject.c | 295 +++- Objects/weakrefobject.c | 4 +- PC/_winreg.c | 2 + PC/config.c | 3 + PC/getpathp.c | 10 + PC/os2emx/Makefile | 24 +- PC/os2emx/README.os2emx | 29 +- PC/os2emx/config.c | 18 +- PC/os2emx/pyconfig.h | 31 + PC/os2emx/python24.def | 1173 ------------- PC/os2emx/python25.def | 1314 +++++++++++++++ PC/os2vacpp/pyconfig.h | 8 + PC/pyconfig.h | 52 +- PC/winsound.c | 8 +- PCbuild/_msi.vcproj | 4 +- PCbuild/_ssl.vcproj | 18 +- PCbuild/build_ssl.bat | 10 + PCbuild/build_ssl.py | 97 +- PCbuild/pythoncore.vcproj | 15 +- PCbuild/readme.txt | 13 +- Python/ast.c | 126 +- Python/ceval.c | 18 +- Python/codecs.c | 9 +- Python/compile.c | 135 +- Python/dynload_win.c | 2 + Python/errors.c | 15 +- Python/future.c | 4 +- Python/getargs.c | 15 +- Python/getopt.c | 31 +- Python/import.c | 201 ++- Python/mactoolboxglue.c | 3 +- Python/mystrtoul.c | 43 +- Python/pyarena.c | 8 +- Python/pystate.c | 57 +- Python/pystrtod.c | 7 + Python/pythonrun.c | 49 +- Python/strtod.c | 2 +- Python/symtable.c | 16 +- Python/sysmodule.c | 90 +- Python/thread.c | 29 + Python/thread_nt.h | 31 +- Python/thread_os2.h | 44 +- Python/thread_pthread.h | 77 +- README | 38 +- RISCOS/pyconfig.h | 25 +- Tools/buildbot/Makefile | 6 + Tools/buildbot/kill_python.c | 14 +- Tools/msi/msi.py | 18 +- Tools/msi/uuids.py | 1 + Tools/pybench/Arithmetic.py | 16 +- Tools/pybench/Calls.py | 107 +- Tools/pybench/CommandLine.py | 2 +- Tools/pybench/Constructs.py | 10 +- Tools/pybench/Dict.py | 78 +- Tools/pybench/Exceptions.py | 30 +- Tools/pybench/Imports.py | 12 +- Tools/pybench/Instances.py | 4 +- Tools/pybench/Lists.py | 146 +- Tools/pybench/Lookups.py | 10 +- Tools/pybench/NewInstances.py | 11 +- Tools/pybench/Numbers.py | 12 +- Tools/pybench/README | 371 ++--- Tools/pybench/Setup.py | 6 +- Tools/pybench/Strings.py | 24 +- Tools/pybench/Tuples.py | 16 +- Tools/pybench/Unicode.py | 18 +- Tools/pybench/clockres.py | 43 + Tools/pybench/pybench.py | 934 ++++++++--- Tools/pybench/systimes.py | 44 +- Tools/scripts/README | 5 +- Tools/webchecker/webchecker.py | 3 +- configure | 816 ++++++--- configure.in | 81 +- pyconfig.h.in | 30 + setup.py | 53 +- 562 files changed, 28787 insertions(+), 15167 deletions(-) delete mode 100644 Doc/lib/libctypesref.tex create mode 100644 Doc/lib/libetree.tex create mode 100644 Doc/lib/libuuid.tex create mode 100755 Doc/lib/libwsgiref.tex create mode 100644 Lib/ctypes/test/test_anon.py create mode 100644 Lib/ctypes/test/test_objects.py create mode 100644 Lib/ctypes/test/test_varsize_struct.py delete mode 100644 Lib/idlelib/buildapp.py create mode 100644 Lib/idlelib/macosxSupport.py create mode 100644 Lib/test/crashers/bogus_code_obj.py create mode 100644 Lib/test/crashers/borrowed_ref_1.py create mode 100644 Lib/test/crashers/borrowed_ref_2.py delete mode 100644 Lib/test/crashers/coerce.py create mode 100644 Lib/test/crashers/gc_inspection.py delete mode 100644 Lib/test/crashers/infinite_rec_3.py create mode 100644 Lib/test/crashers/recursion_limit_too_high.py delete mode 100644 Lib/test/crashers/xml_parsers.py create mode 100644 Lib/test/test_bigaddrspace.py create mode 100644 Lib/test/test_uuid.py create mode 100755 Lib/test/test_wsgiref.py create mode 100644 Lib/test/test_zipfile64.py create mode 100644 Lib/uuid.py create mode 100644 Lib/wsgiref.egg-info create mode 100644 Lib/wsgiref/__init__.py create mode 100644 Lib/wsgiref/handlers.py create mode 100644 Lib/wsgiref/headers.py create mode 100644 Lib/wsgiref/simple_server.py create mode 100644 Lib/wsgiref/util.py create mode 100644 Lib/wsgiref/validate.py delete mode 100644 Lib/xml.py create mode 100644 Lib/xml/__init__.py create mode 100644 Lib/xml/dom/NodeFilter.py create mode 100644 Lib/xml/dom/__init__.py create mode 100644 Lib/xml/dom/domreg.py create mode 100644 Lib/xml/dom/expatbuilder.py create mode 100644 Lib/xml/dom/minicompat.py create mode 100644 Lib/xml/dom/minidom.py create mode 100644 Lib/xml/dom/pulldom.py create mode 100644 Lib/xml/dom/xmlbuilder.py create mode 100644 Lib/xml/etree/ElementInclude.py create mode 100644 Lib/xml/etree/ElementPath.py create mode 100644 Lib/xml/etree/ElementTree.py create mode 100644 Lib/xml/etree/__init__.py create mode 100644 Lib/xml/etree/cElementTree.py create mode 100644 Lib/xml/parsers/__init__.py create mode 100644 Lib/xml/parsers/expat.py create mode 100644 Lib/xml/sax/__init__.py create mode 100644 Lib/xml/sax/_exceptions.py create mode 100644 Lib/xml/sax/expatreader.py create mode 100644 Lib/xml/sax/handler.py create mode 100644 Lib/xml/sax/saxutils.py create mode 100644 Lib/xml/sax/xmlreader.py delete mode 100644 Lib/xmlcore/__init__.py delete mode 100644 Lib/xmlcore/dom/NodeFilter.py delete mode 100644 Lib/xmlcore/dom/__init__.py delete mode 100644 Lib/xmlcore/dom/domreg.py delete mode 100644 Lib/xmlcore/dom/expatbuilder.py delete mode 100644 Lib/xmlcore/dom/minicompat.py delete mode 100644 Lib/xmlcore/dom/minidom.py delete mode 100644 Lib/xmlcore/dom/pulldom.py delete mode 100644 Lib/xmlcore/dom/xmlbuilder.py delete mode 100644 Lib/xmlcore/etree/ElementInclude.py delete mode 100644 Lib/xmlcore/etree/ElementPath.py delete mode 100644 Lib/xmlcore/etree/ElementTree.py delete mode 100644 Lib/xmlcore/etree/__init__.py delete mode 100644 Lib/xmlcore/etree/cElementTree.py delete mode 100644 Lib/xmlcore/parsers/__init__.py delete mode 100644 Lib/xmlcore/parsers/expat.py delete mode 100644 Lib/xmlcore/sax/__init__.py delete mode 100644 Lib/xmlcore/sax/_exceptions.py delete mode 100644 Lib/xmlcore/sax/expatreader.py delete mode 100644 Lib/xmlcore/sax/handler.py delete mode 100644 Lib/xmlcore/sax/saxutils.py delete mode 100644 Lib/xmlcore/sax/xmlreader.py delete mode 100644 Mac/Demo/calldll/readme delete mode 100644 Mac/Demo/calldll/samplecalldll.py delete mode 100644 Mac/Demo/calldll/testcalldll.py create mode 100644 Mac/Modules/MacOS.c delete mode 100644 Mac/Modules/macosmodule.c create mode 100644 Misc/README.coverity create mode 100644 Misc/README.klocwork delete mode 100644 Modules/_ctypes/libffi_msvc/mingwin32.S create mode 100644 Modules/_typesmodule.c delete mode 100644 Modules/expat/Makefile.in create mode 100644 Modules/expat/amigaconfig.h create mode 100644 Modules/expat/pyexpatns.h delete mode 100644 PC/os2emx/python24.def create mode 100644 PC/os2emx/python25.def create mode 100644 PCbuild/build_ssl.bat create mode 100644 Tools/buildbot/Makefile mode change 100755 => 100644 Tools/pybench/NewInstances.py create mode 100644 Tools/pybench/clockres.py diff --git a/Doc/ACKS b/Doc/ACKS index bbb3241..3c2662d 100644 --- a/Doc/ACKS +++ b/Doc/ACKS @@ -190,6 +190,7 @@ Eddy Welbourne Mats Wichmann Gerry Wiener Timothy Wild +Collin Winter Blake Winton Dan Wolfe Steven Work diff --git a/Doc/Makefile b/Doc/Makefile index 0d391af..a435f11 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -691,7 +691,7 @@ distlatex: bziplatex ziplatex # The small amount of additional work is a small price to pay for not # having to remember which order to do it in. ;) paperdist: distpdf distps pkglist -edist: disthtml distinfo zipisilo pkglist +edist: disthtml pkglist # The pkglist.html file is used as part of the download.html page on # python.org; it is not used as intermediate input here or as part of diff --git a/Doc/Makefile.deps b/Doc/Makefile.deps index 2fc3250..f828e1b 100644 --- a/Doc/Makefile.deps +++ b/Doc/Makefile.deps @@ -270,6 +270,7 @@ LIBFILES= $(MANSTYLES) $(INDEXSTYLES) $(COMMONTEX) \ lib/xmlsaxhandler.tex \ lib/xmlsaxutils.tex \ lib/xmlsaxreader.tex \ + lib/libetree.tex \ lib/libqueue.tex \ lib/liblocale.tex \ lib/libgettext.tex \ diff --git a/Doc/api/api.tex b/Doc/api/api.tex index 6fa8c41..cf28f5b 100644 --- a/Doc/api/api.tex +++ b/Doc/api/api.tex @@ -48,11 +48,6 @@ code releases.} \input{newtypes} -% \chapter{Debugging \label{debugging}} -% -% XXX Explain Py_DEBUG, Py_TRACE_REFS, Py_REF_DEBUG. - - \appendix \chapter{Reporting Bugs} \input{reportingbugs} diff --git a/Doc/api/concrete.tex b/Doc/api/concrete.tex index 10247ab..4c7487c 100644 --- a/Doc/api/concrete.tex +++ b/Doc/api/concrete.tex @@ -31,7 +31,7 @@ This section describes Python type objects and the singleton object \begin{cvardesc}{PyObject*}{PyType_Type} This is the type object for type objects; it is the same object as - \code{types.TypeType} in the Python layer. + \code{type} and \code{types.TypeType} in the Python layer. \withsubitem{(in module types)}{\ttindex{TypeType}} \end{cvardesc} @@ -117,7 +117,8 @@ There is no \cfunction{PyNone_Check()} function for the same reason. \begin{cvardesc}{PyTypeObject}{PyInt_Type} This instance of \ctype{PyTypeObject} represents the Python plain - integer type. This is the same object as \code{types.IntType}. + integer type. This is the same object as \code{int} and + \code{types.IntType}. \withsubitem{(in modules types)}{\ttindex{IntType}} \end{cvardesc} @@ -260,7 +261,8 @@ booleans. The following macros are available, however. \begin{cvardesc}{PyTypeObject}{PyLong_Type} This instance of \ctype{PyTypeObject} represents the Python long - integer type. This is the same object as \code{types.LongType}. + integer type. This is the same object as \code{long} and + \code{types.LongType}. \withsubitem{(in modules types)}{\ttindex{LongType}} \end{cvardesc} @@ -376,7 +378,7 @@ booleans. The following macros are available, however. \versionadded{2.3} \end{cfuncdesc} -\begin{cfuncdesc}{unsigned long}{PyLong_AsUnsignedLongLongMask}{PyObject *io} +\begin{cfuncdesc}{unsigned PY_LONG_LONG}{PyLong_AsUnsignedLongLongMask}{PyObject *io} Return a C \ctype{unsigned long long} from a Python long integer, without checking for overflow. \versionadded{2.3} @@ -411,7 +413,8 @@ booleans. The following macros are available, however. \begin{cvardesc}{PyTypeObject}{PyFloat_Type} This instance of \ctype{PyTypeObject} represents the Python floating - point type. This is the same object as \code{types.FloatType}. + point type. This is the same object as \code{float} and + \code{types.FloatType}. \withsubitem{(in modules types)}{\ttindex{FloatType}} \end{cvardesc} @@ -520,7 +523,8 @@ typedef struct { \begin{cvardesc}{PyTypeObject}{PyComplex_Type} This instance of \ctype{PyTypeObject} represents the Python complex - number type. + number type. It is the same object as \code{complex} and + \code{types.ComplexType}. \end{cvardesc} \begin{cfuncdesc}{int}{PyComplex_Check}{PyObject *p} @@ -580,8 +584,8 @@ parameter and are called with a non-string parameter. \begin{cvardesc}{PyTypeObject}{PyString_Type} This instance of \ctype{PyTypeObject} represents the Python string - type; it is the same object as \code{types.TypeType} in the Python - layer. + type; it is the same object as \code{str} and \code{types.StringType} + in the Python layer. \withsubitem{(in module types)}{\ttindex{StringType}}. \end{cvardesc} @@ -850,7 +854,8 @@ Please keep this in mind when writing extensions or interfaces. \begin{cvardesc}{PyTypeObject}{PyUnicode_Type} This instance of \ctype{PyTypeObject} represents the Python Unicode - type. + type. It is exposed to Python code as \code{unicode} and + \code{types.UnicodeType}. \end{cvardesc} The following APIs are really C macros and can be used to do fast @@ -1001,21 +1006,14 @@ use these APIs: const char *errors} Coerce an encoded object \var{obj} to an Unicode object and return a reference with incremented refcount. + + String and other char buffer compatible objects are decoded + according to the given encoding and using the error handling + defined by errors. Both can be \NULL{} to have the interface + use the default values (see the next section for details). - Coercion is done in the following way: - -\begin{enumerate} -\item Unicode objects are passed back as-is with incremented - refcount. \note{These cannot be decoded; passing a non-\NULL{} - value for encoding will result in a \exception{TypeError}.} - -\item String and other char buffer compatible objects are decoded - according to the given encoding and using the error handling - defined by errors. Both can be \NULL{} to have the interface - use the default values (see the next section for details). - -\item All other objects cause an exception. -\end{enumerate} + All other objects, including Unicode objects, cause a + \exception{TypeError} to be set. The API returns \NULL{} if there was an error. The caller is responsible for decref'ing the returned objects. @@ -1431,6 +1429,18 @@ machine running the codec. raised by the codec. \end{cfuncdesc} +\begin{cfuncdesc}{PyObject*}{PyUnicode_DecodeMBCSStateful}{const char *s, + int size, + const char *errors, + int *consumed} + If \var{consumed} is \NULL{}, behave like + \cfunction{PyUnicode_DecodeMBCS()}. If \var{consumed} is not \NULL{}, + \cfunction{PyUnicode_DecodeMBCSStateful()} will not decode trailing lead + byte and the number of bytes that have been decoded will be stored in + \var{consumed}. + \versionadded{2.5} +\end{cfuncdesc} + \begin{cfuncdesc}{PyObject*}{PyUnicode_EncodeMBCS}{const Py_UNICODE *s, Py_ssize_t size, const char *errors} @@ -1618,8 +1628,9 @@ format. \begin{cvardesc}{PyTypeObject}{PyBuffer_Type} The instance of \ctype{PyTypeObject} which represents the Python - buffer type; it is the same object as \code{types.BufferType} in the - Python layer.\withsubitem{(in module types)}{\ttindex{BufferType}}. + buffer type; it is the same object as \code{buffer} and + \code{types.BufferType} in the Python layer. + \withsubitem{(in module types)}{\ttindex{BufferType}}. \end{cvardesc} \begin{cvardesc}{int}{Py_END_OF_BUFFER} @@ -1693,8 +1704,8 @@ format. \begin{cvardesc}{PyTypeObject}{PyTuple_Type} This instance of \ctype{PyTypeObject} represents the Python tuple - type; it is the same object as \code{types.TupleType} in the Python - layer.\withsubitem{(in module types)}{\ttindex{TupleType}}. + type; it is the same object as \code{tuple} and \code{types.TupleType} + in the Python layer.\withsubitem{(in module types)}{\ttindex{TupleType}}. \end{cvardesc} \begin{cfuncdesc}{int}{PyTuple_Check}{PyObject *p} @@ -1790,8 +1801,8 @@ format. \begin{cvardesc}{PyTypeObject}{PyList_Type} This instance of \ctype{PyTypeObject} represents the Python list - type. This is the same object as \code{types.ListType}. - \withsubitem{(in module types)}{\ttindex{ListType}} + type. This is the same object as \code{list} and \code{types.ListType} + in the Python layer.\withsubitem{(in module types)}{\ttindex{ListType}} \end{cvardesc} \begin{cfuncdesc}{int}{PyList_Check}{PyObject *p} @@ -1919,7 +1930,7 @@ format. \begin{cvardesc}{PyTypeObject}{PyDict_Type} This instance of \ctype{PyTypeObject} represents the Python dictionary type. This is exposed to Python programs as - \code{types.DictType} and \code{types.DictionaryType}. + \code{dict} and \code{types.DictType}. \withsubitem{(in module types)}{\ttindex{DictType}\ttindex{DictionaryType}} \end{cvardesc} @@ -2134,7 +2145,8 @@ implementation detail and may change in future releases of Python. \begin{cvardesc}{PyTypeObject}{PyFile_Type} This instance of \ctype{PyTypeObject} represents the Python file - type. This is exposed to Python programs as \code{types.FileType}. + type. This is exposed to Python programs as \code{file} and + \code{types.FileType}. \withsubitem{(in module types)}{\ttindex{FileType}} \end{cvardesc} @@ -2583,7 +2595,7 @@ They are found in the dictionary of type objects. \begin{cvardesc}{PyTypeObject}{PySlice_Type} The type object for slice objects. This is the same as - \code{types.SliceType}. + \code{slice} and \code{types.SliceType}. \withsubitem{(in module types)}{\ttindex{SliceType}} \end{cvardesc} diff --git a/Doc/api/exceptions.tex b/Doc/api/exceptions.tex index 6dbe818..2a9db54 100644 --- a/Doc/api/exceptions.tex +++ b/Doc/api/exceptions.tex @@ -256,10 +256,14 @@ error indicator for each thread. argument. It is mostly for internal use. \end{cfuncdesc} -\begin{cfuncdesc}{int}{PyErr_Warn}{PyObject *category, char *message} +\begin{cfuncdesc}{int}{PyErr_WarnEx}{PyObject *category, char *message, int stacklevel} Issue a warning message. The \var{category} argument is a warning category (see below) or \NULL; the \var{message} argument is a - message string. + message string. \var{stacklevel} is a positive number giving a + number of stack frames; the warning will be issued from the + currently executing line of code in that stack frame. A \var{stacklevel} + of 1 is the function calling \cfunction{PyErr_WarnEx()}, 2 is + the function above that, and so forth. This function normally prints a warning message to \var{sys.stderr}; however, it is also possible that the user has specified that @@ -291,6 +295,16 @@ error indicator for each thread. command line documentation. There is no C API for warning control. \end{cfuncdesc} +\begin{cfuncdesc}{int}{PyErr_Warn}{PyObject *category, char *message} + Issue a warning message. The \var{category} argument is a warning + category (see below) or \NULL; the \var{message} argument is a + message string. The warning will appear to be issued from the function + calling \cfunction{PyErr_Warn()}, equivalent to calling + \cfunction{PyErr_WarnEx()} with a \var{stacklevel} of 1. + + Deprecated; use \cfunction{PyErr_WarnEx()} instead. +\end{cfuncdesc} + \begin{cfuncdesc}{int}{PyErr_WarnExplicit}{PyObject *category, const char *message, const char *filename, int lineno, const char *module, PyObject *registry} diff --git a/Doc/api/intro.tex b/Doc/api/intro.tex index 96f18ec..c2c3fec 100644 --- a/Doc/api/intro.tex +++ b/Doc/api/intro.tex @@ -577,3 +577,59 @@ information about these functions is given in a later chapter. Notice that \cfunction{Py_Finalize} does \emph{not} free all memory allocated by the Python interpreter, e.g. memory allocated by extension modules currently cannot be released. + + +\section{Debugging Builds \label{debugging}} + +Python can be built with several macros to enable extra checks of the +interpreter and extension modules. These checks tend to add a large +amount of overhead to the runtime so they are not enabled by default. + +A full list of the various types of debugging builds is in the file +\file{Misc/SpecialBuilds.txt} in the Python source distribution. +Builds are available that support tracing of reference counts, +debugging the memory allocator, or low-level profiling of the main +interpreter loop. Only the most frequently-used builds will be +described in the remainder of this section. + +Compiling the interpreter with the \csimplemacro{Py_DEBUG} macro +defined produces what is generally meant by "a debug build" of Python. +\csimplemacro{Py_DEBUG} is enabled in the \UNIX{} build by adding +\longprogramopt{with-pydebug} to the \file{configure} command. It is also +implied by the presence of the not-Python-specific +\csimplemacro{_DEBUG} macro. When \csimplemacro{Py_DEBUG} is enabled +in the \UNIX{} build, compiler optimization is disabled. + +In addition to the reference count debugging described below, the +following extra checks are performed: + +\begin{itemize} + \item Extra checks are added to the object allocator. + \item Extra checks are added to the parser and compiler. + \item Downcasts from wide types to narrow types are checked for + loss of information. + \item A number of assertions are added to the dictionary and set + implementations. In addition, the set object acquires a + \method{test_c_api} method. + \item Sanity checks of the input arguments are added to frame + creation. + \item The storage for long ints is initialized with a known + invalid pattern to catch reference to uninitialized + digits. + \item Low-level tracing and extra exception checking are added + to the runtime virtual machine. + \item Extra checks are added to the memory arena implementation. + \item Extra debugging is added to the thread module. +\end{itemize} + +There may be additional checks not mentioned here. + +Defining \csimplemacro{Py_TRACE_REFS} enables reference tracing. When +defined, a circular doubly linked list of active objects is maintained +by adding two extra fields to every \ctype{PyObject}. Total +allocations are tracked as well. Upon exit, all existing references +are printed. (In interactive mode this happens after every statement +run by the interpreter.) Implied by \csimplemacro{Py_DEBUG}. + +Please refer to \file{Misc/SpecialBuilds.txt} in the Python source +distribution for more detailed information. diff --git a/Doc/api/refcounts.dat b/Doc/api/refcounts.dat index ab6d865..b8aaad5 100644 --- a/Doc/api/refcounts.dat +++ b/Doc/api/refcounts.dat @@ -303,6 +303,11 @@ PyErr_Warn:int::: PyErr_Warn:PyObject*:category:0: PyErr_Warn:char*:message:: +PyErr_WarnEx:int::: +PyErr_WarnEx:PyObject*:category:0: +PyErr_WarnEx:const char*:message:: +PyErr_WarnEx:Py_ssize_t:stack_level:: + PyEval_AcquireLock:void::: PyEval_AcquireThread:void::: diff --git a/Doc/commontex/boilerplate.tex b/Doc/commontex/boilerplate.tex index b4c9f48..9749432 100644 --- a/Doc/commontex/boilerplate.tex +++ b/Doc/commontex/boilerplate.tex @@ -5,5 +5,5 @@ Email: \email{docs@python.org} } -\date{\today} % XXX update before final release! +\date{3rd August, 2006} % XXX update before final release! \input{patchlevel} % include Python version information diff --git a/Doc/dist/dist.tex b/Doc/dist/dist.tex index e95c0d3..c1b72ad 100644 --- a/Doc/dist/dist.tex +++ b/Doc/dist/dist.tex @@ -530,7 +530,7 @@ If you need to include header files from some other Python extension, you can take advantage of the fact that header files are installed in a consistent way by the Distutils \command{install\_header} command. For example, the Numerical Python header files are installed (on a standard -Unix installation) to \file{/usr/local/include/python1.5/Numerical}. +\UNIX{} installation) to \file{/usr/local/include/python1.5/Numerical}. (The exact location will differ according to your platform and Python installation.) Since the Python include directory---\file{/usr/local/include/python1.5} in this case---is always @@ -2317,7 +2317,7 @@ constructor \lineiii{name}{the full name of the extension, including any packages --- ie. \emph{not} a filename or pathname, but Python dotted name}{string} \lineiii{sources}{list of source filenames, relative to the distribution -root (where the setup script lives), in Unix form (slash-separated) for +root (where the setup script lives), in \UNIX{} form (slash-separated) for portability. Source files may be C, \Cpp, SWIG (.i), platform-specific resource files, or whatever else is recognized by the \command{build_ext} command as source for a Python extension.}{string} @@ -2873,9 +2873,20 @@ C compiler: \modulesynopsis{Microsoft Compiler} This module provides \class{MSVCCompiler}, an implementation of the abstract -\class{CCompiler} class for Microsoft Visual Studio. It should also work using -the freely available compiler provided as part of the .Net SDK download. XXX -download link. +\class{CCompiler} class for Microsoft Visual Studio. Typically, extension +modules need to be compiled with the same compiler that was used to compile +Python. For Python 2.3 and earlier, the compiler was Visual Studio 6. For +Python 2.4 and 2.5, the compiler is Visual Studio .NET 2003. The AMD64 +and Itanium binaries are created using the Platform SDK. + +\class{MSVCCompiler} will normally choose the right compiler, linker etc. +on its own. To override this choice, the environment variables +\var{DISTUTILS\_USE\_SDK} and \var{MSSdk} must be both set. \var{MSSdk} +indicates that the current environment has been setup by the SDK's +\code{SetEnv.Cmd} script, or that the environment variables had been +registered when the SDK was installed; \var{DISTUTILS\_USE\_SDK} indicates +that the distutils user has made an explicit choice to override the +compiler selection by \class{MSVCCompiler}. \section{\module{distutils.bcppcompiler} --- Borland Compiler} \declaremodule{standard}{distutils.bcppcompiler} @@ -3088,7 +3099,7 @@ name of the output file, and \var{copied} is true if the file was copied Move file \var{src} to \var{dst}. If \var{dst} is a directory, the file will be moved into it with the same name; otherwise, \var{src} is just renamed to \var{dst}. Returns the new full name of the file. -\warning{Handles cross-device moves on Unix using \function{copy_file()}. +\warning{Handles cross-device moves on \UNIX{} using \function{copy_file()}. What about other systems???} \end{funcdesc} @@ -3131,7 +3142,7 @@ For non-\POSIX{} platforms, currently just returns \code{sys.platform}. Return 'pathname' as a name that will work on the native filesystem, i.e. split it on '/' and put it back together again using the current directory separator. Needed because filenames in the setup script are -always supplied in Unix style, and have to be converted to the local +always supplied in \UNIX{} style, and have to be converted to the local convention before we can actually use them in the filesystem. Raises \exception{ValueError} on non-\UNIX-ish systems if \var{pathname} either starts or ends with a slash. @@ -3180,7 +3191,7 @@ with \var{prefix}. \end{funcdesc} \begin{funcdesc}{split_quoted}{s} -Split a string up according to Unix shell-like rules for quotes and +Split a string up according to \UNIX{} shell-like rules for quotes and backslashes. In short: words are delimited by spaces, as long as those spaces are not escaped by a backslash, or inside a quoted string. Single and double quotes are equivalent, and the quote characters can diff --git a/Doc/doc/doc.tex b/Doc/doc/doc.tex index e4b91ac..1d0f279 100644 --- a/Doc/doc/doc.tex +++ b/Doc/doc/doc.tex @@ -187,6 +187,20 @@ text contributions are more than welcome as well. Topics which are not covered in the Apple's style guide will be discussed in this document if necessary. + Footnotes are generally discouraged due to the pain of using + footnotes in the HTML conversion of documents. Footnotes may be + used when they are the best way to present specific information. + When a footnote reference is added at the end of the sentence, it + should follow the sentence-ending punctuation. The \LaTeX{} markup + should appear something like this: + +\begin{verbatim} +This sentence has a footnote reference.% + \footnote{This is the footnote text.} +\end{verbatim} + + Footnotes may appear in the middle of sentences where appropriate. + Many special names are used in the Python documentation, including the names of operating systems, programming languages, standards bodies, and the like. Many of these were assigned \LaTeX{} macros @@ -281,10 +295,10 @@ text contributions are more than welcome as well. to know about \LaTeX{} syntax. A \dfn{comment} is started by the ``percent'' character - (\character{\%}) and continues through the end of the line and all - leading whitespace on the following line. This is a little - different from any programming language I know of, so an example - is in order: + (\character{\%}) and continues through the end of the line + \emph{and all leading whitespace on the following line}. This is + a little different from any programming language I know of, so an + example is in order: \begin{verbatim} This is text.% comment diff --git a/Doc/ext/newtypes.tex b/Doc/ext/newtypes.tex index cd2c045..a485a15 100644 --- a/Doc/ext/newtypes.tex +++ b/Doc/ext/newtypes.tex @@ -16,8 +16,9 @@ get started. The way new types are defined changed dramatically (and for the better) in Python 2.2. This document documents how to define new types for Python 2.2 and later. If you need to support older -versions of Python, you will need to refer to older versions of this -documentation. +versions of Python, you will need to refer to +\ulink{older versions of this documentation} + {http://www.python.org/doc/versions/}. \end{notice} \section{The Basics @@ -479,7 +480,7 @@ this? 1 \item when we know that deallocation of the object\footnote{This is true when we know that the object is a basic type, like a string or - a float} will not cause any + a float.} will not cause any calls back into our type's code \item when decrementing a reference count in a \member{tp_dealloc} handler when garbage-collections is not supported\footnote{We relied @@ -791,9 +792,9 @@ eventually figure out that the list is garbage and free it. In the second version of the \class{Noddy} example, we allowed any kind of object to be stored in the \member{first} or \member{last} -attributes\footnote{Even in the third version, we aren't guaranteed to +attributes.\footnote{Even in the third version, we aren't guaranteed to avoid cycles. Instances of string subclasses are allowed and string -subclasses could allow cycles even if normal strings don't.}. This +subclasses could allow cycles even if normal strings don't.} This means that \class{Noddy} objects can participate in cycles: \begin{verbatim} @@ -1563,6 +1564,85 @@ without setting an exception or it may set \exception{StopIteration}; avoiding the exception can yield slightly better performance. If an actual error occurs, it should set an exception and return \NULL. + +\subsection{Weak Reference Support\label{weakref-support}} + +One of the goals of Python's weak-reference implementation is to allow +any type to participate in the weak reference mechanism without +incurring the overhead on those objects which do not benefit by weak +referencing (such as numbers). + +For an object to be weakly referencable, the extension must include a +\ctype{PyObject*} field in the instance structure for the use of the +weak reference mechanism; it must be initialized to \NULL{} by the +object's constructor. It must also set the \member{tp_weaklistoffset} +field of the corresponding type object to the offset of the field. +For example, the instance type is defined with the following +structure: + +\begin{verbatim} +typedef struct { + PyObject_HEAD + PyClassObject *in_class; /* The class object */ + PyObject *in_dict; /* A dictionary */ + PyObject *in_weakreflist; /* List of weak references */ +} PyInstanceObject; +\end{verbatim} + +The statically-declared type object for instances is defined this way: + +\begin{verbatim} +PyTypeObject PyInstance_Type = { + PyObject_HEAD_INIT(&PyType_Type) + 0, + "module.instance", + + /* Lots of stuff omitted for brevity... */ + + Py_TPFLAGS_DEFAULT, /* tp_flags */ + 0, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + offsetof(PyInstanceObject, in_weakreflist), /* tp_weaklistoffset */ +}; +\end{verbatim} + +The type constructor is responsible for initializing the weak reference +list to \NULL: + +\begin{verbatim} +static PyObject * +instance_new() { + /* Other initialization stuff omitted for brevity */ + + self->in_weakreflist = NULL; + + return (PyObject *) self; +} +\end{verbatim} + +The only further addition is that the destructor needs to call the +weak reference manager to clear any weak references. This should be +done before any other parts of the destruction have occurred, but is +only required if the weak reference list is non-\NULL: + +\begin{verbatim} +static void +instance_dealloc(PyInstanceObject *inst) +{ + /* Allocate temporaries if needed, but do not begin + destruction just yet. + */ + + if (inst->in_weakreflist != NULL) + PyObject_ClearWeakRefs((PyObject *) inst); + + /* Proceed with object destruction normally. */ +} +\end{verbatim} + + \subsection{More Suggestions} Remember that you can omit most of these functions, in which case you diff --git a/Doc/ext/windows.tex b/Doc/ext/windows.tex index ca18a1e..f9de548 100644 --- a/Doc/ext/windows.tex +++ b/Doc/ext/windows.tex @@ -28,13 +28,15 @@ Python; typically Microsoft Visual \Cpp. \section{A Cookbook Approach \label{win-cookbook}} There are two approaches to building extension modules on Windows, -just as there are on \UNIX: use the \refmodule{distutils} package to +just as there are on \UNIX: use the +\ulink{\module{distutils}}{../lib/module-distutils.html} package to control the build process, or do things manually. The distutils approach works well for most extensions; documentation on using -\refmodule{distutils} to build and package extension modules is -available in \citetitle[../dist/dist.html]{Distributing Python -Modules}. This section describes the manual approach to building -Python extensions written in C or \Cpp. +\ulink{\module{distutils}}{../lib/module-distutils.html} to build and +package extension modules is available in +\citetitle[../dist/dist.html]{Distributing Python Modules}. This +section describes the manual approach to building Python extensions +written in C or \Cpp. To build extensions using these instructions, you need to have a copy of the Python sources of the same version as your installed Python. diff --git a/Doc/howto/Makefile b/Doc/howto/Makefile index 19701c6..18110a2 100644 --- a/Doc/howto/Makefile +++ b/Doc/howto/Makefile @@ -1,88 +1,84 @@ +# Makefile for the HOWTO directory +# LaTeX HOWTOs can be turned into HTML, PDF, PS, DVI or plain text output. +# reST HOWTOs can only be turned into HTML. -MKHOWTO=../tools/mkhowto -WEBDIR=. +# Variables to change + +# Paper size for non-HTML formats (letter or a4) +PAPER=letter + +# Arguments to rst2html.py, and location of the script RSTARGS = --input-encoding=utf-8 -VPATH=.:dvi:pdf:ps:txt +RST2HTML = rst2html.py -# List of HOWTOs that aren't to be processed +# List of HOWTOs that aren't to be processed. This should contain the +# base name of the HOWTO without any extension (e.g. 'advocacy', +# 'unicode'). +REMOVE_HOWTOS = -REMOVE_HOWTO = +MKHOWTO=../tools/mkhowto +WEBDIR=. +PAPERDIR=../paper-$(PAPER) +HTMLDIR=../html # Determine list of files to be built +TEX_SOURCES = $(wildcard *.tex) +RST_SOURCES = $(wildcard *.rst) +TEX_NAMES = $(filter-out $(REMOVE_HOWTOS),$(patsubst %.tex,%,$(TEX_SOURCES))) + +PAPER_PATHS=$(addprefix $(PAPERDIR)/,$(TEX_NAMES)) +DVI =$(addsuffix .dvi,$(PAPER_PATHS)) +PDF =$(addsuffix .pdf,$(PAPER_PATHS)) +PS =$(addsuffix .ps,$(PAPER_PATHS)) -HOWTO=$(filter-out $(REMOVE_HOWTO),$(wildcard *.tex)) -RST_SOURCES = $(shell echo *.rst) -DVI =$(patsubst %.tex,%.dvi,$(HOWTO)) -PDF =$(patsubst %.tex,%.pdf,$(HOWTO)) -PS =$(patsubst %.tex,%.ps,$(HOWTO)) -TXT =$(patsubst %.tex,%.txt,$(HOWTO)) -HTML =$(patsubst %.tex,%,$(HOWTO)) +ALL_HOWTO_NAMES = $(TEX_NAMES) $(patsubst %.rst,%,$(RST_SOURCES)) +HOWTO_NAMES = $(filter-out $(REMOVE_HOWTOS),$(ALL_HOWTO_NAMES)) +HTML = $(addprefix $(HTMLDIR)/,$(HOWTO_NAMES)) # Rules for building various formats -%.dvi : %.tex + +# reST to HTML +$(HTMLDIR)/%: %.rst + if [ ! -d $@ ] ; then mkdir $@ ; fi + $(RST2HTML) $(RSTARGS) $< >$@/index.html + +# LaTeX to various output formats +$(PAPERDIR)/%.dvi : %.tex $(MKHOWTO) --dvi $< - mv $@ dvi + mv $*.dvi $@ -%.pdf : %.tex +$(PAPERDIR)/%.pdf : %.tex $(MKHOWTO) --pdf $< - mv $@ pdf + mv $*.pdf $@ -%.ps : %.tex +$(PAPERDIR)/%.ps : %.tex $(MKHOWTO) --ps $< - mv $@ ps + mv $*.ps $@ + +$(HTMLDIR)/% : %.tex + $(MKHOWTO) --html --iconserver="." --dir $@ $< -%.txt : %.tex +# Rule that isn't actually used -- we no longer support the 'txt' target. +$(PAPERDIR)/%.txt : %.tex $(MKHOWTO) --text $< mv $@ txt -% : %.tex - $(MKHOWTO) --html --iconserver="." $< - tar -zcvf html/$*.tgz $* - #zip -r html/$*.zip $* - default: @echo "'all' -- build all files" - @echo "'dvi', 'pdf', 'ps', 'txt', 'html' -- build one format" - -all: $(HTML) - -.PHONY : dvi pdf ps txt html rst -dvi: $(DVI) - -pdf: $(PDF) -ps: $(PS) -txt: $(TXT) -html:$(HTML) - -# Rule to build collected tar files -dist: #all - for i in dvi pdf ps txt ; do \ - cd $$i ; \ - tar -zcf All.tgz *.$$i ;\ - cd .. ;\ - done + @echo "'dvi', 'pdf', 'ps', 'html' -- build one format" -# Rule to copy files to the Web tree on AMK's machine -web: dist - cp dvi/* $(WEBDIR)/dvi - cp ps/* $(WEBDIR)/ps - cp pdf/* $(WEBDIR)/pdf - cp txt/* $(WEBDIR)/txt - for dir in $(HTML) ; do cp -rp $$dir $(WEBDIR) ; done - for ltx in $(HOWTO) ; do cp -p $$ltx $(WEBDIR)/latex ; done +all: dvi pdf ps html -rst: unicode.html - -%.html: %.rst - rst2html $(RSTARGS) $< >$@ +.PHONY : dvi pdf ps html +dvi: $(DVI) +pdf: $(PDF) +ps: $(PS) +html: $(HTML) clean: - rm -f *~ *.log *.ind *.l2h *.aux *.toc *.how - rm -f *.dvi *.ps *.pdf *.bkm - rm -f unicode.html + rm -f *~ *.log *.ind *.l2h *.aux *.toc *.how *.bkm + rm -f *.dvi *.pdf *.ps clobber: - rm dvi/* ps/* pdf/* txt/* html/* - - - + rm -rf $(HTML) + rm -rf $(DVI) $(PDF) $(PS) diff --git a/Doc/howto/doanddont.tex b/Doc/howto/doanddont.tex index adbde66..a105ca1 100644 --- a/Doc/howto/doanddont.tex +++ b/Doc/howto/doanddont.tex @@ -288,8 +288,9 @@ More useful functions in \module{os.path}: \function{basename}, There are also many useful builtin functions people seem not to be aware of for some reason: \function{min()} and \function{max()} can find the minimum/maximum of any sequence with comparable semantics, -for example, yet many people write they own max/min. Another highly -useful function is \function{reduce()}. Classical use of \function{reduce()} +for example, yet many people write their own +\function{max()}/\function{min()}. Another highly useful function is +\function{reduce()}. A classical use of \function{reduce()} is something like \begin{verbatim} diff --git a/Doc/howto/sockets.tex b/Doc/howto/sockets.tex index 4da92a8..0cecbb9 100644 --- a/Doc/howto/sockets.tex +++ b/Doc/howto/sockets.tex @@ -213,34 +213,39 @@ Assuming you don't want to end the connection, the simplest solution is a fixed length message: \begin{verbatim} - class mysocket: - '''demonstration class only - - coded for clarity, not efficiency''' - def __init__(self, sock=None): - if sock is None: - self.sock = socket.socket( - socket.AF_INET, socket.SOCK_STREAM) - else: - self.sock = sock - def connect(host, port): - self.sock.connect((host, port)) - def mysend(msg): - totalsent = 0 - while totalsent < MSGLEN: - sent = self.sock.send(msg[totalsent:]) - if sent == 0: - raise RuntimeError, \\ - "socket connection broken" - totalsent = totalsent + sent - def myreceive(): - msg = '' - while len(msg) < MSGLEN: - chunk = self.sock.recv(MSGLEN-len(msg)) - if chunk == '': - raise RuntimeError, \\ - "socket connection broken" - msg = msg + chunk - return msg +class mysocket: + '''demonstration class only + - coded for clarity, not efficiency + ''' + + def __init__(self, sock=None): + if sock is None: + self.sock = socket.socket( + socket.AF_INET, socket.SOCK_STREAM) + else: + self.sock = sock + + def connect(self, host, port): + self.sock.connect((host, port)) + + def mysend(self, msg): + totalsent = 0 + while totalsent < MSGLEN: + sent = self.sock.send(msg[totalsent:]) + if sent == 0: + raise RuntimeError, \\ + "socket connection broken" + totalsent = totalsent + sent + + def myreceive(self): + msg = '' + while len(msg) < MSGLEN: + chunk = self.sock.recv(MSGLEN-len(msg)) + if chunk == '': + raise RuntimeError, \\ + "socket connection broken" + msg = msg + chunk + return msg \end{verbatim} The sending code here is usable for almost any messaging scheme - in diff --git a/Doc/inst/inst.tex b/Doc/inst/inst.tex index 676f8ae..df7c656 100644 --- a/Doc/inst/inst.tex +++ b/Doc/inst/inst.tex @@ -262,7 +262,7 @@ If you don't choose an installation directory---i.e., if you just run \code{setup.py install}---then the \command{install} command installs to the standard location for third-party Python modules. This location varies by platform and by how you built/installed Python itself. On -\UNIX{} (and Mac OS X, which is also Unix-based), +\UNIX{} (and Mac OS X, which is also \UNIX-based), it also depends on whether the module distribution being installed is pure Python or contains extensions (``non-pure''): \begin{tableiv}{l|l|l|c}{textrm}% diff --git a/Doc/lib/email.tex b/Doc/lib/email.tex index 6853325..ea12705 100644 --- a/Doc/lib/email.tex +++ b/Doc/lib/email.tex @@ -105,7 +105,7 @@ of the package. \lineiii{4.0}{Python 2.5}{Python 2.3 to 2.5} \end{tableiii} -Here are the major differences between \module{email} verson 4 and version 3: +Here are the major differences between \module{email} version 4 and version 3: \begin{itemize} \item All modules have been renamed according to \pep{8} standards. For @@ -126,6 +126,15 @@ Here are the major differences between \module{email} verson 4 and version 3: \item Methods that were deprecated in version 3 have been removed. These include \method{Generator.__call__()}, \method{Message.get_type()}, \method{Message.get_main_type()}, \method{Message.get_subtype()}. + +\item Fixes have been added for \rfc{2231} support which can change some of + the return types for \function{Message.get_param()} and friends. Under + some circumstances, values which used to return a 3-tuple now return + simple strings (specifically, if all extended parameter segments were + unencoded, there is no language and charset designation expected, so the + return type is now a simple string). Also, \%-decoding used to be done + for both encoded and unencoded segments; this decoding is now done only + for encoded segments. \end{itemize} Here are the major differences between \module{email} version 3 and version 2: diff --git a/Doc/lib/emailgenerator.tex b/Doc/lib/emailgenerator.tex index 3415442..b236673 100644 --- a/Doc/lib/emailgenerator.tex +++ b/Doc/lib/emailgenerator.tex @@ -31,11 +31,11 @@ Optional \var{mangle_from_} is a flag that, when \code{True}, puts a \samp{>} character in front of any line in the body that starts exactly as \samp{From }, i.e. \code{From} followed by a space at the beginning of the line. This is the only guaranteed portable way to avoid having such -lines be mistaken for a Unix mailbox format envelope header separator (see +lines be mistaken for a \UNIX{} mailbox format envelope header separator (see \ulink{WHY THE CONTENT-LENGTH FORMAT IS BAD} {http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/content-length.html} for details). \var{mangle_from_} defaults to \code{True}, but you -might want to set this to \code{False} if you are not writing Unix +might want to set this to \code{False} if you are not writing \UNIX{} mailbox format files. Optional \var{maxheaderlen} specifies the longest length for a diff --git a/Doc/lib/lib.tex b/Doc/lib/lib.tex index 0691179..837c759 100644 --- a/Doc/lib/lib.tex +++ b/Doc/lib/lib.tex @@ -71,12 +71,12 @@ and how to embed it in other applications. % BUILT-INs % ============= -\input{libobjs} % Built-in Types, Exceptions and Functions +\input{libobjs} % Built-in Exceptions and Functions \input{libfuncs} -\input{libstdtypes} \input{libexcs} \input{libconsts} +\input{libstdtypes} % Built-in types % ============= @@ -154,8 +154,8 @@ and how to embed it in other applications. % encoding stuff \input{libbase64} -\input{libbinascii} \input{libbinhex} +\input{libbinascii} \input{libquopri} \input{libuu} @@ -171,6 +171,7 @@ and how to embed it in other applications. \input{xmlsaxhandler} \input{xmlsaxutils} \input{xmlsaxreader} +\input{libetree} % \input{libxmllib} \input{fileformats} % Miscellaneous file formats @@ -245,7 +246,6 @@ and how to embed it in other applications. \input{libplatform} \input{liberrno} \input{libctypes} -\input{libctypesref} \input{libsomeos} % Optional Operating System Services \input{libselect} @@ -292,6 +292,7 @@ and how to embed it in other applications. \input{libwebbrowser} \input{libcgi} \input{libcgitb} +\input{libwsgiref} \input{liburllib} \input{liburllib2} \input{libhttplib} @@ -303,6 +304,7 @@ and how to embed it in other applications. \input{libsmtplib} \input{libsmtpd} \input{libtelnetlib} +\input{libuuid} \input{liburlparse} \input{libsocksvr} \input{libbasehttp} diff --git a/Doc/lib/libanydbm.tex b/Doc/lib/libanydbm.tex index 17228dd..badc6ec 100644 --- a/Doc/lib/libanydbm.tex +++ b/Doc/lib/libanydbm.tex @@ -46,6 +46,32 @@ be stored, retrieved, and deleted, and the \method{has_key()} and \method{keys()} methods are available. Keys and values must always be strings. +The following example records some hostnames and a corresponding title, +and then prints out the contents of the database: + +\begin{verbatim} +import anydbm + +# Open database, creating it if necessary. +db = anydbm.open('cache', 'c') + +# Record some values +db['www.python.org'] = 'Python Website' +db['www.cnn.com'] = 'Cable News Network' + +# Loop through contents. Other dictionary methods +# such as .keys(), .values() also work. +for k, v in db.iteritems(): + print k, '\t', v + +# Storing a non-string key or value will raise an exception (most +# likely a TypeError). +db['www.yahoo.com'] = 4 + +# Close when done. +db.close() +\end{verbatim} + \begin{seealso} \seemodule{dbhash}{BSD \code{db} database interface.} diff --git a/Doc/lib/libbase64.tex b/Doc/lib/libbase64.tex index 747d837..0039c84 100644 --- a/Doc/lib/libbase64.tex +++ b/Doc/lib/libbase64.tex @@ -146,6 +146,18 @@ string containing one or more lines of base64-encoded data always including an extra trailing newline (\code{'\e n'}). \end{funcdesc} +An example usage of the module: + +\begin{verbatim} +>>> import base64 +>>> encoded = base64.b64encode('data to be encoded') +>>> encoded +'ZGF0YSB0byBiZSBlbmNvZGVk' +>>> data = base64.b64decode(encoded) +>>> data +'data to be encoded' +\end{verbatim} + \begin{seealso} \seemodule{binascii}{Support module containing \ASCII-to-binary and binary-to-\ASCII{} conversions.} diff --git a/Doc/lib/libbinascii.tex b/Doc/lib/libbinascii.tex index b244b10..84d29c6 100644 --- a/Doc/lib/libbinascii.tex +++ b/Doc/lib/libbinascii.tex @@ -9,10 +9,11 @@ The \module{binascii} module contains a number of methods to convert between binary and various \ASCII-encoded binary representations. Normally, you will not use these functions directly -but use wrapper modules like \refmodule{uu}\refstmodindex{uu} or -\refmodule{binhex}\refstmodindex{binhex} instead, this module solely -exists because bit-manipulation of large amounts of data is slow in -Python. +but use wrapper modules like \refmodule{uu}\refstmodindex{uu}, +\refmodule{base64}\refstmodindex{base64}, or +\refmodule{binhex}\refstmodindex{binhex} instead. The \module{binascii} module +contains low-level functions written in C for greater speed +that are used by the higher-level modules. The \module{binascii} module defines the following functions: diff --git a/Doc/lib/libbsddb.tex b/Doc/lib/libbsddb.tex index a5cda6d..44b9168 100644 --- a/Doc/lib/libbsddb.tex +++ b/Doc/lib/libbsddb.tex @@ -13,23 +13,29 @@ using the appropriate open call. Bsddb objects behave generally like dictionaries. Keys and values must be strings, however, so to use other objects as keys or to store other kinds of objects the user must serialize them somehow, typically using \function{marshal.dumps()} or -\function{pickle.dumps}. +\function{pickle.dumps()}. The \module{bsddb} module requires a Berkeley DB library version from 3.3 thru 4.4. \begin{seealso} - \seeurl{http://pybsddb.sourceforge.net/}{Website with documentation - for the new python Berkeley DB interface that closely mirrors the - sleepycat object oriented interface provided in Berkeley DB 3 and 4.} + \seeurl{http://pybsddb.sourceforge.net/}{The website with documentation + for the \module{bsddb.db} python Berkeley DB interface that closely mirrors + the Sleepycat object oriented interface provided in Berkeley DB 3 and 4.} \seeurl{http://www.sleepycat.com/}{Sleepycat Software produces the - modern Berkeley DB library.} + Berkeley DB library.} \end{seealso} +A more modern DB, DBEnv and DBSequence object interface is available in the +\module{bsddb.db} module which closely matches the Sleepycat Berkeley DB C API +documented at the above URLs. Additional features provided by the +\module{bsddb.db} API include fine tuning, transactions, logging, and +multiprocess concurrent database access. + The following is a description of the legacy \module{bsddb} interface -compatible with the old python bsddb module. For details about the more -modern Db and DbEnv object oriented interface see the above mentioned -pybsddb URL. +compatible with the old python bsddb module. Starting in Python 2.5 this +interface should be safe for multithreaded access. The \module{bsddb.db} +API is recommended for threading users as it provides better control. The \module{bsddb} module defines the following functions that create objects that access the appropriate type of Berkeley DB file. The @@ -88,7 +94,7 @@ interpretation. \begin{notice} -Beginning in 2.3 some Unix versions of Python may have a \module{bsddb185} +Beginning in 2.3 some \UNIX{} versions of Python may have a \module{bsddb185} module. This is present \emph{only} to allow backwards compatibility with systems which ship with the old Berkeley DB 1.85 database library. The \module{bsddb185} module should never be used directly in new code. diff --git a/Doc/lib/libcompileall.tex b/Doc/lib/libcompileall.tex index d39a548..3e9667d 100644 --- a/Doc/lib/libcompileall.tex +++ b/Doc/lib/libcompileall.tex @@ -44,6 +44,19 @@ compile Python sources in directories named on the command line or in \function{compile_dir()} function. \end{funcdesc} +To force a recompile of all the \file{.py} files in the \file{Lib/} +subdirectory and all its subdirectories: + +\begin{verbatim} +import compileall + +compileall.compile_dir('Lib/', force=True) + +# Perform same compilation, excluding files in .svn directories. +import re +compileall.compile_dir('Lib/', rx=re.compile('/[.]svn'), force=True) +\end{verbatim} + \begin{seealso} \seemodule[pycompile]{py_compile}{Byte-compile a single source file.} diff --git a/Doc/lib/libcookielib.tex b/Doc/lib/libcookielib.tex index ef2d833..01f2539 100644 --- a/Doc/lib/libcookielib.tex +++ b/Doc/lib/libcookielib.tex @@ -24,7 +24,7 @@ Internet are Netscape cookies. \module{cookielib} attempts to follow the de-facto Netscape cookie protocol (which differs substantially from that set out in the original Netscape specification), including taking note of the \code{max-age} and \code{port} cookie-attributes -introduced with RFC 2109. \note{The various named parameters found in +introduced with RFC 2965. \note{The various named parameters found in \mailheader{Set-Cookie} and \mailheader{Set-Cookie2} headers (eg. \code{domain} and \code{expires}) are conventionally referred to as \dfn{attributes}. To distinguish them from Python attributes, the diff --git a/Doc/lib/libcsv.tex b/Doc/lib/libcsv.tex index 65053c7..8e10ccf 100644 --- a/Doc/lib/libcsv.tex +++ b/Doc/lib/libcsv.tex @@ -55,7 +55,7 @@ The \module{csv} module defines the following functions: Return a reader object which will iterate over lines in the given {}\var{csvfile}. \var{csvfile} can be any object which supports the iterator protocol and returns a string each time its \method{next} -method is called - file objects and list objects are both suitable. +method is called --- file objects and list objects are both suitable. If \var{csvfile} is a file object, it must be opened with the 'b' flag on platforms where that makes a difference. An optional {}\var{dialect} parameter can be given @@ -70,6 +70,18 @@ Parameters'' for details of these parameters. All data read are returned as strings. No automatic data type conversion is performed. + +\versionchanged[ +The parser is now stricter with respect to multi-line quoted +fields. Previously, if a line ended within a quoted field without a +terminating newline character, a newline would be inserted into the +returned field. This behavior caused problems when reading files +which contained carriage return characters within fields. The +behavior was changed to return the field without inserting newlines. As +a consequence, if newlines embedded within fields are important, the +input should be split into lines in a manner which preserves the newline +characters]{2.5} + \end{funcdesc} \begin{funcdesc}{writer}{csvfile\optional{, @@ -404,7 +416,7 @@ csv.register_dialect('unixpwd', delimiter=':', quoting=csv.QUOTE_NONE) reader = csv.reader(open("passwd", "rb"), 'unixpwd') \end{verbatim} -A slightly more advanced use of the reader - catching and reporting errors: +A slightly more advanced use of the reader --- catching and reporting errors: \begin{verbatim} import csv, sys diff --git a/Doc/lib/libctypes.tex b/Doc/lib/libctypes.tex index dc37749..6206b8c 100755 --- a/Doc/lib/libctypes.tex +++ b/Doc/lib/libctypes.tex @@ -1,4 +1,4 @@ -\newlength{\locallinewidth} +\ifx\locallinewidth\undefined\newlength{\locallinewidth}\fi \setlength{\locallinewidth}{\linewidth} \section{\module{ctypes} --- A foreign function library for Python.} \declaremodule{standard}{ctypes} @@ -6,13 +6,13 @@ \modulesynopsis{A foreign function library for Python.} \versionadded{2.5} -\code{ctypes} is a foreign function library for Python. +\code{ctypes} is a foreign function library for Python. It provides C +compatible data types, and allows to call functions in dlls/shared +libraries. It can be used to wrap these libraries in pure Python. \subsection{ctypes tutorial\label{ctypes-ctypes-tutorial}} -This tutorial describes version 0.9.9 of \code{ctypes}. - Note: The code samples in this tutorial uses \code{doctest} to make sure that they actually work. Since some code samples behave differently under Linux, Windows, or Mac OS X, they contain doctest directives in @@ -66,8 +66,7 @@ calling the constructor: >>> \end{verbatim} - -XXX Add section for Mac OS X. +% XXX Add section for Mac OS X. \subsubsection{Accessing functions from loaded dlls\label{ctypes-accessing-functions-from-loaded-dlls}} @@ -119,7 +118,7 @@ identifiers, like \code{"??2@YAPAXI@Z"}. In this case you have to use On Windows, some dlls export functions not by name but by ordinal. These functions can be accessed by indexing the dll object with the -odinal number: +ordinal number: \begin{verbatim} >>> cdll.kernel32[1] # doctest: +WINDOWS <_FuncPtr object at 0x...> @@ -143,16 +142,18 @@ which returns a win32 module handle. This example calls both functions with a NULL pointer (\code{None} should be used as the NULL pointer): \begin{verbatim} ->>> print libc.time(None) -114... +>>> print libc.time(None) # doctest: +SKIP +1150640792 >>> print hex(windll.kernel32.GetModuleHandleA(None)) # doctest: +WINDOWS 0x1d000000 >>> \end{verbatim} \code{ctypes} tries to protect you from calling functions with the wrong -number of arguments. Unfortunately this only works on Windows. It -does this by examining the stack after the function returns: +number of arguments or the wrong calling convention. Unfortunately +this only works on Windows. It does this by examining the stack after +the function returns, so although an error is raised the function +\emph{has} been called: \begin{verbatim} >>> windll.kernel32.GetModuleHandleA() # doctest: +WINDOWS Traceback (most recent call last): @@ -165,6 +166,25 @@ ValueError: Procedure probably called with too many arguments (4 bytes in excess >>> \end{verbatim} +The same exception is raised when you call an \code{stdcall} function +with the \code{cdecl} calling convention, or vice versa: +\begin{verbatim} +>>> cdll.kernel32.GetModuleHandleA(None) # doctest: +WINDOWS +Traceback (most recent call last): + File "", line 1, in ? +ValueError: Procedure probably called with not enough arguments (4 bytes missing) +>>> + +>>> windll.msvcrt.printf("spam") # doctest: +WINDOWS +Traceback (most recent call last): + File "", line 1, in ? +ValueError: Procedure probably called with too many arguments (4 bytes in excess) +>>> +\end{verbatim} + +To find out the correct calling convention you have to look into the C +header file or the documentation for the function you want to call. + On Windows, \code{ctypes} uses win32 structured exception handling to prevent crashes from general protection faults when functions are called with invalid argument values: @@ -186,158 +206,172 @@ Before we move on calling functions with other parameter types, we have to learn more about \code{ctypes} data types. -\subsubsection{Simple data types\label{ctypes-simple-data-types}} +\subsubsection{Fundamental data types\label{ctypes-fundamental-data-types}} \code{ctypes} defines a number of primitive C compatible data types : \begin{quote} - -\begin{longtable}[c]{|p{0.19\locallinewidth}|p{0.28\locallinewidth}|p{0.14\locallinewidth}|} -\hline -\textbf{ +\begin{tableiii}{l|l|l}{textrm} +{ ctypes type -} & \textbf{ +} +{ C type -} & \textbf{ +} +{ Python type -} \\ -\hline -\endhead - +} +\lineiii{ \class{c{\_}char} - & +} +{ \code{char} - & +} +{ character - \\ -\hline - +} +\lineiii{ \class{c{\_}byte} - & +} +{ \code{char} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}ubyte} - & +} +{ \code{unsigned char} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}short} - & +} +{ \code{short} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}ushort} - & +} +{ \code{unsigned short} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}int} - & +} +{ \code{int} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}uint} - & +} +{ \code{unsigned int} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}long} - & +} +{ \code{long} - & +} +{ integer - \\ -\hline - +} +\lineiii{ \class{c{\_}ulong} - & +} +{ \code{unsigned long} - & +} +{ long - \\ -\hline - +} +\lineiii{ \class{c{\_}longlong} - & +} +{ \code{{\_}{\_}int64} or \code{long long} - & +} +{ long - \\ -\hline - +} +\lineiii{ \class{c{\_}ulonglong} - & +} +{ \code{unsigned {\_}{\_}int64} or \code{unsigned long long} - & +} +{ long - \\ -\hline - +} +\lineiii{ \class{c{\_}float} - & +} +{ \code{float} - & +} +{ float - \\ -\hline - +} +\lineiii{ \class{c{\_}double} - & +} +{ \code{double} - & +} +{ float - \\ -\hline - +} +\lineiii{ \class{c{\_}char{\_}p} - & +} +{ \code{char *} (NUL terminated) - & +} +{ string or \code{None} - \\ -\hline - +} +\lineiii{ \class{c{\_}wchar{\_}p} - & +} +{ \code{wchar{\_}t *} (NUL terminated) - & +} +{ unicode or \code{None} - \\ -\hline - +} +\lineiii{ \class{c{\_}void{\_}p} - & +} +{ \code{void *} - & +} +{ integer or \code{None} - \\ -\hline -\end{longtable} +} +\end{tableiii} \end{quote} All these types can be created by calling them with an optional @@ -380,6 +414,7 @@ c_char_p('Hello, World') c_char_p('Hi, there') >>> print s # first string is unchanged Hello, World +>>> \end{verbatim} You should be careful, however, not to pass them to functions @@ -557,13 +592,12 @@ None >>> \end{verbatim} -XXX Mention the \member{errcheck} protocol... - You can also use a callable Python object (a function or a class for -example) as the \member{restype} attribute. It will be called with the -\code{integer} the C function returns, and the result of this call will -be used as the result of your function call. This is useful to check -for error return values and automatically raise an exception: +example) as the \member{restype} attribute, if the foreign function returns +an integer. The callable will be called with the \code{integer} the C +function returns, and the result of this call will be used as the +result of your function call. This is useful to check for error return +values and automatically raise an exception: \begin{verbatim} >>> GetModuleHandle = windll.kernel32.GetModuleHandleA # doctest: +WINDOWS >>> def ValidHandle(value): @@ -575,7 +609,7 @@ for error return values and automatically raise an exception: >>> GetModuleHandle.restype = ValidHandle # doctest: +WINDOWS >>> GetModuleHandle(None) # doctest: +WINDOWS 486539264 ->>> GetModuleHandle("something silly") # doctest: +WINDOWS +IGNORE_EXCEPTION_DETAIL +>>> GetModuleHandle("something silly") # doctest: +WINDOWS Traceback (most recent call last): File "", line 1, in ? File "", line 3, in ValidHandle @@ -588,6 +622,10 @@ api to get the string representation of an error code, and \emph{returns} an exception. \code{WinError} takes an optional error code parameter, if no one is used, it calls \function{GetLastError()} to retrieve it. +Please note that a much more powerful error checking mechanism is +available through the \member{errcheck} attribute; see the reference manual +for details. + \subsubsection{Passing pointers (or: passing parameters by reference)\label{ctypes-passing-pointers}} @@ -744,6 +782,7 @@ containing 4 POINTs among other stuff: >>> >>> print len(MyStruct().point_array) 4 +>>> \end{verbatim} Instances are created in the usual way, by calling the class: @@ -781,21 +820,31 @@ Pointer instances are created by calling the \code{pointer} function on a >>> \end{verbatim} -XXX XXX Not correct: use indexing, not the contents atribute - Pointer instances have a \code{contents} attribute which returns the -ctypes' type pointed to, the \code{c{\_}int(42)} in the above case: +object to which the pointer points, the \code{i} object above: \begin{verbatim} >>> pi.contents c_long(42) >>> \end{verbatim} +Note that \code{ctypes} does not have OOR (original object return), it +constructs a new, equivalent object each time you retrieve an +attribute: +\begin{verbatim} +>>> pi.contents is i +False +>>> pi.contents is pi.contents +False +>>> +\end{verbatim} + Assigning another \class{c{\_}int} instance to the pointer's contents attribute would cause the pointer to point to the memory location where this is stored: \begin{verbatim} ->>> pi.contents = c_int(99) +>>> i = c_int(99) +>>> pi.contents = i >>> pi.contents c_long(99) >>> @@ -808,26 +857,21 @@ Pointer instances can also be indexed with integers: >>> \end{verbatim} -XXX What is this??? Assigning to an integer index changes the pointed to value: \begin{verbatim} ->>> i2 = pi[0] ->>> i2 -99 +>>> print i +c_long(99) >>> pi[0] = 22 ->>> i2 -99 +>>> print i +c_long(22) >>> \end{verbatim} It is also possible to use indexes different from 0, but you must know -what you're doing when you use this: You access or change arbitrary -memory locations when you do this. Generally you only use this feature -if you receive a pointer from a C function, and you \emph{know} that the -pointer actually points to an array instead of a single item. - - -\subsubsection{Pointer classes/types\label{ctypes-pointer-classestypes}} +what you're doing, just as in C: You can access or change arbitrary +memory locations. Generally you only use this feature if you receive a +pointer from a C function, and you \emph{know} that the pointer actually +points to an array instead of a single item. Behind the scenes, the \code{pointer} function does more than simply create pointer instances, it has to create pointer \emph{types} first. @@ -837,7 +881,7 @@ This is done with the \code{POINTER} function, which accepts any >>> PI = POINTER(c_int) >>> PI ->>> PI(42) # doctest: +IGNORE_EXCEPTION_DETAIL +>>> PI(42) Traceback (most recent call last): File "", line 1, in ? TypeError: expected c_long instead of int @@ -846,6 +890,103 @@ TypeError: expected c_long instead of int >>> \end{verbatim} +Calling the pointer type without an argument creates a \code{NULL} +pointer. \code{NULL} pointers have a \code{False} boolean value: +\begin{verbatim} +>>> null_ptr = POINTER(c_int)() +>>> print bool(null_ptr) +False +>>> +\end{verbatim} + +\code{ctypes} checks for \code{NULL} when dereferencing pointers (but +dereferencing non-\code{NULL} pointers would crash Python): +\begin{verbatim} +>>> null_ptr[0] +Traceback (most recent call last): + .... +ValueError: NULL pointer access +>>> + +>>> null_ptr[0] = 1234 +Traceback (most recent call last): + .... +ValueError: NULL pointer access +>>> +\end{verbatim} + + +\subsubsection{Type conversions\label{ctypes-type-conversions}} + +Usually, ctypes does strict type checking. This means, if you have +\code{POINTER(c{\_}int)} in the \member{argtypes} list of a function or as the +type of a member field in a structure definition, only instances of +exactly the same type are accepted. There are some exceptions to this +rule, where ctypes accepts other objects. For example, you can pass +compatible array instances instead of pointer types. So, for +\code{POINTER(c{\_}int)}, ctypes accepts an array of c{\_}int: +\begin{verbatim} +>>> class Bar(Structure): +... _fields_ = [("count", c_int), ("values", POINTER(c_int))] +... +>>> bar = Bar() +>>> bar.values = (c_int * 3)(1, 2, 3) +>>> bar.count = 3 +>>> for i in range(bar.count): +... print bar.values[i] +... +1 +2 +3 +>>> +\end{verbatim} + +To set a POINTER type field to \code{NULL}, you can assign \code{None}: +\begin{verbatim} +>>> bar.values = None +>>> +\end{verbatim} + +XXX list other conversions... + +Sometimes you have instances of incompatible types. In \code{C}, you can +cast one type into another type. \code{ctypes} provides a \code{cast} +function which can be used in the same way. The \code{Bar} structure +defined above accepts \code{POINTER(c{\_}int)} pointers or \class{c{\_}int} arrays +for its \code{values} field, but not instances of other types: +\begin{verbatim} +>>> bar.values = (c_byte * 4)() +Traceback (most recent call last): + File "", line 1, in ? +TypeError: incompatible types, c_byte_Array_4 instance instead of LP_c_long instance +>>> +\end{verbatim} + +For these cases, the \code{cast} function is handy. + +The \code{cast} function can be used to cast a ctypes instance into a +pointer to a different ctypes data type. \code{cast} takes two +parameters, a ctypes object that is or can be converted to a pointer +of some kind, and a ctypes pointer type. It returns an instance of +the second argument, which references the same memory block as the +first argument: +\begin{verbatim} +>>> a = (c_byte * 4)() +>>> cast(a, POINTER(c_int)) + +>>> +\end{verbatim} + +So, \code{cast} can be used to assign to the \code{values} field of \code{Bar} +the structure: +\begin{verbatim} +>>> bar = Bar() +>>> bar.values = cast((c_byte * 4)(), POINTER(c_int)) +>>> print bar.values[0] +0 +>>> +\end{verbatim} + \subsubsection{Incomplete Types\label{ctypes-incomplete-types}} @@ -1059,7 +1200,10 @@ py_cmp_func 5 7 >>> \end{verbatim} -So, our array sorted now: +It is quite interesting to see that the Windows \function{qsort} function +needs more comparisons than the linux version! + +As we can easily check, our array sorted now: \begin{verbatim} >>> for i in ia: print i, ... @@ -1070,14 +1214,14 @@ So, our array sorted now: \textbf{Important note for callback functions:} Make sure you keep references to CFUNCTYPE objects as long as they are -used from C code. ctypes doesn't, and if you don't, they may be +used from C code. \code{ctypes} doesn't, and if you don't, they may be garbage collected, crashing your program when a callback is made. \subsubsection{Accessing values exported from dlls\label{ctypes-accessing-values-exported-from-dlls}} Sometimes, a dll not only exports functions, it also exports -values. An example in the Python library itself is the +variables. An example in the Python library itself is the \code{Py{\_}OptimizeFlag}, an integer set to 0, 1, or 2, depending on the \programopt{-O} or \programopt{-OO} flag given on startup. @@ -1148,9 +1292,6 @@ The fact that standard Python has a frozen module and a frozen package (indicated by the negative size member) is not wellknown, it is only used for testing. Try it out with \code{import {\_}{\_}hello{\_}{\_}} for example. -XXX Describe how to access the \var{code} member fields, which contain -the byte code for the modules. - \subsubsection{Surprises\label{ctypes-surprises}} @@ -1175,6 +1316,7 @@ Consider the following example: >>> rc.a, rc.b = rc.b, rc.a >>> print rc.a.x, rc.a.y, rc.b.x, rc.b.y 3 4 3 4 +>>> \end{verbatim} Hm. We certainly expected the last statement to print \code{3 4 1 2}. @@ -1184,6 +1326,7 @@ line above: >>> temp0, temp1 = rc.b, rc.a >>> rc.a = temp0 >>> rc.b = temp1 +>>> \end{verbatim} Note that \code{temp0} and \code{temp1} are objects still using the internal @@ -1214,6 +1357,51 @@ the object itself, instead the \code{contents} of the object is stored. Accessing the contents again constructs a new Python each time! +\subsubsection{Variable-sized data types\label{ctypes-variable-sized-data-types}} + +\code{ctypes} provides some support for variable-sized arrays and +structures (this was added in version 0.9.9.7). + +The \code{resize} function can be used to resize the memory buffer of an +existing ctypes object. The function takes the object as first +argument, and the requested size in bytes as the second argument. The +memory block cannot be made smaller than the natural memory block +specified by the objects type, a \code{ValueError} is raised if this is +tried: +\begin{verbatim} +>>> short_array = (c_short * 4)() +>>> print sizeof(short_array) +8 +>>> resize(short_array, 4) +Traceback (most recent call last): + ... +ValueError: minimum size is 8 +>>> resize(short_array, 32) +>>> sizeof(short_array) +32 +>>> sizeof(type(short_array)) +8 +>>> +\end{verbatim} + +This is nice and fine, but how would one access the additional +elements contained in this array? Since the type still only knows +about 4 elements, we get errors accessing other elements: +\begin{verbatim} +>>> short_array[:] +[0, 0, 0, 0] +>>> short_array[7] +Traceback (most recent call last): + ... +IndexError: invalid index +>>> +\end{verbatim} + +Another way to use variable-sized data types with \code{ctypes} is to use +the dynamic nature of Python, and (re-)define the data type after the +required size is already known, on a case by case basis. + + \subsubsection{Bugs, ToDo and non-implemented things\label{ctypes-bugs-todo-non-implemented-things}} Enumeration types are not implemented. You can do it easily yourself, @@ -1224,3 +1412,1014 @@ using \class{c{\_}int} as the base class. % compile-command: "make.bat" % End: + +\subsection{ctypes reference\label{ctypes-ctypes-reference}} + + +\subsubsection{Finding shared libraries\label{ctypes-finding-shared-libraries}} + +When programming in a compiled language, shared libraries are accessed +when compiling/linking a program, and when the program is run. + +The purpose of the \code{find{\_}library} function is to locate a library in +a way similar to what the compiler does (on platforms with several +versions of a shared library the most recent should be loaded), while +the ctypes library loaders act like when a program is run, and call +the runtime loader directly. + +The \code{ctypes.util} module provides a function which can help to +determine the library to load. + +\begin{datadescni}{find_library(name)} +Try to find a library and return a pathname. \var{name} is the +library name without any prefix like \var{lib}, suffix like \code{.so}, +\code{.dylib} or version number (this is the form used for the posix +linker option \programopt{-l}). If no library can be found, returns +\code{None}. +\end{datadescni} + +The exact functionality is system dependend. + +On Linux, \code{find{\_}library} tries to run external programs +(/sbin/ldconfig, gcc, and objdump) to find the library file. It +returns the filename of the library file. Here are sone examples: +\begin{verbatim} +>>> from ctypes.util import find_library +>>> find_library("m") +'libm.so.6' +>>> find_library("c") +'libc.so.6' +>>> find_library("bz2") +'libbz2.so.1.0' +>>> +\end{verbatim} + +On OS X, \code{find{\_}library} tries several predefined naming schemes and +paths to locate the library, and returns a full pathname if successfull: +\begin{verbatim} +>>> from ctypes.util import find_library +>>> find_library("c") +'/usr/lib/libc.dylib' +>>> find_library("m") +'/usr/lib/libm.dylib' +>>> find_library("bz2") +'/usr/lib/libbz2.dylib' +>>> find_library("AGL") +'/System/Library/Frameworks/AGL.framework/AGL' +>>> +\end{verbatim} + +On Windows, \code{find{\_}library} searches along the system search path, +and returns the full pathname, but since there is no predefined naming +scheme a call like \code{find{\_}library("c")} will fail and return +\code{None}. + +If wrapping a shared library with \code{ctypes}, it \emph{may} be better to +determine the shared library name at development type, and hardcode +that into the wrapper module instead of using \code{find{\_}library} to +locate the library at runtime. + + +\subsubsection{Loading shared libraries\label{ctypes-loading-shared-libraries}} + +There are several ways to loaded shared libraries into the Python +process. One way is to instantiate one of the following classes: + +\begin{classdesc}{CDLL}{name, mode=DEFAULT_MODE, handle=None} +Instances of this class represent loaded shared libraries. +Functions in these libraries use the standard C calling +convention, and are assumed to return \code{int}. +\end{classdesc} + +\begin{classdesc}{OleDLL}{name, mode=DEFAULT_MODE, handle=None} +Windows only: Instances of this class represent loaded shared +libraries, functions in these libraries use the \code{stdcall} +calling convention, and are assumed to return the windows specific +\class{HRESULT} code. \class{HRESULT} values contain information +specifying whether the function call failed or succeeded, together +with additional error code. If the return value signals a +failure, an \class{WindowsError} is automatically raised. +\end{classdesc} + +\begin{classdesc}{WinDLL}{name, mode=DEFAULT_MODE, handle=None} +Windows only: Instances of this class represent loaded shared +libraries, functions in these libraries use the \code{stdcall} +calling convention, and are assumed to return \code{int} by default. + +On Windows CE only the standard calling convention is used, for +convenience the \class{WinDLL} and \class{OleDLL} use the standard calling +convention on this platform. +\end{classdesc} + +The Python GIL is released before calling any function exported by +these libraries, and reaquired afterwards. + +\begin{classdesc}{PyDLL}{name, mode=DEFAULT_MODE, handle=None} +Instances of this class behave like \class{CDLL} instances, except +that the Python GIL is \emph{not} released during the function call, +and after the function execution the Python error flag is checked. +If the error flag is set, a Python exception is raised. + +Thus, this is only useful to call Python C api functions directly. +\end{classdesc} + +All these classes can be instantiated by calling them with at least +one argument, the pathname of the shared library. If you have an +existing handle to an already loaded shard library, it can be passed +as the \code{handle} named parameter, otherwise the underlying platforms +\code{dlopen} or \method{LoadLibrary} function is used to load the library +into the process, and to get a handle to it. + +The \var{mode} parameter can be used to specify how the library is +loaded. For details, consult the \code{dlopen(3)} manpage, on Windows, +\var{mode} is ignored. + +\begin{datadescni}{RTLD_GLOBAL} +Flag to use as \var{mode} parameter. On platforms where this flag +is not available, it is defined as the integer zero. +\end{datadescni} + +\begin{datadescni}{RTLD_LOCAL} +Flag to use as \var{mode} parameter. On platforms where this is not +available, it is the same as \var{RTLD{\_}GLOBAL}. +\end{datadescni} + +\begin{datadescni}{DEFAULT_MODE} +The default mode which is used to load shared libraries. On OSX +10.3, this is \var{RTLD{\_}GLOBAL}, otherwise it is the same as +\var{RTLD{\_}LOCAL}. +\end{datadescni} + +Instances of these classes have no public methods, however +\method{{\_}{\_}getattr{\_}{\_}} and \method{{\_}{\_}getitem{\_}{\_}} have special behaviour: functions +exported by the shared library can be accessed as attributes of by +index. Please note that both \method{{\_}{\_}getattr{\_}{\_}} and \method{{\_}{\_}getitem{\_}{\_}} +cache their result, so calling them repeatedly returns the same object +each time. + +The following public attributes are available, their name starts with +an underscore to not clash with exported function names: + +\begin{memberdesc}{_handle} +The system handle used to access the library. +\end{memberdesc} + +\begin{memberdesc}{_name} +The name of the library passed in the contructor. +\end{memberdesc} + +Shared libraries can also be loaded by using one of the prefabricated +objects, which are instances of the \class{LibraryLoader} class, either by +calling the \method{LoadLibrary} method, or by retrieving the library as +attribute of the loader instance. + +\begin{classdesc}{LibraryLoader}{dlltype} +Class which loads shared libraries. \code{dlltype} should be one +of the \class{CDLL}, \class{PyDLL}, \class{WinDLL}, or \class{OleDLL} types. + +\method{{\_}{\_}getattr{\_}{\_}} has special behaviour: It allows to load a shared +library by accessing it as attribute of a library loader +instance. The result is cached, so repeated attribute accesses +return the same library each time. +\end{classdesc} + +\begin{methoddesc}{LoadLibrary}{name} +Load a shared library into the process and return it. This method +always returns a new instance of the library. +\end{methoddesc} + +These prefabricated library loaders are available: + +\begin{datadescni}{cdll} +Creates \class{CDLL} instances. +\end{datadescni} + +\begin{datadescni}{windll} +Windows only: Creates \class{WinDLL} instances. +\end{datadescni} + +\begin{datadescni}{oledll} +Windows only: Creates \class{OleDLL} instances. +\end{datadescni} + +\begin{datadescni}{pydll} +Creates \class{PyDLL} instances. +\end{datadescni} + +For accessing the C Python api directly, a ready-to-use Python shared +library object is available: + +\begin{datadescni}{pythonapi} +An instance of \class{PyDLL} that exposes Python C api functions as +attributes. Note that all these functions are assumed to return +integers, which is of course not always the truth, so you have to +assign the correct \member{restype} attribute to use these functions. +\end{datadescni} + + +\subsubsection{Foreign functions\label{ctypes-foreign-functions}} + +As explained in the previous section, foreign functions can be +accessed as attributes of loaded shared libraries. The function +objects created in this way by default accept any number of arguments, +accept any ctypes data instances as arguments, and return the default +result type specified by the library loader. They are instances of a +private class: + +\begin{classdesc*}{_FuncPtr} +Base class for C callable foreign functions. +\end{classdesc*} + +Instances of foreign functions are also C compatible data types; they +represent C function pointers. + +This behaviour can be customized by assigning to special attributes of +the foreign function object. + +\begin{memberdesc}{restype} +Assign a ctypes type to specify the result type of the foreign +function. Use \code{None} for \code{void} a function not returning +anything. + +It is possible to assign a callable Python object that is not a +ctypes type, in this case the function is assumed to return an +integer, and the callable will be called with this integer, +allowing to do further processing or error checking. Using this +is deprecated, for more flexible postprocessing or error checking +use a ctypes data type as \member{restype} and assign a callable to the +\member{errcheck} attribute. +\end{memberdesc} + +\begin{memberdesc}{argtypes} +Assign a tuple of ctypes types to specify the argument types that +the function accepts. Functions using the \code{stdcall} calling +convention can only be called with the same number of arguments as +the length of this tuple; functions using the C calling convention +accept additional, unspecified arguments as well. + +When a foreign function is called, each actual argument is passed +to the \method{from{\_}param} class method of the items in the +\member{argtypes} tuple, this method allows to adapt the actual +argument to an object that the foreign function accepts. For +example, a \class{c{\_}char{\_}p} item in the \member{argtypes} tuple will +convert a unicode string passed as argument into an byte string +using ctypes conversion rules. + +New: It is now possible to put items in argtypes which are not +ctypes types, but each item must have a \method{from{\_}param} method +which returns a value usable as argument (integer, string, ctypes +instance). This allows to define adapters that can adapt custom +objects as function parameters. +\end{memberdesc} + +\begin{memberdesc}{errcheck} +Assign a Python function or another callable to this attribute. +The callable will be called with three or more arguments: +\end{memberdesc} + +\begin{funcdescni}{callable}{result, func, arguments} +\code{result} is what the foreign function returns, as specified by the +\member{restype} attribute. + +\code{func} is the foreign function object itself, this allows to +reuse the same callable object to check or postprocess the results +of several functions. + +\code{arguments} is a tuple containing the parameters originally +passed to the function call, this allows to specialize the +behaviour on the arguments used. + +The object that this function returns will be returned from the +foreign function call, but it can also check the result value and +raise an exception if the foreign function call failed. +\end{funcdescni} + +\begin{excdesc}{ArgumentError()} +This exception is raised when a foreign function call cannot +convert one of the passed arguments. +\end{excdesc} + + +\subsubsection{Function prototypes\label{ctypes-function-prototypes}} + +Foreign functions can also be created by instantiating function +prototypes. Function prototypes are similar to function prototypes in +C; they describe a function (return type, argument types, calling +convention) without defining an implementation. The factory +functions must be called with the desired result type and the argument +types of the function. + +\begin{funcdesc}{CFUNCTYPE}{restype, *argtypes} +The returned function prototype creates functions that use the +standard C calling convention. The function will release the GIL +during the call. +\end{funcdesc} + +\begin{funcdesc}{WINFUNCTYPE}{restype, *argtypes} +Windows only: The returned function prototype creates functions +that use the \code{stdcall} calling convention, except on Windows CE +where \function{WINFUNCTYPE} is the same as \function{CFUNCTYPE}. The function +will release the GIL during the call. +\end{funcdesc} + +\begin{funcdesc}{PYFUNCTYPE}{restype, *argtypes} +The returned function prototype creates functions that use the +Python calling convention. The function will \emph{not} release the +GIL during the call. +\end{funcdesc} + +Function prototypes created by the factory functions can be +instantiated in different ways, depending on the type and number of +the parameters in the call. + +\begin{funcdescni}{prototype}{address} +Returns a foreign function at the specified address. +\end{funcdescni} + +\begin{funcdescni}{prototype}{callable} +Create a C callable function (a callback function) from a Python +\code{callable}. +\end{funcdescni} + +\begin{funcdescni}{prototype}{func_spec\optional{, paramflags}} +Returns a foreign function exported by a shared library. +\code{func{\_}spec} must be a 2-tuple \code{(name{\_}or{\_}ordinal, library)}. +The first item is the name of the exported function as string, or +the ordinal of the exported function as small integer. The second +item is the shared library instance. +\end{funcdescni} + +\begin{funcdescni}{prototype}{vtbl_index, name\optional{, paramflags\optional{, iid}}} +Returns a foreign function that will call a COM method. +\code{vtbl{\_}index} is the index into the virtual function table, a +small nonnegative integer. \var{name} is name of the COM method. +\var{iid} is an optional pointer to the interface identifier which +is used in extended error reporting. + +COM methods use a special calling convention: They require a +pointer to the COM interface as first argument, in addition to +those parameters that are specified in the \member{argtypes} tuple. +\end{funcdescni} + +The optional \var{paramflags} parameter creates foreign function +wrappers with much more functionality than the features described +above. + +\var{paramflags} must be a tuple of the same length as \member{argtypes}. + +Each item in this tuple contains further information about a +parameter, it must be a tuple containing 1, 2, or 3 items. + +The first item is an integer containing flags for the parameter: + +\begin{datadescni}{1} +Specifies an input parameter to the function. +\end{datadescni} + +\begin{datadescni}{2} +Output parameter. The foreign function fills in a value. +\end{datadescni} + +\begin{datadescni}{4} +Input parameter which defaults to the integer zero. +\end{datadescni} + +The optional second item is the parameter name as string. If this is +specified, the foreign function can be called with named parameters. + +The optional third item is the default value for this parameter. + +This example demonstrates how to wrap the Windows \code{MessageBoxA} +function so that it supports default parameters and named arguments. +The C declaration from the windows header file is this: +\begin{verbatim} +WINUSERAPI int WINAPI +MessageBoxA( + HWND hWnd , + LPCSTR lpText, + LPCSTR lpCaption, + UINT uType); +\end{verbatim} + +Here is the wrapping with \code{ctypes}: +\begin{quote} +\begin{verbatim}>>> from ctypes import c_int, WINFUNCTYPE, windll +>>> from ctypes.wintypes import HWND, LPCSTR, UINT +>>> prototype = WINFUNCTYPE(c_int, HWND, LPCSTR, LPCSTR, c_uint) +>>> paramflags = (1, "hwnd", 0), (1, "text", "Hi"), (1, "caption", None), (1, "flags", 0) +>>> MessageBox = prototype(("MessageBoxA", windll.user32), paramflags) +>>>\end{verbatim} +\end{quote} + +The MessageBox foreign function can now be called in these ways: +\begin{verbatim} +>>> MessageBox() +>>> MessageBox(text="Spam, spam, spam") +>>> MessageBox(flags=2, text="foo bar") +>>> +\end{verbatim} + +A second example demonstrates output parameters. The win32 +\code{GetWindowRect} function retrieves the dimensions of a specified +window by copying them into \code{RECT} structure that the caller has to +supply. Here is the C declaration: +\begin{verbatim} +WINUSERAPI BOOL WINAPI +GetWindowRect( + HWND hWnd, + LPRECT lpRect); +\end{verbatim} + +Here is the wrapping with \code{ctypes}: +\begin{quote} +\begin{verbatim}>>> from ctypes import POINTER, WINFUNCTYPE, windll +>>> from ctypes.wintypes import BOOL, HWND, RECT +>>> prototype = WINFUNCTYPE(BOOL, HWND, POINTER(RECT)) +>>> paramflags = (1, "hwnd"), (2, "lprect") +>>> GetWindowRect = prototype(("GetWindowRect", windll.user32), paramflags) +>>>\end{verbatim} +\end{quote} + +Functions with output parameters will automatically return the output +parameter value if there is a single one, or a tuple containing the +output parameter values when there are more than one, so the +GetWindowRect function now returns a RECT instance, when called. + +Output parameters can be combined with the \member{errcheck} protocol to do +further output processing and error checking. The win32 +\code{GetWindowRect} api function returns a \code{BOOL} to signal success or +failure, so this function could do the error checking, and raises an +exception when the api call failed: +\begin{verbatim} +>>> def errcheck(result, func, args): +... if not result: +... raise WinError() +... return args +>>> GetWindowRect.errcheck = errcheck +>>> +\end{verbatim} + +If the \member{errcheck} function returns the argument tuple it receives +unchanged, \code{ctypes} continues the normal processing it does on the +output parameters. If you want to return a tuple of window +coordinates instead of a \code{RECT} instance, you can retrieve the +fields in the function and return them instead, the normal processing +will no longer take place: +\begin{verbatim} +>>> def errcheck(result, func, args): +... if not result: +... raise WinError() +... rc = args[1] +... return rc.left, rc.top, rc.bottom, rc.right +>>> +>>> GetWindowRect.errcheck = errcheck +>>> +\end{verbatim} + + +\subsubsection{Utility functions\label{ctypes-utility-functions}} + +\begin{funcdesc}{addressof}{obj} +Returns the address of the memory buffer as integer. \code{obj} must +be an instance of a ctypes type. +\end{funcdesc} + +\begin{funcdesc}{alignment}{obj_or_type} +Returns the alignment requirements of a ctypes type. +\code{obj{\_}or{\_}type} must be a ctypes type or instance. +\end{funcdesc} + +\begin{funcdesc}{byref}{obj} +Returns a light-weight pointer to \code{obj}, which must be an +instance of a ctypes type. The returned object can only be used as +a foreign function call parameter. It behaves similar to +\code{pointer(obj)}, but the construction is a lot faster. +\end{funcdesc} + +\begin{funcdesc}{cast}{obj, type} +This function is similar to the cast operator in C. It returns a +new instance of \code{type} which points to the same memory block as +\code{obj}. \code{type} must be a pointer type, and \code{obj} must be an +object that can be interpreted as a pointer. +\end{funcdesc} + +\begin{funcdesc}{create_string_buffer}{init_or_size\optional{, size}} +This function creates a mutable character buffer. The returned +object is a ctypes array of \class{c{\_}char}. + +\code{init{\_}or{\_}size} must be an integer which specifies the size of +the array, or a string which will be used to initialize the array +items. + +If a string is specified as first argument, the buffer is made one +item larger than the length of the string so that the last element +in the array is a NUL termination character. An integer can be +passed as second argument which allows to specify the size of the +array if the length of the string should not be used. + +If the first parameter is a unicode string, it is converted into +an 8-bit string according to ctypes conversion rules. +\end{funcdesc} + +\begin{funcdesc}{create_unicode_buffer}{init_or_size\optional{, size}} +This function creates a mutable unicode character buffer. The +returned object is a ctypes array of \class{c{\_}wchar}. + +\code{init{\_}or{\_}size} must be an integer which specifies the size of +the array, or a unicode string which will be used to initialize +the array items. + +If a unicode string is specified as first argument, the buffer is +made one item larger than the length of the string so that the +last element in the array is a NUL termination character. An +integer can be passed as second argument which allows to specify +the size of the array if the length of the string should not be +used. + +If the first parameter is a 8-bit string, it is converted into an +unicode string according to ctypes conversion rules. +\end{funcdesc} + +\begin{funcdesc}{DllCanUnloadNow}{} +Windows only: This function is a hook which allows to implement +inprocess COM servers with ctypes. It is called from the +DllCanUnloadNow function that the {\_}ctypes extension dll exports. +\end{funcdesc} + +\begin{funcdesc}{DllGetClassObject}{} +Windows only: This function is a hook which allows to implement +inprocess COM servers with ctypes. It is called from the +DllGetClassObject function that the \code{{\_}ctypes} extension dll exports. +\end{funcdesc} + +\begin{funcdesc}{FormatError}{\optional{code}} +Windows only: Returns a textual description of the error code. If +no error code is specified, the last error code is used by calling +the Windows api function GetLastError. +\end{funcdesc} + +\begin{funcdesc}{GetLastError}{} +Windows only: Returns the last error code set by Windows in the +calling thread. +\end{funcdesc} + +\begin{funcdesc}{memmove}{dst, src, count} +Same as the standard C memmove library function: copies \var{count} +bytes from \code{src} to \var{dst}. \var{dst} and \code{src} must be +integers or ctypes instances that can be converted to pointers. +\end{funcdesc} + +\begin{funcdesc}{memset}{dst, c, count} +Same as the standard C memset library function: fills the memory +block at address \var{dst} with \var{count} bytes of value +\var{c}. \var{dst} must be an integer specifying an address, or a +ctypes instance. +\end{funcdesc} + +\begin{funcdesc}{POINTER}{type} +This factory function creates and returns a new ctypes pointer +type. Pointer types are cached an reused internally, so calling +this function repeatedly is cheap. type must be a ctypes type. +\end{funcdesc} + +\begin{funcdesc}{pointer}{obj} +This function creates a new pointer instance, pointing to +\code{obj}. The returned object is of the type POINTER(type(obj)). + +Note: If you just want to pass a pointer to an object to a foreign +function call, you should use \code{byref(obj)} which is much faster. +\end{funcdesc} + +\begin{funcdesc}{resize}{obj, size} +This function resizes the internal memory buffer of obj, which +must be an instance of a ctypes type. It is not possible to make +the buffer smaller than the native size of the objects type, as +given by sizeof(type(obj)), but it is possible to enlarge the +buffer. +\end{funcdesc} + +\begin{funcdesc}{set_conversion_mode}{encoding, errors} +This function sets the rules that ctypes objects use when +converting between 8-bit strings and unicode strings. encoding +must be a string specifying an encoding, like \code{'utf-8'} or +\code{'mbcs'}, errors must be a string specifying the error handling +on encoding/decoding errors. Examples of possible values are +\code{"strict"}, \code{"replace"}, or \code{"ignore"}. + +\code{set{\_}conversion{\_}mode} returns a 2-tuple containing the previous +conversion rules. On windows, the initial conversion rules are +\code{('mbcs', 'ignore')}, on other systems \code{('ascii', 'strict')}. +\end{funcdesc} + +\begin{funcdesc}{sizeof}{obj_or_type} +Returns the size in bytes of a ctypes type or instance memory +buffer. Does the same as the C \code{sizeof()} function. +\end{funcdesc} + +\begin{funcdesc}{string_at}{address\optional{, size}} +This function returns the string starting at memory address +address. If size is specified, it is used as size, otherwise the +string is assumed to be zero-terminated. +\end{funcdesc} + +\begin{funcdesc}{WinError}{code=None, descr=None} +Windows only: this function is probably the worst-named thing in +ctypes. It creates an instance of WindowsError. If \var{code} is not +specified, \code{GetLastError} is called to determine the error +code. If \code{descr} is not spcified, \function{FormatError} is called to +get a textual description of the error. +\end{funcdesc} + +\begin{funcdesc}{wstring_at}{address} +This function returns the wide character string starting at memory +address \code{address} as unicode string. If \code{size} is specified, +it is used as the number of characters of the string, otherwise +the string is assumed to be zero-terminated. +\end{funcdesc} + + +\subsubsection{Data types\label{ctypes-data-types}} + +\begin{classdesc*}{_CData} +This non-public class is the common base class of all ctypes data +types. Among other things, all ctypes type instances contain a +memory block that hold C compatible data; the address of the +memory block is returned by the \code{addressof()} helper function. +Another instance variable is exposed as \member{{\_}objects}; this +contains other Python objects that need to be kept alive in case +the memory block contains pointers. +\end{classdesc*} + +Common methods of ctypes data types, these are all class methods (to +be exact, they are methods of the metaclass): + +\begin{methoddesc}{from_address}{address} +This method returns a ctypes type instance using the memory +specified by address which must be an integer. +\end{methoddesc} + +\begin{methoddesc}{from_param}{obj} +This method adapts obj to a ctypes type. It is called with the +actual object used in a foreign function call, when the type is +present in the foreign functions \member{argtypes} tuple; it must +return an object that can be used as function call parameter. + +All ctypes data types have a default implementation of this +classmethod, normally it returns \code{obj} if that is an instance of +the type. Some types accept other objects as well. +\end{methoddesc} + +\begin{methoddesc}{in_dll}{name, library} +This method returns a ctypes type instance exported by a shared +library. \var{name} is the name of the symbol that exports the data, +\code{library} is the loaded shared library. +\end{methoddesc} + +Common instance variables of ctypes data types: + +\begin{memberdesc}{_b_base_} +Sometimes ctypes data instances do not own the memory block they +contain, instead they share part of the memory block of a base +object. The \member{{\_}b{\_}base{\_}} readonly member is the root ctypes +object that owns the memory block. +\end{memberdesc} + +\begin{memberdesc}{_b_needsfree_} +This readonly variable is true when the ctypes data instance has +allocated the memory block itself, false otherwise. +\end{memberdesc} + +\begin{memberdesc}{_objects} +This member is either \code{None} or a dictionary containing Python +objects that need to be kept alive so that the memory block +contents is kept valid. This object is only exposed for +debugging; never modify the contents of this dictionary. +\end{memberdesc} + + +\subsubsection{Fundamental data types\label{ctypes-fundamental-data-types}} + +\begin{classdesc*}{_SimpleCData} +This non-public class is the base class of all fundamental ctypes +data types. It is mentioned here because it contains the common +attributes of the fundamental ctypes data types. \code{{\_}SimpleCData} +is a subclass of \code{{\_}CData}, so it inherits their methods and +attributes. +\end{classdesc*} + +Instances have a single attribute: + +\begin{memberdesc}{value} +This attribute contains the actual value of the instance. For +integer and pointer types, it is an integer, for character types, +it is a single character string, for character pointer types it +is a Python string or unicode string. + +When the \code{value} attribute is retrieved from a ctypes instance, +usually a new object is returned each time. \code{ctypes} does \emph{not} +implement original object return, always a new object is +constructed. The same is true for all other ctypes object +instances. +\end{memberdesc} + +Fundamental data types, when returned as foreign function call +results, or, for example, by retrieving structure field members or +array items, are transparently converted to native Python types. In +other words, if a foreign function has a \member{restype} of \class{c{\_}char{\_}p}, +you will always receive a Python string, \emph{not} a \class{c{\_}char{\_}p} +instance. + +Subclasses of fundamental data types do \emph{not} inherit this behaviour. +So, if a foreign functions \member{restype} is a subclass of \class{c{\_}void{\_}p}, +you will receive an instance of this subclass from the function call. +Of course, you can get the value of the pointer by accessing the +\code{value} attribute. + +These are the fundamental ctypes data types: + +\begin{classdesc*}{c_byte} +Represents the C signed char datatype, and interprets the value as +small integer. The constructor accepts an optional integer +initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_char} +Represents the C char datatype, and interprets the value as a single +character. The constructor accepts an optional string initializer, +the length of the string must be exactly one character. +\end{classdesc*} + +\begin{classdesc*}{c_char_p} +Represents the C char * datatype, which must be a pointer to a +zero-terminated string. The constructor accepts an integer +address, or a string. +\end{classdesc*} + +\begin{classdesc*}{c_double} +Represents the C double datatype. The constructor accepts an +optional float initializer. +\end{classdesc*} + +\begin{classdesc*}{c_float} +Represents the C double datatype. The constructor accepts an +optional float initializer. +\end{classdesc*} + +\begin{classdesc*}{c_int} +Represents the C signed int datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. On +platforms where \code{sizeof(int) == sizeof(long)} it is an alias to +\class{c{\_}long}. +\end{classdesc*} + +\begin{classdesc*}{c_int8} +Represents the C 8-bit \code{signed int} datatype. Usually an alias for +\class{c{\_}byte}. +\end{classdesc*} + +\begin{classdesc*}{c_int16} +Represents the C 16-bit signed int datatype. Usually an alias for +\class{c{\_}short}. +\end{classdesc*} + +\begin{classdesc*}{c_int32} +Represents the C 32-bit signed int datatype. Usually an alias for +\class{c{\_}int}. +\end{classdesc*} + +\begin{classdesc*}{c_int64} +Represents the C 64-bit \code{signed int} datatype. Usually an alias +for \class{c{\_}longlong}. +\end{classdesc*} + +\begin{classdesc*}{c_long} +Represents the C \code{signed long} datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_longlong} +Represents the C \code{signed long long} datatype. The constructor accepts +an optional integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_short} +Represents the C \code{signed short} datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_size_t} +Represents the C \code{size{\_}t} datatype. +\end{classdesc*} + +\begin{classdesc*}{c_ubyte} +Represents the C \code{unsigned char} datatype, it interprets the +value as small integer. The constructor accepts an optional +integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_uint} +Represents the C \code{unsigned int} datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. On +platforms where \code{sizeof(int) == sizeof(long)} it is an alias for +\class{c{\_}ulong}. +\end{classdesc*} + +\begin{classdesc*}{c_uint8} +Represents the C 8-bit unsigned int datatype. Usually an alias for +\class{c{\_}ubyte}. +\end{classdesc*} + +\begin{classdesc*}{c_uint16} +Represents the C 16-bit unsigned int datatype. Usually an alias for +\class{c{\_}ushort}. +\end{classdesc*} + +\begin{classdesc*}{c_uint32} +Represents the C 32-bit unsigned int datatype. Usually an alias for +\class{c{\_}uint}. +\end{classdesc*} + +\begin{classdesc*}{c_uint64} +Represents the C 64-bit unsigned int datatype. Usually an alias for +\class{c{\_}ulonglong}. +\end{classdesc*} + +\begin{classdesc*}{c_ulong} +Represents the C \code{unsigned long} datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_ulonglong} +Represents the C \code{unsigned long long} datatype. The constructor +accepts an optional integer initializer; no overflow checking is +done. +\end{classdesc*} + +\begin{classdesc*}{c_ushort} +Represents the C \code{unsigned short} datatype. The constructor accepts an +optional integer initializer; no overflow checking is done. +\end{classdesc*} + +\begin{classdesc*}{c_void_p} +Represents the C \code{void *} type. The value is represented as +integer. The constructor accepts an optional integer initializer. +\end{classdesc*} + +\begin{classdesc*}{c_wchar} +Represents the C \code{wchar{\_}t} datatype, and interprets the value as a +single character unicode string. The constructor accepts an +optional string initializer, the length of the string must be +exactly one character. +\end{classdesc*} + +\begin{classdesc*}{c_wchar_p} +Represents the C \code{wchar{\_}t *} datatype, which must be a pointer to +a zero-terminated wide character string. The constructor accepts +an integer address, or a string. +\end{classdesc*} + +\begin{classdesc*}{HRESULT} +Windows only: Represents a \class{HRESULT} value, which contains success +or error information for a function or method call. +\end{classdesc*} + +\begin{classdesc*}{py_object} +Represents the C \code{PyObject *} datatype. +\end{classdesc*} + +The \code{ctypes.wintypes} module provides quite some other Windows +specific data types, for example \code{HWND}, \code{WPARAM}, or \code{DWORD}. +Some useful structures like \code{MSG} or \code{RECT} are also defined. + + +\subsubsection{Structured data types\label{ctypes-structured-data-types}} + +\begin{classdesc}{Union}{*args, **kw} +Abstract base class for unions in native byte order. +\end{classdesc} + +\begin{classdesc}{BigEndianStructure}{*args, **kw} +Abstract base class for structures in \emph{big endian} byte order. +\end{classdesc} + +\begin{classdesc}{LittleEndianStructure}{*args, **kw} +Abstract base class for structures in \emph{little endian} byte order. +\end{classdesc} + +Structures with non-native byte order cannot contain pointer type +fields, or any other data types containing pointer type fields. + +\begin{classdesc}{Structure}{*args, **kw} +Abstract base class for structures in \emph{native} byte order. +\end{classdesc} + +Concrete structure and union types must be created by subclassing one +of these types, and at least define a \member{{\_}fields{\_}} class variable. +\code{ctypes} will create descriptors which allow reading and writing the +fields by direct attribute accesses. These are the + +\begin{memberdesc}{_fields_} +A sequence defining the structure fields. The items must be +2-tuples or 3-tuples. The first item is the name of the field, +the second item specifies the type of the field; it can be any +ctypes data type. + +For integer type fields, a third optional item can be given. It +must be a small positive integer defining the bit width of the +field. + +Field names must be unique within one structure or union. This is +not checked, only one field can be accessed when names are +repeated. + +It is possible to define the \member{{\_}fields{\_}} class variable \emph{after} +the class statement that defines the Structure subclass, this +allows to create data types that directly or indirectly reference +themselves: +\begin{verbatim} +class List(Structure): + pass +List._fields_ = [("pnext", POINTER(List)), + ... + ] +\end{verbatim} + +The \member{{\_}fields{\_}} class variable must, however, be defined before +the type is first used (an instance is created, \code{sizeof()} is +called on it, and so on). Later assignments to the \member{{\_}fields{\_}} +class variable will raise an AttributeError. + +Structure and union subclass constructors accept both positional +and named arguments. Positional arguments are used to initialize +the fields in the same order as they appear in the \member{{\_}fields{\_}} +definition, named arguments are used to initialize the fields with +the corresponding name. + +It is possible to defined sub-subclasses of structure types, they +inherit the fields of the base class plus the \member{{\_}fields{\_}} defined +in the sub-subclass, if any. +\end{memberdesc} + +\begin{memberdesc}{_pack_} +An optional small integer that allows to override the alignment of +structure fields in the instance. \member{{\_}pack{\_}} must already be +defined when \member{{\_}fields{\_}} is assigned, otherwise it will have no +effect. +\end{memberdesc} + +\begin{memberdesc}{_anonymous_} +An optional sequence that lists the names of unnamed (anonymous) +fields. \code{{\_}anonymous{\_}} must be already defined when \member{{\_}fields{\_}} +is assigned, otherwise it will have no effect. + +The fields listed in this variable must be structure or union type +fields. \code{ctypes} will create descriptors in the structure type +that allows to access the nested fields directly, without the need +to create the structure or union field. + +Here is an example type (Windows): +\begin{verbatim} +class _U(Union): + _fields_ = [("lptdesc", POINTER(TYPEDESC)), + ("lpadesc", POINTER(ARRAYDESC)), + ("hreftype", HREFTYPE)] + +class TYPEDESC(Structure): + _fields_ = [("u", _U), + ("vt", VARTYPE)] + + _anonymous_ = ("u",) +\end{verbatim} + +The \code{TYPEDESC} structure describes a COM data type, the \code{vt} +field specifies which one of the union fields is valid. Since the +\code{u} field is defined as anonymous field, it is now possible to +access the members directly off the TYPEDESC instance. +\code{td.lptdesc} and \code{td.u.lptdesc} are equivalent, but the former +is faster since it does not need to create a temporary union +instance: +\begin{verbatim} +td = TYPEDESC() +td.vt = VT_PTR +td.lptdesc = POINTER(some_type) +td.u.lptdesc = POINTER(some_type) +\end{verbatim} +\end{memberdesc} + +It is possible to defined sub-subclasses of structures, they inherit +the fields of the base class. If the subclass definition has a +separate \member{{\_}fields{\_}} variable, the fields specified in this are +appended to the fields of the base class. + +Structure and union constructors accept both positional and +keyword arguments. Positional arguments are used to initialize member +fields in the same order as they are appear in \member{{\_}fields{\_}}. Keyword +arguments in the constructor are interpreted as attribute assignments, +so they will initialize \member{{\_}fields{\_}} with the same name, or create new +attributes for names not present in \member{{\_}fields{\_}}. + + +\subsubsection{Arrays and pointers\label{ctypes-arrays-pointers}} + +XXX + diff --git a/Doc/lib/libctypesref.tex b/Doc/lib/libctypesref.tex deleted file mode 100644 index 6d950f4..0000000 --- a/Doc/lib/libctypesref.tex +++ /dev/null @@ -1,457 +0,0 @@ -\subsection{ctypes reference\label{ctypes-reference}} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% functions -\subsubsection{ctypes functions} - -\begin{funcdesc}{addressof}{obj} -Returns the address of the memory buffer as integer. \var{obj} must -be an instance of a ctypes type. -\end{funcdesc} - -\begin{funcdesc}{alignment}{obj_or_type} -Returns the alignment requirements of a ctypes type. -\var{obj_or_type} must be a ctypes type or an instance. -\end{funcdesc} - -\begin{excclassdesc}{ArgumentError}{} -This exception is raised when a foreign function call cannot convert -one of the passed arguments. -\end{excclassdesc} - -\begin{funcdesc}{byref}{obj} -Returns a light-weight pointer to \var{obj}, which must be an instance -of a ctypes type. The returned object can only be used as a foreign -function call parameter. It behaves similar to \code{pointer(obj)}, -but the construction is a lot faster. -\end{funcdesc} - -\begin{funcdesc}{cast}{obj, type} -This function is similar to the cast operator in C. It returns a new -instance of \var{type} which points to the same memory block as -\code{obj}. \code{type} must be a pointer type, and \code{obj} - must be an object that can be interpreted as a pointer. -\end{funcdesc} - -% XXX separate section for CFUNCTYPE, WINFUNCTYPE, PYFUNCTYPE? - -\begin{funcdesc}{CFUNCTYPE}{restype, *argtypes} -This is a factory function that returns a function prototype. The -function prototype describes a function that has a result type of -\code{restype}, and accepts arguments as specified by \code{argtypes}. -The function prototype can be used to construct several kinds of -functions, depending on how the prototype is called. - -The prototypes returned by \code{CFUNCTYPE} or \code{PYFUNCTYPE} -create functions that use the standard C calling convention, -prototypes returned from \code{WINFUNCTYPE} (on Windows) use the -\code{__stdcall} calling convention. - -Functions created by calling the \code{CFUNCTYPE} and -\code{WINFUNCTYPE} prototypes release the Python GIL -before entering the foreign function, and acquire it back after -leaving the function code. - -% XXX differences between CFUNCTYPE / WINFUNCTYPE / PYFUNCTYPE - -\end{funcdesc} - -\begin{funcdesc}{create_string_buffer}{init_or_size\optional{, size}} -This function creates a mutable character buffer. The returned object -is a ctypes array of \code{c_char}. - -\var{init_or_size} must be an integer which specifies the size of the -array, or a string which will be used to initialize the array items. - -If a string is specified as first argument, the buffer is made one -item larger than the length of the string so that the last element in -the array is a NUL termination character. An integer can be passed as -second argument which allows to specify the size of the array if the -length of the string should not be used. - -If the first parameter is a unicode string, it is converted into an -8-bit string according to ctypes conversion rules. -\end{funcdesc} - -\begin{funcdesc}{create_unicode_buffer}{init_or_size\optional{, size}} -This function creates a mutable unicode character buffer. The -returned object is a ctypes array of \code{c_wchar}. - -\var{init_or_size} must be an integer which specifies the size of the -array, or a unicode string which will be used to initialize the array -items. - -If a unicode string is specified as first argument, the buffer is made -one item larger than the length of the string so that the last element -in the array is a NUL termination character. An integer can be passed -as second argument which allows to specify the size of the array if -the length of the string should not be used. - -If the first parameter is a 8-bit string, it is converted into an -unicode string according to ctypes conversion rules. -\end{funcdesc} - -\begin{funcdesc}{DllCanUnloadNow}{} -Windows only: This function is a hook which allows to implement -inprocess COM servers with ctypes. It is called from the -\code{DllCanUnloadNow} function that the \code{_ctypes} -extension dll exports. -\end{funcdesc} - -\begin{funcdesc}{DllGetClassObject}{} -Windows only: This function is a hook which allows to implement -inprocess COM servers with ctypes. It is called from the -\code{DllGetClassObject} function that the \code{_ctypes} -extension dll exports. -\end{funcdesc} - -\begin{funcdesc}{FormatError}{\optional{code}} -Windows only: Returns a textual description of the error code. If no -error code is specified, the last error code is used by calling the -Windows api function \code{GetLastError}. -\end{funcdesc} - -\begin{funcdesc}{GetLastError}{} -Windows only: Returns the last error code set by Windows in the -calling thread. -\end{funcdesc} - -\begin{funcdesc}{memmove}{dst, src, count} -Same as the standard C \code{memmove} library function: copies -\var{count} bytes from \code{src} to \code{dst}. \code{dst} and -\code{src} must be integers or ctypes instances that can be converted to pointers. -\end{funcdesc} - -\begin{funcdesc}{memset}{dst, c, count} -Same as the standard C \code{memset} library function: fills the -memory clock at address \code{dst} with \var{count} bytes of value -\var{c}. \var{dst} must be an integer specifying an address, or a ctypes instance. -\end{funcdesc} - -\begin{funcdesc}{POINTER}{type} -This factory function creates and returns a new ctypes pointer type. -Pointer types are cached an reused internally, so calling this -function repeatedly is cheap. \var{type} must be a ctypes type. -\end{funcdesc} - -\begin{funcdesc}{pointer}{obj} -This function creates a new pointer instance, pointing to \var{obj}. -The returned object is of the type \code{POINTER(type(obj))}. - -Note: If you just want to pass a pointer to an object to a foreign -function call, you should use \code{byref(obj)} which is much faster. -\end{funcdesc} - -\begin{funcdesc}{PYFUNCTYPE}{restype, *argtypes} -\end{funcdesc} - -\begin{funcdesc}{pythonapi}{} -\end{funcdesc} - -\begin{funcdesc}{resize}{obj, size} -This function resizes the internal memory buffer of \var{obj}, which -must be an instance of a ctypes type. It is not possible to make the -buffer smaller than the native size of the objects type, as given by -\code{sizeof(type(obj))}, but it is possible to enlarge the buffer. -\end{funcdesc} - -\begin{funcdesc}{set_conversion_mode}{encoding, errors} -This function sets the rules that ctypes objects use when converting -between 8-bit strings and unicode strings. \var{encoding} must be a -string specifying an encoding, like 'utf-8' or 'mbcs', \var{errors} -must be a string specifying the error handling on encoding/decoding -errors. Examples of possible values are ``strict'', ``replace'', or -``ignore''. - -\code{set_conversion_mode} returns a 2-tuple containing the previous -conversion rules. On windows, the initial conversion rules are -\code{('mbcs', 'ignore')}, on other systems \code{('ascii', 'strict')}. -\end{funcdesc} - -\begin{funcdesc}{sizeof}{obj_or_type} -Returns the size in bytes of a ctypes type or instance memory buffer. -Does the same as the C sizeof() function. -\end{funcdesc} - -\begin{funcdesc}{string_at}{address\optional{size}} -This function returns the string starting at memory address -\var{address}. If \var{size} is specified, it is used as size, -otherwise the string is assumed to be zero-terminated. -\end{funcdesc} - -\begin{funcdesc}{WinError}{code=None, descr=None} -Windows only: this function is probably the worst-named thing in -ctypes. It creates an instance of \code{WindowsError}. If \var{code} -is not specified, \code{GetLastError} is called to determine the error -code. If \var{descr} is not spcified, \var{FormatError} is called to -get a textual description of the error. -\end{funcdesc} - -\begin{funcdesc}{WINFUNCTYPE}{restype, *argtypes} -\end{funcdesc} - -\begin{funcdesc}{wstring_at}{address} -This function returns the wide character string starting at memory -address \var{address} as unicode string. If \var{size} is specified, -it is used as size, otherwise the string is assumed to be -zero-terminated. -\end{funcdesc} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% data types -\subsubsection{data types} - -ctypes defines a lot of C compatible datatypes, and also allows to -define your own types. Among other things, a ctypes type instance -holds a memory block that contains C compatible data. - -\begin{classdesc}{_ctypes._CData}{} -This non-public class is the base class of all ctypes data types. It -is mentioned here because it contains the common methods of the ctypes -data types. -\end{classdesc} - -Common methods of ctypes data types, these are all class methods (to -be exact, they are methods of the metaclass): - -\begin{methoddesc}{from_address}{address} -This method returns a ctypes type instance using the memory specified -by \code{address}. -\end{methoddesc} - -\begin{methoddesc}{from_param}{obj} -This method adapts \code{obj} to a ctypes type. -\end{methoddesc} - -\begin{methoddesc}{in_dll}{name, library} -This method returns a ctypes type instance exported by a shared -library. \var{name} is the name of the symbol that exports the data, -\var{library} is the loaded shared library. -\end{methoddesc} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% simple data types -\subsubsection{simple data types} - -\begin{classdesc}{_ctypes._SimpleCData}{} -This non-public class is the base class of all ctypes data types. It -is mentioned here because it contains the common attributes of the -ctypes data types. -\end{classdesc} - -\begin{memberdesc}{value} -This attribute contains the actual value of the instance. For integer -types, it is an integer. -\end{memberdesc} - -Here are the simple ctypes data types: - -\begin{classdesc}{c_byte}{\optional{value}} -Represents a C \code{signed char} datatype, and interprets the value -as small integer. The constructor accepts an optional integer -initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_char}{\optional{value}} -Represents a C \code{char} datatype, and interprets the value as a -single character. The constructor accepts an optional string -initializer, the length of the string must be exactly one character. -\end{classdesc} - -\begin{classdesc}{c_char_p}{\optional{value}} -Represents a C \code{char *} datatype, which must be a pointer to a -zero-terminated string. The constructor accepts an integer address, -or a string. -% XXX Explain the difference to POINTER(c_char) -\end{classdesc} - -\begin{classdesc}{c_double}{\optional{value}} -Represents a C \code{double} datatype. The constructor accepts an -optional float initializer. -\end{classdesc} - -\begin{classdesc}{c_float}{\optional{value}} -Represents a C \code{double} datatype. The constructor accepts an -optional float initializer. -\end{classdesc} - -\begin{classdesc}{c_int}{\optional{value}} -Represents a C \code{signed int} datatype. The constructor accepts an -optional integer initializer; no overflow checking is done. On -platforms where \code{sizeof(int) == sizeof(long)} \var{c_int} is an -alias to \var{c_long}. -\end{classdesc} - -\begin{classdesc}{c_int16}{\optional{value}} -Represents a C 16-bit \code{signed int} datatype. Usually an alias -for \var{c_short}. -\end{classdesc} - -\begin{classdesc}{c_int32}{\optional{value}} -Represents a C 32-bit \code{signed int} datatype. Usually an alias -for \code{c_int}. -\end{classdesc} - -\begin{classdesc}{c_int64}{\optional{value}} -Represents a C 64-bit \code{signed int} datatype. Usually an alias -for \code{c_longlong}. -\end{classdesc} - -\begin{classdesc}{c_int8}{\optional{value}} -Represents a C 8-bit \code{signed int} datatype. Usually an alias for \code{c_byte}. -\end{classdesc} - -\begin{classdesc}{c_long}{\optional{value}} -Represents a C \code{signed long} datatype. The constructor accepts -an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_longlong}{\optional{value}} -Represents a C \code{signed long long} datatype. The constructor -accepts an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_short}{\optional{value}} -Represents a C \code{signed short} datatype. The constructor accepts -an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_size_t}{\optional{value}} -Represents a C \code{size_t} datatype. -\end{classdesc} - -\begin{classdesc}{c_ubyte}{\optional{value}} -Represents a C \code{unsigned char} datatype, and interprets the value -as small integer. The constructor accepts an optional integer -initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_uint}{\optional{value}} -Represents a C \code{unsigned int} datatype. The constructor accepts -an optional integer initializer; no overflow checking is done. On -platforms where \code{sizeof(int) == sizeof(long)} \var{c_int} is an -alias to \var{c_long}. -\end{classdesc} - -\begin{classdesc}{c_uint16}{\optional{value}} -Represents a C 16-bit \code{unsigned int} datatype. Usually an alias -for \code{c_ushort}. -\end{classdesc} - -\begin{classdesc}{c_uint32}{\optional{value}} -Represents a C 32-bit \code{unsigned int} datatype. Usually an alias -for \code{c_uint}. -\end{classdesc} - -\begin{classdesc}{c_uint64}{\optional{value}} -Represents a C 64-bit \code{unsigned int} datatype. Usually an alias -for \code{c_ulonglong}. -\end{classdesc} - -\begin{classdesc}{c_uint8}{\optional{value}} -Represents a C 8-bit \code{unsigned int} datatype. Usually an alias -for \code{c_ubyte}. -\end{classdesc} - -\begin{classdesc}{c_ulong}{\optional{value}} -Represents a C \code{unsigned long} datatype. The constructor accepts -an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_ulonglong}{\optional{value}} -Represents a C \code{unsigned long long} datatype. The constructor -accepts an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_ushort}{\optional{value}} -Represents a C \code{unsigned short} datatype. The constructor accepts -an optional integer initializer; no overflow checking is done. -\end{classdesc} - -\begin{classdesc}{c_void_p}{\optional{value}} -Represents a C \code{void *} type. The value is represented as -integer. The constructor accepts an optional integer initializer. -\end{classdesc} - -\begin{classdesc}{c_wchar}{\optional{value}} -Represents a C \code{wchar_t} datatype, and interprets the value as a -single character unicode string. The constructor accepts an optional -string initializer, the length of the string must be exactly one -character. -\end{classdesc} - -\begin{classdesc}{c_wchar_p}{\optional{value}} -Represents a C \code{wchar_t *} datatype, which must be a pointer to a -zero-terminated wide character string. The constructor accepts an -integer address, or a string. -% XXX Explain the difference to POINTER(c_wchar) -\end{classdesc} - -\begin{classdesc}{HRESULT}{} -Windows only: Represents a \code{HRESULT} value, which contains -success or error information for a function or method call. -\end{classdesc} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% structured data types -\subsubsection{structured data types} - -\begin{classdesc}{BigEndianStructure}{} -\end{classdesc} - -\begin{classdesc}{LittleEndianStructure}{} -\end{classdesc} - -\begin{classdesc}{Structure}{} -Base class for Structure data types. - -\end{classdesc} - -\begin{classdesc}{Union}{} -\end{classdesc} - - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% libraries -\subsubsection{libraries} - -\begin{classdesc}{CDLL}{name, mode=RTLD_LOCAL, handle=None} -\end{classdesc} - -\begin{datadesc}{cdll} -\end{datadesc} - -\begin{classdesc}{LibraryLoader}{dlltype} - -\begin{memberdesc}{LoadLibrary}{name, mode=RTLD_LOCAL, handle=None} -\end{memberdesc} - -\end{classdesc} - -\begin{classdesc}{OleDLL}{name, mode=RTLD_LOCAL, handle=None} -\end{classdesc} - -\begin{datadesc}{oledll} -\end{datadesc} - -\begin{classdesc}{py_object}{} -\end{classdesc} - -\begin{classdesc}{PyDLL}{name, mode=RTLD_LOCAL, handle=None} -\end{classdesc} - -\begin{datadesc}{pydll}{} -\end{datadesc} - -\begin{datadesc}{RTLD_GLOBAL} -\end{datadesc} - -\begin{datadesc}{RTLD_LOCAL} -\end{datadesc} - -\begin{classdesc}{WinDLL}{name, mode=RTLD_LOCAL, handle=None} -\end{classdesc} - -\begin{datadesc}{windll} -\end{datadesc} - diff --git a/Doc/lib/libdifflib.tex b/Doc/lib/libdifflib.tex index 765accc..acb5ed1 100644 --- a/Doc/lib/libdifflib.tex +++ b/Doc/lib/libdifflib.tex @@ -419,6 +419,16 @@ of the other sequences. len(\var{b}), 0)}. It is the only triple with \code{\var{n} == 0}. % Explain why a dummy is used! + If + \code{(\var{i}, \var{j}, \var{n})} and + \code{(\var{i'}, \var{j'}, \var{n'})} are adjacent triples in the list, + and the second is not the last triple in the list, then + \code{\var{i}+\var{n} != \var{i'}} or + \code{\var{j}+\var{n} != \var{j'}}; in other words, adjacent triples + always describe non-adjacent equal blocks. + \versionchanged[The guarantee that adjacent triples always describe + non-adjacent blocks was implemented]{2.5} + \begin{verbatim} >>> s = SequenceMatcher(None, "abxcd", "abcd") >>> s.get_matching_blocks() diff --git a/Doc/lib/libetree.tex b/Doc/lib/libetree.tex new file mode 100644 index 0000000..1f29887 --- /dev/null +++ b/Doc/lib/libetree.tex @@ -0,0 +1,367 @@ +\section{\module{elementtree} --- The xml.etree.ElementTree Module} +\declaremodule{standard}{elementtree} +\moduleauthor{Fredrik Lundh}{fredrik@pythonware.com} +\modulesynopsis{This module provides implementations +of the Element and ElementTree types, plus support classes. + +A C version of this API is available as xml.etree.cElementTree.} +\versionadded{2.5} + + +\subsection{Overview\label{elementtree-overview}} + +The Element type is a flexible container object, designed to store +hierarchical data structures in memory. The type can be described as a +cross between a list and a dictionary. + +Each element has a number of properties associated with it: +\begin{itemize} +\item {} +a tag which is a string identifying what kind of data +this element represents (the element type, in other words). + +\item {} +a number of attributes, stored in a Python dictionary. + +\item {} +a text string. + +\item {} +an optional tail string. + +\item {} +a number of child elements, stored in a Python sequence + +\end{itemize} + +To create an element instance, use the Element or SubElement factory +functions. + +The ElementTree class can be used to wrap an element +structure, and convert it from and to XML. + + +\subsection{Functions\label{elementtree-functions}} + +\begin{funcdesc}{Comment}{\optional{text}} +Comment element factory. This factory function creates a special +element that will be serialized as an XML comment. +The comment string can be either an 8-bit ASCII string or a Unicode +string. +\var{text} is a string containing the comment string. + +\begin{datadescni}{Returns:} +An element instance, representing a comment. +\end{datadescni} +\end{funcdesc} + +\begin{funcdesc}{dump}{elem} +Writes an element tree or element structure to sys.stdout. This +function should be used for debugging only. + +The exact output format is implementation dependent. In this +version, it's written as an ordinary XML file. + +\var{elem} is an element tree or an individual element. +\end{funcdesc} + +\begin{funcdesc}{Element}{tag\optional{, attrib}\optional{, **extra}} +Element factory. This function returns an object implementing the +standard Element interface. The exact class or type of that object +is implementation dependent, but it will always be compatible with +the {\_}ElementInterface class in this module. + +The element name, attribute names, and attribute values can be +either 8-bit ASCII strings or Unicode strings. +\var{tag} is the element name. +\var{attrib} is an optional dictionary, containing element attributes. +\var{extra} contains additional attributes, given as keyword arguments. + +\begin{datadescni}{Returns:} +An element instance. +\end{datadescni} +\end{funcdesc} + +\begin{funcdesc}{fromstring}{text} +Parses an XML section from a string constant. Same as XML. +\var{text} is a string containing XML data. + +\begin{datadescni}{Returns:} +An Element instance. +\end{datadescni} +\end{funcdesc} + +\begin{funcdesc}{iselement}{element} +Checks if an object appears to be a valid element object. +\var{element} is an element instance. + +\begin{datadescni}{Returns:} +A true value if this is an element object. +\end{datadescni} +\end{funcdesc} + +\begin{funcdesc}{iterparse}{source\optional{, events}} +Parses an XML section into an element tree incrementally, and reports +what's going on to the user. +\var{source} is a filename or file object containing XML data. +\var{events} is a list of events to report back. If omitted, only ``end'' +events are reported. + +\begin{datadescni}{Returns:} +A (event, elem) iterator. +\end{datadescni} +\end{funcdesc} + +\begin{funcdesc}{parse}{source\optional{, parser}} +Parses an XML section into an element tree. +\var{source} is a filename or file object containing XML data. +\var{parser} is an optional parser instance. If not given, the +standard XMLTreeBuilder parser is used. + +\begin{datadescni}{Returns:} +An ElementTree instance +\end{datadescni} +\end{funcdesc} + +\begin{funcdesc}{ProcessingInstruction}{target\optional{, text}} +PI element factory. This factory function creates a special element +that will be serialized as an XML processing instruction. +\var{target} is a string containing the PI target. +\var{text} is a string containing the PI contents, if given. + +\begin{datadescni}{Returns:} +An element instance, representing a PI. +\end{datadescni} +\end{funcdesc} + +\begin{funcdesc}{SubElement}{parent, tag\optional{, attrib} \optional{, **extra}} +Subelement factory. This function creates an element instance, and +appends it to an existing element. + +The element name, attribute names, and attribute values can be +either 8-bit ASCII strings or Unicode strings. +\var{parent} is the parent element. +\var{tag} is the subelement name. +\var{attrib} is an optional dictionary, containing element attributes. +\var{extra} contains additional attributes, given as keyword arguments. + +\begin{datadescni}{Returns:} +An element instance. +\end{datadescni} +\end{funcdesc} + +\begin{funcdesc}{tostring}{element\optional{, encoding}} +Generates a string representation of an XML element, including all +subelements. +\var{element} is an Element instance. +\var{encoding} is the output encoding (default is US-ASCII). + +\begin{datadescni}{Returns:} +An encoded string containing the XML data. +\end{datadescni} +\end{funcdesc} + +\begin{funcdesc}{XML}{text} +Parses an XML section from a string constant. This function can +be used to embed ``XML literals'' in Python code. +\var{text} is a string containing XML data. + +\begin{datadescni}{Returns:} +An Element instance. +\end{datadescni} +\end{funcdesc} + +\begin{funcdesc}{XMLID}{text} +Parses an XML section from a string constant, and also returns +a dictionary which maps from element id:s to elements. +\var{text} is a string containing XML data. + +\begin{datadescni}{Returns:} +A tuple containing an Element instance and a dictionary. +\end{datadescni} +\end{funcdesc} + + +\subsection{ElementTree Objects\label{elementtree-elementtree-objects}} + +\begin{classdesc}{ElementTree}{\optional{element,} \optional{file}} +ElementTree wrapper class. This class represents an entire element +hierarchy, and adds some extra support for serialization to and from +standard XML. + +\var{element} is the root element. +The tree is initialized with the contents of the XML \var{file} if given. +\end{classdesc} + +\begin{methoddesc}{_setroot}{element} +Replaces the root element for this tree. This discards the +current contents of the tree, and replaces it with the given +element. Use with care. +\var{element} is an element instance. +\end{methoddesc} + +\begin{methoddesc}{find}{path} +Finds the first toplevel element with given tag. +Same as getroot().find(path). +\var{path} is the element to look for. + +\begin{datadescni}{Returns:} +The first matching element, or None if no element was found. +\end{datadescni} +\end{methoddesc} + +\begin{methoddesc}{findall}{path} +Finds all toplevel elements with the given tag. +Same as getroot().findall(path). +\var{path} is the element to look for. + +\begin{datadescni}{Returns:} +A list or iterator containing all matching elements, +in section order. +\end{datadescni} +\end{methoddesc} + +\begin{methoddesc}{findtext}{path\optional{, default}} +Finds the element text for the first toplevel element with given +tag. Same as getroot().findtext(path). +\var{path} is the toplevel element to look for. +\var{default} is the value to return if the element was not found. + +\begin{datadescni}{Returns:} +The text content of the first matching element, or the +default value no element was found. Note that if the element +has is found, but has no text content, this method returns an +empty string. +\end{datadescni} +\end{methoddesc} + +\begin{methoddesc}{getiterator}{\optional{tag}} +Creates a tree iterator for the root element. The iterator loops +over all elements in this tree, in section order. +\var{tag} is the tag to look for (default is to return all elements) + +\begin{datadescni}{Returns:} +An iterator. +\end{datadescni} +\end{methoddesc} + +\begin{methoddesc}{getroot}{} +Gets the root element for this tree. + +\begin{datadescni}{Returns:} +An element instance. +\end{datadescni} +\end{methoddesc} + +\begin{methoddesc}{parse}{source\optional{, parser}} +Loads an external XML section into this element tree. +\var{source} is a file name or file object. +\var{parser} is an optional parser instance. If not given, the +standard XMLTreeBuilder parser is used. + +\begin{datadescni}{Returns:} +The section root element. +\end{datadescni} +\end{methoddesc} + +\begin{methoddesc}{write}{file\optional{, encoding}} +Writes the element tree to a file, as XML. +\var{file} is a file name, or a file object opened for writing. +\var{encoding} is the output encoding (default is US-ASCII). +\end{methoddesc} + + +\subsection{QName Objects\label{elementtree-qname-objects}} + +\begin{classdesc}{QName}{text_or_uri\optional{, tag}} +QName wrapper. This can be used to wrap a QName attribute value, in +order to get proper namespace handling on output. +\var{text_or_uri} is a string containing the QName value, +in the form {\{}uri{\}}local, or, if the tag argument is given, +the URI part of a QName. +If \var{tag} is given, the first argument is interpreted as +an URI, and this argument is interpreted as a local name. + +\begin{datadescni}{Returns:} +An opaque object, representing the QName. +\end{datadescni} +\end{classdesc} + + +\subsection{TreeBuilder Objects\label{elementtree-treebuilder-objects}} + +\begin{classdesc}{TreeBuilder}{\optional{element_factory}} +Generic element structure builder. This builder converts a sequence +of start, data, and end method calls to a well-formed element structure. +You can use this class to build an element structure using a custom XML +parser, or a parser for some other XML-like format. +The \var{element_factory} is called to create new Element instances when +given. +\end{classdesc} + +\begin{methoddesc}{close}{} +Flushes the parser buffers, and returns the toplevel documen +element. + +\begin{datadescni}{Returns:} +An Element instance. +\end{datadescni} +\end{methoddesc} + +\begin{methoddesc}{data}{data} +Adds text to the current element. +\var{data} is a string. This should be either an 8-bit string +containing ASCII text, or a Unicode string. +\end{methoddesc} + +\begin{methoddesc}{end}{tag} +Closes the current element. +\var{tag} is the element name. + +\begin{datadescni}{Returns:} +The closed element. +\end{datadescni} +\end{methoddesc} + +\begin{methoddesc}{start}{tag, attrs} +Opens a new element. +\var{tag} is the element name. +\var{attrs} is a dictionary containing element attributes. + +\begin{datadescni}{Returns:} +The opened element. +\end{datadescni} +\end{methoddesc} + + +\subsection{XMLTreeBuilder Objects\label{elementtree-xmltreebuilder-objects}} + +\begin{classdesc}{XMLTreeBuilder}{\optional{html,} \optional{target}} +Element structure builder for XML source data, based on the +expat parser. +\var{html} are predefined HTML entities. This flag is not supported +by the current implementation. +\var{target} is the target object. If omitted, the builder uses an +instance of the standard TreeBuilder class. +\end{classdesc} + +\begin{methoddesc}{close}{} +Finishes feeding data to the parser. + +\begin{datadescni}{Returns:} +An element structure. +\end{datadescni} +\end{methoddesc} + +\begin{methoddesc}{doctype}{name, pubid, system} +Handles a doctype declaration. +\var{name} is the doctype name. +\var{pubid} is the public identifier. +\var{system} is the system identifier. +\end{methoddesc} + +\begin{methoddesc}{feed}{data} +Feeds data to the parser. + +\var{data} is encoded data. +\end{methoddesc} diff --git a/Doc/lib/libfuncs.tex b/Doc/lib/libfuncs.tex index 8904d5f..65b0bf5 100644 --- a/Doc/lib/libfuncs.tex +++ b/Doc/lib/libfuncs.tex @@ -401,77 +401,27 @@ class C: \end{funcdesc} \begin{funcdesc}{file}{filename\optional{, mode\optional{, bufsize}}} - Return a new file object (described in - section~\ref{bltin-file-objects}, ``\ulink{File - Objects}{bltin-file-objects.html}''). - The first two arguments are the same as for \code{stdio}'s - \cfunction{fopen()}: \var{filename} is the file name to be opened, - \var{mode} indicates how the file is to be opened: \code{'r'} for - reading, \code{'w'} for writing (truncating an existing file), and - \code{'a'} opens it for appending (which on \emph{some} \UNIX{} - systems means that \emph{all} writes append to the end of the file, - regardless of the current seek position). - - Modes \code{'r+'}, \code{'w+'} and \code{'a+'} open the file for - updating (note that \code{'w+'} truncates the file). Append - \code{'b'} to the mode to open the file in binary mode, on systems - that differentiate between binary and text files (else it is - ignored). If the file cannot be opened, \exception{IOError} is - raised. - - In addition to the standard \cfunction{fopen()} values \var{mode} - may be \code{'U'} or \code{'rU'}. If Python is built with universal - newline support (the default) the file is opened as a text file, but - lines may be terminated by any of \code{'\e n'}, the Unix end-of-line - convention, - \code{'\e r'}, the Macintosh convention or \code{'\e r\e n'}, the Windows - convention. All of these external representations are seen as - \code{'\e n'} - by the Python program. If Python is built without universal newline support - \var{mode} \code{'U'} is the same as normal text mode. Note that - file objects so opened also have an attribute called - \member{newlines} which has a value of \code{None} (if no newlines - have yet been seen), \code{'\e n'}, \code{'\e r'}, \code{'\e r\e n'}, - or a tuple containing all the newline types seen. - - Python enforces that the mode, after stripping \code{'U'}, begins with - \code{'r'}, \code{'w'} or \code{'a'}. - - If \var{mode} is omitted, it defaults to \code{'r'}. When opening a - binary file, you should append \code{'b'} to the \var{mode} value - for improved portability. (It's useful even on systems which don't - treat binary and text files differently, where it serves as - documentation.) - \index{line-buffered I/O}\index{unbuffered I/O}\index{buffer size, I/O} - \index{I/O control!buffering} - The optional \var{bufsize} argument specifies the - file's desired buffer size: 0 means unbuffered, 1 means line - buffered, any other positive value means use a buffer of - (approximately) that size. A negative \var{bufsize} means to use - the system default, which is usually line buffered for tty - devices and fully buffered for other files. If omitted, the system - default is used.\footnote{ - Specifying a buffer size currently has no effect on systems that - don't have \cfunction{setvbuf()}. The interface to specify the - buffer size is not done using a method that calls - \cfunction{setvbuf()}, because that may dump core when called - after any I/O has been performed, and there's no reliable way to - determine whether this is the case.} + Constructor function for the \class{file} type, described further + in section~\ref{bltin-file-objects}, ``\ulink{File + Objects}{bltin-file-objects.html}''. The constructor's arguments + are the same as those of the \function{open()} built-in function + described below. + + When opening a file, it's preferable to use \function{open()} instead of + invoking this constructor directly. \class{file} is more suited to + type testing (for example, writing \samp{isinstance(f, file)}). \versionadded{2.2} - - \versionchanged[Restriction on first letter of mode string - introduced]{2.5} \end{funcdesc} \begin{funcdesc}{filter}{function, list} Construct a list from those elements of \var{list} for which \var{function} returns true. \var{list} may be either a sequence, a container which supports iteration, or an iterator, If \var{list} - is a string or a tuple, the result also has that type; otherwise it - is always a list. If \var{function} is \code{None}, the identity - function is assumed, that is, all elements of \var{list} that are false - (zero or empty) are removed. + is a string or a tuple, the result + also has that type; otherwise it is always a list. If \var{function} is + \code{None}, the identity function is assumed, that is, all elements of + \var{list} that are false are removed. Note that \code{filter(function, \var{list})} is equivalent to \code{[item for item in \var{list} if function(item)]} if function is @@ -709,10 +659,71 @@ class C: \end{funcdesc} \begin{funcdesc}{open}{filename\optional{, mode\optional{, bufsize}}} - A wrapper for the \function{file()} function above. The intent is - for \function{open()} to be preferred for use as a factory function - returning a new \class{file} object. \class{file} is more suited to - type testing (for example, writing \samp{isinstance(f, file)}). + Open a file, returning an object of the \class{file} type described + in section~\ref{bltin-file-objects}, ``\ulink{File + Objects}{bltin-file-objects.html}''. If the file cannot be opened, + \exception{IOError} is raised. When opening a file, it's + preferable to use \function{open()} instead of invoking the + \class{file} constructor directly. + + The first two arguments are the same as for \code{stdio}'s + \cfunction{fopen()}: \var{filename} is the file name to be opened, + and \var{mode} is a string indicating how the file is to be opened. + + The most commonly-used values of \var{mode} are \code{'r'} for + reading, \code{'w'} for writing (truncating the file if it already + exists), and \code{'a'} for appending (which on \emph{some} \UNIX{} + systems means that \emph{all} writes append to the end of the file + regardless of the current seek position). If \var{mode} is omitted, + it defaults to \code{'r'}. When opening a binary file, you should + append \code{'b'} to the \var{mode} value to open the file in binary + mode, which will improve portability. (Appending \code{'b'} is + useful even on systems that don't treat binary and text files + differently, where it serves as documentation.) See below for more + possible values of \var{mode}. + + \index{line-buffered I/O}\index{unbuffered I/O}\index{buffer size, I/O} + \index{I/O control!buffering} + The optional \var{bufsize} argument specifies the + file's desired buffer size: 0 means unbuffered, 1 means line + buffered, any other positive value means use a buffer of + (approximately) that size. A negative \var{bufsize} means to use + the system default, which is usually line buffered for tty + devices and fully buffered for other files. If omitted, the system + default is used.\footnote{ + Specifying a buffer size currently has no effect on systems that + don't have \cfunction{setvbuf()}. The interface to specify the + buffer size is not done using a method that calls + \cfunction{setvbuf()}, because that may dump core when called + after any I/O has been performed, and there's no reliable way to + determine whether this is the case.} + + Modes \code{'r+'}, \code{'w+'} and \code{'a+'} open the file for + updating (note that \code{'w+'} truncates the file). Append + \code{'b'} to the mode to open the file in binary mode, on systems + that differentiate between binary and text files; on systems + that don't have this distinction, adding the \code{'b'} has no effect. + + In addition to the standard \cfunction{fopen()} values \var{mode} + may be \code{'U'} or \code{'rU'}. Python is usually built with universal + newline support; supplying \code{'U'} opens the file as a text file, but + lines may be terminated by any of the following: the \UNIX{} end-of-line + convention \code{'\e n'}, + the Macintosh convention \code{'\e r'}, or the Windows + convention \code{'\e r\e n'}. All of these external representations are seen as + \code{'\e n'} + by the Python program. If Python is built without universal newline support + a \var{mode} with \code{'U'} is the same as normal text mode. Note that + file objects so opened also have an attribute called + \member{newlines} which has a value of \code{None} (if no newlines + have yet been seen), \code{'\e n'}, \code{'\e r'}, \code{'\e r\e n'}, + or a tuple containing all the newline types seen. + + Python enforces that the mode, after stripping \code{'U'}, begins with + \code{'r'}, \code{'w'} or \code{'a'}. + + \versionchanged[Restriction on first letter of mode string + introduced]{2.5} \end{funcdesc} \begin{funcdesc}{ord}{c} @@ -764,15 +775,30 @@ class C: \begin{verbatim} class C(object): def __init__(self): self.__x = None - def getx(self): return self.__x - def setx(self, value): self.__x = value - def delx(self): del self.__x + def getx(self): return self._x + def setx(self, value): self._x = value + def delx(self): del self._x x = property(getx, setx, delx, "I'm the 'x' property.") \end{verbatim} If given, \var{doc} will be the docstring of the property attribute. Otherwise, the property will copy \var{fget}'s docstring (if it - exists). + exists). This makes it possible to create read-only properties + easily using \function{property()} as a decorator: + +\begin{verbatim} +class Parrot(object): + def __init__(self): + self._voltage = 100000 + + @property + def voltage(self): + """Get the current voltage.""" + return self._voltage +\end{verbatim} + + turns the \method{voltage()} method into a ``getter'' for a read-only + attribute with the same name. \versionadded{2.2} \versionchanged[Use \var{fget}'s docstring if no \var{doc} given]{2.5} @@ -958,8 +984,30 @@ except NameError: \begin{funcdesc}{sorted}{iterable\optional{, cmp\optional{, key\optional{, reverse}}}} Return a new sorted list from the items in \var{iterable}. - The optional arguments \var{cmp}, \var{key}, and \var{reverse} - have the same meaning as those for the \method{list.sort()} method. + + The optional arguments \var{cmp}, \var{key}, and \var{reverse} have + the same meaning as those for the \method{list.sort()} method + (described in section~\ref{typesseq-mutable}). + + \var{cmp} specifies a custom comparison function of two arguments + (iterable elements) which should return a negative, zero or positive + number depending on whether the first argument is considered smaller + than, equal to, or larger than the second argument: + \samp{\var{cmp}=\keyword{lambda} \var{x},\var{y}: + \function{cmp}(x.lower(), y.lower())} + + \var{key} specifies a function of one argument that is used to + extract a comparison key from each list element: + \samp{\var{key}=\function{str.lower}} + + \var{reverse} is a boolean value. If set to \code{True}, then the + list elements are sorted as if each comparison were reversed. + + In general, the \var{key} and \var{reverse} conversion processes are + much faster than specifying an equivalent \var{cmp} function. This is + because \var{cmp} is called multiple times for each list element while + \var{key} and \var{reverse} touch each element only once. + \versionadded{2.4} \end{funcdesc} diff --git a/Doc/lib/libgettext.tex b/Doc/lib/libgettext.tex index e41f8bf..5c7c6b9 100644 --- a/Doc/lib/libgettext.tex +++ b/Doc/lib/libgettext.tex @@ -549,7 +549,7 @@ The \program{pygettext}\footnote{Fran\c cois Pinard has written a program called \program{xpot} which does a similar job. It is available as part of his \program{po-utils} package at -\url{http://www.iro.umontreal.ca/contrib/po-utils/HTML/}.} program +\url{http://po-utils.progiciels-bpi.ca/}.} program scans all your Python source code looking for the strings you previously marked as translatable. It is similar to the GNU \program{gettext} program except that it understands all the @@ -585,8 +585,8 @@ files are what the \module{gettext} module uses for the actual translation processing during run-time. How you use the \module{gettext} module in your code depends on -whether you are internationalizing your entire application or a single -module. +whether you are internationalizing a single module or your entire application. +The next two sections will discuss each case. \subsubsection{Localizing your module} diff --git a/Doc/lib/libimp.tex b/Doc/lib/libimp.tex index e0a775c..598d351 100644 --- a/Doc/lib/libimp.tex +++ b/Doc/lib/libimp.tex @@ -232,6 +232,24 @@ properly matching byte-compiled file (with suffix \file{.pyc} or source file. \end{funcdesc} +\begin{classdesc}{NullImporter}{path_string} +The \class{NullImporter} type is a \pep{302} import hook that handles +non-directory path strings by failing to find any modules. Calling this +type with an existing directory or empty string raises +\exception{ImportError}. Otherwise, a \class{NullImporter} instance is +returned. + +Python adds instances of this type to \code{sys.path_importer_cache} for +any path entries that are not directories and are not handled by any other +path hooks on \code{sys.path_hooks}. Instances have only one method: + +\begin{methoddesc}{find_module}{fullname \optional{, path}} +This method always returns \code{None}, indicating that the requested +module could not be found. +\end{methoddesc} + +\versionadded{2.5} +\end{classdesc} \subsection{Examples} \label{examples-imp} @@ -257,7 +275,7 @@ def __import__(name, globals=None, locals=None, fromlist=None): # there's a problem we can't handle -- let the caller handle it. fp, pathname, description = imp.find_module(name) - + try: return imp.load_module(name, fp, pathname, description) finally: diff --git a/Doc/lib/libinspect.tex b/Doc/lib/libinspect.tex index 5cabb80..85651f0 100644 --- a/Doc/lib/libinspect.tex +++ b/Doc/lib/libinspect.tex @@ -180,13 +180,32 @@ Note: Return true if the object is a data descriptor. Data descriptors have both a __get__ and a __set__ attribute. Examples are - properties (defined in Python) and getsets and members (defined in C). - Typically, data descriptors will also have __name__ and __doc__ attributes - (properties, getsets, and members have both of these attributes), but this - is not guaranteed. + properties (defined in Python), getsets, and members. The latter two are + defined in C and there are more specific tests available for those types, + which is robust across Python implementations. Typically, data descriptors + will also have __name__ and __doc__ attributes (properties, getsets, and + members have both of these attributes), but this is not guaranteed. \versionadded{2.3} \end{funcdesc} +\begin{funcdesc}{isgetsetdescriptor}{object} + Return true if the object is a getset descriptor. + + getsets are attributes defined in extension modules via \code{PyGetSetDef} + structures. For Python implementations without such types, this method will + always return \code{False}. +\versionadded{2.5} +\end{funcdesc} + +\begin{funcdesc}{ismemberdescriptor}{object} + Return true if the object is a member descriptor. + + Member descriptors are attributes defined in extension modules via + \code{PyMemberDef} structures. For Python implementations without such + types, this method will always return \code{False}. +\versionadded{2.5} +\end{funcdesc} + \subsection{Retrieving source code \label{inspect-source}} @@ -272,18 +291,18 @@ Note: \end{funcdesc} \begin{funcdesc}{formatargspec}{args\optional{, varargs, varkw, defaults, - argformat, varargsformat, varkwformat, defaultformat}} + formatarg, formatvarargs, formatvarkw, formatvalue, join}} Format a pretty argument spec from the four values returned by - \function{getargspec()}. The other four arguments are the + \function{getargspec()}. The format* arguments are the corresponding optional formatting functions that are called to turn names and values into strings. \end{funcdesc} \begin{funcdesc}{formatargvalues}{args\optional{, varargs, varkw, locals, - argformat, varargsformat, varkwformat, valueformat}} + formatarg, formatvarargs, formatvarkw, formatvalue, join}} Format a pretty argument spec from the four values returned by - \function{getargvalues()}. The other four arguments are the + \function{getargvalues()}. The format* arguments are the corresponding optional formatting functions that are called to turn names and values into strings. \end{funcdesc} diff --git a/Doc/lib/liblinecache.tex b/Doc/lib/liblinecache.tex index 1477d3c..72c7743 100644 --- a/Doc/lib/liblinecache.tex +++ b/Doc/lib/liblinecache.tex @@ -38,7 +38,7 @@ files previously read using \function{getline()}. \begin{funcdesc}{checkcache}{\optional{filename}} Check the cache for validity. Use this function if files in the cache may have changed on disk, and you require the updated version. If -\var{filename} is omitted, it will check the whole cache entries. +\var{filename} is omitted, it will check all the entries in the cache. \end{funcdesc} Example: diff --git a/Doc/lib/liblogging.tex b/Doc/lib/liblogging.tex index 576e2e7..cc44294 100644 --- a/Doc/lib/liblogging.tex +++ b/Doc/lib/liblogging.tex @@ -1068,13 +1068,11 @@ list of possible values is, note that they are not case sensitive: \end{tableii} If \var{backupCount} is non-zero, the system will save old log files by -appending the extensions ".1", ".2" etc., to the filename. For example, -with a \var{backupCount} of 5 and a base file name of \file{app.log}, -you would get \file{app.log}, \file{app.log.1}, \file{app.log.2}, up to -\file{app.log.5}. The file being written to is always \file{app.log}. -When this file is filled, it is closed and renamed to \file{app.log.1}, -and if files \file{app.log.1}, \file{app.log.2}, etc. exist, then they -are renamed to \file{app.log.2}, \file{app.log.3} etc. respectively. +appending extensions to the filename. The extensions are date-and-time +based, using the strftime format \code{\%Y-\%m-\%d_\%H-\%M-\%S} or a leading +portion thereof, depending on the rollover interval. At most \var{backupCount} +files will be kept, and if more would be created when rollover occurs, the +oldest one is deleted. \end{classdesc} \begin{methoddesc}{doRollover}{} @@ -1539,7 +1537,7 @@ to start the server, and which you can \method{join()} when appropriate. To stop the server, call \function{stopListening()}. To send a configuration to the socket, read in the configuration file and send it to the socket as a string of bytes preceded by a four-byte length packed in binary using -struct.\code{pack(">L", n)}. +struct.\code{pack('>L', n)}. \end{funcdesc} \begin{funcdesc}{stopListening}{} diff --git a/Doc/lib/libmailbox.tex b/Doc/lib/libmailbox.tex index 0a1f792..75ea7e1 100644 --- a/Doc/lib/libmailbox.tex +++ b/Doc/lib/libmailbox.tex @@ -1367,9 +1367,8 @@ for message in mailbox.mbox('~/mbox'): print subject \end{verbatim} -A (surprisingly) simple example of copying all mail from a Babyl mailbox to an -MH mailbox, converting all of the format-specific information that can be -converted: +To copy all mail from a Babyl mailbox to an MH mailbox, converting all +of the format-specific information that can be converted: \begin{verbatim} import mailbox diff --git a/Doc/lib/libmimetypes.tex b/Doc/lib/libmimetypes.tex index 8e07768..6c46d6f 100644 --- a/Doc/lib/libmimetypes.tex +++ b/Doc/lib/libmimetypes.tex @@ -158,6 +158,20 @@ want more than one MIME-type database: \versionadded{2.2} \end{classdesc} +An example usage of the module: + +\begin{verbatim} +>>> import mimetypes +>>> mimetypes.init() +>>> mimetypes.knownfiles +['/etc/mime.types', '/etc/httpd/mime.types', ... ] +>>> mimetypes.suffix_map['.tgz'] +'.tar.gz' +>>> mimetypes.encodings_map['.gz'] +'gzip' +>>> mimetypes.types_map['.tgz'] +'application/x-tar-gz' +\end{verbatim} \subsection{MimeTypes Objects \label{mimetypes-objects}} diff --git a/Doc/lib/libnew.tex b/Doc/lib/libnew.tex index e3f2a49..18162dc 100644 --- a/Doc/lib/libnew.tex +++ b/Doc/lib/libnew.tex @@ -22,13 +22,16 @@ unbound if \var{instance} is \code{None}. \var{function} must be callable. \end{funcdesc} -\begin{funcdesc}{function}{code, globals\optional{, name\optional{, argdefs}}} +\begin{funcdesc}{function}{code, globals\optional{, name\optional{, + argdefs\optional{, closure}}}} Returns a (Python) function with the given code and globals. If \var{name} is given, it must be a string or \code{None}. If it is a string, the function will have the given name, otherwise the function name will be taken from \code{\var{code}.co_name}. If \var{argdefs} is given, it must be a tuple and will be used to -determine the default values of parameters. +determine the default values of parameters. If \var{closure} is given, +it must be \code{None} or a tuple of cell objects containing objects +to bind to the names in \code{\var{code}.co_freevars}. \end{funcdesc} \begin{funcdesc}{code}{argcount, nlocals, stacksize, flags, codestring, diff --git a/Doc/lib/liboptparse.tex b/Doc/lib/liboptparse.tex index ec43e3d..df96dd4 100644 --- a/Doc/lib/liboptparse.tex +++ b/Doc/lib/liboptparse.tex @@ -1,3 +1,5 @@ +% THIS FILE IS AUTO-GENERATED! DO NOT EDIT! +% (Your changes will be lost the next time it is generated.) \section{\module{optparse} --- More powerful command line option parser} \declaremodule{standard}{optparse} \moduleauthor{Greg Ward}{gward@python.net} @@ -306,7 +308,7 @@ Of these, \member{action} is the most fundamental. Actions tell \module{optparse} what to do when it encounters an option on the command line. There is a fixed set of actions hard-coded into \module{optparse}; -adding new actions is an advanced topic covered in section~\ref{optparse-extending}, Extending \module{optparse}. +adding new actions is an advanced topic covered in section~\ref{optparse-extending-optparse}, Extending \module{optparse}. Most actions tell \module{optparse} to store a value in some variable{---}for example, take a string from the command line and store it in an attribute of \code{options}. @@ -371,7 +373,7 @@ are no long option strings, \module{optparse} looks at the first short option string: the default destination for \code{"-f"} is \code{f}. \module{optparse} also includes built-in \code{long} and \code{complex} types. Adding -types is covered in section~\ref{optparse-extending}, Extending \module{optparse}. +types is covered in section~\ref{optparse-extending-optparse}, Extending \module{optparse}. \subsubsection{Handling boolean (flag) options\label{optparse-handling-boolean-options}} @@ -566,7 +568,7 @@ argument to OptionParser: parser = OptionParser(usage="%prog [-f] [-q]", version="%prog 1.0") \end{verbatim} -Note that \code{"{\%}prog"} is expanded just like it is in \code{usage}. Apart +\code{"{\%}prog"} is expanded just like it is in \code{usage}. Apart from that, \code{version} can contain anything you like. When you supply it, \module{optparse} automatically adds a \code{"-{}-version"} option to your parser. If it encounters this option on the command line, it expands your @@ -659,7 +661,7 @@ def main(): if __name__ == "__main__": main() \end{verbatim} -% $Id: tutorial.txt 505 2005-07-22 01:52:40Z gward $ +% $Id: tutorial.txt 515 2006-06-10 15:37:45Z gward $ \subsection{Reference Guide\label{optparse-reference-guide}} @@ -1146,7 +1148,7 @@ See section~\ref{optparse-tutorial}, the tutorial for an example. \module{optparse} has six built-in option types: \code{string}, \code{int}, \code{long}, \code{choice}, \code{float} and \code{complex}. If you need to add new option -types, see section~\ref{optparse-extending}, Extending \module{optparse}. +types, see section~\ref{optparse-extending-optparse}, Extending \module{optparse}. Arguments to string options are not checked or converted in any way: the text on the command line is stored in the destination (or passed to the @@ -1195,16 +1197,16 @@ its \method{parse{\_}args()} method: where the input parameters are \begin{description} \item[\code{args}] -the list of arguments to process (\code{sys.argv{[}1:]} by default) +the list of arguments to process (default: \code{sys.argv{[}1:]}) \item[\code{options}] -object to store option arguments in (a new instance of -optparse.Values by default) +object to store option arguments in (default: a new instance of +optparse.Values) \end{description} and the return values are \begin{description} \item[\code{options}] -the same object as was passed in as \code{options}, or the new +the same object that was passed in as \code{options}, or the optparse.Values instance created by \module{optparse} \item[\code{args}] the leftover positional arguments after all options have been @@ -1212,9 +1214,9 @@ processed \end{description} The most common usage is to supply neither keyword argument. If you -supply a \code{values} object, it will be repeatedly modified with a -\code{setattr()} call for every option argument written to an option -destination, and finally returned by \method{parse{\_}args()}. +supply \code{options}, it will be modified with repeated \code{setattr()} +calls (roughly one for every option argument stored to an option +destination) and returned by \method{parse{\_}args()}. If \method{parse{\_}args()} encounters any errors in the argument list, it calls the OptionParser's \method{error()} method with an appropriate end-user error @@ -1388,7 +1390,7 @@ parser.add_option("--novice", action="store_const", \end{verbatim} \end{itemize} -% $Id: reference.txt 505 2005-07-22 01:52:40Z gward $ +% $Id: reference.txt 519 2006-06-11 14:39:11Z gward $ \subsection{Option Callbacks\label{optparse-option-callbacks}} @@ -1681,3 +1683,206 @@ further options (probably causing an error), rather than as arguments to \code{"-c"}. Fixing this is left as an exercise for the reader. % $Id: callbacks.txt 415 2004-09-30 02:26:17Z greg $ + +\subsection{Extending \module{optparse}\label{optparse-extending-optparse}} + +Since the two major controlling factors in how \module{optparse} interprets +command-line options are the action and type of each option, the most +likely direction of extension is to add new actions and new types. + + +\subsubsection{Adding new types\label{optparse-adding-new-types}} + +To add new types, you need to define your own subclass of \module{optparse}'s Option +class. This class has a couple of attributes that define \module{optparse}'s types: +\member{TYPES} and \member{TYPE{\_}CHECKER}. + +\member{TYPES} is a tuple of type names; in your subclass, simply define a new +tuple \member{TYPES} that builds on the standard one. + +\member{TYPE{\_}CHECKER} is a dictionary mapping type names to type-checking +functions. A type-checking function has the following signature: +\begin{verbatim} +def check_mytype(option, opt, value) +\end{verbatim} + +where \code{option} is an \class{Option} instance, \code{opt} is an option string +(e.g., \code{"-f"}), and \code{value} is the string from the command line that +must be checked and converted to your desired type. \code{check{\_}mytype()} +should return an object of the hypothetical type \code{mytype}. The value +returned by a type-checking function will wind up in the OptionValues +instance returned by \method{OptionParser.parse{\_}args()}, or be passed to a +callback as the \code{value} parameter. + +Your type-checking function should raise OptionValueError if it +encounters any problems. OptionValueError takes a single string +argument, which is passed as-is to OptionParser's \method{error()} method, +which in turn prepends the program name and the string \code{"error:"} and +prints everything to stderr before terminating the process. + +Here's a silly example that demonstrates adding a \code{complex} option +type to parse Python-style complex numbers on the command line. (This +is even sillier than it used to be, because \module{optparse} 1.3 added built-in +support for complex numbers, but never mind.) + +First, the necessary imports: +\begin{verbatim} +from copy import copy +from optparse import Option, OptionValueError +\end{verbatim} + +You need to define your type-checker first, since it's referred to later +(in the \member{TYPE{\_}CHECKER} class attribute of your Option subclass): +\begin{verbatim} +def check_complex(option, opt, value): + try: + return complex(value) + except ValueError: + raise OptionValueError( + "option %s: invalid complex value: %r" % (opt, value)) +\end{verbatim} + +Finally, the Option subclass: +\begin{verbatim} +class MyOption (Option): + TYPES = Option.TYPES + ("complex",) + TYPE_CHECKER = copy(Option.TYPE_CHECKER) + TYPE_CHECKER["complex"] = check_complex +\end{verbatim} + +(If we didn't make a \function{copy()} of \member{Option.TYPE{\_}CHECKER}, we would end +up modifying the \member{TYPE{\_}CHECKER} attribute of \module{optparse}'s Option class. +This being Python, nothing stops you from doing that except good manners +and common sense.) + +That's it! Now you can write a script that uses the new option type +just like any other \module{optparse}-based script, except you have to instruct your +OptionParser to use MyOption instead of Option: +\begin{verbatim} +parser = OptionParser(option_class=MyOption) +parser.add_option("-c", type="complex") +\end{verbatim} + +Alternately, you can build your own option list and pass it to +OptionParser; if you don't use \method{add{\_}option()} in the above way, you +don't need to tell OptionParser which option class to use: +\begin{verbatim} +option_list = [MyOption("-c", action="store", type="complex", dest="c")] +parser = OptionParser(option_list=option_list) +\end{verbatim} + + +\subsubsection{Adding new actions\label{optparse-adding-new-actions}} + +Adding new actions is a bit trickier, because you have to understand +that \module{optparse} has a couple of classifications for actions: +\begin{description} +\item[``store'' actions] +actions that result in \module{optparse} storing a value to an attribute of the +current OptionValues instance; these options require a \member{dest} +attribute to be supplied to the Option constructor +\item[``typed'' actions] +actions that take a value from the command line and expect it to be +of a certain type; or rather, a string that can be converted to a +certain type. These options require a \member{type} attribute to the +Option constructor. +\end{description} + +These are overlapping sets: some default ``store'' actions are \code{store}, +\code{store{\_}const}, \code{append}, and \code{count}, while the default ``typed'' +actions are \code{store}, \code{append}, and \code{callback}. + +When you add an action, you need to categorize it by listing it in at +least one of the following class attributes of Option (all are lists of +strings): +\begin{description} +\item[\member{ACTIONS}] +all actions must be listed in ACTIONS +\item[\member{STORE{\_}ACTIONS}] +``store'' actions are additionally listed here +\item[\member{TYPED{\_}ACTIONS}] +``typed'' actions are additionally listed here +\item[\code{ALWAYS{\_}TYPED{\_}ACTIONS}] +actions that always take a type (i.e. whose options always take a +value) are additionally listed here. The only effect of this is +that \module{optparse} assigns the default type, \code{string}, to options with no +explicit type whose action is listed in \code{ALWAYS{\_}TYPED{\_}ACTIONS}. +\end{description} + +In order to actually implement your new action, you must override +Option's \method{take{\_}action()} method and add a case that recognizes your +action. + +For example, let's add an \code{extend} action. This is similar to the +standard \code{append} action, but instead of taking a single value from +the command-line and appending it to an existing list, \code{extend} will +take multiple values in a single comma-delimited string, and extend an +existing list with them. That is, if \code{"-{}-names"} is an \code{extend} +option of type \code{string}, the command line +\begin{verbatim} +--names=foo,bar --names blah --names ding,dong +\end{verbatim} + +would result in a list +\begin{verbatim} +["foo", "bar", "blah", "ding", "dong"] +\end{verbatim} + +Again we define a subclass of Option: +\begin{verbatim} +class MyOption (Option): + + ACTIONS = Option.ACTIONS + ("extend",) + STORE_ACTIONS = Option.STORE_ACTIONS + ("extend",) + TYPED_ACTIONS = Option.TYPED_ACTIONS + ("extend",) + ALWAYS_TYPED_ACTIONS = Option.ALWAYS_TYPED_ACTIONS + ("extend",) + + def take_action(self, action, dest, opt, value, values, parser): + if action == "extend": + lvalue = value.split(",") + values.ensure_value(dest, []).extend(lvalue) + else: + Option.take_action( + self, action, dest, opt, value, values, parser) +\end{verbatim} + +Features of note: +\begin{itemize} +\item {} +\code{extend} both expects a value on the command-line and stores that +value somewhere, so it goes in both \member{STORE{\_}ACTIONS} and +\member{TYPED{\_}ACTIONS} + +\item {} +to ensure that \module{optparse} assigns the default type of \code{string} to +\code{extend} actions, we put the \code{extend} action in +\code{ALWAYS{\_}TYPED{\_}ACTIONS} as well + +\item {} +\method{MyOption.take{\_}action()} implements just this one new action, and +passes control back to \method{Option.take{\_}action()} for the standard +\module{optparse} actions + +\item {} +\code{values} is an instance of the optparse{\_}parser.Values class, +which provides the very useful \method{ensure{\_}value()} method. +\method{ensure{\_}value()} is essentially \function{getattr()} with a safety valve; +it is called as +\begin{verbatim} +values.ensure_value(attr, value) +\end{verbatim} + +If the \code{attr} attribute of \code{values} doesn't exist or is None, then +ensure{\_}value() first sets it to \code{value}, and then returns 'value. +This is very handy for actions like \code{extend}, \code{append}, and +\code{count}, all of which accumulate data in a variable and expect that +variable to be of a certain type (a list for the first two, an integer +for the latter). Using \method{ensure{\_}value()} means that scripts using +your action don't have to worry about setting a default value for the +option destinations in question; they can just leave the default as +None and \method{ensure{\_}value()} will take care of getting it right when +it's needed. + +\end{itemize} +% $Id: extending.txt 517 2006-06-10 16:18:11Z gward $ + diff --git a/Doc/lib/libossaudiodev.tex b/Doc/lib/libossaudiodev.tex index 223cf28..4c19aaf 100644 --- a/Doc/lib/libossaudiodev.tex +++ b/Doc/lib/libossaudiodev.tex @@ -68,7 +68,7 @@ raises \exception{IOError}. Errors detected directly by Open an audio device and return an OSS audio device object. This object supports many file-like methods, such as \method{read()}, \method{write()}, and \method{fileno()} (although there are subtle -differences between conventional Unix read/write semantics and those of +differences between conventional \UNIX{} read/write semantics and those of OSS audio devices). It also supports a number of audio-specific methods; see below for the complete list of methods. diff --git a/Doc/lib/libpickle.tex b/Doc/lib/libpickle.tex index 45e80b8..a8ab39e 100644 --- a/Doc/lib/libpickle.tex +++ b/Doc/lib/libpickle.tex @@ -725,7 +725,50 @@ source of the strings your application unpickles. \subsection{Example \label{pickle-example}} -Here's a simple example of how to modify pickling behavior for a +For the simplest code, use the \function{dump()} and \function{load()} +functions. Note that a self-referencing list is pickled and restored +correctly. + +\begin{verbatim} +import pickle + +data1 = {'a': [1, 2.0, 3, 4+6j], + 'b': ('string', u'Unicode string'), + 'c': None} + +selfref_list = [1, 2, 3] +selfref_list.append(selfref_list) + +output = open('data.pkl', 'wb') + +# Pickle dictionary using protocol 0. +pickle.dump(data1, output) + +# Pickle the list using the highest protocol available. +pickle.dump(selfref_list, output, -1) + +output.close() +\end{verbatim} + +The following example reads the resulting pickled data. When reading +a pickle-containing file, you should open the file in binary mode +because you can't be sure if the ASCII or binary format was used. + +\begin{verbatim} +import pprint, pickle + +pkl_file = open('data.pkl', 'rb') + +data1 = pickle.load(pkl_file) +pprint.pprint(data1) + +data2 = pickle.load(pkl_file) +pprint.pprint(data2) + +pkl_file.close() +\end{verbatim} + +Here's a larger example that shows how to modify pickling behavior for a class. The \class{TextReader} class opens a text file, and returns the line number and line contents each time its \method{readline()} method is called. If a \class{TextReader} instance is pickled, all diff --git a/Doc/lib/libpkgutil.tex b/Doc/lib/libpkgutil.tex index 15d866b..a286f00 100644 --- a/Doc/lib/libpkgutil.tex +++ b/Doc/lib/libpkgutil.tex @@ -30,7 +30,7 @@ __path__ = extend_path(__path__, __name__) with \code{import}. A \file{*.pkg} file is trusted at face value: apart from checking for duplicates, all entries found in a \file{*.pkg} file are added to the path, regardless of whether they - exist the filesystem. (This is a feature.) + exist on the filesystem. (This is a feature.) If the input path is not a list (as is the case for frozen packages) it is returned unchanged. The input path is not diff --git a/Doc/lib/libposixpath.tex b/Doc/lib/libposixpath.tex index 9f0de1f..0b2da66 100644 --- a/Doc/lib/libposixpath.tex +++ b/Doc/lib/libposixpath.tex @@ -42,8 +42,11 @@ half of the pair returned by \code{split(\var{path})}. \end{funcdesc} \begin{funcdesc}{exists}{path} -Return \code{True} if \var{path} refers to an existing path. -Returns \code{False} for broken symbolic links. +Return \code{True} if \var{path} refers to an existing path. Returns +\code{False} for broken symbolic links. On some platforms, this +function may return \code{False} if permission is not granted to +execute \function{os.stat()} on the requested file, even if the +\var{path} physically exists. \end{funcdesc} \begin{funcdesc}{lexists}{path} @@ -190,9 +193,8 @@ Availability: Macintosh, \UNIX. \end{funcdesc} \begin{funcdesc}{sameopenfile}{fp1, fp2} -Return \code{True} if the file objects \var{fp1} and \var{fp2} refer to the -same file. The two file objects may represent different file -descriptors. +Return \code{True} if the file descriptors \var{fp1} and \var{fp2} refer +to the same file. Availability: Macintosh, \UNIX. \end{funcdesc} diff --git a/Doc/lib/librandom.tex b/Doc/lib/librandom.tex index 6c2f710..c6b8846 100644 --- a/Doc/lib/librandom.tex +++ b/Doc/lib/librandom.tex @@ -236,7 +236,7 @@ these equations can be found in any statistics text. \var{beta} is the shape parameter. \end{funcdesc} -Alternative Generators +Alternative Generators: \begin{classdesc}{WichmannHill}{\optional{seed}} Class that implements the Wichmann-Hill algorithm as the core generator. @@ -267,6 +267,30 @@ called. \versionadded{2.4} \end{classdesc} +Examples of basic usage: + +\begin{verbatim} +>>> random.random() # Random float x, 0.0 <= x < 1.0 +0.37444887175646646 +>>> random.uniform(1, 10) # Random float x, 1.0 <= x < 10.0 +1.1800146073117523 +>>> random.randint(1, 10) # Integer from 1 to 10, endpoints included +7 +>>> random.randrange(0, 101, 2) # Even integer from 0 to 100 +26 +>>> random.choice('abcdefghij') # Choose a random element +'c' + +>>> items = [1, 2, 3, 4, 5, 6, 7] +>>> random.shuffle(items) +>>> items +[7, 3, 2, 5, 6, 4, 1] + +>>> random.sample([1, 2, 3, 4, 5], 3) # Choose 3 elements +[4, 1, 5] + +\end{verbatim} + \begin{seealso} \seetext{M. Matsumoto and T. Nishimura, ``Mersenne Twister: A 623-dimensionally equidistributed uniform pseudorandom diff --git a/Doc/lib/libre.tex b/Doc/lib/libre.tex index 1404e09..84e382d 100644 --- a/Doc/lib/libre.tex +++ b/Doc/lib/libre.tex @@ -897,7 +897,7 @@ offers some more-or-less equivalent mappings between \lineii{\code{\%d}} {\regexp{[-+]?\e d+}} \lineii{\code{\%e}, \code{\%E}, \code{\%f}, \code{\%g}} - {\regexp{[-+]?(\e d+(\e.\e d*)?|\e d*\e.\e d+)([eE][-+]?\e d+)?}} + {\regexp{[-+]?(\e d+(\e.\e d*)?|\e.\e d+)([eE][-+]?\e d+)?}} \lineii{\code{\%i}} {\regexp{[-+]?(0[xX][\e dA-Fa-f]+|0[0-7]*|\e d+)}} \lineii{\code{\%o}} diff --git a/Doc/lib/libreadline.tex b/Doc/lib/libreadline.tex index ac8e23f..dec37b6 100644 --- a/Doc/lib/libreadline.tex +++ b/Doc/lib/libreadline.tex @@ -7,10 +7,13 @@ \modulesynopsis{GNU readline support for Python.} -The \module{readline} module defines a number of functions used either -directly or from the \refmodule{rlcompleter} module to facilitate -completion and history file read and write from the Python -interpreter. +The \module{readline} module defines a number of functions to +facilitate completion and reading/writing of history files from the +Python interpreter. This module can be used directly or via the +\refmodule{rlcompleter} module. Settings made using +this module affect the behaviour of both the interpreter's interactive prompt +and the prompts offered by the \function{raw_input()} and \function{input()} +built-in functions. The \module{readline} module defines the following functions: diff --git a/Doc/lib/libsgmllib.tex b/Doc/lib/libsgmllib.tex index 3ec1018..1fe0d63 100644 --- a/Doc/lib/libsgmllib.tex +++ b/Doc/lib/libsgmllib.tex @@ -132,27 +132,59 @@ nothing. \begin{methoddesc}{handle_charref}{ref} This method is called to process a character reference of the form -\samp{\&\#\var{ref};}. In the base implementation, \var{ref} must -be a decimal number in the -range 0-255. It translates the character to \ASCII{} and calls the -method \method{handle_data()} with the character as argument. If -\var{ref} is invalid or out of range, the method -\code{unknown_charref(\var{ref})} is called to handle the error. A -subclass must override this method to provide support for named -character entities. +\samp{\&\#\var{ref};}. The base implementation uses +\method{convert_charref()} to convert the reference to a string. If +that method returns a string, it is passed to \method{handle_data()}, +otherwise \method{unknown_charref(\var{ref})} is called to handle the +error. +\versionchanged[Use \method{convert_charref()} instead of hard-coding +the conversion]{2.5} +\end{methoddesc} + +\begin{methoddesc}{convert_charref}{ref} +Convert a character reference to a string, or \code{None}. \var{ref} +is the reference passed in as a string. In the base implementation, +\var{ref} must be a decimal number in the range 0-255. It converts +the code point found using the \method{convert_codepoint()} method. +If \var{ref} is invalid or out of range, this method returns +\code{None}. This method is called by the default +\method{handle_charref()} implementation and by the attribute value +parser. +\versionadded{2.5} +\end{methoddesc} + +\begin{methoddesc}{convert_codepoint}{codepoint} +Convert a codepoint to a \class{str} value. Encodings can be handled +here if appropriate, though the rest of \module{sgmllib} is oblivious +on this matter. +\versionadded{2.5} \end{methoddesc} \begin{methoddesc}{handle_entityref}{ref} This method is called to process a general entity reference of the form \samp{\&\var{ref};} where \var{ref} is an general entity -reference. It looks for \var{ref} in the instance (or class) -variable \member{entitydefs} which should be a mapping from entity -names to corresponding translations. If a translation is found, it +reference. It converts \var{ref} by passing it to +\method{convert_entityref()}. If a translation is returned, it calls the method \method{handle_data()} with the translation; otherwise, it calls the method \code{unknown_entityref(\var{ref})}. The default \member{entitydefs} defines translations for \code{\&}, \code{\&apos}, \code{\>}, \code{\<}, and \code{\"}. +\versionchanged[Use \method{convert_entityref()} instead of hard-coding +the conversion]{2.5} +\end{methoddesc} + +\begin{methoddesc}{convert_entityref}{ref} +Convert a named entity reference to a \class{str} value, or +\code{None}. The resulting value will not be parsed. \var{ref} will +be only the name of the entity. The default implementation looks for +\var{ref} in the instance (or class) variable \member{entitydefs} +which should be a mapping from entity names to corresponding +translations. If no translation is available for \var{ref}, this +method returns \code{None}. This method is called by the default +\method{handle_entityref()} implementation and by the attribute value +parser. +\versionadded{2.5} \end{methoddesc} \begin{methoddesc}{handle_comment}{comment} diff --git a/Doc/lib/libshelve.tex b/Doc/lib/libshelve.tex index 8bd204e..6ca3576 100644 --- a/Doc/lib/libshelve.tex +++ b/Doc/lib/libshelve.tex @@ -143,15 +143,17 @@ data = d[key] # retrieve a COPY of data at key (raise KeyError if no del d[key] # delete data stored at key (raises KeyError # if no such key) flag = d.has_key(key) # true if the key exists -list = d.keys() # a list of all existing keys (slow!) +klist = d.keys() # a list of all existing keys (slow!) # as d was opened WITHOUT writeback=True, beware: d['xx'] = range(4) # this works as expected, but... d['xx'].append(5) # *this doesn't!* -- d['xx'] is STILL range(4)!!! + # having opened d without writeback=True, you need to code carefully: temp = d['xx'] # extracts the copy temp.append(5) # mutates the copy d['xx'] = temp # stores the copy right back, to persist it + # or, d=shelve.open(filename,writeback=True) would let you just code # d['xx'].append(5) and have it work as expected, BUT it would also # consume more memory and make the d.close() operation slower. diff --git a/Doc/lib/libsite.tex b/Doc/lib/libsite.tex index 43b8db2..c079790 100644 --- a/Doc/lib/libsite.tex +++ b/Doc/lib/libsite.tex @@ -16,12 +16,13 @@ search path. It starts by constructing up to four directories from a head and a tail part. For the head part, it uses \code{sys.prefix} and \code{sys.exec_prefix}; empty heads are skipped. For -the tail part, it uses the empty string (on Windows) or -\file{lib/python\shortversion/site-packages} (on \UNIX{} and Macintosh) -and then \file{lib/site-python}. For each of the distinct -head-tail combinations, it sees if it refers to an existing directory, -and if so, adds it to \code{sys.path} and also inspects the newly added -path for configuration files. +the tail part, it uses the empty string and then +\file{lib/site-packages} (on Windows) or +\file{lib/python\shortversion/site-packages} and then +\file{lib/site-python} (on \UNIX{} and Macintosh). For each of the +distinct head-tail combinations, it sees if it refers to an existing +directory, and if so, adds it to \code{sys.path} and also inspects +the newly added path for configuration files. \indexii{site-python}{directory} \indexii{site-packages}{directory} diff --git a/Doc/lib/libsocket.tex b/Doc/lib/libsocket.tex index 8066528..aa75ec9 100644 --- a/Doc/lib/libsocket.tex +++ b/Doc/lib/libsocket.tex @@ -711,6 +711,17 @@ If \var{n} is provided, read \var{n} bytes from the SSL connection, otherwise read until EOF. The return value is a string of the bytes read. \end{methoddesc} +\begin{methoddesc}{server}{} +Returns a string containing the ASN.1 distinguished name identifying the +server's certificate. (See below for an example +showing what distinguished names look like.) +\end{methoddesc} + +\begin{methoddesc}{issuer}{} +Returns a string containing the ASN.1 distinguished name identifying the +issuer of the server's certificate. +\end{methoddesc} + \subsection{Example \label{socket-example}} Here are four minimal example programs using the TCP/IP protocol:\ a @@ -833,3 +844,44 @@ data = s.recv(1024) s.close() print 'Received', repr(data) \end{verbatim} + +This example connects to an SSL server, prints the +server and issuer's distinguished names, sends some bytes, +and reads part of the response: + +\begin{verbatim} +import socket + +s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) +s.connect(('www.verisign.com', 443)) + +ssl_sock = socket.ssl(s) + +print repr(ssl_sock.server()) +print repr(ssl_sock.issuer()) + +# Set a simple HTTP request -- use httplib in actual code. +ssl_sock.write("""GET / HTTP/1.0\r +Host: www.verisign.com\r\n\r\n""") + +# Read a chunk of data. Will not necessarily +# read all the data returned by the server. +data = ssl_sock.read() + +# Note that you need to close the underlying socket, not the SSL object. +del ssl_sock +s.close() +\end{verbatim} + +At this writing, this SSL example prints the following output (line +breaks inserted for readability): + +\begin{verbatim} +'/C=US/ST=California/L=Mountain View/ + O=VeriSign, Inc./OU=Production Services/ + OU=Terms of use at www.verisign.com/rpa (c)00/ + CN=www.verisign.com' +'/O=VeriSign Trust Network/OU=VeriSign, Inc./ + OU=VeriSign International Server CA - Class 3/ + OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign' +\end{verbatim} diff --git a/Doc/lib/libsocksvr.tex b/Doc/lib/libsocksvr.tex index b21e804..c7b28ea 100644 --- a/Doc/lib/libsocksvr.tex +++ b/Doc/lib/libsocksvr.tex @@ -74,9 +74,9 @@ synchronous servers of four types: \end{verbatim} Note that \class{UnixDatagramServer} derives from \class{UDPServer}, not -from \class{UnixStreamServer} -- the only difference between an IP and a -Unix stream server is the address family, which is simply repeated in both -unix server classes. +from \class{UnixStreamServer} --- the only difference between an IP and a +\UNIX{} stream server is the address family, which is simply repeated in both +\UNIX{} server classes. Forking and threading versions of each type of server can be created using the \class{ForkingMixIn} and \class{ThreadingMixIn} mix-in classes. For diff --git a/Doc/lib/libsqlite3.tex b/Doc/lib/libsqlite3.tex index db15c00..d87e064 100644 --- a/Doc/lib/libsqlite3.tex +++ b/Doc/lib/libsqlite3.tex @@ -195,6 +195,14 @@ This can be used to build a shell for SQLite, like in the following example: \verbatiminput{sqlite3/complete_statement.py} \end{funcdesc} +\begin{funcdesc}{}enable_callback_tracebacks{flag} +By default you will not get any tracebacks in user-defined functions, +aggregates, converters, authorizer callbacks etc. If you want to debug them, +you can call this function with \var{flag} as True. Afterwards, you will get +tracebacks from callbacks on \code{sys.stderr}. Use \constant{False} to disable +the feature again. +\end{funcdesc} + \subsection{Connection Objects \label{sqlite3-Connection-Objects}} A \class{Connection} instance has the following attributes and methods: @@ -237,8 +245,7 @@ of parameters the function accepts, and \var{func} is a Python callable that is called as SQL function. The function can return any of the types supported by SQLite: unicode, str, -int, long, float, buffer and None. Exceptions in the function are ignored and -they are handled as if the function returned None. +int, long, float, buffer and None. Example: @@ -254,7 +261,7 @@ number of parameters \var{num_params}, and a \code{finalize} method which will return the final result of the aggregate. The \code{finalize} method can return any of the types supported by SQLite: -unicode, str, int, long, float, buffer and None. Any exceptions are ignored. +unicode, str, int, long, float, buffer and None. Example: @@ -283,6 +290,34 @@ To remove a collation, call \code{create_collation} with None as callable: \end{verbatim} \end{methoddesc} +\begin{methoddesc}{interrupt}{} + +You can call this method from a different thread to abort any queries that +might be executing on the connection. The query will then abort and the caller +will get an exception. +\end{methoddesc} + +\begin{methoddesc}{set_authorizer}{authorizer_callback} + +This routine registers a callback. The callback is invoked for each attempt to +access a column of a table in the database. The callback should return +\constant{SQLITE_OK} if access is allowed, \constant{SQLITE_DENY} if the entire +SQL statement should be aborted with an error and \constant{SQLITE_IGNORE} if +the column should be treated as a NULL value. These constants are available in +the \module{sqlite3} module. + +The first argument to the callback signifies what kind of operation is to be +authorized. The second and third argument will be arguments or \constant{None} +depending on the first argument. The 4th argument is the name of the database +("main", "temp", etc.) if applicable. The 5th argument is the name of the +inner-most trigger or view that is responsible for the access attempt or +\constant{None} if this access attempt is directly from input SQL code. + +Please consult the SQLite documentation about the possible values for the first +argument and the meaning of the second and third argument depending on the +first one. All necessary constants are available in the \module{sqlite3} +module. +\end{methoddesc} \begin{memberdesc}{row_factory} You can change this attribute to a callable that accepts the cursor and @@ -477,10 +512,10 @@ The type/class to adapt must be a new-style class, i. e. it must have \class{object} as one of its bases. \end{notice} -The \module{sqlite3} module has two default adapters for Python's builtin -\class{datetime.date} and \class{datetime.datetime} types. Now let's suppose we -want to store \class{datetime.datetime} objects not in ISO representation, but -as Unix timestamp. +The \module{sqlite3} module has two default adapters for Python's built-in +\class{datetime.date} and \class{datetime.datetime} types. Now let's suppose +we want to store \class{datetime.datetime} objects not in ISO representation, +but as a \UNIX{} timestamp. \verbatiminput{sqlite3/adapter_datetime.py} diff --git a/Doc/lib/libstdtypes.tex b/Doc/lib/libstdtypes.tex index f44360b..95b06f8 100644 --- a/Doc/lib/libstdtypes.tex +++ b/Doc/lib/libstdtypes.tex @@ -1,4 +1,4 @@ -\section{Built-in Types \label{types}} +\chapter{Built-in Types \label{types}} The following sections describe the standard types that are built into the interpreter. @@ -7,14 +7,14 @@ differed from user-defined types because it was not possible to use the built-in types as the basis for object-oriented inheritance. This limitation does not exist any longer.} -The principal built-in types are numerics, sequences, mappings, files +The principal built-in types are numerics, sequences, mappings, files, classes, instances and exceptions. \indexii{built-in}{types} Some operations are supported by several object types; in particular, practically all objects can be compared, tested for truth value, -and converted to a string (with the \code{`\textrm{\ldots}`} notation, -the equivalent \function{repr()} function, or the slightly different +and converted to a string (with +the \function{repr()} function or the slightly different \function{str()} function). The latter function is implicitly used when an object is written by the \keyword{print}\stindex{print} statement. @@ -24,7 +24,7 @@ and other language statements can be found in the \citetitle[../tut/tut.html]{Python Tutorial}.) -\subsection{Truth Value Testing\label{truth}} +\section{Truth Value Testing\label{truth}} Any object can be tested for truth value, for use in an \keyword{if} or \keyword{while} condition or as operand of the Boolean operations below. @@ -71,7 +71,7 @@ return one of their operands.) \index{False} \index{True} -\subsection{Boolean Operations --- +\section{Boolean Operations --- \keyword{and}, \keyword{or}, \keyword{not} \label{boolean}} @@ -107,7 +107,7 @@ These only evaluate their second argument if needed for their outcome. \end{description} -\subsection{Comparisons \label{comparisons}} +\section{Comparisons \label{comparisons}} Comparison operations are supported by all objects. They all have the same priority (which is higher than that of the Boolean operations). @@ -174,7 +174,7 @@ Two more operations with the same syntactic priority, only by sequence types (below). -\subsection{Numeric Types --- +\section{Numeric Types --- \class{int}, \class{float}, \class{long}, \class{complex} \label{typesnumeric}} @@ -307,7 +307,7 @@ though the result's type is not necessarily int. \end{description} % XXXJH exceptions: overflow (when? what operations?) zerodivision -\subsubsection{Bit-string Operations on Integer Types \label{bitstring-ops}} +\subsection{Bit-string Operations on Integer Types \label{bitstring-ops}} \nodename{Bit-string Operations} Plain and long integer types support additional operations that make @@ -350,7 +350,7 @@ division by \code{pow(2, \var{n})} without overflow check. \end{description} -\subsection{Iterator Types \label{typeiter}} +\section{Iterator Types \label{typeiter}} \versionadded{2.2} \index{iterator protocol} @@ -414,7 +414,7 @@ return an iterator object (technically, a generator object) supplying the \method{__iter__()} and \method{next()} methods. -\subsection{Sequence Types --- +\section{Sequence Types --- \class{str}, \class{unicode}, \class{list}, \class{tuple}, \class{buffer}, \class{xrange} \label{typesseq}} @@ -566,7 +566,8 @@ linear concatenation performance across versions and implementations. \end{description} -\subsubsection{String Methods \label{string-methods}} +\subsection{String Methods \label{string-methods}} +\indexii{string}{methods} These are the string methods which both 8-bit strings and Unicode objects support: @@ -618,8 +619,11 @@ For a list of possible encodings, see section~\ref{standard-encodings}. \begin{methoddesc}[string]{endswith}{suffix\optional{, start\optional{, end}}} Return \code{True} if the string ends with the specified \var{suffix}, -otherwise return \code{False}. With optional \var{start}, test beginning at +otherwise return \code{False}. \var{suffix} can also be a tuple of +suffixes to look for. With optional \var{start}, test beginning at that position. With optional \var{end}, stop comparing at that position. + +\versionchanged[Accept tuples as \var{suffix}]{2.5} \end{methoddesc} \begin{methoddesc}[string]{expandtabs}{\optional{tabsize}} @@ -829,9 +833,12 @@ boundaries. Line breaks are not included in the resulting list unless \begin{methoddesc}[string]{startswith}{prefix\optional{, start\optional{, end}}} Return \code{True} if string starts with the \var{prefix}, otherwise -return \code{False}. With optional \var{start}, test string beginning at +return \code{False}. \var{prefix} can also be a tuple of +suffixes to look for. With optional \var{start}, test string beginning at that position. With optional \var{end}, stop comparing string at that position. + +\versionchanged[Accept tuples as \var{prefix}]{2.5} \end{methoddesc} \begin{methoddesc}[string]{strip}{\optional{chars}} @@ -894,7 +901,7 @@ of length \var{width}. The original string is returned if \end{methoddesc} -\subsubsection{String Formatting Operations \label{typesseq-strings}} +\subsection{String Formatting Operations \label{typesseq-strings}} \index{formatting, string (\%{})} \index{interpolation, string (\%{})} @@ -1065,7 +1072,7 @@ Additional string operations are defined in standard modules \refmodule{re}.\refstmodindex{re} -\subsubsection{XRange Type \label{typesseq-xrange}} +\subsection{XRange Type \label{typesseq-xrange}} The \class{xrange}\obindex{xrange} type is an immutable sequence which is commonly used for looping. The advantage of the \class{xrange} @@ -1077,7 +1084,7 @@ XRange objects have very little behavior: they only support indexing, iteration, and the \function{len()} function. -\subsubsection{Mutable Sequence Types \label{typesseq-mutable}} +\subsection{Mutable Sequence Types \label{typesseq-mutable}} List objects support additional operations that allow in-place modification of the object. @@ -1094,7 +1101,8 @@ The following operations are defined on mutable sequence types (where \lineiii{\var{s}[\var{i}] = \var{x}} {item \var{i} of \var{s} is replaced by \var{x}}{} \lineiii{\var{s}[\var{i}:\var{j}] = \var{t}} - {slice of \var{s} from \var{i} to \var{j} is replaced by \var{t}}{} + {slice of \var{s} from \var{i} to \var{j} + is replaced by the contents of the iterable \var{t}}{} \lineiii{del \var{s}[\var{i}:\var{j}]} {same as \code{\var{s}[\var{i}:\var{j}] = []}}{} \lineiii{\var{s}[\var{i}:\var{j}:\var{k}] = \var{t}} @@ -1208,7 +1216,7 @@ Notes: that the list has been mutated during a sort. \end{description} -\subsection{Set Types --- +\section{Set Types --- \class{set}, \class{frozenset} \label{types-set}} \obindex{set} @@ -1347,7 +1355,7 @@ The design of the set types was based on lessons learned from the \end{seealso} -\subsection{Mapping Types --- \class{dict} \label{typesmapping}} +\section{Mapping Types --- \class{dict} \label{typesmapping}} \obindex{mapping} \obindex{dictionary} @@ -1510,7 +1518,7 @@ For an example, see \module{collections}.\class{defaultdict}. \end{description} -\subsection{File Objects +\section{File Objects \label{bltin-file-objects}} File objects\obindex{file} are implemented using C's \code{stdio} @@ -1783,7 +1791,7 @@ implemented in C will have to provide a writable \end{memberdesc} -\subsection{Context Manager Types \label{typecontextmanager}} +\section{Context Manager Types \label{typecontextmanager}} \versionadded{2.5} \index{context manager} @@ -1864,13 +1872,13 @@ runtime context, the overhead of a single class dictionary lookup is negligible. -\subsection{Other Built-in Types \label{typesother}} +\section{Other Built-in Types \label{typesother}} The interpreter supports several other kinds of objects. Most of these support only one or two operations. -\subsubsection{Modules \label{typesmodules}} +\subsection{Modules \label{typesmodules}} The only special operation on a module is attribute access: \code{\var{m}.\var{name}}, where \var{m} is a module and \var{name} @@ -1896,14 +1904,14 @@ written as \code{}. -\subsubsection{Classes and Class Instances \label{typesobjects}} +\subsection{Classes and Class Instances \label{typesobjects}} \nodename{Classes and Instances} See chapters 3 and 7 of the \citetitle[../ref/ref.html]{Python Reference Manual} for these. -\subsubsection{Functions \label{typesfunctions}} +\subsection{Functions \label{typesfunctions}} Function objects are created by function definitions. The only operation on a function object is to call it: @@ -1917,7 +1925,7 @@ different object types. See the \citetitle[../ref/ref.html]{Python Reference Manual} for more information. -\subsubsection{Methods \label{typesmethods}} +\subsection{Methods \label{typesmethods}} \obindex{method} Methods are functions that are called using the attribute notation. @@ -1962,7 +1970,7 @@ See the \citetitle[../ref/ref.html]{Python Reference Manual} for more information. -\subsubsection{Code Objects \label{bltin-code-objects}} +\subsection{Code Objects \label{bltin-code-objects}} \obindex{code} Code objects are used by the implementation to represent @@ -1985,7 +1993,7 @@ See the \citetitle[../ref/ref.html]{Python Reference Manual} for more information. -\subsubsection{Type Objects \label{bltin-type-objects}} +\subsection{Type Objects \label{bltin-type-objects}} Type objects represent the various object types. An object's type is accessed by the built-in function \function{type()}. There are no special @@ -1997,7 +2005,7 @@ for all standard built-in types. Types are written like this: \code{}. -\subsubsection{The Null Object \label{bltin-null-object}} +\subsection{The Null Object \label{bltin-null-object}} This object is returned by functions that don't explicitly return a value. It supports no special operations. There is exactly one null @@ -2006,7 +2014,7 @@ object, named \code{None} (a built-in name). It is written as \code{None}. -\subsubsection{The Ellipsis Object \label{bltin-ellipsis-object}} +\subsection{The Ellipsis Object \label{bltin-ellipsis-object}} This object is used by extended slice notation (see the \citetitle[../ref/ref.html]{Python Reference Manual}). It supports no @@ -2015,7 +2023,7 @@ special operations. There is exactly one ellipsis object, named It is written as \code{Ellipsis}. -\subsubsection{Boolean Values} +\subsection{Boolean Values} Boolean values are the two constant objects \code{False} and \code{True}. They are used to represent truth values (although other @@ -2032,14 +2040,14 @@ They are written as \code{False} and \code{True}, respectively. \indexii{Boolean}{values} -\subsubsection{Internal Objects \label{typesinternal}} +\subsection{Internal Objects \label{typesinternal}} See the \citetitle[../ref/ref.html]{Python Reference Manual} for this information. It describes stack frame objects, traceback objects, and slice objects. -\subsection{Special Attributes \label{specialattrs}} +\section{Special Attributes \label{specialattrs}} The implementation adds a few special read-only attributes to several object types, where they are relevant. Some of these are not reported diff --git a/Doc/lib/libstringio.tex b/Doc/lib/libstringio.tex index 3992e43..2431251 100644 --- a/Doc/lib/libstringio.tex +++ b/Doc/lib/libstringio.tex @@ -37,6 +37,24 @@ such mixing can cause this method to raise \exception{UnicodeError}. Free the memory buffer. \end{methoddesc} +Example usage: + +\begin{verbatim} +import StringIO + +output = StringIO.StringIO() +output.write('First line.\n') +print >>output, 'Second line.' + +# Retrieve file contents -- this will be +# 'First line.\nSecond line.\n' +contents = output.getvalue() + +# Close object and discard memory buffer -- +# .getvalue() will now raise an exception. +output.close() +\end{verbatim} + \section{\module{cStringIO} --- Faster version of \module{StringIO}} @@ -82,3 +100,22 @@ The following data objects are provided as well: There is a C API to the module as well; refer to the module source for more information. + +Example usage: + +\begin{verbatim} +import cStringIO + +output = cStringIO.StringIO() +output.write('First line.\n') +print >>output, 'Second line.' + +# Retrieve file contents -- this will be +# 'First line.\nSecond line.\n' +contents = output.getvalue() + +# Close object and discard memory buffer -- +# .getvalue() will now raise an exception. +output.close() +\end{verbatim} + diff --git a/Doc/lib/libsubprocess.tex b/Doc/lib/libsubprocess.tex index bde92eb..03072f7 100644 --- a/Doc/lib/libsubprocess.tex +++ b/Doc/lib/libsubprocess.tex @@ -107,7 +107,7 @@ for the new process. If \var{universal_newlines} is \constant{True}, the file objects stdout and stderr are opened as text files, but lines may be terminated by -any of \code{'\e n'}, the Unix end-of-line convention, \code{'\e r'}, +any of \code{'\e n'}, the \UNIX{} end-of-line convention, \code{'\e r'}, the Macintosh convention or \code{'\e r\e n'}, the Windows convention. All of these external representations are seen as \code{'\e n'} by the Python program. \note{This feature is only available if Python is built @@ -140,7 +140,7 @@ The arguments are the same as for the Popen constructor. Example: Run command with arguments. Wait for command to complete. If the exit code was zero then return, otherwise raise \exception{CalledProcessError.} The \exception{CalledProcessError} object will have the return code in the -\member{errno} attribute. +\member{returncode} attribute. The arguments are the same as for the Popen constructor. Example: @@ -164,9 +164,8 @@ should prepare for \exception{OSError} exceptions. A \exception{ValueError} will be raised if \class{Popen} is called with invalid arguments. -check_call() will raise \exception{CalledProcessError}, which is a -subclass of \exception{OSError}, if the called process returns a -non-zero return code. +check_call() will raise \exception{CalledProcessError}, if the called +process returns a non-zero return code. \subsubsection{Security} diff --git a/Doc/lib/libsys.tex b/Doc/lib/libsys.tex index 6b5b755..702427a 100644 --- a/Doc/lib/libsys.tex +++ b/Doc/lib/libsys.tex @@ -21,7 +21,7 @@ It is always available. \begin{datadesc}{byteorder} An indicator of the native byte order. This will have the value - \code{'big'} on big-endian (most-signigicant byte first) platforms, + \code{'big'} on big-endian (most-significant byte first) platforms, and \code{'little'} on little-endian (least-significant byte first) platforms. \versionadded{2.0} @@ -41,7 +41,7 @@ It is always available. \code{Include/patchlevel.h} if the branch is a tag. Otherwise, it is \code{None}. \versionadded{2.5} -\end{datadesc} +\end{datadesc} \begin{datadesc}{builtin_module_names} A tuple of strings giving the names of all modules that are compiled @@ -55,6 +55,23 @@ It is always available. interpreter. \end{datadesc} +\begin{funcdesc}{_current_frames}{} + Return a dictionary mapping each thread's identifier to the topmost stack + frame currently active in that thread at the time the function is called. + Note that functions in the \refmodule{traceback} module can build the + call stack given such a frame. + + This is most useful for debugging deadlock: this function does not + require the deadlocked threads' cooperation, and such threads' call stacks + are frozen for as long as they remain deadlocked. The frame returned + for a non-deadlocked thread may bear no relationship to that thread's + current activity by the time calling code examines the frame. + + This function should be used for internal and specialized purposes + only. + \versionadded{2.5} +\end{funcdesc} + \begin{datadesc}{dllhandle} Integer specifying the handle of the Python DLL. Availability: Windows. @@ -142,7 +159,7 @@ It is always available. function, \function{exc_info()} will return three \code{None} values until another exception is raised in the current thread or the execution stack returns to a frame where another exception is being handled. - + This function is only needed in only a few obscure situations. These include logging and error handling systems that report information on the last or current exception. This function can also be used to try to free @@ -241,14 +258,14 @@ It is always available. \begin{itemize} \item On Windows 9x, the encoding is ``mbcs''. \item On Mac OS X, the encoding is ``utf-8''. -\item On Unix, the encoding is the user's preference - according to the result of nl_langinfo(CODESET), or None if - the nl_langinfo(CODESET) failed. +\item On \UNIX, the encoding is the user's preference + according to the result of nl_langinfo(CODESET), or \constant{None} + if the \code{nl_langinfo(CODESET)} failed. \item On Windows NT+, file names are Unicode natively, so no conversion - is performed. \code{getfilesystemencoding} still returns ``mbcs'', - as this is the encoding that applications should use when they - explicitly want to convert Unicode strings to byte strings that - are equivalent when used as file names. + is performed. \function{getfilesystemencoding()} still returns + \code{'mbcs'}, as this is the encoding that applications should use + when they explicitly want to convert Unicode strings to byte strings + that are equivalent when used as file names. \end{itemize} \versionadded{2.3} \end{funcdesc} @@ -279,8 +296,8 @@ It is always available. \end{funcdesc} \begin{funcdesc}{getwindowsversion}{} - Return a tuple containing five components, describing the Windows - version currently running. The elements are \var{major}, \var{minor}, + Return a tuple containing five components, describing the Windows + version currently running. The elements are \var{major}, \var{minor}, \var{build}, \var{platform}, and \var{text}. \var{text} contains a string while all other values are integers. @@ -491,7 +508,7 @@ else: be registered using \function{settrace()} for each thread being debugged. \note{The \function{settrace()} function is intended only for implementing debuggers, profilers, coverage tools and the like. - Its behavior is part of the implementation platform, rather than + Its behavior is part of the implementation platform, rather than part of the language definition, and thus may not be available in all Python implementations.} \end{funcdesc} diff --git a/Doc/lib/libtextwrap.tex b/Doc/lib/libtextwrap.tex index 9fb0816..38f9b03 100644 --- a/Doc/lib/libtextwrap.tex +++ b/Doc/lib/libtextwrap.tex @@ -47,12 +47,17 @@ remove indentation from strings that have unwanted whitespace to the left of the text. \begin{funcdesc}{dedent}{text} -Remove any whitespace that can be uniformly removed from the left -of every line in \var{text}. +Remove any common leading whitespace from every line in \var{text}. -This is typically used to make triple-quoted strings line up with -the left edge of screen/whatever, while still presenting it in the -source code in indented form. +This can be used to make triple-quoted strings line up with the left +edge of the display, while still presenting them in the source code +in indented form. + +Note that tabs and spaces are both treated as whitespace, but they are +not equal: the lines \code{" {} hello"} and \code{"\textbackslash{}thello"} +are considered to have no common leading whitespace. (This behaviour is +new in Python 2.5; older versions of this module incorrectly expanded +tabs before searching for common leading whitespace.) For example: \begin{verbatim} diff --git a/Doc/lib/libthread.tex b/Doc/lib/libthread.tex index 9573ab3..d007eec 100644 --- a/Doc/lib/libthread.tex +++ b/Doc/lib/libthread.tex @@ -74,6 +74,26 @@ data. Thread identifiers may be recycled when a thread exits and another thread is created. \end{funcdesc} +\begin{funcdesc}{stack_size}{\optional{size}} +Return the thread stack size used when creating new threads. The +optional \var{size} argument specifies the stack size to be used for +subsequently created threads, and must be 0 (use platform or +configured default) or a positive integer value of at least 32,768 (32kB). +If changing the thread stack size is unsupported, a \exception{ThreadError} +is raised. If the specified stack size is invalid, a \exception{ValueError} +is raised and the stack size is unmodified. 32kB is currently the minimum +supported stack size value to guarantee sufficient stack space for the +interpreter itself. Note that some platforms may have particular +restrictions on values for the stack size, such as requiring a minimum +stack size > 32kB or requiring allocation in multiples of the system +memory page size - platform documentation should be referred to for +more information (4kB pages are common; using multiples of 4096 for +the stack size is the suggested approach in the absence of more +specific information). +Availability: Windows, systems with \POSIX{} threads. +\versionadded{2.5} +\end{funcdesc} + Lock objects have the following methods: diff --git a/Doc/lib/libthreading.tex b/Doc/lib/libthreading.tex index 8fb3137..0334750 100644 --- a/Doc/lib/libthreading.tex +++ b/Doc/lib/libthreading.tex @@ -125,6 +125,26 @@ method is called. \versionadded{2.3} \end{funcdesc} +\begin{funcdesc}{stack_size}{\optional{size}} +Return the thread stack size used when creating new threads. The +optional \var{size} argument specifies the stack size to be used for +subsequently created threads, and must be 0 (use platform or +configured default) or a positive integer value of at least 32,768 (32kB). +If changing the thread stack size is unsupported, a \exception{ThreadError} +is raised. If the specified stack size is invalid, a \exception{ValueError} +is raised and the stack size is unmodified. 32kB is currently the minimum +supported stack size value to guarantee sufficient stack space for the +interpreter itself. Note that some platforms may have particular +restrictions on values for the stack size, such as requiring a minimum +stack size > 32kB or requiring allocation in multiples of the system +memory page size - platform documentation should be referred to for +more information (4kB pages are common; using multiples of 4096 for +the stack size is the suggested approach in the absence of more +specific information). +Availability: Windows, systems with \POSIX{} threads. +\versionadded{2.5} +\end{funcdesc} + Detailed interfaces for the objects are documented below. The design of this module is loosely based on Java's threading model. diff --git a/Doc/lib/libtime.tex b/Doc/lib/libtime.tex index b39b650..f40838a 100644 --- a/Doc/lib/libtime.tex +++ b/Doc/lib/libtime.tex @@ -226,6 +226,8 @@ if any field in \var{t} is outside of the allowed range. \versionchanged[Allowed \var{t} to be omitted]{2.1} \versionchanged[\exception{ValueError} raised if a field in \var{t} is out of range]{2.4} +\versionchanged[0 is now a legal argument for any position in the time tuple; +if it is normally illegal the value is forced to a correct one.]{2.5} The following directives can be embedded in the \var{format} string. @@ -425,7 +427,7 @@ Where: '16:08:12 05/08/03 AEST' \end{verbatim} -On many Unix systems (including *BSD, Linux, Solaris, and Darwin), it +On many \UNIX{} systems (including *BSD, Linux, Solaris, and Darwin), it is more convenient to use the system's zoneinfo (\manpage{tzfile}{5}) database to specify the timezone rules. To do this, set the \envvar{TZ} environment variable to the path of the required timezone diff --git a/Doc/lib/libturtle.tex b/Doc/lib/libturtle.tex index 638bc07..6161cd9 100644 --- a/Doc/lib/libturtle.tex +++ b/Doc/lib/libturtle.tex @@ -27,6 +27,45 @@ Set angle measurement units to degrees. Set angle measurement units to radians. \end{funcdesc} +\begin{funcdesc}{setup}{**kwargs} +Sets the size and position of the main window. Keywords are: +\begin{itemize} + \item \code{width}: either a size in pixels or a fraction of the screen. + The default is 50\% of the screen. + \item \code{height}: either a size in pixels or a fraction of the screen. + The default is 50\% of the screen. + \item \code{startx}: starting position in pixels from the left edge + of the screen. \code{None} is the default value and + centers the window horizontally on screen. + \item \code{starty}: starting position in pixels from the top edge + of the screen. \code{None} is the default value and + centers the window vertically on screen. +\end{itemize} + + Examples: + +\begin{verbatim} +# Uses default geometry: 50% x 50% of screen, centered. +setup() + +# Sets window to 200x200 pixels, in upper left of screen +setup (width=200, height=200, startx=0, starty=0) + +# Sets window to 75% of screen by 50% of screen, and centers it. +setup(width=.75, height=0.5, startx=None, starty=None) +\end{verbatim} + +\end{funcdesc} + +\begin{funcdesc}{title}{title_str} +Set the window's title to \var{title}. +\end{funcdesc} + +\begin{funcdesc}{done}{} +Enters the Tk main loop. The window will continue to +be displayed until the user closes it or the process is killed. +\end{funcdesc} + \begin{funcdesc}{reset}{} Clear the screen, re-center the pen, and set variables to the default values. @@ -42,6 +81,19 @@ means line are drawn more slowly, with an animation of an arrow along the line. \end{funcdesc} +\begin{funcdesc}{speed}{speed} +Set the speed of the turtle. Valid values for the parameter +\var{speed} are \code{'fastest'} (no delay), \code{'fast'}, +(delay 5ms), \code{'normal'} (delay 10ms), \code{'slow'} +(delay 15ms), and \code{'slowest'} (delay 20ms). +\versionadded{2.5} +\end{funcdesc} + +\begin{funcdesc}{delay}{delay} +Set the speed of the turtle to \var{delay}, which is given +in ms. \versionadded{2.5} +\end{funcdesc} + \begin{funcdesc}{forward}{distance} Go forward \var{distance} steps. \end{funcdesc} @@ -94,6 +146,18 @@ usage is: call \code{fill(1)} before drawing a path you want to fill, and call \code{fill(0)} when you finish to draw the path. \end{funcdesc} +\begin{funcdesc}{begin\_fill}{} +Switch turtle into filling mode; +Must eventually be followed by a corresponding end_fill() call. +Otherwise it will be ignored. +\versionadded{2.5} +\end{funcdesc} + +\begin{funcdesc}{end\_fill}{} +End filling mode, and fill the shape; equivalent to \code{fill(0)}. +\versionadded{2.5} +\end{funcdesc} + \begin{funcdesc}{circle}{radius\optional{, extent}} Draw a circle with radius \var{radius} whose center-point is \var{radius} units left of the turtle. @@ -113,6 +177,49 @@ Go to co-ordinates \var{x}, \var{y}. The co-ordinates may be specified either as two separate arguments or as a 2-tuple. \end{funcdesc} +\begin{funcdesc}{towards}{x, y} +Return the angle of the line from the turtle's position +to the point \var{x}, \var{y}. The co-ordinates may be +specified either as two separate arguments, as a 2-tuple, +or as another pen object. +\versionadded{2.5} +\end{funcdesc} + +\begin{funcdesc}{heading}{} +Return the current orientation of the turtle. +\versionadded{2.3} +\end{funcdesc} + +\begin{funcdesc}{setheading}{angle} +Set the orientation of the turtle to \var{angle}. +\versionadded{2.3} +\end{funcdesc} + +\begin{funcdesc}{position}{} +Return the current location of the turtle as an \code{(x,y)} pair. +\versionadded{2.3} +\end{funcdesc} + +\begin{funcdesc}{setx}{x} +Set the x coordinate of the turtle to \var{x}. +\versionadded{2.3} +\end{funcdesc} + +\begin{funcdesc}{sety}{y} +Set the y coordinate of the turtle to \var{y}. +\versionadded{2.3} +\end{funcdesc} + +\begin{funcdesc}{window\_width}{} +Return the width of the canvas window. +\versionadded{2.3} +\end{funcdesc} + +\begin{funcdesc}{window\_height}{} +Return the height of the canvas window. +\versionadded{2.3} +\end{funcdesc} + This module also does \code{from math import *}, so see the documentation for the \refmodule{math} module for additional constants and functions useful for turtle graphics. @@ -134,19 +241,25 @@ Define a pen. All above functions can be called as a methods on the given pen. The constructor automatically creates a canvas do be drawn on. \end{classdesc} +\begin{classdesc}{Turtle}{} +Define a pen. This is essentially a synonym for \code{Pen()}; +\class{Turtle} is an empty subclass of \class{Pen}. +\end{classdesc} + \begin{classdesc}{RawPen}{canvas} Define a pen which draws on a canvas \var{canvas}. This is useful if you want to use the module to create graphics in a ``real'' program. \end{classdesc} -\subsection{Pen and RawPen Objects \label{pen-rawpen-objects}} +\subsection{Turtle, Pen and RawPen Objects \label{pen-rawpen-objects}} -\class{Pen} and \class{RawPen} objects have all the global functions -described above, except for \function{demo()} as methods, which -manipulate the given pen. +Most of the global functions available in the module are also +available as methods of the \class{Turtle}, \class{Pen} and +\class{RawPen} classes, affecting only the state of the given pen. The only method which is more powerful as a method is -\function{degrees()}. +\function{degrees()}, which takes an optional argument letting +you specify the number of units corresponding to a full circle: \begin{methoddesc}{degrees}{\optional{fullcircle}} \var{fullcircle} is by default 360. This can cause the pen to have any diff --git a/Doc/lib/libtypes.tex b/Doc/lib/libtypes.tex index f8f557d..c80a87a 100644 --- a/Doc/lib/libtypes.tex +++ b/Doc/lib/libtypes.tex @@ -176,6 +176,30 @@ The type of buffer objects created by the \function{buffer()}\bifuncindex{buffer} function. \end{datadesc} +\begin{datadesc}{DictProxyType} +The type of dict proxies, such as \code{TypeType.__dict__}. +\end{datadesc} + +\begin{datadesc}{NotImplementedType} +The type of \code{NotImplemented} +\end{datadesc} + +\begin{datadesc}{GetSetDescriptorType} +The type of objects defined in extension modules with \code{PyGetSetDef}, such +as \code{FrameType.f_locals} or \code{array.array.typecode}. This constant is +not defined in implementations of Python that do not have such extension +types, so for portable code use \code{hasattr(types, 'GetSetDescriptorType')}. +\versionadded{2.5} +\end{datadesc} + +\begin{datadesc}{MemberDescriptorType} +The type of objects defined in extension modules with \code{PyMemberDef}, such +as \code {datetime.timedelta.days}. This constant is not defined in +implementations of Python that do not have such extension types, so for +portable code use \code{hasattr(types, 'MemberDescriptorType')}. +\versionadded{2.5} +\end{datadesc} + \begin{datadesc}{StringTypes} A sequence containing \code{StringType} and \code{UnicodeType} used to facilitate easier checking for any string object. Using this is more diff --git a/Doc/lib/libundoc.tex b/Doc/lib/libundoc.tex index df78152..e7d388f 100644 --- a/Doc/lib/libundoc.tex +++ b/Doc/lib/libundoc.tex @@ -49,7 +49,7 @@ document these. \item[\module{bsddb185}] --- Backwards compatibility module for systems which still use the Berkeley - DB 1.85 module. It is normally only available on certain BSD Unix-based + DB 1.85 module. It is normally only available on certain BSD \UNIX-based systems. It should never be used directly. \end{description} diff --git a/Doc/lib/libunicodedata.tex b/Doc/lib/libunicodedata.tex index dcbda77..435466a 100644 --- a/Doc/lib/libunicodedata.tex +++ b/Doc/lib/libunicodedata.tex @@ -14,11 +14,11 @@ This module provides access to the Unicode Character Database which defines character properties for all Unicode characters. The data in this database is based on the \file{UnicodeData.txt} file version -4.1.0 which is publically available from \url{ftp://ftp.unicode.org/}. +4.1.0 which is publicly available from \url{ftp://ftp.unicode.org/}. The module uses the same names and symbols as defined by the UnicodeData File Format 4.1.0 (see -\url{http://www.unicode.org/Public/4.1-Update/UnicodeData-4.1.0.html}). It +\url{http://www.unicode.org/Public/4.1.0/ucd/UCD.html}). It defines the following functions: \begin{funcdesc}{lookup}{name} @@ -108,7 +108,7 @@ decomposition, and translates each character into its decomposed form. Normal form C (NFC) first applies a canonical decomposition, then composes pre-combined characters again. -In addition to these two forms, there two additional normal forms +In addition to these two forms, there are two additional normal forms based on compatibility equivalence. In Unicode, certain characters are supported which normally would be unified with other characters. For example, U+2160 (ROMAN NUMERAL ONE) is really the same thing as U+0049 @@ -139,3 +139,22 @@ the Unicode database (such as IDNA). \versionadded{2.5} \end{datadesc} + +Examples: + +\begin{verbatim} +>>> unicodedata.lookup('LEFT CURLY BRACKET') +u'{' +>>> unicodedata.name(u'/') +'SOLIDUS' +>>> unicodedata.decimal(u'9') +9 +>>> unicodedata.decimal(u'a') +Traceback (most recent call last): + File "", line 1, in ? +ValueError: not a decimal +>>> unicodedata.category(u'A') # 'L'etter, 'u'ppercase +'Lu' +>>> unicodedata.bidirectional(u'\u0660') # 'A'rabic, 'N'umber +'AN' +\end{verbatim} diff --git a/Doc/lib/libunittest.tex b/Doc/lib/libunittest.tex index 51b321e..f40493d 100644 --- a/Doc/lib/libunittest.tex +++ b/Doc/lib/libunittest.tex @@ -10,19 +10,19 @@ \versionadded{2.1} -The Python unit testing framework, often referred to as ``PyUnit,'' is +The Python unit testing framework, sometimes referred to as ``PyUnit,'' is a Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in turn, a Java version of Kent's Smalltalk testing framework. Each is the de facto standard unit testing framework for its respective language. -PyUnit supports test automation, sharing of setup and shutdown code -for tests, aggregation of tests into collections, and independence of +\module{unittest} supports test automation, sharing of setup and shutdown +code for tests, aggregation of tests into collections, and independence of the tests from the reporting framework. The \module{unittest} module provides classes that make it easy to support these qualities for a set of tests. -To achieve this, PyUnit supports some important concepts: +To achieve this, \module{unittest} supports some important concepts: \begin{definitions} \term{test fixture} @@ -33,10 +33,9 @@ starting a server process. \term{test case} A \dfn{test case} is the smallest unit of testing. It checks for a -specific response to a particular set of inputs. PyUnit provides a -base class, \class{TestCase}, which may be used to create new test -cases. You may provide your own implementation that does not subclass -from \class{TestCase}, of course. +specific response to a particular set of inputs. \module{unittest} +provides a base class, \class{TestCase}, which may be used to create +new test cases. \term{test suite} A \dfn{test suite} is a collection of test cases, test suites, or @@ -54,8 +53,8 @@ indicate the results of executing the tests. The test case and test fixture concepts are supported through the \class{TestCase} and \class{FunctionTestCase} classes; the former should be used when creating new tests, and the latter can be used when -integrating existing test code with a PyUnit-driven framework. When -building test fixtures using \class{TestCase}, the \method{setUp()} +integrating existing test code with a \module{unittest}-driven framework. +When building test fixtures using \class{TestCase}, the \method{setUp()} and \method{tearDown()} methods can be overridden to provide initialization and cleanup for the fixture. With \class{FunctionTestCase}, existing functions can be passed to the @@ -74,19 +73,17 @@ the suite is executed, all tests added directly to the suite and in A test runner is an object that provides a single method, \method{run()}, which accepts a \class{TestCase} or \class{TestSuite} object as a parameter, and returns a result object. The class -\class{TestResult} is provided for use as the result object. PyUnit -provide the \class{TextTestRunner} as an example test runner which -reports test results on the standard error stream by default. -Alternate runners can be implemented for other environments (such as -graphical environments) without any need to derive from a specific -class. +\class{TestResult} is provided for use as the result object. +\module{unittest} provides the \class{TextTestRunner} as an example +test runner which reports test results on the standard error stream by +default. Alternate runners can be implemented for other environments +(such as graphical environments) without any need to derive from a +specific class. \begin{seealso} \seemodule{doctest}{Another test-support module with a very different flavor.} - \seetitle[http://pyunit.sourceforge.net/]{PyUnit Web Site}{The - source for further information on PyUnit.} \seetitle[http://www.XProgramming.com/testfram.htm]{Simple Smalltalk Testing: With Patterns}{Kent Beck's original paper on testing frameworks using the pattern shared by @@ -166,7 +163,7 @@ run from the command line. For example, the last two lines may be replaced with: \begin{verbatim} -suite = unittest.makeSuite(TestSequenceFunctions) +suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions) unittest.TextTestRunner(verbosity=2).run(suite) \end{verbatim} @@ -194,8 +191,8 @@ of the documentation explores the full feature set from first principles. The basic building blocks of unit testing are \dfn{test cases} --- single scenarios that must be set up and checked for correctness. In -PyUnit, test cases are represented by instances of the -\class{TestCase} class in the \refmodule{unittest} module. To make +\module{unittest}, test cases are represented by instances of +\module{unittest}'s \class{TestCase} class. To make your own test cases you must write subclasses of \class{TestCase}, or use \class{FunctionTestCase}. @@ -207,7 +204,7 @@ The testing code of a \class{TestCase} instance should be entirely self contained, such that it can be run either in isolation or in arbitrary combination with any number of other test cases. -The simplest test case subclass will simply override the +The simplest \class{TestCase} subclass will simply override the \method{runTest()} method in order to perform specific testing code: \begin{verbatim} @@ -221,12 +218,13 @@ class DefaultWidgetSizeTestCase(unittest.TestCase): Note that in order to test something, we use the one of the \method{assert*()} or \method{fail*()} methods provided by the -\class{TestCase} base class. If the test fails when the test case -runs, an exception will be raised, and the testing framework will -identify the test case as a \dfn{failure}. Other exceptions that do -not arise from checks made through the \method{assert*()} and -\method{fail*()} methods are identified by the testing framework as -\dfn{errors}. +\class{TestCase} base class. If the test fails, an exception will be +raised, and \module{unittest} will identify the test case as a +\dfn{failure}. Any other exceptions will be treated as \dfn{errors}. +This helps you identify where the problem is: \dfn{failures} are caused by +incorrect results - a 5 where you expected a 6. \dfn{Errors} are caused by +incorrect code - e.g., a \exception{TypeError} caused by an incorrect +function call. The way to run a test case will be described later. For now, note that to construct an instance of such a test case, we call its @@ -237,7 +235,7 @@ testCase = DefaultWidgetSizeTestCase() \end{verbatim} Now, such test cases can be numerous, and their set-up can be -repetitive. In the above case, constructing a ``Widget'' in each of +repetitive. In the above case, constructing a \class{Widget} in each of 100 Widget test case subclasses would mean unsightly duplication. Luckily, we can factor out such set-up code by implementing a method @@ -283,7 +281,7 @@ class SimpleWidgetTestCase(unittest.TestCase): \end{verbatim} If \method{setUp()} succeeded, the \method{tearDown()} method will be -run regardless of whether or not \method{runTest()} succeeded. +run whether \method{runTest()} succeeded or not. Such a working environment for the testing code is called a \dfn{fixture}. @@ -292,8 +290,8 @@ Often, many small test cases will use the same fixture. In this case, we would end up subclassing \class{SimpleWidgetTestCase} into many small one-method classes such as \class{DefaultWidgetSizeTestCase}. This is time-consuming and -discouraging, so in the same vein as JUnit, PyUnit provides a simpler -mechanism: +discouraging, so in the same vein as JUnit, \module{unittest} provides +a simpler mechanism: \begin{verbatim} import unittest @@ -329,9 +327,9 @@ resizeTestCase = WidgetTestCase("testResize") \end{verbatim} Test case instances are grouped together according to the features -they test. PyUnit provides a mechanism for this: the \class{test -suite}, represented by the class \class{TestSuite} in the -\refmodule{unittest} module: +they test. \module{unittest} provides a mechanism for this: the +\dfn{test suite}, represented by \module{unittest}'s \class{TestSuite} +class: \begin{verbatim} widgetTestSuite = unittest.TestSuite() @@ -354,28 +352,30 @@ def suite(): or even: \begin{verbatim} -class WidgetTestSuite(unittest.TestSuite): - def __init__(self): - unittest.TestSuite.__init__(self,map(WidgetTestCase, - ("testDefaultSize", - "testResize"))) -\end{verbatim} +def suite(): + tests = ["testDefaultSize", "testResize"] -(The latter is admittedly not for the faint-hearted!) + return unittest.TestSuite(map(WidgetTestCase, tests)) +\end{verbatim} Since it is a common pattern to create a \class{TestCase} subclass -with many similarly named test functions, there is a convenience -function called \function{makeSuite()} that constructs a test suite -that comprises all of the test cases in a test case class: +with many similarly named test functions, \module{unittest} provides a +\class{TestLoader} class that can be used to automate the process of +creating a test suite and populating it with individual tests. +For example, \begin{verbatim} -suite = unittest.makeSuite(WidgetTestCase) +suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase) \end{verbatim} -Note that when using the \function{makeSuite()} function, the order in -which the various test cases will be run by the test suite is the -order determined by sorting the test function names using the -\function{cmp()} built-in function. +will create a test suite that will run +\code{WidgetTestCase.testDefaultSize()} and \code{WidgetTestCase.testResize}. +\class{TestLoader} uses the \code{'test'} method name prefix to identify +test methods automatically. + +Note that the order in which the various test cases will be run is +determined by sorting the test function names with the built-in +\function{cmp()} function. Often it is desirable to group suites of test cases together, so as to run tests for the whole system at once. This is easy, since @@ -385,13 +385,13 @@ as \class{TestCase} instances can be added to a \class{TestSuite}: \begin{verbatim} suite1 = module1.TheTestSuite() suite2 = module2.TheTestSuite() -alltests = unittest.TestSuite((suite1, suite2)) +alltests = unittest.TestSuite([suite1, suite2]) \end{verbatim} You can place the definitions of test cases and test suites in the same modules as the code they are to test (such as \file{widget.py}), but there are several advantages to placing the test code in a -separate module, such as \file{widgettests.py}: +separate module, such as \file{test_widget.py}: \begin{itemize} \item The test module can be run standalone from the command line. @@ -412,13 +412,12 @@ separate module, such as \file{widgettests.py}: \label{legacy-unit-tests}} Some users will find that they have existing test code that they would -like to run from PyUnit, without converting every old test function to -a \class{TestCase} subclass. +like to run from \module{unittest}, without converting every old test +function to a \class{TestCase} subclass. -For this reason, PyUnit provides a \class{FunctionTestCase} class. -This subclass of \class{TestCase} can be used to wrap an existing test -function. Set-up and tear-down functions can also optionally be -wrapped. +For this reason, \module{unittest} provides a \class{FunctionTestCase} +class. This subclass of \class{TestCase} can be used to wrap an existing +test function. Set-up and tear-down functions can also be provided. Given the following test function: @@ -436,7 +435,8 @@ testcase = unittest.FunctionTestCase(testSomething) \end{verbatim} If there are additional set-up and tear-down methods that should be -called as part of the test case's operation, they can also be provided: +called as part of the test case's operation, they can also be provided +like so: \begin{verbatim} testcase = unittest.FunctionTestCase(testSomething, @@ -444,9 +444,19 @@ testcase = unittest.FunctionTestCase(testSomething, tearDown=deleteSomethingDB) \end{verbatim} -\note{PyUnit supports the use of \exception{AssertionError} -as an indicator of test failure, but does not recommend it. Future -versions may treat \exception{AssertionError} differently.} +To make migrating existing test suites easier, \module{unittest} +supports tests raising \exception{AssertionError} to indicate test failure. +However, it is recommended that you use the explicit +\method{TestCase.fail*()} and \method{TestCase.assert*()} methods instead, +as future versions of \module{unittest} may treat \exception{AssertionError} +differently. + +\note{Even though \class{FunctionTestCase} can be used to quickly convert +an existing test base over to a \module{unittest}-based system, this +approach is not recommended. Taking the time to set up proper +\class{TestCase} subclasses will make future test refactorings infinitely +easier.} + \subsection{Classes and functions @@ -454,11 +464,12 @@ versions may treat \exception{AssertionError} differently.} \begin{classdesc}{TestCase}{} Instances of the \class{TestCase} class represent the smallest - testable units in a set of tests. This class is intended to be used - as a base class, with specific tests being implemented by concrete - subclasses. This class implements the interface needed by the test - runner to allow it to drive the test, and methods that the test code - can use to check for and report various kinds of failures. + testable units in the \module{unittest} universe. This class is + intended to be used as a base class, with specific tests being + implemented by concrete subclasses. This class implements the + interface needed by the test runner to allow it to drive the + test, and methods that the test code can use to check for and + report various kinds of failure. \end{classdesc} \begin{classdesc}{FunctionTestCase}{testFunc\optional{, @@ -474,33 +485,33 @@ versions may treat \exception{AssertionError} differently.} \begin{classdesc}{TestSuite}{\optional{tests}} This class represents an aggregation of individual tests cases and test suites. The class presents the interface needed by the test - runner to allow it to be run as any other test case, but all the - contained tests and test suites are executed. Additional methods - are provided to add test cases and suites to the aggregation. If - \var{tests} is given, it must be a sequence of individual tests that - will be added to the suite. + runner to allow it to be run as any other test case. Running a + \class{TestSuite} instance is the same as iterating over the suite, + running each test individually. + + If \var{tests} is given, it must be an iterable of individual test cases or + other test suites that will be used to build the suite initially. + Additional methods are provided to add test cases and suites to the + collection later on. \end{classdesc} \begin{classdesc}{TestLoader}{} This class is responsible for loading tests according to various criteria and returning them wrapped in a \class{TestSuite}. It can load all tests within a given module or \class{TestCase} - class. When loading from a module, it considers all - \class{TestCase}-derived classes. For each such class, it creates - an instance for each method with a name beginning with the string - \samp{test}. + subclass. \end{classdesc} \begin{datadesc}{defaultTestLoader} - Instance of the \class{TestLoader} class which can be shared. If no + Instance of the \class{TestLoader} class intended to be shared. If no customization of the \class{TestLoader} is needed, this instance can - always be used instead of creating new instances. + be used instead of repeatedly creating new instances. \end{datadesc} \begin{classdesc}{TextTestRunner}{\optional{stream\optional{, descriptions\optional{, verbosity}}}} A basic test runner implementation which prints results on standard - output. It has a few configurable parameters, but is essentially + error. It has a few configurable parameters, but is essentially very simple. Graphical applications which run test suites should provide alternate implementations. \end{classdesc} @@ -510,7 +521,8 @@ versions may treat \exception{AssertionError} differently.} testRunner\optional{, testRunner}}}}}} A command-line program that runs a set of tests; this is primarily for making test modules conveniently executable. The simplest use - for this function is: + for this function is to include the following line at the end of a + test script: \begin{verbatim} if __name__ == '__main__': @@ -518,10 +530,11 @@ if __name__ == '__main__': \end{verbatim} \end{funcdesc} -In some cases, the existing tests may have be written using the +In some cases, the existing tests may have been written using the \refmodule{doctest} module. If so, that module provides a \class{DocTestSuite} class that can automatically build -\class{unittest.TestSuite} instances from the existing test code. +\class{unittest.TestSuite} instances from the existing +\module{doctest}-based tests. \versionadded{2.3} @@ -538,7 +551,7 @@ used to run the test, another used by the test implementation to check conditions and report failures, and some inquiry methods allowing information about the test itself to be gathered. -Methods in the first group are: +Methods in the first group (running the test) are: \begin{methoddesc}[TestCase]{setUp}{} Method called to prepare the test fixture. This is called @@ -562,8 +575,10 @@ Methods in the first group are: Run the test, collecting the result into the test result object passed as \var{result}. If \var{result} is omitted or \constant{None}, a temporary result object is created and used, but is not made - available to the caller. This is equivalent to simply calling the - \class{TestCase} instance. + available to the caller. + + The same effect may be had by simply calling the \class{TestCase} + instance. \end{methoddesc} \begin{methoddesc}[TestCase]{debug}{} @@ -664,10 +679,8 @@ Testing frameworks can use the following methods to collect information on the test: \begin{methoddesc}[TestCase]{countTestCases}{} - Return the number of tests represented by the this test object. For - \class{TestCase} instances, this will always be \code{1}, but this - method is also implemented by the \class{TestSuite} class, which can - return larger values. + Return the number of tests represented by this test object. For + \class{TestCase} instances, this will always be \code{1}. \end{methoddesc} \begin{methoddesc}[TestCase]{defaultTestResult}{} @@ -678,7 +691,7 @@ information on the test: \begin{methoddesc}[TestCase]{id}{} Return a string identifying the specific test case. This is usually the full name of the test method, including the module and class - names. + name. \end{methoddesc} \begin{methoddesc}[TestCase]{shortDescription}{} @@ -694,21 +707,23 @@ information on the test: \class{TestSuite} objects behave much like \class{TestCase} objects, except they do not actually implement a test. Instead, they are used -to aggregate tests into groups that should be run together. Some -additional methods are available to add tests to \class{TestSuite} +to aggregate tests into groups of tests that should be run together. +Some additional methods are available to add tests to \class{TestSuite} instances: \begin{methoddesc}[TestSuite]{addTest}{test} - Add a \class{TestCase} or \class{TestSuite} to the set of tests that - make up the suite. + Add a \class{TestCase} or \class{TestSuite} to the suite. \end{methoddesc} \begin{methoddesc}[TestSuite]{addTests}{tests} - Add all the tests from a sequence of \class{TestCase} and + Add all the tests from an iterable of \class{TestCase} and \class{TestSuite} instances to this test suite. + + This is equivalent to iterating over \var{tests}, calling + \method{addTest()} for each element. \end{methoddesc} -The \method{run()} method is also slightly different: +\class{TestSuite} shares the following methods with \class{TestCase}: \begin{methoddesc}[TestSuite]{run}{result} Run the tests associated with this suite, collecting the result into @@ -717,6 +732,17 @@ The \method{run()} method is also slightly different: result object to be passed in. \end{methoddesc} +\begin{methoddesc}[TestSuite]{debug}{} + Run the tests associated with this suite without collecting the result. + This allows exceptions raised by the test to be propagated to the caller + and can be used to support running tests under a debugger. +\end{methoddesc} + +\begin{methoddesc}[TestSuite]{countTestCases}{} + Return the number of tests represented by this test object, including + all individual tests and sub-suites. +\end{methoddesc} + In the typical usage of a \class{TestSuite} object, the \method{run()} method is invoked by a \class{TestRunner} rather than by the end-user test harness. @@ -727,7 +753,7 @@ test harness. A \class{TestResult} object stores the results of a set of tests. The \class{TestCase} and \class{TestSuite} classes ensure that results are -properly stored; test authors do not need to worry about recording the +properly recorded; test authors do not need to worry about recording the outcome of tests. Testing frameworks built on top of \refmodule{unittest} may want @@ -745,28 +771,41 @@ formatted version of the traceback for the exception. be of interest when inspecting the results of running a set of tests: \begin{memberdesc}[TestResult]{errors} - A list containing pairs of \class{TestCase} instances and the - formatted tracebacks for tests which raised an exception but did not - signal a test failure. + A list containing 2-tuples of \class{TestCase} instances and + formatted tracebacks. Each tuple represents a test which raised an + unexpected exception. \versionchanged[Contains formatted tracebacks instead of \function{sys.exc_info()} results]{2.2} \end{memberdesc} \begin{memberdesc}[TestResult]{failures} - A list containing pairs of \class{TestCase} instances and the - formatted tracebacks for tests which signalled a failure in the code - under test. + A list containing 2-tuples of \class{TestCase} instances and + formatted tracebacks. Each tuple represents a test where a failure + was explicitly signalled using the \method{TestCase.fail*()} or + \method{TestCase.assert*()} methods. \versionchanged[Contains formatted tracebacks instead of \function{sys.exc_info()} results]{2.2} \end{memberdesc} \begin{memberdesc}[TestResult]{testsRun} - The number of tests which have been started. + The total number of tests run so far. \end{memberdesc} \begin{methoddesc}[TestResult]{wasSuccessful}{} - Returns true if all tests run so far have passed, otherwise returns - false. + Returns \constant{True} if all tests run so far have passed, + otherwise returns \constant{False}. +\end{methoddesc} + +\begin{methoddesc}[TestResult]{stop}{} + This method can be called to signal that the set of tests being run + should be aborted by setting the \class{TestResult}'s \code{shouldStop} + attribute to \constant{True}. \class{TestRunner} objects should respect + this flag and return without running any additional tests. + + For example, this feature is used by the \class{TextTestRunner} class + to stop the test framework when the user signals an interrupt from + the keyboard. Interactive tools which provide \class{TestRunner} + implementations can use this in a similar manner. \end{methoddesc} @@ -786,10 +825,9 @@ reporting while tests are being run. \end{methoddesc} \begin{methoddesc}[TestResult]{addError}{test, err} - Called when the test case \var{test} raises an exception without - signalling a test failure. \var{err} is a tuple of the form - returned by \function{sys.exc_info()}: \code{(\var{type}, - \var{value}, \var{traceback})}. + Called when the test case \var{test} raises an unexpected exception + \var{err} is a tuple of the form returned by \function{sys.exc_info()}: + \code{(\var{type}, \var{value}, \var{traceback})}. \end{methoddesc} \begin{methoddesc}[TestResult]{addFailure}{test, err} @@ -800,23 +838,10 @@ reporting while tests are being run. \end{methoddesc} \begin{methoddesc}[TestResult]{addSuccess}{test} - This method is called for a test that does not fail; \var{test} is - the test case object. + Called when the test case \var{test} succeeds. \end{methoddesc} -One additional method is available for \class{TestResult} objects: - -\begin{methoddesc}[TestResult]{stop}{} - This method can be called to signal that the set of tests being run - should be aborted. Once this has been called, the - \class{TestRunner} object return to its caller without running any - additional tests. This is used by the \class{TextTestRunner} class - to stop the test framework when the user signals an interrupt from - the keyboard. Interactive tools which provide runners can use this - in a similar manner. -\end{methoddesc} - \subsection{TestLoader Objects \label{testloader-objects}} @@ -824,15 +849,15 @@ One additional method is available for \class{TestResult} objects: The \class{TestLoader} class is used to create test suites from classes and modules. Normally, there is no need to create an instance of this class; the \refmodule{unittest} module provides an instance -that can be shared as the \code{defaultTestLoader} module attribute. -Using a subclass or instance would allow customization of some +that can be shared as \code{unittest.defaultTestLoader}. +Using a subclass or instance, however, allows customization of some configurable properties. \class{TestLoader} objects have the following methods: \begin{methoddesc}[TestLoader]{loadTestsFromTestCase}{testCaseClass} Return a suite of all tests cases contained in the - \class{TestCase}-derived class \class{testCaseClass}. + \class{TestCase}-derived \class{testCaseClass}. \end{methoddesc} \begin{methoddesc}[TestLoader]{loadTestsFromModule}{module} @@ -842,7 +867,7 @@ configurable properties. method defined for the class. \warning{While using a hierarchy of - \class{Testcase}-derived classes can be convenient in sharing + \class{TestCase}-derived classes can be convenient in sharing fixtures and helper functions, defining test methods on base classes that are not intended to be instantiated directly does not play well with this method. Doing so, however, can be useful when the @@ -853,21 +878,23 @@ configurable properties. Return a suite of all tests cases given a string specifier. The specifier \var{name} is a ``dotted name'' that may resolve - either to a module, a test case class, a test method within a test - case class, or a callable object which returns a \class{TestCase} or - \class{TestSuite} instance. For example, if you have a module - \module{SampleTests} containing a \class{TestCase}-derived class - \class{SampleTestCase} with three test methods (\method{test_one()}, - \method{test_two()}, and \method{test_three()}), the specifier - \code{'SampleTests.SampleTestCase'} would cause this method to - return a suite which will run all three test methods. Using the - specifier \code{'SampleTests.SampleTestCase.test_two'} would cause - it to return a test suite which will run only the + either to a module, a test case class, a \class{TestSuite} instance, + a test method within a test case class, or a callable object which + returns a \class{TestCase} or \class{TestSuite} instance. + + For example, if you have a module \module{SampleTests} containing a + \class{TestCase}-derived class \class{SampleTestCase} with three test + methods (\method{test_one()}, \method{test_two()}, and + \method{test_three()}), the specifier \code{'SampleTests.SampleTestCase'} + would cause this method to return a suite which will run all three test + methods. Using the specifier \code{'SampleTests.SampleTestCase.test_two'} + would cause it to return a test suite which will run only the \method{test_two()} test method. The specifier can refer to modules and packages which have not been imported; they will be imported as a side-effect. - The method optionally resolves \var{name} relative to a given module. + The method optionally resolves \var{name} relative to the given + \var{module}. \end{methoddesc} \begin{methoddesc}[TestLoader]{loadTestsFromNames}{names\optional{, module}} @@ -888,17 +915,22 @@ either by subclassing or assignment on an instance: \begin{memberdesc}[TestLoader]{testMethodPrefix} String giving the prefix of method names which will be interpreted as test methods. The default value is \code{'test'}. + + This affects \method{getTestCaseNames()} and all the + \method{loadTestsFrom*()} methods. \end{memberdesc} \begin{memberdesc}[TestLoader]{sortTestMethodsUsing} Function to be used to compare method names when sorting them in - \method{getTestCaseNames()}. The default value is the built-in - \function{cmp()} function; it can be set to \constant{None} to disable - the sort. + \method{getTestCaseNames()} and all the \method{loadTestsFrom*()} methods. + The default value is the built-in \function{cmp()} function; the attribute + can also be set to \constant{None} to disable the sort. \end{memberdesc} \begin{memberdesc}[TestLoader]{suiteClass} Callable object that constructs a test suite from a list of tests. No methods on the resulting object are needed. The default value is the \class{TestSuite} class. + + This affects all the \method{loadTestsFrom*()} methods. \end{memberdesc} diff --git a/Doc/lib/liburllib.tex b/Doc/lib/liburllib.tex index 0a84c1a..75ee310 100644 --- a/Doc/lib/liburllib.tex +++ b/Doc/lib/liburllib.tex @@ -270,10 +270,10 @@ off completely. Its default value is \code{None}, in which case environmental proxy settings will be used if present, as discussed in the definition of \function{urlopen()}, above. -Additional keyword parameters, collected in \var{x509}, are used for -authentication with the \file{https:} scheme. The keywords -\var{key_file} and \var{cert_file} are supported; both are needed to -actually retrieve a resource at an \file{https:} URL. +Additional keyword parameters, collected in \var{x509}, may be used for +authentication of the client when using the \file{https:} scheme. The keywords +\var{key_file} and \var{cert_file} are supported to provide an +SSL key and certificate; both are needed to support client authentication. \class{URLopener} objects will raise an \exception{IOError} exception if the server returns an error code. diff --git a/Doc/lib/liburllib2.tex b/Doc/lib/liburllib2.tex index f4351c3..542a7b8 100644 --- a/Doc/lib/liburllib2.tex +++ b/Doc/lib/liburllib2.tex @@ -19,7 +19,8 @@ Open the URL \var{url}, which can be either a string or a \class{Request} object. \var{data} may be a string specifying additional data to send to the -server. Currently HTTP requests are the only ones that use \var{data}; +server, or \code{None} if no such data is needed. +Currently HTTP requests are the only ones that use \var{data}; the HTTP request will be a POST instead of a GET when the \var{data} parameter is provided. \var{data} should be a buffer in the standard \mimetype{application/x-www-form-urlencoded} format. The @@ -97,8 +98,17 @@ The following classes are provided: \optional{, origin_req_host}\optional{, unverifiable}} This class is an abstraction of a URL request. -\var{url} should be a string which is a valid URL. For a description -of \var{data} see the \method{add_data()} description. +\var{url} should be a string containing a valid URL. + +\var{data} may be a string specifying additional data to send to the +server, or \code{None} if no such data is needed. +Currently HTTP requests are the only ones that use \var{data}; +the HTTP request will be a POST instead of a GET when the \var{data} +parameter is provided. \var{data} should be a buffer in the standard +\mimetype{application/x-www-form-urlencoded} format. The +\function{urllib.urlencode()} function takes a mapping or sequence of +2-tuples and returns a string in this format. + \var{headers} should be a dictionary, and will be treated as if \method{add_header()} was called with each key and value as arguments. diff --git a/Doc/lib/libuuid.tex b/Doc/lib/libuuid.tex new file mode 100644 index 0000000..a9d5295 --- /dev/null +++ b/Doc/lib/libuuid.tex @@ -0,0 +1,234 @@ +\section{\module{uuid} --- + UUID objects according to RFC 4122} +\declaremodule{builtin}{uuid} +\modulesynopsis{UUID objects (universally unique identifiers) according to RFC 4122} +\moduleauthor{Ka-Ping Yee}{ping@zesty.ca} +\sectionauthor{George Yoshida}{quiver@users.sourceforge.net} + +\versionadded{2.5} + +This module provides immutable \class{UUID} objects (the \class{UUID} class) +and the functions \function{uuid1()}, \function{uuid3()}, +\function{uuid4()}, \function{uuid5()} for generating version 1, 3, 4, +and 5 UUIDs as specified in \rfc{4122}. + +If all you want is a unique ID, you should probably call +\function{uuid1()} or \function{uuid4()}. Note that \function{uuid1()} +may compromise privacy since it creates a UUID containing the computer's +network address. \function{uuid4()} creates a random UUID. + +\begin{classdesc}{UUID}{\optional{hex\optional{, bytes\optional{, +fields\optional{, int\optional{, version}}}}}} + +%Instances of the UUID class represent UUIDs as specified in RFC 4122. +%UUID objects are immutable, hashable, and usable as dictionary keys. +%Converting a UUID to a string with str() yields something in the form +%'12345678-1234-1234-1234-123456789abc'. The UUID constructor accepts +%four possible forms: a similar string of hexadecimal digits, or a +%string of 16 raw bytes as an argument named 'bytes', or a tuple of +%six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and +%48-bit values respectively) as an argument named 'fields', or a single +%128-bit integer as an argument named 'int'. + +Create a UUID from either a string of 32 hexadecimal digits, +a string of 16 bytes as the \var{bytes} argument, a tuple of six +integers (32-bit \var{time_low}, 16-bit \var{time_mid}, +16-bit \var{time_hi_version}, +8-bit \var{clock_seq_hi_variant}, 8-bit \var{clock_seq_low}, 48-bit \var{node}) +as the \var{fields} argument, or a single 128-bit integer as the \var{int} +argument. When a string of hex digits is given, curly braces, +hyphens, and a URN prefix are all optional. For example, these +expressions all yield the same UUID: + +\begin{verbatim} +UUID('{12345678-1234-5678-1234-567812345678}') +UUID('12345678123456781234567812345678') +UUID('urn:uuid:12345678-1234-5678-1234-567812345678') +UUID(bytes='\x12\x34\x56\x78'*4) +UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) +UUID(int=0x12345678123456781234567812345678) +\end{verbatim} + +Exactly one of \var{hex}, \var{bytes}, \var{fields}, or \var{int} must +be given. The \var{version} argument is optional; if given, the +resulting UUID will have its variant and version number set according to +RFC 4122, overriding bits in the given \var{hex}, \var{bytes}, +\var{fields}, or \var{int}. + +\end{classdesc} + +\class{UUID} instances have these read-only attributes: + +\begin{memberdesc}{bytes} +The UUID as a 16-byte string. +\end{memberdesc} + +\begin{memberdesc}{fields} +A tuple of the six integer fields of the UUID, which are also available +as six individual attributes and two derived attributes: + +\begin{tableii}{l|l}{member}{Field}{Meaning} + \lineii{time_low}{the first 32 bits of the UUID} + \lineii{time_mid}{the next 16 bits of the UUID} + \lineii{time_hi_version}{the next 16 bits of the UUID} + \lineii{clock_seq_hi_variant}{the next 8 bits of the UUID} + \lineii{clock_seq_low}{the next 8 bits of the UUID} + \lineii{node}{the last 48 bits of the UUID} + \lineii{time}{the 60-bit timestamp} + \lineii{clock_seq}{the 14-bit sequence number} +\end{tableii} + + +\end{memberdesc} + +\begin{memberdesc}{hex} +The UUID as a 32-character hexadecimal string. +\end{memberdesc} + +\begin{memberdesc}{int} +The UUID as a 128-bit integer. +\end{memberdesc} + +\begin{memberdesc}{urn} +The UUID as a URN as specified in RFC 4122. +\end{memberdesc} + +\begin{memberdesc}{variant} +The UUID variant, which determines the internal layout of the UUID. +This will be an integer equal to one of the constants +\constant{RESERVED_NCS}, +\constant{RFC_4122}, \constant{RESERVED_MICROSOFT}, or +\constant{RESERVED_FUTURE}). +\end{memberdesc} + +\begin{memberdesc}{version} +The UUID version number (1 through 5, meaningful only +when the variant is \constant{RFC_4122}). +\end{memberdesc} + +The \module{uuid} module defines the following functions + +\begin{funcdesc}{getnode}{} +Get the hardware address as a 48-bit positive integer. The first time this +runs, it may launch a separate program, which could be quite slow. If all +attempts to obtain the hardware address fail, we choose a random 48-bit +number with its eighth bit set to 1 as recommended in RFC 4122. "Hardware +address" means the MAC address of a network interface, and on a machine +with multiple network interfaces the MAC address of any one of them may +be returned. +\end{funcdesc} +\index{getnode} + +\begin{funcdesc}{uuid1}{\optional{node\optional{, clock_seq}}} +Generate a UUID from a host ID, sequence number, and the current time. +If \var{node} is not given, \function{getnode()} is used to obtain the +hardware address. +If \var{clock_seq} is given, it is used as the sequence number; +otherwise a random 14-bit sequence number is chosen. +\end{funcdesc} +\index{uuid1} + +\begin{funcdesc}{uuid3}{namespace, name} +Generate a UUID based upon a MD5 hash of the \var{name} string value +drawn from a specified namespace. \var{namespace} +must be one of \constant{NAMESPACE_DNS}, +\constant{NAMESPACE_URL}, \constant{NAMESPACE_OID}, +or \constant{NAMESPACE_X500}. +\end{funcdesc} +\index{uuid3} + +\begin{funcdesc}{uuid4}{} +Generate a random UUID. +\end{funcdesc} +\index{uuid4} + +\begin{funcdesc}{uuid5}{namespace, name} +Generate a UUID based upon a SHA-1 hash of the \var{name} string value +drawn from a specified namespace. \var{namespace} +must be one of \constant{NAMESPACE_DNS}, +\constant{NAMESPACE_URL}, \constant{NAMESPACE_OID}, +or \constant{NAMESPACE_X500}. +\end{funcdesc} +\index{uuid5} + +The \module{uuid} module defines the following namespace constants +for use with \function{uuid3()} or \function{uuid5()}. + +\begin{datadesc}{NAMESPACE_DNS} +Fully-qualified domain name namespace UUID. +\end{datadesc} + +\begin{datadesc}{NAMESPACE_URL} +URL namespace UUID. +\end{datadesc} + +\begin{datadesc}{NAMESPACE_OID} +ISO OID namespace UUID. +\end{datadesc} + +\begin{datadesc}{NAMESPACE_X500} +X.500 DN namespace UUID. +\end{datadesc} + +The \module{uuid} module defines the following constants +for the possible values of the \member{variant} attribute: + +\begin{datadesc}{RESERVED_NCS} +Reserved for NCS compatibility. +\end{datadesc} + +\begin{datadesc}{RFC_4122} +Uses UUID layout specified in \rfc{4122}. +\end{datadesc} + +\begin{datadesc}{RESERVED_MICROSOFT} +Reserved for Microsoft backward compatibility. +\end{datadesc} + +\begin{datadesc}{RESERVED_FUTURE} +Reserved for future definition. +\end{datadesc} + + +\begin{seealso} + \seerfc{4122}{A Universally Unique IDentifier (UUID) URN Namespace}{ + This specifies a Uniform Resource Name namespace for UUIDs.} +\end{seealso} + +\subsection{Example \label{uuid-example}} + +Here is a typical usage: +\begin{verbatim} +>>> import uuid + +# make a UUID based on the host ID and current time +>>> uuid.uuid1() +UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') + +# make a UUID using an MD5 hash of a namespace UUID and a name +>>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') +UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') + +# make a random UUID +>>> uuid.uuid4() +UUID('16fd2706-8baf-433b-82eb-8c7fada847da') + +# make a UUID using a SHA-1 hash of a namespace UUID and a name +>>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') +UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') + +# make a UUID from a string of hex digits (braces and hyphens ignored) +>>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}') + +# convert a UUID to a string of hex digits in standard form +>>> str(x) +'00010203-0405-0607-0809-0a0b0c0d0e0f' + +# get the raw 16 bytes of the UUID +>>> x.bytes +'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f' + +# make a UUID from a 16-byte string +>>> uuid.UUID(bytes=x.bytes) +UUID('00010203-0405-0607-0809-0a0b0c0d0e0f') +\end{verbatim} diff --git a/Doc/lib/libwarnings.tex b/Doc/lib/libwarnings.tex index 7b829a0..08c0340 100644 --- a/Doc/lib/libwarnings.tex +++ b/Doc/lib/libwarnings.tex @@ -71,6 +71,11 @@ runtime features.} \lineii{FutureWarning}{Base category for warnings about constructs that will change semantically in the future.} +\lineii{PendingDeprecationWarning}{Base category for warnings about +features that will be deprecated in the future (ignored by default).} + +\lineii{ImportWarning}{Base category for warnings triggered during the +process of importing a module (ignored by default).} \end{tableii} While these are technically built-in exceptions, they are documented @@ -143,6 +148,17 @@ arguments for all \programopt{-W} options without interpretation in it is first imported (invalid options are ignored, after printing a message to \code{sys.stderr}). +The warnings that are ignored by default may be enabled by passing + \programopt{-Wd} to the interpreter. This enables default handling +for all warnings, including those that are normally ignored by +default. This is particular useful for enabling ImportWarning when +debugging problems importing a developed package. ImportWarning can +also be enabled explicitly in Python code using: + +\begin{verbatim} + warnings.simplefilter('default', ImportWarning) +\end{verbatim} + \subsection{Available Functions \label{warning-functions}} @@ -209,14 +225,26 @@ Insert an entry into the list of warnings filters. The entry is inserted at the front by default; if \var{append} is true, it is inserted at the end. This checks the types of the arguments, compiles the message and -module regular expressions, and inserts them as a tuple in front -of the warnings filter. Entries inserted later override entries -inserted earlier, if both match a particular warning. Omitted -arguments default to a value that matches everything. +module regular expressions, and inserts them as a tuple in the +list of warnings filters. Entries closer to the front of the list +override entries later in the list, if both match a particular +warning. Omitted arguments default to a value that matches +everything. +\end{funcdesc} + +\begin{funcdesc}{simplefilter}{action\optional{, + category\optional{, + lineno\optional{, append}}}} +Insert a simple entry into the list of warnings filters. The meaning +of the function parameters is as for \function{filterwarnings()}, but +regular expressions are not needed as the filter inserted always +matches any message in any module as long as the category and line +number match. \end{funcdesc} \begin{funcdesc}{resetwarnings}{} Reset the warnings filter. This discards the effect of all previous calls to \function{filterwarnings()}, including that of the -\programopt{-W} command line options. +\programopt{-W} command line options and calls to +\function{simplefilter()}. \end{funcdesc} diff --git a/Doc/lib/libweakref.tex b/Doc/lib/libweakref.tex index fc949e6..6f676a2 100644 --- a/Doc/lib/libweakref.tex +++ b/Doc/lib/libweakref.tex @@ -65,10 +65,14 @@ class Dict(dict): obj = Dict(red=1, green=2, blue=3) # this object is weak referencable \end{verbatim} -Extension types can easily be made to support weak references; see section -\ref{weakref-extension}, ``Weak References in Extension Types,'' for more -information. - +Extension types can easily be made to support weak references; see +``\ulink{Weak Reference Support}{../ext/weakref-support.html}'' in +\citetitle[../ext/ext.html]{Extending and Embedding the Python +Interpreter}. +% The referenced section used to appear in this document with the +% \label weakref-extension. It would be good to be able to generate a +% redirect for the corresponding HTML page (weakref-extension.html) +% for on-line versions of this document. \begin{classdesc}{ref}{object\optional{, callback}} Return a weak reference to \var{object}. The original object can be @@ -330,83 +334,3 @@ def remember(obj): def id2obj(oid): return _id2obj_dict[oid] \end{verbatim} - - -\subsection{Weak References in Extension Types - \label{weakref-extension}} - -One of the goals of the implementation is to allow any type to -participate in the weak reference mechanism without incurring the -overhead on those objects which do not benefit by weak referencing -(such as numbers). - -For an object to be weakly referencable, the extension must include a -\ctype{PyObject*} field in the instance structure for the use of the -weak reference mechanism; it must be initialized to \NULL{} by the -object's constructor. It must also set the \member{tp_weaklistoffset} -field of the corresponding type object to the offset of the field. -Also, it needs to add \constant{Py_TPFLAGS_HAVE_WEAKREFS} to the -tp_flags slot. For example, the instance type is defined with the -following structure: - -\begin{verbatim} -typedef struct { - PyObject_HEAD - PyClassObject *in_class; /* The class object */ - PyObject *in_dict; /* A dictionary */ - PyObject *in_weakreflist; /* List of weak references */ -} PyInstanceObject; -\end{verbatim} - -The statically-declared type object for instances is defined this way: - -\begin{verbatim} -PyTypeObject PyInstance_Type = { - PyObject_HEAD_INIT(&PyType_Type) - 0, - "module.instance", - - /* Lots of stuff omitted for brevity... */ - - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_WEAKREFS /* tp_flags */ - 0, /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - offsetof(PyInstanceObject, in_weakreflist), /* tp_weaklistoffset */ -}; -\end{verbatim} - -The type constructor is responsible for initializing the weak reference -list to \NULL: - -\begin{verbatim} -static PyObject * -instance_new() { - /* Other initialization stuff omitted for brevity */ - - self->in_weakreflist = NULL; - - return (PyObject *) self; -} -\end{verbatim} - -The only further addition is that the destructor needs to call the -weak reference manager to clear any weak references. This should be -done before any other parts of the destruction have occurred, but is -only required if the weak reference list is non-\NULL: - -\begin{verbatim} -static void -instance_dealloc(PyInstanceObject *inst) -{ - /* Allocate temporaries if needed, but do not begin - destruction just yet. - */ - - if (inst->in_weakreflist != NULL) - PyObject_ClearWeakRefs((PyObject *) inst); - - /* Proceed with object destruction normally. */ -} -\end{verbatim} diff --git a/Doc/lib/libwebbrowser.tex b/Doc/lib/libwebbrowser.tex index e86b578..11d77a1 100644 --- a/Doc/lib/libwebbrowser.tex +++ b/Doc/lib/libwebbrowser.tex @@ -136,6 +136,18 @@ Library Modules} manual. Only on MacOS X platform. \end{description} +Here are some simple examples: + +\begin{verbatim} +url = 'http://www.python.org' + +# Open URL in a new tab, if a browser window is already open. +webbrowser.open_new_tab(url + '/doc') + +# Open URL in new window, raising the window if possible. +webbrowser.open_new(url) +\end{verbatim} + \subsection{Browser Controller Objects \label{browser-controllers}} diff --git a/Doc/lib/libwsgiref.tex b/Doc/lib/libwsgiref.tex new file mode 100755 index 0000000..4b12e9d --- /dev/null +++ b/Doc/lib/libwsgiref.tex @@ -0,0 +1,781 @@ +\section{\module{wsgiref} --- WSGI Utilities and Reference +Implementation} +\declaremodule{}{wsgiref} +\moduleauthor{Phillip J. Eby}{pje@telecommunity.com} +\sectionauthor{Phillip J. Eby}{pje@telecommunity.com} +\modulesynopsis{WSGI Utilities and Reference Implementation} + +\versionadded{2.5} + +The Web Server Gateway Interface (WSGI) is a standard interface +between web server software and web applications written in Python. +Having a standard interface makes it easy to use an application +that supports WSGI with a number of different web servers. + +Only authors of web servers and programming frameworks need to know +every detail and corner case of the WSGI design. You don't need to +understand every detail of WSGI just to install a WSGI application or +to write a web application using an existing framework. + +\module{wsgiref} is a reference implementation of the WSGI specification +that can be used to add WSGI support to a web server or framework. It +provides utilities for manipulating WSGI environment variables and +response headers, base classes for implementing WSGI servers, a demo +HTTP server that serves WSGI applications, and a validation tool that +checks WSGI servers and applications for conformance to the +WSGI specification (\pep{333}). + +% XXX If you're just trying to write a web application... +% XXX should create a URL on python.org to point people to. + + + + + + + + + + + + + + +\subsection{\module{wsgiref.util} -- WSGI environment utilities} +\declaremodule{}{wsgiref.util} + +This module provides a variety of utility functions for working with +WSGI environments. A WSGI environment is a dictionary containing +HTTP request variables as described in \pep{333}. All of the functions +taking an \var{environ} parameter expect a WSGI-compliant dictionary to +be supplied; please see \pep{333} for a detailed specification. + +\begin{funcdesc}{guess_scheme}{environ} +Return a guess for whether \code{wsgi.url_scheme} should be ``http'' or +``https'', by checking for a \code{HTTPS} environment variable in the +\var{environ} dictionary. The return value is a string. + +This function is useful when creating a gateway that wraps CGI or a +CGI-like protocol such as FastCGI. Typically, servers providing such +protocols will include a \code{HTTPS} variable with a value of ``1'' +``yes'', or ``on'' when a request is received via SSL. So, this +function returns ``https'' if such a value is found, and ``http'' +otherwise. +\end{funcdesc} + +\begin{funcdesc}{request_uri}{environ \optional{, include_query=1}} +Return the full request URI, optionally including the query string, +using the algorithm found in the ``URL Reconstruction'' section of +\pep{333}. If \var{include_query} is false, the query string is +not included in the resulting URI. +\end{funcdesc} + +\begin{funcdesc}{application_uri}{environ} +Similar to \function{request_uri}, except that the \code{PATH_INFO} and +\code{QUERY_STRING} variables are ignored. The result is the base URI +of the application object addressed by the request. +\end{funcdesc} + +\begin{funcdesc}{shift_path_info}{environ} +Shift a single name from \code{PATH_INFO} to \code{SCRIPT_NAME} and +return the name. The \var{environ} dictionary is \emph{modified} +in-place; use a copy if you need to keep the original \code{PATH_INFO} +or \code{SCRIPT_NAME} intact. + +If there are no remaining path segments in \code{PATH_INFO}, \code{None} +is returned. + +Typically, this routine is used to process each portion of a request +URI path, for example to treat the path as a series of dictionary keys. +This routine modifies the passed-in environment to make it suitable for +invoking another WSGI application that is located at the target URI. +For example, if there is a WSGI application at \code{/foo}, and the +request URI path is \code{/foo/bar/baz}, and the WSGI application at +\code{/foo} calls \function{shift_path_info}, it will receive the string +``bar'', and the environment will be updated to be suitable for passing +to a WSGI application at \code{/foo/bar}. That is, \code{SCRIPT_NAME} +will change from \code{/foo} to \code{/foo/bar}, and \code{PATH_INFO} +will change from \code{/bar/baz} to \code{/baz}. + +When \code{PATH_INFO} is just a ``/'', this routine returns an empty +string and appends a trailing slash to \code{SCRIPT_NAME}, even though +empty path segments are normally ignored, and \code{SCRIPT_NAME} doesn't +normally end in a slash. This is intentional behavior, to ensure that +an application can tell the difference between URIs ending in \code{/x} +from ones ending in \code{/x/} when using this routine to do object +traversal. + +\end{funcdesc} + +\begin{funcdesc}{setup_testing_defaults}{environ} +Update \var{environ} with trivial defaults for testing purposes. + +This routine adds various parameters required for WSGI, including +\code{HTTP_HOST}, \code{SERVER_NAME}, \code{SERVER_PORT}, +\code{REQUEST_METHOD}, \code{SCRIPT_NAME}, \code{PATH_INFO}, and all of +the \pep{333}-defined \code{wsgi.*} variables. It only supplies default +values, and does not replace any existing settings for these variables. + +This routine is intended to make it easier for unit tests of WSGI +servers and applications to set up dummy environments. It should NOT +be used by actual WSGI servers or applications, since the data is fake! +\end{funcdesc} + + + +In addition to the environment functions above, the +\module{wsgiref.util} module also provides these miscellaneous +utilities: + +\begin{funcdesc}{is_hop_by_hop}{header_name} +Return true if 'header_name' is an HTTP/1.1 ``Hop-by-Hop'' header, as +defined by \rfc{2616}. +\end{funcdesc} + +\begin{classdesc}{FileWrapper}{filelike \optional{, blksize=8192}} +A wrapper to convert a file-like object to an iterator. The resulting +objects support both \method{__getitem__} and \method{__iter__} +iteration styles, for compatibility with Python 2.1 and Jython. +As the object is iterated over, the optional \var{blksize} parameter +will be repeatedly passed to the \var{filelike} object's \method{read()} +method to obtain strings to yield. When \method{read()} returns an +empty string, iteration is ended and is not resumable. + +If \var{filelike} has a \method{close()} method, the returned object +will also have a \method{close()} method, and it will invoke the +\var{filelike} object's \method{close()} method when called. +\end{classdesc} + + + + + + + + + + + + + + + + + + + +\subsection{\module{wsgiref.headers} -- WSGI response header tools} +\declaremodule{}{wsgiref.headers} + +This module provides a single class, \class{Headers}, for convenient +manipulation of WSGI response headers using a mapping-like interface. + +\begin{classdesc}{Headers}{headers} +Create a mapping-like object wrapping \var{headers}, which must be a +list of header name/value tuples as described in \pep{333}. Any changes +made to the new \class{Headers} object will directly update the +\var{headers} list it was created with. + +\class{Headers} objects support typical mapping operations including +\method{__getitem__}, \method{get}, \method{__setitem__}, +\method{setdefault}, \method{__delitem__}, \method{__contains__} and +\method{has_key}. For each of these methods, the key is the header name +(treated case-insensitively), and the value is the first value +associated with that header name. Setting a header deletes any existing +values for that header, then adds a new value at the end of the wrapped +header list. Headers' existing order is generally maintained, with new +headers added to the end of the wrapped list. + +Unlike a dictionary, \class{Headers} objects do not raise an error when +you try to get or delete a key that isn't in the wrapped header list. +Getting a nonexistent header just returns \code{None}, and deleting +a nonexistent header does nothing. + +\class{Headers} objects also support \method{keys()}, \method{values()}, +and \method{items()} methods. The lists returned by \method{keys()} +and \method{items()} can include the same key more than once if there +is a multi-valued header. The \code{len()} of a \class{Headers} object +is the same as the length of its \method{items()}, which is the same +as the length of the wrapped header list. In fact, the \method{items()} +method just returns a copy of the wrapped header list. + +Calling \code{str()} on a \class{Headers} object returns a formatted +string suitable for transmission as HTTP response headers. Each header +is placed on a line with its value, separated by a colon and a space. +Each line is terminated by a carriage return and line feed, and the +string is terminated with a blank line. + +In addition to their mapping interface and formatting features, +\class{Headers} objects also have the following methods for querying +and adding multi-valued headers, and for adding headers with MIME +parameters: + +\begin{methoddesc}{get_all}{name} +Return a list of all the values for the named header. + +The returned list will be sorted in the order they appeared in the +original header list or were added to this instance, and may contain +duplicates. Any fields deleted and re-inserted are always appended to +the header list. If no fields exist with the given name, returns an +empty list. +\end{methoddesc} + + +\begin{methoddesc}{add_header}{name, value, **_params} +Add a (possibly multi-valued) header, with optional MIME parameters +specified via keyword arguments. + +\var{name} is the header field to add. Keyword arguments can be used to +set MIME parameters for the header field. Each parameter must be a +string or \code{None}. Underscores in parameter names are converted to +dashes, since dashes are illegal in Python identifiers, but many MIME +parameter names include dashes. If the parameter value is a string, it +is added to the header value parameters in the form \code{name="value"}. +If it is \code{None}, only the parameter name is added. (This is used +for MIME parameters without a value.) Example usage: + +\begin{verbatim} +h.add_header('content-disposition', 'attachment', filename='bud.gif') +\end{verbatim} + +The above will add a header that looks like this: + +\begin{verbatim} +Content-Disposition: attachment; filename="bud.gif" +\end{verbatim} +\end{methoddesc} +\end{classdesc} + +\subsection{\module{wsgiref.simple_server} -- a simple WSGI HTTP server} +\declaremodule[wsgiref.simpleserver]{}{wsgiref.simple_server} + +This module implements a simple HTTP server (based on +\module{BaseHTTPServer}) that serves WSGI applications. Each server +instance serves a single WSGI application on a given host and port. If +you want to serve multiple applications on a single host and port, you +should create a WSGI application that parses \code{PATH_INFO} to select +which application to invoke for each request. (E.g., using the +\function{shift_path_info()} function from \module{wsgiref.util}.) + + +\begin{funcdesc}{make_server}{host, port, app +\optional{, server_class=\class{WSGIServer} \optional{, +handler_class=\class{WSGIRequestHandler}}}} +Create a new WSGI server listening on \var{host} and \var{port}, +accepting connections for \var{app}. The return value is an instance of +the supplied \var{server_class}, and will process requests using the +specified \var{handler_class}. \var{app} must be a WSGI application +object, as defined by \pep{333}. + +Example usage: +\begin{verbatim}from wsgiref.simple_server import make_server, demo_app + +httpd = make_server('', 8000, demo_app) +print "Serving HTTP on port 8000..." + +# Respond to requests until process is killed +httpd.serve_forever() + +# Alternative: serve one request, then exit +##httpd.handle_request() +\end{verbatim} + +\end{funcdesc} + + + + + + +\begin{funcdesc}{demo_app}{environ, start_response} +This function is a small but complete WSGI application that +returns a text page containing the message ``Hello world!'' +and a list of the key/value pairs provided in the +\var{environ} parameter. It's useful for verifying that a WSGI server +(such as \module{wsgiref.simple_server}) is able to run a simple WSGI +application correctly. +\end{funcdesc} + + +\begin{classdesc}{WSGIServer}{server_address, RequestHandlerClass} +Create a \class{WSGIServer} instance. \var{server_address} should be +a \code{(host,port)} tuple, and \var{RequestHandlerClass} should be +the subclass of \class{BaseHTTPServer.BaseHTTPRequestHandler} that will +be used to process requests. + +You do not normally need to call this constructor, as the +\function{make_server()} function can handle all the details for you. + +\class{WSGIServer} is a subclass +of \class{BaseHTTPServer.HTTPServer}, so all of its methods (such as +\method{serve_forever()} and \method{handle_request()}) are available. +\class{WSGIServer} also provides these WSGI-specific methods: + +\begin{methoddesc}{set_app}{application} +Sets the callable \var{application} as the WSGI application that will +receive requests. +\end{methoddesc} + +\begin{methoddesc}{get_app}{} +Returns the currently-set application callable. +\end{methoddesc} + +Normally, however, you do not need to use these additional methods, as +\method{set_app()} is normally called by \function{make_server()}, and +the \method{get_app()} exists mainly for the benefit of request handler +instances. +\end{classdesc} + + + +\begin{classdesc}{WSGIRequestHandler}{request, client_address, server} +Create an HTTP handler for the given \var{request} (i.e. a socket), +\var{client_address} (a \code{(\var{host},\var{port})} tuple), and +\var{server} (\class{WSGIServer} instance). + +You do not need to create instances of this class directly; they are +automatically created as needed by \class{WSGIServer} objects. You +can, however, subclass this class and supply it as a \var{handler_class} +to the \function{make_server()} function. Some possibly relevant +methods for overriding in subclasses: + +\begin{methoddesc}{get_environ}{} +Returns a dictionary containing the WSGI environment for a request. The +default implementation copies the contents of the \class{WSGIServer} +object's \member{base_environ} dictionary attribute and then adds +various headers derived from the HTTP request. Each call to this method +should return a new dictionary containing all of the relevant CGI +environment variables as specified in \pep{333}. +\end{methoddesc} + +\begin{methoddesc}{get_stderr}{} +Return the object that should be used as the \code{wsgi.errors} stream. +The default implementation just returns \code{sys.stderr}. +\end{methoddesc} + +\begin{methoddesc}{handle}{} +Process the HTTP request. The default implementation creates a handler +instance using a \module{wsgiref.handlers} class to implement the actual +WSGI application interface. +\end{methoddesc} + +\end{classdesc} + + + + + + + + + +\subsection{\module{wsgiref.validate} -- WSGI conformance checker} +\declaremodule{}{wsgiref.validate} +When creating new WSGI application objects, frameworks, servers, or +middleware, it can be useful to validate the new code's conformance +using \module{wsgiref.validate}. This module provides a function that +creates WSGI application objects that validate communications between +a WSGI server or gateway and a WSGI application object, to check both +sides for protocol conformance. + +Note that this utility does not guarantee complete \pep{333} compliance; +an absence of errors from this module does not necessarily mean that +errors do not exist. However, if this module does produce an error, +then it is virtually certain that either the server or application is +not 100\% compliant. + +This module is based on the \module{paste.lint} module from Ian +Bicking's ``Python Paste'' library. + +\begin{funcdesc}{validator}{application} +Wrap \var{application} and return a new WSGI application object. The +returned application will forward all requests to the original +\var{application}, and will check that both the \var{application} and +the server invoking it are conforming to the WSGI specification and to +RFC 2616. + +Any detected nonconformance results in an \exception{AssertionError} +being raised; note, however, that how these errors are handled is +server-dependent. For example, \module{wsgiref.simple_server} and other +servers based on \module{wsgiref.handlers} (that don't override the +error handling methods to do something else) will simply output a +message that an error has occurred, and dump the traceback to +\code{sys.stderr} or some other error stream. + +This wrapper may also generate output using the \module{warnings} module +to indicate behaviors that are questionable but which may not actually +be prohibited by \pep{333}. Unless they are suppressed using Python +command-line options or the \module{warnings} API, any such warnings +will be written to \code{sys.stderr} (\emph{not} \code{wsgi.errors}, +unless they happen to be the same object). +\end{funcdesc} + +\subsection{\module{wsgiref.handlers} -- server/gateway base classes} +\declaremodule{}{wsgiref.handlers} + +This module provides base handler classes for implementing WSGI servers +and gateways. These base classes handle most of the work of +communicating with a WSGI application, as long as they are given a +CGI-like environment, along with input, output, and error streams. + + +\begin{classdesc}{CGIHandler}{} +CGI-based invocation via \code{sys.stdin}, \code{sys.stdout}, +\code{sys.stderr} and \code{os.environ}. This is useful when you have +a WSGI application and want to run it as a CGI script. Simply invoke +\code{CGIHandler().run(app)}, where \code{app} is the WSGI application +object you wish to invoke. + +This class is a subclass of \class{BaseCGIHandler} that sets +\code{wsgi.run_once} to true, \code{wsgi.multithread} to false, and +\code{wsgi.multiprocess} to true, and always uses \module{sys} and +\module{os} to obtain the necessary CGI streams and environment. +\end{classdesc} + + +\begin{classdesc}{BaseCGIHandler}{stdin, stdout, stderr, environ +\optional{, multithread=True \optional{, multiprocess=False}}} + +Similar to \class{CGIHandler}, but instead of using the \module{sys} and +\module{os} modules, the CGI environment and I/O streams are specified +explicitly. The \var{multithread} and \var{multiprocess} values are +used to set the \code{wsgi.multithread} and \code{wsgi.multiprocess} +flags for any applications run by the handler instance. + +This class is a subclass of \class{SimpleHandler} intended for use with +software other than HTTP ``origin servers''. If you are writing a +gateway protocol implementation (such as CGI, FastCGI, SCGI, etc.) that +uses a \code{Status:} header to send an HTTP status, you probably want +to subclass this instead of \class{SimpleHandler}. +\end{classdesc} + + + +\begin{classdesc}{SimpleHandler}{stdin, stdout, stderr, environ +\optional{,multithread=True \optional{, multiprocess=False}}} + +Similar to \class{BaseCGIHandler}, but designed for use with HTTP origin +servers. If you are writing an HTTP server implementation, you will +probably want to subclass this instead of \class{BaseCGIHandler} + +This class is a subclass of \class{BaseHandler}. It overrides the +\method{__init__()}, \method{get_stdin()}, \method{get_stderr()}, +\method{add_cgi_vars()}, \method{_write()}, and \method{_flush()} +methods to support explicitly setting the environment and streams via +the constructor. The supplied environment and streams are stored in +the \member{stdin}, \member{stdout}, \member{stderr}, and +\member{environ} attributes. +\end{classdesc} + +\begin{classdesc}{BaseHandler}{} +This is an abstract base class for running WSGI applications. Each +instance will handle a single HTTP request, although in principle you +could create a subclass that was reusable for multiple requests. + +\class{BaseHandler} instances have only one method intended for external +use: + +\begin{methoddesc}{run}{app} +Run the specified WSGI application, \var{app}. +\end{methoddesc} + +All of the other \class{BaseHandler} methods are invoked by this method +in the process of running the application, and thus exist primarily to +allow customizing the process. + +The following methods MUST be overridden in a subclass: + +\begin{methoddesc}{_write}{data} +Buffer the string \var{data} for transmission to the client. It's okay +if this method actually transmits the data; \class{BaseHandler} +just separates write and flush operations for greater efficiency +when the underlying system actually has such a distinction. +\end{methoddesc} + +\begin{methoddesc}{_flush}{} +Force buffered data to be transmitted to the client. It's okay if this +method is a no-op (i.e., if \method{_write()} actually sends the data). +\end{methoddesc} + +\begin{methoddesc}{get_stdin}{} +Return an input stream object suitable for use as the \code{wsgi.input} +of the request currently being processed. +\end{methoddesc} + +\begin{methoddesc}{get_stderr}{} +Return an output stream object suitable for use as the +\code{wsgi.errors} of the request currently being processed. +\end{methoddesc} + +\begin{methoddesc}{add_cgi_vars}{} +Insert CGI variables for the current request into the \member{environ} +attribute. +\end{methoddesc} + +Here are some other methods and attributes you may wish to override. +This list is only a summary, however, and does not include every method +that can be overridden. You should consult the docstrings and source +code for additional information before attempting to create a customized +\class{BaseHandler} subclass. + + + + + + + + + + + + + + + + +Attributes and methods for customizing the WSGI environment: + +\begin{memberdesc}{wsgi_multithread} +The value to be used for the \code{wsgi.multithread} environment +variable. It defaults to true in \class{BaseHandler}, but may have +a different default (or be set by the constructor) in the other +subclasses. +\end{memberdesc} + +\begin{memberdesc}{wsgi_multiprocess} +The value to be used for the \code{wsgi.multiprocess} environment +variable. It defaults to true in \class{BaseHandler}, but may have +a different default (or be set by the constructor) in the other +subclasses. +\end{memberdesc} + +\begin{memberdesc}{wsgi_run_once} +The value to be used for the \code{wsgi.run_once} environment +variable. It defaults to false in \class{BaseHandler}, but +\class{CGIHandler} sets it to true by default. +\end{memberdesc} + +\begin{memberdesc}{os_environ} +The default environment variables to be included in every request's +WSGI environment. By default, this is a copy of \code{os.environ} at +the time that \module{wsgiref.handlers} was imported, but subclasses can +either create their own at the class or instance level. Note that the +dictionary should be considered read-only, since the default value is +shared between multiple classes and instances. +\end{memberdesc} + +\begin{memberdesc}{server_software} +If the \member{origin_server} attribute is set, this attribute's value +is used to set the default \code{SERVER_SOFTWARE} WSGI environment +variable, and also to set a default \code{Server:} header in HTTP +responses. It is ignored for handlers (such as \class{BaseCGIHandler} +and \class{CGIHandler}) that are not HTTP origin servers. +\end{memberdesc} + + + +\begin{methoddesc}{get_scheme}{} +Return the URL scheme being used for the current request. The default +implementation uses the \function{guess_scheme()} function from +\module{wsgiref.util} to guess whether the scheme should be ``http'' or +``https'', based on the current request's \member{environ} variables. +\end{methoddesc} + +\begin{methoddesc}{setup_environ}{} +Set the \member{environ} attribute to a fully-populated WSGI +environment. The default implementation uses all of the above methods +and attributes, plus the \method{get_stdin()}, \method{get_stderr()}, +and \method{add_cgi_vars()} methods and the \member{wsgi_file_wrapper} +attribute. It also inserts a \code{SERVER_SOFTWARE} key if not present, +as long as the \member{origin_server} attribute is a true value and the +\member{server_software} attribute is set. +\end{methoddesc} + + + + + + + + + + + + + + + + + + + + + + + + + +Methods and attributes for customizing exception handling: + +\begin{methoddesc}{log_exception}{exc_info} +Log the \var{exc_info} tuple in the server log. \var{exc_info} is a +\code{(\var{type}, \var{value}, \var{traceback})} tuple. The default +implementation simply writes the traceback to the request's +\code{wsgi.errors} stream and flushes it. Subclasses can override this +method to change the format or retarget the output, mail the traceback +to an administrator, or whatever other action may be deemed suitable. +\end{methoddesc} + +\begin{memberdesc}{traceback_limit} +The maximum number of frames to include in tracebacks output by the +default \method{log_exception()} method. If \code{None}, all frames +are included. +\end{memberdesc} + +\begin{methoddesc}{error_output}{environ, start_response} +This method is a WSGI application to generate an error page for the +user. It is only invoked if an error occurs before headers are sent +to the client. + +This method can access the current error information using +\code{sys.exc_info()}, and should pass that information to +\var{start_response} when calling it (as described in the ``Error +Handling'' section of \pep{333}). + +The default implementation just uses the \member{error_status}, +\member{error_headers}, and \member{error_body} attributes to generate +an output page. Subclasses can override this to produce more dynamic +error output. + +Note, however, that it's not recommended from a security perspective to +spit out diagnostics to any old user; ideally, you should have to do +something special to enable diagnostic output, which is why the default +implementation doesn't include any. +\end{methoddesc} + + + + +\begin{memberdesc}{error_status} +The HTTP status used for error responses. This should be a status +string as defined in \pep{333}; it defaults to a 500 code and message. +\end{memberdesc} + +\begin{memberdesc}{error_headers} +The HTTP headers used for error responses. This should be a list of +WSGI response headers (\code{(\var{name}, \var{value})} tuples), as +described in \pep{333}. The default list just sets the content type +to \code{text/plain}. +\end{memberdesc} + +\begin{memberdesc}{error_body} +The error response body. This should be an HTTP response body string. +It defaults to the plain text, ``A server error occurred. Please +contact the administrator.'' +\end{memberdesc} + + + + + + + + + + + + + + + + + + + + + + + + +Methods and attributes for \pep{333}'s ``Optional Platform-Specific File +Handling'' feature: + +\begin{memberdesc}{wsgi_file_wrapper} +A \code{wsgi.file_wrapper} factory, or \code{None}. The default value +of this attribute is the \class{FileWrapper} class from +\module{wsgiref.util}. +\end{memberdesc} + +\begin{methoddesc}{sendfile}{} +Override to implement platform-specific file transmission. This method +is called only if the application's return value is an instance of +the class specified by the \member{wsgi_file_wrapper} attribute. It +should return a true value if it was able to successfully transmit the +file, so that the default transmission code will not be executed. +The default implementation of this method just returns a false value. +\end{methoddesc} + + +Miscellaneous methods and attributes: + +\begin{memberdesc}{origin_server} +This attribute should be set to a true value if the handler's +\method{_write()} and \method{_flush()} are being used to communicate +directly to the client, rather than via a CGI-like gateway protocol that +wants the HTTP status in a special \code{Status:} header. + +This attribute's default value is true in \class{BaseHandler}, but +false in \class{BaseCGIHandler} and \class{CGIHandler}. +\end{memberdesc} + +\begin{memberdesc}{http_version} +If \member{origin_server} is true, this string attribute is used to +set the HTTP version of the response set to the client. It defaults to +\code{"1.0"}. +\end{memberdesc} + + + + + +\end{classdesc} + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Doc/lib/libzipfile.tex b/Doc/lib/libzipfile.tex index 4e06ef6..3d81e50 100644 --- a/Doc/lib/libzipfile.tex +++ b/Doc/lib/libzipfile.tex @@ -13,11 +13,12 @@ The ZIP file format is a common archive and compression standard. This module provides tools to create, read, write, append, and list a ZIP file. Any advanced use of this module will require an understanding of the format, as defined in -\citetitle[http://www.pkware.com/appnote.html]{PKZIP Application -Note}. +\citetitle[http://www.pkware.com/business_and_developers/developer/appnote/] +{PKZIP Application Note}. This module does not currently handle ZIP files which have appended -comments, or multi-disk ZIP files. +comments, or multi-disk ZIP files. It can handle ZIP files that use the +ZIP64 extensions (that is ZIP files that are more than 4 GByte in size). The available attributes of this module are: @@ -25,6 +26,11 @@ The available attributes of this module are: The error raised for bad ZIP files. \end{excdesc} +\begin{excdesc}{LargeZipFile} + The error raised when a ZIP file would require ZIP64 functionality but that + has not been enabled. +\end{excdesc} + \begin{classdesc*}{ZipFile} The class for reading and writing ZIP files. See ``\citetitle{ZipFile Objects}'' (section \ref{zipfile-objects}) for @@ -65,9 +71,9 @@ The available attributes of this module are: \begin{seealso} - \seetitle[http://www.pkware.com/appnote.html]{PKZIP Application - Note}{Documentation on the ZIP file format by Phil - Katz, the creator of the format and algorithms used.} + \seetitle[http://www.pkware.com/business_and_developers/developer/appnote/] + {PKZIP Application Note}{Documentation on the ZIP file format by + Phil Katz, the creator of the format and algorithms used.} \seetitle[http://www.info-zip.org/pub/infozip/]{Info-ZIP Home Page}{ Information about the Info-ZIP project's ZIP archive @@ -77,7 +83,7 @@ The available attributes of this module are: \subsection{ZipFile Objects \label{zipfile-objects}} -\begin{classdesc}{ZipFile}{file\optional{, mode\optional{, compression}}} +\begin{classdesc}{ZipFile}{file\optional{, mode\optional{, compression\optional{, allowZip64}}}} Open a ZIP file, where \var{file} can be either a path to a file (a string) or a file-like object. The \var{mode} parameter should be \code{'r'} to read an existing file, \code{'w'} to @@ -100,6 +106,12 @@ cat myzip.zip >> python.exe is specified but the \refmodule{zlib} module is not available, \exception{RuntimeError} is also raised. The default is \constant{ZIP_STORED}. + If \var{allowZip64} is \code{True} zipfile will create ZIP files that use + the ZIP64 extensions when the zipfile is larger than 2 GB. If it is + false (the default) \module{zipfile} will raise an exception when the + ZIP file would require ZIP64 extensions. ZIP64 extensions are disabled by + default because the default \program{zip} and \program{unzip} commands on + \UNIX{} (the InfoZIP utilities) don't support these extensions. \end{classdesc} \begin{methoddesc}{close}{} @@ -132,8 +144,8 @@ cat myzip.zip >> python.exe \end{methoddesc} \begin{methoddesc}{testzip}{} - Read all the files in the archive and check their CRC's. Return the - name of the first bad file, or else return \code{None}. + Read all the files in the archive and check their CRC's and file + headers. Return the name of the first bad file, or else return \code{None}. \end{methoddesc} \begin{methoddesc}{write}{filename\optional{, arcname\optional{, @@ -243,9 +255,9 @@ Instances have the following attributes: \begin{memberdesc}[ZipInfo]{extra} Expansion field data. The - \citetitle[http://www.pkware.com/appnote.html]{PKZIP Application - Note} contains some comments on the internal structure of the data - contained in this string. + \citetitle[http://www.pkware.com/business_and_developers/developer/appnote/] + {PKZIP Application Note} contains some comments on the internal + structure of the data contained in this string. \end{memberdesc} \begin{memberdesc}[ZipInfo]{create_system} @@ -284,10 +296,6 @@ Instances have the following attributes: Byte offset to the file header. \end{memberdesc} -\begin{memberdesc}[ZipInfo]{file_offset} - Byte offset to the start of the file data. -\end{memberdesc} - \begin{memberdesc}[ZipInfo]{CRC} CRC-32 of the uncompressed file. \end{memberdesc} diff --git a/Doc/lib/libzipimport.tex b/Doc/lib/libzipimport.tex index 770ea21..098e788 100644 --- a/Doc/lib/libzipimport.tex +++ b/Doc/lib/libzipimport.tex @@ -50,9 +50,9 @@ The available attributes of this module are: \begin{seealso} - \seetitle[http://www.pkware.com/appnote.html]{PKZIP Application - Note}{Documentation on the ZIP file format by Phil - Katz, the creator of the format and algorithms used.} + \seetitle[http://www.pkware.com/business_and_developers/developer/appnote/] + {PKZIP Application Note}{Documentation on the ZIP file format by + Phil Katz, the creator of the format and algorithms used.} \seepep{0273}{Import Modules from Zip Archives}{Written by James C. Ahlstrom, who also provided an implementation. Python 2.3 diff --git a/Doc/lib/sqlite3/complete_statement.py b/Doc/lib/sqlite3/complete_statement.py index 89fc250..22525e3 100644 --- a/Doc/lib/sqlite3/complete_statement.py +++ b/Doc/lib/sqlite3/complete_statement.py @@ -24,7 +24,7 @@ while True: if buffer.lstrip().upper().startswith("SELECT"): print cur.fetchall() except sqlite3.Error, e: - print "An error occured:", e.args[0] + print "An error occurred:", e.args[0] buffer = "" con.close() diff --git a/Doc/lib/tkinter.tex b/Doc/lib/tkinter.tex index 405f041..db52cbd 100644 --- a/Doc/lib/tkinter.tex +++ b/Doc/lib/tkinter.tex @@ -18,10 +18,9 @@ implement the Tk widgets as Python classes. In addition, the internal module \module{\_tkinter} provides a threadsafe mechanism which allows Python and Tcl to interact. -Tk is not the only GUI for Python, but is however the most commonly -used one; see section~\ref{other-gui-modules}, ``Other User Interface -Modules and Packages,'' for more information on other GUI toolkits for -Python. +Tk is not the only GUI for Python; see +section~\ref{other-gui-packages}, ``Other User Interface Modules and +Packages,'' for more information on other GUI toolkits for Python. % Other sections I have in mind are % Tkinter internals @@ -103,14 +102,14 @@ of an application. Each instance has its own associated Tcl interpreter. \end{classdesc} \begin{funcdesc}{Tcl}{screenName=None, baseName=None, className='Tk', useTk=0} -The \function{Tcl} function is a factory function which creates an object -much like that created by the \class{Tk} class, except that it does not -initialize the Tk subsystem. This is most often useful when driving the Tcl -interpreter in an environment where one doesn't want to create extraneous -toplevel windows, or where one cannot (i.e. Unix/Linux systems without an X -server). An object created by the \function{Tcl} object can have a Toplevel -window created (and the Tk subsystem initialized) by calling its -\method{loadtk} method. +The \function{Tcl} function is a factory function which creates an +object much like that created by the \class{Tk} class, except that it +does not initialize the Tk subsystem. This is most often useful when +driving the Tcl interpreter in an environment where one doesn't want +to create extraneous toplevel windows, or where one cannot (such as +\UNIX/Linux systems without an X server). An object created by the +\function{Tcl} object can have a Toplevel window created (and the Tk +subsystem initialized) by calling its \method{loadtk} method. \versionadded{2.4} \end{funcdesc} @@ -316,10 +315,10 @@ is called \code{.} (period) and children are delimited by more periods. For example, \code{.myApp.controlPanel.okButton} might be the name of a widget. -\item[\var{options} ] +\item[\var{options}] configure the widget's appearance and in some cases, its behavior. The options come in the form of a list of flags and values. -Flags are proceeded by a `-', like unix shell command flags, and +Flags are proceeded by a `-', like \UNIX{} shell command flags, and values are put in quotes if they are more than one word. \end{description} @@ -1806,24 +1805,29 @@ directly on Python data structures, without having to transfer data through the Tk/Tcl layer.} \end{seealso*} - -Tk is not the only GUI for Python, but is however the -most commonly used one. +Other GUI packages are also available for Python: \begin{seealso*} -\seetitle[http://www.wxwindows.org]{wxWindows}{ -is a GUI toolkit that combines the most attractive attributes of Qt, -Tk, Motif, and GTK+ in one powerful and efficient package. It is -implemented in \Cpp. wxWindows supports two flavors of \UNIX{} -implementation: GTK+ and Motif, and under Windows, it has a standard -Microsoft Foundation Classes (MFC) appearance, because it uses Win32 -widgets. There is a Python class wrapper, independent of Tkinter. - -wxWindows is much richer in widgets than \refmodule{Tkinter}, with its -help system, sophisticated HTML and image viewers, and other -specialized widgets, extensive documentation, and printing capabilities. +\seetitle[http://www.wxpython.org]{wxPython}{ +wxPython is a cross-platform GUI toolkit for Python that is built +around the popular \ulink{wxWidgets}{http://www.wxwidgets.org/} \Cpp{} +toolkit.  It provides a native look and feel for applications on +Windows, Mac OS X, and \UNIX{} systems by using each platform's native +widgets where ever possible, (GTK+ on \UNIX-like systems).  In +addition to an extensive set of widgets, wxPython provides classes for +online documentation and context sensitive help, printing, HTML +viewing, low-level device context drawing, drag and drop, system +clipboard access, an XML-based resource format and more, including an +ever growing library of user-contributed modules.  Both the wxWidgets +and wxPython projects are under active development and continuous +improvement, and have active and helpful user and developer +communities. +} +\seetitle[http://www.amazon.com/exec/obidos/ASIN/1932394621] +{wxPython in Action}{ +The wxPython book, by Noel Rappin and Robin Dunn. } -\seetitle[]{PyQt}{ +\seetitle{PyQt}{ PyQt is a \program{sip}-wrapped binding to the Qt toolkit. Qt is an extensive \Cpp{} GUI toolkit that is available for \UNIX, Windows and Mac OS X. \program{sip} is a tool for generating bindings for \Cpp{} diff --git a/Doc/mac/libmacfs.tex b/Doc/mac/libmacfs.tex index 944ea1b..12a7cc3 100644 --- a/Doc/mac/libmacfs.tex +++ b/Doc/mac/libmacfs.tex @@ -22,10 +22,10 @@ Whenever a function or method expects a \var{file} argument, this argument can be one of three things:\ (1) a full or partial Macintosh pathname, (2) an \class{FSSpec} object or (3) a 3-tuple \code{(\var{wdRefNum}, \var{parID}, \var{name})} as described in -\citetitle{Inside Macintosh:\ Files}. An \class{FSSpec} can point to +\citetitle{Inside Macintosh:\ Files}. An \class{FSSpec} can point to a non-existing file, as long as the folder containing the file exists. -Under MacPython the same is true for a pathname, but not under unix-Pyton -because of the way pathnames and FSRefs works. See Apple's documentation +Under MacPython the same is true for a pathname, but not under \UNIX-Python +because of the way pathnames and FSRefs works. See Apple's documentation for details. A description of aliases and the diff --git a/Doc/mac/libmacos.tex b/Doc/mac/libmacos.tex index b22b39c..e50b99b 100644 --- a/Doc/mac/libmacos.tex +++ b/Doc/mac/libmacos.tex @@ -25,7 +25,7 @@ The way the interpreter has been linked. As extension modules may be incompatible between linking models, packages could use this information to give more decent error messages. The value is one of \code{'static'} for a statically linked Python, \code{'framework'} for Python in a Mac OS X framework, -\code{'shared'} for Python in a standard unix shared library. +\code{'shared'} for Python in a standard \UNIX{} shared library. Older Pythons could also have the value \code{'cfm'} for Mac OS 9-compatible Python. \end{datadesc} diff --git a/Doc/mac/using.tex b/Doc/mac/using.tex index bfa478e..b21a98e 100644 --- a/Doc/mac/using.tex +++ b/Doc/mac/using.tex @@ -6,7 +6,7 @@ Python on any other \UNIX platform, but there are a number of additional features such as the IDE and the Package Manager that are worth pointing out. Python on Mac OS 9 or earlier can be quite different from Python on -Unix or Windows, but is beyond the scope of this manual, as that platform +\UNIX{} or Windows, but is beyond the scope of this manual, as that platform is no longer supported, starting with Python 2.4. See \url{http://www.cwi.nl/\textasciitilde jack/macpython} for installers for the latest 2.3 release for Mac OS 9 and related documentation. diff --git a/Doc/ref/ref2.tex b/Doc/ref/ref2.tex index 2ed8a5d..bad4609 100644 --- a/Doc/ref/ref2.tex +++ b/Doc/ref/ref2.tex @@ -56,7 +56,7 @@ by following the explicit or implicit \emph{line joining} rules. A physical line is a sequence of characters terminated by an end-of-line sequence. In source files, any of the standard platform line -termination sequences can be used - the \UNIX form using \ASCII{} LF +termination sequences can be used - the \UNIX{} form using \ASCII{} LF (linefeed), the Windows form using the \ASCII{} sequence CR LF (return followed by linefeed), or the Macintosh form using the \ASCII{} CR (return) character. All of these forms can be used equally, regardless diff --git a/Doc/ref/ref3.tex b/Doc/ref/ref3.tex index d0c8ccf..15fc188 100644 --- a/Doc/ref/ref3.tex +++ b/Doc/ref/ref3.tex @@ -1307,6 +1307,11 @@ defines mutable objects and implements a \method{__cmp__()} or since the dictionary implementation requires that a key's hash value is immutable (if the object's hash value changes, it will be in the wrong hash bucket). + +\versionchanged[\method{__hash__()} may now also return a long +integer object; the 32-bit integer is then derived from the hash +of that object]{2.5} + \withsubitem{(object method)}{\ttindex{__cmp__()}} \end{methoddesc} @@ -1886,6 +1891,9 @@ method should be the equivalent to using \method{__floordiv__()} and \method{__pow__()} should be defined to accept an optional third argument if the ternary version of the built-in \function{pow()}\bifuncindex{pow} function is to be supported. + +If one of those methods does not support the operation with the +supplied arguments, it should return \code{NotImplemented}. \end{methoddesc} \begin{methoddesc}[numeric object]{__div__}{self, other} @@ -1918,13 +1926,28 @@ called to implement the binary arithmetic operations (\code{+}, \function{pow()}\bifuncindex{pow}, \code{**}, \code{<<}, \code{>>}, \code{\&}, \code{\^}, \code{|}) with reflected (swapped) operands. These functions are only called if the left -operand does not support the corresponding operation. For instance, -to evaluate the expression \var{x}\code{-}\var{y}, where \var{y} is an -instance of a class that has an \method{__rsub__()} method, -\code{\var{y}.__rsub__(\var{x})} is called. Note that ternary +operand does not support the corresponding operation and the +operands are of different types.\footnote{ + For operands of the same type, it is assumed that if the + non-reflected method (such as \method{__add__()}) fails the + operation is not supported, which is why the reflected method + is not called.} +For instance, to evaluate the expression \var{x}\code{-}\var{y}, +where \var{y} is an instance of a class that has an +\method{__rsub__()} method, \code{\var{y}.__rsub__(\var{x})} +is called if \code{\var{x}.__sub__(\var{y})} returns +\var{NotImplemented}. + +Note that ternary \function{pow()}\bifuncindex{pow} will not try calling \method{__rpow__()} (the coercion rules would become too complicated). + +\note{If the right operand's type is a subclass of the left operand's + type and that subclass provides the reflected method for the + operation, this method will be called before the left operand's + non-reflected method. This behavior allows subclasses to + override their ancestors' operations.} \end{methoddesc} \begin{methoddesc}[numeric object]{__iadd__}{self, other} diff --git a/Doc/ref/ref4.tex b/Doc/ref/ref4.tex index dcdc823..12a2b92 100644 --- a/Doc/ref/ref4.tex +++ b/Doc/ref/ref4.tex @@ -97,10 +97,20 @@ searched. The global statement must precede all uses of the name. The built-in namespace associated with the execution of a code block is actually found by looking up the name \code{__builtins__} in its global namespace; this should be a dictionary or a module (in the -latter case the module's dictionary is used). Normally, the -\code{__builtins__} namespace is the dictionary of the built-in module -\module{__builtin__} (note: no `s'). If it isn't, restricted -execution\indexii{restricted}{execution} mode is in effect. +latter case the module's dictionary is used). By default, when in the +\module{__main__} module, \code{__builtins__} is the built-in module +\module{__builtin__} (note: no `s'); when in any other module, +\code{__builtins__} is an alias for the dictionary of the +\module{__builtin__} module itself. \code{__builtins__} can be set +to a user-created dictionary to create a weak form of restricted +execution\indexii{restricted}{execution}. + +\begin{notice} + Users should not touch \code{__builtins__}; it is strictly an + implementation detail. Users wanting to override values in the + built-in namespace should \keyword{import} the \module{__builtin__} + (no `s') module and modify its attributes appropriately. +\end{notice} The namespace for a module is automatically created the first time a module is imported. The main module for a script is always called diff --git a/Doc/ref/ref5.tex b/Doc/ref/ref5.tex index 89f9977..909e5bb 100644 --- a/Doc/ref/ref5.tex +++ b/Doc/ref/ref5.tex @@ -907,7 +907,10 @@ The operators \code{<}, \code{>}, \code{==}, \code{>=}, \code{<=}, and the values of two objects. The objects need not have the same type. If both are numbers, they are converted to a common type. Otherwise, objects of different types \emph{always} compare unequal, and are -ordered consistently but arbitrarily. +ordered consistently but arbitrarily. You can control comparison +behavior of objects of non-builtin types by defining a \code{__cmp__} +method or rich comparison methods like \code{__gt__}, described in +section~\ref{specialnames}. (This unusual definition of comparison was used to simplify the definition of operations like sorting and the \keyword{in} and @@ -952,7 +955,8 @@ otherwise defined.\footnote{Earlier versions of Python used a dictionary for emptiness by comparing it to \code{\{\}}.} \item -Most other types compare unequal unless they are the same object; +Most other objects of builtin types compare unequal unless they are +the same object; the choice whether one object is considered smaller or larger than another one is made arbitrarily but consistently within one execution of a program. diff --git a/Doc/ref/ref8.tex b/Doc/ref/ref8.tex index 801ab58..45be71d 100644 --- a/Doc/ref/ref8.tex +++ b/Doc/ref/ref8.tex @@ -34,7 +34,7 @@ in the namespace of \module{__main__}. \index{interactive mode} \refbimodindex{__main__} -Under {\UNIX}, a complete program can be passed to the interpreter in +Under \UNIX, a complete program can be passed to the interpreter in three forms: with the \programopt{-c} \var{string} command line option, as a file passed as the first command line argument, or as standard input. If the file or standard input is a tty device, the interpreter enters diff --git a/Doc/tut/tut.tex b/Doc/tut/tut.tex index 4e0a26b..1b08a8e 100644 --- a/Doc/tut/tut.tex +++ b/Doc/tut/tut.tex @@ -2931,14 +2931,13 @@ submodules with the same name from different packages. The submodules often need to refer to each other. For example, the \module{surround} module might use the \module{echo} module. In fact, -such references -are so common that the \keyword{import} statement first looks in the -containing package before looking in the standard module search path. -Thus, the surround module can simply use \code{import echo} or -\code{from echo import echofilter}. If the imported module is not -found in the current package (the package of which the current module -is a submodule), the \keyword{import} statement looks for a top-level -module with the given name. +such references are so common that the \keyword{import} statement +first looks in the containing package before looking in the standard +module search path. Thus, the \module{surround} module can simply use +\code{import echo} or \code{from echo import echofilter}. If the +imported module is not found in the current package (the package of +which the current module is a submodule), the \keyword{import} +statement looks for a top-level module with the given name. When packages are structured into subpackages (as with the \module{Sound} package in the example), there's no shortcut to refer @@ -2948,6 +2947,24 @@ must be used. For example, if the module in the \module{Sound.Effects} package, it can use \code{from Sound.Effects import echo}. +Starting with Python 2.5, in addition to the implicit relative imports +described above, you can write explicit relative imports with the +\code{from module import name} form of import statement. These explicit +relative imports use leading dots to indicate the current and parent +packages involved in the relative import. From the \module{surround} +module for example, you might use: + +\begin{verbatim} +from . import echo +from .. import Formats +from ..Filters import equalizer +\end{verbatim} + +Note that both explicit and implicit relative imports are based on the +name of the current module. Since the name of the main module is always +\code{"__main__"}, modules intended for use as the main module of a +Python application should always use absolute imports. + \subsection{Packages in Multiple Directories} Packages support one more special attribute, \member{__path__}. This diff --git a/Doc/whatsnew/whatsnew20.tex b/Doc/whatsnew/whatsnew20.tex index 56d15b8..360d7dc 100644 --- a/Doc/whatsnew/whatsnew20.tex +++ b/Doc/whatsnew/whatsnew20.tex @@ -216,7 +216,7 @@ A new module, \module{unicodedata}, provides an interface to Unicode character properties. For example, \code{unicodedata.category(u'A')} returns the 2-character string 'Lu', the 'L' denoting it's a letter, and 'u' meaning that it's uppercase. -\code{u.bidirectional(u'\e x0660')} returns 'AN', meaning that U+0660 is +\code{unicodedata.bidirectional(u'\e u0660')} returns 'AN', meaning that U+0660 is an Arabic number. The \module{codecs} module contains functions to look up existing encodings @@ -571,7 +571,7 @@ def f(*args, **kw): The \keyword{print} statement can now have its output directed to a file-like object by following the \keyword{print} with -\verb|>> file|, similar to the redirection operator in Unix shells. +\verb|>> file|, similar to the redirection operator in \UNIX{} shells. Previously you'd either have to use the \method{write()} method of the file-like object, which lacks the convenience and simplicity of \keyword{print}, or you could assign a new value to @@ -777,7 +777,7 @@ fact will break in 2.0. Some work has been done to make integers and long integers a bit more interchangeable. In 1.5.2, large-file support was added for Solaris, -to allow reading files larger than 2Gb; this made the \method{tell()} +to allow reading files larger than 2~GiB; this made the \method{tell()} method of file objects return a long integer instead of a regular integer. Some code would subtract two file offsets and attempt to use the result to multiply a sequence or slice a string, but this raised a @@ -894,7 +894,7 @@ to be added, and a third argument for the value to be assigned to the name. This third argument is, respectively, a Python object, a C long, or a C string. -A wrapper API was added for Unix-style signal handlers. +A wrapper API was added for \UNIX-style signal handlers. \function{PyOS_getsig()} gets a signal handler and \function{PyOS_setsig()} will set a new handler. @@ -905,7 +905,7 @@ Before Python 2.0, installing modules was a tedious affair -- there was no way to figure out automatically where Python is installed, or what compiler options to use for extension modules. Software authors had to go through an arduous ritual of editing Makefiles and -configuration files, which only really work on Unix and leave Windows +configuration files, which only really work on \UNIX{} and leave Windows and MacOS unsupported. Python users faced wildly differing installation instructions which varied between different extension packages, which made administering a Python installation something of @@ -1222,7 +1222,7 @@ device on Linux, a twin to the existing \module{sunaudiodev} module. (Contributed by Peter Bosch, with fixes by Jeremy Hylton.) \item{\module{mmap}:} An interface to memory-mapped files on both -Windows and Unix. A file's contents can be mapped directly into +Windows and \UNIX. A file's contents can be mapped directly into memory, at which point it behaves like a mutable string, so its contents can be read and modified. They can even be passed to functions that expect ordinary strings, such as the \module{re} @@ -1262,7 +1262,7 @@ distribution, and enhanced to support Unicode. \item{\module{zipfile}:} A module for reading and writing ZIP-format archives. These are archives produced by \program{PKZIP} on -DOS/Windows or \program{zip} on Unix, not to be confused with +DOS/Windows or \program{zip} on \UNIX, not to be confused with \program{gzip}-format files (which are supported by the \module{gzip} module) (Contributed by James C. Ahlstrom.) diff --git a/Doc/whatsnew/whatsnew21.tex b/Doc/whatsnew/whatsnew21.tex index f3d0245..67cbbe4 100644 --- a/Doc/whatsnew/whatsnew21.tex +++ b/Doc/whatsnew/whatsnew21.tex @@ -325,7 +325,7 @@ Rossum.} When compiling Python, the user had to go in and edit the \file{Modules/Setup} file in order to enable various additional modules; the default set is relatively small and limited to modules -that compile on most Unix platforms. This means that on Unix +that compile on most \UNIX{} platforms. This means that on \Unix{} platforms with many more features, most notably Linux, Python installations often don't contain all useful modules they could. @@ -661,7 +661,7 @@ PyUnit. \item The \module{difflib} module contains a class, \class{SequenceMatcher}, which compares two sequences and computes the changes required to transform one sequence into the other. For -example, this module can be used to write a tool similar to the Unix +example, this module can be used to write a tool similar to the \UNIX{} \program{diff} program, and in fact the sample program \file{Tools/scripts/ndiff.py} demonstrates how to write such a script. diff --git a/Doc/whatsnew/whatsnew23.tex b/Doc/whatsnew/whatsnew23.tex index a122083..72fd306 100644 --- a/Doc/whatsnew/whatsnew23.tex +++ b/Doc/whatsnew/whatsnew23.tex @@ -1479,7 +1479,7 @@ now return enhanced tuples: ('amk', 500) \end{verbatim} -\item The \module{gzip} module can now handle files exceeding 2~Gb. +\item The \module{gzip} module can now handle files exceeding 2~GiB. \item The new \module{heapq} module contains an implementation of a heap queue algorithm. A heap is an array-like data structure that @@ -1979,7 +1979,7 @@ documentation}{../lib/module-datetime.html}. The \module{getopt} module provides simple parsing of command-line arguments. The new \module{optparse} module (originally named Optik) -provides more elaborate command-line parsing that follows the Unix +provides more elaborate command-line parsing that follows the \UNIX{} conventions, automatically creates the output for \longprogramopt{help}, and can perform different actions for different options. diff --git a/Doc/whatsnew/whatsnew24.tex b/Doc/whatsnew/whatsnew24.tex index 51baece..096b1ec 100644 --- a/Doc/whatsnew/whatsnew24.tex +++ b/Doc/whatsnew/whatsnew24.tex @@ -162,7 +162,7 @@ for link in links: Generator expressions always have to be written inside parentheses, as in the above example. The parentheses signalling a function call also -count, so if you want to create a iterator that will be immediately +count, so if you want to create an iterator that will be immediately passed to a function you could write: \begin{verbatim} diff --git a/Doc/whatsnew/whatsnew25.tex b/Doc/whatsnew/whatsnew25.tex index 4015d98..dcb6ab1 100644 --- a/Doc/whatsnew/whatsnew25.tex +++ b/Doc/whatsnew/whatsnew25.tex @@ -3,10 +3,9 @@ % $Id$ % Fix XXX comments -% Count up the patches and bugs \title{What's New in Python 2.5} -\release{0.2} +\release{0.9} \author{A.M. Kuchling} \authoraddress{\email{amk@amk.ca}} @@ -14,31 +13,57 @@ \maketitle \tableofcontents -This article explains the new features in Python 2.5. No release date -for Python 2.5 has been set; it will probably be released in the -autumn of 2006. \pep{356} describes the planned release schedule. - -Comments, suggestions, and error reports are welcome; please e-mail them -to the author or open a bug in the Python bug tracker. - -% XXX Compare with previous release in 2 - 3 sentences here. - -This article doesn't attempt to provide a complete specification of -the new features, but instead provides a convenient overview. For -full details, you should refer to the documentation for Python 2.5. +This article explains the new features in Python 2.5. The final +release of Python 2.5 is scheduled for August 2006; +\pep{356} describes the planned release schedule. + +The changes in Python 2.5 are an interesting mix of language and +library improvements. The library enhancements will be more important +to Python's user community, I think, because several widely-useful +packages were added. New modules include ElementTree for XML +processing (section~\ref{module-etree}), the SQLite database module +(section~\ref{module-sqlite}), and the \module{ctypes} module for +calling C functions (section~\ref{module-ctypes}). + +The language changes are of middling significance. Some pleasant new +features were added, but most of them aren't features that you'll use +every day. Conditional expressions were finally added to the language +using a novel syntax; see section~\ref{pep-308}. The new +'\keyword{with}' statement will make writing cleanup code easier +(section~\ref{pep-343}). Values can now be passed into generators +(section~\ref{pep-342}). Imports are now visible as either absolute +or relative (section~\ref{pep-328}). Some corner cases of exception +handling are handled better (section~\ref{pep-341}). All these +improvements are worthwhile, but they're improvements to one specific +language feature or another; none of them are broad modifications to +Python's semantics. + +As well as the language and library additions, other improvements and +bugfixes were made throughout the source tree. A search through the +SVN change logs finds there were 334 patches applied and 443 bugs +fixed between Python 2.4 and 2.5. (Both figures are likely to be +underestimates.) + +This article doesn't try to be a complete specification of the new +features; instead changes are briefly introduced using helpful +examples. For full details, you should always refer to the +documentation for Python 2.5. % XXX add hyperlink when the documentation becomes available online. If you want to understand the complete implementation and design rationale, refer to the PEP for a particular new feature. +Comments, suggestions, and error reports for this document are +welcome; please e-mail them to the author or open a bug in the Python +bug tracker. %====================================================================== \section{PEP 308: Conditional Expressions\label{pep-308}} For a long time, people have been requesting a way to write -conditional expressions, expressions that return value A or value B -depending on whether a Boolean value is true or false. A conditional -expression lets you write a single assignment statement that has the -same effect as the following: +conditional expressions, which are expressions that return value A or +value B depending on whether a Boolean value is true or false. A +conditional expression lets you write a single assignment statement +that has the same effect as the following: \begin{verbatim} if condition: @@ -54,7 +79,7 @@ but there was no syntax that was preferred by a clear majority. Candidates included C's \code{cond ? true_v : false_v}, \code{if cond then true_v else false_v}, and 16 other variations. -GvR eventually chose a surprising syntax: +Guido van~Rossum eventually chose a surprising syntax: \begin{verbatim} x = true_value if condition else false_value @@ -126,19 +151,16 @@ Wouters.} \section{PEP 309: Partial Function Application\label{pep-309}} The \module{functools} module is intended to contain tools for -functional-style programming. Currently it only contains a -\class{partial()} function, but new functions will probably be added -in future versions of Python. +functional-style programming. -For programs written in a functional style, it can be useful to +One useful tool in this module is the \function{partial()} function. +For programs written in a functional style, you'll sometimes want to construct variants of existing functions that have some of the parameters filled in. Consider a Python function \code{f(a, b, c)}; you could create a new function \code{g(b, c)} that was equivalent to -\code{f(1, b, c)}. This is called ``partial function application'', -and is provided by the \class{partial} class in the new -\module{functools} module. +\code{f(1, b, c)}. This is called ``partial function application''. -The constructor for \class{partial} takes the arguments +\function{partial} takes the arguments \code{(\var{function}, \var{arg1}, \var{arg2}, ... \var{kwarg1}=\var{value1}, \var{kwarg2}=\var{value2})}. The resulting object is callable, so you can just call it to invoke \var{function} @@ -175,11 +197,40 @@ class Application: \end{verbatim} +Another function in the \module{functools} module is the +\function{update_wrapper(\var{wrapper}, \var{wrapped})} function that +helps you write well-behaved decorators. \function{update_wrapper()} +copies the name, module, and docstring attribute to a wrapper function +so that tracebacks inside the wrapped function are easier to +understand. For example, you might write: + +\begin{verbatim} +def my_decorator(f): + def wrapper(*args, **kwds): + print 'Calling decorated function' + return f(*args, **kwds) + functools.update_wrapper(wrapper, f) + return wrapper +\end{verbatim} + +\function{wraps()} is a decorator that can be used inside your own +decorators to copy the wrapped function's information. An alternate +version of the previous example would be: + +\begin{verbatim} +def my_decorator(f): + @functools.wraps(f) + def wrapper(*args, **kwds): + print 'Calling decorated function' + return f(*args, **kwds) + return wrapper +\end{verbatim} + \begin{seealso} \seepep{309}{Partial Function Application}{PEP proposed and written by -Peter Harris; implemented by Hye-Shik Chang, with adaptations by -Raymond Hettinger.} +Peter Harris; implemented by Hye-Shik Chang and Nick Coghlan, with +adaptations by Raymond Hettinger.} \end{seealso} @@ -361,7 +412,7 @@ specific exceptions. You couldn't combine both \keyword{except} blocks and a combined version was complicated and it wasn't clear what the semantics of the combined should be. -GvR spent some time working with Java, which does support the +Guido van~Rossum spent some time working with Java, which does support the equivalent of combining \keyword{except} blocks and a \keyword{finally} block, and this clarified what the statement should mean. In Python 2.5, you can now write: @@ -554,7 +605,11 @@ once the generator has been exhausted. \seepep{342}{Coroutines via Enhanced Generators}{PEP written by Guido van~Rossum and Phillip J. Eby; implemented by Phillip J. Eby. Includes examples of -some fancier uses of generators as coroutines.} +some fancier uses of generators as coroutines. + +Earlier versions of these features were proposed in +\pep{288} by Raymond Hettinger and \pep{325} by Samuele Pedroni. +} \seeurl{http://en.wikipedia.org/wiki/Coroutine}{The Wikipedia entry for coroutines.} @@ -771,7 +826,7 @@ The new \module{contextlib} module provides some functions and a decorator that are useful for writing objects for use with the '\keyword{with}' statement. -The decorator is called \function{contextfactory}, and lets you write +The decorator is called \function{contextmanager}, and lets you write a single generator function instead of defining a new class. The generator should yield exactly one value. The code up to the \keyword{yield} will be executed as the \method{__enter__()} method, and the value @@ -785,9 +840,9 @@ Our database example from the previous section could be written using this decorator as: \begin{verbatim} -from contextlib import contextfactory +from contextlib import contextmanager -@contextfactory +@contextmanager def db_transaction (connection): cursor = connection.cursor() try: @@ -933,7 +988,7 @@ space for a \ctype{PyObject} representing the item. 2147483647*4 is already more bytes than a 32-bit address space can contain. It's possible to address that much memory on a 64-bit platform, -however. The pointers for a list that size would only require 16GiB +however. The pointers for a list that size would only require 16~GiB of space, so it's not unreasonable that Python programmers might construct lists that large. Therefore, the Python interpreter had to be changed to use some type other than \ctype{int}, and this will be a @@ -1044,10 +1099,10 @@ print d[3], d[4] # Prints 0, 0 \item Both 8-bit and Unicode strings have new \method{partition(sep)} and \method{rpartition(sep)} methods that simplify a common use case. + The \method{find(S)} method is often used to get an index which is then used to slice the string and obtain the pieces that are before and after the separator. - \method{partition(sep)} condenses this pattern into a single method call that returns a 3-tuple containing the substring before the separator, the separator itself, and the @@ -1072,6 +1127,17 @@ Some examples: (Implemented by Fredrik Lundh following a suggestion by Raymond Hettinger.) +\item The \method{startswith()} and \method{endswith()} methods +of string types now accept tuples of strings to check for. + +\begin{verbatim} +def is_image_file (filename): + return filename.endswith(('.gif', '.jpg', '.tiff')) +\end{verbatim} + +(Implemented by Georg Brandl following a suggestion by Tom Lynn.) +% RFE #1491485 + \item The \function{min()} and \function{max()} built-in functions gained a \code{key} keyword parameter analogous to the \code{key} argument for \method{sort()}. This parameter supplies a function that @@ -1095,8 +1161,17 @@ print max(L) false values. \function{any()} returns \constant{True} if any value returned by the iterator is true; otherwise it will return \constant{False}. \function{all()} returns \constant{True} only if -all of the values returned by the iterator evaluate as being true. -(Suggested by GvR, and implemented by Raymond Hettinger.) +all of the values returned by the iterator evaluate as true. +(Suggested by Guido van~Rossum, and implemented by Raymond Hettinger.) + +\item The result of a class's \method{__hash__()} method can now +be either a long integer or a regular integer. If a long integer is +returned, the hash of that value is taken. In earlier versions the +hash value was required to be a regular integer, but in 2.5 the +\function{id()} built-in was changed to always return non-negative +numbers, and users often seem to use \code{id(self)} in +\method{__hash__()} methods (though this is discouraged). +% Bug #1536021 \item ASCII is now the default encoding for modules. It's now a syntax error if a module contains string literals with 8-bit @@ -1113,9 +1188,12 @@ a line like this near the top of the source file: to include an \file{__init__.py} module in a package directory. Debugging this mistake can be confusing, and usually requires running Python with the \programopt{-v} switch to log all the paths searched. -In Python 2.5, a new \exception{ImportWarning} warning is raised when +In Python 2.5, a new \exception{ImportWarning} warning is triggered when an import would have picked up a directory as a package but no -\file{__init__.py} was found. (Implemented by Thomas Wouters.) +\file{__init__.py} was found. This warning is silently ignored by default; +provide the \programopt{-Wd} option when running the Python executable +to display the warning message. +(Implemented by Thomas Wouters.) \item The list of base classes in a class definition can now be empty. As an example, this is now legal: @@ -1146,6 +1224,11 @@ produce string representations of themselves, but are also callable. Newbies who try \code{quit()} or \code{exit()} will now exit the interpreter as they expect. (Implemented by Georg Brandl.) +The Python executable now accepts the standard long options +\longprogramopt{help} and \longprogramopt{version}; on Windows, +it also accepts the \programopt{/?} option for displaying a help message. +(Implemented by Georg Brandl.) + %====================================================================== \subsection{Optimizations\label{opts}} @@ -1194,7 +1277,8 @@ Python's allocator functions instead of the system's \item The code generator's peephole optimizer now performs simple constant folding in expressions. If you write something like \code{a = 2+3}, the code generator will do the arithmetic and produce -code corresponding to \code{a = 5}. +code corresponding to \code{a = 5}. (Proposed and implemented +by Raymond Hettinger.) \item Function calls are now faster because code objects now keep the most recently finished frame (a ``zombie frame'') in an internal @@ -1288,10 +1372,13 @@ defaultdict(, {'c': ['cammin', 'che'], 'e': ['era'], 'r': ['ritrovai'], 'u': ['una'], 'v': ['vita', 'via']} \end{verbatim} -The \class{deque} double-ended queue type supplied by the +(Contributed by Guido van~Rossum.) + +\item The \class{deque} double-ended queue type supplied by the \module{collections} module now has a \method{remove(\var{value})} method that removes the first occurrence of \var{value} in the queue, raising \exception{ValueError} if the value isn't found. +(Contributed by Raymond Hettinger.) \item New module: The \module{contextlib} module contains helper functions for use with the new '\keyword{with}' statement. See @@ -1320,6 +1407,17 @@ currently-set limit. The \class{reader} class now has a \member{line_num} attribute that counts the number of physical lines read from the source; records can span multiple physical lines, so \member{line_num} is not the same as the number of records read. + +The CSV parser is now stricter about multi-line quoted +fields. Previously, if a line ended within a quoted field without a +terminating newline character, a newline would be inserted into the +returned field. This behavior caused problems when reading files that +contained carriage return characters within fields, so the code was +changed to return the field without inserting newlines. As a +consequence, if newlines embedded within fields are important, the +input should be split into lines in a manner that preserves the +newline characters. + (Contributed by Skip Montanaro and Andrew McNamara.) \item The \class{datetime} class in the \module{datetime} @@ -1335,11 +1433,27 @@ ts = datetime.strptime('10:13:15 2006-03-07', '%H:%M:%S %Y-%m-%d') \end{verbatim} +\item The \method{SequenceMatcher.get_matching_blocks()} method +in the \module{difflib} module now guarantees to return a minimal list +of blocks describing matching subsequences. Previously, the algorithm would +occasionally break a block of matching elements into two list entries. +(Enhancement by Tim Peters.) + \item The \module{doctest} module gained a \code{SKIP} option that keeps an example from being executed at all. This is intended for code snippets that are usage examples intended for the reader and aren't actually test cases. +An \var{encoding} parameter was added to the \function{testfile()} +function and the \class{DocFileSuite} class to specify the file's +encoding. This makes it easier to use non-ASCII characters in +tests contained within a docstring. (Contributed by Bjorn Tillenius.) +% Patch 1080727 + +\item The \module{email} package has been updated to version 4.0. +% XXX need to provide some more detail here +(Contributed by Barry Warsaw.) + \item The \module{fileinput} module was made more flexible. Unicode filenames are now supported, and a \var{mode} parameter that defaults to \code{"r"} was added to the @@ -1358,6 +1472,7 @@ collector; when these counts reach a specified threshold, a garbage collection sweep will be made. The existing \function{gc.collect()} function now takes an optional \var{generation} argument of 0, 1, or 2 to specify which generation to collect. +(Contributed by Barry Warsaw.) \item The \function{nsmallest()} and \function{nlargest()} functions in the \module{heapq} module @@ -1388,6 +1503,29 @@ itertools.islice(iterable, s.start, s.stop, s.step) (Contributed by Raymond Hettinger.) +\item The \function{format()} function in the \module{locale} module +has been modified and two new functions were added, +\function{format_string()} and \function{currency()}. + +The \function{format()} function's \var{val} parameter could +previously be a string as long as no more than one \%char specifier +appeared; now the parameter must be exactly one \%char specifier with +no surrounding text. An optional \var{monetary} parameter was also +added which, if \code{True}, will use the locale's rules for +formatting currency in placing a separator between groups of three +digits. + +To format strings with multiple \%char specifiers, use the new +\function{format_string()} function that works like \function{format()} +but also supports mixing \%char specifiers with +arbitrary text. + +A new \function{currency()} function was also added that formats a +number according to the current locale's settings. + +(Contributed by Georg Brandl.) +% Patch 1180296 + \item The \module{mailbox} module underwent a massive rewrite to add the capability to modify mailboxes in addition to reading them. A new set of classes that include \class{mbox}, \class{MH}, and @@ -1496,6 +1634,9 @@ tuple slicing, method lookups, and numeric operations, instead of performing many different operations and reducing the result to a single number as \file{pystone.py} does. +\item The \module{pyexpat} module now uses version 2.0 of the Expat parser. +(Contributed by Trent Mick.) + \item The old \module{regex} and \module{regsub} modules, which have been deprecated ever since Python 2.0, have finally been deleted. Other deleted modules: \module{statcache}, \module{tzparse}, @@ -1560,7 +1701,7 @@ year, number, name = s.unpack(data) \end{verbatim} You can also pack and unpack data to and from buffer objects directly -using the \method{pack_to(\var{buffer}, \var{offset}, \var{v1}, +using the \method{pack_into(\var{buffer}, \var{offset}, \var{v1}, \var{v2}, ...)} and \method{unpack_from(\var{buffer}, \var{offset})} methods. This lets you store data directly into an array or a memory-mapped file. @@ -1582,22 +1723,76 @@ string of build information like this: \code{"trunk:45355:45356M, Apr 13 2006, 07:42:19"}. (Contributed by Barry Warsaw.) +\item Another new function, \function{sys._current_frames()}, returns +the current stack frames for all running threads as a dictionary +mapping thread identifiers to the topmost stack frame currently active +in that thread at the time the function is called. (Contributed by +Tim Peters.) + \item The \class{TarFile} class in the \module{tarfile} module now has an \method{extractall()} method that extracts all members from the archive into the current working directory. It's also possible to set a different directory as the extraction target, and to unpack only a -subset of the archive's members. +subset of the archive's members. -A tarfile's compression can be autodetected by -using the mode \code{'r|*'}. +The compression used for a tarfile opened in stream mode can now be +autodetected using the mode \code{'r|*'}. % patch 918101 (Contributed by Lars Gust\"abel.) +\item The \module{threading} module now lets you set the stack size +used when new threads are created. The +\function{stack_size(\optional{\var{size}})} function returns the +currently configured stack size, and supplying the optional \var{size} +parameter sets a new value. Not all platforms support changing the +stack size, but Windows, POSIX threading, and OS/2 all do. +(Contributed by Andrew MacIntyre.) +% Patch 1454481 + \item The \module{unicodedata} module has been updated to use version 4.1.0 of the Unicode character database. Version 3.2.0 is required by some specifications, so it's still available as \member{unicodedata.ucd_3_2_0}. +\item New module: the \module{uuid} module generates +universally unique identifiers (UUIDs) according to \rfc{4122}. The +RFC defines several different UUID versions that are generated from a +starting string, from system properties, or purely randomly. This +module contains a \class{UUID} class and +functions named \function{uuid1()}, +\function{uuid3()}, \function{uuid4()}, and +\function{uuid5()} to generate different versions of UUID. (Version 2 UUIDs +are not specified in \rfc{4122} and are not supported by this module.) + +\begin{verbatim} +>>> import uuid +>>> # make a UUID based on the host ID and current time +>>> uuid.uuid1() +UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') + +>>> # make a UUID using an MD5 hash of a namespace UUID and a name +>>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') +UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') + +>>> # make a random UUID +>>> uuid.uuid4() +UUID('16fd2706-8baf-433b-82eb-8c7fada847da') + +>>> # make a UUID using a SHA-1 hash of a namespace UUID and a name +>>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') +UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') +\end{verbatim} + +(Contributed by Ka-Ping Yee.) + +\item The \module{weakref} module's \class{WeakKeyDictionary} and +\class{WeakValueDictionary} types gained new methods for iterating +over the weak references contained in the dictionary. +\method{iterkeyrefs()} and \method{keyrefs()} methods were +added to \class{WeakKeyDictionary}, and +\method{itervaluerefs()} and \method{valuerefs()} were added to +\class{WeakValueDictionary}. (Contributed by Fred L.~Drake, Jr.) + \item The \module{webbrowser} module received a number of enhancements. It's now usable as a script with \code{python -m webbrowser}, taking a @@ -1609,11 +1804,10 @@ to support this. The module's \function{open()} function supports an additional feature, an \var{autoraise} parameter that signals whether to raise the open window when possible. A number of additional browsers were added to the supported list such as Firefox, Opera, -Konqueror, and elinks. (Contributed by Oleg Broytmann and George +Konqueror, and elinks. (Contributed by Oleg Broytmann and Georg Brandl.) % Patch #754022 - \item The \module{xmlrpclib} module now supports returning \class{datetime} objects for the XML-RPC date type. Supply \code{use_datetime=True} to the \function{loads()} function @@ -1621,6 +1815,12 @@ Brandl.) (Contributed by Skip Montanaro.) % Patch 1120353 +\item The \module{zipfile} module now supports the ZIP64 version of the +format, meaning that a .zip archive can now be larger than 4~GiB and +can contain individual files larger than 4~GiB. (Contributed by +Ronald Oussoren.) +% Patch 1446489 + \item The \module{zlib} module's \class{Compress} and \class{Decompress} objects now support a \method{copy()} method that makes a copy of the object's internal state and returns a new @@ -1711,6 +1911,9 @@ of extension modules, now that \module{ctypes} is included with core Python. \seeurl{http://starship.python.net/crew/theller/ctypes/} {The ctypes web page, with a tutorial, reference, and FAQ.} +\seeurl{../lib/module-ctypes.html}{The documentation +for the \module{ctypes} module.} + \end{seealso} @@ -1822,7 +2025,6 @@ Please read the package's official documentation for more details. \seeurl{http://effbot.org/zone/element-index.htm} {Official documentation for ElementTree.} - \end{seealso} @@ -1873,6 +2075,13 @@ current digest state, \method{digest()} and \method{hexdigest()} return the digest value as a binary string or a string of hex digits, and \method{copy()} returns a new hashing object with the same digest state. +\begin{seealso} + +\seeurl{../lib/module-hashlib.html}{The documentation +for the \module{hashlib} module.} + +\end{seealso} + %====================================================================== \subsection{The sqlite3 package\label{module-sqlite}} @@ -1983,12 +2192,53 @@ For more information about the SQL dialect supported by SQLite, see {The SQLite web page; the documentation describes the syntax and the available data types for the supported SQL dialect.} +\seeurl{../lib/module-sqlite3.html}{The documentation +for the \module{sqlite3} module.} + \seepep{249}{Database API Specification 2.0}{PEP written by Marc-Andr\'e Lemburg.} \end{seealso} +%====================================================================== +\subsection{The wsgiref package\label{module-wsgiref}} + +% XXX should this be in a PEP 333 section instead? + +The Web Server Gateway Interface (WSGI) v1.0 defines a standard +interface between web servers and Python web applications and is +described in \pep{333}. The \module{wsgiref} package is a reference +implementation of the WSGI specification. + +The package includes a basic HTTP server that will run a WSGI +application; this server is useful for debugging but isn't intended for +production use. Setting up a server takes only a few lines of code: + +\begin{verbatim} +from wsgiref import simple_server + +wsgi_app = ... + +host = '' +port = 8000 +httpd = simple_server.make_server(host, port, wsgi_app) +httpd.serve_forever() +\end{verbatim} + +% XXX discuss structure of WSGI applications? +% XXX provide an example using Django or some other framework? + +\begin{seealso} + +\seeurl{http://www.wsgi.org}{A central web site for WSGI-related resources.} + +\seepep{333}{Python Web Server Gateway Interface v1.0}{PEP written by +Phillip J. Eby.} + +\end{seealso} + + % ====================================================================== \section{Build and C API Changes\label{build-api}} @@ -1996,13 +2246,25 @@ Changes to Python's build process and to the C API include: \begin{itemize} +\item The Python source tree was converted from CVS to Subversion, +in a complex migration procedure that was supervised and flawlessly +carried out by Martin von~L\"owis. The procedure was developed as +\pep{347}. + +\item Coverity, a company that markets a source code analysis tool +called Prevent, provided the results of their examination of the Python +source code. The analysis found about 60 bugs that +were quickly fixed. Many of the bugs were refcounting problems, often +occurring in error-handling code. See +\url{http://scan.coverity.com} for the statistics. + \item The largest change to the C API came from \pep{353}, which modifies the interpreter to use a \ctype{Py_ssize_t} type definition instead of \ctype{int}. See the earlier section~\ref{pep-353} for a discussion of this change. -\item The design of the bytecode compiler has changed a great deal, to -no longer generate bytecode by traversing the parse tree. Instead +\item The design of the bytecode compiler has changed a great deal, +no longer generating bytecode by traversing the parse tree. Instead the parse tree is converted to an abstract syntax tree (or AST), and it is the abstract syntax tree that's traversed to produce the bytecode. @@ -2022,12 +2284,13 @@ assignment = ast.body[0] for_loop = ast.body[1] \end{verbatim} -No documentation has been written for the AST code yet. To start -learning about it, read the definition of the various AST nodes in -\file{Parser/Python.asdl}. A Python script reads this file and -generates a set of C structure definitions in -\file{Include/Python-ast.h}. The \cfunction{PyParser_ASTFromString()} -and \cfunction{PyParser_ASTFromFile()}, defined in +No official documentation has been written for the AST code yet, but +\pep{339} discusses the design. To start learning about the code, read the +definition of the various AST nodes in \file{Parser/Python.asdl}. A +Python script reads this file and generates a set of C structure +definitions in \file{Include/Python-ast.h}. The +\cfunction{PyParser_ASTFromString()} and +\cfunction{PyParser_ASTFromFile()}, defined in \file{Include/pythonrun.h}, take Python source as input and return the root of an AST representing the contents. This AST can then be turned into a code object by \cfunction{PyAST_Compile()}. For more @@ -2042,6 +2305,32 @@ Grant Edwards, John Ehresman, Kurt Kaiser, Neal Norwitz, Tim Peters, Armin Rigo, and Neil Schemenauer, plus the participants in a number of AST sprints at conferences such as PyCon. +\item Evan Jones's patch to obmalloc, first described in a talk +at PyCon DC 2005, was applied. Python 2.4 allocated small objects in +256K-sized arenas, but never freed arenas. With this patch, Python +will free arenas when they're empty. The net effect is that on some +platforms, when you allocate many objects, Python's memory usage may +actually drop when you delete them and the memory may be returned to +the operating system. (Implemented by Evan Jones, and reworked by Tim +Peters.) + +Note that this change means extension modules must be more careful +when allocating memory. Python's API has many different +functions for allocating memory that are grouped into families. For +example, \cfunction{PyMem_Malloc()}, \cfunction{PyMem_Realloc()}, and +\cfunction{PyMem_Free()} are one family that allocates raw memory, +while \cfunction{PyObject_Malloc()}, \cfunction{PyObject_Realloc()}, +and \cfunction{PyObject_Free()} are another family that's supposed to +be used for creating Python objects. + +Previously these different families all reduced to the platform's +\cfunction{malloc()} and \cfunction{free()} functions. This meant +it didn't matter if you got things wrong and allocated memory with the +\cfunction{PyMem} function but freed it with the \cfunction{PyObject} +function. With 2.5's changes to obmalloc, these families now do different +things and mismatches will probably result in a segfault. You should +carefully test your C extension modules with Python 2.5. + \item The built-in set types now have an official C API. Call \cfunction{PySet_New()} and \cfunction{PyFrozenSet_New()} to create a new set, \cfunction{PySet_Add()} and \cfunction{PySet_Discard()} to @@ -2072,13 +2361,25 @@ Lundh at the NeedForSpeed sprint.) \var{dict})} can now accept a tuple of base classes as its \var{base} argument. (Contributed by Georg Brandl.) +\item The \cfunction{PyErr_Warn()} function for issuing warnings +is now deprecated in favour of \cfunction{PyErr_WarnEx(category, +message, stacklevel)} which lets you specify the number of stack +frames separating this function and the caller. A \var{stacklevel} of +1 is the function calling \cfunction{PyErr_WarnEx()}, 2 is the +function above that, and so forth. (Added by Neal Norwitz.) + \item The CPython interpreter is still written in C, but the code can now be compiled with a {\Cpp} compiler without errors. (Implemented by Anthony Baxter, Martin von~L\"owis, Skip Montanaro.) \item The \cfunction{PyRange_New()} function was removed. It was never documented, never used in the core code, and had dangerously lax -error checking. +error checking. In the unlikely case that your extensions were using +it, you can replace it by something like the following: +\begin{verbatim} +range = PyObject_CallFunction((PyObject*) &PyRange_Type, "lll", + start, stop, step); +\end{verbatim} \end{itemize} @@ -2105,54 +2406,6 @@ be searched for. %====================================================================== -\section{Other Changes and Fixes \label{section-other}} - -As usual, there were a bunch of other improvements and bugfixes -scattered throughout the source tree. A search through the SVN change -logs finds there were XXX patches applied and YYY bugs fixed between -Python 2.4 and 2.5. Both figures are likely to be underestimates. - -Some of the more notable changes are: - -\begin{itemize} - -\item Evan Jones's patch to obmalloc, first described in a talk -at PyCon DC 2005, was applied. Python 2.4 allocated small objects in -256K-sized arenas, but never freed arenas. With this patch, Python -will free arenas when they're empty. The net effect is that on some -platforms, when you allocate many objects, Python's memory usage may -actually drop when you delete them, and the memory may be returned to -the operating system. (Implemented by Evan Jones, and reworked by Tim -Peters.) - -Note that this change means extension modules need to be more careful -with how they allocate memory. Python's API has many different -functions for allocating memory that are grouped into families. For -example, \cfunction{PyMem_Malloc()}, \cfunction{PyMem_Realloc()}, and -\cfunction{PyMem_Free()} are one family that allocates raw memory, -while \cfunction{PyObject_Malloc()}, \cfunction{PyObject_Realloc()}, -and \cfunction{PyObject_Free()} are another family that's supposed to -be used for creating Python objects. - -Previously these different families all reduced to the platform's -\cfunction{malloc()} and \cfunction{free()} functions. This meant -it didn't matter if you got things wrong and allocated memory with the -\cfunction{PyMem} function but freed it with the \cfunction{PyObject} -function. With the obmalloc change, these families now do different -things, and mismatches will probably result in a segfault. You should -carefully test your C extension modules with Python 2.5. - -\item Coverity, a company that markets a source code analysis tool - called Prevent, provided the results of their examination of the Python - source code. The analysis found about 60 bugs that - were quickly fixed. Many of the bugs were refcounting problems, often - occurring in error-handling code. See - \url{http://scan.coverity.com} for the statistics. - -\end{itemize} - - -%====================================================================== \section{Porting to Python 2.5\label{porting}} This section lists previously described changes that may require @@ -2170,6 +2423,16 @@ was always a frame object. Because of the \pep{342} changes described in section~\ref{pep-342}, it's now possible for \member{gi_frame} to be \code{None}. +\item Library: the \module{csv} module is now stricter about multi-line quoted +fields. If your files contain newlines embedded within fields, the +input should be split into lines in a manner which preserves the +newline characters. + +\item Library: the \module{locale} module's +\function{format()} function's would previously +accept any string as long as no more than one \%char specifier +appeared. In Python 2.5, the argument must be exactly one \%char +specifier with no surrounding text. \item Library: The \module{pickle} and \module{cPickle} modules no longer accept a return value of \code{None} from the @@ -2206,7 +2469,10 @@ freed with the corresponding family's \cfunction{*_Free()} function. The author would like to thank the following people for offering suggestions, corrections and assistance with various drafts of this -article: Phillip J. Eby, Kent Johnson, Martin von~L\"owis, Fredrik Lundh, -Gustavo Niemeyer, James Pryor, Mike Rovner, Scott Weikart, Thomas Wouters. +article: Georg Brandl, Nick Coghlan, Phillip J. Eby, Lars Gust\"abel, +Raymond Hettinger, Ralf W. Grosse-Kunstleve, Kent Johnson, Iain Lowe, +Martin von~L\"owis, Fredrik Lundh, Andrew McNamara, Skip Montanaro, +Gustavo Niemeyer, Paul Prescod, James Pryor, Mike Rovner, Scott +Weikart, Barry Warsaw, Thomas Wouters. \end{document} diff --git a/Include/Python.h b/Include/Python.h index ca16c64..bbb9a08 100644 --- a/Include/Python.h +++ b/Include/Python.h @@ -35,7 +35,7 @@ #endif #include -#ifndef DONT_HAVE_ERRNO_H +#ifdef HAVE_ERRNO_H #include #endif #include diff --git a/Include/frameobject.h b/Include/frameobject.h index cce598b..794f651 100644 --- a/Include/frameobject.h +++ b/Include/frameobject.h @@ -41,8 +41,6 @@ typedef struct _frame { /* As of 2.3 f_lineno is only valid when tracing is active (i.e. when f_trace is set) -- at other times use PyCode_Addr2Line instead. */ int f_lineno; /* Current line number */ - int f_restricted; /* Flag set if restricted operations - in this scope */ int f_iblock; /* index in f_blockstack */ PyTryBlock f_blockstack[CO_MAXBLOCKS]; /* for try and loop blocks */ PyObject *f_localsplus[1]; /* locals+stack, dynamically sized */ @@ -54,6 +52,8 @@ typedef struct _frame { PyAPI_DATA(PyTypeObject) PyFrame_Type; #define PyFrame_Check(op) ((op)->ob_type == &PyFrame_Type) +#define PyFrame_IsRestricted(f) \ + ((f)->f_builtins != (f)->f_tstate->interp->builtins) PyAPI_FUNC(PyFrameObject *) PyFrame_New(PyThreadState *, PyCodeObject *, PyObject *, PyObject *); diff --git a/Include/pyerrors.h b/Include/pyerrors.h index 6006ac7..5df334b 100644 --- a/Include/pyerrors.h +++ b/Include/pyerrors.h @@ -218,10 +218,14 @@ PyAPI_FUNC(PyObject *) PyErr_NewException(char *name, PyObject *base, PyAPI_FUNC(void) PyErr_WriteUnraisable(PyObject *); /* Issue a warning or exception */ -PyAPI_FUNC(int) PyErr_Warn(PyObject *, char *); +PyAPI_FUNC(int) PyErr_WarnEx(PyObject *category, const char *msg, + Py_ssize_t stack_level); PyAPI_FUNC(int) PyErr_WarnExplicit(PyObject *, const char *, const char *, int, const char *, PyObject *); +/* PyErr_Warn is only for backwards compatability and will be removed. + Use PyErr_WarnEx instead. */ +#define PyErr_Warn(category, msg) PyErr_WarnEx(category, msg, 1) /* In sigcheck.c or signalmodule.c */ PyAPI_FUNC(int) PyErr_CheckSignals(void); diff --git a/Include/pyexpat.h b/Include/pyexpat.h index 50ed49f..1e79f4e 100644 --- a/Include/pyexpat.h +++ b/Include/pyexpat.h @@ -16,8 +16,8 @@ struct PyExpat_CAPI the end, if needed */ const XML_LChar * (*ErrorString)(enum XML_Error code); enum XML_Error (*GetErrorCode)(XML_Parser parser); - int (*GetErrorColumnNumber)(XML_Parser parser); - int (*GetErrorLineNumber)(XML_Parser parser); + XML_Size (*GetErrorColumnNumber)(XML_Parser parser); + XML_Size (*GetErrorLineNumber)(XML_Parser parser); enum XML_Status (*Parse)( XML_Parser parser, const char *s, int len, int isFinal); XML_Parser (*ParserCreate_MM)( diff --git a/Include/pyport.h b/Include/pyport.h index 47b9f70..be6c51f 100644 --- a/Include/pyport.h +++ b/Include/pyport.h @@ -240,10 +240,10 @@ typedef Py_intptr_t Py_ssize_t; * to your pyconfig.h. Python code beyond this should check HAVE_STAT and * HAVE_FSTAT instead. * Also - * #define DONT_HAVE_SYS_STAT_H - * if doesn't exist on your platform, and + * #define HAVE_SYS_STAT_H + * if exists on your platform, and * #define HAVE_STAT_H - * if does (don't look at me -- ths mess is inherited). + * if does. */ #ifndef DONT_HAVE_STAT #define HAVE_STAT @@ -258,7 +258,7 @@ typedef Py_intptr_t Py_ssize_t; #include "unixstuff.h" #endif -#ifndef DONT_HAVE_SYS_STAT_H +#ifdef HAVE_SYS_STAT_H #if defined(PYOS_OS2) && defined(PYCC_GCC) #include #endif diff --git a/Include/pystate.h b/Include/pystate.h index bfd3548..cf29695 100644 --- a/Include/pystate.h +++ b/Include/pystate.h @@ -171,6 +171,11 @@ PyAPI_FUNC(void) PyGILState_Release(PyGILState_STATE); */ PyAPI_FUNC(PyThreadState *) PyGILState_GetThisThreadState(void); +/* The implementation of sys._current_frames() Returns a dict mapping + thread id to that thread's current frame. +*/ +PyAPI_FUNC(PyObject *) _PyThread_CurrentFrames(void); + /* Routines for advanced debuggers, requested by David Beazley. Don't use unless you know what you are doing! */ PyAPI_FUNC(PyInterpreterState *) PyInterpreterState_Head(void); diff --git a/Include/pythread.h b/Include/pythread.h index 0fa8db0..f26db16 100644 --- a/Include/pythread.h +++ b/Include/pythread.h @@ -25,6 +25,9 @@ PyAPI_FUNC(int) PyThread_acquire_lock(PyThread_type_lock, int); #define NOWAIT_LOCK 0 PyAPI_FUNC(void) PyThread_release_lock(PyThread_type_lock); +PyAPI_FUNC(size_t) PyThread_get_stacksize(void); +PyAPI_FUNC(int) PyThread_set_stacksize(size_t); + #ifndef NO_EXIT_PROG PyAPI_FUNC(void) PyThread_exit_prog(int); PyAPI_FUNC(void) PyThread__PyThread_exit_prog(int); diff --git a/Include/setobject.h b/Include/setobject.h index cc93968..a16c2f7 100644 --- a/Include/setobject.h +++ b/Include/setobject.h @@ -35,14 +35,14 @@ typedef struct _setobject PySetObject; struct _setobject { PyObject_HEAD - int fill; /* # Active + # Dummy */ - int used; /* # Active */ + Py_ssize_t fill; /* # Active + # Dummy */ + Py_ssize_t used; /* # Active */ /* The table contains mask + 1 slots, and that's a power of 2. * We store the mask instead of the size because the mask is more * frequently needed. */ - int mask; + Py_ssize_t mask; /* table points to smalltable for small tables, else to * additional malloc'ed memory. table is never NULL! This rule diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h index 3177051..c7e07a8 100644 --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -938,6 +938,13 @@ PyAPI_FUNC(PyObject*) PyUnicode_DecodeMBCS( const char *errors /* error handling */ ); +PyAPI_FUNC(PyObject*) PyUnicode_DecodeMBCSStateful( + const char *string, /* MBCS encoded string */ + Py_ssize_t length, /* size of string */ + const char *errors, /* error handling */ + Py_ssize_t *consumed /* bytes consumed */ + ); + PyAPI_FUNC(PyObject*) PyUnicode_AsMBCSString( PyObject *unicode /* Unicode object */ ); diff --git a/Include/weakrefobject.h b/Include/weakrefobject.h index daf490f..0a659b0 100644 --- a/Include/weakrefobject.h +++ b/Include/weakrefobject.h @@ -62,7 +62,7 @@ PyAPI_FUNC(PyObject *) PyWeakref_NewProxy(PyObject *ob, PyObject *callback); PyAPI_FUNC(PyObject *) PyWeakref_GetObject(PyObject *ref); -PyAPI_FUNC(long) _PyWeakref_GetWeakrefCount(PyWeakReference *head); +PyAPI_FUNC(Py_ssize_t) _PyWeakref_GetWeakrefCount(PyWeakReference *head); PyAPI_FUNC(void) _PyWeakref_ClearRef(PyWeakReference *self); diff --git a/Lib/Queue.py b/Lib/Queue.py index 51ad354..0f80584 100644 --- a/Lib/Queue.py +++ b/Lib/Queue.py @@ -14,11 +14,11 @@ class Full(Exception): pass class Queue: - def __init__(self, maxsize=0): - """Initialize a queue object with a given maximum size. + """Create a queue object with a given maximum size. - If maxsize is <= 0, the queue size is infinite. - """ + If maxsize is <= 0, the queue size is infinite. + """ + def __init__(self, maxsize=0): try: import threading except ImportError: diff --git a/Lib/SimpleHTTPServer.py b/Lib/SimpleHTTPServer.py index 089936f..fae551a 100644 --- a/Lib/SimpleHTTPServer.py +++ b/Lib/SimpleHTTPServer.py @@ -192,6 +192,8 @@ class SimpleHTTPRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler): else: return self.extensions_map[''] + if not mimetypes.inited: + mimetypes.init() # try to read system mime.types extensions_map = mimetypes.types_map.copy() extensions_map.update({ '': 'application/octet-stream', # Default diff --git a/Lib/UserString.py b/Lib/UserString.py index 473ee88..60dc34b 100755 --- a/Lib/UserString.py +++ b/Lib/UserString.py @@ -5,14 +5,13 @@ Note: string objects have grown methods in Python 1.6 This module requires Python 1.6 or later. """ -from types import StringTypes import sys __all__ = ["UserString","MutableString"] class UserString: def __init__(self, seq): - if isinstance(seq, StringTypes): + if isinstance(seq, basestring): self.data = seq elif isinstance(seq, UserString): self.data = seq.data[:] @@ -43,12 +42,12 @@ class UserString: def __add__(self, other): if isinstance(other, UserString): return self.__class__(self.data + other.data) - elif isinstance(other, StringTypes): + elif isinstance(other, basestring): return self.__class__(self.data + other) else: return self.__class__(self.data + str(other)) def __radd__(self, other): - if isinstance(other, StringTypes): + if isinstance(other, basestring): return self.__class__(other + self.data) else: return self.__class__(str(other) + self.data) @@ -163,7 +162,7 @@ class MutableString(UserString): start = max(start, 0); end = max(end, 0) if isinstance(sub, UserString): self.data = self.data[:start]+sub.data+self.data[end:] - elif isinstance(sub, StringTypes): + elif isinstance(sub, basestring): self.data = self.data[:start]+sub+self.data[end:] else: self.data = self.data[:start]+str(sub)+self.data[end:] @@ -175,7 +174,7 @@ class MutableString(UserString): def __iadd__(self, other): if isinstance(other, UserString): self.data += other.data - elif isinstance(other, StringTypes): + elif isinstance(other, basestring): self.data += other else: self.data += str(other) diff --git a/Lib/_MozillaCookieJar.py b/Lib/_MozillaCookieJar.py index 1776b93..4fd6de3 100644 --- a/Lib/_MozillaCookieJar.py +++ b/Lib/_MozillaCookieJar.py @@ -63,8 +63,7 @@ class MozillaCookieJar(FileCookieJar): if line.endswith("\n"): line = line[:-1] # skip comments and blank lines XXX what is $ for? - if (line.strip().startswith("#") or - line.strip().startswith("$") or + if (line.strip().startswith(("#", "$")) or line.strip() == ""): continue diff --git a/Lib/binhex.py b/Lib/binhex.py index 4f3882a..0f3e3c4 100644 --- a/Lib/binhex.py +++ b/Lib/binhex.py @@ -44,22 +44,14 @@ RUNCHAR=chr(0x90) # run-length introducer # # Workarounds for non-mac machines. -if os.name == 'mac': - import macfs - import MacOS - try: - openrf = MacOS.openrf - except AttributeError: - # Backward compatibility - openrf = open - - def FInfo(): - return macfs.FInfo() +try: + from Carbon.File import FSSpec, FInfo + from MacOS import openrf def getfileinfo(name): - finfo = macfs.FSSpec(name).GetFInfo() + finfo = FSSpec(name).FSpGetFInfo() dir, file = os.path.split(name) - # XXXX Get resource/data sizes + # XXX Get resource/data sizes fp = open(name, 'rb') fp.seek(0, 2) dlen = fp.tell() @@ -75,7 +67,7 @@ if os.name == 'mac': mode = '*' + mode[0] return openrf(name, mode) -else: +except ImportError: # # Glue code for non-macintosh usage # @@ -183,7 +175,7 @@ class BinHex: ofname = ofp ofp = open(ofname, 'w') if os.name == 'mac': - fss = macfs.FSSpec(ofname) + fss = FSSpec(ofname) fss.SetCreatorType('BnHq', 'TEXT') ofp.write('(This file must be converted with BinHex 4.0)\n\n:') hqxer = _Hqxcoderengine(ofp) @@ -486,7 +478,7 @@ def hexbin(inp, out): if not out: out = ifp.FName if os.name == 'mac': - ofss = macfs.FSSpec(out) + ofss = FSSpec(out) out = ofss.as_pathname() ofp = open(out, 'wb') @@ -519,6 +511,7 @@ def hexbin(inp, out): def _test(): if os.name == 'mac': + import macfs fss, ok = macfs.PromptGetFile('File to convert:') if not ok: sys.exit(0) diff --git a/Lib/bsddb/__init__.py b/Lib/bsddb/__init__.py index 90ed362..cf32668 100644 --- a/Lib/bsddb/__init__.py +++ b/Lib/bsddb/__init__.py @@ -33,7 +33,10 @@ #---------------------------------------------------------------------- -"""Support for BerkeleyDB 3.2 through 4.2. +"""Support for BerkeleyDB 3.3 through 4.4 with a simple interface. + +For the full featured object oriented interface use the bsddb.db module +instead. It mirrors the Sleepycat BerkeleyDB C API. """ try: @@ -43,8 +46,10 @@ try: # python as bsddb._bsddb. import _pybsddb _bsddb = _pybsddb + from bsddb3.dbutils import DeadlockWrap as _DeadlockWrap else: import _bsddb + from bsddb.dbutils import DeadlockWrap as _DeadlockWrap except ImportError: # Remove ourselves from sys.modules import sys @@ -70,7 +75,7 @@ if sys.version >= '2.3': exec """ class _iter_mixin(UserDict.DictMixin): def _make_iter_cursor(self): - cur = self.db.cursor() + cur = _DeadlockWrap(self.db.cursor) key = id(cur) self._cursor_refs[key] = ref(cur, self._gen_cref_cleaner(key)) return cur @@ -90,19 +95,19 @@ class _iter_mixin(UserDict.DictMixin): # since we're only returning keys, we call the cursor # methods with flags=0, dlen=0, dofs=0 - key = cur.first(0,0,0)[0] + key = _DeadlockWrap(cur.first, 0,0,0)[0] yield key next = cur.next while 1: try: - key = next(0,0,0)[0] + key = _DeadlockWrap(next, 0,0,0)[0] yield key except _bsddb.DBCursorClosedError: cur = self._make_iter_cursor() # FIXME-20031101-greg: race condition. cursor could # be closed by another thread before this call. - cur.set(key,0,0,0) + _DeadlockWrap(cur.set, key,0,0,0) next = cur.next except _bsddb.DBNotFoundError: return @@ -119,21 +124,21 @@ class _iter_mixin(UserDict.DictMixin): # FIXME-20031102-greg: race condition. cursor could # be closed by another thread before this call. - kv = cur.first() + kv = _DeadlockWrap(cur.first) key = kv[0] yield kv next = cur.next while 1: try: - kv = next() + kv = _DeadlockWrap(next) key = kv[0] yield kv except _bsddb.DBCursorClosedError: cur = self._make_iter_cursor() # FIXME-20031101-greg: race condition. cursor could # be closed by another thread before this call. - cur.set(key,0,0,0) + _DeadlockWrap(cur.set, key,0,0,0) next = cur.next except _bsddb.DBNotFoundError: return @@ -177,9 +182,9 @@ class _DBWithCursor(_iter_mixin): def _checkCursor(self): if self.dbc is None: - self.dbc = self.db.cursor() + self.dbc = _DeadlockWrap(self.db.cursor) if self.saved_dbc_key is not None: - self.dbc.set(self.saved_dbc_key) + _DeadlockWrap(self.dbc.set, self.saved_dbc_key) self.saved_dbc_key = None # This method is needed for all non-cursor DB calls to avoid @@ -192,15 +197,15 @@ class _DBWithCursor(_iter_mixin): self.dbc = None if save: try: - self.saved_dbc_key = c.current(0,0,0)[0] + self.saved_dbc_key = _DeadlockWrap(c.current, 0,0,0)[0] except db.DBError: pass - c.close() + _DeadlockWrap(c.close) del c for cref in self._cursor_refs.values(): c = cref() if c is not None: - c.close() + _DeadlockWrap(c.close) def _checkOpen(self): if self.db is None: @@ -211,73 +216,77 @@ class _DBWithCursor(_iter_mixin): def __len__(self): self._checkOpen() - return len(self.db) + return _DeadlockWrap(lambda: len(self.db)) # len(self.db) def __getitem__(self, key): self._checkOpen() - return self.db[key] + return _DeadlockWrap(lambda: self.db[key]) # self.db[key] def __setitem__(self, key, value): self._checkOpen() self._closeCursors() - self.db[key] = value + def wrapF(): + self.db[key] = value + _DeadlockWrap(wrapF) # self.db[key] = value def __delitem__(self, key): self._checkOpen() self._closeCursors() - del self.db[key] + def wrapF(): + del self.db[key] + _DeadlockWrap(wrapF) # del self.db[key] def close(self): self._closeCursors(save=0) if self.dbc is not None: - self.dbc.close() + _DeadlockWrap(self.dbc.close) v = 0 if self.db is not None: - v = self.db.close() + v = _DeadlockWrap(self.db.close) self.dbc = None self.db = None return v def keys(self): self._checkOpen() - return self.db.keys() + return _DeadlockWrap(self.db.keys) def has_key(self, key): self._checkOpen() - return self.db.has_key(key) + return _DeadlockWrap(self.db.has_key, key) def set_location(self, key): self._checkOpen() self._checkCursor() - return self.dbc.set_range(key) + return _DeadlockWrap(self.dbc.set_range, key) def next(self): self._checkOpen() self._checkCursor() - rv = self.dbc.next() + rv = _DeadlockWrap(self.dbc.next) return rv def previous(self): self._checkOpen() self._checkCursor() - rv = self.dbc.prev() + rv = _DeadlockWrap(self.dbc.prev) return rv def first(self): self._checkOpen() self._checkCursor() - rv = self.dbc.first() + rv = _DeadlockWrap(self.dbc.first) return rv def last(self): self._checkOpen() self._checkCursor() - rv = self.dbc.last() + rv = _DeadlockWrap(self.dbc.last) return rv def sync(self): self._checkOpen() - return self.db.sync() + return _DeadlockWrap(self.db.sync) #---------------------------------------------------------------------- @@ -385,5 +394,4 @@ try: except ImportError: db.DB_THREAD = 0 - #---------------------------------------------------------------------- diff --git a/Lib/bsddb/dbrecio.py b/Lib/bsddb/dbrecio.py index 22e382a..d439f32 100644 --- a/Lib/bsddb/dbrecio.py +++ b/Lib/bsddb/dbrecio.py @@ -75,7 +75,7 @@ class DBRecIO: dlen = newpos - self.pos - r = self.db.get(key, txn=self.txn, dlen=dlen, doff=self.pos) + r = self.db.get(self.key, txn=self.txn, dlen=dlen, doff=self.pos) self.pos = newpos return r @@ -121,7 +121,7 @@ class DBRecIO: "Negative size not allowed") elif size < self.pos: self.pos = size - self.db.put(key, "", txn=self.txn, dlen=self.len-size, doff=size) + self.db.put(self.key, "", txn=self.txn, dlen=self.len-size, doff=size) def write(self, s): if self.closed: @@ -131,7 +131,7 @@ class DBRecIO: self.buflist.append('\0'*(self.pos - self.len)) self.len = self.pos newpos = self.pos + len(s) - self.db.put(key, s, txn=self.txn, dlen=len(s), doff=self.pos) + self.db.put(self.key, s, txn=self.txn, dlen=len(s), doff=self.pos) self.pos = newpos def writelines(self, list): diff --git a/Lib/bsddb/dbtables.py b/Lib/bsddb/dbtables.py index 369db43..492d5fd 100644 --- a/Lib/bsddb/dbtables.py +++ b/Lib/bsddb/dbtables.py @@ -32,6 +32,12 @@ except ImportError: # For Python 2.3 from bsddb.db import * +# XXX(nnorwitz): is this correct? DBIncompleteError is conditional in _bsddb.c +try: + DBIncompleteError +except NameError: + class DBIncompleteError(Exception): + pass class TableDBError(StandardError): pass diff --git a/Lib/bsddb/dbutils.py b/Lib/bsddb/dbutils.py index 3f63842..6dcfdd5 100644 --- a/Lib/bsddb/dbutils.py +++ b/Lib/bsddb/dbutils.py @@ -22,14 +22,14 @@ # # import the time.sleep function in a namespace safe way to allow -# "from bsddb.db import *" +# "from bsddb.dbutils import *" # from time import sleep as _sleep import db # always sleep at least N seconds between retrys -_deadlock_MinSleepTime = 1.0/64 +_deadlock_MinSleepTime = 1.0/128 # never sleep more than N seconds between retrys _deadlock_MaxSleepTime = 3.14159 @@ -57,7 +57,7 @@ def DeadlockWrap(function, *_args, **_kwargs): max_retries = _kwargs.get('max_retries', -1) if _kwargs.has_key('max_retries'): del _kwargs['max_retries'] - while 1: + while True: try: return function(*_args, **_kwargs) except db.DBLockDeadlockError: diff --git a/Lib/bsddb/test/test_basics.py b/Lib/bsddb/test/test_basics.py index bec5da3..d6d507f 100644 --- a/Lib/bsddb/test/test_basics.py +++ b/Lib/bsddb/test/test_basics.py @@ -562,6 +562,9 @@ class BasicTestCase(unittest.TestCase): num = d.truncate() assert num == 0, "truncate on empty DB returned nonzero (%r)" % (num,) + #---------------------------------------- + + #---------------------------------------------------------------------- @@ -583,18 +586,40 @@ class BasicHashWithThreadFlagTestCase(BasicTestCase): dbopenflags = db.DB_THREAD -class BasicBTreeWithEnvTestCase(BasicTestCase): - dbtype = db.DB_BTREE +class BasicWithEnvTestCase(BasicTestCase): dbopenflags = db.DB_THREAD useEnv = 1 envflags = db.DB_THREAD | db.DB_INIT_MPOOL | db.DB_INIT_LOCK + #---------------------------------------- + + def test07_EnvRemoveAndRename(self): + if not self.env: + return + + if verbose: + print '\n', '-=' * 30 + print "Running %s.test07_EnvRemoveAndRename..." % self.__class__.__name__ + + # can't rename or remove an open DB + self.d.close() + + newname = self.filename + '.renamed' + self.env.dbrename(self.filename, None, newname) + self.env.dbremove(newname) + + # dbremove and dbrename are in 4.1 and later + if db.version() < (4,1): + del test07_EnvRemoveAndRename -class BasicHashWithEnvTestCase(BasicTestCase): + #---------------------------------------- + +class BasicBTreeWithEnvTestCase(BasicWithEnvTestCase): + dbtype = db.DB_BTREE + + +class BasicHashWithEnvTestCase(BasicWithEnvTestCase): dbtype = db.DB_HASH - dbopenflags = db.DB_THREAD - useEnv = 1 - envflags = db.DB_THREAD | db.DB_INIT_MPOOL | db.DB_INIT_LOCK #---------------------------------------------------------------------- diff --git a/Lib/compiler/future.py b/Lib/compiler/future.py index 39c3bb9..fef189e 100644 --- a/Lib/compiler/future.py +++ b/Lib/compiler/future.py @@ -23,14 +23,7 @@ class FutureParser: def visitModule(self, node): stmt = node.node - found_docstring = False for s in stmt.nodes: - # Skip over docstrings - if not found_docstring and isinstance(s, ast.Discard) \ - and isinstance(s.expr, ast.Const) \ - and isinstance(s.expr.value, str): - found_docstring = True - continue if not self.check_stmt(s): break diff --git a/Lib/compiler/pycodegen.py b/Lib/compiler/pycodegen.py index c093128..c8a9779 100644 --- a/Lib/compiler/pycodegen.py +++ b/Lib/compiler/pycodegen.py @@ -380,16 +380,7 @@ class CodeGenerator: self.set_lineno(node) for default in node.defaults: self.visit(default) - frees = gen.scope.get_free_vars() - if frees: - for name in frees: - self.emit('LOAD_CLOSURE', name) - self.emit('LOAD_CONST', gen) - self.emit('MAKE_CLOSURE', len(node.defaults)) - else: - self.emit('LOAD_CONST', gen) - self.emit('MAKE_FUNCTION', len(node.defaults)) - + self._makeClosure(gen, len(node.defaults)) for i in range(ndecorators): self.emit('CALL_FUNCTION', 1) @@ -403,14 +394,7 @@ class CodeGenerator: for base in node.bases: self.visit(base) self.emit('BUILD_TUPLE', len(node.bases)) - frees = gen.scope.get_free_vars() - for name in frees: - self.emit('LOAD_CLOSURE', name) - self.emit('LOAD_CONST', gen) - if frees: - self.emit('MAKE_CLOSURE', 0) - else: - self.emit('MAKE_FUNCTION', 0) + self._makeClosure(gen, 0) self.emit('CALL_FUNCTION', 0) self.emit('BUILD_CLASS') self.storeName(node.name) @@ -642,22 +626,25 @@ class CodeGenerator: self.newBlock() self.emit('POP_TOP') - def visitGenExpr(self, node): - gen = GenExprCodeGenerator(node, self.scopes, self.class_name, - self.get_module()) - walk(node.code, gen) - gen.finish() - self.set_lineno(node) + def _makeClosure(self, gen, args): frees = gen.scope.get_free_vars() if frees: for name in frees: self.emit('LOAD_CLOSURE', name) + self.emit('BUILD_TUPLE', len(frees)) self.emit('LOAD_CONST', gen) - self.emit('MAKE_CLOSURE', 0) + self.emit('MAKE_CLOSURE', args) else: self.emit('LOAD_CONST', gen) - self.emit('MAKE_FUNCTION', 0) + self.emit('MAKE_FUNCTION', args) + def visitGenExpr(self, node): + gen = GenExprCodeGenerator(node, self.scopes, self.class_name, + self.get_module()) + walk(node.code, gen) + gen.finish() + self.set_lineno(node) + self._makeClosure(gen, 0) # precomputation of outmost iterable self.visit(node.code.quals[0].iter) self.emit('GET_ITER') diff --git a/Lib/compiler/symbols.py b/Lib/compiler/symbols.py index c608f64..8eb5fce 100644 --- a/Lib/compiler/symbols.py +++ b/Lib/compiler/symbols.py @@ -191,7 +191,7 @@ class GenExprScope(Scope): self.add_param('[outmost-iterable]') def get_names(self): - keys = Scope.get_names() + keys = Scope.get_names(self) return keys class LambdaScope(FunctionScope): diff --git a/Lib/compiler/transformer.py b/Lib/compiler/transformer.py index 96bcce3..8d256ed 100644 --- a/Lib/compiler/transformer.py +++ b/Lib/compiler/transformer.py @@ -536,12 +536,7 @@ class Transformer: lineno=nodelist[0][2]) def try_stmt(self, nodelist): - # 'try' ':' suite (except_clause ':' suite)+ ['else' ':' suite] - # | 'try' ':' suite 'finally' ':' suite - if nodelist[3][0] != symbol.except_clause: - return self.com_try_finally(nodelist) - - return self.com_try_except(nodelist) + return self.com_try_except_finally(nodelist) def with_stmt(self, nodelist): return self.com_with(nodelist) @@ -729,22 +724,20 @@ class Transformer: def atom(self, nodelist): return self._atom_dispatch[nodelist[0][0]](nodelist) - n.lineno = nodelist[0][2] - return n def atom_lpar(self, nodelist): if nodelist[1][0] == token.RPAR: - return Tuple(()) + return Tuple((), lineno=nodelist[0][2]) return self.com_node(nodelist[1]) def atom_lsqb(self, nodelist): if nodelist[1][0] == token.RSQB: - return List(()) + return List((), lineno=nodelist[0][2]) return self.com_list_constructor(nodelist[1]) def atom_lbrace(self, nodelist): if nodelist[1][0] == token.RBRACE: - return Dict(()) + return Dict((), lineno=nodelist[0][2]) return self.com_dictmaker(nodelist[1]) def atom_backquote(self, nodelist): @@ -919,18 +912,21 @@ class Transformer: bases.append(self.com_node(node[i])) return bases - def com_try_finally(self, nodelist): - # try_fin_stmt: "try" ":" suite "finally" ":" suite - return TryFinally(self.com_node(nodelist[2]), - self.com_node(nodelist[5]), - lineno=nodelist[0][2]) + def com_try_except_finally(self, nodelist): + # ('try' ':' suite + # ((except_clause ':' suite)+ ['else' ':' suite] ['finally' ':' suite] + # | 'finally' ':' suite)) + + if nodelist[3][0] == token.NAME: + # first clause is a finally clause: only try-finally + return TryFinally(self.com_node(nodelist[2]), + self.com_node(nodelist[5]), + lineno=nodelist[0][2]) - def com_try_except(self, nodelist): - # try_except: 'try' ':' suite (except_clause ':' suite)* ['else' suite] #tryexcept: [TryNode, [except_clauses], elseNode)] - stmt = self.com_node(nodelist[2]) clauses = [] elseNode = None + finallyNode = None for i in range(3, len(nodelist), 3): node = nodelist[i] if node[0] == symbol.except_clause: @@ -946,9 +942,16 @@ class Transformer: clauses.append((expr1, expr2, self.com_node(nodelist[i+2]))) if node[0] == token.NAME: - elseNode = self.com_node(nodelist[i+2]) - return TryExcept(self.com_node(nodelist[2]), clauses, elseNode, - lineno=nodelist[0][2]) + if node[1] == 'else': + elseNode = self.com_node(nodelist[i+2]) + elif node[1] == 'finally': + finallyNode = self.com_node(nodelist[i+2]) + try_except = TryExcept(self.com_node(nodelist[2]), clauses, elseNode, + lineno=nodelist[0][2]) + if finallyNode: + return TryFinally(try_except, finallyNode, lineno=nodelist[0][2]) + else: + return try_except def com_with(self, nodelist): # with_stmt: 'with' expr [with_var] ':' suite @@ -1138,7 +1141,7 @@ class Transformer: values = [] for i in range(1, len(nodelist), 2): values.append(self.com_node(nodelist[i])) - return List(values) + return List(values, lineno=values[0].lineno) if hasattr(symbol, 'gen_for'): def com_generator_expression(self, expr, node): @@ -1185,7 +1188,7 @@ class Transformer: for i in range(1, len(nodelist), 4): items.append((self.com_node(nodelist[i]), self.com_node(nodelist[i+2]))) - return Dict(items) + return Dict(items, lineno=items[0][0].lineno) def com_apply_trailer(self, primaryNode, nodelist): t = nodelist[1][0] @@ -1379,6 +1382,7 @@ _doc_nodes = [ symbol.testlist, symbol.testlist_safe, symbol.test, + symbol.or_test, symbol.and_test, symbol.not_test, symbol.comparison, diff --git a/Lib/ctypes/__init__.py b/Lib/ctypes/__init__.py index f2ddbaa..a4e3c36 100644 --- a/Lib/ctypes/__init__.py +++ b/Lib/ctypes/__init__.py @@ -1,9 +1,11 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### """create and manipulate C data types in Python""" import os as _os, sys as _sys -from itertools import chain as _chain -__version__ = "0.9.9.6" +__version__ = "1.0.0" from _ctypes import Union, Structure, Array from _ctypes import _Pointer @@ -20,6 +22,23 @@ if __version__ != _ctypes_version: if _os.name in ("nt", "ce"): from _ctypes import FormatError +DEFAULT_MODE = RTLD_LOCAL +if _os.name == "posix" and _sys.platform == "darwin": + import gestalt + + # gestalt.gestalt("sysv") returns the version number of the + # currently active system file as BCD. + # On OS X 10.4.6 -> 0x1046 + # On OS X 10.2.8 -> 0x1028 + # See also http://www.rgaros.nl/gestalt/ + # + # On OS X 10.3, we use RTLD_GLOBAL as default mode + # because RTLD_LOCAL does not work at least on some + # libraries. + + if gestalt.gestalt("sysv") < 0x1040: + DEFAULT_MODE = RTLD_GLOBAL + from _ctypes import FUNCFLAG_CDECL as _FUNCFLAG_CDECL, \ FUNCFLAG_PYTHONAPI as _FUNCFLAG_PYTHONAPI @@ -67,7 +86,7 @@ def CFUNCTYPE(restype, *argtypes): restype: the result type argtypes: a sequence specifying the argument types - The function prototype can be called in three ways to create a + The function prototype can be called in different ways to create a callable object: prototype(integer address) -> foreign function @@ -111,7 +130,7 @@ if _os.name in ("nt", "ce"): elif _os.name == "posix": from _ctypes import dlopen as _dlopen -from _ctypes import sizeof, byref, addressof, alignment +from _ctypes import sizeof, byref, addressof, alignment, resize from _ctypes import _SimpleCData class py_object(_SimpleCData): @@ -282,7 +301,7 @@ class CDLL(object): _flags_ = _FUNCFLAG_CDECL _restype_ = c_int # default, can be overridden in instances - def __init__(self, name, mode=RTLD_LOCAL, handle=None): + def __init__(self, name, mode=DEFAULT_MODE, handle=None): self._name = name if handle is None: self._handle = _dlopen(self._name, mode) @@ -293,18 +312,19 @@ class CDLL(object): return "<%s '%s', handle %x at %x>" % \ (self.__class__.__name__, self._name, (self._handle & (_sys.maxint*2 + 1)), - id(self)) + id(self) & (_sys.maxint*2 + 1)) def __getattr__(self, name): if name.startswith('__') and name.endswith('__'): raise AttributeError, name - return self.__getitem__(name) + func = self.__getitem__(name) + setattr(self, name, func) + return func def __getitem__(self, name_or_ordinal): func = self._FuncPtr((name_or_ordinal, self)) if not isinstance(name_or_ordinal, (int, long)): func.__name__ = name_or_ordinal - setattr(self, name_or_ordinal, func) return func class PyDLL(CDLL): @@ -419,12 +439,10 @@ def PYFUNCTYPE(restype, *argtypes): _restype_ = restype _flags_ = _FUNCFLAG_CDECL | _FUNCFLAG_PYTHONAPI return CFunctionType -_cast = PYFUNCTYPE(py_object, c_void_p, py_object)(_cast_addr) +_cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr) def cast(obj, typ): - result = _cast(obj, typ) - result.__keepref = obj - return result + return _cast(obj, obj, typ) _string_at = CFUNCTYPE(py_object, c_void_p, c_int)(_string_at_addr) def string_at(ptr, size=0): @@ -446,52 +464,21 @@ else: return _wstring_at(ptr, size) -if _os.name == "nt": # COM stuff +if _os.name in ("nt", "ce"): # COM stuff def DllGetClassObject(rclsid, riid, ppv): - # First ask ctypes.com.server than comtypes.server for the - # class object. - - # trick py2exe by doing dynamic imports - result = -2147221231 # CLASS_E_CLASSNOTAVAILABLE try: - ctcom = __import__("ctypes.com.server", globals(), locals(), ['*']) + ccom = __import__("comtypes.server.inprocserver", globals(), locals(), ['*']) except ImportError: - pass + return -2147221231 # CLASS_E_CLASSNOTAVAILABLE else: - result = ctcom.DllGetClassObject(rclsid, riid, ppv) - - if result == -2147221231: # CLASS_E_CLASSNOTAVAILABLE - try: - ccom = __import__("comtypes.server", globals(), locals(), ['*']) - except ImportError: - pass - else: - result = ccom.DllGetClassObject(rclsid, riid, ppv) - - return result + return ccom.DllGetClassObject(rclsid, riid, ppv) def DllCanUnloadNow(): - # First ask ctypes.com.server than comtypes.server if we can unload or not. - # trick py2exe by doing dynamic imports - result = 0 # S_OK - try: - ctcom = __import__("ctypes.com.server", globals(), locals(), ['*']) - except ImportError: - pass - else: - result = ctcom.DllCanUnloadNow() - if result != 0: # != S_OK - return result - try: - ccom = __import__("comtypes.server", globals(), locals(), ['*']) + ccom = __import__("comtypes.server.inprocserver", globals(), locals(), ['*']) except ImportError: - return result - try: - return ccom.DllCanUnloadNow() - except AttributeError: - pass - return result + return 0 # S_OK + return ccom.DllCanUnloadNow() from ctypes._endian import BigEndianStructure, LittleEndianStructure diff --git a/Lib/ctypes/_endian.py b/Lib/ctypes/_endian.py index 5818ae1..6de0d47 100644 --- a/Lib/ctypes/_endian.py +++ b/Lib/ctypes/_endian.py @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### import sys from ctypes import * diff --git a/Lib/ctypes/macholib/__init__.py b/Lib/ctypes/macholib/__init__.py index 5621def..36149d2 100644 --- a/Lib/ctypes/macholib/__init__.py +++ b/Lib/ctypes/macholib/__init__.py @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### """ Enough Mach-O to make your head spin. diff --git a/Lib/ctypes/macholib/dyld.py b/Lib/ctypes/macholib/dyld.py index a336fd0..14e2139 100644 --- a/Lib/ctypes/macholib/dyld.py +++ b/Lib/ctypes/macholib/dyld.py @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### """ dyld emulation """ diff --git a/Lib/ctypes/macholib/dylib.py b/Lib/ctypes/macholib/dylib.py index aa10750..ea3dd38 100644 --- a/Lib/ctypes/macholib/dylib.py +++ b/Lib/ctypes/macholib/dylib.py @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### """ Generic dylib path manipulation """ diff --git a/Lib/ctypes/macholib/framework.py b/Lib/ctypes/macholib/framework.py index ad6ed55..dd7fb2f 100644 --- a/Lib/ctypes/macholib/framework.py +++ b/Lib/ctypes/macholib/framework.py @@ -1,3 +1,6 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### """ Generic framework path manipulation """ diff --git a/Lib/ctypes/test/test_anon.py b/Lib/ctypes/test/test_anon.py new file mode 100644 index 0000000..99e02cb --- /dev/null +++ b/Lib/ctypes/test/test_anon.py @@ -0,0 +1,60 @@ +import unittest +from ctypes import * + +class AnonTest(unittest.TestCase): + + def test_anon(self): + class ANON(Union): + _fields_ = [("a", c_int), + ("b", c_int)] + + class Y(Structure): + _fields_ = [("x", c_int), + ("_", ANON), + ("y", c_int)] + _anonymous_ = ["_"] + + self.failUnlessEqual(Y.a.offset, sizeof(c_int)) + self.failUnlessEqual(Y.b.offset, sizeof(c_int)) + + self.failUnlessEqual(ANON.a.offset, 0) + self.failUnlessEqual(ANON.b.offset, 0) + + def test_anon_nonseq(self): + # TypeError: _anonymous_ must be a sequence + self.failUnlessRaises(TypeError, + lambda: type(Structure)("Name", + (Structure,), + {"_fields_": [], "_anonymous_": 42})) + + def test_anon_nonmember(self): + # AttributeError: type object 'Name' has no attribute 'x' + self.failUnlessRaises(AttributeError, + lambda: type(Structure)("Name", + (Structure,), + {"_fields_": [], + "_anonymous_": ["x"]})) + + def test_nested(self): + class ANON_S(Structure): + _fields_ = [("a", c_int)] + + class ANON_U(Union): + _fields_ = [("_", ANON_S), + ("b", c_int)] + _anonymous_ = ["_"] + + class Y(Structure): + _fields_ = [("x", c_int), + ("_", ANON_U), + ("y", c_int)] + _anonymous_ = ["_"] + + self.failUnlessEqual(Y.x.offset, 0) + self.failUnlessEqual(Y.a.offset, sizeof(c_int)) + self.failUnlessEqual(Y.b.offset, sizeof(c_int)) + self.failUnlessEqual(Y._.offset, sizeof(c_int)) + self.failUnlessEqual(Y.y.offset, sizeof(c_int) * 2) + +if __name__ == "__main__": + unittest.main() diff --git a/Lib/ctypes/test/test_cast.py b/Lib/ctypes/test/test_cast.py index 821ce3f..09e928f 100644 --- a/Lib/ctypes/test/test_cast.py +++ b/Lib/ctypes/test/test_cast.py @@ -30,17 +30,32 @@ class Test(unittest.TestCase): ptr = cast(address, POINTER(c_int)) self.failUnlessEqual([ptr[i] for i in range(3)], [42, 17, 2]) - - def test_ptr2array(self): - array = (c_int * 3)(42, 17, 2) - - from sys import getrefcount - - before = getrefcount(array) - ptr = cast(array, POINTER(c_int)) - self.failUnlessEqual(getrefcount(array), before + 1) - del ptr - self.failUnlessEqual(getrefcount(array), before) + def test_p2a_objects(self): + array = (c_char_p * 5)() + self.failUnlessEqual(array._objects, None) + array[0] = "foo bar" + self.failUnlessEqual(array._objects, {'0': "foo bar"}) + + p = cast(array, POINTER(c_char_p)) + # array and p share a common _objects attribute + self.failUnless(p._objects is array._objects) + self.failUnlessEqual(array._objects, {'0': "foo bar", id(array): array}) + p[0] = "spam spam" + self.failUnlessEqual(p._objects, {'0': "spam spam", id(array): array}) + self.failUnless(array._objects is p._objects) + p[1] = "foo bar" + self.failUnlessEqual(p._objects, {'1': 'foo bar', '0': "spam spam", id(array): array}) + self.failUnless(array._objects is p._objects) + + def test_other(self): + p = cast((c_int * 4)(1, 2, 3, 4), POINTER(c_int)) + self.failUnlessEqual(p[:4], [1,2, 3, 4]) + c_int() + self.failUnlessEqual(p[:4], [1, 2, 3, 4]) + p[2] = 96 + self.failUnlessEqual(p[:4], [1, 2, 96, 4]) + c_int() + self.failUnlessEqual(p[:4], [1, 2, 96, 4]) if __name__ == "__main__": unittest.main() diff --git a/Lib/ctypes/test/test_keeprefs.py b/Lib/ctypes/test/test_keeprefs.py index 7318f29..80b6ca2 100644 --- a/Lib/ctypes/test/test_keeprefs.py +++ b/Lib/ctypes/test/test_keeprefs.py @@ -61,6 +61,8 @@ class StructureTestCase(unittest.TestCase): r.ul.x = 22 r.ul.y = 44 self.assertEquals(r._objects, {'0': {}}) + r.lr = POINT() + self.assertEquals(r._objects, {'0': {}, '1': {}}) class ArrayTestCase(unittest.TestCase): def test_cint_array(self): @@ -86,9 +88,10 @@ class ArrayTestCase(unittest.TestCase): self.assertEquals(x._objects, {'1': {}}) class PointerTestCase(unittest.TestCase): - def X_test_p_cint(self): - x = pointer(c_int(42)) - print x._objects + def test_p_cint(self): + i = c_int(42) + x = pointer(i) + self.failUnlessEqual(x._objects, {'1': i}) class DeletePointerTestCase(unittest.TestCase): def X_test(self): diff --git a/Lib/ctypes/test/test_loading.py b/Lib/ctypes/test/test_loading.py index 45585ae..28c83fd4 100644 --- a/Lib/ctypes/test/test_loading.py +++ b/Lib/ctypes/test/test_loading.py @@ -9,18 +9,10 @@ if os.name == "nt": libc_name = "msvcrt" elif os.name == "ce": libc_name = "coredll" -elif sys.platform == "darwin": - libc_name = "libc.dylib" elif sys.platform == "cygwin": libc_name = "cygwin1.dll" else: - for line in os.popen("ldd %s" % sys.executable): - if "libc.so" in line: - if sys.platform == "openbsd3": - libc_name = line.split()[4] - else: - libc_name = line.split()[2] - break + libc_name = find_library("c") if is_resource_enabled("printing"): print "libc_name is", libc_name diff --git a/Lib/ctypes/test/test_objects.py b/Lib/ctypes/test/test_objects.py new file mode 100644 index 0000000..4d921d2 --- /dev/null +++ b/Lib/ctypes/test/test_objects.py @@ -0,0 +1,70 @@ +r''' +This tests the '_objects' attribute of ctypes instances. '_objects' +holds references to objects that must be kept alive as long as the +ctypes instance, to make sure that the memory buffer is valid. + +WARNING: The '_objects' attribute is exposed ONLY for debugging ctypes itself, +it MUST NEVER BE MODIFIED! + +'_objects' is initialized to a dictionary on first use, before that it +is None. + +Here is an array of string pointers: + +>>> from ctypes import * +>>> array = (c_char_p * 5)() +>>> print array._objects +None +>>> + +The memory block stores pointers to strings, and the strings itself +assigned from Python must be kept. + +>>> array[4] = 'foo bar' +>>> array._objects +{'4': 'foo bar'} +>>> array[4] +'foo bar' +>>> + +It gets more complicated when the ctypes instance itself is contained +in a 'base' object. + +>>> class X(Structure): +... _fields_ = [("x", c_int), ("y", c_int), ("array", c_char_p * 5)] +... +>>> x = X() +>>> print x._objects +None +>>> + +The'array' attribute of the 'x' object shares part of the memory buffer +of 'x' ('_b_base_' is either None, or the root object owning the memory block): + +>>> print x.array._b_base_ # doctest: +ELLIPSIS + +>>> + +>>> x.array[0] = 'spam spam spam' +>>> x._objects +{'0:2': 'spam spam spam'} +>>> x.array._b_base_._objects +{'0:2': 'spam spam spam'} +>>> + +''' + +import unittest, doctest, sys + +import ctypes.test.test_objects + +class TestCase(unittest.TestCase): + if sys.hexversion > 0x02040000: + # Python 2.3 has no ELLIPSIS flag, so we don't test with this + # version: + def test(self): + doctest.testmod(ctypes.test.test_objects) + +if __name__ == '__main__': + if sys.hexversion > 0x02040000: + doctest.testmod(ctypes.test.test_objects) diff --git a/Lib/ctypes/test/test_parameters.py b/Lib/ctypes/test/test_parameters.py index 9537400..1b7f0dc 100644 --- a/Lib/ctypes/test/test_parameters.py +++ b/Lib/ctypes/test/test_parameters.py @@ -147,6 +147,41 @@ class SimpleTypesTestCase(unittest.TestCase): ## def test_performance(self): ## check_perf() + def test_noctypes_argtype(self): + import _ctypes_test + from ctypes import CDLL, c_void_p, ArgumentError + + func = CDLL(_ctypes_test.__file__)._testfunc_p_p + func.restype = c_void_p + # TypeError: has no from_param method + self.assertRaises(TypeError, setattr, func, "argtypes", (object,)) + + class Adapter(object): + def from_param(cls, obj): + return None + + func.argtypes = (Adapter(),) + self.failUnlessEqual(func(None), None) + self.failUnlessEqual(func(object()), None) + + class Adapter(object): + def from_param(cls, obj): + return obj + + func.argtypes = (Adapter(),) + # don't know how to convert parameter 1 + self.assertRaises(ArgumentError, func, object()) + self.failUnlessEqual(func(c_void_p(42)), 42) + + class Adapter(object): + def from_param(cls, obj): + raise ValueError(obj) + + func.argtypes = (Adapter(),) + # ArgumentError: argument 1: ValueError: 99 + self.assertRaises(ArgumentError, func, 99) + + ################################################################ if __name__ == '__main__': diff --git a/Lib/ctypes/test/test_pointers.py b/Lib/ctypes/test/test_pointers.py index a7a2802..586655a 100644 --- a/Lib/ctypes/test/test_pointers.py +++ b/Lib/ctypes/test/test_pointers.py @@ -157,6 +157,23 @@ class PointersTestCase(unittest.TestCase): q = pointer(y) pp[0] = q # <== self.failUnlessEqual(p[0], 6) + def test_c_void_p(self): + # http://sourceforge.net/tracker/?func=detail&aid=1518190&group_id=5470&atid=105470 + if sizeof(c_void_p) == 4: + self.failUnlessEqual(c_void_p(0xFFFFFFFFL).value, + c_void_p(-1).value) + self.failUnlessEqual(c_void_p(0xFFFFFFFFFFFFFFFFL).value, + c_void_p(-1).value) + elif sizeof(c_void_p) == 8: + self.failUnlessEqual(c_void_p(0xFFFFFFFFL).value, + 0xFFFFFFFFL) + self.failUnlessEqual(c_void_p(0xFFFFFFFFFFFFFFFFL).value, + c_void_p(-1).value) + self.failUnlessEqual(c_void_p(0xFFFFFFFFFFFFFFFFFFFFFFFFL).value, + c_void_p(-1).value) + + self.assertRaises(TypeError, c_void_p, 3.14) # make sure floats are NOT accepted + self.assertRaises(TypeError, c_void_p, object()) # nor other objects if __name__ == '__main__': unittest.main() diff --git a/Lib/ctypes/test/test_slicing.py b/Lib/ctypes/test/test_slicing.py index 08c811e..511c3d3 100644 --- a/Lib/ctypes/test/test_slicing.py +++ b/Lib/ctypes/test/test_slicing.py @@ -35,7 +35,7 @@ class SlicesTestCase(unittest.TestCase): self.assertRaises(ValueError, setslice, a, 0, 5, range(32)) def test_char_ptr(self): - s = "abcdefghijklmnopqrstuvwxyz\0" + s = "abcdefghijklmnopqrstuvwxyz" dll = CDLL(_ctypes_test.__file__) dll.my_strdup.restype = POINTER(c_char) @@ -50,9 +50,31 @@ class SlicesTestCase(unittest.TestCase): dll.my_strdup.restype = POINTER(c_byte) res = dll.my_strdup(s) - self.failUnlessEqual(res[:len(s)-1], range(ord("a"), ord("z")+1)) + self.failUnlessEqual(res[:len(s)], range(ord("a"), ord("z")+1)) dll.my_free(res) + def test_char_ptr_with_free(self): + dll = CDLL(_ctypes_test.__file__) + s = "abcdefghijklmnopqrstuvwxyz" + + class allocated_c_char_p(c_char_p): + pass + + dll.my_free.restype = None + def errcheck(result, func, args): + retval = result.value + dll.my_free(result) + return retval + + dll.my_strdup.restype = allocated_c_char_p + dll.my_strdup.errcheck = errcheck + try: + res = dll.my_strdup(s) + self.failUnlessEqual(res, s) + finally: + del dll.my_strdup.errcheck + + def test_char_array(self): s = "abcdefghijklmnopqrstuvwxyz\0" diff --git a/Lib/ctypes/test/test_structures.py b/Lib/ctypes/test/test_structures.py index 49f064b..8a4531d 100644 --- a/Lib/ctypes/test/test_structures.py +++ b/Lib/ctypes/test/test_structures.py @@ -138,8 +138,8 @@ class StructureTestCase(unittest.TestCase): self.failUnlessEqual(X.y.size, sizeof(c_char)) # readonly - self.assertRaises(AttributeError, setattr, X.x, "offset", 92) - self.assertRaises(AttributeError, setattr, X.x, "size", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "offset", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "size", 92) class X(Union): _fields_ = [("x", c_int), @@ -152,8 +152,8 @@ class StructureTestCase(unittest.TestCase): self.failUnlessEqual(X.y.size, sizeof(c_char)) # readonly - self.assertRaises(AttributeError, setattr, X.x, "offset", 92) - self.assertRaises(AttributeError, setattr, X.x, "size", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "offset", 92) + self.assertRaises((TypeError, AttributeError), setattr, X.x, "size", 92) # XXX Should we check nested data types also? # offset is always relative to the class... @@ -298,7 +298,7 @@ class StructureTestCase(unittest.TestCase): "expected string or Unicode object, int found") else: self.failUnlessEqual(msg, - "(Phone) TypeError: " + "(Phone) exceptions.TypeError: " "expected string or Unicode object, int found") cls, msg = self.get_except(Person, "Someone", ("a", "b", "c")) @@ -307,7 +307,7 @@ class StructureTestCase(unittest.TestCase): self.failUnlessEqual(msg, "(Phone) : too many initializers") else: - self.failUnlessEqual(msg, "(Phone) ValueError: too many initializers") + self.failUnlessEqual(msg, "(Phone) exceptions.ValueError: too many initializers") def get_except(self, func, *args): @@ -371,5 +371,15 @@ class PointerMemberTestCase(unittest.TestCase): items = [s.array[i] for i in range(3)] self.failUnlessEqual(items, [1, 2, 3]) + def test_none_to_pointer_fields(self): + class S(Structure): + _fields_ = [("x", c_int), + ("p", POINTER(c_int))] + + s = S() + s.x = 12345678 + s.p = None + self.failUnlessEqual(s.x, 12345678) + if __name__ == '__main__': unittest.main() diff --git a/Lib/ctypes/test/test_varsize_struct.py b/Lib/ctypes/test/test_varsize_struct.py new file mode 100644 index 0000000..06d2323 --- /dev/null +++ b/Lib/ctypes/test/test_varsize_struct.py @@ -0,0 +1,50 @@ +from ctypes import * +import unittest + +class VarSizeTest(unittest.TestCase): + def test_resize(self): + class X(Structure): + _fields_ = [("item", c_int), + ("array", c_int * 1)] + + self.failUnlessEqual(sizeof(X), sizeof(c_int) * 2) + x = X() + x.item = 42 + x.array[0] = 100 + self.failUnlessEqual(sizeof(x), sizeof(c_int) * 2) + + # make room for one additional item + new_size = sizeof(X) + sizeof(c_int) * 1 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + # make room for 10 additional items + new_size = sizeof(X) + sizeof(c_int) * 9 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + # make room for one additional item + new_size = sizeof(X) + sizeof(c_int) * 1 + resize(x, new_size) + self.failUnlessEqual(sizeof(x), new_size) + self.failUnlessEqual((x.item, x.array[0]), (42, 100)) + + def test_array_invalid_length(self): + # cannot create arrays with non-positive size + self.failUnlessRaises(ValueError, lambda: c_int * -1) + self.failUnlessRaises(ValueError, lambda: c_int * -3) + + def test_zerosized_array(self): + array = (c_int * 0)() + # accessing elements of zero-sized arrays raise IndexError + self.failUnlessRaises(IndexError, array.__setitem__, 0, None) + self.failUnlessRaises(IndexError, array.__getitem__, 0) + self.failUnlessRaises(IndexError, array.__setitem__, 1, None) + self.failUnlessRaises(IndexError, array.__getitem__, 1) + self.failUnlessRaises(IndexError, array.__setitem__, -1, None) + self.failUnlessRaises(IndexError, array.__getitem__, -1) + +if __name__ == "__main__": + unittest.main() diff --git a/Lib/ctypes/test/test_win32.py b/Lib/ctypes/test/test_win32.py index 8247d37..db530d3 100644 --- a/Lib/ctypes/test/test_win32.py +++ b/Lib/ctypes/test/test_win32.py @@ -1,6 +1,7 @@ # Windows specific tests from ctypes import * +from ctypes.test import is_resource_enabled import unittest, sys import _ctypes_test @@ -30,15 +31,10 @@ if sys.platform == "win32": # or wrong calling convention self.assertRaises(ValueError, IsWindow, None) - def test_SEH(self): - # Call functions with invalid arguments, and make sure that access violations - # are trapped and raise an exception. - # - # Normally, in a debug build of the _ctypes extension - # module, exceptions are not trapped, so we can only run - # this test in a release build. - import sys - if not hasattr(sys, "getobjects"): + if is_resource_enabled("SEH"): + def test_SEH(self): + # Call functions with invalid arguments, and make sure that access violations + # are trapped and raise an exception. self.assertRaises(WindowsError, windll.kernel32.GetModuleHandleA, 32) class Structures(unittest.TestCase): diff --git a/Lib/ctypes/util.py b/Lib/ctypes/util.py index d756c1c..2ee2968 100644 --- a/Lib/ctypes/util.py +++ b/Lib/ctypes/util.py @@ -1,5 +1,7 @@ +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### import sys, os -import ctypes # find_library(name) returns the pathname of a library, or None. if os.name == "nt": @@ -41,14 +43,17 @@ if os.name == "posix" and sys.platform == "darwin": elif os.name == "posix": # Andreas Degert's find functions, using gcc, /sbin/ldconfig, objdump - import re, tempfile + import re, tempfile, errno def _findLib_gcc(name): expr = '[^\(\)\s]*lib%s\.[^\(\)\s]*' % name + fdout, ccout = tempfile.mkstemp() + os.close(fdout) cmd = 'if type gcc &>/dev/null; then CC=gcc; else CC=cc; fi;' \ - '$CC -Wl,-t -o /dev/null 2>&1 -l' + name + '$CC -Wl,-t -o ' + ccout + ' 2>&1 -l' + name try: fdout, outfile = tempfile.mkstemp() + os.close(fdout) fd = os.popen(cmd) trace = fd.read() err = fd.close() @@ -58,6 +63,11 @@ elif os.name == "posix": except OSError, e: if e.errno != errno.ENOENT: raise + try: + os.unlink(ccout) + except OSError, e: + if e.errno != errno.ENOENT: + raise res = re.search(expr, trace) if not res: return None diff --git a/Lib/ctypes/wintypes.py b/Lib/ctypes/wintypes.py index 92b79d2..9768233 100644 --- a/Lib/ctypes/wintypes.py +++ b/Lib/ctypes/wintypes.py @@ -1,60 +1,117 @@ -# XXX This module needs cleanup. +###################################################################### +# This file should be kept compatible with Python 2.3, see PEP 291. # +###################################################################### +# The most useful windows datatypes from ctypes import * -DWORD = c_ulong -WORD = c_ushort BYTE = c_byte +WORD = c_ushort +DWORD = c_ulong + +WCHAR = c_wchar +UINT = c_uint + +DOUBLE = c_double + +BOOLEAN = BYTE +BOOL = c_long + +from ctypes import _SimpleCData +class VARIANT_BOOL(_SimpleCData): + _type_ = "v" + def __repr__(self): + return "%s(%r)" % (self.__class__.__name__, self.value) ULONG = c_ulong LONG = c_long -LARGE_INTEGER = c_longlong -ULARGE_INTEGER = c_ulonglong +# in the windows header files, these are structures. +_LARGE_INTEGER = LARGE_INTEGER = c_longlong +_ULARGE_INTEGER = ULARGE_INTEGER = c_ulonglong +LPCOLESTR = LPOLESTR = OLESTR = c_wchar_p +LPCWSTR = LPWSTR = c_wchar_p +LPCSTR = LPSTR = c_char_p +WPARAM = c_uint +LPARAM = c_long + +ATOM = WORD +LANGID = WORD + +COLORREF = DWORD +LGRPID = DWORD +LCTYPE = DWORD + +LCID = DWORD + +################################################################ +# HANDLE types HANDLE = c_ulong # in the header files: void * -HWND = HANDLE +HACCEL = HANDLE +HBITMAP = HANDLE +HBRUSH = HANDLE +HCOLORSPACE = HANDLE HDC = HANDLE -HMODULE = HANDLE +HDESK = HANDLE +HDWP = HANDLE +HENHMETAFILE = HANDLE +HFONT = HANDLE +HGDIOBJ = HANDLE +HGLOBAL = HANDLE +HHOOK = HANDLE +HICON = HANDLE HINSTANCE = HANDLE -HRGN = HANDLE -HTASK = HANDLE HKEY = HANDLE -HPEN = HANDLE -HGDIOBJ = HANDLE +HKL = HANDLE +HLOCAL = HANDLE HMENU = HANDLE +HMETAFILE = HANDLE +HMODULE = HANDLE +HMONITOR = HANDLE +HPALETTE = HANDLE +HPEN = HANDLE +HRGN = HANDLE +HRSRC = HANDLE +HSTR = HANDLE +HTASK = HANDLE +HWINSTA = HANDLE +HWND = HANDLE +SC_HANDLE = HANDLE +SERVICE_STATUS_HANDLE = HANDLE -LCID = DWORD - -WPARAM = c_uint -LPARAM = c_long - -BOOL = c_long -VARIANT_BOOL = c_short - -LPCOLESTR = LPOLESTR = OLESTR = c_wchar_p -LPCWSTR = LPWSTR = c_wchar_p - -LPCSTR = LPSTR = c_char_p +################################################################ +# Some important structure definitions class RECT(Structure): _fields_ = [("left", c_long), ("top", c_long), ("right", c_long), ("bottom", c_long)] -RECTL = RECT +tagRECT = _RECTL = RECTL = RECT + +class _SMALL_RECT(Structure): + _fields_ = [('Left', c_short), + ('Top', c_short), + ('Right', c_short), + ('Bottom', c_short)] +SMALL_RECT = _SMALL_RECT + +class _COORD(Structure): + _fields_ = [('X', c_short), + ('Y', c_short)] class POINT(Structure): _fields_ = [("x", c_long), ("y", c_long)] -POINTL = POINT +tagPOINT = _POINTL = POINTL = POINT class SIZE(Structure): _fields_ = [("cx", c_long), ("cy", c_long)] -SIZEL = SIZE +tagSIZE = SIZEL = SIZE def RGB(red, green, blue): return red + (green << 8) + (blue << 16) @@ -62,6 +119,7 @@ def RGB(red, green, blue): class FILETIME(Structure): _fields_ = [("dwLowDateTime", DWORD), ("dwHighDateTime", DWORD)] +_FILETIME = FILETIME class MSG(Structure): _fields_ = [("hWnd", HWND), @@ -70,6 +128,7 @@ class MSG(Structure): ("lParam", LPARAM), ("time", DWORD), ("pt", POINT)] +tagMSG = MSG MAX_PATH = 260 class WIN32_FIND_DATAA(Structure): @@ -95,3 +154,19 @@ class WIN32_FIND_DATAW(Structure): ("dwReserved1", DWORD), ("cFileName", c_wchar * MAX_PATH), ("cAlternameFileName", c_wchar * 14)] + +__all__ = ['ATOM', 'BOOL', 'BOOLEAN', 'BYTE', 'COLORREF', 'DOUBLE', + 'DWORD', 'FILETIME', 'HACCEL', 'HANDLE', 'HBITMAP', 'HBRUSH', + 'HCOLORSPACE', 'HDC', 'HDESK', 'HDWP', 'HENHMETAFILE', 'HFONT', + 'HGDIOBJ', 'HGLOBAL', 'HHOOK', 'HICON', 'HINSTANCE', 'HKEY', + 'HKL', 'HLOCAL', 'HMENU', 'HMETAFILE', 'HMODULE', 'HMONITOR', + 'HPALETTE', 'HPEN', 'HRGN', 'HRSRC', 'HSTR', 'HTASK', 'HWINSTA', + 'HWND', 'LANGID', 'LARGE_INTEGER', 'LCID', 'LCTYPE', 'LGRPID', + 'LONG', 'LPARAM', 'LPCOLESTR', 'LPCSTR', 'LPCWSTR', 'LPOLESTR', + 'LPSTR', 'LPWSTR', 'MAX_PATH', 'MSG', 'OLESTR', 'POINT', + 'POINTL', 'RECT', 'RECTL', 'RGB', 'SC_HANDLE', + 'SERVICE_STATUS_HANDLE', 'SIZE', 'SIZEL', 'SMALL_RECT', 'UINT', + 'ULARGE_INTEGER', 'ULONG', 'VARIANT_BOOL', 'WCHAR', + 'WIN32_FIND_DATAA', 'WIN32_FIND_DATAW', 'WORD', 'WPARAM', '_COORD', + '_FILETIME', '_LARGE_INTEGER', '_POINTL', '_RECTL', '_SMALL_RECT', + '_ULARGE_INTEGER', 'tagMSG', 'tagPOINT', 'tagRECT', 'tagSIZE'] diff --git a/Lib/difflib.py b/Lib/difflib.py index 55f69ba..3e28b18 100644 --- a/Lib/difflib.py +++ b/Lib/difflib.py @@ -86,8 +86,7 @@ class SequenceMatcher: >>> for block in s.get_matching_blocks(): ... print "a[%d] and b[%d] match for %d elements" % block a[0] and b[0] match for 8 elements - a[8] and b[17] match for 6 elements - a[14] and b[23] match for 15 elements + a[8] and b[17] match for 21 elements a[29] and b[38] match for 0 elements Note that the last tuple returned by .get_matching_blocks() is always a @@ -101,8 +100,7 @@ class SequenceMatcher: ... print "%6s a[%d:%d] b[%d:%d]" % opcode equal a[0:8] b[0:8] insert a[8:8] b[8:17] - equal a[8:14] b[17:23] - equal a[14:29] b[23:38] + equal a[8:29] b[17:38] See the Differ class for a fancy human-friendly file differencer, which uses SequenceMatcher both to compare sequences of lines, and to compare @@ -461,7 +459,11 @@ class SequenceMatcher: Each triple is of the form (i, j, n), and means that a[i:i+n] == b[j:j+n]. The triples are monotonically increasing in - i and in j. + i and in j. New in Python 2.5, it's also guaranteed that if + (i, j, n) and (i', j', n') are adjacent triples in the list, and + the second is not the last triple in the list, then i+n != i' or + j+n != j'. IOW, adjacent triples never describe adjacent equal + blocks. The last triple is a dummy, (len(a), len(b), 0), and is the only triple with n==0. @@ -475,28 +477,52 @@ class SequenceMatcher: return self.matching_blocks la, lb = len(self.a), len(self.b) - indexed_blocks = [] + # This is most naturally expressed as a recursive algorithm, but + # at least one user bumped into extreme use cases that exceeded + # the recursion limit on their box. So, now we maintain a list + # ('queue`) of blocks we still need to look at, and append partial + # results to `matching_blocks` in a loop; the matches are sorted + # at the end. queue = [(0, la, 0, lb)] + matching_blocks = [] while queue: - # builds list of matching blocks covering a[alo:ahi] and - # b[blo:bhi], appending them in increasing order to answer alo, ahi, blo, bhi = queue.pop() - + i, j, k = x = self.find_longest_match(alo, ahi, blo, bhi) # a[alo:i] vs b[blo:j] unknown # a[i:i+k] same as b[j:j+k] # a[i+k:ahi] vs b[j+k:bhi] unknown - i, j, k = x = self.find_longest_match(alo, ahi, blo, bhi) - - if k: + if k: # if k is 0, there was no matching block + matching_blocks.append(x) if alo < i and blo < j: queue.append((alo, i, blo, j)) - indexed_blocks.append((i, x)) if i+k < ahi and j+k < bhi: queue.append((i+k, ahi, j+k, bhi)) - indexed_blocks.sort() - - self.matching_blocks = [elem[1] for elem in indexed_blocks] - self.matching_blocks.append( (la, lb, 0) ) + matching_blocks.sort() + + # It's possible that we have adjacent equal blocks in the + # matching_blocks list now. Starting with 2.5, this code was added + # to collapse them. + i1 = j1 = k1 = 0 + non_adjacent = [] + for i2, j2, k2 in matching_blocks: + # Is this block adjacent to i1, j1, k1? + if i1 + k1 == i2 and j1 + k1 == j2: + # Yes, so collapse them -- this just increases the length of + # the first block by the length of the second, and the first + # block so lengthened remains the block to compare against. + k1 += k2 + else: + # Not adjacent. Remember the first block (k1==0 means it's + # the dummy we started with), and make the second block the + # new block to compare against. + if k1: + non_adjacent.append((i1, j1, k1)) + i1, j1, k1 = i2, j2, k2 + if k1: + non_adjacent.append((i1, j1, k1)) + + non_adjacent.append( (la, lb, 0) ) + self.matching_blocks = non_adjacent return self.matching_blocks def get_opcodes(self): @@ -1422,8 +1448,7 @@ def _mdiff(fromlines, tolines, context=None, linejunk=None, num_blanks_pending -= 1 yield _make_line(lines,'-',0), None, True continue - elif s.startswith('--?+') or s.startswith('--+') or \ - s.startswith('- '): + elif s.startswith(('--?+', '--+', '- ')): # in delete block and see a intraline change or unchanged line # coming: yield the delete line and then blanks from_line,to_line = _make_line(lines,'-',0), None @@ -1447,7 +1472,7 @@ def _mdiff(fromlines, tolines, context=None, linejunk=None, num_blanks_pending += 1 yield None, _make_line(lines,'+',1), True continue - elif s.startswith('+ ') or s.startswith('+-'): + elif s.startswith(('+ ', '+-')): # will be leaving an add block: yield blanks then add line from_line, to_line = None, _make_line(lines,'+',1) num_blanks_to_yield,num_blanks_pending = num_blanks_pending+1,0 diff --git a/Lib/distutils/__init__.py b/Lib/distutils/__init__.py index a1dbb4b..9c60e54 100644 --- a/Lib/distutils/__init__.py +++ b/Lib/distutils/__init__.py @@ -12,4 +12,6 @@ used from a setup script as __revision__ = "$Id$" -__version__ = "2.4.0" +import sys +__version__ = "%d.%d.%d" % sys.version_info[:3] +del sys diff --git a/Lib/distutils/command/bdist_rpm.py b/Lib/distutils/command/bdist_rpm.py index 738e3f7..5b09965 100644 --- a/Lib/distutils/command/bdist_rpm.py +++ b/Lib/distutils/command/bdist_rpm.py @@ -467,7 +467,8 @@ class bdist_rpm (Command): # rpm scripts # figure out default build script - def_build = "%s setup.py build" % self.python + def_setup_call = "%s %s" % (self.python,os.path.basename(sys.argv[0])) + def_build = "%s build" % def_setup_call if self.use_rpm_opt_flags: def_build = 'env CFLAGS="$RPM_OPT_FLAGS" ' + def_build @@ -481,9 +482,9 @@ class bdist_rpm (Command): ('prep', 'prep_script', "%setup"), ('build', 'build_script', def_build), ('install', 'install_script', - ("%s setup.py install " + ("%s install " "--root=$RPM_BUILD_ROOT " - "--record=INSTALLED_FILES") % self.python), + "--record=INSTALLED_FILES") % def_setup_call), ('clean', 'clean_script', "rm -rf $RPM_BUILD_ROOT"), ('verifyscript', 'verify_script', None), ('pre', 'pre_install', None), diff --git a/Lib/distutils/command/upload.py b/Lib/distutils/command/upload.py index 4a9ed39..67ba080 100644 --- a/Lib/distutils/command/upload.py +++ b/Lib/distutils/command/upload.py @@ -185,7 +185,7 @@ class upload(Command): http.endheaders() http.send(body) except socket.error, e: - self.announce(e.msg, log.ERROR) + self.announce(str(e), log.ERROR) return r = http.getresponse() diff --git a/Lib/distutils/msvccompiler.py b/Lib/distutils/msvccompiler.py index d24d0ac..0d72837 100644 --- a/Lib/distutils/msvccompiler.py +++ b/Lib/distutils/msvccompiler.py @@ -131,8 +131,10 @@ class MacroExpander: self.set_macro("FrameworkSDKDir", net, "sdkinstallroot") except KeyError, exc: # raise DistutilsPlatformError, \ - ("The .NET Framework SDK needs to be installed before " - "building extensions for Python.") + ("""Python was built with Visual Studio 2003; +extensions must be built with a compiler than can generate compatible binaries. +Visual Studio 2003 was not found on this system. If you have Cygwin installed, +you can try compiling with MingW32, by passing "-c mingw32" to setup.py.""") p = r"Software\Microsoft\NET Framework Setup\Product" for base in HKEYS: @@ -237,7 +239,7 @@ class MSVCCompiler (CCompiler) : def initialize(self): self.__paths = [] - if os.environ.has_key("MSSdk") and self.find_exe("cl.exe"): + if os.environ.has_key("DISTUTILS_USE_SDK") and os.environ.has_key("MSSdk") and self.find_exe("cl.exe"): # Assume that the SDK set up everything alright; don't try to be # smarter self.cc = "cl.exe" diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py index e1397a1..76fe256 100644 --- a/Lib/distutils/sysconfig.py +++ b/Lib/distutils/sysconfig.py @@ -512,7 +512,7 @@ def get_config_vars(*args): for key in ('LDFLAGS', 'BASECFLAGS'): flags = _config_vars[key] flags = re.sub('-arch\s+\w+\s', ' ', flags) - flags = re.sub('-isysroot [^ \t]* ', ' ', flags) + flags = re.sub('-isysroot [^ \t]*', ' ', flags) _config_vars[key] = flags if args: diff --git a/Lib/distutils/unixccompiler.py b/Lib/distutils/unixccompiler.py index 324819d..6cd14f7 100644 --- a/Lib/distutils/unixccompiler.py +++ b/Lib/distutils/unixccompiler.py @@ -78,7 +78,7 @@ def _darwin_compiler_fixup(compiler_so, cc_args): try: index = compiler_so.index('-isysroot') # Strip this argument and the next one: - del compiler_so[index:index+1] + del compiler_so[index:index+2] except ValueError: pass diff --git a/Lib/doctest.py b/Lib/doctest.py index 47b3aae..fe734b3 100644 --- a/Lib/doctest.py +++ b/Lib/doctest.py @@ -95,7 +95,7 @@ __all__ = [ import __future__ -import sys, traceback, inspect, linecache, os, re, types +import sys, traceback, inspect, linecache, os, re import unittest, difflib, pdb, tempfile import warnings from StringIO import StringIO @@ -821,6 +821,11 @@ class DocTestFinder: # Recursively expore `obj`, extracting DocTests. tests = [] self._find(tests, obj, name, module, source_lines, globs, {}) + # Sort the tests by alpha order of names, for consistency in + # verbose-mode output. This was a feature of doctest in Pythons + # <= 2.3 that got lost by accident in 2.4. It was repaired in + # 2.4.4 and 2.5. + tests.sort() return tests def _from_module(self, module, object): diff --git a/Lib/dummy_thread.py b/Lib/dummy_thread.py index 21fd03f..a72c927 100644 --- a/Lib/dummy_thread.py +++ b/Lib/dummy_thread.py @@ -20,6 +20,7 @@ __all__ = ['error', 'start_new_thread', 'exit', 'get_ident', 'allocate_lock', 'interrupt_main', 'LockType'] import traceback as _traceback +import warnings class error(Exception): """Dummy implementation of thread.error.""" @@ -75,6 +76,12 @@ def allocate_lock(): """Dummy implementation of thread.allocate_lock().""" return LockType() +def stack_size(size=None): + """Dummy implementation of thread.stack_size().""" + if size is not None: + raise error("setting thread stack size not supported") + return 0 + class LockType(object): """Class implementing dummy implementation of thread.LockType. diff --git a/Lib/email/__init__.py b/Lib/email/__init__.py index f01260f..8d230fd 100644 --- a/Lib/email/__init__.py +++ b/Lib/email/__init__.py @@ -4,7 +4,7 @@ """A package for parsing, handling, and generating email messages.""" -__version__ = '4.0a2' +__version__ = '4.0.1' __all__ = [ # Old names diff --git a/Lib/email/message.py b/Lib/email/message.py index 50d90b4..79c5c4c 100644 --- a/Lib/email/message.py +++ b/Lib/email/message.py @@ -747,7 +747,18 @@ class Message: if isinstance(charset, tuple): # RFC 2231 encoded, so decode it, and it better end up as ascii. pcharset = charset[0] or 'us-ascii' - charset = unicode(charset[2], pcharset).encode('us-ascii') + try: + # LookupError will be raised if the charset isn't known to + # Python. UnicodeError will be raised if the encoded text + # contains a character not in the charset. + charset = unicode(charset[2], pcharset).encode('us-ascii') + except (LookupError, UnicodeError): + charset = charset[2] + # charset character must be in us-ascii range + try: + charset = unicode(charset, 'us-ascii').encode('us-ascii') + except UnicodeError: + return failobj # RFC 2046, $4.1.2 says charsets are not case sensitive return charset.lower() diff --git a/Lib/email/test/test_email.py b/Lib/email/test/test_email.py index a197a36..13801dc 100644 --- a/Lib/email/test/test_email.py +++ b/Lib/email/test/test_email.py @@ -3005,14 +3005,29 @@ Content-Type: text/html; NAME*0=file____C__DOCUMENTS_20AND_20SETTINGS_FABIEN_LOC ''' msg = email.message_from_string(m) - self.assertEqual(msg.get_param('NAME'), - (None, None, 'file____C__DOCUMENTS_20AND_20SETTINGS_FABIEN_LOCAL_20SETTINGS_TEMP_nsmail.htm')) + param = msg.get_param('NAME') + self.failIf(isinstance(param, tuple)) + self.assertEqual( + param, + 'file____C__DOCUMENTS_20AND_20SETTINGS_FABIEN_LOCAL_20SETTINGS_TEMP_nsmail.htm') def test_rfc2231_no_language_or_charset_in_filename(self): m = '''\ Content-Disposition: inline; -\tfilename*0="This%20is%20even%20more%20"; -\tfilename*1="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*0*="''This%20is%20even%20more%20"; +\tfilename*1*="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*2="is it not.pdf" + +''' + msg = email.message_from_string(m) + self.assertEqual(msg.get_filename(), + 'This is even more ***fun*** is it not.pdf') + + def test_rfc2231_no_language_or_charset_in_filename_encoded(self): + m = '''\ +Content-Disposition: inline; +\tfilename*0*="''This%20is%20even%20more%20"; +\tfilename*1*="%2A%2A%2Afun%2A%2A%2A%20"; \tfilename*2="is it not.pdf" ''' @@ -3020,11 +3035,37 @@ Content-Disposition: inline; self.assertEqual(msg.get_filename(), 'This is even more ***fun*** is it not.pdf') + def test_rfc2231_partly_encoded(self): + m = '''\ +Content-Disposition: inline; +\tfilename*0="''This%20is%20even%20more%20"; +\tfilename*1*="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*2="is it not.pdf" + +''' + msg = email.message_from_string(m) + self.assertEqual( + msg.get_filename(), + 'This%20is%20even%20more%20***fun*** is it not.pdf') + + def test_rfc2231_partly_nonencoded(self): + m = '''\ +Content-Disposition: inline; +\tfilename*0="This%20is%20even%20more%20"; +\tfilename*1="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*2="is it not.pdf" + +''' + msg = email.message_from_string(m) + self.assertEqual( + msg.get_filename(), + 'This%20is%20even%20more%20%2A%2A%2Afun%2A%2A%2A%20is it not.pdf') + def test_rfc2231_no_language_or_charset_in_boundary(self): m = '''\ Content-Type: multipart/alternative; -\tboundary*0="This%20is%20even%20more%20"; -\tboundary*1="%2A%2A%2Afun%2A%2A%2A%20"; +\tboundary*0*="''This%20is%20even%20more%20"; +\tboundary*1*="%2A%2A%2Afun%2A%2A%2A%20"; \tboundary*2="is it not.pdf" ''' @@ -3036,8 +3077,8 @@ Content-Type: multipart/alternative; # This is a nonsensical charset value, but tests the code anyway m = '''\ Content-Type: text/plain; -\tcharset*0="This%20is%20even%20more%20"; -\tcharset*1="%2A%2A%2Afun%2A%2A%2A%20"; +\tcharset*0*="This%20is%20even%20more%20"; +\tcharset*1*="%2A%2A%2Afun%2A%2A%2A%20"; \tcharset*2="is it not.pdf" ''' @@ -3045,15 +3086,145 @@ Content-Type: text/plain; self.assertEqual(msg.get_content_charset(), 'this is even more ***fun*** is it not.pdf') + def test_rfc2231_bad_encoding_in_filename(self): + m = '''\ +Content-Disposition: inline; +\tfilename*0*="bogus'xx'This%20is%20even%20more%20"; +\tfilename*1*="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*2="is it not.pdf" + +''' + msg = email.message_from_string(m) + self.assertEqual(msg.get_filename(), + 'This is even more ***fun*** is it not.pdf') + + def test_rfc2231_bad_encoding_in_charset(self): + m = """\ +Content-Type: text/plain; charset*=bogus''utf-8%E2%80%9D + +""" + msg = email.message_from_string(m) + # This should return None because non-ascii characters in the charset + # are not allowed. + self.assertEqual(msg.get_content_charset(), None) + + def test_rfc2231_bad_character_in_charset(self): + m = """\ +Content-Type: text/plain; charset*=ascii''utf-8%E2%80%9D + +""" + msg = email.message_from_string(m) + # This should return None because non-ascii characters in the charset + # are not allowed. + self.assertEqual(msg.get_content_charset(), None) + + def test_rfc2231_bad_character_in_filename(self): + m = '''\ +Content-Disposition: inline; +\tfilename*0*="ascii'xx'This%20is%20even%20more%20"; +\tfilename*1*="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*2*="is it not.pdf%E2" + +''' + msg = email.message_from_string(m) + self.assertEqual(msg.get_filename(), + u'This is even more ***fun*** is it not.pdf\ufffd') + def test_rfc2231_unknown_encoding(self): m = """\ Content-Transfer-Encoding: 8bit -Content-Disposition: inline; filename*0=X-UNKNOWN''myfile.txt +Content-Disposition: inline; filename*=X-UNKNOWN''myfile.txt """ msg = email.message_from_string(m) self.assertEqual(msg.get_filename(), 'myfile.txt') + def test_rfc2231_single_tick_in_filename_extended(self): + eq = self.assertEqual + m = """\ +Content-Type: application/x-foo; +\tname*0*=\"Frank's\"; name*1*=\" Document\" + +""" + msg = email.message_from_string(m) + charset, language, s = msg.get_param('name') + eq(charset, None) + eq(language, None) + eq(s, "Frank's Document") + + def test_rfc2231_single_tick_in_filename(self): + m = """\ +Content-Type: application/x-foo; name*0=\"Frank's\"; name*1=\" Document\" + +""" + msg = email.message_from_string(m) + param = msg.get_param('name') + self.failIf(isinstance(param, tuple)) + self.assertEqual(param, "Frank's Document") + + def test_rfc2231_tick_attack_extended(self): + eq = self.assertEqual + m = """\ +Content-Type: application/x-foo; +\tname*0*=\"us-ascii'en-us'Frank's\"; name*1*=\" Document\" + +""" + msg = email.message_from_string(m) + charset, language, s = msg.get_param('name') + eq(charset, 'us-ascii') + eq(language, 'en-us') + eq(s, "Frank's Document") + + def test_rfc2231_tick_attack(self): + m = """\ +Content-Type: application/x-foo; +\tname*0=\"us-ascii'en-us'Frank's\"; name*1=\" Document\" + +""" + msg = email.message_from_string(m) + param = msg.get_param('name') + self.failIf(isinstance(param, tuple)) + self.assertEqual(param, "us-ascii'en-us'Frank's Document") + + def test_rfc2231_no_extended_values(self): + eq = self.assertEqual + m = """\ +Content-Type: application/x-foo; name=\"Frank's Document\" + +""" + msg = email.message_from_string(m) + eq(msg.get_param('name'), "Frank's Document") + + def test_rfc2231_encoded_then_unencoded_segments(self): + eq = self.assertEqual + m = """\ +Content-Type: application/x-foo; +\tname*0*=\"us-ascii'en-us'My\"; +\tname*1=\" Document\"; +\tname*2*=\" For You\" + +""" + msg = email.message_from_string(m) + charset, language, s = msg.get_param('name') + eq(charset, 'us-ascii') + eq(language, 'en-us') + eq(s, 'My Document For You') + + def test_rfc2231_unencoded_then_encoded_segments(self): + eq = self.assertEqual + m = """\ +Content-Type: application/x-foo; +\tname*0=\"us-ascii'en-us'My\"; +\tname*1*=\" Document\"; +\tname*2*=\" For You\" + +""" + msg = email.message_from_string(m) + charset, language, s = msg.get_param('name') + eq(charset, 'us-ascii') + eq(language, 'en-us') + eq(s, 'My Document For You') + def _testclasses(): diff --git a/Lib/email/test/test_email_renamed.py b/Lib/email/test/test_email_renamed.py index 95d06cb..30f39b9 100644 --- a/Lib/email/test/test_email_renamed.py +++ b/Lib/email/test/test_email_renamed.py @@ -3011,14 +3011,29 @@ Content-Type: text/html; NAME*0=file____C__DOCUMENTS_20AND_20SETTINGS_FABIEN_LOC ''' msg = email.message_from_string(m) - self.assertEqual(msg.get_param('NAME'), - (None, None, 'file____C__DOCUMENTS_20AND_20SETTINGS_FABIEN_LOCAL_20SETTINGS_TEMP_nsmail.htm')) + param = msg.get_param('NAME') + self.failIf(isinstance(param, tuple)) + self.assertEqual( + param, + 'file____C__DOCUMENTS_20AND_20SETTINGS_FABIEN_LOCAL_20SETTINGS_TEMP_nsmail.htm') def test_rfc2231_no_language_or_charset_in_filename(self): m = '''\ Content-Disposition: inline; -\tfilename*0="This%20is%20even%20more%20"; -\tfilename*1="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*0*="''This%20is%20even%20more%20"; +\tfilename*1*="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*2="is it not.pdf" + +''' + msg = email.message_from_string(m) + self.assertEqual(msg.get_filename(), + 'This is even more ***fun*** is it not.pdf') + + def test_rfc2231_no_language_or_charset_in_filename_encoded(self): + m = '''\ +Content-Disposition: inline; +\tfilename*0*="''This%20is%20even%20more%20"; +\tfilename*1*="%2A%2A%2Afun%2A%2A%2A%20"; \tfilename*2="is it not.pdf" ''' @@ -3026,11 +3041,37 @@ Content-Disposition: inline; self.assertEqual(msg.get_filename(), 'This is even more ***fun*** is it not.pdf') + def test_rfc2231_partly_encoded(self): + m = '''\ +Content-Disposition: inline; +\tfilename*0="''This%20is%20even%20more%20"; +\tfilename*1*="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*2="is it not.pdf" + +''' + msg = email.message_from_string(m) + self.assertEqual( + msg.get_filename(), + 'This%20is%20even%20more%20***fun*** is it not.pdf') + + def test_rfc2231_partly_nonencoded(self): + m = '''\ +Content-Disposition: inline; +\tfilename*0="This%20is%20even%20more%20"; +\tfilename*1="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*2="is it not.pdf" + +''' + msg = email.message_from_string(m) + self.assertEqual( + msg.get_filename(), + 'This%20is%20even%20more%20%2A%2A%2Afun%2A%2A%2A%20is it not.pdf') + def test_rfc2231_no_language_or_charset_in_boundary(self): m = '''\ Content-Type: multipart/alternative; -\tboundary*0="This%20is%20even%20more%20"; -\tboundary*1="%2A%2A%2Afun%2A%2A%2A%20"; +\tboundary*0*="''This%20is%20even%20more%20"; +\tboundary*1*="%2A%2A%2Afun%2A%2A%2A%20"; \tboundary*2="is it not.pdf" ''' @@ -3042,8 +3083,8 @@ Content-Type: multipart/alternative; # This is a nonsensical charset value, but tests the code anyway m = '''\ Content-Type: text/plain; -\tcharset*0="This%20is%20even%20more%20"; -\tcharset*1="%2A%2A%2Afun%2A%2A%2A%20"; +\tcharset*0*="This%20is%20even%20more%20"; +\tcharset*1*="%2A%2A%2Afun%2A%2A%2A%20"; \tcharset*2="is it not.pdf" ''' @@ -3051,15 +3092,145 @@ Content-Type: text/plain; self.assertEqual(msg.get_content_charset(), 'this is even more ***fun*** is it not.pdf') + def test_rfc2231_bad_encoding_in_filename(self): + m = '''\ +Content-Disposition: inline; +\tfilename*0*="bogus'xx'This%20is%20even%20more%20"; +\tfilename*1*="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*2="is it not.pdf" + +''' + msg = email.message_from_string(m) + self.assertEqual(msg.get_filename(), + 'This is even more ***fun*** is it not.pdf') + + def test_rfc2231_bad_encoding_in_charset(self): + m = """\ +Content-Type: text/plain; charset*=bogus''utf-8%E2%80%9D + +""" + msg = email.message_from_string(m) + # This should return None because non-ascii characters in the charset + # are not allowed. + self.assertEqual(msg.get_content_charset(), None) + + def test_rfc2231_bad_character_in_charset(self): + m = """\ +Content-Type: text/plain; charset*=ascii''utf-8%E2%80%9D + +""" + msg = email.message_from_string(m) + # This should return None because non-ascii characters in the charset + # are not allowed. + self.assertEqual(msg.get_content_charset(), None) + + def test_rfc2231_bad_character_in_filename(self): + m = '''\ +Content-Disposition: inline; +\tfilename*0*="ascii'xx'This%20is%20even%20more%20"; +\tfilename*1*="%2A%2A%2Afun%2A%2A%2A%20"; +\tfilename*2*="is it not.pdf%E2" + +''' + msg = email.message_from_string(m) + self.assertEqual(msg.get_filename(), + u'This is even more ***fun*** is it not.pdf\ufffd') + def test_rfc2231_unknown_encoding(self): m = """\ Content-Transfer-Encoding: 8bit -Content-Disposition: inline; filename*0=X-UNKNOWN''myfile.txt +Content-Disposition: inline; filename*=X-UNKNOWN''myfile.txt """ msg = email.message_from_string(m) self.assertEqual(msg.get_filename(), 'myfile.txt') + def test_rfc2231_single_tick_in_filename_extended(self): + eq = self.assertEqual + m = """\ +Content-Type: application/x-foo; +\tname*0*=\"Frank's\"; name*1*=\" Document\" + +""" + msg = email.message_from_string(m) + charset, language, s = msg.get_param('name') + eq(charset, None) + eq(language, None) + eq(s, "Frank's Document") + + def test_rfc2231_single_tick_in_filename(self): + m = """\ +Content-Type: application/x-foo; name*0=\"Frank's\"; name*1=\" Document\" + +""" + msg = email.message_from_string(m) + param = msg.get_param('name') + self.failIf(isinstance(param, tuple)) + self.assertEqual(param, "Frank's Document") + + def test_rfc2231_tick_attack_extended(self): + eq = self.assertEqual + m = """\ +Content-Type: application/x-foo; +\tname*0*=\"us-ascii'en-us'Frank's\"; name*1*=\" Document\" + +""" + msg = email.message_from_string(m) + charset, language, s = msg.get_param('name') + eq(charset, 'us-ascii') + eq(language, 'en-us') + eq(s, "Frank's Document") + + def test_rfc2231_tick_attack(self): + m = """\ +Content-Type: application/x-foo; +\tname*0=\"us-ascii'en-us'Frank's\"; name*1=\" Document\" + +""" + msg = email.message_from_string(m) + param = msg.get_param('name') + self.failIf(isinstance(param, tuple)) + self.assertEqual(param, "us-ascii'en-us'Frank's Document") + + def test_rfc2231_no_extended_values(self): + eq = self.assertEqual + m = """\ +Content-Type: application/x-foo; name=\"Frank's Document\" + +""" + msg = email.message_from_string(m) + eq(msg.get_param('name'), "Frank's Document") + + def test_rfc2231_encoded_then_unencoded_segments(self): + eq = self.assertEqual + m = """\ +Content-Type: application/x-foo; +\tname*0*=\"us-ascii'en-us'My\"; +\tname*1=\" Document\"; +\tname*2*=\" For You\" + +""" + msg = email.message_from_string(m) + charset, language, s = msg.get_param('name') + eq(charset, 'us-ascii') + eq(language, 'en-us') + eq(s, 'My Document For You') + + def test_rfc2231_unencoded_then_encoded_segments(self): + eq = self.assertEqual + m = """\ +Content-Type: application/x-foo; +\tname*0=\"us-ascii'en-us'My\"; +\tname*1*=\" Document\"; +\tname*2*=\" For You\" + +""" + msg = email.message_from_string(m) + charset, language, s = msg.get_param('name') + eq(charset, 'us-ascii') + eq(language, 'en-us') + eq(s, 'My Document For You') + def _testclasses(): diff --git a/Lib/email/utils.py b/Lib/email/utils.py index 250eb19..26ebb0e 100644 --- a/Lib/email/utils.py +++ b/Lib/email/utils.py @@ -25,6 +25,7 @@ import time import base64 import random import socket +import urllib import warnings from cStringIO import StringIO @@ -45,6 +46,7 @@ COMMASPACE = ', ' EMPTYSTRING = '' UEMPTYSTRING = u'' CRLF = '\r\n' +TICK = "'" specialsre = re.compile(r'[][\\()<>@,:;".]') escapesre = re.compile(r'[][\\()"]') @@ -230,12 +232,14 @@ def unquote(str): # RFC2231-related functions - parameter encoding and decoding def decode_rfc2231(s): """Decode string according to RFC 2231""" - import urllib - parts = s.split("'", 2) - if len(parts) == 1: - return None, None, urllib.unquote(s) - charset, language, s = parts - return charset, language, urllib.unquote(s) + parts = s.split(TICK, 2) + if len(parts) <= 2: + return None, None, s + if len(parts) > 3: + charset, language = parts[:2] + s = TICK.join(parts[2:]) + return charset, language, s + return parts def encode_rfc2231(s, charset=None, language=None): @@ -259,37 +263,54 @@ rfc2231_continuation = re.compile(r'^(?P\w+)\*((?P[0-9]+)\*?)?$') def decode_params(params): """Decode parameters list according to RFC 2231. - params is a sequence of 2-tuples containing (content type, string value). + params is a sequence of 2-tuples containing (param name, string value). """ + # Copy params so we don't mess with the original + params = params[:] new_params = [] - # maps parameter's name to a list of continuations + # Map parameter's name to a list of continuations. The values are a + # 3-tuple of the continuation number, the string value, and a flag + # specifying whether a particular segment is %-encoded. rfc2231_params = {} - # params is a sequence of 2-tuples containing (content_type, string value) - name, value = params[0] + name, value = params.pop(0) new_params.append((name, value)) - # Cycle through each of the rest of the parameters. - for name, value in params[1:]: + while params: + name, value = params.pop(0) + if name.endswith('*'): + encoded = True + else: + encoded = False value = unquote(value) mo = rfc2231_continuation.match(name) if mo: name, num = mo.group('name', 'num') if num is not None: num = int(num) - rfc2231_param1 = rfc2231_params.setdefault(name, []) - rfc2231_param1.append((num, value)) + rfc2231_params.setdefault(name, []).append((num, value, encoded)) else: new_params.append((name, '"%s"' % quote(value))) if rfc2231_params: for name, continuations in rfc2231_params.items(): value = [] + extended = False # Sort by number continuations.sort() - # And now append all values in num order - for num, continuation in continuations: - value.append(continuation) - charset, language, value = decode_rfc2231(EMPTYSTRING.join(value)) - new_params.append( - (name, (charset, language, '"%s"' % quote(value)))) + # And now append all values in numerical order, converting + # %-encodings for the encoded segments. If any of the + # continuation names ends in a *, then the entire string, after + # decoding segments and concatenating, must have the charset and + # language specifiers at the beginning of the string. + for num, s, encoded in continuations: + if encoded: + s = urllib.unquote(s) + extended = True + value.append(s) + value = quote(EMPTYSTRING.join(value)) + if extended: + charset, language, value = decode_rfc2231(value) + new_params.append((name, (charset, language, '"%s"' % value))) + else: + new_params.append((name, '"%s"' % value)) return new_params def collapse_rfc2231_value(value, errors='replace', diff --git a/Lib/encodings/mbcs.py b/Lib/encodings/mbcs.py index ff77fde..baf46cb 100644 --- a/Lib/encodings/mbcs.py +++ b/Lib/encodings/mbcs.py @@ -7,42 +7,39 @@ which was written by Marc-Andre Lemburg (mal@lemburg.com). (c) Copyright CNRI, All Rights Reserved. NO WARRANTY. """ +# Import them explicitly to cause an ImportError +# on non-Windows systems +from codecs import mbcs_encode, mbcs_decode +# for IncrementalDecoder, IncrementalEncoder, ... import codecs ### Codec APIs -class Codec(codecs.Codec): +encode = mbcs_encode - # Note: Binding these as C functions will result in the class not - # converting them to methods. This is intended. - encode = codecs.mbcs_encode - decode = codecs.mbcs_decode +def decode(input, errors='strict'): + return mbcs_decode(input, errors, True) class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return codecs.mbcs_encode(input,self.errors)[0] + return mbcs_encode(input, self.errors)[0] -class IncrementalDecoder(codecs.IncrementalDecoder): - def decode(self, input, final=False): - return codecs.mbcs_decode(input,self.errors)[0] -class StreamWriter(Codec,codecs.StreamWriter): - pass +class IncrementalDecoder(codecs.BufferedIncrementalDecoder): + _buffer_decode = mbcs_decode -class StreamReader(Codec,codecs.StreamReader): - pass +class StreamWriter(codecs.StreamWriter): + encode = mbcs_encode -class StreamConverter(StreamWriter,StreamReader): - - encode = codecs.mbcs_decode - decode = codecs.mbcs_encode +class StreamReader(codecs.StreamReader): + decode = mbcs_decode ### encodings module API def getregentry(): return codecs.CodecInfo( name='mbcs', - encode=Codec.encode, - decode=Codec.decode, + encode=encode, + decode=decode, incrementalencoder=IncrementalEncoder, incrementaldecoder=IncrementalDecoder, streamreader=StreamReader, diff --git a/Lib/encodings/punycode.py b/Lib/encodings/punycode.py index 2cde8b9..d97200f 100644 --- a/Lib/encodings/punycode.py +++ b/Lib/encodings/punycode.py @@ -214,9 +214,9 @@ class IncrementalEncoder(codecs.IncrementalEncoder): class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): - if errors not in ('strict', 'replace', 'ignore'): - raise UnicodeError, "Unsupported error handling "+errors - return punycode_decode(input, errors) + if self.errors not in ('strict', 'replace', 'ignore'): + raise UnicodeError, "Unsupported error handling "+self.errors + return punycode_decode(input, self.errors) class StreamWriter(Codec,codecs.StreamWriter): pass diff --git a/Lib/encodings/utf_8_sig.py b/Lib/encodings/utf_8_sig.py index cd14ab0..f05f6b8 100644 --- a/Lib/encodings/utf_8_sig.py +++ b/Lib/encodings/utf_8_sig.py @@ -30,9 +30,9 @@ class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): if self.first: self.first = False - return codecs.BOM_UTF8 + codecs.utf_8_encode(input, errors)[0] + return codecs.BOM_UTF8 + codecs.utf_8_encode(input, self.errors)[0] else: - return codecs.utf_8_encode(input, errors)[0] + return codecs.utf_8_encode(input, self.errors)[0] def reset(self): codecs.IncrementalEncoder.reset(self) diff --git a/Lib/encodings/uu_codec.py b/Lib/encodings/uu_codec.py index 0877fe1..43fb93c 100644 --- a/Lib/encodings/uu_codec.py +++ b/Lib/encodings/uu_codec.py @@ -102,11 +102,11 @@ class Codec(codecs.Codec): class IncrementalEncoder(codecs.IncrementalEncoder): def encode(self, input, final=False): - return uu_encode(input, errors)[0] + return uu_encode(input, self.errors)[0] class IncrementalDecoder(codecs.IncrementalDecoder): def decode(self, input, final=False): - return uu_decode(input, errors)[0] + return uu_decode(input, self.errors)[0] class StreamWriter(Codec,codecs.StreamWriter): pass diff --git a/Lib/gzip.py b/Lib/gzip.py index 860accc..0bf29e8 100644 --- a/Lib/gzip.py +++ b/Lib/gzip.py @@ -315,7 +315,13 @@ class GzipFile: def close(self): if self.mode == WRITE: self.fileobj.write(self.compress.flush()) - write32(self.fileobj, self.crc) + # The native zlib crc is an unsigned 32-bit integer, but + # the Python wrapper implicitly casts that to a signed C + # long. So, on a 32-bit box self.crc may "look negative", + # while the same crc on a 64-bit box may "look positive". + # To avoid irksome warnings from the `struct` module, force + # it to look positive on all boxes. + write32u(self.fileobj, LOWU32(self.crc)) # self.size may exceed 2GB, or even 4GB write32u(self.fileobj, LOWU32(self.size)) self.fileobj = None diff --git a/Lib/httplib.py b/Lib/httplib.py index 36381de..5ae5efc 100644 --- a/Lib/httplib.py +++ b/Lib/httplib.py @@ -3,7 +3,7 @@ -HTTPConnection go through a number of "states", which defines when a client +HTTPConnection goes through a number of "states", which define when a client may legally make another request or fetch the response for a particular request. This diagram details these state transitions: @@ -926,15 +926,15 @@ class HTTPConnection: self.__state = _CS_IDLE if response.will_close: - # this effectively passes the connection to the response - self.close() + # Pass the socket to the response + self.sock = None else: # remember this, so we can tell when it is complete self.__response = response return response -# The next several classes are used to define FakeSocket,a socket-like +# The next several classes are used to define FakeSocket, a socket-like # interface to an SSL connection. # The primary complexity comes from faking a makefile() method. The diff --git a/Lib/idlelib/Bindings.py b/Lib/idlelib/Bindings.py index b5e90b0..d24be3f 100644 --- a/Lib/idlelib/Bindings.py +++ b/Lib/idlelib/Bindings.py @@ -80,6 +80,32 @@ menudefs = [ ]), ] +import sys +if sys.platform == 'darwin' and '.app' in sys.executable: + # Running as a proper MacOS application bundle. This block restructures + # the menus a little to make them conform better to the HIG. + + quitItem = menudefs[0][1][-1] + closeItem = menudefs[0][1][-2] + + # Remove the last 3 items of the file menu: a separator, close window and + # quit. Close window will be reinserted just above the save item, where + # it should be according to the HIG. Quit is in the application menu. + del menudefs[0][1][-3:] + menudefs[0][1].insert(6, closeItem) + + # Remove the 'About' entry from the help menu, it is in the application + # menu + del menudefs[-1][1][0:2] + + menudefs.insert(0, + ('application', [ + ('About IDLE', '<>'), + None, + ('_Preferences....', '<>'), + ])) + + default_keydefs = idleConf.GetCurrentKeySet() del sys diff --git a/Lib/idlelib/CREDITS.txt b/Lib/idlelib/CREDITS.txt index 6f4e95d..e838c03 100644 --- a/Lib/idlelib/CREDITS.txt +++ b/Lib/idlelib/CREDITS.txt @@ -19,17 +19,18 @@ the integration of the RPC and remote debugger, implemented the threaded subprocess, and made a number of usability enhancements. Other contributors include Raymond Hettinger, Tony Lownds (Mac integration), -Neal Norwitz (code check and clean-up), and Chui Tey (RPC integration, debugger -integration and persistent breakpoints). +Neal Norwitz (code check and clean-up), Ronald Oussoren (Mac integration), +Noam Raphael (Code Context, Call Tips, many other patches), and Chui Tey (RPC +integration, debugger integration and persistent breakpoints). -Scott David Daniels, Hernan Foffani, Christos Georgiou, Martin v. Löwis, -Jason Orendorff, Noam Raphael, Josh Robb, Nigel Rowe, Bruce Sherwood, and -Jeff Shute have submitted useful patches. Thanks, guys! +Scott David Daniels, Tal Einat, Hernan Foffani, Christos Georgiou, +Martin v. Löwis, Jason Orendorff, Josh Robb, Nigel Rowe, Bruce Sherwood, +and Jeff Shute have submitted useful patches. Thanks, guys! For additional details refer to NEWS.txt and Changelog. -Please contact the IDLE maintainer to have yourself included here if you -are one of those we missed! +Please contact the IDLE maintainer (kbk@shore.net) to have yourself included +here if you are one of those we missed! diff --git a/Lib/idlelib/CallTipWindow.py b/Lib/idlelib/CallTipWindow.py index afd4439..2223885 100644 --- a/Lib/idlelib/CallTipWindow.py +++ b/Lib/idlelib/CallTipWindow.py @@ -49,7 +49,11 @@ class CallTip: """ # truncate overly long calltip if len(text) >= 79: - text = text[:75] + ' ...' + textlines = text.splitlines() + for i, line in enumerate(textlines): + if len(line) > 79: + textlines[i] = line[:75] + ' ...' + text = '\n'.join(textlines) self.text = text if self.tipwindow or not self.text: return diff --git a/Lib/idlelib/CallTips.py b/Lib/idlelib/CallTips.py index 47a1d55..997eb13 100644 --- a/Lib/idlelib/CallTips.py +++ b/Lib/idlelib/CallTips.py @@ -127,7 +127,7 @@ def get_arg_text(ob): argText = "" if ob is not None: argOffset = 0 - if type(ob)==types.ClassType: + if type(ob) in (types.ClassType, types.TypeType): # Look for the highest __init__ in the class chain. fob = _find_constructor(ob) if fob is None: diff --git a/Lib/idlelib/CodeContext.py b/Lib/idlelib/CodeContext.py index 5d55f77..63cc82c 100644 --- a/Lib/idlelib/CodeContext.py +++ b/Lib/idlelib/CodeContext.py @@ -11,11 +11,10 @@ not open blocks are not shown in the context hints pane. """ import Tkinter from configHandler import idleConf -from sets import Set import re from sys import maxint as INFINITY -BLOCKOPENERS = Set(["class", "def", "elif", "else", "except", "finally", "for", +BLOCKOPENERS = set(["class", "def", "elif", "else", "except", "finally", "for", "if", "try", "while"]) UPDATEINTERVAL = 100 # millisec FONTUPDATEINTERVAL = 1000 # millisec diff --git a/Lib/idlelib/ColorDelegator.py b/Lib/idlelib/ColorDelegator.py index f258b34..e55f9e6 100644 --- a/Lib/idlelib/ColorDelegator.py +++ b/Lib/idlelib/ColorDelegator.py @@ -8,28 +8,29 @@ from configHandler import idleConf DEBUG = False -def any(name, list): - return "(?P<%s>" % name + "|".join(list) + ")" +def any(name, alternates): + "Return a named group pattern matching list of alternates." + return "(?P<%s>" % name + "|".join(alternates) + ")" def make_pat(): kw = r"\b" + any("KEYWORD", keyword.kwlist) + r"\b" builtinlist = [str(name) for name in dir(__builtin__) if not name.startswith('_')] # self.file = file("file") : - # 1st 'file' colorized normal, 2nd as builtin, 3rd as comment - builtin = r"([^.'\"\\]\b|^)" + any("BUILTIN", builtinlist) + r"\b" + # 1st 'file' colorized normal, 2nd as builtin, 3rd as string + builtin = r"([^.'\"\\#]\b|^)" + any("BUILTIN", builtinlist) + r"\b" comment = any("COMMENT", [r"#[^\n]*"]) - sqstring = r"(\b[rR])?'[^'\\\n]*(\\.[^'\\\n]*)*'?" - dqstring = r'(\b[rR])?"[^"\\\n]*(\\.[^"\\\n]*)*"?' - sq3string = r"(\b[rR])?'''[^'\\]*((\\.|'(?!''))[^'\\]*)*(''')?" - dq3string = r'(\b[rR])?"""[^"\\]*((\\.|"(?!""))[^"\\]*)*(""")?' + sqstring = r"(\b[rRuU])?'[^'\\\n]*(\\.[^'\\\n]*)*'?" + dqstring = r'(\b[rRuU])?"[^"\\\n]*(\\.[^"\\\n]*)*"?' + sq3string = r"(\b[rRuU])?'''[^'\\]*((\\.|'(?!''))[^'\\]*)*(''')?" + dq3string = r'(\b[rRuU])?"""[^"\\]*((\\.|"(?!""))[^"\\]*)*(""")?' string = any("STRING", [sq3string, dq3string, sqstring, dqstring]) return kw + "|" + builtin + "|" + comment + "|" + string +\ "|" + any("SYNC", [r"\n"]) prog = re.compile(make_pat(), re.S) idprog = re.compile(r"\s+(\w+)", re.S) -asprog = re.compile(r".*?\b(as)\b", re.S) +asprog = re.compile(r".*?\b(as)\b") class ColorDelegator(Delegator): @@ -208,10 +209,15 @@ class ColorDelegator(Delegator): head + "+%dc" % a, head + "+%dc" % b) elif value == "import": - # color all the "as" words on same line; - # cheap approximation to the truth + # color all the "as" words on same line, except + # if in a comment; cheap approximation to the + # truth + if '#' in chars: + endpos = chars.index('#') + else: + endpos = len(chars) while True: - m1 = self.asprog.match(chars, b) + m1 = self.asprog.match(chars, b, endpos) if not m1: break a, b = m1.span(1) diff --git a/Lib/idlelib/Debugger.py b/Lib/idlelib/Debugger.py index 7a9d02f..f56460a 100644 --- a/Lib/idlelib/Debugger.py +++ b/Lib/idlelib/Debugger.py @@ -4,6 +4,7 @@ import types from Tkinter import * from WindowList import ListedToplevel from ScrolledList import ScrolledList +import macosxSupport class Idb(bdb.Bdb): @@ -322,7 +323,13 @@ class Debugger: class StackViewer(ScrolledList): def __init__(self, master, flist, gui): - ScrolledList.__init__(self, master, width=80) + if macosxSupport.runningAsOSXApp(): + # At least on with the stock AquaTk version on OSX 10.4 you'll + # get an shaking GUI that eventually kills IDLE if the width + # argument is specified. + ScrolledList.__init__(self, master) + else: + ScrolledList.__init__(self, master, width=80) self.flist = flist self.gui = gui self.stack = [] diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py index 59440f0..6b8ab63 100644 --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -17,6 +17,7 @@ import ReplaceDialog import PyParse from configHandler import idleConf import aboutDialog, textView, configDialog +import macosxSupport # The default tab setting for a Text widget, in average-width characters. TK_TABWIDTH_DEFAULT = 8 @@ -66,26 +67,40 @@ class EditorWindow(object): 'Python%d%d.chm' % sys.version_info[:2]) if os.path.isfile(chmfile): dochome = chmfile + + elif macosxSupport.runningAsOSXApp(): + # documentation is stored inside the python framework + dochome = os.path.join(sys.prefix, + 'Resources/English.lproj/Documentation/index.html') + dochome = os.path.normpath(dochome) if os.path.isfile(dochome): EditorWindow.help_url = dochome + if sys.platform == 'darwin': + # Safari requires real file:-URLs + EditorWindow.help_url = 'file://' + EditorWindow.help_url else: EditorWindow.help_url = "http://www.python.org/doc/current" currentTheme=idleConf.CurrentTheme() self.flist = flist root = root or flist.root self.root = root + try: + sys.ps1 + except AttributeError: + sys.ps1 = '>>> ' self.menubar = Menu(root) self.top = top = WindowList.ListedToplevel(root, menu=self.menubar) if flist: self.tkinter_vars = flist.vars #self.top.instance_dict makes flist.inversedict avalable to #configDialog.py so it can access all EditorWindow instaces - self.top.instance_dict=flist.inversedict + self.top.instance_dict = flist.inversedict else: self.tkinter_vars = {} # keys: Tkinter event names # values: Tkinter variable instances - self.recent_files_path=os.path.join(idleConf.GetUserCfgDir(), + self.top.instance_dict = {} + self.recent_files_path = os.path.join(idleConf.GetUserCfgDir(), 'recent-files.lst') self.vbar = vbar = Scrollbar(top, name='vbar') self.text_frame = text_frame = Frame(top) @@ -111,6 +126,9 @@ class EditorWindow(object): self.top.protocol("WM_DELETE_WINDOW", self.close) self.top.bind("<>", self.close_event) + if macosxSupport.runningAsOSXApp(): + # Command-W on editorwindows doesn't work without this. + text.bind('<>', self.close_event) text.bind("<>", self.cut) text.bind("<>", self.copy) text.bind("<>", self.paste) @@ -278,6 +296,10 @@ class EditorWindow(object): def set_status_bar(self): self.status_bar = self.MultiStatusBar(self.top) + if macosxSupport.runningAsOSXApp(): + # Insert some padding to avoid obscuring some of the statusbar + # by the resize widget. + self.status_bar.set_label('_padding1', ' ', side=RIGHT) self.status_bar.set_label('column', 'Col: ?', side=RIGHT) self.status_bar.set_label('line', 'Ln: ?', side=RIGHT) self.status_bar.pack(side=BOTTOM, fill=X) @@ -301,6 +323,11 @@ class EditorWindow(object): ("help", "_Help"), ] + if macosxSupport.runningAsOSXApp(): + del menu_specs[-3] + menu_specs[-2] = ("windows", "_Window") + + def createmenubar(self): mbar = self.menubar self.menudict = menudict = {} @@ -308,6 +335,12 @@ class EditorWindow(object): underline, label = prepstr(label) menudict[name] = menu = Menu(mbar, name=name) mbar.add_cascade(label=label, menu=menu, underline=underline) + + if sys.platform == 'darwin' and '.framework' in sys.executable: + # Insert the application menu + menudict['application'] = menu = Menu(mbar, name='apple') + mbar.add_cascade(label='IDLE', menu=menu) + self.fill_menus() self.base_helpmenu_length = self.menudict['help'].index(END) self.reset_help_menu_entries() @@ -649,7 +682,7 @@ class EditorWindow(object): def __extra_help_callback(self, helpfile): "Create a callback with the helpfile value frozen at definition time" def display_extra_help(helpfile=helpfile): - if not (helpfile.startswith('www') or helpfile.startswith('http')): + if not helpfile.startswith(('www', 'http')): url = os.path.normpath(helpfile) if sys.platform[:3] == 'win': os.startfile(helpfile) @@ -1244,13 +1277,13 @@ class EditorWindow(object): "Toggle tabs", "Turn tabs " + ("on", "off")[self.usetabs] + "?\nIndent width " + - ("will be", "remains at")[self.usetabs] + " 8.", + ("will be", "remains at")[self.usetabs] + " 8." + + "\n Note: a tab is always 8 columns", parent=self.text): self.usetabs = not self.usetabs - # Try to prevent mixed tabs/spaces. - # User must reset indent width manually after using tabs - # if he insists on getting into trouble. - self.indentwidth = 8 + # Try to prevent inconsistent indentation. + # User must change indent width manually after using tabs. + self.indentwidth = 8 return "break" # XXX this isn't bound to anything -- see tabwidth comments diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 25e5d40..235963e 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -1,3 +1,46 @@ +What's New in IDLE 1.2c1? +========================= + +*Release date: XX-AUG-2006* + +- Changing tokenize (39046) to detect dedent broke tabnanny check (since 1.2a1) + +- ToggleTab dialog was setting indent to 8 even if cancelled (since 1.2a1). + +- When used w/o subprocess, all exceptions were preceded by an error + message claiming they were IDLE internal errors (since 1.2a1). + +What's New in IDLE 1.2b3? +========================= + +*Release date: 03-AUG-2006* + +- EditorWindow.test() was failing. Bug 1417598 + +- EditorWindow failed when used stand-alone if sys.ps1 not set. + Bug 1010370 Dave Florek + +- Tooltips failed on new-syle class __init__ args. Bug 1027566 Loren Guthrie + +- Avoid occasional failure to detect closing paren properly. + Patch 1407280 Tal Einat + +- Rebinding Tab key was inserting 'tab' instead of 'Tab'. Bug 1179168. + +- Colorizer now handles # correctly, also unicode strings and + 'as' keyword in comment directly following import command. Closes 1325071. + Patch 1479219 Tal Einat + +What's New in IDLE 1.2b2? +========================= + +*Release date: 11-JUL-2006* + +What's New in IDLE 1.2b1? +========================= + +*Release date: 20-JUN-2006* + What's New in IDLE 1.2a2? ========================= diff --git a/Lib/idlelib/ParenMatch.py b/Lib/idlelib/ParenMatch.py index 673aee2..250ae8b 100644 --- a/Lib/idlelib/ParenMatch.py +++ b/Lib/idlelib/ParenMatch.py @@ -8,7 +8,7 @@ parentheses, square brackets, and curly braces. from HyperParser import HyperParser from configHandler import idleConf -keysym_opener = {"parenright":'(', "bracketright":'[', "braceright":'{'} +_openers = {')':'(',']':'[','}':'{'} CHECK_DELAY = 100 # miliseconds class ParenMatch: @@ -100,12 +100,13 @@ class ParenMatch: def paren_closed_event(self, event): # If it was a shortcut and not really a closing paren, quit. - if self.text.get("insert-1c") not in (')',']','}'): + closer = self.text.get("insert-1c") + if closer not in _openers: return hp = HyperParser(self.editwin, "insert-1c") if not hp.is_in_code(): return - indices = hp.get_surrounding_brackets(keysym_opener[event.keysym], True) + indices = hp.get_surrounding_brackets(_openers[closer], True) if indices is None: self.warn_mismatched() return diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py index b6abe40..25eb446 100644 --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -11,6 +11,7 @@ import time import threading import traceback import types +import macosxSupport import linecache from code import InteractiveInterpreter @@ -721,8 +722,12 @@ class ModifiedInterpreter(InteractiveInterpreter): else: self.showtraceback() except: - print>>sys.stderr, "IDLE internal error in runcode()" + if use_subprocess: + print >> self.tkconsole.stderr, \ + "IDLE internal error in runcode()" self.showtraceback() + if use_subprocess: + self.tkconsole.endexecuting() finally: if not use_subprocess: self.tkconsole.endexecuting() @@ -777,6 +782,11 @@ class PyShell(OutputWindow): ("help", "_Help"), ] + if macosxSupport.runningAsOSXApp(): + del menu_specs[-3] + menu_specs[-2] = ("windows", "_Window") + + # New classes from IdleHistory import History @@ -1300,10 +1310,6 @@ def main(): script = None startup = False try: - sys.ps1 - except AttributeError: - sys.ps1 = '>>> ' - try: opts, args = getopt.getopt(sys.argv[1:], "c:deihnr:st:") except getopt.error, msg: sys.stderr.write("Error: %s\n" % str(msg)) @@ -1371,9 +1377,12 @@ def main(): enable_shell = enable_shell or not edit_start # start editor and/or shell windows: root = Tk(className="Idle") + fixwordbreaks(root) root.withdraw() flist = PyShellFileList(root) + macosxSupport.setupApp(root, flist) + if enable_edit: if not (cmd or script): for filename in args: @@ -1381,8 +1390,17 @@ def main(): if not args: flist.new() if enable_shell: - if not flist.open_shell(): + shell = flist.open_shell() + if not shell: return # couldn't open shell + + if macosxSupport.runningAsOSXApp() and flist.dict: + # On OSX: when the user has double-clicked on a file that causes + # IDLE to be launched the shell window will open just in front of + # the file she wants to see. Lower the interpreter window when + # there are open files. + shell.top.lower() + shell = flist.pyshell # handle remaining options: if debug: @@ -1403,6 +1421,7 @@ def main(): elif script: shell.interp.prepend_syspath(script) shell.interp.execfile(script) + root.mainloop() root.destroy() diff --git a/Lib/idlelib/ScriptBinding.py b/Lib/idlelib/ScriptBinding.py index 084c607..f325ad1 100644 --- a/Lib/idlelib/ScriptBinding.py +++ b/Lib/idlelib/ScriptBinding.py @@ -51,7 +51,7 @@ class ScriptBinding: # Provide instance variables referenced by Debugger # XXX This should be done differently self.flist = self.editwin.flist - self.root = self.flist.root + self.root = self.editwin.root def check_module_event(self, event): filename = self.getfilename() @@ -76,6 +76,9 @@ class ScriptBinding: self.editwin.gotoline(nag.get_lineno()) self.errorbox("Tab/space error", indent_message) return False + except IndentationError: + # From tokenize(), let compile() in checksyntax find it again. + pass return True def checksyntax(self, filename): diff --git a/Lib/idlelib/ZoomHeight.py b/Lib/idlelib/ZoomHeight.py index 2ab4656..83ca3a6 100644 --- a/Lib/idlelib/ZoomHeight.py +++ b/Lib/idlelib/ZoomHeight.py @@ -2,6 +2,7 @@ import re import sys +import macosxSupport class ZoomHeight: @@ -29,6 +30,14 @@ def zoom_height(top): if sys.platform == 'win32': newy = 0 newheight = newheight - 72 + + elif macosxSupport.runningAsOSXApp(): + # The '88' below is a magic number that avoids placing the bottom + # of the window below the panel on my machine. I don't know how + # to calculate the correct value for this with tkinter. + newy = 22 + newheight = newheight - newy - 88 + else: #newy = 24 newy = 0 diff --git a/Lib/idlelib/buildapp.py b/Lib/idlelib/buildapp.py deleted file mode 100644 index 672eb1e..0000000 --- a/Lib/idlelib/buildapp.py +++ /dev/null @@ -1,17 +0,0 @@ -# -# After running python setup.py install, run this program from the command -# line like so: -# -# % python2.3 buildapp.py build -# -# A double-clickable IDLE application will be created in the build/ directory. -# - -from bundlebuilder import buildapp - -buildapp( - name="IDLE", - mainprogram="idle.py", - argv_emulation=1, - iconfile="Icons/idle.icns", -) diff --git a/Lib/idlelib/config-keys.def b/Lib/idlelib/config-keys.def index 0653746..fb0aaf4 100644 --- a/Lib/idlelib/config-keys.def +++ b/Lib/idlelib/config-keys.def @@ -159,3 +159,56 @@ toggle-tabs= change-indentwidth= del-word-left= del-word-right= + +[IDLE Classic OSX] +toggle-tabs = +interrupt-execution = +untabify-region = +remove-selection = +print-window = +replace = +goto-line = +plain-newline-and-indent = +history-previous = +beginning-of-line = +end-of-line = +comment-region = +redo = +close-window = +restart-shell = +save-window-as-file = +close-all-windows = +view-restart = +tabify-region = +find-again = +find = +toggle-auto-coloring = +select-all = +smart-backspace = +change-indentwidth = +do-nothing = +smart-indent = +center-insert = +history-next = +del-word-right = +undo = +save-window = +uncomment-region = +cut = +find-in-files = +dedent-region = +copy = +paste = +indent-region = +del-word-left = +newline-and-indent = +end-of-file = +open-class-browser = +open-new-window = +open-module = +find-selection = +python-context-help = +save-copy-of-window-as-file = +open-window-from-file = +python-docs = + diff --git a/Lib/idlelib/configHandler.py b/Lib/idlelib/configHandler.py index 191a87c..826fb5d 100644 --- a/Lib/idlelib/configHandler.py +++ b/Lib/idlelib/configHandler.py @@ -20,6 +20,7 @@ configuration problem notification and resolution. import os import sys import string +import macosxSupport from ConfigParser import ConfigParser, NoOptionError, NoSectionError class InvalidConfigType(Exception): pass @@ -406,7 +407,7 @@ class IdleConf: names=extnNameList kbNameIndicies=[] for name in names: - if name.endswith('_bindings') or name.endswith('_cfgBindings'): + if name.endswith(('_bindings', '_cfgBindings')): kbNameIndicies.append(names.index(name)) kbNameIndicies.sort() kbNameIndicies.reverse() @@ -495,7 +496,18 @@ class IdleConf: return binding def GetCurrentKeySet(self): - return self.GetKeySet(self.CurrentKeys()) + result = self.GetKeySet(self.CurrentKeys()) + + if macosxSupport.runningAsOSXApp(): + # We're using AquaTk, replace all keybingings that use the + # Alt key by ones that use the Option key because the former + # don't work reliably. + for k, v in result.items(): + v2 = [ x.replace('':'greater', '/':'slash','?':'question','Page Up':'Prior','Page Down':'Next', 'Left Arrow':'Left','Right Arrow':'Right','Up Arrow':'Up', - 'Down Arrow': 'Down', 'Tab':'tab'} + 'Down Arrow': 'Down', 'Tab':'Tab'} if key in translateDict.keys(): key = translateDict[key] if 'Shift' in modifiers and key in string.ascii_lowercase: diff --git a/Lib/idlelib/macosxSupport.py b/Lib/idlelib/macosxSupport.py new file mode 100644 index 0000000..ad61fff --- /dev/null +++ b/Lib/idlelib/macosxSupport.py @@ -0,0 +1,112 @@ +""" +A number of function that enhance IDLE on MacOSX when it used as a normal +GUI application (as opposed to an X11 application). +""" +import sys + +def runningAsOSXApp(): + """ Returns True iff running from the IDLE.app bundle on OSX """ + return (sys.platform == 'darwin' and 'IDLE.app' in sys.argv[0]) + +def addOpenEventSupport(root, flist): + """ + This ensures that the application will respont to open AppleEvents, which + makes is feaseable to use IDLE as the default application for python files. + """ + def doOpenFile(*args): + for fn in args: + flist.open(fn) + + # The command below is a hook in aquatk that is called whenever the app + # receives a file open event. The callback can have multiple arguments, + # one for every file that should be opened. + root.createcommand("::tk::mac::OpenDocument", doOpenFile) + +def hideTkConsole(root): + root.tk.call('console', 'hide') + +def overrideRootMenu(root, flist): + """ + Replace the Tk root menu by something that's more appropriate for + IDLE. + """ + # The menu that is attached to the Tk root (".") is also used by AquaTk for + # all windows that don't specify a menu of their own. The default menubar + # contains a number of menus, none of which are appropriate for IDLE. The + # Most annoying of those is an 'About Tck/Tk...' menu in the application + # menu. + # + # This function replaces the default menubar by a mostly empty one, it + # should only contain the correct application menu and the window menu. + # + # Due to a (mis-)feature of TkAqua the user will also see an empty Help + # menu. + from Tkinter import Menu, Text, Text + from EditorWindow import prepstr, get_accelerator + import Bindings + import WindowList + from MultiCall import MultiCallCreator + + menubar = Menu(root) + root.configure(menu=menubar) + menudict = {} + + menudict['windows'] = menu = Menu(menubar, name='windows') + menubar.add_cascade(label='Window', menu=menu, underline=0) + + def postwindowsmenu(menu=menu): + end = menu.index('end') + if end is None: + end = -1 + + if end > 0: + menu.delete(0, end) + WindowList.add_windows_to_menu(menu) + WindowList.register_callback(postwindowsmenu) + + menudict['application'] = menu = Menu(menubar, name='apple') + menubar.add_cascade(label='IDLE', menu=menu) + + def about_dialog(event=None): + import aboutDialog + aboutDialog.AboutDialog(root, 'About IDLE') + + def config_dialog(event=None): + import configDialog + configDialog.ConfigDialog(root, 'Settings') + + root.bind('<>', about_dialog) + root.bind('<>', config_dialog) + if flist: + root.bind('<>', flist.close_all_callback) + + for mname, entrylist in Bindings.menudefs: + menu = menudict.get(mname) + if not menu: + continue + for entry in entrylist: + if not entry: + menu.add_separator() + else: + label, eventname = entry + underline, label = prepstr(label) + accelerator = get_accelerator(Bindings.default_keydefs, + eventname) + def command(text=root, eventname=eventname): + text.event_generate(eventname) + menu.add_command(label=label, underline=underline, + command=command, accelerator=accelerator) + + + + + +def setupApp(root, flist): + """ + Perform setup for the OSX application bundle. + """ + if not runningAsOSXApp(): return + + hideTkConsole(root) + overrideRootMenu(root, flist) + addOpenEventSupport(root, flist) diff --git a/Lib/inspect.py b/Lib/inspect.py index bf7f006..0b498b5 100644 --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -89,6 +89,40 @@ def isdatadescriptor(object): is not guaranteed.""" return (hasattr(object, "__set__") and hasattr(object, "__get__")) +if hasattr(types, 'MemberDescriptorType'): + # CPython and equivalent + def ismemberdescriptor(object): + """Return true if the object is a member descriptor. + + Member descriptors are specialized descriptors defined in extension + modules.""" + return isinstance(object, types.MemberDescriptorType) +else: + # Other implementations + def ismemberdescriptor(object): + """Return true if the object is a member descriptor. + + Member descriptors are specialized descriptors defined in extension + modules.""" + return False + +if hasattr(types, 'GetSetDescriptorType'): + # CPython and equivalent + def isgetsetdescriptor(object): + """Return true if the object is a getset descriptor. + + getset descriptors are specialized descriptors defined in extension + modules.""" + return isinstance(object, types.GetSetDescriptorType) +else: + # Other implementations + def isgetsetdescriptor(object): + """Return true if the object is a getset descriptor. + + getset descriptors are specialized descriptors defined in extension + modules.""" + return False + def isfunction(object): """Return true if the object is a user-defined function. @@ -355,40 +389,38 @@ def getsourcefile(object): return None if os.path.exists(filename): return filename - # Ugly but necessary - '' and '' mean that getmodule() - # would infinitely recurse, because they're not real files nor loadable - # Note that this means that writing a PEP 302 loader that uses '<' - # at the start of a filename is now not a good idea. :( - if filename[:1]!='<' and hasattr(getmodule(object), '__loader__'): + # only return a non-existent filename if the module has a PEP 302 loader + if hasattr(getmodule(object, filename), '__loader__'): return filename -def getabsfile(object): +def getabsfile(object, _filename=None): """Return an absolute path to the source or compiled file for an object. The idea is for each object to have a unique origin, so this routine normalizes the result as much as possible.""" - return os.path.normcase( - os.path.abspath(getsourcefile(object) or getfile(object))) + if _filename is None: + _filename = getsourcefile(object) or getfile(object) + return os.path.normcase(os.path.abspath(_filename)) modulesbyfile = {} -def getmodule(object): +def getmodule(object, _filename=None): """Return the module an object was defined in, or None if not found.""" if ismodule(object): return object if hasattr(object, '__module__'): return sys.modules.get(object.__module__) try: - file = getabsfile(object) + file = getabsfile(object, _filename) except TypeError: return None if file in modulesbyfile: return sys.modules.get(modulesbyfile[file]) for module in sys.modules.values(): if ismodule(module) and hasattr(module, '__file__'): - modulesbyfile[ - os.path.realpath( - getabsfile(module))] = module.__name__ + f = getabsfile(module) + modulesbyfile[f] = modulesbyfile[ + os.path.realpath(f)] = module.__name__ if file in modulesbyfile: return sys.modules.get(modulesbyfile[file]) main = sys.modules['__main__'] diff --git a/Lib/lib-tk/Tkinter.py b/Lib/lib-tk/Tkinter.py index 0ba954e..b248031 100644 --- a/Lib/lib-tk/Tkinter.py +++ b/Lib/lib-tk/Tkinter.py @@ -168,18 +168,30 @@ class Variable: Subclasses StringVar, IntVar, DoubleVar, BooleanVar are specializations that constrain the type of the value returned from get().""" _default = "" - def __init__(self, master=None): - """Construct a variable with an optional MASTER as master widget. - The variable is named PY_VAR_number in Tcl. + def __init__(self, master=None, value=None, name=None): + """Construct a variable + + MASTER can be given as master widget. + VALUE is an optional value (defaults to "") + NAME is an optional Tcl name (defaults to PY_VARnum). + + If NAME matches an existing variable and VALUE is omitted + then the existing value is retained. """ global _varnum if not master: master = _default_root self._master = master self._tk = master.tk - self._name = 'PY_VAR' + repr(_varnum) - _varnum = _varnum + 1 - self.set(self._default) + if name: + self._name = name + else: + self._name = 'PY_VAR' + repr(_varnum) + _varnum += 1 + if value != None: + self.set(value) + elif not self._tk.call("info", "exists", self._name): + self.set(self._default) def __del__(self): """Unset the variable in Tcl.""" self._tk.globalunsetvar(self._name) @@ -217,15 +229,29 @@ class Variable: """Return all trace callback information.""" return map(self._tk.split, self._tk.splitlist( self._tk.call("trace", "vinfo", self._name))) + def __eq__(self, other): + """Comparison for equality (==). + + Note: if the Variable's master matters to behavior + also compare self._master == other._master + """ + return self.__class__.__name__ == other.__class__.__name__ \ + and self._name == other._name class StringVar(Variable): """Value holder for strings variables.""" _default = "" - def __init__(self, master=None): + def __init__(self, master=None, value=None, name=None): """Construct a string variable. - MASTER can be given as master widget.""" - Variable.__init__(self, master) + MASTER can be given as master widget. + VALUE is an optional value (defaults to "") + NAME is an optional Tcl name (defaults to PY_VARnum). + + If NAME matches an existing variable and VALUE is omitted + then the existing value is retained. + """ + Variable.__init__(self, master, value, name) def get(self): """Return value of variable as string.""" @@ -237,11 +263,17 @@ class StringVar(Variable): class IntVar(Variable): """Value holder for integer variables.""" _default = 0 - def __init__(self, master=None): + def __init__(self, master=None, value=None, name=None): """Construct an integer variable. - MASTER can be given as master widget.""" - Variable.__init__(self, master) + MASTER can be given as master widget. + VALUE is an optional value (defaults to 0) + NAME is an optional Tcl name (defaults to PY_VARnum). + + If NAME matches an existing variable and VALUE is omitted + then the existing value is retained. + """ + Variable.__init__(self, master, value, name) def set(self, value): """Set the variable to value, converting booleans to integers.""" @@ -256,11 +288,17 @@ class IntVar(Variable): class DoubleVar(Variable): """Value holder for float variables.""" _default = 0.0 - def __init__(self, master=None): + def __init__(self, master=None, value=None, name=None): """Construct a float variable. - MASTER can be given as a master widget.""" - Variable.__init__(self, master) + MASTER can be given as master widget. + VALUE is an optional value (defaults to 0.0) + NAME is an optional Tcl name (defaults to PY_VARnum). + + If NAME matches an existing variable and VALUE is omitted + then the existing value is retained. + """ + Variable.__init__(self, master, value, name) def get(self): """Return the value of the variable as a float.""" @@ -268,12 +306,18 @@ class DoubleVar(Variable): class BooleanVar(Variable): """Value holder for boolean variables.""" - _default = "false" - def __init__(self, master=None): + _default = False + def __init__(self, master=None, value=None, name=None): """Construct a boolean variable. - MASTER can be given as a master widget.""" - Variable.__init__(self, master) + MASTER can be given as master widget. + VALUE is an optional value (defaults to False) + NAME is an optional Tcl name (defaults to PY_VARnum). + + If NAME matches an existing variable and VALUE is omitted + then the existing value is retained. + """ + Variable.__init__(self, master, value, name) def get(self): """Return the value of the variable as a bool.""" @@ -1456,10 +1500,19 @@ class Wm: the group leader of this widget if None is given.""" return self.tk.call('wm', 'group', self._w, pathName) group = wm_group - def wm_iconbitmap(self, bitmap=None): + def wm_iconbitmap(self, bitmap=None, default=None): """Set bitmap for the iconified widget to BITMAP. Return - the bitmap if None is given.""" - return self.tk.call('wm', 'iconbitmap', self._w, bitmap) + the bitmap if None is given. + + Under Windows, the DEFAULT parameter can be used to set the icon + for the widget and any descendents that don't have an icon set + explicitly. DEFAULT can be the relative path to a .ico file + (example: root.iconbitmap(default='myicon.ico') ). See Tk + documentation for more information.""" + if default: + return self.tk.call('wm', 'iconbitmap', self._w, '-default', default) + else: + return self.tk.call('wm', 'iconbitmap', self._w, bitmap) iconbitmap = wm_iconbitmap def wm_iconify(self): """Display widget as icon.""" @@ -1880,9 +1933,9 @@ class BaseWidget(Misc): def destroy(self): """Destroy this and all descendants widgets.""" for c in self.children.values(): c.destroy() + self.tk.call('destroy', self._w) if self.master.children.has_key(self._name): del self.master.children[self._name] - self.tk.call('destroy', self._w) Misc.destroy(self) def _do(self, name, args=()): # XXX Obsolete -- better use self.tk.call directly! diff --git a/Lib/lib-tk/tkMessageBox.py b/Lib/lib-tk/tkMessageBox.py index 25071fe..aff069b 100644 --- a/Lib/lib-tk/tkMessageBox.py +++ b/Lib/lib-tk/tkMessageBox.py @@ -63,9 +63,10 @@ class Message(Dialog): # # convenience stuff -def _show(title=None, message=None, icon=None, type=None, **options): - if icon: options["icon"] = icon - if type: options["type"] = type +# Rename _icon and _type options to allow overriding them in options +def _show(title=None, message=None, _icon=None, _type=None, **options): + if _icon and "icon" not in options: options["icon"] = _icon + if _type and "type" not in options: options["type"] = _type if title: options["title"] = title if message: options["message"] = message res = Message(**options).show() diff --git a/Lib/lib-tk/turtle.py b/Lib/lib-tk/turtle.py index d68e405..01a55b1 100644 --- a/Lib/lib-tk/turtle.py +++ b/Lib/lib-tk/turtle.py @@ -30,6 +30,7 @@ class RawPen: self._tracing = 1 self._arrow = 0 self._delay = 10 # default delay for drawing + self._angle = 0.0 self.degrees() self.reset() @@ -39,6 +40,10 @@ class RawPen: Example: >>> turtle.degrees() """ + # Don't try to change _angle if it is 0, because + # _fullcircle might not be set, yet + if self._angle: + self._angle = (self._angle / self._fullcircle) * fullcircle self._fullcircle = fullcircle self._invradian = pi / (fullcircle * 0.5) @@ -81,7 +86,6 @@ class RawPen: self._color = "black" self._filling = 0 self._path = [] - self._tofill = [] self.clear() canvas._root().tkraise() @@ -301,19 +305,15 @@ class RawPen: {'fill': self._color, 'smooth': smooth}) self._items.append(item) - if self._tofill: - for item in self._tofill: - self._canvas.itemconfigure(item, fill=self._color) - self._items.append(item) self._path = [] - self._tofill = [] self._filling = flag if flag: self._path.append(self._position) - self.forward(0) def begin_fill(self): """ Called just before drawing a shape to be filled. + Must eventually be followed by a corresponding end_fill() call. + Otherwise it will be ignored. Example: >>> turtle.begin_fill() @@ -326,7 +326,8 @@ class RawPen: >>> turtle.forward(100) >>> turtle.end_fill() """ - self.fill(1) + self._path = [self._position] + self._filling = 1 def end_fill(self): """ Called after drawing a shape to be filled. @@ -344,7 +345,7 @@ class RawPen: """ self.fill(0) - def circle(self, radius, extent=None): + def circle(self, radius, extent = None): """ Draw a circle with given radius. The center is radius units left of the turtle; extent determines which part of the circle is drawn. If not given, @@ -361,52 +362,18 @@ class RawPen: """ if extent is None: extent = self._fullcircle - x0, y0 = self._position - xc = x0 - radius * sin(self._angle * self._invradian) - yc = y0 - radius * cos(self._angle * self._invradian) - if radius >= 0.0: - start = self._angle - (self._fullcircle / 4.0) - else: - start = self._angle + (self._fullcircle / 4.0) - extent = -extent - if self._filling: - if abs(extent) >= self._fullcircle: - item = self._canvas.create_oval(xc-radius, yc-radius, - xc+radius, yc+radius, - width=self._width, - outline="") - self._tofill.append(item) - item = self._canvas.create_arc(xc-radius, yc-radius, - xc+radius, yc+radius, - style="chord", - start=start, - extent=extent, - width=self._width, - outline="") - self._tofill.append(item) - if self._drawing: - if abs(extent) >= self._fullcircle: - item = self._canvas.create_oval(xc-radius, yc-radius, - xc+radius, yc+radius, - width=self._width, - outline=self._color) - self._items.append(item) - item = self._canvas.create_arc(xc-radius, yc-radius, - xc+radius, yc+radius, - style="arc", - start=start, - extent=extent, - width=self._width, - outline=self._color) - self._items.append(item) - angle = start + extent - x1 = xc + abs(radius) * cos(angle * self._invradian) - y1 = yc - abs(radius) * sin(angle * self._invradian) - self._angle = (self._angle + extent) % self._fullcircle - self._position = x1, y1 - if self._filling: - self._path.append(self._position) - self._draw_turtle() + frac = abs(extent)/self._fullcircle + steps = 1+int(min(11+abs(radius)/6.0, 59.0)*frac) + w = 1.0 * extent / steps + w2 = 0.5 * w + l = 2.0 * radius * sin(w2*self._invradian) + if radius < 0: + l, w, w2 = -l, -w, -w2 + self.left(w2) + for i in range(steps): + self.forward(l) + self.left(w) + self.right(w2) def heading(self): """ Return the turtle's current heading. @@ -634,6 +601,7 @@ class RawPen: def _draw_turtle(self, position=[]): if not self._tracing: + self._canvas.update() return if position == []: position = self._position @@ -678,7 +646,7 @@ class Pen(RawPen): _canvas = Tkinter.Canvas(_root, background="white") _canvas.pack(expand=1, fill="both") - setup(width=_width, height= _height, startx=_startx, starty=_starty) + setup(width=_width, height= _height, startx=_startx, starty=_starty) RawPen.__init__(self, _canvas) @@ -720,7 +688,7 @@ def color(*args): _getpen().color(*args) def write(arg, move=0): _getpen().write(arg, move) def fill(flag): _getpen().fill(flag) def begin_fill(): _getpen().begin_fill() -def end_fill(): _getpen.end_fill() +def end_fill(): _getpen().end_fill() def circle(radius, extent=None): _getpen().circle(radius, extent) def goto(*args): _getpen().goto(*args) def heading(): return _getpen().heading() @@ -745,7 +713,7 @@ for methodname in dir(RawPen): def setup(**geometry): """ Sets the size and position of the main window. - Keywords are width, height, startx and starty + Keywords are width, height, startx and starty: width: either a size in pixels or a fraction of the screen. Default is 50% of screen. @@ -820,7 +788,7 @@ def setup(**geometry): _root.geometry("%dx%d+%d+%d" % (_width, _height, _startx, _starty)) def title(title): - """ set the window title. + """Set the window title. By default this is set to 'Turtle Graphics' @@ -929,15 +897,30 @@ def demo2(): speed(speeds[sp]) color(0.25,0,0.75) fill(0) - color("green") - left(130) + # draw and fill a concave shape + left(120) up() - forward(90) + forward(70) + right(30) + down() color("red") - speed('fastest') + speed("fastest") + fill(1) + for i in range(4): + circle(50,90) + right(90) + forward(30) + right(90) + color("yellow") + fill(0) + left(90) + up() + forward(30) down(); + color("red") + # create a second turtle and make the original pursue and catch it turtle=Turtle() turtle.reset() diff --git a/Lib/linecache.py b/Lib/linecache.py index f49695a..4838625 100644 --- a/Lib/linecache.py +++ b/Lib/linecache.py @@ -94,6 +94,10 @@ def updatecache(filename, module_globals=None): except (ImportError, IOError): pass else: + if data is None: + # No luck, the PEP302 loader cannot find the source + # for this module. + return [] cache[filename] = ( len(data), None, [line+'\n' for line in data.splitlines()], fullname diff --git a/Lib/logging/config.py b/Lib/logging/config.py index 457ec5c..1d5f8c4 100644 --- a/Lib/logging/config.py +++ b/Lib/logging/config.py @@ -79,6 +79,7 @@ def fileConfig(fname, defaults=None): logging._acquireLock() try: logging._handlers.clear() + logging._handlerList = [] # Handlers add themselves to logging._handlers handlers = _install_handlers(cp, formatters) _install_loggers(cp, handlers) diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py index e0da254..3552950 100644 --- a/Lib/logging/handlers.py +++ b/Lib/logging/handlers.py @@ -128,12 +128,7 @@ class RotatingFileHandler(BaseRotatingHandler): dfn = self.baseFilename + ".1" if os.path.exists(dfn): os.remove(dfn) - try: - os.rename(self.baseFilename, dfn) - except (KeyboardInterrupt, SystemExit): - raise - except: - self.handleError(record) + os.rename(self.baseFilename, dfn) #print "%s -> %s" % (self.baseFilename, dfn) if self.encoding: self.stream = codecs.open(self.baseFilename, 'w', self.encoding) @@ -273,12 +268,7 @@ class TimedRotatingFileHandler(BaseRotatingHandler): dfn = self.baseFilename + "." + time.strftime(self.suffix, timeTuple) if os.path.exists(dfn): os.remove(dfn) - try: - os.rename(self.baseFilename, dfn) - except (KeyboardInterrupt, SystemExit): - raise - except: - self.handleError(record) + os.rename(self.baseFilename, dfn) if self.backupCount > 0: # find the oldest log file and delete it s = glob.glob(self.baseFilename + ".20*") @@ -572,6 +562,18 @@ class SysLogHandler(logging.Handler): "local7": LOG_LOCAL7, } + #The map below appears to be trivially lowercasing the key. However, + #there's more to it than meets the eye - in some locales, lowercasing + #gives unexpected results. See SF #1524081: in the Turkish locale, + #"INFO".lower() != "info" + priority_map = { + "DEBUG" : "debug", + "INFO" : "info", + "WARNING" : "warning", + "ERROR" : "error", + "CRITICAL" : "critical" + } + def __init__(self, address=('localhost', SYSLOG_UDP_PORT), facility=LOG_USER): """ Initialize a handler. @@ -608,7 +610,7 @@ class SysLogHandler(logging.Handler): # necessary. log_format_string = '<%d>%s\000' - def encodePriority (self, facility, priority): + def encodePriority(self, facility, priority): """ Encode the facility and priority. You can pass in strings or integers - if strings are passed, the facility_names and @@ -629,6 +631,16 @@ class SysLogHandler(logging.Handler): self.socket.close() logging.Handler.close(self) + def mapPriority(self, levelName): + """ + Map a logging level name to a key in the priority_names map. + This is useful in two scenarios: when custom levels are being + used, and in the case where you can't do a straightforward + mapping by lowercasing the logging level name because of locale- + specific issues (see SF #1524081). + """ + return self.priority_map.get(levelName, "warning") + def emit(self, record): """ Emit a record. @@ -643,8 +655,8 @@ class SysLogHandler(logging.Handler): """ msg = self.log_format_string % ( self.encodePriority(self.facility, - string.lower(record.levelname)), - msg) + self.mapPriority(record.levelname)), + msg) try: if self.unixsocket: try: diff --git a/Lib/mailbox.py b/Lib/mailbox.py index bb115e1..b72128b 100755 --- a/Lib/mailbox.py +++ b/Lib/mailbox.py @@ -15,7 +15,10 @@ import email.Generator import rfc822 import StringIO try: - import fnctl + if sys.platform == 'os2emx': + # OS/2 EMX fcntl() not adequate + raise ImportError + import fcntl except ImportError: fcntl = None @@ -565,7 +568,8 @@ class _singlefileMailbox(Mailbox): try: os.rename(new_file.name, self._path) except OSError, e: - if e.errno == errno.EEXIST: + if e.errno == errno.EEXIST or \ + (os.name == 'os2' and e.errno == errno.EACCES): os.remove(self._path) os.rename(new_file.name, self._path) else: @@ -1030,6 +1034,9 @@ class MH(Mailbox): if hasattr(os, 'link'): os.link(os.path.join(self._path, str(key)), os.path.join(self._path, str(prev + 1))) + if sys.platform == 'os2emx': + # cannot unlink an open file on OS/2 + f.close() os.unlink(os.path.join(self._path, str(key))) else: f.close() @@ -1798,26 +1805,18 @@ class _PartialFile(_ProxyFile): def _lock_file(f, dotlock=True): - """Lock file f using lockf, flock, and dot locking.""" + """Lock file f using lockf and dot locking.""" dotlock_done = False try: if fcntl: try: fcntl.lockf(f, fcntl.LOCK_EX | fcntl.LOCK_NB) except IOError, e: - if e.errno == errno.EAGAIN: + if e.errno in (errno.EAGAIN, errno.EACCES): raise ExternalClashError('lockf: lock unavailable: %s' % f.name) else: raise - try: - fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB) - except IOError, e: - if e.errno == errno.EWOULDBLOCK: - raise ExternalClashError('flock: lock unavailable: %s' % - f.name) - else: - raise if dotlock: try: pre_lock = _create_temporary(f.name + '.lock') @@ -1836,7 +1835,8 @@ def _lock_file(f, dotlock=True): os.rename(pre_lock.name, f.name + '.lock') dotlock_done = True except OSError, e: - if e.errno == errno.EEXIST: + if e.errno == errno.EEXIST or \ + (os.name == 'os2' and e.errno == errno.EACCES): os.remove(pre_lock.name) raise ExternalClashError('dot lock unavailable: %s' % f.name) @@ -1845,16 +1845,14 @@ def _lock_file(f, dotlock=True): except: if fcntl: fcntl.lockf(f, fcntl.LOCK_UN) - fcntl.flock(f, fcntl.LOCK_UN) if dotlock_done: os.remove(f.name + '.lock') raise def _unlock_file(f): - """Unlock file f using lockf, flock, and dot locking.""" + """Unlock file f using lockf and dot locking.""" if fcntl: fcntl.lockf(f, fcntl.LOCK_UN) - fcntl.flock(f, fcntl.LOCK_UN) if os.path.exists(f.name + '.lock'): os.remove(f.name + '.lock') diff --git a/Lib/mimetypes.py b/Lib/mimetypes.py index bee2ff7..b0d2f18 100644 --- a/Lib/mimetypes.py +++ b/Lib/mimetypes.py @@ -33,6 +33,10 @@ __all__ = [ knownfiles = [ "/etc/mime.types", + "/etc/httpd/mime.types", # Mac OS X + "/etc/httpd/conf/mime.types", # Apache + "/etc/apache/mime.types", # Apache 1 + "/etc/apache2/mime.types", # Apache 2 "/usr/local/etc/httpd/conf/mime.types", "/usr/local/lib/netscape/mime.types", "/usr/local/etc/httpd/conf/mime.types", # Apache 1.2 diff --git a/Lib/msilib/__init__.py b/Lib/msilib/__init__.py index 0881409..4be82b0 100644 --- a/Lib/msilib/__init__.py +++ b/Lib/msilib/__init__.py @@ -187,7 +187,7 @@ class CAB: self.filenames = sets.Set() self.index = 0 - def gen_id(self, dir, file): + def gen_id(self, file): logical = _logical = make_id(file) pos = 1 while logical in self.filenames: @@ -196,9 +196,11 @@ class CAB: self.filenames.add(logical) return logical - def append(self, full, logical): + def append(self, full, file, logical): if os.path.isdir(full): return + if not logical: + logical = self.gen_id(file) self.index += 1 self.files.append((full, logical)) return self.index, logical @@ -328,7 +330,7 @@ class Directory: logical = self.keyfiles[file] else: logical = None - sequence, logical = self.cab.append(absolute, logical) + sequence, logical = self.cab.append(absolute, file, logical) assert logical not in self.ids self.ids.add(logical) short = self.make_short(file) @@ -403,7 +405,7 @@ class Control: [(self.dlg.name, self.name, event, argument, condition, ordering)]) - def mapping(self, mapping, attribute): + def mapping(self, event, attribute): add_data(self.dlg.db, "EventMapping", [(self.dlg.name, self.name, event, attribute)]) diff --git a/Lib/optparse.py b/Lib/optparse.py index 6b8f5d1..62d2f7e 100644 --- a/Lib/optparse.py +++ b/Lib/optparse.py @@ -16,7 +16,7 @@ For support, use the optik-users@lists.sourceforge.net mailing list # Python developers: please do not make changes to this file, since # it is automatically generated from the Optik source code. -__version__ = "1.5.1" +__version__ = "1.5.3" __all__ = ['Option', 'SUPPRESS_HELP', @@ -75,9 +75,9 @@ def _repr(self): # This file was generated from: -# Id: option_parser.py 509 2006-04-20 00:58:24Z gward -# Id: option.py 509 2006-04-20 00:58:24Z gward -# Id: help.py 509 2006-04-20 00:58:24Z gward +# Id: option_parser.py 527 2006-07-23 15:21:30Z greg +# Id: option.py 522 2006-06-11 16:22:03Z gward +# Id: help.py 527 2006-07-23 15:21:30Z greg # Id: errors.py 509 2006-04-20 00:58:24Z gward try: @@ -1629,6 +1629,13 @@ class OptionParser (OptionContainer): result.append(self.format_epilog(formatter)) return "".join(result) + # used by test suite + def _get_encoding(self, file): + encoding = getattr(file, "encoding", None) + if not encoding: + encoding = sys.getdefaultencoding() + return encoding + def print_help(self, file=None): """print_help(file : file = stdout) @@ -1637,7 +1644,8 @@ class OptionParser (OptionContainer): """ if file is None: file = sys.stdout - file.write(self.format_help()) + encoding = self._get_encoding(file) + file.write(self.format_help().encode(encoding, "replace")) # class OptionParser diff --git a/Lib/os.py b/Lib/os.py index 31002ac..2d1b29b 100644 --- a/Lib/os.py +++ b/Lib/os.py @@ -723,7 +723,7 @@ if not _exists("urandom"): """ try: _urandomfd = open("/dev/urandom", O_RDONLY) - except: + except (OSError, IOError): raise NotImplementedError("/dev/urandom (or equivalent) not found") bytes = "" while len(bytes) < n: diff --git a/Lib/pdb.py b/Lib/pdb.py index 94f61f7..06181e7 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -235,7 +235,8 @@ class Pdb(bdb.Bdb, cmd.Cmd): """Interpret the argument as though it had been typed in response to the prompt. - Checks wether this line is typed in the normal prompt or in a breakpoint command list definition + Checks whether this line is typed at the normal prompt or in + a breakpoint command list definition. """ if not self.commands_defining: return cmd.Cmd.onecmd(self, line) diff --git a/Lib/pkgutil.py b/Lib/pkgutil.py index 26c797f..37738e4 100644 --- a/Lib/pkgutil.py +++ b/Lib/pkgutil.py @@ -69,7 +69,33 @@ def simplegeneric(func): def walk_packages(path=None, prefix='', onerror=None): - """Yield submodule names+loaders recursively, for path or sys.path""" + """Yields (module_loader, name, ispkg) for all modules recursively + on path, or, if path is None, all accessible modules. + + 'path' should be either None or a list of paths to look for + modules in. + + 'prefix' is a string to output on the front of every module name + on output. + + Note that this function must import all *packages* (NOT all + modules!) on the given path, in order to access the __path__ + attribute to find submodules. + + 'onerror' is a function which gets called with one argument (the + name of the package which was being imported) if any exception + occurs while trying to import a package. If no onerror function is + supplied, ImportErrors are caught and ignored, while all other + exceptions are propagated, terminating the search. + + Examples: + + # list all modules python can access + walk_packages() + + # list all submodules of ctypes + walk_packages(ctypes.__path__, ctypes.__name__+'.') + """ def seen(p, m={}): if p in m: @@ -84,19 +110,33 @@ def walk_packages(path=None, prefix='', onerror=None): __import__(name) except ImportError: if onerror is not None: - onerror() + onerror(name) + except Exception: + if onerror is not None: + onerror(name) + else: + raise else: path = getattr(sys.modules[name], '__path__', None) or [] # don't traverse path items we've seen before path = [p for p in path if not seen(p)] - for item in walk_packages(path, name+'.'): + for item in walk_packages(path, name+'.', onerror): yield item def iter_modules(path=None, prefix=''): - """Yield submodule names+loaders for path or sys.path""" + """Yields (module_loader, name, ispkg) for all submodules on path, + or, if path is None, all top-level modules on sys.path. + + 'path' should be either None or a list of paths to look for + modules in. + + 'prefix' is a string to output on the front of every module name + on output. + """ + if path is None: importers = iter_importers() else: @@ -208,6 +248,7 @@ class ImpLoader: def _reopen(self): if self.file and self.file.closed: + mod_type = self.etc[2] if mod_type==imp.PY_SOURCE: self.file = open(self.filename, 'rU') elif mod_type in (imp.PY_COMPILED, imp.C_EXTENSION): @@ -340,9 +381,7 @@ def get_importer(path_item): importer = None sys.path_importer_cache.setdefault(path_item, importer) - # The boolean values are used for caching valid and invalid - # file paths for the built-in import machinery - if importer in (None, True, False): + if importer is None: try: importer = ImpImporter(path_item) except ImportError: diff --git a/Lib/popen2.py b/Lib/popen2.py index b966d4c..694979e 100644 --- a/Lib/popen2.py +++ b/Lib/popen2.py @@ -72,14 +72,14 @@ class Popen3: # In case the child hasn't been waited on, check if it's done. self.poll(_deadstate=sys.maxint) if self.sts < 0: - if _active: + if _active is not None: # Child is still running, keep us alive until we can wait on it. _active.append(self) def _run_child(self, cmd): if isinstance(cmd, basestring): cmd = ['/bin/sh', '-c', cmd] - for i in range(3, MAXFD): + for i in xrange(3, MAXFD): try: os.close(i) except OSError: diff --git a/Lib/pstats.py b/Lib/pstats.py index c3a8828..4e94b0c 100644 --- a/Lib/pstats.py +++ b/Lib/pstats.py @@ -548,8 +548,10 @@ if __name__ == '__main__': self.prompt = "% " if profile is not None: self.stats = Stats(profile) + self.stream = self.stats.stream else: self.stats = None + self.stream = sys.stdout def generic(self, fn, line): args = line.split() @@ -667,14 +669,15 @@ if __name__ == '__main__': return None import sys - print >> self.stream, "Welcome to the profile statistics browser." if len(sys.argv) > 1: initprofile = sys.argv[1] else: initprofile = None try: - ProfileBrowser(initprofile).cmdloop() - print >> self.stream, "Goodbye." + browser = ProfileBrowser(initprofile) + print >> browser.stream, "Welcome to the profile statistics browser." + browser.cmdloop() + print >> browser.stream, "Goodbye." except KeyboardInterrupt: pass diff --git a/Lib/pydoc.py b/Lib/pydoc.py index cf38630..29c6cc4 100755 --- a/Lib/pydoc.py +++ b/Lib/pydoc.py @@ -318,6 +318,8 @@ class Doc: # identifies something in a way that pydoc itself has issues handling; # think 'super' and how it is a descriptor (which raises the exception # by lacking a __name__ attribute) and an instance. + if inspect.isgetsetdescriptor(object): return self.docdata(*args) + if inspect.ismemberdescriptor(object): return self.docdata(*args) try: if inspect.ismodule(object): return self.docmodule(*args) if inspect.isclass(object): return self.docclass(*args) @@ -333,7 +335,7 @@ class Doc: name and ' ' + repr(name), type(object).__name__) raise TypeError, message - docmodule = docclass = docroutine = docother = fail + docmodule = docclass = docroutine = docother = docproperty = docdata = fail def getdocloc(self, object): """Return the location of module docs or None""" @@ -915,6 +917,10 @@ class HTMLDoc(Doc): lhs = name and '%s = ' % name or '' return lhs + self.repr(object) + def docdata(self, object, name=None, mod=None, cl=None): + """Produce html documentation for a data descriptor.""" + return self._docdescriptor(name, object, mod) + def index(self, dir, shadowed=None): """Generate an HTML index for a directory of modules.""" modpkgs = [] @@ -1268,6 +1274,10 @@ class TextDoc(Doc): """Produce text documentation for a property.""" return self._docdescriptor(name, object, mod) + def docdata(self, object, name=None, mod=None, cl=None): + """Produce text documentation for a data descriptor.""" + return self._docdescriptor(name, object, mod) + def docother(self, object, name=None, mod=None, parent=None, maxlen=None, doc=None): """Produce text documentation for a data object.""" repr = self.repr(object) @@ -1397,6 +1407,14 @@ def describe(thing): return 'module ' + thing.__name__ if inspect.isbuiltin(thing): return 'built-in function ' + thing.__name__ + if inspect.isgetsetdescriptor(thing): + return 'getset descriptor %s.%s.%s' % ( + thing.__objclass__.__module__, thing.__objclass__.__name__, + thing.__name__) + if inspect.ismemberdescriptor(thing): + return 'member descriptor %s.%s.%s' % ( + thing.__objclass__.__module__, thing.__objclass__.__name__, + thing.__name__) if inspect.isclass(thing): return 'class ' + thing.__name__ if inspect.isfunction(thing): @@ -1453,6 +1471,8 @@ def doc(thing, title='Python Library Documentation: %s', forceload=0): if not (inspect.ismodule(object) or inspect.isclass(object) or inspect.isroutine(object) or + inspect.isgetsetdescriptor(object) or + inspect.ismemberdescriptor(object) or isinstance(object, property)): # If the passed object is a piece of data or an instance, # document its available methods instead of its value. diff --git a/Lib/random.py b/Lib/random.py index 465f477..ae2d434 100644 --- a/Lib/random.py +++ b/Lib/random.py @@ -29,13 +29,12 @@ General notes on the underlying Mersenne Twister core generator: * The period is 2**19937-1. -* It is one of the most extensively tested generators in existence -* Without a direct way to compute N steps forward, the - semantics of jumpahead(n) are weakened to simply jump - to another distant state and rely on the large period - to avoid overlapping sequences. -* The random() method is implemented in C, executes in - a single Python step, and is, therefore, threadsafe. +* It is one of the most extensively tested generators in existence. +* Without a direct way to compute N steps forward, the semantics of + jumpahead(n) are weakened to simply jump to another distant state and rely + on the large period to avoid overlapping sequences. +* The random() method is implemented in C, executes in a single Python step, + and is, therefore, threadsafe. """ @@ -253,11 +252,6 @@ class Random(_random.Random): Optional arg random is a 0-argument function returning a random float in [0.0, 1.0); by default, the standard random.random. - - Note that for even rather small len(x), the total number of - permutations of x is larger than the period of most random number - generators; this implies that "most" permutations of a long - sequence can never be generated. """ if random is None: diff --git a/Lib/sgmllib.py b/Lib/sgmllib.py index 3e85a91..3020d11 100644 --- a/Lib/sgmllib.py +++ b/Lib/sgmllib.py @@ -29,11 +29,16 @@ starttagopen = re.compile('<[>a-zA-Z]') shorttagopen = re.compile('<[a-zA-Z][-.a-zA-Z0-9]*/') shorttag = re.compile('<([a-zA-Z][-.a-zA-Z0-9]*)/([^/]*)/') piclose = re.compile('>') -endbracket = re.compile('[<>]') +starttag = re.compile(r'<[a-zA-Z][-_.:a-zA-Z0-9]*\s*(' + r'\s*([a-zA-Z_][-:.a-zA-Z_0-9]*)(\s*=\s*' + r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~@]' + r'[][\-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~\'"@]*(?=[\s>/<])))?' + r')*\s*/?\s*(?=[<>])') +endtag = re.compile(r'])') tagfind = re.compile('[a-zA-Z][-_.a-zA-Z0-9]*') attrfind = re.compile( r'\s*([a-zA-Z_][-:.a-zA-Z_0-9]*)(\s*=\s*' - r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~\'"@]*))?') + r'(\'[^\']*\'|"[^"]*"|[][\-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~\'"@]*))?') class SGMLParseError(RuntimeError): @@ -53,6 +58,10 @@ class SGMLParseError(RuntimeError): # self.handle_entityref() with the entity reference as argument. class SGMLParser(markupbase.ParserBase): + # Definition of entities -- derived classes may override + entity_or_charref = re.compile('&(?:' + '([a-zA-Z][-.a-zA-Z0-9]*)|#([0-9]+)' + ')(;?)') def __init__(self, verbose=0): """Initialize and reset this instance.""" @@ -245,11 +254,10 @@ class SGMLParser(markupbase.ParserBase): self.finish_shorttag(tag, data) self.__starttag_text = rawdata[start_pos:match.end(1) + 1] return k - # XXX The following should skip matching quotes (' or ") - match = endbracket.search(rawdata, i+1) + match = starttag.match(rawdata, i) if not match: return -1 - j = match.start(0) + j = match.end(0) # Now parse the data between i+1 and j into a tag and attrs attrs = [] if rawdata[i:i+2] == '<>': @@ -274,32 +282,8 @@ class SGMLParser(markupbase.ParserBase): attrvalue[:1] == '"' == attrvalue[-1:]): # strip quotes attrvalue = attrvalue[1:-1] - l = 0 - new_attrvalue = '' - while l < len(attrvalue): - av_match = entityref.match(attrvalue, l) - if (av_match and av_match.group(1) in self.entitydefs and - attrvalue[av_match.end(1)] == ';'): - # only substitute entityrefs ending in ';' since - # otherwise we may break - # which is very common - new_attrvalue += self.entitydefs[av_match.group(1)] - l = av_match.end(0) - continue - ch_match = charref.match(attrvalue, l) - if ch_match: - try: - char = chr(int(ch_match.group(1))) - new_attrvalue += char - l = ch_match.end(0) - continue - except ValueError: - # invalid character reference, don't substitute - pass - # all other cases - new_attrvalue += attrvalue[l] - l += 1 - attrvalue = new_attrvalue + attrvalue = self.entity_or_charref.sub( + self._convert_ref, attrvalue) attrs.append((attrname.lower(), attrvalue)) k = match.end(0) if rawdata[j] == '>': @@ -308,13 +292,24 @@ class SGMLParser(markupbase.ParserBase): self.finish_starttag(tag, attrs) return j + # Internal -- convert entity or character reference + def _convert_ref(self, match): + if match.group(2): + return self.convert_charref(match.group(2)) or \ + '&#%s%s' % match.groups()[1:] + elif match.group(3): + return self.convert_entityref(match.group(1)) or \ + '&%s;' % match.group(1) + else: + return '&%s' % match.group(1) + # Internal -- parse endtag def parse_endtag(self, i): rawdata = self.rawdata - match = endbracket.search(rawdata, i+1) + match = endtag.match(rawdata, i) if not match: return -1 - j = match.start(0) + j = match.end(0) tag = rawdata[i+2:j].strip().lower() if rawdata[j] == '>': j = j+1 @@ -391,35 +386,51 @@ class SGMLParser(markupbase.ParserBase): print '*** Unbalanced ' print '*** Stack:', self.stack - def handle_charref(self, name): - """Handle character reference, no need to override.""" + def convert_charref(self, name): + """Convert character reference, may be overridden.""" try: n = int(name) except ValueError: - self.unknown_charref(name) return if not 0 <= n <= 255: - self.unknown_charref(name) return - self.handle_data(chr(n)) + return self.convert_codepoint(n) + + def convert_codepoint(self, codepoint): + return chr(codepoint) + + def handle_charref(self, name): + """Handle character reference, no need to override.""" + replacement = self.convert_charref(name) + if replacement is None: + self.unknown_charref(name) + else: + self.handle_data(replacement) # Definition of entities -- derived classes may override entitydefs = \ {'lt': '<', 'gt': '>', 'amp': '&', 'quot': '"', 'apos': '\''} - def handle_entityref(self, name): - """Handle entity references. + def convert_entityref(self, name): + """Convert entity references. - There should be no need to override this method; it can be - tailored by setting up the self.entitydefs mapping appropriately. + As an alternative to overriding this method; one can tailor the + results by setting up the self.entitydefs mapping appropriately. """ table = self.entitydefs if name in table: - self.handle_data(table[name]) + return table[name] else: - self.unknown_entityref(name) return + def handle_entityref(self, name): + """Handle entity references, no need to override.""" + replacement = self.convert_entityref(name) + if replacement is None: + self.unknown_entityref(name) + else: + self.handle_data(self.convert_entityref(name)) + # Example -- handle data, should be overridden def handle_data(self, data): pass diff --git a/Lib/shelve.py b/Lib/shelve.py index 4959c26..7a75445 100644 --- a/Lib/shelve.py +++ b/Lib/shelve.py @@ -139,6 +139,9 @@ class Shelf(UserDict.DictMixin): self.dict = 0 def __del__(self): + if not hasattr(self, 'writeback'): + # __init__ didn't succeed, so don't bother closing + return self.close() def sync(self): diff --git a/Lib/shutil.py b/Lib/shutil.py index c50184c..c3ff687 100644 --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -127,7 +127,13 @@ def copytree(src, dst, symlinks=False): # continue with other files except Error, err: errors.extend(err.args[0]) - copystat(src, dst) + try: + copystat(src, dst) + except WindowsError: + # can't copy file access times on Windows + pass + except OSError, why: + errors.extend((src, dst, str(why))) if errors: raise Error, errors diff --git a/Lib/site.py b/Lib/site.py index 47eda24..01086b7 100644 --- a/Lib/site.py +++ b/Lib/site.py @@ -11,10 +11,11 @@ import, this is no longer necessary (but code that does it still works). This will append site-specific paths to the module search path. On -Unix, it starts with sys.prefix and sys.exec_prefix (if different) and -appends lib/python/site-packages as well as lib/site-python. -On other platforms (mainly Mac and Windows), it uses just sys.prefix -(and sys.exec_prefix, if different, but this is unlikely). The +Unix (including Mac OSX), it starts with sys.prefix and +sys.exec_prefix (if different) and appends +lib/python/site-packages as well as lib/site-python. +On other platforms (such as Windows), it tries each of the +prefixes directly, as well as with lib/site-packages appended. The resulting directories, if they exist, are appended to sys.path, and also inspected for path configuration files. diff --git a/Lib/socket.py b/Lib/socket.py index fa0e663..52fb8e3 100644 --- a/Lib/socket.py +++ b/Lib/socket.py @@ -130,35 +130,40 @@ _socketmethods = ( if sys.platform == "riscos": _socketmethods = _socketmethods + ('sleeptaskw',) +# All the method names that must be delegated to either the real socket +# object or the _closedsocket object. +_delegate_methods = ("recv", "recvfrom", "recv_into", "recvfrom_into", + "send", "sendto") + class _closedsocket(object): __slots__ = [] def _dummy(*args): raise error(EBADF, 'Bad file descriptor') - send = recv = sendto = recvfrom = __getattr__ = _dummy + def close(self): + pass + # All _delegate_methods must also be initialized here. + send = recv = recv_into = sendto = recvfrom = recvfrom_into = _dummy + __getattr__ = _dummy class _socketobject(object): __doc__ = _realsocket.__doc__ - __slots__ = ["_sock", - "recv", "recv_into", "recvfrom_into", - "send", "sendto", "recvfrom", - "__weakref__"] + __slots__ = ["_sock", "__weakref__"] + list(_delegate_methods) def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0, _sock=None): if _sock is None: _sock = _realsocket(family, type, proto) self._sock = _sock - self.send = self._sock.send - self.recv = self._sock.recv - self.recv_into = self._sock.recv_into - self.sendto = self._sock.sendto - self.recvfrom = self._sock.recvfrom - self.recvfrom_into = self._sock.recvfrom_into + for method in _delegate_methods: + setattr(self, method, getattr(_sock, method)) def close(self): + self._sock.close() self._sock = _closedsocket() - self.send = self.recv = self.sendto = self.recvfrom = self._sock._dummy + dummy = self._sock._dummy + for method in _delegate_methods: + setattr(self, method, dummy) close.__doc__ = _realsocket.close.__doc__ def accept(self): diff --git a/Lib/sqlite3/test/hooks.py b/Lib/sqlite3/test/hooks.py index b10b3ef..761bdaa 100644 --- a/Lib/sqlite3/test/hooks.py +++ b/Lib/sqlite3/test/hooks.py @@ -48,6 +48,8 @@ class CollationTests(unittest.TestCase): pass def CheckCollationIsUsed(self): + if sqlite.version_info < (3, 2, 1): # old SQLite versions crash on this test + return def mycoll(x, y): # reverse order return -cmp(x, y) diff --git a/Lib/sqlite3/test/regression.py b/Lib/sqlite3/test/regression.py index 25e4b63..c8733b9 100644 --- a/Lib/sqlite3/test/regression.py +++ b/Lib/sqlite3/test/regression.py @@ -61,6 +61,14 @@ class RegressionTests(unittest.TestCase): con.rollback() + def CheckColumnNameWithSpaces(self): + cur = self.con.cursor() + cur.execute('select 1 as "foo bar [datetime]"') + self.failUnlessEqual(cur.description[0][0], "foo bar") + + cur.execute('select 1 as "foo baz"') + self.failUnlessEqual(cur.description[0][0], "foo baz") + def suite(): regression_suite = unittest.makeSuite(RegressionTests, "Check") return unittest.TestSuite((regression_suite,)) diff --git a/Lib/sqlite3/test/types.py b/Lib/sqlite3/test/types.py index e49f7dd..8da5722 100644 --- a/Lib/sqlite3/test/types.py +++ b/Lib/sqlite3/test/types.py @@ -21,7 +21,7 @@ # misrepresented as being the original software. # 3. This notice may not be removed or altered from any source distribution. -import datetime +import bz2, datetime import unittest import sqlite3 as sqlite @@ -101,16 +101,16 @@ class DeclTypesTests(unittest.TestCase): self.cur.execute("create table test(i int, s str, f float, b bool, u unicode, foo foo, bin blob)") # override float, make them always return the same number - sqlite.converters["float"] = lambda x: 47.2 + sqlite.converters["FLOAT"] = lambda x: 47.2 # and implement two custom ones - sqlite.converters["bool"] = lambda x: bool(int(x)) - sqlite.converters["foo"] = DeclTypesTests.Foo + sqlite.converters["BOOL"] = lambda x: bool(int(x)) + sqlite.converters["FOO"] = DeclTypesTests.Foo def tearDown(self): - del sqlite.converters["float"] - del sqlite.converters["bool"] - del sqlite.converters["foo"] + del sqlite.converters["FLOAT"] + del sqlite.converters["BOOL"] + del sqlite.converters["FOO"] self.cur.close() self.con.close() @@ -208,14 +208,14 @@ class ColNamesTests(unittest.TestCase): self.cur = self.con.cursor() self.cur.execute("create table test(x foo)") - sqlite.converters["foo"] = lambda x: "[%s]" % x - sqlite.converters["bar"] = lambda x: "<%s>" % x - sqlite.converters["exc"] = lambda x: 5/0 + sqlite.converters["FOO"] = lambda x: "[%s]" % x + sqlite.converters["BAR"] = lambda x: "<%s>" % x + sqlite.converters["EXC"] = lambda x: 5/0 def tearDown(self): - del sqlite.converters["foo"] - del sqlite.converters["bar"] - del sqlite.converters["exc"] + del sqlite.converters["FOO"] + del sqlite.converters["BAR"] + del sqlite.converters["EXC"] self.cur.close() self.con.close() @@ -231,12 +231,6 @@ class ColNamesTests(unittest.TestCase): val = self.cur.fetchone()[0] self.failUnlessEqual(val, None) - def CheckExc(self): - # Exceptions in type converters result in returned Nones - self.cur.execute('select 5 as "x [exc]"') - val = self.cur.fetchone()[0] - self.failUnlessEqual(val, None) - def CheckColName(self): self.cur.execute("insert into test(x) values (?)", ("xxx",)) self.cur.execute('select x as "x [bar]" from test') @@ -279,6 +273,23 @@ class ObjectAdaptationTests(unittest.TestCase): val = self.cur.fetchone()[0] self.failUnlessEqual(type(val), float) +class BinaryConverterTests(unittest.TestCase): + def convert(s): + return bz2.decompress(s) + convert = staticmethod(convert) + + def setUp(self): + self.con = sqlite.connect(":memory:", detect_types=sqlite.PARSE_COLNAMES) + sqlite.register_converter("bin", BinaryConverterTests.convert) + + def tearDown(self): + self.con.close() + + def CheckBinaryInputForConverter(self): + testdata = "abcdefg" * 10 + result = self.con.execute('select ? as "x [bin]"', (buffer(bz2.compress(testdata)),)).fetchone()[0] + self.failUnlessEqual(testdata, result) + class DateTimeTests(unittest.TestCase): def setUp(self): self.con = sqlite.connect(":memory:", detect_types=sqlite.PARSE_DECLTYPES) @@ -328,8 +339,9 @@ def suite(): decltypes_type_suite = unittest.makeSuite(DeclTypesTests, "Check") colnames_type_suite = unittest.makeSuite(ColNamesTests, "Check") adaptation_suite = unittest.makeSuite(ObjectAdaptationTests, "Check") + bin_suite = unittest.makeSuite(BinaryConverterTests, "Check") date_suite = unittest.makeSuite(DateTimeTests, "Check") - return unittest.TestSuite((sqlite_type_suite, decltypes_type_suite, colnames_type_suite, adaptation_suite, date_suite)) + return unittest.TestSuite((sqlite_type_suite, decltypes_type_suite, colnames_type_suite, adaptation_suite, bin_suite, date_suite)) def test(): runner = unittest.TextTestRunner() diff --git a/Lib/sqlite3/test/userfunctions.py b/Lib/sqlite3/test/userfunctions.py index 78656e7..31bf289 100644 --- a/Lib/sqlite3/test/userfunctions.py +++ b/Lib/sqlite3/test/userfunctions.py @@ -55,6 +55,9 @@ class AggrNoStep: def __init__(self): pass + def finalize(self): + return 1 + class AggrNoFinalize: def __init__(self): pass @@ -144,9 +147,12 @@ class FunctionTests(unittest.TestCase): def CheckFuncRefCount(self): def getfunc(): def f(): - return val + return 1 return f - self.con.create_function("reftest", 0, getfunc()) + f = getfunc() + globals()["foo"] = f + # self.con.create_function("reftest", 0, getfunc()) + self.con.create_function("reftest", 0, f) cur = self.con.cursor() cur.execute("select reftest()") @@ -195,9 +201,12 @@ class FunctionTests(unittest.TestCase): def CheckFuncException(self): cur = self.con.cursor() - cur.execute("select raiseexception()") - val = cur.fetchone()[0] - self.failUnlessEqual(val, None) + try: + cur.execute("select raiseexception()") + cur.fetchone() + self.fail("should have raised OperationalError") + except sqlite.OperationalError, e: + self.failUnlessEqual(e.args[0], 'user-defined function raised exception') def CheckParamString(self): cur = self.con.cursor() @@ -267,31 +276,47 @@ class AggregateTests(unittest.TestCase): def CheckAggrNoStep(self): cur = self.con.cursor() - cur.execute("select nostep(t) from test") + try: + cur.execute("select nostep(t) from test") + self.fail("should have raised an AttributeError") + except AttributeError, e: + self.failUnlessEqual(e.args[0], "AggrNoStep instance has no attribute 'step'") def CheckAggrNoFinalize(self): cur = self.con.cursor() - cur.execute("select nofinalize(t) from test") - val = cur.fetchone()[0] - self.failUnlessEqual(val, None) + try: + cur.execute("select nofinalize(t) from test") + val = cur.fetchone()[0] + self.fail("should have raised an OperationalError") + except sqlite.OperationalError, e: + self.failUnlessEqual(e.args[0], "user-defined aggregate's 'finalize' method raised error") def CheckAggrExceptionInInit(self): cur = self.con.cursor() - cur.execute("select excInit(t) from test") - val = cur.fetchone()[0] - self.failUnlessEqual(val, None) + try: + cur.execute("select excInit(t) from test") + val = cur.fetchone()[0] + self.fail("should have raised an OperationalError") + except sqlite.OperationalError, e: + self.failUnlessEqual(e.args[0], "user-defined aggregate's '__init__' method raised error") def CheckAggrExceptionInStep(self): cur = self.con.cursor() - cur.execute("select excStep(t) from test") - val = cur.fetchone()[0] - self.failUnlessEqual(val, 42) + try: + cur.execute("select excStep(t) from test") + val = cur.fetchone()[0] + self.fail("should have raised an OperationalError") + except sqlite.OperationalError, e: + self.failUnlessEqual(e.args[0], "user-defined aggregate's 'step' method raised error") def CheckAggrExceptionInFinalize(self): cur = self.con.cursor() - cur.execute("select excFinalize(t) from test") - val = cur.fetchone()[0] - self.failUnlessEqual(val, None) + try: + cur.execute("select excFinalize(t) from test") + val = cur.fetchone()[0] + self.fail("should have raised an OperationalError") + except sqlite.OperationalError, e: + self.failUnlessEqual(e.args[0], "user-defined aggregate's 'finalize' method raised error") def CheckAggrCheckParamStr(self): cur = self.con.cursor() @@ -331,10 +356,54 @@ class AggregateTests(unittest.TestCase): val = cur.fetchone()[0] self.failUnlessEqual(val, 60) +def authorizer_cb(action, arg1, arg2, dbname, source): + if action != sqlite.SQLITE_SELECT: + return sqlite.SQLITE_DENY + if arg2 == 'c2' or arg1 == 't2': + return sqlite.SQLITE_DENY + return sqlite.SQLITE_OK + +class AuthorizerTests(unittest.TestCase): + def setUp(self): + self.con = sqlite.connect(":memory:") + self.con.executescript(""" + create table t1 (c1, c2); + create table t2 (c1, c2); + insert into t1 (c1, c2) values (1, 2); + insert into t2 (c1, c2) values (4, 5); + """) + + # For our security test: + self.con.execute("select c2 from t2") + + self.con.set_authorizer(authorizer_cb) + + def tearDown(self): + pass + + def CheckTableAccess(self): + try: + self.con.execute("select * from t2") + except sqlite.DatabaseError, e: + if not e.args[0].endswith("prohibited"): + self.fail("wrong exception text: %s" % e.args[0]) + return + self.fail("should have raised an exception due to missing privileges") + + def CheckColumnAccess(self): + try: + self.con.execute("select c2 from t1") + except sqlite.DatabaseError, e: + if not e.args[0].endswith("prohibited"): + self.fail("wrong exception text: %s" % e.args[0]) + return + self.fail("should have raised an exception due to missing privileges") + def suite(): function_suite = unittest.makeSuite(FunctionTests, "Check") aggregate_suite = unittest.makeSuite(AggregateTests, "Check") - return unittest.TestSuite((function_suite, aggregate_suite)) + authorizer_suite = unittest.makeSuite(AuthorizerTests, "Check") + return unittest.TestSuite((function_suite, aggregate_suite, authorizer_suite)) def test(): runner = unittest.TextTestRunner() diff --git a/Lib/string.py b/Lib/string.py index ba85a49..a5837e9 100644 --- a/Lib/string.py +++ b/Lib/string.py @@ -161,7 +161,7 @@ class Template: val = mapping[named] # We use this idiom instead of str() because the latter will # fail if val is a Unicode containing non-ASCII characters. - return '%s' % val + return '%s' % (val,) if mo.group('escaped') is not None: return self.delimiter if mo.group('invalid') is not None: @@ -186,13 +186,13 @@ class Template: try: # We use this idiom instead of str() because the latter # will fail if val is a Unicode containing non-ASCII - return '%s' % mapping[named] + return '%s' % (mapping[named],) except KeyError: return self.delimiter + named braced = mo.group('braced') if braced is not None: try: - return '%s' % mapping[braced] + return '%s' % (mapping[braced],) except KeyError: return self.delimiter + '{' + braced + '}' if mo.group('escaped') is not None: diff --git a/Lib/struct.py b/Lib/struct.py index 9113e71..07c21bf 100644 --- a/Lib/struct.py +++ b/Lib/struct.py @@ -64,7 +64,7 @@ def pack(fmt, *args): def pack_into(fmt, buf, offset, *args): """ - Pack the values v2, v2, ... according to fmt, write + Pack the values v1, v2, ... according to fmt, write the packed bytes into the writable buffer buf starting at offset. See struct.__doc__ for more on format strings. """ diff --git a/Lib/subprocess.py b/Lib/subprocess.py index a6af7e7..0d19129 100644 --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -121,7 +121,7 @@ check_call(*popenargs, **kwargs): Run command with arguments. Wait for command to complete. If the exit code was zero then return, otherwise raise CalledProcessError. The CalledProcessError object will have the - return code in the errno attribute. + return code in the returncode attribute. The arguments are the same as for the Popen constructor. Example: @@ -141,8 +141,8 @@ should prepare for OSErrors. A ValueError will be raised if Popen is called with invalid arguments. -check_call() will raise CalledProcessError, which is a subclass of -OSError, if the called process returns a non-zero return code. +check_call() will raise CalledProcessError, if the called process +returns a non-zero return code. Security @@ -234,7 +234,7 @@ Replacing os.system() sts = os.system("mycmd" + " myarg") ==> p = Popen("mycmd" + " myarg", shell=True) -sts = os.waitpid(p.pid, 0) +pid, sts = os.waitpid(p.pid, 0) Note: @@ -360,11 +360,16 @@ import types import traceback # Exception classes used by this module. -class CalledProcessError(OSError): +class CalledProcessError(Exception): """This exception is raised when a process run by check_call() returns a non-zero exit status. The exit status will be stored in the - errno attribute. This exception is a subclass of - OSError.""" + returncode attribute.""" + def __init__(self, returncode, cmd): + self.returncode = returncode + self.cmd = cmd + def __str__(self): + return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode) + if mswindows: import threading @@ -442,7 +447,7 @@ def check_call(*popenargs, **kwargs): """Run command with arguments. Wait for command to complete. If the exit code was zero then return, otherwise raise CalledProcessError. The CalledProcessError object will have the - return code in the errno attribute. + return code in the returncode attribute. The arguments are the same as for the Popen constructor. Example: @@ -453,7 +458,7 @@ def check_call(*popenargs, **kwargs): if cmd is None: cmd = popenargs[0] if retcode: - raise CalledProcessError(retcode, "Command %s returned non-zero exit status" % cmd) + raise CalledProcessError(retcode, cmd) return retcode @@ -613,7 +618,7 @@ class Popen(object): return # In case the child hasn't been waited on, check if it's done. self.poll(_deadstate=sys.maxint) - if self.returncode is None: + if self.returncode is None and _active is not None: # Child is still running, keep us alive until we can wait on it. _active.append(self) @@ -941,7 +946,7 @@ class Popen(object): def _close_fds(self, but): - for i in range(3, MAXFD): + for i in xrange(3, MAXFD): if i == but: continue try: diff --git a/Lib/tarfile.py b/Lib/tarfile.py index 061d0f5..c185fbd 100644 --- a/Lib/tarfile.py +++ b/Lib/tarfile.py @@ -417,7 +417,13 @@ class _Stream: self.fileobj.write(self.buf) self.buf = "" if self.comptype == "gz": - self.fileobj.write(struct.pack("= timeout: + break + s_args = s_reply + (timeout-elapsed,) return self.read_very_lazy() def read_all(self): @@ -601,6 +608,9 @@ class Telnet: if not hasattr(list[i], "search"): if not re: import re list[i] = re.compile(list[i]) + if timeout is not None: + from time import time + time_start = time() while 1: self.process_rawq() for i in indices: @@ -613,7 +623,11 @@ class Telnet: if self.eof: break if timeout is not None: - r, w, x = select.select([self.fileno()], [], [], timeout) + elapsed = time() - time_start + if elapsed >= timeout: + break + s_args = ([self.fileno()], [], [], timeout-elapsed) + r, w, x = select.select(*s_args) if not r: break self.fill_rawq() diff --git a/Lib/tempfile.py b/Lib/tempfile.py index dd7e864..2e8cd6d 100644 --- a/Lib/tempfile.py +++ b/Lib/tempfile.py @@ -446,7 +446,7 @@ else: prefix=template, dir=None): """Create and return a temporary file. Arguments: - 'prefix', 'suffix', 'directory' -- as for mkstemp. + 'prefix', 'suffix', 'dir' -- as for mkstemp. 'mode' -- the mode argument to os.fdopen (default "w+b"). 'bufsize' -- the buffer size argument to os.fdopen (default -1). The file is created as mkstemp() would do it. diff --git a/Lib/test/crashers/bogus_code_obj.py b/Lib/test/crashers/bogus_code_obj.py new file mode 100644 index 0000000..613ae51 --- /dev/null +++ b/Lib/test/crashers/bogus_code_obj.py @@ -0,0 +1,19 @@ +""" +Broken bytecode objects can easily crash the interpreter. + +This is not going to be fixed. It is generally agreed that there is no +point in writing a bytecode verifier and putting it in CPython just for +this. Moreover, a verifier is bound to accept only a subset of all safe +bytecodes, so it could lead to unnecessary breakage. + +For security purposes, "restricted" interpreters are not going to let +the user build or load random bytecodes anyway. Otherwise, this is a +"won't fix" case. + +""" + +import types + +co = types.CodeType(0, 0, 0, 0, '\x04\x71\x00\x00', (), + (), (), '', '', 1, '') +exec co diff --git a/Lib/test/crashers/borrowed_ref_1.py b/Lib/test/crashers/borrowed_ref_1.py new file mode 100644 index 0000000..d16ede2 --- /dev/null +++ b/Lib/test/crashers/borrowed_ref_1.py @@ -0,0 +1,29 @@ +""" +_PyType_Lookup() returns a borrowed reference. +This attacks the call in dictobject.c. +""" + +class A(object): + pass + +class B(object): + def __del__(self): + print 'hi' + del D.__missing__ + +class D(dict): + class __missing__: + def __init__(self, *args): + pass + + +d = D() +a = A() +a.cycle = a +a.other = B() +del a + +prev = None +while 1: + d[5] + prev = (prev,) diff --git a/Lib/test/crashers/borrowed_ref_2.py b/Lib/test/crashers/borrowed_ref_2.py new file mode 100644 index 0000000..1a7b3ff --- /dev/null +++ b/Lib/test/crashers/borrowed_ref_2.py @@ -0,0 +1,38 @@ +""" +_PyType_Lookup() returns a borrowed reference. +This attacks PyObject_GenericSetAttr(). + +NB. on my machine this crashes in 2.5 debug but not release. +""" + +class A(object): + pass + +class B(object): + def __del__(self): + print "hi" + del C.d + +class D(object): + def __set__(self, obj, value): + self.hello = 42 + +class C(object): + d = D() + + def g(): + pass + + +c = C() +a = A() +a.cycle = a +a.other = B() + +lst = [None] * 1000000 +i = 0 +del a +while 1: + c.d = 42 # segfaults in PyMethod_New(im_func=D.__set__, im_self=d) + lst[i] = c.g # consume the free list of instancemethod objects + i += 1 diff --git a/Lib/test/crashers/coerce.py b/Lib/test/crashers/coerce.py deleted file mode 100644 index 574956b..0000000 --- a/Lib/test/crashers/coerce.py +++ /dev/null @@ -1,9 +0,0 @@ - -# http://python.org/sf/992017 - -class foo: - def __coerce__(self, other): - return other, self - -if __name__ == '__main__': - foo()+1 # segfault: infinite recursion in C diff --git a/Lib/test/crashers/gc_inspection.py b/Lib/test/crashers/gc_inspection.py new file mode 100644 index 0000000..10caa79 --- /dev/null +++ b/Lib/test/crashers/gc_inspection.py @@ -0,0 +1,32 @@ +""" +gc.get_referrers() can be used to see objects before they are fully built. + +Note that this is only an example. There are many ways to crash Python +by using gc.get_referrers(), as well as many extension modules (even +when they are using perfectly documented patterns to build objects). + +Identifying and removing all places that expose to the GC a +partially-built object is a long-term project. A patch was proposed on +SF specifically for this example but I consider fixing just this single +example a bit pointless (#1517042). + +A fix would include a whole-scale code review, possibly with an API +change to decouple object creation and GC registration, and according +fixes to the documentation for extension module writers. It's unlikely +to happen, though. So this is currently classified as +"gc.get_referrers() is dangerous, use only for debugging". +""" + +import gc + + +def g(): + marker = object() + yield marker + # now the marker is in the tuple being constructed + [tup] = [x for x in gc.get_referrers(marker) if type(x) is tuple] + print tup + print tup[1] + + +tuple(g()) diff --git a/Lib/test/crashers/infinite_rec_3.py b/Lib/test/crashers/infinite_rec_3.py deleted file mode 100644 index 0b04e4c..0000000 --- a/Lib/test/crashers/infinite_rec_3.py +++ /dev/null @@ -1,9 +0,0 @@ - -# http://python.org/sf/1202533 - -class A(object): - pass -A.__call__ = A() - -if __name__ == '__main__': - A()() # segfault: infinite recursion in C diff --git a/Lib/test/crashers/recursion_limit_too_high.py b/Lib/test/crashers/recursion_limit_too_high.py new file mode 100644 index 0000000..1fa4d32 --- /dev/null +++ b/Lib/test/crashers/recursion_limit_too_high.py @@ -0,0 +1,16 @@ +# The following example may crash or not depending on the platform. +# E.g. on 32-bit Intel Linux in a "standard" configuration it seems to +# crash on Python 2.5 (but not 2.4 nor 2.3). On Windows the import +# eventually fails to find the module, possibly because we run out of +# file handles. + +# The point of this example is to show that sys.setrecursionlimit() is a +# hack, and not a robust solution. This example simply exercices a path +# where it takes many C-level recursions, consuming a lot of stack +# space, for each Python-level recursion. So 1000 times this amount of +# stack space may be too much for standard platforms already. + +import sys +if 'recursion_limit_too_high' in sys.modules: + del sys.modules['recursion_limit_too_high'] +import recursion_limit_too_high diff --git a/Lib/test/crashers/recursive_call.py b/Lib/test/crashers/recursive_call.py index 0776479..31c8963 100644 --- a/Lib/test/crashers/recursive_call.py +++ b/Lib/test/crashers/recursive_call.py @@ -1,6 +1,11 @@ #!/usr/bin/env python # No bug report AFAIK, mail on python-dev on 2006-01-10 + +# This is a "won't fix" case. It is known that setting a high enough +# recursion limit crashes by overflowing the stack. Unless this is +# redesigned somehow, it won't go away. + import sys sys.setrecursionlimit(1 << 30) diff --git a/Lib/test/crashers/xml_parsers.py b/Lib/test/crashers/xml_parsers.py deleted file mode 100644 index e6b5727..0000000 --- a/Lib/test/crashers/xml_parsers.py +++ /dev/null @@ -1,56 +0,0 @@ -from xml.parsers import expat - -# http://python.org/sf/1296433 - -def test_parse_only_xml_data(): - # - xml = "%s" % ('a' * 1025) - # this one doesn't crash - #xml = "%s" % ('a' * 10000) - - def handler(text): - raise Exception - - parser = expat.ParserCreate() - parser.CharacterDataHandler = handler - - try: - parser.Parse(xml) - except: - pass - -if __name__ == '__main__': - test_parse_only_xml_data() - -# Invalid read of size 4 -# at 0x43F936: PyObject_Free (obmalloc.c:735) -# by 0x45A7C7: unicode_dealloc (unicodeobject.c:246) -# by 0x1299021D: PyUnknownEncodingHandler (pyexpat.c:1314) -# by 0x12993A66: processXmlDecl (xmlparse.c:3330) -# by 0x12999211: doProlog (xmlparse.c:3678) -# by 0x1299C3F0: prologInitProcessor (xmlparse.c:3550) -# by 0x12991EA3: XML_ParseBuffer (xmlparse.c:1562) -# by 0x1298F8EC: xmlparse_Parse (pyexpat.c:895) -# by 0x47B3A1: PyEval_EvalFrameEx (ceval.c:3565) -# by 0x47CCAC: PyEval_EvalCodeEx (ceval.c:2739) -# by 0x47CDE1: PyEval_EvalCode (ceval.c:490) -# by 0x499820: PyRun_SimpleFileExFlags (pythonrun.c:1198) -# by 0x4117F1: Py_Main (main.c:492) -# by 0x12476D1F: __libc_start_main (in /lib/libc-2.3.5.so) -# by 0x410DC9: (within /home/neal/build/python/svn/clean/python) -# Address 0x12704020 is 264 bytes inside a block of size 592 free'd -# at 0x11B1BA8A: free (vg_replace_malloc.c:235) -# by 0x124B5F18: (within /lib/libc-2.3.5.so) -# by 0x48DE43: find_module (import.c:1320) -# by 0x48E997: import_submodule (import.c:2249) -# by 0x48EC15: load_next (import.c:2083) -# by 0x48F091: import_module_ex (import.c:1914) -# by 0x48F385: PyImport_ImportModuleEx (import.c:1955) -# by 0x46D070: builtin___import__ (bltinmodule.c:44) -# by 0x4186CF: PyObject_Call (abstract.c:1777) -# by 0x474E9B: PyEval_CallObjectWithKeywords (ceval.c:3432) -# by 0x47928E: PyEval_EvalFrameEx (ceval.c:2038) -# by 0x47CCAC: PyEval_EvalCodeEx (ceval.c:2739) -# by 0x47CDE1: PyEval_EvalCode (ceval.c:490) -# by 0x48D0F7: PyImport_ExecCodeModuleEx (import.c:635) -# by 0x48D4F4: load_source_module (import.c:913) diff --git a/Lib/test/fork_wait.py b/Lib/test/fork_wait.py index 5600bdb..7eb55f6 100644 --- a/Lib/test/fork_wait.py +++ b/Lib/test/fork_wait.py @@ -34,7 +34,14 @@ class ForkWait(unittest.TestCase): pass def wait_impl(self, cpid): - spid, status = os.waitpid(cpid, 0) + for i in range(10): + # waitpid() shouldn't hang, but some of the buildbots seem to hang + # in the forking tests. This is an attempt to fix the problem. + spid, status = os.waitpid(cpid, os.WNOHANG) + if spid == cpid: + break + time.sleep(2 * SHORTSLEEP) + self.assertEquals(spid, cpid) self.assertEquals(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) diff --git a/Lib/test/output/test_ossaudiodev b/Lib/test/output/test_ossaudiodev index 9f55afa..f0df5d2 100644 --- a/Lib/test/output/test_ossaudiodev +++ b/Lib/test/output/test_ossaudiodev @@ -1,3 +1,2 @@ test_ossaudiodev -playing test sound file... -elapsed time: 3.1 sec +playing test sound file (expected running time: 2.93 sec) diff --git a/Lib/test/output/test_thread b/Lib/test/output/test_thread index d49651d..68c6a92 100644 --- a/Lib/test/output/test_thread +++ b/Lib/test/output/test_thread @@ -4,3 +4,15 @@ all tasks done *** Barrier Test *** all tasks done + +*** Changing thread stack size *** +caught expected ValueError setting stack_size(4096) +successfully set stack_size(262144) +successfully set stack_size(1048576) +successfully set stack_size(0) +trying stack_size = 262144 +waiting for all tasks to complete +all tasks done +trying stack_size = 1048576 +waiting for all tasks to complete +all tasks done diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py index ca4a3b5..4553838 100755 --- a/Lib/test/regrtest.py +++ b/Lib/test/regrtest.py @@ -66,7 +66,9 @@ reports are written to. These parameters all have defaults (5, 4 and -M runs tests that require an exorbitant amount of memory. These tests typically try to ascertain containers keep working when containing more than -2 bilion objects, and only work on 64-bit systems. The passed-in memlimit, +2 billion objects, which only works on 64-bit systems. There are also some +tests that try to exhaust the address space of the process, which only makes +sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit, which is a string in the form of '2.5Gb', determines howmuch memory the tests will limit themselves to (but they may go slightly over.) The number shouldn't be more memory than the machine has (including swap memory). You @@ -496,14 +498,30 @@ def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS): def runtest(test, generate, verbose, quiet, testdir=None, huntrleaks=False): """Run a single test. + test -- the name of the test generate -- if true, generate output, instead of running the test - and comparing it to a previously created output file + and comparing it to a previously created output file verbose -- if true, print more messages quiet -- if true, don't print 'skipped' messages (probably redundant) testdir -- test directory + huntrleaks -- run multiple times to test for leaks; requires a debug + build; a triple corresponding to -R's three arguments + Return: + -2 test skipped because resource denied + -1 test skipped for some other reason + 0 test failed + 1 test passed """ + try: + return runtest_inner(test, generate, verbose, quiet, testdir, + huntrleaks) + finally: + cleanup_test_droppings(test, verbose) + +def runtest_inner(test, generate, verbose, quiet, + testdir=None, huntrleaks=False): test_support.unload(test) if not testdir: testdir = findtestdir() @@ -595,6 +613,37 @@ def runtest(test, generate, verbose, quiet, testdir=None, huntrleaks=False): sys.stdout.flush() return 0 +def cleanup_test_droppings(testname, verbose): + import shutil + + # Try to clean up junk commonly left behind. While tests shouldn't leave + # any files or directories behind, when a test fails that can be tedious + # for it to arrange. The consequences can be especially nasty on Windows, + # since if a test leaves a file open, it cannot be deleted by name (while + # there's nothing we can do about that here either, we can display the + # name of the offending test, which is a real help). + for name in (test_support.TESTFN, + "db_home", + ): + if not os.path.exists(name): + continue + + if os.path.isdir(name): + kind, nuker = "directory", shutil.rmtree + elif os.path.isfile(name): + kind, nuker = "file", os.unlink + else: + raise SystemError("os.path says %r exists but is neither " + "directory nor file" % name) + + if verbose: + print "%r left behind %s %r" % (testname, kind, name) + try: + nuker(name) + except Exception, msg: + print >> sys.stderr, ("%r left behind %s %r and it couldn't be " + "removed: %s" % (testname, kind, name, msg)) + def dash_R(the_module, test, indirect_test, huntrleaks): # This code is hackish and inelegant, but it seems to do the job. import copy_reg @@ -637,7 +686,7 @@ def dash_R(the_module, test, indirect_test, huntrleaks): def dash_R_cleanup(fs, ps, pic): import gc, copy_reg - import _strptime, linecache, warnings, dircache + import _strptime, linecache, dircache import urlparse, urllib, urllib2, mimetypes, doctest import struct, filecmp from distutils.dir_util import _path_created @@ -1227,6 +1276,37 @@ _expectations = { test_winreg test_winsound """, + 'netbsd3': + """ + test_aepack + test_al + test_applesingle + test_bsddb + test_bsddb185 + test_bsddb3 + test_cd + test_cl + test_ctypes + test_curses + test_dl + test_gdbm + test_gl + test_imgfile + test_linuxaudiodev + test_locale + test_macfs + test_macostools + test_nis + test_ossaudiodev + test_pep277 + test_sqlite + test_startfile + test_sunaudiodev + test_tcl + test_unicode_file + test_winreg + test_winsound + """, } _expectations['freebsd5'] = _expectations['freebsd4'] _expectations['freebsd6'] = _expectations['freebsd4'] diff --git a/Lib/test/string_tests.py b/Lib/test/string_tests.py index aaa2dc2..73447ad 100644 --- a/Lib/test/string_tests.py +++ b/Lib/test/string_tests.py @@ -147,8 +147,8 @@ class CommonTest(unittest.TestCase): else: r2, rem = len(i)+1, 0 if rem or r1 != r2: - self.assertEqual(rem, 0) - self.assertEqual(r1, r2) + self.assertEqual(rem, 0, '%s != 0 for %s' % (rem, i)) + self.assertEqual(r1, r2, '%s != %s for %s' % (r1, r2, i)) def test_find(self): self.checkequal(0, 'abcdefghiabc', 'find', 'abc') @@ -636,6 +636,11 @@ class CommonTest(unittest.TestCase): EQ("bobobXbobob", "bobobobXbobobob", "replace", "bobob", "bob") EQ("BOBOBOB", "BOBOBOB", "replace", "bob", "bobby") + ba = buffer('a') + bb = buffer('b') + EQ("bbc", "abc", "replace", ba, bb) + EQ("aac", "abc", "replace", bb, ba) + # self.checkequal('one@two!three!', 'one!two!three!', 'replace', '!', '@', 1) self.checkequal('onetwothree', 'one!two!three!', 'replace', '!', '') @@ -819,6 +824,21 @@ class MixinStrUnicodeUserStringTest: self.checkraises(TypeError, 'hello', 'startswith') self.checkraises(TypeError, 'hello', 'startswith', 42) + # test tuple arguments + self.checkequal(True, 'hello', 'startswith', ('he', 'ha')) + self.checkequal(False, 'hello', 'startswith', ('lo', 'llo')) + self.checkequal(True, 'hello', 'startswith', ('hellox', 'hello')) + self.checkequal(False, 'hello', 'startswith', ()) + self.checkequal(True, 'helloworld', 'startswith', ('hellowo', + 'rld', 'lowo'), 3) + self.checkequal(False, 'helloworld', 'startswith', ('hellowo', 'ello', + 'rld'), 3) + self.checkequal(True, 'hello', 'startswith', ('lo', 'he'), 0, -1) + self.checkequal(False, 'hello', 'startswith', ('he', 'hel'), 0, 1) + self.checkequal(True, 'hello', 'startswith', ('he', 'hel'), 0, 2) + + self.checkraises(TypeError, 'hello', 'startswith', (42,)) + def test_endswith(self): self.checkequal(True, 'hello', 'endswith', 'lo') self.checkequal(False, 'hello', 'endswith', 'he') @@ -853,6 +873,21 @@ class MixinStrUnicodeUserStringTest: self.checkraises(TypeError, 'hello', 'endswith') self.checkraises(TypeError, 'hello', 'endswith', 42) + # test tuple arguments + self.checkequal(False, 'hello', 'endswith', ('he', 'ha')) + self.checkequal(True, 'hello', 'endswith', ('lo', 'llo')) + self.checkequal(True, 'hello', 'endswith', ('hellox', 'hello')) + self.checkequal(False, 'hello', 'endswith', ()) + self.checkequal(True, 'helloworld', 'endswith', ('hellowo', + 'rld', 'lowo'), 3) + self.checkequal(False, 'helloworld', 'endswith', ('hellowo', 'ello', + 'rld'), 3, -1) + self.checkequal(True, 'hello', 'endswith', ('hell', 'ell'), 0, -1) + self.checkequal(False, 'hello', 'endswith', ('he', 'hel'), 0, 1) + self.checkequal(True, 'hello', 'endswith', ('he', 'hell'), 0, 4) + + self.checkraises(TypeError, 'hello', 'endswith', (42,)) + def test___contains__(self): self.checkequal(True, '', '__contains__', '') # vereq('' in '', True) self.checkequal(True, 'abc', '__contains__', '') # vereq('' in 'abc', True) @@ -872,7 +907,7 @@ class MixinStrUnicodeUserStringTest: self.checkequal(u'abc', 'abc', '__getitem__', slice(0, 1000)) self.checkequal(u'a', 'abc', '__getitem__', slice(0, 1)) self.checkequal(u'', 'abc', '__getitem__', slice(0, 0)) - # FIXME What about negative indizes? This is handled differently by [] and __getitem__(slice) + # FIXME What about negative indices? This is handled differently by [] and __getitem__(slice) self.checkraises(TypeError, 'abc', '__getitem__', 'def') @@ -908,6 +943,8 @@ class MixinStrUnicodeUserStringTest: # test.test_string.StringTest.test_join) self.checkequal('a b c d', ' ', 'join', ['a', 'b', 'c', 'd']) self.checkequal('abcd', '', 'join', ('a', 'b', 'c', 'd')) + self.checkequal('bd', '', 'join', ('', 'b', '', 'd')) + self.checkequal('ac', '', 'join', ('a', '', 'c', '')) self.checkequal('w x y z', ' ', 'join', Sequence()) self.checkequal('abc', 'a', 'join', ('abc',)) self.checkequal('z', 'a', 'join', UserList(['z'])) diff --git a/Lib/test/test__locale.py b/Lib/test/test__locale.py index 9799f89..ec59d71 100644 --- a/Lib/test/test__locale.py +++ b/Lib/test/test__locale.py @@ -113,6 +113,9 @@ class _LocaleTests(unittest.TestCase): "using eval('3.14') failed for %s" % loc) self.assertEquals(int(float('3.14') * 100), 314, "using float('3.14') failed for %s" % loc) + if localeconv()['decimal_point'] != '.': + self.assertRaises(ValueError, float, + localeconv()['decimal_point'].join(['1', '23'])) def test_main(): run_unittest(_LocaleTests) diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py index c64ad28..14fc010 100644 --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -160,7 +160,7 @@ exec_results = [ ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, None, []), [('Return', (1, 8), ('Num', (1, 15), 1))], [])]), ('Module', [('Delete', (1, 0), [('Name', (1, 4), 'v', ('Del',))])]), ('Module', [('Assign', (1, 0), [('Name', (1, 0), 'v', ('Store',))], ('Num', (1, 4), 1))]), -('Module', [('AugAssign', (1, 0), ('Name', (1, 0), 'v', ('Load',)), ('Add',), ('Num', (1, 5), 1))]), +('Module', [('AugAssign', (1, 0), ('Name', (1, 0), 'v', ('Store',)), ('Add',), ('Num', (1, 5), 1))]), ('Module', [('Print', (1, 0), ('Name', (1, 8), 'f', ('Load',)), [('Num', (1, 11), 1)], False)]), ('Module', [('For', (1, 0), ('Name', (1, 4), 'v', ('Store',)), ('Name', (1, 9), 'v', ('Load',)), [('Pass', (1, 11))], [])]), ('Module', [('While', (1, 0), ('Name', (1, 6), 'v', ('Load',)), [('Pass', (1, 8))], [])]), diff --git a/Lib/test/test_asynchat.py b/Lib/test/test_asynchat.py index f93587a..9926167 100644 --- a/Lib/test/test_asynchat.py +++ b/Lib/test/test_asynchat.py @@ -13,7 +13,8 @@ class echo_server(threading.Thread): def run(self): sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - sock.bind((HOST, PORT)) + global PORT + PORT = test_support.bind_port(sock, HOST, PORT) sock.listen(1) conn, client = sock.accept() buffer = "" diff --git a/Lib/test/test_bigaddrspace.py b/Lib/test/test_bigaddrspace.py new file mode 100644 index 0000000..8c215fe --- /dev/null +++ b/Lib/test/test_bigaddrspace.py @@ -0,0 +1,46 @@ +from test import test_support +from test.test_support import bigaddrspacetest, MAX_Py_ssize_t + +import unittest +import operator +import sys + + +class StrTest(unittest.TestCase): + + @bigaddrspacetest + def test_concat(self): + s1 = 'x' * MAX_Py_ssize_t + self.assertRaises(OverflowError, operator.add, s1, '?') + + @bigaddrspacetest + def test_optimized_concat(self): + x = 'x' * MAX_Py_ssize_t + try: + x = x + '?' # this statement uses a fast path in ceval.c + except OverflowError: + pass + else: + self.fail("should have raised OverflowError") + try: + x += '?' # this statement uses a fast path in ceval.c + except OverflowError: + pass + else: + self.fail("should have raised OverflowError") + self.assertEquals(len(x), MAX_Py_ssize_t) + + ### the following test is pending a patch + # (http://mail.python.org/pipermail/python-dev/2006-July/067774.html) + #@bigaddrspacetest + #def test_repeat(self): + # self.assertRaises(OverflowError, operator.mul, 'x', MAX_Py_ssize_t + 1) + + +def test_main(): + test_support.run_unittest(StrTest) + +if __name__ == '__main__': + if len(sys.argv) > 1: + test_support.set_memlimit(sys.argv[1]) + test_main() diff --git a/Lib/test/test_bigmem.py b/Lib/test/test_bigmem.py index 255428f..6d6c37c 100644 --- a/Lib/test/test_bigmem.py +++ b/Lib/test/test_bigmem.py @@ -28,7 +28,7 @@ import sys # - While the bigmemtest decorator speaks of 'minsize', all tests will # actually be called with a much smaller number too, in the normal # test run (5Kb currently.) This is so the tests themselves get frequent -# testing Consequently, always make all large allocations based on the +# testing. Consequently, always make all large allocations based on the # passed-in 'size', and don't rely on the size being very large. Also, # memuse-per-size should remain sane (less than a few thousand); if your # test uses more, adjust 'size' upward, instead. diff --git a/Lib/test/test_bsddb.py b/Lib/test/test_bsddb.py index 513e541..474f3da 100755 --- a/Lib/test/test_bsddb.py +++ b/Lib/test/test_bsddb.py @@ -8,7 +8,6 @@ import bsddb import dbhash # Just so we know it's imported import unittest from test import test_support -from sets import Set class TestBSDDB(unittest.TestCase): openflag = 'c' @@ -53,7 +52,7 @@ class TestBSDDB(unittest.TestCase): self.assertEqual(self.f[k], v) def assertSetEquals(self, seqn1, seqn2): - self.assertEqual(Set(seqn1), Set(seqn2)) + self.assertEqual(set(seqn1), set(seqn2)) def test_mapping_iteration_methods(self): f = self.f diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py index e6e4440..c7e4394 100644 --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -532,13 +532,24 @@ class BuiltinTest(unittest.TestCase): @run_with_locale('LC_NUMERIC', 'fr_FR', 'de_DE') def test_float_with_comma(self): # set locale to something that doesn't use '.' for the decimal point + # float must not accept the locale specific decimal point but + # it still has to accept the normal python syntac import locale if not locale.localeconv()['decimal_point'] == ',': return - self.assertEqual(float(" 3,14 "), 3.14) - self.assertEqual(float(" +3,14 "), 3.14) - self.assertEqual(float(" -3,14 "), -3.14) + self.assertEqual(float(" 3.14 "), 3.14) + self.assertEqual(float("+3.14 "), 3.14) + self.assertEqual(float("-3.14 "), -3.14) + self.assertEqual(float(".14 "), .14) + self.assertEqual(float("3. "), 3.0) + self.assertEqual(float("3.e3 "), 3000.0) + self.assertEqual(float("3.2e3 "), 3200.0) + self.assertEqual(float("2.5e-1 "), 0.25) + self.assertEqual(float("5e-1"), 0.5) + self.assertRaises(ValueError, float, " 3,14 ") + self.assertRaises(ValueError, float, " +3,14 ") + self.assertRaises(ValueError, float, " -3,14 ") self.assertRaises(ValueError, float, " 0x3.1 ") self.assertRaises(ValueError, float, " -0x3.p-1 ") self.assertEqual(float(" 25.e-1 "), 2.5) @@ -603,6 +614,19 @@ class BuiltinTest(unittest.TestCase): def f(): pass self.assertRaises(TypeError, hash, []) self.assertRaises(TypeError, hash, {}) + # Bug 1536021: Allow hash to return long objects + class X: + def __hash__(self): + return 2**100 + self.assertEquals(type(hash(X())), int) + class Y(object): + def __hash__(self): + return 2**100 + self.assertEquals(type(hash(Y())), int) + class Z(long): + def __hash__(self): + return self + self.assertEquals(hash(Z(42)), hash(42L)) def test_hex(self): self.assertEqual(hex(16), '0x10') diff --git a/Lib/test/test_bz2.py b/Lib/test/test_bz2.py index 356c2e3..f198116 100644 --- a/Lib/test/test_bz2.py +++ b/Lib/test/test_bz2.py @@ -250,7 +250,7 @@ class BZ2FileTest(BaseTest): bz2f = BZ2File(self.filename) xlines = list(bz2f.readlines()) bz2f.close() - self.assertEqual(lines, ['Test']) + self.assertEqual(xlines, ['Test']) class BZ2CompressorTest(BaseTest): @@ -344,6 +344,7 @@ def test_main(): BZ2DecompressorTest, FuncTest ) + test_support.reap_children() if __name__ == '__main__': test_main() diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py index ec860d1..5e89863 100644 --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -87,6 +87,7 @@ class CmdLineTest(unittest.TestCase): def test_main(): test.test_support.run_unittest(CmdLineTest) + test.test_support.reap_children() if __name__ == "__main__": test_main() diff --git a/Lib/test/test_code.py b/Lib/test/test_code.py index 52bc894..4e68638 100644 --- a/Lib/test/test_code.py +++ b/Lib/test/test_code.py @@ -61,6 +61,23 @@ nlocals: 1 flags: 67 consts: ('None',) +>>> def optimize_away(): +... 'doc string' +... 'not a docstring' +... 53 +... 53L + +>>> dump(optimize_away.func_code) +name: optimize_away +argcount: 0 +names: () +varnames: () +cellvars: () +freevars: () +nlocals: 0 +flags: 67 +consts: ("'doc string'", 'None') + """ def consts(t): diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py index 6ea49cc..8153979 100644 --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -1166,6 +1166,12 @@ class BasicUnicodeTest(unittest.TestCase): encoder = codecs.getencoder(encoding) self.assertRaises(TypeError, encoder) + def test_encoding_map_type_initialized(self): + from encodings import cp1140 + # This used to crash, we are only verifying there's no crash. + table_type = type(cp1140.encoding_table) + self.assertEqual(table_type, table_type) + class BasicStrTest(unittest.TestCase): def test_basics(self): s = "abc123" diff --git a/Lib/test/test_commands.py b/Lib/test/test_commands.py index 0f7d15f..b72a1b9 100644 --- a/Lib/test/test_commands.py +++ b/Lib/test/test_commands.py @@ -5,7 +5,7 @@ import unittest import os, tempfile, re -from test.test_support import TestSkipped, run_unittest +from test.test_support import TestSkipped, run_unittest, reap_children from commands import * # The module says: @@ -58,6 +58,7 @@ class CommandTests(unittest.TestCase): def test_main(): run_unittest(CommandTests) + reap_children() if __name__ == "__main__": diff --git a/Lib/test/test_compile.py b/Lib/test/test_compile.py index 72c4f7e..a3f15bf 100644 --- a/Lib/test/test_compile.py +++ b/Lib/test/test_compile.py @@ -166,6 +166,16 @@ if 1: pass""" compile(s, "", "exec") + # This test is probably specific to CPython and may not generalize + # to other implementations. We are trying to ensure that when + # the first line of code starts after 256, correct line numbers + # in tracebacks are still produced. + def test_leading_newlines(self): + s256 = "".join(["\n"] * 256 + ["spam"]) + co = compile(s256, 'fn', 'exec') + self.assertEqual(co.co_firstlineno, 257) + self.assertEqual(co.co_lnotab, '') + def test_literals_with_leading_zeroes(self): for arg in ["077787", "0xj", "0x.", "0e", "090000000000000", "080000000000000", "000000000000009", "000000000000008"]: @@ -211,6 +221,25 @@ if 1: self.assertEqual(eval("-" + all_one_bits), -18446744073709551615L) else: self.fail("How many bits *does* this machine have???") + # Verify treatment of contant folding on -(sys.maxint+1) + # i.e. -2147483648 on 32 bit platforms. Should return int, not long. + self.assertTrue(isinstance(eval("%s" % (-sys.maxint - 1)), int)) + self.assertTrue(isinstance(eval("%s" % (-sys.maxint - 2)), long)) + + if sys.maxint == 9223372036854775807: + def test_32_63_bit_values(self): + a = +4294967296 # 1 << 32 + b = -4294967296 # 1 << 32 + c = +281474976710656 # 1 << 48 + d = -281474976710656 # 1 << 48 + e = +4611686018427387904 # 1 << 62 + f = -4611686018427387904 # 1 << 62 + g = +9223372036854775807 # 1 << 63 - 1 + h = -9223372036854775807 # 1 << 63 - 1 + + for variable in self.test_32_63_bit_values.func_code.co_consts: + if variable is not None: + self.assertTrue(isinstance(variable, int)) def test_sequence_unpacking_error(self): # Verify sequence packing/unpacking with "or". SF bug #757818 @@ -238,6 +267,8 @@ if 1: succeed = [ 'import sys', 'import os, sys', + 'import os as bar', + 'import os.path as bar', 'from __future__ import nested_scopes, generators', 'from __future__ import (nested_scopes,\ngenerators)', 'from __future__ import (nested_scopes,\ngenerators,)', @@ -257,6 +288,10 @@ if 1: 'import (sys', 'import sys)', 'import (os,)', + 'import os As bar', + 'import os.path a bar', + 'from sys import stdin As stdout', + 'from sys import stdin a stdout', 'from (sys) import stdin', 'from __future__ import (nested_scopes', 'from __future__ import nested_scopes)', diff --git a/Lib/test/test_compiler.py b/Lib/test/test_compiler.py index 48f1643..1efb6a6 100644 --- a/Lib/test/test_compiler.py +++ b/Lib/test/test_compiler.py @@ -56,13 +56,30 @@ class CompilerTest(unittest.TestCase): def testYieldExpr(self): compiler.compile("def g(): yield\n\n", "", "exec") + def testTryExceptFinally(self): + # Test that except and finally clauses in one try stmt are recognized + c = compiler.compile("try:\n 1/0\nexcept:\n e = 1\nfinally:\n f = 1", + "", "exec") + dct = {} + exec c in dct + self.assertEquals(dct.get('e'), 1) + self.assertEquals(dct.get('f'), 1) + def testDefaultArgs(self): self.assertRaises(SyntaxError, compiler.parse, "def foo(a=1, b): pass") + def testDocstrings(self): + c = compiler.compile('"doc"', '', 'exec') + self.assert_('__doc__' in c.co_names) + c = compiler.compile('def f():\n "doc"', '', 'exec') + g = {} + exec c in g + self.assertEquals(g['f'].__doc__, "doc") + def testLineNo(self): # Test that all nodes except Module have a correct lineno attribute. filename = __file__ - if filename.endswith(".pyc") or filename.endswith(".pyo"): + if filename.endswith((".pyc", ".pyo")): filename = filename[:-1] tree = compiler.parseFile(filename) self.check_lineno(tree) @@ -87,6 +104,19 @@ class CompilerTest(unittest.TestCase): self.assertEquals(flatten([1, [2]]), [1, 2]) self.assertEquals(flatten((1, (2,))), [1, 2]) + def testNestedScope(self): + c = compiler.compile('def g():\n' + ' a = 1\n' + ' def f(): return a + 2\n' + ' return f()\n' + 'result = g()', + '', + 'exec') + dct = {} + exec c in dct + self.assertEquals(dct.get('result'), 3) + + NOLINENO = (compiler.ast.Module, compiler.ast.Stmt, compiler.ast.Discard) ############################################################################### @@ -103,6 +133,12 @@ a, b = 2, 3 l = [(x, y) for x, y in zip(range(5), range(5,10))] l[0] l[3:4] +d = {'a': 2} +d = {} +t = () +t = (1, 2) +l = [] +l = [1, 2] if l: pass else: diff --git a/Lib/test/test_curses.py b/Lib/test/test_curses.py index dc2f20b..4022149 100644 --- a/Lib/test/test_curses.py +++ b/Lib/test/test_curses.py @@ -212,6 +212,13 @@ def module_funcs(stdscr): m = curses.getmouse() curses.ungetmouse(*m) + if hasattr(curses, 'is_term_resized'): + curses.is_term_resized(*stdscr.getmaxyx()) + if hasattr(curses, 'resizeterm'): + curses.resizeterm(*stdscr.getmaxyx()) + if hasattr(curses, 'resize_term'): + curses.resize_term(*stdscr.getmaxyx()) + def unit_tests(): from curses import ascii for ch, expected in [('a', 'a'), ('A', 'A'), diff --git a/Lib/test/test_defaultdict.py b/Lib/test/test_defaultdict.py index b5a6628..134b5a8 100644 --- a/Lib/test/test_defaultdict.py +++ b/Lib/test/test_defaultdict.py @@ -4,6 +4,7 @@ import os import copy import tempfile import unittest +from test import test_support from collections import defaultdict @@ -131,5 +132,8 @@ class TestDefaultDict(unittest.TestCase): self.assertEqual(d2, d1) +def test_main(): + test_support.run_unittest(TestDefaultDict) + if __name__ == "__main__": - unittest.main() + test_main() diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py index 8ee431b..4a39be5 100644 --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -1899,6 +1899,16 @@ def properties(): prop2 = property(fset=setter) vereq(prop2.__doc__, None) + # this segfaulted in 2.5b2 + try: + import _testcapi + except ImportError: + pass + else: + class X(object): + p = property(_testcapi.test_with_docstring) + + def supers(): if verbose: print "Testing super..." @@ -3046,6 +3056,21 @@ def kwdargs(): list.__init__(a, sequence=[0, 1, 2]) vereq(a, [0, 1, 2]) +def recursive__call__(): + if verbose: print ("Testing recursive __call__() by setting to instance of " + "class ...") + class A(object): + pass + + A.__call__ = A() + try: + A()() + except RuntimeError: + pass + else: + raise TestFailed("Recursion limit should have been reached for " + "__call__()") + def delhook(): if verbose: print "Testing __del__ hook..." log = [] @@ -3803,6 +3828,13 @@ def weakref_segfault(): o.whatever = Provoker(o) del o +def wrapper_segfault(): + # SF 927248: deeply nested wrappers could cause stack overflow + f = lambda:None + for i in xrange(1000000): + f = f.__call__ + f = None + # Fix SF #762455, segfault when sys.stdout is changed in getattr def filefault(): if verbose: @@ -3957,6 +3989,7 @@ def notimplemented(): def test_main(): weakref_segfault() # Must be first, somehow + wrapper_segfault() do_this_first() class_docstrings() lists() @@ -4015,6 +4048,7 @@ def test_main(): buffer_inherit() str_of_str_subclass() kwdargs() + recursive__call__() delhook() hashinherit() strops() diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index 081941d..c31092c 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -81,6 +81,13 @@ dis_bug1333982 = """\ bug1333982.func_code.co_firstlineno + 2, bug1333982.func_code.co_firstlineno + 3) +_BIG_LINENO_FORMAT = """\ +%3d 0 LOAD_GLOBAL 0 (spam) + 3 POP_TOP + 4 LOAD_CONST 0 (None) + 7 RETURN_VALUE +""" + class DisTests(unittest.TestCase): def do_disassembly_test(self, func, expected): s = StringIO.StringIO() @@ -124,6 +131,23 @@ class DisTests(unittest.TestCase): if __debug__: self.do_disassembly_test(bug1333982, dis_bug1333982) + def test_big_linenos(self): + def func(count): + namespace = {} + func = "def foo():\n " + "".join(["\n "] * count + ["spam\n"]) + exec func in namespace + return namespace['foo'] + + # Test all small ranges + for i in xrange(1, 300): + expected = _BIG_LINENO_FORMAT % (i + 2) + self.do_disassembly_test(func(i), expected) + + # Test some larger ranges too + for i in xrange(300, 5000, 10): + expected = _BIG_LINENO_FORMAT % (i + 2) + self.do_disassembly_test(func(i), expected) + def test_main(): run_unittest(DisTests) diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py index 01f7acd..e8379c5 100644 --- a/Lib/test/test_doctest.py +++ b/Lib/test/test_doctest.py @@ -419,7 +419,6 @@ methods, classmethods, staticmethods, properties, and nested classes. >>> finder = doctest.DocTestFinder() >>> tests = finder.find(SampleClass) - >>> tests.sort() >>> for t in tests: ... print '%2s %s' % (len(t.examples), t.name) 3 SampleClass @@ -435,7 +434,6 @@ methods, classmethods, staticmethods, properties, and nested classes. New-style classes are also supported: >>> tests = finder.find(SampleNewStyleClass) - >>> tests.sort() >>> for t in tests: ... print '%2s %s' % (len(t.examples), t.name) 1 SampleNewStyleClass @@ -475,7 +473,6 @@ functions, classes, and the `__test__` dictionary, if it exists: >>> # ignoring the objects since they weren't defined in m. >>> import test.test_doctest >>> tests = finder.find(m, module=test.test_doctest) - >>> tests.sort() >>> for t in tests: ... print '%2s %s' % (len(t.examples), t.name) 1 some_module @@ -499,7 +496,6 @@ will only be generated for it once: >>> from test import doctest_aliases >>> tests = excl_empty_finder.find(doctest_aliases) - >>> tests.sort() >>> print len(tests) 2 >>> print tests[0].name @@ -517,7 +513,6 @@ Empty Tests By default, an object with no doctests doesn't create any tests: >>> tests = doctest.DocTestFinder().find(SampleClass) - >>> tests.sort() >>> for t in tests: ... print '%2s %s' % (len(t.examples), t.name) 3 SampleClass @@ -536,7 +531,6 @@ is really to support backward compatibility in what doctest.master.summarize() displays. >>> tests = doctest.DocTestFinder(exclude_empty=False).find(SampleClass) - >>> tests.sort() >>> for t in tests: ... print '%2s %s' % (len(t.examples), t.name) 3 SampleClass @@ -557,7 +551,6 @@ DocTestFinder can be told not to look for tests in contained objects using the `recurse` flag: >>> tests = doctest.DocTestFinder(recurse=False).find(SampleClass) - >>> tests.sort() >>> for t in tests: ... print '%2s %s' % (len(t.examples), t.name) 3 SampleClass diff --git a/Lib/test/test_email_codecs.py b/Lib/test/test_email_codecs.py index aadd537..c550a6f 100644 --- a/Lib/test/test_email_codecs.py +++ b/Lib/test/test_email_codecs.py @@ -1,11 +1,15 @@ # Copyright (C) 2002 Python Software Foundation # email package unit tests for (optional) Asian codecs -import unittest # The specific tests now live in Lib/email/test -from email.test.test_email_codecs import suite +from email.test import test_email_codecs +from email.test import test_email_codecs_renamed +from test import test_support +def test_main(): + suite = test_email_codecs.suite() + suite.addTest(test_email_codecs_renamed.suite()) + test_support.run_suite(suite) - if __name__ == '__main__': - unittest.main(defaultTest='suite') + test_main() diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py index ebe60c1..be2cca1 100644 --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -314,6 +314,18 @@ class ExceptionTests(unittest.TestCase): x = DerivedException(fancy_arg=42) self.assertEquals(x.fancy_arg, 42) + def testInfiniteRecursion(self): + def f(): + return f() + self.assertRaises(RuntimeError, f) + + def g(): + try: + return g() + except ValueError: + return -1 + self.assertRaises(RuntimeError, g) + def test_main(): run_unittest(ExceptionTests) diff --git a/Lib/test/test_fcntl.py b/Lib/test/test_fcntl.py index f53b13a..58a57b5 100755 --- a/Lib/test/test_fcntl.py +++ b/Lib/test/test_fcntl.py @@ -20,9 +20,10 @@ else: if sys.platform.startswith('atheos'): start_len = "qq" -if sys.platform in ('netbsd1', 'netbsd2', 'Darwin1.2', 'darwin', - 'freebsd2', 'freebsd3', 'freebsd4', 'freebsd5', 'freebsd6', - 'freebsd7', +if sys.platform in ('netbsd1', 'netbsd2', 'netbsd3', + 'Darwin1.2', 'darwin', + 'freebsd2', 'freebsd3', 'freebsd4', 'freebsd5', + 'freebsd6', 'freebsd7', 'bsdos2', 'bsdos3', 'bsdos4', 'openbsd', 'openbsd2', 'openbsd3'): if struct.calcsize('l') == 8: diff --git a/Lib/test/test_file.py b/Lib/test/test_file.py index dcfa265..234920d 100644 --- a/Lib/test/test_file.py +++ b/Lib/test/test_file.py @@ -11,14 +11,12 @@ class AutoFileTests(unittest.TestCase): # file tests for which a test file is automatically set up def setUp(self): - self.f = file(TESTFN, 'wb') + self.f = open(TESTFN, 'wb') def tearDown(self): - try: - if self.f: - self.f.close() - except IOError: - pass + if self.f: + self.f.close() + os.remove(TESTFN) def testWeakRefs(self): # verify weak references @@ -80,9 +78,11 @@ class AutoFileTests(unittest.TestCase): def testWritelinesNonString(self): # verify writelines with non-string object - class NonString: pass + class NonString: + pass - self.assertRaises(TypeError, self.f.writelines, [NonString(), NonString()]) + self.assertRaises(TypeError, self.f.writelines, + [NonString(), NonString()]) def testRepr(self): # verify repr works @@ -93,19 +93,21 @@ class AutoFileTests(unittest.TestCase): self.assertEquals(f.name, TESTFN) self.assert_(not f.isatty()) self.assert_(not f.closed) - + self.assertRaises(TypeError, f.readinto, "") f.close() self.assert_(f.closed) def testMethods(self): methods = ['fileno', 'flush', 'isatty', 'next', 'read', 'readinto', - 'readline', 'readlines', 'seek', 'tell', 'truncate', 'write', - '__iter__'] + 'readline', 'readlines', 'seek', 'tell', 'truncate', + 'write', '__iter__'] if sys.platform.startswith('atheos'): methods.remove('truncate') - self.f.close() + # __exit__ should close the file + self.f.__exit__(None, None, None) + self.assert_(self.f.closed) for methodname in methods: method = getattr(self.f, methodname) @@ -113,6 +115,14 @@ class AutoFileTests(unittest.TestCase): self.assertRaises(ValueError, method) self.assertRaises(ValueError, self.f.writelines, []) + # file is closed, __exit__ shouldn't do anything + self.assertEquals(self.f.__exit__(None, None, None), None) + # it must also return None if an exception was given + try: + 1/0 + except: + self.assertEquals(self.f.__exit__(*sys.exc_info()), None) + class OtherFileTests(unittest.TestCase): @@ -120,7 +130,7 @@ class OtherFileTests(unittest.TestCase): # check invalid mode strings for mode in ("", "aU", "wU+"): try: - f = file(TESTFN, mode) + f = open(TESTFN, mode) except ValueError: pass else: @@ -142,6 +152,7 @@ class OtherFileTests(unittest.TestCase): f = open(unicode(TESTFN), "w") self.assert_(repr(f).startswith(" # "file.truncate fault on windows" - f = file(TESTFN, 'wb') + f = open(TESTFN, 'wb') f.write('12345678901') # 11 bytes f.close() - f = file(TESTFN,'rb+') + f = open(TESTFN,'rb+') data = f.read(5) if data != '12345': self.fail("Read on file opened for update failed %r" % data) @@ -208,14 +219,14 @@ class OtherFileTests(unittest.TestCase): os.unlink(TESTFN) def testIteration(self): - # Test the complex interaction when mixing file-iteration and the various - # read* methods. Ostensibly, the mixture could just be tested to work - # when it should work according to the Python language, instead of fail - # when it should fail according to the current CPython implementation. - # People don't always program Python the way they should, though, and the - # implemenation might change in subtle ways, so we explicitly test for - # errors, too; the test will just have to be updated when the - # implementation changes. + # Test the complex interaction when mixing file-iteration and the + # various read* methods. Ostensibly, the mixture could just be tested + # to work when it should work according to the Python language, + # instead of fail when it should fail according to the current CPython + # implementation. People don't always program Python the way they + # should, though, and the implemenation might change in subtle ways, + # so we explicitly test for errors, too; the test will just have to + # be updated when the implementation changes. dataoffset = 16384 filler = "ham\n" assert not dataoffset % len(filler), \ @@ -253,12 +264,13 @@ class OtherFileTests(unittest.TestCase): (methodname, args)) f.close() - # Test to see if harmless (by accident) mixing of read* and iteration - # still works. This depends on the size of the internal iteration - # buffer (currently 8192,) but we can test it in a flexible manner. - # Each line in the bag o' ham is 4 bytes ("h", "a", "m", "\n"), so - # 4096 lines of that should get us exactly on the buffer boundary for - # any power-of-2 buffersize between 4 and 16384 (inclusive). + # Test to see if harmless (by accident) mixing of read* and + # iteration still works. This depends on the size of the internal + # iteration buffer (currently 8192,) but we can test it in a + # flexible manner. Each line in the bag o' ham is 4 bytes + # ("h", "a", "m", "\n"), so 4096 lines of that should get us + # exactly on the buffer boundary for any power-of-2 buffersize + # between 4 and 16384 (inclusive). f = open(TESTFN, 'rb') for i in range(nchunks): f.next() @@ -319,7 +331,13 @@ class OtherFileTests(unittest.TestCase): def test_main(): - run_unittest(AutoFileTests, OtherFileTests) + # Historically, these tests have been sloppy about removing TESTFN. + # So get rid of it no matter what. + try: + run_unittest(AutoFileTests, OtherFileTests) + finally: + if os.path.exists(TESTFN): + os.unlink(TESTFN) if __name__ == '__main__': test_main() diff --git a/Lib/test/test_filecmp.py b/Lib/test/test_filecmp.py index c54119c..503562b 100644 --- a/Lib/test/test_filecmp.py +++ b/Lib/test/test_filecmp.py @@ -1,5 +1,5 @@ -import os, filecmp, shutil, tempfile +import os, filecmp, shutil, tempfile, shutil import unittest from test import test_support @@ -49,6 +49,7 @@ class DirCompareTestCase(unittest.TestCase): self.caseinsensitive = os.path.normcase('A') == os.path.normcase('a') data = 'Contents of file go here.\n' for dir in [self.dir, self.dir_same, self.dir_diff]: + shutil.rmtree(dir, True) os.mkdir(dir) if self.caseinsensitive and dir is self.dir_same: fn = 'FiLe' # Verify case-insensitive comparison diff --git a/Lib/test/test_fork1.py b/Lib/test/test_fork1.py index cba5fc7..e64e398 100644 --- a/Lib/test/test_fork1.py +++ b/Lib/test/test_fork1.py @@ -2,8 +2,9 @@ """ import os +import time from test.fork_wait import ForkWait -from test.test_support import TestSkipped, run_unittest +from test.test_support import TestSkipped, run_unittest, reap_children try: os.fork @@ -12,12 +13,20 @@ except AttributeError: class ForkTest(ForkWait): def wait_impl(self, cpid): - spid, status = os.waitpid(cpid, 0) + for i in range(10): + # waitpid() shouldn't hang, but some of the buildbots seem to hang + # in the forking tests. This is an attempt to fix the problem. + spid, status = os.waitpid(cpid, os.WNOHANG) + if spid == cpid: + break + time.sleep(1.0) + self.assertEqual(spid, cpid) self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) def test_main(): run_unittest(ForkTest) + reap_children() if __name__ == "__main__": test_main() diff --git a/Lib/test/test_generators.py b/Lib/test/test_generators.py index a184a8b..ee36413 100644 --- a/Lib/test/test_generators.py +++ b/Lib/test/test_generators.py @@ -1497,22 +1497,55 @@ And a more sane, but still weird usage: +A yield expression with augmented assignment. + +>>> def coroutine(seq): +... count = 0 +... while count < 200: +... count += yield +... seq.append(count) +>>> seq = [] +>>> c = coroutine(seq) +>>> c.next() +>>> print seq +[] +>>> c.send(10) +>>> print seq +[10] +>>> c.send(10) +>>> print seq +[10, 20] +>>> c.send(10) +>>> print seq +[10, 20, 30] + + Check some syntax errors for yield expressions: >>> f=lambda: (yield 1),(yield 2) Traceback (most recent call last): ... -SyntaxError: 'yield' outside function (, line 1) +SyntaxError: 'yield' outside function (, line 1) >>> def f(): return lambda x=(yield): 1 Traceback (most recent call last): ... -SyntaxError: 'return' with argument inside generator (, line 1) +SyntaxError: 'return' with argument inside generator (, line 1) >>> def f(): x = yield = y Traceback (most recent call last): ... -SyntaxError: assignment to yield expression not possible (, line 1) +SyntaxError: assignment to yield expression not possible (, line 1) + +>>> def f(): (yield bar) = y +Traceback (most recent call last): + ... +SyntaxError: can't assign to yield expression (, line 1) + +>>> def f(): (yield bar) += y +Traceback (most recent call last): + ... +SyntaxError: augmented assignment to yield expression not possible (, line 1) Now check some throw() conditions: diff --git a/Lib/test/test_genexps.py b/Lib/test/test_genexps.py index e414757..2598a79 100644 --- a/Lib/test/test_genexps.py +++ b/Lib/test/test_genexps.py @@ -109,7 +109,7 @@ for iterability Traceback (most recent call last): File "", line 1, in -toplevel- (i for i in 6) - TypeError: iteration over non-sequence + TypeError: 'int' object is not iterable Verify late binding for the outermost if-expression diff --git a/Lib/test/test_getargs2.py b/Lib/test/test_getargs2.py index 8864e8e..c428f45 100644 --- a/Lib/test/test_getargs2.py +++ b/Lib/test/test_getargs2.py @@ -233,8 +233,25 @@ class LongLong_TestCase(unittest.TestCase): self.failUnlessEqual(VERY_LARGE & ULLONG_MAX, getargs_K(VERY_LARGE)) + +class Tuple_TestCase(unittest.TestCase): + def test_tuple(self): + from _testcapi import getargs_tuple + + ret = getargs_tuple(1, (2, 3)) + self.assertEquals(ret, (1,2,3)) + + # make sure invalid tuple arguments are handled correctly + class seq: + def __len__(self): + return 2 + def __getitem__(self, n): + raise ValueError + self.assertRaises(TypeError, getargs_tuple, 1, seq()) + + def test_main(): - tests = [Signed_TestCase, Unsigned_TestCase] + tests = [Signed_TestCase, Unsigned_TestCase, Tuple_TestCase] try: from _testcapi import getargs_L, getargs_K except ImportError: diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py index 4bb4e45..f160867 100644 --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -531,6 +531,11 @@ n = 0 for x in Squares(10): n = n+x if n != 285: raise TestFailed, 'for over growing sequence' +result = [] +for x, in [(1,), (2,), (3,)]: + result.append(x) +vereq(result, [1, 2, 3]) + print 'try_stmt' ### try_stmt: 'try' ':' suite (except_clause ':' suite)+ ['else' ':' suite] ### | 'try' ':' suite 'finally' ':' suite diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py index d9fd93d..99140d2 100644 --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -1,6 +1,8 @@ import sys +import types import unittest import inspect +import datetime from test.test_support import TESTFN, run_unittest @@ -15,7 +17,7 @@ from test import inspect_fodder2 as mod2 # isdatadescriptor modfile = mod.__file__ -if modfile.endswith('c') or modfile.endswith('o'): +if modfile.endswith(('c', 'o')): modfile = modfile[:-1] import __builtin__ @@ -40,10 +42,12 @@ class IsTestBase(unittest.TestCase): self.failIf(other(obj), 'not %s(%s)' % (other.__name__, exp)) class TestPredicates(IsTestBase): - def test_eleven(self): - # Doc/lib/libinspect.tex claims there are 11 such functions + def test_thirteen(self): count = len(filter(lambda x:x.startswith('is'), dir(inspect))) - self.assertEqual(count, 11, "There are %d (not 11) is* functions" % count) + # Doc/lib/libinspect.tex claims there are 13 such functions + expected = 13 + err_msg = "There are %d (not %d) is* functions" % (count, expected) + self.assertEqual(count, expected, err_msg) def test_excluding_predicates(self): self.istest(inspect.isbuiltin, 'sys.exit') @@ -58,6 +62,15 @@ class TestPredicates(IsTestBase): self.istest(inspect.istraceback, 'tb') self.istest(inspect.isdatadescriptor, '__builtin__.file.closed') self.istest(inspect.isdatadescriptor, '__builtin__.file.softspace') + if hasattr(types, 'GetSetDescriptorType'): + self.istest(inspect.isgetsetdescriptor, + 'type(tb.tb_frame).f_locals') + else: + self.failIf(inspect.isgetsetdescriptor(type(tb.tb_frame).f_locals)) + if hasattr(types, 'MemberDescriptorType'): + self.istest(inspect.ismemberdescriptor, 'datetime.timedelta.days') + else: + self.failIf(inspect.ismemberdescriptor(datetime.timedelta.days)) def test_isroutine(self): self.assert_(inspect.isroutine(mod.spam)) @@ -180,6 +193,17 @@ class TestRetrievingSourceCode(GetSourceBase): def test_getfile(self): self.assertEqual(inspect.getfile(mod.StupidGit), mod.__file__) + def test_getmodule_recursion(self): + from new import module + name = '__inspect_dummy' + m = sys.modules[name] = module(name) + m.__file__ = "" # hopefully not a real filename... + m.__loader__ = "dummy" # pretend the filename is understood by a loader + exec "def x(): pass" in m.__dict__ + self.assertEqual(inspect.getsourcefile(m.x.func_code), '') + del sys.modules[name] + inspect.getmodule(compile('a=10','','single')) + class TestDecorators(GetSourceBase): fodderFile = mod2 diff --git a/Lib/test/test_iterlen.py b/Lib/test/test_iterlen.py index bcd0a6f..af4467e 100644 --- a/Lib/test/test_iterlen.py +++ b/Lib/test/test_iterlen.py @@ -235,9 +235,7 @@ class TestSeqIterReversed(TestInvariantWithoutMutations): self.assertEqual(len(it), 0) - -if __name__ == "__main__": - +def test_main(): unittests = [ TestRepeat, TestXrange, @@ -255,3 +253,6 @@ if __name__ == "__main__": TestSeqIterReversed, ] test_support.run_unittest(*unittests) + +if __name__ == "__main__": + test_main() diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index 73f8288..68c23c2 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -480,6 +480,8 @@ def test4(): f.close() try: logging.config.fileConfig(fn) + #call again to make sure cleanup is correct + logging.config.fileConfig(fn) except: t = sys.exc_info()[0] message(str(t)) diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py index 914a20c..45dd118 100644 --- a/Lib/test/test_mailbox.py +++ b/Lib/test/test_mailbox.py @@ -461,7 +461,7 @@ class TestMaildir(TestMailbox): def setUp(self): TestMailbox.setUp(self) - if os.name == 'nt': + if os.name in ('nt', 'os2'): self._box.colon = '!' def test_add_MM(self): @@ -520,7 +520,7 @@ class TestMaildir(TestMailbox): # Initialize an existing mailbox self.tearDown() for subdir in '', 'tmp', 'new', 'cur': - os.mkdir(os.path.join(self._path, subdir)) + os.mkdir(os.path.normpath(os.path.join(self._path, subdir))) self._box = mailbox.Maildir(self._path) self._check_basics(factory=rfc822.Message) self._box = mailbox.Maildir(self._path, factory=None) @@ -720,6 +720,30 @@ class _TestMboxMMDF(TestMailbox): self.assert_(contents == open(self._path, 'rb').read()) self._box = self._factory(self._path) + def test_lock_conflict(self): + # Fork off a subprocess that will lock the file for 2 seconds, + # unlock it, and then exit. + if not hasattr(os, 'fork'): + return + pid = os.fork() + if pid == 0: + # In the child, lock the mailbox. + self._box.lock() + time.sleep(2) + self._box.unlock() + os._exit(0) + + # In the parent, sleep a bit to give the child time to acquire + # the lock. + time.sleep(0.5) + self.assertRaises(mailbox.ExternalClashError, + self._box.lock) + + # Wait for child to exit. Locking should now succeed. + exited_pid, status = os.waitpid(pid, 0) + self._box.lock() + self._box.unlock() + class TestMbox(_TestMboxMMDF): @@ -1761,6 +1785,7 @@ def test_main(): TestMessageConversion, TestProxyFile, TestPartialFile, MaildirTestCase) test_support.run_unittest(*tests) + test_support.reap_children() if __name__ == '__main__': diff --git a/Lib/test/test_mimetools.py b/Lib/test/test_mimetools.py index 96bbb36..b0b5b01 100644 --- a/Lib/test/test_mimetools.py +++ b/Lib/test/test_mimetools.py @@ -1,7 +1,7 @@ import unittest from test import test_support -import string, StringIO, mimetools, sets +import string, StringIO, mimetools msgtext1 = mimetools.Message(StringIO.StringIO( """Content-Type: text/plain; charset=iso-8859-1; format=flowed @@ -25,7 +25,7 @@ class MimeToolsTest(unittest.TestCase): self.assertEqual(o.getvalue(), start) def test_boundary(self): - s = sets.Set([""]) + s = set([""]) for i in xrange(100): nb = mimetools.choose_boundary() self.assert_(nb not in s) diff --git a/Lib/test/test_mimetypes.py b/Lib/test/test_mimetypes.py index 8c584ad..0190c2f 100644 --- a/Lib/test/test_mimetypes.py +++ b/Lib/test/test_mimetypes.py @@ -1,7 +1,6 @@ import mimetypes import StringIO import unittest -from sets import Set from test import test_support @@ -52,8 +51,8 @@ class MimeTypesTestCase(unittest.TestCase): # First try strict. Use a set here for testing the results because if # test_urllib2 is run before test_mimetypes, global state is modified # such that the 'all' set will have more items in it. - all = Set(self.db.guess_all_extensions('text/plain', strict=True)) - unless(all >= Set(['.bat', '.c', '.h', '.ksh', '.pl', '.txt'])) + all = set(self.db.guess_all_extensions('text/plain', strict=True)) + unless(all >= set(['.bat', '.c', '.h', '.ksh', '.pl', '.txt'])) # And now non-strict all = self.db.guess_all_extensions('image/jpg', strict=False) all.sort() diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py index b9377ae..a6d309f 100644 --- a/Lib/test/test_minidom.py +++ b/Lib/test/test_minidom.py @@ -1,4 +1,4 @@ -# test for xmlcore.dom.minidom +# test for xml.dom.minidom import os import sys @@ -7,12 +7,12 @@ import traceback from StringIO import StringIO from test.test_support import verbose -import xmlcore.dom -import xmlcore.dom.minidom -import xmlcore.parsers.expat +import xml.dom +import xml.dom.minidom +import xml.parsers.expat -from xmlcore.dom.minidom import parse, Node, Document, parseString -from xmlcore.dom.minidom import getDOMImplementation +from xml.dom.minidom import parse, Node, Document, parseString +from xml.dom.minidom import getDOMImplementation if __name__ == "__main__": @@ -138,29 +138,29 @@ def testLegalChildren(): text = dom.createTextNode('text') try: dom.appendChild(text) - except xmlcore.dom.HierarchyRequestErr: pass + except xml.dom.HierarchyRequestErr: pass else: print "dom.appendChild didn't raise HierarchyRequestErr" dom.appendChild(elem) try: dom.insertBefore(text, elem) - except xmlcore.dom.HierarchyRequestErr: pass + except xml.dom.HierarchyRequestErr: pass else: print "dom.appendChild didn't raise HierarchyRequestErr" try: dom.replaceChild(text, elem) - except xmlcore.dom.HierarchyRequestErr: pass + except xml.dom.HierarchyRequestErr: pass else: print "dom.appendChild didn't raise HierarchyRequestErr" nodemap = elem.attributes try: nodemap.setNamedItem(text) - except xmlcore.dom.HierarchyRequestErr: pass + except xml.dom.HierarchyRequestErr: pass else: print "NamedNodeMap.setNamedItem didn't raise HierarchyRequestErr" try: nodemap.setNamedItemNS(text) - except xmlcore.dom.HierarchyRequestErr: pass + except xml.dom.HierarchyRequestErr: pass else: print "NamedNodeMap.setNamedItemNS didn't raise HierarchyRequestErr" @@ -439,7 +439,7 @@ def testProcessingInstruction(): and pi.firstChild is None and pi.lastChild is None and pi.localName is None - and pi.namespaceURI == xmlcore.dom.EMPTY_NAMESPACE) + and pi.namespaceURI == xml.dom.EMPTY_NAMESPACE) def testProcessingInstructionRepr(): pass @@ -454,7 +454,7 @@ def testTooManyDocumentElements(): elem = doc.createElement("extra") try: doc.appendChild(elem) - except xmlcore.dom.HierarchyRequestErr: + except xml.dom.HierarchyRequestErr: pass else: print "Failed to catch expected exception when" \ @@ -491,7 +491,7 @@ def testRemoveNamedItem(): confirm(a1.isSameNode(a2)) try: attrs.removeNamedItem("a") - except xmlcore.dom.NotFoundErr: + except xml.dom.NotFoundErr: pass def testRemoveNamedItemNS(): @@ -503,7 +503,7 @@ def testRemoveNamedItemNS(): confirm(a1.isSameNode(a2)) try: attrs.removeNamedItemNS("http://xml.python.org/", "b") - except xmlcore.dom.NotFoundErr: + except xml.dom.NotFoundErr: pass def testAttrListValues(): pass @@ -682,7 +682,7 @@ def check_import_document(deep, testName): doc2 = parseString("") try: doc1.importNode(doc2, deep) - except xmlcore.dom.NotSupportedErr: + except xml.dom.NotSupportedErr: pass else: raise Exception(testName + @@ -705,14 +705,12 @@ def create_nonempty_doctype(): doctype = getDOMImplementation().createDocumentType("doc", None, None) doctype.entities._seq = [] doctype.notations._seq = [] - notation = xmlcore.dom.minidom.Notation( - "my-notation", None, - "http://xml.python.org/notations/my") + notation = xml.dom.minidom.Notation("my-notation", None, + "http://xml.python.org/notations/my") doctype.notations._seq.append(notation) - entity = xmlcore.dom.minidom.Entity( - "my-entity", None, - "http://xml.python.org/entities/my", - "my-notation") + entity = xml.dom.minidom.Entity("my-entity", None, + "http://xml.python.org/entities/my", + "my-notation") entity.version = "1.0" entity.encoding = "utf-8" entity.actualEncoding = "us-ascii" @@ -731,7 +729,7 @@ def testImportDocumentTypeShallow(): target = create_doc_without_doctype() try: imported = target.importNode(src.doctype, 0) - except xmlcore.dom.NotSupportedErr: + except xml.dom.NotSupportedErr: pass else: raise Exception( @@ -742,7 +740,7 @@ def testImportDocumentTypeDeep(): target = create_doc_without_doctype() try: imported = target.importNode(src.doctype, 1) - except xmlcore.dom.NotSupportedErr: + except xml.dom.NotSupportedErr: pass else: raise Exception( @@ -850,7 +848,7 @@ def testNodeListItem(): doc.unlink() def testSAX2DOM(): - from xmlcore.dom import pulldom + from xml.dom import pulldom sax2dom = pulldom.SAX2DOM() sax2dom.startDocument() @@ -940,11 +938,11 @@ def testRenameAttribute(): attr = elem.attributes['a'] # Simple renaming - attr = doc.renameNode(attr, xmlcore.dom.EMPTY_NAMESPACE, "b") + attr = doc.renameNode(attr, xml.dom.EMPTY_NAMESPACE, "b") confirm(attr.name == "b" and attr.nodeName == "b" and attr.localName is None - and attr.namespaceURI == xmlcore.dom.EMPTY_NAMESPACE + and attr.namespaceURI == xml.dom.EMPTY_NAMESPACE and attr.prefix is None and attr.value == "v" and elem.getAttributeNode("a") is None @@ -989,11 +987,11 @@ def testRenameAttribute(): and attrmap[("http://xml.python.org/ns2", "d")].isSameNode(attr)) # Rename back to a simple non-NS node - attr = doc.renameNode(attr, xmlcore.dom.EMPTY_NAMESPACE, "e") + attr = doc.renameNode(attr, xml.dom.EMPTY_NAMESPACE, "e") confirm(attr.name == "e" and attr.nodeName == "e" and attr.localName is None - and attr.namespaceURI == xmlcore.dom.EMPTY_NAMESPACE + and attr.namespaceURI == xml.dom.EMPTY_NAMESPACE and attr.prefix is None and attr.value == "v" and elem.getAttributeNode("a") is None @@ -1007,7 +1005,7 @@ def testRenameAttribute(): try: doc.renameNode(attr, "http://xml.python.org/ns", "xmlns") - except xmlcore.dom.NamespaceErr: + except xml.dom.NamespaceErr: pass else: print "expected NamespaceErr" @@ -1020,11 +1018,11 @@ def testRenameElement(): elem = doc.documentElement # Simple renaming - elem = doc.renameNode(elem, xmlcore.dom.EMPTY_NAMESPACE, "a") + elem = doc.renameNode(elem, xml.dom.EMPTY_NAMESPACE, "a") confirm(elem.tagName == "a" and elem.nodeName == "a" and elem.localName is None - and elem.namespaceURI == xmlcore.dom.EMPTY_NAMESPACE + and elem.namespaceURI == xml.dom.EMPTY_NAMESPACE and elem.prefix is None and elem.ownerDocument.isSameNode(doc)) @@ -1047,11 +1045,11 @@ def testRenameElement(): and elem.ownerDocument.isSameNode(doc)) # Rename back to a simple non-NS node - elem = doc.renameNode(elem, xmlcore.dom.EMPTY_NAMESPACE, "d") + elem = doc.renameNode(elem, xml.dom.EMPTY_NAMESPACE, "d") confirm(elem.tagName == "d" and elem.nodeName == "d" and elem.localName is None - and elem.namespaceURI == xmlcore.dom.EMPTY_NAMESPACE + and elem.namespaceURI == xml.dom.EMPTY_NAMESPACE and elem.prefix is None and elem.ownerDocument.isSameNode(doc)) @@ -1062,15 +1060,15 @@ def checkRenameNodeSharedConstraints(doc, node): # Make sure illegal NS usage is detected: try: doc.renameNode(node, "http://xml.python.org/ns", "xmlns:foo") - except xmlcore.dom.NamespaceErr: + except xml.dom.NamespaceErr: pass else: print "expected NamespaceErr" doc2 = parseString("") try: - doc2.renameNode(node, xmlcore.dom.EMPTY_NAMESPACE, "foo") - except xmlcore.dom.WrongDocumentErr: + doc2.renameNode(node, xml.dom.EMPTY_NAMESPACE, "foo") + except xml.dom.WrongDocumentErr: pass else: print "expected WrongDocumentErr" @@ -1078,12 +1076,12 @@ def checkRenameNodeSharedConstraints(doc, node): def testRenameOther(): # We have to create a comment node explicitly since not all DOM # builders used with minidom add comments to the DOM. - doc = xmlcore.dom.minidom.getDOMImplementation().createDocument( - xmlcore.dom.EMPTY_NAMESPACE, "e", None) + doc = xml.dom.minidom.getDOMImplementation().createDocument( + xml.dom.EMPTY_NAMESPACE, "e", None) node = doc.createComment("comment") try: - doc.renameNode(node, xmlcore.dom.EMPTY_NAMESPACE, "foo") - except xmlcore.dom.NotSupportedErr: + doc.renameNode(node, xml.dom.EMPTY_NAMESPACE, "foo") + except xml.dom.NotSupportedErr: pass else: print "expected NotSupportedErr when renaming comment node" @@ -1194,13 +1192,13 @@ def testSchemaType(): # since each supports a different level of DTD information. t = elem.schemaType confirm(t.name is None - and t.namespace == xmlcore.dom.EMPTY_NAMESPACE) + and t.namespace == xml.dom.EMPTY_NAMESPACE) names = "id notid text enum ref refs ent ents nm nms".split() for name in names: a = elem.getAttributeNode(name) t = a.schemaType confirm(hasattr(t, "name") - and t.namespace == xmlcore.dom.EMPTY_NAMESPACE) + and t.namespace == xml.dom.EMPTY_NAMESPACE) def testSetIdAttribute(): doc = parseString("") @@ -1229,7 +1227,7 @@ def testSetIdAttribute(): and a2.isId and not a3.isId) # renaming an attribute should not affect its ID-ness: - doc.renameNode(a2, xmlcore.dom.EMPTY_NAMESPACE, "an") + doc.renameNode(a2, xml.dom.EMPTY_NAMESPACE, "an") confirm(e.isSameNode(doc.getElementById("w")) and a2.isId) @@ -1265,7 +1263,7 @@ def testSetIdAttributeNS(): confirm(not a3.isId) confirm(doc.getElementById("v") is None) # renaming an attribute should not affect its ID-ness: - doc.renameNode(a2, xmlcore.dom.EMPTY_NAMESPACE, "an") + doc.renameNode(a2, xml.dom.EMPTY_NAMESPACE, "an") confirm(e.isSameNode(doc.getElementById("w")) and a2.isId) @@ -1301,7 +1299,7 @@ def testSetIdAttributeNode(): confirm(not a3.isId) confirm(doc.getElementById("v") is None) # renaming an attribute should not affect its ID-ness: - doc.renameNode(a2, xmlcore.dom.EMPTY_NAMESPACE, "an") + doc.renameNode(a2, xml.dom.EMPTY_NAMESPACE, "an") confirm(e.isSameNode(doc.getElementById("w")) and a2.isId) diff --git a/Lib/test/test_multibytecodec.py b/Lib/test/test_multibytecodec.py index 276b9af..397ebeb 100644 --- a/Lib/test/test_multibytecodec.py +++ b/Lib/test/test_multibytecodec.py @@ -6,17 +6,37 @@ from test import test_support from test import test_multibytecodec_support -import unittest, StringIO, codecs, sys +from test.test_support import TESTFN +import unittest, StringIO, codecs, sys, os + +ALL_CJKENCODINGS = [ +# _codecs_cn + 'gb2312', 'gbk', 'gb18030', 'hz', +# _codecs_hk + 'big5hkscs', +# _codecs_jp + 'cp932', 'shift_jis', 'euc_jp', 'euc_jisx0213', 'shift_jisx0213', + 'euc_jis_2004', 'shift_jis_2004', +# _codecs_kr + 'cp949', 'euc_kr', 'johab', +# _codecs_tw + 'big5', 'cp950', +# _codecs_iso2022 + 'iso2022_jp', 'iso2022_jp_1', 'iso2022_jp_2', 'iso2022_jp_2004', + 'iso2022_jp_3', 'iso2022_jp_ext', 'iso2022_kr', +] class Test_MultibyteCodec(unittest.TestCase): def test_nullcoding(self): - self.assertEqual(''.decode('gb18030'), u'') - self.assertEqual(unicode('', 'gb18030'), u'') - self.assertEqual(u''.encode('gb18030'), '') + for enc in ALL_CJKENCODINGS: + self.assertEqual(''.decode(enc), u'') + self.assertEqual(unicode('', enc), u'') + self.assertEqual(u''.encode(enc), '') def test_str_decode(self): - self.assertEqual('abcd'.encode('gb18030'), 'abcd') + for enc in ALL_CJKENCODINGS: + self.assertEqual('abcd'.encode(enc), 'abcd') def test_errorcallback_longindex(self): dec = codecs.getdecoder('euc-kr') @@ -25,6 +45,14 @@ class Test_MultibyteCodec(unittest.TestCase): self.assertRaises(IndexError, dec, 'apple\x92ham\x93spam', 'test.cjktest') + def test_codingspec(self): + try: + for enc in ALL_CJKENCODINGS: + print >> open(TESTFN, 'w'), '# coding:', enc + exec open(TESTFN) + finally: + os.unlink(TESTFN) + class Test_IncrementalEncoder(unittest.TestCase): def test_stateless(self): diff --git a/Lib/test/test_optparse.py b/Lib/test/test_optparse.py index 79df906..4582fa7 100644 --- a/Lib/test/test_optparse.py +++ b/Lib/test/test_optparse.py @@ -15,7 +15,7 @@ import copy import types import unittest -from cStringIO import StringIO +from StringIO import StringIO from pprint import pprint from test import test_support @@ -164,15 +164,23 @@ and kwargs %(kwargs)r expected_error=None): """Assert the parser prints the expected output on stdout.""" save_stdout = sys.stdout + encoding = getattr(save_stdout, 'encoding', None) try: try: sys.stdout = StringIO() + if encoding: + sys.stdout.encoding = encoding self.parser.parse_args(cmdline_args) finally: output = sys.stdout.getvalue() sys.stdout = save_stdout except InterceptedError, err: + self.assert_( + type(output) is types.StringType, + "expected output to be an ordinary string, not %r" + % type(output)) + if output != expected_output: self.fail("expected: \n'''\n" + expected_output + "'''\nbut got \n'''\n" + output + "'''") @@ -1452,10 +1460,26 @@ class TestHelp(BaseTest): make_option("--foo", action="append", type="string", dest='foo', help="store FOO in the foo list for later fooing"), ] + + # We need to set COLUMNS for the OptionParser constructor, but + # we must restore its original value -- otherwise, this test + # screws things up for other tests when it's part of the Python + # test suite. + orig_columns = os.environ.get('COLUMNS') os.environ['COLUMNS'] = str(columns) - return InterceptingOptionParser(option_list=options) + try: + return InterceptingOptionParser(option_list=options) + finally: + if orig_columns is None: + del os.environ['COLUMNS'] + else: + os.environ['COLUMNS'] = orig_columns def assertHelpEquals(self, expected_output): + if type(expected_output) is types.UnicodeType: + encoding = self.parser._get_encoding(sys.stdout) + expected_output = expected_output.encode(encoding, "replace") + save_argv = sys.argv[:] try: # Make optparse believe bar.py is being executed. @@ -1486,6 +1510,27 @@ class TestHelp(BaseTest): self.parser = self.make_parser(60) self.assertHelpEquals(_expected_help_short_lines) + def test_help_unicode(self): + self.parser = InterceptingOptionParser(usage=SUPPRESS_USAGE) + self.parser.add_option("-a", action="store_true", help=u"ol\u00E9!") + expect = u"""\ +Options: + -h, --help show this help message and exit + -a ol\u00E9! +""" + self.assertHelpEquals(expect) + + def test_help_unicode_description(self): + self.parser = InterceptingOptionParser(usage=SUPPRESS_USAGE, + description=u"ol\u00E9!") + expect = u"""\ +ol\u00E9! + +Options: + -h, --help show this help message and exit +""" + self.assertHelpEquals(expect) + def test_help_description_groups(self): self.parser.set_description( "This is the program description for %prog. %prog has " diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index ffc9420..9497777 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -11,6 +11,19 @@ from test import test_support warnings.filterwarnings("ignore", "tempnam", RuntimeWarning, __name__) warnings.filterwarnings("ignore", "tmpnam", RuntimeWarning, __name__) +# Tests creating TESTFN +class FileTests(unittest.TestCase): + def setUp(self): + if os.path.exists(test_support.TESTFN): + os.unlink(test_support.TESTFN) + tearDown = setUp + + def test_access(self): + f = os.open(test_support.TESTFN, os.O_CREAT|os.O_RDWR) + os.close(f) + self.assert_(os.access(test_support.TESTFN, os.W_OK)) + + class TemporaryFileTests(unittest.TestCase): def setUp(self): self.files = [] @@ -393,6 +406,7 @@ if sys.platform != 'win32': def test_main(): test_support.run_unittest( + FileTests, TemporaryFileTests, StatAttributeTests, EnvironTests, diff --git a/Lib/test/test_ossaudiodev.py b/Lib/test/test_ossaudiodev.py index 8810516..5868ea7 100644 --- a/Lib/test/test_ossaudiodev.py +++ b/Lib/test/test_ossaudiodev.py @@ -40,6 +40,10 @@ def read_sound_file(path): data = audioop.ulaw2lin(data, 2) return (data, rate, 16, nchannels) +# version of assert that still works with -O +def _assert(expr, message=None): + if not expr: + raise AssertionError(message or "assertion failed") def play_sound_file(data, rate, ssize, nchannels): try: @@ -57,9 +61,9 @@ def play_sound_file(data, rate, ssize, nchannels): dsp.fileno() # Make sure the read-only attributes work. - assert dsp.closed is False, "dsp.closed is not False" - assert dsp.name == "/dev/dsp" - assert dsp.mode == 'w', "bad dsp.mode: %r" % dsp.mode + _assert(dsp.closed is False, "dsp.closed is not False") + _assert(dsp.name == "/dev/dsp") + _assert(dsp.mode == 'w', "bad dsp.mode: %r" % dsp.mode) # And make sure they're really read-only. for attr in ('closed', 'name', 'mode'): @@ -69,14 +73,23 @@ def play_sound_file(data, rate, ssize, nchannels): except TypeError: pass + # Compute expected running time of sound sample (in seconds). + expected_time = float(len(data)) / (ssize/8) / nchannels / rate + # set parameters based on .au file headers dsp.setparameters(AFMT_S16_NE, nchannels, rate) + print ("playing test sound file (expected running time: %.2f sec)" + % expected_time) t1 = time.time() - print "playing test sound file..." dsp.write(data) dsp.close() t2 = time.time() - print "elapsed time: %.1f sec" % (t2-t1) + elapsed_time = t2 - t1 + + percent_diff = (abs(elapsed_time - expected_time) / expected_time) * 100 + _assert(percent_diff <= 10.0, \ + ("elapsed time (%.2f sec) > 10%% off of expected time (%.2f sec)" + % (elapsed_time, expected_time))) def test_setparameters(dsp): # Two configurations for testing: @@ -101,11 +114,11 @@ def test_setparameters(dsp): # setparameters() should be able to set this configuration in # either strict or non-strict mode. result = dsp.setparameters(fmt, channels, rate, False) - assert result == (fmt, channels, rate), \ - "setparameters%r: returned %r" % (config + result) + _assert(result == (fmt, channels, rate), + "setparameters%r: returned %r" % (config, result)) result = dsp.setparameters(fmt, channels, rate, True) - assert result == (fmt, channels, rate), \ - "setparameters%r: returned %r" % (config + result) + _assert(result == (fmt, channels, rate), + "setparameters%r: returned %r" % (config, result)) def test_bad_setparameters(dsp): @@ -123,8 +136,8 @@ def test_bad_setparameters(dsp): ]: (fmt, channels, rate) = config result = dsp.setparameters(fmt, channels, rate, False) - assert result != config, \ - "setparameters: unexpectedly got requested configuration" + _assert(result != config, + "setparameters: unexpectedly got requested configuration") try: result = dsp.setparameters(fmt, channels, rate, True) @@ -145,6 +158,6 @@ def test(): #test_bad_setparameters(dsp) finally: dsp.close() - assert dsp.closed is True, "dsp.closed is not True" + _assert(dsp.closed is True, "dsp.closed is not True") test() diff --git a/Lib/test/test_pep292.py b/Lib/test/test_pep292.py index 2a4353a..d1100ea 100644 --- a/Lib/test/test_pep292.py +++ b/Lib/test/test_pep292.py @@ -58,6 +58,13 @@ class TestTemplate(unittest.TestCase): s = Template('tim has eaten ${count} bags of ham today') eq(s.substitute(d), 'tim has eaten 7 bags of ham today') + def test_tupleargs(self): + eq = self.assertEqual + s = Template('$who ate ${meal}') + d = dict(who=('tim', 'fred'), meal=('ham', 'kung pao')) + eq(s.substitute(d), "('tim', 'fred') ate ('ham', 'kung pao')") + eq(s.safe_substitute(d), "('tim', 'fred') ate ('ham', 'kung pao')") + def test_SafeTemplate(self): eq = self.assertEqual s = Template('$who likes ${what} for ${meal}') diff --git a/Lib/test/test_popen.py b/Lib/test/test_popen.py index 2b687ad..fbf5e05 100644 --- a/Lib/test/test_popen.py +++ b/Lib/test/test_popen.py @@ -6,7 +6,7 @@ import os import sys -from test.test_support import TestSkipped +from test.test_support import TestSkipped, reap_children from os import popen # Test that command-lines get down as we expect. @@ -35,5 +35,6 @@ def _test_commandline(): def main(): print "Test popen:" _test_commandline() + reap_children() main() diff --git a/Lib/test/test_popen2.py b/Lib/test/test_popen2.py index 4db3cd1..2d54eb0 100644 --- a/Lib/test/test_popen2.py +++ b/Lib/test/test_popen2.py @@ -5,7 +5,7 @@ import os import sys -from test.test_support import TestSkipped +from test.test_support import TestSkipped, reap_children # popen2 contains its own testing routine # which is especially useful to see if open files @@ -75,3 +75,4 @@ def _test(): main() _test() +reap_children() diff --git a/Lib/test/test_pyexpat.py b/Lib/test/test_pyexpat.py index a9a5e8f..0698818 100644 --- a/Lib/test/test_pyexpat.py +++ b/Lib/test/test_pyexpat.py @@ -365,3 +365,24 @@ parser.Parse(''' ''', 1) + + +def test_parse_only_xml_data(): + # http://python.org/sf/1296433 + # + xml = "%s" % ('a' * 1025) + # this one doesn't crash + #xml = "%s" % ('a' * 10000) + + def handler(text): + raise Exception + + parser = expat.ParserCreate() + parser.CharacterDataHandler = handler + + try: + parser.Parse(xml) + except: + pass + +test_parse_only_xml_data() diff --git a/Lib/test/test_sax.py b/Lib/test/test_sax.py index ded81fb..af4c7dd 100644 --- a/Lib/test/test_sax.py +++ b/Lib/test/test_sax.py @@ -1,17 +1,17 @@ # regression test for SAX 2.0 -*- coding: iso-8859-1 -*- # $Id$ -from xmlcore.sax import make_parser, ContentHandler, \ - SAXException, SAXReaderNotAvailable, SAXParseException +from xml.sax import make_parser, ContentHandler, \ + SAXException, SAXReaderNotAvailable, SAXParseException try: make_parser() except SAXReaderNotAvailable: # don't try to test this module if we cannot create a parser raise ImportError("no XML parsers available") -from xmlcore.sax.saxutils import XMLGenerator, escape, unescape, quoteattr, \ - XMLFilterBase -from xmlcore.sax.expatreader import create_parser -from xmlcore.sax.xmlreader import InputSource, AttributesImpl, AttributesNSImpl +from xml.sax.saxutils import XMLGenerator, escape, unescape, quoteattr, \ + XMLFilterBase +from xml.sax.expatreader import create_parser +from xml.sax.xmlreader import InputSource, AttributesImpl, AttributesNSImpl from cStringIO import StringIO from test.test_support import verify, verbose, TestFailed, findfile import os @@ -36,17 +36,17 @@ def test_make_parser2(): # Creating parsers several times in a row should succeed. # Testing this because there have been failures of this kind # before. - from xmlcore.sax import make_parser + from xml.sax import make_parser p = make_parser() - from xmlcore.sax import make_parser + from xml.sax import make_parser p = make_parser() - from xmlcore.sax import make_parser + from xml.sax import make_parser p = make_parser() - from xmlcore.sax import make_parser + from xml.sax import make_parser p = make_parser() - from xmlcore.sax import make_parser + from xml.sax import make_parser p = make_parser() - from xmlcore.sax import make_parser + from xml.sax import make_parser p = make_parser() except: return 0 @@ -108,7 +108,7 @@ def test_make_parser(): try: # Creating a parser should succeed - it should fall back # to the expatreader - p = make_parser(['xmlcore.parsers.no_such_parser']) + p = make_parser(['xml.parsers.no_such_parser']) except: return 0 else: @@ -175,11 +175,14 @@ def test_xmlgen_attr_escape(): gen.endElement("e") gen.startElement("e", {"a": "'\""}) gen.endElement("e") + gen.startElement("e", {"a": "\n\r\t"}) + gen.endElement("e") gen.endElement("doc") gen.endDocument() - return result.getvalue() == start \ - + "" + return result.getvalue() == start + ("" + "" + "") def test_xmlgen_ignorable(): result = StringIO() @@ -668,6 +671,55 @@ def test_nsattrs_wattr(): attrs.getQNameByName((ns_uri, "attr")) == "ns:attr" +# During the development of Python 2.5, an attempt to move the "xml" +# package implementation to a new package ("xmlcore") proved painful. +# The goal of this change was to allow applications to be able to +# obtain and rely on behavior in the standard library implementation +# of the XML support without needing to be concerned about the +# availability of the PyXML implementation. +# +# While the existing import hackery in Lib/xml/__init__.py can cause +# PyXML's _xmlpus package to supplant the "xml" package, that only +# works because either implementation uses the "xml" package name for +# imports. +# +# The move resulted in a number of problems related to the fact that +# the import machinery's "package context" is based on the name that's +# being imported rather than the __name__ of the actual package +# containment; it wasn't possible for the "xml" package to be replaced +# by a simple module that indirected imports to the "xmlcore" package. +# +# The following two tests exercised bugs that were introduced in that +# attempt. Keeping these tests around will help detect problems with +# other attempts to provide reliable access to the standard library's +# implementation of the XML support. + +def test_sf_1511497(): + # Bug report: http://www.python.org/sf/1511497 + import sys + old_modules = sys.modules.copy() + for modname in sys.modules.keys(): + if modname.startswith("xml."): + del sys.modules[modname] + try: + import xml.sax.expatreader + module = xml.sax.expatreader + return module.__name__ == "xml.sax.expatreader" + finally: + sys.modules.update(old_modules) + +def test_sf_1513611(): + # Bug report: http://www.python.org/sf/1513611 + sio = StringIO("invalid") + parser = make_parser() + from xml.sax import SAXParseException + try: + parser.parse(sio) + except SAXParseException: + return True + else: + return False + # ===== Main program def make_test_output(): diff --git a/Lib/test/test_scope.py b/Lib/test/test_scope.py index f37254c..239745c 100644 --- a/Lib/test/test_scope.py +++ b/Lib/test/test_scope.py @@ -299,6 +299,17 @@ except NameError: else: raise TestFailed +# test for bug #1501934: incorrect LOAD/STORE_GLOBAL generation +global_x = 1 +def f(): + global_x += 1 +try: + f() +except UnboundLocalError: + pass +else: + raise TestFailed, 'scope of global_x not correctly determined' + print "14. complex definitions" def makeReturner(*lst): diff --git a/Lib/test/test_select.py b/Lib/test/test_select.py index eaec52b..d341324 100644 --- a/Lib/test/test_select.py +++ b/Lib/test/test_select.py @@ -1,5 +1,5 @@ # Testing select module -from test.test_support import verbose +from test.test_support import verbose, reap_children import select import os @@ -65,5 +65,6 @@ def test(): continue print 'Unexpected return values from select():', rfd, wfd, xfd p.close() + reap_children() test() diff --git a/Lib/test/test_sgmllib.py b/Lib/test/test_sgmllib.py index 8e8b02f..28a21a4 100644 --- a/Lib/test/test_sgmllib.py +++ b/Lib/test/test_sgmllib.py @@ -1,4 +1,6 @@ +import htmlentitydefs import pprint +import re import sgmllib import unittest from test import test_support @@ -64,6 +66,37 @@ class CDATAEventCollector(EventCollector): self.setliteral() +class HTMLEntityCollector(EventCollector): + + entity_or_charref = re.compile('(?:&([a-zA-Z][-.a-zA-Z0-9]*)' + '|&#(x[0-9a-zA-Z]+|[0-9]+))(;?)') + + def convert_charref(self, name): + self.append(("charref", "convert", name)) + if name[0] != "x": + return EventCollector.convert_charref(self, name) + + def convert_codepoint(self, codepoint): + self.append(("codepoint", "convert", codepoint)) + EventCollector.convert_codepoint(self, codepoint) + + def convert_entityref(self, name): + self.append(("entityref", "convert", name)) + return EventCollector.convert_entityref(self, name) + + # These to record that they were called, then pass the call along + # to the default implementation so that it's actions can be + # recorded. + + def handle_charref(self, data): + self.append(("charref", data)) + sgmllib.SGMLParser.handle_charref(self, data) + + def handle_entityref(self, data): + self.append(("entityref", data)) + sgmllib.SGMLParser.handle_entityref(self, data) + + class SGMLParserTestCase(unittest.TestCase): collector = EventCollector @@ -218,7 +251,9 @@ DOCTYPE html PUBLIC '-//W3C//DTD HTML 4.01//EN' """Substitution of entities and charrefs in attribute values""" # SF bug #1452246 self.check_events("""""", + f="&xxx;" g=' !' h='Ǵ' + i='x?a=b&c=d;' + j='&#42;' k='&#42;'>""", [("starttag", "a", [("b", "<"), ("c", "<>"), ("d", "<->"), @@ -226,13 +261,59 @@ DOCTYPE html PUBLIC '-//W3C//DTD HTML 4.01//EN' ("f", "&xxx;"), ("g", " !"), ("h", "Ǵ"), - ("i", "x?a=b&c=d;"), ])]) + ("i", "x?a=b&c=d;"), + ("j", "*"), + ("k", "*"), + ])]) + + def test_convert_overrides(self): + # This checks that the character and entity reference + # conversion helpers are called at the documented times. No + # attempt is made to really change what the parser accepts. + # + self.collector = HTMLEntityCollector + self.check_events(('foo' + '&foobar;*'), [ + ('entityref', 'convert', 'ldquo'), + ('charref', 'convert', 'x201d'), + ('starttag', 'a', [('title', '“test”')]), + ('data', 'foo'), + ('endtag', 'a'), + ('entityref', 'foobar'), + ('entityref', 'convert', 'foobar'), + ('charref', '42'), + ('charref', 'convert', '42'), + ('codepoint', 'convert', 42), + ]) + + def test_attr_values_quoted_markup(self): + """Multi-line and markup in attribute values""" + self.check_events("""text""", + [("starttag", "a", [("title", "foo\n
bar")]), + ("data", "text"), + ("endtag", "a")]) + self.check_events("""text""", + [("starttag", "a", [("title", "less < than")]), + ("data", "text"), + ("endtag", "a")]) + self.check_events("""text""", + [("starttag", "a", [("title", "greater > than")]), + ("data", "text"), + ("endtag", "a")]) def test_attr_funky_names(self): self.check_events("""""", [ ("starttag", "a", [("a.b", "v"), ("c:d", "v"), ("e-f", "v")]), ]) + def test_attr_value_ip6_url(self): + # http://www.python.org/sf/853506 + self.check_events(("" + ""), [ + ("starttag", "a", [("href", "http://[1080::8:800:200C:417A]/")]), + ("starttag", "a", [("href", "http://[1080::8:800:200C:417A]/")]), + ]) + def test_illegal_declarations(self): s = 'abcdef' self.check_events(s, [ @@ -301,8 +382,8 @@ DOCTYPE html PUBLIC '-//W3C//DTD HTML 4.01//EN' # that needs to be carefully considered before changing it. def _test_starttag_end_boundary(self): - self.check_events("""""", [("starttag", "a", [("b", "<")])]) - self.check_events("""""", [("starttag", "a", [("b", ">")])]) + self.check_events("", [("starttag", "a", [("b", "<")])]) + self.check_events("", [("starttag", "a", [("b", ">")])]) def _test_buffer_artefacts(self): output = [("starttag", "a", [("b", "<")])] @@ -322,17 +403,17 @@ DOCTYPE html PUBLIC '-//W3C//DTD HTML 4.01//EN' self.check_events([""], output) output = [("comment", "abc")] - self._run_check(["", ""], output) - self._run_check(["<", "!--abc-->"], output) - self._run_check([""], output) - self._run_check([""], output) - self._run_check([""], output) - self._run_check([""], output) - self._run_check([""], output) - self._run_check([""], output) - self._run_check(["", ""], output) + self.check_events(["", ""], output) + self.check_events(["<", "!--abc-->"], output) + self.check_events([""], output) + self.check_events([""], output) + self.check_events([""], output) + self.check_events([""], output) + self.check_events([""], output) + self.check_events([""], output) + self.check_events(["", ""], output) def _test_starttag_junk_chars(self): self.check_parse_error("<") diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py index 6ab5a35..da71fa8 100644 --- a/Lib/test/test_shutil.py +++ b/Lib/test/test_shutil.py @@ -74,6 +74,53 @@ class TestShutil(unittest.TestCase): except: pass + def test_copytree_simple(self): + def write_data(path, data): + f = open(path, "w") + f.write(data) + f.close() + + def read_data(path): + f = open(path) + data = f.read() + f.close() + return data + + src_dir = tempfile.mkdtemp() + dst_dir = os.path.join(tempfile.mkdtemp(), 'destination') + + write_data(os.path.join(src_dir, 'test.txt'), '123') + + os.mkdir(os.path.join(src_dir, 'test_dir')) + write_data(os.path.join(src_dir, 'test_dir', 'test.txt'), '456') + + try: + shutil.copytree(src_dir, dst_dir) + self.assertTrue(os.path.isfile(os.path.join(dst_dir, 'test.txt'))) + self.assertTrue(os.path.isdir(os.path.join(dst_dir, 'test_dir'))) + self.assertTrue(os.path.isfile(os.path.join(dst_dir, 'test_dir', + 'test.txt'))) + actual = read_data(os.path.join(dst_dir, 'test.txt')) + self.assertEqual(actual, '123') + actual = read_data(os.path.join(dst_dir, 'test_dir', 'test.txt')) + self.assertEqual(actual, '456') + finally: + for path in ( + os.path.join(src_dir, 'test.txt'), + os.path.join(dst_dir, 'test.txt'), + os.path.join(src_dir, 'test_dir', 'test.txt'), + os.path.join(dst_dir, 'test_dir', 'test.txt'), + ): + if os.path.exists(path): + os.remove(path) + for path in ( + os.path.join(src_dir, 'test_dir'), + os.path.join(dst_dir, 'test_dir'), + ): + if os.path.exists(path): + os.removedirs(path) + + if hasattr(os, "symlink"): def test_dont_copy_file_onto_link_to_itself(self): # bug 851123. diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py index f7fcb04..a6267d2 100644 --- a/Lib/test/test_signal.py +++ b/Lib/test/test_signal.py @@ -25,7 +25,11 @@ script = """ ) & """ % vars() +a_called = b_called = False + def handlerA(*args): + global a_called + a_called = True if verbose: print "handlerA", args @@ -33,11 +37,14 @@ class HandlerBCalled(Exception): pass def handlerB(*args): + global b_called + b_called = True if verbose: print "handlerB", args raise HandlerBCalled, args -signal.alarm(20) # Entire test lasts at most 20 sec. +MAX_DURATION = 20 +signal.alarm(MAX_DURATION) # Entire test should last at most 20 sec. hup = signal.signal(signal.SIGHUP, handlerA) usr1 = signal.signal(signal.SIGUSR1, handlerB) usr2 = signal.signal(signal.SIGUSR2, signal.SIG_IGN) @@ -65,9 +72,35 @@ try: except TypeError: pass +# Set up a child to send an alarm signal to us (the parent) after waiting +# long enough to receive the alarm. It seems we miss the alarm for some +# reason. This will hopefully stop the hangs on Tru64/Alpha. +def force_test_exit(): + # Sigh, both imports seem necessary to avoid errors. + import os + fork_pid = os.fork() + if fork_pid == 0: + # In child + import os, time + try: + # Wait 5 seconds longer than the expected alarm to give enough + # time for the normal sequence of events to occur. This is + # just a stop-gap to prevent the test from hanging. + time.sleep(MAX_DURATION + 5) + print >> sys.__stdout__, ' child should not have to kill parent' + for i in range(3): + os.kill(pid, signal.SIGALARM) + finally: + os._exit(0) + # In parent (or error) + return fork_pid + try: os.system(script) + # Try to ensure this test exits even if there is some problem with alarm. + # Tru64/Alpha sometimes hangs and is ultimately killed by the buildbot. + fork_pid = force_test_exit() print "starting pause() loop..." try: @@ -88,6 +121,22 @@ try: if verbose: print "KeyboardInterrupt (assume the alarm() went off)" + # Forcibly kill the child we created to ping us if there was a test error. + try: + # Make sure we don't kill ourself if there was a fork error. + if fork_pid > 0: + os.kill(fork_pid, signal.SIGKILL) + except: + # If the child killed us, it has probably exited. Killing a + # non-existant process will raise an error which we don't care about. + pass + + if not a_called: + print 'HandlerA not called' + + if not b_called: + print 'HandlerB not called' + finally: signal.signal(signal.SIGHUP, hup) signal.signal(signal.SIGUSR1, usr1) diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index 01b9b5b..356b801 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -11,6 +11,7 @@ import Queue import sys import array from weakref import proxy +import signal PORT = 50007 HOST = 'localhost' @@ -21,7 +22,8 @@ class SocketTCPTest(unittest.TestCase): def setUp(self): self.serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.serv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - self.serv.bind((HOST, PORT)) + global PORT + PORT = test_support.bind_port(self.serv, HOST, PORT) self.serv.listen(1) def tearDown(self): @@ -33,7 +35,8 @@ class SocketUDPTest(unittest.TestCase): def setUp(self): self.serv = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) self.serv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - self.serv.bind((HOST, PORT)) + global PORT + PORT = test_support.bind_port(self.serv, HOST, PORT) def tearDown(self): self.serv.close() @@ -447,7 +450,12 @@ class GeneralModuleTests(unittest.TestCase): sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.bind(("0.0.0.0", PORT+1)) name = sock.getsockname() - self.assertEqual(name, ("0.0.0.0", PORT+1)) + # XXX(nnorwitz): http://tinyurl.com/os5jz seems to indicate + # it reasonable to get the host's addr in addition to 0.0.0.0. + # At least for eCos. This is required for the S/390 to pass. + my_ip_addr = socket.gethostbyname(socket.gethostname()) + self.assert_(name[0] in ("0.0.0.0", my_ip_addr), '%s invalid' % name[0]) + self.assertEqual(name[1], PORT+1) def testGetSockOpt(self): # Testing getsockopt() @@ -575,6 +583,21 @@ class BasicUDPTest(ThreadedUDPSocketTest): def _testRecvFrom(self): self.cli.sendto(MSG, 0, (HOST, PORT)) +class TCPCloserTest(ThreadedTCPSocketTest): + + def testClose(self): + conn, addr = self.serv.accept() + conn.close() + + sd = self.cli + read, write, err = select.select([sd], [], [], 1.0) + self.assertEqual(read, [sd]) + self.assertEqual(sd.recv(1), '') + + def _testClose(self): + self.cli.connect((HOST, PORT)) + time.sleep(1.0) + class BasicSocketPairTest(SocketPairTest): def __init__(self, methodName='runTest'): @@ -795,6 +818,37 @@ class TCPTimeoutTest(SocketTCPTest): if not ok: self.fail("accept() returned success when we did not expect it") + def testInterruptedTimeout(self): + # XXX I don't know how to do this test on MSWindows or any other + # plaform that doesn't support signal.alarm() or os.kill(), though + # the bug should have existed on all platforms. + if not hasattr(signal, "alarm"): + return # can only test on *nix + self.serv.settimeout(5.0) # must be longer than alarm + class Alarm(Exception): + pass + def alarm_handler(signal, frame): + raise Alarm + old_alarm = signal.signal(signal.SIGALRM, alarm_handler) + try: + signal.alarm(2) # POSIX allows alarm to be up to 1 second early + try: + foo = self.serv.accept() + except socket.timeout: + self.fail("caught timeout instead of Alarm") + except Alarm: + pass + except: + self.fail("caught other exception instead of Alarm") + else: + self.fail("nothing caught") + signal.alarm(0) # shut off alarm + except Alarm: + self.fail("got Alarm in wrong place") + finally: + # no alarm can be pending. Safe to restore old handler. + signal.signal(signal.SIGALRM, old_alarm) + class UDPTimeoutTest(SocketTCPTest): def testUDPTimeout(self): @@ -883,8 +937,8 @@ class BufferIOTest(SocketConnectedTest): self.serv_conn.send(buf) def test_main(): - tests = [GeneralModuleTests, BasicTCPTest, TCPTimeoutTest, TestExceptions, - BufferIOTest] + tests = [GeneralModuleTests, BasicTCPTest, TCPCloserTest, TCPTimeoutTest, + TestExceptions, BufferIOTest] if sys.platform != 'mac': tests.extend([ BasicUDPTest, UDPTimeoutTest ]) @@ -899,7 +953,10 @@ def test_main(): tests.append(BasicSocketPairTest) if sys.platform == 'linux2': tests.append(TestLinuxAbstractNamespace) + + thread_info = test_support.threading_setup() test_support.run_unittest(*tests) + test_support.threading_cleanup(*thread_info) if __name__ == "__main__": test_main() diff --git a/Lib/test/test_socket_ssl.py b/Lib/test/test_socket_ssl.py index 1091383..3c9c9f0 100644 --- a/Lib/test/test_socket_ssl.py +++ b/Lib/test/test_socket_ssl.py @@ -3,6 +3,7 @@ import sys from test import test_support import socket +import errno # Optionally test SSL support. This requires the 'network' resource as given # on the regrtest command line. @@ -33,6 +34,13 @@ def test_basic(): def test_timeout(): test_support.requires('network') + def error_msg(extra_msg): + print >> sys.stderr, """\ + WARNING: an attempt to connect to %r %s, in + test_timeout. That may be legitimate, but is not the outcome we hoped + for. If this message is seen often, test_timeout should be changed to + use a more reliable address.""" % (ADDR, extra_msg) + if test_support.verbose: print "test_timeout ..." @@ -48,12 +56,14 @@ def test_timeout(): try: s.connect(ADDR) except socket.timeout: - print >> sys.stderr, """\ - WARNING: an attempt to connect to %r timed out, in - test_timeout. That may be legitimate, but is not the outcome we hoped - for. If this message is seen often, test_timeout should be changed to - use a more reliable address.""" % (ADDR,) + error_msg('timed out') return + except socket.error, exc: # In case connection is refused. + if exc.args[0] == errno.ECONNREFUSED: + error_msg('was refused') + return + else: + raise ss = socket.ssl(s) # Read part of return welcome banner twice. @@ -71,7 +81,7 @@ def test_rude_shutdown(): return # Some random port to connect to. - PORT = 9934 + PORT = [9934] listener_ready = threading.Event() listener_gone = threading.Event() @@ -82,7 +92,7 @@ def test_rude_shutdown(): # know the socket is gone. def listener(): s = socket.socket() - s.bind(('', PORT)) + PORT[0] = test_support.bind_port(s, '', PORT[0]) s.listen(5) listener_ready.set() s.accept() @@ -92,7 +102,7 @@ def test_rude_shutdown(): def connector(): listener_ready.wait() s = socket.socket() - s.connect(('localhost', PORT)) + s.connect(('localhost', PORT[0])) listener_gone.wait() try: ssl_sock = socket.ssl(s) diff --git a/Lib/test/test_socketserver.py b/Lib/test/test_socketserver.py index 1245ba5..dd4532f 100644 --- a/Lib/test/test_socketserver.py +++ b/Lib/test/test_socketserver.py @@ -1,11 +1,13 @@ # Test suite for SocketServer.py from test import test_support -from test.test_support import verbose, verify, TESTFN, TestSkipped +from test.test_support import (verbose, verify, TESTFN, TestSkipped, + reap_children) test_support.requires('network') from SocketServer import * import socket +import errno import select import time import threading @@ -77,6 +79,11 @@ class ServerThread(threading.Thread): pass if verbose: print "thread: creating server" svr = svrcls(self.__addr, self.__hdlrcls) + # pull the address out of the server in case it changed + # this can happen if another process is using the port + addr = getattr(svr, 'server_address') + if addr: + self.__addr = addr if verbose: print "thread: serving three times" svr.serve_a_few() if verbose: print "thread: done" @@ -136,7 +143,25 @@ def testloop(proto, servers, hdlrcls, testfunc): t.join() if verbose: print "done" -tcpservers = [TCPServer, ThreadingTCPServer] +class ForgivingTCPServer(TCPServer): + # prevent errors if another process is using the port we want + def server_bind(self): + host, default_port = self.server_address + # this code shamelessly stolen from test.test_support + # the ports were changed to protect the innocent + import sys + for port in [default_port, 3434, 8798, 23833]: + try: + self.server_address = host, port + TCPServer.server_bind(self) + break + except socket.error, (err, msg): + if err != errno.EADDRINUSE: + raise + print >>sys.__stderr__, \ + ' WARNING: failed to listen on port %d, trying another' % port + +tcpservers = [ForgivingTCPServer, ThreadingTCPServer] if hasattr(os, 'fork') and os.name not in ('os2',): tcpservers.append(ForkingTCPServer) udpservers = [UDPServer, ThreadingUDPServer] @@ -175,6 +200,7 @@ def test_main(): testall() finally: cleanup() + reap_children() if __name__ == "__main__": test_main() diff --git a/Lib/test/test_struct.py b/Lib/test/test_struct.py index aa458e6..66fd667 100644 --- a/Lib/test/test_struct.py +++ b/Lib/test/test_struct.py @@ -15,9 +15,11 @@ try: except ImportError: PY_STRUCT_RANGE_CHECKING = 0 PY_STRUCT_OVERFLOW_MASKING = 1 + PY_STRUCT_FLOAT_COERCE = 2 else: - PY_STRUCT_RANGE_CHECKING = _struct._PY_STRUCT_RANGE_CHECKING - PY_STRUCT_OVERFLOW_MASKING = _struct._PY_STRUCT_OVERFLOW_MASKING + PY_STRUCT_RANGE_CHECKING = getattr(_struct, '_PY_STRUCT_RANGE_CHECKING', 0) + PY_STRUCT_OVERFLOW_MASKING = getattr(_struct, '_PY_STRUCT_OVERFLOW_MASKING', 0) + PY_STRUCT_FLOAT_COERCE = getattr(_struct, '_PY_STRUCT_FLOAT_COERCE', 0) def string_reverse(s): return "".join(reversed(s)) @@ -46,33 +48,40 @@ def any_err(func, *args): raise TestFailed, "%s%s did not raise error" % ( func.__name__, args) +def with_warning_restore(func): + def _with_warning_restore(*args, **kw): + # The `warnings` module doesn't have an advertised way to restore + # its filter list. Cheat. + save_warnings_filters = warnings.filters[:] + # Grrr, we need this function to warn every time. Without removing + # the warningregistry, running test_tarfile then test_struct would fail + # on 64-bit platforms. + globals = func.func_globals + if '__warningregistry__' in globals: + del globals['__warningregistry__'] + warnings.filterwarnings("error", r"""^struct.*""", DeprecationWarning) + warnings.filterwarnings("error", r""".*format requires.*""", + DeprecationWarning) + try: + return func(*args, **kw) + finally: + warnings.filters[:] = save_warnings_filters[:] + return _with_warning_restore + def deprecated_err(func, *args): - # The `warnings` module doesn't have an advertised way to restore - # its filter list. Cheat. - save_warnings_filters = warnings.filters[:] - # Grrr, we need this function to warn every time. Without removing - # the warningregistry, running test_tarfile then test_struct would fail - # on 64-bit platforms. - globals = func.func_globals - if '__warningregistry__' in globals: - del globals['__warningregistry__'] - warnings.filterwarnings("error", r"""^struct.*""", DeprecationWarning) - warnings.filterwarnings("error", r""".*format requires.*""", - DeprecationWarning) try: - try: - func(*args) - except (struct.error, TypeError): - pass - except DeprecationWarning: - if not PY_STRUCT_OVERFLOW_MASKING: - raise TestFailed, "%s%s expected to raise struct.error" % ( - func.__name__, args) - else: - raise TestFailed, "%s%s did not raise error" % ( + func(*args) + except (struct.error, TypeError): + pass + except DeprecationWarning: + if not PY_STRUCT_OVERFLOW_MASKING: + raise TestFailed, "%s%s expected to raise struct.error" % ( func.__name__, args) - finally: - warnings.filters[:] = save_warnings_filters[:] + else: + raise TestFailed, "%s%s did not raise error" % ( + func.__name__, args) +deprecated_err = with_warning_restore(deprecated_err) + simple_err(struct.calcsize, 'Z') @@ -475,6 +484,9 @@ def test_705836(): test_705836() +########################################################################### +# SF bug 1229380. No struct.pack exception for some out of range integers + def test_1229380(): import sys for endian in ('', '>', '<'): @@ -491,6 +503,37 @@ def test_1229380(): if PY_STRUCT_RANGE_CHECKING: test_1229380() +########################################################################### +# SF bug 1530559. struct.pack raises TypeError where it used to convert. + +def check_float_coerce(format, number): + if PY_STRUCT_FLOAT_COERCE == 2: + # Test for pre-2.5 struct module + packed = struct.pack(format, number) + floored = struct.unpack(format, packed)[0] + if floored != int(number): + raise TestFailed("did not correcly coerce float to int") + return + try: + func(*args) + except (struct.error, TypeError): + if PY_STRUCT_FLOAT_COERCE: + raise TestFailed("expected DeprecationWarning for float coerce") + except DeprecationWarning: + if not PY_STRUCT_FLOAT_COERCE: + raise TestFailed("expected to raise struct.error for float coerce") + else: + raise TestFailed("did not raise error for float coerce") + +check_float_coerce = with_warning_restore(deprecated_err) + +def test_1530559(): + for endian in ('', '>', '<'): + for fmt in ('B', 'H', 'I', 'L', 'b', 'h', 'i', 'l'): + check_float_coerce(endian + fmt, 1.0) + check_float_coerce(endian + fmt, 1.5) + +test_1530559() ########################################################################### # Packing and unpacking to/from buffers. diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py index edf5bd0..8c8ac40 100644 --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -27,6 +27,18 @@ def remove_stderr_debug_decorations(stderr): return re.sub(r"\[\d+ refs\]\r?\n?$", "", stderr) class ProcessTestCase(unittest.TestCase): + def setUp(self): + # Try to minimize the number of children we have so this test + # doesn't crash on some buildbots (Alphas in particular). + if hasattr(test_support, "reap_children"): + test_support.reap_children() + + def tearDown(self): + # Try to minimize the number of children we have so this test + # doesn't crash on some buildbots (Alphas in particular). + if hasattr(test_support, "reap_children"): + test_support.reap_children() + def mkstemp(self): """wrapper for mkstemp, calling mktemp if mkstemp is not available""" if hasattr(tempfile, "mkstemp"): @@ -56,7 +68,7 @@ class ProcessTestCase(unittest.TestCase): subprocess.check_call([sys.executable, "-c", "import sys; sys.exit(47)"]) except subprocess.CalledProcessError, e: - self.assertEqual(e.errno, 47) + self.assertEqual(e.returncode, 47) else: self.fail("Expected CalledProcessError") @@ -384,7 +396,8 @@ class ProcessTestCase(unittest.TestCase): def test_no_leaking(self): # Make sure we leak no resources - if test_support.is_resource_enabled("subprocess") and not mswindows: + if not hasattr(test_support, "is_resource_enabled") \ + or test_support.is_resource_enabled("subprocess") and not mswindows: max_handles = 1026 # too much for most UNIX systems else: max_handles = 65 @@ -463,10 +476,36 @@ class ProcessTestCase(unittest.TestCase): else: self.fail("Expected OSError") + def _suppress_core_files(self): + """Try to prevent core files from being created. + Returns previous ulimit if successful, else None. + """ + try: + import resource + old_limit = resource.getrlimit(resource.RLIMIT_CORE) + resource.setrlimit(resource.RLIMIT_CORE, (0,0)) + return old_limit + except (ImportError, ValueError, resource.error): + return None + + def _unsuppress_core_files(self, old_limit): + """Return core file behavior to default.""" + if old_limit is None: + return + try: + import resource + resource.setrlimit(resource.RLIMIT_CORE, old_limit) + except (ImportError, ValueError, resource.error): + return + def test_run_abort(self): # returncode handles signal termination - p = subprocess.Popen([sys.executable, - "-c", "import os; os.abort()"]) + old_limit = self._suppress_core_files() + try: + p = subprocess.Popen([sys.executable, + "-c", "import os; os.abort()"]) + finally: + self._unsuppress_core_files(old_limit) p.wait() self.assertEqual(-p.returncode, signal.SIGABRT) @@ -599,6 +638,8 @@ class ProcessTestCase(unittest.TestCase): def test_main(): test_support.run_unittest(ProcessTestCase) + if hasattr(test_support, "reap_children"): + test_support.reap_children() if __name__ == "__main__": test_main() diff --git a/Lib/test/test_support.py b/Lib/test/test_support.py index 2d08f4d..a9d5dab 100644 --- a/Lib/test/test_support.py +++ b/Lib/test/test_support.py @@ -89,6 +89,24 @@ def requires(resource, msg=None): msg = "Use of the `%s' resource not enabled" % resource raise ResourceDenied(msg) +def bind_port(sock, host='', preferred_port=54321): + """Try to bind the sock to a port. If we are running multiple + tests and we don't try multiple ports, the test can fails. This + makes the test more robust.""" + + import socket, errno + # some random ports that hopefully no one is listening on. + for port in [preferred_port, 9907, 10243, 32999]: + try: + sock.bind((host, port)) + return port + except socket.error, (err, msg): + if err != errno.EADDRINUSE: + raise + print >>sys.__stderr__, \ + ' WARNING: failed to listen on port %d, trying another' % port + raise TestFailed, 'unable to find port to listen on' + FUZZ = 1e-6 def fcmp(x, y): # fuzzy comparison function @@ -296,6 +314,12 @@ _1M = 1024*1024 _1G = 1024 * _1M _2G = 2 * _1G +# Hack to get at the maximum value an internal index can take. +class _Dummy: + def __getslice__(self, i, j): + return j +MAX_Py_ssize_t = _Dummy()[:] + def set_memlimit(limit): import re global max_memuse @@ -310,7 +334,9 @@ def set_memlimit(limit): if m is None: raise ValueError('Invalid memory limit %r' % (limit,)) memlimit = int(float(m.group(1)) * sizes[m.group(3).lower()]) - if memlimit < 2.5*_1G: + if memlimit > MAX_Py_ssize_t: + memlimit = MAX_Py_ssize_t + if memlimit < _2G - 1: raise ValueError('Memory limit %r too low to be useful' % (limit,)) max_memuse = memlimit @@ -353,6 +379,17 @@ def bigmemtest(minsize, memuse, overhead=5*_1M): return wrapper return decorator +def bigaddrspacetest(f): + """Decorator for tests that fill the address space.""" + def wrapper(self): + if max_memuse < MAX_Py_ssize_t: + if verbose: + sys.stderr.write("Skipping %s because of memory " + "constraint\n" % (f.__name__,)) + else: + return f(self) + return wrapper + #======================================================================= # Preliminary PyUNIT integration. @@ -435,3 +472,46 @@ def run_doctest(module, verbosity=None): if verbose: print 'doctest (%s) ... %d tests with zero failures' % (module.__name__, t) return f, t + +#======================================================================= +# Threading support to prevent reporting refleaks when running regrtest.py -R + +def threading_setup(): + import threading + return len(threading._active), len(threading._limbo) + +def threading_cleanup(num_active, num_limbo): + import threading + import time + + _MAX_COUNT = 10 + count = 0 + while len(threading._active) != num_active and count < _MAX_COUNT: + count += 1 + time.sleep(0.1) + + count = 0 + while len(threading._limbo) != num_limbo and count < _MAX_COUNT: + count += 1 + time.sleep(0.1) + +def reap_children(): + """Use this function at the end of test_main() whenever sub-processes + are started. This will help ensure that no extra children (zombies) + stick around to hog resources and create problems when looking + for refleaks. + """ + + # Reap all our dead child processes so we don't leave zombies around. + # These hog resources and might be causing some of the buildbots to die. + import os + if hasattr(os, 'waitpid'): + any_process = -1 + while True: + try: + # This will raise an exception on Windows. That's ok. + pid, status = os.waitpid(any_process, os.WNOHANG) + if pid == 0: + break + except: + break diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index ae2a1c8..f1f1524 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -237,6 +237,90 @@ class SysModuleTest(unittest.TestCase): is sys._getframe().f_code ) + # sys._current_frames() is a CPython-only gimmick. + def test_current_frames(self): + have_threads = True + try: + import thread + except ImportError: + have_threads = False + + if have_threads: + self.current_frames_with_threads() + else: + self.current_frames_without_threads() + + # Test sys._current_frames() in a WITH_THREADS build. + def current_frames_with_threads(self): + import threading, thread + import traceback + + # Spawn a thread that blocks at a known place. Then the main + # thread does sys._current_frames(), and verifies that the frames + # returned make sense. + entered_g = threading.Event() + leave_g = threading.Event() + thread_info = [] # the thread's id + + def f123(): + g456() + + def g456(): + thread_info.append(thread.get_ident()) + entered_g.set() + leave_g.wait() + + t = threading.Thread(target=f123) + t.start() + entered_g.wait() + + # At this point, t has finished its entered_g.set(), although it's + # impossible to guess whether it's still on that line or has moved on + # to its leave_g.wait(). + self.assertEqual(len(thread_info), 1) + thread_id = thread_info[0] + + d = sys._current_frames() + + main_id = thread.get_ident() + self.assert_(main_id in d) + self.assert_(thread_id in d) + + # Verify that the captured main-thread frame is _this_ frame. + frame = d.pop(main_id) + self.assert_(frame is sys._getframe()) + + # Verify that the captured thread frame is blocked in g456, called + # from f123. This is a litte tricky, since various bits of + # threading.py are also in the thread's call stack. + frame = d.pop(thread_id) + stack = traceback.extract_stack(frame) + for i, (filename, lineno, funcname, sourceline) in enumerate(stack): + if funcname == "f123": + break + else: + self.fail("didn't find f123() on thread's call stack") + + self.assertEqual(sourceline, "g456()") + + # And the next record must be for g456(). + filename, lineno, funcname, sourceline = stack[i+1] + self.assertEqual(funcname, "g456") + self.assert_(sourceline in ["leave_g.wait()", "entered_g.set()"]) + + # Reap the spawned thread. + leave_g.set() + t.join() + + # Test sys._current_frames() when thread support doesn't exist. + def current_frames_without_threads(self): + # Not much happens here: there is only one thread, with artificial + # "thread id" 0. + d = sys._current_frames() + self.assertEqual(len(d), 1) + self.assert_(0 in d) + self.assert_(d[0] is sys._getframe()) + def test_attributes(self): self.assert_(isinstance(sys.api_version, int)) self.assert_(isinstance(sys.argv, list)) diff --git a/Lib/test/test_tcl.py b/Lib/test/test_tcl.py index e3fbf98..fa170ef 100644 --- a/Lib/test/test_tcl.py +++ b/Lib/test/test_tcl.py @@ -130,10 +130,8 @@ class TclTest(unittest.TestCase): import os old_display = None import sys - if (sys.platform.startswith('win') or - sys.platform.startswith('darwin') or - sys.platform.startswith('cygwin')): - return # no failure possible on windows? + if sys.platform.startswith(('win', 'darwin', 'cygwin')): + return # no failure possible on windows? if 'DISPLAY' in os.environ: old_display = os.environ['DISPLAY'] del os.environ['DISPLAY'] diff --git a/Lib/test/test_textwrap.py b/Lib/test/test_textwrap.py index 68e4d6d..500eceb 100644 --- a/Lib/test/test_textwrap.py +++ b/Lib/test/test_textwrap.py @@ -460,38 +460,42 @@ some (including a hanging indent).''' # of IndentTestCase! class DedentTestCase(unittest.TestCase): + def assertUnchanged(self, text): + """assert that dedent() has no effect on 'text'""" + self.assertEquals(text, dedent(text)) + def test_dedent_nomargin(self): # No lines indented. text = "Hello there.\nHow are you?\nOh good, I'm glad." - self.assertEquals(dedent(text), text) + self.assertUnchanged(text) # Similar, with a blank line. text = "Hello there.\n\nBoo!" - self.assertEquals(dedent(text), text) + self.assertUnchanged(text) # Some lines indented, but overall margin is still zero. text = "Hello there.\n This is indented." - self.assertEquals(dedent(text), text) + self.assertUnchanged(text) # Again, add a blank line. text = "Hello there.\n\n Boo!\n" - self.assertEquals(dedent(text), text) + self.assertUnchanged(text) def test_dedent_even(self): # All lines indented by two spaces. text = " Hello there.\n How are ya?\n Oh good." expect = "Hello there.\nHow are ya?\nOh good." - self.assertEquals(dedent(text), expect) + self.assertEquals(expect, dedent(text)) # Same, with blank lines. text = " Hello there.\n\n How are ya?\n Oh good.\n" expect = "Hello there.\n\nHow are ya?\nOh good.\n" - self.assertEquals(dedent(text), expect) + self.assertEquals(expect, dedent(text)) # Now indent one of the blank lines. text = " Hello there.\n \n How are ya?\n Oh good.\n" expect = "Hello there.\n\nHow are ya?\nOh good.\n" - self.assertEquals(dedent(text), expect) + self.assertEquals(expect, dedent(text)) def test_dedent_uneven(self): # Lines indented unevenly. @@ -505,18 +509,53 @@ def foo(): while 1: return foo ''' - self.assertEquals(dedent(text), expect) + self.assertEquals(expect, dedent(text)) # Uneven indentation with a blank line. text = " Foo\n Bar\n\n Baz\n" expect = "Foo\n Bar\n\n Baz\n" - self.assertEquals(dedent(text), expect) + self.assertEquals(expect, dedent(text)) # Uneven indentation with a whitespace-only line. text = " Foo\n Bar\n \n Baz\n" expect = "Foo\n Bar\n\n Baz\n" - self.assertEquals(dedent(text), expect) - + self.assertEquals(expect, dedent(text)) + + # dedent() should not mangle internal tabs + def test_dedent_preserve_internal_tabs(self): + text = " hello\tthere\n how are\tyou?" + expect = "hello\tthere\nhow are\tyou?" + self.assertEquals(expect, dedent(text)) + + # make sure that it preserves tabs when it's not making any + # changes at all + self.assertEquals(expect, dedent(expect)) + + # dedent() should not mangle tabs in the margin (i.e. + # tabs and spaces both count as margin, but are *not* + # considered equivalent) + def test_dedent_preserve_margin_tabs(self): + text = " hello there\n\thow are you?" + self.assertUnchanged(text) + + # same effect even if we have 8 spaces + text = " hello there\n\thow are you?" + self.assertUnchanged(text) + + # dedent() only removes whitespace that can be uniformly removed! + text = "\thello there\n\thow are you?" + expect = "hello there\nhow are you?" + self.assertEquals(expect, dedent(text)) + + text = " \thello there\n \thow are you?" + self.assertEquals(expect, dedent(text)) + + text = " \t hello there\n \t how are you?" + self.assertEquals(expect, dedent(text)) + + text = " \thello there\n \t how are you?" + expect = "hello there\n how are you?" + self.assertEquals(expect, dedent(text)) def test_main(): diff --git a/Lib/test/test_thread.py b/Lib/test/test_thread.py index ea345b6..c4c21fe 100644 --- a/Lib/test/test_thread.py +++ b/Lib/test/test_thread.py @@ -115,3 +115,46 @@ for i in range(numtasks): thread.start_new_thread(task2, (i,)) done.acquire() print 'all tasks done' + +# not all platforms support changing thread stack size +print '\n*** Changing thread stack size ***' +if thread.stack_size() != 0: + raise ValueError, "initial stack_size not 0" + +thread.stack_size(0) +if thread.stack_size() != 0: + raise ValueError, "stack_size not reset to default" + +from os import name as os_name +if os_name in ("nt", "os2", "posix"): + + tss_supported = 1 + try: + thread.stack_size(4096) + except ValueError: + print 'caught expected ValueError setting stack_size(4096)' + except thread.error: + tss_supported = 0 + print 'platform does not support changing thread stack size' + + if tss_supported: + failed = lambda s, e: s != e + fail_msg = "stack_size(%d) failed - should succeed" + for tss in (262144, 0x100000, 0): + thread.stack_size(tss) + if failed(thread.stack_size(), tss): + raise ValueError, fail_msg % tss + print 'successfully set stack_size(%d)' % tss + + for tss in (262144, 0x100000): + print 'trying stack_size = %d' % tss + next_ident = 0 + for i in range(numtasks): + newtask() + + print 'waiting for all tasks to complete' + done.acquire() + print 'all tasks done' + + # reset stack size to default + thread.stack_size(0) diff --git a/Lib/test/test_threaded_import.py b/Lib/test/test_threaded_import.py index 0642d25..602ad2a 100644 --- a/Lib/test/test_threaded_import.py +++ b/Lib/test/test_threaded_import.py @@ -30,11 +30,10 @@ def test_import_hangers(): if verbose: print "testing import hangers ...", - from test import threaded_import_hangers - + import test.threaded_import_hangers try: - if threaded_import_hangers.errors: - raise TestFailed(threaded_import_hangers.errors) + if test.threaded_import_hangers.errors: + raise TestFailed(test.threaded_import_hangers.errors) elif verbose: print "OK." finally: diff --git a/Lib/test/test_threadedtempfile.py b/Lib/test/test_threadedtempfile.py index 459ba3a..974333b 100644 --- a/Lib/test/test_threadedtempfile.py +++ b/Lib/test/test_threadedtempfile.py @@ -22,7 +22,7 @@ FILES_PER_THREAD = 50 # change w/ -f option import thread # If this fails, we can't test this module import threading -from test.test_support import TestFailed +from test.test_support import TestFailed, threading_setup, threading_cleanup import StringIO from traceback import print_exc import tempfile @@ -48,6 +48,7 @@ class TempFileGreedy(threading.Thread): def test_main(): threads = [] + thread_info = threading_setup() print "Creating" for i in range(NUM_THREADS): @@ -72,6 +73,7 @@ def test_main(): if errors: raise TestFailed(msg) + threading_cleanup(*thread_info) if __name__ == "__main__": import sys, getopt diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index 7eb9758..79335ea 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -85,6 +85,32 @@ class ThreadTests(unittest.TestCase): print 'all tasks done' self.assertEqual(numrunning.get(), 0) + # run with a small(ish) thread stack size (256kB) + def test_various_ops_small_stack(self): + if verbose: + print 'with 256kB thread stack size...' + try: + threading.stack_size(262144) + except thread.error: + if verbose: + print 'platform does not support changing thread stack size' + return + self.test_various_ops() + threading.stack_size(0) + + # run with a large thread stack size (1MB) + def test_various_ops_large_stack(self): + if verbose: + print 'with 1MB thread stack size...' + try: + threading.stack_size(0x100000) + except thread.error: + if verbose: + print 'platform does not support changing thread stack size' + return + self.test_various_ops() + threading.stack_size(0) + def test_foreign_thread(self): # Check that a "foreign" thread can use the threading module. def f(mutex): diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py index 768e7a0..f4be759 100644 --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -39,9 +39,9 @@ class TimeTestCase(unittest.TestCase): def test_strftime_bounds_checking(self): # Make sure that strftime() checks the bounds of the various parts - #of the time tuple. + #of the time tuple (0 is valid for *all* values). - # Check year + # Check year [1900, max(int)] self.assertRaises(ValueError, time.strftime, '', (1899, 1, 1, 0, 0, 0, 0, 1, -1)) if time.accept2dyear: @@ -49,27 +49,27 @@ class TimeTestCase(unittest.TestCase): (-1, 1, 1, 0, 0, 0, 0, 1, -1)) self.assertRaises(ValueError, time.strftime, '', (100, 1, 1, 0, 0, 0, 0, 1, -1)) - # Check month + # Check month [1, 12] + zero support self.assertRaises(ValueError, time.strftime, '', - (1900, 0, 1, 0, 0, 0, 0, 1, -1)) + (1900, -1, 1, 0, 0, 0, 0, 1, -1)) self.assertRaises(ValueError, time.strftime, '', (1900, 13, 1, 0, 0, 0, 0, 1, -1)) - # Check day of month + # Check day of month [1, 31] + zero support self.assertRaises(ValueError, time.strftime, '', - (1900, 1, 0, 0, 0, 0, 0, 1, -1)) + (1900, 1, -1, 0, 0, 0, 0, 1, -1)) self.assertRaises(ValueError, time.strftime, '', (1900, 1, 32, 0, 0, 0, 0, 1, -1)) - # Check hour + # Check hour [0, 23] self.assertRaises(ValueError, time.strftime, '', (1900, 1, 1, -1, 0, 0, 0, 1, -1)) self.assertRaises(ValueError, time.strftime, '', (1900, 1, 1, 24, 0, 0, 0, 1, -1)) - # Check minute + # Check minute [0, 59] self.assertRaises(ValueError, time.strftime, '', (1900, 1, 1, 0, -1, 0, 0, 1, -1)) self.assertRaises(ValueError, time.strftime, '', (1900, 1, 1, 0, 60, 0, 0, 1, -1)) - # Check second + # Check second [0, 61] self.assertRaises(ValueError, time.strftime, '', (1900, 1, 1, 0, 0, -1, 0, 1, -1)) # C99 only requires allowing for one leap second, but Python's docs say @@ -82,17 +82,25 @@ class TimeTestCase(unittest.TestCase): # modulo. self.assertRaises(ValueError, time.strftime, '', (1900, 1, 1, 0, 0, 0, -2, 1, -1)) - # Check day of the year + # Check day of the year [1, 366] + zero support self.assertRaises(ValueError, time.strftime, '', - (1900, 1, 1, 0, 0, 0, 0, 0, -1)) + (1900, 1, 1, 0, 0, 0, 0, -1, -1)) self.assertRaises(ValueError, time.strftime, '', (1900, 1, 1, 0, 0, 0, 0, 367, -1)) - # Check daylight savings flag + # Check daylight savings flag [-1, 1] self.assertRaises(ValueError, time.strftime, '', (1900, 1, 1, 0, 0, 0, 0, 1, -2)) self.assertRaises(ValueError, time.strftime, '', (1900, 1, 1, 0, 0, 0, 0, 1, 2)) + def test_default_values_for_zero(self): + # Make sure that using all zeros uses the proper default values. + # No test for daylight savings since strftime() does not change output + # based on its value. + expected = "2000 01 01 00 00 00 1 001" + result = time.strftime("%Y %m %d %H %M %S %w %j", (0,)*9) + self.assertEquals(expected, result) + def test_strptime(self): tt = time.gmtime(self.t) for directive in ('a', 'A', 'b', 'B', 'c', 'd', 'H', 'I', @@ -193,13 +201,17 @@ class TimeTestCase(unittest.TestCase): time.ctime(None) def test_gmtime_without_arg(self): - t0 = time.mktime(time.gmtime()) - t1 = time.mktime(time.gmtime(None)) + gt0 = time.gmtime() + gt1 = time.gmtime(None) + t0 = time.mktime(gt0) + t1 = time.mktime(gt1) self.assert_(0 <= (t1-t0) < 0.2) def test_localtime_without_arg(self): - t0 = time.mktime(time.localtime()) - t1 = time.mktime(time.localtime(None)) + lt0 = time.localtime() + lt1 = time.localtime(None) + t0 = time.mktime(lt0) + t1 = time.mktime(lt1) self.assert_(0 <= (t1-t0) < 0.2) def test_main(): diff --git a/Lib/test/test_timeout.py b/Lib/test/test_timeout.py index 4309e8c..2b32b92 100644 --- a/Lib/test/test_timeout.py +++ b/Lib/test/test_timeout.py @@ -100,7 +100,7 @@ class TimeoutTestCase(unittest.TestCase): def setUp(self): self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - self.addr_remote = ('www.python.org', 80) + self.addr_remote = ('www.python.org.', 80) self.addr_local = ('127.0.0.1', 25339) def tearDown(self): diff --git a/Lib/test/test_trace.py b/Lib/test/test_trace.py index 4f946f7..08aec8e 100644 --- a/Lib/test/test_trace.py +++ b/Lib/test/test_trace.py @@ -244,8 +244,8 @@ class TraceTestCase(unittest.TestCase): self.run_test(one_instr_line) def test_04_no_pop_blocks(self): self.run_test(no_pop_blocks) -## def test_05_no_pop_tops(self): -## self.run_test(no_pop_tops) + def test_05_no_pop_tops(self): + self.run_test(no_pop_tops) def test_06_call(self): self.run_test(call) def test_07_raise(self): diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py index 1b59f98..b3c5a50 100644 --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -31,8 +31,9 @@ class TracebackCases(unittest.TestCase): err = self.get_exception_format(self.syntax_error_with_caret, SyntaxError) self.assert_(len(err) == 4) - self.assert_("^" in err[2]) # third line has caret self.assert_(err[1].strip() == "return x!") + self.assert_("^" in err[2]) # third line has caret + self.assert_(err[1].find("!") == err[2].find("^")) # in the right place def test_nocaret(self): if is_jython: @@ -47,8 +48,9 @@ class TracebackCases(unittest.TestCase): err = self.get_exception_format(self.syntax_error_bad_indentation, IndentationError) self.assert_(len(err) == 4) - self.assert_("^" in err[2]) self.assert_(err[1].strip() == "print 2") + self.assert_("^" in err[2]) + self.assert_(err[1].find("2") == err[2].find("^")) def test_bug737473(self): import sys, os, tempfile, time @@ -109,6 +111,45 @@ def test(): lst = traceback.format_exception_only(e.__class__, e) self.assertEqual(lst, ['KeyboardInterrupt\n']) + # String exceptions are deprecated, but legal. The quirky form with + # separate "type" and "value" tends to break things, because + # not isinstance(value, type) + # and a string cannot be the first argument to issubclass. + # + # Note that sys.last_type and sys.last_value do not get set if an + # exception is caught, so we sort of cheat and just emulate them. + # + # test_string_exception1 is equivalent to + # + # >>> raise "String Exception" + # + # test_string_exception2 is equivalent to + # + # >>> raise "String Exception", "String Value" + # + def test_string_exception1(self): + str_type = "String Exception" + err = traceback.format_exception_only(str_type, None) + self.assertEqual(len(err), 1) + self.assertEqual(err[0], str_type + '\n') + + def test_string_exception2(self): + str_type = "String Exception" + str_value = "String Value" + err = traceback.format_exception_only(str_type, str_value) + self.assertEqual(len(err), 1) + self.assertEqual(err[0], str_type + ': ' + str_value + '\n') + + def test_format_exception_only_bad__str__(self): + class X(Exception): + def __str__(self): + 1/0 + err = traceback.format_exception_only(X, X()) + self.assertEqual(len(err), 1) + str_value = '' % X.__name__ + self.assertEqual(err[0], X.__name__ + ': ' + str_value + '\n') + + def test_main(): run_unittest(TracebackCases) diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py index c575c0c..2d299c3 100644 --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -233,6 +233,7 @@ print 'Buffers' try: buffer('asdf', -1) except ValueError: pass else: raise TestFailed, "buffer('asdf', -1) should raise ValueError" +cmp(buffer("abc"), buffer("def")) # used to raise a warning: tp_compare didn't return -1, 0, or 1 try: buffer(None) except TypeError: pass @@ -276,3 +277,10 @@ else: raise TestFailed, "buffer assignment should raise TypeError" try: a[0:1] = 'g' except TypeError: pass else: raise TestFailed, "buffer slice assignment should raise TypeError" + +# array.array() returns an object that does not implement a char buffer, +# something which int() uses for conversion. +import array +try: int(buffer(array.array('c'))) +except TypeError :pass +else: raise TestFailed, "char buffer (at C level) not working" diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index 034b9d0..67218b8 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -676,11 +676,11 @@ class HandlerTests(unittest.TestCase): r = MockResponse(200, "OK", {}, "") newreq = h.do_request_(req) if data is None: # GET - self.assert_("Content-length" not in req.unredirected_hdrs) - self.assert_("Content-type" not in req.unredirected_hdrs) + self.assert_("Content-Length" not in req.unredirected_hdrs) + self.assert_("Content-Type" not in req.unredirected_hdrs) else: # POST - self.assertEqual(req.unredirected_hdrs["Content-length"], "0") - self.assertEqual(req.unredirected_hdrs["Content-type"], + self.assertEqual(req.unredirected_hdrs["Content-Length"], "0") + self.assertEqual(req.unredirected_hdrs["Content-Type"], "application/x-www-form-urlencoded") # XXX the details of Host could be better tested self.assertEqual(req.unredirected_hdrs["Host"], "example.com") @@ -692,8 +692,8 @@ class HandlerTests(unittest.TestCase): req.add_unredirected_header("Host", "baz") req.add_unredirected_header("Spam", "foo") newreq = h.do_request_(req) - self.assertEqual(req.unredirected_hdrs["Content-length"], "foo") - self.assertEqual(req.unredirected_hdrs["Content-type"], "bar") + self.assertEqual(req.unredirected_hdrs["Content-Length"], "foo") + self.assertEqual(req.unredirected_hdrs["Content-Type"], "bar") self.assertEqual(req.unredirected_hdrs["Host"], "baz") self.assertEqual(req.unredirected_hdrs["Spam"], "foo") @@ -847,7 +847,7 @@ class HandlerTests(unittest.TestCase): 407, 'Proxy-Authenticate: Basic realm="%s"\r\n\r\n' % realm) opener.add_handler(auth_handler) opener.add_handler(http_handler) - self._test_basic_auth(opener, auth_handler, "Proxy-authorization", + self._test_basic_auth(opener, auth_handler, "Proxy-Authorization", realm, http_handler, password_manager, "http://acme.example.com:3128/protected", "proxy.example.com:3128", diff --git a/Lib/test/test_urllib2net.py b/Lib/test/test_urllib2net.py index dc3d36d..00cf202 100644 --- a/Lib/test/test_urllib2net.py +++ b/Lib/test/test_urllib2net.py @@ -123,7 +123,7 @@ class urlopenNetworkTests(unittest.TestCase): # domain will be spared to serve its defined # purpose. # urllib2.urlopen, "http://www.sadflkjsasadf.com/") - urllib2.urlopen, "http://www.python.invalid/") + urllib2.urlopen, "http://www.python.invalid./") class OtherNetworkTests(unittest.TestCase): @@ -160,8 +160,8 @@ class OtherNetworkTests(unittest.TestCase): "urllib2$") urls = [ # Thanks to Fred for finding these! - 'gopher://gopher.lib.ncsu.edu/11/library/stacks/Alex', - 'gopher://gopher.vt.edu:10010/10/33', + 'gopher://gopher.lib.ncsu.edu./11/library/stacks/Alex', + 'gopher://gopher.vt.edu.:10010/10/33', ] self._test_urls(urls, self._extra_handlers()) @@ -176,7 +176,7 @@ class OtherNetworkTests(unittest.TestCase): # XXX bug, should raise URLError #('file://nonsensename/etc/passwd', None, urllib2.URLError) - ('file://nonsensename/etc/passwd', None, (OSError, socket.error)) + ('file://nonsensename/etc/passwd', None, (EnvironmentError, socket.error)) ] self._test_urls(urls, self._extra_handlers()) finally: @@ -239,7 +239,9 @@ class OtherNetworkTests(unittest.TestCase): except (IOError, socket.error, OSError), err: debug(err) if expected_err: - self.assert_(isinstance(err, expected_err)) + msg = ("Didn't get expected error(s) %s for %s %s, got %s" % + (expected_err, url, req, err)) + self.assert_(isinstance(err, expected_err), msg) else: buf = f.read() f.close() @@ -259,7 +261,6 @@ class OtherNetworkTests(unittest.TestCase): return handlers - def test_main(): test_support.requires("network") test_support.run_unittest(URLTimeoutTest, urlopenNetworkTests, diff --git a/Lib/test/test_urllibnet.py b/Lib/test/test_urllibnet.py index 80761df..9105afe 100644 --- a/Lib/test/test_urllibnet.py +++ b/Lib/test/test_urllibnet.py @@ -110,7 +110,7 @@ class urlopenNetworkTests(unittest.TestCase): # domain will be spared to serve its defined # purpose. # urllib.urlopen, "http://www.sadflkjsasadf.com/") - urllib.urlopen, "http://www.python.invalid/") + urllib.urlopen, "http://www.python.invalid./") class urlretrieveNetworkTests(unittest.TestCase): """Tests urllib.urlretrieve using the network.""" diff --git a/Lib/test/test_uuid.py b/Lib/test/test_uuid.py new file mode 100644 index 0000000..0586cfd --- /dev/null +++ b/Lib/test/test_uuid.py @@ -0,0 +1,434 @@ +from unittest import TestCase +from test import test_support +import uuid + +def importable(name): + try: + __import__(name) + return True + except: + return False + +class TestUUID(TestCase): + last_node = None + source2node = {} + + def test_UUID(self): + equal = self.assertEqual + ascending = [] + for (string, curly, hex, bytes, fields, integer, urn, + time, clock_seq, variant, version) in [ + ('00000000-0000-0000-0000-000000000000', + '{00000000-0000-0000-0000-000000000000}', + '00000000000000000000000000000000', + '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0', + (0, 0, 0, 0, 0, 0), + 0, + 'urn:uuid:00000000-0000-0000-0000-000000000000', + 0, 0, uuid.RESERVED_NCS, None), + ('00010203-0405-0607-0809-0a0b0c0d0e0f', + '{00010203-0405-0607-0809-0a0b0c0d0e0f}', + '000102030405060708090a0b0c0d0e0f', + '\0\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\x0d\x0e\x0f', + (0x00010203L, 0x0405, 0x0607, 8, 9, 0x0a0b0c0d0e0fL), + 0x000102030405060708090a0b0c0d0e0fL, + 'urn:uuid:00010203-0405-0607-0809-0a0b0c0d0e0f', + 0x607040500010203L, 0x809, uuid.RESERVED_NCS, None), + ('02d9e6d5-9467-382e-8f9b-9300a64ac3cd', + '{02d9e6d5-9467-382e-8f9b-9300a64ac3cd}', + '02d9e6d59467382e8f9b9300a64ac3cd', + '\x02\xd9\xe6\xd5\x94\x67\x38\x2e\x8f\x9b\x93\x00\xa6\x4a\xc3\xcd', + (0x02d9e6d5L, 0x9467, 0x382e, 0x8f, 0x9b, 0x9300a64ac3cdL), + 0x02d9e6d59467382e8f9b9300a64ac3cdL, + 'urn:uuid:02d9e6d5-9467-382e-8f9b-9300a64ac3cd', + 0x82e946702d9e6d5L, 0xf9b, uuid.RFC_4122, 3), + ('12345678-1234-5678-1234-567812345678', + '{12345678-1234-5678-1234-567812345678}', + '12345678123456781234567812345678', + '\x12\x34\x56\x78'*4, + (0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678), + 0x12345678123456781234567812345678, + 'urn:uuid:12345678-1234-5678-1234-567812345678', + 0x678123412345678L, 0x1234, uuid.RESERVED_NCS, None), + ('6ba7b810-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b810-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8109dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x10\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b810L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8109dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b810-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b810L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b811-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b811-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8119dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x11\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b811L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8119dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b811-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b811L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b812-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b812-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8129dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x12\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b812L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8129dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b812-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b812L, 0xb4, uuid.RFC_4122, 1), + ('6ba7b814-9dad-11d1-80b4-00c04fd430c8', + '{6ba7b814-9dad-11d1-80b4-00c04fd430c8}', + '6ba7b8149dad11d180b400c04fd430c8', + '\x6b\xa7\xb8\x14\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + (0x6ba7b814L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), + 0x6ba7b8149dad11d180b400c04fd430c8L, + 'urn:uuid:6ba7b814-9dad-11d1-80b4-00c04fd430c8', + 0x1d19dad6ba7b814L, 0xb4, uuid.RFC_4122, 1), + ('7d444840-9dc0-11d1-b245-5ffdce74fad2', + '{7d444840-9dc0-11d1-b245-5ffdce74fad2}', + '7d4448409dc011d1b2455ffdce74fad2', + '\x7d\x44\x48\x40\x9d\xc0\x11\xd1\xb2\x45\x5f\xfd\xce\x74\xfa\xd2', + (0x7d444840L, 0x9dc0, 0x11d1, 0xb2, 0x45, 0x5ffdce74fad2L), + 0x7d4448409dc011d1b2455ffdce74fad2L, + 'urn:uuid:7d444840-9dc0-11d1-b245-5ffdce74fad2', + 0x1d19dc07d444840L, 0x3245, uuid.RFC_4122, 1), + ('e902893a-9d22-3c7e-a7b8-d6e313b71d9f', + '{e902893a-9d22-3c7e-a7b8-d6e313b71d9f}', + 'e902893a9d223c7ea7b8d6e313b71d9f', + '\xe9\x02\x89\x3a\x9d\x22\x3c\x7e\xa7\xb8\xd6\xe3\x13\xb7\x1d\x9f', + (0xe902893aL, 0x9d22, 0x3c7e, 0xa7, 0xb8, 0xd6e313b71d9fL), + 0xe902893a9d223c7ea7b8d6e313b71d9fL, + 'urn:uuid:e902893a-9d22-3c7e-a7b8-d6e313b71d9f', + 0xc7e9d22e902893aL, 0x27b8, uuid.RFC_4122, 3), + ('eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', + '{eb424026-6f54-4ef8-a4d0-bb658a1fc6cf}', + 'eb4240266f544ef8a4d0bb658a1fc6cf', + '\xeb\x42\x40\x26\x6f\x54\x4e\xf8\xa4\xd0\xbb\x65\x8a\x1f\xc6\xcf', + (0xeb424026L, 0x6f54, 0x4ef8, 0xa4, 0xd0, 0xbb658a1fc6cfL), + 0xeb4240266f544ef8a4d0bb658a1fc6cfL, + 'urn:uuid:eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', + 0xef86f54eb424026L, 0x24d0, uuid.RFC_4122, 4), + ('f81d4fae-7dec-11d0-a765-00a0c91e6bf6', + '{f81d4fae-7dec-11d0-a765-00a0c91e6bf6}', + 'f81d4fae7dec11d0a76500a0c91e6bf6', + '\xf8\x1d\x4f\xae\x7d\xec\x11\xd0\xa7\x65\x00\xa0\xc9\x1e\x6b\xf6', + (0xf81d4faeL, 0x7dec, 0x11d0, 0xa7, 0x65, 0x00a0c91e6bf6L), + 0xf81d4fae7dec11d0a76500a0c91e6bf6L, + 'urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6', + 0x1d07decf81d4faeL, 0x2765, uuid.RFC_4122, 1), + ('fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', + '{fffefdfc-fffe-fffe-fffe-fffefdfcfbfa}', + 'fffefdfcfffefffefffefffefdfcfbfa', + '\xff\xfe\xfd\xfc\xff\xfe\xff\xfe\xff\xfe\xff\xfe\xfd\xfc\xfb\xfa', + (0xfffefdfcL, 0xfffe, 0xfffe, 0xff, 0xfe, 0xfffefdfcfbfaL), + 0xfffefdfcfffefffefffefffefdfcfbfaL, + 'urn:uuid:fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', + 0xffefffefffefdfcL, 0x3ffe, uuid.RESERVED_FUTURE, None), + ('ffffffff-ffff-ffff-ffff-ffffffffffff', + '{ffffffff-ffff-ffff-ffff-ffffffffffff}', + 'ffffffffffffffffffffffffffffffff', + '\xff'*16, + (0xffffffffL, 0xffffL, 0xffffL, 0xff, 0xff, 0xffffffffffffL), + 0xffffffffffffffffffffffffffffffffL, + 'urn:uuid:ffffffff-ffff-ffff-ffff-ffffffffffff', + 0xfffffffffffffffL, 0x3fff, uuid.RESERVED_FUTURE, None), + ]: + equivalents = [] + # Construct each UUID in several different ways. + for u in [uuid.UUID(string), uuid.UUID(curly), uuid.UUID(hex), + uuid.UUID(bytes=bytes), uuid.UUID(fields=fields), + uuid.UUID(int=integer), uuid.UUID(urn)]: + # Test all conversions and properties of the UUID object. + equal(str(u), string) + equal(int(u), integer) + equal(u.bytes, bytes) + equal(u.fields, fields) + equal(u.time_low, fields[0]) + equal(u.time_mid, fields[1]) + equal(u.time_hi_version, fields[2]) + equal(u.clock_seq_hi_variant, fields[3]) + equal(u.clock_seq_low, fields[4]) + equal(u.node, fields[5]) + equal(u.hex, hex) + equal(u.int, integer) + equal(u.urn, urn) + equal(u.time, time) + equal(u.clock_seq, clock_seq) + equal(u.variant, variant) + equal(u.version, version) + equivalents.append(u) + + # Different construction methods should give the same UUID. + for u in equivalents: + for v in equivalents: + equal(u, v) + ascending.append(u) + + # Test comparison of UUIDs. + for i in range(len(ascending)): + for j in range(len(ascending)): + equal(cmp(i, j), cmp(ascending[i], ascending[j])) + + # Test sorting of UUIDs (above list is in ascending order). + resorted = ascending[:] + resorted.reverse() + resorted.sort() + equal(ascending, resorted) + + def test_exceptions(self): + badvalue = lambda f: self.assertRaises(ValueError, f) + badtype = lambda f: self.assertRaises(TypeError, f) + + # Badly formed hex strings. + badvalue(lambda: uuid.UUID('')) + badvalue(lambda: uuid.UUID('abc')) + badvalue(lambda: uuid.UUID('1234567812345678123456781234567')) + badvalue(lambda: uuid.UUID('123456781234567812345678123456789')) + badvalue(lambda: uuid.UUID('123456781234567812345678z2345678')) + + # Badly formed bytes. + badvalue(lambda: uuid.UUID(bytes='abc')) + badvalue(lambda: uuid.UUID(bytes='\0'*15)) + badvalue(lambda: uuid.UUID(bytes='\0'*17)) + + # Badly formed fields. + badvalue(lambda: uuid.UUID(fields=(1,))) + badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5))) + badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5, 6, 7))) + + # Field values out of range. + badvalue(lambda: uuid.UUID(fields=(-1, 0, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0x100000000L, 0, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, -1, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0x10000L, 0, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, -1, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0x10000L, 0, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, -1, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0x100L, 0, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, -1, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0x100L, 0))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, -1))) + badvalue(lambda: uuid.UUID(fields=(0, 0, 0, 0, 0, 0x1000000000000L))) + + # Version number out of range. + badvalue(lambda: uuid.UUID('00'*16, version=0)) + badvalue(lambda: uuid.UUID('00'*16, version=6)) + + # Integer value out of range. + badvalue(lambda: uuid.UUID(int=-1)) + badvalue(lambda: uuid.UUID(int=1<<128L)) + + # Must supply exactly one of hex, bytes, fields, int. + h, b, f, i = '00'*16, '\0'*16, (0, 0, 0, 0, 0, 0), 0 + uuid.UUID(h) + uuid.UUID(hex=h) + uuid.UUID(bytes=b) + uuid.UUID(fields=f) + uuid.UUID(int=i) + + # Wrong number of arguments (positional). + badtype(lambda: uuid.UUID()) + badtype(lambda: uuid.UUID(h, b)) + badtype(lambda: uuid.UUID(h, b, f)) + badtype(lambda: uuid.UUID(h, b, f, i)) + + # Duplicate arguments (named). + badtype(lambda: uuid.UUID(hex=h, bytes=b)) + badtype(lambda: uuid.UUID(hex=h, fields=f)) + badtype(lambda: uuid.UUID(hex=h, int=i)) + badtype(lambda: uuid.UUID(bytes=b, fields=f)) + badtype(lambda: uuid.UUID(bytes=b, int=i)) + badtype(lambda: uuid.UUID(fields=f, int=i)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(hex=h, fields=f, int=i)) + badtype(lambda: uuid.UUID(bytes=b, int=i, fields=f)) + badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i, fields=f)) + + # Duplicate arguments (positional and named). + badtype(lambda: uuid.UUID(h, hex=h)) + badtype(lambda: uuid.UUID(h, bytes=b)) + badtype(lambda: uuid.UUID(h, fields=f)) + badtype(lambda: uuid.UUID(h, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b)) + badtype(lambda: uuid.UUID(h, hex=h, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, int=i)) + badtype(lambda: uuid.UUID(h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(h, fields=f, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i)) + badtype(lambda: uuid.UUID(h, hex=h, fields=f, int=i)) + badtype(lambda: uuid.UUID(h, bytes=b, int=i, fields=f)) + badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i, fields=f)) + + # Immutability. + u = uuid.UUID(h) + badtype(lambda: setattr(u, 'hex', h)) + badtype(lambda: setattr(u, 'bytes', b)) + badtype(lambda: setattr(u, 'fields', f)) + badtype(lambda: setattr(u, 'int', i)) + + def check_node(self, node, source): + individual_group_bit = (node >> 40L) & 1 + universal_local_bit = (node >> 40L) & 2 + message = "%012x doesn't look like a real MAC address" % node + self.assertEqual(individual_group_bit, 0, message) + self.assertEqual(universal_local_bit, 0, message) + self.assertNotEqual(node, 0, message) + self.assertNotEqual(node, 0xffffffffffffL, message) + self.assert_(0 <= node, message) + self.assert_(node < (1L << 48), message) + + TestUUID.source2node[source] = node + if TestUUID.last_node: + if TestUUID.last_node != node: + msg = "different sources disagree on node:\n" + for s, n in TestUUID.source2node.iteritems(): + msg += " from source %r, node was %012x\n" % (s, n) + # There's actually no reason to expect the MAC addresses + # to agree across various methods -- e.g., a box may have + # multiple network interfaces, and different ways of getting + # a MAC address may favor different HW. + ##self.fail(msg) + else: + TestUUID.last_node = node + + def test_ifconfig_getnode(self): + import sys + print >>sys.__stdout__, \ +""" WARNING: uuid._ifconfig_getnode is unreliable on many platforms. + It is disabled until the code and/or test can be fixed properly.""" + return + + import os + if os.name == 'posix': + node = uuid._ifconfig_getnode() + if node is not None: + self.check_node(node, 'ifconfig') + + def test_ipconfig_getnode(self): + import os + if os.name == 'nt': + node = uuid._ipconfig_getnode() + if node is not None: + self.check_node(node, 'ipconfig') + + def test_netbios_getnode(self): + if importable('win32wnet') and importable('netbios'): + self.check_node(uuid._netbios_getnode(), 'netbios') + + def test_random_getnode(self): + node = uuid._random_getnode() + self.assert_(0 <= node) + self.assert_(node < (1L <<48)) + + def test_unixdll_getnode(self): + import sys + print >>sys.__stdout__, \ +""" WARNING: uuid._unixdll_getnode is unreliable on many platforms. + It is disabled until the code and/or test can be fixed properly.""" + return + + import os + if importable('ctypes') and os.name == 'posix': + self.check_node(uuid._unixdll_getnode(), 'unixdll') + + def test_windll_getnode(self): + import os + if importable('ctypes') and os.name == 'nt': + self.check_node(uuid._windll_getnode(), 'windll') + + def test_getnode(self): + import sys + print >>sys.__stdout__, \ +""" WARNING: uuid.getnode is unreliable on many platforms. + It is disabled until the code and/or test can be fixed properly.""" + return + + node1 = uuid.getnode() + self.check_node(node1, "getnode1") + + # Test it again to ensure consistency. + node2 = uuid.getnode() + self.check_node(node2, "getnode2") + + self.assertEqual(node1, node2) + + def test_uuid1(self): + equal = self.assertEqual + + # Make sure uuid4() generates UUIDs that are actually version 1. + for u in [uuid.uuid1() for i in range(10)]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 1) + + # Make sure the supplied node ID appears in the UUID. + u = uuid.uuid1(0) + equal(u.node, 0) + u = uuid.uuid1(0x123456789abc) + equal(u.node, 0x123456789abc) + u = uuid.uuid1(0xffffffffffff) + equal(u.node, 0xffffffffffff) + + # Make sure the supplied clock sequence appears in the UUID. + u = uuid.uuid1(0x123456789abc, 0) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | u.clock_seq_low, 0) + u = uuid.uuid1(0x123456789abc, 0x1234) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | + u.clock_seq_low, 0x1234) + u = uuid.uuid1(0x123456789abc, 0x3fff) + equal(u.node, 0x123456789abc) + equal(((u.clock_seq_hi_variant & 0x3f) << 8) | + u.clock_seq_low, 0x3fff) + + def test_uuid3(self): + equal = self.assertEqual + + # Test some known version-3 UUIDs. + for u, v in [(uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org'), + '6fa459ea-ee8a-3ca4-894e-db77e160355e'), + (uuid.uuid3(uuid.NAMESPACE_URL, 'http://python.org/'), + '9fe8e8c4-aaa8-32a9-a55c-4535a88b748d'), + (uuid.uuid3(uuid.NAMESPACE_OID, '1.3.6.1'), + 'dd1a1cef-13d5-368a-ad82-eca71acd4cd1'), + (uuid.uuid3(uuid.NAMESPACE_X500, 'c=ca'), + '658d3002-db6b-3040-a1d1-8ddd7d189a4d'), + ]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 3) + equal(u, uuid.UUID(v)) + equal(str(u), v) + + def test_uuid4(self): + equal = self.assertEqual + + # Make sure uuid4() generates UUIDs that are actually version 4. + for u in [uuid.uuid4() for i in range(10)]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 4) + + def test_uuid5(self): + equal = self.assertEqual + + # Test some known version-5 UUIDs. + for u, v in [(uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org'), + '886313e1-3b8a-5372-9b90-0c9aee199e5d'), + (uuid.uuid5(uuid.NAMESPACE_URL, 'http://python.org/'), + '4c565f0d-3f5a-5890-b41b-20cf47701c5e'), + (uuid.uuid5(uuid.NAMESPACE_OID, '1.3.6.1'), + '1447fa61-5277-5fef-a9b3-fbc6e44f4af3'), + (uuid.uuid5(uuid.NAMESPACE_X500, 'c=ca'), + 'cc957dd1-a972-5349-98cd-874190002798'), + ]: + equal(u.variant, uuid.RFC_4122) + equal(u.version, 5) + equal(u, uuid.UUID(v)) + equal(str(u), v) + + +def test_main(): + test_support.run_unittest(TestUUID) + +if __name__ == '__main__': + test_main() diff --git a/Lib/test/test_wait3.py b/Lib/test/test_wait3.py index f6a41a6..9de64b2 100644 --- a/Lib/test/test_wait3.py +++ b/Lib/test/test_wait3.py @@ -2,8 +2,9 @@ """ import os +import time from test.fork_wait import ForkWait -from test.test_support import TestSkipped, run_unittest +from test.test_support import TestSkipped, run_unittest, reap_children try: os.fork @@ -17,16 +18,21 @@ except AttributeError: class Wait3Test(ForkWait): def wait_impl(self, cpid): - while 1: - spid, status, rusage = os.wait3(0) + for i in range(10): + # wait3() shouldn't hang, but some of the buildbots seem to hang + # in the forking tests. This is an attempt to fix the problem. + spid, status, rusage = os.wait3(os.WNOHANG) if spid == cpid: break + time.sleep(1.0) + self.assertEqual(spid, cpid) self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) self.assertTrue(rusage) def test_main(): run_unittest(Wait3Test) + reap_children() if __name__ == "__main__": test_main() diff --git a/Lib/test/test_wait4.py b/Lib/test/test_wait4.py index 027e5c3..9f7fc14 100644 --- a/Lib/test/test_wait4.py +++ b/Lib/test/test_wait4.py @@ -2,8 +2,9 @@ """ import os +import time from test.fork_wait import ForkWait -from test.test_support import TestSkipped, run_unittest +from test.test_support import TestSkipped, run_unittest, reap_children try: os.fork @@ -17,13 +18,20 @@ except AttributeError: class Wait4Test(ForkWait): def wait_impl(self, cpid): - spid, status, rusage = os.wait4(cpid, 0) + for i in range(10): + # wait4() shouldn't hang, but some of the buildbots seem to hang + # in the forking tests. This is an attempt to fix the problem. + spid, status, rusage = os.wait4(cpid, os.WNOHANG) + if spid == cpid: + break + time.sleep(1.0) self.assertEqual(spid, cpid) self.assertEqual(status, 0, "cause = %d, exit = %d" % (status&0xff, status>>8)) self.assertTrue(rusage) def test_main(): run_unittest(Wait4Test) + reap_children() if __name__ == "__main__": test_main() diff --git a/Lib/test/test_warnings.py b/Lib/test/test_warnings.py index 5d051a5..a7ccb6b 100644 --- a/Lib/test/test_warnings.py +++ b/Lib/test/test_warnings.py @@ -81,6 +81,19 @@ class TestModule(unittest.TestCase): self.assertEqual(msg.message, text) self.assertEqual(msg.category, 'UserWarning') + def test_options(self): + # Uses the private _setoption() function to test the parsing + # of command-line warning arguments + self.assertRaises(warnings._OptionError, + warnings._setoption, '1:2:3:4:5:6') + self.assertRaises(warnings._OptionError, + warnings._setoption, 'bogus::Warning') + self.assertRaises(warnings._OptionError, + warnings._setoption, 'ignore:2::4:-5') + warnings._setoption('error::Warning::0') + self.assertRaises(UserWarning, warnings.warn, 'convert to error') + + def test_main(verbose=None): # Obscure hack so that this test passes after reloads or repeated calls # to test_main (regrtest -R). diff --git a/Lib/test/test_winreg.py b/Lib/test/test_winreg.py index a9bc962..5830fd6 100644 --- a/Lib/test/test_winreg.py +++ b/Lib/test/test_winreg.py @@ -151,3 +151,6 @@ if remote_name is not None: else: print "Remote registry calls can be tested using", print "'test_winreg.py --remote \\\\machine_name'" + # perform minimal ConnectRegistry test which just invokes it + h = ConnectRegistry(None, HKEY_LOCAL_MACHINE) + h.Close() diff --git a/Lib/test/test_wsgiref.py b/Lib/test/test_wsgiref.py new file mode 100755 index 0000000..1ec271b --- /dev/null +++ b/Lib/test/test_wsgiref.py @@ -0,0 +1,615 @@ +from __future__ import nested_scopes # Backward compat for 2.1 +from unittest import TestSuite, TestCase, makeSuite +from wsgiref.util import setup_testing_defaults +from wsgiref.headers import Headers +from wsgiref.handlers import BaseHandler, BaseCGIHandler +from wsgiref import util +from wsgiref.validate import validator +from wsgiref.simple_server import WSGIServer, WSGIRequestHandler, demo_app +from wsgiref.simple_server import make_server +from StringIO import StringIO +from SocketServer import BaseServer +import re, sys + + +class MockServer(WSGIServer): + """Non-socket HTTP server""" + + def __init__(self, server_address, RequestHandlerClass): + BaseServer.__init__(self, server_address, RequestHandlerClass) + self.server_bind() + + def server_bind(self): + host, port = self.server_address + self.server_name = host + self.server_port = port + self.setup_environ() + + +class MockHandler(WSGIRequestHandler): + """Non-socket HTTP handler""" + def setup(self): + self.connection = self.request + self.rfile, self.wfile = self.connection + + def finish(self): + pass + + + + + +def hello_app(environ,start_response): + start_response("200 OK", [ + ('Content-Type','text/plain'), + ('Date','Mon, 05 Jun 2006 18:49:54 GMT') + ]) + return ["Hello, world!"] + +def run_amock(app=hello_app, data="GET / HTTP/1.0\n\n"): + server = make_server("", 80, app, MockServer, MockHandler) + inp, out, err, olderr = StringIO(data), StringIO(), StringIO(), sys.stderr + sys.stderr = err + + try: + server.finish_request((inp,out), ("127.0.0.1",8888)) + finally: + sys.stderr = olderr + + return out.getvalue(), err.getvalue() + + + + + + + + + + + + + + + + + + + + + + + +def compare_generic_iter(make_it,match): + """Utility to compare a generic 2.1/2.2+ iterator with an iterable + + If running under Python 2.2+, this tests the iterator using iter()/next(), + as well as __getitem__. 'make_it' must be a function returning a fresh + iterator to be tested (since this may test the iterator twice).""" + + it = make_it() + n = 0 + for item in match: + if not it[n]==item: raise AssertionError + n+=1 + try: + it[n] + except IndexError: + pass + else: + raise AssertionError("Too many items from __getitem__",it) + + try: + iter, StopIteration + except NameError: + pass + else: + # Only test iter mode under 2.2+ + it = make_it() + if not iter(it) is it: raise AssertionError + for item in match: + if not it.next()==item: raise AssertionError + try: + it.next() + except StopIteration: + pass + else: + raise AssertionError("Too many items from .next()",it) + + + + + + +class IntegrationTests(TestCase): + + def check_hello(self, out, has_length=True): + self.assertEqual(out, + "HTTP/1.0 200 OK\r\n" + "Server: WSGIServer/0.1 Python/"+sys.version.split()[0]+"\r\n" + "Content-Type: text/plain\r\n" + "Date: Mon, 05 Jun 2006 18:49:54 GMT\r\n" + + (has_length and "Content-Length: 13\r\n" or "") + + "\r\n" + "Hello, world!" + ) + + def test_plain_hello(self): + out, err = run_amock() + self.check_hello(out) + + def test_validated_hello(self): + out, err = run_amock(validator(hello_app)) + # the middleware doesn't support len(), so content-length isn't there + self.check_hello(out, has_length=False) + + def test_simple_validation_error(self): + def bad_app(environ,start_response): + start_response("200 OK", ('Content-Type','text/plain')) + return ["Hello, world!"] + out, err = run_amock(validator(bad_app)) + self.failUnless(out.endswith( + "A server error occurred. Please contact the administrator." + )) + self.assertEqual( + err.splitlines()[-2], + "AssertionError: Headers (('Content-Type', 'text/plain')) must" + " be of type list: " + ) + + + + + + +class UtilityTests(TestCase): + + def checkShift(self,sn_in,pi_in,part,sn_out,pi_out): + env = {'SCRIPT_NAME':sn_in,'PATH_INFO':pi_in} + util.setup_testing_defaults(env) + self.assertEqual(util.shift_path_info(env),part) + self.assertEqual(env['PATH_INFO'],pi_out) + self.assertEqual(env['SCRIPT_NAME'],sn_out) + return env + + def checkDefault(self, key, value, alt=None): + # Check defaulting when empty + env = {} + util.setup_testing_defaults(env) + if isinstance(value,StringIO): + self.failUnless(isinstance(env[key],StringIO)) + else: + self.assertEqual(env[key],value) + + # Check existing value + env = {key:alt} + util.setup_testing_defaults(env) + self.failUnless(env[key] is alt) + + def checkCrossDefault(self,key,value,**kw): + util.setup_testing_defaults(kw) + self.assertEqual(kw[key],value) + + def checkAppURI(self,uri,**kw): + util.setup_testing_defaults(kw) + self.assertEqual(util.application_uri(kw),uri) + + def checkReqURI(self,uri,query=1,**kw): + util.setup_testing_defaults(kw) + self.assertEqual(util.request_uri(kw,query),uri) + + + + + + + def checkFW(self,text,size,match): + + def make_it(text=text,size=size): + return util.FileWrapper(StringIO(text),size) + + compare_generic_iter(make_it,match) + + it = make_it() + self.failIf(it.filelike.closed) + + for item in it: + pass + + self.failIf(it.filelike.closed) + + it.close() + self.failUnless(it.filelike.closed) + + + def testSimpleShifts(self): + self.checkShift('','/', '', '/', '') + self.checkShift('','/x', 'x', '/x', '') + self.checkShift('/','', None, '/', '') + self.checkShift('/a','/x/y', 'x', '/a/x', '/y') + self.checkShift('/a','/x/', 'x', '/a/x', '/') + + + def testNormalizedShifts(self): + self.checkShift('/a/b', '/../y', '..', '/a', '/y') + self.checkShift('', '/../y', '..', '', '/y') + self.checkShift('/a/b', '//y', 'y', '/a/b/y', '') + self.checkShift('/a/b', '//y/', 'y', '/a/b/y', '/') + self.checkShift('/a/b', '/./y', 'y', '/a/b/y', '') + self.checkShift('/a/b', '/./y/', 'y', '/a/b/y', '/') + self.checkShift('/a/b', '///./..//y/.//', '..', '/a', '/y/') + self.checkShift('/a/b', '///', '', '/a/b/', '') + self.checkShift('/a/b', '/.//', '', '/a/b/', '') + self.checkShift('/a/b', '/x//', 'x', '/a/b/x', '/') + self.checkShift('/a/b', '/.', None, '/a/b', '') + + + def testDefaults(self): + for key, value in [ + ('SERVER_NAME','127.0.0.1'), + ('SERVER_PORT', '80'), + ('SERVER_PROTOCOL','HTTP/1.0'), + ('HTTP_HOST','127.0.0.1'), + ('REQUEST_METHOD','GET'), + ('SCRIPT_NAME',''), + ('PATH_INFO','/'), + ('wsgi.version', (1,0)), + ('wsgi.run_once', 0), + ('wsgi.multithread', 0), + ('wsgi.multiprocess', 0), + ('wsgi.input', StringIO("")), + ('wsgi.errors', StringIO()), + ('wsgi.url_scheme','http'), + ]: + self.checkDefault(key,value) + + + def testCrossDefaults(self): + self.checkCrossDefault('HTTP_HOST',"foo.bar",SERVER_NAME="foo.bar") + self.checkCrossDefault('wsgi.url_scheme',"https",HTTPS="on") + self.checkCrossDefault('wsgi.url_scheme',"https",HTTPS="1") + self.checkCrossDefault('wsgi.url_scheme',"https",HTTPS="yes") + self.checkCrossDefault('wsgi.url_scheme',"http",HTTPS="foo") + self.checkCrossDefault('SERVER_PORT',"80",HTTPS="foo") + self.checkCrossDefault('SERVER_PORT',"443",HTTPS="on") + + + def testGuessScheme(self): + self.assertEqual(util.guess_scheme({}), "http") + self.assertEqual(util.guess_scheme({'HTTPS':"foo"}), "http") + self.assertEqual(util.guess_scheme({'HTTPS':"on"}), "https") + self.assertEqual(util.guess_scheme({'HTTPS':"yes"}), "https") + self.assertEqual(util.guess_scheme({'HTTPS':"1"}), "https") + + + + + + def testAppURIs(self): + self.checkAppURI("http://127.0.0.1/") + self.checkAppURI("http://127.0.0.1/spam", SCRIPT_NAME="/spam") + self.checkAppURI("http://spam.example.com:2071/", + HTTP_HOST="spam.example.com:2071", SERVER_PORT="2071") + self.checkAppURI("http://spam.example.com/", + SERVER_NAME="spam.example.com") + self.checkAppURI("http://127.0.0.1/", + HTTP_HOST="127.0.0.1", SERVER_NAME="spam.example.com") + self.checkAppURI("https://127.0.0.1/", HTTPS="on") + self.checkAppURI("http://127.0.0.1:8000/", SERVER_PORT="8000", + HTTP_HOST=None) + + def testReqURIs(self): + self.checkReqURI("http://127.0.0.1/") + self.checkReqURI("http://127.0.0.1/spam", SCRIPT_NAME="/spam") + self.checkReqURI("http://127.0.0.1/spammity/spam", + SCRIPT_NAME="/spammity", PATH_INFO="/spam") + self.checkReqURI("http://127.0.0.1/spammity/spam?say=ni", + SCRIPT_NAME="/spammity", PATH_INFO="/spam",QUERY_STRING="say=ni") + self.checkReqURI("http://127.0.0.1/spammity/spam", 0, + SCRIPT_NAME="/spammity", PATH_INFO="/spam",QUERY_STRING="say=ni") + + def testFileWrapper(self): + self.checkFW("xyz"*50, 120, ["xyz"*40,"xyz"*10]) + + def testHopByHop(self): + for hop in ( + "Connection Keep-Alive Proxy-Authenticate Proxy-Authorization " + "TE Trailers Transfer-Encoding Upgrade" + ).split(): + for alt in hop, hop.title(), hop.upper(), hop.lower(): + self.failUnless(util.is_hop_by_hop(alt)) + + # Not comprehensive, just a few random header names + for hop in ( + "Accept Cache-Control Date Pragma Trailer Via Warning" + ).split(): + for alt in hop, hop.title(), hop.upper(), hop.lower(): + self.failIf(util.is_hop_by_hop(alt)) + +class HeaderTests(TestCase): + + def testMappingInterface(self): + test = [('x','y')] + self.assertEqual(len(Headers([])),0) + self.assertEqual(len(Headers(test[:])),1) + self.assertEqual(Headers(test[:]).keys(), ['x']) + self.assertEqual(Headers(test[:]).values(), ['y']) + self.assertEqual(Headers(test[:]).items(), test) + self.failIf(Headers(test).items() is test) # must be copy! + + h=Headers([]) + del h['foo'] # should not raise an error + + h['Foo'] = 'bar' + for m in h.has_key, h.__contains__, h.get, h.get_all, h.__getitem__: + self.failUnless(m('foo')) + self.failUnless(m('Foo')) + self.failUnless(m('FOO')) + self.failIf(m('bar')) + + self.assertEqual(h['foo'],'bar') + h['foo'] = 'baz' + self.assertEqual(h['FOO'],'baz') + self.assertEqual(h.get_all('foo'),['baz']) + + self.assertEqual(h.get("foo","whee"), "baz") + self.assertEqual(h.get("zoo","whee"), "whee") + self.assertEqual(h.setdefault("foo","whee"), "baz") + self.assertEqual(h.setdefault("zoo","whee"), "whee") + self.assertEqual(h["foo"],"baz") + self.assertEqual(h["zoo"],"whee") + + def testRequireList(self): + self.assertRaises(TypeError, Headers, "foo") + + + def testExtras(self): + h = Headers([]) + self.assertEqual(str(h),'\r\n') + + h.add_header('foo','bar',baz="spam") + self.assertEqual(h['foo'], 'bar; baz="spam"') + self.assertEqual(str(h),'foo: bar; baz="spam"\r\n\r\n') + + h.add_header('Foo','bar',cheese=None) + self.assertEqual(h.get_all('foo'), + ['bar; baz="spam"', 'bar; cheese']) + + self.assertEqual(str(h), + 'foo: bar; baz="spam"\r\n' + 'Foo: bar; cheese\r\n' + '\r\n' + ) + + +class ErrorHandler(BaseCGIHandler): + """Simple handler subclass for testing BaseHandler""" + + def __init__(self,**kw): + setup_testing_defaults(kw) + BaseCGIHandler.__init__( + self, StringIO(''), StringIO(), StringIO(), kw, + multithread=True, multiprocess=True + ) + +class TestHandler(ErrorHandler): + """Simple handler subclass for testing BaseHandler, w/error passthru""" + + def handle_error(self): + raise # for testing, we want to see what's happening + + + + + + + + + + + +class HandlerTests(TestCase): + + def checkEnvironAttrs(self, handler): + env = handler.environ + for attr in [ + 'version','multithread','multiprocess','run_once','file_wrapper' + ]: + if attr=='file_wrapper' and handler.wsgi_file_wrapper is None: + continue + self.assertEqual(getattr(handler,'wsgi_'+attr),env['wsgi.'+attr]) + + def checkOSEnviron(self,handler): + empty = {}; setup_testing_defaults(empty) + env = handler.environ + from os import environ + for k,v in environ.items(): + if not empty.has_key(k): + self.assertEqual(env[k],v) + for k,v in empty.items(): + self.failUnless(env.has_key(k)) + + def testEnviron(self): + h = TestHandler(X="Y") + h.setup_environ() + self.checkEnvironAttrs(h) + self.checkOSEnviron(h) + self.assertEqual(h.environ["X"],"Y") + + def testCGIEnviron(self): + h = BaseCGIHandler(None,None,None,{}) + h.setup_environ() + for key in 'wsgi.url_scheme', 'wsgi.input', 'wsgi.errors': + self.assert_(h.environ.has_key(key)) + + def testScheme(self): + h=TestHandler(HTTPS="on"); h.setup_environ() + self.assertEqual(h.environ['wsgi.url_scheme'],'https') + h=TestHandler(); h.setup_environ() + self.assertEqual(h.environ['wsgi.url_scheme'],'http') + + + def testAbstractMethods(self): + h = BaseHandler() + for name in [ + '_flush','get_stdin','get_stderr','add_cgi_vars' + ]: + self.assertRaises(NotImplementedError, getattr(h,name)) + self.assertRaises(NotImplementedError, h._write, "test") + + + def testContentLength(self): + # Demo one reason iteration is better than write()... ;) + + def trivial_app1(e,s): + s('200 OK',[]) + return [e['wsgi.url_scheme']] + + def trivial_app2(e,s): + s('200 OK',[])(e['wsgi.url_scheme']) + return [] + + h = TestHandler() + h.run(trivial_app1) + self.assertEqual(h.stdout.getvalue(), + "Status: 200 OK\r\n" + "Content-Length: 4\r\n" + "\r\n" + "http") + + h = TestHandler() + h.run(trivial_app2) + self.assertEqual(h.stdout.getvalue(), + "Status: 200 OK\r\n" + "\r\n" + "http") + + + + + + + + def testBasicErrorOutput(self): + + def non_error_app(e,s): + s('200 OK',[]) + return [] + + def error_app(e,s): + raise AssertionError("This should be caught by handler") + + h = ErrorHandler() + h.run(non_error_app) + self.assertEqual(h.stdout.getvalue(), + "Status: 200 OK\r\n" + "Content-Length: 0\r\n" + "\r\n") + self.assertEqual(h.stderr.getvalue(),"") + + h = ErrorHandler() + h.run(error_app) + self.assertEqual(h.stdout.getvalue(), + "Status: %s\r\n" + "Content-Type: text/plain\r\n" + "Content-Length: %d\r\n" + "\r\n%s" % (h.error_status,len(h.error_body),h.error_body)) + + self.failUnless(h.stderr.getvalue().find("AssertionError")<>-1) + + def testErrorAfterOutput(self): + MSG = "Some output has been sent" + def error_app(e,s): + s("200 OK",[])(MSG) + raise AssertionError("This should be caught by handler") + + h = ErrorHandler() + h.run(error_app) + self.assertEqual(h.stdout.getvalue(), + "Status: 200 OK\r\n" + "\r\n"+MSG) + self.failUnless(h.stderr.getvalue().find("AssertionError")<>-1) + + + def testHeaderFormats(self): + + def non_error_app(e,s): + s('200 OK',[]) + return [] + + stdpat = ( + r"HTTP/%s 200 OK\r\n" + r"Date: \w{3}, [ 0123]\d \w{3} \d{4} \d\d:\d\d:\d\d GMT\r\n" + r"%s" r"Content-Length: 0\r\n" r"\r\n" + ) + shortpat = ( + "Status: 200 OK\r\n" "Content-Length: 0\r\n" "\r\n" + ) + + for ssw in "FooBar/1.0", None: + sw = ssw and "Server: %s\r\n" % ssw or "" + + for version in "1.0", "1.1": + for proto in "HTTP/0.9", "HTTP/1.0", "HTTP/1.1": + + h = TestHandler(SERVER_PROTOCOL=proto) + h.origin_server = False + h.http_version = version + h.server_software = ssw + h.run(non_error_app) + self.assertEqual(shortpat,h.stdout.getvalue()) + + h = TestHandler(SERVER_PROTOCOL=proto) + h.origin_server = True + h.http_version = version + h.server_software = ssw + h.run(non_error_app) + if proto=="HTTP/0.9": + self.assertEqual(h.stdout.getvalue(),"") + else: + self.failUnless( + re.match(stdpat%(version,sw), h.stdout.getvalue()), + (stdpat%(version,sw), h.stdout.getvalue()) + ) + +# This epilogue is needed for compatibility with the Python 2.5 regrtest module + +def test_main(): + import unittest + from test.test_support import run_suite + run_suite( + unittest.defaultTestLoader.loadTestsFromModule(sys.modules[__name__]) + ) + +if __name__ == "__main__": + test_main() + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +# the above lines intentionally left blank diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index 86052d7..78adb42 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -1,4 +1,4 @@ -# xmlcore.etree test. This file contains enough tests to make sure that +# xml.etree test. This file contains enough tests to make sure that # all included components work as they should. For a more extensive # test suite, see the selftest script in the ElementTree distribution. @@ -6,8 +6,6 @@ import doctest, sys from test import test_support -from xmlcore.etree import ElementTree as ET - SAMPLE_XML = """ text @@ -32,9 +30,9 @@ def sanity(): """ Import sanity. - >>> from xmlcore.etree import ElementTree - >>> from xmlcore.etree import ElementInclude - >>> from xmlcore.etree import ElementPath + >>> from xml.etree import ElementTree + >>> from xml.etree import ElementInclude + >>> from xml.etree import ElementPath """ def check_method(method): @@ -61,6 +59,8 @@ def interface(): """ Test element tree interface. + >>> from xml.etree import ElementTree as ET + >>> element = ET.Element("tag", key="value") >>> tree = ET.ElementTree(element) @@ -108,6 +108,8 @@ def find(): """ Test find methods (including xpath syntax). + >>> from xml.etree import ElementTree as ET + >>> elem = ET.XML(SAMPLE_XML) >>> elem.find("tag").tag 'tag' @@ -174,6 +176,8 @@ def find(): def parseliteral(): r""" + >>> from xml.etree import ElementTree as ET + >>> element = ET.XML("text") >>> ET.ElementTree(element).write(sys.stdout) text @@ -195,18 +199,20 @@ def parseliteral(): 'body' """ -def check_encoding(encoding): + +def check_encoding(ET, encoding): """ - >>> check_encoding("ascii") - >>> check_encoding("us-ascii") - >>> check_encoding("iso-8859-1") - >>> check_encoding("iso-8859-15") - >>> check_encoding("cp437") - >>> check_encoding("mac-roman") + >>> from xml.etree import ElementTree as ET + + >>> check_encoding(ET, "ascii") + >>> check_encoding(ET, "us-ascii") + >>> check_encoding(ET, "iso-8859-1") + >>> check_encoding(ET, "iso-8859-15") + >>> check_encoding(ET, "cp437") + >>> check_encoding(ET, "mac-roman") """ - ET.XML( - "" % encoding - ) + ET.XML("" % encoding) + # # xinclude tests (samples from appendix C of the xinclude specification) @@ -282,14 +288,16 @@ def xinclude_loader(href, parse="xml", encoding=None): except KeyError: raise IOError("resource not found") if parse == "xml": - return ET.XML(data) + from xml.etree.ElementTree import XML + return XML(data) return data def xinclude(): r""" Basic inclusion example (XInclude C.1) - >>> from xmlcore.etree import ElementInclude + >>> from xml.etree import ElementTree as ET + >>> from xml.etree import ElementInclude >>> document = xinclude_loader("C1.xml") >>> ElementInclude.include(document, xinclude_loader) diff --git a/Lib/test/test_xml_etree_c.py b/Lib/test/test_xml_etree_c.py index 587ea99..56e7fed 100644 --- a/Lib/test/test_xml_etree_c.py +++ b/Lib/test/test_xml_etree_c.py @@ -1,10 +1,10 @@ -# xmlcore.etree test for cElementTree +# xml.etree test for cElementTree import doctest, sys from test import test_support -from xmlcore.etree import cElementTree as ET +from xml.etree import cElementTree as ET SAMPLE_XML = """ @@ -30,7 +30,7 @@ def sanity(): """ Import sanity. - >>> from xmlcore.etree import cElementTree + >>> from xml.etree import cElementTree """ def check_method(method): diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py index 0241348..54684f3 100644 --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -4,7 +4,7 @@ try: except ImportError: zlib = None -import zipfile, os, unittest +import zipfile, os, unittest, sys, shutil from StringIO import StringIO from tempfile import TemporaryFile @@ -28,14 +28,70 @@ class TestsWithSourceFile(unittest.TestCase): zipfp = zipfile.ZipFile(f, "w", compression) zipfp.write(TESTFN, "another"+os.extsep+"name") zipfp.write(TESTFN, TESTFN) + zipfp.writestr("strfile", self.data) zipfp.close() # Read the ZIP archive zipfp = zipfile.ZipFile(f, "r", compression) self.assertEqual(zipfp.read(TESTFN), self.data) self.assertEqual(zipfp.read("another"+os.extsep+"name"), self.data) + self.assertEqual(zipfp.read("strfile"), self.data) + + # Print the ZIP directory + fp = StringIO() + stdout = sys.stdout + try: + sys.stdout = fp + + zipfp.printdir() + finally: + sys.stdout = stdout + + directory = fp.getvalue() + lines = directory.splitlines() + self.assertEquals(len(lines), 4) # Number of files + header + + self.assert_('File Name' in lines[0]) + self.assert_('Modified' in lines[0]) + self.assert_('Size' in lines[0]) + + fn, date, time, size = lines[1].split() + self.assertEquals(fn, 'another.name') + # XXX: timestamp is not tested + self.assertEquals(size, str(len(self.data))) + + # Check the namelist + names = zipfp.namelist() + self.assertEquals(len(names), 3) + self.assert_(TESTFN in names) + self.assert_("another"+os.extsep+"name" in names) + self.assert_("strfile" in names) + + # Check infolist + infos = zipfp.infolist() + names = [ i.filename for i in infos ] + self.assertEquals(len(names), 3) + self.assert_(TESTFN in names) + self.assert_("another"+os.extsep+"name" in names) + self.assert_("strfile" in names) + for i in infos: + self.assertEquals(i.file_size, len(self.data)) + + # check getinfo + for nm in (TESTFN, "another"+os.extsep+"name", "strfile"): + info = zipfp.getinfo(nm) + self.assertEquals(info.filename, nm) + self.assertEquals(info.file_size, len(self.data)) + + # Check that testzip doesn't raise an exception + zipfp.testzip() + + zipfp.close() + + + def testStored(self): for f in (TESTFN2, TemporaryFile(), StringIO()): self.zipTest(f, zipfile.ZIP_STORED) @@ -59,6 +115,197 @@ class TestsWithSourceFile(unittest.TestCase): os.remove(TESTFN) os.remove(TESTFN2) +class TestZip64InSmallFiles(unittest.TestCase): + # These tests test the ZIP64 functionality without using large files, + # see test_zipfile64 for proper tests. + + def setUp(self): + self._limit = zipfile.ZIP64_LIMIT + zipfile.ZIP64_LIMIT = 5 + + line_gen = ("Test of zipfile line %d." % i for i in range(0, 1000)) + self.data = '\n'.join(line_gen) + + # Make a source file with some lines + fp = open(TESTFN, "wb") + fp.write(self.data) + fp.close() + + def largeFileExceptionTest(self, f, compression): + zipfp = zipfile.ZipFile(f, "w", compression) + self.assertRaises(zipfile.LargeZipFile, + zipfp.write, TESTFN, "another"+os.extsep+"name") + zipfp.close() + + def largeFileExceptionTest2(self, f, compression): + zipfp = zipfile.ZipFile(f, "w", compression) + self.assertRaises(zipfile.LargeZipFile, + zipfp.writestr, "another"+os.extsep+"name", self.data) + zipfp.close() + + def testLargeFileException(self): + for f in (TESTFN2, TemporaryFile(), StringIO()): + self.largeFileExceptionTest(f, zipfile.ZIP_STORED) + self.largeFileExceptionTest2(f, zipfile.ZIP_STORED) + + def zipTest(self, f, compression): + # Create the ZIP archive + zipfp = zipfile.ZipFile(f, "w", compression, allowZip64=True) + zipfp.write(TESTFN, "another"+os.extsep+"name") + zipfp.write(TESTFN, TESTFN) + zipfp.writestr("strfile", self.data) + zipfp.close() + + # Read the ZIP archive + zipfp = zipfile.ZipFile(f, "r", compression) + self.assertEqual(zipfp.read(TESTFN), self.data) + self.assertEqual(zipfp.read("another"+os.extsep+"name"), self.data) + self.assertEqual(zipfp.read("strfile"), self.data) + + # Print the ZIP directory + fp = StringIO() + stdout = sys.stdout + try: + sys.stdout = fp + + zipfp.printdir() + finally: + sys.stdout = stdout + + directory = fp.getvalue() + lines = directory.splitlines() + self.assertEquals(len(lines), 4) # Number of files + header + + self.assert_('File Name' in lines[0]) + self.assert_('Modified' in lines[0]) + self.assert_('Size' in lines[0]) + + fn, date, time, size = lines[1].split() + self.assertEquals(fn, 'another.name') + # XXX: timestamp is not tested + self.assertEquals(size, str(len(self.data))) + + # Check the namelist + names = zipfp.namelist() + self.assertEquals(len(names), 3) + self.assert_(TESTFN in names) + self.assert_("another"+os.extsep+"name" in names) + self.assert_("strfile" in names) + + # Check infolist + infos = zipfp.infolist() + names = [ i.filename for i in infos ] + self.assertEquals(len(names), 3) + self.assert_(TESTFN in names) + self.assert_("another"+os.extsep+"name" in names) + self.assert_("strfile" in names) + for i in infos: + self.assertEquals(i.file_size, len(self.data)) + + # check getinfo + for nm in (TESTFN, "another"+os.extsep+"name", "strfile"): + info = zipfp.getinfo(nm) + self.assertEquals(info.filename, nm) + self.assertEquals(info.file_size, len(self.data)) + + # Check that testzip doesn't raise an exception + zipfp.testzip() + + + zipfp.close() + + def testStored(self): + for f in (TESTFN2, TemporaryFile(), StringIO()): + self.zipTest(f, zipfile.ZIP_STORED) + + + if zlib: + def testDeflated(self): + for f in (TESTFN2, TemporaryFile(), StringIO()): + self.zipTest(f, zipfile.ZIP_DEFLATED) + + def testAbsoluteArcnames(self): + zipfp = zipfile.ZipFile(TESTFN2, "w", zipfile.ZIP_STORED, allowZip64=True) + zipfp.write(TESTFN, "/absolute") + zipfp.close() + + zipfp = zipfile.ZipFile(TESTFN2, "r", zipfile.ZIP_STORED) + self.assertEqual(zipfp.namelist(), ["absolute"]) + zipfp.close() + + + def tearDown(self): + zipfile.ZIP64_LIMIT = self._limit + os.remove(TESTFN) + os.remove(TESTFN2) + +class PyZipFileTests(unittest.TestCase): + def testWritePyfile(self): + zipfp = zipfile.PyZipFile(TemporaryFile(), "w") + fn = __file__ + if fn.endswith('.pyc') or fn.endswith('.pyo'): + fn = fn[:-1] + + zipfp.writepy(fn) + + bn = os.path.basename(fn) + self.assert_(bn not in zipfp.namelist()) + self.assert_(bn + 'o' in zipfp.namelist() or bn + 'c' in zipfp.namelist()) + zipfp.close() + + + zipfp = zipfile.PyZipFile(TemporaryFile(), "w") + fn = __file__ + if fn.endswith('.pyc') or fn.endswith('.pyo'): + fn = fn[:-1] + + zipfp.writepy(fn, "testpackage") + + bn = "%s/%s"%("testpackage", os.path.basename(fn)) + self.assert_(bn not in zipfp.namelist()) + self.assert_(bn + 'o' in zipfp.namelist() or bn + 'c' in zipfp.namelist()) + zipfp.close() + + def testWritePythonPackage(self): + import email + packagedir = os.path.dirname(email.__file__) + + zipfp = zipfile.PyZipFile(TemporaryFile(), "w") + zipfp.writepy(packagedir) + + # Check for a couple of modules at different levels of the hieararchy + names = zipfp.namelist() + self.assert_('email/__init__.pyo' in names or 'email/__init__.pyc' in names) + self.assert_('email/mime/text.pyo' in names or 'email/mime/text.pyc' in names) + + def testWritePythonDirectory(self): + os.mkdir(TESTFN2) + try: + fp = open(os.path.join(TESTFN2, "mod1.py"), "w") + fp.write("print 42\n") + fp.close() + + fp = open(os.path.join(TESTFN2, "mod2.py"), "w") + fp.write("print 42 * 42\n") + fp.close() + + fp = open(os.path.join(TESTFN2, "mod2.txt"), "w") + fp.write("bla bla bla\n") + fp.close() + + zipfp = zipfile.PyZipFile(TemporaryFile(), "w") + zipfp.writepy(TESTFN2) + + names = zipfp.namelist() + self.assert_('mod1.pyc' in names or 'mod1.pyo' in names) + self.assert_('mod2.pyc' in names or 'mod2.pyo' in names) + self.assert_('mod2.txt' not in names) + + finally: + shutil.rmtree(TESTFN2) + + + class OtherTests(unittest.TestCase): def testCloseErroneousFile(self): # This test checks that the ZipFile constructor closes the file object @@ -103,7 +350,8 @@ class OtherTests(unittest.TestCase): self.assertRaises(RuntimeError, zipf.testzip) def test_main(): - run_unittest(TestsWithSourceFile, OtherTests) + run_unittest(TestsWithSourceFile, TestZip64InSmallFiles, OtherTests, PyZipFileTests) + #run_unittest(TestZip64InSmallFiles) if __name__ == "__main__": test_main() diff --git a/Lib/test/test_zipfile64.py b/Lib/test/test_zipfile64.py new file mode 100644 index 0000000..449cf39 --- /dev/null +++ b/Lib/test/test_zipfile64.py @@ -0,0 +1,101 @@ +# Tests of the full ZIP64 functionality of zipfile +# The test_support.requires call is the only reason for keeping this separate +# from test_zipfile +from test import test_support +# XXX(nnorwitz): disable this test by looking for extra largfile resource +# which doesn't exist. This test takes over 30 minutes to run in general +# and requires more disk space than most of the buildbots. +test_support.requires( + 'extralargefile', + 'test requires loads of disk-space bytes and a long time to run' + ) + +# We can test part of the module without zlib. +try: + import zlib +except ImportError: + zlib = None + +import zipfile, os, unittest +import time +import sys + +from StringIO import StringIO +from tempfile import TemporaryFile + +from test.test_support import TESTFN, run_unittest + +TESTFN2 = TESTFN + "2" + +# How much time in seconds can pass before we print a 'Still working' message. +_PRINT_WORKING_MSG_INTERVAL = 5 * 60 + +class TestsWithSourceFile(unittest.TestCase): + def setUp(self): + # Create test data. + # xrange() is important here -- don't want to create immortal space + # for a million ints. + line_gen = ("Test of zipfile line %d." % i for i in xrange(1000000)) + self.data = '\n'.join(line_gen) + + # And write it to a file. + fp = open(TESTFN, "wb") + fp.write(self.data) + fp.close() + + def zipTest(self, f, compression): + # Create the ZIP archive. + zipfp = zipfile.ZipFile(f, "w", compression, allowZip64=True) + + # It will contain enough copies of self.data to reach about 6GB of + # raw data to store. + filecount = 6*1024**3 // len(self.data) + + next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL + for num in range(filecount): + zipfp.writestr("testfn%d" % num, self.data) + # Print still working message since this test can be really slow + if next_time <= time.time(): + next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL + print >>sys.__stdout__, ( + ' zipTest still writing %d of %d, be patient...' % + (num, filecount)) + sys.__stdout__.flush() + zipfp.close() + + # Read the ZIP archive + zipfp = zipfile.ZipFile(f, "r", compression) + for num in range(filecount): + self.assertEqual(zipfp.read("testfn%d" % num), self.data) + # Print still working message since this test can be really slow + if next_time <= time.time(): + next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL + print >>sys.__stdout__, ( + ' zipTest still reading %d of %d, be patient...' % + (num, filecount)) + sys.__stdout__.flush() + zipfp.close() + + def testStored(self): + # Try the temp file first. If we do TESTFN2 first, then it hogs + # gigabytes of disk space for the duration of the test. + for f in TemporaryFile(), TESTFN2: + self.zipTest(f, zipfile.ZIP_STORED) + + if zlib: + def testDeflated(self): + # Try the temp file first. If we do TESTFN2 first, then it hogs + # gigabytes of disk space for the duration of the test. + for f in TemporaryFile(), TESTFN2: + self.zipTest(f, zipfile.ZIP_DEFLATED) + + def tearDown(self): + for fname in TESTFN, TESTFN2: + if os.path.exists(fname): + os.remove(fname) + +def test_main(): + run_unittest(TestsWithSourceFile) + +if __name__ == "__main__": + test_main() diff --git a/Lib/test/test_zlib.py b/Lib/test/test_zlib.py index ccbc8fd..4440942 100644 --- a/Lib/test/test_zlib.py +++ b/Lib/test/test_zlib.py @@ -302,63 +302,65 @@ class CompressObjectTestCase(unittest.TestCase): dco = zlib.decompressobj() self.assertEqual(dco.flush(), "") # Returns nothing - def test_compresscopy(self): - # Test copying a compression object - data0 = HAMLET_SCENE - data1 = HAMLET_SCENE.swapcase() - c0 = zlib.compressobj(zlib.Z_BEST_COMPRESSION) - bufs0 = [] - bufs0.append(c0.compress(data0)) - - c1 = c0.copy() - bufs1 = bufs0[:] - - bufs0.append(c0.compress(data0)) - bufs0.append(c0.flush()) - s0 = ''.join(bufs0) - - bufs1.append(c1.compress(data1)) - bufs1.append(c1.flush()) - s1 = ''.join(bufs1) - - self.assertEqual(zlib.decompress(s0),data0+data0) - self.assertEqual(zlib.decompress(s1),data0+data1) - - def test_badcompresscopy(self): - # Test copying a compression object in an inconsistent state - c = zlib.compressobj() - c.compress(HAMLET_SCENE) - c.flush() - self.assertRaises(ValueError, c.copy) - - def test_decompresscopy(self): - # Test copying a decompression object - data = HAMLET_SCENE - comp = zlib.compress(data) - - d0 = zlib.decompressobj() - bufs0 = [] - bufs0.append(d0.decompress(comp[:32])) - - d1 = d0.copy() - bufs1 = bufs0[:] - - bufs0.append(d0.decompress(comp[32:])) - s0 = ''.join(bufs0) - - bufs1.append(d1.decompress(comp[32:])) - s1 = ''.join(bufs1) - - self.assertEqual(s0,s1) - self.assertEqual(s0,data) - - def test_baddecompresscopy(self): - # Test copying a compression object in an inconsistent state - data = zlib.compress(HAMLET_SCENE) - d = zlib.decompressobj() - d.decompress(data) - d.flush() - self.assertRaises(ValueError, d.copy) + if hasattr(zlib.compressobj(), "copy"): + def test_compresscopy(self): + # Test copying a compression object + data0 = HAMLET_SCENE + data1 = HAMLET_SCENE.swapcase() + c0 = zlib.compressobj(zlib.Z_BEST_COMPRESSION) + bufs0 = [] + bufs0.append(c0.compress(data0)) + + c1 = c0.copy() + bufs1 = bufs0[:] + + bufs0.append(c0.compress(data0)) + bufs0.append(c0.flush()) + s0 = ''.join(bufs0) + + bufs1.append(c1.compress(data1)) + bufs1.append(c1.flush()) + s1 = ''.join(bufs1) + + self.assertEqual(zlib.decompress(s0),data0+data0) + self.assertEqual(zlib.decompress(s1),data0+data1) + + def test_badcompresscopy(self): + # Test copying a compression object in an inconsistent state + c = zlib.compressobj() + c.compress(HAMLET_SCENE) + c.flush() + self.assertRaises(ValueError, c.copy) + + if hasattr(zlib.decompressobj(), "copy"): + def test_decompresscopy(self): + # Test copying a decompression object + data = HAMLET_SCENE + comp = zlib.compress(data) + + d0 = zlib.decompressobj() + bufs0 = [] + bufs0.append(d0.decompress(comp[:32])) + + d1 = d0.copy() + bufs1 = bufs0[:] + + bufs0.append(d0.decompress(comp[32:])) + s0 = ''.join(bufs0) + + bufs1.append(d1.decompress(comp[32:])) + s1 = ''.join(bufs1) + + self.assertEqual(s0,s1) + self.assertEqual(s0,data) + + def test_baddecompresscopy(self): + # Test copying a compression object in an inconsistent state + data = zlib.compress(HAMLET_SCENE) + d = zlib.decompressobj() + d.decompress(data) + d.flush() + self.assertRaises(ValueError, d.copy) def genblock(seed, length, step=1024, generator=random): """length-byte stream of random data from a seed (in step-byte blocks).""" diff --git a/Lib/textwrap.py b/Lib/textwrap.py index 7c68280..ccff2ab 100644 --- a/Lib/textwrap.py +++ b/Lib/textwrap.py @@ -317,41 +317,58 @@ def fill(text, width=70, **kwargs): # -- Loosely related functionality ------------------------------------- -def dedent(text): - """dedent(text : string) -> string - - Remove any whitespace than can be uniformly removed from the left - of every line in `text`. +_whitespace_only_re = re.compile('^[ \t]+$', re.MULTILINE) +_leading_whitespace_re = re.compile('(^[ \t]*)(?:[^ \t\n])', re.MULTILINE) - This can be used e.g. to make triple-quoted strings line up with - the left edge of screen/whatever, while still presenting it in the - source code in indented form. +def dedent(text): + """Remove any common leading whitespace from every line in `text`. - For example: + This can be used to make triple-quoted strings line up with the left + edge of the display, while still presenting them in the source code + in indented form. - def test(): - # end first line with \ to avoid the empty line! - s = '''\ - hello - world - ''' - print repr(s) # prints ' hello\n world\n ' - print repr(dedent(s)) # prints 'hello\n world\n' + Note that tabs and spaces are both treated as whitespace, but they + are not equal: the lines " hello" and "\thello" are + considered to have no common leading whitespace. (This behaviour is + new in Python 2.5; older versions of this module incorrectly + expanded tabs before searching for common leading whitespace.) """ - lines = text.expandtabs().split('\n') + # Look for the longest leading string of spaces and tabs common to + # all lines. margin = None - for line in lines: - content = line.lstrip() - if not content: - continue - indent = len(line) - len(content) + text = _whitespace_only_re.sub('', text) + indents = _leading_whitespace_re.findall(text) + for indent in indents: if margin is None: margin = indent - else: - margin = min(margin, indent) - if margin is not None and margin > 0: - for i in range(len(lines)): - lines[i] = lines[i][margin:] + # Current line more deeply indented than previous winner: + # no change (previous winner is still on top). + elif indent.startswith(margin): + pass - return '\n'.join(lines) + # Current line consistent with and no deeper than previous winner: + # it's the new winner. + elif margin.startswith(indent): + margin = indent + + # Current line and previous winner have no common whitespace: + # there is no margin. + else: + margin = "" + break + + # sanity check (testing/debugging only) + if 0 and margin: + for line in text.split("\n"): + assert not line or line.startswith(margin), \ + "line = %r, margin = %r" % (line, margin) + + if margin: + text = re.sub(r'(?m)^' + margin, '', text) + return text + +if __name__ == "__main__": + #print dedent("\tfoo\n\tbar") + #print dedent(" \thello there\n \t how are you?") + print dedent("Hello there.\n This is indented.") diff --git a/Lib/threading.py b/Lib/threading.py index c27140d..5655dde 100644 --- a/Lib/threading.py +++ b/Lib/threading.py @@ -15,7 +15,7 @@ from collections import deque # Rename some stuff so "from threading import *" is safe __all__ = ['activeCount', 'Condition', 'currentThread', 'enumerate', 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread', - 'Timer', 'setprofile', 'settrace', 'local'] + 'Timer', 'setprofile', 'settrace', 'local', 'stack_size'] _start_new_thread = thread.start_new_thread _allocate_lock = thread.allocate_lock @@ -713,6 +713,8 @@ def enumerate(): _active_limbo_lock.release() return active +from thread import stack_size + # Create the main thread object _MainThread() diff --git a/Lib/trace.py b/Lib/trace.py index ca6294e..db36e1d 100644 --- a/Lib/trace.py +++ b/Lib/trace.py @@ -285,7 +285,7 @@ class CoverageResults: if filename == "": continue - if filename.endswith(".pyc") or filename.endswith(".pyo"): + if filename.endswith((".pyc", ".pyo")): filename = filename[:-1] if coverdir is None: diff --git a/Lib/traceback.py b/Lib/traceback.py index f5c4b29..efd0f75 100644 --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -150,50 +150,53 @@ def format_exception_only(etype, value): The arguments are the exception type and value such as given by sys.last_type and sys.last_value. The return value is a list of - strings, each ending in a newline. Normally, the list contains a - single string; however, for SyntaxError exceptions, it contains - several lines that (when printed) display detailed information - about where the syntax error occurred. The message indicating - which exception occurred is the always last string in the list. + strings, each ending in a newline. + + Normally, the list contains a single string; however, for + SyntaxError exceptions, it contains several lines that (when + printed) display detailed information about where the syntax + error occurred. + + The message indicating which exception occurred is always the last + string in the list. + """ - list = [] - if (type(etype) == types.ClassType - or (isinstance(etype, type) and issubclass(etype, BaseException))): - stype = etype.__name__ + + stype = etype.__name__ + + if not issubclass(etype, SyntaxError): + return [_format_final_exc_line(stype, value)] + + # It was a syntax error; show exactly where the problem was found. + lines = [] + try: + msg, (filename, lineno, offset, badline) = value + except Exception: + pass else: - stype = etype - if value is None: - list.append(str(stype) + '\n') + filename = filename or "" + lines.append(' File "%s", line %d\n' % (filename, lineno)) + if badline is not None: + lines.append(' %s\n' % badline.strip()) + if offset is not None: + caretspace = badline[:offset].lstrip() + # non-space whitespace (likes tabs) must be kept for alignment + caretspace = ((c.isspace() and c or ' ') for c in caretspace) + # only three spaces to account for offset1 == pos 0 + lines.append(' %s^\n' % ''.join(caretspace)) + value = msg + + lines.append(_format_final_exc_line(stype, value)) + return lines + +def _format_final_exc_line(etype, value): + """Return a list of a single line -- normal case for format_exception_only""" + valuestr = _some_str(value) + if value is None or not valuestr: + line = "%s\n" % etype else: - if issubclass(etype, SyntaxError): - try: - msg, (filename, lineno, offset, line) = value - except: - pass - else: - if not filename: filename = "" - list.append(' File "%s", line %d\n' % - (filename, lineno)) - if line is not None: - i = 0 - while i < len(line) and line[i].isspace(): - i = i+1 - list.append(' %s\n' % line.strip()) - if offset is not None: - s = ' ' - for c in line[i:offset-1]: - if c.isspace(): - s = s + c - else: - s = s + ' ' - list.append('%s^\n' % s) - value = msg - s = _some_str(value) - if s: - list.append('%s: %s\n' % (str(stype), s)) - else: - list.append('%s\n' % str(stype)) - return list + line = "%s: %s\n" % (etype, valuestr) + return line def _some_str(value): try: diff --git a/Lib/types.py b/Lib/types.py index db63c96..5a89ad1 100644 --- a/Lib/types.py +++ b/Lib/types.py @@ -84,4 +84,16 @@ EllipsisType = type(Ellipsis) DictProxyType = type(TypeType.__dict__) NotImplementedType = type(NotImplemented) -del sys, _f, _g, _C # Not for export +# Extension types defined in a C helper module. XXX There may be no +# equivalent in implementations other than CPython, so it seems better to +# leave them undefined then to set them to e.g. None. +try: + import _types +except ImportError: + pass +else: + GetSetDescriptorType = type(_types.Helper.getter) + MemberDescriptorType = type(_types.Helper.member) + del _types + +del sys, _f, _g, _C, # Not for export diff --git a/Lib/urllib.py b/Lib/urllib.py index f0ae53a..d2a4c48 100644 --- a/Lib/urllib.py +++ b/Lib/urllib.py @@ -118,7 +118,7 @@ class URLopener: self.proxies = proxies self.key_file = x509.get('key_file') self.cert_file = x509.get('cert_file') - self.addheaders = [('User-agent', self.version)] + self.addheaders = [('User-Agent', self.version)] self.__tempfiles = [] self.__unlink = os.unlink # See cleanup() self.tempcache = None @@ -314,8 +314,8 @@ class URLopener: h = httplib.HTTP(host) if data is not None: h.putrequest('POST', selector) - h.putheader('Content-type', 'application/x-www-form-urlencoded') - h.putheader('Content-length', '%d' % len(data)) + h.putheader('Content-Type', 'application/x-www-form-urlencoded') + h.putheader('Content-Length', '%d' % len(data)) else: h.putrequest('GET', selector) if proxy_auth: h.putheader('Proxy-Authorization', 'Basic %s' % proxy_auth) @@ -400,9 +400,9 @@ class URLopener: cert_file=self.cert_file) if data is not None: h.putrequest('POST', selector) - h.putheader('Content-type', + h.putheader('Content-Type', 'application/x-www-form-urlencoded') - h.putheader('Content-length', '%d' % len(data)) + h.putheader('Content-Length', '%d' % len(data)) else: h.putrequest('GET', selector) if proxy_auth: h.putheader('Proxy-Authorization: Basic %s' % proxy_auth) @@ -584,7 +584,7 @@ class URLopener: data = base64.decodestring(data) else: data = unquote(data) - msg.append('Content-length: %d' % len(data)) + msg.append('Content-Length: %d' % len(data)) msg.append('') msg.append(data) msg = '\n'.join(msg) diff --git a/Lib/urllib2.py b/Lib/urllib2.py index 227311c..6ee9e2c 100644 --- a/Lib/urllib2.py +++ b/Lib/urllib2.py @@ -263,11 +263,11 @@ class Request: def add_header(self, key, val): # useful for something like authentication - self.headers[key.capitalize()] = val + self.headers[key.title()] = val def add_unredirected_header(self, key, val): # will not be added to a redirected request - self.unredirected_hdrs[key.capitalize()] = val + self.unredirected_hdrs[key.title()] = val def has_header(self, header_name): return (header_name in self.headers or @@ -286,7 +286,7 @@ class Request: class OpenerDirector: def __init__(self): client_version = "Python-urllib/%s" % __version__ - self.addheaders = [('User-agent', client_version)] + self.addheaders = [('User-Agent', client_version)] # manage the individual handlers self.handlers = [] self.handle_open = {} @@ -675,7 +675,7 @@ class ProxyHandler(BaseHandler): if user and password: user_pass = '%s:%s' % (unquote(user), unquote(password)) creds = base64.encodestring(user_pass).strip() - req.add_header('Proxy-authorization', 'Basic ' + creds) + req.add_header('Proxy-Authorization', 'Basic ' + creds) hostport = unquote(hostport) req.set_proxy(hostport, proxy_type) if orig_type == proxy_type: @@ -819,7 +819,7 @@ class HTTPBasicAuthHandler(AbstractBasicAuthHandler, BaseHandler): class ProxyBasicAuthHandler(AbstractBasicAuthHandler, BaseHandler): - auth_header = 'Proxy-authorization' + auth_header = 'Proxy-Authorization' def http_error_407(self, req, fp, code, msg, headers): # http_error_auth_reqed requires that there is no userinfo component in @@ -1022,20 +1022,20 @@ class AbstractHTTPHandler(BaseHandler): if request.has_data(): # POST data = request.get_data() - if not request.has_header('Content-type'): + if not request.has_header('Content-Type'): request.add_unredirected_header( - 'Content-type', + 'Content-Type', 'application/x-www-form-urlencoded') - if not request.has_header('Content-length'): + if not request.has_header('Content-Length'): request.add_unredirected_header( - 'Content-length', '%d' % len(data)) + 'Content-Length', '%d' % len(data)) scheme, sel = splittype(request.get_selector()) sel_host, sel_path = splithost(sel) if not request.has_header('Host'): request.add_unredirected_header('Host', sel_host or host) for name, value in self.parent.addheaders: - name = name.capitalize() + name = name.title() if not request.has_header(name): request.add_unredirected_header(name, value) @@ -1217,7 +1217,7 @@ class FileHandler(BaseHandler): modified = email.Utils.formatdate(stats.st_mtime, usegmt=True) mtype = mimetypes.guess_type(file)[0] headers = mimetools.Message(StringIO( - 'Content-type: %s\nContent-length: %d\nLast-modified: %s\n' % + 'Content-Type: %s\nContent-Length: %d\nLast-Modified: %s\n' % (mtype or 'text/plain', size, modified))) if host: host, port = splitport(host) @@ -1272,9 +1272,9 @@ class FTPHandler(BaseHandler): headers = "" mtype = mimetypes.guess_type(req.get_full_url())[0] if mtype: - headers += "Content-type: %s\n" % mtype + headers += "Content-Type: %s\n" % mtype if retrlen is not None and retrlen >= 0: - headers += "Content-length: %d\n" % retrlen + headers += "Content-Length: %d\n" % retrlen sf = StringIO(headers) headers = mimetools.Message(sf) return addinfourl(fp, headers, req.get_full_url()) diff --git a/Lib/uuid.py b/Lib/uuid.py new file mode 100644 index 0000000..a6446a1 --- /dev/null +++ b/Lib/uuid.py @@ -0,0 +1,515 @@ +r"""UUID objects (universally unique identifiers) according to RFC 4122. + +This module provides immutable UUID objects (class UUID) and the functions +uuid1(), uuid3(), uuid4(), uuid5() for generating version 1, 3, 4, and 5 +UUIDs as specified in RFC 4122. + +If all you want is a unique ID, you should probably call uuid1() or uuid4(). +Note that uuid1() may compromise privacy since it creates a UUID containing +the computer's network address. uuid4() creates a random UUID. + +Typical usage: + + >>> import uuid + + # make a UUID based on the host ID and current time + >>> uuid.uuid1() + UUID('a8098c1a-f86e-11da-bd1a-00112444be1e') + + # make a UUID using an MD5 hash of a namespace UUID and a name + >>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org') + UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e') + + # make a random UUID + >>> uuid.uuid4() + UUID('16fd2706-8baf-433b-82eb-8c7fada847da') + + # make a UUID using a SHA-1 hash of a namespace UUID and a name + >>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org') + UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d') + + # make a UUID from a string of hex digits (braces and hyphens ignored) + >>> x = uuid.UUID('{00010203-0405-0607-0809-0a0b0c0d0e0f}') + + # convert a UUID to a string of hex digits in standard form + >>> str(x) + '00010203-0405-0607-0809-0a0b0c0d0e0f' + + # get the raw 16 bytes of the UUID + >>> x.bytes + '\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e\x0f' + + # make a UUID from a 16-byte string + >>> uuid.UUID(bytes=x.bytes) + UUID('00010203-0405-0607-0809-0a0b0c0d0e0f') +""" + +__author__ = 'Ka-Ping Yee ' +__date__ = '$Date: 2006/06/12 23:15:40 $'.split()[1].replace('/', '-') +__version__ = '$Revision: 1.30 $'.split()[1] + +RESERVED_NCS, RFC_4122, RESERVED_MICROSOFT, RESERVED_FUTURE = [ + 'reserved for NCS compatibility', 'specified in RFC 4122', + 'reserved for Microsoft compatibility', 'reserved for future definition'] + +class UUID(object): + """Instances of the UUID class represent UUIDs as specified in RFC 4122. + UUID objects are immutable, hashable, and usable as dictionary keys. + Converting a UUID to a string with str() yields something in the form + '12345678-1234-1234-1234-123456789abc'. The UUID constructor accepts + four possible forms: a similar string of hexadecimal digits, or a + string of 16 raw bytes as an argument named 'bytes', or a tuple of + six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and + 48-bit values respectively) as an argument named 'fields', or a single + 128-bit integer as an argument named 'int'. + + UUIDs have these read-only attributes: + + bytes the UUID as a 16-byte string + + fields a tuple of the six integer fields of the UUID, + which are also available as six individual attributes + and two derived attributes: + + time_low the first 32 bits of the UUID + time_mid the next 16 bits of the UUID + time_hi_version the next 16 bits of the UUID + clock_seq_hi_variant the next 8 bits of the UUID + clock_seq_low the next 8 bits of the UUID + node the last 48 bits of the UUID + + time the 60-bit timestamp + clock_seq the 14-bit sequence number + + hex the UUID as a 32-character hexadecimal string + + int the UUID as a 128-bit integer + + urn the UUID as a URN as specified in RFC 4122 + + variant the UUID variant (one of the constants RESERVED_NCS, + RFC_4122, RESERVED_MICROSOFT, or RESERVED_FUTURE) + + version the UUID version number (1 through 5, meaningful only + when the variant is RFC_4122) + """ + + def __init__(self, hex=None, bytes=None, fields=None, int=None, + version=None): + r"""Create a UUID from either a string of 32 hexadecimal digits, + a string of 16 bytes as the 'bytes' argument, a tuple of six + integers (32-bit time_low, 16-bit time_mid, 16-bit time_hi_version, + 8-bit clock_seq_hi_variant, 8-bit clock_seq_low, 48-bit node) as + the 'fields' argument, or a single 128-bit integer as the 'int' + argument. When a string of hex digits is given, curly braces, + hyphens, and a URN prefix are all optional. For example, these + expressions all yield the same UUID: + + UUID('{12345678-1234-5678-1234-567812345678}') + UUID('12345678123456781234567812345678') + UUID('urn:uuid:12345678-1234-5678-1234-567812345678') + UUID(bytes='\x12\x34\x56\x78'*4) + UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) + UUID(int=0x12345678123456781234567812345678) + + Exactly one of 'hex', 'bytes', 'fields', or 'int' must be given. + The 'version' argument is optional; if given, the resulting UUID + will have its variant and version number set according to RFC 4122, + overriding bits in the given 'hex', 'bytes', 'fields', or 'int'. + """ + + if [hex, bytes, fields, int].count(None) != 3: + raise TypeError('need just one of hex, bytes, fields, or int') + if hex is not None: + hex = hex.replace('urn:', '').replace('uuid:', '') + hex = hex.strip('{}').replace('-', '') + if len(hex) != 32: + raise ValueError('badly formed hexadecimal UUID string') + int = long(hex, 16) + if bytes is not None: + if len(bytes) != 16: + raise ValueError('bytes is not a 16-char string') + int = long(('%02x'*16) % tuple(map(ord, bytes)), 16) + if fields is not None: + if len(fields) != 6: + raise ValueError('fields is not a 6-tuple') + (time_low, time_mid, time_hi_version, + clock_seq_hi_variant, clock_seq_low, node) = fields + if not 0 <= time_low < 1<<32L: + raise ValueError('field 1 out of range (need a 32-bit value)') + if not 0 <= time_mid < 1<<16L: + raise ValueError('field 2 out of range (need a 16-bit value)') + if not 0 <= time_hi_version < 1<<16L: + raise ValueError('field 3 out of range (need a 16-bit value)') + if not 0 <= clock_seq_hi_variant < 1<<8L: + raise ValueError('field 4 out of range (need an 8-bit value)') + if not 0 <= clock_seq_low < 1<<8L: + raise ValueError('field 5 out of range (need an 8-bit value)') + if not 0 <= node < 1<<48L: + raise ValueError('field 6 out of range (need a 48-bit value)') + clock_seq = (clock_seq_hi_variant << 8L) | clock_seq_low + int = ((time_low << 96L) | (time_mid << 80L) | + (time_hi_version << 64L) | (clock_seq << 48L) | node) + if int is not None: + if not 0 <= int < 1<<128L: + raise ValueError('int is out of range (need a 128-bit value)') + if version is not None: + if not 1 <= version <= 5: + raise ValueError('illegal version number') + # Set the variant to RFC 4122. + int &= ~(0xc000 << 48L) + int |= 0x8000 << 48L + # Set the version number. + int &= ~(0xf000 << 64L) + int |= version << 76L + self.__dict__['int'] = int + + def __cmp__(self, other): + if isinstance(other, UUID): + return cmp(self.int, other.int) + return NotImplemented + + def __hash__(self): + return hash(self.int) + + def __int__(self): + return self.int + + def __repr__(self): + return 'UUID(%r)' % str(self) + + def __setattr__(self, name, value): + raise TypeError('UUID objects are immutable') + + def __str__(self): + hex = '%032x' % self.int + return '%s-%s-%s-%s-%s' % ( + hex[:8], hex[8:12], hex[12:16], hex[16:20], hex[20:]) + + def get_bytes(self): + bytes = '' + for shift in range(0, 128, 8): + bytes = chr((self.int >> shift) & 0xff) + bytes + return bytes + + bytes = property(get_bytes) + + def get_fields(self): + return (self.time_low, self.time_mid, self.time_hi_version, + self.clock_seq_hi_variant, self.clock_seq_low, self.node) + + fields = property(get_fields) + + def get_time_low(self): + return self.int >> 96L + + time_low = property(get_time_low) + + def get_time_mid(self): + return (self.int >> 80L) & 0xffff + + time_mid = property(get_time_mid) + + def get_time_hi_version(self): + return (self.int >> 64L) & 0xffff + + time_hi_version = property(get_time_hi_version) + + def get_clock_seq_hi_variant(self): + return (self.int >> 56L) & 0xff + + clock_seq_hi_variant = property(get_clock_seq_hi_variant) + + def get_clock_seq_low(self): + return (self.int >> 48L) & 0xff + + clock_seq_low = property(get_clock_seq_low) + + def get_time(self): + return (((self.time_hi_version & 0x0fffL) << 48L) | + (self.time_mid << 32L) | self.time_low) + + time = property(get_time) + + def get_clock_seq(self): + return (((self.clock_seq_hi_variant & 0x3fL) << 8L) | + self.clock_seq_low) + + clock_seq = property(get_clock_seq) + + def get_node(self): + return self.int & 0xffffffffffff + + node = property(get_node) + + def get_hex(self): + return '%032x' % self.int + + hex = property(get_hex) + + def get_urn(self): + return 'urn:uuid:' + str(self) + + urn = property(get_urn) + + def get_variant(self): + if not self.int & (0x8000 << 48L): + return RESERVED_NCS + elif not self.int & (0x4000 << 48L): + return RFC_4122 + elif not self.int & (0x2000 << 48L): + return RESERVED_MICROSOFT + else: + return RESERVED_FUTURE + + variant = property(get_variant) + + def get_version(self): + # The version bits are only meaningful for RFC 4122 UUIDs. + if self.variant == RFC_4122: + return int((self.int >> 76L) & 0xf) + + version = property(get_version) + +def _find_mac(command, args, hw_identifiers, get_index): + import os + for dir in ['', '/sbin/', '/usr/sbin']: + executable = os.path.join(dir, command) + if not os.path.exists(executable): + continue + + try: + # LC_ALL to get English output, 2>/dev/null to + # prevent output on stderr + cmd = 'LC_ALL=C %s %s 2>/dev/null' % (executable, args) + pipe = os.popen(cmd) + except IOError: + continue + + for line in pipe: + words = line.lower().split() + for i in range(len(words)): + if words[i] in hw_identifiers: + return int(words[get_index(i)].replace(':', ''), 16) + return None + +def _ifconfig_getnode(): + """Get the hardware address on Unix by running ifconfig.""" + + # This works on Linux ('' or '-a'), Tru64 ('-av'), but not all Unixes. + for args in ('', '-a', '-av'): + mac = _find_mac('ifconfig', args, ['hwaddr', 'ether'], lambda i: i+1) + if mac: + return mac + + import socket + ip_addr = socket.gethostbyname(socket.gethostname()) + + # Try getting the MAC addr from arp based on our IP address (Solaris). + mac = _find_mac('arp', '-an', [ip_addr], lambda i: -1) + if mac: + return mac + + # This might work on HP-UX. + mac = _find_mac('lanscan', '-ai', ['lan0'], lambda i: 0) + if mac: + return mac + + return None + +def _ipconfig_getnode(): + """Get the hardware address on Windows by running ipconfig.exe.""" + import os, re + dirs = ['', r'c:\windows\system32', r'c:\winnt\system32'] + try: + import ctypes + buffer = ctypes.create_string_buffer(300) + ctypes.windll.kernel32.GetSystemDirectoryA(buffer, 300) + dirs.insert(0, buffer.value.decode('mbcs')) + except: + pass + for dir in dirs: + try: + pipe = os.popen(os.path.join(dir, 'ipconfig') + ' /all') + except IOError: + continue + for line in pipe: + value = line.split(':')[-1].strip().lower() + if re.match('([0-9a-f][0-9a-f]-){5}[0-9a-f][0-9a-f]', value): + return int(value.replace('-', ''), 16) + +def _netbios_getnode(): + """Get the hardware address on Windows using NetBIOS calls. + See http://support.microsoft.com/kb/118623 for details.""" + import win32wnet, netbios + ncb = netbios.NCB() + ncb.Command = netbios.NCBENUM + ncb.Buffer = adapters = netbios.LANA_ENUM() + adapters._pack() + if win32wnet.Netbios(ncb) != 0: + return + adapters._unpack() + for i in range(adapters.length): + ncb.Reset() + ncb.Command = netbios.NCBRESET + ncb.Lana_num = ord(adapters.lana[i]) + if win32wnet.Netbios(ncb) != 0: + continue + ncb.Reset() + ncb.Command = netbios.NCBASTAT + ncb.Lana_num = ord(adapters.lana[i]) + ncb.Callname = '*'.ljust(16) + ncb.Buffer = status = netbios.ADAPTER_STATUS() + if win32wnet.Netbios(ncb) != 0: + continue + status._unpack() + bytes = map(ord, status.adapter_address) + return ((bytes[0]<<40L) + (bytes[1]<<32L) + (bytes[2]<<24L) + + (bytes[3]<<16L) + (bytes[4]<<8L) + bytes[5]) + +# Thanks to Thomas Heller for ctypes and for his help with its use here. + +# If ctypes is available, use it to find system routines for UUID generation. +_uuid_generate_random = _uuid_generate_time = _UuidCreate = None +try: + import ctypes, ctypes.util + _buffer = ctypes.create_string_buffer(16) + + # The uuid_generate_* routines are provided by libuuid on at least + # Linux and FreeBSD, and provided by libc on Mac OS X. + for libname in ['uuid', 'c']: + try: + lib = ctypes.CDLL(ctypes.util.find_library(libname)) + except: + continue + if hasattr(lib, 'uuid_generate_random'): + _uuid_generate_random = lib.uuid_generate_random + if hasattr(lib, 'uuid_generate_time'): + _uuid_generate_time = lib.uuid_generate_time + + # On Windows prior to 2000, UuidCreate gives a UUID containing the + # hardware address. On Windows 2000 and later, UuidCreate makes a + # random UUID and UuidCreateSequential gives a UUID containing the + # hardware address. These routines are provided by the RPC runtime. + # NOTE: at least on Tim's WinXP Pro SP2 desktop box, while the last + # 6 bytes returned by UuidCreateSequential are fixed, they don't appear + # to bear any relationship to the MAC address of any network device + # on the box. + try: + lib = ctypes.windll.rpcrt4 + except: + lib = None + _UuidCreate = getattr(lib, 'UuidCreateSequential', + getattr(lib, 'UuidCreate', None)) +except: + pass + +def _unixdll_getnode(): + """Get the hardware address on Unix using ctypes.""" + _uuid_generate_time(_buffer) + return UUID(bytes=_buffer.raw).node + +def _windll_getnode(): + """Get the hardware address on Windows using ctypes.""" + if _UuidCreate(_buffer) == 0: + return UUID(bytes=_buffer.raw).node + +def _random_getnode(): + """Get a random node ID, with eighth bit set as suggested by RFC 4122.""" + import random + return random.randrange(0, 1<<48L) | 0x010000000000L + +_node = None + +def getnode(): + """Get the hardware address as a 48-bit positive integer. + + The first time this runs, it may launch a separate program, which could + be quite slow. If all attempts to obtain the hardware address fail, we + choose a random 48-bit number with its eighth bit set to 1 as recommended + in RFC 4122. + """ + + global _node + if _node is not None: + return _node + + import sys + if sys.platform == 'win32': + getters = [_windll_getnode, _netbios_getnode, _ipconfig_getnode] + else: + getters = [_unixdll_getnode, _ifconfig_getnode] + + for getter in getters + [_random_getnode]: + try: + _node = getter() + except: + continue + if _node is not None: + return _node + +def uuid1(node=None, clock_seq=None): + """Generate a UUID from a host ID, sequence number, and the current time. + If 'node' is not given, getnode() is used to obtain the hardware + address. If 'clock_seq' is given, it is used as the sequence number; + otherwise a random 14-bit sequence number is chosen.""" + + # When the system provides a version-1 UUID generator, use it (but don't + # use UuidCreate here because its UUIDs don't conform to RFC 4122). + if _uuid_generate_time and node is clock_seq is None: + _uuid_generate_time(_buffer) + return UUID(bytes=_buffer.raw) + + import time + nanoseconds = int(time.time() * 1e9) + # 0x01b21dd213814000 is the number of 100-ns intervals between the + # UUID epoch 1582-10-15 00:00:00 and the Unix epoch 1970-01-01 00:00:00. + timestamp = int(nanoseconds/100) + 0x01b21dd213814000L + if clock_seq is None: + import random + clock_seq = random.randrange(1<<14L) # instead of stable storage + time_low = timestamp & 0xffffffffL + time_mid = (timestamp >> 32L) & 0xffffL + time_hi_version = (timestamp >> 48L) & 0x0fffL + clock_seq_low = clock_seq & 0xffL + clock_seq_hi_variant = (clock_seq >> 8L) & 0x3fL + if node is None: + node = getnode() + return UUID(fields=(time_low, time_mid, time_hi_version, + clock_seq_hi_variant, clock_seq_low, node), version=1) + +def uuid3(namespace, name): + """Generate a UUID from the MD5 hash of a namespace UUID and a name.""" + import md5 + hash = md5.md5(namespace.bytes + name).digest() + return UUID(bytes=hash[:16], version=3) + +def uuid4(): + """Generate a random UUID.""" + + # When the system provides a version-4 UUID generator, use it. + if _uuid_generate_random: + _uuid_generate_random(_buffer) + return UUID(bytes=_buffer.raw) + + # Otherwise, get randomness from urandom or the 'random' module. + try: + import os + return UUID(bytes=os.urandom(16), version=4) + except: + import random + bytes = [chr(random.randrange(256)) for i in range(16)] + return UUID(bytes=bytes, version=4) + +def uuid5(namespace, name): + """Generate a UUID from the SHA-1 hash of a namespace UUID and a name.""" + import sha + hash = sha.sha(namespace.bytes + name).digest() + return UUID(bytes=hash[:16], version=5) + +# The following standard UUIDs are for use with uuid3() or uuid5(). + +NAMESPACE_DNS = UUID('6ba7b810-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_URL = UUID('6ba7b811-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_OID = UUID('6ba7b812-9dad-11d1-80b4-00c04fd430c8') +NAMESPACE_X500 = UUID('6ba7b814-9dad-11d1-80b4-00c04fd430c8') diff --git a/Lib/warnings.py b/Lib/warnings.py index b5d75e4..b7fac69 100644 --- a/Lib/warnings.py +++ b/Lib/warnings.py @@ -46,7 +46,7 @@ def warn(message, category=None, stacklevel=1): filename = globals.get('__file__') if filename: fnl = filename.lower() - if fnl.endswith(".pyc") or fnl.endswith(".pyo"): + if fnl.endswith((".pyc", ".pyo")): filename = filename[:-1] else: if module == "__main__": @@ -254,11 +254,11 @@ def _getcategory(category): cat = getattr(m, klass) except AttributeError: raise _OptionError("unknown warning category: %r" % (category,)) - if (not isinstance(cat, types.ClassType) or - not issubclass(cat, Warning)): + if not issubclass(cat, Warning): raise _OptionError("invalid warning category: %r" % (category,)) return cat # Module initialization _processoptions(sys.warnoptions) simplefilter("ignore", category=PendingDeprecationWarning, append=1) +simplefilter("ignore", category=ImportWarning, append=1) diff --git a/Lib/webbrowser.py b/Lib/webbrowser.py index 4693fe7..7a1a3b4 100644 --- a/Lib/webbrowser.py +++ b/Lib/webbrowser.py @@ -98,8 +98,7 @@ def _synthesize(browser, update_tryorder=1): if sys.platform[:3] == "win": def _isexecutable(cmd): cmd = cmd.lower() - if os.path.isfile(cmd) and (cmd.endswith(".exe") or - cmd.endswith(".bat")): + if os.path.isfile(cmd) and cmd.endswith((".exe", ".bat")): return True for ext in ".exe", ".bat": if os.path.isfile(cmd + ext): @@ -435,13 +434,13 @@ def register_X_browsers(): # The default Gnome browser if _iscommand("gconftool-2"): # get the web browser string from gconftool - gc = 'gconftool-2 -g /desktop/gnome/url-handlers/http/command' + gc = 'gconftool-2 -g /desktop/gnome/url-handlers/http/command 2>/dev/null' out = os.popen(gc) commd = out.read().strip() retncode = out.close() # if successful, register it - if retncode == None and len(commd) != 0: + if retncode is None and commd: register("gnome", None, BackgroundBrowser(commd)) # First, the Mozilla/Netscape browsers diff --git a/Lib/wsgiref.egg-info b/Lib/wsgiref.egg-info new file mode 100644 index 0000000..c0b7893 --- /dev/null +++ b/Lib/wsgiref.egg-info @@ -0,0 +1,8 @@ +Metadata-Version: 1.0 +Name: wsgiref +Version: 0.1.2 +Summary: WSGI (PEP 333) Reference Library +Author: Phillip J. Eby +Author-email: web-sig@python.org +License: PSF or ZPL +Platform: UNKNOWN diff --git a/Lib/wsgiref/__init__.py b/Lib/wsgiref/__init__.py new file mode 100644 index 0000000..46c579f --- /dev/null +++ b/Lib/wsgiref/__init__.py @@ -0,0 +1,23 @@ +"""wsgiref -- a WSGI (PEP 333) Reference Library + +Current Contents: + +* util -- Miscellaneous useful functions and wrappers + +* headers -- Manage response headers + +* handlers -- base classes for server/gateway implementations + +* simple_server -- a simple BaseHTTPServer that supports WSGI + +* validate -- validation wrapper that sits between an app and a server + to detect errors in either + +To-Do: + +* cgi_gateway -- Run WSGI apps under CGI (pending a deployment standard) + +* cgi_wrapper -- Run CGI apps under WSGI + +* router -- a simple middleware component that handles URL traversal +""" diff --git a/Lib/wsgiref/handlers.py b/Lib/wsgiref/handlers.py new file mode 100644 index 0000000..099371b --- /dev/null +++ b/Lib/wsgiref/handlers.py @@ -0,0 +1,492 @@ +"""Base classes for server/gateway implementations""" + +from types import StringType +from util import FileWrapper, guess_scheme, is_hop_by_hop +from headers import Headers + +import sys, os, time + +__all__ = ['BaseHandler', 'SimpleHandler', 'BaseCGIHandler', 'CGIHandler'] + +try: + dict +except NameError: + def dict(items): + d = {} + for k,v in items: + d[k] = v + return d + +try: + True + False +except NameError: + True = not None + False = not True + + +# Weekday and month names for HTTP date/time formatting; always English! +_weekdayname = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"] +_monthname = [None, # Dummy so we can use 1-based month numbers + "Jan", "Feb", "Mar", "Apr", "May", "Jun", + "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"] + +def format_date_time(timestamp): + year, month, day, hh, mm, ss, wd, y, z = time.gmtime(timestamp) + return "%s, %02d %3s %4d %02d:%02d:%02d GMT" % ( + _weekdayname[wd], day, _monthname[month], year, hh, mm, ss + ) + + + +class BaseHandler: + """Manage the invocation of a WSGI application""" + + # Configuration parameters; can override per-subclass or per-instance + wsgi_version = (1,0) + wsgi_multithread = True + wsgi_multiprocess = True + wsgi_run_once = False + + origin_server = True # We are transmitting direct to client + http_version = "1.0" # Version that should be used for response + server_software = None # String name of server software, if any + + # os_environ is used to supply configuration from the OS environment: + # by default it's a copy of 'os.environ' as of import time, but you can + # override this in e.g. your __init__ method. + os_environ = dict(os.environ.items()) + + # Collaborator classes + wsgi_file_wrapper = FileWrapper # set to None to disable + headers_class = Headers # must be a Headers-like class + + # Error handling (also per-subclass or per-instance) + traceback_limit = None # Print entire traceback to self.get_stderr() + error_status = "500 Dude, this is whack!" + error_headers = [('Content-Type','text/plain')] + error_body = "A server error occurred. Please contact the administrator." + + # State variables (don't mess with these) + status = result = None + headers_sent = False + headers = None + bytes_sent = 0 + + + + + + + + + def run(self, application): + """Invoke the application""" + # Note to self: don't move the close()! Asynchronous servers shouldn't + # call close() from finish_response(), so if you close() anywhere but + # the double-error branch here, you'll break asynchronous servers by + # prematurely closing. Async servers must return from 'run()' without + # closing if there might still be output to iterate over. + try: + self.setup_environ() + self.result = application(self.environ, self.start_response) + self.finish_response() + except: + try: + self.handle_error() + except: + # If we get an error handling an error, just give up already! + self.close() + raise # ...and let the actual server figure it out. + + + def setup_environ(self): + """Set up the environment for one request""" + + env = self.environ = self.os_environ.copy() + self.add_cgi_vars() + + env['wsgi.input'] = self.get_stdin() + env['wsgi.errors'] = self.get_stderr() + env['wsgi.version'] = self.wsgi_version + env['wsgi.run_once'] = self.wsgi_run_once + env['wsgi.url_scheme'] = self.get_scheme() + env['wsgi.multithread'] = self.wsgi_multithread + env['wsgi.multiprocess'] = self.wsgi_multiprocess + + if self.wsgi_file_wrapper is not None: + env['wsgi.file_wrapper'] = self.wsgi_file_wrapper + + if self.origin_server and self.server_software: + env.setdefault('SERVER_SOFTWARE',self.server_software) + + + def finish_response(self): + """Send any iterable data, then close self and the iterable + + Subclasses intended for use in asynchronous servers will + want to redefine this method, such that it sets up callbacks + in the event loop to iterate over the data, and to call + 'self.close()' once the response is finished. + """ + if not self.result_is_file() or not self.sendfile(): + for data in self.result: + self.write(data) + self.finish_content() + self.close() + + + def get_scheme(self): + """Return the URL scheme being used""" + return guess_scheme(self.environ) + + + def set_content_length(self): + """Compute Content-Length or switch to chunked encoding if possible""" + try: + blocks = len(self.result) + except (TypeError,AttributeError,NotImplementedError): + pass + else: + if blocks==1: + self.headers['Content-Length'] = str(self.bytes_sent) + return + # XXX Try for chunked encoding if origin server and client is 1.1 + + + def cleanup_headers(self): + """Make any necessary header changes or defaults + + Subclasses can extend this to add other defaults. + """ + if not self.headers.has_key('Content-Length'): + self.set_content_length() + + def start_response(self, status, headers,exc_info=None): + """'start_response()' callable as specified by PEP 333""" + + if exc_info: + try: + if self.headers_sent: + # Re-raise original exception if headers sent + raise exc_info[0], exc_info[1], exc_info[2] + finally: + exc_info = None # avoid dangling circular ref + elif self.headers is not None: + raise AssertionError("Headers already set!") + + assert type(status) is StringType,"Status must be a string" + assert len(status)>=4,"Status must be at least 4 characters" + assert int(status[:3]),"Status message must begin w/3-digit code" + assert status[3]==" ", "Status message must have a space after code" + if __debug__: + for name,val in headers: + assert type(name) is StringType,"Header names must be strings" + assert type(val) is StringType,"Header values must be strings" + assert not is_hop_by_hop(name),"Hop-by-hop headers not allowed" + self.status = status + self.headers = self.headers_class(headers) + return self.write + + + def send_preamble(self): + """Transmit version/status/date/server, via self._write()""" + if self.origin_server: + if self.client_is_modern(): + self._write('HTTP/%s %s\r\n' % (self.http_version,self.status)) + if not self.headers.has_key('Date'): + self._write( + 'Date: %s\r\n' % format_date_time(time.time()) + ) + if self.server_software and not self.headers.has_key('Server'): + self._write('Server: %s\r\n' % self.server_software) + else: + self._write('Status: %s\r\n' % self.status) + + def write(self, data): + """'write()' callable as specified by PEP 333""" + + assert type(data) is StringType,"write() argument must be string" + + if not self.status: + raise AssertionError("write() before start_response()") + + elif not self.headers_sent: + # Before the first output, send the stored headers + self.bytes_sent = len(data) # make sure we know content-length + self.send_headers() + else: + self.bytes_sent += len(data) + + # XXX check Content-Length and truncate if too many bytes written? + self._write(data) + self._flush() + + + def sendfile(self): + """Platform-specific file transmission + + Override this method in subclasses to support platform-specific + file transmission. It is only called if the application's + return iterable ('self.result') is an instance of + 'self.wsgi_file_wrapper'. + + This method should return a true value if it was able to actually + transmit the wrapped file-like object using a platform-specific + approach. It should return a false value if normal iteration + should be used instead. An exception can be raised to indicate + that transmission was attempted, but failed. + + NOTE: this method should call 'self.send_headers()' if + 'self.headers_sent' is false and it is going to attempt direct + transmission of the file. + """ + return False # No platform-specific transmission by default + + + def finish_content(self): + """Ensure headers and content have both been sent""" + if not self.headers_sent: + self.headers['Content-Length'] = "0" + self.send_headers() + else: + pass # XXX check if content-length was too short? + + def close(self): + """Close the iterable (if needed) and reset all instance vars + + Subclasses may want to also drop the client connection. + """ + try: + if hasattr(self.result,'close'): + self.result.close() + finally: + self.result = self.headers = self.status = self.environ = None + self.bytes_sent = 0; self.headers_sent = False + + + def send_headers(self): + """Transmit headers to the client, via self._write()""" + self.cleanup_headers() + self.headers_sent = True + if not self.origin_server or self.client_is_modern(): + self.send_preamble() + self._write(str(self.headers)) + + + def result_is_file(self): + """True if 'self.result' is an instance of 'self.wsgi_file_wrapper'""" + wrapper = self.wsgi_file_wrapper + return wrapper is not None and isinstance(self.result,wrapper) + + + def client_is_modern(self): + """True if client can accept status and headers""" + return self.environ['SERVER_PROTOCOL'].upper() != 'HTTP/0.9' + + + def log_exception(self,exc_info): + """Log the 'exc_info' tuple in the server log + + Subclasses may override to retarget the output or change its format. + """ + try: + from traceback import print_exception + stderr = self.get_stderr() + print_exception( + exc_info[0], exc_info[1], exc_info[2], + self.traceback_limit, stderr + ) + stderr.flush() + finally: + exc_info = None + + def handle_error(self): + """Log current error, and send error output to client if possible""" + self.log_exception(sys.exc_info()) + if not self.headers_sent: + self.result = self.error_output(self.environ, self.start_response) + self.finish_response() + # XXX else: attempt advanced recovery techniques for HTML or text? + + def error_output(self, environ, start_response): + """WSGI mini-app to create error output + + By default, this just uses the 'error_status', 'error_headers', + and 'error_body' attributes to generate an output page. It can + be overridden in a subclass to dynamically generate diagnostics, + choose an appropriate message for the user's preferred language, etc. + + Note, however, that it's not recommended from a security perspective to + spit out diagnostics to any old user; ideally, you should have to do + something special to enable diagnostic output, which is why we don't + include any here! + """ + start_response(self.error_status,self.error_headers[:],sys.exc_info()) + return [self.error_body] + + + # Pure abstract methods; *must* be overridden in subclasses + + def _write(self,data): + """Override in subclass to buffer data for send to client + + It's okay if this method actually transmits the data; BaseHandler + just separates write and flush operations for greater efficiency + when the underlying system actually has such a distinction. + """ + raise NotImplementedError + + def _flush(self): + """Override in subclass to force sending of recent '_write()' calls + + It's okay if this method is a no-op (i.e., if '_write()' actually + sends the data. + """ + raise NotImplementedError + + def get_stdin(self): + """Override in subclass to return suitable 'wsgi.input'""" + raise NotImplementedError + + def get_stderr(self): + """Override in subclass to return suitable 'wsgi.errors'""" + raise NotImplementedError + + def add_cgi_vars(self): + """Override in subclass to insert CGI variables in 'self.environ'""" + raise NotImplementedError + + + + + + + + + + + +class SimpleHandler(BaseHandler): + """Handler that's just initialized with streams, environment, etc. + + This handler subclass is intended for synchronous HTTP/1.0 origin servers, + and handles sending the entire response output, given the correct inputs. + + Usage:: + + handler = SimpleHandler( + inp,out,err,env, multithread=False, multiprocess=True + ) + handler.run(app)""" + + def __init__(self,stdin,stdout,stderr,environ, + multithread=True, multiprocess=False + ): + self.stdin = stdin + self.stdout = stdout + self.stderr = stderr + self.base_env = environ + self.wsgi_multithread = multithread + self.wsgi_multiprocess = multiprocess + + def get_stdin(self): + return self.stdin + + def get_stderr(self): + return self.stderr + + def add_cgi_vars(self): + self.environ.update(self.base_env) + + def _write(self,data): + self.stdout.write(data) + self._write = self.stdout.write + + def _flush(self): + self.stdout.flush() + self._flush = self.stdout.flush + + +class BaseCGIHandler(SimpleHandler): + + """CGI-like systems using input/output/error streams and environ mapping + + Usage:: + + handler = BaseCGIHandler(inp,out,err,env) + handler.run(app) + + This handler class is useful for gateway protocols like ReadyExec and + FastCGI, that have usable input/output/error streams and an environment + mapping. It's also the base class for CGIHandler, which just uses + sys.stdin, os.environ, and so on. + + The constructor also takes keyword arguments 'multithread' and + 'multiprocess' (defaulting to 'True' and 'False' respectively) to control + the configuration sent to the application. It sets 'origin_server' to + False (to enable CGI-like output), and assumes that 'wsgi.run_once' is + False. + """ + + origin_server = False + + + + + + + + + + + + + + + + + + + +class CGIHandler(BaseCGIHandler): + + """CGI-based invocation via sys.stdin/stdout/stderr and os.environ + + Usage:: + + CGIHandler().run(app) + + The difference between this class and BaseCGIHandler is that it always + uses 'wsgi.run_once' of 'True', 'wsgi.multithread' of 'False', and + 'wsgi.multiprocess' of 'True'. It does not take any initialization + parameters, but always uses 'sys.stdin', 'os.environ', and friends. + + If you need to override any of these parameters, use BaseCGIHandler + instead. + """ + + wsgi_run_once = True + + def __init__(self): + BaseCGIHandler.__init__( + self, sys.stdin, sys.stdout, sys.stderr, dict(os.environ.items()), + multithread=False, multiprocess=True + ) + + + + + + + + + + + + + + + + +# diff --git a/Lib/wsgiref/headers.py b/Lib/wsgiref/headers.py new file mode 100644 index 0000000..016eb86 --- /dev/null +++ b/Lib/wsgiref/headers.py @@ -0,0 +1,205 @@ +"""Manage HTTP Response Headers + +Much of this module is red-handedly pilfered from email.Message in the stdlib, +so portions are Copyright (C) 2001,2002 Python Software Foundation, and were +written by Barry Warsaw. +""" + +from types import ListType, TupleType + +# Regular expression that matches `special' characters in parameters, the +# existance of which force quoting of the parameter value. +import re +tspecials = re.compile(r'[ \(\)<>@,;:\\"/\[\]\?=]') + +def _formatparam(param, value=None, quote=1): + """Convenience function to format and return a key=value pair. + + This will quote the value if needed or if quote is true. + """ + if value is not None and len(value) > 0: + if quote or tspecials.search(value): + value = value.replace('\\', '\\\\').replace('"', r'\"') + return '%s="%s"' % (param, value) + else: + return '%s=%s' % (param, value) + else: + return param + + + + + + + + + + + + + + +class Headers: + + """Manage a collection of HTTP response headers""" + + def __init__(self,headers): + if type(headers) is not ListType: + raise TypeError("Headers must be a list of name/value tuples") + self._headers = headers + + def __len__(self): + """Return the total number of headers, including duplicates.""" + return len(self._headers) + + def __setitem__(self, name, val): + """Set the value of a header.""" + del self[name] + self._headers.append((name, val)) + + def __delitem__(self,name): + """Delete all occurrences of a header, if present. + + Does *not* raise an exception if the header is missing. + """ + name = name.lower() + self._headers[:] = [kv for kv in self._headers if kv[0].lower()<>name] + + def __getitem__(self,name): + """Get the first header value for 'name' + + Return None if the header is missing instead of raising an exception. + + Note that if the header appeared multiple times, the first exactly which + occurrance gets returned is undefined. Use getall() to get all + the values matching a header field name. + """ + return self.get(name) + + + + + + def has_key(self, name): + """Return true if the message contains the header.""" + return self.get(name) is not None + + __contains__ = has_key + + + def get_all(self, name): + """Return a list of all the values for the named field. + + These will be sorted in the order they appeared in the original header + list or were added to this instance, and may contain duplicates. Any + fields deleted and re-inserted are always appended to the header list. + If no fields exist with the given name, returns an empty list. + """ + name = name.lower() + return [kv[1] for kv in self._headers if kv[0].lower()==name] + + + def get(self,name,default=None): + """Get the first header value for 'name', or return 'default'""" + name = name.lower() + for k,v in self._headers: + if k.lower()==name: + return v + return default + + + def keys(self): + """Return a list of all the header field names. + + These will be sorted in the order they appeared in the original header + list, or were added to this instance, and may contain duplicates. + Any fields deleted and re-inserted are always appended to the header + list. + """ + return [k for k, v in self._headers] + + + + + def values(self): + """Return a list of all header values. + + These will be sorted in the order they appeared in the original header + list, or were added to this instance, and may contain duplicates. + Any fields deleted and re-inserted are always appended to the header + list. + """ + return [v for k, v in self._headers] + + def items(self): + """Get all the header fields and values. + + These will be sorted in the order they were in the original header + list, or were added to this instance, and may contain duplicates. + Any fields deleted and re-inserted are always appended to the header + list. + """ + return self._headers[:] + + def __repr__(self): + return "Headers(%s)" % `self._headers` + + def __str__(self): + """str() returns the formatted headers, complete with end line, + suitable for direct HTTP transmission.""" + return '\r\n'.join(["%s: %s" % kv for kv in self._headers]+['','']) + + def setdefault(self,name,value): + """Return first matching header value for 'name', or 'value' + + If there is no header named 'name', add a new header with name 'name' + and value 'value'.""" + result = self.get(name) + if result is None: + self._headers.append((name,value)) + return value + else: + return result + + + def add_header(self, _name, _value, **_params): + """Extended header setting. + + _name is the header field to add. keyword arguments can be used to set + additional parameters for the header field, with underscores converted + to dashes. Normally the parameter will be added as key="value" unless + value is None, in which case only the key will be added. + + Example: + + h.add_header('content-disposition', 'attachment', filename='bud.gif') + + Note that unlike the corresponding 'email.Message' method, this does + *not* handle '(charset, language, value)' tuples: all values must be + strings or None. + """ + parts = [] + if _value is not None: + parts.append(_value) + for k, v in _params.items(): + if v is None: + parts.append(k.replace('_', '-')) + else: + parts.append(_formatparam(k.replace('_', '-'), v)) + self._headers.append((_name, "; ".join(parts))) + + + + + + + + + + + + + + + +# diff --git a/Lib/wsgiref/simple_server.py b/Lib/wsgiref/simple_server.py new file mode 100644 index 0000000..95996cc --- /dev/null +++ b/Lib/wsgiref/simple_server.py @@ -0,0 +1,205 @@ +"""BaseHTTPServer that implements the Python WSGI protocol (PEP 333, rev 1.21) + +This is both an example of how WSGI can be implemented, and a basis for running +simple web applications on a local machine, such as might be done when testing +or debugging an application. It has not been reviewed for security issues, +however, and we strongly recommend that you use a "real" web server for +production use. + +For example usage, see the 'if __name__=="__main__"' block at the end of the +module. See also the BaseHTTPServer module docs for other API information. +""" + +from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer +import urllib, sys +from wsgiref.handlers import SimpleHandler + +__version__ = "0.1" +__all__ = ['WSGIServer', 'WSGIRequestHandler', 'demo_app', 'make_server'] + + +server_version = "WSGIServer/" + __version__ +sys_version = "Python/" + sys.version.split()[0] +software_version = server_version + ' ' + sys_version + + +class ServerHandler(SimpleHandler): + + server_software = software_version + + def close(self): + try: + self.request_handler.log_request( + self.status.split(' ',1)[0], self.bytes_sent + ) + finally: + SimpleHandler.close(self) + + + + + +class WSGIServer(HTTPServer): + + """BaseHTTPServer that implements the Python WSGI protocol""" + + application = None + + def server_bind(self): + """Override server_bind to store the server name.""" + HTTPServer.server_bind(self) + self.setup_environ() + + def setup_environ(self): + # Set up base environment + env = self.base_environ = {} + env['SERVER_NAME'] = self.server_name + env['GATEWAY_INTERFACE'] = 'CGI/1.1' + env['SERVER_PORT'] = str(self.server_port) + env['REMOTE_HOST']='' + env['CONTENT_LENGTH']='' + env['SCRIPT_NAME'] = '' + + def get_app(self): + return self.application + + def set_app(self,application): + self.application = application + + + + + + + + + + + + + + + +class WSGIRequestHandler(BaseHTTPRequestHandler): + + server_version = "WSGIServer/" + __version__ + + def get_environ(self): + env = self.server.base_environ.copy() + env['SERVER_PROTOCOL'] = self.request_version + env['REQUEST_METHOD'] = self.command + if '?' in self.path: + path,query = self.path.split('?',1) + else: + path,query = self.path,'' + + env['PATH_INFO'] = urllib.unquote(path) + env['QUERY_STRING'] = query + + host = self.address_string() + if host != self.client_address[0]: + env['REMOTE_HOST'] = host + env['REMOTE_ADDR'] = self.client_address[0] + + if self.headers.typeheader is None: + env['CONTENT_TYPE'] = self.headers.type + else: + env['CONTENT_TYPE'] = self.headers.typeheader + + length = self.headers.getheader('content-length') + if length: + env['CONTENT_LENGTH'] = length + + for h in self.headers.headers: + k,v = h.split(':',1) + k=k.replace('-','_').upper(); v=v.strip() + if k in env: + continue # skip content length, type,etc. + if 'HTTP_'+k in env: + env['HTTP_'+k] += ','+v # comma-separate multiple headers + else: + env['HTTP_'+k] = v + return env + + def get_stderr(self): + return sys.stderr + + def handle(self): + """Handle a single HTTP request""" + + self.raw_requestline = self.rfile.readline() + if not self.parse_request(): # An error code has been sent, just exit + return + + handler = ServerHandler( + self.rfile, self.wfile, self.get_stderr(), self.get_environ() + ) + handler.request_handler = self # backpointer for logging + handler.run(self.server.get_app()) + + + + + + + + + + + + + + + + + + + + + + + + + + +def demo_app(environ,start_response): + from StringIO import StringIO + stdout = StringIO() + print >>stdout, "Hello world!" + print >>stdout + h = environ.items(); h.sort() + for k,v in h: + print >>stdout, k,'=',`v` + start_response("200 OK", [('Content-Type','text/plain')]) + return [stdout.getvalue()] + + +def make_server( + host, port, app, server_class=WSGIServer, handler_class=WSGIRequestHandler +): + """Create a new WSGI server listening on `host` and `port` for `app`""" + server = server_class((host, port), handler_class) + server.set_app(app) + return server + + +if __name__ == '__main__': + httpd = make_server('', 8000, demo_app) + sa = httpd.socket.getsockname() + print "Serving HTTP on", sa[0], "port", sa[1], "..." + import webbrowser + webbrowser.open('http://localhost:8000/xyz?abc') + httpd.handle_request() # serve one request, then exit + + + + + + + + + + + + +# diff --git a/Lib/wsgiref/util.py b/Lib/wsgiref/util.py new file mode 100644 index 0000000..9009b87 --- /dev/null +++ b/Lib/wsgiref/util.py @@ -0,0 +1,205 @@ +"""Miscellaneous WSGI-related Utilities""" + +import posixpath + +__all__ = [ + 'FileWrapper', 'guess_scheme', 'application_uri', 'request_uri', + 'shift_path_info', 'setup_testing_defaults', +] + + +class FileWrapper: + """Wrapper to convert file-like objects to iterables""" + + def __init__(self, filelike, blksize=8192): + self.filelike = filelike + self.blksize = blksize + if hasattr(filelike,'close'): + self.close = filelike.close + + def __getitem__(self,key): + data = self.filelike.read(self.blksize) + if data: + return data + raise IndexError + + def __iter__(self): + return self + + def next(self): + data = self.filelike.read(self.blksize) + if data: + return data + raise StopIteration + + + + + + + + +def guess_scheme(environ): + """Return a guess for whether 'wsgi.url_scheme' should be 'http' or 'https' + """ + if environ.get("HTTPS") in ('yes','on','1'): + return 'https' + else: + return 'http' + +def application_uri(environ): + """Return the application's base URI (no PATH_INFO or QUERY_STRING)""" + url = environ['wsgi.url_scheme']+'://' + from urllib import quote + + if environ.get('HTTP_HOST'): + url += environ['HTTP_HOST'] + else: + url += environ['SERVER_NAME'] + + if environ['wsgi.url_scheme'] == 'https': + if environ['SERVER_PORT'] != '443': + url += ':' + environ['SERVER_PORT'] + else: + if environ['SERVER_PORT'] != '80': + url += ':' + environ['SERVER_PORT'] + + url += quote(environ.get('SCRIPT_NAME') or '/') + return url + +def request_uri(environ, include_query=1): + """Return the full request URI, optionally including the query string""" + url = application_uri(environ) + from urllib import quote + path_info = quote(environ.get('PATH_INFO','')) + if not environ.get('SCRIPT_NAME'): + url += path_info[1:] + else: + url += path_info + if include_query and environ.get('QUERY_STRING'): + url += '?' + environ['QUERY_STRING'] + return url + +def shift_path_info(environ): + """Shift a name from PATH_INFO to SCRIPT_NAME, returning it + + If there are no remaining path segments in PATH_INFO, return None. + Note: 'environ' is modified in-place; use a copy if you need to keep + the original PATH_INFO or SCRIPT_NAME. + + Note: when PATH_INFO is just a '/', this returns '' and appends a trailing + '/' to SCRIPT_NAME, even though empty path segments are normally ignored, + and SCRIPT_NAME doesn't normally end in a '/'. This is intentional + behavior, to ensure that an application can tell the difference between + '/x' and '/x/' when traversing to objects. + """ + path_info = environ.get('PATH_INFO','') + if not path_info: + return None + + path_parts = path_info.split('/') + path_parts[1:-1] = [p for p in path_parts[1:-1] if p and p<>'.'] + name = path_parts[1] + del path_parts[1] + + script_name = environ.get('SCRIPT_NAME','') + script_name = posixpath.normpath(script_name+'/'+name) + if script_name.endswith('/'): + script_name = script_name[:-1] + if not name and not script_name.endswith('/'): + script_name += '/' + + environ['SCRIPT_NAME'] = script_name + environ['PATH_INFO'] = '/'.join(path_parts) + + # Special case: '/.' on PATH_INFO doesn't get stripped, + # because we don't strip the last element of PATH_INFO + # if there's only one path part left. Instead of fixing this + # above, we fix it here so that PATH_INFO gets normalized to + # an empty string in the environ. + if name=='.': + name = None + return name + +def setup_testing_defaults(environ): + """Update 'environ' with trivial defaults for testing purposes + + This adds various parameters required for WSGI, including HTTP_HOST, + SERVER_NAME, SERVER_PORT, REQUEST_METHOD, SCRIPT_NAME, PATH_INFO, + and all of the wsgi.* variables. It only supplies default values, + and does not replace any existing settings for these variables. + + This routine is intended to make it easier for unit tests of WSGI + servers and applications to set up dummy environments. It should *not* + be used by actual WSGI servers or applications, since the data is fake! + """ + + environ.setdefault('SERVER_NAME','127.0.0.1') + environ.setdefault('SERVER_PROTOCOL','HTTP/1.0') + + environ.setdefault('HTTP_HOST',environ['SERVER_NAME']) + environ.setdefault('REQUEST_METHOD','GET') + + if 'SCRIPT_NAME' not in environ and 'PATH_INFO' not in environ: + environ.setdefault('SCRIPT_NAME','') + environ.setdefault('PATH_INFO','/') + + environ.setdefault('wsgi.version', (1,0)) + environ.setdefault('wsgi.run_once', 0) + environ.setdefault('wsgi.multithread', 0) + environ.setdefault('wsgi.multiprocess', 0) + + from StringIO import StringIO + environ.setdefault('wsgi.input', StringIO("")) + environ.setdefault('wsgi.errors', StringIO()) + environ.setdefault('wsgi.url_scheme',guess_scheme(environ)) + + if environ['wsgi.url_scheme']=='http': + environ.setdefault('SERVER_PORT', '80') + elif environ['wsgi.url_scheme']=='https': + environ.setdefault('SERVER_PORT', '443') + + + + +_hoppish = { + 'connection':1, 'keep-alive':1, 'proxy-authenticate':1, + 'proxy-authorization':1, 'te':1, 'trailers':1, 'transfer-encoding':1, + 'upgrade':1 +}.has_key + +def is_hop_by_hop(header_name): + """Return true if 'header_name' is an HTTP/1.1 "Hop-by-Hop" header""" + return _hoppish(header_name.lower()) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +# diff --git a/Lib/wsgiref/validate.py b/Lib/wsgiref/validate.py new file mode 100644 index 0000000..23ab9f8 --- /dev/null +++ b/Lib/wsgiref/validate.py @@ -0,0 +1,432 @@ +# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org) +# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php +# Also licenced under the Apache License, 2.0: http://opensource.org/licenses/apache2.0.php +# Licensed to PSF under a Contributor Agreement +""" +Middleware to check for obedience to the WSGI specification. + +Some of the things this checks: + +* Signature of the application and start_response (including that + keyword arguments are not used). + +* Environment checks: + + - Environment is a dictionary (and not a subclass). + + - That all the required keys are in the environment: REQUEST_METHOD, + SERVER_NAME, SERVER_PORT, wsgi.version, wsgi.input, wsgi.errors, + wsgi.multithread, wsgi.multiprocess, wsgi.run_once + + - That HTTP_CONTENT_TYPE and HTTP_CONTENT_LENGTH are not in the + environment (these headers should appear as CONTENT_LENGTH and + CONTENT_TYPE). + + - Warns if QUERY_STRING is missing, as the cgi module acts + unpredictably in that case. + + - That CGI-style variables (that don't contain a .) have + (non-unicode) string values + + - That wsgi.version is a tuple + + - That wsgi.url_scheme is 'http' or 'https' (@@: is this too + restrictive?) + + - Warns if the REQUEST_METHOD is not known (@@: probably too + restrictive). + + - That SCRIPT_NAME and PATH_INFO are empty or start with / + + - That at least one of SCRIPT_NAME or PATH_INFO are set. + + - That CONTENT_LENGTH is a positive integer. + + - That SCRIPT_NAME is not '/' (it should be '', and PATH_INFO should + be '/'). + + - That wsgi.input has the methods read, readline, readlines, and + __iter__ + + - That wsgi.errors has the methods flush, write, writelines + +* The status is a string, contains a space, starts with an integer, + and that integer is in range (> 100). + +* That the headers is a list (not a subclass, not another kind of + sequence). + +* That the items of the headers are tuples of strings. + +* That there is no 'status' header (that is used in CGI, but not in + WSGI). + +* That the headers don't contain newlines or colons, end in _ or -, or + contain characters codes below 037. + +* That Content-Type is given if there is content (CGI often has a + default content type, but WSGI does not). + +* That no Content-Type is given when there is no content (@@: is this + too restrictive?) + +* That the exc_info argument to start_response is a tuple or None. + +* That all calls to the writer are with strings, and no other methods + on the writer are accessed. + +* That wsgi.input is used properly: + + - .read() is called with zero or one argument + + - That it returns a string + + - That readline, readlines, and __iter__ return strings + + - That .close() is not called + + - No other methods are provided + +* That wsgi.errors is used properly: + + - .write() and .writelines() is called with a string + + - That .close() is not called, and no other methods are provided. + +* The response iterator: + + - That it is not a string (it should be a list of a single string; a + string will work, but perform horribly). + + - That .next() returns a string + + - That the iterator is not iterated over until start_response has + been called (that can signal either a server or application + error). + + - That .close() is called (doesn't raise exception, only prints to + sys.stderr, because we only know it isn't called when the object + is garbage collected). +""" +__all__ = ['validator'] + + +import re +import sys +from types import DictType, StringType, TupleType, ListType +import warnings + +header_re = re.compile(r'^[a-zA-Z][a-zA-Z0-9\-_]*$') +bad_header_value_re = re.compile(r'[\000-\037]') + +class WSGIWarning(Warning): + """ + Raised in response to WSGI-spec-related warnings + """ + +def assert_(cond, *args): + if not cond: + raise AssertionError(*args) + +def validator(application): + + """ + When applied between a WSGI server and a WSGI application, this + middleware will check for WSGI compliancy on a number of levels. + This middleware does not modify the request or response in any + way, but will throw an AssertionError if anything seems off + (except for a failure to close the application iterator, which + will be printed to stderr -- there's no way to throw an exception + at that point). + """ + + def lint_app(*args, **kw): + assert_(len(args) == 2, "Two arguments required") + assert_(not kw, "No keyword arguments allowed") + environ, start_response = args + + check_environ(environ) + + # We use this to check if the application returns without + # calling start_response: + start_response_started = [] + + def start_response_wrapper(*args, **kw): + assert_(len(args) == 2 or len(args) == 3, ( + "Invalid number of arguments: %s" % (args,))) + assert_(not kw, "No keyword arguments allowed") + status = args[0] + headers = args[1] + if len(args) == 3: + exc_info = args[2] + else: + exc_info = None + + check_status(status) + check_headers(headers) + check_content_type(status, headers) + check_exc_info(exc_info) + + start_response_started.append(None) + return WriteWrapper(start_response(*args)) + + environ['wsgi.input'] = InputWrapper(environ['wsgi.input']) + environ['wsgi.errors'] = ErrorWrapper(environ['wsgi.errors']) + + iterator = application(environ, start_response_wrapper) + assert_(iterator is not None and iterator != False, + "The application must return an iterator, if only an empty list") + + check_iterator(iterator) + + return IteratorWrapper(iterator, start_response_started) + + return lint_app + +class InputWrapper: + + def __init__(self, wsgi_input): + self.input = wsgi_input + + def read(self, *args): + assert_(len(args) <= 1) + v = self.input.read(*args) + assert_(type(v) is type("")) + return v + + def readline(self): + v = self.input.readline() + assert_(type(v) is type("")) + return v + + def readlines(self, *args): + assert_(len(args) <= 1) + lines = self.input.readlines(*args) + assert_(type(lines) is type([])) + for line in lines: + assert_(type(line) is type("")) + return lines + + def __iter__(self): + while 1: + line = self.readline() + if not line: + return + yield line + + def close(self): + assert_(0, "input.close() must not be called") + +class ErrorWrapper: + + def __init__(self, wsgi_errors): + self.errors = wsgi_errors + + def write(self, s): + assert_(type(s) is type("")) + self.errors.write(s) + + def flush(self): + self.errors.flush() + + def writelines(self, seq): + for line in seq: + self.write(line) + + def close(self): + assert_(0, "errors.close() must not be called") + +class WriteWrapper: + + def __init__(self, wsgi_writer): + self.writer = wsgi_writer + + def __call__(self, s): + assert_(type(s) is type("")) + self.writer(s) + +class PartialIteratorWrapper: + + def __init__(self, wsgi_iterator): + self.iterator = wsgi_iterator + + def __iter__(self): + # We want to make sure __iter__ is called + return IteratorWrapper(self.iterator, None) + +class IteratorWrapper: + + def __init__(self, wsgi_iterator, check_start_response): + self.original_iterator = wsgi_iterator + self.iterator = iter(wsgi_iterator) + self.closed = False + self.check_start_response = check_start_response + + def __iter__(self): + return self + + def next(self): + assert_(not self.closed, + "Iterator read after closed") + v = self.iterator.next() + if self.check_start_response is not None: + assert_(self.check_start_response, + "The application returns and we started iterating over its body, but start_response has not yet been called") + self.check_start_response = None + return v + + def close(self): + self.closed = True + if hasattr(self.original_iterator, 'close'): + self.original_iterator.close() + + def __del__(self): + if not self.closed: + sys.stderr.write( + "Iterator garbage collected without being closed") + assert_(self.closed, + "Iterator garbage collected without being closed") + +def check_environ(environ): + assert_(type(environ) is DictType, + "Environment is not of the right type: %r (environment: %r)" + % (type(environ), environ)) + + for key in ['REQUEST_METHOD', 'SERVER_NAME', 'SERVER_PORT', + 'wsgi.version', 'wsgi.input', 'wsgi.errors', + 'wsgi.multithread', 'wsgi.multiprocess', + 'wsgi.run_once']: + assert_(key in environ, + "Environment missing required key: %r" % (key,)) + + for key in ['HTTP_CONTENT_TYPE', 'HTTP_CONTENT_LENGTH']: + assert_(key not in environ, + "Environment should not have the key: %s " + "(use %s instead)" % (key, key[5:])) + + if 'QUERY_STRING' not in environ: + warnings.warn( + 'QUERY_STRING is not in the WSGI environment; the cgi ' + 'module will use sys.argv when this variable is missing, ' + 'so application errors are more likely', + WSGIWarning) + + for key in environ.keys(): + if '.' in key: + # Extension, we don't care about its type + continue + assert_(type(environ[key]) is StringType, + "Environmental variable %s is not a string: %r (value: %r)" + % (key, type(environ[key]), environ[key])) + + assert_(type(environ['wsgi.version']) is TupleType, + "wsgi.version should be a tuple (%r)" % (environ['wsgi.version'],)) + assert_(environ['wsgi.url_scheme'] in ('http', 'https'), + "wsgi.url_scheme unknown: %r" % environ['wsgi.url_scheme']) + + check_input(environ['wsgi.input']) + check_errors(environ['wsgi.errors']) + + # @@: these need filling out: + if environ['REQUEST_METHOD'] not in ( + 'GET', 'HEAD', 'POST', 'OPTIONS','PUT','DELETE','TRACE'): + warnings.warn( + "Unknown REQUEST_METHOD: %r" % environ['REQUEST_METHOD'], + WSGIWarning) + + assert_(not environ.get('SCRIPT_NAME') + or environ['SCRIPT_NAME'].startswith('/'), + "SCRIPT_NAME doesn't start with /: %r" % environ['SCRIPT_NAME']) + assert_(not environ.get('PATH_INFO') + or environ['PATH_INFO'].startswith('/'), + "PATH_INFO doesn't start with /: %r" % environ['PATH_INFO']) + if environ.get('CONTENT_LENGTH'): + assert_(int(environ['CONTENT_LENGTH']) >= 0, + "Invalid CONTENT_LENGTH: %r" % environ['CONTENT_LENGTH']) + + if not environ.get('SCRIPT_NAME'): + assert_(environ.has_key('PATH_INFO'), + "One of SCRIPT_NAME or PATH_INFO are required (PATH_INFO " + "should at least be '/' if SCRIPT_NAME is empty)") + assert_(environ.get('SCRIPT_NAME') != '/', + "SCRIPT_NAME cannot be '/'; it should instead be '', and " + "PATH_INFO should be '/'") + +def check_input(wsgi_input): + for attr in ['read', 'readline', 'readlines', '__iter__']: + assert_(hasattr(wsgi_input, attr), + "wsgi.input (%r) doesn't have the attribute %s" + % (wsgi_input, attr)) + +def check_errors(wsgi_errors): + for attr in ['flush', 'write', 'writelines']: + assert_(hasattr(wsgi_errors, attr), + "wsgi.errors (%r) doesn't have the attribute %s" + % (wsgi_errors, attr)) + +def check_status(status): + assert_(type(status) is StringType, + "Status must be a string (not %r)" % status) + # Implicitly check that we can turn it into an integer: + status_code = status.split(None, 1)[0] + assert_(len(status_code) == 3, + "Status codes must be three characters: %r" % status_code) + status_int = int(status_code) + assert_(status_int >= 100, "Status code is invalid: %r" % status_int) + if len(status) < 4 or status[3] != ' ': + warnings.warn( + "The status string (%r) should be a three-digit integer " + "followed by a single space and a status explanation" + % status, WSGIWarning) + +def check_headers(headers): + assert_(type(headers) is ListType, + "Headers (%r) must be of type list: %r" + % (headers, type(headers))) + header_names = {} + for item in headers: + assert_(type(item) is TupleType, + "Individual headers (%r) must be of type tuple: %r" + % (item, type(item))) + assert_(len(item) == 2) + name, value = item + assert_(name.lower() != 'status', + "The Status header cannot be used; it conflicts with CGI " + "script, and HTTP status is not given through headers " + "(value: %r)." % value) + header_names[name.lower()] = None + assert_('\n' not in name and ':' not in name, + "Header names may not contain ':' or '\\n': %r" % name) + assert_(header_re.search(name), "Bad header name: %r" % name) + assert_(not name.endswith('-') and not name.endswith('_'), + "Names may not end in '-' or '_': %r" % name) + if bad_header_value_re.search(value): + assert_(0, "Bad header value: %r (bad char: %r)" + % (value, bad_header_value_re.search(value).group(0))) + +def check_content_type(status, headers): + code = int(status.split(None, 1)[0]) + # @@: need one more person to verify this interpretation of RFC 2616 + # http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html + NO_MESSAGE_BODY = (204, 304) + for name, value in headers: + if name.lower() == 'content-type': + if code not in NO_MESSAGE_BODY: + return + assert_(0, ("Content-Type header found in a %s response, " + "which must not return content.") % code) + if code not in NO_MESSAGE_BODY: + assert_(0, "No Content-Type header found in headers (%s)" % headers) + +def check_exc_info(exc_info): + assert_(exc_info is None or type(exc_info) is type(()), + "exc_info (%r) is not a tuple: %r" % (exc_info, type(exc_info))) + # More exc_info checks? + +def check_iterator(iterator): + # Technically a string is legal, which is why it's a really bad + # idea, because it may cause the response to be returned + # character-by-character + assert_(not isinstance(iterator, str), + "You should not return a string as your application iterator, " + "instead return a single-item list containing that string.") diff --git a/Lib/xml.py b/Lib/xml.py deleted file mode 100644 index 7393c66..0000000 --- a/Lib/xml.py +++ /dev/null @@ -1,47 +0,0 @@ -"""Core XML support for Python. - -This package contains four sub-packages: - -dom -- The W3C Document Object Model. This supports DOM Level 1 + - Namespaces. - -parsers -- Python wrappers for XML parsers (currently only supports Expat). - -sax -- The Simple API for XML, developed by XML-Dev, led by David - Megginson and ported to Python by Lars Marius Garshol. This - supports the SAX 2 API. - -etree -- The ElementTree XML library. This is a subset of the full - ElementTree XML release. - -""" - -import sys -import xmlcore - -__all__ = ["dom", "parsers", "sax", "etree"] - -# When being checked-out without options, this has the form -# "Revision: x.y " -# When exported using -kv, it is "x.y". -__version__ = "$Revision$".split()[-2:][0] - - -_MINIMUM_XMLPLUS_VERSION = (0, 8, 4) - -try: - import _xmlplus -except ImportError: - sys.modules[__name__] = xmlcore -else: - try: - v = _xmlplus.version_info - except AttributeError: - # _xmlplus is too old; ignore it - pass - else: - if v >= _MINIMUM_XMLPLUS_VERSION: - _xmlplus.__path__.extend(xmlcore.__path__) - sys.modules[__name__] = _xmlplus - else: - del v diff --git a/Lib/xml/__init__.py b/Lib/xml/__init__.py new file mode 100644 index 0000000..fa5e8cd --- /dev/null +++ b/Lib/xml/__init__.py @@ -0,0 +1,47 @@ +"""Core XML support for Python. + +This package contains four sub-packages: + +dom -- The W3C Document Object Model. This supports DOM Level 1 + + Namespaces. + +parsers -- Python wrappers for XML parsers (currently only supports Expat). + +sax -- The Simple API for XML, developed by XML-Dev, led by David + Megginson and ported to Python by Lars Marius Garshol. This + supports the SAX 2 API. + +etree -- The ElementTree XML library. This is a subset of the full + ElementTree XML release. + +""" + + +__all__ = ["dom", "parsers", "sax", "etree"] + +# When being checked-out without options, this has the form +# "Revision: x.y " +# When exported using -kv, it is "x.y". +__version__ = "$Revision$".split()[-2:][0] + + +_MINIMUM_XMLPLUS_VERSION = (0, 8, 4) + + +try: + import _xmlplus +except ImportError: + pass +else: + try: + v = _xmlplus.version_info + except AttributeError: + # _xmlplus is too old; ignore it + pass + else: + if v >= _MINIMUM_XMLPLUS_VERSION: + import sys + _xmlplus.__path__.extend(__path__) + sys.modules[__name__] = _xmlplus + else: + del v diff --git a/Lib/xml/dom/NodeFilter.py b/Lib/xml/dom/NodeFilter.py new file mode 100644 index 0000000..fc05245 --- /dev/null +++ b/Lib/xml/dom/NodeFilter.py @@ -0,0 +1,27 @@ +# This is the Python mapping for interface NodeFilter from +# DOM2-Traversal-Range. It contains only constants. + +class NodeFilter: + """ + This is the DOM2 NodeFilter interface. It contains only constants. + """ + FILTER_ACCEPT = 1 + FILTER_REJECT = 2 + FILTER_SKIP = 3 + + SHOW_ALL = 0xFFFFFFFFL + SHOW_ELEMENT = 0x00000001 + SHOW_ATTRIBUTE = 0x00000002 + SHOW_TEXT = 0x00000004 + SHOW_CDATA_SECTION = 0x00000008 + SHOW_ENTITY_REFERENCE = 0x00000010 + SHOW_ENTITY = 0x00000020 + SHOW_PROCESSING_INSTRUCTION = 0x00000040 + SHOW_COMMENT = 0x00000080 + SHOW_DOCUMENT = 0x00000100 + SHOW_DOCUMENT_TYPE = 0x00000200 + SHOW_DOCUMENT_FRAGMENT = 0x00000400 + SHOW_NOTATION = 0x00000800 + + def acceptNode(self, node): + raise NotImplementedError diff --git a/Lib/xml/dom/__init__.py b/Lib/xml/dom/__init__.py new file mode 100644 index 0000000..6363d00 --- /dev/null +++ b/Lib/xml/dom/__init__.py @@ -0,0 +1,139 @@ +"""W3C Document Object Model implementation for Python. + +The Python mapping of the Document Object Model is documented in the +Python Library Reference in the section on the xml.dom package. + +This package contains the following modules: + +minidom -- A simple implementation of the Level 1 DOM with namespace + support added (based on the Level 2 specification) and other + minor Level 2 functionality. + +pulldom -- DOM builder supporting on-demand tree-building for selected + subtrees of the document. + +""" + + +class Node: + """Class giving the NodeType constants.""" + + # DOM implementations may use this as a base class for their own + # Node implementations. If they don't, the constants defined here + # should still be used as the canonical definitions as they match + # the values given in the W3C recommendation. Client code can + # safely refer to these values in all tests of Node.nodeType + # values. + + ELEMENT_NODE = 1 + ATTRIBUTE_NODE = 2 + TEXT_NODE = 3 + CDATA_SECTION_NODE = 4 + ENTITY_REFERENCE_NODE = 5 + ENTITY_NODE = 6 + PROCESSING_INSTRUCTION_NODE = 7 + COMMENT_NODE = 8 + DOCUMENT_NODE = 9 + DOCUMENT_TYPE_NODE = 10 + DOCUMENT_FRAGMENT_NODE = 11 + NOTATION_NODE = 12 + + +#ExceptionCode +INDEX_SIZE_ERR = 1 +DOMSTRING_SIZE_ERR = 2 +HIERARCHY_REQUEST_ERR = 3 +WRONG_DOCUMENT_ERR = 4 +INVALID_CHARACTER_ERR = 5 +NO_DATA_ALLOWED_ERR = 6 +NO_MODIFICATION_ALLOWED_ERR = 7 +NOT_FOUND_ERR = 8 +NOT_SUPPORTED_ERR = 9 +INUSE_ATTRIBUTE_ERR = 10 +INVALID_STATE_ERR = 11 +SYNTAX_ERR = 12 +INVALID_MODIFICATION_ERR = 13 +NAMESPACE_ERR = 14 +INVALID_ACCESS_ERR = 15 +VALIDATION_ERR = 16 + + +class DOMException(Exception): + """Abstract base class for DOM exceptions. + Exceptions with specific codes are specializations of this class.""" + + def __init__(self, *args, **kw): + if self.__class__ is DOMException: + raise RuntimeError( + "DOMException should not be instantiated directly") + Exception.__init__(self, *args, **kw) + + def _get_code(self): + return self.code + + +class IndexSizeErr(DOMException): + code = INDEX_SIZE_ERR + +class DomstringSizeErr(DOMException): + code = DOMSTRING_SIZE_ERR + +class HierarchyRequestErr(DOMException): + code = HIERARCHY_REQUEST_ERR + +class WrongDocumentErr(DOMException): + code = WRONG_DOCUMENT_ERR + +class InvalidCharacterErr(DOMException): + code = INVALID_CHARACTER_ERR + +class NoDataAllowedErr(DOMException): + code = NO_DATA_ALLOWED_ERR + +class NoModificationAllowedErr(DOMException): + code = NO_MODIFICATION_ALLOWED_ERR + +class NotFoundErr(DOMException): + code = NOT_FOUND_ERR + +class NotSupportedErr(DOMException): + code = NOT_SUPPORTED_ERR + +class InuseAttributeErr(DOMException): + code = INUSE_ATTRIBUTE_ERR + +class InvalidStateErr(DOMException): + code = INVALID_STATE_ERR + +class SyntaxErr(DOMException): + code = SYNTAX_ERR + +class InvalidModificationErr(DOMException): + code = INVALID_MODIFICATION_ERR + +class NamespaceErr(DOMException): + code = NAMESPACE_ERR + +class InvalidAccessErr(DOMException): + code = INVALID_ACCESS_ERR + +class ValidationErr(DOMException): + code = VALIDATION_ERR + +class UserDataHandler: + """Class giving the operation constants for UserDataHandler.handle().""" + + # Based on DOM Level 3 (WD 9 April 2002) + + NODE_CLONED = 1 + NODE_IMPORTED = 2 + NODE_DELETED = 3 + NODE_RENAMED = 4 + +XML_NAMESPACE = "http://www.w3.org/XML/1998/namespace" +XMLNS_NAMESPACE = "http://www.w3.org/2000/xmlns/" +XHTML_NAMESPACE = "http://www.w3.org/1999/xhtml" +EMPTY_NAMESPACE = None +EMPTY_PREFIX = None + +from domreg import getDOMImplementation,registerDOMImplementation diff --git a/Lib/xml/dom/domreg.py b/Lib/xml/dom/domreg.py new file mode 100644 index 0000000..684c436 --- /dev/null +++ b/Lib/xml/dom/domreg.py @@ -0,0 +1,99 @@ +"""Registration facilities for DOM. This module should not be used +directly. Instead, the functions getDOMImplementation and +registerDOMImplementation should be imported from xml.dom.""" + +from xml.dom.minicompat import * # isinstance, StringTypes + +# This is a list of well-known implementations. Well-known names +# should be published by posting to xml-sig@python.org, and are +# subsequently recorded in this file. + +well_known_implementations = { + 'minidom':'xml.dom.minidom', + '4DOM': 'xml.dom.DOMImplementation', + } + +# DOM implementations not officially registered should register +# themselves with their + +registered = {} + +def registerDOMImplementation(name, factory): + """registerDOMImplementation(name, factory) + + Register the factory function with the name. The factory function + should return an object which implements the DOMImplementation + interface. The factory function can either return the same object, + or a new one (e.g. if that implementation supports some + customization).""" + + registered[name] = factory + +def _good_enough(dom, features): + "_good_enough(dom, features) -> Return 1 if the dom offers the features" + for f,v in features: + if not dom.hasFeature(f,v): + return 0 + return 1 + +def getDOMImplementation(name = None, features = ()): + """getDOMImplementation(name = None, features = ()) -> DOM implementation. + + Return a suitable DOM implementation. The name is either + well-known, the module name of a DOM implementation, or None. If + it is not None, imports the corresponding module and returns + DOMImplementation object if the import succeeds. + + If name is not given, consider the available implementations to + find one with the required feature set. If no implementation can + be found, raise an ImportError. The features list must be a sequence + of (feature, version) pairs which are passed to hasFeature.""" + + import os + creator = None + mod = well_known_implementations.get(name) + if mod: + mod = __import__(mod, {}, {}, ['getDOMImplementation']) + return mod.getDOMImplementation() + elif name: + return registered[name]() + elif os.environ.has_key("PYTHON_DOM"): + return getDOMImplementation(name = os.environ["PYTHON_DOM"]) + + # User did not specify a name, try implementations in arbitrary + # order, returning the one that has the required features + if isinstance(features, StringTypes): + features = _parse_feature_string(features) + for creator in registered.values(): + dom = creator() + if _good_enough(dom, features): + return dom + + for creator in well_known_implementations.keys(): + try: + dom = getDOMImplementation(name = creator) + except StandardError: # typically ImportError, or AttributeError + continue + if _good_enough(dom, features): + return dom + + raise ImportError,"no suitable DOM implementation found" + +def _parse_feature_string(s): + features = [] + parts = s.split() + i = 0 + length = len(parts) + while i < length: + feature = parts[i] + if feature[0] in "0123456789": + raise ValueError, "bad feature name: %r" % (feature,) + i = i + 1 + version = None + if i < length: + v = parts[i] + if v[0] in "0123456789": + i = i + 1 + version = v + features.append((feature, version)) + return tuple(features) diff --git a/Lib/xml/dom/expatbuilder.py b/Lib/xml/dom/expatbuilder.py new file mode 100644 index 0000000..a2f8a33 --- /dev/null +++ b/Lib/xml/dom/expatbuilder.py @@ -0,0 +1,983 @@ +"""Facility to use the Expat parser to load a minidom instance +from a string or file. + +This avoids all the overhead of SAX and pulldom to gain performance. +""" + +# Warning! +# +# This module is tightly bound to the implementation details of the +# minidom DOM and can't be used with other DOM implementations. This +# is due, in part, to a lack of appropriate methods in the DOM (there is +# no way to create Entity and Notation nodes via the DOM Level 2 +# interface), and for performance. The later is the cause of some fairly +# cryptic code. +# +# Performance hacks: +# +# - .character_data_handler() has an extra case in which continuing +# data is appended to an existing Text node; this can be a +# speedup since pyexpat can break up character data into multiple +# callbacks even though we set the buffer_text attribute on the +# parser. This also gives us the advantage that we don't need a +# separate normalization pass. +# +# - Determining that a node exists is done using an identity comparison +# with None rather than a truth test; this avoids searching for and +# calling any methods on the node object if it exists. (A rather +# nice speedup is achieved this way as well!) + +from xml.dom import xmlbuilder, minidom, Node +from xml.dom import EMPTY_NAMESPACE, EMPTY_PREFIX, XMLNS_NAMESPACE +from xml.parsers import expat +from xml.dom.minidom import _append_child, _set_attribute_node +from xml.dom.NodeFilter import NodeFilter + +from xml.dom.minicompat import * + +TEXT_NODE = Node.TEXT_NODE +CDATA_SECTION_NODE = Node.CDATA_SECTION_NODE +DOCUMENT_NODE = Node.DOCUMENT_NODE + +FILTER_ACCEPT = xmlbuilder.DOMBuilderFilter.FILTER_ACCEPT +FILTER_REJECT = xmlbuilder.DOMBuilderFilter.FILTER_REJECT +FILTER_SKIP = xmlbuilder.DOMBuilderFilter.FILTER_SKIP +FILTER_INTERRUPT = xmlbuilder.DOMBuilderFilter.FILTER_INTERRUPT + +theDOMImplementation = minidom.getDOMImplementation() + +# Expat typename -> TypeInfo +_typeinfo_map = { + "CDATA": minidom.TypeInfo(None, "cdata"), + "ENUM": minidom.TypeInfo(None, "enumeration"), + "ENTITY": minidom.TypeInfo(None, "entity"), + "ENTITIES": minidom.TypeInfo(None, "entities"), + "ID": minidom.TypeInfo(None, "id"), + "IDREF": minidom.TypeInfo(None, "idref"), + "IDREFS": minidom.TypeInfo(None, "idrefs"), + "NMTOKEN": minidom.TypeInfo(None, "nmtoken"), + "NMTOKENS": minidom.TypeInfo(None, "nmtokens"), + } + +class ElementInfo(object): + __slots__ = '_attr_info', '_model', 'tagName' + + def __init__(self, tagName, model=None): + self.tagName = tagName + self._attr_info = [] + self._model = model + + def __getstate__(self): + return self._attr_info, self._model, self.tagName + + def __setstate__(self, state): + self._attr_info, self._model, self.tagName = state + + def getAttributeType(self, aname): + for info in self._attr_info: + if info[1] == aname: + t = info[-2] + if t[0] == "(": + return _typeinfo_map["ENUM"] + else: + return _typeinfo_map[info[-2]] + return minidom._no_type + + def getAttributeTypeNS(self, namespaceURI, localName): + return minidom._no_type + + def isElementContent(self): + if self._model: + type = self._model[0] + return type not in (expat.model.XML_CTYPE_ANY, + expat.model.XML_CTYPE_MIXED) + else: + return False + + def isEmpty(self): + if self._model: + return self._model[0] == expat.model.XML_CTYPE_EMPTY + else: + return False + + def isId(self, aname): + for info in self._attr_info: + if info[1] == aname: + return info[-2] == "ID" + return False + + def isIdNS(self, euri, ename, auri, aname): + # not sure this is meaningful + return self.isId((auri, aname)) + +def _intern(builder, s): + return builder._intern_setdefault(s, s) + +def _parse_ns_name(builder, name): + assert ' ' in name + parts = name.split(' ') + intern = builder._intern_setdefault + if len(parts) == 3: + uri, localname, prefix = parts + prefix = intern(prefix, prefix) + qname = "%s:%s" % (prefix, localname) + qname = intern(qname, qname) + localname = intern(localname, localname) + else: + uri, localname = parts + prefix = EMPTY_PREFIX + qname = localname = intern(localname, localname) + return intern(uri, uri), localname, prefix, qname + + +class ExpatBuilder: + """Document builder that uses Expat to build a ParsedXML.DOM document + instance.""" + + def __init__(self, options=None): + if options is None: + options = xmlbuilder.Options() + self._options = options + if self._options.filter is not None: + self._filter = FilterVisibilityController(self._options.filter) + else: + self._filter = None + # This *really* doesn't do anything in this case, so + # override it with something fast & minimal. + self._finish_start_element = id + self._parser = None + self.reset() + + def createParser(self): + """Create a new parser object.""" + return expat.ParserCreate() + + def getParser(self): + """Return the parser object, creating a new one if needed.""" + if not self._parser: + self._parser = self.createParser() + self._intern_setdefault = self._parser.intern.setdefault + self._parser.buffer_text = True + self._parser.ordered_attributes = True + self._parser.specified_attributes = True + self.install(self._parser) + return self._parser + + def reset(self): + """Free all data structures used during DOM construction.""" + self.document = theDOMImplementation.createDocument( + EMPTY_NAMESPACE, None, None) + self.curNode = self.document + self._elem_info = self.document._elem_info + self._cdata = False + + def install(self, parser): + """Install the callbacks needed to build the DOM into the parser.""" + # This creates circular references! + parser.StartDoctypeDeclHandler = self.start_doctype_decl_handler + parser.StartElementHandler = self.first_element_handler + parser.EndElementHandler = self.end_element_handler + parser.ProcessingInstructionHandler = self.pi_handler + if self._options.entities: + parser.EntityDeclHandler = self.entity_decl_handler + parser.NotationDeclHandler = self.notation_decl_handler + if self._options.comments: + parser.CommentHandler = self.comment_handler + if self._options.cdata_sections: + parser.StartCdataSectionHandler = self.start_cdata_section_handler + parser.EndCdataSectionHandler = self.end_cdata_section_handler + parser.CharacterDataHandler = self.character_data_handler_cdata + else: + parser.CharacterDataHandler = self.character_data_handler + parser.ExternalEntityRefHandler = self.external_entity_ref_handler + parser.XmlDeclHandler = self.xml_decl_handler + parser.ElementDeclHandler = self.element_decl_handler + parser.AttlistDeclHandler = self.attlist_decl_handler + + def parseFile(self, file): + """Parse a document from a file object, returning the document + node.""" + parser = self.getParser() + first_buffer = True + try: + while 1: + buffer = file.read(16*1024) + if not buffer: + break + parser.Parse(buffer, 0) + if first_buffer and self.document.documentElement: + self._setup_subset(buffer) + first_buffer = False + parser.Parse("", True) + except ParseEscape: + pass + doc = self.document + self.reset() + self._parser = None + return doc + + def parseString(self, string): + """Parse a document from a string, returning the document node.""" + parser = self.getParser() + try: + parser.Parse(string, True) + self._setup_subset(string) + except ParseEscape: + pass + doc = self.document + self.reset() + self._parser = None + return doc + + def _setup_subset(self, buffer): + """Load the internal subset if there might be one.""" + if self.document.doctype: + extractor = InternalSubsetExtractor() + extractor.parseString(buffer) + subset = extractor.getSubset() + self.document.doctype.internalSubset = subset + + def start_doctype_decl_handler(self, doctypeName, systemId, publicId, + has_internal_subset): + doctype = self.document.implementation.createDocumentType( + doctypeName, publicId, systemId) + doctype.ownerDocument = self.document + self.document.childNodes.append(doctype) + self.document.doctype = doctype + if self._filter and self._filter.acceptNode(doctype) == FILTER_REJECT: + self.document.doctype = None + del self.document.childNodes[-1] + doctype = None + self._parser.EntityDeclHandler = None + self._parser.NotationDeclHandler = None + if has_internal_subset: + if doctype is not None: + doctype.entities._seq = [] + doctype.notations._seq = [] + self._parser.CommentHandler = None + self._parser.ProcessingInstructionHandler = None + self._parser.EndDoctypeDeclHandler = self.end_doctype_decl_handler + + def end_doctype_decl_handler(self): + if self._options.comments: + self._parser.CommentHandler = self.comment_handler + self._parser.ProcessingInstructionHandler = self.pi_handler + if not (self._elem_info or self._filter): + self._finish_end_element = id + + def pi_handler(self, target, data): + node = self.document.createProcessingInstruction(target, data) + _append_child(self.curNode, node) + if self._filter and self._filter.acceptNode(node) == FILTER_REJECT: + self.curNode.removeChild(node) + + def character_data_handler_cdata(self, data): + childNodes = self.curNode.childNodes + if self._cdata: + if ( self._cdata_continue + and childNodes[-1].nodeType == CDATA_SECTION_NODE): + childNodes[-1].appendData(data) + return + node = self.document.createCDATASection(data) + self._cdata_continue = True + elif childNodes and childNodes[-1].nodeType == TEXT_NODE: + node = childNodes[-1] + value = node.data + data + d = node.__dict__ + d['data'] = d['nodeValue'] = value + return + else: + node = minidom.Text() + d = node.__dict__ + d['data'] = d['nodeValue'] = data + d['ownerDocument'] = self.document + _append_child(self.curNode, node) + + def character_data_handler(self, data): + childNodes = self.curNode.childNodes + if childNodes and childNodes[-1].nodeType == TEXT_NODE: + node = childNodes[-1] + d = node.__dict__ + d['data'] = d['nodeValue'] = node.data + data + return + node = minidom.Text() + d = node.__dict__ + d['data'] = d['nodeValue'] = node.data + data + d['ownerDocument'] = self.document + _append_child(self.curNode, node) + + def entity_decl_handler(self, entityName, is_parameter_entity, value, + base, systemId, publicId, notationName): + if is_parameter_entity: + # we don't care about parameter entities for the DOM + return + if not self._options.entities: + return + node = self.document._create_entity(entityName, publicId, + systemId, notationName) + if value is not None: + # internal entity + # node *should* be readonly, but we'll cheat + child = self.document.createTextNode(value) + node.childNodes.append(child) + self.document.doctype.entities._seq.append(node) + if self._filter and self._filter.acceptNode(node) == FILTER_REJECT: + del self.document.doctype.entities._seq[-1] + + def notation_decl_handler(self, notationName, base, systemId, publicId): + node = self.document._create_notation(notationName, publicId, systemId) + self.document.doctype.notations._seq.append(node) + if self._filter and self._filter.acceptNode(node) == FILTER_ACCEPT: + del self.document.doctype.notations._seq[-1] + + def comment_handler(self, data): + node = self.document.createComment(data) + _append_child(self.curNode, node) + if self._filter and self._filter.acceptNode(node) == FILTER_REJECT: + self.curNode.removeChild(node) + + def start_cdata_section_handler(self): + self._cdata = True + self._cdata_continue = False + + def end_cdata_section_handler(self): + self._cdata = False + self._cdata_continue = False + + def external_entity_ref_handler(self, context, base, systemId, publicId): + return 1 + + def first_element_handler(self, name, attributes): + if self._filter is None and not self._elem_info: + self._finish_end_element = id + self.getParser().StartElementHandler = self.start_element_handler + self.start_element_handler(name, attributes) + + def start_element_handler(self, name, attributes): + node = self.document.createElement(name) + _append_child(self.curNode, node) + self.curNode = node + + if attributes: + for i in range(0, len(attributes), 2): + a = minidom.Attr(attributes[i], EMPTY_NAMESPACE, + None, EMPTY_PREFIX) + value = attributes[i+1] + d = a.childNodes[0].__dict__ + d['data'] = d['nodeValue'] = value + d = a.__dict__ + d['value'] = d['nodeValue'] = value + d['ownerDocument'] = self.document + _set_attribute_node(node, a) + + if node is not self.document.documentElement: + self._finish_start_element(node) + + def _finish_start_element(self, node): + if self._filter: + # To be general, we'd have to call isSameNode(), but this + # is sufficient for minidom: + if node is self.document.documentElement: + return + filt = self._filter.startContainer(node) + if filt == FILTER_REJECT: + # ignore this node & all descendents + Rejecter(self) + elif filt == FILTER_SKIP: + # ignore this node, but make it's children become + # children of the parent node + Skipper(self) + else: + return + self.curNode = node.parentNode + node.parentNode.removeChild(node) + node.unlink() + + # If this ever changes, Namespaces.end_element_handler() needs to + # be changed to match. + # + def end_element_handler(self, name): + curNode = self.curNode + self.curNode = curNode.parentNode + self._finish_end_element(curNode) + + def _finish_end_element(self, curNode): + info = self._elem_info.get(curNode.tagName) + if info: + self._handle_white_text_nodes(curNode, info) + if self._filter: + if curNode is self.document.documentElement: + return + if self._filter.acceptNode(curNode) == FILTER_REJECT: + self.curNode.removeChild(curNode) + curNode.unlink() + + def _handle_white_text_nodes(self, node, info): + if (self._options.whitespace_in_element_content + or not info.isElementContent()): + return + + # We have element type information and should remove ignorable + # whitespace; identify for text nodes which contain only + # whitespace. + L = [] + for child in node.childNodes: + if child.nodeType == TEXT_NODE and not child.data.strip(): + L.append(child) + + # Remove ignorable whitespace from the tree. + for child in L: + node.removeChild(child) + + def element_decl_handler(self, name, model): + info = self._elem_info.get(name) + if info is None: + self._elem_info[name] = ElementInfo(name, model) + else: + assert info._model is None + info._model = model + + def attlist_decl_handler(self, elem, name, type, default, required): + info = self._elem_info.get(elem) + if info is None: + info = ElementInfo(elem) + self._elem_info[elem] = info + info._attr_info.append( + [None, name, None, None, default, 0, type, required]) + + def xml_decl_handler(self, version, encoding, standalone): + self.document.version = version + self.document.encoding = encoding + # This is still a little ugly, thanks to the pyexpat API. ;-( + if standalone >= 0: + if standalone: + self.document.standalone = True + else: + self.document.standalone = False + + +# Don't include FILTER_INTERRUPT, since that's checked separately +# where allowed. +_ALLOWED_FILTER_RETURNS = (FILTER_ACCEPT, FILTER_REJECT, FILTER_SKIP) + +class FilterVisibilityController(object): + """Wrapper around a DOMBuilderFilter which implements the checks + to make the whatToShow filter attribute work.""" + + __slots__ = 'filter', + + def __init__(self, filter): + self.filter = filter + + def startContainer(self, node): + mask = self._nodetype_mask[node.nodeType] + if self.filter.whatToShow & mask: + val = self.filter.startContainer(node) + if val == FILTER_INTERRUPT: + raise ParseEscape + if val not in _ALLOWED_FILTER_RETURNS: + raise ValueError, \ + "startContainer() returned illegal value: " + repr(val) + return val + else: + return FILTER_ACCEPT + + def acceptNode(self, node): + mask = self._nodetype_mask[node.nodeType] + if self.filter.whatToShow & mask: + val = self.filter.acceptNode(node) + if val == FILTER_INTERRUPT: + raise ParseEscape + if val == FILTER_SKIP: + # move all child nodes to the parent, and remove this node + parent = node.parentNode + for child in node.childNodes[:]: + parent.appendChild(child) + # node is handled by the caller + return FILTER_REJECT + if val not in _ALLOWED_FILTER_RETURNS: + raise ValueError, \ + "acceptNode() returned illegal value: " + repr(val) + return val + else: + return FILTER_ACCEPT + + _nodetype_mask = { + Node.ELEMENT_NODE: NodeFilter.SHOW_ELEMENT, + Node.ATTRIBUTE_NODE: NodeFilter.SHOW_ATTRIBUTE, + Node.TEXT_NODE: NodeFilter.SHOW_TEXT, + Node.CDATA_SECTION_NODE: NodeFilter.SHOW_CDATA_SECTION, + Node.ENTITY_REFERENCE_NODE: NodeFilter.SHOW_ENTITY_REFERENCE, + Node.ENTITY_NODE: NodeFilter.SHOW_ENTITY, + Node.PROCESSING_INSTRUCTION_NODE: NodeFilter.SHOW_PROCESSING_INSTRUCTION, + Node.COMMENT_NODE: NodeFilter.SHOW_COMMENT, + Node.DOCUMENT_NODE: NodeFilter.SHOW_DOCUMENT, + Node.DOCUMENT_TYPE_NODE: NodeFilter.SHOW_DOCUMENT_TYPE, + Node.DOCUMENT_FRAGMENT_NODE: NodeFilter.SHOW_DOCUMENT_FRAGMENT, + Node.NOTATION_NODE: NodeFilter.SHOW_NOTATION, + } + + +class FilterCrutch(object): + __slots__ = '_builder', '_level', '_old_start', '_old_end' + + def __init__(self, builder): + self._level = 0 + self._builder = builder + parser = builder._parser + self._old_start = parser.StartElementHandler + self._old_end = parser.EndElementHandler + parser.StartElementHandler = self.start_element_handler + parser.EndElementHandler = self.end_element_handler + +class Rejecter(FilterCrutch): + __slots__ = () + + def __init__(self, builder): + FilterCrutch.__init__(self, builder) + parser = builder._parser + for name in ("ProcessingInstructionHandler", + "CommentHandler", + "CharacterDataHandler", + "StartCdataSectionHandler", + "EndCdataSectionHandler", + "ExternalEntityRefHandler", + ): + setattr(parser, name, None) + + def start_element_handler(self, *args): + self._level = self._level + 1 + + def end_element_handler(self, *args): + if self._level == 0: + # restore the old handlers + parser = self._builder._parser + self._builder.install(parser) + parser.StartElementHandler = self._old_start + parser.EndElementHandler = self._old_end + else: + self._level = self._level - 1 + +class Skipper(FilterCrutch): + __slots__ = () + + def start_element_handler(self, *args): + node = self._builder.curNode + self._old_start(*args) + if self._builder.curNode is not node: + self._level = self._level + 1 + + def end_element_handler(self, *args): + if self._level == 0: + # We're popping back out of the node we're skipping, so we + # shouldn't need to do anything but reset the handlers. + self._builder._parser.StartElementHandler = self._old_start + self._builder._parser.EndElementHandler = self._old_end + self._builder = None + else: + self._level = self._level - 1 + self._old_end(*args) + + +# framework document used by the fragment builder. +# Takes a string for the doctype, subset string, and namespace attrs string. + +_FRAGMENT_BUILDER_INTERNAL_SYSTEM_ID = \ + "http://xml.python.org/entities/fragment-builder/internal" + +_FRAGMENT_BUILDER_TEMPLATE = ( + '''\ + +%%s +]> +&fragment-builder-internal;''' + % _FRAGMENT_BUILDER_INTERNAL_SYSTEM_ID) + + +class FragmentBuilder(ExpatBuilder): + """Builder which constructs document fragments given XML source + text and a context node. + + The context node is expected to provide information about the + namespace declarations which are in scope at the start of the + fragment. + """ + + def __init__(self, context, options=None): + if context.nodeType == DOCUMENT_NODE: + self.originalDocument = context + self.context = context + else: + self.originalDocument = context.ownerDocument + self.context = context + ExpatBuilder.__init__(self, options) + + def reset(self): + ExpatBuilder.reset(self) + self.fragment = None + + def parseFile(self, file): + """Parse a document fragment from a file object, returning the + fragment node.""" + return self.parseString(file.read()) + + def parseString(self, string): + """Parse a document fragment from a string, returning the + fragment node.""" + self._source = string + parser = self.getParser() + doctype = self.originalDocument.doctype + ident = "" + if doctype: + subset = doctype.internalSubset or self._getDeclarations() + if doctype.publicId: + ident = ('PUBLIC "%s" "%s"' + % (doctype.publicId, doctype.systemId)) + elif doctype.systemId: + ident = 'SYSTEM "%s"' % doctype.systemId + else: + subset = "" + nsattrs = self._getNSattrs() # get ns decls from node's ancestors + document = _FRAGMENT_BUILDER_TEMPLATE % (ident, subset, nsattrs) + try: + parser.Parse(document, 1) + except: + self.reset() + raise + fragment = self.fragment + self.reset() +## self._parser = None + return fragment + + def _getDeclarations(self): + """Re-create the internal subset from the DocumentType node. + + This is only needed if we don't already have the + internalSubset as a string. + """ + doctype = self.context.ownerDocument.doctype + s = "" + if doctype: + for i in range(doctype.notations.length): + notation = doctype.notations.item(i) + if s: + s = s + "\n " + s = "%s' \ + % (s, notation.publicId, notation.systemId) + else: + s = '%s SYSTEM "%s">' % (s, notation.systemId) + for i in range(doctype.entities.length): + entity = doctype.entities.item(i) + if s: + s = s + "\n " + s = "%s" + return s + + def _getNSattrs(self): + return "" + + def external_entity_ref_handler(self, context, base, systemId, publicId): + if systemId == _FRAGMENT_BUILDER_INTERNAL_SYSTEM_ID: + # this entref is the one that we made to put the subtree + # in; all of our given input is parsed in here. + old_document = self.document + old_cur_node = self.curNode + parser = self._parser.ExternalEntityParserCreate(context) + # put the real document back, parse into the fragment to return + self.document = self.originalDocument + self.fragment = self.document.createDocumentFragment() + self.curNode = self.fragment + try: + parser.Parse(self._source, 1) + finally: + self.curNode = old_cur_node + self.document = old_document + self._source = None + return -1 + else: + return ExpatBuilder.external_entity_ref_handler( + self, context, base, systemId, publicId) + + +class Namespaces: + """Mix-in class for builders; adds support for namespaces.""" + + def _initNamespaces(self): + # list of (prefix, uri) ns declarations. Namespace attrs are + # constructed from this and added to the element's attrs. + self._ns_ordered_prefixes = [] + + def createParser(self): + """Create a new namespace-handling parser.""" + parser = expat.ParserCreate(namespace_separator=" ") + parser.namespace_prefixes = True + return parser + + def install(self, parser): + """Insert the namespace-handlers onto the parser.""" + ExpatBuilder.install(self, parser) + if self._options.namespace_declarations: + parser.StartNamespaceDeclHandler = ( + self.start_namespace_decl_handler) + + def start_namespace_decl_handler(self, prefix, uri): + """Push this namespace declaration on our storage.""" + self._ns_ordered_prefixes.append((prefix, uri)) + + def start_element_handler(self, name, attributes): + if ' ' in name: + uri, localname, prefix, qname = _parse_ns_name(self, name) + else: + uri = EMPTY_NAMESPACE + qname = name + localname = None + prefix = EMPTY_PREFIX + node = minidom.Element(qname, uri, prefix, localname) + node.ownerDocument = self.document + _append_child(self.curNode, node) + self.curNode = node + + if self._ns_ordered_prefixes: + for prefix, uri in self._ns_ordered_prefixes: + if prefix: + a = minidom.Attr(_intern(self, 'xmlns:' + prefix), + XMLNS_NAMESPACE, prefix, "xmlns") + else: + a = minidom.Attr("xmlns", XMLNS_NAMESPACE, + "xmlns", EMPTY_PREFIX) + d = a.childNodes[0].__dict__ + d['data'] = d['nodeValue'] = uri + d = a.__dict__ + d['value'] = d['nodeValue'] = uri + d['ownerDocument'] = self.document + _set_attribute_node(node, a) + del self._ns_ordered_prefixes[:] + + if attributes: + _attrs = node._attrs + _attrsNS = node._attrsNS + for i in range(0, len(attributes), 2): + aname = attributes[i] + value = attributes[i+1] + if ' ' in aname: + uri, localname, prefix, qname = _parse_ns_name(self, aname) + a = minidom.Attr(qname, uri, localname, prefix) + _attrs[qname] = a + _attrsNS[(uri, localname)] = a + else: + a = minidom.Attr(aname, EMPTY_NAMESPACE, + aname, EMPTY_PREFIX) + _attrs[aname] = a + _attrsNS[(EMPTY_NAMESPACE, aname)] = a + d = a.childNodes[0].__dict__ + d['data'] = d['nodeValue'] = value + d = a.__dict__ + d['ownerDocument'] = self.document + d['value'] = d['nodeValue'] = value + d['ownerElement'] = node + + if __debug__: + # This only adds some asserts to the original + # end_element_handler(), so we only define this when -O is not + # used. If changing one, be sure to check the other to see if + # it needs to be changed as well. + # + def end_element_handler(self, name): + curNode = self.curNode + if ' ' in name: + uri, localname, prefix, qname = _parse_ns_name(self, name) + assert (curNode.namespaceURI == uri + and curNode.localName == localname + and curNode.prefix == prefix), \ + "element stack messed up! (namespace)" + else: + assert curNode.nodeName == name, \ + "element stack messed up - bad nodeName" + assert curNode.namespaceURI == EMPTY_NAMESPACE, \ + "element stack messed up - bad namespaceURI" + self.curNode = curNode.parentNode + self._finish_end_element(curNode) + + +class ExpatBuilderNS(Namespaces, ExpatBuilder): + """Document builder that supports namespaces.""" + + def reset(self): + ExpatBuilder.reset(self) + self._initNamespaces() + + +class FragmentBuilderNS(Namespaces, FragmentBuilder): + """Fragment builder that supports namespaces.""" + + def reset(self): + FragmentBuilder.reset(self) + self._initNamespaces() + + def _getNSattrs(self): + """Return string of namespace attributes from this element and + ancestors.""" + # XXX This needs to be re-written to walk the ancestors of the + # context to build up the namespace information from + # declarations, elements, and attributes found in context. + # Otherwise we have to store a bunch more data on the DOM + # (though that *might* be more reliable -- not clear). + attrs = "" + context = self.context + L = [] + while context: + if hasattr(context, '_ns_prefix_uri'): + for prefix, uri in context._ns_prefix_uri.items(): + # add every new NS decl from context to L and attrs string + if prefix in L: + continue + L.append(prefix) + if prefix: + declname = "xmlns:" + prefix + else: + declname = "xmlns" + if attrs: + attrs = "%s\n %s='%s'" % (attrs, declname, uri) + else: + attrs = " %s='%s'" % (declname, uri) + context = context.parentNode + return attrs + + +class ParseEscape(Exception): + """Exception raised to short-circuit parsing in InternalSubsetExtractor.""" + pass + +class InternalSubsetExtractor(ExpatBuilder): + """XML processor which can rip out the internal document type subset.""" + + subset = None + + def getSubset(self): + """Return the internal subset as a string.""" + return self.subset + + def parseFile(self, file): + try: + ExpatBuilder.parseFile(self, file) + except ParseEscape: + pass + + def parseString(self, string): + try: + ExpatBuilder.parseString(self, string) + except ParseEscape: + pass + + def install(self, parser): + parser.StartDoctypeDeclHandler = self.start_doctype_decl_handler + parser.StartElementHandler = self.start_element_handler + + def start_doctype_decl_handler(self, name, publicId, systemId, + has_internal_subset): + if has_internal_subset: + parser = self.getParser() + self.subset = [] + parser.DefaultHandler = self.subset.append + parser.EndDoctypeDeclHandler = self.end_doctype_decl_handler + else: + raise ParseEscape() + + def end_doctype_decl_handler(self): + s = ''.join(self.subset).replace('\r\n', '\n').replace('\r', '\n') + self.subset = s + raise ParseEscape() + + def start_element_handler(self, name, attrs): + raise ParseEscape() + + +def parse(file, namespaces=True): + """Parse a document, returning the resulting Document node. + + 'file' may be either a file name or an open file object. + """ + if namespaces: + builder = ExpatBuilderNS() + else: + builder = ExpatBuilder() + + if isinstance(file, StringTypes): + fp = open(file, 'rb') + try: + result = builder.parseFile(fp) + finally: + fp.close() + else: + result = builder.parseFile(file) + return result + + +def parseString(string, namespaces=True): + """Parse a document from a string, returning the resulting + Document node. + """ + if namespaces: + builder = ExpatBuilderNS() + else: + builder = ExpatBuilder() + return builder.parseString(string) + + +def parseFragment(file, context, namespaces=True): + """Parse a fragment of a document, given the context from which it + was originally extracted. context should be the parent of the + node(s) which are in the fragment. + + 'file' may be either a file name or an open file object. + """ + if namespaces: + builder = FragmentBuilderNS(context) + else: + builder = FragmentBuilder(context) + + if isinstance(file, StringTypes): + fp = open(file, 'rb') + try: + result = builder.parseFile(fp) + finally: + fp.close() + else: + result = builder.parseFile(file) + return result + + +def parseFragmentString(string, context, namespaces=True): + """Parse a fragment of a document from a string, given the context + from which it was originally extracted. context should be the + parent of the node(s) which are in the fragment. + """ + if namespaces: + builder = FragmentBuilderNS(context) + else: + builder = FragmentBuilder(context) + return builder.parseString(string) + + +def makeBuilder(options): + """Create a builder based on an Options object.""" + if options.namespaces: + return ExpatBuilderNS(options) + else: + return ExpatBuilder(options) diff --git a/Lib/xml/dom/minicompat.py b/Lib/xml/dom/minicompat.py new file mode 100644 index 0000000..d491fb6 --- /dev/null +++ b/Lib/xml/dom/minicompat.py @@ -0,0 +1,110 @@ +"""Python version compatibility support for minidom.""" + +# This module should only be imported using "import *". +# +# The following names are defined: +# +# NodeList -- lightest possible NodeList implementation +# +# EmptyNodeList -- lightest possible NodeList that is guarateed to +# remain empty (immutable) +# +# StringTypes -- tuple of defined string types +# +# defproperty -- function used in conjunction with GetattrMagic; +# using these together is needed to make them work +# as efficiently as possible in both Python 2.2+ +# and older versions. For example: +# +# class MyClass(GetattrMagic): +# def _get_myattr(self): +# return something +# +# defproperty(MyClass, "myattr", +# "return some value") +# +# For Python 2.2 and newer, this will construct a +# property object on the class, which avoids +# needing to override __getattr__(). It will only +# work for read-only attributes. +# +# For older versions of Python, inheriting from +# GetattrMagic will use the traditional +# __getattr__() hackery to achieve the same effect, +# but less efficiently. +# +# defproperty() should be used for each version of +# the relevant _get_() function. + +__all__ = ["NodeList", "EmptyNodeList", "StringTypes", "defproperty"] + +import xml.dom + +try: + unicode +except NameError: + StringTypes = type(''), +else: + StringTypes = type(''), type(unicode('')) + + +class NodeList(list): + __slots__ = () + + def item(self, index): + if 0 <= index < len(self): + return self[index] + + def _get_length(self): + return len(self) + + def _set_length(self, value): + raise xml.dom.NoModificationAllowedErr( + "attempt to modify read-only attribute 'length'") + + length = property(_get_length, _set_length, + doc="The number of nodes in the NodeList.") + + def __getstate__(self): + return list(self) + + def __setstate__(self, state): + self[:] = state + + +class EmptyNodeList(tuple): + __slots__ = () + + def __add__(self, other): + NL = NodeList() + NL.extend(other) + return NL + + def __radd__(self, other): + NL = NodeList() + NL.extend(other) + return NL + + def item(self, index): + return None + + def _get_length(self): + return 0 + + def _set_length(self, value): + raise xml.dom.NoModificationAllowedErr( + "attempt to modify read-only attribute 'length'") + + length = property(_get_length, _set_length, + doc="The number of nodes in the NodeList.") + + +def defproperty(klass, name, doc): + get = getattr(klass, ("_get_" + name)).im_func + def set(self, value, name=name): + raise xml.dom.NoModificationAllowedErr( + "attempt to modify read-only attribute " + repr(name)) + assert not hasattr(klass, "_set_" + name), \ + "expected not to find _set_" + name + prop = property(get, set, doc=doc) + setattr(klass, name, prop) diff --git a/Lib/xml/dom/minidom.py b/Lib/xml/dom/minidom.py new file mode 100644 index 0000000..3a35781 --- /dev/null +++ b/Lib/xml/dom/minidom.py @@ -0,0 +1,1936 @@ +"""\ +minidom.py -- a lightweight DOM implementation. + +parse("foo.xml") + +parseString("") + +Todo: +===== + * convenience methods for getting elements and text. + * more testing + * bring some of the writer and linearizer code into conformance with this + interface + * SAX 2 namespaces +""" + +import xml.dom + +from xml.dom import EMPTY_NAMESPACE, EMPTY_PREFIX, XMLNS_NAMESPACE, domreg +from xml.dom.minicompat import * +from xml.dom.xmlbuilder import DOMImplementationLS, DocumentLS + +# This is used by the ID-cache invalidation checks; the list isn't +# actually complete, since the nodes being checked will never be the +# DOCUMENT_NODE or DOCUMENT_FRAGMENT_NODE. (The node being checked is +# the node being added or removed, not the node being modified.) +# +_nodeTypes_with_children = (xml.dom.Node.ELEMENT_NODE, + xml.dom.Node.ENTITY_REFERENCE_NODE) + + +class Node(xml.dom.Node): + namespaceURI = None # this is non-null only for elements and attributes + parentNode = None + ownerDocument = None + nextSibling = None + previousSibling = None + + prefix = EMPTY_PREFIX # non-null only for NS elements and attributes + + def __nonzero__(self): + return True + + def toxml(self, encoding = None): + return self.toprettyxml("", "", encoding) + + def toprettyxml(self, indent="\t", newl="\n", encoding = None): + # indent = the indentation string to prepend, per level + # newl = the newline string to append + writer = _get_StringIO() + if encoding is not None: + import codecs + # Can't use codecs.getwriter to preserve 2.0 compatibility + writer = codecs.lookup(encoding)[3](writer) + if self.nodeType == Node.DOCUMENT_NODE: + # Can pass encoding only to document, to put it into XML header + self.writexml(writer, "", indent, newl, encoding) + else: + self.writexml(writer, "", indent, newl) + return writer.getvalue() + + def hasChildNodes(self): + if self.childNodes: + return True + else: + return False + + def _get_childNodes(self): + return self.childNodes + + def _get_firstChild(self): + if self.childNodes: + return self.childNodes[0] + + def _get_lastChild(self): + if self.childNodes: + return self.childNodes[-1] + + def insertBefore(self, newChild, refChild): + if newChild.nodeType == self.DOCUMENT_FRAGMENT_NODE: + for c in tuple(newChild.childNodes): + self.insertBefore(c, refChild) + ### The DOM does not clearly specify what to return in this case + return newChild + if newChild.nodeType not in self._child_node_types: + raise xml.dom.HierarchyRequestErr( + "%s cannot be child of %s" % (repr(newChild), repr(self))) + if newChild.parentNode is not None: + newChild.parentNode.removeChild(newChild) + if refChild is None: + self.appendChild(newChild) + else: + try: + index = self.childNodes.index(refChild) + except ValueError: + raise xml.dom.NotFoundErr() + if newChild.nodeType in _nodeTypes_with_children: + _clear_id_cache(self) + self.childNodes.insert(index, newChild) + newChild.nextSibling = refChild + refChild.previousSibling = newChild + if index: + node = self.childNodes[index-1] + node.nextSibling = newChild + newChild.previousSibling = node + else: + newChild.previousSibling = None + newChild.parentNode = self + return newChild + + def appendChild(self, node): + if node.nodeType == self.DOCUMENT_FRAGMENT_NODE: + for c in tuple(node.childNodes): + self.appendChild(c) + ### The DOM does not clearly specify what to return in this case + return node + if node.nodeType not in self._child_node_types: + raise xml.dom.HierarchyRequestErr( + "%s cannot be child of %s" % (repr(node), repr(self))) + elif node.nodeType in _nodeTypes_with_children: + _clear_id_cache(self) + if node.parentNode is not None: + node.parentNode.removeChild(node) + _append_child(self, node) + node.nextSibling = None + return node + + def replaceChild(self, newChild, oldChild): + if newChild.nodeType == self.DOCUMENT_FRAGMENT_NODE: + refChild = oldChild.nextSibling + self.removeChild(oldChild) + return self.insertBefore(newChild, refChild) + if newChild.nodeType not in self._child_node_types: + raise xml.dom.HierarchyRequestErr( + "%s cannot be child of %s" % (repr(newChild), repr(self))) + if newChild is oldChild: + return + if newChild.parentNode is not None: + newChild.parentNode.removeChild(newChild) + try: + index = self.childNodes.index(oldChild) + except ValueError: + raise xml.dom.NotFoundErr() + self.childNodes[index] = newChild + newChild.parentNode = self + oldChild.parentNode = None + if (newChild.nodeType in _nodeTypes_with_children + or oldChild.nodeType in _nodeTypes_with_children): + _clear_id_cache(self) + newChild.nextSibling = oldChild.nextSibling + newChild.previousSibling = oldChild.previousSibling + oldChild.nextSibling = None + oldChild.previousSibling = None + if newChild.previousSibling: + newChild.previousSibling.nextSibling = newChild + if newChild.nextSibling: + newChild.nextSibling.previousSibling = newChild + return oldChild + + def removeChild(self, oldChild): + try: + self.childNodes.remove(oldChild) + except ValueError: + raise xml.dom.NotFoundErr() + if oldChild.nextSibling is not None: + oldChild.nextSibling.previousSibling = oldChild.previousSibling + if oldChild.previousSibling is not None: + oldChild.previousSibling.nextSibling = oldChild.nextSibling + oldChild.nextSibling = oldChild.previousSibling = None + if oldChild.nodeType in _nodeTypes_with_children: + _clear_id_cache(self) + + oldChild.parentNode = None + return oldChild + + def normalize(self): + L = [] + for child in self.childNodes: + if child.nodeType == Node.TEXT_NODE: + data = child.data + if data and L and L[-1].nodeType == child.nodeType: + # collapse text node + node = L[-1] + node.data = node.data + child.data + node.nextSibling = child.nextSibling + child.unlink() + elif data: + if L: + L[-1].nextSibling = child + child.previousSibling = L[-1] + else: + child.previousSibling = None + L.append(child) + else: + # empty text node; discard + child.unlink() + else: + if L: + L[-1].nextSibling = child + child.previousSibling = L[-1] + else: + child.previousSibling = None + L.append(child) + if child.nodeType == Node.ELEMENT_NODE: + child.normalize() + self.childNodes[:] = L + + def cloneNode(self, deep): + return _clone_node(self, deep, self.ownerDocument or self) + + def isSupported(self, feature, version): + return self.ownerDocument.implementation.hasFeature(feature, version) + + def _get_localName(self): + # Overridden in Element and Attr where localName can be Non-Null + return None + + # Node interfaces from Level 3 (WD 9 April 2002) + + def isSameNode(self, other): + return self is other + + def getInterface(self, feature): + if self.isSupported(feature, None): + return self + else: + return None + + # The "user data" functions use a dictionary that is only present + # if some user data has been set, so be careful not to assume it + # exists. + + def getUserData(self, key): + try: + return self._user_data[key][0] + except (AttributeError, KeyError): + return None + + def setUserData(self, key, data, handler): + old = None + try: + d = self._user_data + except AttributeError: + d = {} + self._user_data = d + if d.has_key(key): + old = d[key][0] + if data is None: + # ignore handlers passed for None + handler = None + if old is not None: + del d[key] + else: + d[key] = (data, handler) + return old + + def _call_user_data_handler(self, operation, src, dst): + if hasattr(self, "_user_data"): + for key, (data, handler) in self._user_data.items(): + if handler is not None: + handler.handle(operation, key, data, src, dst) + + # minidom-specific API: + + def unlink(self): + self.parentNode = self.ownerDocument = None + if self.childNodes: + for child in self.childNodes: + child.unlink() + self.childNodes = NodeList() + self.previousSibling = None + self.nextSibling = None + +defproperty(Node, "firstChild", doc="First child node, or None.") +defproperty(Node, "lastChild", doc="Last child node, or None.") +defproperty(Node, "localName", doc="Namespace-local name of this node.") + + +def _append_child(self, node): + # fast path with less checks; usable by DOM builders if careful + childNodes = self.childNodes + if childNodes: + last = childNodes[-1] + node.__dict__["previousSibling"] = last + last.__dict__["nextSibling"] = node + childNodes.append(node) + node.__dict__["parentNode"] = self + +def _in_document(node): + # return True iff node is part of a document tree + while node is not None: + if node.nodeType == Node.DOCUMENT_NODE: + return True + node = node.parentNode + return False + +def _write_data(writer, data): + "Writes datachars to writer." + data = data.replace("&", "&").replace("<", "<") + data = data.replace("\"", """).replace(">", ">") + writer.write(data) + +def _get_elements_by_tagName_helper(parent, name, rc): + for node in parent.childNodes: + if node.nodeType == Node.ELEMENT_NODE and \ + (name == "*" or node.tagName == name): + rc.append(node) + _get_elements_by_tagName_helper(node, name, rc) + return rc + +def _get_elements_by_tagName_ns_helper(parent, nsURI, localName, rc): + for node in parent.childNodes: + if node.nodeType == Node.ELEMENT_NODE: + if ((localName == "*" or node.localName == localName) and + (nsURI == "*" or node.namespaceURI == nsURI)): + rc.append(node) + _get_elements_by_tagName_ns_helper(node, nsURI, localName, rc) + return rc + +class DocumentFragment(Node): + nodeType = Node.DOCUMENT_FRAGMENT_NODE + nodeName = "#document-fragment" + nodeValue = None + attributes = None + parentNode = None + _child_node_types = (Node.ELEMENT_NODE, + Node.TEXT_NODE, + Node.CDATA_SECTION_NODE, + Node.ENTITY_REFERENCE_NODE, + Node.PROCESSING_INSTRUCTION_NODE, + Node.COMMENT_NODE, + Node.NOTATION_NODE) + + def __init__(self): + self.childNodes = NodeList() + + +class Attr(Node): + nodeType = Node.ATTRIBUTE_NODE + attributes = None + ownerElement = None + specified = False + _is_id = False + + _child_node_types = (Node.TEXT_NODE, Node.ENTITY_REFERENCE_NODE) + + def __init__(self, qName, namespaceURI=EMPTY_NAMESPACE, localName=None, + prefix=None): + # skip setattr for performance + d = self.__dict__ + d["nodeName"] = d["name"] = qName + d["namespaceURI"] = namespaceURI + d["prefix"] = prefix + d['childNodes'] = NodeList() + + # Add the single child node that represents the value of the attr + self.childNodes.append(Text()) + + # nodeValue and value are set elsewhere + + def _get_localName(self): + return self.nodeName.split(":", 1)[-1] + + def _get_name(self): + return self.name + + def _get_specified(self): + return self.specified + + def __setattr__(self, name, value): + d = self.__dict__ + if name in ("value", "nodeValue"): + d["value"] = d["nodeValue"] = value + d2 = self.childNodes[0].__dict__ + d2["data"] = d2["nodeValue"] = value + if self.ownerElement is not None: + _clear_id_cache(self.ownerElement) + elif name in ("name", "nodeName"): + d["name"] = d["nodeName"] = value + if self.ownerElement is not None: + _clear_id_cache(self.ownerElement) + else: + d[name] = value + + def _set_prefix(self, prefix): + nsuri = self.namespaceURI + if prefix == "xmlns": + if nsuri and nsuri != XMLNS_NAMESPACE: + raise xml.dom.NamespaceErr( + "illegal use of 'xmlns' prefix for the wrong namespace") + d = self.__dict__ + d['prefix'] = prefix + if prefix is None: + newName = self.localName + else: + newName = "%s:%s" % (prefix, self.localName) + if self.ownerElement: + _clear_id_cache(self.ownerElement) + d['nodeName'] = d['name'] = newName + + def _set_value(self, value): + d = self.__dict__ + d['value'] = d['nodeValue'] = value + if self.ownerElement: + _clear_id_cache(self.ownerElement) + self.childNodes[0].data = value + + def unlink(self): + # This implementation does not call the base implementation + # since most of that is not needed, and the expense of the + # method call is not warranted. We duplicate the removal of + # children, but that's all we needed from the base class. + elem = self.ownerElement + if elem is not None: + del elem._attrs[self.nodeName] + del elem._attrsNS[(self.namespaceURI, self.localName)] + if self._is_id: + self._is_id = False + elem._magic_id_nodes -= 1 + self.ownerDocument._magic_id_count -= 1 + for child in self.childNodes: + child.unlink() + del self.childNodes[:] + + def _get_isId(self): + if self._is_id: + return True + doc = self.ownerDocument + elem = self.ownerElement + if doc is None or elem is None: + return False + + info = doc._get_elem_info(elem) + if info is None: + return False + if self.namespaceURI: + return info.isIdNS(self.namespaceURI, self.localName) + else: + return info.isId(self.nodeName) + + def _get_schemaType(self): + doc = self.ownerDocument + elem = self.ownerElement + if doc is None or elem is None: + return _no_type + + info = doc._get_elem_info(elem) + if info is None: + return _no_type + if self.namespaceURI: + return info.getAttributeTypeNS(self.namespaceURI, self.localName) + else: + return info.getAttributeType(self.nodeName) + +defproperty(Attr, "isId", doc="True if this attribute is an ID.") +defproperty(Attr, "localName", doc="Namespace-local name of this attribute.") +defproperty(Attr, "schemaType", doc="Schema type for this attribute.") + + +class NamedNodeMap(object): + """The attribute list is a transient interface to the underlying + dictionaries. Mutations here will change the underlying element's + dictionary. + + Ordering is imposed artificially and does not reflect the order of + attributes as found in an input document. + """ + + __slots__ = ('_attrs', '_attrsNS', '_ownerElement') + + def __init__(self, attrs, attrsNS, ownerElement): + self._attrs = attrs + self._attrsNS = attrsNS + self._ownerElement = ownerElement + + def _get_length(self): + return len(self._attrs) + + def item(self, index): + try: + return self[self._attrs.keys()[index]] + except IndexError: + return None + + def items(self): + L = [] + for node in self._attrs.values(): + L.append((node.nodeName, node.value)) + return L + + def itemsNS(self): + L = [] + for node in self._attrs.values(): + L.append(((node.namespaceURI, node.localName), node.value)) + return L + + def has_key(self, key): + if isinstance(key, StringTypes): + return self._attrs.has_key(key) + else: + return self._attrsNS.has_key(key) + + def keys(self): + return self._attrs.keys() + + def keysNS(self): + return self._attrsNS.keys() + + def values(self): + return self._attrs.values() + + def get(self, name, value=None): + return self._attrs.get(name, value) + + __len__ = _get_length + + def __cmp__(self, other): + if self._attrs is getattr(other, "_attrs", None): + return 0 + else: + return cmp(id(self), id(other)) + + def __getitem__(self, attname_or_tuple): + if isinstance(attname_or_tuple, tuple): + return self._attrsNS[attname_or_tuple] + else: + return self._attrs[attname_or_tuple] + + # same as set + def __setitem__(self, attname, value): + if isinstance(value, StringTypes): + try: + node = self._attrs[attname] + except KeyError: + node = Attr(attname) + node.ownerDocument = self._ownerElement.ownerDocument + self.setNamedItem(node) + node.value = value + else: + if not isinstance(value, Attr): + raise TypeError, "value must be a string or Attr object" + node = value + self.setNamedItem(node) + + def getNamedItem(self, name): + try: + return self._attrs[name] + except KeyError: + return None + + def getNamedItemNS(self, namespaceURI, localName): + try: + return self._attrsNS[(namespaceURI, localName)] + except KeyError: + return None + + def removeNamedItem(self, name): + n = self.getNamedItem(name) + if n is not None: + _clear_id_cache(self._ownerElement) + del self._attrs[n.nodeName] + del self._attrsNS[(n.namespaceURI, n.localName)] + if n.__dict__.has_key('ownerElement'): + n.__dict__['ownerElement'] = None + return n + else: + raise xml.dom.NotFoundErr() + + def removeNamedItemNS(self, namespaceURI, localName): + n = self.getNamedItemNS(namespaceURI, localName) + if n is not None: + _clear_id_cache(self._ownerElement) + del self._attrsNS[(n.namespaceURI, n.localName)] + del self._attrs[n.nodeName] + if n.__dict__.has_key('ownerElement'): + n.__dict__['ownerElement'] = None + return n + else: + raise xml.dom.NotFoundErr() + + def setNamedItem(self, node): + if not isinstance(node, Attr): + raise xml.dom.HierarchyRequestErr( + "%s cannot be child of %s" % (repr(node), repr(self))) + old = self._attrs.get(node.name) + if old: + old.unlink() + self._attrs[node.name] = node + self._attrsNS[(node.namespaceURI, node.localName)] = node + node.ownerElement = self._ownerElement + _clear_id_cache(node.ownerElement) + return old + + def setNamedItemNS(self, node): + return self.setNamedItem(node) + + def __delitem__(self, attname_or_tuple): + node = self[attname_or_tuple] + _clear_id_cache(node.ownerElement) + node.unlink() + + def __getstate__(self): + return self._attrs, self._attrsNS, self._ownerElement + + def __setstate__(self, state): + self._attrs, self._attrsNS, self._ownerElement = state + +defproperty(NamedNodeMap, "length", + doc="Number of nodes in the NamedNodeMap.") + +AttributeList = NamedNodeMap + + +class TypeInfo(object): + __slots__ = 'namespace', 'name' + + def __init__(self, namespace, name): + self.namespace = namespace + self.name = name + + def __repr__(self): + if self.namespace: + return "" % (self.name, self.namespace) + else: + return "" % self.name + + def _get_name(self): + return self.name + + def _get_namespace(self): + return self.namespace + +_no_type = TypeInfo(None, None) + +class Element(Node): + nodeType = Node.ELEMENT_NODE + nodeValue = None + schemaType = _no_type + + _magic_id_nodes = 0 + + _child_node_types = (Node.ELEMENT_NODE, + Node.PROCESSING_INSTRUCTION_NODE, + Node.COMMENT_NODE, + Node.TEXT_NODE, + Node.CDATA_SECTION_NODE, + Node.ENTITY_REFERENCE_NODE) + + def __init__(self, tagName, namespaceURI=EMPTY_NAMESPACE, prefix=None, + localName=None): + self.tagName = self.nodeName = tagName + self.prefix = prefix + self.namespaceURI = namespaceURI + self.childNodes = NodeList() + + self._attrs = {} # attributes are double-indexed: + self._attrsNS = {} # tagName -> Attribute + # URI,localName -> Attribute + # in the future: consider lazy generation + # of attribute objects this is too tricky + # for now because of headaches with + # namespaces. + + def _get_localName(self): + return self.tagName.split(":", 1)[-1] + + def _get_tagName(self): + return self.tagName + + def unlink(self): + for attr in self._attrs.values(): + attr.unlink() + self._attrs = None + self._attrsNS = None + Node.unlink(self) + + def getAttribute(self, attname): + try: + return self._attrs[attname].value + except KeyError: + return "" + + def getAttributeNS(self, namespaceURI, localName): + try: + return self._attrsNS[(namespaceURI, localName)].value + except KeyError: + return "" + + def setAttribute(self, attname, value): + attr = self.getAttributeNode(attname) + if attr is None: + attr = Attr(attname) + # for performance + d = attr.__dict__ + d["value"] = d["nodeValue"] = value + d["ownerDocument"] = self.ownerDocument + self.setAttributeNode(attr) + elif value != attr.value: + d = attr.__dict__ + d["value"] = d["nodeValue"] = value + if attr.isId: + _clear_id_cache(self) + + def setAttributeNS(self, namespaceURI, qualifiedName, value): + prefix, localname = _nssplit(qualifiedName) + attr = self.getAttributeNodeNS(namespaceURI, localname) + if attr is None: + # for performance + attr = Attr(qualifiedName, namespaceURI, localname, prefix) + d = attr.__dict__ + d["prefix"] = prefix + d["nodeName"] = qualifiedName + d["value"] = d["nodeValue"] = value + d["ownerDocument"] = self.ownerDocument + self.setAttributeNode(attr) + else: + d = attr.__dict__ + if value != attr.value: + d["value"] = d["nodeValue"] = value + if attr.isId: + _clear_id_cache(self) + if attr.prefix != prefix: + d["prefix"] = prefix + d["nodeName"] = qualifiedName + + def getAttributeNode(self, attrname): + return self._attrs.get(attrname) + + def getAttributeNodeNS(self, namespaceURI, localName): + return self._attrsNS.get((namespaceURI, localName)) + + def setAttributeNode(self, attr): + if attr.ownerElement not in (None, self): + raise xml.dom.InuseAttributeErr("attribute node already owned") + old1 = self._attrs.get(attr.name, None) + if old1 is not None: + self.removeAttributeNode(old1) + old2 = self._attrsNS.get((attr.namespaceURI, attr.localName), None) + if old2 is not None and old2 is not old1: + self.removeAttributeNode(old2) + _set_attribute_node(self, attr) + + if old1 is not attr: + # It might have already been part of this node, in which case + # it doesn't represent a change, and should not be returned. + return old1 + if old2 is not attr: + return old2 + + setAttributeNodeNS = setAttributeNode + + def removeAttribute(self, name): + try: + attr = self._attrs[name] + except KeyError: + raise xml.dom.NotFoundErr() + self.removeAttributeNode(attr) + + def removeAttributeNS(self, namespaceURI, localName): + try: + attr = self._attrsNS[(namespaceURI, localName)] + except KeyError: + raise xml.dom.NotFoundErr() + self.removeAttributeNode(attr) + + def removeAttributeNode(self, node): + if node is None: + raise xml.dom.NotFoundErr() + try: + self._attrs[node.name] + except KeyError: + raise xml.dom.NotFoundErr() + _clear_id_cache(self) + node.unlink() + # Restore this since the node is still useful and otherwise + # unlinked + node.ownerDocument = self.ownerDocument + + removeAttributeNodeNS = removeAttributeNode + + def hasAttribute(self, name): + return self._attrs.has_key(name) + + def hasAttributeNS(self, namespaceURI, localName): + return self._attrsNS.has_key((namespaceURI, localName)) + + def getElementsByTagName(self, name): + return _get_elements_by_tagName_helper(self, name, NodeList()) + + def getElementsByTagNameNS(self, namespaceURI, localName): + return _get_elements_by_tagName_ns_helper( + self, namespaceURI, localName, NodeList()) + + def __repr__(self): + return "" % (self.tagName, id(self)) + + def writexml(self, writer, indent="", addindent="", newl=""): + # indent = current indentation + # addindent = indentation to add to higher levels + # newl = newline string + writer.write(indent+"<" + self.tagName) + + attrs = self._get_attributes() + a_names = attrs.keys() + a_names.sort() + + for a_name in a_names: + writer.write(" %s=\"" % a_name) + _write_data(writer, attrs[a_name].value) + writer.write("\"") + if self.childNodes: + writer.write(">%s"%(newl)) + for node in self.childNodes: + node.writexml(writer,indent+addindent,addindent,newl) + writer.write("%s%s" % (indent,self.tagName,newl)) + else: + writer.write("/>%s"%(newl)) + + def _get_attributes(self): + return NamedNodeMap(self._attrs, self._attrsNS, self) + + def hasAttributes(self): + if self._attrs: + return True + else: + return False + + # DOM Level 3 attributes, based on the 22 Oct 2002 draft + + def setIdAttribute(self, name): + idAttr = self.getAttributeNode(name) + self.setIdAttributeNode(idAttr) + + def setIdAttributeNS(self, namespaceURI, localName): + idAttr = self.getAttributeNodeNS(namespaceURI, localName) + self.setIdAttributeNode(idAttr) + + def setIdAttributeNode(self, idAttr): + if idAttr is None or not self.isSameNode(idAttr.ownerElement): + raise xml.dom.NotFoundErr() + if _get_containing_entref(self) is not None: + raise xml.dom.NoModificationAllowedErr() + if not idAttr._is_id: + idAttr.__dict__['_is_id'] = True + self._magic_id_nodes += 1 + self.ownerDocument._magic_id_count += 1 + _clear_id_cache(self) + +defproperty(Element, "attributes", + doc="NamedNodeMap of attributes on the element.") +defproperty(Element, "localName", + doc="Namespace-local name of this element.") + + +def _set_attribute_node(element, attr): + _clear_id_cache(element) + element._attrs[attr.name] = attr + element._attrsNS[(attr.namespaceURI, attr.localName)] = attr + + # This creates a circular reference, but Element.unlink() + # breaks the cycle since the references to the attribute + # dictionaries are tossed. + attr.__dict__['ownerElement'] = element + + +class Childless: + """Mixin that makes childless-ness easy to implement and avoids + the complexity of the Node methods that deal with children. + """ + + attributes = None + childNodes = EmptyNodeList() + firstChild = None + lastChild = None + + def _get_firstChild(self): + return None + + def _get_lastChild(self): + return None + + def appendChild(self, node): + raise xml.dom.HierarchyRequestErr( + self.nodeName + " nodes cannot have children") + + def hasChildNodes(self): + return False + + def insertBefore(self, newChild, refChild): + raise xml.dom.HierarchyRequestErr( + self.nodeName + " nodes do not have children") + + def removeChild(self, oldChild): + raise xml.dom.NotFoundErr( + self.nodeName + " nodes do not have children") + + def replaceChild(self, newChild, oldChild): + raise xml.dom.HierarchyRequestErr( + self.nodeName + " nodes do not have children") + + +class ProcessingInstruction(Childless, Node): + nodeType = Node.PROCESSING_INSTRUCTION_NODE + + def __init__(self, target, data): + self.target = self.nodeName = target + self.data = self.nodeValue = data + + def _get_data(self): + return self.data + def _set_data(self, value): + d = self.__dict__ + d['data'] = d['nodeValue'] = value + + def _get_target(self): + return self.target + def _set_target(self, value): + d = self.__dict__ + d['target'] = d['nodeName'] = value + + def __setattr__(self, name, value): + if name == "data" or name == "nodeValue": + self.__dict__['data'] = self.__dict__['nodeValue'] = value + elif name == "target" or name == "nodeName": + self.__dict__['target'] = self.__dict__['nodeName'] = value + else: + self.__dict__[name] = value + + def writexml(self, writer, indent="", addindent="", newl=""): + writer.write("%s%s" % (indent,self.target, self.data, newl)) + + +class CharacterData(Childless, Node): + def _get_length(self): + return len(self.data) + __len__ = _get_length + + def _get_data(self): + return self.__dict__['data'] + def _set_data(self, data): + d = self.__dict__ + d['data'] = d['nodeValue'] = data + + _get_nodeValue = _get_data + _set_nodeValue = _set_data + + def __setattr__(self, name, value): + if name == "data" or name == "nodeValue": + self.__dict__['data'] = self.__dict__['nodeValue'] = value + else: + self.__dict__[name] = value + + def __repr__(self): + data = self.data + if len(data) > 10: + dotdotdot = "..." + else: + dotdotdot = "" + return "" % ( + self.__class__.__name__, data[0:10], dotdotdot) + + def substringData(self, offset, count): + if offset < 0: + raise xml.dom.IndexSizeErr("offset cannot be negative") + if offset >= len(self.data): + raise xml.dom.IndexSizeErr("offset cannot be beyond end of data") + if count < 0: + raise xml.dom.IndexSizeErr("count cannot be negative") + return self.data[offset:offset+count] + + def appendData(self, arg): + self.data = self.data + arg + + def insertData(self, offset, arg): + if offset < 0: + raise xml.dom.IndexSizeErr("offset cannot be negative") + if offset >= len(self.data): + raise xml.dom.IndexSizeErr("offset cannot be beyond end of data") + if arg: + self.data = "%s%s%s" % ( + self.data[:offset], arg, self.data[offset:]) + + def deleteData(self, offset, count): + if offset < 0: + raise xml.dom.IndexSizeErr("offset cannot be negative") + if offset >= len(self.data): + raise xml.dom.IndexSizeErr("offset cannot be beyond end of data") + if count < 0: + raise xml.dom.IndexSizeErr("count cannot be negative") + if count: + self.data = self.data[:offset] + self.data[offset+count:] + + def replaceData(self, offset, count, arg): + if offset < 0: + raise xml.dom.IndexSizeErr("offset cannot be negative") + if offset >= len(self.data): + raise xml.dom.IndexSizeErr("offset cannot be beyond end of data") + if count < 0: + raise xml.dom.IndexSizeErr("count cannot be negative") + if count: + self.data = "%s%s%s" % ( + self.data[:offset], arg, self.data[offset+count:]) + +defproperty(CharacterData, "length", doc="Length of the string data.") + + +class Text(CharacterData): + # Make sure we don't add an instance __dict__ if we don't already + # have one, at least when that's possible: + # XXX this does not work, CharacterData is an old-style class + # __slots__ = () + + nodeType = Node.TEXT_NODE + nodeName = "#text" + attributes = None + + def splitText(self, offset): + if offset < 0 or offset > len(self.data): + raise xml.dom.IndexSizeErr("illegal offset value") + newText = self.__class__() + newText.data = self.data[offset:] + newText.ownerDocument = self.ownerDocument + next = self.nextSibling + if self.parentNode and self in self.parentNode.childNodes: + if next is None: + self.parentNode.appendChild(newText) + else: + self.parentNode.insertBefore(newText, next) + self.data = self.data[:offset] + return newText + + def writexml(self, writer, indent="", addindent="", newl=""): + _write_data(writer, "%s%s%s"%(indent, self.data, newl)) + + # DOM Level 3 (WD 9 April 2002) + + def _get_wholeText(self): + L = [self.data] + n = self.previousSibling + while n is not None: + if n.nodeType in (Node.TEXT_NODE, Node.CDATA_SECTION_NODE): + L.insert(0, n.data) + n = n.previousSibling + else: + break + n = self.nextSibling + while n is not None: + if n.nodeType in (Node.TEXT_NODE, Node.CDATA_SECTION_NODE): + L.append(n.data) + n = n.nextSibling + else: + break + return ''.join(L) + + def replaceWholeText(self, content): + # XXX This needs to be seriously changed if minidom ever + # supports EntityReference nodes. + parent = self.parentNode + n = self.previousSibling + while n is not None: + if n.nodeType in (Node.TEXT_NODE, Node.CDATA_SECTION_NODE): + next = n.previousSibling + parent.removeChild(n) + n = next + else: + break + n = self.nextSibling + if not content: + parent.removeChild(self) + while n is not None: + if n.nodeType in (Node.TEXT_NODE, Node.CDATA_SECTION_NODE): + next = n.nextSibling + parent.removeChild(n) + n = next + else: + break + if content: + d = self.__dict__ + d['data'] = content + d['nodeValue'] = content + return self + else: + return None + + def _get_isWhitespaceInElementContent(self): + if self.data.strip(): + return False + elem = _get_containing_element(self) + if elem is None: + return False + info = self.ownerDocument._get_elem_info(elem) + if info is None: + return False + else: + return info.isElementContent() + +defproperty(Text, "isWhitespaceInElementContent", + doc="True iff this text node contains only whitespace" + " and is in element content.") +defproperty(Text, "wholeText", + doc="The text of all logically-adjacent text nodes.") + + +def _get_containing_element(node): + c = node.parentNode + while c is not None: + if c.nodeType == Node.ELEMENT_NODE: + return c + c = c.parentNode + return None + +def _get_containing_entref(node): + c = node.parentNode + while c is not None: + if c.nodeType == Node.ENTITY_REFERENCE_NODE: + return c + c = c.parentNode + return None + + +class Comment(Childless, CharacterData): + nodeType = Node.COMMENT_NODE + nodeName = "#comment" + + def __init__(self, data): + self.data = self.nodeValue = data + + def writexml(self, writer, indent="", addindent="", newl=""): + writer.write("%s%s" % (indent, self.data, newl)) + + +class CDATASection(Text): + # Make sure we don't add an instance __dict__ if we don't already + # have one, at least when that's possible: + # XXX this does not work, Text is an old-style class + # __slots__ = () + + nodeType = Node.CDATA_SECTION_NODE + nodeName = "#cdata-section" + + def writexml(self, writer, indent="", addindent="", newl=""): + if self.data.find("]]>") >= 0: + raise ValueError("']]>' not allowed in a CDATA section") + writer.write("" % self.data) + + +class ReadOnlySequentialNamedNodeMap(object): + __slots__ = '_seq', + + def __init__(self, seq=()): + # seq should be a list or tuple + self._seq = seq + + def __len__(self): + return len(self._seq) + + def _get_length(self): + return len(self._seq) + + def getNamedItem(self, name): + for n in self._seq: + if n.nodeName == name: + return n + + def getNamedItemNS(self, namespaceURI, localName): + for n in self._seq: + if n.namespaceURI == namespaceURI and n.localName == localName: + return n + + def __getitem__(self, name_or_tuple): + if isinstance(name_or_tuple, tuple): + node = self.getNamedItemNS(*name_or_tuple) + else: + node = self.getNamedItem(name_or_tuple) + if node is None: + raise KeyError, name_or_tuple + return node + + def item(self, index): + if index < 0: + return None + try: + return self._seq[index] + except IndexError: + return None + + def removeNamedItem(self, name): + raise xml.dom.NoModificationAllowedErr( + "NamedNodeMap instance is read-only") + + def removeNamedItemNS(self, namespaceURI, localName): + raise xml.dom.NoModificationAllowedErr( + "NamedNodeMap instance is read-only") + + def setNamedItem(self, node): + raise xml.dom.NoModificationAllowedErr( + "NamedNodeMap instance is read-only") + + def setNamedItemNS(self, node): + raise xml.dom.NoModificationAllowedErr( + "NamedNodeMap instance is read-only") + + def __getstate__(self): + return [self._seq] + + def __setstate__(self, state): + self._seq = state[0] + +defproperty(ReadOnlySequentialNamedNodeMap, "length", + doc="Number of entries in the NamedNodeMap.") + + +class Identified: + """Mix-in class that supports the publicId and systemId attributes.""" + + # XXX this does not work, this is an old-style class + # __slots__ = 'publicId', 'systemId' + + def _identified_mixin_init(self, publicId, systemId): + self.publicId = publicId + self.systemId = systemId + + def _get_publicId(self): + return self.publicId + + def _get_systemId(self): + return self.systemId + +class DocumentType(Identified, Childless, Node): + nodeType = Node.DOCUMENT_TYPE_NODE + nodeValue = None + name = None + publicId = None + systemId = None + internalSubset = None + + def __init__(self, qualifiedName): + self.entities = ReadOnlySequentialNamedNodeMap() + self.notations = ReadOnlySequentialNamedNodeMap() + if qualifiedName: + prefix, localname = _nssplit(qualifiedName) + self.name = localname + self.nodeName = self.name + + def _get_internalSubset(self): + return self.internalSubset + + def cloneNode(self, deep): + if self.ownerDocument is None: + # it's ok + clone = DocumentType(None) + clone.name = self.name + clone.nodeName = self.name + operation = xml.dom.UserDataHandler.NODE_CLONED + if deep: + clone.entities._seq = [] + clone.notations._seq = [] + for n in self.notations._seq: + notation = Notation(n.nodeName, n.publicId, n.systemId) + clone.notations._seq.append(notation) + n._call_user_data_handler(operation, n, notation) + for e in self.entities._seq: + entity = Entity(e.nodeName, e.publicId, e.systemId, + e.notationName) + entity.actualEncoding = e.actualEncoding + entity.encoding = e.encoding + entity.version = e.version + clone.entities._seq.append(entity) + e._call_user_data_handler(operation, n, entity) + self._call_user_data_handler(operation, self, clone) + return clone + else: + return None + + def writexml(self, writer, indent="", addindent="", newl=""): + writer.write(""+newl) + +class Entity(Identified, Node): + attributes = None + nodeType = Node.ENTITY_NODE + nodeValue = None + + actualEncoding = None + encoding = None + version = None + + def __init__(self, name, publicId, systemId, notation): + self.nodeName = name + self.notationName = notation + self.childNodes = NodeList() + self._identified_mixin_init(publicId, systemId) + + def _get_actualEncoding(self): + return self.actualEncoding + + def _get_encoding(self): + return self.encoding + + def _get_version(self): + return self.version + + def appendChild(self, newChild): + raise xml.dom.HierarchyRequestErr( + "cannot append children to an entity node") + + def insertBefore(self, newChild, refChild): + raise xml.dom.HierarchyRequestErr( + "cannot insert children below an entity node") + + def removeChild(self, oldChild): + raise xml.dom.HierarchyRequestErr( + "cannot remove children from an entity node") + + def replaceChild(self, newChild, oldChild): + raise xml.dom.HierarchyRequestErr( + "cannot replace children of an entity node") + +class Notation(Identified, Childless, Node): + nodeType = Node.NOTATION_NODE + nodeValue = None + + def __init__(self, name, publicId, systemId): + self.nodeName = name + self._identified_mixin_init(publicId, systemId) + + +class DOMImplementation(DOMImplementationLS): + _features = [("core", "1.0"), + ("core", "2.0"), + ("core", "3.0"), + ("core", None), + ("xml", "1.0"), + ("xml", "2.0"), + ("xml", "3.0"), + ("xml", None), + ("ls-load", "3.0"), + ("ls-load", None), + ] + + def hasFeature(self, feature, version): + if version == "": + version = None + return (feature.lower(), version) in self._features + + def createDocument(self, namespaceURI, qualifiedName, doctype): + if doctype and doctype.parentNode is not None: + raise xml.dom.WrongDocumentErr( + "doctype object owned by another DOM tree") + doc = self._create_document() + + add_root_element = not (namespaceURI is None + and qualifiedName is None + and doctype is None) + + if not qualifiedName and add_root_element: + # The spec is unclear what to raise here; SyntaxErr + # would be the other obvious candidate. Since Xerces raises + # InvalidCharacterErr, and since SyntaxErr is not listed + # for createDocument, that seems to be the better choice. + # XXX: need to check for illegal characters here and in + # createElement. + + # DOM Level III clears this up when talking about the return value + # of this function. If namespaceURI, qName and DocType are + # Null the document is returned without a document element + # Otherwise if doctype or namespaceURI are not None + # Then we go back to the above problem + raise xml.dom.InvalidCharacterErr("Element with no name") + + if add_root_element: + prefix, localname = _nssplit(qualifiedName) + if prefix == "xml" \ + and namespaceURI != "http://www.w3.org/XML/1998/namespace": + raise xml.dom.NamespaceErr("illegal use of 'xml' prefix") + if prefix and not namespaceURI: + raise xml.dom.NamespaceErr( + "illegal use of prefix without namespaces") + element = doc.createElementNS(namespaceURI, qualifiedName) + if doctype: + doc.appendChild(doctype) + doc.appendChild(element) + + if doctype: + doctype.parentNode = doctype.ownerDocument = doc + + doc.doctype = doctype + doc.implementation = self + return doc + + def createDocumentType(self, qualifiedName, publicId, systemId): + doctype = DocumentType(qualifiedName) + doctype.publicId = publicId + doctype.systemId = systemId + return doctype + + # DOM Level 3 (WD 9 April 2002) + + def getInterface(self, feature): + if self.hasFeature(feature, None): + return self + else: + return None + + # internal + def _create_document(self): + return Document() + +class ElementInfo(object): + """Object that represents content-model information for an element. + + This implementation is not expected to be used in practice; DOM + builders should provide implementations which do the right thing + using information available to it. + + """ + + __slots__ = 'tagName', + + def __init__(self, name): + self.tagName = name + + def getAttributeType(self, aname): + return _no_type + + def getAttributeTypeNS(self, namespaceURI, localName): + return _no_type + + def isElementContent(self): + return False + + def isEmpty(self): + """Returns true iff this element is declared to have an EMPTY + content model.""" + return False + + def isId(self, aname): + """Returns true iff the named attribte is a DTD-style ID.""" + return False + + def isIdNS(self, namespaceURI, localName): + """Returns true iff the identified attribute is a DTD-style ID.""" + return False + + def __getstate__(self): + return self.tagName + + def __setstate__(self, state): + self.tagName = state + +def _clear_id_cache(node): + if node.nodeType == Node.DOCUMENT_NODE: + node._id_cache.clear() + node._id_search_stack = None + elif _in_document(node): + node.ownerDocument._id_cache.clear() + node.ownerDocument._id_search_stack= None + +class Document(Node, DocumentLS): + _child_node_types = (Node.ELEMENT_NODE, Node.PROCESSING_INSTRUCTION_NODE, + Node.COMMENT_NODE, Node.DOCUMENT_TYPE_NODE) + + nodeType = Node.DOCUMENT_NODE + nodeName = "#document" + nodeValue = None + attributes = None + doctype = None + parentNode = None + previousSibling = nextSibling = None + + implementation = DOMImplementation() + + # Document attributes from Level 3 (WD 9 April 2002) + + actualEncoding = None + encoding = None + standalone = None + version = None + strictErrorChecking = False + errorHandler = None + documentURI = None + + _magic_id_count = 0 + + def __init__(self): + self.childNodes = NodeList() + # mapping of (namespaceURI, localName) -> ElementInfo + # and tagName -> ElementInfo + self._elem_info = {} + self._id_cache = {} + self._id_search_stack = None + + def _get_elem_info(self, element): + if element.namespaceURI: + key = element.namespaceURI, element.localName + else: + key = element.tagName + return self._elem_info.get(key) + + def _get_actualEncoding(self): + return self.actualEncoding + + def _get_doctype(self): + return self.doctype + + def _get_documentURI(self): + return self.documentURI + + def _get_encoding(self): + return self.encoding + + def _get_errorHandler(self): + return self.errorHandler + + def _get_standalone(self): + return self.standalone + + def _get_strictErrorChecking(self): + return self.strictErrorChecking + + def _get_version(self): + return self.version + + def appendChild(self, node): + if node.nodeType not in self._child_node_types: + raise xml.dom.HierarchyRequestErr( + "%s cannot be child of %s" % (repr(node), repr(self))) + if node.parentNode is not None: + # This needs to be done before the next test since this + # may *be* the document element, in which case it should + # end up re-ordered to the end. + node.parentNode.removeChild(node) + + if node.nodeType == Node.ELEMENT_NODE \ + and self._get_documentElement(): + raise xml.dom.HierarchyRequestErr( + "two document elements disallowed") + return Node.appendChild(self, node) + + def removeChild(self, oldChild): + try: + self.childNodes.remove(oldChild) + except ValueError: + raise xml.dom.NotFoundErr() + oldChild.nextSibling = oldChild.previousSibling = None + oldChild.parentNode = None + if self.documentElement is oldChild: + self.documentElement = None + + return oldChild + + def _get_documentElement(self): + for node in self.childNodes: + if node.nodeType == Node.ELEMENT_NODE: + return node + + def unlink(self): + if self.doctype is not None: + self.doctype.unlink() + self.doctype = None + Node.unlink(self) + + def cloneNode(self, deep): + if not deep: + return None + clone = self.implementation.createDocument(None, None, None) + clone.encoding = self.encoding + clone.standalone = self.standalone + clone.version = self.version + for n in self.childNodes: + childclone = _clone_node(n, deep, clone) + assert childclone.ownerDocument.isSameNode(clone) + clone.childNodes.append(childclone) + if childclone.nodeType == Node.DOCUMENT_NODE: + assert clone.documentElement is None + elif childclone.nodeType == Node.DOCUMENT_TYPE_NODE: + assert clone.doctype is None + clone.doctype = childclone + childclone.parentNode = clone + self._call_user_data_handler(xml.dom.UserDataHandler.NODE_CLONED, + self, clone) + return clone + + def createDocumentFragment(self): + d = DocumentFragment() + d.ownerDocument = self + return d + + def createElement(self, tagName): + e = Element(tagName) + e.ownerDocument = self + return e + + def createTextNode(self, data): + if not isinstance(data, StringTypes): + raise TypeError, "node contents must be a string" + t = Text() + t.data = data + t.ownerDocument = self + return t + + def createCDATASection(self, data): + if not isinstance(data, StringTypes): + raise TypeError, "node contents must be a string" + c = CDATASection() + c.data = data + c.ownerDocument = self + return c + + def createComment(self, data): + c = Comment(data) + c.ownerDocument = self + return c + + def createProcessingInstruction(self, target, data): + p = ProcessingInstruction(target, data) + p.ownerDocument = self + return p + + def createAttribute(self, qName): + a = Attr(qName) + a.ownerDocument = self + a.value = "" + return a + + def createElementNS(self, namespaceURI, qualifiedName): + prefix, localName = _nssplit(qualifiedName) + e = Element(qualifiedName, namespaceURI, prefix) + e.ownerDocument = self + return e + + def createAttributeNS(self, namespaceURI, qualifiedName): + prefix, localName = _nssplit(qualifiedName) + a = Attr(qualifiedName, namespaceURI, localName, prefix) + a.ownerDocument = self + a.value = "" + return a + + # A couple of implementation-specific helpers to create node types + # not supported by the W3C DOM specs: + + def _create_entity(self, name, publicId, systemId, notationName): + e = Entity(name, publicId, systemId, notationName) + e.ownerDocument = self + return e + + def _create_notation(self, name, publicId, systemId): + n = Notation(name, publicId, systemId) + n.ownerDocument = self + return n + + def getElementById(self, id): + if self._id_cache.has_key(id): + return self._id_cache[id] + if not (self._elem_info or self._magic_id_count): + return None + + stack = self._id_search_stack + if stack is None: + # we never searched before, or the cache has been cleared + stack = [self.documentElement] + self._id_search_stack = stack + elif not stack: + # Previous search was completed and cache is still valid; + # no matching node. + return None + + result = None + while stack: + node = stack.pop() + # add child elements to stack for continued searching + stack.extend([child for child in node.childNodes + if child.nodeType in _nodeTypes_with_children]) + # check this node + info = self._get_elem_info(node) + if info: + # We have to process all ID attributes before + # returning in order to get all the attributes set to + # be IDs using Element.setIdAttribute*(). + for attr in node.attributes.values(): + if attr.namespaceURI: + if info.isIdNS(attr.namespaceURI, attr.localName): + self._id_cache[attr.value] = node + if attr.value == id: + result = node + elif not node._magic_id_nodes: + break + elif info.isId(attr.name): + self._id_cache[attr.value] = node + if attr.value == id: + result = node + elif not node._magic_id_nodes: + break + elif attr._is_id: + self._id_cache[attr.value] = node + if attr.value == id: + result = node + elif node._magic_id_nodes == 1: + break + elif node._magic_id_nodes: + for attr in node.attributes.values(): + if attr._is_id: + self._id_cache[attr.value] = node + if attr.value == id: + result = node + if result is not None: + break + return result + + def getElementsByTagName(self, name): + return _get_elements_by_tagName_helper(self, name, NodeList()) + + def getElementsByTagNameNS(self, namespaceURI, localName): + return _get_elements_by_tagName_ns_helper( + self, namespaceURI, localName, NodeList()) + + def isSupported(self, feature, version): + return self.implementation.hasFeature(feature, version) + + def importNode(self, node, deep): + if node.nodeType == Node.DOCUMENT_NODE: + raise xml.dom.NotSupportedErr("cannot import document nodes") + elif node.nodeType == Node.DOCUMENT_TYPE_NODE: + raise xml.dom.NotSupportedErr("cannot import document type nodes") + return _clone_node(node, deep, self) + + def writexml(self, writer, indent="", addindent="", newl="", + encoding = None): + if encoding is None: + writer.write(''+newl) + else: + writer.write('%s' % (encoding, newl)) + for node in self.childNodes: + node.writexml(writer, indent, addindent, newl) + + # DOM Level 3 (WD 9 April 2002) + + def renameNode(self, n, namespaceURI, name): + if n.ownerDocument is not self: + raise xml.dom.WrongDocumentErr( + "cannot rename nodes from other documents;\n" + "expected %s,\nfound %s" % (self, n.ownerDocument)) + if n.nodeType not in (Node.ELEMENT_NODE, Node.ATTRIBUTE_NODE): + raise xml.dom.NotSupportedErr( + "renameNode() only applies to element and attribute nodes") + if namespaceURI != EMPTY_NAMESPACE: + if ':' in name: + prefix, localName = name.split(':', 1) + if ( prefix == "xmlns" + and namespaceURI != xml.dom.XMLNS_NAMESPACE): + raise xml.dom.NamespaceErr( + "illegal use of 'xmlns' prefix") + else: + if ( name == "xmlns" + and namespaceURI != xml.dom.XMLNS_NAMESPACE + and n.nodeType == Node.ATTRIBUTE_NODE): + raise xml.dom.NamespaceErr( + "illegal use of the 'xmlns' attribute") + prefix = None + localName = name + else: + prefix = None + localName = None + if n.nodeType == Node.ATTRIBUTE_NODE: + element = n.ownerElement + if element is not None: + is_id = n._is_id + element.removeAttributeNode(n) + else: + element = None + # avoid __setattr__ + d = n.__dict__ + d['prefix'] = prefix + d['localName'] = localName + d['namespaceURI'] = namespaceURI + d['nodeName'] = name + if n.nodeType == Node.ELEMENT_NODE: + d['tagName'] = name + else: + # attribute node + d['name'] = name + if element is not None: + element.setAttributeNode(n) + if is_id: + element.setIdAttributeNode(n) + # It's not clear from a semantic perspective whether we should + # call the user data handlers for the NODE_RENAMED event since + # we're re-using the existing node. The draft spec has been + # interpreted as meaning "no, don't call the handler unless a + # new node is created." + return n + +defproperty(Document, "documentElement", + doc="Top-level element of this document.") + + +def _clone_node(node, deep, newOwnerDocument): + """ + Clone a node and give it the new owner document. + Called by Node.cloneNode and Document.importNode + """ + if node.ownerDocument.isSameNode(newOwnerDocument): + operation = xml.dom.UserDataHandler.NODE_CLONED + else: + operation = xml.dom.UserDataHandler.NODE_IMPORTED + if node.nodeType == Node.ELEMENT_NODE: + clone = newOwnerDocument.createElementNS(node.namespaceURI, + node.nodeName) + for attr in node.attributes.values(): + clone.setAttributeNS(attr.namespaceURI, attr.nodeName, attr.value) + a = clone.getAttributeNodeNS(attr.namespaceURI, attr.localName) + a.specified = attr.specified + + if deep: + for child in node.childNodes: + c = _clone_node(child, deep, newOwnerDocument) + clone.appendChild(c) + + elif node.nodeType == Node.DOCUMENT_FRAGMENT_NODE: + clone = newOwnerDocument.createDocumentFragment() + if deep: + for child in node.childNodes: + c = _clone_node(child, deep, newOwnerDocument) + clone.appendChild(c) + + elif node.nodeType == Node.TEXT_NODE: + clone = newOwnerDocument.createTextNode(node.data) + elif node.nodeType == Node.CDATA_SECTION_NODE: + clone = newOwnerDocument.createCDATASection(node.data) + elif node.nodeType == Node.PROCESSING_INSTRUCTION_NODE: + clone = newOwnerDocument.createProcessingInstruction(node.target, + node.data) + elif node.nodeType == Node.COMMENT_NODE: + clone = newOwnerDocument.createComment(node.data) + elif node.nodeType == Node.ATTRIBUTE_NODE: + clone = newOwnerDocument.createAttributeNS(node.namespaceURI, + node.nodeName) + clone.specified = True + clone.value = node.value + elif node.nodeType == Node.DOCUMENT_TYPE_NODE: + assert node.ownerDocument is not newOwnerDocument + operation = xml.dom.UserDataHandler.NODE_IMPORTED + clone = newOwnerDocument.implementation.createDocumentType( + node.name, node.publicId, node.systemId) + clone.ownerDocument = newOwnerDocument + if deep: + clone.entities._seq = [] + clone.notations._seq = [] + for n in node.notations._seq: + notation = Notation(n.nodeName, n.publicId, n.systemId) + notation.ownerDocument = newOwnerDocument + clone.notations._seq.append(notation) + if hasattr(n, '_call_user_data_handler'): + n._call_user_data_handler(operation, n, notation) + for e in node.entities._seq: + entity = Entity(e.nodeName, e.publicId, e.systemId, + e.notationName) + entity.actualEncoding = e.actualEncoding + entity.encoding = e.encoding + entity.version = e.version + entity.ownerDocument = newOwnerDocument + clone.entities._seq.append(entity) + if hasattr(e, '_call_user_data_handler'): + e._call_user_data_handler(operation, n, entity) + else: + # Note the cloning of Document and DocumentType nodes is + # implemenetation specific. minidom handles those cases + # directly in the cloneNode() methods. + raise xml.dom.NotSupportedErr("Cannot clone node %s" % repr(node)) + + # Check for _call_user_data_handler() since this could conceivably + # used with other DOM implementations (one of the FourThought + # DOMs, perhaps?). + if hasattr(node, '_call_user_data_handler'): + node._call_user_data_handler(operation, node, clone) + return clone + + +def _nssplit(qualifiedName): + fields = qualifiedName.split(':', 1) + if len(fields) == 2: + return fields + else: + return (None, fields[0]) + + +def _get_StringIO(): + # we can't use cStringIO since it doesn't support Unicode strings + from StringIO import StringIO + return StringIO() + +def _do_pulldom_parse(func, args, kwargs): + events = func(*args, **kwargs) + toktype, rootNode = events.getEvent() + events.expandNode(rootNode) + events.clear() + return rootNode + +def parse(file, parser=None, bufsize=None): + """Parse a file into a DOM by filename or file object.""" + if parser is None and not bufsize: + from xml.dom import expatbuilder + return expatbuilder.parse(file) + else: + from xml.dom import pulldom + return _do_pulldom_parse(pulldom.parse, (file,), + {'parser': parser, 'bufsize': bufsize}) + +def parseString(string, parser=None): + """Parse a file into a DOM from a string.""" + if parser is None: + from xml.dom import expatbuilder + return expatbuilder.parseString(string) + else: + from xml.dom import pulldom + return _do_pulldom_parse(pulldom.parseString, (string,), + {'parser': parser}) + +def getDOMImplementation(features=None): + if features: + if isinstance(features, StringTypes): + features = domreg._parse_feature_string(features) + for f, v in features: + if not Document.implementation.hasFeature(f, v): + return None + return Document.implementation diff --git a/Lib/xml/dom/pulldom.py b/Lib/xml/dom/pulldom.py new file mode 100644 index 0000000..18f49b5 --- /dev/null +++ b/Lib/xml/dom/pulldom.py @@ -0,0 +1,351 @@ +import xml.sax +import xml.sax.handler +import types + +try: + _StringTypes = [types.StringType, types.UnicodeType] +except AttributeError: + _StringTypes = [types.StringType] + +START_ELEMENT = "START_ELEMENT" +END_ELEMENT = "END_ELEMENT" +COMMENT = "COMMENT" +START_DOCUMENT = "START_DOCUMENT" +END_DOCUMENT = "END_DOCUMENT" +PROCESSING_INSTRUCTION = "PROCESSING_INSTRUCTION" +IGNORABLE_WHITESPACE = "IGNORABLE_WHITESPACE" +CHARACTERS = "CHARACTERS" + +class PullDOM(xml.sax.ContentHandler): + _locator = None + document = None + + def __init__(self, documentFactory=None): + from xml.dom import XML_NAMESPACE + self.documentFactory = documentFactory + self.firstEvent = [None, None] + self.lastEvent = self.firstEvent + self.elementStack = [] + self.push = self.elementStack.append + try: + self.pop = self.elementStack.pop + except AttributeError: + # use class' pop instead + pass + self._ns_contexts = [{XML_NAMESPACE:'xml'}] # contains uri -> prefix dicts + self._current_context = self._ns_contexts[-1] + self.pending_events = [] + + def pop(self): + result = self.elementStack[-1] + del self.elementStack[-1] + return result + + def setDocumentLocator(self, locator): + self._locator = locator + + def startPrefixMapping(self, prefix, uri): + if not hasattr(self, '_xmlns_attrs'): + self._xmlns_attrs = [] + self._xmlns_attrs.append((prefix or 'xmlns', uri)) + self._ns_contexts.append(self._current_context.copy()) + self._current_context[uri] = prefix or None + + def endPrefixMapping(self, prefix): + self._current_context = self._ns_contexts.pop() + + def startElementNS(self, name, tagName , attrs): + # Retrieve xml namespace declaration attributes. + xmlns_uri = 'http://www.w3.org/2000/xmlns/' + xmlns_attrs = getattr(self, '_xmlns_attrs', None) + if xmlns_attrs is not None: + for aname, value in xmlns_attrs: + attrs._attrs[(xmlns_uri, aname)] = value + self._xmlns_attrs = [] + uri, localname = name + if uri: + # When using namespaces, the reader may or may not + # provide us with the original name. If not, create + # *a* valid tagName from the current context. + if tagName is None: + prefix = self._current_context[uri] + if prefix: + tagName = prefix + ":" + localname + else: + tagName = localname + if self.document: + node = self.document.createElementNS(uri, tagName) + else: + node = self.buildDocument(uri, tagName) + else: + # When the tagname is not prefixed, it just appears as + # localname + if self.document: + node = self.document.createElement(localname) + else: + node = self.buildDocument(None, localname) + + for aname,value in attrs.items(): + a_uri, a_localname = aname + if a_uri == xmlns_uri: + if a_localname == 'xmlns': + qname = a_localname + else: + qname = 'xmlns:' + a_localname + attr = self.document.createAttributeNS(a_uri, qname) + node.setAttributeNodeNS(attr) + elif a_uri: + prefix = self._current_context[a_uri] + if prefix: + qname = prefix + ":" + a_localname + else: + qname = a_localname + attr = self.document.createAttributeNS(a_uri, qname) + node.setAttributeNodeNS(attr) + else: + attr = self.document.createAttribute(a_localname) + node.setAttributeNode(attr) + attr.value = value + + self.lastEvent[1] = [(START_ELEMENT, node), None] + self.lastEvent = self.lastEvent[1] + self.push(node) + + def endElementNS(self, name, tagName): + self.lastEvent[1] = [(END_ELEMENT, self.pop()), None] + self.lastEvent = self.lastEvent[1] + + def startElement(self, name, attrs): + if self.document: + node = self.document.createElement(name) + else: + node = self.buildDocument(None, name) + + for aname,value in attrs.items(): + attr = self.document.createAttribute(aname) + attr.value = value + node.setAttributeNode(attr) + + self.lastEvent[1] = [(START_ELEMENT, node), None] + self.lastEvent = self.lastEvent[1] + self.push(node) + + def endElement(self, name): + self.lastEvent[1] = [(END_ELEMENT, self.pop()), None] + self.lastEvent = self.lastEvent[1] + + def comment(self, s): + if self.document: + node = self.document.createComment(s) + self.lastEvent[1] = [(COMMENT, node), None] + self.lastEvent = self.lastEvent[1] + else: + event = [(COMMENT, s), None] + self.pending_events.append(event) + + def processingInstruction(self, target, data): + if self.document: + node = self.document.createProcessingInstruction(target, data) + self.lastEvent[1] = [(PROCESSING_INSTRUCTION, node), None] + self.lastEvent = self.lastEvent[1] + else: + event = [(PROCESSING_INSTRUCTION, target, data), None] + self.pending_events.append(event) + + def ignorableWhitespace(self, chars): + node = self.document.createTextNode(chars) + self.lastEvent[1] = [(IGNORABLE_WHITESPACE, node), None] + self.lastEvent = self.lastEvent[1] + + def characters(self, chars): + node = self.document.createTextNode(chars) + self.lastEvent[1] = [(CHARACTERS, node), None] + self.lastEvent = self.lastEvent[1] + + def startDocument(self): + if self.documentFactory is None: + import xml.dom.minidom + self.documentFactory = xml.dom.minidom.Document.implementation + + def buildDocument(self, uri, tagname): + # Can't do that in startDocument, since we need the tagname + # XXX: obtain DocumentType + node = self.documentFactory.createDocument(uri, tagname, None) + self.document = node + self.lastEvent[1] = [(START_DOCUMENT, node), None] + self.lastEvent = self.lastEvent[1] + self.push(node) + # Put everything we have seen so far into the document + for e in self.pending_events: + if e[0][0] == PROCESSING_INSTRUCTION: + _,target,data = e[0] + n = self.document.createProcessingInstruction(target, data) + e[0] = (PROCESSING_INSTRUCTION, n) + elif e[0][0] == COMMENT: + n = self.document.createComment(e[0][1]) + e[0] = (COMMENT, n) + else: + raise AssertionError("Unknown pending event ",e[0][0]) + self.lastEvent[1] = e + self.lastEvent = e + self.pending_events = None + return node.firstChild + + def endDocument(self): + self.lastEvent[1] = [(END_DOCUMENT, self.document), None] + self.pop() + + def clear(self): + "clear(): Explicitly release parsing structures" + self.document = None + +class ErrorHandler: + def warning(self, exception): + print exception + def error(self, exception): + raise exception + def fatalError(self, exception): + raise exception + +class DOMEventStream: + def __init__(self, stream, parser, bufsize): + self.stream = stream + self.parser = parser + self.bufsize = bufsize + if not hasattr(self.parser, 'feed'): + self.getEvent = self._slurp + self.reset() + + def reset(self): + self.pulldom = PullDOM() + # This content handler relies on namespace support + self.parser.setFeature(xml.sax.handler.feature_namespaces, 1) + self.parser.setContentHandler(self.pulldom) + + def __getitem__(self, pos): + rc = self.getEvent() + if rc: + return rc + raise IndexError + + def next(self): + rc = self.getEvent() + if rc: + return rc + raise StopIteration + + def __iter__(self): + return self + + def expandNode(self, node): + event = self.getEvent() + parents = [node] + while event: + token, cur_node = event + if cur_node is node: + return + if token != END_ELEMENT: + parents[-1].appendChild(cur_node) + if token == START_ELEMENT: + parents.append(cur_node) + elif token == END_ELEMENT: + del parents[-1] + event = self.getEvent() + + def getEvent(self): + # use IncrementalParser interface, so we get the desired + # pull effect + if not self.pulldom.firstEvent[1]: + self.pulldom.lastEvent = self.pulldom.firstEvent + while not self.pulldom.firstEvent[1]: + buf = self.stream.read(self.bufsize) + if not buf: + self.parser.close() + return None + self.parser.feed(buf) + rc = self.pulldom.firstEvent[1][0] + self.pulldom.firstEvent[1] = self.pulldom.firstEvent[1][1] + return rc + + def _slurp(self): + """ Fallback replacement for getEvent() using the + standard SAX2 interface, which means we slurp the + SAX events into memory (no performance gain, but + we are compatible to all SAX parsers). + """ + self.parser.parse(self.stream) + self.getEvent = self._emit + return self._emit() + + def _emit(self): + """ Fallback replacement for getEvent() that emits + the events that _slurp() read previously. + """ + rc = self.pulldom.firstEvent[1][0] + self.pulldom.firstEvent[1] = self.pulldom.firstEvent[1][1] + return rc + + def clear(self): + """clear(): Explicitly release parsing objects""" + self.pulldom.clear() + del self.pulldom + self.parser = None + self.stream = None + +class SAX2DOM(PullDOM): + + def startElementNS(self, name, tagName , attrs): + PullDOM.startElementNS(self, name, tagName, attrs) + curNode = self.elementStack[-1] + parentNode = self.elementStack[-2] + parentNode.appendChild(curNode) + + def startElement(self, name, attrs): + PullDOM.startElement(self, name, attrs) + curNode = self.elementStack[-1] + parentNode = self.elementStack[-2] + parentNode.appendChild(curNode) + + def processingInstruction(self, target, data): + PullDOM.processingInstruction(self, target, data) + node = self.lastEvent[0][1] + parentNode = self.elementStack[-1] + parentNode.appendChild(node) + + def ignorableWhitespace(self, chars): + PullDOM.ignorableWhitespace(self, chars) + node = self.lastEvent[0][1] + parentNode = self.elementStack[-1] + parentNode.appendChild(node) + + def characters(self, chars): + PullDOM.characters(self, chars) + node = self.lastEvent[0][1] + parentNode = self.elementStack[-1] + parentNode.appendChild(node) + + +default_bufsize = (2 ** 14) - 20 + +def parse(stream_or_string, parser=None, bufsize=None): + if bufsize is None: + bufsize = default_bufsize + if type(stream_or_string) in _StringTypes: + stream = open(stream_or_string) + else: + stream = stream_or_string + if not parser: + parser = xml.sax.make_parser() + return DOMEventStream(stream, parser, bufsize) + +def parseString(string, parser=None): + try: + from cStringIO import StringIO + except ImportError: + from StringIO import StringIO + + bufsize = len(string) + buf = StringIO(string) + if not parser: + parser = xml.sax.make_parser() + return DOMEventStream(buf, parser, bufsize) diff --git a/Lib/xml/dom/xmlbuilder.py b/Lib/xml/dom/xmlbuilder.py new file mode 100644 index 0000000..ac1d448 --- /dev/null +++ b/Lib/xml/dom/xmlbuilder.py @@ -0,0 +1,386 @@ +"""Implementation of the DOM Level 3 'LS-Load' feature.""" + +import copy +import xml.dom + +from xml.dom.NodeFilter import NodeFilter + + +__all__ = ["DOMBuilder", "DOMEntityResolver", "DOMInputSource"] + + +class Options: + """Features object that has variables set for each DOMBuilder feature. + + The DOMBuilder class uses an instance of this class to pass settings to + the ExpatBuilder class. + """ + + # Note that the DOMBuilder class in LoadSave constrains which of these + # values can be set using the DOM Level 3 LoadSave feature. + + namespaces = 1 + namespace_declarations = True + validation = False + external_parameter_entities = True + external_general_entities = True + external_dtd_subset = True + validate_if_schema = False + validate = False + datatype_normalization = False + create_entity_ref_nodes = True + entities = True + whitespace_in_element_content = True + cdata_sections = True + comments = True + charset_overrides_xml_encoding = True + infoset = False + supported_mediatypes_only = False + + errorHandler = None + filter = None + + +class DOMBuilder: + entityResolver = None + errorHandler = None + filter = None + + ACTION_REPLACE = 1 + ACTION_APPEND_AS_CHILDREN = 2 + ACTION_INSERT_AFTER = 3 + ACTION_INSERT_BEFORE = 4 + + _legal_actions = (ACTION_REPLACE, ACTION_APPEND_AS_CHILDREN, + ACTION_INSERT_AFTER, ACTION_INSERT_BEFORE) + + def __init__(self): + self._options = Options() + + def _get_entityResolver(self): + return self.entityResolver + def _set_entityResolver(self, entityResolver): + self.entityResolver = entityResolver + + def _get_errorHandler(self): + return self.errorHandler + def _set_errorHandler(self, errorHandler): + self.errorHandler = errorHandler + + def _get_filter(self): + return self.filter + def _set_filter(self, filter): + self.filter = filter + + def setFeature(self, name, state): + if self.supportsFeature(name): + state = state and 1 or 0 + try: + settings = self._settings[(_name_xform(name), state)] + except KeyError: + raise xml.dom.NotSupportedErr( + "unsupported feature: %r" % (name,)) + else: + for name, value in settings: + setattr(self._options, name, value) + else: + raise xml.dom.NotFoundErr("unknown feature: " + repr(name)) + + def supportsFeature(self, name): + return hasattr(self._options, _name_xform(name)) + + def canSetFeature(self, name, state): + key = (_name_xform(name), state and 1 or 0) + return self._settings.has_key(key) + + # This dictionary maps from (feature,value) to a list of + # (option,value) pairs that should be set on the Options object. + # If a (feature,value) setting is not in this dictionary, it is + # not supported by the DOMBuilder. + # + _settings = { + ("namespace_declarations", 0): [ + ("namespace_declarations", 0)], + ("namespace_declarations", 1): [ + ("namespace_declarations", 1)], + ("validation", 0): [ + ("validation", 0)], + ("external_general_entities", 0): [ + ("external_general_entities", 0)], + ("external_general_entities", 1): [ + ("external_general_entities", 1)], + ("external_parameter_entities", 0): [ + ("external_parameter_entities", 0)], + ("external_parameter_entities", 1): [ + ("external_parameter_entities", 1)], + ("validate_if_schema", 0): [ + ("validate_if_schema", 0)], + ("create_entity_ref_nodes", 0): [ + ("create_entity_ref_nodes", 0)], + ("create_entity_ref_nodes", 1): [ + ("create_entity_ref_nodes", 1)], + ("entities", 0): [ + ("create_entity_ref_nodes", 0), + ("entities", 0)], + ("entities", 1): [ + ("entities", 1)], + ("whitespace_in_element_content", 0): [ + ("whitespace_in_element_content", 0)], + ("whitespace_in_element_content", 1): [ + ("whitespace_in_element_content", 1)], + ("cdata_sections", 0): [ + ("cdata_sections", 0)], + ("cdata_sections", 1): [ + ("cdata_sections", 1)], + ("comments", 0): [ + ("comments", 0)], + ("comments", 1): [ + ("comments", 1)], + ("charset_overrides_xml_encoding", 0): [ + ("charset_overrides_xml_encoding", 0)], + ("charset_overrides_xml_encoding", 1): [ + ("charset_overrides_xml_encoding", 1)], + ("infoset", 0): [], + ("infoset", 1): [ + ("namespace_declarations", 0), + ("validate_if_schema", 0), + ("create_entity_ref_nodes", 0), + ("entities", 0), + ("cdata_sections", 0), + ("datatype_normalization", 1), + ("whitespace_in_element_content", 1), + ("comments", 1), + ("charset_overrides_xml_encoding", 1)], + ("supported_mediatypes_only", 0): [ + ("supported_mediatypes_only", 0)], + ("namespaces", 0): [ + ("namespaces", 0)], + ("namespaces", 1): [ + ("namespaces", 1)], + } + + def getFeature(self, name): + xname = _name_xform(name) + try: + return getattr(self._options, xname) + except AttributeError: + if name == "infoset": + options = self._options + return (options.datatype_normalization + and options.whitespace_in_element_content + and options.comments + and options.charset_overrides_xml_encoding + and not (options.namespace_declarations + or options.validate_if_schema + or options.create_entity_ref_nodes + or options.entities + or options.cdata_sections)) + raise xml.dom.NotFoundErr("feature %s not known" % repr(name)) + + def parseURI(self, uri): + if self.entityResolver: + input = self.entityResolver.resolveEntity(None, uri) + else: + input = DOMEntityResolver().resolveEntity(None, uri) + return self.parse(input) + + def parse(self, input): + options = copy.copy(self._options) + options.filter = self.filter + options.errorHandler = self.errorHandler + fp = input.byteStream + if fp is None and options.systemId: + import urllib2 + fp = urllib2.urlopen(input.systemId) + return self._parse_bytestream(fp, options) + + def parseWithContext(self, input, cnode, action): + if action not in self._legal_actions: + raise ValueError("not a legal action") + raise NotImplementedError("Haven't written this yet...") + + def _parse_bytestream(self, stream, options): + import xml.dom.expatbuilder + builder = xml.dom.expatbuilder.makeBuilder(options) + return builder.parseFile(stream) + + +def _name_xform(name): + return name.lower().replace('-', '_') + + +class DOMEntityResolver(object): + __slots__ = '_opener', + + def resolveEntity(self, publicId, systemId): + assert systemId is not None + source = DOMInputSource() + source.publicId = publicId + source.systemId = systemId + source.byteStream = self._get_opener().open(systemId) + + # determine the encoding if the transport provided it + source.encoding = self._guess_media_encoding(source) + + # determine the base URI is we can + import posixpath, urlparse + parts = urlparse.urlparse(systemId) + scheme, netloc, path, params, query, fragment = parts + # XXX should we check the scheme here as well? + if path and not path.endswith("/"): + path = posixpath.dirname(path) + "/" + parts = scheme, netloc, path, params, query, fragment + source.baseURI = urlparse.urlunparse(parts) + + return source + + def _get_opener(self): + try: + return self._opener + except AttributeError: + self._opener = self._create_opener() + return self._opener + + def _create_opener(self): + import urllib2 + return urllib2.build_opener() + + def _guess_media_encoding(self, source): + info = source.byteStream.info() + if info.has_key("Content-Type"): + for param in info.getplist(): + if param.startswith("charset="): + return param.split("=", 1)[1].lower() + + +class DOMInputSource(object): + __slots__ = ('byteStream', 'characterStream', 'stringData', + 'encoding', 'publicId', 'systemId', 'baseURI') + + def __init__(self): + self.byteStream = None + self.characterStream = None + self.stringData = None + self.encoding = None + self.publicId = None + self.systemId = None + self.baseURI = None + + def _get_byteStream(self): + return self.byteStream + def _set_byteStream(self, byteStream): + self.byteStream = byteStream + + def _get_characterStream(self): + return self.characterStream + def _set_characterStream(self, characterStream): + self.characterStream = characterStream + + def _get_stringData(self): + return self.stringData + def _set_stringData(self, data): + self.stringData = data + + def _get_encoding(self): + return self.encoding + def _set_encoding(self, encoding): + self.encoding = encoding + + def _get_publicId(self): + return self.publicId + def _set_publicId(self, publicId): + self.publicId = publicId + + def _get_systemId(self): + return self.systemId + def _set_systemId(self, systemId): + self.systemId = systemId + + def _get_baseURI(self): + return self.baseURI + def _set_baseURI(self, uri): + self.baseURI = uri + + +class DOMBuilderFilter: + """Element filter which can be used to tailor construction of + a DOM instance. + """ + + # There's really no need for this class; concrete implementations + # should just implement the endElement() and startElement() + # methods as appropriate. Using this makes it easy to only + # implement one of them. + + FILTER_ACCEPT = 1 + FILTER_REJECT = 2 + FILTER_SKIP = 3 + FILTER_INTERRUPT = 4 + + whatToShow = NodeFilter.SHOW_ALL + + def _get_whatToShow(self): + return self.whatToShow + + def acceptNode(self, element): + return self.FILTER_ACCEPT + + def startContainer(self, element): + return self.FILTER_ACCEPT + +del NodeFilter + + +class DocumentLS: + """Mixin to create documents that conform to the load/save spec.""" + + async = False + + def _get_async(self): + return False + def _set_async(self, async): + if async: + raise xml.dom.NotSupportedErr( + "asynchronous document loading is not supported") + + def abort(self): + # What does it mean to "clear" a document? Does the + # documentElement disappear? + raise NotImplementedError( + "haven't figured out what this means yet") + + def load(self, uri): + raise NotImplementedError("haven't written this yet") + + def loadXML(self, source): + raise NotImplementedError("haven't written this yet") + + def saveXML(self, snode): + if snode is None: + snode = self + elif snode.ownerDocument is not self: + raise xml.dom.WrongDocumentErr() + return snode.toxml() + + +class DOMImplementationLS: + MODE_SYNCHRONOUS = 1 + MODE_ASYNCHRONOUS = 2 + + def createDOMBuilder(self, mode, schemaType): + if schemaType is not None: + raise xml.dom.NotSupportedErr( + "schemaType not yet supported") + if mode == self.MODE_SYNCHRONOUS: + return DOMBuilder() + if mode == self.MODE_ASYNCHRONOUS: + raise xml.dom.NotSupportedErr( + "asynchronous builders are not supported") + raise ValueError("unknown value for mode") + + def createDOMWriter(self): + raise NotImplementedError( + "the writer interface hasn't been written yet!") + + def createDOMInputSource(self): + return DOMInputSource() diff --git a/Lib/xml/etree/ElementInclude.py b/Lib/xml/etree/ElementInclude.py new file mode 100644 index 0000000..974cc21 --- /dev/null +++ b/Lib/xml/etree/ElementInclude.py @@ -0,0 +1,143 @@ +# +# ElementTree +# $Id: ElementInclude.py 1862 2004-06-18 07:31:02Z Fredrik $ +# +# limited xinclude support for element trees +# +# history: +# 2003-08-15 fl created +# 2003-11-14 fl fixed default loader +# +# Copyright (c) 2003-2004 by Fredrik Lundh. All rights reserved. +# +# fredrik@pythonware.com +# http://www.pythonware.com +# +# -------------------------------------------------------------------- +# The ElementTree toolkit is +# +# Copyright (c) 1999-2004 by Fredrik Lundh +# +# By obtaining, using, and/or copying this software and/or its +# associated documentation, you agree that you have read, understood, +# and will comply with the following terms and conditions: +# +# Permission to use, copy, modify, and distribute this software and +# its associated documentation for any purpose and without fee is +# hereby granted, provided that the above copyright notice appears in +# all copies, and that both that copyright notice and this permission +# notice appear in supporting documentation, and that the name of +# Secret Labs AB or the author not be used in advertising or publicity +# pertaining to distribution of the software without specific, written +# prior permission. +# +# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD +# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- +# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR +# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY +# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, +# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS +# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE +# OF THIS SOFTWARE. +# -------------------------------------------------------------------- + +# Licensed to PSF under a Contributor Agreement. +# See http://www.python.org/2.4/license for licensing details. + +## +# Limited XInclude support for the ElementTree package. +## + +import copy +import ElementTree + +XINCLUDE = "{http://www.w3.org/2001/XInclude}" + +XINCLUDE_INCLUDE = XINCLUDE + "include" +XINCLUDE_FALLBACK = XINCLUDE + "fallback" + +## +# Fatal include error. + +class FatalIncludeError(SyntaxError): + pass + +## +# Default loader. This loader reads an included resource from disk. +# +# @param href Resource reference. +# @param parse Parse mode. Either "xml" or "text". +# @param encoding Optional text encoding. +# @return The expanded resource. If the parse mode is "xml", this +# is an ElementTree instance. If the parse mode is "text", this +# is a Unicode string. If the loader fails, it can return None +# or raise an IOError exception. +# @throws IOError If the loader fails to load the resource. + +def default_loader(href, parse, encoding=None): + file = open(href) + if parse == "xml": + data = ElementTree.parse(file).getroot() + else: + data = file.read() + if encoding: + data = data.decode(encoding) + file.close() + return data + +## +# Expand XInclude directives. +# +# @param elem Root element. +# @param loader Optional resource loader. If omitted, it defaults +# to {@link default_loader}. If given, it should be a callable +# that implements the same interface as default_loader. +# @throws FatalIncludeError If the function fails to include a given +# resource, or if the tree contains malformed XInclude elements. +# @throws IOError If the function fails to load a given resource. + +def include(elem, loader=None): + if loader is None: + loader = default_loader + # look for xinclude elements + i = 0 + while i < len(elem): + e = elem[i] + if e.tag == XINCLUDE_INCLUDE: + # process xinclude directive + href = e.get("href") + parse = e.get("parse", "xml") + if parse == "xml": + node = loader(href, parse) + if node is None: + raise FatalIncludeError( + "cannot load %r as %r" % (href, parse) + ) + node = copy.copy(node) + if e.tail: + node.tail = (node.tail or "") + e.tail + elem[i] = node + elif parse == "text": + text = loader(href, parse, e.get("encoding")) + if text is None: + raise FatalIncludeError( + "cannot load %r as %r" % (href, parse) + ) + if i: + node = elem[i-1] + node.tail = (node.tail or "") + text + else: + elem.text = (elem.text or "") + text + (e.tail or "") + del elem[i] + continue + else: + raise FatalIncludeError( + "unknown parse type in xi:include tag (%r)" % parse + ) + elif e.tag == XINCLUDE_FALLBACK: + raise FatalIncludeError( + "xi:fallback tag must be child of xi:include (%r)" % e.tag + ) + else: + include(e, loader) + i = i + 1 diff --git a/Lib/xml/etree/ElementPath.py b/Lib/xml/etree/ElementPath.py new file mode 100644 index 0000000..00dbe9d --- /dev/null +++ b/Lib/xml/etree/ElementPath.py @@ -0,0 +1,198 @@ +# +# ElementTree +# $Id: ElementPath.py 1858 2004-06-17 21:31:41Z Fredrik $ +# +# limited xpath support for element trees +# +# history: +# 2003-05-23 fl created +# 2003-05-28 fl added support for // etc +# 2003-08-27 fl fixed parsing of periods in element names +# +# Copyright (c) 2003-2004 by Fredrik Lundh. All rights reserved. +# +# fredrik@pythonware.com +# http://www.pythonware.com +# +# -------------------------------------------------------------------- +# The ElementTree toolkit is +# +# Copyright (c) 1999-2004 by Fredrik Lundh +# +# By obtaining, using, and/or copying this software and/or its +# associated documentation, you agree that you have read, understood, +# and will comply with the following terms and conditions: +# +# Permission to use, copy, modify, and distribute this software and +# its associated documentation for any purpose and without fee is +# hereby granted, provided that the above copyright notice appears in +# all copies, and that both that copyright notice and this permission +# notice appear in supporting documentation, and that the name of +# Secret Labs AB or the author not be used in advertising or publicity +# pertaining to distribution of the software without specific, written +# prior permission. +# +# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD +# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- +# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR +# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY +# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, +# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS +# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE +# OF THIS SOFTWARE. +# -------------------------------------------------------------------- + +# Licensed to PSF under a Contributor Agreement. +# See http://www.python.org/2.4/license for licensing details. + +## +# Implementation module for XPath support. There's usually no reason +# to import this module directly; the ElementTree does this for +# you, if needed. +## + +import re + +xpath_tokenizer = re.compile( + "(::|\.\.|\(\)|[/.*:\[\]\(\)@=])|((?:\{[^}]+\})?[^/:\[\]\(\)@=\s]+)|\s+" + ).findall + +class xpath_descendant_or_self: + pass + +## +# Wrapper for a compiled XPath. + +class Path: + + ## + # Create an Path instance from an XPath expression. + + def __init__(self, path): + tokens = xpath_tokenizer(path) + # the current version supports 'path/path'-style expressions only + self.path = [] + self.tag = None + if tokens and tokens[0][0] == "/": + raise SyntaxError("cannot use absolute path on element") + while tokens: + op, tag = tokens.pop(0) + if tag or op == "*": + self.path.append(tag or op) + elif op == ".": + pass + elif op == "/": + self.path.append(xpath_descendant_or_self()) + continue + else: + raise SyntaxError("unsupported path syntax (%s)" % op) + if tokens: + op, tag = tokens.pop(0) + if op != "/": + raise SyntaxError( + "expected path separator (%s)" % (op or tag) + ) + if self.path and isinstance(self.path[-1], xpath_descendant_or_self): + raise SyntaxError("path cannot end with //") + if len(self.path) == 1 and isinstance(self.path[0], type("")): + self.tag = self.path[0] + + ## + # Find first matching object. + + def find(self, element): + tag = self.tag + if tag is None: + nodeset = self.findall(element) + if not nodeset: + return None + return nodeset[0] + for elem in element: + if elem.tag == tag: + return elem + return None + + ## + # Find text for first matching object. + + def findtext(self, element, default=None): + tag = self.tag + if tag is None: + nodeset = self.findall(element) + if not nodeset: + return default + return nodeset[0].text or "" + for elem in element: + if elem.tag == tag: + return elem.text or "" + return default + + ## + # Find all matching objects. + + def findall(self, element): + nodeset = [element] + index = 0 + while 1: + try: + path = self.path[index] + index = index + 1 + except IndexError: + return nodeset + set = [] + if isinstance(path, xpath_descendant_or_self): + try: + tag = self.path[index] + if not isinstance(tag, type("")): + tag = None + else: + index = index + 1 + except IndexError: + tag = None # invalid path + for node in nodeset: + new = list(node.getiterator(tag)) + if new and new[0] is node: + set.extend(new[1:]) + else: + set.extend(new) + else: + for node in nodeset: + for node in node: + if path == "*" or node.tag == path: + set.append(node) + if not set: + return [] + nodeset = set + +_cache = {} + +## +# (Internal) Compile path. + +def _compile(path): + p = _cache.get(path) + if p is not None: + return p + p = Path(path) + if len(_cache) >= 100: + _cache.clear() + _cache[path] = p + return p + +## +# Find first matching object. + +def find(element, path): + return _compile(path).find(element) + +## +# Find text for first matching object. + +def findtext(element, path, default=None): + return _compile(path).findtext(element, default) + +## +# Find all matching objects. + +def findall(element, path): + return _compile(path).findall(element) diff --git a/Lib/xml/etree/ElementTree.py b/Lib/xml/etree/ElementTree.py new file mode 100644 index 0000000..7dbc72e --- /dev/null +++ b/Lib/xml/etree/ElementTree.py @@ -0,0 +1,1260 @@ +# +# ElementTree +# $Id: ElementTree.py 2326 2005-03-17 07:45:21Z fredrik $ +# +# light-weight XML support for Python 1.5.2 and later. +# +# history: +# 2001-10-20 fl created (from various sources) +# 2001-11-01 fl return root from parse method +# 2002-02-16 fl sort attributes in lexical order +# 2002-04-06 fl TreeBuilder refactoring, added PythonDoc markup +# 2002-05-01 fl finished TreeBuilder refactoring +# 2002-07-14 fl added basic namespace support to ElementTree.write +# 2002-07-25 fl added QName attribute support +# 2002-10-20 fl fixed encoding in write +# 2002-11-24 fl changed default encoding to ascii; fixed attribute encoding +# 2002-11-27 fl accept file objects or file names for parse/write +# 2002-12-04 fl moved XMLTreeBuilder back to this module +# 2003-01-11 fl fixed entity encoding glitch for us-ascii +# 2003-02-13 fl added XML literal factory +# 2003-02-21 fl added ProcessingInstruction/PI factory +# 2003-05-11 fl added tostring/fromstring helpers +# 2003-05-26 fl added ElementPath support +# 2003-07-05 fl added makeelement factory method +# 2003-07-28 fl added more well-known namespace prefixes +# 2003-08-15 fl fixed typo in ElementTree.findtext (Thomas Dartsch) +# 2003-09-04 fl fall back on emulator if ElementPath is not installed +# 2003-10-31 fl markup updates +# 2003-11-15 fl fixed nested namespace bug +# 2004-03-28 fl added XMLID helper +# 2004-06-02 fl added default support to findtext +# 2004-06-08 fl fixed encoding of non-ascii element/attribute names +# 2004-08-23 fl take advantage of post-2.1 expat features +# 2005-02-01 fl added iterparse implementation +# 2005-03-02 fl fixed iterparse support for pre-2.2 versions +# +# Copyright (c) 1999-2005 by Fredrik Lundh. All rights reserved. +# +# fredrik@pythonware.com +# http://www.pythonware.com +# +# -------------------------------------------------------------------- +# The ElementTree toolkit is +# +# Copyright (c) 1999-2005 by Fredrik Lundh +# +# By obtaining, using, and/or copying this software and/or its +# associated documentation, you agree that you have read, understood, +# and will comply with the following terms and conditions: +# +# Permission to use, copy, modify, and distribute this software and +# its associated documentation for any purpose and without fee is +# hereby granted, provided that the above copyright notice appears in +# all copies, and that both that copyright notice and this permission +# notice appear in supporting documentation, and that the name of +# Secret Labs AB or the author not be used in advertising or publicity +# pertaining to distribution of the software without specific, written +# prior permission. +# +# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD +# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- +# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR +# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY +# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, +# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS +# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE +# OF THIS SOFTWARE. +# -------------------------------------------------------------------- + +# Licensed to PSF under a Contributor Agreement. +# See http://www.python.org/2.4/license for licensing details. + +__all__ = [ + # public symbols + "Comment", + "dump", + "Element", "ElementTree", + "fromstring", + "iselement", "iterparse", + "parse", + "PI", "ProcessingInstruction", + "QName", + "SubElement", + "tostring", + "TreeBuilder", + "VERSION", "XML", + "XMLParser", "XMLTreeBuilder", + ] + +## +# The Element type is a flexible container object, designed to +# store hierarchical data structures in memory. The type can be +# described as a cross between a list and a dictionary. +#

+# Each element has a number of properties associated with it: +#

    +#
  • a tag. This is a string identifying what kind of data +# this element represents (the element type, in other words).
    • +#
  • a number of attributes, stored in a Python dictionary.
    • +#
  • a text string.
    • +#
  • an optional tail string.
    • +#
  • a number of child elements, stored in a Python sequence
    • +#
+# +# To create an element instance, use the {@link #Element} or {@link +# #SubElement} factory functions. +#

+# The {@link #ElementTree} class can be used to wrap an element +# structure, and convert it from and to XML. +## + +import string, sys, re + +class _SimpleElementPath: + # emulate pre-1.2 find/findtext/findall behaviour + def find(self, element, tag): + for elem in element: + if elem.tag == tag: + return elem + return None + def findtext(self, element, tag, default=None): + for elem in element: + if elem.tag == tag: + return elem.text or "" + return default + def findall(self, element, tag): + if tag[:3] == ".//": + return element.getiterator(tag[3:]) + result = [] + for elem in element: + if elem.tag == tag: + result.append(elem) + return result + +try: + import ElementPath +except ImportError: + # FIXME: issue warning in this case? + ElementPath = _SimpleElementPath() + +# TODO: add support for custom namespace resolvers/default namespaces +# TODO: add improved support for incremental parsing + +VERSION = "1.2.6" + +## +# Internal element class. This class defines the Element interface, +# and provides a reference implementation of this interface. +#

+# You should not create instances of this class directly. Use the +# appropriate factory functions instead, such as {@link #Element} +# and {@link #SubElement}. +# +# @see Element +# @see SubElement +# @see Comment +# @see ProcessingInstruction + +class _ElementInterface: + # text...tail + + ## + # (Attribute) Element tag. + + tag = None + + ## + # (Attribute) Element attribute dictionary. Where possible, use + # {@link #_ElementInterface.get}, + # {@link #_ElementInterface.set}, + # {@link #_ElementInterface.keys}, and + # {@link #_ElementInterface.items} to access + # element attributes. + + attrib = None + + ## + # (Attribute) Text before first subelement. This is either a + # string or the value None, if there was no text. + + text = None + + ## + # (Attribute) Text after this element's end tag, but before the + # next sibling element's start tag. This is either a string or + # the value None, if there was no text. + + tail = None # text after end tag, if any + + def __init__(self, tag, attrib): + self.tag = tag + self.attrib = attrib + self._children = [] + + def __repr__(self): + return "" % (self.tag, id(self)) + + ## + # Creates a new element object of the same type as this element. + # + # @param tag Element tag. + # @param attrib Element attributes, given as a dictionary. + # @return A new element instance. + + def makeelement(self, tag, attrib): + return Element(tag, attrib) + + ## + # Returns the number of subelements. + # + # @return The number of subelements. + + def __len__(self): + return len(self._children) + + ## + # Returns the given subelement. + # + # @param index What subelement to return. + # @return The given subelement. + # @exception IndexError If the given element does not exist. + + def __getitem__(self, index): + return self._children[index] + + ## + # Replaces the given subelement. + # + # @param index What subelement to replace. + # @param element The new element value. + # @exception IndexError If the given element does not exist. + # @exception AssertionError If element is not a valid object. + + def __setitem__(self, index, element): + assert iselement(element) + self._children[index] = element + + ## + # Deletes the given subelement. + # + # @param index What subelement to delete. + # @exception IndexError If the given element does not exist. + + def __delitem__(self, index): + del self._children[index] + + ## + # Returns a list containing subelements in the given range. + # + # @param start The first subelement to return. + # @param stop The first subelement that shouldn't be returned. + # @return A sequence object containing subelements. + + def __getslice__(self, start, stop): + return self._children[start:stop] + + ## + # Replaces a number of subelements with elements from a sequence. + # + # @param start The first subelement to replace. + # @param stop The first subelement that shouldn't be replaced. + # @param elements A sequence object with zero or more elements. + # @exception AssertionError If a sequence member is not a valid object. + + def __setslice__(self, start, stop, elements): + for element in elements: + assert iselement(element) + self._children[start:stop] = list(elements) + + ## + # Deletes a number of subelements. + # + # @param start The first subelement to delete. + # @param stop The first subelement to leave in there. + + def __delslice__(self, start, stop): + del self._children[start:stop] + + ## + # Adds a subelement to the end of this element. + # + # @param element The element to add. + # @exception AssertionError If a sequence member is not a valid object. + + def append(self, element): + assert iselement(element) + self._children.append(element) + + ## + # Inserts a subelement at the given position in this element. + # + # @param index Where to insert the new subelement. + # @exception AssertionError If the element is not a valid object. + + def insert(self, index, element): + assert iselement(element) + self._children.insert(index, element) + + ## + # Removes a matching subelement. Unlike the find methods, + # this method compares elements based on identity, not on tag + # value or contents. + # + # @param element What element to remove. + # @exception ValueError If a matching element could not be found. + # @exception AssertionError If the element is not a valid object. + + def remove(self, element): + assert iselement(element) + self._children.remove(element) + + ## + # Returns all subelements. The elements are returned in document + # order. + # + # @return A list of subelements. + # @defreturn list of Element instances + + def getchildren(self): + return self._children + + ## + # Finds the first matching subelement, by tag name or path. + # + # @param path What element to look for. + # @return The first matching element, or None if no element was found. + # @defreturn Element or None + + def find(self, path): + return ElementPath.find(self, path) + + ## + # Finds text for the first matching subelement, by tag name or path. + # + # @param path What element to look for. + # @param default What to return if the element was not found. + # @return The text content of the first matching element, or the + # default value no element was found. Note that if the element + # has is found, but has no text content, this method returns an + # empty string. + # @defreturn string + + def findtext(self, path, default=None): + return ElementPath.findtext(self, path, default) + + ## + # Finds all matching subelements, by tag name or path. + # + # @param path What element to look for. + # @return A list or iterator containing all matching elements, + # in document order. + # @defreturn list of Element instances + + def findall(self, path): + return ElementPath.findall(self, path) + + ## + # Resets an element. This function removes all subelements, clears + # all attributes, and sets the text and tail attributes to None. + + def clear(self): + self.attrib.clear() + self._children = [] + self.text = self.tail = None + + ## + # Gets an element attribute. + # + # @param key What attribute to look for. + # @param default What to return if the attribute was not found. + # @return The attribute value, or the default value, if the + # attribute was not found. + # @defreturn string or None + + def get(self, key, default=None): + return self.attrib.get(key, default) + + ## + # Sets an element attribute. + # + # @param key What attribute to set. + # @param value The attribute value. + + def set(self, key, value): + self.attrib[key] = value + + ## + # Gets a list of attribute names. The names are returned in an + # arbitrary order (just like for an ordinary Python dictionary). + # + # @return A list of element attribute names. + # @defreturn list of strings + + def keys(self): + return self.attrib.keys() + + ## + # Gets element attributes, as a sequence. The attributes are + # returned in an arbitrary order. + # + # @return A list of (name, value) tuples for all attributes. + # @defreturn list of (string, string) tuples + + def items(self): + return self.attrib.items() + + ## + # Creates a tree iterator. The iterator loops over this element + # and all subelements, in document order, and returns all elements + # with a matching tag. + #

+ # If the tree structure is modified during iteration, the result + # is undefined. + # + # @param tag What tags to look for (default is to return all elements). + # @return A list or iterator containing all the matching elements. + # @defreturn list or iterator + + def getiterator(self, tag=None): + nodes = [] + if tag == "*": + tag = None + if tag is None or self.tag == tag: + nodes.append(self) + for node in self._children: + nodes.extend(node.getiterator(tag)) + return nodes + +# compatibility +_Element = _ElementInterface + +## +# Element factory. This function returns an object implementing the +# standard Element interface. The exact class or type of that object +# is implementation dependent, but it will always be compatible with +# the {@link #_ElementInterface} class in this module. +#

+# The element name, attribute names, and attribute values can be +# either 8-bit ASCII strings or Unicode strings. +# +# @param tag The element name. +# @param attrib An optional dictionary, containing element attributes. +# @param **extra Additional attributes, given as keyword arguments. +# @return An element instance. +# @defreturn Element + +def Element(tag, attrib={}, **extra): + attrib = attrib.copy() + attrib.update(extra) + return _ElementInterface(tag, attrib) + +## +# Subelement factory. This function creates an element instance, and +# appends it to an existing element. +#

+# The element name, attribute names, and attribute values can be +# either 8-bit ASCII strings or Unicode strings. +# +# @param parent The parent element. +# @param tag The subelement name. +# @param attrib An optional dictionary, containing element attributes. +# @param **extra Additional attributes, given as keyword arguments. +# @return An element instance. +# @defreturn Element + +def SubElement(parent, tag, attrib={}, **extra): + attrib = attrib.copy() + attrib.update(extra) + element = parent.makeelement(tag, attrib) + parent.append(element) + return element + +## +# Comment element factory. This factory function creates a special +# element that will be serialized as an XML comment. +#

+# The comment string can be either an 8-bit ASCII string or a Unicode +# string. +# +# @param text A string containing the comment string. +# @return An element instance, representing a comment. +# @defreturn Element + +def Comment(text=None): + element = Element(Comment) + element.text = text + return element + +## +# PI element factory. This factory function creates a special element +# that will be serialized as an XML processing instruction. +# +# @param target A string containing the PI target. +# @param text A string containing the PI contents, if any. +# @return An element instance, representing a PI. +# @defreturn Element + +def ProcessingInstruction(target, text=None): + element = Element(ProcessingInstruction) + element.text = target + if text: + element.text = element.text + " " + text + return element + +PI = ProcessingInstruction + +## +# QName wrapper. This can be used to wrap a QName attribute value, in +# order to get proper namespace handling on output. +# +# @param text A string containing the QName value, in the form {uri}local, +# or, if the tag argument is given, the URI part of a QName. +# @param tag Optional tag. If given, the first argument is interpreted as +# an URI, and this argument is interpreted as a local name. +# @return An opaque object, representing the QName. + +class QName: + def __init__(self, text_or_uri, tag=None): + if tag: + text_or_uri = "{%s}%s" % (text_or_uri, tag) + self.text = text_or_uri + def __str__(self): + return self.text + def __hash__(self): + return hash(self.text) + def __cmp__(self, other): + if isinstance(other, QName): + return cmp(self.text, other.text) + return cmp(self.text, other) + +## +# ElementTree wrapper class. This class represents an entire element +# hierarchy, and adds some extra support for serialization to and from +# standard XML. +# +# @param element Optional root element. +# @keyparam file Optional file handle or name. If given, the +# tree is initialized with the contents of this XML file. + +class ElementTree: + + def __init__(self, element=None, file=None): + assert element is None or iselement(element) + self._root = element # first node + if file: + self.parse(file) + + ## + # Gets the root element for this tree. + # + # @return An element instance. + # @defreturn Element + + def getroot(self): + return self._root + + ## + # Replaces the root element for this tree. This discards the + # current contents of the tree, and replaces it with the given + # element. Use with care. + # + # @param element An element instance. + + def _setroot(self, element): + assert iselement(element) + self._root = element + + ## + # Loads an external XML document into this element tree. + # + # @param source A file name or file object. + # @param parser An optional parser instance. If not given, the + # standard {@link XMLTreeBuilder} parser is used. + # @return The document root element. + # @defreturn Element + + def parse(self, source, parser=None): + if not hasattr(source, "read"): + source = open(source, "rb") + if not parser: + parser = XMLTreeBuilder() + while 1: + data = source.read(32768) + if not data: + break + parser.feed(data) + self._root = parser.close() + return self._root + + ## + # Creates a tree iterator for the root element. The iterator loops + # over all elements in this tree, in document order. + # + # @param tag What tags to look for (default is to return all elements) + # @return An iterator. + # @defreturn iterator + + def getiterator(self, tag=None): + assert self._root is not None + return self._root.getiterator(tag) + + ## + # Finds the first toplevel element with given tag. + # Same as getroot().find(path). + # + # @param path What element to look for. + # @return The first matching element, or None if no element was found. + # @defreturn Element or None + + def find(self, path): + assert self._root is not None + if path[:1] == "/": + path = "." + path + return self._root.find(path) + + ## + # Finds the element text for the first toplevel element with given + # tag. Same as getroot().findtext(path). + # + # @param path What toplevel element to look for. + # @param default What to return if the element was not found. + # @return The text content of the first matching element, or the + # default value no element was found. Note that if the element + # has is found, but has no text content, this method returns an + # empty string. + # @defreturn string + + def findtext(self, path, default=None): + assert self._root is not None + if path[:1] == "/": + path = "." + path + return self._root.findtext(path, default) + + ## + # Finds all toplevel elements with the given tag. + # Same as getroot().findall(path). + # + # @param path What element to look for. + # @return A list or iterator containing all matching elements, + # in document order. + # @defreturn list of Element instances + + def findall(self, path): + assert self._root is not None + if path[:1] == "/": + path = "." + path + return self._root.findall(path) + + ## + # Writes the element tree to a file, as XML. + # + # @param file A file name, or a file object opened for writing. + # @param encoding Optional output encoding (default is US-ASCII). + + def write(self, file, encoding="us-ascii"): + assert self._root is not None + if not hasattr(file, "write"): + file = open(file, "wb") + if not encoding: + encoding = "us-ascii" + elif encoding != "utf-8" and encoding != "us-ascii": + file.write("\n" % encoding) + self._write(file, self._root, encoding, {}) + + def _write(self, file, node, encoding, namespaces): + # write XML to file + tag = node.tag + if tag is Comment: + file.write("" % _escape_cdata(node.text, encoding)) + elif tag is ProcessingInstruction: + file.write("" % _escape_cdata(node.text, encoding)) + else: + items = node.items() + xmlns_items = [] # new namespaces in this scope + try: + if isinstance(tag, QName) or tag[:1] == "{": + tag, xmlns = fixtag(tag, namespaces) + if xmlns: xmlns_items.append(xmlns) + except TypeError: + _raise_serialization_error(tag) + file.write("<" + _encode(tag, encoding)) + if items or xmlns_items: + items.sort() # lexical order + for k, v in items: + try: + if isinstance(k, QName) or k[:1] == "{": + k, xmlns = fixtag(k, namespaces) + if xmlns: xmlns_items.append(xmlns) + except TypeError: + _raise_serialization_error(k) + try: + if isinstance(v, QName): + v, xmlns = fixtag(v, namespaces) + if xmlns: xmlns_items.append(xmlns) + except TypeError: + _raise_serialization_error(v) + file.write(" %s=\"%s\"" % (_encode(k, encoding), + _escape_attrib(v, encoding))) + for k, v in xmlns_items: + file.write(" %s=\"%s\"" % (_encode(k, encoding), + _escape_attrib(v, encoding))) + if node.text or len(node): + file.write(">") + if node.text: + file.write(_escape_cdata(node.text, encoding)) + for n in node: + self._write(file, n, encoding, namespaces) + file.write("") + else: + file.write(" />") + for k, v in xmlns_items: + del namespaces[v] + if node.tail: + file.write(_escape_cdata(node.tail, encoding)) + +# -------------------------------------------------------------------- +# helpers + +## +# Checks if an object appears to be a valid element object. +# +# @param An element instance. +# @return A true value if this is an element object. +# @defreturn flag + +def iselement(element): + # FIXME: not sure about this; might be a better idea to look + # for tag/attrib/text attributes + return isinstance(element, _ElementInterface) or hasattr(element, "tag") + +## +# Writes an element tree or element structure to sys.stdout. This +# function should be used for debugging only. +#

+# The exact output format is implementation dependent. In this +# version, it's written as an ordinary XML file. +# +# @param elem An element tree or an individual element. + +def dump(elem): + # debugging + if not isinstance(elem, ElementTree): + elem = ElementTree(elem) + elem.write(sys.stdout) + tail = elem.getroot().tail + if not tail or tail[-1] != "\n": + sys.stdout.write("\n") + +def _encode(s, encoding): + try: + return s.encode(encoding) + except AttributeError: + return s # 1.5.2: assume the string uses the right encoding + +if sys.version[:3] == "1.5": + _escape = re.compile(r"[&<>\"\x80-\xff]+") # 1.5.2 +else: + _escape = re.compile(eval(r'u"[&<>\"\u0080-\uffff]+"')) + +_escape_map = { + "&": "&", + "<": "<", + ">": ">", + '"': """, +} + +_namespace_map = { + # "well-known" namespace prefixes + "http://www.w3.org/XML/1998/namespace": "xml", + "http://www.w3.org/1999/xhtml": "html", + "http://www.w3.org/1999/02/22-rdf-syntax-ns#": "rdf", + "http://schemas.xmlsoap.org/wsdl/": "wsdl", +} + +def _raise_serialization_error(text): + raise TypeError( + "cannot serialize %r (type %s)" % (text, type(text).__name__) + ) + +def _encode_entity(text, pattern=_escape): + # map reserved and non-ascii characters to numerical entities + def escape_entities(m, map=_escape_map): + out = [] + append = out.append + for char in m.group(): + text = map.get(char) + if text is None: + text = "&#%d;" % ord(char) + append(text) + return string.join(out, "") + try: + return _encode(pattern.sub(escape_entities, text), "ascii") + except TypeError: + _raise_serialization_error(text) + +# +# the following functions assume an ascii-compatible encoding +# (or "utf-16") + +def _escape_cdata(text, encoding=None, replace=string.replace): + # escape character data + try: + if encoding: + try: + text = _encode(text, encoding) + except UnicodeError: + return _encode_entity(text) + text = replace(text, "&", "&") + text = replace(text, "<", "<") + text = replace(text, ">", ">") + return text + except (TypeError, AttributeError): + _raise_serialization_error(text) + +def _escape_attrib(text, encoding=None, replace=string.replace): + # escape attribute value + try: + if encoding: + try: + text = _encode(text, encoding) + except UnicodeError: + return _encode_entity(text) + text = replace(text, "&", "&") + text = replace(text, "'", "'") # FIXME: overkill + text = replace(text, "\"", """) + text = replace(text, "<", "<") + text = replace(text, ">", ">") + return text + except (TypeError, AttributeError): + _raise_serialization_error(text) + +def fixtag(tag, namespaces): + # given a decorated tag (of the form {uri}tag), return prefixed + # tag and namespace declaration, if any + if isinstance(tag, QName): + tag = tag.text + namespace_uri, tag = string.split(tag[1:], "}", 1) + prefix = namespaces.get(namespace_uri) + if prefix is None: + prefix = _namespace_map.get(namespace_uri) + if prefix is None: + prefix = "ns%d" % len(namespaces) + namespaces[namespace_uri] = prefix + if prefix == "xml": + xmlns = None + else: + xmlns = ("xmlns:%s" % prefix, namespace_uri) + else: + xmlns = None + return "%s:%s" % (prefix, tag), xmlns + +## +# Parses an XML document into an element tree. +# +# @param source A filename or file object containing XML data. +# @param parser An optional parser instance. If not given, the +# standard {@link XMLTreeBuilder} parser is used. +# @return An ElementTree instance + +def parse(source, parser=None): + tree = ElementTree() + tree.parse(source, parser) + return tree + +## +# Parses an XML document into an element tree incrementally, and reports +# what's going on to the user. +# +# @param source A filename or file object containing XML data. +# @param events A list of events to report back. If omitted, only "end" +# events are reported. +# @return A (event, elem) iterator. + +class iterparse: + + def __init__(self, source, events=None): + if not hasattr(source, "read"): + source = open(source, "rb") + self._file = source + self._events = [] + self._index = 0 + self.root = self._root = None + self._parser = XMLTreeBuilder() + # wire up the parser for event reporting + parser = self._parser._parser + append = self._events.append + if events is None: + events = ["end"] + for event in events: + if event == "start": + try: + parser.ordered_attributes = 1 + parser.specified_attributes = 1 + def handler(tag, attrib_in, event=event, append=append, + start=self._parser._start_list): + append((event, start(tag, attrib_in))) + parser.StartElementHandler = handler + except AttributeError: + def handler(tag, attrib_in, event=event, append=append, + start=self._parser._start): + append((event, start(tag, attrib_in))) + parser.StartElementHandler = handler + elif event == "end": + def handler(tag, event=event, append=append, + end=self._parser._end): + append((event, end(tag))) + parser.EndElementHandler = handler + elif event == "start-ns": + def handler(prefix, uri, event=event, append=append): + try: + uri = _encode(uri, "ascii") + except UnicodeError: + pass + append((event, (prefix or "", uri))) + parser.StartNamespaceDeclHandler = handler + elif event == "end-ns": + def handler(prefix, event=event, append=append): + append((event, None)) + parser.EndNamespaceDeclHandler = handler + + def next(self): + while 1: + try: + item = self._events[self._index] + except IndexError: + if self._parser is None: + self.root = self._root + try: + raise StopIteration + except NameError: + raise IndexError + # load event buffer + del self._events[:] + self._index = 0 + data = self._file.read(16384) + if data: + self._parser.feed(data) + else: + self._root = self._parser.close() + self._parser = None + else: + self._index = self._index + 1 + return item + + try: + iter + def __iter__(self): + return self + except NameError: + def __getitem__(self, index): + return self.next() + +## +# Parses an XML document from a string constant. This function can +# be used to embed "XML literals" in Python code. +# +# @param source A string containing XML data. +# @return An Element instance. +# @defreturn Element + +def XML(text): + parser = XMLTreeBuilder() + parser.feed(text) + return parser.close() + +## +# Parses an XML document from a string constant, and also returns +# a dictionary which maps from element id:s to elements. +# +# @param source A string containing XML data. +# @return A tuple containing an Element instance and a dictionary. +# @defreturn (Element, dictionary) + +def XMLID(text): + parser = XMLTreeBuilder() + parser.feed(text) + tree = parser.close() + ids = {} + for elem in tree.getiterator(): + id = elem.get("id") + if id: + ids[id] = elem + return tree, ids + +## +# Parses an XML document from a string constant. Same as {@link #XML}. +# +# @def fromstring(text) +# @param source A string containing XML data. +# @return An Element instance. +# @defreturn Element + +fromstring = XML + +## +# Generates a string representation of an XML element, including all +# subelements. +# +# @param element An Element instance. +# @return An encoded string containing the XML data. +# @defreturn string + +def tostring(element, encoding=None): + class dummy: + pass + data = [] + file = dummy() + file.write = data.append + ElementTree(element).write(file, encoding) + return string.join(data, "") + +## +# Generic element structure builder. This builder converts a sequence +# of {@link #TreeBuilder.start}, {@link #TreeBuilder.data}, and {@link +# #TreeBuilder.end} method calls to a well-formed element structure. +#

+# You can use this class to build an element structure using a custom XML +# parser, or a parser for some other XML-like format. +# +# @param element_factory Optional element factory. This factory +# is called to create new Element instances, as necessary. + +class TreeBuilder: + + def __init__(self, element_factory=None): + self._data = [] # data collector + self._elem = [] # element stack + self._last = None # last element + self._tail = None # true if we're after an end tag + if element_factory is None: + element_factory = _ElementInterface + self._factory = element_factory + + ## + # Flushes the parser buffers, and returns the toplevel documen + # element. + # + # @return An Element instance. + # @defreturn Element + + def close(self): + assert len(self._elem) == 0, "missing end tags" + assert self._last != None, "missing toplevel element" + return self._last + + def _flush(self): + if self._data: + if self._last is not None: + text = string.join(self._data, "") + if self._tail: + assert self._last.tail is None, "internal error (tail)" + self._last.tail = text + else: + assert self._last.text is None, "internal error (text)" + self._last.text = text + self._data = [] + + ## + # Adds text to the current element. + # + # @param data A string. This should be either an 8-bit string + # containing ASCII text, or a Unicode string. + + def data(self, data): + self._data.append(data) + + ## + # Opens a new element. + # + # @param tag The element name. + # @param attrib A dictionary containing element attributes. + # @return The opened element. + # @defreturn Element + + def start(self, tag, attrs): + self._flush() + self._last = elem = self._factory(tag, attrs) + if self._elem: + self._elem[-1].append(elem) + self._elem.append(elem) + self._tail = 0 + return elem + + ## + # Closes the current element. + # + # @param tag The element name. + # @return The closed element. + # @defreturn Element + + def end(self, tag): + self._flush() + self._last = self._elem.pop() + assert self._last.tag == tag,\ + "end tag mismatch (expected %s, got %s)" % ( + self._last.tag, tag) + self._tail = 1 + return self._last + +## +# Element structure builder for XML source data, based on the +# expat parser. +# +# @keyparam target Target object. If omitted, the builder uses an +# instance of the standard {@link #TreeBuilder} class. +# @keyparam html Predefine HTML entities. This flag is not supported +# by the current implementation. +# @see #ElementTree +# @see #TreeBuilder + +class XMLTreeBuilder: + + def __init__(self, html=0, target=None): + try: + from xml.parsers import expat + except ImportError: + raise ImportError( + "No module named expat; use SimpleXMLTreeBuilder instead" + ) + self._parser = parser = expat.ParserCreate(None, "}") + if target is None: + target = TreeBuilder() + self._target = target + self._names = {} # name memo cache + # callbacks + parser.DefaultHandlerExpand = self._default + parser.StartElementHandler = self._start + parser.EndElementHandler = self._end + parser.CharacterDataHandler = self._data + # let expat do the buffering, if supported + try: + self._parser.buffer_text = 1 + except AttributeError: + pass + # use new-style attribute handling, if supported + try: + self._parser.ordered_attributes = 1 + self._parser.specified_attributes = 1 + parser.StartElementHandler = self._start_list + except AttributeError: + pass + encoding = None + if not parser.returns_unicode: + encoding = "utf-8" + # target.xml(encoding, None) + self._doctype = None + self.entity = {} + + def _fixtext(self, text): + # convert text string to ascii, if possible + try: + return _encode(text, "ascii") + except UnicodeError: + return text + + def _fixname(self, key): + # expand qname, and convert name string to ascii, if possible + try: + name = self._names[key] + except KeyError: + name = key + if "}" in name: + name = "{" + name + self._names[key] = name = self._fixtext(name) + return name + + def _start(self, tag, attrib_in): + fixname = self._fixname + tag = fixname(tag) + attrib = {} + for key, value in attrib_in.items(): + attrib[fixname(key)] = self._fixtext(value) + return self._target.start(tag, attrib) + + def _start_list(self, tag, attrib_in): + fixname = self._fixname + tag = fixname(tag) + attrib = {} + if attrib_in: + for i in range(0, len(attrib_in), 2): + attrib[fixname(attrib_in[i])] = self._fixtext(attrib_in[i+1]) + return self._target.start(tag, attrib) + + def _data(self, text): + return self._target.data(self._fixtext(text)) + + def _end(self, tag): + return self._target.end(self._fixname(tag)) + + def _default(self, text): + prefix = text[:1] + if prefix == "&": + # deal with undefined entities + try: + self._target.data(self.entity[text[1:-1]]) + except KeyError: + from xml.parsers import expat + raise expat.error( + "undefined entity %s: line %d, column %d" % + (text, self._parser.ErrorLineNumber, + self._parser.ErrorColumnNumber) + ) + elif prefix == "<" and text[:9] == "": + self._doctype = None + return + text = string.strip(text) + if not text: + return + self._doctype.append(text) + n = len(self._doctype) + if n > 2: + type = self._doctype[1] + if type == "PUBLIC" and n == 4: + name, type, pubid, system = self._doctype + elif type == "SYSTEM" and n == 3: + name, type, system = self._doctype + pubid = None + else: + return + if pubid: + pubid = pubid[1:-1] + self.doctype(name, pubid, system[1:-1]) + self._doctype = None + + ## + # Handles a doctype declaration. + # + # @param name Doctype name. + # @param pubid Public identifier. + # @param system System identifier. + + def doctype(self, name, pubid, system): + pass + + ## + # Feeds data to the parser. + # + # @param data Encoded data. + + def feed(self, data): + self._parser.Parse(data, 0) + + ## + # Finishes feeding data to the parser. + # + # @return An element structure. + # @defreturn Element + + def close(self): + self._parser.Parse("", 1) # end of data + tree = self._target.close() + del self._target, self._parser # get rid of circular references + return tree + +# compatibility +XMLParser = XMLTreeBuilder diff --git a/Lib/xml/etree/__init__.py b/Lib/xml/etree/__init__.py new file mode 100644 index 0000000..3dd2c92 --- /dev/null +++ b/Lib/xml/etree/__init__.py @@ -0,0 +1,33 @@ +# $Id: __init__.py 1821 2004-06-03 16:57:49Z fredrik $ +# elementtree package + +# -------------------------------------------------------------------- +# The ElementTree toolkit is +# +# Copyright (c) 1999-2004 by Fredrik Lundh +# +# By obtaining, using, and/or copying this software and/or its +# associated documentation, you agree that you have read, understood, +# and will comply with the following terms and conditions: +# +# Permission to use, copy, modify, and distribute this software and +# its associated documentation for any purpose and without fee is +# hereby granted, provided that the above copyright notice appears in +# all copies, and that both that copyright notice and this permission +# notice appear in supporting documentation, and that the name of +# Secret Labs AB or the author not be used in advertising or publicity +# pertaining to distribution of the software without specific, written +# prior permission. +# +# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD +# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- +# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR +# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY +# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, +# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS +# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE +# OF THIS SOFTWARE. +# -------------------------------------------------------------------- + +# Licensed to PSF under a Contributor Agreement. +# See http://www.python.org/2.4/license for licensing details. diff --git a/Lib/xml/etree/cElementTree.py b/Lib/xml/etree/cElementTree.py new file mode 100644 index 0000000..a6f127a --- /dev/null +++ b/Lib/xml/etree/cElementTree.py @@ -0,0 +1,3 @@ +# Wrapper module for _elementtree + +from _elementtree import * diff --git a/Lib/xml/parsers/__init__.py b/Lib/xml/parsers/__init__.py new file mode 100644 index 0000000..eb314a3 --- /dev/null +++ b/Lib/xml/parsers/__init__.py @@ -0,0 +1,8 @@ +"""Python interfaces to XML parsers. + +This package contains one module: + +expat -- Python wrapper for James Clark's Expat parser, with namespace + support. + +""" diff --git a/Lib/xml/parsers/expat.py b/Lib/xml/parsers/expat.py new file mode 100644 index 0000000..11359a0 --- /dev/null +++ b/Lib/xml/parsers/expat.py @@ -0,0 +1,4 @@ +"""Interface to the Expat non-validating XML parser.""" +__version__ = '$Revision$' + +from pyexpat import * diff --git a/Lib/xml/sax/__init__.py b/Lib/xml/sax/__init__.py new file mode 100644 index 0000000..6b1b1ba --- /dev/null +++ b/Lib/xml/sax/__init__.py @@ -0,0 +1,108 @@ +"""Simple API for XML (SAX) implementation for Python. + +This module provides an implementation of the SAX 2 interface; +information about the Java version of the interface can be found at +http://www.megginson.com/SAX/. The Python version of the interface is +documented at <...>. + +This package contains the following modules: + +handler -- Base classes and constants which define the SAX 2 API for + the 'client-side' of SAX for Python. + +saxutils -- Implementation of the convenience classes commonly used to + work with SAX. + +xmlreader -- Base classes and constants which define the SAX 2 API for + the parsers used with SAX for Python. + +expatreader -- Driver that allows use of the Expat parser with SAX. +""" + +from xmlreader import InputSource +from handler import ContentHandler, ErrorHandler +from _exceptions import SAXException, SAXNotRecognizedException, \ + SAXParseException, SAXNotSupportedException, \ + SAXReaderNotAvailable + + +def parse(source, handler, errorHandler=ErrorHandler()): + parser = make_parser() + parser.setContentHandler(handler) + parser.setErrorHandler(errorHandler) + parser.parse(source) + +def parseString(string, handler, errorHandler=ErrorHandler()): + try: + from cStringIO import StringIO + except ImportError: + from StringIO import StringIO + + if errorHandler is None: + errorHandler = ErrorHandler() + parser = make_parser() + parser.setContentHandler(handler) + parser.setErrorHandler(errorHandler) + + inpsrc = InputSource() + inpsrc.setByteStream(StringIO(string)) + parser.parse(inpsrc) + +# this is the parser list used by the make_parser function if no +# alternatives are given as parameters to the function + +default_parser_list = ["xml.sax.expatreader"] + +# tell modulefinder that importing sax potentially imports expatreader +_false = 0 +if _false: + import xml.sax.expatreader + +import os, sys +if os.environ.has_key("PY_SAX_PARSER"): + default_parser_list = os.environ["PY_SAX_PARSER"].split(",") +del os + +_key = "python.xml.sax.parser" +if sys.platform[:4] == "java" and sys.registry.containsKey(_key): + default_parser_list = sys.registry.getProperty(_key).split(",") + + +def make_parser(parser_list = []): + """Creates and returns a SAX parser. + + Creates the first parser it is able to instantiate of the ones + given in the list created by doing parser_list + + default_parser_list. The lists must contain the names of Python + modules containing both a SAX parser and a create_parser function.""" + + for parser_name in parser_list + default_parser_list: + try: + return _create_parser(parser_name) + except ImportError,e: + import sys + if sys.modules.has_key(parser_name): + # The parser module was found, but importing it + # failed unexpectedly, pass this exception through + raise + except SAXReaderNotAvailable: + # The parser module detected that it won't work properly, + # so try the next one + pass + + raise SAXReaderNotAvailable("No parsers found", None) + +# --- Internal utility methods used by make_parser + +if sys.platform[ : 4] == "java": + def _create_parser(parser_name): + from org.python.core import imp + drv_module = imp.importName(parser_name, 0, globals()) + return drv_module.create_parser() + +else: + def _create_parser(parser_name): + drv_module = __import__(parser_name,{},{},['create_parser']) + return drv_module.create_parser() + +del sys diff --git a/Lib/xml/sax/_exceptions.py b/Lib/xml/sax/_exceptions.py new file mode 100644 index 0000000..fdd614a --- /dev/null +++ b/Lib/xml/sax/_exceptions.py @@ -0,0 +1,131 @@ +"""Different kinds of SAX Exceptions""" +import sys +if sys.platform[:4] == "java": + from java.lang import Exception +del sys + +# ===== SAXEXCEPTION ===== + +class SAXException(Exception): + """Encapsulate an XML error or warning. This class can contain + basic error or warning information from either the XML parser or + the application: you can subclass it to provide additional + functionality, or to add localization. Note that although you will + receive a SAXException as the argument to the handlers in the + ErrorHandler interface, you are not actually required to throw + the exception; instead, you can simply read the information in + it.""" + + def __init__(self, msg, exception=None): + """Creates an exception. The message is required, but the exception + is optional.""" + self._msg = msg + self._exception = exception + Exception.__init__(self, msg) + + def getMessage(self): + "Return a message for this exception." + return self._msg + + def getException(self): + "Return the embedded exception, or None if there was none." + return self._exception + + def __str__(self): + "Create a string representation of the exception." + return self._msg + + def __getitem__(self, ix): + """Avoids weird error messages if someone does exception[ix] by + mistake, since Exception has __getitem__ defined.""" + raise AttributeError("__getitem__") + + +# ===== SAXPARSEEXCEPTION ===== + +class SAXParseException(SAXException): + """Encapsulate an XML parse error or warning. + + This exception will include information for locating the error in + the original XML document. Note that although the application will + receive a SAXParseException as the argument to the handlers in the + ErrorHandler interface, the application is not actually required + to throw the exception; instead, it can simply read the + information in it and take a different action. + + Since this exception is a subclass of SAXException, it inherits + the ability to wrap another exception.""" + + def __init__(self, msg, exception, locator): + "Creates the exception. The exception parameter is allowed to be None." + SAXException.__init__(self, msg, exception) + self._locator = locator + + # We need to cache this stuff at construction time. + # If this exception is thrown, the objects through which we must + # traverse to get this information may be deleted by the time + # it gets caught. + self._systemId = self._locator.getSystemId() + self._colnum = self._locator.getColumnNumber() + self._linenum = self._locator.getLineNumber() + + def getColumnNumber(self): + """The column number of the end of the text where the exception + occurred.""" + return self._colnum + + def getLineNumber(self): + "The line number of the end of the text where the exception occurred." + return self._linenum + + def getPublicId(self): + "Get the public identifier of the entity where the exception occurred." + return self._locator.getPublicId() + + def getSystemId(self): + "Get the system identifier of the entity where the exception occurred." + return self._systemId + + def __str__(self): + "Create a string representation of the exception." + sysid = self.getSystemId() + if sysid is None: + sysid = "" + linenum = self.getLineNumber() + if linenum is None: + linenum = "?" + colnum = self.getColumnNumber() + if colnum is None: + colnum = "?" + return "%s:%s:%s: %s" % (sysid, linenum, colnum, self._msg) + + +# ===== SAXNOTRECOGNIZEDEXCEPTION ===== + +class SAXNotRecognizedException(SAXException): + """Exception class for an unrecognized identifier. + + An XMLReader will raise this exception when it is confronted with an + unrecognized feature or property. SAX applications and extensions may + use this class for similar purposes.""" + + +# ===== SAXNOTSUPPORTEDEXCEPTION ===== + +class SAXNotSupportedException(SAXException): + """Exception class for an unsupported operation. + + An XMLReader will raise this exception when a service it cannot + perform is requested (specifically setting a state or value). SAX + applications and extensions may use this class for similar + purposes.""" + +# ===== SAXNOTSUPPORTEDEXCEPTION ===== + +class SAXReaderNotAvailable(SAXNotSupportedException): + """Exception class for a missing driver. + + An XMLReader module (driver) should raise this exception when it + is first imported, e.g. when a support module cannot be imported. + It also may be raised during parsing, e.g. if executing an external + program is not permitted.""" diff --git a/Lib/xml/sax/expatreader.py b/Lib/xml/sax/expatreader.py new file mode 100644 index 0000000..bb9c294 --- /dev/null +++ b/Lib/xml/sax/expatreader.py @@ -0,0 +1,414 @@ +""" +SAX driver for the pyexpat C module. This driver works with +pyexpat.__version__ == '2.22'. +""" + +version = "0.20" + +from xml.sax._exceptions import * +from xml.sax.handler import feature_validation, feature_namespaces +from xml.sax.handler import feature_namespace_prefixes +from xml.sax.handler import feature_external_ges, feature_external_pes +from xml.sax.handler import feature_string_interning +from xml.sax.handler import property_xml_string, property_interning_dict + +# xml.parsers.expat does not raise ImportError in Jython +import sys +if sys.platform[:4] == "java": + raise SAXReaderNotAvailable("expat not available in Java", None) +del sys + +try: + from xml.parsers import expat +except ImportError: + raise SAXReaderNotAvailable("expat not supported", None) +else: + if not hasattr(expat, "ParserCreate"): + raise SAXReaderNotAvailable("expat not supported", None) +from xml.sax import xmlreader, saxutils, handler + +AttributesImpl = xmlreader.AttributesImpl +AttributesNSImpl = xmlreader.AttributesNSImpl + +# If we're using a sufficiently recent version of Python, we can use +# weak references to avoid cycles between the parser and content +# handler, otherwise we'll just have to pretend. +try: + import _weakref +except ImportError: + def _mkproxy(o): + return o +else: + import weakref + _mkproxy = weakref.proxy + del weakref, _weakref + +# --- ExpatLocator + +class ExpatLocator(xmlreader.Locator): + """Locator for use with the ExpatParser class. + + This uses a weak reference to the parser object to avoid creating + a circular reference between the parser and the content handler. + """ + def __init__(self, parser): + self._ref = _mkproxy(parser) + + def getColumnNumber(self): + parser = self._ref + if parser._parser is None: + return None + return parser._parser.ErrorColumnNumber + + def getLineNumber(self): + parser = self._ref + if parser._parser is None: + return 1 + return parser._parser.ErrorLineNumber + + def getPublicId(self): + parser = self._ref + if parser is None: + return None + return parser._source.getPublicId() + + def getSystemId(self): + parser = self._ref + if parser is None: + return None + return parser._source.getSystemId() + + +# --- ExpatParser + +class ExpatParser(xmlreader.IncrementalParser, xmlreader.Locator): + """SAX driver for the pyexpat C module.""" + + def __init__(self, namespaceHandling=0, bufsize=2**16-20): + xmlreader.IncrementalParser.__init__(self, bufsize) + self._source = xmlreader.InputSource() + self._parser = None + self._namespaces = namespaceHandling + self._lex_handler_prop = None + self._parsing = 0 + self._entity_stack = [] + self._external_ges = 1 + self._interning = None + + # XMLReader methods + + def parse(self, source): + "Parse an XML document from a URL or an InputSource." + source = saxutils.prepare_input_source(source) + + self._source = source + self.reset() + self._cont_handler.setDocumentLocator(ExpatLocator(self)) + xmlreader.IncrementalParser.parse(self, source) + + def prepareParser(self, source): + if source.getSystemId() != None: + self._parser.SetBase(source.getSystemId()) + + # Redefined setContentHandler to allow changing handlers during parsing + + def setContentHandler(self, handler): + xmlreader.IncrementalParser.setContentHandler(self, handler) + if self._parsing: + self._reset_cont_handler() + + def getFeature(self, name): + if name == feature_namespaces: + return self._namespaces + elif name == feature_string_interning: + return self._interning is not None + elif name in (feature_validation, feature_external_pes, + feature_namespace_prefixes): + return 0 + elif name == feature_external_ges: + return self._external_ges + raise SAXNotRecognizedException("Feature '%s' not recognized" % name) + + def setFeature(self, name, state): + if self._parsing: + raise SAXNotSupportedException("Cannot set features while parsing") + + if name == feature_namespaces: + self._namespaces = state + elif name == feature_external_ges: + self._external_ges = state + elif name == feature_string_interning: + if state: + if self._interning is None: + self._interning = {} + else: + self._interning = None + elif name == feature_validation: + if state: + raise SAXNotSupportedException( + "expat does not support validation") + elif name == feature_external_pes: + if state: + raise SAXNotSupportedException( + "expat does not read external parameter entities") + elif name == feature_namespace_prefixes: + if state: + raise SAXNotSupportedException( + "expat does not report namespace prefixes") + else: + raise SAXNotRecognizedException( + "Feature '%s' not recognized" % name) + + def getProperty(self, name): + if name == handler.property_lexical_handler: + return self._lex_handler_prop + elif name == property_interning_dict: + return self._interning + elif name == property_xml_string: + if self._parser: + if hasattr(self._parser, "GetInputContext"): + return self._parser.GetInputContext() + else: + raise SAXNotRecognizedException( + "This version of expat does not support getting" + " the XML string") + else: + raise SAXNotSupportedException( + "XML string cannot be returned when not parsing") + raise SAXNotRecognizedException("Property '%s' not recognized" % name) + + def setProperty(self, name, value): + if name == handler.property_lexical_handler: + self._lex_handler_prop = value + if self._parsing: + self._reset_lex_handler_prop() + elif name == property_interning_dict: + self._interning = value + elif name == property_xml_string: + raise SAXNotSupportedException("Property '%s' cannot be set" % + name) + else: + raise SAXNotRecognizedException("Property '%s' not recognized" % + name) + + # IncrementalParser methods + + def feed(self, data, isFinal = 0): + if not self._parsing: + self.reset() + self._parsing = 1 + self._cont_handler.startDocument() + + try: + # The isFinal parameter is internal to the expat reader. + # If it is set to true, expat will check validity of the entire + # document. When feeding chunks, they are not normally final - + # except when invoked from close. + self._parser.Parse(data, isFinal) + except expat.error, e: + exc = SAXParseException(expat.ErrorString(e.code), e, self) + # FIXME: when to invoke error()? + self._err_handler.fatalError(exc) + + def close(self): + if self._entity_stack: + # If we are completing an external entity, do nothing here + return + self.feed("", isFinal = 1) + self._cont_handler.endDocument() + self._parsing = 0 + # break cycle created by expat handlers pointing to our methods + self._parser = None + + def _reset_cont_handler(self): + self._parser.ProcessingInstructionHandler = \ + self._cont_handler.processingInstruction + self._parser.CharacterDataHandler = self._cont_handler.characters + + def _reset_lex_handler_prop(self): + lex = self._lex_handler_prop + parser = self._parser + if lex is None: + parser.CommentHandler = None + parser.StartCdataSectionHandler = None + parser.EndCdataSectionHandler = None + parser.StartDoctypeDeclHandler = None + parser.EndDoctypeDeclHandler = None + else: + parser.CommentHandler = lex.comment + parser.StartCdataSectionHandler = lex.startCDATA + parser.EndCdataSectionHandler = lex.endCDATA + parser.StartDoctypeDeclHandler = self.start_doctype_decl + parser.EndDoctypeDeclHandler = lex.endDTD + + def reset(self): + if self._namespaces: + self._parser = expat.ParserCreate(self._source.getEncoding(), " ", + intern=self._interning) + self._parser.namespace_prefixes = 1 + self._parser.StartElementHandler = self.start_element_ns + self._parser.EndElementHandler = self.end_element_ns + else: + self._parser = expat.ParserCreate(self._source.getEncoding(), + intern = self._interning) + self._parser.StartElementHandler = self.start_element + self._parser.EndElementHandler = self.end_element + + self._reset_cont_handler() + self._parser.UnparsedEntityDeclHandler = self.unparsed_entity_decl + self._parser.NotationDeclHandler = self.notation_decl + self._parser.StartNamespaceDeclHandler = self.start_namespace_decl + self._parser.EndNamespaceDeclHandler = self.end_namespace_decl + + self._decl_handler_prop = None + if self._lex_handler_prop: + self._reset_lex_handler_prop() +# self._parser.DefaultHandler = +# self._parser.DefaultHandlerExpand = +# self._parser.NotStandaloneHandler = + self._parser.ExternalEntityRefHandler = self.external_entity_ref + try: + self._parser.SkippedEntityHandler = self.skipped_entity_handler + except AttributeError: + # This pyexpat does not support SkippedEntity + pass + self._parser.SetParamEntityParsing( + expat.XML_PARAM_ENTITY_PARSING_UNLESS_STANDALONE) + + self._parsing = 0 + self._entity_stack = [] + + # Locator methods + + def getColumnNumber(self): + if self._parser is None: + return None + return self._parser.ErrorColumnNumber + + def getLineNumber(self): + if self._parser is None: + return 1 + return self._parser.ErrorLineNumber + + def getPublicId(self): + return self._source.getPublicId() + + def getSystemId(self): + return self._source.getSystemId() + + # event handlers + def start_element(self, name, attrs): + self._cont_handler.startElement(name, AttributesImpl(attrs)) + + def end_element(self, name): + self._cont_handler.endElement(name) + + def start_element_ns(self, name, attrs): + pair = name.split() + if len(pair) == 1: + # no namespace + pair = (None, name) + elif len(pair) == 3: + pair = pair[0], pair[1] + else: + # default namespace + pair = tuple(pair) + + newattrs = {} + qnames = {} + for (aname, value) in attrs.items(): + parts = aname.split() + length = len(parts) + if length == 1: + # no namespace + qname = aname + apair = (None, aname) + elif length == 3: + qname = "%s:%s" % (parts[2], parts[1]) + apair = parts[0], parts[1] + else: + # default namespace + qname = parts[1] + apair = tuple(parts) + + newattrs[apair] = value + qnames[apair] = qname + + self._cont_handler.startElementNS(pair, None, + AttributesNSImpl(newattrs, qnames)) + + def end_element_ns(self, name): + pair = name.split() + if len(pair) == 1: + pair = (None, name) + elif len(pair) == 3: + pair = pair[0], pair[1] + else: + pair = tuple(pair) + + self._cont_handler.endElementNS(pair, None) + + # this is not used (call directly to ContentHandler) + def processing_instruction(self, target, data): + self._cont_handler.processingInstruction(target, data) + + # this is not used (call directly to ContentHandler) + def character_data(self, data): + self._cont_handler.characters(data) + + def start_namespace_decl(self, prefix, uri): + self._cont_handler.startPrefixMapping(prefix, uri) + + def end_namespace_decl(self, prefix): + self._cont_handler.endPrefixMapping(prefix) + + def start_doctype_decl(self, name, sysid, pubid, has_internal_subset): + self._lex_handler_prop.startDTD(name, pubid, sysid) + + def unparsed_entity_decl(self, name, base, sysid, pubid, notation_name): + self._dtd_handler.unparsedEntityDecl(name, pubid, sysid, notation_name) + + def notation_decl(self, name, base, sysid, pubid): + self._dtd_handler.notationDecl(name, pubid, sysid) + + def external_entity_ref(self, context, base, sysid, pubid): + if not self._external_ges: + return 1 + + source = self._ent_handler.resolveEntity(pubid, sysid) + source = saxutils.prepare_input_source(source, + self._source.getSystemId() or + "") + + self._entity_stack.append((self._parser, self._source)) + self._parser = self._parser.ExternalEntityParserCreate(context) + self._source = source + + try: + xmlreader.IncrementalParser.parse(self, source) + except: + return 0 # FIXME: save error info here? + + (self._parser, self._source) = self._entity_stack[-1] + del self._entity_stack[-1] + return 1 + + def skipped_entity_handler(self, name, is_pe): + if is_pe: + # The SAX spec requires to report skipped PEs with a '%' + name = '%'+name + self._cont_handler.skippedEntity(name) + +# --- + +def create_parser(*args, **kwargs): + return ExpatParser(*args, **kwargs) + +# --- + +if __name__ == "__main__": + import xml.sax + p = create_parser() + p.setContentHandler(xml.sax.XMLGenerator()) + p.setErrorHandler(xml.sax.ErrorHandler()) + p.parse("../../../hamlet.xml") diff --git a/Lib/xml/sax/handler.py b/Lib/xml/sax/handler.py new file mode 100644 index 0000000..f9e91b6 --- /dev/null +++ b/Lib/xml/sax/handler.py @@ -0,0 +1,342 @@ +""" +This module contains the core classes of version 2.0 of SAX for Python. +This file provides only default classes with absolutely minimum +functionality, from which drivers and applications can be subclassed. + +Many of these classes are empty and are included only as documentation +of the interfaces. + +$Id$ +""" + +version = '2.0beta' + +#============================================================================ +# +# HANDLER INTERFACES +# +#============================================================================ + +# ===== ERRORHANDLER ===== + +class ErrorHandler: + """Basic interface for SAX error handlers. + + If you create an object that implements this interface, then + register the object with your XMLReader, the parser will call the + methods in your object to report all warnings and errors. There + are three levels of errors available: warnings, (possibly) + recoverable errors, and unrecoverable errors. All methods take a + SAXParseException as the only parameter.""" + + def error(self, exception): + "Handle a recoverable error." + raise exception + + def fatalError(self, exception): + "Handle a non-recoverable error." + raise exception + + def warning(self, exception): + "Handle a warning." + print exception + + +# ===== CONTENTHANDLER ===== + +class ContentHandler: + """Interface for receiving logical document content events. + + This is the main callback interface in SAX, and the one most + important to applications. The order of events in this interface + mirrors the order of the information in the document.""" + + def __init__(self): + self._locator = None + + def setDocumentLocator(self, locator): + """Called by the parser to give the application a locator for + locating the origin of document events. + + SAX parsers are strongly encouraged (though not absolutely + required) to supply a locator: if it does so, it must supply + the locator to the application by invoking this method before + invoking any of the other methods in the DocumentHandler + interface. + + The locator allows the application to determine the end + position of any document-related event, even if the parser is + not reporting an error. Typically, the application will use + this information for reporting its own errors (such as + character content that does not match an application's + business rules). The information returned by the locator is + probably not sufficient for use with a search engine. + + Note that the locator will return correct information only + during the invocation of the events in this interface. The + application should not attempt to use it at any other time.""" + self._locator = locator + + def startDocument(self): + """Receive notification of the beginning of a document. + + The SAX parser will invoke this method only once, before any + other methods in this interface or in DTDHandler (except for + setDocumentLocator).""" + + def endDocument(self): + """Receive notification of the end of a document. + + The SAX parser will invoke this method only once, and it will + be the last method invoked during the parse. The parser shall + not invoke this method until it has either abandoned parsing + (because of an unrecoverable error) or reached the end of + input.""" + + def startPrefixMapping(self, prefix, uri): + """Begin the scope of a prefix-URI Namespace mapping. + + The information from this event is not necessary for normal + Namespace processing: the SAX XML reader will automatically + replace prefixes for element and attribute names when the + http://xml.org/sax/features/namespaces feature is true (the + default). + + There are cases, however, when applications need to use + prefixes in character data or in attribute values, where they + cannot safely be expanded automatically; the + start/endPrefixMapping event supplies the information to the + application to expand prefixes in those contexts itself, if + necessary. + + Note that start/endPrefixMapping events are not guaranteed to + be properly nested relative to each-other: all + startPrefixMapping events will occur before the corresponding + startElement event, and all endPrefixMapping events will occur + after the corresponding endElement event, but their order is + not guaranteed.""" + + def endPrefixMapping(self, prefix): + """End the scope of a prefix-URI mapping. + + See startPrefixMapping for details. This event will always + occur after the corresponding endElement event, but the order + of endPrefixMapping events is not otherwise guaranteed.""" + + def startElement(self, name, attrs): + """Signals the start of an element in non-namespace mode. + + The name parameter contains the raw XML 1.0 name of the + element type as a string and the attrs parameter holds an + instance of the Attributes class containing the attributes of + the element.""" + + def endElement(self, name): + """Signals the end of an element in non-namespace mode. + + The name parameter contains the name of the element type, just + as with the startElement event.""" + + def startElementNS(self, name, qname, attrs): + """Signals the start of an element in namespace mode. + + The name parameter contains the name of the element type as a + (uri, localname) tuple, the qname parameter the raw XML 1.0 + name used in the source document, and the attrs parameter + holds an instance of the Attributes class containing the + attributes of the element. + + The uri part of the name tuple is None for elements which have + no namespace.""" + + def endElementNS(self, name, qname): + """Signals the end of an element in namespace mode. + + The name parameter contains the name of the element type, just + as with the startElementNS event.""" + + def characters(self, content): + """Receive notification of character data. + + The Parser will call this method to report each chunk of + character data. SAX parsers may return all contiguous + character data in a single chunk, or they may split it into + several chunks; however, all of the characters in any single + event must come from the same external entity so that the + Locator provides useful information.""" + + def ignorableWhitespace(self, whitespace): + """Receive notification of ignorable whitespace in element content. + + Validating Parsers must use this method to report each chunk + of ignorable whitespace (see the W3C XML 1.0 recommendation, + section 2.10): non-validating parsers may also use this method + if they are capable of parsing and using content models. + + SAX parsers may return all contiguous whitespace in a single + chunk, or they may split it into several chunks; however, all + of the characters in any single event must come from the same + external entity, so that the Locator provides useful + information.""" + + def processingInstruction(self, target, data): + """Receive notification of a processing instruction. + + The Parser will invoke this method once for each processing + instruction found: note that processing instructions may occur + before or after the main document element. + + A SAX parser should never report an XML declaration (XML 1.0, + section 2.8) or a text declaration (XML 1.0, section 4.3.1) + using this method.""" + + def skippedEntity(self, name): + """Receive notification of a skipped entity. + + The Parser will invoke this method once for each entity + skipped. Non-validating processors may skip entities if they + have not seen the declarations (because, for example, the + entity was declared in an external DTD subset). All processors + may skip external entities, depending on the values of the + http://xml.org/sax/features/external-general-entities and the + http://xml.org/sax/features/external-parameter-entities + properties.""" + + +# ===== DTDHandler ===== + +class DTDHandler: + """Handle DTD events. + + This interface specifies only those DTD events required for basic + parsing (unparsed entities and attributes).""" + + def notationDecl(self, name, publicId, systemId): + "Handle a notation declaration event." + + def unparsedEntityDecl(self, name, publicId, systemId, ndata): + "Handle an unparsed entity declaration event." + + +# ===== ENTITYRESOLVER ===== + +class EntityResolver: + """Basic interface for resolving entities. If you create an object + implementing this interface, then register the object with your + Parser, the parser will call the method in your object to + resolve all external entities. Note that DefaultHandler implements + this interface with the default behaviour.""" + + def resolveEntity(self, publicId, systemId): + """Resolve the system identifier of an entity and return either + the system identifier to read from as a string, or an InputSource + to read from.""" + return systemId + + +#============================================================================ +# +# CORE FEATURES +# +#============================================================================ + +feature_namespaces = "http://xml.org/sax/features/namespaces" +# true: Perform Namespace processing (default). +# false: Optionally do not perform Namespace processing +# (implies namespace-prefixes). +# access: (parsing) read-only; (not parsing) read/write + +feature_namespace_prefixes = "http://xml.org/sax/features/namespace-prefixes" +# true: Report the original prefixed names and attributes used for Namespace +# declarations. +# false: Do not report attributes used for Namespace declarations, and +# optionally do not report original prefixed names (default). +# access: (parsing) read-only; (not parsing) read/write + +feature_string_interning = "http://xml.org/sax/features/string-interning" +# true: All element names, prefixes, attribute names, Namespace URIs, and +# local names are interned using the built-in intern function. +# false: Names are not necessarily interned, although they may be (default). +# access: (parsing) read-only; (not parsing) read/write + +feature_validation = "http://xml.org/sax/features/validation" +# true: Report all validation errors (implies external-general-entities and +# external-parameter-entities). +# false: Do not report validation errors. +# access: (parsing) read-only; (not parsing) read/write + +feature_external_ges = "http://xml.org/sax/features/external-general-entities" +# true: Include all external general (text) entities. +# false: Do not include external general entities. +# access: (parsing) read-only; (not parsing) read/write + +feature_external_pes = "http://xml.org/sax/features/external-parameter-entities" +# true: Include all external parameter entities, including the external +# DTD subset. +# false: Do not include any external parameter entities, even the external +# DTD subset. +# access: (parsing) read-only; (not parsing) read/write + +all_features = [feature_namespaces, + feature_namespace_prefixes, + feature_string_interning, + feature_validation, + feature_external_ges, + feature_external_pes] + + +#============================================================================ +# +# CORE PROPERTIES +# +#============================================================================ + +property_lexical_handler = "http://xml.org/sax/properties/lexical-handler" +# data type: xml.sax.sax2lib.LexicalHandler +# description: An optional extension handler for lexical events like comments. +# access: read/write + +property_declaration_handler = "http://xml.org/sax/properties/declaration-handler" +# data type: xml.sax.sax2lib.DeclHandler +# description: An optional extension handler for DTD-related events other +# than notations and unparsed entities. +# access: read/write + +property_dom_node = "http://xml.org/sax/properties/dom-node" +# data type: org.w3c.dom.Node +# description: When parsing, the current DOM node being visited if this is +# a DOM iterator; when not parsing, the root DOM node for +# iteration. +# access: (parsing) read-only; (not parsing) read/write + +property_xml_string = "http://xml.org/sax/properties/xml-string" +# data type: String +# description: The literal string of characters that was the source for +# the current event. +# access: read-only + +property_encoding = "http://www.python.org/sax/properties/encoding" +# data type: String +# description: The name of the encoding to assume for input data. +# access: write: set the encoding, e.g. established by a higher-level +# protocol. May change during parsing (e.g. after +# processing a META tag) +# read: return the current encoding (possibly established through +# auto-detection. +# initial value: UTF-8 +# + +property_interning_dict = "http://www.python.org/sax/properties/interning-dict" +# data type: Dictionary +# description: The dictionary used to intern common strings in the document +# access: write: Request that the parser uses a specific dictionary, to +# allow interning across different documents +# read: return the current interning dictionary, or None +# + +all_properties = [property_lexical_handler, + property_dom_node, + property_declaration_handler, + property_xml_string, + property_encoding, + property_interning_dict] diff --git a/Lib/xml/sax/saxutils.py b/Lib/xml/sax/saxutils.py new file mode 100644 index 0000000..a496519 --- /dev/null +++ b/Lib/xml/sax/saxutils.py @@ -0,0 +1,299 @@ +"""\ +A library of useful helper classes to the SAX classes, for the +convenience of application and driver writers. +""" + +import os, urlparse, urllib, types +import handler +import xmlreader + +try: + _StringTypes = [types.StringType, types.UnicodeType] +except AttributeError: + _StringTypes = [types.StringType] + +# See whether the xmlcharrefreplace error handler is +# supported +try: + from codecs import xmlcharrefreplace_errors + _error_handling = "xmlcharrefreplace" + del xmlcharrefreplace_errors +except ImportError: + _error_handling = "strict" + +def __dict_replace(s, d): + """Replace substrings of a string using a dictionary.""" + for key, value in d.items(): + s = s.replace(key, value) + return s + +def escape(data, entities={}): + """Escape &, <, and > in a string of data. + + You can escape other strings of data by passing a dictionary as + the optional entities parameter. The keys and values must all be + strings; each key will be replaced with its corresponding value. + """ + + # must do ampersand first + data = data.replace("&", "&") + data = data.replace(">", ">") + data = data.replace("<", "<") + if entities: + data = __dict_replace(data, entities) + return data + +def unescape(data, entities={}): + """Unescape &, <, and > in a string of data. + + You can unescape other strings of data by passing a dictionary as + the optional entities parameter. The keys and values must all be + strings; each key will be replaced with its corresponding value. + """ + data = data.replace("<", "<") + data = data.replace(">", ">") + if entities: + data = __dict_replace(data, entities) + # must do ampersand last + return data.replace("&", "&") + +def quoteattr(data, entities={}): + """Escape and quote an attribute value. + + Escape &, <, and > in a string of data, then quote it for use as + an attribute value. The \" character will be escaped as well, if + necessary. + + You can escape other strings of data by passing a dictionary as + the optional entities parameter. The keys and values must all be + strings; each key will be replaced with its corresponding value. + """ + entities = entities.copy() + entities.update({'\n': ' ', '\r': ' ', '\t':' '}) + data = escape(data, entities) + if '"' in data: + if "'" in data: + data = '"%s"' % data.replace('"', """) + else: + data = "'%s'" % data + else: + data = '"%s"' % data + return data + + +class XMLGenerator(handler.ContentHandler): + + def __init__(self, out=None, encoding="iso-8859-1"): + if out is None: + import sys + out = sys.stdout + handler.ContentHandler.__init__(self) + self._out = out + self._ns_contexts = [{}] # contains uri -> prefix dicts + self._current_context = self._ns_contexts[-1] + self._undeclared_ns_maps = [] + self._encoding = encoding + + def _write(self, text): + if isinstance(text, str): + self._out.write(text) + else: + self._out.write(text.encode(self._encoding, _error_handling)) + + # ContentHandler methods + + def startDocument(self): + self._write('\n' % + self._encoding) + + def startPrefixMapping(self, prefix, uri): + self._ns_contexts.append(self._current_context.copy()) + self._current_context[uri] = prefix + self._undeclared_ns_maps.append((prefix, uri)) + + def endPrefixMapping(self, prefix): + self._current_context = self._ns_contexts[-1] + del self._ns_contexts[-1] + + def startElement(self, name, attrs): + self._write('<' + name) + for (name, value) in attrs.items(): + self._write(' %s=%s' % (name, quoteattr(value))) + self._write('>') + + def endElement(self, name): + self._write('' % name) + + def startElementNS(self, name, qname, attrs): + if name[0] is None: + # if the name was not namespace-scoped, use the unqualified part + name = name[1] + else: + # else try to restore the original prefix from the namespace + name = self._current_context[name[0]] + ":" + name[1] + self._write('<' + name) + + for pair in self._undeclared_ns_maps: + self._write(' xmlns:%s="%s"' % pair) + self._undeclared_ns_maps = [] + + for (name, value) in attrs.items(): + name = self._current_context[name[0]] + ":" + name[1] + self._write(' %s=%s' % (name, quoteattr(value))) + self._write('>') + + def endElementNS(self, name, qname): + if name[0] is None: + name = name[1] + else: + name = self._current_context[name[0]] + ":" + name[1] + self._write('' % name) + + def characters(self, content): + self._write(escape(content)) + + def ignorableWhitespace(self, content): + self._write(content) + + def processingInstruction(self, target, data): + self._write('' % (target, data)) + + +class XMLFilterBase(xmlreader.XMLReader): + """This class is designed to sit between an XMLReader and the + client application's event handlers. By default, it does nothing + but pass requests up to the reader and events on to the handlers + unmodified, but subclasses can override specific methods to modify + the event stream or the configuration requests as they pass + through.""" + + def __init__(self, parent = None): + xmlreader.XMLReader.__init__(self) + self._parent = parent + + # ErrorHandler methods + + def error(self, exception): + self._err_handler.error(exception) + + def fatalError(self, exception): + self._err_handler.fatalError(exception) + + def warning(self, exception): + self._err_handler.warning(exception) + + # ContentHandler methods + + def setDocumentLocator(self, locator): + self._cont_handler.setDocumentLocator(locator) + + def startDocument(self): + self._cont_handler.startDocument() + + def endDocument(self): + self._cont_handler.endDocument() + + def startPrefixMapping(self, prefix, uri): + self._cont_handler.startPrefixMapping(prefix, uri) + + def endPrefixMapping(self, prefix): + self._cont_handler.endPrefixMapping(prefix) + + def startElement(self, name, attrs): + self._cont_handler.startElement(name, attrs) + + def endElement(self, name): + self._cont_handler.endElement(name) + + def startElementNS(self, name, qname, attrs): + self._cont_handler.startElementNS(name, qname, attrs) + + def endElementNS(self, name, qname): + self._cont_handler.endElementNS(name, qname) + + def characters(self, content): + self._cont_handler.characters(content) + + def ignorableWhitespace(self, chars): + self._cont_handler.ignorableWhitespace(chars) + + def processingInstruction(self, target, data): + self._cont_handler.processingInstruction(target, data) + + def skippedEntity(self, name): + self._cont_handler.skippedEntity(name) + + # DTDHandler methods + + def notationDecl(self, name, publicId, systemId): + self._dtd_handler.notationDecl(name, publicId, systemId) + + def unparsedEntityDecl(self, name, publicId, systemId, ndata): + self._dtd_handler.unparsedEntityDecl(name, publicId, systemId, ndata) + + # EntityResolver methods + + def resolveEntity(self, publicId, systemId): + return self._ent_handler.resolveEntity(publicId, systemId) + + # XMLReader methods + + def parse(self, source): + self._parent.setContentHandler(self) + self._parent.setErrorHandler(self) + self._parent.setEntityResolver(self) + self._parent.setDTDHandler(self) + self._parent.parse(source) + + def setLocale(self, locale): + self._parent.setLocale(locale) + + def getFeature(self, name): + return self._parent.getFeature(name) + + def setFeature(self, name, state): + self._parent.setFeature(name, state) + + def getProperty(self, name): + return self._parent.getProperty(name) + + def setProperty(self, name, value): + self._parent.setProperty(name, value) + + # XMLFilter methods + + def getParent(self): + return self._parent + + def setParent(self, parent): + self._parent = parent + +# --- Utility functions + +def prepare_input_source(source, base = ""): + """This function takes an InputSource and an optional base URL and + returns a fully resolved InputSource object ready for reading.""" + + if type(source) in _StringTypes: + source = xmlreader.InputSource(source) + elif hasattr(source, "read"): + f = source + source = xmlreader.InputSource() + source.setByteStream(f) + if hasattr(f, "name"): + source.setSystemId(f.name) + + if source.getByteStream() is None: + sysid = source.getSystemId() + basehead = os.path.dirname(os.path.normpath(base)) + sysidfilename = os.path.join(basehead, sysid) + if os.path.isfile(sysidfilename): + source.setSystemId(sysidfilename) + f = open(sysidfilename, "rb") + else: + source.setSystemId(urlparse.urljoin(base, sysid)) + f = urllib.urlopen(source.getSystemId()) + + source.setByteStream(f) + + return source diff --git a/Lib/xml/sax/xmlreader.py b/Lib/xml/sax/xmlreader.py new file mode 100644 index 0000000..9a2361e --- /dev/null +++ b/Lib/xml/sax/xmlreader.py @@ -0,0 +1,381 @@ +"""An XML Reader is the SAX 2 name for an XML parser. XML Parsers +should be based on this code. """ + +import handler + +from _exceptions import SAXNotSupportedException, SAXNotRecognizedException + + +# ===== XMLREADER ===== + +class XMLReader: + """Interface for reading an XML document using callbacks. + + XMLReader is the interface that an XML parser's SAX2 driver must + implement. This interface allows an application to set and query + features and properties in the parser, to register event handlers + for document processing, and to initiate a document parse. + + All SAX interfaces are assumed to be synchronous: the parse + methods must not return until parsing is complete, and readers + must wait for an event-handler callback to return before reporting + the next event.""" + + def __init__(self): + self._cont_handler = handler.ContentHandler() + self._dtd_handler = handler.DTDHandler() + self._ent_handler = handler.EntityResolver() + self._err_handler = handler.ErrorHandler() + + def parse(self, source): + "Parse an XML document from a system identifier or an InputSource." + raise NotImplementedError("This method must be implemented!") + + def getContentHandler(self): + "Returns the current ContentHandler." + return self._cont_handler + + def setContentHandler(self, handler): + "Registers a new object to receive document content events." + self._cont_handler = handler + + def getDTDHandler(self): + "Returns the current DTD handler." + return self._dtd_handler + + def setDTDHandler(self, handler): + "Register an object to receive basic DTD-related events." + self._dtd_handler = handler + + def getEntityResolver(self): + "Returns the current EntityResolver." + return self._ent_handler + + def setEntityResolver(self, resolver): + "Register an object to resolve external entities." + self._ent_handler = resolver + + def getErrorHandler(self): + "Returns the current ErrorHandler." + return self._err_handler + + def setErrorHandler(self, handler): + "Register an object to receive error-message events." + self._err_handler = handler + + def setLocale(self, locale): + """Allow an application to set the locale for errors and warnings. + + SAX parsers are not required to provide localization for errors + and warnings; if they cannot support the requested locale, + however, they must throw a SAX exception. Applications may + request a locale change in the middle of a parse.""" + raise SAXNotSupportedException("Locale support not implemented") + + def getFeature(self, name): + "Looks up and returns the state of a SAX2 feature." + raise SAXNotRecognizedException("Feature '%s' not recognized" % name) + + def setFeature(self, name, state): + "Sets the state of a SAX2 feature." + raise SAXNotRecognizedException("Feature '%s' not recognized" % name) + + def getProperty(self, name): + "Looks up and returns the value of a SAX2 property." + raise SAXNotRecognizedException("Property '%s' not recognized" % name) + + def setProperty(self, name, value): + "Sets the value of a SAX2 property." + raise SAXNotRecognizedException("Property '%s' not recognized" % name) + +class IncrementalParser(XMLReader): + """This interface adds three extra methods to the XMLReader + interface that allow XML parsers to support incremental + parsing. Support for this interface is optional, since not all + underlying XML parsers support this functionality. + + When the parser is instantiated it is ready to begin accepting + data from the feed method immediately. After parsing has been + finished with a call to close the reset method must be called to + make the parser ready to accept new data, either from feed or + using the parse method. + + Note that these methods must _not_ be called during parsing, that + is, after parse has been called and before it returns. + + By default, the class also implements the parse method of the XMLReader + interface using the feed, close and reset methods of the + IncrementalParser interface as a convenience to SAX 2.0 driver + writers.""" + + def __init__(self, bufsize=2**16): + self._bufsize = bufsize + XMLReader.__init__(self) + + def parse(self, source): + import saxutils + source = saxutils.prepare_input_source(source) + + self.prepareParser(source) + file = source.getByteStream() + buffer = file.read(self._bufsize) + while buffer != "": + self.feed(buffer) + buffer = file.read(self._bufsize) + self.close() + + def feed(self, data): + """This method gives the raw XML data in the data parameter to + the parser and makes it parse the data, emitting the + corresponding events. It is allowed for XML constructs to be + split across several calls to feed. + + feed may raise SAXException.""" + raise NotImplementedError("This method must be implemented!") + + def prepareParser(self, source): + """This method is called by the parse implementation to allow + the SAX 2.0 driver to prepare itself for parsing.""" + raise NotImplementedError("prepareParser must be overridden!") + + def close(self): + """This method is called when the entire XML document has been + passed to the parser through the feed method, to notify the + parser that there are no more data. This allows the parser to + do the final checks on the document and empty the internal + data buffer. + + The parser will not be ready to parse another document until + the reset method has been called. + + close may raise SAXException.""" + raise NotImplementedError("This method must be implemented!") + + def reset(self): + """This method is called after close has been called to reset + the parser so that it is ready to parse new documents. The + results of calling parse or feed after close without calling + reset are undefined.""" + raise NotImplementedError("This method must be implemented!") + +# ===== LOCATOR ===== + +class Locator: + """Interface for associating a SAX event with a document + location. A locator object will return valid results only during + calls to DocumentHandler methods; at any other time, the + results are unpredictable.""" + + def getColumnNumber(self): + "Return the column number where the current event ends." + return -1 + + def getLineNumber(self): + "Return the line number where the current event ends." + return -1 + + def getPublicId(self): + "Return the public identifier for the current event." + return None + + def getSystemId(self): + "Return the system identifier for the current event." + return None + +# ===== INPUTSOURCE ===== + +class InputSource: + """Encapsulation of the information needed by the XMLReader to + read entities. + + This class may include information about the public identifier, + system identifier, byte stream (possibly with character encoding + information) and/or the character stream of an entity. + + Applications will create objects of this class for use in the + XMLReader.parse method and for returning from + EntityResolver.resolveEntity. + + An InputSource belongs to the application, the XMLReader is not + allowed to modify InputSource objects passed to it from the + application, although it may make copies and modify those.""" + + def __init__(self, system_id = None): + self.__system_id = system_id + self.__public_id = None + self.__encoding = None + self.__bytefile = None + self.__charfile = None + + def setPublicId(self, public_id): + "Sets the public identifier of this InputSource." + self.__public_id = public_id + + def getPublicId(self): + "Returns the public identifier of this InputSource." + return self.__public_id + + def setSystemId(self, system_id): + "Sets the system identifier of this InputSource." + self.__system_id = system_id + + def getSystemId(self): + "Returns the system identifier of this InputSource." + return self.__system_id + + def setEncoding(self, encoding): + """Sets the character encoding of this InputSource. + + The encoding must be a string acceptable for an XML encoding + declaration (see section 4.3.3 of the XML recommendation). + + The encoding attribute of the InputSource is ignored if the + InputSource also contains a character stream.""" + self.__encoding = encoding + + def getEncoding(self): + "Get the character encoding of this InputSource." + return self.__encoding + + def setByteStream(self, bytefile): + """Set the byte stream (a Python file-like object which does + not perform byte-to-character conversion) for this input + source. + + The SAX parser will ignore this if there is also a character + stream specified, but it will use a byte stream in preference + to opening a URI connection itself. + + If the application knows the character encoding of the byte + stream, it should set it with the setEncoding method.""" + self.__bytefile = bytefile + + def getByteStream(self): + """Get the byte stream for this input source. + + The getEncoding method will return the character encoding for + this byte stream, or None if unknown.""" + return self.__bytefile + + def setCharacterStream(self, charfile): + """Set the character stream for this input source. (The stream + must be a Python 2.0 Unicode-wrapped file-like that performs + conversion to Unicode strings.) + + If there is a character stream specified, the SAX parser will + ignore any byte stream and will not attempt to open a URI + connection to the system identifier.""" + self.__charfile = charfile + + def getCharacterStream(self): + "Get the character stream for this input source." + return self.__charfile + +# ===== ATTRIBUTESIMPL ===== + +class AttributesImpl: + + def __init__(self, attrs): + """Non-NS-aware implementation. + + attrs should be of the form {name : value}.""" + self._attrs = attrs + + def getLength(self): + return len(self._attrs) + + def getType(self, name): + return "CDATA" + + def getValue(self, name): + return self._attrs[name] + + def getValueByQName(self, name): + return self._attrs[name] + + def getNameByQName(self, name): + if not self._attrs.has_key(name): + raise KeyError, name + return name + + def getQNameByName(self, name): + if not self._attrs.has_key(name): + raise KeyError, name + return name + + def getNames(self): + return self._attrs.keys() + + def getQNames(self): + return self._attrs.keys() + + def __len__(self): + return len(self._attrs) + + def __getitem__(self, name): + return self._attrs[name] + + def keys(self): + return self._attrs.keys() + + def has_key(self, name): + return self._attrs.has_key(name) + + def __contains__(self, name): + return self._attrs.has_key(name) + + def get(self, name, alternative=None): + return self._attrs.get(name, alternative) + + def copy(self): + return self.__class__(self._attrs) + + def items(self): + return self._attrs.items() + + def values(self): + return self._attrs.values() + +# ===== ATTRIBUTESNSIMPL ===== + +class AttributesNSImpl(AttributesImpl): + + def __init__(self, attrs, qnames): + """NS-aware implementation. + + attrs should be of the form {(ns_uri, lname): value, ...}. + qnames of the form {(ns_uri, lname): qname, ...}.""" + self._attrs = attrs + self._qnames = qnames + + def getValueByQName(self, name): + for (nsname, qname) in self._qnames.items(): + if qname == name: + return self._attrs[nsname] + + raise KeyError, name + + def getNameByQName(self, name): + for (nsname, qname) in self._qnames.items(): + if qname == name: + return nsname + + raise KeyError, name + + def getQNameByName(self, name): + return self._qnames[name] + + def getQNames(self): + return self._qnames.values() + + def copy(self): + return self.__class__(self._attrs, self._qnames) + + +def _test(): + XMLReader() + IncrementalParser() + Locator() + +if __name__ == "__main__": + _test() diff --git a/Lib/xmlcore/__init__.py b/Lib/xmlcore/__init__.py deleted file mode 100644 index bf6d8dd..0000000 --- a/Lib/xmlcore/__init__.py +++ /dev/null @@ -1,20 +0,0 @@ -"""Core XML support for Python. - -This package contains four sub-packages: - -dom -- The W3C Document Object Model. This supports DOM Level 1 + - Namespaces. - -parsers -- Python wrappers for XML parsers (currently only supports Expat). - -sax -- The Simple API for XML, developed by XML-Dev, led by David - Megginson and ported to Python by Lars Marius Garshol. This - supports the SAX 2 API. - -etree -- The ElementTree XML library. This is a subset of the full - ElementTree XML release. - -""" - - -__all__ = ["dom", "parsers", "sax", "etree"] diff --git a/Lib/xmlcore/dom/NodeFilter.py b/Lib/xmlcore/dom/NodeFilter.py deleted file mode 100644 index fc05245..0000000 --- a/Lib/xmlcore/dom/NodeFilter.py +++ /dev/null @@ -1,27 +0,0 @@ -# This is the Python mapping for interface NodeFilter from -# DOM2-Traversal-Range. It contains only constants. - -class NodeFilter: - """ - This is the DOM2 NodeFilter interface. It contains only constants. - """ - FILTER_ACCEPT = 1 - FILTER_REJECT = 2 - FILTER_SKIP = 3 - - SHOW_ALL = 0xFFFFFFFFL - SHOW_ELEMENT = 0x00000001 - SHOW_ATTRIBUTE = 0x00000002 - SHOW_TEXT = 0x00000004 - SHOW_CDATA_SECTION = 0x00000008 - SHOW_ENTITY_REFERENCE = 0x00000010 - SHOW_ENTITY = 0x00000020 - SHOW_PROCESSING_INSTRUCTION = 0x00000040 - SHOW_COMMENT = 0x00000080 - SHOW_DOCUMENT = 0x00000100 - SHOW_DOCUMENT_TYPE = 0x00000200 - SHOW_DOCUMENT_FRAGMENT = 0x00000400 - SHOW_NOTATION = 0x00000800 - - def acceptNode(self, node): - raise NotImplementedError diff --git a/Lib/xmlcore/dom/__init__.py b/Lib/xmlcore/dom/__init__.py deleted file mode 100644 index 002cdb7..0000000 --- a/Lib/xmlcore/dom/__init__.py +++ /dev/null @@ -1,139 +0,0 @@ -"""W3C Document Object Model implementation for Python. - -The Python mapping of the Document Object Model is documented in the -Python Library Reference in the section on the xml.dom package. - -This package contains the following modules: - -minidom -- A simple implementation of the Level 1 DOM with namespace - support added (based on the Level 2 specification) and other - minor Level 2 functionality. - -pulldom -- DOM builder supporting on-demand tree-building for selected - subtrees of the document. - -""" - - -class Node: - """Class giving the NodeType constants.""" - - # DOM implementations may use this as a base class for their own - # Node implementations. If they don't, the constants defined here - # should still be used as the canonical definitions as they match - # the values given in the W3C recommendation. Client code can - # safely refer to these values in all tests of Node.nodeType - # values. - - ELEMENT_NODE = 1 - ATTRIBUTE_NODE = 2 - TEXT_NODE = 3 - CDATA_SECTION_NODE = 4 - ENTITY_REFERENCE_NODE = 5 - ENTITY_NODE = 6 - PROCESSING_INSTRUCTION_NODE = 7 - COMMENT_NODE = 8 - DOCUMENT_NODE = 9 - DOCUMENT_TYPE_NODE = 10 - DOCUMENT_FRAGMENT_NODE = 11 - NOTATION_NODE = 12 - - -#ExceptionCode -INDEX_SIZE_ERR = 1 -DOMSTRING_SIZE_ERR = 2 -HIERARCHY_REQUEST_ERR = 3 -WRONG_DOCUMENT_ERR = 4 -INVALID_CHARACTER_ERR = 5 -NO_DATA_ALLOWED_ERR = 6 -NO_MODIFICATION_ALLOWED_ERR = 7 -NOT_FOUND_ERR = 8 -NOT_SUPPORTED_ERR = 9 -INUSE_ATTRIBUTE_ERR = 10 -INVALID_STATE_ERR = 11 -SYNTAX_ERR = 12 -INVALID_MODIFICATION_ERR = 13 -NAMESPACE_ERR = 14 -INVALID_ACCESS_ERR = 15 -VALIDATION_ERR = 16 - - -class DOMException(Exception): - """Abstract base class for DOM exceptions. - Exceptions with specific codes are specializations of this class.""" - - def __init__(self, *args, **kw): - if self.__class__ is DOMException: - raise RuntimeError( - "DOMException should not be instantiated directly") - Exception.__init__(self, *args, **kw) - - def _get_code(self): - return self.code - - -class IndexSizeErr(DOMException): - code = INDEX_SIZE_ERR - -class DomstringSizeErr(DOMException): - code = DOMSTRING_SIZE_ERR - -class HierarchyRequestErr(DOMException): - code = HIERARCHY_REQUEST_ERR - -class WrongDocumentErr(DOMException): - code = WRONG_DOCUMENT_ERR - -class InvalidCharacterErr(DOMException): - code = INVALID_CHARACTER_ERR - -class NoDataAllowedErr(DOMException): - code = NO_DATA_ALLOWED_ERR - -class NoModificationAllowedErr(DOMException): - code = NO_MODIFICATION_ALLOWED_ERR - -class NotFoundErr(DOMException): - code = NOT_FOUND_ERR - -class NotSupportedErr(DOMException): - code = NOT_SUPPORTED_ERR - -class InuseAttributeErr(DOMException): - code = INUSE_ATTRIBUTE_ERR - -class InvalidStateErr(DOMException): - code = INVALID_STATE_ERR - -class SyntaxErr(DOMException): - code = SYNTAX_ERR - -class InvalidModificationErr(DOMException): - code = INVALID_MODIFICATION_ERR - -class NamespaceErr(DOMException): - code = NAMESPACE_ERR - -class InvalidAccessErr(DOMException): - code = INVALID_ACCESS_ERR - -class ValidationErr(DOMException): - code = VALIDATION_ERR - -class UserDataHandler: - """Class giving the operation constants for UserDataHandler.handle().""" - - # Based on DOM Level 3 (WD 9 April 2002) - - NODE_CLONED = 1 - NODE_IMPORTED = 2 - NODE_DELETED = 3 - NODE_RENAMED = 4 - -XML_NAMESPACE = "http://www.w3.org/XML/1998/namespace" -XMLNS_NAMESPACE = "http://www.w3.org/2000/xmlns/" -XHTML_NAMESPACE = "http://www.w3.org/1999/xhtml" -EMPTY_NAMESPACE = None -EMPTY_PREFIX = None - -from .domreg import getDOMImplementation,registerDOMImplementation diff --git a/Lib/xmlcore/dom/domreg.py b/Lib/xmlcore/dom/domreg.py deleted file mode 100644 index d60ed64..0000000 --- a/Lib/xmlcore/dom/domreg.py +++ /dev/null @@ -1,99 +0,0 @@ -"""Registration facilities for DOM. This module should not be used -directly. Instead, the functions getDOMImplementation and -registerDOMImplementation should be imported from xml.dom.""" - -from xmlcore.dom.minicompat import * # isinstance, StringTypes - -# This is a list of well-known implementations. Well-known names -# should be published by posting to xml-sig@python.org, and are -# subsequently recorded in this file. - -well_known_implementations = { - 'minidom':'xml.dom.minidom', - '4DOM': 'xml.dom.DOMImplementation', - } - -# DOM implementations not officially registered should register -# themselves with their - -registered = {} - -def registerDOMImplementation(name, factory): - """registerDOMImplementation(name, factory) - - Register the factory function with the name. The factory function - should return an object which implements the DOMImplementation - interface. The factory function can either return the same object, - or a new one (e.g. if that implementation supports some - customization).""" - - registered[name] = factory - -def _good_enough(dom, features): - "_good_enough(dom, features) -> Return 1 if the dom offers the features" - for f,v in features: - if not dom.hasFeature(f,v): - return 0 - return 1 - -def getDOMImplementation(name = None, features = ()): - """getDOMImplementation(name = None, features = ()) -> DOM implementation. - - Return a suitable DOM implementation. The name is either - well-known, the module name of a DOM implementation, or None. If - it is not None, imports the corresponding module and returns - DOMImplementation object if the import succeeds. - - If name is not given, consider the available implementations to - find one with the required feature set. If no implementation can - be found, raise an ImportError. The features list must be a sequence - of (feature, version) pairs which are passed to hasFeature.""" - - import os - creator = None - mod = well_known_implementations.get(name) - if mod: - mod = __import__(mod, {}, {}, ['getDOMImplementation']) - return mod.getDOMImplementation() - elif name: - return registered[name]() - elif os.environ.has_key("PYTHON_DOM"): - return getDOMImplementation(name = os.environ["PYTHON_DOM"]) - - # User did not specify a name, try implementations in arbitrary - # order, returning the one that has the required features - if isinstance(features, StringTypes): - features = _parse_feature_string(features) - for creator in registered.values(): - dom = creator() - if _good_enough(dom, features): - return dom - - for creator in well_known_implementations.keys(): - try: - dom = getDOMImplementation(name = creator) - except StandardError: # typically ImportError, or AttributeError - continue - if _good_enough(dom, features): - return dom - - raise ImportError,"no suitable DOM implementation found" - -def _parse_feature_string(s): - features = [] - parts = s.split() - i = 0 - length = len(parts) - while i < length: - feature = parts[i] - if feature[0] in "0123456789": - raise ValueError, "bad feature name: %r" % (feature,) - i = i + 1 - version = None - if i < length: - v = parts[i] - if v[0] in "0123456789": - i = i + 1 - version = v - features.append((feature, version)) - return tuple(features) diff --git a/Lib/xmlcore/dom/expatbuilder.py b/Lib/xmlcore/dom/expatbuilder.py deleted file mode 100644 index 32ffa41..0000000 --- a/Lib/xmlcore/dom/expatbuilder.py +++ /dev/null @@ -1,983 +0,0 @@ -"""Facility to use the Expat parser to load a minidom instance -from a string or file. - -This avoids all the overhead of SAX and pulldom to gain performance. -""" - -# Warning! -# -# This module is tightly bound to the implementation details of the -# minidom DOM and can't be used with other DOM implementations. This -# is due, in part, to a lack of appropriate methods in the DOM (there is -# no way to create Entity and Notation nodes via the DOM Level 2 -# interface), and for performance. The later is the cause of some fairly -# cryptic code. -# -# Performance hacks: -# -# - .character_data_handler() has an extra case in which continuing -# data is appended to an existing Text node; this can be a -# speedup since pyexpat can break up character data into multiple -# callbacks even though we set the buffer_text attribute on the -# parser. This also gives us the advantage that we don't need a -# separate normalization pass. -# -# - Determining that a node exists is done using an identity comparison -# with None rather than a truth test; this avoids searching for and -# calling any methods on the node object if it exists. (A rather -# nice speedup is achieved this way as well!) - -from xmlcore.dom import xmlbuilder, minidom, Node -from xmlcore.dom import EMPTY_NAMESPACE, EMPTY_PREFIX, XMLNS_NAMESPACE -from xmlcore.parsers import expat -from xmlcore.dom.minidom import _append_child, _set_attribute_node -from xmlcore.dom.NodeFilter import NodeFilter - -from xmlcore.dom.minicompat import * - -TEXT_NODE = Node.TEXT_NODE -CDATA_SECTION_NODE = Node.CDATA_SECTION_NODE -DOCUMENT_NODE = Node.DOCUMENT_NODE - -FILTER_ACCEPT = xmlbuilder.DOMBuilderFilter.FILTER_ACCEPT -FILTER_REJECT = xmlbuilder.DOMBuilderFilter.FILTER_REJECT -FILTER_SKIP = xmlbuilder.DOMBuilderFilter.FILTER_SKIP -FILTER_INTERRUPT = xmlbuilder.DOMBuilderFilter.FILTER_INTERRUPT - -theDOMImplementation = minidom.getDOMImplementation() - -# Expat typename -> TypeInfo -_typeinfo_map = { - "CDATA": minidom.TypeInfo(None, "cdata"), - "ENUM": minidom.TypeInfo(None, "enumeration"), - "ENTITY": minidom.TypeInfo(None, "entity"), - "ENTITIES": minidom.TypeInfo(None, "entities"), - "ID": minidom.TypeInfo(None, "id"), - "IDREF": minidom.TypeInfo(None, "idref"), - "IDREFS": minidom.TypeInfo(None, "idrefs"), - "NMTOKEN": minidom.TypeInfo(None, "nmtoken"), - "NMTOKENS": minidom.TypeInfo(None, "nmtokens"), - } - -class ElementInfo(object): - __slots__ = '_attr_info', '_model', 'tagName' - - def __init__(self, tagName, model=None): - self.tagName = tagName - self._attr_info = [] - self._model = model - - def __getstate__(self): - return self._attr_info, self._model, self.tagName - - def __setstate__(self, state): - self._attr_info, self._model, self.tagName = state - - def getAttributeType(self, aname): - for info in self._attr_info: - if info[1] == aname: - t = info[-2] - if t[0] == "(": - return _typeinfo_map["ENUM"] - else: - return _typeinfo_map[info[-2]] - return minidom._no_type - - def getAttributeTypeNS(self, namespaceURI, localName): - return minidom._no_type - - def isElementContent(self): - if self._model: - type = self._model[0] - return type not in (expat.model.XML_CTYPE_ANY, - expat.model.XML_CTYPE_MIXED) - else: - return False - - def isEmpty(self): - if self._model: - return self._model[0] == expat.model.XML_CTYPE_EMPTY - else: - return False - - def isId(self, aname): - for info in self._attr_info: - if info[1] == aname: - return info[-2] == "ID" - return False - - def isIdNS(self, euri, ename, auri, aname): - # not sure this is meaningful - return self.isId((auri, aname)) - -def _intern(builder, s): - return builder._intern_setdefault(s, s) - -def _parse_ns_name(builder, name): - assert ' ' in name - parts = name.split(' ') - intern = builder._intern_setdefault - if len(parts) == 3: - uri, localname, prefix = parts - prefix = intern(prefix, prefix) - qname = "%s:%s" % (prefix, localname) - qname = intern(qname, qname) - localname = intern(localname, localname) - else: - uri, localname = parts - prefix = EMPTY_PREFIX - qname = localname = intern(localname, localname) - return intern(uri, uri), localname, prefix, qname - - -class ExpatBuilder: - """Document builder that uses Expat to build a ParsedXML.DOM document - instance.""" - - def __init__(self, options=None): - if options is None: - options = xmlbuilder.Options() - self._options = options - if self._options.filter is not None: - self._filter = FilterVisibilityController(self._options.filter) - else: - self._filter = None - # This *really* doesn't do anything in this case, so - # override it with something fast & minimal. - self._finish_start_element = id - self._parser = None - self.reset() - - def createParser(self): - """Create a new parser object.""" - return expat.ParserCreate() - - def getParser(self): - """Return the parser object, creating a new one if needed.""" - if not self._parser: - self._parser = self.createParser() - self._intern_setdefault = self._parser.intern.setdefault - self._parser.buffer_text = True - self._parser.ordered_attributes = True - self._parser.specified_attributes = True - self.install(self._parser) - return self._parser - - def reset(self): - """Free all data structures used during DOM construction.""" - self.document = theDOMImplementation.createDocument( - EMPTY_NAMESPACE, None, None) - self.curNode = self.document - self._elem_info = self.document._elem_info - self._cdata = False - - def install(self, parser): - """Install the callbacks needed to build the DOM into the parser.""" - # This creates circular references! - parser.StartDoctypeDeclHandler = self.start_doctype_decl_handler - parser.StartElementHandler = self.first_element_handler - parser.EndElementHandler = self.end_element_handler - parser.ProcessingInstructionHandler = self.pi_handler - if self._options.entities: - parser.EntityDeclHandler = self.entity_decl_handler - parser.NotationDeclHandler = self.notation_decl_handler - if self._options.comments: - parser.CommentHandler = self.comment_handler - if self._options.cdata_sections: - parser.StartCdataSectionHandler = self.start_cdata_section_handler - parser.EndCdataSectionHandler = self.end_cdata_section_handler - parser.CharacterDataHandler = self.character_data_handler_cdata - else: - parser.CharacterDataHandler = self.character_data_handler - parser.ExternalEntityRefHandler = self.external_entity_ref_handler - parser.XmlDeclHandler = self.xml_decl_handler - parser.ElementDeclHandler = self.element_decl_handler - parser.AttlistDeclHandler = self.attlist_decl_handler - - def parseFile(self, file): - """Parse a document from a file object, returning the document - node.""" - parser = self.getParser() - first_buffer = True - try: - while 1: - buffer = file.read(16*1024) - if not buffer: - break - parser.Parse(buffer, 0) - if first_buffer and self.document.documentElement: - self._setup_subset(buffer) - first_buffer = False - parser.Parse("", True) - except ParseEscape: - pass - doc = self.document - self.reset() - self._parser = None - return doc - - def parseString(self, string): - """Parse a document from a string, returning the document node.""" - parser = self.getParser() - try: - parser.Parse(string, True) - self._setup_subset(string) - except ParseEscape: - pass - doc = self.document - self.reset() - self._parser = None - return doc - - def _setup_subset(self, buffer): - """Load the internal subset if there might be one.""" - if self.document.doctype: - extractor = InternalSubsetExtractor() - extractor.parseString(buffer) - subset = extractor.getSubset() - self.document.doctype.internalSubset = subset - - def start_doctype_decl_handler(self, doctypeName, systemId, publicId, - has_internal_subset): - doctype = self.document.implementation.createDocumentType( - doctypeName, publicId, systemId) - doctype.ownerDocument = self.document - self.document.childNodes.append(doctype) - self.document.doctype = doctype - if self._filter and self._filter.acceptNode(doctype) == FILTER_REJECT: - self.document.doctype = None - del self.document.childNodes[-1] - doctype = None - self._parser.EntityDeclHandler = None - self._parser.NotationDeclHandler = None - if has_internal_subset: - if doctype is not None: - doctype.entities._seq = [] - doctype.notations._seq = [] - self._parser.CommentHandler = None - self._parser.ProcessingInstructionHandler = None - self._parser.EndDoctypeDeclHandler = self.end_doctype_decl_handler - - def end_doctype_decl_handler(self): - if self._options.comments: - self._parser.CommentHandler = self.comment_handler - self._parser.ProcessingInstructionHandler = self.pi_handler - if not (self._elem_info or self._filter): - self._finish_end_element = id - - def pi_handler(self, target, data): - node = self.document.createProcessingInstruction(target, data) - _append_child(self.curNode, node) - if self._filter and self._filter.acceptNode(node) == FILTER_REJECT: - self.curNode.removeChild(node) - - def character_data_handler_cdata(self, data): - childNodes = self.curNode.childNodes - if self._cdata: - if ( self._cdata_continue - and childNodes[-1].nodeType == CDATA_SECTION_NODE): - childNodes[-1].appendData(data) - return - node = self.document.createCDATASection(data) - self._cdata_continue = True - elif childNodes and childNodes[-1].nodeType == TEXT_NODE: - node = childNodes[-1] - value = node.data + data - d = node.__dict__ - d['data'] = d['nodeValue'] = value - return - else: - node = minidom.Text() - d = node.__dict__ - d['data'] = d['nodeValue'] = data - d['ownerDocument'] = self.document - _append_child(self.curNode, node) - - def character_data_handler(self, data): - childNodes = self.curNode.childNodes - if childNodes and childNodes[-1].nodeType == TEXT_NODE: - node = childNodes[-1] - d = node.__dict__ - d['data'] = d['nodeValue'] = node.data + data - return - node = minidom.Text() - d = node.__dict__ - d['data'] = d['nodeValue'] = node.data + data - d['ownerDocument'] = self.document - _append_child(self.curNode, node) - - def entity_decl_handler(self, entityName, is_parameter_entity, value, - base, systemId, publicId, notationName): - if is_parameter_entity: - # we don't care about parameter entities for the DOM - return - if not self._options.entities: - return - node = self.document._create_entity(entityName, publicId, - systemId, notationName) - if value is not None: - # internal entity - # node *should* be readonly, but we'll cheat - child = self.document.createTextNode(value) - node.childNodes.append(child) - self.document.doctype.entities._seq.append(node) - if self._filter and self._filter.acceptNode(node) == FILTER_REJECT: - del self.document.doctype.entities._seq[-1] - - def notation_decl_handler(self, notationName, base, systemId, publicId): - node = self.document._create_notation(notationName, publicId, systemId) - self.document.doctype.notations._seq.append(node) - if self._filter and self._filter.acceptNode(node) == FILTER_ACCEPT: - del self.document.doctype.notations._seq[-1] - - def comment_handler(self, data): - node = self.document.createComment(data) - _append_child(self.curNode, node) - if self._filter and self._filter.acceptNode(node) == FILTER_REJECT: - self.curNode.removeChild(node) - - def start_cdata_section_handler(self): - self._cdata = True - self._cdata_continue = False - - def end_cdata_section_handler(self): - self._cdata = False - self._cdata_continue = False - - def external_entity_ref_handler(self, context, base, systemId, publicId): - return 1 - - def first_element_handler(self, name, attributes): - if self._filter is None and not self._elem_info: - self._finish_end_element = id - self.getParser().StartElementHandler = self.start_element_handler - self.start_element_handler(name, attributes) - - def start_element_handler(self, name, attributes): - node = self.document.createElement(name) - _append_child(self.curNode, node) - self.curNode = node - - if attributes: - for i in range(0, len(attributes), 2): - a = minidom.Attr(attributes[i], EMPTY_NAMESPACE, - None, EMPTY_PREFIX) - value = attributes[i+1] - d = a.childNodes[0].__dict__ - d['data'] = d['nodeValue'] = value - d = a.__dict__ - d['value'] = d['nodeValue'] = value - d['ownerDocument'] = self.document - _set_attribute_node(node, a) - - if node is not self.document.documentElement: - self._finish_start_element(node) - - def _finish_start_element(self, node): - if self._filter: - # To be general, we'd have to call isSameNode(), but this - # is sufficient for minidom: - if node is self.document.documentElement: - return - filt = self._filter.startContainer(node) - if filt == FILTER_REJECT: - # ignore this node & all descendents - Rejecter(self) - elif filt == FILTER_SKIP: - # ignore this node, but make it's children become - # children of the parent node - Skipper(self) - else: - return - self.curNode = node.parentNode - node.parentNode.removeChild(node) - node.unlink() - - # If this ever changes, Namespaces.end_element_handler() needs to - # be changed to match. - # - def end_element_handler(self, name): - curNode = self.curNode - self.curNode = curNode.parentNode - self._finish_end_element(curNode) - - def _finish_end_element(self, curNode): - info = self._elem_info.get(curNode.tagName) - if info: - self._handle_white_text_nodes(curNode, info) - if self._filter: - if curNode is self.document.documentElement: - return - if self._filter.acceptNode(curNode) == FILTER_REJECT: - self.curNode.removeChild(curNode) - curNode.unlink() - - def _handle_white_text_nodes(self, node, info): - if (self._options.whitespace_in_element_content - or not info.isElementContent()): - return - - # We have element type information and should remove ignorable - # whitespace; identify for text nodes which contain only - # whitespace. - L = [] - for child in node.childNodes: - if child.nodeType == TEXT_NODE and not child.data.strip(): - L.append(child) - - # Remove ignorable whitespace from the tree. - for child in L: - node.removeChild(child) - - def element_decl_handler(self, name, model): - info = self._elem_info.get(name) - if info is None: - self._elem_info[name] = ElementInfo(name, model) - else: - assert info._model is None - info._model = model - - def attlist_decl_handler(self, elem, name, type, default, required): - info = self._elem_info.get(elem) - if info is None: - info = ElementInfo(elem) - self._elem_info[elem] = info - info._attr_info.append( - [None, name, None, None, default, 0, type, required]) - - def xml_decl_handler(self, version, encoding, standalone): - self.document.version = version - self.document.encoding = encoding - # This is still a little ugly, thanks to the pyexpat API. ;-( - if standalone >= 0: - if standalone: - self.document.standalone = True - else: - self.document.standalone = False - - -# Don't include FILTER_INTERRUPT, since that's checked separately -# where allowed. -_ALLOWED_FILTER_RETURNS = (FILTER_ACCEPT, FILTER_REJECT, FILTER_SKIP) - -class FilterVisibilityController(object): - """Wrapper around a DOMBuilderFilter which implements the checks - to make the whatToShow filter attribute work.""" - - __slots__ = 'filter', - - def __init__(self, filter): - self.filter = filter - - def startContainer(self, node): - mask = self._nodetype_mask[node.nodeType] - if self.filter.whatToShow & mask: - val = self.filter.startContainer(node) - if val == FILTER_INTERRUPT: - raise ParseEscape - if val not in _ALLOWED_FILTER_RETURNS: - raise ValueError, \ - "startContainer() returned illegal value: " + repr(val) - return val - else: - return FILTER_ACCEPT - - def acceptNode(self, node): - mask = self._nodetype_mask[node.nodeType] - if self.filter.whatToShow & mask: - val = self.filter.acceptNode(node) - if val == FILTER_INTERRUPT: - raise ParseEscape - if val == FILTER_SKIP: - # move all child nodes to the parent, and remove this node - parent = node.parentNode - for child in node.childNodes[:]: - parent.appendChild(child) - # node is handled by the caller - return FILTER_REJECT - if val not in _ALLOWED_FILTER_RETURNS: - raise ValueError, \ - "acceptNode() returned illegal value: " + repr(val) - return val - else: - return FILTER_ACCEPT - - _nodetype_mask = { - Node.ELEMENT_NODE: NodeFilter.SHOW_ELEMENT, - Node.ATTRIBUTE_NODE: NodeFilter.SHOW_ATTRIBUTE, - Node.TEXT_NODE: NodeFilter.SHOW_TEXT, - Node.CDATA_SECTION_NODE: NodeFilter.SHOW_CDATA_SECTION, - Node.ENTITY_REFERENCE_NODE: NodeFilter.SHOW_ENTITY_REFERENCE, - Node.ENTITY_NODE: NodeFilter.SHOW_ENTITY, - Node.PROCESSING_INSTRUCTION_NODE: NodeFilter.SHOW_PROCESSING_INSTRUCTION, - Node.COMMENT_NODE: NodeFilter.SHOW_COMMENT, - Node.DOCUMENT_NODE: NodeFilter.SHOW_DOCUMENT, - Node.DOCUMENT_TYPE_NODE: NodeFilter.SHOW_DOCUMENT_TYPE, - Node.DOCUMENT_FRAGMENT_NODE: NodeFilter.SHOW_DOCUMENT_FRAGMENT, - Node.NOTATION_NODE: NodeFilter.SHOW_NOTATION, - } - - -class FilterCrutch(object): - __slots__ = '_builder', '_level', '_old_start', '_old_end' - - def __init__(self, builder): - self._level = 0 - self._builder = builder - parser = builder._parser - self._old_start = parser.StartElementHandler - self._old_end = parser.EndElementHandler - parser.StartElementHandler = self.start_element_handler - parser.EndElementHandler = self.end_element_handler - -class Rejecter(FilterCrutch): - __slots__ = () - - def __init__(self, builder): - FilterCrutch.__init__(self, builder) - parser = builder._parser - for name in ("ProcessingInstructionHandler", - "CommentHandler", - "CharacterDataHandler", - "StartCdataSectionHandler", - "EndCdataSectionHandler", - "ExternalEntityRefHandler", - ): - setattr(parser, name, None) - - def start_element_handler(self, *args): - self._level = self._level + 1 - - def end_element_handler(self, *args): - if self._level == 0: - # restore the old handlers - parser = self._builder._parser - self._builder.install(parser) - parser.StartElementHandler = self._old_start - parser.EndElementHandler = self._old_end - else: - self._level = self._level - 1 - -class Skipper(FilterCrutch): - __slots__ = () - - def start_element_handler(self, *args): - node = self._builder.curNode - self._old_start(*args) - if self._builder.curNode is not node: - self._level = self._level + 1 - - def end_element_handler(self, *args): - if self._level == 0: - # We're popping back out of the node we're skipping, so we - # shouldn't need to do anything but reset the handlers. - self._builder._parser.StartElementHandler = self._old_start - self._builder._parser.EndElementHandler = self._old_end - self._builder = None - else: - self._level = self._level - 1 - self._old_end(*args) - - -# framework document used by the fragment builder. -# Takes a string for the doctype, subset string, and namespace attrs string. - -_FRAGMENT_BUILDER_INTERNAL_SYSTEM_ID = \ - "http://xml.python.org/entities/fragment-builder/internal" - -_FRAGMENT_BUILDER_TEMPLATE = ( - '''\ - -%%s -]> -&fragment-builder-internal;''' - % _FRAGMENT_BUILDER_INTERNAL_SYSTEM_ID) - - -class FragmentBuilder(ExpatBuilder): - """Builder which constructs document fragments given XML source - text and a context node. - - The context node is expected to provide information about the - namespace declarations which are in scope at the start of the - fragment. - """ - - def __init__(self, context, options=None): - if context.nodeType == DOCUMENT_NODE: - self.originalDocument = context - self.context = context - else: - self.originalDocument = context.ownerDocument - self.context = context - ExpatBuilder.__init__(self, options) - - def reset(self): - ExpatBuilder.reset(self) - self.fragment = None - - def parseFile(self, file): - """Parse a document fragment from a file object, returning the - fragment node.""" - return self.parseString(file.read()) - - def parseString(self, string): - """Parse a document fragment from a string, returning the - fragment node.""" - self._source = string - parser = self.getParser() - doctype = self.originalDocument.doctype - ident = "" - if doctype: - subset = doctype.internalSubset or self._getDeclarations() - if doctype.publicId: - ident = ('PUBLIC "%s" "%s"' - % (doctype.publicId, doctype.systemId)) - elif doctype.systemId: - ident = 'SYSTEM "%s"' % doctype.systemId - else: - subset = "" - nsattrs = self._getNSattrs() # get ns decls from node's ancestors - document = _FRAGMENT_BUILDER_TEMPLATE % (ident, subset, nsattrs) - try: - parser.Parse(document, 1) - except: - self.reset() - raise - fragment = self.fragment - self.reset() -## self._parser = None - return fragment - - def _getDeclarations(self): - """Re-create the internal subset from the DocumentType node. - - This is only needed if we don't already have the - internalSubset as a string. - """ - doctype = self.context.ownerDocument.doctype - s = "" - if doctype: - for i in range(doctype.notations.length): - notation = doctype.notations.item(i) - if s: - s = s + "\n " - s = "%s' \ - % (s, notation.publicId, notation.systemId) - else: - s = '%s SYSTEM "%s">' % (s, notation.systemId) - for i in range(doctype.entities.length): - entity = doctype.entities.item(i) - if s: - s = s + "\n " - s = "%s" - return s - - def _getNSattrs(self): - return "" - - def external_entity_ref_handler(self, context, base, systemId, publicId): - if systemId == _FRAGMENT_BUILDER_INTERNAL_SYSTEM_ID: - # this entref is the one that we made to put the subtree - # in; all of our given input is parsed in here. - old_document = self.document - old_cur_node = self.curNode - parser = self._parser.ExternalEntityParserCreate(context) - # put the real document back, parse into the fragment to return - self.document = self.originalDocument - self.fragment = self.document.createDocumentFragment() - self.curNode = self.fragment - try: - parser.Parse(self._source, 1) - finally: - self.curNode = old_cur_node - self.document = old_document - self._source = None - return -1 - else: - return ExpatBuilder.external_entity_ref_handler( - self, context, base, systemId, publicId) - - -class Namespaces: - """Mix-in class for builders; adds support for namespaces.""" - - def _initNamespaces(self): - # list of (prefix, uri) ns declarations. Namespace attrs are - # constructed from this and added to the element's attrs. - self._ns_ordered_prefixes = [] - - def createParser(self): - """Create a new namespace-handling parser.""" - parser = expat.ParserCreate(namespace_separator=" ") - parser.namespace_prefixes = True - return parser - - def install(self, parser): - """Insert the namespace-handlers onto the parser.""" - ExpatBuilder.install(self, parser) - if self._options.namespace_declarations: - parser.StartNamespaceDeclHandler = ( - self.start_namespace_decl_handler) - - def start_namespace_decl_handler(self, prefix, uri): - """Push this namespace declaration on our storage.""" - self._ns_ordered_prefixes.append((prefix, uri)) - - def start_element_handler(self, name, attributes): - if ' ' in name: - uri, localname, prefix, qname = _parse_ns_name(self, name) - else: - uri = EMPTY_NAMESPACE - qname = name - localname = None - prefix = EMPTY_PREFIX - node = minidom.Element(qname, uri, prefix, localname) - node.ownerDocument = self.document - _append_child(self.curNode, node) - self.curNode = node - - if self._ns_ordered_prefixes: - for prefix, uri in self._ns_ordered_prefixes: - if prefix: - a = minidom.Attr(_intern(self, 'xmlns:' + prefix), - XMLNS_NAMESPACE, prefix, "xmlns") - else: - a = minidom.Attr("xmlns", XMLNS_NAMESPACE, - "xmlns", EMPTY_PREFIX) - d = a.childNodes[0].__dict__ - d['data'] = d['nodeValue'] = uri - d = a.__dict__ - d['value'] = d['nodeValue'] = uri - d['ownerDocument'] = self.document - _set_attribute_node(node, a) - del self._ns_ordered_prefixes[:] - - if attributes: - _attrs = node._attrs - _attrsNS = node._attrsNS - for i in range(0, len(attributes), 2): - aname = attributes[i] - value = attributes[i+1] - if ' ' in aname: - uri, localname, prefix, qname = _parse_ns_name(self, aname) - a = minidom.Attr(qname, uri, localname, prefix) - _attrs[qname] = a - _attrsNS[(uri, localname)] = a - else: - a = minidom.Attr(aname, EMPTY_NAMESPACE, - aname, EMPTY_PREFIX) - _attrs[aname] = a - _attrsNS[(EMPTY_NAMESPACE, aname)] = a - d = a.childNodes[0].__dict__ - d['data'] = d['nodeValue'] = value - d = a.__dict__ - d['ownerDocument'] = self.document - d['value'] = d['nodeValue'] = value - d['ownerElement'] = node - - if __debug__: - # This only adds some asserts to the original - # end_element_handler(), so we only define this when -O is not - # used. If changing one, be sure to check the other to see if - # it needs to be changed as well. - # - def end_element_handler(self, name): - curNode = self.curNode - if ' ' in name: - uri, localname, prefix, qname = _parse_ns_name(self, name) - assert (curNode.namespaceURI == uri - and curNode.localName == localname - and curNode.prefix == prefix), \ - "element stack messed up! (namespace)" - else: - assert curNode.nodeName == name, \ - "element stack messed up - bad nodeName" - assert curNode.namespaceURI == EMPTY_NAMESPACE, \ - "element stack messed up - bad namespaceURI" - self.curNode = curNode.parentNode - self._finish_end_element(curNode) - - -class ExpatBuilderNS(Namespaces, ExpatBuilder): - """Document builder that supports namespaces.""" - - def reset(self): - ExpatBuilder.reset(self) - self._initNamespaces() - - -class FragmentBuilderNS(Namespaces, FragmentBuilder): - """Fragment builder that supports namespaces.""" - - def reset(self): - FragmentBuilder.reset(self) - self._initNamespaces() - - def _getNSattrs(self): - """Return string of namespace attributes from this element and - ancestors.""" - # XXX This needs to be re-written to walk the ancestors of the - # context to build up the namespace information from - # declarations, elements, and attributes found in context. - # Otherwise we have to store a bunch more data on the DOM - # (though that *might* be more reliable -- not clear). - attrs = "" - context = self.context - L = [] - while context: - if hasattr(context, '_ns_prefix_uri'): - for prefix, uri in context._ns_prefix_uri.items(): - # add every new NS decl from context to L and attrs string - if prefix in L: - continue - L.append(prefix) - if prefix: - declname = "xmlns:" + prefix - else: - declname = "xmlns" - if attrs: - attrs = "%s\n %s='%s'" % (attrs, declname, uri) - else: - attrs = " %s='%s'" % (declname, uri) - context = context.parentNode - return attrs - - -class ParseEscape(Exception): - """Exception raised to short-circuit parsing in InternalSubsetExtractor.""" - pass - -class InternalSubsetExtractor(ExpatBuilder): - """XML processor which can rip out the internal document type subset.""" - - subset = None - - def getSubset(self): - """Return the internal subset as a string.""" - return self.subset - - def parseFile(self, file): - try: - ExpatBuilder.parseFile(self, file) - except ParseEscape: - pass - - def parseString(self, string): - try: - ExpatBuilder.parseString(self, string) - except ParseEscape: - pass - - def install(self, parser): - parser.StartDoctypeDeclHandler = self.start_doctype_decl_handler - parser.StartElementHandler = self.start_element_handler - - def start_doctype_decl_handler(self, name, publicId, systemId, - has_internal_subset): - if has_internal_subset: - parser = self.getParser() - self.subset = [] - parser.DefaultHandler = self.subset.append - parser.EndDoctypeDeclHandler = self.end_doctype_decl_handler - else: - raise ParseEscape() - - def end_doctype_decl_handler(self): - s = ''.join(self.subset).replace('\r\n', '\n').replace('\r', '\n') - self.subset = s - raise ParseEscape() - - def start_element_handler(self, name, attrs): - raise ParseEscape() - - -def parse(file, namespaces=True): - """Parse a document, returning the resulting Document node. - - 'file' may be either a file name or an open file object. - """ - if namespaces: - builder = ExpatBuilderNS() - else: - builder = ExpatBuilder() - - if isinstance(file, StringTypes): - fp = open(file, 'rb') - try: - result = builder.parseFile(fp) - finally: - fp.close() - else: - result = builder.parseFile(file) - return result - - -def parseString(string, namespaces=True): - """Parse a document from a string, returning the resulting - Document node. - """ - if namespaces: - builder = ExpatBuilderNS() - else: - builder = ExpatBuilder() - return builder.parseString(string) - - -def parseFragment(file, context, namespaces=True): - """Parse a fragment of a document, given the context from which it - was originally extracted. context should be the parent of the - node(s) which are in the fragment. - - 'file' may be either a file name or an open file object. - """ - if namespaces: - builder = FragmentBuilderNS(context) - else: - builder = FragmentBuilder(context) - - if isinstance(file, StringTypes): - fp = open(file, 'rb') - try: - result = builder.parseFile(fp) - finally: - fp.close() - else: - result = builder.parseFile(file) - return result - - -def parseFragmentString(string, context, namespaces=True): - """Parse a fragment of a document from a string, given the context - from which it was originally extracted. context should be the - parent of the node(s) which are in the fragment. - """ - if namespaces: - builder = FragmentBuilderNS(context) - else: - builder = FragmentBuilder(context) - return builder.parseString(string) - - -def makeBuilder(options): - """Create a builder based on an Options object.""" - if options.namespaces: - return ExpatBuilderNS(options) - else: - return ExpatBuilder(options) diff --git a/Lib/xmlcore/dom/minicompat.py b/Lib/xmlcore/dom/minicompat.py deleted file mode 100644 index f99b7fe..0000000 --- a/Lib/xmlcore/dom/minicompat.py +++ /dev/null @@ -1,109 +0,0 @@ -"""Python version compatibility support for minidom.""" - -# This module should only be imported using "import *". -# -# The following names are defined: -# -# NodeList -- lightest possible NodeList implementation -# -# EmptyNodeList -- lightest possible NodeList that is guarateed to -# remain empty (immutable) -# -# StringTypes -- tuple of defined string types -# -# defproperty -- function used in conjunction with GetattrMagic; -# using these together is needed to make them work -# as efficiently as possible in both Python 2.2+ -# and older versions. For example: -# -# class MyClass(GetattrMagic): -# def _get_myattr(self): -# return something -# -# defproperty(MyClass, "myattr", -# "return some value") -# -# For Python 2.2 and newer, this will construct a -# property object on the class, which avoids -# needing to override __getattr__(). It will only -# work for read-only attributes. -# -# For older versions of Python, inheriting from -# GetattrMagic will use the traditional -# __getattr__() hackery to achieve the same effect, -# but less efficiently. -# -# defproperty() should be used for each version of -# the relevant _get_() function. - -__all__ = ["NodeList", "EmptyNodeList", "StringTypes", "defproperty"] - -import xmlcore.dom - -try: - unicode -except NameError: - StringTypes = type(''), -else: - StringTypes = type(''), type(unicode('')) - - -class NodeList(list): - __slots__ = () - - def item(self, index): - if 0 <= index < len(self): - return self[index] - - def _get_length(self): - return len(self) - - def _set_length(self, value): - raise xml.dom.NoModificationAllowedErr( - "attempt to modify read-only attribute 'length'") - - length = property(_get_length, _set_length, - doc="The number of nodes in the NodeList.") - - def __getstate__(self): - return list(self) - - def __setstate__(self, state): - self[:] = state - -class EmptyNodeList(tuple): - __slots__ = () - - def __add__(self, other): - NL = NodeList() - NL.extend(other) - return NL - - def __radd__(self, other): - NL = NodeList() - NL.extend(other) - return NL - - def item(self, index): - return None - - def _get_length(self): - return 0 - - def _set_length(self, value): - raise xml.dom.NoModificationAllowedErr( - "attempt to modify read-only attribute 'length'") - - length = property(_get_length, _set_length, - doc="The number of nodes in the NodeList.") - - -def defproperty(klass, name, doc): - get = getattr(klass, ("_get_" + name)).im_func - def set(self, value, name=name): - raise xml.dom.NoModificationAllowedErr( - "attempt to modify read-only attribute " + repr(name)) - assert not hasattr(klass, "_set_" + name), \ - "expected not to find _set_" + name - prop = property(get, set, doc=doc) - setattr(klass, name, prop) diff --git a/Lib/xmlcore/dom/minidom.py b/Lib/xmlcore/dom/minidom.py deleted file mode 100644 index a8abd14..0000000 --- a/Lib/xmlcore/dom/minidom.py +++ /dev/null @@ -1,1936 +0,0 @@ -"""\ -minidom.py -- a lightweight DOM implementation. - -parse("foo.xml") - -parseString("") - -Todo: -===== - * convenience methods for getting elements and text. - * more testing - * bring some of the writer and linearizer code into conformance with this - interface - * SAX 2 namespaces -""" - -import xmlcore.dom - -from xmlcore.dom import EMPTY_NAMESPACE, EMPTY_PREFIX, XMLNS_NAMESPACE, domreg -from xmlcore.dom.minicompat import * -from xmlcore.dom.xmlbuilder import DOMImplementationLS, DocumentLS - -# This is used by the ID-cache invalidation checks; the list isn't -# actually complete, since the nodes being checked will never be the -# DOCUMENT_NODE or DOCUMENT_FRAGMENT_NODE. (The node being checked is -# the node being added or removed, not the node being modified.) -# -_nodeTypes_with_children = (xmlcore.dom.Node.ELEMENT_NODE, - xmlcore.dom.Node.ENTITY_REFERENCE_NODE) - - -class Node(xmlcore.dom.Node): - namespaceURI = None # this is non-null only for elements and attributes - parentNode = None - ownerDocument = None - nextSibling = None - previousSibling = None - - prefix = EMPTY_PREFIX # non-null only for NS elements and attributes - - def __nonzero__(self): - return True - - def toxml(self, encoding = None): - return self.toprettyxml("", "", encoding) - - def toprettyxml(self, indent="\t", newl="\n", encoding = None): - # indent = the indentation string to prepend, per level - # newl = the newline string to append - writer = _get_StringIO() - if encoding is not None: - import codecs - # Can't use codecs.getwriter to preserve 2.0 compatibility - writer = codecs.lookup(encoding)[3](writer) - if self.nodeType == Node.DOCUMENT_NODE: - # Can pass encoding only to document, to put it into XML header - self.writexml(writer, "", indent, newl, encoding) - else: - self.writexml(writer, "", indent, newl) - return writer.getvalue() - - def hasChildNodes(self): - if self.childNodes: - return True - else: - return False - - def _get_childNodes(self): - return self.childNodes - - def _get_firstChild(self): - if self.childNodes: - return self.childNodes[0] - - def _get_lastChild(self): - if self.childNodes: - return self.childNodes[-1] - - def insertBefore(self, newChild, refChild): - if newChild.nodeType == self.DOCUMENT_FRAGMENT_NODE: - for c in tuple(newChild.childNodes): - self.insertBefore(c, refChild) - ### The DOM does not clearly specify what to return in this case - return newChild - if newChild.nodeType not in self._child_node_types: - raise xmlcore.dom.HierarchyRequestErr( - "%s cannot be child of %s" % (repr(newChild), repr(self))) - if newChild.parentNode is not None: - newChild.parentNode.removeChild(newChild) - if refChild is None: - self.appendChild(newChild) - else: - try: - index = self.childNodes.index(refChild) - except ValueError: - raise xmlcore.dom.NotFoundErr() - if newChild.nodeType in _nodeTypes_with_children: - _clear_id_cache(self) - self.childNodes.insert(index, newChild) - newChild.nextSibling = refChild - refChild.previousSibling = newChild - if index: - node = self.childNodes[index-1] - node.nextSibling = newChild - newChild.previousSibling = node - else: - newChild.previousSibling = None - newChild.parentNode = self - return newChild - - def appendChild(self, node): - if node.nodeType == self.DOCUMENT_FRAGMENT_NODE: - for c in tuple(node.childNodes): - self.appendChild(c) - ### The DOM does not clearly specify what to return in this case - return node - if node.nodeType not in self._child_node_types: - raise xmlcore.dom.HierarchyRequestErr( - "%s cannot be child of %s" % (repr(node), repr(self))) - elif node.nodeType in _nodeTypes_with_children: - _clear_id_cache(self) - if node.parentNode is not None: - node.parentNode.removeChild(node) - _append_child(self, node) - node.nextSibling = None - return node - - def replaceChild(self, newChild, oldChild): - if newChild.nodeType == self.DOCUMENT_FRAGMENT_NODE: - refChild = oldChild.nextSibling - self.removeChild(oldChild) - return self.insertBefore(newChild, refChild) - if newChild.nodeType not in self._child_node_types: - raise xmlcore.dom.HierarchyRequestErr( - "%s cannot be child of %s" % (repr(newChild), repr(self))) - if newChild is oldChild: - return - if newChild.parentNode is not None: - newChild.parentNode.removeChild(newChild) - try: - index = self.childNodes.index(oldChild) - except ValueError: - raise xmlcore.dom.NotFoundErr() - self.childNodes[index] = newChild - newChild.parentNode = self - oldChild.parentNode = None - if (newChild.nodeType in _nodeTypes_with_children - or oldChild.nodeType in _nodeTypes_with_children): - _clear_id_cache(self) - newChild.nextSibling = oldChild.nextSibling - newChild.previousSibling = oldChild.previousSibling - oldChild.nextSibling = None - oldChild.previousSibling = None - if newChild.previousSibling: - newChild.previousSibling.nextSibling = newChild - if newChild.nextSibling: - newChild.nextSibling.previousSibling = newChild - return oldChild - - def removeChild(self, oldChild): - try: - self.childNodes.remove(oldChild) - except ValueError: - raise xmlcore.dom.NotFoundErr() - if oldChild.nextSibling is not None: - oldChild.nextSibling.previousSibling = oldChild.previousSibling - if oldChild.previousSibling is not None: - oldChild.previousSibling.nextSibling = oldChild.nextSibling - oldChild.nextSibling = oldChild.previousSibling = None - if oldChild.nodeType in _nodeTypes_with_children: - _clear_id_cache(self) - - oldChild.parentNode = None - return oldChild - - def normalize(self): - L = [] - for child in self.childNodes: - if child.nodeType == Node.TEXT_NODE: - data = child.data - if data and L and L[-1].nodeType == child.nodeType: - # collapse text node - node = L[-1] - node.data = node.data + child.data - node.nextSibling = child.nextSibling - child.unlink() - elif data: - if L: - L[-1].nextSibling = child - child.previousSibling = L[-1] - else: - child.previousSibling = None - L.append(child) - else: - # empty text node; discard - child.unlink() - else: - if L: - L[-1].nextSibling = child - child.previousSibling = L[-1] - else: - child.previousSibling = None - L.append(child) - if child.nodeType == Node.ELEMENT_NODE: - child.normalize() - self.childNodes[:] = L - - def cloneNode(self, deep): - return _clone_node(self, deep, self.ownerDocument or self) - - def isSupported(self, feature, version): - return self.ownerDocument.implementation.hasFeature(feature, version) - - def _get_localName(self): - # Overridden in Element and Attr where localName can be Non-Null - return None - - # Node interfaces from Level 3 (WD 9 April 2002) - - def isSameNode(self, other): - return self is other - - def getInterface(self, feature): - if self.isSupported(feature, None): - return self - else: - return None - - # The "user data" functions use a dictionary that is only present - # if some user data has been set, so be careful not to assume it - # exists. - - def getUserData(self, key): - try: - return self._user_data[key][0] - except (AttributeError, KeyError): - return None - - def setUserData(self, key, data, handler): - old = None - try: - d = self._user_data - except AttributeError: - d = {} - self._user_data = d - if d.has_key(key): - old = d[key][0] - if data is None: - # ignore handlers passed for None - handler = None - if old is not None: - del d[key] - else: - d[key] = (data, handler) - return old - - def _call_user_data_handler(self, operation, src, dst): - if hasattr(self, "_user_data"): - for key, (data, handler) in self._user_data.items(): - if handler is not None: - handler.handle(operation, key, data, src, dst) - - # minidom-specific API: - - def unlink(self): - self.parentNode = self.ownerDocument = None - if self.childNodes: - for child in self.childNodes: - child.unlink() - self.childNodes = NodeList() - self.previousSibling = None - self.nextSibling = None - -defproperty(Node, "firstChild", doc="First child node, or None.") -defproperty(Node, "lastChild", doc="Last child node, or None.") -defproperty(Node, "localName", doc="Namespace-local name of this node.") - - -def _append_child(self, node): - # fast path with less checks; usable by DOM builders if careful - childNodes = self.childNodes - if childNodes: - last = childNodes[-1] - node.__dict__["previousSibling"] = last - last.__dict__["nextSibling"] = node - childNodes.append(node) - node.__dict__["parentNode"] = self - -def _in_document(node): - # return True iff node is part of a document tree - while node is not None: - if node.nodeType == Node.DOCUMENT_NODE: - return True - node = node.parentNode - return False - -def _write_data(writer, data): - "Writes datachars to writer." - data = data.replace("&", "&").replace("<", "<") - data = data.replace("\"", """).replace(">", ">") - writer.write(data) - -def _get_elements_by_tagName_helper(parent, name, rc): - for node in parent.childNodes: - if node.nodeType == Node.ELEMENT_NODE and \ - (name == "*" or node.tagName == name): - rc.append(node) - _get_elements_by_tagName_helper(node, name, rc) - return rc - -def _get_elements_by_tagName_ns_helper(parent, nsURI, localName, rc): - for node in parent.childNodes: - if node.nodeType == Node.ELEMENT_NODE: - if ((localName == "*" or node.localName == localName) and - (nsURI == "*" or node.namespaceURI == nsURI)): - rc.append(node) - _get_elements_by_tagName_ns_helper(node, nsURI, localName, rc) - return rc - -class DocumentFragment(Node): - nodeType = Node.DOCUMENT_FRAGMENT_NODE - nodeName = "#document-fragment" - nodeValue = None - attributes = None - parentNode = None - _child_node_types = (Node.ELEMENT_NODE, - Node.TEXT_NODE, - Node.CDATA_SECTION_NODE, - Node.ENTITY_REFERENCE_NODE, - Node.PROCESSING_INSTRUCTION_NODE, - Node.COMMENT_NODE, - Node.NOTATION_NODE) - - def __init__(self): - self.childNodes = NodeList() - - -class Attr(Node): - nodeType = Node.ATTRIBUTE_NODE - attributes = None - ownerElement = None - specified = False - _is_id = False - - _child_node_types = (Node.TEXT_NODE, Node.ENTITY_REFERENCE_NODE) - - def __init__(self, qName, namespaceURI=EMPTY_NAMESPACE, localName=None, - prefix=None): - # skip setattr for performance - d = self.__dict__ - d["nodeName"] = d["name"] = qName - d["namespaceURI"] = namespaceURI - d["prefix"] = prefix - d['childNodes'] = NodeList() - - # Add the single child node that represents the value of the attr - self.childNodes.append(Text()) - - # nodeValue and value are set elsewhere - - def _get_localName(self): - return self.nodeName.split(":", 1)[-1] - - def _get_name(self): - return self.name - - def _get_specified(self): - return self.specified - - def __setattr__(self, name, value): - d = self.__dict__ - if name in ("value", "nodeValue"): - d["value"] = d["nodeValue"] = value - d2 = self.childNodes[0].__dict__ - d2["data"] = d2["nodeValue"] = value - if self.ownerElement is not None: - _clear_id_cache(self.ownerElement) - elif name in ("name", "nodeName"): - d["name"] = d["nodeName"] = value - if self.ownerElement is not None: - _clear_id_cache(self.ownerElement) - else: - d[name] = value - - def _set_prefix(self, prefix): - nsuri = self.namespaceURI - if prefix == "xmlns": - if nsuri and nsuri != XMLNS_NAMESPACE: - raise xmlcore.dom.NamespaceErr( - "illegal use of 'xmlns' prefix for the wrong namespace") - d = self.__dict__ - d['prefix'] = prefix - if prefix is None: - newName = self.localName - else: - newName = "%s:%s" % (prefix, self.localName) - if self.ownerElement: - _clear_id_cache(self.ownerElement) - d['nodeName'] = d['name'] = newName - - def _set_value(self, value): - d = self.__dict__ - d['value'] = d['nodeValue'] = value - if self.ownerElement: - _clear_id_cache(self.ownerElement) - self.childNodes[0].data = value - - def unlink(self): - # This implementation does not call the base implementation - # since most of that is not needed, and the expense of the - # method call is not warranted. We duplicate the removal of - # children, but that's all we needed from the base class. - elem = self.ownerElement - if elem is not None: - del elem._attrs[self.nodeName] - del elem._attrsNS[(self.namespaceURI, self.localName)] - if self._is_id: - self._is_id = False - elem._magic_id_nodes -= 1 - self.ownerDocument._magic_id_count -= 1 - for child in self.childNodes: - child.unlink() - del self.childNodes[:] - - def _get_isId(self): - if self._is_id: - return True - doc = self.ownerDocument - elem = self.ownerElement - if doc is None or elem is None: - return False - - info = doc._get_elem_info(elem) - if info is None: - return False - if self.namespaceURI: - return info.isIdNS(self.namespaceURI, self.localName) - else: - return info.isId(self.nodeName) - - def _get_schemaType(self): - doc = self.ownerDocument - elem = self.ownerElement - if doc is None or elem is None: - return _no_type - - info = doc._get_elem_info(elem) - if info is None: - return _no_type - if self.namespaceURI: - return info.getAttributeTypeNS(self.namespaceURI, self.localName) - else: - return info.getAttributeType(self.nodeName) - -defproperty(Attr, "isId", doc="True if this attribute is an ID.") -defproperty(Attr, "localName", doc="Namespace-local name of this attribute.") -defproperty(Attr, "schemaType", doc="Schema type for this attribute.") - - -class NamedNodeMap(object): - """The attribute list is a transient interface to the underlying - dictionaries. Mutations here will change the underlying element's - dictionary. - - Ordering is imposed artificially and does not reflect the order of - attributes as found in an input document. - """ - - __slots__ = ('_attrs', '_attrsNS', '_ownerElement') - - def __init__(self, attrs, attrsNS, ownerElement): - self._attrs = attrs - self._attrsNS = attrsNS - self._ownerElement = ownerElement - - def _get_length(self): - return len(self._attrs) - - def item(self, index): - try: - return self[self._attrs.keys()[index]] - except IndexError: - return None - - def items(self): - L = [] - for node in self._attrs.values(): - L.append((node.nodeName, node.value)) - return L - - def itemsNS(self): - L = [] - for node in self._attrs.values(): - L.append(((node.namespaceURI, node.localName), node.value)) - return L - - def has_key(self, key): - if isinstance(key, StringTypes): - return self._attrs.has_key(key) - else: - return self._attrsNS.has_key(key) - - def keys(self): - return self._attrs.keys() - - def keysNS(self): - return self._attrsNS.keys() - - def values(self): - return self._attrs.values() - - def get(self, name, value=None): - return self._attrs.get(name, value) - - __len__ = _get_length - - def __cmp__(self, other): - if self._attrs is getattr(other, "_attrs", None): - return 0 - else: - return cmp(id(self), id(other)) - - def __getitem__(self, attname_or_tuple): - if isinstance(attname_or_tuple, tuple): - return self._attrsNS[attname_or_tuple] - else: - return self._attrs[attname_or_tuple] - - # same as set - def __setitem__(self, attname, value): - if isinstance(value, StringTypes): - try: - node = self._attrs[attname] - except KeyError: - node = Attr(attname) - node.ownerDocument = self._ownerElement.ownerDocument - self.setNamedItem(node) - node.value = value - else: - if not isinstance(value, Attr): - raise TypeError, "value must be a string or Attr object" - node = value - self.setNamedItem(node) - - def getNamedItem(self, name): - try: - return self._attrs[name] - except KeyError: - return None - - def getNamedItemNS(self, namespaceURI, localName): - try: - return self._attrsNS[(namespaceURI, localName)] - except KeyError: - return None - - def removeNamedItem(self, name): - n = self.getNamedItem(name) - if n is not None: - _clear_id_cache(self._ownerElement) - del self._attrs[n.nodeName] - del self._attrsNS[(n.namespaceURI, n.localName)] - if n.__dict__.has_key('ownerElement'): - n.__dict__['ownerElement'] = None - return n - else: - raise xmlcore.dom.NotFoundErr() - - def removeNamedItemNS(self, namespaceURI, localName): - n = self.getNamedItemNS(namespaceURI, localName) - if n is not None: - _clear_id_cache(self._ownerElement) - del self._attrsNS[(n.namespaceURI, n.localName)] - del self._attrs[n.nodeName] - if n.__dict__.has_key('ownerElement'): - n.__dict__['ownerElement'] = None - return n - else: - raise xmlcore.dom.NotFoundErr() - - def setNamedItem(self, node): - if not isinstance(node, Attr): - raise xmlcore.dom.HierarchyRequestErr( - "%s cannot be child of %s" % (repr(node), repr(self))) - old = self._attrs.get(node.name) - if old: - old.unlink() - self._attrs[node.name] = node - self._attrsNS[(node.namespaceURI, node.localName)] = node - node.ownerElement = self._ownerElement - _clear_id_cache(node.ownerElement) - return old - - def setNamedItemNS(self, node): - return self.setNamedItem(node) - - def __delitem__(self, attname_or_tuple): - node = self[attname_or_tuple] - _clear_id_cache(node.ownerElement) - node.unlink() - - def __getstate__(self): - return self._attrs, self._attrsNS, self._ownerElement - - def __setstate__(self, state): - self._attrs, self._attrsNS, self._ownerElement = state - -defproperty(NamedNodeMap, "length", - doc="Number of nodes in the NamedNodeMap.") - -AttributeList = NamedNodeMap - - -class TypeInfo(object): - __slots__ = 'namespace', 'name' - - def __init__(self, namespace, name): - self.namespace = namespace - self.name = name - - def __repr__(self): - if self.namespace: - return "" % (self.name, self.namespace) - else: - return "" % self.name - - def _get_name(self): - return self.name - - def _get_namespace(self): - return self.namespace - -_no_type = TypeInfo(None, None) - -class Element(Node): - nodeType = Node.ELEMENT_NODE - nodeValue = None - schemaType = _no_type - - _magic_id_nodes = 0 - - _child_node_types = (Node.ELEMENT_NODE, - Node.PROCESSING_INSTRUCTION_NODE, - Node.COMMENT_NODE, - Node.TEXT_NODE, - Node.CDATA_SECTION_NODE, - Node.ENTITY_REFERENCE_NODE) - - def __init__(self, tagName, namespaceURI=EMPTY_NAMESPACE, prefix=None, - localName=None): - self.tagName = self.nodeName = tagName - self.prefix = prefix - self.namespaceURI = namespaceURI - self.childNodes = NodeList() - - self._attrs = {} # attributes are double-indexed: - self._attrsNS = {} # tagName -> Attribute - # URI,localName -> Attribute - # in the future: consider lazy generation - # of attribute objects this is too tricky - # for now because of headaches with - # namespaces. - - def _get_localName(self): - return self.tagName.split(":", 1)[-1] - - def _get_tagName(self): - return self.tagName - - def unlink(self): - for attr in self._attrs.values(): - attr.unlink() - self._attrs = None - self._attrsNS = None - Node.unlink(self) - - def getAttribute(self, attname): - try: - return self._attrs[attname].value - except KeyError: - return "" - - def getAttributeNS(self, namespaceURI, localName): - try: - return self._attrsNS[(namespaceURI, localName)].value - except KeyError: - return "" - - def setAttribute(self, attname, value): - attr = self.getAttributeNode(attname) - if attr is None: - attr = Attr(attname) - # for performance - d = attr.__dict__ - d["value"] = d["nodeValue"] = value - d["ownerDocument"] = self.ownerDocument - self.setAttributeNode(attr) - elif value != attr.value: - d = attr.__dict__ - d["value"] = d["nodeValue"] = value - if attr.isId: - _clear_id_cache(self) - - def setAttributeNS(self, namespaceURI, qualifiedName, value): - prefix, localname = _nssplit(qualifiedName) - attr = self.getAttributeNodeNS(namespaceURI, localname) - if attr is None: - # for performance - attr = Attr(qualifiedName, namespaceURI, localname, prefix) - d = attr.__dict__ - d["prefix"] = prefix - d["nodeName"] = qualifiedName - d["value"] = d["nodeValue"] = value - d["ownerDocument"] = self.ownerDocument - self.setAttributeNode(attr) - else: - d = attr.__dict__ - if value != attr.value: - d["value"] = d["nodeValue"] = value - if attr.isId: - _clear_id_cache(self) - if attr.prefix != prefix: - d["prefix"] = prefix - d["nodeName"] = qualifiedName - - def getAttributeNode(self, attrname): - return self._attrs.get(attrname) - - def getAttributeNodeNS(self, namespaceURI, localName): - return self._attrsNS.get((namespaceURI, localName)) - - def setAttributeNode(self, attr): - if attr.ownerElement not in (None, self): - raise xmlcore.dom.InuseAttributeErr("attribute node already owned") - old1 = self._attrs.get(attr.name, None) - if old1 is not None: - self.removeAttributeNode(old1) - old2 = self._attrsNS.get((attr.namespaceURI, attr.localName), None) - if old2 is not None and old2 is not old1: - self.removeAttributeNode(old2) - _set_attribute_node(self, attr) - - if old1 is not attr: - # It might have already been part of this node, in which case - # it doesn't represent a change, and should not be returned. - return old1 - if old2 is not attr: - return old2 - - setAttributeNodeNS = setAttributeNode - - def removeAttribute(self, name): - try: - attr = self._attrs[name] - except KeyError: - raise xmlcore.dom.NotFoundErr() - self.removeAttributeNode(attr) - - def removeAttributeNS(self, namespaceURI, localName): - try: - attr = self._attrsNS[(namespaceURI, localName)] - except KeyError: - raise xmlcore.dom.NotFoundErr() - self.removeAttributeNode(attr) - - def removeAttributeNode(self, node): - if node is None: - raise xmlcore.dom.NotFoundErr() - try: - self._attrs[node.name] - except KeyError: - raise xmlcore.dom.NotFoundErr() - _clear_id_cache(self) - node.unlink() - # Restore this since the node is still useful and otherwise - # unlinked - node.ownerDocument = self.ownerDocument - - removeAttributeNodeNS = removeAttributeNode - - def hasAttribute(self, name): - return self._attrs.has_key(name) - - def hasAttributeNS(self, namespaceURI, localName): - return self._attrsNS.has_key((namespaceURI, localName)) - - def getElementsByTagName(self, name): - return _get_elements_by_tagName_helper(self, name, NodeList()) - - def getElementsByTagNameNS(self, namespaceURI, localName): - return _get_elements_by_tagName_ns_helper( - self, namespaceURI, localName, NodeList()) - - def __repr__(self): - return "" % (self.tagName, id(self)) - - def writexml(self, writer, indent="", addindent="", newl=""): - # indent = current indentation - # addindent = indentation to add to higher levels - # newl = newline string - writer.write(indent+"<" + self.tagName) - - attrs = self._get_attributes() - a_names = attrs.keys() - a_names.sort() - - for a_name in a_names: - writer.write(" %s=\"" % a_name) - _write_data(writer, attrs[a_name].value) - writer.write("\"") - if self.childNodes: - writer.write(">%s"%(newl)) - for node in self.childNodes: - node.writexml(writer,indent+addindent,addindent,newl) - writer.write("%s%s" % (indent,self.tagName,newl)) - else: - writer.write("/>%s"%(newl)) - - def _get_attributes(self): - return NamedNodeMap(self._attrs, self._attrsNS, self) - - def hasAttributes(self): - if self._attrs: - return True - else: - return False - - # DOM Level 3 attributes, based on the 22 Oct 2002 draft - - def setIdAttribute(self, name): - idAttr = self.getAttributeNode(name) - self.setIdAttributeNode(idAttr) - - def setIdAttributeNS(self, namespaceURI, localName): - idAttr = self.getAttributeNodeNS(namespaceURI, localName) - self.setIdAttributeNode(idAttr) - - def setIdAttributeNode(self, idAttr): - if idAttr is None or not self.isSameNode(idAttr.ownerElement): - raise xmlcore.dom.NotFoundErr() - if _get_containing_entref(self) is not None: - raise xmlcore.dom.NoModificationAllowedErr() - if not idAttr._is_id: - idAttr.__dict__['_is_id'] = True - self._magic_id_nodes += 1 - self.ownerDocument._magic_id_count += 1 - _clear_id_cache(self) - -defproperty(Element, "attributes", - doc="NamedNodeMap of attributes on the element.") -defproperty(Element, "localName", - doc="Namespace-local name of this element.") - - -def _set_attribute_node(element, attr): - _clear_id_cache(element) - element._attrs[attr.name] = attr - element._attrsNS[(attr.namespaceURI, attr.localName)] = attr - - # This creates a circular reference, but Element.unlink() - # breaks the cycle since the references to the attribute - # dictionaries are tossed. - attr.__dict__['ownerElement'] = element - - -class Childless: - """Mixin that makes childless-ness easy to implement and avoids - the complexity of the Node methods that deal with children. - """ - - attributes = None - childNodes = EmptyNodeList() - firstChild = None - lastChild = None - - def _get_firstChild(self): - return None - - def _get_lastChild(self): - return None - - def appendChild(self, node): - raise xmlcore.dom.HierarchyRequestErr( - self.nodeName + " nodes cannot have children") - - def hasChildNodes(self): - return False - - def insertBefore(self, newChild, refChild): - raise xmlcore.dom.HierarchyRequestErr( - self.nodeName + " nodes do not have children") - - def removeChild(self, oldChild): - raise xmlcore.dom.NotFoundErr( - self.nodeName + " nodes do not have children") - - def replaceChild(self, newChild, oldChild): - raise xmlcore.dom.HierarchyRequestErr( - self.nodeName + " nodes do not have children") - - -class ProcessingInstruction(Childless, Node): - nodeType = Node.PROCESSING_INSTRUCTION_NODE - - def __init__(self, target, data): - self.target = self.nodeName = target - self.data = self.nodeValue = data - - def _get_data(self): - return self.data - def _set_data(self, value): - d = self.__dict__ - d['data'] = d['nodeValue'] = value - - def _get_target(self): - return self.target - def _set_target(self, value): - d = self.__dict__ - d['target'] = d['nodeName'] = value - - def __setattr__(self, name, value): - if name == "data" or name == "nodeValue": - self.__dict__['data'] = self.__dict__['nodeValue'] = value - elif name == "target" or name == "nodeName": - self.__dict__['target'] = self.__dict__['nodeName'] = value - else: - self.__dict__[name] = value - - def writexml(self, writer, indent="", addindent="", newl=""): - writer.write("%s%s" % (indent,self.target, self.data, newl)) - - -class CharacterData(Childless, Node): - def _get_length(self): - return len(self.data) - __len__ = _get_length - - def _get_data(self): - return self.__dict__['data'] - def _set_data(self, data): - d = self.__dict__ - d['data'] = d['nodeValue'] = data - - _get_nodeValue = _get_data - _set_nodeValue = _set_data - - def __setattr__(self, name, value): - if name == "data" or name == "nodeValue": - self.__dict__['data'] = self.__dict__['nodeValue'] = value - else: - self.__dict__[name] = value - - def __repr__(self): - data = self.data - if len(data) > 10: - dotdotdot = "..." - else: - dotdotdot = "" - return "" % ( - self.__class__.__name__, data[0:10], dotdotdot) - - def substringData(self, offset, count): - if offset < 0: - raise xmlcore.dom.IndexSizeErr("offset cannot be negative") - if offset >= len(self.data): - raise xmlcore.dom.IndexSizeErr("offset cannot be beyond end of data") - if count < 0: - raise xmlcore.dom.IndexSizeErr("count cannot be negative") - return self.data[offset:offset+count] - - def appendData(self, arg): - self.data = self.data + arg - - def insertData(self, offset, arg): - if offset < 0: - raise xmlcore.dom.IndexSizeErr("offset cannot be negative") - if offset >= len(self.data): - raise xmlcore.dom.IndexSizeErr("offset cannot be beyond end of data") - if arg: - self.data = "%s%s%s" % ( - self.data[:offset], arg, self.data[offset:]) - - def deleteData(self, offset, count): - if offset < 0: - raise xmlcore.dom.IndexSizeErr("offset cannot be negative") - if offset >= len(self.data): - raise xmlcore.dom.IndexSizeErr("offset cannot be beyond end of data") - if count < 0: - raise xmlcore.dom.IndexSizeErr("count cannot be negative") - if count: - self.data = self.data[:offset] + self.data[offset+count:] - - def replaceData(self, offset, count, arg): - if offset < 0: - raise xmlcore.dom.IndexSizeErr("offset cannot be negative") - if offset >= len(self.data): - raise xmlcore.dom.IndexSizeErr("offset cannot be beyond end of data") - if count < 0: - raise xmlcore.dom.IndexSizeErr("count cannot be negative") - if count: - self.data = "%s%s%s" % ( - self.data[:offset], arg, self.data[offset+count:]) - -defproperty(CharacterData, "length", doc="Length of the string data.") - - -class Text(CharacterData): - # Make sure we don't add an instance __dict__ if we don't already - # have one, at least when that's possible: - # XXX this does not work, CharacterData is an old-style class - # __slots__ = () - - nodeType = Node.TEXT_NODE - nodeName = "#text" - attributes = None - - def splitText(self, offset): - if offset < 0 or offset > len(self.data): - raise xmlcore.dom.IndexSizeErr("illegal offset value") - newText = self.__class__() - newText.data = self.data[offset:] - newText.ownerDocument = self.ownerDocument - next = self.nextSibling - if self.parentNode and self in self.parentNode.childNodes: - if next is None: - self.parentNode.appendChild(newText) - else: - self.parentNode.insertBefore(newText, next) - self.data = self.data[:offset] - return newText - - def writexml(self, writer, indent="", addindent="", newl=""): - _write_data(writer, "%s%s%s"%(indent, self.data, newl)) - - # DOM Level 3 (WD 9 April 2002) - - def _get_wholeText(self): - L = [self.data] - n = self.previousSibling - while n is not None: - if n.nodeType in (Node.TEXT_NODE, Node.CDATA_SECTION_NODE): - L.insert(0, n.data) - n = n.previousSibling - else: - break - n = self.nextSibling - while n is not None: - if n.nodeType in (Node.TEXT_NODE, Node.CDATA_SECTION_NODE): - L.append(n.data) - n = n.nextSibling - else: - break - return ''.join(L) - - def replaceWholeText(self, content): - # XXX This needs to be seriously changed if minidom ever - # supports EntityReference nodes. - parent = self.parentNode - n = self.previousSibling - while n is not None: - if n.nodeType in (Node.TEXT_NODE, Node.CDATA_SECTION_NODE): - next = n.previousSibling - parent.removeChild(n) - n = next - else: - break - n = self.nextSibling - if not content: - parent.removeChild(self) - while n is not None: - if n.nodeType in (Node.TEXT_NODE, Node.CDATA_SECTION_NODE): - next = n.nextSibling - parent.removeChild(n) - n = next - else: - break - if content: - d = self.__dict__ - d['data'] = content - d['nodeValue'] = content - return self - else: - return None - - def _get_isWhitespaceInElementContent(self): - if self.data.strip(): - return False - elem = _get_containing_element(self) - if elem is None: - return False - info = self.ownerDocument._get_elem_info(elem) - if info is None: - return False - else: - return info.isElementContent() - -defproperty(Text, "isWhitespaceInElementContent", - doc="True iff this text node contains only whitespace" - " and is in element content.") -defproperty(Text, "wholeText", - doc="The text of all logically-adjacent text nodes.") - - -def _get_containing_element(node): - c = node.parentNode - while c is not None: - if c.nodeType == Node.ELEMENT_NODE: - return c - c = c.parentNode - return None - -def _get_containing_entref(node): - c = node.parentNode - while c is not None: - if c.nodeType == Node.ENTITY_REFERENCE_NODE: - return c - c = c.parentNode - return None - - -class Comment(Childless, CharacterData): - nodeType = Node.COMMENT_NODE - nodeName = "#comment" - - def __init__(self, data): - self.data = self.nodeValue = data - - def writexml(self, writer, indent="", addindent="", newl=""): - writer.write("%s%s" % (indent, self.data, newl)) - - -class CDATASection(Text): - # Make sure we don't add an instance __dict__ if we don't already - # have one, at least when that's possible: - # XXX this does not work, Text is an old-style class - # __slots__ = () - - nodeType = Node.CDATA_SECTION_NODE - nodeName = "#cdata-section" - - def writexml(self, writer, indent="", addindent="", newl=""): - if self.data.find("]]>") >= 0: - raise ValueError("']]>' not allowed in a CDATA section") - writer.write("" % self.data) - - -class ReadOnlySequentialNamedNodeMap(object): - __slots__ = '_seq', - - def __init__(self, seq=()): - # seq should be a list or tuple - self._seq = seq - - def __len__(self): - return len(self._seq) - - def _get_length(self): - return len(self._seq) - - def getNamedItem(self, name): - for n in self._seq: - if n.nodeName == name: - return n - - def getNamedItemNS(self, namespaceURI, localName): - for n in self._seq: - if n.namespaceURI == namespaceURI and n.localName == localName: - return n - - def __getitem__(self, name_or_tuple): - if isinstance(name_or_tuple, tuple): - node = self.getNamedItemNS(*name_or_tuple) - else: - node = self.getNamedItem(name_or_tuple) - if node is None: - raise KeyError, name_or_tuple - return node - - def item(self, index): - if index < 0: - return None - try: - return self._seq[index] - except IndexError: - return None - - def removeNamedItem(self, name): - raise xmlcore.dom.NoModificationAllowedErr( - "NamedNodeMap instance is read-only") - - def removeNamedItemNS(self, namespaceURI, localName): - raise xmlcore.dom.NoModificationAllowedErr( - "NamedNodeMap instance is read-only") - - def setNamedItem(self, node): - raise xmlcore.dom.NoModificationAllowedErr( - "NamedNodeMap instance is read-only") - - def setNamedItemNS(self, node): - raise xmlcore.dom.NoModificationAllowedErr( - "NamedNodeMap instance is read-only") - - def __getstate__(self): - return [self._seq] - - def __setstate__(self, state): - self._seq = state[0] - -defproperty(ReadOnlySequentialNamedNodeMap, "length", - doc="Number of entries in the NamedNodeMap.") - - -class Identified: - """Mix-in class that supports the publicId and systemId attributes.""" - - # XXX this does not work, this is an old-style class - # __slots__ = 'publicId', 'systemId' - - def _identified_mixin_init(self, publicId, systemId): - self.publicId = publicId - self.systemId = systemId - - def _get_publicId(self): - return self.publicId - - def _get_systemId(self): - return self.systemId - -class DocumentType(Identified, Childless, Node): - nodeType = Node.DOCUMENT_TYPE_NODE - nodeValue = None - name = None - publicId = None - systemId = None - internalSubset = None - - def __init__(self, qualifiedName): - self.entities = ReadOnlySequentialNamedNodeMap() - self.notations = ReadOnlySequentialNamedNodeMap() - if qualifiedName: - prefix, localname = _nssplit(qualifiedName) - self.name = localname - self.nodeName = self.name - - def _get_internalSubset(self): - return self.internalSubset - - def cloneNode(self, deep): - if self.ownerDocument is None: - # it's ok - clone = DocumentType(None) - clone.name = self.name - clone.nodeName = self.name - operation = xmlcore.dom.UserDataHandler.NODE_CLONED - if deep: - clone.entities._seq = [] - clone.notations._seq = [] - for n in self.notations._seq: - notation = Notation(n.nodeName, n.publicId, n.systemId) - clone.notations._seq.append(notation) - n._call_user_data_handler(operation, n, notation) - for e in self.entities._seq: - entity = Entity(e.nodeName, e.publicId, e.systemId, - e.notationName) - entity.actualEncoding = e.actualEncoding - entity.encoding = e.encoding - entity.version = e.version - clone.entities._seq.append(entity) - e._call_user_data_handler(operation, n, entity) - self._call_user_data_handler(operation, self, clone) - return clone - else: - return None - - def writexml(self, writer, indent="", addindent="", newl=""): - writer.write(""+newl) - -class Entity(Identified, Node): - attributes = None - nodeType = Node.ENTITY_NODE - nodeValue = None - - actualEncoding = None - encoding = None - version = None - - def __init__(self, name, publicId, systemId, notation): - self.nodeName = name - self.notationName = notation - self.childNodes = NodeList() - self._identified_mixin_init(publicId, systemId) - - def _get_actualEncoding(self): - return self.actualEncoding - - def _get_encoding(self): - return self.encoding - - def _get_version(self): - return self.version - - def appendChild(self, newChild): - raise xmlcore.dom.HierarchyRequestErr( - "cannot append children to an entity node") - - def insertBefore(self, newChild, refChild): - raise xmlcore.dom.HierarchyRequestErr( - "cannot insert children below an entity node") - - def removeChild(self, oldChild): - raise xmlcore.dom.HierarchyRequestErr( - "cannot remove children from an entity node") - - def replaceChild(self, newChild, oldChild): - raise xmlcore.dom.HierarchyRequestErr( - "cannot replace children of an entity node") - -class Notation(Identified, Childless, Node): - nodeType = Node.NOTATION_NODE - nodeValue = None - - def __init__(self, name, publicId, systemId): - self.nodeName = name - self._identified_mixin_init(publicId, systemId) - - -class DOMImplementation(DOMImplementationLS): - _features = [("core", "1.0"), - ("core", "2.0"), - ("core", "3.0"), - ("core", None), - ("xml", "1.0"), - ("xml", "2.0"), - ("xml", "3.0"), - ("xml", None), - ("ls-load", "3.0"), - ("ls-load", None), - ] - - def hasFeature(self, feature, version): - if version == "": - version = None - return (feature.lower(), version) in self._features - - def createDocument(self, namespaceURI, qualifiedName, doctype): - if doctype and doctype.parentNode is not None: - raise xmlcore.dom.WrongDocumentErr( - "doctype object owned by another DOM tree") - doc = self._create_document() - - add_root_element = not (namespaceURI is None - and qualifiedName is None - and doctype is None) - - if not qualifiedName and add_root_element: - # The spec is unclear what to raise here; SyntaxErr - # would be the other obvious candidate. Since Xerces raises - # InvalidCharacterErr, and since SyntaxErr is not listed - # for createDocument, that seems to be the better choice. - # XXX: need to check for illegal characters here and in - # createElement. - - # DOM Level III clears this up when talking about the return value - # of this function. If namespaceURI, qName and DocType are - # Null the document is returned without a document element - # Otherwise if doctype or namespaceURI are not None - # Then we go back to the above problem - raise xmlcore.dom.InvalidCharacterErr("Element with no name") - - if add_root_element: - prefix, localname = _nssplit(qualifiedName) - if prefix == "xml" \ - and namespaceURI != "http://www.w3.org/XML/1998/namespace": - raise xmlcore.dom.NamespaceErr("illegal use of 'xml' prefix") - if prefix and not namespaceURI: - raise xmlcore.dom.NamespaceErr( - "illegal use of prefix without namespaces") - element = doc.createElementNS(namespaceURI, qualifiedName) - if doctype: - doc.appendChild(doctype) - doc.appendChild(element) - - if doctype: - doctype.parentNode = doctype.ownerDocument = doc - - doc.doctype = doctype - doc.implementation = self - return doc - - def createDocumentType(self, qualifiedName, publicId, systemId): - doctype = DocumentType(qualifiedName) - doctype.publicId = publicId - doctype.systemId = systemId - return doctype - - # DOM Level 3 (WD 9 April 2002) - - def getInterface(self, feature): - if self.hasFeature(feature, None): - return self - else: - return None - - # internal - def _create_document(self): - return Document() - -class ElementInfo(object): - """Object that represents content-model information for an element. - - This implementation is not expected to be used in practice; DOM - builders should provide implementations which do the right thing - using information available to it. - - """ - - __slots__ = 'tagName', - - def __init__(self, name): - self.tagName = name - - def getAttributeType(self, aname): - return _no_type - - def getAttributeTypeNS(self, namespaceURI, localName): - return _no_type - - def isElementContent(self): - return False - - def isEmpty(self): - """Returns true iff this element is declared to have an EMPTY - content model.""" - return False - - def isId(self, aname): - """Returns true iff the named attribte is a DTD-style ID.""" - return False - - def isIdNS(self, namespaceURI, localName): - """Returns true iff the identified attribute is a DTD-style ID.""" - return False - - def __getstate__(self): - return self.tagName - - def __setstate__(self, state): - self.tagName = state - -def _clear_id_cache(node): - if node.nodeType == Node.DOCUMENT_NODE: - node._id_cache.clear() - node._id_search_stack = None - elif _in_document(node): - node.ownerDocument._id_cache.clear() - node.ownerDocument._id_search_stack= None - -class Document(Node, DocumentLS): - _child_node_types = (Node.ELEMENT_NODE, Node.PROCESSING_INSTRUCTION_NODE, - Node.COMMENT_NODE, Node.DOCUMENT_TYPE_NODE) - - nodeType = Node.DOCUMENT_NODE - nodeName = "#document" - nodeValue = None - attributes = None - doctype = None - parentNode = None - previousSibling = nextSibling = None - - implementation = DOMImplementation() - - # Document attributes from Level 3 (WD 9 April 2002) - - actualEncoding = None - encoding = None - standalone = None - version = None - strictErrorChecking = False - errorHandler = None - documentURI = None - - _magic_id_count = 0 - - def __init__(self): - self.childNodes = NodeList() - # mapping of (namespaceURI, localName) -> ElementInfo - # and tagName -> ElementInfo - self._elem_info = {} - self._id_cache = {} - self._id_search_stack = None - - def _get_elem_info(self, element): - if element.namespaceURI: - key = element.namespaceURI, element.localName - else: - key = element.tagName - return self._elem_info.get(key) - - def _get_actualEncoding(self): - return self.actualEncoding - - def _get_doctype(self): - return self.doctype - - def _get_documentURI(self): - return self.documentURI - - def _get_encoding(self): - return self.encoding - - def _get_errorHandler(self): - return self.errorHandler - - def _get_standalone(self): - return self.standalone - - def _get_strictErrorChecking(self): - return self.strictErrorChecking - - def _get_version(self): - return self.version - - def appendChild(self, node): - if node.nodeType not in self._child_node_types: - raise xmlcore.dom.HierarchyRequestErr( - "%s cannot be child of %s" % (repr(node), repr(self))) - if node.parentNode is not None: - # This needs to be done before the next test since this - # may *be* the document element, in which case it should - # end up re-ordered to the end. - node.parentNode.removeChild(node) - - if node.nodeType == Node.ELEMENT_NODE \ - and self._get_documentElement(): - raise xmlcore.dom.HierarchyRequestErr( - "two document elements disallowed") - return Node.appendChild(self, node) - - def removeChild(self, oldChild): - try: - self.childNodes.remove(oldChild) - except ValueError: - raise xmlcore.dom.NotFoundErr() - oldChild.nextSibling = oldChild.previousSibling = None - oldChild.parentNode = None - if self.documentElement is oldChild: - self.documentElement = None - - return oldChild - - def _get_documentElement(self): - for node in self.childNodes: - if node.nodeType == Node.ELEMENT_NODE: - return node - - def unlink(self): - if self.doctype is not None: - self.doctype.unlink() - self.doctype = None - Node.unlink(self) - - def cloneNode(self, deep): - if not deep: - return None - clone = self.implementation.createDocument(None, None, None) - clone.encoding = self.encoding - clone.standalone = self.standalone - clone.version = self.version - for n in self.childNodes: - childclone = _clone_node(n, deep, clone) - assert childclone.ownerDocument.isSameNode(clone) - clone.childNodes.append(childclone) - if childclone.nodeType == Node.DOCUMENT_NODE: - assert clone.documentElement is None - elif childclone.nodeType == Node.DOCUMENT_TYPE_NODE: - assert clone.doctype is None - clone.doctype = childclone - childclone.parentNode = clone - self._call_user_data_handler(xmlcore.dom.UserDataHandler.NODE_CLONED, - self, clone) - return clone - - def createDocumentFragment(self): - d = DocumentFragment() - d.ownerDocument = self - return d - - def createElement(self, tagName): - e = Element(tagName) - e.ownerDocument = self - return e - - def createTextNode(self, data): - if not isinstance(data, StringTypes): - raise TypeError, "node contents must be a string" - t = Text() - t.data = data - t.ownerDocument = self - return t - - def createCDATASection(self, data): - if not isinstance(data, StringTypes): - raise TypeError, "node contents must be a string" - c = CDATASection() - c.data = data - c.ownerDocument = self - return c - - def createComment(self, data): - c = Comment(data) - c.ownerDocument = self - return c - - def createProcessingInstruction(self, target, data): - p = ProcessingInstruction(target, data) - p.ownerDocument = self - return p - - def createAttribute(self, qName): - a = Attr(qName) - a.ownerDocument = self - a.value = "" - return a - - def createElementNS(self, namespaceURI, qualifiedName): - prefix, localName = _nssplit(qualifiedName) - e = Element(qualifiedName, namespaceURI, prefix) - e.ownerDocument = self - return e - - def createAttributeNS(self, namespaceURI, qualifiedName): - prefix, localName = _nssplit(qualifiedName) - a = Attr(qualifiedName, namespaceURI, localName, prefix) - a.ownerDocument = self - a.value = "" - return a - - # A couple of implementation-specific helpers to create node types - # not supported by the W3C DOM specs: - - def _create_entity(self, name, publicId, systemId, notationName): - e = Entity(name, publicId, systemId, notationName) - e.ownerDocument = self - return e - - def _create_notation(self, name, publicId, systemId): - n = Notation(name, publicId, systemId) - n.ownerDocument = self - return n - - def getElementById(self, id): - if self._id_cache.has_key(id): - return self._id_cache[id] - if not (self._elem_info or self._magic_id_count): - return None - - stack = self._id_search_stack - if stack is None: - # we never searched before, or the cache has been cleared - stack = [self.documentElement] - self._id_search_stack = stack - elif not stack: - # Previous search was completed and cache is still valid; - # no matching node. - return None - - result = None - while stack: - node = stack.pop() - # add child elements to stack for continued searching - stack.extend([child for child in node.childNodes - if child.nodeType in _nodeTypes_with_children]) - # check this node - info = self._get_elem_info(node) - if info: - # We have to process all ID attributes before - # returning in order to get all the attributes set to - # be IDs using Element.setIdAttribute*(). - for attr in node.attributes.values(): - if attr.namespaceURI: - if info.isIdNS(attr.namespaceURI, attr.localName): - self._id_cache[attr.value] = node - if attr.value == id: - result = node - elif not node._magic_id_nodes: - break - elif info.isId(attr.name): - self._id_cache[attr.value] = node - if attr.value == id: - result = node - elif not node._magic_id_nodes: - break - elif attr._is_id: - self._id_cache[attr.value] = node - if attr.value == id: - result = node - elif node._magic_id_nodes == 1: - break - elif node._magic_id_nodes: - for attr in node.attributes.values(): - if attr._is_id: - self._id_cache[attr.value] = node - if attr.value == id: - result = node - if result is not None: - break - return result - - def getElementsByTagName(self, name): - return _get_elements_by_tagName_helper(self, name, NodeList()) - - def getElementsByTagNameNS(self, namespaceURI, localName): - return _get_elements_by_tagName_ns_helper( - self, namespaceURI, localName, NodeList()) - - def isSupported(self, feature, version): - return self.implementation.hasFeature(feature, version) - - def importNode(self, node, deep): - if node.nodeType == Node.DOCUMENT_NODE: - raise xmlcore.dom.NotSupportedErr("cannot import document nodes") - elif node.nodeType == Node.DOCUMENT_TYPE_NODE: - raise xmlcore.dom.NotSupportedErr("cannot import document type nodes") - return _clone_node(node, deep, self) - - def writexml(self, writer, indent="", addindent="", newl="", - encoding = None): - if encoding is None: - writer.write(''+newl) - else: - writer.write('%s' % (encoding, newl)) - for node in self.childNodes: - node.writexml(writer, indent, addindent, newl) - - # DOM Level 3 (WD 9 April 2002) - - def renameNode(self, n, namespaceURI, name): - if n.ownerDocument is not self: - raise xmlcore.dom.WrongDocumentErr( - "cannot rename nodes from other documents;\n" - "expected %s,\nfound %s" % (self, n.ownerDocument)) - if n.nodeType not in (Node.ELEMENT_NODE, Node.ATTRIBUTE_NODE): - raise xmlcore.dom.NotSupportedErr( - "renameNode() only applies to element and attribute nodes") - if namespaceURI != EMPTY_NAMESPACE: - if ':' in name: - prefix, localName = name.split(':', 1) - if ( prefix == "xmlns" - and namespaceURI != xmlcore.dom.XMLNS_NAMESPACE): - raise xmlcore.dom.NamespaceErr( - "illegal use of 'xmlns' prefix") - else: - if ( name == "xmlns" - and namespaceURI != xmlcore.dom.XMLNS_NAMESPACE - and n.nodeType == Node.ATTRIBUTE_NODE): - raise xmlcore.dom.NamespaceErr( - "illegal use of the 'xmlns' attribute") - prefix = None - localName = name - else: - prefix = None - localName = None - if n.nodeType == Node.ATTRIBUTE_NODE: - element = n.ownerElement - if element is not None: - is_id = n._is_id - element.removeAttributeNode(n) - else: - element = None - # avoid __setattr__ - d = n.__dict__ - d['prefix'] = prefix - d['localName'] = localName - d['namespaceURI'] = namespaceURI - d['nodeName'] = name - if n.nodeType == Node.ELEMENT_NODE: - d['tagName'] = name - else: - # attribute node - d['name'] = name - if element is not None: - element.setAttributeNode(n) - if is_id: - element.setIdAttributeNode(n) - # It's not clear from a semantic perspective whether we should - # call the user data handlers for the NODE_RENAMED event since - # we're re-using the existing node. The draft spec has been - # interpreted as meaning "no, don't call the handler unless a - # new node is created." - return n - -defproperty(Document, "documentElement", - doc="Top-level element of this document.") - - -def _clone_node(node, deep, newOwnerDocument): - """ - Clone a node and give it the new owner document. - Called by Node.cloneNode and Document.importNode - """ - if node.ownerDocument.isSameNode(newOwnerDocument): - operation = xmlcore.dom.UserDataHandler.NODE_CLONED - else: - operation = xmlcore.dom.UserDataHandler.NODE_IMPORTED - if node.nodeType == Node.ELEMENT_NODE: - clone = newOwnerDocument.createElementNS(node.namespaceURI, - node.nodeName) - for attr in node.attributes.values(): - clone.setAttributeNS(attr.namespaceURI, attr.nodeName, attr.value) - a = clone.getAttributeNodeNS(attr.namespaceURI, attr.localName) - a.specified = attr.specified - - if deep: - for child in node.childNodes: - c = _clone_node(child, deep, newOwnerDocument) - clone.appendChild(c) - - elif node.nodeType == Node.DOCUMENT_FRAGMENT_NODE: - clone = newOwnerDocument.createDocumentFragment() - if deep: - for child in node.childNodes: - c = _clone_node(child, deep, newOwnerDocument) - clone.appendChild(c) - - elif node.nodeType == Node.TEXT_NODE: - clone = newOwnerDocument.createTextNode(node.data) - elif node.nodeType == Node.CDATA_SECTION_NODE: - clone = newOwnerDocument.createCDATASection(node.data) - elif node.nodeType == Node.PROCESSING_INSTRUCTION_NODE: - clone = newOwnerDocument.createProcessingInstruction(node.target, - node.data) - elif node.nodeType == Node.COMMENT_NODE: - clone = newOwnerDocument.createComment(node.data) - elif node.nodeType == Node.ATTRIBUTE_NODE: - clone = newOwnerDocument.createAttributeNS(node.namespaceURI, - node.nodeName) - clone.specified = True - clone.value = node.value - elif node.nodeType == Node.DOCUMENT_TYPE_NODE: - assert node.ownerDocument is not newOwnerDocument - operation = xmlcore.dom.UserDataHandler.NODE_IMPORTED - clone = newOwnerDocument.implementation.createDocumentType( - node.name, node.publicId, node.systemId) - clone.ownerDocument = newOwnerDocument - if deep: - clone.entities._seq = [] - clone.notations._seq = [] - for n in node.notations._seq: - notation = Notation(n.nodeName, n.publicId, n.systemId) - notation.ownerDocument = newOwnerDocument - clone.notations._seq.append(notation) - if hasattr(n, '_call_user_data_handler'): - n._call_user_data_handler(operation, n, notation) - for e in node.entities._seq: - entity = Entity(e.nodeName, e.publicId, e.systemId, - e.notationName) - entity.actualEncoding = e.actualEncoding - entity.encoding = e.encoding - entity.version = e.version - entity.ownerDocument = newOwnerDocument - clone.entities._seq.append(entity) - if hasattr(e, '_call_user_data_handler'): - e._call_user_data_handler(operation, n, entity) - else: - # Note the cloning of Document and DocumentType nodes is - # implemenetation specific. minidom handles those cases - # directly in the cloneNode() methods. - raise xmlcore.dom.NotSupportedErr("Cannot clone node %s" % repr(node)) - - # Check for _call_user_data_handler() since this could conceivably - # used with other DOM implementations (one of the FourThought - # DOMs, perhaps?). - if hasattr(node, '_call_user_data_handler'): - node._call_user_data_handler(operation, node, clone) - return clone - - -def _nssplit(qualifiedName): - fields = qualifiedName.split(':', 1) - if len(fields) == 2: - return fields - else: - return (None, fields[0]) - - -def _get_StringIO(): - # we can't use cStringIO since it doesn't support Unicode strings - from StringIO import StringIO - return StringIO() - -def _do_pulldom_parse(func, args, kwargs): - events = func(*args, **kwargs) - toktype, rootNode = events.getEvent() - events.expandNode(rootNode) - events.clear() - return rootNode - -def parse(file, parser=None, bufsize=None): - """Parse a file into a DOM by filename or file object.""" - if parser is None and not bufsize: - from xmlcore.dom import expatbuilder - return expatbuilder.parse(file) - else: - from xmlcore.dom import pulldom - return _do_pulldom_parse(pulldom.parse, (file,), - {'parser': parser, 'bufsize': bufsize}) - -def parseString(string, parser=None): - """Parse a file into a DOM from a string.""" - if parser is None: - from xmlcore.dom import expatbuilder - return expatbuilder.parseString(string) - else: - from xmlcore.dom import pulldom - return _do_pulldom_parse(pulldom.parseString, (string,), - {'parser': parser}) - -def getDOMImplementation(features=None): - if features: - if isinstance(features, StringTypes): - features = domreg._parse_feature_string(features) - for f, v in features: - if not Document.implementation.hasFeature(f, v): - return None - return Document.implementation diff --git a/Lib/xmlcore/dom/pulldom.py b/Lib/xmlcore/dom/pulldom.py deleted file mode 100644 index dad3718..0000000 --- a/Lib/xmlcore/dom/pulldom.py +++ /dev/null @@ -1,351 +0,0 @@ -import xmlcore.sax -import xmlcore.sax.handler -import types - -try: - _StringTypes = [types.StringType, types.UnicodeType] -except AttributeError: - _StringTypes = [types.StringType] - -START_ELEMENT = "START_ELEMENT" -END_ELEMENT = "END_ELEMENT" -COMMENT = "COMMENT" -START_DOCUMENT = "START_DOCUMENT" -END_DOCUMENT = "END_DOCUMENT" -PROCESSING_INSTRUCTION = "PROCESSING_INSTRUCTION" -IGNORABLE_WHITESPACE = "IGNORABLE_WHITESPACE" -CHARACTERS = "CHARACTERS" - -class PullDOM(xmlcore.sax.ContentHandler): - _locator = None - document = None - - def __init__(self, documentFactory=None): - from xmlcore.dom import XML_NAMESPACE - self.documentFactory = documentFactory - self.firstEvent = [None, None] - self.lastEvent = self.firstEvent - self.elementStack = [] - self.push = self.elementStack.append - try: - self.pop = self.elementStack.pop - except AttributeError: - # use class' pop instead - pass - self._ns_contexts = [{XML_NAMESPACE:'xml'}] # contains uri -> prefix dicts - self._current_context = self._ns_contexts[-1] - self.pending_events = [] - - def pop(self): - result = self.elementStack[-1] - del self.elementStack[-1] - return result - - def setDocumentLocator(self, locator): - self._locator = locator - - def startPrefixMapping(self, prefix, uri): - if not hasattr(self, '_xmlns_attrs'): - self._xmlns_attrs = [] - self._xmlns_attrs.append((prefix or 'xmlns', uri)) - self._ns_contexts.append(self._current_context.copy()) - self._current_context[uri] = prefix or None - - def endPrefixMapping(self, prefix): - self._current_context = self._ns_contexts.pop() - - def startElementNS(self, name, tagName , attrs): - # Retrieve xml namespace declaration attributes. - xmlns_uri = 'http://www.w3.org/2000/xmlns/' - xmlns_attrs = getattr(self, '_xmlns_attrs', None) - if xmlns_attrs is not None: - for aname, value in xmlns_attrs: - attrs._attrs[(xmlns_uri, aname)] = value - self._xmlns_attrs = [] - uri, localname = name - if uri: - # When using namespaces, the reader may or may not - # provide us with the original name. If not, create - # *a* valid tagName from the current context. - if tagName is None: - prefix = self._current_context[uri] - if prefix: - tagName = prefix + ":" + localname - else: - tagName = localname - if self.document: - node = self.document.createElementNS(uri, tagName) - else: - node = self.buildDocument(uri, tagName) - else: - # When the tagname is not prefixed, it just appears as - # localname - if self.document: - node = self.document.createElement(localname) - else: - node = self.buildDocument(None, localname) - - for aname,value in attrs.items(): - a_uri, a_localname = aname - if a_uri == xmlns_uri: - if a_localname == 'xmlns': - qname = a_localname - else: - qname = 'xmlns:' + a_localname - attr = self.document.createAttributeNS(a_uri, qname) - node.setAttributeNodeNS(attr) - elif a_uri: - prefix = self._current_context[a_uri] - if prefix: - qname = prefix + ":" + a_localname - else: - qname = a_localname - attr = self.document.createAttributeNS(a_uri, qname) - node.setAttributeNodeNS(attr) - else: - attr = self.document.createAttribute(a_localname) - node.setAttributeNode(attr) - attr.value = value - - self.lastEvent[1] = [(START_ELEMENT, node), None] - self.lastEvent = self.lastEvent[1] - self.push(node) - - def endElementNS(self, name, tagName): - self.lastEvent[1] = [(END_ELEMENT, self.pop()), None] - self.lastEvent = self.lastEvent[1] - - def startElement(self, name, attrs): - if self.document: - node = self.document.createElement(name) - else: - node = self.buildDocument(None, name) - - for aname,value in attrs.items(): - attr = self.document.createAttribute(aname) - attr.value = value - node.setAttributeNode(attr) - - self.lastEvent[1] = [(START_ELEMENT, node), None] - self.lastEvent = self.lastEvent[1] - self.push(node) - - def endElement(self, name): - self.lastEvent[1] = [(END_ELEMENT, self.pop()), None] - self.lastEvent = self.lastEvent[1] - - def comment(self, s): - if self.document: - node = self.document.createComment(s) - self.lastEvent[1] = [(COMMENT, node), None] - self.lastEvent = self.lastEvent[1] - else: - event = [(COMMENT, s), None] - self.pending_events.append(event) - - def processingInstruction(self, target, data): - if self.document: - node = self.document.createProcessingInstruction(target, data) - self.lastEvent[1] = [(PROCESSING_INSTRUCTION, node), None] - self.lastEvent = self.lastEvent[1] - else: - event = [(PROCESSING_INSTRUCTION, target, data), None] - self.pending_events.append(event) - - def ignorableWhitespace(self, chars): - node = self.document.createTextNode(chars) - self.lastEvent[1] = [(IGNORABLE_WHITESPACE, node), None] - self.lastEvent = self.lastEvent[1] - - def characters(self, chars): - node = self.document.createTextNode(chars) - self.lastEvent[1] = [(CHARACTERS, node), None] - self.lastEvent = self.lastEvent[1] - - def startDocument(self): - if self.documentFactory is None: - import xmlcore.dom.minidom - self.documentFactory = xmlcore.dom.minidom.Document.implementation - - def buildDocument(self, uri, tagname): - # Can't do that in startDocument, since we need the tagname - # XXX: obtain DocumentType - node = self.documentFactory.createDocument(uri, tagname, None) - self.document = node - self.lastEvent[1] = [(START_DOCUMENT, node), None] - self.lastEvent = self.lastEvent[1] - self.push(node) - # Put everything we have seen so far into the document - for e in self.pending_events: - if e[0][0] == PROCESSING_INSTRUCTION: - _,target,data = e[0] - n = self.document.createProcessingInstruction(target, data) - e[0] = (PROCESSING_INSTRUCTION, n) - elif e[0][0] == COMMENT: - n = self.document.createComment(e[0][1]) - e[0] = (COMMENT, n) - else: - raise AssertionError("Unknown pending event ",e[0][0]) - self.lastEvent[1] = e - self.lastEvent = e - self.pending_events = None - return node.firstChild - - def endDocument(self): - self.lastEvent[1] = [(END_DOCUMENT, self.document), None] - self.pop() - - def clear(self): - "clear(): Explicitly release parsing structures" - self.document = None - -class ErrorHandler: - def warning(self, exception): - print exception - def error(self, exception): - raise exception - def fatalError(self, exception): - raise exception - -class DOMEventStream: - def __init__(self, stream, parser, bufsize): - self.stream = stream - self.parser = parser - self.bufsize = bufsize - if not hasattr(self.parser, 'feed'): - self.getEvent = self._slurp - self.reset() - - def reset(self): - self.pulldom = PullDOM() - # This content handler relies on namespace support - self.parser.setFeature(xmlcore.sax.handler.feature_namespaces, 1) - self.parser.setContentHandler(self.pulldom) - - def __getitem__(self, pos): - rc = self.getEvent() - if rc: - return rc - raise IndexError - - def next(self): - rc = self.getEvent() - if rc: - return rc - raise StopIteration - - def __iter__(self): - return self - - def expandNode(self, node): - event = self.getEvent() - parents = [node] - while event: - token, cur_node = event - if cur_node is node: - return - if token != END_ELEMENT: - parents[-1].appendChild(cur_node) - if token == START_ELEMENT: - parents.append(cur_node) - elif token == END_ELEMENT: - del parents[-1] - event = self.getEvent() - - def getEvent(self): - # use IncrementalParser interface, so we get the desired - # pull effect - if not self.pulldom.firstEvent[1]: - self.pulldom.lastEvent = self.pulldom.firstEvent - while not self.pulldom.firstEvent[1]: - buf = self.stream.read(self.bufsize) - if not buf: - self.parser.close() - return None - self.parser.feed(buf) - rc = self.pulldom.firstEvent[1][0] - self.pulldom.firstEvent[1] = self.pulldom.firstEvent[1][1] - return rc - - def _slurp(self): - """ Fallback replacement for getEvent() using the - standard SAX2 interface, which means we slurp the - SAX events into memory (no performance gain, but - we are compatible to all SAX parsers). - """ - self.parser.parse(self.stream) - self.getEvent = self._emit - return self._emit() - - def _emit(self): - """ Fallback replacement for getEvent() that emits - the events that _slurp() read previously. - """ - rc = self.pulldom.firstEvent[1][0] - self.pulldom.firstEvent[1] = self.pulldom.firstEvent[1][1] - return rc - - def clear(self): - """clear(): Explicitly release parsing objects""" - self.pulldom.clear() - del self.pulldom - self.parser = None - self.stream = None - -class SAX2DOM(PullDOM): - - def startElementNS(self, name, tagName , attrs): - PullDOM.startElementNS(self, name, tagName, attrs) - curNode = self.elementStack[-1] - parentNode = self.elementStack[-2] - parentNode.appendChild(curNode) - - def startElement(self, name, attrs): - PullDOM.startElement(self, name, attrs) - curNode = self.elementStack[-1] - parentNode = self.elementStack[-2] - parentNode.appendChild(curNode) - - def processingInstruction(self, target, data): - PullDOM.processingInstruction(self, target, data) - node = self.lastEvent[0][1] - parentNode = self.elementStack[-1] - parentNode.appendChild(node) - - def ignorableWhitespace(self, chars): - PullDOM.ignorableWhitespace(self, chars) - node = self.lastEvent[0][1] - parentNode = self.elementStack[-1] - parentNode.appendChild(node) - - def characters(self, chars): - PullDOM.characters(self, chars) - node = self.lastEvent[0][1] - parentNode = self.elementStack[-1] - parentNode.appendChild(node) - - -default_bufsize = (2 ** 14) - 20 - -def parse(stream_or_string, parser=None, bufsize=None): - if bufsize is None: - bufsize = default_bufsize - if type(stream_or_string) in _StringTypes: - stream = open(stream_or_string) - else: - stream = stream_or_string - if not parser: - parser = xmlcore.sax.make_parser() - return DOMEventStream(stream, parser, bufsize) - -def parseString(string, parser=None): - try: - from cStringIO import StringIO - except ImportError: - from StringIO import StringIO - - bufsize = len(string) - buf = StringIO(string) - if not parser: - parser = xmlcore.sax.make_parser() - return DOMEventStream(buf, parser, bufsize) diff --git a/Lib/xmlcore/dom/xmlbuilder.py b/Lib/xmlcore/dom/xmlbuilder.py deleted file mode 100644 index 6566d3c..0000000 --- a/Lib/xmlcore/dom/xmlbuilder.py +++ /dev/null @@ -1,386 +0,0 @@ -"""Implementation of the DOM Level 3 'LS-Load' feature.""" - -import copy -import xmlcore.dom - -from xmlcore.dom.NodeFilter import NodeFilter - - -__all__ = ["DOMBuilder", "DOMEntityResolver", "DOMInputSource"] - - -class Options: - """Features object that has variables set for each DOMBuilder feature. - - The DOMBuilder class uses an instance of this class to pass settings to - the ExpatBuilder class. - """ - - # Note that the DOMBuilder class in LoadSave constrains which of these - # values can be set using the DOM Level 3 LoadSave feature. - - namespaces = 1 - namespace_declarations = True - validation = False - external_parameter_entities = True - external_general_entities = True - external_dtd_subset = True - validate_if_schema = False - validate = False - datatype_normalization = False - create_entity_ref_nodes = True - entities = True - whitespace_in_element_content = True - cdata_sections = True - comments = True - charset_overrides_xml_encoding = True - infoset = False - supported_mediatypes_only = False - - errorHandler = None - filter = None - - -class DOMBuilder: - entityResolver = None - errorHandler = None - filter = None - - ACTION_REPLACE = 1 - ACTION_APPEND_AS_CHILDREN = 2 - ACTION_INSERT_AFTER = 3 - ACTION_INSERT_BEFORE = 4 - - _legal_actions = (ACTION_REPLACE, ACTION_APPEND_AS_CHILDREN, - ACTION_INSERT_AFTER, ACTION_INSERT_BEFORE) - - def __init__(self): - self._options = Options() - - def _get_entityResolver(self): - return self.entityResolver - def _set_entityResolver(self, entityResolver): - self.entityResolver = entityResolver - - def _get_errorHandler(self): - return self.errorHandler - def _set_errorHandler(self, errorHandler): - self.errorHandler = errorHandler - - def _get_filter(self): - return self.filter - def _set_filter(self, filter): - self.filter = filter - - def setFeature(self, name, state): - if self.supportsFeature(name): - state = state and 1 or 0 - try: - settings = self._settings[(_name_xform(name), state)] - except KeyError: - raise xmlcore.dom.NotSupportedErr( - "unsupported feature: %r" % (name,)) - else: - for name, value in settings: - setattr(self._options, name, value) - else: - raise xmlcore.dom.NotFoundErr("unknown feature: " + repr(name)) - - def supportsFeature(self, name): - return hasattr(self._options, _name_xform(name)) - - def canSetFeature(self, name, state): - key = (_name_xform(name), state and 1 or 0) - return self._settings.has_key(key) - - # This dictionary maps from (feature,value) to a list of - # (option,value) pairs that should be set on the Options object. - # If a (feature,value) setting is not in this dictionary, it is - # not supported by the DOMBuilder. - # - _settings = { - ("namespace_declarations", 0): [ - ("namespace_declarations", 0)], - ("namespace_declarations", 1): [ - ("namespace_declarations", 1)], - ("validation", 0): [ - ("validation", 0)], - ("external_general_entities", 0): [ - ("external_general_entities", 0)], - ("external_general_entities", 1): [ - ("external_general_entities", 1)], - ("external_parameter_entities", 0): [ - ("external_parameter_entities", 0)], - ("external_parameter_entities", 1): [ - ("external_parameter_entities", 1)], - ("validate_if_schema", 0): [ - ("validate_if_schema", 0)], - ("create_entity_ref_nodes", 0): [ - ("create_entity_ref_nodes", 0)], - ("create_entity_ref_nodes", 1): [ - ("create_entity_ref_nodes", 1)], - ("entities", 0): [ - ("create_entity_ref_nodes", 0), - ("entities", 0)], - ("entities", 1): [ - ("entities", 1)], - ("whitespace_in_element_content", 0): [ - ("whitespace_in_element_content", 0)], - ("whitespace_in_element_content", 1): [ - ("whitespace_in_element_content", 1)], - ("cdata_sections", 0): [ - ("cdata_sections", 0)], - ("cdata_sections", 1): [ - ("cdata_sections", 1)], - ("comments", 0): [ - ("comments", 0)], - ("comments", 1): [ - ("comments", 1)], - ("charset_overrides_xml_encoding", 0): [ - ("charset_overrides_xml_encoding", 0)], - ("charset_overrides_xml_encoding", 1): [ - ("charset_overrides_xml_encoding", 1)], - ("infoset", 0): [], - ("infoset", 1): [ - ("namespace_declarations", 0), - ("validate_if_schema", 0), - ("create_entity_ref_nodes", 0), - ("entities", 0), - ("cdata_sections", 0), - ("datatype_normalization", 1), - ("whitespace_in_element_content", 1), - ("comments", 1), - ("charset_overrides_xml_encoding", 1)], - ("supported_mediatypes_only", 0): [ - ("supported_mediatypes_only", 0)], - ("namespaces", 0): [ - ("namespaces", 0)], - ("namespaces", 1): [ - ("namespaces", 1)], - } - - def getFeature(self, name): - xname = _name_xform(name) - try: - return getattr(self._options, xname) - except AttributeError: - if name == "infoset": - options = self._options - return (options.datatype_normalization - and options.whitespace_in_element_content - and options.comments - and options.charset_overrides_xml_encoding - and not (options.namespace_declarations - or options.validate_if_schema - or options.create_entity_ref_nodes - or options.entities - or options.cdata_sections)) - raise xmlcore.dom.NotFoundErr("feature %s not known" % repr(name)) - - def parseURI(self, uri): - if self.entityResolver: - input = self.entityResolver.resolveEntity(None, uri) - else: - input = DOMEntityResolver().resolveEntity(None, uri) - return self.parse(input) - - def parse(self, input): - options = copy.copy(self._options) - options.filter = self.filter - options.errorHandler = self.errorHandler - fp = input.byteStream - if fp is None and options.systemId: - import urllib2 - fp = urllib2.urlopen(input.systemId) - return self._parse_bytestream(fp, options) - - def parseWithContext(self, input, cnode, action): - if action not in self._legal_actions: - raise ValueError("not a legal action") - raise NotImplementedError("Haven't written this yet...") - - def _parse_bytestream(self, stream, options): - import xmlcore.dom.expatbuilder - builder = xmlcore.dom.expatbuilder.makeBuilder(options) - return builder.parseFile(stream) - - -def _name_xform(name): - return name.lower().replace('-', '_') - - -class DOMEntityResolver(object): - __slots__ = '_opener', - - def resolveEntity(self, publicId, systemId): - assert systemId is not None - source = DOMInputSource() - source.publicId = publicId - source.systemId = systemId - source.byteStream = self._get_opener().open(systemId) - - # determine the encoding if the transport provided it - source.encoding = self._guess_media_encoding(source) - - # determine the base URI is we can - import posixpath, urlparse - parts = urlparse.urlparse(systemId) - scheme, netloc, path, params, query, fragment = parts - # XXX should we check the scheme here as well? - if path and not path.endswith("/"): - path = posixpath.dirname(path) + "/" - parts = scheme, netloc, path, params, query, fragment - source.baseURI = urlparse.urlunparse(parts) - - return source - - def _get_opener(self): - try: - return self._opener - except AttributeError: - self._opener = self._create_opener() - return self._opener - - def _create_opener(self): - import urllib2 - return urllib2.build_opener() - - def _guess_media_encoding(self, source): - info = source.byteStream.info() - if info.has_key("Content-Type"): - for param in info.getplist(): - if param.startswith("charset="): - return param.split("=", 1)[1].lower() - - -class DOMInputSource(object): - __slots__ = ('byteStream', 'characterStream', 'stringData', - 'encoding', 'publicId', 'systemId', 'baseURI') - - def __init__(self): - self.byteStream = None - self.characterStream = None - self.stringData = None - self.encoding = None - self.publicId = None - self.systemId = None - self.baseURI = None - - def _get_byteStream(self): - return self.byteStream - def _set_byteStream(self, byteStream): - self.byteStream = byteStream - - def _get_characterStream(self): - return self.characterStream - def _set_characterStream(self, characterStream): - self.characterStream = characterStream - - def _get_stringData(self): - return self.stringData - def _set_stringData(self, data): - self.stringData = data - - def _get_encoding(self): - return self.encoding - def _set_encoding(self, encoding): - self.encoding = encoding - - def _get_publicId(self): - return self.publicId - def _set_publicId(self, publicId): - self.publicId = publicId - - def _get_systemId(self): - return self.systemId - def _set_systemId(self, systemId): - self.systemId = systemId - - def _get_baseURI(self): - return self.baseURI - def _set_baseURI(self, uri): - self.baseURI = uri - - -class DOMBuilderFilter: - """Element filter which can be used to tailor construction of - a DOM instance. - """ - - # There's really no need for this class; concrete implementations - # should just implement the endElement() and startElement() - # methods as appropriate. Using this makes it easy to only - # implement one of them. - - FILTER_ACCEPT = 1 - FILTER_REJECT = 2 - FILTER_SKIP = 3 - FILTER_INTERRUPT = 4 - - whatToShow = NodeFilter.SHOW_ALL - - def _get_whatToShow(self): - return self.whatToShow - - def acceptNode(self, element): - return self.FILTER_ACCEPT - - def startContainer(self, element): - return self.FILTER_ACCEPT - -del NodeFilter - - -class DocumentLS: - """Mixin to create documents that conform to the load/save spec.""" - - async = False - - def _get_async(self): - return False - def _set_async(self, async): - if async: - raise xmlcore.dom.NotSupportedErr( - "asynchronous document loading is not supported") - - def abort(self): - # What does it mean to "clear" a document? Does the - # documentElement disappear? - raise NotImplementedError( - "haven't figured out what this means yet") - - def load(self, uri): - raise NotImplementedError("haven't written this yet") - - def loadXML(self, source): - raise NotImplementedError("haven't written this yet") - - def saveXML(self, snode): - if snode is None: - snode = self - elif snode.ownerDocument is not self: - raise xmlcore.dom.WrongDocumentErr() - return snode.toxml() - - -class DOMImplementationLS: - MODE_SYNCHRONOUS = 1 - MODE_ASYNCHRONOUS = 2 - - def createDOMBuilder(self, mode, schemaType): - if schemaType is not None: - raise xmlcore.dom.NotSupportedErr( - "schemaType not yet supported") - if mode == self.MODE_SYNCHRONOUS: - return DOMBuilder() - if mode == self.MODE_ASYNCHRONOUS: - raise xmlcore.dom.NotSupportedErr( - "asynchronous builders are not supported") - raise ValueError("unknown value for mode") - - def createDOMWriter(self): - raise NotImplementedError( - "the writer interface hasn't been written yet!") - - def createDOMInputSource(self): - return DOMInputSource() diff --git a/Lib/xmlcore/etree/ElementInclude.py b/Lib/xmlcore/etree/ElementInclude.py deleted file mode 100644 index d7f85b3..0000000 --- a/Lib/xmlcore/etree/ElementInclude.py +++ /dev/null @@ -1,143 +0,0 @@ -# -# ElementTree -# $Id: ElementInclude.py 1862 2004-06-18 07:31:02Z Fredrik $ -# -# limited xinclude support for element trees -# -# history: -# 2003-08-15 fl created -# 2003-11-14 fl fixed default loader -# -# Copyright (c) 2003-2004 by Fredrik Lundh. All rights reserved. -# -# fredrik@pythonware.com -# http://www.pythonware.com -# -# -------------------------------------------------------------------- -# The ElementTree toolkit is -# -# Copyright (c) 1999-2004 by Fredrik Lundh -# -# By obtaining, using, and/or copying this software and/or its -# associated documentation, you agree that you have read, understood, -# and will comply with the following terms and conditions: -# -# Permission to use, copy, modify, and distribute this software and -# its associated documentation for any purpose and without fee is -# hereby granted, provided that the above copyright notice appears in -# all copies, and that both that copyright notice and this permission -# notice appear in supporting documentation, and that the name of -# Secret Labs AB or the author not be used in advertising or publicity -# pertaining to distribution of the software without specific, written -# prior permission. -# -# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD -# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- -# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR -# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY -# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, -# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS -# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE -# OF THIS SOFTWARE. -# -------------------------------------------------------------------- - -# Licensed to PSF under a Contributor Agreement. -# See http://www.python.org/2.4/license for licensing details. - -## -# Limited XInclude support for the ElementTree package. -## - -import copy -from . import ElementTree - -XINCLUDE = "{http://www.w3.org/2001/XInclude}" - -XINCLUDE_INCLUDE = XINCLUDE + "include" -XINCLUDE_FALLBACK = XINCLUDE + "fallback" - -## -# Fatal include error. - -class FatalIncludeError(SyntaxError): - pass - -## -# Default loader. This loader reads an included resource from disk. -# -# @param href Resource reference. -# @param parse Parse mode. Either "xml" or "text". -# @param encoding Optional text encoding. -# @return The expanded resource. If the parse mode is "xml", this -# is an ElementTree instance. If the parse mode is "text", this -# is a Unicode string. If the loader fails, it can return None -# or raise an IOError exception. -# @throws IOError If the loader fails to load the resource. - -def default_loader(href, parse, encoding=None): - file = open(href) - if parse == "xml": - data = ElementTree.parse(file).getroot() - else: - data = file.read() - if encoding: - data = data.decode(encoding) - file.close() - return data - -## -# Expand XInclude directives. -# -# @param elem Root element. -# @param loader Optional resource loader. If omitted, it defaults -# to {@link default_loader}. If given, it should be a callable -# that implements the same interface as default_loader. -# @throws FatalIncludeError If the function fails to include a given -# resource, or if the tree contains malformed XInclude elements. -# @throws IOError If the function fails to load a given resource. - -def include(elem, loader=None): - if loader is None: - loader = default_loader - # look for xinclude elements - i = 0 - while i < len(elem): - e = elem[i] - if e.tag == XINCLUDE_INCLUDE: - # process xinclude directive - href = e.get("href") - parse = e.get("parse", "xml") - if parse == "xml": - node = loader(href, parse) - if node is None: - raise FatalIncludeError( - "cannot load %r as %r" % (href, parse) - ) - node = copy.copy(node) - if e.tail: - node.tail = (node.tail or "") + e.tail - elem[i] = node - elif parse == "text": - text = loader(href, parse, e.get("encoding")) - if text is None: - raise FatalIncludeError( - "cannot load %r as %r" % (href, parse) - ) - if i: - node = elem[i-1] - node.tail = (node.tail or "") + text - else: - elem.text = (elem.text or "") + text + (e.tail or "") - del elem[i] - continue - else: - raise FatalIncludeError( - "unknown parse type in xi:include tag (%r)" % parse - ) - elif e.tag == XINCLUDE_FALLBACK: - raise FatalIncludeError( - "xi:fallback tag must be child of xi:include (%r)" % e.tag - ) - else: - include(e, loader) - i = i + 1 diff --git a/Lib/xmlcore/etree/ElementPath.py b/Lib/xmlcore/etree/ElementPath.py deleted file mode 100644 index 00dbe9d..0000000 --- a/Lib/xmlcore/etree/ElementPath.py +++ /dev/null @@ -1,198 +0,0 @@ -# -# ElementTree -# $Id: ElementPath.py 1858 2004-06-17 21:31:41Z Fredrik $ -# -# limited xpath support for element trees -# -# history: -# 2003-05-23 fl created -# 2003-05-28 fl added support for // etc -# 2003-08-27 fl fixed parsing of periods in element names -# -# Copyright (c) 2003-2004 by Fredrik Lundh. All rights reserved. -# -# fredrik@pythonware.com -# http://www.pythonware.com -# -# -------------------------------------------------------------------- -# The ElementTree toolkit is -# -# Copyright (c) 1999-2004 by Fredrik Lundh -# -# By obtaining, using, and/or copying this software and/or its -# associated documentation, you agree that you have read, understood, -# and will comply with the following terms and conditions: -# -# Permission to use, copy, modify, and distribute this software and -# its associated documentation for any purpose and without fee is -# hereby granted, provided that the above copyright notice appears in -# all copies, and that both that copyright notice and this permission -# notice appear in supporting documentation, and that the name of -# Secret Labs AB or the author not be used in advertising or publicity -# pertaining to distribution of the software without specific, written -# prior permission. -# -# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD -# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- -# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR -# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY -# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, -# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS -# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE -# OF THIS SOFTWARE. -# -------------------------------------------------------------------- - -# Licensed to PSF under a Contributor Agreement. -# See http://www.python.org/2.4/license for licensing details. - -## -# Implementation module for XPath support. There's usually no reason -# to import this module directly; the ElementTree does this for -# you, if needed. -## - -import re - -xpath_tokenizer = re.compile( - "(::|\.\.|\(\)|[/.*:\[\]\(\)@=])|((?:\{[^}]+\})?[^/:\[\]\(\)@=\s]+)|\s+" - ).findall - -class xpath_descendant_or_self: - pass - -## -# Wrapper for a compiled XPath. - -class Path: - - ## - # Create an Path instance from an XPath expression. - - def __init__(self, path): - tokens = xpath_tokenizer(path) - # the current version supports 'path/path'-style expressions only - self.path = [] - self.tag = None - if tokens and tokens[0][0] == "/": - raise SyntaxError("cannot use absolute path on element") - while tokens: - op, tag = tokens.pop(0) - if tag or op == "*": - self.path.append(tag or op) - elif op == ".": - pass - elif op == "/": - self.path.append(xpath_descendant_or_self()) - continue - else: - raise SyntaxError("unsupported path syntax (%s)" % op) - if tokens: - op, tag = tokens.pop(0) - if op != "/": - raise SyntaxError( - "expected path separator (%s)" % (op or tag) - ) - if self.path and isinstance(self.path[-1], xpath_descendant_or_self): - raise SyntaxError("path cannot end with //") - if len(self.path) == 1 and isinstance(self.path[0], type("")): - self.tag = self.path[0] - - ## - # Find first matching object. - - def find(self, element): - tag = self.tag - if tag is None: - nodeset = self.findall(element) - if not nodeset: - return None - return nodeset[0] - for elem in element: - if elem.tag == tag: - return elem - return None - - ## - # Find text for first matching object. - - def findtext(self, element, default=None): - tag = self.tag - if tag is None: - nodeset = self.findall(element) - if not nodeset: - return default - return nodeset[0].text or "" - for elem in element: - if elem.tag == tag: - return elem.text or "" - return default - - ## - # Find all matching objects. - - def findall(self, element): - nodeset = [element] - index = 0 - while 1: - try: - path = self.path[index] - index = index + 1 - except IndexError: - return nodeset - set = [] - if isinstance(path, xpath_descendant_or_self): - try: - tag = self.path[index] - if not isinstance(tag, type("")): - tag = None - else: - index = index + 1 - except IndexError: - tag = None # invalid path - for node in nodeset: - new = list(node.getiterator(tag)) - if new and new[0] is node: - set.extend(new[1:]) - else: - set.extend(new) - else: - for node in nodeset: - for node in node: - if path == "*" or node.tag == path: - set.append(node) - if not set: - return [] - nodeset = set - -_cache = {} - -## -# (Internal) Compile path. - -def _compile(path): - p = _cache.get(path) - if p is not None: - return p - p = Path(path) - if len(_cache) >= 100: - _cache.clear() - _cache[path] = p - return p - -## -# Find first matching object. - -def find(element, path): - return _compile(path).find(element) - -## -# Find text for first matching object. - -def findtext(element, path, default=None): - return _compile(path).findtext(element, default) - -## -# Find all matching objects. - -def findall(element, path): - return _compile(path).findall(element) diff --git a/Lib/xmlcore/etree/ElementTree.py b/Lib/xmlcore/etree/ElementTree.py deleted file mode 100644 index b39760ea..0000000 --- a/Lib/xmlcore/etree/ElementTree.py +++ /dev/null @@ -1,1257 +0,0 @@ -# -# ElementTree -# $Id: ElementTree.py 2326 2005-03-17 07:45:21Z fredrik $ -# -# light-weight XML support for Python 1.5.2 and later. -# -# history: -# 2001-10-20 fl created (from various sources) -# 2001-11-01 fl return root from parse method -# 2002-02-16 fl sort attributes in lexical order -# 2002-04-06 fl TreeBuilder refactoring, added PythonDoc markup -# 2002-05-01 fl finished TreeBuilder refactoring -# 2002-07-14 fl added basic namespace support to ElementTree.write -# 2002-07-25 fl added QName attribute support -# 2002-10-20 fl fixed encoding in write -# 2002-11-24 fl changed default encoding to ascii; fixed attribute encoding -# 2002-11-27 fl accept file objects or file names for parse/write -# 2002-12-04 fl moved XMLTreeBuilder back to this module -# 2003-01-11 fl fixed entity encoding glitch for us-ascii -# 2003-02-13 fl added XML literal factory -# 2003-02-21 fl added ProcessingInstruction/PI factory -# 2003-05-11 fl added tostring/fromstring helpers -# 2003-05-26 fl added ElementPath support -# 2003-07-05 fl added makeelement factory method -# 2003-07-28 fl added more well-known namespace prefixes -# 2003-08-15 fl fixed typo in ElementTree.findtext (Thomas Dartsch) -# 2003-09-04 fl fall back on emulator if ElementPath is not installed -# 2003-10-31 fl markup updates -# 2003-11-15 fl fixed nested namespace bug -# 2004-03-28 fl added XMLID helper -# 2004-06-02 fl added default support to findtext -# 2004-06-08 fl fixed encoding of non-ascii element/attribute names -# 2004-08-23 fl take advantage of post-2.1 expat features -# 2005-02-01 fl added iterparse implementation -# 2005-03-02 fl fixed iterparse support for pre-2.2 versions -# -# Copyright (c) 1999-2005 by Fredrik Lundh. All rights reserved. -# -# fredrik@pythonware.com -# http://www.pythonware.com -# -# -------------------------------------------------------------------- -# The ElementTree toolkit is -# -# Copyright (c) 1999-2005 by Fredrik Lundh -# -# By obtaining, using, and/or copying this software and/or its -# associated documentation, you agree that you have read, understood, -# and will comply with the following terms and conditions: -# -# Permission to use, copy, modify, and distribute this software and -# its associated documentation for any purpose and without fee is -# hereby granted, provided that the above copyright notice appears in -# all copies, and that both that copyright notice and this permission -# notice appear in supporting documentation, and that the name of -# Secret Labs AB or the author not be used in advertising or publicity -# pertaining to distribution of the software without specific, written -# prior permission. -# -# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD -# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- -# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR -# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY -# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, -# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS -# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE -# OF THIS SOFTWARE. -# -------------------------------------------------------------------- - -# Licensed to PSF under a Contributor Agreement. -# See http://www.python.org/2.4/license for licensing details. - -__all__ = [ - # public symbols - "Comment", - "dump", - "Element", "ElementTree", - "fromstring", - "iselement", "iterparse", - "parse", - "PI", "ProcessingInstruction", - "QName", - "SubElement", - "tostring", - "TreeBuilder", - "VERSION", "XML", - "XMLTreeBuilder", - ] - -## -# The Element type is a flexible container object, designed to -# store hierarchical data structures in memory. The type can be -# described as a cross between a list and a dictionary. -#

-# Each element has a number of properties associated with it: -#

    -#
  • a tag. This is a string identifying what kind of data -# this element represents (the element type, in other words).
    • -#
  • a number of attributes, stored in a Python dictionary.
    • -#
  • a text string.
    • -#
  • an optional tail string.
    • -#
  • a number of child elements, stored in a Python sequence
    • -#
-# -# To create an element instance, use the {@link #Element} or {@link -# #SubElement} factory functions. -#

-# The {@link #ElementTree} class can be used to wrap an element -# structure, and convert it from and to XML. -## - -import string, sys, re - -class _SimpleElementPath: - # emulate pre-1.2 find/findtext/findall behaviour - def find(self, element, tag): - for elem in element: - if elem.tag == tag: - return elem - return None - def findtext(self, element, tag, default=None): - for elem in element: - if elem.tag == tag: - return elem.text or "" - return default - def findall(self, element, tag): - if tag[:3] == ".//": - return element.getiterator(tag[3:]) - result = [] - for elem in element: - if elem.tag == tag: - result.append(elem) - return result - -try: - import ElementPath -except ImportError: - # FIXME: issue warning in this case? - ElementPath = _SimpleElementPath() - -# TODO: add support for custom namespace resolvers/default namespaces -# TODO: add improved support for incremental parsing - -VERSION = "1.2.6" - -## -# Internal element class. This class defines the Element interface, -# and provides a reference implementation of this interface. -#

-# You should not create instances of this class directly. Use the -# appropriate factory functions instead, such as {@link #Element} -# and {@link #SubElement}. -# -# @see Element -# @see SubElement -# @see Comment -# @see ProcessingInstruction - -class _ElementInterface: - # text...tail - - ## - # (Attribute) Element tag. - - tag = None - - ## - # (Attribute) Element attribute dictionary. Where possible, use - # {@link #_ElementInterface.get}, - # {@link #_ElementInterface.set}, - # {@link #_ElementInterface.keys}, and - # {@link #_ElementInterface.items} to access - # element attributes. - - attrib = None - - ## - # (Attribute) Text before first subelement. This is either a - # string or the value None, if there was no text. - - text = None - - ## - # (Attribute) Text after this element's end tag, but before the - # next sibling element's start tag. This is either a string or - # the value None, if there was no text. - - tail = None # text after end tag, if any - - def __init__(self, tag, attrib): - self.tag = tag - self.attrib = attrib - self._children = [] - - def __repr__(self): - return "" % (self.tag, id(self)) - - ## - # Creates a new element object of the same type as this element. - # - # @param tag Element tag. - # @param attrib Element attributes, given as a dictionary. - # @return A new element instance. - - def makeelement(self, tag, attrib): - return Element(tag, attrib) - - ## - # Returns the number of subelements. - # - # @return The number of subelements. - - def __len__(self): - return len(self._children) - - ## - # Returns the given subelement. - # - # @param index What subelement to return. - # @return The given subelement. - # @exception IndexError If the given element does not exist. - - def __getitem__(self, index): - return self._children[index] - - ## - # Replaces the given subelement. - # - # @param index What subelement to replace. - # @param element The new element value. - # @exception IndexError If the given element does not exist. - # @exception AssertionError If element is not a valid object. - - def __setitem__(self, index, element): - assert iselement(element) - self._children[index] = element - - ## - # Deletes the given subelement. - # - # @param index What subelement to delete. - # @exception IndexError If the given element does not exist. - - def __delitem__(self, index): - del self._children[index] - - ## - # Returns a list containing subelements in the given range. - # - # @param start The first subelement to return. - # @param stop The first subelement that shouldn't be returned. - # @return A sequence object containing subelements. - - def __getslice__(self, start, stop): - return self._children[start:stop] - - ## - # Replaces a number of subelements with elements from a sequence. - # - # @param start The first subelement to replace. - # @param stop The first subelement that shouldn't be replaced. - # @param elements A sequence object with zero or more elements. - # @exception AssertionError If a sequence member is not a valid object. - - def __setslice__(self, start, stop, elements): - for element in elements: - assert iselement(element) - self._children[start:stop] = list(elements) - - ## - # Deletes a number of subelements. - # - # @param start The first subelement to delete. - # @param stop The first subelement to leave in there. - - def __delslice__(self, start, stop): - del self._children[start:stop] - - ## - # Adds a subelement to the end of this element. - # - # @param element The element to add. - # @exception AssertionError If a sequence member is not a valid object. - - def append(self, element): - assert iselement(element) - self._children.append(element) - - ## - # Inserts a subelement at the given position in this element. - # - # @param index Where to insert the new subelement. - # @exception AssertionError If the element is not a valid object. - - def insert(self, index, element): - assert iselement(element) - self._children.insert(index, element) - - ## - # Removes a matching subelement. Unlike the find methods, - # this method compares elements based on identity, not on tag - # value or contents. - # - # @param element What element to remove. - # @exception ValueError If a matching element could not be found. - # @exception AssertionError If the element is not a valid object. - - def remove(self, element): - assert iselement(element) - self._children.remove(element) - - ## - # Returns all subelements. The elements are returned in document - # order. - # - # @return A list of subelements. - # @defreturn list of Element instances - - def getchildren(self): - return self._children - - ## - # Finds the first matching subelement, by tag name or path. - # - # @param path What element to look for. - # @return The first matching element, or None if no element was found. - # @defreturn Element or None - - def find(self, path): - return ElementPath.find(self, path) - - ## - # Finds text for the first matching subelement, by tag name or path. - # - # @param path What element to look for. - # @param default What to return if the element was not found. - # @return The text content of the first matching element, or the - # default value no element was found. Note that if the element - # has is found, but has no text content, this method returns an - # empty string. - # @defreturn string - - def findtext(self, path, default=None): - return ElementPath.findtext(self, path, default) - - ## - # Finds all matching subelements, by tag name or path. - # - # @param path What element to look for. - # @return A list or iterator containing all matching elements, - # in document order. - # @defreturn list of Element instances - - def findall(self, path): - return ElementPath.findall(self, path) - - ## - # Resets an element. This function removes all subelements, clears - # all attributes, and sets the text and tail attributes to None. - - def clear(self): - self.attrib.clear() - self._children = [] - self.text = self.tail = None - - ## - # Gets an element attribute. - # - # @param key What attribute to look for. - # @param default What to return if the attribute was not found. - # @return The attribute value, or the default value, if the - # attribute was not found. - # @defreturn string or None - - def get(self, key, default=None): - return self.attrib.get(key, default) - - ## - # Sets an element attribute. - # - # @param key What attribute to set. - # @param value The attribute value. - - def set(self, key, value): - self.attrib[key] = value - - ## - # Gets a list of attribute names. The names are returned in an - # arbitrary order (just like for an ordinary Python dictionary). - # - # @return A list of element attribute names. - # @defreturn list of strings - - def keys(self): - return self.attrib.keys() - - ## - # Gets element attributes, as a sequence. The attributes are - # returned in an arbitrary order. - # - # @return A list of (name, value) tuples for all attributes. - # @defreturn list of (string, string) tuples - - def items(self): - return self.attrib.items() - - ## - # Creates a tree iterator. The iterator loops over this element - # and all subelements, in document order, and returns all elements - # with a matching tag. - #

- # If the tree structure is modified during iteration, the result - # is undefined. - # - # @param tag What tags to look for (default is to return all elements). - # @return A list or iterator containing all the matching elements. - # @defreturn list or iterator - - def getiterator(self, tag=None): - nodes = [] - if tag == "*": - tag = None - if tag is None or self.tag == tag: - nodes.append(self) - for node in self._children: - nodes.extend(node.getiterator(tag)) - return nodes - -# compatibility -_Element = _ElementInterface - -## -# Element factory. This function returns an object implementing the -# standard Element interface. The exact class or type of that object -# is implementation dependent, but it will always be compatible with -# the {@link #_ElementInterface} class in this module. -#

-# The element name, attribute names, and attribute values can be -# either 8-bit ASCII strings or Unicode strings. -# -# @param tag The element name. -# @param attrib An optional dictionary, containing element attributes. -# @param **extra Additional attributes, given as keyword arguments. -# @return An element instance. -# @defreturn Element - -def Element(tag, attrib={}, **extra): - attrib = attrib.copy() - attrib.update(extra) - return _ElementInterface(tag, attrib) - -## -# Subelement factory. This function creates an element instance, and -# appends it to an existing element. -#

-# The element name, attribute names, and attribute values can be -# either 8-bit ASCII strings or Unicode strings. -# -# @param parent The parent element. -# @param tag The subelement name. -# @param attrib An optional dictionary, containing element attributes. -# @param **extra Additional attributes, given as keyword arguments. -# @return An element instance. -# @defreturn Element - -def SubElement(parent, tag, attrib={}, **extra): - attrib = attrib.copy() - attrib.update(extra) - element = parent.makeelement(tag, attrib) - parent.append(element) - return element - -## -# Comment element factory. This factory function creates a special -# element that will be serialized as an XML comment. -#

-# The comment string can be either an 8-bit ASCII string or a Unicode -# string. -# -# @param text A string containing the comment string. -# @return An element instance, representing a comment. -# @defreturn Element - -def Comment(text=None): - element = Element(Comment) - element.text = text - return element - -## -# PI element factory. This factory function creates a special element -# that will be serialized as an XML processing instruction. -# -# @param target A string containing the PI target. -# @param text A string containing the PI contents, if any. -# @return An element instance, representing a PI. -# @defreturn Element - -def ProcessingInstruction(target, text=None): - element = Element(ProcessingInstruction) - element.text = target - if text: - element.text = element.text + " " + text - return element - -PI = ProcessingInstruction - -## -# QName wrapper. This can be used to wrap a QName attribute value, in -# order to get proper namespace handling on output. -# -# @param text A string containing the QName value, in the form {uri}local, -# or, if the tag argument is given, the URI part of a QName. -# @param tag Optional tag. If given, the first argument is interpreted as -# an URI, and this argument is interpreted as a local name. -# @return An opaque object, representing the QName. - -class QName: - def __init__(self, text_or_uri, tag=None): - if tag: - text_or_uri = "{%s}%s" % (text_or_uri, tag) - self.text = text_or_uri - def __str__(self): - return self.text - def __hash__(self): - return hash(self.text) - def __cmp__(self, other): - if isinstance(other, QName): - return cmp(self.text, other.text) - return cmp(self.text, other) - -## -# ElementTree wrapper class. This class represents an entire element -# hierarchy, and adds some extra support for serialization to and from -# standard XML. -# -# @param element Optional root element. -# @keyparam file Optional file handle or name. If given, the -# tree is initialized with the contents of this XML file. - -class ElementTree: - - def __init__(self, element=None, file=None): - assert element is None or iselement(element) - self._root = element # first node - if file: - self.parse(file) - - ## - # Gets the root element for this tree. - # - # @return An element instance. - # @defreturn Element - - def getroot(self): - return self._root - - ## - # Replaces the root element for this tree. This discards the - # current contents of the tree, and replaces it with the given - # element. Use with care. - # - # @param element An element instance. - - def _setroot(self, element): - assert iselement(element) - self._root = element - - ## - # Loads an external XML document into this element tree. - # - # @param source A file name or file object. - # @param parser An optional parser instance. If not given, the - # standard {@link XMLTreeBuilder} parser is used. - # @return The document root element. - # @defreturn Element - - def parse(self, source, parser=None): - if not hasattr(source, "read"): - source = open(source, "rb") - if not parser: - parser = XMLTreeBuilder() - while 1: - data = source.read(32768) - if not data: - break - parser.feed(data) - self._root = parser.close() - return self._root - - ## - # Creates a tree iterator for the root element. The iterator loops - # over all elements in this tree, in document order. - # - # @param tag What tags to look for (default is to return all elements) - # @return An iterator. - # @defreturn iterator - - def getiterator(self, tag=None): - assert self._root is not None - return self._root.getiterator(tag) - - ## - # Finds the first toplevel element with given tag. - # Same as getroot().find(path). - # - # @param path What element to look for. - # @return The first matching element, or None if no element was found. - # @defreturn Element or None - - def find(self, path): - assert self._root is not None - if path[:1] == "/": - path = "." + path - return self._root.find(path) - - ## - # Finds the element text for the first toplevel element with given - # tag. Same as getroot().findtext(path). - # - # @param path What toplevel element to look for. - # @param default What to return if the element was not found. - # @return The text content of the first matching element, or the - # default value no element was found. Note that if the element - # has is found, but has no text content, this method returns an - # empty string. - # @defreturn string - - def findtext(self, path, default=None): - assert self._root is not None - if path[:1] == "/": - path = "." + path - return self._root.findtext(path, default) - - ## - # Finds all toplevel elements with the given tag. - # Same as getroot().findall(path). - # - # @param path What element to look for. - # @return A list or iterator containing all matching elements, - # in document order. - # @defreturn list of Element instances - - def findall(self, path): - assert self._root is not None - if path[:1] == "/": - path = "." + path - return self._root.findall(path) - - ## - # Writes the element tree to a file, as XML. - # - # @param file A file name, or a file object opened for writing. - # @param encoding Optional output encoding (default is US-ASCII). - - def write(self, file, encoding="us-ascii"): - assert self._root is not None - if not hasattr(file, "write"): - file = open(file, "wb") - if not encoding: - encoding = "us-ascii" - elif encoding != "utf-8" and encoding != "us-ascii": - file.write("\n" % encoding) - self._write(file, self._root, encoding, {}) - - def _write(self, file, node, encoding, namespaces): - # write XML to file - tag = node.tag - if tag is Comment: - file.write("" % _escape_cdata(node.text, encoding)) - elif tag is ProcessingInstruction: - file.write("" % _escape_cdata(node.text, encoding)) - else: - items = node.items() - xmlns_items = [] # new namespaces in this scope - try: - if isinstance(tag, QName) or tag[:1] == "{": - tag, xmlns = fixtag(tag, namespaces) - if xmlns: xmlns_items.append(xmlns) - except TypeError: - _raise_serialization_error(tag) - file.write("<" + _encode(tag, encoding)) - if items or xmlns_items: - items.sort() # lexical order - for k, v in items: - try: - if isinstance(k, QName) or k[:1] == "{": - k, xmlns = fixtag(k, namespaces) - if xmlns: xmlns_items.append(xmlns) - except TypeError: - _raise_serialization_error(k) - try: - if isinstance(v, QName): - v, xmlns = fixtag(v, namespaces) - if xmlns: xmlns_items.append(xmlns) - except TypeError: - _raise_serialization_error(v) - file.write(" %s=\"%s\"" % (_encode(k, encoding), - _escape_attrib(v, encoding))) - for k, v in xmlns_items: - file.write(" %s=\"%s\"" % (_encode(k, encoding), - _escape_attrib(v, encoding))) - if node.text or len(node): - file.write(">") - if node.text: - file.write(_escape_cdata(node.text, encoding)) - for n in node: - self._write(file, n, encoding, namespaces) - file.write("") - else: - file.write(" />") - for k, v in xmlns_items: - del namespaces[v] - if node.tail: - file.write(_escape_cdata(node.tail, encoding)) - -# -------------------------------------------------------------------- -# helpers - -## -# Checks if an object appears to be a valid element object. -# -# @param An element instance. -# @return A true value if this is an element object. -# @defreturn flag - -def iselement(element): - # FIXME: not sure about this; might be a better idea to look - # for tag/attrib/text attributes - return isinstance(element, _ElementInterface) or hasattr(element, "tag") - -## -# Writes an element tree or element structure to sys.stdout. This -# function should be used for debugging only. -#

-# The exact output format is implementation dependent. In this -# version, it's written as an ordinary XML file. -# -# @param elem An element tree or an individual element. - -def dump(elem): - # debugging - if not isinstance(elem, ElementTree): - elem = ElementTree(elem) - elem.write(sys.stdout) - tail = elem.getroot().tail - if not tail or tail[-1] != "\n": - sys.stdout.write("\n") - -def _encode(s, encoding): - try: - return s.encode(encoding) - except AttributeError: - return s # 1.5.2: assume the string uses the right encoding - -if sys.version[:3] == "1.5": - _escape = re.compile(r"[&<>\"\x80-\xff]+") # 1.5.2 -else: - _escape = re.compile(eval(r'u"[&<>\"\u0080-\uffff]+"')) - -_escape_map = { - "&": "&", - "<": "<", - ">": ">", - '"': """, -} - -_namespace_map = { - # "well-known" namespace prefixes - "http://www.w3.org/XML/1998/namespace": "xml", - "http://www.w3.org/1999/xhtml": "html", - "http://www.w3.org/1999/02/22-rdf-syntax-ns#": "rdf", - "http://schemas.xmlsoap.org/wsdl/": "wsdl", -} - -def _raise_serialization_error(text): - raise TypeError( - "cannot serialize %r (type %s)" % (text, type(text).__name__) - ) - -def _encode_entity(text, pattern=_escape): - # map reserved and non-ascii characters to numerical entities - def escape_entities(m, map=_escape_map): - out = [] - append = out.append - for char in m.group(): - text = map.get(char) - if text is None: - text = "&#%d;" % ord(char) - append(text) - return string.join(out, "") - try: - return _encode(pattern.sub(escape_entities, text), "ascii") - except TypeError: - _raise_serialization_error(text) - -# -# the following functions assume an ascii-compatible encoding -# (or "utf-16") - -def _escape_cdata(text, encoding=None, replace=string.replace): - # escape character data - try: - if encoding: - try: - text = _encode(text, encoding) - except UnicodeError: - return _encode_entity(text) - text = replace(text, "&", "&") - text = replace(text, "<", "<") - text = replace(text, ">", ">") - return text - except (TypeError, AttributeError): - _raise_serialization_error(text) - -def _escape_attrib(text, encoding=None, replace=string.replace): - # escape attribute value - try: - if encoding: - try: - text = _encode(text, encoding) - except UnicodeError: - return _encode_entity(text) - text = replace(text, "&", "&") - text = replace(text, "'", "'") # FIXME: overkill - text = replace(text, "\"", """) - text = replace(text, "<", "<") - text = replace(text, ">", ">") - return text - except (TypeError, AttributeError): - _raise_serialization_error(text) - -def fixtag(tag, namespaces): - # given a decorated tag (of the form {uri}tag), return prefixed - # tag and namespace declaration, if any - if isinstance(tag, QName): - tag = tag.text - namespace_uri, tag = string.split(tag[1:], "}", 1) - prefix = namespaces.get(namespace_uri) - if prefix is None: - prefix = _namespace_map.get(namespace_uri) - if prefix is None: - prefix = "ns%d" % len(namespaces) - namespaces[namespace_uri] = prefix - if prefix == "xml": - xmlns = None - else: - xmlns = ("xmlns:%s" % prefix, namespace_uri) - else: - xmlns = None - return "%s:%s" % (prefix, tag), xmlns - -## -# Parses an XML document into an element tree. -# -# @param source A filename or file object containing XML data. -# @param parser An optional parser instance. If not given, the -# standard {@link XMLTreeBuilder} parser is used. -# @return An ElementTree instance - -def parse(source, parser=None): - tree = ElementTree() - tree.parse(source, parser) - return tree - -## -# Parses an XML document into an element tree incrementally, and reports -# what's going on to the user. -# -# @param source A filename or file object containing XML data. -# @param events A list of events to report back. If omitted, only "end" -# events are reported. -# @return A (event, elem) iterator. - -class iterparse: - - def __init__(self, source, events=None): - if not hasattr(source, "read"): - source = open(source, "rb") - self._file = source - self._events = [] - self._index = 0 - self.root = self._root = None - self._parser = XMLTreeBuilder() - # wire up the parser for event reporting - parser = self._parser._parser - append = self._events.append - if events is None: - events = ["end"] - for event in events: - if event == "start": - try: - parser.ordered_attributes = 1 - parser.specified_attributes = 1 - def handler(tag, attrib_in, event=event, append=append, - start=self._parser._start_list): - append((event, start(tag, attrib_in))) - parser.StartElementHandler = handler - except AttributeError: - def handler(tag, attrib_in, event=event, append=append, - start=self._parser._start): - append((event, start(tag, attrib_in))) - parser.StartElementHandler = handler - elif event == "end": - def handler(tag, event=event, append=append, - end=self._parser._end): - append((event, end(tag))) - parser.EndElementHandler = handler - elif event == "start-ns": - def handler(prefix, uri, event=event, append=append): - try: - uri = _encode(uri, "ascii") - except UnicodeError: - pass - append((event, (prefix or "", uri))) - parser.StartNamespaceDeclHandler = handler - elif event == "end-ns": - def handler(prefix, event=event, append=append): - append((event, None)) - parser.EndNamespaceDeclHandler = handler - - def next(self): - while 1: - try: - item = self._events[self._index] - except IndexError: - if self._parser is None: - self.root = self._root - try: - raise StopIteration - except NameError: - raise IndexError - # load event buffer - del self._events[:] - self._index = 0 - data = self._file.read(16384) - if data: - self._parser.feed(data) - else: - self._root = self._parser.close() - self._parser = None - else: - self._index = self._index + 1 - return item - - try: - iter - def __iter__(self): - return self - except NameError: - def __getitem__(self, index): - return self.next() - -## -# Parses an XML document from a string constant. This function can -# be used to embed "XML literals" in Python code. -# -# @param source A string containing XML data. -# @return An Element instance. -# @defreturn Element - -def XML(text): - parser = XMLTreeBuilder() - parser.feed(text) - return parser.close() - -## -# Parses an XML document from a string constant, and also returns -# a dictionary which maps from element id:s to elements. -# -# @param source A string containing XML data. -# @return A tuple containing an Element instance and a dictionary. -# @defreturn (Element, dictionary) - -def XMLID(text): - parser = XMLTreeBuilder() - parser.feed(text) - tree = parser.close() - ids = {} - for elem in tree.getiterator(): - id = elem.get("id") - if id: - ids[id] = elem - return tree, ids - -## -# Parses an XML document from a string constant. Same as {@link #XML}. -# -# @def fromstring(text) -# @param source A string containing XML data. -# @return An Element instance. -# @defreturn Element - -fromstring = XML - -## -# Generates a string representation of an XML element, including all -# subelements. -# -# @param element An Element instance. -# @return An encoded string containing the XML data. -# @defreturn string - -def tostring(element, encoding=None): - class dummy: - pass - data = [] - file = dummy() - file.write = data.append - ElementTree(element).write(file, encoding) - return string.join(data, "") - -## -# Generic element structure builder. This builder converts a sequence -# of {@link #TreeBuilder.start}, {@link #TreeBuilder.data}, and {@link -# #TreeBuilder.end} method calls to a well-formed element structure. -#

-# You can use this class to build an element structure using a custom XML -# parser, or a parser for some other XML-like format. -# -# @param element_factory Optional element factory. This factory -# is called to create new Element instances, as necessary. - -class TreeBuilder: - - def __init__(self, element_factory=None): - self._data = [] # data collector - self._elem = [] # element stack - self._last = None # last element - self._tail = None # true if we're after an end tag - if element_factory is None: - element_factory = _ElementInterface - self._factory = element_factory - - ## - # Flushes the parser buffers, and returns the toplevel documen - # element. - # - # @return An Element instance. - # @defreturn Element - - def close(self): - assert len(self._elem) == 0, "missing end tags" - assert self._last != None, "missing toplevel element" - return self._last - - def _flush(self): - if self._data: - if self._last is not None: - text = string.join(self._data, "") - if self._tail: - assert self._last.tail is None, "internal error (tail)" - self._last.tail = text - else: - assert self._last.text is None, "internal error (text)" - self._last.text = text - self._data = [] - - ## - # Adds text to the current element. - # - # @param data A string. This should be either an 8-bit string - # containing ASCII text, or a Unicode string. - - def data(self, data): - self._data.append(data) - - ## - # Opens a new element. - # - # @param tag The element name. - # @param attrib A dictionary containing element attributes. - # @return The opened element. - # @defreturn Element - - def start(self, tag, attrs): - self._flush() - self._last = elem = self._factory(tag, attrs) - if self._elem: - self._elem[-1].append(elem) - self._elem.append(elem) - self._tail = 0 - return elem - - ## - # Closes the current element. - # - # @param tag The element name. - # @return The closed element. - # @defreturn Element - - def end(self, tag): - self._flush() - self._last = self._elem.pop() - assert self._last.tag == tag,\ - "end tag mismatch (expected %s, got %s)" % ( - self._last.tag, tag) - self._tail = 1 - return self._last - -## -# Element structure builder for XML source data, based on the -# expat parser. -# -# @keyparam target Target object. If omitted, the builder uses an -# instance of the standard {@link #TreeBuilder} class. -# @keyparam html Predefine HTML entities. This flag is not supported -# by the current implementation. -# @see #ElementTree -# @see #TreeBuilder - -class XMLTreeBuilder: - - def __init__(self, html=0, target=None): - try: - from xmlcore.parsers import expat - except ImportError: - raise ImportError( - "No module named expat; use SimpleXMLTreeBuilder instead" - ) - self._parser = parser = expat.ParserCreate(None, "}") - if target is None: - target = TreeBuilder() - self._target = target - self._names = {} # name memo cache - # callbacks - parser.DefaultHandlerExpand = self._default - parser.StartElementHandler = self._start - parser.EndElementHandler = self._end - parser.CharacterDataHandler = self._data - # let expat do the buffering, if supported - try: - self._parser.buffer_text = 1 - except AttributeError: - pass - # use new-style attribute handling, if supported - try: - self._parser.ordered_attributes = 1 - self._parser.specified_attributes = 1 - parser.StartElementHandler = self._start_list - except AttributeError: - pass - encoding = None - if not parser.returns_unicode: - encoding = "utf-8" - # target.xml(encoding, None) - self._doctype = None - self.entity = {} - - def _fixtext(self, text): - # convert text string to ascii, if possible - try: - return _encode(text, "ascii") - except UnicodeError: - return text - - def _fixname(self, key): - # expand qname, and convert name string to ascii, if possible - try: - name = self._names[key] - except KeyError: - name = key - if "}" in name: - name = "{" + name - self._names[key] = name = self._fixtext(name) - return name - - def _start(self, tag, attrib_in): - fixname = self._fixname - tag = fixname(tag) - attrib = {} - for key, value in attrib_in.items(): - attrib[fixname(key)] = self._fixtext(value) - return self._target.start(tag, attrib) - - def _start_list(self, tag, attrib_in): - fixname = self._fixname - tag = fixname(tag) - attrib = {} - if attrib_in: - for i in range(0, len(attrib_in), 2): - attrib[fixname(attrib_in[i])] = self._fixtext(attrib_in[i+1]) - return self._target.start(tag, attrib) - - def _data(self, text): - return self._target.data(self._fixtext(text)) - - def _end(self, tag): - return self._target.end(self._fixname(tag)) - - def _default(self, text): - prefix = text[:1] - if prefix == "&": - # deal with undefined entities - try: - self._target.data(self.entity[text[1:-1]]) - except KeyError: - from xmlcore.parsers import expat - raise expat.error( - "undefined entity %s: line %d, column %d" % - (text, self._parser.ErrorLineNumber, - self._parser.ErrorColumnNumber) - ) - elif prefix == "<" and text[:9] == "": - self._doctype = None - return - text = string.strip(text) - if not text: - return - self._doctype.append(text) - n = len(self._doctype) - if n > 2: - type = self._doctype[1] - if type == "PUBLIC" and n == 4: - name, type, pubid, system = self._doctype - elif type == "SYSTEM" and n == 3: - name, type, system = self._doctype - pubid = None - else: - return - if pubid: - pubid = pubid[1:-1] - self.doctype(name, pubid, system[1:-1]) - self._doctype = None - - ## - # Handles a doctype declaration. - # - # @param name Doctype name. - # @param pubid Public identifier. - # @param system System identifier. - - def doctype(self, name, pubid, system): - pass - - ## - # Feeds data to the parser. - # - # @param data Encoded data. - - def feed(self, data): - self._parser.Parse(data, 0) - - ## - # Finishes feeding data to the parser. - # - # @return An element structure. - # @defreturn Element - - def close(self): - self._parser.Parse("", 1) # end of data - tree = self._target.close() - del self._target, self._parser # get rid of circular references - return tree diff --git a/Lib/xmlcore/etree/__init__.py b/Lib/xmlcore/etree/__init__.py deleted file mode 100644 index 3dd2c92..0000000 --- a/Lib/xmlcore/etree/__init__.py +++ /dev/null @@ -1,33 +0,0 @@ -# $Id: __init__.py 1821 2004-06-03 16:57:49Z fredrik $ -# elementtree package - -# -------------------------------------------------------------------- -# The ElementTree toolkit is -# -# Copyright (c) 1999-2004 by Fredrik Lundh -# -# By obtaining, using, and/or copying this software and/or its -# associated documentation, you agree that you have read, understood, -# and will comply with the following terms and conditions: -# -# Permission to use, copy, modify, and distribute this software and -# its associated documentation for any purpose and without fee is -# hereby granted, provided that the above copyright notice appears in -# all copies, and that both that copyright notice and this permission -# notice appear in supporting documentation, and that the name of -# Secret Labs AB or the author not be used in advertising or publicity -# pertaining to distribution of the software without specific, written -# prior permission. -# -# SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD -# TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANT- -# ABILITY AND FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR -# BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY -# DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, -# WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS -# ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE -# OF THIS SOFTWARE. -# -------------------------------------------------------------------- - -# Licensed to PSF under a Contributor Agreement. -# See http://www.python.org/2.4/license for licensing details. diff --git a/Lib/xmlcore/etree/cElementTree.py b/Lib/xmlcore/etree/cElementTree.py deleted file mode 100644 index a6f127a..0000000 --- a/Lib/xmlcore/etree/cElementTree.py +++ /dev/null @@ -1,3 +0,0 @@ -# Wrapper module for _elementtree - -from _elementtree import * diff --git a/Lib/xmlcore/parsers/__init__.py b/Lib/xmlcore/parsers/__init__.py deleted file mode 100644 index eb314a3..0000000 --- a/Lib/xmlcore/parsers/__init__.py +++ /dev/null @@ -1,8 +0,0 @@ -"""Python interfaces to XML parsers. - -This package contains one module: - -expat -- Python wrapper for James Clark's Expat parser, with namespace - support. - -""" diff --git a/Lib/xmlcore/parsers/expat.py b/Lib/xmlcore/parsers/expat.py deleted file mode 100644 index 11359a0..0000000 --- a/Lib/xmlcore/parsers/expat.py +++ /dev/null @@ -1,4 +0,0 @@ -"""Interface to the Expat non-validating XML parser.""" -__version__ = '$Revision$' - -from pyexpat import * diff --git a/Lib/xmlcore/sax/__init__.py b/Lib/xmlcore/sax/__init__.py deleted file mode 100644 index 8afbdb0..0000000 --- a/Lib/xmlcore/sax/__init__.py +++ /dev/null @@ -1,108 +0,0 @@ -"""Simple API for XML (SAX) implementation for Python. - -This module provides an implementation of the SAX 2 interface; -information about the Java version of the interface can be found at -http://www.megginson.com/SAX/. The Python version of the interface is -documented at <...>. - -This package contains the following modules: - -handler -- Base classes and constants which define the SAX 2 API for - the 'client-side' of SAX for Python. - -saxutils -- Implementation of the convenience classes commonly used to - work with SAX. - -xmlreader -- Base classes and constants which define the SAX 2 API for - the parsers used with SAX for Python. - -expatreader -- Driver that allows use of the Expat parser with SAX. -""" - -from .xmlreader import InputSource -from .handler import ContentHandler, ErrorHandler -from ._exceptions import (SAXException, SAXNotRecognizedException, - SAXParseException, SAXNotSupportedException, - SAXReaderNotAvailable) - - -def parse(source, handler, errorHandler=ErrorHandler()): - parser = make_parser() - parser.setContentHandler(handler) - parser.setErrorHandler(errorHandler) - parser.parse(source) - -def parseString(string, handler, errorHandler=ErrorHandler()): - try: - from cStringIO import StringIO - except ImportError: - from StringIO import StringIO - - if errorHandler is None: - errorHandler = ErrorHandler() - parser = make_parser() - parser.setContentHandler(handler) - parser.setErrorHandler(errorHandler) - - inpsrc = InputSource() - inpsrc.setByteStream(StringIO(string)) - parser.parse(inpsrc) - -# this is the parser list used by the make_parser function if no -# alternatives are given as parameters to the function - -default_parser_list = ["xmlcore.sax.expatreader"] - -# tell modulefinder that importing sax potentially imports expatreader -_false = 0 -if _false: - import xmlcore.sax.expatreader - -import os, sys -if os.environ.has_key("PY_SAX_PARSER"): - default_parser_list = os.environ["PY_SAX_PARSER"].split(",") -del os - -_key = "python.xml.sax.parser" -if sys.platform[:4] == "java" and sys.registry.containsKey(_key): - default_parser_list = sys.registry.getProperty(_key).split(",") - - -def make_parser(parser_list = []): - """Creates and returns a SAX parser. - - Creates the first parser it is able to instantiate of the ones - given in the list created by doing parser_list + - default_parser_list. The lists must contain the names of Python - modules containing both a SAX parser and a create_parser function.""" - - for parser_name in parser_list + default_parser_list: - try: - return _create_parser(parser_name) - except ImportError,e: - import sys - if sys.modules.has_key(parser_name): - # The parser module was found, but importing it - # failed unexpectedly, pass this exception through - raise - except SAXReaderNotAvailable: - # The parser module detected that it won't work properly, - # so try the next one - pass - - raise SAXReaderNotAvailable("No parsers found", None) - -# --- Internal utility methods used by make_parser - -if sys.platform[ : 4] == "java": - def _create_parser(parser_name): - from org.python.core import imp - drv_module = imp.importName(parser_name, 0, globals()) - return drv_module.create_parser() - -else: - def _create_parser(parser_name): - drv_module = __import__(parser_name,{},{},['create_parser']) - return drv_module.create_parser() - -del sys diff --git a/Lib/xmlcore/sax/_exceptions.py b/Lib/xmlcore/sax/_exceptions.py deleted file mode 100644 index fdd614a..0000000 --- a/Lib/xmlcore/sax/_exceptions.py +++ /dev/null @@ -1,131 +0,0 @@ -"""Different kinds of SAX Exceptions""" -import sys -if sys.platform[:4] == "java": - from java.lang import Exception -del sys - -# ===== SAXEXCEPTION ===== - -class SAXException(Exception): - """Encapsulate an XML error or warning. This class can contain - basic error or warning information from either the XML parser or - the application: you can subclass it to provide additional - functionality, or to add localization. Note that although you will - receive a SAXException as the argument to the handlers in the - ErrorHandler interface, you are not actually required to throw - the exception; instead, you can simply read the information in - it.""" - - def __init__(self, msg, exception=None): - """Creates an exception. The message is required, but the exception - is optional.""" - self._msg = msg - self._exception = exception - Exception.__init__(self, msg) - - def getMessage(self): - "Return a message for this exception." - return self._msg - - def getException(self): - "Return the embedded exception, or None if there was none." - return self._exception - - def __str__(self): - "Create a string representation of the exception." - return self._msg - - def __getitem__(self, ix): - """Avoids weird error messages if someone does exception[ix] by - mistake, since Exception has __getitem__ defined.""" - raise AttributeError("__getitem__") - - -# ===== SAXPARSEEXCEPTION ===== - -class SAXParseException(SAXException): - """Encapsulate an XML parse error or warning. - - This exception will include information for locating the error in - the original XML document. Note that although the application will - receive a SAXParseException as the argument to the handlers in the - ErrorHandler interface, the application is not actually required - to throw the exception; instead, it can simply read the - information in it and take a different action. - - Since this exception is a subclass of SAXException, it inherits - the ability to wrap another exception.""" - - def __init__(self, msg, exception, locator): - "Creates the exception. The exception parameter is allowed to be None." - SAXException.__init__(self, msg, exception) - self._locator = locator - - # We need to cache this stuff at construction time. - # If this exception is thrown, the objects through which we must - # traverse to get this information may be deleted by the time - # it gets caught. - self._systemId = self._locator.getSystemId() - self._colnum = self._locator.getColumnNumber() - self._linenum = self._locator.getLineNumber() - - def getColumnNumber(self): - """The column number of the end of the text where the exception - occurred.""" - return self._colnum - - def getLineNumber(self): - "The line number of the end of the text where the exception occurred." - return self._linenum - - def getPublicId(self): - "Get the public identifier of the entity where the exception occurred." - return self._locator.getPublicId() - - def getSystemId(self): - "Get the system identifier of the entity where the exception occurred." - return self._systemId - - def __str__(self): - "Create a string representation of the exception." - sysid = self.getSystemId() - if sysid is None: - sysid = "" - linenum = self.getLineNumber() - if linenum is None: - linenum = "?" - colnum = self.getColumnNumber() - if colnum is None: - colnum = "?" - return "%s:%s:%s: %s" % (sysid, linenum, colnum, self._msg) - - -# ===== SAXNOTRECOGNIZEDEXCEPTION ===== - -class SAXNotRecognizedException(SAXException): - """Exception class for an unrecognized identifier. - - An XMLReader will raise this exception when it is confronted with an - unrecognized feature or property. SAX applications and extensions may - use this class for similar purposes.""" - - -# ===== SAXNOTSUPPORTEDEXCEPTION ===== - -class SAXNotSupportedException(SAXException): - """Exception class for an unsupported operation. - - An XMLReader will raise this exception when a service it cannot - perform is requested (specifically setting a state or value). SAX - applications and extensions may use this class for similar - purposes.""" - -# ===== SAXNOTSUPPORTEDEXCEPTION ===== - -class SAXReaderNotAvailable(SAXNotSupportedException): - """Exception class for a missing driver. - - An XMLReader module (driver) should raise this exception when it - is first imported, e.g. when a support module cannot be imported. - It also may be raised during parsing, e.g. if executing an external - program is not permitted.""" diff --git a/Lib/xmlcore/sax/expatreader.py b/Lib/xmlcore/sax/expatreader.py deleted file mode 100644 index 6fbd22e..0000000 --- a/Lib/xmlcore/sax/expatreader.py +++ /dev/null @@ -1,414 +0,0 @@ -""" -SAX driver for the pyexpat C module. This driver works with -pyexpat.__version__ == '2.22'. -""" - -version = "0.20" - -from xmlcore.sax._exceptions import * -from xmlcore.sax.handler import feature_validation, feature_namespaces -from xmlcore.sax.handler import feature_namespace_prefixes -from xmlcore.sax.handler import feature_external_ges, feature_external_pes -from xmlcore.sax.handler import feature_string_interning -from xmlcore.sax.handler import property_xml_string, property_interning_dict - -# xmlcore.parsers.expat does not raise ImportError in Jython -import sys -if sys.platform[:4] == "java": - raise SAXReaderNotAvailable("expat not available in Java", None) -del sys - -try: - from xmlcore.parsers import expat -except ImportError: - raise SAXReaderNotAvailable("expat not supported", None) -else: - if not hasattr(expat, "ParserCreate"): - raise SAXReaderNotAvailable("expat not supported", None) -from xmlcore.sax import xmlreader, saxutils, handler - -AttributesImpl = xmlreader.AttributesImpl -AttributesNSImpl = xmlreader.AttributesNSImpl - -# If we're using a sufficiently recent version of Python, we can use -# weak references to avoid cycles between the parser and content -# handler, otherwise we'll just have to pretend. -try: - import _weakref -except ImportError: - def _mkproxy(o): - return o -else: - import weakref - _mkproxy = weakref.proxy - del weakref, _weakref - -# --- ExpatLocator - -class ExpatLocator(xmlreader.Locator): - """Locator for use with the ExpatParser class. - - This uses a weak reference to the parser object to avoid creating - a circular reference between the parser and the content handler. - """ - def __init__(self, parser): - self._ref = _mkproxy(parser) - - def getColumnNumber(self): - parser = self._ref - if parser._parser is None: - return None - return parser._parser.ErrorColumnNumber - - def getLineNumber(self): - parser = self._ref - if parser._parser is None: - return 1 - return parser._parser.ErrorLineNumber - - def getPublicId(self): - parser = self._ref - if parser is None: - return None - return parser._source.getPublicId() - - def getSystemId(self): - parser = self._ref - if parser is None: - return None - return parser._source.getSystemId() - - -# --- ExpatParser - -class ExpatParser(xmlreader.IncrementalParser, xmlreader.Locator): - """SAX driver for the pyexpat C module.""" - - def __init__(self, namespaceHandling=0, bufsize=2**16-20): - xmlreader.IncrementalParser.__init__(self, bufsize) - self._source = xmlreader.InputSource() - self._parser = None - self._namespaces = namespaceHandling - self._lex_handler_prop = None - self._parsing = 0 - self._entity_stack = [] - self._external_ges = 1 - self._interning = None - - # XMLReader methods - - def parse(self, source): - "Parse an XML document from a URL or an InputSource." - source = saxutils.prepare_input_source(source) - - self._source = source - self.reset() - self._cont_handler.setDocumentLocator(ExpatLocator(self)) - xmlreader.IncrementalParser.parse(self, source) - - def prepareParser(self, source): - if source.getSystemId() != None: - self._parser.SetBase(source.getSystemId()) - - # Redefined setContentHandler to allow changing handlers during parsing - - def setContentHandler(self, handler): - xmlreader.IncrementalParser.setContentHandler(self, handler) - if self._parsing: - self._reset_cont_handler() - - def getFeature(self, name): - if name == feature_namespaces: - return self._namespaces - elif name == feature_string_interning: - return self._interning is not None - elif name in (feature_validation, feature_external_pes, - feature_namespace_prefixes): - return 0 - elif name == feature_external_ges: - return self._external_ges - raise SAXNotRecognizedException("Feature '%s' not recognized" % name) - - def setFeature(self, name, state): - if self._parsing: - raise SAXNotSupportedException("Cannot set features while parsing") - - if name == feature_namespaces: - self._namespaces = state - elif name == feature_external_ges: - self._external_ges = state - elif name == feature_string_interning: - if state: - if self._interning is None: - self._interning = {} - else: - self._interning = None - elif name == feature_validation: - if state: - raise SAXNotSupportedException( - "expat does not support validation") - elif name == feature_external_pes: - if state: - raise SAXNotSupportedException( - "expat does not read external parameter entities") - elif name == feature_namespace_prefixes: - if state: - raise SAXNotSupportedException( - "expat does not report namespace prefixes") - else: - raise SAXNotRecognizedException( - "Feature '%s' not recognized" % name) - - def getProperty(self, name): - if name == handler.property_lexical_handler: - return self._lex_handler_prop - elif name == property_interning_dict: - return self._interning - elif name == property_xml_string: - if self._parser: - if hasattr(self._parser, "GetInputContext"): - return self._parser.GetInputContext() - else: - raise SAXNotRecognizedException( - "This version of expat does not support getting" - " the XML string") - else: - raise SAXNotSupportedException( - "XML string cannot be returned when not parsing") - raise SAXNotRecognizedException("Property '%s' not recognized" % name) - - def setProperty(self, name, value): - if name == handler.property_lexical_handler: - self._lex_handler_prop = value - if self._parsing: - self._reset_lex_handler_prop() - elif name == property_interning_dict: - self._interning = value - elif name == property_xml_string: - raise SAXNotSupportedException("Property '%s' cannot be set" % - name) - else: - raise SAXNotRecognizedException("Property '%s' not recognized" % - name) - - # IncrementalParser methods - - def feed(self, data, isFinal = 0): - if not self._parsing: - self.reset() - self._parsing = 1 - self._cont_handler.startDocument() - - try: - # The isFinal parameter is internal to the expat reader. - # If it is set to true, expat will check validity of the entire - # document. When feeding chunks, they are not normally final - - # except when invoked from close. - self._parser.Parse(data, isFinal) - except expat.error, e: - exc = SAXParseException(expat.ErrorString(e.code), e, self) - # FIXME: when to invoke error()? - self._err_handler.fatalError(exc) - - def close(self): - if self._entity_stack: - # If we are completing an external entity, do nothing here - return - self.feed("", isFinal = 1) - self._cont_handler.endDocument() - self._parsing = 0 - # break cycle created by expat handlers pointing to our methods - self._parser = None - - def _reset_cont_handler(self): - self._parser.ProcessingInstructionHandler = \ - self._cont_handler.processingInstruction - self._parser.CharacterDataHandler = self._cont_handler.characters - - def _reset_lex_handler_prop(self): - lex = self._lex_handler_prop - parser = self._parser - if lex is None: - parser.CommentHandler = None - parser.StartCdataSectionHandler = None - parser.EndCdataSectionHandler = None - parser.StartDoctypeDeclHandler = None - parser.EndDoctypeDeclHandler = None - else: - parser.CommentHandler = lex.comment - parser.StartCdataSectionHandler = lex.startCDATA - parser.EndCdataSectionHandler = lex.endCDATA - parser.StartDoctypeDeclHandler = self.start_doctype_decl - parser.EndDoctypeDeclHandler = lex.endDTD - - def reset(self): - if self._namespaces: - self._parser = expat.ParserCreate(self._source.getEncoding(), " ", - intern=self._interning) - self._parser.namespace_prefixes = 1 - self._parser.StartElementHandler = self.start_element_ns - self._parser.EndElementHandler = self.end_element_ns - else: - self._parser = expat.ParserCreate(self._source.getEncoding(), - intern = self._interning) - self._parser.StartElementHandler = self.start_element - self._parser.EndElementHandler = self.end_element - - self._reset_cont_handler() - self._parser.UnparsedEntityDeclHandler = self.unparsed_entity_decl - self._parser.NotationDeclHandler = self.notation_decl - self._parser.StartNamespaceDeclHandler = self.start_namespace_decl - self._parser.EndNamespaceDeclHandler = self.end_namespace_decl - - self._decl_handler_prop = None - if self._lex_handler_prop: - self._reset_lex_handler_prop() -# self._parser.DefaultHandler = -# self._parser.DefaultHandlerExpand = -# self._parser.NotStandaloneHandler = - self._parser.ExternalEntityRefHandler = self.external_entity_ref - try: - self._parser.SkippedEntityHandler = self.skipped_entity_handler - except AttributeError: - # This pyexpat does not support SkippedEntity - pass - self._parser.SetParamEntityParsing( - expat.XML_PARAM_ENTITY_PARSING_UNLESS_STANDALONE) - - self._parsing = 0 - self._entity_stack = [] - - # Locator methods - - def getColumnNumber(self): - if self._parser is None: - return None - return self._parser.ErrorColumnNumber - - def getLineNumber(self): - if self._parser is None: - return 1 - return self._parser.ErrorLineNumber - - def getPublicId(self): - return self._source.getPublicId() - - def getSystemId(self): - return self._source.getSystemId() - - # event handlers - def start_element(self, name, attrs): - self._cont_handler.startElement(name, AttributesImpl(attrs)) - - def end_element(self, name): - self._cont_handler.endElement(name) - - def start_element_ns(self, name, attrs): - pair = name.split() - if len(pair) == 1: - # no namespace - pair = (None, name) - elif len(pair) == 3: - pair = pair[0], pair[1] - else: - # default namespace - pair = tuple(pair) - - newattrs = {} - qnames = {} - for (aname, value) in attrs.items(): - parts = aname.split() - length = len(parts) - if length == 1: - # no namespace - qname = aname - apair = (None, aname) - elif length == 3: - qname = "%s:%s" % (parts[2], parts[1]) - apair = parts[0], parts[1] - else: - # default namespace - qname = parts[1] - apair = tuple(parts) - - newattrs[apair] = value - qnames[apair] = qname - - self._cont_handler.startElementNS(pair, None, - AttributesNSImpl(newattrs, qnames)) - - def end_element_ns(self, name): - pair = name.split() - if len(pair) == 1: - pair = (None, name) - elif len(pair) == 3: - pair = pair[0], pair[1] - else: - pair = tuple(pair) - - self._cont_handler.endElementNS(pair, None) - - # this is not used (call directly to ContentHandler) - def processing_instruction(self, target, data): - self._cont_handler.processingInstruction(target, data) - - # this is not used (call directly to ContentHandler) - def character_data(self, data): - self._cont_handler.characters(data) - - def start_namespace_decl(self, prefix, uri): - self._cont_handler.startPrefixMapping(prefix, uri) - - def end_namespace_decl(self, prefix): - self._cont_handler.endPrefixMapping(prefix) - - def start_doctype_decl(self, name, sysid, pubid, has_internal_subset): - self._lex_handler_prop.startDTD(name, pubid, sysid) - - def unparsed_entity_decl(self, name, base, sysid, pubid, notation_name): - self._dtd_handler.unparsedEntityDecl(name, pubid, sysid, notation_name) - - def notation_decl(self, name, base, sysid, pubid): - self._dtd_handler.notationDecl(name, pubid, sysid) - - def external_entity_ref(self, context, base, sysid, pubid): - if not self._external_ges: - return 1 - - source = self._ent_handler.resolveEntity(pubid, sysid) - source = saxutils.prepare_input_source(source, - self._source.getSystemId() or - "") - - self._entity_stack.append((self._parser, self._source)) - self._parser = self._parser.ExternalEntityParserCreate(context) - self._source = source - - try: - xmlreader.IncrementalParser.parse(self, source) - except: - return 0 # FIXME: save error info here? - - (self._parser, self._source) = self._entity_stack[-1] - del self._entity_stack[-1] - return 1 - - def skipped_entity_handler(self, name, is_pe): - if is_pe: - # The SAX spec requires to report skipped PEs with a '%' - name = '%'+name - self._cont_handler.skippedEntity(name) - -# --- - -def create_parser(*args, **kwargs): - return ExpatParser(*args, **kwargs) - -# --- - -if __name__ == "__main__": - import xmlcore.sax - p = create_parser() - p.setContentHandler(xmlcore.sax.XMLGenerator()) - p.setErrorHandler(xmlcore.sax.ErrorHandler()) - p.parse("../../../hamlet.xml") diff --git a/Lib/xmlcore/sax/handler.py b/Lib/xmlcore/sax/handler.py deleted file mode 100644 index f9e91b6..0000000 --- a/Lib/xmlcore/sax/handler.py +++ /dev/null @@ -1,342 +0,0 @@ -""" -This module contains the core classes of version 2.0 of SAX for Python. -This file provides only default classes with absolutely minimum -functionality, from which drivers and applications can be subclassed. - -Many of these classes are empty and are included only as documentation -of the interfaces. - -$Id$ -""" - -version = '2.0beta' - -#============================================================================ -# -# HANDLER INTERFACES -# -#============================================================================ - -# ===== ERRORHANDLER ===== - -class ErrorHandler: - """Basic interface for SAX error handlers. - - If you create an object that implements this interface, then - register the object with your XMLReader, the parser will call the - methods in your object to report all warnings and errors. There - are three levels of errors available: warnings, (possibly) - recoverable errors, and unrecoverable errors. All methods take a - SAXParseException as the only parameter.""" - - def error(self, exception): - "Handle a recoverable error." - raise exception - - def fatalError(self, exception): - "Handle a non-recoverable error." - raise exception - - def warning(self, exception): - "Handle a warning." - print exception - - -# ===== CONTENTHANDLER ===== - -class ContentHandler: - """Interface for receiving logical document content events. - - This is the main callback interface in SAX, and the one most - important to applications. The order of events in this interface - mirrors the order of the information in the document.""" - - def __init__(self): - self._locator = None - - def setDocumentLocator(self, locator): - """Called by the parser to give the application a locator for - locating the origin of document events. - - SAX parsers are strongly encouraged (though not absolutely - required) to supply a locator: if it does so, it must supply - the locator to the application by invoking this method before - invoking any of the other methods in the DocumentHandler - interface. - - The locator allows the application to determine the end - position of any document-related event, even if the parser is - not reporting an error. Typically, the application will use - this information for reporting its own errors (such as - character content that does not match an application's - business rules). The information returned by the locator is - probably not sufficient for use with a search engine. - - Note that the locator will return correct information only - during the invocation of the events in this interface. The - application should not attempt to use it at any other time.""" - self._locator = locator - - def startDocument(self): - """Receive notification of the beginning of a document. - - The SAX parser will invoke this method only once, before any - other methods in this interface or in DTDHandler (except for - setDocumentLocator).""" - - def endDocument(self): - """Receive notification of the end of a document. - - The SAX parser will invoke this method only once, and it will - be the last method invoked during the parse. The parser shall - not invoke this method until it has either abandoned parsing - (because of an unrecoverable error) or reached the end of - input.""" - - def startPrefixMapping(self, prefix, uri): - """Begin the scope of a prefix-URI Namespace mapping. - - The information from this event is not necessary for normal - Namespace processing: the SAX XML reader will automatically - replace prefixes for element and attribute names when the - http://xml.org/sax/features/namespaces feature is true (the - default). - - There are cases, however, when applications need to use - prefixes in character data or in attribute values, where they - cannot safely be expanded automatically; the - start/endPrefixMapping event supplies the information to the - application to expand prefixes in those contexts itself, if - necessary. - - Note that start/endPrefixMapping events are not guaranteed to - be properly nested relative to each-other: all - startPrefixMapping events will occur before the corresponding - startElement event, and all endPrefixMapping events will occur - after the corresponding endElement event, but their order is - not guaranteed.""" - - def endPrefixMapping(self, prefix): - """End the scope of a prefix-URI mapping. - - See startPrefixMapping for details. This event will always - occur after the corresponding endElement event, but the order - of endPrefixMapping events is not otherwise guaranteed.""" - - def startElement(self, name, attrs): - """Signals the start of an element in non-namespace mode. - - The name parameter contains the raw XML 1.0 name of the - element type as a string and the attrs parameter holds an - instance of the Attributes class containing the attributes of - the element.""" - - def endElement(self, name): - """Signals the end of an element in non-namespace mode. - - The name parameter contains the name of the element type, just - as with the startElement event.""" - - def startElementNS(self, name, qname, attrs): - """Signals the start of an element in namespace mode. - - The name parameter contains the name of the element type as a - (uri, localname) tuple, the qname parameter the raw XML 1.0 - name used in the source document, and the attrs parameter - holds an instance of the Attributes class containing the - attributes of the element. - - The uri part of the name tuple is None for elements which have - no namespace.""" - - def endElementNS(self, name, qname): - """Signals the end of an element in namespace mode. - - The name parameter contains the name of the element type, just - as with the startElementNS event.""" - - def characters(self, content): - """Receive notification of character data. - - The Parser will call this method to report each chunk of - character data. SAX parsers may return all contiguous - character data in a single chunk, or they may split it into - several chunks; however, all of the characters in any single - event must come from the same external entity so that the - Locator provides useful information.""" - - def ignorableWhitespace(self, whitespace): - """Receive notification of ignorable whitespace in element content. - - Validating Parsers must use this method to report each chunk - of ignorable whitespace (see the W3C XML 1.0 recommendation, - section 2.10): non-validating parsers may also use this method - if they are capable of parsing and using content models. - - SAX parsers may return all contiguous whitespace in a single - chunk, or they may split it into several chunks; however, all - of the characters in any single event must come from the same - external entity, so that the Locator provides useful - information.""" - - def processingInstruction(self, target, data): - """Receive notification of a processing instruction. - - The Parser will invoke this method once for each processing - instruction found: note that processing instructions may occur - before or after the main document element. - - A SAX parser should never report an XML declaration (XML 1.0, - section 2.8) or a text declaration (XML 1.0, section 4.3.1) - using this method.""" - - def skippedEntity(self, name): - """Receive notification of a skipped entity. - - The Parser will invoke this method once for each entity - skipped. Non-validating processors may skip entities if they - have not seen the declarations (because, for example, the - entity was declared in an external DTD subset). All processors - may skip external entities, depending on the values of the - http://xml.org/sax/features/external-general-entities and the - http://xml.org/sax/features/external-parameter-entities - properties.""" - - -# ===== DTDHandler ===== - -class DTDHandler: - """Handle DTD events. - - This interface specifies only those DTD events required for basic - parsing (unparsed entities and attributes).""" - - def notationDecl(self, name, publicId, systemId): - "Handle a notation declaration event." - - def unparsedEntityDecl(self, name, publicId, systemId, ndata): - "Handle an unparsed entity declaration event." - - -# ===== ENTITYRESOLVER ===== - -class EntityResolver: - """Basic interface for resolving entities. If you create an object - implementing this interface, then register the object with your - Parser, the parser will call the method in your object to - resolve all external entities. Note that DefaultHandler implements - this interface with the default behaviour.""" - - def resolveEntity(self, publicId, systemId): - """Resolve the system identifier of an entity and return either - the system identifier to read from as a string, or an InputSource - to read from.""" - return systemId - - -#============================================================================ -# -# CORE FEATURES -# -#============================================================================ - -feature_namespaces = "http://xml.org/sax/features/namespaces" -# true: Perform Namespace processing (default). -# false: Optionally do not perform Namespace processing -# (implies namespace-prefixes). -# access: (parsing) read-only; (not parsing) read/write - -feature_namespace_prefixes = "http://xml.org/sax/features/namespace-prefixes" -# true: Report the original prefixed names and attributes used for Namespace -# declarations. -# false: Do not report attributes used for Namespace declarations, and -# optionally do not report original prefixed names (default). -# access: (parsing) read-only; (not parsing) read/write - -feature_string_interning = "http://xml.org/sax/features/string-interning" -# true: All element names, prefixes, attribute names, Namespace URIs, and -# local names are interned using the built-in intern function. -# false: Names are not necessarily interned, although they may be (default). -# access: (parsing) read-only; (not parsing) read/write - -feature_validation = "http://xml.org/sax/features/validation" -# true: Report all validation errors (implies external-general-entities and -# external-parameter-entities). -# false: Do not report validation errors. -# access: (parsing) read-only; (not parsing) read/write - -feature_external_ges = "http://xml.org/sax/features/external-general-entities" -# true: Include all external general (text) entities. -# false: Do not include external general entities. -# access: (parsing) read-only; (not parsing) read/write - -feature_external_pes = "http://xml.org/sax/features/external-parameter-entities" -# true: Include all external parameter entities, including the external -# DTD subset. -# false: Do not include any external parameter entities, even the external -# DTD subset. -# access: (parsing) read-only; (not parsing) read/write - -all_features = [feature_namespaces, - feature_namespace_prefixes, - feature_string_interning, - feature_validation, - feature_external_ges, - feature_external_pes] - - -#============================================================================ -# -# CORE PROPERTIES -# -#============================================================================ - -property_lexical_handler = "http://xml.org/sax/properties/lexical-handler" -# data type: xml.sax.sax2lib.LexicalHandler -# description: An optional extension handler for lexical events like comments. -# access: read/write - -property_declaration_handler = "http://xml.org/sax/properties/declaration-handler" -# data type: xml.sax.sax2lib.DeclHandler -# description: An optional extension handler for DTD-related events other -# than notations and unparsed entities. -# access: read/write - -property_dom_node = "http://xml.org/sax/properties/dom-node" -# data type: org.w3c.dom.Node -# description: When parsing, the current DOM node being visited if this is -# a DOM iterator; when not parsing, the root DOM node for -# iteration. -# access: (parsing) read-only; (not parsing) read/write - -property_xml_string = "http://xml.org/sax/properties/xml-string" -# data type: String -# description: The literal string of characters that was the source for -# the current event. -# access: read-only - -property_encoding = "http://www.python.org/sax/properties/encoding" -# data type: String -# description: The name of the encoding to assume for input data. -# access: write: set the encoding, e.g. established by a higher-level -# protocol. May change during parsing (e.g. after -# processing a META tag) -# read: return the current encoding (possibly established through -# auto-detection. -# initial value: UTF-8 -# - -property_interning_dict = "http://www.python.org/sax/properties/interning-dict" -# data type: Dictionary -# description: The dictionary used to intern common strings in the document -# access: write: Request that the parser uses a specific dictionary, to -# allow interning across different documents -# read: return the current interning dictionary, or None -# - -all_properties = [property_lexical_handler, - property_dom_node, - property_declaration_handler, - property_xml_string, - property_encoding, - property_interning_dict] diff --git a/Lib/xmlcore/sax/saxutils.py b/Lib/xmlcore/sax/saxutils.py deleted file mode 100644 index 880de80..0000000 --- a/Lib/xmlcore/sax/saxutils.py +++ /dev/null @@ -1,297 +0,0 @@ -"""\ -A library of useful helper classes to the SAX classes, for the -convenience of application and driver writers. -""" - -import os, urlparse, urllib, types -from . import handler -from . import xmlreader - -try: - _StringTypes = [types.StringType, types.UnicodeType] -except AttributeError: - _StringTypes = [types.StringType] - -# See whether the xmlcharrefreplace error handler is -# supported -try: - from codecs import xmlcharrefreplace_errors - _error_handling = "xmlcharrefreplace" - del xmlcharrefreplace_errors -except ImportError: - _error_handling = "strict" - -def __dict_replace(s, d): - """Replace substrings of a string using a dictionary.""" - for key, value in d.items(): - s = s.replace(key, value) - return s - -def escape(data, entities={}): - """Escape &, <, and > in a string of data. - - You can escape other strings of data by passing a dictionary as - the optional entities parameter. The keys and values must all be - strings; each key will be replaced with its corresponding value. - """ - - # must do ampersand first - data = data.replace("&", "&") - data = data.replace(">", ">") - data = data.replace("<", "<") - if entities: - data = __dict_replace(data, entities) - return data - -def unescape(data, entities={}): - """Unescape &, <, and > in a string of data. - - You can unescape other strings of data by passing a dictionary as - the optional entities parameter. The keys and values must all be - strings; each key will be replaced with its corresponding value. - """ - data = data.replace("<", "<") - data = data.replace(">", ">") - if entities: - data = __dict_replace(data, entities) - # must do ampersand last - return data.replace("&", "&") - -def quoteattr(data, entities={}): - """Escape and quote an attribute value. - - Escape &, <, and > in a string of data, then quote it for use as - an attribute value. The \" character will be escaped as well, if - necessary. - - You can escape other strings of data by passing a dictionary as - the optional entities parameter. The keys and values must all be - strings; each key will be replaced with its corresponding value. - """ - data = escape(data, entities) - if '"' in data: - if "'" in data: - data = '"%s"' % data.replace('"', """) - else: - data = "'%s'" % data - else: - data = '"%s"' % data - return data - - -class XMLGenerator(handler.ContentHandler): - - def __init__(self, out=None, encoding="iso-8859-1"): - if out is None: - import sys - out = sys.stdout - handler.ContentHandler.__init__(self) - self._out = out - self._ns_contexts = [{}] # contains uri -> prefix dicts - self._current_context = self._ns_contexts[-1] - self._undeclared_ns_maps = [] - self._encoding = encoding - - def _write(self, text): - if isinstance(text, str): - self._out.write(text) - else: - self._out.write(text.encode(self._encoding, _error_handling)) - - # ContentHandler methods - - def startDocument(self): - self._write('\n' % - self._encoding) - - def startPrefixMapping(self, prefix, uri): - self._ns_contexts.append(self._current_context.copy()) - self._current_context[uri] = prefix - self._undeclared_ns_maps.append((prefix, uri)) - - def endPrefixMapping(self, prefix): - self._current_context = self._ns_contexts[-1] - del self._ns_contexts[-1] - - def startElement(self, name, attrs): - self._write('<' + name) - for (name, value) in attrs.items(): - self._write(' %s=%s' % (name, quoteattr(value))) - self._write('>') - - def endElement(self, name): - self._write('' % name) - - def startElementNS(self, name, qname, attrs): - if name[0] is None: - # if the name was not namespace-scoped, use the unqualified part - name = name[1] - else: - # else try to restore the original prefix from the namespace - name = self._current_context[name[0]] + ":" + name[1] - self._write('<' + name) - - for pair in self._undeclared_ns_maps: - self._write(' xmlns:%s="%s"' % pair) - self._undeclared_ns_maps = [] - - for (name, value) in attrs.items(): - name = self._current_context[name[0]] + ":" + name[1] - self._write(' %s=%s' % (name, quoteattr(value))) - self._write('>') - - def endElementNS(self, name, qname): - if name[0] is None: - name = name[1] - else: - name = self._current_context[name[0]] + ":" + name[1] - self._write('' % name) - - def characters(self, content): - self._write(escape(content)) - - def ignorableWhitespace(self, content): - self._write(content) - - def processingInstruction(self, target, data): - self._write('' % (target, data)) - - -class XMLFilterBase(xmlreader.XMLReader): - """This class is designed to sit between an XMLReader and the - client application's event handlers. By default, it does nothing - but pass requests up to the reader and events on to the handlers - unmodified, but subclasses can override specific methods to modify - the event stream or the configuration requests as they pass - through.""" - - def __init__(self, parent = None): - xmlreader.XMLReader.__init__(self) - self._parent = parent - - # ErrorHandler methods - - def error(self, exception): - self._err_handler.error(exception) - - def fatalError(self, exception): - self._err_handler.fatalError(exception) - - def warning(self, exception): - self._err_handler.warning(exception) - - # ContentHandler methods - - def setDocumentLocator(self, locator): - self._cont_handler.setDocumentLocator(locator) - - def startDocument(self): - self._cont_handler.startDocument() - - def endDocument(self): - self._cont_handler.endDocument() - - def startPrefixMapping(self, prefix, uri): - self._cont_handler.startPrefixMapping(prefix, uri) - - def endPrefixMapping(self, prefix): - self._cont_handler.endPrefixMapping(prefix) - - def startElement(self, name, attrs): - self._cont_handler.startElement(name, attrs) - - def endElement(self, name): - self._cont_handler.endElement(name) - - def startElementNS(self, name, qname, attrs): - self._cont_handler.startElementNS(name, qname, attrs) - - def endElementNS(self, name, qname): - self._cont_handler.endElementNS(name, qname) - - def characters(self, content): - self._cont_handler.characters(content) - - def ignorableWhitespace(self, chars): - self._cont_handler.ignorableWhitespace(chars) - - def processingInstruction(self, target, data): - self._cont_handler.processingInstruction(target, data) - - def skippedEntity(self, name): - self._cont_handler.skippedEntity(name) - - # DTDHandler methods - - def notationDecl(self, name, publicId, systemId): - self._dtd_handler.notationDecl(name, publicId, systemId) - - def unparsedEntityDecl(self, name, publicId, systemId, ndata): - self._dtd_handler.unparsedEntityDecl(name, publicId, systemId, ndata) - - # EntityResolver methods - - def resolveEntity(self, publicId, systemId): - return self._ent_handler.resolveEntity(publicId, systemId) - - # XMLReader methods - - def parse(self, source): - self._parent.setContentHandler(self) - self._parent.setErrorHandler(self) - self._parent.setEntityResolver(self) - self._parent.setDTDHandler(self) - self._parent.parse(source) - - def setLocale(self, locale): - self._parent.setLocale(locale) - - def getFeature(self, name): - return self._parent.getFeature(name) - - def setFeature(self, name, state): - self._parent.setFeature(name, state) - - def getProperty(self, name): - return self._parent.getProperty(name) - - def setProperty(self, name, value): - self._parent.setProperty(name, value) - - # XMLFilter methods - - def getParent(self): - return self._parent - - def setParent(self, parent): - self._parent = parent - -# --- Utility functions - -def prepare_input_source(source, base = ""): - """This function takes an InputSource and an optional base URL and - returns a fully resolved InputSource object ready for reading.""" - - if type(source) in _StringTypes: - source = xmlreader.InputSource(source) - elif hasattr(source, "read"): - f = source - source = xmlreader.InputSource() - source.setByteStream(f) - if hasattr(f, "name"): - source.setSystemId(f.name) - - if source.getByteStream() is None: - sysid = source.getSystemId() - basehead = os.path.dirname(os.path.normpath(base)) - sysidfilename = os.path.join(basehead, sysid) - if os.path.isfile(sysidfilename): - source.setSystemId(sysidfilename) - f = open(sysidfilename, "rb") - else: - source.setSystemId(urlparse.urljoin(base, sysid)) - f = urllib.urlopen(source.getSystemId()) - - source.setByteStream(f) - - return source diff --git a/Lib/xmlcore/sax/xmlreader.py b/Lib/xmlcore/sax/xmlreader.py deleted file mode 100644 index 1cade65..0000000 --- a/Lib/xmlcore/sax/xmlreader.py +++ /dev/null @@ -1,381 +0,0 @@ -"""An XML Reader is the SAX 2 name for an XML parser. XML Parsers -should be based on this code. """ - -from . import handler - -from ._exceptions import SAXNotSupportedException, SAXNotRecognizedException - - -# ===== XMLREADER ===== - -class XMLReader: - """Interface for reading an XML document using callbacks. - - XMLReader is the interface that an XML parser's SAX2 driver must - implement. This interface allows an application to set and query - features and properties in the parser, to register event handlers - for document processing, and to initiate a document parse. - - All SAX interfaces are assumed to be synchronous: the parse - methods must not return until parsing is complete, and readers - must wait for an event-handler callback to return before reporting - the next event.""" - - def __init__(self): - self._cont_handler = handler.ContentHandler() - self._dtd_handler = handler.DTDHandler() - self._ent_handler = handler.EntityResolver() - self._err_handler = handler.ErrorHandler() - - def parse(self, source): - "Parse an XML document from a system identifier or an InputSource." - raise NotImplementedError("This method must be implemented!") - - def getContentHandler(self): - "Returns the current ContentHandler." - return self._cont_handler - - def setContentHandler(self, handler): - "Registers a new object to receive document content events." - self._cont_handler = handler - - def getDTDHandler(self): - "Returns the current DTD handler." - return self._dtd_handler - - def setDTDHandler(self, handler): - "Register an object to receive basic DTD-related events." - self._dtd_handler = handler - - def getEntityResolver(self): - "Returns the current EntityResolver." - return self._ent_handler - - def setEntityResolver(self, resolver): - "Register an object to resolve external entities." - self._ent_handler = resolver - - def getErrorHandler(self): - "Returns the current ErrorHandler." - return self._err_handler - - def setErrorHandler(self, handler): - "Register an object to receive error-message events." - self._err_handler = handler - - def setLocale(self, locale): - """Allow an application to set the locale for errors and warnings. - - SAX parsers are not required to provide localization for errors - and warnings; if they cannot support the requested locale, - however, they must throw a SAX exception. Applications may - request a locale change in the middle of a parse.""" - raise SAXNotSupportedException("Locale support not implemented") - - def getFeature(self, name): - "Looks up and returns the state of a SAX2 feature." - raise SAXNotRecognizedException("Feature '%s' not recognized" % name) - - def setFeature(self, name, state): - "Sets the state of a SAX2 feature." - raise SAXNotRecognizedException("Feature '%s' not recognized" % name) - - def getProperty(self, name): - "Looks up and returns the value of a SAX2 property." - raise SAXNotRecognizedException("Property '%s' not recognized" % name) - - def setProperty(self, name, value): - "Sets the value of a SAX2 property." - raise SAXNotRecognizedException("Property '%s' not recognized" % name) - -class IncrementalParser(XMLReader): - """This interface adds three extra methods to the XMLReader - interface that allow XML parsers to support incremental - parsing. Support for this interface is optional, since not all - underlying XML parsers support this functionality. - - When the parser is instantiated it is ready to begin accepting - data from the feed method immediately. After parsing has been - finished with a call to close the reset method must be called to - make the parser ready to accept new data, either from feed or - using the parse method. - - Note that these methods must _not_ be called during parsing, that - is, after parse has been called and before it returns. - - By default, the class also implements the parse method of the XMLReader - interface using the feed, close and reset methods of the - IncrementalParser interface as a convenience to SAX 2.0 driver - writers.""" - - def __init__(self, bufsize=2**16): - self._bufsize = bufsize - XMLReader.__init__(self) - - def parse(self, source): - from . import saxutils - source = saxutils.prepare_input_source(source) - - self.prepareParser(source) - file = source.getByteStream() - buffer = file.read(self._bufsize) - while buffer != "": - self.feed(buffer) - buffer = file.read(self._bufsize) - self.close() - - def feed(self, data): - """This method gives the raw XML data in the data parameter to - the parser and makes it parse the data, emitting the - corresponding events. It is allowed for XML constructs to be - split across several calls to feed. - - feed may raise SAXException.""" - raise NotImplementedError("This method must be implemented!") - - def prepareParser(self, source): - """This method is called by the parse implementation to allow - the SAX 2.0 driver to prepare itself for parsing.""" - raise NotImplementedError("prepareParser must be overridden!") - - def close(self): - """This method is called when the entire XML document has been - passed to the parser through the feed method, to notify the - parser that there are no more data. This allows the parser to - do the final checks on the document and empty the internal - data buffer. - - The parser will not be ready to parse another document until - the reset method has been called. - - close may raise SAXException.""" - raise NotImplementedError("This method must be implemented!") - - def reset(self): - """This method is called after close has been called to reset - the parser so that it is ready to parse new documents. The - results of calling parse or feed after close without calling - reset are undefined.""" - raise NotImplementedError("This method must be implemented!") - -# ===== LOCATOR ===== - -class Locator: - """Interface for associating a SAX event with a document - location. A locator object will return valid results only during - calls to DocumentHandler methods; at any other time, the - results are unpredictable.""" - - def getColumnNumber(self): - "Return the column number where the current event ends." - return -1 - - def getLineNumber(self): - "Return the line number where the current event ends." - return -1 - - def getPublicId(self): - "Return the public identifier for the current event." - return None - - def getSystemId(self): - "Return the system identifier for the current event." - return None - -# ===== INPUTSOURCE ===== - -class InputSource: - """Encapsulation of the information needed by the XMLReader to - read entities. - - This class may include information about the public identifier, - system identifier, byte stream (possibly with character encoding - information) and/or the character stream of an entity. - - Applications will create objects of this class for use in the - XMLReader.parse method and for returning from - EntityResolver.resolveEntity. - - An InputSource belongs to the application, the XMLReader is not - allowed to modify InputSource objects passed to it from the - application, although it may make copies and modify those.""" - - def __init__(self, system_id = None): - self.__system_id = system_id - self.__public_id = None - self.__encoding = None - self.__bytefile = None - self.__charfile = None - - def setPublicId(self, public_id): - "Sets the public identifier of this InputSource." - self.__public_id = public_id - - def getPublicId(self): - "Returns the public identifier of this InputSource." - return self.__public_id - - def setSystemId(self, system_id): - "Sets the system identifier of this InputSource." - self.__system_id = system_id - - def getSystemId(self): - "Returns the system identifier of this InputSource." - return self.__system_id - - def setEncoding(self, encoding): - """Sets the character encoding of this InputSource. - - The encoding must be a string acceptable for an XML encoding - declaration (see section 4.3.3 of the XML recommendation). - - The encoding attribute of the InputSource is ignored if the - InputSource also contains a character stream.""" - self.__encoding = encoding - - def getEncoding(self): - "Get the character encoding of this InputSource." - return self.__encoding - - def setByteStream(self, bytefile): - """Set the byte stream (a Python file-like object which does - not perform byte-to-character conversion) for this input - source. - - The SAX parser will ignore this if there is also a character - stream specified, but it will use a byte stream in preference - to opening a URI connection itself. - - If the application knows the character encoding of the byte - stream, it should set it with the setEncoding method.""" - self.__bytefile = bytefile - - def getByteStream(self): - """Get the byte stream for this input source. - - The getEncoding method will return the character encoding for - this byte stream, or None if unknown.""" - return self.__bytefile - - def setCharacterStream(self, charfile): - """Set the character stream for this input source. (The stream - must be a Python 2.0 Unicode-wrapped file-like that performs - conversion to Unicode strings.) - - If there is a character stream specified, the SAX parser will - ignore any byte stream and will not attempt to open a URI - connection to the system identifier.""" - self.__charfile = charfile - - def getCharacterStream(self): - "Get the character stream for this input source." - return self.__charfile - -# ===== ATTRIBUTESIMPL ===== - -class AttributesImpl: - - def __init__(self, attrs): - """Non-NS-aware implementation. - - attrs should be of the form {name : value}.""" - self._attrs = attrs - - def getLength(self): - return len(self._attrs) - - def getType(self, name): - return "CDATA" - - def getValue(self, name): - return self._attrs[name] - - def getValueByQName(self, name): - return self._attrs[name] - - def getNameByQName(self, name): - if not self._attrs.has_key(name): - raise KeyError, name - return name - - def getQNameByName(self, name): - if not self._attrs.has_key(name): - raise KeyError, name - return name - - def getNames(self): - return self._attrs.keys() - - def getQNames(self): - return self._attrs.keys() - - def __len__(self): - return len(self._attrs) - - def __getitem__(self, name): - return self._attrs[name] - - def keys(self): - return self._attrs.keys() - - def has_key(self, name): - return self._attrs.has_key(name) - - def __contains__(self, name): - return self._attrs.has_key(name) - - def get(self, name, alternative=None): - return self._attrs.get(name, alternative) - - def copy(self): - return self.__class__(self._attrs) - - def items(self): - return self._attrs.items() - - def values(self): - return self._attrs.values() - -# ===== ATTRIBUTESNSIMPL ===== - -class AttributesNSImpl(AttributesImpl): - - def __init__(self, attrs, qnames): - """NS-aware implementation. - - attrs should be of the form {(ns_uri, lname): value, ...}. - qnames of the form {(ns_uri, lname): qname, ...}.""" - self._attrs = attrs - self._qnames = qnames - - def getValueByQName(self, name): - for (nsname, qname) in self._qnames.items(): - if qname == name: - return self._attrs[nsname] - - raise KeyError, name - - def getNameByQName(self, name): - for (nsname, qname) in self._qnames.items(): - if qname == name: - return nsname - - raise KeyError, name - - def getQNameByName(self, name): - return self._qnames[name] - - def getQNames(self): - return self._qnames.values() - - def copy(self): - return self.__class__(self._attrs, self._qnames) - - -def _test(): - XMLReader() - IncrementalParser() - Locator() - -if __name__ == "__main__": - _test() diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 168d245..5c3fff3 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -1,7 +1,8 @@ -"Read and write ZIP files." - +""" +Read and write ZIP files. +""" import struct, os, time, sys -import binascii +import binascii, cStringIO try: import zlib # We may need its compression method @@ -9,12 +10,22 @@ except ImportError: zlib = None __all__ = ["BadZipfile", "error", "ZIP_STORED", "ZIP_DEFLATED", "is_zipfile", - "ZipInfo", "ZipFile", "PyZipFile"] + "ZipInfo", "ZipFile", "PyZipFile", "LargeZipFile" ] class BadZipfile(Exception): pass + + +class LargeZipFile(Exception): + """ + Raised when writing a zipfile, the zipfile requires ZIP64 extensions + and those extensions are disabled. + """ + error = BadZipfile # The exception raised by this module +ZIP64_LIMIT= (1 << 31) - 1 + # constants for Zip file compression methods ZIP_STORED = 0 ZIP_DEFLATED = 8 @@ -27,6 +38,11 @@ structCentralDir = "<4s4B4HlLL5HLl"# 19 items, central directory, 46 bytes stringCentralDir = "PK\001\002" # magic number for central directory structFileHeader = "<4s2B4HlLL2H" # 12 items, file header record, 30 bytes stringFileHeader = "PK\003\004" # magic number for file header +structEndArchive64Locator = "<4slql" # 4 items, locate Zip64 header, 20 bytes +stringEndArchive64Locator = "PK\x06\x07" # magic token for locator header +structEndArchive64 = "<4sqhhllqqqq" # 10 items, end of archive (Zip64), 56 bytes +stringEndArchive64 = "PK\x06\x06" # magic token for Zip64 header + # indexes of entries in the central directory structure _CD_SIGNATURE = 0 @@ -75,6 +91,40 @@ def is_zipfile(filename): pass return False +def _EndRecData64(fpin, offset, endrec): + """ + Read the ZIP64 end-of-archive records and use that to update endrec + """ + locatorSize = struct.calcsize(structEndArchive64Locator) + fpin.seek(offset - locatorSize, 2) + data = fpin.read(locatorSize) + sig, diskno, reloff, disks = struct.unpack(structEndArchive64Locator, data) + if sig != stringEndArchive64Locator: + return endrec + + if diskno != 0 or disks != 1: + raise BadZipfile("zipfiles that span multiple disks are not supported") + + # Assume no 'zip64 extensible data' + endArchiveSize = struct.calcsize(structEndArchive64) + fpin.seek(offset - locatorSize - endArchiveSize, 2) + data = fpin.read(endArchiveSize) + sig, sz, create_version, read_version, disk_num, disk_dir, \ + dircount, dircount2, dirsize, diroffset = \ + struct.unpack(structEndArchive64, data) + if sig != stringEndArchive64: + return endrec + + # Update the original endrec using data from the ZIP64 record + endrec[1] = disk_num + endrec[2] = disk_dir + endrec[3] = dircount + endrec[4] = dircount2 + endrec[5] = dirsize + endrec[6] = diroffset + return endrec + + def _EndRecData(fpin): """Return data from the "End of Central Directory" record, or None. @@ -88,6 +138,8 @@ def _EndRecData(fpin): endrec = list(endrec) endrec.append("") # Append the archive comment endrec.append(filesize - 22) # Append the record start offset + if endrec[-4] == -1 or endrec[-4] == 0xffffffff: + return _EndRecData64(fpin, -22, endrec) return endrec # Search the last END_BLOCK bytes of the file for the record signature. # The comment is appended to the ZIP file and has a 16 bit length. @@ -106,25 +158,50 @@ def _EndRecData(fpin): # Append the archive comment and start offset endrec.append(comment) endrec.append(filesize - END_BLOCK + start) + if endrec[-4] == -1 or endrec[-4] == 0xffffffff: + return _EndRecData64(fpin, - END_BLOCK + start, endrec) return endrec return # Error, return None -class ZipInfo: +class ZipInfo (object): """Class with attributes describing each file in the ZIP archive.""" + __slots__ = ( + 'orig_filename', + 'filename', + 'date_time', + 'compress_type', + 'comment', + 'extra', + 'create_system', + 'create_version', + 'extract_version', + 'reserved', + 'flag_bits', + 'volume', + 'internal_attr', + 'external_attr', + 'header_offset', + 'CRC', + 'compress_size', + 'file_size', + ) + def __init__(self, filename="NoName", date_time=(1980,1,1,0,0,0)): self.orig_filename = filename # Original file name in archive -# Terminate the file name at the first null byte. Null bytes in file -# names are used as tricks by viruses in archives. + + # Terminate the file name at the first null byte. Null bytes in file + # names are used as tricks by viruses in archives. null_byte = filename.find(chr(0)) if null_byte >= 0: filename = filename[0:null_byte] -# This is used to ensure paths in generated ZIP files always use -# forward slashes as the directory separator, as required by the -# ZIP format specification. - if os.sep != "/": + # This is used to ensure paths in generated ZIP files always use + # forward slashes as the directory separator, as required by the + # ZIP format specification. + if os.sep != "/" and os.sep in filename: filename = filename.replace(os.sep, "/") + self.filename = filename # Normalized file name self.date_time = date_time # year, month, day, hour, min, sec # Standard values: @@ -145,7 +222,6 @@ class ZipInfo: self.external_attr = 0 # External file attributes # Other attributes are set by class ZipFile: # header_offset Byte offset to the file header - # file_offset Byte offset to the start of the file data # CRC CRC-32 of the uncompressed file # compress_size Size of the compressed file # file_size Size of the uncompressed file @@ -162,29 +238,85 @@ class ZipInfo: CRC = self.CRC compress_size = self.compress_size file_size = self.file_size + + extra = self.extra + + if file_size > ZIP64_LIMIT or compress_size > ZIP64_LIMIT: + # File is larger than what fits into a 4 byte integer, + # fall back to the ZIP64 extension + fmt = '= 24: + counts = unpack(' ZIP64_LIMIT: + x = endrec[9] - size_cd - 56 - 20 + else: + x = endrec[9] - size_cd # "concat" is zero, unless zip was concatenated to another file concat = x - offset_cd if self.debug > 2: @@ -258,6 +393,8 @@ class ZipFile: # self.start_dir: Position of start of central directory self.start_dir = offset_cd + concat fp.seek(self.start_dir, 0) + data = fp.read(size_cd) + fp = cStringIO.StringIO(data) total = 0 while total < size_cd: centdir = fp.read(46) @@ -275,8 +412,7 @@ class ZipFile: total = (total + centdir[_CD_FILENAME_LENGTH] + centdir[_CD_EXTRA_FIELD_LENGTH] + centdir[_CD_COMMENT_LENGTH]) - x.header_offset = centdir[_CD_LOCAL_HEADER_OFFSET] + concat - # file_offset must be computed below... + x.header_offset = centdir[_CD_LOCAL_HEADER_OFFSET] (x.create_version, x.create_system, x.extract_version, x.reserved, x.flag_bits, x.compress_type, t, d, x.CRC, x.compress_size, x.file_size) = centdir[1:12] @@ -284,28 +420,14 @@ class ZipFile: # Convert date/time code to (year, month, day, hour, min, sec) x.date_time = ( (d>>9)+1980, (d>>5)&0xF, d&0x1F, t>>11, (t>>5)&0x3F, (t&0x1F) * 2 ) + + x._decodeExtra() + x.header_offset = x.header_offset + concat self.filelist.append(x) self.NameToInfo[x.filename] = x if self.debug > 2: print "total", total - for data in self.filelist: - fp.seek(data.header_offset, 0) - fheader = fp.read(30) - if fheader[0:4] != stringFileHeader: - raise BadZipfile, "Bad magic number for file header" - fheader = struct.unpack(structFileHeader, fheader) - # file_offset is computed here, since the extra field for - # the central directory and for the local file header - # refer to different fields, and they can have different - # lengths - data.file_offset = (data.header_offset + 30 - + fheader[_FH_FILENAME_LENGTH] - + fheader[_FH_EXTRA_FIELD_LENGTH]) - fname = fp.read(fheader[_FH_FILENAME_LENGTH]) - if fname != data.orig_filename: - raise RuntimeError, \ - 'File name in directory "%s" and header "%s" differ.' % ( - data.orig_filename, fname) + def namelist(self): """Return a list of file names in the archive.""" @@ -334,6 +456,7 @@ class ZipFile: except BadZipfile: return zinfo.filename + def getinfo(self, name): """Return the instance of ZipInfo given 'name'.""" return self.NameToInfo[name] @@ -347,7 +470,24 @@ class ZipFile: "Attempt to read ZIP archive that was already closed" zinfo = self.getinfo(name) filepos = self.fp.tell() - self.fp.seek(zinfo.file_offset, 0) + + self.fp.seek(zinfo.header_offset, 0) + + # Skip the file header: + fheader = self.fp.read(30) + if fheader[0:4] != stringFileHeader: + raise BadZipfile, "Bad magic number for file header" + + fheader = struct.unpack(structFileHeader, fheader) + fname = self.fp.read(fheader[_FH_FILENAME_LENGTH]) + if fheader[_FH_EXTRA_FIELD_LENGTH]: + self.fp.read(fheader[_FH_EXTRA_FIELD_LENGTH]) + + if fname != zinfo.orig_filename: + raise BadZipfile, \ + 'File name in directory "%s" and header "%s" differ.' % ( + zinfo.orig_filename, fname) + bytes = self.fp.read(zinfo.compress_size) self.fp.seek(filepos, 0) if zinfo.compress_type == ZIP_STORED: @@ -388,6 +528,12 @@ class ZipFile: if zinfo.compress_type not in (ZIP_STORED, ZIP_DEFLATED): raise RuntimeError, \ "That compression method is not supported" + if zinfo.file_size > ZIP64_LIMIT: + if not self._allowZip64: + raise LargeZipFile("Filesize would require ZIP64 extensions") + if zinfo.header_offset > ZIP64_LIMIT: + if not self._allowZip64: + raise LargeZipFile("Zipfile size would require ZIP64 extensions") def write(self, filename, arcname=None, compress_type=None): """Put the bytes from filename into the archive under the name @@ -407,16 +553,19 @@ class ZipFile: zinfo.compress_type = self.compression else: zinfo.compress_type = compress_type - self._writecheck(zinfo) - fp = open(filename, "rb") + + zinfo.file_size = st.st_size zinfo.flag_bits = 0x00 zinfo.header_offset = self.fp.tell() # Start of header bytes + + self._writecheck(zinfo) + self._didModify = True + fp = open(filename, "rb") # Must overwrite CRC and sizes with correct data later zinfo.CRC = CRC = 0 zinfo.compress_size = compress_size = 0 zinfo.file_size = file_size = 0 self.fp.write(zinfo.FileHeader()) - zinfo.file_offset = self.fp.tell() # Start of file bytes if zinfo.compress_type == ZIP_DEFLATED: cmpr = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION, zlib.DEFLATED, -15) @@ -461,8 +610,10 @@ class ZipFile: zinfo.compress_type = self.compression else: zinfo = zinfo_or_arcname - self._writecheck(zinfo) zinfo.file_size = len(bytes) # Uncompressed size + zinfo.header_offset = self.fp.tell() # Start of header bytes + self._writecheck(zinfo) + self._didModify = True zinfo.CRC = binascii.crc32(bytes) # CRC-32 checksum if zinfo.compress_type == ZIP_DEFLATED: co = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION, @@ -473,8 +624,8 @@ class ZipFile: zinfo.compress_size = zinfo.file_size zinfo.header_offset = self.fp.tell() # Start of header bytes self.fp.write(zinfo.FileHeader()) - zinfo.file_offset = self.fp.tell() # Start of file bytes self.fp.write(bytes) + self.fp.flush() if zinfo.flag_bits & 0x08: # Write CRC and file sizes after the file data self.fp.write(struct.pack(" ZIP64_LIMIT \ + or zinfo.compress_size > ZIP64_LIMIT: + extra.append(zinfo.file_size) + extra.append(zinfo.compress_size) + file_size = 0xffffffff #-1 + compress_size = 0xffffffff #-1 + else: + file_size = zinfo.file_size + compress_size = zinfo.compress_size + + if zinfo.header_offset > ZIP64_LIMIT: + extra.append(zinfo.header_offset) + header_offset = -1 # struct "l" format: 32 one bits + else: + header_offset = zinfo.header_offset + + extra_data = zinfo.extra + if extra: + # Append a ZIP64 field to the extra's + extra_data = struct.pack( + ' ZIP64_LIMIT: + # Need to write the ZIP64 end-of-archive records + zip64endrec = struct.pack( + structEndArchive64, stringEndArchive64, + 44, 45, 45, 0, 0, count, count, pos2 - pos1, pos1) + self.fp.write(zip64endrec) + + zip64locrec = struct.pack( + structEndArchive64Locator, + stringEndArchive64Locator, 0, pos2, 1) + self.fp.write(zip64locrec) + + # XXX Why is `pos3` computed next? It's never referenced. + pos3 = self.fp.tell() + endrec = struct.pack(structEndArchive, stringEndArchive, + 0, 0, count, count, pos2 - pos1, -1, 0) + self.fp.write(endrec) + + else: + endrec = struct.pack(structEndArchive, stringEndArchive, + 0, 0, count, count, pos2 - pos1, pos1, 0) + self.fp.write(endrec) self.fp.flush() if not self._filePassed: self.fp.close() @@ -619,3 +821,80 @@ class PyZipFile(ZipFile): if basename: archivename = "%s/%s" % (basename, archivename) return (fname, archivename) + + +def main(args = None): + import textwrap + USAGE=textwrap.dedent("""\ + Usage: + zipfile.py -l zipfile.zip # Show listing of a zipfile + zipfile.py -t zipfile.zip # Test if a zipfile is valid + zipfile.py -e zipfile.zip target # Extract zipfile into target dir + zipfile.py -c zipfile.zip src ... # Create zipfile from sources + """) + if args is None: + args = sys.argv[1:] + + if not args or args[0] not in ('-l', '-c', '-e', '-t'): + print USAGE + sys.exit(1) + + if args[0] == '-l': + if len(args) != 2: + print USAGE + sys.exit(1) + zf = ZipFile(args[1], 'r') + zf.printdir() + zf.close() + + elif args[0] == '-t': + if len(args) != 2: + print USAGE + sys.exit(1) + zf = ZipFile(args[1], 'r') + zf.testzip() + print "Done testing" + + elif args[0] == '-e': + if len(args) != 3: + print USAGE + sys.exit(1) + + zf = ZipFile(args[1], 'r') + out = args[2] + for path in zf.namelist(): + if path.startswith('./'): + tgt = os.path.join(out, path[2:]) + else: + tgt = os.path.join(out, path) + + tgtdir = os.path.dirname(tgt) + if not os.path.exists(tgtdir): + os.makedirs(tgtdir) + fp = open(tgt, 'wb') + fp.write(zf.read(path)) + fp.close() + zf.close() + + elif args[0] == '-c': + if len(args) < 3: + print USAGE + sys.exit(1) + + def addToZip(zf, path, zippath): + if os.path.isfile(path): + zf.write(path, zippath, ZIP_DEFLATED) + elif os.path.isdir(path): + for nm in os.listdir(path): + addToZip(zf, + os.path.join(path, nm), os.path.join(zippath, nm)) + # else: ignore + + zf = ZipFile(args[1], 'w', allowZip64=True) + for src in args[2:]: + addToZip(zf, src, os.path.basename(src)) + + zf.close() + +if __name__ == "__main__": + main() diff --git a/Mac/BuildScript/README.txt b/Mac/BuildScript/README.txt index c556de83..41df0e4 100644 --- a/Mac/BuildScript/README.txt +++ b/Mac/BuildScript/README.txt @@ -8,15 +8,47 @@ $DESTROOT, massages that installation to remove .pyc files and such, creates an Installer package from the installation plus other files in ``resources`` and ``scripts`` and placed that on a ``.dmg`` disk image. -Here are the steps you ned to follow to build a MacPython installer: +Prerequisites +------------- -- Run ``./build-installer.py``. Optionally you can pass a number of arguments - to specify locations of various files. Please see the top of +* A MacOS X 10.4 (or later) + +* XCode 2.2 (or later), with the universal SDK + +* No Fink (in ``/sw``) or DarwinPorts (in ``/opt/local``), those could + interfere with the build. + +* The documentation for the release must be available on python.org + because it is included in the installer. + + +The Recipe +---------- + +Here are the steps you need to follow to build a MacPython installer: + +* Run ``./build-installer.py``. Optionally you can pass a number of arguments + to specify locations of various files. Please see the top of ``build-installer.py`` for its usage. -- When done the script will tell you where the DMG image is. -The script needs to be run on Mac OS X 10.4 with Xcode 2.2 or later and -the 10.4u SDK. + Running this script takes some time, I will not only build Python itself + but also some 3th-party libraries that are needed for extensions. + +* When done the script will tell you where the DMG image is (by default + somewhere in ``/tmp/_py``). + +Testing +------- + +The resulting binaries should work on MacOSX 10.3.9 or later. I usually run +the installer on a 10.3.9, a 10.4.x PPC and a 10.4.x Intel system and then +run the testsuite to make sure. + + +Announcements +------------- + +(This is mostly of historic interest) When all is done, announcements can be posted to at least the following places: diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index 05afe98..083209b 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -67,6 +67,8 @@ DEPSRC=os.path.expanduser('~/Universal/other-sources') SDKPATH="/Developer/SDKs/MacOSX10.4u.sdk" #SDKPATH="/" +ARCHLIST=('i386', 'ppc',) + # Source directory (asume we're in Mac/BuildScript) SRCDIR=os.path.dirname( os.path.dirname( @@ -91,6 +93,26 @@ USAGE=textwrap.dedent("""\ # batteries included python. LIBRARY_RECIPES=[ dict( + name="Bzip2 1.0.3", + url="http://www.bzip.org/1.0.3/bzip2-1.0.3.tar.gz", + configure=None, + install='make install PREFIX=%s/usr/local/ CFLAGS="-arch %s -isysroot %s"'%( + shellQuote(os.path.join(WORKDIR, 'libraries')), + ' -arch '.join(ARCHLIST), + SDKPATH, + ), + ), + dict( + name="ZLib 1.2.3", + url="http://www.gzip.org/zlib/zlib-1.2.3.tar.gz", + configure=None, + install='make install prefix=%s/usr/local/ CFLAGS="-arch %s -isysroot %s"'%( + shellQuote(os.path.join(WORKDIR, 'libraries')), + ' -arch '.join(ARCHLIST), + SDKPATH, + ), + ), + dict( # Note that GNU readline is GPL'd software name="GNU Readline 5.1.4", url="http://ftp.gnu.org/pub/gnu/readline/readline-5.1.tar.gz" , @@ -236,7 +258,7 @@ PKG_RECIPES=[ Mac OS X 10.3 to ensure that you can build new python extensions using that copy of python after installing this version of python. - """ + """, postflight="../Tools/fixapplepython23.py", topdir="/Library/Frameworks/Python.framework", source="/empty-dir", @@ -486,48 +508,49 @@ def buildRecipe(recipe, basedir, archList): runCommand('patch -p%s < %s'%(recipe.get('patchlevel', 1), shellQuote(fn),)) - configure_args = [ - "--prefix=/usr/local", - "--enable-static", - "--disable-shared", - #"CPP=gcc -arch %s -E"%(' -arch '.join(archList,),), - ] - - if 'configure_pre' in recipe: - args = list(recipe['configure_pre']) - if '--disable-static' in args: - configure_args.remove('--enable-static') - if '--enable-shared' in args: - configure_args.remove('--disable-shared') - configure_args.extend(args) - - if recipe.get('useLDFlags', 1): - configure_args.extend([ - "CFLAGS=-arch %s -isysroot %s -I%s/usr/local/include"%( - ' -arch '.join(archList), - shellQuote(SDKPATH)[1:-1], - shellQuote(basedir)[1:-1],), - "LDFLAGS=-syslibroot,%s -L%s/usr/local/lib -arch %s"%( - shellQuote(SDKPATH)[1:-1], - shellQuote(basedir)[1:-1], - ' -arch '.join(archList)), - ]) - else: - configure_args.extend([ - "CFLAGS=-arch %s -isysroot %s -I%s/usr/local/include"%( - ' -arch '.join(archList), + if configure is not None: + configure_args = [ + "--prefix=/usr/local", + "--enable-static", + "--disable-shared", + #"CPP=gcc -arch %s -E"%(' -arch '.join(archList,),), + ] + + if 'configure_pre' in recipe: + args = list(recipe['configure_pre']) + if '--disable-static' in args: + configure_args.remove('--enable-static') + if '--enable-shared' in args: + configure_args.remove('--disable-shared') + configure_args.extend(args) + + if recipe.get('useLDFlags', 1): + configure_args.extend([ + "CFLAGS=-arch %s -isysroot %s -I%s/usr/local/include"%( + ' -arch '.join(archList), + shellQuote(SDKPATH)[1:-1], + shellQuote(basedir)[1:-1],), + "LDFLAGS=-syslibroot,%s -L%s/usr/local/lib -arch %s"%( shellQuote(SDKPATH)[1:-1], - shellQuote(basedir)[1:-1],), - ]) + shellQuote(basedir)[1:-1], + ' -arch '.join(archList)), + ]) + else: + configure_args.extend([ + "CFLAGS=-arch %s -isysroot %s -I%s/usr/local/include"%( + ' -arch '.join(archList), + shellQuote(SDKPATH)[1:-1], + shellQuote(basedir)[1:-1],), + ]) - if 'configure_post' in recipe: - configure_args = configure_args = list(recipe['configure_post']) + if 'configure_post' in recipe: + configure_args = configure_args = list(recipe['configure_post']) - configure_args.insert(0, configure) - configure_args = [ shellQuote(a) for a in configure_args ] + configure_args.insert(0, configure) + configure_args = [ shellQuote(a) for a in configure_args ] - print "Running configure for %s"%(name,) - runCommand(' '.join(configure_args) + ' 2>&1') + print "Running configure for %s"%(name,) + runCommand(' '.join(configure_args) + ' 2>&1') print "Running install for %s"%(name,) runCommand('{ ' + install + ' ;} 2>&1') @@ -550,7 +573,7 @@ def buildLibraries(): os.makedirs(os.path.join(universal, 'usr', 'local', 'include')) for recipe in LIBRARY_RECIPES: - buildRecipe(recipe, universal, ('i386', 'ppc',)) + buildRecipe(recipe, universal, ARCHLIST) @@ -686,6 +709,9 @@ def patchFile(inPath, outPath): data = data.replace('$MACOSX_DEPLOYMENT_TARGET', '10.3 or later') data = data.replace('$ARCHITECTURES', "i386, ppc") data = data.replace('$INSTALL_SIZE', installSize()) + + # This one is not handy as a template variable + data = data.replace('$PYTHONFRAMEWORKINSTALLDIR', '/Library/Frameworks/Python.framework') fp = open(outPath, 'wb') fp.write(data) fp.close() @@ -703,7 +729,10 @@ def patchScript(inPath, outPath): def packageFromRecipe(targetDir, recipe): curdir = os.getcwd() try: - pkgname = recipe['name'] + # The major version (such as 2.5) is included in the pacakge name + # because haveing two version of python installed at the same time is + # common. + pkgname = '%s-%s'%(recipe['name'], getVersion()) srcdir = recipe.get('source') pkgroot = recipe.get('topdir', srcdir) postflight = recipe.get('postflight') @@ -804,7 +833,7 @@ def makeMpkgPlist(path): IFPkgFlagComponentDirectory="Contents/Packages", IFPkgFlagPackageList=[ dict( - IFPkgFlagPackageLocation='%s.pkg'%(item['name']), + IFPkgFlagPackageLocation='%s-%s.pkg'%(item['name'], getVersion()), IFPkgFlagPackageSelection='selected' ) for item in PKG_RECIPES @@ -812,6 +841,7 @@ def makeMpkgPlist(path): IFPkgFormatVersion=0.10000000149011612, IFPkgFlagBackgroundScaling="proportional", IFPkgFlagBackgroundAlignment="left", + IFPkgFlagAuthorizationAction="RootAuthorization", ) writePlist(pl, path) @@ -859,7 +889,7 @@ def buildInstaller(): else: patchFile(os.path.join('resources', fn), os.path.join(rsrcDir, fn)) - shutil.copy("../../../LICENSE", os.path.join(rsrcDir, 'License.txt')) + shutil.copy("../../LICENSE", os.path.join(rsrcDir, 'License.txt')) def installSize(clear=False, _saved=[]): @@ -990,7 +1020,7 @@ def main(): buildPythonDocs() fn = os.path.join(WORKDIR, "_root", "Applications", "MacPython %s"%(getVersion(),), "Update Shell Profile.command") - shutil.copy("scripts/postflight.patch-profile", fn) + patchFile("scripts/postflight.patch-profile", fn) os.chmod(fn, 0755) folder = os.path.join(WORKDIR, "_root", "Applications", "MacPython %s"%( @@ -1005,7 +1035,7 @@ def main(): patchFile('resources/ReadMe.txt', os.path.join(WORKDIR, 'installer', 'ReadMe.txt')) # Ditto for the license file. - shutil.copy('../../../LICENSE', os.path.join(WORKDIR, 'installer', 'License.txt')) + shutil.copy('../../LICENSE', os.path.join(WORKDIR, 'installer', 'License.txt')) fp = open(os.path.join(WORKDIR, 'installer', 'Build.txt'), 'w') print >> fp, "# BUILD INFO" diff --git a/Mac/BuildScript/resources/Welcome.rtf b/Mac/BuildScript/resources/Welcome.rtf index cb65f09..d07e104 100644 --- a/Mac/BuildScript/resources/Welcome.rtf +++ b/Mac/BuildScript/resources/Welcome.rtf @@ -1,4 +1,4 @@ -{\rtf1\mac\ansicpg10000\cocoartf824\cocoasubrtf330 +{\rtf1\mac\ansicpg10000\cocoartf824\cocoasubrtf410 {\fonttbl\f0\fswiss\fcharset77 Helvetica;\f1\fswiss\fcharset77 Helvetica-Bold;} {\colortbl;\red255\green255\blue255;} \paperw11900\paperh16840\margl1440\margr1440\vieww9920\viewh10660\viewkind0 @@ -12,4 +12,7 @@ \ MacPython consists of the Python programming language interpreter, plus a set of programs to allow easy access to it for Mac users (an integrated development environment, an applet builder), plus a set of pre-built extension modules that open up specific Macintosh technologies to Python programs (Carbon, AppleScript, Quicktime, more).\ \ -See the ReadMe file for more information.} \ No newline at end of file +See the ReadMe file for more information.\ +\ +\ +This package will by default update your shell profile to ensure that this version of Python is on the search path of your shell. Please deselect the "Shell profile updater" package on the package customization screen if you want to avoid this modification. } \ No newline at end of file diff --git a/Mac/BuildScript/scripts/postflight.framework b/Mac/BuildScript/scripts/postflight.framework index 532e745..b796a10 100755 --- a/Mac/BuildScript/scripts/postflight.framework +++ b/Mac/BuildScript/scripts/postflight.framework @@ -4,7 +4,7 @@ # PYVER="@PYVER@" -FWK="/Library/Frameworks/Python.framework/Versions/@PYVER@/" +FWK="/Library/Frameworks/Python.framework/Versions/@PYVER@" "${FWK}/bin/python" -Wi -tt \ "${FWK}/lib/python${PYVER}/compileall.py" \ diff --git a/Mac/BuildScript/scripts/postflight.patch-profile b/Mac/BuildScript/scripts/postflight.patch-profile index 48bf701..c42e11e 100755 --- a/Mac/BuildScript/scripts/postflight.patch-profile +++ b/Mac/BuildScript/scripts/postflight.patch-profile @@ -5,14 +5,27 @@ echo "of python is not early enough of the PATH of your shell." echo "These changes will be effective only in shell windows that you open" echo "after running this script." -PYVER=@PYVER@ +PYVER=2.5 PYTHON_ROOT="/Library/Frameworks/Python.framework/Versions/Current" +if [ `id -ur` = 0 ]; then + # Run from the installer, do some trickery to fetch the information + # we need. + theShell="`finger $USER | grep Shell: | head -1 | awk '{ print $NF }'`" + +else + theShell="${SHELL}" +fi + # Make sure the directory ${PYTHON_ROOT}/bin is on the users PATH. -BSH="`basename "${SHELL}"`" +BSH="`basename "${theShell}"`" case "${BSH}" in bash|ksh|sh|*csh) - P="`${SHELL} -c 'echo $PATH'`" + if [ `id -ur` = 0 ]; then + P=`su - ${USER} -c 'echo A-X-4-X@@$PATH@@X-4-X-A' | grep 'A-X-4-X@@.*@@X-4-X-A' | sed -e 's/^A-X-4-X@@//g' -e 's/@@X-4-X-A$//g'` + else + P="`(exec -l ${theShell} -c 'echo $PATH')`" + fi ;; *) echo "Sorry, I don't know how to patch $BSH shells" @@ -42,10 +55,15 @@ case "${BSH}" in echo "# Setting PATH for MacPython ${PYVER}" >> "${HOME}/.cshrc" echo "# The orginal version is saved in .cshrc.pysave" >> "${HOME}/.cshrc" echo "set path=(${PYTHON_ROOT}/bin "'$path'")" >> "${HOME}/.cshrc" + if [ `id -ur` = 0 ]; then + chown "${USER}" "${HOME}/.cshrc" + fi exit 0 ;; bash) - if [ -e "${HOME}/.profile" ]; then + if [ -e "${HOME}/.bash_profile" ]; then + PR="${HOME}/.bash_profile" + elif [ -e "${HOME}/.profile" ]; then PR="${HOME}/.profile" else PR="${HOME}/.bash_profile" @@ -66,6 +84,6 @@ echo "# The orginal version is saved in `basename ${PR}`.pysave" >> "${PR}" echo 'PATH="'"${PYTHON_ROOT}/bin"':${PATH}"' >> "${PR}" echo 'export PATH' >> "${PR}" if [ `id -ur` = 0 ]; then - chown "${LOGNAME}" "${PR}" + chown "${USER}" "${PR}" fi exit 0 diff --git a/Mac/Demo/applescript.html b/Mac/Demo/applescript.html index 014e2ad..5e4fac7 100644 --- a/Mac/Demo/applescript.html +++ b/Mac/Demo/applescript.html @@ -4,6 +4,10 @@

Using the Open Scripting Architecture from Python

+

NOTE: this document describes the OSA support that is shipped with +the core python distribution. Most users are better of with the more +userfriendly appscript library. +

OSA support in Python is still not 100% complete, but there is already enough in place to allow you to do some nifty things with other programs from your python program.

@@ -355,4 +359,4 @@ man pages for more details. - \ No newline at end of file + diff --git a/Mac/Demo/calldll/readme b/Mac/Demo/calldll/readme deleted file mode 100644 index 1e64dfe..0000000 --- a/Mac/Demo/calldll/readme +++ /dev/null @@ -1,48 +0,0 @@ -Preliminary notes/documentation for the calldll module, version 0.2. -==================================================================== - -Calldll allows you to call random C functions from python without writing any -C code. It is mainly meant to call MacOS toolbox routines for which no Python -wrapper module is available. It is also incomplete, in that only a few argument -types are currently supported. Please let me know which other argument types -you need, and/or whether you have any ideas on a general "escape" allowing people -to pass anything. - -The module exports three functions: -- symtable = getlibrary(libraryname) - Get a reference to import library libraryname. "InterfaceLib" is the most commonly - used one, containing most toolbox routines. The symbol table can be used - to lookup routines to be passed to newcall: "symtable.WaitNextEvent" will - return the address of routine WaitNextEvent. and so will "symtable['WaitNextEvent']". - The symtable is a mapping, so you can use keys() and len(...) to inspect it. -- symtable = getdiskfragment(file) - Load the specified file (given by fsspec or filename) and return a reference to - its symboltable. -- callable = newcall(routine, returntype, [argtype, ...]) - Return a callable object. You specify the C routine to be called (as explained above), - the type of the return value and the argument types. The resulting object can - be called from Python code in the normal way, and typechecking on arguments is - performed (but, of course, if you specify incorrect argument types in this call - you may well crash your machine). Printing a callable will give you a description - of the (C-) calling sequence. - -The C return value can be one of 'None', 'Byte', 'Short', 'Long', 'Pstring' (a pascal -string returned by address, copied to a Python string), 'Cobject' (a wrapper around a void -pointer), 'Handle' (a new handle, returned as a Res.Resource object) or 'OSErr' (which raises -MacOS.Error if non-zero). - -Arguments can be any of 'InByte', 'InShort', 'InLong', 'InString' (a python string, with the -address of the data passed to the C routine, so be careful!), 'InPstring' (a python string copied -to a Str255 and passed by address), 'InCobject', 'InHandle', 'OutByte' (storage is allocated for -a single byte, the address passed to C and the resulting value returned to Python), 'OutShort', -'OutLong', 'OutPstring' (again: storage pre-allocated and the address passed to C), 'OutCobject' -(storage for a void * is allocated, this void ** is passed to C and the resulting void * is -encapsulated in the Cobject returned) or 'OutHandle' (ditto, which means that this is usually *not* -what you use, you normally use 'InHandle' because most toolbox calls expect you to preallocate -the handle). - -All values to be returned (from the return value and the Out arguments) are collected. If there -aren't any None is returned, if there is one value this value is returned, if there are multiple -values a tuple is returned. - -There is test code in testcalldll.py, and a minimal example in samplecalldll.py. diff --git a/Mac/Demo/calldll/samplecalldll.py b/Mac/Demo/calldll/samplecalldll.py deleted file mode 100644 index 3a5bb5c..0000000 --- a/Mac/Demo/calldll/samplecalldll.py +++ /dev/null @@ -1,24 +0,0 @@ -# -# Test calldll. Tell the user how often menus flash, and let her change it. -# - -import calldll -import sys - -# Obtain a reference to the library with the toolbox calls -interfacelib = calldll.getlibrary('InterfaceLib') - -# Get the routines we need (see LowMem.h for details) -LMGetMenuFlash = calldll.newcall(interfacelib.LMGetMenuFlash, 'Short') -LMSetMenuFlash = calldll.newcall(interfacelib.LMSetMenuFlash, 'None', 'InShort') - -print "Menus currently flash",LMGetMenuFlash(),"times." -print "How often would you like them to flash?", - -# Note: we use input(), so you can try passing non-integer objects -newflash = input() -LMSetMenuFlash(newflash) - -print "Okay, menus now flash", LMGetMenuFlash(),"times." - -sys.exit(1) # So the window stays on-screen diff --git a/Mac/Demo/calldll/testcalldll.py b/Mac/Demo/calldll/testcalldll.py deleted file mode 100644 index e0f6964..0000000 --- a/Mac/Demo/calldll/testcalldll.py +++ /dev/null @@ -1,132 +0,0 @@ -import calldll -import macfs -import sys -import MacOS -from Carbon import Res - -fss, ok = macfs.PromptGetFile("Show me calldll.ppc.slb") - -lib = calldll.getdiskfragment(fss, 'calldll.ppc.slb') - -cdll_b_bbbbbbbb = calldll.newcall(lib.cdll_b_bbbbbbbb, 'Byte', 'InByte', 'InByte', - 'InByte', 'InByte','InByte', 'InByte','InByte', 'InByte') -cdll_h_hhhhhhhh = calldll.newcall(lib.cdll_h_hhhhhhhh, 'Short', 'InShort', 'InShort', - 'InShort', 'InShort','InShort', 'InShort','InShort', 'InShort') -cdll_l_llllllll = calldll.newcall(lib.cdll_l_llllllll, 'Long', 'InLong', 'InLong', - 'InLong', 'InLong','InLong', 'InLong','InLong', 'InLong') - -cdll_N_ssssssss = calldll.newcall(lib.cdll_N_ssssssss, 'None', 'InString', 'InString', - 'InString', 'InString', 'InString', 'InString', 'InString', 'InString') - -cdll_o_l = calldll.newcall(lib.cdll_o_l, 'OSErr', 'InLong') - -cdll_N_pp = calldll.newcall(lib.cdll_N_pp, 'None', 'InPstring', 'OutPstring') - -cdll_N_bb = calldll.newcall(lib.cdll_N_bb, 'None', 'InByte', 'OutByte') -cdll_N_hh = calldll.newcall(lib.cdll_N_hh, 'None', 'InShort', 'OutShort') -cdll_N_ll = calldll.newcall(lib.cdll_N_ll, 'None', 'InLong', 'OutLong') -cdll_N_sH = calldll.newcall(lib.cdll_N_sH, 'None', 'InString', 'InHandle') - -print 'Test cdll_b_bbbbbbbb' -rv = cdll_b_bbbbbbbb(1, 2, 3, 4, 5, 6, 7, 8) -if rv == 36: - print 'ok.' -else: - print 'Failed, returned', rv - -print 'Test cdll_b_bbbbbbbb negative' -rv = cdll_b_bbbbbbbb(-1, -2, -3, -4, -5, -6, -7, -8) -if rv == -36: - print 'ok.' -else: - print 'Failed, returned', rv - -print 'Test cdll_h_hhhhhhhh' -rv = cdll_h_hhhhhhhh(1, 2, 3, 4, 5, 6, 7, 8) -if rv == 36: - print 'ok.' -else: - print 'Failed, returned', rv - -print 'Test cdll_h_hhhhhhhh negative' -rv = cdll_h_hhhhhhhh(-1, -2, -3, -4, -5, -6, -7, -8) -if rv == -36: - print 'ok.' -else: - print 'Failed, returned', rv - -print 'Test cdll_l_llllllll' -rv = cdll_l_llllllll(1, 2, 3, 4, 5, 6, 7, 8) -if rv == 36: - print 'ok.' -else: - print 'Failed, returned', rv - -print 'Test cdll_l_llllllll negative' -rv = cdll_l_llllllll(-1, -2, -3, -4, -5, -6, -7, -8) -if rv == -36: - print 'ok.' -else: - print 'Failed, returned', rv - -print 'Test cdll_N_ssssssss' -print 'Should print one two three four five six seven eight' -rv = cdll_N_ssssssss('one', 'two', 'three', 'four', 'five', 'six', 'seven', 'eight') -if rv == None: - print 'ok.' -else: - print 'Failed, returned', rv - -print 'Test cdll_o_l(0)' -rv = cdll_o_l(0) -if rv == None: - print 'ok.' -else: - print 'Error, returned', rv - -print 'Test cdll_o_l(-100)' -try: - rv = cdll_o_l(-100) - print 'Error, did not raise exception, returned', rv -except MacOS.Error, arg: - if arg[0] == -100: - print 'ok.' - else: - print 'Error, returned incorrect exception arg:', arg[0] - -print 'Test cdll_N_pp' -rv = cdll_N_pp('pascal string') -if rv == 'Was: pascal string': - print 'ok.' -else: - print 'Failed, returned', repr(rv) - -print 'Test cdll_N_bb' -rv = cdll_N_bb(-100) -if rv == -100: - print 'ok.' -else: - print 'Failed, returned', rv - -print 'Test cdll_N_hh' -rv = cdll_N_hh(-100) -if rv == -100: - print 'ok.' -else: - print 'Failed, returned', rv - -print 'Test cdll_N_ll' -rv = cdll_N_ll(-100) -if rv == -100: - print 'ok.' -else: - print 'Failed, returned', rv - -print 'Test cdll_N_sH' -h = Res.Resource('xyz') -rv = cdll_N_sH('new data', h) -if rv == None and h.data == 'new data': - print 'ok.' -else: - print 'Failed, rv is', rv, 'and handle data is', repr(rv.data) -sys.exit(1) diff --git a/Mac/Demo/index.html b/Mac/Demo/index.html index 443cce9..9e01e9f 100644 --- a/Mac/Demo/index.html +++ b/Mac/Demo/index.html @@ -3,15 +3,16 @@

Macintosh Python crash course

-This set of documents provides an introduction to various aspects of +

This set of documents provides an introduction to various aspects of Python programming on the Mac. It is assumed that the reader is already familiar with Python and, to some extent, with MacOS Toolbox programming. Other readers may find something interesting here too, -your mileage may vary.

+your mileage may vary.

-There is a companion document Using Python on the Mac -which you should read before starting here: it explains the basics of using -python on the Macintosh.

+

As the previous paragraph reveals to the careful observer these examples +are dated, most of them were writting before OSX and haven't been updated +afterwards. They still show how to use the Carbon wrappers but aren't +necessarily the best way to use the Carbon API's in OSX.

Another set of Macintosh-savvy examples, more aimed at beginners, is maintained by Joseph Strout, at Python Tidbits in documentation section on the webserver.

-The W widget set by Just van Rossum, which is used by the Python IDE, does not -have complete documentation as of this writing, but Corran Webster has -documented most of it on his -Python Page.

+

The W widget set by Just van Rossum, does not have complete documentation as +of this writing, but Corran Webster has documented most of it on his +Python Page.

There are also some documentation links, as well as other MacPython-related pages, in the diff --git a/Mac/IDLE/config-main.def b/Mac/IDLE/config-main.def index 1cdc0c5..4691a85 100644 --- a/Mac/IDLE/config-main.def +++ b/Mac/IDLE/config-main.def @@ -71,7 +71,7 @@ name= IDLE Classic [Keys] default= 1 -name= IDLE Classic Mac +name= IDLE Classic OSX [History] cyclic=1 diff --git a/Mac/Makefile.in b/Mac/Makefile.in index 82514b2..11f1438 100644 --- a/Mac/Makefile.in +++ b/Mac/Makefile.in @@ -5,7 +5,7 @@ VERSION=@VERSION@ builddir = .. srcdir=@srcdir@ -prefix=/Library/Frameworks/Python.framework/Versions/$(VERSION) +prefix=@prefix@ LIBDEST=$(prefix)/lib/python$(VERSION) RUNSHARED=@RUNSHARED@ BUILDEXE=@BUILDEXEEXT@ diff --git a/Mac/Modules/MacOS.c b/Mac/Modules/MacOS.c new file mode 100644 index 0000000..4eabb39 --- /dev/null +++ b/Mac/Modules/MacOS.c @@ -0,0 +1,644 @@ +/*********************************************************** +Copyright 1991-1997 by Stichting Mathematisch Centrum, Amsterdam, +The Netherlands. + + All Rights Reserved + +Permission to use, copy, modify, and distribute this software and its +documentation for any purpose and without fee is hereby granted, +provided that the above copyright notice appear in all copies and that +both that copyright notice and this permission notice appear in +supporting documentation, and that the names of Stichting Mathematisch +Centrum or CWI not be used in advertising or publicity pertaining to +distribution of the software without specific, written prior permission. + +STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO +THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND +FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE +FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT +OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + +******************************************************************/ + +/* Macintosh OS-specific interface */ + +#include "Python.h" +#include "pymactoolbox.h" + +#include +#include + +static PyObject *MacOS_Error; /* Exception MacOS.Error */ + +#define PATHNAMELEN 1024 + +/* ----------------------------------------------------- */ + +/* Declarations for objects of type Resource fork */ + +typedef struct { + PyObject_HEAD + short fRefNum; + int isclosed; +} rfobject; + +static PyTypeObject Rftype; + + + +/* ---------------------------------------------------------------- */ + +static void +do_close(rfobject *self) +{ + if (self->isclosed ) return; + (void)FSClose(self->fRefNum); + self->isclosed = 1; +} + +static char rf_read__doc__[] = +"Read data from resource fork" +; + +static PyObject * +rf_read(rfobject *self, PyObject *args) +{ + long n; + PyObject *v; + OSErr err; + + if (self->isclosed) { + PyErr_SetString(PyExc_ValueError, "Operation on closed file"); + return NULL; + } + + if (!PyArg_ParseTuple(args, "l", &n)) + return NULL; + + v = PyString_FromStringAndSize((char *)NULL, n); + if (v == NULL) + return NULL; + + err = FSRead(self->fRefNum, &n, PyString_AsString(v)); + if (err && err != eofErr) { + PyMac_Error(err); + Py_DECREF(v); + return NULL; + } + _PyString_Resize(&v, n); + return v; +} + + +static char rf_write__doc__[] = +"Write to resource fork" +; + +static PyObject * +rf_write(rfobject *self, PyObject *args) +{ + char *buffer; + long size; + OSErr err; + + if (self->isclosed) { + PyErr_SetString(PyExc_ValueError, "Operation on closed file"); + return NULL; + } + if (!PyArg_ParseTuple(args, "s#", &buffer, &size)) + return NULL; + err = FSWrite(self->fRefNum, &size, buffer); + if (err) { + PyMac_Error(err); + return NULL; + } + Py_INCREF(Py_None); + return Py_None; +} + + +static char rf_seek__doc__[] = +"Set file position" +; + +static PyObject * +rf_seek(rfobject *self, PyObject *args) +{ + long amount, pos; + int whence = SEEK_SET; + long eof; + OSErr err; + + if (self->isclosed) { + PyErr_SetString(PyExc_ValueError, "Operation on closed file"); + return NULL; + } + if (!PyArg_ParseTuple(args, "l|i", &amount, &whence)) + return NULL; + + if ((err = GetEOF(self->fRefNum, &eof))) + goto ioerr; + + switch (whence) { + case SEEK_CUR: + if ((err = GetFPos(self->fRefNum, &pos))) + goto ioerr; + break; + case SEEK_END: + pos = eof; + break; + case SEEK_SET: + pos = 0; + break; + default: + PyErr_BadArgument(); + return NULL; + } + + pos += amount; + + /* Don't bother implementing seek past EOF */ + if (pos > eof || pos < 0) { + PyErr_BadArgument(); + return NULL; + } + + if ((err = SetFPos(self->fRefNum, fsFromStart, pos)) ) { +ioerr: + PyMac_Error(err); + return NULL; + } + Py_INCREF(Py_None); + return Py_None; +} + + +static char rf_tell__doc__[] = +"Get file position" +; + +static PyObject * +rf_tell(rfobject *self, PyObject *args) +{ + long where; + OSErr err; + + if (self->isclosed) { + PyErr_SetString(PyExc_ValueError, "Operation on closed file"); + return NULL; + } + if (!PyArg_ParseTuple(args, "")) + return NULL; + if ((err = GetFPos(self->fRefNum, &where)) ) { + PyMac_Error(err); + return NULL; + } + return PyInt_FromLong(where); +} + +static char rf_close__doc__[] = +"Close resource fork" +; + +static PyObject * +rf_close(rfobject *self, PyObject *args) +{ + if (!PyArg_ParseTuple(args, "")) + return NULL; + do_close(self); + Py_INCREF(Py_None); + return Py_None; +} + + +static struct PyMethodDef rf_methods[] = { + {"read", (PyCFunction)rf_read, 1, rf_read__doc__}, + {"write", (PyCFunction)rf_write, 1, rf_write__doc__}, + {"seek", (PyCFunction)rf_seek, 1, rf_seek__doc__}, + {"tell", (PyCFunction)rf_tell, 1, rf_tell__doc__}, + {"close", (PyCFunction)rf_close, 1, rf_close__doc__}, + + {NULL, NULL} /* sentinel */ +}; + +/* ---------- */ + + +static rfobject * +newrfobject(void) +{ + rfobject *self; + + self = PyObject_NEW(rfobject, &Rftype); + if (self == NULL) + return NULL; + self->isclosed = 1; + return self; +} + + +static void +rf_dealloc(rfobject *self) +{ + do_close(self); + PyObject_DEL(self); +} + +static PyObject * +rf_getattr(rfobject *self, char *name) +{ + return Py_FindMethod(rf_methods, (PyObject *)self, name); +} + +static char Rftype__doc__[] = +"Resource fork file object" +; + +static PyTypeObject Rftype = { + PyObject_HEAD_INIT(&PyType_Type) + 0, /*ob_size*/ + "MacOS.ResourceFork", /*tp_name*/ + sizeof(rfobject), /*tp_basicsize*/ + 0, /*tp_itemsize*/ + /* methods */ + (destructor)rf_dealloc, /*tp_dealloc*/ + (printfunc)0, /*tp_print*/ + (getattrfunc)rf_getattr, /*tp_getattr*/ + (setattrfunc)0, /*tp_setattr*/ + (cmpfunc)0, /*tp_compare*/ + (reprfunc)0, /*tp_repr*/ + 0, /*tp_as_number*/ + 0, /*tp_as_sequence*/ + 0, /*tp_as_mapping*/ + (hashfunc)0, /*tp_hash*/ + (ternaryfunc)0, /*tp_call*/ + (reprfunc)0, /*tp_str*/ + + /* Space for future expansion */ + 0L,0L,0L,0L, + Rftype__doc__ /* Documentation string */ +}; + +/* End of code for Resource fork objects */ +/* -------------------------------------------------------- */ + +/*----------------------------------------------------------------------*/ +/* Miscellaneous File System Operations */ + +static char getcrtp_doc[] = "Get MacOS 4-char creator and type for a file"; + +static PyObject * +MacOS_GetCreatorAndType(PyObject *self, PyObject *args) +{ + FSSpec fss; + FInfo info; + PyObject *creator, *type, *res; + OSErr err; + + if (!PyArg_ParseTuple(args, "O&", PyMac_GetFSSpec, &fss)) + return NULL; + if ((err = FSpGetFInfo(&fss, &info)) != noErr) + return PyErr_Mac(MacOS_Error, err); + creator = PyString_FromStringAndSize((char *)&info.fdCreator, 4); + type = PyString_FromStringAndSize((char *)&info.fdType, 4); + res = Py_BuildValue("OO", creator, type); + Py_DECREF(creator); + Py_DECREF(type); + return res; +} + +static char setcrtp_doc[] = "Set MacOS 4-char creator and type for a file"; + +static PyObject * +MacOS_SetCreatorAndType(PyObject *self, PyObject *args) +{ + FSSpec fss; + ResType creator, type; + FInfo info; + OSErr err; + + if (!PyArg_ParseTuple(args, "O&O&O&", + PyMac_GetFSSpec, &fss, PyMac_GetOSType, &creator, PyMac_GetOSType, &type)) + return NULL; + if ((err = FSpGetFInfo(&fss, &info)) != noErr) + return PyErr_Mac(MacOS_Error, err); + info.fdCreator = creator; + info.fdType = type; + if ((err = FSpSetFInfo(&fss, &info)) != noErr) + return PyErr_Mac(MacOS_Error, err); + Py_INCREF(Py_None); + return Py_None; +} + + +static char geterr_doc[] = "Convert OSErr number to string"; + +static PyObject * +MacOS_GetErrorString(PyObject *self, PyObject *args) +{ + int err; + char buf[256]; + Handle h; + char *str; + static int errors_loaded; + + if (!PyArg_ParseTuple(args, "i", &err)) + return NULL; + + h = GetResource('Estr', err); + if (!h && !errors_loaded) { + /* + ** Attempt to open the resource file containing the + ** Estr resources. We ignore all errors. We also try + ** this only once. + */ + PyObject *m, *rv; + errors_loaded = 1; + + m = PyImport_ImportModule("macresource"); + if (!m) { + if (Py_VerboseFlag) + PyErr_Print(); + PyErr_Clear(); + } + else { + rv = PyObject_CallMethod(m, "open_error_resource", ""); + if (!rv) { + if (Py_VerboseFlag) + PyErr_Print(); + PyErr_Clear(); + } + else { + Py_DECREF(rv); + /* And try again... */ + h = GetResource('Estr', err); + } + Py_DECREF(m); + } + } + /* + ** Whether the code above succeeded or not, we won't try + ** again. + */ + errors_loaded = 1; + + if (h) { + HLock(h); + str = (char *)*h; + memcpy(buf, str+1, (unsigned char)str[0]); + buf[(unsigned char)str[0]] = '\0'; + HUnlock(h); + ReleaseResource(h); + } + else { + PyOS_snprintf(buf, sizeof(buf), "Mac OS error code %d", err); + } + + return Py_BuildValue("s", buf); +} + +static char splash_doc[] = "Open a splash-screen dialog by resource-id (0=close)"; + +static PyObject * +MacOS_splash(PyObject *self, PyObject *args) +{ + int resid = -1; + static DialogPtr curdialog = NULL; + DialogPtr olddialog; + WindowRef theWindow; + CGrafPtr thePort; +#if 0 + short xpos, ypos, width, height, swidth, sheight; +#endif + + if (!PyArg_ParseTuple(args, "|i", &resid)) + return NULL; + olddialog = curdialog; + curdialog = NULL; + + if ( resid != -1 ) { + curdialog = GetNewDialog(resid, NULL, (WindowPtr)-1); + if ( curdialog ) { + theWindow = GetDialogWindow(curdialog); + thePort = GetWindowPort(theWindow); +#if 0 + width = thePort->portRect.right - thePort->portRect.left; + height = thePort->portRect.bottom - thePort->portRect.top; + swidth = qd.screenBits.bounds.right - qd.screenBits.bounds.left; + sheight = qd.screenBits.bounds.bottom - qd.screenBits.bounds.top - LMGetMBarHeight(); + xpos = (swidth-width)/2; + ypos = (sheight-height)/5 + LMGetMBarHeight(); + MoveWindow(theWindow, xpos, ypos, 0); + ShowWindow(theWindow); +#endif + DrawDialog(curdialog); + } + } + if (olddialog) + DisposeDialog(olddialog); + Py_INCREF(Py_None); + return Py_None; +} + +static char DebugStr_doc[] = "Switch to low-level debugger with a message"; + +static PyObject * +MacOS_DebugStr(PyObject *self, PyObject *args) +{ + Str255 message; + PyObject *object = 0; + + if (!PyArg_ParseTuple(args, "O&|O", PyMac_GetStr255, message, &object)) + return NULL; + DebugStr(message); + Py_INCREF(Py_None); + return Py_None; +} + +static char SysBeep_doc[] = "BEEEEEP!!!"; + +static PyObject * +MacOS_SysBeep(PyObject *self, PyObject *args) +{ + int duration = 6; + + if (!PyArg_ParseTuple(args, "|i", &duration)) + return NULL; + SysBeep(duration); + Py_INCREF(Py_None); + return Py_None; +} + +static char WMAvailable_doc[] = + "True if this process can interact with the display." + "Will foreground the application on the first call as a side-effect." + ; + +static PyObject * +MacOS_WMAvailable(PyObject *self, PyObject *args) +{ + static PyObject *rv = NULL; + + if (!PyArg_ParseTuple(args, "")) + return NULL; + if (!rv) { + ProcessSerialNumber psn; + + /* + ** This is a fairly innocuous call to make if we don't have a window + ** manager, or if we have no permission to talk to it. It will print + ** a message on stderr, but at least it won't abort the process. + ** It appears the function caches the result itself, and it's cheap, so + ** no need for us to cache. + */ +#ifdef kCGNullDirectDisplay + /* On 10.1 CGMainDisplayID() isn't available, and + ** kCGNullDirectDisplay isn't defined. + */ + if (CGMainDisplayID() == 0) { + rv = Py_False; + } else { +#else + { +#endif + if (GetCurrentProcess(&psn) < 0 || + SetFrontProcess(&psn) < 0) { + rv = Py_False; + } else { + rv = Py_True; + } + } + } + Py_INCREF(rv); + return rv; +} + +static char GetTicks_doc[] = "Return number of ticks since bootup"; + +static PyObject * +MacOS_GetTicks(PyObject *self, PyObject *args) +{ + return Py_BuildValue("i", (int)TickCount()); +} + +static char openrf_doc[] = "Open resource fork of a file"; + +static PyObject * +MacOS_openrf(PyObject *self, PyObject *args) +{ + OSErr err; + char *mode = "r"; + FSSpec fss; + SignedByte permission = 1; + rfobject *fp; + + if (!PyArg_ParseTuple(args, "O&|s", PyMac_GetFSSpec, &fss, &mode)) + return NULL; + while (*mode) { + switch (*mode++) { + case '*': break; + case 'r': permission = 1; break; + case 'w': permission = 2; break; + case 'b': break; + default: + PyErr_BadArgument(); + return NULL; + } + } + + if ( (fp = newrfobject()) == NULL ) + return NULL; + + err = HOpenRF(fss.vRefNum, fss.parID, fss.name, permission, &fp->fRefNum); + + if ( err == fnfErr ) { + /* In stead of doing complicated things here to get creator/type + ** correct we let the standard i/o library handle it + */ + FILE *tfp; + char pathname[PATHNAMELEN]; + + if ( (err=PyMac_GetFullPathname(&fss, pathname, PATHNAMELEN)) ) { + PyMac_Error(err); + Py_DECREF(fp); + return NULL; + } + + if ( (tfp = fopen(pathname, "w")) == NULL ) { + PyMac_Error(fnfErr); /* What else... */ + Py_DECREF(fp); + return NULL; + } + fclose(tfp); + err = HOpenRF(fss.vRefNum, fss.parID, fss.name, permission, &fp->fRefNum); + } + if ( err ) { + Py_DECREF(fp); + PyMac_Error(err); + return NULL; + } + fp->isclosed = 0; + return (PyObject *)fp; +} + + +static PyMethodDef MacOS_Methods[] = { + {"GetCreatorAndType", MacOS_GetCreatorAndType, 1, getcrtp_doc}, + {"SetCreatorAndType", MacOS_SetCreatorAndType, 1, setcrtp_doc}, + {"GetErrorString", MacOS_GetErrorString, 1, geterr_doc}, + {"openrf", MacOS_openrf, 1, openrf_doc}, + {"splash", MacOS_splash, 1, splash_doc}, + {"DebugStr", MacOS_DebugStr, 1, DebugStr_doc}, + {"GetTicks", MacOS_GetTicks, 1, GetTicks_doc}, + {"SysBeep", MacOS_SysBeep, 1, SysBeep_doc}, + {"WMAvailable", MacOS_WMAvailable, 1, WMAvailable_doc}, + {NULL, NULL} /* Sentinel */ +}; + + +void +initMacOS(void) +{ + PyObject *m, *d; + + m = Py_InitModule("MacOS", MacOS_Methods); + d = PyModule_GetDict(m); + + /* Initialize MacOS.Error exception */ + MacOS_Error = PyMac_GetOSErrException(); + if (MacOS_Error == NULL || PyDict_SetItemString(d, "Error", MacOS_Error) != 0) + return; + Rftype.ob_type = &PyType_Type; + Py_INCREF(&Rftype); + if (PyDict_SetItemString(d, "ResourceForkType", (PyObject *)&Rftype) != 0) + return; + /* + ** This is a hack: the following constant added to the id() of a string + ** object gives you the address of the data. Unfortunately, it is needed for + ** some of the image and sound processing interfaces on the mac:-( + */ + { + PyStringObject *p = 0; + long off = (long)&(p->ob_sval[0]); + + if( PyDict_SetItemString(d, "string_id_to_buffer", Py_BuildValue("i", off)) != 0) + return; + } +#define PY_RUNTIMEMODEL "macho" + if (PyDict_SetItemString(d, "runtimemodel", + Py_BuildValue("s", PY_RUNTIMEMODEL)) != 0) + return; +#if defined(WITH_NEXT_FRAMEWORK) +#define PY_LINKMODEL "framework" +#elif defined(Py_ENABLE_SHARED) +#define PY_LINKMODEL "shared" +#else +#define PY_LINKMODEL "static" +#endif + if (PyDict_SetItemString(d, "linkmodel", + Py_BuildValue("s", PY_LINKMODEL)) != 0) + return; + +} diff --git a/Mac/Modules/ae/_AEmodule.c b/Mac/Modules/ae/_AEmodule.c index 68be3bf..fd6ebcc 100644 --- a/Mac/Modules/ae/_AEmodule.c +++ b/Mac/Modules/ae/_AEmodule.c @@ -829,21 +829,19 @@ static PyObject *AEDesc_get_type(AEDescObject *self, void *closure) static PyObject *AEDesc_get_data(AEDescObject *self, void *closure) { + PyObject *res; + Size size; + char *ptr; + OSErr err; - PyObject *res; - Size size; - char *ptr; - OSErr err; - - size = AEGetDescDataSize(&self->ob_itself); - if ( (res = PyString_FromStringAndSize(NULL, size)) == NULL ) - return NULL; - if ( (ptr = PyString_AsString(res)) == NULL ) - return NULL; - if ( (err=AEGetDescData(&self->ob_itself, ptr, size)) < 0 ) - return PyMac_Error(err); - return res; - + size = AEGetDescDataSize(&self->ob_itself); + if ( (res = PyString_FromStringAndSize(NULL, size)) == NULL ) + return NULL; + if ( (ptr = PyString_AsString(res)) == NULL ) + return NULL; + if ( (err=AEGetDescData(&self->ob_itself, ptr, size)) < 0 ) + return PyMac_Error(err); + return res; } #define AEDesc_set_data NULL @@ -1431,14 +1429,11 @@ void init_AE(void) PyObject *m; PyObject *d; - - - upp_AEIdleProc = NewAEIdleUPP(AEIdleProc); - upp_GenericEventHandler = NewAEEventHandlerUPP(GenericEventHandler); - PyMac_INIT_TOOLBOX_OBJECT_NEW(AEDesc *, AEDesc_New); - PyMac_INIT_TOOLBOX_OBJECT_NEW(AEDesc *, AEDesc_NewBorrowed); - PyMac_INIT_TOOLBOX_OBJECT_CONVERT(AEDesc, AEDesc_Convert); - + upp_AEIdleProc = NewAEIdleUPP(AEIdleProc); + upp_GenericEventHandler = NewAEEventHandlerUPP(GenericEventHandler); + PyMac_INIT_TOOLBOX_OBJECT_NEW(AEDesc *, AEDesc_New); + PyMac_INIT_TOOLBOX_OBJECT_NEW(AEDesc *, AEDesc_NewBorrowed); + PyMac_INIT_TOOLBOX_OBJECT_CONVERT(AEDesc, AEDesc_Convert); m = Py_InitModule("_AE", AE_methods); d = PyModule_GetDict(m); diff --git a/Mac/Modules/macosmodule.c b/Mac/Modules/macosmodule.c deleted file mode 100644 index ed86fd0..0000000 --- a/Mac/Modules/macosmodule.c +++ /dev/null @@ -1,643 +0,0 @@ -/*********************************************************** -Copyright 1991-1997 by Stichting Mathematisch Centrum, Amsterdam, -The Netherlands. - - All Rights Reserved - -Permission to use, copy, modify, and distribute this software and its -documentation for any purpose and without fee is hereby granted, -provided that the above copyright notice appear in all copies and that -both that copyright notice and this permission notice appear in -supporting documentation, and that the names of Stichting Mathematisch -Centrum or CWI not be used in advertising or publicity pertaining to -distribution of the software without specific, written prior permission. - -STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO -THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND -FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE -FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES -WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN -ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT -OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. - -******************************************************************/ - -/* Macintosh OS-specific interface */ - -#include "Python.h" -#include "pymactoolbox.h" - -#include -#include - -static PyObject *MacOS_Error; /* Exception MacOS.Error */ - -#define PATHNAMELEN 1024 - -/* ----------------------------------------------------- */ - -/* Declarations for objects of type Resource fork */ - -typedef struct { - PyObject_HEAD - short fRefNum; - int isclosed; -} rfobject; - -static PyTypeObject Rftype; - - - -/* ---------------------------------------------------------------- */ - -static void -do_close(rfobject *self) -{ - if (self->isclosed ) return; - (void)FSClose(self->fRefNum); - self->isclosed = 1; -} - -static char rf_read__doc__[] = -"Read data from resource fork" -; - -static PyObject * -rf_read(rfobject *self, PyObject *args) -{ - long n; - PyObject *v; - OSErr err; - - if (self->isclosed) { - PyErr_SetString(PyExc_ValueError, "Operation on closed file"); - return NULL; - } - - if (!PyArg_ParseTuple(args, "l", &n)) - return NULL; - - v = PyString_FromStringAndSize((char *)NULL, n); - if (v == NULL) - return NULL; - - err = FSRead(self->fRefNum, &n, PyString_AsString(v)); - if (err && err != eofErr) { - PyMac_Error(err); - Py_DECREF(v); - return NULL; - } - _PyString_Resize(&v, n); - return v; -} - - -static char rf_write__doc__[] = -"Write to resource fork" -; - -static PyObject * -rf_write(rfobject *self, PyObject *args) -{ - char *buffer; - long size; - OSErr err; - - if (self->isclosed) { - PyErr_SetString(PyExc_ValueError, "Operation on closed file"); - return NULL; - } - if (!PyArg_ParseTuple(args, "s#", &buffer, &size)) - return NULL; - err = FSWrite(self->fRefNum, &size, buffer); - if (err) { - PyMac_Error(err); - return NULL; - } - Py_INCREF(Py_None); - return Py_None; -} - - -static char rf_seek__doc__[] = -"Set file position" -; - -static PyObject * -rf_seek(rfobject *self, PyObject *args) -{ - long amount, pos; - int whence = SEEK_SET; - long eof; - OSErr err; - - if (self->isclosed) { - PyErr_SetString(PyExc_ValueError, "Operation on closed file"); - return NULL; - } - if (!PyArg_ParseTuple(args, "l|i", &amount, &whence)) - return NULL; - - if ((err = GetEOF(self->fRefNum, &eof))) - goto ioerr; - - switch (whence) { - case SEEK_CUR: - if ((err = GetFPos(self->fRefNum, &pos))) - goto ioerr; - break; - case SEEK_END: - pos = eof; - break; - case SEEK_SET: - pos = 0; - break; - default: - PyErr_BadArgument(); - return NULL; - } - - pos += amount; - - /* Don't bother implementing seek past EOF */ - if (pos > eof || pos < 0) { - PyErr_BadArgument(); - return NULL; - } - - if ((err = SetFPos(self->fRefNum, fsFromStart, pos)) ) { -ioerr: - PyMac_Error(err); - return NULL; - } - Py_INCREF(Py_None); - return Py_None; -} - - -static char rf_tell__doc__[] = -"Get file position" -; - -static PyObject * -rf_tell(rfobject *self, PyObject *args) -{ - long where; - OSErr err; - - if (self->isclosed) { - PyErr_SetString(PyExc_ValueError, "Operation on closed file"); - return NULL; - } - if (!PyArg_ParseTuple(args, "")) - return NULL; - if ((err = GetFPos(self->fRefNum, &where)) ) { - PyMac_Error(err); - return NULL; - } - return PyInt_FromLong(where); -} - -static char rf_close__doc__[] = -"Close resource fork" -; - -static PyObject * -rf_close(rfobject *self, PyObject *args) -{ - if (!PyArg_ParseTuple(args, "")) - return NULL; - do_close(self); - Py_INCREF(Py_None); - return Py_None; -} - - -static struct PyMethodDef rf_methods[] = { - {"read", (PyCFunction)rf_read, 1, rf_read__doc__}, - {"write", (PyCFunction)rf_write, 1, rf_write__doc__}, - {"seek", (PyCFunction)rf_seek, 1, rf_seek__doc__}, - {"tell", (PyCFunction)rf_tell, 1, rf_tell__doc__}, - {"close", (PyCFunction)rf_close, 1, rf_close__doc__}, - - {NULL, NULL} /* sentinel */ -}; - -/* ---------- */ - - -static rfobject * -newrfobject(void) -{ - rfobject *self; - - self = PyObject_NEW(rfobject, &Rftype); - if (self == NULL) - return NULL; - self->isclosed = 1; - return self; -} - - -static void -rf_dealloc(rfobject *self) -{ - do_close(self); - PyObject_DEL(self); -} - -static PyObject * -rf_getattr(rfobject *self, char *name) -{ - return Py_FindMethod(rf_methods, (PyObject *)self, name); -} - -static char Rftype__doc__[] = -"Resource fork file object" -; - -static PyTypeObject Rftype = { - PyObject_HEAD_INIT(&PyType_Type) - 0, /*ob_size*/ - "MacOS.ResourceFork", /*tp_name*/ - sizeof(rfobject), /*tp_basicsize*/ - 0, /*tp_itemsize*/ - /* methods */ - (destructor)rf_dealloc, /*tp_dealloc*/ - (printfunc)0, /*tp_print*/ - (getattrfunc)rf_getattr, /*tp_getattr*/ - (setattrfunc)0, /*tp_setattr*/ - (cmpfunc)0, /*tp_compare*/ - (reprfunc)0, /*tp_repr*/ - 0, /*tp_as_number*/ - 0, /*tp_as_sequence*/ - 0, /*tp_as_mapping*/ - (hashfunc)0, /*tp_hash*/ - (ternaryfunc)0, /*tp_call*/ - (reprfunc)0, /*tp_str*/ - - /* Space for future expansion */ - 0L,0L,0L,0L, - Rftype__doc__ /* Documentation string */ -}; - -/* End of code for Resource fork objects */ -/* -------------------------------------------------------- */ - -/*----------------------------------------------------------------------*/ -/* Miscellaneous File System Operations */ - -static char getcrtp_doc[] = "Get MacOS 4-char creator and type for a file"; - -static PyObject * -MacOS_GetCreatorAndType(PyObject *self, PyObject *args) -{ - FSSpec fss; - FInfo info; - PyObject *creator, *type, *res; - OSErr err; - - if (!PyArg_ParseTuple(args, "O&", PyMac_GetFSSpec, &fss)) - return NULL; - if ((err = FSpGetFInfo(&fss, &info)) != noErr) - return PyErr_Mac(MacOS_Error, err); - creator = PyString_FromStringAndSize((char *)&info.fdCreator, 4); - type = PyString_FromStringAndSize((char *)&info.fdType, 4); - res = Py_BuildValue("OO", creator, type); - Py_DECREF(creator); - Py_DECREF(type); - return res; -} - -static char setcrtp_doc[] = "Set MacOS 4-char creator and type for a file"; - -static PyObject * -MacOS_SetCreatorAndType(PyObject *self, PyObject *args) -{ - FSSpec fss; - ResType creator, type; - FInfo info; - OSErr err; - - if (!PyArg_ParseTuple(args, "O&O&O&", - PyMac_GetFSSpec, &fss, PyMac_GetOSType, &creator, PyMac_GetOSType, &type)) - return NULL; - if ((err = FSpGetFInfo(&fss, &info)) != noErr) - return PyErr_Mac(MacOS_Error, err); - info.fdCreator = creator; - info.fdType = type; - if ((err = FSpSetFInfo(&fss, &info)) != noErr) - return PyErr_Mac(MacOS_Error, err); - Py_INCREF(Py_None); - return Py_None; -} - - -static char geterr_doc[] = "Convert OSErr number to string"; - -static PyObject * -MacOS_GetErrorString(PyObject *self, PyObject *args) -{ - int err; - char buf[256]; - Handle h; - char *str; - static int errors_loaded; - - if (!PyArg_ParseTuple(args, "i", &err)) - return NULL; - - h = GetResource('Estr', err); - if (!h && !errors_loaded) { - /* - ** Attempt to open the resource file containing the - ** Estr resources. We ignore all errors. We also try - ** this only once. - */ - PyObject *m, *rv; - errors_loaded = 1; - - m = PyImport_ImportModule("macresource"); - if (!m) { - if (Py_VerboseFlag) - PyErr_Print(); - PyErr_Clear(); - } - else { - rv = PyObject_CallMethod(m, "open_error_resource", ""); - if (!rv) { - if (Py_VerboseFlag) - PyErr_Print(); - PyErr_Clear(); - } - else { - Py_DECREF(rv); - /* And try again... */ - h = GetResource('Estr', err); - } - } - } - /* - ** Whether the code above succeeded or not, we won't try - ** again. - */ - errors_loaded = 1; - - if (h) { - HLock(h); - str = (char *)*h; - memcpy(buf, str+1, (unsigned char)str[0]); - buf[(unsigned char)str[0]] = '\0'; - HUnlock(h); - ReleaseResource(h); - } - else { - PyOS_snprintf(buf, sizeof(buf), "Mac OS error code %d", err); - } - - return Py_BuildValue("s", buf); -} - -static char splash_doc[] = "Open a splash-screen dialog by resource-id (0=close)"; - -static PyObject * -MacOS_splash(PyObject *self, PyObject *args) -{ - int resid = -1; - static DialogPtr curdialog = NULL; - DialogPtr olddialog; - WindowRef theWindow; - CGrafPtr thePort; -#if 0 - short xpos, ypos, width, height, swidth, sheight; -#endif - - if (!PyArg_ParseTuple(args, "|i", &resid)) - return NULL; - olddialog = curdialog; - curdialog = NULL; - - if ( resid != -1 ) { - curdialog = GetNewDialog(resid, NULL, (WindowPtr)-1); - if ( curdialog ) { - theWindow = GetDialogWindow(curdialog); - thePort = GetWindowPort(theWindow); -#if 0 - width = thePort->portRect.right - thePort->portRect.left; - height = thePort->portRect.bottom - thePort->portRect.top; - swidth = qd.screenBits.bounds.right - qd.screenBits.bounds.left; - sheight = qd.screenBits.bounds.bottom - qd.screenBits.bounds.top - LMGetMBarHeight(); - xpos = (swidth-width)/2; - ypos = (sheight-height)/5 + LMGetMBarHeight(); - MoveWindow(theWindow, xpos, ypos, 0); - ShowWindow(theWindow); -#endif - DrawDialog(curdialog); - } - } - if (olddialog) - DisposeDialog(olddialog); - Py_INCREF(Py_None); - return Py_None; -} - -static char DebugStr_doc[] = "Switch to low-level debugger with a message"; - -static PyObject * -MacOS_DebugStr(PyObject *self, PyObject *args) -{ - Str255 message; - PyObject *object = 0; - - if (!PyArg_ParseTuple(args, "O&|O", PyMac_GetStr255, message, &object)) - return NULL; - DebugStr(message); - Py_INCREF(Py_None); - return Py_None; -} - -static char SysBeep_doc[] = "BEEEEEP!!!"; - -static PyObject * -MacOS_SysBeep(PyObject *self, PyObject *args) -{ - int duration = 6; - - if (!PyArg_ParseTuple(args, "|i", &duration)) - return NULL; - SysBeep(duration); - Py_INCREF(Py_None); - return Py_None; -} - -static char WMAvailable_doc[] = - "True if this process can interact with the display." - "Will foreground the application on the first call as a side-effect." - ; - -static PyObject * -MacOS_WMAvailable(PyObject *self, PyObject *args) -{ - static PyObject *rv = NULL; - - if (!PyArg_ParseTuple(args, "")) - return NULL; - if (!rv) { - ProcessSerialNumber psn; - - /* - ** This is a fairly innocuous call to make if we don't have a window - ** manager, or if we have no permission to talk to it. It will print - ** a message on stderr, but at least it won't abort the process. - ** It appears the function caches the result itself, and it's cheap, so - ** no need for us to cache. - */ -#ifdef kCGNullDirectDisplay - /* On 10.1 CGMainDisplayID() isn't available, and - ** kCGNullDirectDisplay isn't defined. - */ - if (CGMainDisplayID() == 0) { - rv = Py_False; - } else { -#else - { -#endif - if (GetCurrentProcess(&psn) < 0 || - SetFrontProcess(&psn) < 0) { - rv = Py_False; - } else { - rv = Py_True; - } - } - } - Py_INCREF(rv); - return rv; -} - -static char GetTicks_doc[] = "Return number of ticks since bootup"; - -static PyObject * -MacOS_GetTicks(PyObject *self, PyObject *args) -{ - return Py_BuildValue("i", (int)TickCount()); -} - -static char openrf_doc[] = "Open resource fork of a file"; - -static PyObject * -MacOS_openrf(PyObject *self, PyObject *args) -{ - OSErr err; - char *mode = "r"; - FSSpec fss; - SignedByte permission = 1; - rfobject *fp; - - if (!PyArg_ParseTuple(args, "O&|s", PyMac_GetFSSpec, &fss, &mode)) - return NULL; - while (*mode) { - switch (*mode++) { - case '*': break; - case 'r': permission = 1; break; - case 'w': permission = 2; break; - case 'b': break; - default: - PyErr_BadArgument(); - return NULL; - } - } - - if ( (fp = newrfobject()) == NULL ) - return NULL; - - err = HOpenRF(fss.vRefNum, fss.parID, fss.name, permission, &fp->fRefNum); - - if ( err == fnfErr ) { - /* In stead of doing complicated things here to get creator/type - ** correct we let the standard i/o library handle it - */ - FILE *tfp; - char pathname[PATHNAMELEN]; - - if ( (err=PyMac_GetFullPathname(&fss, pathname, PATHNAMELEN)) ) { - PyMac_Error(err); - Py_DECREF(fp); - return NULL; - } - - if ( (tfp = fopen(pathname, "w")) == NULL ) { - PyMac_Error(fnfErr); /* What else... */ - Py_DECREF(fp); - return NULL; - } - fclose(tfp); - err = HOpenRF(fss.vRefNum, fss.parID, fss.name, permission, &fp->fRefNum); - } - if ( err ) { - Py_DECREF(fp); - PyMac_Error(err); - return NULL; - } - fp->isclosed = 0; - return (PyObject *)fp; -} - - -static PyMethodDef MacOS_Methods[] = { - {"GetCreatorAndType", MacOS_GetCreatorAndType, 1, getcrtp_doc}, - {"SetCreatorAndType", MacOS_SetCreatorAndType, 1, setcrtp_doc}, - {"GetErrorString", MacOS_GetErrorString, 1, geterr_doc}, - {"openrf", MacOS_openrf, 1, openrf_doc}, - {"splash", MacOS_splash, 1, splash_doc}, - {"DebugStr", MacOS_DebugStr, 1, DebugStr_doc}, - {"GetTicks", MacOS_GetTicks, 1, GetTicks_doc}, - {"SysBeep", MacOS_SysBeep, 1, SysBeep_doc}, - {"WMAvailable", MacOS_WMAvailable, 1, WMAvailable_doc}, - {NULL, NULL} /* Sentinel */ -}; - - -void -initMacOS(void) -{ - PyObject *m, *d; - - m = Py_InitModule("MacOS", MacOS_Methods); - d = PyModule_GetDict(m); - - /* Initialize MacOS.Error exception */ - MacOS_Error = PyMac_GetOSErrException(); - if (MacOS_Error == NULL || PyDict_SetItemString(d, "Error", MacOS_Error) != 0) - return; - Rftype.ob_type = &PyType_Type; - Py_INCREF(&Rftype); - if (PyDict_SetItemString(d, "ResourceForkType", (PyObject *)&Rftype) != 0) - return; - /* - ** This is a hack: the following constant added to the id() of a string - ** object gives you the address of the data. Unfortunately, it is needed for - ** some of the image and sound processing interfaces on the mac:-( - */ - { - PyStringObject *p = 0; - long off = (long)&(p->ob_sval[0]); - - if( PyDict_SetItemString(d, "string_id_to_buffer", Py_BuildValue("i", off)) != 0) - return; - } -#define PY_RUNTIMEMODEL "macho" - if (PyDict_SetItemString(d, "runtimemodel", - Py_BuildValue("s", PY_RUNTIMEMODEL)) != 0) - return; -#if defined(WITH_NEXT_FRAMEWORK) -#define PY_LINKMODEL "framework" -#elif defined(Py_ENABLE_SHARED) -#define PY_LINKMODEL "shared" -#else -#define PY_LINKMODEL "static" -#endif - if (PyDict_SetItemString(d, "linkmodel", - Py_BuildValue("s", PY_LINKMODEL)) != 0) - return; - -} diff --git a/Mac/PythonLauncher/FileSettings.m b/Mac/PythonLauncher/FileSettings.m index fc3937b..ba375ba 100755 --- a/Mac/PythonLauncher/FileSettings.m +++ b/Mac/PythonLauncher/FileSettings.m @@ -245,12 +245,26 @@ if (value) with_terminal = [value boolValue]; } +- (NSString*)_replaceSingleQuotes: (NSString*)string +{ + /* Replace all single-quotes by '"'"', that way shellquoting will + * be correct when the result value is delimited using single quotes. + */ + NSArray* components = [string componentsSeparatedByString:@"'"]; + + return [components componentsJoinedByString:@"'\"'\"'"]; +} + - (NSString *)commandLineForScript: (NSString *)script { NSString *cur_interp = NULL; + NSString* script_dir = NULL; char hashbangbuf[1024]; FILE *fp; char *p; + + script_dir = [script substringToIndex: + [script length]-[[script lastPathComponent] length]]; if (honourhashbang && (fp=fopen([script cString], "r")) && @@ -266,8 +280,9 @@ cur_interp = interpreter; return [NSString stringWithFormat: - @"\"%@\"%s%s%s%s%s%s %@ \"%@\" %@ %s", - cur_interp, + @"cd '%@' && '%@'%s%s%s%s%s%s %@ '%@' %@ %s", + [self _replaceSingleQuotes:script_dir], + [self _replaceSingleQuotes:cur_interp], debug?" -d":"", verbose?" -v":"", inspect?" -i":"", @@ -275,7 +290,7 @@ nosite?" -S":"", tabs?" -t":"", others, - script, + [self _replaceSingleQuotes:script], scriptargs, with_terminal? "&& echo Exit status: $? && exit 1" : " &"]; } diff --git a/Mac/Tools/fixapplepython23.py b/Mac/Tools/fixapplepython23.py index ef352ce..fb8645a 100644 --- a/Mac/Tools/fixapplepython23.py +++ b/Mac/Tools/fixapplepython23.py @@ -94,9 +94,19 @@ def main(): else: do_apply = True # First check OS version + if sys.byteorder == 'little': + # All intel macs are fine + print "fixapplypython23: no fix is needed on MacOSX on Intel" + sys.exit(0) + if gestalt.gestalt('sysv') < 0x1030: print 'fixapplepython23: no fix needed on MacOSX < 10.3' sys.exit(0) + + if gestalt.gestalt('sysv') >= 0x1040: + print 'fixapplepython23: no fix needed on MacOSX >= 10.4' + sys.exit(0) + # Test that a framework Python is indeed installed if not os.path.exists(MAKEFILE): print 'fixapplepython23: Python framework does not appear to be installed (?), nothing fixed' diff --git a/Makefile.pre.in b/Makefile.pre.in index f822838..74ce8a8 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -318,6 +318,7 @@ OBJECT_OBJS= \ ########################################################################## # objects that get linked into the Python library LIBRARY_OBJS= \ + Modules/_typesmodule.o \ Modules/getbuildinfo.o \ $(PARSER_OBJS) \ $(OBJECT_OBJS) \ @@ -354,6 +355,7 @@ sharedmods: $(BUILDPYTHON) $(LIBRARY): $(LIBRARY_OBJS) -rm -f $@ $(AR) cr $@ Modules/getbuildinfo.o + $(AR) cr $@ Modules/_typesmodule.o $(AR) cr $@ $(PARSER_OBJS) $(AR) cr $@ $(OBJECT_OBJS) $(AR) cr $@ $(PYTHON_OBJS) @@ -486,7 +488,7 @@ $(AST_H): $(AST_ASDL) $(ASDLGEN_FILES) $(AST_C): $(AST_ASDL) $(ASDLGEN_FILES) $(ASDLGEN) -c $(AST_C_DIR) $(AST_ASDL) - + Python/compile.o Python/symtable.o: $(GRAMMAR_H) $(AST_H) Python/getplatform.o: $(srcdir)/Python/getplatform.c @@ -651,16 +653,11 @@ bininstall: altbininstall else true; \ fi (cd $(DESTDIR)$(BINDIR); $(LN) python$(VERSION)$(EXE) $(PYTHON)) + (cd $(DESTDIR)$(BINDIR); $(LN) -sf python$(VERSION)-config python-config) # Install the interpreter with $(VERSION) affixed # This goes into $(exec_prefix) altbininstall: $(BUILDPYTHON) - @if test "$(PYTHONFRAMEWORKDIR)" != no-framework; then \ - if test ! -f $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Versions/Current/Resources/Info.plist; then \ - echo 'Framework build: use "make frameworkinstall" in stead of "make install"'; \ - exit 1; \ - fi; \ - fi @for i in $(BINDIR) $(LIBDIR); \ do \ if test ! -d $(DESTDIR)$$i; then \ @@ -700,7 +697,7 @@ PLATDIR= plat-$(MACHDEP) EXTRAPLATDIR= @EXTRAPLATDIR@ EXTRAMACHDEPPATH=@EXTRAMACHDEPPATH@ MACHDEPS= $(PLATDIR) $(EXTRAPLATDIR) -XMLLIBSUBDIRS= xmlcore xmlcore/dom xmlcore/etree xmlcore/parsers xmlcore/sax +XMLLIBSUBDIRS= xml xml/dom xml/etree xml/parsers xml/sax PLATMACDIRS= plat-mac plat-mac/Carbon plat-mac/lib-scriptpackages \ plat-mac/lib-scriptpackages/_builtinSuites \ plat-mac/lib-scriptpackages/CodeWarrior \ @@ -709,14 +706,14 @@ PLATMACDIRS= plat-mac plat-mac/Carbon plat-mac/lib-scriptpackages \ plat-mac/lib-scriptpackages/Netscape \ plat-mac/lib-scriptpackages/StdSuites \ plat-mac/lib-scriptpackages/SystemEvents \ - plat-mac/lib-scriptpackages/Terminal + plat-mac/lib-scriptpackages/Terminal PLATMACPATH=:plat-mac:plat-mac/lib-scriptpackages -LIBSUBDIRS= lib-old lib-tk site-packages test test/output test/data \ +LIBSUBDIRS= lib-tk site-packages test test/output test/data \ test/decimaltestdata \ encodings compiler hotshot \ email email/mime email/test email/test/data \ sqlite3 sqlite3/test \ - logging bsddb bsddb/test csv \ + logging bsddb bsddb/test csv wsgiref \ ctypes ctypes/test ctypes/macholib idlelib idlelib/Icons \ distutils distutils/command distutils/tests $(XMLLIBSUBDIRS) \ setuptools setuptools/command setuptools/tests setuptools.egg-info \ @@ -741,7 +738,7 @@ libinstall: $(BUILDPYTHON) $(srcdir)/Lib/$(PLATDIR) else true; \ fi; \ done - @for i in $(srcdir)/Lib/*.py $(srcdir)/Lib/*.doc; \ + @for i in $(srcdir)/Lib/*.py $(srcdir)/Lib/*.doc $(srcdir)/Lib/*.egg-info ; \ do \ if test -x $$i; then \ $(INSTALL_SCRIPT) $$i $(DESTDIR)$(LIBDEST); \ @@ -857,8 +854,8 @@ libainstall: all $(INSTALL_SCRIPT) $(srcdir)/install-sh $(DESTDIR)$(LIBPL)/install-sh # Substitution happens here, as the completely-expanded BINDIR # is not available in configure - sed -e "s,@BINDIR@,$(BINDIR)," < $(srcdir)/Misc/python-config.in >python-config - $(INSTALL_SCRIPT) python-config $(DESTDIR)$(BINDIR)/python-config + sed -e "s,@EXENAME@,$(BINDIR)/python$(VERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config + $(INSTALL_SCRIPT) python-config $(DESTDIR)$(BINDIR)/python$(VERSION)-config rm python-config @if [ -s Modules/python.exp -a \ "`echo $(MACHDEP) | sed 's/^\(...\).*/\1/'`" = "aix" ]; then \ @@ -939,7 +936,10 @@ frameworkinstallstructure: $(LDLIBRARY) $(INSTALL_SHARED) $(LDLIBRARY) $(DESTDIR)$(PYTHONFRAMEWORKPREFIX)/$(LDLIBRARY) # This installs Mac/Lib into the framework +# Install a number of symlinks to keep software that expects a normal unix +# install (which includes python-config) happy. frameworkinstallmaclib: + ln -fs "../../../Python" "$(DESTDIR)$(prefix)/lib/python$(VERSION)/config/libpython$(VERSION).a" cd Mac && $(MAKE) installmacsubtree DESTDIR="$(DESTDIR)" # This installs the IDE, the Launcher and other apps into /Applications diff --git a/Misc/ACKS b/Misc/ACKS index e30cc61..e13b594 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -24,6 +24,7 @@ Ross Andrus Jason Asbahr David Ascher Peter Åstrand +Chris AtLee John Aycock Donovan Baarda Attila Babo @@ -202,8 +203,10 @@ Vincent Fiack Russell Finn Nils Fischbeck Frederik Fix +Matt Fleming Hernán Martínez Foffani Doug Fort +John Fouhy Martin Franklin Robin Friedrich Ivan Frohne @@ -432,6 +435,7 @@ Michael Muller Takahiro Nakayama Travers Naran Fredrik Nehr +Tony Nelson Chad Netzer Max Neunhöffer George Neville-Neil @@ -527,6 +531,7 @@ Hugo van Rossum Saskia van Rossum Donald Wallace Rouse II Liam Routt +Sam Ruby Paul Rubin Audun S. Runde Jeff Rush @@ -620,6 +625,7 @@ Laurence Tratt John Tromp Jason Trowbridge Anthony Tuininga +Christopher Tur Lesniewski-Laas Stephen Turner Bill Tutt Doobee R. Tzeck diff --git a/Misc/README.coverity b/Misc/README.coverity new file mode 100644 index 0000000..f5e1bf6 --- /dev/null +++ b/Misc/README.coverity @@ -0,0 +1,22 @@ + +Coverity has a static analysis tool (Prevent) which is similar to Klocwork. +They run their tool on the Python source code (SVN head) on a daily basis. +The results are available at: + + http://scan.coverity.com/ + +About 20 people have access to the analysis reports. Other +people can be added by request. + +Prevent was first run on the Python 2.5 source code in March 2006. +There were originally about 100 defects reported. Some of these +were false positives. Over 70 issues were uncovered. + +Each warning has a unique id and comments that can be made on it. +When checking in changes due to a warning, the unique id +as reported by the tool was added to the SVN commit message. + +False positives were annotated so that the comments can +be reviewed and reversed if the analysis was incorrect. + +Contact python-dev@python.org for more information. diff --git a/Misc/README.klocwork b/Misc/README.klocwork new file mode 100644 index 0000000..a22715e --- /dev/null +++ b/Misc/README.klocwork @@ -0,0 +1,26 @@ + +Klocwork has a static analysis tool (K7) which is similar to Coverity. +They will run their tool on the Python source code on demand. +The results are available at: + + https://opensource.klocwork.com/ + +Currently, only Neal Norwitz has access to the analysis reports. Other +people can be added by request. + +K7 was first run on the Python 2.5 source code in mid-July 2006. +This is after Coverity had been making their results available. +There were originally 175 defects reported. Most of these +were false positives. However, there were numerous real issues +also uncovered. + +Each warning has a unique id and comments that can be made on it. +When checking in changes due to a K7 report, the unique id +as reported by the tool was added to the SVN commit message. +A comment was added to the K7 warning indicating the SVN revision +in addition to any analysis. + +False positives were also annotated so that the comments can +be reviewed and reversed if the analysis was incorrect. + +Contact python-dev@python.org for more information. diff --git a/Misc/RPM/python-2.5.spec b/Misc/RPM/python-2.5.spec index e050454..afbb09e 100644 --- a/Misc/RPM/python-2.5.spec +++ b/Misc/RPM/python-2.5.spec @@ -33,7 +33,7 @@ ################################# %define name python -%define version 2.5a2 +%define version 2.5b3 %define libvers 2.5 %define release 1pydotorg %define __prefix /usr diff --git a/Misc/Vim/python.vim b/Misc/Vim/python.vim index 9711076..d83572a 100644 --- a/Misc/Vim/python.vim +++ b/Misc/Vim/python.vim @@ -85,7 +85,7 @@ if exists("python_highlight_exceptions") syn keyword pythonException LookupError OSError DeprecationWarning syn keyword pythonException UnicodeError UnicodeEncodeError syn keyword pythonException FloatingPointError ReferenceError NameError - syn keyword pythonException OverflowWarning IOError SyntaxError + syn keyword pythonException IOError SyntaxError syn keyword pythonException FutureWarning ImportWarning SystemExit syn keyword pythonException Exception EOFError StandardError ValueError syn keyword pythonException TabError KeyError ZeroDivisionError SystemError diff --git a/Misc/build.sh b/Misc/build.sh index de51539..ff46bba 100755 --- a/Misc/build.sh +++ b/Misc/build.sh @@ -54,13 +54,19 @@ RESULT_FILE="$DIR/build/index.html" INSTALL_DIR="/tmp/python-test/local" RSYNC_OPTS="-aC -e ssh" +# Always run the installed version of Python. +PYTHON=$INSTALL_DIR/bin/python + +# Python options and regression test program that should always be run. +REGRTEST_ARGS="-E -tt $INSTALL_DIR/lib/python2.5/test/regrtest.py" + REFLOG="build/reflog.txt.out" # These tests are not stable and falsely report leaks sometimes. # The entire leak report will be mailed if any test not in this list leaks. # Note: test_XXX (none currently) really leak, but are disabled # so we don't send spam. Any test which really leaks should only # be listed here if there are also test cases under Lib/test/leakers. -LEAKY_TESTS="test_(ctypes|filecmp|socket|threadedtempfile|threading|urllib2)" +LEAKY_TESTS="test_(XXX)" # Currently no tests should report spurious leaks. # Skip these tests altogether when looking for leaks. These tests # do not need to be stored above in LEAKY_TESTS too. @@ -155,21 +161,31 @@ if [ $err = 0 -a "$BUILD_DISABLED" != "yes" ]; then make install >& build/$F update_status "Installing" "$F" $start + if [ ! -x $PYTHON ]; then + ln -s ${PYTHON}2.* $PYTHON + fi + ## make and run basic tests F=make-test.out start=`current_time` - make test >& build/$F + $PYTHON $REGRTEST_ARGS >& build/$F NUM_FAILURES=`grep -ic " failed:" build/$F` update_status "Testing basics ($NUM_FAILURES failures)" "$F" $start - ## FIXME: should mail since -uall below should find same problems mail_on_failure "basics" build/$F + F=make-test-opt.out + start=`current_time` + $PYTHON -O $REGRTEST_ARGS >& build/$F + NUM_FAILURES=`grep -ic " failed:" build/$F` + update_status "Testing opt ($NUM_FAILURES failures)" "$F" $start + mail_on_failure "opt" build/$F + ## run the tests looking for leaks F=make-test-refleak.out start=`current_time` ## ensure that the reflog exists so the grep doesn't fail touch $REFLOG - ./python ./Lib/test/regrtest.py -R 4:3:$REFLOG -u network $LEAKY_SKIPS >& build/$F + $PYTHON $REGRTEST_ARGS -R 4:3:$REFLOG -u network $LEAKY_SKIPS >& build/$F NUM_FAILURES=`egrep -vc "$LEAKY_TESTS" $REFLOG` update_status "Testing refleaks ($NUM_FAILURES failures)" "$F" $start mail_on_failure "refleak" $REFLOG @@ -179,7 +195,7 @@ if [ $err = 0 -a "$BUILD_DISABLED" != "yes" ]; then start=`current_time` ## skip curses when running from cron since there's no terminal ## skip sound since it's not setup on the PSF box (/dev/dsp) - ./python -E -tt ./Lib/test/regrtest.py -uall -x test_curses test_linuxaudiodev test_ossaudiodev >& build/$F + $PYTHON $REGRTEST_ARGS -uall -x test_curses test_linuxaudiodev test_ossaudiodev >& build/$F NUM_FAILURES=`grep -ic " failed:" build/$F` update_status "Testing all except curses and sound ($NUM_FAILURES failures)" "$F" $start mail_on_failure "all" build/$F @@ -189,7 +205,7 @@ fi ## make doc -cd Doc +cd $DIR/Doc F="make-doc.out" start=`current_time` make >& ../build/$F diff --git a/Misc/cheatsheet b/Misc/cheatsheet index 024545d..b0e1cce 100644 --- a/Misc/cheatsheet +++ b/Misc/cheatsheet @@ -1125,7 +1125,6 @@ Exception> DeprecationWarning PendingDeprecationWarning SyntaxWarning - OverflowWarning RuntimeWarning FutureWarning diff --git a/Misc/python-config.in b/Misc/python-config.in index 24e699e..e0215a2 100644 --- a/Misc/python-config.in +++ b/Misc/python-config.in @@ -1,4 +1,4 @@ -#!@BINDIR@/python +#!@EXENAME@ import sys import os @@ -36,13 +36,14 @@ elif opt == '--exec-prefix': print sysconfig.EXEC_PREFIX elif opt in ('--includes', '--cflags'): - flags = ['-I'+dir for dir in getvar('INCLDIRSTOMAKE').split()] + flags = ['-I' + sysconfig.get_python_inc(), + '-I' + sysconfig.get_python_inc(plat_specific=True)] if opt == '--cflags': flags.extend(getvar('CFLAGS').split()) print ' '.join(flags) elif opt in ('--libs', '--ldflags'): - libs = sysconfig.get_config_var('LIBS').split() + libs = getvar('LIBS').split() + getvar('SYSLIBS').split() libs.append('-lpython'+pyver) if opt == '--ldflags': libs.insert(0, '-L' + getvar('LIBPL')) diff --git a/Misc/valgrind-python.supp b/Misc/valgrind-python.supp index 2688f79..4a6710e 100644 --- a/Misc/valgrind-python.supp +++ b/Misc/valgrind-python.supp @@ -134,6 +134,15 @@ ### { + Generic gentoo ld problems + Memcheck:Cond + obj:/lib/ld-2.3.4.so + obj:/lib/ld-2.3.4.so + obj:/lib/ld-2.3.4.so + obj:/lib/ld-2.3.4.so +} + +{ DBM problems, see test_dbm Memcheck:Param write(buf) diff --git a/Modules/Setup.dist b/Modules/Setup.dist index 1b2502d..46f4253 100644 --- a/Modules/Setup.dist +++ b/Modules/Setup.dist @@ -59,7 +59,7 @@ # # In addition, *static* explicitly declares the following modules to # be static. Lines containing "*static*" and "*shared*" may thus -# alternate thoughout this file. +# alternate throughout this file. # NOTE: As a standard policy, as many modules as can be supported by a # platform should be present. The distribution comes with all modules @@ -234,16 +234,19 @@ GLHACK=-Dclear=__GLclear #rgbimg rgbimgmodule.c # Read SGI RGB image files (but coded portably) -# The md5 module implements the RSA Data Security, Inc. MD5 +# Note that the _md5 and _sha modules are normally only built if the +# system does not have the OpenSSL libs containing an optimized version. + +# The _md5 module implements the RSA Data Security, Inc. MD5 # Message-Digest Algorithm, described in RFC 1321. The necessary files # md5.c and md5.h are included here. -#md5 md5module.c md5.c +#_md5 md5module.c md5.c -# The sha module implements the SHA checksum algorithm. +# The _sha module implements the SHA checksum algorithm. # (NIST's Secure Hash Algorithm.) -#sha shamodule.c +#_sha shamodule.c # SGI IRIX specific modules -- off by default. diff --git a/Modules/_bsddb.c b/Modules/_bsddb.c index 9220866..2a3ac5c 100644 --- a/Modules/_bsddb.c +++ b/Modules/_bsddb.c @@ -98,12 +98,12 @@ #error "eek! DBVER can't handle minor versions > 9" #endif -#define PY_BSDDB_VERSION "4.4.4" +#define PY_BSDDB_VERSION "4.4.5" static char *rcs_id = "$Id$"; #if (PY_VERSION_HEX < 0x02050000) -#define Py_ssize_t int +typedef int Py_ssize_t; #endif #ifdef WITH_THREAD @@ -528,6 +528,7 @@ static int makeDBError(int err) PyObject *errObj = NULL; PyObject *errTuple = NULL; int exceptionRaised = 0; + unsigned int bytes_left; switch (err) { case 0: /* successful, no error */ break; @@ -535,12 +536,15 @@ static int makeDBError(int err) #if (DBVER < 41) case DB_INCOMPLETE: #if INCOMPLETE_IS_WARNING - our_strlcpy(errTxt, db_strerror(err), sizeof(errTxt)); - if (_db_errmsg[0]) { + bytes_left = our_strlcpy(errTxt, db_strerror(err), sizeof(errTxt)); + /* Ensure that bytes_left never goes negative */ + if (_db_errmsg[0] && bytes_left < (sizeof(errTxt) - 4)) { + bytes_left = sizeof(errTxt) - bytes_left - 4 - 1; + assert(bytes_left >= 0); strcat(errTxt, " -- "); - strcat(errTxt, _db_errmsg); - _db_errmsg[0] = 0; + strncat(errTxt, _db_errmsg, bytes_left); } + _db_errmsg[0] = 0; #ifdef HAVE_WARNINGS exceptionRaised = PyErr_Warn(PyExc_RuntimeWarning, errTxt); #else @@ -588,12 +592,15 @@ static int makeDBError(int err) } if (errObj != NULL) { - our_strlcpy(errTxt, db_strerror(err), sizeof(errTxt)); - if (_db_errmsg[0]) { + bytes_left = our_strlcpy(errTxt, db_strerror(err), sizeof(errTxt)); + /* Ensure that bytes_left never goes negative */ + if (_db_errmsg[0] && bytes_left < (sizeof(errTxt) - 4)) { + bytes_left = sizeof(errTxt) - bytes_left - 4 - 1; + assert(bytes_left >= 0); strcat(errTxt, " -- "); - strcat(errTxt, _db_errmsg); - _db_errmsg[0] = 0; + strncat(errTxt, _db_errmsg, bytes_left); } + _db_errmsg[0] = 0; errTuple = Py_BuildValue("(is)", err, errTxt); PyErr_SetObject(errObj, errTuple); @@ -798,10 +805,12 @@ newDBObject(DBEnvObject* arg, int flags) MYDB_BEGIN_ALLOW_THREADS; err = db_create(&self->db, db_env, flags); - self->db->set_errcall(self->db, _db_errorCallback); + if (self->db != NULL) { + self->db->set_errcall(self->db, _db_errorCallback); #if (DBVER >= 33) - self->db->app_private = (void*)self; + self->db->app_private = (void*)self; #endif + } MYDB_END_ALLOW_THREADS; /* TODO add a weakref(self) to the self->myenvobj->open_child_weakrefs * list so that a DBEnv can refuse to close without aborting any open @@ -2681,12 +2690,13 @@ DB_set_encrypt(DBObject* self, PyObject* args, PyObject* kwargs) /*-------------------------------------------------------------- */ /* Mapping and Dictionary-like access routines */ -Py_ssize_t DB_length(DBObject* self) +Py_ssize_t DB_length(PyObject* _self) { int err; Py_ssize_t size = 0; int flags = 0; void* sp; + DBObject* self = (DBObject*)_self; if (self->db == NULL) { PyObject *t = Py_BuildValue("(is)", 0, "DB object has been closed"); @@ -2720,6 +2730,8 @@ redo_stat_for_length: */ if (size == 0 && (flags & DB_FAST_STAT)) { flags = 0; + if (!err) + free(sp); goto redo_stat_for_length; } @@ -3864,7 +3876,7 @@ DBEnv_dbremove(DBEnvObject* self, PyObject* args, PyObject* kwargs) static char* kwnames[] = { "file", "database", "txn", "flags", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "ss|Oi:dbremove", kwnames, + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "s|zOi:dbremove", kwnames, &file, &database, &txnobj, &flags)) { return NULL; } @@ -3892,7 +3904,7 @@ DBEnv_dbrename(DBEnvObject* self, PyObject* args, PyObject* kwargs) static char* kwnames[] = { "file", "database", "newname", "txn", "flags", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "sss|Oi:dbrename", kwnames, + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "szs|Oi:dbrename", kwnames, &file, &database, &newname, &txnobj, &flags)) { return NULL; } @@ -5153,11 +5165,7 @@ static PyMethodDef DB_methods[] = { static PyMappingMethods DB_mapping = { -#if (PY_VERSION_HEX < 0x02050000) - (inquiry)DB_length, /*mp_length*/ -#else - (lenfunc)DB_length, /*mp_length*/ -#endif + DB_length, /*mp_length*/ (binaryfunc)DB_subscript, /*mp_subscript*/ (objobjargproc)DB_ass_sub, /*mp_ass_subscript*/ }; @@ -5560,15 +5568,13 @@ DBEnv_construct(PyObject* self, PyObject* args) static PyObject* DBSequence_construct(PyObject* self, PyObject* args, PyObject* kwargs) { - PyObject* dbobj = NULL; + PyObject* dbobj; int flags = 0; static char* kwnames[] = { "db", "flags", NULL}; if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|i:DBSequence", kwnames, &dbobj, &flags)) return NULL; - if (dbobj == Py_None) - dbobj = NULL; - else if (dbobj && !DBObject_Check(dbobj)) { + if (!DBObject_Check(dbobj)) { makeTypeError("DB", dbobj); return NULL; } diff --git a/Modules/_codecsmodule.c b/Modules/_codecsmodule.c index 32fa82f..405fd7a 100644 --- a/Modules/_codecsmodule.c +++ b/Modules/_codecsmodule.c @@ -479,15 +479,20 @@ mbcs_decode(PyObject *self, PyObject *args) { const char *data; - Py_ssize_t size; + Py_ssize_t size, consumed; const char *errors = NULL; + int final = 0; + PyObject *decoded; - if (!PyArg_ParseTuple(args, "t#|z:mbcs_decode", - &data, &size, &errors)) + if (!PyArg_ParseTuple(args, "t#|zi:mbcs_decode", + &data, &size, &errors, &final)) return NULL; - return codec_tuple(PyUnicode_DecodeMBCS(data, size, errors), - size); + decoded = PyUnicode_DecodeMBCSStateful( + data, size, errors, final ? NULL : &consumed); + if (!decoded) + return NULL; + return codec_tuple(decoded, final ? size : consumed); } #endif /* MS_WINDOWS */ diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index a0744ff..76d88de 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -1,3 +1,8 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + + /* ToDo: @@ -342,6 +347,14 @@ static PyMethodDef CDataType_methods[] = { static PyObject * CDataType_repeat(PyObject *self, Py_ssize_t length) { + if (length < 0) + return PyErr_Format(PyExc_ValueError, +#if (PY_VERSION_HEX < 0x02050000) + "Array length must be >= 0, not %d", +#else + "Array length must be >= 0, not %zd", +#endif + length); return CreateArrayType(self, length); } @@ -1371,13 +1384,20 @@ SimpleType_new(PyTypeObject *type, PyObject *args, PyObject *kwds) Py_DECREF(result); return NULL; } + fmt = getentry(PyString_AS_STRING(proto)); + if (fmt == NULL) { + Py_DECREF(result); + PyErr_Format(PyExc_ValueError, + "_type_ '%s' not supported", + PyString_AS_STRING(proto)); + return NULL; + } + stgdict = (StgDictObject *)PyObject_CallObject( (PyObject *)&StgDict_Type, NULL); if (!stgdict) return NULL; - fmt = getentry(PyString_AS_STRING(proto)); - stgdict->ffi_type_pointer = *fmt->pffi_type; stgdict->align = fmt->pffi_type->alignment; stgdict->length = 0; @@ -1620,9 +1640,8 @@ converters_from_argtypes(PyObject *ob) for (i = 0; i < nArgs; ++i) { PyObject *tp = PyTuple_GET_ITEM(ob, i); - StgDictObject *dict = PyType_stgdict(tp); PyObject *cnv = PyObject_GetAttrString(tp, "from_param"); - if (!dict || !cnv) + if (!cnv) goto argtypes_error_1; PyTuple_SET_ITEM(converters, i, cnv); } @@ -1633,7 +1652,7 @@ converters_from_argtypes(PyObject *ob) Py_XDECREF(converters); Py_DECREF(ob); PyErr_Format(PyExc_TypeError, - "item %d in _argtypes_ is not a valid C type", i+1); + "item %d in _argtypes_ has no from_param method", i+1); return NULL; } @@ -1809,23 +1828,53 @@ GetKeepedObjects(CDataObject *target) } static PyObject * -unique_key(CDataObject *target, int index) +unique_key(CDataObject *target, Py_ssize_t index) { - char string[256]; /* XXX is that enough? */ + char string[256]; char *cp = string; - *cp++ = index + '0'; + size_t bytes_left; + + assert(sizeof(string) - 1 > sizeof(Py_ssize_t) * 2); +#if (PY_VERSION_HEX < 0x02050000) + cp += sprintf(cp, "%x", index); +#else + cp += sprintf(cp, "%x", Py_SAFE_DOWNCAST(index, Py_ssize_t, int)); +#endif while (target->b_base) { - *cp++ = target->b_index + '0'; + bytes_left = sizeof(string) - (cp - string) - 1; + /* Hex format needs 2 characters per byte */ + if (bytes_left < sizeof(Py_ssize_t) * 2) { + PyErr_SetString(PyExc_ValueError, + "ctypes object structure too deep"); + return NULL; + } +#if (PY_VERSION_HEX < 0x02050000) + cp += sprintf(cp, ":%x", (int)target->b_index); +#else + cp += sprintf(cp, ":%x", Py_SAFE_DOWNCAST(target->b_index, Py_ssize_t, int)); +#endif target = target->b_base; } return PyString_FromStringAndSize(string, cp-string); } -/* Keep a reference to 'keep' in the 'target', at index 'index' */ + /* - * KeepRef travels the target's b_base pointer down to the root, - * building a sequence of indexes during the path. The indexes, which are a - * couple of small integers, are used to build a byte string usable as - * key int the root object's _objects dict. + * Keep a reference to 'keep' in the 'target', at index 'index'. + * + * If 'keep' is None, do nothing. + * + * Otherwise create a dictionary (if it does not yet exist) id the root + * objects 'b_objects' item, which will store the 'keep' object under a unique + * key. + * + * The unique_key helper travels the target's b_base pointer down to the root, + * building a string containing hex-formatted indexes found during traversal, + * separated by colons. + * + * The index tuple is used as a key into the root object's b_objects dict. + * + * Note: This function steals a refcount of the third argument, even if it + * fails! */ static int KeepRef(CDataObject *target, Py_ssize_t index, PyObject *keep) @@ -1846,6 +1895,10 @@ KeepRef(CDataObject *target, Py_ssize_t index, PyObject *keep) return 0; } key = unique_key(target, index); + if (key == NULL) { + Py_DECREF(keep); + return -1; + } result = PyDict_SetItem(ob->b_objects, key, keep); Py_DECREF(key); Py_DECREF(keep); @@ -2134,7 +2187,7 @@ _CData_set(CDataObject *dst, PyObject *type, SETFUNC setfunc, PyObject *value, Py_DECREF(ob); return result; } else if (value == Py_None && PointerTypeObject_Check(type)) { - *(void **)dst->b_ptr = NULL; + *(void **)ptr = NULL; Py_INCREF(Py_None); return Py_None; } else { @@ -2611,11 +2664,11 @@ CFuncPtr_FromDll(PyTypeObject *type, PyObject *args, PyObject *kwds) *(void **)self->b_ptr = address; + Py_INCREF((PyObject *)dll); /* for KeepRef */ if (-1 == KeepRef((CDataObject *)self, 0, dll)) { Py_DECREF((PyObject *)self); return NULL; } - Py_INCREF((PyObject *)dll); /* for KeepRef above */ Py_INCREF(self); self->callable = (PyObject *)self; @@ -2751,11 +2804,11 @@ CFuncPtr_new(PyTypeObject *type, PyObject *args, PyObject *kwds) correctly... */ + Py_INCREF((PyObject *)self); /* for KeepRef */ if (-1 == KeepRef((CDataObject *)self, 0, (PyObject *)self)) { Py_DECREF((PyObject *)self); return NULL; } - Py_INCREF((PyObject *)self); /* for KeepRef above */ return (PyObject *)self; } @@ -3520,6 +3573,7 @@ Array_item(PyObject *_self, Py_ssize_t index) int offset, size; StgDictObject *stgdict; + if (index < 0 || index >= self->b_length) { PyErr_SetString(PyExc_IndexError, "invalid index"); @@ -4019,7 +4073,8 @@ static PyObject * Pointer_item(PyObject *_self, Py_ssize_t index) { CDataObject *self = (CDataObject *)_self; - int size, offset; + int size; + Py_ssize_t offset; StgDictObject *stgdict, *itemdict; PyObject *proto; @@ -4029,9 +4084,9 @@ Pointer_item(PyObject *_self, Py_ssize_t index) return NULL; } - stgdict = PyObject_stgdict((PyObject *)self); assert(stgdict); + assert(stgdict->proto); proto = stgdict->proto; /* XXXXXX MAKE SURE PROTO IS NOT NULL! */ @@ -4039,7 +4094,7 @@ Pointer_item(PyObject *_self, Py_ssize_t index) size = itemdict->size; offset = index * itemdict->size; - return CData_get(stgdict->proto, stgdict->getfunc, (PyObject *)self, + return CData_get(proto, stgdict->getfunc, (PyObject *)self, index, size, (*(char **)self->b_ptr) + offset); } @@ -4048,7 +4103,9 @@ Pointer_ass_item(PyObject *_self, Py_ssize_t index, PyObject *value) { CDataObject *self = (CDataObject *)_self; int size; - StgDictObject *stgdict; + Py_ssize_t offset; + StgDictObject *stgdict, *itemdict; + PyObject *proto; if (value == NULL) { PyErr_SetString(PyExc_TypeError, @@ -4063,16 +4120,17 @@ Pointer_ass_item(PyObject *_self, Py_ssize_t index, PyObject *value) } stgdict = PyObject_stgdict((PyObject *)self); - if (index != 0) { - PyErr_SetString(PyExc_IndexError, - "invalid index"); - return -1; - } - size = stgdict->size / stgdict->length; + assert(stgdict); + assert(stgdict->proto); - /* XXXXX Make sure proto is NOT NULL! */ - return CData_set((PyObject *)self, stgdict->proto, stgdict->setfunc, value, - index, size, *(void **)self->b_ptr); + proto = stgdict->proto; + /* XXXXXX MAKE SURE PROTO IS NOT NULL! */ + itemdict = PyType_stgdict(proto); + size = itemdict->size; + offset = index * itemdict->size; + + return CData_set((PyObject *)self, proto, stgdict->setfunc, value, + index, size, (*(char **)self->b_ptr) + offset); } static PyObject * @@ -4089,8 +4147,8 @@ Pointer_get_contents(CDataObject *self, void *closure) stgdict = PyObject_stgdict((PyObject *)self); assert(stgdict); return CData_FromBaseObj(stgdict->proto, - (PyObject *)self, 0, - *(void **)self->b_ptr); + (PyObject *)self, 0, + *(void **)self->b_ptr); } static int @@ -4437,7 +4495,7 @@ cast_check_pointertype(PyObject *arg) } static PyObject * -cast(void *ptr, PyObject *ctype) +cast(void *ptr, PyObject *src, PyObject *ctype) { CDataObject *result; if (0 == cast_check_pointertype(ctype)) @@ -4445,9 +4503,46 @@ cast(void *ptr, PyObject *ctype) result = (CDataObject *)PyObject_CallFunctionObjArgs(ctype, NULL); if (result == NULL) return NULL; + + /* + The casted objects '_objects' member: + + It must certainly contain the source objects one. + It must contain the source object itself. + */ + if (CDataObject_Check(src)) { + CDataObject *obj = (CDataObject *)src; + /* CData_GetContainer will initialize src.b_objects, we need + this so it can be shared */ + CData_GetContainer(obj); + /* But we need a dictionary! */ + if (obj->b_objects == Py_None) { + Py_DECREF(Py_None); + obj->b_objects = PyDict_New(); + if (obj->b_objects == NULL) + goto failed; + } + result->b_objects = obj->b_objects; + if (result->b_objects) { + PyObject *index; + int rc; + Py_INCREF(obj->b_objects); + index = PyLong_FromVoidPtr((void *)src); + if (index == NULL) + goto failed; + rc = PyDict_SetItem(result->b_objects, index, src); + Py_DECREF(index); + if (rc == -1) + goto failed; + } + } /* Should we assert that result is a pointer type? */ memcpy(result->b_ptr, &ptr, sizeof(void *)); return (PyObject *)result; + + failed: + Py_DECREF(result); + return NULL; } #ifdef CTYPES_UNICODE @@ -4469,7 +4564,9 @@ init_ctypes(void) ob_type is the metatype (the 'type'), defaults to PyType_Type, tp_base is the base type, defaults to 'object' aka PyBaseObject_Type. */ +#ifdef WITH_THREAD PyEval_InitThreads(); +#endif m = Py_InitModule3("_ctypes", module_methods, module_docs); if (!m) return; @@ -4579,7 +4676,7 @@ init_ctypes(void) #endif PyModule_AddObject(m, "FUNCFLAG_CDECL", PyInt_FromLong(FUNCFLAG_CDECL)); PyModule_AddObject(m, "FUNCFLAG_PYTHONAPI", PyInt_FromLong(FUNCFLAG_PYTHONAPI)); - PyModule_AddStringConstant(m, "__version__", "0.9.9.6"); + PyModule_AddStringConstant(m, "__version__", "1.0.0"); PyModule_AddObject(m, "_memmove_addr", PyLong_FromVoidPtr(memmove)); PyModule_AddObject(m, "_memset_addr", PyLong_FromVoidPtr(memset)); @@ -4617,13 +4714,14 @@ init_ctypes(void) } /***************************************************************** - * replacements for broken Python api functions + * replacements for broken Python api functions (in Python 2.3). + * See #1047269 Buffer overwrite in PyUnicode_AsWideChar */ #ifdef HAVE_WCHAR_H PyObject *My_PyUnicode_FromWideChar(register const wchar_t *w, - int size) + Py_ssize_t size) { PyUnicodeObject *unicode; @@ -4655,7 +4753,7 @@ PyObject *My_PyUnicode_FromWideChar(register const wchar_t *w, int My_PyUnicode_AsWideChar(PyUnicodeObject *unicode, register wchar_t *w, - int size) + Py_ssize_t size) { if (unicode == NULL) { PyErr_BadInternalCall(); diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c index b10d6fe..99cc7a9 100644 --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -1,3 +1,8 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + + #include /* diff --git a/Modules/_ctypes/callbacks.c b/Modules/_ctypes/callbacks.c index 4baf3aa..c091d88 100644 --- a/Modules/_ctypes/callbacks.c +++ b/Modules/_ctypes/callbacks.c @@ -1,3 +1,7 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + #include "Python.h" #include "compile.h" /* required only for 2.3, as it seems */ #include "frameobject.h" @@ -123,7 +127,9 @@ static void _CallPythonObject(void *mem, PyObject *result; PyObject *arglist = NULL; int nArgs; +#ifdef WITH_THREAD PyGILState_STATE state = PyGILState_Ensure(); +#endif nArgs = PySequence_Length(converters); /* Hm. What to return in case of error? @@ -231,8 +237,9 @@ if (x == NULL) _AddTraceback(what, __FILE__, __LINE__ - 1), PyErr_Print() Py_XDECREF(result); Done: Py_XDECREF(arglist); - +#ifdef WITH_THREAD PyGILState_Release(state); +#endif } static void closure_fcn(ffi_cif *cif, @@ -341,7 +348,9 @@ void init_callbacks_in_module(PyObject *m) static void LoadPython(void) { if (!Py_IsInitialized()) { +#ifdef WITH_THREAD PyEval_InitThreads(); +#endif Py_Initialize(); } } @@ -393,12 +402,18 @@ STDAPI DllGetClassObject(REFCLSID rclsid, LPVOID *ppv) { long result; +#ifdef WITH_THREAD PyGILState_STATE state; +#endif LoadPython(); +#ifdef WITH_THREAD state = PyGILState_Ensure(); +#endif result = Call_GetClassObject(rclsid, riid, ppv); +#ifdef WITH_THREAD PyGILState_Release(state); +#endif return result; } @@ -450,9 +465,13 @@ long Call_CanUnloadNow(void) STDAPI DllCanUnloadNow(void) { long result; +#ifdef WITH_THREAD PyGILState_STATE state = PyGILState_Ensure(); +#endif result = Call_CanUnloadNow(); +#ifdef WITH_THREAD PyGILState_Release(state); +#endif return result; } diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index 8108498..d6f875b 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -1,3 +1,8 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + + /* * History: First version dated from 3/97, derived from my SCMLIB version * for win16. @@ -612,7 +617,9 @@ static int _call_function_pointer(int flags, void *resmem, int argcount) { +#ifdef WITH_THREAD PyThreadState *_save = NULL; /* For Py_BLOCK_THREADS and Py_UNBLOCK_THREADS */ +#endif ffi_cif cif; int cc; #ifdef MS_WIN32 @@ -644,8 +651,10 @@ static int _call_function_pointer(int flags, return -1; } +#ifdef WITH_THREAD if ((flags & FUNCFLAG_PYTHONAPI) == 0) Py_UNBLOCK_THREADS +#endif #ifdef MS_WIN32 #ifndef DONT_USE_SEH __try { @@ -662,8 +671,10 @@ static int _call_function_pointer(int flags, } #endif #endif +#ifdef WITH_THREAD if ((flags & FUNCFLAG_PYTHONAPI) == 0) Py_BLOCK_THREADS +#endif #ifdef MS_WIN32 #ifndef DONT_USE_SEH if (dwExceptionCode) { @@ -804,14 +815,22 @@ GetComError(HRESULT errcode, GUID *riid, IUnknown *pIunk) PyObject *obj; TCHAR *text; + /* We absolutely have to release the GIL during COM method calls, + otherwise we may get a deadlock! + */ +#ifdef WITH_THREAD + Py_BEGIN_ALLOW_THREADS +#endif + hr = pIunk->lpVtbl->QueryInterface(pIunk, &IID_ISupportErrorInfo, (void **)&psei); if (FAILED(hr)) goto failed; + hr = psei->lpVtbl->InterfaceSupportsErrorInfo(psei, riid); psei->lpVtbl->Release(psei); - if (FAILED(hr)) goto failed; + hr = GetErrorInfo(0, &pei); if (hr != S_OK) goto failed; @@ -822,24 +841,27 @@ GetComError(HRESULT errcode, GUID *riid, IUnknown *pIunk) pei->lpVtbl->GetHelpFile(pei, &helpfile); pei->lpVtbl->GetSource(pei, &source); + pei->lpVtbl->Release(pei); + failed: - if (pei) - pei->lpVtbl->Release(pei); +#ifdef WITH_THREAD + Py_END_ALLOW_THREADS +#endif progid = NULL; ProgIDFromCLSID(&guid, &progid); -/* XXX Is COMError derived from WindowsError or not? */ text = FormatError(errcode); + obj = Py_BuildValue( #ifdef _UNICODE - obj = Py_BuildValue("iu(uuuiu)", + "iu(uuuiu)", #else - obj = Py_BuildValue("is(uuuiu)", + "is(uuuiu)", #endif - errcode, - text, - descr, source, helpfile, helpcontext, - progid); + errcode, + text, + descr, source, helpfile, helpcontext, + progid); if (obj) { PyErr_SetObject(ComError, obj); Py_DECREF(obj); @@ -1138,7 +1160,7 @@ call_commethod(PyObject *self, PyObject *args) } static char copy_com_pointer_doc[] = -"CopyComPointer(a, b) -> integer\n"; +"CopyComPointer(src, dst) -> HRESULT value\n"; static PyObject * copy_com_pointer(PyObject *self, PyObject *args) @@ -1444,7 +1466,72 @@ set_conversion_mode(PyObject *self, PyObject *args) } #endif +static PyObject * +resize(PyObject *self, PyObject *args) +{ + CDataObject *obj; + StgDictObject *dict; + Py_ssize_t size; + + if (!PyArg_ParseTuple(args, +#if (PY_VERSION_HEX < 0x02050000) + "Oi:resize", +#else + "On:resize", +#endif + (PyObject *)&obj, &size)) + return NULL; + + dict = PyObject_stgdict((PyObject *)obj); + if (dict == NULL) { + PyErr_SetString(PyExc_TypeError, + "excepted ctypes instance"); + return NULL; + } + if (size < dict->size) { + PyErr_Format(PyExc_ValueError, +#if PY_VERSION_HEX < 0x02050000 + "minimum size is %d", +#else + "minimum size is %zd", +#endif + dict->size); + return NULL; + } + if (obj->b_needsfree == 0) { + PyErr_Format(PyExc_ValueError, + "Memory cannot be resized because this object doesn't own it"); + return NULL; + } + if (size <= sizeof(obj->b_value)) { + /* internal default buffer is large enough */ + obj->b_size = size; + goto done; + } + if (obj->b_size <= sizeof(obj->b_value)) { + /* We are currently using the objects default buffer, but it + isn't large enough any more. */ + void *ptr = PyMem_Malloc(size); + if (ptr == NULL) + return PyErr_NoMemory(); + memset(ptr, 0, size); + memmove(ptr, obj->b_ptr, obj->b_size); + obj->b_ptr = ptr; + obj->b_size = size; + } else { + void * ptr = PyMem_Realloc(obj->b_ptr, size); + if (ptr == NULL) + return PyErr_NoMemory(); + obj->b_ptr = ptr; + obj->b_size = size; + } + done: + Py_INCREF(Py_None); + return Py_None; +} + PyMethodDef module_methods[] = { + {"resize", resize, METH_VARARGS, "Resize the memory buffer of a ctypes instance"}, #ifdef CTYPES_UNICODE {"set_conversion_mode", set_conversion_mode, METH_VARARGS, set_conversion_mode_doc}, #endif diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c index ae0290f..3595b05 100644 --- a/Modules/_ctypes/cfield.c +++ b/Modules/_ctypes/cfield.c @@ -1,3 +1,7 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + #include "Python.h" #include @@ -61,10 +65,10 @@ CField_FromDesc(PyObject *desc, int index, } if (bitsize /* this is a bitfield request */ && *pfield_size /* we have a bitfield open */ -#ifdef MS_WIN32 +#if defined(MS_WIN32) && !defined(__MINGW32__) && dict->size * 8 == *pfield_size /* MSVC */ #else - && dict->size * 8 <= *pfield_size /* GCC */ + && dict->size * 8 <= *pfield_size /* GCC, MINGW */ #endif && (*pbitofs + bitsize) <= *pfield_size) { /* continue bit field */ @@ -120,7 +124,7 @@ CField_FromDesc(PyObject *desc, int index, self->getfunc = getfunc; self->index = index; - Py_XINCREF(proto); + Py_INCREF(proto); self->proto = proto; switch (fieldtype) { @@ -1482,16 +1486,27 @@ P_set(void *ptr, PyObject *value, unsigned size) *(void **)ptr = NULL; _RET(value); } - - v = PyLong_AsVoidPtr(value); - if (PyErr_Occurred()) { - /* prevent the SystemError: bad argument to internal function */ - if (!PyInt_Check(value) && !PyLong_Check(value)) { - PyErr_SetString(PyExc_TypeError, - "cannot be converted to pointer"); - } + + if (!PyInt_Check(value) && !PyLong_Check(value)) { + PyErr_SetString(PyExc_TypeError, + "cannot be converted to pointer"); return NULL; } + +#if SIZEOF_VOID_P <= SIZEOF_LONG + v = (void *)PyInt_AsUnsignedLongMask(value); +#else +#ifndef HAVE_LONG_LONG +# error "PyLong_AsVoidPtr: sizeof(void*) > sizeof(long), but no long long" +#elif SIZEOF_LONG_LONG < SIZEOF_VOID_P +# error "PyLong_AsVoidPtr: sizeof(PY_LONG_LONG) < sizeof(void*)" +#endif + v = (void *)PyInt_AsUnsignedLongLongMask(value); +#endif + + if (PyErr_Occurred()) + return NULL; + *(void **)ptr = v; _RET(value); } diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h index 7988595..303eda3 100644 --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -1,4 +1,6 @@ -/******************************************************************/ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ #if (PY_VERSION_HEX < 0x02050000) typedef int Py_ssize_t; @@ -59,7 +61,7 @@ struct tagCDataObject { Py_ssize_t b_length; /* number of references we need */ Py_ssize_t b_index; /* index of this object into base's b_object list */ - PyObject *b_objects; /* list of references we need to keep */ + PyObject *b_objects; /* dictionary of references we need to keep, or Py_None */ union value b_value; }; @@ -181,6 +183,7 @@ typedef struct { PyObject *proto; /* a type or NULL */ GETFUNC getfunc; /* getter function if proto is NULL */ SETFUNC setfunc; /* setter function if proto is NULL */ + int anonymous; } CFieldObject; /* A subclass of PyDictObject, used as the instance dictionary of ctypes @@ -374,8 +377,8 @@ extern char *conversion_mode_errors; # undef PyUnicode_AsWideChar # define PyUnicode_AsWideChar My_PyUnicode_AsWideChar -extern PyObject *My_PyUnicode_FromWideChar(const wchar_t *, int); -extern int My_PyUnicode_AsWideChar(PyUnicodeObject *, wchar_t *, int); +extern PyObject *My_PyUnicode_FromWideChar(const wchar_t *, Py_ssize_t); +extern int My_PyUnicode_AsWideChar(PyUnicodeObject *, wchar_t *, Py_ssize_t); #endif diff --git a/Modules/_ctypes/ctypes_dlfcn.h b/Modules/_ctypes/ctypes_dlfcn.h index 7f632c5..d8bf904 100644 --- a/Modules/_ctypes/ctypes_dlfcn.h +++ b/Modules/_ctypes/ctypes_dlfcn.h @@ -1,4 +1,6 @@ -/******************************************************************/ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ #ifndef _CTYPES_DLFCN_H_ #define _CTYPES_DLFCN_H_ diff --git a/Modules/_ctypes/libffi/configure b/Modules/_ctypes/libffi/configure index 9fdd654..9eded5a 100755 --- a/Modules/_ctypes/libffi/configure +++ b/Modules/_ctypes/libffi/configure @@ -934,7 +934,7 @@ esac else echo "$as_me: WARNING: no configuration information is in $ac_dir" >&2 fi - cd $ac_popdir + cd "$ac_popdir" done fi @@ -1973,8 +1973,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -2032,8 +2031,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -2149,8 +2147,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -2204,8 +2201,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -2250,8 +2246,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -2295,8 +2290,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -2623,8 +2617,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -2794,8 +2787,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -2862,8 +2854,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -3047,8 +3038,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_link\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -3111,8 +3101,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -3290,8 +3279,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_link\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -3408,8 +3396,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -3483,6 +3470,12 @@ fi TARGETDIR="unknown" case "$host" in +mips*-*-openbsd*) TARGET=MIPS; TARGETDIR=mips;; +sparc-*-openbsd*) TARGET=SPARC; TARGETDIR=sparc;; +sparc64-*-openbsd*) TARGET=SPARC; TARGETDIR=sparc;; +alpha*-*-openbsd*) TARGET=ALPHA; TARGETDIR=alpha;; +m68k-*-openbsd*) TARGET=M68K; TARGETDIR=m68k;; +powerpc-*-openbsd*) TARGET=POWERPC; TARGETDIR=powerpc;; i*86-*-darwin*) TARGET=X86_DARWIN; TARGETDIR=x86;; i*86-*-linux*) TARGET=X86; TARGETDIR=x86;; i*86-*-gnu*) TARGET=X86; TARGETDIR=x86;; @@ -3575,8 +3568,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -3777,8 +3769,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_link\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -3841,8 +3832,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_link\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -3923,8 +3913,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_link\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4065,8 +4054,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_link\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4202,8 +4190,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4265,8 +4252,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4306,8 +4292,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4363,8 +4348,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4404,8 +4388,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4469,8 +4452,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4501,10 +4483,8 @@ See \`config.log' for more details." >&2;} esac else if test "$cross_compiling" = yes; then - { { echo "$as_me:$LINENO: error: cannot run test program while cross compiling -See \`config.log' for more details." >&5 -echo "$as_me: error: cannot run test program while cross compiling -See \`config.log' for more details." >&2;} + { { echo "$as_me:$LINENO: error: internal error: not reached in cross-compile" >&5 +echo "$as_me: error: internal error: not reached in cross-compile" >&2;} { (exit 1); exit 1; }; } else cat >conftest.$ac_ext <<_ACEOF @@ -4616,8 +4596,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4679,8 +4658,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4720,8 +4698,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4777,8 +4754,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4818,8 +4794,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4883,8 +4858,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -4915,10 +4889,8 @@ See \`config.log' for more details." >&2;} esac else if test "$cross_compiling" = yes; then - { { echo "$as_me:$LINENO: error: cannot run test program while cross compiling -See \`config.log' for more details." >&5 -echo "$as_me: error: cannot run test program while cross compiling -See \`config.log' for more details." >&2;} + { { echo "$as_me:$LINENO: error: internal error: not reached in cross-compile" >&5 +echo "$as_me: error: internal error: not reached in cross-compile" >&2;} { (exit 1); exit 1; }; } else cat >conftest.$ac_ext <<_ACEOF @@ -5048,8 +5020,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -5091,8 +5062,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -5149,8 +5119,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -5282,8 +5251,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_link\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -5349,8 +5317,7 @@ if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } && - { ac_try='test -z "$ac_c_werror_flag" - || test ! -s conftest.err' + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 (eval $ac_try) 2>&5 ac_status=$? @@ -6309,11 +6276,6 @@ esac - if test x"$ac_file" != x-; then - { echo "$as_me:$LINENO: creating $ac_file" >&5 -echo "$as_me: creating $ac_file" >&6;} - rm -f "$ac_file" - fi # Let's still pretend it is `configure' which instantiates (i.e., don't # use $as_me), people would be surprised to read: # /* config.h. Generated by config.status. */ @@ -6352,6 +6314,12 @@ echo "$as_me: error: cannot find input file: $f" >&2;} fi;; esac done` || { (exit 1); exit 1; } + + if test x"$ac_file" != x-; then + { echo "$as_me:$LINENO: creating $ac_file" >&5 +echo "$as_me: creating $ac_file" >&6;} + rm -f "$ac_file" + fi _ACEOF cat >>$CONFIG_STATUS <<_ACEOF sed "$ac_vpsub diff --git a/Modules/_ctypes/libffi/configure.ac b/Modules/_ctypes/libffi/configure.ac index a0d7513..384959e 100644 --- a/Modules/_ctypes/libffi/configure.ac +++ b/Modules/_ctypes/libffi/configure.ac @@ -21,6 +21,12 @@ AC_FUNC_MMAP_BLACKLIST TARGETDIR="unknown" case "$host" in +mips*-*-openbsd*) TARGET=MIPS; TARGETDIR=mips;; +sparc-*-openbsd*) TARGET=SPARC; TARGETDIR=sparc;; +sparc64-*-openbsd*) TARGET=SPARC; TARGETDIR=sparc;; +alpha*-*-openbsd*) TARGET=ALPHA; TARGETDIR=alpha;; +m68k-*-openbsd*) TARGET=M68K; TARGETDIR=m68k;; +powerpc-*-openbsd*) TARGET=POWERPC; TARGETDIR=powerpc;; i*86-*-darwin*) TARGET=X86_DARWIN; TARGETDIR=x86;; i*86-*-linux*) TARGET=X86; TARGETDIR=x86;; i*86-*-gnu*) TARGET=X86; TARGETDIR=x86;; diff --git a/Modules/_ctypes/libffi/src/x86/darwin.S b/Modules/_ctypes/libffi/src/x86/darwin.S index c5e55b5..d91bdc0 100644 --- a/Modules/_ctypes/libffi/src/x86/darwin.S +++ b/Modules/_ctypes/libffi/src/x86/darwin.S @@ -35,6 +35,13 @@ #include #include +#ifdef PyObjC_STRICT_DEBUGGING + /* XXX: Debugging of stack alignment, to be removed */ +#define ASSERT_STACK_ALIGNED movdqa -16(%esp), %xmm0 +#else +#define ASSERT_STACK_ALIGNED +#endif + .text .globl _ffi_prep_args @@ -47,30 +54,41 @@ _ffi_call_SYSV: pushl %ebp .LCFI0: movl %esp,%ebp + subl $8,%esp + ASSERT_STACK_ALIGNED .LCFI1: /* Make room for all of the new args. */ movl 16(%ebp),%ecx subl %ecx,%esp + ASSERT_STACK_ALIGNED + movl %esp,%eax /* Place all of the ffi_prep_args in position */ + subl $8,%esp pushl 12(%ebp) pushl %eax call *8(%ebp) + ASSERT_STACK_ALIGNED + /* Return stack to previous state and call the function */ - addl $8,%esp + addl $16,%esp - call *28(%ebp) + ASSERT_STACK_ALIGNED - /* Remove the space we pushed for the args */ + call *28(%ebp) + + /* XXX: return returns return with 'ret $4', that upsets the stack! */ movl 16(%ebp),%ecx addl %ecx,%esp + /* Load %ecx with the return type code */ movl 20(%ebp),%ecx + /* If the return value pointer is NULL, assume no return value. */ cmpl $0,24(%ebp) jne retint @@ -117,17 +135,47 @@ retlongdouble: retint64: cmpl $FFI_TYPE_SINT64,%ecx - jne retstruct + jne retstruct1b /* Load %ecx with the pointer to storage for the return value */ movl 24(%ebp),%ecx movl %eax,0(%ecx) movl %edx,4(%ecx) + jmp epilogue + +retstruct1b: + cmpl $FFI_TYPE_SINT8,%ecx + jne retstruct2b + movl 24(%ebp),%ecx + movb %al,0(%ecx) + jmp epilogue + +retstruct2b: + cmpl $FFI_TYPE_SINT16,%ecx + jne retstruct + movl 24(%ebp),%ecx + movw %ax,0(%ecx) + jmp epilogue retstruct: + cmpl $FFI_TYPE_STRUCT,%ecx + jne noretval /* Nothing to do! */ + subl $4,%esp + + ASSERT_STACK_ALIGNED + + addl $8,%esp + movl %ebp, %esp + popl %ebp + ret + noretval: epilogue: + ASSERT_STACK_ALIGNED + addl $8, %esp + + movl %ebp,%esp popl %ebp ret diff --git a/Modules/_ctypes/libffi/src/x86/ffi_darwin.c b/Modules/_ctypes/libffi/src/x86/ffi_darwin.c index 4f82b3a..c9742d8 100644 --- a/Modules/_ctypes/libffi/src/x86/ffi_darwin.c +++ b/Modules/_ctypes/libffi/src/x86/ffi_darwin.c @@ -140,7 +140,7 @@ ffi_status ffi_prep_cif_machdep(ffi_cif *cif) switch (cif->rtype->type) { case FFI_TYPE_VOID: -#if !defined(X86_WIN32) +#if !defined(X86_WIN32) && !defined(X86_DARWIN) case FFI_TYPE_STRUCT: #endif case FFI_TYPE_SINT64: @@ -154,7 +154,7 @@ ffi_status ffi_prep_cif_machdep(ffi_cif *cif) cif->flags = FFI_TYPE_SINT64; break; -#if defined X86_WIN32 +#if defined(X86_WIN32) || defined(X86_DARWIN) case FFI_TYPE_STRUCT: if (cif->rtype->size == 1) @@ -186,10 +186,11 @@ ffi_status ffi_prep_cif_machdep(ffi_cif *cif) } /* Darwin: The stack needs to be aligned to a multiple of 16 bytes */ -#if 0 +#if 1 cif->bytes = (cif->bytes + 15) & ~0xF; #endif + return FFI_OK; } @@ -221,7 +222,6 @@ void ffi_call(/*@dependent@*/ ffi_cif *cif, /*@dependent@*/ void **avalue) { extended_cif ecif; - int flags; ecif.cif = cif; ecif.avalue = avalue; @@ -238,20 +238,6 @@ void ffi_call(/*@dependent@*/ ffi_cif *cif, else ecif.rvalue = rvalue; - flags = cif->flags; - if (flags == FFI_TYPE_STRUCT) { - if (cif->rtype->size == 8) { - flags = FFI_TYPE_SINT64; - } else if (cif->rtype->size == 4) { - flags = FFI_TYPE_INT; - } else if (cif->rtype->size == 2) { - flags = FFI_TYPE_INT; - } else if (cif->rtype->size == 1) { - flags = FFI_TYPE_INT; - } - } - - switch (cif->abi) { case FFI_SYSV: @@ -260,8 +246,8 @@ void ffi_call(/*@dependent@*/ ffi_cif *cif, * block is a multiple of 16. Then add 8 to compensate for local variables * in ffi_call_SYSV. */ - ffi_call_SYSV(ffi_prep_args, &ecif, ALIGN(cif->bytes, 16) +8, - flags, ecif.rvalue, fn); + ffi_call_SYSV(ffi_prep_args, &ecif, cif->bytes, + cif->flags, ecif.rvalue, fn); /*@=usedef@*/ break; #ifdef X86_WIN32 @@ -281,8 +267,6 @@ void ffi_call(/*@dependent@*/ ffi_cif *cif, /** private members **/ -static void ffi_prep_incoming_args_SYSV (char *stack, void **ret, - void** args, ffi_cif* cif); static void ffi_closure_SYSV (ffi_closure *) __attribute__ ((regparm(1))); #if !FFI_NO_RAW_API @@ -290,6 +274,48 @@ static void ffi_closure_raw_SYSV (ffi_raw_closure *) __attribute__ ((regparm(1))); #endif +/*@-exportheader@*/ +static inline void +ffi_prep_incoming_args_SYSV(char *stack, void **rvalue, + void **avalue, ffi_cif *cif) +/*@=exportheader@*/ +{ + register unsigned int i; + register void **p_argv; + register char *argp; + register ffi_type **p_arg; + + argp = stack; + + if (retval_on_stack(cif->rtype)) { + *rvalue = *(void **) argp; + argp += 4; + } + + p_argv = avalue; + + for (i = cif->nargs, p_arg = cif->arg_types; (i != 0); i--, p_arg++) + { + size_t z; + + /* Align if necessary */ + if ((sizeof(int) - 1) & (unsigned) argp) { + argp = (char *) ALIGN(argp, sizeof(int)); + } + + z = (*p_arg)->size; + + /* because we're little endian, this is what it turns into. */ + + *p_argv = (void*) argp; + + p_argv++; + argp += z; + } + + return; +} + /* This function is jumped to by the trampoline */ static void @@ -302,10 +328,10 @@ ffi_closure_SYSV (closure) // our various things... ffi_cif *cif; void **arg_area; - unsigned short rtype; void *resp = (void*)&res; void *args = __builtin_dwarf_cfa (); + cif = closure->cif; arg_area = (void**) alloca (cif->nargs * sizeof (void*)); @@ -319,94 +345,52 @@ ffi_closure_SYSV (closure) (closure->fun) (cif, resp, arg_area, closure->user_data); - rtype = cif->flags; - - if (!retval_on_stack(cif->rtype) && cif->flags == FFI_TYPE_STRUCT) { - if (cif->rtype->size == 8) { - rtype = FFI_TYPE_SINT64; - } else { - rtype = FFI_TYPE_INT; - } - } - /* now, do a generic return based on the value of rtype */ - if (rtype == FFI_TYPE_INT) + if (cif->flags == FFI_TYPE_INT) { asm ("movl (%0),%%eax" : : "r" (resp) : "eax"); } - else if (rtype == FFI_TYPE_FLOAT) + else if (cif->flags == FFI_TYPE_FLOAT) { asm ("flds (%0)" : : "r" (resp) : "st" ); } - else if (rtype == FFI_TYPE_DOUBLE) + else if (cif->flags == FFI_TYPE_DOUBLE) { asm ("fldl (%0)" : : "r" (resp) : "st", "st(1)" ); } - else if (rtype == FFI_TYPE_LONGDOUBLE) + else if (cif->flags == FFI_TYPE_LONGDOUBLE) { asm ("fldt (%0)" : : "r" (resp) : "st", "st(1)" ); } - else if (rtype == FFI_TYPE_SINT64) + else if (cif->flags == FFI_TYPE_SINT64) { asm ("movl 0(%0),%%eax;" "movl 4(%0),%%edx" : : "r"(resp) : "eax", "edx"); } -#ifdef X86_WIN32 - else if (rtype == FFI_TYPE_SINT8) /* 1-byte struct */ +#if defined(X86_WIN32) || defined(X86_DARWIN) + else if (cif->flags == FFI_TYPE_SINT8) /* 1-byte struct */ { asm ("movsbl (%0),%%eax" : : "r" (resp) : "eax"); } - else if (rtype == FFI_TYPE_SINT16) /* 2-bytes struct */ + else if (cif->flags == FFI_TYPE_SINT16) /* 2-bytes struct */ { asm ("movswl (%0),%%eax" : : "r" (resp) : "eax"); } #endif -} -/*@-exportheader@*/ -static void -ffi_prep_incoming_args_SYSV(char *stack, void **rvalue, - void **avalue, ffi_cif *cif) -/*@=exportheader@*/ -{ - register unsigned int i; - register void **p_argv; - register char *argp; - register ffi_type **p_arg; - - argp = stack; - - if (retval_on_stack(cif->rtype)) { - *rvalue = *(void **) argp; - argp += 4; - } - - p_argv = avalue; - - for (i = cif->nargs, p_arg = cif->arg_types; (i != 0); i--, p_arg++) + else if (cif->flags == FFI_TYPE_STRUCT) { - size_t z; - - /* Align if necessary */ - if ((sizeof(int) - 1) & (unsigned) argp) { - argp = (char *) ALIGN(argp, sizeof(int)); - } - - z = (*p_arg)->size; - - /* because we're little endian, this is what it turns into. */ - - *p_argv = (void*) argp; - - p_argv++; - argp += z; + asm ("lea -8(%ebp),%esp;" + "pop %esi;" + "pop %edi;" + "pop %ebp;" + "ret $4"); } - - return; } + /* How to make a trampoline. Derived from gcc/config/i386/i386.c. */ #define FFI_INIT_TRAMPOLINE(TRAMP,FUN,CTX) \ diff --git a/Modules/_ctypes/libffi_msvc/ffi.c b/Modules/_ctypes/libffi_msvc/ffi.c index e5600b2..9af6b71 100644 --- a/Modules/_ctypes/libffi_msvc/ffi.c +++ b/Modules/_ctypes/libffi_msvc/ffi.c @@ -227,11 +227,7 @@ ffi_closure_SYSV (ffi_closure *closure, int *argp) void **arg_area; unsigned short rtype; void *resp = (void*)&res; -//#ifdef _MSC_VER void *args = &argp[1]; -//#else -// void *args = __builtin_dwarf_cfa (); -//#endif cif = closure->cif; arg_area = (void**) alloca (cif->nargs * sizeof (void*)); @@ -353,10 +349,6 @@ ffi_prep_incoming_args_SYSV(char *stack, void **rvalue, /* How to make a trampoline. Derived from gcc/config/i386/i386.c. */ -/* MOV EDX, ESP is 0x8b 0xd4 */ - -//#ifdef _MSC_VER - #define FFI_INIT_TRAMPOLINE(TRAMP,FUN,CTX,BYTES) \ { unsigned char *__tramp = (unsigned char*)(TRAMP); \ unsigned int __fun = (unsigned int)(FUN); \ @@ -365,26 +357,13 @@ ffi_prep_incoming_args_SYSV(char *stack, void **rvalue, *(unsigned char*) &__tramp[0] = 0xb9; \ *(unsigned int*) &__tramp[1] = __ctx; /* mov ecx, __ctx */ \ *(unsigned char*) &__tramp[5] = 0x8b; \ - *(unsigned char*) &__tramp[6] = 0xd4; \ + *(unsigned char*) &__tramp[6] = 0xd4; /* mov edx, esp */ \ *(unsigned char*) &__tramp[7] = 0xe8; \ *(unsigned int*) &__tramp[8] = __dis; /* call __fun */ \ *(unsigned char*) &__tramp[12] = 0xC2; /* ret BYTES */ \ *(unsigned short*) &__tramp[13] = BYTES; \ } -//#else -//#define FFI_INIT_TRAMPOLINE(TRAMP,FUN,CTX,BYTES) \ -//({ unsigned char *__tramp = (unsigned char*)(TRAMP); \ -// unsigned int __fun = (unsigned int)(FUN); \ -// unsigned int __ctx = (unsigned int)(CTX); \ -// unsigned int __dis = __fun - ((unsigned int) __tramp + FFI_TRAMPOLINE_SIZE); \ -// *(unsigned char*) &__tramp[0] = 0xb8; \ -// *(unsigned int*) &__tramp[1] = __ctx; /* movl __ctx, %eax */ \ -// *(unsigned char *) &__tramp[5] = 0xe9; \ -// *(unsigned int*) &__tramp[6] = __dis; /* jmp __fun */ \ -// }) -//#endif - /* the cif must already be prep'ed */ ffi_status diff --git a/Modules/_ctypes/libffi_msvc/fficonfig.h b/Modules/_ctypes/libffi_msvc/fficonfig.h index 25b6305..c14f653 100644 --- a/Modules/_ctypes/libffi_msvc/fficonfig.h +++ b/Modules/_ctypes/libffi_msvc/fficonfig.h @@ -3,8 +3,10 @@ /* fficonfig.h. Generated automatically by configure. */ /* fficonfig.h.in. Generated automatically from configure.in by autoheader. */ -/* Defines for MSVC */ +/* Define this for MSVC, but not for mingw32! */ +#ifdef _MSC_VER #define __attribute__(x) /* */ +#endif #define alloca _alloca /*----------------------------------------------------------------*/ diff --git a/Modules/_ctypes/libffi_msvc/ffitarget.h b/Modules/_ctypes/libffi_msvc/ffitarget.h index 57d275b..78c0c37 100644 --- a/Modules/_ctypes/libffi_msvc/ffitarget.h +++ b/Modules/_ctypes/libffi_msvc/ffitarget.h @@ -71,11 +71,7 @@ typedef enum ffi_abi { #define FFI_TRAMPOLINE_SIZE 24 #define FFI_NATIVE_RAW_API 0 #else -#ifdef _MSC_VER -# define FFI_TRAMPOLINE_SIZE 15 -#else -# define FFI_TRAMPOLINE_SIZE 10 -#endif +#define FFI_TRAMPOLINE_SIZE 15 #define FFI_NATIVE_RAW_API 1 /* x86 has native raw api support */ #endif diff --git a/Modules/_ctypes/libffi_msvc/mingwin32.S b/Modules/_ctypes/libffi_msvc/mingwin32.S deleted file mode 100644 index e71f2b2..0000000 --- a/Modules/_ctypes/libffi_msvc/mingwin32.S +++ /dev/null @@ -1,228 +0,0 @@ -/* ----------------------------------------------------------------------- - win32.S - Copyright (c) 1996, 1998, 2001, 2002 Red Hat, Inc. - Copyright (c) 2001 John Beniton - Copyright (c) 2002 Ranjit Mathew - - - X86 Foreign Function Interface - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - ``Software''), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL CYGNUS SOLUTIONS BE LIABLE FOR ANY CLAIM, DAMAGES OR - OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - OTHER DEALINGS IN THE SOFTWARE. - ----------------------------------------------------------------------- */ - -#define LIBFFI_ASM -#include -#include - -.text - -.globl ffi_prep_args - - # This assumes we are using gas. - .balign 16 -.globl _ffi_call_SYSV - -_ffi_call_SYSV: - pushl %ebp - movl %esp,%ebp - - # Make room for all of the new args. - movl 16(%ebp),%ecx - subl %ecx,%esp - - movl %esp,%eax - - # Place all of the ffi_prep_args in position - pushl 12(%ebp) - pushl %eax - call *8(%ebp) - - # Return stack to previous state and call the function - addl $8,%esp - - # FIXME: Align the stack to a 128-bit boundary to avoid - # potential performance hits. - - call *28(%ebp) - - # Remove the space we pushed for the args - movl 16(%ebp),%ecx - addl %ecx,%esp - - # Load %ecx with the return type code - movl 20(%ebp),%ecx - - # If the return value pointer is NULL, assume no return value. - cmpl $0,24(%ebp) - jne retint - - # Even if there is no space for the return value, we are - # obliged to handle floating-point values. - cmpl $2,%ecx # Float_type - jne noretval - fstp %st(0) - - jmp epilogue - -retint: - cmpl $1,%ecx # Int_type - jne retfloat - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - jmp epilogue - -retfloat: - cmpl $2,%ecx # Float_type - jne retdouble - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstps (%ecx) - jmp epilogue - -retdouble: - cmpl $3,%ecx # Double_type - jne retlongdouble - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstpl (%ecx) - jmp epilogue - -retlongdouble: - cmpl $4,%ecx # Longdouble_type - jne retint64 - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstpt (%ecx) - jmp epilogue - -retint64: - cmpl $12,%ecx # SINT64_type - jne retstruct - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - movl %edx,4(%ecx) - -retstruct: - # Nothing to do! - -noretval: -epilogue: - movl %ebp,%esp - popl %ebp - ret - -.ffi_call_SYSV_end: - - # This assumes we are using gas. - .balign 16 -.globl _ffi_call_STDCALL - -_ffi_call_STDCALL: - pushl %ebp - movl %esp,%ebp - - # Make room for all of the new args. - movl 16(%ebp),%ecx - subl %ecx,%esp - - movl %esp,%eax - - # Place all of the ffi_prep_args in position - pushl 12(%ebp) - pushl %eax - call *8(%ebp) - - # Return stack to previous state and call the function - addl $8,%esp - - # FIXME: Align the stack to a 128-bit boundary to avoid - # potential performance hits. - - call *28(%ebp) - - # stdcall functions pop arguments off the stack themselves - - # Load %ecx with the return type code - movl 20(%ebp),%ecx - - # If the return value pointer is NULL, assume no return value. - cmpl $0,24(%ebp) - jne sc_retint - - # Even if there is no space for the return value, we are - # obliged to handle floating-point values. - cmpl $2,%ecx # Float_type - jne sc_noretval - fstp %st(0) - - jmp sc_epilogue - -sc_retint: - cmpl $1,%ecx # Int_type - jne sc_retfloat - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - jmp sc_epilogue - -sc_retfloat: - cmpl $2,%ecx # Float_type - jne sc_retdouble - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstps (%ecx) - jmp sc_epilogue - -sc_retdouble: - cmpl $2,%ecx # Double_type - jne sc_retlongdouble - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstpl (%ecx) - jmp sc_epilogue - -sc_retlongdouble: - cmpl $4,%ecx # Longdouble_type - jne sc_retint64 - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - fstpt (%ecx) - jmp sc_epilogue - -sc_retint64: - cmpl $12,%ecx # SINT64_Type - jne sc_retstruct - # Load %ecx with the pointer to storage for the return value - movl 24(%ebp),%ecx - movl %eax,0(%ecx) - movl %edx,4(%ecx) - -sc_retstruct: - # Nothing to do! - -sc_noretval: -sc_epilogue: - movl %ebp,%esp - popl %ebp - ret - -.ffi_call_STDCALL_end: - diff --git a/Modules/_ctypes/malloc_closure.c b/Modules/_ctypes/malloc_closure.c index 29e9f4c..4cd5dd6 100644 --- a/Modules/_ctypes/malloc_closure.c +++ b/Modules/_ctypes/malloc_closure.c @@ -1,3 +1,7 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + #include #include #ifdef MS_WIN32 diff --git a/Modules/_ctypes/stgdict.c b/Modules/_ctypes/stgdict.c index c1c30f1..b4d8196 100644 --- a/Modules/_ctypes/stgdict.c +++ b/Modules/_ctypes/stgdict.c @@ -1,3 +1,7 @@ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + #include "Python.h" #include #ifdef MS_WIN32 @@ -128,42 +132,148 @@ PyType_stgdict(PyObject *obj) if (!PyType_Check(obj)) return NULL; type = (PyTypeObject *)obj; - if (!type->tp_dict || !StgDict_Check(type->tp_dict)) + if (!type->tp_dict || !StgDict_CheckExact(type->tp_dict)) return NULL; return (StgDictObject *)type->tp_dict; } /* May return NULL, but does not set an exception! */ +/* + This function should be as fast as possible, so we don't call PyType_stgdict + above but inline the code, and avoid the PyType_Check(). +*/ StgDictObject * PyObject_stgdict(PyObject *self) { - return PyType_stgdict((PyObject *)self->ob_type); + PyTypeObject *type = self->ob_type; + if (!type->tp_dict || !StgDict_CheckExact(type->tp_dict)) + return NULL; + return (StgDictObject *)type->tp_dict; } -#if 0 -/* work in progress: anonymous structure fields */ -int -GetFields(PyObject *desc, int *pindex, int *psize, int *poffset, int *palign, int pack); +/* descr is the descriptor for a field marked as anonymous. Get all the + _fields_ descriptors from descr->proto, create new descriptors with offset + and index adjusted, and stuff them into type. + */ +static int +MakeFields(PyObject *type, CFieldObject *descr, + Py_ssize_t index, Py_ssize_t offset) +{ + Py_ssize_t i; + PyObject *fields; + PyObject *fieldlist; + fields = PyObject_GetAttrString(descr->proto, "_fields_"); + if (fields == NULL) + return -1; + fieldlist = PySequence_Fast(fields, "_fields_ must be a sequence"); + Py_DECREF(fields); + if (fieldlist == NULL) + return -1; + + for (i = 0; i < PySequence_Fast_GET_SIZE(fieldlist); ++i) { + PyObject *pair = PySequence_Fast_GET_ITEM(fieldlist, i); /* borrowed */ + PyObject *fname, *ftype; + CFieldObject *fdescr; + CFieldObject *new_descr; + /* Convert to PyArg_UnpackTuple... */ + if (!PyArg_ParseTuple(pair, "OO", &fname, &ftype)) { + Py_DECREF(fieldlist); + return -1; + } + fdescr = (CFieldObject *)PyObject_GetAttr(descr->proto, fname); + if (fdescr == NULL) { + Py_DECREF(fieldlist); + return -1; + } + if (fdescr->ob_type != &CField_Type) { + PyErr_SetString(PyExc_TypeError, "unexpected type"); + Py_DECREF(fdescr); + Py_DECREF(fieldlist); + return -1; + } + if (fdescr->anonymous) { + int rc = MakeFields(type, fdescr, + index + fdescr->index, + offset + fdescr->offset); + Py_DECREF(fdescr); + if (rc == -1) { + Py_DECREF(fieldlist); + return -1; + } + continue; + } + new_descr = (CFieldObject *)PyObject_CallObject((PyObject *)&CField_Type, NULL); + assert(new_descr->ob_type == &CField_Type); + if (new_descr == NULL) { + Py_DECREF(fdescr); + Py_DECREF(fieldlist); + return -1; + } + new_descr->size = fdescr->size; + new_descr->offset = fdescr->offset + offset; + new_descr->index = fdescr->index + index; + new_descr->proto = fdescr->proto; + Py_XINCREF(new_descr->proto); + new_descr->getfunc = fdescr->getfunc; + new_descr->setfunc = fdescr->setfunc; + + Py_DECREF(fdescr); + + if (-1 == PyObject_SetAttr(type, fname, (PyObject *)new_descr)) { + Py_DECREF(fieldlist); + Py_DECREF(new_descr); + return -1; + } + Py_DECREF(new_descr); + } + Py_DECREF(fieldlist); + return 0; +} + +/* Iterate over the names in the type's _anonymous_ attribute, if present, + */ +static int +MakeAnonFields(PyObject *type) { - int i; - PyObject *tuples = PyObject_GetAttrString(desc, "_fields_"); - if (tuples == NULL) + PyObject *anon; + PyObject *anon_names; + Py_ssize_t i; + + anon = PyObject_GetAttrString(type, "_anonymous_"); + if (anon == NULL) { + PyErr_Clear(); + return 0; + } + anon_names = PySequence_Fast(anon, "_anonymous_ must be a sequence"); + Py_DECREF(anon); + if (anon_names == NULL) return -1; - if (!PyTuple_Check(tuples)) - return -1; /* leak */ - for (i = 0; i < PyTuple_GET_SIZE(tuples); ++i) { - char *fname; - PyObject *dummy; - CFieldObject *field; - PyObject *pair = PyTuple_GET_ITEM(tuples, i); - if (!PyArg_ParseTuple(pair, "sO", &fname, &dummy)) - return -1; /* leak */ - field = PyObject_GetAttrString(desc, fname); - Py_DECREF(field); + + for (i = 0; i < PySequence_Fast_GET_SIZE(anon_names); ++i) { + PyObject *fname = PySequence_Fast_GET_ITEM(anon_names, i); /* borrowed */ + CFieldObject *descr = (CFieldObject *)PyObject_GetAttr(type, fname); + if (descr == NULL) { + Py_DECREF(anon_names); + return -1; + } + assert(descr->ob_type == &CField_Type); + descr->anonymous = 1; + + /* descr is in the field descriptor. */ + if (-1 == MakeFields(type, (CFieldObject *)descr, + ((CFieldObject *)descr)->index, + ((CFieldObject *)descr)->offset)) { + Py_DECREF(descr); + Py_DECREF(anon_names); + return -1; + } + Py_DECREF(descr); } + + Py_DECREF(anon_names); + return 0; } -#endif /* Retrive the (optional) _pack_ attribute from a type, the _fields_ attribute, @@ -366,5 +476,5 @@ StructUnionType_update_stgdict(PyObject *type, PyObject *fields, int isStruct) stgdict->size = size; stgdict->align = total_align; stgdict->length = len; /* ADD ffi_ofs? */ - return 0; + return MakeAnonFields(type); } diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c index 71d7a69..ec55c8e 100644 --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -43,8 +43,8 @@ unsupported functions: del_curterm delscreen dupwin inchnstr inchstr innstr keyok mcprint mvaddchnstr mvaddchstr mvchgat mvcur mvinchnstr mvinchstr mvinnstr mmvwaddchnstr mvwaddchstr mvwchgat - mvwgetnstr mvwinchnstr mvwinchstr mvwinnstr newterm - resizeterm restartterm ripoffline scr_dump + mvwinchnstr mvwinchstr mvwinnstr newterm + restartterm ripoffline scr_dump scr_init scr_restore scr_set scrl set_curterm set_term setterm tgetent tgetflag tgetnum tgetstr tgoto timeout tputs vidattr vidputs waddchnstr waddchstr wchgat @@ -819,14 +819,17 @@ PyCursesWindow_GetStr(PyCursesWindowObject *self, PyObject *args) if (!PyArg_ParseTuple(args,"ii;y,x",&y,&x)) return NULL; Py_BEGIN_ALLOW_THREADS +#ifdef STRICT_SYSV_CURSES + rtn2 = wmove(self->win,y,x)==ERR ? ERR : wgetnstr(self->win, rtn, 1023); +#else rtn2 = mvwgetnstr(self->win,y,x,rtn, 1023); +#endif Py_END_ALLOW_THREADS break; case 3: if (!PyArg_ParseTuple(args,"iii;y,x,n", &y, &x, &n)) return NULL; #ifdef STRICT_SYSV_CURSES - /* Untested */ Py_BEGIN_ALLOW_THREADS rtn2 = wmove(self->win,y,x)==ERR ? ERR : wgetnstr(self->win, rtn, MIN(n, 1023)); @@ -838,7 +841,7 @@ PyCursesWindow_GetStr(PyCursesWindowObject *self, PyObject *args) #endif break; default: - PyErr_SetString(PyExc_TypeError, "getstr requires 0 to 2 arguments"); + PyErr_SetString(PyExc_TypeError, "getstr requires 0 to 3 arguments"); return NULL; } if (rtn2 == ERR) @@ -1950,6 +1953,29 @@ PyCurses_IntrFlush(PyObject *self, PyObject *args) return PyCursesCheckERR(intrflush(NULL,ch), "intrflush"); } +#ifdef HAVE_CURSES_IS_TERM_RESIZED +static PyObject * +PyCurses_Is_Term_Resized(PyObject *self, PyObject *args) +{ + int lines; + int columns; + int result; + + PyCursesInitialised + + if (!PyArg_ParseTuple(args,"ii:is_term_resized", &lines, &columns)) + return NULL; + result = is_term_resized(lines, columns); + if (result == TRUE) { + Py_INCREF(Py_True); + return Py_True; + } else { + Py_INCREF(Py_False); + return Py_False; + } +} +#endif /* HAVE_CURSES_IS_TERM_RESIZED */ + #if !defined(__NetBSD__) static PyObject * PyCurses_KeyName(PyObject *self, PyObject *args) @@ -2170,6 +2196,39 @@ PyCurses_QiFlush(PyObject *self, PyObject *args) } } +#ifdef HAVE_CURSES_RESIZETERM +static PyObject * +PyCurses_ResizeTerm(PyObject *self, PyObject *args) +{ + int lines; + int columns; + + PyCursesInitialised + + if (!PyArg_ParseTuple(args,"ii:resizeterm", &lines, &columns)) + return NULL; + + return PyCursesCheckERR(resizeterm(lines, columns), "resizeterm"); +} + +#endif + +#ifdef HAVE_CURSES_RESIZE_TERM +static PyObject * +PyCurses_Resize_Term(PyObject *self, PyObject *args) +{ + int lines; + int columns; + + PyCursesInitialised + + if (!PyArg_ParseTuple(args,"ii:resize_term", &lines, &columns)) + return NULL; + + return PyCursesCheckERR(resize_term(lines, columns), "resize_term"); +} +#endif /* HAVE_CURSES_RESIZE_TERM */ + static PyObject * PyCurses_setsyx(PyObject *self, PyObject *args) { @@ -2414,6 +2473,9 @@ static PyMethodDef PyCurses_methods[] = { {"initscr", (PyCFunction)PyCurses_InitScr, METH_NOARGS}, {"intrflush", (PyCFunction)PyCurses_IntrFlush, METH_VARARGS}, {"isendwin", (PyCFunction)PyCurses_isendwin, METH_NOARGS}, +#ifdef HAVE_CURSES_IS_TERM_RESIZED + {"is_term_resized", (PyCFunction)PyCurses_Is_Term_Resized, METH_VARARGS}, +#endif #if !defined(__NetBSD__) {"keyname", (PyCFunction)PyCurses_KeyName, METH_VARARGS}, #endif @@ -2441,6 +2503,12 @@ static PyMethodDef PyCurses_methods[] = { {"reset_prog_mode", (PyCFunction)PyCurses_reset_prog_mode, METH_NOARGS}, {"reset_shell_mode", (PyCFunction)PyCurses_reset_shell_mode, METH_NOARGS}, {"resetty", (PyCFunction)PyCurses_resetty, METH_NOARGS}, +#ifdef HAVE_CURSES_RESIZETERM + {"resizeterm", (PyCFunction)PyCurses_ResizeTerm, METH_VARARGS}, +#endif +#ifdef HAVE_CURSES_RESIZE_TERM + {"resize_term", (PyCFunction)PyCurses_Resize_Term, METH_VARARGS}, +#endif {"savetty", (PyCFunction)PyCurses_savetty, METH_NOARGS}, {"setsyx", (PyCFunction)PyCurses_setsyx, METH_VARARGS}, {"setupterm", (PyCFunction)PyCurses_setupterm, diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index dda5042..fe30f30 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -717,7 +717,8 @@ element_deepcopy(ElementObject* self, PyObject* args) LOCAL(int) checkpath(PyObject* tag) { - int i, check = 1; + Py_ssize_t i; + int check = 1; /* check if a tag contains an xpath character */ @@ -1180,7 +1181,7 @@ static int element_setslice(PyObject* self_, Py_ssize_t start, Py_ssize_t end, PyObject* item) { ElementObject* self = (ElementObject*) self_; - int i, new, old; + Py_ssize_t i, new, old; PyObject* recycle = NULL; if (!self->extra) @@ -1426,7 +1427,7 @@ typedef struct { PyObject* data; /* data collector (string or list), or NULL */ PyObject* stack; /* element stack */ - int index; /* current stack size (0=empty) */ + Py_ssize_t index; /* current stack size (0=empty) */ /* element tracing */ PyObject* events; /* list of events, or NULL if not collecting */ @@ -1606,7 +1607,7 @@ treebuilder_handle_data(TreeBuilderObject* self, PyObject* data) PyString_CheckExact(data) && PyString_GET_SIZE(data) == 1) { /* expat often generates single character data sections; handle the most common case by resizing the existing string... */ - int size = PyString_GET_SIZE(self->data); + Py_ssize_t size = PyString_GET_SIZE(self->data); if (_PyString_Resize(&self->data, size + 1) < 0) return NULL; PyString_AS_STRING(self->data)[size] = PyString_AS_STRING(data)[0]; @@ -1988,7 +1989,7 @@ expat_default_handler(XMLParserObject* self, const XML_Char* data_in, Py_XDECREF(res); } else { PyErr_Format( - PyExc_SyntaxError, "undefined entity &%s;: line %d, column %d", + PyExc_SyntaxError, "undefined entity &%s;: line %ld, column %ld", PyString_AS_STRING(key), EXPAT(GetErrorLineNumber)(self->parser), EXPAT(GetErrorColumnNumber)(self->parser) @@ -2349,7 +2350,7 @@ expat_parse(XMLParserObject* self, char* data, int data_len, int final) if (!ok) { PyErr_Format( - PyExc_SyntaxError, "%s: line %d, column %d", + PyExc_SyntaxError, "%s: line %ld, column %ld", EXPAT(ErrorString)(EXPAT(GetErrorCode)(self->parser)), EXPAT(GetErrorLineNumber)(self->parser), EXPAT(GetErrorColumnNumber)(self->parser) @@ -2456,7 +2457,7 @@ xmlparser_setevents(XMLParserObject* self, PyObject* args) { /* activate element event reporting */ - int i; + Py_ssize_t i; TreeBuilderObject* target; PyObject* events; /* event collector */ diff --git a/Modules/_hotshot.c b/Modules/_hotshot.c index 5a81bfb..6d9776f 100644 --- a/Modules/_hotshot.c +++ b/Modules/_hotshot.c @@ -14,7 +14,11 @@ */ #ifdef MS_WINDOWS #include + +#ifdef HAVE_DIRECT_H #include /* for getcwd() */ +#endif + typedef __int64 hs_time; #define GETTIMEOFDAY(P_HS_TIME) \ { LARGE_INTEGER _temp; \ diff --git a/Modules/_localemodule.c b/Modules/_localemodule.c index c016cd7..abfca4e 100644 --- a/Modules/_localemodule.c +++ b/Modules/_localemodule.c @@ -16,7 +16,7 @@ This software comes with no warranty. Use at your own risk. #include #include -#ifndef DONT_HAVE_ERRNO_H +#ifdef HAVE_ERRNO_H #include #endif diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index 64e43eb..703af15 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -34,6 +34,19 @@ static int connection_set_isolation_level(Connection* self, PyObject* isolation_level); + +void _sqlite3_result_error(sqlite3_context* ctx, const char* errmsg, int len) +{ + /* in older SQLite versions, calling sqlite3_result_error in callbacks + * triggers a bug in SQLite that leads either to irritating results or + * segfaults, depending on the SQLite version */ +#if SQLITE_VERSION_NUMBER >= 3003003 + sqlite3_result_error(ctx, errmsg, len); +#else + PyErr_SetString(OperationalError, errmsg); +#endif +} + int connection_init(Connection* self, PyObject* args, PyObject* kwargs) { static char *kwlist[] = {"database", "timeout", "detect_types", "isolation_level", "check_same_thread", "factory", "cached_statements", NULL, NULL}; @@ -405,8 +418,6 @@ void _set_result(sqlite3_context* context, PyObject* py_val) PyObject* stringval; if ((!py_val) || PyErr_Occurred()) { - /* Errors in callbacks are ignored, and we return NULL */ - PyErr_Clear(); sqlite3_result_null(context); } else if (py_val == Py_None) { sqlite3_result_null(context); @@ -519,8 +530,17 @@ void _func_callback(sqlite3_context* context, int argc, sqlite3_value** argv) Py_DECREF(args); } - _set_result(context, py_retval); - Py_XDECREF(py_retval); + if (py_retval) { + _set_result(context, py_retval); + Py_DECREF(py_retval); + } else { + if (_enable_callback_tracebacks) { + PyErr_Print(); + } else { + PyErr_Clear(); + } + _sqlite3_result_error(context, "user-defined function raised exception", -1); + } PyGILState_Release(threadstate); } @@ -545,8 +565,13 @@ static void _step_callback(sqlite3_context *context, int argc, sqlite3_value** p *aggregate_instance = PyObject_CallFunction(aggregate_class, ""); if (PyErr_Occurred()) { - PyErr_Clear(); *aggregate_instance = 0; + if (_enable_callback_tracebacks) { + PyErr_Print(); + } else { + PyErr_Clear(); + } + _sqlite3_result_error(context, "user-defined aggregate's '__init__' method raised error", -1); goto error; } } @@ -565,7 +590,12 @@ static void _step_callback(sqlite3_context *context, int argc, sqlite3_value** p Py_DECREF(args); if (!function_result) { - PyErr_Clear(); + if (_enable_callback_tracebacks) { + PyErr_Print(); + } else { + PyErr_Clear(); + } + _sqlite3_result_error(context, "user-defined aggregate's 'step' method raised error", -1); } error: @@ -597,13 +627,16 @@ void _final_callback(sqlite3_context* context) function_result = PyObject_CallMethod(*aggregate_instance, "finalize", ""); if (!function_result) { - PyErr_Clear(); - Py_INCREF(Py_None); - function_result = Py_None; + if (_enable_callback_tracebacks) { + PyErr_Print(); + } else { + PyErr_Clear(); + } + _sqlite3_result_error(context, "user-defined aggregate's 'finalize' method raised error", -1); + } else { + _set_result(context, function_result); } - _set_result(context, function_result); - error: Py_XDECREF(*aggregate_instance); Py_XDECREF(function_result); @@ -631,7 +664,7 @@ void _drop_unused_statement_references(Connection* self) for (i = 0; i < PyList_Size(self->statements); i++) { weakref = PyList_GetItem(self->statements, i); - if (weakref != Py_None) { + if (PyWeakref_GetObject(weakref) != Py_None) { if (PyList_Append(new_list, weakref) != 0) { Py_DECREF(new_list); return; @@ -699,6 +732,61 @@ PyObject* connection_create_aggregate(Connection* self, PyObject* args, PyObject } } +int _authorizer_callback(void* user_arg, int action, const char* arg1, const char* arg2 , const char* dbname, const char* access_attempt_source) +{ + PyObject *ret; + int rc; + PyGILState_STATE gilstate; + + gilstate = PyGILState_Ensure(); + ret = PyObject_CallFunction((PyObject*)user_arg, "issss", action, arg1, arg2, dbname, access_attempt_source); + + if (!ret) { + if (_enable_callback_tracebacks) { + PyErr_Print(); + } else { + PyErr_Clear(); + } + + rc = SQLITE_DENY; + } else { + if (PyInt_Check(ret)) { + rc = (int)PyInt_AsLong(ret); + } else { + rc = SQLITE_DENY; + } + Py_DECREF(ret); + } + + PyGILState_Release(gilstate); + return rc; +} + +PyObject* connection_set_authorizer(Connection* self, PyObject* args, PyObject* kwargs) +{ + PyObject* authorizer_cb; + + static char *kwlist[] = { "authorizer_callback", NULL }; + int rc; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O:set_authorizer", + kwlist, &authorizer_cb)) { + return NULL; + } + + rc = sqlite3_set_authorizer(self->db, _authorizer_callback, (void*)authorizer_cb); + + if (rc != SQLITE_OK) { + PyErr_SetString(OperationalError, "Error setting authorizer callback"); + return NULL; + } else { + PyDict_SetItem(self->function_pinboard, authorizer_cb, Py_None); + + Py_INCREF(Py_None); + return Py_None; + } +} + int check_thread(Connection* self) { if (self->check_same_thread) { @@ -975,6 +1063,24 @@ finally: } static PyObject * +connection_interrupt(Connection* self, PyObject* args) +{ + PyObject* retval = NULL; + + if (!check_connection(self)) { + goto finally; + } + + sqlite3_interrupt(self->db); + + Py_INCREF(Py_None); + retval = Py_None; + +finally: + return retval; +} + +static PyObject * connection_create_collation(Connection* self, PyObject* args) { PyObject* callable; @@ -1067,6 +1173,8 @@ static PyMethodDef connection_methods[] = { PyDoc_STR("Creates a new function. Non-standard.")}, {"create_aggregate", (PyCFunction)connection_create_aggregate, METH_VARARGS|METH_KEYWORDS, PyDoc_STR("Creates a new aggregate. Non-standard.")}, + {"set_authorizer", (PyCFunction)connection_set_authorizer, METH_VARARGS|METH_KEYWORDS, + PyDoc_STR("Sets authorizer callback. Non-standard.")}, {"execute", (PyCFunction)connection_execute, METH_VARARGS, PyDoc_STR("Executes a SQL statement. Non-standard.")}, {"executemany", (PyCFunction)connection_executemany, METH_VARARGS, @@ -1075,6 +1183,8 @@ static PyMethodDef connection_methods[] = { PyDoc_STR("Executes a multiple SQL statements at once. Non-standard.")}, {"create_collation", (PyCFunction)connection_create_collation, METH_VARARGS, PyDoc_STR("Creates a collation function. Non-standard.")}, + {"interrupt", (PyCFunction)connection_interrupt, METH_NOARGS, + PyDoc_STR("Abort any pending database operation. Non-standard.")}, {NULL, NULL} }; diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c index 95a4931..a2d3cf2 100644 --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -137,6 +137,22 @@ void cursor_dealloc(Cursor* self) self->ob_type->tp_free((PyObject*)self); } +PyObject* _get_converter(PyObject* key) +{ + PyObject* upcase_key; + PyObject* retval; + + upcase_key = PyObject_CallMethod(key, "upper", ""); + if (!upcase_key) { + return NULL; + } + + retval = PyDict_GetItem(converters, upcase_key); + Py_DECREF(upcase_key); + + return retval; +} + int build_row_cast_map(Cursor* self) { int i; @@ -174,7 +190,7 @@ int build_row_cast_map(Cursor* self) break; } - converter = PyDict_GetItem(converters, key); + converter = _get_converter(key); Py_DECREF(key); break; } @@ -195,7 +211,7 @@ int build_row_cast_map(Cursor* self) } } - converter = PyDict_GetItem(converters, py_decltype); + converter = _get_converter(py_decltype); Py_DECREF(py_decltype); } } @@ -228,7 +244,10 @@ PyObject* _build_column_name(const char* colname) } for (pos = colname;; pos++) { - if (*pos == 0 || *pos == ' ') { + if (*pos == 0 || *pos == '[') { + if ((*pos == '[') && (pos > colname) && (*(pos-1) == ' ')) { + pos--; + } return PyString_FromStringAndSize(colname, pos - colname); } } @@ -302,23 +321,21 @@ PyObject* _fetch_one_row(Cursor* self) } if (converter != Py_None) { - val_str = (const char*)sqlite3_column_text(self->statement->st, i); + nbytes = sqlite3_column_bytes(self->statement->st, i); + val_str = (const char*)sqlite3_column_blob(self->statement->st, i); if (!val_str) { Py_INCREF(Py_None); converted = Py_None; } else { - item = PyString_FromString(val_str); + item = PyString_FromStringAndSize(val_str, nbytes); if (!item) { return NULL; } converted = PyObject_CallFunction(converter, "O", item); + Py_DECREF(item); if (!converted) { - /* TODO: have a way to log these errors */ - Py_INCREF(Py_None); - converted = Py_None; - PyErr_Clear(); + break; } - Py_DECREF(item); } } else { Py_BEGIN_ALLOW_THREADS @@ -346,10 +363,10 @@ PyObject* _fetch_one_row(Cursor* self) if (!converted) { colname = sqlite3_column_name(self->statement->st, i); - if (colname) { + if (!colname) { colname = ""; } - PyOS_snprintf(buf, sizeof(buf) - 1, "Could not decode to UTF-8 column %s with text %s", + PyOS_snprintf(buf, sizeof(buf) - 1, "Could not decode to UTF-8 column '%s' with text '%s'", colname , val_str); PyErr_SetString(OperationalError, buf); } @@ -373,7 +390,12 @@ PyObject* _fetch_one_row(Cursor* self) } } - PyTuple_SetItem(row, i, converted); + if (converted) { + PyTuple_SetItem(row, i, converted); + } else { + Py_INCREF(Py_None); + PyTuple_SetItem(row, i, Py_None); + } } if (PyErr_Occurred()) { @@ -598,6 +620,14 @@ PyObject* _query_execute(Cursor* self, int multiple, PyObject* args) goto error; } } else { + if (PyErr_Occurred()) { + /* there was an error that occurred in a user-defined callback */ + if (_enable_callback_tracebacks) { + PyErr_Print(); + } else { + PyErr_Clear(); + } + } _seterror(self->connection->db); goto error; } diff --git a/Modules/_sqlite/module.c b/Modules/_sqlite/module.c index 71d0aaa..606454c 100644 --- a/Modules/_sqlite/module.c +++ b/Modules/_sqlite/module.c @@ -1,25 +1,25 @@ -/* module.c - the module itself - * - * Copyright (C) 2004-2006 Gerhard Häring - * - * This file is part of pysqlite. - * - * This software is provided 'as-is', without any express or implied - * warranty. In no event will the authors be held liable for any damages - * arising from the use of this software. - * - * Permission is granted to anyone to use this software for any purpose, - * including commercial applications, and to alter it and redistribute it - * freely, subject to the following restrictions: - * - * 1. The origin of this software must not be misrepresented; you must not - * claim that you wrote the original software. If you use this software - * in a product, an acknowledgment in the product documentation would be - * appreciated but is not required. - * 2. Altered source versions must be plainly marked as such, and must not be - * misrepresented as being the original software. - * 3. This notice may not be removed or altered from any source distribution. - */ + /* module.c - the module itself + * + * Copyright (C) 2004-2006 Gerhard Häring + * + * This file is part of pysqlite. + * + * This software is provided 'as-is', without any express or implied + * warranty. In no event will the authors be held liable for any damages + * arising from the use of this software. + * + * Permission is granted to anyone to use this software for any purpose, + * including commercial applications, and to alter it and redistribute it + * freely, subject to the following restrictions: + * + * 1. The origin of this software must not be misrepresented; you must not + * claim that you wrote the original software. If you use this software + * in a product, an acknowledgment in the product documentation would be + * appreciated but is not required. + * 2. Altered source versions must be plainly marked as such, and must not be + * misrepresented as being the original software. + * 3. This notice may not be removed or altered from any source distribution. + */ #include "connection.h" #include "statement.h" @@ -40,6 +40,7 @@ PyObject* Error, *Warning, *InterfaceError, *DatabaseError, *InternalError, *NotSupportedError, *OptimizedUnicode; PyObject* converters; +int _enable_callback_tracebacks; static PyObject* module_connect(PyObject* self, PyObject* args, PyObject* kwargs) @@ -140,14 +141,42 @@ static PyObject* module_register_adapter(PyObject* self, PyObject* args, PyObjec static PyObject* module_register_converter(PyObject* self, PyObject* args, PyObject* kwargs) { - PyObject* name; + char* orig_name; + char* name = NULL; + char* c; PyObject* callable; + PyObject* retval = NULL; - if (!PyArg_ParseTuple(args, "OO", &name, &callable)) { + if (!PyArg_ParseTuple(args, "sO", &orig_name, &callable)) { return NULL; } - if (PyDict_SetItem(converters, name, callable) != 0) { + /* convert the name to lowercase */ + name = PyMem_Malloc(strlen(orig_name) + 2); + if (!name) { + goto error; + } + strcpy(name, orig_name); + for (c = name; *c != (char)0; c++) { + *c = (*c) & 0xDF; + } + + if (PyDict_SetItemString(converters, name, callable) != 0) { + goto error; + } + + Py_INCREF(Py_None); + retval = Py_None; +error: + if (name) { + PyMem_Free(name); + } + return retval; +} + +static PyObject* enable_callback_tracebacks(PyObject* self, PyObject* args, PyObject* kwargs) +{ + if (!PyArg_ParseTuple(args, "i", &_enable_callback_tracebacks)) { return NULL; } @@ -174,13 +203,64 @@ static PyMethodDef module_methods[] = { {"register_adapter", (PyCFunction)module_register_adapter, METH_VARARGS, PyDoc_STR("Registers an adapter with pysqlite's adapter registry. Non-standard.")}, {"register_converter", (PyCFunction)module_register_converter, METH_VARARGS, PyDoc_STR("Registers a converter with pysqlite. Non-standard.")}, {"adapt", (PyCFunction)psyco_microprotocols_adapt, METH_VARARGS, psyco_microprotocols_adapt_doc}, + {"enable_callback_tracebacks", (PyCFunction)enable_callback_tracebacks, METH_VARARGS, PyDoc_STR("Enable or disable callback functions throwing errors to stderr.")}, {NULL, NULL} }; +struct _IntConstantPair { + char* constant_name; + int constant_value; +}; + +typedef struct _IntConstantPair IntConstantPair; + +static IntConstantPair _int_constants[] = { + {"PARSE_DECLTYPES", PARSE_DECLTYPES}, + {"PARSE_COLNAMES", PARSE_COLNAMES}, + + {"SQLITE_OK", SQLITE_OK}, + {"SQLITE_DENY", SQLITE_DENY}, + {"SQLITE_IGNORE", SQLITE_IGNORE}, + {"SQLITE_CREATE_INDEX", SQLITE_CREATE_INDEX}, + {"SQLITE_CREATE_TABLE", SQLITE_CREATE_TABLE}, + {"SQLITE_CREATE_TEMP_INDEX", SQLITE_CREATE_TEMP_INDEX}, + {"SQLITE_CREATE_TEMP_TABLE", SQLITE_CREATE_TEMP_TABLE}, + {"SQLITE_CREATE_TEMP_TRIGGER", SQLITE_CREATE_TEMP_TRIGGER}, + {"SQLITE_CREATE_TEMP_VIEW", SQLITE_CREATE_TEMP_VIEW}, + {"SQLITE_CREATE_TRIGGER", SQLITE_CREATE_TRIGGER}, + {"SQLITE_CREATE_VIEW", SQLITE_CREATE_VIEW}, + {"SQLITE_DELETE", SQLITE_DELETE}, + {"SQLITE_DROP_INDEX", SQLITE_DROP_INDEX}, + {"SQLITE_DROP_TABLE", SQLITE_DROP_TABLE}, + {"SQLITE_DROP_TEMP_INDEX", SQLITE_DROP_TEMP_INDEX}, + {"SQLITE_DROP_TEMP_TABLE", SQLITE_DROP_TEMP_TABLE}, + {"SQLITE_DROP_TEMP_TRIGGER", SQLITE_DROP_TEMP_TRIGGER}, + {"SQLITE_DROP_TEMP_VIEW", SQLITE_DROP_TEMP_VIEW}, + {"SQLITE_DROP_TRIGGER", SQLITE_DROP_TRIGGER}, + {"SQLITE_DROP_VIEW", SQLITE_DROP_VIEW}, + {"SQLITE_INSERT", SQLITE_INSERT}, + {"SQLITE_PRAGMA", SQLITE_PRAGMA}, + {"SQLITE_READ", SQLITE_READ}, + {"SQLITE_SELECT", SQLITE_SELECT}, + {"SQLITE_TRANSACTION", SQLITE_TRANSACTION}, + {"SQLITE_UPDATE", SQLITE_UPDATE}, + {"SQLITE_ATTACH", SQLITE_ATTACH}, + {"SQLITE_DETACH", SQLITE_DETACH}, +#if SQLITE_VERSION_NUMBER >= 3002001 + {"SQLITE_ALTER_TABLE", SQLITE_ALTER_TABLE}, + {"SQLITE_REINDEX", SQLITE_REINDEX}, +#endif +#if SQLITE_VERSION_NUMBER >= 3003000 + {"SQLITE_ANALYZE", SQLITE_ANALYZE}, +#endif + {(char*)NULL, 0} +}; + PyMODINIT_FUNC init_sqlite3(void) { PyObject *module, *dict; PyObject *tmp_obj; + int i; module = Py_InitModule("_sqlite3", module_methods); @@ -276,17 +356,15 @@ PyMODINIT_FUNC init_sqlite3(void) OptimizedUnicode = (PyObject*)&PyCell_Type; PyDict_SetItemString(dict, "OptimizedUnicode", OptimizedUnicode); - if (!(tmp_obj = PyInt_FromLong(PARSE_DECLTYPES))) { - goto error; + /* Set integer constants */ + for (i = 0; _int_constants[i].constant_name != 0; i++) { + tmp_obj = PyInt_FromLong(_int_constants[i].constant_value); + if (!tmp_obj) { + goto error; + } + PyDict_SetItemString(dict, _int_constants[i].constant_name, tmp_obj); + Py_DECREF(tmp_obj); } - PyDict_SetItemString(dict, "PARSE_DECLTYPES", tmp_obj); - Py_DECREF(tmp_obj); - - if (!(tmp_obj = PyInt_FromLong(PARSE_COLNAMES))) { - goto error; - } - PyDict_SetItemString(dict, "PARSE_COLNAMES", tmp_obj); - Py_DECREF(tmp_obj); if (!(tmp_obj = PyString_FromString(PYSQLITE_VERSION))) { goto error; @@ -306,6 +384,8 @@ PyMODINIT_FUNC init_sqlite3(void) /* initialize the default converters */ converters_init(dict); + _enable_callback_tracebacks = 0; + /* Original comment form _bsddb.c in the Python core. This is also still * needed nowadays for Python 2.3/2.4. * diff --git a/Modules/_sqlite/module.h b/Modules/_sqlite/module.h index f3e2aa1..e514bd1 100644 --- a/Modules/_sqlite/module.h +++ b/Modules/_sqlite/module.h @@ -25,7 +25,7 @@ #define PYSQLITE_MODULE_H #include "Python.h" -#define PYSQLITE_VERSION "2.2.2" +#define PYSQLITE_VERSION "2.3.2" extern PyObject* Error; extern PyObject* Warning; @@ -50,6 +50,8 @@ extern PyObject* time_sleep; */ extern PyObject* converters; +extern int _enable_callback_tracebacks; + #define PARSE_DECLTYPES 1 #define PARSE_COLNAMES 2 #endif diff --git a/Modules/_sqlite/util.c b/Modules/_sqlite/util.c index 33748a6..f5a7233 100644 --- a/Modules/_sqlite/util.c +++ b/Modules/_sqlite/util.c @@ -38,7 +38,7 @@ int _sqlite_step_with_busyhandler(sqlite3_stmt* statement, Connection* connectio /** * Checks the SQLite error code and sets the appropriate DB-API exception. - * Returns the error code (0 means no error occured). + * Returns the error code (0 means no error occurred). */ int _seterror(sqlite3* db) { diff --git a/Modules/_sqlite/util.h b/Modules/_sqlite/util.h index e99a4dd..7ce3d40 100644 --- a/Modules/_sqlite/util.h +++ b/Modules/_sqlite/util.h @@ -32,7 +32,7 @@ int _sqlite_step_with_busyhandler(sqlite3_stmt* statement, Connection* connectio /** * Checks the SQLite error code and sets the appropriate DB-API exception. - * Returns the error code (0 means no error occured). + * Returns the error code (0 means no error occurred). */ int _seterror(sqlite3* db); #endif diff --git a/Modules/_sre.c b/Modules/_sre.c index d4e6051..68d511e 100644 --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -39,6 +39,8 @@ static char copyright[] = " SRE 2.2.2 Copyright (c) 1997-2002 by Secret Labs AB "; +#define PY_SSIZE_T_CLEAN + #include "Python.h" #include "structmember.h" /* offsetof */ @@ -261,9 +263,9 @@ data_stack_dealloc(SRE_STATE* state) } static int -data_stack_grow(SRE_STATE* state, int size) +data_stack_grow(SRE_STATE* state, Py_ssize_t size) { - int minsize, cursize; + Py_ssize_t minsize, cursize; minsize = state->data_stack_base+size; cursize = state->data_stack_size; if (cursize < minsize) { @@ -335,7 +337,7 @@ SRE_AT(SRE_STATE* state, SRE_CHAR* ptr, SRE_CODE at) { /* check if pointer is at given position */ - int thisp, thatp; + Py_ssize_t thisp, thatp; switch (at) { @@ -476,7 +478,7 @@ SRE_CHARSET(SRE_CODE* set, SRE_CODE ch) case SRE_OP_BIGCHARSET: /* <256 blockindices> */ { - int count, block; + Py_ssize_t count, block; count = *(set++); if (sizeof(SRE_CODE) == 2) { @@ -510,15 +512,15 @@ SRE_CHARSET(SRE_CODE* set, SRE_CODE ch) } } -LOCAL(int) SRE_MATCH(SRE_STATE* state, SRE_CODE* pattern); +LOCAL(Py_ssize_t) SRE_MATCH(SRE_STATE* state, SRE_CODE* pattern); -LOCAL(int) -SRE_COUNT(SRE_STATE* state, SRE_CODE* pattern, int maxcount) +LOCAL(Py_ssize_t) +SRE_COUNT(SRE_STATE* state, SRE_CODE* pattern, Py_ssize_t maxcount) { SRE_CODE chr; SRE_CHAR* ptr = (SRE_CHAR *)state->ptr; SRE_CHAR* end = (SRE_CHAR *)state->end; - int i; + Py_ssize_t i; /* adjust end */ if (maxcount < end - ptr && maxcount != 65535) @@ -608,7 +610,7 @@ SRE_INFO(SRE_STATE* state, SRE_CODE* pattern) SRE_CHAR* end = state->end; SRE_CHAR* ptr = state->ptr; - int i; + Py_ssize_t i; /* check minimal length */ if (pattern[3] && (end - ptr) < pattern[3]) @@ -785,13 +787,13 @@ do { \ while (0) /* gcc doesn't like labels at end of scopes */ \ typedef struct { - int last_ctx_pos; - int jump; + Py_ssize_t last_ctx_pos; + Py_ssize_t jump; SRE_CHAR* ptr; SRE_CODE* pattern; - int count; - int lastmark; - int lastindex; + Py_ssize_t count; + Py_ssize_t lastmark; + Py_ssize_t lastindex; union { SRE_CODE chr; SRE_REPEAT* rep; @@ -800,13 +802,13 @@ typedef struct { /* check if string matches the given pattern. returns <0 for error, 0 for failure, and 1 for success */ -LOCAL(int) +LOCAL(Py_ssize_t) SRE_MATCH(SRE_STATE* state, SRE_CODE* pattern) { SRE_CHAR* end = (SRE_CHAR *)state->end; - int alloc_pos, ctx_pos = -1; - int i, ret = 0; - int jump; + Py_ssize_t alloc_pos, ctx_pos = -1; + Py_ssize_t i, ret = 0; + Py_ssize_t jump; SRE_MATCH_CONTEXT* ctx; SRE_MATCH_CONTEXT* nextctx; @@ -851,7 +853,7 @@ entrance: state->mark array. If it is increased by more than 1, the intervening marks must be set to NULL to signal that these marks have not been encountered. */ - int j = state->lastmark + 1; + Py_ssize_t j = state->lastmark + 1; while (j < i) state->mark[j++] = NULL; state->lastmark = i; @@ -1037,7 +1039,7 @@ entrance: string. check if the rest of the pattern matches, and backtrack if not. */ - if (ctx->count < (int) ctx->pattern[1]) + if (ctx->count < (Py_ssize_t) ctx->pattern[1]) RETURN_FAILURE; if (ctx->pattern[ctx->pattern[0]] == SRE_OP_SUCCESS) { @@ -1053,12 +1055,12 @@ entrance: the rest of the pattern cannot possibly match */ ctx->u.chr = ctx->pattern[ctx->pattern[0]+1]; for (;;) { - while (ctx->count >= (int) ctx->pattern[1] && + while (ctx->count >= (Py_ssize_t) ctx->pattern[1] && (ctx->ptr >= end || *ctx->ptr != ctx->u.chr)) { ctx->ptr--; ctx->count--; } - if (ctx->count < (int) ctx->pattern[1]) + if (ctx->count < (Py_ssize_t) ctx->pattern[1]) break; state->ptr = ctx->ptr; DO_JUMP(JUMP_REPEAT_ONE_1, jump_repeat_one_1, @@ -1076,7 +1078,7 @@ entrance: } else { /* general case */ - while (ctx->count >= (int) ctx->pattern[1]) { + while (ctx->count >= (Py_ssize_t) ctx->pattern[1]) { state->ptr = ctx->ptr; DO_JUMP(JUMP_REPEAT_ONE_2, jump_repeat_one_2, ctx->pattern+ctx->pattern[0]); @@ -1116,7 +1118,7 @@ entrance: ret = SRE_COUNT(state, ctx->pattern+3, ctx->pattern[1]); RETURN_ON_ERROR(ret); DATA_LOOKUP_AT(SRE_MATCH_CONTEXT, ctx, ctx_pos); - if (ret < (int) ctx->pattern[1]) + if (ret < (Py_ssize_t) ctx->pattern[1]) /* didn't match minimum number of times */ RETURN_FAILURE; /* advance past minimum matches of repeat */ @@ -1132,8 +1134,8 @@ entrance: } else { /* general case */ LASTMARK_SAVE(); - while ((int)ctx->pattern[2] == 65535 - || ctx->count <= (int)ctx->pattern[2]) { + while ((Py_ssize_t)ctx->pattern[2] == 65535 + || ctx->count <= (Py_ssize_t)ctx->pattern[2]) { state->ptr = ctx->ptr; DO_JUMP(JUMP_MIN_REPEAT_ONE,jump_min_repeat_one, ctx->pattern+ctx->pattern[0]); @@ -1312,7 +1314,7 @@ entrance: ctx->ptr, ctx->pattern[0])); i = ctx->pattern[0]; { - int groupref = i+i; + Py_ssize_t groupref = i+i; if (groupref >= state->lastmark) { RETURN_FAILURE; } else { @@ -1336,7 +1338,7 @@ entrance: ctx->ptr, ctx->pattern[0])); i = ctx->pattern[0]; { - int groupref = i+i; + Py_ssize_t groupref = i+i; if (groupref >= state->lastmark) { RETURN_FAILURE; } else { @@ -1361,7 +1363,7 @@ entrance: /* codeyes codeno ... */ i = ctx->pattern[0]; { - int groupref = i+i; + Py_ssize_t groupref = i+i; if (groupref >= state->lastmark) { ctx->pattern += ctx->pattern[1]; break; @@ -1474,14 +1476,14 @@ exit: return ret; /* should never get here */ } -LOCAL(int) +LOCAL(Py_ssize_t) SRE_SEARCH(SRE_STATE* state, SRE_CODE* pattern) { SRE_CHAR* ptr = (SRE_CHAR *)state->start; SRE_CHAR* end = (SRE_CHAR *)state->end; - int status = 0; - int prefix_len = 0; - int prefix_skip = 0; + Py_ssize_t status = 0; + Py_ssize_t prefix_len = 0; + Py_ssize_t prefix_skip = 0; SRE_CODE* prefix = NULL; SRE_CODE* charset = NULL; SRE_CODE* overlap = NULL; @@ -1523,7 +1525,7 @@ SRE_SEARCH(SRE_STATE* state, SRE_CODE* pattern) if (prefix_len > 1) { /* pattern starts with a known prefix. use the overlap table to skip forward as fast as we possibly can */ - int i = 0; + Py_ssize_t i = 0; end = (SRE_CHAR *)state->end; while (ptr < end) { for (;;) { @@ -1604,7 +1606,7 @@ SRE_SEARCH(SRE_STATE* state, SRE_CODE* pattern) } LOCAL(int) -SRE_LITERAL_TEMPLATE(SRE_CHAR* ptr, int len) +SRE_LITERAL_TEMPLATE(SRE_CHAR* ptr, Py_ssize_t len) { /* check if given string is a literal template (i.e. no escapes) */ while (len-- > 0) @@ -1625,7 +1627,7 @@ static PyObject*pattern_scanner(PatternObject*, PyObject*); static PyObject * sre_codesize(PyObject* self, PyObject *unused) { - return Py_BuildValue("i", sizeof(SRE_CODE)); + return Py_BuildValue("l", sizeof(SRE_CODE)); } static PyObject * @@ -1660,14 +1662,15 @@ state_reset(SRE_STATE* state) } static void* -getstring(PyObject* string, int* p_length, int* p_charsize) +getstring(PyObject* string, Py_ssize_t* p_length, int* p_charsize) { /* given a python object, return a data pointer, a length (in characters), and a character size. return NULL if the object is not a string (or not compatible) */ PyBufferProcs *buffer; - int size, bytes, charsize; + Py_ssize_t size, bytes; + int charsize; void* ptr; #if defined(HAVE_UNICODE) @@ -1706,7 +1709,7 @@ getstring(PyObject* string, int* p_length, int* p_charsize) if (PyString_Check(string) || bytes == size) charsize = 1; #if defined(HAVE_UNICODE) - else if (bytes == (int) (size * sizeof(Py_UNICODE))) + else if (bytes == (Py_ssize_t) (size * sizeof(Py_UNICODE))) charsize = sizeof(Py_UNICODE); #endif else { @@ -1726,11 +1729,11 @@ getstring(PyObject* string, int* p_length, int* p_charsize) LOCAL(PyObject*) state_init(SRE_STATE* state, PatternObject* pattern, PyObject* string, - int start, int end) + Py_ssize_t start, Py_ssize_t end) { /* prepare state object */ - int length; + Py_ssize_t length; int charsize; void* ptr; @@ -1792,9 +1795,9 @@ state_fini(SRE_STATE* state) (((char*)(member) - (char*)(state)->beginning) / (state)->charsize) LOCAL(PyObject*) -state_getslice(SRE_STATE* state, int index, PyObject* string, int empty) +state_getslice(SRE_STATE* state, Py_ssize_t index, PyObject* string, int empty) { - int i, j; + Py_ssize_t i, j; index = (index - 1) * 2; @@ -1854,10 +1857,10 @@ pattern_match(PatternObject* self, PyObject* args, PyObject* kw) int status; PyObject* string; - int start = 0; - int end = INT_MAX; + Py_ssize_t start = 0; + Py_ssize_t end = PY_SSIZE_T_MAX; static char* kwlist[] = { "pattern", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|ii:match", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:match", kwlist, &string, &start, &end)) return NULL; @@ -1891,10 +1894,10 @@ pattern_search(PatternObject* self, PyObject* args, PyObject* kw) int status; PyObject* string; - int start = 0; - int end = INT_MAX; + Py_ssize_t start = 0; + Py_ssize_t end = PY_SSIZE_T_MAX; static char* kwlist[] = { "pattern", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|ii:search", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:search", kwlist, &string, &start, &end)) return NULL; @@ -2029,13 +2032,13 @@ pattern_findall(PatternObject* self, PyObject* args, PyObject* kw) SRE_STATE state; PyObject* list; int status; - int i, b, e; + Py_ssize_t i, b, e; PyObject* string; - int start = 0; - int end = INT_MAX; + Py_ssize_t start = 0; + Py_ssize_t end = PY_SSIZE_T_MAX; static char* kwlist[] = { "source", "pos", "endpos", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|ii:findall", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "O|nn:findall", kwlist, &string, &start, &end)) return NULL; @@ -2154,18 +2157,18 @@ pattern_split(PatternObject* self, PyObject* args, PyObject* kw) PyObject* list; PyObject* item; int status; - int n; - int i; + Py_ssize_t n; + Py_ssize_t i; void* last; PyObject* string; - int maxsplit = 0; + Py_ssize_t maxsplit = 0; static char* kwlist[] = { "source", "maxsplit", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "O|i:split", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "O|n:split", kwlist, &string, &maxsplit)) return NULL; - string = state_init(&state, self, string, 0, INT_MAX); + string = state_init(&state, self, string, 0, PY_SSIZE_T_MAX); if (!string) return NULL; @@ -2259,7 +2262,7 @@ error: static PyObject* pattern_subx(PatternObject* self, PyObject* ptemplate, PyObject* string, - int count, int subn) + Py_ssize_t count, Py_ssize_t subn) { SRE_STATE state; PyObject* list; @@ -2269,8 +2272,9 @@ pattern_subx(PatternObject* self, PyObject* ptemplate, PyObject* string, PyObject* match; void* ptr; int status; - int n; - int i, b, e; + Py_ssize_t n; + Py_ssize_t i, b, e; + int bint; int filter_is_callable; if (PyCallable_Check(ptemplate)) { @@ -2281,7 +2285,8 @@ pattern_subx(PatternObject* self, PyObject* ptemplate, PyObject* string, } else { /* if not callable, check if it's a literal string */ int literal; - ptr = getstring(ptemplate, &n, &b); + ptr = getstring(ptemplate, &n, &bint); + b = bint; if (ptr) { if (b == 1) { literal = sre_literal_template((unsigned char *)ptr, n); @@ -2310,7 +2315,7 @@ pattern_subx(PatternObject* self, PyObject* ptemplate, PyObject* string, } } - string = state_init(&state, self, string, 0, INT_MAX); + string = state_init(&state, self, string, 0, PY_SSIZE_T_MAX); if (!string) { Py_DECREF(filter); return NULL; @@ -2443,9 +2448,9 @@ pattern_sub(PatternObject* self, PyObject* args, PyObject* kw) { PyObject* ptemplate; PyObject* string; - int count = 0; + Py_ssize_t count = 0; static char* kwlist[] = { "repl", "string", "count", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "OO|i:sub", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "OO|n:sub", kwlist, &ptemplate, &string, &count)) return NULL; @@ -2457,9 +2462,9 @@ pattern_subn(PatternObject* self, PyObject* args, PyObject* kw) { PyObject* ptemplate; PyObject* string; - int count = 0; + Py_ssize_t count = 0; static char* kwlist[] = { "repl", "string", "count", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kw, "OO|i:subn", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kw, "OO|n:subn", kwlist, &ptemplate, &string, &count)) return NULL; @@ -2642,15 +2647,15 @@ _compile(PyObject* self_, PyObject* args) /* "compile" pattern descriptor to pattern object */ PatternObject* self; - int i, n; + Py_ssize_t i, n; PyObject* pattern; int flags = 0; PyObject* code; - int groups = 0; + Py_ssize_t groups = 0; PyObject* groupindex = NULL; PyObject* indexgroup = NULL; - if (!PyArg_ParseTuple(args, "OiO!|iOO", &pattern, &flags, + if (!PyArg_ParseTuple(args, "OiO!|nOO", &pattern, &flags, &PyList_Type, &code, &groups, &groupindex, &indexgroup)) return NULL; @@ -2711,7 +2716,7 @@ match_dealloc(MatchObject* self) } static PyObject* -match_getslice_by_index(MatchObject* self, int index, PyObject* def) +match_getslice_by_index(MatchObject* self, Py_ssize_t index, PyObject* def) { if (index < 0 || index >= self->groups) { /* raise IndexError if we were given a bad group number */ @@ -2735,21 +2740,21 @@ match_getslice_by_index(MatchObject* self, int index, PyObject* def) ); } -static int +static Py_ssize_t match_getindex(MatchObject* self, PyObject* index) { - int i; + Py_ssize_t i; if (PyInt_Check(index)) - return (int) PyInt_AS_LONG(index); + return PyInt_AsSsize_t(index); i = -1; if (self->pattern->groupindex) { index = PyObject_GetItem(self->pattern->groupindex, index); if (index) { - if (PyInt_Check(index)) - i = (int) PyInt_AS_LONG(index); + if (PyInt_Check(index) || PyLong_Check(index)) + i = PyInt_AsSsize_t(index); Py_DECREF(index); } else PyErr_Clear(); @@ -2778,7 +2783,7 @@ static PyObject* match_group(MatchObject* self, PyObject* args) { PyObject* result; - int i, size; + Py_ssize_t i, size; size = PyTuple_GET_SIZE(args); @@ -2813,7 +2818,7 @@ static PyObject* match_groups(MatchObject* self, PyObject* args, PyObject* kw) { PyObject* result; - int index; + Py_ssize_t index; PyObject* def = Py_None; static char* kwlist[] = { "default", NULL }; @@ -2842,7 +2847,7 @@ match_groupdict(MatchObject* self, PyObject* args, PyObject* kw) { PyObject* result; PyObject* keys; - int index; + Py_ssize_t index; PyObject* def = Py_None; static char* kwlist[] = { "default", NULL }; @@ -2888,7 +2893,7 @@ failed: static PyObject* match_start(MatchObject* self, PyObject* args) { - int index; + Py_ssize_t index; PyObject* index_ = Py_False; /* zero */ if (!PyArg_UnpackTuple(args, "start", 0, 1, &index_)) @@ -2911,7 +2916,7 @@ match_start(MatchObject* self, PyObject* args) static PyObject* match_end(MatchObject* self, PyObject* args) { - int index; + Py_ssize_t index; PyObject* index_ = Py_False; /* zero */ if (!PyArg_UnpackTuple(args, "end", 0, 1, &index_)) @@ -2932,7 +2937,7 @@ match_end(MatchObject* self, PyObject* args) } LOCAL(PyObject*) -_pair(int i1, int i2) +_pair(Py_ssize_t i1, Py_ssize_t i2) { PyObject* pair; PyObject* item; @@ -2941,12 +2946,12 @@ _pair(int i1, int i2) if (!pair) return NULL; - item = PyInt_FromLong(i1); + item = PyInt_FromSsize_t(i1); if (!item) goto error; PyTuple_SET_ITEM(pair, 0, item); - item = PyInt_FromLong(i2); + item = PyInt_FromSsize_t(i2); if (!item) goto error; PyTuple_SET_ITEM(pair, 1, item); @@ -2961,7 +2966,7 @@ _pair(int i1, int i2) static PyObject* match_span(MatchObject* self, PyObject* args) { - int index; + Py_ssize_t index; PyObject* index_ = Py_False; /* zero */ if (!PyArg_UnpackTuple(args, "span", 0, 1, &index_)) @@ -2986,7 +2991,7 @@ match_regs(MatchObject* self) { PyObject* regs; PyObject* item; - int index; + Py_ssize_t index; regs = PyTuple_New(self->groups); if (!regs) @@ -3012,7 +3017,7 @@ match_copy(MatchObject* self, PyObject *unused) { #ifdef USE_BUILTIN_COPY MatchObject* copy; - int slots, offset; + Py_ssize_t slots, offset; slots = 2 * (self->pattern->groups+1); @@ -3029,7 +3034,7 @@ match_copy(MatchObject* self, PyObject *unused) Py_XINCREF(self->regs); memcpy((char*) copy + offset, (char*) self + offset, - sizeof(MatchObject) + slots * sizeof(int) - offset); + sizeof(MatchObject) + slots * sizeof(Py_ssize_t) - offset); return (PyObject*) copy; #else @@ -3144,7 +3149,7 @@ match_getattr(MatchObject* self, char* name) static PyTypeObject Match_Type = { PyObject_HEAD_INIT(NULL) 0, "_" SRE_MODULE ".SRE_Match", - sizeof(MatchObject), sizeof(int), + sizeof(MatchObject), sizeof(Py_ssize_t), (destructor)match_dealloc, /*tp_dealloc*/ 0, /*tp_print*/ (getattrfunc)match_getattr /*tp_getattr*/ @@ -3156,7 +3161,7 @@ pattern_new_match(PatternObject* pattern, SRE_STATE* state, int status) /* create match object (from state object) */ MatchObject* match; - int i, j; + Py_ssize_t i, j; char* base; int n; @@ -3329,9 +3334,9 @@ pattern_scanner(PatternObject* pattern, PyObject* args) ScannerObject* self; PyObject* string; - int start = 0; - int end = INT_MAX; - if (!PyArg_ParseTuple(args, "O|ii:scanner", &string, &start, &end)) + Py_ssize_t start = 0; + Py_ssize_t end = PY_SSIZE_T_MAX; + if (!PyArg_ParseTuple(args, "O|nn:scanner", &string, &start, &end)) return NULL; /* create scanner object */ diff --git a/Modules/_ssl.c b/Modules/_ssl.c index f49391d..3b91b24 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -26,6 +26,12 @@ enum py_ssl_error { /* Include symbols from _socket module */ #include "socketmodule.h" +#if defined(HAVE_POLL_H) +#include +#elif defined(HAVE_SYS_POLL_H) +#include +#endif + /* Include OpenSSL header files */ #include "openssl/rsa.h" #include "openssl/crypto.h" @@ -351,7 +357,7 @@ static void PySSL_dealloc(PySSLObject *self) PyObject_Del(self); } -/* If the socket has a timeout, do a select() on the socket. +/* If the socket has a timeout, do a select()/poll() on the socket. The argument writing indicates the direction. Returns one of the possibilities in the timeout_state enum (above). */ @@ -373,6 +379,26 @@ check_socket_and_wait_for_timeout(PySocketSockObject *s, int writing) if (s->sock_fd < 0) return SOCKET_HAS_BEEN_CLOSED; + /* Prefer poll, if available, since you can poll() any fd + * which can't be done with select(). */ +#ifdef HAVE_POLL + { + struct pollfd pollfd; + int timeout; + + pollfd.fd = s->sock_fd; + pollfd.events = writing ? POLLOUT : POLLIN; + + /* s->sock_timeout is in seconds, timeout in ms */ + timeout = (int)(s->sock_timeout * 1000 + 0.5); + Py_BEGIN_ALLOW_THREADS + rc = poll(&pollfd, 1, timeout); + Py_END_ALLOW_THREADS + + goto normal_return; + } +#endif + /* Guard against socket too large for select*/ #ifndef Py_SOCKET_FD_CAN_BE_GE_FD_SETSIZE if (s->sock_fd >= FD_SETSIZE) @@ -393,6 +419,7 @@ check_socket_and_wait_for_timeout(PySocketSockObject *s, int writing) rc = select(s->sock_fd+1, &fds, NULL, NULL, &tv); Py_END_ALLOW_THREADS +normal_return: /* Return SOCKET_TIMED_OUT on timeout, SOCKET_OPERATION_OK otherwise (when we are able to write or when there's something to read) */ return rc == 0 ? SOCKET_HAS_TIMED_OUT : SOCKET_OPERATION_OK; diff --git a/Modules/_struct.c b/Modules/_struct.c index df243ec..8d0a3bc 100644 --- a/Modules/_struct.c +++ b/Modules/_struct.c @@ -31,6 +31,17 @@ static PyObject *pylong_ulong_mask = NULL; static PyObject *pyint_zero = NULL; #endif +/* If PY_STRUCT_FLOAT_COERCE is defined, the struct module will allow float + arguments for integer formats with a warning for backwards + compatibility. */ + +#define PY_STRUCT_FLOAT_COERCE 1 + +#ifdef PY_STRUCT_FLOAT_COERCE +#define FLOAT_COERCE "integer argument expected, got float" +#endif + + /* The translation function for each format character is table driven */ typedef struct _formatdef { char format; @@ -135,6 +146,21 @@ get_long(PyObject *v, long *p) { long x = PyInt_AsLong(v); if (x == -1 && PyErr_Occurred()) { +#ifdef PY_STRUCT_FLOAT_COERCE + if (PyFloat_Check(v)) { + PyObject *o; + int res; + PyErr_Clear(); + if (PyErr_WarnEx(PyExc_DeprecationWarning, FLOAT_COERCE, 2) < 0) + return -1; + o = PyNumber_Int(v); + if (o == NULL) + return -1; + res = get_long(o, p); + Py_DECREF(o); + return res; + } +#endif if (PyErr_ExceptionMatches(PyExc_TypeError)) PyErr_SetString(StructError, "required argument is not an integer"); @@ -214,15 +240,33 @@ get_ulonglong(PyObject *v, unsigned PY_LONG_LONG *p) /* Helper routine to get a Python integer and raise the appropriate error if it isn't one */ +#define INT_OVERFLOW "struct integer overflow masking is deprecated" + static int get_wrapped_long(PyObject *v, long *p) { if (get_long(v, p) < 0) { - if (PyLong_Check(v) && PyErr_ExceptionMatches(PyExc_OverflowError)) { + if (PyLong_Check(v) && + PyErr_ExceptionMatches(PyExc_OverflowError)) { PyObject *wrapped; long x; PyErr_Clear(); - if (PyErr_Warn(PyExc_DeprecationWarning, "struct integer overflow masking is deprecated") < 0) +#ifdef PY_STRUCT_FLOAT_COERCE + if (PyFloat_Check(v)) { + PyObject *o; + int res; + PyErr_Clear(); + if (PyErr_WarnEx(PyExc_DeprecationWarning, FLOAT_COERCE, 2) < 0) + return -1; + o = PyNumber_Int(v); + if (o == NULL) + return -1; + res = get_wrapped_long(o, p); + Py_DECREF(o); + return res; + } +#endif + if (PyErr_WarnEx(PyExc_DeprecationWarning, INT_OVERFLOW, 2) < 0) return -1; wrapped = PyNumber_And(v, pylong_ulong_mask); if (wrapped == NULL) @@ -246,10 +290,25 @@ get_wrapped_ulong(PyObject *v, unsigned long *p) if (x == -1 && PyErr_Occurred()) { PyObject *wrapped; PyErr_Clear(); +#ifdef PY_STRUCT_FLOAT_COERCE + if (PyFloat_Check(v)) { + PyObject *o; + int res; + PyErr_Clear(); + if (PyErr_WarnEx(PyExc_DeprecationWarning, FLOAT_COERCE, 2) < 0) + return -1; + o = PyNumber_Int(v); + if (o == NULL) + return -1; + res = get_wrapped_ulong(o, p); + Py_DECREF(o); + return res; + } +#endif wrapped = PyNumber_And(v, pylong_ulong_mask); if (wrapped == NULL) return -1; - if (PyErr_Warn(PyExc_DeprecationWarning, "struct integer overflow masking is deprecated") < 0) { + if (PyErr_WarnEx(PyExc_DeprecationWarning, INT_OVERFLOW, 2) < 0) { Py_DECREF(wrapped); return -1; } @@ -344,8 +403,8 @@ _range_error(const formatdef *f, int is_unsigned) Py_XDECREF(ptraceback); if (msg == NULL) return -1; - rval = PyErr_Warn(PyExc_DeprecationWarning, - PyString_AS_STRING(msg)); + rval = PyErr_WarnEx(PyExc_DeprecationWarning, + PyString_AS_STRING(msg), 2); Py_DECREF(msg); if (rval == 0) return 0; @@ -1396,23 +1455,17 @@ s_unpack_internal(PyStructObject *soself, char *startfrom) { const char *res = startfrom + code->offset; if (e->format == 's') { v = PyString_FromStringAndSize(res, code->size); - if (v == NULL) - goto fail; - PyTuple_SET_ITEM(result, i++, v); } else if (e->format == 'p') { Py_ssize_t n = *(unsigned char*)res; if (n >= code->size) n = code->size - 1; v = PyString_FromStringAndSize(res + 1, n); - if (v == NULL) - goto fail; - PyTuple_SET_ITEM(result, i++, v); } else { v = e->unpack(res, e); - if (v == NULL) - goto fail; - PyTuple_SET_ITEM(result, i++, v); } + if (v == NULL) + goto fail; + PyTuple_SET_ITEM(result, i++, v); } return result; @@ -1438,7 +1491,8 @@ s_unpack(PyObject *self, PyObject *inputstr) if (inputstr == NULL || !PyString_Check(inputstr) || PyString_GET_SIZE(inputstr) != soself->s_size) { PyErr_Format(StructError, - "unpack requires a string argument of length %zd", soself->s_size); + "unpack requires a string argument of length %zd", + soself->s_size); return NULL; } return s_unpack_internal(soself, PyString_AS_STRING(inputstr)); @@ -1504,17 +1558,18 @@ static int s_pack_internal(PyStructObject *soself, PyObject *args, int offset, char* buf) { formatcode *code; + /* XXX(nnorwitz): why does i need to be a local? can we use + the offset parameter or do we need the wider width? */ Py_ssize_t i; memset(buf, '\0', soself->s_size); i = offset; for (code = soself->s_codes; code->fmtdef != NULL; code++) { Py_ssize_t n; - PyObject *v; + PyObject *v = PyTuple_GET_ITEM(args, i++); const formatdef *e = code->fmtdef; char *res = buf + code->offset; if (e->format == 's') { - v = PyTuple_GET_ITEM(args, i++); if (!PyString_Check(v)) { PyErr_SetString(StructError, "argument for 's' must be a string"); @@ -1526,7 +1581,6 @@ s_pack_internal(PyStructObject *soself, PyObject *args, int offset, char* buf) if (n > 0) memcpy(res, PyString_AS_STRING(v), n); } else if (e->format == 'p') { - v = PyTuple_GET_ITEM(args, i++); if (!PyString_Check(v)) { PyErr_SetString(StructError, "argument for 'p' must be a string"); @@ -1541,7 +1595,6 @@ s_pack_internal(PyStructObject *soself, PyObject *args, int offset, char* buf) n = 255; *res = Py_SAFE_DOWNCAST(n, Py_ssize_t, unsigned char); } else { - v = PyTuple_GET_ITEM(args, i++); if (e->pack(res, v, e) < 0) { if (PyLong_Check(v) && PyErr_ExceptionMatches(PyExc_OverflowError)) PyErr_SetString(StructError, @@ -1818,4 +1871,8 @@ init_struct(void) #ifdef PY_STRUCT_OVERFLOW_MASKING PyModule_AddIntConstant(m, "_PY_STRUCT_OVERFLOW_MASKING", 1); #endif +#ifdef PY_STRUCT_FLOAT_COERCE + PyModule_AddIntConstant(m, "_PY_STRUCT_FLOAT_COERCE", 1); +#endif + } diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 6b9dffd..f5f3ab2 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -294,6 +294,16 @@ test_L_code(PyObject *self) #endif /* ifdef HAVE_LONG_LONG */ +/* Test tuple argument processing */ +static PyObject * +getargs_tuple(PyObject *self, PyObject *args) +{ + int a, b, c; + if (!PyArg_ParseTuple(args, "i(ii)", &a, &b, &c)) + return NULL; + return Py_BuildValue("iii", a, b, c); +} + /* Functions to call PyArg_ParseTuple with integer format codes, and return the result. */ @@ -696,6 +706,13 @@ test_string_from_format(PyObject *self, PyObject *args) #undef CHECK_1_FORMAT } +/* This is here to provide a docstring for test_descr. */ +static PyObject * +test_with_docstring(PyObject *self) +{ + Py_RETURN_NONE; +} + static PyMethodDef TestMethods[] = { {"raise_exception", raise_exception, METH_VARARGS}, {"test_config", (PyCFunction)test_config, METH_NOARGS}, @@ -706,7 +723,10 @@ static PyMethodDef TestMethods[] = { {"test_k_code", (PyCFunction)test_k_code, METH_NOARGS}, {"test_null_strings", (PyCFunction)test_null_strings, METH_NOARGS}, {"test_string_from_format", (PyCFunction)test_string_from_format, METH_NOARGS}, + {"test_with_docstring", (PyCFunction)test_with_docstring, METH_NOARGS, + PyDoc_STR("This is a pretty normal docstring.")}, + {"getargs_tuple", getargs_tuple, METH_VARARGS}, {"getargs_b", getargs_b, METH_VARARGS}, {"getargs_B", getargs_B, METH_VARARGS}, {"getargs_H", getargs_H, METH_VARARGS}, diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c index cec83f5..088fbb9 100644 --- a/Modules/_tkinter.c +++ b/Modules/_tkinter.c @@ -932,12 +932,13 @@ AsObj(PyObject *value) #ifdef Py_USING_UNICODE else if (PyUnicode_Check(value)) { Py_UNICODE *inbuf = PyUnicode_AS_UNICODE(value); - int size = PyUnicode_GET_SIZE(value); + Py_ssize_t size = PyUnicode_GET_SIZE(value); /* This #ifdef assumes that Tcl uses UCS-2. See TCL_UTF_MAX test above. */ #if defined(Py_UNICODE_WIDE) && TCL_UTF_MAX == 3 Tcl_UniChar *outbuf; - int i; + Py_ssize_t i; + assert(size < size * sizeof(Tcl_UniChar)); outbuf = (Tcl_UniChar*)ckalloc(size * sizeof(Tcl_UniChar)); if (!outbuf) { PyErr_NoMemory(); @@ -2103,8 +2104,8 @@ Tkapp_CreateCommand(PyObject *selfptr, PyObject *args) data = PyMem_NEW(PythonCmd_ClientData, 1); if (!data) return PyErr_NoMemory(); - Py_XINCREF(self); - Py_XINCREF(func); + Py_INCREF(self); + Py_INCREF(func); data->self = selfptr; data->func = func; diff --git a/Modules/_typesmodule.c b/Modules/_typesmodule.c new file mode 100644 index 0000000..5a6f2b9 --- /dev/null +++ b/Modules/_typesmodule.c @@ -0,0 +1,94 @@ +/* This extension module exposes some types that are only available at the + * C level. It should not be used directly, but instead through the Python + * level types modules, which imports this. + */ + +#include "Python.h" +#include "structmember.h" + +typedef struct +{ + PyObject_HEAD + int member; +} Helper; + +static PyMemberDef helper_members[] = { + { "member", T_INT, offsetof(Helper, member), READONLY, + PyDoc_STR("A member descriptor") + }, + { NULL } +}; + +static PyObject * +helper_getter(Helper *self, void *unused) +{ + Py_RETURN_NONE; +} + +static PyGetSetDef helper_getset[] = { + { "getter", (getter)helper_getter, NULL, + PyDoc_STR("A getset descriptor"), + }, + { NULL } +}; + +static PyTypeObject HelperType = { + PyObject_HEAD_INIT(NULL) + 0, /* ob_size */ + "_types.Helper", /* tp_name */ + sizeof(Helper), /* tp_basicsize */ + 0, /* tp_itemsize */ + 0, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /* tp_flags */ + 0, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + 0, /* tp_methods */ + helper_members, /* tp_members */ + helper_getset, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + 0, /* tp_new */ + 0, /* tp_free */ +}; + +PyMODINIT_FUNC +init_types(void) +{ + PyObject *m; + + m = Py_InitModule3("_types", NULL, "A types module helper"); + if (!m) + return; + + if (PyType_Ready(&HelperType) < 0) + return; + + Py_INCREF(&HelperType); + PyModule_AddObject(m, "Helper", (PyObject *)&HelperType); +} + + diff --git a/Modules/_weakref.c b/Modules/_weakref.c index 2dfdc14..1712f12 100644 --- a/Modules/_weakref.c +++ b/Modules/_weakref.c @@ -17,7 +17,7 @@ weakref_getweakrefcount(PyObject *self, PyObject *object) if (PyType_SUPPORTS_WEAKREFS(object->ob_type)) { PyWeakReference **list = GET_WEAKREFS_LISTPTR(object); - result = PyInt_FromLong(_PyWeakref_GetWeakrefCount(*list)); + result = PyInt_FromSsize_t(_PyWeakref_GetWeakrefCount(*list)); } else result = PyInt_FromLong(0); @@ -37,12 +37,12 @@ weakref_getweakrefs(PyObject *self, PyObject *object) if (PyType_SUPPORTS_WEAKREFS(object->ob_type)) { PyWeakReference **list = GET_WEAKREFS_LISTPTR(object); - long count = _PyWeakref_GetWeakrefCount(*list); + Py_ssize_t count = _PyWeakref_GetWeakrefCount(*list); result = PyList_New(count); if (result != NULL) { PyWeakReference *current = *list; - long i; + Py_ssize_t i; for (i = 0; i < count; ++i) { PyList_SET_ITEM(result, i, (PyObject *) current); Py_INCREF(current); diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c index 100b341..b739cc8 100644 --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -10,9 +10,9 @@ #ifdef STDC_HEADERS #include #else /* !STDC_HEADERS */ -#ifndef DONT_HAVE_SYS_TYPES_H +#ifdef HAVE_SYS_TYPES_H #include /* For size_t */ -#endif /* DONT_HAVE_SYS_TYPES_H */ +#endif /* HAVE_SYS_TYPES_H */ #endif /* !STDC_HEADERS */ struct arrayobject; /* Forward */ @@ -1785,6 +1785,7 @@ static PyBufferProcs array_as_buffer = { (readbufferproc)array_buffer_getreadbuf, (writebufferproc)array_buffer_getwritebuf, (segcountproc)array_buffer_getsegcount, + NULL, }; static PyObject * diff --git a/Modules/binascii.c b/Modules/binascii.c index 71a9624..3b2c8b2 100644 --- a/Modules/binascii.c +++ b/Modules/binascii.c @@ -909,7 +909,7 @@ binascii_hexlify(PyObject *self, PyObject *args) char* retbuf; Py_ssize_t i, j; - if (!PyArg_ParseTuple(args, "t#:b2a_hex", &argbuf, &arglen)) + if (!PyArg_ParseTuple(args, "s#:b2a_hex", &argbuf, &arglen)) return NULL; retval = PyString_FromStringAndSize(NULL, arglen*2); diff --git a/Modules/bz2module.c b/Modules/bz2module.c index 9576895..b0075a0 100644 --- a/Modules/bz2module.c +++ b/Modules/bz2module.c @@ -1303,7 +1303,11 @@ BZ2File_init(BZ2FileObject *self, PyObject *args, PyObject *kwargs) break; case 'U': +#ifdef __VMS + self->f_univ_newline = 0; +#else self->f_univ_newline = 1; +#endif break; default: @@ -1336,8 +1340,10 @@ BZ2File_init(BZ2FileObject *self, PyObject *args, PyObject *kwargs) #ifdef WITH_THREAD self->lock = PyThread_allocate_lock(); - if (!self->lock) + if (!self->lock) { + PyErr_SetString(PyExc_MemoryError, "unable to allocate lock"); goto error; + } #endif if (mode_char == 'r') @@ -1359,10 +1365,12 @@ BZ2File_init(BZ2FileObject *self, PyObject *args, PyObject *kwargs) return 0; error: - Py_DECREF(self->file); + Py_CLEAR(self->file); #ifdef WITH_THREAD - if (self->lock) + if (self->lock) { PyThread_free_lock(self->lock); + self->lock = NULL; + } #endif return -1; } @@ -1562,7 +1570,7 @@ BZ2Comp_compress(BZ2CompObject *self, PyObject *args) } } - _PyString_Resize(&ret, (int)(BZS_TOTAL_OUT(bzs) - totalout)); + _PyString_Resize(&ret, (Py_ssize_t)(BZS_TOTAL_OUT(bzs) - totalout)); RELEASE_LOCK(self); return ret; @@ -1628,7 +1636,7 @@ BZ2Comp_flush(BZ2CompObject *self) } if (bzs->avail_out != 0) - _PyString_Resize(&ret, (int)(BZS_TOTAL_OUT(bzs) - totalout)); + _PyString_Resize(&ret, (Py_ssize_t)(BZS_TOTAL_OUT(bzs) - totalout)); RELEASE_LOCK(self); return ret; @@ -1670,8 +1678,10 @@ BZ2Comp_init(BZ2CompObject *self, PyObject *args, PyObject *kwargs) #ifdef WITH_THREAD self->lock = PyThread_allocate_lock(); - if (!self->lock) + if (!self->lock) { + PyErr_SetString(PyExc_MemoryError, "unable to allocate lock"); goto error; + } #endif memset(&self->bzs, 0, sizeof(bz_stream)); @@ -1686,8 +1696,10 @@ BZ2Comp_init(BZ2CompObject *self, PyObject *args, PyObject *kwargs) return 0; error: #ifdef WITH_THREAD - if (self->lock) + if (self->lock) { PyThread_free_lock(self->lock); + self->lock = NULL; + } #endif return -1; } @@ -1852,7 +1864,7 @@ BZ2Decomp_decompress(BZ2DecompObject *self, PyObject *args) } if (bzs->avail_out != 0) - _PyString_Resize(&ret, (int)(BZS_TOTAL_OUT(bzs) - totalout)); + _PyString_Resize(&ret, (Py_ssize_t)(BZS_TOTAL_OUT(bzs) - totalout)); RELEASE_LOCK(self); return ret; @@ -1882,8 +1894,10 @@ BZ2Decomp_init(BZ2DecompObject *self, PyObject *args, PyObject *kwargs) #ifdef WITH_THREAD self->lock = PyThread_allocate_lock(); - if (!self->lock) + if (!self->lock) { + PyErr_SetString(PyExc_MemoryError, "unable to allocate lock"); goto error; + } #endif self->unused_data = PyString_FromString(""); @@ -1903,10 +1917,12 @@ BZ2Decomp_init(BZ2DecompObject *self, PyObject *args, PyObject *kwargs) error: #ifdef WITH_THREAD - if (self->lock) + if (self->lock) { PyThread_free_lock(self->lock); + self->lock = NULL; + } #endif - Py_XDECREF(self->unused_data); + Py_CLEAR(self->unused_data); return -1; } @@ -2061,7 +2077,7 @@ bz2_compress(PyObject *self, PyObject *args, PyObject *kwargs) } if (bzs->avail_out != 0) - _PyString_Resize(&ret, (int)BZS_TOTAL_OUT(bzs)); + _PyString_Resize(&ret, (Py_ssize_t)BZS_TOTAL_OUT(bzs)); BZ2_bzCompressEnd(bzs); return ret; @@ -2140,7 +2156,7 @@ bz2_decompress(PyObject *self, PyObject *args) } if (bzs->avail_out != 0) - _PyString_Resize(&ret, (int)BZS_TOTAL_OUT(bzs)); + _PyString_Resize(&ret, (Py_ssize_t)BZS_TOTAL_OUT(bzs)); BZ2_bzDecompressEnd(bzs); return ret; diff --git a/Modules/cPickle.c b/Modules/cPickle.c index 0d29362..24c98cc 100644 --- a/Modules/cPickle.c +++ b/Modules/cPickle.c @@ -196,7 +196,7 @@ Pdata_clear(Pdata *self, int clearto) for (i = self->length, p = self->data + clearto; --i >= clearto; p++) { - Py_DECREF(*p); + Py_CLEAR(*p); } self->length = clearto; @@ -208,6 +208,7 @@ Pdata_grow(Pdata *self) { int bigger; size_t nbytes; + PyObject **tmp; bigger = self->size << 1; if (bigger <= 0) /* was 0, or new value overflows */ @@ -217,14 +218,14 @@ Pdata_grow(Pdata *self) nbytes = (size_t)bigger * sizeof(PyObject *); if (nbytes / sizeof(PyObject *) != (size_t)bigger) goto nomemory; - self->data = realloc(self->data, nbytes); - if (self->data == NULL) + tmp = realloc(self->data, nbytes); + if (tmp == NULL) goto nomemory; + self->data = tmp; self->size = bigger; return 0; nomemory: - self->size = 0; PyErr_NoMemory(); return -1; } @@ -2636,7 +2637,7 @@ Pickle_getvalue(Picklerobject *self, PyObject *args) if (ik >= lm || ik == 0) { PyErr_SetString(PicklingError, "Invalid get data"); - return NULL; + goto err; } if (have_get[ik]) /* with matching get */ rsize += ik < 256 ? 2 : 5; @@ -2648,7 +2649,7 @@ Pickle_getvalue(Picklerobject *self, PyObject *args) ) { PyErr_SetString(PicklingError, "Unexpected data in internal list"); - return NULL; + goto err; } else { /* put */ @@ -3400,11 +3401,11 @@ load_string(Unpicklerobject *self) /********************************************/ str = PyString_DecodeEscape(p, len, NULL, 0, NULL); + free(s); if (str) { PDATA_PUSH(self->stack, str, -1); res = 0; } - free(s); return res; insecure: @@ -3628,10 +3629,14 @@ Instance_New(PyObject *cls, PyObject *args) err: { - PyObject *tp, *v, *tb; + PyObject *tp, *v, *tb, *tmp_value; PyErr_Fetch(&tp, &v, &tb); - if ((r=PyTuple_Pack(3,v,cls,args))) { + tmp_value = v; + /* NULL occurs when there was a KeyboardInterrupt */ + if (tmp_value == NULL) + tmp_value = Py_None; + if ((r = PyTuple_Pack(3, tmp_value, cls, args))) { Py_XDECREF(v); v=r; } @@ -4159,6 +4164,7 @@ do_append(Unpicklerobject *self, int x) int list_len; slice=Pdata_popList(self->stack, x); + if (! slice) return -1; list_len = PyList_GET_SIZE(list); i=PyList_SetSlice(list, list_len, list_len, slice); Py_DECREF(slice); @@ -5163,6 +5169,9 @@ newUnpicklerobject(PyObject *f) if (!( self->memo = PyDict_New())) goto err; + if (!self->stack) + goto err; + Py_INCREF(f); self->file = f; diff --git a/Modules/collectionsmodule.c b/Modules/collectionsmodule.c index 0c7dca6..d761dce 100644 --- a/Modules/collectionsmodule.c +++ b/Modules/collectionsmodule.c @@ -10,7 +10,7 @@ /* The block length may be set to any number over 1. Larger numbers * reduce the number of calls to the memory allocator but take more * memory. Ideally, BLOCKLEN should be set with an eye to the - * length of a cache line. + * length of a cache line. */ #define BLOCKLEN 62 @@ -22,9 +22,9 @@ * element is at d.leftblock[leftindex] and its last element is at * d.rightblock[rightindex]; note that, unlike as for Python slice * indices, these indices are inclusive on both ends. By being inclusive - * on both ends, algorithms for left and right operations become + * on both ends, algorithms for left and right operations become * symmetrical which simplifies the design. - * + * * The list of blocks is never empty, so d.leftblock and d.rightblock * are never equal to NULL. * @@ -37,11 +37,11 @@ * d.leftindex == CENTER+1; and d.rightindex == CENTER. * Checking for d.len == 0 is the intended way to see whether d is empty. * - * Whenever d.leftblock == d.rightblock, + * Whenever d.leftblock == d.rightblock, * d.leftindex + d.len - 1 == d.rightindex. - * + * * However, when d.leftblock != d.rightblock, d.leftindex and d.rightindex - * become indices into distinct blocks and either may be larger than the + * become indices into distinct blocks and either may be larger than the * other. */ @@ -381,7 +381,7 @@ deque_remove(dequeobject *deque, PyObject *value) int cmp = PyObject_RichCompareBool(item, value, Py_EQ); if (deque->len != n) { - PyErr_SetString(PyExc_IndexError, + PyErr_SetString(PyExc_IndexError, "deque mutated during remove()."); return NULL; } @@ -920,7 +920,7 @@ dequeiter_next(dequeiterobject *it) "deque mutated during iteration"); return NULL; } - assert (!(it->b == it->deque->rightblock && + assert (!(it->b == it->deque->rightblock && it->index > it->deque->rightindex)); item = it->b->data[it->index]; @@ -1016,7 +1016,7 @@ dequereviter_next(dequeiterobject *it) "deque mutated during iteration"); return NULL; } - assert (!(it->b == it->deque->leftblock && + assert (!(it->b == it->deque->leftblock && it->index < it->deque->leftindex)); item = it->b->data[it->index]; @@ -1117,7 +1117,7 @@ defdict_copy(defdictobject *dd) static PyObject * defdict_reduce(defdictobject *dd) { - /* __reduce__ must returns a 5-tuple as follows: + /* __reduce__ must return a 5-tuple as follows: - factory function - tuple of args for the factory function @@ -1155,6 +1155,7 @@ defdict_reduce(defdictobject *dd) } result = PyTuple_Pack(5, dd->dict.ob_type, args, Py_None, Py_None, items); + Py_DECREF(items); Py_DECREF(args); return result; } diff --git a/Modules/config.c.in b/Modules/config.c.in index f811991..8c25eea 100644 --- a/Modules/config.c.in +++ b/Modules/config.c.in @@ -28,6 +28,7 @@ extern void PyMarshal_Init(void); extern void initimp(void); extern void initgc(void); extern void init_ast(void); +extern void init_types(void); struct _inittab _PyImport_Inittab[] = { @@ -42,6 +43,9 @@ struct _inittab _PyImport_Inittab[] = { /* This lives in Python/Python-ast.c */ {"_ast", init_ast}, + /* This lives in Python/_types.c */ + {"_types", init_types}, + /* These entries are here for sys.builtin_module_names */ {"__main__", NULL}, {"__builtin__", NULL}, diff --git a/Modules/cryptmodule.c b/Modules/cryptmodule.c index 050a356..6377f84 100644 --- a/Modules/cryptmodule.c +++ b/Modules/cryptmodule.c @@ -5,6 +5,9 @@ #include +#ifdef __VMS +#include +#endif /* Module crypt */ @@ -12,7 +15,9 @@ static PyObject *crypt_crypt(PyObject *self, PyObject *args) { char *word, *salt; +#ifndef __VMS extern char * crypt(const char *, const char *); +#endif if (!PyArg_ParseTuple(args, "ss:crypt", &word, &salt)) { return NULL; diff --git a/Modules/dlmodule.c b/Modules/dlmodule.c index 0955681..5622ed9 100644 --- a/Modules/dlmodule.c +++ b/Modules/dlmodule.c @@ -5,6 +5,10 @@ #include +#ifdef __VMS +#include +#endif + #ifndef RTLD_LAZY #define RTLD_LAZY 1 #endif @@ -77,8 +81,8 @@ dl_call(dlobject *xp, PyObject *args) long, long, long, long, long); long alist[10]; long res; - int i; - int n = PyTuple_Size(args); + Py_ssize_t i; + Py_ssize_t n = PyTuple_Size(args); if (n < 1) { PyErr_SetString(PyExc_TypeError, "at least a name is needed"); return NULL; @@ -186,6 +190,24 @@ dl_open(PyObject *self, PyObject *args) PyErr_SetString(Dlerror, dlerror()); return NULL; } +#ifdef __VMS + /* Under OpenVMS dlopen doesn't do any check, just save the name + * for later use, so we have to check if the file is readable, + * the name can be a logical or a file from SYS$SHARE. + */ + if (access(name, R_OK)) { + char fname[strlen(name) + 20]; + strcpy(fname, "SYS$SHARE:"); + strcat(fname, name); + strcat(fname, ".EXE"); + if (access(fname, R_OK)) { + dlclose(handle); + PyErr_SetString(Dlerror, + "File not found or protection violation"); + return NULL; + } + } +#endif return newdlobject(handle); } diff --git a/Modules/expat/Makefile.in b/Modules/expat/Makefile.in deleted file mode 100644 index 9790739..0000000 --- a/Modules/expat/Makefile.in +++ /dev/null @@ -1,158 +0,0 @@ -################################################################ -# Process this file with top-level configure script to produce Makefile -# -# Copyright 2000 Clark Cooper -# -# This file is part of EXPAT. -# -# EXPAT is free software; you can redistribute it and/or modify it -# under the terms of the License (based on the MIT/X license) contained -# in the file COPYING that comes with this distribution. -# -# EXPAT IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, -# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF -# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. -# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY -# CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, -# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE -# SOFTWARE OR THE USE OR OTHER DEALINGS IN EXPAT. -# - - -SHELL = @SHELL@ - -srcdir = @srcdir@ -top_srcdir = @top_srcdir@ -VPATH = @srcdir@ -prefix = @prefix@ -exec_prefix = @exec_prefix@ - -bindir = @bindir@ -sbindir = @sbindir@ -libexecdir = @libexecdir@ -datadir = @datadir@ -sysconfdir = @sysconfdir@ -sharedstatedir = @sharedstatedir@ -localstatedir = @localstatedir@ -libdir = @libdir@ -infodir = @infodir@ -mandir = @mandir@ -includedir = @includedir@ -oldincludedir = /usr/include - -subdir = lib - -top_builddir = .. - -INSTALL = @INSTALL@ -INSTALL_PROGRAM = @INSTALL_PROGRAM@ -INSTALL_SCRIPT = @INSTALL_SCRIPT@ -INSTALL_DATA = @INSTALL_DATA@ - -host_alias = @host_alias@ -host_triplet = @host@ -AS = @AS@ -CC = @CC@ -DLLTOOL = @DLLTOOL@ -LIBTOOL = @LIBTOOL@ -LN_S = @LN_S@ -OBJDUMP = @OBJDUMP@ -PACKAGE = @PACKAGE@ -RANLIB = @RANLIB@ -VERSION = @VERSION@ - -LIBRARY = libexpat.la -SOURCES = xmlparse.c xmltok.c xmlrole.c -OBJECTS = $(SOURCES:.c=.o) -LTOBJECTS = $(SOURCES:.c=.lo) - -TEMPLATES = xmltok_impl.c xmltok_ns.c -APIHEADER = expat.h -HEADERS = ascii.h iasciitab.h utf8tab.h xmltok.h asciitab.h latin1tab.h \ - nametab.h xmldef.h xmlrole.h xmltok_impl.h - -mkinstalldirs = $(SHELL) $(top_srcdir)/conftools/mkinstalldirs -CONFIG_HEADER = ../config.h -CONFIG_CLEAN_FILES = - -INCLUDES = -I$(srcdir) -I. -I.. -DEFS = @DEFS@ -DPACKAGE='"$(PACKAGE)"' -DVERSION='"$(PACKAGE)_$(VERSION)"' - -CPPFLAGS = @CPPFLAGS@ -LDFLAGS = @LDFLAGS@ -LIBS = @LIBS@ -CFLAGS = @CFLAGS@ - -LIBREVISION = @LIBREVISION@ -LIBCURRENT = @LIBCURRENT@ -LIBAGE = @LIBAGE@ - -COMPILE = $(CC) $(DEFS) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -LTCOMPILE = $(LIBTOOL) --mode=compile $(CC) $(DEFS) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -CCLD = $(CC) -LINK = $(LIBTOOL) --mode=link $(CCLD) -version-info $(LIBCURRENT):$(LIBREVISION):$(LIBAGE) $(CFLAGS) $(LDFLAGS) -o $@ -DIST_COMMON = Makefile.in - - -DISTFILES = $(DIST_COMMON) $(SOURCES) $(TEMPLATES) $(APIHEADER) $(HEADERS) - -TAR = gtar -GZIP_ENV = --best - -all: $(LIBRARY) - -.SUFFIXES: .c .lo .o -.PHONY: all clean distclean maintainer-clean - -.c.o: - $(COMPILE) -c $< - -.c.lo: - $(LTCOMPILE) -c $< - -Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status - cd $(top_builddir) \ - && CONFIG_FILES=$(subdir)/$@ CONFIG_HEADERS= $(SHELL) ./config.status - -$(top_builddir)/config.status: $(top_builddir)/configure - cd $(top_builddir) && $(MAKE) config.status - -$(top_builddir)/config.h: $(top_builddir)/config.h.in - cd $(top_builddir) && $(MAKE) config.h - -clean: - rm -f $(LIBRARY) *.o *.lo *~ - rm -rf .libs _libs - -distclean: clean - rm -f Makefile - -maintainer-clean: distclean - -check: $(SUBDIRS) - @echo - @echo This package does not yet have a regression test. - @echo - -install: $(LIBRARY) $(APIHEADER) - $(mkinstalldirs) $(libdir) $(includedir) - $(LIBTOOL) --mode=install $(INSTALL) $(LIBRARY) $(libdir)/$(LIBRARY) - $(INSTALL_DATA) $(APIHEADER) $(includedir) - -uninstall: - $(LIBTOOL) --mode=uninstall rm -f $(libdir)/$(LIBRARY); - rm -f $(libdir)/$(APIHEADER) - -$(LIBRARY): $(LTOBJECTS) - $(LINK) -rpath $(libdir) $(LDFLAGS) $(LTOBJECTS) - -xmlparse.o \ -xmlparse.lo: xmlparse.c expat.h xmlrole.h xmltok.h $(top_builddir)/config.h - -xmlrole.o \ -xmlrole.lo: xmlrole.c ascii.h xmlrole.h $(top_builddir)/config.h - -xmltok.o \ -xmltok.lo: xmltok.c xmltok_impl.c xmltok_ns.c \ - ascii.h asciitab.h iasciitab.h latin1tab.h nametab.h utf8tab.h \ - xmltok.h xmltok_impl.h $(top_builddir)/config.h diff --git a/Modules/expat/amigaconfig.h b/Modules/expat/amigaconfig.h new file mode 100644 index 0000000..6781a71 --- /dev/null +++ b/Modules/expat/amigaconfig.h @@ -0,0 +1,96 @@ +#ifndef AMIGACONFIG_H +#define AMIGACONFIG_H + +/* 1234 = LIL_ENDIAN, 4321 = BIGENDIAN */ +#define BYTEORDER 4321 + +/* Define to 1 if you have the `bcopy' function. */ +#define HAVE_BCOPY 1 + +/* Define to 1 if you have the header file. */ +#undef HAVE_CHECK_H + +/* Define to 1 if you have the header file. */ +#undef HAVE_DLFCN_H + +/* Define to 1 if you have the header file. */ +#define HAVE_FCNTL_H 1 + +/* Define to 1 if you have the `getpagesize' function. */ +#undef HAVE_GETPAGESIZE + +/* Define to 1 if you have the header file. */ +#define HAVE_INTTYPES_H 1 + +/* Define to 1 if you have the `memmove' function. */ +#define HAVE_MEMMOVE 1 + +/* Define to 1 if you have the header file. */ +#undef HAVE_MEMORY_H + +/* Define to 1 if you have a working `mmap' system call. */ +#undef HAVE_MMAP + +/* Define to 1 if you have the header file. */ +#define HAVE_STDINT_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STDLIB_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STRINGS_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STRING_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_SYS_STAT_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_SYS_TYPES_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_UNISTD_H 1 + +/* Define to the address where bug reports for this package should be sent. */ +#define PACKAGE_BUGREPORT "expat-bugs@mail.libexpat.org" + +/* Define to the full name of this package. */ +#define PACKAGE_NAME "expat" + +/* Define to the full name and version of this package. */ +#define PACKAGE_STRING "expat 1.95.8" + +/* Define to the one symbol short name of this package. */ +#undef PACKAGE_TARNAME + +/* Define to the version of this package. */ +#define PACKAGE_VERSION "1.95.8" + +/* Define to 1 if you have the ANSI C header files. */ +#define STDC_HEADERS 1 + +/* whether byteorder is bigendian */ +#define WORDS_BIGENDIAN + +/* Define to specify how much context to retain around the current parse + point. */ +#define XML_CONTEXT_BYTES 1024 + +/* Define to make parameter entity parsing functionality available. */ +#define XML_DTD + +/* Define to make XML Namespaces functionality available. */ +#define XML_NS + +/* Define to empty if `const' does not conform to ANSI C. */ +#undef const + +/* Define to `long' if does not define. */ +#undef off_t + +/* Define to `unsigned' if does not define. */ +#undef size_t + + +#endif /* AMIGACONFIG_H */ diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index ecba92e..cf113ee 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -2,8 +2,8 @@ See the file COPYING for copying permission. */ -#ifndef XmlParse_INCLUDED -#define XmlParse_INCLUDED 1 +#ifndef Expat_INCLUDED +#define Expat_INCLUDED 1 #ifdef __VMS /* 0 1 2 3 0 1 2 3 @@ -17,6 +17,10 @@ #include #include "expat_external.h" +#ifdef __cplusplus +extern "C" { +#endif + struct XML_ParserStruct; typedef struct XML_ParserStruct *XML_Parser; @@ -87,7 +91,11 @@ enum XML_Error { XML_ERROR_NOT_SUSPENDED, XML_ERROR_ABORTED, XML_ERROR_FINISHED, - XML_ERROR_SUSPEND_PE + XML_ERROR_SUSPEND_PE, + /* Added in 2.0. */ + XML_ERROR_RESERVED_PREFIX_XML, + XML_ERROR_RESERVED_PREFIX_XMLNS, + XML_ERROR_RESERVED_NAMESPACE_URI }; enum XML_Content_Type { @@ -205,8 +213,8 @@ XML_ParserCreate(const XML_Char *encoding); URI, the namespace separator character, and the local part of the name. If the namespace separator is '\0' then the namespace URI and the local part will be concatenated without any separator. - When a namespace is not declared, the name and prefix will be - passed through without expansion. + It is a programming error to use the separator '\0' with namespace + triplets (see XML_SetReturnNSTriplet). */ XMLPARSEAPI(XML_Parser) XML_ParserCreateNS(const XML_Char *encoding, XML_Char namespaceSeparator); @@ -897,9 +905,9 @@ XML_GetErrorCode(XML_Parser parser); was detected; otherwise the location is the location of the last parse event, as described above. */ -XMLPARSEAPI(int) XML_GetCurrentLineNumber(XML_Parser parser); -XMLPARSEAPI(int) XML_GetCurrentColumnNumber(XML_Parser parser); -XMLPARSEAPI(long) XML_GetCurrentByteIndex(XML_Parser parser); +XMLPARSEAPI(XML_Size) XML_GetCurrentLineNumber(XML_Parser parser); +XMLPARSEAPI(XML_Size) XML_GetCurrentColumnNumber(XML_Parser parser); +XMLPARSEAPI(XML_Index) XML_GetCurrentByteIndex(XML_Parser parser); /* Return the number of bytes in the current event. Returns 0 if the event is in an internal entity. @@ -974,7 +982,8 @@ enum XML_FeatureEnum { XML_FEATURE_CONTEXT_BYTES, XML_FEATURE_MIN_SIZE, XML_FEATURE_SIZEOF_XML_CHAR, - XML_FEATURE_SIZEOF_XML_LCHAR + XML_FEATURE_SIZEOF_XML_LCHAR, + XML_FEATURE_NS /* Additional features must be added to the end of this enum. */ }; @@ -993,12 +1002,12 @@ XML_GetFeatureList(void); releases. Micro is bumped with each release, and set to 0 with each change to major or minor version. */ -#define XML_MAJOR_VERSION 1 -#define XML_MINOR_VERSION 95 -#define XML_MICRO_VERSION 8 +#define XML_MAJOR_VERSION 2 +#define XML_MINOR_VERSION 0 +#define XML_MICRO_VERSION 0 #ifdef __cplusplus } #endif -#endif /* not XmlParse_INCLUDED */ +#endif /* not Expat_INCLUDED */ diff --git a/Modules/expat/expat_external.h b/Modules/expat/expat_external.h index 4145cac..f054014 100644 --- a/Modules/expat/expat_external.h +++ b/Modules/expat/expat_external.h @@ -2,8 +2,15 @@ See the file COPYING for copying permission. */ +#ifndef Expat_External_INCLUDED +#define Expat_External_INCLUDED 1 + /* External API definitions */ +/* Namespace external symbols to allow multiple libexpat version to + co-exist. */ +#include "pyexpatns.h" + #if defined(_MSC_EXTENSIONS) && !defined(__BEOS__) && !defined(__CYGWIN__) #define XML_USE_MSC_EXTENSIONS 1 #endif @@ -62,6 +69,7 @@ #endif #endif /* not defined XML_STATIC */ + /* If we didn't define it above, define it away: */ #ifndef XMLIMPORT #define XMLIMPORT @@ -90,3 +98,22 @@ typedef char XML_LChar; typedef char XML_Char; typedef char XML_LChar; #endif /* XML_UNICODE */ + +#ifdef XML_LARGE_SIZE /* Use large integers for file/stream positions. */ +#if defined(XML_USE_MSC_EXTENSIONS) && _MSC_VER < 1400 +typedef __int64 XML_Index; +typedef unsigned __int64 XML_Size; +#else +typedef long long XML_Index; +typedef unsigned long long XML_Size; +#endif +#else +typedef long XML_Index; +typedef unsigned long XML_Size; +#endif /* XML_LARGE_SIZE */ + +#ifdef __cplusplus +} +#endif + +#endif /* not Expat_External_INCLUDED */ diff --git a/Modules/expat/pyexpatns.h b/Modules/expat/pyexpatns.h new file mode 100644 index 0000000..7fbd341 --- /dev/null +++ b/Modules/expat/pyexpatns.h @@ -0,0 +1,124 @@ +/* Copyright (c) 2005-2006 ActiveState Software Inc. + * + * Namespace all expat exported symbols to avoid dynamic loading symbol + * collisions when embedding Python. + * + * The Problem: + * - you embed Python in some app + * - the app dynamically loads libexpat of version X + * - the embedded Python imports pyexpat (which was built against + * libexpat version X+n) + * --> pyexpat gets the expat symbols from the already loaded and *older* + * libexpat: crash (Specifically the crash we observed was in + * getting an old XML_ErrorString (from xmlparse.c) and then calling + * it with newer values in the XML_Error enum: + * + * // pyexpat.c, line 1970 + * ... + * // Added in Expat 1.95.7. + * MYCONST(XML_ERROR_UNBOUND_PREFIX); + * ... + * + * + * The Solution: + * Prefix all a exported symbols with "PyExpat_". This is similar to + * what Mozilla does for some common libs: + * http://lxr.mozilla.org/seamonkey/source/modules/libimg/png/mozpngconf.h#115 + * + * The list of relevant exported symbols can be had with this command: + * + nm pyexpat.so \ + | grep -v " [a-zBUA] " \ + | grep -v "_fini\|_init\|initpyexpat" + * + * If any of those symbols are NOT prefixed with "PyExpat_" then + * a #define should be added for it here. + */ + +#ifndef PYEXPATNS_H +#define PYEXPATNS_H + +#define XML_DefaultCurrent PyExpat_XML_DefaultCurrent +#define XML_ErrorString PyExpat_XML_ErrorString +#define XML_ExpatVersion PyExpat_XML_ExpatVersion +#define XML_ExpatVersionInfo PyExpat_XML_ExpatVersionInfo +#define XML_ExternalEntityParserCreate PyExpat_XML_ExternalEntityParserCreate +#define XML_FreeContentModel PyExpat_XML_FreeContentModel +#define XML_GetBase PyExpat_XML_GetBase +#define XML_GetBuffer PyExpat_XML_GetBuffer +#define XML_GetCurrentByteCount PyExpat_XML_GetCurrentByteCount +#define XML_GetCurrentByteIndex PyExpat_XML_GetCurrentByteIndex +#define XML_GetCurrentColumnNumber PyExpat_XML_GetCurrentColumnNumber +#define XML_GetCurrentLineNumber PyExpat_XML_GetCurrentLineNumber +#define XML_GetErrorCode PyExpat_XML_GetErrorCode +#define XML_GetFeatureList PyExpat_XML_GetFeatureList +#define XML_GetIdAttributeIndex PyExpat_XML_GetIdAttributeIndex +#define XML_GetInputContext PyExpat_XML_GetInputContext +#define XML_GetParsingStatus PyExpat_XML_GetParsingStatus +#define XML_GetSpecifiedAttributeCount PyExpat_XML_GetSpecifiedAttributeCount +#define XmlGetUtf16InternalEncoding PyExpat_XmlGetUtf16InternalEncoding +#define XmlGetUtf16InternalEncodingNS PyExpat_XmlGetUtf16InternalEncodingNS +#define XmlGetUtf8InternalEncoding PyExpat_XmlGetUtf8InternalEncoding +#define XmlGetUtf8InternalEncodingNS PyExpat_XmlGetUtf8InternalEncodingNS +#define XmlInitEncoding PyExpat_XmlInitEncoding +#define XmlInitEncodingNS PyExpat_XmlInitEncodingNS +#define XmlInitUnknownEncoding PyExpat_XmlInitUnknownEncoding +#define XmlInitUnknownEncodingNS PyExpat_XmlInitUnknownEncodingNS +#define XML_MemFree PyExpat_XML_MemFree +#define XML_MemMalloc PyExpat_XML_MemMalloc +#define XML_MemRealloc PyExpat_XML_MemRealloc +#define XML_Parse PyExpat_XML_Parse +#define XML_ParseBuffer PyExpat_XML_ParseBuffer +#define XML_ParserCreate PyExpat_XML_ParserCreate +#define XML_ParserCreate_MM PyExpat_XML_ParserCreate_MM +#define XML_ParserCreateNS PyExpat_XML_ParserCreateNS +#define XML_ParserFree PyExpat_XML_ParserFree +#define XML_ParserReset PyExpat_XML_ParserReset +#define XmlParseXmlDecl PyExpat_XmlParseXmlDecl +#define XmlParseXmlDeclNS PyExpat_XmlParseXmlDeclNS +#define XmlPrologStateInit PyExpat_XmlPrologStateInit +#define XmlPrologStateInitExternalEntity PyExpat_XmlPrologStateInitExternalEntity +#define XML_ResumeParser PyExpat_XML_ResumeParser +#define XML_SetAttlistDeclHandler PyExpat_XML_SetAttlistDeclHandler +#define XML_SetBase PyExpat_XML_SetBase +#define XML_SetCdataSectionHandler PyExpat_XML_SetCdataSectionHandler +#define XML_SetCharacterDataHandler PyExpat_XML_SetCharacterDataHandler +#define XML_SetCommentHandler PyExpat_XML_SetCommentHandler +#define XML_SetDefaultHandler PyExpat_XML_SetDefaultHandler +#define XML_SetDefaultHandlerExpand PyExpat_XML_SetDefaultHandlerExpand +#define XML_SetDoctypeDeclHandler PyExpat_XML_SetDoctypeDeclHandler +#define XML_SetElementDeclHandler PyExpat_XML_SetElementDeclHandler +#define XML_SetElementHandler PyExpat_XML_SetElementHandler +#define XML_SetEncoding PyExpat_XML_SetEncoding +#define XML_SetEndCdataSectionHandler PyExpat_XML_SetEndCdataSectionHandler +#define XML_SetEndDoctypeDeclHandler PyExpat_XML_SetEndDoctypeDeclHandler +#define XML_SetEndElementHandler PyExpat_XML_SetEndElementHandler +#define XML_SetEndNamespaceDeclHandler PyExpat_XML_SetEndNamespaceDeclHandler +#define XML_SetEntityDeclHandler PyExpat_XML_SetEntityDeclHandler +#define XML_SetExternalEntityRefHandler PyExpat_XML_SetExternalEntityRefHandler +#define XML_SetExternalEntityRefHandlerArg PyExpat_XML_SetExternalEntityRefHandlerArg +#define XML_SetNamespaceDeclHandler PyExpat_XML_SetNamespaceDeclHandler +#define XML_SetNotationDeclHandler PyExpat_XML_SetNotationDeclHandler +#define XML_SetNotStandaloneHandler PyExpat_XML_SetNotStandaloneHandler +#define XML_SetParamEntityParsing PyExpat_XML_SetParamEntityParsing +#define XML_SetProcessingInstructionHandler PyExpat_XML_SetProcessingInstructionHandler +#define XML_SetReturnNSTriplet PyExpat_XML_SetReturnNSTriplet +#define XML_SetSkippedEntityHandler PyExpat_XML_SetSkippedEntityHandler +#define XML_SetStartCdataSectionHandler PyExpat_XML_SetStartCdataSectionHandler +#define XML_SetStartDoctypeDeclHandler PyExpat_XML_SetStartDoctypeDeclHandler +#define XML_SetStartElementHandler PyExpat_XML_SetStartElementHandler +#define XML_SetStartNamespaceDeclHandler PyExpat_XML_SetStartNamespaceDeclHandler +#define XML_SetUnknownEncodingHandler PyExpat_XML_SetUnknownEncodingHandler +#define XML_SetUnparsedEntityDeclHandler PyExpat_XML_SetUnparsedEntityDeclHandler +#define XML_SetUserData PyExpat_XML_SetUserData +#define XML_SetXmlDeclHandler PyExpat_XML_SetXmlDeclHandler +#define XmlSizeOfUnknownEncoding PyExpat_XmlSizeOfUnknownEncoding +#define XML_StopParser PyExpat_XML_StopParser +#define XML_UseForeignDTD PyExpat_XML_UseForeignDTD +#define XML_UseParserAsHandlerArg PyExpat_XML_UseParserAsHandlerArg +#define XmlUtf16Encode PyExpat_XmlUtf16Encode +#define XmlUtf8Encode PyExpat_XmlUtf8Encode + + +#endif /* !PYEXPATNS_H */ + diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 42d95b7..882470d 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -8,6 +8,8 @@ #include "winconfig.h" #elif defined(MACOS_CLASSIC) #include "macconfig.h" +#elif defined(__amigaos4__) +#include "amigaconfig.h" #elif defined(HAVE_EXPAT_CONFIG_H) #include #endif /* ndef COMPILED_FROM_DSP */ @@ -456,7 +458,7 @@ struct XML_ParserStruct { char *m_bufferEnd; /* allocated end of buffer */ const char *m_bufferLim; - long m_parseEndByteIndex; + XML_Index m_parseEndByteIndex; const char *m_parseEndPtr; XML_Char *m_dataBuf; XML_Char *m_dataBufEnd; @@ -640,8 +642,8 @@ struct XML_ParserStruct { #define groupSize (parser->m_groupSize) #define namespaceSeparator (parser->m_namespaceSeparator) #define parentParser (parser->m_parentParser) -#define parsing (parser->m_parsingStatus.parsing) -#define finalBuffer (parser->m_parsingStatus.finalBuffer) +#define ps_parsing (parser->m_parsingStatus.parsing) +#define ps_finalBuffer (parser->m_parsingStatus.finalBuffer) #ifdef XML_DTD #define isParamEntity (parser->m_isParamEntity) #define useForeignDTD (parser->m_useForeignDTD) @@ -852,7 +854,7 @@ parserInit(XML_Parser parser, const XML_Char *encodingName) unknownEncodingRelease = NULL; unknownEncodingData = NULL; parentParser = NULL; - parsing = XML_INITIALIZED; + ps_parsing = XML_INITIALIZED; #ifdef XML_DTD isParamEntity = XML_FALSE; useForeignDTD = XML_FALSE; @@ -915,7 +917,7 @@ XML_SetEncoding(XML_Parser parser, const XML_Char *encodingName) XXX There's no way for the caller to determine which of the XXX possible error cases caused the XML_STATUS_ERROR return. */ - if (parsing == XML_PARSING || parsing == XML_SUSPENDED) + if (ps_parsing == XML_PARSING || ps_parsing == XML_SUSPENDED) return XML_STATUS_ERROR; if (encodingName == NULL) protocolEncodingName = NULL; @@ -1143,7 +1145,7 @@ XML_UseForeignDTD(XML_Parser parser, XML_Bool useDTD) { #ifdef XML_DTD /* block after XML_Parse()/XML_ParseBuffer() has been called */ - if (parsing == XML_PARSING || parsing == XML_SUSPENDED) + if (ps_parsing == XML_PARSING || ps_parsing == XML_SUSPENDED) return XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING; useForeignDTD = useDTD; return XML_ERROR_NONE; @@ -1156,7 +1158,7 @@ void XMLCALL XML_SetReturnNSTriplet(XML_Parser parser, int do_nst) { /* block after XML_Parse()/XML_ParseBuffer() has been called */ - if (parsing == XML_PARSING || parsing == XML_SUSPENDED) + if (ps_parsing == XML_PARSING || ps_parsing == XML_SUSPENDED) return; ns_triplets = do_nst ? XML_TRUE : XML_FALSE; } @@ -1408,7 +1410,7 @@ XML_SetParamEntityParsing(XML_Parser parser, enum XML_ParamEntityParsing peParsing) { /* block after XML_Parse()/XML_ParseBuffer() has been called */ - if (parsing == XML_PARSING || parsing == XML_SUSPENDED) + if (ps_parsing == XML_PARSING || ps_parsing == XML_SUSPENDED) return 0; #ifdef XML_DTD paramEntityParsing = peParsing; @@ -1421,7 +1423,7 @@ XML_SetParamEntityParsing(XML_Parser parser, enum XML_Status XMLCALL XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: errorCode = XML_ERROR_SUSPENDED; return XML_STATUS_ERROR; @@ -1429,11 +1431,11 @@ XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) errorCode = XML_ERROR_FINISHED; return XML_STATUS_ERROR; default: - parsing = XML_PARSING; + ps_parsing = XML_PARSING; } if (len == 0) { - finalBuffer = (XML_Bool)isFinal; + ps_finalBuffer = (XML_Bool)isFinal; if (!isFinal) return XML_STATUS_OK; positionPtr = bufferPtr; @@ -1441,19 +1443,19 @@ XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) /* If data are left over from last buffer, and we now know that these data are the final chunk of input, then we have to check them again - to detect errors based on this information. + to detect errors based on that fact. */ errorCode = processor(parser, bufferPtr, parseEndPtr, &bufferPtr); if (errorCode == XML_ERROR_NONE) { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: XmlUpdatePosition(encoding, positionPtr, bufferPtr, &position); positionPtr = bufferPtr; return XML_STATUS_SUSPENDED; case XML_INITIALIZED: case XML_PARSING: - parsing = XML_FINISHED; + ps_parsing = XML_FINISHED; /* fall through */ default: return XML_STATUS_OK; @@ -1470,7 +1472,7 @@ XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) enum XML_Error result; parseEndByteIndex += len; positionPtr = s; - finalBuffer = (XML_Bool)isFinal; + ps_finalBuffer = (XML_Bool)isFinal; errorCode = processor(parser, s, parseEndPtr = s + len, &end); @@ -1480,7 +1482,7 @@ XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) return XML_STATUS_ERROR; } else { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: result = XML_STATUS_SUSPENDED; break; @@ -1488,14 +1490,13 @@ XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) case XML_PARSING: result = XML_STATUS_OK; if (isFinal) { - parsing = XML_FINISHED; + ps_parsing = XML_FINISHED; return result; } } } XmlUpdatePosition(encoding, positionPtr, end, &position); - positionPtr = end; nLeftOver = s + len - end; if (nLeftOver) { if (buffer == NULL || nLeftOver > bufferLim - buffer) { @@ -1518,9 +1519,13 @@ XML_Parse(XML_Parser parser, const char *s, int len, int isFinal) bufferLim = buffer + len * 2; } memcpy(buffer, end, nLeftOver); - bufferPtr = buffer; - bufferEnd = buffer + nLeftOver; } + bufferPtr = buffer; + bufferEnd = buffer + nLeftOver; + positionPtr = bufferPtr; + parseEndPtr = bufferEnd; + eventPtr = bufferPtr; + eventEndPtr = bufferPtr; return result; } #endif /* not defined XML_CONTEXT_BYTES */ @@ -1541,7 +1546,7 @@ XML_ParseBuffer(XML_Parser parser, int len, int isFinal) const char *start; enum XML_Status result = XML_STATUS_OK; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: errorCode = XML_ERROR_SUSPENDED; return XML_STATUS_ERROR; @@ -1549,7 +1554,7 @@ XML_ParseBuffer(XML_Parser parser, int len, int isFinal) errorCode = XML_ERROR_FINISHED; return XML_STATUS_ERROR; default: - parsing = XML_PARSING; + ps_parsing = XML_PARSING; } start = bufferPtr; @@ -1557,7 +1562,7 @@ XML_ParseBuffer(XML_Parser parser, int len, int isFinal) bufferEnd += len; parseEndPtr = bufferEnd; parseEndByteIndex += len; - finalBuffer = (XML_Bool)isFinal; + ps_finalBuffer = (XML_Bool)isFinal; errorCode = processor(parser, start, parseEndPtr, &bufferPtr); @@ -1567,14 +1572,14 @@ XML_ParseBuffer(XML_Parser parser, int len, int isFinal) return XML_STATUS_ERROR; } else { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: result = XML_STATUS_SUSPENDED; break; case XML_INITIALIZED: case XML_PARSING: if (isFinal) { - parsing = XML_FINISHED; + ps_parsing = XML_FINISHED; return result; } default: ; /* should not happen */ @@ -1589,7 +1594,7 @@ XML_ParseBuffer(XML_Parser parser, int len, int isFinal) void * XMLCALL XML_GetBuffer(XML_Parser parser, int len) { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: errorCode = XML_ERROR_SUSPENDED; return NULL; @@ -1601,9 +1606,9 @@ XML_GetBuffer(XML_Parser parser, int len) if (len > bufferLim - bufferEnd) { /* FIXME avoid integer overflow */ - int neededSize = len + (bufferEnd - bufferPtr); + int neededSize = len + (int)(bufferEnd - bufferPtr); #ifdef XML_CONTEXT_BYTES - int keep = bufferPtr - buffer; + int keep = (int)(bufferPtr - buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; @@ -1612,7 +1617,7 @@ XML_GetBuffer(XML_Parser parser, int len) if (neededSize <= bufferLim - buffer) { #ifdef XML_CONTEXT_BYTES if (keep < bufferPtr - buffer) { - int offset = (bufferPtr - buffer) - keep; + int offset = (int)(bufferPtr - buffer) - keep; memmove(buffer, &buffer[offset], bufferEnd - bufferPtr + keep); bufferEnd -= offset; bufferPtr -= offset; @@ -1625,7 +1630,7 @@ XML_GetBuffer(XML_Parser parser, int len) } else { char *newBuf; - int bufferSize = bufferLim - bufferPtr; + int bufferSize = (int)(bufferLim - bufferPtr); if (bufferSize == 0) bufferSize = INIT_BUFFER_SIZE; do { @@ -1639,7 +1644,7 @@ XML_GetBuffer(XML_Parser parser, int len) bufferLim = newBuf + bufferSize; #ifdef XML_CONTEXT_BYTES if (bufferPtr) { - int keep = bufferPtr - buffer; + int keep = (int)(bufferPtr - buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; memcpy(newBuf, &bufferPtr[-keep], bufferEnd - bufferPtr + keep); @@ -1668,13 +1673,13 @@ XML_GetBuffer(XML_Parser parser, int len) enum XML_Status XMLCALL XML_StopParser(XML_Parser parser, XML_Bool resumable) { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: if (resumable) { errorCode = XML_ERROR_SUSPENDED; return XML_STATUS_ERROR; } - parsing = XML_FINISHED; + ps_parsing = XML_FINISHED; break; case XML_FINISHED: errorCode = XML_ERROR_FINISHED; @@ -1687,10 +1692,10 @@ XML_StopParser(XML_Parser parser, XML_Bool resumable) return XML_STATUS_ERROR; } #endif - parsing = XML_SUSPENDED; + ps_parsing = XML_SUSPENDED; } else - parsing = XML_FINISHED; + ps_parsing = XML_FINISHED; } return XML_STATUS_OK; } @@ -1700,11 +1705,11 @@ XML_ResumeParser(XML_Parser parser) { enum XML_Status result = XML_STATUS_OK; - if (parsing != XML_SUSPENDED) { + if (ps_parsing != XML_SUSPENDED) { errorCode = XML_ERROR_NOT_SUSPENDED; return XML_STATUS_ERROR; } - parsing = XML_PARSING; + ps_parsing = XML_PARSING; errorCode = processor(parser, bufferPtr, parseEndPtr, &bufferPtr); @@ -1714,14 +1719,14 @@ XML_ResumeParser(XML_Parser parser) return XML_STATUS_ERROR; } else { - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: result = XML_STATUS_SUSPENDED; break; case XML_INITIALIZED: case XML_PARSING: - if (finalBuffer) { - parsing = XML_FINISHED; + if (ps_finalBuffer) { + ps_parsing = XML_FINISHED; return result; } default: ; @@ -1746,7 +1751,7 @@ XML_GetErrorCode(XML_Parser parser) return errorCode; } -long XMLCALL +XML_Index XMLCALL XML_GetCurrentByteIndex(XML_Parser parser) { if (eventPtr) @@ -1758,7 +1763,7 @@ int XMLCALL XML_GetCurrentByteCount(XML_Parser parser) { if (eventEndPtr && eventPtr) - return eventEndPtr - eventPtr; + return (int)(eventEndPtr - eventPtr); return 0; } @@ -1767,15 +1772,15 @@ XML_GetInputContext(XML_Parser parser, int *offset, int *size) { #ifdef XML_CONTEXT_BYTES if (eventPtr && buffer) { - *offset = eventPtr - buffer; - *size = bufferEnd - buffer; + *offset = (int)(eventPtr - buffer); + *size = (int)(bufferEnd - buffer); return buffer; } #endif /* defined XML_CONTEXT_BYTES */ return (char *) 0; } -int XMLCALL +XML_Size XMLCALL XML_GetCurrentLineNumber(XML_Parser parser) { if (eventPtr && eventPtr >= positionPtr) { @@ -1785,7 +1790,7 @@ XML_GetCurrentLineNumber(XML_Parser parser) return position.lineNumber + 1; } -int XMLCALL +XML_Size XMLCALL XML_GetCurrentColumnNumber(XML_Parser parser) { if (eventPtr && eventPtr >= positionPtr) { @@ -1836,7 +1841,7 @@ XML_DefaultCurrent(XML_Parser parser) const XML_LChar * XMLCALL XML_ErrorString(enum XML_Error code) { - static const XML_LChar *message[] = { + static const XML_LChar* const message[] = { 0, XML_L("out of memory"), XML_L("syntax error"), @@ -1854,7 +1859,7 @@ XML_ErrorString(enum XML_Error code) XML_L("reference to invalid character number"), XML_L("reference to binary entity"), XML_L("reference to external entity in attribute"), - XML_L("xml declaration not at start of external entity"), + XML_L("XML or text declaration not at start of entity"), XML_L("unknown encoding"), XML_L("encoding specified in XML declaration is incorrect"), XML_L("unclosed CDATA section"), @@ -1874,7 +1879,10 @@ XML_ErrorString(enum XML_Error code) XML_L("parser not suspended"), XML_L("parsing aborted"), XML_L("parsing finished"), - XML_L("cannot suspend in external parameter entity") + XML_L("cannot suspend in external parameter entity"), + XML_L("reserved prefix (xml) must not be undeclared or bound to another namespace name"), + XML_L("reserved prefix (xmlns) must not be declared or undeclared"), + XML_L("prefix must not be bound to one of the reserved namespace names") }; if (code > 0 && code < sizeof(message)/sizeof(message[0])) return message[code]; @@ -1916,9 +1924,11 @@ XML_ExpatVersionInfo(void) const XML_Feature * XMLCALL XML_GetFeatureList(void) { - static XML_Feature features[] = { - {XML_FEATURE_SIZEOF_XML_CHAR, XML_L("sizeof(XML_Char)"), 0}, - {XML_FEATURE_SIZEOF_XML_LCHAR, XML_L("sizeof(XML_LChar)"), 0}, + static const XML_Feature features[] = { + {XML_FEATURE_SIZEOF_XML_CHAR, XML_L("sizeof(XML_Char)"), + sizeof(XML_Char)}, + {XML_FEATURE_SIZEOF_XML_LCHAR, XML_L("sizeof(XML_LChar)"), + sizeof(XML_LChar)}, #ifdef XML_UNICODE {XML_FEATURE_UNICODE, XML_L("XML_UNICODE"), 0}, #endif @@ -1935,11 +1945,12 @@ XML_GetFeatureList(void) #ifdef XML_MIN_SIZE {XML_FEATURE_MIN_SIZE, XML_L("XML_MIN_SIZE"), 0}, #endif +#ifdef XML_NS + {XML_FEATURE_NS, XML_L("XML_NS"), 0}, +#endif {XML_FEATURE_END, NULL, 0} }; - features[0].value = sizeof(XML_Char); - features[1].value = sizeof(XML_LChar); return features; } @@ -2000,7 +2011,7 @@ contentProcessor(XML_Parser parser, const char **endPtr) { enum XML_Error result = doContent(parser, 0, encoding, start, end, - endPtr, (XML_Bool)!finalBuffer); + endPtr, (XML_Bool)!ps_finalBuffer); if (result == XML_ERROR_NONE) { if (!storeRawNames(parser)) return XML_ERROR_NO_MEMORY; @@ -2036,21 +2047,21 @@ externalEntityInitProcessor2(XML_Parser parser, doContent (by detecting XML_TOK_NONE) without processing any xml text declaration - causing the error XML_ERROR_MISPLACED_XML_PI in doContent. */ - if (next == end && !finalBuffer) { + if (next == end && !ps_finalBuffer) { *endPtr = next; return XML_ERROR_NONE; } start = next; break; case XML_TOK_PARTIAL: - if (!finalBuffer) { + if (!ps_finalBuffer) { *endPtr = start; return XML_ERROR_NONE; } eventPtr = start; return XML_ERROR_UNCLOSED_TOKEN; case XML_TOK_PARTIAL_CHAR: - if (!finalBuffer) { + if (!ps_finalBuffer) { *endPtr = start; return XML_ERROR_NONE; } @@ -2080,7 +2091,7 @@ externalEntityInitProcessor3(XML_Parser parser, result = processXmlDecl(parser, 1, start, next); if (result != XML_ERROR_NONE) return result; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *endPtr = next; return XML_ERROR_NONE; @@ -2092,13 +2103,13 @@ externalEntityInitProcessor3(XML_Parser parser, } break; case XML_TOK_PARTIAL: - if (!finalBuffer) { + if (!ps_finalBuffer) { *endPtr = start; return XML_ERROR_NONE; } return XML_ERROR_UNCLOSED_TOKEN; case XML_TOK_PARTIAL_CHAR: - if (!finalBuffer) { + if (!ps_finalBuffer) { *endPtr = start; return XML_ERROR_NONE; } @@ -2116,7 +2127,7 @@ externalEntityContentProcessor(XML_Parser parser, const char **endPtr) { enum XML_Error result = doContent(parser, 1, encoding, start, end, - endPtr, (XML_Bool)!finalBuffer); + endPtr, (XML_Bool)!ps_finalBuffer); if (result == XML_ERROR_NONE) { if (!storeRawNames(parser)) return XML_ERROR_NO_MEMORY; @@ -2315,12 +2326,12 @@ doContent(XML_Parser parser, XmlConvert(enc, &fromPtr, rawNameEnd, (ICHAR **)&toPtr, (ICHAR *)tag->bufEnd - 1); - convLen = toPtr - (XML_Char *)tag->buf; + convLen = (int)(toPtr - (XML_Char *)tag->buf); if (fromPtr == rawNameEnd) { tag->name.strLen = convLen; break; } - bufSize = (tag->bufEnd - tag->buf) << 1; + bufSize = (int)(tag->bufEnd - tag->buf) << 1; { char *temp = (char *)REALLOC(tag->buf, bufSize); if (temp == NULL) @@ -2508,12 +2519,12 @@ doContent(XML_Parser parser, ICHAR *dataPtr = (ICHAR *)dataBuf; XmlConvert(enc, &s, end, &dataPtr, (ICHAR *)dataBufEnd); characterDataHandler(handlerArg, dataBuf, - dataPtr - (ICHAR *)dataBuf); + (int)(dataPtr - (ICHAR *)dataBuf)); } else characterDataHandler(handlerArg, (XML_Char *)s, - (XML_Char *)end - (XML_Char *)s); + (int)((XML_Char *)end - (XML_Char *)s)); } else if (defaultHandler) reportDefault(parser, enc, s, end); @@ -2538,7 +2549,7 @@ doContent(XML_Parser parser, XmlConvert(enc, &s, next, &dataPtr, (ICHAR *)dataBufEnd); *eventEndPP = s; characterDataHandler(handlerArg, dataBuf, - dataPtr - (ICHAR *)dataBuf); + (int)(dataPtr - (ICHAR *)dataBuf)); if (s == next) break; *eventPP = s; @@ -2547,7 +2558,7 @@ doContent(XML_Parser parser, else characterDataHandler(handlerArg, (XML_Char *)s, - (XML_Char *)next - (XML_Char *)s); + (int)((XML_Char *)next - (XML_Char *)s)); } else if (defaultHandler) reportDefault(parser, enc, s, next); @@ -2566,7 +2577,7 @@ doContent(XML_Parser parser, break; } *eventPP = s = next; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *nextPtr = next; return XML_ERROR_NONE; @@ -2822,7 +2833,7 @@ storeAtts(XML_Parser parser, const ENCODING *enc, } if (!step) step = PROBE_STEP(uriHash, mask, nsAttsPower); - j < step ? ( j += nsAttsSize - step) : (j -= step); + j < step ? (j += nsAttsSize - step) : (j -= step); } } @@ -2845,8 +2856,10 @@ storeAtts(XML_Parser parser, const ENCODING *enc, nsAtts[j].hash = uriHash; nsAtts[j].uriName = s; - if (!--nPrefixes) + if (!--nPrefixes) { + i += 2; break; + } } else /* not prefixed */ ((XML_Char *)s)[-1] = 0; /* clear flag */ @@ -2879,14 +2892,14 @@ storeAtts(XML_Parser parser, const ENCODING *enc, prefixLen = 0; if (ns_triplets && binding->prefix->name) { for (; binding->prefix->name[prefixLen++];) - ; + ; /* prefixLen includes null terminator */ } tagNamePtr->localPart = localPart; tagNamePtr->uriLen = binding->uriLen; tagNamePtr->prefix = binding->prefix->name; tagNamePtr->prefixLen = prefixLen; for (i = 0; localPart[i++];) - ; + ; /* i includes null terminator */ n = i + binding->uriLen + prefixLen; if (n > binding->uriAlloc) { TAG *p; @@ -2901,12 +2914,13 @@ storeAtts(XML_Parser parser, const ENCODING *enc, FREE(binding->uri); binding->uri = uri; } + /* if namespaceSeparator != '\0' then uri includes it already */ uri = binding->uri + binding->uriLen; memcpy(uri, localPart, i * sizeof(XML_Char)); + /* we always have a namespace separator between localPart and prefix */ if (prefixLen) { - uri = uri + (i - 1); - if (namespaceSeparator) - *uri = namespaceSeparator; + uri += i - 1; + *uri = namespaceSeparator; /* replace null terminator */ memcpy(uri + 1, binding->prefix->name, prefixLen * sizeof(XML_Char)); } tagNamePtr->str = binding->uri; @@ -2920,6 +2934,26 @@ static enum XML_Error addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, const XML_Char *uri, BINDING **bindingsPtr) { + static const XML_Char xmlNamespace[] = { + 'h', 't', 't', 'p', ':', '/', '/', + 'w', 'w', 'w', '.', 'w', '3', '.', 'o', 'r', 'g', '/', + 'X', 'M', 'L', '/', '1', '9', '9', '8', '/', + 'n', 'a', 'm', 'e', 's', 'p', 'a', 'c', 'e', '\0' + }; + static const int xmlLen = + (int)sizeof(xmlNamespace)/sizeof(XML_Char) - 1; + static const XML_Char xmlnsNamespace[] = { + 'h', 't', 't', 'p', ':', '/', '/', + 'w', 'w', 'w', '.', 'w', '3', '.', 'o', 'r', 'g', '/', + '2', '0', '0', '0', '/', 'x', 'm', 'l', 'n', 's', '/', '\0' + }; + static const int xmlnsLen = + (int)sizeof(xmlnsNamespace)/sizeof(XML_Char) - 1; + + XML_Bool mustBeXML = XML_FALSE; + XML_Bool isXML = XML_TRUE; + XML_Bool isXMLNS = XML_TRUE; + BINDING *b; int len; @@ -2927,8 +2961,39 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, if (*uri == XML_T('\0') && prefix->name) return XML_ERROR_UNDECLARING_PREFIX; - for (len = 0; uri[len]; len++) - ; + if (prefix->name + && prefix->name[0] == XML_T('x') + && prefix->name[1] == XML_T('m') + && prefix->name[2] == XML_T('l')) { + + /* Not allowed to bind xmlns */ + if (prefix->name[3] == XML_T('n') + && prefix->name[4] == XML_T('s') + && prefix->name[5] == XML_T('\0')) + return XML_ERROR_RESERVED_PREFIX_XMLNS; + + if (prefix->name[3] == XML_T('\0')) + mustBeXML = XML_TRUE; + } + + for (len = 0; uri[len]; len++) { + if (isXML && (len > xmlLen || uri[len] != xmlNamespace[len])) + isXML = XML_FALSE; + + if (!mustBeXML && isXMLNS + && (len > xmlnsLen || uri[len] != xmlnsNamespace[len])) + isXMLNS = XML_FALSE; + } + isXML = isXML && len == xmlLen; + isXMLNS = isXMLNS && len == xmlnsLen; + + if (mustBeXML != isXML) + return mustBeXML ? XML_ERROR_RESERVED_PREFIX_XML + : XML_ERROR_RESERVED_NAMESPACE_URI; + + if (isXMLNS) + return XML_ERROR_RESERVED_NAMESPACE_URI; + if (namespaceSeparator) len++; if (freeBindingList) { @@ -2985,7 +3050,7 @@ cdataSectionProcessor(XML_Parser parser, const char **endPtr) { enum XML_Error result = doCdataSection(parser, encoding, &start, end, - endPtr, (XML_Bool)!finalBuffer); + endPtr, (XML_Bool)!ps_finalBuffer); if (result != XML_ERROR_NONE) return result; if (start) { @@ -3044,7 +3109,7 @@ doCdataSection(XML_Parser parser, reportDefault(parser, enc, s, next); *startPtr = next; *nextPtr = next; - if (parsing == XML_FINISHED) + if (ps_parsing == XML_FINISHED) return XML_ERROR_ABORTED; else return XML_ERROR_NONE; @@ -3064,7 +3129,7 @@ doCdataSection(XML_Parser parser, XmlConvert(enc, &s, next, &dataPtr, (ICHAR *)dataBufEnd); *eventEndPP = next; characterDataHandler(handlerArg, dataBuf, - dataPtr - (ICHAR *)dataBuf); + (int)(dataPtr - (ICHAR *)dataBuf)); if (s == next) break; *eventPP = s; @@ -3073,7 +3138,7 @@ doCdataSection(XML_Parser parser, else characterDataHandler(handlerArg, (XML_Char *)s, - (XML_Char *)next - (XML_Char *)s); + (int)((XML_Char *)next - (XML_Char *)s)); } else if (defaultHandler) reportDefault(parser, enc, s, next); @@ -3100,7 +3165,7 @@ doCdataSection(XML_Parser parser, } *eventPP = s = next; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *nextPtr = next; return XML_ERROR_NONE; @@ -3124,7 +3189,7 @@ ignoreSectionProcessor(XML_Parser parser, const char **endPtr) { enum XML_Error result = doIgnoreSection(parser, encoding, &start, end, - endPtr, (XML_Bool)!finalBuffer); + endPtr, (XML_Bool)!ps_finalBuffer); if (result != XML_ERROR_NONE) return result; if (start) { @@ -3169,7 +3234,7 @@ doIgnoreSection(XML_Parser parser, reportDefault(parser, enc, s, next); *startPtr = next; *nextPtr = next; - if (parsing == XML_FINISHED) + if (ps_parsing == XML_FINISHED) return XML_ERROR_ABORTED; else return XML_ERROR_NONE; @@ -3409,7 +3474,7 @@ entityValueInitProcessor(XML_Parser parser, tok = XmlPrologTok(encoding, start, end, &next); eventEndPtr = next; if (tok <= 0) { - if (!finalBuffer && tok != XML_TOK_INVALID) { + if (!ps_finalBuffer && tok != XML_TOK_INVALID) { *nextPtr = s; return XML_ERROR_NONE; } @@ -3432,7 +3497,7 @@ entityValueInitProcessor(XML_Parser parser, result = processXmlDecl(parser, 0, start, next); if (result != XML_ERROR_NONE) return result; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *nextPtr = next; return XML_ERROR_NONE; @@ -3452,7 +3517,7 @@ entityValueInitProcessor(XML_Parser parser, then, when this routine is entered the next time, XmlPrologTok will return XML_TOK_INVALID, since the BOM is still in the buffer */ - else if (tok == XML_TOK_BOM && next == end && !finalBuffer) { + else if (tok == XML_TOK_BOM && next == end && !ps_finalBuffer) { *nextPtr = next; return XML_ERROR_NONE; } @@ -3472,7 +3537,7 @@ externalParEntProcessor(XML_Parser parser, tok = XmlPrologTok(encoding, s, end, &next); if (tok <= 0) { - if (!finalBuffer && tok != XML_TOK_INVALID) { + if (!ps_finalBuffer && tok != XML_TOK_INVALID) { *nextPtr = s; return XML_ERROR_NONE; } @@ -3499,7 +3564,7 @@ externalParEntProcessor(XML_Parser parser, processor = prologProcessor; return doProlog(parser, encoding, s, end, tok, next, - nextPtr, (XML_Bool)!finalBuffer); + nextPtr, (XML_Bool)!ps_finalBuffer); } static enum XML_Error PTRCALL @@ -3516,7 +3581,7 @@ entityValueProcessor(XML_Parser parser, for (;;) { tok = XmlPrologTok(enc, start, end, &next); if (tok <= 0) { - if (!finalBuffer && tok != XML_TOK_INVALID) { + if (!ps_finalBuffer && tok != XML_TOK_INVALID) { *nextPtr = s; return XML_ERROR_NONE; } @@ -3549,7 +3614,7 @@ prologProcessor(XML_Parser parser, const char *next = s; int tok = XmlPrologTok(encoding, s, end, &next); return doProlog(parser, encoding, s, end, tok, next, - nextPtr, (XML_Bool)!finalBuffer); + nextPtr, (XML_Bool)!ps_finalBuffer); } static enum XML_Error @@ -3738,7 +3803,8 @@ doProlog(XML_Parser parser, */ #ifdef XML_DTD if (doctypeSysid || useForeignDTD) { - dtd->hasParamEntityRefs = XML_TRUE; /* when docTypeSysid == NULL */ + XML_Bool hadParamEntityRefs = dtd->hasParamEntityRefs; + dtd->hasParamEntityRefs = XML_TRUE; if (paramEntityParsing && externalEntityRefHandler) { ENTITY *entity = (ENTITY *)lookup(&dtd->paramEntities, externalSubsetName, @@ -3754,11 +3820,17 @@ doProlog(XML_Parser parser, entity->systemId, entity->publicId)) return XML_ERROR_EXTERNAL_ENTITY_HANDLING; - if (dtd->paramEntityRead && - !dtd->standalone && - notStandaloneHandler && - !notStandaloneHandler(handlerArg)) - return XML_ERROR_NOT_STANDALONE; + if (dtd->paramEntityRead) { + if (!dtd->standalone && + notStandaloneHandler && + !notStandaloneHandler(handlerArg)) + return XML_ERROR_NOT_STANDALONE; + } + /* if we didn't read the foreign DTD then this means that there + is no external subset and we must reset dtd->hasParamEntityRefs + */ + else if (!doctypeSysid) + dtd->hasParamEntityRefs = hadParamEntityRefs; /* end of DTD - no need to update dtd->keepProcessing */ } useForeignDTD = XML_FALSE; @@ -3775,6 +3847,7 @@ doProlog(XML_Parser parser, last chance to read the foreign DTD */ if (useForeignDTD) { + XML_Bool hadParamEntityRefs = dtd->hasParamEntityRefs; dtd->hasParamEntityRefs = XML_TRUE; if (paramEntityParsing && externalEntityRefHandler) { ENTITY *entity = (ENTITY *)lookup(&dtd->paramEntities, @@ -3790,11 +3863,17 @@ doProlog(XML_Parser parser, entity->systemId, entity->publicId)) return XML_ERROR_EXTERNAL_ENTITY_HANDLING; - if (dtd->paramEntityRead && - !dtd->standalone && - notStandaloneHandler && - !notStandaloneHandler(handlerArg)) - return XML_ERROR_NOT_STANDALONE; + if (dtd->paramEntityRead) { + if (!dtd->standalone && + notStandaloneHandler && + !notStandaloneHandler(handlerArg)) + return XML_ERROR_NOT_STANDALONE; + } + /* if we didn't read the foreign DTD then this means that there + is no external subset and we must reset dtd->hasParamEntityRefs + */ + else + dtd->hasParamEntityRefs = hadParamEntityRefs; /* end of DTD - no need to update dtd->keepProcessing */ } } @@ -3935,7 +4014,7 @@ doProlog(XML_Parser parser, next - enc->minBytesPerChar); if (declEntity) { declEntity->textPtr = poolStart(&dtd->entityValuePool); - declEntity->textLen = poolLength(&dtd->entityValuePool); + declEntity->textLen = (int)(poolLength(&dtd->entityValuePool)); poolFinish(&dtd->entityValuePool); if (entityDeclHandler) { *eventEndPP = s; @@ -4496,7 +4575,7 @@ doProlog(XML_Parser parser, if (handleDefault && defaultHandler) reportDefault(parser, enc, s, next); - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *nextPtr = next; return XML_ERROR_NONE; @@ -4527,7 +4606,7 @@ epilogProcessor(XML_Parser parser, case -XML_TOK_PROLOG_S: if (defaultHandler) { reportDefault(parser, encoding, s, next); - if (parsing == XML_FINISHED) + if (ps_parsing == XML_FINISHED) return XML_ERROR_ABORTED; } *nextPtr = next; @@ -4551,13 +4630,13 @@ epilogProcessor(XML_Parser parser, eventPtr = next; return XML_ERROR_INVALID_TOKEN; case XML_TOK_PARTIAL: - if (!finalBuffer) { + if (!ps_finalBuffer) { *nextPtr = s; return XML_ERROR_NONE; } return XML_ERROR_UNCLOSED_TOKEN; case XML_TOK_PARTIAL_CHAR: - if (!finalBuffer) { + if (!ps_finalBuffer) { *nextPtr = s; return XML_ERROR_NONE; } @@ -4566,7 +4645,7 @@ epilogProcessor(XML_Parser parser, return XML_ERROR_JUNK_AFTER_DOC_ELEMENT; } eventPtr = s = next; - switch (parsing) { + switch (ps_parsing) { case XML_SUSPENDED: *nextPtr = next; return XML_ERROR_NONE; @@ -4619,8 +4698,8 @@ processInternalEntity(XML_Parser parser, ENTITY *entity, textEnd, &next, XML_FALSE); if (result == XML_ERROR_NONE) { - if (textEnd != next && parsing == XML_SUSPENDED) { - entity->processed = next - textStart; + if (textEnd != next && ps_parsing == XML_SUSPENDED) { + entity->processed = (int)(next - textStart); processor = internalEntityProcessor; } else { @@ -4665,8 +4744,8 @@ internalEntityProcessor(XML_Parser parser, if (result != XML_ERROR_NONE) return result; - else if (textEnd != next && parsing == XML_SUSPENDED) { - entity->processed = next - (char *)entity->textPtr; + else if (textEnd != next && ps_parsing == XML_SUSPENDED) { + entity->processed = (int)(next - (char *)entity->textPtr); return result; } else { @@ -4683,7 +4762,7 @@ internalEntityProcessor(XML_Parser parser, processor = prologProcessor; tok = XmlPrologTok(encoding, s, end, &next); return doProlog(parser, encoding, s, end, tok, next, nextPtr, - (XML_Bool)!finalBuffer); + (XML_Bool)!ps_finalBuffer); } else #endif /* XML_DTD */ @@ -4691,7 +4770,7 @@ internalEntityProcessor(XML_Parser parser, processor = contentProcessor; /* see externalEntityContentProcessor vs contentProcessor */ return doContent(parser, parentParser ? 1 : 0, encoding, s, end, - nextPtr, (XML_Bool)!finalBuffer); + nextPtr, (XML_Bool)!ps_finalBuffer); } } @@ -4800,9 +4879,8 @@ appendAttributeValue(XML_Parser parser, const ENCODING *enc, XML_Bool isCdata, return XML_ERROR_NO_MEMORY; entity = (ENTITY *)lookup(&dtd->generalEntities, name, 0); poolDiscard(&temp2Pool); - /* first, determine if a check for an existing declaration is needed; - if yes, check that the entity exists, and that it is internal, - otherwise call the default handler (if called from content) + /* First, determine if a check for an existing declaration is needed; + if yes, check that the entity exists, and that it is internal. */ if (pool == &dtd->pool) /* are we called from prolog? */ checkEntityDecl = @@ -4821,13 +4899,16 @@ appendAttributeValue(XML_Parser parser, const ENCODING *enc, XML_Bool isCdata, return XML_ERROR_ENTITY_DECLARED_IN_PE; } else if (!entity) { - /* cannot report skipped entity here - see comments on - skippedEntityHandler + /* Cannot report skipped entity here - see comments on + skippedEntityHandler. if (skippedEntityHandler) skippedEntityHandler(handlerArg, name, 0); */ + /* Cannot call the default handler because this would be + out of sync with the call to the startElementHandler. if ((pool == &tempPool) && defaultHandler) reportDefault(parser, enc, ptr, next); + */ break; } if (entity->open) { @@ -5127,12 +5208,12 @@ reportDefault(XML_Parser parser, const ENCODING *enc, ICHAR *dataPtr = (ICHAR *)dataBuf; XmlConvert(enc, &s, end, &dataPtr, (ICHAR *)dataBufEnd); *eventEndPP = s; - defaultHandler(handlerArg, dataBuf, dataPtr - (ICHAR *)dataBuf); + defaultHandler(handlerArg, dataBuf, (int)(dataPtr - (ICHAR *)dataBuf)); *eventPP = s; } while (s != end); } else - defaultHandler(handlerArg, (XML_Char *)s, (XML_Char *)end - (XML_Char *)s); + defaultHandler(handlerArg, (XML_Char *)s, (int)((XML_Char *)end - (XML_Char *)s)); } @@ -5286,7 +5367,7 @@ getContext(XML_Parser parser) if (!poolAppendChar(&tempPool, XML_T('='))) return NULL; len = dtd->defaultPrefix.binding->uriLen; - if (namespaceSeparator != XML_T('\0')) + if (namespaceSeparator) len--; for (i = 0; i < len; i++) if (!poolAppendChar(&tempPool, dtd->defaultPrefix.binding->uri[i])) @@ -5312,7 +5393,7 @@ getContext(XML_Parser parser) if (!poolAppendChar(&tempPool, XML_T('='))) return NULL; len = prefix->binding->uriLen; - if (namespaceSeparator != XML_T('\0')) + if (namespaceSeparator) len--; for (i = 0; i < len; i++) if (!poolAppendChar(&tempPool, prefix->binding->uri[i])) @@ -6014,7 +6095,7 @@ poolGrow(STRING_POOL *pool) } } if (pool->blocks && pool->start == pool->blocks->s) { - int blockSize = (pool->end - pool->start)*2; + int blockSize = (int)(pool->end - pool->start)*2; pool->blocks = (BLOCK *) pool->mem->realloc_fcn(pool->blocks, (offsetof(BLOCK, s) @@ -6028,7 +6109,7 @@ poolGrow(STRING_POOL *pool) } else { BLOCK *tem; - int blockSize = pool->end - pool->start; + int blockSize = (int)(pool->end - pool->start); if (blockSize < INIT_BLOCK_SIZE) blockSize = INIT_BLOCK_SIZE; else diff --git a/Modules/expat/xmlrole.c b/Modules/expat/xmlrole.c index 2587fdf..15d4d8f 100644 --- a/Modules/expat/xmlrole.c +++ b/Modules/expat/xmlrole.c @@ -6,6 +6,8 @@ #include "winconfig.h" #elif defined(MACOS_CLASSIC) #include "macconfig.h" +#elif defined(__amigaos4__) +#include "amigaconfig.h" #else #ifdef HAVE_EXPAT_CONFIG_H #include @@ -793,7 +795,7 @@ attlist2(PROLOG_STATE *state, return XML_ROLE_ATTLIST_NONE; case XML_TOK_NAME: { - static const char *types[] = { + static const char * const types[] = { KW_CDATA, KW_ID, KW_IDREF, diff --git a/Modules/expat/xmltok.c b/Modules/expat/xmltok.c index 8b9d997..db92247 100644 --- a/Modules/expat/xmltok.c +++ b/Modules/expat/xmltok.c @@ -6,6 +6,8 @@ #include "winconfig.h" #elif defined(MACOS_CLASSIC) #include "macconfig.h" +#elif defined(__amigaos4__) +#include "amigaconfig.h" #else #ifdef HAVE_EXPAT_CONFIG_H #include @@ -1451,7 +1453,7 @@ static const char KW_UTF_16LE[] = { static int FASTCALL getEncodingIndex(const char *name) { - static const char *encodingNames[] = { + static const char * const encodingNames[] = { KW_ISO_8859_1, KW_US_ASCII, KW_UTF_8, @@ -1484,7 +1486,7 @@ getEncodingIndex(const char *name) static int -initScan(const ENCODING **encodingTable, +initScan(const ENCODING * const *encodingTable, const INIT_ENCODING *enc, int state, const char *ptr, diff --git a/Modules/expat/xmltok.h b/Modules/expat/xmltok.h index 1ecd05f..ca867aa 100644 --- a/Modules/expat/xmltok.h +++ b/Modules/expat/xmltok.h @@ -111,8 +111,8 @@ extern "C" { typedef struct position { /* first line and first column are 0 not 1 */ - unsigned long lineNumber; - unsigned long columnNumber; + XML_Size lineNumber; + XML_Size columnNumber; } POSITION; typedef struct { diff --git a/Modules/expat/xmltok_impl.c b/Modules/expat/xmltok_impl.c index 46569fe..0ee57ab 100644 --- a/Modules/expat/xmltok_impl.c +++ b/Modules/expat/xmltok_impl.c @@ -1714,7 +1714,7 @@ PREFIX(nameLength)(const ENCODING *enc, const char *ptr) ptr += MINBPC(enc); break; default: - return ptr - start; + return (int)(ptr - start); } } } @@ -1750,7 +1750,7 @@ PREFIX(updatePosition)(const ENCODING *enc, LEAD_CASE(2) LEAD_CASE(3) LEAD_CASE(4) #undef LEAD_CASE case BT_LF: - pos->columnNumber = (unsigned)-1; + pos->columnNumber = (XML_Size)-1; pos->lineNumber++; ptr += MINBPC(enc); break; @@ -1759,7 +1759,7 @@ PREFIX(updatePosition)(const ENCODING *enc, ptr += MINBPC(enc); if (ptr != end && BYTE_TYPE(enc, ptr) == BT_LF) ptr += MINBPC(enc); - pos->columnNumber = (unsigned)-1; + pos->columnNumber = (XML_Size)-1; break; default: ptr += MINBPC(enc); diff --git a/Modules/expat/xmltok_ns.c b/Modules/expat/xmltok_ns.c index 5610eb9..d2f8938 100644 --- a/Modules/expat/xmltok_ns.c +++ b/Modules/expat/xmltok_ns.c @@ -19,7 +19,7 @@ NS(XmlGetUtf16InternalEncoding)(void) #endif } -static const ENCODING *NS(encodings)[] = { +static const ENCODING * const NS(encodings)[] = { &ns(latin1_encoding).enc, &ns(ascii_encoding).enc, &ns(utf8_encoding).enc, diff --git a/Modules/fcntlmodule.c b/Modules/fcntlmodule.c index 477af06..4e49a7d 100644 --- a/Modules/fcntlmodule.c +++ b/Modules/fcntlmodule.c @@ -290,7 +290,7 @@ PyDoc_STRVAR(flock_doc, "flock(fd, operation)\n\ \n\ Perform the lock operation op on file descriptor fd. See the Unix \n\ -manual flock(3) for details. (On some systems, this function is\n\ +manual page for flock(3) for details. (On some systems, this function is\n\ emulated using fcntl().)"); @@ -327,7 +327,7 @@ fcntl_lockf(PyObject *self, PyObject *args) l.l_type = F_WRLCK; else { PyErr_SetString(PyExc_ValueError, - "unrecognized flock argument"); + "unrecognized lockf argument"); return NULL; } l.l_start = l.l_len = 0; diff --git a/Modules/fpectlmodule.c b/Modules/fpectlmodule.c index c6d4f77..74354ba 100644 --- a/Modules/fpectlmodule.c +++ b/Modules/fpectlmodule.c @@ -70,6 +70,10 @@ extern "C" { #if defined(__FreeBSD__) # include +#elif defined(__VMS) +#define __NEW_STARLET +#include +#include #endif #ifndef WANT_SIGFPE_HANDLER @@ -190,6 +194,19 @@ static void fpe_reset(Sigfunc *handler) /*-- DEC ALPHA VMS --------------------------------------------------------*/ #elif defined(__ALPHA) && defined(__VMS) + IEEE clrmsk; + IEEE setmsk; + clrmsk.ieee$q_flags = + IEEE$M_TRAP_ENABLE_UNF | IEEE$M_TRAP_ENABLE_INE | + IEEE$M_MAP_UMZ; + setmsk.ieee$q_flags = + IEEE$M_TRAP_ENABLE_INV | IEEE$M_TRAP_ENABLE_DZE | + IEEE$M_TRAP_ENABLE_OVF; + sys$ieee_set_fp_control(&clrmsk, &setmsk, 0); + PyOS_setsig(SIGFPE, handler); + +/*-- HP IA64 VMS --------------------------------------------------------*/ +#elif defined(__ia64) && defined(__VMS) PyOS_setsig(SIGFPE, handler); /*-- Cray Unicos ----------------------------------------------------------*/ @@ -244,6 +261,14 @@ static PyObject *turnoff_sigfpe(PyObject *self,PyObject *args) #ifdef __FreeBSD__ fpresetsticky(fpgetsticky()); fpsetmask(0); +#elif defined(__VMS) + IEEE clrmsk; + clrmsk.ieee$q_flags = + IEEE$M_TRAP_ENABLE_UNF | IEEE$M_TRAP_ENABLE_INE | + IEEE$M_MAP_UMZ | IEEE$M_TRAP_ENABLE_INV | + IEEE$M_TRAP_ENABLE_DZE | IEEE$M_TRAP_ENABLE_OVF | + IEEE$M_INHERIT; + sys$ieee_set_fp_control(&clrmsk, 0, 0); #else fputs("Operation not implemented\n", stderr); #endif diff --git a/Modules/getpath.c b/Modules/getpath.c index 8eba730..78bfaf9 100644 --- a/Modules/getpath.c +++ b/Modules/getpath.c @@ -97,19 +97,19 @@ #ifndef VERSION -#if defined(__VMS) -#define VERSION "2_1" -#else #define VERSION "2.1" #endif -#endif #ifndef VPATH #define VPATH "." #endif #ifndef PREFIX -#define PREFIX "/usr/local" +# ifdef __VMS +# define PREFIX "" +# else +# define PREFIX "/usr/local" +# endif #endif #ifndef EXEC_PREFIX diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index 86b1bbf..d913890 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -357,7 +357,7 @@ teedataobject_jumplink(teedataobject *tdo) { if (tdo->nextlink == NULL) tdo->nextlink = teedataobject_new(tdo->it); - Py_INCREF(tdo->nextlink); + Py_XINCREF(tdo->nextlink); return tdo->nextlink; } @@ -468,7 +468,7 @@ tee_next(teeobject *to) if (to->index >= LINKCELLS) { link = teedataobject_jumplink(to->dataobj); - Py_XDECREF(to->dataobj); + Py_DECREF(to->dataobj); to->dataobj = (teedataobject *)link; to->index = 0; } @@ -522,6 +522,12 @@ tee_fromiterable(PyObject *iterable) if (to == NULL) goto done; to->dataobj = (teedataobject *)teedataobject_new(it); + if (!to->dataobj) { + PyObject_GC_Del(to); + to = NULL; + goto done; + } + to->index = 0; to->weakreflist = NULL; PyObject_GC_Track(to); diff --git a/Modules/main.c b/Modules/main.c index 23a760f..3f39c9b 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -39,7 +39,7 @@ static char **orig_argv; static int orig_argc; /* command line options */ -#define BASE_OPTS "c:dEhim:OStuvVW:xX" +#define BASE_OPTS "c:dEhim:OStuvVW:xX?" #ifndef RISCOS #define PROGRAM_OPTS BASE_OPTS @@ -61,7 +61,7 @@ Options and arguments (and corresponding environment variables):\n\ -c cmd : program passed in as string (terminates option list)\n\ -d : debug output from parser (also PYTHONDEBUG=x)\n\ -E : ignore environment variables (such as PYTHONPATH)\n\ --h : print this help message and exit\n\ +-h : print this help message and exit (also --help)\n\ -i : inspect interactively after running script, (also PYTHONINSPECT=x)\n\ and force prompts, even if stdin does not appear to be a terminal\n\ "; @@ -76,7 +76,7 @@ static char *usage_2 = "\ static char *usage_3 = "\ see man page for details on internal buffering relating to '-u'\n\ -v : verbose (trace import statements) (also PYTHONVERBOSE=x)\n\ --V : print the Python version number and exit\n\ +-V : print the Python version number and exit (also --version)\n\ -W arg : warning control (arg is action:message:category:module:lineno)\n\ -x : skip first line of source, allowing use of non-Unix forms of #!cmd\n\ file : program read from script file\n\ @@ -281,6 +281,7 @@ Py_Main(int argc, char **argv) break; case 'h': + case '?': help++; break; @@ -431,9 +432,10 @@ Py_Main(int argc, char **argv) } if (module != NULL) { - /* Backup _PyOS_optind and force sys.arv[0] = module */ + /* Backup _PyOS_optind and force sys.argv[0] = '-c' + so that PySys_SetArgv correctly sets sys.path[0] to ''*/ _PyOS_optind--; - argv[_PyOS_optind] = module; + argv[_PyOS_optind] = "-c"; } PySys_SetArgv(argc-_PyOS_optind, argv+_PyOS_optind); diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index d7a8944..afe9916 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -50,7 +50,10 @@ my_getpagesize(void) #endif /* UNIX */ #include + +#ifdef HAVE_SYS_TYPES_H #include +#endif /* HAVE_SYS_TYPES_H */ /* Prefer MAP_ANONYMOUS since MAP_ANON is deprecated according to man page. */ #if !defined(MAP_ANONYMOUS) && defined(MAP_ANON) diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index c0280de..d968b6c 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -64,14 +64,21 @@ corresponding Unix manual entries for more information on calls."); #include "osdefs.h" #endif +#ifdef HAVE_SYS_TYPES_H #include +#endif /* HAVE_SYS_TYPES_H */ + +#ifdef HAVE_SYS_STAT_H #include +#endif /* HAVE_SYS_STAT_H */ #ifdef HAVE_SYS_WAIT_H #include /* For WNOHANG */ #endif +#ifdef HAVE_SIGNAL_H #include +#endif #ifdef HAVE_FCNTL_H #include @@ -246,9 +253,15 @@ extern int lstat(const char *, struct stat *); #endif #ifdef _MSC_VER +#ifdef HAVE_DIRECT_H #include +#endif +#ifdef HAVE_IO_H #include +#endif +#ifdef HAVE_PROCESS_H #include +#endif #include "osdefs.h" #define _WIN32_WINNT 0x0400 /* Needed for CryptoAPI on some systems */ #include @@ -1227,7 +1240,9 @@ _pystat_fromstructstat(STRUCT_STAT *st) #define ISSLASHA(c) ((c) == '\\' || (c) == '/') #define ISSLASHW(c) ((c) == L'\\' || (c) == L'/') +#ifndef ARRAYSIZE #define ARRAYSIZE(a) (sizeof(a) / sizeof(a[0])) +#endif static BOOL IsUNCRootA(char *path, int pathlen) @@ -1387,7 +1402,7 @@ finish: return PyBool_FromLong(0); /* Access is possible if either write access wasn't requested, or the file isn't read-only. */ - return PyBool_FromLong(!(mode & 2) || !(attr && FILE_ATTRIBUTE_READONLY)); + return PyBool_FromLong(!(mode & 2) || !(attr & FILE_ATTRIBUTE_READONLY)); #else int res; if (!PyArg_ParseTuple(args, "eti:access", @@ -1847,6 +1862,15 @@ posix_listdir(PyObject *self, PyObject *args) Py_BEGIN_ALLOW_THREADS result = FindNextFileW(hFindFile, &wFileData); Py_END_ALLOW_THREADS + /* FindNextFile sets error to ERROR_NO_MORE_FILES if + it got to the end of the directory. */ + if (!result && GetLastError() != ERROR_NO_MORE_FILES) { + Py_DECREF(d); + win32_error_unicode("FindNextFileW", wnamebuf); + FindClose(hFindFile); + free(wnamebuf); + return NULL; + } } while (result == TRUE); if (FindClose(hFindFile) == FALSE) { @@ -1906,6 +1930,14 @@ posix_listdir(PyObject *self, PyObject *args) Py_BEGIN_ALLOW_THREADS result = FindNextFile(hFindFile, &FileData); Py_END_ALLOW_THREADS + /* FindNextFile sets error to ERROR_NO_MORE_FILES if + it got to the end of the directory. */ + if (!result && GetLastError() != ERROR_NO_MORE_FILES) { + Py_DECREF(d); + win32_error("FindNextFile", namebuf); + FindClose(hFindFile); + return NULL; + } } while (result == TRUE); if (FindClose(hFindFile) == FALSE) { @@ -7867,6 +7899,42 @@ win32_urandom(PyObject *self, PyObject *args) } #endif +#ifdef __VMS +/* Use openssl random routine */ +#include +PyDoc_STRVAR(vms_urandom__doc__, +"urandom(n) -> str\n\n\ +Return a string of n random bytes suitable for cryptographic use."); + +static PyObject* +vms_urandom(PyObject *self, PyObject *args) +{ + int howMany; + PyObject* result; + + /* Read arguments */ + if (! PyArg_ParseTuple(args, "i:urandom", &howMany)) + return NULL; + if (howMany < 0) + return PyErr_Format(PyExc_ValueError, + "negative argument not allowed"); + + /* Allocate bytes */ + result = PyString_FromStringAndSize(NULL, howMany); + if (result != NULL) { + /* Get random data */ + if (RAND_pseudo_bytes((unsigned char*) + PyString_AS_STRING(result), + howMany) < 0) { + Py_DECREF(result); + return PyErr_Format(PyExc_ValueError, + "RAND_pseudo_bytes"); + } + } + return result; +} +#endif + static PyMethodDef posix_methods[] = { {"access", posix_access, METH_VARARGS, posix_access__doc__}, #ifdef HAVE_TTYNAME @@ -8160,6 +8228,9 @@ static PyMethodDef posix_methods[] = { #ifdef MS_WINDOWS {"urandom", win32_urandom, METH_VARARGS, win32_urandom__doc__}, #endif + #ifdef __VMS + {"urandom", vms_urandom, METH_VARARGS, vms_urandom__doc__}, + #endif {NULL, NULL} /* Sentinel */ }; diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index 8a10bab..67f7007 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -238,6 +238,18 @@ error_external_entity_ref_handler(XML_Parser parser, return 0; } +/* Dummy character data handler used when an error (exception) has + been detected, and the actual parsing can be terminated early. + This is needed since character data handler can't be safely removed + from within the character data handler, but can be replaced. It is + used only from the character data handler trampoline, and must be + used right after `flag_error()` is called. */ +static void +noop_character_data_handler(void *userData, const XML_Char *data, int len) +{ + /* Do nothing. */ +} + static void flag_error(xmlparseobject *self) { @@ -457,6 +469,8 @@ call_character_handler(xmlparseobject *self, const XML_Char *buffer, int len) if (temp == NULL) { Py_DECREF(args); flag_error(self); + XML_SetCharacterDataHandler(self->itself, + noop_character_data_handler); return -1; } PyTuple_SET_ITEM(args, 0, temp); @@ -469,6 +483,8 @@ call_character_handler(xmlparseobject *self, const XML_Char *buffer, int len) Py_DECREF(args); if (temp == NULL) { flag_error(self); + XML_SetCharacterDataHandler(self->itself, + noop_character_data_handler); return -1; } Py_DECREF(temp); @@ -1542,8 +1558,22 @@ sethandler(xmlparseobject *self, const char *name, PyObject* v) xmlhandler c_handler = NULL; PyObject *temp = self->handlers[handlernum]; - if (v == Py_None) + if (v == Py_None) { + /* If this is the character data handler, and a character + data handler is already active, we need to be more + careful. What we can safely do is replace the existing + character data handler callback function with a no-op + function that will refuse to call Python. The downside + is that this doesn't completely remove the character + data handler from the C layer if there's any callback + active, so Expat does a little more work than it + otherwise would, but that's really an odd case. A more + elaborate system of handlers and state could remove the + C handler more effectively. */ + if (handlernum == CharacterData && self->in_callback) + c_handler = noop_character_data_handler; v = NULL; + } else if (v != NULL) { Py_INCREF(v); c_handler = handler_info[handlernum].handler; diff --git a/Modules/readline.c b/Modules/readline.c index 8fda228..92f2d1f 100644 --- a/Modules/readline.c +++ b/Modules/readline.c @@ -20,6 +20,12 @@ #include #endif +#ifdef SAVE_LOCALE +# define RESTORE_LOCALE(sl) { setlocale(LC_CTYPE, sl); free(sl); } +#else +# define RESTORE_LOCALE(sl) +#endif + /* GNU readline definitions */ #undef HAVE_CONFIG_H /* Else readline/chardefs.h includes strings.h */ #include @@ -723,10 +729,7 @@ setup_readline(void) */ rl_initialize(); -#ifdef SAVE_LOCALE - setlocale(LC_CTYPE, saved_locale); /* Restore locale */ - free(saved_locale); -#endif + RESTORE_LOCALE(saved_locale) } /* Wrapper around GNU readline that handles signals differently. */ @@ -864,7 +867,8 @@ call_readline(FILE *sys_stdin, FILE *sys_stdout, char *prompt) p = readline_until_enter_or_signal(prompt, &signal); /* we got an interrupt signal */ - if(signal) { + if (signal) { + RESTORE_LOCALE(saved_locale) return NULL; } @@ -873,6 +877,7 @@ call_readline(FILE *sys_stdin, FILE *sys_stdout, char *prompt) p = PyMem_Malloc(1); if (p != NULL) *p = '\0'; + RESTORE_LOCALE(saved_locale) return p; } @@ -905,10 +910,7 @@ call_readline(FILE *sys_stdin, FILE *sys_stdout, char *prompt) p[n+1] = '\0'; } free(q); -#ifdef SAVE_LOCALE - setlocale(LC_CTYPE, saved_locale); /* Restore locale */ - free(saved_locale); -#endif + RESTORE_LOCALE(saved_locale) return p; } diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c index dfa4e85..9eaae84 100644 --- a/Modules/selectmodule.c +++ b/Modules/selectmodule.c @@ -36,7 +36,7 @@ extern void bzero(void *, int); #endif -#ifndef DONT_HAVE_SYS_TYPES_H +#ifdef HAVE_SYS_TYPES_H #include #endif @@ -46,14 +46,14 @@ extern void bzero(void *, int); #endif #ifdef MS_WINDOWS -#include +# include #else -#ifdef __BEOS__ -#include -#define SOCKET int -#else -#define SOCKET int -#endif +# define SOCKET int +# ifdef __BEOS__ +# include +# elif defined(__VMS) +# include +# endif #endif @@ -668,7 +668,7 @@ arguments; each contains the subset of the corresponding file descriptors\n\ that are ready.\n\ \n\ *** IMPORTANT NOTICE ***\n\ -On Windows, only sockets are supported; on Unix, all file descriptors."); +On Windows and OpenVMS, only sockets are supported; on Unix, all file descriptors."); static PyMethodDef select_methods[] = { {"select", select_select, METH_VARARGS, select_doc}, @@ -682,7 +682,7 @@ PyDoc_STRVAR(module_doc, "This module supports asynchronous I/O on multiple file descriptors.\n\ \n\ *** IMPORTANT NOTICE ***\n\ -On Windows, only sockets are supported; on Unix, all file descriptors."); +On Windows and OpenVMS, only sockets are supported; on Unix, all file descriptors."); PyMODINIT_FUNC initselect(void) diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index 262abe8..bb99bde 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -161,7 +161,8 @@ shutdown(how) -- shut down traffic in one or both directions\n\ (this includes the getaddrinfo emulation) protect access with a lock. */ #if defined(WITH_THREAD) && (defined(__APPLE__) || \ (defined(__FreeBSD__) && __FreeBSD_version+0 < 503000) || \ - defined(__OpenBSD__) || defined(__NetBSD__) || !defined(HAVE_GETADDRINFO)) + defined(__OpenBSD__) || defined(__NetBSD__) || \ + defined(__VMS) || !defined(HAVE_GETADDRINFO)) #define USE_GETADDRINFO_LOCK #endif @@ -186,15 +187,8 @@ shutdown(how) -- shut down traffic in one or both directions\n\ #endif #if defined(__VMS) -#if ! defined(_SOCKADDR_LEN) -# ifdef getaddrinfo -# undef getaddrinfo -# endif -# include "TCPIP_IOCTL_ROUTINE" -#else # include #endif -#endif #if defined(PYOS_OS2) # define INCL_DOS @@ -234,7 +228,9 @@ shutdown(how) -- shut down traffic in one or both directions\n\ #endif /* Generic includes */ +#ifdef HAVE_SYS_TYPES_H #include +#endif /* Generic socket object definitions and includes */ #define PySocket_BUILDING_SOCKET @@ -270,7 +266,9 @@ int h_errno; /* not used */ #else /* MS_WINDOWS includes */ -# include +# ifdef HAVE_FCNTL_H +# include +# endif #endif @@ -290,7 +288,7 @@ int h_errno; /* not used */ * _SS_ALIGNSIZE is defined in sys/socket.h by 6.5.21, * for example, but not by 6.5.10. */ -#elif defined(_MSC_VER) && _MSC_VER>1200 +#elif defined(_MSC_VER) && _MSC_VER>1201 /* Do not include addrinfo.h for MSVC7 or greater. 'addrinfo' and * EAI_* constants are defined in (the already included) ws2tcpip.h. */ @@ -359,11 +357,6 @@ const char *inet_ntop(int af, const void *src, char *dst, socklen_t size); #define SOCKETCLOSE close #endif -#ifdef __VMS -/* TCP/IP Services for VMS uses a maximum send/revc buffer length of 65535 */ -#define SEGMENT_SIZE 65535 -#endif - #if defined(HAVE_BLUETOOTH_H) || defined(HAVE_BLUETOOTH_BLUETOOTH_H) #define USE_BLUETOOTH 1 #if defined(__FreeBSD__) @@ -374,6 +367,14 @@ const char *inet_ntop(int af, const void *src, char *dst, socklen_t size); #define _BT_SOCKADDR_MEMB(s, proto) &((s)->sock_addr) #define _BT_L2_MEMB(sa, memb) ((sa)->l2cap_##memb) #define _BT_RC_MEMB(sa, memb) ((sa)->rfcomm_##memb) +#elif defined(__NetBSD__) +#define sockaddr_l2 sockaddr_bt +#define sockaddr_rc sockaddr_bt +#define sockaddr_sco sockaddr_bt +#define _BT_SOCKADDR_MEMB(s, proto) &((s)->sock_addr) +#define _BT_L2_MEMB(sa, memb) ((sa)->bt_##memb) +#define _BT_RC_MEMB(sa, memb) ((sa)->bt_##memb) +#define _BT_SCO_MEMB(sa, memb) ((sa)->bt_##memb) #else #define _BT_SOCKADDR_MEMB(s, proto) (&((s)->sock_addr).bt_##proto) #define _BT_L2_MEMB(sa, memb) ((sa)->l2_##memb) @@ -382,6 +383,11 @@ const char *inet_ntop(int af, const void *src, char *dst, socklen_t size); #endif #endif +#ifdef __VMS +/* TCP/IP Services for VMS uses a maximum send/recv buffer length */ +#define SEGMENT_SIZE (32 * 1024 -1) +#endif + /* * Constants for getnameinfo() */ @@ -413,14 +419,24 @@ static int taskwindow; there has to be a circular reference. */ static PyTypeObject sock_type; -/* Can we call select() with this socket without a buffer overrun? */ +#if defined(HAVE_POLL_H) +#include +#elif defined(HAVE_SYS_POLL_H) +#include +#endif + #ifdef Py_SOCKET_FD_CAN_BE_GE_FD_SETSIZE /* Platform can select file descriptors beyond FD_SETSIZE */ #define IS_SELECTABLE(s) 1 +#elif defined(HAVE_POLL) +/* Instead of select(), we'll use poll() since poll() works on any fd. */ +#define IS_SELECTABLE(s) 1 +/* Can we call select() with this socket without a buffer overrun? */ #else /* POSIX says selecting file descriptors beyond FD_SETSIZE - has undefined behaviour. */ -#define IS_SELECTABLE(s) ((s)->sock_fd < FD_SETSIZE) + has undefined behaviour. If there's no timeout left, we don't have to + call select, so it's a safe, little white lie. */ +#define IS_SELECTABLE(s) ((s)->sock_fd < FD_SETSIZE || s->sock_timeout <= 0.0) #endif static PyObject* @@ -616,6 +632,30 @@ set_gaierror(int error) return NULL; } +#ifdef __VMS +/* Function to send in segments */ +static int +sendsegmented(int sock_fd, char *buf, int len, int flags) +{ + int n = 0; + int remaining = len; + + while (remaining > 0) { + unsigned int segment; + + segment = (remaining >= SEGMENT_SIZE ? SEGMENT_SIZE : remaining); + n = send(sock_fd, buf, segment, flags); + if (n < 0) { + return n; + } + remaining -= segment; + buf += segment; + } /* end while */ + + return len; +} +#endif + /* Function to perform the setting of socket blocking mode internally. block = (1 | 0). */ static int @@ -640,8 +680,8 @@ internal_setblocking(PySocketSockObject *s, int block) ioctl(s->sock_fd, FIONBIO, (caddr_t)&block, sizeof(block)); #elif defined(__VMS) block = !block; - ioctl(s->sock_fd, FIONBIO, (char *)&block); -#else /* !PYOS_OS2 && !_VMS */ + ioctl(s->sock_fd, FIONBIO, (unsigned int *)&block); +#else /* !PYOS_OS2 && !__VMS */ delay_flag = fcntl(s->sock_fd, F_GETFL, 0); if (block) delay_flag &= (~O_NONBLOCK); @@ -664,16 +704,14 @@ internal_setblocking(PySocketSockObject *s, int block) return 1; } -/* Do a select() on the socket, if necessary (sock_timeout > 0). +/* Do a select()/poll() on the socket, if necessary (sock_timeout > 0). The argument writing indicates the direction. This does not raise an exception; we'll let our caller do that after they've reacquired the interpreter lock. - Returns 1 on timeout, 0 otherwise. */ + Returns 1 on timeout, -1 on error, 0 otherwise. */ static int internal_select(PySocketSockObject *s, int writing) { - fd_set fds; - struct timeval tv; int n; /* Nothing to do unless we're in timeout mode (not non-blocking) */ @@ -684,17 +722,40 @@ internal_select(PySocketSockObject *s, int writing) if (s->sock_fd < 0) return 0; - /* Construct the arguments to select */ - tv.tv_sec = (int)s->sock_timeout; - tv.tv_usec = (int)((s->sock_timeout - tv.tv_sec) * 1e6); - FD_ZERO(&fds); - FD_SET(s->sock_fd, &fds); + /* Prefer poll, if available, since you can poll() any fd + * which can't be done with select(). */ +#ifdef HAVE_POLL + { + struct pollfd pollfd; + int timeout; - /* See if the socket is ready */ - if (writing) - n = select(s->sock_fd+1, NULL, &fds, NULL, &tv); - else - n = select(s->sock_fd+1, &fds, NULL, NULL, &tv); + pollfd.fd = s->sock_fd; + pollfd.events = writing ? POLLOUT : POLLIN; + + /* s->sock_timeout is in seconds, timeout in ms */ + timeout = (int)(s->sock_timeout * 1000 + 0.5); + n = poll(&pollfd, 1, timeout); + } +#else + { + /* Construct the arguments to select */ + fd_set fds; + struct timeval tv; + tv.tv_sec = (int)s->sock_timeout; + tv.tv_usec = (int)((s->sock_timeout - tv.tv_sec) * 1e6); + FD_ZERO(&fds); + FD_SET(s->sock_fd, &fds); + + /* See if the socket is ready */ + if (writing) + n = select(s->sock_fd+1, NULL, &fds, NULL, &tv); + else + n = select(s->sock_fd+1, &fds, NULL, NULL, &tv); + } +#endif + + if (n < 0) + return -1; if (n == 0) return 1; return 0; @@ -1494,7 +1555,7 @@ sock_accept(PySocketSockObject *s) &addrlen); Py_END_ALLOW_THREADS - if (timeout) { + if (timeout == 1) { PyErr_SetString(socket_timeout, "timed out"); return NULL; } @@ -1721,6 +1782,8 @@ sock_getsockopt(PySocketSockObject *s, PyObject *args) return PyInt_FromLong(flag); } #ifdef __VMS + /* socklen_t is unsigned so no negative test is needed, + test buflen == 0 is previously done */ if (buflen > 1024) { #else if (buflen <= 0 || buflen > 1024) { @@ -1863,9 +1926,15 @@ internal_connect(PySocketSockObject *s, struct sockaddr *addr, int addrlen, if (s->sock_timeout > 0.0) { if (res < 0 && errno == EINPROGRESS && IS_SELECTABLE(s)) { timeout = internal_select(s, 1); - res = connect(s->sock_fd, addr, addrlen); - if (res < 0 && errno == EISCONN) - res = 0; + if (timeout == 0) { + res = connect(s->sock_fd, addr, addrlen); + if (res < 0 && errno == EISCONN) + res = 0; + } + else if (timeout == -1) + res = errno; /* had error */ + else + res = EWOULDBLOCK; /* timed out */ } } @@ -1895,7 +1964,7 @@ sock_connect(PySocketSockObject *s, PyObject *addro) res = internal_connect(s, addr, addrlen, &timeout); Py_END_ALLOW_THREADS - if (timeout) { + if (timeout == 1) { PyErr_SetString(socket_timeout, "timed out"); return NULL; } @@ -1929,6 +1998,13 @@ sock_connect_ex(PySocketSockObject *s, PyObject *addro) res = internal_connect(s, addr, addrlen, &timeout); Py_END_ALLOW_THREADS + /* Signals are not errors (though they may raise exceptions). Adapted + from PyErr_SetFromErrnoWithFilenameObject(). */ +#ifdef EINTR + if (res == EINTR && PyErr_CheckSignals()) + return NULL; +#endif + return PyInt_FromLong((long) res); } @@ -2149,10 +2225,10 @@ The mode and buffersize arguments are as for the built-in open() function."); static ssize_t sock_recv_guts(PySocketSockObject *s, char* cbuf, int len, int flags) { - ssize_t outlen = 0; + ssize_t outlen = -1; int timeout; #ifdef __VMS - int remaining, nread; + int remaining; char *read_buf; #endif @@ -2168,7 +2244,7 @@ sock_recv_guts(PySocketSockObject *s, char* cbuf, int len, int flags) outlen = recv(s->sock_fd, cbuf, len, flags); Py_END_ALLOW_THREADS - if (timeout) { + if (timeout == 1) { PyErr_SetString(socket_timeout, "timed out"); return -1; } @@ -2183,6 +2259,7 @@ sock_recv_guts(PySocketSockObject *s, char* cbuf, int len, int flags) remaining = len; while (remaining != 0) { unsigned int segment; + int nread = -1; segment = remaining /SEGMENT_SIZE; if (segment != 0) { @@ -2198,7 +2275,7 @@ sock_recv_guts(PySocketSockObject *s, char* cbuf, int len, int flags) nread = recv(s->sock_fd, read_buf, segment, flags); Py_END_ALLOW_THREADS - if (timeout) { + if (timeout == 1) { PyErr_SetString(socket_timeout, "timed out"); return -1; } @@ -2247,7 +2324,7 @@ sock_recv(PySocketSockObject *s, PyObject *args) /* Call the guts */ outlen = sock_recv_guts(s, PyString_AS_STRING(buf), recvlen, flags); if (outlen < 0) { - /* An error occured, release the string and return an + /* An error occurred, release the string and return an error. */ Py_DECREF(buf); return NULL; @@ -2346,7 +2423,7 @@ sock_recvfrom_guts(PySocketSockObject *s, char* cbuf, int len, int flags, { sock_addr_t addrbuf; int timeout; - ssize_t n = 0; + ssize_t n = -1; socklen_t addrlen; *addr = NULL; @@ -2378,7 +2455,7 @@ sock_recvfrom_guts(PySocketSockObject *s, char* cbuf, int len, int flags, } Py_END_ALLOW_THREADS - if (timeout) { + if (timeout == 1) { PyErr_SetString(socket_timeout, "timed out"); return -1; } @@ -2493,10 +2570,7 @@ static PyObject * sock_send(PySocketSockObject *s, PyObject *args) { char *buf; - int len, n = 0, flags = 0, timeout; -#ifdef __VMS - int send_length; -#endif + int len, n = -1, flags = 0, timeout; if (!PyArg_ParseTuple(args, "s#|i:send", &buf, &len, &flags)) return NULL; @@ -2504,49 +2578,22 @@ sock_send(PySocketSockObject *s, PyObject *args) if (!IS_SELECTABLE(s)) return select_error(); -#ifndef __VMS Py_BEGIN_ALLOW_THREADS timeout = internal_select(s, 1); if (!timeout) +#ifdef __VMS + n = sendsegmented(s->sock_fd, buf, len, flags); +#else n = send(s->sock_fd, buf, len, flags); +#endif Py_END_ALLOW_THREADS - if (timeout) { + if (timeout == 1) { PyErr_SetString(socket_timeout, "timed out"); return NULL; } if (n < 0) return s->errorhandler(); -#else - /* Divide packet into smaller segments for */ - /* TCP/IP Services for OpenVMS */ - send_length = len; - while (send_length != 0) { - unsigned int segment; - - segment = send_length / SEGMENT_SIZE; - if (segment != 0) { - segment = SEGMENT_SIZE; - } - else { - segment = send_length; - } - Py_BEGIN_ALLOW_THREADS - timeout = internal_select(s, 1); - if (!timeout) - n = send(s->sock_fd, buf, segment, flags); - Py_END_ALLOW_THREADS - if (timeout) { - PyErr_SetString(socket_timeout, "timed out"); - return NULL; - } - if (n < 0) { - return s->errorhandler(); - } - send_length -= segment; - buf += segment; - } /* end while */ -#endif /* !__VMS */ return PyInt_FromLong((long)n); } @@ -2564,7 +2611,7 @@ static PyObject * sock_sendall(PySocketSockObject *s, PyObject *args) { char *buf; - int len, n = 0, flags = 0, timeout; + int len, n = -1, flags = 0, timeout; if (!PyArg_ParseTuple(args, "s#|i:sendall", &buf, &len, &flags)) return NULL; @@ -2575,9 +2622,14 @@ sock_sendall(PySocketSockObject *s, PyObject *args) Py_BEGIN_ALLOW_THREADS do { timeout = internal_select(s, 1); + n = -1; if (timeout) break; +#ifdef __VMS + n = sendsegmented(s->sock_fd, buf, len, flags); +#else n = send(s->sock_fd, buf, len, flags); +#endif if (n < 0) break; buf += n; @@ -2585,7 +2637,7 @@ sock_sendall(PySocketSockObject *s, PyObject *args) } while (len > 0); Py_END_ALLOW_THREADS - if (timeout) { + if (timeout == 1) { PyErr_SetString(socket_timeout, "timed out"); return NULL; } @@ -2613,7 +2665,7 @@ sock_sendto(PySocketSockObject *s, PyObject *args) PyObject *addro; char *buf; struct sockaddr *addr; - int addrlen, len, n = 0, flags, timeout; + int addrlen, len, n = -1, flags, timeout; flags = 0; if (!PyArg_ParseTuple(args, "s#O:sendto", &buf, &len, &addro)) { @@ -2635,7 +2687,7 @@ sock_sendto(PySocketSockObject *s, PyObject *args) n = sendto(s->sock_fd, buf, len, flags, addr, addrlen); Py_END_ALLOW_THREADS - if (timeout) { + if (timeout == 1) { PyErr_SetString(socket_timeout, "timed out"); return NULL; } diff --git a/Modules/spwdmodule.c b/Modules/spwdmodule.c index b7bf20e..d3f309a 100644 --- a/Modules/spwdmodule.c +++ b/Modules/spwdmodule.c @@ -91,7 +91,7 @@ static PyObject *mkspent(struct spwd *p) #undef SETI if (PyErr_Occurred()) { - Py_XDECREF(v); + Py_DECREF(v); return NULL; } diff --git a/Modules/sre.h b/Modules/sre.h index b07d210..d4af05c 100644 --- a/Modules/sre.h +++ b/Modules/sre.h @@ -23,7 +23,7 @@ typedef struct { PyObject_VAR_HEAD - int groups; /* must be first! */ + Py_ssize_t groups; /* must be first! */ PyObject* groupindex; PyObject* indexgroup; /* compatibility */ @@ -31,7 +31,7 @@ typedef struct { int flags; /* flags used when compiling pattern source */ PyObject *weakreflist; /* List of weak references */ /* pattern code */ - int codesize; + Py_ssize_t codesize; SRE_CODE code[1]; } PatternObject; @@ -42,10 +42,10 @@ typedef struct { PyObject* string; /* link to the target string (must be first) */ PyObject* regs; /* cached list of matching spans */ PatternObject* pattern; /* link to the regex (pattern) object */ - int pos, endpos; /* current target slice */ - int lastindex; /* last index marker seen by the engine (-1 if none) */ - int groups; /* number of groups (start/end marks) */ - int mark[1]; + Py_ssize_t pos, endpos; /* current target slice */ + Py_ssize_t lastindex; /* last index marker seen by the engine (-1 if none) */ + Py_ssize_t groups; /* number of groups (start/end marks) */ + Py_ssize_t mark[1]; } MatchObject; typedef unsigned int (*SRE_TOLOWER_HOOK)(unsigned int ch); @@ -54,7 +54,7 @@ typedef unsigned int (*SRE_TOLOWER_HOOK)(unsigned int ch); #define SRE_MARK_SIZE 200 typedef struct SRE_REPEAT_T { - int count; + Py_ssize_t count; SRE_CODE* pattern; /* points to REPEAT operator arguments */ void* last_ptr; /* helper to check for infinite loops */ struct SRE_REPEAT_T *prev; /* points to previous repeat context */ @@ -68,17 +68,17 @@ typedef struct { void* end; /* end of original string */ /* attributes for the match object */ PyObject* string; - int pos, endpos; + Py_ssize_t pos, endpos; /* character size */ int charsize; /* registers */ - int lastindex; - int lastmark; + Py_ssize_t lastindex; + Py_ssize_t lastmark; void* mark[SRE_MARK_SIZE]; /* dynamically allocated stuff */ char* data_stack; - unsigned int data_stack_size; - unsigned int data_stack_base; + size_t data_stack_size; + size_t data_stack_base; /* current repeat context */ SRE_REPEAT *repeat; /* hooks */ diff --git a/Modules/threadmodule.c b/Modules/threadmodule.c index 6169658..448b11c 100644 --- a/Modules/threadmodule.c +++ b/Modules/threadmodule.c @@ -586,6 +586,60 @@ allocated consecutive numbers starting at 1, this behavior should not\n\ be relied upon, and the number should be seen purely as a magic cookie.\n\ A thread's identity may be reused for another thread after it exits."); +static PyObject * +thread_stack_size(PyObject *self, PyObject *args) +{ + size_t old_size; + Py_ssize_t new_size = 0; + int rc; + + if (!PyArg_ParseTuple(args, "|n:stack_size", &new_size)) + return NULL; + + if (new_size < 0) { + PyErr_SetString(PyExc_ValueError, + "size must be 0 or a positive value"); + return NULL; + } + + old_size = PyThread_get_stacksize(); + + rc = PyThread_set_stacksize((size_t) new_size); + if (rc == -1) { + PyErr_Format(PyExc_ValueError, + "size not valid: %zd bytes", + new_size); + return NULL; + } + if (rc == -2) { + PyErr_SetString(ThreadError, + "setting stack size not supported"); + return NULL; + } + + return PyInt_FromSsize_t((Py_ssize_t) old_size); +} + +PyDoc_STRVAR(stack_size_doc, +"stack_size([size]) -> size\n\ +\n\ +Return the thread stack size used when creating new threads. The\n\ +optional size argument specifies the stack size (in bytes) to be used\n\ +for subsequently created threads, and must be 0 (use platform or\n\ +configured default) or a positive integer value of at least 32,768 (32k).\n\ +If changing the thread stack size is unsupported, a ThreadError\n\ +exception is raised. If the specified size is invalid, a ValueError\n\ +exception is raised, and the stack size is unmodified. 32k bytes\n\ + currently the minimum supported stack size value to guarantee\n\ +sufficient stack space for the interpreter itself.\n\ +\n\ +Note that some platforms may have particular restrictions on values for\n\ +the stack size, such as requiring a minimum stack size larger than 32kB or\n\ +requiring allocation in multiples of the system memory page size\n\ +- platform documentation should be referred to for more information\n\ +(4kB pages are common; using multiples of 4096 for the stack size is\n\ +the suggested approach in the absence of more specific information)."); + static PyMethodDef thread_methods[] = { {"start_new_thread", (PyCFunction)thread_PyThread_start_new_thread, METH_VARARGS, @@ -605,6 +659,9 @@ static PyMethodDef thread_methods[] = { METH_NOARGS, interrupt_doc}, {"get_ident", (PyCFunction)thread_get_ident, METH_NOARGS, get_ident_doc}, + {"stack_size", (PyCFunction)thread_stack_size, + METH_VARARGS, + stack_size_doc}, #ifndef NO_EXIT_PROG {"exit_prog", (PyCFunction)thread_PyThread_exit_prog, METH_VARARGS}, diff --git a/Modules/timemodule.c b/Modules/timemodule.c index 87e543f..444b739 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -19,7 +19,9 @@ #include +#ifdef HAVE_SYS_TYPES_H #include +#endif /* HAVE_SYS_TYPES_H */ #ifdef QUICKWIN #include @@ -404,13 +406,35 @@ time_strftime(PyObject *self, PyObject *args) indexing blindly into some array for a textual representation by some bad index (fixes bug #897625). - No check for year since handled in gettmarg(). + Also support values of zero from Python code for arguments in which + that is out of range by forcing that value to the lowest value that + is valid (fixed bug #1520914). + + Valid ranges based on what is allowed in struct tm: + + - tm_year: [0, max(int)] (1) + - tm_mon: [0, 11] (2) + - tm_mday: [1, 31] + - tm_hour: [0, 23] + - tm_min: [0, 59] + - tm_sec: [0, 60] + - tm_wday: [0, 6] (1) + - tm_yday: [0, 365] (2) + - tm_isdst: [-max(int), max(int)] + + (1) gettmarg() handles bounds-checking. + (2) Python's acceptable range is one greater than the range in C, + thus need to check against automatic decrement by gettmarg(). */ - if (buf.tm_mon < 0 || buf.tm_mon > 11) { + if (buf.tm_mon == -1) + buf.tm_mon = 0; + else if (buf.tm_mon < 0 || buf.tm_mon > 11) { PyErr_SetString(PyExc_ValueError, "month out of range"); return NULL; } - if (buf.tm_mday < 1 || buf.tm_mday > 31) { + if (buf.tm_mday == 0) + buf.tm_mday = 1; + else if (buf.tm_mday < 0 || buf.tm_mday > 31) { PyErr_SetString(PyExc_ValueError, "day of month out of range"); return NULL; } @@ -432,7 +456,9 @@ time_strftime(PyObject *self, PyObject *args) PyErr_SetString(PyExc_ValueError, "day of week out of range"); return NULL; } - if (buf.tm_yday < 0 || buf.tm_yday > 365) { + if (buf.tm_yday == -1) + buf.tm_yday = 0; + else if (buf.tm_yday < 0 || buf.tm_yday > 365) { PyErr_SetString(PyExc_ValueError, "day of year out of range"); return NULL; } @@ -465,6 +491,14 @@ time_strftime(PyObject *self, PyObject *args) return ret; } free(outbuf); +#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) + /* VisualStudio .NET 2005 does this properly */ + if (buflen == 0 && errno == EINVAL) { + PyErr_SetString(PyExc_ValueError, "Invalid format string"); + return 0; + } +#endif + } } diff --git a/Modules/unicodedata.c b/Modules/unicodedata.c index 0660353..1b0be28 100644 --- a/Modules/unicodedata.c +++ b/Modules/unicodedata.c @@ -395,6 +395,7 @@ unicodedata_decomposition(PyObject *self, PyObject *args) PyUnicodeObject *v; char decomp[256]; int code, index, count, i; + unsigned int prefix_index; if (!PyArg_ParseTuple(args, "O!:decomposition", &PyUnicode_Type, &v)) @@ -428,9 +429,15 @@ unicodedata_decomposition(PyObject *self, PyObject *args) /* XXX: could allocate the PyString up front instead (strlen(prefix) + 5 * count + 1 bytes) */ + /* Based on how index is calculated above and decomp_data is generated + from Tools/unicode/makeunicodedata.py, it should not be possible + to overflow decomp_prefix. */ + prefix_index = decomp_data[index] & 255; + assert(prefix_index < (sizeof(decomp_prefix)/sizeof(*decomp_prefix))); + /* copy prefix */ - i = strlen(decomp_prefix[decomp_data[index] & 255]); - memcpy(decomp, decomp_prefix[decomp_data[index] & 255], i); + i = strlen(decomp_prefix[prefix_index]); + memcpy(decomp, decomp_prefix[prefix_index], i); while (count-- > 0) { if (i) diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c index 06b0690..da31e8b 100644 --- a/Modules/zlibmodule.c +++ b/Modules/zlibmodule.c @@ -653,6 +653,7 @@ PyZlib_flush(compobject *self, PyObject *args) return RetVal; } +#ifdef HAVE_ZLIB_COPY PyDoc_STRVAR(comp_copy__doc__, "copy() -- Return a copy of the compression object."); @@ -754,6 +755,7 @@ error: Py_XDECREF(retval); return NULL; } +#endif PyDoc_STRVAR(decomp_flush__doc__, "flush( [length] ) -- Return a string containing any remaining\n" @@ -827,8 +829,10 @@ static PyMethodDef comp_methods[] = comp_compress__doc__}, {"flush", (binaryfunc)PyZlib_flush, METH_VARARGS, comp_flush__doc__}, +#ifdef HAVE_ZLIB_COPY {"copy", (PyCFunction)PyZlib_copy, METH_NOARGS, comp_copy__doc__}, +#endif {NULL, NULL} }; @@ -838,8 +842,10 @@ static PyMethodDef Decomp_methods[] = decomp_decompress__doc__}, {"flush", (binaryfunc)PyZlib_unflush, METH_VARARGS, decomp_flush__doc__}, +#ifdef HAVE_ZLIB_COPY {"copy", (PyCFunction)PyZlib_uncopy, METH_NOARGS, decomp_copy__doc__}, +#endif {NULL, NULL} }; diff --git a/Objects/abstract.c b/Objects/abstract.c index 91ba8c2..de88457 100644 --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -9,9 +9,9 @@ /* Shorthands to return certain errors */ static PyObject * -type_error(const char *msg) +type_error(const char *msg, PyObject *obj) { - PyErr_SetString(PyExc_TypeError, msg); + PyErr_Format(PyExc_TypeError, msg, obj->ob_type->tp_name); return NULL; } @@ -125,10 +125,11 @@ PyObject_GetItem(PyObject *o, PyObject *key) return PySequence_GetItem(o, key_value); } else if (o->ob_type->tp_as_sequence->sq_item) - return type_error("sequence index must be integer"); + return type_error("sequence index must " + "be integer, not '%.200s'", key); } - return type_error("unsubscriptable object"); + return type_error("'%.200s' object is unsubscriptable", o); } int @@ -153,12 +154,13 @@ PyObject_SetItem(PyObject *o, PyObject *key, PyObject *value) return PySequence_SetItem(o, key_value, value); } else if (o->ob_type->tp_as_sequence->sq_ass_item) { - type_error("sequence index must be integer"); + type_error("sequence index must be " + "integer, not '%.200s'", key); return -1; } } - type_error("object does not support item assignment"); + type_error("'%.200s' object does not support item assignment", o); return -1; } @@ -184,12 +186,13 @@ PyObject_DelItem(PyObject *o, PyObject *key) return PySequence_DelItem(o, key_value); } else if (o->ob_type->tp_as_sequence->sq_ass_item) { - type_error("sequence index must be integer"); + type_error("sequence index must be " + "integer, not '%.200s'", key); return -1; } } - type_error("object does not support item deletion"); + type_error("'%.200s' object does not support item deletion", o); return -1; } @@ -408,7 +411,8 @@ static PyObject * binop_type_error(PyObject *v, PyObject *w, const char *op_name) { PyErr_Format(PyExc_TypeError, - "unsupported operand type(s) for %s: '%s' and '%s'", + "unsupported operand type(s) for %.100s: " + "'%.100s' and '%.100s'", op_name, v->ob_type->tp_name, w->ob_type->tp_name); @@ -514,14 +518,14 @@ ternary_op(PyObject *v, PyErr_Format( PyExc_TypeError, "unsupported operand type(s) for ** or pow(): " - "'%s' and '%s'", + "'%.100s' and '%.100s'", v->ob_type->tp_name, w->ob_type->tp_name); else PyErr_Format( PyExc_TypeError, "unsupported operand type(s) for pow(): " - "'%s', '%s', '%s'", + "'%.100s', '%.100s', '%.100s'", v->ob_type->tp_name, w->ob_type->tp_name, z->ob_type->tp_name); @@ -568,8 +572,8 @@ sequence_repeat(ssizeargfunc repeatfunc, PyObject *seq, PyObject *n) return NULL; } else { - return type_error( - "can't multiply sequence by non-int"); + return type_error("can't multiply sequence by " + "non-int of type '%.200s'", n); } return (*repeatfunc)(seq, count); } @@ -776,7 +780,7 @@ PyNumber_Negative(PyObject *o) if (m && m->nb_negative) return (*m->nb_negative)(o); - return type_error("bad operand type for unary -"); + return type_error("bad operand type for unary -: '%.200s'", o); } PyObject * @@ -790,7 +794,7 @@ PyNumber_Positive(PyObject *o) if (m && m->nb_positive) return (*m->nb_positive)(o); - return type_error("bad operand type for unary +"); + return type_error("bad operand type for unary +: '%.200s'", o); } PyObject * @@ -804,7 +808,7 @@ PyNumber_Invert(PyObject *o) if (m && m->nb_invert) return (*m->nb_invert)(o); - return type_error("bad operand type for unary ~"); + return type_error("bad operand type for unary ~: '%.200s'", o); } PyObject * @@ -818,7 +822,7 @@ PyNumber_Absolute(PyObject *o) if (m && m->nb_absolute) return m->nb_absolute(o); - return type_error("bad operand type for abs()"); + return type_error("bad operand type for abs(): '%.200s'", o); } /* Add a check for embedded NULL-bytes in the argument. */ @@ -898,7 +902,8 @@ PyNumber_Int(PyObject *o) if (!PyObject_AsCharBuffer(o, &buffer, &buffer_len)) return int_from_string((char*)buffer, buffer_len); - return type_error("int() argument must be a string or a number"); + return type_error("int() argument must be a string or a " + "number, not '%.200s'", o); } /* Add a check for embedded NULL-bytes in the argument. */ @@ -960,7 +965,8 @@ PyNumber_Long(PyObject *o) if (!PyObject_AsCharBuffer(o, &buffer, &buffer_len)) return long_from_string(buffer, buffer_len); - return type_error("long() argument must be a string or a number"); + return type_error("long() argument must be a string or a " + "number, not '%.200s'", o); } PyObject * @@ -1014,7 +1020,7 @@ PySequence_Size(PyObject *s) if (m && m->sq_length) return m->sq_length(s); - type_error("len() of unsized object"); + type_error("object of type '%.200s' has no len()", s); return -1; } @@ -1047,7 +1053,7 @@ PySequence_Concat(PyObject *s, PyObject *o) return result; Py_DECREF(result); } - return type_error("object can't be concatenated"); + return type_error("'%.200s' object can't be concatenated", s); } PyObject * @@ -1076,7 +1082,7 @@ PySequence_Repeat(PyObject *o, Py_ssize_t count) return result; Py_DECREF(result); } - return type_error("object can't be repeated"); + return type_error("'%.200s' object can't be repeated", o); } PyObject * @@ -1100,7 +1106,7 @@ PySequence_InPlaceConcat(PyObject *s, PyObject *o) return result; Py_DECREF(result); } - return type_error("object can't be concatenated"); + return type_error("'%.200s' object can't be concatenated", s); } PyObject * @@ -1129,7 +1135,7 @@ PySequence_InPlaceRepeat(PyObject *o, Py_ssize_t count) return result; Py_DECREF(result); } - return type_error("object can't be repeated"); + return type_error("'%.200s' object can't be repeated", o); } PyObject * @@ -1153,7 +1159,7 @@ PySequence_GetItem(PyObject *s, Py_ssize_t i) return m->sq_item(s, i); } - return type_error("unindexable object"); + return type_error("'%.200s' object is unindexable", s); } PyObject * @@ -1188,7 +1194,7 @@ PySequence_GetSlice(PyObject *s, Py_ssize_t i1, Py_ssize_t i2) return res; } - return type_error("unsliceable object"); + return type_error("'%.200s' object is unsliceable", s); } int @@ -1214,7 +1220,7 @@ PySequence_SetItem(PyObject *s, Py_ssize_t i, PyObject *o) return m->sq_ass_item(s, i, o); } - type_error("object does not support item assignment"); + type_error("'%.200s' object does not support item assignment", s); return -1; } @@ -1241,7 +1247,7 @@ PySequence_DelItem(PyObject *s, Py_ssize_t i) return m->sq_ass_item(s, i, (PyObject *)NULL); } - type_error("object doesn't support item deletion"); + type_error("'%.200s' object doesn't support item deletion", s); return -1; } @@ -1280,7 +1286,7 @@ PySequence_SetSlice(PyObject *s, Py_ssize_t i1, Py_ssize_t i2, PyObject *o) return res; } - type_error("object doesn't support slice assignment"); + type_error("'%.200s' object doesn't support slice assignment", s); return -1; } @@ -1309,7 +1315,7 @@ PySequence_DelSlice(PyObject *s, Py_ssize_t i1, Py_ssize_t i2) } return m->sq_ass_slice(s, i1, i2, (PyObject *)NULL); } - type_error("object doesn't support slice deletion"); + type_error("'%.200s' object doesn't support slice deletion", s); return -1; } @@ -1440,7 +1446,7 @@ PySequence_Fast(PyObject *v, const char *m) it = PyObject_GetIter(v); if (it == NULL) { if (PyErr_ExceptionMatches(PyExc_TypeError)) - return type_error(m); + PyErr_SetString(PyExc_TypeError, m); return NULL; } @@ -1470,7 +1476,7 @@ _PySequence_IterSearch(PyObject *seq, PyObject *obj, int operation) it = PyObject_GetIter(seq); if (it == NULL) { - type_error("iterable argument required"); + type_error("argument of type '%.200s' is not iterable", seq); return -1; } @@ -1603,7 +1609,7 @@ PyMapping_Size(PyObject *o) if (m && m->mp_length) return m->mp_length(o); - type_error("len() of unsized object"); + type_error("object of type '%.200s' has no len()", o); return -1; } @@ -1701,7 +1707,7 @@ PyObject_Call(PyObject *func, PyObject *arg, PyObject *kw) "NULL result without error in PyObject_Call"); return result; } - PyErr_Format(PyExc_TypeError, "'%s' object is not callable", + PyErr_Format(PyExc_TypeError, "'%.200s' object is not callable", func->ob_type->tp_name); return NULL; } @@ -1790,7 +1796,7 @@ PyObject_CallMethod(PyObject *o, char *name, char *format, ...) } if (!PyCallable_Check(func)) { - type_error("call of non-callable attribute"); + type_error("attribute of type '%.200s' is not callable", func); goto exit; } @@ -1829,7 +1835,7 @@ _PyObject_CallMethod_SizeT(PyObject *o, char *name, char *format, ...) } if (!PyCallable_Check(func)) { - type_error("call of non-callable attribute"); + type_error("attribute of type '%.200s' is not callable", func); goto exit; } @@ -2170,9 +2176,7 @@ PyObject_GetIter(PyObject *o) if (f == NULL) { if (PySequence_Check(o)) return PySeqIter_New(o); - PyErr_SetString(PyExc_TypeError, - "iteration over non-sequence"); - return NULL; + return type_error("'%.200s' object is not iterable", o); } else { PyObject *res = (*f)(o); diff --git a/Objects/bufferobject.c b/Objects/bufferobject.c index d2597b9..3a0e3d5 100644 --- a/Objects/bufferobject.c +++ b/Objects/bufferobject.c @@ -15,8 +15,16 @@ typedef struct { } PyBufferObject; +enum buffer_t { + READ_BUFFER, + WRITE_BUFFER, + CHAR_BUFFER, + ANY_BUFFER, +}; + static int -get_buf(PyBufferObject *self, void **ptr, Py_ssize_t *size) +get_buf(PyBufferObject *self, void **ptr, Py_ssize_t *size, + enum buffer_t buffer_type) { if (self->b_base == NULL) { assert (ptr != NULL); @@ -25,17 +33,43 @@ get_buf(PyBufferObject *self, void **ptr, Py_ssize_t *size) } else { Py_ssize_t count, offset; - readbufferproc proc; + readbufferproc proc = 0; PyBufferProcs *bp = self->b_base->ob_type->tp_as_buffer; if ((*bp->bf_getsegcount)(self->b_base, NULL) != 1) { PyErr_SetString(PyExc_TypeError, "single-segment buffer object expected"); return 0; } - if (self->b_readonly) - proc = bp->bf_getreadbuffer; - else - proc = (readbufferproc)bp->bf_getwritebuffer; + if ((buffer_type == READ_BUFFER) || + ((buffer_type == ANY_BUFFER) && self->b_readonly)) + proc = bp->bf_getreadbuffer; + else if ((buffer_type == WRITE_BUFFER) || + (buffer_type == ANY_BUFFER)) + proc = (readbufferproc)bp->bf_getwritebuffer; + else if (buffer_type == CHAR_BUFFER) { + proc = (readbufferproc)bp->bf_getcharbuffer; + } + if (!proc) { + char *buffer_type_name; + switch (buffer_type) { + case READ_BUFFER: + buffer_type_name = "read"; + break; + case WRITE_BUFFER: + buffer_type_name = "write"; + break; + case CHAR_BUFFER: + buffer_type_name = "char"; + break; + default: + buffer_type_name = "no"; + break; + } + PyErr_Format(PyExc_TypeError, + "%s buffer type not available", + buffer_type_name); + return 0; + } if ((count = (*proc)(self->b_base, 0, ptr)) < 0) return 0; /* apply constraints to the start/end */ @@ -224,15 +258,15 @@ buffer_compare(PyBufferObject *self, PyBufferObject *other) Py_ssize_t len_self, len_other, min_len; int cmp; - if (!get_buf(self, &p1, &len_self)) + if (!get_buf(self, &p1, &len_self, ANY_BUFFER)) return -1; - if (!get_buf(other, &p2, &len_other)) + if (!get_buf(other, &p2, &len_other, ANY_BUFFER)) return -1; min_len = (len_self < len_other) ? len_self : len_other; if (min_len > 0) { cmp = memcmp(p1, p2, min_len); if (cmp != 0) - return cmp; + return cmp < 0 ? -1 : 1; } return (len_self < len_other) ? -1 : (len_self > len_other) ? 1 : 0; } @@ -284,7 +318,7 @@ buffer_hash(PyBufferObject *self) return -1; } - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return -1; p = (unsigned char *) ptr; len = size; @@ -303,7 +337,7 @@ buffer_str(PyBufferObject *self) { void *ptr; Py_ssize_t size; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return NULL; return PyString_FromStringAndSize((const char *)ptr, size); } @@ -315,7 +349,7 @@ buffer_length(PyBufferObject *self) { void *ptr; Py_ssize_t size; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return -1; return size; } @@ -344,7 +378,7 @@ buffer_concat(PyBufferObject *self, PyObject *other) return NULL; } - if (!get_buf(self, &ptr1, &size)) + if (!get_buf(self, &ptr1, &size, ANY_BUFFER)) return NULL; /* optimize special case */ @@ -380,7 +414,7 @@ buffer_repeat(PyBufferObject *self, Py_ssize_t count) if ( count < 0 ) count = 0; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return NULL; ob = PyString_FromStringAndSize(NULL, size * count); if ( ob == NULL ) @@ -404,7 +438,7 @@ buffer_item(PyBufferObject *self, Py_ssize_t idx) { void *ptr; Py_ssize_t size; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return NULL; if ( idx < 0 || idx >= size ) { PyErr_SetString(PyExc_IndexError, "buffer index out of range"); @@ -418,7 +452,7 @@ buffer_slice(PyBufferObject *self, Py_ssize_t left, Py_ssize_t right) { void *ptr; Py_ssize_t size; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return NULL; if ( left < 0 ) left = 0; @@ -446,7 +480,7 @@ buffer_ass_item(PyBufferObject *self, Py_ssize_t idx, PyObject *other) return -1; } - if (!get_buf(self, &ptr1, &size)) + if (!get_buf(self, &ptr1, &size, ANY_BUFFER)) return -1; if (idx < 0 || idx >= size) { @@ -513,7 +547,7 @@ buffer_ass_slice(PyBufferObject *self, Py_ssize_t left, Py_ssize_t right, PyObje "single-segment buffer object expected"); return -1; } - if (!get_buf(self, &ptr1, &size)) + if (!get_buf(self, &ptr1, &size, ANY_BUFFER)) return -1; if ( (count = (*pb->bf_getreadbuffer)(other, 0, &ptr2)) < 0 ) return -1; @@ -552,7 +586,7 @@ buffer_getreadbuf(PyBufferObject *self, Py_ssize_t idx, void **pp) "accessing non-existent buffer segment"); return -1; } - if (!get_buf(self, pp, &size)) + if (!get_buf(self, pp, &size, READ_BUFFER)) return -1; return size; } @@ -560,12 +594,22 @@ buffer_getreadbuf(PyBufferObject *self, Py_ssize_t idx, void **pp) static Py_ssize_t buffer_getwritebuf(PyBufferObject *self, Py_ssize_t idx, void **pp) { + Py_ssize_t size; + if ( self->b_readonly ) { PyErr_SetString(PyExc_TypeError, "buffer is read-only"); return -1; } - return buffer_getreadbuf(self, idx, pp); + + if ( idx != 0 ) { + PyErr_SetString(PyExc_SystemError, + "accessing non-existent buffer segment"); + return -1; + } + if (!get_buf(self, pp, &size, WRITE_BUFFER)) + return -1; + return size; } static Py_ssize_t @@ -573,7 +617,7 @@ buffer_getsegcount(PyBufferObject *self, Py_ssize_t *lenp) { void *ptr; Py_ssize_t size; - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, ANY_BUFFER)) return -1; if (lenp) *lenp = size; @@ -590,13 +634,12 @@ buffer_getcharbuf(PyBufferObject *self, Py_ssize_t idx, const char **pp) "accessing non-existent buffer segment"); return -1; } - if (!get_buf(self, &ptr, &size)) + if (!get_buf(self, &ptr, &size, CHAR_BUFFER)) return -1; *pp = (const char *)ptr; return size; } - static PySequenceMethods buffer_as_sequence = { (lenfunc)buffer_length, /*sq_length*/ (binaryfunc)buffer_concat, /*sq_concat*/ diff --git a/Objects/cellobject.c b/Objects/cellobject.c index da48dea..65a29aa 100644 --- a/Objects/cellobject.c +++ b/Objects/cellobject.c @@ -8,6 +8,8 @@ PyCell_New(PyObject *obj) PyCellObject *op; op = (PyCellObject *)PyObject_GC_New(PyCellObject, &PyCell_Type); + if (op == NULL) + return NULL; op->ob_ref = obj; Py_XINCREF(obj); diff --git a/Objects/classobject.c b/Objects/classobject.c index fa8a3a9..2d80073 100644 --- a/Objects/classobject.c +++ b/Objects/classobject.c @@ -933,11 +933,9 @@ instance_hash(PyInstanceObject *inst) Py_DECREF(func); if (res == NULL) return -1; - if (PyInt_Check(res)) { - outcome = PyInt_AsLong(res); - if (outcome == -1) - outcome = -2; - } + if (PyInt_Check(res) || PyLong_Check(res)) + /* This already converts a -1 result to -2. */ + outcome = res->ob_type->tp_hash(res); else { PyErr_SetString(PyExc_TypeError, "__hash__() should return an int"); @@ -1367,10 +1365,13 @@ half_binop(PyObject *v, PyObject *w, char *opname, binaryfunc thisfunc, * argument */ result = generic_binary_op(v1, w, opname); } else { + if (Py_EnterRecursiveCall(" after coercion")) + return NULL; if (swapped) result = (thisfunc)(w, v1); else result = (thisfunc)(v1, w); + Py_LeaveRecursiveCall(); } Py_DECREF(coerced); return result; diff --git a/Objects/codeobject.c b/Objects/codeobject.c index a9bcb01..89871d6 100644 --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -556,6 +556,7 @@ PyCode_CheckLineNumber(PyCodeObject* co, int lasti, PyAddrPair *bounds) the line increments here, treating them as byte increments gets confusing, to say the least. */ + bounds->ap_lower = 0; while (size > 0) { if (addr + *p > lasti) break; diff --git a/Objects/complexobject.c b/Objects/complexobject.c index c6e3343..aa8fc81 100644 --- a/Objects/complexobject.c +++ b/Objects/complexobject.c @@ -274,16 +274,16 @@ complex_to_buf(char *buf, int bufsz, PyComplexObject *v, int precision) { char format[32]; if (v->cval.real == 0.) { - PyOS_snprintf(format, 32, "%%.%ig", precision); - PyOS_ascii_formatd(buf, bufsz, format, v->cval.imag); - strncat(buf, "j", bufsz); + PyOS_snprintf(format, sizeof(format), "%%.%ig", precision); + PyOS_ascii_formatd(buf, bufsz - 1, format, v->cval.imag); + strncat(buf, "j", 1); } else { char re[64], im[64]; /* Format imaginary part with sign, real part without */ - PyOS_snprintf(format, 32, "%%.%ig", precision); - PyOS_ascii_formatd(re, 64, format, v->cval.real); - PyOS_snprintf(format, 32, "%%+.%ig", precision); - PyOS_ascii_formatd(im, 64, format, v->cval.imag); + PyOS_snprintf(format, sizeof(format), "%%.%ig", precision); + PyOS_ascii_formatd(re, sizeof(re), format, v->cval.real); + PyOS_snprintf(format, sizeof(format), "%%+.%ig", precision); + PyOS_ascii_formatd(im, sizeof(im), format, v->cval.imag); PyOS_snprintf(buf, bufsz, "(%s%sj)", re, im); } } diff --git a/Objects/descrobject.c b/Objects/descrobject.c index 606ef05..914b6d3 100644 --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -892,10 +892,12 @@ typedef struct { static void wrapper_dealloc(wrapperobject *wp) { - _PyObject_GC_UNTRACK(wp); + PyObject_GC_UnTrack(wp); + Py_TRASHCAN_SAFE_BEGIN(wp) Py_XDECREF(wp->descr); Py_XDECREF(wp->self); PyObject_GC_Del(wp); + Py_TRASHCAN_SAFE_END(wp) } static int @@ -1174,7 +1176,6 @@ static int property_init(PyObject *self, PyObject *args, PyObject *kwds) { PyObject *get = NULL, *set = NULL, *del = NULL, *doc = NULL; - PyObject *get_doc = NULL; static char *kwlist[] = {"fget", "fset", "fdel", "doc", 0}; propertyobject *gs = (propertyobject *)self; @@ -1189,20 +1190,22 @@ property_init(PyObject *self, PyObject *args, PyObject *kwds) if (del == Py_None) del = NULL; - /* if no docstring given and the getter has one, use that one */ - if ((doc == NULL || doc == Py_None) && get != NULL && - PyObject_HasAttrString(get, "__doc__")) { - if (!(get_doc = PyObject_GetAttrString(get, "__doc__"))) - return -1; - Py_DECREF(get_doc); /* it is INCREF'd again below */ - doc = get_doc; - } - Py_XINCREF(get); Py_XINCREF(set); Py_XINCREF(del); Py_XINCREF(doc); + /* if no docstring given and the getter has one, use that one */ + if ((doc == NULL || doc == Py_None) && get != NULL) { + PyObject *get_doc = PyObject_GetAttrString(get, "__doc__"); + if (get_doc != NULL) { + Py_XDECREF(doc); + doc = get_doc; /* get_doc already INCREF'd by GetAttr */ + } else { + PyErr_Clear(); + } + } + gs->prop_get = get; gs->prop_set = set; gs->prop_del = del; diff --git a/Objects/dictnotes.txt b/Objects/dictnotes.txt index cb46cb1..b0e59a7 100644 --- a/Objects/dictnotes.txt +++ b/Objects/dictnotes.txt @@ -243,7 +243,7 @@ kept just for iteration. Caching Lookups --------------- The idea is to exploit key access patterns by anticipating future lookups -based of previous lookups. +based on previous lookups. The simplest incarnation is to save the most recently accessed entry. This gives optimal performance for use cases where every get is followed diff --git a/Objects/dictobject.c b/Objects/dictobject.c index f9e45fd..f3b6b7f 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -532,7 +532,7 @@ dictresize(dictobject *mp, Py_ssize_t minused) /* Note that, for historical reasons, PyDict_GetItem() suppresses all errors * that may occur (originally dicts supported only string keys, and exceptions * weren't possible). So, while the original intent was that a NULL return - * meant the key wasn't present, it reality it can mean that, or that an error + * meant the key wasn't present, in reality it can mean that, or that an error * (suppressed) occurred while computing the key's hash, or that some error * (suppressed) occurred when comparing keys in the dict's internal probe * sequence. A nasty example of the latter is when a Python-coded comparison @@ -561,7 +561,7 @@ PyDict_GetItem(PyObject *op, PyObject *key) /* We can arrive here with a NULL tstate during initialization: try running "python -Wi" for an example related to string interning. Let's just hope that no exception occurs then... */ - tstate = PyThreadState_GET(); + tstate = _PyThreadState_Current; if (tstate != NULL && tstate->curexc_type != NULL) { /* preserve the existing exception */ PyObject *err_type, *err_value, *err_tb; @@ -599,6 +599,8 @@ PyDict_SetItem(register PyObject *op, PyObject *key, PyObject *value) PyErr_BadInternalCall(); return -1; } + assert(key); + assert(value); mp = (dictobject *)op; if (PyString_CheckExact(key)) { hash = ((PyStringObject *)key)->ob_shash; @@ -647,6 +649,7 @@ PyDict_DelItem(PyObject *op, PyObject *key) PyErr_BadInternalCall(); return -1; } + assert(key); if (!PyString_CheckExact(key) || (hash = ((PyStringObject *) key)->ob_shash) == -1) { hash = PyObject_Hash(key); diff --git a/Objects/exceptions.c b/Objects/exceptions.c index 369365b..be9627c 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -1967,6 +1967,29 @@ static PyMethodDef functions[] = { if (PyDict_SetItemString(bdict, # TYPE, PyExc_ ## TYPE)) \ Py_FatalError("Module dictionary insertion problem."); +#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) +/* crt variable checking in VisualStudio .NET 2005 */ +#include + +static int prevCrtReportMode; +static _invalid_parameter_handler prevCrtHandler; + +/* Invalid parameter handler. Sets a ValueError exception */ +static void +InvalidParameterHandler( + const wchar_t * expression, + const wchar_t * function, + const wchar_t * file, + unsigned int line, + uintptr_t pReserved) +{ + /* Do nothing, allow execution to continue. Usually this + * means that the CRT will set errno to EINVAL + */ +} +#endif + + PyMODINIT_FUNC _PyExc_Init(void) { @@ -2096,6 +2119,13 @@ _PyExc_Init(void) Py_FatalError("Cannot pre-allocate MemoryError instance\n"); Py_DECREF(bltinmod); + +#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) + /* Set CRT argument error handler */ + prevCrtHandler = _set_invalid_parameter_handler(InvalidParameterHandler); + /* turn off assertions in debug mode */ + prevCrtReportMode = _CrtSetReportMode(_CRT_ASSERT, 0); +#endif } void @@ -2103,4 +2133,9 @@ _PyExc_Fini(void) { Py_XDECREF(PyExc_MemoryErrorInst); PyExc_MemoryErrorInst = NULL; +#if defined _MSC_VER && _MSC_VER >= 1400 && defined(__STDC_SECURE_LIB__) + /* reset CRT error handling */ + _set_invalid_parameter_handler(prevCrtHandler); + _CrtSetReportMode(_CRT_ASSERT, prevCrtReportMode); +#endif } diff --git a/Objects/fileobject.c b/Objects/fileobject.c index a4a43ed..fb8a542 100644 --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -4,9 +4,9 @@ #include "Python.h" #include "structmember.h" -#ifndef DONT_HAVE_SYS_TYPES_H +#ifdef HAVE_SYS_TYPES_H #include -#endif /* DONT_HAVE_SYS_TYPES_H */ +#endif /* HAVE_SYS_TYPES_H */ #ifdef MS_WINDOWS #define fileno _fileno @@ -103,6 +103,7 @@ static PyObject * fill_file_fields(PyFileObject *f, FILE *fp, PyObject *name, char *mode, int (*close)(FILE *)) { + assert(name != NULL); assert(f != NULL); assert(PyFile_Check(f)); assert(f->f_fp == NULL); @@ -111,7 +112,7 @@ fill_file_fields(PyFileObject *f, FILE *fp, PyObject *name, char *mode, Py_DECREF(f->f_mode); Py_DECREF(f->f_encoding); - Py_INCREF (name); + Py_INCREF(name); f->f_name = name; f->f_mode = PyString_FromString(mode); @@ -126,7 +127,7 @@ fill_file_fields(PyFileObject *f, FILE *fp, PyObject *name, char *mode, Py_INCREF(Py_None); f->f_encoding = Py_None; - if (f->f_name == NULL || f->f_mode == NULL) + if (f->f_mode == NULL) return NULL; f->f_fp = fp; f = dircheck(f); @@ -241,13 +242,15 @@ open_the_file(PyFileObject *f, char *name, char *mode) } if (f->f_fp == NULL) { -#ifdef _MSC_VER +#if defined _MSC_VER && (_MSC_VER < 1400 || !defined(__STDC_SECURE_LIB__)) /* MSVC 6 (Microsoft) leaves errno at 0 for bad mode strings, * across all Windows flavors. When it sets EINVAL varies * across Windows flavors, the exact conditions aren't * documented, and the answer lies in the OS's implementation * of Win32's CreateFile function (whose source is secret). * Seems the best we can do is map EINVAL to ENOENT. + * Starting with Visual Studio .NET 2005, EINVAL is correctly + * set by our CRT error handler (set in exceptions.c.) */ if (errno == 0) /* bad mode string */ errno = EINVAL; @@ -276,7 +279,9 @@ PyFile_FromFile(FILE *fp, char *name, char *mode, int (*close)(FILE *)) PyFileObject *f = (PyFileObject *)PyFile_Type.tp_new(&PyFile_Type, NULL, NULL); if (f != NULL) { - PyObject *o_name = PyString_FromString(name); + PyObject *o_name = PyString_FromString(name); + if (o_name == NULL) + return NULL; if (fill_file_fields(f, fp, o_name, mode, close) == NULL) { Py_DECREF(f); f = NULL; @@ -409,11 +414,11 @@ file_repr(PyFileObject *f) if (PyUnicode_Check(f->f_name)) { #ifdef Py_USING_UNICODE PyObject *ret = NULL; - PyObject *name; - name = PyUnicode_AsUnicodeEscapeString(f->f_name); + PyObject *name = PyUnicode_AsUnicodeEscapeString(f->f_name); + const char *name_str = name ? PyString_AsString(name) : "?"; ret = PyString_FromFormat("<%s file u'%s', mode '%s' at %p>", f->f_fp == NULL ? "closed" : "open", - PyString_AsString(name), + name_str, PyString_AsString(f->f_mode), f); Py_XDECREF(name); diff --git a/Objects/frameobject.c b/Objects/frameobject.c index fcb5e4e..3a073b6 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -20,7 +20,6 @@ static PyMemberDef frame_memberlist[] = { {"f_builtins", T_OBJECT, OFF(f_builtins),RO}, {"f_globals", T_OBJECT, OFF(f_globals), RO}, {"f_lasti", T_INT, OFF(f_lasti), RO}, - {"f_restricted",T_INT, OFF(f_restricted),RO}, {"f_exc_type", T_OBJECT, OFF(f_exc_type)}, {"f_exc_value", T_OBJECT, OFF(f_exc_value)}, {"f_exc_traceback", T_OBJECT, OFF(f_exc_traceback)}, @@ -341,11 +340,18 @@ frame_settrace(PyFrameObject *f, PyObject* v, void *closure) return 0; } +static PyObject * +frame_getrestricted(PyFrameObject *f, void *closure) +{ + return PyBool_FromLong(PyFrame_IsRestricted(f)); +} + static PyGetSetDef frame_getsetlist[] = { {"f_locals", (getter)frame_getlocals, NULL, NULL}, {"f_lineno", (getter)frame_getlineno, (setter)frame_setlineno, NULL}, {"f_trace", (getter)frame_gettrace, (setter)frame_settrace, NULL}, + {"f_restricted",(getter)frame_getrestricted,NULL, NULL}, {0} }; @@ -425,7 +431,7 @@ frame_dealloc(PyFrameObject *f) Py_CLEAR(f->f_exc_traceback); co = f->f_code; - if (co != NULL && co->co_zombieframe == NULL) + if (co->co_zombieframe == NULL) co->co_zombieframe = f; else if (numfree < MAXFREELIST) { ++numfree; @@ -435,7 +441,7 @@ frame_dealloc(PyFrameObject *f) else PyObject_GC_Del(f); - Py_XDECREF(co); + Py_DECREF(co); Py_TRASHCAN_SAFE_END(f) } @@ -636,6 +642,7 @@ PyFrame_New(PyThreadState *tstate, PyCodeObject *code, PyObject *globals, f->f_trace = NULL; f->f_exc_type = f->f_exc_value = f->f_exc_traceback = NULL; } + f->f_stacktop = f->f_valuestack; f->f_builtins = builtins; Py_XINCREF(back); f->f_back = back; @@ -664,10 +671,8 @@ PyFrame_New(PyThreadState *tstate, PyCodeObject *code, PyObject *globals, f->f_lasti = -1; f->f_lineno = code->co_firstlineno; - f->f_restricted = (builtins != tstate->interp->builtins); f->f_iblock = 0; - f->f_stacktop = f->f_valuestack; _PyObject_GC_TRACK(f); return f; } diff --git a/Objects/funcobject.c b/Objects/funcobject.c index 59cb519..1ba74c5 100644 --- a/Objects/funcobject.c +++ b/Objects/funcobject.c @@ -109,8 +109,8 @@ PyFunction_SetDefaults(PyObject *op, PyObject *defaults) } if (defaults == Py_None) defaults = NULL; - else if (PyTuple_Check(defaults)) { - Py_XINCREF(defaults); + else if (defaults && PyTuple_Check(defaults)) { + Py_INCREF(defaults); } else { PyErr_SetString(PyExc_SystemError, "non-tuple default args"); @@ -141,7 +141,7 @@ PyFunction_SetClosure(PyObject *op, PyObject *closure) if (closure == Py_None) closure = NULL; else if (PyTuple_Check(closure)) { - Py_XINCREF(closure); + Py_INCREF(closure); } else { PyErr_Format(PyExc_SystemError, diff --git a/Objects/listobject.c b/Objects/listobject.c index e128473..ea03a2a 100644 --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -108,8 +108,10 @@ PyList_New(Py_ssize_t size) op->ob_item = NULL; else { op->ob_item = (PyObject **) PyMem_MALLOC(nbytes); - if (op->ob_item == NULL) + if (op->ob_item == NULL) { + Py_DECREF(op); return PyErr_NoMemory(); + } memset(op->ob_item, 0, nbytes); } op->ob_size = size; diff --git a/Objects/listsort.txt b/Objects/listsort.txt index 68e918d..9226984 100644 --- a/Objects/listsort.txt +++ b/Objects/listsort.txt @@ -494,7 +494,7 @@ and its followup(s). An earlier paper called the same strategy 467-474, Austin, Texas, 25-27 January 1993. and it probably dates back to an earlier paper by Bentley and Yao. The -McIlory paper in particular has good analysis of a mergesort that's +McIlroy paper in particular has good analysis of a mergesort that's probably strongly related to this one in its galloping strategy. diff --git a/Objects/longobject.c b/Objects/longobject.c index 45166fe..9d3685e 100644 --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -1203,7 +1203,7 @@ long_format(PyObject *aa, int base, int addL) register PyLongObject *a = (PyLongObject *)aa; PyStringObject *str; Py_ssize_t i; - const Py_ssize_t size_a = ABS(a->ob_size); + Py_ssize_t size_a; char *p; int bits; char sign = '\0'; @@ -1213,6 +1213,7 @@ long_format(PyObject *aa, int base, int addL) return NULL; } assert(base >= 2 && base <= 36); + size_a = ABS(a->ob_size); /* Compute a rough upper bound for the length of the string */ i = base; @@ -3133,9 +3134,8 @@ long_bitwise(PyLongObject *a, : MAX(size_a, size_b); z = _PyLong_New(size_z); if (z == NULL) { - Py_XDECREF(a); - Py_XDECREF(b); - Py_XDECREF(z); + Py_DECREF(a); + Py_DECREF(b); return NULL; } diff --git a/Objects/object.c b/Objects/object.c index 0af7989..ff13574 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -1067,7 +1067,8 @@ PyObject_Hash(PyObject *v) return _Py_HashPointer(v); /* Use address as hash value */ } /* If there's a cmp but no hash defined, the object can't be hashed */ - PyErr_SetString(PyExc_TypeError, "unhashable type"); + PyErr_Format(PyExc_TypeError, "unhashable type: '%.200s'", + v->ob_type->tp_name); return -1; } @@ -1132,8 +1133,9 @@ PyObject_GetAttr(PyObject *v, PyObject *name) else #endif { - PyErr_SetString(PyExc_TypeError, - "attribute name must be string"); + PyErr_Format(PyExc_TypeError, + "attribute name must be string, not '%.200s'", + name->ob_type->tp_name); return NULL; } } @@ -1178,8 +1180,9 @@ PyObject_SetAttr(PyObject *v, PyObject *name, PyObject *value) else #endif { - PyErr_SetString(PyExc_TypeError, - "attribute name must be string"); + PyErr_Format(PyExc_TypeError, + "attribute name must be string, not '%.200s'", + name->ob_type->tp_name); return -1; } } @@ -1274,8 +1277,9 @@ PyObject_GenericGetAttr(PyObject *obj, PyObject *name) else #endif { - PyErr_SetString(PyExc_TypeError, - "attribute name must be string"); + PyErr_Format(PyExc_TypeError, + "attribute name must be string, not '%.200s'", + name->ob_type->tp_name); return NULL; } } @@ -1395,8 +1399,9 @@ PyObject_GenericSetAttr(PyObject *obj, PyObject *name, PyObject *value) else #endif { - PyErr_SetString(PyExc_TypeError, - "attribute name must be string"); + PyErr_Format(PyExc_TypeError, + "attribute name must be string, not '%.200s'", + name->ob_type->tp_name); return -1; } } @@ -1445,7 +1450,7 @@ PyObject_GenericSetAttr(PyObject *obj, PyObject *name, PyObject *value) if (descr == NULL) { PyErr_Format(PyExc_AttributeError, - "'%.50s' object has no attribute '%.400s'", + "'%.100s' object has no attribute '%.200s'", tp->tp_name, PyString_AS_STRING(name)); goto done; } @@ -1760,8 +1765,9 @@ PyObject_Dir(PyObject *arg) assert(result); if (!PyList_Check(result)) { - PyErr_SetString(PyExc_TypeError, - "Expected keys() to be a list."); + PyErr_Format(PyExc_TypeError, + "Expected keys() to be a list, not '%.200s'", + result->ob_type->tp_name); goto error; } if (PyList_Sort(result) != 0) diff --git a/Objects/setobject.c b/Objects/setobject.c index 91682a2..55a2b85 100644 --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -62,7 +62,7 @@ set_lookkey(PySetObject *so, PyObject *key, register long hash) register Py_ssize_t i; register size_t perturb; register setentry *freeslot; - register unsigned int mask = so->mask; + register size_t mask = so->mask; setentry *table = so->table; register setentry *entry; register int cmp; @@ -140,7 +140,7 @@ set_lookkey_string(PySetObject *so, PyObject *key, register long hash) register Py_ssize_t i; register size_t perturb; register setentry *freeslot; - register unsigned int mask = so->mask; + register size_t mask = so->mask; setentry *table = so->table; register setentry *entry; @@ -221,11 +221,11 @@ keys again. When entries have been deleted, the new table may actually be smaller than the old one. */ static int -set_table_resize(PySetObject *so, int minused) +set_table_resize(PySetObject *so, Py_ssize_t minused) { - int newsize; + Py_ssize_t newsize; setentry *oldtable, *newtable, *entry; - int i; + Py_ssize_t i; int is_oldtable_malloced; setentry small_copy[PySet_MINSIZE]; @@ -314,7 +314,7 @@ set_table_resize(PySetObject *so, int minused) static int set_add_entry(register PySetObject *so, setentry *entry) { - register int n_used; + register Py_ssize_t n_used; assert(so->fill <= so->mask); /* at least one empty slot */ n_used = so->used; @@ -330,7 +330,7 @@ static int set_add_key(register PySetObject *so, PyObject *key) { register long hash; - register int n_used; + register Py_ssize_t n_used; if (!PyString_CheckExact(key) || (hash = ((PyStringObject *) key)->ob_shash) == -1) { @@ -403,10 +403,10 @@ set_clear_internal(PySetObject *so) { setentry *entry, *table; int table_is_malloced; - int fill; + Py_ssize_t fill; setentry small_copy[PySet_MINSIZE]; #ifdef Py_DEBUG - int i, n; + Py_ssize_t i, n; assert (PyAnySet_Check(so)); n = so->mask + 1; @@ -465,7 +465,7 @@ set_clear_internal(PySetObject *so) /* * Iterate over a set table. Use like so: * - * int pos; + * Py_ssize_t pos; * setentry *entry; * pos = 0; # important! pos should not otherwise be changed by you * while (set_next(yourset, &pos, &entry)) { @@ -479,7 +479,7 @@ static int set_next(PySetObject *so, Py_ssize_t *pos_ptr, setentry **entry_ptr) { Py_ssize_t i; - int mask; + Py_ssize_t mask; register setentry *table; assert (PyAnySet_Check(so)); @@ -501,7 +501,7 @@ static void set_dealloc(PySetObject *so) { register setentry *entry; - int fill = so->fill; + Py_ssize_t fill = so->fill; PyObject_GC_UnTrack(so); Py_TRASHCAN_SAFE_BEGIN(so) if (so->weakreflist != NULL) @@ -570,7 +570,7 @@ static int set_merge(PySetObject *so, PyObject *otherset) { PySetObject *other; - register int i; + register Py_ssize_t i; register setentry *entry; assert (PyAnySet_Check(so)); @@ -637,7 +637,7 @@ set_contains_entry(PySetObject *so, setentry *entry) static PyObject * set_pop(PySetObject *so) { - register int i = 0; + register Py_ssize_t i = 0; register setentry *entry; PyObject *key; @@ -655,7 +655,7 @@ set_pop(PySetObject *so) */ entry = &so->table[0]; if (entry->key == NULL || entry->key == dummy) { - i = (int)entry->hash; + i = entry->hash; /* The hash field may be a real hash value, or it may be a * legit search finger, or it may be a once-legit search * finger that's out of bounds now because it wrapped around @@ -730,9 +730,9 @@ set_nohash(PyObject *self) typedef struct { PyObject_HEAD PySetObject *si_set; /* Set to NULL when iterator is exhausted */ - int si_used; - int si_pos; - long len; + Py_ssize_t si_used; + Py_ssize_t si_pos; + Py_ssize_t len; } setiterobject; static void @@ -745,7 +745,7 @@ setiter_dealloc(setiterobject *si) static PyObject * setiter_len(setiterobject *si) { - long len = 0; + Py_ssize_t len = 0; if (si->si_set != NULL && si->si_used == si->si_set->used) len = si->len; return PyInt_FromLong(len); @@ -761,7 +761,7 @@ static PyMethodDef setiter_methods[] = { static PyObject *setiter_iternext(setiterobject *si) { PyObject *key; - register int i, mask; + register Py_ssize_t i, mask; register setentry *entry; PySetObject *so = si->si_set; @@ -1007,7 +1007,7 @@ set_new(PyTypeObject *type, PyObject *args, PyObject *kwds) static void set_swap_bodies(PySetObject *a, PySetObject *b) { - int t; + Py_ssize_t t; setentry *u; setentry *(*f)(PySetObject *so, PyObject *key, long hash); setentry tab[PySet_MINSIZE]; @@ -1380,12 +1380,12 @@ set_symmetric_difference_update(PySetObject *so, PyObject *other) while (set_next(otherset, &pos, &entry)) { int rv = set_discard_entry(so, entry); if (rv == -1) { - Py_XDECREF(otherset); + Py_DECREF(otherset); return NULL; } if (rv == DISCARD_NOTFOUND) { if (set_add_entry(so, entry) == -1) { - Py_XDECREF(otherset); + Py_DECREF(otherset); return NULL; } } @@ -1795,7 +1795,7 @@ static PyNumberMethods set_as_number = { PyDoc_STRVAR(set_doc, "set(iterable) --> set object\n\ \n\ -Build an unordered collection."); +Build an unordered collection of unique elements."); PyTypeObject PySet_Type = { PyObject_HEAD_INIT(&PyType_Type) @@ -1889,7 +1889,7 @@ static PyNumberMethods frozenset_as_number = { PyDoc_STRVAR(frozenset_doc, "frozenset(iterable) --> frozenset object\n\ \n\ -Build an immutable unordered collection."); +Build an immutable unordered collection of unique elements."); PyTypeObject PyFrozenSet_Type = { PyObject_HEAD_INIT(&PyType_Type) @@ -2061,7 +2061,7 @@ _PySet_Update(PyObject *set, PyObject *iterable) static PyObject * test_c_api(PySetObject *so) { - int count; + Py_ssize_t count; char *s; Py_ssize_t i; PyObject *elem, *dup, *t, *f, *dup2; diff --git a/Objects/stringobject.c b/Objects/stringobject.c index 6a760a2..0819c98 100644 --- a/Objects/stringobject.c +++ b/Objects/stringobject.c @@ -1490,7 +1490,6 @@ string_split(PyStringObject *self, PyObject *args) j = i+pos; SPLIT_ADD(s, i, j); i = j + n; - } #else i = j = 0; @@ -1586,7 +1585,7 @@ rsplit_whitespace(const char *s, Py_ssize_t len, Py_ssize_t maxsplit) return NULL; i = j = len-1; - + while (maxsplit-- > 0) { RSKIP_SPACE(s, i); if (i<0) break; @@ -1786,7 +1785,7 @@ string_join(PyStringObject *self, PyObject *orig) sz += seplen; if (sz < old_sz || sz > PY_SSIZE_T_MAX) { PyErr_SetString(PyExc_OverflowError, - "join() is too long for a Python string"); + "join() result is too long for a Python string"); Py_DECREF(seq); return NULL; } @@ -2462,11 +2461,11 @@ return_self(PyStringObject *self) } Py_LOCAL_INLINE(Py_ssize_t) -countchar(char *target, int target_len, char c, Py_ssize_t maxcount) +countchar(const char *target, int target_len, char c, Py_ssize_t maxcount) { Py_ssize_t count=0; - char *start=target; - char *end=target+target_len; + const char *start=target; + const char *end=target+target_len; while ( (start=findchar(start, end-start, c)) != NULL ) { count++; @@ -2478,8 +2477,8 @@ countchar(char *target, int target_len, char c, Py_ssize_t maxcount) } Py_LOCAL(Py_ssize_t) -findstring(char *target, Py_ssize_t target_len, - char *pattern, Py_ssize_t pattern_len, +findstring(const char *target, Py_ssize_t target_len, + const char *pattern, Py_ssize_t pattern_len, Py_ssize_t start, Py_ssize_t end, int direction) @@ -2516,8 +2515,8 @@ findstring(char *target, Py_ssize_t target_len, } Py_LOCAL_INLINE(Py_ssize_t) -countstring(char *target, Py_ssize_t target_len, - char *pattern, Py_ssize_t pattern_len, +countstring(const char *target, Py_ssize_t target_len, + const char *pattern, Py_ssize_t pattern_len, Py_ssize_t start, Py_ssize_t end, int direction, Py_ssize_t maxcount) @@ -2570,22 +2569,21 @@ countstring(char *target, Py_ssize_t target_len, /* len(self)>=1, from="", len(to)>=1, maxcount>=1 */ Py_LOCAL(PyStringObject *) replace_interleave(PyStringObject *self, - PyStringObject *to, + const char *to_s, Py_ssize_t to_len, Py_ssize_t maxcount) { - char *self_s, *to_s, *result_s; - Py_ssize_t self_len, to_len, result_len; + char *self_s, *result_s; + Py_ssize_t self_len, result_len; Py_ssize_t count, i, product; PyStringObject *result; self_len = PyString_GET_SIZE(self); - to_len = PyString_GET_SIZE(to); - + /* 1 at the end plus 1 after every character */ count = self_len+1; if (maxcount < count) count = maxcount; - + /* Check for overflow */ /* result_len = count * to_len + self_len; */ product = count * to_len; @@ -2606,8 +2604,6 @@ replace_interleave(PyStringObject *self, return NULL; self_s = PyString_AS_STRING(self); - to_s = PyString_AS_STRING(to); - to_len = PyString_GET_SIZE(to); result_s = PyString_AS_STRING(result); /* TODO: special case single character, which doesn't need memcpy */ @@ -2668,25 +2664,24 @@ replace_delete_single_character(PyStringObject *self, start = next+1; } Py_MEMCPY(result_s, start, end-start); - + return result; } /* len(self)>=1, len(from)>=2, to="", maxcount>=1 */ Py_LOCAL(PyStringObject *) -replace_delete_substring(PyStringObject *self, PyStringObject *from, +replace_delete_substring(PyStringObject *self, + const char *from_s, Py_ssize_t from_len, Py_ssize_t maxcount) { - char *self_s, *from_s, *result_s; + char *self_s, *result_s; char *start, *next, *end; - Py_ssize_t self_len, from_len, result_len; + Py_ssize_t self_len, result_len; Py_ssize_t count, offset; PyStringObject *result; self_len = PyString_GET_SIZE(self); self_s = PyString_AS_STRING(self); - from_len = PyString_GET_SIZE(from); - from_s = PyString_AS_STRING(from); count = countstring(self_s, self_len, from_s, from_len, @@ -2700,13 +2695,13 @@ replace_delete_substring(PyStringObject *self, PyStringObject *from, result_len = self_len - (count * from_len); assert (result_len>=0); - + if ( (result = (PyStringObject *) PyString_FromStringAndSize(NULL, result_len)) == NULL ) return NULL; - + result_s = PyString_AS_STRING(result); - + start = self_s; end = self_s + self_len; while (count-- > 0) { @@ -2716,9 +2711,9 @@ replace_delete_substring(PyStringObject *self, PyStringObject *from, if (offset == -1) break; next = start + offset; - + Py_MEMCPY(result_s, start, next-start); - + result_s += (next-start); start = next+from_len; } @@ -2735,31 +2730,31 @@ replace_single_character_in_place(PyStringObject *self, char *self_s, *result_s, *start, *end, *next; Py_ssize_t self_len; PyStringObject *result; - + /* The result string will be the same size */ self_s = PyString_AS_STRING(self); self_len = PyString_GET_SIZE(self); - + next = findchar(self_s, self_len, from_c); - + if (next == NULL) { /* No matches; return the original string */ return return_self(self); } - + /* Need to make a new string */ result = (PyStringObject *) PyString_FromStringAndSize(NULL, self_len); if (result == NULL) return NULL; result_s = PyString_AS_STRING(result); Py_MEMCPY(result_s, self_s, self_len); - + /* change everything in-place, starting with this one */ start = result_s + (next-self_s); *start = to_c; start++; end = result_s + self_len; - + while (--maxcount > 0) { next = findchar(start, end-start, from_c); if (next == NULL) @@ -2767,40 +2762,35 @@ replace_single_character_in_place(PyStringObject *self, *next = to_c; start = next+1; } - + return result; } /* len(self)>=1, len(from)==len(to)>=2, maxcount>=1 */ Py_LOCAL(PyStringObject *) replace_substring_in_place(PyStringObject *self, - PyStringObject *from, - PyStringObject *to, + const char *from_s, Py_ssize_t from_len, + const char *to_s, Py_ssize_t to_len, Py_ssize_t maxcount) { char *result_s, *start, *end; - char *self_s, *from_s, *to_s; - Py_ssize_t self_len, from_len, offset; + char *self_s; + Py_ssize_t self_len, offset; PyStringObject *result; - + /* The result string will be the same size */ - + self_s = PyString_AS_STRING(self); self_len = PyString_GET_SIZE(self); - - from_s = PyString_AS_STRING(from); - from_len = PyString_GET_SIZE(from); - to_s = PyString_AS_STRING(to); - + offset = findstring(self_s, self_len, from_s, from_len, 0, self_len, FORWARD); - if (offset == -1) { /* No matches; return the original string */ return return_self(self); } - + /* Need to make a new string */ result = (PyStringObject *) PyString_FromStringAndSize(NULL, self_len); if (result == NULL) @@ -2808,13 +2798,12 @@ replace_substring_in_place(PyStringObject *self, result_s = PyString_AS_STRING(result); Py_MEMCPY(result_s, self_s, self_len); - /* change everything in-place, starting with this one */ start = result_s + offset; Py_MEMCPY(start, to_s, from_len); start += from_len; end = result_s + self_len; - + while ( --maxcount > 0) { offset = findstring(start, end-start, from_s, from_len, @@ -2824,7 +2813,7 @@ replace_substring_in_place(PyStringObject *self, Py_MEMCPY(start+offset, to_s, from_len); start += offset+from_len; } - + return result; } @@ -2832,28 +2821,24 @@ replace_substring_in_place(PyStringObject *self, Py_LOCAL(PyStringObject *) replace_single_character(PyStringObject *self, char from_c, - PyStringObject *to, + const char *to_s, Py_ssize_t to_len, Py_ssize_t maxcount) { - char *self_s, *to_s, *result_s; + char *self_s, *result_s; char *start, *next, *end; - Py_ssize_t self_len, to_len, result_len; + Py_ssize_t self_len, result_len; Py_ssize_t count, product; PyStringObject *result; - + self_s = PyString_AS_STRING(self); self_len = PyString_GET_SIZE(self); - + count = countchar(self_s, self_len, from_c, maxcount); - if (count == 0) { /* no matches, return unchanged */ return return_self(self); } - - to_s = PyString_AS_STRING(to); - to_len = PyString_GET_SIZE(to); - + /* use the difference between current and new, hence the "-1" */ /* result_len = self_len + count * (to_len-1) */ product = count * (to_len-1); @@ -2866,19 +2851,19 @@ replace_single_character(PyStringObject *self, PyErr_SetString(PyExc_OverflowError, "replace string is too long"); return NULL; } - + if ( (result = (PyStringObject *) PyString_FromStringAndSize(NULL, result_len)) == NULL) return NULL; result_s = PyString_AS_STRING(result); - + start = self_s; end = self_s + self_len; while (count-- > 0) { next = findchar(start, end-start, from_c); if (next == NULL) break; - + if (next == start) { /* replace with the 'to' */ Py_MEMCPY(result_s, to_s, to_len); @@ -2895,27 +2880,25 @@ replace_single_character(PyStringObject *self, } /* Copy the remainder of the remaining string */ Py_MEMCPY(result_s, start, end-start); - + return result; } /* len(self)>=1, len(from)>=2, len(to)>=2, maxcount>=1 */ Py_LOCAL(PyStringObject *) replace_substring(PyStringObject *self, - PyStringObject *from, - PyStringObject *to, + const char *from_s, Py_ssize_t from_len, + const char *to_s, Py_ssize_t to_len, Py_ssize_t maxcount) { - char *self_s, *from_s, *to_s, *result_s; + char *self_s, *result_s; char *start, *next, *end; - Py_ssize_t self_len, from_len, to_len, result_len; + Py_ssize_t self_len, result_len; Py_ssize_t count, offset, product; PyStringObject *result; - + self_s = PyString_AS_STRING(self); self_len = PyString_GET_SIZE(self); - from_s = PyString_AS_STRING(from); - from_len = PyString_GET_SIZE(from); - + count = countstring(self_s, self_len, from_s, from_len, 0, self_len, FORWARD, maxcount); @@ -2923,10 +2906,7 @@ replace_substring(PyStringObject *self, /* no matches, return unchanged */ return return_self(self); } - - to_s = PyString_AS_STRING(to); - to_len = PyString_GET_SIZE(to); - + /* Check for overflow */ /* result_len = self_len + count * (to_len-from_len) */ product = count * (to_len-from_len); @@ -2939,12 +2919,12 @@ replace_substring(PyStringObject *self, PyErr_SetString(PyExc_OverflowError, "replace string is too long"); return NULL; } - + if ( (result = (PyStringObject *) PyString_FromStringAndSize(NULL, result_len)) == NULL) return NULL; result_s = PyString_AS_STRING(result); - + start = self_s; end = self_s + self_len; while (count-- > 0) { @@ -2970,29 +2950,24 @@ replace_substring(PyStringObject *self, } /* Copy the remainder of the remaining string */ Py_MEMCPY(result_s, start, end-start); - + return result; } Py_LOCAL(PyStringObject *) replace(PyStringObject *self, - PyStringObject *from, - PyStringObject *to, + const char *from_s, Py_ssize_t from_len, + const char *to_s, Py_ssize_t to_len, Py_ssize_t maxcount) { - Py_ssize_t from_len, to_len; - if (maxcount < 0) { maxcount = PY_SSIZE_T_MAX; } else if (maxcount == 0 || PyString_GET_SIZE(self) == 0) { /* nothing to do; return the original string */ return return_self(self); } - - from_len = PyString_GET_SIZE(from); - to_len = PyString_GET_SIZE(to); - + if (maxcount == 0 || (from_len == 0 && to_len == 0)) { /* nothing to do; return the original string */ @@ -3000,12 +2975,12 @@ replace(PyStringObject *self, } /* Handle zero-length special cases */ - + if (from_len == 0) { /* insert the 'to' string everywhere. */ /* >>> "Python".replace("", ".") */ /* '.P.y.t.h.o.n.' */ - return replace_interleave(self, to, maxcount); + return replace_interleave(self, to_s, to_len, maxcount); } /* Except for "".replace("", "A") == "A" there is no way beyond this */ @@ -3019,9 +2994,9 @@ replace(PyStringObject *self, /* delete all occurances of 'from' string */ if (from_len == 1) { return replace_delete_single_character( - self, PyString_AS_STRING(from)[0], maxcount); + self, from_s[0], maxcount); } else { - return replace_delete_substring(self, from, maxcount); + return replace_delete_substring(self, from_s, from_len, maxcount); } } @@ -3031,22 +3006,22 @@ replace(PyStringObject *self, if (from_len == 1) { return replace_single_character_in_place( self, - PyString_AS_STRING(from)[0], - PyString_AS_STRING(to)[0], + from_s[0], + to_s[0], maxcount); } else { return replace_substring_in_place( - self, from, to, maxcount); + self, from_s, from_len, to_s, to_len, maxcount); } } /* Otherwise use the more generic algorithms */ if (from_len == 1) { - return replace_single_character(self, PyString_AS_STRING(from)[0], - to, maxcount); + return replace_single_character(self, from_s[0], + to_s, to_len, maxcount); } else { /* len('from')>=2, len('to')>=1 */ - return replace_substring(self, from, to, maxcount); + return replace_substring(self, from_s, from_len, to_s, to_len, maxcount); } } @@ -3062,89 +3037,127 @@ string_replace(PyStringObject *self, PyObject *args) { Py_ssize_t count = -1; PyObject *from, *to; - const char *tmp_s; - Py_ssize_t tmp_len; + const char *from_s, *to_s; + Py_ssize_t from_len, to_len; if (!PyArg_ParseTuple(args, "OO|n:replace", &from, &to, &count)) return NULL; if (PyString_Check(from)) { - /* Can this be made a '!check' after the Unicode check? */ + from_s = PyString_AS_STRING(from); + from_len = PyString_GET_SIZE(from); } #ifdef Py_USING_UNICODE if (PyUnicode_Check(from)) return PyUnicode_Replace((PyObject *)self, from, to, count); #endif - else if (PyObject_AsCharBuffer(from, &tmp_s, &tmp_len)) + else if (PyObject_AsCharBuffer(from, &from_s, &from_len)) return NULL; if (PyString_Check(to)) { - /* Can this be made a '!check' after the Unicode check? */ + to_s = PyString_AS_STRING(to); + to_len = PyString_GET_SIZE(to); } #ifdef Py_USING_UNICODE else if (PyUnicode_Check(to)) return PyUnicode_Replace((PyObject *)self, from, to, count); #endif - else if (PyObject_AsCharBuffer(to, &tmp_s, &tmp_len)) + else if (PyObject_AsCharBuffer(to, &to_s, &to_len)) return NULL; return (PyObject *)replace((PyStringObject *) self, - (PyStringObject *) from, - (PyStringObject *) to, count); + from_s, from_len, + to_s, to_len, count); } /** End DALKE **/ +/* Matches the end (direction >= 0) or start (direction < 0) of self + * against substr, using the start and end arguments. Returns + * -1 on error, 0 if not found and 1 if found. + */ +Py_LOCAL(int) +_string_tailmatch(PyStringObject *self, PyObject *substr, Py_ssize_t start, + Py_ssize_t end, int direction) +{ + Py_ssize_t len = PyString_GET_SIZE(self); + Py_ssize_t slen; + const char* sub; + const char* str; + + if (PyString_Check(substr)) { + sub = PyString_AS_STRING(substr); + slen = PyString_GET_SIZE(substr); + } +#ifdef Py_USING_UNICODE + else if (PyUnicode_Check(substr)) + return PyUnicode_Tailmatch((PyObject *)self, + substr, start, end, direction); +#endif + else if (PyObject_AsCharBuffer(substr, &sub, &slen)) + return -1; + str = PyString_AS_STRING(self); + + string_adjust_indices(&start, &end, len); + + if (direction < 0) { + /* startswith */ + if (start+slen > len) + return 0; + } else { + /* endswith */ + if (end-start < slen || start > len) + return 0; + + if (end-slen > start) + start = end - slen; + } + if (end-start >= slen) + return ! memcmp(str+start, sub, slen); + return 0; +} + + PyDoc_STRVAR(startswith__doc__, "S.startswith(prefix[, start[, end]]) -> bool\n\ \n\ Return True if S starts with the specified prefix, False otherwise.\n\ With optional start, test S beginning at that position.\n\ -With optional end, stop comparing S at that position."); +With optional end, stop comparing S at that position.\n\ +prefix can also be a tuple of strings to try."); static PyObject * string_startswith(PyStringObject *self, PyObject *args) { - const char* str = PyString_AS_STRING(self); - Py_ssize_t len = PyString_GET_SIZE(self); - const char* prefix; - Py_ssize_t plen; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; PyObject *subobj; + int result; if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &subobj, _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) return NULL; - if (PyString_Check(subobj)) { - prefix = PyString_AS_STRING(subobj); - plen = PyString_GET_SIZE(subobj); - } -#ifdef Py_USING_UNICODE - else if (PyUnicode_Check(subobj)) { - Py_ssize_t rc; - rc = PyUnicode_Tailmatch((PyObject *)self, - subobj, start, end, -1); - if (rc == -1) - return NULL; - else - return PyBool_FromLong((long) rc); + if (PyTuple_Check(subobj)) { + Py_ssize_t i; + for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) { + result = _string_tailmatch(self, + PyTuple_GET_ITEM(subobj, i), + start, end, -1); + if (result == -1) + return NULL; + else if (result) { + Py_RETURN_TRUE; + } + } + Py_RETURN_FALSE; } -#endif - else if (PyObject_AsCharBuffer(subobj, &prefix, &plen)) + result = _string_tailmatch(self, subobj, start, end, -1); + if (result == -1) return NULL; - - string_adjust_indices(&start, &end, len); - - if (start+plen > len) - return PyBool_FromLong(0); - - if (end-start >= plen) - return PyBool_FromLong(!memcmp(str+start, prefix, plen)); else - return PyBool_FromLong(0); + return PyBool_FromLong(result); } @@ -3153,51 +3166,39 @@ PyDoc_STRVAR(endswith__doc__, \n\ Return True if S ends with the specified suffix, False otherwise.\n\ With optional start, test S beginning at that position.\n\ -With optional end, stop comparing S at that position."); +With optional end, stop comparing S at that position.\n\ +suffix can also be a tuple of strings to try."); static PyObject * string_endswith(PyStringObject *self, PyObject *args) { - const char* str = PyString_AS_STRING(self); - Py_ssize_t len = PyString_GET_SIZE(self); - const char* suffix; - Py_ssize_t slen; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; PyObject *subobj; + int result; if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &subobj, _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) return NULL; - if (PyString_Check(subobj)) { - suffix = PyString_AS_STRING(subobj); - slen = PyString_GET_SIZE(subobj); - } -#ifdef Py_USING_UNICODE - else if (PyUnicode_Check(subobj)) { - Py_ssize_t rc; - rc = PyUnicode_Tailmatch((PyObject *)self, - subobj, start, end, +1); - if (rc == -1) - return NULL; - else - return PyBool_FromLong((long) rc); + if (PyTuple_Check(subobj)) { + Py_ssize_t i; + for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) { + result = _string_tailmatch(self, + PyTuple_GET_ITEM(subobj, i), + start, end, +1); + if (result == -1) + return NULL; + else if (result) { + Py_RETURN_TRUE; + } + } + Py_RETURN_FALSE; } -#endif - else if (PyObject_AsCharBuffer(subobj, &suffix, &slen)) + result = _string_tailmatch(self, subobj, start, end, +1); + if (result == -1) return NULL; - - string_adjust_indices(&start, &end, len); - - if (end-start < slen || start > len) - return PyBool_FromLong(0); - - if (end-slen > start) - start = end - slen; - if (end-start >= slen) - return PyBool_FromLong(!memcmp(str+start, suffix, slen)); else - return PyBool_FromLong(0); + return PyBool_FromLong(result); } diff --git a/Objects/typeobject.c b/Objects/typeobject.c index c8bc61d..1ec345f 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -1466,8 +1466,9 @@ subtype_setdict(PyObject *obj, PyObject *value, void *context) return -1; } if (value != NULL && !PyDict_Check(value)) { - PyErr_SetString(PyExc_TypeError, - "__dict__ must be set to a dictionary"); + PyErr_Format(PyExc_TypeError, + "__dict__ must be set to a dictionary, " + "not a '%.200s'", value->ob_type->tp_name); return -1; } dict = *dictptr; @@ -1485,7 +1486,7 @@ subtype_getweakref(PyObject *obj, void *context) if (obj->ob_type->tp_weaklistoffset == 0) { PyErr_SetString(PyExc_AttributeError, - "This object has no __weaklist__"); + "This object has no __weakref__"); return NULL; } assert(obj->ob_type->tp_weaklistoffset > 0); @@ -1530,8 +1531,9 @@ valid_identifier(PyObject *s) Py_ssize_t i, n; if (!PyString_Check(s)) { - PyErr_SetString(PyExc_TypeError, - "__slots__ must be strings"); + PyErr_Format(PyExc_TypeError, + "__slots__ items must be strings, not '%.200s'", + s->ob_type->tp_name); return 0; } p = (unsigned char *) PyString_AS_STRING(s); @@ -2565,8 +2567,9 @@ reduce_2(PyObject *obj) args = PyObject_CallObject(getnewargs, NULL); Py_DECREF(getnewargs); if (args != NULL && !PyTuple_Check(args)) { - PyErr_SetString(PyExc_TypeError, - "__getnewargs__ should return a tuple"); + PyErr_Format(PyExc_TypeError, + "__getnewargs__ should return a tuple, " + "not '%.200s'", args->ob_type->tp_name); goto end; } } @@ -3206,6 +3209,8 @@ PyType_Ready(PyTypeObject *type) if (PyDict_GetItemString(type->tp_dict, "__doc__") == NULL) { if (type->tp_doc != NULL) { PyObject *doc = PyString_FromString(type->tp_doc); + if (doc == NULL) + goto error; PyDict_SetItemString(type->tp_dict, "__doc__", doc); Py_DECREF(doc); } else { @@ -4294,8 +4299,9 @@ slot_nb_index(PyObject *self) result = temp->ob_type->tp_as_number->nb_index(temp); } else { - PyErr_SetString(PyExc_TypeError, - "__index__ must return an int or a long"); + PyErr_Format(PyExc_TypeError, + "__index__ must return an int or a long, " + "not '%.200s'", temp->ob_type->tp_name); result = -1; } Py_DECREF(temp); @@ -4494,7 +4500,10 @@ slot_tp_hash(PyObject *self) Py_DECREF(func); if (res == NULL) return -1; - h = PyInt_AsLong(res); + if (PyLong_Check(res)) + h = PyLong_Type.tp_hash(res); + else + h = PyInt_AsLong(res); Py_DECREF(res); } else { @@ -4505,8 +4514,9 @@ slot_tp_hash(PyObject *self) func = lookup_method(self, "__cmp__", &cmp_str); } if (func != NULL) { + PyErr_Format(PyExc_TypeError, "unhashable type: '%.200s'", + self->ob_type->tp_name); Py_DECREF(func); - PyErr_SetString(PyExc_TypeError, "unhashable type"); return -1; } PyErr_Clear(); @@ -4526,7 +4536,18 @@ slot_tp_call(PyObject *self, PyObject *args, PyObject *kwds) if (meth == NULL) return NULL; + + /* PyObject_Call() will end up calling slot_tp_call() again if + the object returned for __call__ has __call__ itself defined + upon it. This can be an infinite recursion if you set + __call__ in a class to an instance of it. */ + if (Py_EnterRecursiveCall(" in __call__")) { + Py_DECREF(meth); + return NULL; + } res = PyObject_Call(meth, args, kwds); + Py_LeaveRecursiveCall(); + Py_DECREF(meth); return res; } @@ -4683,8 +4704,9 @@ slot_tp_iter(PyObject *self) PyErr_Clear(); func = lookup_method(self, "__getitem__", &getitem_str); if (func == NULL) { - PyErr_SetString(PyExc_TypeError, - "iteration over non-sequence"); + PyErr_Format(PyExc_TypeError, + "'%.200s' object is not iterable", + self->ob_type->tp_name); return NULL; } Py_DECREF(func); @@ -4757,8 +4779,9 @@ slot_tp_init(PyObject *self, PyObject *args, PyObject *kwds) if (res == NULL) return -1; if (res != Py_None) { - PyErr_SetString(PyExc_TypeError, - "__init__() should return None"); + PyErr_Format(PyExc_TypeError, + "__init__() should return None, not '%.200s'", + res->ob_type->tp_name); Py_DECREF(res); return -1; } diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index c3ab2d8..9e35b61 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -2820,65 +2820,199 @@ PyObject *PyUnicode_AsASCIIString(PyObject *unicode) /* --- MBCS codecs for Windows -------------------------------------------- */ -PyObject *PyUnicode_DecodeMBCS(const char *s, - Py_ssize_t size, - const char *errors) +#if SIZEOF_INT < SIZEOF_SSIZE_T +#define NEED_RETRY +#endif + +/* XXX This code is limited to "true" double-byte encodings, as + a) it assumes an incomplete character consists of a single byte, and + b) IsDBCSLeadByte (probably) does not work for non-DBCS multi-byte + encodings, see IsDBCSLeadByteEx documentation. */ + +static int is_dbcs_lead_byte(const char *s, int offset) +{ + const char *curr = s + offset; + + if (IsDBCSLeadByte(*curr)) { + const char *prev = CharPrev(s, curr); + return (prev == curr) || !IsDBCSLeadByte(*prev) || (curr - prev == 2); + } + return 0; +} + +/* + * Decode MBCS string into unicode object. If 'final' is set, converts + * trailing lead-byte too. Returns consumed size if succeed, -1 otherwise. + */ +static int decode_mbcs(PyUnicodeObject **v, + const char *s, /* MBCS string */ + int size, /* sizeof MBCS string */ + int final) { - PyUnicodeObject *v; Py_UNICODE *p; - DWORD usize; + Py_ssize_t n = 0; + int usize = 0; + + assert(size >= 0); + + /* Skip trailing lead-byte unless 'final' is set */ + if (!final && size >= 1 && is_dbcs_lead_byte(s, size - 1)) + --size; /* First get the size of the result */ - assert(size < INT_MAX); - usize = MultiByteToWideChar(CP_ACP, 0, s, (int)size, NULL, 0); - if (size > 0 && usize==0) - return PyErr_SetFromWindowsErrWithFilename(0, NULL); + if (size > 0) { + usize = MultiByteToWideChar(CP_ACP, 0, s, size, NULL, 0); + if (usize == 0) { + PyErr_SetFromWindowsErrWithFilename(0, NULL); + return -1; + } + } - v = _PyUnicode_New(usize); - if (v == NULL) - return NULL; - if (usize == 0) - return (PyObject *)v; - p = PyUnicode_AS_UNICODE(v); - if (0 == MultiByteToWideChar(CP_ACP, 0, s, (int)size, p, usize)) { - Py_DECREF(v); - return PyErr_SetFromWindowsErrWithFilename(0, NULL); + if (*v == NULL) { + /* Create unicode object */ + *v = _PyUnicode_New(usize); + if (*v == NULL) + return -1; + } + else { + /* Extend unicode object */ + n = PyUnicode_GET_SIZE(*v); + if (_PyUnicode_Resize(v, n + usize) < 0) + return -1; + } + + /* Do the conversion */ + if (size > 0) { + p = PyUnicode_AS_UNICODE(*v) + n; + if (0 == MultiByteToWideChar(CP_ACP, 0, s, size, p, usize)) { + PyErr_SetFromWindowsErrWithFilename(0, NULL); + return -1; + } + } + + return size; +} + +PyObject *PyUnicode_DecodeMBCSStateful(const char *s, + Py_ssize_t size, + const char *errors, + Py_ssize_t *consumed) +{ + PyUnicodeObject *v = NULL; + int done; + + if (consumed) + *consumed = 0; + +#ifdef NEED_RETRY + retry: + if (size > INT_MAX) + done = decode_mbcs(&v, s, INT_MAX, 0); + else +#endif + done = decode_mbcs(&v, s, (int)size, !consumed); + + if (done < 0) { + Py_XDECREF(v); + return NULL; + } + + if (consumed) + *consumed += done; + +#ifdef NEED_RETRY + if (size > INT_MAX) { + s += done; + size -= done; + goto retry; } +#endif return (PyObject *)v; } -PyObject *PyUnicode_EncodeMBCS(const Py_UNICODE *p, +PyObject *PyUnicode_DecodeMBCS(const char *s, Py_ssize_t size, const char *errors) { - PyObject *repr; - char *s; - DWORD mbcssize; + return PyUnicode_DecodeMBCSStateful(s, size, errors, NULL); +} + +/* + * Convert unicode into string object (MBCS). + * Returns 0 if succeed, -1 otherwise. + */ +static int encode_mbcs(PyObject **repr, + const Py_UNICODE *p, /* unicode */ + int size) /* size of unicode */ +{ + int mbcssize = 0; + Py_ssize_t n = 0; - /* If there are no characters, bail now! */ - if (size==0) - return PyString_FromString(""); + assert(size >= 0); /* First get the size of the result */ - assert(size 0) { + mbcssize = WideCharToMultiByte(CP_ACP, 0, p, size, NULL, 0, NULL, NULL); + if (mbcssize == 0) { + PyErr_SetFromWindowsErrWithFilename(0, NULL); + return -1; + } + } - repr = PyString_FromStringAndSize(NULL, mbcssize); - if (repr == NULL) - return NULL; - if (mbcssize == 0) - return repr; + if (*repr == NULL) { + /* Create string object */ + *repr = PyString_FromStringAndSize(NULL, mbcssize); + if (*repr == NULL) + return -1; + } + else { + /* Extend string object */ + n = PyString_Size(*repr); + if (_PyString_Resize(repr, n + mbcssize) < 0) + return -1; + } /* Do the conversion */ - s = PyString_AS_STRING(repr); - assert(size < INT_MAX); - if (0 == WideCharToMultiByte(CP_ACP, 0, p, (int)size, s, mbcssize, NULL, NULL)) { - Py_DECREF(repr); - return PyErr_SetFromWindowsErrWithFilename(0, NULL); + if (size > 0) { + char *s = PyString_AS_STRING(*repr) + n; + if (0 == WideCharToMultiByte(CP_ACP, 0, p, size, s, mbcssize, NULL, NULL)) { + PyErr_SetFromWindowsErrWithFilename(0, NULL); + return -1; + } + } + + return 0; +} + +PyObject *PyUnicode_EncodeMBCS(const Py_UNICODE *p, + Py_ssize_t size, + const char *errors) +{ + PyObject *repr = NULL; + int ret; + +#ifdef NEED_RETRY + retry: + if (size > INT_MAX) + ret = encode_mbcs(&repr, p, INT_MAX); + else +#endif + ret = encode_mbcs(&repr, p, (int)size); + + if (ret < 0) { + Py_XDECREF(repr); + return NULL; } + +#ifdef NEED_RETRY + if (size > INT_MAX) { + p += INT_MAX; + size -= INT_MAX; + goto retry; + } +#endif + return repr; } @@ -2893,6 +3027,8 @@ PyObject *PyUnicode_AsMBCSString(PyObject *unicode) NULL); } +#undef NEED_RETRY + #endif /* MS_WINDOWS */ /* --- Character Mapping Codec -------------------------------------------- */ @@ -4491,11 +4627,11 @@ PyUnicode_Join(PyObject *separator, PyObject *seq) /* Make sure we have enough space for the separator and the item. */ itemlen = PyUnicode_GET_SIZE(item); new_res_used = res_used + itemlen; - if (new_res_used <= 0) + if (new_res_used < 0) goto Overflow; if (i < seqlen - 1) { new_res_used += seplen; - if (new_res_used <= 0) + if (new_res_used < 0) goto Overflow; } if (new_res_used > res_alloc) { @@ -4536,7 +4672,7 @@ PyUnicode_Join(PyObject *separator, PyObject *seq) Overflow: PyErr_SetString(PyExc_OverflowError, - "join() is too long for a Python string"); + "join() result is too long for a Python string"); Py_DECREF(item); /* fall through */ @@ -6667,29 +6803,44 @@ PyDoc_STRVAR(startswith__doc__, \n\ Return True if S starts with the specified prefix, False otherwise.\n\ With optional start, test S beginning at that position.\n\ -With optional end, stop comparing S at that position."); +With optional end, stop comparing S at that position.\n\ +prefix can also be a tuple of strings to try."); static PyObject * unicode_startswith(PyUnicodeObject *self, PyObject *args) { + PyObject *subobj; PyUnicodeObject *substring; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - PyObject *result; + int result; - if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &substring, + if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &subobj, _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) return NULL; - substring = (PyUnicodeObject *)PyUnicode_FromObject( - (PyObject *)substring); + if (PyTuple_Check(subobj)) { + Py_ssize_t i; + for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) { + substring = (PyUnicodeObject *)PyUnicode_FromObject( + PyTuple_GET_ITEM(subobj, i)); + if (substring == NULL) + return NULL; + result = tailmatch(self, substring, start, end, -1); + Py_DECREF(substring); + if (result) { + Py_RETURN_TRUE; + } + } + /* nothing matched */ + Py_RETURN_FALSE; + } + substring = (PyUnicodeObject *)PyUnicode_FromObject(subobj); if (substring == NULL) - return NULL; - - result = PyBool_FromLong(tailmatch(self, substring, start, end, -1)); - + return NULL; + result = tailmatch(self, substring, start, end, -1); Py_DECREF(substring); - return result; + return PyBool_FromLong(result); } @@ -6698,29 +6849,44 @@ PyDoc_STRVAR(endswith__doc__, \n\ Return True if S ends with the specified suffix, False otherwise.\n\ With optional start, test S beginning at that position.\n\ -With optional end, stop comparing S at that position."); +With optional end, stop comparing S at that position.\n\ +suffix can also be a tuple of strings to try."); static PyObject * unicode_endswith(PyUnicodeObject *self, PyObject *args) { + PyObject *subobj; PyUnicodeObject *substring; Py_ssize_t start = 0; Py_ssize_t end = PY_SSIZE_T_MAX; - PyObject *result; + int result; - if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &substring, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) + if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &subobj, + _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) return NULL; - substring = (PyUnicodeObject *)PyUnicode_FromObject( - (PyObject *)substring); + if (PyTuple_Check(subobj)) { + Py_ssize_t i; + for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) { + substring = (PyUnicodeObject *)PyUnicode_FromObject( + PyTuple_GET_ITEM(subobj, i)); + if (substring == NULL) + return NULL; + result = tailmatch(self, substring, start, end, +1); + Py_DECREF(substring); + if (result) { + Py_RETURN_TRUE; + } + } + Py_RETURN_FALSE; + } + substring = (PyUnicodeObject *)PyUnicode_FromObject(subobj); if (substring == NULL) - return NULL; - - result = PyBool_FromLong(tailmatch(self, substring, start, end, +1)); + return NULL; + result = tailmatch(self, substring, start, end, +1); Py_DECREF(substring); - return result; + return PyBool_FromLong(result); } @@ -7748,6 +7914,9 @@ void _PyUnicode_Init(void) unicode_freelist = NULL; unicode_freelist_size = 0; unicode_empty = _PyUnicode_New(0); + if (!unicode_empty) + return; + strcpy(unicode_default_encoding, "ascii"); for (i = 0; i < 256; i++) unicode_latin1[i] = NULL; @@ -7758,6 +7927,8 @@ void _PyUnicode_Init(void) bloom_linebreak = make_bloom_mask( linebreak, sizeof(linebreak) / sizeof(linebreak[0]) ); + + PyType_Ready(&EncodingMapType); } /* Finalize the Unicode implementation */ diff --git a/Objects/weakrefobject.c b/Objects/weakrefobject.c index c55dd4c..f814306 100644 --- a/Objects/weakrefobject.c +++ b/Objects/weakrefobject.c @@ -6,10 +6,10 @@ ((PyWeakReference **) PyObject_GET_WEAKREFS_LISTPTR(o)) -long +Py_ssize_t _PyWeakref_GetWeakrefCount(PyWeakReference *head) { - long count = 0; + Py_ssize_t count = 0; while (head != NULL) { ++count; diff --git a/PC/_winreg.c b/PC/_winreg.c index 5ed58bb..5ed3f6c 100644 --- a/PC/_winreg.c +++ b/PC/_winreg.c @@ -959,7 +959,9 @@ PyConnectRegistry(PyObject *self, PyObject *args) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; + Py_BEGIN_ALLOW_THREADS rc = RegConnectRegistry(szCompName, hKey, &retKey); + Py_END_ALLOW_THREADS if (rc != ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "ConnectRegistry"); diff --git a/PC/config.c b/PC/config.c index b832d7c..a9a280d 100644 --- a/PC/config.c +++ b/PC/config.c @@ -67,6 +67,7 @@ extern void init_codecs_tw(void); extern void init_subprocess(void); extern void init_lsprof(void); extern void init_ast(void); +extern void init_types(void); /* tools/freeze/makeconfig.py marker for additional "extern" */ /* -- ADDMODULE MARKER 1 -- */ @@ -161,6 +162,8 @@ struct _inittab _PyImport_Inittab[] = { {"__builtin__", NULL}, {"sys", NULL}, {"exceptions", NULL}, + + {"_types", init_types}, /* Sentinel */ {0, 0} diff --git a/PC/getpathp.c b/PC/getpathp.c index 37a9c0b..729d2e4 100644 --- a/PC/getpathp.c +++ b/PC/getpathp.c @@ -62,8 +62,14 @@ #include #endif +#ifdef HAVE_SYS_TYPES_H #include +#endif /* HAVE_SYS_TYPES_H */ + +#ifdef HAVE_SYS_STAT_H #include +#endif /* HAVE_SYS_STAT_H */ + #include /* Search in some common locations for the associated Python libraries. @@ -291,6 +297,10 @@ getpythonregpath(HKEY keyBase, int skipcore) } RegCloseKey(subKey); } + + /* return null if no path to return */ + if (dataSize == 0) goto done; + /* original datasize from RegQueryInfo doesn't include the \0 */ dataBuf = malloc((dataSize+1) * sizeof(TCHAR)); if (dataBuf) { diff --git a/PC/os2emx/Makefile b/PC/os2emx/Makefile index 762bfdb..833f3ab 100644 --- a/PC/os2emx/Makefile +++ b/PC/os2emx/Makefile @@ -2,15 +2,15 @@ # # Top-Level Makefile for Building Python 2.4 for OS/2 using GCC/EMX # Originally written by Andrew Zabolotny, for Python 1.5.2 -# Modified by Andrew MacIntyre, for Python 2.4 +# Modified by Andrew MacIntyre, for Python 2.5 # # This makefile was developed for use with [P]GCC/EMX compiler any # version and GNU Make. # -# The output of the build is a largish Python24.DLL containing the +# The output of the build is a largish Python25.DLL containing the # essential modules of Python and a small Python.exe program to start # the interpreter. When embedding Python within another program, only -# Python24.DLL is needed. We also build python_s.a static library (which +# Python25.DLL is needed. We also build python_s.a static library (which # can be converted into OMF (.lib) format using emxomf tool) and both # python.a and python.lib import libraries. Then the optional # extension modules, which are OS/2 DLLs renamed with a PYD file extension. @@ -64,7 +64,7 @@ HAVE_OPENSSL= no # === install locations === # default value of PYTHONHOME -LIB_DIR=C:/Python24 +LIB_DIR=C:/Python25 # default is to have everything in or under PYTHONHOME EXE_DIR=$(LIB_DIR) DLL_DIR=$(EXE_DIR) @@ -236,8 +236,8 @@ $(OUT)%$O: %.c @echo STACKSIZE 2097152 >>$@ # Output file names -PYTHON_VER= 2.4 -PYTHON_LIB= python24 +PYTHON_VER= 2.5 +PYTHON_LIB= python25 PYTHON.LIB= $(PYTHON_LIB)_s$A PYTHON.IMPLIB= $(PYTHON_LIB)$A ifeq ($(EXEOMF),yes) @@ -295,20 +295,23 @@ SRC.MODULES= $(addprefix $(TOP), \ Modules/dlmodule.c \ Modules/errnomodule.c \ Modules/fcntlmodule.c \ + Modules/_functoolsmodule.c \ Modules/_heapqmodule.c \ Modules/imageop.c \ Modules/itertoolsmodule.c \ Modules/_localemodule.c \ Modules/mathmodule.c \ - Modules/md5c.c \ + Modules/md5.c \ Modules/md5module.c \ Modules/operator.c \ Modules/_randommodule.c \ Modules/rgbimgmodule.c \ Modules/shamodule.c \ + Modules/sha256module.c \ + Modules/sha512module.c \ Modules/_sre.c \ Modules/stropmodule.c \ - Modules/structmodule.c \ + Modules/_struct.c \ Modules/symtablemodule.c \ Modules/termios.c \ Modules/timemodule.c \ @@ -331,6 +334,9 @@ SRC.PARSE2= $(addprefix $(TOP), \ SRC.PARSER= $(SRC.PARSE1) \ $(SRC.PARSE2) SRC.PYTHON= $(addprefix $(TOP), \ + Python/Python-ast.c \ + Python/asdl.c \ + Python/ast.c \ Python/bltinmodule.c \ Python/exceptions.c \ Python/ceval.c \ @@ -353,6 +359,7 @@ SRC.PYTHON= $(addprefix $(TOP), \ Python/modsupport.c \ Python/mysnprintf.c \ Python/mystrtoul.c \ + Python/pyarena.c \ Python/pyfpe.c \ Python/pystate.c \ Python/pystrtod.c \ @@ -371,6 +378,7 @@ SRC.OBJECT= $(addprefix $(TOP), \ Objects/cellobject.c \ Objects/classobject.c \ Objects/cobject.c \ + Objects/codeobject.c \ Objects/complexobject.c \ Objects/descrobject.c \ Objects/dictobject.c \ diff --git a/PC/os2emx/README.os2emx b/PC/os2emx/README.os2emx index 9172a2d..4fe5c53 100644 --- a/PC/os2emx/README.os2emx +++ b/PC/os2emx/README.os2emx @@ -612,22 +612,11 @@ functionality. Most of the sub-tests pass, but the "ismount" and test_posixpath should skip these tests on EMX. -24. I have had a report that attempting to use the Bittorrent package -(http://bitconjurer.org/BitTorrent/) with this port causes traps not -long after starting the download; this using the "headless" download -script on eCS v1.1. I have not been able to duplicate this myself, -but the indications I have suggest a failure in the 32 bit TCP/IP -stack (v4.3.2? on eCS v1.1) - on my v4.0 FP12 system with MPTS fixpack -WR8425 applied (16 bit TCP/IP stack v4.02), BitTorrent appears to work -normally in testing on a 100Mbit LAN. With the curses.panel fix (see -item 13 above), the BitTorrent curses downloader works too. I'd -appreciate any success or failure reports with BitTorrent, though -I've regretfully recommended that the person who reported the failure -take this up with eCS support. Since this report, I have received a -followup which suggests that the problem may be addressed by TCP/IP -fixes (IC35005+PJ29457, contained in NEWSTACK.ZIP in the Hobbes -archive). I think it suffices to say that BitTorrent is a fair stress -test of a system's networking capability. +24. I have reports of BitTorrent not working. It appears that the +EMX select() emulation, possibly in concert with bugs in the TCP/IP +stack, runs into problems under the stress imposed by this application. +I think it suffices to say that BitTorrent is a fair stress test of a +system's networking capability. 25. In the absence of an EMX implementation of the link() function, I've implemented a crude Python emulation, in the file @@ -659,14 +648,16 @@ test_subprocess has a number of failures as a result. 29. The default stack size for threads has been 64k. This is proving insufficient for some codebases, such as Zope. The thread stack size -still defaults to 64k, but this can now be increased by defining +still defaults to 64k, but this can now be increased via the stack_size() +function exposed by the threading & thread modules as well as by defining THREAD_STACK_SIZE to an appropriate value in the Makefile (which contains a commented out definition for 128kB thread stacks). I have seen references to heavy Zope/Plone usage requiring 1MB thread stacks on FreeBSD and Linux, but doubt that for most likely usage on OS/2 that more than 256kB is necessary. The size of the required stacks (main and thread) can vary significantly depending on which version of gcc -is used along with the compiler optimisations selected. +is used along with the compiler optimisations selected. Note that the +main thread stack size is set during linking and is currently 2MB. ... probably other issues that I've not encountered, or don't remember :-( @@ -707,4 +698,4 @@ Andrew MacIntyre E-mail: andymac@bullseye.apana.org.au, or andymac@pcug.org.au Web: http://www.andymac.org/ -17 February, 2005. +23 July, 2006. diff --git a/PC/os2emx/config.c b/PC/os2emx/config.c index 40c2cdc..696c3bb 100644 --- a/PC/os2emx/config.c +++ b/PC/os2emx/config.c @@ -58,16 +58,19 @@ extern void initdatetime(); extern void initdl(); extern void initerrno(); extern void initfcntl(); +extern void init_functools(); extern void init_heapq(); extern void initimageop(); extern void inititertools(); extern void initmath(); -extern void initmd5(); +extern void init_md5(); extern void initoperator(); extern void initrgbimg(); -extern void initsha(); +extern void init_sha(); +extern void init_sha256(); +extern void init_sha512(); extern void initstrop(); -extern void initstruct(); +extern void init_struct(); extern void inittermios(); extern void inittime(); extern void inittiming(); @@ -121,16 +124,19 @@ struct _inittab _PyImport_Inittab[] = { {"dl", initdl}, {"errno", initerrno}, {"fcntl", initfcntl}, + {"_functools", init_functools}, {"_heapq", init_heapq}, {"imageop", initimageop}, {"itertools", inititertools}, {"math", initmath}, - {"md5", initmd5}, + {"_md5", init_md5}, {"operator", initoperator}, {"rgbimg", initrgbimg}, - {"sha", initsha}, + {"_sha", init_sha}, + {"_sha256", init_sha256}, + {"_sha512", init_sha512}, {"strop", initstrop}, - {"struct", initstruct}, + {"_struct", init_struct}, {"termios", inittermios}, {"time", inittime}, {"timing", inittiming}, diff --git a/PC/os2emx/pyconfig.h b/PC/os2emx/pyconfig.h index afe79e4..1039d3d 100644 --- a/PC/os2emx/pyconfig.h +++ b/PC/os2emx/pyconfig.h @@ -46,6 +46,7 @@ #define TCPIPV4 1 #define USE_SOCKET 1 #define socklen_t int +#define FD_SETSIZE 1024 /* enable the Python object allocator */ #define WITH_PYMALLOC 1 @@ -61,6 +62,9 @@ #define PY_UNICODE_TYPE wchar_t #define Py_UNICODE_SIZE SIZEOF_SHORT +/* EMX defines ssize_t */ +#define HAVE_SSIZE_T 1 + /* system capabilities */ #define HAVE_TTYNAME 1 #define HAVE_WAIT 1 @@ -137,6 +141,9 @@ typedef long intptr_t; /* The number of bytes in a void *. */ #define SIZEOF_VOID_P 4 +/* The number of bytes in a size_t. */ +#define SIZEOF_SIZE_T 4 + /* Define if you have the alarm function. */ #define HAVE_ALARM 1 @@ -254,15 +261,33 @@ typedef long intptr_t; /* Define if you have the waitpid function. */ #define HAVE_WAITPID 1 +/* Define if you have the header file. */ +#undef HAVE_CONIO_H + +/* Define if you have the header file. */ +#undef HAVE_DIRECT_H + /* Define if you have the header file. */ #define HAVE_DIRENT_H 1 +/* Define if you have the header file. */ +#define HAVE_ERRNO_H 1 + /* Define if you have the header file. */ #define HAVE_FCNTL_H 1 +/* Define if you have the header file. */ +#undef HAVE_IO_H + /* Define if you have the header file. */ #define HAVE_NCURSES_H 1 +/* Define to 1 if you have the header file. */ +#define HAVE_PROCESS_H 1 + +/* Define if you have the header file. */ +#define HAVE_SIGNAL_H 1 + /* Define if you have the header file. */ #define HAVE_SYS_FILE_H 1 @@ -272,12 +297,18 @@ typedef long intptr_t; /* Define if you have the header file. */ #define HAVE_SYS_SELECT_H 1 +/* Define if you have the header file. */ +#define HAVE_SYS_STAT_H 1 + /* Define if you have the header file. */ #define HAVE_SYS_TIME_H 1 /* Define if you have the header file. */ #define HAVE_SYS_TIMES_H 1 +/* Define if you have the header file. */ +#define HAVE_SYS_TYPES_H 1 + /* Define if you have the header file. */ #define HAVE_SYS_UN_H 1 diff --git a/PC/os2emx/python24.def b/PC/os2emx/python24.def deleted file mode 100644 index 534dff8..0000000 --- a/PC/os2emx/python24.def +++ /dev/null @@ -1,1173 +0,0 @@ -LIBRARY python24 INITINSTANCE TERMINSTANCE -DESCRIPTION "Python 2.4 Core DLL" -PROTMODE -DATA MULTIPLE NONSHARED -EXPORTS - -; From python24_s.lib(config) - "_PyImport_Inittab" - -; From python24_s.lib(dlfcn) -; "dlopen" -; "dlsym" -; "dlclose" -; "dlerror" - -; From python24_s.lib(getpathp) - "Py_GetProgramFullPath" - "Py_GetPrefix" - "Py_GetExecPrefix" - "Py_GetPath" - -; From python24_s.lib(getbuildinfo) - "Py_GetBuildInfo" - -; From python24_s.lib(main) - "Py_Main" - "Py_GetArgcArgv" - -; From python24_s.lib(acceler) - "PyGrammar_AddAccelerators" - "PyGrammar_RemoveAccelerators" - -; From python24_s.lib(grammar1) - "PyGrammar_FindDFA" - "PyGrammar_LabelRepr" - -; From python24_s.lib(listnode) - "PyNode_ListTree" - -; From python24_s.lib(node) - "PyNode_AddChild" - "PyNode_New" - "PyNode_Free" - -; From python24_s.lib(parser) - "PyParser_AddToken" - "PyParser_New" - "PyParser_Delete" - -; From python24_s.lib(parsetok) - "Py_TabcheckFlag" - "PyParser_ParseString" - "PyParser_ParseStringFlags" - "PyParser_ParseFile" - "PyParser_ParseFileFlags" - "PyParser_ParseStringFlagsFilename" - -; From python24_s.lib(bitset) - "_Py_newbitset" - "_Py_delbitset" - "_Py_addbit" - "_Py_samebitset" - "_Py_mergebitset" - -; From python24_s.lib(metagrammar) - "_Py_meta_grammar" - "Py_meta_grammar" - -; From python24_s.lib(tokenizer) - "PyToken_OneChar" - "PyToken_TwoChars" - "PyToken_ThreeChars" - "PyTokenizer_FromString" - "PyTokenizer_FromFile" - "PyTokenizer_Free" - "PyTokenizer_Get" - "_PyParser_TokenNames" - -; From python24_s.lib(myreadline) - "_PyOS_ReadlineTState" - "PyOS_ReadlineFunctionPointer" - "PyOS_StdioReadline" - "PyOS_Readline" - "PyOS_InputHook" - -; From python24_s.lib(abstract) - "PyObject_GetItem" - "PySequence_GetItem" - "PyObject_SetItem" - "PySequence_SetItem" - "PyObject_DelItem" - "PySequence_DelItem" - "PyNumber_Multiply" - "PyNumber_InPlaceAdd" - "PyNumber_InPlaceMultiply" - "PyNumber_Int" - "PyNumber_Long" - "PyNumber_Float" - "PySequence_GetSlice" - "PySequence_SetSlice" - "PySequence_Tuple" - "PyObject_GetIter" - "PyMapping_Size" - "PyIter_Next" - "_PySequence_IterSearch" - "PyObject_CallFunction" - "PyObject_CallMethod" - "PyObject_CallMethodObjArgs" - "PyObject_CallFunctionObjArgs" - "PyObject_Cmp" - "PyObject_Call" - "PyObject_CallObject" - "PyObject_Type" - "PyObject_Size" - "PyObject_Length" - "PyObject_DelItemString" - "PyObject_AsCharBuffer" - "PyObject_CheckReadBuffer" - "PyObject_AsReadBuffer" - "PyObject_AsWriteBuffer" - "PyNumber_Check" - "PyNumber_Add" - "PyNumber_Subtract" - "PyNumber_Divide" - "PyNumber_FloorDivide" - "PyNumber_TrueDivide" - "PyNumber_Remainder" - "PyNumber_Divmod" - "PyNumber_Power" - "PyNumber_Negative" - "PyNumber_Positive" - "PyNumber_Absolute" - "PyNumber_Invert" - "PyNumber_Lshift" - "PyNumber_Rshift" - "PyNumber_And" - "PyNumber_Xor" - "PyNumber_Or" - "PyNumber_InPlaceSubtract" - "PyNumber_InPlaceDivide" - "PyNumber_InPlaceFloorDivide" - "PyNumber_InPlaceTrueDivide" - "PyNumber_InPlaceRemainder" - "PyNumber_InPlacePower" - "PyNumber_InPlaceLshift" - "PyNumber_InPlaceRshift" - "PyNumber_InPlaceAnd" - "PyNumber_InPlaceXor" - "PyNumber_InPlaceOr" - "PySequence_Check" - "PySequence_Size" - "PySequence_Length" - "PySequence_Concat" - "PySequence_Repeat" - "PySequence_DelSlice" - "PySequence_List" - "PySequence_Fast" - "PySequence_Count" - "PySequence_Contains" - "PySequence_In" - "PySequence_Index" - "PySequence_InPlaceConcat" - "PySequence_InPlaceRepeat" - "PyMapping_Check" - "PyMapping_Length" - "PyMapping_HasKeyString" - "PyMapping_HasKey" - "PyMapping_GetItemString" - "PyMapping_SetItemString" - "PyObject_IsInstance" - "PyObject_IsSubclass" - -; From python24_s.lib(boolobject) - "PyBool_FromLong" - "PyBool_Type" - "_Py_ZeroStruct" - "_Py_TrueStruct" - -; From python24_s.lib(bufferobject) - "PyBuffer_FromObject" - "PyBuffer_FromReadWriteObject" - "PyBuffer_FromMemory" - "PyBuffer_FromReadWriteMemory" - "PyBuffer_New" - "PyBuffer_Type" - -; From python24_s.lib(cellobject) - "PyCell_New" - "PyCell_Get" - "PyCell_Set" - "PyCell_Type" - -; From python24_s.lib(classobject) - "PyClass_New" - "PyClass_IsSubclass" - "PyInstance_New" - "PyInstance_NewRaw" - "PyMethod_New" - "PyMethod_Function" - "PyMethod_Self" - "PyMethod_Class" - "_PyInstance_Lookup" - "PyMethod_Fini" - "PyClass_Type" - "PyInstance_Type" - "PyMethod_Type" - -; From python24_s.lib(cobject) - "PyCObject_FromVoidPtr" - "PyCObject_FromVoidPtrAndDesc" - "PyCObject_AsVoidPtr" - "PyCObject_GetDesc" - "PyCObject_Import" - "PyCObject_SetVoidPtr" - "PyCObject_Type" - -; From python24_s.lib(complexobject) - "_Py_c_pow" - "_Py_c_sum" - "_Py_c_diff" - "_Py_c_neg" - "_Py_c_prod" - "_Py_c_quot" - "PyComplex_FromCComplex" - "PyComplex_FromDoubles" - "PyComplex_RealAsDouble" - "PyComplex_ImagAsDouble" - "PyComplex_AsCComplex" - "PyComplex_Type" - -; From python24_s.lib(descrobject) - "PyWrapper_New" - "PyDescr_NewMethod" - "PyDescr_NewClassMethod" - "PyDescr_NewMember" - "PyDescr_NewGetSet" - "PyDescr_NewWrapper" - "PyDictProxy_New" - "PyWrapperDescr_Type" - "PyProperty_Type" - -; From python24_s.lib(dictobject) - "PyDict_New" - "PyDict_SetItem" - "PyDict_DelItem" - "PyDict_Clear" - "PyDict_MergeFromSeq2" - "PyDict_Merge" - "PyDict_Keys" - "PyDict_Values" - "PyDict_Contains" - "PyDict_GetItem" - "PyDict_Next" - "PyDict_Items" - "PyDict_Size" - "PyDict_Copy" - "PyDict_Update" - "PyDict_GetItemString" - "PyDict_SetItemString" - "PyDict_DelItemString" - "PyDict_Type" - "PyDictIterKey_Type" - "PyDictIterValue_Type" - "PyDictIterItem_Type" - -; From python24_s.lib(enumobject) - "PyEnum_Type" - "PyReversed_Type" - -; From python24_s.lib(fileobject) - "Py_UniversalNewlineFread" - "PyFile_GetLine" - "PyFile_SoftSpace" - "PyFile_WriteObject" - "PyFile_WriteString" - "PyObject_AsFileDescriptor" - "Py_UniversalNewlineFgets" - "PyFile_FromString" - "PyFile_SetBufSize" - "PyFile_SetEncoding" - "PyFile_FromFile" - "PyFile_AsFile" - "PyFile_Name" - "PyFile_Type" - -; From python24_s.lib(floatobject) - "PyFloat_FromString" - "PyFloat_AsDouble" - "PyFloat_Fini" - "_PyFloat_Pack4" - "_PyFloat_Pack8" - "PyFloat_FromDouble" - "PyFloat_AsReprString" - "PyFloat_AsString" - "_PyFloat_Unpack4" - "_PyFloat_Unpack8" - "PyFloat_AsStringEx" - "PyFloat_Type" - -; From python24_s.lib(frameobject) - "PyFrame_New" - "PyFrame_FastToLocals" - "PyFrame_LocalsToFast" - "_PyFrame_Init" - "PyFrame_Fini" - "PyFrame_BlockSetup" - "PyFrame_BlockPop" - "PyFrame_Type" - -; From python24_s.lib(funcobject) - "PyFunction_New" - "PyFunction_GetCode" - "PyFunction_GetGlobals" - "PyFunction_GetModule" - "PyFunction_GetDefaults" - "PyFunction_SetDefaults" - "PyFunction_GetClosure" - "PyFunction_SetClosure" - "PyClassMethod_New" - "PyStaticMethod_New" - "PyFunction_Type" - "PyClassMethod_Type" - "PyStaticMethod_Type" - -; From python24_s.lib(genobject) - "PyGen_New" - "PyGen_Type" - -; From python24_s.lib(intobject) - "PyInt_AsLong" - "PyInt_AsUnsignedLongMask" - "PyInt_AsUnsignedLongLongMask" - "PyInt_FromString" - "PyInt_Fini" - "PyInt_FromUnicode" - "PyInt_FromLong" - "PyInt_GetMax" - "_PyInt_Init" - "PyInt_Type" - -; From python24_s.lib(iterobject) - "PySeqIter_New" - "PyCallIter_New" - "PySeqIter_Type" - "PyCallIter_Type" - -; From python24_s.lib(listobject) - "PyList_New" - "PyList_Append" - "PyList_Size" - "PyList_GetItem" - "PyList_SetItem" - "PyList_Insert" - "PyList_GetSlice" - "PyList_SetSlice" - "PyList_Sort" - "PyList_Reverse" - "PyList_AsTuple" - "_PyList_Extend" - "PyList_Type" - "PyListIter_Type" - "PyListRevIter_Type" - -; From python24_s.lib(longobject) - "PyLong_FromDouble" - "PyLong_AsLong" - "PyLong_AsUnsignedLong" - "_PyLong_FromByteArray" - "_PyLong_AsByteArray" - "PyLong_AsDouble" - "PyLong_FromString" - "PyLong_FromLong" - "PyLong_FromUnsignedLong" - "PyLong_AsUnsignedLongMask" - "_PyLong_AsScaledDouble" - "PyLong_FromVoidPtr" - "PyLong_AsVoidPtr" - "PyLong_FromLongLong" - "PyLong_FromUnsignedLongLong" - "PyLong_AsLongLong" - "PyLong_AsUnsignedLongLong" - "PyLong_AsUnsignedLongLongMask" - "PyLong_FromUnicode" - "_PyLong_Sign" - "_PyLong_NumBits" - "_PyLong_New" - "_PyLong_Copy" - "PyLong_Type" - -; From python24_s.lib(methodobject) - "PyCFunction_Call" - "Py_FindMethodInChain" - "PyCFunction_GetFunction" - "PyCFunction_GetSelf" - "PyCFunction_GetFlags" - "Py_FindMethod" - "PyCFunction_NewEx" - "PyCFunction_Fini" - "PyCFunction_New" - "PyCFunction_Type" - -; From python24_s.lib(moduleobject) - "PyModule_New" - "_PyModule_Clear" - "PyModule_GetDict" - "PyModule_GetName" - "PyModule_GetFilename" - "PyModule_Type" - -; From python24_s.lib(object) - "Py_DivisionWarningFlag" - "PyObject_Str" - "PyObject_Repr" - "PyObject_Unicode" - "PyObject_GetAttr" - "PyObject_IsTrue" - "PyNumber_CoerceEx" - "PyObject_Compare" - "PyObject_RichCompare" - "_Py_HashDouble" - "PyObject_Hash" - "PyObject_SetAttr" - "PyObject_GenericGetAttr" - "PyObject_GenericSetAttr" - "PyCallable_Check" - "PyObject_Dir" - "PyMem_Malloc" - "PyMem_Realloc" - "PyMem_Free" - "PyObject_Print" - "_PyObject_Dump" - "PyObject_RichCompareBool" - "PyObject_GetAttrString" - "PyObject_SetAttrString" - "PyObject_HasAttrString" - "PyObject_HasAttr" - "_PyObject_GetDictPtr" - "PyObject_SelfIter" - "PyObject_Not" - "PyNumber_Coerce" - "Py_ReprEnter" - "Py_ReprLeave" - "_Py_HashPointer" - "Py_IncRef" - "Py_DecRef" - "_PyTrash_deposit_object" - "_PyTrash_destroy_chain" - "PyObject_Init" - "PyObject_InitVar" - "_PyObject_New" - "_PyObject_NewVar" - "_PyObject_Del" - "_Py_ReadyTypes" - "_Py_SwappedOp" - "_Py_NotImplementedStruct" - "_Py_NoneStruct" - "_Py_cobject_hack" - "_Py_abstract_hack" - "_PyTrash_delete_nesting" - "_PyTrash_delete_later" - -; From python24_s.lib(obmalloc) - "PyObject_Malloc" - "PyObject_Realloc" - "PyObject_Free" - -; From python24_s.lib(rangeobject) - "PyRange_New" - "PyRange_Type" - -; From python24_s.lib(setobject) - "PySet_Type" - "PyFrozenSet_Type" - -; From python24_s.lib(sliceobject) - "PySlice_GetIndices" - "PySlice_GetIndicesEx" - "PySlice_New" - "_Py_EllipsisObject" - "PySlice_Type" - -; From python24_s.lib(stringobject) - "PyString_FromStringAndSize" - "PyString_InternInPlace" - "PyString_FromString" - "PyString_FromFormatV" - "PyString_AsString" - "_PyString_Resize" - "PyString_FromFormat" - "PyString_AsDecodedString" - "PyString_AsEncodedString" - "PyString_DecodeEscape" - "PyString_Size" - "PyString_Repr" - "PyString_AsStringAndSize" - "_PyString_FormatLong" - "PyString_Format" - "_Py_ReleaseInternedStrings" - "PyString_Concat" - "PyString_ConcatAndDel" - "_PyString_Eq" - "PyString_InternImmortal" - "PyString_InternFromString" - "_PyString_Join" - "PyString_Decode" - "PyString_Encode" - "PyString_AsEncodedObject" - "PyString_AsDecodedObject" - "PyString_Fini" - "PyString_Type" - "PyBaseString_Type" - -; From python24_s.lib(structseq) - "PyStructSequence_InitType" - "PyStructSequence_New" - "PyStructSequence_UnnamedField" - -; From python24_s.lib(tupleobject) - "PyTuple_New" - "PyTuple_Pack" - "_PyTuple_Resize" - "PyTuple_Size" - "PyTuple_GetItem" - "PyTuple_SetItem" - "PyTuple_GetSlice" - "PyTuple_Fini" - "PyTuple_Type" - "PyTupleIter_Type" - -; From python24_s.lib(typeobject) - "PyType_IsSubtype" - "_PyType_Lookup" - "PyType_Ready" - "PyType_GenericAlloc" - "_PyObject_SlotCompare" - "PyType_GenericNew" - "PyType_Type" - "PyBaseObject_Type" - "PySuper_Type" - -; From python24_s.lib(unicodeobject) - "PyUnicodeUCS2_Resize" - "PyUnicodeUCS2_FromOrdinal" - "PyUnicodeUCS2_FromObject" - "PyUnicodeUCS2_FromEncodedObject" - "PyUnicodeUCS2_Decode" - "PyUnicodeUCS2_GetDefaultEncoding" - "PyUnicodeUCS2_DecodeUTF8" - "PyUnicodeUCS2_DecodeLatin1" - "PyUnicodeUCS2_DecodeASCII" - "PyUnicodeUCS2_AsEncodedString" - "PyUnicodeUCS2_AsUTF8String" - "PyUnicodeUCS2_AsLatin1String" - "PyUnicodeUCS2_AsASCIIString" - "PyUnicode_DecodeUTF7" - "PyUnicode_EncodeUTF7" - "PyUnicodeUCS2_DecodeUTF8Stateful" - "PyUnicodeUCS2_EncodeUTF8" - "PyUnicodeUCS2_DecodeUTF16Stateful" - "PyUnicodeUCS2_AsUTF16String" - "PyUnicodeUCS2_DecodeUnicodeEscape" - "PyUnicodeUCS2_DecodeRawUnicodeEscape" - "PyUnicodeUCS2_EncodeRawUnicodeEscape" - "PyUnicodeUCS2_DecodeCharmap" - "PyUnicodeUCS2_EncodeCharmap" - "PyUnicodeUCS2_TranslateCharmap" - "PyUnicodeUCS2_EncodeDecimal" - "PyUnicodeUCS2_Count" - "PyUnicodeUCS2_Find" - "PyUnicodeUCS2_Tailmatch" - "PyUnicodeUCS2_Join" - "PyUnicodeUCS2_Splitlines" - "PyUnicodeUCS2_Compare" - "PyUnicodeUCS2_Contains" - "PyUnicodeUCS2_Concat" - "_PyUnicode_XStrip" - "PyUnicodeUCS2_Replace" - "PyUnicodeUCS2_Split" - "PyUnicodeUCS2_RSplit" - "PyUnicodeUCS2_Format" - "_PyUnicodeUCS2_Fini" - "PyUnicodeUCS2_FromUnicode" - "PyUnicodeUCS2_AsUnicode" - "PyUnicodeUCS2_GetSize" - "PyUnicodeUCS2_GetMax" - "_PyUnicodeUCS2_AsDefaultEncodedString" - "PyUnicodeUCS2_SetDefaultEncoding" - "PyUnicodeUCS2_Encode" - "PyUnicodeUCS2_AsEncodedObject" - "PyUnicodeUCS2_DecodeUTF16" - "PyUnicodeUCS2_EncodeUTF16" - "PyUnicodeUCS2_AsUnicodeEscapeString" - "PyUnicodeUCS2_EncodeUnicodeEscape" - "PyUnicodeUCS2_AsRawUnicodeEscapeString" - "PyUnicodeUCS2_EncodeLatin1" - "PyUnicodeUCS2_EncodeASCII" - "PyUnicodeUCS2_AsCharmapString" - "PyUnicodeUCS2_Translate" - "PyUnicode_AsDecodedObject" - "_PyUnicodeUCS2_Init" - "PyUnicode_Type" - -; From python24_s.lib(unicodectype) - "_PyUnicode_TypeRecords" - "_PyUnicodeUCS2_ToNumeric" - "_PyUnicodeUCS2_IsLowercase" - "_PyUnicodeUCS2_IsUppercase" - "_PyUnicodeUCS2_IsTitlecase" - "_PyUnicodeUCS2_IsWhitespace" - "_PyUnicodeUCS2_IsLinebreak" - "_PyUnicodeUCS2_ToLowercase" - "_PyUnicodeUCS2_ToUppercase" - "_PyUnicodeUCS2_ToTitlecase" - "_PyUnicodeUCS2_ToDecimalDigit" - "_PyUnicodeUCS2_ToDigit" - "_PyUnicodeUCS2_IsDecimalDigit" - "_PyUnicodeUCS2_IsDigit" - "_PyUnicodeUCS2_IsNumeric" - "_PyUnicodeUCS2_IsAlpha" - -; From python24_s.lib(weakrefobject) - "PyWeakref_NewRef" - "PyWeakref_NewProxy" - "PyObject_ClearWeakRefs" - "PyWeakref_GetObject" - "_PyWeakref_GetWeakrefCount" - "_PyWeakref_ClearRef" - "_PyWeakref_RefType" - "_PyWeakref_ProxyType" - "_PyWeakref_CallableProxyType" - -; From python24_s.lib(bltinmodule) - "_PyBuiltin_Init" - "Py_FileSystemDefaultEncoding" - -; From python24_s.lib(exceptions) - "PyExc_Exception" - "PyExc_TypeError" - "PyExc_StopIteration" - "PyExc_StandardError" - "PyExc_SystemExit" - "PyExc_KeyboardInterrupt" - "PyExc_ImportError" - "PyExc_EnvironmentError" - "PyExc_IOError" - "PyExc_OSError" - "PyExc_EOFError" - "PyExc_RuntimeError" - "PyExc_NotImplementedError" - "PyExc_NameError" - "PyExc_UnboundLocalError" - "PyExc_AttributeError" - "PyExc_SyntaxError" - "PyExc_IndentationError" - "PyExc_TabError" - "PyExc_AssertionError" - "PyExc_LookupError" - "PyExc_IndexError" - "PyExc_KeyError" - "PyExc_ArithmeticError" - "PyExc_OverflowError" - "PyExc_ZeroDivisionError" - "PyExc_FloatingPointError" - "PyExc_ValueError" - "PyExc_UnicodeError" - "PyExc_UnicodeEncodeError" - "PyExc_UnicodeDecodeError" - "PyExc_UnicodeTranslateError" - "PyExc_ReferenceError" - "PyExc_SystemError" - "PyExc_MemoryError" - "PyExc_Warning" - "PyExc_UserWarning" - "PyExc_DeprecationWarning" - "PyExc_PendingDeprecationWarning" - "PyExc_SyntaxWarning" - "PyExc_OverflowWarning" - "PyExc_RuntimeWarning" - "PyExc_FutureWarning" - "PyExc_MemoryErrorInst" - "PyUnicodeEncodeError_GetStart" - "PyUnicodeDecodeError_GetStart" - "PyUnicodeEncodeError_GetEnd" - "PyUnicodeDecodeError_GetEnd" - "_PyExc_Init" - "_PyExc_Fini" - "PyUnicodeDecodeError_Create" - "PyUnicodeEncodeError_Create" - "PyUnicodeTranslateError_Create" - "PyUnicodeEncodeError_GetEncoding" - "PyUnicodeDecodeError_GetEncoding" - "PyUnicodeEncodeError_GetObject" - "PyUnicodeDecodeError_GetObject" - "PyUnicodeTranslateError_GetObject" - "PyUnicodeTranslateError_GetStart" - "PyUnicodeEncodeError_SetStart" - "PyUnicodeDecodeError_SetStart" - "PyUnicodeTranslateError_SetStart" - "PyUnicodeTranslateError_GetEnd" - "PyUnicodeEncodeError_SetEnd" - "PyUnicodeDecodeError_SetEnd" - "PyUnicodeTranslateError_SetEnd" - "PyUnicodeEncodeError_GetReason" - "PyUnicodeDecodeError_GetReason" - "PyUnicodeTranslateError_GetReason" - "PyUnicodeEncodeError_SetReason" - "PyUnicodeDecodeError_SetReason" - "PyUnicodeTranslateError_SetReason" - -; From python24_s.lib(ceval) - "PyEval_EvalFrame" - "PyEval_CallObjectWithKeywords" - "PyEval_EvalCodeEx" - "_PyEval_SliceIndex" - "PyEval_GetFrame" - "PyEval_CallObject" - "PyEval_SetProfile" - "PyEval_SetTrace" - "PyEval_GetBuiltins" - "PyEval_GetGlobals" - "PyEval_GetLocals" - "PyEval_GetRestricted" - "PyEval_MergeCompilerFlags" - "Py_FlushLine" - "Py_AddPendingCall" - "Py_MakePendingCalls" - "Py_SetRecursionLimit" - "Py_GetRecursionLimit" - "_Py_CheckRecursiveCall" - "PyEval_GetFuncName" - "PyEval_GetFuncDesc" - "PyEval_GetCallStats" - "PyEval_SaveThread" - "PyEval_RestoreThread" - "PyEval_InitThreads" - "PyEval_AcquireLock" - "PyEval_ReleaseLock" - "PyEval_AcquireThread" - "PyEval_ReleaseThread" - "PyEval_ReInitThreads" - "PyEval_EvalCode" - "_PyEval_CallTracing" - "_Py_CheckRecursionLimit" - "_Py_CheckInterval" - "_Py_Ticker" - -; From python24_s.lib(compile) - "PyCode_New" - "PySymtable_Free" - "PyNode_Compile" - "PyNode_CompileFlags" - "PyCode_Addr2Line" - "_Py_Mangle" - "PyNode_CompileSymtable" - "Py_OptimizeFlag" - "PyCode_Type" - -; From python24_s.lib(codecs) - "_PyCodec_Lookup" - "PyCodec_StreamReader" - "PyCodec_StreamWriter" - "PyCodec_Encode" - "PyCodec_Decode" - "PyCodec_IgnoreErrors" - "PyCodec_ReplaceErrors" - "PyCodec_XMLCharRefReplaceErrors" - "PyCodec_BackslashReplaceErrors" - "PyCodec_Register" - "PyCodec_Encoder" - "PyCodec_Decoder" - "PyCodec_RegisterError" - "PyCodec_LookupError" - "PyCodec_StrictErrors" - -; From python24_s.lib(errors) - "PyErr_SetNone" - "PyErr_SetString" - "PyErr_ExceptionMatches" - "PyErr_GivenExceptionMatches" - "PyErr_NormalizeException" - "PyErr_Fetch" - "PyErr_Clear" - "PyErr_NoMemory" - "PyErr_SetFromErrnoWithFilenameObject" - "PyErr_Format" - "PyErr_NewException" - "PyErr_WriteUnraisable" - "PyErr_SyntaxLocation" - "PyErr_ProgramText" - "PyErr_SetObject" - "PyErr_Occurred" - "PyErr_Restore" - "PyErr_BadArgument" - "PyErr_SetFromErrno" - "PyErr_SetFromErrnoWithFilename" - "PyErr_BadInternalCall" - "_PyErr_BadInternalCall" - "PyErr_Warn" - "PyErr_WarnExplicit" - -; From python24_s.lib(frozen) - "PyImport_FrozenModules" - -; From python24_s.lib(frozenmain) - "Py_FrozenMain" - -; From python24_s.lib(future) - "PyNode_Future" - -; From python24_s.lib(getargs) - "PyArg_Parse" - "PyArg_ParseTuple" - "PyArg_ParseTupleAndKeywords" - "PyArg_UnpackTuple" - "PyArg_VaParse" - "PyArg_VaParseTupleAndKeywords" - -; From python24_s.lib(getcompiler) - "Py_GetCompiler" - -; From python24_s.lib(getcopyright) - "Py_GetCopyright" - -; From python24_s.lib(getmtime) - "PyOS_GetLastModificationTime" - -; From python24_s.lib(getplatform) - "Py_GetPlatform" - -; From python24_s.lib(getversion) - "Py_GetVersion" - -; From python24_s.lib(graminit) - "_PyParser_Grammar" - -; From python24_s.lib(import) - "_PyImport_Init" - "_PyImportHooks_Init" - "PyImport_ImportModule" - "PyImport_Cleanup" - "_PyImport_FixupExtension" - "PyImport_AddModule" - "PyImport_ExecCodeModuleEx" - "PyImport_ImportFrozenModule" - "PyImport_ImportModuleEx" - "PyImport_ReloadModule" - "PyImport_Import" -; "initimp" - "_PyImport_Fini" - "PyImport_GetMagicNumber" - "PyImport_ExecCodeModule" - "PyImport_GetModuleDict" - "_PyImport_FindExtension" - "PyImport_AppendInittab" - "PyImport_ExtendInittab" - "PyImport_Inittab" - "_PyImport_Filetab" - -; From python24_s.lib(importdl) - "_PyImport_LoadDynamicModule" - -; From python24_s.lib(marshal) - "PyMarshal_ReadLongFromFile" - "PyMarshal_WriteObjectToString" - "PyMarshal_WriteLongToFile" - "PyMarshal_WriteObjectToFile" - "PyMarshal_ReadShortFromFile" - "PyMarshal_ReadObjectFromFile" - "PyMarshal_ReadLastObjectFromFile" - "PyMarshal_ReadObjectFromString" - "PyMarshal_Init" - -; From python24_s.lib(modsupport) - "Py_InitModule4" - "Py_BuildValue" - "Py_VaBuildValue" - "PyEval_CallFunction" - "PyEval_CallMethod" - "PyModule_AddObject" - "PyModule_AddIntConstant" - "PyModule_AddStringConstant" - "_Py_PackageContext" - -; From python24_s.lib(mysnprintf) - "PyOS_snprintf" - "PyOS_vsnprintf" - -; From python24_s.lib(mystrtoul) - "PyOS_strtoul" - "PyOS_strtol" - -; From python24_s.lib(pyfpe) - "PyFPE_dummy" - -; From python24_s.lib(pystate) - "PyInterpreterState_Clear" - "PyThreadState_Clear" - "PyGILState_Ensure" - "PyGILState_Release" - "PyInterpreterState_New" - "PyInterpreterState_Delete" - "PyThreadState_Delete" - "PyThreadState_New" - "PyThreadState_DeleteCurrent" - "PyThreadState_Get" - "PyThreadState_Swap" - "PyThreadState_GetDict" - "PyThreadState_SetAsyncExc" - "PyGILState_GetThisThreadState" - "PyInterpreterState_Head" - "PyInterpreterState_Next" - "PyInterpreterState_ThreadHead" - "PyThreadState_Next" - "_PyGILState_Init" - "_PyGILState_Fini" - "_PyThreadState_Current" - "_PyThreadState_GetFrame" - -; From python24_s.lib(pystrtod) - "PyOS_ascii_strtod" - "PyOS_ascii_formatd" - "PyOS_ascii_atof" - -; From python24_s.lib(pythonrun) - "Py_IgnoreEnvironmentFlag" - "Py_DebugFlag" - "Py_VerboseFlag" - "Py_NoSiteFlag" - "Py_InteractiveFlag" - "Py_FrozenFlag" - "Py_InitializeEx" - "Py_FatalError" - "Py_NewInterpreter" - "PyErr_Print" - "PyRun_InteractiveOneFlags" - "PyRun_SimpleFileExFlags" - "PyRun_FileExFlags" - "Py_Exit" - "PyErr_PrintEx" - "PyErr_Display" - "Py_SetProgramName" - "Py_GetProgramName" - "Py_SetPythonHome" - "Py_GetPythonHome" - "Py_Initialize" - "Py_Finalize" - "Py_IsInitialized" - "Py_EndInterpreter" - "PyRun_AnyFile" - "PyRun_AnyFileExFlags" - "PyRun_AnyFileEx" - "PyRun_AnyFileFlags" - "Py_FdIsInteractive" - "PyRun_InteractiveLoopFlags" - "PyRun_SimpleString" - "PyRun_SimpleStringFlags" - "PyRun_StringFlags" - "PyRun_SimpleFile" - "PyRun_SimpleFileEx" - "PyRun_InteractiveOne" - "PyRun_InteractiveLoop" - "PyParser_SimpleParseString" - "PyParser_SimpleParseFile" - "PyParser_SimpleParseStringFlags" - "PyParser_SimpleParseStringFlagsFilename" - "PyParser_SimpleParseFileFlags" - "PyRun_String" - "PyRun_File" - "PyRun_FileEx" - "PyRun_FileFlags" - "Py_CompileString" - "Py_CompileStringFlags" - "Py_SymtableString" - "Py_AtExit" - "PyOS_getsig" - "PyOS_setsig" - "PyParser_SetError" - "PyModule_GetWarningsModule" - "PyParser_SimpleParseStringFilename" - "Py_UseClassExceptionsFlag" - "Py_UnicodeFlag" - "_Py_QnewFlag" - -; From python24_s.lib(structmember) - "PyMember_Get" - "PyMember_GetOne" - "PyMember_SetOne" - "PyMember_Set" - -; From python24_s.lib(symtable) - "PySymtableEntry_New" - "PySymtableEntry_Type" - -; From python24_s.lib(sysmodule) - "_PySys_Init" - "PySys_SetPath" - "PySys_SetArgv" - "PySys_WriteStdout" - "PySys_WriteStderr" - "PySys_GetObject" - "PySys_SetObject" - "PySys_GetFile" - "PySys_ResetWarnOptions" - "PySys_AddWarnOption" - -; From python24_s.lib(traceback) - "PyTraceBack_Here" - "PyTraceBack_Print" - "PyTraceBack_Type" - -; From python24_s.lib(getopt) - "_PyOS_GetOpt" - "_PyOS_opterr" - "_PyOS_optind" - "_PyOS_optarg" - -; From python24_s.lib(dynload_shlib) - "_PyImport_DynLoadFiletab" - "_PyImport_GetDynLoadFunc" - -; From python24_s.lib(thread) - "PyThread_delete_key_value" - "PyThread_init_thread" - "PyThread_start_new_thread" - "PyThread_exit_thread" - "PyThread_get_thread_ident" - "PyThread_allocate_lock" - "PyThread_free_lock" - "PyThread_acquire_lock" - "PyThread_release_lock" - "PyThread_create_key" - "PyThread_delete_key" - "PyThread_set_key_value" - "PyThread_get_key_value" - "PyThread__exit_thread" - -; From python24_s.lib(gcmodule) -; "initgc" - "_PyObject_GC_New" - "_PyObject_GC_NewVar" - "PyGC_Collect" - "_PyObject_GC_Resize" - "_PyObject_GC_Malloc" - "PyObject_GC_Track" - "PyObject_GC_UnTrack" - "PyObject_GC_Del" - "_PyGC_Dump" - "_PyObject_GC_Track" - "_PyObject_GC_UnTrack" - "_PyObject_GC_Del" - "_PyGC_generation0" - -; From python24_s.lib(signalmodule) -; "initsignal" - "PyErr_CheckSignals" - "PyErr_SetInterrupt" - "PyOS_FiniInterrupts" - "PyOS_InterruptOccurred" - "PyOS_InitInterrupts" - "PyOS_AfterFork" - -; From python24_s.lib(posixmodule) -; "initos2" - -; From python24_s.lib(threadmodule) -; "initthread" - -; From python24_s.lib(arraymodule) -; "initarray" -; "array_methods" - -; From python24_s.lib(binascii) -; "initbinascii" - -; From python24_s.lib(cmathmodule) -; "initcmath" - -; From python24_s.lib(_codecsmodule) -; "init_codecs" - -; From python24_s.lib(collectionsmodule) -; "initcollections" - "dequeiter_type" - "dequereviter_type" - -; From python24_s.lib(cPickle) -; "initcPickle" -; "fast_save_leave" - -; From python24_s.lib(cStringIO) -; "initcStringIO" - -; From python24_s.lib(_csv) -; "init_csv" - -; From python24_s.lib(datetimemodule) -; "initdatetime" - -; From python24_s.lib(dlmodule) -; "initdl" - -; From python24_s.lib(errnomodule) -; "initerrno" - -; From python24_s.lib(fcntlmodule) -; "initfcntl" - -; From python24_s.lib(_heapqmodule) -; "init_heapq" - -; From python24_s.lib(imageop) -; "initimageop" - -; From python24_s.lib(itertoolsmodule) -; "inititertools" - -; From python24_s.lib(_localemodule) -; "init_locale" - -; From python24_s.lib(mathmodule) -; "initmath" - -; From python24_s.lib(md5c) -; "_Py_MD5Final" -; "_Py_MD5Init" -; "_Py_MD5Update" - -; From python24_s.lib(md5module) -; "initmd5" - -; From python24_s.lib(operator) -; "initoperator" - -; From python24_s.lib(_randommodule) -; "init_random" - -; From python24_s.lib(rgbimgmodule) -; "initrgbimg" - -; From python24_s.lib(shamodule) -; "initsha" - -; From python24_s.lib(_sre) -; "init_sre" - -; From python24_s.lib(stropmodule) -; "initstrop" - -; From python24_s.lib(structmodule) -; "initstruct" - -; From python24_s.lib(symtablemodule) -; "init_symtable" - -; From python24_s.lib(termios) -; "inittermios" - -; From python24_s.lib(timemodule) -; "inittime" - "_PyTime_DoubleToTimet" -; "inittimezone" - -; From python24_s.lib(timingmodule) -; "inittiming" - -; From python24_s.lib(_weakref) -; "init_weakref" - -; From python24_s.lib(xxsubtype) -; "initxxsubtype" - -; From python24_s.lib(zipimport) -; "initzipimport" diff --git a/PC/os2emx/python25.def b/PC/os2emx/python25.def new file mode 100644 index 0000000..3c6004f --- /dev/null +++ b/PC/os2emx/python25.def @@ -0,0 +1,1314 @@ +LIBRARY python25 INITINSTANCE TERMINSTANCE +DESCRIPTION "Python 2.5 Core DLL" +PROTMODE +DATA MULTIPLE NONSHARED +EXPORTS + +; From python25_s.lib(config) + "_PyImport_Inittab" + +; From python25_s.lib(dlfcn) +; "dlopen" +; "dlsym" +; "dlclose" +; "dlerror" + +; From python25_s.lib(getpathp) + "Py_GetProgramFullPath" + "Py_GetPrefix" + "Py_GetExecPrefix" + "Py_GetPath" + +; From python25_s.lib(getbuildinfo) + "Py_GetBuildInfo" + "_Py_svnversion" + +; From python25_s.lib(main) + "Py_Main" + "Py_GetArgcArgv" + +; From python25_s.lib(acceler) + "PyGrammar_AddAccelerators" + "PyGrammar_RemoveAccelerators" + +; From python25_s.lib(grammar1) + "PyGrammar_FindDFA" + "PyGrammar_LabelRepr" + +; From python25_s.lib(listnode) + "PyNode_ListTree" + +; From python25_s.lib(node) + "PyNode_New" + "PyNode_AddChild" + "PyNode_Free" + +; From python25_s.lib(parser) + "PyParser_AddToken" + "PyParser_New" + "PyParser_Delete" + +; From python25_s.lib(parsetok) + "Py_TabcheckFlag" + "PyParser_ParseString" + "PyParser_ParseStringFlagsFilename" + "PyParser_ParseFile" + "PyParser_ParseFileFlags" + "PyParser_ParseStringFlags" + +; From python25_s.lib(bitset) + "_Py_newbitset" + "_Py_delbitset" + "_Py_addbit" + "_Py_samebitset" + "_Py_mergebitset" + +; From python25_s.lib(metagrammar) + "_Py_meta_grammar" + "Py_meta_grammar" + +; From python25_s.lib(tokenizer) + "PyToken_OneChar" + "PyToken_TwoChars" + "PyToken_ThreeChars" + "PyTokenizer_FromString" + "PyTokenizer_Free" + "PyTokenizer_FromFile" + "PyTokenizer_Get" + "_PyParser_TokenNames" + +; From python25_s.lib(myreadline) + "_PyOS_ReadlineTState" + "PyOS_ReadlineFunctionPointer" + "PyOS_StdioReadline" + "PyOS_Readline" + "PyOS_InputHook" + +; From python25_s.lib(abstract) + "_PyObject_LengthHint" + "PyMapping_Size" + "PyObject_CallMethod" + "PyObject_GetItem" + "PySequence_GetItem" + "PyObject_SetItem" + "PySequence_SetItem" + "PyObject_DelItem" + "PySequence_DelItem" + "PyNumber_Multiply" + "PyNumber_InPlaceAdd" + "PyNumber_InPlaceMultiply" + "PyNumber_Int" + "PyNumber_Long" + "PyNumber_Float" + "PySequence_Concat" + "PySequence_Repeat" + "PySequence_InPlaceConcat" + "PySequence_InPlaceRepeat" + "PySequence_GetSlice" + "PySequence_SetSlice" + "PySequence_Tuple" + "PyObject_GetIter" + "PyIter_Next" + "PySequence_Fast" + "_PySequence_IterSearch" + "PyObject_CallFunction" + "_PyObject_CallFunction_SizeT" + "_PyObject_CallMethod_SizeT" + "PyObject_CallMethodObjArgs" + "PyObject_CallFunctionObjArgs" + "PyObject_Cmp" + "PyObject_Call" + "PyObject_CallObject" + "PyObject_Type" + "PyObject_Size" + "PyObject_Length" + "PyObject_DelItemString" + "PyObject_AsCharBuffer" + "PyObject_CheckReadBuffer" + "PyObject_AsReadBuffer" + "PyObject_AsWriteBuffer" + "PyNumber_Check" + "PyNumber_Add" + "PyNumber_Subtract" + "PyNumber_Divide" + "PyNumber_FloorDivide" + "PyNumber_TrueDivide" + "PyNumber_Remainder" + "PyNumber_Divmod" + "PyNumber_Power" + "PyNumber_Negative" + "PyNumber_Positive" + "PyNumber_Absolute" + "PyNumber_Invert" + "PyNumber_Lshift" + "PyNumber_Rshift" + "PyNumber_And" + "PyNumber_Xor" + "PyNumber_Or" + "PyNumber_Index" + "PyNumber_InPlaceSubtract" + "PyNumber_InPlaceDivide" + "PyNumber_InPlaceFloorDivide" + "PyNumber_InPlaceTrueDivide" + "PyNumber_InPlaceRemainder" + "PyNumber_InPlacePower" + "PyNumber_InPlaceLshift" + "PyNumber_InPlaceRshift" + "PyNumber_InPlaceAnd" + "PyNumber_InPlaceXor" + "PyNumber_InPlaceOr" + "PySequence_Check" + "PySequence_Size" + "PySequence_Length" + "PySequence_DelSlice" + "PySequence_List" + "PySequence_Count" + "PySequence_Contains" + "PySequence_In" + "PySequence_Index" + "PyMapping_Check" + "PyMapping_Length" + "PyMapping_HasKeyString" + "PyMapping_HasKey" + "PyMapping_GetItemString" + "PyMapping_SetItemString" + "PyObject_IsInstance" + "PyObject_IsSubclass" + +; From python25_s.lib(boolobject) + "PyBool_FromLong" + "PyBool_Type" + "_Py_ZeroStruct" + "_Py_TrueStruct" + +; From python25_s.lib(bufferobject) + "PyBuffer_FromObject" + "PyBuffer_FromReadWriteObject" + "PyBuffer_FromMemory" + "PyBuffer_FromReadWriteMemory" + "PyBuffer_New" + "PyBuffer_Type" + +; From python25_s.lib(cellobject) + "PyCell_New" + "PyCell_Get" + "PyCell_Set" + "PyCell_Type" + +; From python25_s.lib(classobject) + "PyClass_New" + "PyClass_IsSubclass" + "PyInstance_New" + "PyInstance_NewRaw" + "PyMethod_New" + "PyMethod_Function" + "PyMethod_Self" + "PyMethod_Class" + "_PyInstance_Lookup" + "PyMethod_Fini" + "PyClass_Type" + "PyInstance_Type" + "PyMethod_Type" + +; From python25_s.lib(cobject) + "PyCObject_FromVoidPtr" + "PyCObject_FromVoidPtrAndDesc" + "PyCObject_AsVoidPtr" + "PyCObject_GetDesc" + "PyCObject_Import" + "PyCObject_SetVoidPtr" + "PyCObject_Type" + +; From python25_s.lib(codeobject) + "PyCode_New" + "PyCode_Addr2Line" + "PyCode_CheckLineNumber" + "PyCode_Type" + +; From python25_s.lib(complexobject) + "_Py_c_pow" + "_Py_c_sum" + "_Py_c_diff" + "_Py_c_neg" + "_Py_c_prod" + "_Py_c_quot" + "PyComplex_FromCComplex" + "PyComplex_FromDoubles" + "PyComplex_RealAsDouble" + "PyComplex_ImagAsDouble" + "PyComplex_AsCComplex" + "PyComplex_Type" + +; From python25_s.lib(descrobject) + "PyWrapper_New" + "PyDescr_NewMethod" + "PyDescr_NewClassMethod" + "PyDescr_NewMember" + "PyDescr_NewGetSet" + "PyDescr_NewWrapper" + "PyDictProxy_New" + "PyWrapperDescr_Type" + "PyProperty_Type" + +; From python25_s.lib(dictobject) + "PyDict_New" + "PyDict_GetItem" + "PyDict_SetItem" + "PyDict_DelItem" + "PyDict_Clear" + "PyDict_MergeFromSeq2" + "PyDict_Merge" + "PyDict_Keys" + "PyDict_Values" + "PyDict_Contains" + "PyDict_Next" + "PyDict_Items" + "PyDict_Size" + "PyDict_Copy" + "PyDict_Update" + "PyDict_GetItemString" + "PyDict_SetItemString" + "PyDict_DelItemString" + "PyDict_Type" + "PyDictIterKey_Type" + "PyDictIterValue_Type" + "PyDictIterItem_Type" + +; From python25_s.lib(enumobject) + "PyEnum_Type" + "PyReversed_Type" + +; From python25_s.lib(fileobject) + "PyFile_FromString" + "Py_UniversalNewlineFread" + "PyFile_GetLine" + "PyFile_SoftSpace" + "PyFile_WriteObject" + "PyFile_WriteString" + "PyObject_AsFileDescriptor" + "Py_UniversalNewlineFgets" + "PyFile_SetBufSize" + "PyFile_SetEncoding" + "PyFile_FromFile" + "PyFile_AsFile" + "PyFile_Name" + "PyFile_Type" + +; From python25_s.lib(floatobject) + "PyFloat_FromString" + "PyFloat_AsDouble" + "PyFloat_Fini" + "_PyFloat_Pack4" + "_PyFloat_Pack8" + "_PyFloat_Unpack4" + "_PyFloat_Unpack8" + "PyFloat_FromDouble" + "PyFloat_AsReprString" + "PyFloat_AsString" + "_PyFloat_Init" + "PyFloat_AsStringEx" + "PyFloat_Type" + +; From python25_s.lib(frameobject) + "PyFrame_New" + "PyFrame_FastToLocals" + "PyFrame_LocalsToFast" + "_PyFrame_Init" + "PyFrame_Fini" + "PyFrame_BlockSetup" + "PyFrame_BlockPop" + "PyFrame_Type" + +; From python25_s.lib(funcobject) + "PyFunction_New" + "PyFunction_GetCode" + "PyFunction_GetGlobals" + "PyFunction_GetModule" + "PyFunction_GetDefaults" + "PyFunction_SetDefaults" + "PyFunction_GetClosure" + "PyFunction_SetClosure" + "PyClassMethod_New" + "PyStaticMethod_New" + "PyFunction_Type" + "PyClassMethod_Type" + "PyStaticMethod_Type" + +; From python25_s.lib(genobject) + "PyGen_New" + "PyGen_NeedsFinalizing" + "PyGen_Type" + +; From python25_s.lib(intobject) + "PyInt_AsLong" + "PyInt_AsUnsignedLongMask" + "PyInt_AsUnsignedLongLongMask" + "PyInt_FromString" + "PyInt_AsSsize_t" + "PyInt_Fini" + "PyInt_FromUnicode" + "PyInt_FromLong" + "PyInt_FromSize_t" + "PyInt_FromSsize_t" + "PyInt_GetMax" + "_PyInt_Init" + "PyInt_Type" + +; From python25_s.lib(iterobject) + "PySeqIter_New" + "PyCallIter_New" + "PySeqIter_Type" + "PyCallIter_Type" + +; From python25_s.lib(listobject) + "PyList_New" + "PyList_Append" + "PyList_Size" + "PyList_GetItem" + "PyList_SetItem" + "PyList_Insert" + "PyList_GetSlice" + "PyList_SetSlice" + "PyList_Sort" + "PyList_Reverse" + "PyList_AsTuple" + "_PyList_Extend" + "PyList_Fini" + "PyList_Type" + "PyListIter_Type" + "PyListRevIter_Type" + +; From python25_s.lib(longobject) + "PyLong_FromDouble" + "PyLong_AsLong" + "_PyLong_AsSsize_t" + "PyLong_AsUnsignedLong" + "_PyLong_FromByteArray" + "_PyLong_AsByteArray" + "PyLong_AsDouble" + "PyLong_FromLongLong" + "PyLong_AsLongLong" + "PyLong_FromString" + "PyLong_FromLong" + "PyLong_FromUnsignedLong" + "PyLong_AsUnsignedLongMask" + "_PyLong_FromSize_t" + "_PyLong_FromSsize_t" + "_PyLong_AsScaledDouble" + "PyLong_FromVoidPtr" + "PyLong_AsVoidPtr" + "PyLong_FromUnsignedLongLong" + "PyLong_AsUnsignedLongLong" + "PyLong_AsUnsignedLongLongMask" + "PyLong_FromUnicode" + "_PyLong_Sign" + "_PyLong_NumBits" + "_PyLong_New" + "_PyLong_Copy" + "PyLong_Type" + "_PyLong_DigitValue" + +; From python25_s.lib(methodobject) + "PyCFunction_Call" + "Py_FindMethodInChain" + "PyCFunction_GetFunction" + "PyCFunction_GetSelf" + "PyCFunction_GetFlags" + "Py_FindMethod" + "PyCFunction_NewEx" + "PyCFunction_Fini" + "PyCFunction_New" + "PyCFunction_Type" + +; From python25_s.lib(moduleobject) + "PyModule_New" + "_PyModule_Clear" + "PyModule_GetDict" + "PyModule_GetName" + "PyModule_GetFilename" + "PyModule_Type" + +; From python25_s.lib(object) + "Py_DivisionWarningFlag" + "PyObject_Str" + "PyObject_Repr" + "_PyObject_Str" + "PyObject_Unicode" + "PyObject_GetAttr" + "PyObject_IsTrue" + "PyNumber_CoerceEx" + "PyObject_Compare" + "PyObject_RichCompare" + "_Py_HashDouble" + "PyObject_Hash" + "PyObject_SetAttr" + "PyObject_GenericGetAttr" + "PyObject_GenericSetAttr" + "PyCallable_Check" + "PyObject_Dir" + "PyMem_Malloc" + "PyMem_Realloc" + "PyMem_Free" + "PyObject_Print" + "_PyObject_Dump" + "PyObject_RichCompareBool" + "PyObject_GetAttrString" + "PyObject_SetAttrString" + "PyObject_HasAttrString" + "PyObject_HasAttr" + "_PyObject_GetDictPtr" + "PyObject_SelfIter" + "PyObject_Not" + "PyNumber_Coerce" + "Py_ReprEnter" + "Py_ReprLeave" + "_Py_HashPointer" + "Py_IncRef" + "Py_DecRef" + "_PyTrash_deposit_object" + "_PyTrash_destroy_chain" + "PyObject_Init" + "PyObject_InitVar" + "_PyObject_New" + "_PyObject_NewVar" + "_PyObject_Del" + "_Py_ReadyTypes" + "_Py_SwappedOp" + "_Py_NotImplementedStruct" + "_Py_NoneStruct" + "_Py_cobject_hack" + "_Py_abstract_hack" + "_PyTrash_delete_nesting" + "_PyTrash_delete_later" + +; From python25_s.lib(obmalloc) + "PyObject_Malloc" + "PyObject_Free" + "PyObject_Realloc" + +; From python25_s.lib(rangeobject) + "PyRange_Type" + +; From python25_s.lib(setobject) + "PySet_Pop" + "PySet_New" + "PyFrozenSet_New" + "PySet_Size" + "PySet_Clear" + "PySet_Contains" + "PySet_Discard" + "PySet_Add" + "_PySet_Next" + "_PySet_Update" + "PySet_Fini" + "PySet_Type" + "PyFrozenSet_Type" + +; From python25_s.lib(sliceobject) + "_PySlice_FromIndices" + "PySlice_GetIndices" + "PySlice_GetIndicesEx" + "PySlice_New" + "_Py_EllipsisObject" + "PySlice_Type" + +; From python25_s.lib(stringobject) + "PyString_FromStringAndSize" + "PyString_InternInPlace" + "PyString_FromString" + "PyString_FromFormatV" + "PyString_AsString" + "_PyString_Resize" + "PyString_FromFormat" + "PyString_AsDecodedString" + "PyString_AsEncodedString" + "PyString_DecodeEscape" + "PyString_Repr" + "PyString_AsStringAndSize" + "_PyString_FormatLong" + "PyString_Format" + "_Py_ReleaseInternedStrings" + "PyString_Size" + "PyString_Concat" + "PyString_ConcatAndDel" + "_PyString_Eq" + "PyString_InternImmortal" + "PyString_InternFromString" + "_PyString_Join" + "PyString_Decode" + "PyString_Encode" + "PyString_AsEncodedObject" + "PyString_AsDecodedObject" + "PyString_Fini" + "PyString_Type" + "PyBaseString_Type" + +; From python25_s.lib(structseq) + "PyStructSequence_InitType" + "PyStructSequence_New" + "PyStructSequence_UnnamedField" + +; From python25_s.lib(tupleobject) + "PyTuple_New" + "PyTuple_Pack" + "_PyTuple_Resize" + "PyTuple_Size" + "PyTuple_GetItem" + "PyTuple_SetItem" + "PyTuple_GetSlice" + "PyTuple_Fini" + "PyTuple_Type" + "PyTupleIter_Type" + +; From python25_s.lib(typeobject) + "PyType_IsSubtype" + "_PyType_Lookup" + "PyType_Ready" + "PyType_GenericAlloc" + "_PyObject_SlotCompare" + "PyType_GenericNew" + "PyType_Type" + "PyBaseObject_Type" + "PySuper_Type" + +; From python25_s.lib(unicodeobject) + "PyUnicodeUCS2_Resize" + "PyUnicodeUCS2_FromOrdinal" + "PyUnicodeUCS2_FromObject" + "PyUnicodeUCS2_FromEncodedObject" + "PyUnicodeUCS2_Decode" + "PyUnicodeUCS2_GetDefaultEncoding" + "PyUnicodeUCS2_DecodeUTF8" + "PyUnicodeUCS2_DecodeLatin1" + "PyUnicodeUCS2_DecodeASCII" + "PyUnicodeUCS2_AsEncodedString" + "PyUnicodeUCS2_AsUTF8String" + "PyUnicodeUCS2_AsLatin1String" + "PyUnicodeUCS2_AsASCIIString" + "PyUnicode_DecodeUTF7" + "PyUnicode_EncodeUTF7" + "PyUnicodeUCS2_DecodeUTF8Stateful" + "PyUnicodeUCS2_EncodeUTF8" + "PyUnicodeUCS2_DecodeUTF16Stateful" + "PyUnicodeUCS2_AsUTF16String" + "PyUnicodeUCS2_DecodeUnicodeEscape" + "PyUnicodeUCS2_DecodeRawUnicodeEscape" + "PyUnicodeUCS2_EncodeRawUnicodeEscape" + "_PyUnicode_DecodeUnicodeInternal" + "PyUnicodeUCS2_DecodeCharmap" + "PyUnicode_BuildEncodingMap" + "PyUnicodeUCS2_EncodeCharmap" + "PyUnicodeUCS2_TranslateCharmap" + "PyUnicodeUCS2_EncodeDecimal" + "PyUnicodeUCS2_Count" + "PyUnicodeUCS2_Find" + "PyUnicodeUCS2_Join" + "PyUnicodeUCS2_Splitlines" + "PyUnicodeUCS2_Compare" + "PyUnicodeUCS2_Contains" + "PyUnicodeUCS2_Concat" + "_PyUnicode_XStrip" + "PyUnicodeUCS2_Replace" + "PyUnicodeUCS2_Split" + "PyUnicodeUCS2_RSplit" + "PyUnicodeUCS2_Format" + "_PyUnicodeUCS2_Init" + "_PyUnicodeUCS2_Fini" + "PyUnicodeUCS2_FromUnicode" + "PyUnicodeUCS2_AsUnicode" + "PyUnicodeUCS2_GetSize" + "PyUnicodeUCS2_GetMax" + "_PyUnicodeUCS2_AsDefaultEncodedString" + "PyUnicodeUCS2_SetDefaultEncoding" + "PyUnicodeUCS2_Encode" + "PyUnicodeUCS2_AsEncodedObject" + "PyUnicodeUCS2_DecodeUTF16" + "PyUnicodeUCS2_EncodeUTF16" + "PyUnicodeUCS2_AsUnicodeEscapeString" + "PyUnicodeUCS2_EncodeUnicodeEscape" + "PyUnicodeUCS2_AsRawUnicodeEscapeString" + "PyUnicodeUCS2_EncodeLatin1" + "PyUnicodeUCS2_EncodeASCII" + "PyUnicodeUCS2_AsCharmapString" + "PyUnicodeUCS2_Partition" + "PyUnicodeUCS2_RPartition" + "PyUnicodeUCS2_Translate" + "PyUnicodeUCS2_Tailmatch" + "PyUnicode_AsDecodedObject" + "PyUnicode_Type" + +; From python25_s.lib(unicodectype) + "_PyUnicode_TypeRecords" + "_PyUnicodeUCS2_ToNumeric" + "_PyUnicodeUCS2_IsLowercase" + "_PyUnicodeUCS2_IsUppercase" + "_PyUnicodeUCS2_IsTitlecase" + "_PyUnicodeUCS2_IsWhitespace" + "_PyUnicodeUCS2_IsLinebreak" + "_PyUnicodeUCS2_ToLowercase" + "_PyUnicodeUCS2_ToUppercase" + "_PyUnicodeUCS2_ToTitlecase" + "_PyUnicodeUCS2_ToDecimalDigit" + "_PyUnicodeUCS2_ToDigit" + "_PyUnicodeUCS2_IsDecimalDigit" + "_PyUnicodeUCS2_IsDigit" + "_PyUnicodeUCS2_IsNumeric" + "_PyUnicodeUCS2_IsAlpha" + +; From python25_s.lib(weakrefobject) + "PyWeakref_NewRef" + "PyWeakref_NewProxy" + "PyObject_ClearWeakRefs" + "PyWeakref_GetObject" + "_PyWeakref_GetWeakrefCount" + "_PyWeakref_ClearRef" + "_PyWeakref_RefType" + "_PyWeakref_ProxyType" + "_PyWeakref_CallableProxyType" + +; From python25_s.lib(Python-ast) +; "init_ast" + "Module" + "Interactive" + "Expression" + "Suite" + "FunctionDef" + "ClassDef" + "Return" + "Delete" + "Assign" + "AugAssign" + "Print" + "For" + "While" + "If" + "With" + "Raise" + "TryExcept" + "TryFinally" + "Assert" + "Import" + "ImportFrom" + "Exec" + "Global" + "Expr" + "Pass" + "Break" + "Continue" + "BoolOp" + "BinOp" + "UnaryOp" + "Lambda" + "IfExp" + "Dict" + "ListComp" + "GeneratorExp" + "Yield" + "Compare" + "Call" + "Repr" + "Num" + "Str" + "Attribute" + "Subscript" + "Name" + "List" + "Tuple" + "Ellipsis" + "Slice" + "ExtSlice" + "Index" + "comprehension" + "excepthandler" + "arguments" + "keyword" + "alias" + "PyAST_mod2obj" + +; From python25_s.lib(asdl) + "asdl_seq_new" + "asdl_int_seq_new" + +; From python25_s.lib(ast) + "PyAST_FromNode" + +; From python25_s.lib(bltinmodule) + "_PyBuiltin_Init" + "Py_FileSystemDefaultEncoding" + +; From python25_s.lib(exceptions) + "PyUnicodeEncodeError_GetStart" + "PyUnicodeDecodeError_GetStart" + "PyUnicodeEncodeError_GetEnd" + "PyUnicodeDecodeError_GetEnd" + "_PyExc_Init" + "PyUnicodeDecodeError_Create" + "PyUnicodeEncodeError_Create" + "PyUnicodeTranslateError_Create" + "PyUnicodeEncodeError_GetEncoding" + "PyUnicodeDecodeError_GetEncoding" + "PyUnicodeEncodeError_GetObject" + "PyUnicodeDecodeError_GetObject" + "PyUnicodeTranslateError_GetObject" + "PyUnicodeTranslateError_GetStart" + "PyUnicodeEncodeError_SetStart" + "PyUnicodeDecodeError_SetStart" + "PyUnicodeTranslateError_SetStart" + "PyUnicodeTranslateError_GetEnd" + "PyUnicodeEncodeError_SetEnd" + "PyUnicodeDecodeError_SetEnd" + "PyUnicodeTranslateError_SetEnd" + "PyUnicodeEncodeError_GetReason" + "PyUnicodeDecodeError_GetReason" + "PyUnicodeTranslateError_GetReason" + "PyUnicodeEncodeError_SetReason" + "PyUnicodeDecodeError_SetReason" + "PyUnicodeTranslateError_SetReason" + "_PyExc_Fini" + "PyExc_BaseException" + "PyExc_Exception" + "PyExc_StandardError" + "PyExc_TypeError" + "PyExc_StopIteration" + "PyExc_GeneratorExit" + "PyExc_SystemExit" + "PyExc_KeyboardInterrupt" + "PyExc_ImportError" + "PyExc_EnvironmentError" + "PyExc_IOError" + "PyExc_OSError" + "PyExc_EOFError" + "PyExc_RuntimeError" + "PyExc_NotImplementedError" + "PyExc_NameError" + "PyExc_UnboundLocalError" + "PyExc_AttributeError" + "PyExc_IndexError" + "PyExc_SyntaxError" + "PyExc_IndentationError" + "PyExc_TabError" + "PyExc_LookupError" + "PyExc_KeyError" + "PyExc_ValueError" + "PyExc_UnicodeError" + "PyExc_UnicodeEncodeError" + "PyExc_UnicodeDecodeError" + "PyExc_UnicodeTranslateError" + "PyExc_AssertionError" + "PyExc_ArithmeticError" + "PyExc_FloatingPointError" + "PyExc_OverflowError" + "PyExc_ZeroDivisionError" + "PyExc_SystemError" + "PyExc_ReferenceError" + "PyExc_MemoryError" + "PyExc_Warning" + "PyExc_UserWarning" + "PyExc_DeprecationWarning" + "PyExc_PendingDeprecationWarning" + "PyExc_SyntaxWarning" + "PyExc_RuntimeWarning" + "PyExc_FutureWarning" + "PyExc_ImportWarning" + "PyExc_MemoryErrorInst" + +; From python25_s.lib(ceval) + "PyEval_EvalFrameEx" + "PyEval_CallObjectWithKeywords" + "PyEval_EvalCodeEx" + "PyEval_GetFrame" + "PyEval_CallObject" + "PyEval_SetProfile" + "PyEval_SetTrace" + "PyEval_GetBuiltins" + "PyEval_GetGlobals" + "PyEval_GetLocals" + "PyEval_GetRestricted" + "PyEval_MergeCompilerFlags" + "Py_FlushLine" + "Py_AddPendingCall" + "Py_MakePendingCalls" + "Py_SetRecursionLimit" + "Py_GetRecursionLimit" + "_Py_CheckRecursiveCall" + "PyEval_GetFuncName" + "PyEval_GetFuncDesc" + "PyEval_GetCallStats" + "PyEval_EvalFrame" + "PyEval_SaveThread" + "PyEval_RestoreThread" + "PyEval_ThreadsInitialized" + "PyEval_InitThreads" + "PyEval_AcquireLock" + "PyEval_ReleaseLock" + "PyEval_AcquireThread" + "PyEval_ReleaseThread" + "PyEval_ReInitThreads" + "_PyEval_SliceIndex" + "PyEval_EvalCode" + "_PyEval_CallTracing" + "_Py_CheckRecursionLimit" + "_Py_CheckInterval" + "_Py_Ticker" + +; From python25_s.lib(compile) + "_Py_Mangle" + "PyAST_Compile" + "PyNode_Compile" + "Py_OptimizeFlag" + +; From python25_s.lib(codecs) + "_PyCodec_Lookup" + "PyCodec_Encode" + "PyCodec_Decode" + "PyCodec_IgnoreErrors" + "PyCodec_ReplaceErrors" + "PyCodec_XMLCharRefReplaceErrors" + "PyCodec_BackslashReplaceErrors" + "PyCodec_Register" + "PyCodec_Encoder" + "PyCodec_Decoder" + "PyCodec_IncrementalEncoder" + "PyCodec_IncrementalDecoder" + "PyCodec_StreamReader" + "PyCodec_StreamWriter" + "PyCodec_RegisterError" + "PyCodec_LookupError" + "PyCodec_StrictErrors" + +; From python25_s.lib(errors) + "PyErr_SetNone" + "PyErr_SetString" + "PyErr_GivenExceptionMatches" + "PyErr_NormalizeException" + "PyErr_Fetch" + "PyErr_Clear" + "PyErr_NoMemory" + "PyErr_SetFromErrnoWithFilenameObject" + "PyErr_Format" + "PyErr_NewException" + "PyErr_WriteUnraisable" + "PyErr_SyntaxLocation" + "PyErr_ProgramText" + "PyErr_SetObject" + "PyErr_Occurred" + "PyErr_Restore" + "PyErr_ExceptionMatches" + "PyErr_BadArgument" + "PyErr_SetFromErrno" + "PyErr_SetFromErrnoWithFilename" + "PyErr_BadInternalCall" + "_PyErr_BadInternalCall" + "PyErr_Warn" + "PyErr_WarnExplicit" + +; From python25_s.lib(frozen) + "PyImport_FrozenModules" + +; From python25_s.lib(frozenmain) + "Py_FrozenMain" + +; From python25_s.lib(future) + "PyFuture_FromAST" + +; From python25_s.lib(getargs) + "PyArg_Parse" + "_PyArg_Parse_SizeT" + "PyArg_ParseTuple" + "_PyArg_ParseTuple_SizeT" + "PyArg_ParseTupleAndKeywords" + "_PyArg_ParseTupleAndKeywords_SizeT" + "PyArg_UnpackTuple" + "_PyArg_NoKeywords" + "PyArg_VaParse" + "PyArg_VaParseTupleAndKeywords" + "_PyArg_VaParse_SizeT" + "_PyArg_VaParseTupleAndKeywords_SizeT" + +; From python25_s.lib(getcompiler) + "Py_GetCompiler" + +; From python25_s.lib(getcopyright) + "Py_GetCopyright" + +; From python25_s.lib(getmtime) + "PyOS_GetLastModificationTime" + +; From python25_s.lib(getplatform) + "Py_GetPlatform" + +; From python25_s.lib(getversion) + "Py_GetVersion" + +; From python25_s.lib(graminit) + "_PyParser_Grammar" + +; From python25_s.lib(import) + "_PyImport_Init" + "_PyImportHooks_Init" + "PyImport_ImportModule" + "PyImport_Cleanup" + "_PyImport_FixupExtension" + "PyImport_AddModule" + "PyImport_ExecCodeModuleEx" + "PyImport_ImportFrozenModule" + "PyImport_ImportModuleEx" + "PyImport_ImportModuleLevel" + "PyImport_ReloadModule" + "PyImport_Import" +; "initimp" + "_PyImport_Fini" + "PyImport_GetMagicNumber" + "PyImport_ExecCodeModule" + "PyImport_GetModuleDict" + "_PyImport_FindModule" + "_PyImport_IsScript" + "_PyImport_ReInitLock" + "_PyImport_FindExtension" + "PyImport_AppendInittab" + "PyImport_ExtendInittab" + "PyImport_Inittab" + "_PyImport_Filetab" + +; From python25_s.lib(importdl) + "_PyImport_LoadDynamicModule" + +; From python25_s.lib(marshal) + "PyMarshal_ReadLongFromFile" + "PyMarshal_WriteObjectToString" + "PyMarshal_WriteLongToFile" + "PyMarshal_WriteObjectToFile" + "PyMarshal_ReadShortFromFile" + "PyMarshal_ReadObjectFromFile" + "PyMarshal_ReadLastObjectFromFile" + "PyMarshal_ReadObjectFromString" + "PyMarshal_Init" + +; From python25_s.lib(modsupport) + "Py_InitModule4" + "Py_BuildValue" + "_Py_BuildValue_SizeT" + "PyEval_CallFunction" + "PyEval_CallMethod" + "_Py_VaBuildValue_SizeT" + "Py_VaBuildValue" + "PyModule_AddObject" + "PyModule_AddIntConstant" + "PyModule_AddStringConstant" + "_Py_PackageContext" + +; From python25_s.lib(mysnprintf) + "PyOS_snprintf" + "PyOS_vsnprintf" + +; From python25_s.lib(mystrtoul) + "PyOS_strtoul" + "PyOS_strtol" + +; From python25_s.lib(pyarena) + "PyArena_New" + "PyArena_Free" + "PyArena_Malloc" + "PyArena_AddPyObject" + +; From python25_s.lib(pyfpe) + "PyFPE_dummy" + +; From python25_s.lib(pystate) + "PyInterpreterState_Clear" + "PyThreadState_Clear" + "_PyThread_CurrentFrames" + "PyGILState_Ensure" + "PyGILState_Release" + "PyInterpreterState_New" + "PyInterpreterState_Delete" + "PyThreadState_Delete" + "PyThreadState_New" + "PyThreadState_DeleteCurrent" + "PyThreadState_Get" + "PyThreadState_Swap" + "PyThreadState_GetDict" + "PyThreadState_SetAsyncExc" + "PyGILState_GetThisThreadState" + "PyInterpreterState_Head" + "PyInterpreterState_Next" + "PyInterpreterState_ThreadHead" + "PyThreadState_Next" + "_PyGILState_Init" + "_PyGILState_Fini" + "_PyThreadState_Current" + "_PyThreadState_GetFrame" + +; From python25_s.lib(pystrtod) + "PyOS_ascii_strtod" + "PyOS_ascii_formatd" + "PyOS_ascii_atof" + +; From python25_s.lib(pythonrun) + "Py_IgnoreEnvironmentFlag" + "Py_DebugFlag" + "Py_VerboseFlag" + "Py_NoSiteFlag" + "Py_InteractiveFlag" + "Py_FrozenFlag" + "Py_InitializeEx" + "Py_FatalError" + "Py_NewInterpreter" + "PyErr_Print" + "PyRun_InteractiveOneFlags" + "PyParser_ASTFromFile" + "PyRun_SimpleFileExFlags" + "PyRun_FileExFlags" + "Py_Exit" + "PyErr_PrintEx" + "PyErr_Display" + "Py_SetProgramName" + "Py_GetProgramName" + "Py_SetPythonHome" + "Py_GetPythonHome" + "Py_Initialize" + "Py_Finalize" + "Py_IsInitialized" + "Py_EndInterpreter" + "PyRun_AnyFileFlags" + "Py_FdIsInteractive" + "PyRun_InteractiveLoopFlags" + "PyRun_AnyFileExFlags" + "PyRun_SimpleStringFlags" + "PyRun_StringFlags" + "PyParser_ASTFromString" + "PyParser_SimpleParseStringFlags" + "PyParser_SimpleParseFileFlags" + "Py_CompileStringFlags" + "Py_SymtableString" + "Py_AtExit" + "PyOS_getsig" + "PyOS_setsig" + "PyParser_SetError" + "PyModule_GetWarningsModule" + "PyParser_SimpleParseStringFlagsFilename" + "PyParser_SimpleParseStringFilename" + "PyParser_SimpleParseFile" + "PyParser_SimpleParseString" + "PyRun_AnyFile" + "PyRun_AnyFileEx" + "PyRun_File" + "PyRun_FileEx" + "PyRun_FileFlags" + "PyRun_SimpleFile" + "PyRun_SimpleFileEx" + "PyRun_String" + "PyRun_SimpleString" + "Py_CompileString" + "PyRun_InteractiveOne" + "PyRun_InteractiveLoop" + "Py_UseClassExceptionsFlag" + "Py_UnicodeFlag" + "_Py_QnewFlag" + +; From python25_s.lib(structmember) + "PyMember_Get" + "PyMember_GetOne" + "PyMember_SetOne" + "PyMember_Set" + +; From python25_s.lib(symtable) + "PySymtable_Build" + "PySymtable_Free" + "PyST_GetScope" + "PySymtable_Lookup" + "PySTEntry_Type" + +; From python25_s.lib(sysmodule) + "_PySys_Init" + "PySys_WriteStderr" + "PySys_SetPath" + "PySys_SetArgv" + "PySys_WriteStdout" + "Py_SubversionRevision" + "Py_SubversionShortBranch" + "PySys_GetObject" + "PySys_SetObject" + "PySys_GetFile" + "PySys_ResetWarnOptions" + "PySys_AddWarnOption" + +; From python25_s.lib(traceback) + "PyTraceBack_Here" + "PyTraceBack_Print" + "PyTraceBack_Type" + +; From python25_s.lib(getopt) + "_PyOS_GetOpt" + "_PyOS_opterr" + "_PyOS_optind" + "_PyOS_optarg" + +; From python25_s.lib(dynload_shlib) + "_PyImport_DynLoadFiletab" + "_PyImport_GetDynLoadFunc" + +; From python25_s.lib(thread) + "PyThread_delete_key_value" + "PyThread_init_thread" + "PyThread_start_new_thread" + "PyThread_exit_thread" + "PyThread_get_thread_ident" + "PyThread_allocate_lock" + "PyThread_free_lock" + "PyThread_acquire_lock" + "PyThread_release_lock" + "PyThread_get_stacksize" + "PyThread_set_stacksize" + "PyThread_create_key" + "PyThread_delete_key" + "PyThread_set_key_value" + "PyThread_get_key_value" + "PyThread__exit_thread" + +; From python25_s.lib(gcmodule) +; "initgc" + "_PyObject_GC_New" + "_PyObject_GC_NewVar" + "PyGC_Collect" + "_PyObject_GC_Resize" + "_PyObject_GC_Malloc" + "PyObject_GC_Track" + "PyObject_GC_UnTrack" + "PyObject_GC_Del" + "_PyGC_Dump" + "_PyObject_GC_Track" + "_PyObject_GC_UnTrack" + "_PyObject_GC_Del" + "_PyGC_generation0" + +; From python25_s.lib(signalmodule) +; "initsignal" + "PyErr_CheckSignals" + "PyErr_SetInterrupt" + "PyOS_FiniInterrupts" + "PyOS_InterruptOccurred" + "PyOS_InitInterrupts" + "PyOS_AfterFork" + +; From python25_s.lib(posixmodule) +; "initos2" + +; From python25_s.lib(threadmodule) +; "initthread" + +; From python25_s.lib(arraymodule) +; "initarray" +; "array_methods" + +; From python25_s.lib(binascii) +; "initbinascii" + +; From python25_s.lib(cmathmodule) +; "initcmath" + +; From python25_s.lib(_codecsmodule) +; "init_codecs" + +; From python25_s.lib(collectionsmodule) +; "initcollections" + "dequeiter_type" + "dequereviter_type" + +; From python25_s.lib(cPickle) +; "initcPickle" +; "fast_save_leave" + +; From python25_s.lib(cStringIO) +; "initcStringIO" + +; From python25_s.lib(_csv) +; "init_csv" + +; From python25_s.lib(datetimemodule) +; "initdatetime" + +; From python25_s.lib(dlmodule) +; "initdl" + +; From python25_s.lib(errnomodule) +; "initerrno" + +; From python25_s.lib(fcntlmodule) +; "initfcntl" + +; From python25_s.lib(_functoolsmodule) +; "init_functools" + +; From python25_s.lib(_heapqmodule) +; "init_heapq" + +; From python25_s.lib(imageop) +; "initimageop" + +; From python25_s.lib(itertoolsmodule) +; "inititertools" + +; From python25_s.lib(_localemodule) +; "init_locale" + +; From python25_s.lib(mathmodule) +; "initmath" + +; From python25_s.lib(md5) + "md5_finish" + "md5_init" + "md5_append" + +; From python25_s.lib(md5module) +; "init_md5" + +; From python25_s.lib(operator) +; "initoperator" + +; From python25_s.lib(_randommodule) +; "init_random" + +; From python25_s.lib(rgbimgmodule) +; "initrgbimg" + +; From python25_s.lib(shamodule) +; "init_sha" + +; From python25_s.lib(sha256module) +; "init_sha256" + +; From python25_s.lib(sha512module) +; "init_sha512" + +; From python25_s.lib(_sre) +; "init_sre" + +; From python25_s.lib(stropmodule) +; "initstrop" + +; From python25_s.lib(_struct) +; "init_struct" + +; From python25_s.lib(symtablemodule) +; "init_symtable" + +; From python25_s.lib(termios) +; "inittermios" + +; From python25_s.lib(timemodule) +; "inittime" + "_PyTime_DoubleToTimet" +; "inittimezone" + +; From python25_s.lib(timingmodule) +; "inittiming" + +; From python25_s.lib(_weakref) +; "init_weakref" + +; From python25_s.lib(xxsubtype) +; "initxxsubtype" + +; From python25_s.lib(zipimport) +; "initzipimport" diff --git a/PC/os2vacpp/pyconfig.h b/PC/os2vacpp/pyconfig.h index c858fe9..97f9b80 100644 --- a/PC/os2vacpp/pyconfig.h +++ b/PC/os2vacpp/pyconfig.h @@ -112,6 +112,10 @@ typedef int pid_t; #define HAVE_HYPOT 1 /* hypot() */ #define HAVE_PUTENV 1 /* putenv() */ /* #define VA_LIST_IS_ARRAY 1 */ /* if va_list is an array of some kind */ +/* #define HAVE_CONIO_H 1 */ /* #include */ +#define HAVE_ERRNO_H 1 /* #include */ +#define HAVE_SYS_STAT_H 1 /* #include */ +#define HAVE_SYS_TYPES_H 1 /* #include */ /* Variable-Arguments/Prototypes */ #define HAVE_PROTOTYPES 1 /* VAC++ supports C Function Prototypes */ @@ -124,6 +128,7 @@ typedef int pid_t; #define MALLOC_ZERO_RETURNS_NULL 1 /* Our malloc(0) returns a NULL ptr */ /* Signal Handling */ +#define HAVE_SIGNAL_H 1 /* signal.h */ #define RETSIGTYPE void /* Return type of handlers (int or void) */ /* #undef WANT_SIGFPE_HANDLER */ /* Handle SIGFPE (see Include/pyfpe.h) */ /* #define HAVE_ALARM 1 */ /* alarm() */ @@ -163,7 +168,9 @@ typedef int pid_t; #define HAVE_SETVBUF 1 /* setvbuf() */ #define HAVE_GETCWD 1 /* getcwd() */ #define HAVE_PIPE 1 /* pipe() [OS/2-specific code added] */ +#define HAVE_IO_H 1 /* #include */ #define HAVE_FCNTL_H 1 /* #include */ +#define HAVE_DIRECT_H 1 /* #include */ /* #define HAVE_FLOCK 1 */ /* flock() */ /* #define HAVE_TRUNCATE 1 */ /* truncate() */ /* #define HAVE_FTRUNCATE 1 */ /* ftruncate() */ @@ -172,6 +179,7 @@ typedef int pid_t; /* #define HAVE_OPENDIR 1 */ /* opendir() */ /* Process Operations */ +#define HAVE_PROCESS_H 1 /* #include */ #define HAVE_GETPID 1 /* getpid() */ #define HAVE_SYSTEM 1 /* system() */ #define HAVE_WAIT 1 /* wait() */ diff --git a/PC/pyconfig.h b/PC/pyconfig.h index cb42131..e0df673 100644 --- a/PC/pyconfig.h +++ b/PC/pyconfig.h @@ -78,8 +78,15 @@ MS_CORE_DLL. #endif #ifdef MS_WINCE -#define DONT_HAVE_SYS_STAT_H -#define DONT_HAVE_ERRNO_H +/* Python uses GetVersion() to distinguish between + * Windows NT and 9x/ME where OS Unicode support is concerned. + * Windows CE supports Unicode in the same way as NT so we + * define the missing GetVersion() accordingly. + */ +#define GetVersion() (4) +/* Windows CE does not support environment variables */ +#define getenv(v) (NULL) +#define environ (NULL) #endif /* Compiler specific defines */ @@ -164,6 +171,12 @@ typedef int pid_t; #define Py_IS_INFINITY(X) (!_finite(X) && !_isnan(X)) #define Py_IS_FINITE(X) _finite(X) +/* Turn off warnings about deprecated C runtime functions in + VisualStudio .NET 2005 */ +#if _MSC_VER >= 1400 && !defined _CRT_SECURE_NO_DEPRECATE +#define _CRT_SECURE_NO_DEPRECATE +#endif + #endif /* _MSC_VER */ /* define some ANSI types that are not defined in earlier Win headers */ @@ -350,6 +363,16 @@ Py_NO_ENABLE_SHARED to find out. Also support MS_NO_COREDLL for b/w compat */ /* Define to empty if the keyword does not work. */ /* #define const */ +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_CONIO_H 1 +#endif + +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_DIRECT_H 1 +#endif + /* Define if you have dirent.h. */ /* #define DIRENT 1 */ @@ -555,11 +578,26 @@ Py_NO_ENABLE_SHARED to find out. Also support MS_NO_COREDLL for b/w compat */ /* Define if you have the header file. */ /* #undef HAVE_DLFCN_H */ +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_ERRNO_H 1 +#endif + /* Define if you have the header file. */ #ifndef MS_WINCE #define HAVE_FCNTL_H 1 #endif +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_PROCESS_H 1 +#endif + +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_SIGNAL_H 1 +#endif + /* Define if you have the prototypes. */ #define HAVE_STDARG_PROTOTYPES @@ -575,12 +613,22 @@ Py_NO_ENABLE_SHARED to find out. Also support MS_NO_COREDLL for b/w compat */ /* Define if you have the header file. */ /* #define HAVE_SYS_SELECT_H 1 */ +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_SYS_STAT_H 1 +#endif + /* Define if you have the header file. */ /* #define HAVE_SYS_TIME_H 1 */ /* Define if you have the header file. */ /* #define HAVE_SYS_TIMES_H 1 */ +/* Define to 1 if you have the header file. */ +#ifndef MS_WINCE +#define HAVE_SYS_TYPES_H 1 +#endif + /* Define if you have the header file. */ /* #define HAVE_SYS_UN_H 1 */ diff --git a/PC/winsound.c b/PC/winsound.c index bc30ccc..4e94230 100644 --- a/PC/winsound.c +++ b/PC/winsound.c @@ -37,8 +37,10 @@ #include #include -#include /* port functions on Win9x */ #include +#ifdef HAVE_CONIO_H +#include /* port functions on Win9x */ +#endif PyDoc_STRVAR(sound_playsound_doc, "PlaySound(sound, flags) - a wrapper around the Windows PlaySound API\n" @@ -145,7 +147,7 @@ sound_beep(PyObject *self, PyObject *args) return NULL; } } -#ifdef _M_IX86 +#if defined(_M_IX86) && defined(HAVE_CONIO_H) else if (whichOS == Win9X) { int speaker_state; /* Force timer into oscillator mode via timer control port. */ @@ -170,7 +172,7 @@ sound_beep(PyObject *self, PyObject *args) /* Restore speaker control to original state. */ _outp(0x61, speaker_state); } -#endif /* _M_IX86 */ +#endif /* _M_IX86 && HAVE_CONIO_H */ else { assert(!"winsound's whichOS has insane value"); } diff --git a/PCbuild/_msi.vcproj b/PCbuild/_msi.vcproj index fb8925e..b4afeb4 100644 --- a/PCbuild/_msi.vcproj +++ b/PCbuild/_msi.vcproj @@ -36,11 +36,11 @@ diff --git a/PCbuild/_ssl.vcproj b/PCbuild/_ssl.vcproj index 3ebedbd..bc69ee8 100644 --- a/PCbuild/_ssl.vcproj +++ b/PCbuild/_ssl.vcproj @@ -21,8 +21,8 @@ ATLMinimizesCRunTimeLibraryUsage="FALSE"> @@ -35,8 +35,8 @@ ATLMinimizesCRunTimeLibraryUsage="FALSE"> @@ -49,8 +49,9 @@ ATLMinimizesCRunTimeLibraryUsage="FALSE"> diff --git a/PCbuild/build_ssl.bat b/PCbuild/build_ssl.bat new file mode 100644 index 0000000..82572ed --- /dev/null +++ b/PCbuild/build_ssl.bat @@ -0,0 +1,10 @@ +@echo off +if not defined HOST_PYTHON ( + if %1 EQU Debug ( + set HOST_PYTHON=python_d.exe + ) ELSE ( + set HOST_PYTHON=python.exe + ) +) +%HOST_PYTHON% build_ssl.py %1 %2 + diff --git a/PCbuild/build_ssl.py b/PCbuild/build_ssl.py index 8f485a2..d4beee5 100644 --- a/PCbuild/build_ssl.py +++ b/PCbuild/build_ssl.py @@ -84,9 +84,59 @@ def find_best_ssl_dir(sources): print "Could not find an SSL directory in '%s'" % (sources,) return best_name +def run_32all_py(): + # ms\32all.bat will reconfigure OpenSSL and then try to build + # all outputs (debug/nondebug/dll/lib). So we filter the file + # to exclude any "nmake" commands and then execute. + tempname = "ms\\32all_py.bat" + + in_bat = open("ms\\32all.bat") + temp_bat = open(tempname,"w") + while 1: + cmd = in_bat.readline() + print 'cmd', repr(cmd) + if not cmd: break + if cmd.strip()[:5].lower() == "nmake": + continue + temp_bat.write(cmd) + in_bat.close() + temp_bat.close() + os.system(tempname) + try: + os.remove(tempname) + except: + pass + +def run_configure(configure, do_script): + os.system("perl Configure "+configure) + os.system(do_script) + def main(): - debug = "-d" in sys.argv build_all = "-a" in sys.argv + if sys.argv[1] == "Release": + arch = "x86" + debug = False + configure = "VC-WIN32" + makefile = "32.mak" + elif sys.argv[1] == "Debug": + arch = "x86" + debug = True + configure = "VC-WIN32" + makefile="d32.mak" + elif sys.argv[1] == "ReleaseItanium": + arch = "ia64" + debug = False + configure = "VC-WIN64I" + do_script = "ms\\do_win64i" + makefile = "ms\\nt.mak" + os.environ["VSEXTCOMP_USECL"] = "MS_ITANIUM" + elif sys.argv[1] == "ReleaseAMD64": + arch="amd64" + debug=False + configure = "VC-WIN64A" + do_script = "ms\\do_win64a" + makefile = "ms\\nt.mak" + os.environ["VSEXTCOMP_USECL"] = "MS_OPTERON" make_flags = "" if build_all: make_flags = "-a" @@ -107,49 +157,24 @@ def main(): try: os.chdir(ssl_dir) # If the ssl makefiles do not exist, we invoke Perl to generate them. - if not os.path.isfile(os.path.join(ssl_dir, "32.mak")) or \ - not os.path.isfile(os.path.join(ssl_dir, "d32.mak")): + if not os.path.isfile(makefile): print "Creating the makefiles..." # Put our working Perl at the front of our path os.environ["PATH"] = os.path.split(perl)[0] + \ os.pathsep + \ os.environ["PATH"] - # ms\32all.bat will reconfigure OpenSSL and then try to build - # all outputs (debug/nondebug/dll/lib). So we filter the file - # to exclude any "nmake" commands and then execute. - tempname = "ms\\32all_py.bat" - - in_bat = open("ms\\32all.bat") - temp_bat = open(tempname,"w") - while 1: - cmd = in_bat.readline() - print 'cmd', repr(cmd) - if not cmd: break - if cmd.strip()[:5].lower() == "nmake": - continue - temp_bat.write(cmd) - in_bat.close() - temp_bat.close() - os.system(tempname) - try: - os.remove(tempname) - except: - pass + if arch=="x86": + run_32all_py() + else: + run_configure(configure, do_script) # Now run make. print "Executing nmake over the ssl makefiles..." - if debug: - rc = os.system("nmake /nologo -f d32.mak") - if rc: - print "Executing d32.mak failed" - print rc - sys.exit(rc) - else: - rc = os.system("nmake /nologo -f 32.mak") - if rc: - print "Executing 32.mak failed" - print rc - sys.exit(rc) + rc = os.system("nmake /nologo -f "+makefile) + if rc: + print "Executing d32.mak failed" + print rc + sys.exit(rc) finally: os.chdir(old_cd) # And finally, we can build the _ssl module itself for Python. diff --git a/PCbuild/pythoncore.vcproj b/PCbuild/pythoncore.vcproj index bb9f7af..ef200c4 100644 --- a/PCbuild/pythoncore.vcproj +++ b/PCbuild/pythoncore.vcproj @@ -344,9 +344,6 @@ RelativePath="..\Modules\_bisectmodule.c"> - - + + + + + + - - c_arena) : NULL); if (!args && n_args) - return NULL; /* Don't need to go to NULL; nothing allocated */ + return NULL; /* Don't need to goto error; no objects allocated */ defaults = (n_defaults ? asdl_seq_new(n_defaults, c->c_arena) : NULL); if (!defaults && n_defaults) - goto error; + return NULL; /* Don't need to goto error; no objects allocated */ /* fpdef: NAME | '(' fplist ')' fplist: fpdef (',' fpdef)* [','] @@ -638,8 +641,11 @@ ast_for_arguments(struct compiling *c, const node *n) anything other than EQUAL or a comma? */ /* XXX Should NCH(n) check be made a separate check? */ if (i + 1 < NCH(n) && TYPE(CHILD(n, i + 1)) == EQUAL) { - asdl_seq_SET(defaults, j++, - ast_for_expr(c, CHILD(n, i + 2))); + expr_ty expression = ast_for_expr(c, CHILD(n, i + 2)); + if (!expression) + goto error; + assert(defaults != NULL); + asdl_seq_SET(defaults, j++, expression); i += 2; found_default = 1; } @@ -1484,6 +1490,57 @@ ast_for_trailer(struct compiling *c, const node *n, expr_ty left_expr) } static expr_ty +ast_for_factor(struct compiling *c, const node *n) +{ + node *pfactor, *ppower, *patom, *pnum; + expr_ty expression; + + /* If the unary - operator is applied to a constant, don't generate + a UNARY_NEGATIVE opcode. Just store the approriate value as a + constant. The peephole optimizer already does something like + this but it doesn't handle the case where the constant is + (sys.maxint - 1). In that case, we want a PyIntObject, not a + PyLongObject. + */ + if (TYPE(CHILD(n, 0)) == MINUS + && NCH(n) == 2 + && TYPE((pfactor = CHILD(n, 1))) == factor + && NCH(pfactor) == 1 + && TYPE((ppower = CHILD(pfactor, 0))) == power + && NCH(ppower) == 1 + && TYPE((patom = CHILD(ppower, 0))) == atom + && TYPE((pnum = CHILD(patom, 0))) == NUMBER) { + char *s = PyObject_MALLOC(strlen(STR(pnum)) + 2); + if (s == NULL) + return NULL; + s[0] = '-'; + strcpy(s + 1, STR(pnum)); + PyObject_FREE(STR(pnum)); + STR(pnum) = s; + return ast_for_atom(c, patom); + } + + expression = ast_for_expr(c, CHILD(n, 1)); + if (!expression) + return NULL; + + switch (TYPE(CHILD(n, 0))) { + case PLUS: + return UnaryOp(UAdd, expression, LINENO(n), n->n_col_offset, + c->c_arena); + case MINUS: + return UnaryOp(USub, expression, LINENO(n), n->n_col_offset, + c->c_arena); + case TILDE: + return UnaryOp(Invert, expression, LINENO(n), + n->n_col_offset, c->c_arena); + } + PyErr_Format(PyExc_SystemError, "unhandled factor: %d", + TYPE(CHILD(n, 0))); + return NULL; +} + +static expr_ty ast_for_power(struct compiling *c, const node *n) { /* power: atom trailer* ('**' factor)* @@ -1662,30 +1719,12 @@ ast_for_expr(struct compiling *c, const node *n) } return Yield(exp, LINENO(n), n->n_col_offset, c->c_arena); } - case factor: { - expr_ty expression; - + case factor: if (NCH(n) == 1) { n = CHILD(n, 0); goto loop; } - - expression = ast_for_expr(c, CHILD(n, 1)); - if (!expression) - return NULL; - - switch (TYPE(CHILD(n, 0))) { - case PLUS: - return UnaryOp(UAdd, expression, LINENO(n), n->n_col_offset, c->c_arena); - case MINUS: - return UnaryOp(USub, expression, LINENO(n), n->n_col_offset, c->c_arena); - case TILDE: - return UnaryOp(Invert, expression, LINENO(n), n->n_col_offset, c->c_arena); - } - PyErr_Format(PyExc_SystemError, "unhandled factor: %d", - TYPE(CHILD(n, 0))); - break; - } + return ast_for_factor(c, n); case power: return ast_for_power(c, n); default: @@ -1893,20 +1932,19 @@ ast_for_expr_stmt(struct compiling *c, const node *n) operator_ty newoperator; node *ch = CHILD(n, 0); - if (TYPE(ch) == testlist) - expr1 = ast_for_testlist(c, ch); - else - expr1 = Yield(ast_for_expr(c, CHILD(ch, 0)), LINENO(ch), n->n_col_offset, - c->c_arena); - + expr1 = ast_for_testlist(c, ch); if (!expr1) return NULL; - /* TODO(jhylton): Figure out why set_context() can't be used here. */ + /* TODO(nas): Remove duplicated error checks (set_context does it) */ switch (expr1->kind) { case GeneratorExp_kind: ast_error(ch, "augmented assignment to generator " "expression not possible"); return NULL; + case Yield_kind: + ast_error(ch, "augmented assignment to yield " + "expression not possible"); + return NULL; case Name_kind: { const char *var_name = PyString_AS_STRING(expr1->v.Name.id); if (var_name[0] == 'N' && !strcmp(var_name, "None")) { @@ -1923,12 +1961,13 @@ ast_for_expr_stmt(struct compiling *c, const node *n) "assignment"); return NULL; } + set_context(expr1, Store, ch); ch = CHILD(n, 2); if (TYPE(ch) == testlist) expr2 = ast_for_testlist(c, ch); else - expr2 = Yield(ast_for_expr(c, ch), LINENO(ch), ch->n_col_offset, c->c_arena); + expr2 = ast_for_expr(c, ch); if (!expr2) return NULL; @@ -2142,7 +2181,14 @@ alias_for_import_name(struct compiling *c, const node *n) loop: switch (TYPE(n)) { case import_as_name: - str = (NCH(n) == 3) ? NEW_IDENTIFIER(CHILD(n, 2)) : NULL; + str = NULL; + if (NCH(n) == 3) { + if (strcmp(STR(CHILD(n, 1)), "as") != 0) { + ast_error(n, "must use 'as' in import"); + return NULL; + } + str = NEW_IDENTIFIER(CHILD(n, 2)); + } return alias(NEW_IDENTIFIER(CHILD(n, 0)), str, c->c_arena); case dotted_as_name: if (NCH(n) == 1) { @@ -2151,6 +2197,10 @@ alias_for_import_name(struct compiling *c, const node *n) } else { alias_ty a = alias_for_import_name(c, CHILD(n, 0)); + if (strcmp(STR(CHILD(n, 1)), "as") != 0) { + ast_error(n, "must use 'as' in import"); + return NULL; + } assert(!a->asname); a->asname = NEW_IDENTIFIER(CHILD(n, 2)); return a; @@ -2621,6 +2671,7 @@ ast_for_for_stmt(struct compiling *c, const node *n) asdl_seq *_target, *seq = NULL, *suite_seq; expr_ty expression; expr_ty target; + const node *node_target; /* for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite] */ REQ(n, for_stmt); @@ -2630,10 +2681,13 @@ ast_for_for_stmt(struct compiling *c, const node *n) return NULL; } - _target = ast_for_exprlist(c, CHILD(n, 1), Store); + node_target = CHILD(n, 1); + _target = ast_for_exprlist(c, node_target, Store); if (!_target) return NULL; - if (asdl_seq_LEN(_target) == 1) + /* Check the # of children rather than the length of _target, since + for x, in ... has 1 element in _target, but still requires a Tuple. */ + if (NCH(node_target) == 1) target = (expr_ty)asdl_seq_GET(_target, 0); else target = Tuple(_target, Store, LINENO(n), n->n_col_offset, c->c_arena); diff --git a/Python/ceval.c b/Python/ceval.c index f14e7da..9a9b98e 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -205,7 +205,7 @@ PyEval_GetCallStats(PyObject *self) #ifdef WITH_THREAD -#ifndef DONT_HAVE_ERRNO_H +#ifdef HAVE_ERRNO_H #include #endif #include "pythread.h" @@ -2740,7 +2740,7 @@ PyEval_EvalCodeEx(PyCodeObject *co, PyObject *globals, PyObject *locals, /* Allocate and initialize storage for cell vars, and copy free vars into frame. This isn't too efficient right now. */ if (PyTuple_GET_SIZE(co->co_cellvars)) { - int i = 0, j = 0, nargs, found; + int i, j, nargs, found; char *cellname, *argname; PyObject *c; @@ -3318,7 +3318,7 @@ int PyEval_GetRestricted(void) { PyFrameObject *current_frame = PyEval_GetFrame(); - return current_frame == NULL ? 0 : current_frame->f_restricted; + return current_frame == NULL ? 0 : PyFrame_IsRestricted(current_frame); } int @@ -4187,6 +4187,14 @@ string_concatenate(PyObject *v, PyObject *w, { /* This function implements 'variable += expr' when both arguments are strings. */ + Py_ssize_t v_len = PyString_GET_SIZE(v); + Py_ssize_t w_len = PyString_GET_SIZE(w); + Py_ssize_t new_len = v_len + w_len; + if (new_len < 0) { + PyErr_SetString(PyExc_OverflowError, + "strings are too large to concat"); + return NULL; + } if (v->ob_refcnt == 2) { /* In the common case, there are 2 references to the value @@ -4231,9 +4239,7 @@ string_concatenate(PyObject *v, PyObject *w, /* Now we own the last reference to 'v', so we can resize it * in-place. */ - Py_ssize_t v_len = PyString_GET_SIZE(v); - Py_ssize_t w_len = PyString_GET_SIZE(w); - if (_PyString_Resize(&v, v_len + w_len) != 0) { + if (_PyString_Resize(&v, new_len) != 0) { /* XXX if _PyString_Resize() fails, 'v' has been * deallocated so it cannot be put back into 'variable'. * The MemoryError is raised when there is no value in diff --git a/Python/codecs.c b/Python/codecs.c index 046abe3..4b0f4cb 100644 --- a/Python/codecs.c +++ b/Python/codecs.c @@ -249,14 +249,17 @@ PyObject *codec_getstreamcodec(const char *encoding, const char *errors, const int index) { - PyObject *codecs, *streamcodec; + PyObject *codecs, *streamcodec, *codeccls; codecs = _PyCodec_Lookup(encoding); if (codecs == NULL) return NULL; - streamcodec = PyEval_CallFunction( - PyTuple_GET_ITEM(codecs, index), "Os", stream, errors); + codeccls = PyTuple_GET_ITEM(codecs, index); + if (errors != NULL) + streamcodec = PyObject_CallFunction(codeccls, "Os", stream, errors); + else + streamcodec = PyObject_CallFunction(codeccls, "O", stream); Py_DECREF(codecs); return streamcodec; } diff --git a/Python/compile.c b/Python/compile.c index 6d96006..4464882 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -143,7 +143,7 @@ struct compiler { PyFutureFeatures *c_future; /* pointer to module's __future__ */ PyCompilerFlags *c_flags; - int c_interactive; + int c_interactive; /* true if in interactive mode */ int c_nestlevel; struct compiler_unit *u; /* compiler state for current block */ @@ -300,8 +300,11 @@ PyCodeObject * PyNode_Compile(struct _node *n, const char *filename) { PyCodeObject *co = NULL; + mod_ty mod; PyArena *arena = PyArena_New(); - mod_ty mod = PyAST_FromNode(n, NULL, filename, arena); + if (!arena) + return NULL; + mod = PyAST_FromNode(n, NULL, filename, arena); if (mod) co = PyAST_Compile(mod, filename, NULL, arena); PyArena_Free(arena); @@ -610,8 +613,10 @@ markblocks(unsigned char *code, int len) unsigned int *blocks = (unsigned int *)PyMem_Malloc(len*sizeof(int)); int i,j, opcode, blockcnt = 0; - if (blocks == NULL) + if (blocks == NULL) { + PyErr_NoMemory(); return NULL; + } memset(blocks, 0, len*sizeof(int)); /* Mark labels in the first pass */ @@ -1066,14 +1071,14 @@ compiler_unit_free(struct compiler_unit *u) PyObject_Free((void *)b); b = next; } - Py_XDECREF(u->u_ste); - Py_XDECREF(u->u_name); - Py_XDECREF(u->u_consts); - Py_XDECREF(u->u_names); - Py_XDECREF(u->u_varnames); - Py_XDECREF(u->u_freevars); - Py_XDECREF(u->u_cellvars); - Py_XDECREF(u->u_private); + Py_CLEAR(u->u_ste); + Py_CLEAR(u->u_name); + Py_CLEAR(u->u_consts); + Py_CLEAR(u->u_names); + Py_CLEAR(u->u_varnames); + Py_CLEAR(u->u_freevars); + Py_CLEAR(u->u_cellvars); + Py_CLEAR(u->u_private); PyObject_Free(u); } @@ -1100,8 +1105,17 @@ compiler_enter_scope(struct compiler *c, identifier name, void *key, u->u_name = name; u->u_varnames = list2dict(u->u_ste->ste_varnames); u->u_cellvars = dictbytype(u->u_ste->ste_symbols, CELL, 0, 0); + if (!u->u_varnames || !u->u_cellvars) { + compiler_unit_free(u); + return 0; + } + u->u_freevars = dictbytype(u->u_ste->ste_symbols, FREE, DEF_FREE_CLASS, PyDict_Size(u->u_cellvars)); + if (!u->u_freevars) { + compiler_unit_free(u); + return 0; + } u->u_blocks = NULL; u->u_tmpname = 0; @@ -1125,7 +1139,8 @@ compiler_enter_scope(struct compiler *c, identifier name, void *key, /* Push the old compiler_unit on the stack. */ if (c->u) { PyObject *wrapper = PyCObject_FromVoidPtr(c->u, NULL); - if (PyList_Append(c->c_stack, wrapper) < 0) { + if (!wrapper || PyList_Append(c->c_stack, wrapper) < 0) { + Py_XDECREF(wrapper); compiler_unit_free(u); return 0; } @@ -1251,6 +1266,7 @@ compiler_next_instr(struct compiler *c, basicblock *b) sizeof(struct instr) * DEFAULT_BLOCK_SIZE); } else if (b->b_iused == b->b_ialloc) { + struct instr *tmp; size_t oldsize, newsize; oldsize = b->b_ialloc * sizeof(struct instr); newsize = oldsize << 1; @@ -1259,10 +1275,13 @@ compiler_next_instr(struct compiler *c, basicblock *b) return -1; } b->b_ialloc <<= 1; - b->b_instr = (struct instr *)PyObject_Realloc( + tmp = (struct instr *)PyObject_Realloc( (void *)b->b_instr, newsize); - if (b->b_instr == NULL) + if (tmp == NULL) { + PyErr_NoMemory(); return -1; + } + b->b_instr = tmp; memset((char *)b->b_instr + oldsize, 0, newsize - oldsize); } return b->b_iused++; @@ -1769,7 +1788,8 @@ compiler_mod(struct compiler *c, mod_ty mod) if (!module) return NULL; } - if (!compiler_enter_scope(c, module, mod, 1)) + /* Use 0 for firstlineno initially, will fixup in assemble(). */ + if (!compiler_enter_scope(c, module, mod, 0)) return NULL; switch (mod->kind) { case Module_kind: @@ -1963,11 +1983,8 @@ compiler_function(struct compiler *c, stmt_ty s) n = asdl_seq_LEN(s->v.FunctionDef.body); /* if there was a docstring, we need to skip the first statement */ for (i = docstring; i < n; i++) { - stmt_ty s2 = (stmt_ty)asdl_seq_GET(s->v.FunctionDef.body, i); - if (i == 0 && s2->kind == Expr_kind && - s2->v.Expr.value->kind == Str_kind) - continue; - VISIT_IN_SCOPE(c, stmt, s2); + st = (stmt_ty)asdl_seq_GET(s->v.FunctionDef.body, i); + VISIT_IN_SCOPE(c, stmt, st); } co = assemble(c, 1); compiler_exit_scope(c); @@ -2190,6 +2207,10 @@ compiler_for(struct compiler *c, stmt_ty s) VISIT(c, expr, s->v.For.iter); ADDOP(c, GET_ITER); compiler_use_next_block(c, start); + /* XXX(nnorwitz): is there a better way to handle this? + for loops are special, we want to be able to trace them + each time around, so we need to set an extra line number. */ + c->u->u_lineno_set = false; ADDOP_JREL(c, FOR_ITER, cleanup); VISIT(c, expr, s->v.For.target); VISIT_SEQ(c, stmt, s->v.For.body); @@ -2708,11 +2729,13 @@ compiler_visit_stmt(struct compiler *c, stmt_ty s) case Global_kind: break; case Expr_kind: - VISIT(c, expr, s->v.Expr.value); if (c->c_interactive && c->c_nestlevel <= 1) { + VISIT(c, expr, s->v.Expr.value); ADDOP(c, PRINT_EXPR); } - else { + else if (s->v.Expr.value->kind != Str_kind && + s->v.Expr.value->kind != Num_kind) { + VISIT(c, expr, s->v.Expr.value); ADDOP(c, POP_TOP); } break; @@ -2989,6 +3012,7 @@ compiler_boolop(struct compiler *c, expr_ty e) return 0; s = e->v.BoolOp.values; n = asdl_seq_LEN(s) - 1; + assert(n >= 0); for (i = 0; i < n; ++i) { VISIT(c, expr, (expr_ty)asdl_seq_GET(s, i)); ADDOP_JREL(c, jumpi, end); @@ -3643,7 +3667,7 @@ compiler_augassign(struct compiler *c, stmt_ty s) assert(s->kind == AugAssign_kind); switch (e->kind) { - case Attribute_kind: + case Attribute_kind: auge = Attribute(e->v.Attribute.value, e->v.Attribute.attr, AugLoad, e->lineno, e->col_offset, c->c_arena); if (auge == NULL) @@ -3666,7 +3690,8 @@ compiler_augassign(struct compiler *c, stmt_ty s) VISIT(c, expr, auge); break; case Name_kind: - VISIT(c, expr, s->v.AugAssign.target); + if (!compiler_nameop(c, e->v.Name.id, Load)) + return 0; VISIT(c, expr, s->v.AugAssign.value); ADDOP(c, inplace_binop(c, s->v.AugAssign.op)); return compiler_nameop(c, e->v.Name.id, Store); @@ -3979,6 +4004,8 @@ stackdepth(struct compiler *c) b->b_startdepth = INT_MIN; entryblock = b; } + if (!entryblock) + return 0; return stackdepth_walk(c, entryblock, 0, 0); } @@ -4074,9 +4101,10 @@ corresponding to a bytecode address A should do something like this In order for this to work, when the addr field increments by more than 255, the line # increment in each pair generated must be 0 until the remaining addr -increment is < 256. So, in the example above, com_set_lineno should not (as -was actually done until 2.2) expand 300, 300 to 255, 255, 45, 45, but to -255, 0, 45, 255, 0, 45. +increment is < 256. So, in the example above, assemble_lnotab (it used +to be called com_set_lineno) should not (as was actually done until 2.2) +expand 300, 300 to 255, 255, 45, 45, + but to 255, 0, 45, 255, 0, 45. */ static int @@ -4092,7 +4120,10 @@ assemble_lnotab(struct assembler *a, struct instr *i) assert(d_bytecode >= 0); assert(d_lineno >= 0); - if (d_lineno == 0) + /* XXX(nnorwitz): is there a better way to handle this? + for loops are special, we want to be able to trace them + each time around, so we need to set an extra line number. */ + if (d_lineno == 0 && i->i_opcode != FOR_ITER) return 1; if (d_bytecode > 255) { @@ -4131,12 +4162,12 @@ assemble_lnotab(struct assembler *a, struct instr *i) } lnotab = (unsigned char *) PyString_AS_STRING(a->a_lnotab) + a->a_lnotab_off; - *lnotab++ = 255; *lnotab++ = d_bytecode; + *lnotab++ = 255; d_bytecode = 0; for (j = 1; j < ncodes; j++) { - *lnotab++ = 255; *lnotab++ = 0; + *lnotab++ = 255; } d_lineno -= ncodes * 255; a->a_lnotab_off += ncodes * 2; @@ -4173,7 +4204,7 @@ static int assemble_emit(struct assembler *a, struct instr *i) { int size, arg = 0, ext = 0; - int len = PyString_GET_SIZE(a->a_bytecode); + Py_ssize_t len = PyString_GET_SIZE(a->a_bytecode); char *code; size = instrsize(i); @@ -4397,6 +4428,41 @@ makecode(struct compiler *c, struct assembler *a) return co; } + +/* For debugging purposes only */ +#if 0 +static void +dump_instr(const struct instr *i) +{ + const char *jrel = i->i_jrel ? "jrel " : ""; + const char *jabs = i->i_jabs ? "jabs " : ""; + char arg[128]; + + *arg = '\0'; + if (i->i_hasarg) + sprintf(arg, "arg: %d ", i->i_oparg); + + fprintf(stderr, "line: %d, opcode: %d %s%s%s\n", + i->i_lineno, i->i_opcode, arg, jabs, jrel); +} + +static void +dump_basicblock(const basicblock *b) +{ + const char *seen = b->b_seen ? "seen " : ""; + const char *b_return = b->b_return ? "return " : ""; + fprintf(stderr, "used: %d, depth: %d, offset: %d %s%s\n", + b->b_iused, b->b_startdepth, b->b_offset, seen, b_return); + if (b->b_instr) { + int i; + for (i = 0; i < b->b_iused; i++) { + fprintf(stderr, " [%02d] ", i); + dump_instr(b->b_instr + i); + } + } +} +#endif + static PyCodeObject * assemble(struct compiler *c, int addNone) { @@ -4423,6 +4489,13 @@ assemble(struct compiler *c, int addNone) entryblock = b; } + /* Set firstlineno if it wasn't explicitly set. */ + if (!c->u->u_firstlineno) { + if (entryblock && entryblock->b_instr) + c->u->u_firstlineno = entryblock->b_instr->i_lineno; + else + c->u->u_firstlineno = 1; + } if (!assemble_init(&a, nblocks, c->u->u_firstlineno)) goto error; dfs(c, entryblock, &a); diff --git a/Python/dynload_win.c b/Python/dynload_win.c index 36746e2..37d6d2e 100644 --- a/Python/dynload_win.c +++ b/Python/dynload_win.c @@ -2,7 +2,9 @@ /* Support for dynamic loading of extension modules */ #include +#ifdef HAVE_DIRECT_H #include +#endif #include #include "Python.h" diff --git a/Python/errors.c b/Python/errors.c index d99f261..48ed05c 100644 --- a/Python/errors.c +++ b/Python/errors.c @@ -641,7 +641,7 @@ extern PyObject *PyModule_GetWarningsModule(void); /* Function to issue a warning message; may raise an exception. */ int -PyErr_Warn(PyObject *category, char *message) +PyErr_WarnEx(PyObject *category, const char *message, Py_ssize_t stack_level) { PyObject *dict, *func = NULL; PyObject *warnings_module = PyModule_GetWarningsModule(); @@ -659,7 +659,8 @@ PyErr_Warn(PyObject *category, char *message) if (category == NULL) category = PyExc_RuntimeWarning; - res = PyObject_CallFunction(func, "sO", message, category); + res = PyObject_CallFunction(func, "sOn", + message, category, stack_level); if (res == NULL) return -1; Py_DECREF(res); @@ -667,6 +668,16 @@ PyErr_Warn(PyObject *category, char *message) } } +/* PyErr_Warn is only for backwards compatability and will be removed. + Use PyErr_WarnEx instead. */ + +#undef PyErr_Warn + +PyAPI_FUNC(int) +PyErr_Warn(PyObject *category, char *message) +{ + return PyErr_WarnEx(category, message, 1); +} /* Warning with explicit origin */ int diff --git a/Python/future.c b/Python/future.c index d23fad6..d6f11a4 100644 --- a/Python/future.c +++ b/Python/future.c @@ -121,8 +121,10 @@ PyFuture_FromAST(mod_ty mod, const char *filename) PyFutureFeatures *ff; ff = (PyFutureFeatures *)PyObject_Malloc(sizeof(PyFutureFeatures)); - if (ff == NULL) + if (ff == NULL) { + PyErr_NoMemory(); return NULL; + } ff->ff_features = 0; ff->ff_lineno = -1; diff --git a/Python/getargs.c b/Python/getargs.c index 9637618..8dc5e78 100644 --- a/Python/getargs.c +++ b/Python/getargs.c @@ -206,6 +206,9 @@ vgetargs1(PyObject *args, const char *format, va_list *p_va, int flags) if (level == 0) max++; level++; + if (level >= 30) + Py_FatalError("too many tuple nesting levels " + "in argument format string"); break; case ')': if (level == 0) @@ -351,8 +354,8 @@ seterror(int iarg, const char *msg, int *levels, const char *fname, "argument %d", iarg); i = 0; p += strlen(p); - while (levels[i] > 0 && (int)(p-buf) < 220) { - PyOS_snprintf(p, sizeof(buf) - (buf - p), + while (levels[i] > 0 && i < 32 && (int)(p-buf) < 220) { + PyOS_snprintf(p, sizeof(buf) - (p - buf), ", item %d", levels[i]-1); p += strlen(p); i++; @@ -439,6 +442,13 @@ converttuple(PyObject *arg, const char **p_format, va_list *p_va, int flags, char *msg; PyObject *item; item = PySequence_GetItem(arg, i); + if (item == NULL) { + PyErr_Clear(); + levels[0] = i+1; + levels[1] = 0; + strncpy(msgbuf, "is not retrievable", bufsize); + return msgbuf; + } msg = convertitem(item, &format, p_va, flags, levels+1, msgbuf, bufsize, freelist); /* PySequence_GetItem calls tp->sq_item, which INCREFs */ @@ -1508,6 +1518,7 @@ vgetargskeywords(PyObject *args, PyObject *keywords, const char *format, else { msg = skipitem(&format, p_va, flags); if (msg) { + levels[0] = 0; seterror(i+1, msg, levels, fname, message); return cleanreturn(0, freelist); } diff --git a/Python/getopt.c b/Python/getopt.c index 5429fac5..659efcf 100644 --- a/Python/getopt.c +++ b/Python/getopt.c @@ -24,6 +24,9 @@ * davegottner@delphi.com. *---------------------------------------------------------------------------*/ +/* Modified to support --help and --version, as well as /? on Windows + * by Georg Brandl. */ + #include #include @@ -43,8 +46,17 @@ int _PyOS_GetOpt(int argc, char **argv, char *optstring) if (*opt_ptr == '\0') { - if (_PyOS_optind >= argc || argv[_PyOS_optind][0] != '-' || - argv[_PyOS_optind][1] == '\0' /* lone dash */ ) + if (_PyOS_optind >= argc) + return -1; +#ifdef MS_WINDOWS + else if (strcmp(argv[_PyOS_optind], "/?") == 0) { + ++_PyOS_optind; + return 'h'; + } +#endif + + else if (argv[_PyOS_optind][0] != '-' || + argv[_PyOS_optind][1] == '\0' /* lone dash */ ) return -1; else if (strcmp(argv[_PyOS_optind], "--") == 0) { @@ -52,6 +64,17 @@ int _PyOS_GetOpt(int argc, char **argv, char *optstring) return -1; } + else if (strcmp(argv[_PyOS_optind], "--help") == 0) { + ++_PyOS_optind; + return 'h'; + } + + else if (strcmp(argv[_PyOS_optind], "--version") == 0) { + ++_PyOS_optind; + return 'V'; + } + + opt_ptr = &argv[_PyOS_optind++][1]; } @@ -62,7 +85,7 @@ int _PyOS_GetOpt(int argc, char **argv, char *optstring) if (_PyOS_opterr) fprintf(stderr, "Unknown option: -%c\n", option); - return '?'; + return '_'; } if (*(ptr + 1) == ':') { @@ -76,7 +99,7 @@ int _PyOS_GetOpt(int argc, char **argv, char *optstring) if (_PyOS_opterr) fprintf(stderr, "Argument expected for the -%c option\n", option); - return '?'; + return '_'; } _PyOS_optarg = argv[_PyOS_optind++]; diff --git a/Python/import.c b/Python/import.c index c14e8cd..ef37e8b 100644 --- a/Python/import.c +++ b/Python/import.c @@ -60,6 +60,10 @@ extern time_t PyOS_GetLastModificationTime(char *, FILE *); Python 2.5a0: 62081 (ast-branch) Python 2.5a0: 62091 (with) Python 2.5a0: 62092 (changed WITH_CLEANUP opcode) + Python 2.5b3: 62101 (fix wrong code: for x, in ...) + Python 2.5b3: 62111 (fix wrong code: x += yield) + Python 2.5c1: 62121 (fix wrong lnotab with for loops and + storing constants that should have been removed) Python 3000: 3000 . */ @@ -98,6 +102,8 @@ static const struct filedescr _PyImport_StandardFiletab[] = { }; #endif +static PyTypeObject NullImporterType; /* Forward reference */ + /* Initialize things */ void @@ -116,6 +122,8 @@ _PyImport_Init(void) for (scan = _PyImport_StandardFiletab; scan->suffix != NULL; ++scan) ++countS; filetab = PyMem_NEW(struct filedescr, countD + countS + 1); + if (filetab == NULL) + Py_FatalError("Can't initialize import file table."); memcpy(filetab, _PyImport_DynLoadFiletab, countD * sizeof(struct filedescr)); memcpy(filetab + countD, _PyImport_StandardFiletab, @@ -153,6 +161,8 @@ _PyImportHooks_Init(void) /* adding sys.path_hooks and sys.path_importer_cache, setting up zipimport */ + if (PyType_Ready(&NullImporterType) < 0) + goto error; if (Py_VerboseFlag) PySys_WriteStderr("# installing zipimport hook\n"); @@ -178,9 +188,11 @@ _PyImportHooks_Init(void) if (err) { error: PyErr_Print(); - Py_FatalError("initializing sys.meta_path, sys.path_hooks or " - "path_importer_cache failed"); + Py_FatalError("initializing sys.meta_path, sys.path_hooks, " + "path_importer_cache, or NullImporter failed" + ); } + zimpimport = PyImport_ImportModule("zipimport"); if (zimpimport == NULL) { PyErr_Clear(); /* No zip import module -- okay */ @@ -239,8 +251,11 @@ lock_import(void) long me = PyThread_get_thread_ident(); if (me == -1) return; /* Too bad */ - if (import_lock == NULL) + if (import_lock == NULL) { import_lock = PyThread_allocate_lock(); + if (import_lock == NULL) + return; /* Nothing much we can do. */ + } if (import_lock_thread == me) { import_lock_level++; return; @@ -259,7 +274,7 @@ static int unlock_import(void) { long me = PyThread_get_thread_ident(); - if (me == -1) + if (me == -1 || import_lock == NULL) return 0; /* Too bad */ if (import_lock_thread != me) return -1; @@ -1053,9 +1068,18 @@ get_path_importer(PyObject *path_importer_cache, PyObject *path_hooks, } PyErr_Clear(); } - if (importer == NULL) - importer = Py_None; - else if (importer != Py_None) { + if (importer == NULL) { + importer = PyObject_CallFunctionObjArgs( + (PyObject *)&NullImporterType, p, NULL + ); + if (importer == NULL) { + if (PyErr_ExceptionMatches(PyExc_ImportError)) { + PyErr_Clear(); + return Py_None; + } + } + } + if (importer != NULL) { int err = PyDict_SetItem(path_importer_cache, p, importer); Py_DECREF(importer); if (err != 0) @@ -1238,42 +1262,17 @@ find_module(char *fullname, char *subname, PyObject *path, char *buf, importer = get_path_importer(path_importer_cache, path_hooks, v); - if (importer == NULL) - return NULL; - /* Note: importer is a borrowed reference */ - if (importer == Py_False) { - /* Cached as not being a valid dir. */ + if (importer == NULL) { Py_XDECREF(copy); - continue; - } - else if (importer == Py_True) { - /* Cached as being a valid dir, so just - * continue below. */ - } - else if (importer == Py_None) { - /* No importer was found, so it has to be a file. - * Check if the directory is valid. - * Note that the empty string is a valid path, but - * not stat'able, hence the check for len. */ -#ifdef HAVE_STAT - if (len && stat(buf, &statbuf) != 0) { - /* Directory does not exist. */ - PyDict_SetItem(path_importer_cache, - v, Py_False); - Py_XDECREF(copy); - continue; - } else { - PyDict_SetItem(path_importer_cache, - v, Py_True); - } -#endif + return NULL; } - else { - /* A real import hook importer was found. */ + /* Note: importer is a borrowed reference */ + if (importer != Py_None) { PyObject *loader; loader = PyObject_CallMethod(importer, "find_module", "s", fullname); + Py_XDECREF(copy); if (loader == NULL) return NULL; /* error */ if (loader != Py_None) { @@ -1282,7 +1281,6 @@ find_module(char *fullname, char *subname, PyObject *path, char *buf, return &importhookdescr; } Py_DECREF(loader); - Py_XDECREF(copy); continue; } } @@ -1904,11 +1902,10 @@ PyImport_ImportFrozenModule(char *name) if (co == NULL) return -1; if (!PyCode_Check(co)) { - Py_DECREF(co); PyErr_Format(PyExc_TypeError, "frozen object %.200s is not a code object", name); - return -1; + goto err_return; } if (ispackage) { /* Set __path__ to the package name */ @@ -1916,22 +1913,25 @@ PyImport_ImportFrozenModule(char *name) int err; m = PyImport_AddModule(name); if (m == NULL) - return -1; + goto err_return; d = PyModule_GetDict(m); s = PyString_InternFromString(name); if (s == NULL) - return -1; + goto err_return; err = PyDict_SetItemString(d, "__path__", s); Py_DECREF(s); if (err != 0) - return err; + goto err_return; } m = PyImport_ExecCodeModuleEx(name, co, ""); - Py_DECREF(co); if (m == NULL) - return -1; + goto err_return; + Py_DECREF(co); Py_DECREF(m); return 1; +err_return: + Py_DECREF(co); + return -1; } @@ -2926,11 +2926,120 @@ setint(PyObject *d, char *name, int value) return err; } +typedef struct { + PyObject_HEAD +} NullImporter; + +static int +NullImporter_init(NullImporter *self, PyObject *args, PyObject *kwds) +{ + char *path; + + if (!_PyArg_NoKeywords("NullImporter()", kwds)) + return -1; + + if (!PyArg_ParseTuple(args, "s:NullImporter", + &path)) + return -1; + + if (strlen(path) == 0) { + PyErr_SetString(PyExc_ImportError, "empty pathname"); + return -1; + } else { +#ifndef RISCOS + struct stat statbuf; + int rv; + + rv = stat(path, &statbuf); + if (rv == 0) { + /* it exists */ + if (S_ISDIR(statbuf.st_mode)) { + /* it's a directory */ + PyErr_SetString(PyExc_ImportError, + "existing directory"); + return -1; + } + } +#else + if (object_exists(path)) { + /* it exists */ + if (isdir(path)) { + /* it's a directory */ + PyErr_SetString(PyExc_ImportError, + "existing directory"); + return -1; + } + } +#endif + } + return 0; +} + +static PyObject * +NullImporter_find_module(NullImporter *self, PyObject *args) +{ + Py_RETURN_NONE; +} + +static PyMethodDef NullImporter_methods[] = { + {"find_module", (PyCFunction)NullImporter_find_module, METH_VARARGS, + "Always return None" + }, + {NULL} /* Sentinel */ +}; + + +static PyTypeObject NullImporterType = { + PyObject_HEAD_INIT(NULL) + 0, /*ob_size*/ + "imp.NullImporter", /*tp_name*/ + sizeof(NullImporter), /*tp_basicsize*/ + 0, /*tp_itemsize*/ + 0, /*tp_dealloc*/ + 0, /*tp_print*/ + 0, /*tp_getattr*/ + 0, /*tp_setattr*/ + 0, /*tp_compare*/ + 0, /*tp_repr*/ + 0, /*tp_as_number*/ + 0, /*tp_as_sequence*/ + 0, /*tp_as_mapping*/ + 0, /*tp_hash */ + 0, /*tp_call*/ + 0, /*tp_str*/ + 0, /*tp_getattro*/ + 0, /*tp_setattro*/ + 0, /*tp_as_buffer*/ + Py_TPFLAGS_DEFAULT, /*tp_flags*/ + "Null importer object", /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + NullImporter_methods, /* tp_methods */ + 0, /* tp_members */ + 0, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + (initproc)NullImporter_init, /* tp_init */ + 0, /* tp_alloc */ + PyType_GenericNew /* tp_new */ +}; + + PyMODINIT_FUNC initimp(void) { PyObject *m, *d; + if (PyType_Ready(&NullImporterType) < 0) + goto failure; + m = Py_InitModule4("imp", imp_methods, doc_imp, NULL, PYTHON_API_VERSION); if (m == NULL) @@ -2948,6 +3057,8 @@ initimp(void) if (setint(d, "PY_CODERESOURCE", PY_CODERESOURCE) < 0) goto failure; if (setint(d, "IMP_HOOK", IMP_HOOK) < 0) goto failure; + Py_INCREF(&NullImporterType); + PyModule_AddObject(m, "NullImporter", (PyObject *)&NullImporterType); failure: ; } diff --git a/Python/mactoolboxglue.c b/Python/mactoolboxglue.c index 0aa2cfd..26a1308 100644 --- a/Python/mactoolboxglue.c +++ b/Python/mactoolboxglue.c @@ -60,8 +60,9 @@ PyMac_StrError(int err) strncpy(buf, input, sizeof(buf) - 1); buf[sizeof(buf) - 1] = '\0'; } + Py_DECREF(rv); } - + Py_XDECREF(m); return buf; } diff --git a/Python/mystrtoul.c b/Python/mystrtoul.c index 380b37d..0dda4be 100644 --- a/Python/mystrtoul.c +++ b/Python/mystrtoul.c @@ -17,7 +17,7 @@ #include -#ifndef DONT_HAVE_ERRNO_H +#ifdef HAVE_ERRNO_H #include #endif @@ -69,11 +69,22 @@ static unsigned long smallmax[] = { * calculated by [int(math.floor(math.log(2**32, i))) for i in range(2, 37)]. * Note that this is pessimistic if sizeof(long) > 4. */ +#if SIZEOF_LONG == 4 static int digitlimit[] = { 0, 0, 32, 20, 16, 13, 12, 11, 10, 10, /* 0 - 9 */ 9, 9, 8, 8, 8, 8, 8, 7, 7, 7, /* 10 - 19 */ 7, 7, 7, 7, 6, 6, 6, 6, 6, 6, /* 20 - 29 */ 6, 6, 6, 6, 6, 6, 6}; /* 30 - 36 */ +#elif SIZEOF_LONG == 8 +/* [int(math.floor(math.log(2**64, i))) for i in range(2, 37)] */ +static int digitlimit[] = { + 0, 0, 64, 40, 32, 27, 24, 22, 21, 20, /* 0 - 9 */ + 19, 18, 17, 17, 16, 16, 16, 15, 15, 15, /* 10 - 19 */ + 14, 14, 14, 14, 13, 13, 13, 13, 13, 13, /* 20 - 29 */ + 13, 12, 12, 12, 12, 12, 12}; /* 30 - 36 */ +#else +#error "Need table for SIZEOF_LONG" +#endif /* ** strtoul @@ -184,10 +195,19 @@ overflowed: return (unsigned long)-1; } +/* Checking for overflow in PyOS_strtol is a PITA since C doesn't define + * anything about what happens when a signed integer operation overflows, + * and some compilers think they're doing you a favor by being "clever" + * then. Python assumes a 2's-complement representation, so that the bit + * pattern for the largest postive signed long is LONG_MAX, and for + * the smallest negative signed long is LONG_MAX + 1. + */ + long PyOS_strtol(char *str, char **ptr, int base) { long result; + unsigned long uresult; char sign; while (*str && isspace(Py_CHARMASK(*str))) @@ -197,17 +217,20 @@ PyOS_strtol(char *str, char **ptr, int base) if (sign == '+' || sign == '-') str++; - result = (long) PyOS_strtoul(str, ptr, base); + uresult = PyOS_strtoul(str, ptr, base); - /* Signal overflow if the result appears negative, - except for the largest negative integer */ - if (result < 0 && !(sign == '-' && result == -result)) { + if (uresult <= (unsigned long)LONG_MAX) { + result = (long)uresult; + if (sign == '-') + result = -result; + } + else if (sign == '-' && uresult == (unsigned long)LONG_MAX + 1) { + assert(LONG_MIN == -LONG_MAX-1); + result = LONG_MIN; + } + else { errno = ERANGE; - result = 0x7fffffff; + result = LONG_MAX; } - - if (sign == '-') - result = -result; - return result; } diff --git a/Python/pyarena.c b/Python/pyarena.c index f11a905..f4cc474 100644 --- a/Python/pyarena.c +++ b/Python/pyarena.c @@ -132,19 +132,19 @@ PyArena_New() { PyArena* arena = (PyArena *)malloc(sizeof(PyArena)); if (!arena) - return NULL; + return (PyArena*)PyErr_NoMemory(); arena->a_head = block_new(DEFAULT_BLOCK_SIZE); arena->a_cur = arena->a_head; if (!arena->a_head) { free((void *)arena); - return NULL; + return (PyArena*)PyErr_NoMemory(); } arena->a_objects = PyList_New(0); if (!arena->a_objects) { block_free(arena->a_head); free((void *)arena); - return NULL; + return (PyArena*)PyErr_NoMemory(); } #if defined(Py_DEBUG) arena->total_allocs = 0; @@ -191,7 +191,7 @@ PyArena_Malloc(PyArena *arena, size_t size) { void *p = block_alloc(arena->a_cur, size); if (!p) - return NULL; + return PyErr_NoMemory(); #if defined(Py_DEBUG) arena->total_allocs++; arena->total_size += size; diff --git a/Python/pystate.c b/Python/pystate.c index b8f460f..3fae85b 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -63,6 +63,10 @@ PyInterpreterState_New(void) if (interp != NULL) { HEAD_INIT(); +#ifdef WITH_THREAD + if (head_mutex == NULL) + Py_FatalError("Can't initialize threads for interpreter"); +#endif interp->modules = NULL; interp->sysdict = NULL; interp->builtins = NULL; @@ -387,6 +391,53 @@ PyThreadState_Next(PyThreadState *tstate) { return tstate->next; } +/* The implementation of sys._current_frames(). This is intended to be + called with the GIL held, as it will be when called via + sys._current_frames(). It's possible it would work fine even without + the GIL held, but haven't thought enough about that. +*/ +PyObject * +_PyThread_CurrentFrames(void) +{ + PyObject *result; + PyInterpreterState *i; + + result = PyDict_New(); + if (result == NULL) + return NULL; + + /* for i in all interpreters: + * for t in all of i's thread states: + * if t's frame isn't NULL, map t's id to its frame + * Because these lists can mutute even when the GIL is held, we + * need to grab head_mutex for the duration. + */ + HEAD_LOCK(); + for (i = interp_head; i != NULL; i = i->next) { + PyThreadState *t; + for (t = i->tstate_head; t != NULL; t = t->next) { + PyObject *id; + int stat; + struct _frame *frame = t->frame; + if (frame == NULL) + continue; + id = PyInt_FromLong(t->thread_id); + if (id == NULL) + goto Fail; + stat = PyDict_SetItem(result, id, (PyObject *)frame); + Py_DECREF(id); + if (stat < 0) + goto Fail; + } + } + HEAD_UNLOCK(); + return result; + + Fail: + HEAD_UNLOCK(); + Py_DECREF(result); + return NULL; +} /* Python "auto thread state" API. */ #ifdef WITH_THREAD @@ -444,15 +495,15 @@ _PyGILState_NoteThreadState(PyThreadState* tstate) /* If autoTLSkey is 0, this must be the very first threadstate created in Py_Initialize(). Don't do anything for now (we'll be back here when _PyGILState_Init is called). */ - if (!autoTLSkey) + if (!autoTLSkey) return; - + /* Stick the thread state for this thread in thread local storage. The only situation where you can legitimately have more than one thread state for an OS level thread is when there are multiple interpreters, when: - + a) You shouldn't really be using the PyGILState_ APIs anyway, and: diff --git a/Python/pystrtod.c b/Python/pystrtod.c index e1c84ea..6c19b45 100644 --- a/Python/pystrtod.c +++ b/Python/pystrtod.c @@ -90,6 +90,13 @@ PyOS_ascii_strtod(const char *nptr, char **endptr) p++; end = p; } + else if (strncmp(p, decimal_point, decimal_point_len) == 0) + { + /* Python bug #1417699 */ + *endptr = (char*)nptr; + errno = EINVAL; + return val; + } /* For the other cases, we need not convert the decimal point */ } diff --git a/Python/pythonrun.c b/Python/pythonrun.c index 2dbcf75..555c39d 100644 --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -17,7 +17,9 @@ #include "eval.h" #include "marshal.h" +#ifdef HAVE_SIGNAL_H #include +#endif #ifdef HAVE_LANGINFO_H #include @@ -738,6 +740,11 @@ PyRun_InteractiveOneFlags(FILE *fp, const char *filename, PyCompilerFlags *flags ps2 = PyString_AsString(w); } arena = PyArena_New(); + if (arena == NULL) { + Py_XDECREF(v); + Py_XDECREF(w); + return -1; + } mod = PyParser_ASTFromFile(fp, filename, Py_single_input, ps1, ps2, flags, &errcode, arena); @@ -1056,6 +1063,17 @@ PyErr_PrintEx(int set_sys_last_vars) } PyErr_Fetch(&exception2, &v2, &tb2); PyErr_NormalizeException(&exception2, &v2, &tb2); + /* It should not be possible for exception2 or v2 + to be NULL. However PyErr_Display() can't + tolerate NULLs, so just be safe. */ + if (exception2 == NULL) { + exception2 = Py_None; + Py_INCREF(exception2); + } + if (v2 == NULL) { + v2 = Py_None; + Py_INCREF(v2); + } if (Py_FlushLine()) PyErr_Clear(); fflush(stdout); @@ -1063,8 +1081,8 @@ PyErr_PrintEx(int set_sys_last_vars) PyErr_Display(exception2, v2, tb2); PySys_WriteStderr("\nOriginal exception was:\n"); PyErr_Display(exception, v, tb); - Py_XDECREF(exception2); - Py_XDECREF(v2); + Py_DECREF(exception2); + Py_DECREF(v2); Py_XDECREF(tb2); } Py_XDECREF(result); @@ -1184,9 +1202,12 @@ PyRun_StringFlags(const char *str, int start, PyObject *globals, PyObject *locals, PyCompilerFlags *flags) { PyObject *ret = NULL; + mod_ty mod; PyArena *arena = PyArena_New(); - mod_ty mod = PyParser_ASTFromString(str, "", start, flags, - arena); + if (arena == NULL) + return NULL; + + mod = PyParser_ASTFromString(str, "", start, flags, arena); if (mod != NULL) ret = run_mod(mod, "", globals, locals, flags, arena); PyArena_Free(arena); @@ -1198,9 +1219,13 @@ PyRun_FileExFlags(FILE *fp, const char *filename, int start, PyObject *globals, PyObject *locals, int closeit, PyCompilerFlags *flags) { PyObject *ret; + mod_ty mod; PyArena *arena = PyArena_New(); - mod_ty mod = PyParser_ASTFromFile(fp, filename, start, 0, 0, - flags, NULL, arena); + if (arena == NULL) + return NULL; + + mod = PyParser_ASTFromFile(fp, filename, start, 0, 0, + flags, NULL, arena); if (mod == NULL) { PyArena_Free(arena); return NULL; @@ -1263,8 +1288,12 @@ Py_CompileStringFlags(const char *str, const char *filename, int start, PyCompilerFlags *flags) { PyCodeObject *co; + mod_ty mod; PyArena *arena = PyArena_New(); - mod_ty mod = PyParser_ASTFromString(str, filename, start, flags, arena); + if (arena == NULL) + return NULL; + + mod = PyParser_ASTFromString(str, filename, start, flags, arena); if (mod == NULL) { PyArena_Free(arena); return NULL; @@ -1283,8 +1312,12 @@ struct symtable * Py_SymtableString(const char *str, const char *filename, int start) { struct symtable *st; + mod_ty mod; PyArena *arena = PyArena_New(); - mod_ty mod = PyParser_ASTFromString(str, filename, start, NULL, arena); + if (arena == NULL) + return NULL; + + mod = PyParser_ASTFromString(str, filename, start, NULL, arena); if (mod == NULL) { PyArena_Free(arena); return NULL; diff --git a/Python/strtod.c b/Python/strtod.c index 7911a94..5c084a4 100644 --- a/Python/strtod.c +++ b/Python/strtod.c @@ -54,7 +54,7 @@ static double HUGE = 1.7976931348623157e308; extern double atof(const char *); /* Only called when result known to be ok */ -#ifndef DONT_HAVE_ERRNO_H +#ifdef HAVE_ERRNO_H #include #endif extern int errno; diff --git a/Python/symtable.c b/Python/symtable.c index 1dc2a2e..439a243 100644 --- a/Python/symtable.c +++ b/Python/symtable.c @@ -221,8 +221,12 @@ PySymtable_Build(mod_ty mod, const char *filename, PyFutureFeatures *future) return st; st->st_filename = filename; st->st_future = future; - symtable_enter_block(st, GET_IDENTIFIER(top), ModuleBlock, - (void *)mod, 0); + if (!symtable_enter_block(st, GET_IDENTIFIER(top), ModuleBlock, + (void *)mod, 0)) { + PySymtable_Free(st); + return NULL; + } + st->st_top = st->st_cur; st->st_cur->ste_unoptimized = OPT_TOPLEVEL; /* Any other top-level initialization? */ @@ -525,6 +529,8 @@ update_symbols(PyObject *symbols, PyObject *scope, i = PyInt_AS_LONG(w); flags |= (i << SCOPE_OFF); u = PyInt_FromLong(flags); + if (!u) + return 0; if (PyDict_SetItem(symbols, name, u) < 0) { Py_DECREF(u); return 0; @@ -723,11 +729,13 @@ symtable_exit_block(struct symtable *st, void *ast) { Py_ssize_t end; - Py_DECREF(st->st_cur); + Py_CLEAR(st->st_cur); end = PyList_GET_SIZE(st->st_stack) - 1; if (end >= 0) { st->st_cur = (PySTEntryObject *)PyList_GET_ITEM(st->st_stack, end); + if (st->st_cur == NULL) + return 0; Py_INCREF(st->st_cur); if (PySequence_DelItem(st->st_stack, end) < 0) return 0; @@ -749,6 +757,8 @@ symtable_enter_block(struct symtable *st, identifier name, _Py_block_ty block, Py_DECREF(st->st_cur); } st->st_cur = PySTEntry_New(st, name, block, ast, lineno); + if (st->st_cur == NULL) + return 0; if (name == GET_IDENTIFIER(top)) st->st_global = st->st_cur->ste_symbols; if (prev) { diff --git a/Python/sysmodule.c b/Python/sysmodule.c index fe47fd1..a2df669 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -656,6 +656,21 @@ sys_getframe(PyObject *self, PyObject *args) return (PyObject*)f; } +PyDoc_STRVAR(current_frames_doc, +"_current_frames() -> dictionary\n\ +\n\ +Return a dictionary mapping each current thread T's thread id to T's\n\ +current stack frame.\n\ +\n\ +This function should be used for specialized purposes only." +); + +static PyObject * +sys_current_frames(PyObject *self, PyObject *noargs) +{ + return _PyThread_CurrentFrames(); +} + PyDoc_STRVAR(call_tracing_doc, "call_tracing(func, args) -> object\n\ \n\ @@ -718,6 +733,8 @@ static PyMethodDef sys_methods[] = { /* Might as well keep this in alphabetic order */ {"callstats", (PyCFunction)PyEval_GetCallStats, METH_NOARGS, callstats_doc}, + {"_current_frames", sys_current_frames, METH_NOARGS, + current_frames_doc}, {"displayhook", sys_displayhook, METH_O, displayhook_doc}, {"exc_info", sys_exc_info, METH_NOARGS, exc_info_doc}, {"exc_clear", sys_exc_clear, METH_NOARGS, exc_clear_doc}, @@ -1116,41 +1133,38 @@ _PySys_Init(void) #elif PY_RELEASE_LEVEL == PY_RELEASE_LEVEL_FINAL s = "final"; #endif - PyDict_SetItemString(sysdict, "version_info", - v = Py_BuildValue("iiisi", PY_MAJOR_VERSION, + +#define SET_SYS_FROM_STRING(key, value) \ + v = value; \ + if (v != NULL) \ + PyDict_SetItemString(sysdict, key, v); \ + Py_XDECREF(v) + + SET_SYS_FROM_STRING("version_info", + Py_BuildValue("iiisi", PY_MAJOR_VERSION, PY_MINOR_VERSION, PY_MICRO_VERSION, s, PY_RELEASE_SERIAL)); - Py_XDECREF(v); - PyDict_SetItemString(sysdict, "api_version", - v = PyInt_FromLong(PYTHON_API_VERSION)); - Py_XDECREF(v); - PyDict_SetItemString(sysdict, "copyright", - v = PyString_FromString(Py_GetCopyright())); - Py_XDECREF(v); - PyDict_SetItemString(sysdict, "platform", - v = PyString_FromString(Py_GetPlatform())); - Py_XDECREF(v); - PyDict_SetItemString(sysdict, "executable", - v = PyString_FromString(Py_GetProgramFullPath())); - Py_XDECREF(v); - PyDict_SetItemString(sysdict, "prefix", - v = PyString_FromString(Py_GetPrefix())); - Py_XDECREF(v); - PyDict_SetItemString(sysdict, "exec_prefix", - v = PyString_FromString(Py_GetExecPrefix())); - Py_XDECREF(v); - PyDict_SetItemString(sysdict, "maxint", - v = PyInt_FromLong(PyInt_GetMax())); - Py_XDECREF(v); + SET_SYS_FROM_STRING("api_version", + PyInt_FromLong(PYTHON_API_VERSION)); + SET_SYS_FROM_STRING("copyright", + PyString_FromString(Py_GetCopyright())); + SET_SYS_FROM_STRING("platform", + PyString_FromString(Py_GetPlatform())); + SET_SYS_FROM_STRING("executable", + PyString_FromString(Py_GetProgramFullPath())); + SET_SYS_FROM_STRING("prefix", + PyString_FromString(Py_GetPrefix())); + SET_SYS_FROM_STRING("exec_prefix", + PyString_FromString(Py_GetExecPrefix())); + SET_SYS_FROM_STRING("maxint", + PyInt_FromLong(PyInt_GetMax())); #ifdef Py_USING_UNICODE - PyDict_SetItemString(sysdict, "maxunicode", - v = PyInt_FromLong(PyUnicode_GetMax())); - Py_XDECREF(v); + SET_SYS_FROM_STRING("maxunicode", + PyInt_FromLong(PyUnicode_GetMax())); #endif - PyDict_SetItemString(sysdict, "builtin_module_names", - v = list_builtin_module_names()); - Py_XDECREF(v); + SET_SYS_FROM_STRING("builtin_module_names", + list_builtin_module_names()); { /* Assumes that longs are at least 2 bytes long. Should be safe! */ @@ -1162,18 +1176,16 @@ _PySys_Init(void) value = "big"; else value = "little"; - PyDict_SetItemString(sysdict, "byteorder", - v = PyString_FromString(value)); - Py_XDECREF(v); + SET_SYS_FROM_STRING("byteorder", + PyString_FromString(value)); } #ifdef MS_COREDLL - PyDict_SetItemString(sysdict, "dllhandle", - v = PyLong_FromVoidPtr(PyWin_DLLhModule)); - Py_XDECREF(v); - PyDict_SetItemString(sysdict, "winver", - v = PyString_FromString(PyWin_DLLVersionString)); - Py_XDECREF(v); + SET_SYS_FROM_STRING("dllhandle", + PyLong_FromVoidPtr(PyWin_DLLhModule)); + SET_SYS_FROM_STRING("winver", + PyString_FromString(PyWin_DLLVersionString)); #endif +#undef SET_SYS_FROM_STRING if (warnoptions == NULL) { warnoptions = PyList_New(0); } diff --git a/Python/thread.c b/Python/thread.c index c9356dc..3a2c7af 100644 --- a/Python/thread.c +++ b/Python/thread.c @@ -95,6 +95,11 @@ PyThread_init_thread(void) PyThread__init_thread(); } +/* Support for runtime thread stack size tuning. + A value of 0 means using the platform's default stack size + or the size specified by the THREAD_STACK_SIZE macro. */ +static size_t _pythread_stacksize = 0; + #ifdef SGI_THREADS #include "thread_sgi.h" #endif @@ -150,6 +155,28 @@ PyThread_init_thread(void) #endif */ +/* return the current thread stack size */ +size_t +PyThread_get_stacksize(void) +{ + return _pythread_stacksize; +} + +/* Only platforms defining a THREAD_SET_STACKSIZE() macro + in thread_.h support changing the stack size. + Return 0 if stack size is valid, + -1 if stack size value is invalid, + -2 if setting stack size is not supported. */ +int +PyThread_set_stacksize(size_t size) +{ +#if defined(THREAD_SET_STACKSIZE) + return THREAD_SET_STACKSIZE(size); +#else + return -2; +#endif +} + #ifndef Py_HAVE_NATIVE_TLS /* If the platform has not supplied a platform specific TLS implementation, provide our own. @@ -240,6 +267,8 @@ find_key(int key, void *value) struct key *p; long id = PyThread_get_thread_ident(); + if (!keymutex) + return NULL; PyThread_acquire_lock(keymutex, 1); for (p = keyhead; p != NULL; p = p->next) { if (p->id == id && p->key == key) diff --git a/Python/thread_nt.h b/Python/thread_nt.h index 0b7e84e..67f5ed5 100644 --- a/Python/thread_nt.h +++ b/Python/thread_nt.h @@ -5,7 +5,9 @@ #include #include +#ifdef HAVE_PROCESS_H #include +#endif typedef struct NRMUTEX { LONG owned ; @@ -194,7 +196,7 @@ PyThread_start_new_thread(void (*func)(void *), void *arg) if (obj.done == NULL) return -1; - rv = _beginthread(bootstrap, 0, &obj); /* use default stack size */ + rv = _beginthread(bootstrap, _pythread_stacksize, &obj); if (rv == (Py_uintptr_t)-1) { /* I've seen errno == EAGAIN here, which means "there are * too many threads". @@ -333,3 +335,30 @@ PyThread_release_lock(PyThread_type_lock aLock) if (!(aLock && LeaveNonRecursiveMutex((PNRMUTEX) aLock))) dprintf(("%ld: Could not PyThread_release_lock(%p) error: %l\n", PyThread_get_thread_ident(), aLock, GetLastError())); } + +/* minimum/maximum thread stack sizes supported */ +#define THREAD_MIN_STACKSIZE 0x8000 /* 32kB */ +#define THREAD_MAX_STACKSIZE 0x10000000 /* 256MB */ + +/* set the thread stack size. + * Return 0 if size is valid, -1 otherwise. + */ +static int +_pythread_nt_set_stacksize(size_t size) +{ + /* set to default */ + if (size == 0) { + _pythread_stacksize = 0; + return 0; + } + + /* valid range? */ + if (size >= THREAD_MIN_STACKSIZE && size < THREAD_MAX_STACKSIZE) { + _pythread_stacksize = size; + return 0; + } + + return -1; +} + +#define THREAD_SET_STACKSIZE(x) _pythread_nt_set_stacksize(x) diff --git a/Python/thread_os2.h b/Python/thread_os2.h index 86e91c1..12eeed5 100644 --- a/Python/thread_os2.h +++ b/Python/thread_os2.h @@ -14,10 +14,13 @@ long PyThread_get_thread_ident(void); #endif +/* default thread stack size of 64kB */ #if !defined(THREAD_STACK_SIZE) #define THREAD_STACK_SIZE 0x10000 #endif +#define OS2_STACKSIZE(x) (x ? x : THREAD_STACK_SIZE) + /* * Initialization of the C package, should not be needed. */ @@ -32,18 +35,18 @@ PyThread__init_thread(void) long PyThread_start_new_thread(void (*func)(void *), void *arg) { - int aThread; - int success = 0; + int thread_id; - aThread = _beginthread(func, NULL, THREAD_STACK_SIZE, arg); + thread_id = _beginthread(func, + NULL, + OS2_STACKSIZE(_pythread_stacksize), + arg); - if (aThread == -1) { - success = -1; - fprintf(stderr, "aThread failed == %d", aThread); + if (thread_id == -1) { dprintf(("_beginthread failed. return %ld\n", errno)); } - return success; + return thread_id; } long @@ -275,3 +278,30 @@ PyThread_release_lock(PyThread_type_lock aLock) DosExitCritSec(); #endif } + +/* minimum/maximum thread stack sizes supported */ +#define THREAD_MIN_STACKSIZE 0x8000 /* 32kB */ +#define THREAD_MAX_STACKSIZE 0x2000000 /* 32MB */ + +/* set the thread stack size. + * Return 0 if size is valid, -1 otherwise. + */ +static int +_pythread_os2_set_stacksize(size_t size) +{ + /* set to default */ + if (size == 0) { + _pythread_stacksize = 0; + return 0; + } + + /* valid range? */ + if (size >= THREAD_MIN_STACKSIZE && size < THREAD_MAX_STACKSIZE) { + _pythread_stacksize = size; + return 0; + } + + return -1; +} + +#define THREAD_SET_STACKSIZE(x) _pythread_os2_set_stacksize(x) diff --git a/Python/thread_pthread.h b/Python/thread_pthread.h index c29a61c..60d2fb2 100644 --- a/Python/thread_pthread.h +++ b/Python/thread_pthread.h @@ -12,6 +12,20 @@ #endif #include +/* The POSIX spec requires that use of pthread_attr_setstacksize + be conditional on _POSIX_THREAD_ATTR_STACKSIZE being defined. */ +#ifdef _POSIX_THREAD_ATTR_STACKSIZE +#ifndef THREAD_STACK_SIZE +#define THREAD_STACK_SIZE 0 /* use default stack size */ +#endif +/* for safety, ensure a viable minimum stacksize */ +#define THREAD_STACK_MIN 0x8000 /* 32kB */ +#else /* !_POSIX_THREAD_ATTR_STACKSIZE */ +#ifdef THREAD_STACK_SIZE +#error "THREAD_STACK_SIZE defined but _POSIX_THREAD_ATTR_STACKSIZE undefined" +#endif +#endif + /* The POSIX spec says that implementations supporting the sem_* family of functions must indicate this by defining _POSIX_SEMAPHORES. */ @@ -138,15 +152,27 @@ PyThread_start_new_thread(void (*func)(void *), void *arg) #if defined(THREAD_STACK_SIZE) || defined(PTHREAD_SYSTEM_SCHED_SUPPORTED) pthread_attr_t attrs; #endif +#if defined(THREAD_STACK_SIZE) + size_t tss; +#endif + dprintf(("PyThread_start_new_thread called\n")); if (!initialized) PyThread_init_thread(); #if defined(THREAD_STACK_SIZE) || defined(PTHREAD_SYSTEM_SCHED_SUPPORTED) - pthread_attr_init(&attrs); + if (pthread_attr_init(&attrs) != 0) + return -1; #endif -#ifdef THREAD_STACK_SIZE - pthread_attr_setstacksize(&attrs, THREAD_STACK_SIZE); +#if defined(THREAD_STACK_SIZE) + tss = (_pythread_stacksize != 0) ? _pythread_stacksize + : THREAD_STACK_SIZE; + if (tss != 0) { + if (pthread_attr_setstacksize(&attrs, tss) != 0) { + pthread_attr_destroy(&attrs); + return -1; + } + } #endif #if defined(PTHREAD_SYSTEM_SCHED_SUPPORTED) pthread_attr_setscope(&attrs, PTHREAD_SCOPE_SYSTEM); @@ -460,3 +486,48 @@ PyThread_release_lock(PyThread_type_lock lock) } #endif /* USE_SEMAPHORES */ + +/* set the thread stack size. + * Return 0 if size is valid, -1 if size is invalid, + * -2 if setting stack size is not supported. + */ +static int +_pythread_pthread_set_stacksize(size_t size) +{ +#if defined(THREAD_STACK_SIZE) + pthread_attr_t attrs; + size_t tss_min; + int rc = 0; +#endif + + /* set to default */ + if (size == 0) { + _pythread_stacksize = 0; + return 0; + } + +#if defined(THREAD_STACK_SIZE) +#if defined(PTHREAD_STACK_MIN) + tss_min = PTHREAD_STACK_MIN > THREAD_STACK_MIN ? PTHREAD_STACK_MIN + : THREAD_STACK_MIN; +#else + tss_min = THREAD_STACK_MIN; +#endif + if (size >= tss_min) { + /* validate stack size by setting thread attribute */ + if (pthread_attr_init(&attrs) == 0) { + rc = pthread_attr_setstacksize(&attrs, size); + pthread_attr_destroy(&attrs); + if (rc == 0) { + _pythread_stacksize = size; + return 0; + } + } + } + return -1; +#else + return -2; +#endif +} + +#define THREAD_SET_STACKSIZE(x) _pythread_pthread_set_stacksize(x) diff --git a/README b/README index b3f72e7..72a10cd 100644 --- a/README +++ b/README @@ -226,13 +226,13 @@ compilers from the vendor, or one of the free compilers (gcc). Unsupported systems ------------------- -XXX This section is out of date! - -A number of features are not supported in Python 2.3 anymore. Some -support code is still present, but will be removed in Python 2.4. +A number of features are not supported in Python 2.5 anymore. Some +support code is still present, but will be removed in Python 2.6. If you still need to use current Python versions on these systems, please send a message to python-dev@python.org indicating that you -volunteer to support this system. +volunteer to support this system. For a more detailed discussion +regarding no-longer-supported and resupporting platforms, as well +as a list of platforms that became or will be unsupported, see PEP 11. More specifically, the following systems are not supported any longer: @@ -240,6 +240,7 @@ longer: - DYNIX - dgux - Minix +- NeXT - Irix 4 and --with-sgi-dl - Linux 1 - Systems defining __d6_pthread_create (configure.in) @@ -247,6 +248,25 @@ longer: or PY_PTHREAD_D7 in thread_pthread.h - Systems using --with-dl-dld - Systems using --without-universal-newlines +- MacOS 9 + +The following systems are still supported in Python 2.5, but +support will be dropped in 2.6: +- Systems using --with-wctype-functions +- Win9x, WinME + +Warning on install in Windows 98 and Windows Me +----------------------------------------------- + +Following Microsoft's closing of Extended Support for +Windows 98/ME (July 11, 2006), Python 2.6 will stop +supporting these platforms. Python development and +maintainability becomes easier (and more reliable) when +platform specific code targeting OSes with few users +and no dedicated expert developers is taken out. The +vendor also warns that the OS versions listed above +"can expose customers to security risks" and recommends +upgrade. Platform specific notes ----------------------- @@ -426,14 +446,6 @@ UnixWare: There are known bugs in the math library of the system, as well as thread may interrupt system calls in others). Therefore, test_math and tests involving threads will fail until those problems are fixed. -SunOS 4.x: When using the SunPro C compiler, you may want to use the - '-Xa' option instead of '-Xc', to enable some needed non-ANSI - Sunisms. - THIS SYSTEM IS NO LONGER SUPPORTED. - -NeXT: Not supported anymore. Start with the MacOSX/Darwin code if you - want to revive it. - QNX: Chris Herborth (chrish@qnx.com) writes: configure works best if you use GNU bash; a port is available on ftp.qnx.com in /usr/free. I used the following process to build, diff --git a/RISCOS/pyconfig.h b/RISCOS/pyconfig.h index 45f87be..9a34d41 100644 --- a/RISCOS/pyconfig.h +++ b/RISCOS/pyconfig.h @@ -553,6 +553,9 @@ /* Define if you have the waitpid function. */ #undef HAVE_WAITPID +/* Define if you have the header file. */ +#undef HAVE_CONIO_H + /* Define if you have the header file. */ #undef HAVE_DB_H @@ -562,18 +565,27 @@ /* Define if you have the header file. */ #undef HAVE_DB_185_H +/* Define if you have the header file. */ +#undef HAVE_DIRECT_H + /* Define if you have the header file. */ #undef HAVE_DIRENT_H /* Define if you have the header file. */ #undef HAVE_DLFCN_H +/* Define if you have the header file. */ +#define HAVE_ERRNO_H 1 + /* Define if you have the header file. */ #undef HAVE_FCNTL_H /* Define if you have the header file. */ #undef HAVE_GDBM_NDBM_H +/* Define if you have the header file. */ +#undef HAVE_IO_H + /* Define if you have the header file. */ #undef HAVE_LANGINFO_H @@ -595,12 +607,18 @@ /* Define if you have the header file. */ #undef HAVE_POLL_H +/* Define if you have the header file. */ +#undef HAVE_PROCESS_H + /* Define if you have the header file. */ #undef HAVE_PTHREAD_H /* Define if you have the header file. */ #undef HAVE_PTY_H +/* Define if you have the header file. */ +#define HAVE_SIGNAL_H + /* Define if you have the header file. */ #undef HAVE_SYS_AUDIOIO_H @@ -634,12 +652,18 @@ /* Define if you have the header file. */ #undef HAVE_SYS_SOCKET_H +/* Define if you have the header file. */ +#define HAVE_SYS_STAT_H 1 + /* Define if you have the header file. */ #undef HAVE_SYS_TIME_H /* Define if you have the header file. */ #undef HAVE_SYS_TIMES_H +/* Define if you have the header file. */ +#define HAVE_SYS_TYPES_H 1 + /* Define if you have the header file. */ #undef HAVE_SYS_UN_H @@ -688,7 +712,6 @@ #define DONT_HAVE_FSTAT 1 #define DONT_HAVE_STAT 1 -#undef DONT_HAVE_SYS_STAT_H #define PLATFORM "riscos" diff --git a/Tools/buildbot/Makefile b/Tools/buildbot/Makefile new file mode 100644 index 0000000..1660231 --- /dev/null +++ b/Tools/buildbot/Makefile @@ -0,0 +1,6 @@ +all: kill_python.exe + ./kill_python.exe + +kill_python.exe: kill_python.c + gcc -o kill_python.exe kill_python.c -lpsapi + diff --git a/Tools/buildbot/kill_python.c b/Tools/buildbot/kill_python.c index ebc9aa4..023ff2d 100644 --- a/Tools/buildbot/kill_python.c +++ b/Tools/buildbot/kill_python.c @@ -42,7 +42,19 @@ int main() _strlwr(path); /* printf("%s\n", path); */ - if (strstr(path, "build\\pcbuild\\python_d.exe") != NULL) { + + /* Check if we are running a buildbot version of Python. + + On Windows, this will always be a debug build from the + PCbuild directory. build\\PCbuild\\python_d.exe + + On Cygwin, the pathname is similar to other Unixes. + Use \\build\\python.exe to ensure we don't match + PCbuild\\python.exe which could be a normal instance + of Python running on vanilla Windows. + */ + if ((strstr(path, "build\\pcbuild\\python_d.exe") != NULL) || + (strstr(path, "\\build\\python.exe") != NULL)) { printf("Terminating %s (pid %d)\n", path, pids[i]); if (!TerminateProcess(hProcess, 1)) { printf("Termination failed: %d\n", GetLastError()); diff --git a/Tools/msi/msi.py b/Tools/msi/msi.py index 2576380..aebab98 100644 --- a/Tools/msi/msi.py +++ b/Tools/msi/msi.py @@ -367,7 +367,7 @@ def add_ui(db): ("VerdanaRed9", "Verdana", 9, 255, 0), ]) - compileargs = r"-Wi [TARGETDIR]Lib\compileall.py -f -x bad_coding|badsyntax|site-packages [TARGETDIR]Lib" + compileargs = r'-Wi "[TARGETDIR]Lib\compileall.py" -f -x bad_coding|badsyntax|site-packages "[TARGETDIR]Lib"' # See "CustomAction Table" add_data(db, "CustomAction", [ # msidbCustomActionTypeFirstSequence + msidbCustomActionTypeTextData + msidbCustomActionTypeProperty @@ -962,6 +962,14 @@ def add_files(db): continue dlls.append(f) lib.add_file(f) + # Add sqlite + if msilib.msi_type=="Intel64;1033": + sqlite_arch = "/ia64" + elif msilib.msi_type=="x64;1033": + sqlite_arch = "/amd64" + else: + sqlite_arch = "" + lib.add_file(srcdir+"/"+sqlite_dir+sqlite_arch+"/sqlite3.dll") if have_tcl: if not os.path.exists(srcdir+"/PCBuild/_tkinter.pyd"): print "WARNING: Missing _tkinter.pyd" @@ -972,14 +980,6 @@ def add_files(db): tcldir = os.path.normpath(srcdir+"/../tcltk/bin") for f in glob.glob1(tcldir, "*.dll"): lib.add_file(f, src=os.path.join(tcldir, f)) - # Add sqlite - if msilib.msi_type=="Intel64;1033": - sqlite_arch = "/ia64" - elif msilib.msi_type=="x64;1033": - sqlite_arch = "/amd64" - else: - sqlite_arch = "" - lib.add_file(srcdir+"/"+sqlite_dir+sqlite_arch+"/sqlite3.dll") # check whether there are any unknown extensions for f in glob.glob1(srcdir+"/PCBuild", "*.pyd"): if f.endswith("_d.pyd"): continue # debug version diff --git a/Tools/msi/uuids.py b/Tools/msi/uuids.py index ce7e604..3064975 100644 --- a/Tools/msi/uuids.py +++ b/Tools/msi/uuids.py @@ -27,6 +27,7 @@ product_codes = { '2.5.103': '{73dcd966-ffec-415f-bb39-8342c1f47017}', # 2.5a3 '2.5.111': '{c797ecf8-a8e6-4fec-bb99-526b65f28626}', # 2.5b1 '2.5.112': '{32beb774-f625-439d-b587-7187487baf15}', # 2.5b2 + '2.5.113': '{89f23918-11cf-4f08-be13-b9b2e6463fd9}', # 2.5b3 '2.5.121': '{8e9321bc-6b24-48a3-8fd4-c95f8e531e5f}', # 2.5c1 '2.5.122': '{a6cd508d-9599-45da-a441-cbffa9f7e070}', # 2.5c2 '2.5.150': '{0a2c5854-557e-48c8-835a-3b9f074bdcaa}', # 2.5.0 diff --git a/Tools/pybench/Arithmetic.py b/Tools/pybench/Arithmetic.py index 4ed6219..6923b4b 100644 --- a/Tools/pybench/Arithmetic.py +++ b/Tools/pybench/Arithmetic.py @@ -2,7 +2,7 @@ from pybench import Test class SimpleIntegerArithmetic(Test): - version = 0.3 + version = 2.0 operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) rounds = 120000 @@ -157,9 +157,9 @@ class SimpleIntegerArithmetic(Test): class SimpleFloatArithmetic(Test): - version = 0.3 + version = 2.0 operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) - rounds = 100000 + rounds = 120000 def test(self): @@ -312,7 +312,7 @@ class SimpleFloatArithmetic(Test): class SimpleIntFloatArithmetic(Test): - version = 0.3 + version = 2.0 operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) rounds = 120000 @@ -468,9 +468,9 @@ class SimpleIntFloatArithmetic(Test): class SimpleLongArithmetic(Test): - version = 0.3 + version = 2.0 operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) - rounds = 30000 + rounds = 60000 def test(self): @@ -623,9 +623,9 @@ class SimpleLongArithmetic(Test): class SimpleComplexArithmetic(Test): - version = 0.3 + version = 2.0 operations = 5 * (3 + 5 + 5 + 3 + 3 + 3) - rounds = 40000 + rounds = 80000 def test(self): diff --git a/Tools/pybench/Calls.py b/Tools/pybench/Calls.py index e295243..72ccd0e 100644 --- a/Tools/pybench/Calls.py +++ b/Tools/pybench/Calls.py @@ -2,7 +2,7 @@ from pybench import Test class PythonFunctionCalls(Test): - version = 0.3 + version = 2.0 operations = 5*(1+4+4+2) rounds = 60000 @@ -111,9 +111,9 @@ class PythonFunctionCalls(Test): class BuiltinFunctionCalls(Test): - version = 0.4 + version = 2.0 operations = 5*(2+5+5+5) - rounds = 30000 + rounds = 60000 def test(self): @@ -232,9 +232,9 @@ class BuiltinFunctionCalls(Test): class PythonMethodCalls(Test): - version = 0.3 + version = 2.0 operations = 5*(6 + 5 + 4) - rounds = 20000 + rounds = 30000 def test(self): @@ -374,9 +374,9 @@ class PythonMethodCalls(Test): class Recursion(Test): - version = 0.3 + version = 2.0 operations = 5 - rounds = 50000 + rounds = 100000 def test(self): @@ -407,3 +407,96 @@ class Recursion(Test): for i in xrange(self.rounds): pass + + +### Test to make Fredrik happy... + +if __name__ == '__main__': + import timeit + if 0: + timeit.TestClass = PythonFunctionCalls + timeit.main(['-s', 'test = TestClass(); test.rounds = 1000', + 'test.test()']) + else: + setup = """\ +global f,f1,g,h + +# define functions +def f(): + pass + +def f1(x): + pass + +def g(a,b,c): + return a,b,c + +def h(a,b,c,d=1,e=2,f=3): + return d,e,f + +i = 1 +""" + test = """\ +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) + +f() +f1(i) +f1(i) +f1(i) +f1(i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +g(i,i,i) +h(i,i,3,i,i) +h(i,i,i,2,i,3) +""" + + timeit.main(['-s', setup, + test]) diff --git a/Tools/pybench/CommandLine.py b/Tools/pybench/CommandLine.py index 13e4f9b..6601be5 100644 --- a/Tools/pybench/CommandLine.py +++ b/Tools/pybench/CommandLine.py @@ -358,7 +358,7 @@ class Application: except self.InternalError: print - print '* Internal Error' + print '* Internal Error (use --debug to display the traceback)' if self.debug: print traceback.print_exc(20, sys.stdout) diff --git a/Tools/pybench/Constructs.py b/Tools/pybench/Constructs.py index 00045bd..5105461 100644 --- a/Tools/pybench/Constructs.py +++ b/Tools/pybench/Constructs.py @@ -2,7 +2,7 @@ from pybench import Test class IfThenElse(Test): - version = 0.31 + version = 2.0 operations = 30*3 # hard to say... rounds = 150000 @@ -469,9 +469,9 @@ class IfThenElse(Test): class NestedForLoops(Test): - version = 0.3 + version = 2.0 operations = 1000*10*5 - rounds = 150 + rounds = 300 def test(self): @@ -494,9 +494,9 @@ class NestedForLoops(Test): class ForLoops(Test): - version = 0.1 + version = 2.0 operations = 5 * 5 - rounds = 8000 + rounds = 10000 def test(self): diff --git a/Tools/pybench/Dict.py b/Tools/pybench/Dict.py index 54aeae7..9cdd682 100644 --- a/Tools/pybench/Dict.py +++ b/Tools/pybench/Dict.py @@ -2,9 +2,9 @@ from pybench import Test class DictCreation(Test): - version = 0.3 + version = 2.0 operations = 5*(5 + 5) - rounds = 60000 + rounds = 80000 def test(self): @@ -77,7 +77,7 @@ class DictCreation(Test): class DictWithStringKeys(Test): - version = 0.1 + version = 2.0 operations = 5*(6 + 6) rounds = 200000 @@ -166,9 +166,9 @@ class DictWithStringKeys(Test): class DictWithFloatKeys(Test): - version = 0.1 + version = 2.0 operations = 5*(6 + 6) - rounds = 200000 + rounds = 150000 def test(self): @@ -255,7 +255,7 @@ class DictWithFloatKeys(Test): class DictWithIntegerKeys(Test): - version = 0.1 + version = 2.0 operations = 5*(6 + 6) rounds = 200000 @@ -344,13 +344,14 @@ class DictWithIntegerKeys(Test): class SimpleDictManipulation(Test): - version = 0.3 + version = 2.0 operations = 5*(6 + 6 + 6 + 6) - rounds = 50000 + rounds = 100000 def test(self): d = {} + has_key = d.has_key for i in xrange(self.rounds): @@ -368,12 +369,12 @@ class SimpleDictManipulation(Test): x = d[4] x = d[5] - d.has_key(0) - d.has_key(2) - d.has_key(4) - d.has_key(6) - d.has_key(8) - d.has_key(10) + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) del d[0] del d[1] @@ -396,12 +397,12 @@ class SimpleDictManipulation(Test): x = d[4] x = d[5] - d.has_key(0) - d.has_key(2) - d.has_key(4) - d.has_key(6) - d.has_key(8) - d.has_key(10) + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) del d[0] del d[1] @@ -424,12 +425,12 @@ class SimpleDictManipulation(Test): x = d[4] x = d[5] - d.has_key(0) - d.has_key(2) - d.has_key(4) - d.has_key(6) - d.has_key(8) - d.has_key(10) + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) del d[0] del d[1] @@ -452,12 +453,12 @@ class SimpleDictManipulation(Test): x = d[4] x = d[5] - d.has_key(0) - d.has_key(2) - d.has_key(4) - d.has_key(6) - d.has_key(8) - d.has_key(10) + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) del d[0] del d[1] @@ -480,12 +481,12 @@ class SimpleDictManipulation(Test): x = d[4] x = d[5] - d.has_key(0) - d.has_key(2) - d.has_key(4) - d.has_key(6) - d.has_key(8) - d.has_key(10) + has_key(0) + has_key(2) + has_key(4) + has_key(6) + has_key(8) + has_key(10) del d[0] del d[1] @@ -497,6 +498,7 @@ class SimpleDictManipulation(Test): def calibrate(self): d = {} + has_key = d.has_key for i in xrange(self.rounds): pass diff --git a/Tools/pybench/Exceptions.py b/Tools/pybench/Exceptions.py index 7e55708..eff69c7 100644 --- a/Tools/pybench/Exceptions.py +++ b/Tools/pybench/Exceptions.py @@ -2,9 +2,9 @@ from pybench import Test class TryRaiseExcept(Test): - version = 0.1 - operations = 2 + 3 - rounds = 60000 + version = 2.0 + operations = 2 + 3 + 3 + rounds = 80000 def test(self): @@ -31,6 +31,18 @@ class TryRaiseExcept(Test): raise error,"something" except: pass + try: + raise error("something") + except: + pass + try: + raise error("something") + except: + pass + try: + raise error("something") + except: + pass def calibrate(self): @@ -42,9 +54,9 @@ class TryRaiseExcept(Test): class TryExcept(Test): - version = 0.1 + version = 2.0 operations = 15 * 10 - rounds = 200000 + rounds = 150000 def test(self): @@ -677,3 +689,11 @@ class TryExcept(Test): for i in xrange(self.rounds): pass + +### Test to make Fredrik happy... + +if __name__ == '__main__': + import timeit + timeit.TestClass = TryRaiseExcept + timeit.main(['-s', 'test = TestClass(); test.rounds = 1000', + 'test.test()']) diff --git a/Tools/pybench/Imports.py b/Tools/pybench/Imports.py index 85eb604..afc728b 100644 --- a/Tools/pybench/Imports.py +++ b/Tools/pybench/Imports.py @@ -6,9 +6,9 @@ import package.submodule class SecondImport(Test): - version = 0.1 + version = 2.0 operations = 5 * 5 - rounds = 20000 + rounds = 40000 def test(self): @@ -51,9 +51,9 @@ class SecondImport(Test): class SecondPackageImport(Test): - version = 0.1 + version = 2.0 operations = 5 * 5 - rounds = 20000 + rounds = 40000 def test(self): @@ -95,9 +95,9 @@ class SecondPackageImport(Test): class SecondSubmoduleImport(Test): - version = 0.1 + version = 2.0 operations = 5 * 5 - rounds = 20000 + rounds = 40000 def test(self): diff --git a/Tools/pybench/Instances.py b/Tools/pybench/Instances.py index 9b1929d..1dfc82f 100644 --- a/Tools/pybench/Instances.py +++ b/Tools/pybench/Instances.py @@ -2,9 +2,9 @@ from pybench import Test class CreateInstances(Test): - version = 0.2 + version = 2.0 operations = 3 + 7 + 4 - rounds = 60000 + rounds = 80000 def test(self): diff --git a/Tools/pybench/Lists.py b/Tools/pybench/Lists.py index 4c18e99..67760db 100644 --- a/Tools/pybench/Lists.py +++ b/Tools/pybench/Lists.py @@ -2,22 +2,23 @@ from pybench import Test class SimpleListManipulation(Test): - version = 0.3 + version = 2.0 operations = 5* (6 + 6 + 6) - rounds = 60000 + rounds = 130000 def test(self): l = [] + append = l.append for i in xrange(self.rounds): - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -33,12 +34,12 @@ class SimpleListManipulation(Test): x = l[4] x = l[5] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -54,12 +55,12 @@ class SimpleListManipulation(Test): x = l[4] x = l[5] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -75,12 +76,12 @@ class SimpleListManipulation(Test): x = l[4] x = l[5] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -96,12 +97,12 @@ class SimpleListManipulation(Test): x = l[4] x = l[5] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -124,15 +125,16 @@ class SimpleListManipulation(Test): def calibrate(self): l = [] + append = l.append for i in xrange(self.rounds): pass class ListSlicing(Test): - version = 0.4 + version = 2.0 operations = 25*(3+1+2+1) - rounds = 400 + rounds = 800 def test(self): @@ -141,7 +143,7 @@ class ListSlicing(Test): for i in xrange(self.rounds): - l = range(100) + l = n[:] for j in r: @@ -159,17 +161,14 @@ class ListSlicing(Test): r = range(25) for i in xrange(self.rounds): - - l = range(100) - for j in r: pass class SmallLists(Test): - version = 0.3 + version = 2.0 operations = 5*(1+ 6 + 6 + 3 + 1) - rounds = 60000 + rounds = 80000 def test(self): @@ -177,12 +176,13 @@ class SmallLists(Test): l = [] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -199,12 +199,13 @@ class SmallLists(Test): l = [] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -221,12 +222,13 @@ class SmallLists(Test): l = [] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -243,12 +245,13 @@ class SmallLists(Test): l = [] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -265,12 +268,13 @@ class SmallLists(Test): l = [] - l.append(2) - l.append(3) - l.append(4) - l.append(2) - l.append(3) - l.append(4) + append = l.append + append(2) + append(3) + append(4) + append(2) + append(3) + append(4) l[0] = 3 l[1] = 4 @@ -288,4 +292,4 @@ class SmallLists(Test): def calibrate(self): for i in xrange(self.rounds): - l = [] + pass diff --git a/Tools/pybench/Lookups.py b/Tools/pybench/Lookups.py index e5529cd..f20e7da 100644 --- a/Tools/pybench/Lookups.py +++ b/Tools/pybench/Lookups.py @@ -2,7 +2,7 @@ from pybench import Test class SpecialClassAttribute(Test): - version = 0.3 + version = 2.0 operations = 5*(12 + 12) rounds = 100000 @@ -183,7 +183,7 @@ class SpecialClassAttribute(Test): class NormalClassAttribute(Test): - version = 0.3 + version = 2.0 operations = 5*(12 + 12) rounds = 100000 @@ -369,7 +369,7 @@ class NormalClassAttribute(Test): class SpecialInstanceAttribute(Test): - version = 0.3 + version = 2.0 operations = 5*(12 + 12) rounds = 100000 @@ -557,7 +557,7 @@ class SpecialInstanceAttribute(Test): class NormalInstanceAttribute(Test): - version = 0.3 + version = 2.0 operations = 5*(12 + 12) rounds = 100000 @@ -745,7 +745,7 @@ class NormalInstanceAttribute(Test): class BuiltinMethodLookup(Test): - version = 0.3 + version = 2.0 operations = 5*(3*5 + 3*5) rounds = 70000 diff --git a/Tools/pybench/NewInstances.py b/Tools/pybench/NewInstances.py old mode 100755 new mode 100644 index a352638..258beba --- a/Tools/pybench/NewInstances.py +++ b/Tools/pybench/NewInstances.py @@ -1,8 +1,17 @@ from pybench import Test +# Check for new-style class support: +try: + class c(object): + pass +except NameError: + raise ImportError + +### + class CreateNewInstances(Test): - version = 0.1 + version = 2.0 operations = 3 + 7 + 4 rounds = 60000 diff --git a/Tools/pybench/Numbers.py b/Tools/pybench/Numbers.py index a6aea33..10c8940 100644 --- a/Tools/pybench/Numbers.py +++ b/Tools/pybench/Numbers.py @@ -2,7 +2,7 @@ from pybench import Test class CompareIntegers(Test): - version = 0.1 + version = 2.0 operations = 30 * 5 rounds = 120000 @@ -198,9 +198,9 @@ class CompareIntegers(Test): class CompareFloats(Test): - version = 0.1 + version = 2.0 operations = 30 * 5 - rounds = 60000 + rounds = 80000 def test(self): @@ -394,7 +394,7 @@ class CompareFloats(Test): class CompareFloatsIntegers(Test): - version = 0.1 + version = 2.0 operations = 30 * 5 rounds = 60000 @@ -590,9 +590,9 @@ class CompareFloatsIntegers(Test): class CompareLongs(Test): - version = 0.1 + version = 2.0 operations = 30 * 5 - rounds = 60000 + rounds = 70000 def test(self): diff --git a/Tools/pybench/README b/Tools/pybench/README index 95ae392..022c8de 100644 --- a/Tools/pybench/README +++ b/Tools/pybench/README @@ -28,12 +28,37 @@ and then print out a report to stdout. Micro-Manual ------------ -Run 'pybench.py -h' to see the help screen. -Run 'pybench.py' to just let the benchmark suite do it's thing and -'pybench.py -f ' to have it store the results in a file too. +Run 'pybench.py -h' to see the help screen. Run 'pybench.py' to run +the benchmark suite using default settings and 'pybench.py -f ' +to have it store the results in a file too. + +It is usually a good idea to run pybench.py multiple times to see +whether the environment, timers and benchmark run-times are suitable +for doing benchmark tests. + +You can use the comparison feature of pybench.py ('pybench.py -c +') to check how well the system behaves in comparison to a +reference run. + +If the differences are well below 10% for each test, then you have a +system that is good for doing benchmark testings. Of you get random +differences of more than 10% or significant differences between the +values for minimum and average time, then you likely have some +background processes running which cause the readings to become +inconsistent. Examples include: web-browsers, email clients, RSS +readers, music players, backup programs, etc. + +If you are only interested in a few tests of the whole suite, you can +use the filtering option, e.g. 'pybench.py -t string' will only +run/show the tests that have 'string' in their name. This is the current output of pybench.py --help: +""" +------------------------------------------------------------------------ +PYBENCH - a benchmark test suite for Python interpreters/compilers. +------------------------------------------------------------------------ + Synopsis: pybench.py [option] files... @@ -42,14 +67,14 @@ Options and default settings: -f arg save benchmark to file arg () -c arg compare benchmark with the one in file arg () -s arg show benchmark in file arg, then exit () - -S show statistics of benchmarks (0) - -w arg set warp factor to arg (20) - -d hide noise in compares (0) - --no-gc disable garbage collection (0) - --no-syscheck "disable" sys check interval (set to sys.maxint) (0) - -t arg tests containing substring () - -C arg number of calibration runs (20) - -v generate verbose output + -w arg set warp factor to arg (10) + -t arg run only tests with names matching arg () + -C arg set the number of calibration runs to arg (20) + -d hide noise in comparisons (0) + -v verbose output (not recommended) (0) + --with-gc enable garbage collection (0) + --with-syscheck use default sys check interval (0) + --timer arg use given timer (time.time) -h show this help text --help show this help text --debug enable debugging @@ -57,17 +82,23 @@ Options and default settings: --examples show examples of usage Version: - 1.3 + 2.0 The normal operation is to run the suite and display the -results. Use -f to save them for later reuse or comparisms. +results. Use -f to save them for later reuse or comparisons. -Examples: +Available timers: -python1.5 pybench.py -w 100 -f p15 -python1.4 pybench.py -w 100 -f p14 -python pybench.py -s p15 -c p14 + time.time + time.clock + systimes.processtime +Examples: + +python2.1 pybench.py -f p21.pybench +python2.5 pybench.py -f p25.pybench +python pybench.py -s p25.pybench -c p21.pybench +""" License ------- @@ -78,184 +109,103 @@ See LICENSE file. Sample output ------------- -PYBENCH 1.3 - -Machine Details: - Platform ID: Linux-2.6.8-24.19-default-x86_64-with-SuSE-9.2-x86-64 - Executable: /home/lemburg/projects/Python/Installation/bin/python - Python: 2.5a1.0 - Compiler: GCC 3.3.4 (pre 3.3.5 20040809) - Build: Apr 9 2006 01:50:57 (#trunk) - -Searching for tests... - BuiltinFunctionCalls - BuiltinMethodLookup - CompareFloats - CompareFloatsIntegers - CompareIntegers - CompareInternedStrings - CompareLongs - CompareStrings - CompareUnicode - ConcatStrings - ConcatUnicode - CreateInstances - CreateStringsWithConcat - CreateUnicodeWithConcat - DictCreation - DictWithFloatKeys - DictWithIntegerKeys - DictWithStringKeys - ForLoops - IfThenElse - ListSlicing - NestedForLoops - NormalClassAttribute - NormalInstanceAttribute - PythonFunctionCalls - PythonMethodCalls - Recursion - SecondImport - SecondPackageImport - SecondSubmoduleImport - SimpleComplexArithmetic - SimpleDictManipulation - SimpleFloatArithmetic - SimpleIntFloatArithmetic - SimpleIntegerArithmetic - SimpleListManipulation - SimpleLongArithmetic - SmallLists - SmallTuples - SpecialClassAttribute - SpecialInstanceAttribute - StringMappings - StringPredicates - StringSlicing - TryExcept - TryRaiseExcept - TupleSlicing - UnicodeMappings - UnicodePredicates - UnicodeProperties - UnicodeSlicing - -Running 10 round(s) of the suite: +""" +------------------------------------------------------------------------------- +PYBENCH 2.0 +------------------------------------------------------------------------------- +* using Python 2.4.2 +* disabled garbage collection +* system check interval set to maximum: 2147483647 +* using timer: time.time -... +Calibrating tests. Please wait... - Round 10 real abs overhead - BuiltinFunctionCalls: 0.030r 0.030a 0.000o - BuiltinMethodLookup: 0.059r 0.060a 0.001o - CompareFloats: 0.050r 0.050a 0.000o - CompareFloatsIntegers: 0.050r 0.050a 0.000o - CompareIntegers: 0.070r 0.070a 0.000o - CompareInternedStrings: 0.039r 0.040a 0.001o - CompareLongs: 0.050r 0.050a 0.000o - CompareStrings: 0.060r 0.060a 0.000o - CompareUnicode: 0.060r 0.060a 0.000o - ConcatStrings: 0.040r 0.040a 0.000o - ConcatUnicode: 0.050r 0.050a 0.000o - CreateInstances: 0.050r 0.050a 0.000o - CreateStringsWithConcat: 0.029r 0.030a 0.001o - CreateUnicodeWithConcat: 0.060r 0.060a 0.000o - DictCreation: 0.040r 0.040a 0.000o - DictWithFloatKeys: 0.089r 0.090a 0.000o - DictWithIntegerKeys: 0.059r 0.060a 0.001o - DictWithStringKeys: 0.070r 0.070a 0.001o - ForLoops: 0.050r 0.050a 0.000o - IfThenElse: 0.070r 0.070a 0.000o - ListSlicing: 0.030r 0.030a 0.000o - NestedForLoops: 0.030r 0.030a 0.000o - NormalClassAttribute: 0.060r 0.060a 0.000o - NormalInstanceAttribute: 0.060r 0.060a 0.000o - PythonFunctionCalls: 0.060r 0.060a 0.000o - PythonMethodCalls: 0.050r 0.050a 0.000o - Recursion: 0.050r 0.050a 0.000o - SecondImport: 0.030r 0.030a 0.000o - SecondPackageImport: 0.030r 0.030a 0.000o - SecondSubmoduleImport: 0.040r 0.040a 0.000o - SimpleComplexArithmetic: 0.030r 0.030a 0.000o - SimpleDictManipulation: 0.040r 0.040a 0.000o - SimpleFloatArithmetic: 0.050r 0.050a 0.001o - SimpleIntFloatArithmetic: 0.060r 0.060a 0.000o - SimpleIntegerArithmetic: 0.060r 0.060a 0.000o - SimpleListManipulation: 0.030r 0.030a 0.000o - SimpleLongArithmetic: 0.030r 0.030a 0.000o - SmallLists: 0.050r 0.050a 0.000o - SmallTuples: 0.050r 0.050a 0.000o - SpecialClassAttribute: 0.060r 0.060a 0.000o - SpecialInstanceAttribute: 0.079r 0.080a 0.001o - StringMappings: 0.060r 0.060a 0.000o - StringPredicates: 0.049r 0.050a 0.001o - StringSlicing: 0.039r 0.040a 0.000o - TryExcept: 0.079r 0.080a 0.001o - TryRaiseExcept: 0.059r 0.060a 0.001o - TupleSlicing: 0.050r 0.050a 0.000o - UnicodeMappings: 0.070r 0.070a 0.001o - UnicodePredicates: 0.059r 0.060a 0.001o - UnicodeProperties: 0.059r 0.060a 0.001o - UnicodeSlicing: 0.050r 0.050a 0.000o - ---------------------- - Average round time: 2.937 seconds - - -Tests: per run per oper. overhead ------------------------------------------------------------------------- - BuiltinFunctionCalls: 29.85 ms 0.23 us 0.00 ms - BuiltinMethodLookup: 66.85 ms 0.13 us 0.50 ms - CompareFloats: 43.00 ms 0.10 us 0.00 ms - CompareFloatsIntegers: 51.80 ms 0.12 us 0.00 ms - CompareIntegers: 70.70 ms 0.08 us 0.50 ms - CompareInternedStrings: 41.40 ms 0.08 us 0.50 ms - CompareLongs: 47.90 ms 0.11 us 0.00 ms - CompareStrings: 58.50 ms 0.12 us 0.50 ms - CompareUnicode: 56.55 ms 0.15 us 0.50 ms - ConcatStrings: 44.75 ms 0.30 us 0.00 ms - ConcatUnicode: 54.55 ms 0.36 us 0.50 ms - CreateInstances: 50.95 ms 1.21 us 0.00 ms - CreateStringsWithConcat: 28.85 ms 0.14 us 0.50 ms - CreateUnicodeWithConcat: 53.75 ms 0.27 us 0.00 ms - DictCreation: 41.90 ms 0.28 us 0.00 ms - DictWithFloatKeys: 88.50 ms 0.15 us 0.50 ms - DictWithIntegerKeys: 62.55 ms 0.10 us 0.50 ms - DictWithStringKeys: 60.50 ms 0.10 us 0.50 ms - ForLoops: 46.90 ms 4.69 us 0.00 ms - IfThenElse: 60.55 ms 0.09 us 0.00 ms - ListSlicing: 29.90 ms 8.54 us 0.00 ms - NestedForLoops: 33.95 ms 0.10 us 0.00 ms - NormalClassAttribute: 62.75 ms 0.10 us 0.50 ms - NormalInstanceAttribute: 61.80 ms 0.10 us 0.50 ms - PythonFunctionCalls: 60.00 ms 0.36 us 0.00 ms - PythonMethodCalls: 50.00 ms 0.67 us 0.00 ms - Recursion: 46.85 ms 3.75 us 0.00 ms - SecondImport: 35.00 ms 1.40 us 0.00 ms - SecondPackageImport: 32.00 ms 1.28 us 0.00 ms - SecondSubmoduleImport: 38.00 ms 1.52 us 0.00 ms - SimpleComplexArithmetic: 26.85 ms 0.12 us 0.00 ms - SimpleDictManipulation: 40.85 ms 0.14 us 0.00 ms - SimpleFloatArithmetic: 48.70 ms 0.09 us 0.50 ms - SimpleIntFloatArithmetic: 57.70 ms 0.09 us 0.00 ms - SimpleIntegerArithmetic: 58.75 ms 0.09 us 0.50 ms - SimpleListManipulation: 34.80 ms 0.13 us 0.00 ms - SimpleLongArithmetic: 30.95 ms 0.19 us 0.50 ms - SmallLists: 47.60 ms 0.19 us 0.00 ms - SmallTuples: 48.80 ms 0.20 us 0.50 ms - SpecialClassAttribute: 61.70 ms 0.10 us 0.00 ms - SpecialInstanceAttribute: 76.70 ms 0.13 us 0.50 ms - StringMappings: 58.70 ms 0.47 us 0.00 ms - StringPredicates: 50.00 ms 0.18 us 1.00 ms - StringSlicing: 39.65 ms 0.23 us 0.50 ms - TryExcept: 84.45 ms 0.06 us 0.50 ms - TryRaiseExcept: 61.75 ms 4.12 us 0.50 ms - TupleSlicing: 48.95 ms 0.47 us 0.00 ms - UnicodeMappings: 71.50 ms 3.97 us 0.50 ms - UnicodePredicates: 52.75 ms 0.23 us 1.00 ms - UnicodeProperties: 61.90 ms 0.31 us 1.00 ms - UnicodeSlicing: 53.75 ms 0.31 us 0.50 ms ------------------------------------------------------------------------- - Average round time: 2937.00 ms +Running 10 round(s) of the suite at warp factor 10: +* Round 1 done in 6.388 seconds. +* Round 2 done in 6.485 seconds. +* Round 3 done in 6.786 seconds. +... +* Round 10 done in 6.546 seconds. + +------------------------------------------------------------------------------- +Benchmark: 2006-06-12 12:09:25 +------------------------------------------------------------------------------- + + Rounds: 10 + Warp: 10 + Timer: time.time + + Machine Details: + Platform ID: Linux-2.6.8-24.19-default-x86_64-with-SuSE-9.2-x86-64 + Processor: x86_64 + + Python: + Executable: /usr/local/bin/python + Version: 2.4.2 + Compiler: GCC 3.3.4 (pre 3.3.5 20040809) + Bits: 64bit + Build: Oct 1 2005 15:24:35 (#1) + Unicode: UCS2 + + +Test minimum average operation overhead +------------------------------------------------------------------------------- + BuiltinFunctionCalls: 126ms 145ms 0.28us 0.274ms + BuiltinMethodLookup: 124ms 130ms 0.12us 0.316ms + CompareFloats: 109ms 110ms 0.09us 0.361ms + CompareFloatsIntegers: 100ms 104ms 0.12us 0.271ms + CompareIntegers: 137ms 138ms 0.08us 0.542ms + CompareInternedStrings: 124ms 127ms 0.08us 1.367ms + CompareLongs: 100ms 104ms 0.10us 0.316ms + CompareStrings: 111ms 115ms 0.12us 0.929ms + CompareUnicode: 108ms 128ms 0.17us 0.693ms + ConcatStrings: 142ms 155ms 0.31us 0.562ms + ConcatUnicode: 119ms 127ms 0.42us 0.384ms + CreateInstances: 123ms 128ms 1.14us 0.367ms + CreateNewInstances: 121ms 126ms 1.49us 0.335ms + CreateStringsWithConcat: 130ms 135ms 0.14us 0.916ms + CreateUnicodeWithConcat: 130ms 135ms 0.34us 0.361ms + DictCreation: 108ms 109ms 0.27us 0.361ms + DictWithFloatKeys: 149ms 153ms 0.17us 0.678ms + DictWithIntegerKeys: 124ms 126ms 0.11us 0.915ms + DictWithStringKeys: 114ms 117ms 0.10us 0.905ms + ForLoops: 110ms 111ms 4.46us 0.063ms + IfThenElse: 118ms 119ms 0.09us 0.685ms + ListSlicing: 116ms 120ms 8.59us 0.103ms + NestedForLoops: 125ms 137ms 0.09us 0.019ms + NormalClassAttribute: 124ms 136ms 0.11us 0.457ms + NormalInstanceAttribute: 110ms 117ms 0.10us 0.454ms + PythonFunctionCalls: 107ms 113ms 0.34us 0.271ms + PythonMethodCalls: 140ms 149ms 0.66us 0.141ms + Recursion: 156ms 166ms 3.32us 0.452ms + SecondImport: 112ms 118ms 1.18us 0.180ms + SecondPackageImport: 118ms 127ms 1.27us 0.180ms + SecondSubmoduleImport: 140ms 151ms 1.51us 0.180ms + SimpleComplexArithmetic: 128ms 139ms 0.16us 0.361ms + SimpleDictManipulation: 134ms 136ms 0.11us 0.452ms + SimpleFloatArithmetic: 110ms 113ms 0.09us 0.571ms + SimpleIntFloatArithmetic: 106ms 111ms 0.08us 0.548ms + SimpleIntegerArithmetic: 106ms 109ms 0.08us 0.544ms + SimpleListManipulation: 103ms 113ms 0.10us 0.587ms + SimpleLongArithmetic: 112ms 118ms 0.18us 0.271ms + SmallLists: 105ms 116ms 0.17us 0.366ms + SmallTuples: 108ms 128ms 0.24us 0.406ms + SpecialClassAttribute: 119ms 136ms 0.11us 0.453ms + SpecialInstanceAttribute: 143ms 155ms 0.13us 0.454ms + StringMappings: 115ms 121ms 0.48us 0.405ms + StringPredicates: 120ms 129ms 0.18us 2.064ms + StringSlicing: 111ms 127ms 0.23us 0.781ms + TryExcept: 125ms 126ms 0.06us 0.681ms + TryRaiseExcept: 133ms 137ms 2.14us 0.361ms + TupleSlicing: 117ms 120ms 0.46us 0.066ms + UnicodeMappings: 156ms 160ms 4.44us 0.429ms + UnicodePredicates: 117ms 121ms 0.22us 2.487ms + UnicodeProperties: 115ms 153ms 0.38us 2.070ms + UnicodeSlicing: 126ms 129ms 0.26us 0.689ms +------------------------------------------------------------------------------- +Totals: 6283ms 6673ms +""" ________________________________________________________________________ Writing New Tests @@ -293,7 +243,7 @@ class IntegerCounting(Test): # Number of rounds to execute per test run. This should be # adjusted to a figure that results in a test run-time of between - # 20-50 seconds. + # 1-2 seconds (at warp 1). rounds = 100000 def test(self): @@ -377,6 +327,41 @@ longer strictly comparable with previous runs, the '.version' class variable should be updated. Therefafter, comparisons with previous versions of the test will list as "n/a" to reflect the change. + +Version History +--------------- + + 2.0: rewrote parts of pybench which resulted in more repeatable + timings: + - made timer a parameter + - changed the platform default timer to use high-resolution + timers rather than process timers (which have a much lower + resolution) + - added option to select timer + - added process time timer (using systimes.py) + - changed to use min() as timing estimator (average + is still taken as well to provide an idea of the difference) + - garbage collection is turned off per default + - sys check interval is set to the highest possible value + - calibration is now a separate step and done using + a different strategy that allows measuring the test + overhead more accurately + - modified the tests to each give a run-time of between + 100-200ms using warp 10 + - changed default warp factor to 10 (from 20) + - compared results with timeit.py and confirmed measurements + - bumped all test versions to 2.0 + - updated platform.py to the latest version + - changed the output format a bit to make it look + nicer + - refactored the APIs somewhat + 1.3+: Steve Holden added the NewInstances test and the filtering + option during the NeedForSpeed sprint; this also triggered a long + discussion on how to improve benchmark timing and finally + resulted in the release of 2.0 + 1.3: initial checkin into the Python SVN repository + + Have fun, -- Marc-Andre Lemburg diff --git a/Tools/pybench/Setup.py b/Tools/pybench/Setup.py index f5c5190..f1417e6 100644 --- a/Tools/pybench/Setup.py +++ b/Tools/pybench/Setup.py @@ -14,7 +14,7 @@ # Defaults Number_of_rounds = 10 -Warp_factor = 20 +Warp_factor = 10 # Import tests from Arithmetic import * @@ -24,8 +24,8 @@ from Lookups import * from Instances import * try: from NewInstances import * -except: - print "Cannot test new-style objects" +except ImportError: + pass from Lists import * from Tuples import * from Dict import * diff --git a/Tools/pybench/Strings.py b/Tools/pybench/Strings.py index b01843a..3be8b35 100644 --- a/Tools/pybench/Strings.py +++ b/Tools/pybench/Strings.py @@ -3,9 +3,9 @@ from string import join class ConcatStrings(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 - rounds = 60000 + rounds = 100000 def test(self): @@ -85,7 +85,7 @@ class ConcatStrings(Test): class CompareStrings(Test): - version = 0.2 + version = 2.0 operations = 10 * 5 rounds = 200000 @@ -167,9 +167,9 @@ class CompareStrings(Test): class CompareInternedStrings(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 - rounds = 200000 + rounds = 300000 def test(self): @@ -249,9 +249,9 @@ class CompareInternedStrings(Test): class CreateStringsWithConcat(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 - rounds = 80000 + rounds = 200000 def test(self): @@ -324,9 +324,9 @@ class CreateStringsWithConcat(Test): class StringSlicing(Test): - version = 0.1 + version = 2.0 operations = 5 * 7 - rounds = 100000 + rounds = 160000 def test(self): @@ -387,7 +387,7 @@ if hasattr('', 'lower'): class StringMappings(Test): - version = 0.1 + version = 2.0 operations = 3 * (5 + 4 + 2 + 1) rounds = 70000 @@ -460,9 +460,9 @@ if hasattr('', 'lower'): class StringPredicates(Test): - version = 0.1 + version = 2.0 operations = 10 * 7 - rounds = 80000 + rounds = 100000 def test(self): diff --git a/Tools/pybench/Tuples.py b/Tools/pybench/Tuples.py index e84ea53..8e46989 100644 --- a/Tools/pybench/Tuples.py +++ b/Tools/pybench/Tuples.py @@ -2,18 +2,17 @@ from pybench import Test class TupleSlicing(Test): - version = 0.31 + version = 2.0 operations = 3 * 25 * 10 * 7 - rounds = 400 + rounds = 500 def test(self): r = range(25) + t = tuple(range(100)) for i in xrange(self.rounds): - t = tuple(range(100)) - for j in r: m = t[50:] @@ -259,20 +258,17 @@ class TupleSlicing(Test): def calibrate(self): r = range(25) + t = tuple(range(100)) for i in xrange(self.rounds): - - t = tuple(range(100)) - for j in r: - pass class SmallTuples(Test): - version = 0.3 + version = 2.0 operations = 5*(1 + 3 + 6 + 2) - rounds = 80000 + rounds = 90000 def test(self): diff --git a/Tools/pybench/Unicode.py b/Tools/pybench/Unicode.py index 366f171..153a91e 100644 --- a/Tools/pybench/Unicode.py +++ b/Tools/pybench/Unicode.py @@ -8,7 +8,7 @@ from string import join class ConcatUnicode(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 rounds = 60000 @@ -90,7 +90,7 @@ class ConcatUnicode(Test): class CompareUnicode(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 rounds = 150000 @@ -172,7 +172,7 @@ class CompareUnicode(Test): class CreateUnicodeWithConcat(Test): - version = 0.1 + version = 2.0 operations = 10 * 5 rounds = 80000 @@ -247,9 +247,9 @@ class CreateUnicodeWithConcat(Test): class UnicodeSlicing(Test): - version = 0.1 + version = 2.0 operations = 5 * 7 - rounds = 100000 + rounds = 140000 def test(self): @@ -308,7 +308,7 @@ class UnicodeSlicing(Test): class UnicodeMappings(Test): - version = 0.1 + version = 2.0 operations = 3 * (5 + 4 + 2 + 1) rounds = 10000 @@ -381,9 +381,9 @@ class UnicodeMappings(Test): class UnicodePredicates(Test): - version = 0.1 + version = 2.0 operations = 5 * 9 - rounds = 100000 + rounds = 120000 def test(self): @@ -458,7 +458,7 @@ except ImportError: else: class UnicodeProperties(Test): - version = 0.1 + version = 2.0 operations = 5 * 8 rounds = 100000 diff --git a/Tools/pybench/clockres.py b/Tools/pybench/clockres.py new file mode 100644 index 0000000..64095b3 --- /dev/null +++ b/Tools/pybench/clockres.py @@ -0,0 +1,43 @@ +#!/usr/bin/env python + +""" clockres - calculates the resolution in seconds of a given timer. + + Copyright (c) 2006, Marc-Andre Lemburg (mal@egenix.com). See the + documentation for further information on copyrights, or contact + the author. All Rights Reserved. + +""" +import time + +TEST_TIME = 1.0 + +def clockres(timer): + d = {} + wallclock = time.time + start = wallclock() + stop = wallclock() + TEST_TIME + spin_loops = range(1000) + while 1: + now = wallclock() + if now >= stop: + break + for i in spin_loops: + d[timer()] = 1 + values = d.keys() + values.sort() + min_diff = TEST_TIME + for i in range(len(values) - 1): + diff = values[i+1] - values[i] + if diff < min_diff: + min_diff = diff + return min_diff + +if __name__ == '__main__': + print 'Clock resolution of various timer implementations:' + print 'time.clock: %10.3fus' % (clockres(time.clock) * 1e6) + print 'time.time: %10.3fus' % (clockres(time.time) * 1e6) + try: + import systimes + print 'systimes.processtime: %10.3fus' % (clockres(systimes.processtime) * 1e6) + except ImportError: + pass diff --git a/Tools/pybench/pybench.py b/Tools/pybench/pybench.py index e0110d0..7d90ba1 100755 --- a/Tools/pybench/pybench.py +++ b/Tools/pybench/pybench.py @@ -34,20 +34,7 @@ NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE ! """ -# Version number -__version__ = '1.3' - -# -# NOTE: Use xrange for all test loops unless you want to face -# a 20MB process ! -# -# All tests should have rounds set to values so that a run() -# takes between 20-50 seconds. This is to get fairly good -# clock() values. You can use option -w to speedup the tests -# by a fixed integer factor (the "warp factor"). -# - -import sys,time,operator +import sys, time, operator, string from CommandLine import * try: @@ -56,6 +43,111 @@ try: except ImportError: import pickle +# Version number; version history: see README file ! +__version__ = '2.0' + +### Constants + +# Second fractions +MILLI_SECONDS = 1e3 +MICRO_SECONDS = 1e6 + +# Percent unit +PERCENT = 100 + +# Horizontal line length +LINE = 79 + +# Minimum test run-time +MIN_TEST_RUNTIME = 1e-3 + +# Number of calibration runs to use for calibrating the tests +CALIBRATION_RUNS = 20 + +# Number of calibration loops to run for each calibration run +CALIBRATION_LOOPS = 20 + +# Allow skipping calibration ? +ALLOW_SKIPPING_CALIBRATION = 1 + +# Timer types +TIMER_TIME_TIME = 'time.time' +TIMER_TIME_CLOCK = 'time.clock' +TIMER_SYSTIMES_PROCESSTIME = 'systimes.processtime' + +# Choose platform default timer +if sys.platform[:3] == 'win': + # On WinXP this has 2.5ms resolution + TIMER_PLATFORM_DEFAULT = TIMER_TIME_CLOCK +else: + # On Linux this has 1ms resolution + TIMER_PLATFORM_DEFAULT = TIMER_TIME_TIME + +# Print debug information ? +_debug = 0 + +### Helpers + +def get_timer(timertype): + + if timertype == TIMER_TIME_TIME: + return time.time + elif timertype == TIMER_TIME_CLOCK: + return time.clock + elif timertype == TIMER_SYSTIMES_PROCESSTIME: + import systimes + return systimes.processtime + else: + raise TypeError('unknown timer type: %s' % timertype) + +def get_machine_details(): + + import platform + if _debug: + print 'Getting machine details...' + buildno, builddate = platform.python_build() + python = platform.python_version() + if python > '2.0': + try: + unichr(100000) + except ValueError: + # UCS2 build (standard) + unicode = 'UCS2' + else: + # UCS4 build (most recent Linux distros) + unicode = 'UCS4' + else: + unicode = None + bits, linkage = platform.architecture() + return { + 'platform': platform.platform(), + 'processor': platform.processor(), + 'executable': sys.executable, + 'python': platform.python_version(), + 'compiler': platform.python_compiler(), + 'buildno': buildno, + 'builddate': builddate, + 'unicode': unicode, + 'bits': bits, + } + +def print_machine_details(d, indent=''): + + l = ['Machine Details:', + ' Platform ID: %s' % d.get('platform', 'n/a'), + ' Processor: %s' % d.get('processor', 'n/a'), + '', + 'Python:', + ' Executable: %s' % d.get('executable', 'n/a'), + ' Version: %s' % d.get('python', 'n/a'), + ' Compiler: %s' % d.get('compiler', 'n/a'), + ' Bits: %s' % d.get('bits', 'n/a'), + ' Build: %s (#%s)' % (d.get('builddate', 'n/a'), + d.get('buildno', 'n/a')), + ' Unicode: %s' % d.get('unicode', 'n/a'), + ] + print indent + string.join(l, '\n' + indent) + '\n' + ### Test baseclass class Test: @@ -84,7 +176,7 @@ class Test: # Version number of the test as float (x.yy); this is important # for comparisons of benchmark runs - tests with unequal version # number will not get compared. - version = 1.0 + version = 2.0 # The number of abstract operations done in each round of the # test. An operation is the basic unit of what you want to @@ -97,36 +189,125 @@ class Test: # Number of rounds to execute per test run. This should be # adjusted to a figure that results in a test run-time of between - # 20-50 seconds. - rounds = 10000 + # 1-2 seconds. + rounds = 100000 ### Internal variables # Mark this class as implementing a test is_a_test = 1 - # Misc. internal variables - last_timing = (0,0,0) # last timing (real,run,calibration) - warp = 1 # warp factor this test uses - cruns = 20 # number of calibration runs - overhead = None # list of calibration timings + # Last timing: (real, run, overhead) + last_timing = (0.0, 0.0, 0.0) + + # Warp factor to use for this test + warp = 1 - def __init__(self,warp=1): + # Number of calibration runs to use + calibration_runs = CALIBRATION_RUNS - if warp > 1: - self.rounds = self.rounds / warp + # List of calibration timings + overhead_times = None + + # List of test run timings + times = [] + + # Timer used for the benchmark + timer = TIMER_PLATFORM_DEFAULT + + def __init__(self, warp=None, calibration_runs=None, timer=None): + + # Set parameters + if warp is not None: + self.rounds = int(self.rounds / warp) if self.rounds == 0: - self.rounds = 1 + raise ValueError('warp factor set too high') self.warp = warp + if calibration_runs is not None: + if (not ALLOW_SKIPPING_CALIBRATION and + calibration_runs < 1): + raise ValueError('at least one calibration run is required') + self.calibration_runs = calibration_runs + if timer is not None: + timer = timer + + # Init variables self.times = [] - self.overhead = [] + self.overhead_times = [] + # We want these to be in the instance dict, so that pickle # saves them self.version = self.version self.operations = self.operations self.rounds = self.rounds - def run(self, cruns): + def get_timer(self): + + """ Return the timer function to use for the test. + + """ + return get_timer(self.timer) + + def compatible(self, other): + + """ Return 1/0 depending on whether the test is compatible + with the other Test instance or not. + + """ + if self.version != other.version: + return 0 + if self.rounds != other.rounds: + return 0 + return 1 + + def calibrate_test(self): + + if self.calibration_runs == 0: + self.overhead_times = [0.0] + return + + calibrate = self.calibrate + timer = self.get_timer() + calibration_loops = range(CALIBRATION_LOOPS) + + # Time the calibration loop overhead + prep_times = [] + for i in range(self.calibration_runs): + t = timer() + for i in calibration_loops: + pass + t = timer() - t + prep_times.append(t) + min_prep_time = min(prep_times) + if _debug: + print + print 'Calib. prep time = %.6fms' % ( + min_prep_time * MILLI_SECONDS) + + # Time the calibration runs (doing CALIBRATION_LOOPS loops of + # .calibrate() method calls each) + for i in range(self.calibration_runs): + t = timer() + for i in calibration_loops: + calibrate() + t = timer() - t + self.overhead_times.append(t / CALIBRATION_LOOPS + - min_prep_time) + + # Check the measured times + min_overhead = min(self.overhead_times) + max_overhead = max(self.overhead_times) + if _debug: + print 'Calib. overhead time = %.6fms' % ( + min_overhead * MILLI_SECONDS) + if min_overhead < 0.0: + raise ValueError('calibration setup did not work') + if max_overhead - min_overhead > 0.1: + raise ValueError( + 'overhead calibration timing range too inaccurate: ' + '%r - %r' % (min_overhead, max_overhead)) + + def run(self): """ Run the test in two phases: first calibrate, then do the actual test. Be careful to keep the calibration @@ -134,27 +315,23 @@ class Test: """ test = self.test - calibrate = self.calibrate - clock = time.clock - # first calibrate - t = clock() - calibrate() - offset = clock() - t - if cruns: - for i in range(cruns-1): - t = clock() - calibrate() - t = clock() - t - if t < offset: - offset = t - # now the real thing - t = clock() + timer = self.get_timer() + + # Get calibration + min_overhead = min(self.overhead_times) + + # Test run + t = timer() test() - t = clock() - t - if t < 0.01: - sys.exit("Lower warp required: test times < 10 ms are unreliable") - self.last_timing = (t-offset,t,offset) - self.times.append(t-offset) + t = timer() - t + if t < MIN_TEST_RUNTIME: + raise ValueError('warp factor too high: ' + 'test times are < 10ms') + eff_time = t - min_overhead + if eff_time < 0: + raise ValueError('wrong calibration') + self.last_timing = (eff_time, t, min_overhead) + self.times.append(eff_time) def calibrate(self): @@ -176,33 +353,33 @@ class Test: self.operations number of operations each. """ - # do some tests return def stat(self): - """ Returns four values: - minimum round time - average time per round - average time per operation - average overhead time + """ Return test run statistics as tuple: + + (minimum run time, + average run time, + total run time, + average time per operation, + minimum overhead time) - XXX Should this take warp factors into account? """ runs = len(self.times) if runs == 0: - return 0,0 - mintime = min(self.times) - totaltime = reduce(operator.add,self.times,0.0) - avg = totaltime / float(runs) - op_avg = totaltime / float(runs * self.rounds * self.operations) - if self.overhead: - totaloverhead = reduce(operator.add,self.overhead,0.0) - ov_avg = totaloverhead / float(runs) + return 0.0, 0.0, 0.0, 0.0 + min_time = min(self.times) + total_time = reduce(operator.add, self.times, 0.0) + avg_time = total_time / float(runs) + operation_avg = total_time / float(runs + * self.rounds + * self.operations) + if self.overhead_times: + min_overhead = min(self.overhead_times) else: - # use self.last_timing - not too accurate - ov_avg = self.last_timing[2] - return mintime, avg, op_avg, ov_avg + min_overhead = self.last_timing[2] + return min_time, avg_time, total_time, operation_avg, min_overhead ### Load Setup @@ -215,153 +392,353 @@ import Setup class Benchmark: - name = '?' # Name of the benchmark - rounds = 1 # Number of rounds to run + # Name of the benchmark + name = '' + + # Number of benchmark rounds to run + rounds = 1 + + # Warp factor use to run the tests warp = 1 # Warp factor - roundtime = 0 # Average round time - version = None # Benchmark version number (see __init__) - # as float x.yy - def __init__(self): + # Average benchmark round time + roundtime = 0 - self.tests = {} - self.version = 0.31 + # Benchmark version number as float x.yy + version = 2.0 - def load_tests(self, setupmod, warp=1, limitnames="", verbose=0): + # Produce verbose output ? + verbose = 0 - self.warp = warp - if limitnames: - limitnames = re.compile(limitnames, re.I) + # Dictionary with the machine details + machine_details = None + + # Timer used for the benchmark + timer = TIMER_PLATFORM_DEFAULT + + def __init__(self, name, verbose=None, timer=None, warp=None, + calibration_runs=None): + + if name: + self.name = name else: - limitnames = None - tests = self.tests - if verbose: - print 'Searching for tests ...', - setupmod.__dict__.values() - for c in setupmod.__dict__.values(): - if not hasattr(c,'is_a_test'): + self.name = '%04i-%02i-%02i %02i:%02i:%02i' % \ + (time.localtime(time.time())[:6]) + if verbose is not None: + self.verbose = verbose + if timer is not None: + self.timer = timer + if warp is not None: + self.warp = warp + if calibration_runs is not None: + self.calibration_runs = calibration_runs + + # Init vars + self.tests = {} + if _debug: + print 'Getting machine details...' + self.machine_details = get_machine_details() + + # Make .version an instance attribute to have it saved in the + # Benchmark pickle + self.version = self.version + + def get_timer(self): + + """ Return the timer function to use for the test. + + """ + return get_timer(self.timer) + + def compatible(self, other): + + """ Return 1/0 depending on whether the benchmark is + compatible with the other Benchmark instance or not. + + """ + if self.version != other.version: + return 0 + if (self.machine_details == other.machine_details and + self.timer != other.timer): + return 0 + if (self.calibration_runs == 0 and + other.calibration_runs != 0): + return 0 + if (self.calibration_runs != 0 and + other.calibration_runs == 0): + return 0 + return 1 + + def load_tests(self, setupmod, limitnames=None): + + # Add tests + if self.verbose: + print 'Searching for tests ...' + print '--------------------------------------' + for testclass in setupmod.__dict__.values(): + if not hasattr(testclass, 'is_a_test'): continue - name = c.__name__ + name = testclass.__name__ if name == 'Test': continue - if limitnames is not None and limitnames.search(name) is None: + if (limitnames is not None and + limitnames.search(name) is None): continue - tests[name] = c(warp) - l = tests.keys() + self.tests[name] = testclass( + warp=self.warp, + calibration_runs=self.calibration_runs, + timer=self.timer) + l = self.tests.keys() l.sort() - if verbose: + if self.verbose: + for name in l: + print ' %s' % name + print '--------------------------------------' + print ' %i tests found' % len(l) print - for t in l: - print ' ', t - print len(l), "tests found" + + def calibrate(self): + + print 'Calibrating tests. Please wait...' + if self.verbose: + print + print 'Test min max' + print '-' * LINE + tests = self.tests.items() + tests.sort() + for i in range(len(tests)): + name, test = tests[i] + test.calibrate_test() + if self.verbose: + print '%30s: %6.3fms %6.3fms' % \ + (name, + min(test.overhead_times) * MILLI_SECONDS, + max(test.overhead_times) * MILLI_SECONDS) print - def run(self, verbose, cruns): + def run(self): tests = self.tests.items() tests.sort() - clock = time.clock - print 'Running %i round(s) of the suite at warp factor %i:' % (self.rounds, self.warp) + timer = self.get_timer() + print 'Running %i round(s) of the suite at warp factor %i:' % \ + (self.rounds, self.warp) print - roundtime = clock() + self.roundtimes = [] for i in range(self.rounds): - roundstarttime = clock() - if verbose: - print ' Round %-25i real abs overhead' % (i+1) + if self.verbose: + print ' Round %-25i effective absolute overhead' % (i+1) + total_eff_time = 0.0 for j in range(len(tests)): - name, t = tests[j] - if verbose: + name, test = tests[j] + if self.verbose: print '%30s:' % name, - t.run(cruns) - if verbose: - print ' %.3fr %.3fa %.3fo' % t.last_timing - if verbose: - print ' ----------------------' - print ' Average round time: %.3f seconds' % \ - ((clock() - roundtime)/(i+1)) + test.run() + (eff_time, abs_time, min_overhead) = test.last_timing + total_eff_time = total_eff_time + eff_time + if self.verbose: + print ' %5.0fms %5.0fms %7.3fms' % \ + (eff_time * MILLI_SECONDS, + abs_time * MILLI_SECONDS, + min_overhead * MILLI_SECONDS) + self.roundtimes.append(total_eff_time) + if self.verbose: + print (' ' + ' ------------------------------') + print (' ' + ' Totals: %6.0fms' % + (total_eff_time * MILLI_SECONDS)) print else: - print '%d done in %.3f seconds' % (i+1, (clock() - roundstarttime)) - self.roundtime = (clock() - roundtime) / self.rounds + print '* Round %i done in %.3f seconds.' % (i+1, + total_eff_time) print - def print_stat(self, compare_to=None, hidenoise=0): - - if not compare_to: - print '%-30s min run avg run per oprn overhead' % 'Tests:' - print '-'*77 - tests = self.tests.items() - tests.sort() - totalmintime = 0 - for name,t in tests: - mintime,avg,op_avg,ov_avg = t.stat() - totalmintime += mintime - print '%30s: %9.2f ms %9.2f ms %6.2f us %6.2f' % \ - (name,mintime*1000.0,avg*1000.0,op_avg*1000000.0,ov_avg*1000.0) - print '-'*77 - print '%30s: %9.2f ms' % \ - ('Notional minimum round time', totalmintime * 1000.0) + def stat(self): + + """ Return benchmark run statistics as tuple: - else: - print 'Comparing with: %s (rounds=%i, warp=%i)' % \ - (compare_to.name,compare_to.rounds,compare_to.warp) - print '%-30s min run cmp run avg run diff' % \ - 'Tests:' - print '-'*77 - tests = self.tests.items() - tests.sort() - compatible = 1 - totalmintime = other_totalmintime = 0 - for name, t in tests: - mintime, avg, op_avg, ov_avg = t.stat() - totalmintime += mintime - try: - other = compare_to.tests[name] - except KeyError: - other = None - if other and other.version == t.version and \ - other.operations == t.operations: - mintime1, avg1, op_avg1, ov_avg1 = other.stat() - other_totalmintime += mintime1 - diff = ((mintime*self.warp)/(mintime1*other.warp) - 1.0)*100.0 - if hidenoise and abs(qop_avg) < 10: - diff = '' + (minimum round time, + average round time, + maximum round time) + + XXX Currently not used, since the benchmark does test + statistics across all rounds. + + """ + runs = len(self.roundtimes) + if runs == 0: + return 0.0, 0.0 + min_time = min(self.roundtimes) + total_time = reduce(operator.add, self.roundtimes, 0.0) + avg_time = total_time / float(runs) + max_time = max(self.roundtimes) + return (min_time, avg_time, max_time) + + def print_header(self, title='Benchmark'): + + print '-' * LINE + print '%s: %s' % (title, self.name) + print '-' * LINE + print + print ' Rounds: %s' % self.rounds + print ' Warp: %s' % self.warp + print ' Timer: %s' % self.timer + print + if self.machine_details: + print_machine_details(self.machine_details, indent=' ') + print + + def print_benchmark(self, hidenoise=0, limitnames=None): + + print ('Test ' + ' minimum average operation overhead') + print '-' * LINE + tests = self.tests.items() + tests.sort() + total_min_time = 0.0 + total_avg_time = 0.0 + for name, test in tests: + if (limitnames is not None and + limitnames.search(name) is None): + continue + (min_time, + avg_time, + total_time, + op_avg, + min_overhead) = test.stat() + total_min_time = total_min_time + min_time + total_avg_time = total_avg_time + avg_time + print '%30s: %5.0fms %5.0fms %6.2fus %7.3fms' % \ + (name, + min_time * MILLI_SECONDS, + avg_time * MILLI_SECONDS, + op_avg * MICRO_SECONDS, + min_overhead *MILLI_SECONDS) + print '-' * LINE + print ('Totals: ' + ' %6.0fms %6.0fms' % + (total_min_time * MILLI_SECONDS, + total_avg_time * MILLI_SECONDS, + )) + print + + def print_comparison(self, compare_to, hidenoise=0, limitnames=None): + + # Check benchmark versions + if compare_to.version != self.version: + print ('* Benchmark versions differ: ' + 'cannot compare this benchmark to "%s" !' % + compare_to.name) + print + self.print_benchmark(hidenoise=hidenoise, + limitnames=limitnames) + return + + # Print header + compare_to.print_header('Comparing with') + print ('Test ' + ' minimum run-time average run-time') + print (' ' + ' this other diff this other diff') + print '-' * LINE + + # Print test comparisons + tests = self.tests.items() + tests.sort() + total_min_time = other_total_min_time = 0.0 + total_avg_time = other_total_avg_time = 0.0 + benchmarks_compatible = self.compatible(compare_to) + tests_compatible = 1 + for name, test in tests: + if (limitnames is not None and + limitnames.search(name) is None): + continue + (min_time, + avg_time, + total_time, + op_avg, + min_overhead) = test.stat() + total_min_time = total_min_time + min_time + total_avg_time = total_avg_time + avg_time + try: + other = compare_to.tests[name] + except KeyError: + other = None + if other is None: + # Other benchmark doesn't include the given test + min_diff, avg_diff = 'n/a', 'n/a' + other_min_time = 0.0 + other_avg_time = 0.0 + tests_compatible = 0 + else: + (other_min_time, + other_avg_time, + other_total_time, + other_op_avg, + other_min_overhead) = other.stat() + other_total_min_time = other_total_min_time + other_min_time + other_total_avg_time = other_total_avg_time + other_avg_time + if (benchmarks_compatible and + test.compatible(other)): + # Both benchmark and tests are comparible + min_diff = ((min_time * self.warp) / + (other_min_time * other.warp) - 1.0) + avg_diff = ((avg_time * self.warp) / + (other_avg_time * other.warp) - 1.0) + if hidenoise and abs(min_diff) < 10.0: + min_diff = '' else: - diff = '%+7.2f%%' % diff + min_diff = '%+5.1f%%' % (min_diff * PERCENT) + if hidenoise and abs(avg_diff) < 10.0: + avg_diff = '' + else: + avg_diff = '%+5.1f%%' % (avg_diff * PERCENT) else: - qavg, diff = 'n/a', 'n/a' - compatible = 0 - print '%30s: %8.2f ms %8.2f ms %8.2f ms %8s' % \ - (name,mintime*1000.0,mintime1*1000.0 * compare_to.warp/self.warp, avg*1000.0,diff) - print '-'*77 - # - # Summarise test results - # - if compatible and compare_to.roundtime > 0 and \ - compare_to.version == self.version: - print '%30s: %8.2f ms %8.2f ms %+7.2f%%' % \ - ('Notional minimum round time', totalmintime * 1000.0, - other_totalmintime * 1000.0 * compare_to.warp/self.warp, - ((totalmintime*self.warp)/ - (other_totalmintime*compare_to.warp)-1.0)*100.0) + # Benchmark or tests are not comparible + min_diff, avg_diff = 'n/a', 'n/a' + tests_compatible = 0 + print '%30s: %5.0fms %5.0fms %7s %5.0fms %5.0fms %7s' % \ + (name, + min_time * MILLI_SECONDS, + other_min_time * MILLI_SECONDS * compare_to.warp / self.warp, + min_diff, + avg_time * MILLI_SECONDS, + other_avg_time * MILLI_SECONDS * compare_to.warp / self.warp, + avg_diff) + print '-' * LINE + + # Summarise test results + if not benchmarks_compatible or not tests_compatible: + min_diff, avg_diff = 'n/a', 'n/a' + else: + if other_total_min_time != 0.0: + min_diff = '%+5.1f%%' % ( + ((total_min_time * self.warp) / + (other_total_min_time * compare_to.warp) - 1.0) * PERCENT) + else: + min_diff = 'n/a' + if other_total_avg_time != 0.0: + avg_diff = '%+5.1f%%' % ( + ((total_avg_time * self.warp) / + (other_total_avg_time * compare_to.warp) - 1.0) * PERCENT) else: - print '%30s: %9.2f ms n/a' % \ - ('Notional minimum round time', totalmintime * 1000.0) + avg_diff = 'n/a' + print ('Totals: ' + ' %5.0fms %5.0fms %7s %5.0fms %5.0fms %7s' % + (total_min_time * MILLI_SECONDS, + (other_total_min_time * compare_to.warp/self.warp + * MILLI_SECONDS), + min_diff, + total_avg_time * MILLI_SECONDS, + (other_total_avg_time * compare_to.warp/self.warp + * MILLI_SECONDS), + avg_diff + )) + print + print '(this=%s, other=%s)' % (self.name, + compare_to.name) print - -def print_machine(): - - import platform - print 'Machine Details:' - print ' Platform ID: %s' % platform.platform() - print ' Executable: %s' % sys.executable - # There's a bug in Python 2.2b1+... - if sys.version[:6] == '2.2b1+': - return - print ' Python: %s' % platform.python_version() - print ' Compiler: %s' % platform.python_compiler() - buildno, builddate = platform.python_build() - print ' Build: %s (#%s)' % (builddate, buildno) class PyBenchCmdline(Application): @@ -370,50 +747,64 @@ class PyBenchCmdline(Application): version = __version__ - options = [ArgumentOption('-n','number of rounds',Setup.Number_of_rounds), - ArgumentOption('-f','save benchmark to file arg',''), - ArgumentOption('-c','compare benchmark with the one in file arg',''), - ArgumentOption('-s','show benchmark in file arg, then exit',''), - SwitchOption('-S','show statistics of benchmarks',0), - ArgumentOption('-w','set warp factor to arg',Setup.Warp_factor), - SwitchOption('-d','hide noise in compares', 0), - SwitchOption('-v','verbose output (not recommended)', 0), - SwitchOption('--no-gc','disable garbage collection', 0), - SwitchOption('--no-syscheck', - '"disable" sys check interval (set to sys.maxint)', 0), - ArgumentOption('-t', 'tests containing substring', ''), - ArgumentOption('-C', 'number of calibration runs', 20) + debug = _debug + + options = [ArgumentOption('-n', + 'number of rounds', + Setup.Number_of_rounds), + ArgumentOption('-f', + 'save benchmark to file arg', + ''), + ArgumentOption('-c', + 'compare benchmark with the one in file arg', + ''), + ArgumentOption('-s', + 'show benchmark in file arg, then exit', + ''), + ArgumentOption('-w', + 'set warp factor to arg', + Setup.Warp_factor), + ArgumentOption('-t', + 'run only tests with names matching arg', + ''), + ArgumentOption('-C', + 'set the number of calibration runs to arg', + CALIBRATION_RUNS), + SwitchOption('-d', + 'hide noise in comparisons', + 0), + SwitchOption('-v', + 'verbose output (not recommended)', + 0), + SwitchOption('--with-gc', + 'enable garbage collection', + 0), + SwitchOption('--with-syscheck', + 'use default sys check interval', + 0), + ArgumentOption('--timer', + 'use given timer', + TIMER_PLATFORM_DEFAULT), ] about = """\ The normal operation is to run the suite and display the -results. Use -f to save them for later reuse or comparisms. +results. Use -f to save them for later reuse or comparisons. + +Available timers: + + time.time + time.clock + systimes.processtime Examples: -python1.5 pybench.py -w 100 -f p15 -python1.4 pybench.py -w 100 -f p14 -python pybench.py -s p15 -c p14 +python2.1 pybench.py -f p21.pybench +python2.5 pybench.py -f p25.pybench +python pybench.py -s p25.pybench -c p21.pybench """ copyright = __copyright__ - def handle_S(self, value): - - """ Display one line stats for each benchmark file given on the - command line. - - """ - for benchmark in self.files: - try: - f = open(benchmark, 'rb') - bench = pickle.load(f) - f.close() - except IOError: - print '* Error opening/reading file %s' % repr(benchmark) - else: - print '%s,%-.2f,ms' % (benchmark, bench.roundtime*1000.0) - return 0 - def main(self): rounds = self.values['-n'] @@ -421,38 +812,52 @@ python pybench.py -s p15 -c p14 show_bench = self.values['-s'] compare_to = self.values['-c'] hidenoise = self.values['-d'] - warp = self.values['-w'] - nogc = self.values['--no-gc'] + warp = int(self.values['-w']) + withgc = self.values['--with-gc'] limitnames = self.values['-t'] + if limitnames: + if _debug: + print '* limiting test names to one with substring "%s"' % \ + limitnames + limitnames = re.compile(limitnames, re.I) + else: + limitnames = None verbose = self.verbose - nosyscheck = self.values['--no-syscheck'] - cruns = self.values['-C'] - print "CRUNS:", cruns + withsyscheck = self.values['--with-syscheck'] + calibration_runs = self.values['-C'] + timer = self.values['--timer'] - print 'PYBENCH',__version__ + print '-' * LINE + print 'PYBENCH %s' % __version__ + print '-' * LINE + print '* using Python %s' % (string.split(sys.version)[0]) - # Switch off GC - if nogc: + # Switch off garbage collection + if not withgc: try: import gc except ImportError: - nogc = 0 + print '* Python version doesn\'t support garbage collection' else: - if self.values['--no-gc']: - gc.disable() - print 'NO GC' - - # maximise sys check interval - if nosyscheck: - sys.setcheckinterval(sys.maxint) - print 'CHECKINTERVAL =', sys.maxint + gc.disable() + print '* disabled garbage collection' + + # "Disable" sys check interval + if not withsyscheck: + # Too bad the check interval uses an int instead of a long... + value = 2147483647 + sys.setcheckinterval(value) + print '* system check interval set to maximum: %s' % value + + if timer == TIMER_SYSTIMES_PROCESSTIME: + import systimes + print '* using timer: systimes.processtime (%s)' % \ + systimes.SYSTIMES_IMPLEMENTATION + else: + print '* using timer: %s' % timer print - if not compare_to: - print_machine() - print - if compare_to: try: f = open(compare_to,'rb') @@ -460,8 +865,10 @@ python pybench.py -s p15 -c p14 bench.name = compare_to f.close() compare_to = bench - except IOError: - print '* Error opening/reading file',compare_to + except IOError, reason: + print '* Error opening/reading file %s: %s' % ( + repr(compare_to), + reason) compare_to = None if show_bench: @@ -470,37 +877,52 @@ python pybench.py -s p15 -c p14 bench = pickle.load(f) bench.name = show_bench f.close() - print 'Benchmark: %s (rounds=%i, warp=%i)' % \ - (bench.name,bench.rounds,bench.warp) - print - bench.print_stat(compare_to, hidenoise) + bench.print_header() + if compare_to: + bench.print_comparison(compare_to, + hidenoise=hidenoise, + limitnames=limitnames) + else: + bench.print_benchmark(hidenoise=hidenoise, + limitnames=limitnames) except IOError: - print '* Error opening/reading file',show_bench + print '* Error opening/reading file %s: %s' % ( + repr(show_bench), + reason) print return if reportfile: - if nogc: - print 'Benchmark: %s (rounds=%i, warp=%i, no GC)' % \ - (reportfile,rounds,warp) - else: - print 'Benchmark: %s (rounds=%i, warp=%i)' % \ - (reportfile,rounds,warp) + print 'Creating benchmark: %s (rounds=%i, warp=%i)' % \ + (reportfile, rounds, warp) print # Create benchmark object - bench = Benchmark() + bench = Benchmark(reportfile, + verbose=verbose, + timer=timer, + warp=warp, + calibration_runs=calibration_runs) bench.rounds = rounds - bench.load_tests(Setup, warp, limitnames, verbose) + bench.load_tests(Setup, limitnames=limitnames) try: - bench.run(verbose, cruns) + bench.calibrate() + bench.run() except KeyboardInterrupt: print print '*** KeyboardInterrupt -- Aborting' print return - bench.print_stat(compare_to) - # ring bell + bench.print_header() + if compare_to: + bench.print_comparison(compare_to, + hidenoise=hidenoise, + limitnames=limitnames) + else: + bench.print_benchmark(hidenoise=hidenoise, + limitnames=limitnames) + + # Ring bell sys.stderr.write('\007') if reportfile: diff --git a/Tools/pybench/systimes.py b/Tools/pybench/systimes.py index 79d249f..bf07e36 100644 --- a/Tools/pybench/systimes.py +++ b/Tools/pybench/systimes.py @@ -16,7 +16,7 @@ platforms. If no supported timing methods based on process time can be found, - the module reverts to the highest resolution wall-time timer + the module reverts to the highest resolution wall-clock timer instead. The system time part will then always be 0.0. The module exports one public API: @@ -52,8 +52,8 @@ USE_CTYPES_GETPROCESSTIMES = 'cytpes GetProcessTimes() wrapper' USE_WIN32PROCESS_GETPROCESSTIMES = 'win32process.GetProcessTimes()' USE_RESOURCE_GETRUSAGE = 'resource.getrusage()' USE_PROCESS_TIME_CLOCK = 'time.clock() (process time)' -USE_WALL_TIME_CLOCK = 'time.clock() (wall-time)' -USE_WALL_TIME_TIME = 'time.time() (wall-time)' +USE_WALL_TIME_CLOCK = 'time.clock() (wall-clock)' +USE_WALL_TIME_TIME = 'time.time() (wall-clock)' if sys.platform[:3] == 'win': # Windows platform @@ -63,7 +63,7 @@ if sys.platform[:3] == 'win': try: import ctypes except ImportError: - # Use the wall-time implementation time.clock(), since this + # Use the wall-clock implementation time.clock(), since this # is the highest resolution clock available on Windows SYSTIMES_IMPLEMENTATION = USE_WALL_TIME_CLOCK else: @@ -91,7 +91,7 @@ if SYSTIMES_IMPLEMENTATION is None: # time) SYSTIMES_IMPLEMENTATION = USE_PROCESS_TIME_CLOCK else: - # Use wall-time implementation time.time() since this provides + # Use wall-clock implementation time.time() since this provides # the highest resolution clock on most systems SYSTIMES_IMPLEMENTATION = USE_WALL_TIME_TIME @@ -103,24 +103,27 @@ def getrusage_systimes(): def process_time_clock_systimes(): return (time.clock(), 0.0) -def wall_time_clock_systimes(): +def wall_clock_clock_systimes(): return (time.clock(), 0.0) -def wall_time_time_systimes(): +def wall_clock_time_systimes(): return (time.time(), 0.0) # Number of clock ticks per second for the values returned # by GetProcessTimes() on Windows. # -# Note: Ticks returned by GetProcessTimes() are micro-seconds on -# Windows XP (though the docs say 100ns intervals) -WIN32_PROCESS_TIMES_TICKS_PER_SECOND = 10e6 +# Note: Ticks returned by GetProcessTimes() are 100ns intervals on +# Windows XP. However, the process times are only updated with every +# clock tick and the frequency of these is somewhat lower: depending +# on the OS version between 10ms and 15ms. Even worse, the process +# time seems to be allocated to process currently running when the +# clock interrupt arrives, ie. it is possible that the current time +# slice gets accounted to a different process. + +WIN32_PROCESS_TIMES_TICKS_PER_SECOND = 1e7 def win32process_getprocesstimes_systimes(): d = win32process.GetProcessTimes(win32process.GetCurrentProcess()) - # Note: I'm not sure whether KernelTime on Windows is the same as - # system time on Unix - I've yet to see a non-zero value for - # KernelTime on Windows. return (d['UserTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND, d['KernelTime'] / WIN32_PROCESS_TIMES_TICKS_PER_SECOND) @@ -149,10 +152,10 @@ elif SYSTIMES_IMPLEMENTATION is USE_PROCESS_TIME_CLOCK: systimes = process_time_clock_systimes elif SYSTIMES_IMPLEMENTATION is USE_WALL_TIME_CLOCK: - systimes = wall_time_clock_systimes + systimes = wall_clock_clock_systimes elif SYSTIMES_IMPLEMENTATION is USE_WALL_TIME_TIME: - systimes = wall_time_time_systimes + systimes = wall_clock_time_systimes elif SYSTIMES_IMPLEMENTATION is USE_WIN32PROCESS_GETPROCESSTIMES: systimes = win32process_getprocesstimes_systimes @@ -163,6 +166,17 @@ elif SYSTIMES_IMPLEMENTATION is USE_CTYPES_GETPROCESSTIMES: else: raise TypeError('no suitable systimes() implementation found') +def processtime(): + + """ Return the total time spent on the process. + + This is the sum of user and system time as returned by + systimes(). + + """ + user, system = systimes() + return user + system + ### Testing def some_workload(): diff --git a/Tools/scripts/README b/Tools/scripts/README index 56aa391..9c7b40e 100644 --- a/Tools/scripts/README +++ b/Tools/scripts/README @@ -27,7 +27,6 @@ fixheader.py Add some cpp magic to a C include file fixnotice.py Fix the copyright notice in source files fixps.py Fix Python scripts' first line (if #!) ftpmirror.py FTP mirror script -gencodec.py Create Python codecs from Unicode mapping files google.py Open a webbrowser with Google. gprof2html.py Transform gprof(1) output into useful HTML. h2py.py Translate #define's into Python assignments @@ -53,14 +52,14 @@ pindent.py Indent Python code, giving block-closing comments ptags.py Create vi tags file for Python modules pydoc Python documentation browser. pysource.py Find Python source files -redemo.py Basic regular expression demostration facility +redemo.py Basic regular expression demonstration facility reindent.py Change .py files to use 4-space indents. rgrep.py Reverse grep through a file (useful for big logfiles) setup.py Install all scripts listed here. suff.py Sort a list of files by suffix +svneol.py Sets svn:eol-style on all files in directory. texcheck.py Validate Python LaTeX formatting (Raymond Hettinger) texi2html.py Convert GNU texinfo files into HTML -trace.py Trace Python program or function execution treesync.py Synchronize source trees (very ideosyncratic) untabify.py Replace tabs with spaces in argument files which.py Find a program in $PATH diff --git a/Tools/webchecker/webchecker.py b/Tools/webchecker/webchecker.py index 7199da8..923e8e6 100755 --- a/Tools/webchecker/webchecker.py +++ b/Tools/webchecker/webchecker.py @@ -760,7 +760,8 @@ class MyURLopener(urllib.FancyURLopener): try: names = os.listdir(path) except os.error, msg: - raise IOError, msg, sys.exc_traceback + exc_type, exc_value, exc_tb = sys.exc_info() + raise IOError, msg, exc_tb names.sort() s = MyStringIO("file:"+url, {'content-type': 'text/html'}) s.write('\n' % diff --git a/configure b/configure index b02ebc6..033bb68 100755 --- a/configure +++ b/configure @@ -1,7 +1,7 @@ #! /bin/sh -# From configure.in Revision: 46753 . +# From configure.in Revision: 46754 . # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.59e for python 3.0. +# Generated by GNU Autoconf 2.60 for python 3.0. # # Report bugs to . # @@ -233,8 +233,8 @@ IFS=$as_save_IFS for as_shell in $as_candidate_shells $SHELL; do - # Try only shells which exist, to save several forks. - if test -f "$as_shell" && + # Try only shells that exist, to save several forks. + if { test -f "$as_shell" || test -f "$as_shell.exe"; } && { ("$as_shell") 2> /dev/null <<\_ASEOF # Be Bourne compatible if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then @@ -1408,7 +1408,7 @@ test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF python configure 3.0 -generated by GNU Autoconf 2.59e +generated by GNU Autoconf 2.60 Copyright (C) 1992, 1993, 1994, 1995, 1996, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software Foundation, Inc. @@ -1422,7 +1422,7 @@ This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by python $as_me 3.0, which was -generated by GNU Autoconf 2.59e. Invocation command line was +generated by GNU Autoconf 2.60. Invocation command line was $ $0 $@ @@ -2020,8 +2020,8 @@ case $ac_sys_system/$ac_sys_release in # On Mac OS X 10.4, defining _POSIX_C_SOURCE or _XOPEN_SOURCE # disables platform specific features beyond repair. # On Mac OS X 10.3, defining _POSIX_C_SOURCE or _XOPEN_SOURCE - # has no effect, don't bother defineing them - Darwin/[78].*) + # has no effect, don't bother defining them + Darwin/[789].*) define_xopen_source=no ;; @@ -2846,7 +2846,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_compiler_gnu=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_cv_c_compiler_gnu=$ac_compiler_gnu fi @@ -3032,13 +3032,13 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_c_werror_flag=$ac_save_c_werror_flag fi { echo "$as_me:$LINENO: result: $ac_cv_prog_cc_g" >&5 @@ -3103,6 +3103,11 @@ static char *f (char * (*g) (char **, int), char **p, ...) that's true only with -std. */ int osf4_cc_array ['\x00' == 0 ? 1 : -1]; +/* IBM C 6 for AIX is almost-ANSI by default, but it replaces macro parameters + inside strings and character constants. */ +#define FOO(x) 'x' +int xlc6_cc_array[FOO(a) == 'x' ? 1 : -1]; + int test (int i, double x); struct s1 {int (*f) (int a);}; struct s2 {int (*f) (double a);}; @@ -3163,7 +3168,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext +rm -f core conftest.err conftest.$ac_objext test "x$ac_cv_prog_cc_c89" != "xno" && break done rm -f conftest.$ac_ext @@ -5199,7 +5204,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_header_stdc=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext if test $ac_cv_header_stdc = yes; then # SunOS 4.x string.h does not declare mem*, contrary to ANSI. @@ -5395,7 +5400,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_Header=no" fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_Header'}'` { echo "$as_me:$LINENO: result: $ac_res" >&5 @@ -5450,14 +5455,24 @@ done -for ac_header in asm/types.h curses.h dlfcn.h fcntl.h grp.h \ -shadow.h langinfo.h libintl.h ncurses.h poll.h pthread.h \ -stropts.h termios.h thread.h \ + + + + + + + + +for ac_header in asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +fcntl.h grp.h \ +shadow.h io.h langinfo.h libintl.h ncurses.h poll.h process.h pthread.h \ +signal.h stropts.h termios.h thread.h \ unistd.h utime.h \ sys/audioio.h sys/bsdtty.h sys/file.h sys/loadavg.h sys/lock.h sys/mkdev.h \ sys/modem.h \ -sys/param.h sys/poll.h sys/select.h sys/socket.h sys/statvfs.h sys/time.h \ -sys/times.h sys/un.h sys/utsname.h sys/wait.h pty.h libutil.h \ +sys/param.h sys/poll.h sys/select.h sys/socket.h sys/statvfs.h sys/stat.h \ +sys/time.h \ +sys/times.h sys/types.h sys/un.h sys/utsname.h sys/wait.h pty.h libutil.h \ sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \ bluetooth/bluetooth.h do @@ -5526,7 +5541,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_header_compiler=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $ac_header_compiler" >&5 echo "${ECHO_T}$ac_header_compiler" >&6; } @@ -5700,7 +5715,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_Header=no" fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_Header'}'` { echo "$as_me:$LINENO: result: $ac_res" >&5 @@ -5793,7 +5808,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext if test "${ac_cv_search_opendir+set}" = set; then break @@ -5893,7 +5908,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext if test "${ac_cv_search_opendir+set}" = set; then break @@ -5979,7 +5994,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_header_sys_types_h_makedev=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi @@ -6050,7 +6065,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_header_compiler=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $ac_header_compiler" >&5 echo "${ECHO_T}$ac_header_compiler" >&6; } @@ -6215,7 +6230,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_header_compiler=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $ac_header_compiler" >&5 echo "${ECHO_T}$ac_header_compiler" >&6; } @@ -6385,7 +6400,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_Header=no" fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_Header'}'` { echo "$as_me:$LINENO: result: $ac_res" >&5 @@ -6469,7 +6484,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_Header=no" fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_Header'}'` { echo "$as_me:$LINENO: result: $ac_res" >&5 @@ -6574,7 +6589,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_has_makedev=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext if test "$ac_cv_has_makedev" = "no"; then # we didn't link, try if _OSF_SOURCE will allow us to link @@ -6638,7 +6653,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_has_makedev=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext if test "$ac_cv_has_makedev" = "yes"; then @@ -6729,7 +6744,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 sol_lfs_bug=yes fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $sol_lfs_bug" >&5 echo "${ECHO_T}$sol_lfs_bug" >&6; } if test "$sol_lfs_bug" = "yes"; then @@ -6825,7 +6840,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_mode_t=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_mode_t" >&5 echo "${ECHO_T}$ac_cv_type_mode_t" >&6; } @@ -6905,7 +6920,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_off_t=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_off_t" >&5 echo "${ECHO_T}$ac_cv_type_off_t" >&6; } @@ -6985,7 +7000,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_pid_t=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_pid_t" >&5 echo "${ECHO_T}$ac_cv_type_pid_t" >&6; } @@ -7063,7 +7078,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_signal=void fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_signal" >&5 echo "${ECHO_T}$ac_cv_type_signal" >&6; } @@ -7139,7 +7154,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_size_t=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_size_t" >&5 echo "${ECHO_T}$ac_cv_type_size_t" >&6; } @@ -7257,7 +7272,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_ssize_t=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_ssize_t" >&5 echo "${ECHO_T}$ac_cv_type_ssize_t" >&6; } @@ -7338,7 +7353,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_int=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_int" >&5 echo "${ECHO_T}$ac_cv_type_int" >&6; } @@ -7474,7 +7489,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -7599,7 +7614,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -7608,10 +7623,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo= ac_hi= fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # Binary search between lo and hi bounds. while test "x$ac_lo" != "x$ac_hi"; do ac_mid=`expr '(' $ac_hi - $ac_lo ')' / 2 + $ac_lo` @@ -7675,7 +7690,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo=`expr '(' $ac_mid ')' + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done case $ac_lo in ?*) ac_cv_sizeof_int=$ac_lo;; @@ -7838,7 +7853,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_long=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_long" >&5 echo "${ECHO_T}$ac_cv_type_long" >&6; } @@ -7974,7 +7989,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -8099,7 +8114,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -8108,10 +8123,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo= ac_hi= fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # Binary search between lo and hi bounds. while test "x$ac_lo" != "x$ac_hi"; do ac_mid=`expr '(' $ac_hi - $ac_lo ')' / 2 + $ac_lo` @@ -8175,7 +8190,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo=`expr '(' $ac_mid ')' + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done case $ac_lo in ?*) ac_cv_sizeof_long=$ac_lo;; @@ -8338,7 +8353,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_void_p=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_void_p" >&5 echo "${ECHO_T}$ac_cv_type_void_p" >&6; } @@ -8474,7 +8489,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -8599,7 +8614,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -8608,10 +8623,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo= ac_hi= fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # Binary search between lo and hi bounds. while test "x$ac_lo" != "x$ac_hi"; do ac_mid=`expr '(' $ac_hi - $ac_lo ')' / 2 + $ac_lo` @@ -8675,7 +8690,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo=`expr '(' $ac_mid ')' + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done case $ac_lo in ?*) ac_cv_sizeof_void_p=$ac_lo;; @@ -8838,7 +8853,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_short=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_short" >&5 echo "${ECHO_T}$ac_cv_type_short" >&6; } @@ -8974,7 +8989,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -9099,7 +9114,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -9108,10 +9123,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo= ac_hi= fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # Binary search between lo and hi bounds. while test "x$ac_lo" != "x$ac_hi"; do ac_mid=`expr '(' $ac_hi - $ac_lo ')' / 2 + $ac_lo` @@ -9175,7 +9190,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo=`expr '(' $ac_mid ')' + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done case $ac_lo in ?*) ac_cv_sizeof_short=$ac_lo;; @@ -9338,7 +9353,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_float=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_float" >&5 echo "${ECHO_T}$ac_cv_type_float" >&6; } @@ -9474,7 +9489,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -9599,7 +9614,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -9608,10 +9623,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo= ac_hi= fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # Binary search between lo and hi bounds. while test "x$ac_lo" != "x$ac_hi"; do ac_mid=`expr '(' $ac_hi - $ac_lo ')' / 2 + $ac_lo` @@ -9675,7 +9690,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo=`expr '(' $ac_mid ')' + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done case $ac_lo in ?*) ac_cv_sizeof_float=$ac_lo;; @@ -9838,7 +9853,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_double=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_double" >&5 echo "${ECHO_T}$ac_cv_type_double" >&6; } @@ -9974,7 +9989,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -10099,7 +10114,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -10108,10 +10123,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo= ac_hi= fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # Binary search between lo and hi bounds. while test "x$ac_lo" != "x$ac_hi"; do ac_mid=`expr '(' $ac_hi - $ac_lo ')' / 2 + $ac_lo` @@ -10175,7 +10190,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo=`expr '(' $ac_mid ')' + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done case $ac_lo in ?*) ac_cv_sizeof_double=$ac_lo;; @@ -10338,7 +10353,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_fpos_t=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_fpos_t" >&5 echo "${ECHO_T}$ac_cv_type_fpos_t" >&6; } @@ -10474,7 +10489,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -10599,7 +10614,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -10608,10 +10623,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo= ac_hi= fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # Binary search between lo and hi bounds. while test "x$ac_lo" != "x$ac_hi"; do ac_mid=`expr '(' $ac_hi - $ac_lo ')' / 2 + $ac_lo` @@ -10675,7 +10690,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo=`expr '(' $ac_mid ')' + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done case $ac_lo in ?*) ac_cv_sizeof_fpos_t=$ac_lo;; @@ -10838,7 +10853,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_size_t=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_size_t" >&5 echo "${ECHO_T}$ac_cv_type_size_t" >&6; } @@ -10974,7 +10989,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -11099,7 +11114,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -11108,10 +11123,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo= ac_hi= fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # Binary search between lo and hi bounds. while test "x$ac_lo" != "x$ac_hi"; do ac_mid=`expr '(' $ac_hi - $ac_lo ')' / 2 + $ac_lo` @@ -11175,7 +11190,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo=`expr '(' $ac_mid ')' + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done case $ac_lo in ?*) ac_cv_sizeof_size_t=$ac_lo;; @@ -11340,7 +11355,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $have_long_long" >&5 echo "${ECHO_T}$have_long_long" >&6; } if test "$have_long_long" = yes ; then @@ -11410,7 +11425,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_long_long=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_long_long" >&5 echo "${ECHO_T}$ac_cv_type_long_long" >&6; } @@ -11546,7 +11561,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -11671,7 +11686,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -11680,10 +11695,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo= ac_hi= fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # Binary search between lo and hi bounds. while test "x$ac_lo" != "x$ac_hi"; do ac_mid=`expr '(' $ac_hi - $ac_lo ')' / 2 + $ac_lo` @@ -11747,7 +11762,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo=`expr '(' $ac_mid ')' + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done case $ac_lo in ?*) ac_cv_sizeof_long_long=$ac_lo;; @@ -11913,7 +11928,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $have_uintptr_t" >&5 echo "${ECHO_T}$have_uintptr_t" >&6; } if test "$have_uintptr_t" = yes ; then @@ -11983,7 +11998,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_uintptr_t=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_uintptr_t" >&5 echo "${ECHO_T}$ac_cv_type_uintptr_t" >&6; } @@ -12119,7 +12134,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -12244,7 +12259,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -12253,10 +12268,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo= ac_hi= fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # Binary search between lo and hi bounds. while test "x$ac_lo" != "x$ac_hi"; do ac_mid=`expr '(' $ac_hi - $ac_lo ')' / 2 + $ac_lo` @@ -12320,7 +12335,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo=`expr '(' $ac_mid ')' + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done case $ac_lo in ?*) ac_cv_sizeof_uintptr_t=$ac_lo;; @@ -12644,7 +12659,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $have_pthread_t" >&5 echo "${ECHO_T}$have_pthread_t" >&6; } if test "$have_pthread_t" = yes ; then @@ -13182,7 +13197,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_dl_dlopen=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -13269,7 +13284,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_dld_shl_load=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -13364,7 +13379,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext if test "${ac_cv_search_sem_init+set}" = set; then break @@ -13462,7 +13477,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_intl_textdomain=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -13545,7 +13560,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext;; *) ;; esac @@ -13623,7 +13638,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_nsl_t_open=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -13704,7 +13719,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_socket_socket=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -13788,7 +13803,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_net_socket=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -14071,7 +14086,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_header_compiler=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $ac_header_compiler" >&5 echo "${ECHO_T}$ac_header_compiler" >&6; } @@ -14243,7 +14258,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_header_compiler=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $ac_header_compiler" >&5 echo "${ECHO_T}$ac_header_compiler" >&6; } @@ -14539,7 +14554,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_func_pthread_detach=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_func_pthread_detach" >&5 @@ -14616,7 +14631,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_header_compiler=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $ac_header_compiler" >&5 echo "${ECHO_T}$ac_header_compiler" >&6; } @@ -14783,7 +14798,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_header_compiler=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $ac_header_compiler" >&5 echo "${ECHO_T}$ac_header_compiler" >&6; } @@ -14958,7 +14973,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_pthreads_pthread_create=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -15045,7 +15060,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_c_r_pthread_create=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -15132,7 +15147,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_pthread___pthread_create_system=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -15219,7 +15234,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_cma_pthread_create=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -15255,7 +15270,7 @@ fi fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi @@ -15337,7 +15352,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_mpc_usconfig=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -15426,7 +15441,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_thread_thr_create=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -15647,7 +15662,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -15824,7 +15839,7 @@ echo "${ECHO_T}no" >&6; } ipv6=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi if test "$ipv6" = "yes"; then @@ -16219,7 +16234,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -16471,7 +16486,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -16555,7 +16570,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: checking for link" >&5 echo $ECHO_N "checking for link... $ECHO_C" >&6; } cat >conftest.$ac_ext <<_ACEOF @@ -16623,7 +16638,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: checking for symlink" >&5 echo $ECHO_N "checking for symlink... $ECHO_C" >&6; } cat >conftest.$ac_ext <<_ACEOF @@ -16691,7 +16706,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: checking for fchdir" >&5 echo $ECHO_N "checking for fchdir... $ECHO_C" >&6; } cat >conftest.$ac_ext <<_ACEOF @@ -16759,7 +16774,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: checking for fsync" >&5 echo $ECHO_N "checking for fsync... $ECHO_C" >&6; } cat >conftest.$ac_ext <<_ACEOF @@ -16827,7 +16842,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: checking for fdatasync" >&5 echo $ECHO_N "checking for fdatasync... $ECHO_C" >&6; } cat >conftest.$ac_ext <<_ACEOF @@ -16895,7 +16910,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # On some systems (eg. FreeBSD 5), we would find a definition of the # functions ctermid_r, setgroups in the library, but no prototype @@ -16973,7 +16988,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: checking for flock" >&5 echo $ECHO_N "checking for flock... $ECHO_C" >&6; } @@ -17045,7 +17060,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: checking for getpagesize" >&5 echo $ECHO_N "checking for getpagesize... $ECHO_C" >&6; } @@ -17117,7 +17132,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext for ac_prog in true do @@ -17234,7 +17249,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_c_inet_aton=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -17315,7 +17330,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_resolv_inet_aton=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -17334,6 +17349,108 @@ fi fi +case $ac_sys_system/$ac_sys_release in +Darwin/*) + _CUR_CFLAGS="${CFLAGS}" + _CUR_LDFLAGS="${LDFLAGS}" + CFLAGS="${CFLAGS} -Wl,-search_paths_first" + LDFLAGS="${LDFLAGS} -Wl,-search_paths_first -L/usr/local/lib" + ;; +esac + +{ echo "$as_me:$LINENO: checking for inflateCopy in -lz" >&5 +echo $ECHO_N "checking for inflateCopy in -lz... $ECHO_C" >&6; } +if test "${ac_cv_lib_z_inflateCopy+set}" = set; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + ac_check_lib_save_LIBS=$LIBS +LIBS="-lz $LIBS" +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ + +/* Override any GCC internal prototype to avoid an error. + Use char because int might match the return type of a GCC + builtin and then its argument prototype would still apply. */ +#ifdef __cplusplus +extern "C" +#endif +char inflateCopy (); +int +main () +{ +return inflateCopy (); + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext conftest$ac_exeext +if { (ac_try="$ac_link" +case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_link") 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' + { (case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_try") 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest$ac_exeext' + { (case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_try") 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + ac_cv_lib_z_inflateCopy=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + + ac_cv_lib_z_inflateCopy=no +fi + +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext +LIBS=$ac_check_lib_save_LIBS +fi +{ echo "$as_me:$LINENO: result: $ac_cv_lib_z_inflateCopy" >&5 +echo "${ECHO_T}$ac_cv_lib_z_inflateCopy" >&6; } +if test $ac_cv_lib_z_inflateCopy = yes; then + +cat >>confdefs.h <<\_ACEOF +#define HAVE_ZLIB_COPY 1 +_ACEOF + +fi + + +case $ac_sys_system/$ac_sys_release in +Darwin/*) + CFLAGS="${_CUR_CFLAGS}" + LDFLAGS="${_CUR_LDFLAGS}" + ;; +esac + { echo "$as_me:$LINENO: checking for hstrerror" >&5 echo $ECHO_N "checking for hstrerror... $ECHO_C" >&6; } cat >conftest.$ac_ext <<_ACEOF @@ -17404,7 +17521,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext { echo "$as_me:$LINENO: checking for inet_aton" >&5 @@ -17480,7 +17597,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext { echo "$as_me:$LINENO: checking for inet_pton" >&5 @@ -17556,7 +17673,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # On some systems, setgroups is in unistd.h, on others, in grp.h { echo "$as_me:$LINENO: checking for setgroups" >&5 @@ -17632,7 +17749,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # check for openpty and forkpty @@ -17732,7 +17849,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -17815,7 +17932,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_util_openpty=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -17898,7 +18015,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_bsd_openpty=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -18014,7 +18131,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -18097,7 +18214,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_util_forkpty=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -18180,7 +18297,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_bsd_forkpty=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -18303,7 +18420,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -18418,7 +18535,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -18536,7 +18653,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -18608,7 +18725,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi done @@ -18709,7 +18826,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -18781,7 +18898,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi done @@ -18882,7 +18999,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -18954,7 +19071,7 @@ _ACEOF fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi done @@ -19040,7 +19157,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext # On OSF/1 V5.1, getaddrinfo is available, but a define @@ -19255,7 +19372,7 @@ buggygetaddrinfo=yes fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext if test "$buggygetaddrinfo" = "yes"; then @@ -19367,7 +19484,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -19449,7 +19566,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_header_time=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_header_time" >&5 echo "${ECHO_T}$ac_cv_header_time" >&6; } @@ -19525,7 +19642,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_struct_tm=sys/time.h fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_struct_tm" >&5 echo "${ECHO_T}$ac_cv_struct_tm" >&6; } @@ -19663,10 +19780,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_member_struct_tm_tm_zone=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_member_struct_tm_tm_zone" >&5 echo "${ECHO_T}$ac_cv_member_struct_tm_tm_zone" >&6; } @@ -19753,7 +19870,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_have_decl_tzname=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_have_decl_tzname" >&5 echo "${ECHO_T}$ac_cv_have_decl_tzname" >&6; } @@ -19839,7 +19956,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_var_tzname=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_var_tzname" >&5 @@ -19973,10 +20090,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_member_struct_stat_st_rdev=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_member_struct_stat_st_rdev" >&5 echo "${ECHO_T}$ac_cv_member_struct_stat_st_rdev" >&6; } @@ -20109,10 +20226,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_member_struct_stat_st_blksize=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_member_struct_stat_st_blksize" >&5 echo "${ECHO_T}$ac_cv_member_struct_stat_st_blksize" >&6; } @@ -20245,10 +20362,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_member_struct_stat_st_flags=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_member_struct_stat_st_flags" >&5 echo "${ECHO_T}$ac_cv_member_struct_stat_st_flags" >&6; } @@ -20381,10 +20498,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_member_struct_stat_st_gen=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_member_struct_stat_st_gen" >&5 echo "${ECHO_T}$ac_cv_member_struct_stat_st_gen" >&6; } @@ -20517,10 +20634,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_member_struct_stat_st_birthtime=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_member_struct_stat_st_birthtime" >&5 echo "${ECHO_T}$ac_cv_member_struct_stat_st_birthtime" >&6; } @@ -20653,10 +20770,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_member_struct_stat_st_blocks=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_member_struct_stat_st_blocks" >&5 echo "${ECHO_T}$ac_cv_member_struct_stat_st_blocks" >&6; } @@ -20744,7 +20861,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_header_time_altzone=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_header_time_altzone" >&5 @@ -20828,7 +20945,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $was_it_defined" >&5 echo "${ECHO_T}$was_it_defined" >&6; } @@ -20895,7 +21012,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_struct_addrinfo=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_struct_addrinfo" >&5 @@ -20972,7 +21089,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_struct_sockaddr_storage=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_struct_sockaddr_storage" >&5 @@ -21052,7 +21169,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_c_char_unsigned=yes fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_c_char_unsigned" >&5 echo "${ECHO_T}$ac_cv_c_char_unsigned" >&6; } @@ -21174,7 +21291,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_c_const=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_c_const" >&5 echo "${ECHO_T}$ac_cv_c_const" >&6; } @@ -21252,7 +21369,7 @@ _ACEOF fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $works" >&5 echo "${ECHO_T}$works" >&6; } @@ -21321,7 +21438,7 @@ _ACEOF fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $works" >&5 echo "${ECHO_T}$works" >&6; } @@ -21392,7 +21509,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $have_prototypes" >&5 echo "${ECHO_T}$have_prototypes" >&6; } @@ -21473,7 +21590,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $works" >&5 echo "${ECHO_T}$works" >&6; } @@ -21548,7 +21665,7 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # check if sockaddr has sa_len member { echo "$as_me:$LINENO: checking if sockaddr has sa_len member" >&5 @@ -21619,7 +21736,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext va_list_is_array=no { echo "$as_me:$LINENO: checking whether va_list is an array" >&5 @@ -21694,7 +21811,7 @@ _ACEOF fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $va_list_is_array" >&5 echo "${ECHO_T}$va_list_is_array" >&6; } @@ -21794,7 +21911,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_func_gethostbyname_r=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_func_gethostbyname_r" >&5 @@ -22051,15 +22168,15 @@ echo "${ECHO_T}no" >&6; } fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext CFLAGS=$OLD_CFLAGS else @@ -22160,7 +22277,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -22279,7 +22396,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_func___fpu_control=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_func___fpu_control" >&5 @@ -22359,7 +22476,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_ieee___fpu_control=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -22559,7 +22676,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 eval "$as_ac_var=no" fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi ac_res=`eval echo '${'$as_ac_var'}'` @@ -22647,7 +22764,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_header_compiler=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $ac_header_compiler" >&5 echo "${ECHO_T}$ac_header_compiler" >&6; } @@ -22824,7 +22941,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_wchar_t=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_wchar_t" >&5 echo "${ECHO_T}$ac_cv_type_wchar_t" >&6; } @@ -22962,7 +23079,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -23089,7 +23206,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_mid=`expr 2 '*' $ac_mid` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done else echo "$as_me: failed program was:" >&5 @@ -23098,10 +23215,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo= ac_hi= fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext # Binary search between lo and hi bounds. while test "x$ac_lo" != "x$ac_hi"; do ac_mid=`expr '(' $ac_hi - $ac_lo ')' / 2 + $ac_lo` @@ -23166,7 +23283,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_lo=`expr '(' $ac_mid ')' + 1` fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext done case $ac_lo in ?*) ac_cv_sizeof_wchar_t=$ac_lo;; @@ -23337,7 +23454,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { echo "$as_me:$LINENO: result: $have_ucs4_tcl" >&5 echo "${ECHO_T}$have_ucs4_tcl" >&6; } @@ -23622,7 +23739,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_c_bigendian=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext else echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 @@ -23703,7 +23820,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext else cat >conftest.$ac_ext <<_ACEOF /* confdefs.h. */ @@ -23764,7 +23881,7 @@ fi fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_c_bigendian" >&5 echo "${ECHO_T}$ac_cv_c_bigendian" >&6; } @@ -23927,7 +24044,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_have_getc_unlocked=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi @@ -24017,7 +24134,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_readline_readline=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -24106,7 +24223,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_termcap_readline=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -24195,7 +24312,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_readline_rl_callback_handler_install=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -24346,7 +24463,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_readline_rl_pre_input_hook=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -24433,7 +24550,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_lib_readline_rl_completion_matches=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi @@ -24796,10 +24913,10 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_member_struct_tm_tm_zone=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_member_struct_tm_tm_zone" >&5 echo "${ECHO_T}$ac_cv_member_struct_tm_tm_zone" >&6; } @@ -24886,7 +25003,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_have_decl_tzname=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_have_decl_tzname" >&5 echo "${ECHO_T}$ac_cv_have_decl_tzname" >&6; } @@ -24972,7 +25089,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_var_tzname=no fi -rm -f conftest.err conftest.$ac_objext \ +rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_var_tzname" >&5 @@ -25180,7 +25297,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_stat_tv_nsec=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_stat_tv_nsec" >&5 @@ -25260,7 +25377,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_stat_tv_nsec2=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_stat_tv_nsec2" >&5 @@ -25340,7 +25457,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_mvwdelch_is_expression=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_mvwdelch_is_expression" >&5 @@ -25420,7 +25537,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_window_has_flags=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_window_has_flags" >&5 @@ -25436,39 +25553,263 @@ _ACEOF fi -{ echo "$as_me:$LINENO: checking for /dev/ptmx" >&5 -echo $ECHO_N "checking for /dev/ptmx... $ECHO_C" >&6; } +{ echo "$as_me:$LINENO: checking for is_term_resized" >&5 +echo $ECHO_N "checking for is_term_resized... $ECHO_C" >&6; } +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +#include +int +main () +{ +void *x=is_term_resized + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (ac_try="$ac_compile" +case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_compile") 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' + { (case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_try") 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_try") 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + +cat >>confdefs.h <<\_ACEOF +#define HAVE_CURSES_IS_TERM_RESIZED 1 +_ACEOF -if test -e /dev/ptmx -then { echo "$as_me:$LINENO: result: yes" >&5 echo "${ECHO_T}yes" >&6; } +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + + { echo "$as_me:$LINENO: result: no" >&5 +echo "${ECHO_T}no" >&6; } + +fi + +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext + +{ echo "$as_me:$LINENO: checking for resize_term" >&5 +echo $ECHO_N "checking for resize_term... $ECHO_C" >&6; } +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +#include +int +main () +{ +void *x=resize_term + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (ac_try="$ac_compile" +case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_compile") 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' + { (case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_try") 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_try") 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then cat >>confdefs.h <<\_ACEOF -#define HAVE_DEV_PTMX 1 +#define HAVE_CURSES_RESIZE_TERM 1 _ACEOF + { echo "$as_me:$LINENO: result: yes" >&5 +echo "${ECHO_T}yes" >&6; } else - { echo "$as_me:$LINENO: result: no" >&5 + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + + { echo "$as_me:$LINENO: result: no" >&5 echo "${ECHO_T}no" >&6; } + fi -{ echo "$as_me:$LINENO: checking for /dev/ptc" >&5 -echo $ECHO_N "checking for /dev/ptc... $ECHO_C" >&6; } +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext + +{ echo "$as_me:$LINENO: checking for resizeterm" >&5 +echo $ECHO_N "checking for resizeterm... $ECHO_C" >&6; } +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +#include +int +main () +{ +void *x=resizeterm + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (ac_try="$ac_compile" +case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_compile") 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" || test ! -s conftest.err' + { (case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_try") 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (case "(($ac_try" in + *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; + *) ac_try_echo=$ac_try;; +esac +eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 + (eval "$ac_try") 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + +cat >>confdefs.h <<\_ACEOF +#define HAVE_CURSES_RESIZETERM 1 +_ACEOF -if test -e /dev/ptc -then { echo "$as_me:$LINENO: result: yes" >&5 echo "${ECHO_T}yes" >&6; } +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + + { echo "$as_me:$LINENO: result: no" >&5 +echo "${ECHO_T}no" >&6; } + +fi + +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext + +{ echo "$as_me:$LINENO: checking for /dev/ptmx" >&5 +echo $ECHO_N "checking for /dev/ptmx... $ECHO_C" >&6; } +if test "${ac_cv_file__dev_ptmx+set}" = set; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + test "$cross_compiling" = yes && + { { echo "$as_me:$LINENO: error: cannot check for file existence when cross compiling" >&5 +echo "$as_me: error: cannot check for file existence when cross compiling" >&2;} + { (exit 1); exit 1; }; } +if test -r "/dev/ptmx"; then + ac_cv_file__dev_ptmx=yes +else + ac_cv_file__dev_ptmx=no +fi +fi +{ echo "$as_me:$LINENO: result: $ac_cv_file__dev_ptmx" >&5 +echo "${ECHO_T}$ac_cv_file__dev_ptmx" >&6; } +if test $ac_cv_file__dev_ptmx = yes; then cat >>confdefs.h <<\_ACEOF -#define HAVE_DEV_PTC 1 +#define HAVE_DEV_PTMX 1 _ACEOF +fi + +{ echo "$as_me:$LINENO: checking for /dev/ptc" >&5 +echo $ECHO_N "checking for /dev/ptc... $ECHO_C" >&6; } +if test "${ac_cv_file__dev_ptc+set}" = set; then + echo $ECHO_N "(cached) $ECHO_C" >&6 else - { echo "$as_me:$LINENO: result: no" >&5 -echo "${ECHO_T}no" >&6; } + test "$cross_compiling" = yes && + { { echo "$as_me:$LINENO: error: cannot check for file existence when cross compiling" >&5 +echo "$as_me: error: cannot check for file existence when cross compiling" >&2;} + { (exit 1); exit 1; }; } +if test -r "/dev/ptc"; then + ac_cv_file__dev_ptc=yes +else + ac_cv_file__dev_ptc=no +fi fi +{ echo "$as_me:$LINENO: result: $ac_cv_file__dev_ptc" >&5 +echo "${ECHO_T}$ac_cv_file__dev_ptc" >&6; } +if test $ac_cv_file__dev_ptc = yes; then + +cat >>confdefs.h <<\_ACEOF +#define HAVE_DEV_PTC 1 +_ACEOF + +fi + { echo "$as_me:$LINENO: checking for %zd printf() format support" >&5 echo $ECHO_N "checking for %zd printf() format support... $ECHO_C" >&6; } @@ -25617,7 +25958,7 @@ sed 's/^/| /' conftest.$ac_ext >&5 ac_cv_type_socklen_t=no fi -rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { echo "$as_me:$LINENO: result: $ac_cv_type_socklen_t" >&5 echo "${ECHO_T}$ac_cv_type_socklen_t" >&6; } @@ -26038,7 +26379,7 @@ exec 6>&1 # values after options handling. ac_log=" This file was extended by python $as_me 3.0, which was -generated by GNU Autoconf 2.59e. Invocation command line was +generated by GNU Autoconf 2.60. Invocation command line was CONFIG_FILES = $CONFIG_FILES CONFIG_HEADERS = $CONFIG_HEADERS @@ -26087,7 +26428,7 @@ _ACEOF cat >>$CONFIG_STATUS <<_ACEOF ac_cs_version="\\ python config.status 3.0 -configured by $0, generated by GNU Autoconf 2.59e, +configured by $0, generated by GNU Autoconf 2.60, with options \\"`echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`\\" Copyright (C) 2006 Free Software Foundation, Inc. @@ -26705,6 +27046,7 @@ $ac_datarootdir_hack test -z "$ac_datarootdir_hack$ac_datarootdir_seen" && { ac_out=`sed -n '/\${datarootdir}/p' "$tmp/out"`; test -n "$ac_out"; } && + { ac_out=`sed -n '/^[ ]*datarootdir[ ]*:*=/p' "$tmp/out"`; test -z "$ac_out"; } && { echo "$as_me:$LINENO: WARNING: $ac_file contains a reference to the variable \`datarootdir' which seems to be undefined. Please make sure it is defined." >&5 echo "$as_me: WARNING: $ac_file contains a reference to the variable \`datarootdir' diff --git a/configure.in b/configure.in index d9626f9..5d44cce 100644 --- a/configure.in +++ b/configure.in @@ -235,8 +235,8 @@ case $ac_sys_system/$ac_sys_release in # On Mac OS X 10.4, defining _POSIX_C_SOURCE or _XOPEN_SOURCE # disables platform specific features beyond repair. # On Mac OS X 10.3, defining _POSIX_C_SOURCE or _XOPEN_SOURCE - # has no effect, don't bother defineing them - Darwin/@<:@78@:>@.*) + # has no effect, don't bother defining them + Darwin/@<:@789@:>@.*) define_xopen_source=no ;; @@ -1066,14 +1066,16 @@ dnl AC_MSG_RESULT($cpp_type) # checks for header files AC_HEADER_STDC -AC_CHECK_HEADERS(asm/types.h curses.h dlfcn.h fcntl.h grp.h \ -shadow.h langinfo.h libintl.h ncurses.h poll.h pthread.h \ -stropts.h termios.h thread.h \ +AC_CHECK_HEADERS(asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ +fcntl.h grp.h \ +shadow.h io.h langinfo.h libintl.h ncurses.h poll.h process.h pthread.h \ +signal.h stropts.h termios.h thread.h \ unistd.h utime.h \ sys/audioio.h sys/bsdtty.h sys/file.h sys/loadavg.h sys/lock.h sys/mkdev.h \ sys/modem.h \ -sys/param.h sys/poll.h sys/select.h sys/socket.h sys/statvfs.h sys/time.h \ -sys/times.h sys/un.h sys/utsname.h sys/wait.h pty.h libutil.h \ +sys/param.h sys/poll.h sys/select.h sys/socket.h sys/statvfs.h sys/stat.h \ +sys/time.h \ +sys/times.h sys/types.h sys/un.h sys/utsname.h sys/wait.h pty.h libutil.h \ sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \ bluetooth/bluetooth.h) AC_HEADER_DIRENT @@ -2349,6 +2351,35 @@ AC_CHECK_LIB(c, inet_aton, [$ac_cv_prog_TRUE], AC_CHECK_LIB(resolv, inet_aton) ) +dnl Check if system zlib has *Copy() functions +dnl +dnl On MacOSX the linker will search for dylibs on the entire linker path +dnl before searching for static libraries. setup.py adds -Wl,-search_paths_first +dnl to revert to a more traditional unix behaviour and make it possible to +dnl override the system libz with a local static library of libz. Temporarily +dnl add that flag to our CFLAGS as well to ensure that we check the version +dnl of libz that will be used by setup.py. +dnl The -L/usr/local/lib is needed as wel to get the same compilation +dnl environment as setup.py (and leaving it out can cause configure to use the +dnl wrong version of the library) +case $ac_sys_system/$ac_sys_release in +Darwin/*) + _CUR_CFLAGS="${CFLAGS}" + _CUR_LDFLAGS="${LDFLAGS}" + CFLAGS="${CFLAGS} -Wl,-search_paths_first" + LDFLAGS="${LDFLAGS} -Wl,-search_paths_first -L/usr/local/lib" + ;; +esac + +AC_CHECK_LIB(z, inflateCopy, AC_DEFINE(HAVE_ZLIB_COPY, 1, Define if the zlib library has inflateCopy)) + +case $ac_sys_system/$ac_sys_release in +Darwin/*) + CFLAGS="${_CUR_CFLAGS}" + LDFLAGS="${_CUR_LDFLAGS}" + ;; +esac + AC_MSG_CHECKING(for hstrerror) AC_TRY_LINK([ #include "confdefs.h" @@ -3290,27 +3321,29 @@ then [Define if WINDOW in curses.h offers a field _flags.]) fi -AC_MSG_CHECKING(for /dev/ptmx) - -if test -e /dev/ptmx -then - AC_MSG_RESULT(yes) - AC_DEFINE(HAVE_DEV_PTMX, 1, - [Define if we have /dev/ptmx.]) -else +AC_MSG_CHECKING(for is_term_resized) +AC_TRY_COMPILE([#include ], void *x=is_term_resized, + AC_DEFINE(HAVE_CURSES_IS_TERM_RESIZED, 1, Define if you have the 'is_term_resized' function.) + AC_MSG_RESULT(yes), AC_MSG_RESULT(no) -fi +) -AC_MSG_CHECKING(for /dev/ptc) +AC_MSG_CHECKING(for resize_term) +AC_TRY_COMPILE([#include ], void *x=resize_term, + AC_DEFINE(HAVE_CURSES_RESIZE_TERM, 1, Define if you have the 'resize_term' function.) + AC_MSG_RESULT(yes), + AC_MSG_RESULT(no) +) -if test -e /dev/ptc -then - AC_MSG_RESULT(yes) - AC_DEFINE(HAVE_DEV_PTC, 1, - [Define if we have /dev/ptc.]) -else +AC_MSG_CHECKING(for resizeterm) +AC_TRY_COMPILE([#include ], void *x=resizeterm, + AC_DEFINE(HAVE_CURSES_RESIZETERM, 1, Define if you have the 'resizeterm' function.) + AC_MSG_RESULT(yes), AC_MSG_RESULT(no) -fi +) + +AC_CHECK_FILE(/dev/ptmx, AC_DEFINE(HAVE_DEV_PTMX, 1, [Define if we have /dev/ptmx.])) +AC_CHECK_FILE(/dev/ptc, AC_DEFINE(HAVE_DEV_PTC, 1, [Define if we have /dev/ptc.])) AC_MSG_CHECKING(for %zd printf() format support) AC_TRY_RUN([#include diff --git a/pyconfig.h.in b/pyconfig.h.in index 1468bf6..a3d7f6e 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -73,6 +73,9 @@ /* Define to 1 if you have the `confstr' function. */ #undef HAVE_CONFSTR +/* Define to 1 if you have the header file. */ +#undef HAVE_CONIO_H + /* Define to 1 if you have the `ctermid' function. */ #undef HAVE_CTERMID @@ -82,6 +85,15 @@ /* Define to 1 if you have the header file. */ #undef HAVE_CURSES_H +/* Define if you have the 'is_term_resized' function. */ +#undef HAVE_CURSES_IS_TERM_RESIZED + +/* Define if you have the 'resizeterm' function. */ +#undef HAVE_CURSES_RESIZETERM + +/* Define if you have the 'resize_term' function. */ +#undef HAVE_CURSES_RESIZE_TERM + /* Define to 1 if you have the device macros. */ #undef HAVE_DEVICE_MACROS @@ -91,6 +103,9 @@ /* Define if we have /dev/ptmx. */ #undef HAVE_DEV_PTMX +/* Define to 1 if you have the header file. */ +#undef HAVE_DIRECT_H + /* Define to 1 if you have the header file, and it defines `DIR'. */ #undef HAVE_DIRENT_H @@ -107,6 +122,9 @@ /* Defined when any dynamic module loading is enabled. */ #undef HAVE_DYNAMIC_LOADING +/* Define to 1 if you have the header file. */ +#undef HAVE_ERRNO_H + /* Define to 1 if you have the `execv' function. */ #undef HAVE_EXECV @@ -248,6 +266,9 @@ /* Define to 1 if you have the header file. */ #undef HAVE_INTTYPES_H +/* Define to 1 if you have the header file. */ +#undef HAVE_IO_H + /* Define to 1 if you have the `kill' function. */ #undef HAVE_KILL @@ -353,6 +374,9 @@ /* Define to 1 if you have the header file. */ #undef HAVE_POLL_H +/* Define to 1 if you have the header file. */ +#undef HAVE_PROCESS_H + /* Define if your compiler supports function prototype */ #undef HAVE_PROTOTYPES @@ -446,6 +470,9 @@ /* Define to 1 if you have the `siginterrupt' function. */ #undef HAVE_SIGINTERRUPT +/* Define to 1 if you have the header file. */ +#undef HAVE_SIGNAL_H + /* Define to 1 if you have the `sigrelse' function. */ #undef HAVE_SIGRELSE @@ -692,6 +719,9 @@ */ #undef HAVE_WORKING_TZSET +/* Define if the zlib library has inflateCopy */ +#undef HAVE_ZLIB_COPY + /* Define to 1 if you have the `_getpty' function. */ #undef HAVE__GETPTY diff --git a/setup.py b/setup.py index bd9b8b8..5d71181 100644 --- a/setup.py +++ b/setup.py @@ -638,6 +638,24 @@ class PyBuildExt(build_ext): db_inc_paths.append('/pkg/db-3.%d/include' % x) db_inc_paths.append('/opt/db-3.%d/include' % x) + # Add some common subdirectories for Sleepycat DB to the list, + # based on the standard include directories. This way DB3/4 gets + # picked up when it is installed in a non-standard prefix and + # the user has added that prefix into inc_dirs. + std_variants = [] + for dn in inc_dirs: + std_variants.append(os.path.join(dn, 'db3')) + std_variants.append(os.path.join(dn, 'db4')) + for x in (0,1,2,3,4): + std_variants.append(os.path.join(dn, "db4%d"%x)) + std_variants.append(os.path.join(dn, "db4.%d"%x)) + for x in (2,3): + std_variants.append(os.path.join(dn, "db3%d"%x)) + std_variants.append(os.path.join(dn, "db3.%d"%x)) + + db_inc_paths = std_variants + db_inc_paths + + db_ver_inc_map = {} class db_found(Exception): pass @@ -884,8 +902,12 @@ class PyBuildExt(build_ext): # Curses support, requiring the System V version of curses, often # provided by the ncurses library. + panel_library = 'panel' if (self.compiler.find_library_file(lib_dirs, 'ncursesw')): curses_libs = ['ncursesw'] + # Bug 1464056: If _curses.so links with ncursesw, + # _curses_panel.so must link with panelw. + panel_library = 'panelw' exts.append( Extension('_curses', ['_cursesmodule.c'], libraries = curses_libs) ) elif (self.compiler.find_library_file(lib_dirs, 'ncurses')): @@ -908,9 +930,9 @@ class PyBuildExt(build_ext): # If the curses module is enabled, check for the panel module if (module_enabled(exts, '_curses') and - self.compiler.find_library_file(lib_dirs, 'panel')): + self.compiler.find_library_file(lib_dirs, panel_library)): exts.append( Extension('_curses_panel', ['_curses_panel.c'], - libraries = ['panel'] + curses_libs) ) + libraries = [panel_library] + curses_libs) ) # Andrew Kuchling's zlib module. Note that some versions of zlib @@ -940,13 +962,23 @@ class PyBuildExt(build_ext): break if version >= version_req: if (self.compiler.find_library_file(lib_dirs, 'z')): + if sys.platform == "darwin": + zlib_extra_link_args = ('-Wl,-search_paths_first',) + else: + zlib_extra_link_args = () exts.append( Extension('zlib', ['zlibmodule.c'], - libraries = ['z']) ) + libraries = ['z'], + extra_link_args = zlib_extra_link_args)) # Gustavo Niemeyer's bz2 module. if (self.compiler.find_library_file(lib_dirs, 'bz2')): + if sys.platform == "darwin": + bz2_extra_link_args = ('-Wl,-search_paths_first',) + else: + bz2_extra_link_args = () exts.append( Extension('bz2', ['bz2module.c'], - libraries = ['bz2']) ) + libraries = ['bz2'], + extra_link_args = bz2_extra_link_args) ) # Interface to the Expat XML parser # @@ -1321,6 +1353,7 @@ class PyBuildExt(build_ext): self.use_system_libffi = False include_dirs = [] extra_compile_args = [] + extra_link_args = [] sources = ['_ctypes/_ctypes.c', '_ctypes/callbacks.c', '_ctypes/callproc.c', @@ -1335,9 +1368,21 @@ class PyBuildExt(build_ext): # XXX Is this still needed? ## extra_link_args.extend(['-read_only_relocs', 'warning']) + elif sys.platform == 'sunos5': + # XXX This shouldn't be necessary; it appears that some + # of the assembler code is non-PIC (i.e. it has relocations + # when it shouldn't. The proper fix would be to rewrite + # the assembler code to be PIC. + # This only works with GCC; the Sun compiler likely refuses + # this option. If you want to compile ctypes with the Sun + # compiler, please research a proper solution, instead of + # finding some -z option for the Sun compiler. + extra_link_args.append('-mimpure-text') + ext = Extension('_ctypes', include_dirs=include_dirs, extra_compile_args=extra_compile_args, + extra_link_args=extra_link_args, libraries=[], sources=sources, depends=depends) -- cgit v0.12