Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r60568 | christian.heimes | 2008-02-04 19:48:38 +0100 (Mon, 04 Feb 2008) | 1 line

  Increase debugging to investige failing tests on some build bots
........
  r60570 | christian.heimes | 2008-02-04 20:30:05 +0100 (Mon, 04 Feb 2008) | 1 line

  Small adjustments for test compact freelist test. It's no passing on Windows as well.
........
  r60573 | amaury.forgeotdarc | 2008-02-04 21:53:14 +0100 (Mon, 04 Feb 2008) | 2 lines

  Correct quotes in NEWS file
........
  r60575 | amaury.forgeotdarc | 2008-02-04 22:45:05 +0100 (Mon, 04 Feb 2008) | 13 lines

  #1750076: Debugger did not step on every iteration of a while statement.

  The mapping between bytecode offsets and source lines (lnotab) did not contain
  an entry for the beginning of the loop.

  Now it does, and the lnotab can be a bit larger:
  in particular, several statements on the same line generate several entries.
  However, this does not bother the settrace function, which will trigger only
  one 'line' event.

  The lnotab seems to be exactly the same as with python2.4.
........
  r60584 | amaury.forgeotdarc | 2008-02-05 01:26:21 +0100 (Tue, 05 Feb 2008) | 3 lines

  Change r60575 broke test_compile:
  there is no need to emit co_lnotab item when both offsets are zeros.
........
  r60587 | skip.montanaro | 2008-02-05 03:32:16 +0100 (Tue, 05 Feb 2008) | 1 line

  sync with most recent version from python-mode sf project
........
  r60588 | lars.gustaebel | 2008-02-05 12:51:40 +0100 (Tue, 05 Feb 2008) | 5 lines

  Issue #2004: Use mode 0700 for temporary directories and default
  permissions for missing directories.

  (will backport to 2.5)
........
  r60590 | georg.brandl | 2008-02-05 13:01:24 +0100 (Tue, 05 Feb 2008) | 2 lines

  Convert external links to internal links. Fixes #2010.
........
  r60592 | marc-andre.lemburg | 2008-02-05 15:50:40 +0100 (Tue, 05 Feb 2008) | 3 lines

  Keep distutils Python 2.1 compatible (or even Python 2.4 in this case).
........
  r60593 | andrew.kuchling | 2008-02-05 17:06:57 +0100 (Tue, 05 Feb 2008) | 5 lines

  Update PEP URL.
  (This code is duplicated between pydoc and DocXMLRPCServer; maybe it
  should be refactored as a GHOP project.)

  2.5.2 backport candidate.
........
  r60596 | guido.van.rossum | 2008-02-05 18:32:15 +0100 (Tue, 05 Feb 2008) | 2 lines

  In the experimental 'Scanner' feature, the group count was set wrong.
........
  r60602 | facundo.batista | 2008-02-05 20:03:32 +0100 (Tue, 05 Feb 2008) | 3 lines


  Issue 1951. Converts wave test cases to unittest.
........
  r60603 | georg.brandl | 2008-02-05 20:07:10 +0100 (Tue, 05 Feb 2008) | 2 lines

  Actually run the test.
........
  r60604 | skip.montanaro | 2008-02-05 20:24:30 +0100 (Tue, 05 Feb 2008) | 2 lines

  correct object name
........
  r60605 | georg.brandl | 2008-02-05 20:58:17 +0100 (Tue, 05 Feb 2008) | 7 lines

  * Use the same code to profile for test_profile and test_cprofile.
  * Convert both to unittest.
  * Use the same unit testing code.
  * Include the expected output in both test files.
  * Make it possible to regenerate the expected output by running
    the file as a script with an '-r' argument.
........
  r60613 | raymond.hettinger | 2008-02-06 02:49:00 +0100 (Wed, 06 Feb 2008) | 1 line

  Sync-up with Py3k work.
........
  r60614 | christian.heimes | 2008-02-06 13:44:34 +0100 (Wed, 06 Feb 2008) | 1 line

  Limit free list of method and builtin function objects to 256 entries each.
........
  r60616 | christian.heimes | 2008-02-06 14:33:44 +0100 (Wed, 06 Feb 2008) | 7 lines

  Unified naming convention for free lists and their limits. All free lists
  in Object/ are named ``free_list``, the counter ``numfree`` and the upper
  limit is a macro ``PyName_MAXFREELIST`` inside an #ifndef block.

  The chances should make it easier to adjust Python for platforms with
  less memory, e.g. mobile phones.
........

1 files changed, 170 insertions, 114 deletions

diff --git a/Lib/test/test_profile.py b/Lib/test/test_profile.py
index 95ad8d2..632911d 100644..100755
--- a/Lib/test/test_profile.py
+++ b/Lib/test/test_profile.py
@@ -1,123 +1,179 @@
 """Test suite for the profile module."""
 
-import profile, pstats, sys
+import os
+import sys
+import pstats
+import unittest
+from difflib import unified_diff
+from io import StringIO
+from test.test_support import run_unittest
 
-# In order to have reproducible time, we simulate a timer in the global
-# variable 'ticks', which represents simulated time in milliseconds.
-# (We can't use a helper function increment the timer since it would be
-# included in the profile and would appear to consume all the time.)
-ticks = 0
+import profile
+from test.profilee import testfunc, timer
+
+
+class ProfileTest(unittest.TestCase):
+
+    profilerclass = profile.Profile
+    methodnames = ['print_stats', 'print_callers', 'print_callees']
+    expected_output = {}
+
+    @classmethod
+    def do_profiling(cls):
+        results = []
+        prof = cls.profilerclass(timer, 0.001)
+        prof.runctx("testfunc()", globals(), locals())
+        results.append(timer())
+        for methodname in cls.methodnames:
+            s = StringIO()
+            stats = pstats.Stats(prof, stream=s)
+            stats.strip_dirs().sort_stats("stdname")
+            getattr(stats, methodname)()
+            results.append(s.getvalue())
+        return results
+
+    def test_cprofile(self):
+        results = self.do_profiling()
+        self.assertEqual(results[0], 43000)
+        for i, method in enumerate(self.methodnames):
+            if results[i+1] != self.expected_output[method]:
+                print("Stats.%s output for %s doesn't fit expectation!" %
+                      (method, self.profilerclass.__name__))
+                print('\n'.join(unified_diff(
+                                  results[i+1].split('\n'),
+                                  self.expected_output[method].split('\n'))))
+
+
+def regenerate_expected_output(filename, cls):
+    filename = filename.rstrip('co')
+    print('Regenerating %s...' % filename)
+    results = cls.do_profiling()
+
+    newfile = []
+    with open(filename, 'r') as f:
+        for line in f:
+            newfile.append(line)
+            if line[:6] == '#--cut':
+                break
+
+    with open(filename, 'w') as f:
+        f.writelines(newfile)
+        for i, method in enumerate(cls.methodnames):
+            f.write('%s.expected_output[%r] = """\\\n%s"""\n' % (
+                cls.__name__, method, results[i+1]))
+        f.write('\nif __name__ == "__main__":\n    main()\n')
 
-# IMPORTANT: this is an output test.  *ALL* NUMBERS in the expected
-# output are relevant.  If you change the formatting of pstats,
-# please don't just regenerate output/test_profile without checking
-# very carefully that not a single number has changed.
 
 def test_main():
-    global ticks
-    ticks = 42000
-    prof = profile.Profile(timer)
-    prof.runctx("testfunc()", globals(), locals())
-    assert ticks == 43000, ticks
-    st = pstats.Stats(prof)
-    st.strip_dirs().sort_stats('stdname').print_stats()
-    st.print_callees()
-    st.print_callers()
-
-def timer():
-    return ticks*0.001
-
-def testfunc():
-    # 1 call
-    # 1000 ticks total: 270 ticks local, 730 ticks in subfunctions
-    global ticks
-    ticks += 99
-    helper()                            # 300
-    helper()                            # 300
-    ticks += 171
-    factorial(14)                       # 130
-
-def factorial(n):
-    # 23 calls total
-    # 170 ticks total, 150 ticks local
-    # 3 primitive calls, 130, 20 and 20 ticks total
-    # including 116, 17, 17 ticks local
-    global ticks
-    if n > 0:
-        ticks += n
-        return mul(n, factorial(n-1))
+    run_unittest(ProfileTest)
+
+def main():
+    if '-r' not in sys.argv:
+        test_main()
     else:
-        ticks += 11
-        return 1
-
-def mul(a, b):
-    # 20 calls
-    # 1 tick, local
-    global ticks
-    ticks += 1
-    return a * b
-
-def helper():
-    # 2 calls
-    # 300 ticks total: 20 ticks local, 260 ticks in subfunctions
-    global ticks
-    ticks += 1
-    helper1()                           # 30
-    ticks += 2
-    helper1()                           # 30
-    ticks += 6
-    helper2()                           # 50
-    ticks += 3
-    helper2()                           # 50
-    ticks += 2
-    helper2()                           # 50
-    ticks += 5
-    helper2_indirect()                  # 70
-    ticks += 1
-
-def helper1():
-    # 4 calls
-    # 30 ticks total: 29 ticks local, 1 tick in subfunctions
-    global ticks
-    ticks += 10
-    hasattr(C(), "foo")                 # 1
-    ticks += 19
-    lst = []
-    lst.append(42)                      # 0
-    sys.exc_info()                      # 0
-
-def helper2_indirect():
-    helper2()                           # 50
-    factorial(3)                        # 20
-
-def helper2():
-    # 8 calls
-    # 50 ticks local: 39 ticks local, 11 ticks in subfunctions
-    global ticks
-    ticks += 11
-    hasattr(C(), "bar")                 # 1
-    ticks += 13
-    subhelper()                         # 10
-    ticks += 15
-
-def subhelper():
-    # 8 calls
-    # 10 ticks total: 8 ticks local, 2 ticks in subfunctions
-    global ticks
-    ticks += 2
-    for i in range(2):                  # 0
-        try:
-            C().foo                     # 1 x 2
-        except AttributeError:
-            ticks += 3                  # 3 x 2
-
-class C:
-    def __getattr__(self, name):
-        # 28 calls
-        # 1 tick, local
-        global ticks
-        ticks += 1
-        raise AttributeError
+        regenerate_expected_output(__file__, ProfileTest)
+
+
+# Don't remove this comment. Everything below it is auto-generated.
+#--cut--------------------------------------------------------------------------
+ProfileTest.expected_output['print_stats'] = """\
+         126 function calls (106 primitive calls) in 999.751 CPU seconds
+
+   Ordered by: standard name
+
+   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
+        4   -0.004   -0.001   -0.004   -0.001 :0(append)
+        4   -0.004   -0.001   -0.004   -0.001 :0(exc_info)
+        1   -0.004   -0.004  999.753  999.753 :0(exec)
+       12   -0.024   -0.002   11.964    0.997 :0(hasattr)
+        1    0.000    0.000    0.000    0.000 :0(setprofile)
+        1   -0.002   -0.002  999.767  999.767 <string>:1(<module>)
+        2   -0.004   -0.002   -0.010   -0.005 io.py:1213(flush)
+        2   -0.002   -0.001   -0.002   -0.001 io.py:656(closed)
+        2   -0.004   -0.002   -0.006   -0.003 io.py:874(flush)
+        0    0.000             0.000          profile:0(profiler)
+        1   -0.002   -0.002  999.751  999.751 profile:0(testfunc())
+       28   27.972    0.999   27.972    0.999 profilee.py:110(__getattr__)
+        1  269.996  269.996  999.769  999.769 profilee.py:25(testfunc)
+     23/3  149.937    6.519  169.917   56.639 profilee.py:35(factorial)
+       20   19.980    0.999   19.980    0.999 profilee.py:48(mul)
+        2   39.986   19.993  599.830  299.915 profilee.py:55(helper)
+        4  115.984   28.996  119.964   29.991 profilee.py:73(helper1)
+        2   -0.006   -0.003  139.946   69.973 profilee.py:84(helper2_indirect)
+        8  311.976   38.997  399.912   49.989 profilee.py:88(helper2)
+        8   63.976    7.997   79.960    9.995 profilee.py:98(subhelper)
+
+
+"""
+ProfileTest.expected_output['print_callers'] = """\
+   Ordered by: standard name
+
+Function                          was called by...
+:0(append)                        <- profilee.py:73(helper1)(4)  119.964
+:0(exc_info)                      <- profilee.py:73(helper1)(4)  119.964
+:0(exec)                          <- profile:0(testfunc())(1)  999.751
+:0(hasattr)                       <- profilee.py:73(helper1)(4)  119.964
+                                     profilee.py:88(helper2)(8)  399.912
+:0(setprofile)                    <- profile:0(testfunc())(1)  999.751
+<string>:1(<module>)              <- :0(exec)(1)  999.753
+io.py:1213(flush)                 <- :0(exec)(2)  999.753
+io.py:656(closed)                 <- io.py:874(flush)(2)   -0.006
+io.py:874(flush)                  <- io.py:1213(flush)(2)   -0.010
+profile:0(profiler)               <-
+profile:0(testfunc())             <- profile:0(profiler)(1)    0.000
+profilee.py:110(__getattr__)      <- :0(hasattr)(12)   11.964
+                                     profilee.py:98(subhelper)(16)   79.960
+profilee.py:25(testfunc)          <- <string>:1(<module>)(1)  999.767
+profilee.py:35(factorial)         <- profilee.py:25(testfunc)(1)  999.769
+                                     profilee.py:35(factorial)(20)  169.917
+                                     profilee.py:84(helper2_indirect)(2)  139.946
+profilee.py:48(mul)               <- profilee.py:35(factorial)(20)  169.917
+profilee.py:55(helper)            <- profilee.py:25(testfunc)(2)  999.769
+profilee.py:73(helper1)           <- profilee.py:55(helper)(4)  599.830
+profilee.py:84(helper2_indirect)  <- profilee.py:55(helper)(2)  599.830
+profilee.py:88(helper2)           <- profilee.py:55(helper)(6)  599.830
+                                     profilee.py:84(helper2_indirect)(2)  139.946
+profilee.py:98(subhelper)         <- profilee.py:88(helper2)(8)  399.912
+
+
+"""
+ProfileTest.expected_output['print_callees'] = """\
+   Ordered by: standard name
+
+Function                          called...
+:0(append)                        ->
+:0(exc_info)                      ->
+:0(exec)                          -> <string>:1(<module>)(1)  999.767
+                                     io.py:1213(flush)(2)   -0.010
+:0(hasattr)                       -> profilee.py:110(__getattr__)(12)   27.972
+:0(setprofile)                    ->
+<string>:1(<module>)              -> profilee.py:25(testfunc)(1)  999.769
+io.py:1213(flush)                 -> io.py:874(flush)(2)   -0.006
+io.py:656(closed)                 ->
+io.py:874(flush)                  -> io.py:656(closed)(2)   -0.002
+profile:0(profiler)               -> profile:0(testfunc())(1)  999.751
+profile:0(testfunc())             -> :0(exec)(1)  999.753
+                                     :0(setprofile)(1)    0.000
+profilee.py:110(__getattr__)      ->
+profilee.py:25(testfunc)          -> profilee.py:35(factorial)(1)  169.917
+                                     profilee.py:55(helper)(2)  599.830
+profilee.py:35(factorial)         -> profilee.py:35(factorial)(20)  169.917
+                                     profilee.py:48(mul)(20)   19.980
+profilee.py:48(mul)               ->
+profilee.py:55(helper)            -> profilee.py:73(helper1)(4)  119.964
+                                     profilee.py:84(helper2_indirect)(2)  139.946
+                                     profilee.py:88(helper2)(6)  399.912
+profilee.py:73(helper1)           -> :0(append)(4)   -0.004
+                                     :0(exc_info)(4)   -0.004
+                                     :0(hasattr)(4)   11.964
+profilee.py:84(helper2_indirect)  -> profilee.py:35(factorial)(2)  169.917
+                                     profilee.py:88(helper2)(2)  399.912
+profilee.py:88(helper2)           -> :0(hasattr)(8)   11.964
+                                     profilee.py:98(subhelper)(8)   79.960
+profilee.py:98(subhelper)         -> profilee.py:110(__getattr__)(16)   27.972
+
+
+"""
 
 if __name__ == "__main__":
-    test_main()
+    main()