bpo-30934: Document coverage details for idlelib tests (#2711)

* Add section to idlelib/idle-test/README.txt.
* Include check that branches are taken both ways.
* Exclude IDLE-specific code that does not run during unit tests.

2 files changed, 63 insertions, 0 deletions

diff --git a/Lib/idlelib/idle_test/README.txt b/Lib/idlelib/idle_test/README.txt
index dc7a286..a54e74d 100644
--- a/Lib/idlelib/idle_test/README.txt
+++ b/Lib/idlelib/idle_test/README.txt
@@ -159,3 +159,59 @@ complete, though some tests need improvement. To run all htests, run the
 htest file from an editor or from the command line with:
 
 python -m idlelib.idle_test.htest
+
+
+5. Test Coverage
+
+Install the coverage package into your Python 3.6 site-packages
+directory.  (Its exact location depends on the OS). 
+> python3 -m pip install coverage
+(On Windows, replace 'python3 with 'py -3.6' or perhaps just 'python'.)
+
+The problem with running coverage with repository python is that
+coverage uses absolute imports for its submodules, hence it needs to be
+in a directory in sys.path.  One solution: copy the package to the
+directory containing the cpython repository.  Call it 'dev'.  Then run
+coverage either directly or from a script in that directory so that
+'dev' is prepended to sys.path.
+
+Either edit or add dev/.coveragerc so it looks something like this.
+---
+# .coveragerc sets coverage options.
+[run]
+branch = True
+
+[report]
+# Regexes for lines to exclude from consideration
+exclude_lines =
+    # Don't complain if non-runnable code isn't run:
+    if 0:
+    if __name__ == .__main__.:
+
+    .*# htest #
+    if not _utest:
+    if _htest:
+---
+The additions for IDLE are 'branch = True', to test coverage both ways,
+and the last three exclude lines, to exclude things peculiar to IDLE
+that are not executed during tests.
+
+A script like the following cover.bat (for Windows) is very handy.
+---
+@echo off
+rem Usage: cover filename [test_ suffix] # proper case required by coverage
+rem filename without .py, 2nd parameter if test is not test_filename
+setlocal
+set py=f:\dev\3x\pcbuild\win32\python_d.exe
+set src=idlelib.%1
+if "%2" EQU "" set tst=f:/dev/3x/Lib/idlelib/idle_test/test_%1.py
+if "%2" NEQ "" set tst=f:/dev/ex/Lib/idlelib/idle_test/test_%2.py
+
+%py% -m coverage run --pylib --source=%src% %tst%
+%py% -m coverage report --show-missing
+%py% -m coverage html
+start htmlcov\3x_Lib_idlelib_%1_py.html
+rem Above opens new report; htmlcov\index.html displays report index
+---
+The second parameter was added for tests of module x not named test_x.
+(There were several before modules were renamed, now only one is left.)
diff --git a/Misc/NEWS.d/next/IDLE/2017-07-15-22-26-57.bpo-30934.BanuSB.rst b/Misc/NEWS.d/next/IDLE/2017-07-15-22-26-57.bpo-30934.BanuSB.rst
new file mode 100644
index 0000000..53beb43
--- /dev/null
+++ b/Misc/NEWS.d/next/IDLE/2017-07-15-22-26-57.bpo-30934.BanuSB.rst
@@ -0,0 +1,7 @@
+Document coverage details for idlelib tests.
+
+* Add section to idlelib/idle-test/README.txt.
+
+* Include check that branches are taken both ways.
+
+* Exclude IDLE-specific code that does not run during unit tests.