Tim Peters strikes again:

Ho ho ho -- that's trickier than it sounded!  The colorizer is working with
"line.col" strings instead of Text marks, and the absolute coordinates of
the point of interest can change across the self.update call (voice of
baffled experience, when two quick backspaces no longer fooled it, but a
backspace followed by a quick ENTER did <wink>).

Anyway, the attached appears to do the trick.  CPU usage goes way up when
typing quickly into a long triple-quoted string, but the latency is fine for
me (a relatively fast typist on a relatively slow machine).  Most of the
changes here are left over from reducing the # of vrbl names to help me
reason about the logic better; I hope the code is a *little* easier to

3 files changed, 160 insertions, 44 deletions

diff --git a/Tools/idle/ChangeLog b/Tools/idle/ChangeLog
index ccec63e..167d794 100644
--- a/Tools/idle/ChangeLog
+++ b/Tools/idle/ChangeLog
@@ -1,21 +1,130 @@
+Thu Apr 22 23:20:17 1999  Guido van Rossum  <guido@cnri.reston.va.us>
+
+        * help.txt:
+        Bunch of updates necessary due to recent changes; added docs for File
+        menu, command line and color preferences.
+
+        * Bindings.py: Remove obsolete 'script' menu.
+
+	* TODO.txt: Several wishes fulfilled.
+
+	* OutputWindow.py:
+	Moved classes OnDemandOutputWindow and PseudoFile here,
+	from ScriptBinding.py where they are no longer needed.
+
+	* ScriptBinding.py:
+	Mostly rewritten.  Instead of the old Run module and Debug module,
+	there are two new commands:
+
+	Import module (F5) imports or reloads the module and also adds its
+	name to the __main__ namespace.  This gets executed in the PyShell
+	window under control of its debug settings.
+
+	Run script (Control-F5) is similar but executes the contents of the
+	file directly in the __main__ namespace.
+
+	* PyShell.py: Nits: document use of $IDLESTARTUP; display idle version
+
+	* idlever.py: New version to celebrate new command line
+
+	* OutputWindow.py: Added flush(), for completeness.
+
+	* PyShell.py:
+	A lot of changes to make the command line more useful.  You can now do:
+	  idle.py -e file ...    -- to edit files
+	  idle.py script arg ... -- to run a script
+	  idle.py -c cmd arg ... -- to run a command
+	Other options, see also the usage message (also new!) for more details:
+	  -d       -- enable debugger
+	  -s       -- run $IDLESTARTUP or $PYTHONSTARTUP
+	  -t title -- set Python Shell window's title
+	sys.argv is set accordingly, unless -e is used.
+	sys.path is absolutized, and all relevant paths are inserted into it.
+
+	Other changes:
+	- the environment in which commands are executed is now the
+	  __main__ module
+	- explicitly save sys.stdout etc., don't restore from sys.__stdout__
+	- new interpreter methods execsource(), execfile(), stuffsource()
+	- a few small nits
+
+	* TODO.txt:
+	Some more TODO items.  Made up my mind about command line args,
+	Run/Import, __main__.
+
+	* ColorDelegator.py:
+	Super-elegant patch by Tim Peters that speeds up colorization
+	dramatically (up to 15 times he claims).  Works by reading more than
+	one line at a time, up to 100-line chunks (starting with one line and
+	then doubling up to the limit).  On a typical machine (e.g. Tim's
+	P5-166) this doesn't reduce interactive responsiveness in a noticeable
+	way.
+
+Wed Apr 21 15:49:34 1999  Guido van Rossum  <guido@cnri.reston.va.us>
+
+	* ColorDelegator.py:
+	Patch by Tim Peters to speed up colorizing of big multiline strings.
+
+Tue Apr 20 17:32:52 1999  Guido van Rossum  <guido@cnri.reston.va.us>
+
+	* extend.txt:
+	For an event 'foo-bar', the corresponding method must be called
+	foo_bar_event().  Therefore, fix the references to zoom_height() in
+	the example.
+
+	* IdlePrefs.py: Restored the original IDLE color scheme.
+
+	* PyShell.py, IdlePrefs.py, ColorDelegator.py, EditorWindow.py:
+	Color preferences code by Loren Luke (massaged by me somewhat)
+
+	* SearchEngine.py:
+	Patch by Mark Favas: it fixes the search engine behaviour where an
+	unsuccessful search wraps around and re-searches that part of the file
+	between the start of the search and the end of the file - only really
+	an issue for very large files, but... (also removes a redundant
+	m.span() call).
+
+Mon Apr 19 16:26:02 1999  Guido van Rossum  <guido@cnri.reston.va.us>
+
+	* TODO.txt: A few wishes are now fulfilled.
+
+	* AutoIndent.py: Tim Peters implements some of my wishes:
+
+	o Makes the tab key intelligently insert spaces when appropriate
+	(see Help list banter twixt David Ascher and me; idea stolen from
+	every other editor on earth <wink>).
+
+	o newline_and_indent_event trims trailing whitespace on the old
+	line (pymode and Codewright).
+
+	o newline_and_indent_event no longer fooled by trailing whitespace or
+	comment after ":" (pymode, PTUI).
+
+	o newline_and_indent_event now reduces the new line's indentation after
+	return, break, continue, raise and pass stmts (pymode).
+
+	The last two are easy to fool in the presence of strings &
+	continuations, but pymode requires Emacs's high-powered C parsing
+	functions to avoid that in finite time.
+
 ======================================================================
 	Python release 1.5.2c1, IDLE version 0.4
 ======================================================================
 
-Wed Apr  7 18:41:59 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Wed Apr  7 18:41:59 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* README.txt, NEWS.txt: New version.
 
 	* idlever.py: Version bump awaiting impending new release.
 	(Not much has changed :-( )
 
-Mon Mar 29 14:52:28 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Mon Mar 29 14:52:28 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* ScriptBinding.py, PyShell.py:
 	At Tim Peters' recommendation, add a dummy flush() method to
 	PseudoFile.
 
-Thu Mar 11 23:21:23 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Thu Mar 11 23:21:23 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* PathBrowser.py: Don't crash when sys.path contains an empty string.
 
@@ -29,7 +138,7 @@ Thu Mar 11 23:21:23 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 	- Show a watch cursor when calling pyclbr (since it may take a while
 	recursively parsing imported modules!).
 
-Wed Mar 10 05:18:02 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Wed Mar 10 05:18:02 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* EditorWindow.py, Bindings.py: Add PathBrowser to File module
 
@@ -59,7 +168,7 @@ Wed Mar 10 05:18:02 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 	Python release 1.5.2b2, IDLE version 0.3
 ======================================================================
 	
-Wed Feb 17 22:47:41 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Wed Feb 17 22:47:41 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* NEWS.txt: News in 0.3.
 
@@ -81,18 +190,18 @@ Wed Feb 17 22:47:41 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 
 	* ChangeLog: Oh, why not.  Checking in the Emacs-generated change log.
 
-Tue Feb 16 22:34:17 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Tue Feb 16 22:34:17 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* ScriptBinding.py:
 	Only pop up the stack viewer when requested in the Debug menu.
 
-Mon Feb  8 22:27:49 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Mon Feb  8 22:27:49 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* WindowList.py: Don't crash if a window no longer exists.
 
 	* TODO.txt: Restructured a bit.
 
-Mon Feb  1 23:06:17 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Mon Feb  1 23:06:17 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* PyShell.py: Add current dir or paths of file args to sys.path.
 
@@ -100,12 +209,12 @@ Mon Feb  1 23:06:17 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 
 	* StackViewer.py: Protect against accessing an empty stack.
 
-Fri Jan 29 20:44:45 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Fri Jan 29 20:44:45 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* ZoomHeight.py:
 	Use only the height to decide whether to zoom in or out.
 
-Thu Jan 28 22:24:30 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Thu Jan 28 22:24:30 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* EditorWindow.py, FileList.py:
 	Make sure the Tcl variables are shared between windows.
@@ -125,13 +234,13 @@ Thu Jan 28 22:24:30 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 
 	* idle.py, Attic/idle: Rename idle -> idle.py
 
-Mon Jan 18 15:18:57 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Mon Jan 18 15:18:57 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* EditorWindow.py, WindowList.py: Only deiconify when iconic.
 
 	* TODO.txt: Misc
 
-Tue Jan 12 22:14:34 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Tue Jan 12 22:14:34 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* testcode.py, Attic/test.py:
 	Renamed test.py to testcode.py so one can import Python's
@@ -145,7 +254,7 @@ Tue Jan 12 22:14:34 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 	in arbitrary directions when I try to move it for resizing the frames.
 	This patch makes it more quiet.
 
-Mon Jan 11 14:52:40 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Mon Jan 11 14:52:40 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* TODO.txt: Some requests have been fulfilled.
 
@@ -171,7 +280,7 @@ Mon Jan 11 14:52:40 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 	* Separator.py:
 	Separator classes (draggable divider between two panes).
 
-Sat Jan  9 22:01:33 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Sat Jan  9 22:01:33 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* WindowList.py:
 	Don't traceback when wakeup() is called when the window has been destroyed.
@@ -195,7 +304,7 @@ Sat Jan  9 22:01:33 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 	Python release 1.5.2b1, IDLE version 0.2
 ======================================================================
 	
-Fri Jan  8 17:26:02 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Fri Jan  8 17:26:02 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* README.txt, NEWS.txt: What's new in this release.
 
@@ -203,7 +312,7 @@ Fri Jan  8 17:26:02 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 	Paul Prescod's patches to allow the stack viewer to pop up when a
 	traceback is printed.
 
-Thu Jan  7 00:12:15 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Thu Jan  7 00:12:15 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* FormatParagraph.py:
 	Change paragraph width limit to 70 (like Emacs M-Q).
@@ -214,7 +323,7 @@ Thu Jan  7 00:12:15 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 
 	* TODO.txt: Separating TODO from README.
 
-Mon Jan  4 21:19:09 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Mon Jan  4 21:19:09 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* FormatParagraph.py:
 	Hm.  There was a boundary condition error at the end of the file too.
@@ -242,13 +351,13 @@ Mon Jan  4 21:19:09 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 
 	* ZoomHeight.py: Typo in Win specific height setting.
 
-Sun Jan  3 00:47:35 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Sun Jan  3 00:47:35 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* AutoIndent.py: Added something like Tim Peters' backspace patch.
 
 	* ZoomHeight.py: Adapted to Unix (i.e., more hardcoded constants).
 
-Sat Jan  2 21:28:54 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Sat Jan  2 21:28:54 1999  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* keydefs.py, idlever.py, idle.pyw, idle.bat, help.txt, extend.txt, extend.py, eventparse.py, ZoomHeight.py, WindowList.py, UndoDelegator.py, StackViewer.py, SearchEngine.py, SearchDialogBase.py, SearchDialog.py, ScrolledList.py, SearchBinding.py, ScriptBinding.py, ReplaceDialog.py, Attic/README, README.txt, PyShell.py, Attic/PopupMenu.py, OutputWindow.py, IOBinding.py, Attic/HelpWindow.py, History.py, GrepDialog.py, FileList.py, FrameViewer.py, EditorWindow.py, Debugger.py, Delegator.py, ColorDelegator.py, Bindings.py, ClassBrowser.py, AutoExpand.py, AutoIndent.py:
 	Checking in IDLE 0.2.
@@ -263,16 +372,16 @@ Sat Jan  2 21:28:54 1999  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 	This is exactly as downloaded from my laptop after returning
 	from the holidays -- it hasn't even been tested on Unix yet.
 
-Fri Dec 18 15:52:54 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Fri Dec 18 15:52:54 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* FileList.py, ClassBrowser.py:
 	Fix the class browser to work even when the file is not on sys.path.
 
-Tue Dec  8 20:39:36 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Tue Dec  8 20:39:36 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* Attic/turtle.py: Moved to Python 1.5.2/Lib
 
-Fri Nov 27 03:19:20 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Fri Nov 27 03:19:20 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* help.txt: Typo
 
@@ -283,17 +392,17 @@ Fri Nov 27 03:19:20 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 	definitions (platform-specific), and generating accelerator strings
 	automatically from the key definitions.
 
-Mon Nov 16 18:37:42 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Mon Nov 16 18:37:42 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* Attic/README: Clarify portability and main program.
 
 	* Attic/README: Added intro for 0.1 release and append Grail notes.
 
-Mon Oct 26 18:49:00 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Mon Oct 26 18:49:00 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* Attic/turtle.py: root is now a global called _root
 
-Sat Oct 24 16:38:38 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Sat Oct 24 16:38:38 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* Attic/turtle.py: Raise the root window on reset().
 	Different action on WM_DELETE_WINDOW is more likely to do the right thing,
@@ -310,7 +419,7 @@ Sat Oct 24 16:38:38 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 
 	* Debugger.py: Use of Breakpoint class should be bdb.Breakpoint.
 
-Mon Oct 19 03:33:40 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Mon Oct 19 03:33:40 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* SearchBinding.py:
 	Speed up the search a bit -- don't drag a mark around...
@@ -339,7 +448,7 @@ Mon Oct 19 03:33:40 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 	Add optional 'force' argument (default 0) to load_dict().
 	If set, redo the display even if it's the same dict.
 
-Fri Oct 16 21:10:12 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Fri Oct 16 21:10:12 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* StackViewer.py: Do nothing when loading the same dict as before.
 
@@ -369,7 +478,7 @@ Fri Oct 16 21:10:12 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 
 	* StackViewer.py: Restructured into a browser and a widget.
 
-Thu Oct 15 23:27:08 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Thu Oct 15 23:27:08 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* ClassBrowser.py, ScrolledList.py:
 	Generalized the scrolled list which is the base for the class and
@@ -379,12 +488,12 @@ Thu Oct 15 23:27:08 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 
 	* Debugger.py: Don't show function name if there is none
 
-Wed Oct 14 03:43:05 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Wed Oct 14 03:43:05 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* Debugger.py, PyShell.py: Polish the Debugger GUI a bit.
 	Closing it now also does the right thing.
 
-Tue Oct 13 23:51:13 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Tue Oct 13 23:51:13 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* Debugger.py, PyShell.py, Bindings.py:
 	Ad primitive debugger interface (so far it will step and show you the
@@ -418,7 +527,7 @@ Tue Oct 13 23:51:13 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 	Add new command, "Open module".  You select or type a module name,
 	and it opens the source.
 
-Mon Oct 12 23:59:27 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Mon Oct 12 23:59:27 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* PyShell.py: Subsume functionality from Popup menu in Debug menu.
 	Other stuff so the PyShell window can be resurrected from the Windows menu.
@@ -440,7 +549,7 @@ Mon Oct 12 23:59:27 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
 	* SearchBinding.py:
 	Add parent argument to 'to to line number' dialog box.
 
-Sat Oct 10 19:15:32 1998  Guido van Rossum  <guido@eric.cnri.reston.va.us>
+Sat Oct 10 19:15:32 1998  Guido van Rossum  <guido@cnri.reston.va.us>
 
 	* StackViewer.py:
 	Add a label at the top showing (very basic) help for the stack viewer.
diff --git a/Tools/idle/ColorDelegator.py b/Tools/idle/ColorDelegator.py
index 147f749..68c2d33 100644
--- a/Tools/idle/ColorDelegator.py
+++ b/Tools/idle/ColorDelegator.py
@@ -165,7 +165,6 @@ class ColorDelegator(Delegator):
 
     def recolorize_main(self):
         next = "1.0"
-        was_ok = is_ok = 0
         while 1:
             item = self.tag_nextrange("TODO", next)
             if not item:
@@ -179,14 +178,15 @@ class ColorDelegator(Delegator):
                 head = "1.0"
 
             chars = ""
-            mark = head
+            next = head
             lines_to_get = 1
-            is_ok = was_ok = 0
-            while not (was_ok and is_ok):
+            ok = 0
+            while not ok:
+                mark = next
                 next = self.index(mark + "+%d lines linestart" %
                                          lines_to_get)
                 lines_to_get = min(lines_to_get * 2, 100)
-                was_ok = "SYNC" in self.tag_names(next + "-1c")
+                ok = "SYNC" in self.tag_names(next + "-1c")
                 line = self.get(mark, next)
                 ##print head, "get", mark, next, "->", `line`
                 if not line:
@@ -196,7 +196,6 @@ class ColorDelegator(Delegator):
                 chars = chars + line
                 m = self.prog.search(chars)
                 while m:
-                    i, j = m.span()
                     for key, value in m.groupdict().items():
                         if value:
                             a, b = m.span(key)
@@ -210,12 +209,20 @@ class ColorDelegator(Delegator):
                                     self.tag_add("DEFINITION",
                                                  head + "+%dc" % a,
                                                  head + "+%dc" % b)
-                    m = self.prog.search(chars, j)
-                is_ok = "SYNC" in self.tag_names(next + "-1c")
-                mark = next
-                if is_ok:
-                    head = mark
+                    m = self.prog.search(chars, m.end())
+                if "SYNC" in self.tag_names(next + "-1c"):
+                    head = next
                     chars = ""
+                else:
+                    ok = 0
+                if not ok:
+                    # We're in an inconsistent state, and the call to
+                    # update may tell us to stop.  It may also change
+                    # the correct value for "next" (since this is a
+                    # line.col string, not a true mark).  So leave a
+                    # crumb telling the next invocation to resume here
+                    # in case update tells us to leave.
+                    self.tag_add("TODO", next)
                 self.update()
                 if self.stop_colorizing:
                     if __debug__: print "colorizing stopped"
diff --git a/Tools/idle/PyShell.py b/Tools/idle/PyShell.py
index 76a4beb..64ef2d1 100644
--- a/Tools/idle/PyShell.py
+++ b/Tools/idle/PyShell.py
@@ -407,7 +407,7 @@ class PyShell(OutputWindow):
 
     def begin(self):
         self.resetoutput()
-        self.write("Python %s on %s\n%s\nIDLE %s\n" %
+        self.write("Python %s on %s\n%s\nIDLE %s -- press F1 for help\n" %
                    (sys.version, sys.platform, sys.copyright,
                     idlever.IDLE_VERSION))
         try: