Issue #6953: Rearrange and expand Readline module documentation

* Group functions into six new subsections
* Document the underlying Readline function or variable accessed
* get_history_length() returns the history file limit
* clear_history() is conditionally compiled in
* Clarify zero and one bases for history item indexes
* parse_and_bind() uses its argument directly as an init line
* Change "command line" to "line buffer" for consistency
* read_init_file() also executes the file
* read_history_file() replaces the previous history
* write_history_file() overwrites any existing file
* Differentiate history file lines from history list items, which could be
  multi-line
* Add more information about completion, also addressing Issue #10796
* libedit (Editline) may be used on any platform; detection is OS X specific

3 files changed, 136 insertions, 75 deletions

diff --git a/Doc/library/readline.rst b/Doc/library/readline.rst
index 5433405..42e0ad5 100644
--- a/Doc/library/readline.rst
+++ b/Doc/library/readline.rst
@@ -9,15 +9,18 @@
 
 The :mod:`readline` module defines a number of functions to facilitate
 completion and reading/writing of history files from the Python interpreter.
-This module can be used directly or via the :mod:`rlcompleter` module.  Settings
+This module can be used directly, or via the :mod:`rlcompleter` module, which
+supports completion of Python identifiers at the interactive prompt.  Settings
 made using  this module affect the behaviour of both the interpreter's
 interactive prompt  and the prompts offered by the built-in :func:`input`
 function.
 
 .. note::
 
-  On MacOS X the :mod:`readline` module can be implemented using
+  The underlying Readline library API may be implemented by
   the ``libedit`` library instead of GNU readline.
+  On MacOS X the :mod:`readline` module detects which library is being used
+  at run time.
 
   The configuration file for ``libedit`` is different from that
   of GNU readline. If you programmatically load configuration strings
@@ -25,112 +28,168 @@ function.
   to differentiate between GNU readline and libedit.
 
 
-The :mod:`readline` module defines the following functions:
+Init file
+---------
+
+The following functions relate to the init file and user configuration:
 
 
 .. function:: parse_and_bind(string)
 
-   Parse and execute single line of a readline init file.
+   Execute the init line provided in the *string* argument. This calls
+   :c:func:`rl_parse_and_bind` in the underlying library.
+
+
+.. function:: read_init_file([filename])
+
+   Execute a readline initialization file. The default filename is the last filename
+   used. This calls :c:func:`rl_read_init_file` in the underlying library.
+
+
+Line buffer
+-----------
+
+The following functions operate on the line buffer:
 
 
 .. function:: get_line_buffer()
 
-   Return the current contents of the line buffer.
+   Return the current contents of the line buffer (:c:data:`rl_line_buffer`
+   in the underlying library).
 
 
 .. function:: insert_text(string)
 
-   Insert text into the command line.
+   Insert text into the line buffer at the cursor position.  This calls
+   :c:func:`rl_insert_text` in the underlying library, but ignores
+   the return value.
 
 
-.. function:: read_init_file([filename])
+.. function:: redisplay()
 
-   Parse a readline initialization file. The default filename is the last filename
-   used.
+   Change what's displayed on the screen to reflect the current contents of the
+   line buffer.  This calls :c:func:`rl_redisplay` in the underlying library.
+
+
+History file
+------------
+
+The following functions operate on a history file:
 
 
 .. function:: read_history_file([filename])
 
-   Load a readline history file. The default filename is :file:`~/.history`.
+   Load a readline history file, and append it to the history list.
+   The default filename is :file:`~/.history`.  This calls
+   :c:func:`read_history` in the underlying library.
 
 
 .. function:: write_history_file([filename])
 
-   Save a readline history file. The default filename is :file:`~/.history`.
+   Save the history list to a readline history file, overwriting any
+   existing file.  The default filename is :file:`~/.history`.  This calls
+   :c:func:`write_history` in the underlying library.
 
 
 .. function:: append_history_file(nelements[, filename])
 
-   Append the last *nelements* of history to a file.  The default filename is
-   :file:`~/.history`.  The file must already exist.
+   Append the last *nelements* items of history to a file.  The default filename is
+   :file:`~/.history`.  The file must already exist.  This calls
+   :c:func:`append_history` in the underlying library.
 
    .. versionadded:: 3.5
 
 
-.. function:: clear_history()
+.. function:: get_history_length()
+              set_history_length(length)
 
-   Clear the current history.  (Note: this function is not available if the
-   installed version of GNU readline doesn't support it.)
+   Set or return the desired number of lines to save in the history file.
+   The :func:`write_history_file` function uses this value to truncate
+   the history file, by calling :c:func:`history_truncate_file` in
+   the underlying library.  Negative values imply
+   unlimited history file size.
 
 
-.. function:: get_history_length()
+History list
+------------
 
-   Return the desired length of the history file.  Negative values imply unlimited
-   history file size.
+The following functions operate on a global history list:
 
 
-.. function:: set_history_length(length)
+.. function:: clear_history()
 
-   Set the number of lines to save in the history file. :func:`write_history_file`
-   uses this value to truncate the history file when saving.  Negative values imply
-   unlimited history file size.
+   Clear the current history.  This calls :c:func:`clear_history` in the
+   underlying library.  The Python function only exists if Python was
+   compiled for a version of the library that supports it.
 
 
 .. function:: get_current_history_length()
 
-   Return the number of lines currently in the history.  (This is different from
+   Return the number of items currently in the history.  (This is different from
    :func:`get_history_length`, which returns the maximum number of lines that will
    be written to a history file.)
 
 
 .. function:: get_history_item(index)
 
-   Return the current contents of history item at *index*.
+   Return the current contents of history item at *index*.  The item index
+   is one-based.  This calls :c:func:`history_get` in the underlying library.
 
 
 .. function:: remove_history_item(pos)
 
    Remove history item specified by its position from the history.
+   The position is zero-based.  This calls :c:func:`remove_history` in
+   the underlying library.
 
 
 .. function:: replace_history_item(pos, line)
 
-   Replace history item specified by its position with the given line.
+   Replace history item specified by its position with *line*.
+   The position is zero-based.  This calls :c:func:`replace_history_entry`
+   in the underlying library.
 
 
-.. function:: redisplay()
+.. function:: add_history(line)
+
+   Append *line* to the history buffer, as if it was the last line typed.
+   This calls :c:func:`add_history` in the underlying library.
 
-   Change what's displayed on the screen to reflect the current contents of the
-   line buffer.
+
+Startup hooks
+-------------
 
 
 .. function:: set_startup_hook([function])
 
-   Set or remove the startup_hook function.  If *function* is specified, it will be
-   used as the new startup_hook function; if omitted or ``None``, any hook function
-   already installed is removed.  The startup_hook function is called with no
+   Set or remove the function invoked by the :c:data:`rl_startup_hook`
+   callback of the underlying library.  If *function* is specified, it will
+   be used as the new hook function; if omitted or ``None``, any function
+   already installed is removed.  The hook is called with no
    arguments just before readline prints the first prompt.
 
 
 .. function:: set_pre_input_hook([function])
 
-   Set or remove the pre_input_hook function.  If *function* is specified, it will
-   be used as the new pre_input_hook function; if omitted or ``None``, any hook
-   function already installed is removed.  The pre_input_hook function is called
+   Set or remove the function invoked by the :c:data:`rl_pre_input_hook`
+   callback of the underlying library.  If *function* is specified, it will
+   be used as the new hook function; if omitted or ``None``, any
+   function already installed is removed.  The hook is called
    with no arguments after the first prompt has been printed and just before
    readline starts reading input characters.
 
 
+Completion
+----------
+
+The following functions relate to implementing a custom word completion
+function.  This is typically operated by the Tab key, and can suggest and
+automatically complete a word being typed.  By default, Readline is set up
+to be used by :mod:`rlcompleter` to complete Python identifiers for
+the interactive interpreter.  If the :mod:`readline` module is to be used
+with a custom completer, a different set of word delimiters should be set.
+
+
 .. function:: set_completer([function])
 
    Set or remove the completer function.  If *function* is specified, it will be
@@ -140,6 +199,12 @@ The :mod:`readline` module defines the following functions:
    returns a non-string value.  It should return the next possible completion
    starting with *text*.
 
+   The installed completer function is invoked by the *entry_func* callback
+   passed to :c:func:`rl_completion_matches` in the underlying library.
+   The *text* string comes from the first parameter to the
+   :c:data:`rl_attempted_completion_function` callback of the
+   underlying library.
+
 
 .. function:: get_completer()
 
@@ -148,27 +213,27 @@ The :mod:`readline` module defines the following functions:
 
 .. function:: get_completion_type()
 
-   Get the type of completion being attempted.
+   Get the type of completion being attempted.  This returns the
+   :c:data:`rl_completion_type` variable in the underlying library as
+   an integer.
 
 
 .. function:: get_begidx()
+              get_endidx()
 
-   Get the beginning index of the readline tab-completion scope.
-
-
-.. function:: get_endidx()
-
-   Get the ending index of the readline tab-completion scope.
+   Get the beginning or ending index of the completion scope.
+   These indexes are the *start* and *end* arguments passed to the
+   :c:data:`rl_attempted_completion_function` callback of the
+   underlying library.
 
 
 .. function:: set_completer_delims(string)
+              get_completer_delims()
 
-   Set the readline word delimiters for tab-completion.
-
-
-.. function:: get_completer_delims()
-
-   Get the readline word delimiters for tab-completion.
+   Set or get the word delimiters for completion.  These determine the
+   start of the word to be considered for completion (the completion scope).
+   These functions access the :c:data:`rl_completer_word_break_characters`
+   variable in the underlying library.
 
 
 .. function:: set_completion_display_matches_hook([function])
@@ -176,21 +241,13 @@ The :mod:`readline` module defines the following functions:
    Set or remove the completion display function.  If *function* is
    specified, it will be used as the new completion display function;
    if omitted or ``None``, any completion display function already
-   installed is removed.  The completion display function is called as
+   installed is removed.  This sets or clears the
+   :c:data:`rl_completion_display_matches_hook` callback in the
+   underlying library.  The completion display function is called as
    ``function(substitution, [matches], longest_match_length)`` once
    each time matches need to be displayed.
 
 
-.. function:: add_history(line)
-
-   Append a line to the history buffer, as if it was the last line typed.
-
-.. seealso::
-
-   Module :mod:`rlcompleter`
-      Completion of Python identifiers at the interactive prompt.
-
-
 .. _readline-example:
 
 Example
diff --git a/Misc/NEWS b/Misc/NEWS
index b695831..7027b30 100644
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -340,6 +340,10 @@ Library
 Documentation
 -------------
 
+- Issue #6953: Rework the Readline module documentation to group related
+  functions together, and add more details such as what underlying Readline
+  functions and variables are accessed.
+
 - Issue #23606: Adds note to ctypes documentation regarding cdll.msvcrt.
 
 - Issue #25500: Fix documentation to not claim that __import__ is searched for
diff --git a/Modules/readline.c b/Modules/readline.c
index be57f8f..63be9b4 100644
--- a/Modules/readline.c
+++ b/Modules/readline.c
@@ -149,7 +149,7 @@ parse_and_bind(PyObject *self, PyObject *args)
 
 PyDoc_STRVAR(doc_parse_and_bind,
 "parse_and_bind(string) -> None\n\
-Parse and execute single line of a readline init file.");
+Execute the init line provided in the string argument.");
 
 
 /* Exported function to parse a readline init file */
@@ -174,7 +174,7 @@ read_init_file(PyObject *self, PyObject *args)
 
 PyDoc_STRVAR(doc_read_init_file,
 "read_init_file([filename]) -> None\n\
-Parse a readline initialization file.\n\
+Execute a readline initialization file.\n\
 The default filename is the last filename used.");
 
 
@@ -271,7 +271,7 @@ append_history_file(PyObject *self, PyObject *args)
 
 PyDoc_STRVAR(doc_append_history_file,
 "append_history_file(nelements[, filename]) -> None\n\
-Append the last nelements of the history list to file.\n\
+Append the last nelements items of the history list to file.\n\
 The default filename is ~/.history.");
 #endif
 
@@ -290,7 +290,7 @@ set_history_length(PyObject *self, PyObject *args)
 
 PyDoc_STRVAR(set_history_length_doc,
 "set_history_length(length) -> None\n\
-set the maximal number of items which will be written to\n\
+set the maximal number of lines which will be written to\n\
 the history file. A negative length is used to inhibit\n\
 history truncation.");
 
@@ -305,7 +305,7 @@ get_history_length(PyObject *self, PyObject *noarg)
 
 PyDoc_STRVAR(get_history_length_doc,
 "get_history_length() -> int\n\
-return the maximum number of items that will be written to\n\
+return the maximum number of lines that will be written to\n\
 the history file.");
 
 
@@ -373,7 +373,7 @@ set_startup_hook(PyObject *self, PyObject *args)
 
 PyDoc_STRVAR(doc_set_startup_hook,
 "set_startup_hook([function]) -> None\n\
-Set or remove the startup_hook function.\n\
+Set or remove the function invoked by the rl_startup_hook callback.\n\
 The function is called with no arguments just\n\
 before readline prints the first prompt.");
 
@@ -390,7 +390,7 @@ set_pre_input_hook(PyObject *self, PyObject *args)
 
 PyDoc_STRVAR(doc_set_pre_input_hook,
 "set_pre_input_hook([function]) -> None\n\
-Set or remove the pre_input_hook function.\n\
+Set or remove the function invoked by the rl_pre_input_hook callback.\n\
 The function is called with no arguments after the first prompt\n\
 has been printed and just before readline starts reading input\n\
 characters.");
@@ -421,7 +421,7 @@ get_begidx(PyObject *self, PyObject *noarg)
 
 PyDoc_STRVAR(doc_get_begidx,
 "get_begidx() -> int\n\
-get the beginning index of the readline tab-completion scope");
+get the beginning index of the completion scope");
 
 
 /* Get the ending index for the scope of the tab-completion */
@@ -435,7 +435,7 @@ get_endidx(PyObject *self, PyObject *noarg)
 
 PyDoc_STRVAR(doc_get_endidx,
 "get_endidx() -> int\n\
-get the ending index of the readline tab-completion scope");
+get the ending index of the completion scope");
 
 
 /* Set the tab-completion word-delimiters that readline uses */
@@ -464,7 +464,7 @@ set_completer_delims(PyObject *self, PyObject *args)
 
 PyDoc_STRVAR(doc_set_completer_delims,
 "set_completer_delims(string) -> None\n\
-set the readline word delimiters for tab-completion");
+set the word delimiters for completion");
 
 /* _py_free_history_entry: Utility function to free a history entry. */
 
@@ -504,7 +504,7 @@ py_remove_history(PyObject *self, PyObject *args)
     int entry_number;
     HIST_ENTRY *entry;
 
-    if (!PyArg_ParseTuple(args, "i:remove_history", &entry_number))
+    if (!PyArg_ParseTuple(args, "i:remove_history_item", &entry_number))
         return NULL;
     if (entry_number < 0) {
         PyErr_SetString(PyExc_ValueError,
@@ -534,7 +534,7 @@ py_replace_history(PyObject *self, PyObject *args)
     char *line;
     HIST_ENTRY *old_entry;
 
-    if (!PyArg_ParseTuple(args, "is:replace_history", &entry_number,
+    if (!PyArg_ParseTuple(args, "is:replace_history_item", &entry_number,
                           &line)) {
         return NULL;
     }
@@ -575,7 +575,7 @@ py_add_history(PyObject *self, PyObject *args)
 
 PyDoc_STRVAR(doc_add_history,
 "add_history(string) -> None\n\
-add a line to the history buffer");
+add an item to the history buffer");
 
 
 /* Get the tab-completion word-delimiters that readline uses */
@@ -588,7 +588,7 @@ get_completer_delims(PyObject *self, PyObject *noarg)
 
 PyDoc_STRVAR(doc_get_completer_delims,
 "get_completer_delims() -> string\n\
-get the readline word delimiters for tab-completion");
+get the word delimiters for completion");
 
 
 /* Set the completer function */
@@ -649,7 +649,7 @@ get_history_item(PyObject *self, PyObject *args)
     int idx = 0;
     HIST_ENTRY *hist_ent;
 
-    if (!PyArg_ParseTuple(args, "i:index", &idx))
+    if (!PyArg_ParseTuple(args, "i:get_history_item", &idx))
         return NULL;
 #ifdef  __APPLE__
     if (using_libedit_emulation) {
@@ -741,7 +741,7 @@ insert_text(PyObject *self, PyObject *args)
 
 PyDoc_STRVAR(doc_insert_text,
 "insert_text(string) -> None\n\
-Insert text into the command line.");
+Insert text into the line buffer at the cursor position.");
 
 
 /* Redisplay the line buffer */