gh-126180: Remove getopt and optparse deprecation notices (GH-126227)

* Remove getopt and optparse deprecation notices
* Add new docs sections for command line app helper libraries
* Add guidance on choosing a CLI parsing library to the optparse docs
* Link to the new guidance from the argparse and getopt docs
* Reword intro in docs section for superseded stdlib modules
* Reframe the optparse->argparse guide as a migration guide
  rather than as an upgrade guide

---------

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

8 files changed, 211 insertions, 39 deletions

diff --git a/Doc/library/allos.rst b/Doc/library/allos.rst
index 0223c10..1aed340 100644
--- a/Doc/library/allos.rst
+++ b/Doc/library/allos.rst
@@ -15,14 +15,9 @@ but they are available on most other systems as well.  Here's an overview:
    os.rst
    io.rst
    time.rst
-   argparse.rst
    logging.rst
    logging.config.rst
    logging.handlers.rst
-   getpass.rst
-   curses.rst
-   curses.ascii.rst
-   curses.panel.rst
    platform.rst
    errno.rst
    ctypes.rst
diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst
index da4071d..8d0116d 100644
--- a/Doc/library/argparse.rst
+++ b/Doc/library/argparse.rst
@@ -11,6 +11,18 @@
 
 **Source code:** :source:`Lib/argparse.py`
 
+.. note::
+
+   While :mod:`argparse` is the default recommended standard library module
+   for implementing basic command line applications, authors with more
+   exacting requirements for exactly how their command line applications
+   behave may find it doesn't provide the necessary level of control.
+   Refer to :ref:`choosing-an-argument-parser` for alternatives to
+   consider when ``argparse`` doesn't support behaviors that the application
+   requires (such as entirely disabling support for interspersed options and
+   positional arguments, or accepting option parameter values that start
+   with ``-`` even when they correspond to another defined option).
+
 --------------
 
 .. sidebar:: Tutorial
diff --git a/Doc/library/cmdlinelibs.rst b/Doc/library/cmdlinelibs.rst
new file mode 100644
index 0000000..085d31a
--- /dev/null
+++ b/Doc/library/cmdlinelibs.rst
@@ -0,0 +1,21 @@
+.. _cmdlinelibs:
+
+********************************
+Command Line Interface Libraries
+********************************
+
+The modules described in this chapter assist with implementing
+command line and terminal interfaces for applications.
+
+Here's an overview:
+
+.. toctree::
+   :maxdepth: 1
+
+   argparse.rst
+   optparse.rst
+   getpass.rst
+   fileinput.rst
+   curses.rst
+   curses.ascii.rst
+   curses.panel.rst
diff --git a/Doc/library/filesys.rst b/Doc/library/filesys.rst
index 0ccf2b7..f1ea476 100644
--- a/Doc/library/filesys.rst
+++ b/Doc/library/filesys.rst
@@ -14,7 +14,6 @@ in this chapter is:
 
    pathlib.rst
    os.path.rst
-   fileinput.rst
    stat.rst
    filecmp.rst
    tempfile.rst
diff --git a/Doc/library/getopt.rst b/Doc/library/getopt.rst
index 891885d..5c63009 100644
--- a/Doc/library/getopt.rst
+++ b/Doc/library/getopt.rst
@@ -7,18 +7,13 @@
 
 **Source code:** :source:`Lib/getopt.py`
 
-.. deprecated:: 3.13
-   The :mod:`getopt` module is :term:`soft deprecated` and will not be
-   developed further; development will continue with the :mod:`argparse`
-   module.
-
 .. note::
 
-   The :mod:`getopt` module is a parser for command line options whose API is
-   designed to be familiar to users of the C :c:func:`!getopt` function. Users who
-   are unfamiliar with the C :c:func:`!getopt` function or who would like to write
-   less code and get better help and error messages should consider using the
-   :mod:`argparse` module instead.
+   This module is considered feature complete. A more declarative and
+   extensible alternative to this API is provided in the :mod:`optparse`
+   module. Further functional enhancements for command line parameter
+   processing are provided either as third party modules on PyPI,
+   or else as features in the :mod:`argparse` module.
 
 --------------
 
@@ -28,6 +23,13 @@ the special meanings of arguments of the form '``-``' and '``--``').  Long
 options similar to those supported by GNU software may be used as well via an
 optional third argument.
 
+Users who are unfamiliar with the Unix :c:func:`!getopt` function should consider
+using the :mod:`argparse` module instead. Users who are familiar with the Unix
+:c:func:`!getopt` function, but would like to get equivalent behavior while
+writing less code and getting better help and error messages should consider
+using the :mod:`optparse` module. See :ref:`choosing-an-argument-parser` for
+additional details.
+
 This module provides two functions and an
 exception:
 
@@ -194,13 +196,27 @@ In a script, typical usage is something like this:
                output = a
            else:
                assert False, "unhandled option"
-       # ...
+       process(args, output=output, verbose=verbose)
 
    if __name__ == "__main__":
        main()
 
 Note that an equivalent command line interface could be produced with less code
-and more informative help and error messages by using the :mod:`argparse` module:
+and more informative help and error messages by using the :mod:`optparse` module:
+
+.. testcode::
+
+   import optparse
+
+   if __name__ == '__main__':
+       parser = optparse.OptionParser()
+       parser.add_option('-o', '--output')
+       parser.add_option('-v', dest='verbose', action='store_true')
+       opts, args = parser.parse_args()
+       process(args, output=opts.output, verbose=opts.verbose)
+
+A roughly equivalent command line interface for this case can also be
+produced by using the :mod:`argparse` module:
 
 .. testcode::
 
@@ -210,12 +226,18 @@ and more informative help and error messages by using the :mod:`argparse` module
        parser = argparse.ArgumentParser()
        parser.add_argument('-o', '--output')
        parser.add_argument('-v', dest='verbose', action='store_true')
+       parser.add_argument('rest', nargs='*')
        args = parser.parse_args()
-       # ... do something with args.output ...
-       # ... do something with args.verbose ..
+       process(args.rest, output=args.output, verbose=args.verbose)
+
+See :ref:`choosing-an-argument-parser` for details on how the ``argparse``
+version of this code differs in behaviour from the ``optparse`` (and
+``getopt``) version.
 
 .. seealso::
 
-   Module :mod:`argparse`
-      Alternative command line option and argument parsing library.
+   Module :mod:`optparse`
+      Declarative command line option parsing.
 
+   Module :mod:`argparse`
+      More opinionated command line option and argument parsing library.
diff --git a/Doc/library/index.rst b/Doc/library/index.rst
index 951fbcf..44b2189 100644
--- a/Doc/library/index.rst
+++ b/Doc/library/index.rst
@@ -55,6 +55,7 @@ the `Python Package Index <https://pypi.org>`_.
    fileformats.rst
    crypto.rst
    allos.rst
+   cmdlinelibs.rst
    concurrency.rst
    ipc.rst
    netdata.rst
diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst
index 74a49a8..ff327cf 100644
--- a/Doc/library/optparse.rst
+++ b/Doc/library/optparse.rst
@@ -3,25 +3,135 @@
 
 .. module:: optparse
    :synopsis: Command-line option parsing library.
-   :deprecated:
 
 .. moduleauthor:: Greg Ward <gward@python.net>
 .. sectionauthor:: Greg Ward <gward@python.net>
 
 **Source code:** :source:`Lib/optparse.py`
 
-.. deprecated:: 3.2
-   The :mod:`optparse` module is :term:`soft deprecated` and will not be
-   developed further; development will continue with the :mod:`argparse`
-   module.
-
 --------------
 
+.. _choosing-an-argument-parser:
+
+Choosing an argument parsing library
+------------------------------------
+
+The standard library includes three argument parsing libraries:
+
+* :mod:`getopt`: a module that closely mirrors the procedural C ``getopt`` API.
+  Included in the standard library since before the initial Python 1.0 release.
+* :mod:`optparse`: a declarative replacement for ``getopt`` that
+  provides equivalent functionality without requiring each application
+  to implement its own procedural option parsing logic. Included
+  in the standard library since the Python 2.3 release.
+* :mod:`argparse`: a more opinionated alternative to ``optparse`` that
+  provides more functionality by default, at the expense of reduced application
+  flexibility in controlling exactly how arguments are processed. Included in
+  the standard library since the Python 2.7 and Python 3.2 releases.
+
+In the absence of more specific argument parsing design constraints, :mod:`argparse`
+is the recommended choice for implementing command line applications, as it offers
+the highest level of baseline functionality with the least application level code.
+
+:mod:`getopt` is retained almost entirely for backwards compatibility reasons.
+However, it also serves a niche use case as a tool for prototyping and testing
+command line argument handling in ``getopt``-based C applications.
+
+:mod:`optparse` should be considered as an alternative to :mod:`argparse` in the
+following cases:
+
+* an application is already using :mod:`optparse` and doesn't want to risk the
+  subtle behavioural changes that may arise when migrating to :mod:`argparse`
+* the application requires additional control over the way options and
+  positional parameters are interleaved on the command line (including
+  the ability to disable the interleaving feature completely)
+* the application requires additional control over the incremental parsing
+  of command line elements (while ``argparse`` does support this, the
+  exact way it works in practice is undesirable for some use cases)
+* the application requires additional control over the handling of options
+  which accept parameter values that may start with ``-`` (such as delegated
+  options to be passed to invoked subprocesses)
+* the application requires some other command line parameter processing
+  behavior which ``argparse`` does not support, but which can be implemented
+  in terms of the lower level interface offered by ``optparse``
+
+These considerations also mean that :mod:`optparse` is likely to provide a
+better foundation for library authors writing third party command line
+argument processing libraries.
+
+As a concrete example, consider the following two command line argument
+parsing configurations, the first using ``optparse``, and the second
+using ``argparse``:
+
+.. testcode::
+
+   import optparse
+
+   if __name__ == '__main__':
+       parser = optparse.OptionParser()
+       parser.add_option('-o', '--output')
+       parser.add_option('-v', dest='verbose', action='store_true')
+       opts, args = parser.parse_args()
+       process(args, output=opts.output, verbose=opts.verbose)
+
+.. testcode::
+
+   import argparse
+
+   if __name__ == '__main__':
+       parser = argparse.ArgumentParser()
+       parser.add_argument('-o', '--output')
+       parser.add_argument('-v', dest='verbose', action='store_true')
+       parser.add_argument('rest', nargs='*')
+       args = parser.parse_args()
+       process(args.rest, output=args.output, verbose=args.verbose)
+
+The most obvious difference is that in the ``optparse`` version, the non-option
+arguments are processed separately by the application after the option processing
+is complete. In the ``argparse`` version, positional arguments are declared and
+processed in the same way as the named options.
+
+However, the ``argparse`` version will also handle some parameter combination
+differently from the way the ``optparse`` version would handle them.
+For example (amongst other differences):
+
+* supplying ``-o -v`` gives ``output="-v"`` and ``verbose=False``
+  when using ``optparse``, but a usage error with ``argparse``
+  (complaining that no value has been supplied for ``-o/--output``,
+  since ``-v`` is interpreted as meaning the verbosity flag)
+* similarly, supplying ``-o --`` gives ``output="--"`` and ``args=()``
+  when using ``optparse``, but a usage error with ``argparse``
+  (also complaining that no value has been supplied for ``-o/--output``,
+  since ``--`` is interpreted as terminating the option processing
+  and treating all remaining values as positional arguments)
+* supplying ``-o=foo`` gives ``output="=foo"`` when using ``optparse``,
+  but gives ``output="foo"`` with ``argparse`` (since ``=`` is special
+  cased as an alternative separator for option parameter values)
+
+Whether these differing behaviors in the ``argparse`` version are
+considered desirable or a problem will depend on the specific command line
+application use case.
+
+.. seealso::
+
+    :pypi:`click` is a third party argument processing library (originally
+    based on ``optparse``), which allows command line applications to be
+    developed as a set of decorated command implementation functions.
+
+    Other third party libraries, such as :pypi:`typer` or :pypi:`msgspec-click`,
+    allow command line interfaces to be specified in ways that more effectively
+    integrate with static checking of Python type annotations.
+
+
+Introduction
+------------
+
 :mod:`optparse` is a more convenient, flexible, and powerful library for parsing
-command-line options than the old :mod:`getopt` module.  :mod:`optparse` uses a
-more declarative style of command-line parsing: you create an instance of
-:class:`OptionParser`, populate it with options, and parse the command
-line. :mod:`optparse` allows users to specify options in the conventional
+command-line options than the minimalist :mod:`getopt` module.
+:mod:`optparse` uses a more declarative style of command-line parsing:
+you create an instance of :class:`OptionParser`,
+populate it with options, and parse the command line.
+:mod:`optparse` allows users to specify options in the conventional
 GNU/POSIX syntax, and additionally generates usage and help messages for you.
 
 Here's an example of using :mod:`optparse` in a simple script::
@@ -82,10 +192,11 @@ Background
 ----------
 
 :mod:`optparse` was explicitly designed to encourage the creation of programs
-with straightforward, conventional command-line interfaces.  To that end, it
-supports only the most common command-line syntax and semantics conventionally
-used under Unix.  If you are unfamiliar with these conventions, read this
-section to acquaint yourself with them.
+with straightforward command-line interfaces that follow the conventions
+established by the :c:func:`!getopt` family of functions available to C developers.
+To that end, it supports only the most common command-line syntax and semantics
+conventionally used under Unix.  If you are unfamiliar with these conventions,
+reading this section will allow you to acquaint yourself with them.
 
 
 .. _optparse-terminology:
diff --git a/Doc/library/superseded.rst b/Doc/library/superseded.rst
index 17bfa66..d120c6a 100644
--- a/Doc/library/superseded.rst
+++ b/Doc/library/superseded.rst
@@ -4,12 +4,23 @@
 Superseded Modules
 ******************
 
-The modules described in this chapter are deprecated or :term:`soft deprecated` and only kept for
-backwards compatibility. They have been superseded by other modules.
+The modules described in this chapter have been superseded by other modules
+for most use cases, and are retained primarily to preserve backwards compatibility.
 
+Modules may appear in this chapter because they only cover a limited subset of
+a problem space, and a more generally applicable solution is available elsewhere
+in the standard library (for example, :mod:`getopt` covers the very specific
+task of "mimic the C :c:func:`!getopt` API in Python", rather than the broader
+command line option parsing and argument parsing capabilities offered by
+:mod:`optparse` and :mod:`argparse`).
+
+Alternatively, modules may appear in this chapter because they are deprecated
+outright, and awaiting removal in a future release, or they are
+:term:`soft deprecated` and their use is actively discouraged in new projects.
+With the removal of various obsolete modules through :pep:`594`, there are
+currently no modules in this latter category.
 
 .. toctree::
    :maxdepth: 1
 
    getopt.rst
-   optparse.rst