summaryrefslogtreecommitdiffstats
path: root/Lib/pathlib
Commit message (Collapse)AuthorAgeFilesLines
* [3.13] GH-119169: Implement `pathlib.Path.walk()` using `os.walk()` ↵Miss Islington (bot)2024-05-292-2/+34
| | | | | | | | | | | | | (GH-119573) (#119750) GH-119169: Implement `pathlib.Path.walk()` using `os.walk()` (GH-119573) For silly reasons, pathlib's generic implementation of `walk()` currently resides in `glob._Globber`. This commit moves it into `pathlib._abc.PathBase.walk()` where it really belongs, and makes `pathlib.Path.walk()` call `os.walk()`. (cherry picked from commit 7ff61f51b6f75315291419269295a8ac3933397b) Co-authored-by: Barney Gale <barney.gale@gmail.com>
* [3.13] GH-119113: Raise `TypeError` from ↵Miss Islington (bot)2024-05-191-6/+4
| | | | | | | | `pathlib.PurePath.with_suffix(None)` (GH-119124) (#119183) Restore behaviour from 3.12 when `path.with_suffix(None)` is called. (cherry picked from commit 3c28510b984392b8dac87a17dfc5887366d5c4ab) Co-authored-by: Barney Gale <barney.gale@gmail.com>
* GH-116380: Revert move of pathlib globbing code to `pathlib._glob` (#118678)Barney Gale2024-05-073-336/+5
| | | | | | The previous change made the `glob` module slower to import, because it imported `pathlib._glob` and hence the rest of `pathlib`. Reverts a40f557d7b7a355a55bb90c068e3e9202fd9c8f2.
* Move pathlib implementation out of `__init__.py` (#118582)Barney Gale2024-05-054-905/+910
| | | Use the `__init__.py` file only for imports that define the API, following the example of asyncio.
* GH-116380: Move pathlib globbing implementation into `pathlib._glob` (#118562)Barney Gale2024-05-033-6/+311
| | | | | | | | Moving this code under the `pathlib` package makes it quite a lot easier to backport in the `pathlib-abc` PyPI package. It was a bit foolish of me to add it to `glob` in the first place. Also add `translate()` to `__all__` in `glob`. This function is new in 3.13, so there's no NEWS needed.
* docs: typo: tiny grammar change: "pointed by" -> "pointed to by" (#118411)Andrew Zipperer2024-05-022-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * docs: tiny grammar change: "pointed by" -> "pointed to by" This commit uses "file pointed to by" to replace "file pointed by" in - doc for shutil.copytree - docstring for shutil.copytree - docstring _abc.PathBase.open - docstring for pathlib.Path.open - doc for os.copy_file_range - doc for os.splice The docs use "file pointed to by" more frequently than "file pointed by". So, this commit replaces the uses of "file pointed by" in order to make the uses consistent through the docs. ```bash $ grep -ri 'pointed to by' cpython/ ``` yields more results than ```bash $ grep -ri 'pointed by' cpython/ ``` Separately: There are two occurrences of "tree pointed by": - cpython/Doc/library/xml.etree.elementtree.rst for `xml.etree.ElementInclude.include` - cpython/Lib/xml/etree/ElementInclude.py for `include` For those uses of "tree pointed by", I expect "tree pointed to by" instead. However, I found enough uses online of (a) "tree pointed by" rather than (b) "tree pointed to by" to convince me that (a) is in common use. So, this commit does not replace those occurrences of "tree pointed by" to "tree pointed to by". But I will replace them if a reviewer believes it is correct to replace them. * docs: typo: "exists and executable" -> "exists and is executable" --------- Co-authored-by: Andrew-Zipperer <atzipperer@gmail.com>
* GH-112855: Speed up `pathlib.PurePath` pickling (#112856)Barney Gale2024-04-201-3/+1
| | | | | | | The second item in the tuple returned from `__reduce__()` is a tuple of arguments to supply to path constructor. Previously we returned the `parts` tuple here, which entailed joining, parsing and normalising the path object, and produced a compact pickle representation. With this patch, we instead return a tuple of paths that were originally given to the path constructor. This makes pickling much faster (at the expense of compactness). It's worth noting that, in the olden times, pathlib performed this parsing/normalization up-front in every case, and so using `parts` for pickling was almost free. Nowadays pathlib only parses/normalises paths when it's necessary or advantageous to do so (e.g. computing a path parent, or iterating over a directory, respectively).
* GH-115060: Speed up `pathlib.Path.glob()` by omitting initial `stat()` (#117831)Barney Gale2024-04-132-6/+2
| | | | | | | | Since 6258844c, paths that might not exist can be fed into pathlib's globbing implementation, which will call `os.scandir()` / `os.lstat()` only when strictly necessary. This allows us to drop an initial `self.is_dir()` call, which saves a `stat()`. Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
* GH-117727: Speed up `pathlib.Path.iterdir()` by using `os.scandir()` (#117728)Barney Gale2024-04-121-20/+6
| | | | | Replace use of `os.listdir()` with `os.scandir()`. Forgo setting `_drv`, `_root` and `_tail_cached`, as these usually aren't needed. Use `os.DirEntry.path` to set `_str`.
* GH-115060: Speed up `pathlib.Path.glob()` by not scanning literal parts ↵Barney Gale2024-04-121-1/+7
| | | | | | | | | (#117732) Don't bother calling `os.scandir()` to scan for literal pattern segments, like `foo` in `foo/*.py`. Instead, append the segment(s) as-is and call through to the next selector with `exists=False`, which signals that the path might not exist. Subsequent selectors will call `os.scandir()` or `os.lstat()` to filter out missing paths as needed.
* GH-117586: Speed up `pathlib.Path.walk()` by working with strings (#117726)Barney Gale2024-04-112-71/+14
| | | | | | | | | | | Move `pathlib.Path.walk()` implementation into `glob._Globber`. The new `glob._Globber.walk()` classmethod works with strings internally, which is a little faster than generating `Path` objects and keeping them normalized. The `pathlib.Path.walk()` method converts the strings back to path objects. In the private pathlib ABCs, our existing subclass of `_Globber` ensures that `PathBase` instances are used throughout. Follow-up to #117589.
* GH-117586: Speed up `pathlib.Path.glob()` by working with strings (#117589)Barney Gale2024-04-102-195/+82
| | | | | | | | | | | | Move pathlib globbing implementation into a new private class: `glob._Globber`. This class implements fast string-based globbing. It's called by `pathlib.Path.glob()`, which then converts strings back to path objects. In the private pathlib ABCs, add a `pathlib._abc.Globber` subclass that works with `PathBase` objects rather than strings, and calls user-defined path methods like `PathBase.stat()` rather than `os.stat()`. This sets the stage for two more improvements: - GH-115060: Query non-wildcard segments with `lstat()` - GH-116380: Unify `pathlib` and `glob` implementations of globbing. No change to the implementations of `glob.glob()` and `glob.iglob()`.
* GH-77609: Add recurse_symlinks argument to `pathlib.Path.glob()` (#117311)Barney Gale2024-04-052-16/+12
| | | | | | | | | | | | | | | Replace tri-state `follow_symlinks` with boolean `recurse_symlinks` argument. The new argument controls whether symlinks are followed when expanding recursive `**` wildcards. The possible argument values correspond as follows: follow_symlinks recurse_symlinks =============== ================ False N/A None False True True We therefore drop support for not following symlinks when expanding non-recursive pattern parts; it wasn't requested in the original issue, and it's a feature not found in any shells. This makes the API a easier to grok by eliminating `None` as an option. No news blurb as `follow_symlinks` was new in 3.13.
* GH-114575: Rename `PurePath.pathmod` to `PurePath.parser` (#116513)Barney Gale2024-03-312-52/+52
| | | And rename the private base class from `PathModuleBase` to `ParserBase`.
* pathlib ABCs: follow all symlinks in `PathBase.glob()` (#116293)Barney Gale2024-03-041-2/+2
| | | | | | | Switch the default value of *follow_symlinks* from `None` to `True` in `pathlib._abc.PathBase.glob()` and `rglob()`. This speeds up recursive globbing. No change to the public pathlib classes.
* GH-114610: Fix `pathlib.PurePath.with_stem('')` handling of file extensions ↵Barney Gale2024-02-241-1/+9
| | | | | | | (#114612) Raise `ValueError` if `with_stem('')` is called on a path with a file extension. Paths may only have an empty stem if they also have an empty suffix.
* GH-115060: Speed up `pathlib.Path.glob()` by removing redundant regex ↵Barney Gale2024-02-102-28/+62
| | | | | matching (#115061) When expanding and filtering paths for a `**` wildcard segment, build an `re.Pattern` object from the subsequent pattern parts, rather than the entire pattern, and match against the `os.DirEntry` object prior to instantiating a path object. Also skip compiling a pattern when expanding a `*` wildcard segment.
* GH-106747: Make pathlib ABC globbing more consistent with `glob.glob()` ↵Barney Gale2024-02-061-1/+1
| | | | | | | | | | | | (#115056) When expanding `**` wildcards, ensure we add a trailing slash to the topmost directory path. This matches `glob.glob()` behaviour: >>> glob.glob('dirA/**', recursive=True) ['dirA/', 'dirA/dirB', 'dirA/dirB/dirC'] This does not affect `pathlib.Path.glob()`, because trailing slashes aren't supported in pathlib proper.
* pathlib ABCs: drop partial, broken, untested support for `bytes` paths. ↵Barney Gale2024-01-311-4/+3
| | | | | | | (#114777) Methods like `full_match()`, `glob()`, etc, are difficult to make work with byte paths, and it's not worth the effort. This patch makes `PurePathBase` raise `TypeError` when given non-`str` path segments.
* pathlib ABCs: raise `UnsupportedOperation` directly. (#114776)Barney Gale2024-01-312-33/+31
| | | | | Raise `UnsupportedOperation` directly, rather than via an `_unsupported()` helper, to give human readers and IDEs/typecheckers/etc a bigger hint that these methods are abstract.
* GH-70303: Make `pathlib.Path.glob('**')` return both files and directories ↵Barney Gale2024-01-301-8/+0
| | | | | | | | | | | | (#114684) Return files and directories from `pathlib.Path.glob()` if the pattern ends with `**`. This is more compatible with `PurePath.full_match()` and with other glob implementations such as bash and `glob.glob()`. Users can add a trailing slash to match only directories. In my previous patch I added a `FutureWarning` with the intention of fixing this in Python 3.15. Upon further reflection I think this was an unnecessarily cautious remedy to a clear bug.
* GH-114610: Fix `pathlib._abc.PurePathBase.with_suffix('.ext')` handling of ↵Barney Gale2024-01-301-2/+5
| | | | | | | | | stems (#114613) Raise `ValueError` if `with_suffix('.ext')` is called on a path without a stem. Paths may only have a non-empty suffix if they also have a non-empty stem. ABC-only bugfix; no effect on public classes.
* GH-79634: Speed up pathlib globbing by removing `joinpath()` call. (#114623)Barney Gale2024-01-271-1/+1
| | | | | Remove `self.joinpath('')` call that should have been removed in 6313cdde. This makes `PathBase.glob('')` yield itself *without* adding a trailing slash. It's hard to say whether this is more or less correct, but at least everything else is faster, and there's no behaviour change in the public classes where empty glob patterns are disallowed.
* gh-88569: add `ntpath.isreserved()` (#95486)Barney Gale2024-01-261-21/+7
| | | | | | | | | | | Add `ntpath.isreserved()`, which identifies reserved pathnames such as "NUL", "AUX" and "CON". Deprecate `pathlib.PurePath.is_reserved()`. --------- Co-authored-by: Eryk Sun <eryksun@gmail.com> Co-authored-by: Brett Cannon <brett@python.org> Co-authored-by: Steve Dower <steve.dower@microsoft.com>
* GH-73435: Add `pathlib.PurePath.full_match()` (#114350)Barney Gale2024-01-262-16/+45
| | | | | | | | | | | | | | | | In 49f90ba we added support for the recursive wildcard `**` in `pathlib.PurePath.match()`. This should allow arbitrary prefix and suffix matching, like `p.match('foo/**')` or `p.match('**/foo')`, but there's a problem: for relative patterns only, `match()` implicitly inserts a `**` token on the left hand side, causing all patterns to match from the right. As a result, it's impossible to match relative patterns from the left: `PurePath('foo/bar').match('bar/**')` is true! This commit reverts the changes to `match()`, and instead adds a new `full_match()` method that: - Allows empty patterns - Supports the recursive wildcard `**` - Matches the *entire* path when given a relative pattern
* GH-113225: Speed up `pathlib.Path.walk(top_down=False)` (#113693)Barney Gale2024-01-201-4/+5
| | | | | | | | | | | | | | Use `_make_child_entry()` rather than `_make_child_relpath()` to retrieve path objects for directories to visit. This saves the allocation of one path object per directory in user subclasses of `PathBase`, and avoids a second loop. This trick does not apply when walking top-down, because users can affect the walk by modifying *dirnames* in-place. A side effect of this change is that, in bottom-up mode, subdirectories of each directory are visited in reverse order, and that this order doesn't match that of the names in *dirnames*. I suspect this is fine as the order is arbitrary anyway.
* GH-79634: Accept path-like objects as pathlib glob patterns. (#114017)Barney Gale2024-01-202-69/+78
| | | | | | | | | Allow `os.PathLike` objects to be passed as patterns to `pathlib.Path.glob()` and `rglob()`. (It's already possible to use them in `PurePath.match()`) While we're in the area: - Allow empty glob patterns in `PathBase` (but not `Path`) - Speed up globbing in `PathBase` by generating paths with trailing slashes only as a final step, rather than for every intermediate directory. - Simplify and speed up handling of rare patterns involving both `**` and `..` segments.
* Replace `pathlib._abc.PathModuleBase.splitroot()` with `splitdrive()` (#114065)Barney Gale2024-01-141-10/+8
| | | | | This allows users of the `pathlib-abc` PyPI package to use `posixpath` or `ntpath` as a path module in versions of Python lacking `os.path.splitroot()` (3.11 and before).
* Add `pathlib._abc.PathModuleBase` (#113893)Barney Gale2024-01-142-57/+127
| | | | | | | | | | | | | | | | Path modules provide a subset of the `os.path` API, specifically those functions needed to provide `PurePathBase` functionality. Each `PurePathBase` subclass references its path module via a `pathmod` class attribute. This commit adds a new `PathModuleBase` class, which provides abstract methods that unconditionally raise `UnsupportedOperation`. An instance of this class is assigned to `PurePathBase.pathmod`, replacing `posixpath`. As a result, `PurePathBase` is no longer POSIX-y by default, and all its methods raise `UnsupportedOperation` courtesy of `pathmod`. Users who subclass `PurePathBase` or `PathBase` should choose the path syntax by setting `pathmod` to `posixpath`, `ntpath`, `os.path`, or their own subclass of `PathModuleBase`, as circumstances demand.
* Add module docstring for `pathlib._abc`. (#113691)Barney Gale2024-01-131-0/+13
|
* pathlib ABCs: add `_raw_path` property (#113976)Barney Gale2024-01-132-20/+31
| | | | | | | | | It's wrong for the `PurePathBase` methods to rely so much on `__str__()`. Instead, they should treat the raw path(s) as opaque objects and leave the details to `pathmod`. This commit adds a `PurePathBase._raw_path` property and uses it through many of the other ABC methods. These methods are all redefined in `PurePath` and `Path`, so this has no effect on the public classes.
* GH-44626, GH-105476: Fix `ntpath.isabs()` handling of part-absolute paths ↵Barney Gale2024-01-131-5/+1
| | | | | | | | | | | | | (#113829) On Windows, `os.path.isabs()` now returns `False` when given a path that starts with exactly one (back)slash. This is more compatible with other functions in `os.path`, and with Microsoft's own documentation. Also adjust `pathlib.PureWindowsPath.is_absolute()` to call `ntpath.isabs()`, which corrects its handling of partial UNC/device paths like `//foo`. Co-authored-by: Jon Foster <jon@jon-foster.co.uk>
* pathlib ABCs: Require one or more initialiser arguments (#113885)Barney Gale2024-01-101-8/+2
| | | | | | | Refuse to guess what a user means when they initialise a pathlib ABC without any positional arguments. In mainline pathlib it's normalised to `.`, but in the ABCs this guess isn't appropriate; for example, the path type may not represent the current directory as `.`, or may have no concept of a "current directory" at all.
* GH-113528: Deoptimise `pathlib._abc.PurePathBase` (#113559)Barney Gale2024-01-092-114/+132
| | | | | | | Apply pathlib's normalization and performance tuning in `pathlib.PurePath`, but not `pathlib._abc.PurePathBase`. With this change, the pathlib ABCs do not normalize away alternate path separators, empty segments, or dot segments. A single string given to the initialiser will round-trip by default, i.e. `str(PurePathBase(my_string)) == my_string`. Implementors can set their own path domain-specific normalization scheme by overriding `__str__()` Eliminating path normalization makes maintaining and caching the path's parts and string representation both optional and not very useful, so this commit moves the `_drv`, `_root`, `_tail_cached` and `_str` slots from `PurePathBase` to `PurePath`. Only `_raw_paths` and `_resolving` slots remain in `PurePathBase`. This frees the ABCs from the burden of some of pathlib's hardest-to-understand code.
* GH-113528: Deoptimise `pathlib._abc.PurePathBase.relative_to()` (again) ↵Barney Gale2024-01-092-15/+42
| | | | | | | | | (#113882) Restore full battle-tested implementations of `PurePath.[is_]relative_to()`. These were recently split up in 3375dfe and a15a773. In `PurePathBase`, add entirely new implementations based on `_stack`, which itself calls `pathmod.split()` repeatedly to disassemble a path. These new implementations preserve features like trailing slashes where possible, while still observing that a `..` segment cannot be added to traverse an empty or `.` segment in *walk_up* mode. They do not rely on `parents` nor `__eq__()`, nor do they spin up temporary path objects. Unfortunately calling `pathmod.relpath()` isn't an option, as it calls `abspath()` and in turn `os.getcwd()`, which is impure.
* GH-113528: Deoptimise `pathlib._abc.PurePathBase.parts` (#113883)Barney Gale2024-01-092-4/+13
| | | | | Implement `parts` using `_stack`, which itself calls `pathmod.split()` repeatedly. This avoids use of `_tail`, which will be moved to `PurePath` shortly.
* GH-113528: Deoptimise `pathlib._abc.PathBase.resolve()` (#113782)Barney Gale2024-01-091-25/+40
| | | | | | Replace use of `_from_parsed_parts()` with `with_segments()` in `resolve()`. No effect on `Path.resolve()`, which uses `os.path.realpath()`.
* GH-113528: Deoptimise `pathlib._abc.PathBase._make_child_relpath()` (#113532)Barney Gale2024-01-092-14/+17
| | | | Call straight through to `joinpath()` in `PathBase._make_child_relpath()`. Move optimised/caching code to `pathlib.Path._make_child_relpath()`
* GH-113528: Speed up pathlib ABC tests. (#113788)Barney Gale2024-01-081-4/+4
| | | | | - Add `__slots__` to dummy path classes. - Return namedtuple rather than `os.stat_result` from `DummyPath.stat()`. - Reduce maximum symlink count in `DummyPathWithSymlinks.resolve()`.
* GH-113528: Deoptimise `pathlib._abc.PurePathBase.relative_to()` (#113529)Barney Gale2024-01-062-2/+5
| | | | | Replace use of `_from_parsed_parts()` with `with_segments()` in `PurePathBase.relative_to()`, and move the assignment of `_drv`, `_root` and `_tail_cached` slots into `PurePath.relative_to()`.
* GH-113528: Deoptimise `pathlib._abc.PurePathBase.parent` (#113530)Barney Gale2024-01-062-42/+63
| | | | | Replace use of `_from_parsed_parts()` with `with_segments()`, and move assignments to `_drv`, `_root`, _tail_cached` and `_str` slots into `PurePath`.
* GH-113528: Deoptimise `pathlib._abc.PurePathBase.name` (#113531)Barney Gale2024-01-062-7/+25
| | | | | Replace usage of `_from_parsed_parts()` with `with_segments()` in `with_name()`, and take a similar approach in `name` for consistency's sake.
* GH-113568: Stop raising deprecation warnings from pathlib ABCs (#113757)Barney Gale2024-01-052-17/+31
|
* GH-113568: Stop raising auditing events from pathlib ABCs (#113571)Barney Gale2024-01-052-22/+50
| | | | | Raise auditing events in `pathlib.Path.glob()`, `rglob()` and `walk()`, but not in `pathlib._abc.PathBase` methods. Also move generation of a deprecation warning into `pathlib.Path` so it gets the right stack level.
* GH-113225: Speed up `pathlib.Path.glob()` (#113226)Barney Gale2024-01-041-1/+7
| | | | Use `os.DirEntry.path` as the string representation of child paths, unless the parent path is empty, in which case we use the entry `name`.
* GH-113225: Speed up `pathlib._abc.PathBase.glob()` (#113556)Barney Gale2023-12-282-5/+12
| | | | `PathBase._scandir()` is implemented using `iterdir()`, so we can use its results directly, rather than passing them through `_make_child_relpath()`.
* GH-110109: pathlib ABCs: drop use of `warnings._deprecated()` (#113419)Barney Gale2023-12-271-6/+4
| | | | | | The `pathlib._abc` module will be made available as a PyPI backport supporting Python 3.8+. The `warnings._deprecated()` function was only added last year, and it's private from an external package perspective, so here we switch to `warnings.warn()` instead.
* GH-110109: pathlib ABCs: drop use of `io.text_encoding()` (#113417)Barney Gale2023-12-272-3/+18
| | | | | | Do not use the locale-specific default encoding in `PathBase.read_text()` and `write_text()`. Locale settings shouldn't influence the operation of these base classes, which are intended mostly for implementing rich paths on *nonlocal* filesystems.
* GH-110109: pathlib ABCs: do not vary path syntax by host OS. (#113219)Barney Gale2023-12-222-2/+2
| | | | | | | | | | | | | | | | Change the value of `pathlib._abc.PurePathBase.pathmod` from `os.path` to `posixpath`. User subclasses of `PurePathBase` and `PathBase` previously used the host OS's path syntax, e.g. backslashes as separators on Windows. This is wrong in most use cases, and likely to catch developers out unless they test on both Windows and non-Windows machines. In this patch we change the default to POSIX syntax, regardless of OS. This is somewhat arguable (why not make all aspects of syntax abstract and individually configurable?) but an improvement all the same. This change has no effect on `PurePath`, `Path`, nor their subclasses. Only private APIs are affected.
* GH-110109: Fix misleading `pathlib._abc.PurePathBase` repr (#113376)Barney Gale2023-12-222-3/+3
| | | | | | | | | | `PurePathBase.__repr__()` produces a string like `MyPath('/foo')`. This repr is incorrect/misleading when a subclass's `__init__()` method is customized, which I expect to be the very common. This commit moves the `__repr__()` method to `PurePath`, leaving `PurePathBase` with the default `object` repr. No user-facing changes because the `pathlib._abc` module remains private.
generated by cgit v0.12 at 2025-04-10 10:28:29 (GMT)