summaryrefslogtreecommitdiffstats
path: root/Lib/shutil.py
Commit message (Collapse)AuthorAgeFilesLines
* gh-109653: Defer importing `warnings` in several modules (#110286)Alex Waygood2023-10-041-1/+1
|
* gh-109590: Update shutil.which on Windows to prefer a PATHEXT extension on ↵Charles Machalow2023-10-021-2/+10
| | | | | executable files (GH-109995) The default arguments for shutil.which() request an executable file, but extensionless files are not executable on Windows and should be ignored.
* gh-109748: Fix again venv test_zippath_from_non_installed_posix() (#110149)Victor Stinner2023-09-301-1/+1
| | | | | | | Call also copy_python_src_ignore() on listdir() names. shutil.copytree(): replace set() with an empty tuple. An empty tuple becomes a constant in the compiler and checking if an item is in an empty tuple is cheap.
* gh-99203: shutil.make_archive(): restore select CPython <= 3.10.5 behavior ↵6t8k2023-08-161-0/+4
| | | | | | | | | | | | (GH-99802) Restore following CPython <= 3.10.5 behavior of shutil.make_archive() that went away as part of gh-93160: Do not create an empty archive if root_dir is not a directory, and, in that case, raise FileNotFoundError or NotADirectoryError regardless of format choice. Beyond the brought-back behavior, the function may now also raise these exceptions in dry_run mode.
* gh-88745: Add _winapi.CopyFile2 and update shutil.copy2 to use it (GH-105055)Steve Dower2023-05-301-0/+25
|
* gh-82814: fix shutil access error on WSL (#103790)Allan Lago2023-04-251-1/+1
| | | | | | | gh-82814: Adds `errno.EACCES` to the list of ignored errors on `_copyxattr`. EPERM and EACCES are different constants but in general should be treated the same. News entry authored by: Gregory P. Smith <greg@krypto.org>
* gh-102950: Implement PEP 706 – Filter for tarfile.extractall (#102953)Petr Viktorin2023-04-241-5/+12
|
* gh-102828: set stacklevel on deprecation warning (#103422)Irit Katriel2023-04-111-1/+1
|
* GH-75586: Make shutil.which() on Windows more consistent with the OS (GH-103179)Charles Machalow2023-04-041-41/+48
|
* gh-102828: emit deprecation warning for onerror arg to shutil.rmtree (#102850)Irit Katriel2023-03-211-0/+6
|
* gh-102828: add onexc arg to shutil.rmtree. Deprecate onerror. (#102829)Irit Katriel2023-03-191-44/+62
|
* bpo-45975: Simplify some while-loops with walrus operator (GH-29347)Nick Drozd2022-11-261-4/+1
|
* gh-99547: Add isjunction methods for checking if a path is a junction (GH-99548)Charles Machalow2022-11-221-19/+6
|
* bpo-38523: ignore_dangling_symlinks does not apply recursively (GH-22937)Zackery Spytz2022-11-071-2/+3
|
* gh-74696: Pass root_dir to custom archivers which support it (GH-94251)Serhiy Storchaka2022-10-051-9/+11
| | | | Co-authored-by: Éric <merwok@netwok.org>
* gh-94844: Add pathlib support to shutil archive management (GH-94846)Oleg Iarygin2022-07-201-0/+2
| | | Co-authored-by: Barney Gale <barney.gale@gmail.com>
* gh-74696: Do not change the current working directory in ↵Serhiy Storchaka2022-06-221-34/+65
| | | | | | | | | | | shutil.make_archive() if possible (GH-93160) It is no longer changed when create a zip or tar archive. It is still changed for custom archivers registered with shutil.register_archive_format() if root_dir is not None. Co-authored-by: Éric <merwok@netwok.org> Co-authored-by: Łukasz Langa <lukasz@langa.pl>
* shutil._copyfileobj_readinto: tiny speedup (#92377)Lucinda May Phipps2022-05-201-1/+2
|
* gh-88513: clarify shutil.copytree's dirs_exist_ok arg (GH-91434)Jack DeVries2022-04-121-3/+5
| | | | | * add a paragraph to document this kwarg in detail * update docstring in the source accordingly
* bpo-46245: Add optional parameter dir_fd in shutil.rmtree() (GH-30365)Serhiy Storchaka2022-03-091-5/+12
|
* bpo-39327: Close file descriptors as soon as possible in shutil.rmtree ↵Lital Natan2022-02-201-2/+10
| | | | | | | | | (GH-31384) It fixes the "Text File Busy" OSError when using 'rmtree' on a windows-managed filesystem in via the VirtualBox shared folder (and possible other scenarios like a windows-managed network file system).
* bpo-42174: fallback to sane values if the columns or lines are 0 in ↵Filipe Laíns2021-10-191-2/+2
| | | | | | | | | | | | | get_terminal_size (GH-29046) I considered only falling back when both were 0, but that still seems wrong, and the highly popular rich[1] library does it this way, so I thought we should probably inherit that behavior. [1] https://github.com/willmcgugan/rich Signed-off-by: Filipe Laíns <lains@riseup.net> Co-authored-by: Łukasz Langa <lukasz@langa.pl>
* bpo-45234: Fix FileNotFound exception raised instead of IsADirectoryError in ↵andrei kulakov2021-09-211-29/+30
| | | | | shutil.copyfile() (GH-28421) This was a regression from fixing BPO-43219.
* bpo-33671 fix orphaned comment in shutil.copyfileobj (GH-27516)Anthony Sottile2021-07-311-1/+1
|
* bpo-43219: shutil.copyfile, raise a less confusing exception instead of ↵andrei kulakov2021-07-101-21/+29
| | | | | | IsADirectoryError (GH-27049) Fixes the misleading IsADirectoryError to be FileNotFoundError.
* bpo-43650: Fix MemoryError on zip.read in shutil._unpack_zipfile for large ↵Igor Bolshakov2021-05-171-10/+6
| | | | | | | files (GH-25058) `shutil.unpack_archive()` tries to read the whole file into memory, making no use of any kind of smaller buffer. Process crashes for really large files: I.e. archive: ~1.7G, unpacked: ~10G. Before the crash it can easily take away all available RAM on smaller systems. Had to pull the code form `zipfile.Zipfile.extractall()` to fix this Automerge-Triggered-By: GH:gpshead
* bpo-43743 add comment stating _USE_CP_SENDFILE should not be removed (#26024)Giampaolo Rodola2021-05-101-0/+2
|
* bpo-41718: subprocess imports grp and pwd on demand (GH-24987)Victor Stinner2021-03-231-12/+14
| | | | | The shutil and subprocess modules now only import the grp and pwd modules when they are needed, which is not the case by default in subprocess.
* bpo-42782: Fail fast for permission errors in shutil.move() (GH-24001)Winson Luk2021-03-021-0/+11
| | | | | * Fail fast in shutil.move() to avoid creating destination directories on failure. Co-authored-by: Zackery Spytz <zspytz@gmail.com>
* bpo-42014: shutil.rmtree: call onerror with correct function (GH-22585)Michal Čihař2020-11-101-1/+1
| | | | | | | The onerror is supposed to be called with failed function, but in this case lstat is wrongly used instead of open. Not sure if this needs bug or not... Automerge-Triggered-By: GH:hynek
* bpo-40592: shutil.which will not return None anymore if ; is the last char ↵Christopher Marchfelder2020-10-231-1/+6
| | | | | | in PATHEXT (GH-20088) shutil.which will not return None anymore for empty str in PATHEXT Empty PATHEXT will now be defaulted to _WIN_DEFAULT_PATHEXT
* bpo-39184: Add audit events to functions in `fcntl`, `msvcrt`, `os`, ↵Saiyang Gou2020-02-131-0/+10
| | | | `resource`, `shutil`, `signal`, `syslog` (GH-18407)
* bpo-39390 shutil: fix argument types for ignore callback (GH-18122)mbarkhau2020-01-241-1/+1
|
* bpo-38688, shutil.copytree: consume iterator and create list of entries to ↵Bruno P. Kinoshita2019-11-271-6/+7
| | | | prevent infinite recursion (GH-17098)
* bpo-38319: Fix shutil._fastcopy_sendfile(): set sendfile() max block size ↵Giampaolo Rodola2019-10-011-3/+7
| | | | (GH-16491)
* bpo-32689: Updates shutil.move to allow for Path objects to be used as ↵Maxwell A McKinnon2019-10-011-2/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | source arg (GH-15326) Important work originally done by @emilyemorehouse two years ago and nearly ready to go in. This bug has affected many people and in some cases has been a dealbreaker to the adoption of the otherwise wonderful pathlib and PEP519. https://stackoverflow.com/questions/33625931/copy-file-with-pathlib-in-python. This adds the outstanding test request from that PR @vstinner (https://github.com/python/cpython/pull/5393). Test fails without the change, passes with it, along with every other test in test_shutil. Some variants were experimented with to make the one line change and the most performant one was picked. # Added Test for PathLike directory destination, the current fail case ``` Lib/test/test_shutil.py::TestMove::test_move_file_pathlike FAILED [100%] ============================================================== FAILURES =============================================================== __________________________________________________ TestMove.test_move_file_pathlike ___________________________________________________ self = <test.test_shutil.TestMove testMethod=test_move_file_pathlike> def test_move_file_pathlike(self): # Move a file to another location on the same filesystem. src = pathlib.Path(self.src_file) > self._check_move_file(src, self.dst_dir, self.dst_file) Lib/test/test_shutil.py:1563: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Lib/test/test_shutil.py:1545: in _check_move_file shutil.move(src, dst) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:562: in move real_dst = os.path.join(dst, _basename(src)) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ path = PosixPath('/var/folders/r2/psq74t5x3nbfzlph8bh2pvdw0000gn/T/tmp9ie0wh9_/foo') def _basename(path): # A basename() variant which first strips the trailing slash, if present. # Thus we always get the last component of the path, even for directories. sep = os.path.sep + (os.path.altsep or '') > return os.path.basename(path.rstrip(sep)) E AttributeError: 'PosixPath' object has no attribute 'rstrip' /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:526: AttributeError ============================================== 1 failed, 102 deselected in 0.30 seconds =============================================== ``` After change: ``` ========================================================= test session starts ========================================================= platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7 cachedir: .pytest_cache rootdir: /Users/maxwellmckinnon/dev/cpython plugins: cov-2.7.1, mock-1.10.4 collected 103 items / 102 deselected / 1 selected Lib/test/test_shutil.py::TestMove::test_move_file_pathlike PASSED [100%] ============================================== 1 passed, 102 deselected in 0.06 seconds =============================================== ``` Running all the tests in test_shutil.py ``` ╰─ pytest Lib/test/test_shutil.py -v ========================================================= test session starts ========================================================= platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7 cachedir: .pytest_cache rootdir: /Users/maxwellmckinnon/dev/cpython plugins: cov-2.7.1, mock-1.10.4 collected 103 items Lib/test/test_shutil.py::TestShutil::test_chown PASSED [ 0%] Lib/test/test_shutil.py::TestShutil::test_copy PASSED [ 1%] ... Lib/test/test_shutil.py::TermsizeTests::test_stty_match SKIPPED [ 99%] Lib/test/test_shutil.py::PublicAPITests::test_module_all_attribute PASSED [100%] ================================================ 96 passed, 7 skipped in 1.25 seconds ================================================= ``` # Performance Considerations Is it considered poor form to get rid of _basename altogether and make use of pathlib in the move function? I'm not sure if the idea is for all these modules to strictly avoid circular dependencies. They are already using os.path which is just as much a citizen in 3.8 as pathlib right? e.g. `real_dst = os.path.join(dst, _basename(src))` becomes `real_dst = Path(dst) / Path(src).name` I've looked around and familiarized myself, and I now think importing pathlib here is fine. My only remaining concern is that of performance. Here's the performance difference for this step. ``` In [46]: %timeit real_dst = os.path.join("a/b/c", _basename('b/')) 2.71 µs ± 62.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each) In [47]: %timeit real_dst = Path("a/b/c") / Path('b/').name 12.4 µs ± 65.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each) ``` Is 10us significant or insignificant compared to the least expensive operation this function will do? I don't know. Let's find out. ``` In [55]: %timeit os.rename('/tmp/a/a.txt', '/tmp/a/b.txt'); os.rename('/tmp/a/b.txt', '/tmp/a/a.txt') 124 µs ± 2.18 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each) ``` 62us to rename. 10us seems significant enough that we wouldn't want to favor the Path sugar suggestion. 16% speed decrease from adding the 10us. What do people think? I was hoping to get to use pathlib.Path here, but I suspect for this low level move, it should be as fast as possible, and 16% is not worth one line of sugary code to me. https://bugs.python.org/issue32689 Automerge-Triggered-By: @gvanrossum
* Clarify that shutil's copy functions can accept path-like values (GH-15141)Boris Verhovsky2019-09-091-1/+2
|
* bpo-37834: Prevent shutil.rmtree exception (GH-15602)Ned Deily2019-08-291-1/+1
| | | | when built on non-Windows system without fd system call support, like older versions of macOS.
* bpo-37834: Normalise handling of reparse points on Windows (GH-15231)Steve Dower2019-08-211-7/+41
| | | | | | | | | | bpo-37834: Normalise handling of reparse points on Windows * ntpath.realpath() and nt.stat() will traverse all supported reparse points (previously was mixed) * nt.lstat() will let the OS traverse reparse points that are not name surrogates (previously would not traverse any reparse point) * nt.[l]stat() will only set S_IFLNK for symlinks (previous behaviour) * nt.readlink() will read destinations for symlinks and junction points only bpo-1311: os.path.exists('nul') now returns True on Windows * nt.stat('nul').st_mode is now S_IFCHR (previously was an error)
* bpo-37363: Add audit events for a range of modules (GH-14301)Steve Dower2019-06-241-0/+3
|
* bpo-26660, bpo-35144: Fix permission errors in TemporaryDirectory cleanup. ↵Serhiy Storchaka2019-05-311-3/+8
| | | | | | | (GH-10320) TemporaryDirectory.cleanup() failed when non-writeable or non-searchable files or directories were created inside a temporary directory.
* bpo-36610: shutil.copyfile(): use sendfile() on Linux only (GH-13675)Giampaolo Rodola2019-05-301-6/+6
| | | | ...and avoid using it on Solaris as it can raise EINVAL if offset is equal or bigger than the size of the file
* bpo-24564: shutil.copystat(): ignore EINVAL on os.setxattr() (GH-13369)Ying Wang2019-05-301-2/+3
|
* bpo-24538: Fix bug in shutil involving the copying of xattrs to read-only ↵Olexa Bilaniuk2019-05-101-1/+3
| | | | | | | | files. (PR-13212) Extended attributes can only be set on user-writeable files, but shutil previously first chmod()ed the destination file to the source's permissions and then tried to copy xattrs. This will cause failures if attempting to copy read-only files with xattrs, as occurs with Git clones on Lustre FS.
* bpo-35755: Don't say "to mimick Unix which command behavior" (GH-12861)Victor Stinner2019-04-171-1/+1
|
* bpo-35755: shutil.which() uses os.confstr("CS_PATH") (GH-12858)Victor Stinner2019-04-171-1/+12
| | | | | | | | | | | | | | | | shutil.which() and distutils.spawn.find_executable() now use os.confstr("CS_PATH") if available instead of os.defpath, if the PATH environment variable is not set. Don't use os.confstr("CS_PATH") nor os.defpath if the PATH environment variable is set to an empty string to mimick Unix 'which' command behavior. Changes: * find_executable() now starts by checking for the executable in the current working directly case. Add an explicit "if not path: return None". * Add tests for PATH='' (empty string), PATH=':' and for PATHEXT.
* bpo-36103: change default buffer size of shutil.copyfileobj() (GH-12115)Inada Naoki2019-03-021-1/+1
| | | | | | | | | | | | | | | | It is changed from 16KiB to 64KiB. The previous default value is used since 1990. coreutils chose 128 KiB as minimum buffer size for block device I/O. But shutil.copyfileobj() can be used for non block devices. So I choose more conservative value. As my quick benchmark, performance difference between 64KiB and 128 KiB is up to ~5%. On the other hand, performance difference between 32 KiB and 64 KiB can be more than 10% when file is fully buffered. This is why 64 KiB is rational value.
* bpo-35652: shutil.copytree(copy_function=...) erroneously pass DirEntry ↵Giampaolo Rodola2019-02-261-1/+1
| | | | instead of path str (GH-11997)
* Clean up code which checked presence of os.{stat,lstat,chmod} (#11643)Anthony Sottile2019-02-251-3/+1
|
* bpo-33671: allow setting shutil.copyfile() bufsize globally (GH-12016)Giampaolo Rodola2019-02-241-1/+3
|
generated by cgit v0.12 at 2024-12-03 18:36:59 (GMT)