summaryrefslogtreecommitdiffstats
path: root/Python/thread_pthread.h
Commit message (Collapse)AuthorAgeFilesLines
* bpo-38106: Fix race in pthread PyThread_release_lock() (GH-16047)Kirill Smelkov2019-10-031-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix race in PyThread_release_lock that was leading to memory corruption and deadlocks. The fix applies to POSIX systems where Python locks are implemented with mutex and condition variable because POSIX semaphores are either not provided, or are known to be broken. One particular example of such system is macOS. On Darwin, even though this is considered as POSIX, Python uses mutex+condition variable to implement its lock, and, as of 2019-08-28, Py2.7 implementation, even though similar issue was fixed for Py3 in 2012, contains synchronization bug: the condition is signalled after mutex unlock while the correct protocol is to signal condition from under mutex: https://github.com/python/cpython/blob/v2.7.16-127-g0229b56d8c0/Python/thread_pthread.h#L486-L506 https://github.com/python/cpython/commit/187aa545165d (py3 fix) PyPy has the same bug for both pypy2 and pypy3: https://bitbucket.org/pypy/pypy/src/578667b3fef9/rpython/translator/c/src/thread_pthread.c#lines-443:465 https://bitbucket.org/pypy/pypy/src/5b42890d48c3/rpython/translator/c/src/thread_pthread.c#lines-443:465 Signalling condition outside of corresponding mutex is considered OK by POSIX, but in Python context it can lead to at least memory corruption if we consider the whole lifetime of python level lock. For example the following logical scenario: T1 T2 sema = Lock() sema.acquire() sema.release() sema.acquire() free(sema) ... can translate to the next C-level calls: T1 T2 # sema = Lock() sema = malloc(...) sema.locked = 0 pthread_mutex_init(&sema.mut) pthread_cond_init (&sema.lock_released) # sema.acquire() pthread_mutex_lock(&sema.mut) # sees sema.locked == 0 sema.locked = 1 pthread_mutex_unlock(&sema.mut) # sema.release() pthread_mutex_lock(&sema.mut) sema.locked = 0 pthread_mutex_unlock(&sema.mut) # OS scheduler gets in and relinquishes control from T2 # to another process ... # second sema.acquire() pthread_mutex_lock(&sema.mut) # sees sema.locked == 0 sema.locked = 1 pthread_mutex_unlock(&sema.mut) # free(sema) pthread_mutex_destroy(&sema.mut) pthread_cond_destroy (&sema.lock_released) free(sema) # ... e.g. malloc() which returns memory where sema was ... # OS scheduler returns control to T2 # sema.release() continues # # BUT sema was already freed and writing to anywhere # inside sema block CORRUPTS MEMORY. In particular if # _another_ python-level lock was allocated where sema # block was, writing into the memory can have effect on # further synchronization correctness and in particular # lead to deadlock on lock that was next allocated. pthread_cond_signal(&sema.lock_released) Note that T2.pthread_cond_signal(&sema.lock_released) CORRUPTS MEMORY as it is called when sema memory was already freed and is potentially reallocated for another object. The fix is to move pthread_cond_signal to be done under corresponding mutex: # sema.release() pthread_mutex_lock(&sema.mut) sema.locked = 0 pthread_cond_signal(&sema.lock_released) pthread_mutex_unlock(&sema.mut) To do so this patch cherry-picks thread_pthread.h part of the following 3.2 commit: commit 187aa545165d8d5eac222ecce29c8a77e0282dd4 Author: Kristján Valur Jónsson <kristjan@ccpgames.com> Date: Tue Jun 5 22:17:42 2012 +0000 Signal condition variables with the mutex held. Destroy condition variables before their mutexes. Python/ceval_gil.h | 9 +++++---- Python/thread_pthread.h | 15 +++++++++------ 2 files changed, 14 insertions(+), 10 deletions(-) (ceval_gil.h is Python3 specific and does not apply to Python2.7) The bug was there since 1994 - since at least [1]. It was discussed in 2001 with original code author[2], but the code was still considered to be race-free. In 2010 the place where pthread_cond_signal should be - before or after pthread_mutex_unlock - was discussed with the rationale to avoid threads bouncing[3,4,5], and in 2012 pthread_cond_signal was moved to be called from under mutex, but only for CPython3[6,7]. In 2019 the bug was (re-)discovered while testing Pygolang[8] on macOS with CPython2 and PyPy2 and PyPy3. [1] https://github.com/python/cpython/commit/2c8cb9f3d240 [2] https://bugs.python.org/issue433625 [3] https://bugs.python.org/issue8299#msg103224 [4] https://bugs.python.org/issue8410#msg103313 [5] https://bugs.python.org/issue8411#msg113301 [6] https://bugs.python.org/issue15038#msg163187 [7] https://github.com/python/cpython/commit/187aa545165d [8] https://pypi.org/project/pygolang (cherry picked from commit 187aa545165d8d5eac222ecce29c8a77e0282dd4) Co-Authored-By: Kristján Valur Jónsson <kristjan@ccpgames.com>
* bpo-33015: Use malloc() in PyThread_start_new_thread() (GH-10829)Victor Stinner2018-11-301-3/+3
| | | | | The pthread implementation of PyThread_start_new_thread() now uses malloc/free rather than PyMem_Malloc/PyMem_Free, since the latters are not thread-safe.
* bpo-33015: Fix UB in pthread PyThread_start_new_thread (GH-6008) (GH-10823)Victor Stinner2018-11-301-4/+36
| | | | | | | | | Fix an undefined behaviour in the pthread implementation of PyThread_start_new_thread(): add a function wrapper to always return NULL. Add pythread_callback struct and pythread_wrapper() to thread_pthread.h. (cherry picked from commit 9eea6eaf23067880f4af3a130e3f67c9812e2f30)
* [2.7] bpo-30765: Avoid blocking when PyThread_acquire_lock() is asked not to ↵Antoine Pitrou2017-06-261-18/+28
| | | | | | | | | | | | | | | | | | (GH-2403) (#2420) * [2.7] bpo-30765: Avoid blocking when PyThread_acquire_lock() is asked not to (GH-2403) * bpo-30765: Avoid blocking when PyThread_acquire_lock() is asked not to lock This is especially important if PyThread_acquire_lock() is called reentrantly (for example from a signal handler). * Update 2017-06-26-14-29-50.bpo-30765.Q5iBmf.rst * Avoid core logic when taking the mutex failed. (cherry picked from commit f84ac420c2af98339678744953869cad3c253281) * Remove test undef
* Issue #21963: backout issue #1856 patch (avoid crashes and lockups whenAntoine Pitrou2014-11-211-2/+2
| | | | | | daemon threads run while the interpreter is shutting down; instead, these threads are now killed when they try to take the GIL), as it seems to break some existing code.
* avoid crashes and lockups from daemon threads during interpreter shutdown ↵Benjamin Peterson2014-06-171-2/+2
| | | | (#1856)
* Issue #18256: Compilation fix for recent AIX releases. Patch by David Edelsohn.Antoine Pitrou2013-06-181-0/+3
|
* Closes #16588: Silence unused-but-set warnings in Python/thread_pthread.hJesus Cea2012-12-051-0/+3
|
* Issue #9670: Increase the default stack size for secondary threads onNed Deily2011-05-281-0/+12
| | | | | | Mac OS X and FreeBSD to reduce the chances of a crash instead of a "maximum recursion depth" RuntimeError exception. (patch by Ronald Oussoren)
* Issue9670: Back out changeset b0d2b696da19; test fails on other platformsNed Deily2011-04-091-12/+0
| | | | and on OS X with pydebug.
* Issue #9670: Increase the default stack size for secondary threads onNed Deily2011-04-091-0/+12
| | | | | | Mac OS X and FreeBSD to reduce the chances of a crash instead of a "maximum recursion depth" RuntimeError exception. (Patch by Ronald Oussoren)
* Untabify C files. Will watch buildbots.Antoine Pitrou2010-05-091-219/+219
|
* #4852: Remove dead code in every thread implementation, unused for many years.Amaury Forgeot d'Arc2010-02-231-43/+3
|
* Patch #1454481: Make thread stack size runtime tunable.Andrew MacIntyre2006-06-131-3/+74
| | | | | | | | | | Heavily revised, comprising revisions: 46640 - original trunk revision (backed out in r46655) 46647 - markup fix (backed out in r46655) 46692:46918 merged from branch aimacintyre-sf1454481 branch tested on buildbots (Windows buildbots had problems not related to these changes).
* Revert revisions:Tim Peters2006-06-041-61/+2
| | | | | | | | | | | | | | | | 46640 Patch #1454481: Make thread stack size runtime tunable. 46647 Markup fix The first is causing many buildbots to fail test runs, and there are multiple causes with seemingly no immediate prospects for repairing them. See python-dev discussion. Note that a branch can (and should) be created for resolving these problems, like svn copy svn+ssh://svn.python.org/python/trunk -r46640 svn+ssh://svn.python.org/python/branches/NEW_BRANCH followed by merging rev 46647 to the new branch.
* Patch #1454481: Make thread stack size runtime tunable.Andrew MacIntyre2006-06-041-2/+61
|
* Patch #1396919: Reenable the system scope threads on FreeBSD 5.4Hye-Shik Chang2006-03-231-1/+11
| | | | | and later versions because they bumped the default setting to get our basic tests to run correctly..
* Don't call memset() before checking that the ptr is not NULL.Armin Rigo2005-09-201-1/+1
|
* Add 0 to _POSIX_SEMAPHORES. Will backport to 2.4.Martin v. Löwis2005-03-281-1/+3
|
* Patch #1163249 - Correctly handle _POSIX_SEMAPHORES == -1 to mean noAnthony Baxter2005-03-161-0/+4
| | | | support for posix semaphores.
* This closes patch:Michael W. Hudson2004-07-071-11/+0
| | | | | | | | | | | | | | | | | [ 960406 ] unblock signals in threads although the changes do not correspond exactly to any patch attached to that report. Non-main threads no longer have all signals masked. A different interface to readline is used. The handling of signals inside calls to PyOS_Readline is now rather different. These changes are all a bit scary! Review and cross-platform testing much appreciated.
* SF Patch #902444: Use process scope thread on FreeBSD. System scopeHye-Shik Chang2004-03-041-1/+1
| | | | | is too expensive on FreeBSD's KSE threading infrastructure and even test_threadedimport fails on default setting.
* typoSkip Montanaro2004-03-031-1/+1
|
* Remove support for systems defining Py_PTHREAD_D[467] inSkip Montanaro2004-02-071-72/+0
| | | | Python/thread_pthread.h.
* remove DGUX support.Skip Montanaro2004-01-171-10/+2
|
* Getting rid of code dependent on GUSI or the MetroWerks compiler.Jack Jansen2003-11-191-8/+0
|
* Patch 775605: Cygwin pthread_sigmask() workaround patchJason Tishler2003-07-221-1/+1
| | | | | | | | | | | | | | | | | | Cygwin's pthread_sigmask() implementation appears to be buggy. This patch works around this problem by using sigprocmask() instead. This patch is implemented in a general way so it could be used by other platforms too. If this approach is deemed too risky, then I can work up a patch that just hacks Python/thread_pthread.h for Cygwin. Note that I tested this patch against 2.3c1 under Red Hat Linux 8.0 too. [snip] And finally, I need someone to regenerate pyconfig.h.in and configure with the same versions of the autotools that are normally used by Python. Neal kindly regenerated pyconfig.h.in and configure for me.
* Patch #716969: Detect thread creation failure. Will backport to 2.2.Martin v. Löwis2003-04-191-6/+8
|
* Patch #711835: Remove unnecessary lock operations. Will backport to 2.2.Martin v. Löwis2003-04-181-8/+4
|
* Don't use Posix semaphores on Solaris 8. Fixes #662787.Martin v. Löwis2003-01-211-1/+1
|
* Patch #650415: Avoid redefinition of macros.Martin v. Löwis2002-12-111-0/+12
|
* Add compile-time errors for unsupported systems.Martin v. Löwis2002-12-021-0/+5
|
* Patch #618347: Work around Solaris 2.6 pthread.h bug. Will backport to 2.2.Martin v. Löwis2002-10-041-2/+2
|
* Document that _POSIX_SEMAPHORES is predefined.Martin v. Löwis2002-03-171-0/+4
|
* Patch #525532: Add support for POSIX semaphores.Martin v. Löwis2002-03-171-0/+119
|
* Remove the unused & broken PyThread_*_sema() functions and related constants.Fred Drake2002-01-191-98/+0
| | | | This closes SF patch #504215.
* Workaround for what is probably a problem in Apple's gcc: <pthread.h> failsJack Jansen2002-01-151-0/+6
| | | | | on a function pointer formal argument called "destructor", which is typedeffed as a different function pointer type in object.h.
* Fix memory leak. This is part of SF patch #478006.Fred Drake2001-11-091-1/+1
|
* Partial patch from SF #452266, by Jason Petrone.Guido van Rossum2001-10-161-2/+6
| | | | | | This changes Pythread_start_thread() to return the thread ID, or -1 for an error. (It's technically an incompatible API change, but I doubt anyone calls it.)
* Do not define _POSIX_THREADS if unistd.h defines it.Martin v. Löwis2001-10-151-1/+1
| | | | Check for pthread_sigmask before using it. Fixes remaining problem in #470781.
* Add SF patch #468347 -- mask signals for non-main pthreads, by Jason Lowe:Guido van Rossum2001-10-121-0/+26
| | | | | | | | | | | | | | | | | | | This patch updates Python/thread_pthread.h to mask all signals for any thread created. This will keep all signals masked for any thread that isn't the initial thread. For Solaris and Linux, the two platforms I was able to test it on, it solves bug #465673 (pthreads need signal protection) and probably will solve bug #219772 (Interactive InterPreter+ Thread -> core dump at exit). I'd be great if this could get some testing on other platforms, especially HP-UX pre 11.00 and post 11.00, as I had to make some guesses for the DCE thread case. AIX is also a concern as I saw some mention of using sigthreadmask() as a pthread_sigmask() equivalent, but this patch doesn't use sigthreadmask(). I don't have access to AIX.
* Improve threading on Solaris, according to SF patch #460269, submittedGuido van Rossum2001-09-101-3/+8
| | | | | | | | | | by bbrox@bbrox.org / lionel.ulmer@free.fr. This adds a configure check and if all goes well turns on the PTHREAD_SCOPE_SYSTEM thread attribute for new threads. This should remove the need to add tiny sleeps at the start of threads to allow other threads to be scheduled.
* GUSI on the Mac creates threads with a default stack size of 20KB, which isJack Jansen2001-08-291-1/+21
| | | | | not enough for Python. Increased the stacksize to a (somewhat arbitrary) 64KB.
* REMOVED all CWI, CNRI and BeOpen copyright markings.Guido van Rossum2000-09-011-9/+0
| | | | This should match the situation in the 1.6b1 tree.
* This patch partly (some stuff went in already) ports Python to Monterey.Trent Mick2000-08-231-0/+11
| | | | | | | | | | - Fix bug in thread_pthread.h::PyThread_get_thread_ident() where sizeof(pthread) < sizeof(long). - Add 'configure' for: - SIZEOF_PTHREAD is pthread_t can be included via <pthread.h> - setting Monterey system name - appropriate CC,LINKCC,LDSHARED,OPT, and CCSHARED for Monterey - Add section in README for Monterey build
* Mass ANSIfication of function definitions. Doesn't cover all 'extern'Thomas Wouters2000-07-221-21/+40
| | | | declarations yet, those come later.
* Change copyright notice - 2nd try.Guido van Rossum2000-06-301-6/+0
|
* Change copyright notice.Guido van Rossum2000-06-301-22/+7
|
* Trent Mick <trentm@activestate.com>:Fred Drake2000-06-301-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | The common technique for printing out a pointer has been to cast to a long and use the "%lx" printf modifier. This is incorrect on Win64 where casting to a long truncates the pointer. The "%p" formatter should be used instead. The problem as stated by Tim: > Unfortunately, the C committee refused to define what %p conversion "looks > like" -- they explicitly allowed it to be implementation-defined. Older > versions of Microsoft C even stuck a colon in the middle of the address (in > the days of segment+offset addressing)! The result is that the hex value of a pointer will maybe/maybe not have a 0x prepended to it. Notes on the patch: There are two main classes of changes: - in the various repr() functions that print out pointers - debugging printf's in the various thread_*.h files (these are why the patch is large) Closes SourceForge patch #100505.
* Rob Riggs wrote:Guido van Rossum1999-03-151-2/+2
| | | | | | | | | | | | | | | """ Spec says that on success pthread_create returns 0. It does not say that an error code will be < 0. Linux glibc2 pthread_create() returns ENOMEM (12) when one exceed process limits. (It looks like it should return EAGAIN, but that's another story.) For reference, see: http://www.opengroup.org/onlinepubs/7908799/xsh/pthread_create.html """ [I have a feeling that similar bugs were fixed before; perhaps someone could check that all error checks no check for != 0?]
generated by cgit v0.12 at 2025-12-30 20:17:53 (GMT)