Skip to content

Commit 88be4b3

Browse files
encukouencukou
committed
Expose Py_CriticalSection in Stable ABI
1 parent f2c7c0d commit 88be4b3

8 files changed

Lines changed: 182 additions & 95 deletions

File tree

‎Doc/c-api/synchronization.rst‎

Lines changed: 46 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -84,11 +84,6 @@ there is no :c:type:`PyObject` -- for example, when working with a C type that
8484
does not extend or wrap :c:type:`PyObject` but still needs to call into the C
8585
API in a manner that might lead to deadlocks.
8686
87-
The functions and structs used by the macros are exposed for cases
88-
where C macros are not available. They should only be used as in the
89-
given macro expansions. Note that the sizes and contents of the structures may
90-
change in future Python versions.
91-
9287
.. note::
9388
9489
Operations that need to lock two objects at once must use
@@ -114,12 +109,15 @@ section API avoids potential deadlocks due to reentrancy and lock ordering
114109
by allowing the runtime to temporarily suspend the critical section if the
115110
code triggered by the finalizer blocks and calls :c:func:`PyEval_SaveThread`.
116111
112+
.. _critical-section-macros:
113+
117114
.. c:macro:: Py_BEGIN_CRITICAL_SECTION(op)
118115
119116
Acquires the per-object lock for the object *op* and begins a
120117
critical section.
121118
122-
In the free-threaded build, this macro expands to::
119+
In the free-threaded build, and when building for the
120+
:ref:`Stable ABI <stable-abi>`, this macro expands to::
123121
124122
{
125123
PyCriticalSection _py_cs;
@@ -150,7 +148,8 @@ code triggered by the finalizer blocks and calls :c:func:`PyEval_SaveThread`.
150148
151149
Ends the critical section and releases the per-object lock.
152150
153-
In the free-threaded build, this macro expands to::
151+
In the free-threaded build, and when building for the
152+
:ref:`Stable ABI <stable-abi>`, this macro expands to::
154153
155154
PyCriticalSection_End(&_py_cs);
156155
}
@@ -179,7 +178,8 @@ code triggered by the finalizer blocks and calls :c:func:`PyEval_SaveThread`.
179178
180179
Locks the mutexes *m1* and *m2* and begins a critical section.
181180
182-
In the free-threaded build, this macro expands to::
181+
In the free-threaded build, and when building for the
182+
:ref:`Stable ABI <stable-abi>`, this macro expands to::
183183
184184
{
185185
PyCriticalSection2 _py_cs2;
@@ -196,7 +196,8 @@ code triggered by the finalizer blocks and calls :c:func:`PyEval_SaveThread`.
196196
197197
Ends the critical section and releases the per-object locks.
198198
199-
In the free-threaded build, this macro expands to::
199+
In the free-threaded build, and when building for the
200+
:ref:`Stable ABI <stable-abi>`, this macro expands to::
200201
201202
PyCriticalSection2_End(&_py_cs2);
202203
}
@@ -205,6 +206,42 @@ code triggered by the finalizer blocks and calls :c:func:`PyEval_SaveThread`.
205206
206207
.. versionadded:: 3.13
207208
209+
Low-level critical section API
210+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
211+
212+
The following functions and structs are exposed for cases where C macros
213+
are not available.
214+
215+
.. c:function:: void PyCriticalSection_Begin(PyCriticalSection *c, PyObject *op)
216+
void PyCriticalSection_End(PyCriticalSection *c)
217+
void PyCriticalSection2_Begin(PyCriticalSection2 *c, PyObject *a, PyObject *b)
218+
void PyCriticalSection2_End(PyCriticalSection2 *c);
219+
220+
To be used only as in the macro expansions
221+
listed :ref:`earlier in this section <critical-section-macros>`.
222+
223+
.. versionadded:: 3.13
224+
225+
.. c:type:: PyCriticalSection
226+
PyCriticalSection2
227+
228+
To be used only as in the macro expansions
229+
listed :ref:`earlier in this section <critical-section-macros>`.
230+
Note that the contents of the structures are private and their meaning may
231+
change in future Python versions.
232+
233+
.. versionadded:: 3.13
234+
235+
.. c:function:: void PyCriticalSection_BeginMutex(PyCriticalSection *c, PyMutex *m);
236+
void PyCriticalSection2_BeginMutex(PyCriticalSection2 *c, PyMutex *m1, PyMutex *m2);
237+
238+
.. (These need to be in a separate section without a Stable ABI anotation.)
239+
240+
To be used only as in the macro expansions
241+
listed :ref:`earlier in this section <critical-section-macros>`.
242+
243+
.. versionadded:: 3.14
244+
208245
209246
Legacy locking APIs
210247
-------------------

‎Doc/data/stable_abi.dat‎

Lines changed: 10 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Include/cpython/critical_section.h‎

Lines changed: 7 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,6 @@
22
# error "this header file must not be included directly"
33
#endif
44

5-
// Python critical sections
6-
//
7-
// Conceptually, critical sections are a deadlock avoidance layer on top of
8-
// per-object locks. These helpers, in combination with those locks, replace
9-
// our usage of the global interpreter lock to provide thread-safety for
10-
// otherwise thread-unsafe objects, such as dict.
11-
//
12-
// NOTE: These APIs are no-ops in non-free-threaded builds.
13-
//
145
// Straightforward per-object locking could introduce deadlocks that were not
156
// present when running with the GIL. Threads may hold locks for multiple
167
// objects simultaneously because Python operations can nest. If threads were
@@ -43,52 +34,19 @@
4334
// `_PyThreadState_Attach()`, it resumes the top-most (i.e., most recent)
4435
// critical section by reacquiring the associated lock or locks. See
4536
// `_PyCriticalSection_Resume()`.
46-
//
47-
// NOTE: Only the top-most critical section is guaranteed to be active.
48-
// Operations that need to lock two objects at once must use
49-
// `Py_BEGIN_CRITICAL_SECTION2()`. You *CANNOT* use nested critical sections
50-
// to lock more than one object at once, because the inner critical section
51-
// may suspend the outer critical sections. This API does not provide a way
52-
// to lock more than two objects at once (though it could be added later
53-
// if actually needed).
54-
//
55-
// NOTE: Critical sections implicitly behave like reentrant locks because
56-
// attempting to acquire the same lock will suspend any outer (earlier)
57-
// critical sections. However, they are less efficient for this use case than
58-
// purposefully designed reentrant locks.
59-
//
60-
// Example usage:
61-
// Py_BEGIN_CRITICAL_SECTION(op);
62-
// ...
63-
// Py_END_CRITICAL_SECTION();
64-
//
65-
// To lock two objects at once:
66-
// Py_BEGIN_CRITICAL_SECTION2(op1, op2);
67-
// ...
68-
// Py_END_CRITICAL_SECTION2();
69-
70-
typedef struct PyCriticalSection PyCriticalSection;
71-
typedef struct PyCriticalSection2 PyCriticalSection2;
72-
73-
PyAPI_FUNC(void)
74-
PyCriticalSection_Begin(PyCriticalSection *c, PyObject *op);
7537

7638
PyAPI_FUNC(void)
7739
PyCriticalSection_BeginMutex(PyCriticalSection *c, PyMutex *m);
7840

79-
PyAPI_FUNC(void)
80-
PyCriticalSection_End(PyCriticalSection *c);
81-
82-
PyAPI_FUNC(void)
83-
PyCriticalSection2_Begin(PyCriticalSection2 *c, PyObject *a, PyObject *b);
84-
8541
PyAPI_FUNC(void)
8642
PyCriticalSection2_BeginMutex(PyCriticalSection2 *c, PyMutex *m1, PyMutex *m2);
8743

88-
PyAPI_FUNC(void)
89-
PyCriticalSection2_End(PyCriticalSection2 *c);
90-
9144
#ifndef Py_GIL_DISABLED
45+
#undef Py_BEGIN_CRITICAL_SECTION
46+
#undef Py_END_CRITICAL_SECTION
47+
#undef Py_BEGIN_CRITICAL_SECTION2
48+
#undef Py_END_CRITICAL_SECTION2
49+
9250
# define Py_BEGIN_CRITICAL_SECTION(op) \
9351
{
9452
# define Py_BEGIN_CRITICAL_SECTION_MUTEX(mutex) \
@@ -101,54 +59,17 @@ PyCriticalSection2_End(PyCriticalSection2 *c);
10159
{
10260
# define Py_END_CRITICAL_SECTION2() \
10361
}
104-
#else /* !Py_GIL_DISABLED */
105-
106-
// NOTE: the contents of this struct are private and may change betweeen
107-
// Python releases without a deprecation period.
108-
struct PyCriticalSection {
109-
// Tagged pointer to an outer active critical section (or 0).
110-
uintptr_t _cs_prev;
111-
112-
// Mutex used to protect critical section
113-
PyMutex *_cs_mutex;
114-
};
115-
116-
// A critical section protected by two mutexes. Use
117-
// Py_BEGIN_CRITICAL_SECTION2 and Py_END_CRITICAL_SECTION2.
118-
// NOTE: the contents of this struct are private and may change betweeen
119-
// Python releases without a deprecation period.
120-
struct PyCriticalSection2 {
121-
PyCriticalSection _cs_base;
12262

123-
PyMutex *_cs_mutex2;
124-
};
125-
126-
# define Py_BEGIN_CRITICAL_SECTION(op) \
127-
{ \
128-
PyCriticalSection _py_cs; \
129-
PyCriticalSection_Begin(&_py_cs, _PyObject_CAST(op))
63+
#else /* !Py_GIL_DISABLED */
13064

13165
# define Py_BEGIN_CRITICAL_SECTION_MUTEX(mutex) \
13266
{ \
13367
PyCriticalSection _py_cs; \
13468
PyCriticalSection_BeginMutex(&_py_cs, mutex)
13569

136-
# define Py_END_CRITICAL_SECTION() \
137-
PyCriticalSection_End(&_py_cs); \
138-
}
139-
140-
# define Py_BEGIN_CRITICAL_SECTION2(a, b) \
141-
{ \
142-
PyCriticalSection2 _py_cs2; \
143-
PyCriticalSection2_Begin(&_py_cs2, _PyObject_CAST(a), _PyObject_CAST(b))
144-
14570
# define Py_BEGIN_CRITICAL_SECTION2_MUTEX(m1, m2) \
14671
{ \
14772
PyCriticalSection2 _py_cs2; \
14873
PyCriticalSection2_BeginMutex(&_py_cs2, m1, m2)
14974

150-
# define Py_END_CRITICAL_SECTION2() \
151-
PyCriticalSection2_End(&_py_cs2); \
152-
}
153-
154-
#endif
75+
#endif /* !Py_GIL_DISABLED */

‎Include/critical_section.h‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,91 @@
44
extern "C" {
55
#endif
66

7+
// Python critical sections
8+
//
9+
// Conceptually, critical sections are a deadlock avoidance layer on top of
10+
// per-object locks. These helpers, in combination with those locks, replace
11+
// our usage of the global interpreter lock to provide thread-safety for
12+
// otherwise thread-unsafe objects, such as dict.
13+
//
14+
// NOTE: These APIs are no-ops in non-free-threaded builds.
15+
//
16+
// NOTE: Only the top-most critical section is guaranteed to be active.
17+
// Operations that need to lock two objects at once must use
18+
// `Py_BEGIN_CRITICAL_SECTION2()`. You *CANNOT* use nested critical sections
19+
// to lock more than one object at once, because the inner critical section
20+
// may suspend the outer critical sections. This API does not provide a way
21+
// to lock more than two objects at once (though it could be added later
22+
// if actually needed).
23+
//
24+
// NOTE: Critical sections implicitly behave like reentrant locks because
25+
// attempting to acquire the same lock will suspend any outer (earlier)
26+
// critical sections. However, they are less efficient for this use case than
27+
// purposefully designed reentrant locks.
28+
//
29+
// Example usage:
30+
// Py_BEGIN_CRITICAL_SECTION(op);
31+
// ...
32+
// Py_END_CRITICAL_SECTION();
33+
//
34+
// To lock two objects at once:
35+
// Py_BEGIN_CRITICAL_SECTION2(op1, op2);
36+
// ...
37+
// Py_END_CRITICAL_SECTION2();
38+
39+
// NOTE: the contents of this struct are private and their meaning may
40+
// change betweeen Python releases without a deprecation period.
41+
typedef struct PyCriticalSection {
42+
// Tagged pointer to an outer active critical section (or 0).
43+
uintptr_t _cs_prev;
44+
45+
// Mutex used to protect critical section
46+
struct PyMutex *_cs_mutex;
47+
} PyCriticalSection;
48+
49+
// A critical section protected by two mutexes. Use
50+
// Py_BEGIN_CRITICAL_SECTION2 and Py_END_CRITICAL_SECTION2.
51+
// NOTE: the contents of this struct are private and may change betweeen
52+
// Python releases without a deprecation period.
53+
typedef struct PyCriticalSection2 {
54+
PyCriticalSection _cs_base;
55+
56+
struct PyMutex *_cs_mutex2;
57+
} PyCriticalSection2;
58+
59+
PyAPI_FUNC(void)
60+
PyCriticalSection_Begin(PyCriticalSection *c, PyObject *op);
61+
62+
PyAPI_FUNC(void)
63+
PyCriticalSection_End(PyCriticalSection *c);
64+
65+
PyAPI_FUNC(void)
66+
PyCriticalSection2_Begin(PyCriticalSection2 *c, PyObject *a, PyObject *b);
67+
68+
PyAPI_FUNC(void)
69+
PyCriticalSection2_End(PyCriticalSection2 *c);
70+
71+
// These are definitions for the stable ABI. For GIL-ful builds they're
72+
// conditionally redefined as no-ops in cpython/critical_section.h.
73+
74+
# define Py_BEGIN_CRITICAL_SECTION(op) \
75+
{ \
76+
PyCriticalSection _py_cs; \
77+
PyCriticalSection_Begin(&_py_cs, _PyObject_CAST(op))
78+
79+
# define Py_END_CRITICAL_SECTION() \
80+
PyCriticalSection_End(&_py_cs); \
81+
}
82+
83+
# define Py_BEGIN_CRITICAL_SECTION2(a, b) \
84+
{ \
85+
PyCriticalSection2 _py_cs2; \
86+
PyCriticalSection2_Begin(&_py_cs2, _PyObject_CAST(a), _PyObject_CAST(b))
87+
88+
# define Py_END_CRITICAL_SECTION2() \
89+
PyCriticalSection2_End(&_py_cs2); \
90+
}
91+
792
#ifndef Py_LIMITED_API
893
# define Py_CPYTHON_CRITICAL_SECTION_H
994
# include "cpython/critical_section.h"

‎Lib/test/test_cext/extension.c‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,10 @@ _testcext_exec(PyObject *module)
9999
obj = NULL;
100100
Py_CLEAR(obj);
101101

102+
// Test that Py_BEGIN_CRITICAL_SECTION is available
103+
Py_BEGIN_CRITICAL_SECTION(module);
104+
Py_END_CRITICAL_SECTION();
105+
102106
return 0;
103107
}
104108

‎Lib/test/test_stable_abi_ctypes.py‎

Lines changed: 4 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)