Python to C: What’s new in Cython 3.1

Cython is the most convenient way to bridge C and Python languages and tools in Python programs. Developers use it to write C extensions or wrap C libraries for use in Python. Now, a major new release is on the way. While it’s in early stages, Cython 3.1 adds powerful new type annotation features, and it includes compatibility with the free-threaded or “no-GIL” build of Python. It’s worth a spin, especially for developers who like to stay ahead of the curve.

Support for free-threaded Python

Python 3.13 introduced the first public, if experimental, “free-threaded” or “no-GIL” builds of the language, which we’ll call “3.13t.”

3.13t allows CPU-bound Python threads to run with true parallelism. As a project, it’s still in the early stages of development, so it’s only offered as an opt-in option. You have to specifically install the free-threaded build in order to use it.

For a long time, Cython, and C modules for Python generally, were a way to sidestep the GIL, the global interpreter lock. C code that doesn’t call into the CPython runtime isn’t constrained by the GIL. With 3.13t, conventional Python programs can now enjoy the same freedom.

Cython 3.1 adds basic support for 3.13t. This support is considered experimental, just as free-threading itself is experimental, so you shouldn’t rely on it for production use.

Cython modules need special build instructions to work with 3.31t. If a free-threaded Python tries to load a non-free-threaded-compatible Cython module, the interpreter will drop back to GIL mode for compatibility.

How to build free-threaded Cython modules

There are a few steps to building free-threaded Cython modules.

1. Install the pre-release version of Cython on 3.13t

As of this writing, Cython 3.1 is not yet available on PyPI, so you’ll have to install it from GitHub. To minimize problems you might have with building Cython itself on 3.13, you may want to install the uncompiled version of Cython.

First, set the environment variable NO_CYTHON_COMPILE to the string 'true'. Then install Cython from GitHub:

pip install git+https://github.com/cython/cython

2. Add a compiler directive to Cython modules

To mark a module as being compatible with free-threading, you will need to add this declaration at the top of the module:

# cython: freethreading_compatible = True

Note that this declaration doesn’t by itself cause the module to be compiled for free-threading. It’s just a way to inform the Python runtime that it can stay in free-threaded mode when the module is imported.

3. Add the free-threading build macros

The setup.py file used to build Cython extensions needs to set a C macro to allow your Cython modules to be built with free-threaded compatibility. The macro Py_GIL_DISABLED (note that’s Py, not PY!) should be set to 1.

One convenient way to do this automatically is to use the sys._is_gil_enabled() method to check if the macro needs to be used. A recipe like this might work:

macros = []
if getattr(sys, '_is_gil_enabled', None) and not sys._is_gil_enabled():
macros = [('Py_GIL_DISABLED','1')]

ext_modules = [
Extension(
'compute',
['compute.pyx'],
define_macros=macros
)
]

4. Ensure your code is thread safe

Thread safety in Cython modules has always been the programmer’s responsibility, at least as far as pure C code goes. If you call into the CPython interpreter using Cython, even in the free-threaded build, the interpreter will handle reference counts for those objects (although it won’t do anything about data races). But any Cython functions that run concurrently must not manipulate the same data.

New type annotations for constants, pointers, and volatile values

As Cython has evolved, it has gradually transitioned away from its custom Python-but-not-quite syntax. Modern Cython uses a syntax that is essentially regular Python with Cython-added type hints, context managers, and other constructions.

That new syntax lacked some of the precision and expressiveness of “classic” Cython, however, so it was necessary to drop back to the older Cython syntax to express certain things. Cython 3.1 adds new type hints to help close that gap:

cython.const[T] lets you declare something of a given type as a constant. For instance, x:cython.const[cython.int] = 32 would declare x as the constant integer 32. You can also abbreviate many common constant declarations; e.g., cython.const[cython.int] can be simplified to cython.const_int. Note that these declarations are only valid at the module level; they cannot be made within a function.

cython.volatile[T] creates a declaration that is handled by the compiler in the same manner as something that uses the volatile C keyword. volatile is used to indicate that the compiler should not optimize the code in such a way that the value in question can ever be assumed to be any particular thing at any particular time. This is useful for, say, C modules that interface directly with hardware.

cython.pointer[T] lets you define a pointer type. For instance, x: cython.pointer[cython.int] = cython.address(y) would make x into a pointer to y. Dereferencing a pointer in Cython is still done with the old-school C-style syntax, though: z=x[0] would dereference x into z (and thus set z to the value of y).