Using PyO3 Interop

Overview

A PyO3 extension is a Rust crate that exports symbols Python imports as if they were C extensions. The interesting work is not the bindings — it is the boundary itself: GIL discipline, lifetime contracts between Rust and the Python heap, the cost of crossing, and what happens when one side's invariants leak into the other's runtime.

This pack treats the Python ↔ Rust FFI boundary as a discipline distinct from single-crate Rust engineering and from pure-Python performance work. PyO3 is the connective tissue — but the failure modes are not connective-tissue failures. They are:

• GIL contention or deadlock — Rust holds the GIL through a long computation; Python threads starve; a tokio task tries to acquire the GIL while another path holds it; the process locks.
• Lifetime contract violation — a Bound<'py, T> outlives its Python<'py> token; a NumPy view aliases a Rust buffer that is freed; a #[pyclass] is reborrowed across the GIL boundary.
• Boundary cost amortisation failure — the API is per-element when it should be batched; the workload crosses the FFI 10⁶ times per episode and a 100 ns crossing dominates a 50 ns kernel.
• ABI / packaging mismatch — the wheel built against Python 3.11 fails to import on 3.12 because the build was native and not abi3; manylinux glibc symbols leak into the wheel.
• Interpreter teardown disorder — a Rust-owned tokio::Runtime is dropped after the interpreter is gone; the atexit order is wrong; the process segfaults on exit.

These are not "Rust bugs" or "Python bugs". They are boundary bugs. They are this pack's subject.

When to Use

Use this pack when: