pyo3-0.20.2/.cargo_vcs_info.json0000644000000001360000000000100120070ustar { "git": { "sha1": "bcef18b988a519aca93c29039fa0894a310d2eaf" }, "path_in_vcs": "" }pyo3-0.20.2/.netlify/build.sh000075500000000000000000000057021046102023000137710ustar 00000000000000#!/usr/bin/env bash set -uex rustup default nightly PYO3_VERSION=$(cargo search pyo3 --limit 1 | head -1 | tr -s ' ' | cut -d ' ' -f 3 | tr -d '"') ## Start from the existing gh-pages content. ## By serving it over netlify, we can have better UX for users because ## netlify can then redirect e.g. /v0.17.0 to /v0.17.0/ ## which leads to better loading of CSS assets. wget -qc https://github.com/PyO3/pyo3/archive/gh-pages.tar.gz -O - | tar -xz mv pyo3-gh-pages netlify_build ## Configure netlify _redirects file # Add redirect for each documented version set +x # these loops get very spammy and fill the deploy log for d in netlify_build/v*; do version="${d/netlify_build\/v/}" echo "/v$version/doc/* https://docs.rs/pyo3/$version/:splat" >> netlify_build/_redirects if [ $version != $PYO3_VERSION ]; then # for old versions, mark the files in the latest version as the canonical URL for file in $(find $d -type f); do file_path="${file/$d\//}" # remove index.html and/or .html suffix to match the page URL on the # final netlfiy site url_path="$file_path" if [[ $file_path == index.html ]]; then url_path="" elif [[ $file_path == *.html ]]; then url_path="${file_path%.html}" fi echo "/v$version/$url_path" >> netlify_build/_headers if test -f "netlify_build/v$PYO3_VERSION/$file_path"; then echo " Link: ; rel=\"canonical\"" >> netlify_build/_headers else # this file doesn't exist in the latest guide, don't index it echo " X-Robots-Tag: noindex" >> netlify_build/_headers fi done fi done # Add latest redirect echo "/latest/* /v${PYO3_VERSION}/:splat 302" >> netlify_build/_redirects ## Add landing page redirect if [ "${CONTEXT}" == "deploy-preview" ]; then echo "/ /main/" >> netlify_build/_redirects else echo "/ /v${PYO3_VERSION}/ 302" >> netlify_build/_redirects fi set -x ## Generate towncrier release notes pip install towncrier towncrier build --yes --version Unreleased --date TBC ## Build guide # Install latest mdbook. Netlify will cache the cargo bin dir, so this will # only build mdbook if needed. MDBOOK_VERSION=$(cargo search mdbook --limit 1 | head -1 | tr -s ' ' | cut -d ' ' -f 3 | tr -d '"') INSTALLED_MDBOOK_VERSION=$(mdbook --version || echo "none") if [ "${INSTALLED_MDBOOK_VERSION}" != "mdbook v${MDBOOK_VERSION}" ]; then cargo install mdbook@${MDBOOK_VERSION} --force fi pip install nox nox -s build-guide mv target/guide netlify_build/main/ ## Build public docs nox -s docs mv target/doc netlify_build/main/doc/ echo "" > netlify_build/main/doc/index.html ## Build internal docs nox -s docs -- nightly internal mkdir -p netlify_build/internal mv target/doc netlify_build/internal/ ls -l netlify_build/ pyo3-0.20.2/.netlify/internal_banner.html000064400000000000000000000026211046102023000163570ustar 00000000000000
⚠️ Internal Docs ⚠️ Not Public API 👉 Official Docs Here
pyo3-0.20.2/.python-version000064400000000000000000000000051046102023000135770ustar 000000000000003.12 pyo3-0.20.2/.towncrier.template.md000064400000000000000000000010471046102023000150270ustar 00000000000000{% for section_text, section in sections.items() %}{%- if section %}{{section_text}}{% endif -%} {% if section %} {% for category in ['packaging', 'added', 'changed', 'removed', 'fixed' ] if category in section %} ### {{ definitions[category]['name'] }} {% if definitions[category]['showcontent'] %} {% for text, pull_requests in section[category].items() %} - {{ text }} {{ pull_requests|join(', ') }} {% endfor %} {% else %} - {{ section[category]['']|join(', ') }} {% endif %} {% endfor %}{% else %}No significant changes.{% endif %}{% endfor %} pyo3-0.20.2/Architecture.md000064400000000000000000000246771046102023000135630ustar 00000000000000 # PyO3: Architecture This document roughly describes the high-level architecture of PyO3. If you want to become familiar with the codebase you are in the right place! ## Overview PyO3 provides a bridge between Rust and Python, based on the [Python/C API]. Thus, PyO3 has low-level bindings of these API as its core. On top of that, we have higher-level bindings to operate Python objects safely. Also, to define Python classes and functions in Rust code, we have `trait PyClass` and a set of protocol traits (e.g., `PyIterProtocol`) for supporting object protocols (i.e., `__dunder__` methods). Since implementing `PyClass` requires lots of boilerplate, we have a proc-macro `#[pyclass]`. To summarize, there are six main parts to the PyO3 codebase. 1. [Low-level bindings of Python/C API.](#1-low-level-bindings-of-python-capi) - [`pyo3-ffi`] and [`src/ffi`] 2. [Bindings to Python objects.](#2-bindings-to-python-objects) - [`src/instance.rs`] and [`src/types`] 3. [`PyClass` and related functionalities.](#3-pyclass-and-related-functionalities) - [`src/pycell.rs`], [`src/pyclass.rs`], and more 4. [Protocol methods like `__getitem__`.](#4-protocol-methods) - [`src/class`] 5. [Procedural macros to simplify usage for users.](#5-procedural-macros-to-simplify-usage-for-users) - [`src/derive_utils.rs`], [`pyo3-macros`] and [`pyo3-macros-backend`] 6. [`build.rs` and `pyo3-build-config`](#6-buildrs-and-pyo3-build-config) - [`build.rs`](https://github.com/PyO3/pyo3/tree/main/build.rs) - [`pyo3-build-config`] ## 1. Low-level bindings of Python/C API [`pyo3-ffi`] contains wrappers of [Python/C API]. We aim to provide straight-forward Rust wrappers resembling the file structure of [`cpython/Include`](https://github.com/python/cpython/tree/v3.9.2/Include). However, we still lack some APIs and are continuously updating the module to match the file contents upstream in CPython. The tracking issue is [#1289](https://github.com/PyO3/pyo3/issues/1289), and contribution is welcome. In the [`pyo3-ffi`] crate, there is lots of conditional compilation such as `#[cfg(Py_LIMITED_API)]`, `#[cfg(Py_37)]`, and `#[cfg(PyPy)]`. `Py_LIMITED_API` corresponds to `#define Py_LIMITED_API` macro in Python/C API. With `Py_LIMITED_API`, we can build a Python-version-agnostic binary called an [abi3 wheel](https://pyo3.rs/latest/building_and_distribution.html#py_limited_apiabi3). `Py_37` means that the API is available from Python >= 3.7. There are also `Py_38`, `Py_39`, and so on. `PyPy` means that the API definition is for PyPy. Those flags are set in [`build.rs`](#6-buildrs-and-pyo3-build-config). ## 2. Bindings to Python objects [`src/types`] contains bindings to [built-in types](https://docs.python.org/3/library/stdtypes.html) of Python, such as `dict` and `list`. For historical reasons, Python's `object` is called `PyAny` in PyO3 and located in [`src/types/any.rs`]. Currently, `PyAny` is a straightforward wrapper of `ffi::PyObject`, defined as: ```rust #[repr(transparent)] pub struct PyAny(UnsafeCell); ``` All built-in types are defined as a C struct. For example, `dict` is defined as: ```c typedef struct { /* Base object */ PyObject ob_base; /* Number of items in the dictionary */ Py_ssize_t ma_used; /* Dictionary version */ uint64_t ma_version_tag; PyDictKeysObject *ma_keys; PyObject **ma_values; } PyDictObject; ``` However, we cannot access such a specific data structure with `#[cfg(Py_LIMITED_API)]` set. Thus, all builtin objects are implemented as opaque types by wrapping `PyAny`, e.g.,: ```rust #[repr(transparent)] pub struct PyDict(PyAny); ``` Note that `PyAny` is not a pointer, and it is usually used as a pointer to the object in the Python heap, as `&PyAny`. This design choice can be changed (see the discussion in [#1056](https://github.com/PyO3/pyo3/issues/1056)). Since we need lots of boilerplate for implementing common traits for these types (e.g., `AsPyPointer`, `AsRef`, and `Debug`), we have some macros in [`src/types/mod.rs`]. ## 3. `PyClass` and related functionalities [`src/pycell.rs`], [`src/pyclass.rs`], and [`src/type_object.rs`] contain types and traits to make `#[pyclass]` work. Also, [`src/pyclass_init.rs`] and [`src/impl_/pyclass.rs`] have related functionalities. To realize object-oriented programming in C, all Python objects must have the following two fields at the beginning. ```rust #[repr(C)] pub struct PyObject { pub ob_refcnt: usize, pub ob_type: *mut PyTypeObject, ... } ``` Thanks to this guarantee, casting `*mut A` to `*mut PyObject` is valid if `A` is a Python object. To ensure this guarantee, we have a wrapper struct `PyCell` in [`src/pycell.rs`] which is roughly: ```rust #[repr(C)] pub struct PyCell { object: crate::ffi::PyObject, inner: T, } ``` Thus, when copying a Rust struct to a Python object, we first allocate `PyCell` on the Python heap and then move `T` into it. Also, `PyCell` provides [RefCell](https://doc.rust-lang.org/std/cell/struct.RefCell.html)-like methods to ensure Rust's borrow rules. See [the documentation](https://docs.rs/pyo3/latest/pyo3/pycell/struct.PyCell.html) for more. `PyCell` requires that `T` implements `PyClass`. This trait is somewhat complex and derives many traits, but the most important one is `PyTypeInfo` in [`src/type_object.rs`]. `PyTypeInfo` is also implemented for built-in types. In Python, all objects have their types, and types are also objects of `type`. For example, you can see `type({})` shows `dict` and `type(type({}))` shows `type` in Python REPL. `T: PyTypeInfo` implies that `T` has a corresponding type object. ## 4. Protocol methods Python has some built-in special methods called dunder methods, such as `__iter__`. They are called "slots" in the [abstract objects layer](https://docs.python.org/3/c-api/abstract.html) in Python/C API. We provide a way to implement those protocols similarly, by recognizing special names in `#[pymethods]`, with a few new ones for slots that can not be implemented in Python, such as GC support. ## 5. Procedural macros to simplify usage for users. [`pyo3-macros`] provides five proc-macro APIs: `pymodule`, `pyfunction`, `pyclass`, `pymethods`, and `#[derive(FromPyObject)]`. [`pyo3-macros-backend`] has the actual implementations of these APIs. [`src/derive_utils.rs`] contains some utilities used in code generated by these proc-macros, such as parsing function arguments. ## 6. `build.rs` and `pyo3-build-config` PyO3 supports a wide range of OSes, interpreters and use cases. The correct environment must be detected at build time in order to set up relevant conditional compilation correctly. This logic is captured in the [`pyo3-build-config`] crate, which is a `build-dependency` of `pyo3` and `pyo3-macros`, and can also be used by downstream users in the same way. In [`pyo3-build-config`]'s `build.rs` the build environment is detected and inlined into the crate as a "config file". This works in all cases except for cross-compiling, where it is necessary to capture this from the `pyo3` `build.rs` to get some extra environment variables that Cargo doesn't set for build dependencies. The `pyo3` `build.rs` also runs some safety checks such as ensuring the Python version detected is actually supported. Some of the functionality of `pyo3-build-config`: - Find the interpreter for build and detect the Python version. - We have to set some version flags like `#[cfg(Py_3_7)]`. - If the interpreter is PyPy, we set `#[cfg(PyPy)`. - If the `PYO3_CONFIG_FILE` environment variable is set then that file's contents will be used instead of any detected configuration. - If the `PYO3_NO_PYTHON` environment variable is set then the interpreter detection is bypassed entirely and only abi3 extensions can be built. - Check if we are building a Python extension. - If we are building an extension (e.g., Python library installable by `pip`), we don't link `libpython`. Currently we use the `extension-module` feature for this purpose. This may change in the future. See [#1123](https://github.com/PyO3/pyo3/pull/1123). - Cross-compiling configuration - If `TARGET` architecture and `HOST` architecture differ, we can find cross compile information from environment variables (`PYO3_CROSS_LIB_DIR`, `PYO3_CROSS_PYTHON_VERSION` and `PYO3_CROSS_PYTHON_IMPLEMENTATION`) or system files. When cross compiling extension modules it is often possible to make it work without any additional user input. - When an experimental feature `generate-import-lib` is enabled, the `pyo3-ffi` build script can generate `python3.dll` import libraries for Windows targets automatically via an external [`python3-dll-a`] crate. This enables the users to cross compile Python extensions for Windows without having to install any Windows Python libraries. [python/c api]: https://docs.python.org/3/c-api/ [`python3-dll-a`]: https://docs.rs/python3-dll-a/latest/python3_dll_a/ [`pyo3-macros`]: https://github.com/PyO3/pyo3/tree/main/pyo3-macros [`pyo3-macros-backend`]: https://github.com/PyO3/pyo3/tree/main/pyo3-macros-backend [`pyo3-build-config`]: https://github.com/PyO3/pyo3/tree/main/pyo3-build-config [`pyo3-ffi`]: https://github.com/PyO3/pyo3/tree/main/pyo3-ffi [`src/class`]: https://github.com/PyO3/pyo3/tree/main/src/class [`src/ffi`]: https://github.com/PyO3/pyo3/tree/main/src/ffi [`src/types`]: https://github.com/PyO3/pyo3/tree/main/src/types [`src/derive_utils.rs`]: https://github.com/PyO3/pyo3/blob/main/src/derive_utils.rs [`src/instance.rs`]: https://github.com/PyO3/pyo3/tree/main/src/instance.rs [`src/pycell.rs`]: https://github.com/PyO3/pyo3/tree/main/src/pycell.rs [`src/pyclass.rs`]: https://github.com/PyO3/pyo3/tree/main/src/pyclass.rs [`src/pyclass_init.rs`]: https://github.com/PyO3/pyo3/tree/main/src/pyclass_init.rs [`src/pyclass_slot.rs`]: https://github.com/PyO3/pyo3/tree/main/src/pyclass_slot.rs [`src/type_object.rs`]: https://github.com/PyO3/pyo3/tree/main/src/type_object.rs [`src/class/methods.rs`]: https://github.com/PyO3/pyo3/tree/main/src/class/methods.rs [`src/class/impl_.rs`]: https://github.com/PyO3/pyo3/tree/main/src/class/impl_.rs [`src/types/any.rs`]: https://github.com/PyO3/pyo3/tree/main/src/types/any.rs [`src/types/mod.rs`]: https://github.com/PyO3/pyo3/tree/main/src/types/mod.rs pyo3-0.20.2/CHANGELOG.md000064400000000000000000003571121046102023000124210ustar 00000000000000# Changelog All notable changes to this project will be documented in this file. For help with updating to new PyO3 versions, please see the [migration guide](https://pyo3.rs/latest/migration.html). The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). To see unreleased changes, please see the [CHANGELOG on the main branch guide](https://pyo3.rs/main/changelog.html). ## [0.20.2] - 2024-01-04 ### Packaging - Pin `pyo3` and `pyo3-ffi` dependencies on `pyo3-build-config` to require the same patch version, i.e. `pyo3` 0.20.2 requires _exactly_ `pyo3-build-config` 0.20.2. [#3721](https://github.com/PyO3/pyo3/pull/3721) ### Fixed - Fix compile failure when building `pyo3` 0.20.0 with latest `pyo3-build-config` 0.20.X. [#3724](https://github.com/PyO3/pyo3/pull/3724) - Fix docs.rs build. [#3722](https://github.com/PyO3/pyo3/pull/3722) ## [0.20.1] - 2023-12-30 ### Added - Add optional `either` feature to add conversions for `either::Either` sum type. [#3456](https://github.com/PyO3/pyo3/pull/3456) - Add optional `smallvec` feature to add conversions for `smallvec::SmallVec`. [#3507](https://github.com/PyO3/pyo3/pull/3507) - Add `take` and `into_inner` methods to `GILOnceCell` [#3556](https://github.com/PyO3/pyo3/pull/3556) - `#[classmethod]` methods can now also receive `Py` as their first argument. [#3587](https://github.com/PyO3/pyo3/pull/3587) - `#[pyfunction(pass_module)]` can now also receive `Py` as their first argument. [#3587](https://github.com/PyO3/pyo3/pull/3587) - Add `traverse` method to `GILProtected`. [#3616](https://github.com/PyO3/pyo3/pull/3616) - Added `abi3-py312` feature [#3687](https://github.com/PyO3/pyo3/pull/3687) ### Fixed - Fix minimum version specification for optional `chrono` dependency. [#3512](https://github.com/PyO3/pyo3/pull/3512) - Silenced new `clippy::unnecessary_fallible_conversions` warning when using a `Py` `self` receiver. [#3564](https://github.com/PyO3/pyo3/pull/3564) ## [0.20.0] - 2023-10-11 ### Packaging - Dual-license PyO3 under either the Apache 2.0 OR the MIT license. This makes the project GPLv2 compatible. [#3108](https://github.com/PyO3/pyo3/pull/3108) - Update MSRV to Rust 1.56. [#3208](https://github.com/PyO3/pyo3/pull/3208) - Bump `indoc` dependency to 2.0 and `unindent` dependency to 0.2. [#3237](https://github.com/PyO3/pyo3/pull/3237) - Bump `syn` dependency to 2.0. [#3239](https://github.com/PyO3/pyo3/pull/3239) - Drop support for debug builds of Python 3.7. [#3387](https://github.com/PyO3/pyo3/pull/3387) - Bump `chrono` optional dependency to require 0.4.25 or newer. [#3427](https://github.com/PyO3/pyo3/pull/3427) - Support Python 3.12. [#3488](https://github.com/PyO3/pyo3/pull/3488) ### Added - Support `__lt__`, `__le__`, `__eq__`, `__ne__`, `__gt__` and `__ge__` in `#[pymethods]`. [#3203](https://github.com/PyO3/pyo3/pull/3203) - Add FFI definition `Py_GETENV`. [#3336](https://github.com/PyO3/pyo3/pull/3336) - Add `as_ptr` and `into_ptr` inherent methods for `Py`, `PyAny`, `PyRef`, and `PyRefMut`. [#3359](https://github.com/PyO3/pyo3/pull/3359) - Implement `DoubleEndedIterator` for `PyTupleIterator` and `PyListIterator`. [#3366](https://github.com/PyO3/pyo3/pull/3366) - Add `#[pyclass(rename_all = "...")]` option: this allows renaming all getters and setters of a struct, or all variants of an enum. Available renaming rules are: `"camelCase"`, `"kebab-case"`, `"lowercase"`, `"PascalCase"`, `"SCREAMING-KEBAB-CASE"`, `"SCREAMING_SNAKE_CASE"`, `"snake_case"`, `"UPPERCASE"`. [#3384](https://github.com/PyO3/pyo3/pull/3384) - Add FFI definitions `PyObject_GC_IsTracked` and `PyObject_GC_IsFinalized` on Python 3.9 and up (PyPy 3.10 and up). [#3403](https://github.com/PyO3/pyo3/pull/3403) - Add types for `None`, `Ellipsis`, and `NotImplemented`. [#3408](https://github.com/PyO3/pyo3/pull/3408) - Add FFI definitions for the `Py_mod_multiple_interpreters` constant and its possible values. [#3494](https://github.com/PyO3/pyo3/pull/3494) - Add FFI definitions for `PyInterpreterConfig` struct, its constants and `Py_NewInterpreterFromConfig`. [#3502](https://github.com/PyO3/pyo3/pull/3502) ### Changed - Change `PySet::discard` to return `PyResult` (previously returned nothing). [#3281](https://github.com/PyO3/pyo3/pull/3281) - Optimize implmentation of `IntoPy` for Rust tuples to Python tuples. [#3321](https://github.com/PyO3/pyo3/pull/3321) - Change `PyDict::get_item` to no longer suppress arbitrary exceptions (the return type is now `PyResult>` instead of `Option<&PyAny>`), and deprecate `PyDict::get_item_with_error`. [#3330](https://github.com/PyO3/pyo3/pull/3330) - Deprecate FFI definitions which are deprecated in Python 3.12. [#3336](https://github.com/PyO3/pyo3/pull/3336) - `AsPyPointer` is now an `unsafe trait`. [#3358](https://github.com/PyO3/pyo3/pull/3358) - Accept all `os.PathLike` values in implementation of `FromPyObject` for `PathBuf`. [#3374](https://github.com/PyO3/pyo3/pull/3374) - Add `__builtins__` to globals in `py.run()` and `py.eval()` if they're missing. [#3378](https://github.com/PyO3/pyo3/pull/3378) - Optimize implementation of `FromPyObject` for `BigInt` and `BigUint`. [#3379](https://github.com/PyO3/pyo3/pull/3379) - `PyIterator::from_object` and `PyByteArray::from` now take a single argument of type `&PyAny` (previously took two arguments `Python` and `AsPyPointer`). [#3389](https://github.com/PyO3/pyo3/pull/3389) - Replace `AsPyPointer` with `AsRef` as a bound in the blanket implementation of `From<&T> for PyObject`. [#3391](https://github.com/PyO3/pyo3/pull/3391) - Replace blanket `impl IntoPy for &T where T: AsPyPointer` with implementations of `impl IntoPy` for `&PyAny`, `&T where T: AsRef`, and `&Py`. [#3393](https://github.com/PyO3/pyo3/pull/3393) - Preserve `std::io::Error` kind in implementation of `From` for `PyErr` [#3396](https://github.com/PyO3/pyo3/pull/3396) - Try to select a relevant `ErrorKind` in implementation of `From` for `OSError` subclass. [#3397](https://github.com/PyO3/pyo3/pull/3397) - Retrieve the original `PyErr` in implementation of `From` for `PyErr` if the `std::io::Error` has been built using a Python exception (previously would create a new exception wrapping the `std::io::Error`). [#3402](https://github.com/PyO3/pyo3/pull/3402) - `#[pymodule]` will now return the same module object on repeated import by the same Python interpreter, on Python 3.9 and up. [#3446](https://github.com/PyO3/pyo3/pull/3446) - Truncate leap-seconds and warn when converting `chrono` types to Python `datetime` types (`datetime` cannot represent leap-seconds). [#3458](https://github.com/PyO3/pyo3/pull/3458) - `Err` returned from `#[pyfunction]` will now have a non-None `__context__` if called from inside a `catch` block. [#3455](https://github.com/PyO3/pyo3/pull/3455) - Deprecate undocumented `#[__new__]` form of `#[new]` attribute. [#3505](https://github.com/PyO3/pyo3/pull/3505) ### Removed - Remove all functionality deprecated in PyO3 0.18, including `#[args]` attribute for `#[pymethods]`. [#3232](https://github.com/PyO3/pyo3/pull/3232) - Remove `IntoPyPointer` trait in favour of `into_ptr` inherent methods. [#3385](https://github.com/PyO3/pyo3/pull/3385) ### Fixed - Handle exceptions properly in `PySet::discard`. [#3281](https://github.com/PyO3/pyo3/pull/3281) - The `PyTupleIterator` type returned by `PyTuple::iter` is now public and hence can be named by downstream crates. [#3366](https://github.com/PyO3/pyo3/pull/3366) - Linking of `PyOS_FSPath` on PyPy. [#3374](https://github.com/PyO3/pyo3/pull/3374) - Fix memory leak in `PyTypeBuilder::build`. [#3401](https://github.com/PyO3/pyo3/pull/3401) - Disable removed FFI definitions `_Py_GetAllocatedBlocks`, `_PyObject_GC_Malloc`, and `_PyObject_GC_Calloc` on Python 3.11 and up. [#3403](https://github.com/PyO3/pyo3/pull/3403) - Fix `ResourceWarning` and crashes related to GC when running with debug builds of CPython. [#3404](https://github.com/PyO3/pyo3/pull/3404) - Some-wrapping of `Option` default arguments will no longer re-wrap `Some(T)` or expressions evaluating to `None`. [#3461](https://github.com/PyO3/pyo3/pull/3461) - Fix `IterNextOutput::Return` not returning a value on PyPy. [#3471](https://github.com/PyO3/pyo3/pull/3471) - Emit compile errors instead of ignoring macro invocations inside `#[pymethods]` blocks. [#3491](https://github.com/PyO3/pyo3/pull/3491) - Emit error on invalid arguments to `#[new]`, `#[classmethod]`, `#[staticmethod]`, and `#[classattr]`. [#3484](https://github.com/PyO3/pyo3/pull/3484) - Disable `PyMarshal_WriteObjectToString` from `PyMarshal_ReadObjectFromString` with the `abi3` feature. [#3490](https://github.com/PyO3/pyo3/pull/3490) - Fix FFI definitions for `_PyFrameEvalFunction` on Python 3.11 and up (it now receives a `_PyInterpreterFrame` opaque struct). [#3500](https://github.com/PyO3/pyo3/pull/3500) ## [0.19.2] - 2023-08-01 ### Added - Add FFI definitions `PyState_AddModule`, `PyState_RemoveModule` and `PyState_FindModule` for PyPy 3.9 and up. [#3295](https://github.com/PyO3/pyo3/pull/3295) - Add FFI definitions `_PyObject_CallFunction_SizeT` and `_PyObject_CallMethod_SizeT`. [#3297](https://github.com/PyO3/pyo3/pull/3297) - Add a "performance" section to the guide collecting performance-related tricks and problems. [#3304](https://github.com/PyO3/pyo3/pull/3304) - Add `PyErr::Display` for all Python versions, and FFI symbol `PyErr_DisplayException` for Python 3.12. [#3334](https://github.com/PyO3/pyo3/pull/3334) - Add FFI definition `PyType_GetDict()` for Python 3.12. [#3339](https://github.com/PyO3/pyo3/pull/3339) - Add `PyAny::downcast_exact`. [#3346](https://github.com/PyO3/pyo3/pull/3346) - Add `PySlice::full()` to construct a full slice (`::`). [#3353](https://github.com/PyO3/pyo3/pull/3353) ### Changed - Update `PyErr` for 3.12 betas to avoid deprecated ffi methods. [#3306](https://github.com/PyO3/pyo3/pull/3306) - Update FFI definitions of `object.h` for Python 3.12.0b4. [#3335](https://github.com/PyO3/pyo3/pull/3335) - Update `pyo3::ffi` struct definitions to be compatible with 3.12.0b4. [#3342](https://github.com/PyO3/pyo3/pull/3342) - Optimize conversion of `float` to `f64` (and `PyFloat::value`) on non-abi3 builds. [#3345](https://github.com/PyO3/pyo3/pull/3345) ### Fixed - Fix timezone conversion bug for FixedOffset datetimes that were being incorrectly converted to and from UTC. [#3269](https://github.com/PyO3/pyo3/pull/3269) - Fix `SystemError` raised in `PyUnicodeDecodeError_Create` on PyPy 3.10. [#3297](https://github.com/PyO3/pyo3/pull/3297) - Correct FFI definition `Py_EnterRecursiveCall` to return `c_int` (was incorrectly returning `()`). [#3300](https://github.com/PyO3/pyo3/pull/3300) - Fix case where `PyErr::matches` and `PyErr::is_instance` returned results inconsistent with `PyErr::get_type`. [#3313](https://github.com/PyO3/pyo3/pull/3313) - Fix loss of panic message in `PanicException` when unwinding after the exception was "normalized". [#3326](https://github.com/PyO3/pyo3/pull/3326) - Fix `PyErr::from_value` and `PyErr::into_value` losing traceback on conversion. [#3328](https://github.com/PyO3/pyo3/pull/3328) - Fix reference counting of immortal objects on Python 3.12.0b4. [#3335](https://github.com/PyO3/pyo3/pull/3335) ## [0.19.1] - 2023-07-03 ### Packaging - Extend range of supported versions of `hashbrown` optional dependency to include version 0.14 [#3258](https://github.com/PyO3/pyo3/pull/3258) - Extend range of supported versions of `indexmap` optional dependency to include version 2. [#3277](https://github.com/PyO3/pyo3/pull/3277) - Support PyPy 3.10. [#3289](https://github.com/PyO3/pyo3/pull/3289) ### Added - Add `pyo3::types::PyFrozenSetBuilder` to allow building a `PyFrozenSet` item by item. [#3156](https://github.com/PyO3/pyo3/pull/3156) - Add support for converting to and from Python's `ipaddress.IPv4Address`/`ipaddress.IPv6Address` and `std::net::IpAddr`. [#3197](https://github.com/PyO3/pyo3/pull/3197) - Add support for `num-bigint` feature in combination with `abi3`. [#3198](https://github.com/PyO3/pyo3/pull/3198) - Add `PyErr_GetRaisedException()`, `PyErr_SetRaisedException()` to FFI definitions for Python 3.12 and later. [#3248](https://github.com/PyO3/pyo3/pull/3248) - Add `Python::with_pool` which is a safer but more limited alternative to `Python::new_pool`. [#3263](https://github.com/PyO3/pyo3/pull/3263) - Add `PyDict::get_item_with_error` on PyPy. [#3270](https://github.com/PyO3/pyo3/pull/3270) - Allow `#[new]` methods may to return `Py` in order to return existing instances. [#3287](https://github.com/PyO3/pyo3/pull/3287) ### Fixed - Fix conversion of classes implementing `__complex__` to `Complex` when using `abi3` or PyPy. [#3185](https://github.com/PyO3/pyo3/pull/3185) - Stop suppressing unrelated exceptions in `PyAny::hasattr`. [#3271](https://github.com/PyO3/pyo3/pull/3271) - Fix memory leak when creating `PySet` or `PyFrozenSet` or returning types converted into these internally, e.g. `HashSet` or `BTreeSet`. [#3286](https://github.com/PyO3/pyo3/pull/3286) ## [0.19.0] - 2023-05-31 ### Packaging - Correct dependency on syn to version 1.0.85 instead of the incorrect version 1.0.56. [#3152](https://github.com/PyO3/pyo3/pull/3152) ### Added - Accept `text_signature` option (and automatically generate signature) for `#[new]` in `#[pymethods]`. [#2980](https://github.com/PyO3/pyo3/pull/2980) - Add support for converting to and from Python's `decimal.Decimal` and `rust_decimal::Decimal`. [#3016](https://github.com/PyO3/pyo3/pull/3016) - Add `#[pyo3(from_item_all)]` when deriving `FromPyObject` to specify `get_item` as getter for all fields. [#3120](https://github.com/PyO3/pyo3/pull/3120) - Add `pyo3::exceptions::PyBaseExceptionGroup` for Python 3.11, and corresponding FFI definition `PyExc_BaseExceptionGroup`. [#3141](https://github.com/PyO3/pyo3/pull/3141) - Accept `#[new]` with `#[classmethod]` to create a constructor which receives a (subtype's) class/`PyType` as its first argument. [#3157](https://github.com/PyO3/pyo3/pull/3157) - Add `PyClass::get` and `Py::get` for GIL-indepedent access to classes with `#[pyclass(frozen)]`. [#3158](https://github.com/PyO3/pyo3/pull/3158) - Add `PyAny::is_exact_instance` and `PyAny::is_exact_instance_of`. [#3161](https://github.com/PyO3/pyo3/pull/3161) ### Changed - `PyAny::is_instance_of::(obj)` is now equivalent to `T::is_type_of(obj)`, and now returns `bool` instead of `PyResult`. [#2881](https://github.com/PyO3/pyo3/pull/2881) - Deprecate `text_signature` option on `#[pyclass]` structs. [#2980](https://github.com/PyO3/pyo3/pull/2980) - No longer wrap `anyhow::Error`/`eyre::Report` containing a basic `PyErr` without a chain in a `PyRuntimeError`. [#3004](https://github.com/PyO3/pyo3/pull/3004) - - Change `#[getter]` and `#[setter]` to use a common call "trampoline" to slightly reduce generated code size and compile times. [#3029](https://github.com/PyO3/pyo3/pull/3029) - Improve default values for str, numbers and bool in automatically-generated `text_signature`. [#3050](https://github.com/PyO3/pyo3/pull/3050) - Improve default value for `None` in automatically-generated `text_signature`. [#3066](https://github.com/PyO3/pyo3/pull/3066) - Rename `PySequence::list` and `PySequence::tuple` to `PySequence::to_list` and `PySequence::to_tuple`. (The old names continue to exist as deprecated forms.) [#3111](https://github.com/PyO3/pyo3/pull/3111) - Extend the lifetime of the GIL token returned by `PyRef::py` and `PyRefMut::py` to match the underlying borrow. [#3131](https://github.com/PyO3/pyo3/pull/3131) - Safe access to the GIL, for example via `Python::with_gil`, is now locked inside of implementations of the `__traverse__` slot. [#3168](https://github.com/PyO3/pyo3/pull/3168) ### Removed - Remove all functionality deprecated in PyO3 0.17, most prominently `Python::acquire_gil` is replaced by `Python::with_gil`. [#2981](https://github.com/PyO3/pyo3/pull/2981) ### Fixed - Correct FFI definitions `PyGetSetDef`, `PyMemberDef`, `PyStructSequence_Field` and `PyStructSequence_Desc` to have `*const c_char` members for `name` and `doc` (not `*mut c_char`). [#3036](https://github.com/PyO3/pyo3/pull/3036) - Fix panic on `fmt::Display`, instead return `""` string and report error via `sys.unraisablehook()` [#3062](https://github.com/PyO3/pyo3/pull/3062) - Fix a compile error of "temporary value dropped while borrowed" when `#[pyfunction]`s take references into `#[pyclass]`es [#3142](https://github.com/PyO3/pyo3/pull/3142) - Fix crashes caused by PyO3 applying deferred reference count updates when entering a `__traverse__` implementation. [#3168](https://github.com/PyO3/pyo3/pull/3168) - Forbid running the `Drop` implementations of unsendable classes on other threads. [#3176](https://github.com/PyO3/pyo3/pull/3176) - Fix a compile error when `#[pymethods]` items come from somewhere else (for example, as a macro argument) and a custom receiver like `Py` is used. [#3178](https://github.com/PyO3/pyo3/pull/3178) ## [0.18.3] - 2023-04-13 ### Added - Add `GILProtected` to mediate concurrent access to a value using Python's global interpreter lock (GIL). [#2975](https://github.com/PyO3/pyo3/pull/2975) - Support `PyASCIIObject` / `PyUnicode` and associated methods on big-endian architectures. [#3015](https://github.com/PyO3/pyo3/pull/3015) - Add FFI definition `_PyDict_Contains_KnownHash()` for CPython 3.10 and up. [#3088](https://github.com/PyO3/pyo3/pull/3088) ### Fixed - Fix compile error for `#[pymethods]` and `#[pyfunction]` called "output". [#3022](https://github.com/PyO3/pyo3/pull/3022) - Fix compile error in generated code for magic methods implemented as a `#[staticmethod]`. [#3055](https://github.com/PyO3/pyo3/pull/3055) - Fix `is_instance` for `PyDateTime` (would incorrectly check for a `PyDate`). [#3071](https://github.com/PyO3/pyo3/pull/3071) - Fix upstream deprecation of `PyUnicode_InternImmortal` since Python 3.10. [#3071](https://github.com/PyO3/pyo3/pull/3087) ## [0.18.2] - 2023-03-24 ### Packaging - Disable default features of `chrono` to avoid depending on `time` v0.1.x. [#2939](https://github.com/PyO3/pyo3/pull/2939) ### Added - Implement `IntoPy`, `ToPyObject` and `FromPyObject` for `Cow<[u8]>` to efficiently handle both `bytes` and `bytearray` objects. [#2899](https://github.com/PyO3/pyo3/pull/2899) - Implement `IntoPy`, `ToPyObject` and `FromPyObject` for `Cell`. [#3014](https://github.com/PyO3/pyo3/pull/3014) - Add `PyList::to_tuple()`, as a convenient and efficient conversion from lists to tuples. [#3042](https://github.com/PyO3/pyo3/pull/3042) - Add `PyTuple::to_list()`, as a convenient and efficient conversion from tuples to lists. [#3044](https://github.com/PyO3/pyo3/pull/3044) ### Changed - Optimize `PySequence` conversion for `list` and `tuple` inputs. [#2944](https://github.com/PyO3/pyo3/pull/2944) - Improve exception raised when creating `#[pyclass]` type object fails during module import. [#2947](https://github.com/PyO3/pyo3/pull/2947) - Optimize `PyMapping` conversion for `dict` inputs. [#2954](https://github.com/PyO3/pyo3/pull/2954) - Allow `create_exception!` to take a `dotted.module` to place the exception in a submodule. [#2979](https://github.com/PyO3/pyo3/pull/2979) ### Fixed - Fix a reference counting race condition affecting `PyObject`s cloned in `allow_threads` blocks. [#2952](https://github.com/PyO3/pyo3/pull/2952) - Fix `clippy::redundant_closure` lint on default arguments in `#[pyo3(signature = (...))]` annotations. [#2990](https://github.com/PyO3/pyo3/pull/2990) - Fix `non_snake_case` lint on generated code in `#[pyfunction]` macro. [#2993](https://github.com/PyO3/pyo3/pull/2993) - Fix some FFI definitions for the upcoming PyPy 3.10 release. [#3031](https://github.com/PyO3/pyo3/pull/3031) ## [0.18.1] - 2023-02-07 ### Added - Add `PyErr::write_unraisable()`. [#2889](https://github.com/PyO3/pyo3/pull/2889) - Add `Python::Ellipsis()` and `PyAny::is_ellipsis()` methods. [#2911](https://github.com/PyO3/pyo3/pull/2911) - Add `PyDict::update()` and `PyDict::update_if_missing()` methods. [#2912](https://github.com/PyO3/pyo3/pull/2912) ### Changed - FFI definition `PyIter_Check` on CPython 3.7 is now implemented as `hasattr(type(obj), "__next__")`, which works correctly on all platforms and adds support for `abi3`. [#2914](https://github.com/PyO3/pyo3/pull/2914) - Warn about unknown config keys in `PYO3_CONFIG_FILE` instead of denying. [#2926](https://github.com/PyO3/pyo3/pull/2926) ### Fixed - Send errors returned by `__releasebuffer__` to `sys.unraisablehook` rather than causing `SystemError`. [#2886](https://github.com/PyO3/pyo3/pull/2886) - Fix downcast to `PyIterator` succeeding for Python classes which did not implement `__next__`. [#2914](https://github.com/PyO3/pyo3/pull/2914) - Fix segfault in `__traverse__` when visiting `None` fields of `Option`. [#2921](https://github.com/PyO3/pyo3/pull/2921) - Fix `#[pymethods(crate = "...")]` option being ignored. [#2923](https://github.com/PyO3/pyo3/pull/2923) - Link against `pythonXY_d.dll` for debug Python builds on Windows. [#2937](https://github.com/PyO3/pyo3/pull/2937) ## [0.18.0] - 2023-01-17 ### Packaging - Relax `indexmap` optional depecency to allow `>= 1.6, < 2`. [#2849](https://github.com/PyO3/pyo3/pull/2849) - Relax `hashbrown` optional dependency to allow `>= 0.9, < 0.14`. [#2875](https://github.com/PyO3/pyo3/pull/2875) - Update `memoffset` dependency to 0.8. [#2875](https://github.com/PyO3/pyo3/pull/2875) ### Added - Add `GILOnceCell::get_or_try_init` for fallible `GILOnceCell` initialization. [#2398](https://github.com/PyO3/pyo3/pull/2398) - Add experimental feature `experimental-inspect` with `type_input()` and `type_output()` helpers to get the Python type of any Python-compatible object. [#2490](https://github.com/PyO3/pyo3/pull/2490) [#2882](https://github.com/PyO3/pyo3/pull/2882) - The `#[pyclass]` macro can now take `get_all` and `set_all` to create getters and setters for every field. [#2692](https://github.com/PyO3/pyo3/pull/2692) - Add `#[pyo3(signature = (...))]` option for `#[pyfunction]` and `#[pymethods]`. [#2702](https://github.com/PyO3/pyo3/pull/2702) - `pyo3-build-config`: rebuild when `PYO3_ENVIRONMENT_SIGNATURE` environment variable value changes. [#2727](https://github.com/PyO3/pyo3/pull/2727) - Add conversions between non-zero int types in `std::num` and Python `int`. [#2730](https://github.com/PyO3/pyo3/pull/2730) - Add `Py::downcast()` as a companion to `PyAny::downcast()`, as well as `downcast_unchecked()` for both types. [#2734](https://github.com/PyO3/pyo3/pull/2734) - Add types for all built-in `Warning` classes as well as `PyErr::warn_explicit`. [#2742](https://github.com/PyO3/pyo3/pull/2742) - Add `abi3-py311` feature. [#2776](https://github.com/PyO3/pyo3/pull/2776) - Add FFI definition `_PyErr_ChainExceptions()` for CPython. [#2788](https://github.com/PyO3/pyo3/pull/2788) - Add FFI definitions `PyVectorcall_NARGS` and `PY_VECTORCALL_ARGUMENTS_OFFSET` for PyPy 3.8 and up. [#2811](https://github.com/PyO3/pyo3/pull/2811) - Add `PyList::get_item_unchecked` for PyPy. [#2827](https://github.com/PyO3/pyo3/pull/2827) ### Changed - PyO3's macros now emit a much nicer error message if function return values don't implement the required trait(s). [#2664](https://github.com/PyO3/pyo3/pull/2664) - Use a TypeError, rather than a ValueError, when refusing to treat a str as a Vec. [#2685](https://github.com/PyO3/pyo3/pull/2685) - Change `PyCFunction::new_closure` to take `name` and `doc` arguments. [#2686](https://github.com/PyO3/pyo3/pull/2686) - `PyType::is_subclass`, `PyErr::is_instance` and `PyAny::is_instance` now take `&PyAny` instead of `&PyType` arguments, so that they work with objects that pretend to be types using `__subclasscheck__` and `__instancecheck__`. [#2695](https://github.com/PyO3/pyo3/pull/2695) - Deprecate `#[args]` attribute and passing "args" specification directly to `#[pyfunction]` in favor of the new `#[pyo3(signature = (...))]` option. [#2702](https://github.com/PyO3/pyo3/pull/2702) - Deprecate required arguments after `Option` arguments to `#[pyfunction]` and `#[pymethods]` without also using `#[pyo3(signature)]` to specify whether the arguments should be required or have defaults. [#2703](https://github.com/PyO3/pyo3/pull/2703) - Change `#[pyfunction]` and `#[pymethods]` to use a common call "trampoline" to slightly reduce generated code size and compile times. [#2705](https://github.com/PyO3/pyo3/pull/2705) - `PyAny::cast_as()` and `Py::cast_as()` are now deprecated in favor of `PyAny::downcast()` and the new `Py::downcast()`. [#2734](https://github.com/PyO3/pyo3/pull/2734) - Relax lifetime bounds on `PyAny::downcast()`. [#2734](https://github.com/PyO3/pyo3/pull/2734) - Automatically generate `__text_signature__` for all Python functions created using `#[pyfunction]` and `#[pymethods]`. [#2784](https://github.com/PyO3/pyo3/pull/2784) - Accept any iterator in `PySet::new` and `PyFrozenSet::new`. [#2795](https://github.com/PyO3/pyo3/pull/2795) - Mixing `#[cfg(...)]` and `#[pyo3(...)]` attributes on `#[pyclass]` struct fields will now work. [#2796](https://github.com/PyO3/pyo3/pull/2796) - Re-enable `PyFunction` on when building for abi3 or PyPy. [#2838](https://github.com/PyO3/pyo3/pull/2838) - Improve `derive(FromPyObject)` to use `intern!` when applicable for `#[pyo3(item)]`. [#2879](https://github.com/PyO3/pyo3/pull/2879) ### Removed - Remove the deprecated `pyproto` feature, `#[pyproto]` macro, and all accompanying APIs. [#2587](https://github.com/PyO3/pyo3/pull/2587) - Remove all functionality deprecated in PyO3 0.16. [#2843](https://github.com/PyO3/pyo3/pull/2843) ### Fixed - Disable `PyModule::filename` on PyPy. [#2715](https://github.com/PyO3/pyo3/pull/2715) - `PyCodeObject` is now once again defined with fields on Python 3.7. [#2726](https://github.com/PyO3/pyo3/pull/2726) - Raise a `TypeError` if `#[new]` pymethods with no arguments receive arguments when called from Python. [#2749](https://github.com/PyO3/pyo3/pull/2749) - Use the `NOARGS` argument calling convention for methods that have a single `py: Python` argument (as a performance optimization). [#2760](https://github.com/PyO3/pyo3/pull/2760) - Fix truncation of `isize` values to `c_long` in `PySlice::new`. [#2769](https://github.com/PyO3/pyo3/pull/2769) - Fix soundness issue with FFI definition `PyUnicodeDecodeError_Create` on PyPy leading to indeterminate behavior (typically a `TypeError`). [#2772](https://github.com/PyO3/pyo3/pull/2772) - Allow functions taking `**kwargs` to accept keyword arguments which share a name with a positional-only argument (as permitted by PEP 570). [#2800](https://github.com/PyO3/pyo3/pull/2800) - Fix unresolved symbol for `PyObject_Vectorcall` on PyPy 3.9 and up. [#2811](https://github.com/PyO3/pyo3/pull/2811) - Fix memory leak in `PyCFunction::new_closure`. [#2842](https://github.com/PyO3/pyo3/pull/2842) ## [0.17.3] - 2022-11-01 ### Packaging - Support Python 3.11. (Previous versions of PyO3 0.17 have been tested against Python 3.11 release candidates and are expected to be compatible, this is the first version tested against Python 3.11.0.) [#2708](https://github.com/PyO3/pyo3/pull/2708) ### Added - Implemented `ExactSizeIterator` for `PyListIterator`, `PyDictIterator`, `PySetIterator` and `PyFrozenSetIterator`. [#2676](https://github.com/PyO3/pyo3/pull/2676) ### Fixed - Fix regression of `impl FromPyObject for [T; N]` no longer accepting types passing `PySequence_Check`, e.g. NumPy arrays, since version 0.17.0. This the same fix that was applied `impl FromPyObject for Vec` in version 0.17.1 extended to fixed-size arrays. [#2675](https://github.com/PyO3/pyo3/pull/2675) - Fix UB in `FunctionDescription::extract_arguments_fastcall` due to creating slices from a null pointer. [#2687](https://github.com/PyO3/pyo3/pull/2687) ## [0.17.2] - 2022-10-04 ### Packaging - Added optional `chrono` feature to convert `chrono` types into types in the `datetime` module. [#2612](https://github.com/PyO3/pyo3/pull/2612) ### Added - Add support for `num-bigint` feature on `PyPy`. [#2626](https://github.com/PyO3/pyo3/pull/2626) ### Fixed - Correctly implement `__richcmp__` for enums, fixing `__ne__` returning always returning `True`. [#2622](https://github.com/PyO3/pyo3/pull/2622) - Fix compile error since 0.17.0 with `Option<&SomePyClass>` argument with a default. [#2630](https://github.com/PyO3/pyo3/pull/2630) - Fix regression of `impl FromPyObject for Vec` no longer accepting types passing `PySequence_Check`, e.g. NumPy arrays, since 0.17.0. [#2631](https://github.com/PyO3/pyo3/pull/2631) ## [0.17.1] - 2022-08-28 ### Fixed - Fix visibility of `PyDictItems`, `PyDictKeys`, and `PyDictValues` types added in PyO3 0.17.0. - Fix compile failure when using `#[pyo3(from_py_with = "...")]` attribute on an argument of type `Option`. [#2592](https://github.com/PyO3/pyo3/pull/2592) - Fix clippy `redundant-closure` lint on `**kwargs` arguments for `#[pyfunction]` and `#[pymethods]`. [#2595](https://github.com/PyO3/pyo3/pull/2595) ## [0.17.0] - 2022-08-23 ### Packaging - Update inventory dependency to `0.3` (the `multiple-pymethods` feature now requires Rust 1.62 for correctness). [#2492](https://github.com/PyO3/pyo3/pull/2492) ### Added - Add `timezone_utc`. [#1588](https://github.com/PyO3/pyo3/pull/1588) - Implement `ToPyObject` for `[T; N]`. [#2313](https://github.com/PyO3/pyo3/pull/2313) - Add `PyDictKeys`, `PyDictValues` and `PyDictItems` Rust types. [#2358](https://github.com/PyO3/pyo3/pull/2358) - Add `append_to_inittab`. [#2377](https://github.com/PyO3/pyo3/pull/2377) - Add FFI definition `PyFrame_GetCode`. [#2406](https://github.com/PyO3/pyo3/pull/2406) - Add `PyCode` and `PyFrame` high level objects. [#2408](https://github.com/PyO3/pyo3/pull/2408) - Add FFI definitions `Py_fstring_input`, `sendfunc`, and `_PyErr_StackItem`. [#2423](https://github.com/PyO3/pyo3/pull/2423) - Add `PyDateTime::new_with_fold`, `PyTime::new_with_fold`, `PyTime::get_fold`, and `PyDateTime::get_fold` for PyPy. [#2428](https://github.com/PyO3/pyo3/pull/2428) - Add `#[pyclass(frozen)]`. [#2448](https://github.com/PyO3/pyo3/pull/2448) - Accept `#[pyo3(name)]` on enum variants. [#2457](https://github.com/PyO3/pyo3/pull/2457) - Add `CompareOp::matches` to implement `__richcmp__` as the result of a Rust `std::cmp::Ordering` comparison. [#2460](https://github.com/PyO3/pyo3/pull/2460) - Add `PySuper` type. [#2486](https://github.com/PyO3/pyo3/pull/2486) - Support PyPy on Windows with the `generate-import-lib` feature. [#2506](https://github.com/PyO3/pyo3/pull/2506) - Add FFI definitions `Py_EnterRecursiveCall` and `Py_LeaveRecursiveCall`. [#2511](https://github.com/PyO3/pyo3/pull/2511) - Add `PyDict::get_item_with_error`. [#2536](https://github.com/PyO3/pyo3/pull/2536) - Add `#[pyclass(sequence)]` option. [#2567](https://github.com/PyO3/pyo3/pull/2567) ### Changed - Change datetime constructors taking a `tzinfo` to take `Option<&PyTzInfo>` instead of `Option<&PyObject>`: `PyDateTime::new`, `PyDateTime::new_with_fold`, `PyTime::new`, and `PyTime::new_with_fold`. [#1588](https://github.com/PyO3/pyo3/pull/1588) - Move `PyTypeObject::type_object` method to the `PyTypeInfo` trait, and deprecate the `PyTypeObject` trait. [#2287](https://github.com/PyO3/pyo3/pull/2287) - Methods of `Py` and `PyAny` now accept `impl IntoPy>` rather than just `&str` to allow use of the `intern!` macro. [#2312](https://github.com/PyO3/pyo3/pull/2312) - Change the deprecated `pyproto` feature to be opt-in instead of opt-out. [#2322](https://github.com/PyO3/pyo3/pull/2322) - Emit better error messages when `#[pyfunction]` return types do not implement `IntoPy`. [#2326](https://github.com/PyO3/pyo3/pull/2326) - Require `T: IntoPy` for `impl IntoPy for [T; N]` instead of `T: ToPyObject`. [#2326](https://github.com/PyO3/pyo3/pull/2326) - Deprecate the `ToBorrowedObject` trait. [#2333](https://github.com/PyO3/pyo3/pull/2333) - Iterators over `PySet` and `PyDict` will now panic if the underlying collection is mutated during the iteration. [#2380](https://github.com/PyO3/pyo3/pull/2380) - Iterators over `PySet` and `PyDict` will now panic if the underlying collection is mutated during the iteration. [#2380](https://github.com/PyO3/pyo3/pull/2380) - Allow `#[classattr]` methods to be fallible. [#2385](https://github.com/PyO3/pyo3/pull/2385) - Prevent multiple `#[pymethods]` with the same name for a single `#[pyclass]`. [#2399](https://github.com/PyO3/pyo3/pull/2399) - Fixup `lib_name` when using `PYO3_CONFIG_FILE`. [#2404](https://github.com/PyO3/pyo3/pull/2404) - Add a message to the `ValueError` raised by the `#[derive(FromPyObject)]` implementation for a tuple struct. [#2414](https://github.com/PyO3/pyo3/pull/2414) - Allow `#[classattr]` methods to take `Python` argument. [#2456](https://github.com/PyO3/pyo3/pull/2456) - Rework `PyCapsule` type to resolve soundness issues: [#2485](https://github.com/PyO3/pyo3/pull/2485) - `PyCapsule::new` and `PyCapsule::new_with_destructor` now take `name: Option` instead of `&CStr`. - The destructor `F` in `PyCapsule::new_with_destructor` must now be `Send`. - `PyCapsule::get_context` deprecated in favor of `PyCapsule::context` which doesn't take a `py: Python<'_>` argument. - `PyCapsule::set_context` no longer takes a `py: Python<'_>` argument. - `PyCapsule::name` now returns `PyResult>` instead of `&CStr`. - `FromPyObject::extract` for `Vec` no longer accepts Python `str` inputs. [#2500](https://github.com/PyO3/pyo3/pull/2500) - Ensure each `#[pymodule]` is only initialized once. [#2523](https://github.com/PyO3/pyo3/pull/2523) - `pyo3_build_config::add_extension_module_link_args` now also emits linker arguments for `wasm32-unknown-emscripten`. [#2538](https://github.com/PyO3/pyo3/pull/2538) - Type checks for `PySequence` and `PyMapping` now require inputs to inherit from (or register with) `collections.abc.Sequence` and `collections.abc.Mapping` respectively. [#2477](https://github.com/PyO3/pyo3/pull/2477) - Disable `PyFunction` on when building for abi3 or PyPy. [#2542](https://github.com/PyO3/pyo3/pull/2542) - Deprecate `Python::acquire_gil`. [#2549](https://github.com/PyO3/pyo3/pull/2549) ### Removed - Remove all functionality deprecated in PyO3 0.15. [#2283](https://github.com/PyO3/pyo3/pull/2283) - Make the `Dict`, `WeakRef` and `BaseNativeType` members of the `PyClass` private implementation details. [#2572](https://github.com/PyO3/pyo3/pull/2572) ### Fixed - Enable incorrectly disabled FFI definition `PyThreadState_DeleteCurrent`. [#2357](https://github.com/PyO3/pyo3/pull/2357) - Fix `wrap_pymodule` interactions with name resolution rules: it no longer "sees through" glob imports of `use submodule::*` when `submodule::submodule` is a `#[pymodule]`. [#2363](https://github.com/PyO3/pyo3/pull/2363) - Correct FFI definition `PyEval_EvalCodeEx` to take `*const *mut PyObject` array arguments instead of `*mut *mut PyObject`. [#2368](https://github.com/PyO3/pyo3/pull/2368) - Fix "raw-ident" structs (e.g. `#[pyclass] struct r#RawName`) incorrectly having `r#` at the start of the class name created in Python. [#2395](https://github.com/PyO3/pyo3/pull/2395) - Correct FFI definition `Py_tracefunc` to be `unsafe extern "C" fn` (was previously safe). [#2407](https://github.com/PyO3/pyo3/pull/2407) - Fix compile failure with `#[pyo3(from_py_with = "...")]` annotations on a field in a `#[derive(FromPyObject)]` struct. [#2414](https://github.com/PyO3/pyo3/pull/2414) - Fix FFI definitions `_PyDateTime_BaseTime` and `_PyDateTime_BaseDateTime` lacking leading underscores in their names. [#2421](https://github.com/PyO3/pyo3/pull/2421) - Remove FFI definition `PyArena` on Python 3.10 and up. [#2421](https://github.com/PyO3/pyo3/pull/2421) - Fix FFI definition `PyCompilerFlags` missing member `cf_feature_version` on Python 3.8 and up. [#2423](https://github.com/PyO3/pyo3/pull/2423) - Fix FFI definition `PyAsyncMethods` missing member `am_send` on Python 3.10 and up. [#2423](https://github.com/PyO3/pyo3/pull/2423) - Fix FFI definition `PyGenObject` having multiple incorrect members on various Python versions. [#2423](https://github.com/PyO3/pyo3/pull/2423) - Fix FFI definition `PySyntaxErrorObject` missing members `end_lineno` and `end_offset` on Python 3.10 and up. [#2423](https://github.com/PyO3/pyo3/pull/2423) - Fix FFI definition `PyHeapTypeObject` missing member `ht_module` on Python 3.9 and up. [#2423](https://github.com/PyO3/pyo3/pull/2423) - Fix FFI definition `PyFrameObject` having multiple incorrect members on various Python versions. [#2424](https://github.com/PyO3/pyo3/pull/2424) [#2434](https://github.com/PyO3/pyo3/pull/2434) - Fix FFI definition `PyTypeObject` missing deprecated field `tp_print` on Python 3.8. [#2428](https://github.com/PyO3/pyo3/pull/2428) - Fix FFI definitions `PyDateTime_CAPI`. `PyDateTime_Date`, `PyASCIIObject`, `PyBaseExceptionObject`, `PyListObject`, and `PyTypeObject` on PyPy. [#2428](https://github.com/PyO3/pyo3/pull/2428) - Fix FFI definition `_inittab` field `initfunc` typo'd as `initfun`. [#2431](https://github.com/PyO3/pyo3/pull/2431) - Fix FFI definitions `_PyDateTime_BaseTime` and `_PyDateTime_BaseDateTime` incorrectly having `fold` member. [#2432](https://github.com/PyO3/pyo3/pull/2432) - Fix FFI definitions `PyTypeObject`. `PyHeapTypeObject`, and `PyCFunctionObject` having incorrect members on PyPy 3.9. [#2433](https://github.com/PyO3/pyo3/pull/2433) - Fix FFI definition `PyGetSetDef` to have `*const c_char` for `doc` member (not `*mut c_char`). [#2439](https://github.com/PyO3/pyo3/pull/2439) - Fix `#[pyo3(from_py_with = "...")]` being ignored for 1-element tuple structs and transparent structs. [#2440](https://github.com/PyO3/pyo3/pull/2440) - Use `memoffset` to avoid UB when computing `PyCell` layout. [#2450](https://github.com/PyO3/pyo3/pull/2450) - Fix incorrect enum names being returned by the generated `repr` for enums renamed by `#[pyclass(name = "...")]` [#2457](https://github.com/PyO3/pyo3/pull/2457) - Fix `PyObject_CallNoArgs` incorrectly being available when building for abi3 on Python 3.9. [#2476](https://github.com/PyO3/pyo3/pull/2476) - Fix several clippy warnings generated by `#[pyfunction]` arguments. [#2503](https://github.com/PyO3/pyo3/pull/2503) ## [0.16.6] - 2022-08-23 ### Changed - Fix soundness issues with `PyCapsule` type with select workarounds. Users are encourage to upgrade to PyO3 0.17 at their earliest convenience which contains API breakages which fix the issues in a long-term fashion. [#2522](https://github.com/PyO3/pyo3/pull/2522) - `PyCapsule::new` and `PyCapsule::new_with_destructor` now take ownership of a copy of the `name` to resolve a possible use-after-free. - `PyCapsule::name` now returns an empty `CStr` instead of dereferencing a null pointer if the capsule has no name. - The destructor `F` in `PyCapsule::new_with_destructor` will never be called if the capsule is deleted from a thread other than the one which the capsule was created in (a warning will be emitted). - Panics during drop of panic payload caught by PyO3 will now abort. [#2544](https://github.com/PyO3/pyo3/pull/2544) ## [0.16.5] - 2022-05-15 ### Added - Add an experimental `generate-import-lib` feature to support auto-generating non-abi3 python import libraries for Windows targets. [#2364](https://github.com/PyO3/pyo3/pull/2364) - Add FFI definition `Py_ExitStatusException`. [#2374](https://github.com/PyO3/pyo3/pull/2374) ### Changed - Deprecate experimental `generate-abi3-import-lib` feature in favor of the new `generate-import-lib` feature. [#2364](https://github.com/PyO3/pyo3/pull/2364) ### Fixed - Added missing `warn_default_encoding` field to `PyConfig` on 3.10+. The previously missing field could result in incorrect behavior or crashes. [#2370](https://github.com/PyO3/pyo3/pull/2370) - Fixed order of `pathconfig_warnings` and `program_name` fields of `PyConfig` on 3.10+. Previously, the order of the fields was swapped and this could lead to incorrect behavior or crashes. [#2370](https://github.com/PyO3/pyo3/pull/2370) ## [0.16.4] - 2022-04-14 ### Added - Add `PyTzInfoAccess` trait for safe access to time zone information. [#2263](https://github.com/PyO3/pyo3/pull/2263) - Add an experimental `generate-abi3-import-lib` feature to auto-generate `python3.dll` import libraries for Windows. [#2282](https://github.com/PyO3/pyo3/pull/2282) - Add FFI definitions for `PyDateTime_BaseTime` and `PyDateTime_BaseDateTime`. [#2294](https://github.com/PyO3/pyo3/pull/2294) ### Changed - Improved performance of failing calls to `FromPyObject::extract` which is common when functions accept multiple distinct types. [#2279](https://github.com/PyO3/pyo3/pull/2279) - Default to "m" ABI tag when choosing `libpython` link name for CPython 3.7 on Unix. [#2288](https://github.com/PyO3/pyo3/pull/2288) - Allow to compile "abi3" extensions without a working build host Python interpreter. [#2293](https://github.com/PyO3/pyo3/pull/2293) ### Fixed - Crates depending on PyO3 can collect code coverage via LLVM instrumentation using stable Rust. [#2286](https://github.com/PyO3/pyo3/pull/2286) - Fix segfault when calling FFI methods `PyDateTime_DATE_GET_TZINFO` or `PyDateTime_TIME_GET_TZINFO` on `datetime` or `time` without a tzinfo. [#2289](https://github.com/PyO3/pyo3/pull/2289) - Fix directory names starting with the letter `n` breaking serialization of the interpreter configuration on Windows since PyO3 0.16.3. [#2299](https://github.com/PyO3/pyo3/pull/2299) ## [0.16.3] - 2022-04-05 ### Packaging - Extend `parking_lot` dependency supported versions to include 0.12. [#2239](https://github.com/PyO3/pyo3/pull/2239) ### Added - Add methods to `pyo3_build_config::InterpreterConfig` to run Python scripts using the configured executable. [#2092](https://github.com/PyO3/pyo3/pull/2092) - Add `as_bytes` method to `Py`. [#2235](https://github.com/PyO3/pyo3/pull/2235) - Add FFI definitions for `PyType_FromModuleAndSpec`, `PyType_GetModule`, `PyType_GetModuleState` and `PyModule_AddType`. [#2250](https://github.com/PyO3/pyo3/pull/2250) - Add `pyo3_build_config::cross_compiling_from_to` as a helper to detect when PyO3 is cross-compiling. [#2253](https://github.com/PyO3/pyo3/pull/2253) - Add `#[pyclass(mapping)]` option to leave sequence slots empty in container implementations. [#2265](https://github.com/PyO3/pyo3/pull/2265) - Add `PyString::intern` to enable usage of the Python's built-in string interning. [#2268](https://github.com/PyO3/pyo3/pull/2268) - Add `intern!` macro which can be used to amortize the cost of creating Python strings by storing them inside a `GILOnceCell`. [#2269](https://github.com/PyO3/pyo3/pull/2269) - Add `PYO3_CROSS_PYTHON_IMPLEMENTATION` environment variable for selecting the default cross Python implementation. [#2272](https://github.com/PyO3/pyo3/pull/2272) ### Changed - Allow `#[pyo3(crate = "...", text_signature = "...")]` options to be used directly in `#[pyclass(crate = "...", text_signature = "...")]`. [#2234](https://github.com/PyO3/pyo3/pull/2234) - Make `PYO3_CROSS_LIB_DIR` environment variable optional when cross compiling. [#2241](https://github.com/PyO3/pyo3/pull/2241) - Mark `METH_FASTCALL` calling convention as limited API on Python 3.10. [#2250](https://github.com/PyO3/pyo3/pull/2250) - Deprecate `pyo3_build_config::cross_compiling` in favor of `pyo3_build_config::cross_compiling_from_to`. [#2253](https://github.com/PyO3/pyo3/pull/2253) ### Fixed - Fix `abi3-py310` feature: use Python 3.10 ABI when available instead of silently falling back to the 3.9 ABI. [#2242](https://github.com/PyO3/pyo3/pull/2242) - Use shared linking mode when cross compiling against a [Framework bundle](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPFrameworks/Concepts/FrameworkAnatomy.html) for macOS. [#2233](https://github.com/PyO3/pyo3/pull/2233) - Fix panic during compilation when `PYO3_CROSS_LIB_DIR` is set for some host/target combinations. [#2232](https://github.com/PyO3/pyo3/pull/2232) - Correct dependency version for `syn` to require minimal patch version 1.0.56. [#2240](https://github.com/PyO3/pyo3/pull/2240) ## [0.16.2] - 2022-03-15 ### Packaging - Warn when modules are imported on PyPy 3.7 versions older than PyPy 7.3.8, as they are known to have binary compatibility issues. [#2217](https://github.com/PyO3/pyo3/pull/2217) - Ensure build script of `pyo3-ffi` runs before that of `pyo3` to fix cross compilation. [#2224](https://github.com/PyO3/pyo3/pull/2224) ## [0.16.1] - 2022-03-05 ### Packaging - Extend `hashbrown` optional dependency supported versions to include 0.12. [#2197](https://github.com/PyO3/pyo3/pull/2197) ### Fixed - Fix incorrect platform detection for Windows in `pyo3-build-config`. [#2198](https://github.com/PyO3/pyo3/pull/2198) - Fix regression from 0.16 preventing cross compiling to aarch64 macOS. [#2201](https://github.com/PyO3/pyo3/pull/2201) ## [0.16.0] - 2022-02-27 ### Packaging - Update MSRV to Rust 1.48. [#2004](https://github.com/PyO3/pyo3/pull/2004) - Update `indoc` optional dependency to 1.0. [#2004](https://github.com/PyO3/pyo3/pull/2004) - Drop support for Python 3.6, remove `abi3-py36` feature. [#2006](https://github.com/PyO3/pyo3/pull/2006) - `pyo3-build-config` no longer enables the `resolve-config` feature by default. [#2008](https://github.com/PyO3/pyo3/pull/2008) - Update `inventory` optional dependency to 0.2. [#2019](https://github.com/PyO3/pyo3/pull/2019) - Drop `paste` dependency. [#2081](https://github.com/PyO3/pyo3/pull/2081) - The bindings found in `pyo3::ffi` are now a re-export of a separate `pyo3-ffi` crate. [#2126](https://github.com/PyO3/pyo3/pull/2126) - Support PyPy 3.9. [#2143](https://github.com/PyO3/pyo3/pull/2143) ### Added - Add `PyCapsule` type exposing the [Capsule API](https://docs.python.org/3/c-api/capsule.html#capsules). [#1980](https://github.com/PyO3/pyo3/pull/1980) - Add `pyo3_build_config::Sysconfigdata` and supporting APIs. [#1996](https://github.com/PyO3/pyo3/pull/1996) - Add `Py::setattr` method. [#2009](https://github.com/PyO3/pyo3/pull/2009) - Add `#[pyo3(crate = "some::path")]` option to all attribute macros (except the deprecated `#[pyproto]`). [#2022](https://github.com/PyO3/pyo3/pull/2022) - Enable `create_exception!` macro to take an optional docstring. [#2027](https://github.com/PyO3/pyo3/pull/2027) - Enable `#[pyclass]` for fieldless (aka C-like) enums. [#2034](https://github.com/PyO3/pyo3/pull/2034) - Add buffer magic methods `__getbuffer__` and `__releasebuffer__` to `#[pymethods]`. [#2067](https://github.com/PyO3/pyo3/pull/2067) - Add support for paths in `wrap_pyfunction` and `wrap_pymodule`. [#2081](https://github.com/PyO3/pyo3/pull/2081) - Enable `wrap_pyfunction!` to wrap a `#[pyfunction]` implemented in a different Rust module or crate. [#2091](https://github.com/PyO3/pyo3/pull/2091) - Add `PyAny::contains` method (`in` operator for `PyAny`). [#2115](https://github.com/PyO3/pyo3/pull/2115) - Add `PyMapping::contains` method (`in` operator for `PyMapping`). [#2133](https://github.com/PyO3/pyo3/pull/2133) - Add garbage collection magic magic methods `__traverse__` and `__clear__` to `#[pymethods]`. [#2159](https://github.com/PyO3/pyo3/pull/2159) - Add support for `from_py_with` on struct tuples and enums to override the default from-Python conversion. [#2181](https://github.com/PyO3/pyo3/pull/2181) - Add `eq`, `ne`, `lt`, `le`, `gt`, `ge` methods to `PyAny` that wrap `rich_compare`. [#2175](https://github.com/PyO3/pyo3/pull/2175) - Add `Py::is` and `PyAny::is` methods to check for object identity. [#2183](https://github.com/PyO3/pyo3/pull/2183) - Add support for the `__getattribute__` magic method. [#2187](https://github.com/PyO3/pyo3/pull/2187) ### Changed - `PyType::is_subclass`, `PyErr::is_instance` and `PyAny::is_instance` now operate run-time type object instead of a type known at compile-time. The old behavior is still available as `PyType::is_subclass_of`, `PyErr::is_instance_of` and `PyAny::is_instance_of`. [#1985](https://github.com/PyO3/pyo3/pull/1985) - Rename some methods on `PyErr` (the old names are just marked deprecated for now): [#2026](https://github.com/PyO3/pyo3/pull/2026) - `pytype` -> `get_type` - `pvalue` -> `value` (and deprecate equivalent `instance`) - `ptraceback` -> `traceback` - `from_instance` -> `from_value` - `into_instance` -> `into_value` - `PyErr::new_type` now takes an optional docstring and now returns `PyResult>` rather than a `ffi::PyTypeObject` pointer. [#2027](https://github.com/PyO3/pyo3/pull/2027) - Deprecate `PyType::is_instance`; it is inconsistent with other `is_instance` methods in PyO3. Instead of `typ.is_instance(obj)`, use `obj.is_instance(typ)`. [#2031](https://github.com/PyO3/pyo3/pull/2031) - `__getitem__`, `__setitem__` and `__delitem__` in `#[pymethods]` now implement both a Python mapping and sequence by default. [#2065](https://github.com/PyO3/pyo3/pull/2065) - Improve performance and error messages for `#[derive(FromPyObject)]` for enums. [#2068](https://github.com/PyO3/pyo3/pull/2068) - Reduce generated LLVM code size (to improve compile times) for: - internal `handle_panic` helper [#2074](https://github.com/PyO3/pyo3/pull/2074) [#2158](https://github.com/PyO3/pyo3/pull/2158) - `#[pyfunction]` and `#[pymethods]` argument extraction [#2075](https://github.com/PyO3/pyo3/pull/2075) [#2085](https://github.com/PyO3/pyo3/pull/2085) - `#[pyclass]` type object creation [#2076](https://github.com/PyO3/pyo3/pull/2076) [#2081](https://github.com/PyO3/pyo3/pull/2081) [#2157](https://github.com/PyO3/pyo3/pull/2157) - Respect Rust privacy rules for items wrapped with `wrap_pyfunction` and `wrap_pymodule`. [#2081](https://github.com/PyO3/pyo3/pull/2081) - Add modulo argument to `__ipow__` magic method. [#2083](https://github.com/PyO3/pyo3/pull/2083) - Fix FFI definition for `_PyCFunctionFast`. [#2126](https://github.com/PyO3/pyo3/pull/2126) - `PyDateTimeAPI` and `PyDateTime_TimeZone_UTC` are are now unsafe functions instead of statics. [#2126](https://github.com/PyO3/pyo3/pull/2126) - `PyDateTimeAPI` does not implicitly call `PyDateTime_IMPORT` anymore to reflect the original Python API more closely. Before the first call to `PyDateTime_IMPORT` a null pointer is returned. Therefore before calling any of the following FFI functions `PyDateTime_IMPORT` must be called to avoid undefined behavior: [#2126](https://github.com/PyO3/pyo3/pull/2126) - `PyDateTime_TimeZone_UTC` - `PyDate_Check` - `PyDate_CheckExact` - `PyDateTime_Check` - `PyDateTime_CheckExact` - `PyTime_Check` - `PyTime_CheckExact` - `PyDelta_Check` - `PyDelta_CheckExact` - `PyTZInfo_Check` - `PyTZInfo_CheckExact` - `PyDateTime_FromTimestamp` - `PyDate_FromTimestamp` - Deprecate the `gc` option for `pyclass` (e.g. `#[pyclass(gc)]`). Just implement a `__traverse__` `#[pymethod]`. [#2159](https://github.com/PyO3/pyo3/pull/2159) - The `ml_meth` field of `PyMethodDef` is now represented by the `PyMethodDefPointer` union. [2166](https://github.com/PyO3/pyo3/pull/2166) - Deprecate the `#[pyproto]` traits. [#2173](https://github.com/PyO3/pyo3/pull/2173) ### Removed - Remove all functionality deprecated in PyO3 0.14. [#2007](https://github.com/PyO3/pyo3/pull/2007) - Remove `Default` impl for `PyMethodDef`. [#2166](https://github.com/PyO3/pyo3/pull/2166) - Remove `PartialEq` impl for `Py` and `PyAny` (use the new `is` instead). [#2183](https://github.com/PyO3/pyo3/pull/2183) ### Fixed - Fix undefined symbol for `PyObject_HasAttr` on PyPy. [#2025](https://github.com/PyO3/pyo3/pull/2025) - Fix memory leak in `PyErr::into_value`. [#2026](https://github.com/PyO3/pyo3/pull/2026) - Fix clippy warning `needless-option-as-deref` in code generated by `#[pyfunction]` and `#[pymethods]`. [#2040](https://github.com/PyO3/pyo3/pull/2040) - Fix undefined behavior in `PySlice::indices`. [#2061](https://github.com/PyO3/pyo3/pull/2061) - Fix the `wrap_pymodule!` macro using the wrong name for a `#[pymodule]` with a `#[pyo3(name = "..")]` attribute. [#2081](https://github.com/PyO3/pyo3/pull/2081) - Fix magic methods in `#[pymethods]` accepting implementations with the wrong number of arguments. [#2083](https://github.com/PyO3/pyo3/pull/2083) - Fix panic in `#[pyfunction]` generated code when a required argument following an `Option` was not provided. [#2093](https://github.com/PyO3/pyo3/pull/2093) - Fixed undefined behavior caused by incorrect `ExactSizeIterator` implementations. [#2124](https://github.com/PyO3/pyo3/pull/2124) - Fix missing FFI definition `PyCMethod_New` on Python 3.9 and up. [#2143](https://github.com/PyO3/pyo3/pull/2143) - Add missing FFI definitions `_PyLong_NumBits` and `_PyLong_AsByteArray` on PyPy. [#2146](https://github.com/PyO3/pyo3/pull/2146) - Fix memory leak in implementation of `AsPyPointer` for `Option`. [#2160](https://github.com/PyO3/pyo3/pull/2160) - Fix FFI definition of `_PyLong_NumBits` to return `size_t` instead of `c_int`. [#2161](https://github.com/PyO3/pyo3/pull/2161) - Fix `TypeError` thrown when argument parsing failed missing the originating causes. [2177](https://github.com/PyO3/pyo3/pull/2178) ## [0.15.2] - 2022-04-14 ### Packaging - Backport of PyPy 3.9 support from PyO3 0.16. [#2262](https://github.com/PyO3/pyo3/pull/2262) ## [0.15.1] - 2021-11-19 ### Added - Add implementations for `Py::as_ref` and `Py::into_ref` for `Py`, `Py` and `Py`. [#1682](https://github.com/PyO3/pyo3/pull/1682) - Add `PyTraceback` type to represent and format Python tracebacks. [#1977](https://github.com/PyO3/pyo3/pull/1977) ### Changed - `#[classattr]` constants with a known magic method name (which is lowercase) no longer trigger lint warnings expecting constants to be uppercase. [#1969](https://github.com/PyO3/pyo3/pull/1969) ### Fixed - Fix creating `#[classattr]` by functions with the name of a known magic method. [#1969](https://github.com/PyO3/pyo3/pull/1969) - Fix use of `catch_unwind` in `allow_threads` which can cause fatal crashes. [#1989](https://github.com/PyO3/pyo3/pull/1989) - Fix build failure on PyPy when abi3 features are activated. [#1991](https://github.com/PyO3/pyo3/pull/1991) - Fix mingw platform detection. [#1993](https://github.com/PyO3/pyo3/pull/1993) - Fix panic in `__get__` implementation when accessing descriptor on type object. [#1997](https://github.com/PyO3/pyo3/pull/1997) ## [0.15.0] - 2021-11-03 ### Packaging - `pyo3`'s `Cargo.toml` now advertises `links = "python"` to inform Cargo that it links against *libpython*. [#1819](https://github.com/PyO3/pyo3/pull/1819) - Added optional `anyhow` feature to convert `anyhow::Error` into `PyErr`. [#1822](https://github.com/PyO3/pyo3/pull/1822) - Support Python 3.10. [#1889](https://github.com/PyO3/pyo3/pull/1889) - Added optional `eyre` feature to convert `eyre::Report` into `PyErr`. [#1893](https://github.com/PyO3/pyo3/pull/1893) - Support PyPy 3.8. [#1948](https://github.com/PyO3/pyo3/pull/1948) ### Added - Add `PyList::get_item_unchecked` and `PyTuple::get_item_unchecked` to get items without bounds checks. [#1733](https://github.com/PyO3/pyo3/pull/1733) - Support `#[doc = include_str!(...)]` attributes on Rust 1.54 and up. [#1746](https://github.com/PyO3/pyo3/issues/1746) - Add `PyAny::py` as a convenience for `PyNativeType::py`. [#1751](https://github.com/PyO3/pyo3/pull/1751) - Add implementation of `std::ops::Index` for `PyList`, `PyTuple` and `PySequence`. [#1825](https://github.com/PyO3/pyo3/pull/1825) - Add range indexing implementations of `std::ops::Index` for `PyList`, `PyTuple` and `PySequence`. [#1829](https://github.com/PyO3/pyo3/pull/1829) - Add `PyMapping` type to represent the Python mapping protocol. [#1844](https://github.com/PyO3/pyo3/pull/1844) - Add commonly-used sequence methods to `PyList` and `PyTuple`. [#1849](https://github.com/PyO3/pyo3/pull/1849) - Add `as_sequence` methods to `PyList` and `PyTuple`. [#1860](https://github.com/PyO3/pyo3/pull/1860) - Add support for magic methods in `#[pymethods]`, intended as a replacement for `#[pyproto]`. [#1864](https://github.com/PyO3/pyo3/pull/1864) - Add `abi3-py310` feature. [#1889](https://github.com/PyO3/pyo3/pull/1889) - Add `PyCFunction::new_closure` to create a Python function from a Rust closure. [#1901](https://github.com/PyO3/pyo3/pull/1901) - Add support for positional-only arguments in `#[pyfunction]`. [#1925](https://github.com/PyO3/pyo3/pull/1925) - Add `PyErr::take` to attempt to fetch a Python exception if present. [#1957](https://github.com/PyO3/pyo3/pull/1957) ### Changed - `PyList`, `PyTuple` and `PySequence`'s APIs now accepts only `usize` indices instead of `isize`. [#1733](https://github.com/PyO3/pyo3/pull/1733), [#1802](https://github.com/PyO3/pyo3/pull/1802), [#1803](https://github.com/PyO3/pyo3/pull/1803) - `PyList::get_item` and `PyTuple::get_item` now return `PyResult<&PyAny>` instead of panicking. [#1733](https://github.com/PyO3/pyo3/pull/1733) - `PySequence::in_place_repeat` and `PySequence::in_place_concat` now return `PyResult<&PySequence>` instead of `PyResult<()>`, which is needed in case of immutable sequences such as tuples. [#1803](https://github.com/PyO3/pyo3/pull/1803) - `PySequence::get_slice` now returns `PyResult<&PySequence>` instead of `PyResult<&PyAny>`. [#1829](https://github.com/PyO3/pyo3/pull/1829) - Deprecate `PyTuple::split_from`. [#1804](https://github.com/PyO3/pyo3/pull/1804) - Deprecate `PyTuple::slice`, new method `PyTuple::get_slice` added with `usize` indices. [#1828](https://github.com/PyO3/pyo3/pull/1828) - Deprecate FFI definitions `PyParser_SimpleParseStringFlags`, `PyParser_SimpleParseStringFlagsFilename`, `PyParser_SimpleParseFileFlags` when building for Python 3.9. [#1830](https://github.com/PyO3/pyo3/pull/1830) - Mark FFI definitions removed in Python 3.10 `PyParser_ASTFromString`, `PyParser_ASTFromStringObject`, `PyParser_ASTFromFile`, `PyParser_ASTFromFileObject`, `PyParser_SimpleParseStringFlags`, `PyParser_SimpleParseStringFlagsFilename`, `PyParser_SimpleParseFileFlags`, `PyParser_SimpleParseString`, `PyParser_SimpleParseFile`, `Py_SymtableString`, and `Py_SymtableStringObject`. [#1830](https://github.com/PyO3/pyo3/pull/1830) - `#[pymethods]` now handles magic methods similarly to `#[pyproto]`. In the future, `#[pyproto]` may be deprecated. [#1864](https://github.com/PyO3/pyo3/pull/1864) - Deprecate FFI definitions `PySys_AddWarnOption`, `PySys_AddWarnOptionUnicode` and `PySys_HasWarnOptions`. [#1887](https://github.com/PyO3/pyo3/pull/1887) - Deprecate `#[call]` attribute in favor of using `fn __call__`. [#1929](https://github.com/PyO3/pyo3/pull/1929) - Fix missing FFI definition `_PyImport_FindExtensionObject` on Python 3.10. [#1942](https://github.com/PyO3/pyo3/pull/1942) - Change `PyErr::fetch` to panic in debug mode if no exception is present. [#1957](https://github.com/PyO3/pyo3/pull/1957) ### Fixed - Fix building with a conda environment on Windows. [#1873](https://github.com/PyO3/pyo3/pull/1873) - Fix panic on Python 3.6 when calling `Python::with_gil` with Python initialized but threading not initialized. [#1874](https://github.com/PyO3/pyo3/pull/1874) - Fix incorrect linking to version-specific DLL instead of `python3.dll` when cross-compiling to Windows with `abi3`. [#1880](https://github.com/PyO3/pyo3/pull/1880) - Fix FFI definition for `PyTuple_ClearFreeList` incorrectly being present for Python 3.9 and up. [#1887](https://github.com/PyO3/pyo3/pull/1887) - Fix panic in generated `#[derive(FromPyObject)]` for enums. [#1888](https://github.com/PyO3/pyo3/pull/1888) - Fix cross-compiling to Python 3.7 builds with the "m" abi flag. [#1908](https://github.com/PyO3/pyo3/pull/1908) - Fix `__mod__` magic method fallback to `__rmod__`. [#1934](https://github.com/PyO3/pyo3/pull/1934). - Fix missing FFI definition `_PyImport_FindExtensionObject` on Python 3.10. [#1942](https://github.com/PyO3/pyo3/pull/1942) ## [0.14.5] - 2021-09-05 ### Added - Make `pyo3_build_config::InterpreterConfig` and subfields public. [#1848](https://github.com/PyO3/pyo3/pull/1848) - Add `resolve-config` feature to the `pyo3-build-config` to control whether its build script does anything. [#1856](https://github.com/PyO3/pyo3/pull/1856) ### Fixed - Fix 0.14.4 compile regression on `s390x-unknown-linux-gnu` target. [#1850](https://github.com/PyO3/pyo3/pull/1850) ## [0.14.4] - 2021-08-29 ### Changed - Mark `PyString::data` as `unsafe` and disable it and some supporting PyUnicode FFI APIs (which depend on a C bitfield) on big-endian targets. [#1834](https://github.com/PyO3/pyo3/pull/1834) ## [0.14.3] - 2021-08-22 ### Added - Add `PyString::data` to access the raw bytes stored in a Python string. [#1794](https://github.com/PyO3/pyo3/pull/1794) ### Fixed - Raise `AttributeError` to avoid panic when calling `del` on a `#[setter]` defined class property. [#1779](https://github.com/PyO3/pyo3/pull/1779) - Restrict FFI definitions `PyGILState_Check` and `Py_tracefunc` to the unlimited API. [#1787](https://github.com/PyO3/pyo3/pull/1787) - Add missing `_type` field to `PyStatus` struct definition. [#1791](https://github.com/PyO3/pyo3/pull/1791) - Reduce lower bound `num-complex` optional dependency to support interop with `rust-numpy` and `ndarray` when building with the MSRV of 1.41 [#1799](https://github.com/PyO3/pyo3/pull/1799) - Fix memory leak in `Python::run_code`. [#1806](https://github.com/PyO3/pyo3/pull/1806) - Fix memory leak in `PyModule::from_code`. [#1810](https://github.com/PyO3/pyo3/pull/1810) - Remove use of `pyo3::` in `pyo3::types::datetime` which broke builds using `-Z avoid-dev-deps` [#1811](https://github.com/PyO3/pyo3/pull/1811) ## [0.14.2] - 2021-08-09 ### Added - Add `indexmap` feature to add `ToPyObject`, `IntoPy` and `FromPyObject` implementations for `indexmap::IndexMap`. [#1728](https://github.com/PyO3/pyo3/pull/1728) - Add `pyo3_build_config::add_extension_module_link_args` to use in build scripts to set linker arguments (for macOS). [#1755](https://github.com/PyO3/pyo3/pull/1755) - Add `Python::with_gil_unchecked` unsafe variation of `Python::with_gil` to allow obtaining a `Python` in scenarios where `Python::with_gil` would fail. [#1769](https://github.com/PyO3/pyo3/pull/1769) ### Changed - `PyErr::new` no longer acquires the Python GIL internally. [#1724](https://github.com/PyO3/pyo3/pull/1724) - Reverted PyO3 0.14.0's use of `cargo:rustc-cdylib-link-arg` in its build script, as Cargo unintentionally allowed crates to pass linker args to downstream crates in this way. Projects supporting macOS may need to restore `.cargo/config.toml` files. [#1755](https://github.com/PyO3/pyo3/pull/1755) ### Fixed - Fix regression in 0.14.0 rejecting usage of `#[doc(hidden)]` on structs and functions annotated with PyO3 macros. [#1722](https://github.com/PyO3/pyo3/pull/1722) - Fix regression in 0.14.0 leading to incorrect code coverage being computed for `#[pyfunction]`s. [#1726](https://github.com/PyO3/pyo3/pull/1726) - Fix incorrect FFI definition of `Py_Buffer` on PyPy. [#1737](https://github.com/PyO3/pyo3/pull/1737) - Fix incorrect calculation of `dictoffset` on 32-bit Windows. [#1475](https://github.com/PyO3/pyo3/pull/1475) - Fix regression in 0.13.2 leading to linking to incorrect Python library on Windows "gnu" targets. [#1759](https://github.com/PyO3/pyo3/pull/1759) - Fix compiler warning: deny trailing semicolons in expression macro. [#1762](https://github.com/PyO3/pyo3/pull/1762) - Fix incorrect FFI definition of `Py_DecodeLocale`. The 2nd argument is now `*mut Py_ssize_t` instead of `Py_ssize_t`. [#1766](https://github.com/PyO3/pyo3/pull/1766) ## [0.14.1] - 2021-07-04 ### Added - Implement `IntoPy` for `&PathBuf` and `&OsString`. [#1712](https://github.com/PyO3/pyo3/pull/1712) ### Fixed - Fix crashes on PyPy due to incorrect definitions of `PyList_SET_ITEM`. [#1713](https://github.com/PyO3/pyo3/pull/1713) ## [0.14.0] - 2021-07-03 ### Packaging - Update `num-bigint` optional dependency to 0.4. [#1481](https://github.com/PyO3/pyo3/pull/1481) - Update `num-complex` optional dependency to 0.4. [#1482](https://github.com/PyO3/pyo3/pull/1482) - Extend `hashbrown` optional dependency supported versions to include 0.11. [#1496](https://github.com/PyO3/pyo3/pull/1496) - Support PyPy 3.7. [#1538](https://github.com/PyO3/pyo3/pull/1538) ### Added - Extend conversions for `[T; N]` to all `N` using const generics (on Rust 1.51 and up). [#1128](https://github.com/PyO3/pyo3/pull/1128) - Add conversions between `OsStr`/ `OsString` and Python strings. [#1379](https://github.com/PyO3/pyo3/pull/1379) - Add conversions between `Path`/ `PathBuf` and Python strings (and `pathlib.Path` objects). [#1379](https://github.com/PyO3/pyo3/pull/1379) [#1654](https://github.com/PyO3/pyo3/pull/1654) - Add a new set of `#[pyo3(...)]` attributes to control various PyO3 macro functionality: - `#[pyo3(from_py_with = "...")]` function arguments and struct fields to override the default from-Python conversion. [#1411](https://github.com/PyO3/pyo3/pull/1411) - `#[pyo3(name = "...")]` for setting Python names. [#1567](https://github.com/PyO3/pyo3/pull/1567) - `#[pyo3(text_signature = "...")]` for setting text signature. [#1658](https://github.com/PyO3/pyo3/pull/1658) - Add FFI definition `PyCFunction_CheckExact` for Python 3.9 and later. [#1425](https://github.com/PyO3/pyo3/pull/1425) - Add FFI definition `Py_IS_TYPE`. [#1429](https://github.com/PyO3/pyo3/pull/1429) - Add FFI definition `_Py_InitializeMain`. [#1473](https://github.com/PyO3/pyo3/pull/1473) - Add FFI definitions from `cpython/import.h`.[#1475](https://github.com/PyO3/pyo3/pull/1475) - Add tuple and unit struct support for `#[pyclass]` macro. [#1504](https://github.com/PyO3/pyo3/pull/1504) - Add FFI definition `PyDateTime_TimeZone_UTC`. [#1572](https://github.com/PyO3/pyo3/pull/1572) - Add support for `#[pyclass(extends=Exception)]`. [#1591](https://github.com/PyO3/pyo3/pull/1591) - Add `PyErr::cause` and `PyErr::set_cause`. [#1679](https://github.com/PyO3/pyo3/pull/1679) - Add FFI definitions from `cpython/pystate.h`. [#1687](https://github.com/PyO3/pyo3/pull/1687/) - Add `wrap_pyfunction!` macro to `pyo3::prelude`. [#1695](https://github.com/PyO3/pyo3/pull/1695) ### Changed - Allow only one `#[pymethods]` block per `#[pyclass]` by default, to remove the dependency on `inventory`. Add a `multiple-pymethods` feature to opt-in the original behavior and dependency on `inventory`. [#1457](https://github.com/PyO3/pyo3/pull/1457) - Change `PyTimeAccess::get_fold` to return a `bool` instead of a `u8`. [#1397](https://github.com/PyO3/pyo3/pull/1397) - Deprecate FFI definition `PyCFunction_Call` for Python 3.9 and up. [#1425](https://github.com/PyO3/pyo3/pull/1425) - Deprecate FFI definition `PyModule_GetFilename`. [#1425](https://github.com/PyO3/pyo3/pull/1425) - The `auto-initialize` feature is no longer enabled by default. [#1443](https://github.com/PyO3/pyo3/pull/1443) - Change `PyCFunction::new` and `PyCFunction::new_with_keywords` to take `&'static str` arguments rather than implicitly copying (and leaking) them. [#1450](https://github.com/PyO3/pyo3/pull/1450) - Deprecate `PyModule::call`, `PyModule::call0`, `PyModule::call1` and `PyModule::get`. [#1492](https://github.com/PyO3/pyo3/pull/1492) - Add length information to `PyBufferError`s raised from `PyBuffer::copy_to_slice` and `PyBuffer::copy_from_slice`. [#1534](https://github.com/PyO3/pyo3/pull/1534) - Automatically set `-undefined` and `dynamic_lookup` linker arguments on macOS with the `extension-module` feature. [#1539](https://github.com/PyO3/pyo3/pull/1539) - Deprecate `#[pyproto]` methods which are easier to implement as `#[pymethods]`: [#1560](https://github.com/PyO3/pyo3/pull/1560) - `PyBasicProtocol::__bytes__` and `PyBasicProtocol::__format__` - `PyContextProtocol::__enter__` and `PyContextProtocol::__exit__` - `PyDescrProtocol::__delete__` and `PyDescrProtocol::__set_name__` - `PyMappingProtocol::__reversed__` - `PyNumberProtocol::__complex__` and `PyNumberProtocol::__round__` - `PyAsyncProtocol::__aenter__` and `PyAsyncProtocol::__aexit__` - Deprecate several attributes in favor of the new `#[pyo3(...)]` options: - `#[name = "..."]`, replaced by `#[pyo3(name = "...")]` [#1567](https://github.com/PyO3/pyo3/pull/1567) - `#[pyfn(m, "name")]`, replaced by `#[pyfn(m)] #[pyo3(name = "...")]`. [#1610](https://github.com/PyO3/pyo3/pull/1610) - `#[pymodule(name)]`, replaced by `#[pymodule] #[pyo3(name = "...")]` [#1650](https://github.com/PyO3/pyo3/pull/1650) - `#[text_signature = "..."]`, replaced by `#[pyo3(text_signature = "...")]`. [#1658](https://github.com/PyO3/pyo3/pull/1658) - Reduce LLVM line counts to improve compilation times. [#1604](https://github.com/PyO3/pyo3/pull/1604) - No longer call `PyEval_InitThreads` in `#[pymodule]` init code. [#1630](https://github.com/PyO3/pyo3/pull/1630) - Use `METH_FASTCALL` argument passing convention, when possible, to improve `#[pyfunction]` and method performance. [#1619](https://github.com/PyO3/pyo3/pull/1619), [#1660](https://github.com/PyO3/pyo3/pull/1660) - Filter sysconfigdata candidates by architecture when cross-compiling. [#1626](https://github.com/PyO3/pyo3/pull/1626) ### Removed - Remove deprecated exception names `BaseException` etc. [#1426](https://github.com/PyO3/pyo3/pull/1426) - Remove deprecated methods `Python::is_instance`, `Python::is_subclass`, `Python::release`, `Python::xdecref`, and `Py::from_owned_ptr_or_panic`. [#1426](https://github.com/PyO3/pyo3/pull/1426) - Remove many FFI definitions which never existed in the Python C-API: - (previously deprecated) `PyGetSetDef_INIT`, `PyGetSetDef_DICT`, `PyCoro_Check`, `PyCoroWrapper_Check`, and `PyAsyncGen_Check` [#1426](https://github.com/PyO3/pyo3/pull/1426) - `PyMethodDef_INIT` [#1426](https://github.com/PyO3/pyo3/pull/1426) - `PyTypeObject_INIT` [#1429](https://github.com/PyO3/pyo3/pull/1429) - `PyObject_Check`, `PySuper_Check`, and `FreeFunc` [#1438](https://github.com/PyO3/pyo3/pull/1438) - `PyModuleDef_INIT` [#1630](https://github.com/PyO3/pyo3/pull/1630) - Remove pyclass implementation details from `PyTypeInfo`: - `Type`, `DESCRIPTION`, and `FLAGS` [#1456](https://github.com/PyO3/pyo3/pull/1456) - `BaseType`, `BaseLayout`, `Layout`, `Initializer` [#1596](https://github.com/PyO3/pyo3/pull/1596) - Remove `PYO3_CROSS_INCLUDE_DIR` environment variable and the associated C header parsing functionality. [#1521](https://github.com/PyO3/pyo3/pull/1521) - Remove `raw_pycfunction!` macro. [#1619](https://github.com/PyO3/pyo3/pull/1619) - Remove `PyClassAlloc` trait. [#1657](https://github.com/PyO3/pyo3/pull/1657) - Remove `PyList::get_parked_item`. [#1664](https://github.com/PyO3/pyo3/pull/1664) ### Fixed - Remove FFI definition `PyCFunction_ClearFreeList` for Python 3.9 and later. [#1425](https://github.com/PyO3/pyo3/pull/1425) - `PYO3_CROSS_LIB_DIR` environment variable no long required when compiling for x86-64 Python from macOS arm64 and reverse. [#1428](https://github.com/PyO3/pyo3/pull/1428) - Fix FFI definition `_PyEval_RequestCodeExtraIndex`, which took an argument of the wrong type. [#1429](https://github.com/PyO3/pyo3/pull/1429) - Fix FFI definition `PyIndex_Check` missing with the `abi3` feature. [#1436](https://github.com/PyO3/pyo3/pull/1436) - Fix incorrect `TypeError` raised when keyword-only argument passed along with a positional argument in `*args`. [#1440](https://github.com/PyO3/pyo3/pull/1440) - Fix inability to use a named lifetime for `&PyTuple` of `*args` in `#[pyfunction]`. [#1440](https://github.com/PyO3/pyo3/pull/1440) - Fix use of Python argument for `#[pymethods]` inside macro expansions. [#1505](https://github.com/PyO3/pyo3/pull/1505) - No longer include `__doc__` in `__all__` generated for `#[pymodule]`. [#1509](https://github.com/PyO3/pyo3/pull/1509) - Always use cross-compiling configuration if any of the `PYO3_CROSS` family of environment variables are set. [#1514](https://github.com/PyO3/pyo3/pull/1514) - Support `EnvironmentError`, `IOError`, and `WindowsError` on PyPy. [#1533](https://github.com/PyO3/pyo3/pull/1533) - Fix unnecessary rebuilds when cycling between `cargo check` and `cargo clippy` in a Python virtualenv. [#1557](https://github.com/PyO3/pyo3/pull/1557) - Fix segfault when dereferencing `ffi::PyDateTimeAPI` without the GIL. [#1563](https://github.com/PyO3/pyo3/pull/1563) - Fix memory leak in `FromPyObject` implementations for `u128` and `i128`. [#1638](https://github.com/PyO3/pyo3/pull/1638) - Fix `#[pyclass(extends=PyDict)]` leaking the dict contents on drop. [#1657](https://github.com/PyO3/pyo3/pull/1657) - Fix segfault when calling `PyList::get_item` with negative indices. [#1668](https://github.com/PyO3/pyo3/pull/1668) - Fix FFI definitions of `PyEval_SetProfile`/`PyEval_SetTrace` to take `Option` parameters. [#1692](https://github.com/PyO3/pyo3/pull/1692) - Fix `ToPyObject` impl for `HashSet` to accept non-default hashers. [#1702](https://github.com/PyO3/pyo3/pull/1702) ## [0.13.2] - 2021-02-12 ### Packaging - Lower minimum supported Rust version to 1.41. [#1421](https://github.com/PyO3/pyo3/pull/1421) ### Added - Add unsafe API `with_embedded_python_interpreter` to initialize a Python interpreter, execute a closure, and finalize the interpreter. [#1355](https://github.com/PyO3/pyo3/pull/1355) - Add `serde` feature which provides implementations of `Serialize` and `Deserialize` for `Py`. [#1366](https://github.com/PyO3/pyo3/pull/1366) - Add FFI definition `_PyCFunctionFastWithKeywords` on Python 3.7 and up. [#1384](https://github.com/PyO3/pyo3/pull/1384) - Add `PyDateTime::new_with_fold` method. [#1398](https://github.com/PyO3/pyo3/pull/1398) - Add `size_hint` impls for `{PyDict,PyList,PySet,PyTuple}Iterator`s. [#1699](https://github.com/PyO3/pyo3/pull/1699) ### Changed - `prepare_freethreaded_python` will no longer register an `atexit` handler to call `Py_Finalize`. This resolves a number of issues with incompatible C extensions causing crashes at finalization. [#1355](https://github.com/PyO3/pyo3/pull/1355) - Mark `PyLayout::py_init`, `PyClassDict::clear_dict`, and `opt_to_pyobj` safe, as they do not perform any unsafe operations. [#1404](https://github.com/PyO3/pyo3/pull/1404) ### Fixed - Fix support for using `r#raw_idents` as argument names in pyfunctions. [#1383](https://github.com/PyO3/pyo3/pull/1383) - Fix typo in FFI definition for `PyFunction_GetCode` (was incorrectly `PyFunction_Code`). [#1387](https://github.com/PyO3/pyo3/pull/1387) - Fix FFI definitions `PyMarshal_WriteObjectToString` and `PyMarshal_ReadObjectFromString` as available in limited API. [#1387](https://github.com/PyO3/pyo3/pull/1387) - Fix FFI definitions `PyListObject` and those from `funcobject.h` as requiring non-limited API. [#1387](https://github.com/PyO3/pyo3/pull/1387) - Fix unqualified `Result` usage in `pyobject_native_type_base`. [#1402](https://github.com/PyO3/pyo3/pull/1402) - Fix build on systems where the default Python encoding is not UTF-8. [#1405](https://github.com/PyO3/pyo3/pull/1405) - Fix build on mingw / MSYS2. [#1423](https://github.com/PyO3/pyo3/pull/1423) ## [0.13.1] - 2021-01-10 ### Added - Add support for `#[pyclass(dict)]` and `#[pyclass(weakref)]` with the `abi3` feature on Python 3.9 and up. [#1342](https://github.com/PyO3/pyo3/pull/1342) - Add FFI definitions `PyOS_BeforeFork`, `PyOS_AfterFork_Parent`, `PyOS_AfterFork_Child` for Python 3.7 and up. [#1348](https://github.com/PyO3/pyo3/pull/1348) - Add an `auto-initialize` feature to control whether PyO3 should automatically initialize an embedded Python interpreter. For compatibility this feature is enabled by default in PyO3 0.13.1, but is planned to become opt-in from PyO3 0.14.0. [#1347](https://github.com/PyO3/pyo3/pull/1347) - Add support for cross-compiling to Windows without needing `PYO3_CROSS_INCLUDE_DIR`. [#1350](https://github.com/PyO3/pyo3/pull/1350) ### Deprecated - Deprecate FFI definitions `PyEval_CallObjectWithKeywords`, `PyEval_CallObject`, `PyEval_CallFunction`, `PyEval_CallMethod` when building for Python 3.9. [#1338](https://github.com/PyO3/pyo3/pull/1338) - Deprecate FFI definitions `PyGetSetDef_DICT` and `PyGetSetDef_INIT` which have never been in the Python API. [#1341](https://github.com/PyO3/pyo3/pull/1341) - Deprecate FFI definitions `PyGen_NeedsFinalizing`, `PyImport_Cleanup` (removed in 3.9), and `PyOS_InitInterrupts` (3.10). [#1348](https://github.com/PyO3/pyo3/pull/1348) - Deprecate FFI definition `PyOS_AfterFork` for Python 3.7 and up. [#1348](https://github.com/PyO3/pyo3/pull/1348) - Deprecate FFI definitions `PyCoro_Check`, `PyAsyncGen_Check`, and `PyCoroWrapper_Check`, which have never been in the Python API (for the first two, it is possible to use `PyCoro_CheckExact` and `PyAsyncGen_CheckExact` instead; these are the actual functions provided by the Python API). [#1348](https://github.com/PyO3/pyo3/pull/1348) - Deprecate FFI definitions for `PyUnicode_FromUnicode`, `PyUnicode_AsUnicode` and `PyUnicode_AsUnicodeAndSize`, which will be removed from 3.12 and up due to [PEP 623](https://www.python.org/dev/peps/pep-0623/). [#1370](https://github.com/PyO3/pyo3/pull/1370) ### Removed - Remove FFI definition `PyFrame_ClearFreeList` when building for Python 3.9. [#1341](https://github.com/PyO3/pyo3/pull/1341) - Remove FFI definition `_PyDict_Contains` when building for Python 3.10. [#1341](https://github.com/PyO3/pyo3/pull/1341) - Remove FFI definitions `PyGen_NeedsFinalizing` and `PyImport_Cleanup` (for 3.9 and up), and `PyOS_InitInterrupts` (3.10). [#1348](https://github.com/PyO3/pyo3/pull/1348) ### Fixed - Stop including `Py_TRACE_REFS` config setting automatically if `Py_DEBUG` is set on Python 3.8 and up. [#1334](https://github.com/PyO3/pyo3/pull/1334) - Remove `#[deny(warnings)]` attribute (and instead refuse warnings only in CI). [#1340](https://github.com/PyO3/pyo3/pull/1340) - Fix deprecation warning for missing `__module__` with `#[pyclass]`. [#1343](https://github.com/PyO3/pyo3/pull/1343) - Correct return type of `PyFrozenSet::empty` to `&PyFrozenSet` (was incorrectly `&PySet`). [#1351](https://github.com/PyO3/pyo3/pull/1351) - Fix missing `Py_INCREF` on heap type objects on Python versions before 3.8. [#1365](https://github.com/PyO3/pyo3/pull/1365) ## [0.13.0] - 2020-12-22 ### Packaging - Drop support for Python 3.5 (as it is now end-of-life). [#1250](https://github.com/PyO3/pyo3/pull/1250) - Bump minimum supported Rust version to 1.45. [#1272](https://github.com/PyO3/pyo3/pull/1272) - Bump indoc dependency to 1.0. [#1272](https://github.com/PyO3/pyo3/pull/1272) - Bump paste dependency to 1.0. [#1272](https://github.com/PyO3/pyo3/pull/1272) - Rename internal crates `pyo3cls` and `pyo3-derive-backend` to `pyo3-macros` and `pyo3-macros-backend` respectively. [#1317](https://github.com/PyO3/pyo3/pull/1317) ### Added - Add support for building for CPython limited API. Opting-in to the limited API enables a single extension wheel built with PyO3 to be installable on multiple Python versions. This required a few minor changes to runtime behavior of of PyO3 `#[pyclass]` types. See the migration guide for full details. [#1152](https://github.com/PyO3/pyo3/pull/1152) - Add feature flags `abi3-py36`, `abi3-py37`, `abi3-py38` etc. to set the minimum Python version when using the limited API. [#1263](https://github.com/PyO3/pyo3/pull/1263) - Add argument names to `TypeError` messages generated by pymethod wrappers. [#1212](https://github.com/PyO3/pyo3/pull/1212) - Add FFI definitions for PEP 587 "Python Initialization Configuration". [#1247](https://github.com/PyO3/pyo3/pull/1247) - Add FFI definitions for `PyEval_SetProfile` and `PyEval_SetTrace`. [#1255](https://github.com/PyO3/pyo3/pull/1255) - Add FFI definitions for context.h functions (`PyContext_New`, etc). [#1259](https://github.com/PyO3/pyo3/pull/1259) - Add `PyAny::is_instance` method. [#1276](https://github.com/PyO3/pyo3/pull/1276) - Add support for conversion between `char` and `PyString`. [#1282](https://github.com/PyO3/pyo3/pull/1282) - Add FFI definitions for `PyBuffer_SizeFromFormat`, `PyObject_LengthHint`, `PyObject_CallNoArgs`, `PyObject_CallOneArg`, `PyObject_CallMethodNoArgs`, `PyObject_CallMethodOneArg`, `PyObject_VectorcallDict`, and `PyObject_VectorcallMethod`. [#1287](https://github.com/PyO3/pyo3/pull/1287) - Add conversions between `u128`/`i128` and `PyLong` for PyPy. [#1310](https://github.com/PyO3/pyo3/pull/1310) - Add `Python::version` and `Python::version_info` to get the running interpreter version. [#1322](https://github.com/PyO3/pyo3/pull/1322) - Add conversions for tuples of length 10, 11, and 12. [#1454](https://github.com/PyO3/pyo3/pull/1454) ### Changed - Change return type of `PyType::name` from `Cow` to `PyResult<&str>`. [#1152](https://github.com/PyO3/pyo3/pull/1152) - `#[pyclass(subclass)]` is now required for subclassing from Rust (was previously just required for subclassing from Python). [#1152](https://github.com/PyO3/pyo3/pull/1152) - Change `PyIterator` to be consistent with other native types: it is now used as `&PyIterator` instead of `PyIterator<'a>`. [#1176](https://github.com/PyO3/pyo3/pull/1176) - Change formatting of `PyDowncastError` messages to be closer to Python's builtin error messages. [#1212](https://github.com/PyO3/pyo3/pull/1212) - Change `Debug` and `Display` impls for `PyException` to be consistent with `PyAny`. [#1275](https://github.com/PyO3/pyo3/pull/1275) - Change `Debug` impl of `PyErr` to output more helpful information (acquiring the GIL if necessary). [#1275](https://github.com/PyO3/pyo3/pull/1275) - Rename `PyTypeInfo::is_instance` and `PyTypeInfo::is_exact_instance` to `PyTypeInfo::is_type_of` and `PyTypeInfo::is_exact_type_of`. [#1278](https://github.com/PyO3/pyo3/pull/1278) - Optimize `PyAny::call0`, `Py::call0` and `PyAny::call_method0` and `Py::call_method0` on Python 3.9 and up. [#1287](https://github.com/PyO3/pyo3/pull/1285) - Require double-quotes for pyclass name argument e.g `#[pyclass(name = "MyClass")]`. [#1303](https://github.com/PyO3/pyo3/pull/1303) ### Deprecated - Deprecate `Python::is_instance`, `Python::is_subclass`, `Python::release`, and `Python::xdecref`. [#1292](https://github.com/PyO3/pyo3/pull/1292) ### Removed - Remove deprecated ffi definitions `PyUnicode_AsUnicodeCopy`, `PyUnicode_GetMax`, `_Py_CheckRecursionLimit`, `PyObject_AsCharBuffer`, `PyObject_AsReadBuffer`, `PyObject_CheckReadBuffer` and `PyObject_AsWriteBuffer`, which will be removed in Python 3.10. [#1217](https://github.com/PyO3/pyo3/pull/1217) - Remove unused `python3` feature. [#1235](https://github.com/PyO3/pyo3/pull/1235) ### Fixed - Fix missing field in `PyCodeObject` struct (`co_posonlyargcount`) - caused invalid access to other fields in Python >3.7. [#1260](https://github.com/PyO3/pyo3/pull/1260) - Fix building for `x86_64-unknown-linux-musl` target from `x86_64-unknown-linux-gnu` host. [#1267](https://github.com/PyO3/pyo3/pull/1267) - Fix `#[text_signature]` interacting badly with rust `r#raw_identifiers`. [#1286](https://github.com/PyO3/pyo3/pull/1286) - Fix FFI definitions for `PyObject_Vectorcall` and `PyVectorcall_Call`. [#1287](https://github.com/PyO3/pyo3/pull/1285) - Fix building with Anaconda python inside a virtualenv. [#1290](https://github.com/PyO3/pyo3/pull/1290) - Fix definition of opaque FFI types. [#1312](https://github.com/PyO3/pyo3/pull/1312) - Fix using custom error type in pyclass `#[new]` methods. [#1319](https://github.com/PyO3/pyo3/pull/1319) ## [0.12.4] - 2020-11-28 ### Fixed - Fix reference count bug in implementation of `From>` for `PyObject`, a regression introduced in PyO3 0.12. [#1297](https://github.com/PyO3/pyo3/pull/1297) ## [0.12.3] - 2020-10-12 ### Fixed - Fix support for Rust versions 1.39 to 1.44, broken by an incorrect internal update to paste 1.0 which was done in PyO3 0.12.2. [#1234](https://github.com/PyO3/pyo3/pull/1234) ## [0.12.2] - 2020-10-12 ### Added - Add support for keyword-only arguments without default values in `#[pyfunction]`. [#1209](https://github.com/PyO3/pyo3/pull/1209) - Add `Python::check_signals` as a safe a wrapper for `PyErr_CheckSignals`. [#1214](https://github.com/PyO3/pyo3/pull/1214) ### Fixed - Fix invalid document for protocol methods. [#1169](https://github.com/PyO3/pyo3/pull/1169) - Hide docs of PyO3 private implementation details in `pyo3::class::methods`. [#1169](https://github.com/PyO3/pyo3/pull/1169) - Fix unnecessary rebuild on PATH changes when the python interpreter is provided by PYO3_PYTHON. [#1231](https://github.com/PyO3/pyo3/pull/1231) ## [0.12.1] - 2020-09-16 ### Fixed - Fix building for a 32-bit Python on 64-bit Windows with a 64-bit Rust toolchain. [#1179](https://github.com/PyO3/pyo3/pull/1179) - Fix building on platforms where `c_char` is `u8`. [#1182](https://github.com/PyO3/pyo3/pull/1182) ## [0.12.0] - 2020-09-12 ### Added - Add FFI definitions `Py_FinalizeEx`, `PyOS_getsig`, and `PyOS_setsig`. [#1021](https://github.com/PyO3/pyo3/pull/1021) - Add `PyString::to_str` for accessing `PyString` as `&str`. [#1023](https://github.com/PyO3/pyo3/pull/1023) - Add `Python::with_gil` for executing a closure with the Python GIL. [#1037](https://github.com/PyO3/pyo3/pull/1037) - Add type information to failures in `PyAny::downcast`. [#1050](https://github.com/PyO3/pyo3/pull/1050) - Implement `Debug` for `PyIterator`. [#1051](https://github.com/PyO3/pyo3/pull/1051) - Add `PyBytes::new_with` and `PyByteArray::new_with` for initialising `bytes` and `bytearray` objects using a closure. [#1074](https://github.com/PyO3/pyo3/pull/1074) - Add `#[derive(FromPyObject)]` macro for enums and structs. [#1065](https://github.com/PyO3/pyo3/pull/1065) - Add `Py::as_ref` and `Py::into_ref` for converting `Py` to `&T`. [#1098](https://github.com/PyO3/pyo3/pull/1098) - Add ability to return `Result` types other than `PyResult` from `#[pyfunction]`, `#[pymethod]` and `#[pyproto]` functions. [#1106](https://github.com/PyO3/pyo3/pull/1118). - Implement `ToPyObject`, `IntoPy`, and `FromPyObject` for [hashbrown](https://crates.io/crates/hashbrown)'s `HashMap` and `HashSet` types (requires the `hashbrown` feature). [#1114](https://github.com/PyO3/pyo3/pull/1114) - Add `#[pyfunction(pass_module)]` and `#[pyfn(pass_module)]` to pass the module object as the first function argument. [#1143](https://github.com/PyO3/pyo3/pull/1143) - Add `PyModule::add_function` and `PyModule::add_submodule` as typed alternatives to `PyModule::add_wrapped`. [#1143](https://github.com/PyO3/pyo3/pull/1143) - Add native `PyCFunction` and `PyFunction` types. [#1163](https://github.com/PyO3/pyo3/pull/1163) ### Changed - Rework exception types: [#1024](https://github.com/PyO3/pyo3/pull/1024) [#1115](https://github.com/PyO3/pyo3/pull/1115) - Rename exception types from e.g. `RuntimeError` to `PyRuntimeError`. The old names continue to exist but are deprecated. - Exception objects are now accessible as `&T` or `Py`, just like other Python-native types. - Rename `PyException::py_err` to `PyException::new_err`. - Rename `PyUnicodeDecodeErr::new_err` to `PyUnicodeDecodeErr::new`. - Remove `PyStopIteration::stop_iteration`. - Require `T: Send` for the return value `T` of `Python::allow_threads`. [#1036](https://github.com/PyO3/pyo3/pull/1036) - Rename `PYTHON_SYS_EXECUTABLE` to `PYO3_PYTHON`. The old name will continue to work (undocumented) but will be removed in a future release. [#1039](https://github.com/PyO3/pyo3/pull/1039) - Remove `unsafe` from signature of `PyType::as_type_ptr`. [#1047](https://github.com/PyO3/pyo3/pull/1047) - Change return type of `PyIterator::from_object` to `PyResult` (was `Result`). [#1051](https://github.com/PyO3/pyo3/pull/1051) - `IntoPy` is no longer implied by `FromPy`. [#1063](https://github.com/PyO3/pyo3/pull/1063) - Change `PyObject` to be a type alias for `Py`. [#1063](https://github.com/PyO3/pyo3/pull/1063) - Rework `PyErr` to be compatible with the `std::error::Error` trait: [#1067](https://github.com/PyO3/pyo3/pull/1067) [#1115](https://github.com/PyO3/pyo3/pull/1115) - Implement `Display`, `Error`, `Send` and `Sync` for `PyErr` and `PyErrArguments`. - Add `PyErr::instance` for accessing `PyErr` as `&PyBaseException`. - `PyErr`'s fields are now an implementation detail. The equivalent values can be accessed with `PyErr::ptype`, `PyErr::pvalue` and `PyErr::ptraceback`. - Change receiver of `PyErr::print` and `PyErr::print_and_set_sys_last_vars` to `&self` (was `self`). - Remove `PyErrValue`, `PyErr::from_value`, `PyErr::into_normalized`, and `PyErr::normalize`. - Remove `PyException::into`. - Remove `Into>` for `PyErr` and `PyException`. - Change methods generated by `#[pyproto]` to return `NotImplemented` if Python should try a reversed operation. #[1072](https://github.com/PyO3/pyo3/pull/1072) - Change argument to `PyModule::add` to `impl IntoPy` (was `impl ToPyObject`). #[1124](https://github.com/PyO3/pyo3/pull/1124) ### Removed - Remove many exception and `PyErr` APIs; see the "changed" section above. [#1024](https://github.com/PyO3/pyo3/pull/1024) [#1067](https://github.com/PyO3/pyo3/pull/1067) [#1115](https://github.com/PyO3/pyo3/pull/1115) - Remove `PyString::to_string` (use new `PyString::to_str`). [#1023](https://github.com/PyO3/pyo3/pull/1023) - Remove `PyString::as_bytes`. [#1023](https://github.com/PyO3/pyo3/pull/1023) - Remove `Python::register_any`. [#1023](https://github.com/PyO3/pyo3/pull/1023) - Remove `GILGuard::acquire` from the public API. Use `Python::acquire_gil` or `Python::with_gil`. [#1036](https://github.com/PyO3/pyo3/pull/1036) - Remove the `FromPy` trait. [#1063](https://github.com/PyO3/pyo3/pull/1063) - Remove the `AsPyRef` trait. [#1098](https://github.com/PyO3/pyo3/pull/1098) ### Fixed - Correct FFI definitions `Py_SetProgramName` and `Py_SetPythonHome` to take `*const` arguments (was `*mut`). [#1021](https://github.com/PyO3/pyo3/pull/1021) - Fix `FromPyObject` for `num_bigint::BigInt` for Python objects with an `__index__` method. [#1027](https://github.com/PyO3/pyo3/pull/1027) - Correct FFI definition `_PyLong_AsByteArray` to take `*mut c_uchar` argument (was `*const c_uchar`). [#1029](https://github.com/PyO3/pyo3/pull/1029) - Fix segfault with `#[pyclass(dict, unsendable)]`. [#1058](https://github.com/PyO3/pyo3/pull/1058) [#1059](https://github.com/PyO3/pyo3/pull/1059) - Fix using `&Self` as an argument type for functions in a `#[pymethods]` block. [#1071](https://github.com/PyO3/pyo3/pull/1071) - Fix best-effort build against PyPy 3.6. [#1092](https://github.com/PyO3/pyo3/pull/1092) - Fix many cases of lifetime elision in `#[pyproto]` implementations. [#1093](https://github.com/PyO3/pyo3/pull/1093) - Fix detection of Python build configuration when cross-compiling. [#1095](https://github.com/PyO3/pyo3/pull/1095) - Always link against libpython on android with the `extension-module` feature. [#1095](https://github.com/PyO3/pyo3/pull/1095) - Fix the `+` operator not trying `__radd__` when both `__add__` and `__radd__` are defined in `PyNumberProtocol` (and similar for all other reversible operators). [#1107](https://github.com/PyO3/pyo3/pull/1107) - Fix building with Anaconda python. [#1175](https://github.com/PyO3/pyo3/pull/1175) ## [0.11.1] - 2020-06-30 ### Added - `#[pyclass(unsendable)]`. [#1009](https://github.com/PyO3/pyo3/pull/1009) ### Changed - Update `parking_lot` dependency to `0.11`. [#1010](https://github.com/PyO3/pyo3/pull/1010) ## [0.11.0] - 2020-06-28 ### Added - Support stable versions of Rust (>=1.39). [#969](https://github.com/PyO3/pyo3/pull/969) - Add FFI definition `PyObject_AsFileDescriptor`. [#938](https://github.com/PyO3/pyo3/pull/938) - Add `PyByteArray::data`, `PyByteArray::as_bytes`, and `PyByteArray::as_bytes_mut`. [#967](https://github.com/PyO3/pyo3/pull/967) - Add `GILOnceCell` to use in situations where `lazy_static` or `once_cell` can deadlock. [#975](https://github.com/PyO3/pyo3/pull/975) - Add `Py::borrow`, `Py::borrow_mut`, `Py::try_borrow`, and `Py::try_borrow_mut` for accessing `#[pyclass]` values. [#976](https://github.com/PyO3/pyo3/pull/976) - Add `IterNextOutput` and `IterANextOutput` for returning from `__next__` / `__anext__`. [#997](https://github.com/PyO3/pyo3/pull/997) ### Changed - Simplify internals of `#[pyo3(get)]` attribute. (Remove the hidden API `GetPropertyValue`.) [#934](https://github.com/PyO3/pyo3/pull/934) - Call `Py_Finalize` at exit to flush buffers, etc. [#943](https://github.com/PyO3/pyo3/pull/943) - Add type parameter to PyBuffer. #[951](https://github.com/PyO3/pyo3/pull/951) - Require `Send` bound for `#[pyclass]`. [#966](https://github.com/PyO3/pyo3/pull/966) - Add `Python` argument to most methods on `PyObject` and `Py` to ensure GIL safety. [#970](https://github.com/PyO3/pyo3/pull/970) - Change signature of `PyTypeObject::type_object` - now takes `Python` argument and returns `&PyType`. [#970](https://github.com/PyO3/pyo3/pull/970) - Change return type of `PyTuple::slice` and `PyTuple::split_from` from `Py` to `&PyTuple`. [#970](https://github.com/PyO3/pyo3/pull/970) - Change return type of `PyTuple::as_slice` to `&[&PyAny]`. [#971](https://github.com/PyO3/pyo3/pull/971) - Rename `PyTypeInfo::type_object` to `type_object_raw`, and add `Python` argument. [#975](https://github.com/PyO3/pyo3/pull/975) - Update `num-complex` optional dependendency from `0.2` to `0.3`. [#977](https://github.com/PyO3/pyo3/pull/977) - Update `num-bigint` optional dependendency from `0.2` to `0.3`. [#978](https://github.com/PyO3/pyo3/pull/978) - `#[pyproto]` is re-implemented without specialization. [#961](https://github.com/PyO3/pyo3/pull/961) - `PyClassAlloc::alloc` is renamed to `PyClassAlloc::new`. [#990](https://github.com/PyO3/pyo3/pull/990) - `#[pyproto]` methods can now have return value `T` or `PyResult` (previously only `PyResult` was supported). [#996](https://github.com/PyO3/pyo3/pull/996) - `#[pyproto]` methods can now skip annotating the return type if it is `()`. [#998](https://github.com/PyO3/pyo3/pull/998) ### Removed - Remove `ManagedPyRef` (unused, and needs specialization) [#930](https://github.com/PyO3/pyo3/pull/930) ### Fixed - Fix passing explicit `None` to `Option` argument `#[pyfunction]` with a default value. [#936](https://github.com/PyO3/pyo3/pull/936) - Fix `PyClass.__new__`'s not respecting subclasses when inherited by a Python class. [#990](https://github.com/PyO3/pyo3/pull/990) - Fix returning `Option` from `#[pyproto]` methods. [#996](https://github.com/PyO3/pyo3/pull/996) - Fix accepting `PyRef` and `PyRefMut` to `#[getter]` and `#[setter]` methods. [#999](https://github.com/PyO3/pyo3/pull/999) ## [0.10.1] - 2020-05-14 ### Fixed - Fix deadlock in `Python::acquire_gil` after dropping a `PyObject` or `Py`. [#924](https://github.com/PyO3/pyo3/pull/924) ## [0.10.0] - 2020-05-13 ### Added - Add FFI definition `_PyDict_NewPresized`. [#849](https://github.com/PyO3/pyo3/pull/849) - Implement `IntoPy` for `HashSet` and `BTreeSet`. [#864](https://github.com/PyO3/pyo3/pull/864) - Add `PyAny::dir` method. [#886](https://github.com/PyO3/pyo3/pull/886) - Gate macros behind a `macros` feature (enabled by default). [#897](https://github.com/PyO3/pyo3/pull/897) - Add ability to define class attributes using `#[classattr]` on functions in `#[pymethods]`. [#905](https://github.com/PyO3/pyo3/pull/905) - Implement `Clone` for `PyObject` and `Py`. [#908](https://github.com/PyO3/pyo3/pull/908) - Implement `Deref` for all builtin types. (`PyList`, `PyTuple`, `PyDict` etc.) [#911](https://github.com/PyO3/pyo3/pull/911) - Implement `Deref` for `PyCell`. [#911](https://github.com/PyO3/pyo3/pull/911) - Add `#[classattr]` support for associated constants in `#[pymethods]`. [#914](https://github.com/PyO3/pyo3/pull/914) ### Changed - Panics will now be raised as a Python `PanicException`. [#797](https://github.com/PyO3/pyo3/pull/797) - Change `PyObject` and `Py` reference counts to decrement immediately upon drop when the GIL is held. [#851](https://github.com/PyO3/pyo3/pull/851) - Allow `PyIterProtocol` methods to use either `PyRef` or `PyRefMut` as the receiver type. [#856](https://github.com/PyO3/pyo3/pull/856) - Change the implementation of `FromPyObject` for `Py` to apply to a wider range of `T`, including all `T: PyClass`. [#880](https://github.com/PyO3/pyo3/pull/880) - Move all methods from the `ObjectProtocol` trait to the `PyAny` struct. [#911](https://github.com/PyO3/pyo3/pull/911) - Remove need for `#![feature(specialization)]` in crates depending on PyO3. [#917](https://github.com/PyO3/pyo3/pull/917) ### Removed - Remove `PyMethodsProtocol` trait. [#889](https://github.com/PyO3/pyo3/pull/889) - Remove `num-traits` dependency. [#895](https://github.com/PyO3/pyo3/pull/895) - Remove `ObjectProtocol` trait. [#911](https://github.com/PyO3/pyo3/pull/911) - Remove `PyAny::None`. Users should use `Python::None` instead. [#911](https://github.com/PyO3/pyo3/pull/911) - Remove all `*ProtocolImpl` traits. [#917](https://github.com/PyO3/pyo3/pull/917) ### Fixed - Fix support for `__radd__` and other `__r*__` methods as implementations for Python mathematical operators. [#839](https://github.com/PyO3/pyo3/pull/839) - Fix panics during garbage collection when traversing objects that were already mutably borrowed. [#855](https://github.com/PyO3/pyo3/pull/855) - Prevent `&'static` references to Python objects as arguments to `#[pyfunction]` and `#[pymethods]`. [#869](https://github.com/PyO3/pyo3/pull/869) - Fix lifetime safety bug with `AsPyRef::as_ref`. [#876](https://github.com/PyO3/pyo3/pull/876) - Fix `#[pyo3(get)]` attribute on `Py` fields. [#880](https://github.com/PyO3/pyo3/pull/880) - Fix segmentation faults caused by functions such as `PyList::get_item` returning borrowed objects when it was not safe to do so. [#890](https://github.com/PyO3/pyo3/pull/890) - Fix segmentation faults caused by nested `Python::acquire_gil` calls creating dangling references. [#893](https://github.com/PyO3/pyo3/pull/893) - Fix segmentatation faults when a panic occurs during a call to `Python::allow_threads`. [#912](https://github.com/PyO3/pyo3/pull/912) ## [0.9.2] - 2020-04-09 ### Added - `FromPyObject` implementations for `HashSet` and `BTreeSet`. [#842](https://github.com/PyO3/pyo3/pull/842) ### Fixed - Correctly detect 32bit architecture. [#830](https://github.com/PyO3/pyo3/pull/830) ## [0.9.1] - 2020-03-23 ### Fixed - Error messages for `#[pyclass]`. [#826](https://github.com/PyO3/pyo3/pull/826) - `FromPyObject` implementation for `PySequence`. [#827](https://github.com/PyO3/pyo3/pull/827) ## [0.9.0] - 2020-03-19 ### Added - `PyCell`, which has RefCell-like features. [#770](https://github.com/PyO3/pyo3/pull/770) - `PyClass`, `PyLayout`, `PyClassInitializer`. [#683](https://github.com/PyO3/pyo3/pull/683) - Implemented `IntoIterator` for `PySet` and `PyFrozenSet`. [#716](https://github.com/PyO3/pyo3/pull/716) - `FromPyObject` is now automatically implemented for `T: Clone` pyclasses. [#730](https://github.com/PyO3/pyo3/pull/730) - `#[pyo3(get)]` and `#[pyo3(set)]` will now use the Rust doc-comment from the field for the Python property. [#755](https://github.com/PyO3/pyo3/pull/755) - `#[setter]` functions may now take an argument of `Pyo3::Python`. [#760](https://github.com/PyO3/pyo3/pull/760) - `PyTypeInfo::BaseLayout` and `PyClass::BaseNativeType`. [#770](https://github.com/PyO3/pyo3/pull/770) - `PyDowncastImpl`. [#770](https://github.com/PyO3/pyo3/pull/770) - Implement `FromPyObject` and `IntoPy` traits for arrays (up to 32). [#778](https://github.com/PyO3/pyo3/pull/778) - `migration.md` and `types.md` in the guide. [#795](https://github.com/PyO3/pyo3/pull/795), #[802](https://github.com/PyO3/pyo3/pull/802) - `ffi::{_PyBytes_Resize, _PyDict_Next, _PyDict_Contains, _PyDict_GetDictPtr}`. #[820](https://github.com/PyO3/pyo3/pull/820) ### Changed - `#[new]` does not take `PyRawObject` and can return `Self`. [#683](https://github.com/PyO3/pyo3/pull/683) - The blanket implementations for `FromPyObject` for `&T` and `&mut T` are no longer specializable. Implement `PyTryFrom` for your type to control the behavior of `FromPyObject::extract` for your types. [#713](https://github.com/PyO3/pyo3/pull/713) - The implementation for `IntoPy for T` where `U: FromPy` is no longer specializable. Control the behavior of this via the implementation of `FromPy`. [#713](https://github.com/PyO3/pyo3/pull/713) - Use `parking_lot::Mutex` instead of `spin::Mutex`. [#734](https://github.com/PyO3/pyo3/pull/734) - Bumped minimum Rust version to `1.42.0-nightly 2020-01-21`. [#761](https://github.com/PyO3/pyo3/pull/761) - `PyRef` and `PyRefMut` are renewed for `PyCell`. [#770](https://github.com/PyO3/pyo3/pull/770) - Some new FFI functions for Python 3.8. [#784](https://github.com/PyO3/pyo3/pull/784) - `PyAny` is now on the top level module and prelude. [#816](https://github.com/PyO3/pyo3/pull/816) ### Removed - `PyRawObject`. [#683](https://github.com/PyO3/pyo3/pull/683) - `PyNoArgsFunction`. [#741](https://github.com/PyO3/pyo3/pull/741) - `initialize_type`. To set the module name for a `#[pyclass]`, use the `module` argument to the macro. #[751](https://github.com/PyO3/pyo3/pull/751) - `AsPyRef::as_mut/with/with_mut/into_py/into_mut_py`. [#770](https://github.com/PyO3/pyo3/pull/770) - `PyTryFrom::try_from_mut/try_from_mut_exact/try_from_mut_unchecked`. [#770](https://github.com/PyO3/pyo3/pull/770) - `Python::mut_from_owned_ptr/mut_from_borrowed_ptr`. [#770](https://github.com/PyO3/pyo3/pull/770) - `ObjectProtocol::get_base/get_mut_base`. [#770](https://github.com/PyO3/pyo3/pull/770) ### Fixed - Fixed unsoundness of subclassing. [#683](https://github.com/PyO3/pyo3/pull/683). - Clear error indicator when the exception is handled on the Rust side. [#719](https://github.com/PyO3/pyo3/pull/719) - Usage of raw identifiers with `#[pyo3(set)]`. [#745](https://github.com/PyO3/pyo3/pull/745) - Usage of `PyObject` with `#[pyo3(get)]`. [#760](https://github.com/PyO3/pyo3/pull/760) - `#[pymethods]` used in conjunction with `#[cfg]`. #[769](https://github.com/PyO3/pyo3/pull/769) - `"*"` in a `#[pyfunction()]` argument list incorrectly accepting any number of positional arguments (use `args = "*"` when this behavior is desired). #[792](https://github.com/PyO3/pyo3/pull/792) - `PyModule::dict`. #[809](https://github.com/PyO3/pyo3/pull/809) - Fix the case where `DESCRIPTION` is not null-terminated. #[822](https://github.com/PyO3/pyo3/pull/822) ## [0.8.5] - 2020-01-05 ### Added - Implemented `FromPyObject` for `HashMap` and `BTreeMap` - Support for `#[name = "foo"]` attribute for `#[pyfunction]` and in `#[pymethods]`. [#692](https://github.com/PyO3/pyo3/pull/692) ## [0.8.4] - 2019-12-14 ### Added - Support for `#[text_signature]` attribute. [#675](https://github.com/PyO3/pyo3/pull/675) ## [0.8.3] - 2019-11-23 ### Removed - `#[init]` is removed. [#658](https://github.com/PyO3/pyo3/pull/658) ### Fixed - Now all `&Py~` types have `!Send` bound. [#655](https://github.com/PyO3/pyo3/pull/655) - Fix a compile error raised by the stabilization of `!` type. [#672](https://github.com/PyO3/pyo3/issues/672). ## [0.8.2] - 2019-10-27 ### Added - FFI compatibility for PEP 590 Vectorcall. [#641](https://github.com/PyO3/pyo3/pull/641) ### Fixed - Fix PySequenceProtocol::set_item. [#624](https://github.com/PyO3/pyo3/pull/624) - Fix a corner case of BigInt::FromPyObject. [#630](https://github.com/PyO3/pyo3/pull/630) - Fix index errors in parameter conversion. [#631](https://github.com/PyO3/pyo3/pull/631) - Fix handling of invalid utf-8 sequences in `PyString::as_bytes`. [#639](https://github.com/PyO3/pyo3/pull/639) and `PyString::to_string_lossy` [#642](https://github.com/PyO3/pyo3/pull/642). - Remove `__contains__` and `__iter__` from PyMappingProtocol. [#644](https://github.com/PyO3/pyo3/pull/644) - Fix proc-macro definition of PySetAttrProtocol. [#645](https://github.com/PyO3/pyo3/pull/645) ## [0.8.1] - 2019-10-08 ### Added - Conversion between [num-bigint](https://github.com/rust-num/num-bigint) and Python int. [#608](https://github.com/PyO3/pyo3/pull/608) ### Fixed - Make sure the right Python interpreter is used in OSX builds. [#604](https://github.com/PyO3/pyo3/pull/604) - Patch specialization being broken by Rust 1.40. [#614](https://github.com/PyO3/pyo3/issues/614) - Fix a segfault around PyErr. [#597](https://github.com/PyO3/pyo3/pull/597) ## [0.8.0] - 2019-09-16 ### Added - `module` argument to `pyclass` macro. [#499](https://github.com/PyO3/pyo3/pull/499) - `py_run!` macro [#512](https://github.com/PyO3/pyo3/pull/512) - Use existing fields and methods before calling custom **getattr**. [#505](https://github.com/PyO3/pyo3/pull/505) - `PyBytes` can now be indexed just like `Vec` - Implement `IntoPy` for `PyRef` and `PyRefMut`. ### Changed - Implementing the Using the `gc` parameter for `pyclass` (e.g. `#[pyclass(gc)]`) without implementing the `class::PyGCProtocol` trait is now a compile-time error. Failing to implement this trait could lead to segfaults. [#532](https://github.com/PyO3/pyo3/pull/532) - `PyByteArray::data` has been replaced with `PyDataArray::to_vec` because returning a `&[u8]` is unsound. (See [this comment](https://github.com/PyO3/pyo3/issues/373#issuecomment-512332696) for a great write-up for why that was unsound) - Replace `mashup` with `paste`. - `GILPool` gained a `Python` marker to prevent it from being misused to release Python objects without the GIL held. ### Removed - `IntoPyObject` was replaced with `IntoPy` - `#[pyclass(subclass)]` is hidden a `unsound-subclass` feature because it's causing segmentation faults. ### Fixed - More readable error message for generics in pyclass [#503](https://github.com/PyO3/pyo3/pull/503) ## [0.7.0] - 2019-05-26 ### Added - PyPy support by omerbenamram in [#393](https://github.com/PyO3/pyo3/pull/393) - Have `PyModule` generate an index of its members (`__all__` list). - Allow `slf: PyRef` for pyclass(#419) - Allow to use lifetime specifiers in `pymethods` - Add `marshal` module. [#460](https://github.com/PyO3/pyo3/pull/460) ### Changed - `Python::run` returns `PyResult<()>` instead of `PyResult<&PyAny>`. - Methods decorated with `#[getter]` and `#[setter]` can now omit wrapping the result type in `PyResult` if they don't raise exceptions. ### Fixed - `type_object::PyTypeObject` has been marked unsafe because breaking the contract `type_object::PyTypeObject::init_type` can lead to UB. - Fixed automatic derive of `PySequenceProtocol` implementation in [#423](https://github.com/PyO3/pyo3/pull/423). - Capitalization & better wording to README.md. - Docstrings of properties is now properly set using the doc of the `#[getter]` method. - Fixed issues with `pymethods` crashing on doc comments containing double quotes. - `PySet::new` and `PyFrozenSet::new` now return `PyResult<&Py[Frozen]Set>`; exceptions are raised if the items are not hashable. - Fixed building using `venv` on Windows. - `PyTuple::new` now returns `&PyTuple` instead of `Py`. - Fixed several issues with argument parsing; notable, the `*args` and `**kwargs` tuple/dict now doesn't contain arguments that are otherwise assigned to parameters. ## [0.6.0] - 2019-03-28 ### Regressions - Currently, [#341](https://github.com/PyO3/pyo3/issues/341) causes `cargo test` to fail with weird linking errors when the `extension-module` feature is activated. For now you can work around this by making the `extension-module` feature optional and running the tests with `cargo test --no-default-features`: ```toml [dependencies.pyo3] version = "0.6.0" [features] extension-module = ["pyo3/extension-module"] default = ["extension-module"] ``` ### Added - Added a `wrap_pymodule!` macro similar to the existing `wrap_pyfunction!` macro. Only available on python 3 - Added support for cross compiling (e.g. to arm v7) by mtp401 in [#327](https://github.com/PyO3/pyo3/pull/327). See the "Cross Compiling" section in the "Building and Distribution" chapter of the guide for more details. - The `PyRef` and `PyRefMut` types, which allow to differentiate between an instance of a rust struct on the rust heap and an instance that is embedded inside a python object. By kngwyu in [#335](https://github.com/PyO3/pyo3/pull/335) - Added `FromPy` and `IntoPy` which are equivalent to `From` and `Into` except that they require a gil token. - Added `ManagedPyRef`, which should eventually replace `ToBorrowedObject`. ### Changed - Renamed `PyObjectRef` to `PyAny` in #388 - Renamed `add_function` to `add_wrapped` as it now also supports modules. - Renamed `#[pymodinit]` to `#[pymodule]` - `py.init(|| value)` becomes `Py::new(value)` - `py.init_ref(|| value)` becomes `PyRef::new(value)` - `py.init_mut(|| value)` becomes `PyRefMut::new(value)`. - `PyRawObject::init` is now infallible, e.g. it returns `()` instead of `PyResult<()>`. - Renamed `py_exception!` to `create_exception!` and refactored the error macros. - Renamed `wrap_function!` to `wrap_pyfunction!` - Renamed `#[prop(get, set)]` to `#[pyo3(get, set)]` - `#[pyfunction]` now supports the same arguments as `#[pyfn()]` - Some macros now emit proper spanned errors instead of panics. - Migrated to the 2018 edition - `crate::types::exceptions` moved to `crate::exceptions` - Replace `IntoPyTuple` with `IntoPy>`. - `IntoPyPointer` and `ToPyPointer` moved into the crate root. - `class::CompareOp` moved into `class::basic::CompareOp` - PyTypeObject is now a direct subtrait PyTypeCreate, removing the old cyclical implementation in [#350](https://github.com/PyO3/pyo3/pull/350) - Add `PyList::{sort, reverse}` by chr1sj0nes in [#357](https://github.com/PyO3/pyo3/pull/357) and [#358](https://github.com/PyO3/pyo3/pull/358) - Renamed the `typeob` module to `type_object` ### Removed - `PyToken` was removed due to unsoundness (See [#94](https://github.com/PyO3/pyo3/issues/94)). - Removed the unnecessary type parameter from `PyObjectAlloc` - `NoArgs`. Just use an empty tuple - `PyObjectWithGIL`. `PyNativeType` is sufficient now that PyToken is removed. ### Fixed - A soudness hole where every instances of a `#[pyclass]` struct was considered to be part of a python object, even though you can create instances that are not part of the python heap. This was fixed through `PyRef` and `PyRefMut`. - Fix kwargs support in [#328](https://github.com/PyO3/pyo3/pull/328). - Add full support for `__dict__` in [#403](https://github.com/PyO3/pyo3/pull/403). ## [0.5.3] - 2019-01-04 ### Fixed - Fix memory leak in ArrayList by kngwyu [#316](https://github.com/PyO3/pyo3/pull/316) ## [0.5.2] - 2018-11-25 ### Fixed - Fix undeterministic segfaults when creating many objects by kngwyu in [#281](https://github.com/PyO3/pyo3/pull/281) ## [0.5.1] - 2018-11-24 Yanked ## [0.5.0] - 2018-11-11 ### Added - `#[pyclass]` objects can now be returned from rust functions - `PyComplex` by kngwyu in [#226](https://github.com/PyO3/pyo3/pull/226) - `PyDict::from_sequence`, equivalent to `dict([(key, val), ...])` - Bindings for the `datetime` standard library types: `PyDate`, `PyTime`, `PyDateTime`, `PyTzInfo`, `PyDelta` with associated `ffi` types, by pganssle [#200](https://github.com/PyO3/pyo3/pull/200). - `PyString`, `PyUnicode`, and `PyBytes` now have an `as_bytes` method that returns `&[u8]`. - `PyObjectProtocol::get_type_ptr` by ijl in [#242](https://github.com/PyO3/pyo3/pull/242) ### Changed - Removes the types from the root module and the prelude. They now live in `pyo3::types` instead. - All exceptions are constructed with `py_err` instead of `new`, as they return `PyErr` and not `Self`. - `as_mut` and friends take and `&mut self` instead of `&self` - `ObjectProtocol::call` now takes an `Option<&PyDict>` for the kwargs instead of an `IntoPyDictPointer`. - `IntoPyDictPointer` was replace by `IntoPyDict` which doesn't convert `PyDict` itself anymore and returns a `PyDict` instead of `*mut PyObject`. - `PyTuple::new` now takes an `IntoIterator` instead of a slice - Updated to syn 0.15 - Splitted `PyTypeObject` into `PyTypeObject` without the create method and `PyTypeCreate` with requires `PyObjectAlloc + PyTypeInfo + Sized`. - Ran `cargo edition --fix` which prefixed path with `crate::` for rust 2018 - Renamed `async` to `pyasync` as async will be a keyword in the 2018 edition. - Starting to use `NonNull<*mut PyObject>` for Py and PyObject by ijl [#260](https://github.com/PyO3/pyo3/pull/260) ### Removed - Removed most entries from the prelude. The new prelude is small and clear. - Slowly removing specialization uses - `PyString`, `PyUnicode`, and `PyBytes` no longer have a `data` method (replaced by `as_bytes`) and `PyStringData` has been removed. - The pyobject_extract macro ### Fixed - Added an explanation that the GIL can temporarily be released even while holding a GILGuard. - Lots of clippy errors - Fix segfault on calling an unknown method on a PyObject - Work around a [bug](https://github.com/rust-lang/rust/issues/55380) in the rust compiler by kngwyu [#252](https://github.com/PyO3/pyo3/pull/252) - Fixed a segfault with subclassing pyo3 create classes and using `__class__` by kngwyu [#263](https://github.com/PyO3/pyo3/pull/263) ## [0.4.1] - 2018-08-20 ### Changed - PyTryFrom's error is always to `PyDowncastError` ### Fixed - Fixed compilation on nightly since `use_extern_macros` was stabilized ### Removed - The pyobject_downcast macro ## [0.4.0] - 2018-07-30 ### Changed - Merged both examples into one - Rustfmt all the things :heavy_check_mark: - Switched to [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ### Removed - Conversions from tuples to PyDict due to [rust-lang/rust#52050](https://github.com/rust-lang/rust/issues/52050) ## [0.3.2] - 2018-07-22 ### Changed - Replaced `concat_idents` with mashup ## [0.3.1] - 2018-07-18 ### Fixed - Fixed scoping bug in pyobject_native_type that would break rust-numpy ## [0.3.0] - 2018-07-18 ### Added - A few internal macros became part of the public api ([#155](https://github.com/PyO3/pyo3/pull/155), [#186](https://github.com/PyO3/pyo3/pull/186)) - Always clone in getters. This allows using the get-annotation on all Clone-Types ### Changed - Upgraded to syn 0.14 which means much better error messages :tada: - 128 bit integer support by [kngwyu](https://github.com/kngwyu) ([#137](https://github.com/PyO3/pyo3/pull/173)) - `proc_macro` has been stabilized on nightly ([rust-lang/rust#52081](https://github.com/rust-lang/rust/pull/52081)). This means that we can remove the `proc_macro` feature, but now we need the `use_extern_macros` from the 2018 edition instead. - All proc macro are now prefixed with `py` and live in the prelude. This means you can use `#[pyclass]`, `#[pymethods]`, `#[pyproto]`, `#[pyfunction]` and `#[pymodinit]` directly, at least after a `use pyo3::prelude::*`. They were also moved into a module called `proc_macro`. You shouldn't use `#[pyo3::proc_macro::pyclass]` or other longer paths in attributes because `proc_macro_path_invoc` isn't going to be stabilized soon. - Renamed the `base` option in the `pyclass` macro to `extends`. - `#[pymodinit]` uses the function name as module name, unless the name is overrriden with `#[pymodinit(name)]` - The guide is now properly versioned. ## [0.2.7] - 2018-05-18 ### Fixed - Fix nightly breakage with proc_macro_path ## [0.2.6] - 2018-04-03 ### Fixed - Fix compatibility with TryFrom trait #137 ## [0.2.5] - 2018-02-21 ### Added - CPython 3.7 support ### Fixed - Embedded CPython 3.7b1 crashes on initialization #110 - Generated extension functions are weakly typed #108 - call_method\* crashes when the method does not exist #113 - Allow importing exceptions from nested modules #116 ## [0.2.4] - 2018-01-19 ### Added - Allow to get mutable ref from PyObject #106 - Drop `RefFromPyObject` trait - Add Python::register_any method ### Fixed - Fix impl `FromPyObject` for `Py` - Mark method that work with raw pointers as unsafe #95 ## [0.2.3] - 11-27-2017 ### Changed - Rustup to 1.23.0-nightly 2017-11-07 ### Fixed - Proper `c_char` usage #93 ### Removed - Remove use of now unneeded 'AsciiExt' trait ## [0.2.2] - 09-26-2017 ### Changed - Rustup to 1.22.0-nightly 2017-09-30 ## [0.2.1] - 09-26-2017 ### Fixed - Fix rustc const_fn nightly breakage ## [0.2.0] - 08-12-2017 ### Added - Added inheritance support #15 - Added weakref support #56 - Added subclass support #64 - Added `self.__dict__` supoort #68 - Added `pyo3::prelude` module #70 - Better `Iterator` support for PyTuple, PyList, PyDict #75 - Introduce IntoPyDictPointer similar to IntoPyTuple #69 ### Changed - Allow to add gc support without implementing PyGCProtocol #57 - Refactor `PyErr` implementation. Drop `py` parameter from constructor. ## [0.1.0] - 07-23-2017 ### Added - Initial release [Unreleased]: https://github.com/pyo3/pyo3/compare/v0.20.2...HEAD [0.20.2]: https://github.com/pyo3/pyo3/compare/v0.20.1...v0.20.2 [0.20.1]: https://github.com/pyo3/pyo3/compare/v0.20.0...v0.20.1 [0.20.0]: https://github.com/pyo3/pyo3/compare/v0.19.2...v0.20.0 [0.19.2]: https://github.com/pyo3/pyo3/compare/v0.19.1...v0.19.2 [0.19.1]: https://github.com/pyo3/pyo3/compare/v0.19.0...v0.19.1 [0.19.0]: https://github.com/pyo3/pyo3/compare/v0.18.3...v0.19.0 [0.18.3]: https://github.com/pyo3/pyo3/compare/v0.18.2...v0.18.3 [0.18.2]: https://github.com/pyo3/pyo3/compare/v0.18.1...v0.18.2 [0.18.1]: https://github.com/pyo3/pyo3/compare/v0.18.0...v0.18.1 [0.18.0]: https://github.com/pyo3/pyo3/compare/v0.17.3...v0.18.0 [0.17.3]: https://github.com/pyo3/pyo3/compare/v0.17.2...v0.17.3 [0.17.2]: https://github.com/pyo3/pyo3/compare/v0.17.1...v0.17.2 [0.17.1]: https://github.com/pyo3/pyo3/compare/v0.17.0...v0.17.1 [0.17.0]: https://github.com/pyo3/pyo3/compare/v0.16.6...v0.17.0 [0.16.6]: https://github.com/pyo3/pyo3/compare/v0.16.5...v0.16.6 [0.16.5]: https://github.com/pyo3/pyo3/compare/v0.16.4...v0.16.5 [0.16.4]: https://github.com/pyo3/pyo3/compare/v0.16.3...v0.16.4 [0.16.3]: https://github.com/pyo3/pyo3/compare/v0.16.2...v0.16.3 [0.16.2]: https://github.com/pyo3/pyo3/compare/v0.16.1...v0.16.2 [0.16.1]: https://github.com/pyo3/pyo3/compare/v0.16.0...v0.16.1 [0.16.0]: https://github.com/pyo3/pyo3/compare/v0.15.1...v0.16.0 [0.15.2]: https://github.com/pyo3/pyo3/compare/v0.15.1...v0.15.2 [0.15.1]: https://github.com/pyo3/pyo3/compare/v0.15.0...v0.15.1 [0.15.0]: https://github.com/pyo3/pyo3/compare/v0.14.5...v0.15.0 [0.14.5]: https://github.com/pyo3/pyo3/compare/v0.14.4...v0.14.5 [0.14.4]: https://github.com/pyo3/pyo3/compare/v0.14.3...v0.14.4 [0.14.3]: https://github.com/pyo3/pyo3/compare/v0.14.2...v0.14.3 [0.14.2]: https://github.com/pyo3/pyo3/compare/v0.14.1...v0.14.2 [0.14.1]: https://github.com/pyo3/pyo3/compare/v0.14.0...v0.14.1 [0.14.0]: https://github.com/pyo3/pyo3/compare/v0.13.2...v0.14.0 [0.13.2]: https://github.com/pyo3/pyo3/compare/v0.13.1...v0.13.2 [0.13.1]: https://github.com/pyo3/pyo3/compare/v0.13.0...v0.13.1 [0.13.0]: https://github.com/pyo3/pyo3/compare/v0.12.4...v0.13.0 [0.12.4]: https://github.com/pyo3/pyo3/compare/v0.12.3...v0.12.4 [0.12.3]: https://github.com/pyo3/pyo3/compare/v0.12.2...v0.12.3 [0.12.2]: https://github.com/pyo3/pyo3/compare/v0.12.1...v0.12.2 [0.12.1]: https://github.com/pyo3/pyo3/compare/v0.12.0...v0.12.1 [0.12.0]: https://github.com/pyo3/pyo3/compare/v0.11.1...v0.12.0 [0.11.1]: https://github.com/pyo3/pyo3/compare/v0.11.0...v0.11.1 [0.11.0]: https://github.com/pyo3/pyo3/compare/v0.10.1...v0.11.0 [0.10.1]: https://github.com/pyo3/pyo3/compare/v0.10.0...v0.10.1 [0.10.0]: https://github.com/pyo3/pyo3/compare/v0.9.2...v0.10.0 [0.9.2]: https://github.com/pyo3/pyo3/compare/v0.9.1...v0.9.2 [0.9.1]: https://github.com/pyo3/pyo3/compare/v0.9.0...v0.9.1 [0.9.0]: https://github.com/pyo3/pyo3/compare/v0.8.5...v0.9.0 [0.8.4]: https://github.com/pyo3/pyo3/compare/v0.8.4...v0.8.5 [0.8.4]: https://github.com/pyo3/pyo3/compare/v0.8.3...v0.8.4 [0.8.3]: https://github.com/pyo3/pyo3/compare/v0.8.2...v0.8.3 [0.8.2]: https://github.com/pyo3/pyo3/compare/v0.8.1...v0.8.2 [0.8.1]: https://github.com/pyo3/pyo3/compare/v0.8.0...v0.8.1 [0.8.0]: https://github.com/pyo3/pyo3/compare/v0.7.0...v0.8.0 [0.7.0]: https://github.com/pyo3/pyo3/compare/v0.6.0...v0.7.0 [0.6.0]: https://github.com/pyo3/pyo3/compare/v0.5.3...v0.6.0 [0.5.3]: https://github.com/pyo3/pyo3/compare/v0.5.2...v0.5.3 [0.5.2]: https://github.com/pyo3/pyo3/compare/v0.5.0...v0.5.2 [0.5.0]: https://github.com/pyo3/pyo3/compare/v0.4.1...v0.5.0 [0.4.1]: https://github.com/pyo3/pyo3/compare/v0.4.0...v0.4.1 [0.4.0]: https://github.com/pyo3/pyo3/compare/v0.3.2...v0.4.0 [0.3.2]: https://github.com/pyo3/pyo3/compare/v0.3.1...v0.3.2 [0.3.1]: https://github.com/pyo3/pyo3/compare/v0.3.0...v0.3.1 [0.3.0]: https://github.com/pyo3/pyo3/compare/v0.2.7...v0.3.0 [0.2.7]: https://github.com/pyo3/pyo3/compare/v0.2.6...v0.2.7 [0.2.6]: https://github.com/pyo3/pyo3/compare/v0.2.5...v0.2.6 [0.2.5]: https://github.com/pyo3/pyo3/compare/v0.2.4...v0.2.5 [0.2.4]: https://github.com/pyo3/pyo3/compare/v0.2.3...v0.2.4 [0.2.3]: https://github.com/pyo3/pyo3/compare/v0.2.2...v0.2.3 [0.2.2]: https://github.com/pyo3/pyo3/compare/v0.2.1...v0.2.2 [0.2.1]: https://github.com/pyo3/pyo3/compare/v0.2.0...v0.2.1 [0.2.0]: https://github.com/pyo3/pyo3/compare/v0.1.0...v0.2.0 [0.1.0]: https://github.com/PyO3/pyo3/tree/0.1.0 pyo3-0.20.2/CITATION.cff000064400000000000000000000004431046102023000124720ustar 00000000000000cff-version: 1.2.0 title: PyO3 message: >- If you use this software as part of a publication and wish to cite it, please use the metadata from this file. type: software authors: - name: PyO3 Project and Contributors website: https://github.com/PyO3 license: - Apache-2.0 - MIT pyo3-0.20.2/Cargo.toml0000644000000114140000000000100100060ustar # THIS FILE IS AUTOMATICALLY GENERATED BY CARGO # # When uploading crates to the registry Cargo will automatically # "normalize" Cargo.toml files for maximal compatibility # with all versions of Cargo and also rewrite `path` dependencies # to registry (e.g., crates.io) dependencies. # # If you are reading this file be aware that the original Cargo.toml # will likely look very different (and much more reasonable). # See Cargo.toml.orig for the original contents. [package] edition = "2021" rust-version = "1.56" name = "pyo3" version = "0.20.2" authors = ["PyO3 Project and Contributors "] exclude = [ "/.gitignore", ".cargo/config", "/codecov.yml", "/Makefile", "/pyproject.toml", "/noxfile.py", "/.github", "/tests/test_compile_error.rs", "/tests/ui", ] description = "Bindings to Python interpreter" homepage = "https://github.com/pyo3/pyo3" documentation = "https://docs.rs/crate/pyo3/" readme = "README.md" keywords = [ "pyo3", "python", "cpython", "ffi", ] categories = [ "api-bindings", "development-tools::ffi", ] license = "MIT OR Apache-2.0" repository = "https://github.com/pyo3/pyo3" [package.metadata.docs.rs] features = ["full"] no-default-features = true rustdoc-args = [ "--cfg", "docsrs", ] [dependencies.anyhow] version = "1.0" optional = true [dependencies.cfg-if] version = "1.0" [dependencies.chrono] version = "0.4.25" optional = true default-features = false [dependencies.either] version = "1.9" optional = true [dependencies.eyre] version = ">= 0.4, < 0.7" optional = true [dependencies.hashbrown] version = ">= 0.9, < 0.15" optional = true [dependencies.indexmap] version = ">= 1.6, < 3" optional = true [dependencies.indoc] version = "2.0.1" optional = true [dependencies.inventory] version = "0.3.0" optional = true [dependencies.libc] version = "0.2.62" [dependencies.memoffset] version = "0.9" [dependencies.num-bigint] version = "0.4" optional = true [dependencies.num-complex] version = ">= 0.2, < 0.5" optional = true [dependencies.parking_lot] version = ">= 0.11, < 0.13" [dependencies.pyo3-ffi] version = "=0.20.2" [dependencies.pyo3-macros] version = "=0.20.2" optional = true [dependencies.rust_decimal] version = "1.0.0" optional = true default-features = false [dependencies.serde] version = "1.0" optional = true [dependencies.smallvec] version = "1.0" optional = true [dependencies.unindent] version = "0.2.1" optional = true [dev-dependencies.assert_approx_eq] version = "1.1.0" [dev-dependencies.chrono] version = "0.4.25" [dev-dependencies.proptest] version = "1.0" features = ["std"] default-features = false [dev-dependencies.rayon] version = "1.6.1" [dev-dependencies.send_wrapper] version = "0.6" [dev-dependencies.serde] version = "1.0" features = ["derive"] [dev-dependencies.serde_json] version = "1.0.61" [dev-dependencies.trybuild] version = ">=1.0.70" [dev-dependencies.widestring] version = "0.5.1" [build-dependencies.pyo3-build-config] version = "=0.20.2" features = ["resolve-config"] [features] abi3 = [ "pyo3-build-config/abi3", "pyo3-ffi/abi3", "pyo3-macros/abi3", ] abi3-py310 = [ "abi3-py311", "pyo3-build-config/abi3-py310", "pyo3-ffi/abi3-py310", ] abi3-py311 = [ "abi3-py312", "pyo3-build-config/abi3-py311", "pyo3-ffi/abi3-py311", ] abi3-py312 = [ "abi3", "pyo3-build-config/abi3-py312", "pyo3-ffi/abi3-py312", ] abi3-py37 = [ "abi3-py38", "pyo3-build-config/abi3-py37", "pyo3-ffi/abi3-py37", ] abi3-py38 = [ "abi3-py39", "pyo3-build-config/abi3-py38", "pyo3-ffi/abi3-py38", ] abi3-py39 = [ "abi3-py310", "pyo3-build-config/abi3-py39", "pyo3-ffi/abi3-py39", ] auto-initialize = [] default = ["macros"] experimental-inspect = [] extension-module = ["pyo3-ffi/extension-module"] full = [ "macros", "chrono", "num-bigint", "num-complex", "hashbrown", "smallvec", "serde", "indexmap", "either", "eyre", "anyhow", "experimental-inspect", "rust_decimal", ] generate-import-lib = ["pyo3-ffi/generate-import-lib"] macros = [ "pyo3-macros", "indoc", "unindent", ] multiple-pymethods = [ "inventory", "pyo3-macros/multiple-pymethods", ] nightly = [] [lints.clippy] checked_conversions = "warn" dbg_macro = "warn" explicit_into_iter_loop = "warn" explicit_iter_loop = "warn" filter_map_next = "warn" flat_map_option = "warn" let_unit_value = "warn" manual_assert = "warn" manual_ok_or = "warn" todo = "warn" unnecessary_wraps = "warn" used_underscore_binding = "warn" useless_transmute = "warn" [lints.rust] elided_lifetimes_in_paths = "warn" invalid_doc_attributes = "warn" rust_2018_idioms = "warn" rust_2021_prelude_collisions = "warn" unused_lifetimes = "warn" [lints.rustdoc] bare_urls = "warn" broken_intra_doc_links = "warn" pyo3-0.20.2/Cargo.toml.orig000064400000000000000000000123661046102023000134760ustar 00000000000000[package] name = "pyo3" version = "0.20.2" description = "Bindings to Python interpreter" authors = ["PyO3 Project and Contributors "] readme = "README.md" keywords = ["pyo3", "python", "cpython", "ffi"] homepage = "https://github.com/pyo3/pyo3" repository = "https://github.com/pyo3/pyo3" documentation = "https://docs.rs/crate/pyo3/" categories = ["api-bindings", "development-tools::ffi"] license = "MIT OR Apache-2.0" exclude = ["/.gitignore", ".cargo/config", "/codecov.yml", "/Makefile", "/pyproject.toml", "/noxfile.py", "/.github", "/tests/test_compile_error.rs", "/tests/ui"] edition = "2021" rust-version = "1.56" [dependencies] cfg-if = "1.0" libc = "0.2.62" parking_lot = ">= 0.11, < 0.13" memoffset = "0.9" # ffi bindings to the python interpreter, split into a separate crate so they can be used independently pyo3-ffi = { path = "pyo3-ffi", version = "=0.20.2" } # support crates for macros feature pyo3-macros = { path = "pyo3-macros", version = "=0.20.2", optional = true } indoc = { version = "2.0.1", optional = true } unindent = { version = "0.2.1", optional = true } # support crate for multiple-pymethods feature inventory = { version = "0.3.0", optional = true } # crate integrations that can be added using the eponymous features anyhow = { version = "1.0", optional = true } chrono = { version = "0.4.25", default-features = false, optional = true } either = { version = "1.9", optional = true } eyre = { version = ">= 0.4, < 0.7", optional = true } hashbrown = { version = ">= 0.9, < 0.15", optional = true } indexmap = { version = ">= 1.6, < 3", optional = true } num-bigint = { version = "0.4", optional = true } num-complex = { version = ">= 0.2, < 0.5", optional = true } rust_decimal = { version = "1.0.0", default-features = false, optional = true } serde = { version = "1.0", optional = true } smallvec = { version = "1.0", optional = true } [dev-dependencies] assert_approx_eq = "1.1.0" chrono = { version = "0.4.25" } # Required for "and $N others" normalization trybuild = ">=1.0.70" proptest = { version = "1.0", default-features = false, features = ["std"] } send_wrapper = "0.6" serde = { version = "1.0", features = ["derive"] } serde_json = "1.0.61" rayon = "1.6.1" widestring = "0.5.1" [build-dependencies] pyo3-build-config = { path = "pyo3-build-config", version = "=0.20.2", features = ["resolve-config"] } [features] default = ["macros"] # Enables pyo3::inspect module and additional type information on FromPyObject # and IntoPy traits experimental-inspect = [] # Enables macros: #[pyclass], #[pymodule], #[pyfunction] etc. macros = ["pyo3-macros", "indoc", "unindent"] # Enables multiple #[pymethods] per #[pyclass] multiple-pymethods = ["inventory", "pyo3-macros/multiple-pymethods"] # Use this feature when building an extension module. # It tells the linker to keep the python symbols unresolved, # so that the module can also be used with statically linked python interpreters. extension-module = ["pyo3-ffi/extension-module"] # Use the Python limited API. See https://www.python.org/dev/peps/pep-0384/ for more. abi3 = ["pyo3-build-config/abi3", "pyo3-ffi/abi3", "pyo3-macros/abi3"] # With abi3, we can manually set the minimum Python version. abi3-py37 = ["abi3-py38", "pyo3-build-config/abi3-py37", "pyo3-ffi/abi3-py37"] abi3-py38 = ["abi3-py39", "pyo3-build-config/abi3-py38", "pyo3-ffi/abi3-py38"] abi3-py39 = ["abi3-py310", "pyo3-build-config/abi3-py39", "pyo3-ffi/abi3-py39"] abi3-py310 = ["abi3-py311", "pyo3-build-config/abi3-py310", "pyo3-ffi/abi3-py310"] abi3-py311 = ["abi3-py312", "pyo3-build-config/abi3-py311", "pyo3-ffi/abi3-py311"] abi3-py312 = ["abi3", "pyo3-build-config/abi3-py312", "pyo3-ffi/abi3-py312"] # Automatically generates `python3.dll` import libraries for Windows targets. generate-import-lib = ["pyo3-ffi/generate-import-lib"] # Changes `Python::with_gil` to automatically initialize the Python interpreter if needed. auto-initialize = [] # Optimizes PyObject to Vec conversion and so on. nightly = [] # Activates all additional features # This is mostly intended for testing purposes - activating *all* of these isn't particularly useful. full = [ "macros", # "multiple-pymethods", # TODO re-add this when MSRV is greater than 1.62 "chrono", "num-bigint", "num-complex", "hashbrown", "smallvec", "serde", "indexmap", "either", "eyre", "anyhow", "experimental-inspect", "rust_decimal", ] [workspace] members = [ "pyo3-ffi", "pyo3-build-config", "pyo3-macros", "pyo3-macros-backend", "pytests", "examples", ] [package.metadata.docs.rs] no-default-features = true features = ["full"] rustdoc-args = ["--cfg", "docsrs"] [workspace.lints.clippy] checked_conversions = "warn" dbg_macro = "warn" explicit_into_iter_loop = "warn" explicit_iter_loop = "warn" filter_map_next = "warn" flat_map_option = "warn" let_unit_value = "warn" manual_assert = "warn" manual_ok_or = "warn" todo = "warn" unnecessary_wraps = "warn" useless_transmute = "warn" used_underscore_binding = "warn" [workspace.lints.rust] elided_lifetimes_in_paths = "warn" invalid_doc_attributes = "warn" rust_2018_idioms = "warn" rust_2021_prelude_collisions = "warn" unused_lifetimes = "warn" [workspace.lints.rustdoc] broken_intra_doc_links = "warn" bare_urls = "warn" [lints] workspace = true pyo3-0.20.2/Code-of-Conduct.md000064400000000000000000000064041046102023000137760ustar 00000000000000# Contributor Covenant Code of Conduct ## Our Pledge In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. ## Our Standards Examples of behavior that contributes to creating a positive environment include: * Using welcoming and inclusive language * Being respectful of differing viewpoints and experiences * Gracefully accepting constructive criticism * Focusing on what is best for the community * Showing empathy towards other community members Examples of unacceptable behavior by participants include: * The use of sexualized language or imagery and unwelcome sexual attention or advances * Trolling, insulting/derogatory comments, and personal or political attacks * Public or private harassment * Publishing others' private information, such as a physical or electronic address, without explicit permission * Other conduct which could reasonably be considered inappropriate in a professional setting ## Our Responsibilities Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. ## Scope This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. ## Enforcement Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. ## Attribution This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html [homepage]: https://www.contributor-covenant.org For answers to common questions about this code of conduct, see https://www.contributor-covenant.org/faq pyo3-0.20.2/Contributing.md000064400000000000000000000247101046102023000135740ustar 00000000000000# Contributing Thank you for your interest in contributing to PyO3! All are welcome - please consider reading our [Code of Conduct](https://github.com/PyO3/pyo3/blob/main/Code-of-Conduct.md) to keep our community positive and inclusive. If you are searching for ideas how to contribute, proceed to the ["Getting started contributing"](#getting-started-contributing) section. If you have found a specific issue to contribute to and need information about the development process, you may find the section ["Writing pull requests"](#writing-pull-requests) helpful. If you want to become familiar with the codebase, see [Architecture.md](https://github.com/PyO3/pyo3/blob/main/Architecture.md). ## Getting started contributing Please join in with any part of PyO3 which interests you. We use GitHub issues to record all bugs and ideas. Feel free to request an issue to be assigned to you if you want to work on it. You can browse the API of the non-public parts of PyO3 [here](https://pyo3.netlify.app/internal/doc/pyo3/index.html). The following sections also contain specific ideas on where to start contributing to PyO3. ## Setting up a development environment To work and develop PyO3, you need Python & Rust installed on your system. * We encourage the use of [rustup](https://rustup.rs/) to be able to select and choose specific toolchains based on the project. * [Pyenv](https://github.com/pyenv/pyenv) is also highly recommended for being able to choose a specific Python version. * [virtualenv](https://virtualenv.pypa.io/en/latest/) can also be used with or without Pyenv to use specific installed Python versions. * [`nox`][nox] is used to automate many of our CI tasks. ### Caveats * When using pyenv on macOS, installing a Python version using `--enable-shared` is required to make it work. i.e `env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.7.12` ### Help users identify bugs The [PyO3 Gitter channel](https://gitter.im/PyO3/Lobby) is very active with users who are new to PyO3, and often completely new to Rust. Helping them debug is a great way to get experience with the PyO3 codebase. Helping others often reveals bugs, documentation weaknesses, and missing APIs. It's a good idea to open GitHub issues for these immediately so the resolution can be designed and implemented! ### Implement issues ready for development Issues where the solution is clear and work is not in progress use the [needs-implementer](https://github.com/PyO3/pyo3/issues?q=is%3Aissue+is%3Aopen+label%3Aneeds-implemeter) label. Don't be afraid if the solution is not clear to you! The core PyO3 contributors will be happy to mentor you through any questions you have to help you write the solution. ### Help write great docs PyO3 has a user guide (using mdbook) as well as the usual Rust API docs. The aim is for both of these to be detailed, easy to understand, and up-to-date. Pull requests are always welcome to fix typos, change wording, add examples, etc. There are some specific areas of focus where help is currently needed for the documentation: - Issues requesting documentation improvements are tracked with the [documentation](https://github.com/PyO3/pyo3/issues?q=is%3Aissue+is%3Aopen+label%3Adocumentation) label. - Not all APIs had docs or examples when they were made. The goal is to have documentation on all PyO3 APIs ([#306](https://github.com/PyO3/pyo3/issues/306)). If you see an API lacking a doc, please write one and open a PR! You can build the docs (including all features) with ```shell nox -s docs -- open ``` #### Doctests We use lots of code blocks in our docs. Run `cargo test --doc` when making changes to check that the doctests still work, or `cargo test` to run all the tests including doctests. See https://doc.rust-lang.org/rustdoc/documentation-tests.html for a guide on doctests. #### Building the guide You can preview the user guide by building it locally with `mdbook`. First, install [`mdbook`][mdbook] and [`nox`][nox]. Then, run ```shell nox -s build-guide -- --open ``` ### Help design the next PyO3 Issues which don't yet have a clear solution use the [needs-design](https://github.com/PyO3/pyo3/issues?q=is%3Aissue+is%3Aopen+label%3Aneeds-design) label. If any of these issues interest you, please join in with the conversation on the issue! All opinions are valued, and if you're interested in going further with e.g. draft PRs to experiment with API designs, even better! ### Review pull requests Everybody is welcome to submit comments on open PRs. Please help ensure new PyO3 APIs are safe, performant, tidy, and easy to use! ## Writing pull requests Here are a few things to note when you are writing PRs. ### Continuous Integration The PyO3 repo uses GitHub Actions. PRs are blocked from merging if CI is not successful. Formatting, linting and tests are checked for all Rust and Python code. In addition, all warnings in Rust code are disallowed (using `RUSTFLAGS="-D warnings"`). Tests run with all supported Python versions with the latest stable Rust compiler, as well as for Python 3.9 with the minimum supported Rust version. If you are adding a new feature, you should add it to the `full` feature in our *Cargo.toml** so that it is tested in CI. You can run these tests yourself with ```nox``` and ```nox -l``` lists further commands you can run. ### Documenting changes We use [towncrier](https://towncrier.readthedocs.io/en/stable/index.html) to generate a CHANGELOG for each release. To include your changes in the release notes, you should create one (or more) news items in the `newsfragments` directory. Valid news items should be saved as `..md` where `` is the pull request number and `` is one of the following: - `packaging` - for dependency changes and Python / Rust version compatibility changes - `added` - for new features - `changed` - for features which already existed but have been altered or deprecated - `removed` - for features which have been removed - `fixed` - for "changed" features which were classed as a bugfix Docs-only PRs do not need news items; start your PR title with `docs:` to skip the check. ### Style guide #### Generic code PyO3 has a lot of generic APIs to increase usability. These can come at the cost of generic code bloat. Where reasonable, try to implement a concrete sub-portion of generic functions. There are two forms of this: - If the concrete sub-portion doesn't benefit from re-use by other functions, name it `inner` and keep it as a local to the function. - If the concrete sub-portion is re-used by other functions, preferably name it `_foo` and place it directly below `foo` in the source code (where `foo` is the original generic function). #### FFI calls PyO3 makes a lot of FFI calls to Python's C API using raw pointers. Where possible try to avoid using pointers-to-temporaries in expressions: ```rust // dangerous pyo3::ffi::Something(name.to_object(py).as_ptr()); // because the following refactoring is a use-after-free error: let name = name.to_object(py).as_ptr(); pyo3::ffi::Something(name) ``` Instead, prefer to bind the safe owned `PyObject` wrapper before passing to ffi functions: ```rust let name: PyObject = name.to_object(py); pyo3::ffi::Something(name.as_ptr()) // name will automatically be freed when it falls out of scope ``` ## Python and Rust version support policy PyO3 aims to keep sufficient compatibility to make packaging Python extensions built with PyO3 feasible on most common package managers. To keep package maintainers' lives simpler, PyO3 will commit, wherever possible, to only adjust minimum supported Rust and Python versions at the same time. This bump will only come in an `0.x` release, roughly once per year, after the oldest supported Python version reaches its end-of-life. (Check https://endoflife.date/python for a clear timetable on these.) Below are guidelines on what compatibility all PRs are expected to deliver for each language. ### Python PyO3 supports all officially supported Python versions, as well as the latest PyPy3 release. All of these versions are tested in CI. ### Rust PyO3 aims to make use of up-to-date Rust language features to keep the implementation as efficient as possible. The minimum Rust version supported will be decided when the release which bumps Python and Rust versions is made. At the time, the minimum Rust version will be set no higher than the lowest Rust version shipped in the current Debian, RHEL and Alpine Linux distributions. CI tests both the most recent stable Rust version and the minimum supported Rust version. Because of Rust's stability guarantees this is sufficient to confirm support for all Rust versions in between. ## Benchmarking PyO3 has two sets of benchmarks for evaluating some aspects of its performance. The benchmark suite is currently very small - please open PRs with new benchmarks if you're interested in helping to expand it! First, there are Rust-based benchmarks located in the `pyo3-benches` subdirectory. You can run these benchmarks with: nox -s bench Second, there is a Python-based benchmark contained in the `pytests` subdirectory. You can read more about it [here](pytests). ## Code coverage You can view what code is and isn't covered by PyO3's tests. We aim to have 100% coverage - please check coverage and add tests if you notice a lack of coverage! - First, generate a `lcov.info` file with ```shell nox -s coverage ``` You can install an IDE plugin to view the coverage. For example, if you use VSCode: - Add the [coverage-gutters](https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters) plugin. - Add these settings to VSCode's `settings.json`: ```json { "coverage-gutters.coverageFileNames": [ "lcov.info", "cov.xml", "coverage.xml", ], "coverage-gutters.showLineCoverage": true } ``` - You should now be able to see green highlights for code that is tested, and red highlights for code that is not tested. ## Sponsor this project At the moment there is no official organisation that accepts sponsorship on PyO3's behalf. If you're seeking to provide significant funding to the PyO3 ecosystem, please reach out to us on [GitHub](https://github.com/PyO3/pyo3/issues/new) or [Gitter](https://gitter.im/PyO3/Lobby) and we can discuss. In the meanwhile, some of our maintainers have personal GitHub sponsorship pages and would be grateful for your support: - [davidhewitt](https://github.com/sponsors/davidhewitt) - [messense](https://github.com/sponsors/messense) [mdbook]: https://rust-lang.github.io/mdBook/cli/index.html [nox]: https://github.com/theacodes/nox pyo3-0.20.2/LICENSE-APACHE000064400000000000000000000250351046102023000125300ustar 00000000000000 Copyright (c) 2017-present PyO3 Project and Contributors. https://github.com/PyO3 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. pyo3-0.20.2/LICENSE-MIT000064400000000000000000000021231046102023000122310ustar 00000000000000Copyright (c) 2023-present PyO3 Project and Contributors. https://github.com/PyO3 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. pyo3-0.20.2/README.md000064400000000000000000000370241046102023000120640ustar 00000000000000# PyO3 [![actions status](https://img.shields.io/github/actions/workflow/status/PyO3/pyo3/ci.yml?branch=main&logo=github&style=)](https://github.com/PyO3/pyo3/actions) [![benchmark](https://img.shields.io/badge/benchmark-✓-Green?logo=github)](https://pyo3.rs/dev/bench/) [![codecov](https://img.shields.io/codecov/c/gh/PyO3/pyo3?logo=codecov)](https://codecov.io/gh/PyO3/pyo3) [![crates.io](https://img.shields.io/crates/v/pyo3?logo=rust)](https://crates.io/crates/pyo3) [![minimum rustc 1.56](https://img.shields.io/badge/rustc-1.56+-blue?logo=rust)](https://rust-lang.github.io/rfcs/2495-min-rust-version.html) [![dev chat](https://img.shields.io/gitter/room/PyO3/Lobby?logo=gitter)](https://gitter.im/PyO3/Lobby) [![contributing notes](https://img.shields.io/badge/contribute-on%20github-Green?logo=github)](https://github.com/PyO3/pyo3/blob/main/Contributing.md) [Rust](https://www.rust-lang.org/) bindings for [Python](https://www.python.org/), including tools for creating native Python extension modules. Running and interacting with Python code from a Rust binary is also supported. - User Guide: [stable](https://pyo3.rs) | [main](https://pyo3.rs/main) - API Documentation: [stable](https://docs.rs/pyo3/) | [main](https://pyo3.rs/main/doc) ## Usage PyO3 supports the following software versions: - Python 3.7 and up (CPython and PyPy) - Rust 1.56 and up You can use PyO3 to write a native Python module in Rust, or to embed Python in a Rust binary. The following sections explain each of these in turn. ### Using Rust from Python PyO3 can be used to generate a native Python module. The easiest way to try this out for the first time is to use [`maturin`](https://github.com/PyO3/maturin). `maturin` is a tool for building and publishing Rust-based Python packages with minimal configuration. The following steps install `maturin`, use it to generate and build a new Python package, and then launch Python to import and execute a function from the package. First, follow the commands below to create a new directory containing a new Python `virtualenv`, and install `maturin` into the virtualenv using Python's package manager, `pip`: ```bash # (replace string_sum with the desired package name) $ mkdir string_sum $ cd string_sum $ python -m venv .env $ source .env/bin/activate $ pip install maturin ``` Still inside this `string_sum` directory, now run `maturin init`. This will generate the new package source. When given the choice of bindings to use, select pyo3 bindings: ```bash $ maturin init ✔ 🤷 What kind of bindings to use? · pyo3 ✨ Done! New project created string_sum ``` The most important files generated by this command are `Cargo.toml` and `lib.rs`, which will look roughly like the following: **`Cargo.toml`** ```toml [package] name = "string_sum" version = "0.1.0" edition = "2021" [lib] # The name of the native library. This is the name which will be used in Python to import the # library (i.e. `import string_sum`). If you change this, you must also change the name of the # `#[pymodule]` in `src/lib.rs`. name = "string_sum" # "cdylib" is necessary to produce a shared library for Python to import from. # # Downstream Rust code (including code in `bin/`, `examples/`, and `tests/`) will not be able # to `use string_sum;` unless the "rlib" or "lib" crate type is also included, e.g.: # crate-type = ["cdylib", "rlib"] crate-type = ["cdylib"] [dependencies] pyo3 = { version = "0.20.2", features = ["extension-module"] } ``` **`src/lib.rs`** ```rust use pyo3::prelude::*; /// Formats the sum of two numbers as string. #[pyfunction] fn sum_as_string(a: usize, b: usize) -> PyResult { Ok((a + b).to_string()) } /// A Python module implemented in Rust. The name of this function must match /// the `lib.name` setting in the `Cargo.toml`, else Python will not be able to /// import the module. #[pymodule] fn string_sum(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(sum_as_string, m)?)?; Ok(()) } ``` Finally, run `maturin develop`. This will build the package and install it into the Python virtualenv previously created and activated. The package is then ready to be used from `python`: ```bash $ maturin develop # lots of progress output as maturin runs the compilation... $ python >>> import string_sum >>> string_sum.sum_as_string(5, 20) '25' ``` To make changes to the package, just edit the Rust source code and then re-run `maturin develop` to recompile. To run this all as a single copy-and-paste, use the bash script below (replace `string_sum` in the first command with the desired package name): ```bash mkdir string_sum && cd "$_" python -m venv .env source .env/bin/activate pip install maturin maturin init --bindings pyo3 maturin develop ``` If you want to be able to run `cargo test` or use this project in a Cargo workspace and are running into linker issues, there are some workarounds in [the FAQ](https://pyo3.rs/latest/faq.html#i-cant-run-cargo-test-or-i-cant-build-in-a-cargo-workspace-im-having-linker-issues-like-symbol-not-found-or-undefined-reference-to-_pyexc_systemerror). As well as with `maturin`, it is possible to build using [`setuptools-rust`](https://github.com/PyO3/setuptools-rust) or [manually](https://pyo3.rs/latest/building_and_distribution.html#manual-builds). Both offer more flexibility than `maturin` but require more configuration to get started. ### Using Python from Rust To embed Python into a Rust binary, you need to ensure that your Python installation contains a shared library. The following steps demonstrate how to ensure this (for Ubuntu), and then give some example code which runs an embedded Python interpreter. To install the Python shared library on Ubuntu: ```bash sudo apt install python3-dev ``` To install the Python shared library on RPM based distributions (e.g. Fedora, Red Hat, SuSE), install the `python3-devel` package. Start a new project with `cargo new` and add `pyo3` to the `Cargo.toml` like this: ```toml [dependencies.pyo3] version = "0.20.2" features = ["auto-initialize"] ``` Example program displaying the value of `sys.version` and the current user name: ```rust use pyo3::prelude::*; use pyo3::types::IntoPyDict; fn main() -> PyResult<()> { Python::with_gil(|py| { let sys = py.import("sys")?; let version: String = sys.getattr("version")?.extract()?; let locals = [("os", py.import("os")?)].into_py_dict(py); let code = "os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'"; let user: String = py.eval(code, None, Some(&locals))?.extract()?; println!("Hello {}, I'm Python {}", user, version); Ok(()) }) } ``` The guide has [a section](https://pyo3.rs/latest/python_from_rust.html) with lots of examples about this topic. ## Tools and libraries - [maturin](https://github.com/PyO3/maturin) _Build and publish crates with pyo3, rust-cpython or cffi bindings as well as rust binaries as python packages_ - [setuptools-rust](https://github.com/PyO3/setuptools-rust) _Setuptools plugin for Rust support_. - [pyo3-built](https://github.com/PyO3/pyo3-built) _Simple macro to expose metadata obtained with the [`built`](https://crates.io/crates/built) crate as a [`PyDict`](https://docs.rs/pyo3/*/pyo3/types/struct.PyDict.html)_ - [rust-numpy](https://github.com/PyO3/rust-numpy) _Rust binding of NumPy C-API_ - [dict-derive](https://github.com/gperinazzo/dict-derive) _Derive FromPyObject to automatically transform Python dicts into Rust structs_ - [pyo3-log](https://github.com/vorner/pyo3-log) _Bridge from Rust to Python logging_ - [pythonize](https://github.com/davidhewitt/pythonize) _Serde serializer for converting Rust objects to JSON-compatible Python objects_ - [pyo3-asyncio](https://github.com/awestlake87/pyo3-asyncio) _Utilities for working with Python's Asyncio library and async functions_ - [rustimport](https://github.com/mityax/rustimport) _Directly import Rust files or crates from Python, without manual compilation step. Provides pyo3 integration by default and generates pyo3 binding code automatically._ ## Examples - [autopy](https://github.com/autopilot-rs/autopy) _A simple, cross-platform GUI automation library for Python and Rust._ - Contains an example of building wheels on TravisCI and appveyor using [cibuildwheel](https://github.com/pypa/cibuildwheel) - [ballista-python](https://github.com/apache/arrow-ballista-python) _A Python library that binds to Apache Arrow distributed query engine Ballista._ - [bed-reader](https://github.com/fastlmm/bed-reader) _Read and write the PLINK BED format, simply and efficiently._ - Shows Rayon/ndarray::parallel (including capturing errors, controlling thread num), Python types to Rust generics, Github Actions - [cryptography](https://github.com/pyca/cryptography/tree/main/src/rust) _Python cryptography library with some functionality in Rust._ - [css-inline](https://github.com/Stranger6667/css-inline/tree/master/bindings/python) _CSS inlining for Python implemented in Rust._ - [datafusion-python](https://github.com/apache/arrow-datafusion-python) _A Python library that binds to Apache Arrow in-memory query engine DataFusion._ - [deltalake-python](https://github.com/delta-io/delta-rs/tree/main/python) _Native Delta Lake Python binding based on delta-rs with Pandas integration._ - [fastbloom](https://github.com/yankun1992/fastbloom) _A fast [bloom filter](https://github.com/yankun1992/fastbloom#BloomFilter) | [counting bloom filter](https://github.com/yankun1992/fastbloom#countingbloomfilter) implemented by Rust for Rust and Python!_ - [fastuuid](https://github.com/thedrow/fastuuid/) _Python bindings to Rust's UUID library._ - [feos](https://github.com/feos-org/feos) _Lightning fast thermodynamic modeling in Rust with fully developed Python interface._ - [forust](https://github.com/jinlow/forust) _A lightweight gradient boosted decision tree library written in Rust._ - [haem](https://github.com/BooleanCat/haem) _A Python library for working on Bioinformatics problems._ - [html-py-ever](https://github.com/PyO3/setuptools-rust/tree/main/examples/html-py-ever) _Using [html5ever](https://github.com/servo/html5ever) through [kuchiki](https://github.com/kuchiki-rs/kuchiki) to speed up html parsing and css-selecting._ - [hyperjson](https://github.com/mre/hyperjson) _A hyper-fast Python module for reading/writing JSON data using Rust's serde-json._ - [inline-python](https://github.com/fusion-engineering/inline-python) _Inline Python code directly in your Rust code._ - [jsonschema-rs](https://github.com/Stranger6667/jsonschema-rs/tree/master/bindings/python) _Fast JSON Schema validation library._ - [mocpy](https://github.com/cds-astro/mocpy) _Astronomical Python library offering data structures for describing any arbitrary coverage regions on the unit sphere._ - [opendal](https://github.com/apache/incubator-opendal/tree/main/bindings/python) _A data access layer that allows users to easily and efficiently retrieve data from various storage services in a unified way._ - [orjson](https://github.com/ijl/orjson) _Fast Python JSON library._ - [ormsgpack](https://github.com/aviramha/ormsgpack) _Fast Python msgpack library._ - [point-process](https://github.com/ManifoldFR/point-process-rust/tree/master/pylib) _High level API for pointprocesses as a Python library._ - [polaroid](https://github.com/daggy1234/polaroid) _Hyper Fast and safe image manipulation library for Python written in Rust._ - [polars](https://github.com/pola-rs/polars) _Fast multi-threaded DataFrame library in Rust | Python | Node.js._ - [pydantic-core](https://github.com/pydantic/pydantic-core) _Core validation logic for pydantic written in Rust._ - [pyheck](https://github.com/kevinheavey/pyheck) _Fast case conversion library, built by wrapping [heck](https://github.com/withoutboats/heck)._ - Quite easy to follow as there's not much code. - [pyre](https://github.com/Project-Dream-Weaver/pyre-http) _Fast Python HTTP server written in Rust._ - [ril-py](https://github.com/Cryptex-github/ril-py) _A performant and high-level image processing library for Python written in Rust._ - [river](https://github.com/online-ml/river) _Online machine learning in python, the computationally heavy statistics algorithms are implemented in Rust._ - [rust-python-coverage](https://github.com/cjermain/rust-python-coverage) _Example PyO3 project with automated test coverage for Rust and Python._ - [tiktoken](https://github.com/openai/tiktoken) _A fast BPE tokeniser for use with OpenAI's models._ - [tokenizers](https://github.com/huggingface/tokenizers/tree/main/bindings/python) _Python bindings to the Hugging Face tokenizers (NLP) written in Rust._ - [tzfpy](http://github.com/ringsaturn/tzfpy) _A fast package to convert longitude/latitude to timezone name._ - [wasmer-python](https://github.com/wasmerio/wasmer-python) _Python library to run WebAssembly binaries._ ## Articles and other media - [A Week of PyO3 + rust-numpy (How to Speed Up Your Data Pipeline X Times)](https://terencezl.github.io/blog/2023/06/06/a-week-of-pyo3-rust-numpy/) - Jun 6, 2023 - [(Podcast) PyO3 with David Hewitt](https://rustacean-station.org/episode/david-hewitt/) - May 19, 2023 - [Making Python 100x faster with less than 100 lines of Rust](https://ohadravid.github.io/posts/2023-03-rusty-python/) - Mar 28, 2023 - [How Pydantic V2 leverages Rust's Superpowers](https://fosdem.org/2023/schedule/event/rust_how_pydantic_v2_leverages_rusts_superpowers/) - Feb 4, 2023 - [How we extended the River stats module with Rust using PyO3](https://boring-guy.sh/posts/river-rust/) - Dec 23, 2022 - [Nine Rules for Writing Python Extensions in Rust](https://towardsdatascience.com/nine-rules-for-writing-python-extensions-in-rust-d35ea3a4ec29?sk=f8d808d5f414154fdb811e4137011437) - Dec 31, 2021 - [Calling Rust from Python using PyO3](https://saidvandeklundert.net/learn/2021-11-18-calling-rust-from-python-using-pyo3/) - Nov 18, 2021 - [davidhewitt's 2021 talk at Rust Manchester meetup](https://www.youtube.com/watch?v=-XyWG_klSAw&t=320s) - Aug 19, 2021 - [Incrementally porting a small Python project to Rust](https://blog.waleedkhan.name/port-python-to-rust/) - Apr 29, 2021 - [Vortexa - Integrating Rust into Python](https://www.vortexa.com/insight/integrating-rust-into-python) - Apr 12, 2021 - [Writing and publishing a Python module in Rust](https://blog.yossarian.net/2020/08/02/Writing-and-publishing-a-python-module-in-rust) - Aug 2, 2020 ## Contributing Everyone is welcomed to contribute to PyO3! There are many ways to support the project, such as: - help PyO3 users with issues on GitHub and Gitter - improve documentation - write features and bugfixes - publish blogs and examples of how to use PyO3 Our [contributing notes](https://github.com/PyO3/pyo3/blob/main/Contributing.md) and [architecture guide](https://github.com/PyO3/pyo3/blob/main/Architecture.md) have more resources if you wish to volunteer time for PyO3 and are searching where to start. If you don't have time to contribute yourself but still wish to support the project's future success, some of our maintainers have GitHub sponsorship pages: - [davidhewitt](https://github.com/sponsors/davidhewitt) - [messense](https://github.com/sponsors/messense) ## License PyO3 is licensed under the [Apache-2.0 license](LICENSE-APACHE) or the [MIT license](LICENSE-MIT), at your option. Python is licensed under the [Python License](https://docs.python.org/3/license.html). Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in PyO3 by you, as defined in the Apache License, shall be dual-licensed as above, without any additional terms or conditions. Deploys by Netlify pyo3-0.20.2/Releasing.md000064400000000000000000000063701046102023000130400ustar 00000000000000# Releasing This is notes for the current process of releasing a new PyO3 version. Replace `` in all instructions below with the new version. ## 1. Prepare the release commit Follow the process below to update all required pieces to bump the version. All these changes are done in a single commit because it makes it clear to git readers what happened to bump the version. It also makes it easy to cherry-pick the version bump onto the `main` branch when tidying up branch history at the end of the release process. 1. Replace all instances of the PyO3 current version with the new version to be released. Places to check: - `Cargo.toml` for all PyO3 crates in the repository. - Examples in `README.md` - PyO3 version embedded into documentation like the README. - `pre-script.rhai` templates for the examples. - `[towncrier]` section in `pyproject.toml`. Make sure not to modify the CHANGELOG during this step! 2. Run `towncrier build` to generate the CHANGELOG. The version used by `towncrier` should automatically be correct because of the update to `pyproject.toml` in step 1. 3. Manually edit the CHANGELOG for final notes. Steps to do: - Adjust wording of any release lines to make them clearer for users / fix typos. - Add a new link at the bottom for the new version, and update the `Unreleased` link. 4. Create the commit containing all the above changes, with a message of `release: `. Push to `release-` branch on the main PyO3 repository, where `` depends on whether this is a major or minor release: - for O.X.0 minor releases, just use `0.X`, e.g. `release-0.17`. This will become the maintenance branch after release. - for 0.X.Y patch releases, use the full `0.X.Y`, e.g. `release-0.17.1`. This will be deleted after merge. ## 2. Create the release PR and draft release notes Open a PR for the branch, and confirm that it passes CI. For `0.X.0` minor releases, the PR should be merging into `main`, for `0.X.Y` patch releases, the PR should be merging the `release-0.X` maintenance branch. On https://github.com/PyO3/pyo3/releases, click "Draft a new release". The tag will be a new tag of `v` (note preceding `v`) and target should be the `release-` branch you just pushed. Write release notes which match the style of previous releases. You can get the list of contributors by running `nox -s contributors -- v release-` to get contributors from the previous version tag through to the branch tip you just pushed. (This uses the GitHub API, so you'll need to push the branch first.) Save as a draft and wait for now. ## 3. Leave for a cooling off period Wait a couple of days in case anyone wants to hold up the release to add bugfixes etc. ## 4. Put live To put live: - 1. run `nox -s publish` to put live on crates.io - 2. publish the release on Github - 3. merge the release PR ## 5. Tidy the main branch If the release PR targeted a branch other than main, you will need to cherry-pick the version bumps, CHANGELOG modifications and removal of towncrier `newsfragments` and open another PR to land these on main. ## 6. Delete the release branch (patch releases only) For 0.X.Y patch releases, the release branch is no longer needed, so it should be deleted. pyo3-0.20.2/assets/script.py000064400000000000000000000000641046102023000137570ustar 00000000000000# Used in PyModule examples. class Blah: pass pyo3-0.20.2/build.rs000064400000000000000000000037031046102023000122470ustar 00000000000000use std::env; use pyo3_build_config::pyo3_build_script_impl::{cargo_env_var, errors::Result}; use pyo3_build_config::{bail, print_feature_cfgs, InterpreterConfig}; fn ensure_auto_initialize_ok(interpreter_config: &InterpreterConfig) -> Result<()> { if cargo_env_var("CARGO_FEATURE_AUTO_INITIALIZE").is_some() && !interpreter_config.shared { bail!( "The `auto-initialize` feature is enabled, but your python installation only supports \ embedding the Python interpreter statically. If you are attempting to run tests, or a \ binary which is okay to link dynamically, install a Python distribution which ships \ with the Python shared library.\n\ \n\ Embedding the Python interpreter statically does not yet have first-class support in \ PyO3. If you are sure you intend to do this, disable the `auto-initialize` feature.\n\ \n\ For more information, see \ https://pyo3.rs/v{pyo3_version}/\ building_and_distribution.html#embedding-python-in-rust", pyo3_version = env::var("CARGO_PKG_VERSION").unwrap() ); } Ok(()) } /// Prepares the PyO3 crate for compilation. /// /// This loads the config from pyo3-build-config and then makes some additional checks to improve UX /// for users. /// /// Emits the cargo configuration based on this config as well as a few checks of the Rust compiler /// version to enable features which aren't supported on MSRV. fn configure_pyo3() -> Result<()> { let interpreter_config = pyo3_build_config::get(); ensure_auto_initialize_ok(interpreter_config)?; for cfg in interpreter_config.build_script_outputs() { println!("{}", cfg) } // Emit cfgs like `thread_local_const_init` print_feature_cfgs(); Ok(()) } fn main() { if let Err(e) = configure_pyo3() { eprintln!("error: {}", e.report()); std::process::exit(1) } } pyo3-0.20.2/emscripten/.gitignore000064400000000000000000000000171046102023000147360ustar 00000000000000pybuilddir.txt pyo3-0.20.2/emscripten/Makefile000064400000000000000000000053521046102023000144150ustar 00000000000000CURDIR=$(abspath .) # These three are passed in from nox. BUILDROOT ?= $(CURDIR)/builddir PYMAJORMINORMICRO ?= 3.11.0 EMSCRIPTEN_VERSION=3.1.13 export EMSDKDIR = $(BUILDROOT)/emsdk PLATFORM=wasm32_emscripten SYSCONFIGDATA_NAME=_sysconfigdata__$(PLATFORM) # BASH_ENV tells bash to source emsdk_env.sh on startup. export BASH_ENV := $(CURDIR)/env.sh # Use bash to run each command so that env.sh will be used. SHELL := /bin/bash # Set version variables. version_tuple := $(subst ., ,$(PYMAJORMINORMICRO:v%=%)) PYMAJOR=$(word 1,$(version_tuple)) PYMINOR=$(word 2,$(version_tuple)) PYMICRO=$(word 3,$(version_tuple)) PYVERSION=$(PYMAJORMINORMICRO) PYMAJORMINOR=$(PYMAJOR).$(PYMINOR) PYTHONURL=https://www.python.org/ftp/python/$(PYMAJORMINORMICRO)/Python-$(PYVERSION).tgz PYTHONTARBALL=$(BUILDROOT)/downloads/Python-$(PYVERSION).tgz PYTHONBUILD=$(BUILDROOT)/build/Python-$(PYVERSION) PYTHONLIBDIR=$(BUILDROOT)/install/Python-$(PYVERSION)/lib all: $(PYTHONLIBDIR)/libpython$(PYMAJORMINOR).a $(BUILDROOT)/.exists: mkdir -p $(BUILDROOT) touch $@ # Install emscripten $(EMSDKDIR): $(CURDIR)/emscripten_patches/* $(BUILDROOT)/.exists git clone https://github.com/emscripten-core/emsdk.git --depth 1 --branch $(EMSCRIPTEN_VERSION) $(EMSDKDIR) $(EMSDKDIR)/emsdk install $(EMSCRIPTEN_VERSION) cd $(EMSDKDIR)/upstream/emscripten && cat $(CURDIR)/emscripten_patches/* | patch -p1 $(EMSDKDIR)/emsdk activate $(EMSCRIPTEN_VERSION) $(PYTHONTARBALL): [ -d $(BUILDROOT)/downloads ] || mkdir -p $(BUILDROOT)/downloads wget -q -O $@ $(PYTHONURL) $(PYTHONBUILD)/.patched: $(PYTHONTARBALL) [ -d $(PYTHONBUILD) ] || ( \ mkdir -p $(dir $(PYTHONBUILD));\ tar -C $(dir $(PYTHONBUILD)) -xf $(PYTHONTARBALL) \ ) touch $@ $(PYTHONBUILD)/Makefile: $(PYTHONBUILD)/.patched $(BUILDROOT)/emsdk cd $(PYTHONBUILD) && \ CONFIG_SITE=Tools/wasm/config.site-wasm32-emscripten \ emconfigure ./configure -C \ --host=wasm32-unknown-emscripten \ --build=$(shell $(PYTHONBUILD)/config.guess) \ --with-emscripten-target=browser \ --enable-wasm-dynamic-linking \ --with-build-python=python3.11 $(PYTHONLIBDIR)/libpython$(PYMAJORMINOR).a : $(PYTHONBUILD)/Makefile cd $(PYTHONBUILD) && \ emmake make -j3 libpython$(PYMAJORMINOR).a # Generate sysconfigdata _PYTHON_SYSCONFIGDATA_NAME=$(SYSCONFIGDATA_NAME) _PYTHON_PROJECT_BASE=$(PYTHONBUILD) python3.11 -m sysconfig --generate-posix-vars cp `cat pybuilddir.txt`/$(SYSCONFIGDATA_NAME).py $(PYTHONBUILD)/Lib mkdir -p $(PYTHONLIBDIR) # Copy libexpat.a, libmpdec.a, and libpython3.11.a # In noxfile, we explicitly link libexpat and libmpdec via RUSTFLAGS find $(PYTHONBUILD) -name '*.a' -exec cp {} $(PYTHONLIBDIR) \; # Install Python stdlib cp -r $(PYTHONBUILD)/Lib $(PYTHONLIBDIR)/python$(PYMAJORMINOR) clean: rm -rf $(BUILDROOT) pyo3-0.20.2/emscripten/emscripten_patches/0001-Add-_gxx_personality_v0-stub-to-library.js.patch000064400000000000000000000020261046102023000303700ustar 00000000000000From 4b56f37c3dc9185a235a8314086c4d7a6239b2f8 Mon Sep 17 00:00:00 2001 From: Hood Chatham Date: Sat, 4 Jun 2022 19:19:47 -0700 Subject: [PATCH] Add _gxx_personality_v0 stub to library.js Mitigation for an incompatibility between Rust and Emscripten: https://github.com/rust-lang/rust/issues/85821 https://github.com/emscripten-core/emscripten/issues/17128 --- src/library.js | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/library.js b/src/library.js index e7bb4c38e..7d01744df 100644 --- a/src/library.js +++ b/src/library.js @@ -403,6 +403,8 @@ mergeInto(LibraryManager.library, { abort('Assertion failed: ' + UTF8ToString(condition) + ', at: ' + [filename ? UTF8ToString(filename) : 'unknown filename', line, func ? UTF8ToString(func) : 'unknown function']); }, + __gxx_personality_v0: function() {}, + // ========================================================================== // time.h // ========================================================================== -- 2.25.1 pyo3-0.20.2/emscripten/env.sh000064400000000000000000000003571046102023000141010ustar 00000000000000#!/bin/bash # Activate emsdk environment. emsdk_env.sh writes a lot to stderr so we suppress # the output. This also prevents it from complaining when emscripten isn't yet # installed. source "$EMSDKDIR/emsdk_env.sh" 2> /dev/null || true pyo3-0.20.2/emscripten/pybuilddir.txt000064400000000000000000000000331046102023000156540ustar 00000000000000build/lib.linux-x86_64-3.11pyo3-0.20.2/emscripten/runner.py000075500000000000000000000002361046102023000146370ustar 00000000000000#!/usr/local/bin/python import pathlib import sys import subprocess p = pathlib.Path(sys.argv[1]) sys.exit(subprocess.call(["node", p.name], cwd=p.parent)) pyo3-0.20.2/guide/book.toml000064400000000000000000000005511046102023000135240ustar 00000000000000[book] title = "PyO3 user guide" description = "PyO3 user guide" author = "PyO3 Project and Contributors" [preprocessor.pyo3_version] command = "python3 guide/pyo3_version.py" [output.html] git-repository-url = "https://github.com/PyO3/pyo3/tree/main/guide" edit-url-template = "https://github.com/PyO3/pyo3/edit/main/guide/{path}" playground.runnable = falsepyo3-0.20.2/guide/pyclass_parameters.md000064400000000000000000000067201046102023000161240ustar 00000000000000`#[pyclass]` can be used with the following parameters: | Parameter | Description | | :- | :- | | `crate = "some::path"` | Path to import the `pyo3` crate, if it's not accessible at `::pyo3`. | | `dict` | Gives instances of this class an empty `__dict__` to store custom attributes. | | `extends = BaseType` | Use a custom baseclass. Defaults to [`PyAny`][params-1] | | `freelist = N` | Implements a [free list][params-2] of size N. This can improve performance for types that are often created and deleted in quick succession. Profile your code to see whether `freelist` is right for you. | | `frozen` | Declares that your pyclass is immutable. It removes the borrow checker overhead when retrieving a shared reference to the Rust struct, but disables the ability to get a mutable reference. | | `get_all` | Generates getters for all fields of the pyclass. | | `mapping` | Inform PyO3 that this class is a [`Mapping`][params-mapping], and so leave its implementation of sequence C-API slots empty. | | `module = "module_name"` | Python code will see the class as being defined in this module. Defaults to `builtins`. | | `name = "python_name"` | Sets the name that Python sees this class as. Defaults to the name of the Rust struct. | | `rename_all = "renaming_rule"` | Applies renaming rules to every getters and setters of a struct, or every variants of an enum. Possible values are: "camelCase", "kebab-case", "lowercase", "PascalCase", "SCREAMING-KEBAB-CASE", "SCREAMING_SNAKE_CASE", "snake_case", "UPPERCASE". | | `sequence` | Inform PyO3 that this class is a [`Sequence`][params-sequence], and so leave its C-API mapping length slot empty. | | `set_all` | Generates setters for all fields of the pyclass. | | `subclass` | Allows other Python classes and `#[pyclass]` to inherit from this class. Enums cannot be subclassed. | | `text_signature = "(arg1, arg2, ...)"` | Sets the text signature for the Python class' `__new__` method. | | `unsendable` | Required if your struct is not [`Send`][params-3]. Rather than using `unsendable`, consider implementing your struct in a threadsafe way by e.g. substituting [`Rc`][params-4] with [`Arc`][params-5]. By using `unsendable`, your class will panic when accessed by another thread.| | `weakref` | Allows this class to be [weakly referenceable][params-6]. | All of these parameters can either be passed directly on the `#[pyclass(...)]` annotation, or as one or more accompanying `#[pyo3(...)]` annotations, e.g.: ```rust,ignore // Argument supplied directly to the `#[pyclass]` annotation. #[pyclass(name = "SomeName", subclass)] struct MyClass {} // Argument supplied as a separate annotation. #[pyclass] #[pyo3(name = "SomeName", subclass)] struct MyClass {} ``` [params-1]: https://docs.rs/pyo3/latest/pyo3/struct.PyAny.html [params-2]: https://en.wikipedia.org/wiki/Free_list [params-3]: https://doc.rust-lang.org/std/marker/trait.Send.html [params-4]: https://doc.rust-lang.org/std/rc/struct.Rc.html [params-5]: https://doc.rust-lang.org/std/sync/struct.Arc.html [params-6]: https://docs.python.org/3/library/weakref.html [params-mapping]: https://pyo3.rs/latest/class/protocols.html#mapping--sequence-types [params-sequence]: https://pyo3.rs/latest/class/protocols.html#mapping--sequence-types pyo3-0.20.2/guide/pyo3_version.py000064400000000000000000000032651046102023000147130ustar 00000000000000"""Simple mdbook preprocessor to inject pyo3 version into the guide. It will replace: - {{#PYO3_VERSION_TAG}} with the contents of the PYO3_VERSION_TAG environment var - {{#PYO3_DOCS_URL}} with the location of docs (e.g. 'https://docs.rs/pyo3/0.13.2') - {{#PYO3_CRATE_VERSION}} with a relevant toml snippet (e.g. 'version = "0.13.2"') Tested against mdbook 0.4.10. """ import json import os import sys # Set PYO3_VERSION in CI to build the correct version into links PYO3_VERSION_TAG = os.environ.get("PYO3_VERSION_TAG", "main") if PYO3_VERSION_TAG == "main": PYO3_DOCS_URL = "https://pyo3.rs/main/doc" PYO3_DOCS_VERSION = "latest" PYO3_CRATE_VERSION = 'git = "https://github.com/pyo3/pyo3"' else: # v0.13.2 -> 0.13.2 version = PYO3_VERSION_TAG.lstrip("v") PYO3_DOCS_URL = f"https://docs.rs/pyo3/{version}" PYO3_DOCS_VERSION = version PYO3_CRATE_VERSION = f'version = "{version}"' def replace_section_content(section): if not isinstance(section, dict) or "Chapter" not in section: return # Replace raw and url-encoded forms section["Chapter"]["content"] = ( section["Chapter"]["content"] .replace("{{#PYO3_VERSION_TAG}}", PYO3_VERSION_TAG) .replace("{{#PYO3_DOCS_URL}}", PYO3_DOCS_URL) .replace("{{#PYO3_DOCS_VERSION}}", PYO3_DOCS_VERSION) .replace("{{#PYO3_CRATE_VERSION}}", PYO3_CRATE_VERSION) ) for sub_item in section["Chapter"]["sub_items"]: replace_section_content(sub_item) for line in sys.stdin: if line: [context, book] = json.loads(line) for section in book["sections"]: replace_section_content(section) json.dump(book, fp=sys.stdout) pyo3-0.20.2/guide/src/SUMMARY.md000064400000000000000000000027171046102023000141510ustar 00000000000000# Summary [Introduction](index.md) --- - [Getting started](getting_started.md) - [Python modules](module.md) - [Python functions](function.md) - [Function signatures](function/signature.md) - [Error handling](function/error_handling.md) - [Python classes](class.md) - [Class customizations](class/protocols.md) - [Basic object customization](class/object.md) - [Emulating numeric types](class/numeric.md) - [Emulating callable objects](class/call.md) - [Type conversions](conversions.md) - [Mapping of Rust types to Python types](conversions/tables.md)] - [Conversion traits](conversions/traits.md)] - [Python exceptions](exception.md) - [Calling Python from Rust](python_from_rust.md) - [GIL, mutability and object types](types.md) - [Parallelism](parallelism.md) - [Debugging](debugging.md) - [Features reference](features.md) - [Memory management](memory.md) - [Performance](performance.md) - [Advanced topics](advanced.md) - [Building and distribution](building_and_distribution.md) - [Supporting multiple Python versions](building_and_distribution/multiple_python_versions.md) - [Useful crates](ecosystem.md) - [Logging](ecosystem/logging.md) - [Using `async` and `await`](ecosystem/async-await.md) - [FAQ and troubleshooting](faq.md) --- [Appendix A: Migration guide](migration.md) [Appendix B: Trait bounds](trait_bounds.md) [Appendix C: Python typing hints](python_typing_hints.md) [CHANGELOG](changelog.md) --- [Contributing](contributing.md) pyo3-0.20.2/guide/src/advanced.md000064400000000000000000000013311046102023000145500ustar 00000000000000# Advanced topics ## FFI PyO3 exposes much of Python's C API through the `ffi` module. The C API is naturally unsafe and requires you to manage reference counts, errors and specific invariants yourself. Please refer to the [C API Reference Manual](https://docs.python.org/3/c-api/) and [The Rustonomicon](https://doc.rust-lang.org/nightly/nomicon/ffi.html) before using any function from that API. ## Memory management PyO3's `&PyAny` "owned references" and `Py` smart pointers are used to access memory stored in Python's heap. This memory sometimes lives for longer than expected because of differences in Rust and Python's memory models. See the chapter on [memory management](./memory.md) for more information. pyo3-0.20.2/guide/src/building_and_distribution/multiple_python_versions.md000064400000000000000000000121041046102023000254050ustar 00000000000000# Supporting multiple Python versions PyO3 supports all actively-supported Python 3 and PyPy versions. As much as possible, this is done internally to PyO3 so that your crate's code does not need to adapt to the differences between each version. However, as Python features grow and change between versions, PyO3 cannot a completely identical API for every Python version. This may require you to add conditional compilation to your crate or runtime checks for the Python version. This section of the guide first introduces the `pyo3-build-config` crate, which you can use as a `build-dependency` to add additional `#[cfg]` flags which allow you to support multiple Python versions at compile-time. Second, we'll show how to check the Python version at runtime. This can be useful when building for multiple versions with the `abi3` feature, where the Python API compiled against is not always the same as the one in use. ## Conditional compilation for different Python versions The `pyo3-build-config` exposes multiple [`#[cfg]` flags](https://doc.rust-lang.org/rust-by-example/attribute/cfg.html) which can be used to conditionally compile code for a given Python version. PyO3 itself depends on this crate, so by using it you can be sure that you are configured correctly for the Python version PyO3 is building against. This allows us to write code like the following ```rust,ignore #[cfg(Py_3_7)] fn function_only_supported_on_python_3_7_and_up() {} #[cfg(not(Py_3_8))] fn function_only_supported_before_python_3_8() {} #[cfg(not(Py_LIMITED_API))] fn function_incompatible_with_abi3_feature() {} ``` The following sections first show how to add these `#[cfg]` flags to your build process, and then cover some common patterns flags in a little more detail. To see a full reference of all the `#[cfg]` flags provided, see the [`pyo3-build-cfg` docs](https://docs.rs/pyo3-build-config). ### Using `pyo3-build-config` You can use the `#[cfg]` flags in just two steps: 1. Add `pyo3-build-config` with the [`resolve-config`](../features.md#resolve-config) feature enabled to your crate's build dependencies in `Cargo.toml`: ```toml [build-dependencies] pyo3-build-config = { {{#PYO3_CRATE_VERSION}}, features = ["resolve-config"] } ``` 2. Add a [`build.rs`](https://doc.rust-lang.org/cargo/reference/build-scripts.html) file to your crate with the following contents: ```rust,ignore fn main() { // If you have an existing build.rs file, just add this line to it. pyo3_build_config::use_pyo3_cfgs(); } ``` After these steps you are ready to annotate your code! ### Common usages of `pyo3-build-cfg` flags The `#[cfg]` flags added by `pyo3-build-cfg` can be combined with all of Rust's logic in the `#[cfg]` attribute to create very precise conditional code generation. The following are some common patterns implemented using these flags: ```text #[cfg(Py_3_7)] ``` This `#[cfg]` marks code that will only be present on Python 3.7 and upwards. There are similar options `Py_3_8`, `Py_3_9`, `Py_3_10` and so on for each minor version. ```text #[cfg(not(Py_3_7))] ``` This `#[cfg]` marks code that will only be present on Python versions before (but not including) Python 3.7. ```text #[cfg(not(Py_LIMITED_API))] ``` This `#[cfg]` marks code that is only available when building for the unlimited Python API (i.e. PyO3's `abi3` feature is not enabled). This might be useful if you want to ship your extension module as an `abi3` wheel and also allow users to compile it from source to make use of optimizations only possible with the unlimited API. ```text #[cfg(any(Py_3_9, not(Py_LIMITED_API)))] ``` This `#[cfg]` marks code which is available when running Python 3.9 or newer, or when using the unlimited API with an older Python version. Patterns like this are commonly seen on Python APIs which were added to the limited Python API in a specific minor version. ```text #[cfg(PyPy)] ``` This `#[cfg]` marks code which is running on PyPy. ## Checking the Python version at runtime When building with PyO3's `abi3` feature, your extension module will be compiled against a specific [minimum version](../building_and_distribution.html#minimum-python-version-for-abi3) of Python, but may be running on newer Python versions. For example with PyO3's `abi3-py38` feature, your extension will be compiled as if it were for Python 3.8. If you were using `pyo3-build-config`, `#[cfg(Py_3_8)]` would be present. Your user could freely install and run your abi3 extension on Python 3.9. There's no way to detect your user doing that at compile time, so instead you need to fall back to runtime checks. PyO3 provides the APIs [`Python::version()`] and [`Python::version_info()`] to query the running Python version. This allows you to do the following, for example: ```rust use pyo3::Python; Python::with_gil(|py| { // PyO3 supports Python 3.7 and up. assert!(py.version_info() >= (3, 7)); assert!(py.version_info() >= (3, 7, 0)); }); ``` [`Python::version()`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Python.html#method.version [`Python::version_info()`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Python.html#method.version_info pyo3-0.20.2/guide/src/building_and_distribution.md000064400000000000000000000660271046102023000202360ustar 00000000000000# Building and distribution This chapter of the guide goes into detail on how to build and distribute projects using PyO3. The way to achieve this is very different depending on whether the project is a Python module implemented in Rust, or a Rust binary embedding Python. For both types of project there are also common problems such as the Python version to build for and the [linker](https://en.wikipedia.org/wiki/Linker_(computing)) arguments to use. The material in this chapter is intended for users who have already read the PyO3 [README](#index.md). It covers in turn the choices that can be made for Python modules and for Rust binaries. There is also a section at the end about cross-compiling projects using PyO3. There is an additional sub-chapter dedicated to [supporting multiple Python versions](./building_and_distribution/multiple_python_versions.html). ## Configuring the Python version PyO3 uses a build script (backed by the [`pyo3-build-config`] crate) to determine the Python version and set the correct linker arguments. By default it will attempt to use the following in order: - Any active Python virtualenv. - The `python` executable (if it's a Python 3 interpreter). - The `python3` executable. You can override the Python interpreter by setting the `PYO3_PYTHON` environment variable, e.g. `PYO3_PYTHON=python3.7`, `PYO3_PYTHON=/usr/bin/python3.9`, or even a PyPy interpreter `PYO3_PYTHON=pypy3`. Once the Python interpreter is located, `pyo3-build-config` executes it to query the information in the `sysconfig` module which is needed to configure the rest of the compilation. To validate the configuration which PyO3 will use, you can run a compilation with the environment variable `PYO3_PRINT_CONFIG=1` set. An example output of doing this is shown below: ```console $ PYO3_PRINT_CONFIG=1 cargo build Compiling pyo3 v0.14.1 (/home/david/dev/pyo3) error: failed to run custom build command for `pyo3 v0.14.1 (/home/david/dev/pyo3)` Caused by: process didn't exit successfully: `/home/david/dev/pyo3/target/debug/build/pyo3-7a8cf4fe22e959b7/build-script-build` (exit status: 101) --- stdout cargo:rerun-if-env-changed=PYO3_CROSS cargo:rerun-if-env-changed=PYO3_CROSS_LIB_DIR cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_VERSION cargo:rerun-if-env-changed=PYO3_PRINT_CONFIG -- PYO3_PRINT_CONFIG=1 is set, printing configuration and halting compile -- implementation=CPython version=3.8 shared=true abi3=false lib_name=python3.8 lib_dir=/usr/lib executable=/usr/bin/python pointer_width=64 build_flags= suppress_build_script_link_lines=false ``` The `PYO3_ENVIRONMENT_SIGNATURE` environment variable can be used to trigger rebuilds when its value changes, it has no other effect. ### Advanced: config files If you save the above output config from `PYO3_PRINT_CONFIG` to a file, it is possible to manually override the contents and feed it back into PyO3 using the `PYO3_CONFIG_FILE` env var. If your build environment is unusual enough that PyO3's regular configuration detection doesn't work, using a config file like this will give you the flexibility to make PyO3 work for you. To see the full set of options supported, see the documentation for the [`InterpreterConfig` struct](https://docs.rs/pyo3-build-config/{{#PYO3_DOCS_VERSION}}/pyo3_build_config/struct.InterpreterConfig.html). ## Building Python extension modules Python extension modules need to be compiled differently depending on the OS (and architecture) that they are being compiled for. As well as multiple OSes (and architectures), there are also many different Python versions which are actively supported. Packages uploaded to [PyPI](https://pypi.org/) usually want to upload prebuilt "wheels" covering many OS/arch/version combinations so that users on all these different platforms don't have to compile the package themselves. Package vendors can opt-in to the "abi3" limited Python API which allows their wheels to be used on multiple Python versions, reducing the number of wheels they need to compile, but restricts the functionality they can use. There are many ways to go about this: it is possible to use `cargo` to build the extension module (along with some manual work, which varies with OS). The PyO3 ecosystem has two packaging tools, [`maturin`] and [`setuptools-rust`], which abstract over the OS difference and also support building wheels for PyPI upload. PyO3 has some Cargo features to configure projects for building Python extension modules: - The `extension-module` feature, which must be enabled when building Python extension modules. - The `abi3` feature and its version-specific `abi3-pyXY` companions, which are used to opt-in to the limited Python API in order to support multiple Python versions in a single wheel. This section describes each of these packaging tools before describing how to build manually without them. It then proceeds with an explanation of the `extension-module` feature. Finally, there is a section describing PyO3's `abi3` features. ### Packaging tools The PyO3 ecosystem has two main choices to abstract the process of developing Python extension modules: - [`maturin`] is a command-line tool to build, package and upload Python modules. It makes opinionated choices about project layout meaning it needs very little configuration. This makes it a great choice for users who are building a Python extension from scratch and don't need flexibility. - [`setuptools-rust`] is an add-on for `setuptools` which adds extra keyword arguments to the `setup.py` configuration file. It requires more configuration than `maturin`, however this gives additional flexibility for users adding Rust to an existing Python package that can't satisfy `maturin`'s constraints. Consult each project's documentation for full details on how to get started using them and how to upload wheels to PyPI. It should be noted that while `maturin` is able to build [manylinux](https://github.com/pypa/manylinux)-compliant wheels out-of-the-box, `setuptools-rust` requires a bit more effort, [relying on Docker](https://setuptools-rust.readthedocs.io/en/latest/building_wheels.html) for this purpose. There are also [`maturin-starter`] and [`setuptools-rust-starter`] examples in the PyO3 repository. ### Manual builds To build a PyO3-based Python extension manually, start by running `cargo build` as normal in a library project which uses PyO3's `extension-module` feature and has the [`cdylib` crate type](https://doc.rust-lang.org/cargo/reference/cargo-targets.html#the-crate-type-field). Once built, symlink (or copy) and rename the shared library from Cargo's `target/` directory to your desired output directory: - on macOS, rename `libyour_module.dylib` to `your_module.so`. - on Windows, rename `libyour_module.dll` to `your_module.pyd`. - on Linux, rename `libyour_module.so` to `your_module.so`. You can then open a Python shell in the output directory and you'll be able to run `import your_module`. If you're packaging your library for redistribution, you should indicated the Python interpreter your library is compiled for by including the [platform tag](#platform-tags) in its name. This prevents incompatible interpreters from trying to import your library. If you're compiling for PyPy you *must* include the platform tag, or PyPy will ignore the module. #### Bazel builds To use PyO3 with bazel one needs to manually configure PyO3, PyO3-ffi and PyO3-macros. In particular, one needs to make sure that it is compiled with the right python flags for the version you intend to use. For example see: 1. https://github.com/OliverFM/pytorch_with_gazelle -- for a minimal example of a repo that can use PyO3, PyTorch and Gazelle to generate python Build files. 2. https://github.com/TheButlah/rules_pyo3 -- which has more extensive support, but is outdated. #### Platform tags Rather than using just the `.so` or `.pyd` extension suggested above (depending on OS), you can prefix the shared library extension with a platform tag to indicate the interpreter it is compatible with. You can query your interpreter's platform tag from the `sysconfig` module. Some example outputs of this are seen below: ```bash # CPython 3.10 on macOS .cpython-310-darwin.so # PyPy 7.3 (Python 3.8) on Linux $ python -c 'import sysconfig; print(sysconfig.get_config_var("EXT_SUFFIX"))' .pypy38-pp73-x86_64-linux-gnu.so ``` So, for example, a valid module library name on CPython 3.10 for macOS is `your_module.cpython-310-darwin.so`, and its equivalent when compiled for PyPy 7.3 on Linux would be `your_module.pypy38-pp73-x86_64-linux-gnu.so`. See [PEP 3149](https://peps.python.org/pep-3149/) for more background on platform tags. #### macOS On macOS, because the `extension-module` feature disables linking to `libpython` ([see the next section](#the-extension-module-feature)), some additional linker arguments need to be set. `maturin` and `setuptools-rust` both pass these arguments for PyO3 automatically, but projects using manual builds will need to set these directly in order to support macOS. The easiest way to set the correct linker arguments is to add a [`build.rs`](https://doc.rust-lang.org/cargo/reference/build-scripts.html) with the following content: ```rust,ignore fn main() { pyo3_build_config::add_extension_module_link_args(); } ``` Remember to also add `pyo3-build-config` to the `build-dependencies` section in `Cargo.toml`. An alternative to using `pyo3-build-config` is add the following to a cargo configuration file (e.g. `.cargo/config.toml`): ```toml [target.x86_64-apple-darwin] rustflags = [ "-C", "link-arg=-undefined", "-C", "link-arg=dynamic_lookup", ] [target.aarch64-apple-darwin] rustflags = [ "-C", "link-arg=-undefined", "-C", "link-arg=dynamic_lookup", ] ``` Using the MacOS system python3 (`/usr/bin/python3`, as opposed to python installed via homebrew, pyenv, nix, etc.) may result in runtime errors such as `Library not loaded: @rpath/Python3.framework/Versions/3.8/Python3`. These can be resolved with another addition to `.cargo/config.toml`: ```toml [build] rustflags = [ "-C", "link-args=-Wl,-rpath,/Library/Developer/CommandLineTools/Library/Frameworks", ] ``` Alternatively, on rust >= 1.56, one can include in `build.rs`: ```rust fn main() { println!( "cargo:rustc-link-arg=-Wl,-rpath,/Library/Developer/CommandLineTools/Library/Frameworks" ); } ``` For more discussion on and workarounds for MacOS linking problems [see this issue](https://github.com/PyO3/pyo3/issues/1800#issuecomment-906786649). Finally, don't forget that on MacOS the `extension-module` feature will cause `cargo test` to fail without the `--no-default-features` flag (see [the FAQ](https://pyo3.rs/main/faq.html#i-cant-run-cargo-test-im-having-linker-issues-like-symbol-not-found-or-undefined-reference-to-_pyexc_systemerror)). ### The `extension-module` feature PyO3's `extension-module` feature is used to disable [linking](https://en.wikipedia.org/wiki/Linker_(computing)) to `libpython` on Unix targets. This is necessary because by default PyO3 links to `libpython`. This makes binaries, tests, and examples "just work". However, Python extensions on Unix must not link to libpython for [manylinux](https://www.python.org/dev/peps/pep-0513/) compliance. The downside of not linking to `libpython` is that binaries, tests, and examples (which usually embed Python) will fail to build. If you have an extension module as well as other outputs in a single project, you need to use optional Cargo features to disable the `extension-module` when you're not building the extension module. See [the FAQ](faq.md#i-cant-run-cargo-test-or-i-cant-build-in-a-cargo-workspace-im-having-linker-issues-like-symbol-not-found-or-undefined-reference-to-_pyexc_systemerror) for an example workaround. ### `Py_LIMITED_API`/`abi3` By default, Python extension modules can only be used with the same Python version they were compiled against. For example, an extension module built for Python 3.5 can't be imported in Python 3.8. [PEP 384](https://www.python.org/dev/peps/pep-0384/) introduced the idea of the limited Python API, which would have a stable ABI enabling extension modules built with it to be used against multiple Python versions. This is also known as `abi3`. The advantage of building extension modules using the limited Python API is that package vendors only need to build and distribute a single copy (for each OS / architecture), and users can install it on all Python versions from the [minimum version](#minimum-python-version-for-abi3) and up. The downside of this is that PyO3 can't use optimizations which rely on being compiled against a known exact Python version. It's up to you to decide whether this matters for your extension module. It's also possible to design your extension module such that you can distribute `abi3` wheels but allow users compiling from source to benefit from additional optimizations - see the [support for multiple python versions](./building_and_distribution/multiple_python_versions.html) section of this guide, in particular the `#[cfg(Py_LIMITED_API)]` flag. There are three steps involved in making use of `abi3` when building Python packages as wheels: 1. Enable the `abi3` feature in `pyo3`. This ensures `pyo3` only calls Python C-API functions which are part of the stable API, and on Windows also ensures that the project links against the correct shared object (no special behavior is required on other platforms): ```toml [dependencies] pyo3 = { {{#PYO3_CRATE_VERSION}}, features = ["abi3"] } ``` 2. Ensure that the built shared objects are correctly marked as `abi3`. This is accomplished by telling your build system that you're using the limited API. [`maturin`] >= 0.9.0 and [`setuptools-rust`] >= 0.11.4 support `abi3` wheels. See the [corresponding](https://github.com/PyO3/maturin/pull/353) [PRs](https://github.com/PyO3/setuptools-rust/pull/82) for more. 3. Ensure that the `.whl` is correctly marked as `abi3`. For projects using `setuptools`, this is accomplished by passing `--py-limited-api=cp3x` (where `x` is the minimum Python version supported by the wheel, e.g. `--py-limited-api=cp35` for Python 3.5) to `setup.py bdist_wheel`. #### Minimum Python version for `abi3` Because a single `abi3` wheel can be used with many different Python versions, PyO3 has feature flags `abi3-py37`, `abi3-py38`, `abi3-py39` etc. to set the minimum required Python version for your `abi3` wheel. For example, if you set the `abi3-py37` feature, your extension wheel can be used on all Python 3 versions from Python 3.7 and up. `maturin` and `setuptools-rust` will give the wheel a name like `my-extension-1.0-cp37-abi3-manylinux2020_x86_64.whl`. As your extension module may be run with multiple different Python versions you may occasionally find you need to check the Python version at runtime to customize behavior. See [the relevant section of this guide](./building_and_distribution/multiple_python_versions.html#checking-the-python-version-at-runtime) on supporting multiple Python versions at runtime. PyO3 is only able to link your extension module to abi3 version up to and including your host Python version. E.g., if you set `abi3-py38` and try to compile the crate with a host of Python 3.7, the build will fail. > Note: If you set more that one of these `abi3` version feature flags the lowest version always wins. For example, with both `abi3-py37` and `abi3-py38` set, PyO3 would build a wheel which supports Python 3.7 and up. #### Building `abi3` extensions without a Python interpreter As an advanced feature, you can build PyO3 wheel without calling Python interpreter with the environment variable `PYO3_NO_PYTHON` set. Also, if the build host Python interpreter is not found or is too old or otherwise unusable, PyO3 will still attempt to compile `abi3` extension modules after displaying a warning message. On Unix-like systems this works unconditionally; on Windows you must also set the `RUSTFLAGS` environment variable to contain `-L native=/path/to/python/libs` so that the linker can find `python3.lib`. If the `python3.dll` import library is not available, an experimental `generate-import-lib` crate feature may be enabled, and the required library will be created and used by PyO3 automatically. *Note*: MSVC targets require LLVM binutils (`llvm-dlltool`) to be available in `PATH` for the automatic import library generation feature to work. #### Missing features Due to limitations in the Python API, there are a few `pyo3` features that do not work when compiling for `abi3`. These are: - `#[pyo3(text_signature = "...")]` does not work on classes until Python 3.10 or greater. - The `dict` and `weakref` options on classes are not supported until Python 3.9 or greater. - The buffer API is not supported until Python 3.11 or greater. - Optimizations which rely on knowledge of the exact Python version compiled against. ## Embedding Python in Rust If you want to embed the Python interpreter inside a Rust program, there are two modes in which this can be done: dynamically and statically. We'll cover each of these modes in the following sections. Each of them affect how you must distribute your program. Instead of learning how to do this yourself, you might want to consider using a project like [PyOxidizer] to ship your application and all of its dependencies in a single file. PyO3 automatically switches between the two linking modes depending on whether the Python distribution you have configured PyO3 to use ([see above](#python-version)) contains a shared library or a static library. The static library is most often seen in Python distributions compiled from source without the `--enable-shared` configuration option. For example, this is the default for `pyenv` on macOS. ### Dynamically embedding the Python interpreter Embedding the Python interpreter dynamically is much easier than doing so statically. This is done by linking your program against a Python shared library (such as `libpython.3.9.so` on UNIX, or `python39.dll` on Windows). The implementation of the Python interpreter resides inside the shared library. This means that when the OS runs your Rust program it also needs to be able to find the Python shared library. This mode of embedding works well for Rust tests which need access to the Python interpreter. It is also great for Rust software which is installed inside a Python virtualenv, because the virtualenv sets up appropriate environment variables to locate the correct Python shared library. For distributing your program to non-technical users, you will have to consider including the Python shared library in your distribution as well as setting up wrapper scripts to set the right environment variables (such as `LD_LIBRARY_PATH` on UNIX, or `PATH` on Windows). Note that PyPy cannot be embedded in Rust (or any other software). Support for this is tracked on the [PyPy issue tracker](https://foss.heptapod.net/pypy/pypy/-/issues/3286). ### Statically embedding the Python interpreter Embedding the Python interpreter statically means including the contents of a Python static library directly inside your Rust binary. This means that to distribute your program you only need to ship your binary file: it contains the Python interpreter inside the binary! On Windows static linking is almost never done, so Python distributions don't usually include a static library. The information below applies only to UNIX. The Python static library is usually called `libpython.a`. Static linking has a lot of complications, listed below. For these reasons PyO3 does not yet have first-class support for this embedding mode. See [issue 416 on PyO3's GitHub](https://github.com/PyO3/pyo3/issues/416) for more information and to discuss any issues you encounter. The [`auto-initialize`](features.md#auto-initialize) feature is deliberately disabled when embedding the interpreter statically because this is often unintentionally done by new users to PyO3 running test programs. Trying out PyO3 is much easier using dynamic embedding. The known complications are: - To import compiled extension modules (such as other Rust extension modules, or those written in C), your binary must have the correct linker flags set during compilation to export the original contents of `libpython.a` so that extensions can use them (e.g. `-Wl,--export-dynamic`). - The C compiler and flags which were used to create `libpython.a` must be compatible with your Rust compiler and flags, else you will experience compilation failures. Significantly different compiler versions may see errors like this: ```text lto1: fatal error: bytecode stream in file 'rust-numpy/target/release/deps/libpyo3-6a7fb2ed970dbf26.rlib' generated with LTO version 6.0 instead of the expected 6.2 ``` Mismatching flags may lead to errors like this: ```text /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libpython3.9.a(zlibmodule.o): relocation R_X86_64_32 against `.data' can not be used when making a PIE object; recompile with -fPIE ``` If you encounter these or other complications when linking the interpreter statically, discuss them on [issue 416 on PyO3's GitHub](https://github.com/PyO3/pyo3/issues/416). It is hoped that eventually that discussion will contain enough information and solutions that PyO3 can offer first-class support for static embedding. ### Import your module when embedding the Python interpreter When you run your Rust binary with an embedded interpreter, any `#[pymodule]` created modules won't be accessible to import unless added to a table called `PyImport_Inittab` before the embedded interpreter is initialized. This will cause Python statements in your embedded interpreter such as `import your_new_module` to fail. You can call the macro [`append_to_inittab`]({{#PYO3_DOCS_URL}}/pyo3/macro.append_to_inittab.html) with your module before initializing the Python interpreter to add the module function into that table. (The Python interpreter will be initialized by calling `prepare_freethreaded_python`, `with_embedded_python_interpreter`, or `Python::with_gil` with the [`auto-initialize`](features.md#auto-initialize) feature enabled.) ## Cross Compiling Thanks to Rust's great cross-compilation support, cross-compiling using PyO3 is relatively straightforward. To get started, you'll need a few pieces of software: * A toolchain for your target. * The appropriate options in your Cargo `.config` for the platform you're targeting and the toolchain you are using. * A Python interpreter that's already been compiled for your target (optional when building "abi3" extension modules). * A Python interpreter that is built for your host and available through the `PATH` or setting the [`PYO3_PYTHON`](#python-version) variable (optional when building "abi3" extension modules). After you've obtained the above, you can build a cross-compiled PyO3 module by using Cargo's `--target` flag. PyO3's build script will detect that you are attempting a cross-compile based on your host machine and the desired target. When cross-compiling, PyO3's build script cannot execute the target Python interpreter to query the configuration, so there are a few additional environment variables you may need to set: * `PYO3_CROSS`: If present this variable forces PyO3 to configure as a cross-compilation. * `PYO3_CROSS_LIB_DIR`: This variable can be set to the directory containing the target's libpython DSO and the associated `_sysconfigdata*.py` file for Unix-like targets, or the Python DLL import libraries for the Windows target. This variable is only needed when the output binary must link to libpython explicitly (e.g. when targeting Windows and Android or embedding a Python interpreter), or when it is absolutely required to get the interpreter configuration from `_sysconfigdata*.py`. * `PYO3_CROSS_PYTHON_VERSION`: Major and minor version (e.g. 3.9) of the target Python installation. This variable is only needed if PyO3 cannot determine the version to target from `abi3-py3*` features, or if `PYO3_CROSS_LIB_DIR` is not set, or if there are multiple versions of Python present in `PYO3_CROSS_LIB_DIR`. * `PYO3_CROSS_PYTHON_IMPLEMENTATION`: Python implementation name ("CPython" or "PyPy") of the target Python installation. CPython is assumed by default when this variable is not set, unless `PYO3_CROSS_LIB_DIR` is set for a Unix-like target and PyO3 can get the interpreter configuration from `_sysconfigdata*.py`. An experimental `pyo3` crate feature `generate-import-lib` enables the user to cross-compile extension modules for Windows targets without setting the `PYO3_CROSS_LIB_DIR` environment variable or providing any Windows Python library files. It uses an external [`python3-dll-a`] crate to generate import libraries for the Python DLL for MinGW-w64 and MSVC compile targets. `python3-dll-a` uses the binutils `dlltool` program to generate DLL import libraries for MinGW-w64 targets. It is possible to override the default `dlltool` command name for the cross target by setting `PYO3_MINGW_DLLTOOL` environment variable. *Note*: MSVC targets require LLVM binutils or MSVC build tools to be available on the host system. More specifically, `python3-dll-a` requires `llvm-dlltool` or `lib.exe` executable to be present in `PATH` when targeting `*-pc-windows-msvc`. The Zig compiler executable can be used in place of `llvm-dlltool` when the `ZIG_COMMAND` environment variable is set to the installed Zig program name (`"zig"` or `"python -m ziglang"`). An example might look like the following (assuming your target's sysroot is at `/home/pyo3/cross/sysroot` and that your target is `armv7`): ```sh export PYO3_CROSS_LIB_DIR="/home/pyo3/cross/sysroot/usr/lib" cargo build --target armv7-unknown-linux-gnueabihf ``` If there are multiple python versions at the cross lib directory and you cannot set a more precise location to include both the `libpython` DSO and `_sysconfigdata*.py` files, you can set the required version: ```sh export PYO3_CROSS_PYTHON_VERSION=3.8 export PYO3_CROSS_LIB_DIR="/home/pyo3/cross/sysroot/usr/lib" cargo build --target armv7-unknown-linux-gnueabihf ``` Or another example with the same sys root but building for Windows: ```sh export PYO3_CROSS_PYTHON_VERSION=3.9 export PYO3_CROSS_LIB_DIR="/home/pyo3/cross/sysroot/usr/lib" cargo build --target x86_64-pc-windows-gnu ``` Any of the `abi3-py3*` features can be enabled instead of setting `PYO3_CROSS_PYTHON_VERSION` in the above examples. `PYO3_CROSS_LIB_DIR` can often be omitted when cross compiling extension modules for Unix and macOS targets, or when cross compiling extension modules for Windows and the experimental `generate-import-lib` crate feature is enabled. The following resources may also be useful for cross-compiling: - [github.com/japaric/rust-cross](https://github.com/japaric/rust-cross) is a primer on cross compiling Rust. - [github.com/rust-embedded/cross](https://github.com/rust-embedded/cross) uses Docker to make Rust cross-compilation easier. [`pyo3-build-config`]: https://github.com/PyO3/pyo3/tree/main/pyo3-build-config [`maturin-starter`]: https://github.com/PyO3/pyo3/tree/main/examples/maturin-starter [`setuptools-rust-starter`]: https://github.com/PyO3/pyo3/tree/main/examples/setuptools-rust-starter [`maturin`]: https://github.com/PyO3/maturin [`setuptools-rust`]: https://github.com/PyO3/setuptools-rust [PyOxidizer]: https://github.com/indygreg/PyOxidizer [`python3-dll-a`]: https://docs.rs/python3-dll-a/latest/python3_dll_a/ pyo3-0.20.2/guide/src/changelog.md000064400000000000000000000000401046102023000147260ustar 00000000000000{{#include ../../CHANGELOG.md}} pyo3-0.20.2/guide/src/class/call.md000064400000000000000000000103611046102023000150260ustar 00000000000000# Emulating callable objects Classes can be callable if they have a `#[pymethod]` named `__call__`. This allows instances of a class to behave similar to functions. This method's signature must look like `__call__(, ...) -> object` - here, any argument list can be defined as for normal pymethods ### Example: Implementing a call counter The following pyclass is a basic decorator - its constructor takes a Python object as argument and calls that object when called. An equivalent Python implementation is linked at the end. An example crate containing this pyclass can be found [here](https://github.com/PyO3/pyo3/tree/main/examples/decorator) ```rust,ignore {{#include ../../../examples/decorator/src/lib.rs}} ``` Python code: ```python {{#include ../../../examples/decorator/tests/example.py}} ``` Output: ```text say_hello has been called 1 time(s). hello say_hello has been called 2 time(s). hello say_hello has been called 3 time(s). hello say_hello has been called 4 time(s). hello ``` ### Pure Python implementation A Python implementation of this looks similar to the Rust version: ```python class Counter: def __init__(self, wraps): self.count = 0 self.wraps = wraps def __call__(self, *args, **kwargs): self.count += 1 print(f"{self.wraps.__name__} has been called {self.count} time(s)") self.wraps(*args, **kwargs) ``` Note that it can also be implemented as a higher order function: ```python def Counter(wraps): count = 0 def call(*args, **kwargs): nonlocal count count += 1 print(f"{wraps.__name__} has been called {count} time(s)") return wraps(*args, **kwargs) return call ``` ### What is the `Cell` for? A [previous implementation] used a normal `u64`, which meant it required a `&mut self` receiver to update the count: ```rust,ignore #[pyo3(signature = (*args, **kwargs))] fn __call__( &mut self, py: Python<'_>, args: &PyTuple, kwargs: Option<&PyDict>, ) -> PyResult> { self.count += 1; let name = self.wraps.getattr(py, "__name__")?; println!("{} has been called {} time(s).", name, self.count); // After doing something, we finally forward the call to the wrapped function let ret = self.wraps.call(py, args, kwargs)?; // We could do something with the return value of // the function before returning it Ok(ret) } ``` The problem with this is that the `&mut self` receiver means PyO3 has to borrow it exclusively, and hold this borrow across the`self.wraps.call(py, args, kwargs)` call. This call returns control to the user's Python code which is free to call arbitrary things, *including* the decorated function. If that happens PyO3 is unable to create a second unique borrow and will be forced to raise an exception. As a result, something innocent like this will raise an exception: ```py @Counter def say_hello(): if say_hello.count < 2: print(f"hello from decorator") say_hello() # RuntimeError: Already borrowed ``` The implementation in this chapter fixes that by never borrowing exclusively; all the methods take `&self` as receivers, of which multiple may exist simultaneously. This requires a shared counter and the easiest way to do that is to use [`Cell`], so that's what is used here. This shows the dangers of running arbitrary Python code - note that "running arbitrary Python code" can be far more subtle than the example above: - Python's asynchronous executor may park the current thread in the middle of Python code, even in Python code that *you* control, and let other Python code run. - Dropping arbitrary Python objects may invoke destructors defined in Python (`__del__` methods). - Calling Python's C-api (most PyO3 apis call C-api functions internally) may raise exceptions, which may allow Python code in signal handlers to run. This is especially important if you are writing unsafe code; Python code must never be able to cause undefined behavior. You must ensure that your Rust code is in a consistent state before doing any of the above things. [previous implementation]: https://github.com/PyO3/pyo3/discussions/2598 "Thread Safe Decorator · Discussion #2598 · PyO3/pyo3" [`Cell`]: https://doc.rust-lang.org/std/cell/struct.Cell.html "Cell in std::cell - Rust" pyo3-0.20.2/guide/src/class/numeric.md000064400000000000000000000302461046102023000155610ustar 00000000000000# Emulating numeric types At this point we have a `Number` class that we can't actually do any math on! Before proceeding, we should think about how we want to handle overflows. There are three obvious solutions: - We can have infinite precision just like Python's `int`. However that would be quite boring - we'd be reinventing the wheel. - We can raise exceptions whenever `Number` overflows, but that makes the API painful to use. - We can wrap around the boundary of `i32`. This is the approach we'll take here. To do that we'll just forward to `i32`'s `wrapping_*` methods. ### Fixing our constructor Let's address the first overflow, in `Number`'s constructor: ```python from my_module import Number n = Number(1 << 1337) ``` ```text Traceback (most recent call last): File "example.py", line 3, in n = Number(1 << 1337) OverflowError: Python int too large to convert to C long ``` Instead of relying on the default [`FromPyObject`] extraction to parse arguments, we can specify our own extraction function, using the `#[pyo3(from_py_with = "...")]` attribute. Unfortunately PyO3 doesn't provide a way to wrap Python integers out of the box, but we can do a Python call to mask it and cast it to an `i32`. ```rust # #![allow(dead_code)] use pyo3::prelude::*; fn wrap(obj: &PyAny) -> Result { let val = obj.call_method1("__and__", (0xFFFFFFFF_u32,))?; let val: u32 = val.extract()?; // 👇 This intentionally overflows! Ok(val as i32) } ``` We also add documentation, via `///` comments, which are visible to Python users. ```rust # #![allow(dead_code)] use pyo3::prelude::*; fn wrap(obj: &PyAny) -> Result { let val = obj.call_method1("__and__", (0xFFFFFFFF_u32,))?; let val: u32 = val.extract()?; Ok(val as i32) } /// Did you ever hear the tragedy of Darth Signed The Overfloweth? I thought not. /// It's not a story C would tell you. It's a Rust legend. #[pyclass(module = "my_module")] struct Number(i32); #[pymethods] impl Number { #[new] fn new(#[pyo3(from_py_with = "wrap")] value: i32) -> Self { Self(value) } } ``` With that out of the way, let's implement some operators: ```rust use pyo3::exceptions::{PyZeroDivisionError, PyValueError}; # use pyo3::prelude::*; # # #[pyclass] # struct Number(i32); # #[pymethods] impl Number { fn __add__(&self, other: &Self) -> Self { Self(self.0.wrapping_add(other.0)) } fn __sub__(&self, other: &Self) -> Self { Self(self.0.wrapping_sub(other.0)) } fn __mul__(&self, other: &Self) -> Self { Self(self.0.wrapping_mul(other.0)) } fn __truediv__(&self, other: &Self) -> PyResult { match self.0.checked_div(other.0) { Some(i) => Ok(Self(i)), None => Err(PyZeroDivisionError::new_err("division by zero")), } } fn __floordiv__(&self, other: &Self) -> PyResult { match self.0.checked_div(other.0) { Some(i) => Ok(Self(i)), None => Err(PyZeroDivisionError::new_err("division by zero")), } } fn __rshift__(&self, other: &Self) -> PyResult { match other.0.try_into() { Ok(rhs) => Ok(Self(self.0.wrapping_shr(rhs))), Err(_) => Err(PyValueError::new_err("negative shift count")), } } fn __lshift__(&self, other: &Self) -> PyResult { match other.0.try_into() { Ok(rhs) => Ok(Self(self.0.wrapping_shl(rhs))), Err(_) => Err(PyValueError::new_err("negative shift count")), } } } ``` ### Unary arithmetic operations ```rust # use pyo3::prelude::*; # # #[pyclass] # struct Number(i32); # #[pymethods] impl Number { fn __pos__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> { slf } fn __neg__(&self) -> Self { Self(-self.0) } fn __abs__(&self) -> Self { Self(self.0.abs()) } fn __invert__(&self) -> Self { Self(!self.0) } } ``` ### Support for the `complex()`, `int()` and `float()` built-in functions. ```rust # use pyo3::prelude::*; # # #[pyclass] # struct Number(i32); # use pyo3::types::PyComplex; #[pymethods] impl Number { fn __int__(&self) -> i32 { self.0 } fn __float__(&self) -> f64 { self.0 as f64 } fn __complex__<'py>(&self, py: Python<'py>) -> &'py PyComplex { PyComplex::from_doubles(py, self.0 as f64, 0.0) } } ``` We do not implement the in-place operations like `__iadd__` because we do not wish to mutate `Number`. Similarly we're not interested in supporting operations with different types, so we do not implement the reflected operations like `__radd__` either. Now Python can use our `Number` class: ```python from my_module import Number def hash_djb2(s: str): ''' A version of Daniel J. Bernstein's djb2 string hashing algorithm Like many hashing algorithms, it relies on integer wrapping. ''' n = Number(0) five = Number(5) for x in s: n = Number(ord(x)) + ((n << five) - n) return n assert hash_djb2('l50_50') == Number(-1152549421) ``` ### Final code ```rust use std::collections::hash_map::DefaultHasher; use std::hash::{Hash, Hasher}; use std::convert::TryInto; use pyo3::exceptions::{PyValueError, PyZeroDivisionError}; use pyo3::prelude::*; use pyo3::class::basic::CompareOp; use pyo3::types::PyComplex; fn wrap(obj: &PyAny) -> Result { let val = obj.call_method1("__and__", (0xFFFFFFFF_u32,))?; let val: u32 = val.extract()?; Ok(val as i32) } /// Did you ever hear the tragedy of Darth Signed The Overfloweth? I thought not. /// It's not a story C would tell you. It's a Rust legend. #[pyclass(module = "my_module")] struct Number(i32); #[pymethods] impl Number { #[new] fn new(#[pyo3(from_py_with = "wrap")] value: i32) -> Self { Self(value) } fn __repr__(slf: &PyCell) -> PyResult { // Get the class name dynamically in case `Number` is subclassed let class_name: &str = slf.get_type().name()?; Ok(format!("{}({})", class_name, slf.borrow().0)) } fn __str__(&self) -> String { self.0.to_string() } fn __hash__(&self) -> u64 { let mut hasher = DefaultHasher::new(); self.0.hash(&mut hasher); hasher.finish() } fn __richcmp__(&self, other: &Self, op: CompareOp) -> PyResult { match op { CompareOp::Lt => Ok(self.0 < other.0), CompareOp::Le => Ok(self.0 <= other.0), CompareOp::Eq => Ok(self.0 == other.0), CompareOp::Ne => Ok(self.0 != other.0), CompareOp::Gt => Ok(self.0 > other.0), CompareOp::Ge => Ok(self.0 >= other.0), } } fn __bool__(&self) -> bool { self.0 != 0 } fn __add__(&self, other: &Self) -> Self { Self(self.0.wrapping_add(other.0)) } fn __sub__(&self, other: &Self) -> Self { Self(self.0.wrapping_sub(other.0)) } fn __mul__(&self, other: &Self) -> Self { Self(self.0.wrapping_mul(other.0)) } fn __truediv__(&self, other: &Self) -> PyResult { match self.0.checked_div(other.0) { Some(i) => Ok(Self(i)), None => Err(PyZeroDivisionError::new_err("division by zero")), } } fn __floordiv__(&self, other: &Self) -> PyResult { match self.0.checked_div(other.0) { Some(i) => Ok(Self(i)), None => Err(PyZeroDivisionError::new_err("division by zero")), } } fn __rshift__(&self, other: &Self) -> PyResult { match other.0.try_into() { Ok(rhs) => Ok(Self(self.0.wrapping_shr(rhs))), Err(_) => Err(PyValueError::new_err("negative shift count")), } } fn __lshift__(&self, other: &Self) -> PyResult { match other.0.try_into() { Ok(rhs) => Ok(Self(self.0.wrapping_shl(rhs))), Err(_) => Err(PyValueError::new_err("negative shift count")), } } fn __xor__(&self, other: &Self) -> Self { Self(self.0 ^ other.0) } fn __or__(&self, other: &Self) -> Self { Self(self.0 | other.0) } fn __and__(&self, other: &Self) -> Self { Self(self.0 & other.0) } fn __int__(&self) -> i32 { self.0 } fn __float__(&self) -> f64 { self.0 as f64 } fn __complex__<'py>(&self, py: Python<'py>) -> &'py PyComplex { PyComplex::from_doubles(py, self.0 as f64, 0.0) } } #[pymodule] fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_class::()?; Ok(()) } # const SCRIPT: &'static str = r#" # def hash_djb2(s: str): # n = Number(0) # five = Number(5) # # for x in s: # n = Number(ord(x)) + ((n << five) - n) # return n # # assert hash_djb2('l50_50') == Number(-1152549421) # assert hash_djb2('logo') == Number(3327403) # assert hash_djb2('horizon') == Number(1097468315) # # # assert Number(2) + Number(2) == Number(4) # assert Number(2) + Number(2) != Number(5) # # assert Number(13) - Number(7) == Number(6) # assert Number(13) - Number(-7) == Number(20) # # assert Number(13) / Number(7) == Number(1) # assert Number(13) // Number(7) == Number(1) # # assert Number(13) * Number(7) == Number(13*7) # # assert Number(13) > Number(7) # assert Number(13) < Number(20) # assert Number(13) == Number(13) # assert Number(13) >= Number(7) # assert Number(13) <= Number(20) # assert Number(13) == Number(13) # # # assert (True if Number(1) else False) # assert (False if Number(0) else True) # # # assert int(Number(13)) == 13 # assert float(Number(13)) == 13 # assert Number.__doc__ == "Did you ever hear the tragedy of Darth Signed The Overfloweth? I thought not.\nIt's not a story C would tell you. It's a Rust legend." # assert Number(12345234523452) == Number(1498514748) # try: # import inspect # assert inspect.signature(Number).__str__() == '(value)' # except ValueError: # # Not supported with `abi3` before Python 3.10 # pass # assert Number(1337).__str__() == '1337' # assert Number(1337).__repr__() == 'Number(1337)' "#; # # use pyo3::PyTypeInfo; # # fn main() -> PyResult<()> { # Python::with_gil(|py| -> PyResult<()> { # let globals = PyModule::import(py, "__main__")?.dict(); # globals.set_item("Number", Number::type_object(py))?; # # py.run(SCRIPT, Some(globals), None)?; # Ok(()) # }) # } ``` ## Appendix: Writing some unsafe code At the beginning of this chapter we said that PyO3 doesn't provide a way to wrap Python integers out of the box but that's a half truth. There's not a PyO3 API for it, but there's a Python C API function that does: ```c unsigned long PyLong_AsUnsignedLongMask(PyObject *obj) ``` We can call this function from Rust by using [`pyo3::ffi::PyLong_AsUnsignedLongMask`]. This is an *unsafe* function, which means we have to use an unsafe block to call it and take responsibility for upholding the contracts of this function. Let's review those contracts: - The GIL must be held. If it's not, calling this function causes a data race. - The pointer must be valid, i.e. it must be properly aligned and point to a valid Python object. Let's create that helper function. The signature has to be `fn(&PyAny) -> PyResult`. - `&PyAny` represents a checked borrowed reference, so the pointer derived from it is valid (and not null). - Whenever we have borrowed references to Python objects in scope, it is guaranteed that the GIL is held. This reference is also where we can get a [`Python`] token to use in our call to [`PyErr::take`]. ```rust # #![allow(dead_code)] use std::os::raw::c_ulong; use pyo3::prelude::*; use pyo3::ffi; fn wrap(obj: &PyAny) -> Result { let py: Python<'_> = obj.py(); unsafe { let ptr = obj.as_ptr(); let ret: c_ulong = ffi::PyLong_AsUnsignedLongMask(ptr); if ret == c_ulong::MAX { if let Some(err) = PyErr::take(py) { return Err(err); } } Ok(ret as i32) } } ``` [`PyErr::take`]: {{#PYO3_DOCS_URL}}/pyo3/prelude/struct.PyErr.html#method.take [`Python`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Python.html [`FromPyObject`]: {{#PYO3_DOCS_URL}}/pyo3/conversion/trait.FromPyObject.html [`pyo3::ffi::PyLong_AsUnsignedLongMask`]: {{#PYO3_DOCS_URL}}/pyo3/ffi/fn.PyLong_AsUnsignedLongMask.html pyo3-0.20.2/guide/src/class/object.md000064400000000000000000000167621046102023000153740ustar 00000000000000# Basic object customization Recall the `Number` class from the previous chapter: ```rust use pyo3::prelude::*; #[pyclass] struct Number(i32); #[pymethods] impl Number { #[new] fn new(value: i32) -> Self { Self(value) } } #[pymodule] fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_class::()?; Ok(()) } ``` At this point Python code can import the module, access the class and create class instances - but nothing else. ```python from my_module import Number n = Number(5) print(n) ``` ```text ``` ### String representations It can't even print an user-readable representation of itself! We can fix that by defining the `__repr__` and `__str__` methods inside a `#[pymethods]` block. We do this by accessing the value contained inside `Number`. ```rust # use pyo3::prelude::*; # # #[pyclass] # struct Number(i32); # #[pymethods] impl Number { // For `__repr__` we want to return a string that Python code could use to recreate // the `Number`, like `Number(5)` for example. fn __repr__(&self) -> String { // We use the `format!` macro to create a string. Its first argument is a // format string, followed by any number of parameters which replace the // `{}`'s in the format string. // // 👇 Tuple field access in Rust uses a dot format!("Number({})", self.0) } // `__str__` is generally used to create an "informal" representation, so we // just forward to `i32`'s `ToString` trait implementation to print a bare number. fn __str__(&self) -> String { self.0.to_string() } } ``` #### Accessing the class name In the `__repr__`, we used a hard-coded class name. This is sometimes not ideal, because if the class is subclassed in Python, we would like the repr to reflect the subclass name. This is typically done in Python code by accessing `self.__class__.__name__`. In order to be able to access the Python type information *and* the Rust struct, we need to use a `PyCell` as the `self` argument. ```rust # use pyo3::prelude::*; # # #[pyclass] # struct Number(i32); # #[pymethods] impl Number { fn __repr__(slf: &PyCell) -> PyResult { // This is the equivalent of `self.__class__.__name__` in Python. let class_name: &str = slf.get_type().name()?; // To access fields of the Rust struct, we need to borrow the `PyCell`. Ok(format!("{}({})", class_name, slf.borrow().0)) } } ``` ### Hashing Let's also implement hashing. We'll just hash the `i32`. For that we need a [`Hasher`]. The one provided by `std` is [`DefaultHasher`], which uses the [SipHash] algorithm. ```rust use std::collections::hash_map::DefaultHasher; // Required to call the `.hash` and `.finish` methods, which are defined on traits. use std::hash::{Hash, Hasher}; # use pyo3::prelude::*; # # #[pyclass] # struct Number(i32); # #[pymethods] impl Number { fn __hash__(&self) -> u64 { let mut hasher = DefaultHasher::new(); self.0.hash(&mut hasher); hasher.finish() } } ``` > **Note**: When implementing `__hash__` and comparisons, it is important that the following property holds: > > ```text > k1 == k2 -> hash(k1) == hash(k2) > ``` > > In other words, if two keys are equal, their hashes must also be equal. In addition you must take > care that your classes' hash doesn't change during its lifetime. In this tutorial we do that by not > letting Python code change our `Number` class. In other words, it is immutable. > > By default, all `#[pyclass]` types have a default hash implementation from Python. > Types which should not be hashable can override this by setting `__hash__` to None. > This is the same mechanism as for a pure-Python class. This is done like so: > > ```rust > # use pyo3::prelude::*; > #[pyclass] > struct NotHashable {} > > #[pymethods] > impl NotHashable { > #[classattr] > const __hash__: Option> = None; > } > ``` ### Comparisons PyO3 supports the usual magic comparison methods available in Python such as `__eq__`, `__lt__` and so on. It is also possible to support all six operations at once with `__richcmp__`. This method will be called with a value of `CompareOp` depending on the operation. ```rust use pyo3::class::basic::CompareOp; # use pyo3::prelude::*; # # #[pyclass] # struct Number(i32); # #[pymethods] impl Number { fn __richcmp__(&self, other: &Self, op: CompareOp) -> PyResult { match op { CompareOp::Lt => Ok(self.0 < other.0), CompareOp::Le => Ok(self.0 <= other.0), CompareOp::Eq => Ok(self.0 == other.0), CompareOp::Ne => Ok(self.0 != other.0), CompareOp::Gt => Ok(self.0 > other.0), CompareOp::Ge => Ok(self.0 >= other.0), } } } ``` If you obtain the result by comparing two Rust values, as in this example, you can take a shortcut using `CompareOp::matches`: ```rust use pyo3::class::basic::CompareOp; # use pyo3::prelude::*; # # #[pyclass] # struct Number(i32); # #[pymethods] impl Number { fn __richcmp__(&self, other: &Self, op: CompareOp) -> bool { op.matches(self.0.cmp(&other.0)) } } ``` It checks that the `std::cmp::Ordering` obtained from Rust's `Ord` matches the given `CompareOp`. Alternatively, you can implement just equality using `__eq__`: ```rust # use pyo3::prelude::*; # # #[pyclass] # struct Number(i32); # #[pymethods] impl Number { fn __eq__(&self, other: &Self) -> bool { self.0 == other.0 } } # fn main() -> PyResult<()> { # Python::with_gil(|py| { # let x = PyCell::new(py, Number(4))?; # let y = PyCell::new(py, Number(4))?; # assert!(x.eq(y)?); # assert!(!x.ne(y)?); # Ok(()) # }) # } ``` ### Truthyness We'll consider `Number` to be `True` if it is nonzero: ```rust # use pyo3::prelude::*; # # #[pyclass] # struct Number(i32); # #[pymethods] impl Number { fn __bool__(&self) -> bool { self.0 != 0 } } ``` ### Final code ```rust use std::collections::hash_map::DefaultHasher; use std::hash::{Hash, Hasher}; use pyo3::prelude::*; use pyo3::class::basic::CompareOp; #[pyclass] struct Number(i32); #[pymethods] impl Number { #[new] fn new(value: i32) -> Self { Self(value) } fn __repr__(slf: &PyCell) -> PyResult { let class_name: &str = slf.get_type().name()?; Ok(format!("{}({})", class_name, slf.borrow().0)) } fn __str__(&self) -> String { self.0.to_string() } fn __hash__(&self) -> u64 { let mut hasher = DefaultHasher::new(); self.0.hash(&mut hasher); hasher.finish() } fn __richcmp__(&self, other: &Self, op: CompareOp) -> PyResult { match op { CompareOp::Lt => Ok(self.0 < other.0), CompareOp::Le => Ok(self.0 <= other.0), CompareOp::Eq => Ok(self.0 == other.0), CompareOp::Ne => Ok(self.0 != other.0), CompareOp::Gt => Ok(self.0 > other.0), CompareOp::Ge => Ok(self.0 >= other.0), } } fn __bool__(&self) -> bool { self.0 != 0 } } #[pymodule] fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_class::()?; Ok(()) } ``` [`Hash`]: https://doc.rust-lang.org/std/hash/trait.Hash.html [`Hasher`]: https://doc.rust-lang.org/std/hash/trait.Hasher.html [`DefaultHasher`]: https://doc.rust-lang.org/std/collections/hash_map/struct.DefaultHasher.html [SipHash]: https://en.wikipedia.org/wiki/SipHash pyo3-0.20.2/guide/src/class/protocols.md000064400000000000000000000453541046102023000161510ustar 00000000000000# Magic methods and slots Python's object model defines several protocols for different object behavior, such as the sequence, mapping, and number protocols. You may be familiar with implementing these protocols in Python classes by "magic" methods, such as `__str__` or `__repr__`. Because of the double-underscores surrounding their name, these are also known as "dunder" methods. In the Python C-API which PyO3 is implemented upon, many of these magic methods have to be placed into special "slots" on the class type object, as covered in the previous section. If a function name in `#[pymethods]` is a recognised magic method, it will be automatically placed into the correct slot in the Python type object. The function name is taken from the usual rules for naming `#[pymethods]`: the `#[pyo3(name = "...")]` attribute is used if present, otherwise the Rust function name is used. The magic methods handled by PyO3 are very similar to the standard Python ones on [this page](https://docs.python.org/3/reference/datamodel.html#special-method-names) - in particular they are the the subset which have slots as [defined here](https://docs.python.org/3/c-api/typeobj.html). Some of the slots do not have a magic method in Python, which leads to a few additional magic methods defined only in PyO3: - Magic methods for garbage collection - Magic methods for the buffer protocol When PyO3 handles a magic method, a couple of changes apply compared to other `#[pymethods]`: - The Rust function signature is restricted to match the magic method. - The `#[pyo3(signature = (...)]` and `#[pyo3(text_signature = "...")]` attributes are not allowed. The following sections list of all magic methods PyO3 currently handles. The given signatures should be interpreted as follows: - All methods take a receiver as first argument, shown as ``. It can be `&self`, `&mut self` or a `PyCell` reference like `self_: PyRef<'_, Self>` and `self_: PyRefMut<'_, Self>`, as described [here](../class.md#inheritance). - An optional `Python<'py>` argument is always allowed as the first argument. - Return values can be optionally wrapped in `PyResult`. - `object` means that any type is allowed that can be extracted from a Python object (if argument) or converted to a Python object (if return value). - Other types must match what's given, e.g. `pyo3::basic::CompareOp` for `__richcmp__`'s second argument. - For the comparison and arithmetic methods, extraction errors are not propagated as exceptions, but lead to a return of `NotImplemented`. - For some magic methods, the return values are not restricted by PyO3, but checked by the Python interpreter. For example, `__str__` needs to return a string object. This is indicated by `object (Python type)`. ### Basic object customization - `__str__() -> object (str)` - `__repr__() -> object (str)` - `__hash__() -> isize` Objects that compare equal must have the same hash value. Any type up to 64 bits may be returned instead of `isize`, PyO3 will convert to an isize automatically (wrapping unsigned types like `u64` and `usize`).
Disabling Python's default hash By default, all `#[pyclass]` types have a default hash implementation from Python. Types which should not be hashable can override this by setting `__hash__` to `None`. This is the same mechanism as for a pure-Python class. This is done like so: ```rust # use pyo3::prelude::*; # #[pyclass] struct NotHashable {} #[pymethods] impl NotHashable { #[classattr] const __hash__: Option = None; } ```
- `__lt__(, object) -> object` - `__le__(, object) -> object` - `__eq__(, object) -> object` - `__ne__(, object) -> object` - `__gt__(, object) -> object` - `__ge__(, object) -> object` The implementations of Python's "rich comparison" operators `<`, `<=`, `==`, `!=`, `>` and `>=` respectively. _Note that implementing any of these methods will cause Python not to generate a default `__hash__` implementation, so consider also implementing `__hash__`._
Return type The return type will normally be `bool` or `PyResult`, however any Python object can be returned.
- `__richcmp__(, object, pyo3::basic::CompareOp) -> object` Implements Python comparison operations (`==`, `!=`, `<`, `<=`, `>`, and `>=`) in a single method. The `CompareOp` argument indicates the comparison operation being performed. You can use [`CompareOp::matches`] to adapt a Rust `std::cmp::Ordering` result to the requested comparison. _This method cannot be implemented in combination with any of `__lt__`, `__le__`, `__eq__`, `__ne__`, `__gt__`, or `__ge__`._ _Note that implementing `__richcmp__` will cause Python not to generate a default `__hash__` implementation, so consider implementing `__hash__` when implementing `__richcmp__`._
Return type The return type will normally be `PyResult`, but any Python object can be returned. If you want to leave some operations unimplemented, you can return `py.NotImplemented()` for some of the operations: ```rust use pyo3::class::basic::CompareOp; # use pyo3::prelude::*; # # #[pyclass] # struct Number(i32); # #[pymethods] impl Number { fn __richcmp__(&self, other: &Self, op: CompareOp, py: Python<'_>) -> PyObject { match op { CompareOp::Eq => (self.0 == other.0).into_py(py), CompareOp::Ne => (self.0 != other.0).into_py(py), _ => py.NotImplemented(), } } } ``` If the second argument `object` is not of the type specified in the signature, the generated code will automatically `return NotImplemented`.
- `__getattr__(, object) -> object` - `__getattribute__(, object) -> object`
Differences between `__getattr__` and `__getattribute__` As in Python, `__getattr__` is only called if the attribute is not found by normal attribute lookup. `__getattribute__`, on the other hand, is called for *every* attribute access. If it wants to access existing attributes on `self`, it needs to be very careful not to introduce infinite recursion, and use `baseclass.__getattribute__()`.
- `__setattr__(, value: object) -> ()` - `__delattr__(, object) -> ()` Overrides attribute access. - `__bool__() -> bool` Determines the "truthyness" of an object. - `__call__(, ...) -> object` - here, any argument list can be defined as for normal `pymethods` ### Iterable objects Iterators can be defined using these methods: - `__iter__() -> object` - `__next__() -> Option or IterNextOutput` ([see details](#returning-a-value-from-iteration)) Returning `None` from `__next__` indicates that that there are no further items. Example: ```rust use pyo3::prelude::*; #[pyclass] struct MyIterator { iter: Box + Send>, } #[pymethods] impl MyIterator { fn __iter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> { slf } fn __next__(mut slf: PyRefMut<'_, Self>) -> Option { slf.iter.next() } } ``` In many cases you'll have a distinction between the type being iterated over (i.e. the *iterable*) and the iterator it provides. In this case, the iterable only needs to implement `__iter__()` while the iterator must implement both `__iter__()` and `__next__()`. For example: ```rust # use pyo3::prelude::*; #[pyclass] struct Iter { inner: std::vec::IntoIter, } #[pymethods] impl Iter { fn __iter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> { slf } fn __next__(mut slf: PyRefMut<'_, Self>) -> Option { slf.inner.next() } } #[pyclass] struct Container { iter: Vec, } #[pymethods] impl Container { fn __iter__(slf: PyRef<'_, Self>) -> PyResult> { let iter = Iter { inner: slf.iter.clone().into_iter(), }; Py::new(slf.py(), iter) } } # Python::with_gil(|py| { # let container = Container { iter: vec![1, 2, 3, 4] }; # let inst = pyo3::PyCell::new(py, container).unwrap(); # pyo3::py_run!(py, inst, "assert list(inst) == [1, 2, 3, 4]"); # pyo3::py_run!(py, inst, "assert list(iter(iter(inst))) == [1, 2, 3, 4]"); # }); ``` For more details on Python's iteration protocols, check out [the "Iterator Types" section of the library documentation](https://docs.python.org/library/stdtypes.html#iterator-types). #### Returning a value from iteration This guide has so far shown how to use `Option` to implement yielding values during iteration. In Python a generator can also return a value. To express this in Rust, PyO3 provides the [`IterNextOutput`] enum to both `Yield` values and `Return` a final value - see its docs for further details and an example. ### Awaitable objects - `__await__() -> object` - `__aiter__() -> object` - `__anext__() -> Option or IterANextOutput` ### Mapping & Sequence types The magic methods in this section can be used to implement Python container types. They are two main categories of container in Python: "mappings" such as `dict`, with arbitrary keys, and "sequences" such as `list` and `tuple`, with integer keys. The Python C-API which PyO3 is built upon has separate "slots" for sequences and mappings. When writing a `class` in pure Python, there is no such distinction in the implementation - a `__getitem__` implementation will fill the slots for both the mapping and sequence forms, for example. By default PyO3 reproduces the Python behaviour of filling both mapping and sequence slots. This makes sense for the "simple" case which matches Python, and also for sequences, where the mapping slot is used anyway to implement slice indexing. Mapping types usually will not want the sequence slots filled. Having them filled will lead to outcomes which may be unwanted, such as: - The mapping type will successfully cast to [`PySequence`]. This may lead to consumers of the type handling it incorrectly. - Python provides a default implementation of `__iter__` for sequences, which calls `__getitem__` with consecutive positive integers starting from 0 until an `IndexError` is returned. Unless the mapping only contains consecutive positive integer keys, this `__iter__` implementation will likely not be the intended behavior. Use the `#[pyclass(mapping)]` annotation to instruct PyO3 to only fill the mapping slots, leaving the sequence ones empty. This will apply to `__getitem__`, `__setitem__`, and `__delitem__`. Use the `#[pyclass(sequence)]` annotation to instruct PyO3 to fill the `sq_length` slot instead of the `mp_length` slot for `__len__`. This will help libraries such as `numpy` recognise the class as a sequence, however will also cause CPython to automatically add the sequence length to any negative indices before passing them to `__getitem__`. (`__getitem__`, `__setitem__` and `__delitem__` mapping slots are still used for sequences, for slice operations.) - `__len__() -> usize` Implements the built-in function `len()`. - `__contains__(, object) -> bool` Implements membership test operators. Should return true if `item` is in `self`, false otherwise. For objects that don’t define `__contains__()`, the membership test simply traverses the sequence until it finds a match.
Disabling Python's default contains By default, all `#[pyclass]` types with an `__iter__` method support a default implementation of the `in` operator. Types which do not want this can override this by setting `__contains__` to `None`. This is the same mechanism as for a pure-Python class. This is done like so: ```rust # use pyo3::prelude::*; # #[pyclass] struct NoContains {} #[pymethods] impl NoContains { #[classattr] const __contains__: Option = None; } ```
- `__getitem__(, object) -> object` Implements retrieval of the `self[a]` element. *Note:* Negative integer indexes are not handled specially by PyO3. However, for classes with `#[pyclass(sequence)]`, when a negative index is accessed via `PySequence::get_item`, the underlying C API already adjusts the index to be positive. - `__setitem__(, object, object) -> ()` Implements assignment to the `self[a]` element. Should only be implemented if elements can be replaced. Same behavior regarding negative indices as for `__getitem__`. - `__delitem__(, object) -> ()` Implements deletion of the `self[a]` element. Should only be implemented if elements can be deleted. Same behavior regarding negative indices as for `__getitem__`. * `fn __concat__(&self, other: impl FromPyObject) -> PyResult` Concatenates two sequences. Used by the `+` operator, after trying the numeric addition via the `__add__` and `__radd__` methods. * `fn __repeat__(&self, count: isize) -> PyResult` Repeats the sequence `count` times. Used by the `*` operator, after trying the numeric multiplication via the `__mul__` and `__rmul__` methods. * `fn __inplace_concat__(&self, other: impl FromPyObject) -> PyResult` Concatenates two sequences. Used by the `+=` operator, after trying the numeric addition via the `__iadd__` method. * `fn __inplace_repeat__(&self, count: isize) -> PyResult` Concatenates two sequences. Used by the `*=` operator, after trying the numeric multiplication via the `__imul__` method. ### Descriptors - `__get__(, object, object) -> object` - `__set__(, object, object) -> ()` - `__delete__(, object) -> ()` ### Numeric types Binary arithmetic operations (`+`, `-`, `*`, `@`, `/`, `//`, `%`, `divmod()`, `pow()` and `**`, `<<`, `>>`, `&`, `^`, and `|`) and their reflected versions: (If the `object` is not of the type specified in the signature, the generated code will automatically `return NotImplemented`.) - `__add__(, object) -> object` - `__radd__(, object) -> object` - `__sub__(, object) -> object` - `__rsub__(, object) -> object` - `__mul__(, object) -> object` - `__rmul__(, object) -> object` - `__matmul__(, object) -> object` - `__rmatmul__(, object) -> object` - `__floordiv__(, object) -> object` - `__rfloordiv__(, object) -> object` - `__truediv__(, object) -> object` - `__rtruediv__(, object) -> object` - `__divmod__(, object) -> object` - `__rdivmod__(, object) -> object` - `__mod__(, object) -> object` - `__rmod__(, object) -> object` - `__lshift__(, object) -> object` - `__rlshift__(, object) -> object` - `__rshift__(, object) -> object` - `__rrshift__(, object) -> object` - `__and__(, object) -> object` - `__rand__(, object) -> object` - `__xor__(, object) -> object` - `__rxor__(, object) -> object` - `__or__(, object) -> object` - `__ror__(, object) -> object` - `__pow__(, object, object) -> object` - `__rpow__(, object, object) -> object` In-place assignment operations (`+=`, `-=`, `*=`, `@=`, `/=`, `//=`, `%=`, `**=`, `<<=`, `>>=`, `&=`, `^=`, `|=`): - `__iadd__(, object) -> ()` - `__isub__(, object) -> ()` - `__imul__(, object) -> ()` - `__imatmul__(, object) -> ()` - `__itruediv__(, object) -> ()` - `__ifloordiv__(, object) -> ()` - `__imod__(, object) -> ()` - `__ipow__(, object, object) -> ()` - `__ilshift__(, object) -> ()` - `__irshift__(, object) -> ()` - `__iand__(, object) -> ()` - `__ixor__(, object) -> ()` - `__ior__(, object) -> ()` Unary operations (`-`, `+`, `abs()` and `~`): - `__pos__() -> object` - `__neg__() -> object` - `__abs__() -> object` - `__invert__() -> object` Coercions: - `__index__() -> object (int)` - `__int__() -> object (int)` - `__float__() -> object (float)` ### Buffer objects - `__getbuffer__(, *mut ffi::Py_buffer, flags) -> ()` - `__releasebuffer__(, *mut ffi::Py_buffer) -> ()` Errors returned from `__releasebuffer__` will be sent to `sys.unraiseablehook`. It is strongly advised to never return an error from `__releasebuffer__`, and if it really is necessary, to make best effort to perform any required freeing operations before returning. `__releasebuffer__` will not be called a second time; anything not freed will be leaked. ### Garbage Collector Integration If your type owns references to other Python objects, you will need to integrate with Python's garbage collector so that the GC is aware of those references. To do this, implement the two methods `__traverse__` and `__clear__`. These correspond to the slots `tp_traverse` and `tp_clear` in the Python C API. `__traverse__` must call `visit.call()` for each reference to another Python object. `__clear__` must clear out any mutable references to other Python objects (thus breaking reference cycles). Immutable references do not have to be cleared, as every cycle must contain at least one mutable reference. - `__traverse__(, pyo3::class::gc::PyVisit<'_>) -> Result<(), pyo3::class::gc::PyTraverseError>` - `__clear__() -> ()` Example: ```rust use pyo3::prelude::*; use pyo3::PyTraverseError; use pyo3::gc::PyVisit; #[pyclass] struct ClassWithGCSupport { obj: Option, } #[pymethods] impl ClassWithGCSupport { fn __traverse__(&self, visit: PyVisit<'_>) -> Result<(), PyTraverseError> { if let Some(obj) = &self.obj { visit.call(obj)? } Ok(()) } fn __clear__(&mut self) { // Clear reference, this decrements ref counter. self.obj = None; } } ``` Usually, an implementation of `__traverse__` should do nothing but calls to `visit.call`. Most importantly, safe access to the GIL is prohibited inside implementations of `__traverse__`, i.e. `Python::with_gil` will panic. > Note: these methods are part of the C API, PyPy does not necessarily honor them. If you are building for PyPy you should measure memory consumption to make sure you do not have runaway memory growth. See [this issue on the PyPy bug tracker](https://foss.heptapod.net/pypy/pypy/-/issues/3899). [`IterNextOutput`]: {{#PYO3_DOCS_URL}}/pyo3/pyclass/enum.IterNextOutput.html [`PySequence`]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PySequence.html [`CompareOp::matches`]: {{#PYO3_DOCS_URL}}/pyo3/pyclass/enum.CompareOp.html#method.matches pyo3-0.20.2/guide/src/class.md000064400000000000000000001142651046102023000141230ustar 00000000000000# Python classes PyO3 exposes a group of attributes powered by Rust's proc macro system for defining Python classes as Rust structs. The main attribute is `#[pyclass]`, which is placed upon a Rust `struct` or a fieldless `enum` (a.k.a. C-like enum) to generate a Python type for it. They will usually also have *one* `#[pymethods]`-annotated `impl` block for the struct, which is used to define Python methods and constants for the generated Python type. (If the [`multiple-pymethods`] feature is enabled, each `#[pyclass]` is allowed to have multiple `#[pymethods]` blocks.) `#[pymethods]` may also have implementations for Python magic methods such as `__str__`. This chapter will discuss the functionality and configuration these attributes offer. Below is a list of links to the relevant section of this chapter for each: - [`#[pyclass]`](#defining-a-new-class) - [`#[pyo3(get, set)]`](#object-properties-using-pyo3get-set) - [`#[pymethods]`](#instance-methods) - [`#[new]`](#constructor) - [`#[getter]`](#object-properties-using-getter-and-setter) - [`#[setter]`](#object-properties-using-getter-and-setter) - [`#[staticmethod]`](#static-methods) - [`#[classmethod]`](#class-methods) - [`#[classattr]`](#class-attributes) - [`#[args]`](#method-arguments) - [Magic methods and slots](class/protocols.html) - [Classes as function arguments](#classes-as-function-arguments) ## Defining a new class To define a custom Python class, add the `#[pyclass]` attribute to a Rust struct or a fieldless enum. ```rust # #![allow(dead_code)] use pyo3::prelude::*; #[pyclass] struct Integer { inner: i32, } // A "tuple" struct #[pyclass] struct Number(i32); // PyO3 supports custom discriminants in enums #[pyclass] enum HttpResponse { Ok = 200, NotFound = 404, Teapot = 418, // ... } #[pyclass] enum MyEnum { Variant, OtherVariant = 30, // PyO3 supports custom discriminants. } ``` The above example generates implementations for [`PyTypeInfo`] and [`PyClass`] for `MyClass` and `MyEnum`. To see these generated implementations, refer to the [implementation details](#implementation-details) at the end of this chapter. ### Restrictions To integrate Rust types with Python, PyO3 needs to place some restrictions on the types which can be annotated with `#[pyclass]`. In particular, they must have no lifetime parameters, no generic parameters, and must implement `Send`. The reason for each of these is explained below. #### No lifetime parameters Rust lifetimes are used by the Rust compiler to reason about a program's memory safety. They are a compile-time only concept; there is no way to access Rust lifetimes at runtime from a dynamic language like Python. As soon as Rust data is exposed to Python, there is no guarantee that the Rust compiler can make on how long the data will live. Python is a reference-counted language and those references can be held for an arbitrarily long time which is untraceable by the Rust compiler. The only possible way to express this correctly is to require that any `#[pyclass]` does not borrow data for any lifetime shorter than the `'static` lifetime, i.e. the `#[pyclass]` cannot have any lifetime parameters. When you need to share ownership of data between Python and Rust, instead of using borrowed references with lifetimes consider using reference-counted smart pointers such as [`Arc`] or [`Py`]. #### No generic parameters A Rust `struct Foo` with a generic parameter `T` generates new compiled implementations each time it is used with a different concrete type for `T`. These new implementations are generated by the compiler at each usage site. This is incompatible with wrapping `Foo` in Python, where there needs to be a single compiled implementation of `Foo` which is integrated with the Python interpreter. #### Must be Send Because Python objects are freely shared between threads by the Python interpreter, there is no guarantee which thread will eventually drop the object. Therefore all types annotated with `#[pyclass]` must implement `Send` (unless annotated with [`#[pyclass(unsendable)]`](#customizing-the-class)). ## Constructor By default, it is not possible to create an instance of a custom class from Python code. To declare a constructor, you need to define a method and annotate it with the `#[new]` attribute. Only Python's `__new__` method can be specified, `__init__` is not available. ```rust # use pyo3::prelude::*; # #[pyclass] # struct Number(i32); # #[pymethods] impl Number { #[new] fn new(value: i32) -> Self { Number(value) } } ``` Alternatively, if your `new` method may fail you can return `PyResult`. ```rust # use pyo3::prelude::*; # use pyo3::exceptions::PyValueError; # #[pyclass] # struct Nonzero(i32); # #[pymethods] impl Nonzero { #[new] fn py_new(value: i32) -> PyResult { if value == 0 { Err(PyValueError::new_err("cannot be zero")) } else { Ok(Nonzero(value)) } } } ``` If you want to return an existing object (for example, because your `new` method caches the values it returns), `new` can return `pyo3::Py`. As you can see, the Rust method name is not important here; this way you can still, use `new()` for a Rust-level constructor. If no method marked with `#[new]` is declared, object instances can only be created from Rust, but not from Python. For arguments, see the [`Method arguments`](#method-arguments) section below. ## Adding the class to a module The next step is to create the module initializer and add our class to it: ```rust # use pyo3::prelude::*; # #[pyclass] # struct Number(i32); # #[pymodule] fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_class::()?; Ok(()) } ``` ## PyCell and interior mutability You sometimes need to convert your `pyclass` into a Python object and access it from Rust code (e.g., for testing it). [`PyCell`] is the primary interface for that. `PyCell` is always allocated in the Python heap, so Rust doesn't have ownership of it. In other words, Rust code can only extract a `&PyCell`, not a `PyCell`. Thus, to mutate data behind `&PyCell` safely, PyO3 employs the [Interior Mutability Pattern](https://doc.rust-lang.org/book/ch15-05-interior-mutability.html) like [`RefCell`]. Users who are familiar with `RefCell` can use `PyCell` just like `RefCell`. For users who are not very familiar with `RefCell`, here is a reminder of Rust's rules of borrowing: - At any given time, you can have either (but not both of) one mutable reference or any number of immutable references. - References must always be valid. `PyCell`, like `RefCell`, ensures these borrowing rules by tracking references at runtime. ```rust # use pyo3::prelude::*; #[pyclass] struct MyClass { #[pyo3(get)] num: i32, } Python::with_gil(|py| { let obj = PyCell::new(py, MyClass { num: 3 }).unwrap(); { let obj_ref = obj.borrow(); // Get PyRef assert_eq!(obj_ref.num, 3); // You cannot get PyRefMut unless all PyRefs are dropped assert!(obj.try_borrow_mut().is_err()); } { let mut obj_mut = obj.borrow_mut(); // Get PyRefMut obj_mut.num = 5; // You cannot get any other refs until the PyRefMut is dropped assert!(obj.try_borrow().is_err()); assert!(obj.try_borrow_mut().is_err()); } // You can convert `&PyCell` to a Python object pyo3::py_run!(py, obj, "assert obj.num == 5"); }); ``` `&PyCell` is bounded by the same lifetime as a [`GILGuard`]. To make the object longer lived (for example, to store it in a struct on the Rust side), you can use `Py`, which stores an object longer than the GIL lifetime, and therefore needs a `Python<'_>` token to access. ```rust # use pyo3::prelude::*; #[pyclass] struct MyClass { num: i32, } fn return_myclass() -> Py { Python::with_gil(|py| Py::new(py, MyClass { num: 1 }).unwrap()) } let obj = return_myclass(); Python::with_gil(|py| { let cell = obj.as_ref(py); // Py::as_ref returns &PyCell let obj_ref = cell.borrow(); // Get PyRef assert_eq!(obj_ref.num, 1); }); ``` ### frozen classes: Opting out of interior mutability As detailed above, runtime borrow checking is currently enabled by default. But a class can opt of out it by declaring itself `frozen`. It can still use interior mutability via standard Rust types like `RefCell` or `Mutex`, but it is not bound to the implementation provided by PyO3 and can choose the most appropriate strategy on field-by-field basis. Classes which are `frozen` and also `Sync`, e.g. they do use `Mutex` but not `RefCell`, can be accessed without needing the Python GIL via the `PyCell::get` and `Py::get` methods: ```rust use std::sync::atomic::{AtomicUsize, Ordering}; # use pyo3::prelude::*; #[pyclass(frozen)] struct FrozenCounter { value: AtomicUsize, } let py_counter: Py = Python::with_gil(|py| { let counter = FrozenCounter { value: AtomicUsize::new(0), }; Py::new(py, counter).unwrap() }); py_counter.get().value.fetch_add(1, Ordering::Relaxed); ``` Frozen classes are likely to become the default thereby guiding the PyO3 ecosystem towards a more deliberate application of interior mutability. Eventually, this should enable further optimizations of PyO3's internals and avoid downstream code paying the cost of interior mutability when it is not actually required. ## Customizing the class {{#include ../pyclass_parameters.md}} These parameters are covered in various sections of this guide. ### Return type Generally, `#[new]` methods have to return `T: Into>` or `PyResult where T: Into>`. For constructors that may fail, you should wrap the return type in a PyResult as well. Consult the table below to determine which type your constructor should return: | | **Cannot fail** | **May fail** | |-----------------------------|---------------------------|-----------------------------------| |**No inheritance** | `T` | `PyResult` | |**Inheritance(T Inherits U)**| `(T, U)` | `PyResult<(T, U)>` | |**Inheritance(General Case)**| [`PyClassInitializer`] | `PyResult>` | ## Inheritance By default, `object`, i.e. `PyAny` is used as the base class. To override this default, use the `extends` parameter for `pyclass` with the full path to the base class. Currently, only classes defined in Rust and builtins provided by PyO3 can be inherited from; inheriting from other classes defined in Python is not yet supported ([#991](https://github.com/PyO3/pyo3/issues/991)). For convenience, `(T, U)` implements `Into>` where `U` is the base class of `T`. But for a more deeply nested inheritance, you have to return `PyClassInitializer` explicitly. To get a parent class from a child, use [`PyRef`] instead of `&self` for methods, or [`PyRefMut`] instead of `&mut self`. Then you can access a parent class by `self_.as_ref()` as `&Self::BaseClass`, or by `self_.into_super()` as `PyRef`. ```rust # use pyo3::prelude::*; #[pyclass(subclass)] struct BaseClass { val1: usize, } #[pymethods] impl BaseClass { #[new] fn new() -> Self { BaseClass { val1: 10 } } pub fn method(&self) -> PyResult { Ok(self.val1) } } #[pyclass(extends=BaseClass, subclass)] struct SubClass { val2: usize, } #[pymethods] impl SubClass { #[new] fn new() -> (Self, BaseClass) { (SubClass { val2: 15 }, BaseClass::new()) } fn method2(self_: PyRef<'_, Self>) -> PyResult { let super_ = self_.as_ref(); // Get &BaseClass super_.method().map(|x| x * self_.val2) } } #[pyclass(extends=SubClass)] struct SubSubClass { val3: usize, } #[pymethods] impl SubSubClass { #[new] fn new() -> PyClassInitializer { PyClassInitializer::from(SubClass::new()).add_subclass(SubSubClass { val3: 20 }) } fn method3(self_: PyRef<'_, Self>) -> PyResult { let v = self_.val3; let super_ = self_.into_super(); // Get PyRef<'_, SubClass> SubClass::method2(super_).map(|x| x * v) } #[staticmethod] fn factory_method(py: Python<'_>, val: usize) -> PyResult { let base = PyClassInitializer::from(BaseClass::new()); let sub = base.add_subclass(SubClass { val2: val }); if val % 2 == 0 { Ok(Py::new(py, sub)?.to_object(py)) } else { let sub_sub = sub.add_subclass(SubSubClass { val3: val }); Ok(Py::new(py, sub_sub)?.to_object(py)) } } } # Python::with_gil(|py| { # let subsub = pyo3::PyCell::new(py, SubSubClass::new()).unwrap(); # pyo3::py_run!(py, subsub, "assert subsub.method3() == 3000"); # let subsub = SubSubClass::factory_method(py, 2).unwrap(); # let subsubsub = SubSubClass::factory_method(py, 3).unwrap(); # let cls = py.get_type::(); # pyo3::py_run!(py, subsub cls, "assert not isinstance(subsub, cls)"); # pyo3::py_run!(py, subsubsub cls, "assert isinstance(subsubsub, cls)"); # }); ``` You can inherit native types such as `PyDict`, if they implement [`PySizedLayout`]({{#PYO3_DOCS_URL}}/pyo3/type_object/trait.PySizedLayout.html). This is not supported when building for the Python limited API (aka the `abi3` feature of PyO3). However, because of some technical problems, we don't currently provide safe upcasting methods for types that inherit native types. Even in such cases, you can unsafely get a base class by raw pointer conversion. ```rust # #[cfg(not(Py_LIMITED_API))] { # use pyo3::prelude::*; use pyo3::types::PyDict; use std::collections::HashMap; #[pyclass(extends=PyDict)] #[derive(Default)] struct DictWithCounter { counter: HashMap, } #[pymethods] impl DictWithCounter { #[new] fn new() -> Self { Self::default() } fn set(mut self_: PyRefMut<'_, Self>, key: String, value: &PyAny) -> PyResult<()> { self_.counter.entry(key.clone()).or_insert(0); let py = self_.py(); let dict: &PyDict = unsafe { py.from_borrowed_ptr_or_err(self_.as_ptr())? }; dict.set_item(key, value) } } # Python::with_gil(|py| { # let cnt = pyo3::PyCell::new(py, DictWithCounter::new()).unwrap(); # pyo3::py_run!(py, cnt, "cnt.set('abc', 10); assert cnt['abc'] == 10") # }); # } ``` If `SubClass` does not provide a base class initialization, the compilation fails. ```rust,compile_fail # use pyo3::prelude::*; #[pyclass] struct BaseClass { val1: usize, } #[pyclass(extends=BaseClass)] struct SubClass { val2: usize, } #[pymethods] impl SubClass { #[new] fn new() -> Self { SubClass { val2: 15 } } } ``` The `__new__` constructor of a native base class is called implicitly when creating a new instance from Python. Be sure to accept arguments in the `#[new]` method that you want the base class to get, even if they are not used in that `fn`: ```rust # #[allow(dead_code)] # #[cfg(not(Py_LIMITED_API))] { # use pyo3::prelude::*; use pyo3::types::PyDict; #[pyclass(extends=PyDict)] struct MyDict { private: i32, } #[pymethods] impl MyDict { #[new] #[pyo3(signature = (*args, **kwargs))] fn new(args: &PyAny, kwargs: Option<&PyAny>) -> Self { Self { private: 0 } } // some custom methods that use `private` here... } # Python::with_gil(|py| { # let cls = py.get_type::(); # pyo3::py_run!(py, cls, "cls(a=1, b=2)") # }); # } ``` Here, the `args` and `kwargs` allow creating instances of the subclass passing initial items, such as `MyDict(item_sequence)` or `MyDict(a=1, b=2)`. ## Object properties PyO3 supports two ways to add properties to your `#[pyclass]`: - For simple struct fields with no side effects, a `#[pyo3(get, set)]` attribute can be added directly to the field definition in the `#[pyclass]`. - For properties which require computation you can define `#[getter]` and `#[setter]` functions in the [`#[pymethods]`](#instance-methods) block. We'll cover each of these in the following sections. ### Object properties using `#[pyo3(get, set)]` For simple cases where a member variable is just read and written with no side effects, you can declare getters and setters in your `#[pyclass]` field definition using the `pyo3` attribute, like in the example below: ```rust # use pyo3::prelude::*; #[pyclass] struct MyClass { #[pyo3(get, set)] num: i32, } ``` The above would make the `num` field available for reading and writing as a `self.num` Python property. To expose the property with a different name to the field, specify this alongside the rest of the options, e.g. `#[pyo3(get, set, name = "custom_name")]`. Properties can be readonly or writeonly by using just `#[pyo3(get)]` or `#[pyo3(set)]` respectively. To use these annotations, your field type must implement some conversion traits: - For `get` the field type must implement both `IntoPy` and `Clone`. - For `set` the field type must implement `FromPyObject`. For example, implementations of those traits are provided for the `Cell` type, if the inner type also implements the trait. This means you can use `#[pyo3(get, set)]` on fields wrapped in a `Cell`. ### Object properties using `#[getter]` and `#[setter]` For cases which don't satisfy the `#[pyo3(get, set)]` trait requirements, or need side effects, descriptor methods can be defined in a `#[pymethods]` `impl` block. This is done using the `#[getter]` and `#[setter]` attributes, like in the example below: ```rust # use pyo3::prelude::*; #[pyclass] struct MyClass { num: i32, } #[pymethods] impl MyClass { #[getter] fn num(&self) -> PyResult { Ok(self.num) } } ``` A getter or setter's function name is used as the property name by default. There are several ways how to override the name. If a function name starts with `get_` or `set_` for getter or setter respectively, the descriptor name becomes the function name with this prefix removed. This is also useful in case of Rust keywords like `type` ([raw identifiers](https://doc.rust-lang.org/edition-guide/rust-2018/module-system/raw-identifiers.html) can be used since Rust 2018). ```rust # use pyo3::prelude::*; # #[pyclass] # struct MyClass { # num: i32, # } #[pymethods] impl MyClass { #[getter] fn get_num(&self) -> PyResult { Ok(self.num) } #[setter] fn set_num(&mut self, value: i32) -> PyResult<()> { self.num = value; Ok(()) } } ``` In this case, a property `num` is defined and available from Python code as `self.num`. Both the `#[getter]` and `#[setter]` attributes accept one parameter. If this parameter is specified, it is used as the property name, i.e. ```rust # use pyo3::prelude::*; # #[pyclass] # struct MyClass { # num: i32, # } #[pymethods] impl MyClass { #[getter(number)] fn num(&self) -> PyResult { Ok(self.num) } #[setter(number)] fn set_num(&mut self, value: i32) -> PyResult<()> { self.num = value; Ok(()) } } ``` In this case, the property `number` is defined and available from Python code as `self.number`. Attributes defined by `#[setter]` or `#[pyo3(set)]` will always raise `AttributeError` on `del` operations. Support for defining custom `del` behavior is tracked in [#1778](https://github.com/PyO3/pyo3/issues/1778). ## Instance methods To define a Python compatible method, an `impl` block for your struct has to be annotated with the `#[pymethods]` attribute. PyO3 generates Python compatible wrappers for all functions in this block with some variations, like descriptors, class method static methods, etc. Since Rust allows any number of `impl` blocks, you can easily split methods between those accessible to Python (and Rust) and those accessible only to Rust. However to have multiple `#[pymethods]`-annotated `impl` blocks for the same struct you must enable the [`multiple-pymethods`] feature of PyO3. ```rust # use pyo3::prelude::*; # #[pyclass] # struct MyClass { # num: i32, # } #[pymethods] impl MyClass { fn method1(&self) -> PyResult { Ok(10) } fn set_method(&mut self, value: i32) -> PyResult<()> { self.num = value; Ok(()) } } ``` Calls to these methods are protected by the GIL, so both `&self` and `&mut self` can be used. The return type must be `PyResult` or `T` for some `T` that implements `IntoPy`; the latter is allowed if the method cannot raise Python exceptions. A `Python` parameter can be specified as part of method signature, in this case the `py` argument gets injected by the method wrapper, e.g. ```rust # use pyo3::prelude::*; # #[pyclass] # struct MyClass { # #[allow(dead_code)] # num: i32, # } #[pymethods] impl MyClass { fn method2(&self, py: Python<'_>) -> PyResult { Ok(10) } } ``` From the Python perspective, the `method2` in this example does not accept any arguments. ## Class methods To create a class method for a custom class, the method needs to be annotated with the `#[classmethod]` attribute. This is the equivalent of the Python decorator `@classmethod`. ```rust # use pyo3::prelude::*; # use pyo3::types::PyType; # #[pyclass] # struct MyClass { # #[allow(dead_code)] # num: i32, # } #[pymethods] impl MyClass { #[classmethod] fn cls_method(cls: &PyType) -> PyResult { Ok(10) } } ``` Declares a class method callable from Python. * The first parameter is the type object of the class on which the method is called. This may be the type object of a derived class. * The first parameter implicitly has type `&PyType`. * For details on `parameter-list`, see the documentation of `Method arguments` section. * The return type must be `PyResult` or `T` for some `T` that implements `IntoPy`. ### Constructors which accept a class argument To create a constructor which takes a positional class argument, you can combine the `#[classmethod]` and `#[new]` modifiers: ```rust # use pyo3::prelude::*; # use pyo3::types::PyType; # #[pyclass] # struct BaseClass(PyObject); # #[pymethods] impl BaseClass { #[new] #[classmethod] fn py_new<'p>(cls: &'p PyType, py: Python<'p>) -> PyResult { // Get an abstract attribute (presumably) declared on a subclass of this class. let subclass_attr = cls.getattr("a_class_attr")?; Ok(Self(subclass_attr.to_object(py))) } } ``` ## Static methods To create a static method for a custom class, the method needs to be annotated with the `#[staticmethod]` attribute. The return type must be `T` or `PyResult` for some `T` that implements `IntoPy`. ```rust # use pyo3::prelude::*; # #[pyclass] # struct MyClass { # #[allow(dead_code)] # num: i32, # } #[pymethods] impl MyClass { #[staticmethod] fn static_method(param1: i32, param2: &str) -> PyResult { Ok(10) } } ``` ## Class attributes To create a class attribute (also called [class variable][classattr]), a method without any arguments can be annotated with the `#[classattr]` attribute. ```rust # use pyo3::prelude::*; # #[pyclass] # struct MyClass {} #[pymethods] impl MyClass { #[classattr] fn my_attribute() -> String { "hello".to_string() } } Python::with_gil(|py| { let my_class = py.get_type::(); pyo3::py_run!(py, my_class, "assert my_class.my_attribute == 'hello'") }); ``` > Note: if the method has a `Result` return type and returns an `Err`, PyO3 will panic during class creation. If the class attribute is defined with `const` code only, one can also annotate associated constants: ```rust # use pyo3::prelude::*; # #[pyclass] # struct MyClass {} #[pymethods] impl MyClass { #[classattr] const MY_CONST_ATTRIBUTE: &'static str = "foobar"; } ``` ## Classes as function arguments Free functions defined using `#[pyfunction]` interact with classes through the same mechanisms as the self parameters of instance methods, i.e. they can take GIL-bound references, GIL-bound reference wrappers or GIL-indepedent references: ```rust # #![allow(dead_code)] # use pyo3::prelude::*; #[pyclass] struct MyClass { my_field: i32, } // Take a GIL-bound reference when the underlying `PyCell` is irrelevant. #[pyfunction] fn increment_field(my_class: &mut MyClass) { my_class.my_field += 1; } // Take a GIL-bound reference wrapper when borrowing should be automatic, // but interaction with the underlying `PyCell` is desired. #[pyfunction] fn print_field(my_class: PyRef<'_, MyClass>) { println!("{}", my_class.my_field); } // Take a GIL-bound reference to the underlying cell // when borrowing needs to be managed manually. #[pyfunction] fn increment_then_print_field(my_class: &PyCell) { my_class.borrow_mut().my_field += 1; println!("{}", my_class.borrow().my_field); } // Take a GIL-indepedent reference when you want to store the reference elsewhere. #[pyfunction] fn print_refcnt(my_class: Py, py: Python<'_>) { println!("{}", my_class.get_refcnt(py)); } ``` Classes can also be passed by value if they can be cloned, i.e. they automatically implement `FromPyObject` if they implement `Clone`, e.g. via `#[derive(Clone)]`: ```rust # #![allow(dead_code)] # use pyo3::prelude::*; #[pyclass] #[derive(Clone)] struct MyClass { my_field: Box, } #[pyfunction] fn dissamble_clone(my_class: MyClass) { let MyClass { mut my_field } = my_class; *my_field += 1; } ``` Note that `#[derive(FromPyObject)]` on a class is usually not useful as it tries to construct a new Rust value by filling in the fields by looking up attributes of any given Python value. ## Method arguments Similar to `#[pyfunction]`, the `#[pyo3(signature = (...))]` attribute can be used to specify the way that `#[pymethods]` accept arguments. Consult the documentation for [`function signatures`](./function/signature.md) to see the parameters this attribute accepts. The following example defines a class `MyClass` with a method `method`. This method has a signature that sets default values for `num` and `name`, and indicates that `py_args` should collect all extra positional arguments and `py_kwargs` all extra keyword arguments: ```rust # use pyo3::prelude::*; use pyo3::types::{PyDict, PyTuple}; # # #[pyclass] # struct MyClass { # num: i32, # } #[pymethods] impl MyClass { #[new] #[pyo3(signature = (num=-1))] fn new(num: i32) -> Self { MyClass { num } } #[pyo3(signature = (num=10, *py_args, name="Hello", **py_kwargs))] fn method( &mut self, num: i32, py_args: &PyTuple, name: &str, py_kwargs: Option<&PyDict>, ) -> String { let num_before = self.num; self.num = num; format!( "num={} (was previously={}), py_args={:?}, name={}, py_kwargs={:?} ", num, num_before, py_args, name, py_kwargs, ) } } ``` In Python, this might be used like: ```python >>> import mymodule >>> mc = mymodule.MyClass() >>> print(mc.method(44, False, "World", 666, x=44, y=55)) py_args=('World', 666), py_kwargs=Some({'x': 44, 'y': 55}), name=Hello, num=44, num_before=-1 >>> print(mc.method(num=-1, name="World")) py_args=(), py_kwargs=None, name=World, num=-1, num_before=44 ``` ## Making class method signatures available to Python The [`text_signature = "..."`](./function.md#text_signature) option for `#[pyfunction]` also works for `#[pymethods]`: ```rust # #![allow(dead_code)] use pyo3::prelude::*; use pyo3::types::PyType; #[pyclass] struct MyClass {} #[pymethods] impl MyClass { #[new] #[pyo3(text_signature = "(c, d)")] fn new(c: i32, d: &str) -> Self { Self {} } // the self argument should be written $self #[pyo3(text_signature = "($self, e, f)")] fn my_method(&self, e: i32, f: i32) -> i32 { e + f } // similarly for classmethod arguments, use $cls #[classmethod] #[pyo3(text_signature = "($cls, e, f)")] fn my_class_method(cls: &PyType, e: i32, f: i32) -> i32 { e + f } #[staticmethod] #[pyo3(text_signature = "(e, f)")] fn my_static_method(e: i32, f: i32) -> i32 { e + f } } # # fn main() -> PyResult<()> { # Python::with_gil(|py| { # let inspect = PyModule::import(py, "inspect")?.getattr("signature")?; # let module = PyModule::new(py, "my_module")?; # module.add_class::()?; # let class = module.getattr("MyClass")?; # # if cfg!(not(Py_LIMITED_API)) || py.version_info() >= (3, 10) { # let doc: String = class.getattr("__doc__")?.extract()?; # assert_eq!(doc, ""); # # let sig: String = inspect # .call1((class,))? # .call_method0("__str__")? # .extract()?; # assert_eq!(sig, "(c, d)"); # } else { # let doc: String = class.getattr("__doc__")?.extract()?; # assert_eq!(doc, ""); # # inspect.call1((class,)).expect_err("`text_signature` on classes is not compatible with compilation in `abi3` mode until Python 3.10 or greater"); # } # # { # let method = class.getattr("my_method")?; # # assert!(method.getattr("__doc__")?.is_none()); # # let sig: String = inspect # .call1((method,))? # .call_method0("__str__")? # .extract()?; # assert_eq!(sig, "(self, /, e, f)"); # } # # { # let method = class.getattr("my_class_method")?; # # assert!(method.getattr("__doc__")?.is_none()); # # let sig: String = inspect # .call1((method,))? # .call_method0("__str__")? # .extract()?; # assert_eq!(sig, "(e, f)"); // inspect.signature skips the $cls arg # } # # { # let method = class.getattr("my_static_method")?; # # assert!(method.getattr("__doc__")?.is_none()); # # let sig: String = inspect # .call1((method,))? # .call_method0("__str__")? # .extract()?; # assert_eq!(sig, "(e, f)"); # } # # Ok(()) # }) # } ``` Note that `text_signature` on `#[new]` is not compatible with compilation in `abi3` mode until Python 3.10 or greater. ## #[pyclass] enums Currently PyO3 only supports fieldless enums. PyO3 adds a class attribute for each variant, so you can access them in Python without defining `#[new]`. PyO3 also provides default implementations of `__richcmp__` and `__int__`, so they can be compared using `==`: ```rust # use pyo3::prelude::*; #[pyclass] enum MyEnum { Variant, OtherVariant, } Python::with_gil(|py| { let x = Py::new(py, MyEnum::Variant).unwrap(); let y = Py::new(py, MyEnum::OtherVariant).unwrap(); let cls = py.get_type::(); pyo3::py_run!(py, x y cls, r#" assert x == cls.Variant assert y == cls.OtherVariant assert x != y "#) }) ``` You can also convert your enums into `int`: ```rust # use pyo3::prelude::*; #[pyclass] enum MyEnum { Variant, OtherVariant = 10, } Python::with_gil(|py| { let cls = py.get_type::(); let x = MyEnum::Variant as i32; // The exact value is assigned by the compiler. pyo3::py_run!(py, cls x, r#" assert int(cls.Variant) == x assert int(cls.OtherVariant) == 10 assert cls.OtherVariant == 10 # You can also compare against int. assert 10 == cls.OtherVariant "#) }) ``` PyO3 also provides `__repr__` for enums: ```rust # use pyo3::prelude::*; #[pyclass] enum MyEnum{ Variant, OtherVariant, } Python::with_gil(|py| { let cls = py.get_type::(); let x = Py::new(py, MyEnum::Variant).unwrap(); pyo3::py_run!(py, cls x, r#" assert repr(x) == 'MyEnum.Variant' assert repr(cls.OtherVariant) == 'MyEnum.OtherVariant' "#) }) ``` All methods defined by PyO3 can be overridden. For example here's how you override `__repr__`: ```rust # use pyo3::prelude::*; #[pyclass] enum MyEnum { Answer = 42, } #[pymethods] impl MyEnum { fn __repr__(&self) -> &'static str { "42" } } Python::with_gil(|py| { let cls = py.get_type::(); pyo3::py_run!(py, cls, "assert repr(cls.Answer) == '42'") }) ``` Enums and their variants can also be renamed using `#[pyo3(name)]`. ```rust # use pyo3::prelude::*; #[pyclass(name = "RenamedEnum")] enum MyEnum { #[pyo3(name = "UPPERCASE")] Variant, } Python::with_gil(|py| { let x = Py::new(py, MyEnum::Variant).unwrap(); let cls = py.get_type::(); pyo3::py_run!(py, x cls, r#" assert repr(x) == 'RenamedEnum.UPPERCASE' assert x == cls.UPPERCASE "#) }) ``` You may not use enums as a base class or let enums inherit from other classes. ```rust,compile_fail # use pyo3::prelude::*; #[pyclass(subclass)] enum BadBase { Var1, } ``` ```rust,compile_fail # use pyo3::prelude::*; #[pyclass(subclass)] struct Base; #[pyclass(extends=Base)] enum BadSubclass { Var1, } ``` `#[pyclass]` enums are currently not interoperable with `IntEnum` in Python. ## Implementation details The `#[pyclass]` macros rely on a lot of conditional code generation: each `#[pyclass]` can optionally have a `#[pymethods]` block. To support this flexibility the `#[pyclass]` macro expands to a blob of boilerplate code which sets up the structure for ["dtolnay specialization"](https://github.com/dtolnay/case-studies/blob/master/autoref-specialization/README.md). This implementation pattern enables the Rust compiler to use `#[pymethods]` implementations when they are present, and fall back to default (empty) definitions when they are not. This simple technique works for the case when there is zero or one implementations. To support multiple `#[pymethods]` for a `#[pyclass]` (in the [`multiple-pymethods`] feature), a registry mechanism provided by the [`inventory`](https://github.com/dtolnay/inventory) crate is used instead. This collects `impl`s at library load time, but isn't supported on all platforms. See [inventory: how it works](https://github.com/dtolnay/inventory#how-it-works) for more details. The `#[pyclass]` macro expands to roughly the code seen below. The `PyClassImplCollector` is the type used internally by PyO3 for dtolnay specialization: ```rust # #[cfg(not(feature = "multiple-pymethods"))] { # use pyo3::prelude::*; // Note: the implementation differs slightly with the `multiple-pymethods` feature enabled. struct MyClass { # #[allow(dead_code)] num: i32, } unsafe impl pyo3::type_object::PyTypeInfo for MyClass { type AsRefTarget = pyo3::PyCell; const NAME: &'static str = "MyClass"; const MODULE: ::std::option::Option<&'static str> = ::std::option::Option::None; #[inline] fn type_object_raw(py: pyo3::Python<'_>) -> *mut pyo3::ffi::PyTypeObject { ::lazy_type_object() .get_or_init(py) .as_type_ptr() } } impl pyo3::PyClass for MyClass { type Frozen = pyo3::pyclass::boolean_struct::False; } impl<'a, 'py> pyo3::impl_::extract_argument::PyFunctionArgument<'a, 'py> for &'a MyClass { type Holder = ::std::option::Option>; #[inline] fn extract(obj: &'py pyo3::PyAny, holder: &'a mut Self::Holder) -> pyo3::PyResult { pyo3::impl_::extract_argument::extract_pyclass_ref(obj, holder) } } impl<'a, 'py> pyo3::impl_::extract_argument::PyFunctionArgument<'a, 'py> for &'a mut MyClass { type Holder = ::std::option::Option>; #[inline] fn extract(obj: &'py pyo3::PyAny, holder: &'a mut Self::Holder) -> pyo3::PyResult { pyo3::impl_::extract_argument::extract_pyclass_ref_mut(obj, holder) } } impl pyo3::IntoPy for MyClass { fn into_py(self, py: pyo3::Python<'_>) -> pyo3::PyObject { pyo3::IntoPy::into_py(pyo3::Py::new(py, self).unwrap(), py) } } impl pyo3::impl_::pyclass::PyClassImpl for MyClass { const IS_BASETYPE: bool = false; const IS_SUBCLASS: bool = false; type BaseType = PyAny; type ThreadChecker = pyo3::impl_::pyclass::SendablePyClass; type PyClassMutability = <::PyClassMutability as pyo3::impl_::pycell::PyClassMutability>::MutableChild; type Dict = pyo3::impl_::pyclass::PyClassDummySlot; type WeakRef = pyo3::impl_::pyclass::PyClassDummySlot; type BaseNativeType = pyo3::PyAny; fn items_iter() -> pyo3::impl_::pyclass::PyClassItemsIter { use pyo3::impl_::pyclass::*; let collector = PyClassImplCollector::::new(); static INTRINSIC_ITEMS: PyClassItems = PyClassItems { slots: &[], methods: &[] }; PyClassItemsIter::new(&INTRINSIC_ITEMS, collector.py_methods()) } fn lazy_type_object() -> &'static pyo3::impl_::pyclass::LazyTypeObject { use pyo3::impl_::pyclass::LazyTypeObject; static TYPE_OBJECT: LazyTypeObject = LazyTypeObject::new(); &TYPE_OBJECT } fn doc(py: Python<'_>) -> pyo3::PyResult<&'static ::std::ffi::CStr> { use pyo3::impl_::pyclass::*; static DOC: pyo3::once_cell::GILOnceCell<::std::borrow::Cow<'static, ::std::ffi::CStr>> = pyo3::once_cell::GILOnceCell::new(); DOC.get_or_try_init(py, || { let collector = PyClassImplCollector::::new(); build_pyclass_doc(::NAME, "", None.or_else(|| collector.new_text_signature())) }).map(::std::ops::Deref::deref) } } # Python::with_gil(|py| { # let cls = py.get_type::(); # pyo3::py_run!(py, cls, "assert cls.__name__ == 'MyClass'") # }); # } ``` [`GILGuard`]: {{#PYO3_DOCS_URL}}/pyo3/struct.GILGuard.html [`PyTypeInfo`]: {{#PYO3_DOCS_URL}}/pyo3/type_object/trait.PyTypeInfo.html [`Py`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Py.html [`PyCell`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyCell.html [`PyClass`]: {{#PYO3_DOCS_URL}}/pyo3/pyclass/trait.PyClass.html [`PyRef`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRef.html [`PyRefMut`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRefMut.html [`PyClassInitializer`]: {{#PYO3_DOCS_URL}}/pyo3/pyclass_init/struct.PyClassInitializer.html [`Arc`]: https://doc.rust-lang.org/std/sync/struct.Arc.html [`RefCell`]: https://doc.rust-lang.org/std/cell/struct.RefCell.html [classattr]: https://docs.python.org/3/tutorial/classes.html#class-and-instance-variables [`multiple-pymethods`]: features.md#multiple-pymethods pyo3-0.20.2/guide/src/contributing.md000064400000000000000000000000431046102023000155110ustar 00000000000000{{#include ../../Contributing.md}} pyo3-0.20.2/guide/src/conversions/tables.md000064400000000000000000000165301046102023000166340ustar 00000000000000## Mapping of Rust types to Python types When writing functions callable from Python (such as a `#[pyfunction]` or in a `#[pymethods]` block), the trait `FromPyObject` is required for function arguments, and `IntoPy` is required for function return values. Consult the tables in the following section to find the Rust types provided by PyO3 which implement these traits. ### Argument Types When accepting a function argument, it is possible to either use Rust library types or PyO3's Python-native types. (See the next section for discussion on when to use each.) The table below contains the Python type and the corresponding function argument types that will accept them: | Python | Rust | Rust (Python-native) | | ------------- |:-------------------------------:|:--------------------:| | `object` | - | `&PyAny` | | `str` | `String`, `Cow`, `&str`, `char`, `OsString`, `PathBuf`, `Path` | `&PyString`, `&PyUnicode` | | `bytes` | `Vec`, `&[u8]`, `Cow<[u8]>` | `&PyBytes` | | `bool` | `bool` | `&PyBool` | | `int` | `i8`, `u8`, `i16`, `u16`, `i32`, `u32`, `i64`, `u64`, `i128`, `u128`, `isize`, `usize`, `num_bigint::BigInt`[^1], `num_bigint::BigUint`[^1] | `&PyLong` | | `float` | `f32`, `f64` | `&PyFloat` | | `complex` | `num_complex::Complex`[^2] | `&PyComplex` | | `list[T]` | `Vec` | `&PyList` | | `dict[K, V]` | `HashMap`, `BTreeMap`, `hashbrown::HashMap`[^3], `indexmap::IndexMap`[^4] | `&PyDict` | | `tuple[T, U]` | `(T, U)`, `Vec` | `&PyTuple` | | `set[T]` | `HashSet`, `BTreeSet`, `hashbrown::HashSet`[^3] | `&PySet` | | `frozenset[T]` | `HashSet`, `BTreeSet`, `hashbrown::HashSet`[^3] | `&PyFrozenSet` | | `bytearray` | `Vec`, `Cow<[u8]>` | `&PyByteArray` | | `slice` | - | `&PySlice` | | `type` | - | `&PyType` | | `module` | - | `&PyModule` | | `collections.abc.Buffer` | - | `PyBuffer` | | `datetime.datetime` | - | `&PyDateTime` | | `datetime.date` | - | `&PyDate` | | `datetime.time` | - | `&PyTime` | | `datetime.tzinfo` | - | `&PyTzInfo` | | `datetime.timedelta` | - | `&PyDelta` | | `decimal.Decimal` | `rust_decimal::Decimal`[^5] | - | | `ipaddress.IPv4Address` | `std::net::IpAddr`, `std::net::IpV4Addr` | - | | `ipaddress.IPv6Address` | `std::net::IpAddr`, `std::net::IpV6Addr` | - | | `os.PathLike ` | `PathBuf`, `Path` | `&PyString`, `&PyUnicode` | | `pathlib.Path` | `PathBuf`, `Path` | `&PyString`, `&PyUnicode` | | `typing.Optional[T]` | `Option` | - | | `typing.Sequence[T]` | `Vec` | `&PySequence` | | `typing.Mapping[K, V]` | `HashMap`, `BTreeMap`, `hashbrown::HashMap`[^3], `indexmap::IndexMap`[^4] | `&PyMapping` | | `typing.Iterator[Any]` | - | `&PyIterator` | | `typing.Union[...]` | See [`#[derive(FromPyObject)]`](traits.html#deriving-a-hrefhttpsdocsrspyo3latestpyo3conversiontraitfrompyobjecthtmlfrompyobjecta-for-enums) | - | There are also a few special types related to the GIL and Rust-defined `#[pyclass]`es which may come in useful: | What | Description | | ------------- | ------------------------------------- | | `Python` | A GIL token, used to pass to PyO3 constructors to prove ownership of the GIL | | `Py` | A Python object isolated from the GIL lifetime. This can be sent to other threads. | | `PyObject` | An alias for `Py` | | `&PyCell` | A `#[pyclass]` value owned by Python. | | `PyRef` | A `#[pyclass]` borrowed immutably. | | `PyRefMut` | A `#[pyclass]` borrowed mutably. | For more detail on accepting `#[pyclass]` values as function arguments, see [the section of this guide on Python Classes](../class.md). #### Using Rust library types vs Python-native types Using Rust library types as function arguments will incur a conversion cost compared to using the Python-native types. Using the Python-native types is almost zero-cost (they just require a type check similar to the Python builtin function `isinstance()`). However, once that conversion cost has been paid, the Rust standard library types offer a number of benefits: - You can write functionality in native-speed Rust code (free of Python's runtime costs). - You get better interoperability with the rest of the Rust ecosystem. - You can use `Python::allow_threads` to release the Python GIL and let other Python threads make progress while your Rust code is executing. - You also benefit from stricter type checking. For example you can specify `Vec`, which will only accept a Python `list` containing integers. The Python-native equivalent, `&PyList`, would accept a Python `list` containing Python objects of any type. For most PyO3 usage the conversion cost is worth paying to get these benefits. As always, if you're not sure it's worth it in your case, benchmark it! ### Returning Rust values to Python When returning values from functions callable from Python, Python-native types (`&PyAny`, `&PyDict` etc.) can be used with zero cost. Because these types are references, in some situations the Rust compiler may ask for lifetime annotations. If this is the case, you should use `Py`, `Py` etc. instead - which are also zero-cost. For all of these Python-native types `T`, `Py` can be created from `T` with an `.into()` conversion. If your function is fallible, it should return `PyResult` or `Result` where `E` implements `From for PyErr`. This will raise a `Python` exception if the `Err` variant is returned. Finally, the following Rust types are also able to convert to Python as return values: | Rust type | Resulting Python Type | | ------------- |:-------------------------------:| | `String` | `str` | | `&str` | `str` | | `bool` | `bool` | | Any integer type (`i32`, `u32`, `usize`, etc) | `int` | | `f32`, `f64` | `float` | | `Option` | `Optional[T]` | | `(T, U)` | `Tuple[T, U]` | | `Vec` | `List[T]` | | `Cow<[u8]>` | `bytes` | | `HashMap` | `Dict[K, V]` | | `BTreeMap` | `Dict[K, V]` | | `HashSet` | `Set[T]` | | `BTreeSet` | `Set[T]` | | `&PyCell` | `T` | | `PyRef` | `T` | | `PyRefMut` | `T` | [^1]: Requires the `num-bigint` optional feature. [^2]: Requires the `num-complex` optional feature. [^3]: Requires the `hashbrown` optional feature. [^4]: Requires the `indexmap` optional feature. [^5]: Requires the `rust_decimal` optional feature. pyo3-0.20.2/guide/src/conversions/traits.md000064400000000000000000000404661046102023000166750ustar 00000000000000## Conversion traits PyO3 provides some handy traits to convert between Python types and Rust types. ### `.extract()` and the `FromPyObject` trait The easiest way to convert a Python object to a Rust value is using `.extract()`. It returns a `PyResult` with a type error if the conversion fails, so usually you will use something like ```rust # use pyo3::prelude::*; # use pyo3::types::PyList; # fn main() -> PyResult<()> { # Python::with_gil(|py| { # let list = PyList::new(py, b"foo"); let v: Vec = list.extract()?; # assert_eq!(&v, &[102, 111, 111]); # Ok(()) # }) # } ``` This method is available for many Python object types, and can produce a wide variety of Rust types, which you can check out in the implementor list of [`FromPyObject`]. [`FromPyObject`] is also implemented for your own Rust types wrapped as Python objects (see [the chapter about classes](../class.md)). There, in order to both be able to operate on mutable references *and* satisfy Rust's rules of non-aliasing mutable references, you have to extract the PyO3 reference wrappers [`PyRef`] and [`PyRefMut`]. They work like the reference wrappers of `std::cell::RefCell` and ensure (at runtime) that Rust borrows are allowed. #### Deriving [`FromPyObject`] [`FromPyObject`] can be automatically derived for many kinds of structs and enums if the member types themselves implement `FromPyObject`. This even includes members with a generic type `T: FromPyObject`. Derivation for empty enums, enum variants and structs is not supported. #### Deriving [`FromPyObject`] for structs The derivation generates code that will attempt to access the attribute `my_string` on the Python object, i.e. `obj.getattr("my_string")`, and call `extract()` on the attribute. ```rust use pyo3::prelude::*; #[derive(FromPyObject)] struct RustyStruct { my_string: String, } # # fn main() -> PyResult<()> { # Python::with_gil(|py| -> PyResult<()> { # let module = PyModule::from_code( # py, # "class Foo: # def __init__(self): # self.my_string = 'test'", # "", # "", # )?; # # let class = module.getattr("Foo")?; # let instance = class.call0()?; # let rustystruct: RustyStruct = instance.extract()?; # assert_eq!(rustystruct.my_string, "test"); # Ok(()) # }) # } ``` By setting the `#[pyo3(item)]` attribute on the field, PyO3 will attempt to extract the value by calling the `get_item` method on the Python object. ```rust use pyo3::prelude::*; #[derive(FromPyObject)] struct RustyStruct { #[pyo3(item)] my_string: String, } # # use pyo3::types::PyDict; # fn main() -> PyResult<()> { # Python::with_gil(|py| -> PyResult<()> { # let dict = PyDict::new(py); # dict.set_item("my_string", "test")?; # # let rustystruct: RustyStruct = dict.extract()?; # assert_eq!(rustystruct.my_string, "test"); # Ok(()) # }) # } ``` The argument passed to `getattr` and `get_item` can also be configured: ```rust use pyo3::prelude::*; #[derive(FromPyObject)] struct RustyStruct { #[pyo3(item("key"))] string_in_mapping: String, #[pyo3(attribute("name"))] string_attr: String, } # # fn main() -> PyResult<()> { # Python::with_gil(|py| -> PyResult<()> { # let module = PyModule::from_code( # py, # "class Foo(dict): # def __init__(self): # self.name = 'test' # self['key'] = 'test2'", # "", # "", # )?; # # let class = module.getattr("Foo")?; # let instance = class.call0()?; # let rustystruct: RustyStruct = instance.extract()?; # assert_eq!(rustystruct.string_attr, "test"); # assert_eq!(rustystruct.string_in_mapping, "test2"); # # Ok(()) # }) # } ``` This tries to extract `string_attr` from the attribute `name` and `string_in_mapping` from a mapping with the key `"key"`. The arguments for `attribute` are restricted to non-empty string literals while `item` can take any valid literal that implements `ToBorrowedObject`. You can use `#[pyo3(from_item_all)]` on a struct to extract every field with `get_item` method. In this case, you can't use `#[pyo3(attribute)]` or barely use `#[pyo3(item)]` on any field. However, using `#[pyo3(item("key"))]` to specify the key for a field is still allowed. ```rust use pyo3::prelude::*; #[derive(FromPyObject)] #[pyo3(from_item_all)] struct RustyStruct { foo: String, bar: String, #[pyo3(item("foobar"))] baz: String, } # # fn main() -> PyResult<()> { # Python::with_gil(|py| -> PyResult<()> { # let py_dict = py.eval("{'foo': 'foo', 'bar': 'bar', 'foobar': 'foobar'}", None, None)?; # let rustystruct: RustyStruct = py_dict.extract()?; # assert_eq!(rustystruct.foo, "foo"); # assert_eq!(rustystruct.bar, "bar"); # assert_eq!(rustystruct.baz, "foobar"); # # Ok(()) # }) # } ``` #### Deriving [`FromPyObject`] for tuple structs Tuple structs are also supported but do not allow customizing the extraction. The input is always assumed to be a Python tuple with the same length as the Rust type, the `n`th field is extracted from the `n`th item in the Python tuple. ```rust use pyo3::prelude::*; #[derive(FromPyObject)] struct RustyTuple(String, String); # use pyo3::types::PyTuple; # fn main() -> PyResult<()> { # Python::with_gil(|py| -> PyResult<()> { # let tuple = PyTuple::new(py, vec!["test", "test2"]); # # let rustytuple: RustyTuple = tuple.extract()?; # assert_eq!(rustytuple.0, "test"); # assert_eq!(rustytuple.1, "test2"); # # Ok(()) # }) # } ``` Tuple structs with a single field are treated as wrapper types which are described in the following section. To override this behaviour and ensure that the input is in fact a tuple, specify the struct as ```rust use pyo3::prelude::*; #[derive(FromPyObject)] struct RustyTuple((String,)); # use pyo3::types::PyTuple; # fn main() -> PyResult<()> { # Python::with_gil(|py| -> PyResult<()> { # let tuple = PyTuple::new(py, vec!["test"]); # # let rustytuple: RustyTuple = tuple.extract()?; # assert_eq!((rustytuple.0).0, "test"); # # Ok(()) # }) # } ``` #### Deriving [`FromPyObject`] for wrapper types The `pyo3(transparent)` attribute can be used on structs with exactly one field. This results in extracting directly from the input object, i.e. `obj.extract()`, rather than trying to access an item or attribute. This behaviour is enabled per default for newtype structs and tuple-variants with a single field. ```rust use pyo3::prelude::*; #[derive(FromPyObject)] struct RustyTransparentTupleStruct(String); #[derive(FromPyObject)] #[pyo3(transparent)] struct RustyTransparentStruct { inner: String, } # use pyo3::types::PyString; # fn main() -> PyResult<()> { # Python::with_gil(|py| -> PyResult<()> { # let s = PyString::new(py, "test"); # # let tup: RustyTransparentTupleStruct = s.extract()?; # assert_eq!(tup.0, "test"); # # let stru: RustyTransparentStruct = s.extract()?; # assert_eq!(stru.inner, "test"); # # Ok(()) # }) # } ``` #### Deriving [`FromPyObject`] for enums The `FromPyObject` derivation for enums generates code that tries to extract the variants in the order of the fields. As soon as a variant can be extracted successfully, that variant is returned. This makes it possible to extract Python union types like `str | int`. The same customizations and restrictions described for struct derivations apply to enum variants, i.e. a tuple variant assumes that the input is a Python tuple, and a struct variant defaults to extracting fields as attributes but can be configured in the same manner. The `transparent` attribute can be applied to single-field-variants. ```rust use pyo3::prelude::*; #[derive(FromPyObject)] # #[derive(Debug)] enum RustyEnum<'a> { Int(usize), // input is a positive int String(String), // input is a string IntTuple(usize, usize), // input is a 2-tuple with positive ints StringIntTuple(String, usize), // input is a 2-tuple with String and int Coordinates3d { // needs to be in front of 2d x: usize, y: usize, z: usize, }, Coordinates2d { // only gets checked if the input did not have `z` #[pyo3(attribute("x"))] a: usize, #[pyo3(attribute("y"))] b: usize, }, #[pyo3(transparent)] CatchAll(&'a PyAny), // This extraction never fails } # # use pyo3::types::{PyBytes, PyString}; # fn main() -> PyResult<()> { # Python::with_gil(|py| -> PyResult<()> { # { # let thing = 42_u8.to_object(py); # let rust_thing: RustyEnum<'_> = thing.extract(py)?; # # assert_eq!( # 42, # match rust_thing { # RustyEnum::Int(i) => i, # other => unreachable!("Error extracting: {:?}", other), # } # ); # } # { # let thing = PyString::new(py, "text"); # let rust_thing: RustyEnum<'_> = thing.extract()?; # # assert_eq!( # "text", # match rust_thing { # RustyEnum::String(i) => i, # other => unreachable!("Error extracting: {:?}", other), # } # ); # } # { # let thing = (32_u8, 73_u8).to_object(py); # let rust_thing: RustyEnum<'_> = thing.extract(py)?; # # assert_eq!( # (32, 73), # match rust_thing { # RustyEnum::IntTuple(i, j) => (i, j), # other => unreachable!("Error extracting: {:?}", other), # } # ); # } # { # let thing = ("foo", 73_u8).to_object(py); # let rust_thing: RustyEnum<'_> = thing.extract(py)?; # # assert_eq!( # (String::from("foo"), 73), # match rust_thing { # RustyEnum::StringIntTuple(i, j) => (i, j), # other => unreachable!("Error extracting: {:?}", other), # } # ); # } # { # let module = PyModule::from_code( # py, # "class Foo(dict): # def __init__(self): # self.x = 0 # self.y = 1 # self.z = 2", # "", # "", # )?; # # let class = module.getattr("Foo")?; # let instance = class.call0()?; # let rust_thing: RustyEnum<'_> = instance.extract()?; # # assert_eq!( # (0, 1, 2), # match rust_thing { # RustyEnum::Coordinates3d { x, y, z } => (x, y, z), # other => unreachable!("Error extracting: {:?}", other), # } # ); # } # # { # let module = PyModule::from_code( # py, # "class Foo(dict): # def __init__(self): # self.x = 3 # self.y = 4", # "", # "", # )?; # # let class = module.getattr("Foo")?; # let instance = class.call0()?; # let rust_thing: RustyEnum<'_> = instance.extract()?; # # assert_eq!( # (3, 4), # match rust_thing { # RustyEnum::Coordinates2d { a, b } => (a, b), # other => unreachable!("Error extracting: {:?}", other), # } # ); # } # # { # let thing = PyBytes::new(py, b"text"); # let rust_thing: RustyEnum<'_> = thing.extract()?; # # assert_eq!( # b"text", # match rust_thing { # RustyEnum::CatchAll(i) => i.downcast::()?.as_bytes(), # other => unreachable!("Error extracting: {:?}", other), # } # ); # } # Ok(()) # }) # } ``` If none of the enum variants match, a `PyTypeError` containing the names of the tested variants is returned. The names reported in the error message can be customized through the `#[pyo3(annotation = "name")]` attribute, e.g. to use conventional Python type names: ```rust use pyo3::prelude::*; #[derive(FromPyObject)] # #[derive(Debug)] enum RustyEnum { #[pyo3(transparent, annotation = "str")] String(String), #[pyo3(transparent, annotation = "int")] Int(isize), } # # fn main() -> PyResult<()> { # Python::with_gil(|py| -> PyResult<()> { # { # let thing = 42_u8.to_object(py); # let rust_thing: RustyEnum = thing.extract(py)?; # # assert_eq!( # 42, # match rust_thing { # RustyEnum::Int(i) => i, # other => unreachable!("Error extracting: {:?}", other), # } # ); # } # # { # let thing = "foo".to_object(py); # let rust_thing: RustyEnum = thing.extract(py)?; # # assert_eq!( # "foo", # match rust_thing { # RustyEnum::String(i) => i, # other => unreachable!("Error extracting: {:?}", other), # } # ); # } # # { # let thing = b"foo".to_object(py); # let error = thing.extract::(py).unwrap_err(); # assert!(error.is_instance_of::(py)); # } # # Ok(()) # }) # } ``` If the input is neither a string nor an integer, the error message will be: `"'' cannot be converted to 'str | int'"`. #### `#[derive(FromPyObject)]` Container Attributes - `pyo3(transparent)` - extract the field directly from the object as `obj.extract()` instead of `get_item()` or `getattr()` - Newtype structs and tuple-variants are treated as transparent per default. - only supported for single-field structs and enum variants - `pyo3(annotation = "name")` - changes the name of the failed variant in the generated error message in case of failure. - e.g. `pyo3("int")` reports the variant's type as `int`. - only supported for enum variants #### `#[derive(FromPyObject)]` Field Attributes - `pyo3(attribute)`, `pyo3(attribute("name"))` - retrieve the field from an attribute, possibly with a custom name specified as an argument - argument must be a string-literal. - `pyo3(item)`, `pyo3(item("key"))` - retrieve the field from a mapping, possibly with the custom key specified as an argument. - can be any literal that implements `ToBorrowedObject` - `pyo3(from_py_with = "...")` - apply a custom function to convert the field from Python the desired Rust type. - the argument must be the name of the function as a string. - the function signature must be `fn(&PyAny) -> PyResult` where `T` is the Rust type of the argument. ### `IntoPy` This trait defines the to-python conversion for a Rust type. It is usually implemented as `IntoPy`, which is the trait needed for returning a value from `#[pyfunction]` and `#[pymethods]`. All types in PyO3 implement this trait, as does a `#[pyclass]` which doesn't use `extends`. Occasionally you may choose to implement this for custom types which are mapped to Python types _without_ having a unique python type. ```rust use pyo3::prelude::*; struct MyPyObjectWrapper(PyObject); impl IntoPy for MyPyObjectWrapper { fn into_py(self, py: Python<'_>) -> PyObject { self.0 } } ``` ### The `ToPyObject` trait [`ToPyObject`] is a conversion trait that allows various objects to be converted into [`PyObject`]. `IntoPy` serves the same purpose, except that it consumes `self`. [`IntoPy`]: {{#PYO3_DOCS_URL}}/pyo3/conversion/trait.IntoPy.html [`FromPyObject`]: {{#PYO3_DOCS_URL}}/pyo3/conversion/trait.FromPyObject.html [`ToPyObject`]: {{#PYO3_DOCS_URL}}/pyo3/conversion/trait.ToPyObject.html [`PyObject`]: {{#PYO3_DOCS_URL}}/pyo3/type.PyObject.html [`PyRef`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRef.html [`PyRefMut`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRefMut.html pyo3-0.20.2/guide/src/conversions.md000064400000000000000000000003021046102023000153500ustar 00000000000000# Type conversions In this portion of the guide we'll talk about the mapping of Python types to Rust types offered by PyO3, as well as the traits available to perform conversions between them. pyo3-0.20.2/guide/src/debugging.md000064400000000000000000000051661046102023000147500ustar 00000000000000# Debugging ## Macros PyO3's attributes (`#[pyclass]`, `#[pymodule]`, etc.) are [procedural macros](https://doc.rust-lang.org/reference/procedural-macros.html), which means that they rewrite the source of the annotated item. You can view the generated source with the following command, which also expands a few other things: ```bash cargo rustc --profile=check -- -Z unstable-options --pretty=expanded > expanded.rs; rustfmt expanded.rs ``` (You might need to install [rustfmt](https://github.com/rust-lang-nursery/rustfmt) if you don't already have it.) You can also debug classic `!`-macros by adding `-Z trace-macros`: ```bash cargo rustc --profile=check -- -Z unstable-options --pretty=expanded -Z trace-macros > expanded.rs; rustfmt expanded.rs ``` See [cargo expand](https://github.com/dtolnay/cargo-expand) for a more elaborate version of those commands. ## Running with Valgrind Valgrind is a tool to detect memory management bugs such as memory leaks. You first need to install a debug build of Python, otherwise Valgrind won't produce usable results. In Ubuntu there's e.g. a `python3-dbg` package. Activate an environment with the debug interpreter and recompile. If you're on Linux, use `ldd` with the name of your binary and check that you're linking e.g. `libpython3.7d.so.1.0` instead of `libpython3.7.so.1.0`. [Download the suppressions file for CPython](https://raw.githubusercontent.com/python/cpython/master/Misc/valgrind-python.supp). Run Valgrind with `valgrind --suppressions=valgrind-python.supp ./my-command --with-options` ## Getting a stacktrace The best start to investigate a crash such as an segmentation fault is a backtrace. You can set `RUST_BACKTRACE=1` as an environment variable to get the stack trace on a `panic!`. Alternatively you can use a debugger such as `gdb` to explore the issue. Rust provides a wrapper, `rust-gdb`, which has pretty-printers for inspecting Rust variables. Since PyO3 uses `cdylib` for Python shared objects, it does not receive the pretty-print debug hooks in `rust-gdb` ([rust-lang/rust#96365](https://github.com/rust-lang/rust/issues/96365)). The mentioned issue contains a workaround for enabling pretty-printers in this case. * Link against a debug build of python as described in the previous chapter * Run `rust-gdb ` * Set a breakpoint (`b`) on `rust_panic` if you are investigating a `panic!` * Enter `r` to run * After the crash occurred, enter `bt` or `bt full` to print the stacktrace Often it is helpful to run a small piece of Python code to exercise a section of Rust. ```console rust-gdb --args python -c "import my_package; my_package.sum_to_string(1, 2)" ``` pyo3-0.20.2/guide/src/ecosystem/async-await.md000064400000000000000000000430331046102023000172430ustar 00000000000000# Using `async` and `await` If you are working with a Python library that makes use of async functions or wish to provide Python bindings for an async Rust library, [`pyo3-asyncio`](https://github.com/awestlake87/pyo3-asyncio) likely has the tools you need. It provides conversions between async functions in both Python and Rust and was designed with first-class support for popular Rust runtimes such as [`tokio`](https://tokio.rs/) and [`async-std`](https://async.rs/). In addition, all async Python code runs on the default `asyncio` event loop, so `pyo3-asyncio` should work just fine with existing Python libraries. In the following sections, we'll give a general overview of `pyo3-asyncio` explaining how to call async Python functions with PyO3, how to call async Rust functions from Python, and how to configure your codebase to manage the runtimes of both. ## Quickstart Here are some examples to get you started right away! A more detailed breakdown of the concepts in these examples can be found in the following sections. ### Rust Applications Here we initialize the runtime, import Python's `asyncio` library and run the given future to completion using Python's default `EventLoop` and `async-std`. Inside the future, we convert `asyncio` sleep into a Rust future and await it. ```toml # Cargo.toml dependencies [dependencies] pyo3 = { version = "0.14" } pyo3-asyncio = { version = "0.14", features = ["attributes", "async-std-runtime"] } async-std = "1.9" ``` ```rust //! main.rs use pyo3::prelude::*; #[pyo3_asyncio::async_std::main] async fn main() -> PyResult<()> { let fut = Python::with_gil(|py| { let asyncio = py.import("asyncio")?; // convert asyncio.sleep into a Rust Future pyo3_asyncio::async_std::into_future(asyncio.call_method1("sleep", (1.into_py(py),))?) })?; fut.await?; Ok(()) } ``` The same application can be written to use `tokio` instead using the `#[pyo3_asyncio::tokio::main]` attribute. ```toml # Cargo.toml dependencies [dependencies] pyo3 = { version = "0.14" } pyo3-asyncio = { version = "0.14", features = ["attributes", "tokio-runtime"] } tokio = "1.4" ``` ```rust //! main.rs use pyo3::prelude::*; #[pyo3_asyncio::tokio::main] async fn main() -> PyResult<()> { let fut = Python::with_gil(|py| { let asyncio = py.import("asyncio")?; // convert asyncio.sleep into a Rust Future pyo3_asyncio::tokio::into_future(asyncio.call_method1("sleep", (1.into_py(py),))?) })?; fut.await?; Ok(()) } ``` More details on the usage of this library can be found in the [API docs](https://awestlake87.github.io/pyo3-asyncio/master/doc) and the primer below. ### PyO3 Native Rust Modules PyO3 Asyncio can also be used to write native modules with async functions. Add the `[lib]` section to `Cargo.toml` to make your library a `cdylib` that Python can import. ```toml [lib] name = "my_async_module" crate-type = ["cdylib"] ``` Make your project depend on `pyo3` with the `extension-module` feature enabled and select your `pyo3-asyncio` runtime: For `async-std`: ```toml [dependencies] pyo3 = { version = "0.14", features = ["extension-module"] } pyo3-asyncio = { version = "0.14", features = ["async-std-runtime"] } async-std = "1.9" ``` For `tokio`: ```toml [dependencies] pyo3 = { version = "0.14", features = ["extension-module"] } pyo3-asyncio = { version = "0.14", features = ["tokio-runtime"] } tokio = "1.4" ``` Export an async function that makes use of `async-std`: ```rust //! lib.rs use pyo3::{prelude::*, wrap_pyfunction}; #[pyfunction] fn rust_sleep(py: Python<'_>) -> PyResult<&PyAny> { pyo3_asyncio::async_std::future_into_py(py, async { async_std::task::sleep(std::time::Duration::from_secs(1)).await; Ok(Python::with_gil(|py| py.None())) }) } #[pymodule] fn my_async_module(py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(rust_sleep, m)?)?; Ok(()) } ``` If you want to use `tokio` instead, here's what your module should look like: ```rust //! lib.rs use pyo3::{prelude::*, wrap_pyfunction}; #[pyfunction] fn rust_sleep(py: Python<'_>) -> PyResult<&PyAny> { pyo3_asyncio::tokio::future_into_py(py, async { tokio::time::sleep(std::time::Duration::from_secs(1)).await; Ok(Python::with_gil(|py| py.None())) }) } #[pymodule] fn my_async_module(py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(rust_sleep, m)?)?; Ok(()) } ``` You can build your module with maturin (see the [Using Rust in Python](https://pyo3.rs/main/#using-rust-from-python) section in the PyO3 guide for setup instructions). After that you should be able to run the Python REPL to try it out. ```bash maturin develop && python3 🔗 Found pyo3 bindings 🐍 Found CPython 3.8 at python3 Finished dev [unoptimized + debuginfo] target(s) in 0.04s Python 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import asyncio >>> >>> from my_async_module import rust_sleep >>> >>> async def main(): >>> await rust_sleep() >>> >>> # should sleep for 1s >>> asyncio.run(main()) >>> ``` ## Awaiting an Async Python Function in Rust Let's take a look at a dead simple async Python function: ```python # Sleep for 1 second async def py_sleep(): await asyncio.sleep(1) ``` **Async functions in Python are simply functions that return a `coroutine` object**. For our purposes, we really don't need to know much about these `coroutine` objects. The key factor here is that calling an `async` function is _just like calling a regular function_, the only difference is that we have to do something special with the object that it returns. Normally in Python, that something special is the `await` keyword, but in order to await this coroutine in Rust, we first need to convert it into Rust's version of a `coroutine`: a `Future`. That's where `pyo3-asyncio` comes in. [`pyo3_asyncio::into_future`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/fn.into_future.html) performs this conversion for us. The following example uses `into_future` to call the `py_sleep` function shown above and then await the coroutine object returned from the call: ```rust use pyo3::prelude::*; #[pyo3_asyncio::tokio::main] async fn main() -> PyResult<()> { let future = Python::with_gil(|py| -> PyResult<_> { // import the module containing the py_sleep function let example = py.import("example")?; // calling the py_sleep method like a normal function // returns a coroutine let coroutine = example.call_method0("py_sleep")?; // convert the coroutine into a Rust future using the // tokio runtime pyo3_asyncio::tokio::into_future(coroutine) })?; // await the future future.await?; Ok(()) } ``` Alternatively, the below example shows how to write a `#[pyfunction]` which uses `into_future` to receive and await a coroutine argument: ```rust #[pyfunction] fn await_coro(coro: &PyAny) -> PyResult<()> { // convert the coroutine into a Rust future using the // async_std runtime let f = pyo3_asyncio::async_std::into_future(coro)?; pyo3_asyncio::async_std::run_until_complete(coro.py(), async move { // await the future f.await?; Ok(()) }) } ``` This could be called from Python as: ```python import asyncio async def py_sleep(): asyncio.sleep(1) await_coro(py_sleep()) ``` If for you wanted to pass a callable function to the `#[pyfunction]` instead, (i.e. the last line becomes `await_coro(py_sleep))`, then the above example needs to be tweaked to first call the callable to get the coroutine: ```rust #[pyfunction] fn await_coro(callable: &PyAny) -> PyResult<()> { // get the coroutine by calling the callable let coro = callable.call0()?; // convert the coroutine into a Rust future using the // async_std runtime let f = pyo3_asyncio::async_std::into_future(coro)?; pyo3_asyncio::async_std::run_until_complete(coro.py(), async move { // await the future f.await?; Ok(()) }) } ``` This can be particularly helpful where you need to repeatedly create and await a coroutine. Trying to await the same coroutine multiple times will raise an error: ```python RuntimeError: cannot reuse already awaited coroutine ``` > If you're interested in learning more about `coroutines` and `awaitables` in general, check out the > [Python 3 `asyncio` docs](https://docs.python.org/3/library/asyncio-task.html) for more information. ## Awaiting a Rust Future in Python Here we have the same async function as before written in Rust using the [`async-std`](https://async.rs/) runtime: ```rust /// Sleep for 1 second async fn rust_sleep() { async_std::task::sleep(std::time::Duration::from_secs(1)).await; } ``` Similar to Python, Rust's async functions also return a special object called a `Future`: ```rust let future = rust_sleep(); ``` We can convert this `Future` object into Python to make it `awaitable`. This tells Python that you can use the `await` keyword with it. In order to do this, we'll call [`pyo3_asyncio::async_std::future_into_py`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/async_std/fn.future_into_py.html): ```rust use pyo3::prelude::*; async fn rust_sleep() { async_std::task::sleep(std::time::Duration::from_secs(1)).await; } #[pyfunction] fn call_rust_sleep(py: Python<'_>) -> PyResult<&PyAny> { pyo3_asyncio::async_std::future_into_py(py, async move { rust_sleep().await; Ok(Python::with_gil(|py| py.None())) }) } ``` In Python, we can call this pyo3 function just like any other async function: ```python from example import call_rust_sleep async def rust_sleep(): await call_rust_sleep() ``` ## Managing Event Loops Python's event loop requires some special treatment, especially regarding the main thread. Some of Python's `asyncio` features, like proper signal handling, require control over the main thread, which doesn't always play well with Rust. Luckily, Rust's event loops are pretty flexible and don't _need_ control over the main thread, so in `pyo3-asyncio`, we decided the best way to handle Rust/Python interop was to just surrender the main thread to Python and run Rust's event loops in the background. Unfortunately, since most event loop implementations _prefer_ control over the main thread, this can still make some things awkward. ### PyO3 Asyncio Initialization Because Python needs to control the main thread, we can't use the convenient proc macros from Rust runtimes to handle the `main` function or `#[test]` functions. Instead, the initialization for PyO3 has to be done from the `main` function and the main thread must block on [`pyo3_asyncio::run_forever`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/fn.run_forever.html) or [`pyo3_asyncio::async_std::run_until_complete`](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/async_std/fn.run_until_complete.html). Because we have to block on one of those functions, we can't use [`#[async_std::main]`](https://docs.rs/async-std/latest/async_std/attr.main.html) or [`#[tokio::main]`](https://docs.rs/tokio/1.1.0/tokio/attr.main.html) since it's not a good idea to make long blocking calls during an async function. > Internally, these `#[main]` proc macros are expanded to something like this: > ```rust > fn main() { > // your async main fn > async fn _main_impl() { /* ... */ } > Runtime::new().block_on(_main_impl()); > } > ``` > Making a long blocking call inside the `Future` that's being driven by `block_on` prevents that > thread from doing anything else and can spell trouble for some runtimes (also this will actually > deadlock a single-threaded runtime!). Many runtimes have some sort of `spawn_blocking` mechanism > that can avoid this problem, but again that's not something we can use here since we need it to > block on the _main_ thread. For this reason, `pyo3-asyncio` provides its own set of proc macros to provide you with this initialization. These macros are intended to mirror the initialization of `async-std` and `tokio` while also satisfying the Python runtime's needs. Here's a full example of PyO3 initialization with the `async-std` runtime: ```rust use pyo3::prelude::*; #[pyo3_asyncio::async_std::main] async fn main() -> PyResult<()> { // PyO3 is initialized - Ready to go let fut = Python::with_gil(|py| -> PyResult<_> { let asyncio = py.import("asyncio")?; // convert asyncio.sleep into a Rust Future pyo3_asyncio::async_std::into_future( asyncio.call_method1("sleep", (1.into_py(py),))? ) })?; fut.await?; Ok(()) } ``` ### A Note About `asyncio.run` In Python 3.7+, the recommended way to run a top-level coroutine with `asyncio` is with `asyncio.run`. In `v0.13` we recommended against using this function due to initialization issues, but in `v0.14` it's perfectly valid to use this function... with a caveat. Since our Rust <--> Python conversions require a reference to the Python event loop, this poses a problem. Imagine we have a PyO3 Asyncio module that defines a `rust_sleep` function like in previous examples. You might rightfully assume that you can call pass this directly into `asyncio.run` like this: ```python import asyncio from my_async_module import rust_sleep asyncio.run(rust_sleep()) ``` You might be surprised to find out that this throws an error: ```bash Traceback (most recent call last): File "example.py", line 5, in asyncio.run(rust_sleep()) RuntimeError: no running event loop ``` What's happening here is that we are calling `rust_sleep` _before_ the future is actually running on the event loop created by `asyncio.run`. This is counter-intuitive, but expected behaviour, and unfortunately there doesn't seem to be a good way of solving this problem within PyO3 Asyncio itself. However, we can make this example work with a simple workaround: ```python import asyncio from my_async_module import rust_sleep # Calling main will just construct the coroutine that later calls rust_sleep. # - This ensures that rust_sleep will be called when the event loop is running, # not before. async def main(): await rust_sleep() # Run the main() coroutine at the top-level instead asyncio.run(main()) ``` ### Non-standard Python Event Loops Python allows you to use alternatives to the default `asyncio` event loop. One popular alternative is `uvloop`. In `v0.13` using non-standard event loops was a bit of an ordeal, but in `v0.14` it's trivial. #### Using `uvloop` in a PyO3 Asyncio Native Extensions ```toml # Cargo.toml [lib] name = "my_async_module" crate-type = ["cdylib"] [dependencies] pyo3 = { version = "0.14", features = ["extension-module"] } pyo3-asyncio = { version = "0.14", features = ["tokio-runtime"] } async-std = "1.9" tokio = "1.4" ``` ```rust //! lib.rs use pyo3::{prelude::*, wrap_pyfunction}; #[pyfunction] fn rust_sleep(py: Python<'_>) -> PyResult<&PyAny> { pyo3_asyncio::tokio::future_into_py(py, async { tokio::time::sleep(std::time::Duration::from_secs(1)).await; Ok(Python::with_gil(|py| py.None())) }) } #[pymodule] fn my_async_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(rust_sleep, m)?)?; Ok(()) } ``` ```bash $ maturin develop && python3 🔗 Found pyo3 bindings 🐍 Found CPython 3.8 at python3 Finished dev [unoptimized + debuginfo] target(s) in 0.04s Python 3.8.8 (default, Apr 13 2021, 19:58:26) [GCC 7.3.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information. >>> import asyncio >>> import uvloop >>> >>> import my_async_module >>> >>> uvloop.install() >>> >>> async def main(): ... await my_async_module.rust_sleep() ... >>> asyncio.run(main()) >>> ``` #### Using `uvloop` in Rust Applications Using `uvloop` in Rust applications is a bit trickier, but it's still possible with relatively few modifications. Unfortunately, we can't make use of the `#[pyo3_asyncio::::main]` attribute with non-standard event loops. This is because the `#[pyo3_asyncio::::main]` proc macro has to interact with the Python event loop before we can install the `uvloop` policy. ```toml [dependencies] async-std = "1.9" pyo3 = "0.14" pyo3-asyncio = { version = "0.14", features = ["async-std-runtime"] } ``` ```rust //! main.rs use pyo3::{prelude::*, types::PyType}; fn main() -> PyResult<()> { pyo3::prepare_freethreaded_python(); Python::with_gil(|py| { let uvloop = py.import("uvloop")?; uvloop.call_method0("install")?; // store a reference for the assertion let uvloop = PyObject::from(uvloop); pyo3_asyncio::async_std::run(py, async move { // verify that we are on a uvloop.Loop Python::with_gil(|py| -> PyResult<()> { assert!(pyo3_asyncio::async_std::get_current_loop(py)?.is_instance( uvloop .as_ref(py) .getattr("Loop")? )?); Ok(()) })?; async_std::task::sleep(std::time::Duration::from_secs(1)).await; Ok(()) }) }) } ``` ## Additional Information - Managing event loop references can be tricky with pyo3-asyncio. See [Event Loop References](https://docs.rs/pyo3-asyncio/#event-loop-references) in the API docs to get a better intuition for how event loop references are managed in this library. - Testing pyo3-asyncio libraries and applications requires a custom test harness since Python requires control over the main thread. You can find a testing guide in the [API docs for the `testing` module](https://docs.rs/pyo3-asyncio/latest/pyo3_asyncio/testing) pyo3-0.20.2/guide/src/ecosystem/logging.md000064400000000000000000000061561046102023000164560ustar 00000000000000# Logging It is desirable if both the Python and Rust parts of the application end up logging using the same configuration into the same place. This section of the guide briefly discusses how to connect the two languages' logging ecosystems together. The recommended way for Python extension modules is to configure Rust's logger to send log messages to Python using the `pyo3-log` crate. For users who want to do the opposite and send Python log messages to Rust, see the note at the end of this guide. ## Using `pyo3-log` to send Rust log messages to Python The [pyo3-log] crate allows sending the messages from the Rust side to Python's [logging] system. This is mostly suitable for writing native extensions for Python programs. Use [`pyo3_log::init`][init] to install the logger in its default configuration. It's also possible to tweak its configuration (mostly to tune its performance). ```rust use log::info; use pyo3::prelude::*; #[pyfunction] fn log_something() { // This will use the logger installed in `my_module` to send the `info` // message to the Python logging facilities. info!("Something!"); } #[pymodule] fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { // A good place to install the Rust -> Python logger. pyo3_log::init(); m.add_function(wrap_pyfunction!(log_something))?; Ok(()) } ``` Then it is up to the Python side to actually output the messages somewhere. ```python import logging import my_module FORMAT = '%(levelname)s %(name)s %(asctime)-15s %(filename)s:%(lineno)d %(message)s' logging.basicConfig(format=FORMAT) logging.getLogger().setLevel(logging.INFO) my_module.log_something() ``` It is important to initialize the Python loggers first, before calling any Rust functions that may log. This limitation can be worked around if it is not possible to satisfy, read the documentation about [caching]. ## The Python to Rust direction To have python logs be handled by Rust, one need only register a rust function to handle logs emitted from the core python logging module. This has been implemented within the [pyo3-pylogger] crate. ```rust use log::{info, warn}; use pyo3::prelude::*; fn main() -> PyResult<()> { // register the host handler with python logger, providing a logger target // set the name here to something appropriate for your application pyo3_pylogger::register("example_application_py_logger"); // initialize up a logger env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("trace")).init(); // Log some messages from Rust. info!("Just some normal information!"); warn!("Something spooky happened!"); // Log some messages from Python Python::with_gil(|py| { py.run( " import logging logging.error('Something bad happened') ", None, None, ) }) } ``` [logging]: https://docs.python.org/3/library/logging.html [pyo3-log]: https://crates.io/crates/pyo3-log [init]: https://docs.rs/pyo3-log/*/pyo3_log/fn.init.html [caching]: https://docs.rs/pyo3-log/*/pyo3_log/#performance-filtering-and-caching [pyo3-pylogger]: https://crates.io/crates/pyo3-pylogger pyo3-0.20.2/guide/src/ecosystem.md000064400000000000000000000006031046102023000150170ustar 00000000000000# The PyO3 ecosystem This portion of the guide is dedicated to crates which are external to the main PyO3 project and provide additional functionality you might find useful. Because these projects evolve independently of the PyO3 repository the content of these articles may fall out of date over time; please file issues on the PyO3 GitHub to alert maintainers when this is the case. pyo3-0.20.2/guide/src/exception.md000064400000000000000000000074551046102023000150160ustar 00000000000000# Python exceptions ## Defining a new exception Use the [`create_exception!`] macro: ```rust use pyo3::create_exception; create_exception!(module, MyError, pyo3::exceptions::PyException); ``` * `module` is the name of the containing module. * `MyError` is the name of the new exception type. For example: ```rust use pyo3::prelude::*; use pyo3::create_exception; use pyo3::types::IntoPyDict; use pyo3::exceptions::PyException; create_exception!(mymodule, CustomError, PyException); Python::with_gil(|py| { let ctx = [("CustomError", py.get_type::())].into_py_dict(py); pyo3::py_run!( py, *ctx, "assert str(CustomError) == \"\"" ); pyo3::py_run!(py, *ctx, "assert CustomError('oops').args == ('oops',)"); }); ``` When using PyO3 to create an extension module, you can add the new exception to the module like this, so that it is importable from Python: ```rust use pyo3::prelude::*; use pyo3::exceptions::PyException; pyo3::create_exception!(mymodule, CustomError, PyException); #[pymodule] fn mymodule(py: Python<'_>, m: &PyModule) -> PyResult<()> { // ... other elements added to module ... m.add("CustomError", py.get_type::())?; Ok(()) } ``` ## Raising an exception As described in the [function error handling](./function/error_handling.md) chapter, to raise an exception from a `#[pyfunction]` or `#[pymethods]`, return an `Err(PyErr)`. PyO3 will automatically raise this exception for you when returning the result to Python. You can also manually write and fetch errors in the Python interpreter's global state: ```rust use pyo3::{Python, PyErr}; use pyo3::exceptions::PyTypeError; Python::with_gil(|py| { PyTypeError::new_err("Error").restore(py); assert!(PyErr::occurred(py)); drop(PyErr::fetch(py)); }); ``` ## Checking exception types Python has an [`isinstance`](https://docs.python.org/3/library/functions.html#isinstance) method to check an object's type. In PyO3 every object has the [`PyAny::is_instance`] and [`PyAny::is_instance_of`] methods which do the same thing. ```rust use pyo3::Python; use pyo3::types::{PyBool, PyList}; Python::with_gil(|py| { assert!(PyBool::new(py, true).is_instance_of::()); let list = PyList::new(py, &[1, 2, 3, 4]); assert!(!list.is_instance_of::()); assert!(list.is_instance_of::()); }); ``` To check the type of an exception, you can similarly do: ```rust # use pyo3::exceptions::PyTypeError; # use pyo3::prelude::*; # Python::with_gil(|py| { # let err = PyTypeError::new_err(()); err.is_instance_of::(py); # }); ``` ## Using exceptions defined in Python code It is possible to use an exception defined in Python code as a native Rust type. The `import_exception!` macro allows importing a specific exception class and defines a Rust type for that exception. ```rust #![allow(dead_code)] use pyo3::prelude::*; mod io { pyo3::import_exception!(io, UnsupportedOperation); } fn tell(file: &PyAny) -> PyResult { match file.call_method0("tell") { Err(_) => Err(io::UnsupportedOperation::new_err("not supported: tell")), Ok(x) => x.extract::(), } } ``` [`pyo3::exceptions`]({{#PYO3_DOCS_URL}}/pyo3/exceptions/index.html) defines exceptions for several standard library modules. [`create_exception!`]: {{#PYO3_DOCS_URL}}/pyo3/macro.create_exception.html [`import_exception!`]: {{#PYO3_DOCS_URL}}/pyo3/macro.import_exception.html [`PyErr`]: {{#PYO3_DOCS_URL}}/pyo3/struct.PyErr.html [`PyResult`]: {{#PYO3_DOCS_URL}}/pyo3/type.PyResult.html [`PyErr::from_value`]: {{#PYO3_DOCS_URL}}/pyo3/struct.PyErr.html#method.from_value [`PyAny::is_instance`]: {{#PYO3_DOCS_URL}}/pyo3/struct.PyAny.html#method.is_instance [`PyAny::is_instance_of`]: {{#PYO3_DOCS_URL}}/pyo3/struct.PyAny.html#method.is_instance_of pyo3-0.20.2/guide/src/faq.md000064400000000000000000000177271046102023000135720ustar 00000000000000# Frequently Asked Questions and troubleshooting ## I'm experiencing deadlocks using PyO3 with lazy_static or once_cell! `lazy_static` and `once_cell::sync` both use locks to ensure that initialization is performed only by a single thread. Because the Python GIL is an additional lock this can lead to deadlocks in the following way: 1. A thread (thread A) which has acquired the Python GIL starts initialization of a `lazy_static` value. 2. The initialization code calls some Python API which temporarily releases the GIL e.g. `Python::import`. 3. Another thread (thread B) acquires the Python GIL and attempts to access the same `lazy_static` value. 4. Thread B is blocked, because it waits for `lazy_static`'s initialization to lock to release. 5. Thread A is blocked, because it waits to re-acquire the GIL which thread B still holds. 6. Deadlock. PyO3 provides a struct [`GILOnceCell`] which works equivalently to `OnceCell` but relies solely on the Python GIL for thread safety. This means it can be used in place of `lazy_static` or `once_cell` where you are experiencing the deadlock described above. See the documentation for [`GILOnceCell`] for an example how to use it. [`GILOnceCell`]: {{#PYO3_DOCS_URL}}/pyo3/sync/struct.GILOnceCell.html ## I can't run `cargo test`; or I can't build in a Cargo workspace: I'm having linker issues like "Symbol not found" or "Undefined reference to _PyExc_SystemError"! Currently, [#340](https://github.com/PyO3/pyo3/issues/340) causes `cargo test` to fail with linking errors when the `extension-module` feature is activated. Linking errors can also happen when building in a cargo workspace where a different crate also uses PyO3 (see [#2521](https://github.com/PyO3/pyo3/issues/2521)). For now, there are three ways we can work around these issues. 1. Make the `extension-module` feature optional. Build with `maturin develop --features "extension-module"` ```toml [dependencies.pyo3] {{#PYO3_CRATE_VERSION}} [features] extension-module = ["pyo3/extension-module"] ``` 2. Make the `extension-module` feature optional and default. Run tests with `cargo test --no-default-features`: ```toml [dependencies.pyo3] {{#PYO3_CRATE_VERSION}} [features] extension-module = ["pyo3/extension-module"] default = ["extension-module"] ``` 3. If you are using a [`pyproject.toml`](https://maturin.rs/metadata.html) file to control maturin settings, add the following section: ```toml [tool.maturin] features = ["pyo3/extension-module"] # Or for maturin 0.12: # cargo-extra-args = ["--features", "pyo3/extension-module"] ``` ## I can't run `cargo test`: my crate cannot be found for tests in `tests/` directory! The Rust book suggests to [put integration tests inside a `tests/` directory](https://doc.rust-lang.org/book/ch11-03-test-organization.html#integration-tests). For a PyO3 `extension-module` project where the `crate-type` is set to `"cdylib"` in your `Cargo.toml`, the compiler won't be able to find your crate and will display errors such as `E0432` or `E0463`: ```text error[E0432]: unresolved import `my_crate` --> tests/test_my_crate.rs:1:5 | 1 | use my_crate; | ^^^^^^^^^^^^ no external crate `my_crate` ``` The best solution is to make your crate types include both `rlib` and `cdylib`: ```toml # Cargo.toml [lib] crate-type = ["cdylib", "rlib"] ``` ## Ctrl-C doesn't do anything while my Rust code is executing! This is because Ctrl-C raises a SIGINT signal, which is handled by the calling Python process by simply setting a flag to action upon later. This flag isn't checked while Rust code called from Python is executing, only once control returns to the Python interpreter. You can give the Python interpreter a chance to process the signal properly by calling `Python::check_signals`. It's good practice to call this function regularly if you have a long-running Rust function so that your users can cancel it. ## `#[pyo3(get)]` clones my field! You may have a nested struct similar to this: ```rust # use pyo3::prelude::*; #[pyclass] #[derive(Clone)] struct Inner {/* fields omitted */} #[pyclass] struct Outer { #[pyo3(get)] inner: Inner, } #[pymethods] impl Outer { #[new] fn __new__() -> Self { Self { inner: Inner {} } } } ``` When Python code accesses `Outer`'s field, PyO3 will return a new object on every access (note that their addresses are different): ```python outer = Outer() a = outer.inner b = outer.inner assert a is b, f"a: {a}\nb: {b}" ``` ```text AssertionError: a: b: ``` This can be especially confusing if the field is mutable, as getting the field and then mutating it won't persist - you'll just get a fresh clone of the original on the next access. Unfortunately Python and Rust don't agree about ownership - if PyO3 gave out references to (possibly) temporary Rust objects to Python code, Python code could then keep that reference alive indefinitely. Therefore returning Rust objects requires cloning. If you don't want that cloning to happen, a workaround is to allocate the field on the Python heap and store a reference to that, by using [`Py<...>`]({{#PYO3_DOCS_URL}}/pyo3/struct.Py.html): ```rust # use pyo3::prelude::*; #[pyclass] #[derive(Clone)] struct Inner {/* fields omitted */} #[pyclass] struct Outer { #[pyo3(get)] inner: Py, } #[pymethods] impl Outer { #[new] fn __new__(py: Python<'_>) -> PyResult { Ok(Self { inner: Py::new(py, Inner {})?, }) } } ``` This time `a` and `b` *are* the same object: ```python outer = Outer() a = outer.inner b = outer.inner assert a is b, f"a: {a}\nb: {b}" print(f"a: {a}\nb: {b}") ``` ```text a: b: ``` The downside to this approach is that any Rust code working on the `Outer` struct now has to acquire the GIL to do anything with its field. ## I want to use the `pyo3` crate re-exported from from dependency but the proc-macros fail! All PyO3 proc-macros (`#[pyclass]`, `#[pyfunction]`, `#[derive(FromPyObject)]` and so on) expect the `pyo3` crate to be available under that name in your crate root, which is the normal situation when `pyo3` is a direct dependency of your crate. However, when the dependency is renamed, or your crate only indirectly depends on `pyo3`, you need to let the macro code know where to find the crate. This is done with the `crate` attribute: ```rust # use pyo3::prelude::*; # pub extern crate pyo3; # mod reexported { pub use ::pyo3; } #[pyclass] #[pyo3(crate = "reexported::pyo3")] struct MyClass; ``` ## I'm trying to call Python from Rust but I get `STATUS_DLL_NOT_FOUND` or `STATUS_ENTRYPOINT_NOT_FOUND`! This happens on Windows when linking to the python DLL fails or the wrong one is linked. The Python DLL on Windows will usually be called something like: - `python3X.dll` for Python 3.X, e.g. `python310.dll` for Python 3.10 - `python3.dll` when using PyO3's `abi3` feature The DLL needs to be locatable using the [Windows DLL search order](https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order#standard-search-order-for-unpackaged-apps). Some ways to achieve this are: - Put the Python DLL in the same folder as your build artifacts - Add the directory containing the Python DLL to your `PATH` environment variable, for example `C:\Users\\AppData\Local\Programs\Python\Python310` - If this happens when you are *distributing* your program, consider using [PyOxidizer](https://github.com/indygreg/PyOxidizer) to package it with your binary. If the wrong DLL is linked it is possible that this happened because another program added itself and its own Python DLLs to `PATH`. Rearrange your `PATH` variables to give the correct DLL priority. > **Note**: Changes to `PATH` (or any other environment variable) are not visible to existing shells. Restart it for changes to take effect. For advanced troubleshooting, [Dependency Walker](https://www.dependencywalker.com/) can be used to diagnose linking errors. pyo3-0.20.2/guide/src/features.md000064400000000000000000000206411046102023000146260ustar 00000000000000# Features reference PyO3 provides a number of Cargo features to customize functionality. This chapter of the guide provides detail on each of them. By default, only the `macros` feature is enabled. ## Features for extension module authors ### `extension-module` This feature is required when building a Python extension module using PyO3. It tells PyO3's build script to skip linking against `libpython.so` on Unix platforms, where this must not be done. See the [building and distribution](building_and_distribution.md#linking) section for further detail. ### `abi3` This feature is used when building Python extension modules to create wheels which are compatible with multiple Python versions. It restricts PyO3's API to a subset of the full Python API which is guaranteed by [PEP 384](https://www.python.org/dev/peps/pep-0384/) to be forwards-compatible with future Python versions. See the [building and distribution](building_and_distribution.md#py_limited_apiabi3) section for further detail. ### The `abi3-pyXY` features (`abi3-py37`, `abi3-py38`, `abi3-py39`, `abi3-py310` and `abi3-py311`) These features are extensions of the `abi3` feature to specify the exact minimum Python version which the multiple-version-wheel will support. See the [building and distribution](building_and_distribution.md#minimum-python-version-for-abi3) section for further detail. ### `generate-import-lib` This experimental feature is used to generate import libraries for Python DLL for MinGW-w64 and MSVC (cross-)compile targets. Enabling it allows to (cross-)compile extension modules to any Windows targets without having to install the Windows Python distribution files for the target. See the [building and distribution](building_and_distribution.md#building-abi3-extensions-without-a-python-interpreter) section for further detail. ## Features for embedding Python in Rust ### `auto-initialize` This feature changes [`Python::with_gil`]({{#PYO3_DOCS_URL}}/pyo3/struct.Python.html#method.with_gil) to automatically initialize a Python interpreter (by calling [`prepare_freethreaded_python`]({{#PYO3_DOCS_URL}}/pyo3/fn.prepare_freethreaded_python.html)) if needed. If you do not enable this feature, you should call `pyo3::prepare_freethreaded_python()` before attempting to call any other Python APIs. ## Advanced Features ### `experimental-inspect` This feature adds the `pyo3::inspect` module, as well as `IntoPy::type_output` and `FromPyObject::type_input` APIs to produce Python type "annotations" for Rust types. This is a first step towards adding first-class support for generating type annotations automatically in PyO3, however work is needed to finish this off. All feedback and offers of help welcome on [issue #2454](https://github.com/PyO3/pyo3/issues/2454). ### `macros` This feature enables a dependency on the `pyo3-macros` crate, which provides the procedural macros portion of PyO3's API: - `#[pymodule]` - `#[pyfunction]` - `#[pyclass]` - `#[pymethods]` - `#[derive(FromPyObject)]` It also provides the `py_run!` macro. These macros require a number of dependencies which may not be needed by users who just need PyO3 for Python FFI. Disabling this feature enables faster builds for those users, as these dependencies will not be built if this feature is disabled. > This feature is enabled by default. To disable it, set `default-features = false` for the `pyo3` entry in your Cargo.toml. ### `multiple-pymethods` This feature enables a dependency on `inventory`, which enables each `#[pyclass]` to have more than one `#[pymethods]` block. This feature also requires a minimum Rust version of 1.62 due to limitations in the `inventory` crate. Most users should only need a single `#[pymethods]` per `#[pyclass]`. In addition, not all platforms (e.g. Wasm) are supported by `inventory`. For this reason this feature is not enabled by default, meaning fewer dependencies and faster compilation for the majority of users. See [the `#[pyclass]` implementation details](class.md#implementation-details) for more information. ### `nightly` The `nightly` feature needs the nightly Rust compiler. This allows PyO3 to use the `auto_traits` and `negative_impls` features to fix the `Python::allow_threads` function. ### `resolve-config` The `resolve-config` feature of the `pyo3-build-config` crate controls whether that crate's build script automatically resolves a Python interpreter / build configuration. This feature is primarily useful when building PyO3 itself. By default this feature is not enabled, meaning you can freely use `pyo3-build-config` as a standalone library to read or write PyO3 build configuration files or resolve metadata about a Python interpreter. ## Optional Dependencies These features enable conversions between Python types and types from other Rust crates, enabling easy access to the rest of the Rust ecosystem. ### `anyhow` Adds a dependency on [anyhow](https://docs.rs/anyhow). Enables a conversion from [anyhow](https://docs.rs/anyhow)’s [`Error`](https://docs.rs/anyhow/latest/anyhow/struct.Error.html) type to [`PyErr`]({{#PYO3_DOCS_URL}}/pyo3/struct.PyErr.html), for easy error handling. ### `chrono` Adds a dependency on [chrono](https://docs.rs/chrono). Enables a conversion from [chrono](https://docs.rs/chrono)'s types to python: - [Duration](https://docs.rs/chrono/latest/chrono/struct.Duration.html) -> [`PyDelta`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyDelta.html) - [FixedOffset](https://docs.rs/chrono/latest/chrono/offset/struct.FixedOffset.html) -> [`PyDelta`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyDelta.html) - [Utc](https://docs.rs/chrono/latest/chrono/offset/struct.Utc.html) -> [`PyTzInfo`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyTzInfo.html) - [NaiveDate](https://docs.rs/chrono/latest/chrono/naive/struct.NaiveDate.html) -> [`PyDate`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyDate.html) - [NaiveTime](https://docs.rs/chrono/latest/chrono/naive/struct.NaiveTime.html) -> [`PyTime`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyTime.html) - [DateTime](https://docs.rs/chrono/latest/chrono/struct.DateTime.html) -> [`PyDateTime`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyDateTime.html) ### `either` Adds a dependency on [either](https://docs.rs/either). Enables a conversions into [either](https://docs.rs/either)’s [`Either`](https://docs.rs/either/latest/either/struct.Report.html) type. ### `eyre` Adds a dependency on [eyre](https://docs.rs/eyre). Enables a conversion from [eyre](https://docs.rs/eyre)’s [`Report`](https://docs.rs/eyre/latest/eyre/struct.Report.html) type to [`PyErr`]({{#PYO3_DOCS_URL}}/pyo3/struct.PyErr.html), for easy error handling. ### `hashbrown` Adds a dependency on [hashbrown](https://docs.rs/hashbrown) and enables conversions into its [`HashMap`](https://docs.rs/hashbrown/latest/hashbrown/struct.HashMap.html) and [`HashSet`](https://docs.rs/hashbrown/latest/hashbrown/struct.HashSet.html) types. ### `indexmap` Adds a dependency on [indexmap](https://docs.rs/indexmap) and enables conversions into its [`IndexMap`](https://docs.rs/indexmap/latest/indexmap/map/struct.IndexMap.html) type. ### `num-bigint` Adds a dependency on [num-bigint](https://docs.rs/num-bigint) and enables conversions into its [`BigInt`](https://docs.rs/num-bigint/latest/num_bigint/struct.BigInt.html) and [`BigUint`](https://docs.rs/num-bigint/latest/num_bigint/struct.BigUInt.html) types. ### `num-complex` Adds a dependency on [num-complex](https://docs.rs/num-complex) and enables conversions into its [`Complex`](https://docs.rs/num-complex/latest/num_complex/struct.Complex.html) type. ### `rust_decimal` Adds a dependency on [rust_decimal](https://docs.rs/rust_decimal) and enables conversions into its [`Decimal`](https://docs.rs/rust_decimal/latest/rust_decimal/struct.Decimal.html) type. ### `serde` Enables (de)serialization of `Py` objects via [serde](https://serde.rs/). This allows to use [`#[derive(Serialize, Deserialize)`](https://serde.rs/derive.html) on structs that hold references to `#[pyclass]` instances ```rust # #[cfg(feature = "serde")] # #[allow(dead_code)] # mod serde_only { # use pyo3::prelude::*; # use serde::{Deserialize, Serialize}; #[pyclass] #[derive(Serialize, Deserialize)] struct Permission { name: String, } #[pyclass] #[derive(Serialize, Deserialize)] struct User { username: String, permissions: Vec>, } # } ``` ### `smallvec` Adds a dependency on [smallvec](https://docs.rs/smallvec) and enables conversions into its [`SmallVec`](https://docs.rs/smallvec/latest/smallvec/struct.SmallVec.html) type. pyo3-0.20.2/guide/src/function/error_handling.md000064400000000000000000000216621046102023000176360ustar 00000000000000# Error handling This chapter contains a little background of error handling in Rust and how PyO3 integrates this with Python exceptions. This covers enough detail to create a `#[pyfunction]` which raises Python exceptions from errors originating in Rust. There is a later section of the guide on [Python exceptions](../exception.md) which covers exception types in more detail. ## Representing Python exceptions Rust code uses the generic [`Result`] enum to propagate errors. The error type `E` is chosen by the code author to describe the possible errors which can happen. PyO3 has the [`PyErr`] type which represents a Python exception. If a PyO3 API could result in a Python exception being raised, the return type of that `API` will be [`PyResult`], which is an alias for the type `Result`. In summary: - When Python exceptions are raised and caught by PyO3, the exception will be stored in the `Err` variant of the `PyResult`. - Passing Python exceptions through Rust code then uses all the "normal" techniques such as the `?` operator, with `PyErr` as the error type. - Finally, when a `PyResult` crosses from Rust back to Python via PyO3, if the result is an `Err` variant the contained exception will be raised. (There are many great tutorials on Rust error handling and the `?` operator, so this guide will not go into detail on Rust-specific topics.) ## Raising an exception from a function As indicated in the previous section, when a `PyResult` containing an `Err` crosses from Rust to Python, PyO3 will raise the exception contained within. Accordingly, to raise an exception from a `#[pyfunction]`, change the return type `T` to `PyResult`. When the function returns an `Err` it will raise a Python exception. (Other `Result` types can be used as long as the error `E` has a `From` conversion for `PyErr`, see [implementing a conversion](#implementing-an-error-conversion) below.) This also works for functions in `#[pymethods]`. For example, the following `check_positive` function raises a `ValueError` when the input is negative: ```rust use pyo3::exceptions::PyValueError; use pyo3::prelude::*; #[pyfunction] fn check_positive(x: i32) -> PyResult<()> { if x < 0 { Err(PyValueError::new_err("x is negative")) } else { Ok(()) } } # # fn main(){ # Python::with_gil(|py|{ # let fun = pyo3::wrap_pyfunction!(check_positive, py).unwrap(); # fun.call1((-1,)).unwrap_err(); # fun.call1((1,)).unwrap(); # }); # } ``` All built-in Python exception types are defined in the [`pyo3::exceptions`] module. They have a `new_err` constructor to directly build a `PyErr`, as seen in the example above. ## Custom Rust error types PyO3 will automatically convert a `Result` returned by a `#[pyfunction]` into a `PyResult` as long as there is an implementation of `std::from::From for PyErr`. Many error types in the Rust standard library have a [`From`] conversion defined in this way. If the type `E` you are handling is defined in a third-party crate, see the section on [foreign rust error types](#foreign-rust-error-types) below for ways to work with this error. The following example makes use of the implementation of `From for PyErr` to raise exceptions encountered when parsing strings as integers: ```rust # use pyo3::prelude::*; use std::num::ParseIntError; #[pyfunction] fn parse_int(x: &str) -> Result { x.parse() } # fn main() { # Python::with_gil(|py| { # let fun = pyo3::wrap_pyfunction!(parse_int, py).unwrap(); # let value: usize = fun.call1(("5",)).unwrap().extract().unwrap(); # assert_eq!(value, 5); # }); # } ``` When passed a string which doesn't contain a floating-point number, the exception raised will look like the below: ```python >>> parse_int("bar") Traceback (most recent call last): File "", line 1, in ValueError: invalid digit found in string ``` As a more complete example, the following snippet defines a Rust error named `CustomIOError`. It then defines a `From for PyErr`, which returns a `PyErr` representing Python's `OSError`. Therefore, it can use this error in the result of a `#[pyfunction]` directly, relying on the conversion if it has to be propagated into a Python exception. ```rust use pyo3::exceptions::PyOSError; use pyo3::prelude::*; use std::fmt; #[derive(Debug)] struct CustomIOError; impl std::error::Error for CustomIOError {} impl fmt::Display for CustomIOError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "Oh no!") } } impl std::convert::From for PyErr { fn from(err: CustomIOError) -> PyErr { PyOSError::new_err(err.to_string()) } } pub struct Connection {/* ... */} fn bind(addr: String) -> Result { if &addr == "0.0.0.0" { Err(CustomIOError) } else { Ok(Connection{ /* ... */}) } } #[pyfunction] fn connect(s: String) -> Result<(), CustomIOError> { bind(s)?; // etc. Ok(()) } fn main() { Python::with_gil(|py| { let fun = pyo3::wrap_pyfunction!(connect, py).unwrap(); let err = fun.call1(("0.0.0.0",)).unwrap_err(); assert!(err.is_instance_of::(py)); }); } ``` If lazy construction of the Python exception instance is desired, the [`PyErrArguments`]({{#PYO3_DOCS_URL}}/pyo3/trait.PyErrArguments.html) trait can be implemented instead of `From`. In that case, actual exception argument creation is delayed until the `PyErr` is needed. A final note is that any errors `E` which have a `From` conversion can be used with the `?` ("try") operator with them. An alternative implementation of the above `parse_int` which instead returns `PyResult` is below: ```rust use pyo3::prelude::*; fn parse_int(s: String) -> PyResult { let x = s.parse()?; Ok(x) } # # use pyo3::exceptions::PyValueError; # # fn main() { # Python::with_gil(|py| { # assert_eq!(parse_int(String::from("1")).unwrap(), 1); # assert_eq!(parse_int(String::from("1337")).unwrap(), 1337); # # assert!(parse_int(String::from("-1")) # .unwrap_err() # .is_instance_of::(py)); # assert!(parse_int(String::from("foo")) # .unwrap_err() # .is_instance_of::(py)); # assert!(parse_int(String::from("13.37")) # .unwrap_err() # .is_instance_of::(py)); # }) # } ``` ## Foreign Rust error types The Rust compiler will not permit implementation of traits for types outside of the crate where the type is defined. (This is known as the "orphan rule".) Given a type `OtherError` which is defined in third-party code, there are two main strategies available to integrate it with PyO3: - Create a newtype wrapper, e.g. `MyOtherError`. Then implement `From for PyErr` (or `PyErrArguments`), as well as `From` for `MyOtherError`. - Use Rust's Result combinators such as `map_err` to write code freely to convert `OtherError` into whatever is needed. This requires boilerplate at every usage however gives unlimited flexibility. To detail the newtype strategy a little further, the key trick is to return `Result` from the `#[pyfunction]`. This means that PyO3 will make use of `From for PyErr` to create Python exceptions while the `#[pyfunction]` implementation can use `?` to convert `OtherError` to `MyOtherError` automatically. The following example demonstrates this for some imaginary third-party crate `some_crate` with a function `get_x` returning `Result`: ```rust # mod some_crate { # pub struct OtherError(()); # impl OtherError { # pub fn message(&self) -> &'static str { "some error occurred" } # } # pub fn get_x() -> Result { Ok(5) } # } use pyo3::prelude::*; use pyo3::exceptions::PyValueError; use some_crate::{OtherError, get_x}; struct MyOtherError(OtherError); impl From for PyErr { fn from(error: MyOtherError) -> Self { PyValueError::new_err(error.0.message()) } } impl From for MyOtherError { fn from(other: OtherError) -> Self { Self(other) } } #[pyfunction] fn wrapped_get_x() -> Result { // get_x is a function returning Result let x: i32 = get_x()?; Ok(x) } # fn main() { # Python::with_gil(|py| { # let fun = pyo3::wrap_pyfunction!(wrapped_get_x, py).unwrap(); # let value: usize = fun.call0().unwrap().extract().unwrap(); # assert_eq!(value, 5); # }); # } ``` [`From`]: https://doc.rust-lang.org/stable/std/convert/trait.From.html [`Result`]: https://doc.rust-lang.org/stable/std/result/enum.Result.html [`PyResult`]: {{#PYO3_DOCS_URL}}/pyo3/prelude/type.PyResult.html [`PyResult`]: {{#PYO3_DOCS_URL}}/pyo3/prelude/type.PyResult.html [`PyErr`]: {{#PYO3_DOCS_URL}}/pyo3/struct.PyErr.html [`pyo3::exceptions`]: {{#PYO3_DOCS_URL}}/pyo3/exceptions/index.html pyo3-0.20.2/guide/src/function/signature.md000064400000000000000000000247531046102023000166460ustar 00000000000000# Function signatures The `#[pyfunction]` attribute also accepts parameters to control how the generated Python function accepts arguments. Just like in Python, arguments can be positional-only, keyword-only, or accept either. `*args` lists and `**kwargs` dicts can also be accepted. These parameters also work for `#[pymethods]` which will be introduced in the [Python Classes](../class.md) section of the guide. Like Python, by default PyO3 accepts all arguments as either positional or keyword arguments. Most arguments are required by default, except for trailing `Option<_>` arguments, which are [implicitly given a default of `None`](#trailing-optional-arguments). There are two ways to modify this behaviour: - The `#[pyo3(signature = (...))]` option which allows writing a signature in Python syntax. - Extra arguments directly to `#[pyfunction]`. (See deprecated form) ## Using `#[pyo3(signature = (...))]` For example, below is a function that accepts arbitrary keyword arguments (`**kwargs` in Python syntax) and returns the number that was passed: ```rust use pyo3::prelude::*; use pyo3::types::PyDict; #[pyfunction] #[pyo3(signature = (**kwds))] fn num_kwds(kwds: Option<&PyDict>) -> usize { kwds.map_or(0, |dict| dict.len()) } #[pymodule] fn module_with_functions(py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(num_kwds, m)?).unwrap(); Ok(()) } ``` Just like in Python, the following constructs can be part of the signature:: * `/`: positional-only arguments separator, each parameter defined before `/` is a positional-only parameter. * `*`: var arguments separator, each parameter defined after `*` is a keyword-only parameter. * `*args`: "args" is var args. Type of the `args` parameter has to be `&PyTuple`. * `**kwargs`: "kwargs" receives keyword arguments. The type of the `kwargs` parameter has to be `Option<&PyDict>`. * `arg=Value`: arguments with default value. If the `arg` argument is defined after var arguments, it is treated as a keyword-only argument. Note that `Value` has to be valid rust code, PyO3 just inserts it into the generated code unmodified. Example: ```rust # use pyo3::prelude::*; use pyo3::types::{PyDict, PyTuple}; # # #[pyclass] # struct MyClass { # num: i32, # } #[pymethods] impl MyClass { #[new] #[pyo3(signature = (num=-1))] fn new(num: i32) -> Self { MyClass { num } } #[pyo3(signature = (num=10, *py_args, name="Hello", **py_kwargs))] fn method( &mut self, num: i32, py_args: &PyTuple, name: &str, py_kwargs: Option<&PyDict>, ) -> String { let num_before = self.num; self.num = num; format!( "num={} (was previously={}), py_args={:?}, name={}, py_kwargs={:?} ", num, num_before, py_args, name, py_kwargs, ) } fn make_change(&mut self, num: i32) -> PyResult { self.num = num; Ok(format!("num={}", self.num)) } } ``` Arguments of type `Python` must not be part of the signature: ```rust # #![allow(dead_code)] # use pyo3::prelude::*; #[pyfunction] #[pyo3(signature = (lambda))] pub fn simple_python_bound_function(py: Python<'_>, lambda: PyObject) -> PyResult<()> { Ok(()) } ``` N.B. the position of the `/` and `*` arguments (if included) control the system of handling positional and keyword arguments. In Python: ```python import mymodule mc = mymodule.MyClass() print(mc.method(44, False, "World", 666, x=44, y=55)) print(mc.method(num=-1, name="World")) print(mc.make_change(44, False)) ``` Produces output: ```text py_args=('World', 666), py_kwargs=Some({'x': 44, 'y': 55}), name=Hello, num=44 py_args=(), py_kwargs=None, name=World, num=-1 num=44 num=-1 ``` > Note: to use keywords like `struct` as a function argument, use "raw identifier" syntax `r#struct` in both the signature and the function definition: > > ```rust > # #![allow(dead_code)] > # use pyo3::prelude::*; > #[pyfunction(signature = (r#struct = "foo"))] > fn function_with_keyword(r#struct: &str) { > # let _ = r#struct; > /* ... */ > } > ``` ## Trailing optional arguments As a convenience, functions without a `#[pyo3(signature = (...))]` option will treat trailing `Option` arguments as having a default of `None`. In the example below, PyO3 will create `increment` with a signature of `increment(x, amount=None)`. ```rust use pyo3::prelude::*; /// Returns a copy of `x` increased by `amount`. /// /// If `amount` is unspecified or `None`, equivalent to `x + 1`. #[pyfunction] fn increment(x: u64, amount: Option) -> u64 { x + amount.unwrap_or(1) } # # fn main() -> PyResult<()> { # Python::with_gil(|py| { # let fun = pyo3::wrap_pyfunction!(increment, py)?; # # let inspect = PyModule::import(py, "inspect")?.getattr("signature")?; # let sig: String = inspect # .call1((fun,))? # .call_method0("__str__")? # .extract()?; # # #[cfg(Py_3_8)] // on 3.7 the signature doesn't render b, upstream bug? # assert_eq!(sig, "(x, amount=None)"); # # Ok(()) # }) # } ``` To make trailing `Option` arguments required, but still accept `None`, add a `#[pyo3(signature = (...))]` annotation. For the example above, this would be `#[pyo3(signature = (x, amount))]`: ```rust # use pyo3::prelude::*; #[pyfunction] #[pyo3(signature = (x, amount))] fn increment(x: u64, amount: Option) -> u64 { x + amount.unwrap_or(1) } # # fn main() -> PyResult<()> { # Python::with_gil(|py| { # let fun = pyo3::wrap_pyfunction!(increment, py)?; # # let inspect = PyModule::import(py, "inspect")?.getattr("signature")?; # let sig: String = inspect # .call1((fun,))? # .call_method0("__str__")? # .extract()?; # # #[cfg(Py_3_8)] // on 3.7 the signature doesn't render b, upstream bug? # assert_eq!(sig, "(x, amount)"); # # Ok(()) # }) # } ``` To help avoid confusion, PyO3 requires `#[pyo3(signature = (...))]` when an `Option` argument is surrounded by arguments which aren't `Option`. ## Making the function signature available to Python The function signature is exposed to Python via the `__text_signature__` attribute. PyO3 automatically generates this for every `#[pyfunction]` and all `#[pymethods]` directly from the Rust function, taking into account any override done with the `#[pyo3(signature = (...))]` option. This automatic generation can only display the value of default arguments for strings, integers, boolean types, and `None`. Any other default arguments will be displayed as `...`. (`.pyi` type stub files commonly also use `...` for default arguments in the same way.) In cases where the automatically-generated signature needs adjusting, it can [be overridden](#overriding-the-generated-signature) using the `#[pyo3(text_signature)]` option.) The example below creates a function `add` which accepts two positional-only arguments `a` and `b`, where `b` has a default value of zero. ```rust use pyo3::prelude::*; /// This function adds two unsigned 64-bit integers. #[pyfunction] #[pyo3(signature = (a, b=0, /))] fn add(a: u64, b: u64) -> u64 { a + b } # # fn main() -> PyResult<()> { # Python::with_gil(|py| { # let fun = pyo3::wrap_pyfunction!(add, py)?; # # let doc: String = fun.getattr("__doc__")?.extract()?; # assert_eq!(doc, "This function adds two unsigned 64-bit integers."); # # let inspect = PyModule::import(py, "inspect")?.getattr("signature")?; # let sig: String = inspect # .call1((fun,))? # .call_method0("__str__")? # .extract()?; # # #[cfg(Py_3_8)] // on 3.7 the signature doesn't render b, upstream bug? # assert_eq!(sig, "(a, b=0, /)"); # # Ok(()) # }) # } ``` The following IPython output demonstrates how this generated signature will be seen from Python tooling: ```text >>> pyo3_test.add.__text_signature__ '(a, b=..., /)' >>> pyo3_test.add? Signature: pyo3_test.add(a, b=0, /) Docstring: This function adds two unsigned 64-bit integers. Type: builtin_function_or_method ``` ### Overriding the generated signature The `#[pyo3(text_signature = "()")]` attribute can be used to override the default generated signature. In the snippet below, the text signature attribute is used to include the default value of `0` for the argument `b`, instead of the automatically-generated default value of `...`: ```rust use pyo3::prelude::*; /// This function adds two unsigned 64-bit integers. #[pyfunction] #[pyo3(signature = (a, b=0, /), text_signature = "(a, b=0, /)")] fn add(a: u64, b: u64) -> u64 { a + b } # # fn main() -> PyResult<()> { # Python::with_gil(|py| { # let fun = pyo3::wrap_pyfunction!(add, py)?; # # let doc: String = fun.getattr("__doc__")?.extract()?; # assert_eq!(doc, "This function adds two unsigned 64-bit integers."); # # let inspect = PyModule::import(py, "inspect")?.getattr("signature")?; # let sig: String = inspect # .call1((fun,))? # .call_method0("__str__")? # .extract()?; # assert_eq!(sig, "(a, b=0, /)"); # # Ok(()) # }) # } ``` PyO3 will include the contents of the annotation unmodified as the `__text_signature`. Below shows how IPython will now present this (see the default value of 0 for b): ```text >>> pyo3_test.add.__text_signature__ '(a, b=0, /)' >>> pyo3_test.add? Signature: pyo3_test.add(a, b=0, /) Docstring: This function adds two unsigned 64-bit integers. Type: builtin_function_or_method ``` If no signature is wanted at all, `#[pyo3(text_signature = None)]` will disable the built-in signature. The snippet below demonstrates use of this: ```rust use pyo3::prelude::*; /// This function adds two unsigned 64-bit integers. #[pyfunction] #[pyo3(signature = (a, b=0, /), text_signature = None)] fn add(a: u64, b: u64) -> u64 { a + b } # # fn main() -> PyResult<()> { # Python::with_gil(|py| { # let fun = pyo3::wrap_pyfunction!(add, py)?; # # let doc: String = fun.getattr("__doc__")?.extract()?; # assert_eq!(doc, "This function adds two unsigned 64-bit integers."); # assert!(fun.getattr("__text_signature__")?.is_none()); # # Ok(()) # }) # } ``` Now the function's `__text_signature__` will be set to `None`, and IPython will not display any signature in the help: ```text >>> pyo3_test.add.__text_signature__ == None True >>> pyo3_test.add? Docstring: This function adds two unsigned 64-bit integers. Type: builtin_function_or_method ``` pyo3-0.20.2/guide/src/function.md000064400000000000000000000173731046102023000146450ustar 00000000000000# Python functions The `#[pyfunction]` attribute is used to define a Python function from a Rust function. Once defined, the function needs to be added to a [module](./module.md) using the `wrap_pyfunction!` macro. The following example defines a function called `double` in a Python module called `my_extension`: ```rust use pyo3::prelude::*; #[pyfunction] fn double(x: usize) -> usize { x * 2 } #[pymodule] fn my_extension(py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(double, m)?)?; Ok(()) } ``` This chapter of the guide explains full usage of the `#[pyfunction]` attribute. In this first section, the following topics are covered: - [Function options](#function-options) - [`#[pyo3(name = "...")]`](#name) - [`#[pyo3(signature = (...))]`](#signature) - [`#[pyo3(text_signature = "...")]`](#text_signature) - [`#[pyo3(pass_module)]`](#pass_module) - [Per-argument options](#per-argument-options) - [Advanced function patterns](#advanced-function-patterns) - [`#[pyfn]` shorthand](#pyfn-shorthand) There are also additional sections on the following topics: - [Function Signatures](./function/signature.md) ## Function options The `#[pyo3]` attribute can be used to modify properties of the generated Python function. It can take any combination of the following options: - `#[pyo3(name = "...")]` Overrides the name exposed to Python. In the following example, the Rust function `no_args_py` will be added to the Python module `module_with_functions` as the Python function `no_args`: ```rust use pyo3::prelude::*; #[pyfunction] #[pyo3(name = "no_args")] fn no_args_py() -> usize { 42 } #[pymodule] fn module_with_functions(py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(no_args_py, m)?)?; Ok(()) } # Python::with_gil(|py| { # let m = pyo3::wrap_pymodule!(module_with_functions)(py); # assert!(m.getattr(py, "no_args").is_ok()); # assert!(m.getattr(py, "no_args_py").is_err()); # }); ``` - `#[pyo3(signature = (...))]` Defines the function signature in Python. See [Function Signatures](./function/signature.md). - `#[pyo3(text_signature = "...")]` Overrides the PyO3-generated function signature visible in Python tooling (such as via [`inspect.signature`]). See the [corresponding topic in the Function Signatures subchapter](./function/signature.md#making-the-function-signature-available-to-python). - `#[pyo3(pass_module)]` Set this option to make PyO3 pass the containing module as the first argument to the function. It is then possible to use the module in the function body. The first argument **must** be of type `&PyModule`. The following example creates a function `pyfunction_with_module` which returns the containing module's name (i.e. `module_with_fn`): ```rust use pyo3::prelude::*; #[pyfunction] #[pyo3(pass_module)] fn pyfunction_with_module(module: &PyModule) -> PyResult<&str> { module.name() } #[pymodule] fn module_with_fn(py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(pyfunction_with_module, m)?) } ``` ## Per-argument options The `#[pyo3]` attribute can be used on individual arguments to modify properties of them in the generated function. It can take any combination of the following options: - `#[pyo3(from_py_with = "...")]` Set this on an option to specify a custom function to convert the function argument from Python to the desired Rust type, instead of using the default `FromPyObject` extraction. The function signature must be `fn(&PyAny) -> PyResult` where `T` is the Rust type of the argument. The following example uses `from_py_with` to convert the input Python object to its length: ```rust use pyo3::prelude::*; fn get_length(obj: &PyAny) -> PyResult { let length = obj.len()?; Ok(length) } #[pyfunction] fn object_length(#[pyo3(from_py_with = "get_length")] argument: usize) -> usize { argument } # Python::with_gil(|py| { # let f = pyo3::wrap_pyfunction!(object_length)(py).unwrap(); # assert_eq!(f.call1((vec![1, 2, 3],)).unwrap().extract::().unwrap(), 3); # }); ``` ## Advanced function patterns ### Calling Python functions in Rust You can pass Python `def`'d functions and built-in functions to Rust functions [`PyFunction`] corresponds to regular Python functions while [`PyCFunction`] describes built-ins such as `repr()`. You can also use [`PyAny::is_callable`] to check if you have a callable object. `is_callable` will return `true` for functions (including lambdas), methods and objects with a `__call__` method. You can call the object with [`PyAny::call`] with the args as first parameter and the kwargs (or `None`) as second parameter. There are also [`PyAny::call0`] with no args and [`PyAny::call1`] with only positional args. ### Calling Rust functions in Python The ways to convert a Rust function into a Python object vary depending on the function: - Named functions, e.g. `fn foo()`: add `#[pyfunction]` and then use [`wrap_pyfunction!`] to get the corresponding [`PyCFunction`]. - Anonymous functions (or closures), e.g. `foo: fn()` either: - use a `#[pyclass]` struct which stores the function as a field and implement `__call__` to call the stored function. - use `PyCFunction::new_closure` to create an object directly from the function. [`PyAny::is_callable`]: {{#PYO3_DOCS_URL}}/pyo3/struct.PyAny.html#tymethod.is_callable [`PyAny::call`]: {{#PYO3_DOCS_URL}}/pyo3/struct.PyAny.html#tymethod.call [`PyAny::call0`]: {{#PYO3_DOCS_URL}}/pyo3/struct.PyAny.html#tymethod.call0 [`PyAny::call1`]: {{#PYO3_DOCS_URL}}/pyo3/struct.PyAny.html#tymethod.call1 [`PyObject`]: {{#PYO3_DOCS_URL}}/pyo3/type.PyObject.html [`wrap_pyfunction!`]: {{#PYO3_DOCS_URL}}/pyo3/macro.wrap_pyfunction.html [`PyFunction`]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PyFunction.html [`PyCFunction`]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PyCFunction.html ### Accessing the FFI functions In order to make Rust functions callable from Python, PyO3 generates an `extern "C"` function whose exact signature depends on the Rust signature. (PyO3 chooses the optimal Python argument passing convention.) It then embeds the call to the Rust function inside this FFI-wrapper function. This wrapper handles extraction of the regular arguments and the keyword arguments from the input `PyObject`s. The `wrap_pyfunction` macro can be used to directly get a `PyCFunction` given a `#[pyfunction]` and a `PyModule`: `wrap_pyfunction!(rust_fun, module)`. ## `#[pyfn]` shorthand There is a shorthand to `#[pyfunction]` and `wrap_pymodule!`: the function can be placed inside the module definition and annotated with `#[pyfn]`. To simplify PyO3, it is expected that `#[pyfn]` may be removed in a future release (See [#694](https://github.com/PyO3/pyo3/issues/694)). An example of `#[pyfn]` is below: ```rust use pyo3::prelude::*; #[pymodule] fn my_extension(py: Python<'_>, m: &PyModule) -> PyResult<()> { #[pyfn(m)] fn double(x: usize) -> usize { x * 2 } Ok(()) } ``` `#[pyfn(m)]` is just syntactic sugar for `#[pyfunction]`, and takes all the same options documented in the rest of this chapter. The code above is expanded to the following: ```rust use pyo3::prelude::*; #[pymodule] fn my_extension(py: Python<'_>, m: &PyModule) -> PyResult<()> { #[pyfunction] fn double(x: usize) -> usize { x * 2 } m.add_function(wrap_pyfunction!(double, m)?)?; Ok(()) } ``` [`inspect.signature`]: https://docs.python.org/3/library/inspect.html#inspect.signature pyo3-0.20.2/guide/src/getting_started.md000064400000000000000000000142511046102023000161770ustar 00000000000000# Installation To get started using PyO3 you will need three things: a Rust toolchain, a Python environment, and a way to build. We'll cover each of these below. ## Rust First, make sure you have Rust installed on your system. If you haven't already done so, try following the instructions [here](https://www.rust-lang.org/tools/install). PyO3 runs on both the `stable` and `nightly` versions so you can choose whichever one fits you best. The minimum required Rust version is 1.56. If you can run `rustc --version` and the version is new enough you're good to go! ## Python To use PyO3, you need at least Python 3.7. While you can simply use the default Python interpreter on your system, it is recommended to use a virtual environment. ## Virtualenvs While you can use any virtualenv manager you like, we recommend the use of `pyenv` in particular if you want to develop or test for multiple different Python versions, so that is what the examples in this book will use. The installation instructions for `pyenv` can be found [here](https://github.com/pyenv/pyenv#getting-pyenv). (Note: To get the `pyenv activate` and `pyenv virtualenv` commands, you will also need to install the [`pyenv-virtualenv`](https://github.com/pyenv/pyenv-virtualenv) plugin. The [pyenv installer](https://github.com/pyenv/pyenv-installer#installation--update--uninstallation) will install both together.) If you intend to run Python from Rust (for example in unit tests) you should set the following environment variable when installing a new Python version using `pyenv`: ```bash PYTHON_CONFIGURE_OPTS="--enable-shared" ``` For example: ```bash env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.12 ``` You can read more about `pyenv`'s configuration options [here](https://github.com/pyenv/pyenv/blob/master/plugins/python-build/README.md#building-with---enable-shared). ### Building There are a number of build and Python package management systems such as [`setuptools-rust`](https://github.com/PyO3/setuptools-rust) or [manually](https://pyo3.rs/latest/building_and_distribution.html#manual-builds). We recommend the use of `maturin`, which you can install [here](https://maturin.rs/installation.html). It is developed to work with PyO3 and provides the most "batteries included" experience, especially if you are aiming to publish to PyPI. `maturin` is just a Python package, so you can add it in the same you already install Python packages. System Python: ```bash pip install maturin --user ``` pipx: ```bash pipx install maturin ``` pyenv: ```bash pyenv activate pyo3 pip install maturin ``` poetry: ```bash poetry add -G dev maturin ``` After installation, you can run `maturin --version` to check that you have correctly installed it. # Starting a new project First you should create the folder and virtual environment that are going to contain your new project. Here we will use the recommended `pyenv`: ```bash mkdir pyo3-example cd pyo3-example pyenv virtualenv pyo3 pyenv local pyo3 ``` After this, you should install your build manager. In this example, we will use `maturin`. After you've activated your virtualenv, add `maturin` to it: ```bash pip install maturin ``` Now you can initialize the new project: ```bash maturin init ``` If `maturin` is already installed, you can create a new project using that directly as well: ```bash maturin new -b pyo3 pyo3-example cd pyo3-example pyenv virtualenv pyo3 pyenv local pyo3 ``` # Adding to an existing project Sadly, `maturin` cannot currently be run in existing projects, so if you want to use Python in an existing project you basically have two options: 1. Create a new project as above and move your existing code into that project 2. Manually edit your project configuration as necessary If you opt for the second option, here are the things you need to pay attention to: ## Cargo.toml Make sure that the Rust crate you want to be able to access from Python is compiled into a library. You can have a binary output as well, but the code you want to access from Python has to be in the library part. Also, make sure that the crate type is `cdylib` and add PyO3 as a dependency as so: ```toml # If you already have [package] information in `Cargo.toml`, you can ignore # this section! [package] # `name` here is name of the package. name = "pyo3_start" # these are good defaults: version = "0.1.0" edition = "2021" [lib] # The name of the native library. This is the name which will be used in Python to import the # library (i.e. `import string_sum`). If you change this, you must also change the name of the # `#[pymodule]` in `src/lib.rs`. name = "pyo3_example" # "cdylib" is necessary to produce a shared library for Python to import from. crate-type = ["cdylib"] [dependencies] pyo3 = { {{#PYO3_CRATE_VERSION}}, features = ["extension-module"] } ``` ## pyproject.toml You should also create a `pyproject.toml` with the following contents: ```toml [build-system] requires = ["maturin>=1,<2"] build-backend = "maturin" [project] name = "pyo3_example" requires-python = ">=3.7" classifiers = [ "Programming Language :: Rust", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy", ] ``` ## Running code After this you can setup Rust code to be available in Python as below; for example, you can place this code in `src/lib.rs`: ```rust use pyo3::prelude::*; /// Formats the sum of two numbers as string. #[pyfunction] fn sum_as_string(a: usize, b: usize) -> PyResult { Ok((a + b).to_string()) } /// A Python module implemented in Rust. The name of this function must match /// the `lib.name` setting in the `Cargo.toml`, else Python will not be able to /// import the module. #[pymodule] fn pyo3_example(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(sum_as_string, m)?)?; Ok(()) } ``` Now you can run `maturin develop` to prepare the Python package, after which you can use it like so: ```bash $ maturin develop # lots of progress output as maturin runs the compilation... $ python >>> import pyo3_example >>> pyo3_example.sum_as_string(5, 20) '25' ``` For more instructions on how to use Python code from Rust, see the [Python from Rust](python_from_rust.md) page. pyo3-0.20.2/guide/src/index.md000064400000000000000000000005511046102023000141150ustar 00000000000000# The PyO3 user guide Welcome to the PyO3 user guide! This book is a companion to [PyO3's API docs](https://docs.rs/pyo3). It contains examples and documentation to explain all of PyO3's use cases in detail. Please choose from the chapters on the left to jump to individual topics, or continue below to start with PyO3's README. {{#include ../../README.md}} pyo3-0.20.2/guide/src/memory.md000064400000000000000000000212451046102023000143210ustar 00000000000000# Memory management Rust and Python have very different notions of memory management. Rust has a strict memory model with concepts of ownership, borrowing, and lifetimes, where memory is freed at predictable points in program execution. Python has a looser memory model in which variables are reference-counted with shared, mutable state by default. A global interpreter lock (GIL) is needed to prevent race conditions, and a garbage collector is needed to break reference cycles. Memory in Python is freed eventually by the garbage collector, but not usually in a predictable way. PyO3 bridges the Rust and Python memory models with two different strategies for accessing memory allocated on Python's heap from inside Rust. These are GIL-bound, or "owned" references, and GIL-independent `Py` smart pointers. ## GIL-bound memory PyO3's GIL-bound, "owned references" (`&PyAny` etc.) make PyO3 more ergonomic to use by ensuring that their lifetime can never be longer than the duration the Python GIL is held. This means that most of PyO3's API can assume the GIL is held. (If PyO3 could not assume this, every PyO3 API would need to take a `Python` GIL token to prove that the GIL is held.) This allows us to write very simple and easy-to-understand programs like this: ```rust # use pyo3::prelude::*; # use pyo3::types::PyString; # fn main() -> PyResult<()> { Python::with_gil(|py| -> PyResult<()> { let hello: &PyString = py.eval("\"Hello World!\"", None, None)?.extract()?; println!("Python says: {}", hello); Ok(()) })?; # Ok(()) # } ``` Internally, calling `Python::with_gil()` creates a `GILPool` which owns the memory pointed to by the reference. In the example above, the lifetime of the reference `hello` is bound to the `GILPool`. When the `with_gil()` closure ends the `GILPool` is also dropped and the Python reference counts of the variables it owns are decreased, releasing them to the Python garbage collector. Most of the time we don't have to think about this, but consider the following: ```rust # use pyo3::prelude::*; # use pyo3::types::PyString; # fn main() -> PyResult<()> { Python::with_gil(|py| -> PyResult<()> { for _ in 0..10 { let hello: &PyString = py.eval("\"Hello World!\"", None, None)?.extract()?; println!("Python says: {}", hello); } // There are 10 copies of `hello` on Python's heap here. Ok(()) })?; # Ok(()) # } ``` We might assume that the `hello` variable's memory is freed at the end of each loop iteration, but in fact we create 10 copies of `hello` on Python's heap. This may seem surprising at first, but it is completely consistent with Rust's memory model. The `hello` variable is dropped at the end of each loop, but it is only a reference to the memory owned by the `GILPool`, and its lifetime is bound to the `GILPool`, not the for loop. The `GILPool` isn't dropped until the end of the `with_gil()` closure, at which point the 10 copies of `hello` are finally released to the Python garbage collector. In general we don't want unbounded memory growth during loops! One workaround is to acquire and release the GIL with each iteration of the loop. ```rust # use pyo3::prelude::*; # use pyo3::types::PyString; # fn main() -> PyResult<()> { for _ in 0..10 { Python::with_gil(|py| -> PyResult<()> { let hello: &PyString = py.eval("\"Hello World!\"", None, None)?.extract()?; println!("Python says: {}", hello); Ok(()) })?; // only one copy of `hello` at a time } # Ok(()) # } ``` It might not be practical or performant to acquire and release the GIL so many times. Another workaround is to work with the `GILPool` object directly, but this is unsafe. ```rust # use pyo3::prelude::*; # use pyo3::types::PyString; # fn main() -> PyResult<()> { Python::with_gil(|py| -> PyResult<()> { for _ in 0..10 { let pool = unsafe { py.new_pool() }; let py = pool.python(); let hello: &PyString = py.eval("\"Hello World!\"", None, None)?.extract()?; println!("Python says: {}", hello); } Ok(()) })?; # Ok(()) # } ``` The unsafe method `Python::new_pool` allows you to create a nested `GILPool` from which you can retrieve a new `py: Python` GIL token. Variables created with this new GIL token are bound to the nested `GILPool` and will be released when the nested `GILPool` is dropped. Here, the nested `GILPool` is dropped at the end of each loop iteration, before the `with_gil()` closure ends. When doing this, you must be very careful to ensure that once the `GILPool` is dropped you do not retain access to any owned references created after the `GILPool` was created. Read the [documentation for `Python::new_pool()`]({{#PYO3_DOCS_URL}}/pyo3/prelude/struct.Python.html#method.new_pool) for more information on safety. This memory management can also be applicable when writing extension modules. `#[pyfunction]` and `#[pymethods]` will create a `GILPool` which lasts the entire function call, releasing objects when the function returns. Most functions only create a few objects, meaning this doesn't have a significant impact. Occasionally functions with long complex loops may need to use `Python::new_pool` as shown above. This behavior may change in future, see [issue #1056](https://github.com/PyO3/pyo3/issues/1056). ## GIL-independent memory Sometimes we need a reference to memory on Python's heap that can outlive the GIL. Python's `Py` is analogous to `Arc`, but for variables whose memory is allocated on Python's heap. Cloning a `Py` increases its internal reference count just like cloning `Arc`. The smart pointer can outlive the "GIL is held" period in which it was created. It isn't magic, though. We need to reacquire the GIL to access the memory pointed to by the `Py`. What happens to the memory when the last `Py` is dropped and its reference count reaches zero? It depends whether or not we are holding the GIL. ```rust # use pyo3::prelude::*; # use pyo3::types::PyString; # fn main() -> PyResult<()> { Python::with_gil(|py| -> PyResult<()> { let hello: Py = py.eval("\"Hello World!\"", None, None)?.extract()?; println!("Python says: {}", hello.as_ref(py)); Ok(()) })?; # Ok(()) # } ``` At the end of the `Python::with_gil()` closure `hello` is dropped, and then the GIL is dropped. Since `hello` is dropped while the GIL is still held by the current thread, its memory is released to the Python garbage collector immediately. This example wasn't very interesting. We could have just used a GIL-bound `&PyString` reference. What happens when the last `Py` is dropped while we are *not* holding the GIL? ```rust # use pyo3::prelude::*; # use pyo3::types::PyString; # fn main() -> PyResult<()> { let hello: Py = Python::with_gil(|py| { py.eval("\"Hello World!\"", None, None)?.extract() })?; // Do some stuff... // Now sometime later in the program we want to access `hello`. Python::with_gil(|py| { println!("Python says: {}", hello.as_ref(py)); }); // Now we're done with `hello`. drop(hello); // Memory *not* released here. // Sometime later we need the GIL again for something... Python::with_gil(|py| // Memory for `hello` is released here. # () ); # Ok(()) # } ``` When `hello` is dropped *nothing* happens to the pointed-to memory on Python's heap because nothing _can_ happen if we're not holding the GIL. Fortunately, the memory isn't leaked. PyO3 keeps track of the memory internally and will release it the next time we acquire the GIL. We can avoid the delay in releasing memory if we are careful to drop the `Py` while the GIL is held. ```rust # use pyo3::prelude::*; # use pyo3::types::PyString; # fn main() -> PyResult<()> { let hello: Py = Python::with_gil(|py| py.eval("\"Hello World!\"", None, None)?.extract())?; // Do some stuff... // Now sometime later in the program: Python::with_gil(|py| { println!("Python says: {}", hello.as_ref(py)); drop(hello); // Memory released here. }); # Ok(()) # } ``` We could also have used `Py::into_ref()`, which consumes `self`, instead of `Py::as_ref()`. But note that in addition to being slower than `as_ref()`, `into_ref()` binds the memory to the lifetime of the `GILPool`, which means that rather than being released immediately, the memory will not be released until the GIL is dropped. ```rust # use pyo3::prelude::*; # use pyo3::types::PyString; # fn main() -> PyResult<()> { let hello: Py = Python::with_gil(|py| py.eval("\"Hello World!\"", None, None)?.extract())?; // Do some stuff... // Now sometime later in the program: Python::with_gil(|py| { println!("Python says: {}", hello.into_ref(py)); // Memory not released yet. // Do more stuff... // Memory released here at end of `with_gil()` closure. }); # Ok(()) # } ``` pyo3-0.20.2/guide/src/migration.md000064400000000000000000001172661046102023000150130ustar 00000000000000# Migrating from older PyO3 versions This guide can help you upgrade code through breaking changes from one PyO3 version to the next. For a detailed list of all changes, see the [CHANGELOG](changelog.md). ## from 0.19.* to 0.20 ### Drop support for older technologies PyO3 0.20 has increased minimum Rust version to 1.56. This enables use of newer language features and simplifies maintenance of the project. ### `PyDict::get_item` now returns a `Result` `PyDict::get_item` in PyO3 0.19 and older was implemented using a Python API which would suppress all exceptions and return `None` in those cases. This included errors in `__hash__` and `__eq__` implementations of the key being looked up. Newer recommendations by the Python core developers advise against using these APIs which suppress exceptions, instead allowing exceptions to bubble upwards. `PyDict::get_item_with_error` already implemented this recommended behavior, so that API has been renamed to `PyDict::get_item`. Before: ```rust,ignore use pyo3::prelude::*; use pyo3::exceptions::PyTypeError; use pyo3::types::{PyDict, IntoPyDict}; # fn main() { # let _ = Python::with_gil(|py| { let dict: &PyDict = [("a", 1)].into_py_dict(py); // `a` is in the dictionary, with value 1 assert!(dict.get_item("a").map_or(Ok(false), |x| x.eq(1))?); // `b` is not in the dictionary assert!(dict.get_item("b").is_none()); // `dict` is not hashable, so this fails with a `TypeError` assert!(dict.get_item_with_error(dict).unwrap_err().is_instance_of::(py)); }); # } ``` After: ```rust use pyo3::prelude::*; use pyo3::exceptions::PyTypeError; use pyo3::types::{PyDict, IntoPyDict}; # fn main() { # let _ = Python::with_gil(|py| -> PyResult<()> { let dict: &PyDict = [("a", 1)].into_py_dict(py); // `a` is in the dictionary, with value 1 assert!(dict.get_item("a")?.map_or(Ok(false), |x| x.eq(1))?); // `b` is not in the dictionary assert!(dict.get_item("b")?.is_none()); // `dict` is not hashable, so this fails with a `TypeError` assert!(dict.get_item(dict).unwrap_err().is_instance_of::(py)); Ok(()) }); # } ``` ### Required arguments are no longer accepted after optional arguments [Trailing `Option` arguments](./function/signature.md#trailing-optional-arguments) have an automatic default of `None`. To avoid unwanted changes when modifying function signatures, in PyO3 0.18 it was deprecated to have a required argument after an `Option` argument without using `#[pyo3(signature = (...))]` to specify the intended defaults. In PyO3 0.20, this becomes a hard error. Before: ```rust,ignore #[pyfunction] fn x_or_y(x: Option, y: u64) -> u64 { x.unwrap_or(y) } ``` After: ```rust # #![allow(dead_code)] # use pyo3::prelude::*; #[pyfunction] #[pyo3(signature = (x, y))] // both x and y have no defaults and are required fn x_or_y(x: Option, y: u64) -> u64 { x.unwrap_or(y) } ``` ### Remove deprecated function forms In PyO3 0.18 the `#[args]` attribute for `#[pymethods]`, and directly specifying the function signature in `#[pyfunction]`, was deprecated. This functionality has been removed in PyO3 0.20. Before: ```rust,ignore #[pyfunction] #[pyo3(a, b = "0", "/")] fn add(a: u64, b: u64) -> u64 { a + b } ``` After: ```rust # #![allow(dead_code)] # use pyo3::prelude::*; #[pyfunction] #[pyo3(signature = (a, b=0, /))] fn add(a: u64, b: u64) -> u64 { a + b } ``` ### `IntoPyPointer` trait removed The trait `IntoPyPointer`, which provided the `into_ptr` method on many types, has been removed. `into_ptr` is now available as an inherent method on all types that previously implemented this trait. ### `AsPyPointer` now `unsafe` trait The trait `AsPyPointer` is now `unsafe trait`, meaning any external implementation of it must be marked as `unsafe impl`, and ensure that they uphold the invariant of returning valid pointers. ## from 0.18.* to 0.19 ### Access to `Python` inside `__traverse__` implementations are now forbidden During `__traverse__` implementations for Python's Garbage Collection it is forbidden to do anything other than visit the members of the `#[pyclass]` being traversed. This means making Python function calls or other API calls are forbidden. Previous versions of PyO3 would allow access to `Python` (e.g. via `Python::with_gil`), which could cause the Python interpreter to crash or otherwise confuse the garbage collection algorithm. Attempts to acquire the GIL will now panic. See [#3165](https://github.com/PyO3/pyo3/issues/3165) for more detail. ```rust,ignore # use pyo3::prelude::*; #[pyclass] struct SomeClass {} impl SomeClass { fn __traverse__(&self, pyo3::class::gc::PyVisit<'_>) -> Result<(), pyo3::class::gc::PyTraverseError>` { Python::with_gil(|| { /*...*/ }) // ERROR: this will panic } } ``` ### Smarter `anyhow::Error` / `eyre::Report` conversion when inner error is "simple" `PyErr` When converting from `anyhow::Error` or `eyre::Report` to `PyErr`, if the inner error is a "simple" `PyErr` (with no source error), then the inner error will be used directly as the `PyErr` instead of wrapping it in a new `PyRuntimeError` with the original information converted into a string. ```rust # #[cfg(feature = "anyhow")] # #[allow(dead_code)] # mod anyhow_only { # use pyo3::prelude::*; # use pyo3::exceptions::PyValueError; #[pyfunction] fn raise_err() -> anyhow::Result<()> { Err(PyValueError::new_err("original error message").into()) } fn main() { Python::with_gil(|py| { let rs_func = wrap_pyfunction!(raise_err, py).unwrap(); pyo3::py_run!( py, rs_func, r" try: rs_func() except Exception as e: print(repr(e)) " ); }) } # } ``` Before, the above code would have printed `RuntimeError('ValueError: original error message')`, which might be confusing. After, the same code will print `ValueError: original error message`, which is more straightforward. However, if the `anyhow::Error` or `eyre::Report` has a source, then the original exception will still be wrapped in a `PyRuntimeError`. ### The deprecated `Python::acquire_gil` was removed and `Python::with_gil` must be used instead While the API provided by [`Python::acquire_gil`](https://docs.rs/pyo3/0.18.3/pyo3/marker/struct.Python.html#method.acquire_gil) seems convenient, it is somewhat brittle as the design of the GIL token [`Python`](https://docs.rs/pyo3/0.18.3/pyo3/marker/struct.Python.html) relies on proper nesting and panics if not used correctly, e.g. ```rust,ignore # #![allow(dead_code, deprecated)] # use pyo3::prelude::*; #[pyclass] struct SomeClass {} struct ObjectAndGuard { object: Py, guard: GILGuard, } impl ObjectAndGuard { fn new() -> Self { let guard = Python::acquire_gil(); let object = Py::new(guard.python(), SomeClass {}).unwrap(); Self { object, guard } } } let first = ObjectAndGuard::new(); let second = ObjectAndGuard::new(); // Panics because the guard within `second` is still alive. drop(first); drop(second); ``` The replacement is [`Python::with_gil`]() which is more cumbersome but enforces the proper nesting by design, e.g. ```rust # #![allow(dead_code)] # use pyo3::prelude::*; #[pyclass] struct SomeClass {} struct Object { object: Py, } impl Object { fn new(py: Python<'_>) -> Self { let object = Py::new(py, SomeClass {}).unwrap(); Self { object } } } // It either forces us to release the GIL before aquiring it again. let first = Python::with_gil(|py| Object::new(py)); let second = Python::with_gil(|py| Object::new(py)); drop(first); drop(second); // Or it ensure releasing the inner lock before the outer one. Python::with_gil(|py| { let first = Object::new(py); let second = Python::with_gil(|py| Object::new(py)); drop(first); drop(second); }); ``` Furthermore, `Python::acquire_gil` provides ownership of a `GILGuard` which can be freely stored and passed around. This is usually not helpful as it may keep the lock held for a long time thereby blocking progress in other parts of the program. Due to the generative lifetime attached to the GIL token supplied by `Python::with_gil`, the problem is avoided as the GIL token can only be passed down the call chain. Often, this issue can also be avoided entirely as any GIL-bound reference `&'py PyAny` implies access to a GIL token `Python<'py>` via the [`PyAny::py`](https://docs.rs/pyo3/latest/pyo3/types/struct.PyAny.html#method.py) method. ## from 0.17.* to 0.18 ### Required arguments after `Option<_>` arguments will no longer be automatically inferred In `#[pyfunction]` and `#[pymethods]`, if a "required" function input such as `i32` came after an `Option<_>` input, then the `Option<_>` would be implicitly treated as required. (All trailing `Option<_>` arguments were treated as optional with a default value of `None`). Starting with PyO3 0.18, this is deprecated and a future PyO3 version will require a [`#[pyo3(signature = (...))]` option](./function/signature.md) to explicitly declare the programmer's intention. Before, x in the below example would be required to be passed from Python code: ```rust,compile_fail # #![allow(dead_code)] # use pyo3::prelude::*; #[pyfunction] fn required_argument_after_option(x: Option, y: i32) {} ``` After, specify the intended Python signature explicitly: ```rust # #![allow(dead_code)] # use pyo3::prelude::*; // If x really was intended to be required #[pyfunction(signature = (x, y))] fn required_argument_after_option_a(x: Option, y: i32) {} // If x was intended to be optional, y needs a default too #[pyfunction(signature = (x=None, y=0))] fn required_argument_after_option_b(x: Option, y: i32) {} ``` ### `__text_signature__` is now automatically generated for `#[pyfunction]` and `#[pymethods]` The [`#[pyo3(text_signature = "...")]` option](./function/signature.md#making-the-function-signature-available-to-python) was previously the only supported way to set the `__text_signature__` attribute on generated Python functions. PyO3 is now able to automatically populate `__text_signature__` for all functions automatically based on their Rust signature (or the [new `#[pyo3(signature = (...))]` option](./function/signature.md)). These automatically-generated `__text_signature__` values will currently only render `...` for all default values. Many `#[pyo3(text_signature = "...")]` options can be removed from functions when updating to PyO3 0.18, however in cases with default values a manual implementation may still be preferred for now. As examples: ```rust # use pyo3::prelude::*; // The `text_signature` option here is no longer necessary, as PyO3 will automatically // generate exactly the same value. #[pyfunction(text_signature = "(a, b, c)")] fn simple_function(a: i32, b: i32, c: i32) {} // The `text_signature` still provides value here as of PyO3 0.18, because the automatically // generated signature would be "(a, b=..., c=...)". #[pyfunction(signature = (a, b = 1, c = 2), text_signature = "(a, b=1, c=2)")] fn function_with_defaults(a: i32, b: i32, c: i32) {} # fn main() { # Python::with_gil(|py| { # let simple = wrap_pyfunction!(simple_function, py).unwrap(); # assert_eq!(simple.getattr("__text_signature__").unwrap().to_string(), "(a, b, c)"); # let defaulted = wrap_pyfunction!(function_with_defaults, py).unwrap(); # assert_eq!(defaulted.getattr("__text_signature__").unwrap().to_string(), "(a, b=1, c=2)"); # }) # } ``` ## from 0.16.* to 0.17 ### Type checks have been changed for `PyMapping` and `PySequence` types Previously the type checks for `PyMapping` and `PySequence` (implemented in `PyTryFrom`) used the Python C-API functions `PyMapping_Check` and `PySequence_Check`. Unfortunately these functions are not sufficient for distinguishing such types, leading to inconsistent behavior (see [pyo3/pyo3#2072](https://github.com/PyO3/pyo3/issues/2072)). PyO3 0.17 changes these downcast checks to explicitly test if the type is a subclass of the corresponding abstract base class `collections.abc.Mapping` or `collections.abc.Sequence`. Note this requires calling into Python, which may incur a performance penalty over the previous method. If this performance penalty is a problem, you may be able to perform your own checks and use `try_from_unchecked` (unsafe). Another side-effect is that a pyclass defined in Rust with PyO3 will need to be _registered_ with the corresponding Python abstract base class for downcasting to succeed. `PySequence::register` and `PyMapping:register` have been added to make it easy to do this from Rust code. These are equivalent to calling `collections.abc.Mapping.register(MappingPyClass)` or `collections.abc.Sequence.register(SequencePyClass)` from Python. For example, for a mapping class defined in Rust: ```rust,compile_fail use pyo3::prelude::*; use std::collections::HashMap; #[pyclass(mapping)] struct Mapping { index: HashMap, } #[pymethods] impl Mapping { #[new] fn new(elements: Option<&PyList>) -> PyResult { // ... // truncated implementation of this mapping pyclass - basically a wrapper around a HashMap } ``` You must register the class with `collections.abc.Mapping` before the downcast will work: ```rust,compile_fail let m = Py::new(py, Mapping { index }).unwrap(); assert!(m.as_ref(py).downcast::().is_err()); PyMapping::register::(py).unwrap(); assert!(m.as_ref(py).downcast::().is_ok()); ``` Note that this requirement may go away in the future when a pyclass is able to inherit from the abstract base class directly (see [pyo3/pyo3#991](https://github.com/PyO3/pyo3/issues/991)). ### The `multiple-pymethods` feature now requires Rust 1.62 Due to limitations in the `inventory` crate which the `multiple-pymethods` feature depends on, this feature now requires Rust 1.62. For more information see [dtolnay/inventory#32](https://github.com/dtolnay/inventory/issues/32). ### Added `impl IntoPy> for &str` This may cause inference errors. Before: ```rust,compile_fail # use pyo3::prelude::*; # # fn main() { Python::with_gil(|py| { // Cannot infer either `Py` or `Py` let _test = "test".into_py(py); }); # } ``` After, some type annotations may be necessary: ```rust # use pyo3::prelude::*; # # fn main() { Python::with_gil(|py| { let _test: Py = "test".into_py(py); }); # } ``` ### The `pyproto` feature is now disabled by default In preparation for removing the deprecated `#[pyproto]` attribute macro in a future PyO3 version, it is now gated behind an opt-in feature flag. This also gives a slight saving to compile times for code which does not use the deprecated macro. ### `PyTypeObject` trait has been deprecated The `PyTypeObject` trait already was near-useless; almost all functionality was already on the `PyTypeInfo` trait, which `PyTypeObject` had a blanket implementation based upon. In PyO3 0.17 the final method, `PyTypeObject::type_object` was moved to `PyTypeInfo::type_object`. To migrate, update trait bounds and imports from `PyTypeObject` to `PyTypeInfo`. Before: ```rust,compile_fail use pyo3::Python; use pyo3::type_object::PyTypeObject; use pyo3::types::PyType; fn get_type_object(py: Python<'_>) -> &PyType { T::type_object(py) } ``` After ```rust use pyo3::{Python, PyTypeInfo}; use pyo3::types::PyType; fn get_type_object(py: Python<'_>) -> &PyType { T::type_object(py) } # Python::with_gil(|py| { get_type_object::(py); }); ``` ### `impl IntoPy for [T; N]` now requires `T: IntoPy` rather than `T: ToPyObject` If this leads to errors, simply implement `IntoPy`. Because pyclasses already implement `IntoPy`, you probably don't need to worry about this. ### Each `#[pymodule]` can now only be initialized once per process To make PyO3 modules sound in the presence of Python sub-interpreters, for now it has been necessary to explicitly disable the ability to initialize a `#[pymodule]` more than once in the same process. Attempting to do this will now raise an `ImportError`. ## from 0.15.* to 0.16 ### Drop support for older technologies PyO3 0.16 has increased minimum Rust version to 1.48 and minimum Python version to 3.7. This enables use of newer language features (enabling some of the other additions in 0.16) and simplifies maintenance of the project. ### `#[pyproto]` has been deprecated In PyO3 0.15, the `#[pymethods]` attribute macro gained support for implementing "magic methods" such as `__str__` (aka "dunder" methods). This implementation was not quite finalized at the time, with a few edge cases to be decided upon. The existing `#[pyproto]` attribute macro was left untouched, because it covered these edge cases. In PyO3 0.16, the `#[pymethods]` implementation has been completed and is now the preferred way to implement magic methods. To allow the PyO3 project to move forward, `#[pyproto]` has been deprecated (with expected removal in PyO3 0.18). Migration from `#[pyproto]` to `#[pymethods]` is straightforward; copying the existing methods directly from the `#[pyproto]` trait implementation is all that is needed in most cases. Before: ```rust,compile_fail use pyo3::prelude::*; use pyo3::class::{PyObjectProtocol, PyIterProtocol}; use pyo3::types::PyString; #[pyclass] struct MyClass {} #[pyproto] impl PyObjectProtocol for MyClass { fn __str__(&self) -> &'static [u8] { b"hello, world" } } #[pyproto] impl PyIterProtocol for MyClass { fn __iter__(slf: PyRef) -> PyResult<&PyAny> { PyString::new(slf.py(), "hello, world").iter() } } ``` After ```rust,compile_fail use pyo3::prelude::*; use pyo3::types::PyString; #[pyclass] struct MyClass {} #[pymethods] impl MyClass { fn __str__(&self) -> &'static [u8] { b"hello, world" } fn __iter__(slf: PyRef) -> PyResult<&PyAny> { PyString::new(slf.py(), "hello, world").iter() } } ``` ### Removed `PartialEq` for object wrappers The Python object wrappers `Py` and `PyAny` had implementations of `PartialEq` so that `object_a == object_b` would compare the Python objects for pointer equality, which corresponds to the `is` operator, not the `==` operator in Python. This has been removed in favor of a new method: use `object_a.is(object_b)`. This also has the advantage of not requiring the same wrapper type for `object_a` and `object_b`; you can now directly compare a `Py` with a `&PyAny` without having to convert. To check for Python object equality (the Python `==` operator), use the new method `eq()`. ### Container magic methods now match Python behavior In PyO3 0.15, `__getitem__`, `__setitem__` and `__delitem__` in `#[pymethods]` would generate only the _mapping_ implementation for a `#[pyclass]`. To match the Python behavior, these methods now generate both the _mapping_ **and** _sequence_ implementations. This means that classes implementing these `#[pymethods]` will now also be treated as sequences, same as a Python `class` would be. Small differences in behavior may result: - PyO3 will allow instances of these classes to be cast to `PySequence` as well as `PyMapping`. - Python will provide a default implementation of `__iter__` (if the class did not have one) which repeatedly calls `__getitem__` with integers (starting at 0) until an `IndexError` is raised. To explain this in detail, consider the following Python class: ```python class ExampleContainer: def __len__(self): return 5 def __getitem__(self, idx: int) -> int: if idx < 0 or idx > 5: raise IndexError() return idx ``` This class implements a Python [sequence](https://docs.python.org/3/glossary.html#term-sequence). The `__len__` and `__getitem__` methods are also used to implement a Python [mapping](https://docs.python.org/3/glossary.html#term-mapping). In the Python C-API, these methods are not shared: the sequence `__len__` and `__getitem__` are defined by the `sq_length` and `sq_item` slots, and the mapping equivalents are `mp_length` and `mp_subscript`. There are similar distinctions for `__setitem__` and `__delitem__`. Because there is no such distinction from Python, implementing these methods will fill the mapping and sequence slots simultaneously. A Python class with `__len__` implemented, for example, will have both the `sq_length` and `mp_length` slots filled. The PyO3 behavior in 0.16 has been changed to be closer to this Python behavior by default. ### `wrap_pymodule!` and `wrap_pyfunction!` now respect privacy correctly Prior to PyO3 0.16 the `wrap_pymodule!` and `wrap_pyfunction!` macros could use modules and functions whose defining `fn` was not reachable according Rust privacy rules. For example, the following code was legal before 0.16, but in 0.16 is rejected because the `wrap_pymodule!` macro cannot access the `private_submodule` function: ```rust,compile_fail mod foo { use pyo3::prelude::*; #[pymodule] fn private_submodule(_py: Python<'_>, m: &PyModule) -> PyResult<()> { Ok(()) } } use pyo3::prelude::*; use foo::*; #[pymodule] fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_wrapped(wrap_pymodule!(private_submodule))?; Ok(()) } ``` To fix it, make the private submodule visible, e.g. with `pub` or `pub(crate)`. ```rust mod foo { use pyo3::prelude::*; #[pymodule] pub(crate) fn private_submodule(_py: Python<'_>, m: &PyModule) -> PyResult<()> { Ok(()) } } use pyo3::prelude::*; use pyo3::wrap_pymodule; use foo::*; #[pymodule] fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_wrapped(wrap_pymodule!(private_submodule))?; Ok(()) } ``` ## from 0.14.* to 0.15 ### Changes in sequence indexing For all types that take sequence indices (`PyList`, `PyTuple` and `PySequence`), the API has been made consistent to only take `usize` indices, for consistency with Rust's indexing conventions. Negative indices, which were only sporadically supported even in APIs that took `isize`, now aren't supported anywhere. Further, the `get_item` methods now always return a `PyResult` instead of panicking on invalid indices. The `Index` trait has been implemented instead, and provides the same panic behavior as on Rust vectors. Note that *slice* indices (accepted by `PySequence::get_slice` and other) still inherit the Python behavior of clamping the indices to the actual length, and not panicking/returning an error on out of range indices. An additional advantage of using Rust's indexing conventions for these types is that these types can now also support Rust's indexing operators as part of a consistent API: ```rust use pyo3::{Python, types::PyList}; Python::with_gil(|py| { let list = PyList::new(py, &[1, 2, 3]); assert_eq!(list[0..2].to_string(), "[1, 2]"); }); ``` ## from 0.13.* to 0.14 ### `auto-initialize` feature is now opt-in For projects embedding Python in Rust, PyO3 no longer automatically initializes a Python interpreter on the first call to `Python::with_gil` (or `Python::acquire_gil`) unless the [`auto-initialize` feature](features.md#auto-initialize) is enabled. ### New `multiple-pymethods` feature `#[pymethods]` have been reworked with a simpler default implementation which removes the dependency on the `inventory` crate. This reduces dependencies and compile times for the majority of users. The limitation of the new default implementation is that it cannot support multiple `#[pymethods]` blocks for the same `#[pyclass]`. If you need this functionality, you must enable the `multiple-pymethods` feature which will switch `#[pymethods]` to the inventory-based implementation. ### Deprecated `#[pyproto]` methods Some protocol (aka `__dunder__`) methods such as `__bytes__` and `__format__` have been possible to implement two ways in PyO3 for some time: via a `#[pyproto]` (e.g. `PyObjectProtocol` for the methods listed here), or by writing them directly in `#[pymethods]`. This is only true for a handful of the `#[pyproto]` methods (for technical reasons to do with the way PyO3 currently interacts with the Python C-API). In the interest of having only one way to do things, the `#[pyproto]` forms of these methods have been deprecated. To migrate just move the affected methods from a `#[pyproto]` to a `#[pymethods]` block. Before: ```rust,compile_fail use pyo3::prelude::*; use pyo3::class::basic::PyObjectProtocol; #[pyclass] struct MyClass {} #[pyproto] impl PyObjectProtocol for MyClass { fn __bytes__(&self) -> &'static [u8] { b"hello, world" } } ``` After: ```rust use pyo3::prelude::*; #[pyclass] struct MyClass {} #[pymethods] impl MyClass { fn __bytes__(&self) -> &'static [u8] { b"hello, world" } } ``` ## from 0.12.* to 0.13 ### Minimum Rust version increased to Rust 1.45 PyO3 `0.13` makes use of new Rust language features stabilized between Rust 1.40 and Rust 1.45. If you are using a Rust compiler older than Rust 1.45, you will need to update your toolchain to be able to continue using PyO3. ### Runtime changes to support the CPython limited API In PyO3 `0.13` support was added for compiling against the CPython limited API. This had a number of implications for _all_ PyO3 users, described here. The largest of these is that all types created from PyO3 are what CPython calls "heap" types. The specific implications of this are: - If you wish to subclass one of these types _from Rust_ you must mark it `#[pyclass(subclass)]`, as you would if you wished to allow subclassing it from Python code. - Type objects are now mutable - Python code can set attributes on them. - `__module__` on types without `#[pyclass(module="mymodule")]` no longer returns `builtins`, it now raises `AttributeError`. ## from 0.11.* to 0.12 ### `PyErr` has been reworked In PyO3 `0.12` the `PyErr` type has been re-implemented to be significantly more compatible with the standard Rust error handling ecosystem. Specifically `PyErr` now implements `Error + Send + Sync`, which are the standard traits used for error types. While this has necessitated the removal of a number of APIs, the resulting `PyErr` type should now be much more easier to work with. The following sections list the changes in detail and how to migrate to the new APIs. #### `PyErr::new` and `PyErr::from_type` now require `Send + Sync` for their argument For most uses no change will be needed. If you are trying to construct `PyErr` from a value that is not `Send + Sync`, you will need to first create the Python object and then use `PyErr::from_instance`. Similarly, any types which implemented `PyErrArguments` will now need to be `Send + Sync`. #### `PyErr`'s contents are now private It is no longer possible to access the fields `.ptype`, `.pvalue` and `.ptraceback` of a `PyErr`. You should instead now use the new methods `PyErr::ptype`, `PyErr::pvalue` and `PyErr::ptraceback`. #### `PyErrValue` and `PyErr::from_value` have been removed As these were part the internals of `PyErr` which have been reworked, these APIs no longer exist. If you used this API, it is recommended to use `PyException::new_err` (see [the section on Exception types](#exception-types-have-been-reworked)). #### `Into>` for `PyErr` has been removed This implementation was redundant. Just construct the `Result::Err` variant directly. Before: ```rust,compile_fail let result: PyResult<()> = PyErr::new::("error message").into(); ``` After (also using the new reworked exception types; see the following section): ```rust # use pyo3::{PyResult, exceptions::PyTypeError}; let result: PyResult<()> = Err(PyTypeError::new_err("error message")); ``` ### Exception types have been reworked Previously exception types were zero-sized marker types purely used to construct `PyErr`. In PyO3 0.12, these types have been replaced with full definitions and are usable in the same way as `PyAny`, `PyDict` etc. This makes it possible to interact with Python exception objects. The new types also have names starting with the "Py" prefix. For example, before: ```rust,compile_fail let err: PyErr = TypeError::py_err("error message"); ``` After: ```rust,compile_fail # use pyo3::{PyErr, PyResult, Python, type_object::PyTypeObject}; # use pyo3::exceptions::{PyBaseException, PyTypeError}; # Python::with_gil(|py| -> PyResult<()> { let err: PyErr = PyTypeError::new_err("error message"); // Uses Display for PyErr, new for PyO3 0.12 assert_eq!(err.to_string(), "TypeError: error message"); // Now possible to interact with exception instances, new for PyO3 0.12 let instance: &PyBaseException = err.instance(py); assert_eq!( instance.getattr("__class__")?, PyTypeError::type_object(py).as_ref() ); # Ok(()) # }).unwrap(); ``` ### `FromPy` has been removed To simplify the PyO3 conversion traits, the `FromPy` trait has been removed. Previously there were two ways to define the to-Python conversion for a type: `FromPy for PyObject` and `IntoPy for T`. Now there is only one way to define the conversion, `IntoPy`, so downstream crates may need to adjust accordingly. Before: ```rust,compile_fail # use pyo3::prelude::*; struct MyPyObjectWrapper(PyObject); impl FromPy for PyObject { fn from_py(other: MyPyObjectWrapper, _py: Python<'_>) -> Self { other.0 } } ``` After ```rust # use pyo3::prelude::*; struct MyPyObjectWrapper(PyObject); impl IntoPy for MyPyObjectWrapper { fn into_py(self, _py: Python<'_>) -> PyObject { self.0 } } ``` Similarly, code which was using the `FromPy` trait can be trivially rewritten to use `IntoPy`. Before: ```rust,compile_fail # use pyo3::prelude::*; # Python::with_gil(|py| { let obj = PyObject::from_py(1.234, py); # }) ``` After: ```rust # use pyo3::prelude::*; # Python::with_gil(|py| { let obj: PyObject = 1.234.into_py(py); # }) ``` ### `PyObject` is now a type alias of `Py` This should change very little from a usage perspective. If you implemented traits for both `PyObject` and `Py`, you may find you can just remove the `PyObject` implementation. ### `AsPyRef` has been removed As `PyObject` has been changed to be just a type alias, the only remaining implementor of `AsPyRef` was `Py`. This removed the need for a trait, so the `AsPyRef::as_ref` method has been moved to `Py::as_ref`. This should require no code changes except removing `use pyo3::AsPyRef` for code which did not use `pyo3::prelude::*`. Before: ```rust,compile_fail use pyo3::{AsPyRef, Py, types::PyList}; # pyo3::Python::with_gil(|py| { let list_py: Py = PyList::empty(py).into(); let list_ref: &PyList = list_py.as_ref(py); # }) ``` After: ```rust use pyo3::{Py, types::PyList}; # pyo3::Python::with_gil(|py| { let list_py: Py = PyList::empty(py).into(); let list_ref: &PyList = list_py.as_ref(py); # }) ``` ## from 0.10.* to 0.11 ### Stable Rust PyO3 now supports the stable Rust toolchain. The minimum required version is 1.39.0. ### `#[pyclass]` structs must now be `Send` or `unsendable` Because `#[pyclass]` structs can be sent between threads by the Python interpreter, they must implement `Send` or declared as `unsendable` (by `#[pyclass(unsendable)]`). Note that `unsendable` is added in PyO3 `0.11.1` and `Send` is always required in PyO3 `0.11.0`. This may "break" some code which previously was accepted, even though it could be unsound. There can be two fixes: 1. If you think that your `#[pyclass]` actually must be `Send`able, then let's implement `Send`. A common, safer way is using thread-safe types. E.g., `Arc` instead of `Rc`, `Mutex` instead of `RefCell`, and `Box` instead of `Box`. Before: ```rust,compile_fail use pyo3::prelude::*; use std::rc::Rc; use std::cell::RefCell; #[pyclass] struct NotThreadSafe { shared_bools: Rc>>, closure: Box, } ``` After: ```rust # #![allow(dead_code)] use pyo3::prelude::*; use std::sync::{Arc, Mutex}; #[pyclass] struct ThreadSafe { shared_bools: Arc>>, closure: Box, } ``` In situations where you cannot change your `#[pyclass]` to automatically implement `Send` (e.g., when it contains a raw pointer), you can use `unsafe impl Send`. In such cases, care should be taken to ensure the struct is actually thread safe. See [the Rustonomicon](https://doc.rust-lang.org/nomicon/send-and-sync.html) for more. 2. If you think that your `#[pyclass]` should not be accessed by another thread, you can use `unsendable` flag. A class marked with `unsendable` panics when accessed by another thread, making it thread-safe to expose an unsendable object to the Python interpreter. Before: ```rust,compile_fail use pyo3::prelude::*; #[pyclass] struct Unsendable { pointers: Vec<*mut std::os::raw::c_char>, } ``` After: ```rust # #![allow(dead_code)] use pyo3::prelude::*; #[pyclass(unsendable)] struct Unsendable { pointers: Vec<*mut std::os::raw::c_char>, } ``` ### All `PyObject` and `Py` methods now take `Python` as an argument Previously, a few methods such as `Object::get_refcnt` did not take `Python` as an argument (to ensure that the Python GIL was held by the current thread). Technically, this was not sound. To migrate, just pass a `py` argument to any calls to these methods. Before: ```rust,compile_fail # pyo3::Python::with_gil(|py| { py.None().get_refcnt(); # }) ``` After: ```rust # pyo3::Python::with_gil(|py| { py.None().get_refcnt(py); # }) ``` ## from 0.9.* to 0.10 ### `ObjectProtocol` is removed All methods are moved to [`PyAny`]. And since now all native types (e.g., `PyList`) implements `Deref`, all you need to do is remove `ObjectProtocol` from your code. Or if you use `ObjectProtocol` by `use pyo3::prelude::*`, you have to do nothing. Before: ```rust,compile_fail use pyo3::ObjectProtocol; # pyo3::Python::with_gil(|py| { let obj = py.eval("lambda: 'Hi :)'", None, None).unwrap(); let hi: &pyo3::types::PyString = obj.call0().unwrap().downcast().unwrap(); assert_eq!(hi.len().unwrap(), 5); # }) ``` After: ```rust # pyo3::Python::with_gil(|py| { let obj = py.eval("lambda: 'Hi :)'", None, None).unwrap(); let hi: &pyo3::types::PyString = obj.call0().unwrap().downcast().unwrap(); assert_eq!(hi.len().unwrap(), 5); # }) ``` ### No `#![feature(specialization)]` in user code While PyO3 itself still requires specialization and nightly Rust, now you don't have to use `#![feature(specialization)]` in your crate. ## from 0.8.* to 0.9 ### `#[new]` interface [`PyRawObject`](https://docs.rs/pyo3/0.8.5/pyo3/type_object/struct.PyRawObject.html) is now removed and our syntax for constructors has changed. Before: ```rust,compile_fail #[pyclass] struct MyClass {} #[pymethods] impl MyClass { #[new] fn new(obj: &PyRawObject) { obj.init(MyClass {}) } } ``` After: ```rust # use pyo3::prelude::*; #[pyclass] struct MyClass {} #[pymethods] impl MyClass { #[new] fn new() -> Self { MyClass {} } } ``` Basically you can return `Self` or `Result` directly. For more, see [the constructor section](class.html#constructor) of this guide. ### PyCell PyO3 0.9 introduces [`PyCell`], which is a [`RefCell`]-like object wrapper for ensuring Rust's rules regarding aliasing of references are upheld. For more detail, see the [Rust Book's section on Rust's rules of references](https://doc.rust-lang.org/book/ch04-02-references-and-borrowing.html#the-rules-of-references) For `#[pymethods]` or `#[pyfunction]`s, your existing code should continue to work without any change. Python exceptions will automatically be raised when your functions are used in a way which breaks Rust's rules of references. Here is an example. ```rust # use pyo3::prelude::*; #[pyclass] struct Names { names: Vec, } #[pymethods] impl Names { #[new] fn new() -> Self { Names { names: vec![] } } fn merge(&mut self, other: &mut Names) { self.names.append(&mut other.names) } } # Python::with_gil(|py| { # let names = PyCell::new(py, Names::new()).unwrap(); # pyo3::py_run!(py, names, r" # try: # names.merge(names) # assert False, 'Unreachable' # except RuntimeError as e: # assert str(e) == 'Already borrowed' # "); # }) ``` `Names` has a `merge` method, which takes `&mut self` and another argument of type `&mut Self`. Given this `#[pyclass]`, calling `names.merge(names)` in Python raises a [`PyBorrowMutError`] exception, since it requires two mutable borrows of `names`. However, for `#[pyproto]` and some functions, you need to manually fix the code. #### Object creation In 0.8 object creation was done with `PyRef::new` and `PyRefMut::new`. In 0.9 these have both been removed. To upgrade code, please use [`PyCell::new`]({{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyCell.html#method.new) instead. If you need [`PyRef`] or [`PyRefMut`], just call `.borrow()` or `.borrow_mut()` on the newly-created `PyCell`. Before: ```rust,compile_fail # use pyo3::prelude::*; # #[pyclass] # struct MyClass {} # Python::with_gil(|py| { let obj_ref = PyRef::new(py, MyClass {}).unwrap(); # }) ``` After: ```rust # use pyo3::prelude::*; # #[pyclass] # struct MyClass {} # Python::with_gil(|py| { let obj = PyCell::new(py, MyClass {}).unwrap(); let obj_ref = obj.borrow(); # }) ``` #### Object extraction For `PyClass` types `T`, `&T` and `&mut T` no longer have [`FromPyObject`] implementations. Instead you should extract `PyRef` or `PyRefMut`, respectively. If `T` implements `Clone`, you can extract `T` itself. In addition, you can also extract `&PyCell`, though you rarely need it. Before: ```compile_fail let obj: &PyAny = create_obj(); let obj_ref: &MyClass = obj.extract().unwrap(); let obj_ref_mut: &mut MyClass = obj.extract().unwrap(); ``` After: ```rust # use pyo3::prelude::*; # use pyo3::types::IntoPyDict; # #[pyclass] #[derive(Clone)] struct MyClass {} # #[pymethods] impl MyClass { #[new]fn new() -> Self { MyClass {} }} # Python::with_gil(|py| { # let typeobj = py.get_type::(); # let d = [("c", typeobj)].into_py_dict(py); # let create_obj = || py.eval("c()", None, Some(d)).unwrap(); let obj: &PyAny = create_obj(); let obj_cell: &PyCell = obj.extract().unwrap(); let obj_cloned: MyClass = obj.extract().unwrap(); // extracted by cloning the object { let obj_ref: PyRef<'_, MyClass> = obj.extract().unwrap(); // we need to drop obj_ref before we can extract a PyRefMut due to Rust's rules of references } let obj_ref_mut: PyRefMut<'_, MyClass> = obj.extract().unwrap(); # }) ``` #### `#[pyproto]` Most of the arguments to methods in `#[pyproto]` impls require a [`FromPyObject`] implementation. So if your protocol methods take `&T` or `&mut T` (where `T: PyClass`), please use [`PyRef`] or [`PyRefMut`] instead. Before: ```rust,compile_fail # use pyo3::prelude::*; # use pyo3::class::PySequenceProtocol; #[pyclass] struct ByteSequence { elements: Vec, } #[pyproto] impl PySequenceProtocol for ByteSequence { fn __concat__(&self, other: &Self) -> PyResult { let mut elements = self.elements.clone(); elements.extend_from_slice(&other.elements); Ok(Self { elements }) } } ``` After: ```rust,compile_fail # use pyo3::prelude::*; # use pyo3::class::PySequenceProtocol; #[pyclass] struct ByteSequence { elements: Vec, } #[pyproto] impl PySequenceProtocol for ByteSequence { fn __concat__(&self, other: PyRef<'p, Self>) -> PyResult { let mut elements = self.elements.clone(); elements.extend_from_slice(&other.elements); Ok(Self { elements }) } } ``` [`FromPyObject`]: {{#PYO3_DOCS_URL}}/pyo3/conversion/trait.FromPyObject.html [`PyAny`]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PyAny.html [`PyCell`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyCell.html [`PyBorrowMutError`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyBorrowMutError.html [`PyRef`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRef.html [`PyRefMut`]: {{#PYO3_DOCS_URL}}/pyo3/pycell/struct.PyRef.html [`RefCell`]: https://doc.rust-lang.org/std/cell/struct.RefCell.html pyo3-0.20.2/guide/src/module.md000064400000000000000000000064051046102023000142770ustar 00000000000000# Python modules You can create a module using `#[pymodule]`: ```rust use pyo3::prelude::*; #[pyfunction] fn double(x: usize) -> usize { x * 2 } /// This module is implemented in Rust. #[pymodule] fn my_extension(py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(double, m)?)?; Ok(()) } ``` The `#[pymodule]` procedural macro takes care of exporting the initialization function of your module to Python. The module's name defaults to the name of the Rust function. You can override the module name by using `#[pyo3(name = "custom_name")]`: ```rust use pyo3::prelude::*; #[pyfunction] fn double(x: usize) -> usize { x * 2 } #[pymodule] #[pyo3(name = "custom_name")] fn my_extension(py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(double, m)?)?; Ok(()) } ``` The name of the module must match the name of the `.so` or `.pyd` file. Otherwise, you will get an import error in Python with the following message: `ImportError: dynamic module does not define module export function (PyInit_name_of_your_module)` To import the module, either: - copy the shared library as described in [Manual builds](building_and_distribution.html#manual-builds), or - use a tool, e.g. `maturin develop` with [maturin](https://github.com/PyO3/maturin) or `python setup.py develop` with [setuptools-rust](https://github.com/PyO3/setuptools-rust). ## Documentation The [Rust doc comments](https://doc.rust-lang.org/stable/book/ch03-04-comments.html) of the module initialization function will be applied automatically as the Python docstring of your module. For example, building off of the above code, this will print `This module is implemented in Rust.`: ```python import my_extension print(my_extension.__doc__) ``` ## Python submodules You can create a module hierarchy within a single extension module by using [`PyModule.add_submodule()`]({{#PYO3_DOCS_URL}}/pyo3/prelude/struct.PyModule.html#method.add_submodule). For example, you could define the modules `parent_module` and `parent_module.child_module`. ```rust use pyo3::prelude::*; #[pymodule] fn parent_module(py: Python<'_>, m: &PyModule) -> PyResult<()> { register_child_module(py, m)?; Ok(()) } fn register_child_module(py: Python<'_>, parent_module: &PyModule) -> PyResult<()> { let child_module = PyModule::new(py, "child_module")?; child_module.add_function(wrap_pyfunction!(func, child_module)?)?; parent_module.add_submodule(child_module)?; Ok(()) } #[pyfunction] fn func() -> String { "func".to_string() } # Python::with_gil(|py| { # use pyo3::wrap_pymodule; # use pyo3::types::IntoPyDict; # let parent_module = wrap_pymodule!(parent_module)(py); # let ctx = [("parent_module", parent_module)].into_py_dict(py); # # py.run("assert parent_module.child_module.func() == 'func'", None, Some(&ctx)).unwrap(); # }) ``` Note that this does not define a package, so this won’t allow Python code to directly import submodules by using `from parent_module import child_module`. For more information, see [#759](https://github.com/PyO3/pyo3/issues/759) and [#1517](https://github.com/PyO3/pyo3/issues/1517#issuecomment-808664021). It is not necessary to add `#[pymodule]` on nested modules, which is only required on the top-level module. pyo3-0.20.2/guide/src/parallelism.md000064400000000000000000000136121046102023000153150ustar 00000000000000# Parallelism CPython has the infamous [Global Interpreter Lock](https://docs.python.org/3/glossary.html#term-global-interpreter-lock), which prevents several threads from executing Python bytecode in parallel. This makes threading in Python a bad fit for [CPU-bound](https://stackoverflow.com/questions/868568/) tasks and often forces developers to accept the overhead of multiprocessing. In PyO3 parallelism can be easily achieved in Rust-only code. Let's take a look at our [word-count](https://github.com/PyO3/pyo3/blob/main/examples/word-count/src/lib.rs) example, where we have a `search` function that utilizes the [rayon](https://github.com/rayon-rs/rayon) crate to count words in parallel. ```rust,no_run # #![allow(dead_code)] use pyo3::prelude::*; // These traits let us use `par_lines` and `map`. use rayon::str::ParallelString; use rayon::iter::ParallelIterator; /// Count the occurrences of needle in line, case insensitive fn count_line(line: &str, needle: &str) -> usize { let mut total = 0; for word in line.split(' ') { if word == needle { total += 1; } } total } #[pyfunction] fn search(contents: &str, needle: &str) -> usize { contents .par_lines() .map(|line| count_line(line, needle)) .sum() } ``` But let's assume you have a long running Rust function which you would like to execute several times in parallel. For the sake of example let's take a sequential version of the word count: ```rust,no_run # #![allow(dead_code)] # fn count_line(line: &str, needle: &str) -> usize { # let mut total = 0; # for word in line.split(' ') { # if word == needle { # total += 1; # } # } # total # } # fn search_sequential(contents: &str, needle: &str) -> usize { contents.lines().map(|line| count_line(line, needle)).sum() } ``` To enable parallel execution of this function, the [`Python::allow_threads`] method can be used to temporarily release the GIL, thus allowing other Python threads to run. We then have a function exposed to the Python runtime which calls `search_sequential` inside a closure passed to [`Python::allow_threads`] to enable true parallelism: ```rust,no_run # #![allow(dead_code)] # use pyo3::prelude::*; # # fn count_line(line: &str, needle: &str) -> usize { # let mut total = 0; # for word in line.split(' ') { # if word == needle { # total += 1; # } # } # total # } # # fn search_sequential(contents: &str, needle: &str) -> usize { # contents.lines().map(|line| count_line(line, needle)).sum() # } #[pyfunction] fn search_sequential_allow_threads(py: Python<'_>, contents: &str, needle: &str) -> usize { py.allow_threads(|| search_sequential(contents, needle)) } ``` Now Python threads can use more than one CPU core, resolving the limitation which usually makes multi-threading in Python only good for IO-bound tasks: ```Python from concurrent.futures import ThreadPoolExecutor from word_count import search_sequential_allow_threads executor = ThreadPoolExecutor(max_workers=2) future_1 = executor.submit( word_count.search_sequential_allow_threads, contents, needle ) future_2 = executor.submit( word_count.search_sequential_allow_threads, contents, needle ) result_1 = future_1.result() result_2 = future_2.result() ``` ## Benchmark Let's benchmark the `word-count` example to verify that we really did unlock parallelism with PyO3. We are using `pytest-benchmark` to benchmark four word count functions: 1. Pure Python version 2. Rust parallel version 3. Rust sequential version 4. Rust sequential version executed twice with two Python threads The benchmark script can be found [here](https://github.com/PyO3/pyo3/blob/main/examples/word-count/tests/test_word_count.py), and we can run `nox` in the `word-count` folder to benchmark these functions. While the results of the benchmark of course depend on your machine, the relative results should be similar to this (mid 2020): ```text -------------------------------------------------------------------------------------------------- benchmark: 4 tests ------------------------------------------------------------------------------------------------- Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- test_word_count_rust_parallel 1.7315 (1.0) 4.6495 (1.0) 1.9972 (1.0) 0.4299 (1.0) 1.8142 (1.0) 0.2049 (1.0) 40;46 500.6943 (1.0) 375 1 test_word_count_rust_sequential 7.3348 (4.24) 10.3556 (2.23) 8.0035 (4.01) 0.7785 (1.81) 7.5597 (4.17) 0.8641 (4.22) 26;5 124.9457 (0.25) 121 1 test_word_count_rust_sequential_twice_with_threads 7.9839 (4.61) 10.3065 (2.22) 8.4511 (4.23) 0.4709 (1.10) 8.2457 (4.55) 0.3927 (1.92) 17;17 118.3274 (0.24) 114 1 test_word_count_python_sequential 27.3985 (15.82) 45.4527 (9.78) 28.9604 (14.50) 4.1449 (9.64) 27.5781 (15.20) 0.4638 (2.26) 3;5 34.5299 (0.07) 35 1 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ``` You can see that the Python threaded version is not much slower than the Rust sequential version, which means compared to an execution on a single CPU core the speed has doubled. [`Python::allow_threads`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Python.html#method.allow_threads pyo3-0.20.2/guide/src/performance.md000064400000000000000000000062371046102023000153160ustar 00000000000000# Performance To achieve the best possible performance, it is useful to be aware of several tricks and sharp edges concerning PyO3's API. ## `extract` versus `downcast` Pythonic API implemented using PyO3 are often polymorphic, i.e. they will accept `&PyAny` and try to turn this into multiple more concrete types to which the requested operation is applied. This often leads to chains of calls to `extract`, e.g. ```rust # #![allow(dead_code)] # use pyo3::prelude::*; # use pyo3::{exceptions::PyTypeError, types::PyList}; fn frobnicate_list(list: &PyList) -> PyResult<&PyAny> { todo!() } fn frobnicate_vec(vec: Vec<&PyAny>) -> PyResult<&PyAny> { todo!() } #[pyfunction] fn frobnicate(value: &PyAny) -> PyResult<&PyAny> { if let Ok(list) = value.extract::<&PyList>() { frobnicate_list(list) } else if let Ok(vec) = value.extract::>() { frobnicate_vec(vec) } else { Err(PyTypeError::new_err("Cannot frobnicate that type.")) } } ``` This suboptimal as the `FromPyObject` trait requires `extract` to have a `Result` return type. For native types like `PyList`, it faster to use `downcast` (which `extract` calls internally) when the error value is ignored. This avoids the costly conversion of a `PyDowncastError` to a `PyErr` required to fulfil the `FromPyObject` contract, i.e. ```rust # #![allow(dead_code)] # use pyo3::prelude::*; # use pyo3::{exceptions::PyTypeError, types::PyList}; # fn frobnicate_list(list: &PyList) -> PyResult<&PyAny> { todo!() } # fn frobnicate_vec(vec: Vec<&PyAny>) -> PyResult<&PyAny> { todo!() } # #[pyfunction] fn frobnicate(value: &PyAny) -> PyResult<&PyAny> { // Use `downcast` instead of `extract` as turning `PyDowncastError` into `PyErr` is quite costly. if let Ok(list) = value.downcast::() { frobnicate_list(list) } else if let Ok(vec) = value.extract::>() { frobnicate_vec(vec) } else { Err(PyTypeError::new_err("Cannot frobnicate that type.")) } } ``` ## Access to GIL-bound reference implies access to GIL token Calling `Python::with_gil` is effectively a no-op when the GIL is already held, but checking that this is the case still has a cost. If an existing GIL token can not be accessed, for example when implementing a pre-existing trait, but a GIL-bound reference is available, this cost can be avoided by exploiting that access to GIL-bound reference gives zero-cost access to a GIL token via `PyAny::py`. For example, instead of writing ```rust # #![allow(dead_code)] # use pyo3::prelude::*; # use pyo3::types::PyList; struct Foo(Py); struct FooRef<'a>(&'a PyList); impl PartialEq for FooRef<'_> { fn eq(&self, other: &Foo) -> bool { Python::with_gil(|py| self.0.len() == other.0.as_ref(py).len()) } } ``` use more efficient ```rust # #![allow(dead_code)] # use pyo3::prelude::*; # use pyo3::types::PyList; # struct Foo(Py); # struct FooRef<'a>(&'a PyList); # impl PartialEq for FooRef<'_> { fn eq(&self, other: &Foo) -> bool { // Access to `&'a PyAny` implies access to `Python<'a>`. let py = self.0.py(); self.0.len() == other.0.as_ref(py).len() } } ``` pyo3-0.20.2/guide/src/python_from_rust.md000064400000000000000000000340541046102023000164340ustar 00000000000000# Calling Python in Rust code This chapter of the guide documents some ways to interact with Python code from Rust: - How to call Python functions - How to execute existing Python code ## Calling Python functions Any Python-native object reference (such as `&PyAny`, `&PyList`, or `&PyCell`) can be used to call Python functions. PyO3 offers two APIs to make function calls: * [`call`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyAny.html#method.call) - call any callable Python object. * [`call_method`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyAny.html#method.call_method) - call a method on the Python object. Both of these APIs take `args` and `kwargs` arguments (for positional and keyword arguments respectively). There are variants for less complex calls: * [`call1`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyAny.html#method.call1) and [`call_method1`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyAny.html#method.call_method1) to call only with positional `args`. * [`call0`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyAny.html#method.call0) and [`call_method0`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyAny.html#method.call_method0) to call with no arguments. For convenience the [`Py`](types.html#pyt-and-pyobject) smart pointer also exposes these same six API methods, but needs a `Python` token as an additional first argument to prove the GIL is held. The example below calls a Python function behind a `PyObject` (aka `Py`) reference: ```rust use pyo3::prelude::*; use pyo3::types::PyTuple; fn main() -> PyResult<()> { let arg1 = "arg1"; let arg2 = "arg2"; let arg3 = "arg3"; Python::with_gil(|py| { let fun: Py = PyModule::from_code( py, "def example(*args, **kwargs): if args != (): print('called with args', args) if kwargs != {}: print('called with kwargs', kwargs) if args == () and kwargs == {}: print('called with no arguments')", "", "", )? .getattr("example")? .into(); // call object without any arguments fun.call0(py)?; // call object with PyTuple let args = PyTuple::new(py, &[arg1, arg2, arg3]); fun.call1(py, args)?; // pass arguments as rust tuple let args = (arg1, arg2, arg3); fun.call1(py, args)?; Ok(()) }) } ``` ### Creating keyword arguments For the `call` and `call_method` APIs, `kwargs` can be `None` or `Some(&PyDict)`. You can use the [`IntoPyDict`]({{#PYO3_DOCS_URL}}/pyo3/types/trait.IntoPyDict.html) trait to convert other dict-like containers, e.g. `HashMap` or `BTreeMap`, as well as tuples with up to 10 elements and `Vec`s where each element is a two-element tuple. ```rust use pyo3::prelude::*; use pyo3::types::IntoPyDict; use std::collections::HashMap; fn main() -> PyResult<()> { let key1 = "key1"; let val1 = 1; let key2 = "key2"; let val2 = 2; Python::with_gil(|py| { let fun: Py = PyModule::from_code( py, "def example(*args, **kwargs): if args != (): print('called with args', args) if kwargs != {}: print('called with kwargs', kwargs) if args == () and kwargs == {}: print('called with no arguments')", "", "", )? .getattr("example")? .into(); // call object with PyDict let kwargs = [(key1, val1)].into_py_dict(py); fun.call(py, (), Some(kwargs))?; // pass arguments as Vec let kwargs = vec![(key1, val1), (key2, val2)]; fun.call(py, (), Some(kwargs.into_py_dict(py)))?; // pass arguments as HashMap let mut kwargs = HashMap::<&str, i32>::new(); kwargs.insert(key1, 1); fun.call(py, (), Some(kwargs.into_py_dict(py)))?; Ok(()) }) } ``` ## Executing existing Python code If you already have some existing Python code that you need to execute from Rust, the following FAQs can help you select the right PyO3 functionality for your situation: ### Want to access Python APIs? Then use `PyModule::import`. [`Pymodule::import`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyModule.html#method.import) can be used to get handle to a Python module from Rust. You can use this to import and use any Python module available in your environment. ```rust use pyo3::prelude::*; fn main() -> PyResult<()> { Python::with_gil(|py| { let builtins = PyModule::import(py, "builtins")?; let total: i32 = builtins .getattr("sum")? .call1((vec![1, 2, 3],))? .extract()?; assert_eq!(total, 6); Ok(()) }) } ``` ### Want to run just an expression? Then use `eval`. [`Python::eval`]({{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#method.eval) is a method to execute a [Python expression](https://docs.python.org/3.7/reference/expressions.html) and return the evaluated value as a `&PyAny` object. ```rust use pyo3::prelude::*; # fn main() -> Result<(), ()> { Python::with_gil(|py| { let result = py .eval("[i * 10 for i in range(5)]", None, None) .map_err(|e| { e.print_and_set_sys_last_vars(py); })?; let res: Vec = result.extract().unwrap(); assert_eq!(res, vec![0, 10, 20, 30, 40]); Ok(()) }) # } ``` ### Want to run statements? Then use `run`. [`Python::run`] is a method to execute one or more [Python statements](https://docs.python.org/3.7/reference/simple_stmts.html). This method returns nothing (like any Python statement), but you can get access to manipulated objects via the `locals` dict. You can also use the [`py_run!`] macro, which is a shorthand for [`Python::run`]. Since [`py_run!`] panics on exceptions, we recommend you use this macro only for quickly testing your Python extensions. ```rust use pyo3::prelude::*; use pyo3::{PyCell, py_run}; # fn main() { #[pyclass] struct UserData { id: u32, name: String, } #[pymethods] impl UserData { fn as_tuple(&self) -> (u32, String) { (self.id, self.name.clone()) } fn __repr__(&self) -> PyResult { Ok(format!("User {}(id: {})", self.name, self.id)) } } Python::with_gil(|py| { let userdata = UserData { id: 34, name: "Yu".to_string(), }; let userdata = PyCell::new(py, userdata).unwrap(); let userdata_as_tuple = (34, "Yu"); py_run!(py, userdata userdata_as_tuple, r#" assert repr(userdata) == "User Yu(id: 34)" assert userdata.as_tuple() == userdata_as_tuple "#); }) # } ``` ## You have a Python file or code snippet? Then use `PyModule::from_code`. [`PyModule::from_code`]({{#PYO3_DOCS_URL}}/pyo3/types/struct.PyModule.html#method.from_code) can be used to generate a Python module which can then be used just as if it was imported with `PyModule::import`. **Warning**: This will compile and execute code. **Never** pass untrusted code to this function! ```rust use pyo3::{ prelude::*, types::{IntoPyDict, PyModule}, }; # fn main() -> PyResult<()> { Python::with_gil(|py| { let activators = PyModule::from_code( py, r#" def relu(x): """see https://en.wikipedia.org/wiki/Rectifier_(neural_networks)""" return max(0.0, x) def leaky_relu(x, slope=0.01): return x if x >= 0 else x * slope "#, "activators.py", "activators", )?; let relu_result: f64 = activators.getattr("relu")?.call1((-1.0,))?.extract()?; assert_eq!(relu_result, 0.0); let kwargs = [("slope", 0.2)].into_py_dict(py); let lrelu_result: f64 = activators .getattr("leaky_relu")? .call((-1.0,), Some(kwargs))? .extract()?; assert_eq!(lrelu_result, -0.2); # Ok(()) }) # } ``` ### Want to embed Python in Rust with additional modules? Python maintains the `sys.modules` dict as a cache of all imported modules. An import in Python will first attempt to lookup the module from this dict, and if not present will use various strategies to attempt to locate and load the module. The [`append_to_inittab`]({{#PYO3_DOCS_URL}}/pyo3/macro.append_to_inittab.html) macro can be used to add additional `#[pymodule]` modules to an embedded Python interpreter. The macro **must** be invoked _before_ initializing Python. As an example, the below adds the module `foo` to the embedded interpreter: ```rust use pyo3::prelude::*; #[pyfunction] fn add_one(x: i64) -> i64 { x + 1 } #[pymodule] fn foo(_py: Python<'_>, foo_module: &PyModule) -> PyResult<()> { foo_module.add_function(wrap_pyfunction!(add_one, foo_module)?)?; Ok(()) } fn main() -> PyResult<()> { pyo3::append_to_inittab!(foo); Python::with_gil(|py| Python::run(py, "import foo; foo.add_one(6)", None, None)) } ``` If `append_to_inittab` cannot be used due to constraints in the program, an alternative is to create a module using [`PyModule::new`] and insert it manually into `sys.modules`: ```rust use pyo3::prelude::*; use pyo3::types::PyDict; #[pyfunction] pub fn add_one(x: i64) -> i64 { x + 1 } fn main() -> PyResult<()> { Python::with_gil(|py| { // Create new module let foo_module = PyModule::new(py, "foo")?; foo_module.add_function(wrap_pyfunction!(add_one, foo_module)?)?; // Import and get sys.modules let sys = PyModule::import(py, "sys")?; let py_modules: &PyDict = sys.getattr("modules")?.downcast()?; // Insert foo into sys.modules py_modules.set_item("foo", foo_module)?; // Now we can import + run our python code Python::run(py, "import foo; foo.add_one(6)", None, None) }) } ``` ### Include multiple Python files You can include a file at compile time by using [`std::include_str`](https://doc.rust-lang.org/std/macro.include_str.html) macro. Or you can load a file at runtime by using [`std::fs::read_to_string`](https://doc.rust-lang.org/std/fs/fn.read_to_string.html) function. Many Python files can be included and loaded as modules. If one file depends on another you must preserve correct order while declaring `PyModule`. Example directory structure: ```text . ├── Cargo.lock ├── Cargo.toml ├── python_app │ ├── app.py │ └── utils │ └── foo.py └── src └── main.rs ``` `python_app/app.py`: ```python from utils.foo import bar def run(): return bar() ``` `python_app/utils/foo.py`: ```python def bar(): return "baz" ``` The example below shows: * how to include content of `app.py` and `utils/foo.py` into your rust binary * how to call function `run()` (declared in `app.py`) that needs function imported from `utils/foo.py` `src/main.rs`: ```rust,ignore use pyo3::prelude::*; fn main() -> PyResult<()> { let py_foo = include_str!(concat!( env!("CARGO_MANIFEST_DIR"), "/python_app/utils/foo.py" )); let py_app = include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/python_app/app.py")); let from_python = Python::with_gil(|py| -> PyResult> { PyModule::from_code(py, py_foo, "utils.foo", "utils.foo")?; let app: Py = PyModule::from_code(py, py_app, "", "")? .getattr("run")? .into(); app.call0(py) }); println!("py: {}", from_python?); Ok(()) } ``` The example below shows: * how to load content of `app.py` at runtime so that it sees its dependencies automatically * how to call function `run()` (declared in `app.py`) that needs function imported from `utils/foo.py` It is recommended to use absolute paths because then your binary can be run from anywhere as long as your `app.py` is in the expected directory (in this example that directory is `/usr/share/python_app`). `src/main.rs`: ```rust,no_run use pyo3::prelude::*; use pyo3::types::PyList; use std::fs; use std::path::Path; fn main() -> PyResult<()> { let path = Path::new("/usr/share/python_app"); let py_app = fs::read_to_string(path.join("app.py"))?; let from_python = Python::with_gil(|py| -> PyResult> { let syspath: &PyList = py.import("sys")?.getattr("path")?.downcast()?; syspath.insert(0, &path)?; let app: Py = PyModule::from_code(py, &py_app, "", "")? .getattr("run")? .into(); app.call0(py) }); println!("py: {}", from_python?); Ok(()) } ``` [`Python::run`]: {{#PYO3_DOCS_URL}}/pyo3/struct.Python.html#method.run [`py_run!`]: {{#PYO3_DOCS_URL}}/pyo3/macro.py_run.html ## Need to use a context manager from Rust? Use context managers by directly invoking `__enter__` and `__exit__`. ```rust use pyo3::prelude::*; use pyo3::types::PyModule; fn main() { Python::with_gil(|py| { let custom_manager = PyModule::from_code( py, r#" class House(object): def __init__(self, address): self.address = address def __enter__(self): print(f"Welcome to {self.address}!") def __exit__(self, type, value, traceback): if type: print(f"Sorry you had {type} trouble at {self.address}") else: print(f"Thank you for visiting {self.address}, come again soon!") "#, "house.py", "house", ) .unwrap(); let house_class = custom_manager.getattr("House").unwrap(); let house = house_class.call1(("123 Main Street",)).unwrap(); house.call_method0("__enter__").unwrap(); let result = py.eval("undefined_variable + 1", None, None); // If the eval threw an exception we'll pass it through to the context manager. // Otherwise, __exit__ is called with empty arguments (Python "None"). match result { Ok(_) => { let none = py.None(); house .call_method1("__exit__", (&none, &none, &none)) .unwrap(); } Err(e) => { house .call_method1("__exit__", (e.get_type(py), e.value(py), e.traceback(py))) .unwrap(); } } }) } ``` [`PyModule::new`]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PyModule.html#method.new pyo3-0.20.2/guide/src/python_typing_hints.md000064400000000000000000000212361046102023000171310ustar 00000000000000# Typing and IDE hints for your Python package PyO3 provides an easy to use interface to code native Python libraries in Rust. The accompanying Maturin allows you to build and publish them as a package. Yet, for a better user experience, Python libraries should provide typing hints and documentation for all public entities, so that IDEs can show them during development and type analyzing tools such as `mypy` can use them to properly verify the code. Currently the best solution for the problem is to manually maintain `*.pyi` files and ship them along with the package. There is a sketch of a roadmap towards completing [the `experimental-inspect` feature](./features.md#experimental-inspect) which may eventually lead to automatic type annotations generated by PyO3. This needs more testing and implementation, please see [issue #2454](https://github.com/PyO3/pyo3/issues/2454). ## Introduction to `pyi` files `pyi` files (an abbreviation for `Python Interface`) are called "stub files" in most of the documentation related to them. A very good definition of what it is can be found in [old MyPy documentation](https://github.com/python/mypy/wiki/Creating-Stubs-For-Python-Modules): > A stubs file only contains a description of the public interface of the module without any implementations. There is also [extensive documentation on type stubs on the official Python typing documentation](https://typing.readthedocs.io/en/latest/source/stubs.html). Most Python developers probably already encountered them when trying to use their IDE's "Go to Definition" function on any builtin type. For example, the definitions of a few standard exceptions look like this: ```python class BaseException(object): args: Tuple[Any, ...] __cause__: BaseException | None __context__: BaseException | None __suppress_context__: bool __traceback__: TracebackType | None def __init__(self, *args: object) -> None: ... def __str__(self) -> str: ... def __repr__(self) -> str: ... def with_traceback(self: _TBE, tb: TracebackType | None) -> _TBE: ... class SystemExit(BaseException): code: int class Exception(BaseException): ... class StopIteration(Exception): value: Any ``` As we can see, those are not full definitions containing implementation, but just a description of the interface. It is usually all that the user of the library needs. ### What do the PEPs say? At the time of writing this documentation, the `pyi` files are referenced in three PEPs. [PEP8 - Style Guide for Python Code - #Function Annotations](https://www.python.org/dev/peps/pep-0008/#function-annotations) (last point) recommends all third party library creators to provide stub files as the source of knowledge about the package for type checker tools. > (...) it is expected that users of third party library packages may want to run type checkers over those packages. For this purpose [PEP 484](https://www.python.org/dev/peps/pep-0484) recommends the use of stub files: .pyi files that are read by the type checker in preference of the corresponding .py files. (...) [PEP484 - Type Hints - #Stub Files](https://www.python.org/dev/peps/pep-0484/#stub-files) defines stub files as follows. > Stub files are files containing type hints that are only for use by the type checker, not at runtime. It contains a specification for them (highly recommended reading, since it contains at least one thing that is not used in normal Python code) and also some general information about where to store the stub files. [PEP561 - Distributing and Packaging Type Information](https://www.python.org/dev/peps/pep-0561/) describes in detail how to build packages that will enable type checking. In particular it contains information about how the stub files must be distributed in order for type checkers to use them. ## How to do it? [PEP561](https://www.python.org/dev/peps/pep-0561/) recognizes three ways of distributing type information: * `inline` - the typing is placed directly in source (`py`) files; * `separate package with stub files` - the typing is placed in `pyi` files distributed in their own, separate package; * `in-package stub files` - the typing is placed in `pyi` files distributed in the same package as source files. The first way is tricky with PyO3 since we do not have `py` files. When it has been investigated and necessary changes are implemented, this document will be updated. The second way is easy to do, and the whole work can be fully separated from the main library code. The example repo for the package with stub files can be found in [PEP561 references section](https://www.python.org/dev/peps/pep-0561/#references): [Stub package repository](https://github.com/ethanhs/stub-package) The third way is described below. ### Including `pyi` files in your PyO3/Maturin build package When source files are in the same package as stub files, they should be placed next to each other. We need a way to do that with Maturin. Also, in order to mark our package as typing-enabled we need to add an empty file named `py.typed` to the package. #### If you do not have other Python files If you do not need to add any other Python files apart from `pyi` to the package, Maturin provides a way to do most of the work for you. As documented in the [Maturin Guide](https://github.com/PyO3/maturin/#mixed-rustpython-projects), the only thing you need to do is to create a stub file for your module named `.pyi` in your project root and Maturin will do the rest. ```text my-rust-project/ ├── Cargo.toml ├── my_project.pyi # <<< add type stubs for Rust functions in the my_project module here ├── pyproject.toml └── src └── lib.rs ``` For an example `pyi` file see the [`my_project.pyi` content](#my_projectpyi-content) section. #### If you need other Python files If you need to add other Python files apart from `pyi` to the package, you can do it also, but that requires some more work. Maturin provides an easy way to add files to a package ([documentation](https://github.com/PyO3/maturin/blob/0dee40510083c03607834c821eea76964140a126/Readme.md#mixed-rustpython-projects)). You just need to create a folder with the name of your module next to the `Cargo.toml` file (for customization see documentation linked above). The folder structure would be: ```text my-project ├── Cargo.toml ├── my_project │ ├── __init__.py │ ├── my_project.pyi │ ├── other_python_file.py │ └── py.typed ├── pyproject.toml ├── Readme.md └── src └── lib.rs ``` Let's go a little bit more into detail regarding the files inside the package folder. ##### `__init__.py` content As we now specify our own package content, we have to provide the `__init__.py` file, so the folder is treated as a package and we can import things from it. We can always use the same content that Maturin creates for us if we do not specify a Python source folder. For PyO3 bindings it would be: ```python from .my_project import * ``` That way everything that is exposed by our native module can be imported directly from the package. ##### `py.typed` requirement As stated in [PEP561](https://www.python.org/dev/peps/pep-0561/): > Package maintainers who wish to support type checking of their code MUST add a marker file named py.typed to their package supporting typing. This marker applies recursively: if a top-level package includes it, all its sub-packages MUST support type checking as well. If we do not include that file, some IDEs might still use our `pyi` files to show hints, but the type checkers might not. MyPy will raise an error in this situation: ```text error: Skipping analyzing "my_project": found module but no type hints or library stubs ``` The file is just a marker file, so it should be empty. ##### `my_project.pyi` content Our module stub file. This document does not aim at describing how to write them, since you can find a lot of documentation on it, starting from the already quoted [PEP484](https://www.python.org/dev/peps/pep-0484/#stub-files). The example can look like this: ```python class Car: """ A class representing a car. :param body_type: the name of body type, e.g. hatchback, sedan :param horsepower: power of the engine in horsepower """ def __init__(self, body_type: str, horsepower: int) -> None: ... @classmethod def from_unique_name(cls, name: str) -> 'Car': """ Creates a Car based on unique name :param name: model name of a car to be created :return: a Car instance with default data """ def best_color(self) -> str: """ Gets the best color for the car. :return: the name of the color our great algorithm thinks is the best for this car """ ``` pyo3-0.20.2/guide/src/trait_bounds.md000064400000000000000000000412631046102023000155100ustar 00000000000000# Using in Python a Rust function with trait bounds PyO3 allows for easy conversion from Rust to Python for certain functions and classes (see the [conversion table](conversions/tables.html). However, it is not always straightforward to convert Rust code that requires a given trait implementation as an argument. This tutorial explains how to convert a Rust function that takes a trait as argument for use in Python with classes implementing the same methods as the trait. Why is this useful? ### Pros - Make your Rust code available to Python users - Code complex algorithms in Rust with the help of the borrow checker ### Cons - Not as fast as native Rust (type conversion has to be performed and one part of the code runs in Python) - You need to adapt your code to expose it ## Example Let's work with the following basic example of an implementation of a optimization solver operating on a given model. Let's say we have a function `solve` that operates on a model and mutates its state. The argument of the function can be any model that implements the `Model` trait : ```rust # #![allow(dead_code)] pub trait Model { fn set_variables(&mut self, inputs: &Vec); fn compute(&mut self); fn get_results(&self) -> Vec; } pub fn solve(model: &mut T) { println!("Magic solver that mutates the model into a resolved state"); } ``` Let's assume we have the following constraints: - We cannot change that code as it runs on many Rust models. - We also have many Python models that cannot be solved as this solver is not available in that language. Rewriting it in Python would be cumbersome and error-prone, as everything is already available in Rust. How could we expose this solver to Python thanks to PyO3 ? ## Implementation of the trait bounds for the Python class If a Python class implements the same three methods as the `Model` trait, it seems logical it could be adapted to use the solver. However, it is not possible to pass a `PyObject` to it as it does not implement the Rust trait (even if the Python model has the required methods). In order to implement the trait, we must write a wrapper around the calls in Rust to the Python model. The method signatures must be the same as the trait, keeping in mind that the Rust trait cannot be changed for the purpose of making the code available in Python. The Python model we want to expose is the following one, which already contains all the required methods: ```python class Model: def set_variables(self, inputs): self.inputs = inputs def compute(self): self.results = [elt**2 - 3 for elt in self.inputs] def get_results(self): return self.results ``` The following wrapper will call the Python model from Rust, using a struct to hold the model as a `PyAny` object: ```rust use pyo3::prelude::*; # pub trait Model { # fn set_variables(&mut self, inputs: &Vec); # fn compute(&mut self); # fn get_results(&self) -> Vec; # } struct UserModel { model: Py, } impl Model for UserModel { fn set_variables(&mut self, var: &Vec) { println!("Rust calling Python to set the variables"); Python::with_gil(|py| { let values: Vec = var.clone(); let list: PyObject = values.into_py(py); let py_model = self.model.as_ref(py); py_model .call_method("set_variables", (list,), None) .unwrap(); }) } fn get_results(&self) -> Vec { println!("Rust calling Python to get the results"); Python::with_gil(|py| { self.model .as_ref(py) .call_method("get_results", (), None) .unwrap() .extract() .unwrap() }) } fn compute(&mut self) { println!("Rust calling Python to perform the computation"); Python::with_gil(|py| { self.model .as_ref(py) .call_method("compute", (), None) .unwrap(); }) } } ``` Now that this bit is implemented, let's expose the model wrapper to Python. Let's add the PyO3 annotations and add a constructor: ```rust # #![allow(dead_code)] # pub trait Model { # fn set_variables(&mut self, inputs: &Vec); # fn compute(&mut self); # fn get_results(&self) -> Vec; # } # use pyo3::prelude::*; #[pyclass] struct UserModel { model: Py, } #[pymodule] fn trait_exposure(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_class::()?; Ok(()) } #[pymethods] impl UserModel { #[new] pub fn new(model: Py) -> Self { UserModel { model } } } ``` Now we add the PyO3 annotations to the trait implementation: ```rust,ignore #[pymethods] impl Model for UserModel { // the previous trait implementation } ``` However, the previous code will not compile. The compilation error is the following one: `error: #[pymethods] cannot be used on trait impl blocks` That's a bummer! However, we can write a second wrapper around these functions to call them directly. This wrapper will also perform the type conversions between Python and Rust. ```rust # use pyo3::prelude::*; # # pub trait Model { # fn set_variables(&mut self, inputs: &Vec); # fn compute(&mut self); # fn get_results(&self) -> Vec; # } # # #[pyclass] # struct UserModel { # model: Py, # } # # impl Model for UserModel { # fn set_variables(&mut self, var: &Vec) { # println!("Rust calling Python to set the variables"); # Python::with_gil(|py| { # let values: Vec = var.clone(); # let list: PyObject = values.into_py(py); # let py_model = self.model.as_ref(py); # py_model # .call_method("set_variables", (list,), None) # .unwrap(); # }) # } # # fn get_results(&self) -> Vec { # println!("Rust calling Python to get the results"); # Python::with_gil(|py| { # self.model # .as_ref(py) # .call_method("get_results", (), None) # .unwrap() # .extract() # .unwrap() # }) # } # # fn compute(&mut self) { # println!("Rust calling Python to perform the computation"); # Python::with_gil(|py| { # self.model # .as_ref(py) # .call_method("compute", (), None) # .unwrap(); # }) # # } # } #[pymethods] impl UserModel { pub fn set_variables(&mut self, var: Vec) { println!("Set variables from Python calling Rust"); Model::set_variables(self, &var) } pub fn get_results(&mut self) -> Vec { println!("Get results from Python calling Rust"); Model::get_results(self) } pub fn compute(&mut self) { println!("Compute from Python calling Rust"); Model::compute(self) } } ``` This wrapper handles the type conversion between the PyO3 requirements and the trait. In order to meet PyO3 requirements, this wrapper must: - return an object of type `PyResult` - use only values, not references in the method signatures Let's run the file python file: ```python class Model: def set_variables(self, inputs): self.inputs = inputs def compute(self): self.results = [elt**2 - 3 for elt in self.inputs] def get_results(self): return self.results if __name__=="__main__": import trait_exposure myModel = Model() my_rust_model = trait_exposure.UserModel(myModel) my_rust_model.set_variables([2.0]) print("Print value from Python: ", myModel.inputs) my_rust_model.compute() print("Print value from Python through Rust: ", my_rust_model.get_results()) print("Print value directly from Python: ", myModel.get_results()) ``` This outputs: ```block Set variables from Python calling Rust Set variables from Rust calling Python Print value from Python: [2.0] Compute from Python calling Rust Compute from Rust calling Python Get results from Python calling Rust Get results from Rust calling Python Print value from Python through Rust: [1.0] Print value directly from Python: [1.0] ``` We have now successfully exposed a Rust model that implements the `Model` trait to Python! We will now expose the `solve` function, but before, let's talk about types errors. ## Type errors in Python What happens if you have type errors when using Python and how can you improve the error messages? ### Wrong types in Python function arguments Let's assume in the first case that you will use in your Python file `my_rust_model.set_variables(2.0)` instead of `my_rust_model.set_variables([2.0])`. The Rust signature expects a vector, which corresponds to a list in Python. What happens if instead of a vector, we pass a single value ? At the execution of Python, we get : ```block File "main.py", line 15, in my_rust_model.set_variables(2) TypeError ``` It is a type error and Python points to it, so it's easy to identify and solve. ### Wrong types in Python method signatures Let's assume now that the return type of one of the methods of our Model class is wrong, for example the `get_results` method that is expected to return a `Vec` in Rust, a list in Python. ```python class Model: def set_variables(self, inputs): self.inputs = inputs def compute(self): self.results = [elt**2 -3 for elt in self.inputs] def get_results(self): return self.results[0] #return self.results <-- this is the expected output ``` This call results in the following panic: ```block pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: PyErr { type: Py(0x10dcf79f0, PhantomData) } ``` This error code is not helpful for a Python user that does not know anything about Rust, or someone that does not know PyO3 was used to interface the Rust code. However, as we are responsible for making the Rust code available to Python, we can do something about it. The issue is that we called `unwrap` anywhere we could, and therefore any panic from PyO3 will be directly forwarded to the end user. Let's modify the code performing the type conversion to give a helpful error message to the Python user: We used in our `get_results` method the following call that performs the type conversion: ```rust # use pyo3::prelude::*; # # pub trait Model { # fn set_variables(&mut self, inputs: &Vec); # fn compute(&mut self); # fn get_results(&self) -> Vec; # } # # #[pyclass] # struct UserModel { # model: Py, # } impl Model for UserModel { fn get_results(&self) -> Vec { println!("Rust calling Python to get the results"); Python::with_gil(|py| { self.model .as_ref(py) .call_method("get_results", (), None) .unwrap() .extract() .unwrap() }) } # fn set_variables(&mut self, var: &Vec) { # println!("Rust calling Python to set the variables"); # Python::with_gil(|py| { # let values: Vec = var.clone(); # let list: PyObject = values.into_py(py); # let py_model = self.model.as_ref(py); # py_model # .call_method("set_variables", (list,), None) # .unwrap(); # }) # } # # fn compute(&mut self) { # println!("Rust calling Python to perform the computation"); # Python::with_gil(|py| { # self.model # .as_ref(py) # .call_method("compute", (), None) # .unwrap(); # }) # } } ``` Let's break it down in order to perform better error handling: ```rust # use pyo3::prelude::*; # # pub trait Model { # fn set_variables(&mut self, inputs: &Vec); # fn compute(&mut self); # fn get_results(&self) -> Vec; # } # # #[pyclass] # struct UserModel { # model: Py, # } impl Model for UserModel { fn get_results(&self) -> Vec { println!("Get results from Rust calling Python"); Python::with_gil(|py| { let py_result: &PyAny = self .model .as_ref(py) .call_method("get_results", (), None) .unwrap(); if py_result.get_type().name().unwrap() != "list" { panic!( "Expected a list for the get_results() method signature, got {}", py_result.get_type().name().unwrap() ); } py_result.extract() }) .unwrap() } # fn set_variables(&mut self, var: &Vec) { # println!("Rust calling Python to set the variables"); # Python::with_gil(|py| { # let values: Vec = var.clone(); # let list: PyObject = values.into_py(py); # let py_model = self.model.as_ref(py); # py_model # .call_method("set_variables", (list,), None) # .unwrap(); # }) # } # # fn compute(&mut self) { # println!("Rust calling Python to perform the computation"); # Python::with_gil(|py| { # self.model # .as_ref(py) # .call_method("compute", (), None) # .unwrap(); # }) # } } ``` By doing so, you catch the result of the Python computation and check its type in order to be able to deliver a better error message before performing the unwrapping. Of course, it does not cover all the possible wrong outputs: the user could return a list of strings instead of a list of floats. In this case, a runtime panic would still occur due to PyO3, but with an error message much more difficult to decipher for non-rust user. It is up to the developer exposing the rust code to decide how much effort to invest into Python type error handling and improved error messages. ## The final code Now let's expose the `solve()` function to make it available from Python. It is not possible to directly expose the `solve` function to Python, as the type conversion cannot be performed. It requires an object implementing the `Model` trait as input. However, the `UserModel` already implements this trait. Because of this, we can write a function wrapper that takes the `UserModel`--which has already been exposed to Python--as an argument in order to call the core function `solve`. It is also required to make the struct public. ```rust use pyo3::prelude::*; pub trait Model { fn set_variables(&mut self, var: &Vec); fn get_results(&self) -> Vec; fn compute(&mut self); } pub fn solve(model: &mut T) { println!("Magic solver that mutates the model into a resolved state"); } #[pyfunction] #[pyo3(name = "solve")] pub fn solve_wrapper(model: &mut UserModel) { solve(model); } #[pyclass] pub struct UserModel { model: Py, } #[pymodule] fn trait_exposure(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_class::()?; m.add_function(wrap_pyfunction!(solve_wrapper, m)?)?; Ok(()) } #[pymethods] impl UserModel { #[new] pub fn new(model: Py) -> Self { UserModel { model } } pub fn set_variables(&mut self, var: Vec) { println!("Set variables from Python calling Rust"); Model::set_variables(self, &var) } pub fn get_results(&mut self) -> Vec { println!("Get results from Python calling Rust"); Model::get_results(self) } pub fn compute(&mut self) { Model::compute(self) } } impl Model for UserModel { fn set_variables(&mut self, var: &Vec) { println!("Rust calling Python to set the variables"); Python::with_gil(|py| { let values: Vec = var.clone(); let list: PyObject = values.into_py(py); let py_model = self.model.as_ref(py); py_model .call_method("set_variables", (list,), None) .unwrap(); }) } fn get_results(&self) -> Vec { println!("Get results from Rust calling Python"); Python::with_gil(|py| { let py_result: &PyAny = self .model .as_ref(py) .call_method("get_results", (), None) .unwrap(); if py_result.get_type().name().unwrap() != "list" { panic!( "Expected a list for the get_results() method signature, got {}", py_result.get_type().name().unwrap() ); } py_result.extract() }) .unwrap() } fn compute(&mut self) { println!("Rust calling Python to perform the computation"); Python::with_gil(|py| { self.model .as_ref(py) .call_method("compute", (), None) .unwrap(); }) } } ``` pyo3-0.20.2/guide/src/types.md000064400000000000000000000240161046102023000141540ustar 00000000000000# GIL lifetimes, mutability and Python object types On first glance, PyO3 provides a huge number of different types that can be used to wrap or refer to Python objects. This page delves into the details and gives an overview of their intended meaning, with examples when each type is best used. ## The Python GIL, mutability, and Rust types Since Python has no concept of ownership, and works solely with boxed objects, any Python object can be referenced any number of times, and mutation is allowed from any reference. The situation is helped a little by the Global Interpreter Lock (GIL), which ensures that only one thread can use the Python interpreter and its API at the same time, while non-Python operations (system calls and extension code) can unlock the GIL. (See [the section on parallelism](parallelism.md) for how to do that in PyO3.) In PyO3, holding the GIL is modeled by acquiring a token of the type `Python<'py>`, which serves three purposes: * It provides some global API for the Python interpreter, such as [`eval`][eval]. * It can be passed to functions that require a proof of holding the GIL, such as [`Py::clone_ref`][clone_ref]. * Its lifetime can be used to create Rust references that implicitly guarantee holding the GIL, such as [`&'py PyAny`][PyAny]. The latter two points are the reason why some APIs in PyO3 require the `py: Python` argument, while others don't. The PyO3 API for Python objects is written such that instead of requiring a mutable Rust reference for mutating operations such as [`PyList::append`][PyList_append], a shared reference (which, in turn, can only be created through `Python<'_>` with a GIL lifetime) is sufficient. However, Rust structs wrapped as Python objects (called `pyclass` types) usually *do* need `&mut` access. Due to the GIL, PyO3 *can* guarantee thread-safe access to them, but it cannot statically guarantee uniqueness of `&mut` references once an object's ownership has been passed to the Python interpreter, ensuring references is done at runtime using `PyCell`, a scheme very similar to `std::cell::RefCell`. ### Accessing the Python GIL To get hold of a `Python<'py>` token to prove the GIL is held, consult [PyO3's documentation][obtaining-py]. ## Object types ### [`PyAny`][PyAny] **Represents:** a Python object of unspecified type, restricted to a GIL lifetime. Currently, `PyAny` can only ever occur as a reference, `&PyAny`. **Used:** Whenever you want to refer to some Python object and will have the GIL for the whole duration you need to access that object. For example, intermediate values and arguments to `pyfunction`s or `pymethod`s implemented in Rust where any type is allowed. Many general methods for interacting with Python objects are on the `PyAny` struct, such as `getattr`, `setattr`, and `.call`. **Conversions:** For a `&PyAny` object reference `any` where the underlying object is a Python-native type such as a list: ```rust # use pyo3::prelude::*; # use pyo3::types::PyList; # Python::with_gil(|py| -> PyResult<()> { let obj: &PyAny = PyList::empty(py); // To &PyList with PyAny::downcast let _: &PyList = obj.downcast()?; // To Py (aka PyObject) with .into() let _: Py = obj.into(); // To Py with PyAny::extract let _: Py = obj.extract()?; # Ok(()) # }).unwrap(); ``` For a `&PyAny` object reference `any` where the underlying object is a `#[pyclass]`: ```rust # use pyo3::prelude::*; # #[pyclass] #[derive(Clone)] struct MyClass { } # Python::with_gil(|py| -> PyResult<()> { let obj: &PyAny = Py::new(py, MyClass {})?.into_ref(py); // To &PyCell with PyAny::downcast let _: &PyCell = obj.downcast()?; // To Py (aka PyObject) with .into() let _: Py = obj.into(); // To Py with PyAny::extract let _: Py = obj.extract()?; // To MyClass with PyAny::extract, if MyClass: Clone let _: MyClass = obj.extract()?; // To PyRef<'_, MyClass> or PyRefMut<'_, MyClass> with PyAny::extract let _: PyRef<'_, MyClass> = obj.extract()?; let _: PyRefMut<'_, MyClass> = obj.extract()?; # Ok(()) # }).unwrap(); ``` ### `PyTuple`, `PyDict`, and many more **Represents:** a native Python object of known type, restricted to a GIL lifetime just like `PyAny`. **Used:** Whenever you want to operate with native Python types while holding the GIL. Like `PyAny`, this is the most convenient form to use for function arguments and intermediate values. These types all implement `Deref`, so they all expose the same methods which can be found on `PyAny`. To see all Python types exposed by `PyO3` you should consult the [`pyo3::types`][pyo3::types] module. **Conversions:** ```rust # use pyo3::prelude::*; # use pyo3::types::PyList; # Python::with_gil(|py| -> PyResult<()> { let list = PyList::empty(py); // Use methods from PyAny on all Python types with Deref implementation let _ = list.repr()?; // To &PyAny automatically with Deref implementation let _: &PyAny = list; // To &PyAny explicitly with .as_ref() let _: &PyAny = list.as_ref(); // To Py with .into() or Py::from() let _: Py = list.into(); // To PyObject with .into() or .to_object(py) let _: PyObject = list.into(); # Ok(()) # }).unwrap(); ``` ### `Py` and `PyObject` **Represents:** a GIL-independent reference to a Python object. This can be a Python native type (like `PyTuple`), or a `pyclass` type implemented in Rust. The most commonly-used variant, `Py`, is also known as `PyObject`. **Used:** Whenever you want to carry around references to a Python object without caring about a GIL lifetime. For example, storing Python object references in a Rust struct that outlives the Python-Rust FFI boundary, or returning objects from functions implemented in Rust back to Python. Can be cloned using Python reference counts with `.clone()`. **Conversions:** For a `Py`, the conversions are as below: ```rust # use pyo3::prelude::*; # use pyo3::types::PyList; # Python::with_gil(|py| { let list: Py = PyList::empty(py).into(); // To &PyList with Py::as_ref() (borrows from the Py) let _: &PyList = list.as_ref(py); # let list_clone = list.clone(); // Because `.into_ref()` will consume `list`. // To &PyList with Py::into_ref() (moves the pointer into PyO3's object storage) let _: &PyList = list.into_ref(py); # let list = list_clone; // To Py (aka PyObject) with .into() let _: Py = list.into(); # }) ``` For a `#[pyclass] struct MyClass`, the conversions for `Py` are below: ```rust # use pyo3::prelude::*; # Python::with_gil(|py| { # #[pyclass] struct MyClass { } # Python::with_gil(|py| -> PyResult<()> { let my_class: Py = Py::new(py, MyClass { })?; // To &PyCell with Py::as_ref() (borrows from the Py) let _: &PyCell = my_class.as_ref(py); # let my_class_clone = my_class.clone(); // Because `.into_ref()` will consume `my_class`. // To &PyCell with Py::into_ref() (moves the pointer into PyO3's object storage) let _: &PyCell = my_class.into_ref(py); # let my_class = my_class_clone.clone(); // To Py (aka PyObject) with .into_py(py) let _: Py = my_class.into_py(py); # let my_class = my_class_clone; // To PyRef<'_, MyClass> with Py::borrow or Py::try_borrow let _: PyRef<'_, MyClass> = my_class.try_borrow(py)?; // To PyRefMut<'_, MyClass> with Py::borrow_mut or Py::try_borrow_mut let _: PyRefMut<'_, MyClass> = my_class.try_borrow_mut(py)?; # Ok(()) # }).unwrap(); # }); ``` ### `PyCell` **Represents:** a reference to a Rust object (instance of `PyClass`) which is wrapped in a Python object. The cell part is an analog to stdlib's [`RefCell`][RefCell] to allow access to `&mut` references. **Used:** for accessing pure-Rust API of the instance (members and functions taking `&SomeType` or `&mut SomeType`) while maintaining the aliasing rules of Rust references. Like PyO3's Python native types, `PyCell` implements `Deref`, so it also exposes all of the methods on `PyAny`. **Conversions:** `PyCell` can be used to access `&T` and `&mut T` via `PyRef` and `PyRefMut` respectively. ```rust # use pyo3::prelude::*; # #[pyclass] struct MyClass { } # Python::with_gil(|py| -> PyResult<()> { let cell: &PyCell = PyCell::new(py, MyClass {})?; // To PyRef with .borrow() or .try_borrow() let py_ref: PyRef<'_, MyClass> = cell.try_borrow()?; let _: &MyClass = &*py_ref; # drop(py_ref); // To PyRefMut with .borrow_mut() or .try_borrow_mut() let mut py_ref_mut: PyRefMut<'_, MyClass> = cell.try_borrow_mut()?; let _: &mut MyClass = &mut *py_ref_mut; # Ok(()) # }).unwrap(); ``` `PyCell` can also be accessed like a Python-native type. ```rust # use pyo3::prelude::*; # #[pyclass] struct MyClass { } # Python::with_gil(|py| -> PyResult<()> { let cell: &PyCell = PyCell::new(py, MyClass {})?; // Use methods from PyAny on PyCell with Deref implementation let _ = cell.repr()?; // To &PyAny automatically with Deref implementation let _: &PyAny = cell; // To &PyAny explicitly with .as_ref() let _: &PyAny = cell.as_ref(); # Ok(()) # }).unwrap(); ``` ### `PyRef` and `PyRefMut` **Represents:** reference wrapper types employed by `PyCell` to keep track of borrows, analog to `Ref` and `RefMut` used by `RefCell`. **Used:** while borrowing a `PyCell`. They can also be used with `.extract()` on types like `Py` and `PyAny` to get a reference quickly. ## Related traits and types ### `PyClass` This trait marks structs defined in Rust that are also usable as Python classes, usually defined using the `#[pyclass]` macro. ### `PyNativeType` This trait marks structs that mirror native Python types, such as `PyList`. [eval]: {{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#method.eval [clone_ref]: {{#PYO3_DOCS_URL}}/pyo3/struct.Py.html#method.clone_ref [pyo3::types]: {{#PYO3_DOCS_URL}}/pyo3/types/index.html [PyAny]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PyAny.html [PyList_append]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.PyList.html#method.append [RefCell]: https://doc.rust-lang.org/std/cell/struct.RefCell.html [obtaining-py]: {{#PYO3_DOCS_URL}}/pyo3/marker/struct.Python.html#obtaining-a-python-token pyo3-0.20.2/netlify.toml000064400000000000000000000001551046102023000131470ustar 00000000000000[build] publish = "netlify_build/" command = ".netlify/build.sh" [build.environment] PYTHON_VERSION = "3.8" pyo3-0.20.2/newsfragments/.gitignore000064400000000000000000000000001046102023000154400ustar 00000000000000pyo3-0.20.2/pyo3-runtime/LICENSE-APACHE000064400000000000000000000250351046102023000151030ustar 00000000000000 Copyright (c) 2017-present PyO3 Project and Contributors. https://github.com/PyO3 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. pyo3-0.20.2/pyo3-runtime/LICENSE-MIT000064400000000000000000000021231046102023000146040ustar 00000000000000Copyright (c) 2023-present PyO3 Project and Contributors. https://github.com/PyO3 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. pyo3-0.20.2/pyo3-runtime/README.md000064400000000000000000000000151046102023000144250ustar 00000000000000Coming soon! pyo3-0.20.2/pyo3-runtime/pyproject.toml000064400000000000000000000016661046102023000160770ustar 00000000000000[build-system] requires = ["hatchling"] build-backend = "hatchling.build" [project] name = "pyo3-runtime" dynamic = ["version"] description = '' readme = "README.md" requires-python = ">=3.7" license = "MIT OR Apache-2.0" keywords = [] authors = [ { name = "David Hewitt", email = "1939362+davidhewitt@users.noreply.github.com" }, ] classifiers = [ "Development Status :: 4 - Beta", "Programming Language :: Python", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy", ] dependencies = [] [project.urls] Homepage = "https://github.com/PyO3/pyo3" [tool.hatch.version] path = "src/pyo3_runtime/__init__.py" pyo3-0.20.2/pyo3-runtime/src/pyo3_runtime/__init__.py000064400000000000000000000000261046102023000205050ustar 00000000000000__version__ = "0.0.1" pyo3-0.20.2/pyo3-runtime/tests/__init__.py000064400000000000000000000000001046102023000164130ustar 00000000000000pyo3-0.20.2/src/buffer.rs000064400000000000000000001036761046102023000132220ustar 00000000000000#![cfg(any(not(Py_LIMITED_API), Py_3_11))] // Copyright (c) 2017 Daniel Grunwald // // Permission is hereby granted, free of charge, to any person obtaining a copy of this // software and associated documentation files (the "Software"), to deal in the Software // without restriction, including without limitation the rights to use, copy, modify, merge, // publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons // to whom the Software is furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included in all copies or // substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, // INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR // PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE // FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR // OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER // DEALINGS IN THE SOFTWARE. //! `PyBuffer` implementation use crate::{err, exceptions::PyBufferError, ffi, FromPyObject, PyAny, PyResult, Python}; use std::marker::PhantomData; use std::os::raw; use std::pin::Pin; use std::{cell, mem, ptr, slice}; use std::{ffi::CStr, fmt::Debug}; /// Allows access to the underlying buffer used by a python object such as `bytes`, `bytearray` or `array.array`. // use Pin because Python expects that the Py_buffer struct has a stable memory address #[repr(transparent)] pub struct PyBuffer(Pin>, PhantomData); // PyBuffer is thread-safe: the shape of the buffer is immutable while a Py_buffer exists. // Accessing the buffer contents is protected using the GIL. unsafe impl Send for PyBuffer {} unsafe impl Sync for PyBuffer {} impl Debug for PyBuffer { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { f.debug_struct("PyBuffer") .field("buf", &self.0.buf) .field("obj", &self.0.obj) .field("len", &self.0.len) .field("itemsize", &self.0.itemsize) .field("readonly", &self.0.readonly) .field("ndim", &self.0.ndim) .field("format", &self.0.format) .field("shape", &self.0.shape) .field("strides", &self.0.strides) .field("suboffsets", &self.0.suboffsets) .field("internal", &self.0.internal) .finish() } } /// Represents the type of a Python buffer element. #[derive(Copy, Clone, Debug, Eq, PartialEq)] pub enum ElementType { /// A signed integer type. SignedInteger { /// The width of the signed integer in bytes. bytes: usize, }, /// An unsigned integer type. UnsignedInteger { /// The width of the unsigned integer in bytes. bytes: usize, }, /// A boolean type. Bool, /// A float type. Float { /// The width of the float in bytes. bytes: usize, }, /// An unknown type. This may occur when parsing has failed. Unknown, } impl ElementType { /// Determines the `ElementType` from a Python `struct` module format string. /// /// See for more information /// about struct format strings. pub fn from_format(format: &CStr) -> ElementType { match format.to_bytes() { [size] | [b'@', size] => native_element_type_from_type_char(*size), [b'=' | b'<' | b'>' | b'!', size] => standard_element_type_from_type_char(*size), _ => ElementType::Unknown, } } } fn native_element_type_from_type_char(type_char: u8) -> ElementType { use self::ElementType::*; match type_char { b'c' => UnsignedInteger { bytes: mem::size_of::(), }, b'b' => SignedInteger { bytes: mem::size_of::(), }, b'B' => UnsignedInteger { bytes: mem::size_of::(), }, b'?' => Bool, b'h' => SignedInteger { bytes: mem::size_of::(), }, b'H' => UnsignedInteger { bytes: mem::size_of::(), }, b'i' => SignedInteger { bytes: mem::size_of::(), }, b'I' => UnsignedInteger { bytes: mem::size_of::(), }, b'l' => SignedInteger { bytes: mem::size_of::(), }, b'L' => UnsignedInteger { bytes: mem::size_of::(), }, b'q' => SignedInteger { bytes: mem::size_of::(), }, b'Q' => UnsignedInteger { bytes: mem::size_of::(), }, b'n' => SignedInteger { bytes: mem::size_of::(), }, b'N' => UnsignedInteger { bytes: mem::size_of::(), }, b'e' => Float { bytes: 2 }, b'f' => Float { bytes: 4 }, b'd' => Float { bytes: 8 }, _ => Unknown, } } fn standard_element_type_from_type_char(type_char: u8) -> ElementType { use self::ElementType::*; match type_char { b'c' | b'B' => UnsignedInteger { bytes: 1 }, b'b' => SignedInteger { bytes: 1 }, b'?' => Bool, b'h' => SignedInteger { bytes: 2 }, b'H' => UnsignedInteger { bytes: 2 }, b'i' | b'l' => SignedInteger { bytes: 4 }, b'I' | b'L' => UnsignedInteger { bytes: 4 }, b'q' => SignedInteger { bytes: 8 }, b'Q' => UnsignedInteger { bytes: 8 }, b'e' => Float { bytes: 2 }, b'f' => Float { bytes: 4 }, b'd' => Float { bytes: 8 }, _ => Unknown, } } #[cfg(target_endian = "little")] fn is_matching_endian(c: u8) -> bool { c == b'@' || c == b'=' || c == b'>' } #[cfg(target_endian = "big")] fn is_matching_endian(c: u8) -> bool { c == b'@' || c == b'=' || c == b'>' || c == b'!' } /// Trait implemented for possible element types of `PyBuffer`. /// /// # Safety /// /// This trait must only be implemented for types which represent valid elements of Python buffers. pub unsafe trait Element: Copy { /// Gets whether the element specified in the format string is potentially compatible. /// Alignment and size are checked separately from this function. fn is_compatible_format(format: &CStr) -> bool; } impl<'source, T: Element> FromPyObject<'source> for PyBuffer { fn extract(obj: &PyAny) -> PyResult> { Self::get(obj) } } impl PyBuffer { /// Gets the underlying buffer from the specified python object. pub fn get(obj: &PyAny) -> PyResult> { // TODO: use nightly API Box::new_uninit() once stable let mut buf = Box::new(mem::MaybeUninit::uninit()); let buf: Box = { err::error_on_minusone(obj.py(), unsafe { ffi::PyObject_GetBuffer(obj.as_ptr(), buf.as_mut_ptr(), ffi::PyBUF_FULL_RO) })?; // Safety: buf is initialized by PyObject_GetBuffer. // TODO: use nightly API Box::assume_init() once stable unsafe { mem::transmute(buf) } }; // Create PyBuffer immediately so that if validation checks fail, the PyBuffer::drop code // will call PyBuffer_Release (thus avoiding any leaks). let buf = PyBuffer(Pin::from(buf), PhantomData); if buf.0.shape.is_null() { Err(PyBufferError::new_err("shape is null")) } else if buf.0.strides.is_null() { Err(PyBufferError::new_err("strides is null")) } else if mem::size_of::() != buf.item_size() || !T::is_compatible_format(buf.format()) { Err(PyBufferError::new_err(format!( "buffer contents are not compatible with {}", std::any::type_name::() ))) } else if buf.0.buf.align_offset(mem::align_of::()) != 0 { Err(PyBufferError::new_err(format!( "buffer contents are insufficiently aligned for {}", std::any::type_name::() ))) } else { Ok(buf) } } /// Gets the pointer to the start of the buffer memory. /// /// Warning: the buffer memory might be mutated by other Python functions, /// and thus may only be accessed while the GIL is held. #[inline] pub fn buf_ptr(&self) -> *mut raw::c_void { self.0.buf } /// Gets a pointer to the specified item. /// /// If `indices.len() < self.dimensions()`, returns the start address of the sub-array at the specified dimension. pub fn get_ptr(&self, indices: &[usize]) -> *mut raw::c_void { let shape = &self.shape()[..indices.len()]; for i in 0..indices.len() { assert!(indices[i] < shape[i]); } unsafe { ffi::PyBuffer_GetPointer( #[cfg(Py_3_11)] &*self.0, #[cfg(not(Py_3_11))] { &*self.0 as *const ffi::Py_buffer as *mut ffi::Py_buffer }, #[cfg(Py_3_11)] { indices.as_ptr() as *const ffi::Py_ssize_t }, #[cfg(not(Py_3_11))] { indices.as_ptr() as *mut ffi::Py_ssize_t }, ) } } /// Gets whether the underlying buffer is read-only. #[inline] pub fn readonly(&self) -> bool { self.0.readonly != 0 } /// Gets the size of a single element, in bytes. /// Important exception: when requesting an unformatted buffer, item_size still has the value #[inline] pub fn item_size(&self) -> usize { self.0.itemsize as usize } /// Gets the total number of items. #[inline] pub fn item_count(&self) -> usize { (self.0.len as usize) / (self.0.itemsize as usize) } /// `item_size() * item_count()`. /// For contiguous arrays, this is the length of the underlying memory block. /// For non-contiguous arrays, it is the length that the logical structure would have if it were copied to a contiguous representation. #[inline] pub fn len_bytes(&self) -> usize { self.0.len as usize } /// Gets the number of dimensions. /// /// May be 0 to indicate a single scalar value. #[inline] pub fn dimensions(&self) -> usize { self.0.ndim as usize } /// Returns an array of length `dimensions`. `shape()[i]` is the length of the array in dimension number `i`. /// /// May return None for single-dimensional arrays or scalar values (`dimensions() <= 1`); /// You can call `item_count()` to get the length of the single dimension. /// /// Despite Python using an array of signed integers, the values are guaranteed to be non-negative. /// However, dimensions of length 0 are possible and might need special attention. #[inline] pub fn shape(&self) -> &[usize] { unsafe { slice::from_raw_parts(self.0.shape as *const usize, self.0.ndim as usize) } } /// Returns an array that holds, for each dimension, the number of bytes to skip to get to the next element in the dimension. /// /// Stride values can be any integer. For regular arrays, strides are usually positive, /// but a consumer MUST be able to handle the case `strides[n] <= 0`. #[inline] pub fn strides(&self) -> &[isize] { unsafe { slice::from_raw_parts(self.0.strides, self.0.ndim as usize) } } /// An array of length ndim. /// If `suboffsets[n] >= 0`, the values stored along the nth dimension are pointers and the suboffset value dictates how many bytes to add to each pointer after de-referencing. /// A suboffset value that is negative indicates that no de-referencing should occur (striding in a contiguous memory block). /// /// If all suboffsets are negative (i.e. no de-referencing is needed), then this field must be NULL (the default value). #[inline] pub fn suboffsets(&self) -> Option<&[isize]> { unsafe { if self.0.suboffsets.is_null() { None } else { Some(slice::from_raw_parts( self.0.suboffsets, self.0.ndim as usize, )) } } } /// A NUL terminated string in struct module style syntax describing the contents of a single item. #[inline] pub fn format(&self) -> &CStr { if self.0.format.is_null() { CStr::from_bytes_with_nul(b"B\0").unwrap() } else { unsafe { CStr::from_ptr(self.0.format) } } } /// Gets whether the buffer is contiguous in C-style order (last index varies fastest when visiting items in order of memory address). #[inline] pub fn is_c_contiguous(&self) -> bool { unsafe { ffi::PyBuffer_IsContiguous( &*self.0 as *const ffi::Py_buffer, b'C' as std::os::raw::c_char, ) != 0 } } /// Gets whether the buffer is contiguous in Fortran-style order (first index varies fastest when visiting items in order of memory address). #[inline] pub fn is_fortran_contiguous(&self) -> bool { unsafe { ffi::PyBuffer_IsContiguous( &*self.0 as *const ffi::Py_buffer, b'F' as std::os::raw::c_char, ) != 0 } } /// Gets the buffer memory as a slice. /// /// This function succeeds if: /// * the buffer format is compatible with `T` /// * alignment and size of buffer elements is matching the expectations for type `T` /// * the buffer is C-style contiguous /// /// The returned slice uses type `Cell` because it's theoretically possible for any call into the Python runtime /// to modify the values in the slice. pub fn as_slice<'a>(&'a self, _py: Python<'a>) -> Option<&'a [ReadOnlyCell]> { if self.is_c_contiguous() { unsafe { Some(slice::from_raw_parts( self.0.buf as *mut ReadOnlyCell, self.item_count(), )) } } else { None } } /// Gets the buffer memory as a slice. /// /// This function succeeds if: /// * the buffer is not read-only /// * the buffer format is compatible with `T` /// * alignment and size of buffer elements is matching the expectations for type `T` /// * the buffer is C-style contiguous /// /// The returned slice uses type `Cell` because it's theoretically possible for any call into the Python runtime /// to modify the values in the slice. pub fn as_mut_slice<'a>(&'a self, _py: Python<'a>) -> Option<&'a [cell::Cell]> { if !self.readonly() && self.is_c_contiguous() { unsafe { Some(slice::from_raw_parts( self.0.buf as *mut cell::Cell, self.item_count(), )) } } else { None } } /// Gets the buffer memory as a slice. /// /// This function succeeds if: /// * the buffer format is compatible with `T` /// * alignment and size of buffer elements is matching the expectations for type `T` /// * the buffer is Fortran-style contiguous /// /// The returned slice uses type `Cell` because it's theoretically possible for any call into the Python runtime /// to modify the values in the slice. pub fn as_fortran_slice<'a>(&'a self, _py: Python<'a>) -> Option<&'a [ReadOnlyCell]> { if mem::size_of::() == self.item_size() && self.is_fortran_contiguous() { unsafe { Some(slice::from_raw_parts( self.0.buf as *mut ReadOnlyCell, self.item_count(), )) } } else { None } } /// Gets the buffer memory as a slice. /// /// This function succeeds if: /// * the buffer is not read-only /// * the buffer format is compatible with `T` /// * alignment and size of buffer elements is matching the expectations for type `T` /// * the buffer is Fortran-style contiguous /// /// The returned slice uses type `Cell` because it's theoretically possible for any call into the Python runtime /// to modify the values in the slice. pub fn as_fortran_mut_slice<'a>(&'a self, _py: Python<'a>) -> Option<&'a [cell::Cell]> { if !self.readonly() && self.is_fortran_contiguous() { unsafe { Some(slice::from_raw_parts( self.0.buf as *mut cell::Cell, self.item_count(), )) } } else { None } } /// Copies the buffer elements to the specified slice. /// If the buffer is multi-dimensional, the elements are written in C-style order. /// /// * Fails if the slice does not have the correct length (`buf.item_count()`). /// * Fails if the buffer format is not compatible with type `T`. /// /// To check whether the buffer format is compatible before calling this method, /// you can use `::is_compatible_format(buf.format())`. /// Alternatively, `match buffer::ElementType::from_format(buf.format())`. pub fn copy_to_slice(&self, py: Python<'_>, target: &mut [T]) -> PyResult<()> { self._copy_to_slice(py, target, b'C') } /// Copies the buffer elements to the specified slice. /// If the buffer is multi-dimensional, the elements are written in Fortran-style order. /// /// * Fails if the slice does not have the correct length (`buf.item_count()`). /// * Fails if the buffer format is not compatible with type `T`. /// /// To check whether the buffer format is compatible before calling this method, /// you can use `::is_compatible_format(buf.format())`. /// Alternatively, `match buffer::ElementType::from_format(buf.format())`. pub fn copy_to_fortran_slice(&self, py: Python<'_>, target: &mut [T]) -> PyResult<()> { self._copy_to_slice(py, target, b'F') } fn _copy_to_slice(&self, py: Python<'_>, target: &mut [T], fort: u8) -> PyResult<()> { if mem::size_of_val(target) != self.len_bytes() { return Err(PyBufferError::new_err(format!( "slice to copy to (of length {}) does not match buffer length of {}", target.len(), self.item_count() ))); } err::error_on_minusone(py, unsafe { ffi::PyBuffer_ToContiguous( target.as_mut_ptr().cast(), #[cfg(Py_3_11)] &*self.0, #[cfg(not(Py_3_11))] { &*self.0 as *const ffi::Py_buffer as *mut ffi::Py_buffer }, self.0.len, fort as std::os::raw::c_char, ) }) } /// Copies the buffer elements to a newly allocated vector. /// If the buffer is multi-dimensional, the elements are written in C-style order. /// /// Fails if the buffer format is not compatible with type `T`. pub fn to_vec(&self, py: Python<'_>) -> PyResult> { self._to_vec(py, b'C') } /// Copies the buffer elements to a newly allocated vector. /// If the buffer is multi-dimensional, the elements are written in Fortran-style order. /// /// Fails if the buffer format is not compatible with type `T`. pub fn to_fortran_vec(&self, py: Python<'_>) -> PyResult> { self._to_vec(py, b'F') } fn _to_vec(&self, py: Python<'_>, fort: u8) -> PyResult> { let item_count = self.item_count(); let mut vec: Vec = Vec::with_capacity(item_count); // Copy the buffer into the uninitialized space in the vector. // Due to T:Copy, we don't need to be concerned with Drop impls. err::error_on_minusone(py, unsafe { ffi::PyBuffer_ToContiguous( vec.as_ptr() as *mut raw::c_void, #[cfg(Py_3_11)] &*self.0, #[cfg(not(Py_3_11))] { &*self.0 as *const ffi::Py_buffer as *mut ffi::Py_buffer }, self.0.len, fort as std::os::raw::c_char, ) })?; // set vector length to mark the now-initialized space as usable unsafe { vec.set_len(item_count) }; Ok(vec) } /// Copies the specified slice into the buffer. /// If the buffer is multi-dimensional, the elements in the slice are expected to be in C-style order. /// /// * Fails if the buffer is read-only. /// * Fails if the slice does not have the correct length (`buf.item_count()`). /// * Fails if the buffer format is not compatible with type `T`. /// /// To check whether the buffer format is compatible before calling this method, /// use `::is_compatible_format(buf.format())`. /// Alternatively, `match buffer::ElementType::from_format(buf.format())`. pub fn copy_from_slice(&self, py: Python<'_>, source: &[T]) -> PyResult<()> { self._copy_from_slice(py, source, b'C') } /// Copies the specified slice into the buffer. /// If the buffer is multi-dimensional, the elements in the slice are expected to be in Fortran-style order. /// /// * Fails if the buffer is read-only. /// * Fails if the slice does not have the correct length (`buf.item_count()`). /// * Fails if the buffer format is not compatible with type `T`. /// /// To check whether the buffer format is compatible before calling this method, /// use `::is_compatible_format(buf.format())`. /// Alternatively, `match buffer::ElementType::from_format(buf.format())`. pub fn copy_from_fortran_slice(&self, py: Python<'_>, source: &[T]) -> PyResult<()> { self._copy_from_slice(py, source, b'F') } fn _copy_from_slice(&self, py: Python<'_>, source: &[T], fort: u8) -> PyResult<()> { if self.readonly() { return Err(PyBufferError::new_err("cannot write to read-only buffer")); } else if mem::size_of_val(source) != self.len_bytes() { return Err(PyBufferError::new_err(format!( "slice to copy from (of length {}) does not match buffer length of {}", source.len(), self.item_count() ))); } err::error_on_minusone(py, unsafe { ffi::PyBuffer_FromContiguous( #[cfg(Py_3_11)] &*self.0, #[cfg(not(Py_3_11))] { &*self.0 as *const ffi::Py_buffer as *mut ffi::Py_buffer }, #[cfg(Py_3_11)] { source.as_ptr() as *const raw::c_void }, #[cfg(not(Py_3_11))] { source.as_ptr() as *mut raw::c_void }, self.0.len, fort as std::os::raw::c_char, ) }) } /// Releases the buffer object, freeing the reference to the Python object /// which owns the buffer. /// /// This will automatically be called on drop. pub fn release(self, _py: Python<'_>) { // First move self into a ManuallyDrop, so that PyBuffer::drop will // never be called. (It would acquire the GIL and call PyBuffer_Release // again.) let mut mdself = mem::ManuallyDrop::new(self); unsafe { // Next, make the actual PyBuffer_Release call. ffi::PyBuffer_Release(&mut *mdself.0); // Finally, drop the contained Pin> in place, to free the // Box memory. let inner: *mut Pin> = &mut mdself.0; ptr::drop_in_place(inner); } } } impl Drop for PyBuffer { fn drop(&mut self) { Python::with_gil(|_| unsafe { ffi::PyBuffer_Release(&mut *self.0) }); } } /// Like [std::cell::Cell], but only provides read-only access to the data. /// /// `&ReadOnlyCell` is basically a safe version of `*const T`: /// The data cannot be modified through the reference, but other references may /// be modifying the data. #[repr(transparent)] pub struct ReadOnlyCell(cell::UnsafeCell); impl ReadOnlyCell { /// Returns a copy of the current value. #[inline] pub fn get(&self) -> T { unsafe { *self.0.get() } } /// Returns a pointer to the current value. #[inline] pub fn as_ptr(&self) -> *const T { self.0.get() } } macro_rules! impl_element( ($t:ty, $f:ident) => { unsafe impl Element for $t { fn is_compatible_format(format: &CStr) -> bool { let slice = format.to_bytes(); if slice.len() > 1 && !is_matching_endian(slice[0]) { return false; } ElementType::from_format(format) == ElementType::$f { bytes: mem::size_of::<$t>() } } } } ); impl_element!(u8, UnsignedInteger); impl_element!(u16, UnsignedInteger); impl_element!(u32, UnsignedInteger); impl_element!(u64, UnsignedInteger); impl_element!(usize, UnsignedInteger); impl_element!(i8, SignedInteger); impl_element!(i16, SignedInteger); impl_element!(i32, SignedInteger); impl_element!(i64, SignedInteger); impl_element!(isize, SignedInteger); impl_element!(f32, Float); impl_element!(f64, Float); #[cfg(test)] mod tests { use super::PyBuffer; use crate::ffi; use crate::Python; #[test] fn test_debug() { Python::with_gil(|py| { let bytes = py.eval("b'abcde'", None, None).unwrap(); let buffer: PyBuffer = PyBuffer::get(bytes).unwrap(); let expected = format!( concat!( "PyBuffer {{ buf: {:?}, obj: {:?}, ", "len: 5, itemsize: 1, readonly: 1, ", "ndim: 1, format: {:?}, shape: {:?}, ", "strides: {:?}, suboffsets: {:?}, internal: {:?} }}", ), buffer.0.buf, buffer.0.obj, buffer.0.format, buffer.0.shape, buffer.0.strides, buffer.0.suboffsets, buffer.0.internal ); let debug_repr = format!("{:?}", buffer); assert_eq!(debug_repr, expected); }); } #[test] fn test_element_type_from_format() { use super::ElementType; use super::ElementType::*; use std::ffi::CStr; use std::mem::size_of; use std::os::raw; for (cstr, expected) in &[ // @ prefix goes to native_element_type_from_type_char ( "@b\0", SignedInteger { bytes: size_of::(), }, ), ( "@c\0", UnsignedInteger { bytes: size_of::(), }, ), ( "@b\0", SignedInteger { bytes: size_of::(), }, ), ( "@B\0", UnsignedInteger { bytes: size_of::(), }, ), ("@?\0", Bool), ( "@h\0", SignedInteger { bytes: size_of::(), }, ), ( "@H\0", UnsignedInteger { bytes: size_of::(), }, ), ( "@i\0", SignedInteger { bytes: size_of::(), }, ), ( "@I\0", UnsignedInteger { bytes: size_of::(), }, ), ( "@l\0", SignedInteger { bytes: size_of::(), }, ), ( "@L\0", UnsignedInteger { bytes: size_of::(), }, ), ( "@q\0", SignedInteger { bytes: size_of::(), }, ), ( "@Q\0", UnsignedInteger { bytes: size_of::(), }, ), ( "@n\0", SignedInteger { bytes: size_of::(), }, ), ( "@N\0", UnsignedInteger { bytes: size_of::(), }, ), ("@e\0", Float { bytes: 2 }), ("@f\0", Float { bytes: 4 }), ("@d\0", Float { bytes: 8 }), ("@z\0", Unknown), // = prefix goes to standard_element_type_from_type_char ("=b\0", SignedInteger { bytes: 1 }), ("=c\0", UnsignedInteger { bytes: 1 }), ("=B\0", UnsignedInteger { bytes: 1 }), ("=?\0", Bool), ("=h\0", SignedInteger { bytes: 2 }), ("=H\0", UnsignedInteger { bytes: 2 }), ("=l\0", SignedInteger { bytes: 4 }), ("=l\0", SignedInteger { bytes: 4 }), ("=I\0", UnsignedInteger { bytes: 4 }), ("=L\0", UnsignedInteger { bytes: 4 }), ("=q\0", SignedInteger { bytes: 8 }), ("=Q\0", UnsignedInteger { bytes: 8 }), ("=e\0", Float { bytes: 2 }), ("=f\0", Float { bytes: 4 }), ("=d\0", Float { bytes: 8 }), ("=z\0", Unknown), ("=0\0", Unknown), // unknown prefix -> Unknown (":b\0", Unknown), ] { assert_eq!( ElementType::from_format(CStr::from_bytes_with_nul(cstr.as_bytes()).unwrap()), *expected, "element from format &Cstr: {:?}", cstr, ); } } #[test] fn test_compatible_size() { // for the cast in PyBuffer::shape() assert_eq!( std::mem::size_of::(), std::mem::size_of::() ); } #[test] fn test_bytes_buffer() { Python::with_gil(|py| { let bytes = py.eval("b'abcde'", None, None).unwrap(); let buffer = PyBuffer::get(bytes).unwrap(); assert_eq!(buffer.dimensions(), 1); assert_eq!(buffer.item_count(), 5); assert_eq!(buffer.format().to_str().unwrap(), "B"); assert_eq!(buffer.shape(), [5]); // single-dimensional buffer is always contiguous assert!(buffer.is_c_contiguous()); assert!(buffer.is_fortran_contiguous()); let slice = buffer.as_slice(py).unwrap(); assert_eq!(slice.len(), 5); assert_eq!(slice[0].get(), b'a'); assert_eq!(slice[2].get(), b'c'); assert_eq!(unsafe { *(buffer.get_ptr(&[1]) as *mut u8) }, b'b'); assert!(buffer.as_mut_slice(py).is_none()); assert!(buffer.copy_to_slice(py, &mut [0u8]).is_err()); let mut arr = [0; 5]; buffer.copy_to_slice(py, &mut arr).unwrap(); assert_eq!(arr, b"abcde" as &[u8]); assert!(buffer.copy_from_slice(py, &[0u8; 5]).is_err()); assert_eq!(buffer.to_vec(py).unwrap(), b"abcde"); }); } #[test] fn test_array_buffer() { Python::with_gil(|py| { let array = py .import("array") .unwrap() .call_method("array", ("f", (1.0, 1.5, 2.0, 2.5)), None) .unwrap(); let buffer = PyBuffer::get(array).unwrap(); assert_eq!(buffer.dimensions(), 1); assert_eq!(buffer.item_count(), 4); assert_eq!(buffer.format().to_str().unwrap(), "f"); assert_eq!(buffer.shape(), [4]); // array creates a 1D contiguious buffer, so it's both C and F contiguous. This would // be more interesting if we can come up with a 2D buffer but I think it would need a // third-party lib or a custom class. // C-contiguous fns let slice = buffer.as_slice(py).unwrap(); assert_eq!(slice.len(), 4); assert_eq!(slice[0].get(), 1.0); assert_eq!(slice[3].get(), 2.5); let mut_slice = buffer.as_mut_slice(py).unwrap(); assert_eq!(mut_slice.len(), 4); assert_eq!(mut_slice[0].get(), 1.0); mut_slice[3].set(2.75); assert_eq!(slice[3].get(), 2.75); buffer .copy_from_slice(py, &[10.0f32, 11.0, 12.0, 13.0]) .unwrap(); assert_eq!(slice[2].get(), 12.0); assert_eq!(buffer.to_vec(py).unwrap(), [10.0, 11.0, 12.0, 13.0]); // F-contiguous fns let buffer = PyBuffer::get(array).unwrap(); let slice = buffer.as_fortran_slice(py).unwrap(); assert_eq!(slice.len(), 4); assert_eq!(slice[1].get(), 11.0); let mut_slice = buffer.as_fortran_mut_slice(py).unwrap(); assert_eq!(mut_slice.len(), 4); assert_eq!(mut_slice[2].get(), 12.0); mut_slice[3].set(2.75); assert_eq!(slice[3].get(), 2.75); buffer .copy_from_fortran_slice(py, &[10.0f32, 11.0, 12.0, 13.0]) .unwrap(); assert_eq!(slice[2].get(), 12.0); assert_eq!(buffer.to_fortran_vec(py).unwrap(), [10.0, 11.0, 12.0, 13.0]); }); } } pyo3-0.20.2/src/callback.rs000064400000000000000000000101571046102023000134740ustar 00000000000000//! Utilities for a Python callable object that invokes a Rust function. use crate::err::{PyErr, PyResult}; use crate::exceptions::PyOverflowError; use crate::ffi::{self, Py_hash_t}; use crate::{IntoPy, PyObject, Python}; use std::isize; use std::os::raw::c_int; /// A type which can be the return type of a python C-API callback pub trait PyCallbackOutput: Copy { /// The error value to return to python if the callback raised an exception const ERR_VALUE: Self; } impl PyCallbackOutput for *mut ffi::PyObject { const ERR_VALUE: Self = std::ptr::null_mut(); } impl PyCallbackOutput for std::os::raw::c_int { const ERR_VALUE: Self = -1; } impl PyCallbackOutput for ffi::Py_ssize_t { const ERR_VALUE: Self = -1; } /// Convert the result of callback function into the appropriate return value. pub trait IntoPyCallbackOutput { fn convert(self, py: Python<'_>) -> PyResult; } impl IntoPyCallbackOutput for Result where T: IntoPyCallbackOutput, E: Into, { #[inline] fn convert(self, py: Python<'_>) -> PyResult { match self { Ok(v) => v.convert(py), Err(e) => Err(e.into()), } } } impl IntoPyCallbackOutput<*mut ffi::PyObject> for T where T: IntoPy, { #[inline] fn convert(self, py: Python<'_>) -> PyResult<*mut ffi::PyObject> { Ok(self.into_py(py).into_ptr()) } } impl IntoPyCallbackOutput for *mut ffi::PyObject { #[inline] fn convert(self, _: Python<'_>) -> PyResult { Ok(self) } } impl IntoPyCallbackOutput for () { #[inline] fn convert(self, _: Python<'_>) -> PyResult { Ok(0) } } impl IntoPyCallbackOutput for bool { #[inline] fn convert(self, _: Python<'_>) -> PyResult { Ok(self as c_int) } } impl IntoPyCallbackOutput<()> for () { #[inline] fn convert(self, _: Python<'_>) -> PyResult<()> { Ok(()) } } impl IntoPyCallbackOutput for usize { #[inline] fn convert(self, _py: Python<'_>) -> PyResult { if self <= (isize::MAX as usize) { Ok(self as isize) } else { Err(PyOverflowError::new_err(())) } } } // Converters needed for `#[pyproto]` implementations impl IntoPyCallbackOutput for bool { #[inline] fn convert(self, _: Python<'_>) -> PyResult { Ok(self) } } impl IntoPyCallbackOutput for usize { #[inline] fn convert(self, _: Python<'_>) -> PyResult { Ok(self) } } impl IntoPyCallbackOutput for T where T: IntoPy, { #[inline] fn convert(self, py: Python<'_>) -> PyResult { Ok(self.into_py(py)) } } pub trait WrappingCastTo { fn wrapping_cast(self) -> T; } macro_rules! wrapping_cast { ($from:ty, $to:ty) => { impl WrappingCastTo<$to> for $from { #[inline] fn wrapping_cast(self) -> $to { self as $to } } }; } wrapping_cast!(u8, Py_hash_t); wrapping_cast!(u16, Py_hash_t); wrapping_cast!(u32, Py_hash_t); wrapping_cast!(usize, Py_hash_t); wrapping_cast!(u64, Py_hash_t); wrapping_cast!(i8, Py_hash_t); wrapping_cast!(i16, Py_hash_t); wrapping_cast!(i32, Py_hash_t); wrapping_cast!(isize, Py_hash_t); wrapping_cast!(i64, Py_hash_t); pub struct HashCallbackOutput(Py_hash_t); impl IntoPyCallbackOutput for HashCallbackOutput { #[inline] fn convert(self, _py: Python<'_>) -> PyResult { let hash = self.0; if hash == -1 { Ok(-2) } else { Ok(hash) } } } impl IntoPyCallbackOutput for T where T: WrappingCastTo, { #[inline] fn convert(self, _py: Python<'_>) -> PyResult { Ok(HashCallbackOutput(self.wrapping_cast())) } } #[doc(hidden)] #[inline] pub fn convert(py: Python<'_>, value: T) -> PyResult where T: IntoPyCallbackOutput, { value.convert(py) } pyo3-0.20.2/src/conversion.rs000064400000000000000000000476741046102023000141430ustar 00000000000000//! Defines conversions between Rust and Python types. use crate::err::{self, PyDowncastError, PyResult}; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::pyclass::boolean_struct::False; use crate::type_object::PyTypeInfo; use crate::types::PyTuple; use crate::{ ffi, gil, Py, PyAny, PyCell, PyClass, PyNativeType, PyObject, PyRef, PyRefMut, Python, }; use std::cell::Cell; use std::ptr::NonNull; /// Returns a borrowed pointer to a Python object. /// /// The returned pointer will be valid for as long as `self` is. It may be null depending on the /// implementation. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyString; /// use pyo3::ffi; /// /// Python::with_gil(|py| { /// let s: Py = "foo".into_py(py); /// let ptr = s.as_ptr(); /// /// let is_really_a_pystring = unsafe { ffi::PyUnicode_CheckExact(ptr) }; /// assert_eq!(is_really_a_pystring, 1); /// }); /// ``` /// /// # Safety /// /// For callers, it is your responsibility to make sure that the underlying Python object is not dropped too /// early. For example, the following code will cause undefined behavior: /// /// ```rust,no_run /// # use pyo3::prelude::*; /// # use pyo3::ffi; /// # /// Python::with_gil(|py| { /// let ptr: *mut ffi::PyObject = 0xabad1dea_u32.into_py(py).as_ptr(); /// /// let isnt_a_pystring = unsafe { /// // `ptr` is dangling, this is UB /// ffi::PyUnicode_CheckExact(ptr) /// }; /// # assert_eq!(isnt_a_pystring, 0); /// }); /// ``` /// /// This happens because the pointer returned by `as_ptr` does not carry any lifetime information /// and the Python object is dropped immediately after the `0xabad1dea_u32.into_py(py).as_ptr()` /// expression is evaluated. To fix the problem, bind Python object to a local variable like earlier /// to keep the Python object alive until the end of its scope. /// /// Implementors must ensure this returns a valid pointer to a Python object, which borrows a reference count from `&self`. pub unsafe trait AsPyPointer { /// Returns the underlying FFI pointer as a borrowed pointer. fn as_ptr(&self) -> *mut ffi::PyObject; } /// Convert `None` into a null pointer. unsafe impl AsPyPointer for Option where T: AsPyPointer, { #[inline] fn as_ptr(&self) -> *mut ffi::PyObject { self.as_ref() .map_or_else(std::ptr::null_mut, |t| t.as_ptr()) } } /// Conversion trait that allows various objects to be converted into `PyObject`. pub trait ToPyObject { /// Converts self into a Python object. fn to_object(&self, py: Python<'_>) -> PyObject; } /// Defines a conversion from a Rust type to a Python object. /// /// It functions similarly to std's [`Into`] trait, but requires a [GIL token](Python) /// as an argument. Many functions and traits internal to PyO3 require this trait as a bound, /// so a lack of this trait can manifest itself in different error messages. /// /// # Examples /// ## With `#[pyclass]` /// The easiest way to implement `IntoPy` is by exposing a struct as a native Python object /// by annotating it with [`#[pyclass]`](crate::prelude::pyclass). /// /// ```rust /// use pyo3::prelude::*; /// /// #[pyclass] /// struct Number { /// #[pyo3(get, set)] /// value: i32, /// } /// ``` /// Python code will see this as an instance of the `Number` class with a `value` attribute. /// /// ## Conversion to a Python object /// /// However, it may not be desirable to expose the existence of `Number` to Python code. /// `IntoPy` allows us to define a conversion to an appropriate Python object. /// ```rust /// use pyo3::prelude::*; /// /// struct Number { /// value: i32, /// } /// /// impl IntoPy for Number { /// fn into_py(self, py: Python<'_>) -> PyObject { /// // delegates to i32's IntoPy implementation. /// self.value.into_py(py) /// } /// } /// ``` /// Python code will see this as an `int` object. /// /// ## Dynamic conversion into Python objects. /// It is also possible to return a different Python object depending on some condition. /// This is useful for types like enums that can carry different types. /// /// ```rust /// use pyo3::prelude::*; /// /// enum Value { /// Integer(i32), /// String(String), /// None, /// } /// /// impl IntoPy for Value { /// fn into_py(self, py: Python<'_>) -> PyObject { /// match self { /// Self::Integer(val) => val.into_py(py), /// Self::String(val) => val.into_py(py), /// Self::None => py.None(), /// } /// } /// } /// # fn main() { /// # Python::with_gil(|py| { /// # let v = Value::Integer(73).into_py(py); /// # let v = v.extract::(py).unwrap(); /// # /// # let v = Value::String("foo".into()).into_py(py); /// # let v = v.extract::(py).unwrap(); /// # /// # let v = Value::None.into_py(py); /// # let v = v.extract::>>(py).unwrap(); /// # }); /// # } /// ``` /// Python code will see this as any of the `int`, `string` or `None` objects. #[doc(alias = "IntoPyCallbackOutput")] pub trait IntoPy: Sized { /// Performs the conversion. fn into_py(self, py: Python<'_>) -> T; /// Extracts the type hint information for this type when it appears as a return value. /// /// For example, `Vec` would return `List[int]`. /// The default implementation returns `Any`, which is correct for any type. /// /// For most types, the return value for this method will be identical to that of [`FromPyObject::type_input`]. /// It may be different for some types, such as `Dict`, to allow duck-typing: functions return `Dict` but take `Mapping` as argument. #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::Any } } /// Extract a type from a Python object. /// /// /// Normal usage is through the `extract` methods on [`Py`] and [`PyAny`], which forward to this trait. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyString; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// let obj: Py = PyString::new(py, "blah").into(); /// /// // Straight from an owned reference /// let s: &str = obj.extract(py)?; /// # assert_eq!(s, "blah"); /// /// // Or from a borrowed reference /// let obj: &PyString = obj.as_ref(py); /// let s: &str = obj.extract()?; /// # assert_eq!(s, "blah"); /// # Ok(()) /// }) /// # } /// ``` /// /// Note: depending on the implementation, the lifetime of the extracted result may /// depend on the lifetime of the `obj` or the `prepared` variable. /// /// For example, when extracting `&str` from a Python byte string, the resulting string slice will /// point to the existing string data (lifetime: `'source`). /// On the other hand, when extracting `&str` from a Python Unicode string, the preparation step /// will convert the string to UTF-8, and the resulting string slice will have lifetime `'prepared`. /// Since which case applies depends on the runtime type of the Python object, /// both the `obj` and `prepared` variables must outlive the resulting string slice. /// /// The trait's conversion method takes a `&PyAny` argument but is called /// `FromPyObject` for historical reasons. pub trait FromPyObject<'source>: Sized { /// Extracts `Self` from the source `PyObject`. fn extract(ob: &'source PyAny) -> PyResult; /// Extracts the type hint information for this type when it appears as an argument. /// /// For example, `Vec` would return `Sequence[int]`. /// The default implementation returns `Any`, which is correct for any type. /// /// For most types, the return value for this method will be identical to that of [`IntoPy::type_output`]. /// It may be different for some types, such as `Dict`, to allow duck-typing: functions return `Dict` but take `Mapping` as argument. #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { TypeInfo::Any } } /// Identity conversion: allows using existing `PyObject` instances where /// `T: ToPyObject` is expected. impl ToPyObject for &'_ T { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { ::to_object(*self, py) } } /// `Option::Some` is converted like `T`. /// `Option::None` is converted to Python `None`. impl ToPyObject for Option where T: ToPyObject, { fn to_object(&self, py: Python<'_>) -> PyObject { self.as_ref() .map_or_else(|| py.None(), |val| val.to_object(py)) } } impl IntoPy for Option where T: IntoPy, { fn into_py(self, py: Python<'_>) -> PyObject { self.map_or_else(|| py.None(), |val| val.into_py(py)) } } impl IntoPy for &'_ PyAny { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { unsafe { PyObject::from_borrowed_ptr(py, self.as_ptr()) } } } impl IntoPy for &'_ T where T: AsRef, { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { unsafe { PyObject::from_borrowed_ptr(py, self.as_ref().as_ptr()) } } } impl ToPyObject for Cell { fn to_object(&self, py: Python<'_>) -> PyObject { self.get().to_object(py) } } impl> IntoPy for Cell { fn into_py(self, py: Python<'_>) -> PyObject { self.get().into_py(py) } } impl<'a, T: FromPyObject<'a>> FromPyObject<'a> for Cell { fn extract(ob: &'a PyAny) -> PyResult { T::extract(ob).map(Cell::new) } } impl<'a, T> FromPyObject<'a> for &'a PyCell where T: PyClass, { fn extract(obj: &'a PyAny) -> PyResult { PyTryFrom::try_from(obj).map_err(Into::into) } } impl<'a, T> FromPyObject<'a> for T where T: PyClass + Clone, { fn extract(obj: &'a PyAny) -> PyResult { let cell: &PyCell = PyTryFrom::try_from(obj)?; Ok(unsafe { cell.try_borrow_unguarded()?.clone() }) } } impl<'a, T> FromPyObject<'a> for PyRef<'a, T> where T: PyClass, { fn extract(obj: &'a PyAny) -> PyResult { let cell: &PyCell = PyTryFrom::try_from(obj)?; cell.try_borrow().map_err(Into::into) } } impl<'a, T> FromPyObject<'a> for PyRefMut<'a, T> where T: PyClass, { fn extract(obj: &'a PyAny) -> PyResult { let cell: &PyCell = PyTryFrom::try_from(obj)?; cell.try_borrow_mut().map_err(Into::into) } } impl<'a, T> FromPyObject<'a> for Option where T: FromPyObject<'a>, { fn extract(obj: &'a PyAny) -> PyResult { if obj.as_ptr() == unsafe { ffi::Py_None() } { Ok(None) } else { T::extract(obj).map(Some) } } } /// Trait implemented by Python object types that allow a checked downcast. /// If `T` implements `PyTryFrom`, we can convert `&PyAny` to `&T`. /// /// This trait is similar to `std::convert::TryFrom` pub trait PyTryFrom<'v>: Sized + PyNativeType { /// Cast from a concrete Python object type to PyObject. fn try_from>(value: V) -> Result<&'v Self, PyDowncastError<'v>>; /// Cast from a concrete Python object type to PyObject. With exact type check. fn try_from_exact>(value: V) -> Result<&'v Self, PyDowncastError<'v>>; /// Cast a PyAny to a specific type of PyObject. The caller must /// have already verified the reference is for this type. /// /// # Safety /// /// Callers must ensure that the type is valid or risk type confusion. unsafe fn try_from_unchecked>(value: V) -> &'v Self; } /// Trait implemented by Python object types that allow a checked downcast. /// This trait is similar to `std::convert::TryInto` pub trait PyTryInto: Sized { /// Cast from PyObject to a concrete Python object type. fn try_into(&self) -> Result<&T, PyDowncastError<'_>>; /// Cast from PyObject to a concrete Python object type. With exact type check. fn try_into_exact(&self) -> Result<&T, PyDowncastError<'_>>; } // TryFrom implies TryInto impl PyTryInto for PyAny where U: for<'v> PyTryFrom<'v>, { fn try_into(&self) -> Result<&U, PyDowncastError<'_>> { >::try_from(self) } fn try_into_exact(&self) -> Result<&U, PyDowncastError<'_>> { U::try_from_exact(self) } } impl<'v, T> PyTryFrom<'v> for T where T: PyTypeInfo + PyNativeType, { fn try_from>(value: V) -> Result<&'v Self, PyDowncastError<'v>> { let value = value.into(); unsafe { if T::is_type_of(value) { Ok(Self::try_from_unchecked(value)) } else { Err(PyDowncastError::new(value, T::NAME)) } } } fn try_from_exact>(value: V) -> Result<&'v Self, PyDowncastError<'v>> { let value = value.into(); unsafe { if T::is_exact_type_of(value) { Ok(Self::try_from_unchecked(value)) } else { Err(PyDowncastError::new(value, T::NAME)) } } } #[inline] unsafe fn try_from_unchecked>(value: V) -> &'v Self { Self::unchecked_downcast(value.into()) } } impl<'v, T> PyTryFrom<'v> for PyCell where T: 'v + PyClass, { fn try_from>(value: V) -> Result<&'v Self, PyDowncastError<'v>> { let value = value.into(); unsafe { if T::is_type_of(value) { Ok(Self::try_from_unchecked(value)) } else { Err(PyDowncastError::new(value, T::NAME)) } } } fn try_from_exact>(value: V) -> Result<&'v Self, PyDowncastError<'v>> { let value = value.into(); unsafe { if T::is_exact_type_of(value) { Ok(Self::try_from_unchecked(value)) } else { Err(PyDowncastError::new(value, T::NAME)) } } } #[inline] unsafe fn try_from_unchecked>(value: V) -> &'v Self { Self::unchecked_downcast(value.into()) } } /// Converts `()` to an empty Python tuple. impl IntoPy> for () { fn into_py(self, py: Python<'_>) -> Py { PyTuple::empty(py).into() } } /// Raw level conversion between `*mut ffi::PyObject` and PyO3 types. /// /// # Safety /// /// See safety notes on individual functions. pub unsafe trait FromPyPointer<'p>: Sized { /// Convert from an arbitrary `PyObject`. /// /// # Safety /// /// Implementations must ensure the object does not get freed during `'p` /// and ensure that `ptr` is of the correct type. /// Note that it must be safe to decrement the reference count of `ptr`. unsafe fn from_owned_ptr_or_opt(py: Python<'p>, ptr: *mut ffi::PyObject) -> Option<&'p Self>; /// Convert from an arbitrary `PyObject` or panic. /// /// # Safety /// /// Relies on [`from_owned_ptr_or_opt`](#method.from_owned_ptr_or_opt). unsafe fn from_owned_ptr_or_panic(py: Python<'p>, ptr: *mut ffi::PyObject) -> &'p Self { Self::from_owned_ptr_or_opt(py, ptr).unwrap_or_else(|| err::panic_after_error(py)) } /// Convert from an arbitrary `PyObject` or panic. /// /// # Safety /// /// Relies on [`from_owned_ptr_or_opt`](#method.from_owned_ptr_or_opt). unsafe fn from_owned_ptr(py: Python<'p>, ptr: *mut ffi::PyObject) -> &'p Self { Self::from_owned_ptr_or_panic(py, ptr) } /// Convert from an arbitrary `PyObject`. /// /// # Safety /// /// Relies on [`from_owned_ptr_or_opt`](#method.from_owned_ptr_or_opt). unsafe fn from_owned_ptr_or_err(py: Python<'p>, ptr: *mut ffi::PyObject) -> PyResult<&'p Self> { Self::from_owned_ptr_or_opt(py, ptr).ok_or_else(|| err::PyErr::fetch(py)) } /// Convert from an arbitrary borrowed `PyObject`. /// /// # Safety /// /// Implementations must ensure the object does not get freed during `'p` and avoid type confusion. unsafe fn from_borrowed_ptr_or_opt(py: Python<'p>, ptr: *mut ffi::PyObject) -> Option<&'p Self>; /// Convert from an arbitrary borrowed `PyObject`. /// /// # Safety /// /// Relies on unsafe fn [`from_borrowed_ptr_or_opt`](#method.from_borrowed_ptr_or_opt). unsafe fn from_borrowed_ptr_or_panic(py: Python<'p>, ptr: *mut ffi::PyObject) -> &'p Self { Self::from_borrowed_ptr_or_opt(py, ptr).unwrap_or_else(|| err::panic_after_error(py)) } /// Convert from an arbitrary borrowed `PyObject`. /// /// # Safety /// /// Relies on unsafe fn [`from_borrowed_ptr_or_opt`](#method.from_borrowed_ptr_or_opt). unsafe fn from_borrowed_ptr(py: Python<'p>, ptr: *mut ffi::PyObject) -> &'p Self { Self::from_borrowed_ptr_or_panic(py, ptr) } /// Convert from an arbitrary borrowed `PyObject`. /// /// # Safety /// /// Relies on unsafe fn [`from_borrowed_ptr_or_opt`](#method.from_borrowed_ptr_or_opt). unsafe fn from_borrowed_ptr_or_err( py: Python<'p>, ptr: *mut ffi::PyObject, ) -> PyResult<&'p Self> { Self::from_borrowed_ptr_or_opt(py, ptr).ok_or_else(|| err::PyErr::fetch(py)) } } unsafe impl<'p, T> FromPyPointer<'p> for T where T: 'p + crate::PyNativeType, { unsafe fn from_owned_ptr_or_opt(py: Python<'p>, ptr: *mut ffi::PyObject) -> Option<&'p Self> { gil::register_owned(py, NonNull::new(ptr)?); Some(&*(ptr as *mut Self)) } unsafe fn from_borrowed_ptr_or_opt( _py: Python<'p>, ptr: *mut ffi::PyObject, ) -> Option<&'p Self> { NonNull::new(ptr as *mut Self).map(|p| &*p.as_ptr()) } } /// ```rust,compile_fail /// use pyo3::prelude::*; /// /// #[pyclass] /// struct TestClass { /// num: u32, /// } /// /// let t = TestClass { num: 10 }; /// /// Python::with_gil(|py| { /// let pyvalue = Py::new(py, t).unwrap().to_object(py); /// let t: TestClass = pyvalue.extract(py).unwrap(); /// }) /// ``` mod test_no_clone {} #[cfg(test)] mod tests { use crate::types::{IntoPyDict, PyAny, PyDict, PyList}; use crate::{PyObject, Python, ToPyObject}; use super::PyTryFrom; #[test] fn test_try_from() { Python::with_gil(|py| { let list: &PyAny = vec![3, 6, 5, 4, 7].to_object(py).into_ref(py); let dict: &PyAny = vec![("reverse", true)].into_py_dict(py).as_ref(); assert!(>::try_from(list).is_ok()); assert!(>::try_from(dict).is_ok()); assert!(>::try_from(list).is_ok()); assert!(>::try_from(dict).is_ok()); }); } #[test] fn test_try_from_exact() { Python::with_gil(|py| { let list: &PyAny = vec![3, 6, 5, 4, 7].to_object(py).into_ref(py); let dict: &PyAny = vec![("reverse", true)].into_py_dict(py).as_ref(); assert!(PyList::try_from_exact(list).is_ok()); assert!(PyDict::try_from_exact(dict).is_ok()); assert!(PyAny::try_from_exact(list).is_err()); assert!(PyAny::try_from_exact(dict).is_err()); }); } #[test] fn test_try_from_unchecked() { Python::with_gil(|py| { let list = PyList::new(py, [1, 2, 3]); let val = unsafe { ::try_from_unchecked(list.as_ref()) }; assert!(list.is(val)); }); } #[test] fn test_option_as_ptr() { Python::with_gil(|py| { use crate::AsPyPointer; let mut option: Option = None; assert_eq!(option.as_ptr(), std::ptr::null_mut()); let none = py.None(); option = Some(none.clone()); let ref_cnt = none.get_refcnt(py); assert_eq!(option.as_ptr(), none.as_ptr()); // Ensure ref count not changed by as_ptr call assert_eq!(none.get_refcnt(py), ref_cnt); }); } } pyo3-0.20.2/src/conversions/anyhow.rs000064400000000000000000000151141046102023000156130ustar 00000000000000#![cfg(feature = "anyhow")] //! A conversion from //! [anyhow](https://docs.rs/anyhow/ "A trait object based error system for easy idiomatic error handling in Rust applications.")’s //! [`Error`](https://docs.rs/anyhow/latest/anyhow/struct.Error.html "Anyhows `Error` type, a wrapper around a dynamic error type") //! type to [`PyErr`]. //! //! Use of an error handling library like [anyhow] is common in application code and when you just //! want error handling to be easy. If you are writing a library or you need more control over your //! errors you might want to design your own error type instead. //! //! When the inner error is a [`PyErr`] without source, it will be extracted out. //! Otherwise a Python [`RuntimeError`] will be created. //! You might find that you need to map the error from your Rust code into another Python exception. //! See [`PyErr::new`] for more information about that. //! //! For information about error handling in general, see the [Error handling] chapter of the Rust //! book. //! //! # Setup //! //! To use this feature, add this to your **`Cargo.toml`**: //! //! ```toml //! [dependencies] //! ## change * to the version you want to use, ideally the latest. //! anyhow = "*" #![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"anyhow\"] }")] //! ``` //! //! Note that you must use compatible versions of anyhow and PyO3. //! The required anyhow version may vary based on the version of PyO3. //! //! # Example: Propagating a `PyErr` into [`anyhow::Error`] //! //! ```rust //! use pyo3::prelude::*; //! use pyo3::wrap_pyfunction; //! use std::path::PathBuf; //! //! // A wrapper around a Rust function. //! // The pyfunction macro performs the conversion to a PyErr //! #[pyfunction] //! fn py_open(filename: PathBuf) -> anyhow::Result> { //! let data = std::fs::read(filename)?; //! Ok(data) //! } //! //! fn main() { //! let error = Python::with_gil(|py| -> PyResult> { //! let fun = wrap_pyfunction!(py_open, py)?; //! let text = fun.call1(("foo.txt",))?.extract::>()?; //! Ok(text) //! }).unwrap_err(); //! //! println!("{}", error); //! } //! ``` //! //! # Example: Using `anyhow` in general //! //! Note that you don't need this feature to convert a [`PyErr`] into an [`anyhow::Error`], because //! it can already convert anything that implements [`Error`](std::error::Error): //! //! ```rust //! use pyo3::prelude::*; //! use pyo3::types::PyBytes; //! //! // An example function that must handle multiple error types. //! // //! // To do this you usually need to design your own error type or use //! // `Box`. `anyhow` is a convenient alternative for this. //! pub fn decompress(bytes: &[u8]) -> anyhow::Result { //! // An arbitrary example of a Python api you //! // could call inside an application... //! // This might return a `PyErr`. //! let res = Python::with_gil(|py| { //! let zlib = PyModule::import(py, "zlib")?; //! let decompress = zlib.getattr("decompress")?; //! let bytes = PyBytes::new(py, bytes); //! let value = decompress.call1((bytes,))?; //! value.extract::>() //! })?; //! //! // This might be a `FromUtf8Error`. //! let text = String::from_utf8(res)?; //! //! Ok(text) //! } //! //! fn main() -> anyhow::Result<()> { //! let bytes: &[u8] = b"x\x9c\x8b\xcc/U(\xce\xc8/\xcdIQ((\xcaOJL\xca\xa9T\ //! (-NU(\xc9HU\xc8\xc9LJ\xcbI,IUH.\x02\x91\x99y\xc5%\ //! \xa9\x89)z\x00\xf2\x15\x12\xfe"; //! let text = decompress(bytes)?; //! //! println!("The text is \"{}\"", text); //! # assert_eq!(text, "You should probably use the libflate crate instead."); //! Ok(()) //! } //! ``` //! //! [`RuntimeError`]: https://docs.python.org/3/library/exceptions.html#RuntimeError "Built-in Exceptions — Python documentation" //! [Error handling]: https://doc.rust-lang.org/book/ch09-02-recoverable-errors-with-result.html "Recoverable Errors with Result - The Rust Programming Language" use crate::exceptions::PyRuntimeError; use crate::PyErr; impl From for PyErr { fn from(mut error: anyhow::Error) -> Self { // Errors containing a PyErr without chain or context are returned as the underlying error if error.source().is_none() { error = match error.downcast::() { Ok(py_err) => return py_err, Err(error) => error, }; } PyRuntimeError::new_err(format!("{:?}", error)) } } #[cfg(test)] mod test_anyhow { use crate::exceptions::{PyRuntimeError, PyValueError}; use crate::prelude::*; use crate::types::IntoPyDict; use anyhow::{anyhow, bail, Context, Result}; fn f() -> Result<()> { use std::io; bail!(io::Error::new(io::ErrorKind::PermissionDenied, "oh no!")); } fn g() -> Result<()> { f().context("f failed") } fn h() -> Result<()> { g().context("g failed") } #[test] fn test_pyo3_exception_contents() { let err = h().unwrap_err(); let expected_contents = format!("{:?}", err); let pyerr = PyErr::from(err); Python::with_gil(|py| { let locals = [("err", pyerr)].into_py_dict(py); let pyerr = py.run("raise err", None, Some(locals)).unwrap_err(); assert_eq!(pyerr.value(py).to_string(), expected_contents); }) } fn k() -> Result<()> { Err(anyhow!("Some sort of error")) } #[test] fn test_pyo3_exception_contents2() { let err = k().unwrap_err(); let expected_contents = format!("{:?}", err); let pyerr = PyErr::from(err); Python::with_gil(|py| { let locals = [("err", pyerr)].into_py_dict(py); let pyerr = py.run("raise err", None, Some(locals)).unwrap_err(); assert_eq!(pyerr.value(py).to_string(), expected_contents); }) } #[test] fn test_pyo3_unwrap_simple_err() { let origin_exc = PyValueError::new_err("Value Error"); let err: anyhow::Error = origin_exc.into(); let converted: PyErr = err.into(); assert!(Python::with_gil( |py| converted.is_instance_of::(py) )) } #[test] fn test_pyo3_unwrap_complex_err() { let origin_exc = PyValueError::new_err("Value Error"); let mut err: anyhow::Error = origin_exc.into(); err = err.context("Context"); let converted: PyErr = err.into(); assert!(Python::with_gil( |py| converted.is_instance_of::(py) )) } } pyo3-0.20.2/src/conversions/chrono.rs000064400000000000000000001255651046102023000156120ustar 00000000000000#![cfg(all(feature = "chrono", not(Py_LIMITED_API)))] //! Conversions to and from [chrono](https://docs.rs/chrono/)’s `Duration`, //! `NaiveDate`, `NaiveTime`, `DateTime`, `FixedOffset`, and `Utc`. //! //! Unavailable with the `abi3` feature. //! //! # Setup //! //! To use this feature, add this to your **`Cargo.toml`**: //! //! ```toml //! [dependencies] //! # change * to the latest versions //! pyo3 = { version = "*", features = ["chrono"] } //! chrono = "0.4" #![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"chrono\"] }")] //! ``` //! //! Note that you must use compatible versions of chrono and PyO3. //! The required chrono version may vary based on the version of PyO3. //! //! # Example: Convert a `PyDateTime` to chrono's `DateTime` //! //! ```rust //! use chrono::{Utc, DateTime}; //! use pyo3::{Python, ToPyObject, types::PyDateTime}; //! //! fn main() { //! pyo3::prepare_freethreaded_python(); //! Python::with_gil(|py| { //! // Create an UTC datetime in python //! let py_tz = Utc.to_object(py); //! let py_tz = py_tz.downcast(py).unwrap(); //! let py_datetime = PyDateTime::new(py, 2022, 1, 1, 12, 0, 0, 0, Some(py_tz)).unwrap(); //! println!("PyDateTime: {}", py_datetime); //! // Now convert it to chrono's DateTime //! let chrono_datetime: DateTime = py_datetime.extract().unwrap(); //! println!("DateTime: {}", chrono_datetime); //! }); //! } //! ``` use crate::exceptions::{PyTypeError, PyUserWarning, PyValueError}; use crate::types::{ timezone_utc, PyDate, PyDateAccess, PyDateTime, PyDelta, PyDeltaAccess, PyTime, PyTimeAccess, PyTzInfo, PyTzInfoAccess, PyUnicode, }; use crate::{FromPyObject, IntoPy, PyAny, PyErr, PyObject, PyResult, Python, ToPyObject}; use chrono::offset::{FixedOffset, Utc}; use chrono::{ DateTime, Datelike, Duration, NaiveDate, NaiveDateTime, NaiveTime, Offset, TimeZone, Timelike, }; use pyo3_ffi::{PyDateTime_IMPORT, PyTimeZone_FromOffset}; use std::convert::TryInto; impl ToPyObject for Duration { fn to_object(&self, py: Python<'_>) -> PyObject { // Total number of days let days = self.num_days(); // Remainder of seconds let secs_dur = *self - Duration::days(days); // .try_into() converts i64 to i32, but this should never overflow // since it's at most the number of seconds per day let secs = secs_dur.num_seconds().try_into().unwrap(); // Fractional part of the microseconds let micros = (secs_dur - Duration::seconds(secs_dur.num_seconds())) .num_microseconds() // This should never panic since we are just getting the fractional // part of the total microseconds, which should never overflow. .unwrap() // Same for the conversion from i64 to i32 .try_into() .unwrap(); // We do not need to check i64 to i32 cast from rust because // python will panic with OverflowError. // We pass true as the `normalize` parameter since we'd need to do several checks here to // avoid that, and it shouldn't have a big performance impact. let delta = PyDelta::new(py, days.try_into().unwrap_or(i32::MAX), secs, micros, true) .expect("failed to construct delta"); delta.into() } } impl IntoPy for Duration { fn into_py(self, py: Python<'_>) -> PyObject { ToPyObject::to_object(&self, py) } } impl FromPyObject<'_> for Duration { fn extract(ob: &PyAny) -> PyResult { let delta: &PyDelta = ob.downcast()?; // Python size are much lower than rust size so we do not need bound checks. // 0 <= microseconds < 1000000 // 0 <= seconds < 3600*24 // -999999999 <= days <= 999999999 Ok(Duration::days(delta.get_days().into()) + Duration::seconds(delta.get_seconds().into()) + Duration::microseconds(delta.get_microseconds().into())) } } impl ToPyObject for NaiveDate { fn to_object(&self, py: Python<'_>) -> PyObject { (*self).into_py(py) } } impl IntoPy for NaiveDate { fn into_py(self, py: Python<'_>) -> PyObject { let DateArgs { year, month, day } = self.into(); PyDate::new(py, year, month, day) .expect("failed to construct date") .into() } } impl FromPyObject<'_> for NaiveDate { fn extract(ob: &PyAny) -> PyResult { let date: &PyDate = ob.downcast()?; py_date_to_naive_date(date) } } impl ToPyObject for NaiveTime { fn to_object(&self, py: Python<'_>) -> PyObject { (*self).into_py(py) } } impl IntoPy for NaiveTime { fn into_py(self, py: Python<'_>) -> PyObject { let TimeArgs { hour, min, sec, micro, truncated_leap_second, } = self.into(); let time = PyTime::new(py, hour, min, sec, micro, None).expect("Failed to construct time"); if truncated_leap_second { warn_truncated_leap_second(time); } time.into() } } impl FromPyObject<'_> for NaiveTime { fn extract(ob: &PyAny) -> PyResult { let time: &PyTime = ob.downcast()?; py_time_to_naive_time(time) } } impl ToPyObject for NaiveDateTime { fn to_object(&self, py: Python<'_>) -> PyObject { naive_datetime_to_py_datetime(py, self, None) .expect("failed to construct datetime") .into() } } impl IntoPy for NaiveDateTime { fn into_py(self, py: Python<'_>) -> PyObject { ToPyObject::to_object(&self, py) } } impl FromPyObject<'_> for NaiveDateTime { fn extract(ob: &PyAny) -> PyResult { let dt: &PyDateTime = ob.downcast()?; // If the user tries to convert a timezone aware datetime into a naive one, // we return a hard error. We could silently remove tzinfo, or assume local timezone // and do a conversion, but better leave this decision to the user of the library. if dt.get_tzinfo().is_some() { return Err(PyTypeError::new_err("expected a datetime without tzinfo")); } let dt = NaiveDateTime::new(py_date_to_naive_date(dt)?, py_time_to_naive_time(dt)?); Ok(dt) } } impl ToPyObject for DateTime { fn to_object(&self, py: Python<'_>) -> PyObject { // FIXME: convert to better timezone representation here than just convert to fixed offset // See https://github.com/PyO3/pyo3/issues/3266 let tz = self.offset().fix().to_object(py); let tz = tz.downcast(py).unwrap(); naive_datetime_to_py_datetime(py, &self.naive_local(), Some(tz)) .expect("failed to construct datetime") .into() } } impl IntoPy for DateTime { fn into_py(self, py: Python<'_>) -> PyObject { ToPyObject::to_object(&self, py) } } impl FromPyObject<'_> for DateTime { fn extract(ob: &PyAny) -> PyResult> { let dt: &PyDateTime = ob.downcast()?; let tz: FixedOffset = if let Some(tzinfo) = dt.get_tzinfo() { tzinfo.extract()? } else { return Err(PyTypeError::new_err( "expected a datetime with non-None tzinfo", )); }; let dt = NaiveDateTime::new(py_date_to_naive_date(dt)?, py_time_to_naive_time(dt)?); // `FixedOffset` cannot have ambiguities so we don't have to worry about DST folds and such Ok(dt.and_local_timezone(tz).unwrap()) } } impl FromPyObject<'_> for DateTime { fn extract(ob: &PyAny) -> PyResult> { let dt: &PyDateTime = ob.downcast()?; let _: Utc = if let Some(tzinfo) = dt.get_tzinfo() { tzinfo.extract()? } else { return Err(PyTypeError::new_err( "expected a datetime with non-None tzinfo", )); }; let dt = NaiveDateTime::new(py_date_to_naive_date(dt)?, py_time_to_naive_time(dt)?); Ok(dt.and_utc()) } } // Utility function used to convert PyDelta to timezone fn py_timezone_from_offset<'a>(py: &Python<'a>, td: &PyDelta) -> &'a PyAny { // Safety: py.from_owned_ptr needs the cast to be valid. // Since we are forcing a &PyDelta as input, the cast should always be valid. unsafe { PyDateTime_IMPORT(); py.from_owned_ptr(PyTimeZone_FromOffset(td.as_ptr())) } } impl ToPyObject for FixedOffset { fn to_object(&self, py: Python<'_>) -> PyObject { let seconds_offset = self.local_minus_utc(); let td = PyDelta::new(py, 0, seconds_offset, 0, true).expect("failed to construct timedelta"); py_timezone_from_offset(&py, td).into() } } impl IntoPy for FixedOffset { fn into_py(self, py: Python<'_>) -> PyObject { ToPyObject::to_object(&self, py) } } impl FromPyObject<'_> for FixedOffset { /// Convert python tzinfo to rust [`FixedOffset`]. /// /// Note that the conversion will result in precision lost in microseconds as chrono offset /// does not supports microseconds. fn extract(ob: &PyAny) -> PyResult { let py_tzinfo: &PyTzInfo = ob.downcast()?; // Passing `ob.py().None()` (so Python's None) to the `utcoffset` function will only // work for timezones defined as fixed offsets in Python. // Any other timezone would require a datetime as the parameter, and return // None if the datetime is not provided. // Trying to convert None to a PyDelta in the next line will then fail. let py_timedelta = py_tzinfo.call_method1("utcoffset", (ob.py().None(),))?; let py_timedelta: &PyDelta = py_timedelta.downcast().map_err(|_| { PyTypeError::new_err(format!( "{:?} is not a fixed offset timezone", py_tzinfo .repr() .unwrap_or_else(|_| PyUnicode::new(ob.py(), "repr failed")) )) })?; let days = py_timedelta.get_days() as i64; let seconds = py_timedelta.get_seconds() as i64; // Here we won't get microseconds as noted before // let microseconds = py_timedelta.get_microseconds() as i64; let total_seconds = Duration::days(days) + Duration::seconds(seconds); // This cast is safe since the timedelta is limited to -24 hours and 24 hours. let total_seconds = total_seconds.num_seconds() as i32; FixedOffset::east_opt(total_seconds) .ok_or_else(|| PyValueError::new_err("fixed offset out of bounds")) } } impl ToPyObject for Utc { fn to_object(&self, py: Python<'_>) -> PyObject { timezone_utc(py).to_object(py) } } impl IntoPy for Utc { fn into_py(self, py: Python<'_>) -> PyObject { ToPyObject::to_object(&self, py) } } impl FromPyObject<'_> for Utc { fn extract(ob: &PyAny) -> PyResult { let py_tzinfo: &PyTzInfo = ob.downcast()?; let py_utc = timezone_utc(ob.py()); if py_tzinfo.eq(py_utc)? { Ok(Utc) } else { Err(PyValueError::new_err("expected datetime.timezone.utc")) } } } struct DateArgs { year: i32, month: u8, day: u8, } impl From for DateArgs { fn from(value: NaiveDate) -> Self { Self { year: value.year(), month: value.month() as u8, day: value.day() as u8, } } } struct TimeArgs { hour: u8, min: u8, sec: u8, micro: u32, truncated_leap_second: bool, } impl From for TimeArgs { fn from(value: NaiveTime) -> Self { let ns = value.nanosecond(); let checked_sub = ns.checked_sub(1_000_000_000); let truncated_leap_second = checked_sub.is_some(); let micro = checked_sub.unwrap_or(ns) / 1000; Self { hour: value.hour() as u8, min: value.minute() as u8, sec: value.second() as u8, micro, truncated_leap_second, } } } fn naive_datetime_to_py_datetime<'py>( py: Python<'py>, naive_datetime: &NaiveDateTime, tzinfo: Option<&PyTzInfo>, ) -> PyResult<&'py PyDateTime> { let DateArgs { year, month, day } = naive_datetime.date().into(); let TimeArgs { hour, min, sec, micro, truncated_leap_second, } = naive_datetime.time().into(); let datetime = PyDateTime::new(py, year, month, day, hour, min, sec, micro, tzinfo)?; if truncated_leap_second { warn_truncated_leap_second(datetime); } Ok(datetime) } fn warn_truncated_leap_second(obj: &PyAny) { let py = obj.py(); if let Err(e) = PyErr::warn( py, py.get_type::(), "ignored leap-second, `datetime` does not support leap-seconds", 0, ) { e.write_unraisable(py, Some(obj)) }; } fn py_date_to_naive_date(py_date: &impl PyDateAccess) -> PyResult { NaiveDate::from_ymd_opt( py_date.get_year(), py_date.get_month().into(), py_date.get_day().into(), ) .ok_or_else(|| PyValueError::new_err("invalid or out-of-range date")) } fn py_time_to_naive_time(py_time: &impl PyTimeAccess) -> PyResult { NaiveTime::from_hms_micro_opt( py_time.get_hour().into(), py_time.get_minute().into(), py_time.get_second().into(), py_time.get_microsecond(), ) .ok_or_else(|| PyValueError::new_err("invalid or out-of-range time")) } #[cfg(test)] mod tests { use std::{cmp::Ordering, panic}; use crate::{tests::common::CatchWarnings, PyTypeInfo}; use super::*; #[test] // Only Python>=3.9 has the zoneinfo package // We skip the test on windows too since we'd need to install // tzdata there to make this work. #[cfg(all(Py_3_9, not(target_os = "windows")))] fn test_zoneinfo_is_not_fixed_offset() { Python::with_gil(|py| { let locals = crate::types::PyDict::new(py); py.run( "import zoneinfo; zi = zoneinfo.ZoneInfo('Europe/London')", None, Some(locals), ) .unwrap(); let result: PyResult = locals.get_item("zi").unwrap().unwrap().extract(); assert!(result.is_err()); let res = result.err().unwrap(); // Also check the error message is what we expect let msg = res.value(py).repr().unwrap().to_string(); assert_eq!(msg, "TypeError('\"zoneinfo.ZoneInfo(key=\\'Europe/London\\')\" is not a fixed offset timezone')"); }); } #[test] fn test_timezone_aware_to_naive_fails() { // Test that if a user tries to convert a python's timezone aware datetime into a naive // one, the conversion fails. Python::with_gil(|py| { let utc = timezone_utc(py); let py_datetime = PyDateTime::new(py, 2022, 1, 1, 1, 0, 0, 0, Some(utc)).unwrap(); // Now test that converting a PyDateTime with tzinfo to a NaiveDateTime fails let res: PyResult = py_datetime.extract(); assert!(res.is_err()); let res = res.err().unwrap(); // Also check the error message is what we expect let msg = res.value(py).repr().unwrap().to_string(); assert_eq!(msg, "TypeError('expected a datetime without tzinfo')"); }); } #[test] fn test_naive_to_timezone_aware_fails() { // Test that if a user tries to convert a python's timezone aware datetime into a naive // one, the conversion fails. Python::with_gil(|py| { let py_datetime = PyDateTime::new(py, 2022, 1, 1, 1, 0, 0, 0, None).unwrap(); // Now test that converting a PyDateTime with tzinfo to a NaiveDateTime fails let res: PyResult> = py_datetime.extract(); assert!(res.is_err()); let res = res.err().unwrap(); // Also check the error message is what we expect let msg = res.value(py).repr().unwrap().to_string(); assert_eq!(msg, "TypeError('expected a datetime with non-None tzinfo')"); // Now test that converting a PyDateTime with tzinfo to a NaiveDateTime fails let res: PyResult> = py_datetime.extract(); assert!(res.is_err()); let res = res.err().unwrap(); // Also check the error message is what we expect let msg = res.value(py).repr().unwrap().to_string(); assert_eq!(msg, "TypeError('expected a datetime with non-None tzinfo')"); }); } #[test] fn test_invalid_types_fail() { // Test that if a user tries to convert a python's timezone aware datetime into a naive // one, the conversion fails. Python::with_gil(|py| { let none = py.None().into_ref(py); assert_eq!( none.extract::().unwrap_err().to_string(), "TypeError: 'NoneType' object cannot be converted to 'PyDelta'" ); assert_eq!( none.extract::().unwrap_err().to_string(), "TypeError: 'NoneType' object cannot be converted to 'PyTzInfo'" ); assert_eq!( none.extract::().unwrap_err().to_string(), "TypeError: 'NoneType' object cannot be converted to 'PyTzInfo'" ); assert_eq!( none.extract::().unwrap_err().to_string(), "TypeError: 'NoneType' object cannot be converted to 'PyTime'" ); assert_eq!( none.extract::().unwrap_err().to_string(), "TypeError: 'NoneType' object cannot be converted to 'PyDate'" ); assert_eq!( none.extract::().unwrap_err().to_string(), "TypeError: 'NoneType' object cannot be converted to 'PyDateTime'" ); assert_eq!( none.extract::>().unwrap_err().to_string(), "TypeError: 'NoneType' object cannot be converted to 'PyDateTime'" ); assert_eq!( none.extract::>() .unwrap_err() .to_string(), "TypeError: 'NoneType' object cannot be converted to 'PyDateTime'" ); }); } #[test] fn test_pyo3_timedelta_topyobject() { // Utility function used to check different durations. // The `name` parameter is used to identify the check in case of a failure. let check = |name: &'static str, delta: Duration, py_days, py_seconds, py_ms| { Python::with_gil(|py| { let delta = delta.to_object(py); let delta: &PyDelta = delta.extract(py).unwrap(); let py_delta = PyDelta::new(py, py_days, py_seconds, py_ms, true).unwrap(); assert!( delta.eq(py_delta).unwrap(), "{}: {} != {}", name, delta, py_delta ); }); }; let delta = Duration::days(-1) + Duration::seconds(1) + Duration::microseconds(-10); check("delta normalization", delta, -1, 1, -10); // Check the minimum value allowed by PyDelta, which is different // from the minimum value allowed in Duration. This should pass. let delta = Duration::seconds(-86399999913600); // min check("delta min value", delta, -999999999, 0, 0); // Same, for max value let delta = Duration::seconds(86399999999999) + Duration::nanoseconds(999999000); // max check("delta max value", delta, 999999999, 86399, 999999); // Also check that trying to convert an out of bound value panics. Python::with_gil(|py| { assert!(panic::catch_unwind(|| Duration::min_value().to_object(py)).is_err()); assert!(panic::catch_unwind(|| Duration::max_value().to_object(py)).is_err()); }); } #[test] fn test_pyo3_timedelta_frompyobject() { // Utility function used to check different durations. // The `name` parameter is used to identify the check in case of a failure. let check = |name: &'static str, delta: Duration, py_days, py_seconds, py_ms| { Python::with_gil(|py| { let py_delta = PyDelta::new(py, py_days, py_seconds, py_ms, true).unwrap(); let py_delta: Duration = py_delta.extract().unwrap(); assert_eq!(py_delta, delta, "{}: {} != {}", name, py_delta, delta); }) }; // Check the minimum value allowed by PyDelta, which is different // from the minimum value allowed in Duration. This should pass. check( "min py_delta value", Duration::seconds(-86399999913600), -999999999, 0, 0, ); // Same, for max value check( "max py_delta value", Duration::seconds(86399999999999) + Duration::microseconds(999999), 999999999, 86399, 999999, ); // This check is to assert that we can't construct every possible Duration from a PyDelta // since they have different bounds. Python::with_gil(|py| { let low_days: i32 = -1000000000; // This is possible assert!(panic::catch_unwind(|| Duration::days(low_days as i64)).is_ok()); // This panics on PyDelta::new assert!(panic::catch_unwind(|| { let py_delta = PyDelta::new(py, low_days, 0, 0, true).unwrap(); if let Ok(_duration) = py_delta.extract::() { // So we should never get here } }) .is_err()); let high_days: i32 = 1000000000; // This is possible assert!(panic::catch_unwind(|| Duration::days(high_days as i64)).is_ok()); // This panics on PyDelta::new assert!(panic::catch_unwind(|| { let py_delta = PyDelta::new(py, high_days, 0, 0, true).unwrap(); if let Ok(_duration) = py_delta.extract::() { // So we should never get here } }) .is_err()); }); } #[test] fn test_pyo3_date_topyobject() { let eq_ymd = |name: &'static str, year, month, day| { Python::with_gil(|py| { let date = NaiveDate::from_ymd_opt(year, month, day) .unwrap() .to_object(py); let date: &PyDate = date.extract(py).unwrap(); let py_date = PyDate::new(py, year, month as u8, day as u8).unwrap(); assert_eq!( date.compare(py_date).unwrap(), Ordering::Equal, "{}: {} != {}", name, date, py_date ); }) }; eq_ymd("past date", 2012, 2, 29); eq_ymd("min date", 1, 1, 1); eq_ymd("future date", 3000, 6, 5); eq_ymd("max date", 9999, 12, 31); } #[test] fn test_pyo3_date_frompyobject() { let eq_ymd = |name: &'static str, year, month, day| { Python::with_gil(|py| { let py_date = PyDate::new(py, year, month as u8, day as u8).unwrap(); let py_date: NaiveDate = py_date.extract().unwrap(); let date = NaiveDate::from_ymd_opt(year, month, day).unwrap(); assert_eq!(py_date, date, "{}: {} != {}", name, date, py_date); }) }; eq_ymd("past date", 2012, 2, 29); eq_ymd("min date", 1, 1, 1); eq_ymd("future date", 3000, 6, 5); eq_ymd("max date", 9999, 12, 31); } #[test] fn test_pyo3_datetime_topyobject_utc() { Python::with_gil(|py| { let check_utc = |name: &'static str, year, month, day, hour, minute, second, ms, py_ms| { let datetime = NaiveDate::from_ymd_opt(year, month, day) .unwrap() .and_hms_micro_opt(hour, minute, second, ms) .unwrap() .and_utc(); let datetime = datetime.to_object(py); let datetime: &PyDateTime = datetime.extract(py).unwrap(); let py_tz = timezone_utc(py); let py_datetime = PyDateTime::new( py, year, month as u8, day as u8, hour as u8, minute as u8, second as u8, py_ms, Some(py_tz), ) .unwrap(); assert_eq!( datetime.compare(py_datetime).unwrap(), Ordering::Equal, "{}: {} != {}", name, datetime, py_datetime ); }; check_utc("regular", 2014, 5, 6, 7, 8, 9, 999_999, 999_999); assert_warnings!( py, check_utc("leap second", 2014, 5, 6, 7, 8, 59, 1_999_999, 999_999), [( PyUserWarning, "ignored leap-second, `datetime` does not support leap-seconds" )] ); }) } #[test] fn test_pyo3_datetime_topyobject_fixed_offset() { Python::with_gil(|py| { let check_fixed_offset = |name: &'static str, year, month, day, hour, minute, second, ms, py_ms| { let offset = FixedOffset::east_opt(3600).unwrap(); let datetime = NaiveDate::from_ymd_opt(year, month, day) .unwrap() .and_hms_micro_opt(hour, minute, second, ms) .unwrap() .and_local_timezone(offset) .unwrap(); let datetime = datetime.to_object(py); let datetime: &PyDateTime = datetime.extract(py).unwrap(); let py_tz = offset.to_object(py); let py_tz = py_tz.downcast(py).unwrap(); let py_datetime = PyDateTime::new( py, year, month as u8, day as u8, hour as u8, minute as u8, second as u8, py_ms, Some(py_tz), ) .unwrap(); assert_eq!( datetime.compare(py_datetime).unwrap(), Ordering::Equal, "{}: {} != {}", name, datetime, py_datetime ); }; check_fixed_offset("regular", 2014, 5, 6, 7, 8, 9, 999_999, 999_999); assert_warnings!( py, check_fixed_offset("leap second", 2014, 5, 6, 7, 8, 59, 1_999_999, 999_999), [( PyUserWarning, "ignored leap-second, `datetime` does not support leap-seconds" )] ); }) } #[test] fn test_pyo3_datetime_frompyobject_utc() { Python::with_gil(|py| { let year = 2014; let month = 5; let day = 6; let hour = 7; let minute = 8; let second = 9; let micro = 999_999; let py_tz = timezone_utc(py); let py_datetime = PyDateTime::new( py, year, month, day, hour, minute, second, micro, Some(py_tz), ) .unwrap(); let py_datetime: DateTime = py_datetime.extract().unwrap(); let datetime = NaiveDate::from_ymd_opt(year, month.into(), day.into()) .unwrap() .and_hms_micro_opt(hour.into(), minute.into(), second.into(), micro) .unwrap() .and_utc(); assert_eq!(py_datetime, datetime,); }) } #[test] fn test_pyo3_datetime_frompyobject_fixed_offset() { Python::with_gil(|py| { let year = 2014; let month = 5; let day = 6; let hour = 7; let minute = 8; let second = 9; let micro = 999_999; let offset = FixedOffset::east_opt(3600).unwrap(); let py_tz = offset.to_object(py); let py_tz = py_tz.downcast(py).unwrap(); let py_datetime = PyDateTime::new( py, year, month, day, hour, minute, second, micro, Some(py_tz), ) .unwrap(); let datetime_from_py: DateTime = py_datetime.extract().unwrap(); let datetime = NaiveDate::from_ymd_opt(year, month.into(), day.into()) .unwrap() .and_hms_micro_opt(hour.into(), minute.into(), second.into(), micro) .unwrap(); let datetime = datetime.and_local_timezone(offset).unwrap(); assert_eq!(datetime_from_py, datetime); assert!( py_datetime.extract::>().is_err(), "Extracting Utc from nonzero FixedOffset timezone will fail" ); let utc = timezone_utc(py); let py_datetime_utc = PyDateTime::new(py, year, month, day, hour, minute, second, micro, Some(utc)) .unwrap(); assert!( py_datetime_utc.extract::>().is_ok(), "Extracting FixedOffset from Utc timezone will succeed" ); }) } #[test] fn test_pyo3_offset_fixed_topyobject() { Python::with_gil(|py| { // Chrono offset let offset = FixedOffset::east_opt(3600).unwrap().to_object(py); // Python timezone from timedelta let td = PyDelta::new(py, 0, 3600, 0, true).unwrap(); let py_timedelta = py_timezone_from_offset(&py, td); // Should be equal assert!(offset.as_ref(py).eq(py_timedelta).unwrap()); // Same but with negative values let offset = FixedOffset::east_opt(-3600).unwrap().to_object(py); let td = PyDelta::new(py, 0, -3600, 0, true).unwrap(); let py_timedelta = py_timezone_from_offset(&py, td); assert!(offset.as_ref(py).eq(py_timedelta).unwrap()); }) } #[test] fn test_pyo3_offset_fixed_frompyobject() { Python::with_gil(|py| { let py_timedelta = PyDelta::new(py, 0, 3600, 0, true).unwrap(); let py_tzinfo = py_timezone_from_offset(&py, py_timedelta); let offset: FixedOffset = py_tzinfo.extract().unwrap(); assert_eq!(FixedOffset::east_opt(3600).unwrap(), offset); }) } #[test] fn test_pyo3_offset_utc_topyobject() { Python::with_gil(|py| { let utc = Utc.to_object(py); let py_utc = timezone_utc(py); assert!(utc.as_ref(py).is(py_utc)); }) } #[test] fn test_pyo3_offset_utc_frompyobject() { Python::with_gil(|py| { let py_utc = timezone_utc(py); let py_utc: Utc = py_utc.extract().unwrap(); assert_eq!(Utc, py_utc); let py_timedelta = PyDelta::new(py, 0, 0, 0, true).unwrap(); let py_timezone_utc = py_timezone_from_offset(&py, py_timedelta); let py_timezone_utc: Utc = py_timezone_utc.extract().unwrap(); assert_eq!(Utc, py_timezone_utc); let py_timedelta = PyDelta::new(py, 0, 3600, 0, true).unwrap(); let py_timezone = py_timezone_from_offset(&py, py_timedelta); assert!(py_timezone.extract::().is_err()); }) } #[test] fn test_pyo3_time_topyobject() { Python::with_gil(|py| { let check_time = |name: &'static str, hour, minute, second, ms, py_ms| { let time = NaiveTime::from_hms_micro_opt(hour, minute, second, ms) .unwrap() .to_object(py); let time: &PyTime = time.extract(py).unwrap(); let py_time = PyTime::new(py, hour as u8, minute as u8, second as u8, py_ms, None).unwrap(); assert!( time.eq(py_time).unwrap(), "{}: {} != {}", name, time, py_time ); }; check_time("regular", 3, 5, 7, 999_999, 999_999); assert_warnings!( py, check_time("leap second", 3, 5, 59, 1_999_999, 999_999), [( PyUserWarning, "ignored leap-second, `datetime` does not support leap-seconds" )] ); }) } #[test] fn test_pyo3_time_frompyobject() { let hour = 3; let minute = 5; let second = 7; let micro = 999_999; Python::with_gil(|py| { let py_time = PyTime::new(py, hour as u8, minute as u8, second as u8, micro, None).unwrap(); let py_time: NaiveTime = py_time.extract().unwrap(); let time = NaiveTime::from_hms_micro_opt(hour, minute, second, micro).unwrap(); assert_eq!(py_time, time); }) } #[cfg(all(test, not(target_arch = "wasm32")))] mod proptests { use super::*; use crate::types::IntoPyDict; use proptest::prelude::*; proptest! { // Range is limited to 1970 to 2038 due to windows limitations #[test] fn test_pyo3_offset_fixed_frompyobject_created_in_python(timestamp in 0..(i32::MAX as i64), timedelta in -86399i32..=86399i32) { Python::with_gil(|py| { let globals = [("datetime", py.import("datetime").unwrap())].into_py_dict(py); let code = format!("datetime.datetime.fromtimestamp({}).replace(tzinfo=datetime.timezone(datetime.timedelta(seconds={})))", timestamp, timedelta); let t = py.eval(&code, Some(globals), None).unwrap(); // Get ISO 8601 string from python let py_iso_str = t.call_method0("isoformat").unwrap(); // Get ISO 8601 string from rust let t = t.extract::>().unwrap(); // Python doesn't print the seconds of the offset if they are 0 let rust_iso_str = if timedelta % 60 == 0 { t.format("%Y-%m-%dT%H:%M:%S%:z").to_string() } else { t.format("%Y-%m-%dT%H:%M:%S%::z").to_string() }; // They should be equal assert_eq!(py_iso_str.to_string(), rust_iso_str); }) } #[test] fn test_duration_roundtrip(days in -999999999i64..=999999999i64) { // Test roundtrip conversion rust->python->rust for all allowed // python values of durations (from -999999999 to 999999999 days), Python::with_gil(|py| { let dur = Duration::days(days); let py_delta = dur.into_py(py); let roundtripped: Duration = py_delta.extract(py).expect("Round trip"); assert_eq!(dur, roundtripped); }) } #[test] fn test_fixed_offset_roundtrip(secs in -86399i32..=86399i32) { Python::with_gil(|py| { let offset = FixedOffset::east_opt(secs).unwrap(); let py_offset = offset.into_py(py); let roundtripped: FixedOffset = py_offset.extract(py).expect("Round trip"); assert_eq!(offset, roundtripped); }) } #[test] fn test_naive_date_roundtrip( year in 1i32..=9999i32, month in 1u32..=12u32, day in 1u32..=31u32 ) { // Test roundtrip conversion rust->python->rust for all allowed // python dates (from year 1 to year 9999) Python::with_gil(|py| { // We use to `from_ymd_opt` constructor so that we only test valid `NaiveDate`s. // This is to skip the test if we are creating an invalid date, like February 31. if let Some(date) = NaiveDate::from_ymd_opt(year, month, day) { let py_date = date.to_object(py); let roundtripped: NaiveDate = py_date.extract(py).expect("Round trip"); assert_eq!(date, roundtripped); } }) } #[test] fn test_naive_time_roundtrip( hour in 0u32..=23u32, min in 0u32..=59u32, sec in 0u32..=59u32, micro in 0u32..=1_999_999u32 ) { // Test roundtrip conversion rust->python->rust for naive times. // Python time has a resolution of microseconds, so we only test // NaiveTimes with microseconds resolution, even if NaiveTime has nanosecond // resolution. Python::with_gil(|py| { if let Some(time) = NaiveTime::from_hms_micro_opt(hour, min, sec, micro) { // Wrap in CatchWarnings to avoid to_object firing warning for truncated leap second let py_time = CatchWarnings::enter(py, |_| Ok(time.to_object(py))).unwrap(); let roundtripped: NaiveTime = py_time.extract(py).expect("Round trip"); // Leap seconds are not roundtripped let expected_roundtrip_time = micro.checked_sub(1_000_000).map(|micro| NaiveTime::from_hms_micro_opt(hour, min, sec, micro).unwrap()).unwrap_or(time); assert_eq!(expected_roundtrip_time, roundtripped); } }) } #[test] fn test_naive_datetime_roundtrip( year in 1i32..=9999i32, month in 1u32..=12u32, day in 1u32..=31u32, hour in 0u32..=24u32, min in 0u32..=60u32, sec in 0u32..=60u32, micro in 0u32..=999_999u32 ) { Python::with_gil(|py| { let date_opt = NaiveDate::from_ymd_opt(year, month, day); let time_opt = NaiveTime::from_hms_micro_opt(hour, min, sec, micro); if let (Some(date), Some(time)) = (date_opt, time_opt) { let dt = NaiveDateTime::new(date, time); let pydt = dt.to_object(py); let roundtripped: NaiveDateTime = pydt.extract(py).expect("Round trip"); assert_eq!(dt, roundtripped); } }) } #[test] fn test_utc_datetime_roundtrip( year in 1i32..=9999i32, month in 1u32..=12u32, day in 1u32..=31u32, hour in 0u32..=23u32, min in 0u32..=59u32, sec in 0u32..=59u32, micro in 0u32..=1_999_999u32 ) { Python::with_gil(|py| { let date_opt = NaiveDate::from_ymd_opt(year, month, day); let time_opt = NaiveTime::from_hms_micro_opt(hour, min, sec, micro); if let (Some(date), Some(time)) = (date_opt, time_opt) { let dt: DateTime = NaiveDateTime::new(date, time).and_utc(); // Wrap in CatchWarnings to avoid into_py firing warning for truncated leap second let py_dt = CatchWarnings::enter(py, |_| Ok(dt.into_py(py))).unwrap(); let roundtripped: DateTime = py_dt.extract(py).expect("Round trip"); // Leap seconds are not roundtripped let expected_roundtrip_time = micro.checked_sub(1_000_000).map(|micro| NaiveTime::from_hms_micro_opt(hour, min, sec, micro).unwrap()).unwrap_or(time); let expected_roundtrip_dt: DateTime = NaiveDateTime::new(date, expected_roundtrip_time).and_utc(); assert_eq!(expected_roundtrip_dt, roundtripped); } }) } #[test] fn test_fixed_offset_datetime_roundtrip( year in 1i32..=9999i32, month in 1u32..=12u32, day in 1u32..=31u32, hour in 0u32..=23u32, min in 0u32..=59u32, sec in 0u32..=59u32, micro in 0u32..=1_999_999u32, offset_secs in -86399i32..=86399i32 ) { Python::with_gil(|py| { let date_opt = NaiveDate::from_ymd_opt(year, month, day); let time_opt = NaiveTime::from_hms_micro_opt(hour, min, sec, micro); let offset = FixedOffset::east_opt(offset_secs).unwrap(); if let (Some(date), Some(time)) = (date_opt, time_opt) { let dt: DateTime = NaiveDateTime::new(date, time).and_local_timezone(offset).unwrap(); // Wrap in CatchWarnings to avoid into_py firing warning for truncated leap second let py_dt = CatchWarnings::enter(py, |_| Ok(dt.into_py(py))).unwrap(); let roundtripped: DateTime = py_dt.extract(py).expect("Round trip"); // Leap seconds are not roundtripped let expected_roundtrip_time = micro.checked_sub(1_000_000).map(|micro| NaiveTime::from_hms_micro_opt(hour, min, sec, micro).unwrap()).unwrap_or(time); let expected_roundtrip_dt: DateTime = NaiveDateTime::new(date, expected_roundtrip_time).and_local_timezone(offset).unwrap(); assert_eq!(expected_roundtrip_dt, roundtripped); } }) } } } } pyo3-0.20.2/src/conversions/either.rs000064400000000000000000000112441046102023000155660ustar 00000000000000#![cfg(feature = "either")] //! Conversion to/from //! [either](https://docs.rs/either/ "A library for easy idiomatic error handling and reporting in Rust applications")’s //! [`Either`] type to a union of two Python types. //! //! Use of a generic sum type like [either] is common when you want to either accept one of two possible //! types as an argument or return one of two possible types from a function, without having to define //! a helper type manually yourself. //! //! # Setup //! //! To use this feature, add this to your **`Cargo.toml`**: //! //! ```toml //! [dependencies] //! ## change * to the version you want to use, ideally the latest. //! either = "*" #![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"either\"] }")] //! ``` //! //! Note that you must use compatible versions of either and PyO3. //! The required either version may vary based on the version of PyO3. //! //! # Example: Convert a `int | str` to `Either`. //! //! ```rust //! use either::Either; //! use pyo3::{Python, ToPyObject}; //! //! fn main() { //! pyo3::prepare_freethreaded_python(); //! Python::with_gil(|py| { //! // Create a string and an int in Python. //! let py_str = "crab".to_object(py); //! let py_int = 42.to_object(py); //! // Now convert it to an Either. //! let either_str: Either = py_str.extract(py).unwrap(); //! let either_int: Either = py_int.extract(py).unwrap(); //! }); //! } //! ``` //! //! [either](https://docs.rs/either/ "A library for easy idiomatic error handling and reporting in Rust applications")’s #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{ exceptions::PyTypeError, FromPyObject, IntoPy, PyAny, PyObject, PyResult, Python, ToPyObject, }; use either::Either; #[cfg_attr(docsrs, doc(cfg(feature = "either")))] impl IntoPy for Either where L: IntoPy, R: IntoPy, { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { match self { Either::Left(l) => l.into_py(py), Either::Right(r) => r.into_py(py), } } } #[cfg_attr(docsrs, doc(cfg(feature = "either")))] impl ToPyObject for Either where L: ToPyObject, R: ToPyObject, { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { match self { Either::Left(l) => l.to_object(py), Either::Right(r) => r.to_object(py), } } } #[cfg_attr(docsrs, doc(cfg(feature = "either")))] impl<'source, L, R> FromPyObject<'source> for Either where L: FromPyObject<'source>, R: FromPyObject<'source>, { #[inline] fn extract(obj: &'source PyAny) -> PyResult { if let Ok(l) = obj.extract::() { Ok(Either::Left(l)) } else if let Ok(r) = obj.extract::() { Ok(Either::Right(r)) } else { let err_msg = format!("failed to convert the value to '{}'", Self::type_input()); Err(PyTypeError::new_err(err_msg)) } } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { TypeInfo::union_of(&[L::type_input(), R::type_input()]) } } #[cfg(test)] mod tests { use crate::exceptions::PyTypeError; use crate::{Python, ToPyObject}; use either::Either; #[test] fn test_either_conversion() { type E = Either; type E1 = Either; type E2 = Either; Python::with_gil(|py| { let l = E::Left(42); let obj_l = l.to_object(py); assert_eq!(obj_l.extract::(py).unwrap(), 42); assert_eq!(obj_l.extract::(py).unwrap(), l); let r = E::Right("foo".to_owned()); let obj_r = r.to_object(py); assert_eq!(obj_r.extract::<&str>(py).unwrap(), "foo"); assert_eq!(obj_r.extract::(py).unwrap(), r); let obj_s = "foo".to_object(py); let err = obj_s.extract::(py).unwrap_err(); assert!(err.is_instance_of::(py)); assert_eq!( err.to_string(), "TypeError: failed to convert the value to 'Union[int, float]'" ); let obj_i = 42.to_object(py); assert_eq!(obj_i.extract::(py).unwrap(), E1::Left(42)); assert_eq!(obj_i.extract::(py).unwrap(), E2::Left(42.0)); let obj_f = 42.0.to_object(py); assert_eq!(obj_f.extract::(py).unwrap(), E1::Right(42.0)); assert_eq!(obj_f.extract::(py).unwrap(), E2::Left(42.0)); }); } } pyo3-0.20.2/src/conversions/eyre.rs000064400000000000000000000154171046102023000152600ustar 00000000000000#![cfg(feature = "eyre")] //! A conversion from //! [eyre](https://docs.rs/eyre/ "A library for easy idiomatic error handling and reporting in Rust applications.")’s //! [`Report`] type to [`PyErr`]. //! //! Use of an error handling library like [eyre] is common in application code and when you just //! want error handling to be easy. If you are writing a library or you need more control over your //! errors you might want to design your own error type instead. //! //! When the inner error is a [`PyErr`] without source, it will be extracted out. //! Otherwise a Python [`RuntimeError`] will be created. //! You might find that you need to map the error from your Rust code into another Python exception. //! See [`PyErr::new`] for more information about that. //! //! For information about error handling in general, see the [Error handling] chapter of the Rust //! book. //! //! # Setup //! //! To use this feature, add this to your **`Cargo.toml`**: //! //! ```toml //! [dependencies] //! ## change * to the version you want to use, ideally the latest. //! eyre = "*" #![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"eyre\"] }")] //! ``` //! //! Note that you must use compatible versions of eyre and PyO3. //! The required eyre version may vary based on the version of PyO3. //! //! # Example: Propagating a `PyErr` into [`eyre::Report`] //! //! ```rust //! use pyo3::prelude::*; //! use pyo3::wrap_pyfunction; //! use std::path::PathBuf; //! //! // A wrapper around a Rust function. //! // The pyfunction macro performs the conversion to a PyErr //! #[pyfunction] //! fn py_open(filename: PathBuf) -> eyre::Result> { //! let data = std::fs::read(filename)?; //! Ok(data) //! } //! //! fn main() { //! let error = Python::with_gil(|py| -> PyResult> { //! let fun = wrap_pyfunction!(py_open, py)?; //! let text = fun.call1(("foo.txt",))?.extract::>()?; //! Ok(text) //! }).unwrap_err(); //! //! println!("{}", error); //! } //! ``` //! //! # Example: Using `eyre` in general //! //! Note that you don't need this feature to convert a [`PyErr`] into an [`eyre::Report`], because //! it can already convert anything that implements [`Error`](std::error::Error): //! //! ```rust //! use pyo3::prelude::*; //! use pyo3::types::PyBytes; //! //! // An example function that must handle multiple error types. //! // //! // To do this you usually need to design your own error type or use //! // `Box`. `eyre` is a convenient alternative for this. //! pub fn decompress(bytes: &[u8]) -> eyre::Result { //! // An arbitrary example of a Python api you //! // could call inside an application... //! // This might return a `PyErr`. //! let res = Python::with_gil(|py| { //! let zlib = PyModule::import(py, "zlib")?; //! let decompress = zlib.getattr("decompress")?; //! let bytes = PyBytes::new(py, bytes); //! let value = decompress.call1((bytes,))?; //! value.extract::>() //! })?; //! //! // This might be a `FromUtf8Error`. //! let text = String::from_utf8(res)?; //! //! Ok(text) //! } //! //! fn main() -> eyre::Result<()> { //! let bytes: &[u8] = b"x\x9c\x8b\xcc/U(\xce\xc8/\xcdIQ((\xcaOJL\xca\xa9T\ //! (-NU(\xc9HU\xc8\xc9LJ\xcbI,IUH.\x02\x91\x99y\xc5%\ //! \xa9\x89)z\x00\xf2\x15\x12\xfe"; //! let text = decompress(bytes)?; //! //! println!("The text is \"{}\"", text); //! # assert_eq!(text, "You should probably use the libflate crate instead."); //! Ok(()) //! } //! ``` //! //! [eyre]: https://docs.rs/eyre/ "A library for easy idiomatic error handling and reporting in Rust applications." //! [`RuntimeError`]: https://docs.python.org/3/library/exceptions.html#RuntimeError "Built-in Exceptions — Python documentation" //! [Error handling]: https://doc.rust-lang.org/book/ch09-02-recoverable-errors-with-result.html "Recoverable Errors with Result - The Rust Programming Language" use crate::exceptions::PyRuntimeError; use crate::PyErr; use eyre::Report; /// Converts [`eyre::Report`] to a [`PyErr`] containing a [`PyRuntimeError`]. /// /// If you want to raise a different Python exception you will have to do so manually. See /// [`PyErr::new`] for more information about that. impl From for PyErr { fn from(mut error: Report) -> Self { // Errors containing a PyErr without chain or context are returned as the underlying error if error.source().is_none() { error = match error.downcast::() { Ok(py_err) => return py_err, Err(error) => error, }; } PyRuntimeError::new_err(format!("{:?}", error)) } } #[cfg(test)] mod tests { use crate::exceptions::{PyRuntimeError, PyValueError}; use crate::prelude::*; use crate::types::IntoPyDict; use eyre::{bail, eyre, Report, Result, WrapErr}; fn f() -> Result<()> { use std::io; bail!(io::Error::new(io::ErrorKind::PermissionDenied, "oh no!")); } fn g() -> Result<()> { f().wrap_err("f failed") } fn h() -> Result<()> { g().wrap_err("g failed") } #[test] fn test_pyo3_exception_contents() { let err = h().unwrap_err(); let expected_contents = format!("{:?}", err); let pyerr = PyErr::from(err); Python::with_gil(|py| { let locals = [("err", pyerr)].into_py_dict(py); let pyerr = py.run("raise err", None, Some(locals)).unwrap_err(); assert_eq!(pyerr.value(py).to_string(), expected_contents); }) } fn k() -> Result<()> { Err(eyre!("Some sort of error")) } #[test] fn test_pyo3_exception_contents2() { let err = k().unwrap_err(); let expected_contents = format!("{:?}", err); let pyerr = PyErr::from(err); Python::with_gil(|py| { let locals = [("err", pyerr)].into_py_dict(py); let pyerr = py.run("raise err", None, Some(locals)).unwrap_err(); assert_eq!(pyerr.value(py).to_string(), expected_contents); }) } #[test] fn test_pyo3_unwrap_simple_err() { let origin_exc = PyValueError::new_err("Value Error"); let report: Report = origin_exc.into(); let converted: PyErr = report.into(); assert!(Python::with_gil( |py| converted.is_instance_of::(py) )) } #[test] fn test_pyo3_unwrap_complex_err() { let origin_exc = PyValueError::new_err("Value Error"); let mut report: Report = origin_exc.into(); report = report.wrap_err("Wrapped"); let converted: PyErr = report.into(); assert!(Python::with_gil( |py| converted.is_instance_of::(py) )) } } pyo3-0.20.2/src/conversions/hashbrown.rs000064400000000000000000000123531046102023000163030ustar 00000000000000#![cfg(feature = "hashbrown")] //! Conversions to and from [hashbrown](https://docs.rs/hashbrown/)’s //! `HashMap` and `HashSet`. //! //! # Setup //! //! To use this feature, add this to your **`Cargo.toml`**: //! //! ```toml //! [dependencies] //! # change * to the latest versions //! hashbrown = "*" #![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"hashbrown\"] }")] //! ``` //! //! Note that you must use compatible versions of hashbrown and PyO3. //! The required hashbrown version may vary based on the version of PyO3. use crate::{ types::set::new_from_iter, types::{IntoPyDict, PyDict, PySet}, FromPyObject, IntoPy, PyAny, PyErr, PyObject, PyResult, Python, ToPyObject, }; use std::{cmp, hash}; impl ToPyObject for hashbrown::HashMap where K: hash::Hash + cmp::Eq + ToPyObject, V: ToPyObject, H: hash::BuildHasher, { fn to_object(&self, py: Python<'_>) -> PyObject { IntoPyDict::into_py_dict(self, py).into() } } impl IntoPy for hashbrown::HashMap where K: hash::Hash + cmp::Eq + IntoPy, V: IntoPy, H: hash::BuildHasher, { fn into_py(self, py: Python<'_>) -> PyObject { let iter = self .into_iter() .map(|(k, v)| (k.into_py(py), v.into_py(py))); IntoPyDict::into_py_dict(iter, py).into() } } impl<'source, K, V, S> FromPyObject<'source> for hashbrown::HashMap where K: FromPyObject<'source> + cmp::Eq + hash::Hash, V: FromPyObject<'source>, S: hash::BuildHasher + Default, { fn extract(ob: &'source PyAny) -> Result { let dict: &PyDict = ob.downcast()?; let mut ret = hashbrown::HashMap::with_capacity_and_hasher(dict.len(), S::default()); for (k, v) in dict { ret.insert(K::extract(k)?, V::extract(v)?); } Ok(ret) } } impl ToPyObject for hashbrown::HashSet where T: hash::Hash + Eq + ToPyObject, { fn to_object(&self, py: Python<'_>) -> PyObject { new_from_iter(py, self) .expect("Failed to create Python set from hashbrown::HashSet") .into() } } impl IntoPy for hashbrown::HashSet where K: IntoPy + Eq + hash::Hash, S: hash::BuildHasher + Default, { fn into_py(self, py: Python<'_>) -> PyObject { new_from_iter(py, self.into_iter().map(|item| item.into_py(py))) .expect("Failed to create Python set from hashbrown::HashSet") .into() } } impl<'source, K, S> FromPyObject<'source> for hashbrown::HashSet where K: FromPyObject<'source> + cmp::Eq + hash::Hash, S: hash::BuildHasher + Default, { fn extract(ob: &'source PyAny) -> PyResult { let set: &PySet = ob.downcast()?; set.iter().map(K::extract).collect() } } #[cfg(test)] mod tests { use super::*; #[test] fn test_hashbrown_hashmap_to_python() { Python::with_gil(|py| { let mut map = hashbrown::HashMap::::new(); map.insert(1, 1); let m = map.to_object(py); let py_map: &PyDict = m.downcast(py).unwrap(); assert!(py_map.len() == 1); assert!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap() == 1 ); assert_eq!(map, py_map.extract().unwrap()); }); } #[test] fn test_hashbrown_hashmap_into_python() { Python::with_gil(|py| { let mut map = hashbrown::HashMap::::new(); map.insert(1, 1); let m: PyObject = map.into_py(py); let py_map: &PyDict = m.downcast(py).unwrap(); assert!(py_map.len() == 1); assert!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap() == 1 ); }); } #[test] fn test_hashbrown_hashmap_into_dict() { Python::with_gil(|py| { let mut map = hashbrown::HashMap::::new(); map.insert(1, 1); let py_map = map.into_py_dict(py); assert_eq!(py_map.len(), 1); assert_eq!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap(), 1 ); }); } #[test] fn test_extract_hashbrown_hashset() { Python::with_gil(|py| { let set = PySet::new(py, &[1, 2, 3, 4, 5]).unwrap(); let hash_set: hashbrown::HashSet = set.extract().unwrap(); assert_eq!(hash_set, [1, 2, 3, 4, 5].iter().copied().collect()); }); } #[test] fn test_hashbrown_hashset_into_py() { Python::with_gil(|py| { let hs: hashbrown::HashSet = [1, 2, 3, 4, 5].iter().cloned().collect(); let hso: PyObject = hs.clone().into_py(py); assert_eq!(hs, hso.extract(py).unwrap()); }); } } pyo3-0.20.2/src/conversions/indexmap.rs000064400000000000000000000156351046102023000161230ustar 00000000000000#![cfg(feature = "indexmap")] //! Conversions to and from [indexmap](https://docs.rs/indexmap/)’s //! `IndexMap`. //! //! [`indexmap::IndexMap`] is a hash table that is closely compatible with the standard [`std::collections::HashMap`], //! with the difference that it preserves the insertion order when iterating over keys. It was inspired //! by Python's 3.6+ dict implementation. //! //! Dictionary order is guaranteed to be insertion order in Python, hence IndexMap is a good candidate //! for maintaining an equivalent behaviour in Rust. //! //! # Setup //! //! To use this feature, add this to your **`Cargo.toml`**: //! //! ```toml //! [dependencies] //! # change * to the latest versions //! indexmap = "*" #![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"indexmap\"] }")] //! ``` //! //! Note that you must use compatible versions of indexmap and PyO3. //! The required indexmap version may vary based on the version of PyO3. //! //! # Examples //! //! Using [indexmap](https://docs.rs/indexmap) to return a dictionary with some statistics //! about a list of numbers. Because of the insertion order guarantees, the Python code will //! always print the same result, matching users' expectations about Python's dict. //! ```rust //! use indexmap::{indexmap, IndexMap}; //! use pyo3::prelude::*; //! //! fn median(data: &Vec) -> f32 { //! let sorted_data = data.clone().sort(); //! let mid = data.len() / 2; //! if data.len() % 2 == 0 { //! data[mid] as f32 //! } //! else { //! (data[mid] + data[mid - 1]) as f32 / 2.0 //! } //! } //! //! fn mean(data: &Vec) -> f32 { //! data.iter().sum::() as f32 / data.len() as f32 //! } //! fn mode(data: &Vec) -> f32 { //! let mut frequency = IndexMap::new(); // we can use IndexMap as any hash table //! //! for &element in data { //! *frequency.entry(element).or_insert(0) += 1; //! } //! //! frequency //! .iter() //! .max_by(|a, b| a.1.cmp(&b.1)) //! .map(|(k, _v)| *k) //! .unwrap() as f32 //! } //! //! #[pyfunction] //! fn calculate_statistics(data: Vec) -> IndexMap<&'static str, f32> { //! indexmap! { //! "median" => median(&data), //! "mean" => mean(&data), //! "mode" => mode(&data), //! } //! } //! //! #[pymodule] //! fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { //! m.add_function(wrap_pyfunction!(calculate_statistics, m)?)?; //! Ok(()) //! } //! ``` //! //! Python code: //! ```python //! from my_module import calculate_statistics //! //! data = [1, 1, 1, 3, 4, 5] //! print(calculate_statistics(data)) //! # always prints {"median": 2.0, "mean": 2.5, "mode": 1.0} in the same order //! # if another hash table was used, the order could be random //! ``` use crate::types::*; use crate::{FromPyObject, IntoPy, PyErr, PyObject, Python, ToPyObject}; use std::{cmp, hash}; impl ToPyObject for indexmap::IndexMap where K: hash::Hash + cmp::Eq + ToPyObject, V: ToPyObject, H: hash::BuildHasher, { fn to_object(&self, py: Python<'_>) -> PyObject { IntoPyDict::into_py_dict(self, py).into() } } impl IntoPy for indexmap::IndexMap where K: hash::Hash + cmp::Eq + IntoPy, V: IntoPy, H: hash::BuildHasher, { fn into_py(self, py: Python<'_>) -> PyObject { let iter = self .into_iter() .map(|(k, v)| (k.into_py(py), v.into_py(py))); IntoPyDict::into_py_dict(iter, py).into() } } impl<'source, K, V, S> FromPyObject<'source> for indexmap::IndexMap where K: FromPyObject<'source> + cmp::Eq + hash::Hash, V: FromPyObject<'source>, S: hash::BuildHasher + Default, { fn extract(ob: &'source PyAny) -> Result { let dict: &PyDict = ob.downcast()?; let mut ret = indexmap::IndexMap::with_capacity_and_hasher(dict.len(), S::default()); for (k, v) in dict { ret.insert(K::extract(k)?, V::extract(v)?); } Ok(ret) } } #[cfg(test)] mod test_indexmap { use crate::types::*; use crate::{IntoPy, PyObject, Python, ToPyObject}; #[test] fn test_indexmap_indexmap_to_python() { Python::with_gil(|py| { let mut map = indexmap::IndexMap::::new(); map.insert(1, 1); let m = map.to_object(py); let py_map: &PyDict = m.downcast(py).unwrap(); assert!(py_map.len() == 1); assert!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap() == 1 ); assert_eq!( map, py_map.extract::>().unwrap() ); }); } #[test] fn test_indexmap_indexmap_into_python() { Python::with_gil(|py| { let mut map = indexmap::IndexMap::::new(); map.insert(1, 1); let m: PyObject = map.into_py(py); let py_map: &PyDict = m.downcast(py).unwrap(); assert!(py_map.len() == 1); assert!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap() == 1 ); }); } #[test] fn test_indexmap_indexmap_into_dict() { Python::with_gil(|py| { let mut map = indexmap::IndexMap::::new(); map.insert(1, 1); let py_map = map.into_py_dict(py); assert_eq!(py_map.len(), 1); assert_eq!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap(), 1 ); }); } #[test] fn test_indexmap_indexmap_insertion_order_round_trip() { Python::with_gil(|py| { let n = 20; let mut map = indexmap::IndexMap::::new(); for i in 1..=n { if i % 2 == 1 { map.insert(i, i); } else { map.insert(n - i, i); } } let py_map = map.clone().into_py_dict(py); let trip_map = py_map.extract::>().unwrap(); for (((k1, v1), (k2, v2)), (k3, v3)) in map.iter().zip(py_map.iter()).zip(trip_map.iter()) { let k2 = k2.extract::().unwrap(); let v2 = v2.extract::().unwrap(); assert_eq!((k1, v1), (&k2, &v2)); assert_eq!((k1, v1), (k3, v3)); assert_eq!((&k2, &v2), (k3, v3)); } }); } } pyo3-0.20.2/src/conversions/mod.rs000064400000000000000000000004621046102023000150650ustar 00000000000000//! This module contains conversions between various Rust object and their representation in Python. pub mod anyhow; pub mod chrono; pub mod either; pub mod eyre; pub mod hashbrown; pub mod indexmap; pub mod num_bigint; pub mod num_complex; pub mod rust_decimal; pub mod serde; pub mod smallvec; mod std; pyo3-0.20.2/src/conversions/num_bigint.rs000064400000000000000000000305551046102023000164470ustar 00000000000000#![cfg(feature = "num-bigint")] //! Conversions to and from [num-bigint](https://docs.rs/num-bigint)’s [`BigInt`] and [`BigUint`] types. //! //! This is useful for converting Python integers when they may not fit in Rust's built-in integer types. //! //! # Setup //! //! To use this feature, add this to your **`Cargo.toml`**: //! //! ```toml //! [dependencies] //! num-bigint = "*" #![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"num-bigint\"] }")] //! ``` //! //! Note that you must use compatible versions of num-bigint and PyO3. //! The required num-bigint version may vary based on the version of PyO3. //! //! ## Examples //! //! Using [`BigInt`] to correctly increment an arbitrary precision integer. //! This is not possible with Rust's native integers if the Python integer is too large, //! in which case it will fail its conversion and raise `OverflowError`. //! ```rust //! use num_bigint::BigInt; //! use pyo3::prelude::*; //! //! #[pyfunction] //! fn add_one(n: BigInt) -> BigInt { //! n + 1 //! } //! //! #[pymodule] //! fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { //! m.add_function(wrap_pyfunction!(add_one, m)?)?; //! Ok(()) //! } //! ``` //! //! Python code: //! ```python //! from my_module import add_one //! //! n = 1 << 1337 //! value = add_one(n) //! //! assert n + 1 == value //! ``` use crate::{ ffi, types::*, FromPyObject, IntoPy, Py, PyAny, PyObject, PyResult, Python, ToPyObject, }; use num_bigint::{BigInt, BigUint}; #[cfg(not(Py_LIMITED_API))] use num_bigint::Sign; // for identical functionality between BigInt and BigUint macro_rules! bigint_conversion { ($rust_ty: ty, $is_signed: expr, $to_bytes: path) => { #[cfg_attr(docsrs, doc(cfg(feature = "num-bigint")))] impl ToPyObject for $rust_ty { #[cfg(not(Py_LIMITED_API))] fn to_object(&self, py: Python<'_>) -> PyObject { let bytes = $to_bytes(self); unsafe { let obj = ffi::_PyLong_FromByteArray( bytes.as_ptr().cast(), bytes.len(), 1, $is_signed, ); PyObject::from_owned_ptr(py, obj) } } #[cfg(Py_LIMITED_API)] fn to_object(&self, py: Python<'_>) -> PyObject { let bytes = $to_bytes(self); let bytes_obj = PyBytes::new(py, &bytes); let kwargs = if $is_signed > 0 { let kwargs = PyDict::new(py); kwargs.set_item(crate::intern!(py, "signed"), true).unwrap(); Some(kwargs) } else { None }; py.get_type::() .call_method("from_bytes", (bytes_obj, "little"), kwargs) .expect("int.from_bytes() failed during to_object()") // FIXME: #1813 or similar .into() } } #[cfg_attr(docsrs, doc(cfg(feature = "num-bigint")))] impl IntoPy for $rust_ty { fn into_py(self, py: Python<'_>) -> PyObject { self.to_object(py) } } }; } bigint_conversion!(BigUint, 0, BigUint::to_bytes_le); bigint_conversion!(BigInt, 1, BigInt::to_signed_bytes_le); #[cfg_attr(docsrs, doc(cfg(feature = "num-bigint")))] impl<'source> FromPyObject<'source> for BigInt { fn extract(ob: &'source PyAny) -> PyResult { let py = ob.py(); // fast path - checking for subclass of `int` just checks a bit in the type object let num_owned: Py; let num = if let Ok(long) = ob.downcast::() { long } else { num_owned = unsafe { Py::from_owned_ptr_or_err(py, ffi::PyNumber_Index(ob.as_ptr()))? }; num_owned.as_ref(py) }; let n_bits = int_n_bits(num)?; if n_bits == 0 { return Ok(BigInt::from(0isize)); } #[cfg(not(Py_LIMITED_API))] { let mut buffer = int_to_u32_vec(num, (n_bits + 32) / 32, true)?; let sign = if buffer.last().copied().map_or(false, |last| last >> 31 != 0) { // BigInt::new takes an unsigned array, so need to convert from two's complement // flip all bits, 'subtract' 1 (by adding one to the unsigned array) let mut elements = buffer.iter_mut(); for element in elements.by_ref() { *element = (!*element).wrapping_add(1); if *element != 0 { // if the element didn't wrap over, no need to keep adding further ... break; } } // ... so just two's complement the rest for element in elements { *element = !*element; } Sign::Minus } else { Sign::Plus }; Ok(BigInt::new(sign, buffer)) } #[cfg(Py_LIMITED_API)] { let bytes = int_to_py_bytes(num, (n_bits + 8) / 8, true)?; Ok(BigInt::from_signed_bytes_le(bytes.as_bytes())) } } } #[cfg_attr(docsrs, doc(cfg(feature = "num-bigint")))] impl<'source> FromPyObject<'source> for BigUint { fn extract(ob: &'source PyAny) -> PyResult { let py = ob.py(); // fast path - checking for subclass of `int` just checks a bit in the type object let num_owned: Py; let num = if let Ok(long) = ob.downcast::() { long } else { num_owned = unsafe { Py::from_owned_ptr_or_err(py, ffi::PyNumber_Index(ob.as_ptr()))? }; num_owned.as_ref(py) }; let n_bits = int_n_bits(num)?; if n_bits == 0 { return Ok(BigUint::from(0usize)); } #[cfg(not(Py_LIMITED_API))] { let buffer = int_to_u32_vec(num, (n_bits + 31) / 32, false)?; Ok(BigUint::new(buffer)) } #[cfg(Py_LIMITED_API)] { let bytes = int_to_py_bytes(num, (n_bits + 7) / 8, false)?; Ok(BigUint::from_bytes_le(bytes.as_bytes())) } } } #[cfg(not(Py_LIMITED_API))] #[inline] fn int_to_u32_vec(long: &PyLong, n_digits: usize, is_signed: bool) -> PyResult> { let mut buffer = Vec::with_capacity(n_digits); unsafe { crate::err::error_on_minusone( long.py(), ffi::_PyLong_AsByteArray( long.as_ptr().cast(), buffer.as_mut_ptr() as *mut u8, n_digits * 4, 1, is_signed.into(), ), )?; buffer.set_len(n_digits) }; buffer .iter_mut() .for_each(|chunk| *chunk = u32::from_le(*chunk)); Ok(buffer) } #[cfg(Py_LIMITED_API)] fn int_to_py_bytes(long: &PyLong, n_bytes: usize, is_signed: bool) -> PyResult<&PyBytes> { use crate::intern; let py = long.py(); let kwargs = if is_signed { let kwargs = PyDict::new(py); kwargs.set_item(intern!(py, "signed"), true)?; Some(kwargs) } else { None }; let bytes = long.call_method( intern!(py, "to_bytes"), (n_bytes, intern!(py, "little")), kwargs, )?; Ok(bytes.downcast()?) } #[inline] fn int_n_bits(long: &PyLong) -> PyResult { let py = long.py(); #[cfg(not(Py_LIMITED_API))] { // fast path let n_bits = unsafe { ffi::_PyLong_NumBits(long.as_ptr()) }; if n_bits == (-1isize as usize) { return Err(crate::PyErr::fetch(py)); } Ok(n_bits) } #[cfg(Py_LIMITED_API)] { // slow path long.call_method0(crate::intern!(py, "bit_length")) .and_then(PyAny::extract) } } #[cfg(test)] mod tests { use super::*; use crate::types::{PyDict, PyModule}; use indoc::indoc; fn rust_fib() -> impl Iterator where T: From, for<'a> &'a T: std::ops::Add, { let mut f0: T = T::from(1); let mut f1: T = T::from(1); std::iter::from_fn(move || { let f2 = &f0 + &f1; Some(std::mem::replace(&mut f0, std::mem::replace(&mut f1, f2))) }) } fn python_fib(py: Python<'_>) -> impl Iterator + '_ { let mut f0 = 1.to_object(py); let mut f1 = 1.to_object(py); std::iter::from_fn(move || { let f2 = f0.call_method1(py, "__add__", (f1.as_ref(py),)).unwrap(); Some(std::mem::replace(&mut f0, std::mem::replace(&mut f1, f2))) }) } #[test] fn convert_biguint() { Python::with_gil(|py| { // check the first 2000 numbers in the fibonacci sequence for (py_result, rs_result) in python_fib(py).zip(rust_fib::()).take(2000) { // Python -> Rust assert_eq!(py_result.extract::(py).unwrap(), rs_result); // Rust -> Python assert!(py_result.as_ref(py).eq(rs_result).unwrap()); } }); } #[test] fn convert_bigint() { Python::with_gil(|py| { // check the first 2000 numbers in the fibonacci sequence for (py_result, rs_result) in python_fib(py).zip(rust_fib::()).take(2000) { // Python -> Rust assert_eq!(py_result.extract::(py).unwrap(), rs_result); // Rust -> Python assert!(py_result.as_ref(py).eq(&rs_result).unwrap()); // negate let rs_result = rs_result * -1; let py_result = py_result.call_method0(py, "__neg__").unwrap(); // Python -> Rust assert_eq!(py_result.extract::(py).unwrap(), rs_result); // Rust -> Python assert!(py_result.as_ref(py).eq(rs_result).unwrap()); } }); } fn python_index_class(py: Python<'_>) -> &PyModule { let index_code = indoc!( r#" class C: def __init__(self, x): self.x = x def __index__(self): return self.x "# ); PyModule::from_code(py, index_code, "index.py", "index").unwrap() } #[test] fn convert_index_class() { Python::with_gil(|py| { let index = python_index_class(py); let locals = PyDict::new(py); locals.set_item("index", index).unwrap(); let ob = py.eval("index.C(10)", None, Some(locals)).unwrap(); let _: BigInt = FromPyObject::extract(ob).unwrap(); }); } #[test] fn handle_zero() { Python::with_gil(|py| { let zero: BigInt = 0.to_object(py).extract(py).unwrap(); assert_eq!(zero, BigInt::from(0)); }) } /// `OverflowError` on converting Python int to BigInt, see issue #629 #[test] fn check_overflow() { Python::with_gil(|py| { macro_rules! test { ($T:ty, $value:expr, $py:expr) => { let value = $value; println!("{}: {}", stringify!($T), value); let python_value = value.clone().into_py(py); let roundtrip_value = python_value.extract::<$T>(py).unwrap(); assert_eq!(value, roundtrip_value); }; } for i in 0..=256usize { // test a lot of values to help catch other bugs too test!(BigInt, BigInt::from(i), py); test!(BigUint, BigUint::from(i), py); test!(BigInt, -BigInt::from(i), py); test!(BigInt, BigInt::from(1) << i, py); test!(BigUint, BigUint::from(1u32) << i, py); test!(BigInt, -BigInt::from(1) << i, py); test!(BigInt, (BigInt::from(1) << i) + 1u32, py); test!(BigUint, (BigUint::from(1u32) << i) + 1u32, py); test!(BigInt, (-BigInt::from(1) << i) + 1u32, py); test!(BigInt, (BigInt::from(1) << i) - 1u32, py); test!(BigUint, (BigUint::from(1u32) << i) - 1u32, py); test!(BigInt, (-BigInt::from(1) << i) - 1u32, py); } }); } } pyo3-0.20.2/src/conversions/num_complex.rs000064400000000000000000000257531046102023000166460ustar 00000000000000#![cfg(feature = "num-complex")] //! Conversions to and from [num-complex](https://docs.rs/num-complex)’ //! [`Complex`]`<`[`f32`]`>` and [`Complex`]`<`[`f64`]`>`. //! //! num-complex’ [`Complex`] supports more operations than PyO3's [`PyComplex`] //! and can be used with the rest of the Rust ecosystem. //! //! # Setup //! //! To use this feature, add this to your **`Cargo.toml`**: //! //! ```toml //! [dependencies] //! # change * to the latest versions //! num-complex = "*" #![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"num-complex\"] }")] //! ``` //! //! Note that you must use compatible versions of num-complex and PyO3. //! The required num-complex version may vary based on the version of PyO3. //! //! # Examples //! //! Using [num-complex](https://docs.rs/num-complex) and [nalgebra](https://docs.rs/nalgebra) //! to create a pyfunction that calculates the eigenvalues of a 2x2 matrix. //! ```ignore //! # // not tested because nalgebra isn't supported on msrv //! # // please file an issue if it breaks! //! use nalgebra::base::{dimension::Const, storage::Storage, Matrix}; //! use num_complex::Complex; //! use pyo3::prelude::*; //! //! type T = Complex; //! //! #[pyfunction] //! fn get_eigenvalues(m11: T, m12: T, m21: T, m22: T) -> Vec { //! let mat = Matrix::, Const<2>, _>::new(m11, m12, m21, m22); //! //! match mat.eigenvalues() { //! Some(e) => e.data.as_slice().to_vec(), //! None => vec![], //! } //! } //! //! #[pymodule] //! fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { //! m.add_function(wrap_pyfunction!(get_eigenvalues, m)?)?; //! Ok(()) //! } //! # // test //! # use assert_approx_eq::assert_approx_eq; //! # use nalgebra::ComplexField; //! # use pyo3::types::PyComplex; //! # //! # fn main() -> PyResult<()> { //! # Python::with_gil(|py| -> PyResult<()> { //! # let module = PyModule::new(py, "my_module")?; //! # //! # module.add_function(wrap_pyfunction!(get_eigenvalues, module)?)?; //! # //! # let m11 = PyComplex::from_doubles(py, 0_f64, -1_f64); //! # let m12 = PyComplex::from_doubles(py, 1_f64, 0_f64); //! # let m21 = PyComplex::from_doubles(py, 2_f64, -1_f64); //! # let m22 = PyComplex::from_doubles(py, -1_f64, 0_f64); //! # //! # let result = module //! # .getattr("get_eigenvalues")? //! # .call1((m11, m12, m21, m22))?; //! # println!("eigenvalues: {:?}", result); //! # //! # let result = result.extract::>()?; //! # let e0 = result[0]; //! # let e1 = result[1]; //! # //! # assert_approx_eq!(e0, Complex::new(1_f64, -1_f64)); //! # assert_approx_eq!(e1, Complex::new(-2_f64, 0_f64)); //! # //! # Ok(()) //! # }) //! # } //! ``` //! //! Python code: //! ```python //! from my_module import get_eigenvalues //! //! m11 = complex(0,-1) //! m12 = complex(1,0) //! m21 = complex(2,-1) //! m22 = complex(-1,0) //! //! result = get_eigenvalues(m11,m12,m21,m22) //! assert result == [complex(1,-1), complex(-2,0)] //! ``` use crate::{ ffi, types::PyComplex, FromPyObject, PyAny, PyErr, PyObject, PyResult, Python, ToPyObject, }; use num_complex::Complex; use std::os::raw::c_double; impl PyComplex { /// Creates a new Python `PyComplex` object from `num_complex`'s [`Complex`]. pub fn from_complex>(py: Python<'_>, complex: Complex) -> &PyComplex { unsafe { let ptr = ffi::PyComplex_FromDoubles(complex.re.into(), complex.im.into()); py.from_owned_ptr(ptr) } } } macro_rules! complex_conversion { ($float: ty) => { #[cfg_attr(docsrs, doc(cfg(feature = "num-complex")))] impl ToPyObject for Complex<$float> { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { crate::IntoPy::::into_py(self.to_owned(), py) } } #[cfg_attr(docsrs, doc(cfg(feature = "num-complex")))] impl crate::IntoPy for Complex<$float> { fn into_py(self, py: Python<'_>) -> PyObject { unsafe { let raw_obj = ffi::PyComplex_FromDoubles(self.re as c_double, self.im as c_double); PyObject::from_owned_ptr(py, raw_obj) } } } #[cfg_attr(docsrs, doc(cfg(feature = "num-complex")))] impl<'source> FromPyObject<'source> for Complex<$float> { fn extract(obj: &'source PyAny) -> PyResult> { #[cfg(not(any(Py_LIMITED_API, PyPy)))] unsafe { let val = ffi::PyComplex_AsCComplex(obj.as_ptr()); if val.real == -1.0 { if let Some(err) = PyErr::take(obj.py()) { return Err(err); } } Ok(Complex::new(val.real as $float, val.imag as $float)) } #[cfg(any(Py_LIMITED_API, PyPy))] unsafe { let obj = if obj.is_instance_of::() { obj } else if let Some(method) = obj.lookup_special(crate::intern!(obj.py(), "__complex__"))? { method.call0()? } else { // `obj` might still implement `__float__` or `__index__`, which will be // handled by `PyComplex_{Real,Imag}AsDouble`, including propagating any // errors if those methods don't exist / raise exceptions. obj }; let ptr = obj.as_ptr(); let real = ffi::PyComplex_RealAsDouble(ptr); if real == -1.0 { if let Some(err) = PyErr::take(obj.py()) { return Err(err); } } let imag = ffi::PyComplex_ImagAsDouble(ptr); Ok(Complex::new(real as $float, imag as $float)) } } } }; } complex_conversion!(f32); complex_conversion!(f64); #[cfg(test)] mod tests { use super::*; use crate::types::PyModule; #[test] fn from_complex() { Python::with_gil(|py| { let complex = Complex::new(3.0, 1.2); let py_c = PyComplex::from_complex(py, complex); assert_eq!(py_c.real(), 3.0); assert_eq!(py_c.imag(), 1.2); }); } #[test] fn to_from_complex() { Python::with_gil(|py| { let val = Complex::new(3.0, 1.2); let obj = val.to_object(py); assert_eq!(obj.extract::>(py).unwrap(), val); }); } #[test] fn from_complex_err() { Python::with_gil(|py| { let obj = vec![1].to_object(py); assert!(obj.extract::>(py).is_err()); }); } #[test] fn from_python_magic() { Python::with_gil(|py| { let module = PyModule::from_code( py, r#" class A: def __complex__(self): return 3.0+1.2j class B: def __float__(self): return 3.0 class C: def __index__(self): return 3 "#, "test.py", "test", ) .unwrap(); let from_complex = module.getattr("A").unwrap().call0().unwrap(); assert_eq!( from_complex.extract::>().unwrap(), Complex::new(3.0, 1.2) ); let from_float = module.getattr("B").unwrap().call0().unwrap(); assert_eq!( from_float.extract::>().unwrap(), Complex::new(3.0, 0.0) ); // Before Python 3.8, `__index__` wasn't tried by `float`/`complex`. #[cfg(Py_3_8)] { let from_index = module.getattr("C").unwrap().call0().unwrap(); assert_eq!( from_index.extract::>().unwrap(), Complex::new(3.0, 0.0) ); } }) } #[test] fn from_python_inherited_magic() { Python::with_gil(|py| { let module = PyModule::from_code( py, r#" class First: pass class ComplexMixin: def __complex__(self): return 3.0+1.2j class FloatMixin: def __float__(self): return 3.0 class IndexMixin: def __index__(self): return 3 class A(First, ComplexMixin): pass class B(First, FloatMixin): pass class C(First, IndexMixin): pass "#, "test.py", "test", ) .unwrap(); let from_complex = module.getattr("A").unwrap().call0().unwrap(); assert_eq!( from_complex.extract::>().unwrap(), Complex::new(3.0, 1.2) ); let from_float = module.getattr("B").unwrap().call0().unwrap(); assert_eq!( from_float.extract::>().unwrap(), Complex::new(3.0, 0.0) ); #[cfg(Py_3_8)] { let from_index = module.getattr("C").unwrap().call0().unwrap(); assert_eq!( from_index.extract::>().unwrap(), Complex::new(3.0, 0.0) ); } }) } #[test] fn from_python_noncallable_descriptor_magic() { // Functions and lambdas implement the descriptor protocol in a way that makes // `type(inst).attr(inst)` equivalent to `inst.attr()` for methods, but this isn't the only // way the descriptor protocol might be implemented. Python::with_gil(|py| { let module = PyModule::from_code( py, r#" class A: @property def __complex__(self): return lambda: 3.0+1.2j "#, "test.py", "test", ) .unwrap(); let obj = module.getattr("A").unwrap().call0().unwrap(); assert_eq!( obj.extract::>().unwrap(), Complex::new(3.0, 1.2) ); }) } #[test] fn from_python_nondescriptor_magic() { // Magic methods don't need to implement the descriptor protocol, if they're callable. Python::with_gil(|py| { let module = PyModule::from_code( py, r#" class MyComplex: def __call__(self): return 3.0+1.2j class A: __complex__ = MyComplex() "#, "test.py", "test", ) .unwrap(); let obj = module.getattr("A").unwrap().call0().unwrap(); assert_eq!( obj.extract::>().unwrap(), Complex::new(3.0, 1.2) ); }) } } pyo3-0.20.2/src/conversions/rust_decimal.rs000064400000000000000000000155421046102023000167660ustar 00000000000000#![cfg(feature = "rust_decimal")] //! Conversions to and from [rust_decimal](https://docs.rs/rust_decimal)'s [`Decimal`] type. //! //! This is useful for converting Python's decimal.Decimal into and from a native Rust type. //! //! # Setup //! //! To use this feature, add to your **`Cargo.toml`**: //! //! ```toml //! [dependencies] #![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"rust_decimal\"] }")] //! rust_decimal = "1.0" //! ``` //! //! Note that you must use a compatible version of rust_decimal and PyO3. //! The required rust_decimal version may vary based on the version of PyO3. //! //! # Example //! //! Rust code to create a function that adds one to a Decimal //! //! ```rust //! use rust_decimal::Decimal; //! use pyo3::prelude::*; //! //! #[pyfunction] //! fn add_one(d: Decimal) -> Decimal { //! d + Decimal::ONE //! } //! //! #[pymodule] //! fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { //! m.add_function(wrap_pyfunction!(add_one, m)?)?; //! Ok(()) //! } //! ``` //! //! Python code that validates the functionality //! //! //! ```python //! from my_module import add_one //! from decimal import Decimal //! //! d = Decimal("2") //! value = add_one(d) //! //! assert d + 1 == value //! ``` use crate::exceptions::PyValueError; use crate::once_cell::GILOnceCell; use crate::types::PyType; use crate::{intern, FromPyObject, IntoPy, Py, PyAny, PyObject, PyResult, Python, ToPyObject}; use rust_decimal::Decimal; use std::str::FromStr; impl FromPyObject<'_> for Decimal { fn extract(obj: &PyAny) -> PyResult { // use the string representation to not be lossy if let Ok(val) = obj.extract() { Ok(Decimal::new(val, 0)) } else { Decimal::from_str(obj.str()?.to_str()?) .map_err(|e| PyValueError::new_err(e.to_string())) } } } static DECIMAL_CLS: GILOnceCell> = GILOnceCell::new(); fn get_decimal_cls(py: Python<'_>) -> PyResult<&PyType> { DECIMAL_CLS .get_or_try_init(py, || { py.import(intern!(py, "decimal"))? .getattr(intern!(py, "Decimal"))? .extract() }) .map(|ty| ty.as_ref(py)) } impl ToPyObject for Decimal { fn to_object(&self, py: Python<'_>) -> PyObject { // TODO: handle error gracefully when ToPyObject can error // look up the decimal.Decimal let dec_cls = get_decimal_cls(py).expect("failed to load decimal.Decimal"); // now call the constructor with the Rust Decimal string-ified // to not be lossy let ret = dec_cls .call1((self.to_string(),)) .expect("failed to call decimal.Decimal(value)"); ret.to_object(py) } } impl IntoPy for Decimal { fn into_py(self, py: Python<'_>) -> PyObject { self.to_object(py) } } #[cfg(test)] mod test_rust_decimal { use super::*; use crate::err::PyErr; use crate::types::PyDict; use rust_decimal::Decimal; #[cfg(not(target_arch = "wasm32"))] use proptest::prelude::*; macro_rules! convert_constants { ($name:ident, $rs:expr, $py:literal) => { #[test] fn $name() { Python::with_gil(|py| { let rs_orig = $rs; let rs_dec = rs_orig.into_py(py); let locals = PyDict::new(py); locals.set_item("rs_dec", &rs_dec).unwrap(); // Checks if Rust Decimal -> Python Decimal conversion is correct py.run( &format!( "import decimal\npy_dec = decimal.Decimal({})\nassert py_dec == rs_dec", $py ), None, Some(locals), ) .unwrap(); // Checks if Python Decimal -> Rust Decimal conversion is correct let py_dec = locals.get_item("py_dec").unwrap().unwrap(); let py_result: Decimal = FromPyObject::extract(py_dec).unwrap(); assert_eq!(rs_orig, py_result); }) } }; } convert_constants!(convert_zero, Decimal::ZERO, "0"); convert_constants!(convert_one, Decimal::ONE, "1"); convert_constants!(convert_neg_one, Decimal::NEGATIVE_ONE, "-1"); convert_constants!(convert_two, Decimal::TWO, "2"); convert_constants!(convert_ten, Decimal::TEN, "10"); convert_constants!(convert_one_hundred, Decimal::ONE_HUNDRED, "100"); convert_constants!(convert_one_thousand, Decimal::ONE_THOUSAND, "1000"); #[cfg(not(target_arch = "wasm32"))] proptest! { #[test] fn test_roundtrip( lo in any::(), mid in any::(), high in any::(), negative in any::(), scale in 0..28u32 ) { let num = Decimal::from_parts(lo, mid, high, negative, scale); Python::with_gil(|py| { let rs_dec = num.into_py(py); let locals = PyDict::new(py); locals.set_item("rs_dec", &rs_dec).unwrap(); py.run( &format!( "import decimal\npy_dec = decimal.Decimal(\"{}\")\nassert py_dec == rs_dec", num), None, Some(locals)).unwrap(); let roundtripped: Decimal = rs_dec.extract(py).unwrap(); assert_eq!(num, roundtripped); }) } #[test] fn test_integers(num in any::()) { Python::with_gil(|py| { let py_num = num.into_py(py); let roundtripped: Decimal = py_num.extract(py).unwrap(); let rs_dec = Decimal::new(num, 0); assert_eq!(rs_dec, roundtripped); }) } } #[test] fn test_nan() { Python::with_gil(|py| { let locals = PyDict::new(py); py.run( "import decimal\npy_dec = decimal.Decimal(\"NaN\")", None, Some(locals), ) .unwrap(); let py_dec = locals.get_item("py_dec").unwrap().unwrap(); let roundtripped: Result = FromPyObject::extract(py_dec); assert!(roundtripped.is_err()); }) } #[test] fn test_infinity() { Python::with_gil(|py| { let locals = PyDict::new(py); py.run( "import decimal\npy_dec = decimal.Decimal(\"Infinity\")", None, Some(locals), ) .unwrap(); let py_dec = locals.get_item("py_dec").unwrap().unwrap(); let roundtripped: Result = FromPyObject::extract(py_dec); assert!(roundtripped.is_err()); }) } } pyo3-0.20.2/src/conversions/serde.rs000064400000000000000000000023731046102023000154130ustar 00000000000000#![cfg(feature = "serde")] //! Enables (de)serialization of [`Py`]`` objects via [serde](https://docs.rs/serde). //! //! # Setup //! //! To use this feature, add this to your **`Cargo.toml`**: //! //! ```toml //! [dependencies] #![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"serde\"] }")] //! serde = "1.0" //! ``` use crate::{Py, PyAny, PyClass, Python}; use serde::{de, ser, Deserialize, Deserializer, Serialize, Serializer}; impl Serialize for Py where T: Serialize + PyClass, { fn serialize(&self, serializer: S) -> Result<::Ok, ::Error> where S: Serializer, { Python::with_gil(|py| { self.try_borrow(py) .map_err(|e| ser::Error::custom(e.to_string()))? .serialize(serializer) }) } } impl<'de, T> Deserialize<'de> for Py where T: PyClass + Deserialize<'de>, { fn deserialize(deserializer: D) -> Result, D::Error> where D: Deserializer<'de>, { let deserialized = T::deserialize(deserializer)?; Python::with_gil(|py| { Py::new(py, deserialized).map_err(|e| de::Error::custom(e.to_string())) }) } } pyo3-0.20.2/src/conversions/smallvec.rs000064400000000000000000000076741046102023000161300ustar 00000000000000#![cfg(feature = "smallvec")] //! Conversions to and from [smallvec](https://docs.rs/smallvec/). //! //! # Setup //! //! To use this feature, add this to your **`Cargo.toml`**: //! //! ```toml //! [dependencies] //! # change * to the latest versions //! smallvec = "*" #![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"smallvec\"] }")] //! ``` //! //! Note that you must use compatible versions of smallvec and PyO3. //! The required smallvec version may vary based on the version of PyO3. use crate::exceptions::PyTypeError; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::types::list::new_from_iter; use crate::types::{PySequence, PyString}; use crate::{ ffi, FromPyObject, IntoPy, PyAny, PyDowncastError, PyObject, PyResult, Python, ToPyObject, }; use smallvec::{Array, SmallVec}; impl ToPyObject for SmallVec where A: Array, A::Item: ToPyObject, { fn to_object(&self, py: Python<'_>) -> PyObject { self.as_slice().to_object(py) } } impl IntoPy for SmallVec where A: Array, A::Item: IntoPy, { fn into_py(self, py: Python<'_>) -> PyObject { let mut iter = self.into_iter().map(|e| e.into_py(py)); let list = new_from_iter(py, &mut iter); list.into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::list_of(A::Item::type_output()) } } impl<'a, A> FromPyObject<'a> for SmallVec where A: Array, A::Item: FromPyObject<'a>, { fn extract(obj: &'a PyAny) -> PyResult { if obj.is_instance_of::() { return Err(PyTypeError::new_err("Can't extract `str` to `SmallVec`")); } extract_sequence(obj) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { TypeInfo::sequence_of(A::Item::type_input()) } } fn extract_sequence<'s, A>(obj: &'s PyAny) -> PyResult> where A: Array, A::Item: FromPyObject<'s>, { // Types that pass `PySequence_Check` usually implement enough of the sequence protocol // to support this function and if not, we will only fail extraction safely. let seq: &PySequence = unsafe { if ffi::PySequence_Check(obj.as_ptr()) != 0 { obj.downcast_unchecked() } else { return Err(PyDowncastError::new(obj, "Sequence").into()); } }; let mut sv = SmallVec::with_capacity(seq.len().unwrap_or(0)); for item in seq.iter()? { sv.push(item?.extract::()?); } Ok(sv) } #[cfg(test)] mod tests { use super::*; use crate::types::{PyDict, PyList}; #[test] fn test_smallvec_into_py() { Python::with_gil(|py| { let sv: SmallVec<[u64; 8]> = [1, 2, 3, 4, 5].iter().cloned().collect(); let hso: PyObject = sv.clone().into_py(py); let l = PyList::new(py, [1, 2, 3, 4, 5]); assert!(l.eq(hso).unwrap()); }); } #[test] fn test_smallvec_from_py_object() { Python::with_gil(|py| { let l = PyList::new(py, [1, 2, 3, 4, 5]); let sv: SmallVec<[u64; 8]> = l.extract().unwrap(); assert_eq!(sv.as_slice(), [1, 2, 3, 4, 5]); }); } #[test] fn test_smallvec_from_py_object_fails() { Python::with_gil(|py| { let dict = PyDict::new(py); let sv: PyResult> = dict.extract(); assert_eq!( sv.unwrap_err().to_string(), "TypeError: 'dict' object cannot be converted to 'Sequence'" ); }); } #[test] fn test_smallvec_to_object() { Python::with_gil(|py| { let sv: SmallVec<[u64; 8]> = [1, 2, 3, 4, 5].iter().cloned().collect(); let hso: PyObject = sv.to_object(py); let l = PyList::new(py, [1, 2, 3, 4, 5]); assert!(l.eq(hso).unwrap()); }); } } pyo3-0.20.2/src/conversions/std/array.rs000064400000000000000000000176341046102023000162270ustar 00000000000000use crate::types::PySequence; use crate::{exceptions, PyErr}; use crate::{ ffi, FromPyObject, IntoPy, Py, PyAny, PyDowncastError, PyObject, PyResult, Python, ToPyObject, }; impl IntoPy for [T; N] where T: IntoPy, { fn into_py(self, py: Python<'_>) -> PyObject { unsafe { #[allow(deprecated)] // we're not on edition 2021 yet let elements = std::array::IntoIter::new(self); let len = N as ffi::Py_ssize_t; let ptr = ffi::PyList_New(len); // We create the `Py` pointer here for two reasons: // - panics if the ptr is null // - its Drop cleans up the list if user code panics. let list: Py = Py::from_owned_ptr(py, ptr); for (i, obj) in (0..len).zip(elements) { let obj = obj.into_py(py).into_ptr(); #[cfg(not(Py_LIMITED_API))] ffi::PyList_SET_ITEM(ptr, i, obj); #[cfg(Py_LIMITED_API)] ffi::PyList_SetItem(ptr, i, obj); } list } } } impl ToPyObject for [T; N] where T: ToPyObject, { fn to_object(&self, py: Python<'_>) -> PyObject { self.as_ref().to_object(py) } } impl<'a, T, const N: usize> FromPyObject<'a> for [T; N] where T: FromPyObject<'a>, { fn extract(obj: &'a PyAny) -> PyResult { create_array_from_obj(obj) } } fn create_array_from_obj<'s, T, const N: usize>(obj: &'s PyAny) -> PyResult<[T; N]> where T: FromPyObject<'s>, { // Types that pass `PySequence_Check` usually implement enough of the sequence protocol // to support this function and if not, we will only fail extraction safely. let seq: &PySequence = unsafe { if ffi::PySequence_Check(obj.as_ptr()) != 0 { obj.downcast_unchecked() } else { return Err(PyDowncastError::new(obj, "Sequence").into()); } }; let seq_len = seq.len()?; if seq_len != N { return Err(invalid_sequence_length(N, seq_len)); } array_try_from_fn(|idx| seq.get_item(idx).and_then(PyAny::extract)) } // TODO use std::array::try_from_fn, if that stabilises: // (https://github.com/rust-lang/rust/pull/75644) fn array_try_from_fn(mut cb: F) -> Result<[T; N], E> where F: FnMut(usize) -> Result, { // Helper to safely create arrays since the standard library doesn't // provide one yet. Shouldn't be necessary in the future. struct ArrayGuard { dst: *mut T, initialized: usize, } impl Drop for ArrayGuard { fn drop(&mut self) { debug_assert!(self.initialized <= N); let initialized_part = core::ptr::slice_from_raw_parts_mut(self.dst, self.initialized); unsafe { core::ptr::drop_in_place(initialized_part); } } } // [MaybeUninit; N] would be "nicer" but is actually difficult to create - there are nightly // APIs which would make this easier. let mut array: core::mem::MaybeUninit<[T; N]> = core::mem::MaybeUninit::uninit(); let mut guard: ArrayGuard = ArrayGuard { dst: array.as_mut_ptr() as _, initialized: 0, }; unsafe { let mut value_ptr = array.as_mut_ptr() as *mut T; for i in 0..N { core::ptr::write(value_ptr, cb(i)?); value_ptr = value_ptr.offset(1); guard.initialized += 1; } core::mem::forget(guard); Ok(array.assume_init()) } } fn invalid_sequence_length(expected: usize, actual: usize) -> PyErr { exceptions::PyValueError::new_err(format!( "expected a sequence of length {} (got {})", expected, actual )) } #[cfg(test)] mod tests { use std::{ panic, sync::atomic::{AtomicUsize, Ordering}, }; use crate::{types::PyList, IntoPy, PyResult, Python, ToPyObject}; #[test] fn array_try_from_fn() { static DROP_COUNTER: AtomicUsize = AtomicUsize::new(0); struct CountDrop; impl Drop for CountDrop { fn drop(&mut self) { DROP_COUNTER.fetch_add(1, Ordering::SeqCst); } } let _ = catch_unwind_silent(move || { let _: Result<[CountDrop; 4], ()> = super::array_try_from_fn(|idx| { #[allow(clippy::manual_assert)] if idx == 2 { panic!("peek a boo"); } Ok(CountDrop) }); }); assert_eq!(DROP_COUNTER.load(Ordering::SeqCst), 2); } #[test] fn test_extract_bytearray_to_array() { Python::with_gil(|py| { let v: [u8; 33] = py .eval( "bytearray(b'abcabcabcabcabcabcabcabcabcabcabc')", None, None, ) .unwrap() .extract() .unwrap(); assert!(&v == b"abcabcabcabcabcabcabcabcabcabcabc"); }) } #[test] fn test_extract_small_bytearray_to_array() { Python::with_gil(|py| { let v: [u8; 3] = py .eval("bytearray(b'abc')", None, None) .unwrap() .extract() .unwrap(); assert!(&v == b"abc"); }); } #[test] fn test_topyobject_array_conversion() { Python::with_gil(|py| { let array: [f32; 4] = [0.0, -16.0, 16.0, 42.0]; let pyobject = array.to_object(py); let pylist: &PyList = pyobject.extract(py).unwrap(); assert_eq!(pylist[0].extract::().unwrap(), 0.0); assert_eq!(pylist[1].extract::().unwrap(), -16.0); assert_eq!(pylist[2].extract::().unwrap(), 16.0); assert_eq!(pylist[3].extract::().unwrap(), 42.0); }); } #[test] fn test_extract_invalid_sequence_length() { Python::with_gil(|py| { let v: PyResult<[u8; 3]> = py .eval("bytearray(b'abcdefg')", None, None) .unwrap() .extract(); assert_eq!( v.unwrap_err().to_string(), "ValueError: expected a sequence of length 3 (got 7)" ); }) } #[test] fn test_intopy_array_conversion() { Python::with_gil(|py| { let array: [f32; 4] = [0.0, -16.0, 16.0, 42.0]; let pyobject = array.into_py(py); let pylist: &PyList = pyobject.extract(py).unwrap(); assert_eq!(pylist[0].extract::().unwrap(), 0.0); assert_eq!(pylist[1].extract::().unwrap(), -16.0); assert_eq!(pylist[2].extract::().unwrap(), 16.0); assert_eq!(pylist[3].extract::().unwrap(), 42.0); }); } #[test] fn test_extract_non_iterable_to_array() { Python::with_gil(|py| { let v = py.eval("42", None, None).unwrap(); v.extract::().unwrap(); v.extract::<[i32; 1]>().unwrap_err(); }); } #[cfg(feature = "macros")] #[test] fn test_pyclass_intopy_array_conversion() { #[crate::pyclass(crate = "crate")] struct Foo; Python::with_gil(|py| { let array: [Foo; 8] = [Foo, Foo, Foo, Foo, Foo, Foo, Foo, Foo]; let pyobject = array.into_py(py); let list: &PyList = pyobject.downcast(py).unwrap(); let _cell: &crate::PyCell = list.get_item(4).unwrap().extract().unwrap(); }); } // https://stackoverflow.com/a/59211505 fn catch_unwind_silent(f: F) -> std::thread::Result where F: FnOnce() -> R + panic::UnwindSafe, { let prev_hook = panic::take_hook(); panic::set_hook(Box::new(|_| {})); let result = panic::catch_unwind(f); panic::set_hook(prev_hook); result } } pyo3-0.20.2/src/conversions/std/ipaddr.rs000075500000000000000000000070221046102023000163450ustar 00000000000000use std::net::{IpAddr, Ipv4Addr, Ipv6Addr}; use crate::exceptions::PyValueError; use crate::sync::GILOnceCell; use crate::types::PyType; use crate::{intern, FromPyObject, IntoPy, Py, PyAny, PyObject, PyResult, Python, ToPyObject}; impl FromPyObject<'_> for IpAddr { fn extract(obj: &PyAny) -> PyResult { match obj.getattr(intern!(obj.py(), "packed")) { Ok(packed) => { if let Ok(packed) = packed.extract::<[u8; 4]>() { Ok(IpAddr::V4(Ipv4Addr::from(packed))) } else if let Ok(packed) = packed.extract::<[u8; 16]>() { Ok(IpAddr::V6(Ipv6Addr::from(packed))) } else { Err(PyValueError::new_err("invalid packed length")) } } Err(_) => { // We don't have a .packed attribute, so we try to construct an IP from str(). obj.str()?.to_str()?.parse().map_err(PyValueError::new_err) } } } } impl ToPyObject for Ipv4Addr { fn to_object(&self, py: Python<'_>) -> PyObject { static IPV4_ADDRESS: GILOnceCell> = GILOnceCell::new(); IPV4_ADDRESS .get_or_try_init_type_ref(py, "ipaddress", "IPv4Address") .expect("failed to load ipaddress.IPv4Address") .call1((u32::from_be_bytes(self.octets()),)) .expect("failed to construct ipaddress.IPv4Address") .to_object(py) } } impl ToPyObject for Ipv6Addr { fn to_object(&self, py: Python<'_>) -> PyObject { static IPV6_ADDRESS: GILOnceCell> = GILOnceCell::new(); IPV6_ADDRESS .get_or_try_init_type_ref(py, "ipaddress", "IPv6Address") .expect("failed to load ipaddress.IPv6Address") .call1((u128::from_be_bytes(self.octets()),)) .expect("failed to construct ipaddress.IPv6Address") .to_object(py) } } impl ToPyObject for IpAddr { fn to_object(&self, py: Python<'_>) -> PyObject { match self { IpAddr::V4(ip) => ip.to_object(py), IpAddr::V6(ip) => ip.to_object(py), } } } impl IntoPy for IpAddr { fn into_py(self, py: Python<'_>) -> PyObject { self.to_object(py) } } #[cfg(test)] mod test_ipaddr { use std::str::FromStr; use crate::types::PyString; use super::*; #[test] fn test_roundtrip() { Python::with_gil(|py| { fn roundtrip(py: Python<'_>, ip: &str) { let ip = IpAddr::from_str(ip).unwrap(); let py_cls = if ip.is_ipv4() { "IPv4Address" } else { "IPv6Address" }; let pyobj = ip.into_py(py); let repr = pyobj.as_ref(py).repr().unwrap().to_string_lossy(); assert_eq!(repr, format!("{}('{}')", py_cls, ip)); let ip2: IpAddr = pyobj.extract(py).unwrap(); assert_eq!(ip, ip2); } roundtrip(py, "127.0.0.1"); roundtrip(py, "::1"); roundtrip(py, "0.0.0.0"); }); } #[test] fn test_from_pystring() { Python::with_gil(|py| { let py_str = PyString::new(py, "0:0:0:0:0:0:0:1"); let ip: IpAddr = py_str.to_object(py).extract(py).unwrap(); assert_eq!(ip, IpAddr::from_str("::1").unwrap()); let py_str = PyString::new(py, "invalid"); assert!(py_str.to_object(py).extract::(py).is_err()); }); } } pyo3-0.20.2/src/conversions/std/map.rs000064400000000000000000000126621046102023000156620ustar 00000000000000use std::{cmp, collections, hash}; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{ types::{IntoPyDict, PyDict}, FromPyObject, IntoPy, PyAny, PyErr, PyObject, Python, ToPyObject, }; impl ToPyObject for collections::HashMap where K: hash::Hash + cmp::Eq + ToPyObject, V: ToPyObject, H: hash::BuildHasher, { fn to_object(&self, py: Python<'_>) -> PyObject { IntoPyDict::into_py_dict(self, py).into() } } impl ToPyObject for collections::BTreeMap where K: cmp::Eq + ToPyObject, V: ToPyObject, { fn to_object(&self, py: Python<'_>) -> PyObject { IntoPyDict::into_py_dict(self, py).into() } } impl IntoPy for collections::HashMap where K: hash::Hash + cmp::Eq + IntoPy, V: IntoPy, H: hash::BuildHasher, { fn into_py(self, py: Python<'_>) -> PyObject { let iter = self .into_iter() .map(|(k, v)| (k.into_py(py), v.into_py(py))); IntoPyDict::into_py_dict(iter, py).into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::dict_of(K::type_output(), V::type_output()) } } impl IntoPy for collections::BTreeMap where K: cmp::Eq + IntoPy, V: IntoPy, { fn into_py(self, py: Python<'_>) -> PyObject { let iter = self .into_iter() .map(|(k, v)| (k.into_py(py), v.into_py(py))); IntoPyDict::into_py_dict(iter, py).into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::dict_of(K::type_output(), V::type_output()) } } impl<'source, K, V, S> FromPyObject<'source> for collections::HashMap where K: FromPyObject<'source> + cmp::Eq + hash::Hash, V: FromPyObject<'source>, S: hash::BuildHasher + Default, { fn extract(ob: &'source PyAny) -> Result { let dict: &PyDict = ob.downcast()?; let mut ret = collections::HashMap::with_capacity_and_hasher(dict.len(), S::default()); for (k, v) in dict { ret.insert(K::extract(k)?, V::extract(v)?); } Ok(ret) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { TypeInfo::mapping_of(K::type_input(), V::type_input()) } } impl<'source, K, V> FromPyObject<'source> for collections::BTreeMap where K: FromPyObject<'source> + cmp::Ord, V: FromPyObject<'source>, { fn extract(ob: &'source PyAny) -> Result { let dict: &PyDict = ob.downcast()?; let mut ret = collections::BTreeMap::new(); for (k, v) in dict { ret.insert(K::extract(k)?, V::extract(v)?); } Ok(ret) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { TypeInfo::mapping_of(K::type_input(), V::type_input()) } } #[cfg(test)] mod tests { use super::*; use crate::{IntoPy, PyObject, Python, ToPyObject}; use std::collections::{BTreeMap, HashMap}; #[test] fn test_hashmap_to_python() { Python::with_gil(|py| { let mut map = HashMap::::new(); map.insert(1, 1); let m = map.to_object(py); let py_map: &PyDict = m.downcast(py).unwrap(); assert!(py_map.len() == 1); assert!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap() == 1 ); assert_eq!(map, py_map.extract().unwrap()); }); } #[test] fn test_btreemap_to_python() { Python::with_gil(|py| { let mut map = BTreeMap::::new(); map.insert(1, 1); let m = map.to_object(py); let py_map: &PyDict = m.downcast(py).unwrap(); assert!(py_map.len() == 1); assert!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap() == 1 ); assert_eq!(map, py_map.extract().unwrap()); }); } #[test] fn test_hashmap_into_python() { Python::with_gil(|py| { let mut map = HashMap::::new(); map.insert(1, 1); let m: PyObject = map.into_py(py); let py_map: &PyDict = m.downcast(py).unwrap(); assert!(py_map.len() == 1); assert!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap() == 1 ); }); } #[test] fn test_btreemap_into_py() { Python::with_gil(|py| { let mut map = BTreeMap::::new(); map.insert(1, 1); let m: PyObject = map.into_py(py); let py_map: &PyDict = m.downcast(py).unwrap(); assert!(py_map.len() == 1); assert!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap() == 1 ); }); } } pyo3-0.20.2/src/conversions/std/mod.rs000064400000000000000000000001471046102023000156570ustar 00000000000000mod array; mod ipaddr; mod map; mod num; mod osstr; mod path; mod set; mod slice; mod string; mod vec; pyo3-0.20.2/src/conversions/std/num.rs000064400000000000000000000614161046102023000157050ustar 00000000000000#[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{ exceptions, ffi, FromPyObject, IntoPy, PyAny, PyErr, PyObject, PyResult, Python, ToPyObject, }; use std::convert::TryFrom; use std::num::{ NonZeroI128, NonZeroI16, NonZeroI32, NonZeroI64, NonZeroI8, NonZeroIsize, NonZeroU128, NonZeroU16, NonZeroU32, NonZeroU64, NonZeroU8, NonZeroUsize, }; use std::os::raw::c_long; macro_rules! int_fits_larger_int { ($rust_type:ty, $larger_type:ty) => { impl ToPyObject for $rust_type { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { (*self as $larger_type).into_py(py) } } impl IntoPy for $rust_type { fn into_py(self, py: Python<'_>) -> PyObject { (self as $larger_type).into_py(py) } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { <$larger_type>::type_output() } } impl<'source> FromPyObject<'source> for $rust_type { fn extract(obj: &'source PyAny) -> PyResult { let val: $larger_type = obj.extract()?; <$rust_type>::try_from(val) .map_err(|e| exceptions::PyOverflowError::new_err(e.to_string())) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { <$larger_type>::type_input() } } }; } macro_rules! int_convert_u64_or_i64 { ($rust_type:ty, $pylong_from_ll_or_ull:expr, $pylong_as_ll_or_ull:expr) => { impl ToPyObject for $rust_type { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { unsafe { PyObject::from_owned_ptr(py, $pylong_from_ll_or_ull(*self)) } } } impl IntoPy for $rust_type { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { unsafe { PyObject::from_owned_ptr(py, $pylong_from_ll_or_ull(self)) } } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::builtin("int") } } impl<'source> FromPyObject<'source> for $rust_type { fn extract(ob: &'source PyAny) -> PyResult<$rust_type> { let ptr = ob.as_ptr(); unsafe { let num = ffi::PyNumber_Index(ptr); if num.is_null() { Err(PyErr::fetch(ob.py())) } else { let result = err_if_invalid_value(ob.py(), !0, $pylong_as_ll_or_ull(num)); ffi::Py_DECREF(num); result } } } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { Self::type_output() } } }; } macro_rules! int_fits_c_long { ($rust_type:ty) => { impl ToPyObject for $rust_type { fn to_object(&self, py: Python<'_>) -> PyObject { unsafe { PyObject::from_owned_ptr(py, ffi::PyLong_FromLong(*self as c_long)) } } } impl IntoPy for $rust_type { fn into_py(self, py: Python<'_>) -> PyObject { unsafe { PyObject::from_owned_ptr(py, ffi::PyLong_FromLong(self as c_long)) } } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::builtin("int") } } impl<'source> FromPyObject<'source> for $rust_type { fn extract(obj: &'source PyAny) -> PyResult { let ptr = obj.as_ptr(); let val = unsafe { let num = ffi::PyNumber_Index(ptr); if num.is_null() { Err(PyErr::fetch(obj.py())) } else { let val = err_if_invalid_value(obj.py(), -1, ffi::PyLong_AsLong(num)); ffi::Py_DECREF(num); val } }?; <$rust_type>::try_from(val) .map_err(|e| exceptions::PyOverflowError::new_err(e.to_string())) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { Self::type_output() } } }; } int_fits_c_long!(i8); int_fits_c_long!(u8); int_fits_c_long!(i16); int_fits_c_long!(u16); int_fits_c_long!(i32); // If c_long is 64-bits, we can use more types with int_fits_c_long!: #[cfg(all(target_pointer_width = "64", not(target_os = "windows")))] int_fits_c_long!(u32); #[cfg(any(target_pointer_width = "32", target_os = "windows"))] int_fits_larger_int!(u32, u64); #[cfg(all(target_pointer_width = "64", not(target_os = "windows")))] int_fits_c_long!(i64); // manual implementation for i64 on systems with 32-bit long #[cfg(any(target_pointer_width = "32", target_os = "windows"))] int_convert_u64_or_i64!(i64, ffi::PyLong_FromLongLong, ffi::PyLong_AsLongLong); #[cfg(all(target_pointer_width = "64", not(target_os = "windows")))] int_fits_c_long!(isize); #[cfg(any(target_pointer_width = "32", target_os = "windows"))] int_fits_larger_int!(isize, i64); int_fits_larger_int!(usize, u64); // u64 has a manual implementation as it never fits into signed long int_convert_u64_or_i64!( u64, ffi::PyLong_FromUnsignedLongLong, ffi::PyLong_AsUnsignedLongLong ); #[cfg(not(Py_LIMITED_API))] mod fast_128bit_int_conversion { use super::*; // for 128bit Integers macro_rules! int_convert_128 { ($rust_type: ty, $is_signed: expr) => { impl ToPyObject for $rust_type { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { (*self).into_py(py) } } impl IntoPy for $rust_type { fn into_py(self, py: Python<'_>) -> PyObject { // Always use little endian let bytes = self.to_le_bytes(); unsafe { PyObject::from_owned_ptr( py, ffi::_PyLong_FromByteArray( bytes.as_ptr() as *const std::os::raw::c_uchar, bytes.len(), 1, $is_signed, ), ) } } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::builtin("int") } } impl<'source> FromPyObject<'source> for $rust_type { fn extract(ob: &'source PyAny) -> PyResult<$rust_type> { let num = unsafe { PyObject::from_owned_ptr_or_err(ob.py(), ffi::PyNumber_Index(ob.as_ptr()))? }; let mut buffer = [0; std::mem::size_of::<$rust_type>()]; crate::err::error_on_minusone(ob.py(), unsafe { ffi::_PyLong_AsByteArray( num.as_ptr() as *mut ffi::PyLongObject, buffer.as_mut_ptr(), buffer.len(), 1, $is_signed, ) })?; Ok(<$rust_type>::from_le_bytes(buffer)) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { Self::type_output() } } }; } int_convert_128!(i128, 1); int_convert_128!(u128, 0); } // For ABI3 we implement the conversion manually. #[cfg(Py_LIMITED_API)] mod slow_128bit_int_conversion { use super::*; const SHIFT: usize = 64; // for 128bit Integers macro_rules! int_convert_128 { ($rust_type: ty, $half_type: ty) => { impl ToPyObject for $rust_type { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { (*self).into_py(py) } } impl IntoPy for $rust_type { fn into_py(self, py: Python<'_>) -> PyObject { let lower = (self as u64).into_py(py); let upper = ((self >> SHIFT) as $half_type).into_py(py); let shift = SHIFT.into_py(py); unsafe { let shifted = PyObject::from_owned_ptr( py, ffi::PyNumber_Lshift(upper.as_ptr(), shift.as_ptr()), ); PyObject::from_owned_ptr( py, ffi::PyNumber_Or(shifted.as_ptr(), lower.as_ptr()), ) } } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::builtin("int") } } impl<'source> FromPyObject<'source> for $rust_type { fn extract(ob: &'source PyAny) -> PyResult<$rust_type> { let py = ob.py(); unsafe { let lower = err_if_invalid_value( py, -1 as _, ffi::PyLong_AsUnsignedLongLongMask(ob.as_ptr()), )? as $rust_type; let shift = SHIFT.into_py(py); let shifted = PyObject::from_owned_ptr_or_err( py, ffi::PyNumber_Rshift(ob.as_ptr(), shift.as_ptr()), )?; let upper: $half_type = shifted.extract(py)?; Ok((<$rust_type>::from(upper) << SHIFT) | lower) } } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { Self::type_output() } } }; } int_convert_128!(i128, i64); int_convert_128!(u128, u64); } fn err_if_invalid_value( py: Python<'_>, invalid_value: T, actual_value: T, ) -> PyResult { if actual_value == invalid_value { if let Some(err) = PyErr::take(py) { return Err(err); } } Ok(actual_value) } macro_rules! nonzero_int_impl { ($nonzero_type:ty, $primitive_type:ty) => { impl ToPyObject for $nonzero_type { fn to_object(&self, py: Python<'_>) -> PyObject { self.get().to_object(py) } } impl IntoPy for $nonzero_type { fn into_py(self, py: Python<'_>) -> PyObject { self.get().into_py(py) } } impl<'source> FromPyObject<'source> for $nonzero_type { fn extract(obj: &'source PyAny) -> PyResult { let val: $primitive_type = obj.extract()?; <$nonzero_type>::try_from(val) .map_err(|_| exceptions::PyValueError::new_err("invalid zero value")) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { <$primitive_type>::type_input() } } }; } nonzero_int_impl!(NonZeroI8, i8); nonzero_int_impl!(NonZeroI16, i16); nonzero_int_impl!(NonZeroI32, i32); nonzero_int_impl!(NonZeroI64, i64); nonzero_int_impl!(NonZeroI128, i128); nonzero_int_impl!(NonZeroIsize, isize); nonzero_int_impl!(NonZeroU8, u8); nonzero_int_impl!(NonZeroU16, u16); nonzero_int_impl!(NonZeroU32, u32); nonzero_int_impl!(NonZeroU64, u64); nonzero_int_impl!(NonZeroU128, u128); nonzero_int_impl!(NonZeroUsize, usize); #[cfg(test)] mod test_128bit_integers { use super::*; #[cfg(not(target_arch = "wasm32"))] use crate::types::PyDict; #[cfg(not(target_arch = "wasm32"))] use proptest::prelude::*; #[cfg(not(target_arch = "wasm32"))] proptest! { #[test] fn test_i128_roundtrip(x: i128) { Python::with_gil(|py| { let x_py = x.into_py(py); let locals = PyDict::new(py); locals.set_item("x_py", x_py.clone_ref(py)).unwrap(); py.run(&format!("assert x_py == {}", x), None, Some(locals)).unwrap(); let roundtripped: i128 = x_py.extract(py).unwrap(); assert_eq!(x, roundtripped); }) } #[test] fn test_nonzero_i128_roundtrip( x in any::() .prop_filter("Values must not be 0", |x| x != &0) .prop_map(|x| NonZeroI128::new(x).unwrap()) ) { Python::with_gil(|py| { let x_py = x.into_py(py); let locals = PyDict::new(py); locals.set_item("x_py", x_py.clone_ref(py)).unwrap(); py.run(&format!("assert x_py == {}", x), None, Some(locals)).unwrap(); let roundtripped: NonZeroI128 = x_py.extract(py).unwrap(); assert_eq!(x, roundtripped); }) } } #[cfg(not(target_arch = "wasm32"))] proptest! { #[test] fn test_u128_roundtrip(x: u128) { Python::with_gil(|py| { let x_py = x.into_py(py); let locals = PyDict::new(py); locals.set_item("x_py", x_py.clone_ref(py)).unwrap(); py.run(&format!("assert x_py == {}", x), None, Some(locals)).unwrap(); let roundtripped: u128 = x_py.extract(py).unwrap(); assert_eq!(x, roundtripped); }) } #[test] fn test_nonzero_u128_roundtrip( x in any::() .prop_filter("Values must not be 0", |x| x != &0) .prop_map(|x| NonZeroU128::new(x).unwrap()) ) { Python::with_gil(|py| { let x_py = x.into_py(py); let locals = PyDict::new(py); locals.set_item("x_py", x_py.clone_ref(py)).unwrap(); py.run(&format!("assert x_py == {}", x), None, Some(locals)).unwrap(); let roundtripped: NonZeroU128 = x_py.extract(py).unwrap(); assert_eq!(x, roundtripped); }) } } #[test] fn test_i128_max() { Python::with_gil(|py| { let v = std::i128::MAX; let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert_eq!(v as u128, obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); }) } #[test] fn test_i128_min() { Python::with_gil(|py| { let v = std::i128::MIN; let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); assert!(obj.extract::(py).is_err()); }) } #[test] fn test_u128_max() { Python::with_gil(|py| { let v = std::u128::MAX; let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); }) } #[test] fn test_i128_overflow() { Python::with_gil(|py| { let obj = py.eval("(1 << 130) * -1", None, None).unwrap(); let err = obj.extract::().unwrap_err(); assert!(err.is_instance_of::(py)); }) } #[test] fn test_u128_overflow() { Python::with_gil(|py| { let obj = py.eval("1 << 130", None, None).unwrap(); let err = obj.extract::().unwrap_err(); assert!(err.is_instance_of::(py)); }) } #[test] fn test_nonzero_i128_max() { Python::with_gil(|py| { let v = NonZeroI128::new(std::i128::MAX).unwrap(); let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert_eq!( NonZeroU128::new(v.get() as u128).unwrap(), obj.extract::(py).unwrap() ); assert!(obj.extract::(py).is_err()); }) } #[test] fn test_nonzero_i128_min() { Python::with_gil(|py| { let v = NonZeroI128::new(std::i128::MIN).unwrap(); let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); assert!(obj.extract::(py).is_err()); }) } #[test] fn test_nonzero_u128_max() { Python::with_gil(|py| { let v = NonZeroU128::new(std::u128::MAX).unwrap(); let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); }) } #[test] fn test_nonzero_i128_overflow() { Python::with_gil(|py| { let obj = py.eval("(1 << 130) * -1", None, None).unwrap(); let err = obj.extract::().unwrap_err(); assert!(err.is_instance_of::(py)); }) } #[test] fn test_nonzero_u128_overflow() { Python::with_gil(|py| { let obj = py.eval("1 << 130", None, None).unwrap(); let err = obj.extract::().unwrap_err(); assert!(err.is_instance_of::(py)); }) } #[test] fn test_nonzero_i128_zero_value() { Python::with_gil(|py| { let obj = py.eval("0", None, None).unwrap(); let err = obj.extract::().unwrap_err(); assert!(err.is_instance_of::(py)); }) } #[test] fn test_nonzero_u128_zero_value() { Python::with_gil(|py| { let obj = py.eval("0", None, None).unwrap(); let err = obj.extract::().unwrap_err(); assert!(err.is_instance_of::(py)); }) } } #[cfg(test)] mod tests { use crate::Python; use crate::ToPyObject; use std::num::*; #[test] fn test_u32_max() { Python::with_gil(|py| { let v = std::u32::MAX; let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert_eq!(u64::from(v), obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); }); } #[test] fn test_i64_max() { Python::with_gil(|py| { let v = std::i64::MAX; let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert_eq!(v as u64, obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); }); } #[test] fn test_i64_min() { Python::with_gil(|py| { let v = std::i64::MIN; let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); assert!(obj.extract::(py).is_err()); }); } #[test] fn test_u64_max() { Python::with_gil(|py| { let v = std::u64::MAX; let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); }); } macro_rules! test_common ( ($test_mod_name:ident, $t:ty) => ( mod $test_mod_name { use crate::exceptions; use crate::ToPyObject; use crate::Python; #[test] fn from_py_string_type_error() { Python::with_gil(|py| { let obj = ("123").to_object(py); let err = obj.extract::<$t>(py).unwrap_err(); assert!(err.is_instance_of::(py)); }); } #[test] fn from_py_float_type_error() { Python::with_gil(|py| { let obj = (12.3).to_object(py); let err = obj.extract::<$t>(py).unwrap_err(); assert!(err.is_instance_of::(py));}); } #[test] fn to_py_object_and_back() { Python::with_gil(|py| { let val = 123 as $t; let obj = val.to_object(py); assert_eq!(obj.extract::<$t>(py).unwrap(), val as $t);}); } } ) ); test_common!(i8, i8); test_common!(u8, u8); test_common!(i16, i16); test_common!(u16, u16); test_common!(i32, i32); test_common!(u32, u32); test_common!(i64, i64); test_common!(u64, u64); test_common!(isize, isize); test_common!(usize, usize); test_common!(i128, i128); test_common!(u128, u128); #[test] fn test_nonzero_u32_max() { Python::with_gil(|py| { let v = NonZeroU32::new(std::u32::MAX).unwrap(); let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert_eq!(NonZeroU64::from(v), obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); }); } #[test] fn test_nonzero_i64_max() { Python::with_gil(|py| { let v = NonZeroI64::new(std::i64::MAX).unwrap(); let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert_eq!( NonZeroU64::new(v.get() as u64).unwrap(), obj.extract::(py).unwrap() ); assert!(obj.extract::(py).is_err()); }); } #[test] fn test_nonzero_i64_min() { Python::with_gil(|py| { let v = NonZeroI64::new(std::i64::MIN).unwrap(); let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); assert!(obj.extract::(py).is_err()); }); } #[test] fn test_nonzero_u64_max() { Python::with_gil(|py| { let v = NonZeroU64::new(std::u64::MAX).unwrap(); let obj = v.to_object(py); assert_eq!(v, obj.extract::(py).unwrap()); assert!(obj.extract::(py).is_err()); }); } macro_rules! test_nonzero_common ( ($test_mod_name:ident, $t:ty) => ( mod $test_mod_name { use crate::exceptions; use crate::ToPyObject; use crate::Python; use std::num::*; #[test] fn from_py_string_type_error() { Python::with_gil(|py| { let obj = ("123").to_object(py); let err = obj.extract::<$t>(py).unwrap_err(); assert!(err.is_instance_of::(py)); }); } #[test] fn from_py_float_type_error() { Python::with_gil(|py| { let obj = (12.3).to_object(py); let err = obj.extract::<$t>(py).unwrap_err(); assert!(err.is_instance_of::(py));}); } #[test] fn to_py_object_and_back() { Python::with_gil(|py| { let val = <$t>::new(123).unwrap(); let obj = val.to_object(py); assert_eq!(obj.extract::<$t>(py).unwrap(), val);}); } } ) ); test_nonzero_common!(nonzero_i8, NonZeroI8); test_nonzero_common!(nonzero_u8, NonZeroU8); test_nonzero_common!(nonzero_i16, NonZeroI16); test_nonzero_common!(nonzero_u16, NonZeroU16); test_nonzero_common!(nonzero_i32, NonZeroI32); test_nonzero_common!(nonzero_u32, NonZeroU32); test_nonzero_common!(nonzero_i64, NonZeroI64); test_nonzero_common!(nonzero_u64, NonZeroU64); test_nonzero_common!(nonzero_isize, NonZeroIsize); test_nonzero_common!(nonzero_usize, NonZeroUsize); test_nonzero_common!(nonzero_i128, NonZeroI128); test_nonzero_common!(nonzero_u128, NonZeroU128); } pyo3-0.20.2/src/conversions/std/osstr.rs000064400000000000000000000164621046102023000162610ustar 00000000000000use crate::types::PyString; use crate::{ffi, FromPyObject, IntoPy, PyAny, PyObject, PyResult, Python, ToPyObject}; use std::borrow::Cow; use std::ffi::{OsStr, OsString}; #[cfg(not(windows))] use std::os::raw::c_char; impl ToPyObject for OsStr { fn to_object(&self, py: Python<'_>) -> PyObject { // If the string is UTF-8, take the quick and easy shortcut if let Some(valid_utf8_path) = self.to_str() { return valid_utf8_path.to_object(py); } // All targets besides windows support the std::os::unix::ffi::OsStrExt API: // https://doc.rust-lang.org/src/std/sys_common/mod.rs.html#59 #[cfg(not(windows))] { #[cfg(target_os = "wasi")] let bytes = std::os::wasi::ffi::OsStrExt::as_bytes(self); #[cfg(not(target_os = "wasi"))] let bytes = std::os::unix::ffi::OsStrExt::as_bytes(self); let ptr = bytes.as_ptr() as *const c_char; let len = bytes.len() as ffi::Py_ssize_t; unsafe { // DecodeFSDefault automatically chooses an appropriate decoding mechanism to // parse os strings losslessly (i.e. surrogateescape most of the time) let pystring = ffi::PyUnicode_DecodeFSDefaultAndSize(ptr, len); PyObject::from_owned_ptr(py, pystring) } } #[cfg(windows)] { let wstr: Vec = std::os::windows::ffi::OsStrExt::encode_wide(self).collect(); unsafe { // This will not panic because the data from encode_wide is well-formed Windows // string data PyObject::from_owned_ptr( py, ffi::PyUnicode_FromWideChar(wstr.as_ptr(), wstr.len() as ffi::Py_ssize_t), ) } } } } // There's no FromPyObject implementation for &OsStr because albeit possible on Unix, this would // be impossible to implement on Windows. Hence it's omitted entirely impl FromPyObject<'_> for OsString { fn extract(ob: &PyAny) -> PyResult { let pystring: &PyString = ob.downcast()?; #[cfg(not(windows))] { // Decode from Python's lossless bytes string representation back into raw bytes let fs_encoded_bytes = unsafe { crate::Py::::from_owned_ptr( ob.py(), ffi::PyUnicode_EncodeFSDefault(pystring.as_ptr()), ) }; // Create an OsStr view into the raw bytes from Python #[cfg(target_os = "wasi")] let os_str: &OsStr = std::os::wasi::ffi::OsStrExt::from_bytes( fs_encoded_bytes.as_ref(ob.py()).as_bytes(), ); #[cfg(not(target_os = "wasi"))] let os_str: &OsStr = std::os::unix::ffi::OsStrExt::from_bytes( fs_encoded_bytes.as_ref(ob.py()).as_bytes(), ); Ok(os_str.to_os_string()) } #[cfg(windows)] { // Take the quick and easy shortcut if UTF-8 if let Ok(utf8_string) = pystring.to_str() { return Ok(utf8_string.to_owned().into()); } // Get an owned allocated wide char buffer from PyString, which we have to deallocate // ourselves let size = unsafe { ffi::PyUnicode_AsWideChar(pystring.as_ptr(), std::ptr::null_mut(), 0) }; crate::err::error_on_minusone(ob.py(), size)?; let mut buffer = vec![0; size as usize]; let bytes_read = unsafe { ffi::PyUnicode_AsWideChar(pystring.as_ptr(), buffer.as_mut_ptr(), size) }; assert_eq!(bytes_read, size); // Copy wide char buffer into OsString let os_string = std::os::windows::ffi::OsStringExt::from_wide(&buffer); Ok(os_string) } } } impl IntoPy for &'_ OsStr { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { self.to_object(py) } } impl ToPyObject for Cow<'_, OsStr> { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { (self as &OsStr).to_object(py) } } impl IntoPy for Cow<'_, OsStr> { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { self.to_object(py) } } impl ToPyObject for OsString { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { (self as &OsStr).to_object(py) } } impl IntoPy for OsString { fn into_py(self, py: Python<'_>) -> PyObject { self.to_object(py) } } impl<'a> IntoPy for &'a OsString { fn into_py(self, py: Python<'_>) -> PyObject { self.to_object(py) } } #[cfg(test)] mod tests { use crate::{types::PyString, IntoPy, PyObject, Python, ToPyObject}; use std::fmt::Debug; use std::{ borrow::Cow, ffi::{OsStr, OsString}, }; #[test] #[cfg(not(windows))] fn test_non_utf8_conversion() { Python::with_gil(|py| { #[cfg(not(target_os = "wasi"))] use std::os::unix::ffi::OsStrExt; #[cfg(target_os = "wasi")] use std::os::wasi::ffi::OsStrExt; // this is not valid UTF-8 let payload = &[250, 251, 252, 253, 254, 255, 0, 255]; let os_str = OsStr::from_bytes(payload); // do a roundtrip into Pythonland and back and compare let py_str: PyObject = os_str.into_py(py); let os_str_2: OsString = py_str.extract(py).unwrap(); assert_eq!(os_str, os_str_2); }); } #[test] fn test_topyobject_roundtrip() { Python::with_gil(|py| { fn test_roundtrip + Debug>(py: Python<'_>, obj: T) { let pyobject = obj.to_object(py); let pystring: &PyString = pyobject.extract(py).unwrap(); assert_eq!(pystring.to_string_lossy(), obj.as_ref().to_string_lossy()); let roundtripped_obj: OsString = pystring.extract().unwrap(); assert_eq!(obj.as_ref(), roundtripped_obj.as_os_str()); } let os_str = OsStr::new("Hello\0\n🐍"); test_roundtrip::<&OsStr>(py, os_str); test_roundtrip::>(py, Cow::Borrowed(os_str)); test_roundtrip::>(py, Cow::Owned(os_str.to_os_string())); test_roundtrip::(py, os_str.to_os_string()); }); } #[test] fn test_intopy_roundtrip() { Python::with_gil(|py| { fn test_roundtrip + AsRef + Debug + Clone>( py: Python<'_>, obj: T, ) { let pyobject = obj.clone().into_py(py); let pystring: &PyString = pyobject.extract(py).unwrap(); assert_eq!(pystring.to_string_lossy(), obj.as_ref().to_string_lossy()); let roundtripped_obj: OsString = pystring.extract().unwrap(); assert!(obj.as_ref() == roundtripped_obj.as_os_str()); } let os_str = OsStr::new("Hello\0\n🐍"); test_roundtrip::<&OsStr>(py, os_str); test_roundtrip::(py, os_str.to_os_string()); test_roundtrip::<&OsString>(py, &os_str.to_os_string()); }) } } pyo3-0.20.2/src/conversions/std/path.rs000064400000000000000000000101661046102023000160360ustar 00000000000000use crate::{ ffi, FromPyObject, FromPyPointer, IntoPy, PyAny, PyObject, PyResult, Python, ToPyObject, }; use std::borrow::Cow; use std::ffi::OsString; use std::path::{Path, PathBuf}; impl ToPyObject for Path { fn to_object(&self, py: Python<'_>) -> PyObject { self.as_os_str().to_object(py) } } // See osstr.rs for why there's no FromPyObject impl for &Path impl FromPyObject<'_> for PathBuf { fn extract(ob: &PyAny) -> PyResult { // We use os.fspath to get the underlying path as bytes or str let path = unsafe { PyAny::from_owned_ptr_or_err(ob.py(), ffi::PyOS_FSPath(ob.as_ptr())) }?; Ok(OsString::extract(path)?.into()) } } impl<'a> IntoPy for &'a Path { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { self.as_os_str().to_object(py) } } impl<'a> ToPyObject for Cow<'a, Path> { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { self.as_os_str().to_object(py) } } impl<'a> IntoPy for Cow<'a, Path> { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { self.to_object(py) } } impl ToPyObject for PathBuf { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { self.as_os_str().to_object(py) } } impl IntoPy for PathBuf { fn into_py(self, py: Python<'_>) -> PyObject { self.into_os_string().to_object(py) } } impl<'a> IntoPy for &'a PathBuf { fn into_py(self, py: Python<'_>) -> PyObject { self.as_os_str().to_object(py) } } #[cfg(test)] mod tests { use crate::{types::PyString, IntoPy, PyObject, Python, ToPyObject}; use std::borrow::Cow; use std::fmt::Debug; use std::path::{Path, PathBuf}; #[test] #[cfg(not(windows))] fn test_non_utf8_conversion() { Python::with_gil(|py| { use std::ffi::OsStr; #[cfg(not(target_os = "wasi"))] use std::os::unix::ffi::OsStrExt; #[cfg(target_os = "wasi")] use std::os::wasi::ffi::OsStrExt; // this is not valid UTF-8 let payload = &[250, 251, 252, 253, 254, 255, 0, 255]; let path = Path::new(OsStr::from_bytes(payload)); // do a roundtrip into Pythonland and back and compare let py_str: PyObject = path.into_py(py); let path_2: PathBuf = py_str.extract(py).unwrap(); assert_eq!(path, path_2); }); } #[test] fn test_topyobject_roundtrip() { Python::with_gil(|py| { fn test_roundtrip + Debug>(py: Python<'_>, obj: T) { let pyobject = obj.to_object(py); let pystring: &PyString = pyobject.extract(py).unwrap(); assert_eq!(pystring.to_string_lossy(), obj.as_ref().to_string_lossy()); let roundtripped_obj: PathBuf = pystring.extract().unwrap(); assert_eq!(obj.as_ref(), roundtripped_obj.as_path()); } let path = Path::new("Hello\0\n🐍"); test_roundtrip::<&Path>(py, path); test_roundtrip::>(py, Cow::Borrowed(path)); test_roundtrip::>(py, Cow::Owned(path.to_path_buf())); test_roundtrip::(py, path.to_path_buf()); }); } #[test] fn test_intopy_roundtrip() { Python::with_gil(|py| { fn test_roundtrip + AsRef + Debug + Clone>( py: Python<'_>, obj: T, ) { let pyobject = obj.clone().into_py(py); let pystring: &PyString = pyobject.extract(py).unwrap(); assert_eq!(pystring.to_string_lossy(), obj.as_ref().to_string_lossy()); let roundtripped_obj: PathBuf = pystring.extract().unwrap(); assert_eq!(obj.as_ref(), roundtripped_obj.as_path()); } let path = Path::new("Hello\0\n🐍"); test_roundtrip::<&Path>(py, path); test_roundtrip::(py, path.to_path_buf()); test_roundtrip::<&PathBuf>(py, &path.to_path_buf()); }) } } pyo3-0.20.2/src/conversions/std/set.rs000064400000000000000000000103261046102023000156730ustar 00000000000000use std::{cmp, collections, hash}; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{ types::set::new_from_iter, types::PySet, FromPyObject, IntoPy, PyAny, PyObject, PyResult, Python, ToPyObject, }; impl ToPyObject for collections::HashSet where T: hash::Hash + Eq + ToPyObject, S: hash::BuildHasher + Default, { fn to_object(&self, py: Python<'_>) -> PyObject { new_from_iter(py, self) .expect("Failed to create Python set from HashSet") .into() } } impl ToPyObject for collections::BTreeSet where T: hash::Hash + Eq + ToPyObject, { fn to_object(&self, py: Python<'_>) -> PyObject { new_from_iter(py, self) .expect("Failed to create Python set from BTreeSet") .into() } } impl IntoPy for collections::HashSet where K: IntoPy + Eq + hash::Hash, S: hash::BuildHasher + Default, { fn into_py(self, py: Python<'_>) -> PyObject { new_from_iter(py, self.into_iter().map(|item| item.into_py(py))) .expect("Failed to create Python set from HashSet") .into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::set_of(K::type_output()) } } impl<'source, K, S> FromPyObject<'source> for collections::HashSet where K: FromPyObject<'source> + cmp::Eq + hash::Hash, S: hash::BuildHasher + Default, { fn extract(ob: &'source PyAny) -> PyResult { let set: &PySet = ob.downcast()?; set.iter().map(K::extract).collect() } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { TypeInfo::set_of(K::type_input()) } } impl IntoPy for collections::BTreeSet where K: IntoPy + cmp::Ord, { fn into_py(self, py: Python<'_>) -> PyObject { new_from_iter(py, self.into_iter().map(|item| item.into_py(py))) .expect("Failed to create Python set from BTreeSet") .into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::set_of(K::type_output()) } } impl<'source, K> FromPyObject<'source> for collections::BTreeSet where K: FromPyObject<'source> + cmp::Ord, { fn extract(ob: &'source PyAny) -> PyResult { let set: &PySet = ob.downcast()?; set.iter().map(K::extract).collect() } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { TypeInfo::set_of(K::type_input()) } } #[cfg(test)] mod tests { use super::PySet; use crate::{IntoPy, PyObject, Python, ToPyObject}; use std::collections::{BTreeSet, HashSet}; #[test] fn test_extract_hashset() { Python::with_gil(|py| { let set = PySet::new(py, &[1, 2, 3, 4, 5]).unwrap(); let hash_set: HashSet = set.extract().unwrap(); assert_eq!(hash_set, [1, 2, 3, 4, 5].iter().copied().collect()); }); } #[test] fn test_extract_btreeset() { Python::with_gil(|py| { let set = PySet::new(py, &[1, 2, 3, 4, 5]).unwrap(); let hash_set: BTreeSet = set.extract().unwrap(); assert_eq!(hash_set, [1, 2, 3, 4, 5].iter().copied().collect()); }); } #[test] fn test_set_into_py() { Python::with_gil(|py| { let bt: BTreeSet = [1, 2, 3, 4, 5].iter().cloned().collect(); let hs: HashSet = [1, 2, 3, 4, 5].iter().cloned().collect(); let bto: PyObject = bt.clone().into_py(py); let hso: PyObject = hs.clone().into_py(py); assert_eq!(bt, bto.extract(py).unwrap()); assert_eq!(hs, hso.extract(py).unwrap()); }); } #[test] fn test_set_to_object() { Python::with_gil(|py| { let bt: BTreeSet = [1, 2, 3, 4, 5].iter().cloned().collect(); let hs: HashSet = [1, 2, 3, 4, 5].iter().cloned().collect(); let bto: PyObject = bt.to_object(py); let hso: PyObject = hs.to_object(py); assert_eq!(bt, bto.extract(py).unwrap()); assert_eq!(hs, hso.extract(py).unwrap()); }); } } pyo3-0.20.2/src/conversions/std/slice.rs000064400000000000000000000020671046102023000162020ustar 00000000000000#[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{types::PyBytes, FromPyObject, IntoPy, PyAny, PyObject, PyResult, Python, ToPyObject}; impl<'a> IntoPy for &'a [u8] { fn into_py(self, py: Python<'_>) -> PyObject { PyBytes::new(py, self).to_object(py) } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::builtin("bytes") } } impl<'a> FromPyObject<'a> for &'a [u8] { fn extract(obj: &'a PyAny) -> PyResult { Ok(obj.downcast::()?.as_bytes()) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { Self::type_output() } } #[cfg(test)] mod tests { use crate::FromPyObject; use crate::Python; #[test] fn test_extract_bytes() { Python::with_gil(|py| { let py_bytes = py.eval("b'Hello Python'", None, None).unwrap(); let bytes: &[u8] = FromPyObject::extract(py_bytes).unwrap(); assert_eq!(bytes, b"Hello Python"); }); } } pyo3-0.20.2/src/conversions/std/string.rs000064400000000000000000000153501046102023000164100ustar 00000000000000use std::borrow::Cow; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{ types::PyString, FromPyObject, IntoPy, Py, PyAny, PyObject, PyResult, Python, ToPyObject, }; /// Converts a Rust `str` to a Python object. /// See `PyString::new` for details on the conversion. impl ToPyObject for str { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { PyString::new(py, self).into() } } impl<'a> IntoPy for &'a str { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { PyString::new(py, self).into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { ::type_output() } } impl<'a> IntoPy> for &'a str { #[inline] fn into_py(self, py: Python<'_>) -> Py { PyString::new(py, self).into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { ::type_output() } } /// Converts a Rust `Cow<'_, str>` to a Python object. /// See `PyString::new` for details on the conversion. impl ToPyObject for Cow<'_, str> { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { PyString::new(py, self).into() } } impl IntoPy for Cow<'_, str> { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { self.to_object(py) } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { ::type_output() } } /// Converts a Rust `String` to a Python object. /// See `PyString::new` for details on the conversion. impl ToPyObject for String { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { PyString::new(py, self).into() } } impl ToPyObject for char { fn to_object(&self, py: Python<'_>) -> PyObject { self.into_py(py) } } impl IntoPy for char { fn into_py(self, py: Python<'_>) -> PyObject { let mut bytes = [0u8; 4]; PyString::new(py, self.encode_utf8(&mut bytes)).into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { ::type_output() } } impl IntoPy for String { fn into_py(self, py: Python<'_>) -> PyObject { PyString::new(py, &self).into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::builtin("str") } } impl<'a> IntoPy for &'a String { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { PyString::new(py, self).into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { ::type_output() } } /// Allows extracting strings from Python objects. /// Accepts Python `str` and `unicode` objects. impl<'source> FromPyObject<'source> for &'source str { fn extract(ob: &'source PyAny) -> PyResult { ob.downcast::()?.to_str() } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { ::type_input() } } /// Allows extracting strings from Python objects. /// Accepts Python `str` and `unicode` objects. impl FromPyObject<'_> for String { fn extract(obj: &PyAny) -> PyResult { obj.downcast::()?.to_str().map(ToOwned::to_owned) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { Self::type_output() } } impl FromPyObject<'_> for char { fn extract(obj: &PyAny) -> PyResult { let s = obj.downcast::()?.to_str()?; let mut iter = s.chars(); if let (Some(ch), None) = (iter.next(), iter.next()) { Ok(ch) } else { Err(crate::exceptions::PyValueError::new_err( "expected a string of length 1", )) } } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { ::type_input() } } #[cfg(test)] mod tests { use crate::Python; use crate::{FromPyObject, IntoPy, PyObject, ToPyObject}; use std::borrow::Cow; #[test] fn test_cow_into_py() { Python::with_gil(|py| { let s = "Hello Python"; let py_string: PyObject = Cow::Borrowed(s).into_py(py); assert_eq!(s, py_string.extract::<&str>(py).unwrap()); let py_string: PyObject = Cow::::Owned(s.into()).into_py(py); assert_eq!(s, py_string.extract::<&str>(py).unwrap()); }) } #[test] fn test_cow_to_object() { Python::with_gil(|py| { let s = "Hello Python"; let py_string = Cow::Borrowed(s).to_object(py); assert_eq!(s, py_string.extract::<&str>(py).unwrap()); let py_string = Cow::::Owned(s.into()).to_object(py); assert_eq!(s, py_string.extract::<&str>(py).unwrap()); }) } #[test] fn test_non_bmp() { Python::with_gil(|py| { let s = "\u{1F30F}"; let py_string = s.to_object(py); assert_eq!(s, py_string.extract::(py).unwrap()); }) } #[test] fn test_extract_str() { Python::with_gil(|py| { let s = "Hello Python"; let py_string = s.to_object(py); let s2: &str = FromPyObject::extract(py_string.as_ref(py)).unwrap(); assert_eq!(s, s2); }) } #[test] fn test_extract_char() { Python::with_gil(|py| { let ch = '😃'; let py_string = ch.to_object(py); let ch2: char = FromPyObject::extract(py_string.as_ref(py)).unwrap(); assert_eq!(ch, ch2); }) } #[test] fn test_extract_char_err() { Python::with_gil(|py| { let s = "Hello Python"; let py_string = s.to_object(py); let err: crate::PyResult = FromPyObject::extract(py_string.as_ref(py)); assert!(err .unwrap_err() .to_string() .contains("expected a string of length 1")); }) } #[test] fn test_string_into_py() { Python::with_gil(|py| { let s = "Hello Python"; let s2 = s.to_owned(); let s3 = &s2; assert_eq!( s, IntoPy::::into_py(s3, py) .extract::<&str>(py) .unwrap() ); assert_eq!( s, IntoPy::::into_py(s2, py) .extract::<&str>(py) .unwrap() ); assert_eq!( s, IntoPy::::into_py(s, py) .extract::<&str>(py) .unwrap() ); }) } } pyo3-0.20.2/src/conversions/std/vec.rs000064400000000000000000000017061046102023000156570ustar 00000000000000#[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::types::list::new_from_iter; use crate::{IntoPy, PyObject, Python, ToPyObject}; impl ToPyObject for [T] where T: ToPyObject, { fn to_object(&self, py: Python<'_>) -> PyObject { let mut iter = self.iter().map(|e| e.to_object(py)); let list = new_from_iter(py, &mut iter); list.into() } } impl ToPyObject for Vec where T: ToPyObject, { fn to_object(&self, py: Python<'_>) -> PyObject { self.as_slice().to_object(py) } } impl IntoPy for Vec where T: IntoPy, { fn into_py(self, py: Python<'_>) -> PyObject { let mut iter = self.into_iter().map(|e| e.into_py(py)); let list = new_from_iter(py, &mut iter); list.into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::list_of(T::type_output()) } } pyo3-0.20.2/src/derive_utils.rs000064400000000000000000000016751046102023000144430ustar 00000000000000//! Functionality for the code generated by the derive backend use crate::{types::PyModule, Python}; /// Enum to abstract over the arguments of Python function wrappers. pub enum PyFunctionArguments<'a> { Python(Python<'a>), PyModule(&'a PyModule), } impl<'a> PyFunctionArguments<'a> { pub fn into_py_and_maybe_module(self) -> (Python<'a>, Option<&'a PyModule>) { match self { PyFunctionArguments::Python(py) => (py, None), PyFunctionArguments::PyModule(module) => { let py = module.py(); (py, Some(module)) } } } } impl<'a> From> for PyFunctionArguments<'a> { fn from(py: Python<'a>) -> PyFunctionArguments<'a> { PyFunctionArguments::Python(py) } } impl<'a> From<&'a PyModule> for PyFunctionArguments<'a> { fn from(module: &'a PyModule) -> PyFunctionArguments<'a> { PyFunctionArguments::PyModule(module) } } pyo3-0.20.2/src/err/err_state.rs000064400000000000000000000172111046102023000145160ustar 00000000000000use crate::{ exceptions::{PyBaseException, PyTypeError}, ffi, types::{PyTraceback, PyType}, IntoPy, Py, PyAny, PyObject, PyTypeInfo, Python, }; #[derive(Clone)] pub(crate) struct PyErrStateNormalized { #[cfg(not(Py_3_12))] ptype: Py, pub pvalue: Py, #[cfg(not(Py_3_12))] ptraceback: Option>, } impl PyErrStateNormalized { #[cfg(not(Py_3_12))] pub(crate) fn ptype<'py>(&'py self, py: Python<'py>) -> &'py PyType { self.ptype.as_ref(py) } #[cfg(Py_3_12)] pub(crate) fn ptype<'py>(&'py self, py: Python<'py>) -> &'py PyType { self.pvalue.as_ref(py).get_type() } #[cfg(not(Py_3_12))] pub(crate) fn ptraceback<'py>(&'py self, py: Python<'py>) -> Option<&'py PyTraceback> { self.ptraceback .as_ref() .map(|traceback| traceback.as_ref(py)) } #[cfg(Py_3_12)] pub(crate) fn ptraceback<'py>(&'py self, py: Python<'py>) -> Option<&'py PyTraceback> { unsafe { py.from_owned_ptr_or_opt(ffi::PyException_GetTraceback(self.pvalue.as_ptr())) } } #[cfg(Py_3_12)] pub(crate) fn take(py: Python<'_>) -> Option { unsafe { Py::from_owned_ptr_or_opt(py, ffi::PyErr_GetRaisedException()) } .map(|pvalue| PyErrStateNormalized { pvalue }) } #[cfg(not(Py_3_12))] unsafe fn from_normalized_ffi_tuple( py: Python<'_>, ptype: *mut ffi::PyObject, pvalue: *mut ffi::PyObject, ptraceback: *mut ffi::PyObject, ) -> Self { PyErrStateNormalized { ptype: Py::from_owned_ptr_or_opt(py, ptype).expect("Exception type missing"), pvalue: Py::from_owned_ptr_or_opt(py, pvalue).expect("Exception value missing"), ptraceback: Py::from_owned_ptr_or_opt(py, ptraceback), } } } pub(crate) struct PyErrStateLazyFnOutput { pub(crate) ptype: PyObject, pub(crate) pvalue: PyObject, } pub(crate) type PyErrStateLazyFn = dyn for<'py> FnOnce(Python<'py>) -> PyErrStateLazyFnOutput + Send + Sync; pub(crate) enum PyErrState { Lazy(Box), #[cfg(not(Py_3_12))] FfiTuple { ptype: PyObject, pvalue: Option, ptraceback: Option, }, Normalized(PyErrStateNormalized), } /// Helper conversion trait that allows to use custom arguments for lazy exception construction. pub trait PyErrArguments: Send + Sync { /// Arguments for exception fn arguments(self, py: Python<'_>) -> PyObject; } impl PyErrArguments for T where T: IntoPy + Send + Sync, { fn arguments(self, py: Python<'_>) -> PyObject { self.into_py(py) } } impl PyErrState { pub(crate) fn lazy(ptype: &PyAny, args: impl PyErrArguments + 'static) -> Self { let ptype = ptype.into(); PyErrState::Lazy(Box::new(move |py| PyErrStateLazyFnOutput { ptype, pvalue: args.arguments(py), })) } pub(crate) fn normalized(pvalue: &PyBaseException) -> Self { Self::Normalized(PyErrStateNormalized { #[cfg(not(Py_3_12))] ptype: pvalue.get_type().into(), pvalue: pvalue.into(), #[cfg(not(Py_3_12))] ptraceback: unsafe { Py::from_owned_ptr_or_opt( pvalue.py(), ffi::PyException_GetTraceback(pvalue.as_ptr()), ) }, }) } pub(crate) fn normalize(self, py: Python<'_>) -> PyErrStateNormalized { match self { #[cfg(not(Py_3_12))] PyErrState::Lazy(lazy) => { let (ptype, pvalue, ptraceback) = lazy_into_normalized_ffi_tuple(py, lazy); unsafe { PyErrStateNormalized::from_normalized_ffi_tuple(py, ptype, pvalue, ptraceback) } } #[cfg(Py_3_12)] PyErrState::Lazy(lazy) => { // To keep the implementation simple, just write the exception into the interpreter, // which will cause it to be normalized raise_lazy(py, lazy); PyErrStateNormalized::take(py) .expect("exception missing after writing to the interpreter") } #[cfg(not(Py_3_12))] PyErrState::FfiTuple { ptype, pvalue, ptraceback, } => { let mut ptype = ptype.into_ptr(); let mut pvalue = pvalue.map_or(std::ptr::null_mut(), Py::into_ptr); let mut ptraceback = ptraceback.map_or(std::ptr::null_mut(), Py::into_ptr); unsafe { ffi::PyErr_NormalizeException(&mut ptype, &mut pvalue, &mut ptraceback); PyErrStateNormalized::from_normalized_ffi_tuple(py, ptype, pvalue, ptraceback) } } PyErrState::Normalized(normalized) => normalized, } } #[cfg(not(Py_3_12))] pub(crate) fn restore(self, py: Python<'_>) { let (ptype, pvalue, ptraceback) = match self { PyErrState::Lazy(lazy) => lazy_into_normalized_ffi_tuple(py, lazy), PyErrState::FfiTuple { ptype, pvalue, ptraceback, } => ( ptype.into_ptr(), pvalue.map_or(std::ptr::null_mut(), Py::into_ptr), ptraceback.map_or(std::ptr::null_mut(), Py::into_ptr), ), PyErrState::Normalized(PyErrStateNormalized { ptype, pvalue, ptraceback, }) => ( ptype.into_ptr(), pvalue.into_ptr(), ptraceback.map_or(std::ptr::null_mut(), Py::into_ptr), ), }; unsafe { ffi::PyErr_Restore(ptype, pvalue, ptraceback) } } #[cfg(Py_3_12)] pub(crate) fn restore(self, py: Python<'_>) { match self { PyErrState::Lazy(lazy) => raise_lazy(py, lazy), PyErrState::Normalized(PyErrStateNormalized { pvalue }) => unsafe { ffi::PyErr_SetRaisedException(pvalue.into_ptr()) }, } } } #[cfg(not(Py_3_12))] fn lazy_into_normalized_ffi_tuple( py: Python<'_>, lazy: Box, ) -> (*mut ffi::PyObject, *mut ffi::PyObject, *mut ffi::PyObject) { // To be consistent with 3.12 logic, go via raise_lazy, but also then normalize // the resulting exception raise_lazy(py, lazy); let mut ptype = std::ptr::null_mut(); let mut pvalue = std::ptr::null_mut(); let mut ptraceback = std::ptr::null_mut(); unsafe { ffi::PyErr_Fetch(&mut ptype, &mut pvalue, &mut ptraceback); ffi::PyErr_NormalizeException(&mut ptype, &mut pvalue, &mut ptraceback); } (ptype, pvalue, ptraceback) } /// Raises a "lazy" exception state into the Python interpreter. /// /// In principle this could be split in two; first a function to create an exception /// in a normalized state, and then a call to `PyErr_SetRaisedException` to raise it. /// /// This would require either moving some logic from C to Rust, or requesting a new /// API in CPython. fn raise_lazy(py: Python<'_>, lazy: Box) { let PyErrStateLazyFnOutput { ptype, pvalue } = lazy(py); unsafe { if ffi::PyExceptionClass_Check(ptype.as_ptr()) == 0 { ffi::PyErr_SetString( PyTypeError::type_object_raw(py).cast(), "exceptions must derive from BaseException\0" .as_ptr() .cast(), ) } else { ffi::PyErr_SetObject(ptype.as_ptr(), pvalue.as_ptr()) } } } pyo3-0.20.2/src/err/impls.rs000064400000000000000000000147201046102023000136540ustar 00000000000000use crate::{err::PyErrArguments, exceptions, IntoPy, PyErr, PyObject, Python}; use std::io; /// Convert `PyErr` to `io::Error` impl From for io::Error { fn from(err: PyErr) -> Self { let kind = Python::with_gil(|py| { if err.is_instance_of::(py) { io::ErrorKind::BrokenPipe } else if err.is_instance_of::(py) { io::ErrorKind::ConnectionRefused } else if err.is_instance_of::(py) { io::ErrorKind::ConnectionAborted } else if err.is_instance_of::(py) { io::ErrorKind::ConnectionReset } else if err.is_instance_of::(py) { io::ErrorKind::Interrupted } else if err.is_instance_of::(py) { io::ErrorKind::NotFound } else if err.is_instance_of::(py) { io::ErrorKind::PermissionDenied } else if err.is_instance_of::(py) { io::ErrorKind::AlreadyExists } else if err.is_instance_of::(py) { io::ErrorKind::WouldBlock } else if err.is_instance_of::(py) { io::ErrorKind::TimedOut } else { io::ErrorKind::Other } }); io::Error::new(kind, err) } } /// Create `PyErr` from `io::Error` /// (`OSError` except if the `io::Error` is wrapping a Python exception, /// in this case the exception is returned) impl From for PyErr { fn from(err: io::Error) -> PyErr { // If the error wraps a Python error we return it if err.get_ref().map_or(false, |e| e.is::()) { return *err.into_inner().unwrap().downcast().unwrap(); } match err.kind() { io::ErrorKind::BrokenPipe => exceptions::PyBrokenPipeError::new_err(err), io::ErrorKind::ConnectionRefused => exceptions::PyConnectionRefusedError::new_err(err), io::ErrorKind::ConnectionAborted => exceptions::PyConnectionAbortedError::new_err(err), io::ErrorKind::ConnectionReset => exceptions::PyConnectionResetError::new_err(err), io::ErrorKind::Interrupted => exceptions::PyInterruptedError::new_err(err), io::ErrorKind::NotFound => exceptions::PyFileNotFoundError::new_err(err), io::ErrorKind::PermissionDenied => exceptions::PyPermissionError::new_err(err), io::ErrorKind::AlreadyExists => exceptions::PyFileExistsError::new_err(err), io::ErrorKind::WouldBlock => exceptions::PyBlockingIOError::new_err(err), io::ErrorKind::TimedOut => exceptions::PyTimeoutError::new_err(err), _ => exceptions::PyOSError::new_err(err), } } } impl PyErrArguments for io::Error { fn arguments(self, py: Python<'_>) -> PyObject { self.to_string().into_py(py) } } impl From> for PyErr { fn from(err: io::IntoInnerError) -> PyErr { err.into_error().into() } } impl PyErrArguments for io::IntoInnerError { fn arguments(self, py: Python<'_>) -> PyObject { self.into_error().arguments(py) } } impl From for PyErr { fn from(_: std::convert::Infallible) -> PyErr { unreachable!() } } macro_rules! impl_to_pyerr { ($err: ty, $pyexc: ty) => { impl PyErrArguments for $err { fn arguments(self, py: Python<'_>) -> PyObject { self.to_string().into_py(py) } } impl std::convert::From<$err> for PyErr { fn from(err: $err) -> PyErr { <$pyexc>::new_err(err) } } }; } impl_to_pyerr!(std::array::TryFromSliceError, exceptions::PyValueError); impl_to_pyerr!(std::num::ParseIntError, exceptions::PyValueError); impl_to_pyerr!(std::num::ParseFloatError, exceptions::PyValueError); impl_to_pyerr!(std::num::TryFromIntError, exceptions::PyValueError); impl_to_pyerr!(std::str::ParseBoolError, exceptions::PyValueError); impl_to_pyerr!(std::ffi::IntoStringError, exceptions::PyUnicodeDecodeError); impl_to_pyerr!(std::ffi::NulError, exceptions::PyValueError); impl_to_pyerr!(std::str::Utf8Error, exceptions::PyUnicodeDecodeError); impl_to_pyerr!(std::string::FromUtf8Error, exceptions::PyUnicodeDecodeError); impl_to_pyerr!( std::string::FromUtf16Error, exceptions::PyUnicodeDecodeError ); impl_to_pyerr!( std::char::DecodeUtf16Error, exceptions::PyUnicodeDecodeError ); impl_to_pyerr!(std::net::AddrParseError, exceptions::PyValueError); #[cfg(test)] mod tests { use crate::{PyErr, Python}; use std::io; #[test] fn io_errors() { let check_err = |kind, expected_ty| { Python::with_gil(|py| { let rust_err = io::Error::new(kind, "some error msg"); let py_err: PyErr = rust_err.into(); let py_err_msg = format!("{}: some error msg", expected_ty); assert_eq!(py_err.to_string(), py_err_msg); let py_error_clone = py_err.clone_ref(py); let rust_err_from_py_err: io::Error = py_err.into(); assert_eq!(rust_err_from_py_err.to_string(), py_err_msg); assert_eq!(rust_err_from_py_err.kind(), kind); let py_err_recovered_from_rust_err: PyErr = rust_err_from_py_err.into(); assert!(py_err_recovered_from_rust_err .value(py) .is(py_error_clone.value(py))); // It should be the same exception }) }; check_err(io::ErrorKind::BrokenPipe, "BrokenPipeError"); check_err(io::ErrorKind::ConnectionRefused, "ConnectionRefusedError"); check_err(io::ErrorKind::ConnectionAborted, "ConnectionAbortedError"); check_err(io::ErrorKind::ConnectionReset, "ConnectionResetError"); check_err(io::ErrorKind::Interrupted, "InterruptedError"); check_err(io::ErrorKind::NotFound, "FileNotFoundError"); check_err(io::ErrorKind::PermissionDenied, "PermissionError"); check_err(io::ErrorKind::AlreadyExists, "FileExistsError"); check_err(io::ErrorKind::WouldBlock, "BlockingIOError"); check_err(io::ErrorKind::TimedOut, "TimeoutError"); } } pyo3-0.20.2/src/err/mod.rs000064400000000000000000001100401046102023000132770ustar 00000000000000use crate::panic::PanicException; use crate::type_object::PyTypeInfo; use crate::types::{PyTraceback, PyType}; use crate::{ exceptions::{self, PyBaseException}, ffi, }; use crate::{IntoPy, Py, PyAny, PyObject, Python, ToPyObject}; use std::borrow::Cow; use std::cell::UnsafeCell; use std::ffi::CString; mod err_state; mod impls; pub use err_state::PyErrArguments; use err_state::{PyErrState, PyErrStateLazyFnOutput, PyErrStateNormalized}; /// Represents a Python exception. /// /// To avoid needing access to [`Python`] in `Into` conversions to create `PyErr` (thus improving /// compatibility with `?` and other Rust errors) this type supports creating exceptions instances /// in a lazy fashion, where the full Python object for the exception is created only when needed. /// /// Accessing the contained exception in any way, such as with [`value`](PyErr::value), /// [`get_type`](PyErr::get_type), or [`is_instance`](PyErr::is_instance) will create the full /// exception object if it was not already created. pub struct PyErr { // Safety: can only hand out references when in the "normalized" state. Will never change // after normalization. // // The state is temporarily removed from the PyErr during normalization, to avoid // concurrent modifications. state: UnsafeCell>, } // The inner value is only accessed through ways that require proving the gil is held #[cfg(feature = "nightly")] unsafe impl crate::marker::Ungil for PyErr {} unsafe impl Send for PyErr {} unsafe impl Sync for PyErr {} /// Represents the result of a Python call. pub type PyResult = Result; /// Error that indicates a failure to convert a PyAny to a more specific Python type. #[derive(Debug)] pub struct PyDowncastError<'a> { from: &'a PyAny, to: Cow<'static, str>, } impl<'a> PyDowncastError<'a> { /// Create a new `PyDowncastError` representing a failure to convert the object /// `from` into the type named in `to`. pub fn new(from: &'a PyAny, to: impl Into>) -> Self { PyDowncastError { from, to: to.into(), } } } impl PyErr { /// Creates a new PyErr of type `T`. /// /// `args` can be: /// * a tuple: the exception instance will be created using the equivalent to the Python /// expression `T(*tuple)` /// * any other value: the exception instance will be created using the equivalent to the Python /// expression `T(value)` /// /// This exception instance will be initialized lazily. This avoids the need for the Python GIL /// to be held, but requires `args` to be `Send` and `Sync`. If `args` is not `Send` or `Sync`, /// consider using [`PyErr::from_value`] instead. /// /// If `T` does not inherit from `BaseException`, then a `TypeError` will be returned. /// /// If calling T's constructor with `args` raises an exception, that exception will be returned. /// /// # Examples /// /// ``` /// use pyo3::prelude::*; /// use pyo3::exceptions::PyTypeError; /// /// #[pyfunction] /// fn always_throws() -> PyResult<()> { /// Err(PyErr::new::("Error message")) /// } /// # /// # Python::with_gil(|py| { /// # let fun = pyo3::wrap_pyfunction!(always_throws, py).unwrap(); /// # let err = fun.call0().expect_err("called a function that should always return an error but the return value was Ok"); /// # assert!(err.is_instance_of::(py)) /// # }); /// ``` /// /// In most cases, you can use a concrete exception's constructor instead: /// /// ``` /// use pyo3::prelude::*; /// use pyo3::exceptions::PyTypeError; /// /// #[pyfunction] /// fn always_throws() -> PyResult<()> { /// Err(PyTypeError::new_err("Error message")) /// } /// # /// # Python::with_gil(|py| { /// # let fun = pyo3::wrap_pyfunction!(always_throws, py).unwrap(); /// # let err = fun.call0().expect_err("called a function that should always return an error but the return value was Ok"); /// # assert!(err.is_instance_of::(py)) /// # }); /// ``` #[inline] pub fn new(args: A) -> PyErr where T: PyTypeInfo, A: PyErrArguments + Send + Sync + 'static, { PyErr::from_state(PyErrState::Lazy(Box::new(move |py| { PyErrStateLazyFnOutput { ptype: T::type_object(py).into(), pvalue: args.arguments(py), } }))) } /// Constructs a new PyErr from the given Python type and arguments. /// /// `ty` is the exception type; usually one of the standard exceptions /// like `exceptions::PyRuntimeError`. /// /// `args` is either a tuple or a single value, with the same meaning as in [`PyErr::new`]. /// /// If `ty` does not inherit from `BaseException`, then a `TypeError` will be returned. /// /// If calling `ty` with `args` raises an exception, that exception will be returned. pub fn from_type(ty: &PyType, args: A) -> PyErr where A: PyErrArguments + Send + Sync + 'static, { PyErr::from_state(PyErrState::lazy(ty, args)) } /// Creates a new PyErr. /// /// If `obj` is a Python exception object, the PyErr will contain that object. /// /// If `obj` is a Python exception type object, this is equivalent to `PyErr::from_type(obj, ())`. /// /// Otherwise, a `TypeError` is created. /// /// # Examples /// ```rust /// use pyo3::prelude::*; /// use pyo3::exceptions::PyTypeError; /// use pyo3::types::{PyType, PyString}; /// /// Python::with_gil(|py| { /// // Case #1: Exception object /// let err = PyErr::from_value(PyTypeError::new_err("some type error").value(py)); /// assert_eq!(err.to_string(), "TypeError: some type error"); /// /// // Case #2: Exception type /// let err = PyErr::from_value(PyType::new::(py)); /// assert_eq!(err.to_string(), "TypeError: "); /// /// // Case #3: Invalid exception value /// let err = PyErr::from_value(PyString::new(py, "foo").into()); /// assert_eq!( /// err.to_string(), /// "TypeError: exceptions must derive from BaseException" /// ); /// }); /// ``` pub fn from_value(obj: &PyAny) -> PyErr { let state = if let Ok(obj) = obj.downcast::() { PyErrState::normalized(obj) } else { // Assume obj is Type[Exception]; let later normalization handle if this // is not the case PyErrState::lazy(obj, obj.py().None()) }; PyErr::from_state(state) } /// Returns the type of this exception. /// /// # Examples /// ```rust /// use pyo3::{exceptions::PyTypeError, types::PyType, PyErr, Python}; /// /// Python::with_gil(|py| { /// let err: PyErr = PyTypeError::new_err(("some type error",)); /// assert!(err.get_type(py).is(PyType::new::(py))); /// }); /// ``` pub fn get_type<'py>(&'py self, py: Python<'py>) -> &'py PyType { self.normalized(py).ptype(py) } /// Returns the value of this exception. /// /// # Examples /// /// ```rust /// use pyo3::{exceptions::PyTypeError, PyErr, Python}; /// /// Python::with_gil(|py| { /// let err: PyErr = PyTypeError::new_err(("some type error",)); /// assert!(err.is_instance_of::(py)); /// assert_eq!(err.value(py).to_string(), "some type error"); /// }); /// ``` pub fn value<'py>(&'py self, py: Python<'py>) -> &'py PyBaseException { self.normalized(py).pvalue.as_ref(py) } /// Consumes self to take ownership of the exception value contained in this error. pub fn into_value(self, py: Python<'_>) -> Py { // NB technically this causes one reference count increase and decrease in quick succession // on pvalue, but it's probably not worth optimizing this right now for the additional code // complexity. let normalized = self.normalized(py); let exc = normalized.pvalue.clone_ref(py); if let Some(tb) = normalized.ptraceback(py) { unsafe { ffi::PyException_SetTraceback(exc.as_ptr(), tb.as_ptr()); } } exc } /// Returns the traceback of this exception object. /// /// # Examples /// ```rust /// use pyo3::{exceptions::PyTypeError, Python}; /// /// Python::with_gil(|py| { /// let err = PyTypeError::new_err(("some type error",)); /// assert!(err.traceback(py).is_none()); /// }); /// ``` pub fn traceback<'py>(&'py self, py: Python<'py>) -> Option<&'py PyTraceback> { self.normalized(py).ptraceback(py) } /// Gets whether an error is present in the Python interpreter's global state. #[inline] pub fn occurred(_: Python<'_>) -> bool { unsafe { !ffi::PyErr_Occurred().is_null() } } /// Takes the current error from the Python interpreter's global state and clears the global /// state. If no error is set, returns `None`. /// /// If the error is a `PanicException` (which would have originated from a panic in a pyo3 /// callback) then this function will resume the panic. /// /// Use this function when it is not known if an error should be present. If the error is /// expected to have been set, for example from [`PyErr::occurred`] or by an error return value /// from a C FFI function, use [`PyErr::fetch`]. pub fn take(py: Python<'_>) -> Option { Self::_take(py) } #[cfg(not(Py_3_12))] fn _take(py: Python<'_>) -> Option { let (ptype, pvalue, ptraceback) = unsafe { let mut ptype: *mut ffi::PyObject = std::ptr::null_mut(); let mut pvalue: *mut ffi::PyObject = std::ptr::null_mut(); let mut ptraceback: *mut ffi::PyObject = std::ptr::null_mut(); ffi::PyErr_Fetch(&mut ptype, &mut pvalue, &mut ptraceback); // Convert to Py immediately so that any references are freed by early return. let ptype = PyObject::from_owned_ptr_or_opt(py, ptype); let pvalue = PyObject::from_owned_ptr_or_opt(py, pvalue); let ptraceback = PyObject::from_owned_ptr_or_opt(py, ptraceback); // A valid exception state should always have a non-null ptype, but the other two may be // null. let ptype = match ptype { Some(ptype) => ptype, None => { debug_assert!( pvalue.is_none(), "Exception type was null but value was not null" ); debug_assert!( ptraceback.is_none(), "Exception type was null but traceback was not null" ); return None; } }; (ptype, pvalue, ptraceback) }; if ptype.as_ptr() == PanicException::type_object_raw(py).cast() { let msg = pvalue .as_ref() .and_then(|obj| obj.as_ref(py).str().ok()) .map(|py_str| py_str.to_string_lossy().into()) .unwrap_or_else(|| String::from("Unwrapped panic from Python code")); let state = PyErrState::FfiTuple { ptype, pvalue, ptraceback, }; Self::print_panic_and_unwind(py, state, msg) } Some(PyErr::from_state(PyErrState::FfiTuple { ptype, pvalue, ptraceback, })) } #[cfg(Py_3_12)] fn _take(py: Python<'_>) -> Option { let state = PyErrStateNormalized::take(py)?; let pvalue = state.pvalue.as_ref(py); if pvalue.get_type().as_ptr() == PanicException::type_object_raw(py).cast() { let msg: String = pvalue .str() .map(|py_str| py_str.to_string_lossy().into()) .unwrap_or_else(|_| String::from("Unwrapped panic from Python code")); Self::print_panic_and_unwind(py, PyErrState::Normalized(state), msg) } Some(PyErr::from_state(PyErrState::Normalized(state))) } fn print_panic_and_unwind(py: Python<'_>, state: PyErrState, msg: String) -> ! { eprintln!("--- PyO3 is resuming a panic after fetching a PanicException from Python. ---"); eprintln!("Python stack trace below:"); state.restore(py); unsafe { ffi::PyErr_PrintEx(0); } std::panic::resume_unwind(Box::new(msg)) } /// Equivalent to [PyErr::take], but when no error is set: /// - Panics in debug mode. /// - Returns a `SystemError` in release mode. /// /// This behavior is consistent with Python's internal handling of what happens when a C return /// value indicates an error occurred but the global error state is empty. (A lack of exception /// should be treated as a bug in the code which returned an error code but did not set an /// exception.) /// /// Use this function when the error is expected to have been set, for example from /// [PyErr::occurred] or by an error return value from a C FFI function. #[cfg_attr(all(debug_assertions, track_caller), track_caller)] #[inline] pub fn fetch(py: Python<'_>) -> PyErr { const FAILED_TO_FETCH: &str = "attempted to fetch exception but none was set"; match PyErr::take(py) { Some(err) => err, #[cfg(debug_assertions)] None => panic!("{}", FAILED_TO_FETCH), #[cfg(not(debug_assertions))] None => exceptions::PySystemError::new_err(FAILED_TO_FETCH), } } /// Creates a new exception type with the given name and docstring. /// /// - `base` can be an existing exception type to subclass, or a tuple of classes. /// - `dict` specifies an optional dictionary of class variables and methods. /// - `doc` will be the docstring seen by python users. /// /// /// # Errors /// /// This function returns an error if `name` is not of the form `.`. /// /// # Panics /// /// This function will panic if `name` or `doc` cannot be converted to [`CString`]s. pub fn new_type( py: Python<'_>, name: &str, doc: Option<&str>, base: Option<&PyType>, dict: Option, ) -> PyResult> { let base: *mut ffi::PyObject = match base { None => std::ptr::null_mut(), Some(obj) => obj.as_ptr(), }; let dict: *mut ffi::PyObject = match dict { None => std::ptr::null_mut(), Some(obj) => obj.as_ptr(), }; let null_terminated_name = CString::new(name).expect("Failed to initialize nul terminated exception name"); let null_terminated_doc = doc.map(|d| CString::new(d).expect("Failed to initialize nul terminated docstring")); let null_terminated_doc_ptr = match null_terminated_doc.as_ref() { Some(c) => c.as_ptr(), None => std::ptr::null(), }; let ptr = unsafe { ffi::PyErr_NewExceptionWithDoc( null_terminated_name.as_ptr(), null_terminated_doc_ptr, base, dict, ) }; unsafe { Py::from_owned_ptr_or_err(py, ptr) } } /// Prints a standard traceback to `sys.stderr`. pub fn display(&self, py: Python<'_>) { #[cfg(Py_3_12)] unsafe { ffi::PyErr_DisplayException(self.value(py).as_ptr()) } #[cfg(not(Py_3_12))] unsafe { ffi::PyErr_Display( self.get_type(py).as_ptr(), self.value(py).as_ptr(), self.traceback(py) .map_or(std::ptr::null_mut(), |traceback| traceback.as_ptr()), ) } } /// Calls `sys.excepthook` and then prints a standard traceback to `sys.stderr`. pub fn print(&self, py: Python<'_>) { self.clone_ref(py).restore(py); unsafe { ffi::PyErr_PrintEx(0) } } /// Calls `sys.excepthook` and then prints a standard traceback to `sys.stderr`. /// /// Additionally sets `sys.last_{type,value,traceback,exc}` attributes to this exception. pub fn print_and_set_sys_last_vars(&self, py: Python<'_>) { self.clone_ref(py).restore(py); unsafe { ffi::PyErr_PrintEx(1) } } /// Returns true if the current exception matches the exception in `exc`. /// /// If `exc` is a class object, this also returns `true` when `self` is an instance of a subclass. /// If `exc` is a tuple, all exceptions in the tuple (and recursively in subtuples) are searched for a match. pub fn matches(&self, py: Python<'_>, exc: T) -> bool where T: ToPyObject, { self.is_instance(py, exc.to_object(py).as_ref(py)) } /// Returns true if the current exception is instance of `T`. #[inline] pub fn is_instance(&self, py: Python<'_>, ty: &PyAny) -> bool { (unsafe { ffi::PyErr_GivenExceptionMatches(self.get_type(py).as_ptr(), ty.as_ptr()) }) != 0 } /// Returns true if the current exception is instance of `T`. #[inline] pub fn is_instance_of(&self, py: Python<'_>) -> bool where T: PyTypeInfo, { self.is_instance(py, T::type_object(py)) } /// Writes the error back to the Python interpreter's global state. /// This is the opposite of `PyErr::fetch()`. #[inline] pub fn restore(self, py: Python<'_>) { self.state .into_inner() .expect("PyErr state should never be invalid outside of normalization") .restore(py) } /// Reports the error as unraisable. /// /// This calls `sys.unraisablehook()` using the current exception and obj argument. /// /// This method is useful to report errors in situations where there is no good mechanism /// to report back to the Python land. In Python this is used to indicate errors in /// background threads or destructors which are protected. In Rust code this is commonly /// useful when you are calling into a Python callback which might fail, but there is no /// obvious way to handle this error other than logging it. /// /// Calling this method has the benefit that the error goes back into a standardized callback /// in Python which for instance allows unittests to ensure that no unraisable error /// actually happend by hooking `sys.unraisablehook`. /// /// Example: /// ```rust /// # use pyo3::prelude::*; /// # use pyo3::exceptions::PyRuntimeError; /// # fn failing_function() -> PyResult<()> { Err(PyRuntimeError::new_err("foo")) } /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// match failing_function() { /// Err(pyerr) => pyerr.write_unraisable(py, None), /// Ok(..) => { /* do something here */ } /// } /// Ok(()) /// }) /// # } #[inline] pub fn write_unraisable(self, py: Python<'_>, obj: Option<&PyAny>) { self.restore(py); unsafe { ffi::PyErr_WriteUnraisable(obj.map_or(std::ptr::null_mut(), |x| x.as_ptr())) } } /// Issues a warning message. /// /// May return an `Err(PyErr)` if warnings-as-errors is enabled. /// /// Equivalent to `warnings.warn()` in Python. /// /// The `category` should be one of the `Warning` classes available in /// [`pyo3::exceptions`](crate::exceptions), or a subclass. The Python /// object can be retrieved using [`Python::get_type()`]. /// /// Example: /// ```rust /// # use pyo3::prelude::*; /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// let user_warning = py.get_type::(); /// PyErr::warn(py, user_warning, "I am warning you", 0)?; /// Ok(()) /// }) /// # } /// ``` pub fn warn(py: Python<'_>, category: &PyAny, message: &str, stacklevel: i32) -> PyResult<()> { let message = CString::new(message)?; error_on_minusone(py, unsafe { ffi::PyErr_WarnEx( category.as_ptr(), message.as_ptr(), stacklevel as ffi::Py_ssize_t, ) }) } /// Issues a warning message, with more control over the warning attributes. /// /// May return a `PyErr` if warnings-as-errors is enabled. /// /// Equivalent to `warnings.warn_explicit()` in Python. /// /// The `category` should be one of the `Warning` classes available in /// [`pyo3::exceptions`](crate::exceptions), or a subclass. pub fn warn_explicit( py: Python<'_>, category: &PyAny, message: &str, filename: &str, lineno: i32, module: Option<&str>, registry: Option<&PyAny>, ) -> PyResult<()> { let message = CString::new(message)?; let filename = CString::new(filename)?; let module = module.map(CString::new).transpose()?; let module_ptr = match module { None => std::ptr::null_mut(), Some(s) => s.as_ptr(), }; let registry: *mut ffi::PyObject = match registry { None => std::ptr::null_mut(), Some(obj) => obj.as_ptr(), }; error_on_minusone(py, unsafe { ffi::PyErr_WarnExplicit( category.as_ptr(), message.as_ptr(), filename.as_ptr(), lineno, module_ptr, registry, ) }) } /// Clone the PyErr. This requires the GIL, which is why PyErr does not implement Clone. /// /// # Examples /// ```rust /// use pyo3::{exceptions::PyTypeError, PyErr, Python}; /// Python::with_gil(|py| { /// let err: PyErr = PyTypeError::new_err(("some type error",)); /// let err_clone = err.clone_ref(py); /// assert!(err.get_type(py).is(err_clone.get_type(py))); /// assert!(err.value(py).is(err_clone.value(py))); /// match err.traceback(py) { /// None => assert!(err_clone.traceback(py).is_none()), /// Some(tb) => assert!(err_clone.traceback(py).unwrap().is(tb)), /// } /// }); /// ``` #[inline] pub fn clone_ref(&self, py: Python<'_>) -> PyErr { PyErr::from_state(PyErrState::Normalized(self.normalized(py).clone())) } /// Return the cause (either an exception instance, or None, set by `raise ... from ...`) /// associated with the exception, as accessible from Python through `__cause__`. pub fn cause(&self, py: Python<'_>) -> Option { let value = self.value(py); let obj = unsafe { py.from_owned_ptr_or_opt::(ffi::PyException_GetCause(value.as_ptr())) }; obj.map(Self::from_value) } /// Set the cause associated with the exception, pass `None` to clear it. pub fn set_cause(&self, py: Python<'_>, cause: Option) { let value = self.value(py); let cause = cause.map(|err| err.into_value(py)); unsafe { // PyException_SetCause _steals_ a reference to cause, so must use .into_ptr() ffi::PyException_SetCause( value.as_ptr(), cause.map_or(std::ptr::null_mut(), Py::into_ptr), ); } } #[inline] fn from_state(state: PyErrState) -> PyErr { PyErr { state: UnsafeCell::new(Some(state)), } } #[inline] fn normalized(&self, py: Python<'_>) -> &PyErrStateNormalized { if let Some(PyErrState::Normalized(n)) = unsafe { // Safety: self.state will never be written again once normalized. &*self.state.get() } { return n; } self.make_normalized(py) } #[cold] fn make_normalized(&self, py: Python<'_>) -> &PyErrStateNormalized { // This process is safe because: // - Access is guaranteed not to be concurrent thanks to `Python` GIL token // - Write happens only once, and then never will change again. // - State is set to None during the normalization process, so that a second // concurrent normalization attempt will panic before changing anything. let state = unsafe { (*self.state.get()) .take() .expect("Cannot normalize a PyErr while already normalizing it.") }; unsafe { let self_state = &mut *self.state.get(); *self_state = Some(PyErrState::Normalized(state.normalize(py))); match self_state { Some(PyErrState::Normalized(n)) => n, _ => unreachable!(), } } } } impl std::fmt::Debug for PyErr { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> Result<(), std::fmt::Error> { Python::with_gil(|py| { f.debug_struct("PyErr") .field("type", self.get_type(py)) .field("value", self.value(py)) .field("traceback", &self.traceback(py)) .finish() }) } } impl std::fmt::Display for PyErr { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { Python::with_gil(|py| { let value = self.value(py); let type_name = value.get_type().name().map_err(|_| std::fmt::Error)?; write!(f, "{}", type_name)?; if let Ok(s) = value.str() { write!(f, ": {}", &s.to_string_lossy()) } else { write!(f, ": ") } }) } } impl std::error::Error for PyErr {} impl IntoPy for PyErr { fn into_py(self, py: Python<'_>) -> PyObject { self.into_value(py).into() } } impl ToPyObject for PyErr { fn to_object(&self, py: Python<'_>) -> PyObject { self.clone_ref(py).into_py(py) } } impl<'a> IntoPy for &'a PyErr { fn into_py(self, py: Python<'_>) -> PyObject { self.clone_ref(py).into_py(py) } } struct PyDowncastErrorArguments { from: Py, to: Cow<'static, str>, } impl PyErrArguments for PyDowncastErrorArguments { fn arguments(self, py: Python<'_>) -> PyObject { format!( "'{}' object cannot be converted to '{}'", self.from .as_ref(py) .name() .unwrap_or(""), self.to ) .to_object(py) } } /// Convert `PyDowncastError` to Python `TypeError`. impl<'a> std::convert::From> for PyErr { fn from(err: PyDowncastError<'_>) -> PyErr { let args = PyDowncastErrorArguments { from: err.from.get_type().into(), to: err.to, }; exceptions::PyTypeError::new_err(args) } } impl<'a> std::error::Error for PyDowncastError<'a> {} impl<'a> std::fmt::Display for PyDowncastError<'a> { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> Result<(), std::fmt::Error> { write!( f, "'{}' object cannot be converted to '{}'", self.from.get_type().name().map_err(|_| std::fmt::Error)?, self.to ) } } pub fn panic_after_error(_py: Python<'_>) -> ! { unsafe { ffi::PyErr_Print(); } panic!("Python API call failed"); } /// Returns Ok if the error code is not -1. #[inline] pub(crate) fn error_on_minusone(py: Python<'_>, result: T) -> PyResult<()> { if result != T::MINUS_ONE { Ok(()) } else { Err(PyErr::fetch(py)) } } pub(crate) trait SignedInteger: Eq { const MINUS_ONE: Self; } macro_rules! impl_signed_integer { ($t:ty) => { impl SignedInteger for $t { const MINUS_ONE: Self = -1; } }; } impl_signed_integer!(i8); impl_signed_integer!(i16); impl_signed_integer!(i32); impl_signed_integer!(i64); impl_signed_integer!(i128); impl_signed_integer!(isize); #[cfg(test)] mod tests { use super::PyErrState; use crate::exceptions::{self, PyTypeError, PyValueError}; use crate::tests::common::CatchWarnings; use crate::{PyErr, PyTypeInfo, Python}; #[test] fn no_error() { assert!(Python::with_gil(PyErr::take).is_none()); } #[test] fn set_valueerror() { Python::with_gil(|py| { let err: PyErr = exceptions::PyValueError::new_err("some exception message"); assert!(err.is_instance_of::(py)); err.restore(py); assert!(PyErr::occurred(py)); let err = PyErr::fetch(py); assert!(err.is_instance_of::(py)); assert_eq!(err.to_string(), "ValueError: some exception message"); }) } #[test] fn invalid_error_type() { Python::with_gil(|py| { let err: PyErr = PyErr::new::(()); assert!(err.is_instance_of::(py)); err.restore(py); let err = PyErr::fetch(py); assert!(err.is_instance_of::(py)); assert_eq!( err.to_string(), "TypeError: exceptions must derive from BaseException" ); }) } #[test] fn set_typeerror() { Python::with_gil(|py| { let err: PyErr = exceptions::PyTypeError::new_err(()); err.restore(py); assert!(PyErr::occurred(py)); drop(PyErr::fetch(py)); }); } #[test] #[should_panic(expected = "new panic")] fn fetching_panic_exception_resumes_unwind() { use crate::panic::PanicException; Python::with_gil(|py| { let err: PyErr = PanicException::new_err("new panic"); err.restore(py); assert!(PyErr::occurred(py)); // should resume unwind let _ = PyErr::fetch(py); }); } #[test] #[should_panic(expected = "new panic")] #[cfg(not(Py_3_12))] fn fetching_normalized_panic_exception_resumes_unwind() { use crate::panic::PanicException; Python::with_gil(|py| { let err: PyErr = PanicException::new_err("new panic"); // Restoring an error doesn't normalize it before Python 3.12, // so we have to explicitly test this case. let _ = err.normalized(py); err.restore(py); assert!(PyErr::occurred(py)); // should resume unwind let _ = PyErr::fetch(py); }); } #[test] fn err_debug() { // Debug representation should be like the following (without the newlines): // PyErr { // type: , // value: Exception('banana'), // traceback: Some("); assert_eq!(fields.next().unwrap(), "value: Exception('banana')"); let traceback = fields.next().unwrap(); assert!(traceback.starts_with("traceback: Some()")); assert!(fields.next().is_none()); }); } #[test] fn err_display() { Python::with_gil(|py| { let err = py .run("raise Exception('banana')", None, None) .expect_err("raising should have given us an error"); assert_eq!(err.to_string(), "Exception: banana"); }); } #[test] fn test_pyerr_send_sync() { fn is_send() {} fn is_sync() {} is_send::(); is_sync::(); is_send::(); is_sync::(); } #[test] fn test_pyerr_matches() { Python::with_gil(|py| { let err = PyErr::new::("foo"); assert!(err.matches(py, PyValueError::type_object(py))); assert!(err.matches( py, (PyValueError::type_object(py), PyTypeError::type_object(py)) )); assert!(!err.matches(py, PyTypeError::type_object(py))); // String is not a valid exception class, so we should get a TypeError let err: PyErr = PyErr::from_type(crate::types::PyString::type_object(py), "foo"); assert!(err.matches(py, PyTypeError::type_object(py))); }) } #[test] fn test_pyerr_cause() { Python::with_gil(|py| { let err = py .run("raise Exception('banana')", None, None) .expect_err("raising should have given us an error"); assert!(err.cause(py).is_none()); let err = py .run( "raise Exception('banana') from Exception('apple')", None, None, ) .expect_err("raising should have given us an error"); let cause = err .cause(py) .expect("raising from should have given us a cause"); assert_eq!(cause.to_string(), "Exception: apple"); err.set_cause(py, None); assert!(err.cause(py).is_none()); let new_cause = exceptions::PyValueError::new_err("orange"); err.set_cause(py, Some(new_cause)); let cause = err .cause(py) .expect("set_cause should have given us a cause"); assert_eq!(cause.to_string(), "ValueError: orange"); }); } #[test] fn warnings() { // Note: although the warning filter is interpreter global, keeping the // GIL locked should prevent effects to be visible to other testing // threads. Python::with_gil(|py| { let cls = py.get_type::(); // Reset warning filter to default state let warnings = py.import("warnings").unwrap(); warnings.call_method0("resetwarnings").unwrap(); // First, test the warning is emitted assert_warnings!( py, { PyErr::warn(py, cls, "I am warning you", 0).unwrap() }, [(exceptions::PyUserWarning, "I am warning you")] ); // Test with raising warnings .call_method1("simplefilter", ("error", cls)) .unwrap(); PyErr::warn(py, cls, "I am warning you", 0).unwrap_err(); // Test with error for an explicit module warnings.call_method0("resetwarnings").unwrap(); warnings .call_method1("filterwarnings", ("error", "", cls, "pyo3test")) .unwrap(); // This has the wrong module and will not raise, just be emitted assert_warnings!( py, { PyErr::warn(py, cls, "I am warning you", 0).unwrap() }, [(exceptions::PyUserWarning, "I am warning you")] ); let err = PyErr::warn_explicit(py, cls, "I am warning you", "pyo3test.py", 427, None, None) .unwrap_err(); assert!(err .value(py) .getattr("args") .unwrap() .get_item(0) .unwrap() .eq("I am warning you") .unwrap()); // Finally, reset filter again warnings.call_method0("resetwarnings").unwrap(); }); } } pyo3-0.20.2/src/exceptions.rs000064400000000000000000001040231046102023000141150ustar 00000000000000//! Exception and warning types defined by Python. //! //! The structs in this module represent Python's built-in exceptions and //! warnings, while the modules comprise structs representing errors defined in //! Python code. //! //! The latter are created with the //! [`import_exception`](crate::import_exception) macro, which you can use //! yourself to import Python classes that are ultimately derived from //! `BaseException`. use crate::{ffi, PyResult, Python}; use std::ffi::CStr; use std::ops; use std::os::raw::c_char; /// The boilerplate to convert between a Rust type and a Python exception. #[doc(hidden)] #[macro_export] macro_rules! impl_exception_boilerplate { ($name: ident) => { impl ::std::convert::From<&$name> for $crate::PyErr { #[inline] fn from(err: &$name) -> $crate::PyErr { $crate::PyErr::from_value(err) } } impl $name { /// Creates a new [`PyErr`] of this type. /// /// [`PyErr`]: https://docs.rs/pyo3/latest/pyo3/struct.PyErr.html "PyErr in pyo3" #[inline] pub fn new_err(args: A) -> $crate::PyErr where A: $crate::PyErrArguments + ::std::marker::Send + ::std::marker::Sync + 'static, { $crate::PyErr::new::<$name, A>(args) } } impl ::std::error::Error for $name { fn source(&self) -> ::std::option::Option<&(dyn ::std::error::Error + 'static)> { unsafe { let cause: &$crate::exceptions::PyBaseException = self .py() .from_owned_ptr_or_opt($crate::ffi::PyException_GetCause(self.as_ptr()))?; ::std::option::Option::Some(cause) } } } }; } /// Defines a Rust type for an exception defined in Python code. /// /// # Syntax /// /// ```import_exception!(module, MyError)``` /// /// * `module` is the name of the containing module. /// * `MyError` is the name of the new exception type. /// /// # Examples /// ``` /// use pyo3::import_exception; /// use pyo3::types::IntoPyDict; /// use pyo3::Python; /// /// import_exception!(socket, gaierror); /// /// Python::with_gil(|py| { /// let ctx = [("gaierror", py.get_type::())].into_py_dict(py); /// pyo3::py_run!(py, *ctx, "import socket; assert gaierror is socket.gaierror"); /// }); /// /// ``` #[macro_export] macro_rules! import_exception { ($module: expr, $name: ident) => { /// A Rust type representing an exception defined in Python code. /// /// This type was created by the [`pyo3::import_exception!`] macro - see its documentation /// for more information. /// /// [`pyo3::import_exception!`]: https://docs.rs/pyo3/latest/pyo3/macro.import_exception.html "import_exception in pyo3" #[repr(transparent)] #[allow(non_camel_case_types)] // E.g. `socket.herror` pub struct $name($crate::PyAny); $crate::impl_exception_boilerplate!($name); $crate::pyobject_native_type_core!( $name, $name::type_object_raw, #module=::std::option::Option::Some(stringify!($module)) ); impl $name { fn type_object_raw(py: $crate::Python<'_>) -> *mut $crate::ffi::PyTypeObject { use $crate::sync::GILOnceCell; static TYPE_OBJECT: GILOnceCell<$crate::Py<$crate::types::PyType>> = GILOnceCell::new(); TYPE_OBJECT .get_or_init(py, || { let imp = py .import(stringify!($module)) .unwrap_or_else(|err| { let traceback = err .traceback(py) .map(|tb| tb.format().expect("raised exception will have a traceback")) .unwrap_or_default(); ::std::panic!("Can not import module {}: {}\n{}", stringify!($module), err, traceback); }); let cls = imp.getattr(stringify!($name)).expect(concat!( "Can not load exception class: {}.{}", stringify!($module), ".", stringify!($name) )); cls.extract() .expect("Imported exception should be a type object") }) .as_ptr() as *mut _ } } }; } /// Defines a new exception type. /// /// # Syntax /// /// * `module` is the name of the containing module. /// * `name` is the name of the new exception type. /// * `base` is the base class of `MyError`, usually [`PyException`]. /// * `doc` (optional) is the docstring visible to users (with `.__doc__` and `help()`) and /// accompanies your error type in your crate's documentation. /// /// # Examples /// /// ``` /// use pyo3::prelude::*; /// use pyo3::create_exception; /// use pyo3::exceptions::PyException; /// /// create_exception!(my_module, MyError, PyException, "Some description."); /// /// #[pyfunction] /// fn raise_myerror() -> PyResult<()> { /// let err = MyError::new_err("Some error happened."); /// Err(err) /// } /// /// #[pymodule] /// fn my_module(py: Python<'_>, m: &PyModule) -> PyResult<()> { /// m.add("MyError", py.get_type::())?; /// m.add_function(wrap_pyfunction!(raise_myerror, py)?)?; /// Ok(()) /// } /// # fn main() -> PyResult<()> { /// # Python::with_gil(|py| -> PyResult<()> { /// # let fun = wrap_pyfunction!(raise_myerror, py)?; /// # let locals = pyo3::types::PyDict::new(py); /// # locals.set_item("MyError", py.get_type::())?; /// # locals.set_item("raise_myerror", fun)?; /// # /// # py.run( /// # "try: /// # raise_myerror() /// # except MyError as e: /// # assert e.__doc__ == 'Some description.' /// # assert str(e) == 'Some error happened.'", /// # None, /// # Some(locals), /// # )?; /// # /// # Ok(()) /// # }) /// # } /// ``` /// /// Python code can handle this exception like any other exception: /// /// ```python /// from my_module import MyError, raise_myerror /// /// try: /// raise_myerror() /// except MyError as e: /// assert e.__doc__ == 'Some description.' /// assert str(e) == 'Some error happened.' /// ``` /// #[macro_export] macro_rules! create_exception { ($module: expr, $name: ident, $base: ty) => { #[repr(transparent)] #[allow(non_camel_case_types)] // E.g. `socket.herror` pub struct $name($crate::PyAny); $crate::impl_exception_boilerplate!($name); $crate::create_exception_type_object!($module, $name, $base, ::std::option::Option::None); }; ($module: expr, $name: ident, $base: ty, $doc: expr) => { #[repr(transparent)] #[allow(non_camel_case_types)] // E.g. `socket.herror` #[doc = $doc] pub struct $name($crate::PyAny); $crate::impl_exception_boilerplate!($name); $crate::create_exception_type_object!( $module, $name, $base, ::std::option::Option::Some($doc) ); }; } /// `impl PyTypeInfo for $name` where `$name` is an /// exception newly defined in Rust code. #[doc(hidden)] #[macro_export] macro_rules! create_exception_type_object { ($module: expr, $name: ident, $base: ty, $doc: expr) => { $crate::pyobject_native_type_core!( $name, $name::type_object_raw, #module=::std::option::Option::Some(stringify!($module)) ); impl $name { fn type_object_raw(py: $crate::Python<'_>) -> *mut $crate::ffi::PyTypeObject { use $crate::sync::GILOnceCell; static TYPE_OBJECT: GILOnceCell<$crate::Py<$crate::types::PyType>> = GILOnceCell::new(); TYPE_OBJECT .get_or_init(py, || $crate::PyErr::new_type( py, concat!(stringify!($module), ".", stringify!($name)), $doc, ::std::option::Option::Some(py.get_type::<$base>()), ::std::option::Option::None, ).expect("Failed to initialize new exception type.") ).as_ptr() as *mut $crate::ffi::PyTypeObject } } }; } macro_rules! impl_native_exception ( ($name:ident, $exc_name:ident, $doc:expr, $layout:path $(, #checkfunction=$checkfunction:path)?) => ( #[doc = $doc] #[allow(clippy::upper_case_acronyms)] pub struct $name($crate::PyAny); $crate::impl_exception_boilerplate!($name); $crate::pyobject_native_type!($name, $layout, |_py| unsafe { $crate::ffi::$exc_name as *mut $crate::ffi::PyTypeObject } $(, #checkfunction=$checkfunction)?); ); ($name:ident, $exc_name:ident, $doc:expr) => ( impl_native_exception!($name, $exc_name, $doc, $crate::ffi::PyBaseExceptionObject); ) ); #[cfg(windows)] macro_rules! impl_windows_native_exception ( ($name:ident, $exc_name:ident, $doc:expr, $layout:path) => ( #[cfg(windows)] #[doc = $doc] #[allow(clippy::upper_case_acronyms)] pub struct $name($crate::PyAny); $crate::impl_exception_boilerplate!($name); $crate::pyobject_native_type!($name, $layout, |_py| unsafe { $crate::ffi::$exc_name as *mut $crate::ffi::PyTypeObject }); ); ($name:ident, $exc_name:ident, $doc:expr) => ( impl_windows_native_exception!($name, $exc_name, $doc, $crate::ffi::PyBaseExceptionObject); ) ); macro_rules! native_doc( ($name: literal, $alt: literal) => ( concat!( "Represents Python's [`", $name, "`](https://docs.python.org/3/library/exceptions.html#", $name, ") exception. ", $alt ) ); ($name: literal) => ( concat!( " Represents Python's [`", $name, "`](https://docs.python.org/3/library/exceptions.html#", $name, ") exception. # Example: Raising ", $name, " from Rust This exception can be sent to Python code by converting it into a [`PyErr`](crate::PyErr), where Python code can then catch it. ``` use pyo3::prelude::*; use pyo3::exceptions::Py", $name, "; #[pyfunction] fn always_throws() -> PyResult<()> { let message = \"I'm ", $name ,", and I was raised from Rust.\"; Err(Py", $name, "::new_err(message)) } # # Python::with_gil(|py| { # let fun = pyo3::wrap_pyfunction!(always_throws, py).unwrap(); # let err = fun.call0().expect_err(\"called a function that should always return an error but the return value was Ok\"); # assert!(err.is_instance_of::(py)) # }); ``` Python code: ```python from my_module import always_throws try: always_throws() except ", $name, " as e: print(f\"Caught an exception: {e}\") ``` # Example: Catching ", $name, " in Rust ``` use pyo3::prelude::*; use pyo3::exceptions::Py", $name, "; Python::with_gil(|py| { let result: PyResult<()> = py.run(\"raise ", $name, "\", None, None); let error_type = match result { Ok(_) => \"Not an error\", Err(error) if error.is_instance_of::(py) => \"" , $name, "\", Err(_) => \"Some other error\", }; assert_eq!(error_type, \"", $name, "\"); }); ``` " ) ); ); impl_native_exception!( PyBaseException, PyExc_BaseException, native_doc!("BaseException"), ffi::PyBaseExceptionObject, #checkfunction=ffi::PyExceptionInstance_Check ); impl_native_exception!(PyException, PyExc_Exception, native_doc!("Exception")); impl_native_exception!( PyStopAsyncIteration, PyExc_StopAsyncIteration, native_doc!("StopAsyncIteration") ); impl_native_exception!( PyStopIteration, PyExc_StopIteration, native_doc!("StopIteration"), ffi::PyStopIterationObject ); impl_native_exception!( PyGeneratorExit, PyExc_GeneratorExit, native_doc!("GeneratorExit") ); impl_native_exception!( PyArithmeticError, PyExc_ArithmeticError, native_doc!("ArithmeticError") ); impl_native_exception!(PyLookupError, PyExc_LookupError, native_doc!("LookupError")); impl_native_exception!( PyAssertionError, PyExc_AssertionError, native_doc!("AssertionError") ); impl_native_exception!( PyAttributeError, PyExc_AttributeError, native_doc!("AttributeError") ); impl_native_exception!(PyBufferError, PyExc_BufferError, native_doc!("BufferError")); impl_native_exception!(PyEOFError, PyExc_EOFError, native_doc!("EOFError")); impl_native_exception!( PyFloatingPointError, PyExc_FloatingPointError, native_doc!("FloatingPointError") ); #[cfg(not(PyPy))] impl_native_exception!( PyOSError, PyExc_OSError, native_doc!("OSError"), ffi::PyOSErrorObject ); #[cfg(PyPy)] impl_native_exception!(PyOSError, PyExc_OSError, native_doc!("OSError")); impl_native_exception!(PyImportError, PyExc_ImportError, native_doc!("ImportError")); impl_native_exception!( PyModuleNotFoundError, PyExc_ModuleNotFoundError, native_doc!("ModuleNotFoundError") ); impl_native_exception!(PyIndexError, PyExc_IndexError, native_doc!("IndexError")); impl_native_exception!(PyKeyError, PyExc_KeyError, native_doc!("KeyError")); impl_native_exception!( PyKeyboardInterrupt, PyExc_KeyboardInterrupt, native_doc!("KeyboardInterrupt") ); impl_native_exception!(PyMemoryError, PyExc_MemoryError, native_doc!("MemoryError")); impl_native_exception!(PyNameError, PyExc_NameError, native_doc!("NameError")); impl_native_exception!( PyOverflowError, PyExc_OverflowError, native_doc!("OverflowError") ); impl_native_exception!( PyRuntimeError, PyExc_RuntimeError, native_doc!("RuntimeError") ); impl_native_exception!( PyRecursionError, PyExc_RecursionError, native_doc!("RecursionError") ); impl_native_exception!( PyNotImplementedError, PyExc_NotImplementedError, native_doc!("NotImplementedError") ); #[cfg(not(PyPy))] impl_native_exception!( PySyntaxError, PyExc_SyntaxError, native_doc!("SyntaxError"), ffi::PySyntaxErrorObject ); #[cfg(PyPy)] impl_native_exception!(PySyntaxError, PyExc_SyntaxError, native_doc!("SyntaxError")); impl_native_exception!( PyReferenceError, PyExc_ReferenceError, native_doc!("ReferenceError") ); impl_native_exception!(PySystemError, PyExc_SystemError, native_doc!("SystemError")); #[cfg(not(PyPy))] impl_native_exception!( PySystemExit, PyExc_SystemExit, native_doc!("SystemExit"), ffi::PySystemExitObject ); #[cfg(PyPy)] impl_native_exception!(PySystemExit, PyExc_SystemExit, native_doc!("SystemExit")); impl_native_exception!(PyTypeError, PyExc_TypeError, native_doc!("TypeError")); impl_native_exception!( PyUnboundLocalError, PyExc_UnboundLocalError, native_doc!("UnboundLocalError") ); #[cfg(not(PyPy))] impl_native_exception!( PyUnicodeError, PyExc_UnicodeError, native_doc!("UnicodeError"), ffi::PyUnicodeErrorObject ); #[cfg(PyPy)] impl_native_exception!( PyUnicodeError, PyExc_UnicodeError, native_doc!("UnicodeError") ); // these four errors need arguments, so they're too annoying to write tests for using macros... impl_native_exception!( PyUnicodeDecodeError, PyExc_UnicodeDecodeError, native_doc!("UnicodeDecodeError", "") ); impl_native_exception!( PyUnicodeEncodeError, PyExc_UnicodeEncodeError, native_doc!("UnicodeEncodeError", "") ); impl_native_exception!( PyUnicodeTranslateError, PyExc_UnicodeTranslateError, native_doc!("UnicodeTranslateError", "") ); #[cfg(Py_3_11)] impl_native_exception!( PyBaseExceptionGroup, PyExc_BaseExceptionGroup, native_doc!("BaseExceptionGroup", "") ); impl_native_exception!(PyValueError, PyExc_ValueError, native_doc!("ValueError")); impl_native_exception!( PyZeroDivisionError, PyExc_ZeroDivisionError, native_doc!("ZeroDivisionError") ); impl_native_exception!( PyBlockingIOError, PyExc_BlockingIOError, native_doc!("BlockingIOError") ); impl_native_exception!( PyBrokenPipeError, PyExc_BrokenPipeError, native_doc!("BrokenPipeError") ); impl_native_exception!( PyChildProcessError, PyExc_ChildProcessError, native_doc!("ChildProcessError") ); impl_native_exception!( PyConnectionError, PyExc_ConnectionError, native_doc!("ConnectionError") ); impl_native_exception!( PyConnectionAbortedError, PyExc_ConnectionAbortedError, native_doc!("ConnectionAbortedError") ); impl_native_exception!( PyConnectionRefusedError, PyExc_ConnectionRefusedError, native_doc!("ConnectionRefusedError") ); impl_native_exception!( PyConnectionResetError, PyExc_ConnectionResetError, native_doc!("ConnectionResetError") ); impl_native_exception!( PyFileExistsError, PyExc_FileExistsError, native_doc!("FileExistsError") ); impl_native_exception!( PyFileNotFoundError, PyExc_FileNotFoundError, native_doc!("FileNotFoundError") ); impl_native_exception!( PyInterruptedError, PyExc_InterruptedError, native_doc!("InterruptedError") ); impl_native_exception!( PyIsADirectoryError, PyExc_IsADirectoryError, native_doc!("IsADirectoryError") ); impl_native_exception!( PyNotADirectoryError, PyExc_NotADirectoryError, native_doc!("NotADirectoryError") ); impl_native_exception!( PyPermissionError, PyExc_PermissionError, native_doc!("PermissionError") ); impl_native_exception!( PyProcessLookupError, PyExc_ProcessLookupError, native_doc!("ProcessLookupError") ); impl_native_exception!( PyTimeoutError, PyExc_TimeoutError, native_doc!("TimeoutError") ); impl_native_exception!( PyEnvironmentError, PyExc_EnvironmentError, native_doc!("EnvironmentError") ); impl_native_exception!(PyIOError, PyExc_IOError, native_doc!("IOError")); #[cfg(windows)] impl_windows_native_exception!( PyWindowsError, PyExc_WindowsError, native_doc!("WindowsError") ); impl PyUnicodeDecodeError { /// Creates a Python `UnicodeDecodeError`. pub fn new<'p>( py: Python<'p>, encoding: &CStr, input: &[u8], range: ops::Range, reason: &CStr, ) -> PyResult<&'p PyUnicodeDecodeError> { unsafe { py.from_owned_ptr_or_err(ffi::PyUnicodeDecodeError_Create( encoding.as_ptr(), input.as_ptr() as *const c_char, input.len() as ffi::Py_ssize_t, range.start as ffi::Py_ssize_t, range.end as ffi::Py_ssize_t, reason.as_ptr(), )) } } /// Creates a Python `UnicodeDecodeError` from a Rust UTF-8 decoding error. /// /// # Examples /// /// ``` /// #![cfg_attr(invalid_from_utf8_lint, allow(invalid_from_utf8))] /// use pyo3::prelude::*; /// use pyo3::exceptions::PyUnicodeDecodeError; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// let invalid_utf8 = b"fo\xd8o"; /// let err = std::str::from_utf8(invalid_utf8).expect_err("should be invalid utf8"); /// let decode_err = PyUnicodeDecodeError::new_utf8(py, invalid_utf8, err)?; /// assert_eq!( /// decode_err.to_string(), /// "'utf-8' codec can't decode byte 0xd8 in position 2: invalid utf-8" /// ); /// Ok(()) /// }) /// # } pub fn new_utf8<'p>( py: Python<'p>, input: &[u8], err: std::str::Utf8Error, ) -> PyResult<&'p PyUnicodeDecodeError> { let pos = err.valid_up_to(); PyUnicodeDecodeError::new( py, CStr::from_bytes_with_nul(b"utf-8\0").unwrap(), input, pos..(pos + 1), CStr::from_bytes_with_nul(b"invalid utf-8\0").unwrap(), ) } } impl_native_exception!(PyWarning, PyExc_Warning, native_doc!("Warning")); impl_native_exception!(PyUserWarning, PyExc_UserWarning, native_doc!("UserWarning")); impl_native_exception!( PyDeprecationWarning, PyExc_DeprecationWarning, native_doc!("DeprecationWarning") ); impl_native_exception!( PyPendingDeprecationWarning, PyExc_PendingDeprecationWarning, native_doc!("PendingDeprecationWarning") ); impl_native_exception!( PySyntaxWarning, PyExc_SyntaxWarning, native_doc!("SyntaxWarning") ); impl_native_exception!( PyRuntimeWarning, PyExc_RuntimeWarning, native_doc!("RuntimeWarning") ); impl_native_exception!( PyFutureWarning, PyExc_FutureWarning, native_doc!("FutureWarning") ); impl_native_exception!( PyImportWarning, PyExc_ImportWarning, native_doc!("ImportWarning") ); impl_native_exception!( PyUnicodeWarning, PyExc_UnicodeWarning, native_doc!("UnicodeWarning") ); impl_native_exception!( PyBytesWarning, PyExc_BytesWarning, native_doc!("BytesWarning") ); impl_native_exception!( PyResourceWarning, PyExc_ResourceWarning, native_doc!("ResourceWarning") ); #[cfg(Py_3_10)] impl_native_exception!( PyEncodingWarning, PyExc_EncodingWarning, native_doc!("EncodingWarning") ); #[cfg(test)] macro_rules! test_exception { ($exc_ty:ident $(, |$py:tt| $constructor:expr )?) => { #[allow(non_snake_case)] #[test] fn $exc_ty () { use super::$exc_ty; $crate::Python::with_gil(|py| { use std::error::Error; let err: $crate::PyErr = { None $( .or(Some({ let $py = py; $constructor })) )? .unwrap_or($exc_ty::new_err("a test exception")) }; assert!(err.is_instance_of::<$exc_ty>(py)); let value: &$exc_ty = err.value(py).downcast().unwrap(); assert!(value.source().is_none()); err.set_cause(py, Some($crate::exceptions::PyValueError::new_err("a cause"))); assert!(value.source().is_some()); assert!($crate::PyErr::from(value).is_instance_of::<$exc_ty>(py)); }) } }; } /// Exceptions defined in Python's [`asyncio`](https://docs.python.org/3/library/asyncio.html) /// module. pub mod asyncio { import_exception!(asyncio, CancelledError); import_exception!(asyncio, InvalidStateError); import_exception!(asyncio, TimeoutError); import_exception!(asyncio, IncompleteReadError); import_exception!(asyncio, LimitOverrunError); import_exception!(asyncio, QueueEmpty); import_exception!(asyncio, QueueFull); #[cfg(test)] mod tests { test_exception!(CancelledError); test_exception!(InvalidStateError); test_exception!(TimeoutError); test_exception!(IncompleteReadError, |_| IncompleteReadError::new_err(( "partial", "expected" ))); test_exception!(LimitOverrunError, |_| LimitOverrunError::new_err(( "message", "consumed" ))); test_exception!(QueueEmpty); test_exception!(QueueFull); } } /// Exceptions defined in Python's [`socket`](https://docs.python.org/3/library/socket.html) /// module. pub mod socket { import_exception!(socket, herror); import_exception!(socket, gaierror); import_exception!(socket, timeout); #[cfg(test)] mod tests { test_exception!(herror); test_exception!(gaierror); test_exception!(timeout); } } #[cfg(test)] mod tests { use super::*; use crate::types::{IntoPyDict, PyDict}; use crate::{PyErr, Python}; import_exception!(socket, gaierror); import_exception!(email.errors, MessageError); #[test] fn test_check_exception() { Python::with_gil(|py| { let err: PyErr = gaierror::new_err(()); let socket = py .import("socket") .map_err(|e| e.display(py)) .expect("could not import socket"); let d = PyDict::new(py); d.set_item("socket", socket) .map_err(|e| e.display(py)) .expect("could not setitem"); d.set_item("exc", err) .map_err(|e| e.display(py)) .expect("could not setitem"); py.run("assert isinstance(exc, socket.gaierror)", None, Some(d)) .map_err(|e| e.display(py)) .expect("assertion failed"); }); } #[test] fn test_check_exception_nested() { Python::with_gil(|py| { let err: PyErr = MessageError::new_err(()); let email = py .import("email") .map_err(|e| e.display(py)) .expect("could not import email"); let d = PyDict::new(py); d.set_item("email", email) .map_err(|e| e.display(py)) .expect("could not setitem"); d.set_item("exc", err) .map_err(|e| e.display(py)) .expect("could not setitem"); py.run( "assert isinstance(exc, email.errors.MessageError)", None, Some(d), ) .map_err(|e| e.display(py)) .expect("assertion failed"); }); } #[test] fn custom_exception() { create_exception!(mymodule, CustomError, PyException); Python::with_gil(|py| { let error_type = py.get_type::(); let ctx = [("CustomError", error_type)].into_py_dict(py); let type_description: String = py .eval("str(CustomError)", None, Some(ctx)) .unwrap() .extract() .unwrap(); assert_eq!(type_description, ""); py.run( "assert CustomError('oops').args == ('oops',)", None, Some(ctx), ) .unwrap(); py.run("assert CustomError.__doc__ is None", None, Some(ctx)) .unwrap(); }); } #[test] fn custom_exception_dotted_module() { create_exception!(mymodule.exceptions, CustomError, PyException); Python::with_gil(|py| { let error_type = py.get_type::(); let ctx = [("CustomError", error_type)].into_py_dict(py); let type_description: String = py .eval("str(CustomError)", None, Some(ctx)) .unwrap() .extract() .unwrap(); assert_eq!( type_description, "" ); }); } #[test] fn custom_exception_doc() { create_exception!(mymodule, CustomError, PyException, "Some docs"); Python::with_gil(|py| { let error_type = py.get_type::(); let ctx = [("CustomError", error_type)].into_py_dict(py); let type_description: String = py .eval("str(CustomError)", None, Some(ctx)) .unwrap() .extract() .unwrap(); assert_eq!(type_description, ""); py.run( "assert CustomError('oops').args == ('oops',)", None, Some(ctx), ) .unwrap(); py.run("assert CustomError.__doc__ == 'Some docs'", None, Some(ctx)) .unwrap(); }); } #[test] fn custom_exception_doc_expr() { create_exception!( mymodule, CustomError, PyException, concat!("Some", " more ", stringify!(docs)) ); Python::with_gil(|py| { let error_type = py.get_type::(); let ctx = [("CustomError", error_type)].into_py_dict(py); let type_description: String = py .eval("str(CustomError)", None, Some(ctx)) .unwrap() .extract() .unwrap(); assert_eq!(type_description, ""); py.run( "assert CustomError('oops').args == ('oops',)", None, Some(ctx), ) .unwrap(); py.run( "assert CustomError.__doc__ == 'Some more docs'", None, Some(ctx), ) .unwrap(); }); } #[test] fn native_exception_debug() { Python::with_gil(|py| { let exc = py .run("raise Exception('banana')", None, None) .expect_err("raising should have given us an error") .into_value(py) .into_ref(py); assert_eq!( format!("{:?}", exc), exc.repr().unwrap().extract::().unwrap() ); }); } #[test] fn native_exception_display() { Python::with_gil(|py| { let exc = py .run("raise Exception('banana')", None, None) .expect_err("raising should have given us an error") .into_value(py) .into_ref(py); assert_eq!( exc.to_string(), exc.str().unwrap().extract::().unwrap() ); }); } #[test] fn native_exception_chain() { use std::error::Error; Python::with_gil(|py| { let exc = py .run( "raise Exception('banana') from TypeError('peach')", None, None, ) .expect_err("raising should have given us an error") .into_value(py) .into_ref(py); assert_eq!(format!("{:?}", exc), "Exception('banana')"); let source = exc.source().expect("cause should exist"); assert_eq!(format!("{:?}", source), "TypeError('peach')"); let source_source = source.source(); assert!(source_source.is_none(), "source_source should be None"); }); } #[test] fn unicode_decode_error() { let invalid_utf8 = b"fo\xd8o"; #[cfg_attr(invalid_from_utf8_lint, allow(invalid_from_utf8))] let err = std::str::from_utf8(invalid_utf8).expect_err("should be invalid utf8"); Python::with_gil(|py| { let decode_err = PyUnicodeDecodeError::new_utf8(py, invalid_utf8, err).unwrap(); assert_eq!( format!("{:?}", decode_err), "UnicodeDecodeError('utf-8', b'fo\\xd8o', 2, 3, 'invalid utf-8')" ); // Restoring should preserve the same error let e: PyErr = decode_err.into(); e.restore(py); assert_eq!( PyErr::fetch(py).to_string(), "UnicodeDecodeError: \'utf-8\' codec can\'t decode byte 0xd8 in position 2: invalid utf-8" ); }); } #[cfg(Py_3_11)] test_exception!(PyBaseExceptionGroup, |_| PyBaseExceptionGroup::new_err(( "msg", vec![PyValueError::new_err("err")] ))); test_exception!(PyBaseException); test_exception!(PyException); test_exception!(PyStopAsyncIteration); test_exception!(PyStopIteration); test_exception!(PyGeneratorExit); test_exception!(PyArithmeticError); test_exception!(PyLookupError); test_exception!(PyAssertionError); test_exception!(PyAttributeError); test_exception!(PyBufferError); test_exception!(PyEOFError); test_exception!(PyFloatingPointError); test_exception!(PyOSError); test_exception!(PyImportError); test_exception!(PyModuleNotFoundError); test_exception!(PyIndexError); test_exception!(PyKeyError); test_exception!(PyKeyboardInterrupt); test_exception!(PyMemoryError); test_exception!(PyNameError); test_exception!(PyOverflowError); test_exception!(PyRuntimeError); test_exception!(PyRecursionError); test_exception!(PyNotImplementedError); test_exception!(PySyntaxError); test_exception!(PyReferenceError); test_exception!(PySystemError); test_exception!(PySystemExit); test_exception!(PyTypeError); test_exception!(PyUnboundLocalError); test_exception!(PyUnicodeError); test_exception!(PyUnicodeDecodeError, |py| { let invalid_utf8 = b"fo\xd8o"; #[cfg_attr(invalid_from_utf8_lint, allow(invalid_from_utf8))] let err = std::str::from_utf8(invalid_utf8).expect_err("should be invalid utf8"); PyErr::from_value(PyUnicodeDecodeError::new_utf8(py, invalid_utf8, err).unwrap()) }); test_exception!(PyUnicodeEncodeError, |py| py .eval("chr(40960).encode('ascii')", None, None) .unwrap_err()); test_exception!(PyUnicodeTranslateError, |_| { PyUnicodeTranslateError::new_err(("\u{3042}", 0, 1, "ouch")) }); test_exception!(PyValueError); test_exception!(PyZeroDivisionError); test_exception!(PyBlockingIOError); test_exception!(PyBrokenPipeError); test_exception!(PyChildProcessError); test_exception!(PyConnectionError); test_exception!(PyConnectionAbortedError); test_exception!(PyConnectionRefusedError); test_exception!(PyConnectionResetError); test_exception!(PyFileExistsError); test_exception!(PyFileNotFoundError); test_exception!(PyInterruptedError); test_exception!(PyIsADirectoryError); test_exception!(PyNotADirectoryError); test_exception!(PyPermissionError); test_exception!(PyProcessLookupError); test_exception!(PyTimeoutError); test_exception!(PyEnvironmentError); test_exception!(PyIOError); #[cfg(windows)] test_exception!(PyWindowsError); test_exception!(PyWarning); test_exception!(PyUserWarning); test_exception!(PyDeprecationWarning); test_exception!(PyPendingDeprecationWarning); test_exception!(PySyntaxWarning); test_exception!(PyRuntimeWarning); test_exception!(PyFutureWarning); test_exception!(PyImportWarning); test_exception!(PyUnicodeWarning); test_exception!(PyBytesWarning); #[cfg(Py_3_10)] test_exception!(PyEncodingWarning); } pyo3-0.20.2/src/ffi/mod.rs000064400000000000000000000023151046102023000132600ustar 00000000000000//! Raw FFI declarations for Python's C API. //! //! This module provides low level bindings to the Python interpreter. //! It is meant for advanced users only - regular PyO3 users shouldn't //! need to interact with this module at all. //! //! The contents of this module are not documented here, as it would entail //! basically copying the documentation from CPython. Consult the [Python/C API Reference //! Manual][capi] for up-to-date documentation. //! //! # Safety //! //! The functions in this module lack individual safety documentation, but //! generally the following apply: //! - Pointer arguments have to point to a valid Python object of the correct type, //! although null pointers are sometimes valid input. //! - The vast majority can only be used safely while the GIL is held. //! - Some functions have additional safety requirements, consult the //! [Python/C API Reference Manual][capi] //! for more information. //! //! [capi]: https://docs.python.org/3/c-api/index.html #[cfg(test)] mod tests; // reexport raw bindings exposed in pyo3_ffi pub use pyo3_ffi::*; /// Helper to enable #\[pymethods\] to see the workaround for __ipow__ on Python 3.7 #[doc(hidden)] pub use crate::impl_::pymethods::ipowfunc; pyo3-0.20.2/src/ffi/tests.rs000064400000000000000000000225771046102023000136570ustar 00000000000000use crate::ffi::*; use crate::Python; #[cfg(not(Py_LIMITED_API))] use crate::{ types::{PyDict, PyString}, IntoPy, Py, PyAny, }; #[cfg(not(any(Py_3_12, Py_LIMITED_API)))] use libc::wchar_t; #[cfg(not(Py_LIMITED_API))] #[cfg_attr(target_arch = "wasm32", ignore)] // DateTime import fails on wasm for mysterious reasons #[test] fn test_datetime_fromtimestamp() { Python::with_gil(|py| { let args: Py = (100,).into_py(py); let dt: &PyAny = unsafe { PyDateTime_IMPORT(); py.from_owned_ptr(PyDateTime_FromTimestamp(args.as_ptr())) }; let locals = PyDict::new(py); locals.set_item("dt", dt).unwrap(); py.run( "import datetime; assert dt == datetime.datetime.fromtimestamp(100)", None, Some(locals), ) .unwrap(); }) } #[cfg(not(Py_LIMITED_API))] #[cfg_attr(target_arch = "wasm32", ignore)] // DateTime import fails on wasm for mysterious reasons #[test] fn test_date_fromtimestamp() { Python::with_gil(|py| { let args: Py = (100,).into_py(py); let dt: &PyAny = unsafe { PyDateTime_IMPORT(); py.from_owned_ptr(PyDate_FromTimestamp(args.as_ptr())) }; let locals = PyDict::new(py); locals.set_item("dt", dt).unwrap(); py.run( "import datetime; assert dt == datetime.date.fromtimestamp(100)", None, Some(locals), ) .unwrap(); }) } #[cfg(not(Py_LIMITED_API))] #[cfg_attr(target_arch = "wasm32", ignore)] // DateTime import fails on wasm for mysterious reasons #[test] fn test_utc_timezone() { Python::with_gil(|py| { let utc_timezone: &PyAny = unsafe { PyDateTime_IMPORT(); py.from_borrowed_ptr(PyDateTime_TimeZone_UTC()) }; let locals = PyDict::new(py); locals.set_item("utc_timezone", utc_timezone).unwrap(); py.run( "import datetime; assert utc_timezone is datetime.timezone.utc", None, Some(locals), ) .unwrap(); }) } #[test] #[cfg(not(Py_LIMITED_API))] #[cfg(feature = "macros")] #[cfg_attr(target_arch = "wasm32", ignore)] // DateTime import fails on wasm for mysterious reasons fn test_timezone_from_offset() { use crate::types::PyDelta; Python::with_gil(|py| { let delta = PyDelta::new(py, 0, 100, 0, false).unwrap(); let tz: &PyAny = unsafe { py.from_borrowed_ptr(PyTimeZone_FromOffset(delta.as_ptr())) }; crate::py_run!( py, tz, "import datetime; assert tz == datetime.timezone(datetime.timedelta(seconds=100))" ); }) } #[test] #[cfg(not(Py_LIMITED_API))] #[cfg(feature = "macros")] #[cfg_attr(target_arch = "wasm32", ignore)] // DateTime import fails on wasm for mysterious reasons fn test_timezone_from_offset_and_name() { use crate::types::PyDelta; Python::with_gil(|py| { let delta = PyDelta::new(py, 0, 100, 0, false).unwrap(); let tzname = PyString::new(py, "testtz"); let tz: &PyAny = unsafe { py.from_borrowed_ptr(PyTimeZone_FromOffsetAndName( delta.as_ptr(), tzname.as_ptr(), )) }; crate::py_run!( py, tz, "import datetime; assert tz == datetime.timezone(datetime.timedelta(seconds=100), 'testtz')" ); }) } #[test] #[cfg(not(Py_LIMITED_API))] fn ascii_object_bitfield() { let ob_base: PyObject = unsafe { std::mem::zeroed() }; let mut o = PyASCIIObject { ob_base, length: 0, #[cfg(not(PyPy))] hash: 0, state: 0u32, #[cfg(not(Py_3_12))] wstr: std::ptr::null_mut() as *mut wchar_t, }; unsafe { assert_eq!(o.interned(), 0); assert_eq!(o.kind(), 0); assert_eq!(o.compact(), 0); assert_eq!(o.ascii(), 0); #[cfg(not(Py_3_12))] assert_eq!(o.ready(), 0); let interned_count = if cfg!(Py_3_12) { 2 } else { 4 }; for i in 0..interned_count { o.set_interned(i); assert_eq!(o.interned(), i); } for i in 0..8 { o.set_kind(i); assert_eq!(o.kind(), i); } o.set_compact(1); assert_eq!(o.compact(), 1); o.set_ascii(1); assert_eq!(o.ascii(), 1); #[cfg(not(Py_3_12))] o.set_ready(1); #[cfg(not(Py_3_12))] assert_eq!(o.ready(), 1); } } #[test] #[cfg(not(Py_LIMITED_API))] #[cfg_attr(Py_3_10, allow(deprecated))] fn ascii() { Python::with_gil(|py| { // This test relies on implementation details of PyString. let s = PyString::new(py, "hello, world"); let ptr = s.as_ptr(); unsafe { let ascii_ptr = ptr as *mut PyASCIIObject; let ascii = ascii_ptr.as_ref().unwrap(); assert_eq!(ascii.interned(), 0); assert_eq!(ascii.kind(), PyUnicode_1BYTE_KIND); assert_eq!(ascii.compact(), 1); assert_eq!(ascii.ascii(), 1); #[cfg(not(Py_3_12))] assert_eq!(ascii.ready(), 1); assert_eq!(PyUnicode_IS_ASCII(ptr), 1); assert_eq!(PyUnicode_IS_COMPACT(ptr), 1); assert_eq!(PyUnicode_IS_COMPACT_ASCII(ptr), 1); assert!(!PyUnicode_1BYTE_DATA(ptr).is_null()); // 2 and 4 byte macros return nonsense for this string instance. assert_eq!(PyUnicode_KIND(ptr), PyUnicode_1BYTE_KIND); assert!(!_PyUnicode_COMPACT_DATA(ptr).is_null()); // _PyUnicode_NONCOMPACT_DATA isn't valid for compact strings. assert!(!PyUnicode_DATA(ptr).is_null()); assert_eq!(PyUnicode_GET_LENGTH(ptr), s.len().unwrap() as Py_ssize_t); assert_eq!(PyUnicode_IS_READY(ptr), 1); // This has potential to mutate object. But it should be a no-op since // we're already ready. assert_eq!(PyUnicode_READY(ptr), 0); } }) } #[test] #[cfg(not(Py_LIMITED_API))] #[cfg_attr(Py_3_10, allow(deprecated))] fn ucs4() { Python::with_gil(|py| { let s = "哈哈🐈"; let py_string = PyString::new(py, s); let ptr = py_string.as_ptr(); unsafe { let ascii_ptr = ptr as *mut PyASCIIObject; let ascii = ascii_ptr.as_ref().unwrap(); assert_eq!(ascii.interned(), 0); assert_eq!(ascii.kind(), PyUnicode_4BYTE_KIND); assert_eq!(ascii.compact(), 1); assert_eq!(ascii.ascii(), 0); #[cfg(not(Py_3_12))] assert_eq!(ascii.ready(), 1); assert_eq!(PyUnicode_IS_ASCII(ptr), 0); assert_eq!(PyUnicode_IS_COMPACT(ptr), 1); assert_eq!(PyUnicode_IS_COMPACT_ASCII(ptr), 0); assert!(!PyUnicode_4BYTE_DATA(ptr).is_null()); assert_eq!(PyUnicode_KIND(ptr), PyUnicode_4BYTE_KIND); assert!(!_PyUnicode_COMPACT_DATA(ptr).is_null()); // _PyUnicode_NONCOMPACT_DATA isn't valid for compact strings. assert!(!PyUnicode_DATA(ptr).is_null()); assert_eq!( PyUnicode_GET_LENGTH(ptr), py_string.len().unwrap() as Py_ssize_t ); assert_eq!(PyUnicode_IS_READY(ptr), 1); // This has potential to mutate object. But it should be a no-op since // we're already ready. assert_eq!(PyUnicode_READY(ptr), 0); } }) } #[test] #[cfg(not(Py_LIMITED_API))] #[cfg_attr(target_arch = "wasm32", ignore)] // DateTime import fails on wasm for mysterious reasons #[cfg(not(PyPy))] fn test_get_tzinfo() { use crate::types::timezone_utc; crate::Python::with_gil(|py| { use crate::types::{PyDateTime, PyTime}; use crate::PyAny; let utc = timezone_utc(py); let dt = PyDateTime::new(py, 2018, 1, 1, 0, 0, 0, 0, Some(utc)).unwrap(); assert!( unsafe { py.from_borrowed_ptr::(PyDateTime_DATE_GET_TZINFO(dt.as_ptr())) } .is(utc) ); let dt = PyDateTime::new(py, 2018, 1, 1, 0, 0, 0, 0, None).unwrap(); assert!( unsafe { py.from_borrowed_ptr::(PyDateTime_DATE_GET_TZINFO(dt.as_ptr())) } .is_none() ); let t = PyTime::new(py, 0, 0, 0, 0, Some(utc)).unwrap(); assert!( unsafe { py.from_borrowed_ptr::(PyDateTime_TIME_GET_TZINFO(t.as_ptr())) } .is(utc) ); let t = PyTime::new(py, 0, 0, 0, 0, None).unwrap(); assert!( unsafe { py.from_borrowed_ptr::(PyDateTime_TIME_GET_TZINFO(t.as_ptr())) } .is_none() ); }) } #[test] fn test_inc_dec_ref() { Python::with_gil(|py| { let obj = py.eval("object()", None, None).unwrap(); let ref_count = obj.get_refcnt(); let ptr = obj.as_ptr(); unsafe { Py_INCREF(ptr) }; assert_eq!(obj.get_refcnt(), ref_count + 1); unsafe { Py_DECREF(ptr) }; assert_eq!(obj.get_refcnt(), ref_count); }) } #[test] #[cfg(Py_3_12)] fn test_inc_dec_ref_immortal() { Python::with_gil(|py| { let obj = py.None(); let ref_count = obj.get_refcnt(py); let ptr = obj.as_ptr(); unsafe { Py_INCREF(ptr) }; assert_eq!(obj.get_refcnt(py), ref_count); unsafe { Py_DECREF(ptr) }; assert_eq!(obj.get_refcnt(py), ref_count); }) } pyo3-0.20.2/src/gil.rs000064400000000000000000000754141046102023000125220ustar 00000000000000//! Interaction with Python's global interpreter lock use crate::impl_::not_send::{NotSend, NOT_SEND}; use crate::{ffi, Python}; use parking_lot::{const_mutex, Mutex, Once}; use std::cell::Cell; #[cfg(debug_assertions)] use std::cell::RefCell; #[cfg(not(debug_assertions))] use std::cell::UnsafeCell; use std::{mem, ptr::NonNull}; static START: Once = Once::new(); cfg_if::cfg_if! { if #[cfg(thread_local_const_init)] { use std::thread_local as thread_local_const_init; } else { macro_rules! thread_local_const_init { ($($(#[$attr:meta])* static $name:ident: $ty:ty = const { $init:expr };)*) => ( thread_local! { $($(#[$attr])* static $name: $ty = $init;)* } ) } } } thread_local_const_init! { /// This is an internal counter in pyo3 monitoring whether this thread has the GIL. /// /// It will be incremented whenever a GILGuard or GILPool is created, and decremented whenever /// they are dropped. /// /// As a result, if this thread has the GIL, GIL_COUNT is greater than zero. /// /// Additionally, we sometimes need to prevent safe access to the GIL, /// e.g. when implementing `__traverse__`, which is represented by a negative value. static GIL_COUNT: Cell = const { Cell::new(0) }; /// Temporarily hold objects that will be released when the GILPool drops. #[cfg(debug_assertions)] static OWNED_OBJECTS: RefCell = const { RefCell::new(Vec::new()) }; #[cfg(not(debug_assertions))] static OWNED_OBJECTS: UnsafeCell = const { UnsafeCell::new(Vec::new()) }; } const GIL_LOCKED_DURING_TRAVERSE: isize = -1; /// Checks whether the GIL is acquired. /// /// Note: This uses pyo3's internal count rather than PyGILState_Check for two reasons: /// 1) for performance /// 2) PyGILState_Check always returns 1 if the sub-interpreter APIs have ever been called, /// which could lead to incorrect conclusions that the GIL is held. #[inline(always)] fn gil_is_acquired() -> bool { GIL_COUNT.try_with(|c| c.get() > 0).unwrap_or(false) } /// Prepares the use of Python in a free-threaded context. /// /// If the Python interpreter is not already initialized, this function will initialize it with /// signal handling disabled (Python will not raise the `KeyboardInterrupt` exception). Python /// signal handling depends on the notion of a 'main thread', which must be the thread that /// initializes the Python interpreter. /// /// If the Python interpreter is already initialized, this function has no effect. /// /// This function is unavailable under PyPy because PyPy cannot be embedded in Rust (or any other /// software). Support for this is tracked on the /// [PyPy issue tracker](https://foss.heptapod.net/pypy/pypy/-/issues/3286). /// /// # Examples /// ```rust /// use pyo3::prelude::*; /// /// # fn main() -> PyResult<()> { /// pyo3::prepare_freethreaded_python(); /// Python::with_gil(|py| py.run("print('Hello World')", None, None)) /// # } /// ``` #[cfg(not(PyPy))] pub fn prepare_freethreaded_python() { // Protect against race conditions when Python is not yet initialized and multiple threads // concurrently call 'prepare_freethreaded_python()'. Note that we do not protect against // concurrent initialization of the Python runtime by other users of the Python C API. START.call_once_force(|_| unsafe { // Use call_once_force because if initialization panics, it's okay to try again. if ffi::Py_IsInitialized() == 0 { ffi::Py_InitializeEx(0); // Release the GIL. ffi::PyEval_SaveThread(); } }); } /// Executes the provided closure with an embedded Python interpreter. /// /// This function initializes the Python interpreter, executes the provided closure, and then /// finalizes the Python interpreter. /// /// After execution all Python resources are cleaned up, and no further Python APIs can be called. /// Because many Python modules implemented in C do not support multiple Python interpreters in a /// single process, it is not safe to call this function more than once. (Many such modules will not /// initialize correctly on the second run.) /// /// # Panics /// - If the Python interpreter is already initialized before calling this function. /// /// # Safety /// - This function should only ever be called once per process (usually as part of the `main` /// function). It is also not thread-safe. /// - No Python APIs can be used after this function has finished executing. /// - The return value of the closure must not contain any Python value, _including_ `PyResult`. /// /// # Examples /// /// ```rust /// unsafe { /// pyo3::with_embedded_python_interpreter(|py| { /// if let Err(e) = py.run("print('Hello World')", None, None) { /// // We must make sure to not return a `PyErr`! /// e.print(py); /// } /// }); /// } /// ``` #[cfg(not(PyPy))] pub unsafe fn with_embedded_python_interpreter(f: F) -> R where F: for<'p> FnOnce(Python<'p>) -> R, { assert_eq!( ffi::Py_IsInitialized(), 0, "called `with_embedded_python_interpreter` but a Python interpreter is already running." ); ffi::Py_InitializeEx(0); // Safety: the GIL is already held because of the Py_IntializeEx call. let pool = GILPool::new(); // Import the threading module - this ensures that it will associate this thread as the "main" // thread, which is important to avoid an `AssertionError` at finalization. pool.python().import("threading").unwrap(); // Execute the closure. let result = f(pool.python()); // Drop the pool before finalizing. drop(pool); // Finalize the Python interpreter. ffi::Py_Finalize(); result } /// RAII type that represents the Global Interpreter Lock acquisition. pub(crate) struct GILGuard { gstate: ffi::PyGILState_STATE, pool: mem::ManuallyDrop, } impl GILGuard { /// PyO3 internal API for acquiring the GIL. The public API is Python::with_gil. /// /// If the GIL was already acquired via PyO3, this returns `None`. Otherwise, /// the GIL will be acquired and a new `GILPool` created. pub(crate) fn acquire() -> Option { if gil_is_acquired() { return None; } // Maybe auto-initialize the GIL: // - If auto-initialize feature set and supported, try to initialize the interpreter. // - If the auto-initialize feature is set but unsupported, emit hard errors only when the // extension-module feature is not activated - extension modules don't care about // auto-initialize so this avoids breaking existing builds. // - Otherwise, just check the GIL is initialized. cfg_if::cfg_if! { if #[cfg(all(feature = "auto-initialize", not(PyPy)))] { prepare_freethreaded_python(); } else { // This is a "hack" to make running `cargo test` for PyO3 convenient (i.e. no need // to specify `--features auto-initialize` manually. Tests within the crate itself // all depend on the auto-initialize feature for conciseness but Cargo does not // provide a mechanism to specify required features for tests. #[cfg(not(PyPy))] if option_env!("CARGO_PRIMARY_PACKAGE").is_some() { prepare_freethreaded_python(); } START.call_once_force(|_| unsafe { // Use call_once_force because if there is a panic because the interpreter is // not initialized, it's fine for the user to initialize the interpreter and // retry. assert_ne!( ffi::Py_IsInitialized(), 0, "The Python interpreter is not initialized and the `auto-initialize` \ feature is not enabled.\n\n\ Consider calling `pyo3::prepare_freethreaded_python()` before attempting \ to use Python APIs." ); }); } } Self::acquire_unchecked() } /// Acquires the `GILGuard` without performing any state checking. /// /// This can be called in "unsafe" contexts where the normal interpreter state /// checking performed by `GILGuard::acquire` may fail. This includes calling /// as part of multi-phase interpreter initialization. pub(crate) fn acquire_unchecked() -> Option { if gil_is_acquired() { return None; } let gstate = unsafe { ffi::PyGILState_Ensure() }; // acquire GIL let pool = unsafe { mem::ManuallyDrop::new(GILPool::new()) }; Some(GILGuard { gstate, pool }) } } /// The Drop implementation for `GILGuard` will release the GIL. impl Drop for GILGuard { fn drop(&mut self) { unsafe { // Drop the objects in the pool before attempting to release the thread state mem::ManuallyDrop::drop(&mut self.pool); ffi::PyGILState_Release(self.gstate); } } } // Vector of PyObject type PyObjVec = Vec>; /// Thread-safe storage for objects which were inc_ref / dec_ref while the GIL was not held. struct ReferencePool { // .0 is INCREFs, .1 is DECREFs pointer_ops: Mutex<(PyObjVec, PyObjVec)>, } impl ReferencePool { const fn new() -> Self { Self { pointer_ops: const_mutex((Vec::new(), Vec::new())), } } fn register_incref(&self, obj: NonNull) { self.pointer_ops.lock().0.push(obj); } fn register_decref(&self, obj: NonNull) { self.pointer_ops.lock().1.push(obj); } fn update_counts(&self, _py: Python<'_>) { let mut ops = self.pointer_ops.lock(); if ops.0.is_empty() && ops.1.is_empty() { return; } let (increfs, decrefs) = mem::take(&mut *ops); drop(ops); // Always increase reference counts first - as otherwise objects which have a // nonzero total reference count might be incorrectly dropped by Python during // this update. for ptr in increfs { unsafe { ffi::Py_INCREF(ptr.as_ptr()) }; } for ptr in decrefs { unsafe { ffi::Py_DECREF(ptr.as_ptr()) }; } } } unsafe impl Sync for ReferencePool {} static POOL: ReferencePool = ReferencePool::new(); /// A guard which can be used to temporarily release the GIL and restore on `Drop`. pub(crate) struct SuspendGIL { count: isize, tstate: *mut ffi::PyThreadState, } impl SuspendGIL { pub(crate) unsafe fn new() -> Self { let count = GIL_COUNT.with(|c| c.replace(0)); let tstate = ffi::PyEval_SaveThread(); Self { count, tstate } } } impl Drop for SuspendGIL { fn drop(&mut self) { GIL_COUNT.with(|c| c.set(self.count)); unsafe { ffi::PyEval_RestoreThread(self.tstate); // Update counts of PyObjects / Py that were cloned or dropped while the GIL was released. POOL.update_counts(Python::assume_gil_acquired()); } } } /// Used to lock safe access to the GIL pub(crate) struct LockGIL { count: isize, } impl LockGIL { /// Lock access to the GIL while an implementation of `__traverse__` is running pub fn during_traverse() -> Self { Self::new(GIL_LOCKED_DURING_TRAVERSE) } fn new(reason: isize) -> Self { let count = GIL_COUNT.with(|c| c.replace(reason)); Self { count } } #[cold] fn bail(current: isize) { match current { GIL_LOCKED_DURING_TRAVERSE => panic!( "Access to the GIL is prohibited while a __traverse__ implmentation is running." ), _ => panic!("Access to the GIL is currently prohibited."), } } } impl Drop for LockGIL { fn drop(&mut self) { GIL_COUNT.with(|c| c.set(self.count)); } } /// A RAII pool which PyO3 uses to store owned Python references. /// /// See the [Memory Management] chapter of the guide for more information about how PyO3 uses /// [`GILPool`] to manage memory. /// /// [Memory Management]: https://pyo3.rs/main/memory.html#gil-bound-memory pub struct GILPool { /// Initial length of owned objects and anys. /// `Option` is used since TSL can be broken when `new` is called from `atexit`. start: Option, _not_send: NotSend, } impl GILPool { /// Creates a new [`GILPool`]. This function should only ever be called with the GIL held. /// /// It is recommended not to use this API directly, but instead to use [`Python::new_pool`], as /// that guarantees the GIL is held. /// /// # Safety /// /// As well as requiring the GIL, see the safety notes on [`Python::new_pool`]. #[inline] pub unsafe fn new() -> GILPool { increment_gil_count(); // Update counts of PyObjects / Py that have been cloned or dropped since last acquisition POOL.update_counts(Python::assume_gil_acquired()); GILPool { start: OWNED_OBJECTS .try_with(|owned_objects| { #[cfg(debug_assertions)] let len = owned_objects.borrow().len(); #[cfg(not(debug_assertions))] // SAFETY: This is not re-entrant. let len = unsafe { (*owned_objects.get()).len() }; len }) .ok(), _not_send: NOT_SEND, } } /// Gets the Python token associated with this [`GILPool`]. #[inline] pub fn python(&self) -> Python<'_> { unsafe { Python::assume_gil_acquired() } } } impl Drop for GILPool { fn drop(&mut self) { if let Some(start) = self.start { let owned_objects = OWNED_OBJECTS.with(|owned_objects| { #[cfg(debug_assertions)] let mut owned_objects = owned_objects.borrow_mut(); #[cfg(not(debug_assertions))] // SAFETY: `OWNED_OBJECTS` is released before calling Py_DECREF, // or Py_DECREF may call `GILPool::drop` recursively, resulting in invalid borrowing. let owned_objects = unsafe { &mut *owned_objects.get() }; if start < owned_objects.len() { owned_objects.split_off(start) } else { Vec::new() } }); for obj in owned_objects { unsafe { ffi::Py_DECREF(obj.as_ptr()); } } } decrement_gil_count(); } } /// Registers a Python object pointer inside the release pool, to have its reference count increased /// the next time the GIL is acquired in pyo3. /// /// If the GIL is held, the reference count will be increased immediately instead of being queued /// for later. /// /// # Safety /// The object must be an owned Python reference. pub unsafe fn register_incref(obj: NonNull) { if gil_is_acquired() { ffi::Py_INCREF(obj.as_ptr()) } else { POOL.register_incref(obj); } } /// Registers a Python object pointer inside the release pool, to have its reference count decreased /// the next time the GIL is acquired in pyo3. /// /// If the GIL is held, the reference count will be decreased immediately instead of being queued /// for later. /// /// # Safety /// The object must be an owned Python reference. pub unsafe fn register_decref(obj: NonNull) { if gil_is_acquired() { ffi::Py_DECREF(obj.as_ptr()) } else { POOL.register_decref(obj); } } /// Registers an owned object inside the GILPool, to be released when the GILPool drops. /// /// # Safety /// The object must be an owned Python reference. pub unsafe fn register_owned(_py: Python<'_>, obj: NonNull) { debug_assert!(gil_is_acquired()); // Ignores the error in case this function called from `atexit`. let _ = OWNED_OBJECTS.try_with(|owned_objects| { #[cfg(debug_assertions)] owned_objects.borrow_mut().push(obj); #[cfg(not(debug_assertions))] // SAFETY: This is not re-entrant. unsafe { (*owned_objects.get()).push(obj); } }); } /// Increments pyo3's internal GIL count - to be called whenever GILPool or GILGuard is created. #[inline(always)] fn increment_gil_count() { // Ignores the error in case this function called from `atexit`. let _ = GIL_COUNT.try_with(|c| { let current = c.get(); if current < 0 { LockGIL::bail(current); } c.set(current + 1); }); } /// Decrements pyo3's internal GIL count - to be called whenever GILPool or GILGuard is dropped. #[inline(always)] fn decrement_gil_count() { // Ignores the error in case this function called from `atexit`. let _ = GIL_COUNT.try_with(|c| { let current = c.get(); debug_assert!( current > 0, "Negative GIL count detected. Please report this error to the PyO3 repo as a bug." ); c.set(current - 1); }); } #[cfg(test)] mod tests { use super::{gil_is_acquired, GILPool, GIL_COUNT, OWNED_OBJECTS, POOL}; use crate::{ffi, gil, PyObject, Python, ToPyObject}; #[cfg(not(target_arch = "wasm32"))] use parking_lot::{const_mutex, Condvar, Mutex}; use std::ptr::NonNull; fn get_object(py: Python<'_>) -> PyObject { // Convenience function for getting a single unique object, using `new_pool` so as to leave // the original pool state unchanged. let pool = unsafe { py.new_pool() }; let py = pool.python(); let obj = py.eval("object()", None, None).unwrap(); obj.to_object(py) } fn owned_object_count() -> usize { #[cfg(debug_assertions)] let len = OWNED_OBJECTS.with(|owned_objects| owned_objects.borrow().len()); #[cfg(not(debug_assertions))] let len = OWNED_OBJECTS.with(|owned_objects| unsafe { (*owned_objects.get()).len() }); len } fn pool_inc_refs_does_not_contain(obj: &PyObject) -> bool { !POOL .pointer_ops .lock() .0 .contains(&unsafe { NonNull::new_unchecked(obj.as_ptr()) }) } fn pool_dec_refs_does_not_contain(obj: &PyObject) -> bool { !POOL .pointer_ops .lock() .1 .contains(&unsafe { NonNull::new_unchecked(obj.as_ptr()) }) } #[cfg(not(target_arch = "wasm32"))] fn pool_dirty_with( inc_refs: Vec>, dec_refs: Vec>, ) -> bool { *POOL.pointer_ops.lock() == (inc_refs, dec_refs) } #[test] fn test_owned() { Python::with_gil(|py| { let obj = get_object(py); let obj_ptr = obj.as_ptr(); // Ensure that obj does not get freed let _ref = obj.clone_ref(py); unsafe { { let pool = py.new_pool(); gil::register_owned(pool.python(), NonNull::new_unchecked(obj.into_ptr())); assert_eq!(owned_object_count(), 1); assert_eq!(ffi::Py_REFCNT(obj_ptr), 2); } { let _pool = py.new_pool(); assert_eq!(owned_object_count(), 0); assert_eq!(ffi::Py_REFCNT(obj_ptr), 1); } } }) } #[test] fn test_owned_nested() { Python::with_gil(|py| { let obj = get_object(py); // Ensure that obj does not get freed let _ref = obj.clone_ref(py); let obj_ptr = obj.as_ptr(); unsafe { { let _pool = py.new_pool(); assert_eq!(owned_object_count(), 0); gil::register_owned(py, NonNull::new_unchecked(obj.into_ptr())); assert_eq!(owned_object_count(), 1); assert_eq!(ffi::Py_REFCNT(obj_ptr), 2); { let _pool = py.new_pool(); let obj = get_object(py); gil::register_owned(py, NonNull::new_unchecked(obj.into_ptr())); assert_eq!(owned_object_count(), 2); } assert_eq!(owned_object_count(), 1); } { assert_eq!(owned_object_count(), 0); assert_eq!(ffi::Py_REFCNT(obj_ptr), 1); } } }); } #[test] fn test_pyobject_drop_with_gil_decreases_refcnt() { Python::with_gil(|py| { let obj = get_object(py); // Create a reference to drop with the GIL. let reference = obj.clone_ref(py); assert_eq!(obj.get_refcnt(py), 2); assert!(pool_inc_refs_does_not_contain(&obj)); // With the GIL held, reference cound will be decreased immediately. drop(reference); assert_eq!(obj.get_refcnt(py), 1); assert!(pool_dec_refs_does_not_contain(&obj)); }); } #[test] #[cfg(not(target_arch = "wasm32"))] // We are building wasm Python with pthreads disabled fn test_pyobject_drop_without_gil_doesnt_decrease_refcnt() { let obj = Python::with_gil(|py| { let obj = get_object(py); // Create a reference to drop without the GIL. let reference = obj.clone_ref(py); assert_eq!(obj.get_refcnt(py), 2); assert!(pool_inc_refs_does_not_contain(&obj)); // Drop reference in a separate thread which doesn't have the GIL. std::thread::spawn(move || drop(reference)).join().unwrap(); // The reference count should not have changed (the GIL has always // been held by this thread), it is remembered to release later. assert_eq!(obj.get_refcnt(py), 2); assert!(pool_dirty_with( vec![], vec![NonNull::new(obj.as_ptr()).unwrap()] )); obj }); // Next time the GIL is acquired, the reference is released Python::with_gil(|py| { assert_eq!(obj.get_refcnt(py), 1); let non_null = unsafe { NonNull::new_unchecked(obj.as_ptr()) }; assert!(!POOL.pointer_ops.lock().0.contains(&non_null)); assert!(!POOL.pointer_ops.lock().1.contains(&non_null)); }); } #[test] fn test_gil_counts() { // Check with_gil and GILPool both increase counts correctly let get_gil_count = || GIL_COUNT.with(|c| c.get()); assert_eq!(get_gil_count(), 0); Python::with_gil(|_| { assert_eq!(get_gil_count(), 1); let pool = unsafe { GILPool::new() }; assert_eq!(get_gil_count(), 2); let pool2 = unsafe { GILPool::new() }; assert_eq!(get_gil_count(), 3); drop(pool); assert_eq!(get_gil_count(), 2); Python::with_gil(|_| { // nested with_gil doesn't update gil count assert_eq!(get_gil_count(), 2); }); assert_eq!(get_gil_count(), 2); drop(pool2); assert_eq!(get_gil_count(), 1); }); assert_eq!(get_gil_count(), 0); } #[test] fn test_allow_threads() { assert!(!gil_is_acquired()); Python::with_gil(|py| { assert!(gil_is_acquired()); py.allow_threads(move || { assert!(!gil_is_acquired()); Python::with_gil(|_| assert!(gil_is_acquired())); assert!(!gil_is_acquired()); }); assert!(gil_is_acquired()); }); assert!(!gil_is_acquired()); } #[test] fn test_allow_threads_updates_refcounts() { Python::with_gil(|py| { // Make a simple object with 1 reference let obj = get_object(py); assert!(obj.get_refcnt(py) == 1); // Clone the object without the GIL to use internal tracking let escaped_ref = py.allow_threads(|| obj.clone()); // But after the block the refcounts are updated assert!(obj.get_refcnt(py) == 2); drop(escaped_ref); assert!(obj.get_refcnt(py) == 1); drop(obj); }); } #[test] fn dropping_gil_does_not_invalidate_references() { // Acquiring GIL for the second time should be safe - see #864 Python::with_gil(|py| { let obj = Python::with_gil(|_| py.eval("object()", None, None).unwrap()); // After gil2 drops, obj should still have a reference count of one assert_eq!(obj.get_refcnt(), 1); }) } #[test] fn test_clone_with_gil() { Python::with_gil(|py| { let obj = get_object(py); let count = obj.get_refcnt(py); // Cloning with the GIL should increase reference count immediately #[allow(clippy::redundant_clone)] let c = obj.clone(); assert_eq!(count + 1, c.get_refcnt(py)); }) } #[cfg(not(target_arch = "wasm32"))] struct Event { set: Mutex, wait: Condvar, } #[cfg(not(target_arch = "wasm32"))] impl Event { const fn new() -> Self { Self { set: const_mutex(false), wait: Condvar::new(), } } fn set(&self) { *self.set.lock() = true; self.wait.notify_all(); } fn wait(&self) { let mut set = self.set.lock(); while !*set { self.wait.wait(&mut set); } } } #[test] #[cfg(not(target_arch = "wasm32"))] // We are building wasm Python with pthreads disabled fn test_clone_without_gil() { use crate::{Py, PyAny}; use std::{sync::Arc, thread}; // Some events for synchronizing static GIL_ACQUIRED: Event = Event::new(); static OBJECT_CLONED: Event = Event::new(); static REFCNT_CHECKED: Event = Event::new(); Python::with_gil(|py| { let obj: Arc> = Arc::new(get_object(py)); let thread_obj = Arc::clone(&obj); let count = obj.get_refcnt(py); println!( "1: The object has been created and its reference count is {}", count ); let handle = thread::spawn(move || { Python::with_gil(move |py| { println!("3. The GIL has been acquired on another thread."); GIL_ACQUIRED.set(); // Wait while the main thread registers obj in POOL OBJECT_CLONED.wait(); println!("5. Checking refcnt"); assert_eq!(thread_obj.get_refcnt(py), count); REFCNT_CHECKED.set(); }) }); let cloned = py.allow_threads(|| { println!("2. The GIL has been released."); // Wait until the GIL has been acquired on the thread. GIL_ACQUIRED.wait(); println!("4. The other thread is now hogging the GIL, we clone without it held"); // Cloning without GIL should not update reference count let cloned = Py::clone(&*obj); OBJECT_CLONED.set(); cloned }); REFCNT_CHECKED.wait(); println!("6. The main thread has acquired the GIL again and processed the pool."); // Total reference count should be one higher assert_eq!(obj.get_refcnt(py), count + 1); // Clone dropped drop(cloned); // Ensure refcount of the arc is 1 handle.join().unwrap(); // Overall count is now back to the original, and should be no pending change assert_eq!(Arc::try_unwrap(obj).unwrap().get_refcnt(py), count); }); } #[test] #[cfg(not(target_arch = "wasm32"))] // We are building wasm Python with pthreads disabled fn test_clone_in_other_thread() { use crate::Py; use std::{sync::Arc, thread}; // Some events for synchronizing static OBJECT_CLONED: Event = Event::new(); let (obj, count, ptr) = Python::with_gil(|py| { let obj = Arc::new(get_object(py)); let count = obj.get_refcnt(py); let thread_obj = Arc::clone(&obj); // Start a thread which does not have the GIL, and clone it let t = thread::spawn(move || { // Cloning without GIL should not update reference count #[allow(clippy::redundant_clone)] let _ = Py::clone(&*thread_obj); OBJECT_CLONED.set(); }); OBJECT_CLONED.wait(); assert_eq!(count, obj.get_refcnt(py)); t.join().unwrap(); let ptr = NonNull::new(obj.as_ptr()).unwrap(); // The pointer should appear once in the incref pool, and once in the // decref pool (for the clone being created and also dropped) assert!(POOL.pointer_ops.lock().0.contains(&ptr)); assert!(POOL.pointer_ops.lock().1.contains(&ptr)); (obj, count, ptr) }); Python::with_gil(|py| { // Acquiring the gil clears the pool assert!(!POOL.pointer_ops.lock().0.contains(&ptr)); assert!(!POOL.pointer_ops.lock().1.contains(&ptr)); // Overall count is still unchanged assert_eq!(count, obj.get_refcnt(py)); }); } #[test] fn test_update_counts_does_not_deadlock() { // update_counts can run arbitrary Python code during Py_DECREF. // if the locking is implemented incorrectly, it will deadlock. Python::with_gil(|py| { let obj = get_object(py); unsafe extern "C" fn capsule_drop(capsule: *mut ffi::PyObject) { // This line will implicitly call update_counts // -> and so cause deadlock if update_counts is not handling recursion correctly. let pool = GILPool::new(); // Rebuild obj so that it can be dropped PyObject::from_owned_ptr( pool.python(), ffi::PyCapsule_GetPointer(capsule, std::ptr::null()) as _, ); } let ptr = obj.into_ptr(); let capsule = unsafe { ffi::PyCapsule_New(ptr as _, std::ptr::null(), Some(capsule_drop)) }; POOL.register_decref(NonNull::new(capsule).unwrap()); // Updating the counts will call decref on the capsule, which calls capsule_drop POOL.update_counts(py); }) } } pyo3-0.20.2/src/impl_/deprecations.rs000064400000000000000000000005371046102023000155210ustar 00000000000000//! Symbols used to denote deprecated usages of PyO3's proc macros. #[deprecated( since = "0.19.0", note = "put `text_signature` on `#[new]` instead of `#[pyclass]`" )] pub const PYCLASS_TEXT_SIGNATURE: () = (); #[deprecated(since = "0.20.0", note = "use `#[new]` instead of `#[__new__]`")] pub const PYMETHODS_NEW_DEPRECATED_FORM: () = (); pyo3-0.20.2/src/impl_/extract_argument.rs000064400000000000000000000701551046102023000164200ustar 00000000000000use crate::{ exceptions::PyTypeError, ffi, pyclass::boolean_struct::False, types::{PyDict, PyString, PyTuple}, FromPyObject, PyAny, PyClass, PyErr, PyRef, PyRefMut, PyResult, Python, }; /// A trait which is used to help PyO3 macros extract function arguments. /// /// `#[pyclass]` structs need to extract as `PyRef` and `PyRefMut` /// wrappers rather than extracting `&T` and `&mut T` directly. The `Holder` type is used /// to hold these temporary wrappers - the way the macro is constructed, these wrappers /// will be dropped as soon as the pyfunction call ends. /// /// There exists a trivial blanket implementation for `T: FromPyObject` with `Holder = ()`. pub trait PyFunctionArgument<'a, 'py>: Sized + 'a { type Holder: FunctionArgumentHolder; fn extract(obj: &'py PyAny, holder: &'a mut Self::Holder) -> PyResult; } impl<'a, 'py, T> PyFunctionArgument<'a, 'py> for T where T: FromPyObject<'py> + 'a, { type Holder = (); #[inline] fn extract(obj: &'py PyAny, _: &'a mut ()) -> PyResult { obj.extract() } } /// Trait for types which can be a function argument holder - they should /// to be able to const-initialize to an empty value. pub trait FunctionArgumentHolder: Sized { const INIT: Self; } impl FunctionArgumentHolder for () { const INIT: Self = (); } impl FunctionArgumentHolder for Option { const INIT: Self = None; } #[inline] pub fn extract_pyclass_ref<'a, 'py: 'a, T: PyClass>( obj: &'py PyAny, holder: &'a mut Option>, ) -> PyResult<&'a T> { Ok(&*holder.insert(obj.extract()?)) } #[inline] pub fn extract_pyclass_ref_mut<'a, 'py: 'a, T: PyClass>( obj: &'py PyAny, holder: &'a mut Option>, ) -> PyResult<&'a mut T> { Ok(&mut *holder.insert(obj.extract()?)) } /// The standard implementation of how PyO3 extracts a `#[pyfunction]` or `#[pymethod]` function argument. #[doc(hidden)] pub fn extract_argument<'a, 'py, T>( obj: &'py PyAny, holder: &'a mut T::Holder, arg_name: &str, ) -> PyResult where T: PyFunctionArgument<'a, 'py>, { match PyFunctionArgument::extract(obj, holder) { Ok(value) => Ok(value), Err(e) => Err(argument_extraction_error(obj.py(), arg_name, e)), } } /// Alternative to [`extract_argument`] used for `Option` arguments. This is necessary because Option<&T> /// does not implement `PyFunctionArgument` for `T: PyClass`. #[doc(hidden)] pub fn extract_optional_argument<'a, 'py, T>( obj: Option<&'py PyAny>, holder: &'a mut T::Holder, arg_name: &str, default: fn() -> Option, ) -> PyResult> where T: PyFunctionArgument<'a, 'py>, { match obj { Some(obj) => { if obj.is_none() { // Explicit `None` will result in None being used as the function argument Ok(None) } else { extract_argument(obj, holder, arg_name).map(Some) } } _ => Ok(default()), } } /// Alternative to [`extract_argument`] used when the argument has a default value provided by an annotation. #[doc(hidden)] pub fn extract_argument_with_default<'a, 'py, T>( obj: Option<&'py PyAny>, holder: &'a mut T::Holder, arg_name: &str, default: fn() -> T, ) -> PyResult where T: PyFunctionArgument<'a, 'py>, { match obj { Some(obj) => extract_argument(obj, holder, arg_name), None => Ok(default()), } } /// Alternative to [`extract_argument`] used when the argument has a `#[pyo3(from_py_with)]` annotation. #[doc(hidden)] pub fn from_py_with<'py, T>( obj: &'py PyAny, arg_name: &str, extractor: fn(&'py PyAny) -> PyResult, ) -> PyResult { match extractor(obj) { Ok(value) => Ok(value), Err(e) => Err(argument_extraction_error(obj.py(), arg_name, e)), } } /// Alternative to [`extract_argument`] used when the argument has a `#[pyo3(from_py_with)]` annotation and also a default value. #[doc(hidden)] pub fn from_py_with_with_default<'py, T>( obj: Option<&'py PyAny>, arg_name: &str, extractor: fn(&'py PyAny) -> PyResult, default: fn() -> T, ) -> PyResult { match obj { Some(obj) => from_py_with(obj, arg_name, extractor), None => Ok(default()), } } /// Adds the argument name to the error message of an error which occurred during argument extraction. /// /// Only modifies TypeError. (Cannot guarantee all exceptions have constructors from /// single string.) #[doc(hidden)] #[cold] pub fn argument_extraction_error(py: Python<'_>, arg_name: &str, error: PyErr) -> PyErr { if error.get_type(py).is(py.get_type::()) { let remapped_error = PyTypeError::new_err(format!("argument '{}': {}", arg_name, error.value(py))); remapped_error.set_cause(py, error.cause(py)); remapped_error } else { error } } /// Unwraps the Option<&PyAny> produced by the FunctionDescription `extract_arguments_` methods. /// They check if required methods are all provided. /// /// # Safety /// `argument` must not be `None` #[doc(hidden)] #[inline] pub unsafe fn unwrap_required_argument(argument: Option<&PyAny>) -> &PyAny { match argument { Some(value) => value, #[cfg(debug_assertions)] None => unreachable!("required method argument was not extracted"), #[cfg(not(debug_assertions))] None => std::hint::unreachable_unchecked(), } } pub struct KeywordOnlyParameterDescription { pub name: &'static str, pub required: bool, } /// Function argument specification for a `#[pyfunction]` or `#[pymethod]`. pub struct FunctionDescription { pub cls_name: Option<&'static str>, pub func_name: &'static str, pub positional_parameter_names: &'static [&'static str], pub positional_only_parameters: usize, pub required_positional_parameters: usize, pub keyword_only_parameters: &'static [KeywordOnlyParameterDescription], } impl FunctionDescription { fn full_name(&self) -> String { if let Some(cls_name) = self.cls_name { format!("{}.{}()", cls_name, self.func_name) } else { format!("{}()", self.func_name) } } /// Equivalent of `extract_arguments_tuple_dict` which uses the Python C-API "fastcall" convention. /// /// # Safety /// - `args` must be a pointer to a C-style array of valid `ffi::PyObject` pointers, or NULL. /// - `kwnames` must be a pointer to a PyTuple, or NULL. /// - `nargs + kwnames.len()` is the total length of the `args` array. #[cfg(not(Py_LIMITED_API))] pub unsafe fn extract_arguments_fastcall<'py, V, K>( &self, py: Python<'py>, args: *const *mut ffi::PyObject, nargs: ffi::Py_ssize_t, kwnames: *mut ffi::PyObject, output: &mut [Option<&'py PyAny>], ) -> PyResult<(V::Varargs, K::Varkeywords)> where V: VarargsHandler<'py>, K: VarkeywordsHandler<'py>, { let num_positional_parameters = self.positional_parameter_names.len(); debug_assert!(nargs >= 0); debug_assert!(self.positional_only_parameters <= num_positional_parameters); debug_assert!(self.required_positional_parameters <= num_positional_parameters); debug_assert_eq!( output.len(), num_positional_parameters + self.keyword_only_parameters.len() ); // Handle positional arguments // Safety: Option<&PyAny> has the same memory layout as `*mut ffi::PyObject` let args: *const Option<&PyAny> = args.cast(); let positional_args_provided = nargs as usize; let remaining_positional_args = if args.is_null() { debug_assert_eq!(positional_args_provided, 0); &[] } else { // Can consume at most the number of positional parameters in the function definition, // the rest are varargs. let positional_args_to_consume = num_positional_parameters.min(positional_args_provided); let (positional_parameters, remaining) = std::slice::from_raw_parts(args, positional_args_provided) .split_at(positional_args_to_consume); output[..positional_args_to_consume].copy_from_slice(positional_parameters); remaining }; let varargs = V::handle_varargs_fastcall(py, remaining_positional_args, self)?; // Handle keyword arguments let mut varkeywords = K::Varkeywords::default(); if let Some(kwnames) = py.from_borrowed_ptr_or_opt::(kwnames) { // Safety: &PyAny has the same memory layout as `*mut ffi::PyObject` let kwargs = ::std::slice::from_raw_parts((args as *const &PyAny).offset(nargs), kwnames.len()); self.handle_kwargs::( kwnames.iter().zip(kwargs.iter().copied()), &mut varkeywords, num_positional_parameters, output, )? } // Once all inputs have been processed, check that all required arguments have been provided. self.ensure_no_missing_required_positional_arguments(output, positional_args_provided)?; self.ensure_no_missing_required_keyword_arguments(output)?; Ok((varargs, varkeywords)) } /// Extracts the `args` and `kwargs` provided into `output`, according to this function /// definition. /// /// `output` must have the same length as this function has positional and keyword-only /// parameters (as per the `positional_parameter_names` and `keyword_only_parameters` /// respectively). /// /// Unexpected, duplicate or invalid arguments will cause this function to return `TypeError`. /// /// # Safety /// - `args` must be a pointer to a PyTuple. /// - `kwargs` must be a pointer to a PyDict, or NULL. pub unsafe fn extract_arguments_tuple_dict<'py, V, K>( &self, py: Python<'py>, args: *mut ffi::PyObject, kwargs: *mut ffi::PyObject, output: &mut [Option<&'py PyAny>], ) -> PyResult<(V::Varargs, K::Varkeywords)> where V: VarargsHandler<'py>, K: VarkeywordsHandler<'py>, { let args = py.from_borrowed_ptr::(args); let kwargs: ::std::option::Option<&PyDict> = py.from_borrowed_ptr_or_opt(kwargs); let num_positional_parameters = self.positional_parameter_names.len(); debug_assert!(self.positional_only_parameters <= num_positional_parameters); debug_assert!(self.required_positional_parameters <= num_positional_parameters); debug_assert_eq!( output.len(), num_positional_parameters + self.keyword_only_parameters.len() ); // Copy positional arguments into output for (i, arg) in args.iter().take(num_positional_parameters).enumerate() { output[i] = Some(arg); } // If any arguments remain, push them to varargs (if possible) or error let varargs = V::handle_varargs_tuple(args, self)?; // Handle keyword arguments let mut varkeywords = K::Varkeywords::default(); if let Some(kwargs) = kwargs { self.handle_kwargs::(kwargs, &mut varkeywords, num_positional_parameters, output)? } // Once all inputs have been processed, check that all required arguments have been provided. self.ensure_no_missing_required_positional_arguments(output, args.len())?; self.ensure_no_missing_required_keyword_arguments(output)?; Ok((varargs, varkeywords)) } #[inline] fn handle_kwargs<'py, K, I>( &self, kwargs: I, varkeywords: &mut K::Varkeywords, num_positional_parameters: usize, output: &mut [Option<&'py PyAny>], ) -> PyResult<()> where K: VarkeywordsHandler<'py>, I: IntoIterator, { debug_assert_eq!( num_positional_parameters, self.positional_parameter_names.len() ); debug_assert_eq!( output.len(), num_positional_parameters + self.keyword_only_parameters.len() ); let mut positional_only_keyword_arguments = Vec::new(); for (kwarg_name_py, value) in kwargs { // All keyword arguments should be UTF-8 strings, but we'll check, just in case. // If it isn't, then it will be handled below as a varkeyword (which may raise an // error if this function doesn't accept **kwargs). Rust source is always UTF-8 // and so all argument names in `#[pyfunction]` signature must be UTF-8. if let Ok(kwarg_name) = kwarg_name_py.downcast::()?.to_str() { // Try to place parameter in keyword only parameters if let Some(i) = self.find_keyword_parameter_in_keyword_only(kwarg_name) { if output[i + num_positional_parameters] .replace(value) .is_some() { return Err(self.multiple_values_for_argument(kwarg_name)); } continue; } // Repeat for positional parameters if let Some(i) = self.find_keyword_parameter_in_positional(kwarg_name) { if i < self.positional_only_parameters { // If accepting **kwargs, then it's allowed for the name of the // kwarg to conflict with a postional-only argument - the value // will go into **kwargs anyway. if K::handle_varkeyword(varkeywords, kwarg_name_py, value, self).is_err() { positional_only_keyword_arguments.push(kwarg_name); } } else if output[i].replace(value).is_some() { return Err(self.multiple_values_for_argument(kwarg_name)); } continue; } }; K::handle_varkeyword(varkeywords, kwarg_name_py, value, self)? } if !positional_only_keyword_arguments.is_empty() { return Err(self.positional_only_keyword_arguments(&positional_only_keyword_arguments)); } Ok(()) } #[inline] fn find_keyword_parameter_in_positional(&self, kwarg_name: &str) -> Option { self.positional_parameter_names .iter() .position(|¶m_name| param_name == kwarg_name) } #[inline] fn find_keyword_parameter_in_keyword_only(&self, kwarg_name: &str) -> Option { // Compare the keyword name against each parameter in turn. This is exactly the same method // which CPython uses to map keyword names. Although it's O(num_parameters), the number of // parameters is expected to be small so it's not worth constructing a mapping. self.keyword_only_parameters .iter() .position(|param_desc| param_desc.name == kwarg_name) } #[inline] fn ensure_no_missing_required_positional_arguments( &self, output: &[Option<&PyAny>], positional_args_provided: usize, ) -> PyResult<()> { if positional_args_provided < self.required_positional_parameters { for out in &output[positional_args_provided..self.required_positional_parameters] { if out.is_none() { return Err(self.missing_required_positional_arguments(output)); } } } Ok(()) } #[inline] fn ensure_no_missing_required_keyword_arguments( &self, output: &[Option<&PyAny>], ) -> PyResult<()> { let keyword_output = &output[self.positional_parameter_names.len()..]; for (param, out) in self.keyword_only_parameters.iter().zip(keyword_output) { if param.required && out.is_none() { return Err(self.missing_required_keyword_arguments(keyword_output)); } } Ok(()) } #[cold] fn too_many_positional_arguments(&self, args_provided: usize) -> PyErr { let was = if args_provided == 1 { "was" } else { "were" }; let msg = if self.required_positional_parameters != self.positional_parameter_names.len() { format!( "{} takes from {} to {} positional arguments but {} {} given", self.full_name(), self.required_positional_parameters, self.positional_parameter_names.len(), args_provided, was ) } else { format!( "{} takes {} positional arguments but {} {} given", self.full_name(), self.positional_parameter_names.len(), args_provided, was ) }; PyTypeError::new_err(msg) } #[cold] fn multiple_values_for_argument(&self, argument: &str) -> PyErr { PyTypeError::new_err(format!( "{} got multiple values for argument '{}'", self.full_name(), argument )) } #[cold] fn unexpected_keyword_argument(&self, argument: &PyAny) -> PyErr { PyTypeError::new_err(format!( "{} got an unexpected keyword argument '{}'", self.full_name(), argument )) } #[cold] fn positional_only_keyword_arguments(&self, parameter_names: &[&str]) -> PyErr { let mut msg = format!( "{} got some positional-only arguments passed as keyword arguments: ", self.full_name() ); push_parameter_list(&mut msg, parameter_names); PyTypeError::new_err(msg) } #[cold] fn missing_required_arguments(&self, argument_type: &str, parameter_names: &[&str]) -> PyErr { let arguments = if parameter_names.len() == 1 { "argument" } else { "arguments" }; let mut msg = format!( "{} missing {} required {} {}: ", self.full_name(), parameter_names.len(), argument_type, arguments, ); push_parameter_list(&mut msg, parameter_names); PyTypeError::new_err(msg) } #[cold] fn missing_required_keyword_arguments(&self, keyword_outputs: &[Option<&PyAny>]) -> PyErr { debug_assert_eq!(self.keyword_only_parameters.len(), keyword_outputs.len()); let missing_keyword_only_arguments: Vec<_> = self .keyword_only_parameters .iter() .zip(keyword_outputs) .filter_map(|(keyword_desc, out)| { if keyword_desc.required && out.is_none() { Some(keyword_desc.name) } else { None } }) .collect(); debug_assert!(!missing_keyword_only_arguments.is_empty()); self.missing_required_arguments("keyword", &missing_keyword_only_arguments) } #[cold] fn missing_required_positional_arguments(&self, output: &[Option<&PyAny>]) -> PyErr { let missing_positional_arguments: Vec<_> = self .positional_parameter_names .iter() .take(self.required_positional_parameters) .zip(output) .filter_map(|(param, out)| if out.is_none() { Some(*param) } else { None }) .collect(); debug_assert!(!missing_positional_arguments.is_empty()); self.missing_required_arguments("positional", &missing_positional_arguments) } } /// A trait used to control whether to accept varargs in FunctionDescription::extract_argument_(method) functions. pub trait VarargsHandler<'py> { type Varargs; /// Called by `FunctionDescription::extract_arguments_fastcall` with any additional arguments. fn handle_varargs_fastcall( py: Python<'py>, varargs: &[Option<&PyAny>], function_description: &FunctionDescription, ) -> PyResult; /// Called by `FunctionDescription::extract_arguments_tuple_dict` with the original tuple. /// /// Additional arguments are those in the tuple slice starting from `function_description.positional_parameter_names.len()`. fn handle_varargs_tuple( args: &'py PyTuple, function_description: &FunctionDescription, ) -> PyResult; } /// Marker struct which indicates varargs are not allowed. pub struct NoVarargs; impl<'py> VarargsHandler<'py> for NoVarargs { type Varargs = (); #[inline] fn handle_varargs_fastcall( _py: Python<'py>, varargs: &[Option<&PyAny>], function_description: &FunctionDescription, ) -> PyResult { let extra_arguments = varargs.len(); if extra_arguments > 0 { return Err(function_description.too_many_positional_arguments( function_description.positional_parameter_names.len() + extra_arguments, )); } Ok(()) } #[inline] fn handle_varargs_tuple( args: &'py PyTuple, function_description: &FunctionDescription, ) -> PyResult { let positional_parameter_count = function_description.positional_parameter_names.len(); let provided_args_count = args.len(); if provided_args_count <= positional_parameter_count { Ok(()) } else { Err(function_description.too_many_positional_arguments(provided_args_count)) } } } /// Marker struct which indicates varargs should be collected into a `PyTuple`. pub struct TupleVarargs; impl<'py> VarargsHandler<'py> for TupleVarargs { type Varargs = &'py PyTuple; #[inline] fn handle_varargs_fastcall( py: Python<'py>, varargs: &[Option<&PyAny>], _function_description: &FunctionDescription, ) -> PyResult { Ok(PyTuple::new(py, varargs)) } #[inline] fn handle_varargs_tuple( args: &'py PyTuple, function_description: &FunctionDescription, ) -> PyResult { let positional_parameters = function_description.positional_parameter_names.len(); Ok(args.get_slice(positional_parameters, args.len())) } } /// A trait used to control whether to accept varkeywords in FunctionDescription::extract_argument_(method) functions. pub trait VarkeywordsHandler<'py> { type Varkeywords: Default; fn handle_varkeyword( varkeywords: &mut Self::Varkeywords, name: &'py PyAny, value: &'py PyAny, function_description: &FunctionDescription, ) -> PyResult<()>; } /// Marker struct which indicates unknown keywords are not permitted. pub struct NoVarkeywords; impl<'py> VarkeywordsHandler<'py> for NoVarkeywords { type Varkeywords = (); #[inline] fn handle_varkeyword( _varkeywords: &mut Self::Varkeywords, name: &'py PyAny, _value: &'py PyAny, function_description: &FunctionDescription, ) -> PyResult<()> { Err(function_description.unexpected_keyword_argument(name)) } } /// Marker struct which indicates unknown keywords should be collected into a `PyDict`. pub struct DictVarkeywords; impl<'py> VarkeywordsHandler<'py> for DictVarkeywords { type Varkeywords = Option<&'py PyDict>; #[inline] fn handle_varkeyword( varkeywords: &mut Self::Varkeywords, name: &'py PyAny, value: &'py PyAny, _function_description: &FunctionDescription, ) -> PyResult<()> { varkeywords .get_or_insert_with(|| PyDict::new(name.py())) .set_item(name, value) } } fn push_parameter_list(msg: &mut String, parameter_names: &[&str]) { for (i, parameter) in parameter_names.iter().enumerate() { if i != 0 { if parameter_names.len() > 2 { msg.push(','); } if i == parameter_names.len() - 1 { msg.push_str(" and ") } else { msg.push(' ') } } msg.push('\''); msg.push_str(parameter); msg.push('\''); } } #[cfg(test)] mod tests { use crate::{ types::{IntoPyDict, PyTuple}, PyAny, Python, ToPyObject, }; use super::{push_parameter_list, FunctionDescription, NoVarargs, NoVarkeywords}; #[test] fn unexpected_keyword_argument() { let function_description = FunctionDescription { cls_name: None, func_name: "example", positional_parameter_names: &[], positional_only_parameters: 0, required_positional_parameters: 0, keyword_only_parameters: &[], }; Python::with_gil(|py| { let args = PyTuple::new(py, Vec::<&PyAny>::new()); let kwargs = [("foo".to_object(py).into_ref(py), 0u8)].into_py_dict(py); let err = unsafe { function_description .extract_arguments_tuple_dict::( py, args.as_ptr(), kwargs.as_ptr(), &mut [], ) .unwrap_err() }; assert_eq!( err.to_string(), "TypeError: example() got an unexpected keyword argument 'foo'" ); }) } #[test] fn keyword_not_string() { let function_description = FunctionDescription { cls_name: None, func_name: "example", positional_parameter_names: &[], positional_only_parameters: 0, required_positional_parameters: 0, keyword_only_parameters: &[], }; Python::with_gil(|py| { let args = PyTuple::new(py, Vec::<&PyAny>::new()); let kwargs = [(1u8.to_object(py).into_ref(py), 1u8)].into_py_dict(py); let err = unsafe { function_description .extract_arguments_tuple_dict::( py, args.as_ptr(), kwargs.as_ptr(), &mut [], ) .unwrap_err() }; assert_eq!( err.to_string(), "TypeError: 'int' object cannot be converted to 'PyString'" ); }) } #[test] fn missing_required_arguments() { let function_description = FunctionDescription { cls_name: None, func_name: "example", positional_parameter_names: &["foo", "bar"], positional_only_parameters: 0, required_positional_parameters: 2, keyword_only_parameters: &[], }; Python::with_gil(|py| { let args = PyTuple::new(py, Vec::<&PyAny>::new()); let mut output = [None, None]; let err = unsafe { function_description.extract_arguments_tuple_dict::( py, args.as_ptr(), std::ptr::null_mut(), &mut output, ) } .unwrap_err(); assert_eq!( err.to_string(), "TypeError: example() missing 2 required positional arguments: 'foo' and 'bar'" ); }) } #[test] fn push_parameter_list_empty() { let mut s = String::new(); push_parameter_list(&mut s, &[]); assert_eq!(&s, ""); } #[test] fn push_parameter_list_one() { let mut s = String::new(); push_parameter_list(&mut s, &["a"]); assert_eq!(&s, "'a'"); } #[test] fn push_parameter_list_two() { let mut s = String::new(); push_parameter_list(&mut s, &["a", "b"]); assert_eq!(&s, "'a' and 'b'"); } #[test] fn push_parameter_list_three() { let mut s = String::new(); push_parameter_list(&mut s, &["a", "b", "c"]); assert_eq!(&s, "'a', 'b', and 'c'"); } #[test] fn push_parameter_list_four() { let mut s = String::new(); push_parameter_list(&mut s, &["a", "b", "c", "d"]); assert_eq!(&s, "'a', 'b', 'c', and 'd'"); } } pyo3-0.20.2/src/impl_/freelist.rs000064400000000000000000000035031046102023000146520ustar 00000000000000//! Support for [free allocation lists][1]. //! //! This can improve performance for types that are often created and deleted in quick succession. //! //! Rather than implementing this manually, //! implement it by annotating a struct with `#[pyclass(freelist = N)]`, //! where `N` is the size of the freelist. //! //! [1]: https://en.wikipedia.org/wiki/Free_list use std::mem; /// Represents a slot of a [`FreeList`]. pub enum Slot { /// A free slot. Empty, /// An allocated slot. Filled(T), } /// A free allocation list. /// /// See [the parent module](crate::impl_::freelist) for more details. pub struct FreeList { entries: Vec>, split: usize, capacity: usize, } impl FreeList { /// Creates a new `FreeList` instance with specified capacity. pub fn with_capacity(capacity: usize) -> FreeList { let entries = (0..capacity).map(|_| Slot::Empty).collect::>(); FreeList { entries, split: 0, capacity, } } /// Pops the first non empty item. pub fn pop(&mut self) -> Option { let idx = self.split; if idx == 0 { None } else { match mem::replace(&mut self.entries[idx - 1], Slot::Empty) { Slot::Filled(v) => { self.split = idx - 1; Some(v) } _ => panic!("FreeList is corrupt"), } } } /// Inserts a value into the list. Returns `Some(val)` if the `FreeList` is full. pub fn insert(&mut self, val: T) -> Option { let next = self.split + 1; if next < self.capacity { self.entries[self.split] = Slot::Filled(val); self.split = next; None } else { Some(val) } } } pyo3-0.20.2/src/impl_/frompyobject.rs000064400000000000000000000066641046102023000155530ustar 00000000000000use crate::{exceptions::PyTypeError, FromPyObject, PyAny, PyErr, PyResult, Python}; #[cold] pub fn failed_to_extract_enum( py: Python<'_>, type_name: &str, variant_names: &[&str], error_names: &[&str], errors: &[PyErr], ) -> PyErr { // TODO maybe use ExceptionGroup on Python 3.11+ ? let mut err_msg = format!( "failed to extract enum {} ('{}')", type_name, error_names.join(" | ") ); for ((variant_name, error_name), error) in variant_names.iter().zip(error_names).zip(errors) { use std::fmt::Write; write!( &mut err_msg, "\n- variant {variant_name} ({error_name}): {error_msg}", variant_name = variant_name, error_name = error_name, error_msg = extract_traceback(py, error.clone_ref(py)), ) .unwrap(); } PyTypeError::new_err(err_msg) } /// Flattens a chain of errors into a single string. fn extract_traceback(py: Python<'_>, mut error: PyErr) -> String { use std::fmt::Write; let mut error_msg = error.to_string(); while let Some(cause) = error.cause(py) { write!(&mut error_msg, ", caused by {}", cause).unwrap(); error = cause } error_msg } pub fn extract_struct_field<'py, T>( obj: &'py PyAny, struct_name: &str, field_name: &str, ) -> PyResult where T: FromPyObject<'py>, { match obj.extract() { Ok(value) => Ok(value), Err(err) => Err(failed_to_extract_struct_field( obj.py(), err, struct_name, field_name, )), } } pub fn extract_struct_field_with<'py, T>( extractor: impl FnOnce(&'py PyAny) -> PyResult, obj: &'py PyAny, struct_name: &str, field_name: &str, ) -> PyResult { match extractor(obj) { Ok(value) => Ok(value), Err(err) => Err(failed_to_extract_struct_field( obj.py(), err, struct_name, field_name, )), } } #[cold] fn failed_to_extract_struct_field( py: Python<'_>, inner_err: PyErr, struct_name: &str, field_name: &str, ) -> PyErr { let new_err = PyTypeError::new_err(format!( "failed to extract field {}.{}", struct_name, field_name )); new_err.set_cause(py, ::std::option::Option::Some(inner_err)); new_err } pub fn extract_tuple_struct_field<'py, T>( obj: &'py PyAny, struct_name: &str, index: usize, ) -> PyResult where T: FromPyObject<'py>, { match obj.extract() { Ok(value) => Ok(value), Err(err) => Err(failed_to_extract_tuple_struct_field( obj.py(), err, struct_name, index, )), } } pub fn extract_tuple_struct_field_with<'py, T>( extractor: impl FnOnce(&'py PyAny) -> PyResult, obj: &'py PyAny, struct_name: &str, index: usize, ) -> PyResult { match extractor(obj) { Ok(value) => Ok(value), Err(err) => Err(failed_to_extract_tuple_struct_field( obj.py(), err, struct_name, index, )), } } #[cold] fn failed_to_extract_tuple_struct_field( py: Python<'_>, inner_err: PyErr, struct_name: &str, index: usize, ) -> PyErr { let new_err = PyTypeError::new_err(format!("failed to extract field {}.{}", struct_name, index)); new_err.set_cause(py, ::std::option::Option::Some(inner_err)); new_err } pyo3-0.20.2/src/impl_/not_send.rs000064400000000000000000000004711046102023000146470ustar 00000000000000use std::marker::PhantomData; use crate::Python; /// A marker type that makes the type !Send. /// Workaround for lack of !Send on stable (). pub(crate) struct NotSend(PhantomData<*mut Python<'static>>); pub(crate) const NOT_SEND: NotSend = NotSend(PhantomData); pyo3-0.20.2/src/impl_/panic.rs000064400000000000000000000011251046102023000141250ustar 00000000000000/// Type which will panic if dropped. /// /// If this is dropped during a panic, this will cause an abort. /// /// Use this to avoid letting unwinds cross through the FFI boundary, which is UB. pub struct PanicTrap { msg: &'static str, } impl PanicTrap { #[inline] pub const fn new(msg: &'static str) -> Self { Self { msg } } #[inline] pub const fn disarm(self) { std::mem::forget(self) } } impl Drop for PanicTrap { fn drop(&mut self) { // Panic here will abort the process, assuming in an unwind. panic!("{}", self.msg) } } pyo3-0.20.2/src/impl_/pycell.rs000064400000000000000000000001701046102023000143220ustar 00000000000000//! Externally-accessible implementation of pycell pub use crate::pycell::impl_::{GetBorrowChecker, PyClassMutability}; pyo3-0.20.2/src/impl_/pyclass/lazy_type_object.rs000064400000000000000000000166421046102023000200710ustar 00000000000000use std::{ borrow::Cow, cell::RefCell, ffi::CStr, marker::PhantomData, thread::{self, ThreadId}, }; use crate::{ exceptions::PyRuntimeError, ffi, pyclass::{create_type_object, PyClassTypeObject}, sync::{GILOnceCell, GILProtected}, types::PyType, PyClass, PyErr, PyMethodDefType, PyObject, PyResult, Python, }; use super::PyClassItemsIter; /// Lazy type object for PyClass. #[doc(hidden)] pub struct LazyTypeObject(LazyTypeObjectInner, PhantomData); // Non-generic inner of LazyTypeObject to keep code size down struct LazyTypeObjectInner { value: GILOnceCell, // Threads which have begun initialization of the `tp_dict`. Used for // reentrant initialization detection. initializing_threads: GILProtected>>, tp_dict_filled: GILOnceCell<()>, } impl LazyTypeObject { /// Creates an uninitialized `LazyTypeObject`. pub const fn new() -> Self { LazyTypeObject( LazyTypeObjectInner { value: GILOnceCell::new(), initializing_threads: GILProtected::new(RefCell::new(Vec::new())), tp_dict_filled: GILOnceCell::new(), }, PhantomData, ) } } impl LazyTypeObject { /// Gets the type object contained by this `LazyTypeObject`, initializing it if needed. pub fn get_or_init<'py>(&'py self, py: Python<'py>) -> &'py PyType { self.get_or_try_init(py).unwrap_or_else(|err| { err.print(py); panic!("failed to create type object for {}", T::NAME) }) } /// Fallible version of the above. pub(crate) fn get_or_try_init<'py>(&'py self, py: Python<'py>) -> PyResult<&'py PyType> { self.0 .get_or_try_init(py, create_type_object::, T::NAME, T::items_iter()) } } impl LazyTypeObjectInner { // Uses dynamically dispatched fn(Python<'py>) -> PyResult // so that this code is only instantiated once, instead of for every T // like the generic LazyTypeObject methods above. fn get_or_try_init<'py>( &'py self, py: Python<'py>, init: fn(Python<'py>) -> PyResult, name: &str, items_iter: PyClassItemsIter, ) -> PyResult<&'py PyType> { (|| -> PyResult<_> { let type_object = self .value .get_or_try_init(py, || init(py))? .type_object .as_ref(py); self.ensure_init(type_object, name, items_iter)?; Ok(type_object) })() .map_err(|err| { wrap_in_runtime_error( py, err, format!("An error occurred while initializing class {}", name), ) }) } fn ensure_init( &self, type_object: &PyType, name: &str, items_iter: PyClassItemsIter, ) -> PyResult<()> { let py = type_object.py(); // We might want to fill the `tp_dict` with python instances of `T` // itself. In order to do so, we must first initialize the type object // with an empty `tp_dict`: now we can create instances of `T`. // // Then we fill the `tp_dict`. Multiple threads may try to fill it at // the same time, but only one of them will succeed. // // More importantly, if a thread is performing initialization of the // `tp_dict`, it can still request the type object through `get_or_init`, // but the `tp_dict` may appear empty of course. if self.tp_dict_filled.get(py).is_some() { // `tp_dict` is already filled: ok. return Ok(()); } let thread_id = thread::current().id(); { let mut threads = self.initializing_threads.get(py).borrow_mut(); if threads.contains(&thread_id) { // Reentrant call: just return the type object, even if the // `tp_dict` is not filled yet. return Ok(()); } threads.push(thread_id); } struct InitializationGuard<'a> { initializing_threads: &'a GILProtected>>, py: Python<'a>, thread_id: ThreadId, } impl Drop for InitializationGuard<'_> { fn drop(&mut self) { let mut threads = self.initializing_threads.get(self.py).borrow_mut(); threads.retain(|id| *id != self.thread_id); } } let guard = InitializationGuard { initializing_threads: &self.initializing_threads, py, thread_id, }; // Pre-compute the class attribute objects: this can temporarily // release the GIL since we're calling into arbitrary user code. It // means that another thread can continue the initialization in the // meantime: at worst, we'll just make a useless computation. let mut items = vec![]; for class_items in items_iter { for def in class_items.methods { if let PyMethodDefType::ClassAttribute(attr) = def { let key = attr.attribute_c_string().unwrap(); match (attr.meth.0)(py) { Ok(val) => items.push((key, val)), Err(err) => { return Err(wrap_in_runtime_error( py, err, format!( "An error occurred while initializing `{}.{}`", name, attr.name.trim_end_matches('\0') ), )) } } } } } // Now we hold the GIL and we can assume it won't be released until we // return from the function. let result = self.tp_dict_filled.get_or_try_init(py, move || { let result = initialize_tp_dict(py, type_object.as_ptr(), items); // Initialization successfully complete, can clear the thread list. // (No further calls to get_or_init() will try to init, on any thread.) std::mem::forget(guard); self.initializing_threads.get(py).replace(Vec::new()); result }); if let Err(err) = result { return Err(wrap_in_runtime_error( py, err.clone_ref(py), format!("An error occurred while initializing `{}.__dict__`", name), )); } Ok(()) } } fn initialize_tp_dict( py: Python<'_>, type_object: *mut ffi::PyObject, items: Vec<(Cow<'static, CStr>, PyObject)>, ) -> PyResult<()> { // We hold the GIL: the dictionary update can be considered atomic from // the POV of other threads. for (key, val) in items { crate::err::error_on_minusone(py, unsafe { ffi::PyObject_SetAttrString(type_object, key.as_ptr(), val.into_ptr()) })?; } Ok(()) } // This is necessary for making static `LazyTypeObject`s unsafe impl Sync for LazyTypeObject {} #[cold] fn wrap_in_runtime_error(py: Python<'_>, err: PyErr, message: String) -> PyErr { let runtime_err = PyRuntimeError::new_err(message); runtime_err.set_cause(py, Some(err)); runtime_err } pyo3-0.20.2/src/impl_/pyclass.rs000064400000000000000000001024761046102023000145240ustar 00000000000000use crate::{ exceptions::{PyAttributeError, PyNotImplementedError, PyRuntimeError, PyValueError}, ffi, impl_::freelist::FreeList, impl_::pycell::{GetBorrowChecker, PyClassMutability}, internal_tricks::extract_c_string, pycell::PyCellLayout, pyclass_init::PyObjectInit, types::PyBool, Py, PyAny, PyCell, PyClass, PyErr, PyMethodDefType, PyNativeType, PyResult, PyTypeInfo, Python, }; use std::{ borrow::Cow, ffi::{CStr, CString}, marker::PhantomData, os::raw::{c_int, c_void}, ptr::NonNull, thread, }; mod lazy_type_object; pub use lazy_type_object::LazyTypeObject; /// Gets the offset of the dictionary from the start of the object in bytes. #[inline] pub fn dict_offset() -> ffi::Py_ssize_t { PyCell::::dict_offset() } /// Gets the offset of the weakref list from the start of the object in bytes. #[inline] pub fn weaklist_offset() -> ffi::Py_ssize_t { PyCell::::weaklist_offset() } /// Represents the `__dict__` field for `#[pyclass]`. pub trait PyClassDict { /// Initial form of a [PyObject](crate::ffi::PyObject) `__dict__` reference. const INIT: Self; /// Empties the dictionary of its key-value pairs. #[inline] fn clear_dict(&mut self, _py: Python<'_>) {} private_decl! {} } /// Represents the `__weakref__` field for `#[pyclass]`. pub trait PyClassWeakRef { /// Initializes a `weakref` instance. const INIT: Self; /// Clears the weak references to the given object. /// /// # Safety /// - `_obj` must be a pointer to the pyclass instance which contains `self`. /// - The GIL must be held. #[inline] unsafe fn clear_weakrefs(&mut self, _obj: *mut ffi::PyObject, _py: Python<'_>) {} private_decl! {} } /// Zero-sized dummy field. pub struct PyClassDummySlot; impl PyClassDict for PyClassDummySlot { private_impl! {} const INIT: Self = PyClassDummySlot; } impl PyClassWeakRef for PyClassDummySlot { private_impl! {} const INIT: Self = PyClassDummySlot; } /// Actual dict field, which holds the pointer to `__dict__`. /// /// `#[pyclass(dict)]` automatically adds this. #[repr(transparent)] pub struct PyClassDictSlot(*mut ffi::PyObject); impl PyClassDict for PyClassDictSlot { private_impl! {} const INIT: Self = Self(std::ptr::null_mut()); #[inline] fn clear_dict(&mut self, _py: Python<'_>) { if !self.0.is_null() { unsafe { ffi::PyDict_Clear(self.0) } } } } /// Actual weakref field, which holds the pointer to `__weakref__`. /// /// `#[pyclass(weakref)]` automatically adds this. #[repr(transparent)] pub struct PyClassWeakRefSlot(*mut ffi::PyObject); impl PyClassWeakRef for PyClassWeakRefSlot { private_impl! {} const INIT: Self = Self(std::ptr::null_mut()); #[inline] unsafe fn clear_weakrefs(&mut self, obj: *mut ffi::PyObject, _py: Python<'_>) { if !self.0.is_null() { ffi::PyObject_ClearWeakRefs(obj) } } } /// This type is used as a "dummy" type on which dtolnay specializations are /// applied to apply implementations from `#[pymethods]` pub struct PyClassImplCollector(PhantomData); impl PyClassImplCollector { pub fn new() -> Self { Self(PhantomData) } } impl Default for PyClassImplCollector { fn default() -> Self { Self::new() } } impl Clone for PyClassImplCollector { fn clone(&self) -> Self { *self } } impl Copy for PyClassImplCollector {} pub struct PyClassItems { pub methods: &'static [PyMethodDefType], pub slots: &'static [ffi::PyType_Slot], } // Allow PyClassItems in statics unsafe impl Sync for PyClassItems {} /// Implements the underlying functionality of `#[pyclass]`, assembled by various proc macros. /// /// Users are discouraged from implementing this trait manually; it is a PyO3 implementation detail /// and may be changed at any time. pub trait PyClassImpl: Sized + 'static { /// #[pyclass(subclass)] const IS_BASETYPE: bool = false; /// #[pyclass(extends=...)] const IS_SUBCLASS: bool = false; /// #[pyclass(mapping)] const IS_MAPPING: bool = false; /// #[pyclass(sequence)] const IS_SEQUENCE: bool = false; /// Base class type BaseType: PyTypeInfo + PyClassBaseType; /// Immutable or mutable type PyClassMutability: PyClassMutability + GetBorrowChecker; /// Specify this class has `#[pyclass(dict)]` or not. type Dict: PyClassDict; /// Specify this class has `#[pyclass(weakref)]` or not. type WeakRef: PyClassWeakRef; /// The closest native ancestor. This is `PyAny` by default, and when you declare /// `#[pyclass(extends=PyDict)]`, it's `PyDict`. type BaseNativeType: PyTypeInfo + PyNativeType; /// This handles following two situations: /// 1. In case `T` is `Send`, stub `ThreadChecker` is used and does nothing. /// This implementation is used by default. Compile fails if `T: !Send`. /// 2. In case `T` is `!Send`, `ThreadChecker` panics when `T` is accessed by another thread. /// This implementation is used when `#[pyclass(unsendable)]` is given. /// Panicking makes it safe to expose `T: !Send` to the Python interpreter, where all objects /// can be accessed by multiple threads by `threading` module. type ThreadChecker: PyClassThreadChecker; #[cfg(feature = "multiple-pymethods")] type Inventory: PyClassInventory; /// Rendered class doc fn doc(py: Python<'_>) -> PyResult<&'static CStr>; fn items_iter() -> PyClassItemsIter; #[inline] fn dict_offset() -> Option { None } #[inline] fn weaklist_offset() -> Option { None } fn lazy_type_object() -> &'static LazyTypeObject; } /// Runtime helper to build a class docstring from the `doc` and `text_signature`. /// /// This is done at runtime because the class text signature is collected via dtolnay /// specialization in to the `#[pyclass]` macro from the `#[pymethods]` macro. pub fn build_pyclass_doc( class_name: &'static str, doc: &'static str, text_signature: Option<&'static str>, ) -> PyResult> { if let Some(text_signature) = text_signature { let doc = CString::new(format!( "{}{}\n--\n\n{}", class_name, text_signature, doc.trim_end_matches('\0') )) .map_err(|_| PyValueError::new_err("class doc cannot contain nul bytes"))?; Ok(Cow::Owned(doc)) } else { extract_c_string(doc, "class doc cannot contain nul bytes") } } /// Iterator used to process all class items during type instantiation. pub struct PyClassItemsIter { /// Iteration state idx: usize, /// Items from the `#[pyclass]` macro pyclass_items: &'static PyClassItems, /// Items from the `#[pymethods]` macro #[cfg(not(feature = "multiple-pymethods"))] pymethods_items: &'static PyClassItems, /// Items from the `#[pymethods]` macro with inventory #[cfg(feature = "multiple-pymethods")] pymethods_items: Box>, } impl PyClassItemsIter { pub fn new( pyclass_items: &'static PyClassItems, #[cfg(not(feature = "multiple-pymethods"))] pymethods_items: &'static PyClassItems, #[cfg(feature = "multiple-pymethods")] pymethods_items: Box< dyn Iterator, >, ) -> Self { Self { idx: 0, pyclass_items, pymethods_items, } } } impl Iterator for PyClassItemsIter { type Item = &'static PyClassItems; #[cfg(not(feature = "multiple-pymethods"))] fn next(&mut self) -> Option { match self.idx { 0 => { self.idx += 1; Some(self.pyclass_items) } 1 => { self.idx += 1; Some(self.pymethods_items) } // Termination clause _ => None, } } #[cfg(feature = "multiple-pymethods")] fn next(&mut self) -> Option { match self.idx { 0 => { self.idx += 1; Some(self.pyclass_items) } // Termination clause _ => self.pymethods_items.next(), } } } // Traits describing known special methods. macro_rules! slot_fragment_trait { ($trait_name:ident, $($default_method:tt)*) => { #[allow(non_camel_case_types)] pub trait $trait_name: Sized { $($default_method)* } impl $trait_name for &'_ PyClassImplCollector {} } } slot_fragment_trait! { PyClass__getattribute__SlotFragment, /// # Safety: _slf and _attr must be valid non-null Python objects #[inline] unsafe fn __getattribute__( self, py: Python<'_>, slf: *mut ffi::PyObject, attr: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { let res = ffi::PyObject_GenericGetAttr(slf, attr); if res.is_null() { Err(PyErr::fetch(py)) } else { Ok(res) } } } slot_fragment_trait! { PyClass__getattr__SlotFragment, /// # Safety: _slf and _attr must be valid non-null Python objects #[inline] unsafe fn __getattr__( self, py: Python<'_>, _slf: *mut ffi::PyObject, attr: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { Err(PyErr::new::( (Py::::from_borrowed_ptr(py, attr),) )) } } #[doc(hidden)] #[macro_export] macro_rules! generate_pyclass_getattro_slot { ($cls:ty) => {{ unsafe extern "C" fn __wrap( _slf: *mut $crate::ffi::PyObject, attr: *mut $crate::ffi::PyObject, ) -> *mut $crate::ffi::PyObject { $crate::impl_::trampoline::getattrofunc(_slf, attr, |py, _slf, attr| { use ::std::result::Result::*; use $crate::impl_::pyclass::*; let collector = PyClassImplCollector::<$cls>::new(); // Strategy: // - Try __getattribute__ first. Its default is PyObject_GenericGetAttr. // - If it returns a result, use it. // - If it fails with AttributeError, try __getattr__. // - If it fails otherwise, reraise. match collector.__getattribute__(py, _slf, attr) { Ok(obj) => Ok(obj), Err(e) if e.is_instance_of::<$crate::exceptions::PyAttributeError>(py) => { collector.__getattr__(py, _slf, attr) } Err(e) => Err(e), } }) } $crate::ffi::PyType_Slot { slot: $crate::ffi::Py_tp_getattro, pfunc: __wrap as $crate::ffi::getattrofunc as _, } }}; } pub use generate_pyclass_getattro_slot; /// Macro which expands to three items /// - Trait for a __setitem__ dunder /// - Trait for the corresponding __delitem__ dunder /// - A macro which will use dtolnay specialisation to generate the shared slot for the two dunders macro_rules! define_pyclass_setattr_slot { ( $set_trait:ident, $del_trait:ident, $set:ident, $del:ident, $set_error:expr, $del_error:expr, $generate_macro:ident, $slot:ident, $func_ty:ident, ) => { slot_fragment_trait! { $set_trait, /// # Safety: _slf and _attr must be valid non-null Python objects #[inline] unsafe fn $set( self, _py: Python<'_>, _slf: *mut ffi::PyObject, _attr: *mut ffi::PyObject, _value: NonNull, ) -> PyResult<()> { $set_error } } slot_fragment_trait! { $del_trait, /// # Safety: _slf and _attr must be valid non-null Python objects #[inline] unsafe fn $del( self, _py: Python<'_>, _slf: *mut ffi::PyObject, _attr: *mut ffi::PyObject, ) -> PyResult<()> { $del_error } } #[doc(hidden)] #[macro_export] macro_rules! $generate_macro { ($cls:ty) => {{ unsafe extern "C" fn __wrap( _slf: *mut $crate::ffi::PyObject, attr: *mut $crate::ffi::PyObject, value: *mut $crate::ffi::PyObject, ) -> ::std::os::raw::c_int { $crate::impl_::trampoline::setattrofunc( _slf, attr, value, |py, _slf, attr, value| { use ::std::option::Option::*; use $crate::callback::IntoPyCallbackOutput; use $crate::impl_::pyclass::*; let collector = PyClassImplCollector::<$cls>::new(); if let Some(value) = ::std::ptr::NonNull::new(value) { collector.$set(py, _slf, attr, value).convert(py) } else { collector.$del(py, _slf, attr).convert(py) } }, ) } $crate::ffi::PyType_Slot { slot: $crate::ffi::$slot, pfunc: __wrap as $crate::ffi::$func_ty as _, } }}; } pub use $generate_macro; }; } define_pyclass_setattr_slot! { PyClass__setattr__SlotFragment, PyClass__delattr__SlotFragment, __setattr__, __delattr__, Err(PyAttributeError::new_err("can't set attribute")), Err(PyAttributeError::new_err("can't delete attribute")), generate_pyclass_setattr_slot, Py_tp_setattro, setattrofunc, } define_pyclass_setattr_slot! { PyClass__set__SlotFragment, PyClass__delete__SlotFragment, __set__, __delete__, Err(PyNotImplementedError::new_err("can't set descriptor")), Err(PyNotImplementedError::new_err("can't delete descriptor")), generate_pyclass_setdescr_slot, Py_tp_descr_set, descrsetfunc, } define_pyclass_setattr_slot! { PyClass__setitem__SlotFragment, PyClass__delitem__SlotFragment, __setitem__, __delitem__, Err(PyNotImplementedError::new_err("can't set item")), Err(PyNotImplementedError::new_err("can't delete item")), generate_pyclass_setitem_slot, Py_mp_ass_subscript, objobjargproc, } /// Macro which expands to three items /// - Trait for a lhs dunder e.g. __add__ /// - Trait for the corresponding rhs e.g. __radd__ /// - A macro which will use dtolnay specialisation to generate the shared slot for the two dunders macro_rules! define_pyclass_binary_operator_slot { ( $lhs_trait:ident, $rhs_trait:ident, $lhs:ident, $rhs:ident, $generate_macro:ident, $slot:ident, $func_ty:ident, ) => { slot_fragment_trait! { $lhs_trait, /// # Safety: _slf and _other must be valid non-null Python objects #[inline] unsafe fn $lhs( self, _py: Python<'_>, _slf: *mut ffi::PyObject, _other: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { Ok(ffi::_Py_NewRef(ffi::Py_NotImplemented())) } } slot_fragment_trait! { $rhs_trait, /// # Safety: _slf and _other must be valid non-null Python objects #[inline] unsafe fn $rhs( self, _py: Python<'_>, _slf: *mut ffi::PyObject, _other: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { Ok(ffi::_Py_NewRef(ffi::Py_NotImplemented())) } } #[doc(hidden)] #[macro_export] macro_rules! $generate_macro { ($cls:ty) => {{ unsafe extern "C" fn __wrap( _slf: *mut $crate::ffi::PyObject, _other: *mut $crate::ffi::PyObject, ) -> *mut $crate::ffi::PyObject { $crate::impl_::trampoline::binaryfunc(_slf, _other, |py, _slf, _other| { use $crate::impl_::pyclass::*; let collector = PyClassImplCollector::<$cls>::new(); let lhs_result = collector.$lhs(py, _slf, _other)?; if lhs_result == $crate::ffi::Py_NotImplemented() { $crate::ffi::Py_DECREF(lhs_result); collector.$rhs(py, _other, _slf) } else { ::std::result::Result::Ok(lhs_result) } }) } $crate::ffi::PyType_Slot { slot: $crate::ffi::$slot, pfunc: __wrap as $crate::ffi::$func_ty as _, } }}; } pub use $generate_macro; }; } define_pyclass_binary_operator_slot! { PyClass__add__SlotFragment, PyClass__radd__SlotFragment, __add__, __radd__, generate_pyclass_add_slot, Py_nb_add, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__sub__SlotFragment, PyClass__rsub__SlotFragment, __sub__, __rsub__, generate_pyclass_sub_slot, Py_nb_subtract, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__mul__SlotFragment, PyClass__rmul__SlotFragment, __mul__, __rmul__, generate_pyclass_mul_slot, Py_nb_multiply, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__mod__SlotFragment, PyClass__rmod__SlotFragment, __mod__, __rmod__, generate_pyclass_mod_slot, Py_nb_remainder, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__divmod__SlotFragment, PyClass__rdivmod__SlotFragment, __divmod__, __rdivmod__, generate_pyclass_divmod_slot, Py_nb_divmod, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__lshift__SlotFragment, PyClass__rlshift__SlotFragment, __lshift__, __rlshift__, generate_pyclass_lshift_slot, Py_nb_lshift, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__rshift__SlotFragment, PyClass__rrshift__SlotFragment, __rshift__, __rrshift__, generate_pyclass_rshift_slot, Py_nb_rshift, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__and__SlotFragment, PyClass__rand__SlotFragment, __and__, __rand__, generate_pyclass_and_slot, Py_nb_and, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__or__SlotFragment, PyClass__ror__SlotFragment, __or__, __ror__, generate_pyclass_or_slot, Py_nb_or, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__xor__SlotFragment, PyClass__rxor__SlotFragment, __xor__, __rxor__, generate_pyclass_xor_slot, Py_nb_xor, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__matmul__SlotFragment, PyClass__rmatmul__SlotFragment, __matmul__, __rmatmul__, generate_pyclass_matmul_slot, Py_nb_matrix_multiply, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__truediv__SlotFragment, PyClass__rtruediv__SlotFragment, __truediv__, __rtruediv__, generate_pyclass_truediv_slot, Py_nb_true_divide, binaryfunc, } define_pyclass_binary_operator_slot! { PyClass__floordiv__SlotFragment, PyClass__rfloordiv__SlotFragment, __floordiv__, __rfloordiv__, generate_pyclass_floordiv_slot, Py_nb_floor_divide, binaryfunc, } slot_fragment_trait! { PyClass__pow__SlotFragment, /// # Safety: _slf and _other must be valid non-null Python objects #[inline] unsafe fn __pow__( self, _py: Python<'_>, _slf: *mut ffi::PyObject, _other: *mut ffi::PyObject, _mod: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { Ok(ffi::_Py_NewRef(ffi::Py_NotImplemented())) } } slot_fragment_trait! { PyClass__rpow__SlotFragment, /// # Safety: _slf and _other must be valid non-null Python objects #[inline] unsafe fn __rpow__( self, _py: Python<'_>, _slf: *mut ffi::PyObject, _other: *mut ffi::PyObject, _mod: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { Ok(ffi::_Py_NewRef(ffi::Py_NotImplemented())) } } #[doc(hidden)] #[macro_export] macro_rules! generate_pyclass_pow_slot { ($cls:ty) => {{ unsafe extern "C" fn __wrap( _slf: *mut $crate::ffi::PyObject, _other: *mut $crate::ffi::PyObject, _mod: *mut $crate::ffi::PyObject, ) -> *mut $crate::ffi::PyObject { $crate::impl_::trampoline::ternaryfunc(_slf, _other, _mod, |py, _slf, _other, _mod| { use $crate::impl_::pyclass::*; let collector = PyClassImplCollector::<$cls>::new(); let lhs_result = collector.__pow__(py, _slf, _other, _mod)?; if lhs_result == $crate::ffi::Py_NotImplemented() { $crate::ffi::Py_DECREF(lhs_result); collector.__rpow__(py, _other, _slf, _mod) } else { ::std::result::Result::Ok(lhs_result) } }) } $crate::ffi::PyType_Slot { slot: $crate::ffi::Py_nb_power, pfunc: __wrap as $crate::ffi::ternaryfunc as _, } }}; } pub use generate_pyclass_pow_slot; slot_fragment_trait! { PyClass__lt__SlotFragment, /// # Safety: _slf and _other must be valid non-null Python objects #[inline] unsafe fn __lt__( self, _py: Python<'_>, _slf: *mut ffi::PyObject, _other: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { Ok(ffi::_Py_NewRef(ffi::Py_NotImplemented())) } } slot_fragment_trait! { PyClass__le__SlotFragment, /// # Safety: _slf and _other must be valid non-null Python objects #[inline] unsafe fn __le__( self, _py: Python<'_>, _slf: *mut ffi::PyObject, _other: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { Ok(ffi::_Py_NewRef(ffi::Py_NotImplemented())) } } slot_fragment_trait! { PyClass__eq__SlotFragment, /// # Safety: _slf and _other must be valid non-null Python objects #[inline] unsafe fn __eq__( self, _py: Python<'_>, _slf: *mut ffi::PyObject, _other: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { Ok(ffi::_Py_NewRef(ffi::Py_NotImplemented())) } } slot_fragment_trait! { PyClass__ne__SlotFragment, /// # Safety: _slf and _other must be valid non-null Python objects #[inline] unsafe fn __ne__( self, py: Python<'_>, slf: *mut ffi::PyObject, other: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { // By default `__ne__` will try `__eq__` and invert the result let slf: &PyAny = py.from_borrowed_ptr(slf); let other: &PyAny = py.from_borrowed_ptr(other); slf.eq(other).map(|is_eq| PyBool::new(py, !is_eq).into_ptr()) } } slot_fragment_trait! { PyClass__gt__SlotFragment, /// # Safety: _slf and _other must be valid non-null Python objects #[inline] unsafe fn __gt__( self, _py: Python<'_>, _slf: *mut ffi::PyObject, _other: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { Ok(ffi::_Py_NewRef(ffi::Py_NotImplemented())) } } slot_fragment_trait! { PyClass__ge__SlotFragment, /// # Safety: _slf and _other must be valid non-null Python objects #[inline] unsafe fn __ge__( self, _py: Python<'_>, _slf: *mut ffi::PyObject, _other: *mut ffi::PyObject, ) -> PyResult<*mut ffi::PyObject> { Ok(ffi::_Py_NewRef(ffi::Py_NotImplemented())) } } #[doc(hidden)] #[macro_export] macro_rules! generate_pyclass_richcompare_slot { ($cls:ty) => {{ impl $cls { #[allow(non_snake_case)] unsafe extern "C" fn __pymethod___richcmp____( slf: *mut $crate::ffi::PyObject, other: *mut $crate::ffi::PyObject, op: ::std::os::raw::c_int, ) -> *mut $crate::ffi::PyObject { $crate::impl_::trampoline::richcmpfunc(slf, other, op, |py, slf, other, op| { use $crate::class::basic::CompareOp; use $crate::impl_::pyclass::*; let collector = PyClassImplCollector::<$cls>::new(); match CompareOp::from_raw(op).expect("invalid compareop") { CompareOp::Lt => collector.__lt__(py, slf, other), CompareOp::Le => collector.__le__(py, slf, other), CompareOp::Eq => collector.__eq__(py, slf, other), CompareOp::Ne => collector.__ne__(py, slf, other), CompareOp::Gt => collector.__gt__(py, slf, other), CompareOp::Ge => collector.__ge__(py, slf, other), } }) } } $crate::ffi::PyType_Slot { slot: $crate::ffi::Py_tp_richcompare, pfunc: <$cls>::__pymethod___richcmp____ as $crate::ffi::richcmpfunc as _, } }}; } pub use generate_pyclass_richcompare_slot; /// Implements a freelist. /// /// Do not implement this trait manually. Instead, use `#[pyclass(freelist = N)]` /// on a Rust struct to implement it. pub trait PyClassWithFreeList: PyClass { fn get_free_list(py: Python<'_>) -> &mut FreeList<*mut ffi::PyObject>; } /// Implementation of tp_alloc for `freelist` classes. /// /// # Safety /// - `subtype` must be a valid pointer to the type object of T or a subclass. /// - The GIL must be held. pub unsafe extern "C" fn alloc_with_freelist( subtype: *mut ffi::PyTypeObject, nitems: ffi::Py_ssize_t, ) -> *mut ffi::PyObject { let py = Python::assume_gil_acquired(); #[cfg(not(Py_3_8))] bpo_35810_workaround(py, subtype); let self_type = T::type_object_raw(py); // If this type is a variable type or the subtype is not equal to this type, we cannot use the // freelist if nitems == 0 && subtype == self_type { if let Some(obj) = T::get_free_list(py).pop() { ffi::PyObject_Init(obj, subtype); return obj as _; } } ffi::PyType_GenericAlloc(subtype, nitems) } /// Implementation of tp_free for `freelist` classes. /// /// # Safety /// - `obj` must be a valid pointer to an instance of T (not a subclass). /// - The GIL must be held. pub unsafe extern "C" fn free_with_freelist(obj: *mut c_void) { let obj = obj as *mut ffi::PyObject; debug_assert_eq!( T::type_object_raw(Python::assume_gil_acquired()), ffi::Py_TYPE(obj) ); if let Some(obj) = T::get_free_list(Python::assume_gil_acquired()).insert(obj) { let ty = ffi::Py_TYPE(obj); // Deduce appropriate inverse of PyType_GenericAlloc let free = if ffi::PyType_IS_GC(ty) != 0 { ffi::PyObject_GC_Del } else { ffi::PyObject_Free }; free(obj as *mut c_void); #[cfg(Py_3_8)] if ffi::PyType_HasFeature(ty, ffi::Py_TPFLAGS_HEAPTYPE) != 0 { ffi::Py_DECREF(ty as *mut ffi::PyObject); } } } /// Workaround for Python issue 35810; no longer necessary in Python 3.8 #[inline] #[cfg(not(Py_3_8))] unsafe fn bpo_35810_workaround(py: Python<'_>, ty: *mut ffi::PyTypeObject) { #[cfg(Py_LIMITED_API)] { // Must check version at runtime for abi3 wheels - they could run against a higher version // than the build config suggests. use crate::sync::GILOnceCell; static IS_PYTHON_3_8: GILOnceCell = GILOnceCell::new(); if *IS_PYTHON_3_8.get_or_init(py, || py.version_info() >= (3, 8)) { // No fix needed - the wheel is running on a sufficiently new interpreter. return; } } #[cfg(not(Py_LIMITED_API))] { // suppress unused variable warning let _ = py; } ffi::Py_INCREF(ty as *mut ffi::PyObject); } /// Implementation detail. Only to be used through our proc macro code. /// Method storage for `#[pyclass]`. /// Allows arbitrary `#[pymethod]` blocks to submit their methods, /// which are eventually collected by `#[pyclass]`. #[cfg(feature = "multiple-pymethods")] pub trait PyClassInventory: inventory::Collect { /// Returns the items for a single `#[pymethods] impl` block fn items(&'static self) -> &'static PyClassItems; } // Items from #[pymethods] if not using inventory. #[cfg(not(feature = "multiple-pymethods"))] pub trait PyMethods { fn py_methods(self) -> &'static PyClassItems; } #[cfg(not(feature = "multiple-pymethods"))] impl PyMethods for &'_ PyClassImplCollector { fn py_methods(self) -> &'static PyClassItems { &PyClassItems { methods: &[], slots: &[], } } } // Text signature for __new__ pub trait PyClassNewTextSignature { fn new_text_signature(self) -> Option<&'static str>; } impl PyClassNewTextSignature for &'_ PyClassImplCollector { #[inline] fn new_text_signature(self) -> Option<&'static str> { None } } // Thread checkers #[doc(hidden)] pub trait PyClassThreadChecker: Sized { fn ensure(&self); fn can_drop(&self, py: Python<'_>) -> bool; fn new() -> Self; private_decl! {} } /// Default thread checker for `#[pyclass]`. /// /// Keeping the T: Send bound here slightly improves the compile /// error message to hint to users to figure out what's wrong /// when `#[pyclass]` types do not implement `Send`. #[doc(hidden)] pub struct SendablePyClass(PhantomData); impl PyClassThreadChecker for SendablePyClass { fn ensure(&self) {} fn can_drop(&self, _py: Python<'_>) -> bool { true } #[inline] fn new() -> Self { SendablePyClass(PhantomData) } private_impl! {} } /// Thread checker for `#[pyclass(unsendable)]` types. /// Panics when the value is accessed by another thread. #[doc(hidden)] pub struct ThreadCheckerImpl(thread::ThreadId); impl ThreadCheckerImpl { fn ensure(&self, type_name: &'static str) { assert_eq!( thread::current().id(), self.0, "{} is unsendable, but sent to another thread", type_name ); } fn can_drop(&self, py: Python<'_>, type_name: &'static str) -> bool { if thread::current().id() != self.0 { PyRuntimeError::new_err(format!( "{} is unsendable, but is being dropped on another thread", type_name )) .write_unraisable(py, None); return false; } true } } impl PyClassThreadChecker for ThreadCheckerImpl { fn ensure(&self) { self.ensure(std::any::type_name::()); } fn can_drop(&self, py: Python<'_>) -> bool { self.can_drop(py, std::any::type_name::()) } fn new() -> Self { ThreadCheckerImpl(thread::current().id()) } private_impl! {} } /// Trait denoting that this class is suitable to be used as a base type for PyClass. pub trait PyClassBaseType: Sized { type LayoutAsBase: PyCellLayout; type BaseNativeType; type Initializer: PyObjectInit; type PyClassMutability: PyClassMutability; } /// All mutable PyClasses can be used as a base type. /// /// In the future this will be extended to immutable PyClasses too. impl PyClassBaseType for T { type LayoutAsBase = crate::pycell::PyCell; type BaseNativeType = T::BaseNativeType; type Initializer = crate::pyclass_init::PyClassInitializer; type PyClassMutability = T::PyClassMutability; } /// Implementation of tp_dealloc for pyclasses without gc pub(crate) unsafe extern "C" fn tp_dealloc(obj: *mut ffi::PyObject) { crate::impl_::trampoline::dealloc(obj, PyCell::::tp_dealloc) } /// Implementation of tp_dealloc for pyclasses with gc pub(crate) unsafe extern "C" fn tp_dealloc_with_gc(obj: *mut ffi::PyObject) { #[cfg(not(PyPy))] { ffi::PyObject_GC_UnTrack(obj.cast()); } crate::impl_::trampoline::dealloc(obj, PyCell::::tp_dealloc) } pub(crate) unsafe extern "C" fn get_sequence_item_from_mapping( obj: *mut ffi::PyObject, index: ffi::Py_ssize_t, ) -> *mut ffi::PyObject { let index = ffi::PyLong_FromSsize_t(index); if index.is_null() { return std::ptr::null_mut(); } let result = ffi::PyObject_GetItem(obj, index); ffi::Py_DECREF(index); result } pub(crate) unsafe extern "C" fn assign_sequence_item_from_mapping( obj: *mut ffi::PyObject, index: ffi::Py_ssize_t, value: *mut ffi::PyObject, ) -> c_int { let index = ffi::PyLong_FromSsize_t(index); if index.is_null() { return -1; } let result = if value.is_null() { ffi::PyObject_DelItem(obj, index) } else { ffi::PyObject_SetItem(obj, index, value) }; ffi::Py_DECREF(index); result } pyo3-0.20.2/src/impl_/pyfunction.rs000064400000000000000000000005211046102023000152300ustar 00000000000000use crate::{derive_utils::PyFunctionArguments, types::PyCFunction, PyResult}; pub use crate::impl_::pymethods::PyMethodDef; pub fn _wrap_pyfunction<'a>( method_def: &PyMethodDef, py_or_module: impl Into>, ) -> PyResult<&'a PyCFunction> { PyCFunction::internal_new(method_def, py_or_module.into()) } pyo3-0.20.2/src/impl_/pymethods.rs000064400000000000000000000227231046102023000150560ustar 00000000000000use crate::gil::LockGIL; use crate::impl_::panic::PanicTrap; use crate::internal_tricks::extract_c_string; use crate::{ffi, PyAny, PyCell, PyClass, PyObject, PyResult, PyTraverseError, PyVisit, Python}; use std::borrow::Cow; use std::ffi::CStr; use std::fmt; use std::os::raw::{c_int, c_void}; use std::panic::{catch_unwind, AssertUnwindSafe}; /// Python 3.8 and up - __ipow__ has modulo argument correctly populated. #[cfg(Py_3_8)] #[repr(transparent)] pub struct IPowModulo(*mut ffi::PyObject); /// Python 3.7 and older - __ipow__ does not have modulo argument correctly populated. #[cfg(not(Py_3_8))] #[repr(transparent)] pub struct IPowModulo(std::mem::MaybeUninit<*mut ffi::PyObject>); /// Helper to use as pymethod ffi definition #[allow(non_camel_case_types)] pub type ipowfunc = unsafe extern "C" fn( arg1: *mut ffi::PyObject, arg2: *mut ffi::PyObject, arg3: IPowModulo, ) -> *mut ffi::PyObject; impl IPowModulo { #[cfg(Py_3_8)] #[inline] pub fn to_borrowed_any(self, py: Python<'_>) -> &PyAny { unsafe { py.from_borrowed_ptr::(self.0) } } #[cfg(not(Py_3_8))] #[inline] pub fn to_borrowed_any(self, py: Python<'_>) -> &PyAny { unsafe { py.from_borrowed_ptr::(ffi::Py_None()) } } } /// `PyMethodDefType` represents different types of Python callable objects. /// It is used by the `#[pymethods]` attribute. pub enum PyMethodDefType { /// Represents class method Class(PyMethodDef), /// Represents static method Static(PyMethodDef), /// Represents normal method Method(PyMethodDef), /// Represents class attribute, used by `#[attribute]` ClassAttribute(PyClassAttributeDef), /// Represents getter descriptor, used by `#[getter]` Getter(PyGetterDef), /// Represents setter descriptor, used by `#[setter]` Setter(PySetterDef), } #[derive(Copy, Clone, Debug)] pub enum PyMethodType { PyCFunction(PyCFunction), PyCFunctionWithKeywords(PyCFunctionWithKeywords), #[cfg(not(Py_LIMITED_API))] PyCFunctionFastWithKeywords(PyCFunctionFastWithKeywords), } // These newtype structs serve no purpose other than wrapping which are function pointers - because // function pointers aren't allowed in const fn, but types wrapping them are! #[derive(Clone, Copy, Debug)] pub struct PyCFunction(pub ffi::PyCFunction); #[derive(Clone, Copy, Debug)] pub struct PyCFunctionWithKeywords(pub ffi::PyCFunctionWithKeywords); #[cfg(not(Py_LIMITED_API))] #[derive(Clone, Copy, Debug)] pub struct PyCFunctionFastWithKeywords(pub ffi::_PyCFunctionFastWithKeywords); #[derive(Clone, Copy)] pub struct PyGetter(pub Getter); #[derive(Clone, Copy)] pub struct PySetter(pub Setter); #[derive(Clone, Copy)] pub struct PyClassAttributeFactory(pub for<'p> fn(Python<'p>) -> PyResult); // TODO: it would be nice to use CStr in these types, but then the constructors can't be const fn // until `CStr::from_bytes_with_nul_unchecked` is const fn. #[derive(Clone, Debug)] pub struct PyMethodDef { pub(crate) ml_name: &'static str, pub(crate) ml_meth: PyMethodType, pub(crate) ml_flags: c_int, pub(crate) ml_doc: &'static str, } #[derive(Copy, Clone)] pub struct PyClassAttributeDef { pub(crate) name: &'static str, pub(crate) meth: PyClassAttributeFactory, } impl PyClassAttributeDef { pub(crate) fn attribute_c_string(&self) -> PyResult> { extract_c_string(self.name, "class attribute name cannot contain nul bytes") } } #[derive(Clone)] pub struct PyGetterDef { pub(crate) name: &'static str, pub(crate) meth: PyGetter, pub(crate) doc: &'static str, } #[derive(Clone)] pub struct PySetterDef { pub(crate) name: &'static str, pub(crate) meth: PySetter, pub(crate) doc: &'static str, } unsafe impl Sync for PyMethodDef {} unsafe impl Sync for PyGetterDef {} unsafe impl Sync for PySetterDef {} impl PyMethodDef { /// Define a function with no `*args` and `**kwargs`. pub const fn noargs(name: &'static str, cfunction: PyCFunction, doc: &'static str) -> Self { Self { ml_name: name, ml_meth: PyMethodType::PyCFunction(cfunction), ml_flags: ffi::METH_NOARGS, ml_doc: doc, } } /// Define a function that can take `*args` and `**kwargs`. pub const fn cfunction_with_keywords( name: &'static str, cfunction: PyCFunctionWithKeywords, doc: &'static str, ) -> Self { Self { ml_name: name, ml_meth: PyMethodType::PyCFunctionWithKeywords(cfunction), ml_flags: ffi::METH_VARARGS | ffi::METH_KEYWORDS, ml_doc: doc, } } /// Define a function that can take `*args` and `**kwargs`. #[cfg(not(Py_LIMITED_API))] pub const fn fastcall_cfunction_with_keywords( name: &'static str, cfunction: PyCFunctionFastWithKeywords, doc: &'static str, ) -> Self { Self { ml_name: name, ml_meth: PyMethodType::PyCFunctionFastWithKeywords(cfunction), ml_flags: ffi::METH_FASTCALL | ffi::METH_KEYWORDS, ml_doc: doc, } } pub const fn flags(mut self, flags: c_int) -> Self { self.ml_flags |= flags; self } /// Convert `PyMethodDef` to Python method definition struct `ffi::PyMethodDef` pub(crate) fn as_method_def(&self) -> PyResult<(ffi::PyMethodDef, PyMethodDefDestructor)> { let meth = match self.ml_meth { PyMethodType::PyCFunction(meth) => ffi::PyMethodDefPointer { PyCFunction: meth.0, }, PyMethodType::PyCFunctionWithKeywords(meth) => ffi::PyMethodDefPointer { PyCFunctionWithKeywords: meth.0, }, #[cfg(not(Py_LIMITED_API))] PyMethodType::PyCFunctionFastWithKeywords(meth) => ffi::PyMethodDefPointer { _PyCFunctionFastWithKeywords: meth.0, }, }; let name = get_name(self.ml_name)?; let doc = get_doc(self.ml_doc)?; let def = ffi::PyMethodDef { ml_name: name.as_ptr(), ml_meth: meth, ml_flags: self.ml_flags, ml_doc: doc.as_ptr(), }; let destructor = PyMethodDefDestructor { name, doc }; Ok((def, destructor)) } } impl PyClassAttributeDef { /// Define a class attribute. pub const fn new(name: &'static str, meth: PyClassAttributeFactory) -> Self { Self { name, meth } } } // Manual implementation because `Python<'_>` does not implement `Debug` and // trait bounds on `fn` compiler-generated derive impls are too restrictive. impl fmt::Debug for PyClassAttributeDef { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { f.debug_struct("PyClassAttributeDef") .field("name", &self.name) .finish() } } /// Class getter / setters pub(crate) type Getter = for<'py> unsafe fn(Python<'py>, *mut ffi::PyObject) -> PyResult<*mut ffi::PyObject>; pub(crate) type Setter = for<'py> unsafe fn(Python<'py>, *mut ffi::PyObject, *mut ffi::PyObject) -> PyResult; impl PyGetterDef { /// Define a getter. pub const fn new(name: &'static str, getter: PyGetter, doc: &'static str) -> Self { Self { name, meth: getter, doc, } } } impl PySetterDef { /// Define a setter. pub const fn new(name: &'static str, setter: PySetter, doc: &'static str) -> Self { Self { name, meth: setter, doc, } } } /// Calls an implementation of __traverse__ for tp_traverse #[doc(hidden)] pub unsafe fn _call_traverse( slf: *mut ffi::PyObject, impl_: fn(&T, PyVisit<'_>) -> Result<(), PyTraverseError>, visit: ffi::visitproc, arg: *mut c_void, ) -> c_int where T: PyClass, { // It is important the implementation of `__traverse__` cannot safely access the GIL, // c.f. https://github.com/PyO3/pyo3/issues/3165, and hence we do not expose our GIL // token to the user code and lock safe methods for acquiring the GIL. // (This includes enforcing the `&self` method receiver as e.g. `PyRef` could // reconstruct a GIL token via `PyRef::py`.) // Since we do not create a `GILPool` at all, it is important that our usage of the GIL // token does not produce any owned objects thereby calling into `register_owned`. let trap = PanicTrap::new("uncaught panic inside __traverse__ handler"); let py = Python::assume_gil_acquired(); let slf = py.from_borrowed_ptr::>(slf); let borrow = slf.try_borrow(); let visit = PyVisit::from_raw(visit, arg, py); let retval = if let Ok(borrow) = borrow { let _lock = LockGIL::during_traverse(); match catch_unwind(AssertUnwindSafe(move || impl_(&*borrow, visit))) { Ok(res) => match res { Ok(()) => 0, Err(PyTraverseError(value)) => value, }, Err(_err) => -1, } } else { 0 }; trap.disarm(); retval } pub(crate) struct PyMethodDefDestructor { // These members are just to avoid leaking CStrings when possible #[allow(dead_code)] name: Cow<'static, CStr>, #[allow(dead_code)] doc: Cow<'static, CStr>, } pub(crate) fn get_name(name: &'static str) -> PyResult> { extract_c_string(name, "function name cannot contain NUL byte.") } pub(crate) fn get_doc(doc: &'static str) -> PyResult> { extract_c_string(doc, "function doc cannot contain NUL byte.") } pyo3-0.20.2/src/impl_/pymodule.rs000064400000000000000000000173261046102023000147030ustar 00000000000000//! Implementation details of `#[pymodule]` which need to be accessible from proc-macro generated code. use std::cell::UnsafeCell; #[cfg(all(not(PyPy), Py_3_9, not(all(windows, Py_LIMITED_API, not(Py_3_10)))))] use std::sync::atomic::{AtomicI64, Ordering}; #[cfg(not(PyPy))] use crate::exceptions::PyImportError; use crate::{ffi, sync::GILOnceCell, types::PyModule, Py, PyResult, Python}; /// `Sync` wrapper of `ffi::PyModuleDef`. pub struct ModuleDef { // wrapped in UnsafeCell so that Rust compiler treats this as interior mutability ffi_def: UnsafeCell, initializer: ModuleInitializer, /// Interpreter ID where module was initialized (not applicable on PyPy). #[cfg(all(not(PyPy), Py_3_9, not(all(windows, Py_LIMITED_API, not(Py_3_10)))))] interpreter: AtomicI64, /// Initialized module object, cached to avoid reinitialization. module: GILOnceCell>, } /// Wrapper to enable initializer to be used in const fns. pub struct ModuleInitializer(pub for<'py> fn(Python<'py>, &PyModule) -> PyResult<()>); unsafe impl Sync for ModuleDef {} impl ModuleDef { /// Make new module definition with given module name. /// /// # Safety /// `name` and `doc` must be null-terminated strings. pub const unsafe fn new( name: &'static str, doc: &'static str, initializer: ModuleInitializer, ) -> Self { const INIT: ffi::PyModuleDef = ffi::PyModuleDef { m_base: ffi::PyModuleDef_HEAD_INIT, m_name: std::ptr::null(), m_doc: std::ptr::null(), m_size: 0, m_methods: std::ptr::null_mut(), m_slots: std::ptr::null_mut(), m_traverse: None, m_clear: None, m_free: None, }; let ffi_def = UnsafeCell::new(ffi::PyModuleDef { m_name: name.as_ptr() as *const _, m_doc: doc.as_ptr() as *const _, ..INIT }); ModuleDef { ffi_def, initializer, // -1 is never expected to be a valid interpreter ID #[cfg(all(not(PyPy), Py_3_9, not(all(windows, Py_LIMITED_API, not(Py_3_10)))))] interpreter: AtomicI64::new(-1), module: GILOnceCell::new(), } } /// Builds a module using user given initializer. Used for [`#[pymodule]`][crate::pymodule]. pub fn make_module(&'static self, py: Python<'_>) -> PyResult> { #[cfg(all(PyPy, not(Py_3_8)))] { const PYPY_GOOD_VERSION: [u8; 3] = [7, 3, 8]; let version = py .import("sys")? .getattr("implementation")? .getattr("version")?; if version.lt(crate::types::PyTuple::new(py, PYPY_GOOD_VERSION))? { let warn = py.import("warnings")?.getattr("warn")?; warn.call1(( "PyPy 3.7 versions older than 7.3.8 are known to have binary \ compatibility issues which may cause segfaults. Please upgrade.", ))?; } } // Check the interpreter ID has not changed, since we currently have no way to guarantee // that static data is not reused across interpreters. // // PyPy does not have subinterpreters, so no need to check interpreter ID. #[cfg(not(PyPy))] { // PyInterpreterState_Get is only available on 3.9 and later, but is missing // from python3.dll for Windows stable API on 3.9 #[cfg(all(Py_3_9, not(all(windows, Py_LIMITED_API, not(Py_3_10)))))] { let current_interpreter = unsafe { ffi::PyInterpreterState_GetID(ffi::PyInterpreterState_Get()) }; crate::err::error_on_minusone(py, current_interpreter)?; if let Err(initialized_interpreter) = self.interpreter.compare_exchange( -1, current_interpreter, Ordering::SeqCst, Ordering::SeqCst, ) { if initialized_interpreter != current_interpreter { return Err(PyImportError::new_err( "PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576", )); } } } #[cfg(not(all(Py_3_9, not(all(windows, Py_LIMITED_API, not(Py_3_10))))))] { // CPython before 3.9 does not have APIs to check the interpreter ID, so best that can be // done to guard against subinterpreters is fail if the module is initialized twice if self.module.get(py).is_some() { return Err(PyImportError::new_err( "PyO3 modules compiled for CPython 3.8 or older may only be initialized once per interpreter process" )); } } } self.module .get_or_try_init(py, || { let module = unsafe { Py::::from_owned_ptr_or_err( py, ffi::PyModule_Create(self.ffi_def.get()), )? }; (self.initializer.0)(py, module.as_ref(py))?; Ok(module) }) .map(|py_module| py_module.clone_ref(py)) } } #[cfg(test)] mod tests { use std::sync::atomic::{AtomicBool, Ordering}; use crate::{types::PyModule, PyResult, Python}; use super::{ModuleDef, ModuleInitializer}; #[test] fn module_init() { static MODULE_DEF: ModuleDef = unsafe { ModuleDef::new( "test_module\0", "some doc\0", ModuleInitializer(|_, m| { m.add("SOME_CONSTANT", 42)?; Ok(()) }), ) }; Python::with_gil(|py| { let module = MODULE_DEF.make_module(py).unwrap().into_ref(py); assert_eq!( module .getattr("__name__") .unwrap() .extract::<&str>() .unwrap(), "test_module", ); assert_eq!( module .getattr("__doc__") .unwrap() .extract::<&str>() .unwrap(), "some doc", ); assert_eq!( module .getattr("SOME_CONSTANT") .unwrap() .extract::() .unwrap(), 42, ); }) } #[test] fn module_def_new() { // To get coverage for ModuleDef::new() need to create a non-static ModuleDef, however init // etc require static ModuleDef, so this test needs to be separated out. static NAME: &str = "test_module\0"; static DOC: &str = "some doc\0"; static INIT_CALLED: AtomicBool = AtomicBool::new(false); #[allow(clippy::unnecessary_wraps)] fn init(_: Python<'_>, _: &PyModule) -> PyResult<()> { INIT_CALLED.store(true, Ordering::SeqCst); Ok(()) } unsafe { let module_def: ModuleDef = ModuleDef::new(NAME, DOC, ModuleInitializer(init)); assert_eq!((*module_def.ffi_def.get()).m_name, NAME.as_ptr() as _); assert_eq!((*module_def.ffi_def.get()).m_doc, DOC.as_ptr() as _); Python::with_gil(|py| { module_def.initializer.0(py, py.import("builtins").unwrap()).unwrap(); assert!(INIT_CALLED.load(Ordering::SeqCst)); }) } } } pyo3-0.20.2/src/impl_/trampoline.rs000064400000000000000000000154101046102023000152070ustar 00000000000000//! Trampolines for various pyfunction and pymethod implementations. //! //! They exist to monomorphise std::panic::catch_unwind once into PyO3, rather than inline in every //! function, thus saving a huge amount of compile-time complexity. use std::{ any::Any, os::raw::c_int, panic::{self, UnwindSafe}, }; use crate::{ callback::PyCallbackOutput, ffi, impl_::panic::PanicTrap, methods::IPowModulo, panic::PanicException, types::PyModule, GILPool, Py, PyResult, Python, }; #[inline] pub unsafe fn module_init( f: for<'py> unsafe fn(Python<'py>) -> PyResult>, ) -> *mut ffi::PyObject { trampoline(|py| f(py).map(|module| module.into_ptr())) } #[inline] pub unsafe fn noargs( slf: *mut ffi::PyObject, args: *mut ffi::PyObject, f: for<'py> unsafe fn(Python<'py>, *mut ffi::PyObject) -> PyResult<*mut ffi::PyObject>, ) -> *mut ffi::PyObject { debug_assert!(args.is_null()); trampoline(|py| f(py, slf)) } macro_rules! trampoline { (pub fn $name:ident($($arg_names:ident: $arg_types:ty),* $(,)?) -> $ret:ty;) => { #[inline] pub unsafe fn $name( $($arg_names: $arg_types,)* f: for<'py> unsafe fn (Python<'py>, $($arg_types),*) -> PyResult<$ret>, ) -> $ret { trampoline(|py| f(py, $($arg_names,)*)) } } } macro_rules! trampolines { ($(pub fn $name:ident($($arg_names:ident: $arg_types:ty),* $(,)?) -> $ret:ty);* ;) => { $(trampoline!(pub fn $name($($arg_names: $arg_types),*) -> $ret;));*; } } trampolines!( pub fn fastcall_with_keywords( slf: *mut ffi::PyObject, args: *const *mut ffi::PyObject, nargs: ffi::Py_ssize_t, kwnames: *mut ffi::PyObject, ) -> *mut ffi::PyObject; pub fn cfunction_with_keywords( slf: *mut ffi::PyObject, args: *mut ffi::PyObject, kwargs: *mut ffi::PyObject, ) -> *mut ffi::PyObject; ); // Trampolines used by slot methods trampolines!( pub fn getattrofunc(slf: *mut ffi::PyObject, attr: *mut ffi::PyObject) -> *mut ffi::PyObject; pub fn setattrofunc( slf: *mut ffi::PyObject, attr: *mut ffi::PyObject, value: *mut ffi::PyObject, ) -> c_int; pub fn binaryfunc(slf: *mut ffi::PyObject, arg1: *mut ffi::PyObject) -> *mut ffi::PyObject; pub fn descrgetfunc( slf: *mut ffi::PyObject, arg1: *mut ffi::PyObject, arg2: *mut ffi::PyObject, ) -> *mut ffi::PyObject; pub fn getiterfunc(slf: *mut ffi::PyObject) -> *mut ffi::PyObject; pub fn hashfunc(slf: *mut ffi::PyObject) -> ffi::Py_hash_t; pub fn inquiry(slf: *mut ffi::PyObject) -> c_int; pub fn iternextfunc(slf: *mut ffi::PyObject) -> *mut ffi::PyObject; pub fn lenfunc(slf: *mut ffi::PyObject) -> ffi::Py_ssize_t; pub fn newfunc( subtype: *mut ffi::PyTypeObject, args: *mut ffi::PyObject, kwargs: *mut ffi::PyObject, ) -> *mut ffi::PyObject; pub fn objobjproc(slf: *mut ffi::PyObject, arg1: *mut ffi::PyObject) -> c_int; pub fn reprfunc(slf: *mut ffi::PyObject) -> *mut ffi::PyObject; pub fn richcmpfunc( slf: *mut ffi::PyObject, other: *mut ffi::PyObject, op: c_int, ) -> *mut ffi::PyObject; pub fn ssizeargfunc(arg1: *mut ffi::PyObject, arg2: ffi::Py_ssize_t) -> *mut ffi::PyObject; pub fn ternaryfunc( slf: *mut ffi::PyObject, arg1: *mut ffi::PyObject, arg2: *mut ffi::PyObject, ) -> *mut ffi::PyObject; pub fn unaryfunc(slf: *mut ffi::PyObject) -> *mut ffi::PyObject; ); #[cfg(any(not(Py_LIMITED_API), Py_3_11))] trampoline! { pub fn getbufferproc(slf: *mut ffi::PyObject, buf: *mut ffi::Py_buffer, flags: c_int) -> c_int; } #[cfg(any(not(Py_LIMITED_API), Py_3_11))] #[inline] pub unsafe fn releasebufferproc( slf: *mut ffi::PyObject, buf: *mut ffi::Py_buffer, f: for<'py> unsafe fn(Python<'py>, *mut ffi::PyObject, *mut ffi::Py_buffer) -> PyResult<()>, ) { trampoline_unraisable(|py| f(py, slf, buf), slf) } #[inline] pub(crate) unsafe fn dealloc( slf: *mut ffi::PyObject, f: for<'py> unsafe fn(Python<'py>, *mut ffi::PyObject) -> (), ) { // After calling tp_dealloc the object is no longer valid, // so pass null_mut() to the context. // // (Note that we don't allow the implementation `f` to fail.) trampoline_unraisable( |py| { f(py, slf); Ok(()) }, std::ptr::null_mut(), ) } // Ipowfunc is a unique case where PyO3 has its own type // to workaround a problem on 3.7 (see IPowModulo type definition). // Once 3.7 support dropped can just remove this. trampoline!( pub fn ipowfunc( arg1: *mut ffi::PyObject, arg2: *mut ffi::PyObject, arg3: IPowModulo, ) -> *mut ffi::PyObject; ); /// Implementation of trampoline functions, which sets up a GILPool and calls F. /// /// Panics during execution are trapped so that they don't propagate through any /// outer FFI boundary. #[inline] pub(crate) fn trampoline(body: F) -> R where F: for<'py> FnOnce(Python<'py>) -> PyResult + UnwindSafe, R: PyCallbackOutput, { let trap = PanicTrap::new("uncaught panic at ffi boundary"); let pool = unsafe { GILPool::new() }; let py = pool.python(); let out = panic_result_into_callback_output( py, panic::catch_unwind(move || -> PyResult<_> { body(py) }), ); trap.disarm(); out } /// Converts the output of std::panic::catch_unwind into a Python function output, either by raising a Python /// exception or by unwrapping the contained success output. #[inline] fn panic_result_into_callback_output( py: Python<'_>, panic_result: Result, Box>, ) -> R where R: PyCallbackOutput, { let py_err = match panic_result { Ok(Ok(value)) => return value, Ok(Err(py_err)) => py_err, Err(payload) => PanicException::from_panic_payload(payload), }; py_err.restore(py); R::ERR_VALUE } /// Implementation of trampoline for functions which can't return an error. /// /// Panics during execution are trapped so that they don't propagate through any /// outer FFI boundary. /// /// Exceptions produced are sent to `sys.unraisablehook`. /// /// # Safety /// /// ctx must be either a valid ffi::PyObject or NULL #[inline] unsafe fn trampoline_unraisable(body: F, ctx: *mut ffi::PyObject) where F: for<'py> FnOnce(Python<'py>) -> PyResult<()> + UnwindSafe, { let trap = PanicTrap::new("uncaught panic at ffi boundary"); let pool = GILPool::new(); let py = pool.python(); if let Err(py_err) = panic::catch_unwind(move || body(py)) .unwrap_or_else(|payload| Err(PanicException::from_panic_payload(payload))) { py_err.write_unraisable(py, py.from_borrowed_ptr_or_opt(ctx)); } trap.disarm(); } pyo3-0.20.2/src/impl_/wrap.rs000064400000000000000000000024231046102023000140060ustar 00000000000000use crate::{IntoPy, Py, PyAny, PyErr, PyObject, PyResult, Python}; /// Used to wrap values in `Option` for default arguments. pub trait SomeWrap { fn wrap(self) -> T; } impl SomeWrap> for T { fn wrap(self) -> Option { Some(self) } } impl SomeWrap> for Option { fn wrap(self) -> Self { self } } /// Used to wrap the result of `#[pyfunction]` and `#[pymethods]`. pub trait OkWrap { type Error; fn wrap(self, py: Python<'_>) -> Result, Self::Error>; } // The T: IntoPy bound here is necessary to prevent the // implementation for Result from conflicting impl OkWrap for T where T: IntoPy, { type Error = PyErr; fn wrap(self, py: Python<'_>) -> PyResult> { Ok(self.into_py(py)) } } impl OkWrap for Result where T: IntoPy, { type Error = E; fn wrap(self, py: Python<'_>) -> Result, Self::Error> { self.map(|o| o.into_py(py)) } } #[cfg(test)] mod tests { use super::*; #[test] fn wrap_option() { let a: Option = SomeWrap::wrap(42); assert_eq!(a, Some(42)); let b: Option = SomeWrap::wrap(None); assert_eq!(b, None); } } pyo3-0.20.2/src/impl_.rs000064400000000000000000000011221046102023000130300ustar 00000000000000#![allow(missing_docs)] //! Internals of PyO3 which are accessed by code expanded from PyO3's procedural macros. //! //! Usage of any of these APIs in downstream code is implicitly acknowledging that these //! APIs may may change at any time without documentation in the CHANGELOG and without //! breaking semver guarantees. pub mod deprecations; pub mod extract_argument; pub mod freelist; pub mod frompyobject; pub(crate) mod not_send; pub mod panic; pub mod pycell; pub mod pyclass; pub mod pyfunction; pub mod pymethods; pub mod pymodule; #[doc(hidden)] pub mod trampoline; pub mod wrap; pyo3-0.20.2/src/inspect/mod.rs000064400000000000000000000002101046102023000141510ustar 00000000000000//! Runtime inspection of objects exposed to Python. //! //! Tracking issue: . pub mod types; pyo3-0.20.2/src/inspect/types.rs000064400000000000000000000363151046102023000145550ustar 00000000000000//! Data types used to describe runtime Python types. use std::borrow::Cow; use std::fmt::{Display, Formatter}; /// Designation of a Python type. /// /// This enum is used to handle advanced types, such as types with generics. /// Its [`Display`] implementation can be used to convert to the type hint notation (e.g. `List[int]`). #[derive(Debug, Clone, Eq, PartialEq)] pub enum TypeInfo { /// The type `typing.Any`, which represents any possible value (unknown type). Any, /// The type `typing.None`. None, /// The type `typing.NoReturn`, which represents functions that never return (they can still panic / throw, similar to `never` in Rust). NoReturn, /// The type `typing.Callable`. /// /// The first argument represents the parameters of the callable: /// - `Some` of a vector of types to represent the signature, /// - `None` if the signature is unknown (allows any number of arguments with type `Any`). /// /// The second argument represents the return type. Callable(Option>, Box), /// The type `typing.tuple`. /// /// The argument represents the contents of the tuple: /// - `Some` of a vector of types to represent the accepted types, /// - `Some` of an empty vector for the empty tuple, /// - `None` if the number and type of accepted values is unknown. /// /// If the number of accepted values is unknown, but their type is, use [`Self::UnsizedTypedTuple`]. Tuple(Option>), /// The type `typing.Tuple`. /// /// Use this variant to represent a tuple of unknown size but of known types. /// /// If the type is unknown, or if the number of elements is known, use [`Self::Tuple`]. UnsizedTypedTuple(Box), /// A Python class. Class { /// The module this class comes from. module: ModuleName, /// The name of this class, as it appears in a type hint. name: Cow<'static, str>, /// The generics accepted by this class (empty vector if this class is not generic). type_vars: Vec, }, } /// Declares which module a type is a part of. #[derive(Debug, Clone, Eq, PartialEq)] pub enum ModuleName { /// The type is built-in: it doesn't need to be imported. Builtin, /// The type is in the current module: it doesn't need to be imported in this module, but needs to be imported in others. CurrentModule, /// The type is in the specified module. Module(Cow<'static, str>), } impl TypeInfo { /// Returns the module in which a type is declared. /// /// Returns `None` if the type is declared in the current module. pub fn module_name(&self) -> Option<&str> { match self { TypeInfo::Any | TypeInfo::None | TypeInfo::NoReturn | TypeInfo::Callable(_, _) | TypeInfo::Tuple(_) | TypeInfo::UnsizedTypedTuple(_) => Some("typing"), TypeInfo::Class { module, .. } => match module { ModuleName::Builtin => Some("builtins"), ModuleName::CurrentModule => None, ModuleName::Module(name) => Some(name), }, } } /// Returns the name of a type. /// /// The name of a type is the part of the hint that is not generic (e.g. `List` instead of `List[int]`). pub fn name(&self) -> Cow<'_, str> { Cow::from(match self { TypeInfo::Any => "Any", TypeInfo::None => "None", TypeInfo::NoReturn => "NoReturn", TypeInfo::Callable(_, _) => "Callable", TypeInfo::Tuple(_) => "Tuple", TypeInfo::UnsizedTypedTuple(_) => "Tuple", TypeInfo::Class { name, .. } => name, }) } } // Utilities for easily instantiating TypeInfo structures for built-in/common types. impl TypeInfo { /// The Python `Optional` type. pub fn optional_of(t: TypeInfo) -> TypeInfo { TypeInfo::Class { module: ModuleName::Module(Cow::from("typing")), name: Cow::from("Optional"), type_vars: vec![t], } } /// The Python `Union` type. pub fn union_of(types: &[TypeInfo]) -> TypeInfo { TypeInfo::Class { module: ModuleName::Module(Cow::from("typing")), name: Cow::from("Union"), type_vars: types.to_vec(), } } /// The Python `List` type. pub fn list_of(t: TypeInfo) -> TypeInfo { TypeInfo::Class { module: ModuleName::Module(Cow::from("typing")), name: Cow::from("List"), type_vars: vec![t], } } /// The Python `Sequence` type. pub fn sequence_of(t: TypeInfo) -> TypeInfo { TypeInfo::Class { module: ModuleName::Module(Cow::from("typing")), name: Cow::from("Sequence"), type_vars: vec![t], } } /// The Python `Set` type. pub fn set_of(t: TypeInfo) -> TypeInfo { TypeInfo::Class { module: ModuleName::Module(Cow::from("typing")), name: Cow::from("Set"), type_vars: vec![t], } } /// The Python `FrozenSet` type. pub fn frozen_set_of(t: TypeInfo) -> TypeInfo { TypeInfo::Class { module: ModuleName::Module(Cow::from("typing")), name: Cow::from("FrozenSet"), type_vars: vec![t], } } /// The Python `Iterable` type. pub fn iterable_of(t: TypeInfo) -> TypeInfo { TypeInfo::Class { module: ModuleName::Module(Cow::from("typing")), name: Cow::from("Iterable"), type_vars: vec![t], } } /// The Python `Iterator` type. pub fn iterator_of(t: TypeInfo) -> TypeInfo { TypeInfo::Class { module: ModuleName::Module(Cow::from("typing")), name: Cow::from("Iterator"), type_vars: vec![t], } } /// The Python `Dict` type. pub fn dict_of(k: TypeInfo, v: TypeInfo) -> TypeInfo { TypeInfo::Class { module: ModuleName::Module(Cow::from("typing")), name: Cow::from("Dict"), type_vars: vec![k, v], } } /// The Python `Mapping` type. pub fn mapping_of(k: TypeInfo, v: TypeInfo) -> TypeInfo { TypeInfo::Class { module: ModuleName::Module(Cow::from("typing")), name: Cow::from("Mapping"), type_vars: vec![k, v], } } /// Convenience factory for non-generic builtins (e.g. `int`). pub fn builtin(name: &'static str) -> TypeInfo { TypeInfo::Class { module: ModuleName::Builtin, name: Cow::from(name), type_vars: vec![], } } } impl Display for TypeInfo { fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result { match self { TypeInfo::Any | TypeInfo::None | TypeInfo::NoReturn => write!(f, "{}", self.name()), TypeInfo::Callable(input, output) => { write!(f, "Callable[")?; if let Some(input) = input { write!(f, "[")?; let mut comma = false; for arg in input { if comma { write!(f, ", ")?; } write!(f, "{}", arg)?; comma = true; } write!(f, "]")?; } else { write!(f, "...")?; } write!(f, ", {}]", output) } TypeInfo::Tuple(types) => { write!(f, "Tuple[")?; if let Some(types) = types { if types.is_empty() { write!(f, "()")?; } else { let mut comma = false; for t in types { if comma { write!(f, ", ")?; } write!(f, "{}", t)?; comma = true; } } } else { write!(f, "...")?; } write!(f, "]") } TypeInfo::UnsizedTypedTuple(t) => write!(f, "Tuple[{}, ...]", t), TypeInfo::Class { name, type_vars, .. } => { write!(f, "{}", name)?; if !type_vars.is_empty() { write!(f, "[")?; let mut comma = false; for var in type_vars { if comma { write!(f, ", ")?; } write!(f, "{}", var)?; comma = true; } write!(f, "]") } else { Ok(()) } } } } } #[cfg(test)] mod test { use std::borrow::Cow; use crate::inspect::types::{ModuleName, TypeInfo}; pub fn assert_display(t: &TypeInfo, expected: &str) { assert_eq!(format!("{}", t), expected) } #[test] fn basic() { assert_display(&TypeInfo::Any, "Any"); assert_display(&TypeInfo::None, "None"); assert_display(&TypeInfo::NoReturn, "NoReturn"); assert_display(&TypeInfo::builtin("int"), "int"); } #[test] fn callable() { let any_to_int = TypeInfo::Callable(None, Box::new(TypeInfo::builtin("int"))); assert_display(&any_to_int, "Callable[..., int]"); let sum = TypeInfo::Callable( Some(vec![TypeInfo::builtin("int"), TypeInfo::builtin("int")]), Box::new(TypeInfo::builtin("int")), ); assert_display(&sum, "Callable[[int, int], int]"); } #[test] fn tuple() { let any = TypeInfo::Tuple(None); assert_display(&any, "Tuple[...]"); let triple = TypeInfo::Tuple(Some(vec![ TypeInfo::builtin("int"), TypeInfo::builtin("str"), TypeInfo::builtin("bool"), ])); assert_display(&triple, "Tuple[int, str, bool]"); let empty = TypeInfo::Tuple(Some(vec![])); assert_display(&empty, "Tuple[()]"); let typed = TypeInfo::UnsizedTypedTuple(Box::new(TypeInfo::builtin("bool"))); assert_display(&typed, "Tuple[bool, ...]"); } #[test] fn class() { let class1 = TypeInfo::Class { module: ModuleName::CurrentModule, name: Cow::from("MyClass"), type_vars: vec![], }; assert_display(&class1, "MyClass"); let class2 = TypeInfo::Class { module: ModuleName::CurrentModule, name: Cow::from("MyClass"), type_vars: vec![TypeInfo::builtin("int"), TypeInfo::builtin("bool")], }; assert_display(&class2, "MyClass[int, bool]"); } #[test] fn collections() { let int = TypeInfo::builtin("int"); let bool = TypeInfo::builtin("bool"); let str = TypeInfo::builtin("str"); let list = TypeInfo::list_of(int.clone()); assert_display(&list, "List[int]"); let sequence = TypeInfo::sequence_of(bool.clone()); assert_display(&sequence, "Sequence[bool]"); let optional = TypeInfo::optional_of(str.clone()); assert_display(&optional, "Optional[str]"); let iterable = TypeInfo::iterable_of(int.clone()); assert_display(&iterable, "Iterable[int]"); let iterator = TypeInfo::iterator_of(bool); assert_display(&iterator, "Iterator[bool]"); let dict = TypeInfo::dict_of(int.clone(), str.clone()); assert_display(&dict, "Dict[int, str]"); let mapping = TypeInfo::mapping_of(int, str.clone()); assert_display(&mapping, "Mapping[int, str]"); let set = TypeInfo::set_of(str.clone()); assert_display(&set, "Set[str]"); let frozen_set = TypeInfo::frozen_set_of(str); assert_display(&frozen_set, "FrozenSet[str]"); } #[test] fn complicated() { let int = TypeInfo::builtin("int"); assert_display(&int, "int"); let bool = TypeInfo::builtin("bool"); assert_display(&bool, "bool"); let str = TypeInfo::builtin("str"); assert_display(&str, "str"); let any = TypeInfo::Any; assert_display(&any, "Any"); let params = TypeInfo::union_of(&[int.clone(), str]); assert_display(¶ms, "Union[int, str]"); let func = TypeInfo::Callable(Some(vec![params, any]), Box::new(bool)); assert_display(&func, "Callable[[Union[int, str], Any], bool]"); let dict = TypeInfo::mapping_of(int, func); assert_display( &dict, "Mapping[int, Callable[[Union[int, str], Any], bool]]", ); } } #[cfg(test)] mod conversion { use std::collections::{HashMap, HashSet}; use crate::inspect::types::test::assert_display; use crate::{FromPyObject, IntoPy}; #[test] fn unsigned_int() { assert_display(&usize::type_output(), "int"); assert_display(&usize::type_input(), "int"); assert_display(&u8::type_output(), "int"); assert_display(&u8::type_input(), "int"); assert_display(&u16::type_output(), "int"); assert_display(&u16::type_input(), "int"); assert_display(&u32::type_output(), "int"); assert_display(&u32::type_input(), "int"); assert_display(&u64::type_output(), "int"); assert_display(&u64::type_input(), "int"); } #[test] fn signed_int() { assert_display(&isize::type_output(), "int"); assert_display(&isize::type_input(), "int"); assert_display(&i8::type_output(), "int"); assert_display(&i8::type_input(), "int"); assert_display(&i16::type_output(), "int"); assert_display(&i16::type_input(), "int"); assert_display(&i32::type_output(), "int"); assert_display(&i32::type_input(), "int"); assert_display(&i64::type_output(), "int"); assert_display(&i64::type_input(), "int"); } #[test] fn float() { assert_display(&f32::type_output(), "float"); assert_display(&f32::type_input(), "float"); assert_display(&f64::type_output(), "float"); assert_display(&f64::type_input(), "float"); } #[test] fn bool() { assert_display(&bool::type_output(), "bool"); assert_display(&bool::type_input(), "bool"); } #[test] fn text() { assert_display(&String::type_output(), "str"); assert_display(&String::type_input(), "str"); assert_display(&<&[u8]>::type_output(), "bytes"); assert_display(&<&[u8]>::type_input(), "bytes"); } #[test] fn collections() { assert_display(&>::type_output(), "List[int]"); assert_display(&>::type_input(), "Sequence[int]"); assert_display(&>::type_output(), "Set[int]"); assert_display(&>::type_input(), "Set[int]"); assert_display(&>::type_output(), "Dict[int, float]"); assert_display(&>::type_input(), "Mapping[int, float]"); assert_display(&<(usize, f32)>::type_input(), "Tuple[int, float]"); } } pyo3-0.20.2/src/instance.rs000064400000000000000000001203551046102023000135460ustar 00000000000000use crate::conversion::PyTryFrom; use crate::err::{self, PyDowncastError, PyErr, PyResult}; use crate::gil; use crate::pycell::{PyBorrowError, PyBorrowMutError, PyCell}; use crate::pyclass::boolean_struct::{False, True}; use crate::types::{PyDict, PyString, PyTuple}; use crate::{ ffi, AsPyPointer, FromPyObject, IntoPy, PyAny, PyClass, PyClassInitializer, PyRef, PyRefMut, PyTypeInfo, Python, ToPyObject, }; use std::marker::PhantomData; use std::mem; use std::ptr::NonNull; /// Types that are built into the Python interpreter. /// /// PyO3 is designed in a way that all references to those types are bound /// to the GIL, which is why you can get a token from all references of those /// types. /// /// # Safety /// /// This trait must only be implemented for types which cannot be accessed without the GIL. pub unsafe trait PyNativeType: Sized { /// Returns a GIL marker constrained to the lifetime of this type. #[inline] fn py(&self) -> Python<'_> { unsafe { Python::assume_gil_acquired() } } /// Cast `&PyAny` to `&Self` without no type checking. /// /// # Safety /// /// `obj` must have the same layout as `*const ffi::PyObject` and must be /// an instance of a type corresponding to `Self`. unsafe fn unchecked_downcast(obj: &PyAny) -> &Self { &*(obj.as_ptr() as *const Self) } } /// A GIL-independent reference to an object allocated on the Python heap. /// /// This type does not auto-dereference to the inner object because you must prove you hold the GIL to access it. /// Instead, call one of its methods to access the inner object: /// - [`Py::as_ref`], to borrow a GIL-bound reference to the contained object. /// - [`Py::borrow`], [`Py::try_borrow`], [`Py::borrow_mut`], or [`Py::try_borrow_mut`], /// to get a (mutable) reference to a contained pyclass, using a scheme similar to std's [`RefCell`]. /// See the [`PyCell` guide entry](https://pyo3.rs/latest/class.html#pycell-and-interior-mutability) /// for more information. /// - You can call methods directly on `Py` with [`Py::call`], [`Py::call_method`] and friends. /// These require passing in the [`Python<'py>`](crate::Python) token but are otherwise similar to the corresponding /// methods on [`PyAny`]. /// /// # Example: Storing Python objects in structs /// /// As all the native Python objects only appear as references, storing them in structs doesn't work well. /// For example, this won't compile: /// /// ```compile_fail /// # use pyo3::prelude::*; /// # use pyo3::types::PyDict; /// # /// #[pyclass] /// struct Foo<'py> { /// inner: &'py PyDict, /// } /// /// impl Foo { /// fn new() -> Foo { /// let foo = Python::with_gil(|py| { /// // `py` will only last for this scope. /// /// // `&PyDict` derives its lifetime from `py` and /// // so won't be able to outlive this closure. /// let dict: &PyDict = PyDict::new(py); /// /// // because `Foo` contains `dict` its lifetime /// // is now also tied to `py`. /// Foo { inner: dict } /// }); /// // Foo is no longer valid. /// // Returning it from this function is a 💥 compiler error 💥 /// foo /// } /// } /// ``` /// /// [`Py`]`` can be used to get around this by converting `dict` into a GIL-independent reference: /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyDict; /// /// #[pyclass] /// struct Foo { /// inner: Py, /// } /// /// #[pymethods] /// impl Foo { /// #[new] /// fn __new__() -> Foo { /// Python::with_gil(|py| { /// let dict: Py = PyDict::new(py).into(); /// Foo { inner: dict } /// }) /// } /// } /// # /// # fn main() -> PyResult<()> { /// # Python::with_gil(|py| { /// # let m = pyo3::types::PyModule::new(py, "test")?; /// # m.add_class::()?; /// # /// # let foo: &PyCell = m.getattr("Foo")?.call0()?.downcast()?; /// # let dict = &foo.borrow().inner; /// # let dict: &PyDict = dict.as_ref(py); /// # /// # Ok(()) /// # }) /// # } /// ``` /// /// This can also be done with other pyclasses: /// ```rust /// use pyo3::prelude::*; /// /// #[pyclass] /// struct Bar {/* ... */} /// /// #[pyclass] /// struct Foo { /// inner: Py, /// } /// /// #[pymethods] /// impl Foo { /// #[new] /// fn __new__() -> PyResult { /// Python::with_gil(|py| { /// let bar: Py = Py::new(py, Bar {})?; /// Ok(Foo { inner: bar }) /// }) /// } /// } /// # /// # fn main() -> PyResult<()> { /// # Python::with_gil(|py| { /// # let m = pyo3::types::PyModule::new(py, "test")?; /// # m.add_class::()?; /// # /// # let foo: &PyCell = m.getattr("Foo")?.call0()?.downcast()?; /// # let bar = &foo.borrow().inner; /// # let bar: &Bar = &*bar.borrow(py); /// # /// # Ok(()) /// # }) /// # } /// ``` /// /// # Example: Shared ownership of Python objects /// /// `Py` can be used to share ownership of a Python object, similar to std's [`Rc`]``. /// As with [`Rc`]``, cloning it increases its reference count rather than duplicating /// the underlying object. /// /// This can be done using either [`Py::clone_ref`] or [`Py`]``'s [`Clone`] trait implementation. /// [`Py::clone_ref`] will be faster if you happen to be already holding the GIL. /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyDict; /// /// # fn main() { /// Python::with_gil(|py| { /// let first: Py = PyDict::new(py).into(); /// /// // All of these are valid syntax /// let second = Py::clone_ref(&first, py); /// let third = first.clone_ref(py); /// let fourth = Py::clone(&first); /// let fifth = first.clone(); /// /// // Disposing of our original `Py` just decrements the reference count. /// drop(first); /// /// // They all point to the same object /// assert!(second.is(&third)); /// assert!(fourth.is(&fifth)); /// assert!(second.is(&fourth)); /// }); /// # } /// ``` /// /// # Preventing reference cycles /// /// It is easy to accidentally create reference cycles using [`Py`]``. /// The Python interpreter can break these reference cycles within pyclasses if they /// [integrate with the garbage collector][gc]. If your pyclass contains other Python /// objects you should implement it to avoid leaking memory. /// /// # A note on Python reference counts /// /// Dropping a [`Py`]`` will eventually decrease Python's reference count /// of the pointed-to variable, allowing Python's garbage collector to free /// the associated memory, but this may not happen immediately. This is /// because a [`Py`]`` can be dropped at any time, but the Python reference /// count can only be modified when the GIL is held. /// /// If a [`Py`]`` is dropped while its thread happens to be holding the /// GIL then the Python reference count will be decreased immediately. /// Otherwise, the reference count will be decreased the next time the GIL is /// reacquired. /// /// # A note on `Send` and `Sync` /// /// Accessing this object is threadsafe, since any access to its API requires a [`Python<'py>`](crate::Python) token. /// As you can only get this by acquiring the GIL, `Py<...>` "implements [`Send`] and [`Sync`]. /// /// [`Rc`]: std::rc::Rc /// [`RefCell`]: std::cell::RefCell /// [gc]: https://pyo3.rs/main/class/protocols.html#garbage-collector-integration #[repr(transparent)] pub struct Py(NonNull, PhantomData); // The inner value is only accessed through ways that require proving the gil is held #[cfg(feature = "nightly")] unsafe impl crate::marker::Ungil for Py {} unsafe impl Send for Py {} unsafe impl Sync for Py {} impl Py where T: PyClass, { /// Creates a new instance `Py` of a `#[pyclass]` on the Python heap. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// /// #[pyclass] /// struct Foo {/* fields omitted */} /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult> { /// let foo: Py = Py::new(py, Foo {})?; /// Ok(foo) /// })?; /// # Ok(()) /// # } /// ``` pub fn new(py: Python<'_>, value: impl Into>) -> PyResult> { let initializer = value.into(); let obj = initializer.create_cell(py)?; let ob = unsafe { Py::from_owned_ptr(py, obj as _) }; Ok(ob) } } impl Py where T: PyTypeInfo, { /// Borrows a GIL-bound reference to the contained `T`. /// /// By binding to the GIL lifetime, this allows the GIL-bound reference to not require /// [`Python<'py>`](crate::Python) for any of its methods, which makes calling methods /// on it more ergonomic. /// /// For native types, this reference is `&T`. For pyclasses, this is `&PyCell`. /// /// Note that the lifetime of the returned reference is the shortest of `&self` and /// [`Python<'py>`](crate::Python). /// Consider using [`Py::into_ref`] instead if this poses a problem. /// /// # Examples /// /// Get access to `&PyList` from `Py`: /// /// ``` /// # use pyo3::prelude::*; /// # use pyo3::types::PyList; /// # /// Python::with_gil(|py| { /// let list: Py = PyList::empty(py).into(); /// let list: &PyList = list.as_ref(py); /// assert_eq!(list.len(), 0); /// }); /// ``` /// /// Get access to `&PyCell` from `Py`: /// /// ``` /// # use pyo3::prelude::*; /// # /// #[pyclass] /// struct MyClass {} /// /// Python::with_gil(|py| { /// let my_class: Py = Py::new(py, MyClass {}).unwrap(); /// let my_class_cell: &PyCell = my_class.as_ref(py); /// assert!(my_class_cell.try_borrow().is_ok()); /// }); /// ``` pub fn as_ref<'py>(&'py self, _py: Python<'py>) -> &'py T::AsRefTarget { let any = self.as_ptr() as *const PyAny; unsafe { PyNativeType::unchecked_downcast(&*any) } } /// Borrows a GIL-bound reference to the contained `T` independently of the lifetime of `T`. /// /// This method is similar to [`as_ref`](#method.as_ref) but consumes `self` and registers the /// Python object reference in PyO3's object storage. The reference count for the Python /// object will not be decreased until the GIL lifetime ends. /// /// You should prefer using [`as_ref`](#method.as_ref) if you can as it'll have less overhead. /// /// # Examples /// /// [`Py::as_ref`]'s lifetime limitation forbids creating a function that references a /// variable created inside the function. /// /// ```rust,compile_fail /// # use pyo3::prelude::*; /// # /// fn new_py_any<'py>(py: Python<'py>, value: impl IntoPy>) -> &'py PyAny { /// let obj: Py = value.into_py(py); /// /// // The lifetime of the return value of this function is the shortest /// // of `obj` and `py`. As `obj` is owned by the current function, /// // Rust won't let the return value escape this function! /// obj.as_ref(py) /// } /// ``` /// /// This can be solved by using [`Py::into_ref`] instead, which does not suffer from this issue. /// Note that the lifetime of the [`Python<'py>`](crate::Python) token is transferred to /// the returned reference. /// /// ```rust /// # use pyo3::prelude::*; /// # #[allow(dead_code)] // This is just to show it compiles. /// fn new_py_any<'py>(py: Python<'py>, value: impl IntoPy>) -> &'py PyAny { /// let obj: Py = value.into_py(py); /// /// // This reference's lifetime is determined by `py`'s lifetime. /// // Because that originates from outside this function, /// // this return value is allowed. /// obj.into_ref(py) /// } /// ``` pub fn into_ref(self, py: Python<'_>) -> &T::AsRefTarget { unsafe { py.from_owned_ptr(self.into_ptr()) } } } impl Py { /// Returns the raw FFI pointer represented by self. /// /// # Safety /// /// Callers are responsible for ensuring that the pointer does not outlive self. /// /// The reference is borrowed; callers should not decrease the reference count /// when they are finished with the pointer. #[inline] pub fn as_ptr(&self) -> *mut ffi::PyObject { self.0.as_ptr() } /// Returns an owned raw FFI pointer represented by self. /// /// # Safety /// /// The reference is owned; when finished the caller should either transfer ownership /// of the pointer or decrease the reference count (e.g. with [`pyo3::ffi::Py_DecRef`](crate::ffi::Py_DecRef)). #[inline] pub fn into_ptr(self) -> *mut ffi::PyObject { let ptr = self.0.as_ptr(); std::mem::forget(self); ptr } } impl Py where T: PyClass, { /// Immutably borrows the value `T`. /// /// This borrow lasts while the returned [`PyRef`] exists. /// Multiple immutable borrows can be taken out at the same time. /// /// For frozen classes, the simpler [`get`][Self::get] is available. /// /// Equivalent to `self.as_ref(py).borrow()` - /// see [`PyCell::borrow`](crate::pycell::PyCell::borrow). /// /// # Examples /// /// ```rust /// # use pyo3::prelude::*; /// # /// #[pyclass] /// struct Foo { /// inner: u8, /// } /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let foo: Py = Py::new(py, Foo { inner: 73 })?; /// let inner: &u8 = &foo.borrow(py).inner; /// /// assert_eq!(*inner, 73); /// Ok(()) /// })?; /// # Ok(()) /// # } /// ``` /// /// # Panics /// /// Panics if the value is currently mutably borrowed. For a non-panicking variant, use /// [`try_borrow`](#method.try_borrow). pub fn borrow<'py>(&'py self, py: Python<'py>) -> PyRef<'py, T> { self.as_ref(py).borrow() } /// Mutably borrows the value `T`. /// /// This borrow lasts while the returned [`PyRefMut`] exists. /// /// Equivalent to `self.as_ref(py).borrow_mut()` - /// see [`PyCell::borrow_mut`](crate::pycell::PyCell::borrow_mut). /// /// # Examples /// /// ``` /// # use pyo3::prelude::*; /// # /// #[pyclass] /// struct Foo { /// inner: u8, /// } /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let foo: Py = Py::new(py, Foo { inner: 73 })?; /// foo.borrow_mut(py).inner = 35; /// /// assert_eq!(foo.borrow(py).inner, 35); /// Ok(()) /// })?; /// # Ok(()) /// # } /// ``` /// /// # Panics /// Panics if the value is currently borrowed. For a non-panicking variant, use /// [`try_borrow_mut`](#method.try_borrow_mut). pub fn borrow_mut<'py>(&'py self, py: Python<'py>) -> PyRefMut<'py, T> where T: PyClass, { self.as_ref(py).borrow_mut() } /// Attempts to immutably borrow the value `T`, returning an error if the value is currently mutably borrowed. /// /// The borrow lasts while the returned [`PyRef`] exists. /// /// This is the non-panicking variant of [`borrow`](#method.borrow). /// /// For frozen classes, the simpler [`get`][Self::get] is available. /// /// Equivalent to `self.as_ref(py).borrow_mut()` - /// see [`PyCell::try_borrow`](crate::pycell::PyCell::try_borrow). pub fn try_borrow<'py>(&'py self, py: Python<'py>) -> Result, PyBorrowError> { self.as_ref(py).try_borrow() } /// Attempts to mutably borrow the value `T`, returning an error if the value is currently borrowed. /// /// The borrow lasts while the returned [`PyRefMut`] exists. /// /// This is the non-panicking variant of [`borrow_mut`](#method.borrow_mut). /// /// Equivalent to `self.as_ref(py).try_borrow_mut()` - /// see [`PyCell::try_borrow_mut`](crate::pycell::PyCell::try_borrow_mut). pub fn try_borrow_mut<'py>( &'py self, py: Python<'py>, ) -> Result, PyBorrowMutError> where T: PyClass, { self.as_ref(py).try_borrow_mut() } /// Provide an immutable borrow of the value `T` without acquiring the GIL. /// /// This is available if the class is [`frozen`][macro@crate::pyclass] and [`Sync`]. /// /// # Examples /// /// ``` /// use std::sync::atomic::{AtomicUsize, Ordering}; /// # use pyo3::prelude::*; /// /// #[pyclass(frozen)] /// struct FrozenCounter { /// value: AtomicUsize, /// } /// /// let cell = Python::with_gil(|py| { /// let counter = FrozenCounter { value: AtomicUsize::new(0) }; /// /// Py::new(py, counter).unwrap() /// }); /// /// cell.get().value.fetch_add(1, Ordering::Relaxed); /// ``` pub fn get(&self) -> &T where T: PyClass + Sync, { let any = self.as_ptr() as *const PyAny; // SAFETY: The class itself is frozen and `Sync` and we do not access anything but `cell.contents.value`. unsafe { let cell: &PyCell = PyNativeType::unchecked_downcast(&*any); &*cell.get_ptr() } } } impl Py { /// Returns whether `self` and `other` point to the same object. To compare /// the equality of two objects (the `==` operator), use [`eq`](PyAny::eq). /// /// This is equivalent to the Python expression `self is other`. #[inline] pub fn is(&self, o: &U) -> bool { self.as_ptr() == o.as_ptr() } /// Gets the reference count of the `ffi::PyObject` pointer. #[inline] pub fn get_refcnt(&self, _py: Python<'_>) -> isize { unsafe { ffi::Py_REFCNT(self.0.as_ptr()) } } /// Makes a clone of `self`. /// /// This creates another pointer to the same object, increasing its reference count. /// /// You should prefer using this method over [`Clone`] if you happen to be holding the GIL already. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyDict; /// /// # fn main() { /// Python::with_gil(|py| { /// let first: Py = PyDict::new(py).into(); /// let second = Py::clone_ref(&first, py); /// /// // Both point to the same object /// assert!(first.is(&second)); /// }); /// # } /// ``` #[inline] pub fn clone_ref(&self, py: Python<'_>) -> Py { unsafe { Py::from_borrowed_ptr(py, self.0.as_ptr()) } } /// Returns whether the object is considered to be None. /// /// This is equivalent to the Python expression `self is None`. pub fn is_none(&self, _py: Python<'_>) -> bool { unsafe { ffi::Py_None() == self.as_ptr() } } /// Returns whether the object is Ellipsis, e.g. `...`. /// /// This is equivalent to the Python expression `self is ...`. pub fn is_ellipsis(&self) -> bool { unsafe { ffi::Py_Ellipsis() == self.as_ptr() } } /// Returns whether the object is considered to be true. /// /// This is equivalent to the Python expression `bool(self)`. pub fn is_true(&self, py: Python<'_>) -> PyResult { let v = unsafe { ffi::PyObject_IsTrue(self.as_ptr()) }; err::error_on_minusone(py, v)?; Ok(v != 0) } /// Extracts some type from the Python object. /// /// This is a wrapper function around `FromPyObject::extract()`. pub fn extract<'p, D>(&'p self, py: Python<'p>) -> PyResult where D: FromPyObject<'p>, { FromPyObject::extract(unsafe { py.from_borrowed_ptr(self.as_ptr()) }) } /// Retrieves an attribute value. /// /// This is equivalent to the Python expression `self.attr_name`. /// /// If calling this method becomes performance-critical, the [`intern!`](crate::intern) macro /// can be used to intern `attr_name`, thereby avoiding repeated temporary allocations of /// Python strings. /// /// # Example: `intern!`ing the attribute name /// /// ``` /// # use pyo3::{intern, pyfunction, types::PyModule, IntoPy, Py, Python, PyObject, PyResult}; /// # /// #[pyfunction] /// fn version(sys: Py, py: Python<'_>) -> PyResult { /// sys.getattr(py, intern!(py, "version")) /// } /// # /// # Python::with_gil(|py| { /// # let sys = py.import("sys").unwrap().into_py(py); /// # version(sys, py).unwrap(); /// # }); /// ``` pub fn getattr(&self, py: Python<'_>, attr_name: N) -> PyResult where N: IntoPy>, { let attr_name = attr_name.into_py(py); unsafe { PyObject::from_owned_ptr_or_err( py, ffi::PyObject_GetAttr(self.as_ptr(), attr_name.as_ptr()), ) } } /// Sets an attribute value. /// /// This is equivalent to the Python expression `self.attr_name = value`. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`](crate::intern) /// macro can be used to intern `attr_name`. /// /// # Example: `intern!`ing the attribute name /// /// ``` /// # use pyo3::{intern, pyfunction, types::PyModule, IntoPy, PyObject, Python, PyResult}; /// # /// #[pyfunction] /// fn set_answer(ob: PyObject, py: Python<'_>) -> PyResult<()> { /// ob.setattr(py, intern!(py, "answer"), 42) /// } /// # /// # Python::with_gil(|py| { /// # let ob = PyModule::new(py, "empty").unwrap().into_py(py); /// # set_answer(ob, py).unwrap(); /// # }); /// ``` pub fn setattr(&self, py: Python<'_>, attr_name: N, value: V) -> PyResult<()> where N: IntoPy>, V: IntoPy>, { let attr_name = attr_name.into_py(py); let value = value.into_py(py); err::error_on_minusone(py, unsafe { ffi::PyObject_SetAttr(self.as_ptr(), attr_name.as_ptr(), value.as_ptr()) }) } /// Calls the object. /// /// This is equivalent to the Python expression `self(*args, **kwargs)`. pub fn call( &self, py: Python<'_>, args: impl IntoPy>, kwargs: Option<&PyDict>, ) -> PyResult { let args = args.into_py(py); let kwargs = kwargs.map_or(std::ptr::null_mut(), |p| p.into_ptr()); unsafe { let ret = PyObject::from_owned_ptr_or_err( py, ffi::PyObject_Call(self.as_ptr(), args.as_ptr(), kwargs), ); ffi::Py_XDECREF(kwargs); ret } } /// Calls the object with only positional arguments. /// /// This is equivalent to the Python expression `self(*args)`. pub fn call1(&self, py: Python<'_>, args: impl IntoPy>) -> PyResult { self.call(py, args, None) } /// Calls the object without arguments. /// /// This is equivalent to the Python expression `self()`. pub fn call0(&self, py: Python<'_>) -> PyResult { cfg_if::cfg_if! { if #[cfg(all( not(PyPy), any(Py_3_10, all(not(Py_LIMITED_API), Py_3_9)) // PyObject_CallNoArgs was added to python in 3.9 but to limited API in 3.10 ))] { // Optimized path on python 3.9+ unsafe { PyObject::from_owned_ptr_or_err(py, ffi::PyObject_CallNoArgs(self.as_ptr())) } } else { self.call(py, (), None) } } } /// Calls a method on the object. /// /// This is equivalent to the Python expression `self.name(*args, **kwargs)`. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`](crate::intern) /// macro can be used to intern `name`. pub fn call_method( &self, py: Python<'_>, name: N, args: A, kwargs: Option<&PyDict>, ) -> PyResult where N: IntoPy>, A: IntoPy>, { let callee = self.getattr(py, name)?; let args: Py = args.into_py(py); let kwargs = kwargs.map_or(std::ptr::null_mut(), |p| p.into_ptr()); unsafe { let result = PyObject::from_owned_ptr_or_err( py, ffi::PyObject_Call(callee.as_ptr(), args.as_ptr(), kwargs), ); ffi::Py_XDECREF(kwargs); result } } /// Calls a method on the object with only positional arguments. /// /// This is equivalent to the Python expression `self.name(*args)`. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`](crate::intern) /// macro can be used to intern `name`. pub fn call_method1(&self, py: Python<'_>, name: N, args: A) -> PyResult where N: IntoPy>, A: IntoPy>, { self.call_method(py, name, args, None) } /// Calls a method on the object with no arguments. /// /// This is equivalent to the Python expression `self.name()`. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`](crate::intern) /// macro can be used to intern `name`. pub fn call_method0(&self, py: Python<'_>, name: N) -> PyResult where N: IntoPy>, { cfg_if::cfg_if! { if #[cfg(all(Py_3_9, not(any(Py_LIMITED_API, PyPy))))] { // Optimized path on python 3.9+ unsafe { let name: Py = name.into_py(py); PyObject::from_owned_ptr_or_err(py, ffi::PyObject_CallMethodNoArgs(self.as_ptr(), name.as_ptr())) } } else { self.call_method(py, name, (), None) } } } /// Create a `Py` instance by taking ownership of the given FFI pointer. /// /// # Safety /// `ptr` must be a pointer to a Python object of type T. /// /// Callers must own the object referred to by `ptr`, as this function /// implicitly takes ownership of that object. /// /// # Panics /// Panics if `ptr` is null. #[inline] pub unsafe fn from_owned_ptr(py: Python<'_>, ptr: *mut ffi::PyObject) -> Py { match NonNull::new(ptr) { Some(nonnull_ptr) => Py(nonnull_ptr, PhantomData), None => crate::err::panic_after_error(py), } } /// Create a `Py` instance by taking ownership of the given FFI pointer. /// /// If `ptr` is null then the current Python exception is fetched as a [`PyErr`]. /// /// # Safety /// If non-null, `ptr` must be a pointer to a Python object of type T. #[inline] pub unsafe fn from_owned_ptr_or_err( py: Python<'_>, ptr: *mut ffi::PyObject, ) -> PyResult> { match NonNull::new(ptr) { Some(nonnull_ptr) => Ok(Py(nonnull_ptr, PhantomData)), None => Err(PyErr::fetch(py)), } } /// Create a `Py` instance by taking ownership of the given FFI pointer. /// /// If `ptr` is null then `None` is returned. /// /// # Safety /// If non-null, `ptr` must be a pointer to a Python object of type T. #[inline] pub unsafe fn from_owned_ptr_or_opt(_py: Python<'_>, ptr: *mut ffi::PyObject) -> Option { NonNull::new(ptr).map(|nonnull_ptr| Py(nonnull_ptr, PhantomData)) } /// Create a `Py` instance by creating a new reference from the given FFI pointer. /// /// # Safety /// `ptr` must be a pointer to a Python object of type T. /// /// # Panics /// Panics if `ptr` is null. #[inline] pub unsafe fn from_borrowed_ptr(py: Python<'_>, ptr: *mut ffi::PyObject) -> Py { match Self::from_borrowed_ptr_or_opt(py, ptr) { Some(slf) => slf, None => crate::err::panic_after_error(py), } } /// Create a `Py` instance by creating a new reference from the given FFI pointer. /// /// If `ptr` is null then the current Python exception is fetched as a `PyErr`. /// /// # Safety /// `ptr` must be a pointer to a Python object of type T. #[inline] pub unsafe fn from_borrowed_ptr_or_err( py: Python<'_>, ptr: *mut ffi::PyObject, ) -> PyResult { Self::from_borrowed_ptr_or_opt(py, ptr).ok_or_else(|| PyErr::fetch(py)) } /// Create a `Py` instance by creating a new reference from the given FFI pointer. /// /// If `ptr` is null then `None` is returned. /// /// # Safety /// `ptr` must be a pointer to a Python object of type T. #[inline] pub unsafe fn from_borrowed_ptr_or_opt( _py: Python<'_>, ptr: *mut ffi::PyObject, ) -> Option { NonNull::new(ptr).map(|nonnull_ptr| { ffi::Py_INCREF(ptr); Py(nonnull_ptr, PhantomData) }) } /// For internal conversions. /// /// # Safety /// `ptr` must point to a Python object of type T. #[inline] unsafe fn from_non_null(ptr: NonNull) -> Self { Self(ptr, PhantomData) } /// Returns the inner pointer without decreasing the refcount. #[inline] fn into_non_null(self) -> NonNull { let pointer = self.0; mem::forget(self); pointer } } impl ToPyObject for Py { /// Converts `Py` instance -> PyObject. fn to_object(&self, py: Python<'_>) -> PyObject { unsafe { PyObject::from_borrowed_ptr(py, self.as_ptr()) } } } impl IntoPy for Py { /// Converts a `Py` instance to `PyObject`. /// Consumes `self` without calling `Py_DECREF()`. #[inline] fn into_py(self, _py: Python<'_>) -> PyObject { unsafe { PyObject::from_non_null(self.into_non_null()) } } } impl IntoPy for &'_ Py { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { self.to_object(py) } } unsafe impl crate::AsPyPointer for Py { /// Gets the underlying FFI pointer, returns a borrowed pointer. #[inline] fn as_ptr(&self) -> *mut ffi::PyObject { self.0.as_ptr() } } impl std::convert::From<&'_ PyAny> for PyObject { fn from(obj: &PyAny) -> Self { unsafe { Py::from_borrowed_ptr(obj.py(), obj.as_ptr()) } } } impl std::convert::From<&'_ T> for PyObject where T: PyNativeType + AsRef, { fn from(obj: &T) -> Self { unsafe { Py::from_borrowed_ptr(obj.py(), obj.as_ref().as_ptr()) } } } impl std::convert::From> for PyObject where T: AsRef, { #[inline] fn from(other: Py) -> Self { unsafe { Self::from_non_null(other.into_non_null()) } } } // `&PyCell` can be converted to `Py` impl std::convert::From<&PyCell> for Py where T: PyClass, { fn from(cell: &PyCell) -> Self { unsafe { Py::from_borrowed_ptr(cell.py(), cell.as_ptr()) } } } impl<'a, T> std::convert::From> for Py where T: PyClass, { fn from(pyref: PyRef<'a, T>) -> Self { unsafe { Py::from_borrowed_ptr(pyref.py(), pyref.as_ptr()) } } } impl<'a, T> std::convert::From> for Py where T: PyClass, { fn from(pyref: PyRefMut<'a, T>) -> Self { unsafe { Py::from_borrowed_ptr(pyref.py(), pyref.as_ptr()) } } } /// If the GIL is held this increments `self`'s reference count. /// Otherwise this registers the [`Py`]`` instance to have its reference count /// incremented the next time PyO3 acquires the GIL. impl Clone for Py { fn clone(&self) -> Self { unsafe { gil::register_incref(self.0); } Self(self.0, PhantomData) } } /// Dropping a `Py` instance decrements the reference count on the object by 1. impl Drop for Py { fn drop(&mut self) { unsafe { gil::register_decref(self.0); } } } impl<'a, T> FromPyObject<'a> for Py where T: PyTypeInfo, &'a T::AsRefTarget: FromPyObject<'a>, T::AsRefTarget: 'a + AsPyPointer, { /// Extracts `Self` from the source `PyObject`. fn extract(ob: &'a PyAny) -> PyResult { unsafe { ob.extract::<&T::AsRefTarget>() .map(|val| Py::from_borrowed_ptr(ob.py(), val.as_ptr())) } } } /// `Py` can be used as an error when T is an Error. /// /// However for GIL lifetime reasons, cause() cannot be implemented for `Py`. /// Use .as_ref() to get the GIL-scoped error if you need to inspect the cause. impl std::error::Error for Py where T: std::error::Error + PyTypeInfo, T::AsRefTarget: std::fmt::Display, { } impl std::fmt::Display for Py where T: PyTypeInfo, T::AsRefTarget: std::fmt::Display, { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { Python::with_gil(|py| std::fmt::Display::fmt(self.as_ref(py), f)) } } impl std::fmt::Debug for Py { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { f.debug_tuple("Py").field(&self.0.as_ptr()).finish() } } /// A commonly-used alias for `Py`. /// /// This is an owned reference a Python object without any type information. This value can also be /// safely sent between threads. /// /// See the documentation for [`Py`](struct.Py.html). pub type PyObject = Py; impl PyObject { /// Downcast this `PyObject` to a concrete Python type or pyclass. /// /// Note that you can often avoid downcasting yourself by just specifying /// the desired type in function or method signatures. /// However, manual downcasting is sometimes necessary. /// /// For extracting a Rust-only type, see [`Py::extract`](struct.Py.html#method.extract). /// /// # Example: Downcasting to a specific Python object /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::{PyDict, PyList}; /// /// Python::with_gil(|py| { /// let any: PyObject = PyDict::new(py).into(); /// /// assert!(any.downcast::(py).is_ok()); /// assert!(any.downcast::(py).is_err()); /// }); /// ``` /// /// # Example: Getting a reference to a pyclass /// /// This is useful if you want to mutate a `PyObject` that /// might actually be a pyclass. /// /// ```rust /// # fn main() -> Result<(), pyo3::PyErr> { /// use pyo3::prelude::*; /// /// #[pyclass] /// struct Class { /// i: i32, /// } /// /// Python::with_gil(|py| { /// let class: PyObject = Py::new(py, Class { i: 0 }).unwrap().into_py(py); /// /// let class_cell: &PyCell = class.downcast(py)?; /// /// class_cell.borrow_mut().i += 1; /// /// // Alternatively you can get a `PyRefMut` directly /// let class_ref: PyRefMut<'_, Class> = class.extract(py)?; /// assert_eq!(class_ref.i, 1); /// Ok(()) /// }) /// # } /// ``` #[inline] pub fn downcast<'p, T>(&'p self, py: Python<'p>) -> Result<&T, PyDowncastError<'_>> where T: PyTryFrom<'p>, { >::try_from(self.as_ref(py)) } /// Casts the PyObject to a concrete Python object type without checking validity. /// /// # Safety /// /// Callers must ensure that the type is valid or risk type confusion. #[inline] pub unsafe fn downcast_unchecked<'p, T>(&'p self, py: Python<'p>) -> &T where T: PyTryFrom<'p>, { >::try_from_unchecked(self.as_ref(py)) } } #[cfg(test)] mod tests { use super::{Py, PyObject}; use crate::types::{PyDict, PyString}; use crate::{PyAny, PyResult, Python, ToPyObject}; #[test] fn test_call0() { Python::with_gil(|py| { let obj = py.get_type::().to_object(py); assert_eq!( obj.call0(py) .unwrap() .as_ref(py) .repr() .unwrap() .to_string_lossy(), "{}" ); }) } #[test] fn test_call_for_non_existing_method() { Python::with_gil(|py| { let obj: PyObject = PyDict::new(py).into(); assert!(obj.call_method0(py, "asdf").is_err()); assert!(obj .call_method(py, "nonexistent_method", (1,), None) .is_err()); assert!(obj.call_method0(py, "nonexistent_method").is_err()); assert!(obj.call_method1(py, "nonexistent_method", (1,)).is_err()); }); } #[test] fn py_from_dict() { let dict: Py = Python::with_gil(|py| { let native = PyDict::new(py); Py::from(native) }); assert_eq!(Python::with_gil(|py| dict.get_refcnt(py)), 1); } #[test] fn pyobject_from_py() { Python::with_gil(|py| { let dict: Py = PyDict::new(py).into(); let cnt = dict.get_refcnt(py); let p: PyObject = dict.into(); assert_eq!(p.get_refcnt(py), cnt); }); } #[test] fn attr() -> PyResult<()> { use crate::types::PyModule; Python::with_gil(|py| { const CODE: &str = r#" class A: pass a = A() "#; let module = PyModule::from_code(py, CODE, "", "")?; let instance: Py = module.getattr("a")?.into(); instance.getattr(py, "foo").unwrap_err(); instance.setattr(py, "foo", "bar")?; assert!(instance .getattr(py, "foo")? .as_ref(py) .eq(PyString::new(py, "bar"))?); instance.getattr(py, "foo")?; Ok(()) }) } #[test] fn pystring_attr() -> PyResult<()> { use crate::types::PyModule; Python::with_gil(|py| { const CODE: &str = r#" class A: pass a = A() "#; let module = PyModule::from_code(py, CODE, "", "")?; let instance: Py = module.getattr("a")?.into(); let foo = crate::intern!(py, "foo"); let bar = crate::intern!(py, "bar"); instance.getattr(py, foo).unwrap_err(); instance.setattr(py, foo, bar)?; assert!(instance.getattr(py, foo)?.as_ref(py).eq(bar)?); Ok(()) }) } #[test] fn invalid_attr() -> PyResult<()> { Python::with_gil(|py| { let instance: Py = py.eval("object()", None, None)?.into(); instance.getattr(py, "foo").unwrap_err(); // Cannot assign arbitrary attributes to `object` instance.setattr(py, "foo", "bar").unwrap_err(); Ok(()) }) } #[test] fn test_is_ellipsis() { Python::with_gil(|py| { let v = py .eval("...", None, None) .map_err(|e| e.display(py)) .unwrap() .to_object(py); assert!(v.is_ellipsis()); let not_ellipsis = 5.to_object(py); assert!(!not_ellipsis.is_ellipsis()); }); } #[cfg(feature = "macros")] mod using_macros { use super::*; #[crate::pyclass] #[pyo3(crate = "crate")] struct SomeClass(i32); #[test] fn instance_borrow_methods() { // More detailed tests of the underlying semantics in pycell.rs Python::with_gil(|py| { let instance = Py::new(py, SomeClass(0)).unwrap(); assert_eq!(instance.borrow(py).0, 0); assert_eq!(instance.try_borrow(py).unwrap().0, 0); assert_eq!(instance.borrow_mut(py).0, 0); assert_eq!(instance.try_borrow_mut(py).unwrap().0, 0); instance.borrow_mut(py).0 = 123; assert_eq!(instance.borrow(py).0, 123); assert_eq!(instance.try_borrow(py).unwrap().0, 123); assert_eq!(instance.borrow_mut(py).0, 123); assert_eq!(instance.try_borrow_mut(py).unwrap().0, 123); }) } } } pyo3-0.20.2/src/internal_tricks.rs000064400000000000000000000143611046102023000151340ustar 00000000000000use std::{ borrow::Cow, ffi::{CStr, CString}, }; use crate::{ exceptions::PyValueError, ffi::{Py_ssize_t, PY_SSIZE_T_MAX}, PyResult, }; pub struct PrivateMarker; macro_rules! private_decl { () => { /// This trait is private to implement; this method exists to make it /// impossible to implement outside the crate. fn __private__(&self) -> crate::internal_tricks::PrivateMarker; }; } macro_rules! private_impl { () => { fn __private__(&self) -> crate::internal_tricks::PrivateMarker { crate::internal_tricks::PrivateMarker } }; } macro_rules! pyo3_exception { ($doc: expr, $name: ident, $base: ty) => { #[doc = $doc] #[repr(transparent)] #[allow(non_camel_case_types)] pub struct $name($crate::PyAny); $crate::impl_exception_boilerplate!($name); $crate::create_exception_type_object!(pyo3_runtime, $name, $base, Some($doc)); }; } /// Convert an usize index into a Py_ssize_t index, clamping overflow to /// PY_SSIZE_T_MAX. pub(crate) fn get_ssize_index(index: usize) -> Py_ssize_t { index.min(PY_SSIZE_T_MAX as usize) as Py_ssize_t } /// Implementations used for slice indexing PySequence, PyTuple, and PyList macro_rules! index_impls { ( $ty:ty, $ty_name:literal, $len:expr, $get_slice:expr $(,)? ) => { impl std::ops::Index for $ty { // Always PyAny output (even if the slice operation returns something else) type Output = PyAny; #[track_caller] fn index(&self, index: usize) -> &Self::Output { self.get_item(index).unwrap_or_else(|_| { crate::internal_tricks::index_len_fail(index, $ty_name, $len(self)) }) } } impl std::ops::Index> for $ty { type Output = $ty; #[track_caller] fn index( &self, std::ops::Range { start, end }: std::ops::Range, ) -> &Self::Output { let len = $len(self); if start > len { crate::internal_tricks::slice_start_index_len_fail(start, $ty_name, len) } else if end > len { crate::internal_tricks::slice_end_index_len_fail(end, $ty_name, len) } else if start > end { crate::internal_tricks::slice_index_order_fail(start, end) } else { $get_slice(self, start, end) } } } impl std::ops::Index> for $ty { type Output = $ty; #[track_caller] fn index( &self, std::ops::RangeFrom { start }: std::ops::RangeFrom, ) -> &Self::Output { let len = $len(self); if start > len { crate::internal_tricks::slice_start_index_len_fail(start, $ty_name, len) } else { $get_slice(self, start, len) } } } impl std::ops::Index for $ty { type Output = $ty; #[track_caller] fn index(&self, _: std::ops::RangeFull) -> &Self::Output { let len = $len(self); $get_slice(self, 0, len) } } impl std::ops::Index> for $ty { type Output = $ty; #[track_caller] fn index(&self, range: std::ops::RangeInclusive) -> &Self::Output { let exclusive_end = range .end() .checked_add(1) .expect("range end exceeds Python limit"); &self[*range.start()..exclusive_end] } } impl std::ops::Index> for $ty { type Output = $ty; #[track_caller] fn index(&self, std::ops::RangeTo { end }: std::ops::RangeTo) -> &Self::Output { &self[0..end] } } impl std::ops::Index> for $ty { type Output = $ty; #[track_caller] fn index( &self, std::ops::RangeToInclusive { end }: std::ops::RangeToInclusive, ) -> &Self::Output { &self[0..=end] } } }; } // these error messages are shamelessly "borrowed" from std. #[inline(never)] #[cold] #[track_caller] pub(crate) fn index_len_fail(index: usize, ty_name: &str, len: usize) -> ! { panic!( "index {} out of range for {} of length {}", index, ty_name, len ); } #[inline(never)] #[cold] #[track_caller] pub(crate) fn slice_start_index_len_fail(index: usize, ty_name: &str, len: usize) -> ! { panic!( "range start index {} out of range for {} of length {}", index, ty_name, len ); } #[inline(never)] #[cold] #[track_caller] pub(crate) fn slice_end_index_len_fail(index: usize, ty_name: &str, len: usize) -> ! { panic!( "range end index {} out of range for {} of length {}", index, ty_name, len ); } #[inline(never)] #[cold] #[track_caller] pub(crate) fn slice_index_order_fail(index: usize, end: usize) -> ! { panic!("slice index starts at {} but ends at {}", index, end); } pub(crate) fn extract_c_string( src: &'static str, err_msg: &'static str, ) -> PyResult> { let bytes = src.as_bytes(); let cow = match bytes { [] => { // Empty string, we can trivially refer to a static "\0" string Cow::Borrowed(unsafe { CStr::from_bytes_with_nul_unchecked(b"\0") }) } [.., 0] => { // Last byte is a nul; try to create as a CStr let c_str = CStr::from_bytes_with_nul(bytes).map_err(|_| PyValueError::new_err(err_msg))?; Cow::Borrowed(c_str) } _ => { // Allocate a new CString for this let c_string = CString::new(bytes).map_err(|_| PyValueError::new_err(err_msg))?; Cow::Owned(c_string) } }; Ok(cow) } pyo3-0.20.2/src/lib.rs000064400000000000000000000523331046102023000125100ustar 00000000000000#![warn(missing_docs)] #![cfg_attr(feature = "nightly", feature(auto_traits, negative_impls))] #![cfg_attr(docsrs, feature(doc_cfg, doc_auto_cfg))] // Deny some lints in doctests. // Use `#[allow(...)]` locally to override. #![doc(test(attr( deny( rust_2018_idioms, unused_lifetimes, rust_2021_prelude_collisions, warnings ), allow(unused_variables, unused_assignments, unused_extern_crates) )))] //! Rust bindings to the Python interpreter. //! //! PyO3 can be used to write native Python modules or run Python code and modules from Rust. //! //! See [the guide] for a detailed introduction. //! //! # PyO3's object types //! //! PyO3 has several core types that you should familiarize yourself with: //! //! ## The Python<'py> object //! //! Holding the [global interpreter lock] (GIL) is modeled with the [`Python<'py>`](crate::Python) //! token. All APIs that require that the GIL is held require this token as proof that you really //! are holding the GIL. It can be explicitly acquired and is also implicitly acquired by PyO3 as //! it wraps Rust functions and structs into Python functions and objects. //! //! ## The GIL-dependent types //! //! For example `&`[`PyAny`]. These are only ever seen as references, with a lifetime that is only //! valid for as long as the GIL is held, which is why using them doesn't require a //! [`Python<'py>`](crate::Python) token. The underlying Python object, if mutable, can be mutated //! through any reference. //! //! See the [guide][types] for an explanation of the different Python object types. //! //! ## The GIL-independent types //! //! When wrapped in [`Py`]`<...>`, like with [`Py`]`<`[`PyAny`]`>` or [`Py`]``, Python //! objects no longer have a limited lifetime which makes them easier to store in structs and pass //! between functions. However, you cannot do much with them without a //! [`Python<'py>`](crate::Python) token, for which you’d need to reacquire the GIL. //! //! ## PyErr //! //! The vast majority of operations in this library will return [`PyResult<...>`](PyResult). //! This is an alias for the type `Result<..., PyErr>`. //! //! A `PyErr` represents a Python exception. A `PyErr` returned to Python code will be raised as a //! Python exception. Errors from `PyO3` itself are also exposed as Python exceptions. //! //! # Feature flags //! //! PyO3 uses [feature flags] to enable you to opt-in to additional functionality. For a detailed //! description, see the [Features chapter of the guide]. //! //! ## Default feature flags //! //! The following features are turned on by default: //! - `macros`: Enables various macros, including all the attribute macros. //! //! ## Optional feature flags //! //! The following features customize PyO3's behavior: //! //! - `abi3`: Restricts PyO3's API to a subset of the full Python API which is guaranteed by //! [PEP 384] to be forward-compatible with future Python versions. //! - `auto-initialize`: Changes [`Python::with_gil`] to automatically initialize the Python //! interpreter if needed. //! - `extension-module`: This will tell the linker to keep the Python symbols unresolved, so that //! your module can also be used with statically linked Python interpreters. Use this feature when //! building an extension module. //! - `multiple-pymethods`: Enables the use of multiple [`#[pymethods]`](macro@crate::pymethods) //! blocks per [`#[pyclass]`](macro@crate::pyclass). This adds a dependency on the [inventory] //! crate, which is not supported on all platforms. //! //! The following features enable interactions with other crates in the Rust ecosystem: //! - [`anyhow`]: Enables a conversion from [anyhow]’s [`Error`][anyhow_error] type to [`PyErr`]. //! - [`chrono`]: Enables a conversion from [chrono]'s structures to the equivalent Python ones. //! - [`either`]: Enables conversions between Python objects and [either]'s [`Either`] type. //! - [`eyre`]: Enables a conversion from [eyre]’s [`Report`] type to [`PyErr`]. //! - [`hashbrown`]: Enables conversions between Python objects and [hashbrown]'s [`HashMap`] and //! [`HashSet`] types. //! - [`indexmap`][indexmap_feature]: Enables conversions between Python dictionary and [indexmap]'s [`IndexMap`]. //! - [`num-bigint`]: Enables conversions between Python objects and [num-bigint]'s [`BigInt`] and //! [`BigUint`] types. //! - [`num-complex`]: Enables conversions between Python objects and [num-complex]'s [`Complex`] //! type. //! - [`rust_decimal`]: Enables conversions between Python's decimal.Decimal and [rust_decimal]'s //! [`Decimal`] type. //! - [`serde`]: Allows implementing [serde]'s [`Serialize`] and [`Deserialize`] traits for //! [`Py`]`` for all `T` that implement [`Serialize`] and [`Deserialize`]. //! - [`smallvec`][smallvec]: Enables conversions between Python list and [smallvec]'s [`SmallVec`]. //! //! ## Unstable features //! //! - `nightly`: Uses `#![feature(auto_traits, negative_impls)]` to define [`Ungil`] as an auto trait. // //! ## `rustc` environment flags //! //! PyO3 uses `rustc`'s `--cfg` flags to enable or disable code used for different Python versions. //! If you want to do this for your own crate, you can do so with the [`pyo3-build-config`] crate. //! //! - `Py_3_7`, `Py_3_8`, `Py_3_9`, `Py_3_10`: Marks code that is only enabled when //! compiling for a given minimum Python version. //! - `Py_LIMITED_API`: Marks code enabled when the `abi3` feature flag is enabled. //! - `PyPy` - Marks code enabled when compiling for PyPy. //! //! # Minimum supported Rust and Python versions //! //! PyO3 supports the following software versions: //! - Python 3.7 and up (CPython and PyPy) //! - Rust 1.56 and up //! //! # Example: Building a native Python module //! //! PyO3 can be used to generate a native Python module. The easiest way to try this out for the //! first time is to use [`maturin`]. `maturin` is a tool for building and publishing Rust-based //! Python packages with minimal configuration. The following steps set up some files for an example //! Python module, install `maturin`, and then show how to build and import the Python module. //! //! First, create a new folder (let's call it `string_sum`) containing the following two files: //! //! **`Cargo.toml`** //! //! ```toml //! [package] //! name = "string-sum" //! version = "0.1.0" //! edition = "2021" //! //! [lib] //! name = "string_sum" //! # "cdylib" is necessary to produce a shared library for Python to import from. //! # //! # Downstream Rust code (including code in `bin/`, `examples/`, and `tests/`) will not be able //! # to `use string_sum;` unless the "rlib" or "lib" crate type is also included, e.g.: //! # crate-type = ["cdylib", "rlib"] //! crate-type = ["cdylib"] //! //! [dependencies.pyo3] #![doc = concat!("version = \"", env!("CARGO_PKG_VERSION"), "\"")] //! features = ["extension-module"] //! ``` //! //! **`src/lib.rs`** //! ```rust //! use pyo3::prelude::*; //! //! /// Formats the sum of two numbers as string. //! #[pyfunction] //! fn sum_as_string(a: usize, b: usize) -> PyResult { //! Ok((a + b).to_string()) //! } //! //! /// A Python module implemented in Rust. //! #[pymodule] //! fn string_sum(py: Python<'_>, m: &PyModule) -> PyResult<()> { //! m.add_function(wrap_pyfunction!(sum_as_string, m)?)?; //! //! Ok(()) //! } //! ``` //! //! With those two files in place, now `maturin` needs to be installed. This can be done using //! Python's package manager `pip`. First, load up a new Python `virtualenv`, and install `maturin` //! into it: //! ```bash //! $ cd string_sum //! $ python -m venv .env //! $ source .env/bin/activate //! $ pip install maturin //! ``` //! //! Now build and execute the module: //! ```bash //! $ maturin develop //! # lots of progress output as maturin runs the compilation... //! $ python //! >>> import string_sum //! >>> string_sum.sum_as_string(5, 20) //! '25' //! ``` //! //! As well as with `maturin`, it is possible to build using [setuptools-rust] or //! [manually][manual_builds]. Both offer more flexibility than `maturin` but require further //! configuration. //! //! # Example: Using Python from Rust //! //! To embed Python into a Rust binary, you need to ensure that your Python installation contains a //! shared library. The following steps demonstrate how to ensure this (for Ubuntu), and then give //! some example code which runs an embedded Python interpreter. //! //! To install the Python shared library on Ubuntu: //! ```bash //! sudo apt install python3-dev //! ``` //! //! Start a new project with `cargo new` and add `pyo3` to the `Cargo.toml` like this: //! ```toml //! [dependencies.pyo3] #![doc = concat!("version = \"", env!("CARGO_PKG_VERSION"), "\"")] //! # this is necessary to automatically initialize the Python interpreter //! features = ["auto-initialize"] //! ``` //! //! Example program displaying the value of `sys.version` and the current user name: //! ```rust //! use pyo3::prelude::*; //! use pyo3::types::IntoPyDict; //! //! fn main() -> PyResult<()> { //! Python::with_gil(|py| { //! let sys = py.import("sys")?; //! let version: String = sys.getattr("version")?.extract()?; //! //! let locals = [("os", py.import("os")?)].into_py_dict(py); //! let code = "os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'"; //! let user: String = py.eval(code, None, Some(&locals))?.extract()?; //! //! println!("Hello {}, I'm Python {}", user, version); //! Ok(()) //! }) //! } //! ``` //! //! The guide has [a section][calling_rust] with lots of examples about this topic. //! //! # Other Examples //! //! The PyO3 [README](https://github.com/PyO3/pyo3#readme) contains quick-start examples for both //! using [Rust from Python] and [Python from Rust]. //! //! The PyO3 repository's [examples subdirectory] //! contains some basic packages to demonstrate usage of PyO3. //! //! There are many projects using PyO3 - see a list of some at //! . //! //! [anyhow]: https://docs.rs/anyhow/ "A trait object based error system for easy idiomatic error handling in Rust applications." //! [anyhow_error]: https://docs.rs/anyhow/latest/anyhow/struct.Error.html "Anyhows `Error` type, a wrapper around a dynamic error type" //! [`anyhow`]: ./anyhow/index.html "Documentation about the `anyhow` feature." //! [inventory]: https://docs.rs/inventory //! [`HashMap`]: https://docs.rs/hashbrown/latest/hashbrown/struct.HashMap.html //! [`HashSet`]: https://docs.rs/hashbrown/latest/hashbrown/struct.HashSet.html //! [`SmallVec`]: https://docs.rs/smallvec/latest/smallvec/struct.SmallVec.html //! [`IndexMap`]: https://docs.rs/indexmap/latest/indexmap/map/struct.IndexMap.html //! [`BigInt`]: https://docs.rs/num-bigint/latest/num_bigint/struct.BigInt.html //! [`BigUint`]: https://docs.rs/num-bigint/latest/num_bigint/struct.BigUint.html //! [`Complex`]: https://docs.rs/num-complex/latest/num_complex/struct.Complex.html //! [`Deserialize`]: https://docs.rs/serde/latest/serde/trait.Deserialize.html //! [`Serialize`]: https://docs.rs/serde/latest/serde/trait.Serialize.html //! [chrono]: https://docs.rs/chrono/ "Date and Time for Rust." //! [`chrono`]: ./chrono/index.html "Documentation about the `chrono` feature." //! [either]: https://docs.rs/either/ "A type that represents one of two alternatives." //! [`either`]: ./either/index.html "Documentation about the `either` feature." //! [`Either`]: https://docs.rs/either/latest/either/enum.Either.html //! [eyre]: https://docs.rs/eyre/ "A library for easy idiomatic error handling and reporting in Rust applications." //! [`Report`]: https://docs.rs/eyre/latest/eyre/struct.Report.html //! [`eyre`]: ./eyre/index.html "Documentation about the `eyre` feature." //! [`hashbrown`]: ./hashbrown/index.html "Documentation about the `hashbrown` feature." //! [indexmap_feature]: ./indexmap/index.html "Documentation about the `indexmap` feature." //! [`maturin`]: https://github.com/PyO3/maturin "Build and publish crates with pyo3, rust-cpython and cffi bindings as well as rust binaries as python packages" //! [`num-bigint`]: ./num_bigint/index.html "Documentation about the `num-bigint` feature." //! [`num-complex`]: ./num_complex/index.html "Documentation about the `num-complex` feature." //! [`pyo3-build-config`]: https://docs.rs/pyo3-build-config //! [rust_decimal]: https://docs.rs/rust_decimal //! [`rust_decimal`]: ./rust_decimal/index.html "Documenation about the `rust_decimal` feature." //! [`Decimal`]: https://docs.rs/rust_decimal/latest/rust_decimal/struct.Decimal.html //! [`serde`]: <./serde/index.html> "Documentation about the `serde` feature." //! [calling_rust]: https://pyo3.rs/latest/python_from_rust.html "Calling Python from Rust - PyO3 user guide" //! [examples subdirectory]: https://github.com/PyO3/pyo3/tree/main/examples //! [feature flags]: https://doc.rust-lang.org/cargo/reference/features.html "Features - The Cargo Book" //! [global interpreter lock]: https://docs.python.org/3/glossary.html#term-global-interpreter-lock //! [hashbrown]: https://docs.rs/hashbrown //! [smallvec]: https://docs.rs/smallvec //! [indexmap]: https://docs.rs/indexmap //! [manual_builds]: https://pyo3.rs/latest/building_and_distribution.html#manual-builds "Manual builds - Building and Distribution - PyO3 user guide" //! [num-bigint]: https://docs.rs/num-bigint //! [num-complex]: https://docs.rs/num-complex //! [serde]: https://docs.rs/serde //! [setuptools-rust]: https://github.com/PyO3/setuptools-rust "Setuptools plugin for Rust extensions" //! [the guide]: https://pyo3.rs "PyO3 user guide" //! [types]: https://pyo3.rs/latest/types.html "GIL lifetimes, mutability and Python object types" //! [PEP 384]: https://www.python.org/dev/peps/pep-0384 "PEP 384 -- Defining a Stable ABI" //! [Python from Rust]: https://github.com/PyO3/pyo3#using-python-from-rust //! [Rust from Python]: https://github.com/PyO3/pyo3#using-rust-from-python //! [Features chapter of the guide]: https://pyo3.rs/latest/features.html#features-reference "Features Reference - PyO3 user guide" //! [`Ungil`]: crate::marker::Ungil pub use crate::class::*; pub use crate::conversion::{ AsPyPointer, FromPyObject, FromPyPointer, IntoPy, PyTryFrom, PyTryInto, ToPyObject, }; pub use crate::err::{PyDowncastError, PyErr, PyErrArguments, PyResult}; pub use crate::gil::GILPool; #[cfg(not(PyPy))] pub use crate::gil::{prepare_freethreaded_python, with_embedded_python_interpreter}; pub use crate::instance::{Py, PyNativeType, PyObject}; pub use crate::marker::Python; pub use crate::pycell::{PyCell, PyRef, PyRefMut}; pub use crate::pyclass::PyClass; pub use crate::pyclass_init::PyClassInitializer; pub use crate::type_object::PyTypeInfo; pub use crate::types::PyAny; pub use crate::version::PythonVersionInfo; /// Old module which contained some implementation details of the `#[pyproto]` module. /// /// Prefer using the same content from `pyo3::pyclass`, e.g. `use pyo3::pyclass::CompareOp` instead /// of `use pyo3::class::basic::CompareOp`. /// /// For compatibility reasons this has not yet been removed, however will be done so /// once is resolved. pub mod class { #[doc(hidden)] pub use crate::impl_::pymethods as methods; pub use self::gc::{PyTraverseError, PyVisit}; #[doc(hidden)] pub use self::methods::{ PyClassAttributeDef, PyGetterDef, PyMethodDef, PyMethodDefType, PyMethodType, PySetterDef, }; /// Old module which contained some implementation details of the `#[pyproto]` module. /// /// Prefer using the same content from `pyo3::pyclass`, e.g. `use pyo3::pyclass::CompareOp` instead /// of `use pyo3::class::basic::CompareOp`. /// /// For compatibility reasons this has not yet been removed, however will be done so /// once is resolved. pub mod basic { pub use crate::pyclass::CompareOp; } /// Old module which contained some implementation details of the `#[pyproto]` module. /// /// Prefer using the same content from `pyo3::pyclass`, e.g. `use pyo3::pyclass::IterANextOutput` instead /// of `use pyo3::class::pyasync::IterANextOutput`. /// /// For compatibility reasons this has not yet been removed, however will be done so /// once is resolved. pub mod pyasync { pub use crate::pyclass::{IterANextOutput, PyIterANextOutput}; } /// Old module which contained some implementation details of the `#[pyproto]` module. /// /// Prefer using the same content from `pyo3::pyclass`, e.g. `use pyo3::pyclass::IterNextOutput` instead /// of `use pyo3::class::pyasync::IterNextOutput`. /// /// For compatibility reasons this has not yet been removed, however will be done so /// once is resolved. pub mod iter { pub use crate::pyclass::{IterNextOutput, PyIterNextOutput}; } /// Old module which contained some implementation details of the `#[pyproto]` module. /// /// Prefer using the same content from `pyo3::pyclass`, e.g. `use pyo3::pyclass::PyTraverseError` instead /// of `use pyo3::class::gc::PyTraverseError`. /// /// For compatibility reasons this has not yet been removed, however will be done so /// once is resolved. pub mod gc { pub use crate::pyclass::{PyTraverseError, PyVisit}; } } #[cfg(feature = "macros")] #[doc(hidden)] pub use { indoc, // Re-exported for py_run unindent, // Re-exported for py_run }; #[cfg(all(feature = "macros", feature = "multiple-pymethods"))] #[doc(hidden)] pub use inventory; // Re-exported for `#[pyclass]` and `#[pymethods]` with `multiple-pymethods`. /// Tests and helpers which reside inside PyO3's main library. Declared first so that macros /// are available in unit tests. #[cfg(test)] #[macro_use] mod tests; #[macro_use] mod internal_tricks; pub mod buffer; #[doc(hidden)] pub mod callback; pub mod conversion; mod conversions; #[macro_use] #[doc(hidden)] pub mod derive_utils; mod err; pub mod exceptions; pub mod ffi; mod gil; #[doc(hidden)] pub mod impl_; mod instance; pub mod marker; pub mod marshal; #[macro_use] pub mod sync; pub mod panic; pub mod prelude; pub mod pycell; pub mod pyclass; pub mod pyclass_init; pub mod type_object; pub mod types; mod version; #[doc(hidden)] #[deprecated(since = "0.19.0", note = "Please use the `sync` module instead.")] pub mod once_cell { // FIXME: We want to deprecate these, // but that does not yet work for re-exports, // c.f. https://github.com/rust-lang/rust/issues/30827 pub use crate::sync::{GILOnceCell, Interned}; } #[allow(unused_imports)] // with no features enabled this module has no public exports pub use crate::conversions::*; #[cfg(feature = "macros")] pub use pyo3_macros::{pyfunction, pymethods, pymodule, FromPyObject}; /// A proc macro used to expose Rust structs and fieldless enums as Python objects. /// #[doc = include_str!("../guide/pyclass_parameters.md")] /// /// For more on creating Python classes, /// see the [class section of the guide][1]. /// /// [1]: https://pyo3.rs/latest/class.html #[cfg(feature = "macros")] pub use pyo3_macros::pyclass; #[cfg(feature = "macros")] #[macro_use] mod macros; #[cfg(feature = "experimental-inspect")] pub mod inspect; /// Test readme and user guide #[cfg(doctest)] pub mod doc_test { macro_rules! doctests { ($($path:expr => $mod:ident),* $(,)?) => { $( #[doc = include_str!(concat!("../", $path))] mod $mod{} )* }; } doctests! { "README.md" => readme_md, "guide/src/advanced.md" => guide_advanced_md, "guide/src/building_and_distribution.md" => guide_building_and_distribution_md, "guide/src/building_and_distribution/multiple_python_versions.md" => guide_bnd_multiple_python_versions_md, "guide/src/class.md" => guide_class_md, "guide/src/class/call.md" => guide_class_call, "guide/src/class/object.md" => guide_class_object, "guide/src/class/numeric.md" => guide_class_numeric, "guide/src/class/protocols.md" => guide_class_protocols_md, "guide/src/conversions.md" => guide_conversions_md, "guide/src/conversions/tables.md" => guide_conversions_tables_md, "guide/src/conversions/traits.md" => guide_conversions_traits_md, "guide/src/debugging.md" => guide_debugging_md, // deliberate choice not to test guide/ecosystem because those pages depend on external // crates such as pyo3_asyncio. "guide/src/exception.md" => guide_exception_md, "guide/src/faq.md" => guide_faq_md, "guide/src/features.md" => guide_features_md, "guide/src/function.md" => guide_function_md, "guide/src/function/error_handling.md" => guide_function_error_handling_md, "guide/src/function/signature.md" => guide_function_signature_md, "guide/src/memory.md" => guide_memory_md, "guide/src/migration.md" => guide_migration_md, "guide/src/module.md" => guide_module_md, "guide/src/parallelism.md" => guide_parallelism_md, "guide/src/performance.md" => guide_performance_md, "guide/src/python_from_rust.md" => guide_python_from_rust_md, "guide/src/python_typing_hints.md" => guide_python_typing_hints_md, "guide/src/trait_bounds.md" => guide_trait_bounds_md, "guide/src/types.md" => guide_types_md, } } pyo3-0.20.2/src/macros.rs000064400000000000000000000127541046102023000132310ustar 00000000000000/// A convenient macro to execute a Python code snippet, with some local variables set. /// /// # Panics /// /// This macro internally calls [`Python::run`](crate::Python::run) and panics /// if it returns `Err`, after printing the error to stdout. /// /// If you need to handle failures, please use [`Python::run`](crate::marker::Python::run) instead. /// /// # Examples /// ``` /// use pyo3::{prelude::*, py_run, types::PyList}; /// /// Python::with_gil(|py| { /// let list = PyList::new(py, &[1, 2, 3]); /// py_run!(py, list, "assert list == [1, 2, 3]"); /// }); /// ``` /// /// You can use this macro to test pyfunctions or pyclasses quickly. /// /// ``` /// use pyo3::{prelude::*, py_run}; /// /// #[pyclass] /// #[derive(Debug)] /// struct Time { /// hour: u32, /// minute: u32, /// second: u32, /// } /// /// #[pymethods] /// impl Time { /// fn repl_japanese(&self) -> String { /// format!("{}時{}分{}秒", self.hour, self.minute, self.second) /// } /// #[getter] /// fn hour(&self) -> u32 { /// self.hour /// } /// fn as_tuple(&self) -> (u32, u32, u32) { /// (self.hour, self.minute, self.second) /// } /// } /// /// Python::with_gil(|py| { /// let time = PyCell::new(py, Time {hour: 8, minute: 43, second: 16}).unwrap(); /// let time_as_tuple = (8, 43, 16); /// py_run!(py, time time_as_tuple, r#" /// assert time.hour == 8 /// assert time.repl_japanese() == "8時43分16秒" /// assert time.as_tuple() == time_as_tuple /// "#); /// }); /// ``` /// /// If you need to prepare the `locals` dict by yourself, you can pass it as `*locals`. /// /// ``` /// use pyo3::prelude::*; /// use pyo3::types::IntoPyDict; /// /// #[pyclass] /// struct MyClass; /// /// #[pymethods] /// impl MyClass { /// #[new] /// fn new() -> Self { /// MyClass {} /// } /// } /// /// Python::with_gil(|py| { /// let locals = [("C", py.get_type::())].into_py_dict(py); /// pyo3::py_run!(py, *locals, "c = C()"); /// }); /// ``` #[macro_export] macro_rules! py_run { ($py:expr, $($val:ident)+, $code:literal) => {{ $crate::py_run_impl!($py, $($val)+, $crate::indoc::indoc!($code)) }}; ($py:expr, $($val:ident)+, $code:expr) => {{ $crate::py_run_impl!($py, $($val)+, &$crate::unindent::unindent($code)) }}; ($py:expr, *$dict:expr, $code:literal) => {{ $crate::py_run_impl!($py, *$dict, $crate::indoc::indoc!($code)) }}; ($py:expr, *$dict:expr, $code:expr) => {{ $crate::py_run_impl!($py, *$dict, &$crate::unindent::unindent($code)) }}; } #[macro_export] #[doc(hidden)] macro_rules! py_run_impl { ($py:expr, $($val:ident)+, $code:expr) => {{ use $crate::types::IntoPyDict; use $crate::ToPyObject; let d = [$((stringify!($val), $val.to_object($py)),)+].into_py_dict($py); $crate::py_run_impl!($py, *d, $code) }}; ($py:expr, *$dict:expr, $code:expr) => {{ use ::std::option::Option::*; if let ::std::result::Result::Err(e) = $py.run($code, None, Some($dict)) { e.print($py); // So when this c api function the last line called printed the error to stderr, // the output is only written into a buffer which is never flushed because we // panic before flushing. This is where this hack comes into place $py.run("import sys; sys.stderr.flush()", None, None) .unwrap(); ::std::panic!("{}", $code) } }}; } /// Wraps a Rust function annotated with [`#[pyfunction]`](macro@crate::pyfunction). /// /// This can be used with [`PyModule::add_function`](crate::types::PyModule::add_function) to add free /// functions to a [`PyModule`](crate::types::PyModule) - see its documentation for more information. #[macro_export] macro_rules! wrap_pyfunction { ($function:path) => { &|py_or_module| { use $function as wrapped_pyfunction; $crate::impl_::pyfunction::_wrap_pyfunction(&wrapped_pyfunction::DEF, py_or_module) } }; ($function:path, $py_or_module:expr) => {{ use $function as wrapped_pyfunction; $crate::impl_::pyfunction::_wrap_pyfunction(&wrapped_pyfunction::DEF, $py_or_module) }}; } /// Returns a function that takes a [`Python`](crate::Python) instance and returns a /// Python module. /// /// Use this together with [`#[pymodule]`](crate::pymodule) and /// [`PyModule::add_wrapped`](crate::types::PyModule::add_wrapped). #[macro_export] macro_rules! wrap_pymodule { ($module:path) => { &|py| { use $module as wrapped_pymodule; wrapped_pymodule::DEF .make_module(py) .expect("failed to wrap pymodule") } }; } /// Add the module to the initialization table in order to make embedded Python code to use it. /// Module name is the argument. /// /// Use it before [`prepare_freethreaded_python`](crate::prepare_freethreaded_python) and /// leave feature `auto-initialize` off #[cfg(not(PyPy))] #[macro_export] macro_rules! append_to_inittab { ($module:ident) => { unsafe { if $crate::ffi::Py_IsInitialized() != 0 { ::std::panic!( "called `append_to_inittab` but a Python interpreter is already running." ); } $crate::ffi::PyImport_AppendInittab( $module::NAME.as_ptr() as *const ::std::os::raw::c_char, ::std::option::Option::Some($module::init), ); } }; } pyo3-0.20.2/src/marker.rs000064400000000000000000001265711046102023000132310ustar 00000000000000//! Fundamental properties of objects tied to the Python interpreter. //! //! The Python interpreter is not threadsafe. To protect the Python interpreter in multithreaded //! scenarios there is a global lock, the *global interpreter lock* (hereafter referred to as *GIL*) //! that must be held to safely interact with Python objects. This is why in PyO3 when you acquire //! the GIL you get a [`Python`] marker token that carries the *lifetime* of holding the GIL and all //! borrowed references to Python objects carry this lifetime as well. This will statically ensure //! that you can never use Python objects after dropping the lock - if you mess this up it will be //! caught at compile time and your program will fail to compile. //! //! It also supports this pattern that many extension modules employ: //! - Drop the GIL, so that other Python threads can acquire it and make progress themselves //! - Do something independently of the Python interpreter, like IO, a long running calculation or //! awaiting a future //! - Once that is done, reacquire the GIL //! //! That API is provided by [`Python::allow_threads`] and enforced via the [`Ungil`] bound on the //! closure and the return type. This is done by relying on the [`Send`] auto trait. `Ungil` is //! defined as the following: //! //! ```rust //! pub unsafe trait Ungil {} //! //! unsafe impl Ungil for T {} //! ``` //! //! We piggy-back off the `Send` auto trait because it is not possible to implement custom auto //! traits on stable Rust. This is the solution which enables it for as many types as possible while //! making the API usable. //! //! In practice this API works quite well, but it comes with some drawbacks: //! //! ## Drawbacks //! //! There is no reason to prevent `!Send` types like [`Rc`] from crossing the closure. After all, //! [`Python::allow_threads`] just lets other Python threads run - it does not itself launch a new //! thread. //! //! ```rust, compile_fail //! # #[cfg(feature = "nightly")] //! # compile_error!("this actually works on nightly") //! use pyo3::prelude::*; //! use std::rc::Rc; //! //! fn main() { //! Python::with_gil(|py| { //! let rc = Rc::new(5); //! //! py.allow_threads(|| { //! // This would actually be fine... //! println!("{:?}", *rc); //! }); //! }); //! } //! ``` //! //! Because we are using `Send` for something it's not quite meant for, other code that //! (correctly) upholds the invariants of [`Send`] can cause problems. //! //! [`SendWrapper`] is one of those. Per its documentation: //! //! > A wrapper which allows you to move around non-Send-types between threads, as long as you //! > access the contained value only from within the original thread and make sure that it is //! > dropped from within the original thread. //! //! This will "work" to smuggle Python references across the closure, because we're not actually //! doing anything with threads: //! //! ```rust, no_run //! use pyo3::prelude::*; //! use pyo3::types::PyString; //! use send_wrapper::SendWrapper; //! //! Python::with_gil(|py| { //! let string = PyString::new(py, "foo"); //! //! let wrapped = SendWrapper::new(string); //! //! py.allow_threads(|| { //! # #[cfg(not(feature = "nightly"))] //! # { //! // 💥 Unsound! 💥 //! let smuggled: &PyString = *wrapped; //! println!("{:?}", smuggled); //! # } //! }); //! }); //! ``` //! //! For now the answer to that is "don't do that". //! //! # A proper implementation using an auto trait //! //! However on nightly Rust and when PyO3's `nightly` feature is //! enabled, `Ungil` is defined as the following: //! //! ```rust //! # #[cfg(FALSE)] //! # { //! #![feature(auto_traits, negative_impls)] //! //! pub unsafe auto trait Ungil {} //! //! // It is unimplemented for the `Python` struct and Python objects. //! impl !Ungil for Python<'_> {} //! impl !Ungil for ffi::PyObject {} //! //! // `Py` wraps it in a safe api, so this is OK //! unsafe impl Ungil for Py {} //! # } //! ``` //! //! With this feature enabled, the above two examples will start working and not working, respectively. //! //! [`SendWrapper`]: https://docs.rs/send_wrapper/latest/send_wrapper/struct.SendWrapper.html //! [`Rc`]: std::rc::Rc //! [`Py`]: crate::Py use crate::err::{self, PyDowncastError, PyErr, PyResult}; use crate::gil::{GILGuard, GILPool, SuspendGIL}; use crate::impl_::not_send::NotSend; use crate::types::{ PyAny, PyDict, PyEllipsis, PyModule, PyNone, PyNotImplemented, PyString, PyType, }; use crate::version::PythonVersionInfo; use crate::{ffi, FromPyPointer, IntoPy, Py, PyNativeType, PyObject, PyTryFrom, PyTypeInfo}; use std::ffi::{CStr, CString}; use std::marker::PhantomData; use std::os::raw::c_int; /// Types that are safe to access while the GIL is not held. /// /// # Safety /// /// The type must not carry borrowed Python references or, if it does, not allow access to them if /// the GIL is not held. /// /// See the [module-level documentation](self) for more information. /// /// # Examples /// /// This tracking is currently imprecise as it relies on the [`Send`] auto trait on stable Rust. /// For example, an `Rc` smart pointer should be usable without the GIL, but we currently prevent that: /// /// ```compile_fail /// # use pyo3::prelude::*; /// use std::rc::Rc; /// /// Python::with_gil(|py| { /// let rc = Rc::new(42); /// /// py.allow_threads(|| { /// println!("{:?}", rc); /// }); /// }); /// ``` /// /// This also implies that the interplay between `with_gil` and `allow_threads` is unsound, for example /// one can circumvent this protection using the [`send_wrapper`](https://docs.rs/send_wrapper/) crate: /// /// ```no_run /// # use pyo3::prelude::*; /// # use pyo3::types::PyString; /// use send_wrapper::SendWrapper; /// /// Python::with_gil(|py| { /// let string = PyString::new(py, "foo"); /// /// let wrapped = SendWrapper::new(string); /// /// py.allow_threads(|| { /// let sneaky: &PyString = *wrapped; /// /// println!("{:?}", sneaky); /// }); /// }); /// ``` /// /// Fixing this loophole on stable Rust has significant ergonomic issues, but it is fixed when using /// nightly Rust and the `nightly` feature, c.f. [#2141](https://github.com/PyO3/pyo3/issues/2141). #[cfg_attr(docsrs, doc(cfg(all())))] // Hide the cfg flag #[cfg(not(feature = "nightly"))] pub unsafe trait Ungil {} #[cfg_attr(docsrs, doc(cfg(all())))] // Hide the cfg flag #[cfg(not(feature = "nightly"))] unsafe impl Ungil for T {} #[cfg(feature = "nightly")] mod nightly { macro_rules! define { ($($tt:tt)*) => { $($tt)* } } define! { /// Types that are safe to access while the GIL is not held. /// /// # Safety /// /// The type must not carry borrowed Python references or, if it does, not allow access to them if /// the GIL is not held. /// /// See the [module-level documentation](self) for more information. /// /// # Examples /// /// Types which are `Ungil` cannot be used in contexts where the GIL was released, e.g. /// /// ```compile_fail /// # use pyo3::prelude::*; /// # use pyo3::types::PyString; /// Python::with_gil(|py| { /// let string = PyString::new(py, "foo"); /// /// py.allow_threads(|| { /// println!("{:?}", string); /// }); /// }); /// ``` /// /// This applies to the GIL token `Python` itself as well, e.g. /// /// ```compile_fail /// # use pyo3::prelude::*; /// Python::with_gil(|py| { /// py.allow_threads(|| { /// drop(py); /// }); /// }); /// ``` /// /// On nightly Rust, this is not based on the [`Send`] auto trait and hence we are able /// to prevent incorrectly circumventing it using e.g. the [`send_wrapper`](https://docs.rs/send_wrapper/) crate: /// /// ```compile_fail /// # use pyo3::prelude::*; /// # use pyo3::types::PyString; /// use send_wrapper::SendWrapper; /// /// Python::with_gil(|py| { /// let string = PyString::new(py, "foo"); /// /// let wrapped = SendWrapper::new(string); /// /// py.allow_threads(|| { /// let sneaky: &PyString = *wrapped; /// /// println!("{:?}", sneaky); /// }); /// }); /// ``` /// /// This also enables using non-[`Send`] types in `allow_threads`, /// at least if they are not also bound to the GIL: /// /// ```rust /// # use pyo3::prelude::*; /// use std::rc::Rc; /// /// Python::with_gil(|py| { /// let rc = Rc::new(42); /// /// py.allow_threads(|| { /// println!("{:?}", rc); /// }); /// }); /// ``` pub unsafe auto trait Ungil {} } impl !Ungil for crate::Python<'_> {} // This means that PyString, PyList, etc all inherit !Ungil from this. impl !Ungil for crate::PyAny {} // All the borrowing wrappers impl !Ungil for crate::PyCell {} impl !Ungil for crate::PyRef<'_, T> {} impl !Ungil for crate::PyRefMut<'_, T> {} // FFI pointees impl !Ungil for crate::ffi::PyObject {} impl !Ungil for crate::ffi::PyLongObject {} impl !Ungil for crate::ffi::PyThreadState {} impl !Ungil for crate::ffi::PyInterpreterState {} impl !Ungil for crate::ffi::PyWeakReference {} impl !Ungil for crate::ffi::PyFrameObject {} impl !Ungil for crate::ffi::PyCodeObject {} #[cfg(not(Py_LIMITED_API))] impl !Ungil for crate::ffi::PyDictKeysObject {} #[cfg(not(any(Py_LIMITED_API, Py_3_10)))] impl !Ungil for crate::ffi::PyArena {} } #[cfg(feature = "nightly")] pub use nightly::Ungil; /// A marker token that represents holding the GIL. /// /// It serves three main purposes: /// - It provides a global API for the Python interpreter, such as [`Python::eval`]. /// - It can be passed to functions that require a proof of holding the GIL, such as /// [`Py::clone_ref`]. /// - Its lifetime represents the scope of holding the GIL which can be used to create Rust /// references that are bound to it, such as `&`[`PyAny`]. /// /// Note that there are some caveats to using it that you might need to be aware of. See the /// [Deadlocks](#deadlocks) and [Releasing and freeing memory](#releasing-and-freeing-memory) /// paragraphs for more information about that. /// /// # Obtaining a Python token /// /// The following are the recommended ways to obtain a [`Python`] token, in order of preference: /// - In a function or method annotated with [`#[pyfunction]`](crate::pyfunction) or [`#[pymethods]`](crate::pymethods) you can declare it /// as a parameter, and PyO3 will pass in the token when Python code calls it. /// - If you already have something with a lifetime bound to the GIL, such as `&`[`PyAny`], you can /// use its [`.py()`][PyAny::py] method to get a token. /// - When you need to acquire the GIL yourself, such as when calling Python code from Rust, you /// should call [`Python::with_gil`] to do that and pass your code as a closure to it. /// /// # Deadlocks /// /// Note that the GIL can be temporarily released by the Python interpreter during a function call /// (e.g. importing a module). In general, you don't need to worry about this because the GIL is /// reacquired before returning to the Rust code: /// /// ```text /// `Python` exists |=====================================| /// GIL actually held |==========| |================| /// Rust code running |=======| |==| |======| /// ``` /// /// This behaviour can cause deadlocks when trying to lock a Rust mutex while holding the GIL: /// /// * Thread 1 acquires the GIL /// * Thread 1 locks a mutex /// * Thread 1 makes a call into the Python interpreter which releases the GIL /// * Thread 2 acquires the GIL /// * Thread 2 tries to locks the mutex, blocks /// * Thread 1's Python interpreter call blocks trying to reacquire the GIL held by thread 2 /// /// To avoid deadlocking, you should release the GIL before trying to lock a mutex or `await`ing in /// asynchronous code, e.g. with [`Python::allow_threads`]. /// /// # Releasing and freeing memory /// /// The [`Python`] type can be used to create references to variables owned by the Python /// interpreter, using functions such as [`Python::eval`] and [`PyModule::import`]. These /// references are tied to a [`GILPool`] whose references are not cleared until it is dropped. /// This can cause apparent "memory leaks" if it is kept around for a long time. /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyString; /// /// # fn main () -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// for _ in 0..10 { /// let hello: &PyString = py.eval("\"Hello World!\"", None, None)?.extract()?; /// println!("Python says: {}", hello.to_str()?); /// // Normally variables in a loop scope are dropped here, but `hello` is a reference to /// // something owned by the Python interpreter. Dropping this reference does nothing. /// } /// Ok(()) /// }) /// // This is where the `hello`'s reference counts start getting decremented. /// # } /// ``` /// /// The variable `hello` is dropped at the end of each loop iteration, but the lifetime of the /// pointed-to memory is bound to [`Python::with_gil`]'s [`GILPool`] which will not be dropped until /// the end of [`Python::with_gil`]'s scope. Only then is each `hello`'s Python reference count /// decreased. This means that at the last line of the example there are 10 copies of `hello` in /// Python's memory, not just one at a time as we might expect from Rust's [scoping rules]. /// /// See the [Memory Management] chapter of the guide for more information about how PyO3 uses /// [`GILPool`] to manage memory. /// /// [scoping rules]: https://doc.rust-lang.org/stable/book/ch04-01-what-is-ownership.html#ownership-rules /// [`Py::clone_ref`]: crate::Py::clone_ref /// [Memory Management]: https://pyo3.rs/main/memory.html#gil-bound-memory #[derive(Copy, Clone)] pub struct Python<'py>(PhantomData<(&'py GILGuard, NotSend)>); impl Python<'_> { /// Acquires the global interpreter lock, allowing access to the Python interpreter. The /// provided closure `F` will be executed with the acquired `Python` marker token. /// /// If implementing [`#[pymethods]`](crate::pymethods) or [`#[pyfunction]`](crate::pyfunction), /// declare `py: Python` as an argument. PyO3 will pass in the token to grant access to the GIL /// context in which the function is running, avoiding the need to call `with_gil`. /// /// If the [`auto-initialize`] feature is enabled and the Python runtime is not already /// initialized, this function will initialize it. See #[cfg_attr( not(PyPy), doc = "[`prepare_freethreaded_python`](crate::prepare_freethreaded_python)" )] #[cfg_attr(PyPy, doc = "`prepare_freethreaded_python`")] /// for details. /// /// If the current thread does not yet have a Python "thread state" associated with it, /// a new one will be automatically created before `F` is executed and destroyed after `F` /// completes. /// /// # Panics /// /// - If the [`auto-initialize`] feature is not enabled and the Python interpreter is not /// initialized. /// /// # Examples /// /// ``` /// use pyo3::prelude::*; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let x: i32 = py.eval("5", None, None)?.extract()?; /// assert_eq!(x, 5); /// Ok(()) /// }) /// # } /// ``` /// /// [`auto-initialize`]: https://pyo3.rs/main/features.html#auto-initialize #[inline] pub fn with_gil(f: F) -> R where F: for<'py> FnOnce(Python<'py>) -> R, { let _guard = GILGuard::acquire(); // SAFETY: Either the GIL was already acquired or we just created a new `GILGuard`. f(unsafe { Python::assume_gil_acquired() }) } /// Like [`Python::with_gil`] except Python interpreter state checking is skipped. /// /// Normally when the GIL is acquired, we check that the Python interpreter is an /// appropriate state (e.g. it is fully initialized). This function skips those /// checks. /// /// # Safety /// /// If [`Python::with_gil`] would succeed, it is safe to call this function. /// /// In most cases, you should use [`Python::with_gil`]. /// /// A justified scenario for calling this function is during multi-phase interpreter /// initialization when [`Python::with_gil`] would fail before // this link is only valid on 3.8+not pypy and up. #[cfg_attr( all(Py_3_8, not(PyPy)), doc = "[`_Py_InitializeMain`](crate::ffi::_Py_InitializeMain)" )] #[cfg_attr(any(not(Py_3_8), PyPy), doc = "`_Py_InitializeMain`")] /// is called because the interpreter is only partially initialized. /// /// Behavior in other scenarios is not documented. #[inline] pub unsafe fn with_gil_unchecked(f: F) -> R where F: for<'py> FnOnce(Python<'py>) -> R, { let _guard = GILGuard::acquire_unchecked(); // SAFETY: Either the GIL was already acquired or we just created a new `GILGuard`. f(Python::assume_gil_acquired()) } } impl<'py> Python<'py> { /// Temporarily releases the GIL, thus allowing other Python threads to run. The GIL will be /// reacquired when `F`'s scope ends. /// /// If you don't need to touch the Python /// interpreter for some time and have other Python threads around, this will let you run /// Rust-only code while letting those other Python threads make progress. /// /// Only types that implement [`Ungil`] can cross the closure. See the /// [module level documentation](self) for more information. /// /// If you need to pass Python objects into the closure you can use [`Py`]``to create a /// reference independent of the GIL lifetime. However, you cannot do much with those without a /// [`Python`] token, for which you'd need to reacquire the GIL. /// /// # Example: Releasing the GIL while running a computation in Rust-only code /// /// ``` /// use pyo3::prelude::*; /// /// #[pyfunction] /// fn sum_numbers(py: Python<'_>, numbers: Vec) -> PyResult { /// // We release the GIL here so any other Python threads get a chance to run. /// py.allow_threads(move || { /// // An example of an "expensive" Rust calculation /// let sum = numbers.iter().sum(); /// /// Ok(sum) /// }) /// } /// # /// # fn main() -> PyResult<()> { /// # Python::with_gil(|py| -> PyResult<()> { /// # let fun = pyo3::wrap_pyfunction!(sum_numbers, py)?; /// # let res = fun.call1((vec![1_u32, 2, 3],))?; /// # assert_eq!(res.extract::()?, 6_u32); /// # Ok(()) /// # }) /// # } /// ``` /// /// Please see the [Parallelism] chapter of the guide for a thorough discussion of using /// [`Python::allow_threads`] in this manner. /// /// # Example: Passing borrowed Python references into the closure is not allowed /// /// ```compile_fail /// use pyo3::prelude::*; /// use pyo3::types::PyString; /// /// fn parallel_print(py: Python<'_>) { /// let s = PyString::new(py, "This object cannot be accessed without holding the GIL >_<"); /// py.allow_threads(move || { /// println!("{:?}", s); // This causes a compile error. /// }); /// } /// ``` /// /// [`Py`]: crate::Py /// [`PyString`]: crate::types::PyString /// [auto-traits]: https://doc.rust-lang.org/nightly/unstable-book/language-features/auto-traits.html /// [Parallelism]: https://pyo3.rs/main/parallelism.html pub fn allow_threads(self, f: F) -> T where F: Ungil + FnOnce() -> T, T: Ungil, { // Use a guard pattern to handle reacquiring the GIL, // so that the GIL will be reacquired even if `f` panics. // The `Send` bound on the closure prevents the user from // transferring the `Python` token into the closure. let _guard = unsafe { SuspendGIL::new() }; f() } /// Evaluates a Python expression in the given context and returns the result. /// /// If `globals` is `None`, it defaults to Python module `__main__`. /// If `locals` is `None`, it defaults to the value of `globals`. /// /// If `globals` doesn't contain `__builtins__`, default `__builtins__` /// will be added automatically. /// /// # Examples /// /// ``` /// # use pyo3::prelude::*; /// # Python::with_gil(|py| { /// let result = py.eval("[i * 10 for i in range(5)]", None, None).unwrap(); /// let res: Vec = result.extract().unwrap(); /// assert_eq!(res, vec![0, 10, 20, 30, 40]) /// # }); /// ``` pub fn eval( self, code: &str, globals: Option<&PyDict>, locals: Option<&PyDict>, ) -> PyResult<&'py PyAny> { self.run_code(code, ffi::Py_eval_input, globals, locals) } /// Executes one or more Python statements in the given context. /// /// If `globals` is `None`, it defaults to Python module `__main__`. /// If `locals` is `None`, it defaults to the value of `globals`. /// /// If `globals` doesn't contain `__builtins__`, default `__builtins__` /// will be added automatically. /// /// # Examples /// ``` /// use pyo3::{ /// prelude::*, /// types::{PyBytes, PyDict}, /// }; /// Python::with_gil(|py| { /// let locals = PyDict::new(py); /// py.run( /// r#" /// import base64 /// s = 'Hello Rust!' /// ret = base64.b64encode(s.encode('utf-8')) /// "#, /// None, /// Some(locals), /// ) /// .unwrap(); /// let ret = locals.get_item("ret").unwrap().unwrap(); /// let b64: &PyBytes = ret.downcast().unwrap(); /// assert_eq!(b64.as_bytes(), b"SGVsbG8gUnVzdCE="); /// }); /// ``` /// /// You can use [`py_run!`](macro.py_run.html) for a handy alternative of `run` /// if you don't need `globals` and unwrapping is OK. pub fn run( self, code: &str, globals: Option<&PyDict>, locals: Option<&PyDict>, ) -> PyResult<()> { let res = self.run_code(code, ffi::Py_file_input, globals, locals); res.map(|obj| { debug_assert!(obj.is_none()); }) } /// Runs code in the given context. /// /// `start` indicates the type of input expected: one of `Py_single_input`, /// `Py_file_input`, or `Py_eval_input`. /// /// If `globals` is `None`, it defaults to Python module `__main__`. /// If `locals` is `None`, it defaults to the value of `globals`. fn run_code( self, code: &str, start: c_int, globals: Option<&PyDict>, locals: Option<&PyDict>, ) -> PyResult<&'py PyAny> { let code = CString::new(code)?; unsafe { let mptr = ffi::PyImport_AddModule("__main__\0".as_ptr() as *const _); if mptr.is_null() { return Err(PyErr::fetch(self)); } let globals = globals .map(|dict| dict.as_ptr()) .unwrap_or_else(|| ffi::PyModule_GetDict(mptr)); let locals = locals.map(|dict| dict.as_ptr()).unwrap_or(globals); // If `globals` don't provide `__builtins__`, most of the code will fail if Python // version is <3.10. That's probably not what user intended, so insert `__builtins__` // for them. // // See also: // - https://github.com/python/cpython/pull/24564 (the same fix in CPython 3.10) // - https://github.com/PyO3/pyo3/issues/3370 let builtins_s = crate::intern!(self, "__builtins__").as_ptr(); let has_builtins = ffi::PyDict_Contains(globals, builtins_s); if has_builtins == -1 { return Err(PyErr::fetch(self)); } if has_builtins == 0 { // Inherit current builtins. let builtins = ffi::PyEval_GetBuiltins(); // `PyDict_SetItem` doesn't take ownership of `builtins`, but `PyEval_GetBuiltins` // seems to return a borrowed reference, so no leak here. if ffi::PyDict_SetItem(globals, builtins_s, builtins) == -1 { return Err(PyErr::fetch(self)); } } let code_obj = ffi::Py_CompileString(code.as_ptr(), "\0".as_ptr() as _, start); if code_obj.is_null() { return Err(PyErr::fetch(self)); } let res_ptr = ffi::PyEval_EvalCode(code_obj, globals, locals); ffi::Py_DECREF(code_obj); self.from_owned_ptr_or_err(res_ptr) } } /// Gets the Python type object for type `T`. #[inline] pub fn get_type(self) -> &'py PyType where T: PyTypeInfo, { T::type_object(self) } /// Imports the Python module with the specified name. pub fn import(self, name: N) -> PyResult<&'py PyModule> where N: IntoPy>, { PyModule::import(self, name) } /// Gets the Python builtin value `None`. #[allow(non_snake_case)] // the Python keyword starts with uppercase #[inline] pub fn None(self) -> PyObject { PyNone::get(self).into() } /// Gets the Python builtin value `Ellipsis`, or `...`. #[allow(non_snake_case)] // the Python keyword starts with uppercase #[inline] pub fn Ellipsis(self) -> PyObject { PyEllipsis::get(self).into() } /// Gets the Python builtin value `NotImplemented`. #[allow(non_snake_case)] // the Python keyword starts with uppercase #[inline] pub fn NotImplemented(self) -> PyObject { PyNotImplemented::get(self).into() } /// Gets the running Python interpreter version as a string. /// /// # Examples /// ```rust /// # use pyo3::Python; /// Python::with_gil(|py| { /// // The full string could be, for example: /// // "3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)]" /// assert!(py.version().starts_with("3.")); /// }); /// ``` pub fn version(self) -> &'py str { unsafe { CStr::from_ptr(ffi::Py_GetVersion()) .to_str() .expect("Python version string not UTF-8") } } /// Gets the running Python interpreter version as a struct similar to /// `sys.version_info`. /// /// # Examples /// ```rust /// # use pyo3::Python; /// Python::with_gil(|py| { /// // PyO3 supports Python 3.7 and up. /// assert!(py.version_info() >= (3, 7)); /// assert!(py.version_info() >= (3, 7, 0)); /// }); /// ``` pub fn version_info(self) -> PythonVersionInfo<'py> { let version_str = self.version(); // Portion of the version string returned by Py_GetVersion up to the first space is the // version number. let version_number_str = version_str.split(' ').next().unwrap_or(version_str); PythonVersionInfo::from_str(version_number_str).unwrap() } /// Registers the object in the release pool, and tries to downcast to specific type. pub fn checked_cast_as(self, obj: PyObject) -> Result<&'py T, PyDowncastError<'py>> where T: PyTryFrom<'py>, { let any: &PyAny = unsafe { self.from_owned_ptr(obj.into_ptr()) }; ::try_from(any) } /// Registers the object in the release pool, and does an unchecked downcast /// to the specific type. /// /// # Safety /// /// Callers must ensure that ensure that the cast is valid. pub unsafe fn cast_as(self, obj: PyObject) -> &'py T where T: PyNativeType + PyTypeInfo, { let any: &PyAny = self.from_owned_ptr(obj.into_ptr()); T::unchecked_downcast(any) } /// Registers the object pointer in the release pool, /// and does an unchecked downcast to the specific type. /// /// # Safety /// /// Callers must ensure that ensure that the cast is valid. #[allow(clippy::wrong_self_convention)] pub unsafe fn from_owned_ptr(self, ptr: *mut ffi::PyObject) -> &'py T where T: FromPyPointer<'py>, { FromPyPointer::from_owned_ptr(self, ptr) } /// Registers the owned object pointer in the release pool. /// /// Returns `Err(PyErr)` if the pointer is NULL. /// Does an unchecked downcast to the specific type. /// /// # Safety /// /// Callers must ensure that ensure that the cast is valid. #[allow(clippy::wrong_self_convention)] pub unsafe fn from_owned_ptr_or_err(self, ptr: *mut ffi::PyObject) -> PyResult<&'py T> where T: FromPyPointer<'py>, { FromPyPointer::from_owned_ptr_or_err(self, ptr) } /// Registers the owned object pointer in release pool. /// /// Returns `None` if the pointer is NULL. /// Does an unchecked downcast to the specific type. /// /// # Safety /// /// Callers must ensure that ensure that the cast is valid. #[allow(clippy::wrong_self_convention)] pub unsafe fn from_owned_ptr_or_opt(self, ptr: *mut ffi::PyObject) -> Option<&'py T> where T: FromPyPointer<'py>, { FromPyPointer::from_owned_ptr_or_opt(self, ptr) } /// Does an unchecked downcast to the specific type. /// /// Panics if the pointer is NULL. /// /// # Safety /// /// Callers must ensure that ensure that the cast is valid. #[allow(clippy::wrong_self_convention)] pub unsafe fn from_borrowed_ptr(self, ptr: *mut ffi::PyObject) -> &'py T where T: FromPyPointer<'py>, { FromPyPointer::from_borrowed_ptr(self, ptr) } /// Does an unchecked downcast to the specific type. /// /// Returns `Err(PyErr)` if the pointer is NULL. /// /// # Safety /// /// Callers must ensure that ensure that the cast is valid. #[allow(clippy::wrong_self_convention)] pub unsafe fn from_borrowed_ptr_or_err(self, ptr: *mut ffi::PyObject) -> PyResult<&'py T> where T: FromPyPointer<'py>, { FromPyPointer::from_borrowed_ptr_or_err(self, ptr) } /// Does an unchecked downcast to the specific type. /// /// Returns `None` if the pointer is NULL. /// /// # Safety /// /// Callers must ensure that ensure that the cast is valid. #[allow(clippy::wrong_self_convention)] pub unsafe fn from_borrowed_ptr_or_opt(self, ptr: *mut ffi::PyObject) -> Option<&'py T> where T: FromPyPointer<'py>, { FromPyPointer::from_borrowed_ptr_or_opt(self, ptr) } /// Lets the Python interpreter check and handle any pending signals. This will invoke the /// corresponding signal handlers registered in Python (if any). /// /// Returns `Err(`[`PyErr`]`)` if any signal handler raises an exception. /// /// These signals include `SIGINT` (normally raised by CTRL + C), which by default raises /// `KeyboardInterrupt`. For this reason it is good practice to call this function regularly /// as part of long-running Rust functions so that users can cancel it. /// /// # Example /// /// ```rust /// # #![allow(dead_code)] // this example is quite impractical to test /// use pyo3::prelude::*; /// /// # fn main() { /// #[pyfunction] /// fn loop_forever(py: Python<'_>) -> PyResult<()> { /// loop { /// // As this loop is infinite it should check for signals every once in a while. /// // Using `?` causes any `PyErr` (potentially containing `KeyboardInterrupt`) /// // to break out of the loop. /// py.check_signals()?; /// /// // do work here /// # break Ok(()) // don't actually loop forever /// } /// } /// # } /// ``` /// /// # Note /// /// This function calls [`PyErr_CheckSignals()`][1] which in turn may call signal handlers. /// As Python's [`signal`][2] API allows users to define custom signal handlers, calling this /// function allows arbitrary Python code inside signal handlers to run. /// /// If the function is called from a non-main thread, or under a non-main Python interpreter, /// it does nothing yet still returns `Ok(())`. /// /// [1]: https://docs.python.org/3/c-api/exceptions.html?highlight=pyerr_checksignals#c.PyErr_CheckSignals /// [2]: https://docs.python.org/3/library/signal.html pub fn check_signals(self) -> PyResult<()> { err::error_on_minusone(self, unsafe { ffi::PyErr_CheckSignals() }) } /// Create a new pool for managing PyO3's owned references. /// /// When this `GILPool` is dropped, all PyO3 owned references created after this `GILPool` will /// all have their Python reference counts decremented, potentially allowing Python to drop /// the corresponding Python objects. /// /// Typical usage of PyO3 will not need this API, as [`Python::with_gil`] automatically creates /// a `GILPool` where appropriate. /// /// Advanced uses of PyO3 which perform long-running tasks which never free the GIL may need /// to use this API to clear memory, as PyO3 usually does not clear memory until the GIL is /// released. /// /// # Examples /// /// ```rust /// # use pyo3::prelude::*; /// Python::with_gil(|py| { /// // Some long-running process like a webserver, which never releases the GIL. /// loop { /// // Create a new pool, so that PyO3 can clear memory at the end of the loop. /// let pool = unsafe { py.new_pool() }; /// /// // It is recommended to *always* immediately set py to the pool's Python, to help /// // avoid creating references with invalid lifetimes. /// let py = pool.python(); /// /// // do stuff... /// # break; // Exit the loop so that doctest terminates! /// } /// }); /// ``` /// /// # Safety /// /// Extreme care must be taken when using this API, as misuse can lead to accessing invalid /// memory. In addition, the caller is responsible for guaranteeing that the GIL remains held /// for the entire lifetime of the returned `GILPool`. /// /// Two best practices are required when using this API: /// - From the moment `new_pool()` is called, only the `Python` token from the returned /// `GILPool` (accessible using [`.python()`]) should be used in PyO3 APIs. All other older /// `Python` tokens with longer lifetimes are unsafe to use until the `GILPool` is dropped, /// because they can be used to create PyO3 owned references which have lifetimes which /// outlive the `GILPool`. /// - Similarly, methods on existing owned references will implicitly refer back to the /// `Python` token which that reference was originally created with. If the returned values /// from these methods are owned references they will inherit the same lifetime. As a result, /// Rust's lifetime rules may allow them to outlive the `GILPool`, even though this is not /// safe for reasons discussed above. Care must be taken to never access these return values /// after the `GILPool` is dropped, unless they are converted to `Py` *before* the pool /// is dropped. /// /// [`.python()`]: crate::GILPool::python #[inline] pub unsafe fn new_pool(self) -> GILPool { GILPool::new() } } impl Python<'_> { /// Creates a scope using a new pool for managing PyO3's owned references. /// /// This is a safe alterantive to [`new_pool`][Self::new_pool] as /// it limits the closure to using the new GIL token at the cost of /// being unable to capture existing GIL-bound references. /// /// Note that on stable Rust, this API suffers from the same the `SendWrapper` loophole /// as [`allow_threads`][Self::allow_threads], c.f. the documentation of the [`Ungil`] trait, /// /// # Examples /// /// ```rust /// # use pyo3::prelude::*; /// Python::with_gil(|py| { /// // Some long-running process like a webserver, which never releases the GIL. /// loop { /// // Create a new scope, so that PyO3 can clear memory at the end of the loop. /// py.with_pool(|py| { /// // do stuff... /// }); /// # break; // Exit the loop so that doctest terminates! /// } /// }); /// ``` /// /// The `Ungil` bound on the closure does prevent hanging on to existing GIL-bound references /// /// ```compile_fail /// # use pyo3::prelude::*; /// # use pyo3::types::PyString; /// /// Python::with_gil(|py| { /// let old_str = PyString::new(py, "a message from the past"); /// /// py.with_pool(|_py| { /// print!("{:?}", old_str); /// }); /// }); /// ``` /// /// or continuing to use the old GIL token /// /// ```compile_fail /// # use pyo3::prelude::*; /// /// Python::with_gil(|old_py| { /// old_py.with_pool(|_new_py| { /// let _none = old_py.None(); /// }); /// }); /// ``` #[inline] pub fn with_pool(&self, f: F) -> R where F: for<'py> FnOnce(Python<'py>) -> R + Ungil, { // SAFETY: The closure is `Ungil`, // i.e. it does not capture any GIL-bound references // and accesses only the newly created GIL token. let pool = unsafe { GILPool::new() }; f(pool.python()) } } impl<'unbound> Python<'unbound> { /// Unsafely creates a Python token with an unbounded lifetime. /// /// Many of PyO3 APIs use `Python<'_>` as proof that the GIL is held, but this function can be /// used to call them unsafely. /// /// # Safety /// /// - This token and any borrowed Python references derived from it can only be safely used /// whilst the currently executing thread is actually holding the GIL. /// - This function creates a token with an *unbounded* lifetime. Safe code can assume that /// holding a `Python<'py>` token means the GIL is and stays acquired for the lifetime `'py`. /// If you let it or borrowed Python references escape to safe code you are /// responsible for bounding the lifetime `'unbound` appropriately. For more on unbounded /// lifetimes, see the [nomicon]. /// /// [nomicon]: https://doc.rust-lang.org/nomicon/unbounded-lifetimes.html #[inline] pub unsafe fn assume_gil_acquired() -> Python<'unbound> { Python(PhantomData) } } #[cfg(test)] mod tests { use super::*; use crate::types::{IntoPyDict, PyDict, PyList}; use crate::Py; use std::sync::Arc; #[test] fn test_eval() { Python::with_gil(|py| { // Make sure builtin names are accessible let v: i32 = py .eval("min(1, 2)", None, None) .map_err(|e| e.display(py)) .unwrap() .extract() .unwrap(); assert_eq!(v, 1); let d = [("foo", 13)].into_py_dict(py); // Inject our own global namespace let v: i32 = py .eval("foo + 29", Some(d), None) .unwrap() .extract() .unwrap(); assert_eq!(v, 42); // Inject our own local namespace let v: i32 = py .eval("foo + 29", None, Some(d)) .unwrap() .extract() .unwrap(); assert_eq!(v, 42); // Make sure builtin names are still accessible when using a local namespace let v: i32 = py .eval("min(foo, 2)", None, Some(d)) .unwrap() .extract() .unwrap(); assert_eq!(v, 2); }); } #[test] #[cfg(not(target_arch = "wasm32"))] // We are building wasm Python with pthreads disabled fn test_allow_threads_releases_and_acquires_gil() { Python::with_gil(|py| { let b = std::sync::Arc::new(std::sync::Barrier::new(2)); let b2 = b.clone(); std::thread::spawn(move || Python::with_gil(|_| b2.wait())); py.allow_threads(|| { // If allow_threads does not release the GIL, this will deadlock because // the thread spawned above will never be able to acquire the GIL. b.wait(); }); unsafe { // If the GIL is not reacquired at the end of allow_threads, this call // will crash the Python interpreter. let tstate = ffi::PyEval_SaveThread(); ffi::PyEval_RestoreThread(tstate); } }); } #[test] fn test_allow_threads_panics_safely() { Python::with_gil(|py| { let result = std::panic::catch_unwind(|| unsafe { let py = Python::assume_gil_acquired(); py.allow_threads(|| { panic!("There was a panic!"); }); }); // Check panic was caught assert!(result.is_err()); // If allow_threads is implemented correctly, this thread still owns the GIL here // so the following Python calls should not cause crashes. let list = PyList::new(py, [1, 2, 3, 4]); assert_eq!(list.extract::>().unwrap(), vec![1, 2, 3, 4]); }); } #[test] fn test_allow_threads_pass_stuff_in() { let list: Py = Python::with_gil(|py| { let list = PyList::new(py, vec!["foo", "bar"]); list.into() }); let mut v = vec![1, 2, 3]; let a = Arc::new(String::from("foo")); Python::with_gil(|py| { py.allow_threads(|| { drop((list, &mut v, a)); }); }); } #[test] #[cfg(not(Py_LIMITED_API))] fn test_acquire_gil() { const GIL_NOT_HELD: c_int = 0; const GIL_HELD: c_int = 1; let state = unsafe { crate::ffi::PyGILState_Check() }; assert_eq!(state, GIL_NOT_HELD); Python::with_gil(|_| { let state = unsafe { crate::ffi::PyGILState_Check() }; assert_eq!(state, GIL_HELD); }); let state = unsafe { crate::ffi::PyGILState_Check() }; assert_eq!(state, GIL_NOT_HELD); } #[test] fn test_ellipsis() { Python::with_gil(|py| { assert_eq!(py.Ellipsis().to_string(), "Ellipsis"); let v = py .eval("...", None, None) .map_err(|e| e.display(py)) .unwrap(); assert!(v.eq(py.Ellipsis()).unwrap()); }); } #[test] fn test_py_run_inserts_globals() { Python::with_gil(|py| { let namespace = PyDict::new(py); py.run("class Foo: pass", Some(namespace), Some(namespace)) .unwrap(); assert!(matches!(namespace.get_item("Foo"), Ok(Some(..)))); assert!(matches!(namespace.get_item("__builtins__"), Ok(Some(..)))); }) } } pyo3-0.20.2/src/marshal.rs000064400000000000000000000047621046102023000133740ustar 00000000000000#![cfg(not(Py_LIMITED_API))] //! Support for the Python `marshal` format. use crate::ffi; use crate::types::{PyAny, PyBytes}; use crate::{AsPyPointer, FromPyPointer, PyResult, Python}; use std::os::raw::{c_char, c_int}; /// The current version of the marshal binary format. pub const VERSION: i32 = 4; /// Serialize an object to bytes using the Python built-in marshal module. /// /// The built-in marshalling only supports a limited range of objects. /// The exact types supported depend on the version argument. /// The [`VERSION`] constant holds the highest version currently supported. /// /// See the [Python documentation](https://docs.python.org/3/library/marshal.html) for more details. /// /// # Examples /// ``` /// # use pyo3::{marshal, types::PyDict}; /// # pyo3::Python::with_gil(|py| { /// let dict = PyDict::new(py); /// dict.set_item("aap", "noot").unwrap(); /// dict.set_item("mies", "wim").unwrap(); /// dict.set_item("zus", "jet").unwrap(); /// /// let bytes = marshal::dumps(py, dict, marshal::VERSION); /// # }); /// ``` pub fn dumps<'a>(py: Python<'a>, object: &impl AsPyPointer, version: i32) -> PyResult<&'a PyBytes> { unsafe { let bytes = ffi::PyMarshal_WriteObjectToString(object.as_ptr(), version as c_int); FromPyPointer::from_owned_ptr_or_err(py, bytes) } } /// Deserialize an object from bytes using the Python built-in marshal module. pub fn loads<'a, B>(py: Python<'a>, data: &B) -> PyResult<&'a PyAny> where B: AsRef<[u8]> + ?Sized, { let data = data.as_ref(); unsafe { let c_str = data.as_ptr() as *const c_char; let object = ffi::PyMarshal_ReadObjectFromString(c_str, data.len() as isize); FromPyPointer::from_owned_ptr_or_err(py, object) } } #[cfg(test)] mod tests { use super::*; use crate::types::PyDict; #[test] fn marshal_roundtrip() { Python::with_gil(|py| { let dict = PyDict::new(py); dict.set_item("aap", "noot").unwrap(); dict.set_item("mies", "wim").unwrap(); dict.set_item("zus", "jet").unwrap(); let bytes = dumps(py, dict, VERSION) .expect("marshalling failed") .as_bytes(); let deserialized = loads(py, bytes).expect("unmarshalling failed"); assert!(equal(py, dict, deserialized)); }); } fn equal(_py: Python<'_>, a: &impl AsPyPointer, b: &impl AsPyPointer) -> bool { unsafe { ffi::PyObject_RichCompareBool(a.as_ptr(), b.as_ptr(), ffi::Py_EQ) != 0 } } } pyo3-0.20.2/src/panic.rs000064400000000000000000000017471046102023000130370ustar 00000000000000//! Helper to convert Rust panics to Python exceptions. use crate::exceptions::PyBaseException; use crate::PyErr; use std::any::Any; pyo3_exception!( " The exception raised when Rust code called from Python panics. Like SystemExit, this exception is derived from BaseException so that it will typically propagate all the way through the stack and cause the Python interpreter to exit. ", PanicException, PyBaseException ); impl PanicException { /// Creates a new PanicException from a panic payload. /// /// Attempts to format the error in the same way panic does. #[cold] pub(crate) fn from_panic_payload(payload: Box) -> PyErr { if let Some(string) = payload.downcast_ref::() { Self::new_err((string.clone(),)) } else if let Some(s) = payload.downcast_ref::<&str>() { Self::new_err((s.to_string(),)) } else { Self::new_err(("panic from Rust code",)) } } } pyo3-0.20.2/src/prelude.rs000064400000000000000000000014011046102023000133700ustar 00000000000000//! PyO3's prelude. //! //! The purpose of this module is to alleviate imports of many commonly used items of the PyO3 crate //! by adding a glob import to the top of pyo3 heavy modules: //! //! ``` //! # #![allow(unused_imports)] //! use pyo3::prelude::*; //! ``` pub use crate::conversion::{FromPyObject, IntoPy, PyTryFrom, PyTryInto, ToPyObject}; pub use crate::err::{PyErr, PyResult}; pub use crate::instance::{Py, PyObject}; pub use crate::marker::Python; pub use crate::pycell::{PyCell, PyRef, PyRefMut}; pub use crate::pyclass_init::PyClassInitializer; pub use crate::types::{PyAny, PyModule}; #[cfg(feature = "macros")] pub use pyo3_macros::{pyclass, pyfunction, pymethods, pymodule, FromPyObject}; #[cfg(feature = "macros")] pub use crate::wrap_pyfunction; pyo3-0.20.2/src/pycell/impl_.rs000064400000000000000000000312621046102023000143300ustar 00000000000000#![allow(missing_docs)] //! Crate-private implementation of pycell use std::cell::Cell; use std::marker::PhantomData; use crate::impl_::pyclass::{PyClassBaseType, PyClassImpl}; use crate::PyCell; use super::{PyBorrowError, PyBorrowMutError}; pub trait PyClassMutability { // The storage for this inheritance layer. Only the first mutable class in // an inheritance hierarchy needs to store the borrow flag. type Storage: PyClassBorrowChecker; // The borrow flag needed to implement this class' mutability. Empty until // the first mutable class, at which point it is BorrowChecker and will be // for all subclasses. type Checker: PyClassBorrowChecker; type ImmutableChild: PyClassMutability; type MutableChild: PyClassMutability; } pub struct ImmutableClass(()); pub struct MutableClass(()); pub struct ExtendsMutableAncestor(PhantomData); impl PyClassMutability for ImmutableClass { type Storage = EmptySlot; type Checker = EmptySlot; type ImmutableChild = ImmutableClass; type MutableChild = MutableClass; } impl PyClassMutability for MutableClass { type Storage = BorrowChecker; type Checker = BorrowChecker; type ImmutableChild = ExtendsMutableAncestor; type MutableChild = ExtendsMutableAncestor; } impl PyClassMutability for ExtendsMutableAncestor { type Storage = EmptySlot; type Checker = BorrowChecker; type ImmutableChild = ExtendsMutableAncestor; type MutableChild = ExtendsMutableAncestor; } #[derive(Debug, Copy, Clone, Eq, PartialEq)] struct BorrowFlag(usize); impl BorrowFlag { pub(crate) const UNUSED: BorrowFlag = BorrowFlag(0); const HAS_MUTABLE_BORROW: BorrowFlag = BorrowFlag(usize::max_value()); const fn increment(self) -> Self { Self(self.0 + 1) } const fn decrement(self) -> Self { Self(self.0 - 1) } } pub struct EmptySlot(()); pub struct BorrowChecker(Cell); pub trait PyClassBorrowChecker { /// Initial value for self fn new() -> Self; /// Increments immutable borrow count, if possible fn try_borrow(&self) -> Result<(), PyBorrowError>; fn try_borrow_unguarded(&self) -> Result<(), PyBorrowError>; /// Decrements immutable borrow count fn release_borrow(&self); /// Increments mutable borrow count, if possible fn try_borrow_mut(&self) -> Result<(), PyBorrowMutError>; /// Decremements mutable borrow count fn release_borrow_mut(&self); } impl PyClassBorrowChecker for EmptySlot { #[inline] fn new() -> Self { EmptySlot(()) } #[inline] fn try_borrow(&self) -> Result<(), PyBorrowError> { Ok(()) } #[inline] fn try_borrow_unguarded(&self) -> Result<(), PyBorrowError> { Ok(()) } #[inline] fn release_borrow(&self) {} #[inline] fn try_borrow_mut(&self) -> Result<(), PyBorrowMutError> { unreachable!() } #[inline] fn release_borrow_mut(&self) { unreachable!() } } impl PyClassBorrowChecker for BorrowChecker { #[inline] fn new() -> Self { Self(Cell::new(BorrowFlag::UNUSED)) } fn try_borrow(&self) -> Result<(), PyBorrowError> { let flag = self.0.get(); if flag != BorrowFlag::HAS_MUTABLE_BORROW { self.0.set(flag.increment()); Ok(()) } else { Err(PyBorrowError { _private: () }) } } fn try_borrow_unguarded(&self) -> Result<(), PyBorrowError> { let flag = self.0.get(); if flag != BorrowFlag::HAS_MUTABLE_BORROW { Ok(()) } else { Err(PyBorrowError { _private: () }) } } fn release_borrow(&self) { let flag = self.0.get(); self.0.set(flag.decrement()) } fn try_borrow_mut(&self) -> Result<(), PyBorrowMutError> { let flag = self.0.get(); if flag == BorrowFlag::UNUSED { self.0.set(BorrowFlag::HAS_MUTABLE_BORROW); Ok(()) } else { Err(PyBorrowMutError { _private: () }) } } fn release_borrow_mut(&self) { self.0.set(BorrowFlag::UNUSED) } } pub trait GetBorrowChecker { fn borrow_checker(cell: &PyCell) -> &::Checker; } impl> GetBorrowChecker for MutableClass { fn borrow_checker(cell: &PyCell) -> &BorrowChecker { &cell.contents.borrow_checker } } impl> GetBorrowChecker for ImmutableClass { fn borrow_checker(cell: &PyCell) -> &EmptySlot { &cell.contents.borrow_checker } } impl, M: PyClassMutability> GetBorrowChecker for ExtendsMutableAncestor where T::BaseType: PyClassImpl + PyClassBaseType>, ::PyClassMutability: PyClassMutability, { fn borrow_checker(cell: &PyCell) -> &BorrowChecker { <::PyClassMutability as GetBorrowChecker>::borrow_checker(&cell.ob_base) } } #[cfg(test)] #[cfg(feature = "macros")] mod tests { use super::*; use crate::prelude::*; use crate::pyclass::boolean_struct::{False, True}; use crate::PyClass; #[pyclass(crate = "crate", subclass)] struct MutableBase; #[pyclass(crate = "crate", extends = MutableBase, subclass)] struct MutableChildOfMutableBase; #[pyclass(crate = "crate", extends = MutableBase, frozen, subclass)] struct ImmutableChildOfMutableBase; #[pyclass(crate = "crate", extends = MutableChildOfMutableBase)] struct MutableChildOfMutableChildOfMutableBase; #[pyclass(crate = "crate", extends = ImmutableChildOfMutableBase)] struct MutableChildOfImmutableChildOfMutableBase; #[pyclass(crate = "crate", extends = MutableChildOfMutableBase, frozen)] struct ImmutableChildOfMutableChildOfMutableBase; #[pyclass(crate = "crate", extends = ImmutableChildOfMutableBase, frozen)] struct ImmutableChildOfImmutableChildOfMutableBase; #[pyclass(crate = "crate", frozen, subclass)] struct ImmutableBase; #[pyclass(crate = "crate", extends = ImmutableBase, subclass)] struct MutableChildOfImmutableBase; #[pyclass(crate = "crate", extends = ImmutableBase, frozen, subclass)] struct ImmutableChildOfImmutableBase; #[pyclass(crate = "crate", extends = MutableChildOfImmutableBase)] struct MutableChildOfMutableChildOfImmutableBase; #[pyclass(crate = "crate", extends = ImmutableChildOfImmutableBase)] struct MutableChildOfImmutableChildOfImmutableBase; #[pyclass(crate = "crate", extends = MutableChildOfImmutableBase, frozen)] struct ImmutableChildOfMutableChildOfImmutableBase; #[pyclass(crate = "crate", extends = ImmutableChildOfImmutableBase, frozen)] struct ImmutableChildOfImmutableChildOfImmutableBase; fn assert_mutable>() {} fn assert_immutable>() {} fn assert_mutable_with_mutable_ancestor< T: PyClass>, >() { } fn assert_immutable_with_mutable_ancestor< T: PyClass>, >() { } #[test] fn test_inherited_mutability() { // mutable base assert_mutable::(); // children of mutable base have a mutable ancestor assert_mutable_with_mutable_ancestor::(); assert_immutable_with_mutable_ancestor::(); // grandchildren of mutable base have a mutable ancestor assert_mutable_with_mutable_ancestor::(); assert_mutable_with_mutable_ancestor::(); assert_immutable_with_mutable_ancestor::(); assert_immutable_with_mutable_ancestor::(); // immutable base and children assert_immutable::(); assert_immutable::(); assert_immutable::(); // mutable children of immutable at any level are simply mutable assert_mutable::(); assert_mutable::(); // children of the mutable child display this property assert_mutable_with_mutable_ancestor::(); assert_immutable_with_mutable_ancestor::(); } #[test] fn test_mutable_borrow_prevents_further_borrows() { Python::with_gil(|py| { let mmm = Py::new( py, PyClassInitializer::from(MutableBase) .add_subclass(MutableChildOfMutableBase) .add_subclass(MutableChildOfMutableChildOfMutableBase), ) .unwrap(); let mmm_cell: &PyCell = mmm.as_ref(py); let mmm_refmut = mmm_cell.borrow_mut(); // Cannot take any other mutable or immutable borrows whilst the object is borrowed mutably assert!(mmm_cell .extract::>() .is_err()); assert!(mmm_cell .extract::>() .is_err()); assert!(mmm_cell.extract::>().is_err()); assert!(mmm_cell .extract::>() .is_err()); assert!(mmm_cell .extract::>() .is_err()); assert!(mmm_cell.extract::>().is_err()); // With the borrow dropped, all other borrow attempts will succeed drop(mmm_refmut); assert!(mmm_cell .extract::>() .is_ok()); assert!(mmm_cell .extract::>() .is_ok()); assert!(mmm_cell.extract::>().is_ok()); assert!(mmm_cell .extract::>() .is_ok()); assert!(mmm_cell .extract::>() .is_ok()); assert!(mmm_cell.extract::>().is_ok()); }) } #[test] fn test_immutable_borrows_prevent_mutable_borrows() { Python::with_gil(|py| { let mmm = Py::new( py, PyClassInitializer::from(MutableBase) .add_subclass(MutableChildOfMutableBase) .add_subclass(MutableChildOfMutableChildOfMutableBase), ) .unwrap(); let mmm_cell: &PyCell = mmm.as_ref(py); let mmm_refmut = mmm_cell.borrow(); // Further immutable borrows are ok assert!(mmm_cell .extract::>() .is_ok()); assert!(mmm_cell .extract::>() .is_ok()); assert!(mmm_cell.extract::>().is_ok()); // Further mutable borrows are not ok assert!(mmm_cell .extract::>() .is_err()); assert!(mmm_cell .extract::>() .is_err()); assert!(mmm_cell.extract::>().is_err()); // With the borrow dropped, all mutable borrow attempts will succeed drop(mmm_refmut); assert!(mmm_cell .extract::>() .is_ok()); assert!(mmm_cell .extract::>() .is_ok()); assert!(mmm_cell.extract::>().is_ok()); }) } } pyo3-0.20.2/src/pycell.rs000064400000000000000000001047151046102023000132340ustar 00000000000000//! PyO3's interior mutability primitive. //! //! Rust has strict aliasing rules - you can either have any number of immutable (shared) references or one mutable //! reference. Python's ownership model is the complete opposite of that - any Python object //! can be referenced any number of times, and mutation is allowed from any reference. //! //! PyO3 deals with these differences by employing the [Interior Mutability] //! pattern. This requires that PyO3 enforces the borrowing rules and it has two mechanisms for //! doing so: //! - Statically it can enforce threadsafe access with the [`Python<'py>`](crate::Python) token. //! All Rust code holding that token, or anything derived from it, can assume that they have //! safe access to the Python interpreter's state. For this reason all the native Python objects //! can be mutated through shared references. //! - However, methods and functions in Rust usually *do* need `&mut` references. While PyO3 can //! use the [`Python<'py>`](crate::Python) token to guarantee thread-safe access to them, it cannot //! statically guarantee uniqueness of `&mut` references. As such those references have to be tracked //! dynamically at runtime, using [`PyCell`] and the other types defined in this module. This works //! similar to std's [`RefCell`](std::cell::RefCell) type. //! //! # When *not* to use PyCell //! //! Usually you can use `&mut` references as method and function receivers and arguments, and you //! won't need to use [`PyCell`] directly: //! //! ```rust //! use pyo3::prelude::*; //! //! #[pyclass] //! struct Number { //! inner: u32, //! } //! //! #[pymethods] //! impl Number { //! fn increment(&mut self) { //! self.inner += 1; //! } //! } //! ``` //! //! The [`#[pymethods]`](crate::pymethods) proc macro will generate this wrapper function (and more), //! using [`PyCell`] under the hood: //! //! ```rust //! # use pyo3::prelude::*; //! # #[pyclass] //! # struct Number { //! # inner: u32, //! # } //! # //! # #[pymethods] //! # impl Number { //! # fn increment(&mut self) { //! # self.inner += 1; //! # } //! # } //! # //! // The function which is exported to Python looks roughly like the following //! unsafe extern "C" fn __pymethod_increment__( //! _slf: *mut pyo3::ffi::PyObject, //! _args: *mut pyo3::ffi::PyObject, //! ) -> *mut pyo3::ffi::PyObject { //! use :: pyo3 as _pyo3; //! _pyo3::impl_::trampoline::noargs(_slf, _args, |py, _slf| { //! let _cell = py //! .from_borrowed_ptr::<_pyo3::PyAny>(_slf) //! .downcast::<_pyo3::PyCell>()?; //! let mut _ref = _cell.try_borrow_mut()?; //! let _slf: &mut Number = &mut *_ref; //! _pyo3::callback::convert(py, Number::increment(_slf)) //! }) //! } //! ``` //! //! # When to use PyCell //! ## Using pyclasses from Rust //! //! However, we *do* need [`PyCell`] if we want to call its methods from Rust: //! ```rust //! # use pyo3::prelude::*; //! # //! # #[pyclass] //! # struct Number { //! # inner: u32, //! # } //! # //! # #[pymethods] //! # impl Number { //! # fn increment(&mut self) { //! # self.inner += 1; //! # } //! # } //! # fn main() -> PyResult<()> { //! Python::with_gil(|py| { //! let n = Py::new(py, Number { inner: 0 })?; //! //! // We borrow the guard and then dereference //! // it to get a mutable reference to Number //! let mut guard: PyRefMut<'_, Number> = n.as_ref(py).borrow_mut(); //! let n_mutable: &mut Number = &mut *guard; //! //! n_mutable.increment(); //! //! // To avoid panics we must dispose of the //! // `PyRefMut` before borrowing again. //! drop(guard); //! //! let n_immutable: &Number = &n.as_ref(py).borrow(); //! assert_eq!(n_immutable.inner, 1); //! //! Ok(()) //! }) //! # } //! ``` //! ## Dealing with possibly overlapping mutable references //! //! It is also necessary to use [`PyCell`] if you can receive mutable arguments that may overlap. //! Suppose the following function that swaps the values of two `Number`s: //! ``` //! # use pyo3::prelude::*; //! # #[pyclass] //! # pub struct Number { //! # inner: u32, //! # } //! #[pyfunction] //! fn swap_numbers(a: &mut Number, b: &mut Number) { //! std::mem::swap(&mut a.inner, &mut b.inner); //! } //! # fn main() { //! # Python::with_gil(|py| { //! # let n = Py::new(py, Number{inner: 35}).unwrap(); //! # let n2 = n.clone_ref(py); //! # assert!(n.is(&n2)); //! # let fun = pyo3::wrap_pyfunction!(swap_numbers, py).unwrap(); //! # fun.call1((n, n2)).expect_err("Managed to create overlapping mutable references. Note: this is undefined behaviour."); //! # }); //! # } //! ``` //! When users pass in the same `Number` as both arguments, one of the mutable borrows will //! fail and raise a `RuntimeError`: //! ```text //! >>> a = Number() //! >>> swap_numbers(a, a) //! Traceback (most recent call last): //! File "", line 1, in //! RuntimeError: Already borrowed //! ``` //! //! It is better to write that function like this: //! ```rust //! # use pyo3::prelude::*; //! # #[pyclass] //! # pub struct Number { //! # inner: u32, //! # } //! #[pyfunction] //! fn swap_numbers(a: &PyCell, b: &PyCell) { //! // Check that the pointers are unequal //! if !a.is(b) { //! std::mem::swap(&mut a.borrow_mut().inner, &mut b.borrow_mut().inner); //! } else { //! // Do nothing - they are the same object, so don't need swapping. //! } //! } //! # fn main() { //! # // With duplicate numbers //! # Python::with_gil(|py| { //! # let n = Py::new(py, Number{inner: 35}).unwrap(); //! # let n2 = n.clone_ref(py); //! # assert!(n.is(&n2)); //! # let fun = pyo3::wrap_pyfunction!(swap_numbers, py).unwrap(); //! # fun.call1((n, n2)).unwrap(); //! # }); //! # //! # // With two different numbers //! # Python::with_gil(|py| { //! # let n = Py::new(py, Number{inner: 35}).unwrap(); //! # let n2 = Py::new(py, Number{inner: 42}).unwrap(); //! # assert!(!n.is(&n2)); //! # let fun = pyo3::wrap_pyfunction!(swap_numbers, py).unwrap(); //! # fun.call1((&n, &n2)).unwrap(); //! # let n: u32 = n.borrow(py).inner; //! # let n2: u32 = n2.borrow(py).inner; //! # assert_eq!(n, 42); //! # assert_eq!(n2, 35); //! # }); //! # } //! ``` //! See the [guide] for more information. //! //! [guide]: https://pyo3.rs/latest/class.html#pycell-and-interior-mutability "PyCell and interior mutability" //! [Interior Mutability]: https://doc.rust-lang.org/book/ch15-05-interior-mutability.html "RefCell and the Interior Mutability Pattern - The Rust Programming Language" use crate::exceptions::PyRuntimeError; use crate::impl_::pyclass::{ PyClassBaseType, PyClassDict, PyClassImpl, PyClassThreadChecker, PyClassWeakRef, }; use crate::pyclass::{ boolean_struct::{False, True}, PyClass, }; use crate::pyclass_init::PyClassInitializer; use crate::type_object::{PyLayout, PySizedLayout}; use crate::types::PyAny; use crate::{ conversion::{AsPyPointer, FromPyPointer, ToPyObject}, type_object::get_tp_free, PyTypeInfo, }; use crate::{ffi, IntoPy, PyErr, PyNativeType, PyObject, PyResult, Python}; use std::cell::UnsafeCell; use std::fmt; use std::mem::ManuallyDrop; use std::ops::{Deref, DerefMut}; pub(crate) mod impl_; use impl_::{GetBorrowChecker, PyClassBorrowChecker, PyClassMutability}; /// Base layout of PyCell. #[doc(hidden)] #[repr(C)] pub struct PyCellBase { ob_base: T, } unsafe impl PyLayout for PyCellBase where U: PySizedLayout {} /// A container type for (mutably) accessing [`PyClass`] values /// /// `PyCell` autodereferences to [`PyAny`], so you can call `PyAny`'s methods on a `PyCell`. /// /// # Examples /// /// This example demonstrates getting a mutable reference of the contained `PyClass`. /// ```rust /// use pyo3::prelude::*; /// /// #[pyclass] /// struct Number { /// inner: u32, /// } /// /// #[pymethods] /// impl Number { /// fn increment(&mut self) { /// self.inner += 1; /// } /// } /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// let n = PyCell::new(py, Number { inner: 0 })?; /// /// let n_mutable: &mut Number = &mut n.borrow_mut(); /// n_mutable.increment(); /// /// Ok(()) /// }) /// # } /// ``` /// For more information on how, when and why (not) to use `PyCell` please see the /// [module-level documentation](self). #[repr(C)] pub struct PyCell { ob_base: ::LayoutAsBase, contents: PyCellContents, } #[repr(C)] pub(crate) struct PyCellContents { pub(crate) value: ManuallyDrop>, pub(crate) borrow_checker: ::Storage, pub(crate) thread_checker: T::ThreadChecker, pub(crate) dict: T::Dict, pub(crate) weakref: T::WeakRef, } unsafe impl PyNativeType for PyCell {} impl PyCell { /// Makes a new `PyCell` on the Python heap and return the reference to it. /// /// In cases where the value in the cell does not need to be accessed immediately after /// creation, consider [`Py::new`](crate::Py::new) as a more efficient alternative. pub fn new(py: Python<'_>, value: impl Into>) -> PyResult<&Self> { unsafe { let initializer = value.into(); let self_ = initializer.create_cell(py)?; FromPyPointer::from_owned_ptr_or_err(py, self_ as _) } } /// Immutably borrows the value `T`. This borrow lasts as long as the returned `PyRef` exists. /// /// For frozen classes, the simpler [`get`][Self::get] is available. /// /// # Panics /// /// Panics if the value is currently mutably borrowed. For a non-panicking variant, use /// [`try_borrow`](#method.try_borrow). pub fn borrow(&self) -> PyRef<'_, T> { self.try_borrow().expect("Already mutably borrowed") } /// Mutably borrows the value `T`. This borrow lasts as long as the returned `PyRefMut` exists. /// /// # Panics /// /// Panics if the value is currently borrowed. For a non-panicking variant, use /// [`try_borrow_mut`](#method.try_borrow_mut). pub fn borrow_mut(&self) -> PyRefMut<'_, T> where T: PyClass, { self.try_borrow_mut().expect("Already borrowed") } /// Immutably borrows the value `T`, returning an error if the value is currently /// mutably borrowed. This borrow lasts as long as the returned `PyRef` exists. /// /// This is the non-panicking variant of [`borrow`](#method.borrow). /// /// For frozen classes, the simpler [`get`][Self::get] is available. /// /// # Examples /// /// ``` /// # use pyo3::prelude::*; /// #[pyclass] /// struct Class {} /// /// Python::with_gil(|py| { /// let c = PyCell::new(py, Class {}).unwrap(); /// { /// let m = c.borrow_mut(); /// assert!(c.try_borrow().is_err()); /// } /// /// { /// let m = c.borrow(); /// assert!(c.try_borrow().is_ok()); /// } /// }); /// ``` pub fn try_borrow(&self) -> Result, PyBorrowError> { self.ensure_threadsafe(); self.borrow_checker() .try_borrow() .map(|_| PyRef { inner: self }) } /// Mutably borrows the value `T`, returning an error if the value is currently borrowed. /// This borrow lasts as long as the returned `PyRefMut` exists. /// /// This is the non-panicking variant of [`borrow_mut`](#method.borrow_mut). /// /// # Examples /// /// ``` /// # use pyo3::prelude::*; /// #[pyclass] /// struct Class {} /// Python::with_gil(|py| { /// let c = PyCell::new(py, Class {}).unwrap(); /// { /// let m = c.borrow(); /// assert!(c.try_borrow_mut().is_err()); /// } /// /// assert!(c.try_borrow_mut().is_ok()); /// }); /// ``` pub fn try_borrow_mut(&self) -> Result, PyBorrowMutError> where T: PyClass, { self.ensure_threadsafe(); self.borrow_checker() .try_borrow_mut() .map(|_| PyRefMut { inner: self }) } /// Immutably borrows the value `T`, returning an error if the value is /// currently mutably borrowed. /// /// # Safety /// /// This method is unsafe because it does not return a `PyRef`, /// thus leaving the borrow flag untouched. Mutably borrowing the `PyCell` /// while the reference returned by this method is alive is undefined behaviour. /// /// # Examples /// /// ``` /// # use pyo3::prelude::*; /// #[pyclass] /// struct Class {} /// Python::with_gil(|py| { /// let c = PyCell::new(py, Class {}).unwrap(); /// /// { /// let m = c.borrow_mut(); /// assert!(unsafe { c.try_borrow_unguarded() }.is_err()); /// } /// /// { /// let m = c.borrow(); /// assert!(unsafe { c.try_borrow_unguarded() }.is_ok()); /// } /// }); /// ``` pub unsafe fn try_borrow_unguarded(&self) -> Result<&T, PyBorrowError> { self.ensure_threadsafe(); self.borrow_checker() .try_borrow_unguarded() .map(|_: ()| &*self.contents.value.get()) } /// Provide an immutable borrow of the value `T` without acquiring the GIL. /// /// This is available if the class is [`frozen`][macro@crate::pyclass] and [`Sync`]. /// /// While the GIL is usually required to get access to `&PyCell`, /// compared to [`borrow`][Self::borrow] or [`try_borrow`][Self::try_borrow] /// this avoids any thread or borrow checking overhead at runtime. /// /// # Examples /// /// ``` /// use std::sync::atomic::{AtomicUsize, Ordering}; /// # use pyo3::prelude::*; /// /// #[pyclass(frozen)] /// struct FrozenCounter { /// value: AtomicUsize, /// } /// /// Python::with_gil(|py| { /// let counter = FrozenCounter { value: AtomicUsize::new(0) }; /// /// let cell = PyCell::new(py, counter).unwrap(); /// /// cell.get().value.fetch_add(1, Ordering::Relaxed); /// }); /// ``` pub fn get(&self) -> &T where T: PyClass + Sync, { // SAFETY: The class itself is frozen and `Sync` and we do not access anything but `self.contents.value`. unsafe { &*self.get_ptr() } } /// Replaces the wrapped value with a new one, returning the old value. /// /// # Panics /// /// Panics if the value is currently borrowed. #[inline] pub fn replace(&self, t: T) -> T where T: PyClass, { std::mem::replace(&mut *self.borrow_mut(), t) } /// Replaces the wrapped value with a new one computed from `f`, returning the old value. /// /// # Panics /// /// Panics if the value is currently borrowed. pub fn replace_with T>(&self, f: F) -> T where T: PyClass, { let mut_borrow = &mut *self.borrow_mut(); let replacement = f(mut_borrow); std::mem::replace(mut_borrow, replacement) } /// Swaps the wrapped value of `self` with the wrapped value of `other`. /// /// # Panics /// /// Panics if the value in either `PyCell` is currently borrowed. #[inline] pub fn swap(&self, other: &Self) where T: PyClass, { std::mem::swap(&mut *self.borrow_mut(), &mut *other.borrow_mut()) } pub(crate) fn get_ptr(&self) -> *mut T { self.contents.value.get() } /// Gets the offset of the dictionary from the start of the struct in bytes. pub(crate) fn dict_offset() -> ffi::Py_ssize_t { use memoffset::offset_of; let offset = offset_of!(PyCell, contents) + offset_of!(PyCellContents, dict); // Py_ssize_t may not be equal to isize on all platforms #[allow(clippy::useless_conversion)] offset.try_into().expect("offset should fit in Py_ssize_t") } /// Gets the offset of the weakref list from the start of the struct in bytes. pub(crate) fn weaklist_offset() -> ffi::Py_ssize_t { use memoffset::offset_of; let offset = offset_of!(PyCell, contents) + offset_of!(PyCellContents, weakref); // Py_ssize_t may not be equal to isize on all platforms #[allow(clippy::useless_conversion)] offset.try_into().expect("offset should fit in Py_ssize_t") } } impl PyCell { fn borrow_checker(&self) -> &::Checker { T::PyClassMutability::borrow_checker(self) } } unsafe impl PyLayout for PyCell {} impl PySizedLayout for PyCell {} unsafe impl AsPyPointer for PyCell { fn as_ptr(&self) -> *mut ffi::PyObject { (self as *const _) as *mut _ } } impl ToPyObject for &PyCell { fn to_object(&self, py: Python<'_>) -> PyObject { unsafe { PyObject::from_borrowed_ptr(py, self.as_ptr()) } } } impl AsRef for PyCell { fn as_ref(&self) -> &PyAny { unsafe { self.py().from_borrowed_ptr(self.as_ptr()) } } } impl Deref for PyCell { type Target = PyAny; fn deref(&self) -> &PyAny { unsafe { self.py().from_borrowed_ptr(self.as_ptr()) } } } impl fmt::Debug for PyCell { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self.try_borrow() { Ok(borrow) => f.debug_struct("RefCell").field("value", &borrow).finish(), Err(_) => { struct BorrowedPlaceholder; impl fmt::Debug for BorrowedPlaceholder { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { f.write_str("") } } f.debug_struct("RefCell") .field("value", &BorrowedPlaceholder) .finish() } } } } /// A wrapper type for an immutably borrowed value from a [`PyCell`]``. /// /// See the [`PyCell`] documentation for more information. /// /// # Examples /// /// You can use `PyRef` as an alternative to a `&self` receiver when /// - you need to access the pointer of the `PyCell`, or /// - you want to get a super class. /// ``` /// # use pyo3::prelude::*; /// #[pyclass(subclass)] /// struct Parent { /// basename: &'static str, /// } /// /// #[pyclass(extends=Parent)] /// struct Child { /// name: &'static str, /// } /// /// #[pymethods] /// impl Child { /// #[new] /// fn new() -> (Self, Parent) { /// (Child { name: "Caterpillar" }, Parent { basename: "Butterfly" }) /// } /// /// fn format(slf: PyRef<'_, Self>) -> String { /// // We can get *mut ffi::PyObject from PyRef /// let refcnt = unsafe { pyo3::ffi::Py_REFCNT(slf.as_ptr()) }; /// // We can get &Self::BaseType by as_ref /// let basename = slf.as_ref().basename; /// format!("{}(base: {}, cnt: {})", slf.name, basename, refcnt) /// } /// } /// # Python::with_gil(|py| { /// # let sub = PyCell::new(py, Child::new()).unwrap(); /// # pyo3::py_run!(py, sub, "assert sub.format() == 'Caterpillar(base: Butterfly, cnt: 3)'"); /// # }); /// ``` /// /// See the [module-level documentation](self) for more information. pub struct PyRef<'p, T: PyClass> { inner: &'p PyCell, } impl<'p, T: PyClass> PyRef<'p, T> { /// Returns a `Python` token that is bound to the lifetime of the `PyRef`. pub fn py(&self) -> Python<'p> { self.inner.py() } } impl<'p, T, U> AsRef for PyRef<'p, T> where T: PyClass, U: PyClass, { fn as_ref(&self) -> &T::BaseType { unsafe { &*self.inner.ob_base.get_ptr() } } } impl<'p, T: PyClass> PyRef<'p, T> { /// Returns the raw FFI pointer represented by self. /// /// # Safety /// /// Callers are responsible for ensuring that the pointer does not outlive self. /// /// The reference is borrowed; callers should not decrease the reference count /// when they are finished with the pointer. #[inline] pub fn as_ptr(&self) -> *mut ffi::PyObject { self.inner.as_ptr() } /// Returns an owned raw FFI pointer represented by self. /// /// # Safety /// /// The reference is owned; when finished the caller should either transfer ownership /// of the pointer or decrease the reference count (e.g. with [`pyo3::ffi::Py_DecRef`](crate::ffi::Py_DecRef)). #[inline] pub fn into_ptr(self) -> *mut ffi::PyObject { self.inner.into_ptr() } } impl<'p, T, U> PyRef<'p, T> where T: PyClass, U: PyClass, { /// Gets a `PyRef`. /// /// While `as_ref()` returns a reference of type `&T::BaseType`, this cannot be /// used to get the base of `T::BaseType`. /// /// But with the help of this method, you can get hold of instances of the /// super-superclass when needed. /// /// # Examples /// ``` /// # use pyo3::prelude::*; /// #[pyclass(subclass)] /// struct Base1 { /// name1: &'static str, /// } /// /// #[pyclass(extends=Base1, subclass)] /// struct Base2 { /// name2: &'static str, /// } /// /// #[pyclass(extends=Base2)] /// struct Sub { /// name3: &'static str, /// } /// /// #[pymethods] /// impl Sub { /// #[new] /// fn new() -> PyClassInitializer { /// PyClassInitializer::from(Base1 { name1: "base1" }) /// .add_subclass(Base2 { name2: "base2" }) /// .add_subclass(Self { name3: "sub" }) /// } /// fn name(slf: PyRef<'_, Self>) -> String { /// let subname = slf.name3; /// let super_ = slf.into_super(); /// format!("{} {} {}", super_.as_ref().name1, super_.name2, subname) /// } /// } /// # Python::with_gil(|py| { /// # let sub = PyCell::new(py, Sub::new()).unwrap(); /// # pyo3::py_run!(py, sub, "assert sub.name() == 'base1 base2 sub'") /// # }); /// ``` pub fn into_super(self) -> PyRef<'p, U> { let PyRef { inner } = self; std::mem::forget(self); PyRef { inner: &inner.ob_base, } } } impl<'p, T: PyClass> Deref for PyRef<'p, T> { type Target = T; #[inline] fn deref(&self) -> &T { unsafe { &*self.inner.get_ptr() } } } impl<'p, T: PyClass> Drop for PyRef<'p, T> { fn drop(&mut self) { self.inner.borrow_checker().release_borrow() } } impl IntoPy for PyRef<'_, T> { fn into_py(self, py: Python<'_>) -> PyObject { unsafe { PyObject::from_borrowed_ptr(py, self.inner.as_ptr()) } } } impl IntoPy for &'_ PyRef<'_, T> { fn into_py(self, py: Python<'_>) -> PyObject { self.inner.into_py(py) } } impl<'a, T: PyClass> std::convert::TryFrom<&'a PyCell> for crate::PyRef<'a, T> { type Error = PyBorrowError; fn try_from(cell: &'a crate::PyCell) -> Result { cell.try_borrow() } } unsafe impl<'a, T: PyClass> AsPyPointer for PyRef<'a, T> { fn as_ptr(&self) -> *mut ffi::PyObject { self.inner.as_ptr() } } impl fmt::Debug for PyRef<'_, T> { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { fmt::Debug::fmt(&**self, f) } } /// A wrapper type for a mutably borrowed value from a[`PyCell`]``. /// /// See the [module-level documentation](self) for more information. pub struct PyRefMut<'p, T: PyClass> { inner: &'p PyCell, } impl<'p, T: PyClass> PyRefMut<'p, T> { /// Returns a `Python` token that is bound to the lifetime of the `PyRefMut`. pub fn py(&self) -> Python<'p> { self.inner.py() } } impl<'p, T, U> AsRef for PyRefMut<'p, T> where T: PyClass, U: PyClass, { fn as_ref(&self) -> &T::BaseType { unsafe { &*self.inner.ob_base.get_ptr() } } } impl<'p, T, U> AsMut for PyRefMut<'p, T> where T: PyClass, U: PyClass, { fn as_mut(&mut self) -> &mut T::BaseType { unsafe { &mut *self.inner.ob_base.get_ptr() } } } impl<'p, T: PyClass> PyRefMut<'p, T> { /// Returns the raw FFI pointer represented by self. /// /// # Safety /// /// Callers are responsible for ensuring that the pointer does not outlive self. /// /// The reference is borrowed; callers should not decrease the reference count /// when they are finished with the pointer. #[inline] pub fn as_ptr(&self) -> *mut ffi::PyObject { self.inner.as_ptr() } /// Returns an owned raw FFI pointer represented by self. /// /// # Safety /// /// The reference is owned; when finished the caller should either transfer ownership /// of the pointer or decrease the reference count (e.g. with [`pyo3::ffi::Py_DecRef`](crate::ffi::Py_DecRef)). #[inline] pub fn into_ptr(self) -> *mut ffi::PyObject { self.inner.into_ptr() } } impl<'p, T, U> PyRefMut<'p, T> where T: PyClass, U: PyClass, { /// Gets a `PyRef`. /// /// See [`PyRef::into_super`] for more. pub fn into_super(self) -> PyRefMut<'p, U> { let PyRefMut { inner } = self; std::mem::forget(self); PyRefMut { inner: &inner.ob_base, } } } impl<'p, T: PyClass> Deref for PyRefMut<'p, T> { type Target = T; #[inline] fn deref(&self) -> &T { unsafe { &*self.inner.get_ptr() } } } impl<'p, T: PyClass> DerefMut for PyRefMut<'p, T> { #[inline] fn deref_mut(&mut self) -> &mut T { unsafe { &mut *self.inner.get_ptr() } } } impl<'p, T: PyClass> Drop for PyRefMut<'p, T> { fn drop(&mut self) { self.inner.borrow_checker().release_borrow_mut() } } impl> IntoPy for PyRefMut<'_, T> { fn into_py(self, py: Python<'_>) -> PyObject { unsafe { PyObject::from_borrowed_ptr(py, self.inner.as_ptr()) } } } impl> IntoPy for &'_ PyRefMut<'_, T> { fn into_py(self, py: Python<'_>) -> PyObject { self.inner.into_py(py) } } unsafe impl<'a, T: PyClass> AsPyPointer for PyRefMut<'a, T> { fn as_ptr(&self) -> *mut ffi::PyObject { self.inner.as_ptr() } } impl<'a, T: PyClass> std::convert::TryFrom<&'a PyCell> for crate::PyRefMut<'a, T> { type Error = PyBorrowMutError; fn try_from(cell: &'a crate::PyCell) -> Result { cell.try_borrow_mut() } } impl + fmt::Debug> fmt::Debug for PyRefMut<'_, T> { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { fmt::Debug::fmt(self.deref(), f) } } /// An error type returned by [`PyCell::try_borrow`]. /// /// If this error is allowed to bubble up into Python code it will raise a `RuntimeError`. pub struct PyBorrowError { _private: (), } impl fmt::Debug for PyBorrowError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { f.debug_struct("PyBorrowError").finish() } } impl fmt::Display for PyBorrowError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { fmt::Display::fmt("Already mutably borrowed", f) } } impl From for PyErr { fn from(other: PyBorrowError) -> Self { PyRuntimeError::new_err(other.to_string()) } } /// An error type returned by [`PyCell::try_borrow_mut`]. /// /// If this error is allowed to bubble up into Python code it will raise a `RuntimeError`. pub struct PyBorrowMutError { _private: (), } impl fmt::Debug for PyBorrowMutError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { f.debug_struct("PyBorrowMutError").finish() } } impl fmt::Display for PyBorrowMutError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { fmt::Display::fmt("Already borrowed", f) } } impl From for PyErr { fn from(other: PyBorrowMutError) -> Self { PyRuntimeError::new_err(other.to_string()) } } #[doc(hidden)] pub trait PyCellLayout: PyLayout { fn ensure_threadsafe(&self); /// Implementation of tp_dealloc. /// # Safety /// - slf must be a valid pointer to an instance of a T or a subclass. /// - slf must not be used after this call (as it will be freed). unsafe fn tp_dealloc(py: Python<'_>, slf: *mut ffi::PyObject); } impl PyCellLayout for PyCellBase where U: PySizedLayout, T: PyTypeInfo, { fn ensure_threadsafe(&self) {} unsafe fn tp_dealloc(py: Python<'_>, slf: *mut ffi::PyObject) { let type_obj = T::type_object_raw(py); // For `#[pyclass]` types which inherit from PyAny, we can just call tp_free if type_obj == std::ptr::addr_of_mut!(ffi::PyBaseObject_Type) { return get_tp_free(ffi::Py_TYPE(slf))(slf as _); } // More complex native types (e.g. `extends=PyDict`) require calling the base's dealloc. #[cfg(not(Py_LIMITED_API))] { if let Some(dealloc) = (*type_obj).tp_dealloc { // Before CPython 3.11 BaseException_dealloc would use Py_GC_UNTRACK which // assumes the exception is currently GC tracked, so we have to re-track // before calling the dealloc so that it can safely call Py_GC_UNTRACK. #[cfg(not(any(Py_3_11, PyPy)))] if ffi::PyType_FastSubclass(type_obj, ffi::Py_TPFLAGS_BASE_EXC_SUBCLASS) == 1 { ffi::PyObject_GC_Track(slf.cast()); } dealloc(slf as _); } else { get_tp_free(ffi::Py_TYPE(slf))(slf as _); } } #[cfg(Py_LIMITED_API)] unreachable!("subclassing native types is not possible with the `abi3` feature"); } } impl PyCellLayout for PyCell where ::LayoutAsBase: PyCellLayout, { fn ensure_threadsafe(&self) { self.contents.thread_checker.ensure(); self.ob_base.ensure_threadsafe(); } unsafe fn tp_dealloc(py: Python<'_>, slf: *mut ffi::PyObject) { // Safety: Python only calls tp_dealloc when no references to the object remain. let cell = &mut *(slf as *mut PyCell); if cell.contents.thread_checker.can_drop(py) { ManuallyDrop::drop(&mut cell.contents.value); } cell.contents.dict.clear_dict(py); cell.contents.weakref.clear_weakrefs(slf, py); ::LayoutAsBase::tp_dealloc(py, slf) } } #[cfg(test)] #[cfg(feature = "macros")] mod tests { use super::*; #[crate::pyclass] #[pyo3(crate = "crate")] #[derive(Copy, Clone, PartialEq, Eq, Debug)] struct SomeClass(i32); #[test] fn pycell_replace() { Python::with_gil(|py| { let cell = PyCell::new(py, SomeClass(0)).unwrap(); assert_eq!(*cell.borrow(), SomeClass(0)); let previous = cell.replace(SomeClass(123)); assert_eq!(previous, SomeClass(0)); assert_eq!(*cell.borrow(), SomeClass(123)); }) } #[test] #[should_panic(expected = "Already borrowed: PyBorrowMutError")] fn pycell_replace_panic() { Python::with_gil(|py| { let cell = PyCell::new(py, SomeClass(0)).unwrap(); let _guard = cell.borrow(); cell.replace(SomeClass(123)); }) } #[test] fn pycell_replace_with() { Python::with_gil(|py| { let cell = PyCell::new(py, SomeClass(0)).unwrap(); assert_eq!(*cell.borrow(), SomeClass(0)); let previous = cell.replace_with(|value| { *value = SomeClass(2); SomeClass(123) }); assert_eq!(previous, SomeClass(2)); assert_eq!(*cell.borrow(), SomeClass(123)); }) } #[test] #[should_panic(expected = "Already borrowed: PyBorrowMutError")] fn pycell_replace_with_panic() { Python::with_gil(|py| { let cell = PyCell::new(py, SomeClass(0)).unwrap(); let _guard = cell.borrow(); cell.replace_with(|_| SomeClass(123)); }) } #[test] fn pycell_swap() { Python::with_gil(|py| { let cell = PyCell::new(py, SomeClass(0)).unwrap(); let cell2 = PyCell::new(py, SomeClass(123)).unwrap(); assert_eq!(*cell.borrow(), SomeClass(0)); assert_eq!(*cell2.borrow(), SomeClass(123)); cell.swap(cell2); assert_eq!(*cell.borrow(), SomeClass(123)); assert_eq!(*cell2.borrow(), SomeClass(0)); }) } #[test] #[should_panic(expected = "Already borrowed: PyBorrowMutError")] fn pycell_swap_panic() { Python::with_gil(|py| { let cell = PyCell::new(py, SomeClass(0)).unwrap(); let cell2 = PyCell::new(py, SomeClass(123)).unwrap(); let _guard = cell.borrow(); cell.swap(cell2); }) } #[test] #[should_panic(expected = "Already borrowed: PyBorrowMutError")] fn pycell_swap_panic_other_borrowed() { Python::with_gil(|py| { let cell = PyCell::new(py, SomeClass(0)).unwrap(); let cell2 = PyCell::new(py, SomeClass(123)).unwrap(); let _guard = cell2.borrow(); cell.swap(cell2); }) } #[test] fn test_as_ptr() { Python::with_gil(|py| { let cell = PyCell::new(py, SomeClass(0)).unwrap(); let ptr = cell.as_ptr(); assert_eq!(cell.borrow().as_ptr(), ptr); assert_eq!(cell.borrow_mut().as_ptr(), ptr); }) } #[test] fn test_into_ptr() { Python::with_gil(|py| { let cell = PyCell::new(py, SomeClass(0)).unwrap(); let ptr = cell.as_ptr(); assert_eq!(cell.borrow().into_ptr(), ptr); unsafe { ffi::Py_DECREF(ptr) }; assert_eq!(cell.borrow_mut().into_ptr(), ptr); unsafe { ffi::Py_DECREF(ptr) }; }) } } pyo3-0.20.2/src/pyclass/create_type_object.rs000064400000000000000000000536001046102023000172500ustar 00000000000000use pyo3_ffi::PyType_IS_GC; use crate::{ exceptions::PyTypeError, ffi, impl_::pyclass::{ assign_sequence_item_from_mapping, get_sequence_item_from_mapping, tp_dealloc, tp_dealloc_with_gc, PyClassItemsIter, }, impl_::{ pymethods::{get_doc, get_name, Getter, Setter}, trampoline::trampoline, }, types::PyType, Py, PyCell, PyClass, PyGetterDef, PyMethodDefType, PyResult, PySetterDef, PyTypeInfo, Python, }; use std::{ borrow::Cow, collections::HashMap, convert::TryInto, ffi::{CStr, CString}, os::raw::{c_char, c_int, c_ulong, c_void}, ptr, }; pub(crate) struct PyClassTypeObject { pub type_object: Py, #[allow(dead_code)] // This is purely a cache that must live as long as the type object getset_destructors: Vec, } pub(crate) fn create_type_object(py: Python<'_>) -> PyResult where T: PyClass, { // Written this way to monomorphize the majority of the logic. #[allow(clippy::too_many_arguments)] unsafe fn inner( py: Python<'_>, base: *mut ffi::PyTypeObject, dealloc: unsafe extern "C" fn(*mut ffi::PyObject), dealloc_with_gc: unsafe extern "C" fn(*mut ffi::PyObject), is_mapping: bool, is_sequence: bool, doc: &'static CStr, dict_offset: Option, weaklist_offset: Option, is_basetype: bool, items_iter: PyClassItemsIter, name: &'static str, module: Option<&'static str>, size_of: usize, ) -> PyResult { PyTypeBuilder { slots: Vec::new(), method_defs: Vec::new(), getset_builders: HashMap::new(), cleanup: Vec::new(), tp_base: base, tp_dealloc: dealloc, tp_dealloc_with_gc: dealloc_with_gc, is_mapping, is_sequence, has_new: false, has_dealloc: false, has_getitem: false, has_setitem: false, has_traverse: false, has_clear: false, has_dict: false, class_flags: 0, #[cfg(all(not(Py_3_9), not(Py_LIMITED_API)))] buffer_procs: Default::default(), } .type_doc(doc) .offsets(dict_offset, weaklist_offset) .set_is_basetype(is_basetype) .class_items(items_iter) .build(py, name, module, size_of) } unsafe { inner( py, T::BaseType::type_object_raw(py), tp_dealloc::, tp_dealloc_with_gc::, T::IS_MAPPING, T::IS_SEQUENCE, T::doc(py)?, T::dict_offset(), T::weaklist_offset(), T::IS_BASETYPE, T::items_iter(), T::NAME, T::MODULE, std::mem::size_of::>(), ) } } type PyTypeBuilderCleanup = Box; struct PyTypeBuilder { slots: Vec, method_defs: Vec, getset_builders: HashMap<&'static str, GetSetDefBuilder>, /// Used to patch the type objects for the things there's no /// PyType_FromSpec API for... there's no reason this should work, /// except for that it does and we have tests. cleanup: Vec, tp_base: *mut ffi::PyTypeObject, tp_dealloc: ffi::destructor, tp_dealloc_with_gc: ffi::destructor, is_mapping: bool, is_sequence: bool, has_new: bool, has_dealloc: bool, has_getitem: bool, has_setitem: bool, has_traverse: bool, has_clear: bool, has_dict: bool, class_flags: c_ulong, // Before Python 3.9, need to patch in buffer methods manually (they don't work in slots) #[cfg(all(not(Py_3_9), not(Py_LIMITED_API)))] buffer_procs: ffi::PyBufferProcs, } impl PyTypeBuilder { /// # Safety /// The given pointer must be of the correct type for the given slot unsafe fn push_slot(&mut self, slot: c_int, pfunc: *mut T) { match slot { ffi::Py_tp_new => self.has_new = true, ffi::Py_tp_dealloc => self.has_dealloc = true, ffi::Py_mp_subscript => self.has_getitem = true, ffi::Py_mp_ass_subscript => self.has_setitem = true, ffi::Py_tp_traverse => { self.has_traverse = true; self.class_flags |= ffi::Py_TPFLAGS_HAVE_GC; } ffi::Py_tp_clear => self.has_clear = true, #[cfg(all(not(Py_3_9), not(Py_LIMITED_API)))] ffi::Py_bf_getbuffer => { // Safety: slot.pfunc is a valid function pointer self.buffer_procs.bf_getbuffer = Some(std::mem::transmute(pfunc)); } #[cfg(all(not(Py_3_9), not(Py_LIMITED_API)))] ffi::Py_bf_releasebuffer => { // Safety: slot.pfunc is a valid function pointer self.buffer_procs.bf_releasebuffer = Some(std::mem::transmute(pfunc)); } _ => {} } self.slots.push(ffi::PyType_Slot { slot, pfunc: pfunc as _, }); } /// # Safety /// It is the caller's responsibility that `data` is of the correct type for the given slot. unsafe fn push_raw_vec_slot(&mut self, slot: c_int, mut data: Vec) { if !data.is_empty() { // Python expects a zeroed entry to mark the end of the defs data.push(std::mem::zeroed()); self.push_slot(slot, Box::into_raw(data.into_boxed_slice()) as *mut c_void); } } fn pymethod_def(&mut self, def: &PyMethodDefType) { match def { PyMethodDefType::Getter(getter) => { self.getset_builders .entry(getter.name) .or_default() .add_getter(getter); } PyMethodDefType::Setter(setter) => { self.getset_builders .entry(setter.name) .or_default() .add_setter(setter); } PyMethodDefType::Method(def) | PyMethodDefType::Class(def) | PyMethodDefType::Static(def) => { let (def, destructor) = def.as_method_def().unwrap(); // FIXME: stop leaking destructor std::mem::forget(destructor); self.method_defs.push(def); } // These class attributes are added after the type gets created by LazyStaticType PyMethodDefType::ClassAttribute(_) => {} } } fn finalize_methods_and_properties(&mut self) -> PyResult> { let method_defs: Vec = std::mem::take(&mut self.method_defs); // Safety: Py_tp_methods expects a raw vec of PyMethodDef unsafe { self.push_raw_vec_slot(ffi::Py_tp_methods, method_defs) }; let mut getset_destructors = Vec::with_capacity(self.getset_builders.len()); #[allow(unused_mut)] let mut property_defs: Vec<_> = self .getset_builders .iter() .map(|(name, builder)| { let (def, destructor) = builder.as_get_set_def(name)?; getset_destructors.push(destructor); Ok(def) }) .collect::>()?; // PyPy doesn't automatically add __dict__ getter / setter. // PyObject_GenericGetDict not in the limited API until Python 3.10. if self.has_dict { #[cfg(not(any(PyPy, all(Py_LIMITED_API, not(Py_3_10)))))] property_defs.push(ffi::PyGetSetDef { name: "__dict__\0".as_ptr().cast(), get: Some(ffi::PyObject_GenericGetDict), set: Some(ffi::PyObject_GenericSetDict), doc: ptr::null(), closure: ptr::null_mut(), }); } // Safety: Py_tp_members expects a raw vec of PyGetSetDef unsafe { self.push_raw_vec_slot(ffi::Py_tp_getset, property_defs) }; // If mapping methods implemented, define sequence methods get implemented too. // CPython does the same for Python `class` statements. // NB we don't implement sq_length to avoid annoying CPython behaviour of automatically adding // the length to negative indices. // Don't add these methods for "pure" mappings. if !self.is_mapping && self.has_getitem { // Safety: This is the correct slot type for Py_sq_item unsafe { self.push_slot( ffi::Py_sq_item, get_sequence_item_from_mapping as *mut c_void, ) } } if !self.is_mapping && self.has_setitem { // Safety: This is the correct slot type for Py_sq_ass_item unsafe { self.push_slot( ffi::Py_sq_ass_item, assign_sequence_item_from_mapping as *mut c_void, ) } } Ok(getset_destructors) } fn set_is_basetype(mut self, is_basetype: bool) -> Self { if is_basetype { self.class_flags |= ffi::Py_TPFLAGS_BASETYPE; } self } /// # Safety /// All slots in the PyClassItemsIter should be correct unsafe fn class_items(mut self, iter: PyClassItemsIter) -> Self { for items in iter { for slot in items.slots { self.push_slot(slot.slot, slot.pfunc); } for method in items.methods { self.pymethod_def(method); } } self } fn type_doc(mut self, type_doc: &'static CStr) -> Self { let slice = type_doc.to_bytes(); if !slice.is_empty() { unsafe { self.push_slot(ffi::Py_tp_doc, type_doc.as_ptr() as *mut c_char) } // Running this causes PyPy to segfault. #[cfg(all(not(PyPy), not(Py_LIMITED_API), not(Py_3_10)))] { // Until CPython 3.10, tp_doc was treated specially for // heap-types, and it removed the text_signature value from it. // We go in after the fact and replace tp_doc with something // that _does_ include the text_signature value! self.cleanup .push(Box::new(move |_self, type_object| unsafe { ffi::PyObject_Free((*type_object).tp_doc as _); let data = ffi::PyMem_Malloc(slice.len()); data.copy_from(slice.as_ptr() as _, slice.len()); (*type_object).tp_doc = data as _; })) } } self } fn offsets( mut self, dict_offset: Option, #[allow(unused_variables)] weaklist_offset: Option, ) -> Self { self.has_dict = dict_offset.is_some(); #[cfg(Py_3_9)] { #[inline(always)] fn offset_def( name: &'static str, offset: ffi::Py_ssize_t, ) -> ffi::structmember::PyMemberDef { ffi::structmember::PyMemberDef { name: name.as_ptr() as _, type_code: ffi::structmember::T_PYSSIZET, offset, flags: ffi::structmember::READONLY, doc: std::ptr::null_mut(), } } let mut members = Vec::new(); // __dict__ support if let Some(dict_offset) = dict_offset { members.push(offset_def("__dictoffset__\0", dict_offset)); } // weakref support if let Some(weaklist_offset) = weaklist_offset { members.push(offset_def("__weaklistoffset__\0", weaklist_offset)); } // Safety: Py_tp_members expects a raw vec of PyMemberDef unsafe { self.push_raw_vec_slot(ffi::Py_tp_members, members) }; } // Setting buffer protocols, tp_dictoffset and tp_weaklistoffset via slots doesn't work until // Python 3.9, so on older versions we must manually fixup the type object. #[cfg(all(not(Py_LIMITED_API), not(Py_3_9)))] { self.cleanup .push(Box::new(move |builder, type_object| unsafe { (*(*type_object).tp_as_buffer).bf_getbuffer = builder.buffer_procs.bf_getbuffer; (*(*type_object).tp_as_buffer).bf_releasebuffer = builder.buffer_procs.bf_releasebuffer; if let Some(dict_offset) = dict_offset { (*type_object).tp_dictoffset = dict_offset; } if let Some(weaklist_offset) = weaklist_offset { (*type_object).tp_weaklistoffset = weaklist_offset; } })); } self } fn build( mut self, py: Python<'_>, name: &'static str, module_name: Option<&'static str>, basicsize: usize, ) -> PyResult { // `c_ulong` and `c_uint` have the same size // on some platforms (like windows) #![allow(clippy::useless_conversion)] let getset_destructors = self.finalize_methods_and_properties()?; unsafe { self.push_slot(ffi::Py_tp_base, self.tp_base) } if !self.has_new { // Safety: This is the correct slot type for Py_tp_new unsafe { self.push_slot(ffi::Py_tp_new, no_constructor_defined as *mut c_void) } } let tp_dealloc = if self.has_traverse || unsafe { PyType_IS_GC(self.tp_base) == 1 } { self.tp_dealloc_with_gc } else { self.tp_dealloc }; unsafe { self.push_slot(ffi::Py_tp_dealloc, tp_dealloc as *mut c_void) } if self.has_clear && !self.has_traverse { return Err(PyTypeError::new_err(format!( "`#[pyclass]` {} implements __clear__ without __traverse__", name ))); } // For sequences, implement sq_length instead of mp_length if self.is_sequence { for slot in &mut self.slots { if slot.slot == ffi::Py_mp_length { slot.slot = ffi::Py_sq_length; } } } // Add empty sentinel at the end // Safety: python expects this empty slot unsafe { self.push_slot(0, ptr::null_mut::()) } let class_name = py_class_qualified_name(module_name, name)?; let mut spec = ffi::PyType_Spec { name: class_name.as_ptr() as _, basicsize: basicsize as c_int, itemsize: 0, flags: (ffi::Py_TPFLAGS_DEFAULT | self.class_flags) .try_into() .unwrap(), slots: self.slots.as_mut_ptr(), }; // Safety: We've correctly setup the PyType_Spec at this point let type_object: Py = unsafe { Py::from_owned_ptr_or_err(py, ffi::PyType_FromSpec(&mut spec))? }; #[cfg(not(Py_3_11))] bpo_45315_workaround(py, class_name); for cleanup in std::mem::take(&mut self.cleanup) { cleanup(&self, type_object.as_ref(py).as_type_ptr()); } Ok(PyClassTypeObject { type_object, getset_destructors, }) } } fn py_class_qualified_name(module_name: Option<&str>, class_name: &str) -> PyResult { Ok(CString::new(format!( "{}.{}", module_name.unwrap_or("builtins"), class_name ))?) } /// Workaround for Python issue 45315; no longer necessary in Python 3.11 #[inline] #[cfg(not(Py_3_11))] fn bpo_45315_workaround(py: Python<'_>, class_name: CString) { #[cfg(Py_LIMITED_API)] { // Must check version at runtime for abi3 wheels - they could run against a higher version // than the build config suggests. use crate::sync::GILOnceCell; static IS_PYTHON_3_11: GILOnceCell = GILOnceCell::new(); if *IS_PYTHON_3_11.get_or_init(py, || py.version_info() >= (3, 11)) { // No fix needed - the wheel is running on a sufficiently new interpreter. return; } } #[cfg(not(Py_LIMITED_API))] { // suppress unused variable warning let _ = py; } std::mem::forget(class_name); } /// Default new implementation unsafe extern "C" fn no_constructor_defined( _subtype: *mut ffi::PyTypeObject, _args: *mut ffi::PyObject, _kwds: *mut ffi::PyObject, ) -> *mut ffi::PyObject { trampoline(|_| { Err(crate::exceptions::PyTypeError::new_err( "No constructor defined", )) }) } #[derive(Default)] struct GetSetDefBuilder { doc: Option<&'static str>, getter: Option, setter: Option, } impl GetSetDefBuilder { fn add_getter(&mut self, getter: &PyGetterDef) { // TODO: be smarter about merging getter and setter docs if self.doc.is_none() { self.doc = Some(getter.doc); } // TODO: return an error if getter already defined? self.getter = Some(getter.meth.0) } fn add_setter(&mut self, setter: &PySetterDef) { // TODO: be smarter about merging getter and setter docs if self.doc.is_none() { self.doc = Some(setter.doc); } // TODO: return an error if setter already defined? self.setter = Some(setter.meth.0) } fn as_get_set_def( &self, name: &'static str, ) -> PyResult<(ffi::PyGetSetDef, GetSetDefDestructor)> { let name = get_name(name)?; let doc = self.doc.map(get_doc).transpose()?; let getset_type = match (self.getter, self.setter) { (Some(getter), None) => GetSetDefType::Getter(getter), (None, Some(setter)) => GetSetDefType::Setter(setter), (Some(getter), Some(setter)) => { GetSetDefType::GetterAndSetter(Box::new(GetterAndSetter { getter, setter })) } (None, None) => { unreachable!("GetSetDefBuilder expected to always have either getter or setter") } }; let getset_def = getset_type.create_py_get_set_def(&name, doc.as_deref()); let destructor = GetSetDefDestructor { name, doc, closure: getset_type, }; Ok((getset_def, destructor)) } } #[allow(dead_code)] // a stack of fields which are purely to cache until dropped struct GetSetDefDestructor { name: Cow<'static, CStr>, doc: Option>, closure: GetSetDefType, } /// Possible forms of property - either a getter, setter, or both enum GetSetDefType { Getter(Getter), Setter(Setter), // The box is here so that the `GetterAndSetter` has a stable // memory address even if the `GetSetDefType` enum is moved GetterAndSetter(Box), } pub(crate) struct GetterAndSetter { getter: Getter, setter: Setter, } impl GetSetDefType { /// Fills a PyGetSetDef structure /// It is only valid for as long as this GetSetDefType remains alive, /// as well as name and doc members pub(crate) fn create_py_get_set_def( &self, name: &CStr, doc: Option<&CStr>, ) -> ffi::PyGetSetDef { let (get, set, closure): (Option, Option, *mut c_void) = match self { &Self::Getter(closure) => { unsafe extern "C" fn getter( slf: *mut ffi::PyObject, closure: *mut c_void, ) -> *mut ffi::PyObject { // Safety: PyO3 sets the closure when constructing the ffi getter so this cast should always be valid let getter: Getter = std::mem::transmute(closure); trampoline(|py| getter(py, slf)) } (Some(getter), None, closure as Getter as _) } &Self::Setter(closure) => { unsafe extern "C" fn setter( slf: *mut ffi::PyObject, value: *mut ffi::PyObject, closure: *mut c_void, ) -> c_int { // Safety: PyO3 sets the closure when constructing the ffi setter so this cast should always be valid let setter: Setter = std::mem::transmute(closure); trampoline(|py| setter(py, slf, value)) } (None, Some(setter), closure as Setter as _) } Self::GetterAndSetter(closure) => { unsafe extern "C" fn getset_getter( slf: *mut ffi::PyObject, closure: *mut c_void, ) -> *mut ffi::PyObject { let getset: &GetterAndSetter = &*(closure as *const GetterAndSetter); trampoline(|py| (getset.getter)(py, slf)) } unsafe extern "C" fn getset_setter( slf: *mut ffi::PyObject, value: *mut ffi::PyObject, closure: *mut c_void, ) -> c_int { let getset: &GetterAndSetter = &*(closure as *const GetterAndSetter); trampoline(|py| (getset.setter)(py, slf, value)) } ( Some(getset_getter), Some(getset_setter), closure.as_ref() as *const GetterAndSetter as _, ) } }; ffi::PyGetSetDef { name: name.as_ptr(), doc: doc.map_or(ptr::null(), CStr::as_ptr), get, set, closure, } } } pyo3-0.20.2/src/pyclass/gc.rs000064400000000000000000000023521046102023000140050ustar 00000000000000use std::os::raw::{c_int, c_void}; use crate::{ffi, AsPyPointer, Python}; /// Error returned by a `__traverse__` visitor implementation. #[repr(transparent)] pub struct PyTraverseError(pub(crate) c_int); /// Object visitor for GC. #[derive(Clone)] pub struct PyVisit<'p> { pub(crate) visit: ffi::visitproc, pub(crate) arg: *mut c_void, /// VisitProc contains a Python instance to ensure that /// 1) it is cannot be moved out of the traverse() call /// 2) it cannot be sent to other threads pub(crate) _py: Python<'p>, } impl<'p> PyVisit<'p> { /// Visit `obj`. pub fn call(&self, obj: &T) -> Result<(), PyTraverseError> where T: AsPyPointer, { let ptr = obj.as_ptr(); if !ptr.is_null() { let r = unsafe { (self.visit)(ptr, self.arg) }; if r == 0 { Ok(()) } else { Err(PyTraverseError(r)) } } else { Ok(()) } } /// Creates the PyVisit from the arguments to tp_traverse #[doc(hidden)] pub unsafe fn from_raw(visit: ffi::visitproc, arg: *mut c_void, py: Python<'p>) -> Self { Self { visit, arg, _py: py, } } } pyo3-0.20.2/src/pyclass.rs000064400000000000000000000171701046102023000134200ustar 00000000000000//! `PyClass` and related traits. use crate::{ callback::IntoPyCallbackOutput, ffi, impl_::pyclass::PyClassImpl, IntoPy, PyCell, PyObject, PyResult, PyTypeInfo, Python, }; use std::{cmp::Ordering, os::raw::c_int}; mod create_type_object; mod gc; pub(crate) use self::create_type_object::{create_type_object, PyClassTypeObject}; pub use self::gc::{PyTraverseError, PyVisit}; /// Types that can be used as Python classes. /// /// The `#[pyclass]` attribute implements this trait for your Rust struct - /// you shouldn't implement this trait directly. pub trait PyClass: PyTypeInfo> + PyClassImpl { /// Whether the pyclass is frozen. /// /// This can be enabled via `#[pyclass(frozen)]`. type Frozen: Frozen; } /// Operators for the `__richcmp__` method #[derive(Debug, Clone, Copy)] pub enum CompareOp { /// The *less than* operator. Lt = ffi::Py_LT as isize, /// The *less than or equal to* operator. Le = ffi::Py_LE as isize, /// The equality operator. Eq = ffi::Py_EQ as isize, /// The *not equal to* operator. Ne = ffi::Py_NE as isize, /// The *greater than* operator. Gt = ffi::Py_GT as isize, /// The *greater than or equal to* operator. Ge = ffi::Py_GE as isize, } impl CompareOp { /// Conversion from the C enum. pub fn from_raw(op: c_int) -> Option { match op { ffi::Py_LT => Some(CompareOp::Lt), ffi::Py_LE => Some(CompareOp::Le), ffi::Py_EQ => Some(CompareOp::Eq), ffi::Py_NE => Some(CompareOp::Ne), ffi::Py_GT => Some(CompareOp::Gt), ffi::Py_GE => Some(CompareOp::Ge), _ => None, } } /// Returns if a Rust [`std::cmp::Ordering`] matches this ordering query. /// /// Usage example: /// /// ```rust /// # use pyo3::prelude::*; /// # use pyo3::class::basic::CompareOp; /// /// #[pyclass] /// struct Size { /// size: usize, /// } /// /// #[pymethods] /// impl Size { /// fn __richcmp__(&self, other: &Size, op: CompareOp) -> bool { /// op.matches(self.size.cmp(&other.size)) /// } /// } /// ``` pub fn matches(&self, result: Ordering) -> bool { match self { CompareOp::Eq => result == Ordering::Equal, CompareOp::Ne => result != Ordering::Equal, CompareOp::Lt => result == Ordering::Less, CompareOp::Le => result != Ordering::Greater, CompareOp::Gt => result == Ordering::Greater, CompareOp::Ge => result != Ordering::Less, } } } /// Output of `__next__` which can either `yield` the next value in the iteration, or /// `return` a value to raise `StopIteration` in Python. /// /// Usage example: /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::iter::IterNextOutput; /// /// #[pyclass] /// struct PyClassIter { /// count: usize, /// } /// /// #[pymethods] /// impl PyClassIter { /// #[new] /// pub fn new() -> Self { /// PyClassIter { count: 0 } /// } /// /// fn __next__(&mut self) -> IterNextOutput { /// if self.count < 5 { /// self.count += 1; /// // Given an instance `counter`, First five `next(counter)` calls yield 1, 2, 3, 4, 5. /// IterNextOutput::Yield(self.count) /// } else { /// // At the sixth time, we get a `StopIteration` with `'Ended'`. /// // try: /// // next(counter) /// // except StopIteration as e: /// // assert e.value == 'Ended' /// IterNextOutput::Return("Ended") /// } /// } /// } /// ``` pub enum IterNextOutput { /// The value yielded by the iterator. Yield(T), /// The `StopIteration` object. Return(U), } /// Alias of `IterNextOutput` with `PyObject` yield & return values. pub type PyIterNextOutput = IterNextOutput; impl IntoPyCallbackOutput<*mut ffi::PyObject> for PyIterNextOutput { fn convert(self, _py: Python<'_>) -> PyResult<*mut ffi::PyObject> { match self { IterNextOutput::Yield(o) => Ok(o.into_ptr()), IterNextOutput::Return(opt) => Err(crate::exceptions::PyStopIteration::new_err((opt,))), } } } impl IntoPyCallbackOutput for IterNextOutput where T: IntoPy, U: IntoPy, { fn convert(self, py: Python<'_>) -> PyResult { match self { IterNextOutput::Yield(o) => Ok(IterNextOutput::Yield(o.into_py(py))), IterNextOutput::Return(o) => Ok(IterNextOutput::Return(o.into_py(py))), } } } impl IntoPyCallbackOutput for Option where T: IntoPy, { fn convert(self, py: Python<'_>) -> PyResult { match self { Some(o) => Ok(PyIterNextOutput::Yield(o.into_py(py))), None => Ok(PyIterNextOutput::Return(py.None())), } } } /// Output of `__anext__`. /// /// pub enum IterANextOutput { /// An expression which the generator yielded. Yield(T), /// A `StopAsyncIteration` object. Return(U), } /// An [IterANextOutput] of Python objects. pub type PyIterANextOutput = IterANextOutput; impl IntoPyCallbackOutput<*mut ffi::PyObject> for PyIterANextOutput { fn convert(self, _py: Python<'_>) -> PyResult<*mut ffi::PyObject> { match self { IterANextOutput::Yield(o) => Ok(o.into_ptr()), IterANextOutput::Return(opt) => { Err(crate::exceptions::PyStopAsyncIteration::new_err((opt,))) } } } } impl IntoPyCallbackOutput for IterANextOutput where T: IntoPy, U: IntoPy, { fn convert(self, py: Python<'_>) -> PyResult { match self { IterANextOutput::Yield(o) => Ok(IterANextOutput::Yield(o.into_py(py))), IterANextOutput::Return(o) => Ok(IterANextOutput::Return(o.into_py(py))), } } } impl IntoPyCallbackOutput for Option where T: IntoPy, { fn convert(self, py: Python<'_>) -> PyResult { match self { Some(o) => Ok(PyIterANextOutput::Yield(o.into_py(py))), None => Ok(PyIterANextOutput::Return(py.None())), } } } /// A workaround for [associated const equality](https://github.com/rust-lang/rust/issues/92827). /// /// This serves to have True / False values in the [`PyClass`] trait's `Frozen` type. #[doc(hidden)] pub mod boolean_struct { pub(crate) mod private { use super::*; /// A way to "seal" the boolean traits. pub trait Boolean {} impl Boolean for True {} impl Boolean for False {} } pub struct True(()); pub struct False(()); } /// A trait which is used to describe whether a `#[pyclass]` is frozen. #[doc(hidden)] pub trait Frozen: boolean_struct::private::Boolean {} impl Frozen for boolean_struct::True {} impl Frozen for boolean_struct::False {} mod tests { #[test] fn test_compare_op_matches() { use super::CompareOp; use std::cmp::Ordering; assert!(CompareOp::Eq.matches(Ordering::Equal)); assert!(CompareOp::Ne.matches(Ordering::Less)); assert!(CompareOp::Ge.matches(Ordering::Greater)); assert!(CompareOp::Gt.matches(Ordering::Greater)); assert!(CompareOp::Le.matches(Ordering::Equal)); assert!(CompareOp::Lt.matches(Ordering::Less)); } } pyo3-0.20.2/src/pyclass_init.rs000064400000000000000000000234021046102023000144360ustar 00000000000000//! Contains initialization utilities for `#[pyclass]`. use crate::callback::IntoPyCallbackOutput; use crate::impl_::pyclass::{PyClassBaseType, PyClassDict, PyClassThreadChecker, PyClassWeakRef}; use crate::{ffi, Py, PyCell, PyClass, PyErr, PyResult, Python}; use crate::{ ffi::PyTypeObject, pycell::{ impl_::{PyClassBorrowChecker, PyClassMutability}, PyCellContents, }, type_object::{get_tp_alloc, PyTypeInfo}, }; use std::{ cell::UnsafeCell, marker::PhantomData, mem::{ManuallyDrop, MaybeUninit}, }; /// Initializer for Python types. /// /// This trait is intended to use internally for distinguishing `#[pyclass]` and /// Python native types. pub trait PyObjectInit: Sized { /// # Safety /// - `subtype` must be a valid pointer to a type object of T or a subclass. unsafe fn into_new_object( self, py: Python<'_>, subtype: *mut PyTypeObject, ) -> PyResult<*mut ffi::PyObject>; private_decl! {} } /// Initializer for Python native types, like `PyDict`. pub struct PyNativeTypeInitializer(PhantomData); impl PyObjectInit for PyNativeTypeInitializer { unsafe fn into_new_object( self, py: Python<'_>, subtype: *mut PyTypeObject, ) -> PyResult<*mut ffi::PyObject> { unsafe fn inner( py: Python<'_>, type_object: *mut PyTypeObject, subtype: *mut PyTypeObject, ) -> PyResult<*mut ffi::PyObject> { // HACK (due to FIXME below): PyBaseObject_Type's tp_new isn't happy with NULL arguments let is_base_object = type_object == std::ptr::addr_of_mut!(ffi::PyBaseObject_Type); if is_base_object { let alloc = get_tp_alloc(subtype).unwrap_or(ffi::PyType_GenericAlloc); let obj = alloc(subtype, 0); return if obj.is_null() { Err(PyErr::fetch(py)) } else { Ok(obj) }; } #[cfg(Py_LIMITED_API)] unreachable!("subclassing native types is not possible with the `abi3` feature"); #[cfg(not(Py_LIMITED_API))] { match (*type_object).tp_new { // FIXME: Call __new__ with actual arguments Some(newfunc) => { let obj = newfunc(subtype, std::ptr::null_mut(), std::ptr::null_mut()); if obj.is_null() { Err(PyErr::fetch(py)) } else { Ok(obj) } } None => Err(crate::exceptions::PyTypeError::new_err( "base type without tp_new", )), } } } let type_object = T::type_object_raw(py); inner(py, type_object, subtype) } private_impl! {} } /// Initializer for our `#[pyclass]` system. /// /// You can use this type to initialize complicatedly nested `#[pyclass]`. /// /// # Examples /// /// ``` /// # use pyo3::prelude::*; /// # use pyo3::py_run; /// #[pyclass(subclass)] /// struct BaseClass { /// #[pyo3(get)] /// basename: &'static str, /// } /// #[pyclass(extends=BaseClass, subclass)] /// struct SubClass { /// #[pyo3(get)] /// subname: &'static str, /// } /// #[pyclass(extends=SubClass)] /// struct SubSubClass { /// #[pyo3(get)] /// subsubname: &'static str, /// } /// /// #[pymethods] /// impl SubSubClass { /// #[new] /// fn new() -> PyClassInitializer { /// PyClassInitializer::from(BaseClass { basename: "base" }) /// .add_subclass(SubClass { subname: "sub" }) /// .add_subclass(SubSubClass { /// subsubname: "subsub", /// }) /// } /// } /// Python::with_gil(|py| { /// let typeobj = py.get_type::(); /// let sub_sub_class = typeobj.call((), None).unwrap(); /// py_run!( /// py, /// sub_sub_class, /// r#" /// assert sub_sub_class.basename == 'base' /// assert sub_sub_class.subname == 'sub' /// assert sub_sub_class.subsubname == 'subsub'"# /// ); /// }); /// ``` pub struct PyClassInitializer(PyClassInitializerImpl); enum PyClassInitializerImpl { Existing(Py), New { init: T, super_init: ::Initializer, }, } impl PyClassInitializer { /// Constructs a new initializer from value `T` and base class' initializer. /// /// It is recommended to use `add_subclass` instead of this method for most usage. pub fn new(init: T, super_init: ::Initializer) -> Self { Self(PyClassInitializerImpl::New { init, super_init }) } /// Constructs a new initializer from an initializer for the base class. /// /// # Examples /// ``` /// use pyo3::prelude::*; /// /// #[pyclass(subclass)] /// struct BaseClass { /// #[pyo3(get)] /// value: i32, /// } /// /// impl BaseClass { /// fn new(value: i32) -> PyResult { /// Ok(Self { value }) /// } /// } /// /// #[pyclass(extends=BaseClass)] /// struct SubClass {} /// /// #[pymethods] /// impl SubClass { /// #[new] /// fn new(value: i32) -> PyResult> { /// let base_init = PyClassInitializer::from(BaseClass::new(value)?); /// Ok(base_init.add_subclass(SubClass {})) /// } /// } /// /// fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// let m = PyModule::new(py, "example")?; /// m.add_class::()?; /// m.add_class::()?; /// /// let instance = m.getattr("SubClass")?.call1((92,))?; /// /// // `SubClass` does not have a `value` attribute, but `BaseClass` does. /// let n = instance.getattr("value")?.extract::()?; /// assert_eq!(n, 92); /// /// Ok(()) /// }) /// } /// ``` pub fn add_subclass(self, subclass_value: S) -> PyClassInitializer where S: PyClass, S::BaseType: PyClassBaseType, { PyClassInitializer::new(subclass_value, self) } /// Creates a new PyCell and initializes it. #[doc(hidden)] pub fn create_cell(self, py: Python<'_>) -> PyResult<*mut PyCell> where T: PyClass, { unsafe { self.create_cell_from_subtype(py, T::type_object_raw(py)) } } /// Creates a new PyCell and initializes it given a typeobject `subtype`. /// Called by the Python `tp_new` implementation generated by a `#[new]` function in a `#[pymethods]` block. /// /// # Safety /// `subtype` must be a valid pointer to the type object of T or a subclass. #[doc(hidden)] pub unsafe fn create_cell_from_subtype( self, py: Python<'_>, subtype: *mut crate::ffi::PyTypeObject, ) -> PyResult<*mut PyCell> where T: PyClass, { self.into_new_object(py, subtype).map(|obj| obj as _) } } impl PyObjectInit for PyClassInitializer { unsafe fn into_new_object( self, py: Python<'_>, subtype: *mut PyTypeObject, ) -> PyResult<*mut ffi::PyObject> { /// Layout of a PyCell after base new has been called, but the contents have not yet been /// written. #[repr(C)] struct PartiallyInitializedPyCell { _ob_base: ::LayoutAsBase, contents: MaybeUninit>, } let (init, super_init) = match self.0 { PyClassInitializerImpl::Existing(value) => return Ok(value.into_ptr()), PyClassInitializerImpl::New { init, super_init } => (init, super_init), }; let obj = super_init.into_new_object(py, subtype)?; let cell: *mut PartiallyInitializedPyCell = obj as _; std::ptr::write( (*cell).contents.as_mut_ptr(), PyCellContents { value: ManuallyDrop::new(UnsafeCell::new(init)), borrow_checker: ::Storage::new(), thread_checker: T::ThreadChecker::new(), dict: T::Dict::INIT, weakref: T::WeakRef::INIT, }, ); Ok(obj) } private_impl! {} } impl From for PyClassInitializer where T: PyClass, T::BaseType: PyClassBaseType>, { #[inline] fn from(value: T) -> PyClassInitializer { Self::new(value, PyNativeTypeInitializer(PhantomData)) } } impl From<(S, B)> for PyClassInitializer where S: PyClass, B: PyClass, B::BaseType: PyClassBaseType>, { fn from(sub_and_base: (S, B)) -> PyClassInitializer { let (sub, base) = sub_and_base; PyClassInitializer::from(base).add_subclass(sub) } } impl From> for PyClassInitializer { #[inline] fn from(value: Py) -> PyClassInitializer { PyClassInitializer(PyClassInitializerImpl::Existing(value)) } } // Implementation used by proc macros to allow anything convertible to PyClassInitializer to be // the return value of pyclass #[new] method (optionally wrapped in `Result`). impl IntoPyCallbackOutput> for U where T: PyClass, U: Into>, { #[inline] fn convert(self, _py: Python<'_>) -> PyResult> { Ok(self.into()) } } pyo3-0.20.2/src/sync.rs000064400000000000000000000245551046102023000127230ustar 00000000000000//! Synchronization mechanisms based on the Python GIL. use crate::{types::PyString, types::PyType, Py, PyErr, PyVisit, Python}; use std::cell::UnsafeCell; /// Value with concurrent access protected by the GIL. /// /// This is a synchronization primitive based on Python's global interpreter lock (GIL). /// It ensures that only one thread at a time can access the inner value via shared references. /// It can be combined with interior mutability to obtain mutable references. /// /// # Example /// /// Combining `GILProtected` with `RefCell` enables mutable access to static data: /// /// ``` /// # use pyo3::prelude::*; /// use pyo3::sync::GILProtected; /// use std::cell::RefCell; /// /// static NUMBERS: GILProtected>> = GILProtected::new(RefCell::new(Vec::new())); /// /// Python::with_gil(|py| { /// NUMBERS.get(py).borrow_mut().push(42); /// }); /// ``` pub struct GILProtected { value: T, } impl GILProtected { /// Place the given value under the protection of the GIL. pub const fn new(value: T) -> Self { Self { value } } /// Gain access to the inner value by giving proof of having acquired the GIL. pub fn get<'py>(&'py self, _py: Python<'py>) -> &'py T { &self.value } /// Gain access to the inner value by giving proof that garbage collection is happening. pub fn traverse<'py>(&'py self, _visit: PyVisit<'py>) -> &'py T { &self.value } } unsafe impl Sync for GILProtected where T: Send {} /// A write-once cell similar to [`once_cell::OnceCell`](https://docs.rs/once_cell/latest/once_cell/). /// /// Unlike `once_cell::sync` which blocks threads to achieve thread safety, this implementation /// uses the Python GIL to mediate concurrent access. This helps in cases where `once_cell` or /// `lazy_static`'s synchronization strategy can lead to deadlocks when interacting with the Python /// GIL. For an example, see [the FAQ section](https://pyo3.rs/latest/faq.html) of the guide. /// /// Note that: /// 1) `get_or_init` and `get_or_try_init` do not protect against infinite recursion /// from reentrant initialization. /// 2) If the initialization function `f` provided to `get_or_init` (or `get_or_try_init`) /// temporarily releases the GIL (e.g. by calling `Python::import`) then it is possible /// for a second thread to also begin initializing the `GITOnceCell`. Even when this /// happens `GILOnceCell` guarantees that only **one** write to the cell ever occurs /// - this is treated as a race, other threads will discard the value they compute and /// return the result of the first complete computation. /// /// # Examples /// /// The following example shows how to use `GILOnceCell` to share a reference to a Python list /// between threads: /// /// ``` /// use pyo3::sync::GILOnceCell; /// use pyo3::prelude::*; /// use pyo3::types::PyList; /// /// static LIST_CELL: GILOnceCell> = GILOnceCell::new(); /// /// pub fn get_shared_list(py: Python<'_>) -> &PyList { /// LIST_CELL /// .get_or_init(py, || PyList::empty(py).into()) /// .as_ref(py) /// } /// # Python::with_gil(|py| assert_eq!(get_shared_list(py).len(), 0)); /// ``` pub struct GILOnceCell(UnsafeCell>); // T: Send is needed for Sync because the thread which drops the GILOnceCell can be different // to the thread which fills it. unsafe impl Sync for GILOnceCell {} unsafe impl Send for GILOnceCell {} impl GILOnceCell { /// Create a `GILOnceCell` which does not yet contain a value. pub const fn new() -> Self { Self(UnsafeCell::new(None)) } /// Get a reference to the contained value, or `None` if the cell has not yet been written. #[inline] pub fn get(&self, _py: Python<'_>) -> Option<&T> { // Safe because if the cell has not yet been written, None is returned. unsafe { &*self.0.get() }.as_ref() } /// Get a reference to the contained value, initializing it if needed using the provided /// closure. /// /// See the type-level documentation for detail on re-entrancy and concurrent initialization. #[inline] pub fn get_or_init(&self, py: Python<'_>, f: F) -> &T where F: FnOnce() -> T, { if let Some(value) = self.get(py) { return value; } match self.init(py, || Ok::(f())) { Ok(value) => value, Err(void) => match void {}, } } /// Like `get_or_init`, but accepts a fallible initialization function. If it fails, the cell /// is left uninitialized. /// /// See the type-level documentation for detail on re-entrancy and concurrent initialization. #[inline] pub fn get_or_try_init(&self, py: Python<'_>, f: F) -> Result<&T, E> where F: FnOnce() -> Result, { if let Some(value) = self.get(py) { return Ok(value); } self.init(py, f) } #[cold] fn init(&self, py: Python<'_>, f: F) -> Result<&T, E> where F: FnOnce() -> Result, { // Note that f() could temporarily release the GIL, so it's possible that another thread // writes to this GILOnceCell before f() finishes. That's fine; we'll just have to discard // the value computed here and accept a bit of wasted computation. let value = f()?; let _ = self.set(py, value); Ok(self.get(py).unwrap()) } /// Get the contents of the cell mutably. This is only possible if the reference to the cell is /// unique. pub fn get_mut(&mut self) -> Option<&mut T> { self.0.get_mut().as_mut() } /// Set the value in the cell. /// /// If the cell has already been written, `Err(value)` will be returned containing the new /// value which was not written. pub fn set(&self, _py: Python<'_>, value: T) -> Result<(), T> { // Safe because GIL is held, so no other thread can be writing to this cell concurrently. let inner = unsafe { &mut *self.0.get() }; if inner.is_some() { return Err(value); } *inner = Some(value); Ok(()) } /// Takes the value out of the cell, moving it back to an uninitialized state. /// /// Has no effect and returns None if the cell has not yet been written. pub fn take(&mut self) -> Option { self.0.get_mut().take() } /// Consumes the cell, returning the wrapped value. /// /// Returns None if the cell has not yet been written. pub fn into_inner(self) -> Option { self.0.into_inner() } } impl GILOnceCell> { /// Get a reference to the contained Python type, initializing it if needed. /// /// This is a shorthand method for `get_or_init` which imports the type from Python on init. pub(crate) fn get_or_try_init_type_ref<'py>( &'py self, py: Python<'py>, module_name: &str, attr_name: &str, ) -> Result<&'py PyType, PyErr> { self.get_or_try_init(py, || py.import(module_name)?.getattr(attr_name)?.extract()) .map(|ty| ty.as_ref(py)) } } /// Interns `text` as a Python string and stores a reference to it in static storage. /// /// A reference to the same Python string is returned on each invocation. /// /// # Example: Using `intern!` to avoid needlessly recreating the same Python string /// /// ``` /// use pyo3::intern; /// # use pyo3::{pyfunction, types::PyDict, wrap_pyfunction, PyResult, Python}; /// /// #[pyfunction] /// fn create_dict(py: Python<'_>) -> PyResult<&PyDict> { /// let dict = PyDict::new(py); /// // 👇 A new `PyString` is created /// // for every call of this function. /// dict.set_item("foo", 42)?; /// Ok(dict) /// } /// /// #[pyfunction] /// fn create_dict_faster(py: Python<'_>) -> PyResult<&PyDict> { /// let dict = PyDict::new(py); /// // 👇 A `PyString` is created once and reused /// // for the lifetime of the program. /// dict.set_item(intern!(py, "foo"), 42)?; /// Ok(dict) /// } /// # /// # Python::with_gil(|py| { /// # let fun_slow = wrap_pyfunction!(create_dict, py).unwrap(); /// # let dict = fun_slow.call0().unwrap(); /// # assert!(dict.contains("foo").unwrap()); /// # let fun = wrap_pyfunction!(create_dict_faster, py).unwrap(); /// # let dict = fun.call0().unwrap(); /// # assert!(dict.contains("foo").unwrap()); /// # }); /// ``` #[macro_export] macro_rules! intern { ($py: expr, $text: expr) => {{ static INTERNED: $crate::sync::Interned = $crate::sync::Interned::new($text); INTERNED.get($py) }}; } /// Implementation detail for `intern!` macro. #[doc(hidden)] pub struct Interned(&'static str, GILOnceCell>); impl Interned { /// Creates an empty holder for an interned `str`. pub const fn new(value: &'static str) -> Self { Interned(value, GILOnceCell::new()) } /// Gets or creates the interned `str` value. #[inline] pub fn get<'py>(&'py self, py: Python<'py>) -> &'py PyString { self.1 .get_or_init(py, || PyString::intern(py, self.0).into()) .as_ref(py) } } #[cfg(test)] mod tests { use super::*; use crate::types::PyDict; #[test] fn test_intern() { Python::with_gil(|py| { let foo1 = "foo"; let foo2 = intern!(py, "foo"); let foo3 = intern!(py, stringify!(foo)); let dict = PyDict::new(py); dict.set_item(foo1, 42_usize).unwrap(); assert!(dict.contains(foo2).unwrap()); assert_eq!( dict.get_item(foo3) .unwrap() .unwrap() .extract::() .unwrap(), 42 ); }); } #[test] fn test_once_cell() { Python::with_gil(|py| { let mut cell = GILOnceCell::new(); assert!(cell.get(py).is_none()); assert_eq!(cell.get_or_try_init(py, || Err(5)), Err(5)); assert!(cell.get(py).is_none()); assert_eq!(cell.get_or_try_init(py, || Ok::<_, ()>(2)), Ok(&2)); assert_eq!(cell.get(py), Some(&2)); assert_eq!(cell.get_or_try_init(py, || Err(5)), Ok(&2)); assert_eq!(cell.take(), Some(2)); assert_eq!(cell.into_inner(), None) }) } } pyo3-0.20.2/src/test_utils.rs000064400000000000000000000000631046102023000141320ustar 00000000000000use crate as pyo3; include!("../tests/common.rs"); pyo3-0.20.2/src/tests/common.rs000064400000000000000000000126421046102023000143730ustar 00000000000000// the inner mod enables the #![allow(dead_code)] to // be applied - `test_utils.rs` uses `include!` to pull in this file /// Common macros and helpers for tests #[allow(dead_code)] // many tests do not use the complete set of functionality offered here #[macro_use] mod inner { #[allow(unused_imports)] // pulls in `use crate as pyo3` in `test_utils.rs` use super::*; use pyo3::prelude::*; use pyo3::types::{IntoPyDict, PyList}; #[macro_export] macro_rules! py_assert { ($py:expr, $($val:ident)+, $assertion:literal) => { pyo3::py_run!($py, $($val)+, concat!("assert ", $assertion)) }; ($py:expr, *$dict:expr, $assertion:literal) => { pyo3::py_run!($py, *$dict, concat!("assert ", $assertion)) }; } #[macro_export] macro_rules! py_expect_exception { // Case1: idents & no err_msg ($py:expr, $($val:ident)+, $code:expr, $err:ident) => {{ use pyo3::types::IntoPyDict; let d = [$((stringify!($val), $val.to_object($py)),)+].into_py_dict($py); py_expect_exception!($py, *d, $code, $err) }}; // Case2: dict & no err_msg ($py:expr, *$dict:expr, $code:expr, $err:ident) => {{ let res = $py.run($code, None, Some($dict)); let err = res.expect_err(&format!("Did not raise {}", stringify!($err))); if !err.matches($py, $py.get_type::()) { panic!("Expected {} but got {:?}", stringify!($err), err) } err }}; // Case3: idents & err_msg ($py:expr, $($val:ident)+, $code:expr, $err:ident, $err_msg:literal) => {{ let err = py_expect_exception!($py, $($val)+, $code, $err); // Suppose that the error message looks like 'TypeError: ~' assert_eq!(format!("Py{}", err), concat!(stringify!($err), ": ", $err_msg)); err }}; // Case4: dict & err_msg ($py:expr, *$dict:expr, $code:expr, $err:ident, $err_msg:literal) => {{ let err = py_expect_exception!($py, *$dict, $code, $err); assert_eq!(format!("Py{}", err), concat!(stringify!($err), ": ", $err_msg)); err }}; } // sys.unraisablehook not available until Python 3.8 #[cfg(all(feature = "macros", Py_3_8))] #[pyclass(crate = "pyo3")] pub struct UnraisableCapture { pub capture: Option<(PyErr, PyObject)>, old_hook: Option, } #[cfg(all(feature = "macros", Py_3_8))] #[pymethods(crate = "pyo3")] impl UnraisableCapture { pub fn hook(&mut self, unraisable: &PyAny) { let err = PyErr::from_value(unraisable.getattr("exc_value").unwrap()); let instance = unraisable.getattr("object").unwrap(); self.capture = Some((err, instance.into())); } } #[cfg(all(feature = "macros", Py_3_8))] impl UnraisableCapture { pub fn install(py: Python<'_>) -> Py { let sys = py.import("sys").unwrap(); let old_hook = sys.getattr("unraisablehook").unwrap().into(); let capture = Py::new( py, UnraisableCapture { capture: None, old_hook: Some(old_hook), }, ) .unwrap(); sys.setattr("unraisablehook", capture.getattr(py, "hook").unwrap()) .unwrap(); capture } pub fn uninstall(&mut self, py: Python<'_>) { let old_hook = self.old_hook.take().unwrap(); let sys = py.import("sys").unwrap(); sys.setattr("unraisablehook", old_hook).unwrap(); } } pub struct CatchWarnings<'py> { catch_warnings: &'py PyAny, } impl<'py> CatchWarnings<'py> { pub fn enter(py: Python<'py>, f: impl FnOnce(&PyList) -> PyResult) -> PyResult { let warnings = py.import("warnings")?; let kwargs = [("record", true)].into_py_dict(py); let catch_warnings = warnings.getattr("catch_warnings")?.call((), Some(kwargs))?; let list = catch_warnings.call_method0("__enter__")?.extract()?; let _guard = Self { catch_warnings }; f(list) } } impl Drop for CatchWarnings<'_> { fn drop(&mut self) { let py = self.catch_warnings.py(); self.catch_warnings .call_method1("__exit__", (py.None(), py.None(), py.None())) .unwrap(); } } #[macro_export] macro_rules! assert_warnings { ($py:expr, $body:expr, [$(($category:ty, $message:literal)),+] $(,)? ) => {{ CatchWarnings::enter($py, |w| { $body; let expected_warnings = [$((<$category>::type_object($py), $message)),+]; assert_eq!(w.len(), expected_warnings.len()); for (warning, (category, message)) in w.iter().zip(expected_warnings) { assert!(warning.getattr("category").unwrap().is(category)); assert_eq!( warning.getattr("message").unwrap().str().unwrap().to_string_lossy(), message ); } Ok(()) }) .unwrap(); }}; } } #[allow(unused_imports)] // some tests use just the macros and none of the other functionality pub use inner::*; pyo3-0.20.2/src/tests/hygiene/misc.rs000064400000000000000000000021631046102023000154630ustar 00000000000000#![no_implicit_prelude] #[derive(crate::FromPyObject)] #[pyo3(crate = "crate")] struct Derive1(i32); // newtype case #[derive(crate::FromPyObject)] #[pyo3(crate = "crate")] #[allow(dead_code)] struct Derive2(i32, i32); // tuple case #[derive(crate::FromPyObject)] #[pyo3(crate = "crate")] #[allow(dead_code)] struct Derive3 { f: i32, g: i32, } // struct case #[derive(crate::FromPyObject)] #[pyo3(crate = "crate")] #[allow(dead_code)] enum Derive4 { A(i32), B { f: i32 }, } // enum case crate::create_exception!(mymodule, CustomError, crate::exceptions::PyException); crate::import_exception!(socket, gaierror); #[allow(dead_code)] fn intern(py: crate::Python<'_>) { let _foo = crate::intern!(py, "foo"); let _bar = crate::intern!(py, stringify!(bar)); } #[allow(dead_code)] #[cfg(not(PyPy))] fn append_to_inittab() { #[crate::pymodule] #[pyo3(crate = "crate")] #[allow(clippy::unnecessary_wraps)] fn module_for_inittab(_: crate::Python<'_>, _: &crate::types::PyModule) -> crate::PyResult<()> { ::std::result::Result::Ok(()) } crate::append_to_inittab!(module_for_inittab); } pyo3-0.20.2/src/tests/hygiene/mod.rs000064400000000000000000000005371046102023000153120ustar 00000000000000// The modules in this test are used to check PyO3 macro expansion is hygienic. By locating the test // inside the crate the global `::pyo3` namespace is not available, so in combination with // #[pyo3(crate = "crate")] this validates that all macro expansion respects the setting. mod misc; mod pyclass; mod pyfunction; mod pymethods; mod pymodule; pyo3-0.20.2/src/tests/hygiene/pyclass.rs000064400000000000000000000017571046102023000162160ustar 00000000000000#![no_implicit_prelude] #![allow(unused_variables)] #[crate::pyclass] #[pyo3(crate = "crate")] #[derive(::std::clone::Clone)] pub struct Foo; #[crate::pyclass] #[pyo3(crate = "crate")] pub struct Foo2; #[crate::pyclass( name = "ActuallyBar", freelist = 8, weakref, unsendable, subclass, extends = crate::types::PyAny, module = "Spam" )] #[pyo3(crate = "crate")] pub struct Bar { #[pyo3(get, set)] a: u8, #[pyo3(get, set)] b: Foo, #[pyo3(get, set)] c: ::std::option::Option>, } #[crate::pyclass] #[pyo3(crate = "crate")] pub enum Enum { Var0, } #[crate::pyclass] #[pyo3(crate = "crate")] pub struct Foo3 { #[pyo3(get, set)] #[cfg(FALSE)] field: i32, #[pyo3(get, set)] #[cfg(not(FALSE))] field: u32, } #[crate::pyclass] #[pyo3(crate = "crate")] pub struct Foo4 { #[pyo3(get, set)] #[cfg(FALSE)] #[cfg(not(FALSE))] field: i32, #[pyo3(get, set)] #[cfg(not(FALSE))] field: u32, } pyo3-0.20.2/src/tests/hygiene/pyfunction.rs000064400000000000000000000006361046102023000167310ustar 00000000000000#![no_implicit_prelude] #![allow(unused_variables, clippy::unnecessary_wraps)] #[crate::pyfunction] #[pyo3(crate = "crate")] fn do_something(x: i32) -> crate::PyResult { ::std::result::Result::Ok(x) } #[test] fn invoke_wrap_pyfunction() { crate::Python::with_gil(|py| { let func = crate::wrap_pyfunction!(do_something)(py).unwrap(); crate::py_run!(py, func, r#"func(5)"#); }); } pyo3-0.20.2/src/tests/hygiene/pymethods.rs000064400000000000000000000441111046102023000165430ustar 00000000000000#![no_implicit_prelude] #![allow(unused_variables, clippy::unnecessary_wraps)] #[crate::pyclass] #[pyo3(crate = "crate")] pub struct Dummy; #[crate::pyclass] #[pyo3(crate = "crate")] pub struct DummyIter; #[cfg(Py_3_8)] #[crate::pymethods] #[pyo3(crate = "crate")] impl Dummy { ////////////////////// // Basic customization ////////////////////// fn __repr__(&self) -> &'static str { "Dummy" } fn __str__(&self) -> &'static str { "Dummy" } fn __bytes__<'py>(&self, py: crate::Python<'py>) -> &'py crate::types::PyBytes { crate::types::PyBytes::new(py, &[0]) } fn __format__(&self, format_spec: ::std::string::String) -> ::std::string::String { ::std::panic!("unimplemented isn't hygienic before 1.50") } fn __lt__(&self, other: &Self) -> bool { false } fn __le__(&self, other: &Self) -> bool { false } fn __eq__(&self, other: &Self) -> bool { false } fn __ne__(&self, other: &Self) -> bool { false } fn __gt__(&self, other: &Self) -> bool { false } fn __ge__(&self, other: &Self) -> bool { false } fn __hash__(&self) -> u64 { 42 } fn __bool__(&self) -> bool { true } ////////////////////// // Customizing attribute access ////////////////////// fn __getattr__(&self, name: ::std::string::String) -> &crate::PyAny { ::std::panic!("unimplemented isn't hygienic before 1.50") } fn __getattribute__(&self, name: ::std::string::String) -> &crate::PyAny { ::std::panic!("unimplemented isn't hygienic before 1.50") } fn __setattr__(&mut self, name: ::std::string::String, value: ::std::string::String) {} fn __delattr__(&mut self, name: ::std::string::String) {} fn __dir__<'py>(&self, py: crate::Python<'py>) -> &'py crate::types::PyList { crate::types::PyList::new(py, ::std::vec![0_u8]) } ////////////////////// // Implementing Descriptors ////////////////////// fn __get__( &self, instance: &crate::PyAny, owner: &crate::PyAny, ) -> crate::PyResult<&crate::PyAny> { ::std::panic!("unimplemented isn't hygienic before 1.50") } fn __set__(&self, instance: &crate::PyAny, owner: &crate::PyAny) {} fn __delete__(&self, instance: &crate::PyAny) {} fn __set_name__(&self, owner: &crate::PyAny, name: &crate::PyAny) {} ////////////////////// // Implementing Descriptors ////////////////////// fn __len__(&self) -> usize { 0 } fn __getitem__(&self, key: u32) -> crate::PyResult { ::std::result::Result::Err(crate::exceptions::PyKeyError::new_err("boo")) } fn __setitem__(&self, key: u32, value: u32) {} fn __delitem__(&self, key: u32) {} fn __iter__(_: crate::pycell::PyRef<'_, Self>, py: crate::Python<'_>) -> crate::Py { crate::Py::new(py, DummyIter {}).unwrap() } fn __next__(&mut self) -> ::std::option::Option<()> { ::std::option::Option::None } fn __reversed__( slf: crate::pycell::PyRef<'_, Self>, py: crate::Python<'_>, ) -> crate::Py { crate::Py::new(py, DummyIter {}).unwrap() } fn __contains__(&self, item: u32) -> bool { false } ////////////////////// // Emulating numeric types ////////////////////// fn __add__(&self, other: &Self) -> Dummy { Dummy {} } fn __sub__(&self, other: &Self) -> Dummy { Dummy {} } fn __mul__(&self, other: &Self) -> Dummy { Dummy {} } fn __truediv__(&self, _other: &Self) -> crate::PyResult<()> { ::std::result::Result::Err(crate::exceptions::PyZeroDivisionError::new_err("boo")) } fn __floordiv__(&self, _other: &Self) -> crate::PyResult<()> { ::std::result::Result::Err(crate::exceptions::PyZeroDivisionError::new_err("boo")) } fn __mod__(&self, _other: &Self) -> u32 { 0 } fn __divmod__(&self, _other: &Self) -> (u32, u32) { (0, 0) } fn __pow__(&self, _other: &Self, modulo: ::std::option::Option) -> Dummy { Dummy {} } fn __lshift__(&self, other: &Self) -> Dummy { Dummy {} } fn __rshift__(&self, other: &Self) -> Dummy { Dummy {} } fn __and__(&self, other: &Self) -> Dummy { Dummy {} } fn __xor__(&self, other: &Self) -> Dummy { Dummy {} } fn __or__(&self, other: &Self) -> Dummy { Dummy {} } fn __radd__(&self, other: &Self) -> Dummy { Dummy {} } fn __rrsub__(&self, other: &Self) -> Dummy { Dummy {} } fn __rmul__(&self, other: &Self) -> Dummy { Dummy {} } fn __rtruediv__(&self, _other: &Self) -> crate::PyResult<()> { ::std::result::Result::Err(crate::exceptions::PyZeroDivisionError::new_err("boo")) } fn __rfloordiv__(&self, _other: &Self) -> crate::PyResult<()> { ::std::result::Result::Err(crate::exceptions::PyZeroDivisionError::new_err("boo")) } fn __rmod__(&self, _other: &Self) -> u32 { 0 } fn __rdivmod__(&self, _other: &Self) -> (u32, u32) { (0, 0) } fn __rpow__(&self, _other: &Self, modulo: ::std::option::Option) -> Dummy { Dummy {} } fn __rlshift__(&self, other: &Self) -> Dummy { Dummy {} } fn __rrshift__(&self, other: &Self) -> Dummy { Dummy {} } fn __rand__(&self, other: &Self) -> Dummy { Dummy {} } fn __rxor__(&self, other: &Self) -> Dummy { Dummy {} } fn __ror__(&self, other: &Self) -> Dummy { Dummy {} } fn __iadd__(&mut self, other: &Self) {} fn __irsub__(&mut self, other: &Self) {} fn __imul__(&mut self, other: &Self) {} fn __itruediv__(&mut self, _other: &Self) {} fn __ifloordiv__(&mut self, _other: &Self) {} fn __imod__(&mut self, _other: &Self) {} fn __ipow__(&mut self, _other: &Self, modulo: ::std::option::Option) {} fn __ilshift__(&mut self, other: &Self) {} fn __irshift__(&mut self, other: &Self) {} fn __iand__(&mut self, other: &Self) {} fn __ixor__(&mut self, other: &Self) {} fn __ior__(&mut self, other: &Self) {} fn __neg__(slf: crate::pycell::PyRef<'_, Self>) -> crate::pycell::PyRef<'_, Self> { slf } fn __pos__(slf: crate::pycell::PyRef<'_, Self>) -> crate::pycell::PyRef<'_, Self> { slf } fn __abs__(slf: crate::pycell::PyRef<'_, Self>) -> crate::pycell::PyRef<'_, Self> { slf } fn __invert__(slf: crate::pycell::PyRef<'_, Self>) -> crate::pycell::PyRef<'_, Self> { slf } fn __complex__<'py>(&self, py: crate::Python<'py>) -> &'py crate::types::PyComplex { crate::types::PyComplex::from_doubles(py, 0.0, 0.0) } fn __int__(&self) -> u32 { 0 } fn __float__(&self) -> f64 { 0.0 } fn __index__(&self) -> u32 { 0 } fn __round__(&self, ndigits: ::std::option::Option) -> u32 { 0 } fn __trunc__(&self) -> u32 { 0 } fn __floor__(&self) -> u32 { 0 } fn __ceil__(&self) -> u32 { 0 } ////////////////////// // With Statement Context Managers ////////////////////// fn __enter__(&mut self) {} fn __exit__( &mut self, exc_type: &crate::PyAny, exc_value: &crate::PyAny, traceback: &crate::PyAny, ) { } ////////////////////// // Awaitable Objects ////////////////////// fn __await__(slf: crate::pycell::PyRef<'_, Self>) -> crate::pycell::PyRef<'_, Self> { slf } ////////////////////// // Asynchronous Iterators ////////////////////// fn __aiter__( slf: crate::pycell::PyRef<'_, Self>, py: crate::Python<'_>, ) -> crate::Py { crate::Py::new(py, DummyIter {}).unwrap() } fn __anext__(&mut self) -> ::std::option::Option<()> { ::std::option::Option::None } ////////////////////// // Asynchronous Context Managers ////////////////////// fn __aenter__(&mut self) {} fn __aexit__( &mut self, exc_type: &crate::PyAny, exc_value: &crate::PyAny, traceback: &crate::PyAny, ) { } // Things with attributes #[pyo3(signature = (_y, *, _z=2))] fn test(&self, _y: &Dummy, _z: i32) {} #[staticmethod] fn staticmethod() {} #[classmethod] fn clsmethod(_: &crate::types::PyType) {} #[pyo3(signature = (*_args, **_kwds))] fn __call__( &self, _args: &crate::types::PyTuple, _kwds: ::std::option::Option<&crate::types::PyDict>, ) -> crate::PyResult { ::std::panic!("unimplemented isn't hygienic before 1.50") } #[new] fn new(a: u8) -> Self { Dummy {} } #[getter] fn get(&self) -> i32 { 0 } #[setter] fn set(&mut self, _v: i32) {} #[classattr] fn class_attr() -> i32 { 0 } // Dunder methods invented for protocols // PyGcProtocol // Buffer protocol? } #[cfg(not(Py_3_8))] #[crate::pymethods] #[pyo3(crate = "crate")] impl Dummy { ////////////////////// // Basic customization ////////////////////// fn __repr__(&self) -> &'static str { "Dummy" } fn __str__(&self) -> &'static str { "Dummy" } fn __bytes__<'py>(&self, py: crate::Python<'py>) -> &'py crate::types::PyBytes { crate::types::PyBytes::new(py, &[0]) } fn __format__(&self, format_spec: ::std::string::String) -> ::std::string::String { ::std::panic!("unimplemented isn't hygienic before 1.50") } fn __lt__(&self, other: &Self) -> bool { false } fn __le__(&self, other: &Self) -> bool { false } fn __eq__(&self, other: &Self) -> bool { false } fn __ne__(&self, other: &Self) -> bool { false } fn __gt__(&self, other: &Self) -> bool { false } fn __ge__(&self, other: &Self) -> bool { false } fn __hash__(&self) -> u64 { 42 } fn __bool__(&self) -> bool { true } ////////////////////// // Customizing attribute access ////////////////////// fn __getattr__(&self, name: ::std::string::String) -> &crate::PyAny { ::std::panic!("unimplemented isn't hygienic before 1.50") } fn __getattribute__(&self, name: ::std::string::String) -> &crate::PyAny { ::std::panic!("unimplemented isn't hygienic before 1.50") } fn __setattr__(&mut self, name: ::std::string::String, value: ::std::string::String) {} fn __delattr__(&mut self, name: ::std::string::String) {} fn __dir__<'py>(&self, py: crate::Python<'py>) -> &'py crate::types::PyList { crate::types::PyList::new(py, ::std::vec![0_u8]) } ////////////////////// // Implementing Descriptors ////////////////////// fn __get__( &self, instance: &crate::PyAny, owner: &crate::PyAny, ) -> crate::PyResult<&crate::PyAny> { ::std::panic!("unimplemented isn't hygienic before 1.50") } fn __set__(&self, instance: &crate::PyAny, owner: &crate::PyAny) {} fn __delete__(&self, instance: &crate::PyAny) {} fn __set_name__(&self, owner: &crate::PyAny, name: &crate::PyAny) {} ////////////////////// // Implementing Descriptors ////////////////////// fn __len__(&self) -> usize { 0 } fn __getitem__(&self, key: u32) -> crate::PyResult { ::std::result::Result::Err(crate::exceptions::PyKeyError::new_err("boo")) } fn __setitem__(&self, key: u32, value: u32) {} fn __delitem__(&self, key: u32) {} fn __iter__(_: crate::pycell::PyRef<'_, Self>, py: crate::Python<'_>) -> crate::Py { crate::Py::new(py, DummyIter {}).unwrap() } fn __next__(&mut self) -> ::std::option::Option<()> { ::std::option::Option::None } fn __reversed__( slf: crate::pycell::PyRef<'_, Self>, py: crate::Python<'_>, ) -> crate::Py { crate::Py::new(py, DummyIter {}).unwrap() } fn __contains__(&self, item: u32) -> bool { false } ////////////////////// // Emulating numeric types ////////////////////// fn __add__(&self, other: &Self) -> Dummy { Dummy {} } fn __sub__(&self, other: &Self) -> Dummy { Dummy {} } fn __mul__(&self, other: &Self) -> Dummy { Dummy {} } fn __truediv__(&self, _other: &Self) -> crate::PyResult<()> { ::std::result::Result::Err(crate::exceptions::PyZeroDivisionError::new_err("boo")) } fn __floordiv__(&self, _other: &Self) -> crate::PyResult<()> { ::std::result::Result::Err(crate::exceptions::PyZeroDivisionError::new_err("boo")) } fn __mod__(&self, _other: &Self) -> u32 { 0 } fn __divmod__(&self, _other: &Self) -> (u32, u32) { (0, 0) } fn __pow__(&self, _other: &Self, modulo: ::std::option::Option) -> Dummy { Dummy {} } fn __lshift__(&self, other: &Self) -> Dummy { Dummy {} } fn __rshift__(&self, other: &Self) -> Dummy { Dummy {} } fn __and__(&self, other: &Self) -> Dummy { Dummy {} } fn __xor__(&self, other: &Self) -> Dummy { Dummy {} } fn __or__(&self, other: &Self) -> Dummy { Dummy {} } fn __radd__(&self, other: &Self) -> Dummy { Dummy {} } fn __rrsub__(&self, other: &Self) -> Dummy { Dummy {} } fn __rmul__(&self, other: &Self) -> Dummy { Dummy {} } fn __rtruediv__(&self, _other: &Self) -> crate::PyResult<()> { ::std::result::Result::Err(crate::exceptions::PyZeroDivisionError::new_err("boo")) } fn __rfloordiv__(&self, _other: &Self) -> crate::PyResult<()> { ::std::result::Result::Err(crate::exceptions::PyZeroDivisionError::new_err("boo")) } fn __rmod__(&self, _other: &Self) -> u32 { 0 } fn __rdivmod__(&self, _other: &Self) -> (u32, u32) { (0, 0) } fn __rpow__(&self, _other: &Self, modulo: ::std::option::Option) -> Dummy { Dummy {} } fn __rlshift__(&self, other: &Self) -> Dummy { Dummy {} } fn __rrshift__(&self, other: &Self) -> Dummy { Dummy {} } fn __rand__(&self, other: &Self) -> Dummy { Dummy {} } fn __rxor__(&self, other: &Self) -> Dummy { Dummy {} } fn __ror__(&self, other: &Self) -> Dummy { Dummy {} } fn __iadd__(&mut self, other: &Self) {} fn __irsub__(&mut self, other: &Self) {} fn __imul__(&mut self, other: &Self) {} fn __itruediv__(&mut self, _other: &Self) {} fn __ifloordiv__(&mut self, _other: &Self) {} fn __imod__(&mut self, _other: &Self) {} fn __ipow__(&mut self, _other: &Self, _modulo: ::std::option::Option) {} fn __ilshift__(&mut self, other: &Self) {} fn __irshift__(&mut self, other: &Self) {} fn __iand__(&mut self, other: &Self) {} fn __ixor__(&mut self, other: &Self) {} fn __ior__(&mut self, other: &Self) {} fn __neg__(slf: crate::pycell::PyRef<'_, Self>) -> crate::pycell::PyRef<'_, Self> { slf } fn __pos__(slf: crate::pycell::PyRef<'_, Self>) -> crate::pycell::PyRef<'_, Self> { slf } fn __abs__(slf: crate::pycell::PyRef<'_, Self>) -> crate::pycell::PyRef<'_, Self> { slf } fn __invert__(slf: crate::pycell::PyRef<'_, Self>) -> crate::pycell::PyRef<'_, Self> { slf } fn __complex__<'py>(&self, py: crate::Python<'py>) -> &'py crate::types::PyComplex { crate::types::PyComplex::from_doubles(py, 0.0, 0.0) } fn __int__(&self) -> u32 { 0 } fn __float__(&self) -> f64 { 0.0 } fn __index__(&self) -> u32 { 0 } fn __round__(&self, ndigits: ::std::option::Option) -> u32 { 0 } fn __trunc__(&self) -> u32 { 0 } fn __floor__(&self) -> u32 { 0 } fn __ceil__(&self) -> u32 { 0 } ////////////////////// // With Statement Context Managers ////////////////////// fn __enter__(&mut self) {} fn __exit__( &mut self, exc_type: &crate::PyAny, exc_value: &crate::PyAny, traceback: &crate::PyAny, ) { } ////////////////////// // Awaitable Objects ////////////////////// fn __await__(slf: crate::pycell::PyRef<'_, Self>) -> crate::pycell::PyRef<'_, Self> { slf } ////////////////////// // Asynchronous Iterators ////////////////////// fn __aiter__( slf: crate::pycell::PyRef<'_, Self>, py: crate::Python<'_>, ) -> crate::Py { crate::Py::new(py, DummyIter {}).unwrap() } fn __anext__(&mut self) -> ::std::option::Option<()> { ::std::option::Option::None } ////////////////////// // Asynchronous Context Managers ////////////////////// fn __aenter__(&mut self) {} fn __aexit__( &mut self, exc_type: &crate::PyAny, exc_value: &crate::PyAny, traceback: &crate::PyAny, ) { } // Things with attributes #[pyo3(signature = (_y, *, _z=2))] fn test(&self, _y: &Dummy, _z: i32) {} #[staticmethod] fn staticmethod() {} #[classmethod] fn clsmethod(_: &crate::types::PyType) {} #[pyo3(signature = (*_args, **_kwds))] fn __call__( &self, _args: &crate::types::PyTuple, _kwds: ::std::option::Option<&crate::types::PyDict>, ) -> crate::PyResult { ::std::panic!("unimplemented isn't hygienic before 1.50") } #[new] fn new(a: u8) -> Self { Dummy {} } #[getter] fn get(&self) -> i32 { 0 } #[setter] fn set(&mut self, _v: i32) {} #[classattr] fn class_attr() -> i32 { 0 } // Dunder methods invented for protocols // PyGcProtocol // Buffer protocol? } // Ensure that crate argument is also accepted inline #[crate::pyclass(crate = "crate")] struct Dummy2; #[crate::pymethods(crate = "crate")] impl Dummy2 {} pyo3-0.20.2/src/tests/hygiene/pymodule.rs000064400000000000000000000012251046102023000163640ustar 00000000000000#![no_implicit_prelude] #![allow(unused_variables, clippy::unnecessary_wraps)] #[crate::pyfunction] #[pyo3(crate = "crate")] fn do_something(x: i32) -> crate::PyResult { ::std::result::Result::Ok(x) } #[crate::pymodule] #[pyo3(crate = "crate")] fn foo(_py: crate::Python<'_>, _m: &crate::types::PyModule) -> crate::PyResult<()> { ::std::result::Result::Ok(()) } #[crate::pymodule] #[pyo3(crate = "crate")] fn my_module(_py: crate::Python<'_>, m: &crate::types::PyModule) -> crate::PyResult<()> { m.add_function(crate::wrap_pyfunction!(do_something, m)?)?; m.add_wrapped(crate::wrap_pymodule!(foo))?; ::std::result::Result::Ok(()) } pyo3-0.20.2/src/tests/mod.rs000064400000000000000000000003711046102023000136560ustar 00000000000000#[macro_use] pub(crate) mod common { use crate as pyo3; include!("./common.rs"); } /// Test macro hygiene - this is in the crate since we won't have /// `pyo3` available in the crate root. #[cfg(all(test, feature = "macros"))] mod hygiene; pyo3-0.20.2/src/type_object.rs000064400000000000000000000057671046102023000142620ustar 00000000000000//! Python type object information use crate::types::{PyAny, PyType}; use crate::{ffi, PyNativeType, Python}; /// `T: PyLayout` represents that `T` is a concrete representation of `U` in the Python heap. /// E.g., `PyCell` is a concrete representation of all `pyclass`es, and `ffi::PyObject` /// is of `PyAny`. /// /// This trait is intended to be used internally. /// /// # Safety /// /// This trait must only be implemented for types which represent valid layouts of Python objects. pub unsafe trait PyLayout {} /// `T: PySizedLayout` represents that `T` is not a instance of /// [`PyVarObject`](https://docs.python.org/3.8/c-api/structures.html?highlight=pyvarobject#c.PyVarObject). /// In addition, that `T` is a concrete representation of `U`. pub trait PySizedLayout: PyLayout + Sized {} /// Python type information. /// All Python native types (e.g., `PyDict`) and `#[pyclass]` structs implement this trait. /// /// This trait is marked unsafe because: /// - specifying the incorrect layout can lead to memory errors /// - the return value of type_object must always point to the same PyTypeObject instance /// /// It is safely implemented by the `pyclass` macro. /// /// # Safety /// /// Implementations must provide an implementation for `type_object_raw` which infallibly produces a /// non-null pointer to the corresponding Python type object. pub unsafe trait PyTypeInfo: Sized { /// Class name. const NAME: &'static str; /// Module name, if any. const MODULE: Option<&'static str>; /// Utility type to make Py::as_ref work. type AsRefTarget: PyNativeType; /// Returns the PyTypeObject instance for this type. fn type_object_raw(py: Python<'_>) -> *mut ffi::PyTypeObject; /// Returns the safe abstraction over the type object. #[inline] fn type_object(py: Python<'_>) -> &PyType { unsafe { py.from_borrowed_ptr(Self::type_object_raw(py) as _) } } /// Checks if `object` is an instance of this type or a subclass of this type. #[inline] fn is_type_of(object: &PyAny) -> bool { unsafe { ffi::PyObject_TypeCheck(object.as_ptr(), Self::type_object_raw(object.py())) != 0 } } /// Checks if `object` is an instance of this type. #[inline] fn is_exact_type_of(object: &PyAny) -> bool { unsafe { ffi::Py_TYPE(object.as_ptr()) == Self::type_object_raw(object.py()) } } } #[inline] pub(crate) unsafe fn get_tp_alloc(tp: *mut ffi::PyTypeObject) -> Option { #[cfg(not(Py_LIMITED_API))] { (*tp).tp_alloc } #[cfg(Py_LIMITED_API)] { let ptr = ffi::PyType_GetSlot(tp, ffi::Py_tp_alloc); std::mem::transmute(ptr) } } #[inline] pub(crate) unsafe fn get_tp_free(tp: *mut ffi::PyTypeObject) -> ffi::freefunc { #[cfg(not(Py_LIMITED_API))] { (*tp).tp_free.unwrap() } #[cfg(Py_LIMITED_API)] { let ptr = ffi::PyType_GetSlot(tp, ffi::Py_tp_free); debug_assert_ne!(ptr, std::ptr::null_mut()); std::mem::transmute(ptr) } } pyo3-0.20.2/src/types/any.rs000064400000000000000000001363011046102023000136730ustar 00000000000000use crate::class::basic::CompareOp; use crate::conversion::{AsPyPointer, FromPyObject, IntoPy, PyTryFrom, ToPyObject}; use crate::err::{PyDowncastError, PyErr, PyResult}; use crate::exceptions::{PyAttributeError, PyTypeError}; use crate::type_object::PyTypeInfo; #[cfg(not(PyPy))] use crate::types::PySuper; use crate::types::{PyDict, PyIterator, PyList, PyString, PyTuple, PyType}; use crate::{err, ffi, Py, PyNativeType, PyObject, Python}; use std::cell::UnsafeCell; use std::cmp::Ordering; use std::os::raw::c_int; /// Represents any Python object. /// /// It currently only appears as a *reference*, `&PyAny`, /// with a lifetime that represents the scope during which the GIL is held. /// /// `PyAny` has some interesting properties, which it shares /// with the other [native Python types](crate::types): /// /// - It can only be obtained and used while the GIL is held, /// therefore its API does not require a [`Python<'py>`](crate::Python) token. /// - It can't be used in situations where the GIL is temporarily released, /// such as [`Python::allow_threads`](crate::Python::allow_threads)'s closure. /// - The underlying Python object, if mutable, can be mutated through any reference. /// - It can be converted to the GIL-independent [`Py`]`<`[`PyAny`]`>`, /// allowing it to outlive the GIL scope. However, using [`Py`]`<`[`PyAny`]`>`'s API /// *does* require a [`Python<'py>`](crate::Python) token. /// /// It can be cast to a concrete type with PyAny::downcast (for native Python types only) /// and FromPyObject::extract. See their documentation for more information. /// /// See [the guide](https://pyo3.rs/latest/types.html) for an explanation /// of the different Python object types. #[repr(transparent)] pub struct PyAny(UnsafeCell); unsafe impl AsPyPointer for PyAny { #[inline] fn as_ptr(&self) -> *mut ffi::PyObject { self.0.get() } } #[allow(non_snake_case)] // Copied here as the macro does not accept deprecated functions. // Originally ffi::object::PyObject_Check, but this is not in the Python C API. fn PyObject_Check(_: *mut ffi::PyObject) -> c_int { 1 } pyobject_native_type_base!(PyAny); pyobject_native_type_info!( PyAny, pyobject_native_static_type_object!(ffi::PyBaseObject_Type), Some("builtins"), #checkfunction=PyObject_Check ); pyobject_native_type_extract!(PyAny); pyobject_native_type_sized!(PyAny, ffi::PyObject); impl PyAny { /// Returns whether `self` and `other` point to the same object. To compare /// the equality of two objects (the `==` operator), use [`eq`](PyAny::eq). /// /// This is equivalent to the Python expression `self is other`. #[inline] pub fn is(&self, other: &T) -> bool { self.as_ptr() == other.as_ptr() } /// Determines whether this object has the given attribute. /// /// This is equivalent to the Python expression `hasattr(self, attr_name)`. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used /// to intern `attr_name`. /// /// # Example: `intern!`ing the attribute name /// /// ``` /// # use pyo3::{intern, pyfunction, types::PyModule, Python, PyResult}; /// # /// #[pyfunction] /// fn has_version(sys: &PyModule) -> PyResult { /// sys.hasattr(intern!(sys.py(), "version")) /// } /// # /// # Python::with_gil(|py| { /// # let sys = py.import("sys").unwrap(); /// # has_version(sys).unwrap(); /// # }); /// ``` pub fn hasattr(&self, attr_name: N) -> PyResult where N: IntoPy>, { fn inner(any: &PyAny, attr_name: Py) -> PyResult { // PyObject_HasAttr suppresses all exceptions, which was the behaviour of `hasattr` in Python 2. // Use an implementation which suppresses only AttributeError, which is consistent with `hasattr` in Python 3. match any._getattr(attr_name) { Ok(_) => Ok(true), Err(err) if err.is_instance_of::(any.py()) => Ok(false), Err(e) => Err(e), } } inner(self, attr_name.into_py(self.py())) } /// Retrieves an attribute value. /// /// This is equivalent to the Python expression `self.attr_name`. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used /// to intern `attr_name`. /// /// # Example: `intern!`ing the attribute name /// /// ``` /// # use pyo3::{intern, pyfunction, types::PyModule, PyAny, Python, PyResult}; /// # /// #[pyfunction] /// fn version(sys: &PyModule) -> PyResult<&PyAny> { /// sys.getattr(intern!(sys.py(), "version")) /// } /// # /// # Python::with_gil(|py| { /// # let sys = py.import("sys").unwrap(); /// # version(sys).unwrap(); /// # }); /// ``` pub fn getattr(&self, attr_name: N) -> PyResult<&PyAny> where N: IntoPy>, { fn inner(any: &PyAny, attr_name: Py) -> PyResult<&PyAny> { any._getattr(attr_name) .map(|object| object.into_ref(any.py())) } inner(self, attr_name.into_py(self.py())) } fn _getattr(&self, attr_name: Py) -> PyResult { unsafe { Py::from_owned_ptr_or_err( self.py(), ffi::PyObject_GetAttr(self.as_ptr(), attr_name.as_ptr()), ) } } /// Retrieve an attribute value, skipping the instance dictionary during the lookup but still /// binding the object to the instance. /// /// This is useful when trying to resolve Python's "magic" methods like `__getitem__`, which /// are looked up starting from the type object. This returns an `Option` as it is not /// typically a direct error for the special lookup to fail, as magic methods are optional in /// many situations in which they might be called. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used /// to intern `attr_name`. #[allow(dead_code)] // Currently only used with num-complex+abi3, so dead without that. pub(crate) fn lookup_special(&self, attr_name: N) -> PyResult> where N: IntoPy>, { let py = self.py(); let self_type = self.get_type(); let attr = if let Ok(attr) = self_type.getattr(attr_name) { attr } else { return Ok(None); }; // Manually resolve descriptor protocol. if cfg!(Py_3_10) || unsafe { ffi::PyType_HasFeature(attr.get_type_ptr(), ffi::Py_TPFLAGS_HEAPTYPE) } != 0 { // This is the preferred faster path, but does not work on static types (generally, // types defined in extension modules) before Python 3.10. unsafe { let descr_get_ptr = ffi::PyType_GetSlot(attr.get_type_ptr(), ffi::Py_tp_descr_get); if descr_get_ptr.is_null() { return Ok(Some(attr)); } let descr_get: ffi::descrgetfunc = std::mem::transmute(descr_get_ptr); let ret = descr_get(attr.as_ptr(), self.as_ptr(), self_type.as_ptr()); py.from_owned_ptr_or_err(ret).map(Some) } } else if let Ok(descr_get) = attr.get_type().getattr(crate::intern!(py, "__get__")) { descr_get.call1((attr, self, self_type)).map(Some) } else { Ok(Some(attr)) } } /// Sets an attribute value. /// /// This is equivalent to the Python expression `self.attr_name = value`. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used /// to intern `name`. /// /// # Example: `intern!`ing the attribute name /// /// ``` /// # use pyo3::{intern, pyfunction, types::PyModule, PyAny, Python, PyResult}; /// # /// #[pyfunction] /// fn set_answer(ob: &PyAny) -> PyResult<()> { /// ob.setattr(intern!(ob.py(), "answer"), 42) /// } /// # /// # Python::with_gil(|py| { /// # let ob = PyModule::new(py, "empty").unwrap(); /// # set_answer(ob).unwrap(); /// # }); /// ``` pub fn setattr(&self, attr_name: N, value: V) -> PyResult<()> where N: IntoPy>, V: ToPyObject, { fn inner(any: &PyAny, attr_name: Py, value: PyObject) -> PyResult<()> { err::error_on_minusone(any.py(), unsafe { ffi::PyObject_SetAttr(any.as_ptr(), attr_name.as_ptr(), value.as_ptr()) }) } let py = self.py(); inner(self, attr_name.into_py(py), value.to_object(py)) } /// Deletes an attribute. /// /// This is equivalent to the Python statement `del self.attr_name`. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used /// to intern `attr_name`. pub fn delattr(&self, attr_name: N) -> PyResult<()> where N: IntoPy>, { fn inner(any: &PyAny, attr_name: Py) -> PyResult<()> { err::error_on_minusone(any.py(), unsafe { ffi::PyObject_DelAttr(any.as_ptr(), attr_name.as_ptr()) }) } inner(self, attr_name.into_py(self.py())) } /// Returns an [`Ordering`] between `self` and `other`. /// /// This is equivalent to the following Python code: /// ```python /// if self == other: /// return Equal /// elif a < b: /// return Less /// elif a > b: /// return Greater /// else: /// raise TypeError("PyAny::compare(): All comparisons returned false") /// ``` /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyFloat; /// use std::cmp::Ordering; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let a = PyFloat::new(py, 0_f64); /// let b = PyFloat::new(py, 42_f64); /// assert_eq!(a.compare(b)?, Ordering::Less); /// Ok(()) /// })?; /// # Ok(())} /// ``` /// /// It will return `PyErr` for values that cannot be compared: /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::{PyFloat, PyString}; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let a = PyFloat::new(py, 0_f64); /// let b = PyString::new(py, "zero"); /// assert!(a.compare(b).is_err()); /// Ok(()) /// })?; /// # Ok(())} /// ``` pub fn compare(&self, other: O) -> PyResult where O: ToPyObject, { self._compare(other.to_object(self.py())) } fn _compare(&self, other: PyObject) -> PyResult { let py = self.py(); let other = other.as_ptr(); // Almost the same as ffi::PyObject_RichCompareBool, but this one doesn't try self == other. // See https://github.com/PyO3/pyo3/issues/985 for more. let do_compare = |other, op| unsafe { PyObject::from_owned_ptr_or_err(py, ffi::PyObject_RichCompare(self.as_ptr(), other, op)) .and_then(|obj| obj.is_true(py)) }; if do_compare(other, ffi::Py_EQ)? { Ok(Ordering::Equal) } else if do_compare(other, ffi::Py_LT)? { Ok(Ordering::Less) } else if do_compare(other, ffi::Py_GT)? { Ok(Ordering::Greater) } else { Err(PyTypeError::new_err( "PyAny::compare(): All comparisons returned false", )) } } /// Tests whether two Python objects obey a given [`CompareOp`]. /// /// [`lt`](Self::lt), [`le`](Self::le), [`eq`](Self::eq), [`ne`](Self::ne), /// [`gt`](Self::gt) and [`ge`](Self::ge) are the specialized versions /// of this function. /// /// Depending on the value of `compare_op`, this is equivalent to one of the /// following Python expressions: /// /// | `compare_op` | Python expression | /// | :---: | :----: | /// | [`CompareOp::Eq`] | `self == other` | /// | [`CompareOp::Ne`] | `self != other` | /// | [`CompareOp::Lt`] | `self < other` | /// | [`CompareOp::Le`] | `self <= other` | /// | [`CompareOp::Gt`] | `self > other` | /// | [`CompareOp::Ge`] | `self >= other` | /// /// # Examples /// /// ```rust /// use pyo3::class::basic::CompareOp; /// use pyo3::prelude::*; /// use pyo3::types::PyInt; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let a: &PyInt = 0_u8.into_py(py).into_ref(py).downcast()?; /// let b: &PyInt = 42_u8.into_py(py).into_ref(py).downcast()?; /// assert!(a.rich_compare(b, CompareOp::Le)?.is_true()?); /// Ok(()) /// })?; /// # Ok(())} /// ``` pub fn rich_compare(&self, other: O, compare_op: CompareOp) -> PyResult<&PyAny> where O: ToPyObject, { fn inner(slf: &PyAny, other: PyObject, compare_op: CompareOp) -> PyResult<&PyAny> { unsafe { slf.py().from_owned_ptr_or_err(ffi::PyObject_RichCompare( slf.as_ptr(), other.as_ptr(), compare_op as c_int, )) } } inner(self, other.to_object(self.py()), compare_op) } /// Tests whether this object is less than another. /// /// This is equivalent to the Python expression `self < other`. pub fn lt(&self, other: O) -> PyResult where O: ToPyObject, { self.rich_compare(other, CompareOp::Lt)?.is_true() } /// Tests whether this object is less than or equal to another. /// /// This is equivalent to the Python expression `self <= other`. pub fn le(&self, other: O) -> PyResult where O: ToPyObject, { self.rich_compare(other, CompareOp::Le)?.is_true() } /// Tests whether this object is equal to another. /// /// This is equivalent to the Python expression `self == other`. pub fn eq(&self, other: O) -> PyResult where O: ToPyObject, { self.rich_compare(other, CompareOp::Eq)?.is_true() } /// Tests whether this object is not equal to another. /// /// This is equivalent to the Python expression `self != other`. pub fn ne(&self, other: O) -> PyResult where O: ToPyObject, { self.rich_compare(other, CompareOp::Ne)?.is_true() } /// Tests whether this object is greater than another. /// /// This is equivalent to the Python expression `self > other`. pub fn gt(&self, other: O) -> PyResult where O: ToPyObject, { self.rich_compare(other, CompareOp::Gt)?.is_true() } /// Tests whether this object is greater than or equal to another. /// /// This is equivalent to the Python expression `self >= other`. pub fn ge(&self, other: O) -> PyResult where O: ToPyObject, { self.rich_compare(other, CompareOp::Ge)?.is_true() } /// Determines whether this object appears callable. /// /// This is equivalent to Python's [`callable()`][1] function. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let builtins = PyModule::import(py, "builtins")?; /// let print = builtins.getattr("print")?; /// assert!(print.is_callable()); /// Ok(()) /// })?; /// # Ok(())} /// ``` /// /// This is equivalent to the Python statement `assert callable(print)`. /// /// Note that unless an API needs to distinguish between callable and /// non-callable objects, there is no point in checking for callability. /// Instead, it is better to just do the call and handle potential /// exceptions. /// /// [1]: https://docs.python.org/3/library/functions.html#callable pub fn is_callable(&self) -> bool { unsafe { ffi::PyCallable_Check(self.as_ptr()) != 0 } } /// Calls the object. /// /// This is equivalent to the Python expression `self(*args, **kwargs)`. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyDict; /// /// const CODE: &str = r#" /// def function(*args, **kwargs): /// assert args == ("hello",) /// assert kwargs == {"cruel": "world"} /// return "called with args and kwargs" /// "#; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// let module = PyModule::from_code(py, CODE, "", "")?; /// let fun = module.getattr("function")?; /// let args = ("hello",); /// let kwargs = PyDict::new(py); /// kwargs.set_item("cruel", "world")?; /// let result = fun.call(args, Some(kwargs))?; /// assert_eq!(result.extract::<&str>()?, "called with args and kwargs"); /// Ok(()) /// }) /// # } /// ``` pub fn call( &self, args: impl IntoPy>, kwargs: Option<&PyDict>, ) -> PyResult<&PyAny> { let py = self.py(); let args = args.into_py(py); let kwargs = kwargs.map_or(std::ptr::null_mut(), |kwargs| kwargs.as_ptr()); unsafe { let return_value = ffi::PyObject_Call(self.as_ptr(), args.as_ptr(), kwargs); let ret = py.from_owned_ptr_or_err(return_value); ret } } /// Calls the object without arguments. /// /// This is equivalent to the Python expression `self()`. /// /// # Examples /// /// ```no_run /// use pyo3::prelude::*; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let module = PyModule::import(py, "builtins")?; /// let help = module.getattr("help")?; /// help.call0()?; /// Ok(()) /// })?; /// # Ok(())} /// ``` /// /// This is equivalent to the Python expression `help()`. pub fn call0(&self) -> PyResult<&PyAny> { cfg_if::cfg_if! { if #[cfg(all( not(PyPy), any(Py_3_10, all(not(Py_LIMITED_API), Py_3_9)) // PyObject_CallNoArgs was added to python in 3.9 but to limited API in 3.10 ))] { // Optimized path on python 3.9+ unsafe { self.py().from_owned_ptr_or_err(ffi::PyObject_CallNoArgs(self.as_ptr())) } } else { self.call((), None) } } } /// Calls the object with only positional arguments. /// /// This is equivalent to the Python expression `self(*args)`. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// /// const CODE: &str = r#" /// def function(*args, **kwargs): /// assert args == ("hello",) /// assert kwargs == {} /// return "called with args" /// "#; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// let module = PyModule::from_code(py, CODE, "", "")?; /// let fun = module.getattr("function")?; /// let args = ("hello",); /// let result = fun.call1(args)?; /// assert_eq!(result.extract::<&str>()?, "called with args"); /// Ok(()) /// }) /// # } /// ``` pub fn call1(&self, args: impl IntoPy>) -> PyResult<&PyAny> { self.call(args, None) } /// Calls a method on the object. /// /// This is equivalent to the Python expression `self.name(*args, **kwargs)`. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used /// to intern `name`. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyDict; /// /// const CODE: &str = r#" /// class A: /// def method(self, *args, **kwargs): /// assert args == ("hello",) /// assert kwargs == {"cruel": "world"} /// return "called with args and kwargs" /// a = A() /// "#; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// let module = PyModule::from_code(py, CODE, "", "")?; /// let instance = module.getattr("a")?; /// let args = ("hello",); /// let kwargs = PyDict::new(py); /// kwargs.set_item("cruel", "world")?; /// let result = instance.call_method("method", args, Some(kwargs))?; /// assert_eq!(result.extract::<&str>()?, "called with args and kwargs"); /// Ok(()) /// }) /// # } /// ``` pub fn call_method(&self, name: N, args: A, kwargs: Option<&PyDict>) -> PyResult<&PyAny> where N: IntoPy>, A: IntoPy>, { let py = self.py(); let callee = self.getattr(name)?; let args: Py = args.into_py(py); let kwargs = kwargs.map_or(std::ptr::null_mut(), |kwargs| kwargs.as_ptr()); unsafe { let result_ptr = ffi::PyObject_Call(callee.as_ptr(), args.as_ptr(), kwargs); let result = py.from_owned_ptr_or_err(result_ptr); result } } /// Calls a method on the object without arguments. /// /// This is equivalent to the Python expression `self.name()`. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used /// to intern `name`. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// /// const CODE: &str = r#" /// class A: /// def method(self, *args, **kwargs): /// assert args == () /// assert kwargs == {} /// return "called with no arguments" /// a = A() /// "#; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// let module = PyModule::from_code(py, CODE, "", "")?; /// let instance = module.getattr("a")?; /// let result = instance.call_method0("method")?; /// assert_eq!(result.extract::<&str>()?, "called with no arguments"); /// Ok(()) /// }) /// # } /// ``` pub fn call_method0(&self, name: N) -> PyResult<&PyAny> where N: IntoPy>, { cfg_if::cfg_if! { if #[cfg(all(Py_3_9, not(any(Py_LIMITED_API, PyPy))))] { let py = self.py(); // Optimized path on python 3.9+ unsafe { let name: Py = name.into_py(py); let ptr = ffi::PyObject_CallMethodNoArgs(self.as_ptr(), name.as_ptr()); py.from_owned_ptr_or_err(ptr) } } else { self.call_method(name, (), None) } } } /// Calls a method on the object with only positional arguments. /// /// This is equivalent to the Python expression `self.name(*args)`. /// /// To avoid repeated temporary allocations of Python strings, the [`intern!`] macro can be used /// to intern `name`. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// /// const CODE: &str = r#" /// class A: /// def method(self, *args, **kwargs): /// assert args == ("hello",) /// assert kwargs == {} /// return "called with args" /// a = A() /// "#; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// let module = PyModule::from_code(py, CODE, "", "")?; /// let instance = module.getattr("a")?; /// let args = ("hello",); /// let result = instance.call_method1("method", args)?; /// assert_eq!(result.extract::<&str>()?, "called with args"); /// Ok(()) /// }) /// # } /// ``` pub fn call_method1(&self, name: N, args: A) -> PyResult<&PyAny> where N: IntoPy>, A: IntoPy>, { self.call_method(name, args, None) } /// Returns whether the object is considered to be true. /// /// This is equivalent to the Python expression `bool(self)`. pub fn is_true(&self) -> PyResult { let v = unsafe { ffi::PyObject_IsTrue(self.as_ptr()) }; err::error_on_minusone(self.py(), v)?; Ok(v != 0) } /// Returns whether the object is considered to be None. /// /// This is equivalent to the Python expression `self is None`. #[inline] pub fn is_none(&self) -> bool { unsafe { ffi::Py_None() == self.as_ptr() } } /// Returns whether the object is Ellipsis, e.g. `...`. /// /// This is equivalent to the Python expression `self is ...`. pub fn is_ellipsis(&self) -> bool { unsafe { ffi::Py_Ellipsis() == self.as_ptr() } } /// Returns true if the sequence or mapping has a length of 0. /// /// This is equivalent to the Python expression `len(self) == 0`. pub fn is_empty(&self) -> PyResult { self.len().map(|l| l == 0) } /// Gets an item from the collection. /// /// This is equivalent to the Python expression `self[key]`. pub fn get_item(&self, key: K) -> PyResult<&PyAny> where K: ToPyObject, { fn inner(slf: &PyAny, key: PyObject) -> PyResult<&PyAny> { unsafe { slf.py() .from_owned_ptr_or_err(ffi::PyObject_GetItem(slf.as_ptr(), key.as_ptr())) } } inner(self, key.to_object(self.py())) } /// Sets a collection item value. /// /// This is equivalent to the Python expression `self[key] = value`. pub fn set_item(&self, key: K, value: V) -> PyResult<()> where K: ToPyObject, V: ToPyObject, { fn inner(slf: &PyAny, key: PyObject, value: PyObject) -> PyResult<()> { err::error_on_minusone(slf.py(), unsafe { ffi::PyObject_SetItem(slf.as_ptr(), key.as_ptr(), value.as_ptr()) }) } let py = self.py(); inner(self, key.to_object(py), value.to_object(py)) } /// Deletes an item from the collection. /// /// This is equivalent to the Python expression `del self[key]`. pub fn del_item(&self, key: K) -> PyResult<()> where K: ToPyObject, { fn inner(slf: &PyAny, key: PyObject) -> PyResult<()> { err::error_on_minusone(slf.py(), unsafe { ffi::PyObject_DelItem(slf.as_ptr(), key.as_ptr()) }) } inner(self, key.to_object(self.py())) } /// Takes an object and returns an iterator for it. /// /// This is typically a new iterator but if the argument is an iterator, /// this returns itself. pub fn iter(&self) -> PyResult<&PyIterator> { PyIterator::from_object(self) } /// Returns the Python type object for this object's type. pub fn get_type(&self) -> &PyType { unsafe { PyType::from_type_ptr(self.py(), ffi::Py_TYPE(self.as_ptr())) } } /// Returns the Python type pointer for this object. #[inline] pub fn get_type_ptr(&self) -> *mut ffi::PyTypeObject { unsafe { ffi::Py_TYPE(self.as_ptr()) } } /// Downcast this `PyAny` to a concrete Python type or pyclass. /// /// Note that you can often avoid downcasting yourself by just specifying /// the desired type in function or method signatures. /// However, manual downcasting is sometimes necessary. /// /// For extracting a Rust-only type, see [`PyAny::extract`](struct.PyAny.html#method.extract). /// /// # Example: Downcasting to a specific Python object /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::{PyDict, PyList}; /// /// Python::with_gil(|py| { /// let dict = PyDict::new(py); /// assert!(dict.is_instance_of::()); /// let any: &PyAny = dict.as_ref(); /// /// assert!(any.downcast::().is_ok()); /// assert!(any.downcast::().is_err()); /// }); /// ``` /// /// # Example: Getting a reference to a pyclass /// /// This is useful if you want to mutate a `PyObject` that /// might actually be a pyclass. /// /// ```rust /// # fn main() -> Result<(), pyo3::PyErr> { /// use pyo3::prelude::*; /// /// #[pyclass] /// struct Class { /// i: i32, /// } /// /// Python::with_gil(|py| { /// let class: &PyAny = Py::new(py, Class { i: 0 }).unwrap().into_ref(py); /// /// let class_cell: &PyCell = class.downcast()?; /// /// class_cell.borrow_mut().i += 1; /// /// // Alternatively you can get a `PyRefMut` directly /// let class_ref: PyRefMut<'_, Class> = class.extract()?; /// assert_eq!(class_ref.i, 1); /// Ok(()) /// }) /// # } /// ``` #[inline] pub fn downcast<'p, T>(&'p self) -> Result<&'p T, PyDowncastError<'_>> where T: PyTryFrom<'p>, { ::try_from(self) } /// Downcast this `PyAny` to a concrete Python type or pyclass (but not a subclass of it). /// /// It is almost always better to use [`PyAny::downcast`] because it accounts for Python /// subtyping. Use this method only when you do not want to allow subtypes. /// /// The advantage of this method over [`PyAny::downcast`] is that it is faster. The implementation /// of `downcast_exact` uses the equivalent of the Python expression `type(self) is T`, whereas /// `downcast` uses `isinstance(self, T)`. /// /// For extracting a Rust-only type, see [`PyAny::extract`](struct.PyAny.html#method.extract). /// /// # Example: Downcasting to a specific Python object but not a subtype /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::{PyBool, PyLong}; /// /// Python::with_gil(|py| { /// let b = PyBool::new(py, true); /// assert!(b.is_instance_of::()); /// let any: &PyAny = b.as_ref(); /// /// // `bool` is a subtype of `int`, so `downcast` will accept a `bool` as an `int` /// // but `downcast_exact` will not. /// assert!(any.downcast::().is_ok()); /// assert!(any.downcast_exact::().is_err()); /// /// assert!(any.downcast_exact::().is_ok()); /// }); /// ``` #[inline] pub fn downcast_exact<'p, T>(&'p self) -> Result<&'p T, PyDowncastError<'_>> where T: PyTryFrom<'p>, { ::try_from_exact(self) } /// Converts this `PyAny` to a concrete Python type without checking validity. /// /// # Safety /// /// Callers must ensure that the type is valid or risk type confusion. #[inline] pub unsafe fn downcast_unchecked<'p, T>(&'p self) -> &'p T where T: PyTryFrom<'p>, { ::try_from_unchecked(self) } /// Extracts some type from the Python object. /// /// This is a wrapper function around [`FromPyObject::extract()`]. pub fn extract<'a, D>(&'a self) -> PyResult where D: FromPyObject<'a>, { FromPyObject::extract(self) } /// Returns the reference count for the Python object. pub fn get_refcnt(&self) -> isize { unsafe { ffi::Py_REFCNT(self.as_ptr()) } } /// Computes the "repr" representation of self. /// /// This is equivalent to the Python expression `repr(self)`. pub fn repr(&self) -> PyResult<&PyString> { unsafe { self.py() .from_owned_ptr_or_err(ffi::PyObject_Repr(self.as_ptr())) } } /// Computes the "str" representation of self. /// /// This is equivalent to the Python expression `str(self)`. pub fn str(&self) -> PyResult<&PyString> { unsafe { self.py() .from_owned_ptr_or_err(ffi::PyObject_Str(self.as_ptr())) } } /// Retrieves the hash code of self. /// /// This is equivalent to the Python expression `hash(self)`. pub fn hash(&self) -> PyResult { let v = unsafe { ffi::PyObject_Hash(self.as_ptr()) }; crate::err::error_on_minusone(self.py(), v)?; Ok(v) } /// Returns the length of the sequence or mapping. /// /// This is equivalent to the Python expression `len(self)`. pub fn len(&self) -> PyResult { let v = unsafe { ffi::PyObject_Size(self.as_ptr()) }; crate::err::error_on_minusone(self.py(), v)?; Ok(v as usize) } /// Returns the list of attributes of this object. /// /// This is equivalent to the Python expression `dir(self)`. pub fn dir(&self) -> &PyList { unsafe { self.py().from_owned_ptr(ffi::PyObject_Dir(self.as_ptr())) } } /// Checks whether this object is an instance of type `ty`. /// /// This is equivalent to the Python expression `isinstance(self, ty)`. #[inline] pub fn is_instance(&self, ty: &PyAny) -> PyResult { let result = unsafe { ffi::PyObject_IsInstance(self.as_ptr(), ty.as_ptr()) }; err::error_on_minusone(self.py(), result)?; Ok(result == 1) } /// Checks whether this object is an instance of exactly type `ty` (not a subclass). /// /// This is equivalent to the Python expression `type(self) is ty`. #[inline] pub fn is_exact_instance(&self, ty: &PyAny) -> bool { self.get_type().is(ty) } /// Checks whether this object is an instance of type `T`. /// /// This is equivalent to the Python expression `isinstance(self, T)`, /// if the type `T` is known at compile time. #[inline] pub fn is_instance_of(&self) -> bool { T::is_type_of(self) } /// Checks whether this object is an instance of exactly type `T`. /// /// This is equivalent to the Python expression `type(self) is T`, /// if the type `T` is known at compile time. #[inline] pub fn is_exact_instance_of(&self) -> bool { T::is_exact_type_of(self) } /// Determines if self contains `value`. /// /// This is equivalent to the Python expression `value in self`. pub fn contains(&self, value: V) -> PyResult where V: ToPyObject, { self._contains(value.to_object(self.py())) } fn _contains(&self, value: PyObject) -> PyResult { match unsafe { ffi::PySequence_Contains(self.as_ptr(), value.as_ptr()) } { 0 => Ok(false), 1 => Ok(true), _ => Err(PyErr::fetch(self.py())), } } /// Returns a GIL marker constrained to the lifetime of this type. #[inline] pub fn py(&self) -> Python<'_> { PyNativeType::py(self) } /// Returns the raw FFI pointer represented by self. /// /// # Safety /// /// Callers are responsible for ensuring that the pointer does not outlive self. /// /// The reference is borrowed; callers should not decrease the reference count /// when they are finished with the pointer. #[inline] pub fn as_ptr(&self) -> *mut ffi::PyObject { self as *const PyAny as *mut ffi::PyObject } /// Returns an owned raw FFI pointer represented by self. /// /// # Safety /// /// The reference is owned; when finished the caller should either transfer ownership /// of the pointer or decrease the reference count (e.g. with [`pyo3::ffi::Py_DecRef`](crate::ffi::Py_DecRef)). #[inline] pub fn into_ptr(&self) -> *mut ffi::PyObject { // Safety: self.as_ptr() returns a valid non-null pointer unsafe { ffi::_Py_NewRef(self.as_ptr()) } } /// Return a proxy object that delegates method calls to a parent or sibling class of type. /// /// This is equivalent to the Python expression `super()` #[cfg(not(PyPy))] pub fn py_super(&self) -> PyResult<&PySuper> { PySuper::new(self.get_type(), self) } } #[cfg(test)] mod tests { use crate::{ types::{IntoPyDict, PyAny, PyBool, PyList, PyLong, PyModule}, Python, ToPyObject, }; #[test] fn test_lookup_special() { Python::with_gil(|py| { let module = PyModule::from_code( py, r#" class CustomCallable: def __call__(self): return 1 class SimpleInt: def __int__(self): return 1 class InheritedInt(SimpleInt): pass class NoInt: pass class NoDescriptorInt: __int__ = CustomCallable() class InstanceOverrideInt: def __int__(self): return 1 instance_override = InstanceOverrideInt() instance_override.__int__ = lambda self: 2 class ErrorInDescriptorInt: @property def __int__(self): raise ValueError("uh-oh!") class NonHeapNonDescriptorInt: # A static-typed callable that doesn't implement `__get__`. These are pretty hard to come by. __int__ = int "#, "test.py", "test", ) .unwrap(); let int = crate::intern!(py, "__int__"); let eval_int = |obj: &PyAny| obj.lookup_special(int)?.unwrap().call0()?.extract::(); let simple = module.getattr("SimpleInt").unwrap().call0().unwrap(); assert_eq!(eval_int(simple).unwrap(), 1); let inherited = module.getattr("InheritedInt").unwrap().call0().unwrap(); assert_eq!(eval_int(inherited).unwrap(), 1); let no_descriptor = module.getattr("NoDescriptorInt").unwrap().call0().unwrap(); assert_eq!(eval_int(no_descriptor).unwrap(), 1); let missing = module.getattr("NoInt").unwrap().call0().unwrap(); assert!(missing.lookup_special(int).unwrap().is_none()); // Note the instance override should _not_ call the instance method that returns 2, // because that's not how special lookups are meant to work. let instance_override = module.getattr("instance_override").unwrap(); assert_eq!(eval_int(instance_override).unwrap(), 1); let descriptor_error = module .getattr("ErrorInDescriptorInt") .unwrap() .call0() .unwrap(); assert!(descriptor_error.lookup_special(int).is_err()); let nonheap_nondescriptor = module .getattr("NonHeapNonDescriptorInt") .unwrap() .call0() .unwrap(); assert_eq!(eval_int(nonheap_nondescriptor).unwrap(), 0); }) } #[test] fn test_call_for_non_existing_method() { Python::with_gil(|py| { let a = py.eval("42", None, None).unwrap(); a.call_method0("__str__").unwrap(); // ok assert!(a.call_method("nonexistent_method", (1,), None).is_err()); assert!(a.call_method0("nonexistent_method").is_err()); assert!(a.call_method1("nonexistent_method", (1,)).is_err()); }); } #[test] fn test_call_with_kwargs() { Python::with_gil(|py| { let list = vec![3, 6, 5, 4, 7].to_object(py); let dict = vec![("reverse", true)].into_py_dict(py); list.call_method(py, "sort", (), Some(dict)).unwrap(); assert_eq!(list.extract::>(py).unwrap(), vec![7, 6, 5, 4, 3]); }); } #[test] fn test_call_method0() { Python::with_gil(|py| { let module = PyModule::from_code( py, r#" class SimpleClass: def foo(self): return 42 "#, file!(), "test_module", ) .expect("module creation failed"); let simple_class = module.getattr("SimpleClass").unwrap().call0().unwrap(); assert_eq!( simple_class .call_method0("foo") .unwrap() .extract::() .unwrap(), 42 ); }) } #[test] fn test_type() { Python::with_gil(|py| { let obj = py.eval("42", None, None).unwrap(); assert_eq!(obj.get_type().as_type_ptr(), obj.get_type_ptr()); }); } #[test] fn test_dir() { Python::with_gil(|py| { let obj = py.eval("42", None, None).unwrap(); let dir = py .eval("dir(42)", None, None) .unwrap() .downcast::() .unwrap(); let a = obj .dir() .into_iter() .map(|x| x.extract::().unwrap()); let b = dir.into_iter().map(|x| x.extract::().unwrap()); assert!(a.eq(b)); }); } #[test] fn test_hasattr() { Python::with_gil(|py| { let x = 5.to_object(py).into_ref(py); assert!(x.is_instance_of::()); assert!(x.hasattr("to_bytes").unwrap()); assert!(!x.hasattr("bbbbbbytes").unwrap()); }) } #[cfg(feature = "macros")] #[test] fn test_hasattr_error() { use crate::exceptions::PyValueError; use crate::prelude::*; #[pyclass(crate = "crate")] struct GetattrFail; #[pymethods(crate = "crate")] impl GetattrFail { fn __getattr__(&self, attr: PyObject) -> PyResult { Err(PyValueError::new_err(attr)) } } Python::with_gil(|py| { let obj = Py::new(py, GetattrFail).unwrap(); let obj = obj.as_ref(py).as_ref(); assert!(obj .hasattr("foo") .unwrap_err() .is_instance_of::(py)); }) } #[test] fn test_nan_eq() { Python::with_gil(|py| { let nan = py.eval("float('nan')", None, None).unwrap(); assert!(nan.compare(nan).is_err()); }); } #[test] fn test_any_is_instance_of() { Python::with_gil(|py| { let x = 5.to_object(py).into_ref(py); assert!(x.is_instance_of::()); let l = vec![x, x].to_object(py).into_ref(py); assert!(l.is_instance_of::()); }); } #[test] fn test_any_is_instance() { Python::with_gil(|py| { let l = vec![1u8, 2].to_object(py).into_ref(py); assert!(l.is_instance(py.get_type::()).unwrap()); }); } #[test] fn test_any_is_exact_instance_of() { Python::with_gil(|py| { let x = 5.to_object(py).into_ref(py); assert!(x.is_exact_instance_of::()); let t = PyBool::new(py, true); assert!(t.is_instance_of::()); assert!(!t.is_exact_instance_of::()); assert!(t.is_exact_instance_of::()); let l = vec![x, x].to_object(py).into_ref(py); assert!(l.is_exact_instance_of::()); }); } #[test] fn test_any_is_exact_instance() { Python::with_gil(|py| { let t = PyBool::new(py, true); assert!(t.is_instance(py.get_type::()).unwrap()); assert!(!t.is_exact_instance(py.get_type::())); assert!(t.is_exact_instance(py.get_type::())); }); } #[test] fn test_any_contains() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2, 3, 5, 8]; let ob = v.to_object(py).into_ref(py); let bad_needle = 7i32.to_object(py); assert!(!ob.contains(&bad_needle).unwrap()); let good_needle = 8i32.to_object(py); assert!(ob.contains(&good_needle).unwrap()); let type_coerced_needle = 8f32.to_object(py); assert!(ob.contains(&type_coerced_needle).unwrap()); let n: u32 = 42; let bad_haystack = n.to_object(py).into_ref(py); let irrelevant_needle = 0i32.to_object(py); assert!(bad_haystack.contains(&irrelevant_needle).is_err()); }); } // This is intentionally not a test, it's a generic function used by the tests below. fn test_eq_methods_generic(list: &[T]) where T: PartialEq + PartialOrd + ToPyObject, { Python::with_gil(|py| { for a in list { for b in list { let a_py = a.to_object(py).into_ref(py); let b_py = b.to_object(py).into_ref(py); assert_eq!( a.lt(b), a_py.lt(b_py).unwrap(), "{} < {} should be {}.", a_py, b_py, a.lt(b) ); assert_eq!( a.le(b), a_py.le(b_py).unwrap(), "{} <= {} should be {}.", a_py, b_py, a.le(b) ); assert_eq!( a.eq(b), a_py.eq(b_py).unwrap(), "{} == {} should be {}.", a_py, b_py, a.eq(b) ); assert_eq!( a.ne(b), a_py.ne(b_py).unwrap(), "{} != {} should be {}.", a_py, b_py, a.ne(b) ); assert_eq!( a.gt(b), a_py.gt(b_py).unwrap(), "{} > {} should be {}.", a_py, b_py, a.gt(b) ); assert_eq!( a.ge(b), a_py.ge(b_py).unwrap(), "{} >= {} should be {}.", a_py, b_py, a.ge(b) ); } } }); } #[test] fn test_eq_methods_integers() { let ints = [-4, -4, 1, 2, 0, -100, 1_000_000]; test_eq_methods_generic(&ints); } #[test] fn test_eq_methods_strings() { let strings = ["Let's", "test", "some", "eq", "methods"]; test_eq_methods_generic(&strings); } #[test] fn test_eq_methods_floats() { let floats = [ -1.0, 2.5, 0.0, 3.0, std::f64::consts::PI, 10.0, 10.0 / 3.0, -1_000_000.0, ]; test_eq_methods_generic(&floats); } #[test] fn test_eq_methods_bools() { let bools = [true, false]; test_eq_methods_generic(&bools); } #[test] fn test_is_ellipsis() { Python::with_gil(|py| { let v = py .eval("...", None, None) .map_err(|e| e.display(py)) .unwrap(); assert!(v.is_ellipsis()); let not_ellipsis = 5.to_object(py).into_ref(py); assert!(!not_ellipsis.is_ellipsis()); }); } } pyo3-0.20.2/src/types/boolobject.rs000064400000000000000000000050251046102023000152240ustar 00000000000000#[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{ffi, FromPyObject, IntoPy, PyAny, PyObject, PyResult, Python, ToPyObject}; /// Represents a Python `bool`. #[repr(transparent)] pub struct PyBool(PyAny); pyobject_native_type!(PyBool, ffi::PyObject, pyobject_native_static_type_object!(ffi::PyBool_Type), #checkfunction=ffi::PyBool_Check); impl PyBool { /// Depending on `val`, returns `true` or `false`. #[inline] pub fn new(py: Python<'_>, val: bool) -> &PyBool { unsafe { py.from_borrowed_ptr(if val { ffi::Py_True() } else { ffi::Py_False() }) } } /// Gets whether this boolean is `true`. #[inline] pub fn is_true(&self) -> bool { self.as_ptr() == unsafe { crate::ffi::Py_True() } } } /// Converts a Rust `bool` to a Python `bool`. impl ToPyObject for bool { #[inline] fn to_object(&self, py: Python<'_>) -> PyObject { unsafe { PyObject::from_borrowed_ptr( py, if *self { ffi::Py_True() } else { ffi::Py_False() }, ) } } } impl IntoPy for bool { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { PyBool::new(py, self).into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::builtin("bool") } } /// Converts a Python `bool` to a Rust `bool`. /// /// Fails with `TypeError` if the input is not a Python `bool`. impl<'source> FromPyObject<'source> for bool { fn extract(obj: &'source PyAny) -> PyResult { Ok(obj.downcast::()?.is_true()) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { Self::type_output() } } #[cfg(test)] mod tests { use crate::types::{PyAny, PyBool}; use crate::Python; use crate::ToPyObject; #[test] fn test_true() { Python::with_gil(|py| { assert!(PyBool::new(py, true).is_true()); let t: &PyAny = PyBool::new(py, true).into(); assert!(t.extract::().unwrap()); assert!(true.to_object(py).is(PyBool::new(py, true))); }); } #[test] fn test_false() { Python::with_gil(|py| { assert!(!PyBool::new(py, false).is_true()); let t: &PyAny = PyBool::new(py, false).into(); assert!(!t.extract::().unwrap()); assert!(false.to_object(py).is(PyBool::new(py, false))); }); } } pyo3-0.20.2/src/types/bytearray.rs000064400000000000000000000312531046102023000151060ustar 00000000000000use crate::err::{PyErr, PyResult}; use crate::{ffi, AsPyPointer, Py, PyAny, Python}; use std::os::raw::c_char; use std::slice; /// Represents a Python `bytearray`. #[repr(transparent)] pub struct PyByteArray(PyAny); pyobject_native_type_core!(PyByteArray, pyobject_native_static_type_object!(ffi::PyByteArray_Type), #checkfunction=ffi::PyByteArray_Check); impl PyByteArray { /// Creates a new Python bytearray object. /// /// The byte string is initialized by copying the data from the `&[u8]`. pub fn new<'p>(py: Python<'p>, src: &[u8]) -> &'p PyByteArray { let ptr = src.as_ptr() as *const c_char; let len = src.len() as ffi::Py_ssize_t; unsafe { py.from_owned_ptr::(ffi::PyByteArray_FromStringAndSize(ptr, len)) } } /// Creates a new Python `bytearray` object with an `init` closure to write its contents. /// Before calling `init` the bytearray is zero-initialised. /// * If Python raises a MemoryError on the allocation, `new_with` will return /// it inside `Err`. /// * If `init` returns `Err(e)`, `new_with` will return `Err(e)`. /// * If `init` returns `Ok(())`, `new_with` will return `Ok(&PyByteArray)`. /// /// # Examples /// /// ``` /// use pyo3::{prelude::*, types::PyByteArray}; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let py_bytearray = PyByteArray::new_with(py, 10, |bytes: &mut [u8]| { /// bytes.copy_from_slice(b"Hello Rust"); /// Ok(()) /// })?; /// let bytearray: &[u8] = unsafe { py_bytearray.as_bytes() }; /// assert_eq!(bytearray, b"Hello Rust"); /// Ok(()) /// }) /// # } /// ``` pub fn new_with(py: Python<'_>, len: usize, init: F) -> PyResult<&PyByteArray> where F: FnOnce(&mut [u8]) -> PyResult<()>, { unsafe { let pyptr = ffi::PyByteArray_FromStringAndSize(std::ptr::null(), len as ffi::Py_ssize_t); // Check for an allocation error and return it let pypybytearray: Py = Py::from_owned_ptr_or_err(py, pyptr)?; let buffer: *mut u8 = ffi::PyByteArray_AsString(pyptr).cast(); debug_assert!(!buffer.is_null()); // Zero-initialise the uninitialised bytearray std::ptr::write_bytes(buffer, 0u8, len); // (Further) Initialise the bytearray in init // If init returns an Err, pypybytearray will automatically deallocate the buffer init(std::slice::from_raw_parts_mut(buffer, len)).map(|_| pypybytearray.into_ref(py)) } } /// Creates a new Python `bytearray` object from another Python object that /// implements the buffer protocol. pub fn from(src: &PyAny) -> PyResult<&PyByteArray> { unsafe { src.py() .from_owned_ptr_or_err(ffi::PyByteArray_FromObject(src.as_ptr())) } } /// Gets the length of the bytearray. #[inline] pub fn len(&self) -> usize { // non-negative Py_ssize_t should always fit into Rust usize unsafe { ffi::PyByteArray_Size(self.as_ptr()) as usize } } /// Checks if the bytearray is empty. pub fn is_empty(&self) -> bool { self.len() == 0 } /// Gets the start of the buffer containing the contents of the bytearray. /// /// # Safety /// /// See the safety requirements of [`PyByteArray::as_bytes`] and [`PyByteArray::as_bytes_mut`]. pub fn data(&self) -> *mut u8 { unsafe { ffi::PyByteArray_AsString(self.as_ptr()).cast() } } /// Extracts a slice of the `ByteArray`'s entire buffer. /// /// # Safety /// /// Mutation of the `bytearray` invalidates the slice. If it is used afterwards, the behavior is /// undefined. /// /// These mutations may occur in Python code as well as from Rust: /// - Calling methods like [`PyByteArray::as_bytes_mut`] and [`PyByteArray::resize`] will /// invalidate the slice. /// - Actions like dropping objects or raising exceptions can invoke `__del__`methods or signal /// handlers, which may execute arbitrary Python code. This means that if Python code has a /// reference to the `bytearray` you cannot safely use the vast majority of PyO3's API whilst /// using the slice. /// /// As a result, this slice should only be used for short-lived operations without executing any /// Python code, such as copying into a Vec. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::exceptions::PyRuntimeError; /// use pyo3::types::PyByteArray; /// /// #[pyfunction] /// fn a_valid_function(bytes: &PyByteArray) -> PyResult<()> { /// let section = { /// // SAFETY: We promise to not let the interpreter regain control /// // or invoke any PyO3 APIs while using the slice. /// let slice = unsafe { bytes.as_bytes() }; /// /// // Copy only a section of `bytes` while avoiding /// // `to_vec` which copies the entire thing. /// let section = slice /// .get(6..11) /// .ok_or_else(|| PyRuntimeError::new_err("input is not long enough"))?; /// Vec::from(section) /// }; /// /// // Now we can do things with `section` and call PyO3 APIs again. /// // ... /// # assert_eq!(§ion, b"world"); /// /// Ok(()) /// } /// # fn main() -> PyResult<()> { /// # Python::with_gil(|py| -> PyResult<()> { /// # let fun = wrap_pyfunction!(a_valid_function, py)?; /// # let locals = pyo3::types::PyDict::new(py); /// # locals.set_item("a_valid_function", fun)?; /// # /// # py.run( /// # r#"b = bytearray(b"hello world") /// # a_valid_function(b) /// # /// # try: /// # a_valid_function(bytearray()) /// # except RuntimeError as e: /// # assert str(e) == 'input is not long enough'"#, /// # None, /// # Some(locals), /// # )?; /// # /// # Ok(()) /// # }) /// # } /// ``` /// /// # Incorrect usage /// /// The following `bug` function is unsound ⚠️ /// /// ```rust,no_run /// # use pyo3::prelude::*; /// # use pyo3::types::PyByteArray; /// /// # #[allow(dead_code)] /// #[pyfunction] /// fn bug(py: Python<'_>, bytes: &PyByteArray) { /// let slice = unsafe { bytes.as_bytes() }; /// /// // This explicitly yields control back to the Python interpreter... /// // ...but it's not always this obvious. Many things do this implicitly. /// py.allow_threads(|| { /// // Python code could be mutating through its handle to `bytes`, /// // which makes reading it a data race, which is undefined behavior. /// println!("{:?}", slice[0]); /// }); /// /// // Python code might have mutated it, so we can not rely on the slice /// // remaining valid. As such this is also undefined behavior. /// println!("{:?}", slice[0]); /// } /// ``` pub unsafe fn as_bytes(&self) -> &[u8] { slice::from_raw_parts(self.data(), self.len()) } /// Extracts a mutable slice of the `ByteArray`'s entire buffer. /// /// # Safety /// /// Any other accesses of the `bytearray`'s buffer invalidate the slice. If it is used /// afterwards, the behavior is undefined. The safety requirements of [`PyByteArray::as_bytes`] /// apply to this function as well. #[allow(clippy::mut_from_ref)] pub unsafe fn as_bytes_mut(&self) -> &mut [u8] { slice::from_raw_parts_mut(self.data(), self.len()) } /// Copies the contents of the bytearray to a Rust vector. /// /// # Examples /// /// ``` /// # use pyo3::prelude::*; /// # use pyo3::types::PyByteArray; /// # Python::with_gil(|py| { /// let bytearray = PyByteArray::new(py, b"Hello World."); /// let mut copied_message = bytearray.to_vec(); /// assert_eq!(b"Hello World.", copied_message.as_slice()); /// /// copied_message[11] = b'!'; /// assert_eq!(b"Hello World!", copied_message.as_slice()); /// /// pyo3::py_run!(py, bytearray, "assert bytearray == b'Hello World.'"); /// # }); /// ``` pub fn to_vec(&self) -> Vec { unsafe { self.as_bytes() }.to_vec() } /// Resizes the bytearray object to the new length `len`. /// /// Note that this will invalidate any pointers obtained by [PyByteArray::data], as well as /// any (unsafe) slices obtained from [PyByteArray::as_bytes] and [PyByteArray::as_bytes_mut]. pub fn resize(&self, len: usize) -> PyResult<()> { unsafe { let result = ffi::PyByteArray_Resize(self.as_ptr(), len as ffi::Py_ssize_t); if result == 0 { Ok(()) } else { Err(PyErr::fetch(self.py())) } } } } #[cfg(test)] mod tests { use crate::exceptions; use crate::types::PyByteArray; use crate::{PyObject, Python}; #[test] fn test_len() { Python::with_gil(|py| { let src = b"Hello Python"; let bytearray = PyByteArray::new(py, src); assert_eq!(src.len(), bytearray.len()); }); } #[test] fn test_as_bytes() { Python::with_gil(|py| { let src = b"Hello Python"; let bytearray = PyByteArray::new(py, src); let slice = unsafe { bytearray.as_bytes() }; assert_eq!(src, slice); assert_eq!(bytearray.data() as *const _, slice.as_ptr()); }); } #[test] fn test_as_bytes_mut() { Python::with_gil(|py| { let src = b"Hello Python"; let bytearray = PyByteArray::new(py, src); let slice = unsafe { bytearray.as_bytes_mut() }; assert_eq!(src, slice); assert_eq!(bytearray.data(), slice.as_mut_ptr()); slice[0..5].copy_from_slice(b"Hi..."); assert_eq!( bytearray.str().unwrap().to_str().unwrap(), "bytearray(b'Hi... Python')" ); }); } #[test] fn test_to_vec() { Python::with_gil(|py| { let src = b"Hello Python"; let bytearray = PyByteArray::new(py, src); let vec = bytearray.to_vec(); assert_eq!(src, vec.as_slice()); }); } #[test] fn test_from() { Python::with_gil(|py| { let src = b"Hello Python"; let bytearray = PyByteArray::new(py, src); let ba: PyObject = bytearray.into(); let bytearray = PyByteArray::from(ba.as_ref(py)).unwrap(); assert_eq!(src, unsafe { bytearray.as_bytes() }); }); } #[test] fn test_from_err() { Python::with_gil(|py| { if let Err(err) = PyByteArray::from(py.None().as_ref(py)) { assert!(err.is_instance_of::(py)); } else { panic!("error"); } }); } #[test] fn test_resize() { Python::with_gil(|py| { let src = b"Hello Python"; let bytearray = PyByteArray::new(py, src); bytearray.resize(20).unwrap(); assert_eq!(20, bytearray.len()); }); } #[test] fn test_byte_array_new_with() -> super::PyResult<()> { Python::with_gil(|py| -> super::PyResult<()> { let py_bytearray = PyByteArray::new_with(py, 10, |b: &mut [u8]| { b.copy_from_slice(b"Hello Rust"); Ok(()) })?; let bytearray: &[u8] = unsafe { py_bytearray.as_bytes() }; assert_eq!(bytearray, b"Hello Rust"); Ok(()) }) } #[test] fn test_byte_array_new_with_zero_initialised() -> super::PyResult<()> { Python::with_gil(|py| -> super::PyResult<()> { let py_bytearray = PyByteArray::new_with(py, 10, |_b: &mut [u8]| Ok(()))?; let bytearray: &[u8] = unsafe { py_bytearray.as_bytes() }; assert_eq!(bytearray, &[0; 10]); Ok(()) }) } #[test] fn test_byte_array_new_with_error() { use crate::exceptions::PyValueError; Python::with_gil(|py| { let py_bytearray_result = PyByteArray::new_with(py, 10, |_b: &mut [u8]| { Err(PyValueError::new_err("Hello Crustaceans!")) }); assert!(py_bytearray_result.is_err()); assert!(py_bytearray_result .err() .unwrap() .is_instance_of::(py)); }) } } pyo3-0.20.2/src/types/bytes.rs000064400000000000000000000201001046102023000142170ustar 00000000000000use crate::{ffi, FromPyObject, IntoPy, Py, PyAny, PyResult, Python, ToPyObject}; use std::borrow::Cow; use std::ops::Index; use std::os::raw::c_char; use std::slice::SliceIndex; use std::str; use super::bytearray::PyByteArray; /// Represents a Python `bytes` object. /// /// This type is immutable. #[repr(transparent)] pub struct PyBytes(PyAny); pyobject_native_type_core!(PyBytes, pyobject_native_static_type_object!(ffi::PyBytes_Type), #checkfunction=ffi::PyBytes_Check); impl PyBytes { /// Creates a new Python bytestring object. /// The bytestring is initialized by copying the data from the `&[u8]`. /// /// Panics if out of memory. pub fn new<'p>(py: Python<'p>, s: &[u8]) -> &'p PyBytes { let ptr = s.as_ptr() as *const c_char; let len = s.len() as ffi::Py_ssize_t; unsafe { py.from_owned_ptr(ffi::PyBytes_FromStringAndSize(ptr, len)) } } /// Creates a new Python `bytes` object with an `init` closure to write its contents. /// Before calling `init` the bytes' contents are zero-initialised. /// * If Python raises a MemoryError on the allocation, `new_with` will return /// it inside `Err`. /// * If `init` returns `Err(e)`, `new_with` will return `Err(e)`. /// * If `init` returns `Ok(())`, `new_with` will return `Ok(&PyBytes)`. /// /// # Examples /// /// ``` /// use pyo3::{prelude::*, types::PyBytes}; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let py_bytes = PyBytes::new_with(py, 10, |bytes: &mut [u8]| { /// bytes.copy_from_slice(b"Hello Rust"); /// Ok(()) /// })?; /// let bytes: &[u8] = FromPyObject::extract(py_bytes)?; /// assert_eq!(bytes, b"Hello Rust"); /// Ok(()) /// }) /// # } /// ``` pub fn new_with(py: Python<'_>, len: usize, init: F) -> PyResult<&PyBytes> where F: FnOnce(&mut [u8]) -> PyResult<()>, { unsafe { let pyptr = ffi::PyBytes_FromStringAndSize(std::ptr::null(), len as ffi::Py_ssize_t); // Check for an allocation error and return it let pypybytes: Py = Py::from_owned_ptr_or_err(py, pyptr)?; let buffer: *mut u8 = ffi::PyBytes_AsString(pyptr).cast(); debug_assert!(!buffer.is_null()); // Zero-initialise the uninitialised bytestring std::ptr::write_bytes(buffer, 0u8, len); // (Further) Initialise the bytestring in init // If init returns an Err, pypybytearray will automatically deallocate the buffer init(std::slice::from_raw_parts_mut(buffer, len)).map(|_| pypybytes.into_ref(py)) } } /// Creates a new Python byte string object from a raw pointer and length. /// /// Panics if out of memory. /// /// # Safety /// /// This function dereferences the raw pointer `ptr` as the /// leading pointer of a slice of length `len`. [As with /// `std::slice::from_raw_parts`, this is /// unsafe](https://doc.rust-lang.org/std/slice/fn.from_raw_parts.html#safety). pub unsafe fn from_ptr(py: Python<'_>, ptr: *const u8, len: usize) -> &PyBytes { py.from_owned_ptr(ffi::PyBytes_FromStringAndSize( ptr as *const _, len as isize, )) } /// Gets the Python string as a byte slice. #[inline] pub fn as_bytes(&self) -> &[u8] { unsafe { let buffer = ffi::PyBytes_AsString(self.as_ptr()) as *const u8; let length = ffi::PyBytes_Size(self.as_ptr()) as usize; debug_assert!(!buffer.is_null()); std::slice::from_raw_parts(buffer, length) } } } impl Py { /// Gets the Python bytes as a byte slice. Because Python bytes are /// immutable, the result may be used for as long as the reference to /// `self` is held, including when the GIL is released. pub fn as_bytes<'a>(&'a self, _py: Python<'_>) -> &'a [u8] { // py is required here because `PyBytes_AsString` and `PyBytes_Size` // can both technically raise exceptions which require the GIL to be // held. The only circumstance in which they raise is if the value // isn't really a `PyBytes`, but better safe than sorry. unsafe { let buffer = ffi::PyBytes_AsString(self.as_ptr()) as *const u8; let length = ffi::PyBytes_Size(self.as_ptr()) as usize; debug_assert!(!buffer.is_null()); std::slice::from_raw_parts(buffer, length) } } } /// This is the same way [Vec] is indexed. impl> Index for PyBytes { type Output = I::Output; fn index(&self, index: I) -> &Self::Output { &self.as_bytes()[index] } } /// Special-purpose trait impl to efficiently handle both `bytes` and `bytearray` /// /// If the source object is a `bytes` object, the `Cow` will be borrowed and /// pointing into the source object, and no copying or heap allocations will happen. /// If it is a `bytearray`, its contents will be copied to an owned `Cow`. impl<'source> FromPyObject<'source> for Cow<'source, [u8]> { fn extract(ob: &'source PyAny) -> PyResult { if let Ok(bytes) = ob.downcast::() { return Ok(Cow::Borrowed(bytes.as_bytes())); } let byte_array = ob.downcast::()?; Ok(Cow::Owned(byte_array.to_vec())) } } impl ToPyObject for Cow<'_, [u8]> { fn to_object(&self, py: Python<'_>) -> Py { PyBytes::new(py, self.as_ref()).into() } } impl IntoPy> for Cow<'_, [u8]> { fn into_py(self, py: Python<'_>) -> Py { self.to_object(py) } } #[cfg(test)] mod tests { use super::*; #[test] fn test_bytes_index() { Python::with_gil(|py| { let bytes = PyBytes::new(py, b"Hello World"); assert_eq!(bytes[1], b'e'); }); } #[test] fn test_bytes_new_with() -> super::PyResult<()> { Python::with_gil(|py| -> super::PyResult<()> { let py_bytes = PyBytes::new_with(py, 10, |b: &mut [u8]| { b.copy_from_slice(b"Hello Rust"); Ok(()) })?; let bytes: &[u8] = FromPyObject::extract(py_bytes)?; assert_eq!(bytes, b"Hello Rust"); Ok(()) }) } #[test] fn test_bytes_new_with_zero_initialised() -> super::PyResult<()> { Python::with_gil(|py| -> super::PyResult<()> { let py_bytes = PyBytes::new_with(py, 10, |_b: &mut [u8]| Ok(()))?; let bytes: &[u8] = FromPyObject::extract(py_bytes)?; assert_eq!(bytes, &[0; 10]); Ok(()) }) } #[test] fn test_bytes_new_with_error() { use crate::exceptions::PyValueError; Python::with_gil(|py| { let py_bytes_result = PyBytes::new_with(py, 10, |_b: &mut [u8]| { Err(PyValueError::new_err("Hello Crustaceans!")) }); assert!(py_bytes_result.is_err()); assert!(py_bytes_result .err() .unwrap() .is_instance_of::(py)); }); } #[test] fn test_cow_impl() { Python::with_gil(|py| { let bytes = py.eval(r#"b"foobar""#, None, None).unwrap(); let cow = bytes.extract::>().unwrap(); assert_eq!(cow, Cow::<[u8]>::Borrowed(b"foobar")); let byte_array = py.eval(r#"bytearray(b"foobar")"#, None, None).unwrap(); let cow = byte_array.extract::>().unwrap(); assert_eq!(cow, Cow::<[u8]>::Owned(b"foobar".to_vec())); let something_else_entirely = py.eval("42", None, None).unwrap(); something_else_entirely .extract::>() .unwrap_err(); let cow = Cow::<[u8]>::Borrowed(b"foobar").to_object(py); assert!(cow.as_ref(py).is_instance_of::()); let cow = Cow::<[u8]>::Owned(b"foobar".to_vec()).to_object(py); assert!(cow.as_ref(py).is_instance_of::()); }); } } pyo3-0.20.2/src/types/capsule.rs000064400000000000000000000371541046102023000145460ustar 00000000000000use crate::Python; use crate::{ffi, PyAny}; use crate::{pyobject_native_type_core, PyErr, PyResult}; use std::ffi::{CStr, CString}; use std::os::raw::{c_char, c_int, c_void}; /// Represents a Python Capsule /// as described in [Capsules](https://docs.python.org/3/c-api/capsule.html#capsules): /// > This subtype of PyObject represents an opaque value, useful for C extension /// > modules who need to pass an opaque value (as a void* pointer) through Python /// > code to other C code. It is often used to make a C function pointer defined /// > in one module available to other modules, so the regular import mechanism can /// > be used to access C APIs defined in dynamically loaded modules. /// /// /// # Example /// ``` /// use pyo3::{prelude::*, types::PyCapsule}; /// use std::ffi::CString; /// /// #[repr(C)] /// struct Foo { /// pub val: u32, /// } /// /// let r = Python::with_gil(|py| -> PyResult<()> { /// let foo = Foo { val: 123 }; /// let name = CString::new("builtins.capsule").unwrap(); /// /// let capsule = PyCapsule::new(py, foo, Some(name.clone()))?; /// /// let module = PyModule::import(py, "builtins")?; /// module.add("capsule", capsule)?; /// /// let cap: &Foo = unsafe { PyCapsule::import(py, name.as_ref())? }; /// assert_eq!(cap.val, 123); /// Ok(()) /// }); /// assert!(r.is_ok()); /// ``` #[repr(transparent)] pub struct PyCapsule(PyAny); pyobject_native_type_core!(PyCapsule, pyobject_native_static_type_object!(ffi::PyCapsule_Type), #checkfunction=ffi::PyCapsule_CheckExact); impl PyCapsule { /// Constructs a new capsule whose contents are `value`, associated with `name`. /// `name` is the identifier for the capsule; if it is stored as an attribute of a module, /// the name should be in the format `"modulename.attribute"`. /// /// It is checked at compile time that the type T is not zero-sized. Rust function items /// need to be cast to a function pointer (`fn(args) -> result`) to be put into a capsule. /// /// # Example /// /// ``` /// use pyo3::{prelude::*, types::PyCapsule}; /// use std::ffi::CString; /// /// Python::with_gil(|py| { /// let name = CString::new("foo").unwrap(); /// let capsule = PyCapsule::new(py, 123_u32, Some(name)).unwrap(); /// let val = unsafe { capsule.reference::() }; /// assert_eq!(*val, 123); /// }); /// ``` /// /// However, attempting to construct a `PyCapsule` with a zero-sized type will not compile: /// /// ```compile_fail /// use pyo3::{prelude::*, types::PyCapsule}; /// use std::ffi::CString; /// /// Python::with_gil(|py| { /// let capsule = PyCapsule::new(py, (), None).unwrap(); // Oops! `()` is zero sized! /// }); /// ``` pub fn new( py: Python<'_>, value: T, name: Option, ) -> PyResult<&Self> { Self::new_with_destructor(py, value, name, |_, _| {}) } /// Constructs a new capsule whose contents are `value`, associated with `name`. /// /// Also provides a destructor: when the `PyCapsule` is destroyed, it will be passed the original object, /// as well as a `*mut c_void` which will point to the capsule's context, if any. /// /// The `destructor` must be `Send`, because there is no guarantee which thread it will eventually /// be called from. pub fn new_with_destructor< T: 'static + Send + AssertNotZeroSized, F: FnOnce(T, *mut c_void) + Send, >( py: Python<'_>, value: T, name: Option, destructor: F, ) -> PyResult<&'_ Self> { AssertNotZeroSized::assert_not_zero_sized(&value); // Sanity check for capsule layout debug_assert_eq!(memoffset::offset_of!(CapsuleContents::, value), 0); let name_ptr = name.as_ref().map_or(std::ptr::null(), |name| name.as_ptr()); let val = Box::new(CapsuleContents { value, destructor, name, }); unsafe { let cap_ptr = ffi::PyCapsule_New( Box::into_raw(val) as *mut c_void, name_ptr, Some(capsule_destructor::), ); py.from_owned_ptr_or_err(cap_ptr) } } /// Imports an existing capsule. /// /// The `name` should match the path to the module attribute exactly in the form /// of `"module.attribute"`, which should be the same as the name within the capsule. /// /// # Safety /// /// It must be known that the capsule imported by `name` contains an item of type `T`. pub unsafe fn import<'py, T>(py: Python<'py>, name: &CStr) -> PyResult<&'py T> { let ptr = ffi::PyCapsule_Import(name.as_ptr(), false as c_int); if ptr.is_null() { Err(PyErr::fetch(py)) } else { Ok(&*(ptr as *const T)) } } /// Sets the context pointer in the capsule. /// /// Returns an error if this capsule is not valid. /// /// # Notes /// /// The context is treated much like the value of the capsule, but should likely act as /// a place to store any state management when using the capsule. /// /// If you want to store a Rust value as the context, and drop it from the destructor, use /// `Box::into_raw` to convert it into a pointer, see the example. /// /// # Example /// /// ``` /// use std::sync::mpsc::{channel, Sender}; /// use libc::c_void; /// use pyo3::{prelude::*, types::PyCapsule}; /// /// let (tx, rx) = channel::(); /// /// fn destructor(val: u32, context: *mut c_void) { /// let ctx = unsafe { *Box::from_raw(context as *mut Sender) }; /// ctx.send("Destructor called!".to_string()).unwrap(); /// } /// /// Python::with_gil(|py| { /// let capsule = /// PyCapsule::new_with_destructor(py, 123, None, destructor as fn(u32, *mut c_void)) /// .unwrap(); /// let context = Box::new(tx); // `Sender` is our context, box it up and ship it! /// capsule.set_context(Box::into_raw(context) as *mut c_void).unwrap(); /// // This scope will end, causing our destructor to be called... /// }); /// /// assert_eq!(rx.recv(), Ok("Destructor called!".to_string())); /// ``` #[allow(clippy::not_unsafe_ptr_arg_deref)] pub fn set_context(&self, context: *mut c_void) -> PyResult<()> { let result = unsafe { ffi::PyCapsule_SetContext(self.as_ptr(), context) }; if result != 0 { Err(PyErr::fetch(self.py())) } else { Ok(()) } } /// Gets the current context stored in the capsule. If there is no context, the pointer /// will be null. /// /// Returns an error if this capsule is not valid. pub fn context(&self) -> PyResult<*mut c_void> { let ctx = unsafe { ffi::PyCapsule_GetContext(self.as_ptr()) }; if ctx.is_null() { ensure_no_error(self.py())? } Ok(ctx) } /// Obtains a reference to the value of this capsule. /// /// # Safety /// /// It must be known that this capsule is valid and its pointer is to an item of type `T`. pub unsafe fn reference(&self) -> &T { &*(self.pointer() as *const T) } /// Gets the raw `c_void` pointer to the value in this capsule. /// /// Returns null if this capsule is not valid. pub fn pointer(&self) -> *mut c_void { unsafe { let ptr = ffi::PyCapsule_GetPointer(self.0.as_ptr(), self.name_ptr_ignore_error()); if ptr.is_null() { ffi::PyErr_Clear(); } ptr } } /// Checks if this is a valid capsule. /// /// Returns true if the stored `pointer()` is non-null. pub fn is_valid(&self) -> bool { // As well as if the stored pointer is null, PyCapsule_IsValid also returns false if // self.as_ptr() is null or not a ptr to a PyCapsule object. Both of these are guaranteed // to not be the case thanks to invariants of this PyCapsule struct. let r = unsafe { ffi::PyCapsule_IsValid(self.as_ptr(), self.name_ptr_ignore_error()) }; r != 0 } /// Retrieves the name of this capsule, if set. /// /// Returns an error if this capsule is not valid. pub fn name(&self) -> PyResult> { unsafe { let ptr = ffi::PyCapsule_GetName(self.as_ptr()); if ptr.is_null() { ensure_no_error(self.py())?; Ok(None) } else { Ok(Some(CStr::from_ptr(ptr))) } } } /// Attempts to retrieve the raw name pointer of this capsule. /// /// On error, clears the error indicator and returns NULL. This is a private function and next /// use of this capsule will error anyway. fn name_ptr_ignore_error(&self) -> *const c_char { let ptr = unsafe { ffi::PyCapsule_GetName(self.as_ptr()) }; if ptr.is_null() { unsafe { ffi::PyErr_Clear() }; } ptr } } // C layout, as PyCapsule::get_reference depends on `T` being first. #[repr(C)] struct CapsuleContents { /// Value of the capsule value: T, /// Destructor to be used by the capsule destructor: D, /// Name used when creating the capsule name: Option, } // Wrapping ffi::PyCapsule_Destructor for a user supplied FnOnce(T) for capsule destructor unsafe extern "C" fn capsule_destructor( capsule: *mut ffi::PyObject, ) { let ptr = ffi::PyCapsule_GetPointer(capsule, ffi::PyCapsule_GetName(capsule)); let ctx = ffi::PyCapsule_GetContext(capsule); let CapsuleContents { value, destructor, .. } = *Box::from_raw(ptr as *mut CapsuleContents); destructor(value, ctx) } /// Guarantee `T` is not zero sized at compile time. // credit: `` #[doc(hidden)] pub trait AssertNotZeroSized: Sized { const _CONDITION: usize = (std::mem::size_of::() == 0) as usize; const _CHECK: &'static str = ["PyCapsule value type T must not be zero-sized!"][Self::_CONDITION]; #[allow(path_statements, clippy::no_effect)] fn assert_not_zero_sized(&self) { ::_CHECK; } } impl AssertNotZeroSized for T {} fn ensure_no_error(py: Python<'_>) -> PyResult<()> { if let Some(err) = PyErr::take(py) { Err(err) } else { Ok(()) } } #[cfg(test)] mod tests { use libc::c_void; use crate::prelude::PyModule; use crate::{types::PyCapsule, Py, PyResult, Python}; use std::ffi::CString; use std::sync::mpsc::{channel, Sender}; #[test] fn test_pycapsule_struct() -> PyResult<()> { #[repr(C)] struct Foo { pub val: u32, } impl Foo { fn get_val(&self) -> u32 { self.val } } Python::with_gil(|py| -> PyResult<()> { let foo = Foo { val: 123 }; let name = CString::new("foo").unwrap(); let cap = PyCapsule::new(py, foo, Some(name.clone()))?; assert!(cap.is_valid()); let foo_capi = unsafe { cap.reference::() }; assert_eq!(foo_capi.val, 123); assert_eq!(foo_capi.get_val(), 123); assert_eq!(cap.name().unwrap(), Some(name.as_ref())); Ok(()) }) } #[test] fn test_pycapsule_func() { fn foo(x: u32) -> u32 { x } let cap: Py = Python::with_gil(|py| { let name = CString::new("foo").unwrap(); let cap = PyCapsule::new(py, foo as fn(u32) -> u32, Some(name)).unwrap(); cap.into() }); Python::with_gil(|py| { let f = unsafe { cap.as_ref(py).reference:: u32>() }; assert_eq!(f(123), 123); }); } #[test] fn test_pycapsule_context() -> PyResult<()> { Python::with_gil(|py| { let name = CString::new("foo").unwrap(); let cap = PyCapsule::new(py, 0, Some(name))?; let c = cap.context()?; assert!(c.is_null()); let ctx = Box::new(123_u32); cap.set_context(Box::into_raw(ctx) as _)?; let ctx_ptr: *mut c_void = cap.context()?; let ctx = unsafe { *Box::from_raw(ctx_ptr as *mut u32) }; assert_eq!(ctx, 123); Ok(()) }) } #[test] fn test_pycapsule_import() -> PyResult<()> { #[repr(C)] struct Foo { pub val: u32, } Python::with_gil(|py| -> PyResult<()> { let foo = Foo { val: 123 }; let name = CString::new("builtins.capsule").unwrap(); let capsule = PyCapsule::new(py, foo, Some(name.clone()))?; let module = PyModule::import(py, "builtins")?; module.add("capsule", capsule)?; // check error when wrong named passed for capsule. let wrong_name = CString::new("builtins.non_existant").unwrap(); let result: PyResult<&Foo> = unsafe { PyCapsule::import(py, wrong_name.as_ref()) }; assert!(result.is_err()); // corret name is okay. let cap: &Foo = unsafe { PyCapsule::import(py, name.as_ref())? }; assert_eq!(cap.val, 123); Ok(()) }) } #[test] fn test_vec_storage() { let cap: Py = Python::with_gil(|py| { let name = CString::new("foo").unwrap(); let stuff: Vec = vec![1, 2, 3, 4]; let cap = PyCapsule::new(py, stuff, Some(name)).unwrap(); cap.into() }); Python::with_gil(|py| { let ctx: &Vec = unsafe { cap.as_ref(py).reference() }; assert_eq!(ctx, &[1, 2, 3, 4]); }) } #[test] fn test_vec_context() { let context: Vec = vec![1, 2, 3, 4]; let cap: Py = Python::with_gil(|py| { let name = CString::new("foo").unwrap(); let cap = PyCapsule::new(py, 0, Some(name)).unwrap(); cap.set_context(Box::into_raw(Box::new(&context)) as _) .unwrap(); cap.into() }); Python::with_gil(|py| { let ctx_ptr: *mut c_void = cap.as_ref(py).context().unwrap(); let ctx = unsafe { *Box::from_raw(ctx_ptr as *mut &Vec) }; assert_eq!(ctx, &vec![1_u8, 2, 3, 4]); }) } #[test] fn test_pycapsule_destructor() { let (tx, rx) = channel::(); fn destructor(_val: u32, ctx: *mut c_void) { assert!(!ctx.is_null()); let context = unsafe { *Box::from_raw(ctx as *mut Sender) }; context.send(true).unwrap(); } Python::with_gil(|py| { let name = CString::new("foo").unwrap(); let cap = PyCapsule::new_with_destructor(py, 0, Some(name), destructor).unwrap(); cap.set_context(Box::into_raw(Box::new(tx)) as _).unwrap(); }); // the destructor was called. assert_eq!(rx.recv(), Ok(true)); } #[test] fn test_pycapsule_no_name() { Python::with_gil(|py| { let cap = PyCapsule::new(py, 0usize, None).unwrap(); assert_eq!(unsafe { cap.reference::() }, &0usize); assert_eq!(cap.name().unwrap(), None); assert_eq!(cap.context().unwrap(), std::ptr::null_mut()); }); } } pyo3-0.20.2/src/types/code.rs000064400000000000000000000004031046102023000140070ustar 00000000000000use crate::ffi; use crate::PyAny; /// Represents a Python code object. #[repr(transparent)] pub struct PyCode(PyAny); pyobject_native_type_core!( PyCode, pyobject_native_static_type_object!(ffi::PyCode_Type), #checkfunction=ffi::PyCode_Check ); pyo3-0.20.2/src/types/complex.rs000064400000000000000000000156501046102023000145560ustar 00000000000000use crate::{ffi, PyAny, Python}; use std::os::raw::c_double; /// Represents a Python [`complex`](https://docs.python.org/3/library/functions.html#complex) object. /// /// Note that `PyComplex` supports only basic operations. For advanced operations /// consider using [num-complex](https://docs.rs/num-complex)'s [`Complex`] type instead. /// This optional dependency can be activated with the `num-complex` feature flag. /// /// [`Complex`]: https://docs.rs/num-complex/latest/num_complex/struct.Complex.html #[repr(transparent)] pub struct PyComplex(PyAny); pyobject_native_type!( PyComplex, ffi::PyComplexObject, pyobject_native_static_type_object!(ffi::PyComplex_Type), #checkfunction=ffi::PyComplex_Check ); impl PyComplex { /// Creates a new `PyComplex` from the given real and imaginary values. pub fn from_doubles(py: Python<'_>, real: c_double, imag: c_double) -> &PyComplex { unsafe { let ptr = ffi::PyComplex_FromDoubles(real, imag); py.from_owned_ptr(ptr) } } /// Returns the real part of the complex number. pub fn real(&self) -> c_double { unsafe { ffi::PyComplex_RealAsDouble(self.as_ptr()) } } /// Returns the imaginary part of the complex number. pub fn imag(&self) -> c_double { unsafe { ffi::PyComplex_ImagAsDouble(self.as_ptr()) } } } #[cfg(not(any(Py_LIMITED_API, PyPy)))] mod not_limited_impls { use super::*; use std::ops::{Add, Div, Mul, Neg, Sub}; impl PyComplex { /// Returns `|self|`. pub fn abs(&self) -> c_double { unsafe { let val = (*(self.as_ptr() as *mut ffi::PyComplexObject)).cval; ffi::_Py_c_abs(val) } } /// Returns `self` raised to the power of `other`. pub fn pow(&self, other: &PyComplex) -> &PyComplex { unsafe { self.py() .from_owned_ptr(complex_operation(self, other, ffi::_Py_c_pow)) } } } #[inline(always)] unsafe fn complex_operation( l: &PyComplex, r: &PyComplex, operation: unsafe extern "C" fn(ffi::Py_complex, ffi::Py_complex) -> ffi::Py_complex, ) -> *mut ffi::PyObject { let l_val = (*(l.as_ptr() as *mut ffi::PyComplexObject)).cval; let r_val = (*(r.as_ptr() as *mut ffi::PyComplexObject)).cval; ffi::PyComplex_FromCComplex(operation(l_val, r_val)) } impl<'py> Add for &'py PyComplex { type Output = &'py PyComplex; fn add(self, other: &'py PyComplex) -> &'py PyComplex { unsafe { self.py() .from_owned_ptr(complex_operation(self, other, ffi::_Py_c_sum)) } } } impl<'py> Sub for &'py PyComplex { type Output = &'py PyComplex; fn sub(self, other: &'py PyComplex) -> &'py PyComplex { unsafe { self.py() .from_owned_ptr(complex_operation(self, other, ffi::_Py_c_diff)) } } } impl<'py> Mul for &'py PyComplex { type Output = &'py PyComplex; fn mul(self, other: &'py PyComplex) -> &'py PyComplex { unsafe { self.py() .from_owned_ptr(complex_operation(self, other, ffi::_Py_c_prod)) } } } impl<'py> Div for &'py PyComplex { type Output = &'py PyComplex; fn div(self, other: &'py PyComplex) -> &'py PyComplex { unsafe { self.py() .from_owned_ptr(complex_operation(self, other, ffi::_Py_c_quot)) } } } impl<'py> Neg for &'py PyComplex { type Output = &'py PyComplex; fn neg(self) -> &'py PyComplex { unsafe { let val = (*(self.as_ptr() as *mut ffi::PyComplexObject)).cval; self.py() .from_owned_ptr(ffi::PyComplex_FromCComplex(ffi::_Py_c_neg(val))) } } } #[cfg(test)] mod tests { use super::PyComplex; use crate::Python; use assert_approx_eq::assert_approx_eq; #[test] fn test_add() { Python::with_gil(|py| { let l = PyComplex::from_doubles(py, 3.0, 1.2); let r = PyComplex::from_doubles(py, 1.0, 2.6); let res = l + r; assert_approx_eq!(res.real(), 4.0); assert_approx_eq!(res.imag(), 3.8); }); } #[test] fn test_sub() { Python::with_gil(|py| { let l = PyComplex::from_doubles(py, 3.0, 1.2); let r = PyComplex::from_doubles(py, 1.0, 2.6); let res = l - r; assert_approx_eq!(res.real(), 2.0); assert_approx_eq!(res.imag(), -1.4); }); } #[test] fn test_mul() { Python::with_gil(|py| { let l = PyComplex::from_doubles(py, 3.0, 1.2); let r = PyComplex::from_doubles(py, 1.0, 2.6); let res = l * r; assert_approx_eq!(res.real(), -0.12); assert_approx_eq!(res.imag(), 9.0); }); } #[test] fn test_div() { Python::with_gil(|py| { let l = PyComplex::from_doubles(py, 3.0, 1.2); let r = PyComplex::from_doubles(py, 1.0, 2.6); let res = l / r; assert_approx_eq!(res.real(), 0.788_659_793_814_432_9); assert_approx_eq!(res.imag(), -0.850_515_463_917_525_7); }); } #[test] fn test_neg() { Python::with_gil(|py| { let val = PyComplex::from_doubles(py, 3.0, 1.2); let res = -val; assert_approx_eq!(res.real(), -3.0); assert_approx_eq!(res.imag(), -1.2); }); } #[test] fn test_abs() { Python::with_gil(|py| { let val = PyComplex::from_doubles(py, 3.0, 1.2); assert_approx_eq!(val.abs(), 3.231_098_884_280_702_2); }); } #[test] fn test_pow() { Python::with_gil(|py| { let l = PyComplex::from_doubles(py, 3.0, 1.2); let r = PyComplex::from_doubles(py, 1.2, 2.6); let val = l.pow(r); assert_approx_eq!(val.real(), -1.419_309_997_016_603_7); assert_approx_eq!(val.imag(), -0.541_297_466_033_544_6); }); } } } #[cfg(test)] mod tests { use super::PyComplex; use crate::Python; use assert_approx_eq::assert_approx_eq; #[test] fn test_from_double() { use assert_approx_eq::assert_approx_eq; Python::with_gil(|py| { let complex = PyComplex::from_doubles(py, 3.0, 1.2); assert_approx_eq!(complex.real(), 3.0); assert_approx_eq!(complex.imag(), 1.2); }); } } pyo3-0.20.2/src/types/datetime.rs000064400000000000000000000464211046102023000147030ustar 00000000000000//! Safe Rust wrappers for types defined in the Python `datetime` library //! //! For more details about these types, see the [Python //! documentation](https://docs.python.org/3/library/datetime.html) use crate::err::PyResult; use crate::ffi::{ self, PyDateTime_CAPI, PyDateTime_FromTimestamp, PyDateTime_IMPORT, PyDate_FromTimestamp, }; use crate::ffi::{ PyDateTime_DATE_GET_FOLD, PyDateTime_DATE_GET_HOUR, PyDateTime_DATE_GET_MICROSECOND, PyDateTime_DATE_GET_MINUTE, PyDateTime_DATE_GET_SECOND, }; use crate::ffi::{ PyDateTime_DELTA_GET_DAYS, PyDateTime_DELTA_GET_MICROSECONDS, PyDateTime_DELTA_GET_SECONDS, }; use crate::ffi::{PyDateTime_GET_DAY, PyDateTime_GET_MONTH, PyDateTime_GET_YEAR}; use crate::ffi::{ PyDateTime_TIME_GET_FOLD, PyDateTime_TIME_GET_HOUR, PyDateTime_TIME_GET_MICROSECOND, PyDateTime_TIME_GET_MINUTE, PyDateTime_TIME_GET_SECOND, }; use crate::instance::PyNativeType; use crate::types::PyTuple; use crate::{IntoPy, Py, PyAny, Python}; use std::os::raw::c_int; fn ensure_datetime_api(_py: Python<'_>) -> &'static PyDateTime_CAPI { unsafe { if pyo3_ffi::PyDateTimeAPI().is_null() { PyDateTime_IMPORT() } &*pyo3_ffi::PyDateTimeAPI() } } // Type Check macros // // These are bindings around the C API typecheck macros, all of them return // `1` if True and `0` if False. In all type check macros, the argument (`op`) // must not be `NULL`. The implementations here all call ensure_datetime_api // to ensure that the PyDateTimeAPI is initialized before use // // // # Safety // // These functions must only be called when the GIL is held! macro_rules! ffi_fun_with_autoinit { ($(#[$outer:meta] unsafe fn $name: ident($arg: ident: *mut PyObject) -> $ret: ty;)*) => { $( #[$outer] #[allow(non_snake_case)] /// # Safety /// /// Must only be called while the GIL is held unsafe fn $name($arg: *mut crate::ffi::PyObject) -> $ret { let _ = ensure_datetime_api(Python::assume_gil_acquired()); crate::ffi::$name($arg) } )* }; } ffi_fun_with_autoinit! { /// Check if `op` is a `PyDateTimeAPI.DateType` or subtype. unsafe fn PyDate_Check(op: *mut PyObject) -> c_int; /// Check if `op` is a `PyDateTimeAPI.DateTimeType` or subtype. unsafe fn PyDateTime_Check(op: *mut PyObject) -> c_int; /// Check if `op` is a `PyDateTimeAPI.TimeType` or subtype. unsafe fn PyTime_Check(op: *mut PyObject) -> c_int; /// Check if `op` is a `PyDateTimeAPI.DetaType` or subtype. unsafe fn PyDelta_Check(op: *mut PyObject) -> c_int; /// Check if `op` is a `PyDateTimeAPI.TZInfoType` or subtype. unsafe fn PyTZInfo_Check(op: *mut PyObject) -> c_int; } // Access traits /// Trait for accessing the date components of a struct containing a date. pub trait PyDateAccess { /// Returns the year, as a positive int. /// /// Implementations should conform to the upstream documentation: /// fn get_year(&self) -> i32; /// Returns the month, as an int from 1 through 12. /// /// Implementations should conform to the upstream documentation: /// fn get_month(&self) -> u8; /// Returns the day, as an int from 1 through 31. /// /// Implementations should conform to the upstream documentation: /// fn get_day(&self) -> u8; } /// Trait for accessing the components of a struct containing a timedelta. /// /// Note: These access the individual components of a (day, second, /// microsecond) representation of the delta, they are *not* intended as /// aliases for calculating the total duration in each of these units. pub trait PyDeltaAccess { /// Returns the number of days, as an int from -999999999 to 999999999. /// /// Implementations should conform to the upstream documentation: /// fn get_days(&self) -> i32; /// Returns the number of seconds, as an int from 0 through 86399. /// /// Implementations should conform to the upstream documentation: /// fn get_seconds(&self) -> i32; /// Returns the number of microseconds, as an int from 0 through 999999. /// /// Implementations should conform to the upstream documentation: /// fn get_microseconds(&self) -> i32; } /// Trait for accessing the time components of a struct containing a time. pub trait PyTimeAccess { /// Returns the hour, as an int from 0 through 23. /// /// Implementations should conform to the upstream documentation: /// fn get_hour(&self) -> u8; /// Returns the minute, as an int from 0 through 59. /// /// Implementations should conform to the upstream documentation: /// fn get_minute(&self) -> u8; /// Returns the second, as an int from 0 through 59. /// /// Implementations should conform to the upstream documentation: /// fn get_second(&self) -> u8; /// Returns the microsecond, as an int from 0 through 999999. /// /// Implementations should conform to the upstream documentation: /// fn get_microsecond(&self) -> u32; /// Returns whether this date is the later of two moments with the /// same representation, during a repeated interval. /// /// This typically occurs at the end of daylight savings time. Only valid if the /// represented time is ambiguous. /// See [PEP 495](https://www.python.org/dev/peps/pep-0495/) for more detail. fn get_fold(&self) -> bool; } /// Trait for accessing the components of a struct containing a tzinfo. pub trait PyTzInfoAccess { /// Returns the tzinfo (which may be None). /// /// Implementations should conform to the upstream documentation: /// /// fn get_tzinfo(&self) -> Option<&PyTzInfo>; } /// Bindings around `datetime.date` #[repr(transparent)] pub struct PyDate(PyAny); pyobject_native_type!( PyDate, crate::ffi::PyDateTime_Date, |py| ensure_datetime_api(py).DateType, #module=Some("datetime"), #checkfunction=PyDate_Check ); impl PyDate { /// Creates a new `datetime.date`. pub fn new(py: Python<'_>, year: i32, month: u8, day: u8) -> PyResult<&PyDate> { unsafe { let ptr = (ensure_datetime_api(py).Date_FromDate)( year, c_int::from(month), c_int::from(day), ensure_datetime_api(py).DateType, ); py.from_owned_ptr_or_err(ptr) } } /// Construct a `datetime.date` from a POSIX timestamp /// /// This is equivalent to `datetime.date.fromtimestamp` pub fn from_timestamp(py: Python<'_>, timestamp: i64) -> PyResult<&PyDate> { let time_tuple = PyTuple::new(py, [timestamp]); // safety ensure that the API is loaded let _api = ensure_datetime_api(py); unsafe { let ptr = PyDate_FromTimestamp(time_tuple.as_ptr()); py.from_owned_ptr_or_err(ptr) } } } impl PyDateAccess for PyDate { fn get_year(&self) -> i32 { unsafe { PyDateTime_GET_YEAR(self.as_ptr()) } } fn get_month(&self) -> u8 { unsafe { PyDateTime_GET_MONTH(self.as_ptr()) as u8 } } fn get_day(&self) -> u8 { unsafe { PyDateTime_GET_DAY(self.as_ptr()) as u8 } } } /// Bindings for `datetime.datetime` #[repr(transparent)] pub struct PyDateTime(PyAny); pyobject_native_type!( PyDateTime, crate::ffi::PyDateTime_DateTime, |py| ensure_datetime_api(py).DateTimeType, #module=Some("datetime"), #checkfunction=PyDateTime_Check ); impl PyDateTime { /// Creates a new `datetime.datetime` object. #[allow(clippy::too_many_arguments)] pub fn new<'p>( py: Python<'p>, year: i32, month: u8, day: u8, hour: u8, minute: u8, second: u8, microsecond: u32, tzinfo: Option<&PyTzInfo>, ) -> PyResult<&'p PyDateTime> { let api = ensure_datetime_api(py); unsafe { let ptr = (api.DateTime_FromDateAndTime)( year, c_int::from(month), c_int::from(day), c_int::from(hour), c_int::from(minute), c_int::from(second), microsecond as c_int, opt_to_pyobj(tzinfo), api.DateTimeType, ); py.from_owned_ptr_or_err(ptr) } } /// Alternate constructor that takes a `fold` parameter. A `true` value for this parameter /// signifies this this datetime is the later of two moments with the same representation, /// during a repeated interval. /// /// This typically occurs at the end of daylight savings time. Only valid if the /// represented time is ambiguous. /// See [PEP 495](https://www.python.org/dev/peps/pep-0495/) for more detail. #[allow(clippy::too_many_arguments)] pub fn new_with_fold<'p>( py: Python<'p>, year: i32, month: u8, day: u8, hour: u8, minute: u8, second: u8, microsecond: u32, tzinfo: Option<&PyTzInfo>, fold: bool, ) -> PyResult<&'p PyDateTime> { let api = ensure_datetime_api(py); unsafe { let ptr = (api.DateTime_FromDateAndTimeAndFold)( year, c_int::from(month), c_int::from(day), c_int::from(hour), c_int::from(minute), c_int::from(second), microsecond as c_int, opt_to_pyobj(tzinfo), c_int::from(fold), api.DateTimeType, ); py.from_owned_ptr_or_err(ptr) } } /// Construct a `datetime` object from a POSIX timestamp /// /// This is equivalent to `datetime.datetime.fromtimestamp` pub fn from_timestamp<'p>( py: Python<'p>, timestamp: f64, tzinfo: Option<&PyTzInfo>, ) -> PyResult<&'p PyDateTime> { let args: Py = (timestamp, tzinfo).into_py(py); // safety ensure API is loaded let _api = ensure_datetime_api(py); unsafe { let ptr = PyDateTime_FromTimestamp(args.as_ptr()); py.from_owned_ptr_or_err(ptr) } } } impl PyDateAccess for PyDateTime { fn get_year(&self) -> i32 { unsafe { PyDateTime_GET_YEAR(self.as_ptr()) } } fn get_month(&self) -> u8 { unsafe { PyDateTime_GET_MONTH(self.as_ptr()) as u8 } } fn get_day(&self) -> u8 { unsafe { PyDateTime_GET_DAY(self.as_ptr()) as u8 } } } impl PyTimeAccess for PyDateTime { fn get_hour(&self) -> u8 { unsafe { PyDateTime_DATE_GET_HOUR(self.as_ptr()) as u8 } } fn get_minute(&self) -> u8 { unsafe { PyDateTime_DATE_GET_MINUTE(self.as_ptr()) as u8 } } fn get_second(&self) -> u8 { unsafe { PyDateTime_DATE_GET_SECOND(self.as_ptr()) as u8 } } fn get_microsecond(&self) -> u32 { unsafe { PyDateTime_DATE_GET_MICROSECOND(self.as_ptr()) as u32 } } fn get_fold(&self) -> bool { unsafe { PyDateTime_DATE_GET_FOLD(self.as_ptr()) > 0 } } } impl PyTzInfoAccess for PyDateTime { fn get_tzinfo(&self) -> Option<&PyTzInfo> { let ptr = self.as_ptr() as *mut ffi::PyDateTime_DateTime; unsafe { if (*ptr).hastzinfo != 0 { Some(self.py().from_borrowed_ptr((*ptr).tzinfo)) } else { None } } } } /// Bindings for `datetime.time` #[repr(transparent)] pub struct PyTime(PyAny); pyobject_native_type!( PyTime, crate::ffi::PyDateTime_Time, |py| ensure_datetime_api(py).TimeType, #module=Some("datetime"), #checkfunction=PyTime_Check ); impl PyTime { /// Creates a new `datetime.time` object. pub fn new<'p>( py: Python<'p>, hour: u8, minute: u8, second: u8, microsecond: u32, tzinfo: Option<&PyTzInfo>, ) -> PyResult<&'p PyTime> { let api = ensure_datetime_api(py); unsafe { let ptr = (api.Time_FromTime)( c_int::from(hour), c_int::from(minute), c_int::from(second), microsecond as c_int, opt_to_pyobj(tzinfo), api.TimeType, ); py.from_owned_ptr_or_err(ptr) } } /// Alternate constructor that takes a `fold` argument. See [`PyDateTime::new_with_fold`]. pub fn new_with_fold<'p>( py: Python<'p>, hour: u8, minute: u8, second: u8, microsecond: u32, tzinfo: Option<&PyTzInfo>, fold: bool, ) -> PyResult<&'p PyTime> { let api = ensure_datetime_api(py); unsafe { let ptr = (api.Time_FromTimeAndFold)( c_int::from(hour), c_int::from(minute), c_int::from(second), microsecond as c_int, opt_to_pyobj(tzinfo), fold as c_int, api.TimeType, ); py.from_owned_ptr_or_err(ptr) } } } impl PyTimeAccess for PyTime { fn get_hour(&self) -> u8 { unsafe { PyDateTime_TIME_GET_HOUR(self.as_ptr()) as u8 } } fn get_minute(&self) -> u8 { unsafe { PyDateTime_TIME_GET_MINUTE(self.as_ptr()) as u8 } } fn get_second(&self) -> u8 { unsafe { PyDateTime_TIME_GET_SECOND(self.as_ptr()) as u8 } } fn get_microsecond(&self) -> u32 { unsafe { PyDateTime_TIME_GET_MICROSECOND(self.as_ptr()) as u32 } } fn get_fold(&self) -> bool { unsafe { PyDateTime_TIME_GET_FOLD(self.as_ptr()) != 0 } } } impl PyTzInfoAccess for PyTime { fn get_tzinfo(&self) -> Option<&PyTzInfo> { let ptr = self.as_ptr() as *mut ffi::PyDateTime_Time; unsafe { if (*ptr).hastzinfo != 0 { Some(self.py().from_borrowed_ptr((*ptr).tzinfo)) } else { None } } } } /// Bindings for `datetime.tzinfo`. /// /// This is an abstract base class and cannot be constructed directly. /// For concrete time zone implementations, see [`timezone_utc`] and /// the [`zoneinfo` module](https://docs.python.org/3/library/zoneinfo.html). #[repr(transparent)] pub struct PyTzInfo(PyAny); pyobject_native_type!( PyTzInfo, crate::ffi::PyObject, |py| ensure_datetime_api(py).TZInfoType, #module=Some("datetime"), #checkfunction=PyTZInfo_Check ); /// Equivalent to `datetime.timezone.utc` pub fn timezone_utc(py: Python<'_>) -> &PyTzInfo { unsafe { &*(ensure_datetime_api(py).TimeZone_UTC as *const PyTzInfo) } } /// Bindings for `datetime.timedelta` #[repr(transparent)] pub struct PyDelta(PyAny); pyobject_native_type!( PyDelta, crate::ffi::PyDateTime_Delta, |py| ensure_datetime_api(py).DeltaType, #module=Some("datetime"), #checkfunction=PyDelta_Check ); impl PyDelta { /// Creates a new `timedelta`. pub fn new( py: Python<'_>, days: i32, seconds: i32, microseconds: i32, normalize: bool, ) -> PyResult<&PyDelta> { let api = ensure_datetime_api(py); unsafe { let ptr = (api.Delta_FromDelta)( days as c_int, seconds as c_int, microseconds as c_int, normalize as c_int, api.DeltaType, ); py.from_owned_ptr_or_err(ptr) } } } impl PyDeltaAccess for PyDelta { fn get_days(&self) -> i32 { unsafe { PyDateTime_DELTA_GET_DAYS(self.as_ptr()) } } fn get_seconds(&self) -> i32 { unsafe { PyDateTime_DELTA_GET_SECONDS(self.as_ptr()) } } fn get_microseconds(&self) -> i32 { unsafe { PyDateTime_DELTA_GET_MICROSECONDS(self.as_ptr()) } } } // Utility function which returns a borrowed reference to either // the underlying tzinfo or None. fn opt_to_pyobj(opt: Option<&PyTzInfo>) -> *mut ffi::PyObject { match opt { Some(tzi) => tzi.as_ptr(), None => unsafe { ffi::Py_None() }, } } #[cfg(test)] mod tests { use super::*; #[cfg(feature = "macros")] use crate::py_run; #[test] #[cfg(feature = "macros")] #[cfg_attr(target_arch = "wasm32", ignore)] // DateTime import fails on wasm for mysterious reasons fn test_datetime_fromtimestamp() { Python::with_gil(|py| { let dt = PyDateTime::from_timestamp(py, 100.0, None).unwrap(); py_run!( py, dt, "import datetime; assert dt == datetime.datetime.fromtimestamp(100)" ); let dt = PyDateTime::from_timestamp(py, 100.0, Some(timezone_utc(py))).unwrap(); py_run!( py, dt, "import datetime; assert dt == datetime.datetime.fromtimestamp(100, datetime.timezone.utc)" ); }) } #[test] #[cfg(feature = "macros")] #[cfg_attr(target_arch = "wasm32", ignore)] // DateTime import fails on wasm for mysterious reasons fn test_date_fromtimestamp() { Python::with_gil(|py| { let dt = PyDate::from_timestamp(py, 100).unwrap(); py_run!( py, dt, "import datetime; assert dt == datetime.date.fromtimestamp(100)" ); }) } #[test] #[cfg_attr(target_arch = "wasm32", ignore)] // DateTime import fails on wasm for mysterious reasons fn test_new_with_fold() { Python::with_gil(|py| { let a = PyDateTime::new_with_fold(py, 2021, 1, 23, 20, 32, 40, 341516, None, false); let b = PyDateTime::new_with_fold(py, 2021, 1, 23, 20, 32, 40, 341516, None, true); assert!(!a.unwrap().get_fold()); assert!(b.unwrap().get_fold()); }); } #[test] #[cfg_attr(target_arch = "wasm32", ignore)] // DateTime import fails on wasm for mysterious reasons fn test_get_tzinfo() { crate::Python::with_gil(|py| { let utc = timezone_utc(py); let dt = PyDateTime::new(py, 2018, 1, 1, 0, 0, 0, 0, Some(utc)).unwrap(); assert!(dt.get_tzinfo().unwrap().eq(utc).unwrap()); let dt = PyDateTime::new(py, 2018, 1, 1, 0, 0, 0, 0, None).unwrap(); assert!(dt.get_tzinfo().is_none()); let t = PyTime::new(py, 0, 0, 0, 0, Some(utc)).unwrap(); assert!(t.get_tzinfo().unwrap().eq(utc).unwrap()); let t = PyTime::new(py, 0, 0, 0, 0, None).unwrap(); assert!(t.get_tzinfo().is_none()); }); } } pyo3-0.20.2/src/types/dict.rs000064400000000000000000001060341046102023000140270ustar 00000000000000use super::PyMapping; use crate::err::{self, PyErr, PyResult}; use crate::ffi::Py_ssize_t; use crate::types::{PyAny, PyList}; use crate::{ffi, PyObject, Python, ToPyObject}; /// Represents a Python `dict`. #[repr(transparent)] pub struct PyDict(PyAny); pyobject_native_type!( PyDict, ffi::PyDictObject, pyobject_native_static_type_object!(ffi::PyDict_Type), #checkfunction=ffi::PyDict_Check ); /// Represents a Python `dict_keys`. #[cfg(not(PyPy))] #[repr(transparent)] pub struct PyDictKeys(PyAny); #[cfg(not(PyPy))] pyobject_native_type_core!( PyDictKeys, pyobject_native_static_type_object!(ffi::PyDictKeys_Type), #checkfunction=ffi::PyDictKeys_Check ); /// Represents a Python `dict_values`. #[cfg(not(PyPy))] #[repr(transparent)] pub struct PyDictValues(PyAny); #[cfg(not(PyPy))] pyobject_native_type_core!( PyDictValues, pyobject_native_static_type_object!(ffi::PyDictValues_Type), #checkfunction=ffi::PyDictValues_Check ); /// Represents a Python `dict_items`. #[cfg(not(PyPy))] #[repr(transparent)] pub struct PyDictItems(PyAny); #[cfg(not(PyPy))] pyobject_native_type_core!( PyDictItems, pyobject_native_static_type_object!(ffi::PyDictItems_Type), #checkfunction=ffi::PyDictItems_Check ); impl PyDict { /// Creates a new empty dictionary. pub fn new(py: Python<'_>) -> &PyDict { unsafe { py.from_owned_ptr::(ffi::PyDict_New()) } } /// Creates a new dictionary from the sequence given. /// /// The sequence must consist of `(PyObject, PyObject)`. This is /// equivalent to `dict([("a", 1), ("b", 2)])`. /// /// Returns an error on invalid input. In the case of key collisions, /// this keeps the last entry seen. #[cfg(not(PyPy))] pub fn from_sequence(py: Python<'_>, seq: PyObject) -> PyResult<&PyDict> { let dict = Self::new(py); err::error_on_minusone(py, unsafe { ffi::PyDict_MergeFromSeq2(dict.into_ptr(), seq.into_ptr(), 1) })?; Ok(dict) } /// Returns a new dictionary that contains the same key-value pairs as self. /// /// This is equivalent to the Python expression `self.copy()`. pub fn copy(&self) -> PyResult<&PyDict> { unsafe { self.py() .from_owned_ptr_or_err::(ffi::PyDict_Copy(self.as_ptr())) } } /// Empties an existing dictionary of all key-value pairs. pub fn clear(&self) { unsafe { ffi::PyDict_Clear(self.as_ptr()) } } /// Return the number of items in the dictionary. /// /// This is equivalent to the Python expression `len(self)`. pub fn len(&self) -> usize { self._len() as usize } fn _len(&self) -> Py_ssize_t { #[cfg(any(not(Py_3_8), PyPy, Py_LIMITED_API))] unsafe { ffi::PyDict_Size(self.as_ptr()) } #[cfg(all(Py_3_8, not(PyPy), not(Py_LIMITED_API)))] unsafe { (*self.as_ptr().cast::()).ma_used } } /// Checks if the dict is empty, i.e. `len(self) == 0`. pub fn is_empty(&self) -> bool { self.len() == 0 } /// Determines if the dictionary contains the specified key. /// /// This is equivalent to the Python expression `key in self`. pub fn contains(&self, key: K) -> PyResult where K: ToPyObject, { fn inner(dict: &PyDict, key: PyObject) -> PyResult { match unsafe { ffi::PyDict_Contains(dict.as_ptr(), key.as_ptr()) } { 1 => Ok(true), 0 => Ok(false), _ => Err(PyErr::fetch(dict.py())), } } inner(self, key.to_object(self.py())) } /// Gets an item from the dictionary. /// /// Returns `Ok(None)` if the item is not present. To get a `KeyError` for /// non-existing keys, use [`PyAny::get_item`]. /// /// Returns `Err(PyErr)` if Python magic methods `__hash__` or `__eq__` used in dictionary /// lookup raise an exception, for example if the key `K` is not hashable. Usually it is /// best to bubble this error up to the caller using the `?` operator. /// /// # Examples /// /// The following example calls `get_item` for the dictionary `{"a": 1}` with various /// keys. /// - `get_item("a")` returns `Ok(Some(...))`, with the `PyAny` being a reference to the Python /// int `1`. /// - `get_item("b")` returns `Ok(None)`, because "b" is not in the dictionary. /// - `get_item(dict)` returns an `Err(PyErr)`. The error will be a `TypeError` because a dict is not /// hashable. /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::{PyDict, IntoPyDict}; /// use pyo3::exceptions::{PyTypeError, PyKeyError}; /// /// # fn main() { /// # let _ = /// Python::with_gil(|py| -> PyResult<()> { /// let dict: &PyDict = [("a", 1)].into_py_dict(py); /// // `a` is in the dictionary, with value 1 /// assert!(dict.get_item("a")?.map_or(Ok(false), |x| x.eq(1))?); /// // `b` is not in the dictionary /// assert!(dict.get_item("b")?.is_none()); /// // `dict` is not hashable, so this returns an error /// assert!(dict.get_item(dict).unwrap_err().is_instance_of::(py)); /// /// // `PyAny::get_item("b")` will raise a `KeyError` instead of returning `None` /// let any: &PyAny = dict.as_ref(); /// assert!(any.get_item("b").unwrap_err().is_instance_of::(py)); /// Ok(()) /// }); /// # } /// ``` pub fn get_item(&self, key: K) -> PyResult> where K: ToPyObject, { fn inner(dict: &PyDict, key: PyObject) -> PyResult> { let py = dict.py(); // PyDict_GetItemWithError returns a borrowed ptr, must make it owned for safety (see #890). // PyObject::from_borrowed_ptr_or_opt will take ownership in this way. unsafe { PyObject::from_borrowed_ptr_or_opt( py, ffi::PyDict_GetItemWithError(dict.as_ptr(), key.as_ptr()), ) } .map(|pyobject| Ok(pyobject.into_ref(py))) .or_else(|| PyErr::take(py).map(Err)) .transpose() } inner(self, key.to_object(self.py())) } /// Deprecated version of `get_item`. #[deprecated( since = "0.20.0", note = "this is now equivalent to `PyDict::get_item`" )] #[inline] pub fn get_item_with_error(&self, key: K) -> PyResult> where K: ToPyObject, { self.get_item(key) } /// Sets an item value. /// /// This is equivalent to the Python statement `self[key] = value`. pub fn set_item(&self, key: K, value: V) -> PyResult<()> where K: ToPyObject, V: ToPyObject, { fn inner(dict: &PyDict, key: PyObject, value: PyObject) -> PyResult<()> { err::error_on_minusone(dict.py(), unsafe { ffi::PyDict_SetItem(dict.as_ptr(), key.as_ptr(), value.as_ptr()) }) } let py = self.py(); inner(self, key.to_object(py), value.to_object(py)) } /// Deletes an item. /// /// This is equivalent to the Python statement `del self[key]`. pub fn del_item(&self, key: K) -> PyResult<()> where K: ToPyObject, { fn inner(dict: &PyDict, key: PyObject) -> PyResult<()> { err::error_on_minusone(dict.py(), unsafe { ffi::PyDict_DelItem(dict.as_ptr(), key.as_ptr()) }) } inner(self, key.to_object(self.py())) } /// Returns a list of dict keys. /// /// This is equivalent to the Python expression `list(dict.keys())`. pub fn keys(&self) -> &PyList { unsafe { self.py() .from_owned_ptr::(ffi::PyDict_Keys(self.as_ptr())) } } /// Returns a list of dict values. /// /// This is equivalent to the Python expression `list(dict.values())`. pub fn values(&self) -> &PyList { unsafe { self.py() .from_owned_ptr::(ffi::PyDict_Values(self.as_ptr())) } } /// Returns a list of dict items. /// /// This is equivalent to the Python expression `list(dict.items())`. pub fn items(&self) -> &PyList { unsafe { self.py() .from_owned_ptr::(ffi::PyDict_Items(self.as_ptr())) } } /// Returns an iterator of `(key, value)` pairs in this dictionary. /// /// # Panics /// /// If PyO3 detects that the dictionary is mutated during iteration, it will panic. /// It is allowed to modify values as you iterate over the dictionary, but only /// so long as the set of keys does not change. pub fn iter(&self) -> PyDictIterator<'_> { IntoIterator::into_iter(self) } /// Returns `self` cast as a `PyMapping`. pub fn as_mapping(&self) -> &PyMapping { unsafe { self.downcast_unchecked() } } /// Update this dictionary with the key/value pairs from another. /// /// This is equivalent to the Python expression `self.update(other)`. If `other` is a `PyDict`, you may want /// to use `self.update(other.as_mapping())`, note: `PyDict::as_mapping` is a zero-cost conversion. pub fn update(&self, other: &PyMapping) -> PyResult<()> { let py = self.py(); err::error_on_minusone(py, unsafe { ffi::PyDict_Update(self.as_ptr(), other.as_ptr()) }) } /// Add key/value pairs from another dictionary to this one only when they do not exist in this. /// /// This is equivalent to the Python expression `self.update({k: v for k, v in other.items() if k not in self})`. /// If `other` is a `PyDict`, you may want to use `self.update_if_missing(other.as_mapping())`, /// note: `PyDict::as_mapping` is a zero-cost conversion. /// /// This method uses [`PyDict_Merge`](https://docs.python.org/3/c-api/dict.html#c.PyDict_Merge) internally, /// so should have the same performance as `update`. pub fn update_if_missing(&self, other: &PyMapping) -> PyResult<()> { let py = self.py(); err::error_on_minusone(py, unsafe { ffi::PyDict_Merge(self.as_ptr(), other.as_ptr(), 0) }) } } /// PyO3 implementation of an iterator for a Python `dict` object. pub struct PyDictIterator<'py> { dict: &'py PyDict, ppos: ffi::Py_ssize_t, di_used: ffi::Py_ssize_t, len: ffi::Py_ssize_t, } impl<'py> Iterator for PyDictIterator<'py> { type Item = (&'py PyAny, &'py PyAny); #[inline] fn next(&mut self) -> Option { let ma_used = self.dict._len(); // These checks are similar to what CPython does. // // If the dimension of the dict changes e.g. key-value pairs are removed // or added during iteration, this will panic next time when `next` is called if self.di_used != ma_used { self.di_used = -1; panic!("dictionary changed size during iteration"); }; // If the dict is changed in such a way that the length remains constant // then this will panic at the end of iteration - similar to this: // // d = {"a":1, "b":2, "c": 3} // // for k, v in d.items(): // d[f"{k}_"] = 4 // del d[k] // print(k) // if self.len == -1 { self.di_used = -1; panic!("dictionary keys changed during iteration"); }; let ret = unsafe { self.next_unchecked() }; if ret.is_some() { self.len -= 1 } ret } #[inline] fn size_hint(&self) -> (usize, Option) { let len = self.len(); (len, Some(len)) } } impl<'py> ExactSizeIterator for PyDictIterator<'py> { fn len(&self) -> usize { self.len as usize } } impl<'a> std::iter::IntoIterator for &'a PyDict { type Item = (&'a PyAny, &'a PyAny); type IntoIter = PyDictIterator<'a>; fn into_iter(self) -> Self::IntoIter { PyDictIterator { dict: self, ppos: 0, di_used: self._len(), len: self._len(), } } } impl<'py> PyDictIterator<'py> { /// Advances the iterator without checking for concurrent modification. /// /// See [`PyDict_Next`](https://docs.python.org/3/c-api/dict.html#c.PyDict_Next) /// for more information. unsafe fn next_unchecked(&mut self) -> Option<(&'py PyAny, &'py PyAny)> { let mut key: *mut ffi::PyObject = std::ptr::null_mut(); let mut value: *mut ffi::PyObject = std::ptr::null_mut(); if ffi::PyDict_Next(self.dict.as_ptr(), &mut self.ppos, &mut key, &mut value) != 0 { let py = self.dict.py(); // PyDict_Next returns borrowed values; for safety must make them owned (see #890) Some(( py.from_owned_ptr(ffi::_Py_NewRef(key)), py.from_owned_ptr(ffi::_Py_NewRef(value)), )) } else { None } } } /// Conversion trait that allows a sequence of tuples to be converted into `PyDict` /// Primary use case for this trait is `call` and `call_method` methods as keywords argument. pub trait IntoPyDict { /// Converts self into a `PyDict` object pointer. Whether pointer owned or borrowed /// depends on implementation. fn into_py_dict(self, py: Python<'_>) -> &PyDict; } impl IntoPyDict for I where T: PyDictItem, I: IntoIterator, { fn into_py_dict(self, py: Python<'_>) -> &PyDict { let dict = PyDict::new(py); for item in self { dict.set_item(item.key(), item.value()) .expect("Failed to set_item on dict"); } dict } } /// Represents a tuple which can be used as a PyDict item. pub trait PyDictItem { type K: ToPyObject; type V: ToPyObject; fn key(&self) -> &Self::K; fn value(&self) -> &Self::V; } impl PyDictItem for (K, V) where K: ToPyObject, V: ToPyObject, { type K = K; type V = V; fn key(&self) -> &Self::K { &self.0 } fn value(&self) -> &Self::V { &self.1 } } impl PyDictItem for &(K, V) where K: ToPyObject, V: ToPyObject, { type K = K; type V = V; fn key(&self) -> &Self::K { &self.0 } fn value(&self) -> &Self::V { &self.1 } } #[cfg(test)] mod tests { use super::*; #[cfg(not(PyPy))] use crate::exceptions; #[cfg(not(PyPy))] use crate::types::PyList; use crate::{types::PyTuple, Python, ToPyObject}; use std::collections::{BTreeMap, HashMap}; #[test] fn test_new() { Python::with_gil(|py| { let dict = [(7, 32)].into_py_dict(py); assert_eq!( 32, dict.get_item(7i32) .unwrap() .unwrap() .extract::() .unwrap() ); assert!(dict.get_item(8i32).unwrap().is_none()); let map: HashMap = [(7, 32)].iter().cloned().collect(); assert_eq!(map, dict.extract().unwrap()); let map: BTreeMap = [(7, 32)].iter().cloned().collect(); assert_eq!(map, dict.extract().unwrap()); }); } #[test] #[cfg(not(PyPy))] fn test_from_sequence() { Python::with_gil(|py| { let items = PyList::new(py, &vec![("a", 1), ("b", 2)]); let dict = PyDict::from_sequence(py, items.to_object(py)).unwrap(); assert_eq!( 1, dict.get_item("a") .unwrap() .unwrap() .extract::() .unwrap() ); assert_eq!( 2, dict.get_item("b") .unwrap() .unwrap() .extract::() .unwrap() ); let map: HashMap<&str, i32> = [("a", 1), ("b", 2)].iter().cloned().collect(); assert_eq!(map, dict.extract().unwrap()); let map: BTreeMap<&str, i32> = [("a", 1), ("b", 2)].iter().cloned().collect(); assert_eq!(map, dict.extract().unwrap()); }); } #[test] #[cfg(not(PyPy))] fn test_from_sequence_err() { Python::with_gil(|py| { let items = PyList::new(py, &vec!["a", "b"]); assert!(PyDict::from_sequence(py, items.to_object(py)).is_err()); }); } #[test] fn test_copy() { Python::with_gil(|py| { let dict = [(7, 32)].into_py_dict(py); let ndict = dict.copy().unwrap(); assert_eq!( 32, ndict .get_item(7i32) .unwrap() .unwrap() .extract::() .unwrap() ); assert!(ndict.get_item(8i32).unwrap().is_none()); }); } #[test] fn test_len() { Python::with_gil(|py| { let mut v = HashMap::new(); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); assert_eq!(0, dict.len()); v.insert(7, 32); let ob = v.to_object(py); let dict2: &PyDict = ob.downcast(py).unwrap(); assert_eq!(1, dict2.len()); }); } #[test] fn test_contains() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); assert!(dict.contains(7i32).unwrap()); assert!(!dict.contains(8i32).unwrap()); }); } #[test] fn test_get_item() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); assert_eq!( 32, dict.get_item(7i32) .unwrap() .unwrap() .extract::() .unwrap() ); assert!(dict.get_item(8i32).unwrap().is_none()); }); } #[test] #[allow(deprecated)] #[cfg(not(PyPy))] fn test_get_item_with_error() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); assert_eq!( 32, dict.get_item_with_error(7i32) .unwrap() .unwrap() .extract::() .unwrap() ); assert!(dict.get_item_with_error(8i32).unwrap().is_none()); assert!(dict .get_item_with_error(dict) .unwrap_err() .is_instance_of::(py)); }); } #[test] fn test_set_item() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); assert!(dict.set_item(7i32, 42i32).is_ok()); // change assert!(dict.set_item(8i32, 123i32).is_ok()); // insert assert_eq!( 42i32, dict.get_item(7i32) .unwrap() .unwrap() .extract::() .unwrap() ); assert_eq!( 123i32, dict.get_item(8i32) .unwrap() .unwrap() .extract::() .unwrap() ); }); } #[test] fn test_set_item_refcnt() { Python::with_gil(|py| { let cnt; let obj = py.eval("object()", None, None).unwrap(); { let _pool = unsafe { crate::GILPool::new() }; cnt = obj.get_refcnt(); let _dict = [(10, obj)].into_py_dict(py); } { assert_eq!(cnt, obj.get_refcnt()); } }); } #[test] fn test_set_item_does_not_update_original_object() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); assert!(dict.set_item(7i32, 42i32).is_ok()); // change assert!(dict.set_item(8i32, 123i32).is_ok()); // insert assert_eq!(32i32, v[&7i32]); // not updated! assert_eq!(None, v.get(&8i32)); }); } #[test] fn test_del_item() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); assert!(dict.del_item(7i32).is_ok()); assert_eq!(0, dict.len()); assert!(dict.get_item(7i32).unwrap().is_none()); }); } #[test] fn test_del_item_does_not_update_original_object() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); assert!(dict.del_item(7i32).is_ok()); // change assert_eq!(32i32, *v.get(&7i32).unwrap()); // not updated! }); } #[test] fn test_items() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); v.insert(8, 42); v.insert(9, 123); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); // Can't just compare against a vector of tuples since we don't have a guaranteed ordering. let mut key_sum = 0; let mut value_sum = 0; for el in dict.items() { let tuple = el.downcast::().unwrap(); key_sum += tuple.get_item(0).unwrap().extract::().unwrap(); value_sum += tuple.get_item(1).unwrap().extract::().unwrap(); } assert_eq!(7 + 8 + 9, key_sum); assert_eq!(32 + 42 + 123, value_sum); }); } #[test] fn test_keys() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); v.insert(8, 42); v.insert(9, 123); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); // Can't just compare against a vector of tuples since we don't have a guaranteed ordering. let mut key_sum = 0; for el in dict.keys() { key_sum += el.extract::().unwrap(); } assert_eq!(7 + 8 + 9, key_sum); }); } #[test] fn test_values() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); v.insert(8, 42); v.insert(9, 123); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); // Can't just compare against a vector of tuples since we don't have a guaranteed ordering. let mut values_sum = 0; for el in dict.values() { values_sum += el.extract::().unwrap(); } assert_eq!(32 + 42 + 123, values_sum); }); } #[test] fn test_iter() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); v.insert(8, 42); v.insert(9, 123); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); let mut key_sum = 0; let mut value_sum = 0; for (key, value) in dict { key_sum += key.extract::().unwrap(); value_sum += value.extract::().unwrap(); } assert_eq!(7 + 8 + 9, key_sum); assert_eq!(32 + 42 + 123, value_sum); }); } #[test] fn test_iter_value_mutated() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); v.insert(8, 42); v.insert(9, 123); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); for (key, value) in dict { dict.set_item(key, value.extract::().unwrap() + 7) .unwrap(); } }); } #[test] #[should_panic] fn test_iter_key_mutated() { Python::with_gil(|py| { let mut v = HashMap::new(); for i in 0..10 { v.insert(i * 2, i * 2); } let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); for (i, (key, value)) in dict.iter().enumerate() { let key = key.extract::().unwrap(); let value = value.extract::().unwrap(); dict.set_item(key + 1, value + 1).unwrap(); if i > 1000 { // avoid this test just running out of memory if it fails break; }; } }); } #[test] #[should_panic] fn test_iter_key_mutated_constant_len() { Python::with_gil(|py| { let mut v = HashMap::new(); for i in 0..10 { v.insert(i * 2, i * 2); } let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); for (i, (key, value)) in dict.iter().enumerate() { let key = key.extract::().unwrap(); let value = value.extract::().unwrap(); dict.del_item(key).unwrap(); dict.set_item(key + 1, value + 1).unwrap(); if i > 1000 { // avoid this test just running out of memory if it fails break; }; } }); } #[test] fn test_iter_size_hint() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); v.insert(8, 42); v.insert(9, 123); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); let mut iter = dict.iter(); assert_eq!(iter.size_hint(), (v.len(), Some(v.len()))); iter.next(); assert_eq!(iter.size_hint(), (v.len() - 1, Some(v.len() - 1))); // Exhaust iterator. for _ in &mut iter {} assert_eq!(iter.size_hint(), (0, Some(0))); assert!(iter.next().is_none()); assert_eq!(iter.size_hint(), (0, Some(0))); }); } #[test] fn test_into_iter() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); v.insert(8, 42); v.insert(9, 123); let ob = v.to_object(py); let dict: &PyDict = ob.downcast(py).unwrap(); let mut key_sum = 0; let mut value_sum = 0; for (key, value) in dict { key_sum += key.extract::().unwrap(); value_sum += value.extract::().unwrap(); } assert_eq!(7 + 8 + 9, key_sum); assert_eq!(32 + 42 + 123, value_sum); }); } #[test] fn test_hashmap_into_dict() { Python::with_gil(|py| { let mut map = HashMap::::new(); map.insert(1, 1); let py_map = map.into_py_dict(py); assert_eq!(py_map.len(), 1); assert_eq!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap(), 1 ); }); } #[test] fn test_btreemap_into_dict() { Python::with_gil(|py| { let mut map = BTreeMap::::new(); map.insert(1, 1); let py_map = map.into_py_dict(py); assert_eq!(py_map.len(), 1); assert_eq!( py_map .get_item(1) .unwrap() .unwrap() .extract::() .unwrap(), 1 ); }); } #[test] fn test_vec_into_dict() { Python::with_gil(|py| { let vec = vec![("a", 1), ("b", 2), ("c", 3)]; let py_map = vec.into_py_dict(py); assert_eq!(py_map.len(), 3); assert_eq!( py_map .get_item("b") .unwrap() .unwrap() .extract::() .unwrap(), 2 ); }); } #[test] fn test_slice_into_dict() { Python::with_gil(|py| { let arr = [("a", 1), ("b", 2), ("c", 3)]; let py_map = arr.into_py_dict(py); assert_eq!(py_map.len(), 3); assert_eq!( py_map .get_item("b") .unwrap() .unwrap() .extract::() .unwrap(), 2 ); }); } #[test] fn dict_as_mapping() { Python::with_gil(|py| { let mut map = HashMap::::new(); map.insert(1, 1); let py_map = map.into_py_dict(py); assert_eq!(py_map.as_mapping().len().unwrap(), 1); assert_eq!( py_map .as_mapping() .get_item(1) .unwrap() .extract::() .unwrap(), 1 ); }); } #[cfg(not(PyPy))] fn abc_dict(py: Python<'_>) -> &PyDict { let mut map = HashMap::<&'static str, i32>::new(); map.insert("a", 1); map.insert("b", 2); map.insert("c", 3); map.into_py_dict(py) } #[test] #[cfg(not(PyPy))] fn dict_keys_view() { Python::with_gil(|py| { let dict = abc_dict(py); let keys = dict.call_method0("keys").unwrap(); assert!(keys.is_instance(py.get_type::()).unwrap()); }) } #[test] #[cfg(not(PyPy))] fn dict_values_view() { Python::with_gil(|py| { let dict = abc_dict(py); let values = dict.call_method0("values").unwrap(); assert!(values.is_instance(py.get_type::()).unwrap()); }) } #[test] #[cfg(not(PyPy))] fn dict_items_view() { Python::with_gil(|py| { let dict = abc_dict(py); let items = dict.call_method0("items").unwrap(); assert!(items.is_instance(py.get_type::()).unwrap()); }) } #[test] fn dict_update() { Python::with_gil(|py| { let dict = [("a", 1), ("b", 2), ("c", 3)].into_py_dict(py); let other = [("b", 4), ("c", 5), ("d", 6)].into_py_dict(py); dict.update(other.as_mapping()).unwrap(); assert_eq!(dict.len(), 4); assert_eq!( dict.get_item("a") .unwrap() .unwrap() .extract::() .unwrap(), 1 ); assert_eq!( dict.get_item("b") .unwrap() .unwrap() .extract::() .unwrap(), 4 ); assert_eq!( dict.get_item("c") .unwrap() .unwrap() .extract::() .unwrap(), 5 ); assert_eq!( dict.get_item("d") .unwrap() .unwrap() .extract::() .unwrap(), 6 ); assert_eq!(other.len(), 3); assert_eq!( other .get_item("b") .unwrap() .unwrap() .extract::() .unwrap(), 4 ); assert_eq!( other .get_item("c") .unwrap() .unwrap() .extract::() .unwrap(), 5 ); assert_eq!( other .get_item("d") .unwrap() .unwrap() .extract::() .unwrap(), 6 ); }) } #[test] fn dict_update_if_missing() { Python::with_gil(|py| { let dict = [("a", 1), ("b", 2), ("c", 3)].into_py_dict(py); let other = [("b", 4), ("c", 5), ("d", 6)].into_py_dict(py); dict.update_if_missing(other.as_mapping()).unwrap(); assert_eq!(dict.len(), 4); assert_eq!( dict.get_item("a") .unwrap() .unwrap() .extract::() .unwrap(), 1 ); assert_eq!( dict.get_item("b") .unwrap() .unwrap() .extract::() .unwrap(), 2 ); assert_eq!( dict.get_item("c") .unwrap() .unwrap() .extract::() .unwrap(), 3 ); assert_eq!( dict.get_item("d") .unwrap() .unwrap() .extract::() .unwrap(), 6 ); assert_eq!(other.len(), 3); assert_eq!( other .get_item("b") .unwrap() .unwrap() .extract::() .unwrap(), 4 ); assert_eq!( other .get_item("c") .unwrap() .unwrap() .extract::() .unwrap(), 5 ); assert_eq!( other .get_item("d") .unwrap() .unwrap() .extract::() .unwrap(), 6 ); }) } } pyo3-0.20.2/src/types/ellipsis.rs000064400000000000000000000031331046102023000147240ustar 00000000000000use crate::{ffi, PyAny, PyDowncastError, PyTryFrom, Python}; /// Represents the Python `Ellipsis` object. #[repr(transparent)] pub struct PyEllipsis(PyAny); pyobject_native_type_named!(PyEllipsis); pyobject_native_type_extract!(PyEllipsis); impl PyEllipsis { /// Returns the `Ellipsis` object. #[inline] pub fn get(py: Python<'_>) -> &PyEllipsis { unsafe { py.from_borrowed_ptr(ffi::Py_Ellipsis()) } } } impl<'v> PyTryFrom<'v> for PyEllipsis { fn try_from>(value: V) -> Result<&'v Self, crate::PyDowncastError<'v>> { let value: &PyAny = value.into(); if unsafe { ffi::Py_Ellipsis() == value.as_ptr() } { return unsafe { Ok(value.downcast_unchecked()) }; } Err(PyDowncastError::new(value, "ellipsis")) } fn try_from_exact>( value: V, ) -> Result<&'v Self, crate::PyDowncastError<'v>> { value.into().downcast() } unsafe fn try_from_unchecked>(value: V) -> &'v Self { let ptr = value.into() as *const _ as *const PyEllipsis; &*ptr } } #[cfg(test)] mod tests { use crate::types::{PyDict, PyEllipsis}; use crate::Python; #[test] fn test_ellipsis_is_itself() { Python::with_gil(|py| { assert!(PyEllipsis::get(py) .downcast::() .unwrap() .is_ellipsis()); }) } #[test] fn test_dict_is_not_ellipsis() { Python::with_gil(|py| { assert!(PyDict::new(py).downcast::().is_err()); }) } } pyo3-0.20.2/src/types/floatob.rs000064400000000000000000000100151046102023000145230ustar 00000000000000#[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::{ ffi, FromPyObject, IntoPy, PyAny, PyErr, PyNativeType, PyObject, PyResult, Python, ToPyObject, }; use std::os::raw::c_double; /// Represents a Python `float` object. /// /// You can usually avoid directly working with this type /// by using [`ToPyObject`] and [`extract`](PyAny::extract) /// with `f32`/`f64`. #[repr(transparent)] pub struct PyFloat(PyAny); pyobject_native_type!( PyFloat, ffi::PyFloatObject, pyobject_native_static_type_object!(ffi::PyFloat_Type), #checkfunction=ffi::PyFloat_Check ); impl PyFloat { /// Creates a new Python `float` object. pub fn new(py: Python<'_>, val: c_double) -> &PyFloat { unsafe { py.from_owned_ptr(ffi::PyFloat_FromDouble(val)) } } /// Gets the value of this float. pub fn value(&self) -> c_double { #[cfg(not(Py_LIMITED_API))] unsafe { // Safety: self is PyFloat object ffi::PyFloat_AS_DOUBLE(self.as_ptr()) } #[cfg(Py_LIMITED_API)] unsafe { ffi::PyFloat_AsDouble(self.as_ptr()) } } } impl ToPyObject for f64 { fn to_object(&self, py: Python<'_>) -> PyObject { PyFloat::new(py, *self).into() } } impl IntoPy for f64 { fn into_py(self, py: Python<'_>) -> PyObject { PyFloat::new(py, self).into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::builtin("float") } } impl<'source> FromPyObject<'source> for f64 { // PyFloat_AsDouble returns -1.0 upon failure #![allow(clippy::float_cmp)] fn extract(obj: &'source PyAny) -> PyResult { // On non-limited API, .value() uses PyFloat_AS_DOUBLE which // allows us to have an optimized fast path for the case when // we have exactly a `float` object (it's not worth going through // `isinstance` machinery for subclasses). #[cfg(not(Py_LIMITED_API))] if let Ok(float) = obj.downcast_exact::() { return Ok(float.value()); } let v = unsafe { ffi::PyFloat_AsDouble(obj.as_ptr()) }; if v == -1.0 { if let Some(err) = PyErr::take(obj.py()) { return Err(err); } } Ok(v) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { Self::type_output() } } impl ToPyObject for f32 { fn to_object(&self, py: Python<'_>) -> PyObject { PyFloat::new(py, f64::from(*self)).into() } } impl IntoPy for f32 { fn into_py(self, py: Python<'_>) -> PyObject { PyFloat::new(py, f64::from(self)).into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::builtin("float") } } impl<'source> FromPyObject<'source> for f32 { fn extract(obj: &'source PyAny) -> PyResult { Ok(obj.extract::()? as f32) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { Self::type_output() } } #[cfg(test)] mod tests { use crate::{types::PyFloat, Python, ToPyObject}; macro_rules! num_to_py_object_and_back ( ($func_name:ident, $t1:ty, $t2:ty) => ( #[test] fn $func_name() { use assert_approx_eq::assert_approx_eq; Python::with_gil(|py| { let val = 123 as $t1; let obj = val.to_object(py); assert_approx_eq!(obj.extract::<$t2>(py).unwrap(), val as $t2); }); } ) ); num_to_py_object_and_back!(to_from_f64, f64, f64); num_to_py_object_and_back!(to_from_f32, f32, f32); num_to_py_object_and_back!(int_to_float, i32, f64); #[test] fn test_float_value() { use assert_approx_eq::assert_approx_eq; Python::with_gil(|py| { let v = 1.23f64; let obj = PyFloat::new(py, 1.23); assert_approx_eq!(v, obj.value()); }); } } pyo3-0.20.2/src/types/frame.rs000064400000000000000000000004011046102023000141650ustar 00000000000000use crate::ffi; use crate::PyAny; /// Represents a Python frame. #[repr(transparent)] pub struct PyFrame(PyAny); pyobject_native_type_core!( PyFrame, pyobject_native_static_type_object!(ffi::PyFrame_Type), #checkfunction=ffi::PyFrame_Check ); pyo3-0.20.2/src/types/frozenset.rs000064400000000000000000000176331046102023000151310ustar 00000000000000#[cfg(Py_LIMITED_API)] use crate::types::PyIterator; use crate::{ err::{self, PyErr, PyResult}, Py, PyObject, }; use crate::{ffi, PyAny, Python, ToPyObject}; use std::ptr; /// Allows building a Python `frozenset` one item at a time pub struct PyFrozenSetBuilder<'py> { py_frozen_set: &'py PyFrozenSet, } impl<'py> PyFrozenSetBuilder<'py> { /// Create a new `FrozenSetBuilder`. /// Since this allocates a `PyFrozenSet` internally it may /// panic when running out of memory. pub fn new(py: Python<'py>) -> PyResult> { Ok(PyFrozenSetBuilder { py_frozen_set: PyFrozenSet::empty(py)?, }) } /// Adds an element to the set. pub fn add(&mut self, key: K) -> PyResult<()> where K: ToPyObject, { fn inner(frozenset: &PyFrozenSet, key: PyObject) -> PyResult<()> { err::error_on_minusone(frozenset.py(), unsafe { ffi::PySet_Add(frozenset.as_ptr(), key.as_ptr()) }) } inner(self.py_frozen_set, key.to_object(self.py_frozen_set.py())) } /// Finish building the set and take ownership of its current value pub fn finalize(self) -> &'py PyFrozenSet { self.py_frozen_set } } /// Represents a Python `frozenset` #[repr(transparent)] pub struct PyFrozenSet(PyAny); #[cfg(not(PyPy))] pyobject_native_type!( PyFrozenSet, ffi::PySetObject, pyobject_native_static_type_object!(ffi::PyFrozenSet_Type), #checkfunction=ffi::PyFrozenSet_Check ); #[cfg(PyPy)] pyobject_native_type_core!( PyFrozenSet, pyobject_native_static_type_object!(ffi::PyFrozenSet_Type), #checkfunction=ffi::PyFrozenSet_Check ); impl PyFrozenSet { /// Creates a new frozenset. /// /// May panic when running out of memory. #[inline] pub fn new<'a, 'p, T: ToPyObject + 'a>( py: Python<'p>, elements: impl IntoIterator, ) -> PyResult<&'p PyFrozenSet> { new_from_iter(py, elements).map(|set| set.into_ref(py)) } /// Creates a new empty frozen set pub fn empty(py: Python<'_>) -> PyResult<&PyFrozenSet> { unsafe { py.from_owned_ptr_or_err(ffi::PyFrozenSet_New(ptr::null_mut())) } } /// Return the number of items in the set. /// This is equivalent to len(p) on a set. #[inline] pub fn len(&self) -> usize { unsafe { ffi::PySet_Size(self.as_ptr()) as usize } } /// Check if set is empty. pub fn is_empty(&self) -> bool { self.len() == 0 } /// Determine if the set contains the specified key. /// This is equivalent to the Python expression `key in self`. pub fn contains(&self, key: K) -> PyResult where K: ToPyObject, { fn inner(frozenset: &PyFrozenSet, key: PyObject) -> PyResult { match unsafe { ffi::PySet_Contains(frozenset.as_ptr(), key.as_ptr()) } { 1 => Ok(true), 0 => Ok(false), _ => Err(PyErr::fetch(frozenset.py())), } } inner(self, key.to_object(self.py())) } /// Returns an iterator of values in this frozen set. pub fn iter(&self) -> PyFrozenSetIterator<'_> { IntoIterator::into_iter(self) } } #[cfg(Py_LIMITED_API)] mod impl_ { use super::*; impl<'a> std::iter::IntoIterator for &'a PyFrozenSet { type Item = &'a PyAny; type IntoIter = PyFrozenSetIterator<'a>; fn into_iter(self) -> Self::IntoIter { PyFrozenSetIterator { it: PyIterator::from_object(self).unwrap(), } } } /// PyO3 implementation of an iterator for a Python `frozenset` object. pub struct PyFrozenSetIterator<'p> { it: &'p PyIterator, } impl<'py> Iterator for PyFrozenSetIterator<'py> { type Item = &'py super::PyAny; #[inline] fn next(&mut self) -> Option { self.it.next().map(Result::unwrap) } } } #[cfg(not(Py_LIMITED_API))] mod impl_ { use super::*; impl<'a> std::iter::IntoIterator for &'a PyFrozenSet { type Item = &'a PyAny; type IntoIter = PyFrozenSetIterator<'a>; fn into_iter(self) -> Self::IntoIter { PyFrozenSetIterator { set: self, pos: 0 } } } /// PyO3 implementation of an iterator for a Python `frozenset` object. pub struct PyFrozenSetIterator<'py> { set: &'py PyFrozenSet, pos: ffi::Py_ssize_t, } impl<'py> Iterator for PyFrozenSetIterator<'py> { type Item = &'py PyAny; #[inline] fn next(&mut self) -> Option { unsafe { let mut key: *mut ffi::PyObject = std::ptr::null_mut(); let mut hash: ffi::Py_hash_t = 0; if ffi::_PySet_NextEntry(self.set.as_ptr(), &mut self.pos, &mut key, &mut hash) != 0 { // _PySet_NextEntry returns borrowed object; for safety must make owned (see #890) Some(self.set.py().from_owned_ptr(ffi::_Py_NewRef(key))) } else { None } } } #[inline] fn size_hint(&self) -> (usize, Option) { let len = self.len(); (len, Some(len)) } } impl<'py> ExactSizeIterator for PyFrozenSetIterator<'py> { fn len(&self) -> usize { self.set.len().saturating_sub(self.pos as usize) } } } pub use impl_::*; #[inline] pub(crate) fn new_from_iter( py: Python<'_>, elements: impl IntoIterator, ) -> PyResult> { fn inner( py: Python<'_>, elements: &mut dyn Iterator, ) -> PyResult> { let set: Py = unsafe { // We create the `Py` pointer because its Drop cleans up the set if user code panics. Py::from_owned_ptr_or_err(py, ffi::PyFrozenSet_New(std::ptr::null_mut()))? }; let ptr = set.as_ptr(); for obj in elements { err::error_on_minusone(py, unsafe { ffi::PySet_Add(ptr, obj.as_ptr()) })?; } Ok(set) } let mut iter = elements.into_iter().map(|e| e.to_object(py)); inner(py, &mut iter) } #[cfg(test)] mod tests { use super::*; #[test] fn test_frozenset_new_and_len() { Python::with_gil(|py| { let set = PyFrozenSet::new(py, &[1]).unwrap(); assert_eq!(1, set.len()); let v = vec![1]; assert!(PyFrozenSet::new(py, &[v]).is_err()); }); } #[test] fn test_frozenset_empty() { Python::with_gil(|py| { let set = PyFrozenSet::empty(py).unwrap(); assert_eq!(0, set.len()); }); } #[test] fn test_frozenset_contains() { Python::with_gil(|py| { let set = PyFrozenSet::new(py, &[1]).unwrap(); assert!(set.contains(1).unwrap()); }); } #[test] fn test_frozenset_iter() { Python::with_gil(|py| { let set = PyFrozenSet::new(py, &[1]).unwrap(); // iter method for el in set { assert_eq!(1i32, el.extract::().unwrap()); } // intoiterator iteration for el in set { assert_eq!(1i32, el.extract::().unwrap()); } }); } #[test] fn test_frozenset_builder() { use super::PyFrozenSetBuilder; Python::with_gil(|py| { let mut builder = PyFrozenSetBuilder::new(py).unwrap(); // add an item builder.add(1).unwrap(); builder.add(2).unwrap(); builder.add(2).unwrap(); // finalize it let set = builder.finalize(); assert!(set.contains(1).unwrap()); assert!(set.contains(2).unwrap()); assert!(!set.contains(3).unwrap()); }); } } pyo3-0.20.2/src/types/function.rs000064400000000000000000000136431046102023000147340ustar 00000000000000use crate::derive_utils::PyFunctionArguments; use crate::methods::PyMethodDefDestructor; use crate::prelude::*; use crate::{ ffi, impl_::pymethods::{self, PyMethodDef}, types::{PyCapsule, PyDict, PyTuple}, }; use std::cell::UnsafeCell; use std::ffi::CStr; /// Represents a builtin Python function object. #[repr(transparent)] pub struct PyCFunction(PyAny); pyobject_native_type_core!(PyCFunction, pyobject_native_static_type_object!(ffi::PyCFunction_Type), #checkfunction=ffi::PyCFunction_Check); impl PyCFunction { /// Create a new built-in function with keywords (*args and/or **kwargs). pub fn new_with_keywords<'a>( fun: ffi::PyCFunctionWithKeywords, name: &'static str, doc: &'static str, py_or_module: PyFunctionArguments<'a>, ) -> PyResult<&'a Self> { Self::internal_new( &PyMethodDef::cfunction_with_keywords( name, pymethods::PyCFunctionWithKeywords(fun), doc, ), py_or_module, ) } /// Create a new built-in function which takes no arguments. pub fn new<'a>( fun: ffi::PyCFunction, name: &'static str, doc: &'static str, py_or_module: PyFunctionArguments<'a>, ) -> PyResult<&'a Self> { Self::internal_new( &PyMethodDef::noargs(name, pymethods::PyCFunction(fun), doc), py_or_module, ) } /// Create a new function from a closure. /// /// # Examples /// /// ``` /// # use pyo3::prelude::*; /// # use pyo3::{py_run, types::{PyCFunction, PyDict, PyTuple}}; /// /// Python::with_gil(|py| { /// let add_one = |args: &PyTuple, _kwargs: Option<&PyDict>| -> PyResult<_> { /// let i = args.extract::<(i64,)>()?.0; /// Ok(i+1) /// }; /// let add_one = PyCFunction::new_closure(py, None, None, add_one).unwrap(); /// py_run!(py, add_one, "assert add_one(42) == 43"); /// }); /// ``` pub fn new_closure<'a, F, R>( py: Python<'a>, name: Option<&'static str>, doc: Option<&'static str>, closure: F, ) -> PyResult<&'a PyCFunction> where F: Fn(&PyTuple, Option<&PyDict>) -> R + Send + 'static, R: crate::callback::IntoPyCallbackOutput<*mut ffi::PyObject>, { let method_def = pymethods::PyMethodDef::cfunction_with_keywords( name.unwrap_or("pyo3-closure\0"), pymethods::PyCFunctionWithKeywords(run_closure::), doc.unwrap_or("\0"), ); let (def, def_destructor) = method_def.as_method_def()?; let capsule = PyCapsule::new( py, ClosureDestructor:: { closure, def: UnsafeCell::new(def), def_destructor, }, Some(closure_capsule_name().to_owned()), )?; // Safety: just created the capsule with type ClosureDestructor above let data = unsafe { capsule.reference::>() }; unsafe { py.from_owned_ptr_or_err::(ffi::PyCFunction_NewEx( data.def.get(), capsule.as_ptr(), std::ptr::null_mut(), )) } } #[doc(hidden)] pub fn internal_new<'py>( method_def: &PyMethodDef, py_or_module: PyFunctionArguments<'py>, ) -> PyResult<&'py Self> { let (py, module) = py_or_module.into_py_and_maybe_module(); let (mod_ptr, module_name) = if let Some(m) = module { let mod_ptr = m.as_ptr(); let name: Py = m.name()?.into_py(py); (mod_ptr, name.as_ptr()) } else { (std::ptr::null_mut(), std::ptr::null_mut()) }; let (def, destructor) = method_def.as_method_def()?; // FIXME: stop leaking the def and destructor let def = Box::into_raw(Box::new(def)); std::mem::forget(destructor); unsafe { py.from_owned_ptr_or_err::(ffi::PyCFunction_NewEx( def, mod_ptr, module_name, )) } } } fn closure_capsule_name() -> &'static CStr { // TODO replace this with const CStr once MSRV new enough CStr::from_bytes_with_nul(b"pyo3-closure\0").unwrap() } unsafe extern "C" fn run_closure( capsule_ptr: *mut ffi::PyObject, args: *mut ffi::PyObject, kwargs: *mut ffi::PyObject, ) -> *mut ffi::PyObject where F: Fn(&PyTuple, Option<&PyDict>) -> R + Send + 'static, R: crate::callback::IntoPyCallbackOutput<*mut ffi::PyObject>, { crate::impl_::trampoline::cfunction_with_keywords( capsule_ptr, args, kwargs, |py, capsule_ptr, args, kwargs| { let boxed_fn: &ClosureDestructor = &*(ffi::PyCapsule_GetPointer(capsule_ptr, closure_capsule_name().as_ptr()) as *mut ClosureDestructor); let args = py.from_borrowed_ptr::(args); let kwargs = py.from_borrowed_ptr_or_opt::(kwargs); crate::callback::convert(py, (boxed_fn.closure)(args, kwargs)) }, ) } struct ClosureDestructor { closure: F, // Wrapped in UnsafeCell because Python C-API wants a *mut pointer // to this member. def: UnsafeCell, // Used to destroy the cstrings in `def`, if necessary. #[allow(dead_code)] def_destructor: PyMethodDefDestructor, } // Safety: F is send and none of the fields are ever mutated unsafe impl Send for ClosureDestructor {} /// Represents a Python function object. #[repr(transparent)] #[cfg(all(not(Py_LIMITED_API), not(all(PyPy, not(Py_3_8)))))] pub struct PyFunction(PyAny); #[cfg(all(not(Py_LIMITED_API), not(all(PyPy, not(Py_3_8)))))] pyobject_native_type_core!(PyFunction, pyobject_native_static_type_object!(ffi::PyFunction_Type), #checkfunction=ffi::PyFunction_Check); pyo3-0.20.2/src/types/iterator.rs000064400000000000000000000240171046102023000147350ustar 00000000000000use crate::{ffi, AsPyPointer, Py, PyAny, PyErr, PyNativeType, PyResult, Python}; use crate::{PyDowncastError, PyTryFrom}; /// A Python iterator object. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let list = py.eval("iter([1, 2, 3, 4])", None, None)?; /// let numbers: PyResult> = list /// .iter()? /// .map(|i| i.and_then(PyAny::extract::)) /// .collect(); /// let sum: usize = numbers?.iter().sum(); /// assert_eq!(sum, 10); /// Ok(()) /// }) /// # } /// ``` #[repr(transparent)] pub struct PyIterator(PyAny); pyobject_native_type_named!(PyIterator); pyobject_native_type_extract!(PyIterator); impl PyIterator { /// Constructs a `PyIterator` from a Python iterable object. /// /// Equivalent to Python's built-in `iter` function. pub fn from_object(obj: &PyAny) -> PyResult<&PyIterator> { unsafe { obj.py() .from_owned_ptr_or_err(ffi::PyObject_GetIter(obj.as_ptr())) } } } impl<'p> Iterator for &'p PyIterator { type Item = PyResult<&'p PyAny>; /// Retrieves the next item from an iterator. /// /// Returns `None` when the iterator is exhausted. /// If an exception occurs, returns `Some(Err(..))`. /// Further `next()` calls after an exception occurs are likely /// to repeatedly result in the same exception. fn next(&mut self) -> Option { let py = self.0.py(); match unsafe { py.from_owned_ptr_or_opt(ffi::PyIter_Next(self.0.as_ptr())) } { Some(obj) => Some(Ok(obj)), None => PyErr::take(py).map(Err), } } #[cfg(not(Py_LIMITED_API))] fn size_hint(&self) -> (usize, Option) { let hint = unsafe { ffi::PyObject_LengthHint(self.0.as_ptr(), 0) }; (hint.max(0) as usize, None) } } // PyIter_Check does not exist in the limited API until 3.8 impl<'v> PyTryFrom<'v> for PyIterator { fn try_from>(value: V) -> Result<&'v PyIterator, PyDowncastError<'v>> { let value = value.into(); unsafe { if ffi::PyIter_Check(value.as_ptr()) != 0 { Ok(value.downcast_unchecked()) } else { Err(PyDowncastError::new(value, "Iterator")) } } } fn try_from_exact>(value: V) -> Result<&'v PyIterator, PyDowncastError<'v>> { value.into().downcast() } #[inline] unsafe fn try_from_unchecked>(value: V) -> &'v PyIterator { let ptr = value.into() as *const _ as *const PyIterator; &*ptr } } impl Py { /// Borrows a GIL-bound reference to the PyIterator. By binding to the GIL lifetime, this /// allows the GIL-bound reference to not require `Python` for any of its methods. pub fn as_ref<'py>(&'py self, _py: Python<'py>) -> &'py PyIterator { let any = self.as_ptr() as *const PyAny; unsafe { PyNativeType::unchecked_downcast(&*any) } } /// Similar to [`as_ref`](#method.as_ref), and also consumes this `Py` and registers the /// Python object reference in PyO3's object storage. The reference count for the Python /// object will not be decreased until the GIL lifetime ends. pub fn into_ref(self, py: Python<'_>) -> &PyIterator { unsafe { py.from_owned_ptr(self.into_ptr()) } } } #[cfg(test)] mod tests { use super::PyIterator; use crate::exceptions::PyTypeError; use crate::gil::GILPool; use crate::types::{PyDict, PyList}; use crate::{Py, PyAny, Python, ToPyObject}; #[test] fn vec_iter() { Python::with_gil(|py| { let obj = vec![10, 20].to_object(py); let inst = obj.as_ref(py); let mut it = inst.iter().unwrap(); assert_eq!( 10_i32, it.next().unwrap().unwrap().extract::<'_, i32>().unwrap() ); assert_eq!( 20_i32, it.next().unwrap().unwrap().extract::<'_, i32>().unwrap() ); assert!(it.next().is_none()); }); } #[test] fn iter_refcnt() { let (obj, count) = Python::with_gil(|py| { let obj = vec![10, 20].to_object(py); let count = obj.get_refcnt(py); (obj, count) }); Python::with_gil(|py| { let inst = obj.as_ref(py); let mut it = inst.iter().unwrap(); assert_eq!( 10_i32, it.next().unwrap().unwrap().extract::<'_, i32>().unwrap() ); }); Python::with_gil(|py| { assert_eq!(count, obj.get_refcnt(py)); }); } #[test] fn iter_item_refcnt() { Python::with_gil(|py| { let count; let obj = py.eval("object()", None, None).unwrap(); let list = { let _pool = unsafe { GILPool::new() }; let list = PyList::empty(py); list.append(10).unwrap(); list.append(obj).unwrap(); count = obj.get_refcnt(); list.to_object(py) }; { let _pool = unsafe { GILPool::new() }; let inst = list.as_ref(py); let mut it = inst.iter().unwrap(); assert_eq!( 10_i32, it.next().unwrap().unwrap().extract::<'_, i32>().unwrap() ); assert!(it.next().unwrap().unwrap().is(obj)); assert!(it.next().is_none()); } assert_eq!(count, obj.get_refcnt()); }); } #[test] fn fibonacci_generator() { let fibonacci_generator = r#" def fibonacci(target): a = 1 b = 1 for _ in range(target): yield a a, b = b, a + b "#; Python::with_gil(|py| { let context = PyDict::new(py); py.run(fibonacci_generator, None, Some(context)).unwrap(); let generator = py.eval("fibonacci(5)", None, Some(context)).unwrap(); for (actual, expected) in generator.iter().unwrap().zip(&[1, 1, 2, 3, 5]) { let actual = actual.unwrap().extract::().unwrap(); assert_eq!(actual, *expected) } }); } #[test] fn int_not_iterable() { Python::with_gil(|py| { let x = 5.to_object(py); let err = PyIterator::from_object(x.as_ref(py)).unwrap_err(); assert!(err.is_instance_of::(py)); }); } #[test] fn iterator_try_from() { Python::with_gil(|py| { let obj: Py = vec![10, 20].to_object(py).as_ref(py).iter().unwrap().into(); let iter: &PyIterator = obj.downcast(py).unwrap(); assert!(obj.is(iter)); }); } #[test] fn test_as_ref() { Python::with_gil(|py| { let iter: Py = PyAny::iter(PyList::empty(py)).unwrap().into(); let mut iter_ref: &PyIterator = iter.as_ref(py); assert!(iter_ref.next().is_none()); }) } #[test] fn test_into_ref() { Python::with_gil(|py| { let bare_iter = PyAny::iter(PyList::empty(py)).unwrap(); assert_eq!(bare_iter.get_refcnt(), 1); let iter: Py = bare_iter.into(); assert_eq!(bare_iter.get_refcnt(), 2); let mut iter_ref = iter.into_ref(py); assert!(iter_ref.next().is_none()); assert_eq!(iter_ref.get_refcnt(), 2); }) } #[test] #[cfg(feature = "macros")] fn python_class_not_iterator() { use crate::PyErr; #[crate::pyclass(crate = "crate")] struct Downcaster { failed: Option, } #[crate::pymethods(crate = "crate")] impl Downcaster { fn downcast_iterator(&mut self, obj: &PyAny) { self.failed = Some(obj.downcast::().unwrap_err().into()); } } // Regression test for 2913 Python::with_gil(|py| { let downcaster = Py::new(py, Downcaster { failed: None }).unwrap(); crate::py_run!( py, downcaster, r#" from collections.abc import Sequence class MySequence(Sequence): def __init__(self): self._data = [1, 2, 3] def __getitem__(self, index): return self._data[index] def __len__(self): return len(self._data) downcaster.downcast_iterator(MySequence()) "# ); assert_eq!( downcaster.borrow_mut(py).failed.take().unwrap().to_string(), "TypeError: 'MySequence' object cannot be converted to 'Iterator'" ); }); } #[test] #[cfg(feature = "macros")] fn python_class_iterator() { #[crate::pyfunction(crate = "crate")] fn assert_iterator(obj: &PyAny) { assert!(obj.downcast::().is_ok()) } // Regression test for 2913 Python::with_gil(|py| { let assert_iterator = crate::wrap_pyfunction!(assert_iterator, py).unwrap(); crate::py_run!( py, assert_iterator, r#" class MyIter: def __next__(self): raise StopIteration assert_iterator(MyIter()) "# ); }); } #[test] #[cfg(not(Py_LIMITED_API))] fn length_hint_becomes_size_hint_lower_bound() { Python::with_gil(|py| { let list = py.eval("[1, 2, 3]", None, None).unwrap(); let iter = list.iter().unwrap(); let hint = iter.size_hint(); assert_eq!(hint, (3, None)); }); } } pyo3-0.20.2/src/types/list.rs000064400000000000000000000723401046102023000140610ustar 00000000000000use std::convert::TryInto; use std::iter::FusedIterator; use crate::err::{self, PyResult}; use crate::ffi::{self, Py_ssize_t}; use crate::internal_tricks::get_ssize_index; use crate::types::{PySequence, PyTuple}; use crate::{Py, PyAny, PyObject, Python, ToPyObject}; /// Represents a Python `list`. #[repr(transparent)] pub struct PyList(PyAny); pyobject_native_type_core!(PyList, pyobject_native_static_type_object!(ffi::PyList_Type), #checkfunction=ffi::PyList_Check); #[inline] #[track_caller] pub(crate) fn new_from_iter( py: Python<'_>, elements: &mut dyn ExactSizeIterator, ) -> Py { unsafe { // PyList_New checks for overflow but has a bad error message, so we check ourselves let len: Py_ssize_t = elements .len() .try_into() .expect("out of range integral type conversion attempted on `elements.len()`"); let ptr = ffi::PyList_New(len); // We create the `Py` pointer here for two reasons: // - panics if the ptr is null // - its Drop cleans up the list if user code or the asserts panic. let list: Py = Py::from_owned_ptr(py, ptr); let mut counter: Py_ssize_t = 0; for obj in elements.take(len as usize) { #[cfg(not(Py_LIMITED_API))] ffi::PyList_SET_ITEM(ptr, counter, obj.into_ptr()); #[cfg(Py_LIMITED_API)] ffi::PyList_SetItem(ptr, counter, obj.into_ptr()); counter += 1; } assert!(elements.next().is_none(), "Attempted to create PyList but `elements` was larger than reported by its `ExactSizeIterator` implementation."); assert_eq!(len, counter, "Attempted to create PyList but `elements` was smaller than reported by its `ExactSizeIterator` implementation."); list } } impl PyList { /// Constructs a new list with the given elements. /// /// If you want to create a [`PyList`] with elements of different or unknown types, or from an /// iterable that doesn't implement [`ExactSizeIterator`], use [`PyList::append`]. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyList; /// /// # fn main() { /// Python::with_gil(|py| { /// let elements: Vec = vec![0, 1, 2, 3, 4, 5]; /// let list: &PyList = PyList::new(py, elements); /// assert_eq!(format!("{:?}", list), "[0, 1, 2, 3, 4, 5]"); /// }); /// # } /// ``` /// /// # Panics /// /// This function will panic if `element`'s [`ExactSizeIterator`] implementation is incorrect. /// All standard library structures implement this trait correctly, if they do, so calling this /// function with (for example) [`Vec`]`` or `&[T]` will always succeed. #[track_caller] pub fn new(py: Python<'_>, elements: impl IntoIterator) -> &PyList where T: ToPyObject, U: ExactSizeIterator, { let mut iter = elements.into_iter().map(|e| e.to_object(py)); let list = new_from_iter(py, &mut iter); list.into_ref(py) } /// Constructs a new empty list. pub fn empty(py: Python<'_>) -> &PyList { unsafe { py.from_owned_ptr::(ffi::PyList_New(0)) } } /// Returns the length of the list. pub fn len(&self) -> usize { unsafe { #[cfg(not(Py_LIMITED_API))] let size = ffi::PyList_GET_SIZE(self.as_ptr()); #[cfg(Py_LIMITED_API)] let size = ffi::PyList_Size(self.as_ptr()); // non-negative Py_ssize_t should always fit into Rust usize size as usize } } /// Checks if the list is empty. pub fn is_empty(&self) -> bool { self.len() == 0 } /// Returns `self` cast as a `PySequence`. pub fn as_sequence(&self) -> &PySequence { unsafe { self.downcast_unchecked() } } /// Gets the list item at the specified index. /// # Example /// ``` /// use pyo3::{prelude::*, types::PyList}; /// Python::with_gil(|py| { /// let list = PyList::new(py, [2, 3, 5, 7]); /// let obj = list.get_item(0); /// assert_eq!(obj.unwrap().extract::().unwrap(), 2); /// }); /// ``` pub fn get_item(&self, index: usize) -> PyResult<&PyAny> { unsafe { let item = ffi::PyList_GetItem(self.as_ptr(), index as Py_ssize_t); // PyList_GetItem return borrowed ptr; must make owned for safety (see #890). ffi::Py_XINCREF(item); self.py().from_owned_ptr_or_err(item) } } /// Gets the list item at the specified index. Undefined behavior on bad index. Use with caution. /// /// # Safety /// /// Caller must verify that the index is within the bounds of the list. #[cfg(not(Py_LIMITED_API))] pub unsafe fn get_item_unchecked(&self, index: usize) -> &PyAny { let item = ffi::PyList_GET_ITEM(self.as_ptr(), index as Py_ssize_t); // PyList_GET_ITEM return borrowed ptr; must make owned for safety (see #890). ffi::Py_XINCREF(item); self.py().from_owned_ptr(item) } /// Takes the slice `self[low:high]` and returns it as a new list. /// /// Indices must be nonnegative, and out-of-range indices are clipped to /// `self.len()`. pub fn get_slice(&self, low: usize, high: usize) -> &PyList { unsafe { self.py().from_owned_ptr(ffi::PyList_GetSlice( self.as_ptr(), get_ssize_index(low), get_ssize_index(high), )) } } /// Sets the item at the specified index. /// /// Raises `IndexError` if the index is out of range. pub fn set_item(&self, index: usize, item: I) -> PyResult<()> where I: ToPyObject, { fn inner(list: &PyList, index: usize, item: PyObject) -> PyResult<()> { err::error_on_minusone(list.py(), unsafe { ffi::PyList_SetItem(list.as_ptr(), get_ssize_index(index), item.into_ptr()) }) } inner(self, index, item.to_object(self.py())) } /// Deletes the `index`th element of self. /// /// This is equivalent to the Python statement `del self[i]`. #[inline] pub fn del_item(&self, index: usize) -> PyResult<()> { self.as_sequence().del_item(index) } /// Assigns the sequence `seq` to the slice of `self` from `low` to `high`. /// /// This is equivalent to the Python statement `self[low:high] = v`. #[inline] pub fn set_slice(&self, low: usize, high: usize, seq: &PyAny) -> PyResult<()> { err::error_on_minusone(self.py(), unsafe { ffi::PyList_SetSlice( self.as_ptr(), get_ssize_index(low), get_ssize_index(high), seq.as_ptr(), ) }) } /// Deletes the slice from `low` to `high` from `self`. /// /// This is equivalent to the Python statement `del self[low:high]`. #[inline] pub fn del_slice(&self, low: usize, high: usize) -> PyResult<()> { self.as_sequence().del_slice(low, high) } /// Appends an item to the list. pub fn append(&self, item: I) -> PyResult<()> where I: ToPyObject, { fn inner(list: &PyList, item: PyObject) -> PyResult<()> { err::error_on_minusone(list.py(), unsafe { ffi::PyList_Append(list.as_ptr(), item.as_ptr()) }) } inner(self, item.to_object(self.py())) } /// Inserts an item at the specified index. /// /// If `index >= self.len()`, inserts at the end. pub fn insert(&self, index: usize, item: I) -> PyResult<()> where I: ToPyObject, { fn inner(list: &PyList, index: usize, item: PyObject) -> PyResult<()> { err::error_on_minusone(list.py(), unsafe { ffi::PyList_Insert(list.as_ptr(), get_ssize_index(index), item.as_ptr()) }) } inner(self, index, item.to_object(self.py())) } /// Determines if self contains `value`. /// /// This is equivalent to the Python expression `value in self`. #[inline] pub fn contains(&self, value: V) -> PyResult where V: ToPyObject, { self.as_sequence().contains(value) } /// Returns the first index `i` for which `self[i] == value`. /// /// This is equivalent to the Python expression `self.index(value)`. #[inline] pub fn index(&self, value: V) -> PyResult where V: ToPyObject, { self.as_sequence().index(value) } /// Returns an iterator over this list's items. pub fn iter(&self) -> PyListIterator<'_> { PyListIterator { list: self, index: 0, length: self.len(), } } /// Sorts the list in-place. Equivalent to the Python expression `l.sort()`. pub fn sort(&self) -> PyResult<()> { err::error_on_minusone(self.py(), unsafe { ffi::PyList_Sort(self.as_ptr()) }) } /// Reverses the list in-place. Equivalent to the Python expression `l.reverse()`. pub fn reverse(&self) -> PyResult<()> { err::error_on_minusone(self.py(), unsafe { ffi::PyList_Reverse(self.as_ptr()) }) } /// Return a new tuple containing the contents of the list; equivalent to the Python expression `tuple(list)`. /// /// This method is equivalent to `self.as_sequence().to_tuple()` and faster than `PyTuple::new(py, this_list)`. pub fn to_tuple(&self) -> &PyTuple { unsafe { self.py().from_owned_ptr(ffi::PyList_AsTuple(self.as_ptr())) } } } index_impls!(PyList, "list", PyList::len, PyList::get_slice); /// Used by `PyList::iter()`. pub struct PyListIterator<'a> { list: &'a PyList, index: usize, length: usize, } impl<'a> PyListIterator<'a> { unsafe fn get_item(&self, index: usize) -> &'a PyAny { #[cfg(any(Py_LIMITED_API, PyPy))] let item = self.list.get_item(index).expect("list.get failed"); #[cfg(not(any(Py_LIMITED_API, PyPy)))] let item = self.list.get_item_unchecked(index); item } } impl<'a> Iterator for PyListIterator<'a> { type Item = &'a PyAny; #[inline] fn next(&mut self) -> Option { let length = self.length.min(self.list.len()); if self.index < length { let item = unsafe { self.get_item(self.index) }; self.index += 1; Some(item) } else { None } } #[inline] fn size_hint(&self) -> (usize, Option) { let len = self.len(); (len, Some(len)) } } impl<'a> DoubleEndedIterator for PyListIterator<'a> { #[inline] fn next_back(&mut self) -> Option { let length = self.length.min(self.list.len()); if self.index < length { let item = unsafe { self.get_item(length - 1) }; self.length = length - 1; Some(item) } else { None } } } impl<'a> ExactSizeIterator for PyListIterator<'a> { fn len(&self) -> usize { self.length.saturating_sub(self.index) } } impl FusedIterator for PyListIterator<'_> {} impl<'a> IntoIterator for &'a PyList { type Item = &'a PyAny; type IntoIter = PyListIterator<'a>; fn into_iter(self) -> Self::IntoIter { self.iter() } } #[cfg(test)] mod tests { use crate::types::{PyList, PyTuple}; use crate::Python; use crate::{IntoPy, PyObject, ToPyObject}; #[test] fn test_new() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5, 7]); assert_eq!(2, list[0].extract::().unwrap()); assert_eq!(3, list[1].extract::().unwrap()); assert_eq!(5, list[2].extract::().unwrap()); assert_eq!(7, list[3].extract::().unwrap()); }); } #[test] fn test_len() { Python::with_gil(|py| { let list = PyList::new(py, [1, 2, 3, 4]); assert_eq!(4, list.len()); }); } #[test] fn test_get_item() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5, 7]); assert_eq!(2, list.get_item(0).unwrap().extract::().unwrap()); assert_eq!(3, list.get_item(1).unwrap().extract::().unwrap()); assert_eq!(5, list.get_item(2).unwrap().extract::().unwrap()); assert_eq!(7, list.get_item(3).unwrap().extract::().unwrap()); }); } #[test] fn test_get_slice() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5, 7]); let slice = list.get_slice(1, 3); assert_eq!(2, slice.len()); let slice = list.get_slice(1, 7); assert_eq!(3, slice.len()); }); } #[test] fn test_set_item() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5, 7]); let val = 42i32.to_object(py); let val2 = 42i32.to_object(py); assert_eq!(2, list[0].extract::().unwrap()); list.set_item(0, val).unwrap(); assert_eq!(42, list[0].extract::().unwrap()); assert!(list.set_item(10, val2).is_err()); }); } #[test] fn test_set_item_refcnt() { Python::with_gil(|py| { let obj = py.eval("object()", None, None).unwrap(); let cnt; { let _pool = unsafe { crate::GILPool::new() }; let v = vec![2]; let ob = v.to_object(py); let list: &PyList = ob.downcast(py).unwrap(); cnt = obj.get_refcnt(); list.set_item(0, obj).unwrap(); } assert_eq!(cnt, obj.get_refcnt()); }); } #[test] fn test_insert() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5, 7]); let val = 42i32.to_object(py); let val2 = 43i32.to_object(py); assert_eq!(4, list.len()); assert_eq!(2, list[0].extract::().unwrap()); list.insert(0, val).unwrap(); list.insert(1000, val2).unwrap(); assert_eq!(6, list.len()); assert_eq!(42, list[0].extract::().unwrap()); assert_eq!(2, list[1].extract::().unwrap()); assert_eq!(43, list[5].extract::().unwrap()); }); } #[test] fn test_insert_refcnt() { Python::with_gil(|py| { let cnt; let obj = py.eval("object()", None, None).unwrap(); { let _pool = unsafe { crate::GILPool::new() }; let list = PyList::empty(py); cnt = obj.get_refcnt(); list.insert(0, obj).unwrap(); } assert_eq!(cnt, obj.get_refcnt()); }); } #[test] fn test_append() { Python::with_gil(|py| { let list = PyList::new(py, [2]); list.append(3).unwrap(); assert_eq!(2, list[0].extract::().unwrap()); assert_eq!(3, list[1].extract::().unwrap()); }); } #[test] fn test_append_refcnt() { Python::with_gil(|py| { let cnt; let obj = py.eval("object()", None, None).unwrap(); { let _pool = unsafe { crate::GILPool::new() }; let list = PyList::empty(py); cnt = obj.get_refcnt(); list.append(obj).unwrap(); } assert_eq!(cnt, obj.get_refcnt()); }); } #[test] fn test_iter() { Python::with_gil(|py| { let v = vec![2, 3, 5, 7]; let list = PyList::new(py, &v); let mut idx = 0; for el in list { assert_eq!(v[idx], el.extract::().unwrap()); idx += 1; } assert_eq!(idx, v.len()); }); } #[test] fn test_iter_size_hint() { Python::with_gil(|py| { let v = vec![2, 3, 5, 7]; let ob = v.to_object(py); let list: &PyList = ob.downcast(py).unwrap(); let mut iter = list.iter(); assert_eq!(iter.size_hint(), (v.len(), Some(v.len()))); iter.next(); assert_eq!(iter.size_hint(), (v.len() - 1, Some(v.len() - 1))); // Exhaust iterator. for _ in &mut iter {} assert_eq!(iter.size_hint(), (0, Some(0))); }); } #[test] fn test_iter_rev() { Python::with_gil(|py| { let v = vec![2, 3, 5, 7]; let ob = v.to_object(py); let list: &PyList = ob.downcast(py).unwrap(); let mut iter = list.iter().rev(); assert_eq!(iter.size_hint(), (4, Some(4))); assert_eq!(iter.next().unwrap().extract::().unwrap(), 7); assert_eq!(iter.size_hint(), (3, Some(3))); assert_eq!(iter.next().unwrap().extract::().unwrap(), 5); assert_eq!(iter.size_hint(), (2, Some(2))); assert_eq!(iter.next().unwrap().extract::().unwrap(), 3); assert_eq!(iter.size_hint(), (1, Some(1))); assert_eq!(iter.next().unwrap().extract::().unwrap(), 2); assert_eq!(iter.size_hint(), (0, Some(0))); assert!(iter.next().is_none()); assert!(iter.next().is_none()); }); } #[test] fn test_into_iter() { Python::with_gil(|py| { let list = PyList::new(py, [1, 2, 3, 4]); for (i, item) in list.iter().enumerate() { assert_eq!((i + 1) as i32, item.extract::().unwrap()); } }); } #[test] fn test_extract() { Python::with_gil(|py| { let v = vec![2, 3, 5, 7]; let list = PyList::new(py, &v); let v2 = list.as_ref().extract::>().unwrap(); assert_eq!(v, v2); }); } #[test] fn test_sort() { Python::with_gil(|py| { let v = vec![7, 3, 2, 5]; let list = PyList::new(py, &v); assert_eq!(7, list[0].extract::().unwrap()); assert_eq!(3, list[1].extract::().unwrap()); assert_eq!(2, list[2].extract::().unwrap()); assert_eq!(5, list[3].extract::().unwrap()); list.sort().unwrap(); assert_eq!(2, list[0].extract::().unwrap()); assert_eq!(3, list[1].extract::().unwrap()); assert_eq!(5, list[2].extract::().unwrap()); assert_eq!(7, list[3].extract::().unwrap()); }); } #[test] fn test_reverse() { Python::with_gil(|py| { let v = vec![2, 3, 5, 7]; let list = PyList::new(py, &v); assert_eq!(2, list[0].extract::().unwrap()); assert_eq!(3, list[1].extract::().unwrap()); assert_eq!(5, list[2].extract::().unwrap()); assert_eq!(7, list[3].extract::().unwrap()); list.reverse().unwrap(); assert_eq!(7, list[0].extract::().unwrap()); assert_eq!(5, list[1].extract::().unwrap()); assert_eq!(3, list[2].extract::().unwrap()); assert_eq!(2, list[3].extract::().unwrap()); }); } #[test] fn test_array_into_py() { Python::with_gil(|py| { let array: PyObject = [1, 2].into_py(py); let list: &PyList = array.downcast(py).unwrap(); assert_eq!(1, list[0].extract::().unwrap()); assert_eq!(2, list[1].extract::().unwrap()); }); } #[test] fn test_list_get_item_invalid_index() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5, 7]); let obj = list.get_item(5); assert!(obj.is_err()); assert_eq!( obj.unwrap_err().to_string(), "IndexError: list index out of range" ); }); } #[test] fn test_list_get_item_sanity() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5, 7]); let obj = list.get_item(0); assert_eq!(obj.unwrap().extract::().unwrap(), 2); }); } #[cfg(not(any(Py_LIMITED_API, PyPy)))] #[test] fn test_list_get_item_unchecked_sanity() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5, 7]); let obj = unsafe { list.get_item_unchecked(0) }; assert_eq!(obj.extract::().unwrap(), 2); }); } #[test] fn test_list_index_trait() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5]); assert_eq!(2, list[0].extract::().unwrap()); assert_eq!(3, list[1].extract::().unwrap()); assert_eq!(5, list[2].extract::().unwrap()); }); } #[test] #[should_panic] fn test_list_index_trait_panic() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5]); let _ = &list[7]; }); } #[test] fn test_list_index_trait_ranges() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5]); assert_eq!(vec![3, 5], list[1..3].extract::>().unwrap()); assert_eq!(Vec::::new(), list[3..3].extract::>().unwrap()); assert_eq!(vec![3, 5], list[1..].extract::>().unwrap()); assert_eq!(Vec::::new(), list[3..].extract::>().unwrap()); assert_eq!(vec![2, 3, 5], list[..].extract::>().unwrap()); assert_eq!(vec![3, 5], list[1..=2].extract::>().unwrap()); assert_eq!(vec![2, 3], list[..2].extract::>().unwrap()); assert_eq!(vec![2, 3], list[..=1].extract::>().unwrap()); }) } #[test] #[should_panic = "range start index 5 out of range for list of length 3"] fn test_list_index_trait_range_panic_start() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5]); list[5..10].extract::>().unwrap(); }) } #[test] #[should_panic = "range end index 10 out of range for list of length 3"] fn test_list_index_trait_range_panic_end() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5]); list[1..10].extract::>().unwrap(); }) } #[test] #[should_panic = "slice index starts at 2 but ends at 1"] fn test_list_index_trait_range_panic_wrong_order() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5]); #[allow(clippy::reversed_empty_ranges)] list[2..1].extract::>().unwrap(); }) } #[test] #[should_panic = "range start index 8 out of range for list of length 3"] fn test_list_index_trait_range_from_panic() { Python::with_gil(|py| { let list = PyList::new(py, [2, 3, 5]); list[8..].extract::>().unwrap(); }) } #[test] fn test_list_del_item() { Python::with_gil(|py| { let list = PyList::new(py, [1, 1, 2, 3, 5, 8]); assert!(list.del_item(10).is_err()); assert_eq!(1, list[0].extract::().unwrap()); assert!(list.del_item(0).is_ok()); assert_eq!(1, list[0].extract::().unwrap()); assert!(list.del_item(0).is_ok()); assert_eq!(2, list[0].extract::().unwrap()); assert!(list.del_item(0).is_ok()); assert_eq!(3, list[0].extract::().unwrap()); assert!(list.del_item(0).is_ok()); assert_eq!(5, list[0].extract::().unwrap()); assert!(list.del_item(0).is_ok()); assert_eq!(8, list[0].extract::().unwrap()); assert!(list.del_item(0).is_ok()); assert_eq!(0, list.len()); assert!(list.del_item(0).is_err()); }); } #[test] fn test_list_set_slice() { Python::with_gil(|py| { let list = PyList::new(py, [1, 1, 2, 3, 5, 8]); let ins = PyList::new(py, [7, 4]); list.set_slice(1, 4, ins).unwrap(); assert_eq!([1, 7, 4, 5, 8], list.extract::<[i32; 5]>().unwrap()); list.set_slice(3, 100, PyList::empty(py)).unwrap(); assert_eq!([1, 7, 4], list.extract::<[i32; 3]>().unwrap()); }); } #[test] fn test_list_del_slice() { Python::with_gil(|py| { let list = PyList::new(py, [1, 1, 2, 3, 5, 8]); list.del_slice(1, 4).unwrap(); assert_eq!([1, 5, 8], list.extract::<[i32; 3]>().unwrap()); list.del_slice(1, 100).unwrap(); assert_eq!([1], list.extract::<[i32; 1]>().unwrap()); }); } #[test] fn test_list_contains() { Python::with_gil(|py| { let list = PyList::new(py, [1, 1, 2, 3, 5, 8]); assert_eq!(6, list.len()); let bad_needle = 7i32.to_object(py); assert!(!list.contains(&bad_needle).unwrap()); let good_needle = 8i32.to_object(py); assert!(list.contains(&good_needle).unwrap()); let type_coerced_needle = 8f32.to_object(py); assert!(list.contains(&type_coerced_needle).unwrap()); }); } #[test] fn test_list_index() { Python::with_gil(|py| { let list = PyList::new(py, [1, 1, 2, 3, 5, 8]); assert_eq!(0, list.index(1i32).unwrap()); assert_eq!(2, list.index(2i32).unwrap()); assert_eq!(3, list.index(3i32).unwrap()); assert_eq!(4, list.index(5i32).unwrap()); assert_eq!(5, list.index(8i32).unwrap()); assert!(list.index(42i32).is_err()); }); } use std::ops::Range; // An iterator that lies about its `ExactSizeIterator` implementation. // See https://github.com/PyO3/pyo3/issues/2118 struct FaultyIter(Range, usize); impl Iterator for FaultyIter { type Item = usize; fn next(&mut self) -> Option { self.0.next() } } impl ExactSizeIterator for FaultyIter { fn len(&self) -> usize { self.1 } } #[test] #[should_panic( expected = "Attempted to create PyList but `elements` was larger than reported by its `ExactSizeIterator` implementation." )] fn too_long_iterator() { Python::with_gil(|py| { let iter = FaultyIter(0..usize::MAX, 73); let _list = PyList::new(py, iter); }) } #[test] #[should_panic( expected = "Attempted to create PyList but `elements` was smaller than reported by its `ExactSizeIterator` implementation." )] fn too_short_iterator() { Python::with_gil(|py| { let iter = FaultyIter(0..35, 73); let _list = PyList::new(py, iter); }) } #[test] #[should_panic( expected = "out of range integral type conversion attempted on `elements.len()`" )] fn overflowing_size() { Python::with_gil(|py| { let iter = FaultyIter(0..0, usize::MAX); let _list = PyList::new(py, iter); }) } #[cfg(feature = "macros")] #[test] fn bad_clone_mem_leaks() { use crate::{Py, PyAny}; use std::sync::atomic::{AtomicUsize, Ordering::SeqCst}; static NEEDS_DESTRUCTING_COUNT: AtomicUsize = AtomicUsize::new(0); #[crate::pyclass] #[pyo3(crate = "crate")] struct Bad(usize); impl Clone for Bad { fn clone(&self) -> Self { // This panic should not lead to a memory leak assert_ne!(self.0, 42); NEEDS_DESTRUCTING_COUNT.fetch_add(1, SeqCst); Bad(self.0) } } impl Drop for Bad { fn drop(&mut self) { NEEDS_DESTRUCTING_COUNT.fetch_sub(1, SeqCst); } } impl ToPyObject for Bad { fn to_object(&self, py: Python<'_>) -> Py { self.to_owned().into_py(py) } } struct FaultyIter(Range, usize); impl Iterator for FaultyIter { type Item = Bad; fn next(&mut self) -> Option { self.0.next().map(|i| { NEEDS_DESTRUCTING_COUNT.fetch_add(1, SeqCst); Bad(i) }) } } impl ExactSizeIterator for FaultyIter { fn len(&self) -> usize { self.1 } } Python::with_gil(|py| { std::panic::catch_unwind(|| { let iter = FaultyIter(0..50, 50); let _list = PyList::new(py, iter); }) .unwrap_err(); }); assert_eq!( NEEDS_DESTRUCTING_COUNT.load(SeqCst), 0, "Some destructors did not run" ); } #[test] fn test_list_to_tuple() { Python::with_gil(|py| { let list = PyList::new(py, vec![1, 2, 3]); let tuple = list.to_tuple(); let tuple_expected = PyTuple::new(py, vec![1, 2, 3]); assert!(tuple.eq(tuple_expected).unwrap()); }) } } pyo3-0.20.2/src/types/mapping.rs000064400000000000000000000266251046102023000145460ustar 00000000000000use crate::err::{PyDowncastError, PyResult}; use crate::sync::GILOnceCell; use crate::type_object::PyTypeInfo; use crate::types::{PyAny, PyDict, PySequence, PyType}; use crate::{ffi, Py, PyNativeType, PyTryFrom, Python, ToPyObject}; /// Represents a reference to a Python object supporting the mapping protocol. #[repr(transparent)] pub struct PyMapping(PyAny); pyobject_native_type_named!(PyMapping); pyobject_native_type_extract!(PyMapping); impl PyMapping { /// Returns the number of objects in the mapping. /// /// This is equivalent to the Python expression `len(self)`. #[inline] pub fn len(&self) -> PyResult { let v = unsafe { ffi::PyMapping_Size(self.as_ptr()) }; crate::err::error_on_minusone(self.py(), v)?; Ok(v as usize) } /// Returns whether the mapping is empty. #[inline] pub fn is_empty(&self) -> PyResult { self.len().map(|l| l == 0) } /// Determines if the mapping contains the specified key. /// /// This is equivalent to the Python expression `key in self`. pub fn contains(&self, key: K) -> PyResult where K: ToPyObject, { PyAny::contains(self, key) } /// Gets the item in self with key `key`. /// /// Returns an `Err` if the item with specified key is not found, usually `KeyError`. /// /// This is equivalent to the Python expression `self[key]`. #[inline] pub fn get_item(&self, key: K) -> PyResult<&PyAny> where K: ToPyObject, { PyAny::get_item(self, key) } /// Sets the item in self with key `key`. /// /// This is equivalent to the Python expression `self[key] = value`. #[inline] pub fn set_item(&self, key: K, value: V) -> PyResult<()> where K: ToPyObject, V: ToPyObject, { PyAny::set_item(self, key, value) } /// Deletes the item with key `key`. /// /// This is equivalent to the Python statement `del self[key]`. #[inline] pub fn del_item(&self, key: K) -> PyResult<()> where K: ToPyObject, { PyAny::del_item(self, key) } /// Returns a sequence containing all keys in the mapping. #[inline] pub fn keys(&self) -> PyResult<&PySequence> { unsafe { self.py() .from_owned_ptr_or_err(ffi::PyMapping_Keys(self.as_ptr())) } } /// Returns a sequence containing all values in the mapping. #[inline] pub fn values(&self) -> PyResult<&PySequence> { unsafe { self.py() .from_owned_ptr_or_err(ffi::PyMapping_Values(self.as_ptr())) } } /// Returns a sequence of tuples of all (key, value) pairs in the mapping. #[inline] pub fn items(&self) -> PyResult<&PySequence> { unsafe { self.py() .from_owned_ptr_or_err(ffi::PyMapping_Items(self.as_ptr())) } } /// Register a pyclass as a subclass of `collections.abc.Mapping` (from the Python standard /// library). This is equvalent to `collections.abc.Mapping.register(T)` in Python. /// This registration is required for a pyclass to be downcastable from `PyAny` to `PyMapping`. pub fn register(py: Python<'_>) -> PyResult<()> { let ty = T::type_object(py); get_mapping_abc(py)?.call_method1("register", (ty,))?; Ok(()) } } static MAPPING_ABC: GILOnceCell> = GILOnceCell::new(); fn get_mapping_abc(py: Python<'_>) -> PyResult<&PyType> { MAPPING_ABC .get_or_try_init(py, || { py.import("collections.abc")?.getattr("Mapping")?.extract() }) .map(|ty| ty.as_ref(py)) } impl<'v> PyTryFrom<'v> for PyMapping { /// Downcasting to `PyMapping` requires the concrete class to be a subclass (or registered /// subclass) of `collections.abc.Mapping` (from the Python standard library) - i.e. /// `isinstance(, collections.abc.Mapping) == True`. fn try_from>(value: V) -> Result<&'v PyMapping, PyDowncastError<'v>> { let value = value.into(); // Using `is_instance` for `collections.abc.Mapping` is slow, so provide // optimized case dict as a well-known mapping if PyDict::is_type_of(value) || get_mapping_abc(value.py()) .and_then(|abc| value.is_instance(abc)) // TODO: surface errors in this chain to the user .unwrap_or(false) { unsafe { return Ok(value.downcast_unchecked()) } } Err(PyDowncastError::new(value, "Mapping")) } #[inline] fn try_from_exact>(value: V) -> Result<&'v PyMapping, PyDowncastError<'v>> { value.into().downcast() } #[inline] unsafe fn try_from_unchecked>(value: V) -> &'v PyMapping { let ptr = value.into() as *const _ as *const PyMapping; &*ptr } } impl Py { /// Borrows a GIL-bound reference to the PyMapping. By binding to the GIL lifetime, this /// allows the GIL-bound reference to not require `Python` for any of its methods. pub fn as_ref<'py>(&'py self, _py: Python<'py>) -> &'py PyMapping { let any = self.as_ptr() as *const PyAny; unsafe { PyNativeType::unchecked_downcast(&*any) } } /// Similar to [`as_ref`](#method.as_ref), and also consumes this `Py` and registers the /// Python object reference in PyO3's object storage. The reference count for the Python /// object will not be decreased until the GIL lifetime ends. pub fn into_ref(self, py: Python<'_>) -> &PyMapping { unsafe { py.from_owned_ptr(self.into_ptr()) } } } #[cfg(test)] mod tests { use std::collections::HashMap; use crate::{ exceptions::PyKeyError, types::{PyDict, PyTuple}, Python, }; use super::*; #[test] fn test_len() { Python::with_gil(|py| { let mut v = HashMap::new(); let ob = v.to_object(py); let mapping: &PyMapping = ob.downcast(py).unwrap(); assert_eq!(0, mapping.len().unwrap()); assert!(mapping.is_empty().unwrap()); v.insert(7, 32); let ob = v.to_object(py); let mapping2: &PyMapping = ob.downcast(py).unwrap(); assert_eq!(1, mapping2.len().unwrap()); assert!(!mapping2.is_empty().unwrap()); }); } #[test] fn test_contains() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert("key0", 1234); let ob = v.to_object(py); let mapping: &PyMapping = ob.downcast(py).unwrap(); mapping.set_item("key1", "foo").unwrap(); assert!(mapping.contains("key0").unwrap()); assert!(mapping.contains("key1").unwrap()); assert!(!mapping.contains("key2").unwrap()); }); } #[test] fn test_get_item() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); let ob = v.to_object(py); let mapping: &PyMapping = ob.downcast(py).unwrap(); assert_eq!( 32, mapping.get_item(7i32).unwrap().extract::().unwrap() ); assert!(mapping .get_item(8i32) .unwrap_err() .is_instance_of::(py)); }); } #[test] fn test_set_item() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); let ob = v.to_object(py); let mapping: &PyMapping = ob.downcast(py).unwrap(); assert!(mapping.set_item(7i32, 42i32).is_ok()); // change assert!(mapping.set_item(8i32, 123i32).is_ok()); // insert assert_eq!( 42i32, mapping.get_item(7i32).unwrap().extract::().unwrap() ); assert_eq!( 123i32, mapping.get_item(8i32).unwrap().extract::().unwrap() ); }); } #[test] fn test_del_item() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); let ob = v.to_object(py); let mapping: &PyMapping = ob.downcast(py).unwrap(); assert!(mapping.del_item(7i32).is_ok()); assert_eq!(0, mapping.len().unwrap()); assert!(mapping .get_item(7i32) .unwrap_err() .is_instance_of::(py)); }); } #[test] fn test_items() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); v.insert(8, 42); v.insert(9, 123); let ob = v.to_object(py); let mapping: &PyMapping = ob.downcast(py).unwrap(); // Can't just compare against a vector of tuples since we don't have a guaranteed ordering. let mut key_sum = 0; let mut value_sum = 0; for el in mapping.items().unwrap().iter().unwrap() { let tuple = el.unwrap().downcast::().unwrap(); key_sum += tuple.get_item(0).unwrap().extract::().unwrap(); value_sum += tuple.get_item(1).unwrap().extract::().unwrap(); } assert_eq!(7 + 8 + 9, key_sum); assert_eq!(32 + 42 + 123, value_sum); }); } #[test] fn test_keys() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); v.insert(8, 42); v.insert(9, 123); let ob = v.to_object(py); let mapping: &PyMapping = ob.downcast(py).unwrap(); // Can't just compare against a vector of tuples since we don't have a guaranteed ordering. let mut key_sum = 0; for el in mapping.keys().unwrap().iter().unwrap() { key_sum += el.unwrap().extract::().unwrap(); } assert_eq!(7 + 8 + 9, key_sum); }); } #[test] fn test_values() { Python::with_gil(|py| { let mut v = HashMap::new(); v.insert(7, 32); v.insert(8, 42); v.insert(9, 123); let ob = v.to_object(py); let mapping: &PyMapping = ob.downcast(py).unwrap(); // Can't just compare against a vector of tuples since we don't have a guaranteed ordering. let mut values_sum = 0; for el in mapping.values().unwrap().iter().unwrap() { values_sum += el.unwrap().extract::().unwrap(); } assert_eq!(32 + 42 + 123, values_sum); }); } #[test] fn test_as_ref() { Python::with_gil(|py| { let mapping: Py = PyDict::new(py).as_mapping().into(); let mapping_ref: &PyMapping = mapping.as_ref(py); assert_eq!(mapping_ref.len().unwrap(), 0); }) } #[test] fn test_into_ref() { Python::with_gil(|py| { let bare_mapping = PyDict::new(py).as_mapping(); assert_eq!(bare_mapping.get_refcnt(), 1); let mapping: Py = bare_mapping.into(); assert_eq!(bare_mapping.get_refcnt(), 2); let mapping_ref = mapping.into_ref(py); assert_eq!(mapping_ref.len().unwrap(), 0); assert_eq!(mapping_ref.get_refcnt(), 2); }) } } pyo3-0.20.2/src/types/mod.rs000064400000000000000000000244551046102023000136710ustar 00000000000000//! Various types defined by the Python interpreter such as `int`, `str` and `tuple`. pub use self::any::PyAny; pub use self::boolobject::PyBool; pub use self::bytearray::PyByteArray; pub use self::bytes::PyBytes; pub use self::capsule::PyCapsule; #[cfg(not(Py_LIMITED_API))] pub use self::code::PyCode; pub use self::complex::PyComplex; #[cfg(not(Py_LIMITED_API))] pub use self::datetime::{ timezone_utc, PyDate, PyDateAccess, PyDateTime, PyDelta, PyDeltaAccess, PyTime, PyTimeAccess, PyTzInfo, PyTzInfoAccess, }; pub use self::dict::{IntoPyDict, PyDict}; #[cfg(not(PyPy))] pub use self::dict::{PyDictItems, PyDictKeys, PyDictValues}; pub use self::ellipsis::PyEllipsis; pub use self::floatob::PyFloat; #[cfg(all(not(Py_LIMITED_API), not(PyPy)))] pub use self::frame::PyFrame; pub use self::frozenset::{PyFrozenSet, PyFrozenSetBuilder}; pub use self::function::PyCFunction; #[cfg(all(not(Py_LIMITED_API), not(PyPy)))] pub use self::function::PyFunction; pub use self::iterator::PyIterator; pub use self::list::PyList; pub use self::mapping::PyMapping; pub use self::module::PyModule; pub use self::none::PyNone; pub use self::notimplemented::PyNotImplemented; pub use self::num::PyLong; pub use self::num::PyLong as PyInt; #[cfg(not(PyPy))] pub use self::pysuper::PySuper; pub use self::sequence::PySequence; pub use self::set::PySet; pub use self::slice::{PySlice, PySliceIndices}; #[cfg(not(Py_LIMITED_API))] pub use self::string::PyStringData; pub use self::string::{PyString, PyString as PyUnicode}; pub use self::traceback::PyTraceback; pub use self::tuple::PyTuple; pub use self::typeobject::PyType; /// Iteration over Python collections. /// /// When working with a Python collection, one approach is to convert it to a Rust collection such /// as `Vec` or `HashMap`. However this is a relatively expensive operation. If you just want to /// visit all their items, consider iterating over the collections directly: /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyDict; /// /// # pub fn main() -> PyResult<()> { /// Python::with_gil(|py| { /// let dict: &PyDict = py.eval("{'a':'b', 'c':'d'}", None, None)?.downcast()?; /// /// for (key, value) in dict { /// println!("key: {}, value: {}", key, value); /// } /// /// Ok(()) /// }) /// # } /// ``` /// /// If PyO3 detects that the collection is mutated during iteration, it will panic. /// /// These iterators use Python's C-API directly. However in certain cases, like when compiling for /// the Limited API and PyPy, the underlying structures are opaque and that may not be possible. /// In these cases the iterators are implemented by forwarding to [`PyIterator`]. pub mod iter { pub use super::dict::PyDictIterator; pub use super::frozenset::PyFrozenSetIterator; pub use super::set::PySetIterator; pub use super::tuple::PyTupleIterator; } // Implementations core to all native types #[doc(hidden)] #[macro_export] macro_rules! pyobject_native_type_base( ($name:ty $(;$generics:ident)* ) => { unsafe impl<$($generics,)*> $crate::PyNativeType for $name {} impl<$($generics,)*> ::std::fmt::Debug for $name { fn fmt(&self, f: &mut ::std::fmt::Formatter<'_>) -> ::std::result::Result<(), ::std::fmt::Error> { let s = self.repr().or(::std::result::Result::Err(::std::fmt::Error))?; f.write_str(&s.to_string_lossy()) } } impl<$($generics,)*> ::std::fmt::Display for $name { fn fmt(&self, f: &mut ::std::fmt::Formatter<'_>) -> ::std::result::Result<(), ::std::fmt::Error> { match self.str() { ::std::result::Result::Ok(s) => return f.write_str(&s.to_string_lossy()), ::std::result::Result::Err(err) => err.write_unraisable(self.py(), ::std::option::Option::Some(self)), } match self.get_type().name() { ::std::result::Result::Ok(name) => ::std::write!(f, "", name), ::std::result::Result::Err(_err) => f.write_str(""), } } } impl<$($generics,)*> $crate::ToPyObject for $name { #[inline] fn to_object(&self, py: $crate::Python<'_>) -> $crate::PyObject { unsafe { $crate::PyObject::from_borrowed_ptr(py, self.as_ptr()) } } } }; ); // Implementations core to all native types except for PyAny (because they don't // make sense on PyAny / have different implementations). #[doc(hidden)] #[macro_export] macro_rules! pyobject_native_type_named ( ($name:ty $(;$generics:ident)*) => { $crate::pyobject_native_type_base!($name $(;$generics)*); impl<$($generics,)*> ::std::convert::AsRef<$crate::PyAny> for $name { #[inline] fn as_ref(&self) -> &$crate::PyAny { &self.0 } } impl<$($generics,)*> ::std::ops::Deref for $name { type Target = $crate::PyAny; #[inline] fn deref(&self) -> &$crate::PyAny { &self.0 } } unsafe impl<$($generics,)*> $crate::AsPyPointer for $name { /// Gets the underlying FFI pointer, returns a borrowed pointer. #[inline] fn as_ptr(&self) -> *mut $crate::ffi::PyObject { self.0.as_ptr() } } impl<$($generics,)*> $crate::IntoPy<$crate::Py<$name>> for &'_ $name { #[inline] fn into_py(self, py: $crate::Python<'_>) -> $crate::Py<$name> { unsafe { $crate::Py::from_borrowed_ptr(py, self.as_ptr()) } } } impl<$($generics,)*> ::std::convert::From<&'_ $name> for $crate::Py<$name> { #[inline] fn from(other: &$name) -> Self { use $crate::PyNativeType; unsafe { $crate::Py::from_borrowed_ptr(other.py(), other.as_ptr()) } } } impl<'a, $($generics,)*> ::std::convert::From<&'a $name> for &'a $crate::PyAny { fn from(ob: &'a $name) -> Self { unsafe{&*(ob as *const $name as *const $crate::PyAny)} } } }; ); #[doc(hidden)] #[macro_export] macro_rules! pyobject_native_static_type_object( ($typeobject:expr) => { |_py| unsafe { ::std::ptr::addr_of_mut!($typeobject) } }; ); #[doc(hidden)] #[macro_export] macro_rules! pyobject_native_type_info( ($name:ty, $typeobject:expr, $module:expr $(, #checkfunction=$checkfunction:path)? $(;$generics:ident)*) => { unsafe impl<$($generics,)*> $crate::type_object::PyTypeInfo for $name { type AsRefTarget = Self; const NAME: &'static str = stringify!($name); const MODULE: ::std::option::Option<&'static str> = $module; #[inline] #[allow(clippy::redundant_closure_call)] fn type_object_raw(py: $crate::Python<'_>) -> *mut $crate::ffi::PyTypeObject { $typeobject(py) } $( #[inline] fn is_type_of(ptr: &$crate::PyAny) -> bool { #[allow(unused_unsafe)] unsafe { $checkfunction(ptr.as_ptr()) > 0 } } )? } }; ); // NOTE: This macro is not included in pyobject_native_type_base! // because rust-numpy has a special implementation. #[doc(hidden)] #[macro_export] macro_rules! pyobject_native_type_extract { ($name:ty $(;$generics:ident)*) => { impl<'py, $($generics,)*> $crate::FromPyObject<'py> for &'py $name { fn extract(obj: &'py $crate::PyAny) -> $crate::PyResult { obj.downcast().map_err(::std::convert::Into::into) } } } } /// Declares all of the boilerplate for Python types. #[doc(hidden)] #[macro_export] macro_rules! pyobject_native_type_core { ($name:ty, $typeobject:expr, #module=$module:expr $(, #checkfunction=$checkfunction:path)? $(;$generics:ident)*) => { $crate::pyobject_native_type_named!($name $(;$generics)*); $crate::pyobject_native_type_info!($name, $typeobject, $module $(, #checkfunction=$checkfunction)? $(;$generics)*); $crate::pyobject_native_type_extract!($name $(;$generics)*); }; ($name:ty, $typeobject:expr $(, #checkfunction=$checkfunction:path)? $(;$generics:ident)*) => { $crate::pyobject_native_type_core!($name, $typeobject, #module=::std::option::Option::Some("builtins") $(, #checkfunction=$checkfunction)? $(;$generics)*); }; } #[doc(hidden)] #[macro_export] macro_rules! pyobject_native_type_sized { ($name:ty, $layout:path $(;$generics:ident)*) => { unsafe impl $crate::type_object::PyLayout<$name> for $layout {} impl $crate::type_object::PySizedLayout<$name> for $layout {} impl<$($generics,)*> $crate::impl_::pyclass::PyClassBaseType for $name { type LayoutAsBase = $crate::pycell::PyCellBase<$layout>; type BaseNativeType = $name; type Initializer = $crate::pyclass_init::PyNativeTypeInitializer; type PyClassMutability = $crate::pycell::impl_::ImmutableClass; } } } /// Declares all of the boilerplate for Python types which can be inherited from (because the exact /// Python layout is known). #[doc(hidden)] #[macro_export] macro_rules! pyobject_native_type { ($name:ty, $layout:path, $typeobject:expr $(, #module=$module:expr)? $(, #checkfunction=$checkfunction:path)? $(;$generics:ident)*) => { $crate::pyobject_native_type_core!($name, $typeobject $(, #module=$module)? $(, #checkfunction=$checkfunction)? $(;$generics)*); // To prevent inheriting native types with ABI3 #[cfg(not(Py_LIMITED_API))] $crate::pyobject_native_type_sized!($name, $layout $(;$generics)*); }; } mod any; mod boolobject; mod bytearray; mod bytes; mod capsule; #[cfg(not(Py_LIMITED_API))] mod code; mod complex; #[cfg(not(Py_LIMITED_API))] mod datetime; mod dict; mod ellipsis; mod floatob; #[cfg(all(not(Py_LIMITED_API), not(PyPy)))] mod frame; mod frozenset; mod function; mod iterator; pub(crate) mod list; mod mapping; mod module; mod none; mod notimplemented; mod num; #[cfg(not(PyPy))] mod pysuper; mod sequence; pub(crate) mod set; mod slice; mod string; mod traceback; mod tuple; mod typeobject; pyo3-0.20.2/src/types/module.rs000064400000000000000000000314441046102023000143730ustar 00000000000000use crate::callback::IntoPyCallbackOutput; use crate::err::{PyErr, PyResult}; use crate::exceptions; use crate::ffi; use crate::pyclass::PyClass; use crate::types::{PyAny, PyCFunction, PyDict, PyList, PyString}; use crate::{IntoPy, Py, PyObject, Python}; use std::ffi::{CStr, CString}; use std::str; /// Represents a Python [`module`][1] object. /// /// As with all other Python objects, modules are first class citizens. /// This means they can be passed to or returned from functions, /// created dynamically, assigned to variables and so forth. /// /// [1]: https://docs.python.org/3/tutorial/modules.html #[repr(transparent)] pub struct PyModule(PyAny); pyobject_native_type_core!(PyModule, pyobject_native_static_type_object!(ffi::PyModule_Type), #checkfunction=ffi::PyModule_Check); impl PyModule { /// Creates a new module object with the `__name__` attribute set to `name`. /// /// # Examples /// /// ``` rust /// use pyo3::prelude::*; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let module = PyModule::new(py, "my_module")?; /// /// assert_eq!(module.name()?, "my_module"); /// Ok(()) /// })?; /// # Ok(())} /// ``` pub fn new<'p>(py: Python<'p>, name: &str) -> PyResult<&'p PyModule> { // Could use PyModule_NewObject, but it doesn't exist on PyPy. let name = CString::new(name)?; unsafe { py.from_owned_ptr_or_err(ffi::PyModule_New(name.as_ptr())) } } /// Imports the Python module with the specified name. /// /// # Examples /// /// ```no_run /// # fn main() { /// use pyo3::prelude::*; /// /// Python::with_gil(|py| { /// let module = PyModule::import(py, "antigravity").expect("No flying for you."); /// }); /// # } /// ``` /// /// This is equivalent to the following Python expression: /// ```python /// import antigravity /// ``` pub fn import(py: Python<'_>, name: N) -> PyResult<&PyModule> where N: IntoPy>, { let name: Py = name.into_py(py); unsafe { py.from_owned_ptr_or_err(ffi::PyImport_Import(name.as_ptr())) } } /// Creates and loads a module named `module_name`, /// containing the Python code passed to `code` /// and pretending to live at `file_name`. /// ///
///
⚠ ️
///
    //
    ///  Warning: This will compile and execute code. Never pass untrusted code to this function!
    ///
    /// 
/// /// # Errors /// /// Returns `PyErr` if: /// - `code` is not syntactically correct Python. /// - Any Python exceptions are raised while initializing the module. /// - Any of the arguments cannot be converted to [`CString`]s. /// /// # Example: bundle in a file at compile time with [`include_str!`][std::include_str]: /// /// ```rust /// use pyo3::prelude::*; /// /// # fn main() -> PyResult<()> { /// // This path is resolved relative to this file. /// let code = include_str!("../../assets/script.py"); /// /// Python::with_gil(|py| -> PyResult<()> { /// PyModule::from_code(py, code, "example.py", "example")?; /// Ok(()) /// })?; /// # Ok(()) /// # } /// ``` /// /// # Example: Load a file at runtime with [`std::fs::read_to_string`]. /// /// ```rust /// use pyo3::prelude::*; /// /// # fn main() -> PyResult<()> { /// // This path is resolved by however the platform resolves paths, /// // which also makes this less portable. Consider using `include_str` /// // if you just want to bundle a script with your module. /// let code = std::fs::read_to_string("assets/script.py")?; /// /// Python::with_gil(|py| -> PyResult<()> { /// PyModule::from_code(py, &code, "example.py", "example")?; /// Ok(()) /// })?; /// Ok(()) /// # } /// ``` pub fn from_code<'p>( py: Python<'p>, code: &str, file_name: &str, module_name: &str, ) -> PyResult<&'p PyModule> { let data = CString::new(code)?; let filename = CString::new(file_name)?; let module = CString::new(module_name)?; unsafe { let cptr = ffi::Py_CompileString(data.as_ptr(), filename.as_ptr(), ffi::Py_file_input); if cptr.is_null() { return Err(PyErr::fetch(py)); } let mptr = ffi::PyImport_ExecCodeModuleEx(module.as_ptr(), cptr, filename.as_ptr()); ffi::Py_DECREF(cptr); if mptr.is_null() { return Err(PyErr::fetch(py)); } <&PyModule as crate::FromPyObject>::extract(py.from_owned_ptr_or_err(mptr)?) } } /// Returns the module's `__dict__` attribute, which contains the module's symbol table. pub fn dict(&self) -> &PyDict { unsafe { // PyModule_GetDict returns borrowed ptr; must make owned for safety (see #890). let ptr = ffi::PyModule_GetDict(self.as_ptr()); self.py().from_owned_ptr(ffi::_Py_NewRef(ptr)) } } /// Returns the index (the `__all__` attribute) of the module, /// creating one if needed. /// /// `__all__` declares the items that will be imported with `from my_module import *`. pub fn index(&self) -> PyResult<&PyList> { let __all__ = __all__(self.py()); match self.getattr(__all__) { Ok(idx) => idx.downcast().map_err(PyErr::from), Err(err) => { if err.is_instance_of::(self.py()) { let l = PyList::empty(self.py()); self.setattr(__all__, l).map_err(PyErr::from)?; Ok(l) } else { Err(err) } } } } /// Returns the name (the `__name__` attribute) of the module. /// /// May fail if the module does not have a `__name__` attribute. pub fn name(&self) -> PyResult<&str> { let ptr = unsafe { ffi::PyModule_GetName(self.as_ptr()) }; if ptr.is_null() { Err(PyErr::fetch(self.py())) } else { let name = unsafe { CStr::from_ptr(ptr) } .to_str() .expect("PyModule_GetName expected to return utf8"); Ok(name) } } /// Returns the filename (the `__file__` attribute) of the module. /// /// May fail if the module does not have a `__file__` attribute. #[cfg(not(PyPy))] pub fn filename(&self) -> PyResult<&str> { unsafe { self.py() .from_owned_ptr_or_err::(ffi::PyModule_GetFilenameObject(self.as_ptr()))? .to_str() } } /// Adds an attribute to the module. /// /// For adding classes, functions or modules, prefer to use [`PyModule::add_class`], /// [`PyModule::add_function`] or [`PyModule::add_submodule`] instead, respectively. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// /// #[pymodule] /// fn my_module(_py: Python<'_>, module: &PyModule) -> PyResult<()> { /// module.add("c", 299_792_458)?; /// Ok(()) /// } /// ``` /// /// Python code can then do the following: /// /// ```python /// from my_module import c /// /// print("c is", c) /// ``` /// /// This will result in the following output: /// /// ```text /// c is 299792458 /// ``` pub fn add(&self, name: &str, value: V) -> PyResult<()> where V: IntoPy, { self.index()? .append(name) .expect("could not append __name__ to __all__"); self.setattr(name, value.into_py(self.py())) } /// Adds a new class to the module. /// /// Notice that this method does not take an argument. /// Instead, this method is *generic*, and requires us to use the /// "turbofish" syntax to specify the class we want to add. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// /// #[pyclass] /// struct Foo { /* fields omitted */ } /// /// #[pymodule] /// fn my_module(_py: Python<'_>, module: &PyModule) -> PyResult<()> { /// module.add_class::()?; /// Ok(()) /// } /// ``` /// /// Python code can see this class as such: /// ```python /// from my_module import Foo /// /// print("Foo is", Foo) /// ``` /// /// This will result in the following output: /// ```text /// Foo is /// ``` /// /// Note that as we haven't defined a [constructor][1], Python code can't actually /// make an *instance* of `Foo` (or *get* one for that matter, as we haven't exported /// anything that can return instances of `Foo`). /// /// [1]: https://pyo3.rs/latest/class.html#constructor pub fn add_class(&self) -> PyResult<()> where T: PyClass, { let py = self.py(); self.add(T::NAME, T::lazy_type_object().get_or_try_init(py)?) } /// Adds a function or a (sub)module to a module, using the functions name as name. /// /// Prefer to use [`PyModule::add_function`] and/or [`PyModule::add_submodule`] instead. pub fn add_wrapped<'a, T>(&'a self, wrapper: &impl Fn(Python<'a>) -> T) -> PyResult<()> where T: IntoPyCallbackOutput, { self._add_wrapped(wrapper(self.py()).convert(self.py())?) } fn _add_wrapped(&self, object: PyObject) -> PyResult<()> { let py = self.py(); let name = object.getattr(py, __name__(py))?; let name = name.extract(py)?; self.add(name, object) } /// Adds a submodule to a module. /// /// This is especially useful for creating module hierarchies. /// /// Note that this doesn't define a *package*, so this won't allow Python code /// to directly import submodules by using /// `from my_module import submodule`. /// For more information, see [#759][1] and [#1517][2]. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// /// #[pymodule] /// fn my_module(py: Python<'_>, module: &PyModule) -> PyResult<()> { /// let submodule = PyModule::new(py, "submodule")?; /// submodule.add("super_useful_constant", "important")?; /// /// module.add_submodule(submodule)?; /// Ok(()) /// } /// ``` /// /// Python code can then do the following: /// /// ```python /// import my_module /// /// print("super_useful_constant is", my_module.submodule.super_useful_constant) /// ``` /// /// This will result in the following output: /// /// ```text /// super_useful_constant is important /// ``` /// /// [1]: https://github.com/PyO3/pyo3/issues/759 /// [2]: https://github.com/PyO3/pyo3/issues/1517#issuecomment-808664021 pub fn add_submodule(&self, module: &PyModule) -> PyResult<()> { let name = module.name()?; self.add(name, module) } /// Add a function to a module. /// /// Note that this also requires the [`wrap_pyfunction!`][2] macro /// to wrap a function annotated with [`#[pyfunction]`][1]. /// /// ```rust /// use pyo3::prelude::*; /// /// #[pyfunction] /// fn say_hello() { /// println!("Hello world!") /// } /// #[pymodule] /// fn my_module(_py: Python<'_>, module: &PyModule) -> PyResult<()> { /// module.add_function(wrap_pyfunction!(say_hello, module)?) /// } /// ``` /// /// Python code can then do the following: /// /// ```python /// from my_module import say_hello /// /// say_hello() /// ``` /// /// This will result in the following output: /// /// ```text /// Hello world! /// ``` /// /// [1]: crate::prelude::pyfunction /// [2]: crate::wrap_pyfunction pub fn add_function<'a>(&'a self, fun: &'a PyCFunction) -> PyResult<()> { let name = fun.getattr(__name__(self.py()))?.extract()?; self.add(name, fun) } } fn __all__(py: Python<'_>) -> &PyString { intern!(py, "__all__") } fn __name__(py: Python<'_>) -> &PyString { intern!(py, "__name__") } #[cfg(test)] mod tests { use crate::{types::PyModule, Python}; #[test] fn module_import_and_name() { Python::with_gil(|py| { let builtins = PyModule::import(py, "builtins").unwrap(); assert_eq!(builtins.name().unwrap(), "builtins"); }) } } pyo3-0.20.2/src/types/none.rs000064400000000000000000000042361046102023000140440ustar 00000000000000use crate::{ffi, IntoPy, PyAny, PyDowncastError, PyObject, PyTryFrom, Python, ToPyObject}; /// Represents the Python `None` object. #[repr(transparent)] pub struct PyNone(PyAny); pyobject_native_type_named!(PyNone); pyobject_native_type_extract!(PyNone); impl PyNone { /// Returns the `None` object. #[inline] pub fn get(py: Python<'_>) -> &PyNone { unsafe { py.from_borrowed_ptr(ffi::Py_None()) } } } impl<'v> PyTryFrom<'v> for PyNone { fn try_from>(value: V) -> Result<&'v Self, crate::PyDowncastError<'v>> { let value: &PyAny = value.into(); if value.is_none() { return unsafe { Ok(value.downcast_unchecked()) }; } Err(PyDowncastError::new(value, "NoneType")) } fn try_from_exact>( value: V, ) -> Result<&'v Self, crate::PyDowncastError<'v>> { value.into().downcast() } unsafe fn try_from_unchecked>(value: V) -> &'v Self { let ptr = value.into() as *const _ as *const PyNone; &*ptr } } /// `()` is converted to Python `None`. impl ToPyObject for () { fn to_object(&self, py: Python<'_>) -> PyObject { PyNone::get(py).into() } } impl IntoPy for () { #[inline] fn into_py(self, py: Python<'_>) -> PyObject { PyNone::get(py).into() } } #[cfg(test)] mod tests { use crate::types::{PyDict, PyNone}; use crate::{IntoPy, PyObject, Python, ToPyObject}; #[test] fn test_none_is_none() { Python::with_gil(|py| { assert!(PyNone::get(py).downcast::().unwrap().is_none()); }) } #[test] fn test_unit_to_object_is_none() { Python::with_gil(|py| { assert!(().to_object(py).downcast::(py).is_ok()); }) } #[test] fn test_unit_into_py_is_none() { Python::with_gil(|py| { let obj: PyObject = ().into_py(py); assert!(obj.downcast::(py).is_ok()); }) } #[test] fn test_dict_is_not_none() { Python::with_gil(|py| { assert!(PyDict::new(py).downcast::().is_err()); }) } } pyo3-0.20.2/src/types/notimplemented.rs000064400000000000000000000033261046102023000161300ustar 00000000000000use crate::{ffi, PyAny, PyDowncastError, PyTryFrom, Python}; /// Represents the Python `NotImplemented` object. #[repr(transparent)] pub struct PyNotImplemented(PyAny); pyobject_native_type_named!(PyNotImplemented); pyobject_native_type_extract!(PyNotImplemented); impl PyNotImplemented { /// Returns the `NotImplemented` object. #[inline] pub fn get(py: Python<'_>) -> &PyNotImplemented { unsafe { py.from_borrowed_ptr(ffi::Py_NotImplemented()) } } } impl<'v> PyTryFrom<'v> for PyNotImplemented { fn try_from>(value: V) -> Result<&'v Self, crate::PyDowncastError<'v>> { let value: &PyAny = value.into(); if unsafe { ffi::Py_NotImplemented() == value.as_ptr() } { return unsafe { Ok(value.downcast_unchecked()) }; } Err(PyDowncastError::new(value, "NotImplementedType")) } fn try_from_exact>( value: V, ) -> Result<&'v Self, crate::PyDowncastError<'v>> { value.into().downcast() } unsafe fn try_from_unchecked>(value: V) -> &'v Self { let ptr = value.into() as *const _ as *const PyNotImplemented; &*ptr } } #[cfg(test)] mod tests { use crate::types::{PyDict, PyNotImplemented}; use crate::Python; #[test] fn test_notimplemented_is_itself() { Python::with_gil(|py| { assert!(PyNotImplemented::get(py) .downcast::() .unwrap() .is(&py.NotImplemented())); }) } #[test] fn test_dict_is_not_notimplemented() { Python::with_gil(|py| { assert!(PyDict::new(py).downcast::().is_err()); }) } } pyo3-0.20.2/src/types/num.rs000064400000000000000000000006651046102023000137060ustar 00000000000000use crate::{ffi, PyAny}; /// Represents a Python `int` object. /// /// You can usually avoid directly working with this type /// by using [`ToPyObject`](crate::conversion::ToPyObject) /// and [`extract`](PyAny::extract) /// with the primitive Rust integer types. #[repr(transparent)] pub struct PyLong(PyAny); pyobject_native_type_core!(PyLong, pyobject_native_static_type_object!(ffi::PyLong_Type), #checkfunction=ffi::PyLong_Check); pyo3-0.20.2/src/types/pysuper.rs000064400000000000000000000031001046102023000146010ustar 00000000000000use crate::ffi; use crate::types::PyType; use crate::{PyAny, PyResult}; /// Represents a Python `super` object. /// /// This type is immutable. #[repr(transparent)] pub struct PySuper(PyAny); pyobject_native_type_core!( PySuper, pyobject_native_static_type_object!(ffi::PySuper_Type) ); impl PySuper { /// Constructs a new super object. More read about super object: [docs](https://docs.python.org/3/library/functions.html#super) /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// /// #[pyclass(subclass)] /// struct BaseClass { /// val1: usize, /// } /// /// #[pymethods] /// impl BaseClass { /// #[new] /// fn new() -> Self { /// BaseClass { val1: 10 } /// } /// /// pub fn method(&self) -> usize { /// self.val1 /// } /// } /// /// #[pyclass(extends=BaseClass)] /// struct SubClass {} /// /// #[pymethods] /// impl SubClass { /// #[new] /// fn new() -> (Self, BaseClass) { /// (SubClass {}, BaseClass::new()) /// } /// /// fn method(self_: &PyCell) -> PyResult<&PyAny> { /// let super_ = self_.py_super()?; /// super_.call_method("method", (), None) /// } /// } /// ``` pub fn new<'py>(ty: &'py PyType, obj: &'py PyAny) -> PyResult<&'py PySuper> { let py = ty.py(); let super_ = py.get_type::().call1((ty, obj))?; let super_ = super_.downcast::()?; Ok(super_) } } pyo3-0.20.2/src/types/sequence.rs000064400000000000000000000755431046102023000147260ustar 00000000000000use crate::err::{self, PyDowncastError, PyErr, PyResult}; use crate::exceptions::PyTypeError; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::internal_tricks::get_ssize_index; use crate::sync::GILOnceCell; use crate::type_object::PyTypeInfo; use crate::types::{PyAny, PyList, PyString, PyTuple, PyType}; use crate::{ffi, PyNativeType, PyObject, ToPyObject}; use crate::{FromPyObject, PyTryFrom}; use crate::{Py, Python}; /// Represents a reference to a Python object supporting the sequence protocol. #[repr(transparent)] pub struct PySequence(PyAny); pyobject_native_type_named!(PySequence); pyobject_native_type_extract!(PySequence); impl PySequence { /// Returns the number of objects in sequence. /// /// This is equivalent to the Python expression `len(self)`. #[inline] pub fn len(&self) -> PyResult { let v = unsafe { ffi::PySequence_Size(self.as_ptr()) }; crate::err::error_on_minusone(self.py(), v)?; Ok(v as usize) } /// Returns whether the sequence is empty. #[inline] pub fn is_empty(&self) -> PyResult { self.len().map(|l| l == 0) } /// Returns the concatenation of `self` and `other`. /// /// This is equivalent to the Python expression `self + other`. #[inline] pub fn concat(&self, other: &PySequence) -> PyResult<&PySequence> { unsafe { self.py() .from_owned_ptr_or_err(ffi::PySequence_Concat(self.as_ptr(), other.as_ptr())) } } /// Returns the result of repeating a sequence object `count` times. /// /// This is equivalent to the Python expression `self * count`. #[inline] pub fn repeat(&self, count: usize) -> PyResult<&PySequence> { unsafe { self.py().from_owned_ptr_or_err(ffi::PySequence_Repeat( self.as_ptr(), get_ssize_index(count), )) } } /// Concatenates `self` and `other`, in place if possible. /// /// This is equivalent to the Python expression `self.__iadd__(other)`. /// /// The Python statement `self += other` is syntactic sugar for `self = /// self.__iadd__(other)`. `__iadd__` should modify and return `self` if /// possible, but create and return a new object if not. #[inline] pub fn in_place_concat(&self, other: &PySequence) -> PyResult<&PySequence> { unsafe { self.py() .from_owned_ptr_or_err(ffi::PySequence_InPlaceConcat(self.as_ptr(), other.as_ptr())) } } /// Repeats the sequence object `count` times and updates `self`, if possible. /// /// This is equivalent to the Python expression `self.__imul__(other)`. /// /// The Python statement `self *= other` is syntactic sugar for `self = /// self.__imul__(other)`. `__imul__` should modify and return `self` if /// possible, but create and return a new object if not. #[inline] pub fn in_place_repeat(&self, count: usize) -> PyResult<&PySequence> { unsafe { self.py() .from_owned_ptr_or_err(ffi::PySequence_InPlaceRepeat( self.as_ptr(), get_ssize_index(count), )) } } /// Returns the `index`th element of the Sequence. /// /// This is equivalent to the Python expression `self[index]` without support of negative indices. #[inline] pub fn get_item(&self, index: usize) -> PyResult<&PyAny> { unsafe { self.py().from_owned_ptr_or_err(ffi::PySequence_GetItem( self.as_ptr(), get_ssize_index(index), )) } } /// Returns the slice of sequence object between `begin` and `end`. /// /// This is equivalent to the Python expression `self[begin:end]`. #[inline] pub fn get_slice(&self, begin: usize, end: usize) -> PyResult<&PySequence> { unsafe { self.py().from_owned_ptr_or_err(ffi::PySequence_GetSlice( self.as_ptr(), get_ssize_index(begin), get_ssize_index(end), )) } } /// Assigns object `item` to the `i`th element of self. /// /// This is equivalent to the Python statement `self[i] = v`. #[inline] pub fn set_item(&self, i: usize, item: I) -> PyResult<()> where I: ToPyObject, { fn inner(seq: &PySequence, i: usize, item: PyObject) -> PyResult<()> { err::error_on_minusone(seq.py(), unsafe { ffi::PySequence_SetItem(seq.as_ptr(), get_ssize_index(i), item.as_ptr()) }) } inner(self, i, item.to_object(self.py())) } /// Deletes the `i`th element of self. /// /// This is equivalent to the Python statement `del self[i]`. #[inline] pub fn del_item(&self, i: usize) -> PyResult<()> { err::error_on_minusone(self.py(), unsafe { ffi::PySequence_DelItem(self.as_ptr(), get_ssize_index(i)) }) } /// Assigns the sequence `v` to the slice of `self` from `i1` to `i2`. /// /// This is equivalent to the Python statement `self[i1:i2] = v`. #[inline] pub fn set_slice(&self, i1: usize, i2: usize, v: &PyAny) -> PyResult<()> { err::error_on_minusone(self.py(), unsafe { ffi::PySequence_SetSlice( self.as_ptr(), get_ssize_index(i1), get_ssize_index(i2), v.as_ptr(), ) }) } /// Deletes the slice from `i1` to `i2` from `self`. /// /// This is equivalent to the Python statement `del self[i1:i2]`. #[inline] pub fn del_slice(&self, i1: usize, i2: usize) -> PyResult<()> { err::error_on_minusone(self.py(), unsafe { ffi::PySequence_DelSlice(self.as_ptr(), get_ssize_index(i1), get_ssize_index(i2)) }) } /// Returns the number of occurrences of `value` in self, that is, return the /// number of keys for which `self[key] == value`. #[inline] #[cfg(not(PyPy))] pub fn count(&self, value: V) -> PyResult where V: ToPyObject, { fn inner(seq: &PySequence, value: PyObject) -> PyResult { let r = unsafe { ffi::PySequence_Count(seq.as_ptr(), value.as_ptr()) }; crate::err::error_on_minusone(seq.py(), r)?; Ok(r as usize) } inner(self, value.to_object(self.py())) } /// Determines if self contains `value`. /// /// This is equivalent to the Python expression `value in self`. #[inline] pub fn contains(&self, value: V) -> PyResult where V: ToPyObject, { fn inner(seq: &PySequence, value: PyObject) -> PyResult { let r = unsafe { ffi::PySequence_Contains(seq.as_ptr(), value.as_ptr()) }; match r { 0 => Ok(false), 1 => Ok(true), _ => Err(PyErr::fetch(seq.py())), } } inner(self, value.to_object(self.py())) } /// Returns the first index `i` for which `self[i] == value`. /// /// This is equivalent to the Python expression `self.index(value)`. #[inline] pub fn index(&self, value: V) -> PyResult where V: ToPyObject, { fn inner(seq: &PySequence, value: PyObject) -> PyResult { let r = unsafe { ffi::PySequence_Index(seq.as_ptr(), value.as_ptr()) }; crate::err::error_on_minusone(seq.py(), r)?; Ok(r as usize) } inner(self, value.to_object(self.py())) } /// Returns a fresh list based on the Sequence. #[inline] pub fn to_list(&self) -> PyResult<&PyList> { unsafe { self.py() .from_owned_ptr_or_err(ffi::PySequence_List(self.as_ptr())) } } /// Returns a fresh list based on the Sequence. #[inline] #[deprecated(since = "0.19.0", note = "renamed to .to_list()")] pub fn list(&self) -> PyResult<&PyList> { self.to_list() } /// Returns a fresh tuple based on the Sequence. #[inline] pub fn to_tuple(&self) -> PyResult<&PyTuple> { unsafe { self.py() .from_owned_ptr_or_err(ffi::PySequence_Tuple(self.as_ptr())) } } /// Returns a fresh tuple based on the Sequence. #[inline] #[deprecated(since = "0.19.0", note = "renamed to .to_tuple()")] pub fn tuple(&self) -> PyResult<&PyTuple> { self.to_tuple() } /// Register a pyclass as a subclass of `collections.abc.Sequence` (from the Python standard /// library). This is equvalent to `collections.abc.Sequence.register(T)` in Python. /// This registration is required for a pyclass to be downcastable from `PyAny` to `PySequence`. pub fn register(py: Python<'_>) -> PyResult<()> { let ty = T::type_object(py); get_sequence_abc(py)?.call_method1("register", (ty,))?; Ok(()) } } #[inline] fn sequence_len(seq: &PySequence) -> usize { seq.len().expect("failed to get sequence length") } #[inline] fn sequence_slice(seq: &PySequence, start: usize, end: usize) -> &PySequence { seq.get_slice(start, end) .expect("sequence slice operation failed") } index_impls!(PySequence, "sequence", sequence_len, sequence_slice); impl<'a, T> FromPyObject<'a> for Vec where T: FromPyObject<'a>, { fn extract(obj: &'a PyAny) -> PyResult { if obj.is_instance_of::() { return Err(PyTypeError::new_err("Can't extract `str` to `Vec`")); } extract_sequence(obj) } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { TypeInfo::sequence_of(T::type_input()) } } fn extract_sequence<'s, T>(obj: &'s PyAny) -> PyResult> where T: FromPyObject<'s>, { // Types that pass `PySequence_Check` usually implement enough of the sequence protocol // to support this function and if not, we will only fail extraction safely. let seq: &PySequence = unsafe { if ffi::PySequence_Check(obj.as_ptr()) != 0 { obj.downcast_unchecked() } else { return Err(PyDowncastError::new(obj, "Sequence").into()); } }; let mut v = Vec::with_capacity(seq.len().unwrap_or(0)); for item in seq.iter()? { v.push(item?.extract::()?); } Ok(v) } static SEQUENCE_ABC: GILOnceCell> = GILOnceCell::new(); fn get_sequence_abc(py: Python<'_>) -> PyResult<&PyType> { SEQUENCE_ABC .get_or_try_init(py, || { py.import("collections.abc")?.getattr("Sequence")?.extract() }) .map(|ty| ty.as_ref(py)) } impl<'v> PyTryFrom<'v> for PySequence { /// Downcasting to `PySequence` requires the concrete class to be a subclass (or registered /// subclass) of `collections.abc.Sequence` (from the Python standard library) - i.e. /// `isinstance(, collections.abc.Sequence) == True`. fn try_from>(value: V) -> Result<&'v PySequence, PyDowncastError<'v>> { let value = value.into(); // Using `is_instance` for `collections.abc.Sequence` is slow, so provide // optimized cases for list and tuples as common well-known sequences if PyList::is_type_of(value) || PyTuple::is_type_of(value) || get_sequence_abc(value.py()) .and_then(|abc| value.is_instance(abc)) // TODO: surface errors in this chain to the user .unwrap_or(false) { unsafe { return Ok(value.downcast_unchecked::()) } } Err(PyDowncastError::new(value, "Sequence")) } fn try_from_exact>(value: V) -> Result<&'v PySequence, PyDowncastError<'v>> { value.into().downcast() } #[inline] unsafe fn try_from_unchecked>(value: V) -> &'v PySequence { let ptr = value.into() as *const _ as *const PySequence; &*ptr } } impl Py { /// Borrows a GIL-bound reference to the PySequence. By binding to the GIL lifetime, this /// allows the GIL-bound reference to not require `Python` for any of its methods. /// /// ``` /// # use pyo3::prelude::*; /// # use pyo3::types::{PyList, PySequence}; /// # Python::with_gil(|py| { /// let seq: Py = PyList::empty(py).as_sequence().into(); /// let seq: &PySequence = seq.as_ref(py); /// assert_eq!(seq.len().unwrap(), 0); /// # }); /// ``` pub fn as_ref<'py>(&'py self, _py: Python<'py>) -> &'py PySequence { let any = self.as_ptr() as *const PyAny; unsafe { PyNativeType::unchecked_downcast(&*any) } } /// Similar to [`as_ref`](#method.as_ref), and also consumes this `Py` and registers the /// Python object reference in PyO3's object storage. The reference count for the Python /// object will not be decreased until the GIL lifetime ends. pub fn into_ref(self, py: Python<'_>) -> &PySequence { unsafe { py.from_owned_ptr(self.into_ptr()) } } } #[cfg(test)] mod tests { use crate::types::{PyList, PySequence, PyTuple}; use crate::{Py, PyObject, Python, ToPyObject}; fn get_object() -> PyObject { // Convenience function for getting a single unique object Python::with_gil(|py| { let obj = py.eval("object()", None, None).unwrap(); obj.to_object(py) }) } #[test] fn test_numbers_are_not_sequences() { Python::with_gil(|py| { let v = 42i32; assert!(v.to_object(py).downcast::(py).is_err()); }); } #[test] fn test_strings_are_sequences() { Python::with_gil(|py| { let v = "London Calling"; assert!(v.to_object(py).downcast::(py).is_ok()); }); } #[test] fn test_strings_cannot_be_extracted_to_vec() { Python::with_gil(|py| { let v = "London Calling"; let ob = v.to_object(py); assert!(ob.extract::>(py).is_err()); assert!(ob.extract::>(py).is_err()); assert!(ob.extract::>(py).is_err()); }); } #[test] fn test_seq_empty() { Python::with_gil(|py| { let v: Vec = vec![]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert_eq!(0, seq.len().unwrap()); let needle = 7i32.to_object(py); assert!(!seq.contains(&needle).unwrap()); }); } #[test] fn test_seq_is_empty() { Python::with_gil(|py| { let list = vec![1].to_object(py); let seq = list.downcast::(py).unwrap(); assert!(!seq.is_empty().unwrap()); let vec: Vec = Vec::new(); let empty_list = vec.to_object(py); let empty_seq = empty_list.downcast::(py).unwrap(); assert!(empty_seq.is_empty().unwrap()); }); } #[test] fn test_seq_contains() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2, 3, 5, 8]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert_eq!(6, seq.len().unwrap()); let bad_needle = 7i32.to_object(py); assert!(!seq.contains(&bad_needle).unwrap()); let good_needle = 8i32.to_object(py); assert!(seq.contains(&good_needle).unwrap()); let type_coerced_needle = 8f32.to_object(py); assert!(seq.contains(&type_coerced_needle).unwrap()); }); } #[test] fn test_seq_get_item() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2, 3, 5, 8]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert_eq!(1, seq.get_item(0).unwrap().extract::().unwrap()); assert_eq!(1, seq.get_item(1).unwrap().extract::().unwrap()); assert_eq!(2, seq.get_item(2).unwrap().extract::().unwrap()); assert_eq!(3, seq.get_item(3).unwrap().extract::().unwrap()); assert_eq!(5, seq.get_item(4).unwrap().extract::().unwrap()); assert_eq!(8, seq.get_item(5).unwrap().extract::().unwrap()); assert!(seq.get_item(10).is_err()); }); } #[test] fn test_seq_index_trait() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert_eq!(1, seq[0].extract::().unwrap()); assert_eq!(1, seq[1].extract::().unwrap()); assert_eq!(2, seq[2].extract::().unwrap()); }); } #[test] #[should_panic = "index 7 out of range for sequence"] fn test_seq_index_trait_panic() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); let _ = &seq[7]; }); } #[test] fn test_seq_index_trait_ranges() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert_eq!(vec![1, 2], seq[1..3].extract::>().unwrap()); assert_eq!(Vec::::new(), seq[3..3].extract::>().unwrap()); assert_eq!(vec![1, 2], seq[1..].extract::>().unwrap()); assert_eq!(Vec::::new(), seq[3..].extract::>().unwrap()); assert_eq!(vec![1, 1, 2], seq[..].extract::>().unwrap()); assert_eq!(vec![1, 2], seq[1..=2].extract::>().unwrap()); assert_eq!(vec![1, 1], seq[..2].extract::>().unwrap()); assert_eq!(vec![1, 1], seq[..=1].extract::>().unwrap()); }) } #[test] #[should_panic = "range start index 5 out of range for sequence of length 3"] fn test_seq_index_trait_range_panic_start() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); seq[5..10].extract::>().unwrap(); }) } #[test] #[should_panic = "range end index 10 out of range for sequence of length 3"] fn test_seq_index_trait_range_panic_end() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); seq[1..10].extract::>().unwrap(); }) } #[test] #[should_panic = "slice index starts at 2 but ends at 1"] fn test_seq_index_trait_range_panic_wrong_order() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); #[allow(clippy::reversed_empty_ranges)] seq[2..1].extract::>().unwrap(); }) } #[test] #[should_panic = "range start index 8 out of range for sequence of length 3"] fn test_seq_index_trait_range_from_panic() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); seq[8..].extract::>().unwrap(); }) } #[test] fn test_seq_del_item() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2, 3, 5, 8]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert!(seq.del_item(10).is_err()); assert_eq!(1, seq[0].extract::().unwrap()); assert!(seq.del_item(0).is_ok()); assert_eq!(1, seq[0].extract::().unwrap()); assert!(seq.del_item(0).is_ok()); assert_eq!(2, seq[0].extract::().unwrap()); assert!(seq.del_item(0).is_ok()); assert_eq!(3, seq[0].extract::().unwrap()); assert!(seq.del_item(0).is_ok()); assert_eq!(5, seq[0].extract::().unwrap()); assert!(seq.del_item(0).is_ok()); assert_eq!(8, seq[0].extract::().unwrap()); assert!(seq.del_item(0).is_ok()); assert_eq!(0, seq.len().unwrap()); assert!(seq.del_item(0).is_err()); }); } #[test] fn test_seq_set_item() { Python::with_gil(|py| { let v: Vec = vec![1, 2]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert_eq!(2, seq[1].extract::().unwrap()); assert!(seq.set_item(1, 10).is_ok()); assert_eq!(10, seq[1].extract::().unwrap()); }); } #[test] fn test_seq_set_item_refcnt() { let obj = get_object(); Python::with_gil(|py| { let v: Vec = vec![1, 2]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert!(seq.set_item(1, &obj).is_ok()); assert!(seq[1].as_ptr() == obj.as_ptr()); }); Python::with_gil(|py| { assert_eq!(1, obj.get_refcnt(py)); }); } #[test] fn test_seq_get_slice() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2, 3, 5, 8]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert_eq!( [1, 2, 3], seq.get_slice(1, 4).unwrap().extract::<[i32; 3]>().unwrap() ); assert_eq!( [3, 5, 8], seq.get_slice(3, 100) .unwrap() .extract::<[i32; 3]>() .unwrap() ); }); } #[test] fn test_set_slice() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2, 3, 5, 8]; let w: Vec = vec![7, 4]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); let ins = w.to_object(py); seq.set_slice(1, 4, ins.as_ref(py)).unwrap(); assert_eq!([1, 7, 4, 5, 8], seq.extract::<[i32; 5]>().unwrap()); seq.set_slice(3, 100, PyList::empty(py)).unwrap(); assert_eq!([1, 7, 4], seq.extract::<[i32; 3]>().unwrap()); }); } #[test] fn test_del_slice() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2, 3, 5, 8]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); seq.del_slice(1, 4).unwrap(); assert_eq!([1, 5, 8], seq.extract::<[i32; 3]>().unwrap()); seq.del_slice(1, 100).unwrap(); assert_eq!([1], seq.extract::<[i32; 1]>().unwrap()); }); } #[test] fn test_seq_index() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2, 3, 5, 8]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert_eq!(0, seq.index(1i32).unwrap()); assert_eq!(2, seq.index(2i32).unwrap()); assert_eq!(3, seq.index(3i32).unwrap()); assert_eq!(4, seq.index(5i32).unwrap()); assert_eq!(5, seq.index(8i32).unwrap()); assert!(seq.index(42i32).is_err()); }); } #[test] #[cfg(not(PyPy))] fn test_seq_count() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2, 3, 5, 8]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert_eq!(2, seq.count(1i32).unwrap()); assert_eq!(1, seq.count(2i32).unwrap()); assert_eq!(1, seq.count(3i32).unwrap()); assert_eq!(1, seq.count(5i32).unwrap()); assert_eq!(1, seq.count(8i32).unwrap()); assert_eq!(0, seq.count(42i32).unwrap()); }); } #[test] fn test_seq_iter() { Python::with_gil(|py| { let v: Vec = vec![1, 1, 2, 3, 5, 8]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); let mut idx = 0; for el in seq.iter().unwrap() { assert_eq!(v[idx], el.unwrap().extract::().unwrap()); idx += 1; } assert_eq!(idx, v.len()); }); } #[test] fn test_seq_strings() { Python::with_gil(|py| { let v = vec!["It", "was", "the", "worst", "of", "times"]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); let bad_needle = "blurst".to_object(py); assert!(!seq.contains(bad_needle).unwrap()); let good_needle = "worst".to_object(py); assert!(seq.contains(good_needle).unwrap()); }); } #[test] fn test_seq_concat() { Python::with_gil(|py| { let v: Vec = vec![1, 2, 3]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); let concat_seq = seq.concat(seq).unwrap(); assert_eq!(6, concat_seq.len().unwrap()); let concat_v: Vec = vec![1, 2, 3, 1, 2, 3]; for (el, cc) in concat_seq.iter().unwrap().zip(concat_v) { assert_eq!(cc, el.unwrap().extract::().unwrap()); } }); } #[test] fn test_seq_concat_string() { Python::with_gil(|py| { let v = "string"; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); let concat_seq = seq.concat(seq).unwrap(); assert_eq!(12, concat_seq.len().unwrap()); let concat_v = "stringstring".to_owned(); for (el, cc) in seq.iter().unwrap().zip(concat_v.chars()) { assert_eq!(cc, el.unwrap().extract::().unwrap()); } }); } #[test] fn test_seq_repeat() { Python::with_gil(|py| { let v = vec!["foo", "bar"]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); let repeat_seq = seq.repeat(3).unwrap(); assert_eq!(6, repeat_seq.len().unwrap()); let repeated = ["foo", "bar", "foo", "bar", "foo", "bar"]; for (el, rpt) in repeat_seq.iter().unwrap().zip(repeated.iter()) { assert_eq!(*rpt, el.unwrap().extract::().unwrap()); } }); } #[test] fn test_seq_inplace() { Python::with_gil(|py| { let v = vec!["foo", "bar"]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); let rep_seq = seq.in_place_repeat(3).unwrap(); assert_eq!(6, seq.len().unwrap()); assert!(seq.is(rep_seq)); let conc_seq = seq.in_place_concat(seq).unwrap(); assert_eq!(12, seq.len().unwrap()); assert!(seq.is(conc_seq)); }); } #[test] fn test_list_coercion() { Python::with_gil(|py| { let v = vec!["foo", "bar"]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert!(seq.to_list().unwrap().eq(PyList::new(py, &v)).unwrap()); #[allow(deprecated)] { assert!(seq.list().is_ok()); } }); } #[test] fn test_strings_coerce_to_lists() { Python::with_gil(|py| { let v = "foo"; let ob = v.to_object(py); let seq: &PySequence = ob.downcast(py).unwrap(); assert!(seq .to_list() .unwrap() .eq(PyList::new(py, ["f", "o", "o"])) .unwrap()); #[allow(deprecated)] { assert!(seq.list().is_ok()); } }); } #[test] fn test_tuple_coercion() { Python::with_gil(|py| { let v = ("foo", "bar"); let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert!(seq .to_tuple() .unwrap() .eq(PyTuple::new(py, ["foo", "bar"])) .unwrap()); #[allow(deprecated)] { assert!(seq.tuple().is_ok()); } }); } #[test] fn test_lists_coerce_to_tuples() { Python::with_gil(|py| { let v = vec!["foo", "bar"]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); assert!(seq.to_tuple().unwrap().eq(PyTuple::new(py, &v)).unwrap()); #[allow(deprecated)] { assert!(seq.tuple().is_ok()); } }); } #[test] fn test_extract_tuple_to_vec() { Python::with_gil(|py| { let v: Vec = py.eval("(1, 2)", None, None).unwrap().extract().unwrap(); assert!(v == [1, 2]); }); } #[test] fn test_extract_range_to_vec() { Python::with_gil(|py| { let v: Vec = py .eval("range(1, 5)", None, None) .unwrap() .extract() .unwrap(); assert!(v == [1, 2, 3, 4]); }); } #[test] fn test_extract_bytearray_to_vec() { Python::with_gil(|py| { let v: Vec = py .eval("bytearray(b'abc')", None, None) .unwrap() .extract() .unwrap(); assert!(v == b"abc"); }); } #[test] fn test_seq_downcast_unchecked() { Python::with_gil(|py| { let v = vec!["foo", "bar"]; let ob = v.to_object(py); let seq = ob.downcast::(py).unwrap(); let type_ptr = seq.as_ref(); let seq_from = unsafe { type_ptr.downcast_unchecked::() }; assert!(seq_from.to_list().is_ok()); }); } #[test] fn test_as_ref() { Python::with_gil(|py| { let seq: Py = PyList::empty(py).as_sequence().into(); let seq_ref: &PySequence = seq.as_ref(py); assert_eq!(seq_ref.len().unwrap(), 0); }) } #[test] fn test_into_ref() { Python::with_gil(|py| { let bare_seq = PyList::empty(py).as_sequence(); assert_eq!(bare_seq.get_refcnt(), 1); let seq: Py = bare_seq.into(); assert_eq!(bare_seq.get_refcnt(), 2); let seq_ref = seq.into_ref(py); assert_eq!(seq_ref.len().unwrap(), 0); assert_eq!(seq_ref.get_refcnt(), 2); }) } } pyo3-0.20.2/src/types/set.rs000064400000000000000000000267411046102023000137050ustar 00000000000000#[cfg(Py_LIMITED_API)] use crate::types::PyIterator; use crate::{ err::{self, PyErr, PyResult}, Py, }; use crate::{ffi, PyAny, PyObject, Python, ToPyObject}; use std::ptr; /// Represents a Python `set` #[repr(transparent)] pub struct PySet(PyAny); #[cfg(not(PyPy))] pyobject_native_type!( PySet, ffi::PySetObject, pyobject_native_static_type_object!(ffi::PySet_Type), #checkfunction=ffi::PySet_Check ); #[cfg(PyPy)] pyobject_native_type_core!( PySet, pyobject_native_static_type_object!(ffi::PySet_Type), #checkfunction=ffi::PySet_Check ); impl PySet { /// Creates a new set with elements from the given slice. /// /// Returns an error if some element is not hashable. #[inline] pub fn new<'a, 'p, T: ToPyObject + 'a>( py: Python<'p>, elements: impl IntoIterator, ) -> PyResult<&'p PySet> { new_from_iter(py, elements).map(|set| set.into_ref(py)) } /// Creates a new empty set. pub fn empty(py: Python<'_>) -> PyResult<&PySet> { unsafe { py.from_owned_ptr_or_err(ffi::PySet_New(ptr::null_mut())) } } /// Removes all elements from the set. #[inline] pub fn clear(&self) { unsafe { ffi::PySet_Clear(self.as_ptr()); } } /// Returns the number of items in the set. /// /// This is equivalent to the Python expression `len(self)`. #[inline] pub fn len(&self) -> usize { unsafe { ffi::PySet_Size(self.as_ptr()) as usize } } /// Checks if set is empty. pub fn is_empty(&self) -> bool { self.len() == 0 } /// Determines if the set contains the specified key. /// /// This is equivalent to the Python expression `key in self`. pub fn contains(&self, key: K) -> PyResult where K: ToPyObject, { fn inner(set: &PySet, key: PyObject) -> PyResult { match unsafe { ffi::PySet_Contains(set.as_ptr(), key.as_ptr()) } { 1 => Ok(true), 0 => Ok(false), _ => Err(PyErr::fetch(set.py())), } } inner(self, key.to_object(self.py())) } /// Removes the element from the set if it is present. /// /// Returns `true` if the element was present in the set. pub fn discard(&self, key: K) -> PyResult where K: ToPyObject, { fn inner(set: &PySet, key: PyObject) -> PyResult { match unsafe { ffi::PySet_Discard(set.as_ptr(), key.as_ptr()) } { 1 => Ok(true), 0 => Ok(false), _ => Err(PyErr::fetch(set.py())), } } inner(self, key.to_object(self.py())) } /// Adds an element to the set. pub fn add(&self, key: K) -> PyResult<()> where K: ToPyObject, { fn inner(set: &PySet, key: PyObject) -> PyResult<()> { err::error_on_minusone(set.py(), unsafe { ffi::PySet_Add(set.as_ptr(), key.as_ptr()) }) } inner(self, key.to_object(self.py())) } /// Removes and returns an arbitrary element from the set. pub fn pop(&self) -> Option { let element = unsafe { PyObject::from_owned_ptr_or_err(self.py(), ffi::PySet_Pop(self.as_ptr())) }; match element { Ok(e) => Some(e), Err(_) => None, } } /// Returns an iterator of values in this set. /// /// # Panics /// /// If PyO3 detects that the set is mutated during iteration, it will panic. pub fn iter(&self) -> PySetIterator<'_> { IntoIterator::into_iter(self) } } #[cfg(Py_LIMITED_API)] mod impl_ { use super::*; impl<'a> std::iter::IntoIterator for &'a PySet { type Item = &'a PyAny; type IntoIter = PySetIterator<'a>; /// Returns an iterator of values in this set. /// /// # Panics /// /// If PyO3 detects that the set is mutated during iteration, it will panic. fn into_iter(self) -> Self::IntoIter { PySetIterator { it: PyIterator::from_object(self).unwrap(), } } } /// PyO3 implementation of an iterator for a Python `set` object. pub struct PySetIterator<'p> { it: &'p PyIterator, } impl<'py> Iterator for PySetIterator<'py> { type Item = &'py super::PyAny; /// Advances the iterator and returns the next value. /// /// # Panics /// /// If PyO3 detects that the set is mutated during iteration, it will panic. #[inline] fn next(&mut self) -> Option { self.it.next().map(Result::unwrap) } } } #[cfg(not(Py_LIMITED_API))] mod impl_ { use super::*; /// PyO3 implementation of an iterator for a Python `set` object. pub struct PySetIterator<'py> { set: &'py super::PySet, pos: ffi::Py_ssize_t, used: ffi::Py_ssize_t, } impl<'a> std::iter::IntoIterator for &'a PySet { type Item = &'a PyAny; type IntoIter = PySetIterator<'a>; /// Returns an iterator of values in this set. /// /// # Panics /// /// If PyO3 detects that the set is mutated during iteration, it will panic. fn into_iter(self) -> Self::IntoIter { PySetIterator { set: self, pos: 0, used: unsafe { ffi::PySet_Size(self.as_ptr()) }, } } } impl<'py> Iterator for PySetIterator<'py> { type Item = &'py super::PyAny; /// Advances the iterator and returns the next value. /// /// # Panics /// /// If PyO3 detects that the set is mutated during iteration, it will panic. #[inline] fn next(&mut self) -> Option { unsafe { let len = ffi::PySet_Size(self.set.as_ptr()); assert_eq!(self.used, len, "Set changed size during iteration"); let mut key: *mut ffi::PyObject = std::ptr::null_mut(); let mut hash: ffi::Py_hash_t = 0; if ffi::_PySet_NextEntry(self.set.as_ptr(), &mut self.pos, &mut key, &mut hash) != 0 { // _PySet_NextEntry returns borrowed object; for safety must make owned (see #890) Some(self.set.py().from_owned_ptr(ffi::_Py_NewRef(key))) } else { None } } } #[inline] fn size_hint(&self) -> (usize, Option) { let len = self.len(); (len, Some(len)) } } impl<'py> ExactSizeIterator for PySetIterator<'py> { fn len(&self) -> usize { self.set.len().saturating_sub(self.pos as usize) } } } pub use impl_::*; #[inline] pub(crate) fn new_from_iter( py: Python<'_>, elements: impl IntoIterator, ) -> PyResult> { fn inner(py: Python<'_>, elements: &mut dyn Iterator) -> PyResult> { let set: Py = unsafe { // We create the `Py` pointer because its Drop cleans up the set if user code panics. Py::from_owned_ptr_or_err(py, ffi::PySet_New(std::ptr::null_mut()))? }; let ptr = set.as_ptr(); for obj in elements { err::error_on_minusone(py, unsafe { ffi::PySet_Add(ptr, obj.as_ptr()) })?; } Ok(set) } let mut iter = elements.into_iter().map(|e| e.to_object(py)); inner(py, &mut iter) } #[cfg(test)] mod tests { use super::PySet; use crate::{Python, ToPyObject}; use std::collections::HashSet; #[test] fn test_set_new() { Python::with_gil(|py| { let set = PySet::new(py, &[1]).unwrap(); assert_eq!(1, set.len()); let v = vec![1]; assert!(PySet::new(py, &[v]).is_err()); }); } #[test] fn test_set_empty() { Python::with_gil(|py| { let set = PySet::empty(py).unwrap(); assert_eq!(0, set.len()); }); } #[test] fn test_set_len() { Python::with_gil(|py| { let mut v = HashSet::new(); let ob = v.to_object(py); let set: &PySet = ob.downcast(py).unwrap(); assert_eq!(0, set.len()); v.insert(7); let ob = v.to_object(py); let set2: &PySet = ob.downcast(py).unwrap(); assert_eq!(1, set2.len()); }); } #[test] fn test_set_clear() { Python::with_gil(|py| { let set = PySet::new(py, &[1]).unwrap(); assert_eq!(1, set.len()); set.clear(); assert_eq!(0, set.len()); }); } #[test] fn test_set_contains() { Python::with_gil(|py| { let set = PySet::new(py, &[1]).unwrap(); assert!(set.contains(1).unwrap()); }); } #[test] fn test_set_discard() { Python::with_gil(|py| { let set = PySet::new(py, &[1]).unwrap(); assert!(!set.discard(2).unwrap()); assert_eq!(1, set.len()); assert!(set.discard(1).unwrap()); assert_eq!(0, set.len()); assert!(!set.discard(1).unwrap()); assert!(set.discard(vec![1, 2]).is_err()); }); } #[test] fn test_set_add() { Python::with_gil(|py| { let set = PySet::new(py, &[1, 2]).unwrap(); set.add(1).unwrap(); // Add a dupliated element assert!(set.contains(1).unwrap()); }); } #[test] fn test_set_pop() { Python::with_gil(|py| { let set = PySet::new(py, &[1]).unwrap(); let val = set.pop(); assert!(val.is_some()); let val2 = set.pop(); assert!(val2.is_none()); assert!(py .eval("print('Exception state should not be set.')", None, None) .is_ok()); }); } #[test] fn test_set_iter() { Python::with_gil(|py| { let set = PySet::new(py, &[1]).unwrap(); // iter method for el in set { assert_eq!(1i32, el.extract::<'_, i32>().unwrap()); } // intoiterator iteration for el in set { assert_eq!(1i32, el.extract::<'_, i32>().unwrap()); } }); } #[test] #[should_panic] fn test_set_iter_mutation() { Python::with_gil(|py| { let set = PySet::new(py, &[1, 2, 3, 4, 5]).unwrap(); for _ in set { let _ = set.add(42); } }); } #[test] #[should_panic] fn test_set_iter_mutation_same_len() { Python::with_gil(|py| { let set = PySet::new(py, &[1, 2, 3, 4, 5]).unwrap(); for item in set { let item: i32 = item.extract().unwrap(); let _ = set.del_item(item); let _ = set.add(item + 10); } }); } #[test] fn test_set_iter_size_hint() { Python::with_gil(|py| { let set = PySet::new(py, &[1]).unwrap(); let mut iter = set.iter(); if cfg!(Py_LIMITED_API) { assert_eq!(iter.size_hint(), (0, None)); } else { assert_eq!(iter.size_hint(), (1, Some(1))); iter.next(); assert_eq!(iter.size_hint(), (0, Some(0))); } }); } } pyo3-0.20.2/src/types/slice.rs000064400000000000000000000131741046102023000142050ustar 00000000000000use crate::err::{PyErr, PyResult}; use crate::ffi::{self, Py_ssize_t}; use crate::{PyAny, PyObject, Python, ToPyObject}; use std::os::raw::c_long; /// Represents a Python `slice`. /// /// Only `c_long` indices supported at the moment by the `PySlice` object. #[repr(transparent)] pub struct PySlice(PyAny); pyobject_native_type!( PySlice, ffi::PySliceObject, pyobject_native_static_type_object!(ffi::PySlice_Type), #checkfunction=ffi::PySlice_Check ); /// Return value from [`PySlice::indices`]. #[derive(Debug, Eq, PartialEq)] pub struct PySliceIndices { /// Start of the slice pub start: isize, /// End of the slice pub stop: isize, /// Increment to use when iterating the slice from `start` to `stop`. pub step: isize, /// The length of the slice calculated from the original input sequence. pub slicelength: isize, } impl PySliceIndices { /// Creates a new `PySliceIndices`. pub fn new(start: isize, stop: isize, step: isize) -> PySliceIndices { PySliceIndices { start, stop, step, slicelength: 0, } } } impl PySlice { /// Constructs a new slice with the given elements. pub fn new(py: Python<'_>, start: isize, stop: isize, step: isize) -> &PySlice { unsafe { let ptr = ffi::PySlice_New( ffi::PyLong_FromSsize_t(start), ffi::PyLong_FromSsize_t(stop), ffi::PyLong_FromSsize_t(step), ); py.from_owned_ptr(ptr) } } /// Constructs a new full slice that is equivalent to `::`. pub fn full(py: Python<'_>) -> &PySlice { unsafe { let ptr = ffi::PySlice_New(ffi::Py_None(), ffi::Py_None(), ffi::Py_None()); py.from_owned_ptr(ptr) } } /// Retrieves the start, stop, and step indices from the slice object, /// assuming a sequence of length `length`, and stores the length of the /// slice in its `slicelength` member. #[inline] pub fn indices(&self, length: c_long) -> PyResult { // non-negative Py_ssize_t should always fit into Rust usize unsafe { let mut slicelength: isize = 0; let mut start: isize = 0; let mut stop: isize = 0; let mut step: isize = 0; let r = ffi::PySlice_GetIndicesEx( self.as_ptr(), length as Py_ssize_t, &mut start, &mut stop, &mut step, &mut slicelength, ); if r == 0 { Ok(PySliceIndices { start, stop, step, slicelength, }) } else { Err(PyErr::fetch(self.py())) } } } } impl ToPyObject for PySliceIndices { fn to_object(&self, py: Python<'_>) -> PyObject { PySlice::new(py, self.start, self.stop, self.step).into() } } #[cfg(test)] mod tests { use super::*; #[test] fn test_py_slice_new() { Python::with_gil(|py| { let slice = PySlice::new(py, isize::MIN, isize::MAX, 1); assert_eq!( slice.getattr("start").unwrap().extract::().unwrap(), isize::MIN ); assert_eq!( slice.getattr("stop").unwrap().extract::().unwrap(), isize::MAX ); assert_eq!( slice.getattr("step").unwrap().extract::().unwrap(), 1 ); }); } #[test] fn test_py_slice_full() { Python::with_gil(|py| { let slice = PySlice::full(py); assert!(slice.getattr("start").unwrap().is_none(),); assert!(slice.getattr("stop").unwrap().is_none(),); assert!(slice.getattr("step").unwrap().is_none(),); assert_eq!( slice.indices(0).unwrap(), PySliceIndices { start: 0, stop: 0, step: 1, slicelength: 0, }, ); assert_eq!( slice.indices(42).unwrap(), PySliceIndices { start: 0, stop: 42, step: 1, slicelength: 42, }, ); }); } #[test] fn test_py_slice_indices_new() { let start = 0; let stop = 0; let step = 0; assert_eq!( PySliceIndices::new(start, stop, step), PySliceIndices { start, stop, step, slicelength: 0 } ); let start = 0; let stop = 100; let step = 10; assert_eq!( PySliceIndices::new(start, stop, step), PySliceIndices { start, stop, step, slicelength: 0 } ); let start = 0; let stop = -10; let step = -1; assert_eq!( PySliceIndices::new(start, stop, step), PySliceIndices { start, stop, step, slicelength: 0 } ); let start = 0; let stop = -10; let step = 20; assert_eq!( PySliceIndices::new(start, stop, step), PySliceIndices { start, stop, step, slicelength: 0 } ); } } pyo3-0.20.2/src/types/string.rs000064400000000000000000000425511046102023000144150ustar 00000000000000#[cfg(not(Py_LIMITED_API))] use crate::exceptions::PyUnicodeDecodeError; use crate::types::PyBytes; use crate::{ffi, PyAny, PyResult, Python}; use std::borrow::Cow; use std::os::raw::c_char; use std::str; /// Represents raw data backing a Python `str`. /// /// Python internally stores strings in various representations. This enumeration /// represents those variations. #[cfg(not(Py_LIMITED_API))] #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum PyStringData<'a> { /// UCS1 representation. Ucs1(&'a [u8]), /// UCS2 representation. Ucs2(&'a [u16]), /// UCS4 representation. Ucs4(&'a [u32]), } #[cfg(not(Py_LIMITED_API))] impl<'a> PyStringData<'a> { /// Obtain the raw bytes backing this instance as a [u8] slice. pub fn as_bytes(&self) -> &[u8] { match self { Self::Ucs1(s) => s, Self::Ucs2(s) => unsafe { std::slice::from_raw_parts( s.as_ptr() as *const u8, s.len() * self.value_width_bytes(), ) }, Self::Ucs4(s) => unsafe { std::slice::from_raw_parts( s.as_ptr() as *const u8, s.len() * self.value_width_bytes(), ) }, } } /// Size in bytes of each value/item in the underlying slice. #[inline] pub fn value_width_bytes(&self) -> usize { match self { Self::Ucs1(_) => 1, Self::Ucs2(_) => 2, Self::Ucs4(_) => 4, } } /// Convert the raw data to a Rust string. /// /// For UCS-1 / UTF-8, returns a borrow into the original slice. For UCS-2 and UCS-4, /// returns an owned string. /// /// Returns [PyUnicodeDecodeError] if the string data isn't valid in its purported /// storage format. This should only occur for strings that were created via Python /// C APIs that skip input validation (like `PyUnicode_FromKindAndData`) and should /// never occur for strings that were created from Python code. pub fn to_string(self, py: Python<'_>) -> PyResult> { use std::ffi::CStr; match self { Self::Ucs1(data) => match str::from_utf8(data) { Ok(s) => Ok(Cow::Borrowed(s)), Err(e) => Err(crate::PyErr::from_value(PyUnicodeDecodeError::new_utf8( py, data, e, )?)), }, Self::Ucs2(data) => match String::from_utf16(data) { Ok(s) => Ok(Cow::Owned(s)), Err(e) => { let mut message = e.to_string().as_bytes().to_vec(); message.push(0); Err(crate::PyErr::from_value(PyUnicodeDecodeError::new( py, CStr::from_bytes_with_nul(b"utf-16\0").unwrap(), self.as_bytes(), 0..self.as_bytes().len(), CStr::from_bytes_with_nul(&message).unwrap(), )?)) } }, Self::Ucs4(data) => match data.iter().map(|&c| std::char::from_u32(c)).collect() { Some(s) => Ok(Cow::Owned(s)), None => Err(crate::PyErr::from_value(PyUnicodeDecodeError::new( py, CStr::from_bytes_with_nul(b"utf-32\0").unwrap(), self.as_bytes(), 0..self.as_bytes().len(), CStr::from_bytes_with_nul(b"error converting utf-32\0").unwrap(), )?)), }, } } /// Convert the raw data to a Rust string, possibly with data loss. /// /// Invalid code points will be replaced with `U+FFFD REPLACEMENT CHARACTER`. /// /// Returns a borrow into original data, when possible, or owned data otherwise. /// /// The return value of this function should only disagree with [Self::to_string] /// when that method would error. pub fn to_string_lossy(self) -> Cow<'a, str> { match self { Self::Ucs1(data) => String::from_utf8_lossy(data), Self::Ucs2(data) => Cow::Owned(String::from_utf16_lossy(data)), Self::Ucs4(data) => Cow::Owned( data.iter() .map(|&c| std::char::from_u32(c).unwrap_or('\u{FFFD}')) .collect(), ), } } } /// Represents a Python `string` (a Unicode string object). /// /// This type is immutable. #[repr(transparent)] pub struct PyString(PyAny); pyobject_native_type_core!(PyString, pyobject_native_static_type_object!(ffi::PyUnicode_Type), #checkfunction=ffi::PyUnicode_Check); impl PyString { /// Creates a new Python string object. /// /// Panics if out of memory. pub fn new<'p>(py: Python<'p>, s: &str) -> &'p PyString { let ptr = s.as_ptr() as *const c_char; let len = s.len() as ffi::Py_ssize_t; unsafe { py.from_owned_ptr(ffi::PyUnicode_FromStringAndSize(ptr, len)) } } /// Intern the given string /// /// This will return a reference to the same Python string object if called repeatedly with the same string. /// /// Note that while this is more memory efficient than [`PyString::new`], it unconditionally allocates a /// temporary Python string object and is thereby slower than [`PyString::new`]. /// /// Panics if out of memory. pub fn intern<'p>(py: Python<'p>, s: &str) -> &'p PyString { let ptr = s.as_ptr() as *const c_char; let len = s.len() as ffi::Py_ssize_t; unsafe { let mut ob = ffi::PyUnicode_FromStringAndSize(ptr, len); if !ob.is_null() { ffi::PyUnicode_InternInPlace(&mut ob); } py.from_owned_ptr(ob) } } /// Attempts to create a Python string from a Python [bytes-like object]. /// /// [bytes-like object]: (https://docs.python.org/3/glossary.html#term-bytes-like-object). pub fn from_object<'p>(src: &'p PyAny, encoding: &str, errors: &str) -> PyResult<&'p PyString> { unsafe { src.py() .from_owned_ptr_or_err::(ffi::PyUnicode_FromEncodedObject( src.as_ptr(), encoding.as_ptr() as *const c_char, errors.as_ptr() as *const c_char, )) } } /// Gets the Python string as a byte slice. /// /// Returns a `UnicodeEncodeError` if the input is not valid unicode /// (containing unpaired surrogates). #[inline] pub fn to_str(&self) -> PyResult<&str> { let utf8_slice = { cfg_if::cfg_if! { if #[cfg(any(Py_3_10, not(Py_LIMITED_API)))] { // PyUnicode_AsUTF8AndSize only available on limited API starting with 3.10. let mut size: ffi::Py_ssize_t = 0; let data: *const u8 = unsafe { ffi::PyUnicode_AsUTF8AndSize(self.as_ptr(), &mut size).cast() }; if data.is_null() { return Err(crate::PyErr::fetch(self.py())); } else { unsafe { std::slice::from_raw_parts(data, size as usize) } } } else { let bytes = unsafe { self.py().from_owned_ptr_or_err::(ffi::PyUnicode_AsUTF8String(self.as_ptr()))? }; bytes.as_bytes() } } }; Ok(unsafe { std::str::from_utf8_unchecked(utf8_slice) }) } /// Converts the `PyString` into a Rust string. /// /// Unpaired surrogates invalid UTF-8 sequences are /// replaced with `U+FFFD REPLACEMENT CHARACTER`. pub fn to_string_lossy(&self) -> Cow<'_, str> { match self.to_str() { Ok(s) => Cow::Borrowed(s), Err(_) => { let bytes = unsafe { self.py() .from_owned_ptr::(ffi::PyUnicode_AsEncodedString( self.as_ptr(), b"utf-8\0" as *const _ as _, b"surrogatepass\0" as *const _ as _, )) }; String::from_utf8_lossy(bytes.as_bytes()) } } } /// Obtains the raw data backing the Python string. /// /// If the Python string object was created through legacy APIs, its internal storage format /// will be canonicalized before data is returned. /// /// # Safety /// /// This function implementation relies on manually decoding a C bitfield. In practice, this /// works well on common little-endian architectures such as x86_64, where the bitfield has a /// common representation (even if it is not part of the C spec). The PyO3 CI tests this API on /// x86_64 platforms. /// /// By using this API, you accept responsibility for testing that PyStringData behaves as /// expected on the targets where you plan to distribute your software. #[cfg(not(Py_LIMITED_API))] pub unsafe fn data(&self) -> PyResult> { let ptr = self.as_ptr(); #[cfg(not(Py_3_12))] #[allow(deprecated)] { let ready = ffi::PyUnicode_READY(ptr); if ready != 0 { // Exception was created on failure. return Err(crate::PyErr::fetch(self.py())); } } // The string should be in its canonical form after calling `PyUnicode_READY()`. // And non-canonical form not possible after Python 3.12. So it should be safe // to call these APIs. let length = ffi::PyUnicode_GET_LENGTH(ptr) as usize; let raw_data = ffi::PyUnicode_DATA(ptr); let kind = ffi::PyUnicode_KIND(ptr); match kind { ffi::PyUnicode_1BYTE_KIND => Ok(PyStringData::Ucs1(std::slice::from_raw_parts( raw_data as *const u8, length, ))), ffi::PyUnicode_2BYTE_KIND => Ok(PyStringData::Ucs2(std::slice::from_raw_parts( raw_data as *const u16, length, ))), ffi::PyUnicode_4BYTE_KIND => Ok(PyStringData::Ucs4(std::slice::from_raw_parts( raw_data as *const u32, length, ))), _ => unreachable!(), } } } #[cfg(test)] mod tests { use super::*; use crate::Python; use crate::{PyObject, ToPyObject}; #[cfg(not(Py_LIMITED_API))] use std::borrow::Cow; #[test] fn test_to_str_ascii() { Python::with_gil(|py| { let s = "ascii 🐈"; let obj: PyObject = PyString::new(py, s).into(); let py_string: &PyString = obj.downcast(py).unwrap(); assert_eq!(s, py_string.to_str().unwrap()); }) } #[test] fn test_to_str_surrogate() { Python::with_gil(|py| { let obj: PyObject = py.eval(r"'\ud800'", None, None).unwrap().into(); let py_string: &PyString = obj.downcast(py).unwrap(); assert!(py_string.to_str().is_err()); }) } #[test] fn test_to_str_unicode() { Python::with_gil(|py| { let s = "哈哈🐈"; let obj: PyObject = PyString::new(py, s).into(); let py_string: &PyString = obj.downcast(py).unwrap(); assert_eq!(s, py_string.to_str().unwrap()); }) } #[test] fn test_to_string_lossy() { Python::with_gil(|py| { let obj: PyObject = py .eval(r"'🐈 Hello \ud800World'", None, None) .unwrap() .into(); let py_string: &PyString = obj.downcast(py).unwrap(); assert_eq!(py_string.to_string_lossy(), "🐈 Hello ���World"); }) } #[test] fn test_debug_string() { Python::with_gil(|py| { let v = "Hello\n".to_object(py); let s: &PyString = v.downcast(py).unwrap(); assert_eq!(format!("{:?}", s), "'Hello\\n'"); }) } #[test] fn test_display_string() { Python::with_gil(|py| { let v = "Hello\n".to_object(py); let s: &PyString = v.downcast(py).unwrap(); assert_eq!(format!("{}", s), "Hello\n"); }) } #[test] #[cfg(not(Py_LIMITED_API))] fn test_string_data_ucs1() { Python::with_gil(|py| { let s = PyString::new(py, "hello, world"); let data = unsafe { s.data().unwrap() }; assert_eq!(data, PyStringData::Ucs1(b"hello, world")); assert_eq!(data.to_string(py).unwrap(), Cow::Borrowed("hello, world")); assert_eq!(data.to_string_lossy(), Cow::Borrowed("hello, world")); }) } #[test] #[cfg(not(Py_LIMITED_API))] fn test_string_data_ucs1_invalid() { Python::with_gil(|py| { // 0xfe is not allowed in UTF-8. let buffer = b"f\xfe\0"; let ptr = unsafe { crate::ffi::PyUnicode_FromKindAndData( crate::ffi::PyUnicode_1BYTE_KIND as _, buffer.as_ptr() as *const _, 2, ) }; assert!(!ptr.is_null()); let s: &PyString = unsafe { py.from_owned_ptr(ptr) }; let data = unsafe { s.data().unwrap() }; assert_eq!(data, PyStringData::Ucs1(b"f\xfe")); let err = data.to_string(py).unwrap_err(); assert!(err.get_type(py).is(py.get_type::())); assert!(err .to_string() .contains("'utf-8' codec can't decode byte 0xfe in position 1")); assert_eq!(data.to_string_lossy(), Cow::Borrowed("f�")); }); } #[test] #[cfg(not(Py_LIMITED_API))] fn test_string_data_ucs2() { Python::with_gil(|py| { let s = py.eval("'foo\\ud800'", None, None).unwrap(); let py_string = s.downcast::().unwrap(); let data = unsafe { py_string.data().unwrap() }; assert_eq!(data, PyStringData::Ucs2(&[102, 111, 111, 0xd800])); assert_eq!( data.to_string_lossy(), Cow::Owned::("foo�".to_string()) ); }) } #[test] #[cfg(all(not(Py_LIMITED_API), target_endian = "little"))] fn test_string_data_ucs2_invalid() { Python::with_gil(|py| { // U+FF22 (valid) & U+d800 (never valid) let buffer = b"\x22\xff\x00\xd8\x00\x00"; let ptr = unsafe { crate::ffi::PyUnicode_FromKindAndData( crate::ffi::PyUnicode_2BYTE_KIND as _, buffer.as_ptr() as *const _, 2, ) }; assert!(!ptr.is_null()); let s: &PyString = unsafe { py.from_owned_ptr(ptr) }; let data = unsafe { s.data().unwrap() }; assert_eq!(data, PyStringData::Ucs2(&[0xff22, 0xd800])); let err = data.to_string(py).unwrap_err(); assert!(err.get_type(py).is(py.get_type::())); assert!(err .to_string() .contains("'utf-16' codec can't decode bytes in position 0-3")); assert_eq!(data.to_string_lossy(), Cow::Owned::("B�".into())); }); } #[test] #[cfg(not(Py_LIMITED_API))] fn test_string_data_ucs4() { Python::with_gil(|py| { let s = "哈哈🐈"; let py_string = PyString::new(py, s); let data = unsafe { py_string.data().unwrap() }; assert_eq!(data, PyStringData::Ucs4(&[21704, 21704, 128008])); assert_eq!(data.to_string_lossy(), Cow::Owned::(s.to_string())); }) } #[test] #[cfg(all(not(Py_LIMITED_API), target_endian = "little"))] fn test_string_data_ucs4_invalid() { Python::with_gil(|py| { // U+20000 (valid) & U+d800 (never valid) let buffer = b"\x00\x00\x02\x00\x00\xd8\x00\x00\x00\x00\x00\x00"; let ptr = unsafe { crate::ffi::PyUnicode_FromKindAndData( crate::ffi::PyUnicode_4BYTE_KIND as _, buffer.as_ptr() as *const _, 2, ) }; assert!(!ptr.is_null()); let s: &PyString = unsafe { py.from_owned_ptr(ptr) }; let data = unsafe { s.data().unwrap() }; assert_eq!(data, PyStringData::Ucs4(&[0x20000, 0xd800])); let err = data.to_string(py).unwrap_err(); assert!(err.get_type(py).is(py.get_type::())); assert!(err .to_string() .contains("'utf-32' codec can't decode bytes in position 0-7")); assert_eq!(data.to_string_lossy(), Cow::Owned::("𠀀�".into())); }); } #[test] fn test_intern_string() { Python::with_gil(|py| { let py_string1 = PyString::intern(py, "foo"); assert_eq!(py_string1.to_str().unwrap(), "foo"); let py_string2 = PyString::intern(py, "foo"); assert_eq!(py_string2.to_str().unwrap(), "foo"); assert_eq!(py_string1.as_ptr(), py_string2.as_ptr()); let py_string3 = PyString::intern(py, "bar"); assert_eq!(py_string3.to_str().unwrap(), "bar"); assert_ne!(py_string1.as_ptr(), py_string3.as_ptr()); }); } } pyo3-0.20.2/src/types/traceback.rs000064400000000000000000000074201046102023000150220ustar 00000000000000use crate::err::{error_on_minusone, PyResult}; use crate::ffi; use crate::types::PyString; use crate::PyAny; /// Represents a Python traceback. #[repr(transparent)] pub struct PyTraceback(PyAny); pyobject_native_type_core!( PyTraceback, pyobject_native_static_type_object!(ffi::PyTraceBack_Type), #checkfunction=ffi::PyTraceBack_Check ); impl PyTraceback { /// Formats the traceback as a string. /// /// This does not include the exception type and value. The exception type and value can be /// formatted using the `Display` implementation for `PyErr`. /// /// # Example /// /// The following code formats a Python traceback and exception pair from Rust: /// /// ```rust /// # use pyo3::{Python, PyResult}; /// # let result: PyResult<()> = /// Python::with_gil(|py| { /// let err = py /// .run("raise Exception('banana')", None, None) /// .expect_err("raise will create a Python error"); /// /// let traceback = err.traceback(py).expect("raised exception will have a traceback"); /// assert_eq!( /// format!("{}{}", traceback.format()?, err), /// "\ /// Traceback (most recent call last): /// File \"\", line 1, in /// Exception: banana\ /// " /// ); /// Ok(()) /// }) /// # ; /// # result.expect("example failed"); /// ``` pub fn format(&self) -> PyResult { let py = self.py(); let string_io = py .import(intern!(py, "io"))? .getattr(intern!(py, "StringIO"))? .call0()?; let result = unsafe { ffi::PyTraceBack_Print(self.as_ptr(), string_io.as_ptr()) }; error_on_minusone(py, result)?; let formatted = string_io .getattr(intern!(py, "getvalue"))? .call0()? .downcast::()? .to_str()? .to_owned(); Ok(formatted) } } #[cfg(test)] mod tests { use crate::{prelude::*, types::PyDict}; #[test] fn format_traceback() { Python::with_gil(|py| { let err = py .run("raise Exception('banana')", None, None) .expect_err("raising should have given us an error"); assert_eq!( err.traceback(py).unwrap().format().unwrap(), "Traceback (most recent call last):\n File \"\", line 1, in \n" ); }) } #[test] fn test_err_from_value() { Python::with_gil(|py| { let locals = PyDict::new(py); // Produce an error from python so that it has a traceback py.run( r" try: raise ValueError('raised exception') except Exception as e: err = e ", None, Some(locals), ) .unwrap(); let err = PyErr::from_value(locals.get_item("err").unwrap().unwrap()); let traceback = err.value(py).getattr("__traceback__").unwrap(); assert!(err.traceback(py).unwrap().is(traceback)); }) } #[test] fn test_err_into_py() { Python::with_gil(|py| { let locals = PyDict::new(py); // Produce an error from python so that it has a traceback py.run( r" def f(): raise ValueError('raised exception') ", None, Some(locals), ) .unwrap(); let f = locals.get_item("f").unwrap().unwrap(); let err = f.call0().unwrap_err(); let traceback = err.traceback(py).unwrap(); let err_object = err.clone_ref(py).into_py(py).into_ref(py); assert!(err_object.getattr("__traceback__").unwrap().is(traceback)); }) } } pyo3-0.20.2/src/types/tuple.rs000064400000000000000000000731421046102023000142400ustar 00000000000000use std::convert::TryInto; use std::iter::FusedIterator; use crate::ffi::{self, Py_ssize_t}; #[cfg(feature = "experimental-inspect")] use crate::inspect::types::TypeInfo; use crate::internal_tricks::get_ssize_index; use crate::types::PyList; use crate::types::PySequence; use crate::{ exceptions, FromPyObject, IntoPy, Py, PyAny, PyErr, PyObject, PyResult, Python, ToPyObject, }; #[inline] #[track_caller] fn new_from_iter( py: Python<'_>, elements: &mut dyn ExactSizeIterator, ) -> Py { unsafe { // PyTuple_New checks for overflow but has a bad error message, so we check ourselves let len: Py_ssize_t = elements .len() .try_into() .expect("out of range integral type conversion attempted on `elements.len()`"); let ptr = ffi::PyTuple_New(len); // - Panics if the ptr is null // - Cleans up the tuple if `convert` or the asserts panic let tup: Py = Py::from_owned_ptr(py, ptr); let mut counter: Py_ssize_t = 0; for obj in elements.take(len as usize) { #[cfg(not(any(Py_LIMITED_API, PyPy)))] ffi::PyTuple_SET_ITEM(ptr, counter, obj.into_ptr()); #[cfg(any(Py_LIMITED_API, PyPy))] ffi::PyTuple_SetItem(ptr, counter, obj.into_ptr()); counter += 1; } assert!(elements.next().is_none(), "Attempted to create PyTuple but `elements` was larger than reported by its `ExactSizeIterator` implementation."); assert_eq!(len, counter, "Attempted to create PyTuple but `elements` was smaller than reported by its `ExactSizeIterator` implementation."); tup } } /// Represents a Python `tuple` object. /// /// This type is immutable. #[repr(transparent)] pub struct PyTuple(PyAny); pyobject_native_type_core!(PyTuple, pyobject_native_static_type_object!(ffi::PyTuple_Type), #checkfunction=ffi::PyTuple_Check); impl PyTuple { /// Constructs a new tuple with the given elements. /// /// If you want to create a [`PyTuple`] with elements of different or unknown types, or from an /// iterable that doesn't implement [`ExactSizeIterator`], create a Rust tuple with the given /// elements and convert it at once using `into_py`. /// /// # Examples /// /// ```rust /// use pyo3::prelude::*; /// use pyo3::types::PyTuple; /// /// # fn main() { /// Python::with_gil(|py| { /// let elements: Vec = vec![0, 1, 2, 3, 4, 5]; /// let tuple: &PyTuple = PyTuple::new(py, elements); /// assert_eq!(format!("{:?}", tuple), "(0, 1, 2, 3, 4, 5)"); /// }); /// # } /// ``` /// /// # Panics /// /// This function will panic if `element`'s [`ExactSizeIterator`] implementation is incorrect. /// All standard library structures implement this trait correctly, if they do, so calling this /// function using [`Vec`]`` or `&[T]` will always succeed. #[track_caller] pub fn new( py: Python<'_>, elements: impl IntoIterator, ) -> &PyTuple where T: ToPyObject, U: ExactSizeIterator, { let mut elements = elements.into_iter().map(|e| e.to_object(py)); let tup = new_from_iter(py, &mut elements); tup.into_ref(py) } /// Constructs an empty tuple (on the Python side, a singleton object). pub fn empty(py: Python<'_>) -> &PyTuple { unsafe { py.from_owned_ptr(ffi::PyTuple_New(0)) } } /// Gets the length of the tuple. pub fn len(&self) -> usize { unsafe { #[cfg(not(any(Py_LIMITED_API, PyPy)))] let size = ffi::PyTuple_GET_SIZE(self.as_ptr()); #[cfg(any(Py_LIMITED_API, PyPy))] let size = ffi::PyTuple_Size(self.as_ptr()); // non-negative Py_ssize_t should always fit into Rust uint size as usize } } /// Checks if the tuple is empty. pub fn is_empty(&self) -> bool { self.len() == 0 } /// Returns `self` cast as a `PySequence`. pub fn as_sequence(&self) -> &PySequence { unsafe { self.downcast_unchecked() } } /// Takes the slice `self[low:high]` and returns it as a new tuple. /// /// Indices must be nonnegative, and out-of-range indices are clipped to /// `self.len()`. pub fn get_slice(&self, low: usize, high: usize) -> &PyTuple { unsafe { self.py().from_owned_ptr(ffi::PyTuple_GetSlice( self.as_ptr(), get_ssize_index(low), get_ssize_index(high), )) } } /// Gets the tuple item at the specified index. /// # Example /// ``` /// use pyo3::{prelude::*, types::PyTuple}; /// /// # fn main() -> PyResult<()> { /// Python::with_gil(|py| -> PyResult<()> { /// let ob = (1, 2, 3).to_object(py); /// let tuple: &PyTuple = ob.downcast(py).unwrap(); /// let obj = tuple.get_item(0); /// assert_eq!(obj.unwrap().extract::().unwrap(), 1); /// Ok(()) /// }) /// # } /// ``` pub fn get_item(&self, index: usize) -> PyResult<&PyAny> { unsafe { let item = ffi::PyTuple_GetItem(self.as_ptr(), index as Py_ssize_t); self.py().from_borrowed_ptr_or_err(item) } } /// Gets the tuple item at the specified index. Undefined behavior on bad index. Use with caution. /// /// # Safety /// /// Caller must verify that the index is within the bounds of the tuple. #[cfg(not(any(Py_LIMITED_API, PyPy)))] pub unsafe fn get_item_unchecked(&self, index: usize) -> &PyAny { let item = ffi::PyTuple_GET_ITEM(self.as_ptr(), index as Py_ssize_t); self.py().from_borrowed_ptr(item) } /// Returns `self` as a slice of objects. #[cfg(not(Py_LIMITED_API))] pub fn as_slice(&self) -> &[&PyAny] { // This is safe because &PyAny has the same memory layout as *mut ffi::PyObject, // and because tuples are immutable. unsafe { let ptr = self.as_ptr() as *mut ffi::PyTupleObject; let slice = std::slice::from_raw_parts((*ptr).ob_item.as_ptr(), self.len()); &*(slice as *const [*mut ffi::PyObject] as *const [&PyAny]) } } /// Determines if self contains `value`. /// /// This is equivalent to the Python expression `value in self`. #[inline] pub fn contains(&self, value: V) -> PyResult where V: ToPyObject, { self.as_sequence().contains(value) } /// Returns the first index `i` for which `self[i] == value`. /// /// This is equivalent to the Python expression `self.index(value)`. #[inline] pub fn index(&self, value: V) -> PyResult where V: ToPyObject, { self.as_sequence().index(value) } /// Returns an iterator over the tuple items. pub fn iter(&self) -> PyTupleIterator<'_> { PyTupleIterator { tuple: self, index: 0, length: self.len(), } } /// Return a new list containing the contents of this tuple; equivalent to the Python expression `list(tuple)`. /// /// This method is equivalent to `self.as_sequence().to_list()` and faster than `PyList::new(py, self)`. pub fn to_list(&self) -> &PyList { self.as_sequence() .to_list() .expect("failed to convert tuple to list") } } index_impls!(PyTuple, "tuple", PyTuple::len, PyTuple::get_slice); /// Used by `PyTuple::iter()`. pub struct PyTupleIterator<'a> { tuple: &'a PyTuple, index: usize, length: usize, } impl<'a> PyTupleIterator<'a> { unsafe fn get_item(&self, index: usize) -> &'a PyAny { #[cfg(any(Py_LIMITED_API, PyPy))] let item = self.tuple.get_item(index).expect("tuple.get failed"); #[cfg(not(any(Py_LIMITED_API, PyPy)))] let item = self.tuple.get_item_unchecked(index); item } } impl<'a> Iterator for PyTupleIterator<'a> { type Item = &'a PyAny; #[inline] fn next(&mut self) -> Option { if self.index < self.length { let item = unsafe { self.get_item(self.index) }; self.index += 1; Some(item) } else { None } } #[inline] fn size_hint(&self) -> (usize, Option) { let len = self.len(); (len, Some(len)) } } impl<'a> DoubleEndedIterator for PyTupleIterator<'a> { #[inline] fn next_back(&mut self) -> Option { if self.index < self.length { let item = unsafe { self.get_item(self.length - 1) }; self.length -= 1; Some(item) } else { None } } } impl<'a> ExactSizeIterator for PyTupleIterator<'a> { fn len(&self) -> usize { self.length.saturating_sub(self.index) } } impl FusedIterator for PyTupleIterator<'_> {} impl<'a> IntoIterator for &'a PyTuple { type Item = &'a PyAny; type IntoIter = PyTupleIterator<'a>; fn into_iter(self) -> Self::IntoIter { self.iter() } } #[cold] fn wrong_tuple_length(t: &PyTuple, expected_length: usize) -> PyErr { let msg = format!( "expected tuple of length {}, but got tuple of length {}", expected_length, t.len() ); exceptions::PyValueError::new_err(msg) } macro_rules! tuple_conversion ({$length:expr,$(($refN:ident, $n:tt, $T:ident)),+} => { impl <$($T: ToPyObject),+> ToPyObject for ($($T,)+) { fn to_object(&self, py: Python<'_>) -> PyObject { array_into_tuple(py, [$(self.$n.to_object(py)),+]).into() } } impl <$($T: IntoPy),+> IntoPy for ($($T,)+) { fn into_py(self, py: Python<'_>) -> PyObject { array_into_tuple(py, [$(self.$n.into_py(py)),+]).into() } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::Tuple(Some(vec![$( $T::type_output() ),+])) } } impl <$($T: IntoPy),+> IntoPy> for ($($T,)+) { fn into_py(self, py: Python<'_>) -> Py { array_into_tuple(py, [$(self.$n.into_py(py)),+]) } #[cfg(feature = "experimental-inspect")] fn type_output() -> TypeInfo { TypeInfo::Tuple(Some(vec![$( $T::type_output() ),+])) } } impl<'s, $($T: FromPyObject<'s>),+> FromPyObject<'s> for ($($T,)+) { fn extract(obj: &'s PyAny) -> PyResult { let t: &PyTuple = obj.downcast()?; if t.len() == $length { #[cfg(any(Py_LIMITED_API, PyPy))] return Ok(($(t.get_item($n)?.extract::<$T>()?,)+)); #[cfg(not(any(Py_LIMITED_API, PyPy)))] unsafe {return Ok(($(t.get_item_unchecked($n).extract::<$T>()?,)+));} } else { Err(wrong_tuple_length(t, $length)) } } #[cfg(feature = "experimental-inspect")] fn type_input() -> TypeInfo { TypeInfo::Tuple(Some(vec![$( $T::type_input() ),+])) } } }); fn array_into_tuple(py: Python<'_>, array: [PyObject; N]) -> Py { unsafe { let ptr = ffi::PyTuple_New(N.try_into().expect("0 < N <= 12")); let tup = Py::from_owned_ptr(py, ptr); for (index, obj) in array.into_iter().enumerate() { #[cfg(not(any(Py_LIMITED_API, PyPy)))] ffi::PyTuple_SET_ITEM(ptr, index as ffi::Py_ssize_t, obj.into_ptr()); #[cfg(any(Py_LIMITED_API, PyPy))] ffi::PyTuple_SetItem(ptr, index as ffi::Py_ssize_t, obj.into_ptr()); } tup } } tuple_conversion!(1, (ref0, 0, T0)); tuple_conversion!(2, (ref0, 0, T0), (ref1, 1, T1)); tuple_conversion!(3, (ref0, 0, T0), (ref1, 1, T1), (ref2, 2, T2)); tuple_conversion!( 4, (ref0, 0, T0), (ref1, 1, T1), (ref2, 2, T2), (ref3, 3, T3) ); tuple_conversion!( 5, (ref0, 0, T0), (ref1, 1, T1), (ref2, 2, T2), (ref3, 3, T3), (ref4, 4, T4) ); tuple_conversion!( 6, (ref0, 0, T0), (ref1, 1, T1), (ref2, 2, T2), (ref3, 3, T3), (ref4, 4, T4), (ref5, 5, T5) ); tuple_conversion!( 7, (ref0, 0, T0), (ref1, 1, T1), (ref2, 2, T2), (ref3, 3, T3), (ref4, 4, T4), (ref5, 5, T5), (ref6, 6, T6) ); tuple_conversion!( 8, (ref0, 0, T0), (ref1, 1, T1), (ref2, 2, T2), (ref3, 3, T3), (ref4, 4, T4), (ref5, 5, T5), (ref6, 6, T6), (ref7, 7, T7) ); tuple_conversion!( 9, (ref0, 0, T0), (ref1, 1, T1), (ref2, 2, T2), (ref3, 3, T3), (ref4, 4, T4), (ref5, 5, T5), (ref6, 6, T6), (ref7, 7, T7), (ref8, 8, T8) ); tuple_conversion!( 10, (ref0, 0, T0), (ref1, 1, T1), (ref2, 2, T2), (ref3, 3, T3), (ref4, 4, T4), (ref5, 5, T5), (ref6, 6, T6), (ref7, 7, T7), (ref8, 8, T8), (ref9, 9, T9) ); tuple_conversion!( 11, (ref0, 0, T0), (ref1, 1, T1), (ref2, 2, T2), (ref3, 3, T3), (ref4, 4, T4), (ref5, 5, T5), (ref6, 6, T6), (ref7, 7, T7), (ref8, 8, T8), (ref9, 9, T9), (ref10, 10, T10) ); tuple_conversion!( 12, (ref0, 0, T0), (ref1, 1, T1), (ref2, 2, T2), (ref3, 3, T3), (ref4, 4, T4), (ref5, 5, T5), (ref6, 6, T6), (ref7, 7, T7), (ref8, 8, T8), (ref9, 9, T9), (ref10, 10, T10), (ref11, 11, T11) ); #[cfg(test)] mod tests { use crate::types::{PyAny, PyList, PyTuple}; use crate::{Python, ToPyObject}; use std::collections::HashSet; #[test] fn test_new() { Python::with_gil(|py| { let ob = PyTuple::new(py, [1, 2, 3]); assert_eq!(3, ob.len()); let ob: &PyAny = ob.into(); assert_eq!((1, 2, 3), ob.extract().unwrap()); let mut map = HashSet::new(); map.insert(1); map.insert(2); PyTuple::new(py, map); }); } #[test] fn test_len() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); assert_eq!(3, tuple.len()); let ob: &PyAny = tuple.into(); assert_eq!((1, 2, 3), ob.extract().unwrap()); }); } #[test] fn test_slice() { Python::with_gil(|py| { let tup = PyTuple::new(py, [2, 3, 5, 7]); let slice = tup.get_slice(1, 3); assert_eq!(2, slice.len()); let slice = tup.get_slice(1, 7); assert_eq!(3, slice.len()); }); } #[test] fn test_iter() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); assert_eq!(3, tuple.len()); let mut iter = tuple.iter(); assert_eq!(iter.size_hint(), (3, Some(3))); assert_eq!(1_i32, iter.next().unwrap().extract::<'_, i32>().unwrap()); assert_eq!(iter.size_hint(), (2, Some(2))); assert_eq!(2_i32, iter.next().unwrap().extract::<'_, i32>().unwrap()); assert_eq!(iter.size_hint(), (1, Some(1))); assert_eq!(3_i32, iter.next().unwrap().extract::<'_, i32>().unwrap()); assert_eq!(iter.size_hint(), (0, Some(0))); assert!(iter.next().is_none()); assert!(iter.next().is_none()); }); } #[test] fn test_iter_rev() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); assert_eq!(3, tuple.len()); let mut iter = tuple.iter().rev(); assert_eq!(iter.size_hint(), (3, Some(3))); assert_eq!(3_i32, iter.next().unwrap().extract::<'_, i32>().unwrap()); assert_eq!(iter.size_hint(), (2, Some(2))); assert_eq!(2_i32, iter.next().unwrap().extract::<'_, i32>().unwrap()); assert_eq!(iter.size_hint(), (1, Some(1))); assert_eq!(1_i32, iter.next().unwrap().extract::<'_, i32>().unwrap()); assert_eq!(iter.size_hint(), (0, Some(0))); assert!(iter.next().is_none()); assert!(iter.next().is_none()); }); } #[test] fn test_into_iter() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); assert_eq!(3, tuple.len()); for (i, item) in tuple.iter().enumerate() { assert_eq!(i + 1, item.extract::<'_, usize>().unwrap()); } }); } #[test] #[cfg(not(Py_LIMITED_API))] fn test_as_slice() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); let slice = tuple.as_slice(); assert_eq!(3, slice.len()); assert_eq!(1_i32, slice[0].extract::<'_, i32>().unwrap()); assert_eq!(2_i32, slice[1].extract::<'_, i32>().unwrap()); assert_eq!(3_i32, slice[2].extract::<'_, i32>().unwrap()); }); } #[test] fn test_tuple_lengths_up_to_12() { Python::with_gil(|py| { let t0 = (0,).to_object(py); let t1 = (0, 1).to_object(py); let t2 = (0, 1, 2).to_object(py); let t3 = (0, 1, 2, 3).to_object(py); let t4 = (0, 1, 2, 3, 4).to_object(py); let t5 = (0, 1, 2, 3, 4, 5).to_object(py); let t6 = (0, 1, 2, 3, 4, 5, 6).to_object(py); let t7 = (0, 1, 2, 3, 4, 5, 6, 7).to_object(py); let t8 = (0, 1, 2, 3, 4, 5, 6, 7, 8).to_object(py); let t9 = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9).to_object(py); let t10 = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10).to_object(py); let t11 = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11).to_object(py); assert_eq!(t0.extract::<(i32,)>(py).unwrap(), (0,)); assert_eq!(t1.extract::<(i32, i32)>(py).unwrap(), (0, 1,)); assert_eq!(t2.extract::<(i32, i32, i32)>(py).unwrap(), (0, 1, 2,)); assert_eq!( t3.extract::<(i32, i32, i32, i32,)>(py).unwrap(), (0, 1, 2, 3,) ); assert_eq!( t4.extract::<(i32, i32, i32, i32, i32,)>(py).unwrap(), (0, 1, 2, 3, 4,) ); assert_eq!( t5.extract::<(i32, i32, i32, i32, i32, i32,)>(py).unwrap(), (0, 1, 2, 3, 4, 5,) ); assert_eq!( t6.extract::<(i32, i32, i32, i32, i32, i32, i32,)>(py) .unwrap(), (0, 1, 2, 3, 4, 5, 6,) ); assert_eq!( t7.extract::<(i32, i32, i32, i32, i32, i32, i32, i32,)>(py) .unwrap(), (0, 1, 2, 3, 4, 5, 6, 7,) ); assert_eq!( t8.extract::<(i32, i32, i32, i32, i32, i32, i32, i32, i32,)>(py) .unwrap(), (0, 1, 2, 3, 4, 5, 6, 7, 8,) ); assert_eq!( t9.extract::<(i32, i32, i32, i32, i32, i32, i32, i32, i32, i32,)>(py) .unwrap(), (0, 1, 2, 3, 4, 5, 6, 7, 8, 9,) ); assert_eq!( t10.extract::<(i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32,)>(py) .unwrap(), (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,) ); assert_eq!( t11.extract::<(i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32,)>(py) .unwrap(), (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,) ); }) } #[test] fn test_tuple_get_item_invalid_index() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); let obj = tuple.get_item(5); assert!(obj.is_err()); assert_eq!( obj.unwrap_err().to_string(), "IndexError: tuple index out of range" ); }); } #[test] fn test_tuple_get_item_sanity() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); let obj = tuple.get_item(0); assert_eq!(obj.unwrap().extract::().unwrap(), 1); }); } #[cfg(not(any(Py_LIMITED_API, PyPy)))] #[test] fn test_tuple_get_item_unchecked_sanity() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); let obj = unsafe { tuple.get_item_unchecked(0) }; assert_eq!(obj.extract::().unwrap(), 1); }); } #[test] fn test_tuple_index_trait() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); assert_eq!(1, tuple[0].extract::().unwrap()); assert_eq!(2, tuple[1].extract::().unwrap()); assert_eq!(3, tuple[2].extract::().unwrap()); }); } #[test] #[should_panic] fn test_tuple_index_trait_panic() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); let _ = &tuple[7]; }); } #[test] fn test_tuple_index_trait_ranges() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); assert_eq!(vec![2, 3], tuple[1..3].extract::>().unwrap()); assert_eq!( Vec::::new(), tuple[3..3].extract::>().unwrap() ); assert_eq!(vec![2, 3], tuple[1..].extract::>().unwrap()); assert_eq!(Vec::::new(), tuple[3..].extract::>().unwrap()); assert_eq!(vec![1, 2, 3], tuple[..].extract::>().unwrap()); assert_eq!(vec![2, 3], tuple[1..=2].extract::>().unwrap()); assert_eq!(vec![1, 2], tuple[..2].extract::>().unwrap()); assert_eq!(vec![1, 2], tuple[..=1].extract::>().unwrap()); }) } #[test] #[should_panic = "range start index 5 out of range for tuple of length 3"] fn test_tuple_index_trait_range_panic_start() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); tuple[5..10].extract::>().unwrap(); }) } #[test] #[should_panic = "range end index 10 out of range for tuple of length 3"] fn test_tuple_index_trait_range_panic_end() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); tuple[1..10].extract::>().unwrap(); }) } #[test] #[should_panic = "slice index starts at 2 but ends at 1"] fn test_tuple_index_trait_range_panic_wrong_order() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); #[allow(clippy::reversed_empty_ranges)] tuple[2..1].extract::>().unwrap(); }) } #[test] #[should_panic = "range start index 8 out of range for tuple of length 3"] fn test_tuple_index_trait_range_from_panic() { Python::with_gil(|py| { let ob = (1, 2, 3).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); tuple[8..].extract::>().unwrap(); }) } #[test] fn test_tuple_contains() { Python::with_gil(|py| { let ob = (1, 1, 2, 3, 5, 8).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); assert_eq!(6, tuple.len()); let bad_needle = 7i32.to_object(py); assert!(!tuple.contains(&bad_needle).unwrap()); let good_needle = 8i32.to_object(py); assert!(tuple.contains(&good_needle).unwrap()); let type_coerced_needle = 8f32.to_object(py); assert!(tuple.contains(&type_coerced_needle).unwrap()); }); } #[test] fn test_tuple_index() { Python::with_gil(|py| { let ob = (1, 1, 2, 3, 5, 8).to_object(py); let tuple: &PyTuple = ob.downcast(py).unwrap(); assert_eq!(0, tuple.index(1i32).unwrap()); assert_eq!(2, tuple.index(2i32).unwrap()); assert_eq!(3, tuple.index(3i32).unwrap()); assert_eq!(4, tuple.index(5i32).unwrap()); assert_eq!(5, tuple.index(8i32).unwrap()); assert!(tuple.index(42i32).is_err()); }); } use std::ops::Range; // An iterator that lies about its `ExactSizeIterator` implementation. // See https://github.com/PyO3/pyo3/issues/2118 struct FaultyIter(Range, usize); impl Iterator for FaultyIter { type Item = usize; fn next(&mut self) -> Option { self.0.next() } } impl ExactSizeIterator for FaultyIter { fn len(&self) -> usize { self.1 } } #[test] #[should_panic( expected = "Attempted to create PyTuple but `elements` was larger than reported by its `ExactSizeIterator` implementation." )] fn too_long_iterator() { Python::with_gil(|py| { let iter = FaultyIter(0..usize::MAX, 73); let _tuple = PyTuple::new(py, iter); }) } #[test] #[should_panic( expected = "Attempted to create PyTuple but `elements` was smaller than reported by its `ExactSizeIterator` implementation." )] fn too_short_iterator() { Python::with_gil(|py| { let iter = FaultyIter(0..35, 73); let _tuple = PyTuple::new(py, iter); }) } #[test] #[should_panic( expected = "out of range integral type conversion attempted on `elements.len()`" )] fn overflowing_size() { Python::with_gil(|py| { let iter = FaultyIter(0..0, usize::MAX); let _tuple = PyTuple::new(py, iter); }) } #[cfg(feature = "macros")] #[test] fn bad_clone_mem_leaks() { use crate::{IntoPy, Py}; use std::sync::atomic::{AtomicUsize, Ordering::SeqCst}; static NEEDS_DESTRUCTING_COUNT: AtomicUsize = AtomicUsize::new(0); #[crate::pyclass] #[pyo3(crate = "crate")] struct Bad(usize); impl Clone for Bad { fn clone(&self) -> Self { // This panic should not lead to a memory leak assert_ne!(self.0, 42); NEEDS_DESTRUCTING_COUNT.fetch_add(1, SeqCst); Bad(self.0) } } impl Drop for Bad { fn drop(&mut self) { NEEDS_DESTRUCTING_COUNT.fetch_sub(1, SeqCst); } } impl ToPyObject for Bad { fn to_object(&self, py: Python<'_>) -> Py { self.to_owned().into_py(py) } } struct FaultyIter(Range, usize); impl Iterator for FaultyIter { type Item = Bad; fn next(&mut self) -> Option { self.0.next().map(|i| { NEEDS_DESTRUCTING_COUNT.fetch_add(1, SeqCst); Bad(i) }) } } impl ExactSizeIterator for FaultyIter { fn len(&self) -> usize { self.1 } } Python::with_gil(|py| { std::panic::catch_unwind(|| { let iter = FaultyIter(0..50, 50); let _tuple = PyTuple::new(py, iter); }) .unwrap_err(); }); assert_eq!( NEEDS_DESTRUCTING_COUNT.load(SeqCst), 0, "Some destructors did not run" ); } #[cfg(feature = "macros")] #[test] fn bad_clone_mem_leaks_2() { use crate::{IntoPy, Py}; use std::sync::atomic::{AtomicUsize, Ordering::SeqCst}; static NEEDS_DESTRUCTING_COUNT: AtomicUsize = AtomicUsize::new(0); #[crate::pyclass] #[pyo3(crate = "crate")] struct Bad(usize); impl Clone for Bad { fn clone(&self) -> Self { // This panic should not lead to a memory leak assert_ne!(self.0, 3); NEEDS_DESTRUCTING_COUNT.fetch_add(1, SeqCst); Bad(self.0) } } impl Drop for Bad { fn drop(&mut self) { NEEDS_DESTRUCTING_COUNT.fetch_sub(1, SeqCst); } } impl ToPyObject for Bad { fn to_object(&self, py: Python<'_>) -> Py { self.to_owned().into_py(py) } } let s = (Bad(1), Bad(2), Bad(3), Bad(4)); NEEDS_DESTRUCTING_COUNT.store(4, SeqCst); Python::with_gil(|py| { std::panic::catch_unwind(|| { let _tuple: Py = s.to_object(py); }) .unwrap_err(); }); drop(s); assert_eq!( NEEDS_DESTRUCTING_COUNT.load(SeqCst), 0, "Some destructors did not run" ); } #[test] fn test_tuple_to_list() { Python::with_gil(|py| { let tuple = PyTuple::new(py, vec![1, 2, 3]); let list = tuple.to_list(); let list_expected = PyList::new(py, vec![1, 2, 3]); assert!(list.eq(list_expected).unwrap()); }) } } pyo3-0.20.2/src/types/typeobject.rs000064400000000000000000000046451046102023000152610ustar 00000000000000use crate::err::{self, PyResult}; use crate::{ffi, PyAny, PyTypeInfo, Python}; /// Represents a reference to a Python `type object`. #[repr(transparent)] pub struct PyType(PyAny); pyobject_native_type_core!(PyType, pyobject_native_static_type_object!(ffi::PyType_Type), #checkfunction=ffi::PyType_Check); impl PyType { /// Creates a new type object. #[inline] pub fn new(py: Python<'_>) -> &PyType { T::type_object(py) } /// Retrieves the underlying FFI pointer associated with this Python object. #[inline] pub fn as_type_ptr(&self) -> *mut ffi::PyTypeObject { self.as_ptr() as *mut ffi::PyTypeObject } /// Retrieves the `PyType` instance for the given FFI pointer. /// /// # Safety /// - The pointer must be non-null. /// - The pointer must be valid for the entire of the lifetime for which the reference is used. #[inline] pub unsafe fn from_type_ptr(py: Python<'_>, p: *mut ffi::PyTypeObject) -> &PyType { py.from_borrowed_ptr(p as *mut ffi::PyObject) } /// Gets the name of the `PyType`. pub fn name(&self) -> PyResult<&str> { self.getattr(intern!(self.py(), "__qualname__"))?.extract() } /// Checks whether `self` is a subclass of `other`. /// /// Equivalent to the Python expression `issubclass(self, other)`. pub fn is_subclass(&self, other: &PyAny) -> PyResult { let result = unsafe { ffi::PyObject_IsSubclass(self.as_ptr(), other.as_ptr()) }; err::error_on_minusone(self.py(), result)?; Ok(result == 1) } /// Checks whether `self` is a subclass of type `T`. /// /// Equivalent to the Python expression `issubclass(self, T)`, if the type /// `T` is known at compile time. pub fn is_subclass_of(&self) -> PyResult where T: PyTypeInfo, { self.is_subclass(T::type_object(self.py())) } } #[cfg(test)] mod tests { use crate::types::{PyBool, PyLong}; use crate::Python; #[test] fn test_type_is_subclass() { Python::with_gil(|py| { let bool_type = py.get_type::(); let long_type = py.get_type::(); assert!(bool_type.is_subclass(long_type).unwrap()); }); } #[test] fn test_type_is_subclass_of() { Python::with_gil(|py| { assert!(py.get_type::().is_subclass_of::().unwrap()); }); } } pyo3-0.20.2/src/version.rs000064400000000000000000000115701046102023000134250ustar 00000000000000/// Represents the major, minor, and patch (if any) versions of this interpreter. /// /// This struct is usually created with [`Python::version`]. /// /// # Examples /// /// ```rust /// # use pyo3::Python; /// Python::with_gil(|py| { /// // PyO3 supports Python 3.7 and up. /// assert!(py.version_info() >= (3, 7)); /// assert!(py.version_info() >= (3, 7, 0)); /// }); /// ``` /// /// [`Python::version`]: crate::marker::Python::version #[derive(Debug)] pub struct PythonVersionInfo<'py> { /// Python major version (e.g. `3`). pub major: u8, /// Python minor version (e.g. `11`). pub minor: u8, /// Python patch version (e.g. `0`). pub patch: u8, /// Python version suffix, if applicable (e.g. `a0`). pub suffix: Option<&'py str>, } impl<'py> PythonVersionInfo<'py> { /// Parses a hard-coded Python interpreter version string (e.g. 3.9.0a4+). pub(crate) fn from_str(version_number_str: &'py str) -> Result { fn split_and_parse_number(version_part: &str) -> (u8, Option<&str>) { match version_part.find(|c: char| !c.is_ascii_digit()) { None => (version_part.parse().unwrap(), None), Some(version_part_suffix_start) => { let (version_part, version_part_suffix) = version_part.split_at(version_part_suffix_start); (version_part.parse().unwrap(), Some(version_part_suffix)) } } } let mut parts = version_number_str.split('.'); let major_str = parts.next().ok_or("Python major version missing")?; let minor_str = parts.next().ok_or("Python minor version missing")?; let patch_str = parts.next(); if parts.next().is_some() { return Err("Python version string has too many parts"); }; let major = major_str .parse() .map_err(|_| "Python major version not an integer")?; let (minor, suffix) = split_and_parse_number(minor_str); if suffix.is_some() { assert!(patch_str.is_none()); return Ok(PythonVersionInfo { major, minor, patch: 0, suffix, }); } let (patch, suffix) = patch_str.map(split_and_parse_number).unwrap_or_default(); Ok(PythonVersionInfo { major, minor, patch, suffix, }) } } impl PartialEq<(u8, u8)> for PythonVersionInfo<'_> { fn eq(&self, other: &(u8, u8)) -> bool { self.major == other.0 && self.minor == other.1 } } impl PartialEq<(u8, u8, u8)> for PythonVersionInfo<'_> { fn eq(&self, other: &(u8, u8, u8)) -> bool { self.major == other.0 && self.minor == other.1 && self.patch == other.2 } } impl PartialOrd<(u8, u8)> for PythonVersionInfo<'_> { fn partial_cmp(&self, other: &(u8, u8)) -> Option { (self.major, self.minor).partial_cmp(other) } } impl PartialOrd<(u8, u8, u8)> for PythonVersionInfo<'_> { fn partial_cmp(&self, other: &(u8, u8, u8)) -> Option { (self.major, self.minor, self.patch).partial_cmp(other) } } #[cfg(test)] mod test { use super::*; use crate::Python; #[test] fn test_python_version_info() { Python::with_gil(|py| { let version = py.version_info(); #[cfg(Py_3_7)] assert!(version >= (3, 7)); #[cfg(Py_3_7)] assert!(version >= (3, 7, 0)); #[cfg(Py_3_8)] assert!(version >= (3, 8)); #[cfg(Py_3_8)] assert!(version >= (3, 8, 0)); #[cfg(Py_3_9)] assert!(version >= (3, 9)); #[cfg(Py_3_9)] assert!(version >= (3, 9, 0)); #[cfg(Py_3_10)] assert!(version >= (3, 10)); #[cfg(Py_3_10)] assert!(version >= (3, 10, 0)); #[cfg(Py_3_11)] assert!(version >= (3, 11)); #[cfg(Py_3_11)] assert!(version >= (3, 11, 0)); }); } #[test] fn test_python_version_info_parse() { assert!(PythonVersionInfo::from_str("3.5.0a1").unwrap() >= (3, 5, 0)); assert!(PythonVersionInfo::from_str("3.5+").unwrap() >= (3, 5, 0)); assert!(PythonVersionInfo::from_str("3.5+").unwrap() == (3, 5, 0)); assert!(PythonVersionInfo::from_str("3.5+").unwrap() != (3, 5, 1)); assert!(PythonVersionInfo::from_str("3.5.2a1+").unwrap() < (3, 5, 3)); assert!(PythonVersionInfo::from_str("3.5.2a1+").unwrap() == (3, 5, 2)); assert!(PythonVersionInfo::from_str("3.5.2a1+").unwrap() == (3, 5)); assert!(PythonVersionInfo::from_str("3.5+").unwrap() == (3, 5)); assert!(PythonVersionInfo::from_str("3.5.2a1+").unwrap() < (3, 6)); assert!(PythonVersionInfo::from_str("3.5.2a1+").unwrap() > (3, 4)); } } pyo3-0.20.2/tests/test_anyhow.rs000064400000000000000000000021051046102023000146510ustar 00000000000000#![cfg(feature = "anyhow")] #[test] fn test_anyhow_py_function_ok_result() { use pyo3::{py_run, pyfunction, wrap_pyfunction, Python}; #[pyfunction] #[allow(clippy::unnecessary_wraps)] fn produce_ok_result() -> anyhow::Result { Ok(String::from("OK buddy")) } Python::with_gil(|py| { let func = wrap_pyfunction!(produce_ok_result)(py).unwrap(); py_run!( py, func, r#" func() "# ); }); } #[test] fn test_anyhow_py_function_err_result() { use pyo3::{pyfunction, types::PyDict, wrap_pyfunction, Python}; #[pyfunction] fn produce_err_result() -> anyhow::Result { anyhow::bail!("error time") } Python::with_gil(|py| { let func = wrap_pyfunction!(produce_err_result)(py).unwrap(); let locals = PyDict::new(py); locals.set_item("func", func).unwrap(); py.run( r#" func() "#, None, Some(locals), ) .unwrap_err(); }); } pyo3-0.20.2/tests/test_append_to_inittab.rs000064400000000000000000000012201046102023000170240ustar 00000000000000#![cfg(all(feature = "macros", not(PyPy)))] use pyo3::prelude::*; #[pyfunction] fn foo() -> usize { 123 } #[pymodule] fn module_with_functions(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(foo, m)?).unwrap(); Ok(()) } #[cfg(not(PyPy))] #[test] fn test_module_append_to_inittab() { use pyo3::append_to_inittab; append_to_inittab!(module_with_functions); Python::with_gil(|py| { py.run( r#" import module_with_functions assert module_with_functions.foo() == 123 "#, None, None, ) .map_err(|e| e.display(py)) .unwrap(); }) } pyo3-0.20.2/tests/test_arithmetics.rs000064400000000000000000000522321046102023000156660ustar 00000000000000#![cfg(feature = "macros")] use pyo3::class::basic::CompareOp; use pyo3::prelude::*; use pyo3::py_run; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct UnaryArithmetic { inner: f64, } #[pymethods] impl UnaryArithmetic { #[new] fn new(value: f64) -> Self { UnaryArithmetic { inner: value } } fn __repr__(&self) -> String { format!("UA({})", self.inner) } fn __neg__(&self) -> Self { Self::new(-self.inner) } fn __pos__(&self) -> Self { Self::new(self.inner) } fn __abs__(&self) -> Self { Self::new(self.inner.abs()) } fn __round__(&self, _ndigits: Option) -> Self { Self::new(self.inner.round()) } } #[test] fn unary_arithmetic() { Python::with_gil(|py| { let c = PyCell::new(py, UnaryArithmetic::new(2.7)).unwrap(); py_run!(py, c, "assert repr(-c) == 'UA(-2.7)'"); py_run!(py, c, "assert repr(+c) == 'UA(2.7)'"); py_run!(py, c, "assert repr(abs(c)) == 'UA(2.7)'"); py_run!(py, c, "assert repr(round(c)) == 'UA(3)'"); py_run!(py, c, "assert repr(round(c, 1)) == 'UA(3)'"); }); } #[pyclass] struct Indexable(i32); #[pymethods] impl Indexable { fn __index__(&self) -> i32 { self.0 } fn __int__(&self) -> i32 { self.0 } fn __float__(&self) -> f64 { f64::from(self.0) } fn __invert__(&self) -> Self { Self(!self.0) } } #[test] fn indexable() { Python::with_gil(|py| { let i = PyCell::new(py, Indexable(5)).unwrap(); py_run!(py, i, "assert int(i) == 5"); py_run!(py, i, "assert [0, 1, 2, 3, 4, 5][i] == 5"); py_run!(py, i, "assert float(i) == 5.0"); py_run!(py, i, "assert int(~i) == -6"); }) } #[pyclass] struct InPlaceOperations { value: u32, } #[pymethods] impl InPlaceOperations { fn __repr__(&self) -> String { format!("IPO({:?})", self.value) } fn __iadd__(&mut self, other: u32) { self.value += other; } fn __isub__(&mut self, other: u32) { self.value -= other; } fn __imul__(&mut self, other: u32) { self.value *= other; } fn __ilshift__(&mut self, other: u32) { self.value <<= other; } fn __irshift__(&mut self, other: u32) { self.value >>= other; } fn __iand__(&mut self, other: u32) { self.value &= other; } fn __ixor__(&mut self, other: u32) { self.value ^= other; } fn __ior__(&mut self, other: u32) { self.value |= other; } fn __ipow__(&mut self, other: u32, _modulo: Option) { self.value = self.value.pow(other); } } #[test] fn inplace_operations() { Python::with_gil(|py| { let init = |value, code| { let c = PyCell::new(py, InPlaceOperations { value }).unwrap(); py_run!(py, c, code); }; init(0, "d = c; c += 1; assert repr(c) == repr(d) == 'IPO(1)'"); init(10, "d = c; c -= 1; assert repr(c) == repr(d) == 'IPO(9)'"); init(3, "d = c; c *= 3; assert repr(c) == repr(d) == 'IPO(9)'"); init(3, "d = c; c <<= 2; assert repr(c) == repr(d) == 'IPO(12)'"); init(12, "d = c; c >>= 2; assert repr(c) == repr(d) == 'IPO(3)'"); init(12, "d = c; c &= 10; assert repr(c) == repr(d) == 'IPO(8)'"); init(12, "d = c; c |= 3; assert repr(c) == repr(d) == 'IPO(15)'"); init(12, "d = c; c ^= 5; assert repr(c) == repr(d) == 'IPO(9)'"); init(3, "d = c; c **= 4; assert repr(c) == repr(d) == 'IPO(81)'"); init( 3, "d = c; c.__ipow__(4); assert repr(c) == repr(d) == 'IPO(81)'", ); }); } #[pyclass] struct BinaryArithmetic {} #[pymethods] impl BinaryArithmetic { fn __repr__(&self) -> &'static str { "BA" } fn __add__(&self, rhs: &PyAny) -> String { format!("BA + {:?}", rhs) } fn __sub__(&self, rhs: &PyAny) -> String { format!("BA - {:?}", rhs) } fn __mul__(&self, rhs: &PyAny) -> String { format!("BA * {:?}", rhs) } fn __lshift__(&self, rhs: &PyAny) -> String { format!("BA << {:?}", rhs) } fn __rshift__(&self, rhs: &PyAny) -> String { format!("BA >> {:?}", rhs) } fn __and__(&self, rhs: &PyAny) -> String { format!("BA & {:?}", rhs) } fn __xor__(&self, rhs: &PyAny) -> String { format!("BA ^ {:?}", rhs) } fn __or__(&self, rhs: &PyAny) -> String { format!("BA | {:?}", rhs) } fn __pow__(&self, rhs: &PyAny, mod_: Option) -> String { format!("BA ** {:?} (mod: {:?})", rhs, mod_) } } #[test] fn binary_arithmetic() { Python::with_gil(|py| { let c = PyCell::new(py, BinaryArithmetic {}).unwrap(); py_run!(py, c, "assert c + c == 'BA + BA'"); py_run!(py, c, "assert c.__add__(c) == 'BA + BA'"); py_run!(py, c, "assert c + 1 == 'BA + 1'"); py_run!(py, c, "assert c - 1 == 'BA - 1'"); py_run!(py, c, "assert c * 1 == 'BA * 1'"); py_run!(py, c, "assert c << 1 == 'BA << 1'"); py_run!(py, c, "assert c >> 1 == 'BA >> 1'"); py_run!(py, c, "assert c & 1 == 'BA & 1'"); py_run!(py, c, "assert c ^ 1 == 'BA ^ 1'"); py_run!(py, c, "assert c | 1 == 'BA | 1'"); py_run!(py, c, "assert c ** 1 == 'BA ** 1 (mod: None)'"); // Class with __add__ only should not allow the reverse op; // this is consistent with Python classes. py_expect_exception!(py, c, "1 + c", PyTypeError); py_expect_exception!(py, c, "1 - c", PyTypeError); py_expect_exception!(py, c, "1 * c", PyTypeError); py_expect_exception!(py, c, "1 << c", PyTypeError); py_expect_exception!(py, c, "1 >> c", PyTypeError); py_expect_exception!(py, c, "1 & c", PyTypeError); py_expect_exception!(py, c, "1 ^ c", PyTypeError); py_expect_exception!(py, c, "1 | c", PyTypeError); py_expect_exception!(py, c, "1 ** c", PyTypeError); py_run!(py, c, "assert pow(c, 1, 100) == 'BA ** 1 (mod: Some(100))'"); }); } #[pyclass] struct RhsArithmetic {} #[pymethods] impl RhsArithmetic { fn __radd__(&self, other: &PyAny) -> String { format!("{:?} + RA", other) } fn __rsub__(&self, other: &PyAny) -> String { format!("{:?} - RA", other) } fn __rmul__(&self, other: &PyAny) -> String { format!("{:?} * RA", other) } fn __rlshift__(&self, other: &PyAny) -> String { format!("{:?} << RA", other) } fn __rrshift__(&self, other: &PyAny) -> String { format!("{:?} >> RA", other) } fn __rand__(&self, other: &PyAny) -> String { format!("{:?} & RA", other) } fn __rxor__(&self, other: &PyAny) -> String { format!("{:?} ^ RA", other) } fn __ror__(&self, other: &PyAny) -> String { format!("{:?} | RA", other) } fn __rpow__(&self, other: &PyAny, _mod: Option<&PyAny>) -> String { format!("{:?} ** RA", other) } } #[test] fn rhs_arithmetic() { Python::with_gil(|py| { let c = PyCell::new(py, RhsArithmetic {}).unwrap(); py_run!(py, c, "assert c.__radd__(1) == '1 + RA'"); py_run!(py, c, "assert 1 + c == '1 + RA'"); py_run!(py, c, "assert c.__rsub__(1) == '1 - RA'"); py_run!(py, c, "assert 1 - c == '1 - RA'"); py_run!(py, c, "assert c.__rmul__(1) == '1 * RA'"); py_run!(py, c, "assert 1 * c == '1 * RA'"); py_run!(py, c, "assert c.__rlshift__(1) == '1 << RA'"); py_run!(py, c, "assert 1 << c == '1 << RA'"); py_run!(py, c, "assert c.__rrshift__(1) == '1 >> RA'"); py_run!(py, c, "assert 1 >> c == '1 >> RA'"); py_run!(py, c, "assert c.__rand__(1) == '1 & RA'"); py_run!(py, c, "assert 1 & c == '1 & RA'"); py_run!(py, c, "assert c.__rxor__(1) == '1 ^ RA'"); py_run!(py, c, "assert 1 ^ c == '1 ^ RA'"); py_run!(py, c, "assert c.__ror__(1) == '1 | RA'"); py_run!(py, c, "assert 1 | c == '1 | RA'"); py_run!(py, c, "assert c.__rpow__(1) == '1 ** RA'"); py_run!(py, c, "assert 1 ** c == '1 ** RA'"); }); } #[pyclass] struct LhsAndRhs {} impl std::fmt::Debug for LhsAndRhs { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { write!(f, "LR") } } #[pymethods] impl LhsAndRhs { // fn __repr__(&self) -> &'static str { // "BA" // } fn __add__(lhs: PyRef<'_, Self>, rhs: &PyAny) -> String { format!("{:?} + {:?}", lhs, rhs) } fn __sub__(lhs: PyRef<'_, Self>, rhs: &PyAny) -> String { format!("{:?} - {:?}", lhs, rhs) } fn __mul__(lhs: PyRef<'_, Self>, rhs: &PyAny) -> String { format!("{:?} * {:?}", lhs, rhs) } fn __lshift__(lhs: PyRef<'_, Self>, rhs: &PyAny) -> String { format!("{:?} << {:?}", lhs, rhs) } fn __rshift__(lhs: PyRef<'_, Self>, rhs: &PyAny) -> String { format!("{:?} >> {:?}", lhs, rhs) } fn __and__(lhs: PyRef<'_, Self>, rhs: &PyAny) -> String { format!("{:?} & {:?}", lhs, rhs) } fn __xor__(lhs: PyRef<'_, Self>, rhs: &PyAny) -> String { format!("{:?} ^ {:?}", lhs, rhs) } fn __or__(lhs: PyRef<'_, Self>, rhs: &PyAny) -> String { format!("{:?} | {:?}", lhs, rhs) } fn __pow__(lhs: PyRef<'_, Self>, rhs: &PyAny, _mod: Option) -> String { format!("{:?} ** {:?}", lhs, rhs) } fn __matmul__(lhs: PyRef<'_, Self>, rhs: &PyAny) -> String { format!("{:?} @ {:?}", lhs, rhs) } fn __radd__(&self, other: &PyAny) -> String { format!("{:?} + RA", other) } fn __rsub__(&self, other: &PyAny) -> String { format!("{:?} - RA", other) } fn __rmul__(&self, other: &PyAny) -> String { format!("{:?} * RA", other) } fn __rlshift__(&self, other: &PyAny) -> String { format!("{:?} << RA", other) } fn __rrshift__(&self, other: &PyAny) -> String { format!("{:?} >> RA", other) } fn __rand__(&self, other: &PyAny) -> String { format!("{:?} & RA", other) } fn __rxor__(&self, other: &PyAny) -> String { format!("{:?} ^ RA", other) } fn __ror__(&self, other: &PyAny) -> String { format!("{:?} | RA", other) } fn __rpow__(&self, other: &PyAny, _mod: Option<&PyAny>) -> String { format!("{:?} ** RA", other) } fn __rmatmul__(&self, other: &PyAny) -> String { format!("{:?} @ RA", other) } fn __rtruediv__(&self, other: &PyAny) -> String { format!("{:?} / RA", other) } fn __rfloordiv__(&self, other: &PyAny) -> String { format!("{:?} // RA", other) } } #[test] fn lhs_fellback_to_rhs() { Python::with_gil(|py| { let c = PyCell::new(py, LhsAndRhs {}).unwrap(); // If the light hand value is `LhsAndRhs`, LHS is used. py_run!(py, c, "assert c + 1 == 'LR + 1'"); py_run!(py, c, "assert c - 1 == 'LR - 1'"); py_run!(py, c, "assert c * 1 == 'LR * 1'"); py_run!(py, c, "assert c << 1 == 'LR << 1'"); py_run!(py, c, "assert c >> 1 == 'LR >> 1'"); py_run!(py, c, "assert c & 1 == 'LR & 1'"); py_run!(py, c, "assert c ^ 1 == 'LR ^ 1'"); py_run!(py, c, "assert c | 1 == 'LR | 1'"); py_run!(py, c, "assert c ** 1 == 'LR ** 1'"); py_run!(py, c, "assert c @ 1 == 'LR @ 1'"); // Fellback to RHS because of type mismatching py_run!(py, c, "assert 1 + c == '1 + RA'"); py_run!(py, c, "assert 1 - c == '1 - RA'"); py_run!(py, c, "assert 1 * c == '1 * RA'"); py_run!(py, c, "assert 1 << c == '1 << RA'"); py_run!(py, c, "assert 1 >> c == '1 >> RA'"); py_run!(py, c, "assert 1 & c == '1 & RA'"); py_run!(py, c, "assert 1 ^ c == '1 ^ RA'"); py_run!(py, c, "assert 1 | c == '1 | RA'"); py_run!(py, c, "assert 1 ** c == '1 ** RA'"); py_run!(py, c, "assert 1 @ c == '1 @ RA'"); }); } #[pyclass] struct RichComparisons {} #[pymethods] impl RichComparisons { fn __repr__(&self) -> &'static str { "RC" } fn __richcmp__(&self, other: &PyAny, op: CompareOp) -> String { match op { CompareOp::Lt => format!("{} < {:?}", self.__repr__(), other), CompareOp::Le => format!("{} <= {:?}", self.__repr__(), other), CompareOp::Eq => format!("{} == {:?}", self.__repr__(), other), CompareOp::Ne => format!("{} != {:?}", self.__repr__(), other), CompareOp::Gt => format!("{} > {:?}", self.__repr__(), other), CompareOp::Ge => format!("{} >= {:?}", self.__repr__(), other), } } } #[pyclass] struct RichComparisons2 {} #[pymethods] impl RichComparisons2 { fn __repr__(&self) -> &'static str { "RC2" } fn __richcmp__(&self, other: &PyAny, op: CompareOp) -> PyObject { match op { CompareOp::Eq => true.into_py(other.py()), CompareOp::Ne => false.into_py(other.py()), _ => other.py().NotImplemented(), } } } #[test] fn rich_comparisons() { Python::with_gil(|py| { let c = PyCell::new(py, RichComparisons {}).unwrap(); py_run!(py, c, "assert (c < c) == 'RC < RC'"); py_run!(py, c, "assert (c < 1) == 'RC < 1'"); py_run!(py, c, "assert (1 < c) == 'RC > 1'"); py_run!(py, c, "assert (c <= c) == 'RC <= RC'"); py_run!(py, c, "assert (c <= 1) == 'RC <= 1'"); py_run!(py, c, "assert (1 <= c) == 'RC >= 1'"); py_run!(py, c, "assert (c == c) == 'RC == RC'"); py_run!(py, c, "assert (c == 1) == 'RC == 1'"); py_run!(py, c, "assert (1 == c) == 'RC == 1'"); py_run!(py, c, "assert (c != c) == 'RC != RC'"); py_run!(py, c, "assert (c != 1) == 'RC != 1'"); py_run!(py, c, "assert (1 != c) == 'RC != 1'"); py_run!(py, c, "assert (c > c) == 'RC > RC'"); py_run!(py, c, "assert (c > 1) == 'RC > 1'"); py_run!(py, c, "assert (1 > c) == 'RC < 1'"); py_run!(py, c, "assert (c >= c) == 'RC >= RC'"); py_run!(py, c, "assert (c >= 1) == 'RC >= 1'"); py_run!(py, c, "assert (1 >= c) == 'RC <= 1'"); }); } #[test] fn rich_comparisons_python_3_type_error() { Python::with_gil(|py| { let c2 = PyCell::new(py, RichComparisons2 {}).unwrap(); py_expect_exception!(py, c2, "c2 < c2", PyTypeError); py_expect_exception!(py, c2, "c2 < 1", PyTypeError); py_expect_exception!(py, c2, "1 < c2", PyTypeError); py_expect_exception!(py, c2, "c2 <= c2", PyTypeError); py_expect_exception!(py, c2, "c2 <= 1", PyTypeError); py_expect_exception!(py, c2, "1 <= c2", PyTypeError); py_run!(py, c2, "assert (c2 == c2) == True"); py_run!(py, c2, "assert (c2 == 1) == True"); py_run!(py, c2, "assert (1 == c2) == True"); py_run!(py, c2, "assert (c2 != c2) == False"); py_run!(py, c2, "assert (c2 != 1) == False"); py_run!(py, c2, "assert (1 != c2) == False"); py_expect_exception!(py, c2, "c2 > c2", PyTypeError); py_expect_exception!(py, c2, "c2 > 1", PyTypeError); py_expect_exception!(py, c2, "1 > c2", PyTypeError); py_expect_exception!(py, c2, "c2 >= c2", PyTypeError); py_expect_exception!(py, c2, "c2 >= 1", PyTypeError); py_expect_exception!(py, c2, "1 >= c2", PyTypeError); }); } // Checks that binary operations for which the arguments don't match the // required type, return NotImplemented. mod return_not_implemented { use super::*; #[pyclass] struct RichComparisonToSelf {} #[pymethods] impl RichComparisonToSelf { fn __repr__(&self) -> &'static str { "RC_Self" } fn __richcmp__(&self, other: PyRef<'_, Self>, _op: CompareOp) -> PyObject { other.py().None() } fn __add__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __sub__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __mul__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __matmul__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __truediv__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __floordiv__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __mod__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __pow__(slf: PyRef<'_, Self>, _other: u8, _modulo: Option) -> PyRef<'_, Self> { slf } fn __lshift__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __rshift__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __divmod__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __and__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __or__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } fn __xor__<'p>(slf: PyRef<'p, Self>, _other: PyRef<'p, Self>) -> PyRef<'p, Self> { slf } // Inplace assignments fn __iadd__(&mut self, _other: PyRef<'_, Self>) {} fn __isub__(&mut self, _other: PyRef<'_, Self>) {} fn __imul__(&mut self, _other: PyRef<'_, Self>) {} fn __imatmul__(&mut self, _other: PyRef<'_, Self>) {} fn __itruediv__(&mut self, _other: PyRef<'_, Self>) {} fn __ifloordiv__(&mut self, _other: PyRef<'_, Self>) {} fn __imod__(&mut self, _other: PyRef<'_, Self>) {} fn __ilshift__(&mut self, _other: PyRef<'_, Self>) {} fn __irshift__(&mut self, _other: PyRef<'_, Self>) {} fn __iand__(&mut self, _other: PyRef<'_, Self>) {} fn __ior__(&mut self, _other: PyRef<'_, Self>) {} fn __ixor__(&mut self, _other: PyRef<'_, Self>) {} fn __ipow__(&mut self, _other: PyRef<'_, Self>, _modulo: Option) {} } fn _test_binary_dunder(dunder: &str) { Python::with_gil(|py| { let c2 = PyCell::new(py, RichComparisonToSelf {}).unwrap(); py_run!( py, c2, &format!( "class Other: pass\nassert c2.__{}__(Other()) is NotImplemented", dunder ) ); }); } fn _test_binary_operator(operator: &str, dunder: &str) { _test_binary_dunder(dunder); Python::with_gil(|py| { let c2 = PyCell::new(py, RichComparisonToSelf {}).unwrap(); py_expect_exception!( py, c2, &format!("class Other: pass\nc2 {} Other()", operator), PyTypeError ); }); } fn _test_inplace_binary_operator(operator: &str, dunder: &str) { _test_binary_operator(operator, dunder); } #[test] fn equality() { _test_binary_dunder("eq"); _test_binary_dunder("ne"); } #[test] fn ordering() { _test_binary_operator("<", "lt"); _test_binary_operator("<=", "le"); _test_binary_operator(">", "gt"); _test_binary_operator(">=", "ge"); } #[test] fn bitwise() { _test_binary_operator("&", "and"); _test_binary_operator("|", "or"); _test_binary_operator("^", "xor"); _test_binary_operator("<<", "lshift"); _test_binary_operator(">>", "rshift"); } #[test] fn arith() { _test_binary_operator("+", "add"); _test_binary_operator("-", "sub"); _test_binary_operator("*", "mul"); _test_binary_operator("@", "matmul"); _test_binary_operator("/", "truediv"); _test_binary_operator("//", "floordiv"); _test_binary_operator("%", "mod"); _test_binary_operator("**", "pow"); } #[test] fn reverse_arith() { _test_binary_dunder("radd"); _test_binary_dunder("rsub"); _test_binary_dunder("rmul"); _test_binary_dunder("rmatmul"); _test_binary_dunder("rtruediv"); _test_binary_dunder("rfloordiv"); _test_binary_dunder("rmod"); _test_binary_dunder("rdivmod"); _test_binary_dunder("rpow"); } #[test] fn inplace_bitwise() { _test_inplace_binary_operator("&=", "iand"); _test_inplace_binary_operator("|=", "ior"); _test_inplace_binary_operator("^=", "ixor"); _test_inplace_binary_operator("<<=", "ilshift"); _test_inplace_binary_operator(">>=", "irshift"); } #[test] fn inplace_arith() { _test_inplace_binary_operator("+=", "iadd"); _test_inplace_binary_operator("-=", "isub"); _test_inplace_binary_operator("*=", "imul"); _test_inplace_binary_operator("@=", "imatmul"); _test_inplace_binary_operator("/=", "itruediv"); _test_inplace_binary_operator("//=", "ifloordiv"); _test_inplace_binary_operator("%=", "imod"); _test_inplace_binary_operator("**=", "ipow"); } } pyo3-0.20.2/tests/test_buffer.rs000064400000000000000000000076501046102023000146270ustar 00000000000000#![cfg(feature = "macros")] #![cfg(any(not(Py_LIMITED_API), Py_3_11))] use pyo3::{buffer::PyBuffer, exceptions::PyBufferError, ffi, prelude::*}; use std::{ ffi::CStr, os::raw::{c_int, c_void}, ptr, }; #[macro_use] #[path = "../src/tests/common.rs"] mod common; enum TestGetBufferError { NullShape, NullStrides, IncorrectItemSize, IncorrectFormat, IncorrectAlignment, } #[pyclass] struct TestBufferErrors { buf: Vec, error: Option, } #[pymethods] impl TestBufferErrors { unsafe fn __getbuffer__( slf: PyRefMut<'_, Self>, view: *mut ffi::Py_buffer, flags: c_int, ) -> PyResult<()> { if view.is_null() { return Err(PyBufferError::new_err("View is null")); } if (flags & ffi::PyBUF_WRITABLE) == ffi::PyBUF_WRITABLE { return Err(PyBufferError::new_err("Object is not writable")); } (*view).obj = ffi::_Py_NewRef(slf.as_ptr()); let bytes = &slf.buf; (*view).buf = bytes.as_ptr() as *mut c_void; (*view).len = bytes.len() as isize; (*view).readonly = 1; (*view).itemsize = std::mem::size_of::() as isize; let msg = CStr::from_bytes_with_nul(b"I\0").unwrap(); (*view).format = msg.as_ptr() as *mut _; (*view).ndim = 1; (*view).shape = &mut (*view).len; (*view).strides = &mut (*view).itemsize; (*view).suboffsets = ptr::null_mut(); (*view).internal = ptr::null_mut(); if let Some(err) = &slf.error { use TestGetBufferError::*; match err { NullShape => { (*view).shape = std::ptr::null_mut(); } NullStrides => { (*view).strides = std::ptr::null_mut(); } IncorrectItemSize => { (*view).itemsize += 1; } IncorrectFormat => { (*view).format = CStr::from_bytes_with_nul(b"B\0").unwrap().as_ptr() as _; } IncorrectAlignment => (*view).buf = (*view).buf.add(1), } } Ok(()) } } #[test] fn test_get_buffer_errors() { Python::with_gil(|py| { let instance = Py::new( py, TestBufferErrors { buf: vec![0, 1, 2, 3], error: None, }, ) .unwrap(); assert!(PyBuffer::::get(instance.as_ref(py)).is_ok()); instance.borrow_mut(py).error = Some(TestGetBufferError::NullShape); assert_eq!( PyBuffer::::get(instance.as_ref(py)) .unwrap_err() .to_string(), "BufferError: shape is null" ); instance.borrow_mut(py).error = Some(TestGetBufferError::NullStrides); assert_eq!( PyBuffer::::get(instance.as_ref(py)) .unwrap_err() .to_string(), "BufferError: strides is null" ); instance.borrow_mut(py).error = Some(TestGetBufferError::IncorrectItemSize); assert_eq!( PyBuffer::::get(instance.as_ref(py)) .unwrap_err() .to_string(), "BufferError: buffer contents are not compatible with u32" ); instance.borrow_mut(py).error = Some(TestGetBufferError::IncorrectFormat); assert_eq!( PyBuffer::::get(instance.as_ref(py)) .unwrap_err() .to_string(), "BufferError: buffer contents are not compatible with u32" ); instance.borrow_mut(py).error = Some(TestGetBufferError::IncorrectAlignment); assert_eq!( PyBuffer::::get(instance.as_ref(py)) .unwrap_err() .to_string(), "BufferError: buffer contents are insufficiently aligned for u32" ); }); } pyo3-0.20.2/tests/test_buffer_protocol.rs000064400000000000000000000115341046102023000165440ustar 00000000000000#![cfg(feature = "macros")] #![cfg(any(not(Py_LIMITED_API), Py_3_11))] use pyo3::buffer::PyBuffer; use pyo3::exceptions::PyBufferError; use pyo3::ffi; use pyo3::prelude::*; use pyo3::types::IntoPyDict; use std::ffi::CString; use std::os::raw::{c_int, c_void}; use std::ptr; use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::Arc; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct TestBufferClass { vec: Vec, drop_called: Arc, } #[pymethods] impl TestBufferClass { unsafe fn __getbuffer__( slf: &PyCell, view: *mut ffi::Py_buffer, flags: c_int, ) -> PyResult<()> { fill_view_from_readonly_data(view, flags, &slf.borrow().vec, slf) } unsafe fn __releasebuffer__(&self, view: *mut ffi::Py_buffer) { // Release memory held by the format string drop(CString::from_raw((*view).format)); } } impl Drop for TestBufferClass { fn drop(&mut self) { print!("dropped"); self.drop_called.store(true, Ordering::Relaxed); } } #[test] fn test_buffer() { let drop_called = Arc::new(AtomicBool::new(false)); Python::with_gil(|py| { let instance = Py::new( py, TestBufferClass { vec: vec![b' ', b'2', b'3'], drop_called: drop_called.clone(), }, ) .unwrap(); let env = [("ob", instance)].into_py_dict(py); py_assert!(py, *env, "bytes(ob) == b' 23'"); }); assert!(drop_called.load(Ordering::Relaxed)); } #[test] fn test_buffer_referenced() { let drop_called = Arc::new(AtomicBool::new(false)); let buf = { let input = vec![b' ', b'2', b'3']; Python::with_gil(|py| { let instance: PyObject = TestBufferClass { vec: input.clone(), drop_called: drop_called.clone(), } .into_py(py); let buf = PyBuffer::::get(instance.as_ref(py)).unwrap(); assert_eq!(buf.to_vec(py).unwrap(), input); drop(instance); buf }) }; assert!(!drop_called.load(Ordering::Relaxed)); Python::with_gil(|_| { drop(buf); }); assert!(drop_called.load(Ordering::Relaxed)); } #[test] #[cfg(Py_3_8)] // sys.unraisablehook not available until Python 3.8 fn test_releasebuffer_unraisable_error() { use common::UnraisableCapture; use pyo3::exceptions::PyValueError; #[pyclass] struct ReleaseBufferError {} #[pymethods] impl ReleaseBufferError { unsafe fn __getbuffer__( slf: &PyCell, view: *mut ffi::Py_buffer, flags: c_int, ) -> PyResult<()> { static BUF_BYTES: &[u8] = b"hello world"; fill_view_from_readonly_data(view, flags, BUF_BYTES, slf) } unsafe fn __releasebuffer__(&self, _view: *mut ffi::Py_buffer) -> PyResult<()> { Err(PyValueError::new_err("oh dear")) } } Python::with_gil(|py| { let capture = UnraisableCapture::install(py); let instance = Py::new(py, ReleaseBufferError {}).unwrap(); let env = [("ob", instance.clone())].into_py_dict(py); assert!(capture.borrow(py).capture.is_none()); py_assert!(py, *env, "bytes(ob) == b'hello world'"); let (err, object) = capture.borrow_mut(py).capture.take().unwrap(); assert_eq!(err.to_string(), "ValueError: oh dear"); assert!(object.is(&instance)); capture.borrow_mut(py).uninstall(py); }); } /// # Safety /// /// `view` must be a valid pointer to ffi::Py_buffer, or null /// `data` must outlive the Python lifetime of `owner` (i.e. data must be owned by owner, or data /// must be static data) unsafe fn fill_view_from_readonly_data( view: *mut ffi::Py_buffer, flags: c_int, data: &[u8], owner: &PyAny, ) -> PyResult<()> { if view.is_null() { return Err(PyBufferError::new_err("View is null")); } if (flags & ffi::PyBUF_WRITABLE) == ffi::PyBUF_WRITABLE { return Err(PyBufferError::new_err("Object is not writable")); } (*view).obj = ffi::_Py_NewRef(owner.as_ptr()); (*view).buf = data.as_ptr() as *mut c_void; (*view).len = data.len() as isize; (*view).readonly = 1; (*view).itemsize = 1; (*view).format = if (flags & ffi::PyBUF_FORMAT) == ffi::PyBUF_FORMAT { let msg = CString::new("B").unwrap(); msg.into_raw() } else { ptr::null_mut() }; (*view).ndim = 1; (*view).shape = if (flags & ffi::PyBUF_ND) == ffi::PyBUF_ND { &mut (*view).len } else { ptr::null_mut() }; (*view).strides = if (flags & ffi::PyBUF_STRIDES) == ffi::PyBUF_STRIDES { &mut (*view).itemsize } else { ptr::null_mut() }; (*view).suboffsets = ptr::null_mut(); (*view).internal = ptr::null_mut(); Ok(()) } pyo3-0.20.2/tests/test_bytes.rs000064400000000000000000000023501046102023000144740ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::types::PyBytes; #[path = "../src/tests/common.rs"] mod common; #[pyfunction] fn bytes_pybytes_conversion(bytes: &[u8]) -> &[u8] { bytes } #[test] fn test_pybytes_bytes_conversion() { Python::with_gil(|py| { let f = wrap_pyfunction!(bytes_pybytes_conversion)(py).unwrap(); py_assert!(py, f, "f(b'Hello World') == b'Hello World'"); }); } #[pyfunction] fn bytes_vec_conversion(py: Python<'_>, bytes: Vec) -> &PyBytes { PyBytes::new(py, bytes.as_slice()) } #[test] fn test_pybytes_vec_conversion() { Python::with_gil(|py| { let f = wrap_pyfunction!(bytes_vec_conversion)(py).unwrap(); py_assert!(py, f, "f(b'Hello World') == b'Hello World'"); }); } #[test] fn test_bytearray_vec_conversion() { Python::with_gil(|py| { let f = wrap_pyfunction!(bytes_vec_conversion)(py).unwrap(); py_assert!(py, f, "f(bytearray(b'Hello World')) == b'Hello World'"); }); } #[test] fn test_py_as_bytes() { let pyobj: pyo3::Py = Python::with_gil(|py| pyo3::types::PyBytes::new(py, b"abc").into_py(py)); let data = Python::with_gil(|py| pyobj.as_bytes(py)); assert_eq!(data, b"abc"); } pyo3-0.20.2/tests/test_class_attributes.rs000064400000000000000000000154561046102023000167340ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct Foo { #[pyo3(get)] x: i32, } #[pyclass] struct Bar { #[pyo3(get)] x: i32, } #[pymethods] impl Foo { #[classattr] const MY_CONST: &'static str = "foobar"; #[classattr] #[pyo3(name = "RENAMED_CONST")] const MY_CONST_2: &'static str = "foobar_2"; #[classattr] fn a() -> i32 { 5 } #[classattr] #[pyo3(name = "B")] fn b() -> String { "bar".to_string() } #[classattr] fn bar() -> Bar { Bar { x: 2 } } #[classattr] fn a_foo() -> Foo { Foo { x: 1 } } #[classattr] fn a_foo_with_py(py: Python<'_>) -> Py { Py::new(py, Foo { x: 1 }).unwrap() } } #[test] fn class_attributes() { Python::with_gil(|py| { let foo_obj = py.get_type::(); py_assert!(py, foo_obj, "foo_obj.MY_CONST == 'foobar'"); py_assert!(py, foo_obj, "foo_obj.RENAMED_CONST == 'foobar_2'"); py_assert!(py, foo_obj, "foo_obj.a == 5"); py_assert!(py, foo_obj, "foo_obj.B == 'bar'"); py_assert!(py, foo_obj, "foo_obj.a_foo.x == 1"); py_assert!(py, foo_obj, "foo_obj.a_foo_with_py.x == 1"); }); } // Ignored because heap types are not immutable: // https://github.com/python/cpython/blob/master/Objects/typeobject.c#L3399-L3409 #[test] #[ignore] fn class_attributes_are_immutable() { Python::with_gil(|py| { let foo_obj = py.get_type::(); py_expect_exception!(py, foo_obj, "foo_obj.a = 6", PyTypeError); }); } #[pymethods] impl Bar { #[classattr] fn a_foo() -> Foo { Foo { x: 3 } } } #[test] fn recursive_class_attributes() { Python::with_gil(|py| { let foo_obj = py.get_type::(); let bar_obj = py.get_type::(); py_assert!(py, foo_obj, "foo_obj.a_foo.x == 1"); py_assert!(py, foo_obj, "foo_obj.bar.x == 2"); py_assert!(py, bar_obj, "bar_obj.a_foo.x == 3"); }); } #[test] fn test_fallible_class_attribute() { use pyo3::{exceptions::PyValueError, types::PyString}; struct CaptureStdErr<'py> { oldstderr: &'py PyAny, string_io: &'py PyAny, } impl<'py> CaptureStdErr<'py> { fn new(py: Python<'py>) -> PyResult { let sys = py.import("sys")?; let oldstderr = sys.getattr("stderr")?; let string_io = py.import("io")?.getattr("StringIO")?.call0()?; sys.setattr("stderr", string_io)?; Ok(Self { oldstderr, string_io, }) } fn reset(self) -> PyResult { let py = self.string_io.py(); let payload = self .string_io .getattr("getvalue")? .call0()? .downcast::()? .to_str()? .to_owned(); let sys = py.import("sys")?; sys.setattr("stderr", self.oldstderr)?; Ok(payload) } } #[pyclass] struct BrokenClass; #[pymethods] impl BrokenClass { #[classattr] fn fails_to_init() -> PyResult { Err(PyValueError::new_err("failed to create class attribute")) } } Python::with_gil(|py| { let stderr = CaptureStdErr::new(py).unwrap(); assert!(std::panic::catch_unwind(|| py.get_type::()).is_err()); assert_eq!( stderr.reset().unwrap().trim(), "\ ValueError: failed to create class attribute The above exception was the direct cause of the following exception: RuntimeError: An error occurred while initializing `BrokenClass.fails_to_init` The above exception was the direct cause of the following exception: RuntimeError: An error occurred while initializing class BrokenClass" ) }); } #[pyclass(get_all, set_all, rename_all = "camelCase")] struct StructWithRenamedFields { first_field: bool, second_field: u8, #[pyo3(name = "third_field")] fourth_field: bool, } #[pymethods] impl StructWithRenamedFields { #[new] fn new() -> Self { Self { first_field: true, second_field: 5, fourth_field: false, } } } #[test] fn test_renaming_all_struct_fields() { use pyo3::types::PyBool; Python::with_gil(|py| { let struct_class = py.get_type::(); let struct_obj = struct_class.call0().unwrap(); assert!(struct_obj .setattr("firstField", PyBool::new(py, false)) .is_ok()); py_assert!(py, struct_obj, "struct_obj.firstField == False"); py_assert!(py, struct_obj, "struct_obj.secondField == 5"); assert!(struct_obj .setattr("third_field", PyBool::new(py, true)) .is_ok()); py_assert!(py, struct_obj, "struct_obj.third_field == True"); }); } macro_rules! test_case { ($struct_name: ident, $rule: literal, $field_name: ident, $renamed_field_name: literal, $test_name: ident) => { #[pyclass(get_all, set_all, rename_all = $rule)] #[allow(non_snake_case)] struct $struct_name { $field_name: u8, } #[pymethods] impl $struct_name { #[new] fn new() -> Self { Self { $field_name: 0 } } } #[test] fn $test_name() { //use pyo3::types::PyInt; Python::with_gil(|py| { let struct_class = py.get_type::<$struct_name>(); let struct_obj = struct_class.call0().unwrap(); assert!(struct_obj.setattr($renamed_field_name, 2).is_ok()); let attr = struct_obj.getattr($renamed_field_name).unwrap(); assert_eq!(2, PyAny::extract::(attr).unwrap()); }); } }; } test_case!( LowercaseTest, "lowercase", fieldOne, "fieldone", test_rename_all_lowercase ); test_case!( CamelCaseTest, "camelCase", field_one, "fieldOne", test_rename_all_camel_case ); test_case!( KebabCaseTest, "kebab-case", field_one, "field-one", test_rename_all_kebab_case ); test_case!( PascalCaseTest, "PascalCase", field_one, "FieldOne", test_rename_all_pascal_case ); test_case!( ScreamingSnakeCaseTest, "SCREAMING_SNAKE_CASE", field_one, "FIELD_ONE", test_rename_all_screaming_snake_case ); test_case!( ScreamingKebabCaseTest, "SCREAMING-KEBAB-CASE", field_one, "FIELD-ONE", test_rename_all_screaming_kebab_case ); test_case!( SnakeCaseTest, "snake_case", fieldOne, "field_one", test_rename_all_snake_case ); test_case!( UppercaseTest, "UPPERCASE", fieldOne, "FIELDONE", test_rename_all_uppercase ); pyo3-0.20.2/tests/test_class_basics.rs000064400000000000000000000337241046102023000160100ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::types::PyType; use pyo3::{py_run, PyClass}; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct EmptyClass {} #[test] fn empty_class() { Python::with_gil(|py| { let typeobj = py.get_type::(); // By default, don't allow creating instances from python. assert!(typeobj.call((), None).is_err()); py_assert!(py, typeobj, "typeobj.__name__ == 'EmptyClass'"); }); } #[pyclass] struct UnitClass; #[test] fn unit_class() { Python::with_gil(|py| { let typeobj = py.get_type::(); // By default, don't allow creating instances from python. assert!(typeobj.call((), None).is_err()); py_assert!(py, typeobj, "typeobj.__name__ == 'UnitClass'"); }); } /// Line1 ///Line2 /// Line3 // this is not doc string #[pyclass] struct ClassWithDocs { /// Property field #[pyo3(get, set)] value: i32, /// Read-only property field #[pyo3(get)] readonly: i32, /// Write-only property field #[pyo3(set)] #[allow(dead_code)] // Rust detects field is never read writeonly: i32, } #[test] fn class_with_docstr() { Python::with_gil(|py| { let typeobj = py.get_type::(); py_run!( py, typeobj, "assert typeobj.__doc__ == 'Line1\\nLine2\\n Line3'" ); py_run!( py, typeobj, "assert typeobj.value.__doc__ == 'Property field'" ); py_run!( py, typeobj, "assert typeobj.readonly.__doc__ == 'Read-only property field'" ); py_run!( py, typeobj, "assert typeobj.writeonly.__doc__ == 'Write-only property field'" ); }); } #[pyclass(name = "CustomName")] struct EmptyClass2 {} #[pymethods] impl EmptyClass2 { #[pyo3(name = "custom_fn")] fn bar(&self) {} #[staticmethod] #[pyo3(name = "custom_static")] fn bar_static() {} #[getter] #[pyo3(name = "custom_getter")] fn foo(&self) -> i32 { 5 } } #[test] fn custom_names() { Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!(py, typeobj, "typeobj.__name__ == 'CustomName'"); py_assert!(py, typeobj, "typeobj.custom_fn.__name__ == 'custom_fn'"); py_assert!( py, typeobj, "typeobj.custom_static.__name__ == 'custom_static'" ); py_assert!( py, typeobj, "typeobj.custom_getter.__name__ == 'custom_getter'" ); py_assert!(py, typeobj, "not hasattr(typeobj, 'bar')"); py_assert!(py, typeobj, "not hasattr(typeobj, 'bar_static')"); py_assert!(py, typeobj, "not hasattr(typeobj, 'foo')"); }); } #[pyclass] struct RawIdents { #[pyo3(get, set)] r#type: i64, } #[pymethods] impl RawIdents { fn r#fn(&self) {} } #[test] fn test_raw_idents() { Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!(py, typeobj, "not hasattr(typeobj, 'r#fn')"); py_assert!(py, typeobj, "hasattr(typeobj, 'fn')"); py_assert!(py, typeobj, "hasattr(typeobj, 'type')"); }); } #[pyclass] struct EmptyClassInModule {} // Ignored because heap types do not show up as being in builtins, instead they // raise AttributeError: // https://github.com/python/cpython/blob/v3.11.1/Objects/typeobject.c#L541-L570 #[test] #[ignore] fn empty_class_in_module() { Python::with_gil(|py| { let module = PyModule::new(py, "test_module.nested").unwrap(); module.add_class::().unwrap(); let ty = module.getattr("EmptyClassInModule").unwrap(); assert_eq!( ty.getattr("__name__").unwrap().extract::().unwrap(), "EmptyClassInModule" ); let module: String = ty.getattr("__module__").unwrap().extract().unwrap(); // Rationale: The class can be added to many modules, but will only be initialized once. // We currently have no way of determining a canonical module, so builtins is better // than using whatever calls init first. assert_eq!(module, "builtins"); }); } #[pyclass] struct ClassWithObjectField { // It used to be that PyObject was not supported with (get, set) // - this test is just ensuring it compiles. #[pyo3(get, set)] value: PyObject, } #[pymethods] impl ClassWithObjectField { #[new] fn new(value: PyObject) -> ClassWithObjectField { Self { value } } } #[test] fn class_with_object_field() { Python::with_gil(|py| { let ty = py.get_type::(); py_assert!(py, ty, "ty(5).value == 5"); py_assert!(py, ty, "ty(None).value == None"); }); } #[pyclass(unsendable, subclass)] struct UnsendableBase { value: std::rc::Rc, } #[pymethods] impl UnsendableBase { #[new] fn new(value: usize) -> UnsendableBase { Self { value: std::rc::Rc::new(value), } } #[getter] fn value(&self) -> usize { *self.value } } #[pyclass(extends=UnsendableBase)] struct UnsendableChild {} #[pymethods] impl UnsendableChild { #[new] fn new(value: usize) -> (UnsendableChild, UnsendableBase) { (UnsendableChild {}, UnsendableBase::new(value)) } } fn test_unsendable() -> PyResult<()> { let obj = Python::with_gil(|py| -> PyResult<_> { let obj: Py = PyType::new::(py).call1((5,))?.extract()?; // Accessing the value inside this thread should not panic let caught_panic = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| -> PyResult<_> { assert_eq!(obj.as_ref(py).getattr("value")?.extract::()?, 5); Ok(()) })) .is_err(); assert!(!caught_panic); Ok(obj) })?; let keep_obj_here = obj.clone(); let caught_panic = std::thread::spawn(move || { // This access must panic Python::with_gil(|py| { obj.borrow(py); }); }) .join(); Python::with_gil(|_py| drop(keep_obj_here)); if let Err(err) = caught_panic { if let Some(msg) = err.downcast_ref::() { panic!("{}", msg); } } Ok(()) } /// If a class is marked as `unsendable`, it panics when accessed by another thread. #[test] #[cfg_attr(target_arch = "wasm32", ignore)] #[should_panic( expected = "test_class_basics::UnsendableBase is unsendable, but sent to another thread" )] fn panic_unsendable_base() { test_unsendable::().unwrap(); } #[test] #[cfg_attr(target_arch = "wasm32", ignore)] #[should_panic( expected = "test_class_basics::UnsendableBase is unsendable, but sent to another thread" )] fn panic_unsendable_child() { test_unsendable::().unwrap(); } fn get_length(obj: &PyAny) -> PyResult { let length = obj.len()?; Ok(length) } #[pyclass] struct ClassWithFromPyWithMethods {} #[pymethods] impl ClassWithFromPyWithMethods { fn instance_method(&self, #[pyo3(from_py_with = "get_length")] argument: usize) -> usize { argument } #[classmethod] fn classmethod(_cls: &PyType, #[pyo3(from_py_with = "PyAny::len")] argument: usize) -> usize { argument } #[staticmethod] fn staticmethod(#[pyo3(from_py_with = "get_length")] argument: usize) -> usize { argument } } #[test] fn test_pymethods_from_py_with() { Python::with_gil(|py| { let instance = Py::new(py, ClassWithFromPyWithMethods {}).unwrap(); py_run!( py, instance, r#" arg = {1: 1, 2: 3} assert instance.instance_method(arg) == 2 assert instance.classmethod(arg) == 2 assert instance.staticmethod(arg) == 2 "# ); }) } #[pyclass] struct TupleClass(#[pyo3(get, set, name = "value")] i32); #[test] fn test_tuple_struct_class() { Python::with_gil(|py| { let typeobj = py.get_type::(); assert!(typeobj.call((), None).is_err()); py_assert!(py, typeobj, "typeobj.__name__ == 'TupleClass'"); let instance = Py::new(py, TupleClass(5)).unwrap(); py_run!( py, instance, r#" assert instance.value == 5; instance.value = 1234; assert instance.value == 1234; "# ); assert_eq!(instance.borrow(py).0, 1234); }); } #[pyclass(dict, subclass)] struct DunderDictSupport { // Make sure that dict_offset runs with non-zero sized Self _pad: [u8; 32], } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_9)), ignore)] fn dunder_dict_support() { Python::with_gil(|py| { let inst = PyCell::new( py, DunderDictSupport { _pad: *b"DEADBEEFDEADBEEFDEADBEEFDEADBEEF", }, ) .unwrap(); py_run!( py, inst, r#" inst.a = 1 assert inst.a == 1 "# ); }); } // Accessing inst.__dict__ only supported in limited API from Python 3.10 #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_10)), ignore)] fn access_dunder_dict() { Python::with_gil(|py| { let inst = PyCell::new( py, DunderDictSupport { _pad: *b"DEADBEEFDEADBEEFDEADBEEFDEADBEEF", }, ) .unwrap(); py_run!( py, inst, r#" inst.a = 1 assert inst.__dict__ == {'a': 1} "# ); }); } // If the base class has dict support, child class also has dict #[pyclass(extends=DunderDictSupport)] struct InheritDict { _value: usize, } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_9)), ignore)] fn inherited_dict() { Python::with_gil(|py| { let inst = PyCell::new( py, ( InheritDict { _value: 0 }, DunderDictSupport { _pad: *b"DEADBEEFDEADBEEFDEADBEEFDEADBEEF", }, ), ) .unwrap(); py_run!( py, inst, r#" inst.a = 1 assert inst.a == 1 "# ); }); } #[pyclass(weakref, dict)] struct WeakRefDunderDictSupport { // Make sure that weaklist_offset runs with non-zero sized Self _pad: [u8; 32], } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_9)), ignore)] fn weakref_dunder_dict_support() { Python::with_gil(|py| { let inst = PyCell::new( py, WeakRefDunderDictSupport { _pad: *b"DEADBEEFDEADBEEFDEADBEEFDEADBEEF", }, ) .unwrap(); py_run!( py, inst, "import weakref; assert weakref.ref(inst)() is inst; inst.a = 1; assert inst.a == 1" ); }); } #[pyclass(weakref, subclass)] struct WeakRefSupport { _pad: [u8; 32], } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_9)), ignore)] fn weakref_support() { Python::with_gil(|py| { let inst = PyCell::new( py, WeakRefSupport { _pad: *b"DEADBEEFDEADBEEFDEADBEEFDEADBEEF", }, ) .unwrap(); py_run!( py, inst, "import weakref; assert weakref.ref(inst)() is inst" ); }); } // If the base class has weakref support, child class also has weakref. #[pyclass(extends=WeakRefSupport)] struct InheritWeakRef { _value: usize, } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_9)), ignore)] fn inherited_weakref() { Python::with_gil(|py| { let inst = PyCell::new( py, ( InheritWeakRef { _value: 0 }, WeakRefSupport { _pad: *b"DEADBEEFDEADBEEFDEADBEEFDEADBEEF", }, ), ) .unwrap(); py_run!( py, inst, "import weakref; assert weakref.ref(inst)() is inst" ); }); } #[test] fn access_frozen_class_without_gil() { use std::sync::atomic::{AtomicUsize, Ordering}; #[pyclass(frozen)] struct FrozenCounter { value: AtomicUsize, } let py_counter: Py = Python::with_gil(|py| { let counter = FrozenCounter { value: AtomicUsize::new(0), }; let cell = PyCell::new(py, counter).unwrap(); cell.get().value.fetch_add(1, Ordering::Relaxed); cell.into() }); assert_eq!(py_counter.get().value.load(Ordering::Relaxed), 1); } #[test] #[cfg(Py_3_8)] // sys.unraisablehook not available until Python 3.8 #[cfg_attr(target_arch = "wasm32", ignore)] fn drop_unsendable_elsewhere() { use common::UnraisableCapture; use std::sync::{ atomic::{AtomicBool, Ordering}, Arc, }; use std::thread::spawn; #[pyclass(unsendable)] struct Unsendable { dropped: Arc, } impl Drop for Unsendable { fn drop(&mut self) { self.dropped.store(true, Ordering::SeqCst); } } Python::with_gil(|py| { let capture = UnraisableCapture::install(py); let dropped = Arc::new(AtomicBool::new(false)); let unsendable = Py::new( py, Unsendable { dropped: dropped.clone(), }, ) .unwrap(); py.allow_threads(|| { spawn(move || { Python::with_gil(move |_py| { drop(unsendable); }); }) .join() .unwrap(); }); assert!(!dropped.load(Ordering::SeqCst)); let (err, object) = capture.borrow_mut(py).capture.take().unwrap(); assert_eq!(err.to_string(), "RuntimeError: test_class_basics::drop_unsendable_elsewhere::Unsendable is unsendable, but is being dropped on another thread"); assert!(object.is_none(py)); capture.borrow_mut(py).uninstall(py); }); } pyo3-0.20.2/tests/test_class_conversion.rs000064400000000000000000000072451046102023000167300ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::ToPyObject; #[macro_use] #[path = "../src/tests/common.rs"] mod common; #[pyclass] #[derive(Clone, Debug, PartialEq)] struct Cloneable { x: i32, } #[test] fn test_cloneable_pyclass() { let c = Cloneable { x: 10 }; Python::with_gil(|py| { let py_c = Py::new(py, c.clone()).unwrap().to_object(py); let c2: Cloneable = py_c.extract(py).unwrap(); assert_eq!(c, c2); { let rc: PyRef<'_, Cloneable> = py_c.extract(py).unwrap(); assert_eq!(&c, &*rc); // Drops PyRef before taking PyRefMut } let mrc: PyRefMut<'_, Cloneable> = py_c.extract(py).unwrap(); assert_eq!(&c, &*mrc); }); } #[pyclass(subclass)] #[derive(Default)] struct BaseClass { value: i32, } #[pymethods] impl BaseClass { fn foo(&self) -> &'static str { "BaseClass" } } #[pyclass(extends=BaseClass)] struct SubClass {} #[pymethods] impl SubClass { fn foo(&self) -> &'static str { "SubClass" } } #[pyclass] struct PolymorphicContainer { #[pyo3(get, set)] inner: Py, } #[test] fn test_polymorphic_container_stores_base_class() { Python::with_gil(|py| { let p = PyCell::new( py, PolymorphicContainer { inner: Py::new(py, BaseClass::default()).unwrap(), }, ) .unwrap() .to_object(py); py_assert!(py, p, "p.inner.foo() == 'BaseClass'"); }); } #[test] fn test_polymorphic_container_stores_sub_class() { Python::with_gil(|py| { let p = PyCell::new( py, PolymorphicContainer { inner: Py::new(py, BaseClass::default()).unwrap(), }, ) .unwrap() .to_object(py); p.as_ref(py) .setattr( "inner", PyCell::new( py, PyClassInitializer::from(BaseClass::default()).add_subclass(SubClass {}), ) .unwrap(), ) .unwrap(); py_assert!(py, p, "p.inner.foo() == 'SubClass'"); }); } #[test] fn test_polymorphic_container_does_not_accept_other_types() { Python::with_gil(|py| { let p = PyCell::new( py, PolymorphicContainer { inner: Py::new(py, BaseClass::default()).unwrap(), }, ) .unwrap() .to_object(py); let setattr = |value: PyObject| p.as_ref(py).setattr("inner", value); assert!(setattr(1i32.into_py(py)).is_err()); assert!(setattr(py.None()).is_err()); assert!(setattr((1i32, 2i32).into_py(py)).is_err()); }); } #[test] fn test_pyref_as_base() { Python::with_gil(|py| { let cell = PyCell::new(py, (SubClass {}, BaseClass { value: 120 })).unwrap(); // First try PyRefMut let sub: PyRefMut<'_, SubClass> = cell.borrow_mut(); let mut base: PyRefMut<'_, BaseClass> = sub.into_super(); assert_eq!(120, base.value); base.value = 999; assert_eq!(999, base.value); drop(base); // Repeat for PyRef let sub: PyRef<'_, SubClass> = cell.borrow(); let base: PyRef<'_, BaseClass> = sub.into_super(); assert_eq!(999, base.value); }); } #[test] fn test_pycell_deref() { Python::with_gil(|py| { let cell = PyCell::new(py, (SubClass {}, BaseClass { value: 120 })).unwrap(); // Should be able to deref as PyAny assert_eq!( cell.call_method0("foo") .and_then(PyAny::extract::<&str>) .unwrap(), "SubClass" ); }); } pyo3-0.20.2/tests/test_class_new.rs000064400000000000000000000143401046102023000153260ustar 00000000000000#![cfg(feature = "macros")] use pyo3::exceptions::PyValueError; use pyo3::prelude::*; use pyo3::sync::GILOnceCell; use pyo3::types::IntoPyDict; #[pyclass] struct EmptyClassWithNew {} #[pymethods] impl EmptyClassWithNew { #[new] fn new() -> EmptyClassWithNew { EmptyClassWithNew {} } } #[test] fn empty_class_with_new() { Python::with_gil(|py| { let typeobj = py.get_type::(); assert!(typeobj .call((), None) .unwrap() .downcast::>() .is_ok()); // Calling with arbitrary args or kwargs is not ok assert!(typeobj.call(("some", "args"), None).is_err()); assert!(typeobj .call((), Some([("some", "kwarg")].into_py_dict(py))) .is_err()); }); } #[pyclass] struct UnitClassWithNew; #[pymethods] impl UnitClassWithNew { #[new] fn new() -> Self { Self } } #[test] fn unit_class_with_new() { Python::with_gil(|py| { let typeobj = py.get_type::(); assert!(typeobj .call((), None) .unwrap() .downcast::>() .is_ok()); }); } #[pyclass] struct TupleClassWithNew(i32); #[pymethods] impl TupleClassWithNew { #[new] fn new(arg: i32) -> Self { Self(arg) } } #[test] fn tuple_class_with_new() { Python::with_gil(|py| { let typeobj = py.get_type::(); let wrp = typeobj.call((42,), None).unwrap(); let obj = wrp.downcast::>().unwrap(); let obj_ref = obj.borrow(); assert_eq!(obj_ref.0, 42); }); } #[pyclass] #[derive(Debug)] struct NewWithOneArg { data: i32, } #[pymethods] impl NewWithOneArg { #[new] fn new(arg: i32) -> NewWithOneArg { NewWithOneArg { data: arg } } } #[test] fn new_with_one_arg() { Python::with_gil(|py| { let typeobj = py.get_type::(); let wrp = typeobj.call((42,), None).unwrap(); let obj = wrp.downcast::>().unwrap(); let obj_ref = obj.borrow(); assert_eq!(obj_ref.data, 42); }); } #[pyclass] struct NewWithTwoArgs { data1: i32, data2: i32, } #[pymethods] impl NewWithTwoArgs { #[new] fn new(arg1: i32, arg2: i32) -> Self { NewWithTwoArgs { data1: arg1, data2: arg2, } } } #[test] fn new_with_two_args() { Python::with_gil(|py| { let typeobj = py.get_type::(); let wrp = typeobj .call((10, 20), None) .map_err(|e| e.display(py)) .unwrap(); let obj = wrp.downcast::>().unwrap(); let obj_ref = obj.borrow(); assert_eq!(obj_ref.data1, 10); assert_eq!(obj_ref.data2, 20); }); } #[pyclass(subclass)] struct SuperClass { #[pyo3(get)] from_rust: bool, } #[pymethods] impl SuperClass { #[new] fn new() -> Self { SuperClass { from_rust: true } } } /// Checks that `subclass.__new__` works correctly. /// See https://github.com/PyO3/pyo3/issues/947 for the corresponding bug. #[test] fn subclass_new() { Python::with_gil(|py| { let super_cls = py.get_type::(); let source = pyo3::indoc::indoc!( r#" class Class(SuperClass): def __new__(cls): return super().__new__(cls) # This should return an instance of Class @property def from_rust(self): return False c = Class() assert c.from_rust is False "# ); let globals = PyModule::import(py, "__main__").unwrap().dict(); globals.set_item("SuperClass", super_cls).unwrap(); py.run(source, Some(globals), None) .map_err(|e| e.display(py)) .unwrap(); }); } #[pyclass] #[derive(Debug)] struct NewWithCustomError {} struct CustomError; impl From for PyErr { fn from(_error: CustomError) -> PyErr { PyValueError::new_err("custom error") } } #[pymethods] impl NewWithCustomError { #[new] fn new() -> Result { Err(CustomError) } } #[test] fn new_with_custom_error() { Python::with_gil(|py| { let typeobj = py.get_type::(); let err = typeobj.call0().unwrap_err(); assert_eq!(err.to_string(), "ValueError: custom error"); }); } #[pyclass] struct NewExisting { #[pyo3(get)] num: usize, } #[pymethods] impl NewExisting { #[new] fn new(py: pyo3::Python<'_>, val: usize) -> pyo3::Py { static PRE_BUILT: GILOnceCell<[pyo3::Py; 2]> = GILOnceCell::new(); let existing = PRE_BUILT.get_or_init(py, || { [ pyo3::PyCell::new(py, NewExisting { num: 0 }) .unwrap() .into(), pyo3::PyCell::new(py, NewExisting { num: 1 }) .unwrap() .into(), ] }); if val < existing.len() { return existing[val].clone_ref(py); } pyo3::PyCell::new(py, NewExisting { num: val }) .unwrap() .into() } } #[test] fn test_new_existing() { Python::with_gil(|py| { let typeobj = py.get_type::(); let obj1 = typeobj.call1((0,)).unwrap(); let obj2 = typeobj.call1((0,)).unwrap(); let obj3 = typeobj.call1((1,)).unwrap(); let obj4 = typeobj.call1((1,)).unwrap(); let obj5 = typeobj.call1((2,)).unwrap(); let obj6 = typeobj.call1((2,)).unwrap(); assert!(obj1.getattr("num").unwrap().extract::().unwrap() == 0); assert!(obj2.getattr("num").unwrap().extract::().unwrap() == 0); assert!(obj3.getattr("num").unwrap().extract::().unwrap() == 1); assert!(obj4.getattr("num").unwrap().extract::().unwrap() == 1); assert!(obj5.getattr("num").unwrap().extract::().unwrap() == 2); assert!(obj6.getattr("num").unwrap().extract::().unwrap() == 2); assert!(obj1.is(obj2)); assert!(obj3.is(obj4)); assert!(!obj1.is(obj3)); assert!(!obj1.is(obj5)); assert!(!obj5.is(obj6)); }); } pyo3-0.20.2/tests/test_datetime.rs000064400000000000000000000143711046102023000151500ustar 00000000000000#![cfg(not(Py_LIMITED_API))] use pyo3::prelude::*; use pyo3::types::{timezone_utc, IntoPyDict, PyDate, PyDateTime, PyTime}; use pyo3_ffi::PyDateTime_IMPORT; fn _get_subclasses<'p>( py: Python<'p>, py_type: &str, args: &str, ) -> PyResult<(&'p PyAny, &'p PyAny, &'p PyAny)> { // Import the class from Python and create some subclasses let datetime = py.import("datetime")?; let locals = [(py_type, datetime.getattr(py_type)?)].into_py_dict(py); let make_subclass_py = format!("class Subklass({}):\n pass", py_type); let make_sub_subclass_py = "class SubSubklass(Subklass):\n pass"; py.run(&make_subclass_py, None, Some(locals))?; py.run(make_sub_subclass_py, None, Some(locals))?; // Construct an instance of the base class let obj = py.eval(&format!("{}({})", py_type, args), None, Some(locals))?; // Construct an instance of the subclass let sub_obj = py.eval(&format!("Subklass({})", args), None, Some(locals))?; // Construct an instance of the sub-subclass let sub_sub_obj = py.eval(&format!("SubSubklass({})", args), None, Some(locals))?; Ok((obj, sub_obj, sub_sub_obj)) } macro_rules! assert_check_exact { ($check_func:ident, $check_func_exact:ident, $obj: expr) => { unsafe { use pyo3::ffi::*; assert!($check_func(($obj).as_ptr()) != 0); assert!($check_func_exact(($obj).as_ptr()) != 0); } }; } macro_rules! assert_check_only { ($check_func:ident, $check_func_exact:ident, $obj: expr) => { unsafe { use pyo3::ffi::*; assert!($check_func(($obj).as_ptr()) != 0); assert!($check_func_exact(($obj).as_ptr()) == 0); } }; } #[test] fn test_date_check() { Python::with_gil(|py| { let (obj, sub_obj, sub_sub_obj) = _get_subclasses(py, "date", "2018, 1, 1").unwrap(); unsafe { PyDateTime_IMPORT() } assert_check_exact!(PyDate_Check, PyDate_CheckExact, obj); assert_check_only!(PyDate_Check, PyDate_CheckExact, sub_obj); assert_check_only!(PyDate_Check, PyDate_CheckExact, sub_sub_obj); assert!(obj.is_instance_of::()); assert!(!obj.is_instance_of::()); assert!(!obj.is_instance_of::()); }); } #[test] fn test_time_check() { Python::with_gil(|py| { let (obj, sub_obj, sub_sub_obj) = _get_subclasses(py, "time", "12, 30, 15").unwrap(); unsafe { PyDateTime_IMPORT() } assert_check_exact!(PyTime_Check, PyTime_CheckExact, obj); assert_check_only!(PyTime_Check, PyTime_CheckExact, sub_obj); assert_check_only!(PyTime_Check, PyTime_CheckExact, sub_sub_obj); assert!(!obj.is_instance_of::()); assert!(obj.is_instance_of::()); assert!(!obj.is_instance_of::()); }); } #[test] fn test_datetime_check() { Python::with_gil(|py| { let (obj, sub_obj, sub_sub_obj) = _get_subclasses(py, "datetime", "2018, 1, 1, 13, 30, 15") .map_err(|e| e.display(py)) .unwrap(); unsafe { PyDateTime_IMPORT() } assert_check_only!(PyDate_Check, PyDate_CheckExact, obj); assert_check_exact!(PyDateTime_Check, PyDateTime_CheckExact, obj); assert_check_only!(PyDateTime_Check, PyDateTime_CheckExact, sub_obj); assert_check_only!(PyDateTime_Check, PyDateTime_CheckExact, sub_sub_obj); assert!(obj.is_instance_of::()); assert!(!obj.is_instance_of::()); assert!(obj.is_instance_of::()); }); } #[test] fn test_delta_check() { Python::with_gil(|py| { let (obj, sub_obj, sub_sub_obj) = _get_subclasses(py, "timedelta", "1, -3").unwrap(); unsafe { PyDateTime_IMPORT() } assert_check_exact!(PyDelta_Check, PyDelta_CheckExact, obj); assert_check_only!(PyDelta_Check, PyDelta_CheckExact, sub_obj); assert_check_only!(PyDelta_Check, PyDelta_CheckExact, sub_sub_obj); }); } #[test] fn test_datetime_utc() { use assert_approx_eq::assert_approx_eq; use pyo3::types::PyDateTime; Python::with_gil(|py| { let utc = timezone_utc(py); let dt = PyDateTime::new(py, 2018, 1, 1, 0, 0, 0, 0, Some(utc)).unwrap(); let locals = [("dt", dt)].into_py_dict(py); let offset: f32 = py .eval("dt.utcoffset().total_seconds()", None, Some(locals)) .unwrap() .extract() .unwrap(); assert_approx_eq!(offset, 0f32); }); } static INVALID_DATES: &[(i32, u8, u8)] = &[ (-1, 1, 1), (0, 1, 1), (10000, 1, 1), (2 << 30, 1, 1), (2018, 0, 1), (2018, 13, 1), (2018, 1, 0), (2017, 2, 29), (2018, 1, 32), ]; static INVALID_TIMES: &[(u8, u8, u8, u32)] = &[(25, 0, 0, 0), (255, 0, 0, 0), (0, 60, 0, 0), (0, 0, 61, 0)]; #[test] fn test_pydate_out_of_bounds() { use pyo3::types::PyDate; Python::with_gil(|py| { for val in INVALID_DATES { let (year, month, day) = val; let dt = PyDate::new(py, *year, *month, *day); dt.unwrap_err(); } }); } #[test] fn test_pytime_out_of_bounds() { use pyo3::types::PyTime; Python::with_gil(|py| { for val in INVALID_TIMES { let (hour, minute, second, microsecond) = val; let dt = PyTime::new(py, *hour, *minute, *second, *microsecond, None); dt.unwrap_err(); } }); } #[test] fn test_pydatetime_out_of_bounds() { use pyo3::types::PyDateTime; use std::iter; Python::with_gil(|py| { let valid_time = (0, 0, 0, 0); let valid_date = (2018, 1, 1); let invalid_dates = INVALID_DATES.iter().zip(iter::repeat(&valid_time)); let invalid_times = iter::repeat(&valid_date).zip(INVALID_TIMES.iter()); let vals = invalid_dates.chain(invalid_times); for val in vals { let (date, time) = val; let (year, month, day) = date; let (hour, minute, second, microsecond) = time; let dt = PyDateTime::new( py, *year, *month, *day, *hour, *minute, *second, *microsecond, None, ); dt.unwrap_err(); } }); } pyo3-0.20.2/tests/test_default_impls.rs000064400000000000000000000014521046102023000162000ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; #[path = "../src/tests/common.rs"] mod common; // Test default generated __repr__. #[pyclass] enum TestDefaultRepr { Var, } #[test] fn test_default_slot_exists() { Python::with_gil(|py| { let test_object = Py::new(py, TestDefaultRepr::Var).unwrap(); py_assert!( py, test_object, "repr(test_object) == 'TestDefaultRepr.Var'" ); }) } #[pyclass] enum OverrideSlot { Var, } #[pymethods] impl OverrideSlot { fn __repr__(&self) -> &str { "overridden" } } #[test] fn test_override_slot() { Python::with_gil(|py| { let test_object = Py::new(py, OverrideSlot::Var).unwrap(); py_assert!(py, test_object, "repr(test_object) == 'overridden'"); }) } pyo3-0.20.2/tests/test_dict_iter.rs000064400000000000000000000007451046102023000153220ustar 00000000000000use pyo3::prelude::*; use pyo3::types::IntoPyDict; #[test] #[cfg_attr(target_arch = "wasm32", ignore)] // Not sure why this fails. fn iter_dict_nosegv() { Python::with_gil(|py| { const LEN: usize = 10_000_000; let dict = (0..LEN as u64).map(|i| (i, i * 2)).into_py_dict(py); let mut sum = 0; for (k, _v) in dict { let i: u64 = k.extract().unwrap(); sum += i; } assert_eq!(sum, 49_999_995_000_000); }); } pyo3-0.20.2/tests/test_enum.rs000064400000000000000000000117311046102023000143150ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::{py_run, wrap_pyfunction}; #[path = "../src/tests/common.rs"] mod common; #[pyclass] #[derive(Debug, PartialEq, Eq, Clone)] pub enum MyEnum { Variant, OtherVariant, } #[test] fn test_enum_class_attr() { Python::with_gil(|py| { let my_enum = py.get_type::(); let var = Py::new(py, MyEnum::Variant).unwrap(); py_assert!(py, my_enum var, "my_enum.Variant == var"); }) } #[pyfunction] fn return_enum() -> MyEnum { MyEnum::Variant } #[test] fn test_return_enum() { Python::with_gil(|py| { let f = wrap_pyfunction!(return_enum)(py).unwrap(); let mynum = py.get_type::(); py_run!(py, f mynum, "assert f() == mynum.Variant") }); } #[pyfunction] fn enum_arg(e: MyEnum) { assert_eq!(MyEnum::OtherVariant, e) } #[test] fn test_enum_arg() { Python::with_gil(|py| { let f = wrap_pyfunction!(enum_arg)(py).unwrap(); let mynum = py.get_type::(); py_run!(py, f mynum, "f(mynum.OtherVariant)") }) } #[test] fn test_enum_eq_enum() { Python::with_gil(|py| { let var1 = Py::new(py, MyEnum::Variant).unwrap(); let var2 = Py::new(py, MyEnum::Variant).unwrap(); let other_var = Py::new(py, MyEnum::OtherVariant).unwrap(); py_assert!(py, var1 var2, "var1 == var2"); py_assert!(py, var1 other_var, "var1 != other_var"); py_assert!(py, var1 var2, "(var1 != var2) == False"); }) } #[test] fn test_enum_eq_incomparable() { Python::with_gil(|py| { let var1 = Py::new(py, MyEnum::Variant).unwrap(); py_assert!(py, var1, "(var1 == 'foo') == False"); py_assert!(py, var1, "(var1 != 'foo') == True"); }) } #[pyclass] enum CustomDiscriminant { One = 1, Two = 2, } #[test] fn test_custom_discriminant() { Python::with_gil(|py| { #[allow(non_snake_case)] let CustomDiscriminant = py.get_type::(); let one = Py::new(py, CustomDiscriminant::One).unwrap(); let two = Py::new(py, CustomDiscriminant::Two).unwrap(); py_run!(py, CustomDiscriminant one two, r#" assert CustomDiscriminant.One == one assert CustomDiscriminant.Two == two assert one != two "#); }) } #[test] fn test_enum_to_int() { Python::with_gil(|py| { let one = Py::new(py, CustomDiscriminant::One).unwrap(); py_assert!(py, one, "int(one) == 1"); let v = Py::new(py, MyEnum::Variant).unwrap(); let v_value = MyEnum::Variant as isize; py_run!(py, v v_value, "int(v) == v_value"); }) } #[test] fn test_enum_compare_int() { Python::with_gil(|py| { let one = Py::new(py, CustomDiscriminant::One).unwrap(); py_run!( py, one, r#" assert one == 1 assert 1 == one assert one != 2 "# ) }) } #[pyclass] #[repr(u8)] enum SmallEnum { V = 1, } #[test] fn test_enum_compare_int_no_throw_when_overflow() { Python::with_gil(|py| { let v = Py::new(py, SmallEnum::V).unwrap(); py_assert!(py, v, "v != 1<<30") }) } #[pyclass] #[repr(usize)] #[allow(clippy::enum_clike_unportable_variant)] enum BigEnum { V = usize::MAX, } #[test] fn test_big_enum_no_overflow() { Python::with_gil(|py| { let usize_max = usize::MAX; let v = Py::new(py, BigEnum::V).unwrap(); py_assert!(py, usize_max v, "v == usize_max"); py_assert!(py, usize_max v, "int(v) == usize_max"); }) } #[pyclass] #[repr(u16, align(8))] enum TestReprParse { V, } #[test] fn test_repr_parse() { assert_eq!(std::mem::align_of::(), 8); } #[pyclass(name = "MyEnum")] #[derive(Debug, PartialEq, Eq, Clone)] pub enum RenameEnum { Variant, } #[test] fn test_rename_enum_repr_correct() { Python::with_gil(|py| { let var1 = Py::new(py, RenameEnum::Variant).unwrap(); py_assert!(py, var1, "repr(var1) == 'MyEnum.Variant'"); }) } #[pyclass] #[derive(Debug, PartialEq, Eq, Clone)] pub enum RenameVariantEnum { #[pyo3(name = "VARIANT")] Variant, } #[test] fn test_rename_variant_repr_correct() { Python::with_gil(|py| { let var1 = Py::new(py, RenameVariantEnum::Variant).unwrap(); py_assert!(py, var1, "repr(var1) == 'RenameVariantEnum.VARIANT'"); }) } #[pyclass(rename_all = "SCREAMING_SNAKE_CASE")] #[allow(clippy::enum_variant_names)] enum RenameAllVariantsEnum { VariantOne, VariantTwo, #[pyo3(name = "VariantThree")] VariantFour, } #[test] fn test_renaming_all_enum_variants() { Python::with_gil(|py| { let enum_obj = py.get_type::(); py_assert!(py, enum_obj, "enum_obj.VARIANT_ONE == enum_obj.VARIANT_ONE"); py_assert!(py, enum_obj, "enum_obj.VARIANT_TWO == enum_obj.VARIANT_TWO"); py_assert!( py, enum_obj, "enum_obj.VariantThree == enum_obj.VariantThree" ); }); } pyo3-0.20.2/tests/test_exceptions.rs000064400000000000000000000060671046102023000155400ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::{exceptions, py_run, PyErr, PyResult}; use std::error::Error; use std::fmt; #[cfg(not(target_os = "windows"))] use std::fs::File; #[path = "../src/tests/common.rs"] mod common; #[pyfunction] #[cfg(not(target_os = "windows"))] fn fail_to_open_file() -> PyResult<()> { File::open("not_there.txt")?; Ok(()) } #[test] #[cfg_attr(target_arch = "wasm32", ignore)] // Not sure why this fails. #[cfg(not(target_os = "windows"))] fn test_filenotfounderror() { Python::with_gil(|py| { let fail_to_open_file = wrap_pyfunction!(fail_to_open_file)(py).unwrap(); py_run!( py, fail_to_open_file, r#" try: fail_to_open_file() except FileNotFoundError as e: assert str(e) == "No such file or directory (os error 2)" "# ); }); } #[derive(Debug)] struct CustomError; impl Error for CustomError {} impl fmt::Display for CustomError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "Oh no!") } } impl std::convert::From for PyErr { fn from(err: CustomError) -> PyErr { exceptions::PyOSError::new_err(err.to_string()) } } fn fail_with_custom_error() -> Result<(), CustomError> { Err(CustomError) } #[pyfunction] fn call_fail_with_custom_error() -> PyResult<()> { fail_with_custom_error()?; Ok(()) } #[test] fn test_custom_error() { Python::with_gil(|py| { let call_fail_with_custom_error = wrap_pyfunction!(call_fail_with_custom_error)(py).unwrap(); py_run!( py, call_fail_with_custom_error, r#" try: call_fail_with_custom_error() except OSError as e: assert str(e) == "Oh no!" "# ); }); } #[test] fn test_exception_nosegfault() { use std::net::TcpListener; fn io_err() -> PyResult<()> { TcpListener::bind("no:address")?; Ok(()) } fn parse_int() -> PyResult<()> { "@_@".parse::()?; Ok(()) } assert!(io_err().is_err()); assert!(parse_int().is_err()); } #[test] #[cfg(Py_3_8)] fn test_write_unraisable() { use common::UnraisableCapture; use pyo3::{exceptions::PyRuntimeError, ffi}; Python::with_gil(|py| { let capture = UnraisableCapture::install(py); assert!(capture.borrow(py).capture.is_none()); let err = PyRuntimeError::new_err("foo"); err.write_unraisable(py, None); let (err, object) = capture.borrow_mut(py).capture.take().unwrap(); assert_eq!(err.to_string(), "RuntimeError: foo"); assert!(object.is_none(py)); let err = PyRuntimeError::new_err("bar"); err.write_unraisable(py, Some(py.NotImplemented().as_ref(py))); let (err, object) = capture.borrow_mut(py).capture.take().unwrap(); assert_eq!(err.to_string(), "RuntimeError: bar"); assert!(object.as_ptr() == unsafe { ffi::Py_NotImplemented() }); capture.borrow_mut(py).uninstall(py); }); } pyo3-0.20.2/tests/test_field_cfg.rs000064400000000000000000000011761046102023000152550ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; #[pyclass] struct CfgClass { #[pyo3(get, set)] #[cfg(any())] pub a: u32, #[pyo3(get, set)] // This is always true #[cfg(any( target_family = "unix", target_family = "windows", target_family = "wasm" ))] pub b: u32, } #[test] fn test_cfg() { Python::with_gil(|py| { let cfg = CfgClass { b: 3 }; let py_cfg = Py::new(py, cfg).unwrap(); assert!(py_cfg.as_ref(py).getattr("a").is_err()); let b: u32 = py_cfg.as_ref(py).getattr("b").unwrap().extract().unwrap(); assert_eq!(b, 3); }); } pyo3-0.20.2/tests/test_frompyobject.rs000064400000000000000000000416161046102023000160610ustar 00000000000000#![cfg(feature = "macros")] use pyo3::exceptions::PyValueError; use pyo3::prelude::*; use pyo3::types::{PyDict, PyList, PyString, PyTuple}; #[macro_use] #[path = "../src/tests/common.rs"] mod common; /// Helper function that concatenates the error message from /// each error in the traceback into a single string that can /// be tested. fn extract_traceback(py: Python<'_>, mut error: PyErr) -> String { let mut error_msg = error.to_string(); while let Some(cause) = error.cause(py) { error_msg.push_str(": "); error_msg.push_str(&cause.to_string()); error = cause } error_msg } #[derive(Debug, FromPyObject)] pub struct A<'a> { #[pyo3(attribute)] s: String, #[pyo3(item)] t: &'a PyString, #[pyo3(attribute("foo"))] p: &'a PyAny, } #[pyclass] pub struct PyA { #[pyo3(get)] s: String, #[pyo3(get)] foo: Option, } #[pymethods] impl PyA { fn __getitem__(&self, key: String) -> pyo3::PyResult { if key == "t" { Ok("bar".into()) } else { Err(PyValueError::new_err("Failed")) } } } #[test] fn test_named_fields_struct() { Python::with_gil(|py| { let pya = PyA { s: "foo".into(), foo: None, }; let py_c = Py::new(py, pya).unwrap(); let a: A<'_> = FromPyObject::extract(py_c.as_ref(py)).expect("Failed to extract A from PyA"); assert_eq!(a.s, "foo"); assert_eq!(a.t.to_string_lossy(), "bar"); assert!(a.p.is_none()); }); } #[derive(Debug, FromPyObject)] #[pyo3(transparent)] pub struct B { test: String, } #[test] fn test_transparent_named_field_struct() { Python::with_gil(|py| { let test: PyObject = "test".into_py(py); let b: B = FromPyObject::extract(test.as_ref(py)).expect("Failed to extract B from String"); assert_eq!(b.test, "test"); let test: PyObject = 1.into_py(py); let b = B::extract(test.as_ref(py)); assert!(b.is_err()); }); } #[derive(Debug, FromPyObject)] #[pyo3(transparent)] pub struct D { test: T, } #[test] fn test_generic_transparent_named_field_struct() { Python::with_gil(|py| { let test: PyObject = "test".into_py(py); let d: D = D::extract(test.as_ref(py)).expect("Failed to extract D from String"); assert_eq!(d.test, "test"); let test = 1usize.into_py(py); let d: D = D::extract(test.as_ref(py)).expect("Failed to extract D from String"); assert_eq!(d.test, 1); }); } #[derive(Debug, FromPyObject)] pub struct E { test: T, test2: T2, } #[pyclass] #[derive(Clone)] pub struct PyE { #[pyo3(get)] test: String, #[pyo3(get)] test2: usize, } #[test] fn test_generic_named_fields_struct() { Python::with_gil(|py| { let pye = PyE { test: "test".into(), test2: 2, } .into_py(py); let e: E = E::extract(pye.as_ref(py)).expect("Failed to extract E from PyE"); assert_eq!(e.test, "test"); assert_eq!(e.test2, 2); let e = E::::extract(pye.as_ref(py)); assert!(e.is_err()); }); } #[derive(Debug, FromPyObject)] pub struct C { #[pyo3(attribute("test"))] test: String, } #[test] fn test_named_field_with_ext_fn() { Python::with_gil(|py| { let pyc = PyE { test: "foo".into(), test2: 0, } .into_py(py); let c = C::extract(pyc.as_ref(py)).expect("Failed to extract C from PyE"); assert_eq!(c.test, "foo"); }); } #[derive(Debug, FromPyObject)] pub struct Tuple(String, usize); #[test] fn test_tuple_struct() { Python::with_gil(|py| { let tup = PyTuple::new(py, &[1.into_py(py), "test".into_py(py)]); let tup = Tuple::extract(tup.as_ref()); assert!(tup.is_err()); let tup = PyTuple::new(py, &["test".into_py(py), 1.into_py(py)]); let tup = Tuple::extract(tup.as_ref()).expect("Failed to extract Tuple from PyTuple"); assert_eq!(tup.0, "test"); assert_eq!(tup.1, 1); }); } #[derive(Debug, FromPyObject)] pub struct TransparentTuple(String); #[test] fn test_transparent_tuple_struct() { Python::with_gil(|py| { let tup: PyObject = 1.into_py(py); let tup = TransparentTuple::extract(tup.as_ref(py)); assert!(tup.is_err()); let test: PyObject = "test".into_py(py); let tup = TransparentTuple::extract(test.as_ref(py)) .expect("Failed to extract TransparentTuple from PyTuple"); assert_eq!(tup.0, "test"); }); } #[pyclass] struct PyBaz { #[pyo3(get)] tup: (String, String), #[pyo3(get)] e: PyE, } #[derive(Debug, FromPyObject)] #[allow(dead_code)] struct Baz { e: E, tup: Tuple, } #[test] fn test_struct_nested_type_errors() { Python::with_gil(|py| { let pybaz = PyBaz { tup: ("test".into(), "test".into()), e: PyE { test: "foo".into(), test2: 0, }, } .into_py(py); let test: PyResult> = FromPyObject::extract(pybaz.as_ref(py)); assert!(test.is_err()); assert_eq!( extract_traceback(py,test.unwrap_err()), "TypeError: failed to extract field Baz.tup: TypeError: failed to extract field Tuple.1: \ TypeError: \'str\' object cannot be interpreted as an integer" ); }); } #[test] fn test_struct_nested_type_errors_with_generics() { Python::with_gil(|py| { let pybaz = PyBaz { tup: ("test".into(), "test".into()), e: PyE { test: "foo".into(), test2: 0, }, } .into_py(py); let test: PyResult> = FromPyObject::extract(pybaz.as_ref(py)); assert!(test.is_err()); assert_eq!( extract_traceback(py, test.unwrap_err()), "TypeError: failed to extract field Baz.e: TypeError: failed to extract field E.test: \ TypeError: \'str\' object cannot be interpreted as an integer", ); }); } #[test] fn test_transparent_struct_error_message() { Python::with_gil(|py| { let tup: PyObject = 1.into_py(py); let tup = B::extract(tup.as_ref(py)); assert!(tup.is_err()); assert_eq!( extract_traceback(py,tup.unwrap_err()), "TypeError: failed to extract field B.test: TypeError: \'int\' object cannot be converted \ to \'PyString\'" ); }); } #[test] fn test_tuple_struct_error_message() { Python::with_gil(|py| { let tup: PyObject = (1, "test").into_py(py); let tup = Tuple::extract(tup.as_ref(py)); assert!(tup.is_err()); assert_eq!( extract_traceback(py, tup.unwrap_err()), "TypeError: failed to extract field Tuple.0: TypeError: \'int\' object cannot be \ converted to \'PyString\'" ); }); } #[test] fn test_transparent_tuple_error_message() { Python::with_gil(|py| { let tup: PyObject = 1.into_py(py); let tup = TransparentTuple::extract(tup.as_ref(py)); assert!(tup.is_err()); assert_eq!( extract_traceback(py, tup.unwrap_err()), "TypeError: failed to extract field TransparentTuple.0: TypeError: 'int' object \ cannot be converted to 'PyString'", ); }); } #[derive(Debug, FromPyObject)] pub enum Foo<'a> { TupleVar(usize, String), StructVar { test: &'a PyString, }, #[pyo3(transparent)] TransparentTuple(usize), #[pyo3(transparent)] TransparentStructVar { a: Option, }, StructVarGetAttrArg { #[pyo3(attribute("bla"))] a: bool, }, StructWithGetItem { #[pyo3(item)] a: String, }, StructWithGetItemArg { #[pyo3(item("foo"))] a: String, }, } #[pyclass] pub struct PyBool { #[pyo3(get)] bla: bool, } #[test] fn test_enum() { Python::with_gil(|py| { let tup = PyTuple::new(py, &[1.into_py(py), "test".into_py(py)]); let f = Foo::extract(tup.as_ref()).expect("Failed to extract Foo from tuple"); match f { Foo::TupleVar(test, test2) => { assert_eq!(test, 1); assert_eq!(test2, "test"); } _ => panic!("Expected extracting Foo::TupleVar, got {:?}", f), } let pye = PyE { test: "foo".into(), test2: 0, } .into_py(py); let f = Foo::extract(pye.as_ref(py)).expect("Failed to extract Foo from PyE"); match f { Foo::StructVar { test } => assert_eq!(test.to_string_lossy(), "foo"), _ => panic!("Expected extracting Foo::StructVar, got {:?}", f), } let int: PyObject = 1.into_py(py); let f = Foo::extract(int.as_ref(py)).expect("Failed to extract Foo from int"); match f { Foo::TransparentTuple(test) => assert_eq!(test, 1), _ => panic!("Expected extracting Foo::TransparentTuple, got {:?}", f), } let none = py.None(); let f = Foo::extract(none.as_ref(py)).expect("Failed to extract Foo from int"); match f { Foo::TransparentStructVar { a } => assert!(a.is_none()), _ => panic!("Expected extracting Foo::TransparentStructVar, got {:?}", f), } let pybool = PyBool { bla: true }.into_py(py); let f = Foo::extract(pybool.as_ref(py)).expect("Failed to extract Foo from PyBool"); match f { Foo::StructVarGetAttrArg { a } => assert!(a), _ => panic!("Expected extracting Foo::StructVarGetAttrArg, got {:?}", f), } let dict = PyDict::new(py); dict.set_item("a", "test").expect("Failed to set item"); let f = Foo::extract(dict.as_ref()).expect("Failed to extract Foo from dict"); match f { Foo::StructWithGetItem { a } => assert_eq!(a, "test"), _ => panic!("Expected extracting Foo::StructWithGetItem, got {:?}", f), } let dict = PyDict::new(py); dict.set_item("foo", "test").expect("Failed to set item"); let f = Foo::extract(dict.as_ref()).expect("Failed to extract Foo from dict"); match f { Foo::StructWithGetItemArg { a } => assert_eq!(a, "test"), _ => panic!("Expected extracting Foo::StructWithGetItemArg, got {:?}", f), } }); } #[test] fn test_enum_error() { Python::with_gil(|py| { let dict = PyDict::new(py); let err = Foo::extract(dict.as_ref()).unwrap_err(); assert_eq!( err.to_string(), "\ TypeError: failed to extract enum Foo ('TupleVar | StructVar | TransparentTuple | TransparentStructVar | StructVarGetAttrArg | StructWithGetItem | StructWithGetItemArg') - variant TupleVar (TupleVar): TypeError: 'dict' object cannot be converted to 'PyTuple' - variant StructVar (StructVar): AttributeError: 'dict' object has no attribute 'test' - variant TransparentTuple (TransparentTuple): TypeError: failed to extract field Foo::TransparentTuple.0, caused by TypeError: 'dict' object cannot be interpreted as an integer - variant TransparentStructVar (TransparentStructVar): TypeError: failed to extract field Foo::TransparentStructVar.a, caused by TypeError: 'dict' object cannot be converted to 'PyString' - variant StructVarGetAttrArg (StructVarGetAttrArg): AttributeError: 'dict' object has no attribute 'bla' - variant StructWithGetItem (StructWithGetItem): KeyError: 'a' - variant StructWithGetItemArg (StructWithGetItemArg): KeyError: 'foo'" ); let tup = PyTuple::empty(py); let err = Foo::extract(tup.as_ref()).unwrap_err(); assert_eq!( err.to_string(), "\ TypeError: failed to extract enum Foo ('TupleVar | StructVar | TransparentTuple | TransparentStructVar | StructVarGetAttrArg | StructWithGetItem | StructWithGetItemArg') - variant TupleVar (TupleVar): ValueError: expected tuple of length 2, but got tuple of length 0 - variant StructVar (StructVar): AttributeError: 'tuple' object has no attribute 'test' - variant TransparentTuple (TransparentTuple): TypeError: failed to extract field Foo::TransparentTuple.0, caused by TypeError: 'tuple' object cannot be interpreted as an integer - variant TransparentStructVar (TransparentStructVar): TypeError: failed to extract field Foo::TransparentStructVar.a, caused by TypeError: 'tuple' object cannot be converted to 'PyString' - variant StructVarGetAttrArg (StructVarGetAttrArg): AttributeError: 'tuple' object has no attribute 'bla' - variant StructWithGetItem (StructWithGetItem): TypeError: tuple indices must be integers or slices, not str - variant StructWithGetItemArg (StructWithGetItemArg): TypeError: tuple indices must be integers or slices, not str" ); }); } #[derive(Debug, FromPyObject)] enum EnumWithCatchAll<'a> { #[pyo3(transparent)] Foo(Foo<'a>), #[pyo3(transparent)] CatchAll(&'a PyAny), } #[test] fn test_enum_catch_all() { Python::with_gil(|py| { let dict = PyDict::new(py); let f = EnumWithCatchAll::extract(dict.as_ref()) .expect("Failed to extract EnumWithCatchAll from dict"); match f { EnumWithCatchAll::CatchAll(any) => { let d = <&PyDict>::extract(any).expect("Expected pydict"); assert!(d.is_empty()); } _ => panic!( "Expected extracting EnumWithCatchAll::CatchAll, got {:?}", f ), } }); } #[derive(Debug, FromPyObject)] pub enum Bar { #[pyo3(annotation = "str")] A(String), #[pyo3(annotation = "uint")] B(usize), #[pyo3(annotation = "int", transparent)] C(isize), } #[test] fn test_err_rename() { Python::with_gil(|py| { let dict = PyDict::new(py); let f = Bar::extract(dict.as_ref()); assert!(f.is_err()); assert_eq!( f.unwrap_err().to_string(), "\ TypeError: failed to extract enum Bar ('str | uint | int') - variant A (str): TypeError: failed to extract field Bar::A.0, caused by TypeError: 'dict' object cannot be converted to 'PyString' - variant B (uint): TypeError: failed to extract field Bar::B.0, caused by TypeError: 'dict' object cannot be interpreted as an integer - variant C (int): TypeError: failed to extract field Bar::C.0, caused by TypeError: 'dict' object cannot be interpreted as an integer" ); }); } #[derive(Debug, FromPyObject)] pub struct Zap { #[pyo3(item)] name: String, #[pyo3(from_py_with = "PyAny::len", item("my_object"))] some_object_length: usize, } #[test] fn test_from_py_with() { Python::with_gil(|py| { let py_zap = py .eval( r#"{"name": "whatever", "my_object": [1, 2, 3]}"#, None, None, ) .expect("failed to create dict"); let zap = Zap::extract(py_zap).unwrap(); assert_eq!(zap.name, "whatever"); assert_eq!(zap.some_object_length, 3usize); }); } #[derive(Debug, FromPyObject)] pub struct ZapTuple(String, #[pyo3(from_py_with = "PyAny::len")] usize); #[test] fn test_from_py_with_tuple_struct() { Python::with_gil(|py| { let py_zap = py .eval(r#"("whatever", [1, 2, 3])"#, None, None) .expect("failed to create tuple"); let zap = ZapTuple::extract(py_zap).unwrap(); assert_eq!(zap.0, "whatever"); assert_eq!(zap.1, 3usize); }); } #[test] fn test_from_py_with_tuple_struct_error() { Python::with_gil(|py| { let py_zap = py .eval(r#"("whatever", [1, 2, 3], "third")"#, None, None) .expect("failed to create tuple"); let f = ZapTuple::extract(py_zap); assert!(f.is_err()); assert_eq!( f.unwrap_err().to_string(), "ValueError: expected tuple of length 2, but got tuple of length 3" ); }); } #[derive(Debug, FromPyObject, PartialEq, Eq)] pub enum ZapEnum { Zip(#[pyo3(from_py_with = "PyAny::len")] usize), Zap(String, #[pyo3(from_py_with = "PyAny::len")] usize), } #[test] fn test_from_py_with_enum() { Python::with_gil(|py| { let py_zap = py .eval(r#"("whatever", [1, 2, 3])"#, None, None) .expect("failed to create tuple"); let zap = ZapEnum::extract(py_zap).unwrap(); let expected_zap = ZapEnum::Zip(2); assert_eq!(zap, expected_zap); }); } #[derive(Debug, FromPyObject, PartialEq, Eq)] #[pyo3(transparent)] pub struct TransparentFromPyWith { #[pyo3(from_py_with = "PyAny::len")] len: usize, } #[test] fn test_transparent_from_py_with() { Python::with_gil(|py| { let result = TransparentFromPyWith::extract(PyList::new(py, [1, 2, 3])).unwrap(); let expected = TransparentFromPyWith { len: 3 }; assert_eq!(result, expected); }); } pyo3-0.20.2/tests/test_gc.rs000064400000000000000000000324621046102023000137460ustar 00000000000000#![cfg(feature = "macros")] use pyo3::class::PyTraverseError; use pyo3::class::PyVisit; use pyo3::prelude::*; use pyo3::{py_run, PyCell, PyTryInto}; use std::cell::Cell; use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::Arc; #[path = "../src/tests/common.rs"] mod common; #[pyclass(freelist = 2)] struct ClassWithFreelist {} #[test] fn class_with_freelist() { let ptr = Python::with_gil(|py| { let inst = Py::new(py, ClassWithFreelist {}).unwrap(); let _inst2 = Py::new(py, ClassWithFreelist {}).unwrap(); let ptr = inst.as_ptr(); drop(inst); ptr }); Python::with_gil(|py| { let inst3 = Py::new(py, ClassWithFreelist {}).unwrap(); assert_eq!(ptr, inst3.as_ptr()); let inst4 = Py::new(py, ClassWithFreelist {}).unwrap(); assert_ne!(ptr, inst4.as_ptr()) }); } struct TestDropCall { drop_called: Arc, } impl Drop for TestDropCall { fn drop(&mut self) { self.drop_called.store(true, Ordering::Relaxed); } } #[allow(dead_code)] #[pyclass] struct DataIsDropped { member1: TestDropCall, member2: TestDropCall, } #[test] fn data_is_dropped() { let drop_called1 = Arc::new(AtomicBool::new(false)); let drop_called2 = Arc::new(AtomicBool::new(false)); Python::with_gil(|py| { let data_is_dropped = DataIsDropped { member1: TestDropCall { drop_called: Arc::clone(&drop_called1), }, member2: TestDropCall { drop_called: Arc::clone(&drop_called2), }, }; let inst = Py::new(py, data_is_dropped).unwrap(); assert!(!drop_called1.load(Ordering::Relaxed)); assert!(!drop_called2.load(Ordering::Relaxed)); drop(inst); }); assert!(drop_called1.load(Ordering::Relaxed)); assert!(drop_called2.load(Ordering::Relaxed)); } #[allow(dead_code)] #[pyclass] struct GcIntegration { self_ref: PyObject, dropped: TestDropCall, } #[pymethods] impl GcIntegration { fn __traverse__(&self, visit: PyVisit<'_>) -> Result<(), PyTraverseError> { visit.call(&self.self_ref) } fn __clear__(&mut self) { Python::with_gil(|py| { self.self_ref = py.None(); }); } } #[test] fn gc_integration() { let drop_called = Arc::new(AtomicBool::new(false)); Python::with_gil(|py| { let inst = PyCell::new( py, GcIntegration { self_ref: py.None(), dropped: TestDropCall { drop_called: Arc::clone(&drop_called), }, }, ) .unwrap(); let mut borrow = inst.borrow_mut(); borrow.self_ref = inst.to_object(py); py_run!(py, inst, "import gc; assert inst in gc.get_objects()"); }); Python::with_gil(|py| { py.run("import gc; gc.collect()", None, None).unwrap(); assert!(drop_called.load(Ordering::Relaxed)); }); } #[pyclass] struct GcNullTraversal { cycle: Option>, null: Option>, } #[pymethods] impl GcNullTraversal { fn __traverse__(&self, visit: PyVisit<'_>) -> Result<(), PyTraverseError> { visit.call(&self.cycle)?; visit.call(&self.null)?; // Should not segfault Ok(()) } fn __clear__(&mut self) { self.cycle = None; self.null = None; } } #[test] fn gc_null_traversal() { Python::with_gil(|py| { let obj = Py::new( py, GcNullTraversal { cycle: None, null: None, }, ) .unwrap(); obj.borrow_mut(py).cycle = Some(obj.clone_ref(py)); // the object doesn't have to be cleaned up, it just needs to be traversed. py.run("import gc; gc.collect()", None, None).unwrap(); }); } #[pyclass(subclass)] struct BaseClassWithDrop { data: Option>, } #[pymethods] impl BaseClassWithDrop { #[new] fn new() -> BaseClassWithDrop { BaseClassWithDrop { data: None } } } impl Drop for BaseClassWithDrop { fn drop(&mut self) { if let Some(data) = &self.data { data.store(true, Ordering::Relaxed); } } } #[pyclass(extends = BaseClassWithDrop)] struct SubClassWithDrop { data: Option>, } #[pymethods] impl SubClassWithDrop { #[new] fn new() -> (Self, BaseClassWithDrop) { ( SubClassWithDrop { data: None }, BaseClassWithDrop { data: None }, ) } } impl Drop for SubClassWithDrop { fn drop(&mut self) { if let Some(data) = &self.data { data.store(true, Ordering::Relaxed); } } } #[test] fn inheritance_with_new_methods_with_drop() { let drop_called1 = Arc::new(AtomicBool::new(false)); let drop_called2 = Arc::new(AtomicBool::new(false)); Python::with_gil(|py| { let _typebase = py.get_type::(); let typeobj = py.get_type::(); let inst = typeobj.call((), None).unwrap(); let obj: &PyCell = PyTryInto::try_into(inst).unwrap(); let mut obj_ref_mut = obj.borrow_mut(); obj_ref_mut.data = Some(Arc::clone(&drop_called1)); let base: &mut BaseClassWithDrop = obj_ref_mut.as_mut(); base.data = Some(Arc::clone(&drop_called2)); }); assert!(drop_called1.load(Ordering::Relaxed)); assert!(drop_called2.load(Ordering::Relaxed)); } #[pyclass] struct TraversableClass { traversed: AtomicBool, } impl TraversableClass { fn new() -> Self { Self { traversed: AtomicBool::new(false), } } } #[pymethods] impl TraversableClass { fn __clear__(&mut self) {} #[allow(clippy::unnecessary_wraps)] fn __traverse__(&self, _visit: PyVisit<'_>) -> Result<(), PyTraverseError> { self.traversed.store(true, Ordering::Relaxed); Ok(()) } } #[test] fn gc_during_borrow() { Python::with_gil(|py| { unsafe { // get the traverse function let ty = py.get_type::().as_type_ptr(); let traverse = get_type_traverse(ty).unwrap(); // create an object and check that traversing it works normally // when it's not borrowed let cell = PyCell::new(py, TraversableClass::new()).unwrap(); let obj = cell.to_object(py); assert!(!cell.borrow().traversed.load(Ordering::Relaxed)); traverse(obj.as_ptr(), novisit, std::ptr::null_mut()); assert!(cell.borrow().traversed.load(Ordering::Relaxed)); // create an object and check that it is not traversed if the GC // is invoked while it is already borrowed mutably let cell2 = PyCell::new(py, TraversableClass::new()).unwrap(); let obj2 = cell2.to_object(py); let guard = cell2.borrow_mut(); assert!(!guard.traversed.load(Ordering::Relaxed)); traverse(obj2.as_ptr(), novisit, std::ptr::null_mut()); assert!(!guard.traversed.load(Ordering::Relaxed)); drop(guard); } }); } #[pyclass] struct PartialTraverse { member: PyObject, } impl PartialTraverse { fn new(py: Python<'_>) -> Self { Self { member: py.None() } } } #[pymethods] impl PartialTraverse { fn __traverse__(&self, visit: PyVisit<'_>) -> Result<(), PyTraverseError> { visit.call(&self.member)?; // In the test, we expect this to never be hit unreachable!() } } #[test] fn traverse_partial() { Python::with_gil(|py| unsafe { // get the traverse function let ty = py.get_type::().as_type_ptr(); let traverse = get_type_traverse(ty).unwrap(); // confirm that traversing errors let obj = Py::new(py, PartialTraverse::new(py)).unwrap(); assert_eq!( traverse(obj.as_ptr(), visit_error, std::ptr::null_mut()), -1 ); }) } #[pyclass] struct PanickyTraverse { member: PyObject, } impl PanickyTraverse { fn new(py: Python<'_>) -> Self { Self { member: py.None() } } } #[pymethods] impl PanickyTraverse { fn __traverse__(&self, visit: PyVisit<'_>) -> Result<(), PyTraverseError> { visit.call(&self.member)?; panic!("at the disco"); } } #[test] fn traverse_panic() { Python::with_gil(|py| unsafe { // get the traverse function let ty = py.get_type::().as_type_ptr(); let traverse = get_type_traverse(ty).unwrap(); // confirm that traversing errors let obj = Py::new(py, PanickyTraverse::new(py)).unwrap(); assert_eq!(traverse(obj.as_ptr(), novisit, std::ptr::null_mut()), -1); }) } #[pyclass] struct TriesGILInTraverse {} #[pymethods] impl TriesGILInTraverse { fn __traverse__(&self, _visit: PyVisit<'_>) -> Result<(), PyTraverseError> { Python::with_gil(|_py| Ok(())) } } #[test] fn tries_gil_in_traverse() { Python::with_gil(|py| unsafe { // get the traverse function let ty = py.get_type::().as_type_ptr(); let traverse = get_type_traverse(ty).unwrap(); // confirm that traversing panicks let obj = Py::new(py, TriesGILInTraverse {}).unwrap(); assert_eq!(traverse(obj.as_ptr(), novisit, std::ptr::null_mut()), -1); }) } #[pyclass] struct HijackedTraverse { traversed: Cell, hijacked: Cell, } impl HijackedTraverse { fn new() -> Self { Self { traversed: Cell::new(false), hijacked: Cell::new(false), } } fn traversed_and_hijacked(&self) -> (bool, bool) { (self.traversed.get(), self.hijacked.get()) } } #[pymethods] impl HijackedTraverse { #[allow(clippy::unnecessary_wraps)] fn __traverse__(&self, _visit: PyVisit<'_>) -> Result<(), PyTraverseError> { self.traversed.set(true); Ok(()) } } trait Traversable { fn __traverse__(&self, visit: PyVisit<'_>) -> Result<(), PyTraverseError>; } impl<'a> Traversable for PyRef<'a, HijackedTraverse> { fn __traverse__(&self, _visit: PyVisit<'_>) -> Result<(), PyTraverseError> { self.hijacked.set(true); Ok(()) } } #[test] fn traverse_cannot_be_hijacked() { Python::with_gil(|py| unsafe { // get the traverse function let ty = py.get_type::().as_type_ptr(); let traverse = get_type_traverse(ty).unwrap(); let cell = PyCell::new(py, HijackedTraverse::new()).unwrap(); let obj = cell.to_object(py); assert_eq!(cell.borrow().traversed_and_hijacked(), (false, false)); traverse(obj.as_ptr(), novisit, std::ptr::null_mut()); assert_eq!(cell.borrow().traversed_and_hijacked(), (true, false)); }) } #[allow(dead_code)] #[pyclass] struct DropDuringTraversal { cycle: Cell>>, dropped: TestDropCall, } #[pymethods] impl DropDuringTraversal { #[allow(clippy::unnecessary_wraps)] fn __traverse__(&self, _visit: PyVisit<'_>) -> Result<(), PyTraverseError> { self.cycle.take(); Ok(()) } fn __clear__(&mut self) { self.cycle.take(); } } #[test] fn drop_during_traversal_with_gil() { let drop_called = Arc::new(AtomicBool::new(false)); Python::with_gil(|py| { let inst = Py::new( py, DropDuringTraversal { cycle: Cell::new(None), dropped: TestDropCall { drop_called: Arc::clone(&drop_called), }, }, ) .unwrap(); inst.borrow_mut(py).cycle.set(Some(inst.clone_ref(py))); drop(inst); }); // due to the internal GC mechanism, we may need multiple // (but not too many) collections to get `inst` actually dropped. for _ in 0..10 { Python::with_gil(|py| { py.run("import gc; gc.collect()", None, None).unwrap(); }); } assert!(drop_called.load(Ordering::Relaxed)); } #[test] fn drop_during_traversal_without_gil() { let drop_called = Arc::new(AtomicBool::new(false)); let inst = Python::with_gil(|py| { let inst = Py::new( py, DropDuringTraversal { cycle: Cell::new(None), dropped: TestDropCall { drop_called: Arc::clone(&drop_called), }, }, ) .unwrap(); inst.borrow_mut(py).cycle.set(Some(inst.clone_ref(py))); inst }); drop(inst); // due to the internal GC mechanism, we may need multiple // (but not too many) collections to get `inst` actually dropped. for _ in 0..10 { Python::with_gil(|py| { py.run("import gc; gc.collect()", None, None).unwrap(); }); } assert!(drop_called.load(Ordering::Relaxed)); } // Manual traversal utilities unsafe fn get_type_traverse(tp: *mut pyo3::ffi::PyTypeObject) -> Option { std::mem::transmute(pyo3::ffi::PyType_GetSlot(tp, pyo3::ffi::Py_tp_traverse)) } // a dummy visitor function extern "C" fn novisit( _object: *mut pyo3::ffi::PyObject, _arg: *mut core::ffi::c_void, ) -> std::os::raw::c_int { 0 } // a visitor function which errors (returns nonzero code) extern "C" fn visit_error( _object: *mut pyo3::ffi::PyObject, _arg: *mut core::ffi::c_void, ) -> std::os::raw::c_int { -1 } pyo3-0.20.2/tests/test_getter_setter.rs000064400000000000000000000121171046102023000162300ustar 00000000000000#![cfg(feature = "macros")] use std::cell::Cell; use pyo3::prelude::*; use pyo3::py_run; use pyo3::types::{IntoPyDict, PyList}; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct ClassWithProperties { num: i32, } #[pymethods] impl ClassWithProperties { fn get_num(&self) -> i32 { self.num } #[getter(DATA)] /// a getter for data fn get_data(&self) -> i32 { self.num } #[setter(DATA)] fn set_data(&mut self, value: i32) { self.num = value; } #[getter] /// a getter with a type un-wrapped by PyResult fn get_unwrapped(&self) -> i32 { self.num } #[setter] fn set_unwrapped(&mut self, value: i32) { self.num = value; } #[getter] fn get_data_list<'py>(&self, py: Python<'py>) -> &'py PyList { PyList::new(py, [self.num]) } } #[test] fn class_with_properties() { Python::with_gil(|py| { let inst = Py::new(py, ClassWithProperties { num: 10 }).unwrap(); py_run!(py, inst, "assert inst.get_num() == 10"); py_run!(py, inst, "assert inst.get_num() == inst.DATA"); py_run!(py, inst, "inst.DATA = 20"); py_run!(py, inst, "assert inst.get_num() == 20 == inst.DATA"); py_expect_exception!(py, inst, "del inst.DATA", PyAttributeError); py_run!(py, inst, "assert inst.get_num() == inst.unwrapped == 20"); py_run!(py, inst, "inst.unwrapped = 42"); py_run!(py, inst, "assert inst.get_num() == inst.unwrapped == 42"); py_run!(py, inst, "assert inst.data_list == [42]"); let d = [("C", py.get_type::())].into_py_dict(py); py_assert!(py, *d, "C.DATA.__doc__ == 'a getter for data'"); }); } #[pyclass] struct GetterSetter { #[pyo3(get, set)] num: i32, #[pyo3(get, set)] text: String, } #[pymethods] impl GetterSetter { fn get_num2(&self) -> i32 { self.num } } #[test] fn getter_setter_autogen() { Python::with_gil(|py| { let inst = Py::new( py, GetterSetter { num: 10, text: "Hello".to_string(), }, ) .unwrap(); py_run!(py, inst, "assert inst.num == 10"); py_run!(py, inst, "inst.num = 20; assert inst.num == 20"); py_run!( py, inst, "assert inst.text == 'Hello'; inst.text = 'There'; assert inst.text == 'There'" ); }); } #[pyclass] struct RefGetterSetter { num: i32, } #[pymethods] impl RefGetterSetter { #[getter] fn get_num(slf: PyRef<'_, Self>) -> i32 { slf.num } #[setter] fn set_num(mut slf: PyRefMut<'_, Self>, value: i32) { slf.num = value; } } #[test] fn ref_getter_setter() { // Regression test for #837 Python::with_gil(|py| { let inst = Py::new(py, RefGetterSetter { num: 10 }).unwrap(); py_run!(py, inst, "assert inst.num == 10"); py_run!(py, inst, "inst.num = 20; assert inst.num == 20"); }); } #[pyclass] struct TupleClassGetterSetter(i32); #[pymethods] impl TupleClassGetterSetter { #[getter(num)] fn get_num(&self) -> i32 { self.0 } #[setter(num)] fn set_num(&mut self, value: i32) { self.0 = value; } } #[test] fn tuple_struct_getter_setter() { Python::with_gil(|py| { let inst = Py::new(py, TupleClassGetterSetter(10)).unwrap(); py_assert!(py, inst, "inst.num == 10"); py_run!(py, inst, "inst.num = 20"); py_assert!(py, inst, "inst.num == 20"); }); } #[pyclass(get_all, set_all)] struct All { num: i32, } #[test] fn get_set_all() { Python::with_gil(|py| { let inst = Py::new(py, All { num: 10 }).unwrap(); py_run!(py, inst, "assert inst.num == 10"); py_run!(py, inst, "inst.num = 20; assert inst.num == 20"); }); } #[pyclass(get_all)] struct All2 { #[pyo3(set)] num: i32, } #[test] fn get_all_and_set() { Python::with_gil(|py| { let inst = Py::new(py, All2 { num: 10 }).unwrap(); py_run!(py, inst, "assert inst.num == 10"); py_run!(py, inst, "inst.num = 20; assert inst.num == 20"); }); } #[pyclass] struct CellGetterSetter { #[pyo3(get, set)] cell_inner: Cell, } #[test] fn cell_getter_setter() { let c = CellGetterSetter { cell_inner: Cell::new(10), }; Python::with_gil(|py| { let inst = Py::new(py, c).unwrap().to_object(py); let cell = Cell::new(20).to_object(py); py_run!(py, cell, "assert cell == 20"); py_run!(py, inst, "assert inst.cell_inner == 10"); py_run!( py, inst, "inst.cell_inner = 20; assert inst.cell_inner == 20" ); }); } #[test] fn borrowed_value_with_lifetime_of_self() { #[pyclass] struct BorrowedValue {} #[pymethods] impl BorrowedValue { #[getter] fn value(&self) -> &str { "value" } } Python::with_gil(|py| { let inst = Py::new(py, BorrowedValue {}).unwrap().to_object(py); py_run!(py, inst, "assert inst.value == 'value'"); }); } pyo3-0.20.2/tests/test_inheritance.rs000064400000000000000000000210271046102023000156410ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::py_run; use pyo3::types::IntoPyDict; #[path = "../src/tests/common.rs"] mod common; #[pyclass(subclass)] struct BaseClass { #[pyo3(get)] val1: usize, } #[pyclass(subclass)] struct SubclassAble {} #[test] fn subclass() { Python::with_gil(|py| { let d = [("SubclassAble", py.get_type::())].into_py_dict(py); py.run( "class A(SubclassAble): pass\nassert issubclass(A, SubclassAble)", None, Some(d), ) .map_err(|e| e.display(py)) .unwrap(); }); } #[pymethods] impl BaseClass { #[new] fn new() -> Self { BaseClass { val1: 10 } } fn base_method(&self, x: usize) -> usize { x * self.val1 } fn base_set(&mut self, fn_: &pyo3::PyAny) -> PyResult<()> { let value: usize = fn_.call0()?.extract()?; self.val1 = value; Ok(()) } } #[pyclass(extends=BaseClass)] struct SubClass { #[pyo3(get)] val2: usize, } #[pymethods] impl SubClass { #[new] fn new() -> (Self, BaseClass) { (SubClass { val2: 5 }, BaseClass { val1: 10 }) } fn sub_method(&self, x: usize) -> usize { x * self.val2 } fn sub_set_and_ret(&mut self, x: usize) -> usize { self.val2 = x; x } } #[test] fn inheritance_with_new_methods() { Python::with_gil(|py| { let typeobj = py.get_type::(); let inst = typeobj.call((), None).unwrap(); py_run!(py, inst, "assert inst.val1 == 10; assert inst.val2 == 5"); }); } #[test] fn call_base_and_sub_methods() { Python::with_gil(|py| { let obj = PyCell::new(py, SubClass::new()).unwrap(); py_run!( py, obj, r#" assert obj.base_method(10) == 100 assert obj.sub_method(10) == 50 "# ); }); } #[test] fn mutation_fails() { Python::with_gil(|py| { let obj = PyCell::new(py, SubClass::new()).unwrap(); let global = Some([("obj", obj)].into_py_dict(py)); let e = py .run("obj.base_set(lambda: obj.sub_set_and_ret(1))", global, None) .unwrap_err(); assert_eq!(&e.to_string(), "RuntimeError: Already borrowed"); }); } #[test] fn is_subclass_and_is_instance() { Python::with_gil(|py| { let sub_ty = py.get_type::(); let base_ty = py.get_type::(); assert!(sub_ty.is_subclass_of::().unwrap()); assert!(sub_ty.is_subclass(base_ty).unwrap()); let obj = PyCell::new(py, SubClass::new()).unwrap(); assert!(obj.is_instance_of::()); assert!(obj.is_instance_of::()); assert!(obj.is_instance(sub_ty).unwrap()); assert!(obj.is_instance(base_ty).unwrap()); }); } #[pyclass(subclass)] struct BaseClassWithResult { _val: usize, } #[pymethods] impl BaseClassWithResult { #[new] fn new(value: isize) -> PyResult { Ok(Self { _val: std::convert::TryFrom::try_from(value)?, }) } } #[pyclass(extends=BaseClassWithResult)] struct SubClass2 {} #[pymethods] impl SubClass2 { #[new] fn new(value: isize) -> PyResult<(Self, BaseClassWithResult)> { let base = BaseClassWithResult::new(value)?; Ok((Self {}, base)) } } #[test] fn handle_result_in_new() { Python::with_gil(|py| { let subclass = py.get_type::(); py_run!( py, subclass, r#" try: subclass(-10) assert Fals except ValueError as e: pass except Exception as e: raise e "# ); }); } // Subclassing builtin types is not allowed in the LIMITED API. #[cfg(not(Py_LIMITED_API))] mod inheriting_native_type { use super::*; use pyo3::exceptions::PyException; use pyo3::types::{IntoPyDict, PyDict}; #[cfg(not(PyPy))] #[test] fn inherit_set() { use pyo3::types::PySet; #[cfg(not(PyPy))] #[pyclass(extends=PySet)] #[derive(Debug)] struct SetWithName { #[pyo3(get, name = "name")] _name: &'static str, } #[cfg(not(PyPy))] #[pymethods] impl SetWithName { #[new] fn new() -> Self { SetWithName { _name: "Hello :)" } } } Python::with_gil(|py| { let set_sub = pyo3::PyCell::new(py, SetWithName::new()).unwrap(); py_run!( py, set_sub, r#"set_sub.add(10); assert list(set_sub) == [10]; assert set_sub.name == "Hello :)""# ); }); } #[pyclass(extends=PyDict)] #[derive(Debug)] struct DictWithName { #[pyo3(get, name = "name")] _name: &'static str, } #[pymethods] impl DictWithName { #[new] fn new() -> Self { DictWithName { _name: "Hello :)" } } } #[test] fn inherit_dict() { Python::with_gil(|py| { let dict_sub = pyo3::PyCell::new(py, DictWithName::new()).unwrap(); py_run!( py, dict_sub, r#"dict_sub[0] = 1; assert dict_sub[0] == 1; assert dict_sub.name == "Hello :)""# ); }); } #[test] fn inherit_dict_drop() { Python::with_gil(|py| { let dict_sub = pyo3::Py::new(py, DictWithName::new()).unwrap(); assert_eq!(dict_sub.get_refcnt(py), 1); let item = py.eval("object()", None, None).unwrap(); assert_eq!(item.get_refcnt(), 1); dict_sub.as_ref(py).set_item("foo", item).unwrap(); assert_eq!(item.get_refcnt(), 2); drop(dict_sub); assert_eq!(item.get_refcnt(), 1); }) } #[pyclass(extends=PyException)] struct CustomException { #[pyo3(get)] context: &'static str, } #[pymethods] impl CustomException { #[new] fn new(_exc_arg: &PyAny) -> Self { CustomException { context: "Hello :)", } } } #[test] fn custom_exception() { Python::with_gil(|py| { let cls = py.get_type::(); let dict = [("cls", cls)].into_py_dict(py); let res = py.run( "e = cls('hello'); assert str(e) == 'hello'; assert e.context == 'Hello :)'; raise e", None, Some(dict) ); let err = res.unwrap_err(); assert!(err.matches(py, cls), "{}", err); // catching the exception in Python also works: py_run!( py, cls, r#" try: raise cls("foo") except cls: pass "# ) }) } } #[pyclass(subclass)] struct SimpleClass {} #[pymethods] impl SimpleClass { #[new] fn new() -> Self { Self {} } } #[test] fn test_subclass_ref_counts() { // regression test for issue #1363 Python::with_gil(|py| { #[allow(non_snake_case)] let SimpleClass = py.get_type::(); py_run!( py, SimpleClass, r#" import gc import sys class SubClass(SimpleClass): pass gc.collect() count = sys.getrefcount(SubClass) for i in range(1000): c = SubClass() del c gc.collect() after = sys.getrefcount(SubClass) # depending on Python's GC the count may be either identical or exactly 1000 higher, # both are expected values that are not representative of the issue. # # (With issue #1363 the count will be decreased.) assert after == count or (after == count + 1000), f"{after} vs {count}" "# ); }) } #[test] #[cfg(not(Py_LIMITED_API))] fn module_add_class_inherit_bool_fails() { use pyo3::types::PyBool; #[pyclass(extends = PyBool)] struct ExtendsBool; Python::with_gil(|py| { let m = PyModule::new(py, "test_module").unwrap(); let err = m.add_class::().unwrap_err(); assert_eq!( err.to_string(), "RuntimeError: An error occurred while initializing class ExtendsBool" ); assert_eq!( err.cause(py).unwrap().to_string(), "TypeError: type 'bool' is not an acceptable base type" ); }) } pyo3-0.20.2/tests/test_macro_docs.rs000064400000000000000000000015261046102023000154630ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::types::IntoPyDict; #[macro_use] #[path = "../src/tests/common.rs"] mod common; #[pyclass] /// The MacroDocs class. #[doc = concat!("Some macro ", "class ", "docs.")] /// A very interesting type! struct MacroDocs {} #[pymethods] impl MacroDocs { #[doc = concat!("A macro ", "example.")] /// With mixed doc types. fn macro_doc(&self) {} } #[test] fn meth_doc() { Python::with_gil(|py| { let d = [("C", py.get_type::())].into_py_dict(py); py_assert!( py, *d, "C.__doc__ == 'The MacroDocs class.\\nSome macro class docs.\\nA very interesting type!'" ); py_assert!( py, *d, "C.macro_doc.__doc__ == 'A macro example.\\nWith mixed doc types.'" ); }); } pyo3-0.20.2/tests/test_macros.rs000064400000000000000000000046051046102023000146370ustar 00000000000000#![cfg(feature = "macros")] //! Ensure that pyo3 macros can be used inside macro_rules! use pyo3::prelude::*; #[macro_use] #[path = "../src/tests/common.rs"] mod common; macro_rules! make_struct_using_macro { // Ensure that one doesn't need to fall back on the escape type: tt // in order to macro create pyclass. ($class_name:ident, $py_name:literal) => { #[pyclass(name=$py_name)] struct $class_name {} }; } make_struct_using_macro!(MyBaseClass, "MyClass"); macro_rules! set_extends_via_macro { ($class_name:ident, $base_class:path) => { // Try and pass a variable into the extends parameter #[pyclass(extends=$base_class)] struct $class_name {} }; } set_extends_via_macro!(MyClass2, MyBaseClass); // // Check that pyfunctiona nd text_signature can be called with macro arguments. // macro_rules! fn_macro { ($sig:literal, $a_exp:expr, $b_exp:expr, $c_exp: expr) => { // Try and pass a variable into the signature parameter #[pyfunction(signature = ($a_exp, $b_exp, *, $c_exp))] #[pyo3(text_signature = $sig)] fn my_function_in_macro(a: i32, b: Option, c: i32) { let _ = (a, b, c); } }; } fn_macro!("(a, b=None, *, c=42)", a, b = None, c = 42); macro_rules! property_rename_via_macro { ($prop_name:ident) => { #[pyclass] struct ClassWithProperty { member: u64, } #[pymethods] impl ClassWithProperty { #[getter($prop_name)] fn get_member(&self) -> u64 { self.member } #[setter($prop_name)] fn set_member(&mut self, member: u64) { self.member = member; } } }; } property_rename_via_macro!(my_new_property_name); #[test] fn test_macro_rules_interactions() { Python::with_gil(|py| { let my_base = py.get_type::(); py_assert!(py, my_base, "my_base.__name__ == 'MyClass'"); let my_func = wrap_pyfunction!(my_function_in_macro, py).unwrap(); py_assert!( py, my_func, "my_func.__text_signature__ == '(a, b=None, *, c=42)'" ); let renamed_prop = py.get_type::(); py_assert!( py, renamed_prop, "hasattr(renamed_prop, 'my_new_property_name')" ); }); } pyo3-0.20.2/tests/test_mapping.rs000064400000000000000000000065221046102023000150060ustar 00000000000000#![cfg(feature = "macros")] use std::collections::HashMap; use pyo3::exceptions::PyKeyError; use pyo3::prelude::*; use pyo3::py_run; use pyo3::types::IntoPyDict; use pyo3::types::PyList; use pyo3::types::PyMapping; use pyo3::types::PySequence; #[path = "../src/tests/common.rs"] mod common; #[pyclass(mapping)] struct Mapping { index: HashMap, } #[pymethods] impl Mapping { #[new] fn new(elements: Option<&PyList>) -> PyResult { if let Some(pylist) = elements { let mut elems = HashMap::with_capacity(pylist.len()); for (i, pyelem) in pylist.into_iter().enumerate() { let elem = String::extract(pyelem)?; elems.insert(elem, i); } Ok(Self { index: elems }) } else { Ok(Self { index: HashMap::new(), }) } } fn __len__(&self) -> usize { self.index.len() } fn __getitem__(&self, query: String) -> PyResult { self.index .get(&query) .copied() .ok_or_else(|| PyKeyError::new_err("unknown key")) } fn __setitem__(&mut self, key: String, value: usize) { self.index.insert(key, value); } fn __delitem__(&mut self, key: String) -> PyResult<()> { if self.index.remove(&key).is_none() { Err(PyKeyError::new_err("unknown key")) } else { Ok(()) } } fn get(&self, py: Python<'_>, key: &str, default: Option) -> Option { self.index .get(key) .map(|value| value.into_py(py)) .or(default) } } /// Return a dict with `m = Mapping(['1', '2', '3'])`. fn map_dict(py: Python<'_>) -> &pyo3::types::PyDict { let d = [("Mapping", py.get_type::())].into_py_dict(py); py_run!(py, *d, "m = Mapping(['1', '2', '3'])"); d } #[test] fn test_getitem() { Python::with_gil(|py| { let d = map_dict(py); py_assert!(py, *d, "m['1'] == 0"); py_assert!(py, *d, "m['2'] == 1"); py_assert!(py, *d, "m['3'] == 2"); py_expect_exception!(py, *d, "print(m['4'])", PyKeyError); }); } #[test] fn test_setitem() { Python::with_gil(|py| { let d = map_dict(py); py_run!(py, *d, "m['1'] = 4; assert m['1'] == 4"); py_run!(py, *d, "m['0'] = 0; assert m['0'] == 0"); py_assert!(py, *d, "len(m) == 4"); py_expect_exception!(py, *d, "m[0] = 'hello'", PyTypeError); py_expect_exception!(py, *d, "m[0] = -1", PyTypeError); }); } #[test] fn test_delitem() { Python::with_gil(|py| { let d = map_dict(py); py_run!( py, *d, "del m['1']; assert len(m) == 2 and m['2'] == 1 and m['3'] == 2" ); py_expect_exception!(py, *d, "del m[-1]", PyTypeError); py_expect_exception!(py, *d, "del m['4']", PyKeyError); }); } #[test] fn mapping_is_not_sequence() { Python::with_gil(|py| { let mut index = HashMap::new(); index.insert("Foo".into(), 1); index.insert("Bar".into(), 2); let m = Py::new(py, Mapping { index }).unwrap(); PyMapping::register::(py).unwrap(); assert!(m.as_ref(py).downcast::().is_ok()); assert!(m.as_ref(py).downcast::().is_err()); }); } pyo3-0.20.2/tests/test_methods.rs000064400000000000000000000724471046102023000150270ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::py_run; use pyo3::types::{IntoPyDict, PyDict, PyList, PySet, PyString, PyTuple, PyType}; use pyo3::PyCell; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct InstanceMethod { member: i32, } #[pymethods] impl InstanceMethod { /// Test method fn method(&self) -> i32 { self.member } // Checks that &Self works fn add_other(&self, other: &Self) -> i32 { self.member + other.member } } #[test] fn instance_method() { Python::with_gil(|py| { let obj = PyCell::new(py, InstanceMethod { member: 42 }).unwrap(); let obj_ref = obj.borrow(); assert_eq!(obj_ref.method(), 42); py_assert!(py, obj, "obj.method() == 42"); py_assert!(py, obj, "obj.add_other(obj) == 84"); py_assert!(py, obj, "obj.method.__doc__ == 'Test method'"); }); } #[pyclass] struct InstanceMethodWithArgs { member: i32, } #[pymethods] impl InstanceMethodWithArgs { fn method(&self, multiplier: i32) -> i32 { self.member * multiplier } } #[test] fn instance_method_with_args() { Python::with_gil(|py| { let obj = PyCell::new(py, InstanceMethodWithArgs { member: 7 }).unwrap(); let obj_ref = obj.borrow(); assert_eq!(obj_ref.method(6), 42); py_assert!(py, obj, "obj.method(3) == 21"); py_assert!(py, obj, "obj.method(multiplier=6) == 42"); }); } #[pyclass] struct ClassMethod {} #[pymethods] impl ClassMethod { #[new] fn new() -> Self { ClassMethod {} } #[classmethod] /// Test class method. fn method(cls: &PyType) -> PyResult { Ok(format!("{}.method()!", cls.name()?)) } #[classmethod] fn method_owned(cls: Py) -> PyResult { Ok(format!( "{}.method_owned()!", Python::with_gil(|gil| cls.as_ref(gil).name().map(ToString::to_string))? )) } } #[test] fn class_method() { Python::with_gil(|py| { let d = [("C", py.get_type::())].into_py_dict(py); py_assert!(py, *d, "C.method() == 'ClassMethod.method()!'"); py_assert!(py, *d, "C().method() == 'ClassMethod.method()!'"); py_assert!( py, *d, "C().method_owned() == 'ClassMethod.method_owned()!'" ); py_assert!(py, *d, "C.method.__doc__ == 'Test class method.'"); py_assert!(py, *d, "C().method.__doc__ == 'Test class method.'"); }); } #[pyclass] struct ClassMethodWithArgs {} #[pymethods] impl ClassMethodWithArgs { #[classmethod] fn method(cls: &PyType, input: &PyString) -> PyResult { Ok(format!("{}.method({})", cls.name()?, input)) } } #[test] fn class_method_with_args() { Python::with_gil(|py| { let d = [("C", py.get_type::())].into_py_dict(py); py_assert!( py, *d, "C.method('abc') == 'ClassMethodWithArgs.method(abc)'" ); }); } #[pyclass] struct StaticMethod {} #[pymethods] impl StaticMethod { #[new] fn new() -> Self { StaticMethod {} } #[staticmethod] /// Test static method. fn method(_py: Python<'_>) -> &'static str { "StaticMethod.method()!" } } #[test] fn static_method() { Python::with_gil(|py| { assert_eq!(StaticMethod::method(py), "StaticMethod.method()!"); let d = [("C", py.get_type::())].into_py_dict(py); py_assert!(py, *d, "C.method() == 'StaticMethod.method()!'"); py_assert!(py, *d, "C().method() == 'StaticMethod.method()!'"); py_assert!(py, *d, "C.method.__doc__ == 'Test static method.'"); py_assert!(py, *d, "C().method.__doc__ == 'Test static method.'"); }); } #[pyclass] struct StaticMethodWithArgs {} #[pymethods] impl StaticMethodWithArgs { #[staticmethod] fn method(_py: Python<'_>, input: i32) -> String { format!("0x{:x}", input) } } #[test] fn static_method_with_args() { Python::with_gil(|py| { assert_eq!(StaticMethodWithArgs::method(py, 1234), "0x4d2"); let d = [("C", py.get_type::())].into_py_dict(py); py_assert!(py, *d, "C.method(1337) == '0x539'"); }); } #[pyclass] struct MethSignature {} #[pymethods] impl MethSignature { #[pyo3(signature = (test = None))] fn get_optional(&self, test: Option) -> i32 { test.unwrap_or(10) } #[pyo3(signature = (test = None))] fn get_optional2(&self, test: Option) -> Option { test } fn get_optional_positional( &self, _t1: Option, t2: Option, _t3: Option, ) -> Option { t2 } #[pyo3(signature = (test = 10))] fn get_default(&self, test: i32) -> i32 { test } #[pyo3(signature = (*, test = 10))] fn get_kwarg(&self, test: i32) -> i32 { test } #[pyo3(signature = (*args, **kwargs))] fn get_kwargs(&self, py: Python<'_>, args: &PyTuple, kwargs: Option<&PyDict>) -> PyObject { [args.into(), kwargs.to_object(py)].to_object(py) } #[pyo3(signature = (a, *args, **kwargs))] fn get_pos_arg_kw( &self, py: Python<'_>, a: i32, args: &PyTuple, kwargs: Option<&PyDict>, ) -> PyObject { [a.to_object(py), args.into(), kwargs.to_object(py)].to_object(py) } #[pyo3(signature = (a, b, /))] fn get_pos_only(&self, a: i32, b: i32) -> i32 { a + b } #[pyo3(signature = (a, /, b))] fn get_pos_only_and_pos(&self, a: i32, b: i32) -> i32 { a + b } #[pyo3(signature = (a, /, b, c = 5))] fn get_pos_only_and_pos_and_kw(&self, a: i32, b: i32, c: i32) -> i32 { a + b + c } #[pyo3(signature = (a, /, *, b))] fn get_pos_only_and_kw_only(&self, a: i32, b: i32) -> i32 { a + b } #[pyo3(signature = (a, /, *, b = 3))] fn get_pos_only_and_kw_only_with_default(&self, a: i32, b: i32) -> i32 { a + b } #[pyo3(signature = (a, /, b, *, c, d = 5))] fn get_all_arg_types_together(&self, a: i32, b: i32, c: i32, d: i32) -> i32 { a + b + c + d } #[pyo3(signature = (a, /, *args))] fn get_pos_only_with_varargs(&self, a: i32, args: Vec) -> i32 { a + args.iter().sum::() } #[pyo3(signature = (a, /, **kwargs))] fn get_pos_only_with_kwargs( &self, py: Python<'_>, a: i32, kwargs: Option<&PyDict>, ) -> PyObject { [a.to_object(py), kwargs.to_object(py)].to_object(py) } #[pyo3(signature = (a=0, /, **kwargs))] fn get_optional_pos_only_with_kwargs( &self, py: Python<'_>, a: i32, kwargs: Option<&PyDict>, ) -> PyObject { [a.to_object(py), kwargs.to_object(py)].to_object(py) } #[pyo3(signature = (*, a = 2, b = 3))] fn get_kwargs_only_with_defaults(&self, a: i32, b: i32) -> i32 { a + b } #[pyo3(signature = (*, a, b))] fn get_kwargs_only(&self, a: i32, b: i32) -> i32 { a + b } #[pyo3(signature = (*, a = 1, b))] fn get_kwargs_only_with_some_default(&self, a: i32, b: i32) -> i32 { a + b } #[pyo3(signature = (*args, a))] fn get_args_and_required_keyword(&self, py: Python<'_>, args: &PyTuple, a: i32) -> PyObject { (args, a).to_object(py) } #[pyo3(signature = (a, b = 2, *, c = 3))] fn get_pos_arg_kw_sep1(&self, a: i32, b: i32, c: i32) -> i32 { a + b + c } #[pyo3(signature = (a, *, b = 2, c = 3))] fn get_pos_arg_kw_sep2(&self, a: i32, b: i32, c: i32) -> i32 { a + b + c } #[pyo3(signature = (a, **kwargs))] fn get_pos_kw(&self, py: Python<'_>, a: i32, kwargs: Option<&PyDict>) -> PyObject { [a.to_object(py), kwargs.to_object(py)].to_object(py) } // "args" can be anything that can be extracted from PyTuple #[pyo3(signature = (*args))] fn args_as_vec(&self, args: Vec) -> i32 { args.iter().sum() } } #[test] fn meth_signature() { Python::with_gil(|py| { let inst = Py::new(py, MethSignature {}).unwrap(); py_run!(py, inst, "assert inst.get_optional() == 10"); py_run!(py, inst, "assert inst.get_optional(100) == 100"); py_run!(py, inst, "assert inst.get_optional2() == None"); py_run!(py, inst, "assert inst.get_optional2(100) == 100"); py_run!( py, inst, "assert inst.get_optional_positional(1, 2, 3) == 2" ); py_run!(py, inst, "assert inst.get_optional_positional(1) == None"); py_run!(py, inst, "assert inst.get_default() == 10"); py_run!(py, inst, "assert inst.get_default(100) == 100"); py_run!(py, inst, "assert inst.get_kwarg() == 10"); py_expect_exception!(py, inst, "inst.get_kwarg(100)", PyTypeError); py_run!(py, inst, "assert inst.get_kwarg(test=100) == 100"); py_run!(py, inst, "assert inst.get_kwargs() == [(), None]"); py_run!(py, inst, "assert inst.get_kwargs(1,2,3) == [(1,2,3), None]"); py_run!( py, inst, "assert inst.get_kwargs(t=1,n=2) == [(), {'t': 1, 'n': 2}]" ); py_run!( py, inst, "assert inst.get_kwargs(1,2,3,t=1,n=2) == [(1,2,3), {'t': 1, 'n': 2}]" ); py_run!(py, inst, "assert inst.get_pos_arg_kw(1) == [1, (), None]"); py_run!( py, inst, "assert inst.get_pos_arg_kw(1, 2, 3) == [1, (2, 3), None]" ); py_run!( py, inst, "assert inst.get_pos_arg_kw(1, b=2) == [1, (), {'b': 2}]" ); py_run!(py, inst, "assert inst.get_pos_arg_kw(a=1) == [1, (), None]"); py_expect_exception!(py, inst, "inst.get_pos_arg_kw()", PyTypeError); py_expect_exception!(py, inst, "inst.get_pos_arg_kw(1, a=1)", PyTypeError); py_expect_exception!(py, inst, "inst.get_pos_arg_kw(b=2)", PyTypeError); py_run!(py, inst, "assert inst.get_pos_only(10, 11) == 21"); py_expect_exception!(py, inst, "inst.get_pos_only(10, b = 11)", PyTypeError); py_expect_exception!(py, inst, "inst.get_pos_only(a = 10, b = 11)", PyTypeError); py_run!(py, inst, "assert inst.get_pos_only_and_pos(10, 11) == 21"); py_run!( py, inst, "assert inst.get_pos_only_and_pos(10, b = 11) == 21" ); py_expect_exception!( py, inst, "inst.get_pos_only_and_pos(a = 10, b = 11)", PyTypeError ); py_run!( py, inst, "assert inst.get_pos_only_and_pos_and_kw(10, 11) == 26" ); py_run!( py, inst, "assert inst.get_pos_only_and_pos_and_kw(10, b = 11) == 26" ); py_run!( py, inst, "assert inst.get_pos_only_and_pos_and_kw(10, 11, c = 0) == 21" ); py_run!( py, inst, "assert inst.get_pos_only_and_pos_and_kw(10, b = 11, c = 0) == 21" ); py_expect_exception!( py, inst, "inst.get_pos_only_and_pos_and_kw(a = 10, b = 11)", PyTypeError ); py_run!( py, inst, "assert inst.get_pos_only_and_kw_only(10, b = 11) == 21" ); py_expect_exception!( py, inst, "inst.get_pos_only_and_kw_only(10, 11)", PyTypeError ); py_expect_exception!( py, inst, "inst.get_pos_only_and_kw_only(a = 10, b = 11)", PyTypeError ); py_run!( py, inst, "assert inst.get_pos_only_and_kw_only_with_default(10) == 13" ); py_run!( py, inst, "assert inst.get_pos_only_and_kw_only_with_default(10, b = 11) == 21" ); py_expect_exception!( py, inst, "inst.get_pos_only_and_kw_only_with_default(10, 11)", PyTypeError ); py_expect_exception!( py, inst, "inst.get_pos_only_and_kw_only_with_default(a = 10, b = 11)", PyTypeError ); py_run!( py, inst, "assert inst.get_all_arg_types_together(10, 10, c = 10) == 35" ); py_run!( py, inst, "assert inst.get_all_arg_types_together(10, 10, c = 10, d = 10) == 40" ); py_run!( py, inst, "assert inst.get_all_arg_types_together(10, b = 10, c = 10, d = 10) == 40" ); py_expect_exception!( py, inst, "inst.get_all_arg_types_together(10, 10, 10)", PyTypeError ); py_expect_exception!( py, inst, "inst.get_all_arg_types_together(a = 10, b = 10, c = 10)", PyTypeError ); py_run!(py, inst, "assert inst.get_pos_only_with_varargs(10) == 10"); py_run!( py, inst, "assert inst.get_pos_only_with_varargs(10, 10) == 20" ); py_run!( py, inst, "assert inst.get_pos_only_with_varargs(10, 10, 10, 10, 10) == 50" ); py_expect_exception!( py, inst, "inst.get_pos_only_with_varargs(a = 10)", PyTypeError ); py_run!( py, inst, "assert inst.get_pos_only_with_kwargs(10) == [10, None]" ); py_run!( py, inst, "assert inst.get_pos_only_with_kwargs(10, b = 10) == [10, {'b': 10}]" ); py_run!( py, inst, "assert inst.get_pos_only_with_kwargs(10, b = 10, c = 10, d = 10, e = 10) == [10, {'b': 10, 'c': 10, 'd': 10, 'e': 10}]" ); py_expect_exception!( py, inst, "inst.get_pos_only_with_kwargs(a = 10)", PyTypeError ); py_expect_exception!( py, inst, "inst.get_pos_only_with_kwargs(a = 10, b = 10)", PyTypeError ); py_run!( py, inst, "assert inst.get_optional_pos_only_with_kwargs() == [0, None]" ); py_run!( py, inst, "assert inst.get_optional_pos_only_with_kwargs(10) == [10, None]" ); py_run!( py, inst, "assert inst.get_optional_pos_only_with_kwargs(a=10) == [0, {'a': 10}]" ); py_run!(py, inst, "assert inst.get_kwargs_only_with_defaults() == 5"); py_run!( py, inst, "assert inst.get_kwargs_only_with_defaults(a = 8) == 11" ); py_run!( py, inst, "assert inst.get_kwargs_only_with_defaults(b = 8) == 10" ); py_run!( py, inst, "assert inst.get_kwargs_only_with_defaults(a = 1, b = 1) == 2" ); py_run!( py, inst, "assert inst.get_kwargs_only_with_defaults(b = 1, a = 1) == 2" ); py_run!(py, inst, "assert inst.get_kwargs_only(a = 1, b = 1) == 2"); py_run!(py, inst, "assert inst.get_kwargs_only(b = 1, a = 1) == 2"); py_run!( py, inst, "assert inst.get_kwargs_only_with_some_default(a = 2, b = 1) == 3" ); py_run!( py, inst, "assert inst.get_kwargs_only_with_some_default(b = 1) == 2" ); py_run!( py, inst, "assert inst.get_kwargs_only_with_some_default(b = 1, a = 2) == 3" ); py_expect_exception!( py, inst, "inst.get_kwargs_only_with_some_default()", PyTypeError ); py_run!( py, inst, "assert inst.get_args_and_required_keyword(1, 2, a=3) == ((1, 2), 3)" ); py_run!( py, inst, "assert inst.get_args_and_required_keyword(a=1) == ((), 1)" ); py_expect_exception!( py, inst, "inst.get_args_and_required_keyword()", PyTypeError ); py_run!(py, inst, "assert inst.get_pos_arg_kw_sep1(1) == 6"); py_run!(py, inst, "assert inst.get_pos_arg_kw_sep1(1, 2) == 6"); py_run!( py, inst, "assert inst.get_pos_arg_kw_sep1(1, 2, c=13) == 16" ); py_run!( py, inst, "assert inst.get_pos_arg_kw_sep1(a=1, b=2, c=13) == 16" ); py_run!( py, inst, "assert inst.get_pos_arg_kw_sep1(b=2, c=13, a=1) == 16" ); py_run!( py, inst, "assert inst.get_pos_arg_kw_sep1(c=13, b=2, a=1) == 16" ); py_expect_exception!(py, inst, "inst.get_pos_arg_kw_sep1(1, 2, 3)", PyTypeError); py_run!(py, inst, "assert inst.get_pos_arg_kw_sep2(1) == 6"); py_run!( py, inst, "assert inst.get_pos_arg_kw_sep2(1, b=12, c=13) == 26" ); py_expect_exception!(py, inst, "inst.get_pos_arg_kw_sep2(1, 2)", PyTypeError); py_run!(py, inst, "assert inst.get_pos_kw(1, b=2) == [1, {'b': 2}]"); py_expect_exception!(py, inst, "inst.get_pos_kw(1,2)", PyTypeError); py_run!(py, inst, "assert inst.args_as_vec(1,2,3) == 6"); }); } #[pyclass] /// A class with "documentation". struct MethDocs { x: i32, } #[pymethods] impl MethDocs { /// A method with "documentation" as well. fn method(&self) -> i32 { 0 } #[getter] /// `int`: a very "important" member of 'this' instance. fn get_x(&self) -> i32 { self.x } } #[test] fn meth_doc() { Python::with_gil(|py| { let d = [("C", py.get_type::())].into_py_dict(py); py_assert!(py, *d, "C.__doc__ == 'A class with \"documentation\".'"); py_assert!( py, *d, "C.method.__doc__ == 'A method with \"documentation\" as well.'" ); py_assert!( py, *d, "C.x.__doc__ == '`int`: a very \"important\" member of \\'this\\' instance.'" ); }); } #[pyclass] struct MethodWithLifeTime {} #[pymethods] impl MethodWithLifeTime { fn set_to_list<'py>(&self, py: Python<'py>, set: &'py PySet) -> PyResult<&'py PyList> { let mut items = vec![]; for _ in 0..set.len() { items.push(set.pop().unwrap()); } let list = PyList::new(py, items); list.sort()?; Ok(list) } } #[test] fn method_with_lifetime() { Python::with_gil(|py| { let obj = PyCell::new(py, MethodWithLifeTime {}).unwrap(); py_run!( py, obj, "assert obj.set_to_list(set((1, 2, 3))) == [1, 2, 3]" ); }); } #[pyclass] struct MethodWithPyClassArg { #[pyo3(get)] value: i64, } #[pymethods] impl MethodWithPyClassArg { fn add(&self, other: &MethodWithPyClassArg) -> MethodWithPyClassArg { MethodWithPyClassArg { value: self.value + other.value, } } fn add_pyref(&self, other: PyRef<'_, MethodWithPyClassArg>) -> MethodWithPyClassArg { MethodWithPyClassArg { value: self.value + other.value, } } fn inplace_add(&self, other: &mut MethodWithPyClassArg) { other.value += self.value; } fn inplace_add_pyref(&self, mut other: PyRefMut<'_, MethodWithPyClassArg>) { other.value += self.value; } fn optional_add(&self, other: Option<&MethodWithPyClassArg>) -> MethodWithPyClassArg { MethodWithPyClassArg { value: self.value + other.map(|o| o.value).unwrap_or(10), } } fn optional_inplace_add(&self, other: Option<&mut MethodWithPyClassArg>) { if let Some(other) = other { other.value += self.value; } } } #[test] fn method_with_pyclassarg() { Python::with_gil(|py| { let obj1 = PyCell::new(py, MethodWithPyClassArg { value: 10 }).unwrap(); let obj2 = PyCell::new(py, MethodWithPyClassArg { value: 10 }).unwrap(); let d = [("obj1", obj1), ("obj2", obj2)].into_py_dict(py); py_run!(py, *d, "obj = obj1.add(obj2); assert obj.value == 20"); py_run!(py, *d, "obj = obj1.add_pyref(obj2); assert obj.value == 20"); py_run!(py, *d, "obj = obj1.optional_add(); assert obj.value == 20"); py_run!( py, *d, "obj = obj1.optional_add(obj2); assert obj.value == 20" ); py_run!(py, *d, "obj1.inplace_add(obj2); assert obj.value == 20"); py_run!( py, *d, "obj1.inplace_add_pyref(obj2); assert obj2.value == 30" ); py_run!( py, *d, "obj1.optional_inplace_add(); assert obj2.value == 30" ); py_run!( py, *d, "obj1.optional_inplace_add(obj2); assert obj2.value == 40" ); }); } #[pyclass] #[cfg(unix)] struct CfgStruct {} #[pyclass] #[cfg(not(unix))] struct CfgStruct {} #[pymethods] #[cfg(unix)] impl CfgStruct { fn unix_method(&self) -> &str { "unix" } #[cfg(not(unix))] fn never_compiled_method(&self) {} } #[pymethods] #[cfg(not(unix))] impl CfgStruct { fn not_unix_method(&self) -> &str { "not unix" } #[cfg(unix)] fn never_compiled_method(&self) {} } #[test] fn test_cfg_attrs() { Python::with_gil(|py| { let inst = Py::new(py, CfgStruct {}).unwrap(); #[cfg(unix)] { py_assert!(py, inst, "inst.unix_method() == 'unix'"); py_assert!(py, inst, "not hasattr(inst, 'not_unix_method')"); } #[cfg(not(unix))] { py_assert!(py, inst, "not hasattr(inst, 'unix_method')"); py_assert!(py, inst, "inst.not_unix_method() == 'not unix'"); } py_assert!(py, inst, "not hasattr(inst, 'never_compiled_method')"); }); } #[pyclass] #[derive(Default)] struct FromSequence { #[pyo3(get)] numbers: Vec, } #[pymethods] impl FromSequence { #[new] fn new(seq: Option<&pyo3::types::PySequence>) -> PyResult { if let Some(seq) = seq { Ok(FromSequence { numbers: seq.as_ref().extract::>()?, }) } else { Ok(FromSequence::default()) } } } #[test] fn test_from_sequence() { Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!(py, typeobj, "typeobj(range(0, 4)).numbers == [0, 1, 2, 3]"); }); } #[pyclass] struct r#RawIdents { #[pyo3(get, set)] r#type: PyObject, r#subtype: PyObject, r#subsubtype: PyObject, } #[pymethods] impl r#RawIdents { #[new] pub fn r#new( r#_py: Python<'_>, r#type: PyObject, r#subtype: PyObject, r#subsubtype: PyObject, ) -> Self { Self { r#type, r#subtype, r#subsubtype, } } #[getter(r#subtype)] pub fn r#get_subtype(&self) -> PyObject { self.r#subtype.clone() } #[setter(r#subtype)] pub fn r#set_subtype(&mut self, r#subtype: PyObject) { self.r#subtype = r#subtype; } #[getter] pub fn r#get_subsubtype(&self) -> PyObject { self.r#subsubtype.clone() } #[setter] pub fn r#set_subsubtype(&mut self, r#subsubtype: PyObject) { self.r#subsubtype = r#subsubtype; } pub fn r#__call__(&mut self, r#type: PyObject) { self.r#type = r#type; } #[staticmethod] pub fn r#static_method(r#type: PyObject) -> PyObject { r#type } #[classmethod] pub fn r#class_method(_: &PyType, r#type: PyObject) -> PyObject { r#type } #[classattr] pub fn r#class_attr_fn() -> i32 { 5 } #[classattr] const r#CLASS_ATTR_CONST: i32 = 6; #[pyo3(signature = (r#struct = "foo"))] fn method_with_keyword<'a>(&self, r#struct: &'a str) -> &'a str { r#struct } } #[test] fn test_raw_idents() { Python::with_gil(|py| { let raw_idents_type = py.get_type::(); assert_eq!(raw_idents_type.name().unwrap(), "RawIdents"); py_run!( py, raw_idents_type, r#" instance = raw_idents_type(type=None, subtype=5, subsubtype="foo") assert instance.type is None assert instance.subtype == 5 assert instance.subsubtype == "foo" instance.type = 1 instance.subtype = 2 instance.subsubtype = 3 assert instance.type == 1 assert instance.subtype == 2 assert instance.subsubtype == 3 assert raw_idents_type.static_method(type=30) == 30 assert instance.class_method(type=40) == 40 instance(type=50) assert instance.type == 50 assert raw_idents_type.class_attr_fn == 5 assert raw_idents_type.CLASS_ATTR_CONST == 6 assert instance.method_with_keyword() == "foo" assert instance.method_with_keyword("bar") == "bar" assert instance.method_with_keyword(struct="baz") == "baz" "# ); }) } // Regression test for issue 1505 - Python argument not detected correctly when inside a macro. #[pyclass] struct Issue1505 {} macro_rules! pymethods { ( #[pymethods] impl $ty: ty { fn $fn:ident (&self, $arg:ident : $arg_ty:ty) {} } ) => { #[pymethods] impl $ty { fn $fn(&self, $arg: $arg_ty) {} } }; } pymethods!( #[pymethods] impl Issue1505 { fn issue_1505(&self, _py: Python<'_>) {} } ); // Regression test for issue 1506 - incorrect macro hygiene. // By applying the `#[pymethods]` attribute inside a macro_rules! macro, this separates the macro // call scope from the scope of the impl block. For this to work our macros must be careful to not // cheat hygiene! #[pyclass] struct Issue1506 {} macro_rules! issue_1506 { (#[pymethods] $($body:tt)*) => { #[pymethods] $($body)* }; } issue_1506!( #[pymethods] impl Issue1506 { fn issue_1506( &self, _py: Python<'_>, _arg: &PyAny, _args: &PyTuple, _kwargs: Option<&PyDict>, ) { } fn issue_1506_mut( &mut self, _py: Python<'_>, _arg: &PyAny, _args: &PyTuple, _kwargs: Option<&PyDict>, ) { } fn issue_1506_custom_receiver( _slf: Py, _py: Python<'_>, _arg: &PyAny, _args: &PyTuple, _kwargs: Option<&PyDict>, ) { } fn issue_1506_custom_receiver_explicit( _slf: Py, _py: Python<'_>, _arg: &PyAny, _args: &PyTuple, _kwargs: Option<&PyDict>, ) { } #[new] fn issue_1506_new( _py: Python<'_>, _arg: &PyAny, _args: &PyTuple, _kwargs: Option<&PyDict>, ) -> Self { Issue1506 {} } #[getter("foo")] fn issue_1506_getter(&self, _py: Python<'_>) -> i32 { 5 } #[setter("foo")] fn issue_1506_setter(&self, _py: Python<'_>, _value: i32) {} #[staticmethod] fn issue_1506_static( _py: Python<'_>, _arg: &PyAny, _args: &PyTuple, _kwargs: Option<&PyDict>, ) { } #[classmethod] fn issue_1506_class( _cls: &PyType, _py: Python<'_>, _arg: &PyAny, _args: &PyTuple, _kwargs: Option<&PyDict>, ) { } } ); #[pyclass] struct Issue1696 {} pymethods!( #[pymethods] impl Issue1696 { fn issue_1696(&self, _x: &InstanceMethod) {} } ); #[test] fn test_option_pyclass_arg() { // Option<&PyClass> argument with a default set in a signature regressed to a compile // error in PyO3 0.17.0 - this test it continues to be accepted. #[pyclass] struct SomePyClass {} #[pyfunction(signature = (arg=None))] fn option_class_arg(arg: Option<&SomePyClass>) -> Option { arg.map(|_| SomePyClass {}) } Python::with_gil(|py| { let f = wrap_pyfunction!(option_class_arg, py).unwrap(); assert!(f.call0().unwrap().is_none()); let obj = Py::new(py, SomePyClass {}).unwrap(); assert!(f .call1((obj,)) .unwrap() .extract::>() .is_ok()); }) } #[test] fn test_issue_2988() { #[pyfunction] #[pyo3(signature = ( _data = vec![], _data2 = vec![], ))] pub fn _foo( _data: Vec, // The from_py_with here looks a little odd, we just need some way // to encourage the macro to expand the from_py_with default path too #[pyo3(from_py_with = "PyAny::extract")] _data2: Vec, ) { } } pyo3-0.20.2/tests/test_module.rs000064400000000000000000000275311046102023000146430ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::py_run; use pyo3::types::{IntoPyDict, PyDict, PyTuple}; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct AnonClass {} #[pyclass] struct ValueClass { value: usize, } #[pymethods] impl ValueClass { #[new] fn new(value: usize) -> ValueClass { ValueClass { value } } } #[pyclass(module = "module")] struct LocatedClass {} #[pyfunction] /// Doubles the given value fn double(x: usize) -> usize { x * 2 } /// This module is implemented in Rust. #[pymodule] fn module_with_functions(_py: Python<'_>, m: &PyModule) -> PyResult<()> { #[pyfn(m)] #[pyo3(name = "no_parameters")] fn function_with_name() -> usize { 42 } #[pyfn(m)] #[pyo3(pass_module)] fn with_module(module: &PyModule) -> PyResult<&str> { module.name() } #[pyfn(m)] fn double_value(v: &ValueClass) -> usize { v.value * 2 } m.add_class::().unwrap(); m.add_class::().unwrap(); m.add_class::().unwrap(); m.add("foo", "bar").unwrap(); m.add_function(wrap_pyfunction!(double, m)?).unwrap(); m.add("also_double", wrap_pyfunction!(double, m)?).unwrap(); Ok(()) } #[test] fn test_module_with_functions() { use pyo3::wrap_pymodule; Python::with_gil(|py| { let d = [( "module_with_functions", wrap_pymodule!(module_with_functions)(py), )] .into_py_dict(py); py_assert!( py, *d, "module_with_functions.__doc__ == 'This module is implemented in Rust.'" ); py_assert!(py, *d, "module_with_functions.no_parameters() == 42"); py_assert!(py, *d, "module_with_functions.foo == 'bar'"); py_assert!(py, *d, "module_with_functions.AnonClass != None"); py_assert!(py, *d, "module_with_functions.LocatedClass != None"); py_assert!( py, *d, "module_with_functions.LocatedClass.__module__ == 'module'" ); py_assert!(py, *d, "module_with_functions.double(3) == 6"); py_assert!( py, *d, "module_with_functions.double.__doc__ == 'Doubles the given value'" ); py_assert!(py, *d, "module_with_functions.also_double(3) == 6"); py_assert!( py, *d, "module_with_functions.also_double.__doc__ == 'Doubles the given value'" ); py_assert!( py, *d, "module_with_functions.double_value(module_with_functions.ValueClass(1)) == 2" ); py_assert!( py, *d, "module_with_functions.with_module() == 'module_with_functions'" ); }); } #[pymodule] #[pyo3(name = "other_name")] fn some_name(_: Python<'_>, m: &PyModule) -> PyResult<()> { m.add("other_name", "other_name")?; Ok(()) } #[test] fn test_module_renaming() { use pyo3::wrap_pymodule; Python::with_gil(|py| { let d = [("different_name", wrap_pymodule!(some_name)(py))].into_py_dict(py); py_run!(py, *d, "assert different_name.__name__ == 'other_name'"); }); } #[test] fn test_module_from_code() { Python::with_gil(|py| { let adder_mod = PyModule::from_code( py, "def add(a,b):\n\treturn a+b", "adder_mod.py", "adder_mod", ) .expect("Module code should be loaded"); let add_func = adder_mod .getattr("add") .expect("Add function should be in the module") .to_object(py); let ret_value: i32 = add_func .call1(py, (1, 2)) .expect("A value should be returned") .extract(py) .expect("The value should be able to be converted to an i32"); assert_eq!(ret_value, 3); }); } #[pyfunction] fn r#move() -> usize { 42 } #[pymodule] fn raw_ident_module(_py: Python<'_>, module: &PyModule) -> PyResult<()> { module.add_function(wrap_pyfunction!(r#move, module)?) } #[test] fn test_raw_idents() { use pyo3::wrap_pymodule; Python::with_gil(|py| { let module = wrap_pymodule!(raw_ident_module)(py); py_assert!(py, module, "module.move() == 42"); }); } #[pyfunction] #[pyo3(name = "foobar")] fn custom_named_fn() -> usize { 42 } #[test] fn test_custom_names() { #[pymodule] fn custom_names(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(custom_named_fn, m)?)?; Ok(()) } Python::with_gil(|py| { let module = pyo3::wrap_pymodule!(custom_names)(py); py_assert!(py, module, "not hasattr(module, 'custom_named_fn')"); py_assert!(py, module, "module.foobar() == 42"); }); } #[test] fn test_module_dict() { #[pymodule] fn module_dict(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.dict().set_item("yay", "me")?; Ok(()) } Python::with_gil(|py| { let module = pyo3::wrap_pymodule!(module_dict)(py); py_assert!(py, module, "module.yay == 'me'"); }); } #[test] fn test_module_dunder_all() { Python::with_gil(|py| { #[pymodule] fn dunder_all(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.dict().set_item("yay", "me")?; m.add_function(wrap_pyfunction!(custom_named_fn, m)?)?; Ok(()) } let module = pyo3::wrap_pymodule!(dunder_all)(py); py_assert!(py, module, "module.__all__ == ['foobar']"); }); } #[pyfunction] fn subfunction() -> String { "Subfunction".to_string() } fn submodule(module: &PyModule) -> PyResult<()> { module.add_function(wrap_pyfunction!(subfunction, module)?)?; Ok(()) } #[pymodule] fn submodule_with_init_fn(_py: Python<'_>, module: &PyModule) -> PyResult<()> { module.add_function(wrap_pyfunction!(subfunction, module)?)?; Ok(()) } #[pyfunction] fn superfunction() -> String { "Superfunction".to_string() } #[pymodule] fn supermodule(py: Python<'_>, module: &PyModule) -> PyResult<()> { module.add_function(wrap_pyfunction!(superfunction, module)?)?; let module_to_add = PyModule::new(py, "submodule")?; submodule(module_to_add)?; module.add_submodule(module_to_add)?; let module_to_add = PyModule::new(py, "submodule_with_init_fn")?; submodule_with_init_fn(py, module_to_add)?; module.add_submodule(module_to_add)?; Ok(()) } #[test] fn test_module_nesting() { use pyo3::wrap_pymodule; Python::with_gil(|py| { let supermodule = wrap_pymodule!(supermodule)(py); py_assert!( py, supermodule, "supermodule.superfunction() == 'Superfunction'" ); py_assert!( py, supermodule, "supermodule.submodule.subfunction() == 'Subfunction'" ); py_assert!( py, supermodule, "supermodule.submodule_with_init_fn.subfunction() == 'Subfunction'" ); }); } // Test that argument parsing specification works for pyfunctions #[pyfunction(signature = (a=5, *args))] fn ext_vararg_fn(py: Python<'_>, a: i32, args: &PyTuple) -> PyObject { [a.to_object(py), args.into()].to_object(py) } #[pymodule] fn vararg_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { #[pyfn(m, signature = (a=5, *args))] fn int_vararg_fn(py: Python<'_>, a: i32, args: &PyTuple) -> PyObject { ext_vararg_fn(py, a, args) } m.add_function(wrap_pyfunction!(ext_vararg_fn, m)?).unwrap(); Ok(()) } #[test] fn test_vararg_module() { Python::with_gil(|py| { let m = pyo3::wrap_pymodule!(vararg_module)(py); py_assert!(py, m, "m.ext_vararg_fn() == [5, ()]"); py_assert!(py, m, "m.ext_vararg_fn(1, 2) == [1, (2,)]"); py_assert!(py, m, "m.int_vararg_fn() == [5, ()]"); py_assert!(py, m, "m.int_vararg_fn(1, 2) == [1, (2,)]"); }); } #[test] fn test_module_with_constant() { // Regression test for #1102 #[pymodule] fn module_with_constant(_py: Python<'_>, m: &PyModule) -> PyResult<()> { const ANON: AnonClass = AnonClass {}; m.add("ANON", ANON)?; m.add_class::()?; Ok(()) } Python::with_gil(|py| { let m = pyo3::wrap_pymodule!(module_with_constant)(py); py_assert!(py, m, "isinstance(m.ANON, m.AnonClass)"); }); } #[pyfunction] #[pyo3(pass_module)] fn pyfunction_with_module(module: &PyModule) -> PyResult<&str> { module.name() } #[pyfunction] #[pyo3(pass_module)] fn pyfunction_with_module_owned(module: Py) -> PyResult { Python::with_gil(|gil| module.as_ref(gil).name().map(Into::into)) } #[pyfunction] #[pyo3(pass_module)] fn pyfunction_with_module_and_py<'a>( module: &'a PyModule, _python: Python<'a>, ) -> PyResult<&'a str> { module.name() } #[pyfunction] #[pyo3(pass_module)] fn pyfunction_with_module_and_arg(module: &PyModule, string: String) -> PyResult<(&str, String)> { module.name().map(|s| (s, string)) } #[pyfunction(signature = (string="foo"))] #[pyo3(pass_module)] fn pyfunction_with_module_and_default_arg<'a>( module: &'a PyModule, string: &str, ) -> PyResult<(&'a str, String)> { module.name().map(|s| (s, string.into())) } #[pyfunction(signature = (*args, **kwargs))] #[pyo3(pass_module)] fn pyfunction_with_module_and_args_kwargs<'a>( module: &'a PyModule, args: &PyTuple, kwargs: Option<&PyDict>, ) -> PyResult<(&'a str, usize, Option)> { module .name() .map(|s| (s, args.len(), kwargs.map(|d| d.len()))) } #[pyfunction] #[pyo3(pass_module)] fn pyfunction_with_pass_module_in_attribute(module: &PyModule) -> PyResult<&str> { module.name() } #[pymodule] fn module_with_functions_with_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { m.add_function(wrap_pyfunction!(pyfunction_with_module, m)?)?; m.add_function(wrap_pyfunction!(pyfunction_with_module_owned, m)?)?; m.add_function(wrap_pyfunction!(pyfunction_with_module_and_py, m)?)?; m.add_function(wrap_pyfunction!(pyfunction_with_module_and_arg, m)?)?; m.add_function(wrap_pyfunction!(pyfunction_with_module_and_default_arg, m)?)?; m.add_function(wrap_pyfunction!(pyfunction_with_module_and_args_kwargs, m)?)?; m.add_function(wrap_pyfunction!( pyfunction_with_pass_module_in_attribute, m )?)?; m.add_function(wrap_pyfunction!(pyfunction_with_module, m)?)?; Ok(()) } #[test] fn test_module_functions_with_module() { Python::with_gil(|py| { let m = pyo3::wrap_pymodule!(module_with_functions_with_module)(py); py_assert!( py, m, "m.pyfunction_with_module() == 'module_with_functions_with_module'" ); py_assert!( py, m, "m.pyfunction_with_module_owned() == 'module_with_functions_with_module'" ); py_assert!( py, m, "m.pyfunction_with_module_and_py() == 'module_with_functions_with_module'" ); py_assert!( py, m, "m.pyfunction_with_module_and_default_arg() \ == ('module_with_functions_with_module', 'foo')" ); py_assert!( py, m, "m.pyfunction_with_module_and_args_kwargs(1, x=1, y=2) \ == ('module_with_functions_with_module', 1, 2)" ); py_assert!( py, m, "m.pyfunction_with_pass_module_in_attribute() == 'module_with_functions_with_module'" ); }); } #[test] fn test_module_doc_hidden() { #[doc(hidden)] #[allow(clippy::unnecessary_wraps)] #[pymodule] fn my_module(_py: Python<'_>, _m: &PyModule) -> PyResult<()> { Ok(()) } Python::with_gil(|py| { let m = pyo3::wrap_pymodule!(my_module)(py); py_assert!(py, m, "m.__doc__ == ''"); }) } pyo3-0.20.2/tests/test_multiple_pymethods.rs000064400000000000000000000031551046102023000173010ustar 00000000000000#![cfg(feature = "multiple-pymethods")] use pyo3::prelude::*; use pyo3::types::PyType; #[macro_use] #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct PyClassWithMultiplePyMethods {} #[pymethods] impl PyClassWithMultiplePyMethods { #[new] fn new() -> Self { Self {} } } #[pymethods] impl PyClassWithMultiplePyMethods { fn __call__(&self) -> &'static str { "call" } } #[pymethods] impl PyClassWithMultiplePyMethods { fn method(&self) -> &'static str { "method" } } #[pymethods] impl PyClassWithMultiplePyMethods { #[classmethod] fn classmethod(_ty: &PyType) -> &'static str { "classmethod" } } #[pymethods] impl PyClassWithMultiplePyMethods { #[staticmethod] fn staticmethod() -> &'static str { "staticmethod" } } #[pymethods] impl PyClassWithMultiplePyMethods { #[classattr] fn class_attribute() -> &'static str { "class_attribute" } } #[pymethods] impl PyClassWithMultiplePyMethods { #[classattr] const CLASS_ATTRIBUTE: &'static str = "CLASS_ATTRIBUTE"; } #[test] fn test_class_with_multiple_pymethods() { Python::with_gil(|py| { let cls = py.get_type::(); py_assert!(py, cls, "cls()() == 'call'"); py_assert!(py, cls, "cls().method() == 'method'"); py_assert!(py, cls, "cls.classmethod() == 'classmethod'"); py_assert!(py, cls, "cls.staticmethod() == 'staticmethod'"); py_assert!(py, cls, "cls.class_attribute == 'class_attribute'"); py_assert!(py, cls, "cls.CLASS_ATTRIBUTE == 'CLASS_ATTRIBUTE'"); }) } pyo3-0.20.2/tests/test_no_imports.rs000064400000000000000000000073731046102023000155510ustar 00000000000000//! Tests that various macros work correctly without any PyO3 imports. #![cfg(feature = "macros")] #[pyo3::pyfunction] #[pyo3(name = "identity", signature = (x = None))] fn basic_function(py: pyo3::Python<'_>, x: Option) -> pyo3::PyObject { x.unwrap_or_else(|| py.None()) } #[pyo3::pymodule] fn basic_module(_py: pyo3::Python<'_>, m: &pyo3::types::PyModule) -> pyo3::PyResult<()> { #[pyfn(m)] fn answer() -> usize { 42 } m.add_function(pyo3::wrap_pyfunction!(basic_function, m)?)?; Ok(()) } #[pyo3::pyclass] struct BasicClass { #[pyo3(get)] v: usize, #[pyo3(get, set)] s: String, } #[pyo3::pymethods] impl BasicClass { #[classattr] const OKAY: bool = true; #[new] fn new(arg: &pyo3::PyAny) -> pyo3::PyResult { if let Ok(v) = arg.extract::() { Ok(Self { v, s: "".to_string(), }) } else { Ok(Self { v: 0, s: arg.extract()?, }) } } #[getter] fn get_property(&self) -> usize { self.v * 100 } #[setter] fn set_property(&mut self, value: usize) { self.v = value / 100 } /// Some documentation here #[classmethod] fn classmethod(cls: &pyo3::types::PyType) -> &pyo3::types::PyType { cls } #[staticmethod] fn staticmethod(py: pyo3::Python<'_>, v: usize) -> pyo3::Py { use pyo3::IntoPy; v.to_string().into_py(py) } fn __add__(&self, other: usize) -> usize { self.v + other } fn __iadd__(&mut self, other: pyo3::PyRef<'_, Self>) { self.v += other.v; self.s.push_str(&other.s); } fn mutate(mut slf: pyo3::PyRefMut<'_, Self>) { slf.v += slf.v; slf.s.push('!'); } } #[test] fn test_basic() { pyo3::Python::with_gil(|py| { let module = pyo3::wrap_pymodule!(basic_module)(py); let cls = py.get_type::(); let d = pyo3::types::IntoPyDict::into_py_dict( [ ("mod", module.as_ref(py).as_ref()), ("cls", cls.as_ref()), ("a", cls.call1((8,)).unwrap()), ("b", cls.call1(("foo",)).unwrap()), ], py, ); pyo3::py_run!(py, *d, "assert mod.answer() == 42"); pyo3::py_run!(py, *d, "assert mod.identity() is None"); pyo3::py_run!(py, *d, "v = object(); assert mod.identity(v) is v"); pyo3::py_run!(py, *d, "assert cls.OKAY"); pyo3::py_run!(py, *d, "assert (a.v, a.s) == (8, '')"); pyo3::py_run!(py, *d, "assert (b.v, b.s) == (0, 'foo')"); pyo3::py_run!(py, *d, "b.property = 314"); pyo3::py_run!(py, *d, "assert b.property == 300"); pyo3::py_run!( py, *d, "assert cls.classmethod.__doc__ == 'Some documentation here'" ); pyo3::py_run!(py, *d, "assert cls.classmethod() is cls"); pyo3::py_run!(py, *d, "assert cls.staticmethod(5) == '5'"); pyo3::py_run!(py, *d, "a.s = 'bar'; assert a.s == 'bar'"); pyo3::py_run!(py, *d, "a.mutate(); assert (a.v, a.s) == (16, 'bar!')"); pyo3::py_run!(py, *d, "assert a + 9 == 25"); pyo3::py_run!(py, *d, "b += a; assert (b.v, b.s) == (19, 'foobar!')"); }); } #[pyo3::pyclass] struct NewClassMethod { #[pyo3(get)] cls: pyo3::PyObject, } #[pyo3::pymethods] impl NewClassMethod { #[new] #[classmethod] fn new(cls: &pyo3::types::PyType) -> Self { Self { cls: cls.into() } } } #[test] fn test_new_class_method() { pyo3::Python::with_gil(|py| { let cls = py.get_type::(); pyo3::py_run!(py, cls, "assert cls().cls is cls"); }); } pyo3-0.20.2/tests/test_pep_587.rs000064400000000000000000000033221046102023000145350ustar 00000000000000#![cfg(all(Py_3_8, not(any(PyPy, Py_LIMITED_API))))] use pyo3::ffi; #[cfg(Py_3_10)] use widestring::WideCString; #[test] fn test_default_interpreter() { macro_rules! ensure { ($py_call:expr) => {{ let status = $py_call; unsafe { if ffi::PyStatus_Exception(status) != 0 { ffi::Py_ExitStatusException(status); } } }}; } let mut preconfig = unsafe { std::mem::zeroed() }; unsafe { ffi::PyPreConfig_InitPythonConfig(&mut preconfig) }; preconfig.utf8_mode = 1; ensure!(unsafe { ffi::Py_PreInitialize(&preconfig) }); let mut config = unsafe { std::mem::zeroed() }; unsafe { ffi::PyConfig_InitPythonConfig(&mut config) }; // Require manually calling _Py_InitializeMain to exercise more ffi code #[allow(clippy::used_underscore_binding)] { config._init_main = 0; } #[cfg(Py_3_10)] unsafe { ffi::PyConfig_SetBytesString( &mut config, &mut config.program_name, "some_test\0".as_ptr().cast(), ); } ensure!(unsafe { ffi::Py_InitializeFromConfig(&config) }); // The GIL is held. assert_eq!(unsafe { ffi::PyGILState_Check() }, 1); // Now proceed with the Python main initialization. ensure!(unsafe { ffi::_Py_InitializeMain() }); // The GIL is held after finishing initialization. assert_eq!(unsafe { ffi::PyGILState_Check() }, 1); // Confirm program name set above was picked up correctly #[cfg(Py_3_10)] { let program_name = unsafe { WideCString::from_ptr_str(ffi::Py_GetProgramName().cast()) }; assert_eq!(program_name.to_string().unwrap(), "some_test"); } } pyo3-0.20.2/tests/test_proto_methods.rs000064400000000000000000000515771046102023000162530ustar 00000000000000#![cfg(feature = "macros")] use pyo3::exceptions::{PyAttributeError, PyIndexError, PyValueError}; use pyo3::types::{PyDict, PyList, PyMapping, PySequence, PySlice, PyType}; use pyo3::{prelude::*, py_run, PyCell}; use std::{isize, iter}; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct EmptyClass; #[pyclass] struct ExampleClass { #[pyo3(get, set)] value: i32, custom_attr: Option, } #[pymethods] impl ExampleClass { fn __getattr__(&self, py: Python<'_>, attr: &str) -> PyResult { if attr == "special_custom_attr" { Ok(self.custom_attr.into_py(py)) } else { Err(PyAttributeError::new_err(attr.to_string())) } } fn __setattr__(&mut self, attr: &str, value: &PyAny) -> PyResult<()> { if attr == "special_custom_attr" { self.custom_attr = Some(value.extract()?); Ok(()) } else { Err(PyAttributeError::new_err(attr.to_string())) } } fn __delattr__(&mut self, attr: &str) -> PyResult<()> { if attr == "special_custom_attr" { self.custom_attr = None; Ok(()) } else { Err(PyAttributeError::new_err(attr.to_string())) } } fn __str__(&self) -> String { self.value.to_string() } fn __repr__(&self) -> String { format!("ExampleClass(value={})", self.value) } fn __hash__(&self) -> u64 { let i64_value: i64 = self.value.into(); i64_value as u64 } fn __bool__(&self) -> bool { self.value != 0 } } fn make_example(py: Python<'_>) -> &PyCell { Py::new( py, ExampleClass { value: 5, custom_attr: Some(20), }, ) .unwrap() .into_ref(py) } #[test] fn test_getattr() { Python::with_gil(|py| { let example_py = make_example(py); assert_eq!( example_py .getattr("value") .unwrap() .extract::() .unwrap(), 5, ); assert_eq!( example_py .getattr("special_custom_attr") .unwrap() .extract::() .unwrap(), 20, ); assert!(example_py .getattr("other_attr") .unwrap_err() .is_instance_of::(py)); }) } #[test] fn test_setattr() { Python::with_gil(|py| { let example_py = make_example(py); example_py.setattr("special_custom_attr", 15).unwrap(); assert_eq!( example_py .getattr("special_custom_attr") .unwrap() .extract::() .unwrap(), 15, ); }) } #[test] fn test_delattr() { Python::with_gil(|py| { let example_py = make_example(py); example_py.delattr("special_custom_attr").unwrap(); assert!(example_py.getattr("special_custom_attr").unwrap().is_none()); }) } #[test] fn test_str() { Python::with_gil(|py| { let example_py = make_example(py); assert_eq!(example_py.str().unwrap().to_str().unwrap(), "5"); }) } #[test] fn test_repr() { Python::with_gil(|py| { let example_py = make_example(py); assert_eq!( example_py.repr().unwrap().to_str().unwrap(), "ExampleClass(value=5)" ); }) } #[test] fn test_hash() { Python::with_gil(|py| { let example_py = make_example(py); assert_eq!(example_py.hash().unwrap(), 5); }) } #[test] fn test_bool() { Python::with_gil(|py| { let example_py = make_example(py); assert!(example_py.is_true().unwrap()); example_py.borrow_mut().value = 0; assert!(!example_py.is_true().unwrap()); }) } #[pyclass] pub struct LenOverflow; #[pymethods] impl LenOverflow { fn __len__(&self) -> usize { (isize::MAX as usize) + 1 } } #[test] fn len_overflow() { Python::with_gil(|py| { let inst = Py::new(py, LenOverflow).unwrap(); py_expect_exception!(py, inst, "len(inst)", PyOverflowError); }); } #[pyclass] pub struct Mapping { values: Py, } #[pymethods] impl Mapping { fn __len__(&self, py: Python<'_>) -> usize { self.values.as_ref(py).len() } fn __getitem__<'a>(&'a self, key: &'a PyAny) -> PyResult<&'a PyAny> { let any: &PyAny = self.values.as_ref(key.py()).as_ref(); any.get_item(key) } fn __setitem__(&self, key: &PyAny, value: &PyAny) -> PyResult<()> { self.values.as_ref(key.py()).set_item(key, value) } fn __delitem__(&self, key: &PyAny) -> PyResult<()> { self.values.as_ref(key.py()).del_item(key) } } #[test] fn mapping() { Python::with_gil(|py| { PyMapping::register::(py).unwrap(); let inst = Py::new( py, Mapping { values: PyDict::new(py).into(), }, ) .unwrap(); let mapping: &PyMapping = inst.as_ref(py).downcast().unwrap(); py_assert!(py, inst, "len(inst) == 0"); py_run!(py, inst, "inst['foo'] = 'foo'"); py_assert!(py, inst, "inst['foo'] == 'foo'"); py_run!(py, inst, "del inst['foo']"); py_expect_exception!(py, inst, "inst['foo']", PyKeyError); // Default iteration will call __getitem__ with integer indices // which fails with a KeyError py_expect_exception!(py, inst, "[*inst] == []", PyKeyError, "0"); // check mapping protocol assert_eq!(mapping.len().unwrap(), 0); mapping.set_item(0, 5).unwrap(); assert_eq!(mapping.len().unwrap(), 1); assert_eq!(mapping.get_item(0).unwrap().extract::().unwrap(), 5); mapping.del_item(0).unwrap(); assert_eq!(mapping.len().unwrap(), 0); }); } #[derive(FromPyObject)] enum SequenceIndex<'a> { Integer(isize), Slice(&'a PySlice), } #[pyclass] pub struct Sequence { values: Vec, } #[pymethods] impl Sequence { fn __len__(&self) -> usize { self.values.len() } fn __getitem__(&self, index: SequenceIndex<'_>) -> PyResult { match index { SequenceIndex::Integer(index) => { let uindex = self.usize_index(index)?; self.values .get(uindex) .map(Clone::clone) .ok_or_else(|| PyIndexError::new_err(index)) } // Just to prove that slicing can be implemented SequenceIndex::Slice(s) => Ok(s.into()), } } fn __setitem__(&mut self, index: isize, value: PyObject) -> PyResult<()> { let uindex = self.usize_index(index)?; self.values .get_mut(uindex) .map(|place| *place = value) .ok_or_else(|| PyIndexError::new_err(index)) } fn __delitem__(&mut self, index: isize) -> PyResult<()> { let uindex = self.usize_index(index)?; if uindex >= self.values.len() { Err(PyIndexError::new_err(index)) } else { self.values.remove(uindex); Ok(()) } } fn append(&mut self, value: PyObject) { self.values.push(value); } } impl Sequence { fn usize_index(&self, index: isize) -> PyResult { if index < 0 { let corrected_index = index + self.values.len() as isize; if corrected_index < 0 { Err(PyIndexError::new_err(index)) } else { Ok(corrected_index as usize) } } else { Ok(index as usize) } } } #[test] fn sequence() { Python::with_gil(|py| { PySequence::register::(py).unwrap(); let inst = Py::new(py, Sequence { values: vec![] }).unwrap(); let sequence: &PySequence = inst.as_ref(py).downcast().unwrap(); py_assert!(py, inst, "len(inst) == 0"); py_expect_exception!(py, inst, "inst[0]", PyIndexError); py_run!(py, inst, "inst.append('foo')"); py_assert!(py, inst, "inst[0] == 'foo'"); py_assert!(py, inst, "inst[-1] == 'foo'"); py_expect_exception!(py, inst, "inst[1]", PyIndexError); py_expect_exception!(py, inst, "inst[-2]", PyIndexError); py_assert!(py, inst, "[*inst] == ['foo']"); py_run!(py, inst, "del inst[0]"); py_expect_exception!(py, inst, "inst['foo']", PyTypeError); py_assert!(py, inst, "inst[0:2] == slice(0, 2)"); // check sequence protocol // we don't implement sequence length so that CPython doesn't attempt to correct negative // indices. assert!(sequence.len().is_err()); // however regular python len() works thanks to mp_len slot assert_eq!(inst.as_ref(py).len().unwrap(), 0); py_run!(py, inst, "inst.append(0)"); sequence.set_item(0, 5).unwrap(); assert_eq!(inst.as_ref(py).len().unwrap(), 1); assert_eq!(sequence.get_item(0).unwrap().extract::().unwrap(), 5); sequence.del_item(0).unwrap(); assert_eq!(inst.as_ref(py).len().unwrap(), 0); }); } #[pyclass] struct Iterator { iter: Box + Send>, } #[pymethods] impl Iterator { fn __iter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> { slf } fn __next__(mut slf: PyRefMut<'_, Self>) -> Option { slf.iter.next() } } #[test] fn iterator() { Python::with_gil(|py| { let inst = Py::new( py, Iterator { iter: Box::new(5..8), }, ) .unwrap(); py_assert!(py, inst, "iter(inst) is inst"); py_assert!(py, inst, "list(inst) == [5, 6, 7]"); }); } #[pyclass] struct Callable; #[pymethods] impl Callable { fn __call__(&self, arg: i32) -> i32 { arg * 6 } } #[pyclass] struct NotCallable; #[test] fn callable() { Python::with_gil(|py| { let c = Py::new(py, Callable).unwrap(); py_assert!(py, c, "callable(c)"); py_assert!(py, c, "c(7) == 42"); let nc = Py::new(py, NotCallable).unwrap(); py_assert!(py, nc, "not callable(nc)"); }); } #[pyclass] #[derive(Debug)] struct SetItem { key: i32, val: i32, } #[pymethods] impl SetItem { fn __setitem__(&mut self, key: i32, val: i32) { self.key = key; self.val = val; } } #[test] fn setitem() { Python::with_gil(|py| { let c = PyCell::new(py, SetItem { key: 0, val: 0 }).unwrap(); py_run!(py, c, "c[1] = 2"); { let c = c.borrow(); assert_eq!(c.key, 1); assert_eq!(c.val, 2); } py_expect_exception!(py, c, "del c[1]", PyNotImplementedError); }); } #[pyclass] struct DelItem { key: i32, } #[pymethods] impl DelItem { fn __delitem__(&mut self, key: i32) { self.key = key; } } #[test] fn delitem() { Python::with_gil(|py| { let c = PyCell::new(py, DelItem { key: 0 }).unwrap(); py_run!(py, c, "del c[1]"); { let c = c.borrow(); assert_eq!(c.key, 1); } py_expect_exception!(py, c, "c[1] = 2", PyNotImplementedError); }); } #[pyclass] struct SetDelItem { val: Option, } #[pymethods] impl SetDelItem { fn __setitem__(&mut self, _key: i32, val: i32) { self.val = Some(val); } fn __delitem__(&mut self, _key: i32) { self.val = None; } } #[test] fn setdelitem() { Python::with_gil(|py| { let c = PyCell::new(py, SetDelItem { val: None }).unwrap(); py_run!(py, c, "c[1] = 2"); { let c = c.borrow(); assert_eq!(c.val, Some(2)); } py_run!(py, c, "del c[1]"); let c = c.borrow(); assert_eq!(c.val, None); }); } #[pyclass] struct Contains {} #[pymethods] impl Contains { fn __contains__(&self, item: i32) -> bool { item >= 0 } } #[test] fn contains() { Python::with_gil(|py| { let c = Py::new(py, Contains {}).unwrap(); py_run!(py, c, "assert 1 in c"); py_run!(py, c, "assert -1 not in c"); py_expect_exception!(py, c, "assert 'wrong type' not in c", PyTypeError); }); } #[pyclass] struct GetItem {} #[pymethods] impl GetItem { fn __getitem__(&self, idx: &PyAny) -> PyResult<&'static str> { if let Ok(slice) = idx.downcast::() { let indices = slice.indices(1000)?; if indices.start == 100 && indices.stop == 200 && indices.step == 1 { return Ok("slice"); } } else if let Ok(idx) = idx.extract::() { if idx == 1 { return Ok("int"); } } Err(PyValueError::new_err("error")) } } #[test] fn test_getitem() { Python::with_gil(|py| { let ob = Py::new(py, GetItem {}).unwrap(); py_assert!(py, ob, "ob[1] == 'int'"); py_assert!(py, ob, "ob[100:200:1] == 'slice'"); }); } #[pyclass] struct ClassWithGetAttr { #[pyo3(get, set)] data: u32, } #[pymethods] impl ClassWithGetAttr { fn __getattr__(&self, _name: &str) -> u32 { self.data * 2 } } #[test] fn getattr_doesnt_override_member() { Python::with_gil(|py| { let inst = PyCell::new(py, ClassWithGetAttr { data: 4 }).unwrap(); py_assert!(py, inst, "inst.data == 4"); py_assert!(py, inst, "inst.a == 8"); }); } #[pyclass] struct ClassWithGetAttribute { #[pyo3(get, set)] data: u32, } #[pymethods] impl ClassWithGetAttribute { fn __getattribute__(&self, _name: &str) -> u32 { self.data * 2 } } #[test] fn getattribute_overrides_member() { Python::with_gil(|py| { let inst = PyCell::new(py, ClassWithGetAttribute { data: 4 }).unwrap(); py_assert!(py, inst, "inst.data == 8"); py_assert!(py, inst, "inst.y == 8"); }); } #[pyclass] struct ClassWithGetAttrAndGetAttribute; #[pymethods] impl ClassWithGetAttrAndGetAttribute { fn __getattribute__(&self, name: &str) -> PyResult { if name == "exists" { Ok(42) } else if name == "error" { Err(PyValueError::new_err("bad")) } else { Err(PyAttributeError::new_err("fallback")) } } fn __getattr__(&self, name: &str) -> PyResult { if name == "lucky" { Ok(57) } else { Err(PyAttributeError::new_err("no chance")) } } } #[test] fn getattr_and_getattribute() { Python::with_gil(|py| { let inst = PyCell::new(py, ClassWithGetAttrAndGetAttribute).unwrap(); py_assert!(py, inst, "inst.exists == 42"); py_assert!(py, inst, "inst.lucky == 57"); py_expect_exception!(py, inst, "inst.error", PyValueError); py_expect_exception!(py, inst, "inst.unlucky", PyAttributeError); }); } /// Wraps a Python future and yield it once. #[pyclass] #[derive(Debug)] struct OnceFuture { future: PyObject, polled: bool, } #[pymethods] impl OnceFuture { #[new] fn new(future: PyObject) -> Self { OnceFuture { future, polled: false, } } fn __await__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> { slf } fn __iter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> { slf } fn __next__(mut slf: PyRefMut<'_, Self>) -> Option { if !slf.polled { slf.polled = true; Some(slf.future.clone()) } else { None } } } #[test] #[cfg(not(target_arch = "wasm32"))] // Won't work without wasm32 event loop (e.g., Pyodide has WebLoop) fn test_await() { Python::with_gil(|py| { let once = py.get_type::(); let source = r#" import asyncio import sys async def main(): res = await Once(await asyncio.sleep(0.1)) assert res is None # For an odd error similar to https://bugs.python.org/issue38563 if sys.platform == "win32" and sys.version_info >= (3, 8, 0): asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) asyncio.run(main()) "#; let globals = PyModule::import(py, "__main__").unwrap().dict(); globals.set_item("Once", once).unwrap(); py.run(source, Some(globals), None) .map_err(|e| e.display(py)) .unwrap(); }); } #[pyclass] struct AsyncIterator { future: Option>, } #[pymethods] impl AsyncIterator { #[new] fn new(future: Py) -> Self { Self { future: Some(future), } } fn __aiter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> { slf } fn __anext__(&mut self) -> Option> { self.future.take() } } #[test] #[cfg(not(target_arch = "wasm32"))] // Won't work without wasm32 event loop (e.g., Pyodide has WebLoop) fn test_anext_aiter() { Python::with_gil(|py| { let once = py.get_type::(); let source = r#" import asyncio import sys async def main(): count = 0 async for result in AsyncIterator(Once(await asyncio.sleep(0.1))): # The Once is awaited as part of the `async for` and produces None assert result is None count +=1 assert count == 1 # For an odd error similar to https://bugs.python.org/issue38563 if sys.platform == "win32" and sys.version_info >= (3, 8, 0): asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) asyncio.run(main()) "#; let globals = PyModule::import(py, "__main__").unwrap().dict(); globals.set_item("Once", once).unwrap(); globals .set_item("AsyncIterator", py.get_type::()) .unwrap(); py.run(source, Some(globals), None) .map_err(|e| e.display(py)) .unwrap(); }); } /// Increment the count when `__get__` is called. #[pyclass] struct DescrCounter { #[pyo3(get)] count: usize, } #[pymethods] impl DescrCounter { #[new] fn new() -> Self { DescrCounter { count: 0 } } /// Each access will increase the count fn __get__<'a>( mut slf: PyRefMut<'a, Self>, _instance: &PyAny, _owner: Option<&PyType>, ) -> PyRefMut<'a, Self> { slf.count += 1; slf } /// Allow assigning a new counter to the descriptor, copying the count across fn __set__(&self, _instance: &PyAny, new_value: &mut Self) { new_value.count = self.count; } /// Delete to reset the counter fn __delete__(&mut self, _instance: &PyAny) { self.count = 0; } } #[test] fn descr_getset() { Python::with_gil(|py| { let counter = py.get_type::(); let source = pyo3::indoc::indoc!( r#" class Class: counter = Counter() # access via type counter = Class.counter assert counter.count == 1 # access with instance directly assert Counter.__get__(counter, Class()).count == 2 # access via instance c = Class() assert c.counter.count == 3 # __set__ c.counter = Counter() assert c.counter.count == 4 # __delete__ del c.counter assert c.counter.count == 1 "# ); let globals = PyModule::import(py, "__main__").unwrap().dict(); globals.set_item("Counter", counter).unwrap(); py.run(source, Some(globals), None) .map_err(|e| e.display(py)) .unwrap(); }); } #[pyclass] struct NotHashable; #[pymethods] impl NotHashable { #[classattr] const __hash__: Option = None; } #[test] fn test_hash_opt_out() { // By default Python provides a hash implementation, which can be disabled by setting __hash__ // to None. Python::with_gil(|py| { let empty = Py::new(py, EmptyClass).unwrap(); py_assert!(py, empty, "hash(empty) is not None"); let not_hashable = Py::new(py, NotHashable).unwrap(); py_expect_exception!(py, not_hashable, "hash(not_hashable)", PyTypeError); }) } /// Class with __iter__ gets default contains from CPython. #[pyclass] struct DefaultedContains; #[pymethods] impl DefaultedContains { fn __iter__(&self, py: Python<'_>) -> PyObject { PyList::new(py, ["a", "b", "c"]) .as_ref() .iter() .unwrap() .into() } } #[pyclass] struct NoContains; #[pymethods] impl NoContains { fn __iter__(&self, py: Python<'_>) -> PyObject { PyList::new(py, ["a", "b", "c"]) .as_ref() .iter() .unwrap() .into() } // Equivalent to the opt-out const form in NotHashable above, just more verbose, to confirm this // also works. #[classattr] fn __contains__() -> Option { None } } #[test] fn test_contains_opt_out() { Python::with_gil(|py| { let defaulted_contains = Py::new(py, DefaultedContains).unwrap(); py_assert!(py, defaulted_contains, "'a' in defaulted_contains"); let no_contains = Py::new(py, NoContains).unwrap(); py_expect_exception!(py, no_contains, "'a' in no_contains", PyTypeError); }) } pyo3-0.20.2/tests/test_pyfunction.rs000064400000000000000000000356251046102023000155570ustar 00000000000000#![cfg(feature = "macros")] use std::collections::HashMap; #[cfg(not(Py_LIMITED_API))] use pyo3::buffer::PyBuffer; use pyo3::prelude::*; #[cfg(not(Py_LIMITED_API))] use pyo3::types::PyDateTime; #[cfg(not(any(Py_LIMITED_API, PyPy)))] use pyo3::types::PyFunction; use pyo3::types::{self, PyCFunction}; #[path = "../src/tests/common.rs"] mod common; #[pyfunction(signature = (arg = true))] fn optional_bool(arg: Option) -> String { format!("{:?}", arg) } #[test] fn test_optional_bool() { // Regression test for issue #932 Python::with_gil(|py| { let f = wrap_pyfunction!(optional_bool)(py).unwrap(); py_assert!(py, f, "f() == 'Some(true)'"); py_assert!(py, f, "f(True) == 'Some(true)'"); py_assert!(py, f, "f(False) == 'Some(false)'"); py_assert!(py, f, "f(None) == 'None'"); }); } #[cfg(not(Py_LIMITED_API))] #[pyfunction] fn buffer_inplace_add(py: Python<'_>, x: PyBuffer, y: PyBuffer) { let x = x.as_mut_slice(py).unwrap(); let y = y.as_slice(py).unwrap(); for (xi, yi) in x.iter().zip(y) { let xi_plus_yi = xi.get() + yi.get(); xi.set(xi_plus_yi); } } #[cfg(not(Py_LIMITED_API))] #[test] fn test_buffer_add() { Python::with_gil(|py| { let f = wrap_pyfunction!(buffer_inplace_add)(py).unwrap(); py_expect_exception!( py, f, r#" import array a = array.array("i", [0, 1, 2, 3]) b = array.array("I", [0, 1, 2, 3]) f(a, b) "#, PyBufferError ); pyo3::py_run!( py, f, r#" import array a = array.array("i", [0, 1, 2, 3]) b = array.array("i", [2, 3, 4, 5]) f(a, b) assert a, array.array("i", [2, 4, 6, 8]) "# ); }); } #[cfg(not(any(Py_LIMITED_API, PyPy)))] #[pyfunction] fn function_with_pyfunction_arg(fun: &PyFunction) -> PyResult<&PyAny> { fun.call((), None) } #[pyfunction] fn function_with_pycfunction_arg(fun: &PyCFunction) -> PyResult<&PyAny> { fun.call((), None) } #[test] fn test_functions_with_function_args() { Python::with_gil(|py| { let py_cfunc_arg = wrap_pyfunction!(function_with_pycfunction_arg)(py).unwrap(); let bool_to_string = wrap_pyfunction!(optional_bool)(py).unwrap(); pyo3::py_run!( py, py_cfunc_arg bool_to_string, r#" assert py_cfunc_arg(bool_to_string) == "Some(true)" "# ); #[cfg(not(any(Py_LIMITED_API, PyPy)))] { let py_func_arg = wrap_pyfunction!(function_with_pyfunction_arg)(py).unwrap(); pyo3::py_run!( py, py_func_arg, r#" def foo(): return "bar" assert py_func_arg(foo) == "bar" "# ); } }); } #[cfg(not(Py_LIMITED_API))] fn datetime_to_timestamp(dt: &PyAny) -> PyResult { let dt: &PyDateTime = dt.extract()?; let ts: f64 = dt.call_method0("timestamp")?.extract()?; Ok(ts as i64) } #[cfg(not(Py_LIMITED_API))] #[pyfunction] fn function_with_custom_conversion( #[pyo3(from_py_with = "datetime_to_timestamp")] timestamp: i64, ) -> i64 { timestamp } #[cfg(not(Py_LIMITED_API))] #[test] fn test_function_with_custom_conversion() { Python::with_gil(|py| { let custom_conv_func = wrap_pyfunction!(function_with_custom_conversion)(py).unwrap(); pyo3::py_run!( py, custom_conv_func, r#" import datetime dt = datetime.datetime.fromtimestamp(1612040400) assert custom_conv_func(dt) == 1612040400 "# ) }); } #[cfg(not(Py_LIMITED_API))] #[test] fn test_function_with_custom_conversion_error() { Python::with_gil(|py| { let custom_conv_func = wrap_pyfunction!(function_with_custom_conversion)(py).unwrap(); py_expect_exception!( py, custom_conv_func, "custom_conv_func(['a'])", PyTypeError, "argument 'timestamp': 'list' object cannot be converted to 'PyDateTime'" ); }); } #[test] fn test_from_py_with_defaults() { fn optional_int(x: &PyAny) -> PyResult> { if x.is_none() { Ok(None) } else { Some(x.extract()).transpose() } } // issue 2280 combination of from_py_with and Option did not compile #[pyfunction] fn from_py_with_option(#[pyo3(from_py_with = "optional_int")] int: Option) -> i32 { int.unwrap_or(0) } #[pyfunction(signature = (len=0))] fn from_py_with_default(#[pyo3(from_py_with = "PyAny::len")] len: usize) -> usize { len } Python::with_gil(|py| { let f = wrap_pyfunction!(from_py_with_option)(py).unwrap(); assert_eq!(f.call0().unwrap().extract::().unwrap(), 0); assert_eq!(f.call1((123,)).unwrap().extract::().unwrap(), 123); assert_eq!(f.call1((999,)).unwrap().extract::().unwrap(), 999); let f2 = wrap_pyfunction!(from_py_with_default)(py).unwrap(); assert_eq!(f2.call0().unwrap().extract::().unwrap(), 0); assert_eq!(f2.call1(("123",)).unwrap().extract::().unwrap(), 3); assert_eq!(f2.call1(("1234",)).unwrap().extract::().unwrap(), 4); }); } #[pyclass] #[derive(Debug, FromPyObject)] struct ValueClass { #[pyo3(get)] value: usize, } #[pyfunction] fn conversion_error( str_arg: &str, int_arg: i64, tuple_arg: (&str, f64), option_arg: Option, struct_arg: Option, ) { println!( "{:?} {:?} {:?} {:?} {:?}", str_arg, int_arg, tuple_arg, option_arg, struct_arg ); } #[test] fn test_conversion_error() { Python::with_gil(|py| { let conversion_error = wrap_pyfunction!(conversion_error)(py).unwrap(); py_expect_exception!( py, conversion_error, "conversion_error(None, None, None, None, None)", PyTypeError, "argument 'str_arg': 'NoneType' object cannot be converted to 'PyString'" ); py_expect_exception!( py, conversion_error, "conversion_error(100, None, None, None, None)", PyTypeError, "argument 'str_arg': 'int' object cannot be converted to 'PyString'" ); py_expect_exception!( py, conversion_error, "conversion_error('string1', 'string2', None, None, None)", PyTypeError, "argument 'int_arg': 'str' object cannot be interpreted as an integer" ); py_expect_exception!( py, conversion_error, "conversion_error('string1', -100, 'string2', None, None)", PyTypeError, "argument 'tuple_arg': 'str' object cannot be converted to 'PyTuple'" ); py_expect_exception!( py, conversion_error, "conversion_error('string1', -100, ('string2', 10.), 'string3', None)", PyTypeError, "argument 'option_arg': 'str' object cannot be interpreted as an integer" ); let exception = py_expect_exception!( py, conversion_error, " class ValueClass: def __init__(self, value): self.value = value conversion_error('string1', -100, ('string2', 10.), None, ValueClass(\"no_expected_type\"))", PyTypeError ); assert_eq!( extract_traceback(py, exception), "TypeError: argument 'struct_arg': failed to \ extract field ValueClass.value: TypeError: 'str' object cannot be interpreted as an integer" ); let exception = py_expect_exception!( py, conversion_error, " class ValueClass: def __init__(self, value): self.value = value conversion_error('string1', -100, ('string2', 10.), None, ValueClass(-5))", PyTypeError ); assert_eq!( extract_traceback(py, exception), "TypeError: argument 'struct_arg': failed to \ extract field ValueClass.value: OverflowError: can't convert negative int to unsigned" ); }); } /// Helper function that concatenates the error message from /// each error in the traceback into a single string that can /// be tested. fn extract_traceback(py: Python<'_>, mut error: PyErr) -> String { let mut error_msg = error.to_string(); while let Some(cause) = error.cause(py) { error_msg.push_str(": "); error_msg.push_str(&cause.to_string()); error = cause } error_msg } #[test] fn test_pycfunction_new() { use pyo3::ffi; Python::with_gil(|py| { unsafe extern "C" fn c_fn( _self: *mut ffi::PyObject, _args: *mut ffi::PyObject, ) -> *mut ffi::PyObject { ffi::PyLong_FromLong(4200) } let py_fn = PyCFunction::new( c_fn, "py_fn", "py_fn for test (this is the docstring)", py.into(), ) .unwrap(); py_assert!(py, py_fn, "py_fn() == 4200"); py_assert!( py, py_fn, "py_fn.__doc__ == 'py_fn for test (this is the docstring)'" ); }); } #[test] fn test_pycfunction_new_with_keywords() { use pyo3::ffi; use std::ffi::CString; use std::os::raw::{c_char, c_long}; use std::ptr; Python::with_gil(|py| { unsafe extern "C" fn c_fn( _self: *mut ffi::PyObject, args: *mut ffi::PyObject, kwds: *mut ffi::PyObject, ) -> *mut ffi::PyObject { let mut foo: c_long = 0; let mut bar: c_long = 0; let foo_ptr: *mut c_long = &mut foo; let bar_ptr: *mut c_long = &mut bar; let foo_name = CString::new("foo").unwrap(); let foo_name_raw: *mut c_char = foo_name.into_raw(); let kw_bar_name = CString::new("kw_bar").unwrap(); let kw_bar_name_raw: *mut c_char = kw_bar_name.into_raw(); let mut arglist = vec![foo_name_raw, kw_bar_name_raw, ptr::null_mut()]; let arglist_ptr: *mut *mut c_char = arglist.as_mut_ptr(); let arg_pattern: *const c_char = CString::new("l|l").unwrap().into_raw(); ffi::PyArg_ParseTupleAndKeywords( args, kwds, arg_pattern, arglist_ptr, foo_ptr, bar_ptr, ); ffi::PyLong_FromLong(foo * bar) } let py_fn = PyCFunction::new_with_keywords( c_fn, "py_fn", "py_fn for test (this is the docstring)", py.into(), ) .unwrap(); py_assert!(py, py_fn, "py_fn(42, kw_bar=100) == 4200"); py_assert!(py, py_fn, "py_fn(foo=42, kw_bar=100) == 4200"); py_assert!( py, py_fn, "py_fn.__doc__ == 'py_fn for test (this is the docstring)'" ); }); } #[test] fn test_closure() { Python::with_gil(|py| { let f = |args: &types::PyTuple, _kwargs: Option<&types::PyDict>| -> PyResult<_> { Python::with_gil(|py| { let res: Vec<_> = args .iter() .map(|elem| { if let Ok(i) = elem.extract::() { (i + 1).into_py(py) } else if let Ok(f) = elem.extract::() { (2. * f).into_py(py) } else if let Ok(mut s) = elem.extract::() { s.push_str("-py"); s.into_py(py) } else { panic!("unexpected argument type for {:?}", elem) } }) .collect(); Ok(res) }) }; let closure_py = PyCFunction::new_closure(py, Some("test_fn"), Some("test_fn doc"), f).unwrap(); py_assert!(py, closure_py, "closure_py(42) == [43]"); py_assert!(py, closure_py, "closure_py.__name__ == 'test_fn'"); py_assert!(py, closure_py, "closure_py.__doc__ == 'test_fn doc'"); py_assert!( py, closure_py, "closure_py(42, 3.14, 'foo') == [43, 6.28, 'foo-py']" ); }); } #[test] fn test_closure_counter() { Python::with_gil(|py| { let counter = std::cell::RefCell::new(0); let counter_fn = move |_args: &types::PyTuple, _kwargs: Option<&types::PyDict>| -> PyResult { let mut counter = counter.borrow_mut(); *counter += 1; Ok(*counter) }; let counter_py = PyCFunction::new_closure(py, None, None, counter_fn).unwrap(); py_assert!(py, counter_py, "counter_py() == 1"); py_assert!(py, counter_py, "counter_py() == 2"); py_assert!(py, counter_py, "counter_py() == 3"); }); } #[test] fn use_pyfunction() { mod function_in_module { use pyo3::prelude::*; #[pyfunction] pub fn foo(x: i32) -> i32 { x } } Python::with_gil(|py| { use function_in_module::foo; // check imported name can be wrapped let f = wrap_pyfunction!(foo, py).unwrap(); assert_eq!(f.call1((5,)).unwrap().extract::().unwrap(), 5); assert_eq!(f.call1((42,)).unwrap().extract::().unwrap(), 42); // check path import can be wrapped let f2 = wrap_pyfunction!(function_in_module::foo, py).unwrap(); assert_eq!(f2.call1((5,)).unwrap().extract::().unwrap(), 5); assert_eq!(f2.call1((42,)).unwrap().extract::().unwrap(), 42); }) } #[pyclass] struct Key(String); #[pyclass] struct Value(i32); #[pyfunction] fn return_value_borrows_from_arguments<'py>( py: Python<'py>, key: &'py Key, value: &'py Value, ) -> HashMap<&'py str, i32> { py.allow_threads(move || { let mut map = HashMap::new(); map.insert(key.0.as_str(), value.0); map }) } #[test] fn test_return_value_borrows_from_arguments() { Python::with_gil(|py| { let function = wrap_pyfunction!(return_value_borrows_from_arguments, py).unwrap(); let key = Py::new(py, Key("key".to_owned())).unwrap(); let value = Py::new(py, Value(42)).unwrap(); py_assert!(py, function key value, "function(key, value) == { \"key\": 42 }"); }); } #[test] fn test_some_wrap_arguments() { // https://github.com/PyO3/pyo3/issues/3460 const NONE: Option = None; #[pyfunction(signature = (a = 1, b = Some(2), c = None, d = NONE))] fn some_wrap_arguments( a: Option, b: Option, c: Option, d: Option, ) -> [Option; 4] { [a, b, c, d] } Python::with_gil(|py| { let function = wrap_pyfunction!(some_wrap_arguments, py).unwrap(); py_assert!(py, function, "function() == [1, 2, None, None]"); }) } pyo3-0.20.2/tests/test_pyself.rs000064400000000000000000000060101046102023000146450ustar 00000000000000#![cfg(feature = "macros")] //! Test slf: PyRef/PyMutRef(especially, slf.into::) works use pyo3::prelude::*; use pyo3::types::{PyBytes, PyString}; use pyo3::PyCell; use std::collections::HashMap; #[path = "../src/tests/common.rs"] mod common; /// Assumes it's a file reader or so. /// Inspired by https://github.com/jothan/cordoba, thanks. #[pyclass] #[derive(Clone, Debug)] struct Reader { inner: HashMap, } #[pymethods] impl Reader { fn clone_ref(slf: &PyCell) -> &PyCell { slf } fn clone_ref_with_py<'py>(slf: &'py PyCell, _py: Python<'py>) -> &'py PyCell { slf } fn get_iter(slf: &PyCell, keys: Py) -> Iter { Iter { reader: slf.into(), keys, idx: 0, } } fn get_iter_and_reset( mut slf: PyRefMut<'_, Self>, keys: Py, py: Python<'_>, ) -> PyResult { let reader = Py::new(py, slf.clone())?; slf.inner.clear(); Ok(Iter { reader, keys, idx: 0, }) } } #[pyclass] #[derive(Debug)] struct Iter { reader: Py, keys: Py, idx: usize, } #[pymethods] impl Iter { #[allow(clippy::self_named_constructors)] fn __iter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> { slf } fn __next__(mut slf: PyRefMut<'_, Self>) -> PyResult> { let bytes = slf.keys.as_ref(slf.py()).as_bytes(); match bytes.get(slf.idx) { Some(&b) => { slf.idx += 1; let py = slf.py(); let reader = slf.reader.as_ref(py); let reader_ref = reader.try_borrow()?; let res = reader_ref .inner .get(&b) .map(|s| PyString::new(py, s).into()); Ok(res) } None => Ok(None), } } } fn reader() -> Reader { let reader = [(1, "a"), (2, "b"), (3, "c"), (4, "d"), (5, "e")]; Reader { inner: reader.iter().map(|(k, v)| (*k, (*v).to_string())).collect(), } } #[test] fn test_nested_iter() { Python::with_gil(|py| { let reader: PyObject = reader().into_py(py); py_assert!( py, reader, "list(reader.get_iter(bytes([3, 5, 2]))) == ['c', 'e', 'b']" ); }); } #[test] fn test_clone_ref() { Python::with_gil(|py| { let reader: PyObject = reader().into_py(py); py_assert!(py, reader, "reader == reader.clone_ref()"); py_assert!(py, reader, "reader == reader.clone_ref_with_py()"); }); } #[test] fn test_nested_iter_reset() { Python::with_gil(|py| { let reader = PyCell::new(py, reader()).unwrap(); py_assert!( py, reader, "list(reader.get_iter_and_reset(bytes([3, 5, 2]))) == ['c', 'e', 'b']" ); let reader_ref = reader.borrow(); assert!(reader_ref.inner.is_empty()); }); } pyo3-0.20.2/tests/test_sequence.rs000064400000000000000000000216171046102023000151650ustar 00000000000000#![cfg(feature = "macros")] use pyo3::exceptions::{PyIndexError, PyValueError}; use pyo3::types::{IntoPyDict, PyList, PyMapping, PySequence}; use pyo3::{ffi, prelude::*}; use pyo3::py_run; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct ByteSequence { elements: Vec, } #[pymethods] impl ByteSequence { #[new] fn new(elements: Option<&PyList>) -> PyResult { if let Some(pylist) = elements { let mut elems = Vec::with_capacity(pylist.len()); for pyelem in pylist { let elem = u8::extract(pyelem)?; elems.push(elem); } Ok(Self { elements: elems }) } else { Ok(Self { elements: Vec::new(), }) } } fn __len__(&self) -> usize { self.elements.len() } fn __getitem__(&self, idx: isize) -> PyResult { self.elements .get(idx as usize) .copied() .ok_or_else(|| PyIndexError::new_err("list index out of range")) } fn __setitem__(&mut self, idx: isize, value: u8) { self.elements[idx as usize] = value; } fn __delitem__(&mut self, mut idx: isize) -> PyResult<()> { let self_len = self.elements.len() as isize; if idx < 0 { idx += self_len; } if (idx < self_len) && (idx >= 0) { self.elements.remove(idx as usize); Ok(()) } else { Err(PyIndexError::new_err("list index out of range")) } } fn __contains__(&self, other: &PyAny) -> bool { match u8::extract(other) { Ok(x) => self.elements.contains(&x), Err(_) => false, } } fn __concat__(&self, other: &Self) -> Self { let mut elements = self.elements.clone(); elements.extend_from_slice(&other.elements); Self { elements } } fn __inplace_concat__(mut slf: PyRefMut<'_, Self>, other: &Self) -> Py { slf.elements.extend_from_slice(&other.elements); slf.into() } fn __repeat__(&self, count: isize) -> PyResult { if count >= 0 { let mut elements = Vec::with_capacity(self.elements.len() * count as usize); for _ in 0..count { elements.extend(&self.elements); } Ok(Self { elements }) } else { Err(PyValueError::new_err("invalid repeat count")) } } fn __inplace_repeat__(mut slf: PyRefMut<'_, Self>, count: isize) -> PyResult> { if count >= 0 { let mut elements = Vec::with_capacity(slf.elements.len() * count as usize); for _ in 0..count { elements.extend(&slf.elements); } slf.elements = elements; Ok(slf.into()) } else { Err(PyValueError::new_err("invalid repeat count")) } } } /// Return a dict with `s = ByteSequence([1, 2, 3])`. fn seq_dict(py: Python<'_>) -> &pyo3::types::PyDict { let d = [("ByteSequence", py.get_type::())].into_py_dict(py); // Though we can construct `s` in Rust, let's test `__new__` works. py_run!(py, *d, "s = ByteSequence([1, 2, 3])"); d } #[test] fn test_getitem() { Python::with_gil(|py| { let d = seq_dict(py); py_assert!(py, *d, "s[0] == 1"); py_assert!(py, *d, "s[1] == 2"); py_assert!(py, *d, "s[2] == 3"); py_expect_exception!(py, *d, "print(s[-4])", PyIndexError); py_expect_exception!(py, *d, "print(s[4])", PyIndexError); }); } #[test] fn test_setitem() { Python::with_gil(|py| { let d = seq_dict(py); py_run!(py, *d, "s[0] = 4; assert list(s) == [4, 2, 3]"); py_expect_exception!(py, *d, "s[0] = 'hello'", PyTypeError); }); } #[test] fn test_delitem() { Python::with_gil(|py| { let d = [("ByteSequence", py.get_type::())].into_py_dict(py); py_run!( py, *d, "s = ByteSequence([1, 2, 3]); del s[0]; assert list(s) == [2, 3]" ); py_run!( py, *d, "s = ByteSequence([1, 2, 3]); del s[1]; assert list(s) == [1, 3]" ); py_run!( py, *d, "s = ByteSequence([1, 2, 3]); del s[-1]; assert list(s) == [1, 2]" ); py_run!( py, *d, "s = ByteSequence([1, 2, 3]); del s[-2]; assert list(s) == [1, 3]" ); py_expect_exception!( py, *d, "s = ByteSequence([1, 2, 3]); del s[-4]; print(list(s))", PyIndexError ); py_expect_exception!( py, *d, "s = ByteSequence([1, 2, 3]); del s[4]", PyIndexError ); }); } #[test] fn test_contains() { Python::with_gil(|py| { let d = seq_dict(py); py_assert!(py, *d, "1 in s"); py_assert!(py, *d, "2 in s"); py_assert!(py, *d, "3 in s"); py_assert!(py, *d, "4 not in s"); py_assert!(py, *d, "'hello' not in s"); }); } #[test] fn test_concat() { Python::with_gil(|py| { let d = seq_dict(py); py_run!( py, *d, "s1 = ByteSequence([1, 2]); s2 = ByteSequence([3, 4]); assert list(s1 + s2) == [1, 2, 3, 4]" ); py_expect_exception!( py, *d, "s1 = ByteSequence([1, 2]); s2 = 'hello'; s1 + s2", PyTypeError ); }); } #[test] fn test_inplace_concat() { Python::with_gil(|py| { let d = seq_dict(py); py_run!( py, *d, "s += ByteSequence([4, 5]); assert list(s) == [1, 2, 3, 4, 5]" ); py_expect_exception!(py, *d, "s += 'hello'", PyTypeError); }); } #[test] fn test_repeat() { Python::with_gil(|py| { let d = seq_dict(py); py_run!(py, *d, "s2 = s * 2; assert list(s2) == [1, 2, 3, 1, 2, 3]"); py_expect_exception!(py, *d, "s2 = s * -1", PyValueError); }); } #[test] fn test_inplace_repeat() { Python::with_gil(|py| { let d = [("ByteSequence", py.get_type::())].into_py_dict(py); py_run!( py, *d, "s = ByteSequence([1, 2]); s *= 3; assert list(s) == [1, 2, 1, 2, 1, 2]" ); py_expect_exception!(py, *d, "s = ByteSequence([1, 2]); s *= -1", PyValueError); }); } // Check that #[pyo3(get, set)] works correctly for Vec #[pyclass] struct GenericList { #[pyo3(get, set)] items: Vec, } #[test] fn test_generic_list_get() { Python::with_gil(|py| { let list: PyObject = GenericList { items: [1, 2, 3].iter().map(|i| i.to_object(py)).collect(), } .into_py(py); py_assert!(py, list, "list.items == [1, 2, 3]"); }); } #[test] fn test_generic_list_set() { Python::with_gil(|py| { let list = PyCell::new(py, GenericList { items: vec![] }).unwrap(); py_run!(py, list, "list.items = [1, 2, 3]"); assert!(list .borrow() .items .iter() .zip(&[1u32, 2, 3]) .all(|(a, b)| a.as_ref(py).eq(&b.into_py(py)).unwrap())); }); } #[pyclass(sequence)] struct OptionList { #[pyo3(get, set)] items: Vec>, } #[pymethods] impl OptionList { fn __len__(&self) -> usize { self.items.len() } fn __getitem__(&self, idx: isize) -> PyResult> { match self.items.get(idx as usize) { Some(x) => Ok(*x), None => Err(PyIndexError::new_err("Index out of bounds")), } } } #[test] fn test_option_list_get() { // Regression test for #798 Python::with_gil(|py| { let list = PyCell::new( py, OptionList { items: vec![Some(1), None], }, ) .unwrap(); py_assert!(py, list, "list[0] == 1"); py_assert!(py, list, "list[1] == None"); py_expect_exception!(py, list, "list[2]", PyIndexError); }); } #[test] fn sequence_is_not_mapping() { Python::with_gil(|py| { let list = PyCell::new( py, OptionList { items: vec![Some(1), None], }, ) .unwrap(); PySequence::register::(py).unwrap(); assert!(list.as_ref().downcast::().is_err()); assert!(list.as_ref().downcast::().is_ok()); }) } #[test] fn sequence_length() { Python::with_gil(|py| { let list = PyCell::new( py, OptionList { items: vec![Some(1), None], }, ) .unwrap(); assert_eq!(list.len().unwrap(), 2); assert_eq!(unsafe { ffi::PySequence_Length(list.as_ptr()) }, 2); assert_eq!(unsafe { ffi::PyMapping_Length(list.as_ptr()) }, -1); unsafe { ffi::PyErr_Clear() }; }) } pyo3-0.20.2/tests/test_serde.rs000064400000000000000000000044621046102023000144560ustar 00000000000000#[cfg(feature = "serde")] mod test_serde { use pyo3::prelude::*; use serde::{Deserialize, Serialize}; #[pyclass] #[derive(Debug, Serialize, Deserialize)] struct Group { name: String, } #[pyclass] #[derive(Debug, Clone, Serialize, Deserialize)] struct User { username: String, group: Option>, friends: Vec>, } #[test] fn test_serialize() { let friend1 = User { username: "friend 1".into(), group: None, friends: vec![], }; let friend2 = User { username: "friend 2".into(), ..friend1.clone() }; let user = Python::with_gil(|py| { let py_friend1 = Py::new(py, friend1).expect("failed to create friend 1"); let py_friend2 = Py::new(py, friend2).expect("failed to create friend 2"); let friends = vec![py_friend1, py_friend2]; let py_group = Py::new( py, Group { name: "group name".into(), }, ) .unwrap(); User { username: "danya".into(), group: Some(py_group), friends, } }); let serialized = serde_json::to_string(&user).expect("failed to serialize"); assert_eq!( serialized, r#"{"username":"danya","group":{"name":"group name"},"friends":[{"username":"friend 1","group":null,"friends":[]},{"username":"friend 2","group":null,"friends":[]}]}"# ); } #[test] fn test_deserialize() { let serialized = r#"{"username": "danya", "friends": [{"username": "friend", "group": {"name": "danya's friends"}, "friends": []}]}"#; let user: User = serde_json::from_str(serialized).expect("failed to deserialize"); assert_eq!(user.username, "danya"); assert!(user.group.is_none()); assert_eq!(user.friends.len(), 1usize); let friend = user.friends.first().unwrap(); Python::with_gil(|py| { assert_eq!(friend.borrow(py).username, "friend"); assert_eq!( friend.borrow(py).group.as_ref().unwrap().borrow(py).name, "danya's friends" ) }); } } pyo3-0.20.2/tests/test_static_slots.rs000064400000000000000000000026141046102023000160640ustar 00000000000000#![cfg(feature = "macros")] use pyo3::exceptions::PyIndexError; use pyo3::prelude::*; use pyo3::types::IntoPyDict; use pyo3::py_run; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct Count5(); #[pymethods] impl Count5 { #[new] fn new() -> Self { Self() } #[staticmethod] fn __len__() -> usize { 5 } #[staticmethod] fn __getitem__(idx: isize) -> PyResult { if idx < 0 { Err(PyIndexError::new_err("Count5 cannot count backwards")) } else if idx > 4 { Err(PyIndexError::new_err("Count5 cannot count higher than 5")) } else { Ok(idx as f64 + 1.0) } } } /// Return a dict with `s = Count5()`. fn test_dict(py: Python<'_>) -> &pyo3::types::PyDict { let d = [("Count5", py.get_type::())].into_py_dict(py); // Though we can construct `s` in Rust, let's test `__new__` works. py_run!(py, *d, "s = Count5()"); d } #[test] fn test_len() { Python::with_gil(|py| { let d = test_dict(py); py_assert!(py, *d, "len(s) == 5"); }); } #[test] fn test_getitem() { Python::with_gil(|py| { let d = test_dict(py); py_assert!(py, *d, "s[4] == 5.0"); }); } #[test] fn test_list() { Python::with_gil(|py| { let d = test_dict(py); py_assert!(py, *d, "list(s) == [1.0, 2.0, 3.0, 4.0, 5.0]"); }); } pyo3-0.20.2/tests/test_string.rs000064400000000000000000000010221046102023000146470ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; #[path = "../src/tests/common.rs"] mod common; #[pyfunction] fn take_str(_s: &str) {} #[test] fn test_unicode_encode_error() { Python::with_gil(|py| { let take_str = wrap_pyfunction!(take_str)(py).unwrap(); py_expect_exception!( py, take_str, "take_str('\\ud800')", PyUnicodeEncodeError, "'utf-8' codec can't encode character '\\ud800' in position 0: surrogates not allowed" ); }); } pyo3-0.20.2/tests/test_super.rs000064400000000000000000000015641046102023000145120ustar 00000000000000#![cfg(all(feature = "macros", not(PyPy)))] use pyo3::prelude::*; #[pyclass(subclass)] struct BaseClass { val1: usize, } #[pymethods] impl BaseClass { #[new] fn new() -> Self { BaseClass { val1: 10 } } pub fn method(&self) -> usize { self.val1 } } #[pyclass(extends=BaseClass)] struct SubClass {} #[pymethods] impl SubClass { #[new] fn new() -> (Self, BaseClass) { (SubClass {}, BaseClass::new()) } fn method(self_: &PyCell) -> PyResult<&PyAny> { let super_ = self_.py_super()?; super_.call_method("method", (), None) } } #[test] fn test_call_super_method() { Python::with_gil(|py| { let cls = py.get_type::(); pyo3::py_run!( py, cls, r#" obj = cls() assert obj.method() == 10 "# ) }); } pyo3-0.20.2/tests/test_text_signature.rs000064400000000000000000000340541046102023000164210ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::types::{PyDict, PyTuple}; use pyo3::{types::PyType, wrap_pymodule, PyCell}; #[path = "../src/tests/common.rs"] mod common; #[test] fn class_without_docs_or_signature() { #[pyclass] struct MyClass {} Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!(py, typeobj, "typeobj.__doc__ is None"); py_assert!(py, typeobj, "typeobj.__text_signature__ is None"); }); } #[test] fn class_with_docs() { /// docs line1 #[pyclass] /// docs line2 struct MyClass {} Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!(py, typeobj, "typeobj.__doc__ == 'docs line1\\ndocs line2'"); py_assert!(py, typeobj, "typeobj.__text_signature__ is None"); }); } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_10)), ignore)] fn class_with_signature_no_doc() { #[pyclass] struct MyClass {} #[pymethods] impl MyClass { #[new] #[pyo3(signature = (a, b=None, *, c=42), text_signature = "(a, b=None, *, c=42)")] fn __new__(a: i32, b: Option, c: i32) -> Self { let _ = (a, b, c); Self {} } } Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!(py, typeobj, "typeobj.__doc__ == ''"); py_assert!( py, typeobj, "typeobj.__text_signature__ == '(a, b=None, *, c=42)'" ); }); } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_10)), ignore)] fn class_with_docs_and_signature() { /// docs line1 #[pyclass] /// docs line2 struct MyClass {} #[pymethods] impl MyClass { #[new] #[pyo3(signature = (a, b=None, *, c=42), text_signature = "(a, b=None, *, c=42)")] fn __new__(a: i32, b: Option, c: i32) -> Self { let _ = (a, b, c); Self {} } } Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!(py, typeobj, "typeobj.__doc__ == 'docs line1\\ndocs line2'"); py_assert!( py, typeobj, "typeobj.__text_signature__ == '(a, b=None, *, c=42)'" ); }); } #[test] fn test_function() { #[pyfunction(signature = (a, b=None, *, c=42))] #[pyo3(text_signature = "(a, b=None, *, c=42)")] fn my_function(a: i32, b: Option, c: i32) { let _ = (a, b, c); } Python::with_gil(|py| { let f = wrap_pyfunction!(my_function)(py).unwrap(); py_assert!(py, f, "f.__text_signature__ == '(a, b=None, *, c=42)'"); }); } #[test] fn test_auto_test_signature_function() { #[pyfunction] fn my_function(a: i32, b: i32, c: i32) { let _ = (a, b, c); } #[pyfunction(pass_module)] fn my_function_2(module: &PyModule, a: i32, b: i32, c: i32) { let _ = (module, a, b, c); } #[pyfunction(signature = (a, /, b = None, *, c = 5))] fn my_function_3(a: i32, b: Option, c: i32) { let _ = (a, b, c); } #[pyfunction(signature = (a, /, b = None, *args, c, d=5, **kwargs))] fn my_function_4( a: i32, b: Option, args: &PyTuple, c: i32, d: i32, kwargs: Option<&PyDict>, ) { let _ = (a, b, args, c, d, kwargs); } #[pyfunction(signature = (a = 1, /, b = None, c = 1.5, d=5, e = "pyo3", f = 'f', h = true))] fn my_function_5(a: i32, b: Option, c: f32, d: i32, e: &str, f: char, h: bool) { let _ = (a, b, c, d, e, f, h); } #[pyfunction] fn my_function_6(a: i32, b: Option, c: Option) { let _ = (a, b, c); } Python::with_gil(|py| { let f = wrap_pyfunction!(my_function)(py).unwrap(); py_assert!( py, f, "f.__text_signature__ == '(a, b, c)', f.__text_signature__" ); let f = wrap_pyfunction!(my_function_2)(py).unwrap(); py_assert!( py, f, "f.__text_signature__ == '($module, a, b, c)', f.__text_signature__" ); let f = wrap_pyfunction!(my_function_3)(py).unwrap(); py_assert!( py, f, "f.__text_signature__ == '(a, /, b=None, *, c=5)', f.__text_signature__" ); let f = wrap_pyfunction!(my_function_4)(py).unwrap(); py_assert!( py, f, "f.__text_signature__ == '(a, /, b=None, *args, c, d=5, **kwargs)', f.__text_signature__" ); let f = wrap_pyfunction!(my_function_5)(py).unwrap(); py_assert!( py, f, "f.__text_signature__ == '(a=1, /, b=None, c=1.5, d=5, e=\"pyo3\", f=\\'f\\', h=True)', f.__text_signature__" ); let f = wrap_pyfunction!(my_function_6)(py).unwrap(); py_assert!( py, f, "f.__text_signature__ == '(a, b=None, c=None)', f.__text_signature__" ); }); } #[test] fn test_auto_test_signature_method() { #[pyclass] struct MyClass {} #[pymethods] impl MyClass { #[new] fn new(a: i32, b: i32, c: i32) -> Self { let _ = (a, b, c); Self {} } fn method(&self, a: i32, b: i32, c: i32) { let _ = (a, b, c); } #[pyo3(signature = (a, /, b = None, *, c = 5))] fn method_2(&self, a: i32, b: Option, c: i32) { let _ = (a, b, c); } #[pyo3(signature = (a, /, b = None, *args, c, d=5, **kwargs))] fn method_3( &self, a: i32, b: Option, args: &PyTuple, c: i32, d: i32, kwargs: Option<&PyDict>, ) { let _ = (a, b, args, c, d, kwargs); } #[staticmethod] fn staticmethod(a: i32, b: i32, c: i32) { let _ = (a, b, c); } #[classmethod] fn classmethod(cls: &PyType, a: i32, b: i32, c: i32) { let _ = (cls, a, b, c); } } Python::with_gil(|py| { let cls = py.get_type::(); #[cfg(any(not(Py_LIMITED_API), Py_3_10))] py_assert!(py, cls, "cls.__text_signature__ == '(a, b, c)'"); py_assert!( py, cls, "cls.method.__text_signature__ == '($self, a, b, c)'" ); py_assert!( py, cls, "cls.method_2.__text_signature__ == '($self, a, /, b=None, *, c=5)'" ); py_assert!( py, cls, "cls.method_3.__text_signature__ == '($self, a, /, b=None, *args, c, d=5, **kwargs)'" ); py_assert!( py, cls, "cls.staticmethod.__text_signature__ == '(a, b, c)'" ); py_assert!( py, cls, "cls.classmethod.__text_signature__ == '($cls, a, b, c)'" ); }); } #[test] fn test_auto_test_signature_opt_out() { #[pyfunction(text_signature = None)] fn my_function(a: i32, b: i32, c: i32) { let _ = (a, b, c); } #[pyfunction(signature = (a, /, b = None, *, c = 5), text_signature = None)] fn my_function_2(a: i32, b: Option, c: i32) { let _ = (a, b, c); } #[pyclass] struct MyClass {} #[pymethods] impl MyClass { #[new] #[pyo3(text_signature = None)] fn new(a: i32, b: i32, c: i32) -> Self { let _ = (a, b, c); Self {} } #[pyo3(text_signature = None)] fn method(&self, a: i32, b: i32, c: i32) { let _ = (a, b, c); } #[pyo3(signature = (a, /, b = None, *, c = 5), text_signature = None)] fn method_2(&self, a: i32, b: Option, c: i32) { let _ = (a, b, c); } #[staticmethod] #[pyo3(text_signature = None)] fn staticmethod(a: i32, b: i32, c: i32) { let _ = (a, b, c); } #[classmethod] #[pyo3(text_signature = None)] fn classmethod(cls: &PyType, a: i32, b: i32, c: i32) { let _ = (cls, a, b, c); } } Python::with_gil(|py| { let f = wrap_pyfunction!(my_function)(py).unwrap(); py_assert!(py, f, "f.__text_signature__ == None"); let f = wrap_pyfunction!(my_function_2)(py).unwrap(); py_assert!(py, f, "f.__text_signature__ == None"); let cls = py.get_type::(); py_assert!(py, cls, "cls.__text_signature__ == None"); py_assert!(py, cls, "cls.method.__text_signature__ == None"); py_assert!(py, cls, "cls.method_2.__text_signature__ == None"); py_assert!(py, cls, "cls.staticmethod.__text_signature__ == None"); py_assert!(py, cls, "cls.classmethod.__text_signature__ == None"); }); } #[test] fn test_pyfn() { #[pymodule] fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> { #[pyfn(m, signature = (a, b=None, *, c=42))] #[pyo3(text_signature = "(a, b=None, *, c=42)")] fn my_function(a: i32, b: Option, c: i32) { let _ = (a, b, c); } Ok(()) } Python::with_gil(|py| { let m = wrap_pymodule!(my_module)(py); py_assert!( py, m, "m.my_function.__text_signature__ == '(a, b=None, *, c=42)'" ); }); } #[test] fn test_methods() { #[pyclass] struct MyClass {} #[pymethods] impl MyClass { #[pyo3(text_signature = "($self, a)")] fn method(&self, a: i32) { let _ = a; } #[pyo3(text_signature = "($self, b)")] fn pyself_method(_this: &PyCell, b: i32) { let _ = b; } #[classmethod] #[pyo3(text_signature = "($cls, c)")] fn class_method(_cls: &PyType, c: i32) { let _ = c; } #[staticmethod] #[pyo3(text_signature = "(d)")] fn static_method(d: i32) { let _ = d; } } Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!( py, typeobj, "typeobj.method.__text_signature__ == '($self, a)'" ); py_assert!( py, typeobj, "typeobj.pyself_method.__text_signature__ == '($self, b)'" ); py_assert!( py, typeobj, "typeobj.class_method.__text_signature__ == '($cls, c)'" ); py_assert!( py, typeobj, "typeobj.static_method.__text_signature__ == '(d)'" ); }); } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_10)), ignore)] fn test_raw_identifiers() { #[pyclass] struct r#MyClass {} #[pymethods] impl MyClass { #[new] fn new() -> MyClass { MyClass {} } fn r#method(&self) {} } Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!(py, typeobj, "typeobj.__text_signature__ == '()'"); py_assert!( py, typeobj, "typeobj.method.__text_signature__ == '($self)'" ); }); } #[allow(deprecated)] mod deprecated { use crate::py_assert; use pyo3::prelude::*; #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_10)), ignore)] fn class_with_docs_and_signature() { /// docs line1 #[pyclass] /// docs line2 #[pyo3(text_signature = "(a, b=None, *, c=42)")] /// docs line3 struct MyClass {} #[pymethods] impl MyClass { #[new] #[pyo3(signature = (a, b=None, *, c=42))] fn __new__(a: i32, b: Option, c: i32) -> Self { let _ = (a, b, c); Self {} } } Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!( py, typeobj, "typeobj.__doc__ == 'docs line1\\ndocs line2\\ndocs line3'" ); py_assert!( py, typeobj, "typeobj.__text_signature__ == '(a, b=None, *, c=42)'" ); }); } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_10)), ignore)] fn class_with_deprecated_text_signature() { #[pyclass] #[pyo3(text_signature = "(a, b=None, *, c=42)")] struct MyClass {} #[pymethods] impl MyClass { #[new] #[pyo3(signature = (a, b=None, *, c=42))] fn __new__(a: i32, b: Option, c: i32) -> Self { let _ = (a, b, c); Self {} } } Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!( py, typeobj, "typeobj.__doc__ is None or typeobj.__doc__ == ''" ); py_assert!( py, typeobj, "typeobj.__text_signature__ == '(a, b=None, *, c=42)'" ); }); } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_10)), ignore)] fn class_with_deprecated_text_signature_and_on_new() { #[pyclass(text_signature = "(a, b=None, *, c=42)")] struct MyClass {} #[pymethods] impl MyClass { #[new] #[pyo3(signature = (a, b=None, *, c=42), text_signature = "(NOT, THIS, ONE)")] fn __new__(a: i32, b: Option, c: i32) -> Self { let _ = (a, b, c); Self {} } } Python::with_gil(|py| { let typeobj = py.get_type::(); py_assert!( py, typeobj, "typeobj.__doc__ is None or typeobj.__doc__ == ''" ); // Deprecated `#[pyclass(text_signature)]` attribute will be preferred // for backwards-compatibility. py_assert!( py, typeobj, "typeobj.__text_signature__ == '(a, b=None, *, c=42)'" ); }); } } pyo3-0.20.2/tests/test_unsendable_dict.rs000064400000000000000000000021401046102023000164660ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::py_run; #[pyclass(dict, unsendable)] struct UnsendableDictClass {} #[pymethods] impl UnsendableDictClass { #[new] fn new() -> Self { UnsendableDictClass {} } } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_10)), ignore)] fn test_unsendable_dict() { Python::with_gil(|py| { let inst = Py::new(py, UnsendableDictClass {}).unwrap(); py_run!(py, inst, "assert inst.__dict__ == {}"); }); } #[pyclass(dict, unsendable, weakref)] struct UnsendableDictClassWithWeakRef {} #[pymethods] impl UnsendableDictClassWithWeakRef { #[new] fn new() -> Self { Self {} } } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_10)), ignore)] fn test_unsendable_dict_with_weakref() { Python::with_gil(|py| { let inst = Py::new(py, UnsendableDictClassWithWeakRef {}).unwrap(); py_run!(py, inst, "assert inst.__dict__ == {}"); py_run!( py, inst, "import weakref; assert weakref.ref(inst)() is inst; inst.a = 1; assert inst.a == 1" ); }); } pyo3-0.20.2/tests/test_variable_arguments.rs000064400000000000000000000022161046102023000172210ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::types::{PyDict, PyTuple}; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct MyClass {} #[pymethods] impl MyClass { #[staticmethod] #[pyo3(signature = (*args))] fn test_args(args: &PyTuple) -> &PyTuple { args } #[staticmethod] #[pyo3(signature = (**kwargs))] fn test_kwargs(kwargs: Option<&PyDict>) -> Option<&PyDict> { kwargs } } #[test] fn variable_args() { Python::with_gil(|py| { let my_obj = py.get_type::(); py_assert!(py, my_obj, "my_obj.test_args() == ()"); py_assert!(py, my_obj, "my_obj.test_args(1) == (1,)"); py_assert!(py, my_obj, "my_obj.test_args(1, 2) == (1, 2)"); }); } #[test] fn variable_kwargs() { Python::with_gil(|py| { let my_obj = py.get_type::(); py_assert!(py, my_obj, "my_obj.test_kwargs() == None"); py_assert!(py, my_obj, "my_obj.test_kwargs(test=1) == {'test': 1}"); py_assert!( py, my_obj, "my_obj.test_kwargs(test1=1, test2=2) == {'test1':1, 'test2':2}" ); }); } pyo3-0.20.2/tests/test_various.rs000064400000000000000000000117361046102023000150460ustar 00000000000000#![cfg(feature = "macros")] use pyo3::prelude::*; use pyo3::types::{PyDict, PyTuple}; use pyo3::{py_run, PyCell}; use std::fmt; #[path = "../src/tests/common.rs"] mod common; #[pyclass] struct MutRefArg { n: i32, } #[pymethods] impl MutRefArg { fn get(&self) -> i32 { self.n } fn set_other(&self, mut other: PyRefMut<'_, MutRefArg>) { other.n = 100; } } #[test] fn mut_ref_arg() { Python::with_gil(|py| { let inst1 = Py::new(py, MutRefArg { n: 0 }).unwrap(); let inst2 = Py::new(py, MutRefArg { n: 0 }).unwrap(); py_run!(py, inst1 inst2, "inst1.set_other(inst2)"); let inst2 = inst2.as_ref(py).borrow(); assert_eq!(inst2.n, 100); }); } #[pyclass] struct PyUsize { #[pyo3(get)] pub value: usize, } #[pyfunction] fn get_zero() -> PyUsize { PyUsize { value: 0 } } #[test] /// Checks that we can use return a custom class in arbitrary function and use those functions /// both in rust and python fn return_custom_class() { Python::with_gil(|py| { // Using from rust assert_eq!(get_zero().value, 0); // Using from python let get_zero = wrap_pyfunction!(get_zero)(py).unwrap(); py_assert!(py, get_zero, "get_zero().value == 0"); }); } #[test] fn intopytuple_primitive() { Python::with_gil(|py| { let tup = (1, 2, "foo"); py_assert!(py, tup, "tup == (1, 2, 'foo')"); py_assert!(py, tup, "tup[0] == 1"); py_assert!(py, tup, "tup[1] == 2"); py_assert!(py, tup, "tup[2] == 'foo'"); }); } #[pyclass] struct SimplePyClass {} #[test] fn intopytuple_pyclass() { Python::with_gil(|py| { let tup = ( PyCell::new(py, SimplePyClass {}).unwrap(), PyCell::new(py, SimplePyClass {}).unwrap(), ); py_assert!(py, tup, "type(tup[0]).__name__ == 'SimplePyClass'"); py_assert!(py, tup, "type(tup[0]).__name__ == type(tup[1]).__name__"); py_assert!(py, tup, "tup[0] != tup[1]"); }); } #[test] fn pytuple_primitive_iter() { Python::with_gil(|py| { let tup = PyTuple::new(py, [1u32, 2, 3].iter()); py_assert!(py, tup, "tup == (1, 2, 3)"); }); } #[test] fn pytuple_pyclass_iter() { Python::with_gil(|py| { let tup = PyTuple::new( py, [ PyCell::new(py, SimplePyClass {}).unwrap(), PyCell::new(py, SimplePyClass {}).unwrap(), ] .iter(), ); py_assert!(py, tup, "type(tup[0]).__name__ == 'SimplePyClass'"); py_assert!(py, tup, "type(tup[0]).__name__ == type(tup[0]).__name__"); py_assert!(py, tup, "tup[0] != tup[1]"); }); } #[pyclass(dict, module = "test_module")] struct PickleSupport {} #[pymethods] impl PickleSupport { #[new] fn new() -> PickleSupport { PickleSupport {} } pub fn __reduce__<'py>( slf: &'py PyCell, py: Python<'py>, ) -> PyResult<(PyObject, &'py PyTuple, PyObject)> { let cls = slf.to_object(py).getattr(py, "__class__")?; let dict = slf.to_object(py).getattr(py, "__dict__")?; Ok((cls, PyTuple::empty(py), dict)) } } fn add_module(py: Python<'_>, module: &PyModule) -> PyResult<()> { py.import("sys")? .dict() .get_item("modules") .unwrap() .unwrap() .downcast::()? .set_item(module.name()?, module) } #[test] #[cfg_attr(all(Py_LIMITED_API, not(Py_3_10)), ignore)] fn test_pickle() { Python::with_gil(|py| { let module = PyModule::new(py, "test_module").unwrap(); module.add_class::().unwrap(); add_module(py, module).unwrap(); let inst = PyCell::new(py, PickleSupport {}).unwrap(); py_run!( py, inst, r#" inst.a = 1 assert inst.__dict__ == {'a': 1} import pickle inst2 = pickle.loads(pickle.dumps(inst)) assert inst2.__dict__ == {'a': 1} "# ); }); } /// Testing https://github.com/PyO3/pyo3/issues/1106. A result type that /// implements `From for PyErr` should be automatically converted /// when using `#[pyfunction]`. /// /// This only makes sure that valid `Result` types do work. For an invalid /// enum type, see `ui/invalid_result_conversion.py`. #[derive(Debug)] struct MyError { pub descr: &'static str, } impl fmt::Display for MyError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "My error message: {}", self.descr) } } /// Important for the automatic conversion to `PyErr`. impl From for PyErr { fn from(err: MyError) -> pyo3::PyErr { pyo3::exceptions::PyOSError::new_err(err.to_string()) } } #[pyfunction] fn result_conversion_function() -> Result<(), MyError> { Err(MyError { descr: "something went wrong", }) } #[test] fn test_result_conversion() { Python::with_gil(|py| { wrap_pyfunction!(result_conversion_function)(py).unwrap(); }); } pyo3-0.20.2/tests/test_wrap_pyfunction_deduction.rs000064400000000000000000000004321046102023000206320ustar 00000000000000#![cfg(feature = "macros")] use pyo3::{prelude::*, types::PyCFunction}; #[pyfunction] fn f() {} pub fn add_wrapped(wrapper: &impl Fn(Python<'_>) -> PyResult<&PyCFunction>) { let _ = wrapper; } #[test] fn wrap_pyfunction_deduction() { add_wrapped(wrap_pyfunction!(f)); }