Developing and Building
This section won't go into the actual coding of your core code in rust - see the excursions for that. Assuming that you have your core code written (in rust):
Wrapping with pyo3
The relevant section of the pyo3 book does a great job of explaining how to wrap your code, so I'll just touch the highlights here:
rust/fizzbuzzo3/Cargo.toml
- Name your library the way you want the module to appear in python. For example to import using
from fizzbuzz import fizzbuzzo3
- Use the
cdylib
library type - Add a dependency to
pyo3
Add the following to ./rust/fizzbuzzo3/Cargo.toml
...
[lib]
name = "fizzbuzzo3"
path = "src/lib.rs"
crate-type = ["cdylib"] # cdylib required for python import, rlib required for rust tests.
[dependencies]
pyo3 = { git = "https://github.com/MusicalNinjaDad/pyo3.git", branch = "pyo3-testing" }
...
Note: for now this uses a git dependency to a branch on my fork - until either PR pyo3/#4099 lands or I pull the testing support out into an independent crate
Use the same name for the library and exported module
I have not spent much time trying but I couldn't get the import to work if you have different names for the library and imported module. Trying to rename the library to fizzbuzzo3lib
leads to a file like python/fizzbuzz/fizzbuzzo3lib.cpython-312-x86_64-linux-gnu.so
being generated but unusable:
>>> from fizzbuzz import fizzbuzzo3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name 'fizzbuzzo3' from 'fizzbuzz' (/workspaces/FizzBuzz/python/fizzbuzz/__init__.py)
>>> from fizzbuzz import fizzbuzzo3lib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dynamic module does not define module export function (PyInit_fizzbuzzo3lib)
rust/fizzbuzzo3/Cargo.toml
- full source
[package]
name = "fizzbuzzo3"
version = "2.1.0" # Tracks python version
edition = "2021"
[lib]
name = "fizzbuzzo3"
path = "src/lib.rs"
crate-type = ["cdylib"] # cdylib required for python import, rlib required for rust tests.
[dependencies]
fizzbuzz = { path = "../fizzbuzz" }
pyo3 = "0.21.2"
rayon = "1.10.0"
[dev-dependencies]
pyo3-testing = "0.3.4"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
Adding the wrapped module to your project
I chose to use setuptools
& setuptools-rust
as my build backend. Pyo3 offer two backends setuptools-rust
and maturin
. I preferred to try the first because:
- I am already used to using
setuptools
and didn't want to change out a working system for something else - I found
setuptools-rust
to be very easy to use - The docs point out that it offers more flexibility and fits better to a use case where you may also have independent python code
Add the following to ./pyproject.toml
...
[build-system]
requires = ["setuptools", "setuptools-rust"]
build-backend = "setuptools.build_meta"
...
[[tool.setuptools-rust.ext-modules]]
# The last part of the name (e.g. "_lib") has to match lib.name in Cargo.toml,
# but you can add a prefix to nest it inside of a Python package.
target = "fizzbuzz.fizzbuzzo3"
path = "rust/fizzbuzzo3/Cargo.toml"
binding = "PyO3"
features = ["pyo3/extension-module"] # IMPORTANT!!!
debug = false # Adds `--release` to `pip install -e .` builds, necessary for performance testing
...
Avoid errors packaging for linux
It is important to specify features = ["pyo3/extension-module"]
in ./pyproject.toml
to avoid linking the python interpreter into your library and failing quality checks when trying to package for linux.
Background is available by combining the pyo3 FAQ and manylinux specification
Missing out on 4-7x performance gains
For performance benchmarking or faster tests, add debug = false
to ./pyproject.toml
. This ensures your Rust code is built with the right optimisations even when installing in editable mode via pip install -e .
. Without this, only install
and wheel
builds will be optimised.
The default release profile is well documented in The Cargo Book. I found a 4-7x performance boost when I enabled this!
./pyproject.toml
- full source
[project]
name = "fizzbuzz"
description = "An implementation of the traditional fizzbuzz kata"
# readme = "README.md"
license = { text = "MIT" }
authors = [{ name = "Mike Foster" }]
dynamic = ["version"]
requires-python = ">=3.8"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Topic :: Software Development :: Libraries :: Python Modules",
"Intended Audience :: Developers",
"Natural Language :: English",
]
[project.urls]
# Homepage = ""
# Documentation = ""
Repository = "https://github.com/MusicalNinjaDad/fizzbuzz"
Issues = "https://github.com/MusicalNinjaDad/fizzbuzz/issues"
# Changelog = ""
[tool.setuptools.dynamic]
version = { file = "__pyversion__" }
# ===========================
# Build, package
# ===========================
[build-system]
requires = ["setuptools", "setuptools-rust"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["python"]
[[tool.setuptools-rust.ext-modules]]
# The last part of the name (e.g. "_lib") has to match lib.name in Cargo.toml,
# but you can add a prefix to nest it inside of a Python package.
target = "fizzbuzz.fizzbuzzo3"
path = "rust/fizzbuzzo3/Cargo.toml"
binding = "PyO3"
features = ["pyo3/extension-module"] # IMPORTANT!!!
debug = false # Adds `--release` to `pip install -e .` builds, necessary for performance testing
[tool.cibuildwheel]
skip = [
"*-musllinux_i686", # No musllinux-i686 rust compiler available.
"*-musllinux_ppc64le", # no rust target available.
"*-musllinux_s390x", # no rust target available.
]
# Use prebuilt copies of pypa images with rust and just ready compiled (saves up to 25 mins on emulated archs)
manylinux-x86_64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_x86_64-rust"
manylinux-i686-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_i686-rust"
manylinux-aarch64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_aarch64-rust"
manylinux-ppc64le-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_ppc64le-rust"
manylinux-s390x-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_s390x-rust"
manylinux-pypy_x86_64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_x86_64-rust"
manylinux-pypy_i686-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_i686-rust"
manylinux-pypy_aarch64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_aarch64-rust"
musllinux-x86_64-image = "ghcr.io/musicalninjas/quay.io/pypa/musllinux_1_2_x86_64-rust"
musllinux-aarch64-image = "ghcr.io/musicalninjas/quay.io/pypa/musllinux_1_2_aarch64-rust"
# `test-command` will FAIL if only passed `pytest {package}`
# as this tries to resolve imports based on source files (where there is no `.so` for rust libraries), not installed wheel
test-command = "pytest {package}/tests"
test-extras = "test"
[tool.cibuildwheel.linux]
before-all = "just clean"
archs = "all"
[tool.cibuildwheel.macos]
before-all = "curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal && cargo install just && just clean"
[tool.cibuildwheel.windows]
before-all = "rustup target add aarch64-pc-windows-msvc i586-pc-windows-msvc i686-pc-windows-msvc x86_64-pc-windows-msvc && cargo install just && just clean"
before-build = "pip install delvewheel"
repair-wheel-command = "delvewheel repair -w {dest_dir} {wheel}"
# ===========================
# Test, lint, documentation
# ===========================
[project.optional-dependencies]
stubs = ["pyo3-stubgen"]
lint = ["ruff"]
test = ["pytest", "pytest-doctest-mkdocstrings"]
cov = ["fizzbuzz[test]", "pytest-cov"]
doc = ["black", "mkdocs", "mkdocstrings[python]", "mkdocs-material", "fizzbuzz[stubs]"]
dev = ["fizzbuzz[stubs,lint,test,cov,doc]"]
[tool.pytest.ini_options]
xfail_strict = true
addopts = [
"--doctest-modules",
"--doctest-mdcodeblocks",
"--doctest-glob=*.pyi",
"--doctest-glob=*.md",
]
testpaths = ["tests", "python"]
[tool.coverage.run]
branch = true
omit = ["tests/*"]
dynamic_context = "test_function"
[tool.ruff]
line-length = 120
format.skip-magic-trailing-comma = false
format.quote-style = "double"
[tool.ruff.lint]
select = ["ALL"]
flake8-pytest-style.fixture-parentheses = false
flake8-annotations.mypy-init-return = true
extend-ignore = [
"E701", # One-line ifs are not great, one-liners with suppression are worse
"ANN101", # Type annotation for `self`
"ANN202", # Return type annotation for private functions
"ANN401", # Using Any often enough that supressing individually is pointless
"W291", # Double space at EOL is linebreak in md-docstring
"W292", # Too much typing to add newline at end of each file
"W293", # Whitespace only lines are OK for us
"D401", # First line of docstring should be in imperative mood ("Returns ..." is OK, for example)
"D203", # 1 blank line required before class docstring (No thank you, only after the class docstring)
"D212", # Multi-line docstring summary should start at the first line (We start on new line, D209)
"D215", # Section underline is over-indented (No underlines)
"D400", # First line should end with a period (We allow any punctuation, D415)
"D406", # Section name should end with a newline (We use a colon, D416)
"D407", # Missing dashed underline after section (No underlines)
"D408", # Section underline should be in the line following the section's name (No underlines)
"D409", # Section underline should match the length of its name (No underlines)
"D413", # Missing blank line after last section (Not required)
]
[tool.ruff.lint.per-file-ignores]
# Additional ignores for tests
"**/test_*.py" = [
"INP001", # Missing __init__.py
"ANN", # Missing type annotations
"S101", # Use of `assert`
"PLR2004", # Magic number comparisons are OK in tests
"D1", # Don't REQUIRE docstrings for tests - but they are nice
]
"**/__init__.py" = [
"D104", # Don't require module docstring in __init__.py
"F401", # Unused imports are fine: using __init__.py to expose them with implicit __ALL__
]
Python virtual environment & build
I like to keep things as simple as possible. Python has many virtual environment managers, venv
is part of the core library and does everything we need while leaving us in control of the entire build and integration process.
Quick start with justfile
The justfile ./justfile
handles all of this for you. Feel free to copy it.
./justfile
- full source
# list available recipes
list:
@just --list --justfile {{justfile()}}
# remove pre-built rust and python libraries (excluding .venv)
clean:
- cargo clean
rm -rf .pytest_cache
rm -rf build
rm -rf dist
rm -rf wheelhouse
rm -rf .ruff_cache
find . -depth -type d -not -path "./.venv/*" -name "__pycache__" -exec rm -rf "{}" \;
find . -depth -type d -path "*.egg-info" -exec rm -rf "{}" \;
find . -type f -name "*.egg" -delete
find . -type f -name "*.so" -delete
# clean out coverage files
clean-cov:
find . -type f -name "*.profraw" -delete
rm -rf pycov
rm -rf rustcov
# clean, remove existing .venv and rebuild the venv with pip install -e .[dev]
reset: clean clean-cov
rm -rf .venv
python -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/pip install -e .[dev]
# lint rust files with fmt & clippy
lint-rust:
- cargo fmt --check
- cargo clippy --workspace
# test rust workspace
test-rust:
- cargo test --quiet --workspace
# lint and test rust
check-rust: lint-rust test-rust
# lint python with ruff
lint-python:
- .venv/bin/ruff check .
# test python
test-python:
- .venv/bin/pytest
# lint and test python
check-python: lint-python test-python
# lint and test both rust and python
check: check-rust check-python
#run coverage analysis on rust & python code
cov:
RUSTFLAGS="-Cinstrument-coverage" cargo build
RUSTFLAGS="-Cinstrument-coverage" LLVM_PROFILE_FILE="tests-%p-%m.profraw" cargo test
-mkdir rustcov
grcov . -s . --binary-path ./target/debug/ -t html,lcov --branch --ignore-not-existing --excl-line GRCOV_EXCL_LINE --output-path ./rustcov
pytest --cov --cov-report html:pycov --cov-report term
# serve rust coverage results on localhost:8002 (doesn't run coverage analysis)
rust-cov:
python -m http.server -d ./rustcov/html 8002
# serve python coverage results on localhost:8003 (doesn't run coverage analysis)
py-cov:
python -m http.server -d ./pycov 8003
# serve python docs on localhost:8000
py-docs:
mkdocs serve
#build and serve rust API docs on localhost:8001
rust-docs:
cargo doc
python -m http.server -d target/doc 8001
# build and test a wheel (a suitable venv must already by active!)
test-wheel: clean
cibuildwheel --only cp312-manylinux_x86_64
Creating a virtual environment with venv
If you are unfamiliar with venv here are the docs
Depending on your distro you may need to install venv as a separate package
Creating a virtual environment is as simple as:
/projectroot$ python -m venv .venv
Sourcing development dependencies from ./pyproject.toml
To provide a single line python build for local development you will need to source your development dependencies from ./pyproject.toml
. These can be split into multiple groups to give more control during automated processes where you don't need everything.
Add the following to ./pyproject.toml
...
[project.optional-dependencies]
lint = ["ruff"]
test = ["pytest", "pytest-doctest-mkdocstrings"]
cov = ["fizzbuzz[test]", "pytest-cov"]
doc = ["black", "mkdocs", "mkdocstrings[python]", "mkdocs-material"]
dev = ["fizzbuzz[lint,test,cov,doc]"]
...
./pyproject.toml
- full source
[project]
name = "fizzbuzz"
description = "An implementation of the traditional fizzbuzz kata"
# readme = "README.md"
license = { text = "MIT" }
authors = [{ name = "Mike Foster" }]
dynamic = ["version"]
requires-python = ">=3.8"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Topic :: Software Development :: Libraries :: Python Modules",
"Intended Audience :: Developers",
"Natural Language :: English",
]
[project.urls]
# Homepage = ""
# Documentation = ""
Repository = "https://github.com/MusicalNinjaDad/fizzbuzz"
Issues = "https://github.com/MusicalNinjaDad/fizzbuzz/issues"
# Changelog = ""
[tool.setuptools.dynamic]
version = { file = "__pyversion__" }
# ===========================
# Build, package
# ===========================
[build-system]
requires = ["setuptools", "setuptools-rust"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["python"]
[[tool.setuptools-rust.ext-modules]]
# The last part of the name (e.g. "_lib") has to match lib.name in Cargo.toml,
# but you can add a prefix to nest it inside of a Python package.
target = "fizzbuzz.fizzbuzzo3"
path = "rust/fizzbuzzo3/Cargo.toml"
binding = "PyO3"
features = ["pyo3/extension-module"] # IMPORTANT!!!
debug = false # Adds `--release` to `pip install -e .` builds, necessary for performance testing
[tool.cibuildwheel]
skip = [
"*-musllinux_i686", # No musllinux-i686 rust compiler available.
"*-musllinux_ppc64le", # no rust target available.
"*-musllinux_s390x", # no rust target available.
]
# Use prebuilt copies of pypa images with rust and just ready compiled (saves up to 25 mins on emulated archs)
manylinux-x86_64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_x86_64-rust"
manylinux-i686-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_i686-rust"
manylinux-aarch64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_aarch64-rust"
manylinux-ppc64le-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_ppc64le-rust"
manylinux-s390x-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_s390x-rust"
manylinux-pypy_x86_64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_x86_64-rust"
manylinux-pypy_i686-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_i686-rust"
manylinux-pypy_aarch64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_aarch64-rust"
musllinux-x86_64-image = "ghcr.io/musicalninjas/quay.io/pypa/musllinux_1_2_x86_64-rust"
musllinux-aarch64-image = "ghcr.io/musicalninjas/quay.io/pypa/musllinux_1_2_aarch64-rust"
# `test-command` will FAIL if only passed `pytest {package}`
# as this tries to resolve imports based on source files (where there is no `.so` for rust libraries), not installed wheel
test-command = "pytest {package}/tests"
test-extras = "test"
[tool.cibuildwheel.linux]
before-all = "just clean"
archs = "all"
[tool.cibuildwheel.macos]
before-all = "curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal && cargo install just && just clean"
[tool.cibuildwheel.windows]
before-all = "rustup target add aarch64-pc-windows-msvc i586-pc-windows-msvc i686-pc-windows-msvc x86_64-pc-windows-msvc && cargo install just && just clean"
before-build = "pip install delvewheel"
repair-wheel-command = "delvewheel repair -w {dest_dir} {wheel}"
# ===========================
# Test, lint, documentation
# ===========================
[project.optional-dependencies]
stubs = ["pyo3-stubgen"]
lint = ["ruff"]
test = ["pytest", "pytest-doctest-mkdocstrings"]
cov = ["fizzbuzz[test]", "pytest-cov"]
doc = ["black", "mkdocs", "mkdocstrings[python]", "mkdocs-material", "fizzbuzz[stubs]"]
dev = ["fizzbuzz[stubs,lint,test,cov,doc]"]
[tool.pytest.ini_options]
xfail_strict = true
addopts = [
"--doctest-modules",
"--doctest-mdcodeblocks",
"--doctest-glob=*.pyi",
"--doctest-glob=*.md",
]
testpaths = ["tests", "python"]
[tool.coverage.run]
branch = true
omit = ["tests/*"]
dynamic_context = "test_function"
[tool.ruff]
line-length = 120
format.skip-magic-trailing-comma = false
format.quote-style = "double"
[tool.ruff.lint]
select = ["ALL"]
flake8-pytest-style.fixture-parentheses = false
flake8-annotations.mypy-init-return = true
extend-ignore = [
"E701", # One-line ifs are not great, one-liners with suppression are worse
"ANN101", # Type annotation for `self`
"ANN202", # Return type annotation for private functions
"ANN401", # Using Any often enough that supressing individually is pointless
"W291", # Double space at EOL is linebreak in md-docstring
"W292", # Too much typing to add newline at end of each file
"W293", # Whitespace only lines are OK for us
"D401", # First line of docstring should be in imperative mood ("Returns ..." is OK, for example)
"D203", # 1 blank line required before class docstring (No thank you, only after the class docstring)
"D212", # Multi-line docstring summary should start at the first line (We start on new line, D209)
"D215", # Section underline is over-indented (No underlines)
"D400", # First line should end with a period (We allow any punctuation, D415)
"D406", # Section name should end with a newline (We use a colon, D416)
"D407", # Missing dashed underline after section (No underlines)
"D408", # Section underline should be in the line following the section's name (No underlines)
"D409", # Section underline should match the length of its name (No underlines)
"D413", # Missing blank line after last section (Not required)
]
[tool.ruff.lint.per-file-ignores]
# Additional ignores for tests
"**/test_*.py" = [
"INP001", # Missing __init__.py
"ANN", # Missing type annotations
"S101", # Use of `assert`
"PLR2004", # Magic number comparisons are OK in tests
"D1", # Don't REQUIRE docstrings for tests - but they are nice
]
"**/__init__.py" = [
"D104", # Don't require module docstring in __init__.py
"F401", # Unused imports are fine: using __init__.py to expose them with implicit __ALL__
]
Building and installing the wrapped rust code for use in python development
Before you can use the wrapped rust code you need to build the equivalent of python/fizzbuzz/fizzbuzzo3.cpython-312-x86_64-linux-gnu.so
:
- Make sure your virtual environment is active. If not run
. .venv/bin/activate
(note the leading dot, which is easier than typingsource
all the time) - Then simply use
pip
to create an editable installation of your codebase:(.venv)/projectroot$ pip -e.[dev]
cleaning
Cleaning up old build artefacts
As with any built language it is a good idea to clean up old build artefacts before generating new ones, or at least before finalising a change. Cargo offers a simple cargo clean
for this, but you will also have the python library and various python caches in place which can sometimes cause problems.
To clean both languages:
(.venv)/projectroot$ cargo clean || true
(.venv)/projectroot$ rm -rf .pytest_cache
(.venv)/projectroot$ rm -rf build
(.venv)/projectroot$ rm -rf dist
(.venv)/projectroot$ rm -rf wheelhouse
(.venv)/projectroot$ rm -rf .ruff_cache
(.venv)/projectroot$ find . -depth -type d -not -path "./.venv/*" -name "__pycache__" -exec rm -rf "{}" \;
(.venv)/projectroot$ find . -depth -type d -path "*.egg-info" -exec rm -rf "{}" \;
(.venv)/projectroot$ find . -type f -name "*.egg" -delete
(.venv)/projectroot$ find . -type f -name "*.so" -delete
Or just use just clean
from the ./justfile
./justfile
- full source
# list available recipes
list:
@just --list --justfile {{justfile()}}
# remove pre-built rust and python libraries (excluding .venv)
clean:
- cargo clean
rm -rf .pytest_cache
rm -rf build
rm -rf dist
rm -rf wheelhouse
rm -rf .ruff_cache
find . -depth -type d -not -path "./.venv/*" -name "__pycache__" -exec rm -rf "{}" \;
find . -depth -type d -path "*.egg-info" -exec rm -rf "{}" \;
find . -type f -name "*.egg" -delete
find . -type f -name "*.so" -delete
# clean out coverage files
clean-cov:
find . -type f -name "*.profraw" -delete
rm -rf pycov
rm -rf rustcov
# clean, remove existing .venv and rebuild the venv with pip install -e .[dev]
reset: clean clean-cov
rm -rf .venv
python -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/pip install -e .[dev]
# lint rust files with fmt & clippy
lint-rust:
- cargo fmt --check
- cargo clippy --workspace
# test rust workspace
test-rust:
- cargo test --quiet --workspace
# lint and test rust
check-rust: lint-rust test-rust
# lint python with ruff
lint-python:
- .venv/bin/ruff check .
# test python
test-python:
- .venv/bin/pytest
# lint and test python
check-python: lint-python test-python
# lint and test both rust and python
check: check-rust check-python
#run coverage analysis on rust & python code
cov:
RUSTFLAGS="-Cinstrument-coverage" cargo build
RUSTFLAGS="-Cinstrument-coverage" LLVM_PROFILE_FILE="tests-%p-%m.profraw" cargo test
-mkdir rustcov
grcov . -s . --binary-path ./target/debug/ -t html,lcov --branch --ignore-not-existing --excl-line GRCOV_EXCL_LINE --output-path ./rustcov
pytest --cov --cov-report html:pycov --cov-report term
# serve rust coverage results on localhost:8002 (doesn't run coverage analysis)
rust-cov:
python -m http.server -d ./rustcov/html 8002
# serve python coverage results on localhost:8003 (doesn't run coverage analysis)
py-cov:
python -m http.server -d ./pycov 8003
# serve python docs on localhost:8000
py-docs:
mkdocs serve
#build and serve rust API docs on localhost:8001
rust-docs:
cargo doc
python -m http.server -d target/doc 8001
# build and test a wheel (a suitable venv must already by active!)
test-wheel: clean
cibuildwheel --only cp312-manylinux_x86_64
API design
There are a few things to consider when designing your API for python users.
Performance
Assuming part of the reason you are doing this is to provide a performance over native python, you will want to consider the (small but noticeable) performance cost each time you cross the python-rust boundary. Discussion pyo3/#4085 covers this topic, further improvements are promised for the next versions of pyo3.
Pass Container
s, don't make multiple calls
One simple way to avoid crossing the boundary often is to pass a list
or similar rather than making multiple individual calls. The performance difference can be seen below:
Rust: [1 calls of 10 runs fizzbuzzing up to 1_000_000]
[12.454665303001093]
Python: [1 calls of 10 runs fizzbuzzing up to 1_000_000]
[39.32552230800138]
Rust vector: [1 calls of 10 runs fizzbuzzing a list of numbers up to 1_000_000]
[6.319926773001498]
Use the rayon
crate to break the GIL and run parallel calculations
Adding parallel processing to a rust iterator is insanely simple. My impression of rust is "easy things are hard, hard things are easy!"
I simply added the following to my core rust code /rust/fizzbuzz/src/lib.rs
:
...
use rayon::prelude::*;
static BIG_VECTOR: usize = 300_000; // Size from which parallelisation makes sense
...
impl<Num> MultiFizzBuzz for Vec<Num>
where
Num: FizzBuzz + Sync,
{
fn fizzbuzz(&self) -> FizzBuzzAnswer {
if self.len() < BIG_VECTOR {
FizzBuzzAnswer::Many(self.iter().map(|n| n.fizzbuzz().into()).collect())
} else {
FizzBuzzAnswer::Many(self.par_iter().map(|n| n.fizzbuzz().into()).collect())
}
}
}
Check it makes sense
Adding parallel processing doesn't always make sense as it adds overhead ramping and managing a threadpool. You will want to do some benchmarking to find the sweet-spot. Benchmarking and performance testing is a topic for itself, so I'll add a dedicated section ...
Even more speed by passing a range (and implementing IntoPy and FromPyObject traits)
The world obviously needs the most performant fizzbuzz available! In an attempt to squeeze out even more speed I tried completely avoiding the need to build and pass a list
and instead (ab)used a python slice
to provide start
, stop
, and optional step
values. This gave another 1.5x speed boost. Surprisingly most of that comes from passing the list to rust, not creating it or processing it:
Timeit results
Rust: [3 calls of 10 runs fizzbuzzing up to 1000000]
[13.941677560000244, 12.671054376998654, 12.669853160998173]
Rust vector: [3 calls of 10 runs fizzbuzzing a list of numbers up to 1000000]
[5.104824486003054, 4.96210950999739, 4.903727466000419]
Rust vector, with python list overhead: [3 calls of 10 runs creating and fizzbuzzing a list of numbers up to 1000000]
[5.363066075999086, 5.316481181002018, 5.361383773997659]
Rust range: [3 calls of 10 runs fizzbuzzing a range of numbers up to 1000000]
[3.8294942710017494, 3.8227306799999496, 3.800879727001302]
Criterion bench results
multifizzbuzz_trait_from_vec_as_answer
time: [62.035 ms 63.960 ms 65.921 ms]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
multifizzbuzz_trait_from_range_as_answer
time: [60.295 ms 62.228 ms 64.228 ms]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Excursion to follow ...
This change was quite in depth, so expect an excursion later on the changes to the core rust/fizzbuzz/src/lib.rs
...
On the pyo3 side this involved the following in rust/fizzbuzzo3/src/lib.rs
:
- Creating a
struct MySlice
to hold the start, stop and step values which:- Can be created from a python
slice
:#[derive(FromPyObject)] struct MySlice { start: isize, stop: isize, step: Option<isize>, }
- Can be converted into a pyo3
PySlice
:Note: There is no rust standard type which pyo3 maps to a slice.impl IntoPy<Py<PyAny>> for MySlice { fn into_py(self, py: Python<'_>) -> Py<PyAny> { PySlice::new_bound(py, self.start, self.stop, self.step.unwrap_or(1)).into_py(py) } }
- Can be created from a python
- Parsing the slice to provide equivalent logic to python for negative steps:
fn py_fizzbuzz(num: FizzBuzzable) -> PyResult<String> { match num { ... FizzBuzzable::Slice(s) => match s.step { None => Ok((s.start..s.stop).fizzbuzz().into()), Some(1) => Ok((s.start..s.stop).fizzbuzz().into()), Some(step) => match step { 1.. => Ok((s.start..s.stop) .into_par_iter() .step_by(step.try_into().unwrap()) .fizzbuzz() .into()), // ```python // >>> foo[1:5:0] // Traceback (most recent call last): // File "<stdin>", line 1, in <module> // ValueError: slice step cannot be zero // ``` 0 => Err(PyValueError::new_err("step cannot be zero")), // ```python // >>> foo=[0,1,2,3,4,5,6] // >>> foo[6:0:-2] // [6, 4, 2] // ``` // Rust doesn't accept step < 0 or stop < start so need some trickery ..=-1 => Ok((s.start.neg()..s.stop.neg()) .into_par_iter() .step_by(step.neg().try_into().unwrap()) .map(|x| x.neg()) .fizzbuzz() .into()), }, }, ...
- Quite a bit of extra testing ...
rust/fizzbuzzo3/src/lib.rs
- full source
use std::{borrow::Cow, ops::Neg};
use fizzbuzz::{FizzBuzz, FizzBuzzAnswer, MultiFizzBuzz};
use pyo3::{exceptions::PyValueError, prelude::*, types::PySlice};
use rayon::iter::{IndexedParallelIterator, IntoParallelIterator, ParallelIterator};
#[derive(FromPyObject)]
enum FizzBuzzable {
Int(isize),
Float(f64),
Vec(Vec<isize>),
Slice(MySlice),
}
#[derive(FromPyObject)]
struct MySlice {
start: isize,
stop: isize,
step: Option<isize>,
}
impl IntoPy<Py<PyAny>> for MySlice {
fn into_py(self, py: Python<'_>) -> Py<PyAny> {
PySlice::new_bound(py, self.start, self.stop, self.step.unwrap_or(1)).into_py(py)
}
}
/// A wrapper struct for FizzBuzzAnswer to provide a custom implementation of `IntoPy`.
enum FizzBuzzReturn {
One(String),
Many(Vec<Cow<'static, str>>),
}
impl From<FizzBuzzAnswer> for FizzBuzzReturn {
fn from(value: FizzBuzzAnswer) -> Self {
FizzBuzzReturn::One(value.into())
}
}
impl IntoPy<Py<PyAny>> for FizzBuzzReturn {
fn into_py(self, py: Python<'_>) -> Py<PyAny> {
match self {
FizzBuzzReturn::One(answer) => answer.into_py(py),
FizzBuzzReturn::Many(answers) => answers.into_py(py),
}
}
}
/// Returns the correct fizzbuzz answer for any number or list/range of numbers.
///
/// This is an optimised algorithm compiled in rust. Large lists will utilise multiple CPU cores for processing.
/// Passing a slice, to represent a range, is fastest.
///
/// Arguments:
/// n: the number(s) to fizzbuzz
///
/// Returns:
/// In the case of a single number: a `str` with the correct fizzbuzz answer.
/// In the case of a list or range of inputs: a `list` of `str` with the correct fizzbuzz answers.
///
/// Examples:
/// a single `int`:
/// ```
/// >>> from fizzbuzz.fizzbuzzo3 import fizzbuzz
/// >>> fizzbuzz(1)
/// '1'
/// >>> fizzbuzz(3)
/// 'fizz'
/// ```
/// a `list`:
/// ```
/// from fizzbuzz.fizzbuzzo3 import fizzbuzz
/// >>> fizzbuzz([1,3])
/// ['1', 'fizz']
/// ```
/// a `slice` representing a range:
/// ```
/// from fizzbuzz.fizzbuzzo3 import fizzbuzz
/// >>> fizzbuzz(slice(1,4,2))
/// ['1', 'fizz']
/// >>> fizzbuzz(slice(1,4))
/// ['1', '2', 'fizz']
/// >>> fizzbuzz(slice(4,1,-1))
/// ['4', 'fizz', '2']
/// >>> fizzbuzz(slice(1,5,-1))
/// []
/// ```
/// Note: Slices are inclusive on the left, exclusive on the right and can contain an optional step.
/// Note: Slices are inclusive on the left, exclusive on the right and can contain an optional step.
/// Negative steps require start > stop, positive steps require stop > start; other combinations return `[]`.
/// A step of zero is invalid and will raise a `ValueError`.
#[pyfunction]
#[pyo3(name = "fizzbuzz", text_signature = "(n)")]
fn py_fizzbuzz(num: FizzBuzzable) -> PyResult<FizzBuzzReturn> {
match num {
FizzBuzzable::Int(n) => Ok(n.fizzbuzz().into()),
FizzBuzzable::Float(n) => Ok(n.fizzbuzz().into()),
FizzBuzzable::Vec(v) => Ok(FizzBuzzReturn::Many(v.fizzbuzz().collect())),
FizzBuzzable::Slice(s) => match s.step {
// Can only be tested from python: Cannot create a PySlice with no step in rust.
None => Ok(FizzBuzzReturn::Many((s.start..s.stop).fizzbuzz().collect())), // GRCOV_EXCL_LINE
Some(1) => Ok(FizzBuzzReturn::Many((s.start..s.stop).fizzbuzz().collect())),
Some(step) => match step {
1.. => Ok(FizzBuzzReturn::Many(
(s.start..s.stop)
.into_par_iter()
.step_by(step.try_into().unwrap())
.fizzbuzz()
.collect(),
)),
// ```python
// >>> foo[1:5:0]
// Traceback (most recent call last):
// File "<stdin>", line 1, in <module>
// ValueError: slice step cannot be zero
// ```
0 => Err(PyValueError::new_err("step cannot be zero")),
// ```python
// >>> foo=[0,1,2,3,4,5,6]
// >>> foo[6:0:-2]
// [6, 4, 2]
// ```
// Rust doesn't accept step < 0 or stop < start so need some trickery
..=-1 => Ok(FizzBuzzReturn::Many(
(s.start.neg()..s.stop.neg())
.into_par_iter()
.step_by(step.neg().try_into().unwrap())
.map(|x| x.neg())
.fizzbuzz()
.collect(),
)),
},
},
}
}
#[pymodule]
#[pyo3(name = "fizzbuzzo3")]
fn py_fizzbuzzo3(module: &Bound<'_, PyModule>) -> PyResult<()> {
module.add_function(wrap_pyfunction!(py_fizzbuzz, module)?)?;
Ok(())
}
#[cfg(test)]
mod tests {
use pyo3::exceptions::{PyTypeError, PyValueError};
use pyo3_testing::{pyo3test, with_py_raises};
use super::*;
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz() {
let result: String = fizzbuzz!(1i32);
assert_eq!(result, "1");
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_float() {
let result: String = fizzbuzz!(1f32);
assert_eq!(result, "1");
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_vec() {
let input = vec![1, 2, 3, 4, 5];
let expected = vec![
"1".to_string(),
"2".to_string(),
"fizz".to_string(),
"4".to_string(),
"buzz".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[allow(unused_macros)]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_string() {
with_py_raises!(PyTypeError, { fizzbuzz.call1(("4",)) })
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice() {
let input = MySlice {
start: 1,
stop: 6,
step: Some(1),
};
let expected = vec![
"1".to_string(),
"2".to_string(),
"fizz".to_string(),
"4".to_string(),
"buzz".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_no_step() {
let input = MySlice {
start: 1,
stop: 6,
step: None,
};
let expected = vec![
"1".to_string(),
"2".to_string(),
"fizz".to_string(),
"4".to_string(),
"buzz".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_step() {
let input = MySlice {
start: 1,
stop: 6,
step: Some(2),
};
let expected = vec!["1".to_string(), "fizz".to_string(), "buzz".to_string()];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_backwards() {
let input = MySlice {
start: 5,
stop: 0,
step: Some(1),
};
let result: Vec<String> = fizzbuzz!(input);
let expected: Vec<String> = vec![];
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_negative_step() {
let input = MySlice {
start: 5,
stop: 0,
step: Some(-2),
};
let expected = vec!["buzz".to_string(), "fizz".to_string(), "1".to_string()];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_negative_step_boundaries() {
let input = MySlice {
start: 5,
stop: 1,
step: Some(-1),
};
let expected = vec![
"buzz".to_string(),
"4".to_string(),
"fizz".to_string(),
"2".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_negative_step_boundaries_2() {
let input = MySlice {
start: 6,
stop: 0,
step: Some(-2),
};
let expected = vec!["fizz".to_string(), "4".to_string(), "2".to_string()];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[allow(unused_macros)]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_zero_step() {
let slice: MySlice = py
.eval_bound("slice(1,2,0)", None, None)
.unwrap()
.extract()
.unwrap();
with_py_raises!(PyValueError, { fizzbuzz.call1((slice,)) });
}
}
rust/fizzbuzz/src/lib.rs
- full source
//! A Trait implementation of "fizzbuzz" providing a default implementation for all types which can be
//! constructed from `0`,`3`,`5` and support the `%` operator.
//!
//!
//! ## Example usage for single item:
//!
//! ```
//! use fizzbuzz::FizzBuzz;
//! use std::borrow::Cow;
//!
//! let one: Cow<str> = 1.fizzbuzz().into();
//! let three: String = 3.fizzbuzz().into();
//! assert_eq!(&one, "1");
//! assert_eq!(three, "fizz");
//! ```
//!
//! ## Example usage for multiple items:
//!
//! ```
//! use fizzbuzz::MultiFizzBuzz;
//! use rayon::iter::ParallelIterator; // required to `.collect()` the results
//!
//! let one_to_five = vec![1,2,3,4,5];
//! let fizzbuzzed: Vec<String> = one_to_five.fizzbuzz().collect();
//! assert_eq!(fizzbuzzed, vec!["1", "2", "fizz", "4", "buzz"]);
//! ```
use std::borrow::Cow;
use rayon::prelude::*;
static BIG_VECTOR: usize = 300_000; // Size from which parallelisation makes sense
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord)]
/// Represents a valid answer to fizzbuzz and provides conversion to `String` and `Cow<&str>` via `.into()`
pub enum FizzBuzzAnswer {
Fizz,
Buzz,
Fizzbuzz,
Number(String),
}
impl From<FizzBuzzAnswer> for Cow<'static, str> {
fn from(answer: FizzBuzzAnswer) -> Self {
match answer {
FizzBuzzAnswer::Fizz => "fizz".into(),
FizzBuzzAnswer::Buzz => "buzz".into(),
FizzBuzzAnswer::Fizzbuzz => "fizzbuzz".into(),
FizzBuzzAnswer::Number(n) => n.into(),
}
}
}
impl From<FizzBuzzAnswer> for String {
fn from(answer: FizzBuzzAnswer) -> Self {
match answer {
FizzBuzzAnswer::Fizz => "fizz".into(),
FizzBuzzAnswer::Buzz => "buzz".into(),
FizzBuzzAnswer::Fizzbuzz => "fizzbuzz".into(),
FizzBuzzAnswer::Number(n) => n,
}
}
}
/// Used to obtain the correct fizzbuzz answer for a given number
///
/// ### Required:
/// - fn fizzbuzz() -> String
pub trait FizzBuzz {
/// Computes the FizzBuzz value for the implementing type.
/// Returns a `FizzBuzzAnswer` which provides a structured representation
/// of the FizzBuzz result.
///
/// - `fizzbuzz` if the number is directly divisible by 5 *and* 3
/// - `fizz` if the number is directly divisible by 3
/// - `buzz` if the number is directly divisible by 5
/// - the number in other cases
///
/// A default implementation is available for any type `<Num>` which supports
/// - `<Num>::try_from(<u8>)`: `Num` must be able to be constructed from `0`, `3` & `5`.
/// - `std::fmt::Display`: Allows `Num` to be formatted as a `String`.
/// - `PartialEq`: Enables comparison operations for `Num`.
/// - `<&Num>::Rem<Num, Output = Num>`: Allows `&Num % Num`.
fn fizzbuzz(&self) -> FizzBuzzAnswer;
}
/// Implements the FizzBuzz trait for any type `<T>` which supports `<T>::from(<u8>)`
/// and `<T> % <T>`
impl<Num> FizzBuzz for Num
where
Num: TryFrom<u8> + std::fmt::Display + PartialEq,
for<'a> &'a Num: std::ops::Rem<Num, Output = Num>,
{
fn fizzbuzz(&self) -> FizzBuzzAnswer {
let three = match <Num>::try_from(3_u8) {
Ok(three) => three,
Err(_) => return FizzBuzzAnswer::Number(self.to_string()),
};
let five = match <Num>::try_from(5_u8) {
Ok(five) => five,
Err(_) => return FizzBuzzAnswer::Number(self.to_string()),
};
let zero = match <Num>::try_from(0_u8) {
Ok(zero) => zero,
Err(_) => return FizzBuzzAnswer::Number(self.to_string()),
};
match (self % three == zero, self % five == zero) {
(true, true) => FizzBuzzAnswer::Fizzbuzz,
(true, false) => FizzBuzzAnswer::Fizz,
(false, true) => FizzBuzzAnswer::Buzz,
_ => FizzBuzzAnswer::Number(self.to_string()),
}
}
}
/// Used to obtain the correct `FizzBuzzAnswer` for a multiple fizzbuzz-able numbers
pub trait MultiFizzBuzz {
/// Returns an iterator which provides the FizzBuzz values for the elements of the implementing type.
///
/// Note:
/// - This function **consumes** the input
/// - The returned iterator is a `rayon::iter::IndexedParallelIterator`
/// - The Items in the returned iterator will be converted to a requested type
/// (e.g. `FizzBuzzAnswer`, `String`, `Cow<str>`)
fn fizzbuzz<Rtn>(self) -> impl IndexedParallelIterator<Item = Rtn>
where
Rtn: From<FizzBuzzAnswer> + Send;
}
/// Implements the MultiFizzBuzz trait for any type which can be easily converted into a
/// `rayon::iter::IndexedParallelIterator` over Items which implement `fizzbuzz::FizzBuzz`
///
/// Note:
/// - The returned iterator is _lazy_ - no calculations are performed until you use it
/// - Collecting this iterator requires that `rayon::iter::ParallelIterator` is in scope
/// - This implementation will decide whether it is worth the overhead of spawning multiple parallel threads
impl<Iterable, Num> MultiFizzBuzz for Iterable
where
Iterable: rayon::iter::IntoParallelIterator<Item = Num>,
<Iterable as IntoParallelIterator>::Iter: IndexedParallelIterator,
Num: FizzBuzz,
{
fn fizzbuzz<Rtn>(self) -> impl IndexedParallelIterator<Item = Rtn>
where
Rtn: From<FizzBuzzAnswer> + Send,
{
let par_iter = self.into_par_iter();
let min_len = if par_iter.len() < BIG_VECTOR {
BIG_VECTOR //Don't parallelise when small
} else {
1
};
par_iter.with_min_len(min_len).map(|n| n.fizzbuzz().into())
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn big_vector_is_well_ordered() {
let input: Vec<_> = (1..BIG_VECTOR + 2).collect();
let output: Vec<FizzBuzzAnswer> = input.clone().fizzbuzz().collect();
let mut expected: Vec<FizzBuzzAnswer> = vec![];
for i in input.iter() {
expected.push(i.fizzbuzz())
}
assert_eq!(output, expected);
}
#[test]
fn fizzbuzz_range() {
let input = 1..20;
let mut expected: Vec<FizzBuzzAnswer> = vec![];
for i in 1..20 {
expected.push(i.fizzbuzz().into())
}
let output: Vec<FizzBuzzAnswer> = input.fizzbuzz().collect();
assert_eq!(output, expected)
}
}
Ducktyping & Union
types
Remember your primary users are python coders who are used to duck typing . They will expect fizzbuzz(3.0)
to return '3.0'
and fizzbuzz(3.1)
to return '3.1'
unless something is documented regarding rounding to the nearest integer or similar. (Leaving aside any discussion on why floats are inaccurate).
Python also often provides single functions which can receive multiple significantly different types for a single argument: e.g. fizzbuzz([1,2,3])
and fizzbuzz(3)
could easily both work. The function signature would be def fizzbuzz(n: int | list[int]) -> str:
.
Use a custom enum and match to allow multiple types
This is best done directly in your wrapping library as it is part of the rust-python interface not the core functionality.
In rust/fizzbuzzo3/src/lib.rs
I used this pattern:
...
#[derive(FromPyObject)]
enum FizzBuzzable {
Int(isize),
Float(f64),
Vec(Vec<isize>),
}
...
#[pyfunction]
#[pyo3(name = "fizzbuzz", text_signature = "(n)")]
fn py_fizzbuzz(num: FizzBuzzable) -> String {
match num {
FizzBuzzable::Int(n) => n.fizzbuzz().into(),
FizzBuzzable::Float(n) => n.fizzbuzz().into(),
FizzBuzzable::Vec(v) => v.fizzbuzz().into(),
}
}
rust/fizzbuzzo3/src/lib.rs
- full source
use std::{borrow::Cow, ops::Neg};
use fizzbuzz::{FizzBuzz, FizzBuzzAnswer, MultiFizzBuzz};
use pyo3::{exceptions::PyValueError, prelude::*, types::PySlice};
use rayon::iter::{IndexedParallelIterator, IntoParallelIterator, ParallelIterator};
#[derive(FromPyObject)]
enum FizzBuzzable {
Int(isize),
Float(f64),
Vec(Vec<isize>),
Slice(MySlice),
}
#[derive(FromPyObject)]
struct MySlice {
start: isize,
stop: isize,
step: Option<isize>,
}
impl IntoPy<Py<PyAny>> for MySlice {
fn into_py(self, py: Python<'_>) -> Py<PyAny> {
PySlice::new_bound(py, self.start, self.stop, self.step.unwrap_or(1)).into_py(py)
}
}
/// A wrapper struct for FizzBuzzAnswer to provide a custom implementation of `IntoPy`.
enum FizzBuzzReturn {
One(String),
Many(Vec<Cow<'static, str>>),
}
impl From<FizzBuzzAnswer> for FizzBuzzReturn {
fn from(value: FizzBuzzAnswer) -> Self {
FizzBuzzReturn::One(value.into())
}
}
impl IntoPy<Py<PyAny>> for FizzBuzzReturn {
fn into_py(self, py: Python<'_>) -> Py<PyAny> {
match self {
FizzBuzzReturn::One(answer) => answer.into_py(py),
FizzBuzzReturn::Many(answers) => answers.into_py(py),
}
}
}
/// Returns the correct fizzbuzz answer for any number or list/range of numbers.
///
/// This is an optimised algorithm compiled in rust. Large lists will utilise multiple CPU cores for processing.
/// Passing a slice, to represent a range, is fastest.
///
/// Arguments:
/// n: the number(s) to fizzbuzz
///
/// Returns:
/// In the case of a single number: a `str` with the correct fizzbuzz answer.
/// In the case of a list or range of inputs: a `list` of `str` with the correct fizzbuzz answers.
///
/// Examples:
/// a single `int`:
/// ```
/// >>> from fizzbuzz.fizzbuzzo3 import fizzbuzz
/// >>> fizzbuzz(1)
/// '1'
/// >>> fizzbuzz(3)
/// 'fizz'
/// ```
/// a `list`:
/// ```
/// from fizzbuzz.fizzbuzzo3 import fizzbuzz
/// >>> fizzbuzz([1,3])
/// ['1', 'fizz']
/// ```
/// a `slice` representing a range:
/// ```
/// from fizzbuzz.fizzbuzzo3 import fizzbuzz
/// >>> fizzbuzz(slice(1,4,2))
/// ['1', 'fizz']
/// >>> fizzbuzz(slice(1,4))
/// ['1', '2', 'fizz']
/// >>> fizzbuzz(slice(4,1,-1))
/// ['4', 'fizz', '2']
/// >>> fizzbuzz(slice(1,5,-1))
/// []
/// ```
/// Note: Slices are inclusive on the left, exclusive on the right and can contain an optional step.
/// Note: Slices are inclusive on the left, exclusive on the right and can contain an optional step.
/// Negative steps require start > stop, positive steps require stop > start; other combinations return `[]`.
/// A step of zero is invalid and will raise a `ValueError`.
#[pyfunction]
#[pyo3(name = "fizzbuzz", text_signature = "(n)")]
fn py_fizzbuzz(num: FizzBuzzable) -> PyResult<FizzBuzzReturn> {
match num {
FizzBuzzable::Int(n) => Ok(n.fizzbuzz().into()),
FizzBuzzable::Float(n) => Ok(n.fizzbuzz().into()),
FizzBuzzable::Vec(v) => Ok(FizzBuzzReturn::Many(v.fizzbuzz().collect())),
FizzBuzzable::Slice(s) => match s.step {
// Can only be tested from python: Cannot create a PySlice with no step in rust.
None => Ok(FizzBuzzReturn::Many((s.start..s.stop).fizzbuzz().collect())), // GRCOV_EXCL_LINE
Some(1) => Ok(FizzBuzzReturn::Many((s.start..s.stop).fizzbuzz().collect())),
Some(step) => match step {
1.. => Ok(FizzBuzzReturn::Many(
(s.start..s.stop)
.into_par_iter()
.step_by(step.try_into().unwrap())
.fizzbuzz()
.collect(),
)),
// ```python
// >>> foo[1:5:0]
// Traceback (most recent call last):
// File "<stdin>", line 1, in <module>
// ValueError: slice step cannot be zero
// ```
0 => Err(PyValueError::new_err("step cannot be zero")),
// ```python
// >>> foo=[0,1,2,3,4,5,6]
// >>> foo[6:0:-2]
// [6, 4, 2]
// ```
// Rust doesn't accept step < 0 or stop < start so need some trickery
..=-1 => Ok(FizzBuzzReturn::Many(
(s.start.neg()..s.stop.neg())
.into_par_iter()
.step_by(step.neg().try_into().unwrap())
.map(|x| x.neg())
.fizzbuzz()
.collect(),
)),
},
},
}
}
#[pymodule]
#[pyo3(name = "fizzbuzzo3")]
fn py_fizzbuzzo3(module: &Bound<'_, PyModule>) -> PyResult<()> {
module.add_function(wrap_pyfunction!(py_fizzbuzz, module)?)?;
Ok(())
}
#[cfg(test)]
mod tests {
use pyo3::exceptions::{PyTypeError, PyValueError};
use pyo3_testing::{pyo3test, with_py_raises};
use super::*;
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz() {
let result: String = fizzbuzz!(1i32);
assert_eq!(result, "1");
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_float() {
let result: String = fizzbuzz!(1f32);
assert_eq!(result, "1");
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_vec() {
let input = vec![1, 2, 3, 4, 5];
let expected = vec![
"1".to_string(),
"2".to_string(),
"fizz".to_string(),
"4".to_string(),
"buzz".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[allow(unused_macros)]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_string() {
with_py_raises!(PyTypeError, { fizzbuzz.call1(("4",)) })
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice() {
let input = MySlice {
start: 1,
stop: 6,
step: Some(1),
};
let expected = vec![
"1".to_string(),
"2".to_string(),
"fizz".to_string(),
"4".to_string(),
"buzz".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_no_step() {
let input = MySlice {
start: 1,
stop: 6,
step: None,
};
let expected = vec![
"1".to_string(),
"2".to_string(),
"fizz".to_string(),
"4".to_string(),
"buzz".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_step() {
let input = MySlice {
start: 1,
stop: 6,
step: Some(2),
};
let expected = vec!["1".to_string(), "fizz".to_string(), "buzz".to_string()];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_backwards() {
let input = MySlice {
start: 5,
stop: 0,
step: Some(1),
};
let result: Vec<String> = fizzbuzz!(input);
let expected: Vec<String> = vec![];
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_negative_step() {
let input = MySlice {
start: 5,
stop: 0,
step: Some(-2),
};
let expected = vec!["buzz".to_string(), "fizz".to_string(), "1".to_string()];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_negative_step_boundaries() {
let input = MySlice {
start: 5,
stop: 1,
step: Some(-1),
};
let expected = vec![
"buzz".to_string(),
"4".to_string(),
"fizz".to_string(),
"2".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_negative_step_boundaries_2() {
let input = MySlice {
start: 6,
stop: 0,
step: Some(-2),
};
let expected = vec!["fizz".to_string(), "4".to_string(), "2".to_string()];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[allow(unused_macros)]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_zero_step() {
let slice: MySlice = py
.eval_bound("slice(1,2,0)", None, None)
.unwrap()
.extract()
.unwrap();
with_py_raises!(PyValueError, { fizzbuzz.call1((slice,)) });
}
}
Union
type returns: def fizzbuzz(n: int | list[int]) -> str | list[str]
If you would like to provide different return types for different cases:
- Implement an
enum
, or a wrapperstruct
around an existingenum
, that holds the different types. - Provide one or more conversion
From
traits to convert from the return of your core rust functions. - Provide a conversion
IntoPy
trait to convert to the relevant PyO3 types. -
Use this new type as the return of your wrapped function.
In
/rust/fizzbuzzo3/src/lib.rs
:... struct FizzBuzzReturn(FizzBuzzAnswer); impl From<FizzBuzzAnswer> for FizzBuzzReturn { fn from(value: FizzBuzzAnswer) -> Self { FizzBuzzReturn(value) } } impl IntoPy<Py<PyAny>> for FizzBuzzReturn { fn into_py(self, py: Python<'_>) -> Py<PyAny> { match self.0 { FizzBuzzAnswer::One(string) => string.into_py(py), FizzBuzzAnswer::Many(list) => list.into_py(py), } } } ... #[pyfunction] #[pyo3(name = "fizzbuzz", text_signature = "(n)")] fn py_fizzbuzz(num: FizzBuzzable) -> PyResult<FizzBuzzReturn> { ...
Thanks to the comments in Issue pyo3/#1637 for pointers on how to get this working.
-
Add
@overload
hints for your IDE (see IDE type & doc hinting), so that it understands the relationships between input and output types:In
/python/fizzbuzz/fizzbuzzo3.pyi
:... from typing import overload @overload def fizzbuzz(n: int) -> str: ... @overload def fizzbuzz(n: list[int] | slice) -> list[str]: ... def fizzbuzz(n): """ Returns the correct fizzbuzz answer for any number or list/range of numbers. ...
rust/fizzbuzzo3/src/lib.rs
- full source
use std::{borrow::Cow, ops::Neg};
use fizzbuzz::{FizzBuzz, FizzBuzzAnswer, MultiFizzBuzz};
use pyo3::{exceptions::PyValueError, prelude::*, types::PySlice};
use rayon::iter::{IndexedParallelIterator, IntoParallelIterator, ParallelIterator};
#[derive(FromPyObject)]
enum FizzBuzzable {
Int(isize),
Float(f64),
Vec(Vec<isize>),
Slice(MySlice),
}
#[derive(FromPyObject)]
struct MySlice {
start: isize,
stop: isize,
step: Option<isize>,
}
impl IntoPy<Py<PyAny>> for MySlice {
fn into_py(self, py: Python<'_>) -> Py<PyAny> {
PySlice::new_bound(py, self.start, self.stop, self.step.unwrap_or(1)).into_py(py)
}
}
/// A wrapper struct for FizzBuzzAnswer to provide a custom implementation of `IntoPy`.
enum FizzBuzzReturn {
One(String),
Many(Vec<Cow<'static, str>>),
}
impl From<FizzBuzzAnswer> for FizzBuzzReturn {
fn from(value: FizzBuzzAnswer) -> Self {
FizzBuzzReturn::One(value.into())
}
}
impl IntoPy<Py<PyAny>> for FizzBuzzReturn {
fn into_py(self, py: Python<'_>) -> Py<PyAny> {
match self {
FizzBuzzReturn::One(answer) => answer.into_py(py),
FizzBuzzReturn::Many(answers) => answers.into_py(py),
}
}
}
/// Returns the correct fizzbuzz answer for any number or list/range of numbers.
///
/// This is an optimised algorithm compiled in rust. Large lists will utilise multiple CPU cores for processing.
/// Passing a slice, to represent a range, is fastest.
///
/// Arguments:
/// n: the number(s) to fizzbuzz
///
/// Returns:
/// In the case of a single number: a `str` with the correct fizzbuzz answer.
/// In the case of a list or range of inputs: a `list` of `str` with the correct fizzbuzz answers.
///
/// Examples:
/// a single `int`:
/// ```
/// >>> from fizzbuzz.fizzbuzzo3 import fizzbuzz
/// >>> fizzbuzz(1)
/// '1'
/// >>> fizzbuzz(3)
/// 'fizz'
/// ```
/// a `list`:
/// ```
/// from fizzbuzz.fizzbuzzo3 import fizzbuzz
/// >>> fizzbuzz([1,3])
/// ['1', 'fizz']
/// ```
/// a `slice` representing a range:
/// ```
/// from fizzbuzz.fizzbuzzo3 import fizzbuzz
/// >>> fizzbuzz(slice(1,4,2))
/// ['1', 'fizz']
/// >>> fizzbuzz(slice(1,4))
/// ['1', '2', 'fizz']
/// >>> fizzbuzz(slice(4,1,-1))
/// ['4', 'fizz', '2']
/// >>> fizzbuzz(slice(1,5,-1))
/// []
/// ```
/// Note: Slices are inclusive on the left, exclusive on the right and can contain an optional step.
/// Note: Slices are inclusive on the left, exclusive on the right and can contain an optional step.
/// Negative steps require start > stop, positive steps require stop > start; other combinations return `[]`.
/// A step of zero is invalid and will raise a `ValueError`.
#[pyfunction]
#[pyo3(name = "fizzbuzz", text_signature = "(n)")]
fn py_fizzbuzz(num: FizzBuzzable) -> PyResult<FizzBuzzReturn> {
match num {
FizzBuzzable::Int(n) => Ok(n.fizzbuzz().into()),
FizzBuzzable::Float(n) => Ok(n.fizzbuzz().into()),
FizzBuzzable::Vec(v) => Ok(FizzBuzzReturn::Many(v.fizzbuzz().collect())),
FizzBuzzable::Slice(s) => match s.step {
// Can only be tested from python: Cannot create a PySlice with no step in rust.
None => Ok(FizzBuzzReturn::Many((s.start..s.stop).fizzbuzz().collect())), // GRCOV_EXCL_LINE
Some(1) => Ok(FizzBuzzReturn::Many((s.start..s.stop).fizzbuzz().collect())),
Some(step) => match step {
1.. => Ok(FizzBuzzReturn::Many(
(s.start..s.stop)
.into_par_iter()
.step_by(step.try_into().unwrap())
.fizzbuzz()
.collect(),
)),
// ```python
// >>> foo[1:5:0]
// Traceback (most recent call last):
// File "<stdin>", line 1, in <module>
// ValueError: slice step cannot be zero
// ```
0 => Err(PyValueError::new_err("step cannot be zero")),
// ```python
// >>> foo=[0,1,2,3,4,5,6]
// >>> foo[6:0:-2]
// [6, 4, 2]
// ```
// Rust doesn't accept step < 0 or stop < start so need some trickery
..=-1 => Ok(FizzBuzzReturn::Many(
(s.start.neg()..s.stop.neg())
.into_par_iter()
.step_by(step.neg().try_into().unwrap())
.map(|x| x.neg())
.fizzbuzz()
.collect(),
)),
},
},
}
}
#[pymodule]
#[pyo3(name = "fizzbuzzo3")]
fn py_fizzbuzzo3(module: &Bound<'_, PyModule>) -> PyResult<()> {
module.add_function(wrap_pyfunction!(py_fizzbuzz, module)?)?;
Ok(())
}
#[cfg(test)]
mod tests {
use pyo3::exceptions::{PyTypeError, PyValueError};
use pyo3_testing::{pyo3test, with_py_raises};
use super::*;
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz() {
let result: String = fizzbuzz!(1i32);
assert_eq!(result, "1");
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_float() {
let result: String = fizzbuzz!(1f32);
assert_eq!(result, "1");
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_vec() {
let input = vec![1, 2, 3, 4, 5];
let expected = vec![
"1".to_string(),
"2".to_string(),
"fizz".to_string(),
"4".to_string(),
"buzz".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[allow(unused_macros)]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_string() {
with_py_raises!(PyTypeError, { fizzbuzz.call1(("4",)) })
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice() {
let input = MySlice {
start: 1,
stop: 6,
step: Some(1),
};
let expected = vec![
"1".to_string(),
"2".to_string(),
"fizz".to_string(),
"4".to_string(),
"buzz".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_no_step() {
let input = MySlice {
start: 1,
stop: 6,
step: None,
};
let expected = vec![
"1".to_string(),
"2".to_string(),
"fizz".to_string(),
"4".to_string(),
"buzz".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_step() {
let input = MySlice {
start: 1,
stop: 6,
step: Some(2),
};
let expected = vec!["1".to_string(), "fizz".to_string(), "buzz".to_string()];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_backwards() {
let input = MySlice {
start: 5,
stop: 0,
step: Some(1),
};
let result: Vec<String> = fizzbuzz!(input);
let expected: Vec<String> = vec![];
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_negative_step() {
let input = MySlice {
start: 5,
stop: 0,
step: Some(-2),
};
let expected = vec!["buzz".to_string(), "fizz".to_string(), "1".to_string()];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_negative_step_boundaries() {
let input = MySlice {
start: 5,
stop: 1,
step: Some(-1),
};
let expected = vec![
"buzz".to_string(),
"4".to_string(),
"fizz".to_string(),
"2".to_string(),
];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_negative_step_boundaries_2() {
let input = MySlice {
start: 6,
stop: 0,
step: Some(-2),
};
let expected = vec!["fizz".to_string(), "4".to_string(), "2".to_string()];
let result: Vec<String> = fizzbuzz!(input);
assert_eq!(result, expected);
}
#[pyo3test]
#[allow(unused_macros)]
#[pyo3import(py_fizzbuzzo3: from fizzbuzzo3 import fizzbuzz)]
fn test_fizzbuzz_slice_zero_step() {
let slice: MySlice = py
.eval_bound("slice(1,2,0)", None, None)
.unwrap()
.extract()
.unwrap();
with_py_raises!(PyValueError, { fizzbuzz.call1((slice,)) });
}
}
python/fizzbuzz/fizzbuzzo3.pyi
- full source
# flake8: noqa: PYI021
"""
An optimised rust version of fizzbuzz.
Provides a fizzbuzz() function which will run on multiple CPU cores if needed.
Usage:
```
>>> from fizzbuzz.fizzbuzzo3 import fizzbuzz
```
"""
from typing import overload
@overload
def fizzbuzz(n: int) -> str:
...
@overload
def fizzbuzz(n: list[int] | slice) -> list[str]:
...
def fizzbuzz(n):
"""
Returns the correct fizzbuzz answer for any number or list/range of numbers.
This is an optimised algorithm compiled in rust. Large lists will utilise multiple CPU cores for processing.
Passing a slice, to represent a range, is fastest.
Arguments:
n: the number(s) to fizzbuzz
Returns:
In the case of a single number: a `str` with the correct fizzbuzz answer.
In the case of a list or range of inputs: a `list` of `str` with the correct fizzbuzz answers.
Examples:
a single `int`:
```
>>> from fizzbuzz.fizzbuzzo3 import fizzbuzz
>>> fizzbuzz(1)
'1'
>>> fizzbuzz(3)
'fizz'
```
a `list`:
```
from fizzbuzz.fizzbuzzo3 import fizzbuzz
>>> fizzbuzz([1,3])
['1', 'fizz']
```
a `slice` representing a range:
```
from fizzbuzz.fizzbuzzo3 import fizzbuzz
>>> fizzbuzz(slice(1,4,2))
['1', 'fizz']
>>> fizzbuzz(slice(1,4))
['1', '2', 'fizz']
>>> fizzbuzz(slice(4,1,-1))
['4', 'fizz', '2']
>>> fizzbuzz(slice(1,5,-1))
[]
```
Note: Slices are inclusive on the left, exclusive on the right and can contain an optional step.
Negative steps require start > stop, positive steps require stop > start; other combinations return `[]`.
A step of zero is invalid and will raise a `ValueError`.
"""
IDE type & doc hinting
Pyo3 does a great job automatically exporting inline rust documentation (using /// ...
) as python docstrings. It also creates a simple attribute detailling the function signature, which you can manually adjust.
IDEs, linters, etc. don't actually import your code to read the docstrings and signatures, they parse the source-code; and with the source in rust, they can't do this directly for your wrapped modules.
Autogenerating hints
Because I hate copy-pasting stuff I created pyo3-stubgen to auto-generate the information. It is available on pypi: pip install pyo3-stubgen
, has a simple command line interface and can also be called from python if you prefer.
Create a .pyi
file
- Create a stub file with:
- the same name as your exported module
- the extension
.pyi
- in the location you would otherwise have placed the
module.py
file
- Add function definitions with type hints and docstrings but no code
- For functions with no docstrings enter
...
as the function body - Add the
.pyi
extension to the files checked by doctest:./pyproject.toml
:[tool.pytest.ini_options] ... addopts = [ "--doctest-modules", ... "--doctest-glob=*.pyi", ... ]
python/fizzbuzz/fizzbuzzo3.pyi
- full source
# flake8: noqa: PYI021
"""
An optimised rust version of fizzbuzz.
Provides a fizzbuzz() function which will run on multiple CPU cores if needed.
Usage:
```
>>> from fizzbuzz.fizzbuzzo3 import fizzbuzz
```
"""
from typing import overload
@overload
def fizzbuzz(n: int) -> str:
...
@overload
def fizzbuzz(n: list[int] | slice) -> list[str]:
...
def fizzbuzz(n):
"""
Returns the correct fizzbuzz answer for any number or list/range of numbers.
This is an optimised algorithm compiled in rust. Large lists will utilise multiple CPU cores for processing.
Passing a slice, to represent a range, is fastest.
Arguments:
n: the number(s) to fizzbuzz
Returns:
In the case of a single number: a `str` with the correct fizzbuzz answer.
In the case of a list or range of inputs: a `list` of `str` with the correct fizzbuzz answers.
Examples:
a single `int`:
```
>>> from fizzbuzz.fizzbuzzo3 import fizzbuzz
>>> fizzbuzz(1)
'1'
>>> fizzbuzz(3)
'fizz'
```
a `list`:
```
from fizzbuzz.fizzbuzzo3 import fizzbuzz
>>> fizzbuzz([1,3])
['1', 'fizz']
```
a `slice` representing a range:
```
from fizzbuzz.fizzbuzzo3 import fizzbuzz
>>> fizzbuzz(slice(1,4,2))
['1', 'fizz']
>>> fizzbuzz(slice(1,4))
['1', '2', 'fizz']
>>> fizzbuzz(slice(4,1,-1))
['4', 'fizz', '2']
>>> fizzbuzz(slice(1,5,-1))
[]
```
Note: Slices are inclusive on the left, exclusive on the right and can contain an optional step.
Negative steps require start > stop, positive steps require stop > start; other combinations return `[]`.
A step of zero is invalid and will raise a `ValueError`.
"""
./pyproject.toml
- full source
[project]
name = "fizzbuzz"
description = "An implementation of the traditional fizzbuzz kata"
# readme = "README.md"
license = { text = "MIT" }
authors = [{ name = "Mike Foster" }]
dynamic = ["version"]
requires-python = ">=3.8"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Topic :: Software Development :: Libraries :: Python Modules",
"Intended Audience :: Developers",
"Natural Language :: English",
]
[project.urls]
# Homepage = ""
# Documentation = ""
Repository = "https://github.com/MusicalNinjaDad/fizzbuzz"
Issues = "https://github.com/MusicalNinjaDad/fizzbuzz/issues"
# Changelog = ""
[tool.setuptools.dynamic]
version = { file = "__pyversion__" }
# ===========================
# Build, package
# ===========================
[build-system]
requires = ["setuptools", "setuptools-rust"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["python"]
[[tool.setuptools-rust.ext-modules]]
# The last part of the name (e.g. "_lib") has to match lib.name in Cargo.toml,
# but you can add a prefix to nest it inside of a Python package.
target = "fizzbuzz.fizzbuzzo3"
path = "rust/fizzbuzzo3/Cargo.toml"
binding = "PyO3"
features = ["pyo3/extension-module"] # IMPORTANT!!!
debug = false # Adds `--release` to `pip install -e .` builds, necessary for performance testing
[tool.cibuildwheel]
skip = [
"*-musllinux_i686", # No musllinux-i686 rust compiler available.
"*-musllinux_ppc64le", # no rust target available.
"*-musllinux_s390x", # no rust target available.
]
# Use prebuilt copies of pypa images with rust and just ready compiled (saves up to 25 mins on emulated archs)
manylinux-x86_64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_x86_64-rust"
manylinux-i686-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_i686-rust"
manylinux-aarch64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_aarch64-rust"
manylinux-ppc64le-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_ppc64le-rust"
manylinux-s390x-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_s390x-rust"
manylinux-pypy_x86_64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_x86_64-rust"
manylinux-pypy_i686-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_i686-rust"
manylinux-pypy_aarch64-image = "ghcr.io/musicalninjas/quay.io/pypa/manylinux2014_aarch64-rust"
musllinux-x86_64-image = "ghcr.io/musicalninjas/quay.io/pypa/musllinux_1_2_x86_64-rust"
musllinux-aarch64-image = "ghcr.io/musicalninjas/quay.io/pypa/musllinux_1_2_aarch64-rust"
# `test-command` will FAIL if only passed `pytest {package}`
# as this tries to resolve imports based on source files (where there is no `.so` for rust libraries), not installed wheel
test-command = "pytest {package}/tests"
test-extras = "test"
[tool.cibuildwheel.linux]
before-all = "just clean"
archs = "all"
[tool.cibuildwheel.macos]
before-all = "curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal && cargo install just && just clean"
[tool.cibuildwheel.windows]
before-all = "rustup target add aarch64-pc-windows-msvc i586-pc-windows-msvc i686-pc-windows-msvc x86_64-pc-windows-msvc && cargo install just && just clean"
before-build = "pip install delvewheel"
repair-wheel-command = "delvewheel repair -w {dest_dir} {wheel}"
# ===========================
# Test, lint, documentation
# ===========================
[project.optional-dependencies]
stubs = ["pyo3-stubgen"]
lint = ["ruff"]
test = ["pytest", "pytest-doctest-mkdocstrings"]
cov = ["fizzbuzz[test]", "pytest-cov"]
doc = ["black", "mkdocs", "mkdocstrings[python]", "mkdocs-material", "fizzbuzz[stubs]"]
dev = ["fizzbuzz[stubs,lint,test,cov,doc]"]
[tool.pytest.ini_options]
xfail_strict = true
addopts = [
"--doctest-modules",
"--doctest-mdcodeblocks",
"--doctest-glob=*.pyi",
"--doctest-glob=*.md",
]
testpaths = ["tests", "python"]
[tool.coverage.run]
branch = true
omit = ["tests/*"]
dynamic_context = "test_function"
[tool.ruff]
line-length = 120
format.skip-magic-trailing-comma = false
format.quote-style = "double"
[tool.ruff.lint]
select = ["ALL"]
flake8-pytest-style.fixture-parentheses = false
flake8-annotations.mypy-init-return = true
extend-ignore = [
"E701", # One-line ifs are not great, one-liners with suppression are worse
"ANN101", # Type annotation for `self`
"ANN202", # Return type annotation for private functions
"ANN401", # Using Any often enough that supressing individually is pointless
"W291", # Double space at EOL is linebreak in md-docstring
"W292", # Too much typing to add newline at end of each file
"W293", # Whitespace only lines are OK for us
"D401", # First line of docstring should be in imperative mood ("Returns ..." is OK, for example)
"D203", # 1 blank line required before class docstring (No thank you, only after the class docstring)
"D212", # Multi-line docstring summary should start at the first line (We start on new line, D209)
"D215", # Section underline is over-indented (No underlines)
"D400", # First line should end with a period (We allow any punctuation, D415)
"D406", # Section name should end with a newline (We use a colon, D416)
"D407", # Missing dashed underline after section (No underlines)
"D408", # Section underline should be in the line following the section's name (No underlines)
"D409", # Section underline should match the length of its name (No underlines)
"D413", # Missing blank line after last section (Not required)
]
[tool.ruff.lint.per-file-ignores]
# Additional ignores for tests
"**/test_*.py" = [
"INP001", # Missing __init__.py
"ANN", # Missing type annotations
"S101", # Use of `assert`
"PLR2004", # Magic number comparisons are OK in tests
"D1", # Don't REQUIRE docstrings for tests - but they are nice
]
"**/__init__.py" = [
"D104", # Don't require module docstring in __init__.py
"F401", # Unused imports are fine: using __init__.py to expose them with implicit __ALL__
]
Pyo3 discusses this topic in Appendix C.