Development Environment

To work with both python and rust you're going to need a toolchain in place for both eco-systems and be on a Linux(-like) system. I've done all this on Windows with WSL and it's simply awesome!

I, personally, like to keep my working machine free of the range of different packages and tools that I need for all the different projects I look at. I also like to be able to rebuild a reproducible working environment easily, and let others know which tools they need if they want to contribute.

Dev Containers make this easy, they consist of:

a Dockerfile with the entire set of tools needed for development
a json file with IDE settings, extensions and project specific build scripts

Think of it like this: you wouldn't develop a python project without a virtual environment ...

Quick-Start

Getting up and running fast

Install docker (any set up will do but docker desktop works easiest in my experience if you're starting from scratch)
Install the ms-vscode-remote.remote-containers extension in VSCode
Copy .devcontainer/devcontainer.json from MusicalNinjaDad/FizzBuzz into your project (keeping the location and filename)
Select Reopen in containerfrom the toastie or the command bar

Packages and tools (Dockerfile)

Pre-Build Docker Image

You can grab a pre-build image to use in your devcontainer for either linux/amd64 or linux/arm64

Add to .devcontainer/devcontainer.json:

"image": "ghcr.io/musicalninjas/pyo3-devcontainer"

If you want to run it directly docker run -it ghcr.io/musicalninjas/pyo3-devcontainer:latest. The source is at MusicalNinjas/devcontainers-pyo3

I like to use fedora for my dev environment base as it provides the most up-to-date versions of tools via the package manager dnf, I also like to keep the environment as clean as possible, with only the tools that the project needs. You'll find similarly named packages in most distros.

MusicalNinjas/devcontainers-pyo3/Dockerfile - full source

FROM docker.io/library/fedora:40

# ---
# Setup base system ...
# ---

# Enable man pages by commenting out the nodocs flag
COPY <<EOF /etc/dnf/dnf.conf
[main]
gpgcheck=True
installonly_limit=3
clean_requirements_on_remove=True
best=False
skip_if_unavailable=True
# tsflags=nodocs
EOF

# Rust stuff goes in /opt so we don't end up with system and user installs: this is a single user system.
ENV RUSTUP_HOME=/opt/rustup \
    CARGO_HOME=/opt/cargo \
    PATH=/opt/cargo/bin:$PATH
RUN mkdir --mode=777 --parents $RUSTUP_HOME \
&& mkdir --mode=777 --parents $CARGO_HOME

# Create the default user - most agents mount workspace directory chowned to 1000:1000
ARG USERNAME=pyo3
ARG USER_UID=1000
ARG USER_GID=${USER_UID}
RUN groupadd --gid ${USER_GID} ${USERNAME} \
&& useradd --uid ${USER_UID} --gid ${USER_GID} -m ${USERNAME} \
&& echo ${USERNAME} ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/${USERNAME} \
&& chmod 0440 /etc/sudoers.d/${USERNAME}

# ---
# Install ...
# ---

# Man pages for all the stuff which is already installed, man itself and basic manpages
RUN dnf -y --setopt=install_weak_deps=False reinstall $(dnf list --installed | awk '{print $1}') \
&& dnf -y --setopt=install_weak_deps=False install \
    man \
    man-db \
    man-pages \
&& dnf -y update

# Basic development tools
RUN dnf -y --setopt=install_weak_deps=False install \
    bash-completion \
    git \
    just \
    which

# Python
RUN dnf -y install \
    python \
    python-pip

# Rust (and python headers)
# and chown CARGO_HOME and RUSTUP_HOME to the default user 
RUN dnf -y install \
    clang \
    python3-devel \
    rustup \
&& rustup-init -v -y \
&& rustup component add \
    llvm-tools-preview \
    rust-src \
&& cargo install \ 
    cargo-expand \
    cargo-cyclonedx \    
    grcov \
    mdbook \
&& chmod -R a+rwX ${CARGO_HOME} \
&& chmod -R a+rwX ${RUSTUP_HOME}

# ---
# Final setup steps
# ---

# Set the default user
USER ${USERNAME}

If you are putting together an environment for yourself the important things to note are the packages in the # Python and # Rust (and python headers) sections. As well as the usual tools for developing in each language you will also need the python headers for pyo3 to use - in the case of fedora that's python3-devel.

Python

Most distributions need you to install the python package manager pip with a dedicated package. Python is so embedded into the OS that you really don't wan't to go installing a load of packages and a different version of python at the system level.

There is no point installing a range of other python packages as you will be re-downloading these in a virtual environment anyway to keep your project set up and your system setup separate.

dnf -y install python-pip

Rust

Most distributions don't bundle a recent version of rust-up, so you need to install rust by curl-ing a shell-script as described at rustup.rs. Fedora is nice.

To get rust working you'll also need clang as a C-compiler and at least rust-src so you have the sources of the core libraries

dnf -y install \
    clang \
    python3-devel \
    rustup \
&& rustup-init -v -y \
&& rustup component add rust-src

If you're installing rust inside a docker container which will not be run as root you need to use RUSTUP_HOME and CARGO_HOME environment variables to select where to install, make sure they are accessible to all users, add cargo to the PATH and then chown or chmod the entire contents to make them usable by a non-root user. Otherwise the installation goes under /root and the files are rw-rw---- root root

Unlike python installing tools up-front via cargo can save quite some time when rebuilding / creating the dev environment as rust tools are compiled from source when you install them. Getting this done once when you build the container rather than every time you create an environment saves a significant number of minutes.

Other useful tools

In addition to the core language components I also found the following tools useful (they're in the Dockerfile):

just: You're going to be running a few multi-step commands. I came across it for the first time during this project and liked the simple approach. I guess you could use nox, but that means mentally translating from shell to python and context switching from rust to python if you're in the middle of rust-ing. make would be another option, but again, overly complicated for this use case, in my view
llvm-tools-preview and grcov for getting test coverage of your rust code. There are a few different ways to get rust coverage, after much research and a few unsuccessful starts I found grcov, from mozilla, which pretty much just worked, gave the best quality results and sounds to me like it works in a way that makes most sense
mdbook is what to use if you want to create rust-style manuals rather than python-style manuals
cargo-expand is invaluable if you end up writing rust proc-macros

IDE extensions and settings (json)

I use VSCode and love language-agnostic approach it takes. Devcontainers will run in IntelliJ as well (I've tried it with a colleague) and in neoVIM (I've read), many of these tips will probably translate to those IDEs too without too much google-ing.

.devcontainer/devcontainer.json - full source

{
    "image": "ghcr.io/musicalninjas/pyo3-devcontainer",
    "customizations": {
        "vscode": {
            "extensions": [
                // rust
                "rust-lang.rust-analyzer",
                "vadimcn.vscode-lldb",
                // python
                "ms-python.python",
                "ms-python.vscode-pylance",
                "charliermarsh.ruff",
                // configs, docs, etc.
                "DavidAnson.vscode-markdownlint",
                "tamasfe.even-better-toml"
                // DISABLE spell checking for now as exclusions not working ... TODO
                // "streetsidesoftware.code-spell-checker",
                // "streetsidesoftware.code-spell-checker-british-english"
            ],
            "settings": {
                // rust
                "rust-analyzer.interpret.tests": true,
                "rust-analyzer.testExplorer": true,
                "[rust]": {
                    "editor.rulers": [
                        100
                    ]
                },
                // python
                "python.defaultInterpreterPath": "./.venv/bin/python3",
                "python.testing.pytestEnabled": true,
                "[python]": {
                    "editor.rulers": [
                        120
                    ]
                },
                // docs
                "markdownlint.config": {
                    "MD013": false, // let the editor wrap lines not the author
                    // multi-paragraph admonitions in mkdocs-material are considered indented code blocks
                    // see also ... for possible improvements via a plugin:
                    // - https://github.com/DavidAnson/vscode-markdownlint/issues/180
                    // - https://github.com/DavidAnson/vscode-markdownlint/issues/302
                    // - https://github.com/DavidAnson/markdownlint/issues/209
                    "MD046": false // {"style": "fenced"} leads to errors on codeblocks in admonitions
                },
                "[markdown]": {
                    "editor.tabSize": 2,
                    "editor.detectIndentation": false
                },
                // DISABLE spell checking for now as exclusions not working ... TODO
                // "cSpell.language": "en",
                // "cSpell.checkOnlyEnabledFileTypes": true,
                // "cSpell.enabledFileTypes": {"markdown": true, "json": false},

                // shell
                "terminal.integrated.defaultProfile.linux": "bash",
                "terminal.integrated.profiles.linux": {
                    "bash": {
                        "path": "/usr/bin/bash"
                    }
                }
            }
        }
    },
    "updateContentCommand": "just reset check"
}

Python

Extensions:

ms-python.python & ms-python.vscode-pylance give you language services, hints and intellisense
charliermarsh.ruff integrates ruff as your linter (It's written in rust!)

Settings:

"python.defaultInterpreterPath": "./.venv/bin/python3" sets VSCode to use the interpreter your virtual environment
"python.testing.pytestEnabled": true sets VSCode to use pytest and integrates the tests into the Test Explorer

Rust

Extensions:

rust-lang.rust-analyzer the only extension you need is the official LSP implementation for rust. This handles everything including testing

Settings:

"rust-analyzer.interpret.tests": true & "rust-analyzer.testExplorer": true enable integrating tests into VSCode
"[rust]": {"editor.rulers": [100]}: rust has a standard formatter which is as opinionated as black! You're going to want a ruler at 100 chars!

Development Environment

Quick-Start

Packages and tools (Dockerfile)

IDE extensions and settings (json)

TODO - .gitignore