beginnerintermediateadvanced

Python · Python Packaging

Python Packaging

pyproject.toml, building real wheels, dependency ranges and conflicts, and the publish workflow — verified against a real built package, not just docs

pyproject.tomlone static file for build config, metadata, and dependencies
Wheelsa pre-built, ready-to-install ZIP — no build step at install time
Ranges vs. pinsflexible-but-current vs. exact-but-reproducible
Transitive depsinstalling one package can pull in many more automatically

1.pip and venv

pip installs, upgrades, and removes Python packages. venv creates an isolated environment with its own pip and package directory, so installing something for one project never affects another.

Key features

python -m venv .venv creates an isolated environment in the .venv directory
Activating a venv puts its own python/pip first on PATH (.venv/bin or .venv\Scripts)
pip install <package> installs into whichever environment is currently active
pip list / pip freeze show what is installed in the active environment
A venv is disposable — deleting and recreating it affects nothing outside it

pip — the commands worth knowing

pip — the commands worth knowing
Command	Effect
pip install <pkg>	installs the latest compatible version
pip install <pkg>==1.2.3	installs an exact pinned version
pip install -r requirements.txt	installs everything listed in a file
pip uninstall <pkg>	removes an installed package
pip list	shows every package installed in the active environment

No venv vs. an activated venv

No venv active

pip install targets the ONE global Python
Every project on the machine shares it
Two projects needing different versions collide

.venv activated

Its own python and pip, first on PATH
Its own isolated site-packages directory
Disposable — delete and recreate, affects nothing else

Command

python

python -m venv .venv
.venv\Scripts\activate     # Windows
source .venv/bin/activate  # macOS/Linux

pip install requests

venv gives each project its own isolated Python and package set; pip installs into whichever environment is currently active.

2.pyproject.toml and package layout

pyproject.toml is one TOML file declaring a project's name, version, dependencies, and how to build it. The src/ layout puts package code inside a src/ directory, which is the modern recommended shape over placing it at the project root.

Key features

[build-system] declares what tool builds the package (setuptools, hatchling, poetry-core)
[project] holds name, version, dependencies, and requires-python
The src/ layout prevents accidentally importing uninstalled source over a real install
A flat layout still works, but src/ is the modern recommendation
pyproject.toml is TOML, not JSON or YAML — a different, simpler syntax for config files

pyproject.toml — the tables worth knowing

pyproject.toml — the tables worth knowing
Table	Holds
[build-system]	requires (build dependencies) and build-backend
[project]	name, version, description, dependencies, requires-python
[project.optional-dependencies]	extras like dev, test — installed with pip install pkg[dev]
[tool.*]	per-tool config — [tool.pytest.ini_options], [tool.ruff], etc.

One file, four tables

[build-system] requires = ["setuptools>=68"] [project] name = "example-pkg" dependencies = ["requests>=2.31"] [project.optional-dependencies] dev = ["pytest>=8.0"]

[build-system] requires = ["setuptools>=68"]

build-system — what tool builds the package, and its own build-time deps

[project] name = "example-pkg" dependencies = ["requests>=2.31"]

project — name, version, runtime dependencies — the package's ID card

[project.optional-dependencies] dev = ["pytest>=8.0"]

optional-dependencies — extras like dev, installed with pip install pkg[dev]

Command

python

myproject/
    pyproject.toml
    src/
        mypackage/
            __init__.py
    tests/
        test_mypackage.py

pyproject.toml is one file for build config, metadata, and dependencies; src/ layout avoids accidental imports.

3.Build systems and wheels

A build backend turns your source code and pyproject.toml into installable artifacts. A wheel (.whl) is one such artifact — a zip file laid out exactly how pip expects, ready to install without running any code.

Key features

[build-system] in pyproject.toml names the backend (setuptools, hatchling, poetry-core)
python -m build produces both a wheel (.whl) and a source distribution (.tar.gz)
A wheel is literally a ZIP file — unzipping shows the package plus a *.dist-info dir
pip prefers a wheel when available, since it needs no build step at install time
A wheel filename encodes compatibility: name-version-pythontag-abitag-platformtag.whl

Wheel filename — reading the tags

Wheel filename — reading the tags
Segment	Example	Means
name-version	example_pkg-0.1.0	the package and its version
python tag	py3	compatible with any Python 3.x
abi tag	none	no C-extension ABI dependency
platform tag	any	not tied to a specific OS/architecture

Source to installable artifact

pyproject.toml + source

python -m build

runs the [build-system] backend

example_pkg-0.1.0-py3-none-any.whl

a ZIP, ready to install — no build step needed

example_pkg-0.1.0.tar.gz

source distribution

Command

python

pip install build
python -m build
# produces:
#   dist/example_pkg-0.1.0-py3-none-any.whl
#   dist/example_pkg-0.1.0.tar.gz

A build backend (named in [build-system]) turns source into artifacts; a wheel is a ready-to-install ZIP that needs no build step at install time.

4.Source distributions and editable installs

A source distribution (sdist) ships the raw source, built on the installing machine. pip install -e . instead links a project's source directly into the environment, so edits take effect immediately, with no reinstall.

Key features

python -m build also produces an sdist (.tar.gz) alongside the wheel, by default
An sdist needs a build step at install time; a wheel does not
pip install -e . installs a project in "editable" mode from the current directory
Editing source after an editable install is reflected immediately — no reinstall needed
Editable installs are for active development; a release ships a wheel/sdist instead

Command

python

pip install -e .
# or, with dev extras:
pip install -e ".[dev]"

# any edit to the source is now live immediately
python -c "import mypackage; mypackage.some_function()"

An sdist ships raw source, built wherever it lands; pip install -e . links source directly into the environment so edits take effect with no reinstall.

5.Dependencies, pinning, and ranges

A dependency is another package your code needs. Pinning locks to one exact version; a range accepts any version in a window — pinning is reproducible but can go stale, ranges stay current but can break unexpectedly.

Key features

dependencies = ["requests>=2.31,<3.0"] in pyproject.toml declares a version range
== pins an exact version; >= / <= / != / ~= express ranges and exclusions
~=2.31 ("compatible release") means >=2.31,<3.0 — allow patch/minor updates, not major
pip freeze > requirements.txt captures every currently-installed version as an exact pin
Applications usually pin exactly; libraries declare ranges, so they compose with others

Version specifiers

Version specifiers
Specifier	Means
==2.31.0	exactly this version
>=2.31,<3.0	a range — 2.31 or newer, but not 3.0
~=2.31	compatible release — >=2.31, <3.0
!=2.31.5	anything except this specific version (a known-bad release)

Reading a version specifier

dependencies = ["requests>=2.31,<3.0", "pydantic~=2.5"]

>=2.31,<3.0

a range — 2.31 or newer, but never 3.0 — pip picks the latest that fits

~=2.5

compatible release — shorthand for >=2.5,<3.0 — allows minor/patch, not major

Command

python

# pyproject.toml
dependencies = ["requests>=2.31,<3.0"]

# a pinned requirements.txt, generated from what's actually installed
pip freeze > requirements.txt

Pin exact versions for reproducible deployments (pip freeze); declare ranges in a library so it composes with other packages' requirements.

6.Semantic versioning

Semantic versioning (semver) numbers releases as MAJOR.MINOR.PATCH. Bump PATCH for a bug fix, MINOR for a backward-compatible new feature, and MAJOR for a breaking change — the numbers themselves signal what kind of change happened.

Key features

MAJOR.MINOR.PATCH — e.g. 2.31.0 is major 2, minor 31, patch 0
PATCH increments for backward-compatible bug fixes
MINOR increments for backward-compatible new functionality
MAJOR increments for any breaking (backward-incompatible) change
A range like >=2.31,<3.0 only makes sense if the library actually follows semver

Command

python

2.31.0
│  │  └── PATCH: bug fix, no new features, nothing breaks
│  └───── MINOR: new feature, still backward-compatible
└──────── MAJOR: breaking change

MAJOR.MINOR.PATCH — PATCH is a bug fix, MINOR is a compatible new feature, MAJOR is a breaking change.

7.Transitive dependencies and dependency conflicts

A transitive dependency is something your dependency depends on, installed automatically even though you never listed it. A dependency conflict happens when two requirements cannot both be satisfied at the same time.

Key features

Installing one package can pull in many more automatically — its transitive dependencies
pip resolves the whole dependency graph, not just the packages you directly listed
A conflict happens when two requirements name incompatible ranges for the same package
pip's resolver reports the conflict explicitly, not silently installing something broken
Widening an overly narrow range is usually how a real conflict gets fixed

Direct vs. transitive

Direct vs. transitive
Kind	Example
Direct	requests, listed in your own pyproject.toml
Transitive	urllib3, idna, certifi, charset-normalizer — requests' own dependencies

One direct dependency pulls in four more

Declared

requests>=2.31,<3.0

the only line you actually wrote

Installed (pip freeze)

requests==2.34.2

direct

urllib3, idna, certifi

requests' own transitive deps

charset-normalizer

transitive

Command

python

pip install pkg-a "urllib3>=2.7"
# ERROR: Cannot install pkg-a and urllib3>=2.7 because
# these package versions have conflicting dependencies.

A transitive dependency is installed automatically by something you depend on; pip reports a real conflict explicitly.

8.Private packages and internal package registries

A private package is code shared across an organization's projects, not published publicly. An internal package registry hosts these, and pip can install from it exactly like PyPI, just pointed at a different URL.

Key features

pip install --index-url <private-url> mypackage installs from a private index
--extra-index-url adds a second index to search, alongside the default (public PyPI)
A private index typically requires authentication — a token or credentials in the URL
Common registries: Artifactory, AWS CodeArtifact, Google Artifact Registry, devpi
pip.conf can set a default index-url so every install in an environment uses it

Command

python

pip install --index-url https://internal.example.com/simple/ mypackage

# or set it as the default for this environment:
pip config set global.index-url https://internal.example.com/simple/
pip install mypackage

--index-url points pip at a private registry instead of PyPI; --extra-index-url searches both, so public and private packages both remain installable.

9.Publishing packages

Publishing means uploading a built wheel and sdist to a package index (PyPI, or an internal one) so others can pip install it. twine is the standard tool for the upload step, run after python -m build produces the artifacts.

Key features

python -m build produces dist/*.whl and dist/*.tar.gz — the artifacts to publish
twine upload dist/* uploads to PyPI, or --repository-url for a private index
A PyPI API token (not a password) is the recommended way to authenticate an upload
A version number, once published to PyPI, can never be reused — even if deleted
twine check dist/* validates the artifacts before uploading, catching problems early

The publish workflow, in order

The publish workflow, in order
Step	Command
1. Build	python -m build
2. Validate	twine check dist/*
3. Upload	twine upload dist/*
4. Verify	pip install yourpackage (in a fresh venv)

Command

python

pip install build twine
python -m build
twine check dist/*
twine upload dist/*   # prompts for a PyPI API token

Build, then twine check validates, then twine upload publishes — a published version number can never be reused.

10.Why pyproject.toml over legacy setup.py

setup.py is executable Python code pip used to run to learn a package's metadata — a risk, since installing meant running the author's arbitrary code first. pyproject.toml is static data instead, read without executing anything.

Key features

setup.py is Python code, run by pip to get metadata — arbitrary code runs at build time
pyproject.toml is static TOML data, parsed without executing any of the project's own code
Before pyproject.toml, metadata was scattered across setup.py, setup.cfg, requirements.txt
pyproject.toml is standardized (PEP 517/518/621) — any build backend reads the same format
Legacy setup.py-only projects still work, but new projects should use pyproject.toml

Command

python

# OLD: setup.py -- executable code
from setuptools import setup
setup(name="mypackage", version="0.1.0", install_requires=["requests"])

# NEW: pyproject.toml -- static data, no code execution needed
[project]
name = "mypackage"
version = "0.1.0"
dependencies = ["requests"]

setup.py is executable code pip must run to read metadata; pyproject.toml is static data, parsed without running any code.

Spotted a mistake on this page? Tell us.

Read Sheet