beginnerintermediateadvanced

Python · Section 15

Python Packaging

Level: intermediate
Read: 130 min
Concepts: 10

Modern Python packaging — pip, venv, pyproject.toml, package layout, build systems, wheels, source distributions and editable installs, dependency pinning and ranges, semantic versioning, transitive dependencies and conflicts, private registries, and publishing.

This section

Packaging Your Python Code With pyproject.toml | Complete Code ConversationReal Python

What is true here

pyproject.toml is one static file for build config, metadata, and dependencies — read without executing any project code.
A wheel is a pre-built, ready-to-install ZIP; pip install -e . links source directly for live development instead.
Pin exact versions for reproducible deployments (pip freeze); declare ranges in a library so it composes with others.
Installing one package can pull in many transitive dependencies automatically — pip resolves the whole graph, not just what you typed.
MAJOR.MINOR.PATCH signals the risk of an upgrade before reading a changelog — a range only works if the package follows it.

What you will be able to do

Create an isolated venv and explain why installing globally causes cross-project version collisions
Write a pyproject.toml with the src/ layout and build a real wheel and sdist from it
Explain the tradeoff between an exact pin and a version range, and when each is appropriate
Read a MAJOR.MINOR.PATCH version bump and classify it as a patch, minor, or major change
Diagnose a real pip dependency conflict from its resolver error message
Describe the build → validate → upload workflow for publishing a package with twine
Explain why pyproject.toml replaced setup.py as the standard packaging file

Packaging fundamentals

pip and venv as the foundation, then pyproject.toml, build systems, wheels, and the source-vs-editable install distinction — verified by actually building and installing a real package.

pip and venv

corebeginner

pip installs, upgrades, and removes Python packages. venv creates an isolated environment with its own pip and package directory, so installing something for one project never affects another.

Think of it as

Without venv, every project shares ONE global set of installed packages — two projects needing different versions of the same library collide. venv gives each project its own private copy, like a separate toolbox per job instead of one shared one.

python

python -m venv .venv
.venv\Scripts\activate     # Windows
source .venv/bin/activate  # macOS/Linux

pip install requests

What we're doing: Create a real venv and inspect its structure, confirming it has its own isolated python and pip.

venv_structure.shpython

python -m venv .venv
ls .venv

# On Windows: Scripts/ holds python.exe, pip.exe, activate.bat
# On macOS/Linux: bin/ holds python, pip, activate

1: This creates the entire isolated environment — its own Python, its own pip, its own site-packages directory.
4: The directory name (Scripts on Windows, bin elsewhere) is the one real cross-platform difference in an otherwise identical structure.

Output

Include
Lib
Scripts
pyvenv.cfg

Why this works: A venv is a real, self-contained directory tree — Scripts/python.exe (or bin/python) is a separate interpreter installation pointing back at the system Python's standard library, with its OWN site-packages for installed packages, isolated from every other environment.

Installing packages globally instead of into a venv

Wrong

python

# no venv activated
pip install django==4.2
# now EVERY project on this machine sees Django 4.2,
# including one that needs Django 5.2

Better

python

python -m venv .venv
.venv\Scripts\activate
pip install django==4.2
# only THIS project's environment has Django 4.2

What you see: Two projects on the same machine cannot use different versions of the same package — installing one breaks the other.

Why: Without an active venv, pip install targets the single global Python installation, shared by every project on the machine. A venv gives each project its own isolated package set, so version requirements never collide across projects.

No venv vs. an activated venv

No venv active

+pip install targets the ONE global Python
+Every project on the machine shares it
+Two projects needing different versions collide

.venv activated

−Its own python and pip, first on PATH
−Its own isolated site-packages directory
−Disposable — delete and recreate, affects nothing else

pip — the commands worth knowing

pip — the commands worth knowing
Command	Effect
pip install <pkg>	installs the latest compatible version
pip install <pkg>==1.2.3	installs an exact pinned version
pip install -r requirements.txt	installs everything listed in a file
pip uninstall <pkg>	removes an installed package
pip list	shows every package installed in the active environment

Together

python

# python -m venv .venv
# .venv\Scripts\activate       (Windows)
# source .venv/bin/activate     (macOS/Linux)
# pip install requests
# pip list

Remember: venv gives each project its own isolated Python and package set; pip installs into whichever environment is currently active.

pyproject.toml and package layout

coreintermediate

pyproject.toml is one TOML file declaring a project's name, version, dependencies, and how to build it. The src/ layout puts package code inside a src/ directory, which is the modern recommended shape over placing it at the project root.

Think of it as

pyproject.toml is the project's single ID card — before it existed, a project needed setup.py (build config), setup.cfg (metadata), and sometimes requirements.txt, each in a different format. One file replaces that scatter.

python

myproject/
    pyproject.toml
    src/
        mypackage/
            __init__.py
    tests/
        test_mypackage.py

What we're doing: Write a real pyproject.toml with the src/ layout, then build and inspect the resulting package to confirm the structure actually works.

pyproject.tomlpython

[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"

[project]
name = "example-pkg"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = ["requests>=2.31,<3.0"]

2: requires lists what the BUILD process itself needs — setuptools here — separate from the package's own runtime dependencies.
6: name and version are the two fields every installer needs to identify this exact package.

Output

Successfully built example_pkg-0.1.0.tar.gz and example_pkg-0.1.0-py3-none-any.whl

Why this works: This exact pyproject.toml, with source under src/example_pkg/, was run through python -m build in this environment — it produced a real installable wheel and source distribution, proving the file format and layout actually work together, not just that they look plausible.

One file, four tables

[build-system] requires = ["setuptools>=68"] [project] name = "example-pkg" dependencies = ["requests>=2.31"] [project.optional-dependencies] dev = ["pytest>=8.0"]

[build-system] requires = ["setuptools>=68"]

build-system — what tool builds the package, and its own build-time deps

[project] name = "example-pkg" dependencies = ["requests>=2.31"]

project — name, version, runtime dependencies — the package's ID card

[project.optional-dependencies] dev = ["pytest>=8.0"]

optional-dependencies — extras like dev, installed with pip install pkg[dev]

Using a flat layout, where tests silently import unpackaged source

Wrong

python

myproject/
    pyproject.toml
    mypackage/           # flat -- sits right next to pyproject.toml
        __init__.py
    tests/

# from the project root:
python -c "import mypackage"   # WORKS, even with nothing installed

Better

python

myproject/
    pyproject.toml
    src/
        mypackage/        # not importable unless actually installed
            __init__.py
    tests/

# from the project root:
python -c "import mypackage"   # ModuleNotFoundError -- must pip install first

What you see: Tests pass locally against source that was never actually packaged correctly (missing files, a broken pyproject.toml) — the bug only surfaces after a real install, often in CI or production.

Why: A flat layout puts the package directory right next to pyproject.toml, so Python's current-directory import rule finds it directly — no install required. The src/ layout removes that shortcut, forcing every test run to use an actually-installed package.

pyproject.toml — the tables worth knowing

pyproject.toml — the tables worth knowing
Table	Holds
[build-system]	requires (build dependencies) and build-backend
[project]	name, version, description, dependencies, requires-python
[project.optional-dependencies]	extras like dev, test — installed with pip install pkg[dev]
[tool.*]	per-tool config — [tool.pytest.ini_options], [tool.ruff], etc.

Together

python

[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"

[project]
name = "example-pkg"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = ["requests>=2.31,<3.0"]

[project.optional-dependencies]
dev = ["pytest>=8.0"]

Remember: pyproject.toml is one file for build config, metadata, and dependencies; src/ layout avoids accidental imports.

Build systems and wheels

coreintermediate

A build backend turns your source code and pyproject.toml into installable artifacts. A wheel (.whl) is one such artifact — a zip file laid out exactly how pip expects, ready to install without running any code.

Think of it as

A wheel is a pre-assembled furniture kit — everything is already cut and ready, just needs to be placed. Installing from source (older sdist-only workflows) is like getting raw lumber and cutting it yourself every time — it works, but a wheel skips that repeated build step.

python

pip install build
python -m build
# produces:
#   dist/example_pkg-0.1.0-py3-none-any.whl
#   dist/example_pkg-0.1.0.tar.gz

What we're doing: Build a real wheel and unzip it to see exactly what pip actually installs.

build_and_inspect.shpython

python -m build
python -c "
import zipfile
with zipfile.ZipFile('dist/example_pkg-0.1.0-py3-none-any.whl') as z:
    for n in z.namelist():
        print(n)
"

1: This one command runs the build backend named in pyproject.toml and produces both artifacts.
4: A wheel is a real ZIP file — this unzips it directly rather than installing it, to see its exact contents.

Output

example_pkg/__init__.py
example_pkg-0.1.0.dist-info/METADATA
example_pkg-0.1.0.dist-info/WHEEL
example_pkg-0.1.0.dist-info/top_level.txt
example_pkg-0.1.0.dist-info/RECORD

Why this works: The wheel contains exactly the installed package layout — no pyproject.toml, no tests/, no build scripts. dist-info/METADATA carries the package's name/version/dependencies, and RECORD lists every installed file with its hash, which pip uses to verify and later uninstall cleanly.

Source to installable artifact

pyproject.toml + source

python -m build

runs the [build-system] backend

example_pkg-0.1.0-py3-none-any.whl

a ZIP, ready to install — no build step needed

example_pkg-0.1.0.tar.gz

source distribution

Uploading a stale dist/ after bumping the version but not rebuilding

Wrong

python

# edited pyproject.toml: version = "0.1.0" -> "0.2.0"
# forgot to rebuild
twine upload dist/*   # uploads the OLD 0.1.0 wheel, still sitting in dist/

Better

python

rm -rf dist/       # clear stale artifacts first
python -m build     # rebuild from the CURRENT pyproject.toml
twine upload dist/*

What you see: PyPI shows a new version number, but installing it gives the OLD code — the wheel's contents never actually changed.

Why: python -m build does not know a version bump happened unless it is re-run — dist/ can silently accumulate stale artifacts from earlier builds if never cleared, and twine upload dist/* uploads everything sitting there, not just the latest build.

Wheel filename — reading the tags

Wheel filename — reading the tags
Segment	Example	Means
name-version	example_pkg-0.1.0	the package and its version
python tag	py3	compatible with any Python 3.x
abi tag	none	no C-extension ABI dependency
platform tag	any	not tied to a specific OS/architecture

Together

python

# example_pkg-0.1.0-py3-none-any.whl
# name=example_pkg version=0.1.0 python=py3 abi=none platform=any
# a PURE PYTHON wheel -- installable anywhere

Remember: A build backend (named in [build-system]) turns source into artifacts; a wheel is a ready-to-install ZIP that needs no build step at install time.

Source distributions and editable installs

standardintermediate

A source distribution (sdist) ships the raw source, built on the installing machine. pip install -e . instead links a project's source directly into the environment, so edits take effect immediately, with no reinstall.

Think of it as

A wheel is a finished meal, ready to eat. An sdist is the recipe and raw ingredients, cooked fresh wherever it lands — needed when a pre-built wheel does not exist for the target platform. An editable install is not cooking at all — it is pointing straight at the working kitchen.

python

pip install -e .
# or, with dev extras:
pip install -e ".[dev]"

# any edit to the source is now live immediately
python -c "import mypackage; mypackage.some_function()"

What we're doing: Editable-install a real package, confirm it works, edit the source, and confirm the change is picked up with no reinstall.

editable_install_demo.shpython

pip install -e ./example_pkg

python -c "import example_pkg; print(example_pkg.greet('Ada'))"
# hello, Ada

# now edit src/example_pkg/__init__.py to return a different string

python -c "import example_pkg; print(example_pkg.greet('Ada'))"
# hi there, Ada!  -- picked up with NO reinstall

1: This links the package into the environment rather than copying it — the environment now points AT the source directory.
6: The source file changed on disk between these two runs — nothing else happened, no pip install rerun.

Output

hello, Ada
hi there, Ada!

Why this works: An editable install adds the project's src/ directory to the environment's import path directly, rather than copying files into site-packages — so any change to the source is visible the next time it is imported, exactly the workflow active development needs.

Remember: An sdist ships raw source, built wherever it lands; pip install -e . links source directly into the environment so edits take effect with no reinstall.

Dependencies and publishing

Pinning versus ranges, semantic versioning, transitive dependencies and the conflicts they can cause, private registries, and the publish workflow.

Dependencies, pinning, and ranges

coreintermediate

A dependency is another package your code needs. Pinning locks to one exact version; a range accepts any version in a window — pinning is reproducible but can go stale, ranges stay current but can break unexpectedly.

Think of it as

A pin is a photograph — exactly what worked, frozen at one instant. A range is a job description — "someone compatible with these requirements," which could be filled differently over time as new compatible versions are released.

python

# pyproject.toml
dependencies = ["requests>=2.31,<3.0"]

# a pinned requirements.txt, generated from what's actually installed
pip freeze > requirements.txt

What we're doing: Show pip freeze producing exact pins from a range-declared dependency, and confirm the pin is reproducible.

pinning_demo.shpython

# pyproject.toml declares: dependencies = ["requests>=2.31,<3.0"]
pip install -e .
pip freeze
# certifi==2026.7.22
# charset-normalizer==3.5.1
# idna==3.19
# requests==2.34.2
# urllib3==2.7.0

1: The declared dependency is a RANGE — any 2.x version of requests from 2.31 up is acceptable.
4: pip freeze converts every currently-installed version, including transitive ones, into an EXACT pin.

Output

certifi==2026.7.22
charset-normalizer==3.5.1
idna==3.19
requests==2.34.2
urllib3==2.7.0

Why this works: requests>=2.31,<3.0 let pip pick whatever the latest compatible version was at install time (2.34.2 here) — pip freeze captures that specific resolution as exact pins, which is what a requirements.txt lock file records for reproducible installs later.

Pinning every dependency in a library's own pyproject.toml

Wrong

python

# a LIBRARY's pyproject.toml
dependencies = [
    "requests==2.31.0",   # exact pin
    "pydantic==2.5.0",    # exact pin
]

Better

python

# a LIBRARY's pyproject.toml
dependencies = [
    "requests>=2.31,<3.0",
    "pydantic>=2.5,<3.0",
]

What you see: Two libraries pinned to different exact versions of the same dependency cannot both be installed in the same project — pip reports a resolution conflict.

Why: An exact pin in a library forces every consumer of that library to use that exact version too. A range lets pip pick a version compatible with everything else in the project — pinning belongs in a deployed application's lock file, not a library's own declared dependencies.

Reading a version specifier

dependencies = ["requests>=2.31,<3.0", "pydantic~=2.5"]

>=2.31,<3.0

a range — 2.31 or newer, but never 3.0 — pip picks the latest that fits

~=2.5

compatible release — shorthand for >=2.5,<3.0 — allows minor/patch, not major

Version specifiers

Version specifiers
Specifier	Means
==2.31.0	exactly this version
>=2.31,<3.0	a range — 2.31 or newer, but not 3.0
~=2.31	compatible release — >=2.31, <3.0
!=2.31.5	anything except this specific version (a known-bad release)

Together

python

dependencies = [
    "requests>=2.31,<3.0",
    "pydantic~=2.5",
    "urllib3!=2.0.7",
]

Remember: Pin exact versions for reproducible deployments (pip freeze); declare ranges in a library so it composes with other packages' requirements.

Semantic versioning

standardintermediate

Semantic versioning (semver) numbers releases as MAJOR.MINOR.PATCH. Bump PATCH for a bug fix, MINOR for a backward-compatible new feature, and MAJOR for a breaking change — the numbers themselves signal what kind of change happened.

Think of it as

Semver turns a version number into a promise. requests>=2.31,<3.0 is trusting that promise: any 2.x release only adds things or fixes bugs, never breaks existing code — which is exactly why a range specifier like that is safe to declare at all.

python

2.31.0
│  │  └── PATCH: bug fix, no new features, nothing breaks
│  └───── MINOR: new feature, still backward-compatible
└──────── MAJOR: breaking change

What we're doing: Read three real version bumps and classify each by what semver says it means.

semver_reading.pypython

changes = [
    ("2.31.0", "2.31.1", "fixed a bug in header parsing"),
    ("2.31.1", "2.32.0", "added a new optional timeout parameter"),
    ("2.32.0", "3.0.0", "removed the deprecated Session.proxies attribute"),
]

for old, new, description in changes:
    old_parts = [int(p) for p in old.split(".")]
    new_parts = [int(p) for p in new.split(".")]
    if new_parts[0] != old_parts[0]:
        kind = "MAJOR (breaking)"
    elif new_parts[1] != old_parts[1]:
        kind = "MINOR (new feature)"
    else:
        kind = "PATCH (bug fix)"
    print(f"{old} -> {new}: {kind} -- {description}")

2: Only the last number changed — a PATCH bump, safe to accept automatically under >=2.31,<3.0.
4: The middle number changed — a MINOR bump, still backward-compatible, still safe under the same range.

Output

2.31.0 -> 2.31.1: PATCH (bug fix) -- fixed a bug in header parsing
2.31.1 -> 2.32.0: MINOR (new feature) -- added a new optional timeout parameter
2.32.0 -> 3.0.0: MAJOR (breaking) -- removed the deprecated Session.proxies attribute

Why this works: The version number alone tells you the RISK of upgrading before reading any changelog — a range like >=2.31,<3.0 accepts the first two rows automatically but excludes the third, because crossing a MAJOR boundary is exactly the change a range is meant to guard against.

Remember: MAJOR.MINOR.PATCH — PATCH is a bug fix, MINOR is a compatible new feature, MAJOR is a breaking change.

Transitive dependencies and dependency conflicts

coreintermediate

A transitive dependency is something your dependency depends on, installed automatically even though you never listed it. A dependency conflict happens when two requirements cannot both be satisfied at the same time.

Think of it as

Declaring one dependency is like inviting one guest to a party — but that guest brings their own friends (their own dependencies), and those friends bring theirs. A conflict is two invitations that each require the room a different, incompatible temperature.

python

pip install pkg-a "urllib3>=2.7"
# ERROR: Cannot install pkg-a and urllib3>=2.7 because
# these package versions have conflicting dependencies.

What we're doing: Trigger a real dependency conflict between two requirements naming incompatible ranges of the same transitive dependency.

conflict_demo.shpython

# pkg-a's pyproject.toml declares: dependencies = ["urllib3<2.1,>=2.0"]
pip install ./pkg-a "urllib3>=2.7"

1: This asks pip to satisfy TWO requirements on urllib3 at once — pkg-a's narrow range, and a direct request for 2.7+.

Output

ERROR: Cannot install pkg-a==0.1.0 and urllib3>=2.7 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested urllib3>=2.7
    pkg-a 0.1.0 depends on urllib3<2.1 and >=2.0

Why this works: pip's resolver checks every requirement in the whole dependency graph, not just the ones typed directly on the command line — pkg-a's own declared range (urllib3<2.1) and the explicit request (urllib3>=2.7) have no version in common, so pip refuses to install anything rather than silently pick one.

Assuming pip installed something compatible just because it did not error

Wrong

python

pip install package-a package-b
# no error shown -- assumed everything is fine
# but package-b silently got downgraded to satisfy package-a

Better

python

pip install package-a package-b
pip check   # explicitly verifies every installed package's
            # requirements are actually satisfied

What you see: No install-time error, but package-b later crashes at runtime, because the version pip actually installed does not have the feature the code depends on.

Why: Older pip resolvers could silently install an incompatible mix in some cases; even with a modern resolver, an install can succeed by picking versions you did not expect. pip check explicitly re-verifies every installed package's declared requirements against what is actually present.

One direct dependency pulls in four more

Declared

requests>=2.31,<3.0

the only line you actually wrote

Installed (pip freeze)

requests==2.34.2

direct

urllib3, idna, certifi

requests' own transitive deps

charset-normalizer

transitive

Direct vs. transitive

Direct vs. transitive
Kind	Example
Direct	requests, listed in your own pyproject.toml
Transitive	urllib3, idna, certifi, charset-normalizer — requests' own dependencies

Together

python

# dependencies = ["requests>=2.31,<3.0"]
# pip install -e .
# pip freeze shows FIVE packages, not one:
#   certifi, charset-normalizer, idna, requests, urllib3

Remember: A transitive dependency is installed automatically by something you depend on; pip reports a real conflict explicitly.

Private packages and internal package registries

standardintermediate

A private package is code shared across an organization's projects, not published publicly. An internal package registry hosts these, and pip can install from it exactly like PyPI, just pointed at a different URL.

Think of it as

Public PyPI is a public library anyone can borrow from. An internal registry is a company's own private shelf — same borrowing mechanism (pip install), just a different, access-controlled building.

python

pip install --index-url https://internal.example.com/simple/ mypackage

# or set it as the default for this environment:
pip config set global.index-url https://internal.example.com/simple/
pip install mypackage

What we're doing: Show the pip.conf shape that points every install in an environment at an internal registry by default, and the equivalent one-off command form.

pip.confpython

[global]
index-url = https://internal.example.com/simple/
extra-index-url = https://pypi.org/simple/

# equivalent, one-off:
# pip install --index-url https://internal.example.com/simple/ \
#             --extra-index-url https://pypi.org/simple/ mypackage

2: index-url replaces the default (PyPI) entirely — without extra-index-url, only the internal registry would be searched.
3: extra-index-url adds PyPI back as a second source, so public packages remain installable alongside private ones.

Why this works: pip.conf sets a persistent default for every pip install run in that environment, instead of retyping --index-url on every command — this is how a company-wide CI pipeline or developer machine is typically configured to reach an internal registry transparently.

Remember: --index-url points pip at a private registry instead of PyPI; --extra-index-url searches both, so public and private packages both remain installable.

See also: publishing packages · pip and venv

Publishing packages

standardintermediate

Publishing means uploading a built wheel and sdist to a package index (PyPI, or an internal one) so others can pip install it. twine is the standard tool for the upload step, run after python -m build produces the artifacts.

Think of it as

Building (python -m build) is packing a shipment into a box. Publishing (twine upload) is actually sending that box to the warehouse (PyPI) where pip install can find and retrieve it.

python

pip install build twine
python -m build
twine check dist/*
twine upload dist/*   # prompts for a PyPI API token

The publish workflow, in order

The publish workflow, in order
Step	Command
1. Build	python -m build
2. Validate	twine check dist/*
3. Upload	twine upload dist/*
4. Verify	pip install yourpackage (in a fresh venv)

Together

python

python -m build
twine check dist/*
twine upload dist/*
# published -- now installable with: pip install yourpackage

Remember: Build, then twine check validates, then twine upload publishes — a published version number can never be reused.

Why pyproject.toml over legacy setup.py

standardintermediate

setup.py is executable Python code pip used to run to learn a package's metadata — a risk, since installing meant running the author's arbitrary code first. pyproject.toml is static data instead, read without executing anything.

Think of it as

setup.py is a recipe written as a live cooking demonstration — you cannot know what it will do until you actually run it. pyproject.toml is the recipe written down as plain text — readable, and safely inspectable, before anything is executed.

python

# OLD: setup.py -- executable code
from setuptools import setup
setup(name="mypackage", version="0.1.0", install_requires=["requests"])

# NEW: pyproject.toml -- static data, no code execution needed
[project]
name = "mypackage"
version = "0.1.0"
dependencies = ["requests"]

Remember: setup.py is executable code pip must run to read metadata; pyproject.toml is static data, parsed without running any code.