Python for AI Engineers

◆ Chapter 02

Environment & tooling: the Python project skeleton (npm → pip/uv mental map)

Virtual envs, pip and uv, pyproject.toml — the Python project skeleton mapped straight onto the npm workflow you already know.

~1,929 words · chapter 2 of 15

A Python project's skeleton looks unfamiliar for one structural reason: Python has no node_modules. There's no per-project dependency folder by default — packages install into a shared, interpreter-wide location. Understanding how the ecosystem works around that fact is 80% of "setting up Python correctly."

Interpreter versions

Unlike Node where node is one binary you upgrade, macOS/Linux often have several Pythons: a system python3, a Homebrew one, others. You pin the version you want per project. pyenv is the classic tool for installing and switching Python versions (analogous to nvm), though the modern uv (below) can also download and manage interpreters for you. Rule of thumb: never build on the OS's system Python — install your own 3.12 or 3.13 and target that.

python3 --version        # => Python 3.12.7

Virtual environments — why they exist

Here's the trap. By default, pip install requests installs requests into the interpreter's global site-packages, shared by every project using that interpreter. Two projects needing different versions of the same library collide. There's no node_modules to isolate them.

A virtual environment solves this: a lightweight, per-project folder holding its own copy of the interpreter link and its own site-packages. It's Python's answer to node_modules, but you must create and activate it explicitly.

python -m venv .venv      # create an env in the .venv/ folder
source .venv/bin/activate # activate it (macOS/Linux)
# .venv\Scripts\activate  # (Windows)

"Activated" means your shell's PATH now points python and pip at .venv/'s copies. Your prompt usually shows (.venv). Now pip install writes into .venv/, isolated from every other project. deactivate reverses it.

JS → Python: node_modules is automatic, local, and per-project — you never think about it. Python's .venv is manual, local, per-project, but you must create and activate it, and forgetting to means you're accidentally polluting a global or system Python. The single most common beginner Python mistake is pip install-ing into the global interpreter because no venv was active. Always check for (.venv) in your prompt.

pip and requirements.txt

pip is the classic installer (npm's rough analogue). The traditional way to record dependencies is a plain-text requirements.txt:

anthropic==0.40.0
python-dotenv>=1.0
pip install -r requirements.txt   # install everything listed
pip freeze > requirements.txt     # snapshot current exact versions

requirements.txt is just a list — it isn't a real manifest with metadata, scripts, or a lockfile. That's a weakness modern tooling fixes.

The modern stack: uv + pyproject.toml

pyproject.toml (standardised by PEP 621) is Python's real package.json — one declarative file for project metadata, dependencies, and tool config. uv is the fast, modern package/project manager (written in Rust) that has become the 2026 default: it resolves and installs dependencies dramatically faster than pip, manages virtual environments and even Python versions, and writes a lockfile.

A minimal pyproject.toml:

[project]
name = "agent-demo"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
    "anthropic>=0.40",
    "httpx>=0.27",
]

[dependency-groups]              # dev-only deps (like devDependencies)
dev = ["ruff>=0.6", "mypy>=1.11", "pytest>=8"]

[tool.ruff]                      # tool config lives here too
line-length = 100

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Core uv commands:

uv venv                       # create .venv (auto-detects/downloads Python)
uv add anthropic              # add a dep: installs + updates pyproject.toml + uv.lock
uv pip install anthropic      # pip-compatible install into the active env
uv sync                       # install exactly what uv.lock pins (reproducible)
uv run python main.py         # run a command inside the project env (no activate needed)
uvx ruff check .              # run a tool in a throwaway env, no install

uv.lock is the auto-generated lockfile (like package-lock.json) recording the exact resolved versions of the whole dependency tree for byte-reproducible installs. uv run is a killer feature: it runs your command in the project's env without needing manual activation — great for scripts and CI. uvx (alias for uv tool run) is the npx equivalent: run a CLI tool one-off without permanently installing it.

JS → Python:

JavaScript / Node Python (modern) Role
npm / pnpm pip / uv install packages
package.json pyproject.toml project manifest + config
node_modules/ .venv/ installed deps (isolated)
npm install uv sync / uv add install / add deps
package-lock.json uv.lock exact-version lockfile
npx <tool> uvx <tool> run a tool without installing
dependencies / devDependencies dependencies / [dependency-groups] dev prod vs dev deps
tsc (types) mypy / pyright static type checking
eslint + prettier ruff (lint + format) lint + format

Installing and importing packages, python -m

Installing a package makes it importable by its import name, which may differ from its install name — a real gotcha. You uv add python-dotenv but import dotenv; you uv add beautifulsoup4 but import bs4. There's no guaranteed 1:1 mapping like npm's.

from dotenv import load_dotenv   # installed as "python-dotenv"
import anthropic                  # installed as "anthropic" (matches, this time)

python -m module runs an installed module as a script using the current environment's interpreter — this is why you see python -m venv, python -m pytest, python -m http.server. It's the safe way to invoke a tool because it guarantees the active interpreter runs it, sidestepping PATH ambiguity.

Project layout: the src/ layout, packages, imports

A well-structured Python project (the recommended src/ layout):

agent-demo/
├── pyproject.toml
├── uv.lock
├── README.md
├── .venv/                      # git-ignored
├── src/
│   └── agent_demo/             # the package (snake_case, importable name)
│       ├── __init__.py         # marks this dir as a package
│       ├── client.py           # a module
│       └── tools/              # a sub-package
│           ├── __init__.py
│           └── search.py
└── tests/
    └── test_client.py
  • A module is a single .py file. A package is a directory of modules.
  • __init__.py marks a directory as a package and runs when the package is first imported (often empty; sometimes it re-exports things for a cleaner public API). Modern Python also supports "namespace packages" without it, but explicit __init__.py is clearer and expected in most repos.
  • src/ layout puts your code one level down, so tests must import it the same way an installed user would — catching "works on my machine because the current directory was on the path" bugs. It's the mark of a serious project.

Imports — absolute vs relative:

# absolute (preferred — clear, unambiguous, works everywhere):
from agent_demo.tools.search import web_search

# relative (from inside the package; the dot = "this package"):
from .tools.search import web_search      # . = current package
from ..client import Client               # .. = parent package

JS → Python: Python has no import ./file with a path and extension. You import by dotted module path (agent_demo.tools.search), resolved against the interpreter's search path — not by filesystem path. There's no index.js; __init__.py is the closest analogue but its purpose is "this folder is a package," not "this is the entry point." And unlike Node's implicit relative resolution, absolute imports are the community-preferred default.

The quality toolchain

ruff is the ESLint + Prettier of Python, collapsed into one blazing-fast (Rust) tool. It lints and formats:

uvx ruff check .        # lint (find problems)
uvx ruff check --fix .  # lint and auto-fix what it can
uvx ruff format .       # format (the Black-compatible formatter)

It has essentially replaced the older trio of flake8 (lint) + black (format) + isort (import sorting) that you'll still see referenced in older repos.

mypy and pyright are the static type checkers — Python's tsc. They read your type hints and flag mismatches without running the code:

def greet(name: str) -> str:
    return "hi " + name

greet(42)   # mypy error: Argument 1 has incompatible type "int"; expected "str"

Type hints are optional and not enforced at runtime by default, but in serious GenAI codebases they're near-universal — LLM SDKs, Pydantic (used everywhere for validating model I/O and tool schemas), and FastAPI all lean heavily on them. pyright (from Microsoft, powers the VS Code Pylance extension) is stricter and faster on large codebases; mypy is the long-standing reference. Either is a strong signal of a disciplined project.

Try it: uv init demo && cd demo && uv add anthropic && uv run python -c "import anthropic; print(anthropic.__version__)". In four commands uv created a project, a venv, a pyproject.toml, a lockfile, installed a dep, and ran code in the isolated env — no manual activation.

Reading & judging — spotting a well-structured Python repo at a glance:

Signal Healthy Red flag
Manifest pyproject.toml with [project] deps only a bare requirements.txt, or worse, no dep file at all
Version pinning a lockfile (uv.lock/poetry.lock) or pinned == versions everything unpinned (anthropic, no version) — non-reproducible
Isolation .venv/ git-ignored, lockfile committed .venv/ committed to git, or deps installed globally
Layout src/ layout, real package with __init__.py, tests/ one giant main.py, or all logic in .ipynb notebooks
Tooling ruff + mypy/pyright config in pyproject.toml, CI running them no linter/type-checker, camelCase names, mixed styles
Secrets .env in .gitignore, config via env vars API keys hardcoded in source (an instant fail)

The fastest 10-second read: open pyproject.toml. If it exists, lists pinned dependencies, and has [tool.ruff]/[tool.mypy] sections, you're almost certainly looking at code written by someone who knows what they're doing. A repo with a stray requirements.txt, no lockfile, unpinned versions, and all its logic in notebooks is one to review with both eyes open — the code might be clever, but the engineering around it isn't there yet, which in GenAI work (where reproducibility and dependency drift break everything) is exactly what you're being asked to judge.

The notebook workflow — your experiment sandbox

Chapter 1 explained what a Jupyter notebook is and its hidden-state trap. Here is how to actually run one — because it's where you'll prototype almost every model call before it hardens into a .py file.

pip install notebook      # or: uv add --dev notebook
python -m notebook        # launches the notebook UI in your browser

A notebook is a column of cells. Press Ctrl+Enter to run a code cell (or Shift+Enter to run it and drop to the next); its output appears directly beneath. A few keyboard shortcuts do most of the work: B adds a cell below, A above, and M turns a cell into a markdown cell so you can write prose notes between experiments. "Run All" executes the whole notebook top to bottom.

Why AI people live here: you load a 5 GB model or a big dataset once, then iterate on prompts and plots cell by cell for an hour — the state stays live in memory, so you never pay the load cost twice.

Try it: pip install notebook && python -m notebook, make a code cell with print(2 ** 10) and run it with Ctrl+Enter, then add a markdown cell above it (A, then M) and write a heading. Finally use Kernel → Restart & Run All and confirm you get the same output — the reproducibility check from Chapter 1, in your fingers.