The data & HTTP layer you must be able to read (numpy, pandas, httpx)

You will not write much numpy or pandas as a GenAI engineer. But you will read a lot of it — every RAG pipeline, every eval harness, every embeddings notebook is soaked in these two libraries. And every LLM SDK is, underneath, an HTTP client. This chapter gives you read-level fluency: enough to understand what AI code is doing to your vectors and your data, and enough to spot when it's doing it badly.

numpy: the array that exists because Python lists are slow

Python's built-in list is a box of pointers to arbitrary objects. Doing math on a million floats stored in a list means a million separate Python operations — slow, and heavy on memory. numpy's ndarray is a flat block of raw numbers of one fixed type, laid out contiguously in memory, with the math implemented in C. This is why embeddings, tensors, and similarity scores are numpy arrays: the speed is not optional at scale.

import numpy as np

# Create an array from a list. dtype is inferred (float64 here).
v = np.array([0.1, 0.2, 0.3, 0.4])
print(v.shape)   # => (4,)      one dimension, 4 elements
print(v.dtype)   # => float64   every element is the same type

# A 2-D array (a matrix) — e.g. 3 embeddings, each of length 4
m = np.array([[1.0, 0.0, 0.0, 0.0],
              [0.0, 1.0, 0.0, 0.0],
              [0.5, 0.5, 0.0, 0.0]])
print(m.shape)   # => (3, 4)    3 rows, 4 columns

shape (the size along each dimension) and dtype (the element type) are the two attributes you check constantly. A shape mismatch is the single most common bug in AI glue code.

Vectorized operations are the whole point. Instead of looping, you apply an operation to the entire array at once and numpy runs the loop in C:

a = np.array([1.0, 2.0, 3.0])
b = np.array([10.0, 20.0, 30.0])

print(a + b)        # => [11. 22. 33.]   element-wise, no loop
print(a * 2)        # => [2. 4. 6.]      scalar applied to every element
print(a @ b)        # => 140.0           dot product (1*10 + 2*20 + 3*30)

The pure-Python equivalent of a + b — [x + y for x, y in zip(a, b)] — gives the same answer but is often 10–100× slower on real-sized arrays. When you see a Python for loop iterating over array elements to do arithmetic, that is usually a performance bug (more on this below).

Broadcasting is the rule that lets arrays of different shapes combine. numpy stretches the smaller one to fit, without copying data. a * 2 above already used it — the scalar 2 was broadcast across all elements. It also works across dimensions: adding a shape-(4,) row to a shape-(3, 4) matrix applies that row to all 3 rows. You mostly need to recognize broadcasting, not author it — when a (3, 4) and a (4,) combine cleanly, that's broadcasting.

Indexing and slicing look like lists but extend to multiple dimensions with a comma:

print(m[0])        # => [1. 0. 0. 0.]   first row
print(m[0, 2])     # => 0.0             row 0, column 2
print(m[:, 1])     # => [0. 1. 0.5]     every row, column 1 (a whole column)
print(m[0:2])      # => first two rows

Cosine similarity — the workhorse of semantic search — is two lines. It measures the angle between two embedding vectors: 1.0 means identical direction, 0.0 means unrelated.

def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
    # dot product divided by the product of the vectors' lengths (norms)
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))

query = np.array([0.9, 0.1, 0.0])
doc   = np.array([0.8, 0.2, 0.0])
print(cosine_similarity(query, doc))   # => 0.98...  very similar

When a retrieval system "finds the most relevant chunk," this — or its batched matrix form docs @ query — is what's running.

JS → Python: A numpy array is what a JS Float64Array/typed array wishes it were: fixed dtype, contiguous, but with math operators and slicing built in. a + b in JS concatenates or coerces; in numpy it's element-wise math. There's no native JS equivalent of broadcasting or a @ b — you'd reach for a library like ml-matrix. Slicing m[:, 1] has no JS analogue; you'd write a .map().

Reading & judging: The red flag is a Python for loop computing arithmetic element-by-element over arrays — for i in range(len(a)): result[i] = a[i] * b[i]. That throws away numpy's entire reason for existing; the vectorized a * b is both shorter and dramatically faster. Also watch dtypes: integer arrays silently truncate (np.array([1,2,3]) / 2 is fine, but in-place division on an int array floors). And check that shapes are asserted or at least commented near model boundaries — a silent shape mismatch that happens to broadcast produces wrong numbers, not an error.

pandas: the spreadsheet as a variable

A pandas DataFrame is a table — labelled rows, named columns, mixed types per column — that you manipulate in code. A Series is a single column. If numpy is the array, pandas is the spreadsheet you can query. In GenAI work you meet it in data prep and, above all, evaluation datasets: the CSV of test prompts, expected answers, and model scores that tells you whether your system actually works.

import pandas as pd

# Read a CSV of eval cases into a DataFrame
df = pd.read_csv("eval_results.csv")

# Peek at the first few rows — always do this first
print(df.head())
#    id             prompt        model   latency_ms  score
# 0   1   Summarize this...     gpt-5.1          820   0.91
# 1   2   Translate to fr...    gpt-5.1          640   0.88
# ...

Selecting columns and rows. df["col"] gives a Series (one column). .loc selects by label, .iloc by integer position:

df["score"]            # the score column (a Series)
df[["prompt", "score"]]  # two columns (a DataFrame — note the double brackets)
df.loc[0, "prompt"]    # value at row label 0, column "prompt"
df.iloc[0]             # the entire first row, by position

Filtering with boolean masks is the pattern you must be able to read. df["score"] < 0.5 produces a Series of True/False; passing it back into df[...] keeps only the True rows:

# Every eval case the model scored below 0.5 — the failures worth inspecting
failures = df[df["score"] < 0.5]

# Combine conditions with & (and) / | (or) — each condition MUST be parenthesised
slow_failures = df[(df["score"] < 0.5) & (df["latency_ms"] > 1000)]

Aggregation and groupby. To turn a table of results into a summary:

print(df["score"].mean())          # => 0.87   average score across all rows
print(df["latency_ms"].max())      # => 2100

# Average score per model — the core of any model comparison
print(df.groupby("model")["score"].mean())
# model
# gpt-5.1            0.87
# claude-sonnet-4-5  0.89

groupby("model") splits the table into one group per model, then ["score"].mean() computes the average within each. This three-token expression is the heart of most eval reports.

.apply runs a function over each row or value — used when a computation doesn't have a built-in vectorized form (e.g. calling a scoring function on each prompt):

# Add a column classifying each row by its score
df["grade"] = df["score"].apply(lambda s: "pass" if s >= 0.8 else "fail")

Reading a data-prep pipeline is then a matter of following the DataFrame through a chain: load → filter → transform → group → summarize. Each step returns a new DataFrame or Series.

JS → Python: There is no first-class pandas equivalent in JS — the closest mental model is an array of objects (records) that you query, except pandas operates column-first and vectorized. df[df["score"] < 0.5] is like records.filter(r => r.score < 0.5), but it runs in C over whole columns. df.groupby("model")["score"].mean() is a one-liner for what you'd write as a manual reduce-into-a-Map in JS. Think of a DataFrame as a queryable, in-memory table — closer to a SQL result set than to a plain array.

Reading & judging: Watch for giant DataFrames loaded whole into memory — pd.read_csv("huge.csv") pulls the entire file in; for multi-GB eval logs you want chunking or a database, and a reviewer should flag the unbounded read. Watch for long chained operations with no intermediate variables — df[df.x>0].groupby("y").z.mean().reset_index().sort_values("z").head(10) is correct but opaque; a clear pipeline names its steps. And note pandas is for wrangling, not for production request paths — it's memory-hungry and single-machine; seeing it inside a per-request web handler is a smell.

HTTP with httpx / requests: what every SDK is underneath

Every LLM call is an HTTPS POST with a JSON body and a bearer token, and a JSON (or streamed) response. The SDKs hide this, but when you debug a timeout, a 429, or a hanging stream, you're back at the HTTP layer. requests is the classic synchronous library; httpx is the modern one that does the same thing plus async and HTTP/2 — and it's what the OpenAI and Anthropic SDKs use internally.

import httpx

# A GET request
resp = httpx.get("https://api.example.com/models", timeout=10.0)
print(resp.status_code)   # => 200
print(resp.json())        # parse the JSON body into a Python dict/list

# A POST with a JSON body and an auth header
resp = httpx.post(
    "https://api.example.com/v1/chat",
    headers={"Authorization": "Bearer sk-...", "Content-Type": "application/json"},
    json={"model": "gpt-5.1", "messages": [{"role": "user", "content": "Hi"}]},
    timeout=30.0,
)
resp.raise_for_status()   # raise an exception on any 4xx/5xx status
data = resp.json()

Two calls carry outsized weight. raise_for_status() turns a failed HTTP status (401 auth error, 429 rate limit, 500 server error) into a Python exception instead of letting your code sail on with an error body it treats as data. timeout bounds how long you'll wait — without it, a stalled connection hangs your program forever, which in an agent loop means a wedged worker.

Clients as context managers. Creating a fresh connection per request is wasteful. A Client (sync) or AsyncClient (async) pools and reuses connections; the with block guarantees it's closed:

# Reuse one connection pool across many requests
with httpx.Client(timeout=30.0, headers={"Authorization": "Bearer sk-..."}) as client:
    for prompt in prompts:
        r = client.post("https://api.example.com/v1/chat", json={"input": prompt})
        r.raise_for_status()

# Async version — for firing many calls concurrently
async with httpx.AsyncClient() as client:
    r = await client.post("https://api.example.com/v1/chat", json={"input": "Hi"})

Streaming responses — how token-by-token LLM output arrives over the wire — uses a streaming context so you don't buffer the whole body:

with httpx.Client() as client:
    with client.stream("POST", url, json=body) as resp:
        resp.raise_for_status()
        for line in resp.iter_lines():   # process each chunk as it arrives
            print(line)

The SDK's for chunk in stream: that you'll meet in the next chapter is this, dressed up.

JS → Python: httpx/requests are fetch/axios. httpx.get(url).json() ≈ (await fetch(url)).json(). The big difference: fetch does not reject on HTTP 4xx/5xx (you check res.ok yourself), and neither does httpx by default — raise_for_status() is the explicit equivalent of that check. httpx.Client() as a context manager is like a reused axios instance; AsyncClient + await maps to fetch in async functions. iter_lines() over a stream is the Python analogue of consuming a ReadableStream reader.

Reading & judging: Three failures dominate AI HTTP code. Missing timeout — a call with no timeout can hang an entire agent; every request to an external service must have one. Missing raise_for_status() (or an unchecked status) — code that reads resp.json() without checking the status will happily parse a {"error": ...} body as if it were a success, producing baffling downstream failures. Unbounded concurrency — firing hundreds of AsyncClient requests at once with no semaphore will trip rate limits or exhaust connections; look for a concurrency cap. Also flag a new Client() created inside a hot loop instead of reused, and any auth token that's hard-coded in the source rather than read from the environment.

Try it: Load any CSV with pandas and run df.describe() to see summary stats, then df.groupby(some_column).size() to count rows per group. Separately, pip install httpx and hit a public JSON API (e.g. httpx.get("https://api.github.com/repos/openai/openai-python").json()["stargazers_count"]) — then remove the .raise_for_status() mentally and ask what would happen if that endpoint returned a 500.

---