Builder session 3 · Random variables, E and Var

Part 0

Expectation is a balance point, not a prediction

The single most misused idea in this session is the expected value. "Expected revenue per session is $2.37" does not mean any session earns $2.37 - almost none do. A session earns $0 (no purchase) or roughly $74 (a purchase), never $2.37. The expectation is the long-run average, the point where the distribution balances. Get that straight now and the rest of probability, all the way to confidence intervals, stays honest.

Live - built in session Self-study - run after ★ The one big idea The case: Lumen Skincare

★ What you walk out with today Discrete vs continuous random variables, the pmf / pdf / cdf trio and when each applies, expectation E[X] = sum of x times p(x), variance Var[X] = E[(X - mu) squared], and standard deviation as its square root. You will compute all of them from the definition with np.average(values, weights=probs), cross-check against np.mean / np.var, peek at covariance, and meet the scipy frozen distribution that hands you .mean(), .var(), .pmf(), and .cdf() in one line.

Part 1 · the vocabulary in code

Expectation, variance, and the pmf / pdf / cdf trio 7 min live

Picture a distribution as weights sitting on a ruler. The expected value E[X] is the fulcrum - the spot where the ruler balances. The variance is how far, on average, the weights sit from that fulcrum. Everything below is those two ideas plus the three functions (pmf, pdf, cdf) that describe the distribution itself.

🔍 Click to zoom - bake the parameters in once, then ask anything

🔍 Click to zoom - E[X] is the balance point, variance is the spread around it

LiveThe definitions, and the functions that compute them4 min▶

Discrete vs continuous: a discrete RV takes countable values (orders per day: 0, 1, 2, ...); a continuous RV takes any value in a range (order value: any dollar amount). Discrete uses a pmf; continuous uses a pdf.
Expectation: E[X] = sum of x times p(x) for discrete, an integral for continuous. In code it is a probability-weighted average: np.average(values, weights=probs).
Variance: Var[X] = E[(X - mu) squared] = E[X squared] - (E[X]) squared. The average squared distance from the mean.
Standard deviation: sigma = sqrt(Var[X]). Same units as X, which is why we report it instead of variance.
The trio: pmf/pdf gives the height at a value, cdf gives P(X <= value), and they are two views of the same distribution.

pmf for counts, pdf for measures A pmf value is a real probability - binom.pmf(7) is literally P(X = 7). A pdf value is a density, not a probability: for a continuous RV, P(X = exactly 74.00) is 0, so you always ask for a range via the cdf, e.g. P(X < 100) = cdf(100). Mixing these up is the classic continuous-variable trap.

★ everyone builds · E and Var by hand and by scipy

Compute expectation and variance four ways ★ 22 min · everyone builds

Start with the number that pays the bills - expected revenue per session - then build E and Var from the definition, cross-check against numpy, peek at covariance, and finish with the scipy frozen distribution that does it all for you.

Expected revenue per session. Probability of converting times average order value. This is an expectation, not a guarantee.

🔍 Click to zoom - the $2.37 expectation is never a session's actual outcome

expected_revenue.py

p_convert = 0.032      # Lumen checkout conversion rate
aov       = 74.0       # average order value in dollars

exp_rev = p_convert * aov
print("expected revenue / session:", round(exp_rev, 2))   # 2.37

# scale it up: ~4,000 sessions a day
print("expected revenue / day    :", round(exp_rev * 4000, 2))  # ~ 9,472
# NOTE: no single session earns $2.37 - each earns $0 or ~$74.
# $2.37 is the long-run average, the balance point.

Real world

When Lumen's finance team models a paid-traffic campaign, $2.37 per session is the number that decides the bid ceiling: if a click costs more than about $2.37, the campaign loses money on average. But an analyst who promised "every visitor is worth $2.37" would be wrong about every individual visitor - the value is either $0 or a full order. The expectation guides the budget; it never describes a person.

E and Var from the definition. Use a probability-weighted average, then confirm by simulating draws.

e_and_var.py

import numpy as np

values = np.array([0, 1, 2, 3, 4, 5])          # items per order
probs  = np.array([.05, .15, .30, .25, .15, .10])

EX  = np.average(values, weights=probs)        # 2.6  = sum x*p(x)
EX2 = np.average(values**2, weights=probs)     # 8.5
var = EX2 - EX**2                              # 1.74 = E[X^2] - E[X]^2
print("E[X] :", round(EX, 3), " Var[X] :", round(var, 3), " SD :", round(var**0.5, 3))

# cross-check by drawing 500,000 samples from this pmf
rng   = np.random.default_rng(42)
draws = rng.choice(values, size=500_000, p=probs)
print("np.mean:", round(draws.mean(), 3), " np.var:", round(draws.var(), 3))
# the sample mean/var land on 2.6 and 1.74 - the definition and the data agree

Covariance - a preview of correlation. How order value and prior spend move together, on the real Lumen columns.

covariance.py

import numpy as np, pandas as pd

df = pd.read_parquet("lumen_orders.parquet")

cov = np.cov(df["order_value"], df["prior_30d_spend"])
print("covariance matrix:\n", np.round(cov, 1))
# the off-diagonal is the covariance - positive, so they rise together

r = np.corrcoef(df["order_value"], df["prior_30d_spend"])[0, 1]
print("correlation preview:", round(r, 2))   # ~ 0.6  (full story in b10)

The scipy frozen distribution. Bake the parameters in once, then ask it anything. No hand formulas.

scipy_frozen.py

import scipy.stats as st

rv = st.binom(25, 0.3)      # 25 trials, success prob 0.3 - a "frozen" distribution

print("mean :", rv.mean())      # 7.5  = n*p
print("var  :", rv.var())       # 5.25 = n*p*(1-p)
print("pmf 7:", round(rv.pmf(7), 4))    # P(X = 7)
print("cdf 7:", round(rv.cdf(7), 4))    # P(X <= 7)
# .mean() and .var() match n*p and n*p*(1-p) - scipy did the E and Var for you

Before Session 4

This week ◐ 40 min total

Expected value of an order. Compute E[order_value] directly with df["order_value"].mean() and confirm it lands near $74. Then compute expected revenue per session as P(convert) x E[order_value] yourself.
Linearity of expectation. Show that E[2X + 10] = 2 E[X] + 10 on the items-per-order pmf, both by transforming values first and by transforming E[X]. They must match.
Variance is not linear. Show Var[2X] = 4 Var[X], not 2 Var[X]. Write one sentence on why variance scales with the square.
Optional: freeze st.poisson(128) for daily orders and read off its mean and variance. Notice they are equal - that is the Poisson signature, and it sets up b4.

Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · Lumen's expected revenue per session is $2.37. What does that mean for a single session?

Expectation is a balance point, not a prediction for any one outcome. Each session earns $0 (no purchase) or a full order value; the $2.37 only appears as an average over many sessions.

2 · Which numpy call computes E[X] from a list of values and their probabilities?

E[X] is a probability-weighted average, so you weight the values by their probabilities: np.average(values, weights=probs). np.mean would treat every value as equally likely, which is wrong unless the pmf is uniform.

3 · For rv = scipy.stats.binom(25, 0.3), what does rv.pmf(7) return?

pmf is the probability mass function: for a discrete distribution it gives the probability of exactly that value. P(X <= 7) would be rv.cdf(7); a percentile would come from rv.ppf.

Source material

What this session covers

This builder session covers ~80% of the random-variables content in the sources below, computed on Lumen. Named distributions as scipy objects - normal, binomial, Poisson, lognormal - get their own session in b4.

✓Discrete vs continuous RVs, E[X], Var[X], SD, pmf / pdf / cdf, covariancePart 1 + demo

✓E and Var from the definition with np.average; scipy frozen distributionsdemo

◐OpenIntro Statistics ch. 3 - Distributions of random variablestheory spine; free PDF

◐Khan Academy - Random variables and probability distributionsunit refresher

◐StatQuest - Expected values, clearly explainedvideo, balance-point intuition

Builder Session 3 cheat sheet · pin this

ExpectationE[X] = sum x*p(x). In code: np.average(values, weights=probs). It is a balance point, not a prediction.

VarianceVar[X] = E[(X-mu)^2] = E[X^2] - E[X]^2. Average squared distance from the mean.

Std deviationsigma = sqrt(Var). Same units as X - report this, not variance.

pmf vs pdfpmf = real probability (discrete). pdf = density (continuous); for a range use the cdf.

Frozen distrv = st.binom(25, .3) then rv.mean(), rv.var(), rv.pmf(k), rv.cdf(k).

Lumen numberExpected revenue / session = 0.032 x $74 = $2.37. The bid ceiling, not a per-visitor promise.