When one feature depends on another 7 min live
An interaction exists when the effect of one feature depends on the value of another. Lumen has a real one, and you have been carrying it since b1: spend matters more for returning customers than for new ones. A returning customer with high prior spend is not simply "returning plus high spend" - the combination behaves differently from what either column predicts on its own. A linear model, given only the two main effects, has no way to say that.
LiveThe four shapes an interaction takes5 min▶
Rule #20 from Google's Rules of Machine Learning is the licence for all of this: "Combine and modify existing features to create new features in human-understandable ways." The last four words are the discipline - if you cannot say out loud what the new column means, it will not survive review or debugging.
- Products.
spend * returningis the Lumen one, and it is worth +0.006 on the workbench ladder. Multiplication is the plain arithmetic form of "these two matter together." - Ratios. Spend per page viewed, items per order, revenue per session. Ratios are often the most human-readable features on the whole sheet, because they are the numbers the business already talks in.
- Differences. Price minus category median, order value minus this customer's own average. A difference reframes an absolute number as "relative to what is normal here."
- Polynomial expansion. When you want every pair at once rather than a chosen few.
from sklearn.preprocessing import PolynomialFeatures
# every pairwise product, no squares, no constant column
PolynomialFeatures(degree=2, interaction_only=True, include_bias=False)interaction_only=True on twenty columns gives you 190 new ones, and Rule #21 is waiting: "The number of feature weights you can learn in a linear model is roughly proportional to the amount of data you have." Expansion is a fine way to explore. Shipping the three pairs you can explain is usually the better model.
Crosses, and the number that should stop you 8 min live
A feature cross is the categorical version of an interaction. Google's definition: feature crosses are "created by crossing (taking the Cartesian product of) two or more categorical or bucketed features". Instead of asking the model to combine two one-hot blocks itself, you hand it one block whose levels are the combinations.
LiveGoogle's leaf example, exactly4 min▶
Two categorical features describing a leaf. Edges: {smooth, toothed, lobed}. Arrangement: {opposite, alternate}. Crossing them gives 3 x 2 = 6 combinations, and a leaf that is lobed with alternate arrangement is represented as {0, 0, 0, 0, 0, 1}.
Notice what the model gains. With separate one-hot blocks it can learn "lobed matters" and "alternate matters" and add them. With the cross it can learn that lobed-and-alternate specifically behaves unlike either piece - the same thing the 2x2 grid above showed for numerics.
Google's warning, verbatim. "Crossing two sparse features produces an even sparser new feature than the two original features." Their worked case: a 100-element feature crossed with a 200-element feature produces a 20,000-element sparse feature. On Lumen the same arithmetic decides the design for you - channel x device is 9 x 3 = 27 columns, perfectly reasonable, while channel x postal_code is 9 x 1,200 = roughly 10,800 columns, almost all of them empty almost all of the time. Rule #21 again: you cannot learn 10,800 weights from 40,000 rows with a 3.2% base rate.
Self-studyThe discipline, in three checks3 min read▶
- Bucket first, then cross. Continuous features get discretized before crossing, which is why b2's binning work comes before tonight. A crossed pair of 4-bucket features is 16 columns, not thousands.
- Multiply the cardinalities before you write the code. Two numbers and ten seconds. If the product is in the thousands, you are building a sparsity problem, not a feature.
- For genuinely high-cardinality categoricals, cross is the wrong tool. b3 gave you the right one: target encoding with cross-fitting, which turns
postal_codeinto a single column instead of 1,200. And per Google, "For natural-language data, the main method of reducing dimensionality is to convert feature vectors to embedding vectors" - the b9 bridge.
Group aggregates and entity history 8 min live
Every feature so far describes a row. Aggregations describe the group the row belongs to: this customer's mean order value, this channel's conversion rate, this category's return rate. They give a single session the context of everything else in its group, and they are usually cheap, because the data is already in your warehouse.
The version most teams never build is the one worth the most: features describing an entity's history up to the prediction moment. Orders in the prior 30 and 90 days. Days since last order. Lifetime value to date. Sessions in the last week. These are not group statistics, they are a small biography of the customer as of the instant you have to decide something about them.
LiveThe groupby and merge, with the cutoff in it5 min▶
Read the filter on the first line before anything else. It is the whole session.
hist = (
orders[orders["order_ts"] < orders["cutoff_ts"]] # the rail
.groupby("customer_id")
.agg(
orders_prior_90d=("order_id", "count"),
mean_order_value=("order_value", "mean"),
last_order_ts=("order_ts", "max"),
)
.reset_index()
)
sessions = sessions.merge(hist, on="customer_id", how="left")
sessions["days_since_last_order"] = (
sessions["prediction_ts"] - sessions["last_order_ts"]
).dt.daysWithout that filter this is a four-line snippet that any analyst would wave through in review. With it, it is a correct feature. The difference is invisible in the output and worth everything in production.
groupby over the whole table is a leak by construction: a customer's mean order value computed across all time includes orders they placed after the session you are scoring, so the feature knows the future. This is exactly the case that point-in-time joins exist to handle (leader session a4), and it is the same rule b4 applied to tenure and recency.
Self-studyWhy this one survives code review3 min read▶
b5 named the four leakage families and gave you the three rules. Aggregation leakage is the family that slips past all of your existing habits, and it is worth understanding why.
- It passes the train/test split test. You can split correctly, fit only on train, wrap everything in a Pipeline, and still leak - because the leak happened upstream, in the table you built before sklearn ever saw the data.
- It looks like good engineering. A clean groupby joined onto a session table is the shape of code that gets praised in review.
- It shows up as a better score, which is sklearn's whole warning: leakage "results in overly optimistic performance estimates... and thus poorer performance when the model is used on actually novel data."
- The fix is structural, not clever. Every aggregate carries a cutoff timestamp, or it does not ship. Feast's own value list names the same thing: "Avoid data leakage by generating point-in-time correct feature sets."
Turn on the interactions lever ★ 10 min · everyone runs it
Numerics, encoding and datetime are on, and the ladder is at 0.806. Add the interaction and watch a smaller, more honest kind of win.
Confirm the rung you are standing on. Press Score on held-out with the first three levers on: 0.806. Everything up to here has been about improving individual columns.
Switch on Interactions and score again: 0.812. The lever adds spend-times-returning, the relationship from the 2x2 grid at the top of this page.
Read the honest size of it: +0.006. Encoding bought 0.089 and datetime bought 0.021. One interaction is a fraction of either, and that is the normal shape of this technique.
Understand why it is still worth your evening. Encoding wins are one-time - once a column is readable, it is readable. Interaction wins compound across pairs, and the search space is enormous, which is why judgement about which pairs to build matters more here than anywhere else in the track.
Try it yourself - this week ◐ 20-30 min total
- Write down three interactions from your own domain as sentences first: "X matters more when Y is high." Only then turn them into columns. If the sentence is hard to write, the feature is probably not real.
- List your categorical columns with their cardinalities next to them, then compute the width of every pairwise cross. Circle the ones under about fifty columns - those are your candidates. Everything else needs bucketing or target encoding.
- Build one entity-history feature with an explicit cutoff timestamp, then build the same feature without the cutoff and compare held-out scores. The gap is your personal version of the leakage demo.
- Audit an existing aggregate feature in a model you already run. Ask one question: could any row that contributed to this number have happened after the moment being predicted? Bring the answer to b10.
Official sources covered
This session teaches the working core of Google's feature-cross material and the interaction-building content from Kaggle Learn, plus the aggregation pattern that sits underneath every feature store. Certificates and graded exercises stay with the official providers.
Three questions before you go 🎯 ◐ 90 seconds
1 · Google documents a cross of a 100-element feature with a 200-element feature. How wide is the result?
A cross is the Cartesian product, so widths multiply. Google states the consequence directly: "Crossing two sparse features produces an even sparser new feature than the two original features." On Lumen, channel x device is 9 x 3 = 27 columns and fine; channel x postal_code is 9 x 1,200 = roughly 10,800 and is not.
2 · What makes something an interaction rather than two ordinary features?
Dependence is the definition. On Lumen, spend matters more for returning customers than for new ones, so the returning-and-high-spend cell sits further from baseline than the two main effects add up to. A linear model given only main effects cannot express that. Rule #20 licenses building it: "Combine and modify existing features to create new features in human-understandable ways."
3 · What is the rail on group aggregates and entity-history features?
A naive groupby over the whole table is a leak by construction - a customer's mean order value across all time includes orders placed after the session being scored. Splitting correctly does not save you, because the leak happened upstream while building the table. Point-in-time joins exist for exactly this (leader session a4); the full leakage family is b5.