From traits to a playbook

"What do the top streams share?" sounds like one question. Run it against the summary table and you get the traffic answer: the top ten by GMV are all human hosts, nine of the ten opened with a giveaway, and giveaway streams average 10,364 views against 7,546. That is real. The platform's boost rewards a giveaway opener, x1.93 against x1.25, and the room fills.

Run it against the minute table, comparing every stream with itself, and you get the selling answer: inside a stream, the minutes where the giveaway is live convert 2.31 points worse than the same stream's other minutes. Also real. The room is bigger and browsing for a free serum, and it clicks less on the card that costs 29 dollars.

The pooled regression sees streams, so it sees traffic. The within-stream regression sees minutes of one stream, so the traffic is held fixed and only the pitch is left to vary. Neither can answer the other's question, and the mistake is not picking the wrong one, it is not knowing there are two.

• Pooled regression - one OLS over all rows as if they were exchangeable. Across streams it compares stream A to stream B; across minutes it compares minute 43 of stream A to minute 12 of stream B.
• Fixed effects - a separate intercept per stream, so each stream is compared only with itself. Everything constant inside a stream (host, hour, weekday, the traffic draw, the giveaway opener) is absorbed and cannot confound.
• Demeaning - the arithmetic that implements fixed effects: subtract each stream's mean from every column, then run OLS without an intercept. The stream dummies never have to be built.
• Clustered standard errors - minutes of one stream are not independent draws; they share a room, a host, a mood. Clustering by stream tells the standard error to expect that, so the p-values are honest about 60 streams rather than pretending to 7,200 free observations.
• Confound - a variable that moves both the treatment and the outcome. Here, the platform's boost moves both "giveaway" and "GMV", which is why the pooled coefficient is a traffic number wearing a giveaway costume.
• SHAP - a per-row, per-feature share of a model's prediction, summing to the prediction. Mean absolute SHAP over rows is a feature ranking.
• Held-out stream - a stream the model never trained on. Holding out rows instead lets the model see minute 42 and be asked about minute 43 of the same stream, which is not a test.

Start where everyone starts, with the summary table sorted by GMV. Nine of the top ten opened with a giveaway, against 32 of 60 overall. Giveaway streams average 16,904 dollars of GMV against 15,265, and 10,364 views against 7,546. The eye says "giveaway wins", and a regression that treats streams as the unit says the same thing with a p-value attached.

The pooled model is gmv ~ giveawayOpen + C(hostType) + C(hour) on the 60-row summary. Controlling host type and hour is the responsible-looking move; it removes the AI-host penalty and the evening effect. The giveaway coefficient survives it: plus 5,576 dollars, p below 1e-7. That is not a fluke and it is not wrong. It is the platform's boost, x1.93 against x1.25, showing up as GMV, because GMV is traffic times selling and the giveaway bought traffic.

Look at the same summary table's other columns and the seam is already visible. Giveaway streams have a lower CTOR, 11.16 against 13.10, a lower GPM, 1,547 against 1,983, and a shorter watch, 8.67 against 9.38 minutes. More people, converting less. The pooled GMV regression cannot see that because GMV adds the two together.

Fixed effects sounds like a technique; it is a subtraction. For every column in the model, subtract that stream's mean from every one of its minutes. Then run OLS on the demeaned columns with no intercept. Whatever was constant within a stream is now zero everywhere, so the host, the hour, the weekday, the traffic draw and the giveaway opener all vanish from the comparison. What is left is minute 43 against minute 12 of the same stream. Standard errors are clustered by stream, because 120 minutes of one room are not 120 independent draws.

The outcome is CTOR in points, orders over clicks, and the regressors are the segment indicators, the three visual columns, and whether the giveaway is live in that minute.

Three rows say nothing, and that is the finding of the night, not a disappointment. Product in hand, close-up ratio and a price on screen do not move CTOR. Run the identical regression with CTR as the outcome and the same three rows come alive.

Put the CTR table over the CTOR table and a clean split appears. What you show moves the click. Product in hand, a close-up, a price card on screen: each adds about a point of CTR and none of them touches CTOR. The viewer taps the card because they saw the thing. What you say moves the order. CTA, price, a short burst of urgency: each adds points of CTOR, and the CTA adds points of both. The viewer buys because they were asked, told the number, and given a reason to do it now.

The Q&A row is the one worth pausing on. It lowers CTR by a point (the host is looking at comments, not the product) and raises CTOR by a point (the person whose question got answered converts). A per-viewer read from session 5 says Q&A minutes sell 3.68 orders per thousand viewer-minutes, below the mean of 5.80, so on its own it is a weak selling minute. Session 8's bench is where "answer the room every ten minutes" gets priced properly, and it comes out at plus 4.4 percent GMV with the longest watch of any single cue.

You do not have to demean by hand. linearmodels 7.0 does the same arithmetic in one line: PanelOLS(y, X, entity_effects=True) on a MultiIndex of (stream_id, t), then fit(cov_type="clustered", cluster_entity=True). Its coefficients match the hand version to the third decimal. Tonight's build-along does it by hand once, so you know exactly what the one line is doing and can explain it to the person who asks.

The within-stream regression answers "on average, inside a stream, what does a CTA minute do to CTOR". It does not answer "for which hosts, in which rooms, does it do the most". That is a heterogeneous-effects question, and two libraries in the panel are built for it.

• DoWhy 0.14 makes the causal claim explicit before estimating it: CausalModel(data, treatment, outcome, graph), then identify_effect(), estimate_effect(), and a refute_estimate() step that adds a random common cause or a placebo treatment and checks the estimate survives. The graph is where you write down that the boost moves both the giveaway and the room, which is exactly what the pooled regression could not say.
• EconML 0.17.0 estimates conditional effects: LinearDML or CausalForestDML with the minute's context (room size, minute of show, host) as the effect modifiers. The output is a per-minute estimate of what a CTA is worth, which is what a coach needs at minute 43 and a pooled average cannot give.

Both need the identification argument that fixed effects gave you tonight: within a stream, what varies is the pitch. The method ladder for that argument, from DAGs through backdoor adjustment to doubly robust estimators, is a course of its own: learn causal inference with phoebe.

The model is LightGBM 4.7.0's LGBMRegressor on per-minute orders, with 24 features: the traffic columns (room, impressions), the content columns (the three visual scores, scene cuts, words per minute, the urgency run counter, the eight segment indicators), the giveaway state, the clock (hour, minute of show), the engagement counts, and whether the host is an AI avatar. The settings are modest on purpose: n_estimators=400, learning_rate=0.03, num_leaves=15, min_child_samples=40, subsample=0.8, subsample_freq=1, colsample_bytree=0.8, random_state=20260924. Fifteen leaves and forty minimum samples per leaf keep it from memorising individual minutes.

The split is the part that matters. Twelve streams, chosen by a seeded shuffle, are held out whole, every minute of them unseen by the model. Splitting by row would let the model train on minute 42 of stream 9 and be tested on minute 43 of the same stream, and adjacent minutes of one room are near copies of each other. That test scores brilliantly and tells you nothing about a stream you have not seen. On the 12 held-out streams the model's MAE is 1.43 orders per minute against 2.56 for predicting the mean; the model's error is a little over half the baseline's.

SHAP 0.52.0's TreeExplainer is exact for tree ensembles: for every held-out minute and every feature it returns the share of that minute's prediction the feature is responsible for, and the shares sum to the prediction. Take the absolute value, average over the held-out minutes, sort, and you have the ranking in the diagram.

Impressions is first at 1.058 and room is fifth at 0.414. That is the traffic, and it should dominate a model of order counts: a minute in a room of 1,500 has more orders than a minute in a room of 300 whatever the host does. The model is not being clever there. The interesting part is everything between and after: is_cta at 0.684, close_up_ratio at 0.463, product_in_hand at 0.417, fomo_run at 0.310, is_hook at 0.261, then price_overlay and is_price at 0.19. Those are, almost column for column, the rows of the two within-stream tables that had a p-value below .05. The model found them without being told, from a different objective (orders, not a rate) with a different method (trees, not demeaned OLS).

Note what is not in the top ten: giveaway_live and giveaway_open. In a model that already has the room and impressions as features, the giveaway has nothing left to explain. Its whole effect on orders was the traffic it bought, which the model reads directly from the room. That is the pooled-versus-within lesson again, from a third angle.

is_ai is tenth at 0.153. The host being an avatar matters, and it matters less than what the host is doing in the minute. On TikTok Shop the point is moot: the Requirements for High-Quality Videos and LIVEs, effective 2026-09-17, require real-time verbal interaction and do not permit AI-generated voices on a promotional LIVE, so the AI-host streams in this data describe a trade that is not on offer there.

A playbook line a host will follow needs a verb, a moment and a number. The number comes from the within-stream table, because that is the one that compares a host with herself; the model is the corroborating witness, not the source. Here is the table the leader track's sessions 3 and 4 hand out, as the build-along writes it.

Short urgency on this data is minutes one and two of a run, with CTOR at 15.12 and 15.44 for those two minutes and 8.30 from the third. The playbook says two minutes because that is where the number turns, and because the TikTok Shop Content Policy (effective 2026-09-17) and the UK DMCC Act both treat a limited-time claim that is not genuine as a violation; a countdown that runs six minutes is not a countdown. The giveaway line is not a "never". It is worth 5,576 dollars of GMV through traffic on the pooled read and costs 2.31 points of CTOR while it is live, and whoever owns the traffic budget makes that call with both numbers in front of them.

A SHAP value is a share of a prediction. If the model had learned that CTA minutes tend to come after price minutes, and price minutes tend to be high-order minutes, the CTA feature would carry a large SHAP value for reasons that have nothing to do with the CTA. SHAP faithfully reports what the model used; it makes no promise that what the model used is what caused the orders. The within-stream regression, with the segment indicators entered together and the stream fixed, is the design that lets you read a coefficient as "holding the rest of the minute's content and the stream fixed, this segment does this". That is why the playbook cites the regression and merely mentions the model.

Two more things the model cannot say. It cannot price a cue the host has never done, because a tree has no leaf for it; the bench in session 8 is where a policy is actually applied. And it cannot separate "the host did a CTA" from "the host did a CTA at the right moment", because the moment is exactly the thing session 7's nowcast exists to find. The model tells you the CTA column matters. It does not tell you when.