LIVE Commerce Analytics / Builder session 1 of 8
Learn LIVE Commerce Analytics with Phoebe · Builder session 1 of 8

The stream as data

A two-hour TikTok Shop LIVE leaves four recordings behind: the video file, the audio inside it, the comment feed, and the seller dashboard's ledger of who was in the room and who bought. They are recorded on four clocks, at four grains, by four systems that were never designed to be joined. Tonight you pick the one grain they all fit into, the minute, and build the table the rest of the builder track is built on: sixty streams of one skincare brand, 7,200 rows, one row per minute, with what was said, what was shown and what was sold sitting side by side.

🟢 Foundational Data scientists and analytics engineers Anyone who owns a LIVE dashboard 45 min
0-3 · Welcome 3-20 · Four sources, one grain 20-40 · Build-along: the minute table 40-45 · Q&A
Part 0

How the builder track works

Eight sessions, 45 minutes each, one pipeline. Tonight the grain and the table. Session 2 gets the four recordings onto one clock. Sessions 3 and 4 add what the camera shows and what the host says, as columns. Session 5 scores every minute, session 6 works out which traits actually move the money and which only look like they do, session 7 forecasts the next five minutes while the stream is still running, and session 8 cues the host on a replay bench and measures what the cue was worth. The brand throughout is Lumen Glow, one hero SKU, a 29 dollar serum sold by four human hosts and two AI avatars across 60 streams.

Live = presented in the 45 minutes Self-study = the full depth, after class 🔍 Every diagram zooms

★ What you walk out with tonight. A 26-column minute table you can open in pandas, with every later column already reserved: segment, product in hand, close-up ratio, price on screen, comments, replies, room size, product impressions, clicks, orders, GMV. The funnel arithmetic that turns those columns into the metrics a seller dashboard reports, checked to the cent against the stream totals. And a clear head about grain, because the three most common analysis mistakes in live commerce are all grain mistakes wearing a costume.

LiveWhy the minute, and not the frame or the stream3 min▶

Every source has a natural grain. Video is frames, 30 a second. Speech is words, with a start and an end time each. Comments are events with a timestamp. The seller dashboard reports the whole stream as one row, or at best a curve you can hover. None of those grains can hold the others. A frame does not know what was sold, a stream total does not know what was said at minute 43.

The minute is the coarsest grain at which a host can still change something. A host cannot react to a frame, and a stream total arrives after the room has gone home. Every question the leader track asks (which minutes convert, what the host was doing in them, whether a cue at minute 43 paid off by minute 48) is a minute question. So the table is built at the minute, and finer sources are aggregated up to it: share of frames with the product in hand, count of comments, dominant speech segment.

Real world

Lumen Glow's first attempt at this joined comments to orders by second and found nothing. The comment "is it good for oily skin?" and the order it produced were 40 to 90 seconds apart, so no second-level join ever matched them. At the minute grain, with a one-minute lag column, the pattern was obvious: minutes with an answered question were followed by minutes with more orders.

Self-studyThe vocabulary the whole track uses4 min▶
  • Room - the number of viewers watching at a given minute. Also called concurrent viewers. Rises with arrivals, falls with leavers.
  • Views - the count of entries into the stream. One person leaving and coming back is two views. TikTok Shop reports Viewers (unique people) beside it, plus Peak Concurrent Viewers and Average Concurrent Viewers.
  • Product impressions - how many times the pinned product card was shown. Roughly the room times the share of viewers who see the card each minute.
  • Product clicks and CTR - taps on the product card, and clicks divided by impressions.
  • Orders and click-to-order rate - purchases, and orders divided by clicks. TikTok Shop's analytics call the rate CTOR; the course's tables write CVR, the same number.
  • GMV - gross merchandise value, the sum of order values before refunds.
  • GPM - GMV per thousand views, Watch GPM in TikTok Shop's LIVE analytics. The one number that lets a small room and a large room be compared on how well they sold.
  • Segment - what the host is doing in a minute, as a tagger would label it from the transcript: hook, product build, demo, price, CTA, urgency, Q&A, filler, dead air. Session 4 builds the tagger; tonight it is a column.
Part 1 · covers the four sources and the event model that joins them

Four recordings, one row per minute

9 min live

The four sources arrive at wildly different rates, and the temptation is to build four tables and join them later. Build the minute table first and treat every source as something that contributes columns to it. The join key is the stream id and the minute number, and nothing else.

Four sources, four grains, one join key Video 30 frames a second what the camera showed Audio words with start and end what the host said Comment feed one event per comment what the room said Seller ledger room, impressions, clicks, orders, GMV per minute Aggregate to the minute share of frames: product in hand mean close-up ratio, scene cuts dominant segment label words per minute count(comments), count(likes) count(host replies) already per minute shift by the stream delay (b2) one minute row key: stream_id, t segment · product_in_hand · close_up_ratio price_overlay · scene_cuts · wpm comments · likes · replies room · impressions · clicks · orders · gmv 26 columns, 7,200 rows across 60 streams Every later session adds or reads a column here. Nothing downstream joins on anything but stream_id and t.
🔍 Click to zoom - four grains aggregated up to the minute and landing as columns of one row.
LiveThe four sources and what each one actually knows3 min▶

Each source answers one question and is nearly blind to the others. Knowing the blind spots is what stops you asking a source for something it never recorded.

SourceKnowsBlind toNative grain
Video fileWhat was in frame: the host, the product, a price card, a cutWho was watching and what they didFrame
Audio trackWhat was said, when, how fastWhether anyone heard it or believed itWord
Comment feedWhat the room said back, and how much of it the host answeredThe silent majority who never typeEvent
Seller ledgerRoom size, product impressions, clicks, orders, GMVWhy. It has no idea what the host was doingMinute, or stream

The whole course is the fourth row's blind spot filled in by the first three. The ledger says minute 43 sold 19 units. The other three say the host was holding the serum up to the camera, had just said the price, and was answering a question about oily skin. Neither half is analysis on its own.

LiveTwo conventions that save you a week3 min▶

Minute t is the interval [t-1, t) from the start of the stream, indexed from 1. Not from the top of the wall-clock hour, not from when the recording started. Every source is shifted to this clock in session 2, and once it is, "minute 43" means the same thing in every column.

Counts are sums, states are end-of-minute. Arrivals, comments, clicks, orders and GMV are what happened during the minute, so they add up: summing a stream's minute rows must give the stream total exactly, and tonight's demo checks that it does. Room size is a state, so it is the value at the end of the minute and it does not add up to anything. Mixing the two is the first grain mistake, and it produces "average room" numbers that are quietly wrong.

Write the check, not the note. A one-line assertion that the minute GMV sums to the stream GMV for every stream is worth more than a paragraph in the README saying it should. It ran on tonight's data and passed for 60 of 60.

Self-studyThe event model behind the minute table4 min▶

The minute table is a derived table. Underneath it, when you have real recordings, sit five raw tables that you keep and never edit:

  • streams - one row per stream: id, host, host type, start time, weekday, whether it opened with a giveaway, the platform's stream id.
  • frames - one row per sampled frame (1 per second is plenty): stream, offset seconds, then detector outputs as columns. Session 3 fills it.
  • words - one row per transcribed word: stream, start, end, text, confidence. Session 4 fills it and tags segments from it.
  • comments - one row per comment or like event: stream, offset, author hash, text, whether the host replied to it.
  • ledger - the seller dashboard export at whatever grain it gives you, with its own timestamps untouched.

The minute table is rebuilt from these every time a detector or tagger improves. That is the point of keeping them: an analysis you cannot rebuild from raw is an analysis you cannot correct.

Part 2 · covers the funnel arithmetic and the three grains that lie

The funnel, and what a ratio is a ratio of

8 min live

Every metric a seller dashboard reports is a count or a ratio of two counts from the minute table. Once you can rebuild each one from the columns, you also know exactly which denominator it stands on, and that is what protects you from the three ways a live-commerce ratio lies.

The funnel of an average Lumen Glow stream, rebuilt from the minute table means over 60 streams; every number is a column sum or a ratio of two Views 9,049 sum(arrivals) Impressions 60,000 sum(impressions) Clicks 3,997 sum(clicks) Orders 484 sum(orders) GMV $16,139 sum(gmv) CTR 6.6% clicks / impressions CVR 12.1% orders / clicks $33.36 per order 22% take the $49 bundle GPM $1,750 gmv / views x 1000 Why impressions are bigger than views A viewer who stays nine minutes sees the pinned card nine times. Impressions count exposures, views count people arriving. So CTR is a per-exposure rate and it falls as watch time rises, even when nothing about the pitch changed. Why GPM and not GMV GMV is traffic times selling. GPM divides the traffic out. Two rooms, one of 400 and one of 1,400, can be judged on what they did with what they were given. Across these 60 streams GPM tracks CVR at 0.86; GMV tracks peak room at 0.96. Peak room tells you about the traffic a stream got. GPM tells you how well it sold it. Keep them on separate lines.
🔍 Click to zoom - the average stream's funnel, with the column arithmetic under every number.
LiveRebuilding the dashboard's numbers from columns4 min▶

The seller dashboard reports these per stream. Every one of them is a one-line aggregate of the minute table, which means you can recompute them, compare, and find out where your export and the platform disagree before you build anything on top.

Dashboard numberFrom the minute tableKind
Viewssum(arrivals)Count
Peak viewersmax(room)State
Average watch timesum(room) / sum(arrivals) minutesRatio of viewer-minutes to people
Product impressionssum(impressions)Count
Product clicks, CTRsum(clicks), / sum(impressions)Count, ratio
Orders, click-to-order rate (CTOR)sum(orders), / sum(clicks)Count, ratio
GMVsum(gmv)Count, in dollars
Watch GPM1000 * sum(gmv) / sum(arrivals)Ratio

Average watch time is the one people get wrong. It is viewer-minutes divided by viewers, which is the sum of the room column divided by the sum of arrivals. Averaging the room column gives you the average room size, a different number with a different meaning.

LiveThe three grains that lie4 min▶

Three mistakes account for most wrong conclusions in this field, and each one is a ratio computed at the wrong grain.

  • The per-minute ratio on a small denominator. A minute with 3 clicks and 2 orders has a 67 percent order rate. It means nothing. Session 5 uses orders per thousand viewer-minutes, whose denominator is the room, and weights ratios by their denominators when averaging them.
  • The per-stream comparison that is really a traffic comparison. Stream A made 30,000 dollars and stream B made 12,000. If A had three times the room, B sold better. Comparing streams on GMV is comparing them on how much traffic the platform sent. Session 6 is entirely about this.
  • The state summed like a count. Adding up room sizes gives viewer-minutes, which is useful and is not "total viewers". Reporting it as viewers overstates the audience by a factor of the average watch time, which here is about nine.
Real world

A brand-side report once claimed 74,000 viewers for a stream whose dashboard showed 8,200 views. Nobody had lied: somebody had summed the concurrent-viewer curve, which had been sampled every minute. Nine minutes of average watch time, nine times the audience. The deck went to a CFO before anyone checked the grain.

Self-studyWhat the platform reports, and what it does not4 min▶

TikTok Shop's LIVE analytics reports the stream-level figures in the table above under its own names (Views, Viewers, Peak Concurrent Viewers, Average Watch Duration, Product Impressions, Product Clicks, CTR, CTOR, GMV, Watch GPM, Items Sold), plus New Followers, Comments, Likes, Shares and a traffic-source split (Feed, Promoted, Follow). Its LIVE Diagnosis page reads the funnel in four stages: tap-through rate into the room, average viewing duration per view, CTR, then CTOR. One stage is missing from the minute table on purpose: tap-through rate needs the feed impressions of the LIVE card, which the platform holds and the recording never sees.

It also offers per-product breakdowns. What it does not give you is the minute-by-minute ledger joined to the content. The room curve is visible but not trivially exportable, orders carry a timestamp in the order export, and nothing in the platform knows what the host was saying. That gap is why the pipeline exists.

Where the platform's definitions and yours differ, yours must be documented and reconciled, not silently preferred. Two common gaps: the platform counts a view on entry while your recording starts a few seconds later, so the first minute is short; and the platform's GPM may be computed on a slightly different view count from the one you see. Neither matters for the analysis. Both matter the first time you put your number next to the platform's on a slide.

Build-along

Open the minute table and rebuild the funnel

★ 20 min · everyone builds

Two files, twenty minutes, pandas only. The 7,200-row minute table and the 60-row stream summary that the same generator produced. You rebuild every dashboard number from the minute rows, check them against the summary, and find the one that does not reconcile the way you expected.

Load both files and look at the grain. Twenty-six columns, one row per stream and minute. Check the key is unique before anything else: 60 streams times 120 minutes must be exactly 7,200 rows.

★ Load and check the key import pandas as pd m = pd.read_csv("data/streams.csv") # one row per stream_id, t s = pd.read_csv("data/summary.csv") # one row per stream assert len(m) == 60 * 120 assert not m.duplicated(["stream_id", "t"]).any() m.head(3)[["stream_id", "t", "segment", "room", "impressions", "clicks", "orders", "gmv"]]

Rebuild the funnel per stream from the minute rows. Counts are sums, the room is a state. Average watch time is viewer-minutes over arrivals, not the mean of the room column.

★ The funnel, one groupby f = (m.groupby("stream_id") .agg(views=("arrivals", "sum"), peak=("room", "max"), viewer_min=("room", "sum"), impressions=("impressions", "sum"), clicks=("clicks", "sum"), orders=("orders", "sum"), gmv=("gmv", "sum")) .reset_index()) f["watch_min"] = f.viewer_min / f.views f["ctr"] = 100 * f.clicks / f.impressions f["cvr"] = 100 * f.orders / f.clicks f["gpm"] = 1000 * f.gmv / f.views f.round(2).head()

Reconcile against the summary file. GMV, orders, clicks, impressions and views must match to the unit. Watch time and the ratios match to rounding. If anything is off by more than rounding, the grain is wrong somewhere and you stop here until you know where.

★ The check that replaces a README note chk = f.merge(s, left_on="stream_id", right_on="id", suffixes=("_m", "_s")) for col in ["views", "impressions", "clicks", "orders", "gmv"]: assert (chk[f"{col}_m"] == chk[f"{col}_s"]).all(), col assert (chk.watch_min - chk.avgWatchMin).abs().max() < 0.01 print("60 of 60 streams reconcile")

Now do the wrong thing on purpose. Compute the "audience" as the sum of the room column, and the "average order rate" as the mean of the per-minute orders-over-clicks ratio. Put them next to the right numbers. The first is about nine times too big. The second is undefined for every minute with no clicks and wildly noisy for the rest; the weighted version is the one the dashboard reports.

✗ Two grain mistakes, computed so you recognise them later wrong_audience = m.groupby("stream_id").room.sum() # viewer-minutes, not viewers right_audience = m.groupby("stream_id").arrivals.sum() print((wrong_audience / right_audience).mean()) # about 9: the average watch time m["cvr_min"] = m.orders / m.clicks # NaN where clicks == 0, 100% where 1 of 1 print(m.groupby("stream_id").cvr_min.mean().describe()) # unweighted mean of a noisy ratio print((100 * m.groupby("stream_id").orders.sum() / m.groupby("stream_id").clicks.sum()).describe())

Save the funnel table. It is the leader track's session 1 material and the base for session 5. Write it next to the raw files, never over them.

★ Derived, never raw f.to_csv("data/derived/funnel_by_stream.csv", index=False)

The ledger has a delay you have not corrected yet. In real recordings the seller's per-minute figures land a few seconds after the host's words reached the viewer, because the stream itself is delayed. Tonight's data is already on one clock. Session 2 shows what happens to every result if it is not.

Homework

Before session 2

30 min
  1. Export one real stream's data from a seller dashboard you have access to, or take any two-hour LIVE recording you are allowed to use. List which of the four sources you actually have, at what grain, and with what timestamp format. Bring the list.
  2. From the funnel table, rank the 60 streams by GMV and by GPM. Count how many streams are in the top ten on both lists. Write one sentence about what the streams in the GMV top ten but not the GPM top ten have in common. Session 2 of the leader track answers it.
  3. Compute average watch time for human hosts and AI hosts separately, the right way. Then compute the mean of the room column for both and notice which comparison changes.
  4. Read the columns you have not used yet: segment, product_in_hand, close_up_ratio, price_overlay, replies. For each, write down which of the four sources it will come from once the pipeline is real.
Check yourself

Three questions

A report sums the concurrent-viewer curve, sampled once a minute, and calls the result "total viewers". The dashboard shows 8,200 views and the report shows 74,000. What happened?

Room size is a state, not an event. Adding it across minutes gives viewer-minutes, which divided by the true view count is the average watch time. Nine minutes, nine times the audience.

Why does GPM, not GMV, belong on the line that judges how well a host sold?

Across the 60 streams GMV tracks peak room at 0.96 and GPM tracks conversion at 0.86. The first number is mostly about what the platform sent; the second is mostly about the pitch.

Product impressions for an average stream are 60,000 while views are 9,049. Is something broken?

A viewer who stays nine minutes is shown the card about nine times. It also means CTR is a per-exposure rate that falls as watch time rises even when the pitch is unchanged, which is why session 5 uses a per-viewer-minute rate instead.

Sources covered

What tonight maps to

✓ TikTok Shop LIVE analytics metrics: views, viewers, watch time, product impressions, clicks, orders, GMV, GPM Each rebuilt as a column aggregate of the minute table and reconciled against the stream totals.
✓ The minute-grain event model for multimodal stream data Four sources, native grains, aggregation to the minute, the five raw tables the minute table is rebuilt from.
✓ Counts versus states, and ratio denominators The three grain mistakes, each computed on purpose in the build-along so it is recognisable later.
◐ Clock alignment across sources Named tonight as the reason the ledger cannot be joined raw. Session 2 measures the stream delay and shifts every source onto one clock.
◐ Content columns: segment, product in hand, close-up, price on screen Present in the table tonight as reserved columns. Sessions 3 and 4 build the detectors and the tagger that fill them from real recordings.
· Order-level data: refunds, SKUs, buyer cohorts Not covered by design. The course works at the minute grain with GMV before refunds; buyer-level analysis belongs to the customer retention and e-commerce metrics courses.

Builder session 1 cheat sheet

The grainOne row per stream and minute. The coarsest grain at which a host can still change something.
Four sourcesVideo (frames), audio (words), comment feed (events), seller ledger (minutes or stream). Each blind to the others.
One join keystream_id and t. Every source is aggregated up to it; nothing downstream joins on anything else.
Counts sum, states do notArrivals, clicks, orders, GMV add up to the stream total. Room is end-of-minute and never summed as viewers.
Watch timesum(room) / sum(arrivals). Not the mean of the room column, which is average room size.
GPM1000 x GMV / views. Traffic divided out. Tracks CVR at 0.86 here; GMV tracks peak room at 0.96.
Impressions exceed viewsOne exposure per viewer per minute they stay. CTR is per exposure and falls as watch time rises.
Write the checkMinute sums must equal stream totals for every stream. An assertion beats a README sentence.