See it: what the camera shows

A detector budget is easier to justify with the number it protects. In the Lumen Glow data, comparing minutes within the same stream so the room size and the host cancel out, the visual columns move the click-through rate on the pinned card: a minute with the product in hand runs +1.38 points of CTR, a minute of close-up runs +1.01, and a minute with the price on screen runs +0.97. Those are among the largest within-stream CTR effects in the table; only the CTA and demo segments beat them.

The same three columns do almost nothing to the click-to-order rate: product in hand +0.04 points, close-up +0.46, price overlay -0.13, none of them distinguishable from zero. The camera earns the click. What the host says earns the order. Session 4 measures that half.

The bench in session 8 turns the click effect into money. Cueing the host to have the product in hand on every CTA is worth +9.7 percent of GMV on a Friday night, as a mean over 40 seeded replays. You cannot cue what you cannot measure, and you cannot measure it without tonight's column.

• Closed-vocabulary detection - a detector trained on a fixed list of classes and blind to anything else. YOLO26's COCO weights know "person" and "bottle" and have never heard of Dew Serum.
• Open-vocabulary detection - a detector that takes a text prompt at run time and finds whatever the prompt names. "serum bottle", "pump dispenser", the SKU name on the label. No training, one string.
• Landmark - a named keypoint on a body part. MediaPipe's HandLandmarker returns 21 per hand, PoseLandmarker returns 33 per body, each with normalised x, y and a depth estimate.
• Zone - a polygon on the frame that reports which detections fall inside it. supervision's PolygonZone is the standard implementation.
• Tracker id - a stable integer attached to one object across consecutive frames, so "the same bottle" can be followed. ByteTrack is the usual algorithm.
• OCR - optical character recognition, reading text off pixels. Florence-2's OCR_WITH_REGION task returns each string with its box.
• Licence: AGPL-3.0 - copyleft that reaches over the network. Software that uses an AGPL component must offer its own source to users, including users who only reach it through a server. Apache-2.0 and MIT are permissive: use, modify, ship closed, keep the notice.

Ultralytics' own licence page draws the line. AGPL-3.0 fits research, personal and fully open-source use. The Enterprise licence is required for commercial products, closed-source internal tools, and SaaS or APIs with a YOLO backend, and its pricing is quote-based. YOLOE, the open-vocabulary model, ships inside the same package, so the same terms apply to it.

Now place the thing you are building. An MCN's stream-quality dashboard, run on the MCN's own servers for the MCN's own coaches, is a closed-source internal tool. That is on the Enterprise side of the line by the licensor's own wording. So the choice is one of three, and it is made before a single frame is processed:

• Buy Enterprise. Get the quote, budget it, ship YOLO26 and YOLOE with a clear conscience.
• Open-source the pipeline. AGPL is satisfied if the code that uses YOLO is offered to its users. For an internal tool that means publishing it, which some MCNs will not do and most brands will not allow.
• Use Apache-2.0 and MIT models. MediaPipe for landmarks, Grounding DINO or OWLv2 for the SKU, Florence-2 for OCR, supervision and trackers for the glue. Tonight's build-along takes this route, and the accuracy it gives up is smaller than the licence it keeps.

Each column is one rule applied per frame, then one aggregation per minute. The rules are deliberately simple; the sophistication is in the detectors underneath, not in the arithmetic on top.

The one that needs care is close-up. A host who leans in until only her face and the bottle fill the frame has no hips in shot, so a naive "person box" from pose landmarks shrinks to the visible points and the ratio drops when it should rise. Treat missing hips as a signal, not as missing data: shoulders-and-up with no hips is a close-up by construction, and the ratio is computed from the shoulder-to-crown box scaled to the frame.

Both quoted from the fetched documentation. The Grounding DINO call is from the transformers v5.17.0 model docs; note the kwarg name, because the model card on the Hub still shows the older box_threshold while the current processor takes threshold. Signature defaults are 0.25 for both thresholds; the docs' example raises them to 0.4 and 0.3.

MediaPipe's task options all share one shape: a base_options pointing at a model file, a running mode, and a handful of confidence thresholds. The hand landmarker's, as documented:

For a host who sells with both hands, set num_hands=2. The landmarks are normalised to the frame, so a box from the detector in pixels has to be divided by the frame width and height before the two are compared, or the landmarks multiplied up. Pick one convention and assert it in the code.

• Booleans become shares. A detector says true or false per frame. Sixty of those in a minute are a proportion, and the proportion is what carries information: a minute where the product was in hand for 55 frames is different from one where it was for 12. Round it to 0 or 1 only when a downstream method needs a flag, and keep the share beside it.
• Ratios become means. Close-up ratio is already a number per frame; average it over the usable frames. A median is defensible too, but the mean is what the course table carries, so use the mean to reconcile.
• Events become counts. Scene cuts are boundaries; a minute has an integer number of them. Session 2 built this one.
• Thin minutes become null. If fewer than 30 of the minute's 60 frames were usable, whether because the detector crashed, the frames were black, or you sampled down for speed, write null. A share of 3 out of 4 frames is not a share, it is a coin flip with a decimal point. Session 5 skips nulls; it cannot skip a zero it does not know is fake.

The course table stores product_in_hand and price_overlay as 0 or 1 per minute, which is the share rounded at one half. Your own pipeline should keep both: the share for modelling, the flag for reconciling against the table and for the cue rules in session 8.

Nobody hands you labelled frames of your host. You make them, and it is cheaper than it sounds. Sample 200 frames at random across streams, hosts and hours; sit with a coffee and mark each one by hand for each boolean column and a rough close-up bucket. Then run the detector on the same 200 and report precision and recall per column, not one accuracy number: a detector that never says "in hand" has 90 percent accuracy on a stream where the product is rarely held, and is useless.

Keep the 200 labels in the repository. They are the regression test for every future change of model, threshold or prompt, and they are the answer when someone asks how much to trust a column. A detector whose recall on price cards is low will understate the price-overlay effect, and knowing that before session 6 is worth more than a better model after it.

Two of the six Lumen Glow hosts are AI avatars, and the camera columns spot them before anyone reads the host name. Their scene cuts sit at 1 to 3 per minute against 2 to 7 for the human hosts, because an avatar has no operator switching to the product camera. Their speech rate holds at a steady 158 words a minute with almost no variance, where a human host's rate moves with the segment and collapses in dead air. The columns are honest about it: the avatars are consistent, and consistency is not what sells.

On TikTok Shop this is a compliance question before it is an analytics question. The Requirements for High-Quality Videos and LIVEs, effective 2026-09-17 in the US, say "Do not use non-real-time verbal interaction such as AI-generated voices, audio recordings, or radio" and "Do not use animated figures or content that covers more than 50% of the screen." A fully AI-voiced avatar host is not permitted on a promotional LIVE there; the AI streams in the data show what the trade would be on a platform that allows it. Leader session 4 covers the rule; tonight's close_up_ratio is, incidentally, the measurement of "more than 50% of the screen".

There is a fourth way to get columns from frames: send the video to a vision-language model and ask it. Gemini 3.8 Flash reads video at about 100 tokens per second at low resolution and is priced at $0.75 per 1M input tokens under promotional pricing that runs through 2026-12-31, which works out to about $0.006 per minute of video. Session 4 uses it as a segment labeller, where a model that understands "she is now doing the price pitch" earns its keep.

For tonight's columns it is the wrong tool for three reasons. A VLM returns a sentence, not a box, so "in hand" becomes a judgement you cannot audit against a zone margin. Its answer per frame is not stable across runs, so the 200-frame validation drifts. And the price is per minute for as long as you keep running it, where a local Apache-2.0 landmarker costs the electricity. Use the VLM for what needs language, the detector for what needs geometry, and keep the receipts for both.