learn-prescriptive-analytics-with-phoebe / Leader session 6 of 6
Learn Prescriptive Analytics with Phoebe · Leader session 6 of 6

Reading someone else's optimum

Most of the optimal plans you will meet will not be yours. They will arrive finished, from a team or a supplier, with a recommendation and a number attached, and the useful skill is not building one but reading one. Five sessions at Ravensworth have produced exactly five questions worth putting to a plan you did not produce, and each one comes with the answer that should change your mind. The point is not that optimisation is untrustworthy. It is that the plan and the problem statement have to travel together.

🟡 Foundations Anyone handed a finished recommendation Anyone buying analytics from outside 45 min
0-3 · The handover 3-18 · Five questions 18-28 · Ravensworth's answers 28-45 · Build-along and Q&A
Part 0

What arrives, and what does not arrive with it

What arrives is a plan, a headline number and a recommendation. What almost never arrives is the problem it solved: the objective as written, which ceilings were measured and which were assumed, what each scarce thing was worth and over what range, and what demand assumption the whole thing rests on. Every one of those is answerable in a sentence by whoever built it, and asking for them is not scepticism. It is reading the thing properly.

Live = presented in the 45 minutes Self-study = the full depth, after class 🔍 Every diagram zooms

★ What you walk out with tonight. Five questions, short enough to remember without a card, each paired with the specific answer that should make you change your mind rather than nod. Ravensworth's own answer to all five, so you have heard what a good answer sounds like. And one sentence to carry into any meeting where the word optimal appears.

LiveWhy a finished plan is hard to argue with4 min▶

A recommendation with arithmetic behind it has a particular kind of authority. It is specific, it is internally consistent, and any objection raised from experience sounds like an objection to the arithmetic, which is the one part that is certainly correct. That is an uncomfortable position to be in for someone who has to sign it and who suspects something is off.

The way out is not to argue with the arithmetic, which will lose. It is to ask what problem the arithmetic solved. Every one of tonight's five questions is about the problem statement rather than about the computation, and all five can be asked by someone who could not derive a single line of the method.

Real world

Ravensworth's own plan is a good test case precisely because it is correct. The solver passes its checks, the plan is genuinely the best one available for the problem as written, and the five questions still find three things a decision maker would want to know before signing: two constraints were never separated into measured and assumed, the prices hold over a very short range, and the plan sells out about a quarter of the time.

Self-studyEverything the five questions are built from4 min▶
  • The objective. The single number being maximised, and whether it is the one the business wants. Session 1 put it in a box; session 5 showed what happens when it is wrong.
  • Binding and non-binding. Which ceilings the plan ran out of. Session 3: at Ravensworth, milling and premium protein bind, blending does not.
  • Measured against assumed. A ceiling that came off the plant floor and a ceiling that came off a commissioning document are not the same object, and the plan cannot tell them apart.
  • Shadow price. What one more unit of a binding limit is worth. Milling 33.7931, blending 0, premium protein 8.2759.
  • Validity range. How far that rate survives. Milling to about +0.46 hours, premium protein to about +0.77 tonnes.
  • The demand assumption. What the plan was optimised against, and what happens when the week is not that. Session 4: a shortfall of 325 to 338 and a 5th percentile around 3,540 to 3,591.
  • Stability. How far the plan moves when the objective shifts slightly. Cheap to test, rarely tested.
  • Degeneracy. Several plans tied for best, which would matter here if it were true. It is not: exactly one integer plan reaches 4,656.00.
Part 1 · covers the five questions

Five questions, and the answer that changes your mind

15 min live

A question you would nod at whatever the answer is not a question, it is a courtesy. Each of these five is paired with the specific reply that should stop the meeting, so that asking it commits you to acting on what comes back. They are in order, because the later ones only make sense once the earlier ones have been answered.

Five questions for a plan you did not build, each with its own stopping answer THE QUESTION WHAT WOULD CHANGE YOUR MIND 1 What is the objective, and is it the one we want? The single number being made as large as possible, in a sentence, with what it leaves out. CHANGE YOUR MIND IF it turns out to be a proxy that happened to be available, rather than the thing the business actually wants. 2 Which constraints bind, and which were assumed? A ceiling off the plant floor and a ceiling off a commissioning document are not the same object. CHANGE YOUR MIND IF a binding ceiling turns out to be somebody's estimate. The plan is resting its whole weight on that number. 3 What is each binding constraint worth, and over what range? Both halves. A price with no range attached is half a number, and the missing half gets multiplied. CHANGE YOUR MIND IF there is no range, or the quantity being discussed sits outside it. Then the business case is quoting the old world. 4 What demand is this optimal for, and what is the bad week? The assumption the plan rests on, and the 5th percentile of what it actually delivers. CHANGE YOUR MIND IF the answer is one average figure and nobody has run the plan against a spread. The headline is then untested. 5 What would the plan be if the objective shifted slightly? One more solver run. It converts a hidden assumption into a visible spread. CHANGE YOUR MIND IF the plan moves a long way. The objective is then the decision, and it deserves the attention going to the model. Every one of the five is about the problem statement, not the computation. None of them requires you to check the arithmetic.
🔍 Click to zoom - the right-hand column is what makes these questions rather than courtesies.
LiveWhy the order matters4 min▶

The objective comes first because everything downstream is conditional on it. There is no point discussing whether a milling hour is worth 33.7931 if the plan was maximising tonnage, because in that plan the figure means something else entirely. Similarly there is little point pressing on the demand assumption before knowing which ceilings are real, since a plan resting on an invented ceiling has a bigger problem than its percentiles.

Question five sits last for a different reason. It is the only one that is a test rather than a request for information, and it works best when you already know what answer you expected. If the objective is sound and the constraints are measured, a plan that moves a long way under a small change of objective is telling you something worth hearing.

LiveThe question about measured and assumed4 min▶

This is the one most often skipped, and it is the cheapest to ask. A solver treats every ceiling identically. It does not know that milling hours were timed on the line last month and that the protein figure came from a spreadsheet nobody has opened since the store was reorganised. Both arrive as numbers, both get respected exactly, and the plan pushes right up against whichever of them binds.

What makes it worth asking is the interaction with the binding set. An assumed number that is not binding costs nothing, because the plan never reaches it. An assumed number that binds is carrying the plan. Two questions in sequence, and the pair of answers tells you where the real risk in the model sits without anyone opening the code.

Self-studyQuestions that sound similar and are not useful4 min▶

"How confident are you in this?" invites a number that means nothing, because the plan has no interval and the honest answer is that the arithmetic is certain and the problem statement is not. "What is the accuracy of the model?" imports a word from prediction that does not apply to a plan. "Has this been validated?" gets a yes, because the solver was checked, which was not what you meant to ask.

All three fail the same way: they ask about the computation, which is the part least likely to be wrong, and they let the conversation end with reassurance. The five questions are deliberately about inputs and assumptions instead, because those are the parts that carry the risk and the parts a non-specialist can evaluate.

Part 2 · covers Ravensworth's answers

The five questions, asked of a plan that is correct

10 min live

Ravensworth's plan is right. The solver passes its checks, exactly one integer plan reaches 4,656.00, and nobody involved has done anything careless. Putting the five questions to it is therefore the fair test: this is what the answers look like when the work is good, and it is still a materially different conversation from the one that starts with "the optimal plan is worth 4,656".

Ravensworth Mills answers all five, and the plan was correct all along QUESTION 1 · THE OBJECTIVE Total margin across Standard, Dairy and Show, for one week. It says nothing about what the Show relationship is worth beyond Friday. QUESTION 2 · WHICH CONSTRAINTS BIND BINDING Milling hours, ceiling 131 BINDING Premium protein, ceiling 29 SLACK Blending hours, ceiling 77 0.2414 hours left unused, so its price is exactly 0. NOT ANSWERED Which of the three ceilings were measured and which were assumed is recorded nowhere. Two of them carry the plan. QUESTION 3 · WHAT IT IS WORTH, AND OVER WHAT RANGE WORTH PER UNIT THE RATE HOLDS TO Milling hours 33.7931 an hour +0.46 hours Premium protein 8.2759 a tonne +0.77 tonnes BOTH RANGES ARE SHORT Neither survives a purchase order. QUESTION 4 · WHAT DEMAND IT IS OPTIMAL FOR Optimal for demand fixed at a single average figure per product. Run against a spread, it delivers a mean of 4,318 to 4,331 against the promised 4,656.00, a shortfall of 325 to 338. The 5th percentile is roughly 3,540 to 3,591, and the plan sells out about a quarter of the time. QUESTION 5 · HOW FAR THE PLAN MOVES Change the objective and the plan changes with it. The box below runs that test live, on this page. An optimal plan is optimal for a stated problem. The question is almost always whether that was the right problem.
🔍 Click to zoom - four of the five have good answers. The fifth row of question 2 is the one nobody wrote down.
Live The real solver, running in this page. Pick a question and it runs the check on Ravensworth's own plan.

...

...
LiveQuestion 5, run properly4 min▶

Select the last option in the box. It nudges Dairy's margin up and down by small amounts and re-solves each time, which is exactly the check nobody runs and which costs one solver call per row. Read down the plan column and see how far the mix travels for a change in a single margin figure that most businesses would consider noise.

Whatever the result, the reason to run it is the same. Either the plan is stable, in which case the objective was not carrying much weight and you can stop worrying about it, or the plan is not stable, in which case the objective is the decision and it should be argued about by people rather than assumed by whoever filled the box in.

LiveWhat a good answer sounds like3 min▶

Notice what Ravensworth's answers are not. They are not reassurances, they are not confidence scores, and they are not a defence of the method. They are five short factual statements, four of which are genuinely satisfying and one of which is an honest gap. That mix is what a well-built piece of work looks like when it is read properly, and a plan that answers all five without a single gap is worth a second look for a different reason.

If the person who built it can answer all five in a couple of minutes, you are in good hands regardless of what the answers are. If the answers take a week to assemble, the problem statement was never written down anywhere, which is itself the finding.

Self-studyWhere serious people disagree5 min▶

Grant the five questions and there is still a live argument about who should be answering them, and it turns on what you think a handover is for.

One camp says the builder should ship the problem statement as part of the deliverable, every time, without being asked. Objective in a sentence, the binding set, every shadow price with its range, and the plan run against a spread. On this view a plan without its problem statement is an incomplete artefact, in the same way that a figure without units is, and the fact that the recipient can ask for it is no excuse for not providing it. Most recipients do not know to ask, and the ones who do are usually the ones who least need to.

The other camp says that shipping all of it every time is how the important parts get ignored. A deliverable carrying five appendices trains people to skim, and the caveat that matters this week drowns in four that do not. On this view the builder's job is to surface the one or two things that would actually change the decision, in the covering sentence, and the discipline is editorial rather than procedural. Publishing everything is not transparency, it is transferring the work of judgement onto somebody with less context.

Both camps have watched the other one fail. The first produces documents nobody reads; the second produces handovers where the omitted caveat turns out to be the one that mattered, and the builder's judgement about what to surface was the error. Notice that this is a disagreement about how organisations read, not about optimisation, and that the five questions work either way: they are the recipient's half of a contract that the builder may or may not have honoured.

Part 3 · build-along

Read a plan you did not build

17 min hands on

No code. Six steps, using the plan you brought from session 5 and the live box for the ones your own plan cannot answer. The aim is to leave able to run all five questions from memory, in a meeting, without notes.

Say the five out loud

Objective. Binding and assumed. Worth and range. Demand and the bad week. Stability. Say them in order, twice, then cover the page and say them again. They are meant to be recalled under pressure and not read off a card.

Put question 1 to your own plan

Write the objective of the recommendation you brought, in one sentence, as you believe it to be. Then write what it leaves out. If you cannot write the first sentence, that is the answer to the question.

Separate measured from assumed

List every limit in your plan and mark each one M or A. Then mark which ones the plan pushed right up against. Any cell that is both A and binding is where the whole recommendation is resting, and it took two minutes to find.

Ask for a range you will not be given

Find one per-unit figure in your plan and ask over what quantity it holds. Compare the answer with Ravensworth, where milling holds to +0.46 hours and protein to +0.77 tonnes. Note whether your figure came with any range at all.

Run question 5 on the mill

Use the last option in the live box and watch the plan move as the objective is nudged. Then say, for your own plan, whether anybody has ever run that test, and what it would have cost them to.

Write the covering sentence you would want

One sentence that should have arrived with the plan you brought: what it optimises, what binds, what the bad case is. If the plan had come with that sentence, say honestly whether any of tonight's questions would still have been needed.

The habit to keep. When a plan arrives with a number on it, ask for the problem it solved before you argue about the number. The arithmetic will be correct and the argument about the arithmetic is unwinnable. The problem statement is where every question worth asking lives, and five of them fit on the back of an envelope.

Check yourself

Three questions

A supplier presents an optimal plan. Which pair of answers tells you most about where the risk in it sits?

A solver treats a timed measurement and a guess identically, and the plan pushes right up against whichever ceiling binds. Those two questions asked in sequence locate the risk without anyone opening the code.

A plan quotes a binding resource at 33.7931 a unit. What must travel with that figure?

A shadow price is a local rate. Without its range it cannot be multiplied by anything, and the quantities people actually want to buy sit well outside the ranges measured here.

What does "this plan is optimal" actually assert?

The word is accurate and narrow. It carries no claim that the objective was the right one, that the ceilings were measured, that the rates survive being multiplied, or that the week will arrive as assumed. That is why the plan and the problem statement have to travel together.

After the leader track

Homework

Twenty minutes, on the next recommendation that reaches your desk.

Run all five, in order, and write the answers down

On a real plan, from a real team. Keep the answers on one page. The page is worth more than the plan, because it is the thing that will still make sense when somebody asks in six months why the decision was made.

Ship your own problem statement once

Take something your team is about to hand over and attach the covering sentence: what it optimises, what binds, what the bad case is. Then watch which of the three the recipient asks about, and whether they would have asked at all.

Covered tonight

What this session covers

✓The five questionsObjective, binding and assumed, worth and range, demand and the bad week, stability.
✓The answer that changes your mindEach question paired with the reply that should stop the meeting.
✓Measured against assumedA solver respects a guess exactly as hard as a measurement, and cannot tell them apart.
✓Both validity rangesMilling holds to +0.46 hours, premium protein to +0.77 tonnes. Both short.
✓Ravensworth answered in fullFour good answers and one honest gap, on a plan that was correct throughout.
✓The one-line versionAn optimal plan is optimal for a stated problem, and that is the thing to examine.
◐Stability under a changed objectiveRun live tonight on Ravensworth as a check, not derived as a method.
◐DegeneracyNamed as a concept that would matter here if it applied. Exactly one plan reaches 4,656.00.
○How shadow prices and ranges are computedThe dual and sensitivity analysis belong to the practitioner track.
○Writing the problem down yourselfDecision variables, objective and constraints as a formulation start the practitioner track.
○Choosing which decision to make at allOptions and framing live in the decision intelligence course.
Take this with you

Cheat sheet

The one sentence. An optimal plan is optimal for a stated problem, and the question is whether that was the right problem.
Question 1. What is the objective, and is it the one we want. Change your mind if it is a proxy.
Question 2. Which constraints bind, and which were assumed. Assumed and binding is where the plan rests.
Question 3. What is each binding constraint worth, and over what range. Milling +0.46 hours, protein +0.77 tonnes.
Question 4. What demand is it optimal for, and what is the 5th percentile. Ravensworth: roughly 3,540 to 3,591.
Question 5. How far does the plan move if the objective shifts. One solver run, and rarely done.
Do not ask. How confident are you, what is the accuracy, has it been validated. All three end in reassurance.
The handover rule. The plan and the problem statement travel together, or the plan is incomplete.