learn-prescriptive-analytics-with-phoebe / Practitioner session 5 of 10
Learn Prescriptive Analytics with Phoebe · Practitioner session 5 of 10

Integers change the answer

The mill cannot bag 30.6897 tonnes of Standard, and the obvious repair is to round it. That repair is usually described as harmless and here it is neither small nor in the right direction: rounding down gives 30 tonnes and 4,596.00, the true whole-number optimum is 33 tonnes and 4,656.00, and the rounded plan has moved away from the answer rather than towards it. Tonight is the method that actually finds 33: solve the easy problem, split on a fraction, and throw away every branch that cannot win.

🟠 Working depth Anyone whose quantities are countable Anyone who has rounded an optimum 45 min
0-3 · Setup 3-17 · Relax, branch, bound, prune 17-29 · What rounding costs 29-45 · The branch that died, and build-along
Part 0

Three plans, and only one of them is the answer

Put the three side by side before any method is discussed, because the numbers make the case better than an argument does. The continuous optimum is 30.6897 Standard, 64.8276 Dairy, no Show, worth 4,666.90. Round it down and you get 30, 64, 0, worth 4,596.00, and it is perfectly feasible. The best whole-number plan is 33, 63, 0, worth 4,656.00. Rounding gave up 60.00 of margin, and it did so while moving Standard down when the right move was up.

Live = presented in the 45 minutes Self-study = the full depth, after class 🔍 Every diagram zooms

★ What you walk out with tonight. Branch and bound as four moves you can perform on paper: relax, branch, bound, prune, and the reason pruning is what makes it finish rather than what makes it fast. The measured cost of rounding at Ravensworth, 60.00 and in the wrong direction. And a real defect in the solver on this page, which is that a lower bound written the natural way produced a negative right hand side, the simplex refused it without saying so, and every upward branch in the search died in silence for a while.

LiveThe one-paragraph version4 min▶

Drop the whole-number requirement and solve the easy problem. If the answer happens to be whole, you are finished. If some variable comes back at 30.6897, no whole-number plan can sit between 30 and 31 for that variable, so split the problem in two: one where it is at most 30, one where it is at least 31. Solve each. Each relaxed answer is an upper bound on everything inside that branch, so the moment a branch's relaxed answer falls below the best whole plan you have already found, the whole branch can be discarded without being explored. Keep splitting what is left. Ravensworth finishes in 29 nodes.

Real world

The rounded plan is not wrong because it is worse. It is wrong because it is confidently in the wrong place. A planner handed 30.6897 will write 30 or 31 on the sheet and reasonably believe the answer is within a tonne. The answer is 33, three tonnes away and in the other direction, and nothing in the continuous output hints at it.

Self-studyThe words tonight adds4 min▶
  • Integer programme. The same objective and constraints, plus a requirement that some or all variables be whole numbers.
  • Relaxation. The same problem with the whole-number requirement dropped. Easy to solve and always at least as good as the real answer.
  • Bound. The relaxed value of a branch. Nothing inside that branch can beat it, which is what makes it useful.
  • Incumbent. The best genuinely whole-number plan found so far. It only ever improves.
  • Prune. Discard a branch whose bound cannot beat the incumbent. No solving, no exploring, no apology.
  • Node. One subproblem in the search. Ravensworth takes 29 of them.
  • Gap. The distance between the best bound and the incumbent. When it closes, the incumbent is proved optimal.
  • Origin shift. Applying a lower bound by substituting x = L + x' rather than by adding a row. Tonight it is the difference between a search that works and one that quietly does not.
Part 1 · covers branch and bound

Relax, branch, bound, prune

18 min live

There is no version of simplex that respects whole numbers, and there is no clever pivot rule that would make one. The whole-number requirement is not a constraint the tableau can hold, because it is not linear and it is not about a ceiling. So integer problems are solved by an outer method that calls the linear solver many times, and the interesting design question is not how to solve each piece but how few pieces you can get away with solving.

One linear problem becomes a tree of linear problems, and most of the tree is never visited ROOT · SOLVE THE RELAXATION 4,666.90 Standard comes back at 30.6897, which the mill cannot bag BRANCH DOWN Standard <= 30 solve this as its own relaxation BRANCH UP Standard >= 31 solve this as its own relaxation no whole plan lives between 30 and 31 STILL FRACTIONAL Another variable came back with a fraction in it, so this node splits again, on that variable. PRUNED Its relaxed value is already below the best whole plan found, so nothing inside it can win. Never explored. The four moves RELAX Drop whole numbers. Solve what is left. BRANCH Pick a fractional variable. Split into at most, and at least. BOUND A branch can never beat its own relaxed value. PRUNE Bound below the incumbent? Discard it. Ravensworth closes in 29 nodes, and the plan that survives is 33 Standard, 63 Dairy, no Show, worth 4,656.00. Pruning is not an optimisation of the search. Without it the tree is every combination of whole numbers, and it does not finish.
🔍 Click to zoom - the root value 4,666.90 is an upper bound on every whole plan that exists. The answer 4,656.00 sits below it, and it has to.
LiveWhy the relaxation is always an upper bound4 min▶

Every whole-number plan is also a legal fractional plan, because a whole number is a number. So the set of plans the relaxation searches contains the set of plans the integer problem searches, and the best of a larger set is at least as good as the best of a smaller one. That one sentence is what the entire method rests on, and it is worth being able to say it without hedging, because it is also what lets you prune.

Read Ravensworth's two numbers with that in hand. The relaxation says 4,666.90 and the whole-number answer is 4,656.00. The second must be at most the first, and the gap between them, 10.90, is the price of being unable to bag fractions of a tonne. That gap is the honest cost of integrality, and the rounded plan's shortfall of 60.00 is something else entirely: it is the cost of a bad method.

LiveWhat the code does at each node5 min▶

The recursion carries two arrays, lo and hi, holding the current bounds on every variable. At each node it builds the node's own linear problem, solves it with the same simplex from session 3, and then does three tests in order. If the status is not optimal, return. If the bound cannot beat the incumbent, return, which is the line if (bound <= bestObj + 1e-7) return; and it is the prune. If every integer variable came back whole, this is a new incumbent, so record it and return.

Only if all three tests fall through does it branch: take the first variable that is still fractional, and call itself twice, once with that variable's upper bound set to Math.floor of the fractional value and once with its lower bound set to Math.ceil. There is a node counter with a ceiling of 6,000 and a depth limit of 60, both of which are guards rather than parts of the method, in the same spirit as the iteration counter inside the simplex.

Self-studyWhy the order of exploration matters so much4 min▶

Pruning needs an incumbent to prune against, and at the start of the search there is not one. Until some branch runs all the way down to a whole-number plan, bestObj is negative infinity and no branch can be discarded, so the early part of the search is pure exploration with no savings at all. This is why the order matters: find a decent whole plan early and a large fraction of the tree evaporates, find one late and you paid to walk it.

This implementation takes the first fractional variable it finds, and explores the down branch before the up branch, which is a choice with no theory behind it. Choosing the variable that most changes the bound, or diving towards a feasible plan first and only then searching properly, are both standard and both cost something per node. The node count you see is a property of those choices as much as of the problem.

Part 2 · covers what rounding actually does

Rounding is not a rounding error

16 min live

The case against rounding is usually made as a warning about feasibility: round up and you may break a ceiling. That is real and it is the easy half. The harder half is visible here, where rounding down is perfectly feasible, loses 60.00, and puts Standard at 30 when the answer is 33. The rounded plan is not a slightly degraded version of the right answer. It is in a different place.

The same mill, three answers PLAN STANDARD DAIRY SHOW MARGIN STATUS Continuous optimum the relaxation, solved by simplex 30.6897 64.8276 0 4,666.90 cannot be bagged Rounded down what people actually do 30 64 0 4,596.00 feasible, 60.00 worse Integer optimum branch and bound 33 63 0 4,656.00 found in 29 nodes Tonnes of Standard, on a line 29 30 31 32 33 34 30.6897, THE RELAXATION 30, WHERE ROUNDING LANDS 33, THE ANSWER Rounding moved Standard to 30. The answer is 33. The repair travelled away from the thing it was repairing.
🔍 Click to zoom - the amber arrow is what rounding does, and the answer is off to the right of it. Feasible and 60.00 short.
Live The real solver. Three answers computed here, now, plus the two ways of writing a lower bound.

...

...
LiveWhy rounding can point the wrong way at all4 min▶

The continuous optimum sits on a corner where two ceilings meet exactly. Chop a fraction off one variable and the plan is no longer on that corner, and the capacity you just released is not free, it is stranded. Nothing puts it back to work, because rounding is not an optimisation, it is an edit. The whole-number optimum is a genuinely different corner of a genuinely different problem, found by searching, and there is no reason for it to be near the continuous one.

Here the search discovers that giving up a tonne of Dairy, 64.8276 down to 63, frees enough milling and protein to add nearly three tonnes of Standard, and the swap pays. That trade is invisible from the continuous answer, and rounding cannot find it because rounding never re-solves anything.

LiveWhen rounding is fine, honestly3 min▶

If the mill were planning 30,689 tonnes rather than 30.6897, one tonne either way would be a rounding error in the ordinary sense, and nobody should spend a search on it. The damage from rounding scales with how large one unit is relative to the quantities. Small numbers of large things are where integrality bites, and Ravensworth's whole-tonne batches are exactly that case.

The test is cheap enough that you do not have to guess. Solve both, compare, and keep the comparison in the write-up. If the gap is negligible on your problem, you now know rather than assume, and you have a number to point at when somebody asks why the model does not respect whole units.

Part 3 · covers a real defect in this solver

The branch that died in silence

10 min live

Branching up means forcing a variable to be at least 31, and the natural way to write that is as another constraint row. Multiply through by minus one to get it into the at-most form the solver takes, and you have -x <= -31. The right hand side is negative. This simplex begins by scanning the right hand sides and refuses anything negative, because a negative ceiling destroys the free starting corner. It returns a status instead of throwing, the branch function returns early on any non-optimal status, and so every upward branch in the entire search vanished without a message.

The same branch, written twice AS A ROW, WHICH LOOKS RIGHT AND IS NOT Force Standard to at least 31 by adding a row: A.push([-1, 0, 0]); b.push(-31); The right hand side is negative, so the very first loop in solveLP returns before a single pivot happens: status: "needs-phase-one" The branch function returns early on any status that is not optimal. No error, no warning, no node counted. The search simply never goes up. down, alive up, dead and silent, every time AS A SHIFT OF THE ORIGIN, WHICH WORKS Force it by substitution instead: Standard = 31 + Standard' Take 31 lots of that product's rate out of every ceiling: b2[i] = b[i] - A[i][0] * 31 Every right hand side stays non-negative, so the free starting corner survives and the node solves normally. Add the 31 back afterwards, to the answer and to the objective, and the node is the node it was meant to be. down, alive up, alive, and it is the one that finds 33 A solver that only searches downward returns a confident, plausible, smaller number. Nothing about the output looks wrong. There is no exception, the status is optimal, the node count is a normal number, and the plan is feasible. The fault was invisible to a source read and invisible to the three hand-checkable problems the solver ships, because none of them needed an upward branch. Session 9 tells the whole story.
🔍 Click to zoom - the left panel is not a typo, it is the encoding almost everyone writes first. The refusal was correct; the silence was the defect.
The three answers, and both encodings of a lower bound
var c = [38, 54, 71];
var A = [[1.1, 1.5, 2.3], [0.6, 0.9, 1.7], [0.1, 0.4, 1.0]];
var b = [131, 77, 29];

OPT.solveLP(c, A, b).objective;        // the relaxation, an upper bound on everything
OPT.solveMILP(c, A, b).objective;      // the whole-number answer
OPT.solveMILP(c, A, b).nodes;          // how much of the tree it had to visit

var lp = OPT.solveLP(c, A, b);
var rd = OPT.roundDown(lp, A, b);      // { x: [...], feasible: true }
OPT.objectiveOf(c, rd.x);              // what the edit costs

// a lower bound of 31 on Standard, written as a row
OPT.solveLP(c, A.concat([[-1, 0, 0]]), b.concat([-31])).status;

// the same bound, written as a shift of the origin
var b2 = b.map(function (bi, i) { return bi - A[i][0] * 31; });
var shifted = OPT.solveLP(c, A, b2);
shifted.objective + 38 * 31;           // add the shifted margin back on
LiveWhy this class of fault is so hard to see4 min▶

Three things lined up. The refusal was legitimate: a negative right hand side really does need a preliminary phase this solver does not implement, and returning a status was the honest thing to do. The caller's early return on a non-optimal status was also reasonable, because a node that cannot be solved is usually a node with no feasible plans in it. And the output of a search that only ever goes down is a real, feasible, whole-number plan with a normal-looking objective. Every individual decision was defensible and the combination was wrong.

Note what would not have caught it. Reading the code catches it only if you are already thinking about signs. The three shipped self-test problems do not catch it, because none of them has an optimum that requires branching upward. A unit test asserting that solveMILP returns an optimal status passes. What caught it was solving many generated problems a second way and comparing, which is the subject of session 9.

Self-studyWhere serious people disagree5 min▶

Given a model with fractional quantities, should you enforce integrality at all? Practitioners split, and not along a line of sophistication.

One camp treats integrality as something to avoid wherever it can honestly be avoided. Solving the relaxation is fast and predictable; solving the integer version can take exponentially longer and the time is impossible to forecast, which is fatal for anything that has to run on a schedule. In most operational settings the quantities are large enough that a unit is noise, the input data are far less precise than the fraction being argued about, and a plan that arrives late is worth nothing however optimal it is. On this view, add integrality only after demonstrating that rounding actually costs something.

The other camp holds that the demonstration is the whole point and it is cheap to run, so run it. Rounding does not merely lose a little, it can land in a different part of the space, as it does here, and it silently hides trades the search would have found. Worse, a great many real decisions are not quantities at all but yes-or-no choices, where there is no such thing as rounding and the relaxed answer is meaningless on its own. On this view, treating integrality as optional is a habit formed on problems where it happened not to matter.

They agree on more than they appear to: both would solve both and compare. They differ on which one is the default when the comparison has not been done yet, and that default decides what ships when somebody is in a hurry.

Part 4 · build-along

Find 33 yourself, then break the branch on purpose

14 min hands on

Six steps on the mill. The aim is that branch and bound stops being a name and becomes something you could sketch on a whiteboard, and that a silently dead branch is a failure mode you would now look for.

Round it yourself before you are shown

Take 30.6897 and 64.8276 and write the whole-number plan you would send to the floor. Check with OPT.isFeasible that it does not break a ceiling, then compute its margin with OPT.objectiveOf. Write the number down before running anything else.

Solve it properly and read the gap

Call OPT.solveMILP(c, A, b) and compare the plan and the margin with yours. Then work out two separate differences: 4,666.90 against 4,656.00, which is the cost of whole tonnes, and 4,656.00 against 4,596.00, which is the cost of your method. Say which one you could have avoided.

Sketch the first two nodes on paper

Root relaxation, Standard at 30.6897, so the children are Standard at most 30 and Standard at least 31. Write both child problems out in full, with their own A and b. You will need the second one written correctly for step 5.

Watch the node count move

Run solveMILP on the mill and note 29 nodes. Then change one margin, say Show from 71 to 90, and run it again. The node count is a property of the problem and of the search order together, and seeing it swing is the fastest way to stop treating it as a constant.

Write the up branch the wrong way, deliberately

Add the row [-1, 0, 0] with right hand side -31 and call solveLP. Read the status. Then write the same bound as a shift of the origin and call it again. Two encodings of one constraint, one of which returns a plan and one of which returns a refusal that a careless caller will read as "nothing here".

Say what would have caught it

In one sentence: what test, written before the bug existed, would have failed on the downward-only search. It has to be a test that needs an upward branch to pass, which means constructing a problem whose optimum lies above the relaxation's fractional value. Bring your sentence to session 9.

The habit to keep. When a solver returns a status rather than an answer, find out what the caller does with it. A status that is handled by returning early is indistinguishable, from the outside, from a branch that genuinely contained nothing. Any place in a search where "could not solve" and "nothing to find" are treated the same way is a place where a real answer can be thrown away without anybody hearing it happen.

Check yourself

Three questions

The continuous optimum is 30.6897 tonnes of Standard. The rounded plan uses 30 and the integer optimum uses 33. What does that tell you about rounding?

Rounding is an edit, not an optimisation. Chopping a fraction off strands the capacity it releases. Branch and bound gives up a tonne of Dairy and gains nearly three of Standard, which pays, and neither 30 nor 31 is anywhere near that plan.

Why can a branch be discarded without being explored?

Every whole-number plan is also a legal fractional plan, so the relaxation searches a larger set and its optimum is at least as good. That makes the relaxed value a ceiling on the branch. Pruning against the incumbent is what stops the tree being every combination of whole numbers, which is why the method finishes at all.

A lower bound of 31 on Standard is added as the row -x1 <= -31 and the search then returns a feasible, whole-number, optimal-looking plan. What is the danger?

The refusal is correct in itself, because a negative right hand side destroys the free starting corner and needs a preliminary phase this solver does not have. The defect is that the refusal is silent at the point where it is consumed: the caller cannot tell a node it could not solve from a node with nothing in it. Substituting x = L + x' keeps every right hand side non-negative and the branch alive.

Before session 6

Homework

Thirty minutes, console and paper.

Build a problem where rounding is fine

Keep Ravensworth's rates and margins and multiply all three ceilings by a thousand. Solve continuous, round down, and solve as integers. Report the gap as a share of the margin and compare with tonight's. Then say, in one sentence, what actually changed, given that the model is otherwise identical.

Write down a model you are sure is right

Take any small decision from your own work and write it as an objective plus constraints. Bring it to session 6 with a note of anything you were unsure how to express, especially anything you wrote as a preference because you could not see how to write it as a limit.

Covered tonight

What this session covers

✓Why simplex cannot do whole numbersIntegrality is not a ceiling, so it cannot live in the tableau.
✓The relaxation as an upper bound4,666.90 is a ceiling on every whole plan, including the 4,656.00 that wins.
✓Branch and bound, the four movesRelax, branch on a fractional variable, bound the branch, prune what cannot win.
✓Ravensworth's integer optimum33 Standard, 63 Dairy, no Show, worth 4,656.00, found in 29 nodes.
✓What rounding costs30 / 64 / 0 at 4,596.00. Feasible, 60.00 worse, and pointing the wrong way.
✓Two costs, told apart10.90 for whole tonnes, 60.00 for the method. Only one of them was necessary.
✓The negative right hand side faultA lower bound as a row is silently refused, so every upward branch died.
✓The origin shift that fixes itSubstitute x = L + x' and every right hand side stays non-negative.
◐Search order and node countFirst fractional variable, down before up. A choice, and the node count reflects it.
◐Yes-or-no decisionsNamed as the case where rounding has no meaning at all. Session 6 takes the mechanism.
○Cutting planesTightening the relaxation instead of splitting it is a different family, not taught.
○Shadow prices for integer problemsThere is no clean dual once the variables are whole. Session 4's machinery is for the relaxation.
○How the fault was foundThe cross-check over generated problems, and the checker that was also wrong, are session 9.
Take this with you

Cheat sheet

The three answers. Continuous 30.6897 / 64.8276 / 0 at 4,666.90. Integer 33 / 63 / 0 at 4,656.00. Rounded 30 / 64 / 0 at 4,596.00.
Rounding costs 60.00 here. And it moves Standard to 30 when the answer is 33.
Relax. Drop whole numbers, solve, and you have an upper bound on everything.
Branch. Pick a fractional variable at 30.6897. Split into at most 30 and at least 31.
Bound and prune. A branch below the incumbent is discarded unexplored. This is what makes it finish.
29 nodes. The node count belongs to the problem and the search order together, not to the problem alone.
The trap. A lower bound as -x <= -L has a negative right hand side, which this simplex refuses in silence.
The fix. Substitute x = L + x', subtract L lots of the rate from every ceiling, add L back at the end.