The Challenge Problem
MLB gave every team two chances a game to overrule the umpire. Players used them on a blurry read of the zone, often didn't act on what they saw, and barely adjusted for the stakes. Optimal play on the same pitches recovers 36.0 more wins across the league.
2026 regular season716,792 pitches10,558 challenges
The automated ball-strike challenge system arrived in 2026 with simple rules. Each team starts a game with two challenges. Only the batter, pitcher or catcher can ask for one, immediately after the call. A correct challenge is kept and a wrong one is lost. Under those rules a challenge works like currency: spending it on a coin flip in the 2nd inning means you may not have it in the 9th.
This study treats the challenge as a sequential decision under uncertainty. It asks three questions of the full 2026 season: how well players actually see the zone, what a challenge is worth at each moment of a game, and how far real teams fell from the best possible policy. The accompanying paper has the formal model, proofs and full statistical checks. This page tells the story.
- The ABS zone can be reconstructed exactly. A pitch is a strike if any part of the ball touches a 17-inch-wide box running from 27% to 53.5% of the batter's height. That rule matches 99.95% of the season's challenge rulings, so every umpire miss in 2026 is known, not estimated.
- Catchers see the zone 34% more precisely than batters, with perception noise of 1.65 inches against 2.51. Batters lose precision on velocity and breaking balls, and both groups see better when the stakes rise.
- The optimal rule is a cutoff that falls with the stakes and with the clock. We prove that, for a given number of challenges held, a challenge's value can only be expected to fall as a game goes on, so the bar for spending one should fall too.
- Half the gap is not acting on what players see; a third is ignoring the stakes. Optimal play would challenge 1.7× less in low-stakes spots and 2.3× more in high-stakes ones. A Shapley decomposition puts 52% of the gap on attention and 36% on the response to stakes.
- A three-number card closes 48% of the gap. "Be 40% sure on low-stakes pitches, 25% on medium, 10% on high" would have added 18.5 wins league-wide. The gap survives six alternative models of how players see the zone, though its size depends on how much non-response is inattention.
- Teams are hard to tell apart, and outcome-based leaderboards are mostly noise. Every team fell short, but the differences between teams are borderline at best, and player challenge value has low split-season reliability.
Players see a blurry zone
For every called pitch, one side has a reason to challenge: the batter on a strike, the catcher or pitcher on a ball. Call w the distance, in inches, by which the call was wrong from that side's point of view. Positive means ABS would overturn it, negative means the umpire was right.
If players saw the zone perfectly, the challenge rate would jump from 0 to 100% at w = 0. Instead it rises slowly. I model each player's read as the true miss plus Gaussian noise; this is a signal-detection model. The player challenges when that read clears a threshold, and an attention parameter covers clear misses that nobody reacts to. Fit across 345,365 opportunities, the model tracks the observed rates closely.
Challenge rate by how wrong the call was
Dots are observed rates in half-inch bins; lines are the fitted perception model.
What blurs the read
Batters' noise rises with velocity and on breaking balls, while catchers' doesn't, since they see the full flight into the glove. Both groups are worse on the top and bottom edges than on the sides, and both see more sharply when the stakes rise. On the threshold side the clearest effect is salience. Holding the stakes constant, batters lower their bar by 3.33 inches with two strikes.
Every piece of the model earns its place. Each block of terms passes a likelihood-ratio test (all p ≤ 10⁻⁴), and BIC improves at every step. Out of sample, the 14-parameter model comes within 1.8% of gradient-boosted trees on log-loss and is the best calibrated for batters.
What a challenge is worth
To price a call, I built run-expectancy and win-expectancy models from 2026 data alone. The win model matches Statcast's published win expectancy (Brier score 0.1590 against 0.1591). The stake of a call, g, is the win-probability gap between that pitch being a ball and being a strike. Measured in runs, it rises from 0.088 at 0-0 to 0.604 at 3-2.
A challenge left in your pocket is an option on future misses. I solve for its value by backward induction. The model tracks challenges held, the score margin (which changes between innings), and extra innings, where a team with none left gets one back at the start of each extra inning (MLB's rule). The value of holding k challenges rather than k−1 is the challenge's shadow price, λ.
p is how likely you are to be right given what you saw, g what the call is worth, λ what the challenge is worth if you lose it. A correct challenge costs nothing because you keep it, so you should challenge when you are more than λ / (g + λ) sure. The paper proves that λ is concave in the number of challenges held and that, for a given number held, its expected value only falls as the game goes on.
The shadow price of a challenge, inning by inning
Win probability a team gives up by losing a challenge.
In a tie game the last challenge is worth 1.90 points of win probability at first pitch and still 0.95 in the 9th. Up four in the 9th, it is worth 0.08. In 59% of real opportunities, the break-even confidence is below 50%. The table gives the rule for a close game.
How sure do you need to be? (tie game)
Required confidence that the call is wrong, λ/(g+λ). Stakes columns are the 25th, 50th and 90th percentiles of win value across all called pitches.
Four ways to spend a challenge
Each policy below runs on the same simulated games and the same perception noise; only the decision rule changes. The intervals combine three sources of uncertainty: resampling games, drawing the perception parameters from their estimated uncertainty, and fresh simulation noise.
Wins added per team over 162 games
95% intervals from 60 joint bootstrap replicates.
In the model's simplified game the optimal policy is worth 1.52 more wins per team per season than 2026 behavior (95% CI 1.37–1.63). Replayed on the real 2026 games, where misses cluster within games so that two challenges run out in some games and go unused in others, the gain is smaller, 1.28 per team, and that is the number the headline uses. The intuitive "more likely right than wrong" rule does worst, because it ignores that a correct challenge is free. Giving batters catchers' eyesight would add only 0.62 wins to the optimal policy; the decision rule matters more than vision.
Where players go wrong
For every real opportunity in 2026, I computed what the optimal policy would have done given the team's actual challenges left, the score and the inning, then compared it with what players did.
Challenges by stakes: actual vs optimal
Opportunities grouped into fifths by the win value of the call.
1. They under-react to stakes. In the lowest-stakes fifth, batters challenged 856 times where the optimum is 498. In the highest-stakes fifth they challenged 1,628 times against 3,823. Players do respond to stakes, but far too weakly.
2. They protect their success rate. Optimal play wins 40% of batter challenges and 46% of fielder challenges; players won 48.9% and 58.0%. When the stakes exceed the challenge's shadow price, a policy that wins fewer than half its challenges is the right one.
Challenges by inning: actual vs optimal
3. They hold challenges until they expire. Optimal 9th-inning challenging is 2.6× what batters did. 74.6% of team-games ended with at least one unused challenge and 28.2% with both. Meanwhile 1,703 umpire misses came after a team had already run out.
The exception: strike three. At 0-2 and 1-2, batters challenged 246 and 410 times against optimal counts of 217 and 421. Strike three is a moment where the stakes are high and players can feel it. The failures come at moments that matter just as much but don't feel that way.
A card players can actually use
The optimal rule needs the exact stakes of every pitch, which no one can compute in two seconds. So I asked how much of the optimum a simple card can capture. The player answers one question, "how sure am I that the call is wrong?", and compares the answer with a threshold on the card. The thresholds were tuned inside the model, then tested on fresh simulations and on a replay of the whole season.
The stakes card
Which situations count as low, medium or high stakes.
How much each rule recovers
Share of the gap between 2026 behaviour and the optimal policy.
This closes 48% of the gap and would have added 18.5 wins across the league. Adding the game phase (9 numbers) closes 54%, and a tuned 18-cell card 61%. The intuitive "more likely right than wrong" rule does worse than what players actually did.
What rule should MLB use?
The same model can score rules MLB didn't adopt. For each of nine rules I re-solved the optimal policy and replayed every 2026 team-game 200 times, under that policy and under 2026-style behaviour. The rules vary the number of challenges, whether a won challenge is kept, who may challenge, and whether offence and defence share a pool. The league cares about two things: how much of the umpires' error gets fixed, and how long reviews take. Time is counted at 13.8 seconds per challenge, MLB's spring-training average.
Accuracy vs review volume, nine rules
Share of the win value of umpire misses corrected, against challenges per game (both teams).
1. Keeping won challenges matters more than adding a third. Going from two non-retained challenges to two retained ones is worth 1.24 wins per team under optimal play and 0.84 under 2026 behaviour. A third retained challenge adds 1.05 and 0.41.
2. One shared pool beats separate ones. Two shared challenges beat one for offence and one for defence by 0.72 wins under optimal play, because a shared pool can follow the stakes to either side of the ball.
3. Limiting who may challenge is costly. Catchers and pitchers alone keep 65% of the value; batters alone keep 60%.
4. Better decisions beat more challenges. Under the 2026 rule, optimal play is worth 1.28 more wins per team than 2026 behaviour. With 2026-style decisions, a third challenge would add only 0.41 wins and about 9 seconds of review per game, because teams already leave challenges unused. One challenge used optimally (3.11 wins) is worth nearly as much as two used the 2026 way (3.39).
For the league, the 2026 rule sits on the efficient frontier. Even so, under optimal play it corrects only 44% of the win value of umpire misses, and 32% as players actually decided. Full automation would correct all of it.
Teams: all short, hard to rank
Each team's season was replayed 200 times on its exact sequence of called pitches under the optimal policy. Every team captured less than its optimal replay, but most of the variation between teams is noise.
Shortfall against the optimal replay, with 95% intervals
Players: style is a trait, results are mostly noise
To tell skill from luck, I split each player's games into odd and even and estimated everything twice. How often a player challenges is very stable from one half to the other. How well they see and where they set the bar are moderately stable. The results of their challenges are not.
Split-half reliability of player traits
Spearman–Brown full-season reliability with 95% intervals. 1 = a pure trait, 0 = pure noise.
Challenge Value Added (CVA) credits each overturn with the win value of the call and charges each lost challenge its shadow price at that moment. Because its reliability is only 0.36 for batters and 0.17 for catchers, the leaderboards below show raw CVA in wins next to a shrunken rate in wins per 1,000 opportunities. Read them as descriptive, not as rankings of skill.
Batters
Catchers
Sharpest and blurriest reads of the zone
Estimated perception noise σ in inches (lower is sharper), shrunk toward the role average. Reliability of this trait is about 0.5.
Catchers adapted and batters didn't
Umpire accuracy was flat all season. Catchers' share of the optimal policy's value rose from 65% in April to 80% in September, and a formal test of change over the season is overwhelming (p < 10⁻¹⁸). Batters show no change in any part of their behaviour (p = 0.59).
Share of optimal value captured, by month
What exactly changed is not identified. Catchers became more willing to challenge, but the data can't separate "paying more attention" from "lowering the bar", whose estimated trends correlate at 0.93. Better eyesight alone is the least supported explanation. That's as far as the evidence goes.
How sure are we?
Each claim above was checked against a way it could be wrong. The paper has the details; here's the summary.
Robustness to the perception model
Each row refits the model, re-solves the policy and replays the season. Fit is the change in log-likelihood against the baseline (higher is better).
Method and limitations
- Data
- Every 2026 regular-season pitch from Baseball Savant (716,792 pitches, 2,429 games) joined to the batter, catcher and pitcher challenge files (10,558 challenges, all matched). Statcast records the post-challenge call, so each umpire's original call was reconstructed from the challenge result.
- Models
- Run and win expectancy from 2026 run distributions by half, base, out and count. Perception as P(challenge) = c · Φ((w − τ)/σ), fitted by maximum likelihood with analytic gradients. The optimal policy comes from backward induction over (opportunity, challenges held, score margin), with Bayesian posteriors and an extra-inning fixed point.
- Limitations
- Perception is modelled as Gaussian noise around the true location, so other cues players use are absorbed by the attention term. Win expectancy ignores team strength. Opportunities are independent within a game, so an umpire's zone drifting mid-game isn't modelled. The ball radius and zone heights were inferred from the data rather than the rulebook, although they fit 99.95% of rulings.