Robo strikezone: Not as simple as you think -- Baseball Prospectus

VORP Speed · Feb 13, 2019

Perfect is the enemy of good

InstaFace · Feb 13, 2019

yeah, if it's 98% accurate to within 1/8", the umps' union might as well fold up shop. These worries are angels-dancing-on-the-head-of-a-pin in terms of impact on the overall experience.

edoug · Feb 13, 2019

What about foul tips? Wild pitches or passed balls with a man on 3rd? Wild pitches or passed balls with men on 2nd and 3rd? HBP that graze the batters uniform?

jercra · Feb 13, 2019

edoug said:
What about foul tips? Wild pitches or passed balls with a man on 3rd? Wild pitches or passed balls with men on 2nd and 3rd? HBP that graze the batters uniform?

I don't think the proposal needs to be that the home plate umpire is eliminated, just that they don't call balls and strikes anymore. The first base ump still exists and he doesn't call balls and strikes.

Beomoose · Feb 13, 2019

How about a compromise: the Ump wears an AR headset that projects the zone into the world so he can clearly see the ball in relation to it, but he's still the one that makes the call.

Marceline · Feb 14, 2019

Pace of play is an entirely separate issue being discussed elsewhere on this site, and I think an important one, but not to be conflated with the missed calls issue.

This article is highly flawed from my perspective. For example:

"It is widely known that both Doppler-based and camera-based machines are not accurately reporting 100 percent of their measured data 100 percent of the time. Even if they were accurate 99.9 percent of the time, what if that 0.1 percent is a strike but is called a ball on a 3-2 count in the bottom of the ninth inning with bases loaded and two outs in a tie game?"

This argument makes no sense, because I think we all know that no human umpire is 99.9% accurate, so why would a robo ump be held to that standard? Not to mention what the example used brings to mind since we saw a perfect game ruined by a human umpire in real life. (I know, not a ball/strike call)

Worrying about whether the seams of the baseball will be picked up is silly because right now the entire baseball can be over the plate and the human ump may call it a ball.

Lose Remerswaal · Feb 14, 2019

charlieoscar said:
Overall, batters hit .248 with an on-base percentage of .318 during the 2018 regular season. Who made more mistakes...the batters of the home plate umpires? I've been watching baseball for a long, long time and missed calls by umpires (and mistakes by everyone from the owners on down) have been part of the game. What I want to see is getting the game time back down to about two-and-a-half hours. I posted a link to that article to bring it to the attention of those people who think that robo-calling balls and strikes is a piece of cake to implement.

Welcome to SoSH Mr. Cafardo!

Doing it quickly is better than doing it right? I hope I never meet your surgeon!

Of course if batters hit 1.000 or had an OBP of 1.000 you could turn your argument around to say the pitchers are making too many mistakes . . . .

Van Everyman · Feb 14, 2019

My assumption all along has been that umpires will wear a Google Glass-type of eyewear that allows them to see the ball go over the plate Gameday-style.

Really, it’s not that hard.

Max Power · Feb 14, 2019

Lose Remerswaal said:
Welcome to SoSH Mr. Cafardo!

Doing it quickly is better than doing it right? I hope I never meet your surgeon!

Of course if batters hit 1.000 or had an OBP of 1.000 you could turn your argument around to say the pitchers are making too many mistakes . . . .

Imagine how long the games would be if that were the case.

DJnVa · Feb 14, 2019

People thought robot umps would be a piece of cake to implement?

TomTerrific · Feb 14, 2019

The article isn't bad for the most part (see below), but it kind of misses the forest for the trees.

(BTW, one of the bad parts of the article is the machine calibration part--calibrating to the tolerances required is not challenging at all. And the discussion of RF issues is uninformed, to put it mildly.)

What a baseball umpire calling balls and strikes needs to have, more than anything else, is consistency within a game, and consistency from game to game, and park to park. This is very much easier to achieve with automated equipment than humans. Even if the strike zone is different than the average one called by umpires, given the tolerances already achievable by machines, I'll bet the in-game variance for them is way smaller than the variance from ump to ump. And as I said above, the game-to-game and park-to-park variance just isn't going to be anything nearly as big as the in-game variance if any reasonable effort is put into it.

The thing that drives people nuts aren't any of the things focused on in the article. What bugs everyone are lack of consistency and glaring errors--and humans just can't do nearly as good a job at avoiding those two things. As for any residual errors--if batters can adjust to the vagaries (i.e., biases) of human umpires, they can adjust to the ones caused by machines. Which will be far more constant and predictable than the human ones. And BTW, if the K-zone shown on the screen is drawn from the same sensing that calls the balls and strikes, it will look right to everyone and stop pissing people off. Which people, BTW, who have a really, really poor idea based on just their eyeballs as to whether the pitch really was a ball or strike.

TECHNICAL RANT I CAN'T RESIST MAKING:

From a physics point of view, humans are kind of poorly designed relative to machines. We depend almost exclusively on two angles that our eyes measure, and a not-very-well-stablized platform (our heads) to track an object that is decreasing in range by almost 98%. Anyone who's ever worked this kind of problem knows you're going to be heavily dependent on estimating the entire trajectory in order to try and get that last bit (where the ball crosses the strike zone) right, and humans definitely do that. Given the forces at play (e.g., baseball's response to the spin vector, air motion-temp-pressure-humidity) this is a really variable process, too.

In contrast, these other systems are range-Doppler multilateration systems (meaning they construct a trajectory based on a series of range-Doppler measurements from a set of fixed locations). There are very few situations where we cannot reconstruct trajectories far, far better using range-Doppler multilateration than we can using angle information along, even multi-angular measurements (aka triangulation). The basic reason is that, for most situations, and relative to the required accuracy, we can measure range far, far better than we can angles.

Marbleheader · Feb 14, 2019

Beomoose said:
How about a compromise: the Ump wears an AR headset that projects the zone into the world so he can clearly see the ball in relation to it, but he's still the one that makes the call.

All he's really need is an earpiece. The ump gets a ding for a strike and a buzzer for a ball but still relays the call on the field.

geoduck no quahog · Feb 14, 2019

What on earth is relevant about "batters making more mistakes than umpires"? Seriously...what's your point?

That aside, the killer paragraph for me is:

With human umpires the zone expands and contracts depending on the count and whether the batter or pitcher has the advantage. Knowing that, pitchers who fall behind know that an expanding zone can help them battle back. When the zone doesn’t change size based on count, pitchers will be incentivized to try to get ahead early so that hitters have to cover areas they haven’t traditionally had to defend when behind in the count (e.g., the top corners of the zone).

There is nothing fair or sporting about that sentence. Is that the way players should be trained? The zone changes with the count? Robo-Umps aside, it's ridiculous that an umpire should change the zone within a single plate appearance.

I'll repeat what an NCAA umpire once told me (take it for what it's worth - I'd be interested in a more direct interpretation): Umpires have a hard time judging pitch height, particularly in the 2-dimensions that include plate depth. He says he was taught to call a pitch caught above the catcher's knees and below the catcher's eyes a strike. He said that college umps give catchers a lot of shit if they try to repeatedly game that system. Anyone have any insight?

Everything about that article led me to believe automatic strike zone calls are more fair than human perception.

Bergs · Feb 14, 2019

charlieoscar said:
Why does a batter take a pitch? Sometimes it's because it wasn't the pitch he was looking for, sometimes because he was fooled, sometimes because he thought it would be called a ball. In this latter case, he is umpiring. So, who makes more mistakes umpiring? The batter or the umpire? I'd say the batter and if you had a perfect ball/strike calling robo-umpire you would still see batters misjudge close pitches. For one reason, batters are watching the ball and especially with pitches that are up a bit, they won't be seeing the edge of the strike zone.

Oh, the umpire missed a call. Boo! But does anyone boo the batter? You have these fans watching a depiction of the strike zone and path of the ball in a little display on the corner of their television screen, a display that is not always accurate, calling for the head of the umpire. Or better, yet, someone down past the foul pole in right field screaming at the umpire for missing a ball or strike call.

I think baseball has bigger things to worry about, and doing things quicker is one of them.

This is an argument against the very concept of logic.

Plympton91 · Feb 14, 2019

charlieoscar said:
Oh, the umpire missed a call. Boo! But does anyone boo the batter? You have these fans watching a depiction of the strike zone and path of the ball in a little display on the corner of their television screen, a display that is not always accurate, calling for the head of the umpire. Or better, yet, someone down past the foul pole in right field screaming at the umpire for missing a ball or strike call.

.

It’s been about 20 years now that we’ve had games with that display screen for balls and strikes at least intermittently. In all the games I’ve watched, there was 1 time - one - where I said, “boy, the computer clearly missed that call, I’m glad the home plate umpire was there.” There is at least 1 time per game where the umpire calls a pitch completely and obviously wrong.

The argument against computerized strike zones is, “Old man yells at cloud.” That’s it.

joe dokes · Feb 14, 2019

charlieoscar said:
Why does a batter take a pitch? Sometimes it's because it wasn't the pitch he was looking for, sometimes because he was fooled, sometimes because he thought it would be called a ball. In this latter case, he is umpiring. So, who makes more mistakes umpiring? The batter or the umpire? I'd say the batter and if you had a perfect ball/strike calling robo-umpire you would still see batters misjudge close pitches. For one reason, batters are watching the ball and especially with pitches that are up a bit, they won't be seeing the edge of the strike zone.

Oh, the umpire missed a call. Boo! But does anyone boo the batter? You have these fans watching a depiction of the strike zone and path of the ball in a little display on the corner of their television screen, a display that is not always accurate, calling for the head of the umpire. Or better, yet, someone down past the foul pole in right field screaming at the umpire for missing a ball or strike call.

I think baseball has bigger things to worry about, and doing things quicker is one of them.

I dont know how hard it would be to implement, but I'm not sure your points go to that issue.

Who makes more mistakes delivering babies? Obstetricians or orthodontists?

Does anyone boo the batter? In my experience, that happens occasionally.

How does this do things not quicker?

This is an argument against the very concept of logic.
#17 Bergs, 27 minutes ago

If true, it is even an argument?
(see "The Argument Sketch," Monty Python.)

Bergs · Feb 14, 2019

joe dokes said:
If true, it is even an argument?
(see "The Argument Sketch," Monty Python.)

I'm sorry, this is abuse.

Dewey'sCannon · Feb 14, 2019

Look at these shoes, I've only worn them a week and already the soles are worn out. Why bother to complain, no one even listens. . . .

FWIW, I always though the first step should be rob-assisted umpiring - rigging their ball/strike counters so that they would vibrate if the ball is within the automated strike zone. Ump can rely on this, over override if they really think the robot-call is wrong. And that data would be available for review, at least by MLB.

Plympton91 · Feb 14, 2019

It just seems like a vibration signal, or a beep, or a heads up display would be so distracting that it would make the umpires moot anyway. I can’t imagine that something that breaks your concentration as the pitch crosses home plate is conducive to making a better call. And, if they had the buzzer or the vibration on a one-second delay, it’s just going to train umps to not make a borderline call and rely on the computer anyway.

There is no reason to not make this change other than nastalgia for a less sophisticated time.

geoduck no quahog · Feb 14, 2019

...and they'd still be responsible for check swings (along with another umpire), plays at the plate, HBP, 3rd strikes in the dirt, foul balls up to the bag, calling time and generally running the game...a little more than they do the other 3 games they officiate - and they could now do better.

edoug · Feb 15, 2019

geoduck no quahog said:
...and they'd still be responsible for check swings (along with another umpire), plays at the plate, HBP, 3rd strikes in the dirt, foul balls up to the bag, calling time and generally running the game...a little more than they do the other 3 games they officiate - and they could now do better.

Yeah, there absolutely has to be someone behind home plate.

Lose Remerswaal · Feb 15, 2019

edoug said:
Yeah, there absolutely has to be someone behind home plate.

Well duh, or else you are going to have a shit-ton of passed balls /Casey Stengel (sort of)

edoug · Feb 15, 2019

Lose Remerswaal said:
Well duh, or else you are going to have a shit-ton of passed balls /Casey Stengel (sort of)

Yeah it's only logical.

InstaFace · Feb 16, 2019

Plympton91 said:
It’s been about 20 years now that we’ve had games with that display screen for balls and strikes at least intermittently. In all the games I’ve watched, there was 1 time - one - where I said, “boy, the computer clearly missed that call, I’m glad the home plate umpire was there.” There is at least 1 time per game where the umpire calls a pitch completely and obviously wrong.

The argument against computerized strike zones is, “Old man yells at cloud.” That’s it.

Since it was clearly so memorable, though, I'm now kinda curious what that one instance was.

uncannymanny · Feb 16, 2019

charlieoscar said:
Why does a batter take a pitch? Sometimes it's because it wasn't the pitch he was looking for, sometimes because he was fooled, sometimes because he thought it would be called a ball. In this latter case, he is umpiring. So, who makes more mistakes umpiring? The batter or the umpire? I'd say the batter and if you had a perfect ball/strike calling robo-umpire you would still see batters misjudge close pitches. For one reason, batters are watching the ball and especially with pitches that are up a bit, they won't be seeing the edge of the strike zone.

Oh, the umpire missed a call. Boo! But does anyone boo the batter?

The batter is also trying to decide whether or not to swing and to get his body parts into position to do so in a split second. The umpire has the luxury of having one job as the pitch is delivered. This is a really odd argument.

Plympton91 · Feb 17, 2019

InstaFace said:
Since it was clearly so memorable, though, I'm now kinda curious what that one instance was.

It was the Ortiz at bat against Miller early in 2016 season. Of course, the umpire missed that call too, so it didn’t really matter.

shaggydog2000 · Feb 17, 2019

Trackman/Statcast, which I assume would be the robo-ump of choice given that it is in current use at ball-parks, is off vertically and horizontally by up to 3 inches:

https://tht.fangraphs.com/the-lurking-error-in-statcast-pitch-data/

Now, those three inch errors are pretty rare. There are most likely errors that are even worse, but happen even more rarely. But we've all seen Umpires make even worse mistakes. I don't know of data on umpire accuracy, but I am sure the MLB has it somewhere. I'd guess it's somewhat equal, but not fairly distributed error. Combining an Ump with an ability to over-rule the system with the trackman buzzer would be fine by me. The Ump could catch the obvious errors, and the robo-ump could be used to train the Ump to be more accurate and consistent. I know they do post-game training involving video and the Umpire calls, but a buzzer in your ear is a good reminder of how you're supposed to be calling things. It's going to come eventually, let's figure out how to do it in the most fair and accurate manner.

santadevil · Apr 8, 2019

Good umpire breakdown of numbers article from ESPN

http://www.espn.com/mlb/story/_/id/26467659/study-young-umps-top-older-peers-calls

A new study reports that younger umpires are better at calling balls and strikes than older umpires and that umpires selected for recent World Series games have not necessarily been the best performers.

Umpires showed a persistent two-strike bias. With two strikes, umpires were twice as likely to call a true ball a strike (29 percent of the time) than with a lower count (15 percent). The error rates have declined since 2008, when it was 35.2 percent, but over the 11 seasons, nearly one-third of the batters who struck out looking were called out on miscalls.

InstaFace · Jul 11, 2019

Robot Umpire debuts in Atlantic League ASG.

Umpires can overrule the computer.

"Until we can trust this system 100 percent, I still have to go back there with the intention of getting a pitch correct because if the system fails, it doesn't pick a pitch up, or if it registers a pitch that's a foot-and-a-half off the plate as a strike, I have to be prepared to correct that," deBrauwere said before the game.

The umpires have the ability to override the computer, which considers a pitch a strike when the ball bounces and then crosses the zone. TrackMan also does not evaluate check swings.

Some interesting quotes from Rob Manfred at the end there.

wade boggs chicken dinner · Jul 11, 2019

Lose Remerswaal said:
Welcome to SoSH Mr. Cafardo!

Doing it quickly is better than doing it right? I hope I never meet your surgeon!

Of course if batters hit 1.000 or had an OBP of 1.000 you could turn your argument around to say the pitchers are making too many mistakes . . . .

Need to get robot pitchers.

mauf · Jul 11, 2019

As with automobiles, the talk is all about automating the job and eliminating the human element entirely, but in the near term some combination of human and machine is likely to yield the best results. To some degree, this is already happening with balls and strikes — no one who follows baseball closely would dispute that ball-strike calls have improved greatly in the past 20 years, and technology has been a big part of that. Giving umpires the benefit of real-time technological input is the logical next step.

budcrew08 · Jul 11, 2019

The first pitch from the All-Star Game in the Atlantic League was a called strike. https://mobile.twitter.com/RobRoseSports/status/1149099581667446784

shaggydog2000 · Jul 11, 2019

budcrew08 said:
The first pitch from the All-Star Game in the Atlantic League was a called strike. https://mobile.twitter.com/RobRoseSports/status/1149099581667446784

But did they get it right?

InstaFace · Jul 11, 2019

wade boggs chicken dinner said:
Need to get robot pitchers.

I know some that can throw strikes...

soxhop411 · Jul 12, 2019

budcrew08 said:
The first pitch from the All-Star Game in the Atlantic League was a called strike. https://mobile.twitter.com/RobRoseSports/status/1149099581667446784

shaggydog2000 said:
But did they get it right?

Close call sports has a bunch of highlights from the debut of the robo ump in the Atlantic league.

https://youtu.be/8kqAr5hRPUg
Cc @budcrew08

Mueller's Twin Grannies · Jul 12, 2019

This is basically going to eliminate the need for pitch framing as a skill for catchers, which is kind of sad. Stealing a strike is a great skill that has made the difference for a lot of pitchers over the years and catchers too.

InstaFace said:
I know some that can throw strikes...

I don't think that's legal since it never touches the lane. It's also terrifying while simultaneously reminding me of Walter throw the ball into the Nihilist (Flea)'s stomach in TBL.

phrenile · Jul 12, 2019

Mueller's Twin Grannies said:
I don't think that's legal since it never touches the lane. It's also terrifying while simultaneously reminding me of Walter throw the ball into the Nihilist (Flea)'s stomach in TBL.

Like TBL, it's well-executed fiction.

InstaFace · Jul 15, 2019

Ok, mark it zero. As you were.

Anyone have opinions (yours or someone else's) as to how Trackman did against their own assessment of pitch calling? The video up there had some called strikes that would surely have been low balls to an MLB ump. That's surprising, I always figured the most squeezing happened on high pitches, where umps have collectively decided to ignore the rulebook. Left/right looked good to me.

Wonder where else they'll use it next.

CFB_Rules · Jul 15, 2019

I think having robots call balls and strikes is certainly doable and desirable, but there are a few things that would have to be demonstrated for it to work well.

1) I'm less concerned with the error bars of the machine, because humans have those too. However, it needs to be shown that machine errors are essentially randomly distributed. If the machine is more likely to make mistakes with one pitcher because it doesn't like the spin on his pitches or something, that's not acceptable.

2) The machine needs to be able to make decisions quickly. Less than a second after the ball is caught the umpire should know the result from the computer.

The umpire also needs to have the final say to potentially overrule the machine. Overrules should be rare and documented (such as a ball bouncing in the dirt and then catching part of the zone).

One thing that I haven't heard a lot of people talk about...as machines become more proficient at calling strikes correctly according to rule, pitching and hitting will change as both become accustomed to the "new" zone. Pitches up and away have historically almost always been called balls since they are practically unhittable, and breaking pitches just off the plate have been called strikes. Will players adjust the way they've played to the newly called strike zone, or will they adjust the machine parameters to match what players expect?

InstaFace · Jul 16, 2019

how would you conclusively determine whether a given machine pitch call is an error? frame-by-frame review of the video?

I believe your #2 is already the case, it just took the Atlantic League ump more time to communicate than players were used to. And your #3 is the case in the Atlantic League's implementation - see my quote in post #34.

CFB_Rules · Jul 16, 2019

InstaFace said:
how would you conclusively determine whether a given machine pitch call is an error? frame-by-frame review of the video?

The article cites this as their source for PitchFX accuracy: https://www.baseballprospectus.com/news/article/13109/spinning-yarn-how-accurate-is-pitchtrax/

Some of the data in the article would cause concern. For example, it is obviously unacceptable that Milwaukee's PitchFX system was shifted 2 inches to the 3rd base side for 4 straight years. Hopefully that is something that routine calibrations, done by MLB and not the home club, would fix.

geoduck no quahog · Jul 16, 2019

Don't know if they're accurate or not, but the 3D strike zone graphics used by ESPN are spectacular. I wonder if teams use that technology to help train/tune-up their pitchers instead of coaches or catchers calling out location and, if they don't - why not?

It seems incredibly useful for a pitcher to see that he missed his target, say, by a half a baseball width inside, but still high enough to be called a strike...or if he caught the front end of the strike zone on a low curve.

InstaFace · Jul 16, 2019

Great article.

Also love that tidbit: the worst park for calibration and systematic error? Florida, of course.

HowBoutDemSox · Aug 12, 2019

Interesting piece of umpire accuracy and the elusive “perfect game” of umpiring:

Since the start of 2008, the first full season when pitch-tracking systems were installed in every stadium, MLB players have recorded 51 cycles, 48 triple plays, and 38 no-hitters (not counting combined no-hitters). All those infrequent events have occurred more often than the umpire perfect game. Over the same span, home-plate umpires have made every pitch call correctly on one team only 24 times in games that lasted at least nine innings, with West’s the most recent. That’s roughly once per 1,200 games, or a little more than twice per season, on average. It’s not quite as uncommon as the pitcher perfect game—we’ve seen only six of those since ’08—but it’s rarer than most of the events considered deserving of push notifications.

No umpire on record has achieved perfection when forced to make more than 71 calls on a team. Naturally, no umpire has been perfect for both teams in a game; Scott Barry came closest on July 22, 2017, when he made 138 of 142 calls correctly (97.2 percent). When West was perfect on Cubs calls, he nailed only 87.5 percent of his calls on Pirates pitches.

https://www.theringer.com/mlb/2019/8/9/20798375/umpire-perfect-game-accurate-robo-umps

jon abbey · Nov 5, 2019

One step closer:

"Tested already in the independent Atlantic League and the MLB-run Arizona Fall League, the automated strike zone is slated to arrive in Minor League Baseball in some capacity in 2020.

Commissioner Rob Manfred told MLB Network last week that the technology used for the automated strike zone was due for a large upgrade this winter, and that it would make its way into standard minor-league play next year.

“Here’s our thinking on the automated strike zone: The technology exists. We have the technology,” Manfred said on MLB Now with Brian Kenny. “We’re actually going through a big upgrade of that piece of our technology during this offseason. I think we need to be ready to use an automated strike zone when the time is right. That’s why we experimented in the Atlantic League. It’s why we went to the Arizona Fall League. It’s why we’re using it in Minor League Baseball next year, in some ballparks at least.

“I think it’s incumbent upon us to see if we can get the system to the point we’re comfortable it can work. I only would go to an automated strike zone when we were sure that it was absolutely the best it can be. Getting out there too early with it and not having it work well, that’d be a big mistake.”"

https://theathletic.com/1344428/2019/11/05/automated-strike-zone-to-debut-in-minor-league-baseball-in-2020-rob-manfred-says/?source=twittered

Robo strikezone: Not as simple as you think -- Baseball Prospectus

Member

Member

The Ultimate One

Member

No longer respects DeChambeau

is insoxicated

Member

Well-Known Member

Experiencing Furry Panic

Member

thai good. you like shirt?

Dorito Dawg

Member

Moderator

Member

not particularly consistent

don't Judge me

bubble burster

Member

don't Judge me

Member

bubble burster

not particularly consistent

Member

Experiencing Furry Panic

Member

The Ultimate One

Member

bubble burster

Member

Member

wears depends

Member

The Ultimate One

Member

Anderson Cooper × Mr. Rogers

Member

Member

The Ultimate One

news aggravator

critical thinker

Well-Known Member

The Ultimate One

Member

The Ultimate One

Member

not particularly consistent

The Ultimate One

Member

Shanghai Warrior