cricket news: Umpiring In The Age Of Cable TV

[ADMIN 322]

In recent years, I’ve been struck by a strange feature of the way cricket is discussed. On the one hand, the discussion of tactics, technique and the playing conditions is more sophisticated than ever before. On the other, the discussion of umpiring is less nuanced than ever before. It is as if there is an emerging consensus in cricket that the game is complicated, but refereeing it is not, and decisions made by umpires would be self-evident to anybody else who might fill their shoes. Why might this be? The simplest explanation is that umpires are rarely present in commentary boxes or on the sports pages to explain what they do, while former players, captains and coaches proliferate. This may explain why the umpire’s point of view is not presented to cricket fans by umpires. But it doesn’t explain why those who do explain the game to us, who otherwise understand its texture so well, seem increasingly lost when it comes to explaining umpiring decisions. This essay is my attempt first, to illustrate this problem, and second, to explain why it exists. I argue that the the game has been enchanted by technology, ultimately to its detriment.

********************

“Hold on Gunner. I’ll let you know when you are on the screen so you can give your decision.”

Steve Davis sat in the third umpire’s box as he spoke these words into Ian “Gunner” Gould’s ears. And ours. “You can give your decision now. You are on the screen.”

Gould confirmed a not-out decision he had given a few seconds earlier.

Right-arm-over, fast-medium, to lefty from over-the-wicket. The ball pitched somewhere around the line of leg stump, beat the inside edge of the batsman’s forward stroke and hit the pad at knee height. There wasn’t much movement. If it was going to be LBW, it would be tight – pitched on leg stump, hit off. The batsman was playing forward too. Put together, all this meant that a good umpire might not be convinced that it was out. It was uncontroversially a close appeal, not an obvious one. One defining feature of a close appeal is that partisans on both sides simultaneously think it is obvious. In the moment, a good umpire might be sufficiently convinced to give it out. But then again, maybe not.

Once the decision was made, television commentators had their say. A former batsman said that it should have been given out. Another said the batsman was extremely lucky. Still another said that the technology had vindicated the umpire. There was something for partisans of every stripe, including the new ones on the technophile-luddite fault line. The large range of opinions confirmed that the appeal was marginal. That between the bowler, the batsman, the umpire and laws designed more than 130 years ago, the best that could be said about the matter was: “It could have reasonably gone either way.”

We have become incapable of living with this conclusion. What’s more, the game has become fearful of it. Considered expert judgement has become anathema and is being replaced by impersonal bureaucratic box-ticking. When reality defies these boxes, cricket has taken to constructing more rules that take autonomy away from the umpire.

This development is not simply the result of a desire for better rulings. The media-administrative-complex which engulfs the playing fields of professional cricket these days can no longer tolerate the idea of any one person, however accomplished, being able to exercise the discretion in a situation which “could go either way”. Discretion is the right to make a choice in a given situation. The range of situations within which umpires are allowed the right to make make a choice is declining. The game is poorer for it. Let me explain why.

****************************

As I heard umpire Davis’ monotone, I couldn’t help wondering what was going on in his head. Here was an expert looking at a delivery on a television screen. Here was an expert capable of evaluating an umpiring decision better than anybody else in the world. Now he was reduced to ticking off a checklist. “Let’s look at the no-ball. Yes, that looks fine.” “May I see spin-vision when you are ready.” “Lets see the ball-tracker when you are ready.” “Pitched outside leg.”

A part of me wanted umpires Davis and Gould to discuss Gould’s original decision. Not because the ball-tracker showed the ball pitching a hair’s breadth outside leg stump, but because it was marginal based on the available replay, regardless of what the ball-tracker said. I wanted these two veteran umpires to explain to the world how an umpiring judgment is arrived at. I would have given a lot just hear Gould say, “Steve, I thought it was a close shout, but there was just too much going on with where it pitched and where it was heading for me to be certain.” And for Steve to reply, “You’d have to guess correctly on too many 50-50 calls in order to be sure about that one. If you weren’t sure about any one of them, I agree with the not-out decision.”

Gould’s decision would still have been correct even if the ball-tracker had shown the ball to be pitching a hairs breadth to the right, on leg stump. This is because we are dealing with two methods of evaluating the course of the ball which are fundamentally incompatible. If you think about it, the ball-tracker’s predictive element performs three basic steps. The unit of the ball-tracker's results is distance.

1. The ball-tracker estimates where the ball will cross the plane of the stumps.

2. It evaluates whether or not this location meets a condition. The condition could be "pitched outside leg-stump", or "pitched on leg stump" or "hitting leg stump" or "hitting 30% of leg-stump" and so on.

3. Then the ICC-specified conclusion based on the ball-tracker's evaluation is provided to viewers. At the stumps, for example, the options are "Hitting", "Missing" and "Umpire's Call". The conclusion "Missing" is self-evident. The conclusion "Hitting" or "Umpire's Call" depends on a standard set by the ICC. Currently, the standard for "hitting" is "50% of the ball hitting at least 50% of the stump." Anything which lies between missing and hitting, falls under “Umpire’s Call”.

The unit of the umpire's work is not distance. It is categories. Umpires do not conclude "Missing by 2mm" or "Missing by 3/8th of an inch". They pursue factual observations. "Tall batsman", "Hit above the knee roll", "Pitched outside off, impact on middle-and-leg", "batsman well-forward.", "batsman batting outside his crease.", "batsman caught on the crease", and so on. It can be observed that umpires systematically give more LBWs which are tight on leg-stump when the batsman is playing back, than they do when a batsman is playing forward. Umpires put these observations together to reach one of two conclusions "sure" or "not sure".

So Gould, making his decision in real time, would be correct to decide "not sure" for a ball which pitched (according to the ball-tracker) one hair's breadth outside leg-stump, and for a ball which pitched (according to the ball-tracker) one hair's breadth to the off-side of leg-stump. The difference is infinitesimal. If Umpires are going to be judged to have made a mistake for being unable to discriminate between these two cases, then no human being has a chance of ever being a good umpire. More importantly, to expect umpires to evaluate LBWs like a ball-tracker is to misunderstand not only umpires, ball-tracking and the LBW law, but also the very nature of evidence and proof.

Umpire Gould once found himself at the other end of a review. Think of the LBW decision he gave against Sachin Tendulkar in the World Cup Semi Final at Mohali in 2011. There was nothing obviously wrong with the decision, but Tendulkar reviewed it (his wicket was important to his team). The review went against Gould by a hair’s breadth. One hair to the left, and it would have been clipping leg-stump and classed as ‘Umpire’s Call’. Tendulkar would have been Out. In the moment, Gould was convinced that Ajmal had trapped Tendulkar. Was it Out? Arguably yes. Before Tendulkar reviewed Gould’s decision, even Ravi Shastri conceded in the commentary box that the appeal looked very close.

After the game, there were conspiracy theories about the ball-tracker being doctored. This is not surprising given the stakes. Hawkeye Innovations Ltd., the company which makes Hawkeye, issued two documents clarifying the decision and dismissing these conspiracy theories. There was talk of how Tendulkar had been saved by technology. While the appeal was being made, commentators were more or less unanimous that it was very close. This is commentary jargon for “batsman will be lucky to survive.” After the review, the commentators turned their attention to the spectacle of the big wicket being reversed in India’s favor because of Hawkeye. There was very little discussion about Leg Before Wicket.

It wasn’t always so. Before the ball-tracker provided its box-ticking apparatus, commentary about umpiring decisions used to be enriching and informative. Sachin Tendulkar was given out LBW ducking against Glenn McGrath in the 2nd innings at the Adelaide Oval in 1999 by umpire Daryl Harper. The LBW appeal is available on YouTube in edited form thanks to Rob Moody’s superb archive.

McGrath delivered a short ball which pitched outside off stump and moved inwards. Tendulkar ducked and turned his back to the ball. He was hit on the left shoulder.

Ian Chappell was on commentary with Sunil Gavaskar and even as the Australians appealed, observed “You don’t want to be ducking too much here at Adelaide..”.

Umpire Harper was making up his mind and as he motioned to raise the finger, Chappell continued, “And he’s been given out LBW to a ball that didn’t rise. (pause) I think it got Tendulkar on the arm, but obviously Umpire Harper thinks that it was right in front.”

Gavaskar was on commentary alongside Chappell. His first reaction came a few seconds after Chappell had called the delivery. “An attempted bouncer from Glenn McGrath. He’s got that fielder at forward short-leg and there’s one at backward short-leg as well. And, it’s hit him on the shoulder. Took a long time, Daryl Harper, to think about it. And then [gave] the Indian captain out, Leg Before Wicket. He’s got a duck. Big disappointment for him as well as for the crowd here. India four for twenty seven.”

We heard these first thoughts on our screens as we saw the dismissal live. Tendulkar pursed his lips and looked, not at the umpire, but down at the wicket. The sort of glance a batsman gives when he knows he’s been done in by the pitch. He was not wrong. The ball had misbehaved by about 3 feet in the vertical dimension. Tendulkar walked away ruefully.

Now, this was a controversial decision. But the controversy was fully described by Ian Chappell during play. Nothing that was said after the game added any information or insight to the issue. Here is Chappell’s comment (made as the delivery was replayed on our screen) in full:

“Dangerous LBW decision for an Umpire to give because there are so many moving parts. It’s not like the batsman being hit on the pad. There’s a lot of movement when the batsman’s ducking like that. It’s hit him up under the back of the arm. (pauses as the ball leaves McGrath’s hand and reaches Tendulkar) Oh, it’s not the easiest decision to give at all. Because with all those things moving, you’ve got to be very sure.”

During the recap next day, it fell to Richie Benaud to verify the decision.

“And then, Tendulkar. This was contro.. it’s been made to be controversial, but the shots I saw yesterday indicate to me that the ball definitely wouldn’t have gone over the stumps. There’s a very good picture this morning in the local papers and in papers around the world.”

The comments by Benaud and Chappell, and Gavaskar’s wonderfully disciplined observations on live commentary illustrate the complexity of the decision. Of course, each of them, and probably every other player and umpire who was alive has an opinion about that decision, and must have, at one point or another, expressed it. On Cricinfo (as it was then), Partab Ramchand wrote a wonderful short article on the reaction to this dismissal. Ramchand wrote

“There are two ways of looking at Tendulkar's dismissal. One view is that the ball was going over the stumps and so he was not out. The other view is that the ball would have hit the stumps and so he was out. It's as simple as that.”

The live commentary by Chappell and Gavaskar is much richer than that. Chappell was trying to describe the circumstances which shape whether or not an umpire is able to give an LBW decision to the bowler. Benaud (much like Ramchand) gave his view after the fact, on whether or not Umpire Harper was right.

Chappell’s point, had he made it to Benaud, would have been, Umpire Harper may well have been proven right in the end, but it was still a bold decision to give it out given what he had to go on. Chappell’s point was not to say that Harper was wrong. It was to say “He may well be right, but I fail to see how he can know for sure. Bold decisions imply at least some element of guesswork. I’m not sure that that’s a good way to give LBWs. Having said that, in the moment, he may well have been sure and may well have been right.”

If Gould’s decision against Tendulkar was arguably the right one even if it was reversed, Harper’s decision against Tendulkar was arguably wrong, even if, a ball-tracker (had one been available) had returned three reds and confirmed it. The best that can be said is, that in the moment, Umpire Harper seemed to be sure that it was Out and ruled accordingly.

Chappell’s commentary was exemplary. He did not simply condemn or praise the umpire, he actually explained why he thought it was a bold decision to give. Not only that, Gavaskar and Chappell both alluded to the field, to the line of attack, to an explanation of why Tendulkar ducked and why it was a bad option. The complex context they built up provided a rich picture to anyone who was prepared to listen.

In the 2015 World Cup quarter final, instead of a discussion about a subtle umpiring judgement by two of foremost experts in the contemporary game, we heard Ian Gould and Steve Davis telling us about the superficially obvious pictures on our screen.

How often have you seen it? An interestingly close LBW decision gets an ex-player in the commentary box all excited. Then the bureaucratic box-ticking of DRS takes over and thinking stops.

**********************************

You may well ask if some umpiring decisions really are as nuanced as I’m suggesting they are. What if the ball-tracker is right, and using its superior accuracy to verify the umpire is in fact justified. Technology is not just used for LBWs (even though a high majority of player reviews are for LBWs). It is also used for catches. Surely, in the case of catches, where only events which actually take place are at issue, the technology can do better than an umpire.

Perhaps so. The question of the technology being “better” than the umpire is a vexed one. As we saw with the three LBW decisions i’ve discussed so far, using the ball-tracker to verify the umpire’s work, even if we assume that the ball-tracker is objectively both more precise and more accurate than an umpire, is of questionable merit. If we look under the hood of the ball-tracker and other technologies, matters become even murkier. There are limits to the extent to which we can do this given that the technologies are proprietary. Let’s consider the ball-tracker first.

The ICC is responsible for the adoption of ball-trackers for umpiring. Within the ICC, the Cricket Committee is charged with considering this question. Nearly 7 years after the ball-tracker was first used under DRS (or UDRS as it was known then), the technology has not been field tested by the ICC. The International Tennis Federation, which also uses technology developed by Hawkeye Innovations (which is one of the vendors of ball-tracking technology in cricket), has an elaborate protocol for field evaluation line-call technology. If nothing else, the detail with which it has been laid out for the public to see suggests that the ITF wants people to believe that it takes testing seriously.

Consider the following episode from the ICC’s treatment of ball-tracker testing from 2012 as a point of comparison. The ICC commissioned Dr. Edward Rosten, a Cambridge expert in Computer Vision, to evaluate the ball-tracker. In a media release dated June 1, 2012, the ICC reported the following:

“Following some concerns about the accuracy and reliability of the ball tracking technology, ICC engaged Dr Edward Rosten, a former Cambridge University lecturer and an expert in this field, to provide an independent evaluation of the accuracy of the two ICC DRS accredited ball tracking suppliers.

The committee was presented with a provisional report covering a review of ball trackings provided in the recent South Africa v Australia series. In the 14 examined sequences Dr Rosten said that his company's results were in 100 per cent agreement with the ball tracking system in use in that series.”

“The Cricket Committee re-iterated its view that, depending on the ability to finance the technology, that DRS should be implemented universally in Test and ODI cricket.”

The ICC asked Dr. Rosten to examine 2 accredited vendors. He reported back to them provisionally about one of the two. Based on this, the ICC recommended that DRS should be implemented “universally” - that both vendors products should be used even though only one had been evaluated. I asked the ICC about this in an email at the time. Their response was a remarkable feat of bureaucratese in which they sought to separate the ICC from its cricket committee.

“It is correct that the Rosten report considered by the Cricket Committee only covered one of the two accredited ball tracking systems. It was a provisional report. Dr Rosten is still busy with his research on the other accredited supplier and will furnish ICC with a final report once he is finished. The Cricket Committee recommended the mandatory use of the DRS on the basis that accredited ball tracking suppliers will be used. The Cricket Committee does not itself accredit the ball tracking companies. This is ICC’s responsibility (in practice, the ICC cricket operations department). At this stage, two ball tracking companies are accredited by ICC.”

This problem is exacerbated by the fact that the two vendors of ball-tracking use different methods to produce the ball-track, and have disagreed publicly about the merits of each other’s methods. What’s more, they disagree about the extent to which ball-tracking should be used to determine the predictive path in LBW decisions. Paul Hawkins of Hawkeye told me in an interview in 2011, and said again in 2013, that cricket had adopted ball-tracking without adequate field testing.

My point here is not to place the ICC in the dock about its cavalier attitude towards scientific testing. What I’ve recounted here is old news. It may well be, just as it was with Daryl Harper’s LBW against Tendulkar in 1999, that the ICC has guessed right about the reliability of ball-tracking. My point is only to say that we don’t know how good ball-tracking is in the specific context of cricket, in which, unlike other sports like baseball or tennis, a counterfactual has to be evaluated (“if the pad was not in the way, would the ball would have gone on to hit the stumps?”). Using DRS the way it does, the ICC is engaged in bad science.

Like ball-tracking all other technology which is part of DRS today came into the game as a feature of the broadcast. Even today, the contract to provide the technology exists between the vendor of the technology and the broadcaster. The beautiful beguiling pictures are a mixture of what looks good on the broadcast and what is scientifically sound. That’s the polite way to put it. The pictures are bad science masquerading as accurate descriptions of reality.

To this day, I have never seen a snickometer with units marked along the vertical axis. Nor have I ever seen any data which might help viewers (and umpires) distinguish systematically between types of contact (bat on ball, glove on ball, bat on pad, bat on ground and so on) by sound signature. The tolerance of snicko is not revealed to viewers either. What is the minimum sound that snicko can record? If snicko doesn’t record something, does it mean there was no contact? Or does it only mean that there was not enough contact for snicko to be activated?

Absolute silence exists only in theory and there are limits to the extent to which a signal can be separated from noise. There is, doubtless, a lot of optimization which has gone into the development of the system, but the presentation of the results on TV is poor. This is not a trivial problem. A brief digression into swimming may be useful here.

In the 2008 Olympics, Michael Phelps won the 100m Butterfly final by 0.01 seconds over Milorad Cavic of Serbia. Later, Omega, the company which provided the timing system at the Olympics, clarified that while Cavic touched the wall first, Phelps was the first to touch with sufficient pressure to stop the clock. In swimming races, the clock is stopped by the swimmers, not by officials (whose only duty is to monitor legality of stroke and turn).

A similar problem exists in cricket in compounded fashion. When umpires use the snickometer technology, they typically use it along with other evidence - visual evidence of a deflection, or of distance between and ball at the right moment in time. Matters get even more complicated when snicko and hotspot are both used. Snicko uses a sound signature to record contact and hotspot uses a heat signature. It’s not clear that the minimum contact each method can detect is the same. Unsurprisingly, in many instances one shows contact while the other shows nothing. Other problems about hotspot have been reported as well.

All this suggests that introducing technology into umpiring introduces new problems. But cricket today adds a special problem to all these difficulties. It is that all this technology has meant that commentators are no longer experts when it comes to umpiring. How could they be? Understanding the ins and outs of ball-tracking, heat signatures, sound signatures and their many combinations would require a substantial understanding of applied mathematics, or at least, of the nature of statistical evidence at the very least. A more general understanding of the nature of scientific evidence, perhaps the ability to discuss Popper, Kuhn, Feyerabend and Polanyi intelligently might be useful as well. This is material which most engineers would have to work at learning, let alone professional cricketers turned commentators who have gotten where they have by being dedicated to the game in single minded fashion for most of their lives.

*************************************

One commentator I spoke to said that commentators “praise umpires when they are right, and criticize them when they are wrong.” This seemed to be a fair approach to this commentator. This is a dangerously simplistic approach which would justify nearly anything that is said or done under the guise of criticism. In the case of both players and umpires, critics have a duty to give fair respect to their expertise. When you criticize an expert action, you have to first describe that expert action and demonstrate that you understand the terms of that expertise, not just the results of such actions. Commentators have a duty to actually describe what the umpire might have been trying to do. What facts were available to the umpire and how might the umpire have put them together? What did the umpire miss? Or was it simply a very difficult appeal?

What I have described in the previous paragraph is not that complicated. Replace umpiring with say, opening batting on a green wicket, and the same commentator would be capable of endless nuance. We would be treated to a disquisition on footwork, on late adjustments with the wrists, on what it really means to see the ball early and play it late and many other things. We might hear how all these points about footwork and still heads are really about one thing - balance. Test players in the commentary box rarely “criticize batsmen when they get out, and praise batsmen when they score runs”. What’s more, if the commentator I spoke to heard a lay journalist speaking about a batsman’s innings on that green wicket in the same cavalier fashion, saying something along the lines of “When the batsman fails, he deserves blame, just as he gets praise when he makes a hundred.”, we would probably see a disapproving frown on his face about the journalist’s ignorance.

The irony in all this is that in an age in which umpiring has been closed off to commentary, the evidence from DRS suggests that today elite umpires are highly competent experts who rarely make mistakes. This is so not because of technology, but simply because umpires are better paid, better trained and the best ones do the job full time.  The ICC, which seems to be so cavalier about adopting technology, tracks the performance of elite Umpires in painstaking detail. Every single decision is recorded. Under the laws of cricket, a decision is said to be made whenever an appeal is answered. Any appeal must either be upheld or be denied. The ICC classifies every appeal into one of three categories - “easy”, “hard” and “very hard”. For many dismissals, like clean bowled, or an outfield catch or even most slip-catches, no appeal is answered. Batsmen walk for such obvious dismissals. Umpires do not have to make decisions in these cases.

According to the ICC’s official figures, Umpires made 663 decisions during the 2015 World Cup. Of these, 583 were made on the field and players reviewed 84 out of the 541 reviewable decisions (or 1 in 5 decisions). 20 reviews were successful. All in all, one in every 27 umpiring decisions was reversed.

Type of Decision	Total	Umpire Review	Player Review	Reversal due to Player Review
LBW	312	0	57	8
Caught	229	8	27	12
Run Out	34 (on field)	55	0	0
Stumped	8 (on field)	16	0	0

Total On-field decisions: 583

Total Umpire Reviews: 80

Total Player Reviews: 84

Total Successful Player Reviews: 20

Total Errors: 32

Errors not corrected because Review was unavailable: 2

Errors not corrected because Review was available but not taken: 10

Total Errors during Umpire Review: 1

Total Errors during Player Review: 0

Correct Easy Decisions: 263/263 (100%)

Correct Hard Decisions: 260/279 (93%)

Correct Very Hard Decisions: 28/41 (68%)

The data shows that players are wrong three times out of four when they use the player review. In 48 matches (1 match was washed out), there were 22 errors, of which 20 were corrected because of the player review.

In these numbers are instances in which the technology, thanks to the arbitrary but ultimately wise concept of “Umpire’s Call”, gives some limited leeway to the umpire’s judgment. There are other instances in which , much like Ian Gould in the 2011 World Cup Semi Final against Tendulkar, an umpire is deemed to have made a mistake by a fraction of a centimeter.

*****************************************

Given the evidence of the last 8 years, the idea that the ICC has implemented DRS simply because it wants to eliminate obvious umpiring mistakes is implausible. By definition, obvious mistakes ought to be obviously evident. If it takes a combination of arbitrary thresholds defined by the ICC and a suite of technologies compromised by the imperative to produce imprecise but beautiful pictures to say “maybe the umpire wrong by a fraction of a millimetre”,  then clearly, the umpire is not obviously wrong. Further, a system which requires, as it turns out, that 4 unsuccessful reviews be made for each successful review has to be bad at correcting umpiring decisions, especially when the same system shows that the umpires are already right at least 9 times out of 10.

No, the only explanation for the persistence of DRS in its current form, other than bureaucratic, institutional inertia, is that DRS was designed primarily to create rules and systems to limit umpiring discretion. Its chief ambition is to manufacture truth without anybody being responsible for exercising expert judgment.

Why did this happen? Here is a brief sketch of an answer in conclusion to this essay.

The cable broadcast, which brought so much money into the game, also brought new gadgetry into it. Umpires began to look foolish as their rare mistakes were magnified thanks to increasingly partisan commentary and forensically sophisticated gadgetry. But far more significant, were the ways in which the gadgetry provided ammunition for partisan commentators (partisanship being a professional necessity for commentator employed by cable TV), both during the game and in later accounts, to challenge the discretion of umpires in situations where no obvious mistakes were evident. The set of obvious umpiring mistakes varied for each side. For the first time, this set came to be systematically built on the broadcast. The pressure on umpires and the ICC built up over time and could have only one consequence, given the game’s dependence on revenue from television - the gadgetry would be invited inside the tent. Under the circumstances, ‘Umpire’s Call’ is a sophisticated compromise which preserves the umpire’s discretion to a limited extent even if it disfigures it in the process.

Not surprisingly, ‘Umpire’ Call’ is also the one part of DRS which attracts the most adverse attention from commentators. Scrutiny of the technology is beyond them. Should teams lose a review is “Umpire’s Call” is returned? How many unsuccessful reviews should be permitted per innings? These two questions form the universe of things commentators can argue about with reference to DRS today. The second question is not particularly interesting. But the first one does get to the heart of the matter. It effectively asks whether players should be penalized for questioning the umpire’s discretion and failing? Many distinguished former players like Shane Warne and Ian Botham think that players shouldn’t be penalized for this. Their view results inescapably in the argument that in a system ostensibly designed to correct obvious mistakes, players who request reviews in situation where no obvious mistake has been made should not be penalized.

It is said that big decisions should never been taking when one is scared or angry. DRS is a system born in fear. It is as much about media-management as it is about umpiring. It is an effort to produce truth without judgment. In the process it has muted expertise and impoverished cricket as a venue for thoughtful observation. The game and its fans are poorer for it.

View the full article