More than 100 years ago, a guy asked a whole lot of people to guess how much an ox weighed. Stay with me, I promise this is related to the Trump-Kim nuclear summit.
The man was a statistician named Francis Galton and in 1906 he got 800 people at a country fair to look an unassuming ox up and down and then estimate the beast’s weight. Some guesses were presumably pretty bad, but when Galton ran the numbers, he found that the average of the guesses was really good. (NPR repeated this experiment in 2015 and the internet’s average guess was impressive as well.)
The ox-weight guessing game is a common example of what’s known as the wisdom of crowds or, to keep the farmland theme going, the wisdom of the herd. It basically means that while a few of us might be idiots about a particular topic, in mass, even non-expert humans aren’t too bad when their insights are averaged out. So what happens if we apply this phenomenon to non-bovine-related matters, maybe something as complex as geopolitics?
That’s where Good Judgment Inc. comes in. Good Judgment is the commercial outgrowth of the Good Judgment Project, a research team assembled for a government-sponsored future prediction competition back in 2011. Drawing predictions from a bunch of people and armed with demographic information about those people, the project’s data scientists “then applied groundbreaking aggregation methods to convert the individual predictions into collective ‘wisdom of the crowd’ forecasts,” the company’s website says.
According to the researchers, the best of these forecasters, known as “superforecasters,” at times beat professional U.S. government analyst forecasts. Good Judgment maintains a stable of more than 100 of these “superforecasters” – contractors whose day job is anything from traders on Wall Street to archaeologists to scientists – to make predictions for paying clients. But for the rest of us, Good Judgment also runs GJOpen, in which anyone can sign up to take a stab at being a modern Nostradamus, with the help of thousands of strangers the world over.
Will the Mosul Dam survive the conflict in Iraq? The GJOpen participants were right (at least so far). How will the Supreme Court rule on a recent gambling issue? Respondents got that one too. The herd gave Villanova the best chance of winning March Madness for the men after the 2018 NCAA tourney’s first round. Who would win the 2016 election? Well, okay that was a rough one, but Good Judgment’s team did better than most.
The 2016 election is not alone. After all, the crowd is made up of humans and, of course, they’re not always right. Cast in point: In March GJOpen asked respondents if President Trump and North Korean leader Kim Jong Un would meet before July 2018. For some context, when the question was asked Trump had already accepted an invitation to meet with Kim. But the more than 350 forecasters who answered the question weren’t buying it.
For the first month, the average forecasts said there was a 90 percent chance the two leaders would not sit down face-to-face by the deadline, according to data on the site.
In late April, however, feelings started to change. That month then-CIA Director Mike Pompeo secretly met with Kim, North Korea said it suspended nuclear and ICBM tests, and Kim met with South Korean President Moon Jae-in. By April 30, the forecasters gave the sit-down a 50-50 shot.
The upward trend continued as Pompeo made another visit to North Korea, and it jumped up to 99 percent sure-to-happen on May 10 when Trump announced he’d meet Kim in Singapore.
Then National Security Advisor John Bolton kicked in the door, suggested North Korea follow the “Libya model” and sparked a verbal back-and-forth that ended in Trump calling off the meeting in a huff on May 24.
Watching all this happen, forecasters had the odds of a meeting rolling slowly downhill until falling off the May 24 cliff. By the early hours of May 25, GJOpen predicters put the chance of a meeting at around
2 percent.
But by the next day, the forecast started another upward climb spurred on by nice words from the North Koreans and the White House’s continued summit preparations. Fifty-fifty odds came again on May 31, after North Korean envoy Kim Yong Chol arrived in the U.S. to meet with Pompeo in New York before visiting Trump in the White House. Trump said on June 1 the meeting was back on.
Less than two weeks later, here we are. As Trump alighted in Air Force One for Singapore this weekend, GJOpen forecasters gave the meeting a 99 percent chance of happening.
Marc Koehler, Vice President at Good Judgment, Inc., told RealClearLife that his stable of superforecasters was also very skeptical of a possible meeting when they were asked the question in May, but as various diplomatic teams went about their international shuffling, the expectations rose. In general, he said, the superforecasters predictions aren’t as volatile as the GJOpen participants’ because they’re more accustomed to taking the long view rather than reacting to daily news spasms. So what if Trump canceled the summit – what are White House officials and North Korean diplomats actually doing on the ground?
The actions of North Korea, Koehler said, are famously hard to anticipate – so much so that there’s a running half-joke among superforecasters that no one should judge anything as less than 10 percent or more than 90 percent likely to happen, because who knows. With the addition of Trump, a man who prides himself on unpredictability, the job of predicting the future got may have gotten a lot harder.
At least an ox stays about the same weight from day to day.
This article was featured in the InsideHook newsletter. Sign up now.
