In mid-2016, Aviv Ovadya realized there was something fundamentally wrong with the internet — so wrong that he abandoned his work and sounded an alarm. A few weeks before the 2016 election, he presented his concerns to technologists in San Francisco’s Bay Area and warned of an impending crisis of misinformation in a presentation he titled “Infocalypse.”
The web and the information ecosystem that had developed around it was wildly unhealthy, Ovadya argued. The incentives that governed its biggest platforms were calibrated to reward information that was often misleading and polarizing, or both.
Platforms like Facebook, Twitter, and Google prioritized clicks, shares, ads, and money over quality of information, and Ovadya couldn’t shake the feeling that it was all building toward something bad — a kind of critical threshold of addictive and toxic misinformation. The presentation was largely ignored by employees from the Big Tech platforms — including a few from Facebook who would later go on to drive the company’s NewsFeed integrity effort.
“At the time, it felt like we were in a car careening out of control and it wasn’t just that everyone was saying, ‘we’ll be fine’ — it’s that they didn’t even see the car,” he said.
Ovadya saw early what many — including lawmakers, journalists, and Big Tech CEOs — wouldn’t grasp until months later: Our platformed and algorithmically optimized world is vulnerable — to propaganda, to misinformation, to dark targeted advertising from foreign governments — so much so that it threatens to undermine a cornerstone of human discourse: the credibility of fact.
But it’s what he sees coming next that will really scare the shit out of you.
“Alarmism can be good — you should be alarmist about this stuff,” Ovadya said one January afternoon before calmly outlining a deeply unsettling projection about the next two decades of fake news, artificial intelligence–assisted misinformation campaigns, and propaganda. “We are so screwed it’s beyond what most of us can imagine,” he said. “We were utterly screwed a year and a half ago and we’re even more screwed now. And depending how far you look into the future it just gets worse.”
That future, according to Ovadya, will arrive with a slew of slick, easy-to-use, and eventually seamless technological tools for manipulating perception and falsifying reality, for which terms have already been coined — “reality apathy,” “automated laser phishing,” and “human puppets.”
Which is why Ovadya, an MIT grad with engineering stints at tech companies like Quora, dropped everything in early 2016 to try to prevent what he saw as a Big Tech–enabled information crisis. “One day something just clicked,” he said of his awakening. It became clear to him that, if somebody were to exploit our attention economy and use the platforms that undergird it to distort the truth, there were no real checks and balances to stop it. “I realized if these systems were going to go out of control, there’d be nothing to reign them in and it was going to get bad, and quick,” he said.
Read more – this is only the intro to an incredibly important analysis…
And take a look at what can be done today to deceive us…
Illustration
Doug Chayka for BuzzFeed News
Jan Oberg comments
One must ask what we can safely trust, if anything, in the future. How will we be able to do research and form opinions if, basically, everything can be more or less manipulated and ordinary citizens will have no solid ground to stand on.
Who will exploit such technologies for what purposes?
Will democracy be able to survive that?
What could this imply for education and knowledge-based public debate?
Will media be able to set up filters to avoid just conveying totally constructed and manipulated videos, photos and texts?
Knowledge is structured information. If information cannot be trusted, true knowledge is gone.
One must believe that education is the single most important tool to combat the mentioned global information crisis. Including using educational materials produced before all this happened.
3 Responses
Hvem fortæller, hvad fortælles og hvorfor fortælles der: Alle skoleelever skal lære at stille disse spørgsmål.
Yes education and information. I know from myself the feeling of giving up. Which this article helps me not to do. What is explained in combination with politicians repeating the same lies and/or misinformation is no good.
Yes