Twitter:
A on Monday on Twitter An AI-generated image suggesting a large explosion at the Pentagon led to brief confusion that included a small drop in the stock market. It originated from a verified Twitter account called “Bloomberg Feed,” which is not affiliated with the well-known media company Bloomberg, and was quickly exposed as a hoax. However, by the time it was revealed, major accounts such as Russia Today had already spread the misinformation, The Washington Post reported.
The fake image showed a large plume of black smoke next to a building that vaguely resembles the Pentagon: “Large explosion near the Pentagon complex in Washington. preliminary report” with a note. Upon closer inspection, local authorities confirmed that the image was not an accurate representation of the Pentagon. Also, with the blurry fences and building columns, it looks like a rather sloppy AI-generated image generated by a model like Stable Diffusion.
By the time Twitter suspended the fake Bloomberg account, it had tweeted 224,000 times and reached fewer than 1,000 followers, according to the Post, but it’s unclear who ran it or what their motivations were for sharing the fake image. In addition to Bloomberg Feed, other accounts sharing the fake report include “Walter Bloomberg” and “Breaking Market News,” both of which are not affiliated with the real Bloomberg organization.
The incident highlights the potential dangers posed by the rapid spread of AI-generated images on social media and Twitter’s paid verification system. In March, fake images of Donald Trump’s arrest, created with Midjourney, reached a wide audience. Although clearly marked as fakes, they have sparked fears of mistaking them for real photographs due to their realism. That same month, AI-generated images of Pope Francis in a white coat fooled many who saw them on social media.
Twitter:
A Pope in puffy coats is one thing, but when someone impersonates a government entity like the headquarters of the United States Department of Defense with a fake tweet, the consequences can be more severe. In addition to general confusion on Twitter, a deceptive tweet can affect the stock market. The Washington Post reported that the Dow Jones industrial average fell 85 points in the four minutes after the tweets, but quickly reversed.
Much of the confusion surrounding fake tweets has been made possible by changes at Twitter under its new owner, Elon Musk. Musk launched teams of content moderators shortly after his speculation and largely automated the account verification process, moving it to a system where anyone can pay to have a blue check mark. Critics argue that the practice makes the platform more susceptible to misinformation.
While the authorities easily picked off the explosion photo as a fake due to inaccuracies, the availability of image synthesis models such as Midjourney and Stable Diffusion mean that artistic skill is no longer required to create convincing fakes, lowering barriers to entry and opening the door for potential automation. disinformation machines. The ease with which hoaxes can be created, combined with the viral nature of a platform like Twitter, means that false information can spread faster than it can be fact-checked.
But in this case, the image did not need to be of high quality to make an impact. Sam Gregory, executive director of the rights group Witness, pointed out to The Washington Post that when people want to believe, they let their guard down and fail to verify information before sharing it. He described the Pentagon hoax as a “shallow fake” (as opposed to the more convincing “deep fake”).
“The way people are exposed to those surface fakes, something doesn’t need to look like something else to get attention,” he said. “People will willingly pick up and share things that don’t seem right, but they feel right.”