AI Slop Erodes Internet Trust, Demanding New Media Literacy

Key Quotes

"I am easily convinced to quickly reshare any sort of funny you know man on the street content and so there was one of a guy saying something that I thought was very funny in a way that I found hilarious and I shared it very quickly this was Instagram I'm pretty sure but something in my gut said that was too funny you know it was too perfect for me and I went back and rewatched it and then caught the sort of unnatural emotion on the face which I think is for the time being anyway still a tell for AI slop and so I unshared it so that I wouldn't participate in the AI slop economy but my defenses are much lower when the content is funny which I suspect is true for all of us."

Tressie McMillan Cottom describes a personal experience of being fooled by AI-generated content, highlighting how humor can lower defenses and lead to the unwitting sharing of "AI slop." She notes that an "unnatural emotion on the face" was a tell, and that her defenses are lower when content is funny, a sentiment she suspects is common.

"Well I think honestly I'm really sad to admit this but I was fooled by the photographs from last weekend of the Nicolas Maduro capture in Venezuela there were a few that went around that were horizontal and show him handcuffed outside of a of a plane and the reason that I believed them to be real and I believed them to be real for like 24 hours was because they had been shared on Instagram by someone who I trusted and that person had created a reel about how they looked very similar to Saddam Hussein pictures from 2003 so I had just kind of gone with it and when we see images on these platforms we're seeing them very small we're seeing them very quickly and both of those things make it very hard to sit with an image and decode it and make sure that it's real."

Emily Keagan recounts being deceived by AI-generated images of Nicolas Maduro's capture, explaining that her trust in the person who shared them and the rapid, small-scale consumption of content on platforms like Instagram made it difficult to discern the fakes. She emphasizes that the platform's design, which encourages quick viewing, hinders critical evaluation of images.

"I think one of the things that people certainly in my world who think about the digital space and the social world are pretty much in agreement about is that this is not a problem that developing the right skill set is going to solve as Emily points out everything about the affordances of digital technology meaning what the app or the tool allows you to do how it sets up and controls and directs your attention is designed to overcome pretty much anything that we would train a person to do."

Tressie McMillan Cottom argues that developing individual skill sets will not solve the problem of AI-generated content, as digital technologies are designed to capture and direct attention in ways that override human critical abilities. She explains that the "affordances of digital technology" are built to overcome our capacity for discernment.

"I would say I'd be even more pointed and say there is no economic incentive for these platforms to do a better job of making consumers more informed and making them more media literate in fact the incentives are in the other direction so when the tools become cheap enough and accessible enough which is what we have seen with things like you know grog or sora which is really popular with to manipulate video and photo images in particular is that the ability to manipulate images has existed for a very long time it is now however sort of democratized it is available to so many more actors."

Emily Keagan asserts that social media platforms lack economic motivation to improve media literacy among users, and in fact, their incentives lie in the opposite direction. She notes that the increasing accessibility and affordability of AI tools for image and video manipulation have democratized this capability, making it available to a wider range of actors.

"I think the feeling that we are having when we see an AI image or an AI video is very similar to the feeling we have when we read text written by AI which is the words can be in the right order you can recognize the form of it as being a sentence a paragraph a story or a book but you do not have the appropriate emotional response to it now we can get mystical and say that there's something about the human spirit right that we infuse into our art and I am not disinclined to believe that but I think that whatever that process is AI can look like reality but it cannot communicate emotionally to us in a way that's that resonates as being authentic."

Tressie McMillan Cottom suggests that the emotional disconnect experienced with AI-generated images and text stems from AI's inability to imbue its creations with authentic human emotion, even if they mimic reality aesthetically. She posits that while AI can replicate the form of communication, it cannot convey the emotional resonance that connects with audiences.

"I think the reason you know that there's not a shark swimming in the flood outside your door is because it's not in the papers of the new york times I think that at this point the only way to know if an image is quote unquote real is if the person who's trafficking it is a place or person that you trust and is verified so listen you're on tiktok at 2 am and you think that the cute baby saying the you know the funny thing you know if it's funny no harm no foul right I think the question becomes when do we allow the image to move us to act whether that means we share it whether that means we get very angry or anxious about you know this is some you know major news development right then the question becomes wait is this real that question becomes all the more important."

Tressie McMillan Cottom advises that in the current landscape, the trustworthiness of the source is the primary indicator of an image's authenticity, rather than inherent visual cues. She suggests that the critical question arises when content prompts action, at which point verifying the source becomes paramount, especially when engaging with content on platforms like TikTok.