it is The Stepback, A weekly newsletter telling an essential story from the tech world. Follow Robert Hart for more on AI illusions. stepback Delivers to our subscribers’ inboxes at 8am ET. opt in for stepback Here.
At first, AI influencers were relatively easy to identify and ignore. Apart from the occasional promotion, they didn’t seem to change much in the way social media worked. First Virtual Influencer – Lil Miquela With his blunt tusks and freckles, Imma With her bubblegum pink bob, and Shudu Village With her flawless complexion – obviously there were digital productions. were cooperation announced with fanfare. Post requires a studio, money, coordination, and a lot of polish.
Over time, I’ve noticed that the fake people on my Timeline have started to look like everyone else on it. characters like Emily Pellegrini And aitana lopez Got a little closer to reality — or at least the reality of that well-traveled, nice friend from college who you haven’t kept in touch with, always posting from nice restaurants and beautiful places, or from Coachella and Wimbledon. Not exactly related, but, then again, most professionals aren’t influencers either.
Nevertheless, many of these accounts are not standard by any means. Lopez is the product of a Spanish creative agency called The Clueless, which manages a group of AI influencers. Creator Pellegrini, who goes by the pseudonym Professor EP, told me he used to manage OnlyFans Creators. He now sells courses that teach people how to become AI influencers for themselves.
This is what people are starting to do. Many people.
The novelty has worn off. Early AI influencers emerged because there were so few of them. Now they’re part of a larger mess of AI-generated content flooding social media: low-quality drivel easily copied from chatbots, sloppy images and videos, and that’s tempting. lord of the rings disco song Which took over my TikTok for a month.
Fake people are everywhere now. They’re selling drop-ship junk, defrauding men of money through fake photospush disinformation And casteism talking points, and having an increasingly awkward, frequently fulfilling sexual relationship. Of course, there are Very Of thirst trap. It also has a lot of worldly content, just with the incarnations. copy Whatever is popular among human creators is often appropriate putting my fake face on it.
This makes it difficult to measure the scale of an AI content creator’s impact. Platforms do not publish statistics on how many of their users are fake people, and most AI avatars do not become popular or influential enough to justify the media attention the previous wave received. like database virtual man Track hundreds of popular avatars, but only those accounts are weird, awkward, or big enough to attract attention. Beneath them is an ocean of accounts that are flying completely under the radar.
One reason these accounts avoid detection is that the technology used to create them has improved massively. A still image of a fake person can now be good enough to come across as real at a glance, especially in a feed full of real influencers making liberal use of staging, filters and editing effects. Video and audio are quickly catching on, giving virtual people voices and movements that can fool unwitting scrollers. Equipment is no longer specialized or prohibitively expensive. Mainstream products from companies like Google and OpenAI exist alongside specialized services from companies like Higgsfield, Hagen, and ElevenLabs. With a little effort, almost anyone can create an AI influencer – or a stable of them – without the need for a studio, special equipment, or (much) money.
All this leaves social media platforms with a problem they aren’t particularly interested in solving. After years of grappling with AI-generated images, video and audio, most major platforms now have some form of policy covering synthetic media. But beyond requiring labels for AI-generated content, such rules often do little more than include content in existing categories covering things like scams, spam, impersonation, and graphic content. AI people, especially people designed to behave like real people, don’t fit neatly into any of these buckets. They don’t necessarily have to be running a scam, posting graphic content, or impersonating someone – who would they be impersonating? And if they do disclose that their posts are AI-generated, it’s unclear what rules they might be breaking.
For now, the platforms seem content to live in obscurity, neither fully welcoming nor disdainful of AI creators. They have created a paradoxical situation, promoting AI as a creative tool while also trying to stem the tide of sloppiness that is overwhelming their services. youtube, tiktokInstagram and other platforms have developed rules for labeling synthetic media, particularly the realistic kind, as well as promoted their own suites of AI tools, including some that can clone or simulate users. But those rules focus on individual posts rather than the accounts and the personalities behind them, leaving AI influencers in a gray area.
In that uncertainty, the AI influencer ecosystem is thriving. Some market research firms guess The virtual influencer market could be worth more than $60 billion by 2030, up from about $12 billion this year. Cultural influence is also increasing. AI has impressive rewards, beauty contestsDedicated talent agencies Representing synthetic creators, and an emerging market of synthetic creators, is selling courses and tools that promise to help people build and run their own fake creators, often with the promise of faceless passive income. Some of it has the faint pyramidal smell of an online gold rush, there are few visible success stories and a lot of people selling shovels.
My guess is that there is going to be a reckoning. AI slop is already troubling, and there’s only so much a platform can take until it makes it practically unusable, especially given their persistent refusal to let AI slop filter out users. Fake people pretending to be real are an even more intimate version of the same problem. But beyond labeling and enforcing existing rules, platforms seem mostly content to see what happens. For the platform, engagement is still engagement, whether it comes from a fake creator or a real one. As long as synthetic creators keep posting and don’t go outside the existing rules, there’s no incentive to put a stop to it.
There’s also the question of how sustainable the whole idea of running AI avatars online is. If so many creations are built just to make money from human users, what will happen when the pool of human users dries up? For example, there are many people who would be willing to purchase courses and tools to create their own influencer. It is believed that social media can survive the influx of AI influencers. By definition, it takes some critical mass of humanity to keep things social. If left unchecked, the network will become burdened by these impersonators, as human users will inevitably turn away.
This may change if public anger continues to grow. The backlash over deepfakes, impersonation, and synthetic spam is already forcing lawmakers and regulators to pay closer attention, especially after incidents involving non-consensual sexual deepfakes generated from tools like Grok. Europe’s AI Act could be a driver, at least when its transparency obligations apply to AI-generated content. rules Deployers of generative AI systems would be required to clearly disclose AI-generated or manipulated content, which could put pressure on companies to flag AI content or potentially face hefty fines. But even so, the focus is still largely on the content, not whether the account posting it represents a real person.
As with so much on social media, the burden falls back on users. Many platforms have effectively handed over the task of moderating AI content to users, relying on them to identify and report suspicious profiles. But self-control is a poor and lasting response to something designed to avoid notice. The appetite for AI-free spaces is already growing. If the platforms themselves refuse to draw the boundaries between real and unrealistic, I hope users will draw them instead.
- Many of the high-profile AI influencers I’ve encountered recently are openly political leanings, which I think could accelerate the regulatory reckoning. Danny Bones, A Fake white nationalist rapper funded by a far right political party In the UK, probably the best example of this I’ve seen so far.
- Like human influencers, many AI avatars are created around specific identities and communities, such as race, disability, politics, and nationality, such as the MAGA fantasy girl. jessica fosterWhich leans heavily on the military’s erotic aesthetics and Trumpism. Not all avatars align with their creators: for example, the black AI model Shudu Gram was made by a white man. Emily Pellegrini is also the product of a man, Professor EP, who told me the character was created using material licensed from an anonymous OnlyFans creator.
- Title of Jess Weatherbed’s recent story The Verge It says it all: “Let us filter out the AI slop, you coward.”
- The Verge Recently it was reported that grifters are using AI avatars of fake black people to sell mass-produced products via drop shipping at inflated prices on social media.
- wired informed On the rapidly growing “AI pimping” industry, where human creators are having their content stolen and monetized by AI avatars without their permission.
- Charlie Warzel’s Podcast examined The impetus behind the proliferation of AI influencers and the weariness many people feel when it comes to worrying about whether what we consume is real or not.
