Manipulated by Machines
CyberGunk 04
Membership content
Key Research
The Century of the Self
Specifically part 1 here has large chunks lifted and (hopefully) flipped. I never actually watched it back or used it as reference because I didn’t want to directly copy anything, but I know that this is where half the story comes from.
Research List
Filtered to Astroturfing View allTitle | URL | Notes | Video | Relevance | |
---|---|---|---|---|---|
Amazon Recruited Twitter Army to Defend Company and CEO Jeff Bezos, Leaked Document Reveals | https://www.ign.com/articles/amazon-twitter-army-defend-company-ceo-jeff-bezos-leaked-document-reveals | Amazon’s in-house astroturfing campaign | Astroturfing | ★★ | |
An Examination of the Impact of Astroturfing on Nationalism: A Persuasion Knowledge Perspective | https://www.mdpi.com/402650 | Actual research into astroturfing | Astroturfing | ★★★ | |
Bird Box challenge: How Netflix made a few tweets into a global trend | https://inews.co.uk/news/bird-box-challenge-netflix-twitter-warning-explained-241879 | Faked “Bird Box Challenge” | Astroturfing | ★★★ | |
Bird Box: How Netflix Harnesses It’s Memeing-ful Relationship with Fans | Research World | https://www.researchworld.com/bird-box-how-netflix-harnesses-its-memeing-ful-relationship-with-fans/ | Bird Box campaign overview | Astroturfing | ★★★ | |
Characterizing Retweet Bots: The Case of Black Market Accounts | https://arxiv.org/abs/2112.02366 | “Bot checkers” don’t catch retweet bots | Astroturfing | ★★★★★ | |
Data Against Democracy | https://logicmag.io/failure/data-against-democracy/ | Platforms warping politics | Astroturfing | ★★ | |
Elon Musk’s not-so-secret weapon: An army of Twitter bots touting Tesla | https://www.msn.com/en-us/news/technology/elon-musk-s-not-so-secret-weapon-an-army-of-twitter-bots-touting-tesla/ | Tesla astroturfing | Astroturfing | ★★ | |
Ephemeral Astroturfing Attacks: The Case of Fake Twitter Trends | https://arxiv.org/abs/1910.07783 | How to fake a twitter trend | Astroturfing | ★★★★★ | |
Facebook Campaign Nets Big Returns For ‘Bird Box’ | https://www.mediapost.com/publications/article/330557/facebook-campaign-nets-big-returns-for-bird-box.html | No trace of official Twitter Bird Box ad spend beyond hashflag | Astroturfing | ★★★ | |
Group topic-author model for efficient discovery of latent social astroturfing groups in tourism domain | https://link.springer.com/article/10.1186/s42400-019-0029-8 | Actual research into astroturfing | Astroturfing | ★★★★ | |
How Custom Emojis Crowned Twitter the Hashtag Registrar | https://medium.com/@justinogarrity/how-custom-emojis-crowned-twitter-the-hashtag-registrar-43e5fa60c557 | History of hashflags | Astroturfing | ★★ | |
Lil Nas X Was A Popular Twitter User Before ‘Old Town Road’ | https://nymag.com/intelligencer/2019/04/lil-nas-x-was-a-popular-twitter-user-before-old-town-road.html | Lil Nas X, the tweetdecker | Astroturfing | ★★★ | |
Misleading Repurposing on Twitter | https://arxiv.org/abs/2010.10600 | Sockpuppets start as real accounts | Astroturfing | ★★★★ | |
Netflix apologizes for using actors to meet press at Canadian launch | https://www.theglobeandmail.com/technology/netflix-apologizes-for-using-actors-to-meet-press-at-canadian-launch/article4326706/ | Netflix doing good old irl astroturfing | Astroturfing | ★★ | |
Netflix denies spreading ‘Bird Box’ memes with Twitter bots | https://www.dailydot.com/unclick/bird-box-memes-fake-netflix/ | Bird Box wasn’t astroturfed, but why are all these links dead | Astroturfing | ★★ | |
Political Astroturfing on Twitter: How to Coordinate a Disinformation Campaign | https://www.tandfonline.com/doi/abs/10.1080/10584609.2019.1661888 | Actual research into astroturfing | Astroturfing | ★★★ | |
Power of Suggestion | https://www.chronicle.com/article/power-of-suggestion/ | Many priming studies can’t be replicated | Astroturfing | ★★★ | |
Public Relations and Participatory Culture: Fandom, social media and community engagement | https://www.worldcat.org/title/public-relations-and-participatory-culture-fandom-social-media-and-community-engagement/oclc/1127300821&referer=brief_results | PR and fandoms | Astroturfing | ★★★ | |
Social Media and Public Relations: Fake friends and powerful publics | https://www.worldcat.org/title/social-media-and-public-relations-fake-friends-and-powerful-publics/oclc/1089492876&referer=brief_results | General new media PR textbook | Astroturfing | ★★★ | |
Strategic Communication, Social Media and Democracy: The Challenge of the Digital Naturals | https://www.worldcat.org/title/strategic-communication-social-media-and-democracy-the-challenge-of-the-digital-naturals/oclc/1064560421&referer=brief_results | General new media PR textbook | Astroturfing | ★★★ | |
Survive Reddit And Make An Impact | https://wearesocial.com/uk/blog/2019/10/survive-reddit-and-make-an-impact/ | How to infiltrate Reddit | Astroturfing | ★★★★ | |
The Bird Box Effect: How Memes Drive Users to Netflix | https://www.theringer.com/movies/2019/1/3/18167278/bird-box-memes-netflix-bots-marketing | Bird Box wasn’t astroturfed, but whats all this bot activity | Astroturfing | ★★ | |
The disconcerting potential of online disinformation: Persuasive effects of astroturfing comments and three strategies for inoculation against them | https://journals.sagepub.com/doi/abs/10.1177/1461444820908530 | Actual research into astroturfing | Astroturfing | ★★★ | |
The lesson of the dust-up over Trump’s fake Twitter followers. | https://slate.com/technology/2017/06/the-lesson-of-the-dust-up-over-trumps-fake-twitter-followers.html | We only have conjecture but there is manipulation happening | Astroturfing | ★★ | |
The Stans Who Post So Much, Twitter Thinks They’re Bots — And Shuts Them Down | https://melmagazine.com/en-us/story/the-stans-who-post-so-much-twitter-thinks-theyre-bots-and-shuts-them-down | Stans and “harmless bots, including ones that promoted Netflix shows” | Astroturfing | ★ | |
There’s Nothing Social About Social Priming: Derailing The “Train Wreck” | https://www.tandfonline.com/doi/abs/10.1080/1047840X.2021.1889312 | On “priming” post replication crisis | Astroturfing | ★★★ | |
Thinking Taxonomically about Fake Accounts: Classification, False Dichotomies, and the Need for Nuance | https://arxiv.org/abs/2006.04959 | “Bots” encompasses so much | Astroturfing | ★★★ | |
TikTok “Deeply Concerned” by Facebook’s Paid Campaign to Promote Negative Coverage of Social App | https://www.hollywoodreporter.com/business/digital/tiktok-facebook-meta-targeted-victory-1235122641/ | Facebook shady anti-competitive campaign | Astroturfing | ★ | |
Weekly Nielsen Streaming Charts – Archive | https://www.spoilertv.com/2020/11/weekly-nielsen-streaming-charts-updated.html | Viewership numbers | Astroturfing | ★★★★ | |
Who Really Writes Twitter’s ‘Trending’ Summaries | https://onezero.medium.com/who-really-writes-twitters-trending-summaries-89e819255599 | On the new team behind the twitter trends | Astroturfing | ★★★ |
Audio Draft
This content is for members only. Join Now
Shooting Script
The tech platforms are successfully pretending that online advertising works. Which is great for them, financially. But when they actually need to do some advertising, it’s a little embarrassing because there’s nowhere to go. So there’s been a bit of a return to an earlier form of marketing: Public relations.
PROPAGANDA MACHINES
So, just quickly here, there was a guy called Edward Bernays, who invented public relations, because, in his words, he needed a nicer way of saying “propaganda.” Now, maybe he is the inventor of public relations or maybe it’s just because he ran a public relations campaign which made him seem like the inventor of public relations. Either way, a relatively influential guy.
He was a nephew of Sigmund Freud, and he took a lot of inspiration from Freud’s ideas of the subconscious, y’know: that we’re all fundamentally violent irrational beings that must be controlled. So public relations is all about controlling the masses through secret messaging. Mostly, it involves sending press releases to journalists.
And ever since, that’s made up about 40% of all news, right up until recently, when journalism was destroyed by tech, and now it’s about 80%.
But I guess Bernays sort of famously did the campaign to get women smoking Lucky Strike. There were two problems with that 1) the packaging was a really ugly green and 2) women weren’t smoking.
So first, he goes to a doctor, and says “what are the health benefits of cigarettes,” and the doctor goes, “look, it’s 1928, and I’m a fucking idiot so it’s probably healthier than eating because it keeps you thin.” Bernays goes “fantastic, can I get that in writing.” Then he starts promoting the ideal of thinness, and then promotes the cigarettes as something that’ll help you with that.
He also couldn’t change the colour of the packaging, so instead he persuaded designers, tastemakers and charities to make this shade of green “cool.”
But his big one was inspired by the psychoanalytic idea that women will actually want cigarettes in their mouth because of… Y’know. Freud. The idea is that it’s actually like an empowerment thing. “I reject your penis and substitute my own.”
So he paid activists and models to march smoking cigarettes. Which he took a bunch of photos of and released a bunch of articles about, calling them “Torches of Freedom,” and made it out to be this grassroots activism sort of thing.
And that’s what’s come to be known as astroturfing. Astroturfing because it looks like it’s grassroots but it’s fake. That’s not my wordplay, that’s a sociologists wordplay.
And recently that’s become a bit of a buzzword for American journalists complaining about social media.
Actually, my favourite twitter account of all time was a Russian bot.
It exclusively posted one blurry picture of an older Russian woman, links to articles about Trump, and misspellings of “Harrison Ford.” Incredible, next level posting. Blurry picture, Trump article, Hairson Ford. Blurry picture, Trump article, Harasion Ford. Blurry picture, Trump article, Harasion Ford. That’s not a joke I came up with for this, that was an actual thing. But it’s not the sort of thing that’s going to change election outcomes.
Just because bots are an exciting idea, they’re not the biggest problem. But there are much more effective techniques, and the fact that bots like that are even being tried, is indicative of a much larger presence of manipulation online.
Now, one of the descendants of Edward Bernays is a man– called Mark Randolph, and he founded Netflix. The first DVD mail service. Instead of going to a video store to get your DVD’s, you order them online, but then you still had to go and post them back, so it’s not… It’s a stupid idea. But they were able to beat out the video stores because they were willing to take on massive losses.
And before they launched Mark Randolph hired Corey Bridges to do something that they called “black ops.” Where he would infiltrate DVD forums, ingratiate himself with the community over the course of months, and then go “Hey, have you guys heard of this new service Netflix, I love this new service Netflix.” And that’s how Netflix got their first 1000 subscribers. By lying to people.
And this was insanely effective, the effective thing about it wasn’t convincing individual people to buy Netflix, it was using the communication power of the internet to exploit network effects: You convince a few people to do your marketing for you, and then they convince a few people and so on and so on.
Like Nutella, right? Y’know, Nutella is one of the worst inventions of all time: Worse chocolate that’s less healthy than chocolate. Obviously they got away with it in the 60’s because people used to be idiots, but you wouldn’t expect them to get away with it now. Except, at the start of the 2010’s they were one of the first brands to have a robust social media presence. So now all the TikTokers and Instagramers do their marketing for them.
Netflix did really well at encouraging social media engagement in the early days, so they’ve got sort of a footing.
Any sort of organic campaign, transmedia narrative, encourages engagement and transforms fans into brand ambassadors. Or “stans.”
The stan accounts are interesting to me because they openly engage in the manipulation of social media. Proudly falsifying social movements, inflating streaming numbers, manipulating algorithms, to promote a thing they like. I’ve found that almost every time Taylor Swift’s entered the Twitter trending topics over the past few years, it was organised and orchestrated by Taylor Swift fans. Grassroots astroturfing.
[explain how?]
A stan account of particular interest to me was nasmaraj, he was a Nicki Minaj fan, and tweetdecker. Which is basically a type of guy who would dig up old viral posts and repost them verbatim, then use the platform tweetdeck to immediately give those posts outsized engagement, which would force them into people’s feeds. Now he was trying to turn this into a career when Cowboy memes started trending so he quickly whipped up a meme song called Old Town Road. And then used his network to promote this across social media.
You can go back and find Reddit posts of him impersonating Redditors going, “hey what’s the name of the song that goes take my horse to the Old Town Road,” and it all has obviously manipulated upvote ratios, and likes. And through this aggressive push, he was able to develop this song into a bonafide TikTok meme. And then through chart manipulation and continual re-releases it became an actual hit.
And I mean, I don’t. I’m not saying that that’s bad or anything. I think there’s something genuinely aspirational that one person, alone, working day-in-day-out to manipulate social media, could actually have that big of an effect.
The only problem is that it seems to be an approach that’s been adopted across the music industry. But there’s no way to actually say who’s doing this manipulation, or how they did it or why. No one’s gonna tell you that. The interesting thing about Lil Nas X, is the second he got an actual PR team, they spent years successfully sweeping his social media manipulation under the rug, even though most of it’s still online and hundreds of thousands of people had actually seen it happen.
People have talked about Netflix manipulating social media for years. It is weird that “Netflix and Chill” suddenly changed meaning on the 6th of October 2014, without any inciting meme or popular post, just everyone collectively decided to start using it in an entirely new way.
And people noticed that the viral memes around Netflix releases were coming from accounts with no followers, so people theorised about bots, or sockpuppets. But then people got in touch with some of those accounts and a lot of them were actually real people. And ever since then, that has been a trend that we see with the Netflix and Disney releases: small, new accounts with viral tweets.
I also noticed this. So when Bird Box came out, I was running analysis on the tweets. Remember that? Bird Box? The biggest movie of all time? No? Ok.
The Bird Box campaign began with a lot of physical advertising. Billboards, posters, installations, a travelling escape room, a theatrical pre-release. And that’s all they need to do, they don’t need to spend on programmatic advertising, they can do this sort of bullshit, and then fans will do their social media for them. The only demographic they did spend on was boomer Facebook. Because they’re the only people still falling for that sort of shit.
They also spent on influencer marketing, they worked with some Twitch channels to do “the Bird Box challenge,” which was where you had to play a game blindfolded. They worked with popular influencer Birbs Rights Activist, to make some tweets and videos. And they bought a hashflag, which is that little emoji beside a hashtag. When those came out they were $1 million a piece, nowadays you get a few bundled in with every 400k quarterly ad spend. Anyway, the point is, big marketing campaign, but a lot of that money going in bizarre directions.
Then it’s released, and immediately it starts trending on Twitter and does so for several days, but with different keywords, regularly cycling out: hashtag birdbox, bird space box, hashtag birdbox netflix, birdbox space netflix… Which is interesting, because it’s difficult for something to stay in the trends, trends are looking at what’s increasing in volume. So it’d be much easier to keep something trending, if you could convince people to change what keywords they were using about every 6 hours.
Particularly egregious is when #BirbBox starts trending. For seemingly no reason. No one was tweeting it, and now it is trending, most people talking about it are saying, “why is this trending it’s a typo.”
Now, Netflix had used that hashtag before Bird Box came out, for their collaboration with Birbs Rights Activist. But now, a week later, with like 60 tweets from all time using that hashtag, it’s trending.
Now there’s a quirk in the trending algorithm, it counts tweets as they come in. It doesn’t account for deleted tweets. So using bots or compromised accounts, you make thousands of tweets and immediately delete them, and it will count those towards a trending topic. This is actually incredibly common, it happens 30-40 times a day. And in the few minutes before #BirbBox enters trending, we do see tens of thousands of tweets being made and immediately deleted.
[jokey disclaimer]
You also have 9,000 people posting the exact same meme with the exact same typo. And yet, run those accounts through a bot-checking tool and none of them show up as bots.
[no one talks about the show on any other social media until after the memes kick off]
[jokey disclaimer]
[i did data collection of the tweets at the time]
Then there’s these tiny accounts with the viral memes. Also seemingly not automated. But what I saw at the time was that tweets, from small accounts, about Bird Box, were getting their likes inflated.
The recommendation algorithm takes into account how much engagement you’re getting versus your account average. So if you’re a 10 follower account getting 0-1 likes per post. A targeted attack to give you 12 likes will disproportionately push that post into people’s feeds. So just with a few likes, you can have an outsized effect on what’s being shoved into people’s faces.
And these likes were coming from the same few people, with multiple alts operating in a network, like the tweetdeckers.
Now I wanted to check in with these accounts, 5 years later, follow up with them, actually ask them some questions, “What was all of that liking Netflix tweets about?” But they’re all suspended now, obviously. So, we can’t know why this happened, maybe it was just the Sara Paulson stan accounts, who were rabid at the time.
[segue]
Now at the end of this first week Netflix comes out and says that 45 million accounts have watched Bird Box in its first week, or at least watched 70% of it. And that’s just accounts right, so if 4 people watched it twice, that only gets counted once. Which, people have pointed out if you convert that to Box Office that would be at least in the top 5 openings of 2019, sort of the equivalent of Star Wars: The Rise of Skywalker, easily beating out Captain Marvel or Joker, franchises like Mission Impossible or Fast and Furious, and out-performing A Quiet Place 5 to 1.
But it’s maybe possible, there wasn’t really any good third-party tracking at the time.
But now we do have much better stats, so we can look at something like Army of the Dead which Netflix says was watched by 75 million accounts in its first 4 weeks. Nielsen says it was 2 billion minutes domestically, which would be about 36 million views total. I think a good little check to do on any number that Netflix releases is ask “Have they just doubled it?”
These days they count a “view” as being 2 minutes watched. Still, it feels very improbable to me, that the majority of people would watch a movie for 2-10 minutes then give up. But to be fair, it is a Zack Snyder movie.
At the very least, they’ve found a creative way of counting. But whenever they release these numbers it creates a big news story, and it creates the movie’s success. The biggest day for Bird Box on Twitter, on Google Trends, on IMDb, on Metacritic, on Letterboxd was the day after they said that it had 45 million views. They told people it was successful and then it became successful.
Now another week passes and they put out another tweet warning people not to do the Bird Box challenge. Again, big news story, big day for Bird Box in the trends. But– no one had actually been doing the Bird Box challenge. There were a couple of videos that were basically unnoticed, and there was their own promotion that they paid for. The Twitch streamers.
Of course as soon as they say not to do it, y’know Logan Paul’s out there blindfolding himself and running over pedestrians, but it didn’t exist before then.
[I dont have all the answers I dont even have all the questions.]
[It’s widespread now]
Now, this does all paint quite a bleak picture. That you could manipulate the masses from the shadows, just with targeted attacks on social media. But interestingly, I saw a similar behavior on the day Tiger King came out. And Tiger King wasn’t a success until weeks later.
So I think it tells us that yes, you can put things in front of people. You can force people to see things on social media, but you can’t make them like it.
So yeah, the technology is there for Russian bots to make you see tweets about Donald Trump, but it’s not going to change anyone’s beliefs. Cambridge Analytica were doing campaigns to make people feel like voting less, which is a scary idea, but their own metrics show that they were entirely unsuccessful.
Recently in psychology, it turns out a lot of foundational studies into the subconscious, and priming, the idea that you could change the way people act, just using words and symbols, cannot be replicated. Obviously you can manipulate people, by talking to them. But the fundamentals of this mass-scale Bernays-style subconscious manipulation, doesn’t seem to work at all.
The fundamentals of advertising still work, putting something in front of someone over and over again makes them think of that thing more often than if you hadn’t done that. Obviously that works, but the rest is a grey area.
When Edward Bernays hired activists to smoke, it had an outsized effect on Lucky Strike sales. Now he thought that this was because of the subconscious associations with oral fixation and that women everywhere were being subconsciously manipulated into smoking.
But, his campaign to promote the ideal of thinness didn’t seem to help sales, and despite his campaign to make the color green cool, Lucky Strike had to change the colour of their packaging not long after.
Maybe the “Torches of Freedom” appealed to people who already liked smoking and women’s rights. He wasn’t luring people with their subconscious desire for empowerment and penises, he was unknowingly converting cigarette fans into stans.
So yes, there’s manipulation, more than ever we are surrounded by media manipulation. But mostly, it’s algorithms manipulating algorithms, and we seem to be stronger than we think.