What are Filter bubbles?
Filter Bubbles and Extremism
There is a paucity of research studying the effects of personalisation algorithms on extremist content.
YouTube's recommended videos can propel users into an immersive bubble of right-wing extremism (O’Callaghan et al., 2015)
Twitter’s “Who to Follow” suggested violent extremist Islamist groups if the user followed al-Qaeda affiliated group (Berger, 2013)
Facebook’s “Recommended Friends” function had likely actively connected at least two Islamic State supporters in SE Asia (Waters and Postings, 2018)
The architecture of the platforms may facilitate closer interactions than would otherwise exist.
Research Question
Do algorithms promote extremist material once a user begins to interact with such content?
How to measure extremist content?
Research Question
Do algorithms promote extremist material once a user begins to interact with such content?
How to measure extremist content?
- Hand-coding content with Holbrook's Extremist Media Index
Holbrook's Extremism Media Index (2015)
Overview of the data:
Of the 1443 videos coded on YouTube (749 unique)
- 65.77% moderate
- 28.34% fringe
- 5.89% extremist
Figure on the left shows the EMI scores for each session with a rank from one to eighteen, depending on where the video appears on the “Recommended Videos” section. Figure on the right shows the percentage distribution of the three categories of content before and after each treatment.
After extreme interaction treatment:
- The incident rate for fringe content is 1.72 times the incident rate for the reference group
- The incident rate for extreme content is 2.47 times the incident rate for the reference group
After extreme interaction treatment:
- Fringe content 1.37 (p < 0.01) times more likely than before
- Extreme content 2.00 (p < 0.01) times more likely than before
- In the neutral interaction account
- only three sessions had an extreme content piece after interaction
- In the extreme interaction account
- all but one session had an extreme content piece after interaction
- In the extreme interaction account
- Extreme content ranked sig. higher (p = 0.028) than moderate
- Almost all extreme content shows up in the upper half (< 8) of the recommendations
(Median Rank = 5)
Of the 2100 posts coded on Reddit (834 unique)
- 78.76% Moderate
- 19.81% Fringe
- 1.43% Extreme
No sig. observable in-/decrease of content
No sig. difference in ranking of content
Different setup due to technical difficulties
3 Different News Feeds: “Popular”, “Controversial”, and “Latest”
3 Topics: “Politics”, “News”, “Humour”
Collected data over five sessions
1271 Rated posts (746 unique)
- 63.73% Moderate
- 28.8% Fringe
- 7.47% Extreme
No statistically sig. difference between extreme & non-extreme content in any of the timelines
Discussion
Do algorithms promote extremist material once a user begins to interact with such content?
Only YouTube has an effect which, after engaging with extreme content, prioritises it further.
Users which engage with extreme and fringe content are more likely to be recommended more of the same.
Extreme content is pushed up the ranking of recommended videos on YouTube
Support for O’Callaghan et al. (2015)
Discussion
Safe haven for right wing extremists and home to terrorists such as Pittsburgh Synagogue Shooter
Anecdotally, by far the most extreme of the platforms.
Lack of evidence of Gab’s algorithm suggests that it is the user’s choices which are responsible for this environment.
Recommendations
Removing Problematic Content from Recommendations
Content which does not clearly violate site rules or policies
Google’s “limited features” policy
Reddit’s “Quarantine” System
Opt in content
No monetisation or recommendation
Constructive balance between freedom of speech and harmful content.
Need for Clarity and Consistency
OILAB: Availability of YouTube videos posted in Nazi threads on 4chan/pol/ on 5 and 6 June 2019.
Recommendations
Ensuring Video Recommendations are from Quality Sources
Provide users with more context and alternative perspectives
Google introduced changes to make quality count and give more context to searches.
Provide high quality sources on the same topic
Jigsaw’s redirect method as a model.
Recommendations
Greater Transparency
Users should have a clear option to request why content has been recommended to them.
Opportunity for Explainable AI
Facebook’s “Why am I seeing this ad/post”
Future Research
More accounts over a longer period of time
Research Project constrained by the number and type of social media platforms that we could research.
Closed nature of the platform (Facebook)
Terms of Service Restrictions (Twitter)
Increasing knowledge gap which can only be answered through close collaboration with social media companies.
What are Personalisation Algorithms?
Pariser 2011
Algorithms are responsible for content that users see in their feeds
Eli Pariser suggests that they can create a “filter bubble” effect or “autopropaganda”
by controlling what users do and do not see it can – and is in fact, designed to – dramatically amplify confirmation bias
The pre-filtering of content leads to bubbles in which people never view or read about opposing viewpoints
Creates seperated spaces that make communication between opposing viewpoints harder: undermines democracy itself
Figure shows the first appearance of a piece of content (Moderate, Fringe or Extreme) for each session.
Figure shows the first appearance of an extreme content piece for each session.
- In the neutral interaction account
- only three sessions had an extreme content piece after interaction
In the extreme interaction account
- all but one session had an extreme content piece after interaction
- Almost all content shows up in the upper half (< 8) of the recommendation list
(Median Rank = 5)
Poisson Regression Results:
Reddit Personalisation
