Recommender systems and algorithms – position statement

As part of eSafety’s commitment to remain current on and anticipate technological change, we review and update our published position statements where necessary.

In December 2022, we published a position statement on recommender systems and algorithms. Since then, there have been a number of changes in the technology landscape.

The updated position statement outlines current and emerging risks and harms, identifies opportunities, explores regulatory challenges and provides guidance on Safety by Design measures. The paper considers new risks and impacts and the increasing influence of artificial intelligence (AI) in terms of platform approaches and content.

Recommender systems present unique risks and harms, which are affected by context and specific design choices and can extend beyond the impacts of individual pieces of content. There are a number of important factors, related to underlying system design, which can impact these risks and harms:

• Social media platforms prioritise engagement when sorting and presenting content algorithmically, but these systems rarely distinguish between positive and negative engagement.
• The impact, severity and type of harm a user may experience from recommended content varies based on a user’s particular circumstances.
• Recommending large volumes of harmful content to a user may cause cumulative effects that can be more impactful and harmful than any one individual post, creating deeper and longer lasting harm.

The continued concern about the role of algorithms and recommender systems in a range of online harms – exacerbated by rapidly shifting platform norms and new technologies – highlights the need for greater, coordinated action.

Industry should take a proactive and deliberate approach to safer design by implementing Safety by Design. This includes giving users more control over algorithm settings and improving transparency about how recommender systems and algorithms are designed and used. User safety, best interests and rights are paramount considerations when developing and implementing recommender systems and algorithms. Industry should make user safety the first and foremost priority. 

Further information about recommender systems is provided in eSafety’s position statement. Topics covered include:

• Background to the position statement
• What are algorithms and recommender systems
• Exploring the risks, harms and impacts of recommender systems
• Current harm mitigation strategies and challenges
• Designing safer systems, including Safety by Design
• Looking ahead.

Click on the file link to download the full position statement. 

If you or someone in your care is experiencing serious online abuse or harm, you can make a report to eSafety. You can also speak to a mental health professional through an expert counselling and support service.

Report incidents of child sexual exploitation and abuse material to the Australian Centre to Counter Child Exploitation (ACCCE). If you believe a child is in immediate danger, call Triple Zero (000) or contact your local police station.

eSafety is developing education and training programs to raise awareness of the impacts of recommender systems, including both their opportunities and risks, and the tools available to manage them.

Targeted online safety education includes:

• Webinars for educators, youth serving professionals, and parents and carers about recommender systems and algorithms, including how algorithms can influence and reinforce harmful beliefs, and the rewards and risks of recommender systems for young people.
• Discussion through key advisory and consultation mechanisms, including with the Trusted eSafety Providers, National Online Safety Education Council (which includes ACARA – the Australian Curriculum and Assessment Authority), eSafety’s Parent Advisory Group, and the eSafety Youth Council, to inform the development of information on our page, Drifting into an ‘echo chamber’? Take control of the algorithms that shape your feed.
• Published advisories on topics such as how algorithms are shaping our adolescents and how violent content is reaching children and what you can do.
• Updates to eSafety’s Toolkit for Schools and Best Practice Framework for Online Safety Education to include guidance on a whole-of-school approach to online safety education and support prevention and responses to online harms.
• Additional resources to support digital literacy are also available.