App Stores Curb Underage Access to Adult Content

With the prevalence of phone usage among children and teens, the youth of today are now more susceptible than ever to the dangers of consuming age-inappropriate content online. The Code of Practice for Online Safety for App Distribution Services aims to tackle this problem by protecting children below the age of 18 from harmful online content.

This article will cover:

• What is the Code of Practice for Online Safety for App Distribution Services
• The key features of the Code
• The penalties in place for App Distribution Services that breach the Code
• Whether you can file a claim of negligence against App Distribution Services and how

What is the Code of Practice for Online Safety for App Distribution Services? 

The Code of Practice for Online Safety for App Distribution Services (‘the Code’) aims to minimise users’ risk of exposure to harmful content, particularly for children, by controlling and limiting the spread of harmful content on App Distribution Services (ADSs). This includes the distribution and storage of harmful content on the apps found on such services. More specifically, the Code requires designated ADSs, such as the Apple App Store and Google Play Store, to implement system-level measures reducing such risk of exposure.

Under the Code, there are six different categories of harmful content. The categories are as follows:

1. Sexual content
2. Violent content
3. Suicide and self-harm content
4. Cyberbullying content
5. Content endangering public health
6. Content facilitating vice and organised crime

The Code of Practice for Online Safety for App Distribution Services complements the Code of Practice for Online Safety for Social Media Services, which aims to protect children from harmful online content on designated social media platforms such as Facebook, Instagram, TikTok and YouTube. The Infocomm Media Development Authority (IMDA) has regulatory oversight of the Code.

To whom does the code apply? 

The Code currently applies to the following designated app distribution services that are deemed to have significant reach or impact:

1. Apple App Store
2. Google Play Store
3. Huawei AppGallery
4. Microsoft Store
5. Samsung Galaxy Store

What are the Key Features of the Code? 

The Code sets out the obligations of ADSs across three areas:

1. User safety;
2. User reporting and resolution; and
3. Accountability.

1. User safety

Content guidelines and moderation measures 

Under User Safety, ADSs must put in place measures to minimise users’ access to harmful content. These include setting content guidelines and standards for app providers to adhere to, alongside content moderation measures such as reviewing apps and app updates before they are released on the ADS and taking appropriate enforcement action if content guidelines have been breached. Examples of such enforcement actions may include warning, suspending or banning the relevant app providers from the ADS.

Removing harmful content from ADSs 

Generally, ADSs must leverage technologies and processes to limit users’ exposure to child sexual exploitation and abuse material (e.g. online grooming for child sexual abuse), as well as content involving terrorism (e.g. content glorifying or promoting terrorist activities or recruitment). Such content must be quickly detected and removed from ADSs. Examples of steps that may be taken by ADSs in this regard include requiring app providers to prohibit such content from being facilitated via their apps, reviewing apps and updates before their release on the platform, and suspending or removing apps that carry such content.

Prohibiting harmful advertising towards children

With regard to children, ADSs must implement content guidelines and content moderation measures that are appropriate for children, with stricter standards applying to sexual content, violent, suicide and self-harm content, as well as cyberbullying content. Children should not be targeted to receive content that could negatively impact their physical or mental well-being, such as by receiving advertisements (including in-app advertisements), promoted apps and recommendations for potentially harmful apps. For example, a children’s game on the app store should not contain third-party advertisements promoting weight loss products or dietary supplements as this could potentially lead to children who play the game and come across the advertisements developing body dysmorphia and even eating disorders.

Restricting children’s access to harmful content

ADSs are required to deploy specific measures designed to protect children from dangerous online content. For example, if the ADS does not generally restrict children’s access to content across all accounts, children must be given differentiated accounts equipped with robust settings for tools to reduce their risk of exposure to detrimental content. These settings must be set to more restrictive levels that are, by default, age-appropriate. Besides restricting access to harmful content, ADSs must also prevent children from accessing apps that are rated 18+ and above by implementing age assurance measures to verify the age of a user.

Implementing age assurance measures 

In line with the measures described above, ADSs must have a system of age assurance (which includes age estimation and age verification) in order to determine the age or age range of account users with reasonable accuracy.

Age estimation uses systems and processes to gauge a user’s likely age or age range. Examples of such processes include Artificial Intelligence (AI), machine learning technology or a facial age analysis algorithm. For instance, ADSs could use the front-facing camera of smartphones to analyse a user’s facial features and approximate their age from there. Such a measure has been implemented by Yubo, a French social networking app, which requires users to take a real-time photograph of themselves within the app, which is then analysed by tech company Yoti to estimate the user’s age. The app goes a step further by using a liveness algorithm to take a short video of the user to ensure that the photo provided is genuine, rather than a screenshot or an image obtained from Google.

Age verification, on the other hand, depends on verified sources of identification such as a digital ID or credit card to determine a user’s age or age range. For example, in Singapore, this could take the form of ADSs redirecting users to Singpass to verify their age before being allowed to download an age-restricted app. ADSs can decide whether to use age estimation or age verification for their system of age assurance, or even both.

Tools and resources for parents 

If children or their parents choose to opt out of the default settings of the child accounts, they must be clearly warned of the potential implications of doing so. Furthermore, children and parents must be able to easily access information and tools to manage children’s online safety and mitigate the risk of exposure to harmful online content. Such resources include app information that would allow parents to evaluate whether a particular app is suitable for children (e.g. descriptions of the app’s features and the type of content on the app). They could also include parental controls or search restrictions, which would enable parents to supervise their children’s usage of the apps.

2. User reporting and resolution

Measures regarding user reporting

In the event that users of ADSs come across any form of harmful content under the six categories described above, they must be able to easily report the offending content via accessible, effective and transparent reporting mechanisms provided by the ADSs. To this end, ADSs must assess users’ reports and take relevant action quickly, with more urgent action being necessary for content relating to child sexual exploitation and abuse, as well as terrorism.

Measures regarding user resolution 

Examples of relevant actions that ADSs can take to resolve users’ issues include the prompt removal of reported content, restricting access to such content, as well as warning, suspending or even banning the app providers of offending apps altogether. If the ADS makes the decision to remove or disable an offending app, users of the app who have downloaded it within the past six months must be informed as soon as reasonably practicable.

ADSs must also follow up expeditiously with the user who submitted the report by informing them of their decision and relevant action taken to address the offending content (unless the user’s report is deemed frivolous or vexatious, or the user had expressly indicated their desire not to be further contacted). The user can then request for a review of the decision and action undertaken.

3. Accountability

Finally, ADSs must submit annual online safety reports to IMDA which will be published on IMDA’s website. The reports must include appropriate information and metrics (subject to IMDA’s agreement) which exemplify Singapore users’ experience on the platform. This is to ensure that users have ready access to easily understandable information empowering them to evaluate the safety strategies adopted by the respective ADSs and make more informed decisions for themselves. For example, parents can use the information to assess whether the ADS is a safe space for their child, or whether they need to tighten control over their child’s access to the service and keep close tabs on the apps they are downloading.

The content of the safety reports could include the specific measures taken by the ADS to manage users’ exposure to harmful content and enhance online safety, alongside an evaluation of the effectiveness of these measures. The report could also include actions taken by the ADS to address reports made by ADS users in Singapore.

Examples of metrics within the report may include:

• The volume of harmful content found on the ADS platform, broken down or classified according to the six categories (i.e. sexual content, violent content, suicide and self-harm content, cyberbullying content, content endangering public health, content facilitating vice and organised crime);
• The outcomes of ADS user reports with respect to each content category;
• Specific apps and app providers offering apps related to the content categories;
• The timeliness of the ADS’ systems and processes in dealing with user reports; and
• The level of effectiveness of the ADS’ measures in detecting and handling offending apps.

What Penalties are in Place For App Distribution Services That Breach the Code?

ADSs that have failed to comply with the Code may be ordered by IMDA to pay a financial penalty of up to $1 million or take certain measures within a specified timeframe to remedy the breach.

If an ADS does not comply with a direction issued by IMDA, that ADS may be convicted of an offence and subject to a fine of up to $1 million. If the offence continues, the ADS could be further fined up to $ 100,000 for every day (or part thereof) that the offence continues after conviction.

Filing a Claim of Negligence Against App Distribution Services

Affected individual users who wish to take action against errant ADSs may consider bringing a civil action against the platform via a claim of negligence. For example, suppose a parent wishes to sue an app store for failing to remove an app that romanticises suicidal ideation and self-harm, causing their child to be influenced by the content within the app and consequently develop suicidal tendencies. Such a scenario is not unprecedented — seven French families have sued TikTok for exposing their adolescent children to harmful content promoting suicide, self-harm and eating disorders, causing two of the children to commit suicide.

First, the user would have to satisfy the threshold requirement that the general harm caused by such an app is factually foreseeable. Hence, if the app features different methods of self-harm but aestheticises the content via attractive visuals such that it appeals to the target audience of children, it could be argued that it is foreseeable that impressionable individuals, especially at a young age, could easily be influenced to try out these methods on their own, causing harm to themselves.

Next, the user would have to prove that the app store in question owes them a duty of care. To do this, the user needs to show the following:

• Legal proximity: It must be shown that there was sufficient legal proximity between their child and the offending app on the app store. In the case of app stores, legal proximity may take the form of causal proximity in which the suicidal tendencies of the user’s child are closely connected to the child’s exposure to harmful content being promulgated by the app, which is directly accessible via the app store.
• Voluntary assumption of responsibility: It must also be questioned whether the app store had voluntarily assumed responsibility towards the user (i.e. whether the app store had taken on the responsibility to make the platform a safe space for children) and whether the user had consequently relied on the app store’s assumption of responsibility (i.e. by letting their child use the app store freely), resulting in harm.
• Public policy considerations: The court also considers public policy when determining whether the app store owes a duty of care to the affected user. An example of a public policy consideration that the court might consider in this case could be whether holding the app store liable for third-party content (such as user generated content) would cause the app store to be overly stringent in its regulation of app providers, resulting in a chilling effect on the variety of content offered by app providers.

Once a duty of care has been established, the user needs to demonstrate that the app store has breached its duty of care to the user. There are two steps involved in this:

• Standard of care: First, the user would have to establish the standard of care owed to them, for instance by showing what a reasonable app store would have done in the same circumstances (e.g. removing the app immediately upon discovery).
• Breach of the standard: Second, the user needs to show that the app store had actually breached this standard of care (e.g. by failing to remove the app despite multiple complaints from other concerned users).

Finally, the user would have to prove causation by showing that but for the app store’s failure to remove the offending app, their child would not have developed suicidal ideation and suffered harm as a result. The user also needs to ensure that the type of harm caused to their child (e.g. psychological harm stemming from suicidal ideation) is not too remote or far-fetched and that it was reasonably foreseeable by the app store.

As shown above, there are many elements that make up a negligence claim and its complexity varies according to the specific facts of the case. Hence, it would be best to seek legal advice from a lawyer to determine whether a user has an actionable claim against the ADS.

For a more in-depth discussion on bringing a civil negligence suit, do refer to our article on suing for negligence in Singapore.

—

To conclude, the Code of Practice for Online Safety for App Distribution Services aims to strengthen the safety of app stores or ADSs (especially for children below 18) by minimising harmful content found on such platforms, equipping users with accessible reporting mechanisms and enhancing the accountability of ADSs. ADSs should be mindful of complying with the Code to improve the safety of their platforms, while individual users of such platforms would do well to be wary of harmful content on these platforms and keep abreast of the relevant reporting mechanisms they can turn to.

If you work for an ADS and require additional legal advice or guidance on complying with the Code, you may wish to reach out to a regulatory compliance lawyer who can assess your company’s specific situation and ensure that your app platform is in compliance with the regulations laid down by the Code. Users may also consult a regulatory compliance lawyer if they wish to check whether a particular ADS has breached the Code, and what actions they can take against the platform. You can use our Find a Lawyer service to kickstart your search for a regulatory compliance lawyer that best suits your needs.