Meta Promises to Take Action Following Discovery of Instagram Algorithm Encouraging Pedophilia Content

According to The Wall Street Journal, Meta has established an internal team to address the discovery that its systems facilitated the promotion and connection of a large network of accounts dedicated to underage sexual content. Instagram, unlike other platforms, not only hosts such content but also promotes it through its algorithms. The company has acknowledged the challenges in enforcing its policies and has taken measures such as limiting its systems from suggesting searches related to sexual abuse.

“Child abuse is a horrific crime,” Meta said in a statement to the WSJ. “We are constantly investigating ways to actively defend against this behavior.”

Together with the task force, Meta told reporters that it is working to prevent child sexual abuse material (CSAM) networks and is taking steps to change its systems. In the last two years, it has taken down 27 pedophile networks and is working to take down more. It has blocked thousands of related hashtags (some millions of posts) and has taken steps to prevent its systems from recommending CSAM-related terms. It also tries to prevent its systems from connecting potential abusers to each other.

However, the report should be a wake-up call for Metal, the company’s former security chief Alex Stamos told the WSJ. “If a team of three academics with limited access could find such a huge network, there should be an alarm at Meta,” he said, noting that the company has far better tools than outside researchers to map CSAM networks. “I hope the company will reinvest in human researchers.”

According to the report, researchers at Stanford’s Internet Observatory and UMass Rescue Lab were able to quickly find “large-scale communities that promote criminal sexual exploitation.” After creating and reviewing one account for test users, they immediately received “suggested for you” recommendations from potential CSAM sellers and buyers, as well as accounts that linked to content sites outside the platform. Just following several recommendations caused test accounts to be flooded with sexual abuse content.

“Instagram is an on-ramp to places on the Internet where child sexual abuse is more evident,” said Brian Levine, director of the UMass Rescue Lab. The Stanford team also found CSAM content to be “particularly serious” on the site. “The most important platform for these networks of buyers and sellers seems to be Instagram.”

Meta said the company is actively working to remove such users, having removed 490,000 accounts that violated child safety policies in January alone. Its internal statistics show that child abuse occurs in less than one in 10,000 messages, it added.

However, until the journalists’ survey, Instagram allowed users to search for terms that its own systems knew might be related to CSAM material. The pop-up warned users that “these results may contain images of child sexual abuse” that could cause “extreme harm” to children. However, after that, users were given either “Find resources” or “See results anyway”. The latter option is now disabled, but Meta did not respond when the WSJ asked why it was allowed in the first place.

In addition, Instagram’s algorithms often ignored users’ attempts to report child sex content. And sometimes the systems overrode Facebook’s own efforts to exclude hashtags and terms, suggesting users try variations of the name. In testing, the researchers found that viewing even one underage seller account prompted the algorithm to recommend new ones. “Instagram’s suggestions helped rebuild the network that the platform’s own security staff was trying to dismantle.”

A Meta spokesman said it is currently building a system to prevent such recommendations, but Levine said now is the time to act. “Pull the emergency brake. Are the financial benefits worth the harm to these children?” ReturnByte has reached out to Meta for comment.