MIT Technology Review · 5 hours ago
by Kelly Clancy · Humans and technology
Pam Wisniewski’s digital adolescence showed her the best and the worst of the internet. At 14, she left an abusive home, where she’d been isolated in a fifth-wheel trailer at the end of a seven-mile dirt road. She moved in with her older sister and taught herself to type on AOL Instant Messenger. Online, she sought out the support and the community she’d lacked at home. She also discovered how thin the ice can be. “I sent my address to some guy in New Mexico to send me a mug with my name on it,” she recalls. “And then I found a news story like five, 10 years later that he killed somebody.”
Those experiences set the course of her career. Wisniewski—now a principal research scientist at the International Computer Science Institute, a nonprofit affiliated with the University of California, Berkeley—has spent well over a decade asking what safety should look like for families navigating an evolving tech landscape, and how to achieve it without sacrificing trust. “I really see the internet as this double-edged sword,” she says.
Digital harms have become the defining fear of American parenthood. In the University of Michigan’s 2025 National Poll on Children’s Health, parents’ top three worries had to do with social media, screen time, and internet safety. Nearly half of American teenagers say they have been bullied or harassed online, according to the Pew Research Center. From there, the dangers escalate. Online drug dealers sell counterfeit pills laced with fentanyl. In the first half of 2025, the National Center for Missing & Exploited Children fielded more than 23,000 reports of financial sextortion, in which a predator posing as a peer extracts a sexual image from a child and threatens to publish it unless paid. Chatbots are the newest danger, with companies like OpenAI facing lawsuits for allegedly coaching children toward suicide.
Most parents’ first defense is conversation. In Pew surveys, more than nine in 10 say they’ve talked with their teens about what’s appropriate to share online and how to treat peers. They get an assist in limiting exposure from tools that come preinstalled on phones: Apple’s Screen Time and Google’s Family Link let parents cap screen time, block or approve apps, filter web content, and track devices. There are also apps that let family members share locations; Life360, the largest, has nearly 98 million monthly users.
But a growing number of adults are opting for tools that go further. Rather than simply restrict or locate, content-monitoring apps scan a child’s texts, photos, emails, and chats and alert parents whenever an algorithm flags something it deems dangerous. Business is booming, and the next wave of growth is already being marketed around AI, with companies positioning themselves as foils to chatbot companions and other risks. The broader market for parental control software, encompassing dozens of apps, was worth an estimated $1.57 billion in 2025 and is expected to nearly triple in value by 2034. Bark Technologies, the current leader among content-monitoring apps, says it scanned 11 billion messages to or from 7.5 million children in the US in 2025. Its free school program is in more than 3,700 districts, covering about one in 10 kids in the US. While Bark aims to flag only content that trips its filters, some competitors, like FlashGet Kids, include features like screen mirroring and camera access.
These apps have had genuine successes: They’ve prevented suicide attempts, intercepted predators, averted school shootings. But they can also cause harm themselves. To get a read on how digital surveillance affects young people, I scraped more than 600,000 reviews of the leading apps and talked to kids, parents, and people who were monitored as children and are now grown. Some kids were grateful for their parents’ protection. Others described false alarms that got them punished, secrets revealed before they were ready, breakdowns of trust, and anxiety they carried into adulthood. (To protect their privacy, we’re not using their full names.)
FRANZISKA BARCZYK
None of this is easily fixed. But child-safety researchers and advocates say better approaches exist. One is to make the platforms themselves safer, forcing social media companies to build guardrails instead of leaving families to find their own. This “duty of care” approach is now advancing in the UK, Australia, and a number of US states. Another is to spend less effort watching kids and more effort helping them recognize risk, cope with it, and turn to a trusted adult when something goes wrong. This approach is called resilience.
Wisniewski and others are developing tools that don’t aggressively monitor kids but train them in resilience and build trust with parents. “If we define safety as the absence of risk, the way you keep them safe is by keeping them off the platforms entirely,” she says. “But if we define safety as the ability to protect oneself and engage without engaging a risk, the solutions look very different.” It is the difference between abstinence lectures and sex ed.
Content-monitoring apps typically involve two pieces of software—one on the child’s device, one on the parent’s. A parent installs the kid-side app, connects the kid’s accounts, and chooses what to surveil: messages, social apps, browsing, location, screen time. The kid-side software forwards activity to the company’s algorithmic classifiers, which scan for content related to sex, drugs, bullying, and self-harm, among other categories, and push alerts to a parent-facing dashboard. The same type of software can be loaded into school-issued accounts and devices.
Sometimes the safety net works. Titania Jordan, Bark’s chief marketing officer, told me the FBI has thanked the company on multiple occasions for bringing credible school-shooting threats to its attention. “I hate that we have to exist,” she says. “But I’m so thankful that we do.” According to Bark’s annual report, its classifiers alerted hundreds of thousands of families to severe self-harm risks in 2025. Gaggle, a school-monitoring competitor, makes a similar claim in its own annual report, crediting its software with saving over 1,000 lives in the 2024–’25 school year.
Nearly one in five kids noted feeling watched or stripped of privacy. About one in 10 said monitoring broke their trust in their parents; one in 12 described anxiety or distress.
Real-world efficacy across the market is harder to measure, though there are hundreds of anecdotes in the reviews—predominantly about how location-sharing apps like Life360 have helped parents find lost kids. Dozens of reviewers also praise content-monitoring apps for averting major crises like potential suicide or self-harm. Of the reviews I scraped (reviews, it’s worth noting, represent a self-selected sample that’s skewed toward strong feelings), over 200,000 had some indication of whether they were written by a parent or a kid. Parents liked whatever worked. On average, screen-time limiters and location trackers drew four- or five-star reviews 74% to 78% of the time. OS-level controls and content-scanning apps fared worse (44% and 48% positive reviews, respectively), with parents’ top grievance being that the apps were unreliable: They disconnected, missed genuinely troubling content, and raised alarms over nothing.
The kids being watched had different concerns. Of the reviews I could attribute to either a parent or a kid, about 14,000 came from kids being monitored now or from adults looking back at being monitored in their youth. Roughly one in seven were, on balance, grateful. (Gavin, a teen I messaged with, said that monitoring “taught me self-accountability and integrity.”) But nearly one in five noted feeling watched or stripped of privacy. About one in 10 said it broke their trust in their parents; one in 12 described anxiety or distress. That’s not surprising, given that child-development experts have long said testing boundaries and building autonomy are what adolescence is for. A 2019 meta-analysis in the European Journal of Developmental Psychology, which pooled 31 long-term studies on how parent-child communication changes, found that children naturally disclose less to their parents as they age—the work of building an independent self.
Making matters harder for kids, a lot gets caught in the dragnet unnecessarily. S., an 11-year-old on the autism spectrum, recalls messaging her best friend at school about whether a service dog might help with her meltdowns or keep her from harming herself. The school used Bark, which alerted the principal, S.’s parents, and her friend’s parents, citing content involving “self-harm.” Perhaps frightened by the alert, the friend’s parents cut off contact between the girls. “My first thought was, why?” S. told me. “It’s not like I’m hiding anything. But I’d rather people who weren’t in the situation not see it.” Over a year later, the girls still don’t speak.
While apps aren’t in charge of parents’ reactions, false flags do seem to be more the rule than the exception. “Ninety-nine percent of the alerts are garbage,” says Grant Callaghan, an Australian dad who tried several monitoring apps with his 13-year-old son before deciding to build his own. “It flagged two kids calling a third one annoying as bullying. It flagged a kid complaining of a headache as ‘medically concerning content.’” When I ran Bark on a test phone set up as a 10-year-old’s, it manufactured a grooming scenario out of a random password I saved in the Notes app.
Bark’s Jordan doesn’t dispute that false alarms happen, volunteering an example of a soccer player whose photo of a net-scraped wrist was marked as potential self-harm. She says that over-flagging is safer than under-flagging, though: “You’d rather know that than not know that.” In a 2021 study with Bark, CDC researchers found that students who tripped multiple risk flags were far likelier to trigger an alert for severe self-harm later—indicating that flags aren’t necessarily noise.
Another problem is that Bark and similar monitoring apps surface things kids may not be ready to share. In a national survey by the Center for Democracy and Technology, nearly a third of LGBTQ+ students said they or someone they knew had been outed by school monitoring software. Jordan said Bark does not flag sexual orientation, but discussions about it could be captured when it screens chats for sexual content. Again, what adults do with such alerts, she added, is beyond any app’s control. “If a parent responds to a child’s identity with rejection,” she told me, “that is a parenting failure, not a child safety feature working as intended.”
Damage caused by monitoring tools can follow kids into adulthood. One in eight reviews left by adults looking back on their years of being monitored describes lasting harm. Only one of the people I contacted agreed to go on the record at all, the others citing privacy worries and anxiety that continues to haunt them.
“There’s so many ways for teens to get around [controls]. And at the end of the day, the biggest thing that most of these apps are telling kids is that we don’t trust you.”Pam Wisniewski
That woman, M., is now 19. She got her first smartphone at 13 and says her mother installed a monitoring app before handing it over. Though she’d been abused from a young age, the first serious beating, M. tells me, came shortly after she texted a friend about her depression. “She didn’t want me to be able to talk to people about the things that she was doing to me,” M. speculates. She eventually got a secret second phone and ultimately ended contact with her mother. Today, M. won’t let even her fiancé touch her phone. “I have been watched my whole life,” she says. “I deserve this sense of privacy.”
M.’s case represents the extreme end of a spectrum: an app enabling abusive behavior. But it could reflect a broader issue, particularly with tools downloaded outside official app stores—aka, ones that are sideloaded. A 2025 audit led by researchers at St. Pölten University of Applied Sciences and University College London found that nearly half the sideloaded monitoring apps they looked at are functionally indistinguishable from stalkerware.
Above all else, it’s not clear how well these apps fulfill their core promise of keeping kids safer. No commercial monitoring app has produced a controlled trial that I was able to find showing that it reduces harm.
Kids swap tips online on how to bypass the surveillance. About 7% of the app reviews left by children describe a concrete workaround. “If you red-team them at all,” Wisniewski says, “there’s so many ways for teens to get around them. And at the end of the day, the biggest thing that most of these apps are telling kids is that we don’t trust you.” In a 2018 survey, her team found that use of parental controls was associated with an increase in the online risk kids encountered, including exposure to harassment. Causality, though, is hard to pin down, because monitoring may follow trouble rather than cause it.
Risks may also migrate out of view, as potential predators actively seek out unmonitored spaces. According to Bark’s 2025 annual report, alerts about predators and grooming behavior have fallen in recent years as conversations move into blind spots. Thorn, a child-safety nonprofit, has found that offenders routinely direct targets onto encrypted apps like WhatsApp and Telegram. After Meta encrypted Messenger by default, the number of reports reaching the National Center for Missing & Exploited Children dropped nearly 20% in a year—a trend the center attributed to lost visibility. Parents can block access to such spaces, but tech-fluent kids can slip around that through browser versions, borrowed phones, or secret accounts.
Despite their flaws, content-monitoring apps thrive because parents feel overwhelmed. There is no federal law in the US requiring platforms to design their products to be safe for users, and any safeguards that do exist have proved insufficient time and time again. In 2024 the Molly Rose Foundation (MRF), a suicide prevention nonprofit, analyzed 12 million moderation decisions related to suicide and self-harm that had been logged by six major social platforms. More than 95% came from just two of them: Pinterest and TikTok. Instagram and Facebook each accounted for 1% and X.com for even less—evidence, the researchers argue, not that they host less worrisome content but that they’re less inclined to moderate it.
Meta’s Instagram Teen Accounts fared little better in MRF audits. Of the safety tools tested in a 2025 review coauthored by Arturo Béjar, a former Meta engineering director who designed many of the company’s antibullying tools in the mid-2010s, only 17% worked as advertised. Josh Golin, who runs the children’s advocacy group Fairplay, offers the simplest explanation for such failures: “Almost any change that’s going to make kids safer is going to mean less money.” Documents unsealed in lawsuits against Meta allege that the company weighed safety fixes against the engagement they would cost—and chose growth.
Some groups are working to create more accountability. ParentsSOS is a coalition of families who have lost children to online harms. The group has been a driving force behind several state laws. In 2017, David’s Law made cyberbullying a crime in Texas, and Mississippi criminalized sextortion after 16-year-old Walker Montgomery died by suicide within hours of being targeted by a scammer posing as a teenage girl.
The courts are moving too. Nearly 2,900 federal lawsuits by families and school districts are pending against social media platforms. And in March 2026, New Mexico became the first state to win its own child-safety case against Meta, netting a $375 million verdict. A newer wave of suits targets AI: The parents of 16-year-old Adam Raine are suing OpenAI, alleging that ChatGPT coached him toward suicide. And in January, Character.AI settled a wrongful-death suit brought by the mother of a 14-year-old who had formed an intense attachment to one of its chatbots before taking his life.
FRANZISKA BARCZYK
ParentsSOS members have also been instrumental in advancing the Kids Online Safety Act in Congress. In its original form, the law would have imposed a duty of care on social media companies, meaning they’d have to design their features to mitigate harms to minors rather than to maximize engagement. They would also have to give kids tools to limit who can contact them. It overwhelmingly passed the Senate in 2024, but its chances of becoming law remain in flux. The House’s new rewrite—blessed by the tech industry—strips the duty-of-care provision.
But even as they push to fill the regulatory void, parents who have faced tragedy don’t see monitoring tools as an answer. “You don’t pay extra for seat belts in the car, or airbags,” says ParentsSOS cofounder Maurine Molak, whose son David (of David’s Law) died by suicide in 2016 after months of cyberbullying.
Other critics of monitoring tools say the real issue is with how they’re built. The systems, says Wisniewski, need to be reworked so they keep kids safe without violating the privacy and autonomy that adolescents require. The goal isn’t no monitoring or social apps, but better ones—and a principled way of preparing young people to handle risk while also promoting trust between them and their parents.
Wisniewski’s resilience-based approach focuses on developing three pillars: self-regulation to help kids decide what healthy use looks like, risk coping to teach them what to do when something bad happens, and a support system of trusted adults and peers to turn to. She didn’t invent the idea of resilience; the finding that children grow stronger through manageable exposure to adversity has anchored developmental psychology since the 1970s. Sonia Livingstone, who has studied children’s online lives at the London School of Economics for over two decades, has amassed evidence from 33 countries that online risk and opportunity are inseparable, and that children shielded from every risk never get the practice to handle any.
A growing body of research indicates that inculcating resilience works in practice. In 2024, development economists at Hiroshima and Hitotsubashi Universities published a randomized trial of a digital-safety curriculum that was taught to junior high students in Vietnam. Across four modules, students learned how scammers and groomers operate, how to protect themselves, and how to find help. When tested a month later, only about 15% of those who’d taken the training engaged in risky online activity, compared with 50% to 60% of the control group. Similarly, Wisniewski’s own research suggests that teens high in resilience, exposed to the same risks as everyone else, show less negative impact from that exposure.
The systems need to be reworked so they keep kids safe without violating the privacy and autonomy adolescents require. The goal isn’t no apps, but better ones.
Her contribution is pushing resilience into technology itself. Her research group, the Socio-Technical Interaction Research Lab, has developed alternatives to the current crop of apps, though it’s never commercialized them. An app called Circle of Trust monitors teens’ messages but shows them the same dashboard a parent sees, and it lets each pair work out a set of trusted contacts whose chats the teen can keep private. When 17 parent-kid pairs tested it in 2020, most rated it as more useful and less corrosive of trust than a stricter app.
Her focus now is less a product than a method. Through a program called Teenovate, she trains teenagers as research apprentices and co-creators of their own safety tools. Since 2019, more than 85 teens have worked on prototypes and safety features dealing with cyberbullying, sexting, and privacy. They developed the prototype app MOSafely, which reports potential harms to kids directly, so they can review alerts and decide when to involve their parents. The end goal of Teenovate is to fashion evidence-based design patterns that tech platforms can leverage—though none have adopted them yet.
To be fair, some existing apps do factor in kids’ privacy and agency. Aura, a Bark competitor, shows parents a weekly well-being score built from sleep, usage, and engagement patterns. The app holds back most of the content but sends parents the self-harm alerts along with guidance on how to navigate a conversation. Jordan told me that Bark continues to consider whether it should show alerts to kids alongside the flags it sends parents, but at this point, she says, the best use of the app involves open dialogue within the family from installation through alert.
Some parents are even building their own solutions. Frustrated by available offerings, Callaghan, the Australian dad, created a web-based tool called Joey Family. The idea is not to catch his son doing something wrong but to answer a simple question: “Is he happy?” He wanted to know if his kid was lonely, whether he sent more messages than he got back. The data opened conversations. “The people I’m trying to help are people like me who want to have that relationship with their kids,” Callaghan told me. “They want to be guides and learners.”
If anything is fundamental, it’s the relationship: Online safety efforts work best when anchored by a young person’s trust in someone else. For kids whose homes aren’t safe—like M., or Wisniewski when she was young—that someone can even be an online community. “There are so many ways where we can find real community online,” says Caitlyn Vergara, a child-safety researcher at Harvard who studies youth mental health. “Especially for youth of color coming from majority-white spaces, it is a legitimate way to find community when community is not tangible around you.” Walling off kids in the name of safety can isolate them, she says; the point is to make sure every kid has somewhere safe to belong.
The core tension is that families are being asked to solve at the kitchen table a problem engineered in boardrooms. Monitoring is one answer to that assignment, but while it might mitigate risks from outside the home, it can also damage the trust inside it. The best parents can do is refuse to become one more thing their kids have to hide from—and stop believing they should have to fix a dangerous situation alone.
Kelly Clancy is a neuroscientist and writer based in New Hampshire. She’s also the author of Playing with Reality: How Games Have Shaped Our World.
Donald Trump holds a press conference with Elon Musk in the Oval Office on May 30, 2025. © Kevin Dietsch
