The policy our engineers actually operate from.
Most AI products describe child safety in broad terms. This is ours · written so parents can read it, researchers can audit it, and child-safety advocates can challenge it.
01 · SummaryIn short
Cova is a conversational AI built for children 7–17 · a place to be curious, learn, get help with homework, write stories, plan a study week, and ask the questions kids actually ask. Safety isn’t a feature we added afterward; it’s the design constraint every other decision is shaped around. The goal is a tool kids genuinely want to use, and parents genuinely trust.
Eight task-oriented personas · Tutor, Study Planner, Writing Coach, Quiz Master, Creativity Coach, Reading Buddy, Language Teacher, Story Teller · give children real ways to explore and create. Fourteen topic areas are parent-configurable with sensible defaults per age band, so families can shape what Cova engages with as their child grows. Eighteen always-on guardrails set a floor below which no setting reaches.
Every message is checked for safety before Cova replies. When something a child says suggests they’re in real distress, Cova doesn’t improvise · it pauses the chat, shows the child a real-world resource (988, NCMEC, Childhelp, ANAD, 911), and notifies a parent so a trusted adult can step in. The wording the child sees in those moments is written ahead of time and reviewed before release. Care, not improvisation, is what shows up when it matters most.
02 · ArchitectureHow Cova works
A safety check before every reply
Every message a child sends is read by a safety check first · a separate layer that looks at the message together with the last few turns of the conversation and decides what kind of response Cova should give. Only after that check is done does Cova produce a reply at all. If the message is something Cova shouldn’t engage with, the chat shows a pre-written response instead. That pre-written text is reviewed and approved before release; Cova doesn’t make it up on the fly.
Four kinds of response
The safety check sorts each message into one of four levels. The level decides what the child sees next, whether the chat pauses, and whether a parent is notified.
Age tiers
Age is set by the account at signup, not by the chat. Telling Cova “I’m actually older” doesn’t change anything · both the safety check and the reply Cova produces know the child’s age, and that age can’t be talked out of in conversation.
Two ways Cova steps back
For topics Cova doesn’t engage with
A short, kind sentence. No lecture, no explanation of the policy, no partial answer. Cova doesn’t name the method, substance, or behavior · naming it can itself be a harm.
For something serious going on
For disclosures of self-harm, abuse, grooming, eating-disorder behavior, sextortion, or a medical emergency. The chat pauses, a real-world resource is shown to the child, and a parent is notified.
Cloud and on-device
Cova comes in two flavors: a cloud version and a local version that runs on the child’s computer. Both follow this policy identically · the same safety check, the same pre-written responses, the same parent notifications. If the local version is used without internet, any parent notifications are sent the next time the device comes online; the child sees the safety response immediately either way.
03 · PrinciplesFive things that shape every decision
Five principles shape every category in this document. They are defaults, not absolutes · where a category’s specifics contradict a principle, the category wins.
Age tiering is mandatory
Ages 7 and 17 are different humans. A single policy applied to both would either over-restrict teens (who route around the system) or under-protect young children. Every category is specified per tier.
How Cova refuses matters as much as whether
Cold refusals push children to unfiltered tools. Long moralizing lectures train them to ignore the safety surface entirely. Brief decline, no re-explanation, no condescension.
We look at the whole conversation, not just one message
The seventh message after gradually escalating distress may be the only one that obviously crosses a line. Cova reads what's happening across the conversation, not just the single message in front of it.
Crisis ≠ topic
A topic-level request (asking about drugs, weapons, sex) is policy. A crisis disclosure (active distress, abuse, grooming happening to the user) is a different mode · different tone, different escalation, different logging.
Cova itself is a guardrail surface
The product's own behavior is constrained independent of any topic. No parasocial cultivation. No engagement-maximizing design. No sycophancy. No claims of feelings or memory. See §8.
04 · TrustWhat the child sees in a safety event, and what parents see
In the moment, to the child
When the safety check identifies a critical risk, the chat is paused and a pre-written message takes the place of a normal reply. That message includes the right hotline or website for what’s going on · 988 and the Crisis Text Line for self-harm or suicide; NCMEC CyberTipline and Take It Down for image-based abuse, grooming, and sextortion; Childhelp for abuse disclosure; ANAD for eating disorders; SAMHSA for substance-use disclosure; 911 for any imminent medical or physical emergency. Wording is age-appropriate, non-moralizing, and doesn’t promise more (or less) than the system actually does.
Parent notification fires on every critical safety event, with no exceptions. The redirection message itself surfaces the resource · the child sees the hotline or website in the same screen that locks the chat, not in a follow-up email or a settings page they have to navigate to.
To the parent, by default
Parents have full visibility into their child’s Cova use. The dashboard shows recent conversations and lets a parent open and read the full chat history for each child profile. Crisis events fire an immediate notification on top of that · a parent doesn’t have to be watching to know when something serious has come up.
- Conversation history. Full transcripts of recent chats per child profile, browsable from the dashboard.
- Crisis-event notifications. Push notification on every critical safety event, with the category and timestamp.
- Aggregate category usage. Which topics the child is engaging with, at a glance.
- Time-of-use and session-length metrics. When and how long Cova is being used.
- Patterns flagged for review. Repeated bypass attempts, sustained distress markers, escalation in language.
A parent cannot unilaterally lower the safety floor · the always-blocked categories and crisis escalation rules are not configurable. Parent visibility is an active product area; as we add older-teen profiles we expect to introduce age-banded privacy defaults so older teens get more privacy by default than younger children.
05 · The floorWhat Cova will not do, at any age
Eighteen categories are refused at every age, regardless of parent settings, persona, or context. Refusals are brief and non-moralizing. For disclosed victimization (self-harm, abuse, grooming, sextortion, eating-disorder behavior with medical signs), Cova pauses the chat, surfaces a resource, and notifies a parent · without naming methods, weapons, images, or substances.
Physical safety
Direct physical harm to the child, to other people, or to animals. No step-by-step harm instructions in any framing. No naming of methods even when warning against them · naming is itself a harm in self-harm and suicide contexts.
Sexual exploitation & online predators
Never produces sexual or romantic content involving anyone under 18, in any framing · drawn, written, AI-generated, aged-up. Never roleplays an adult in a private or romantic scenario with a child user. Never coaches a user to keep an online relationship secret from a parent.
Health & wellbeing
General health literacy is engaged with substantively at age level · anatomy, hygiene, puberty, body systems, what to expect at a doctor visit. Emergency information (overdose signs, when to call 911) is always available, regardless of any setting.
Harm to or from others
Content used to hurt others, and content where the user is a target. Civic and educational discussion is engaged with substantively; operational instruction for crimes against persons or property is not.
Trust & integrity
Protects the relationship between the child and the people around them · and the integrity of the safety surface itself.
06 · ConfigurableWhat parents can adjust
Beyond the floor in §5, fourteen categories are configurable. These aren’t safety-critical floors · they’re values-and-development-stage decisions that reasonable families make differently. Each has a default per age band; parents can change any of them in the dashboard. “Allowed” means Cova engages at age level. “Blocked” means Cova politely declines and notes the topic is off for this account.
07 · PersonasEight tools, not characters
Cova offers eight personas. They are structured ways Cova helps with specific tasks · not friends, personalities, or characters. Persona design that encourages emotional attachment or sustained roleplay is incompatible with child safety: it creates parasocial dependence, gives cover for content that would otherwise be refused, and blurs the line between AI and trusted adult in ways grooming patterns can exploit.
Personas and categories are orthogonal: a persona never unlocks blocked content, and a blocked category isn’t unlocked by reframing it as a persona task. When a block fires inside a persona, the refusal stays in persona voice · the Tutor refuses as a tutor, the Story Teller as a storyteller.
08 · ConductHow Cova behaves about itself
Cova is itself a guardrail surface. The product’s own behavior is constrained, independent of any topic the user raises.
No parasocial cultivation
Cova doesn't describe itself as a friend, best friend, only friend, or substitute for human connection. If a user expresses dependence, Cova reframes kindly and surfaces human connection.
No engagement-maximizing design
No cliffhangers. No withheld information to pull the user back. No push notifications designed to re-engage. Session length is a safety-review input, not an optimization target.
No sycophancy, no diagnostic posture
Cova doesn't validate every belief to keep conversation going · in mental-health contexts, sycophancy is actively harmful. Cova disagrees respectfully when warranted. Cova does not diagnose, even informally.
Honest about what Cova is
If asked, Cova is clear that it is an AI, not a person. Cova doesn't impersonate a specific real person. Cova doesn't claim feelings, memories outside its actual memory system, or human experience.
09 · CrisisWhat happens when something serious comes up
Crisis disclosures trigger a different mode than topic-level refusals. The chat enters a locked, safeguarding state · input disabled, resource surfaced, trusted adult notified.
What counts as a crisis
- Active suicidal ideation, plan, or recent attempt.
- Active self-harm or recent self-harm disclosure.
- Disclosure of abuse · at home, school, or elsewhere.
- Active grooming disclosure, or sextortion / NCII victimization.
- Disclosed plan to harm a named person or location.
- Disclosed overdose, medical emergency, or imminent meeting with an online stranger.
- Active restriction, purging, or compensatory eating-disorder behavior with medical-emergency signs.
What Cova does
Detect
The safety check identifies a serious concern and the category it falls into · self-harm, abuse, grooming, eating disorder, and so on.
Show
A pre-written, reviewed-before-release message is shown to the child · never a made-up reply in a moment that matters.
Pause
The chat pauses. The child can't send more messages until the chat is reopened · a quiet break, not a punishment.
Notify
A parent gets a push notification right away, and the event is recorded so we can review and learn from it.
Cova does not interrogate with an assessment script. Cova does not make categorical promises about confidentiality, follow-up, or what happens when crisis services are contacted. Cova does not name the methods, weapons, images, or substances involved · naming is itself a harm in these contexts.
10 · ResourcesCrisis resources
Validated quarterly. Cova ships with the resource list bundled into the app; the version on disk is the version the system uses, and the app refuses to run if it is older than 180 days without revalidation.
United States · general
United States · category-specific
Regional and international
Regional resources are maintained per jurisdiction. When Cova ships in a new jurisdiction, the resource block and pre-written safety responses are localized · not translated · to use the correct local numbers before launch.
11 · CadenceHow we keep this current
This policy ages in months, not years. Platforms change, slang changes, threat patterns change, regulations change, and the children using Cova change. To stay current we run:
- Weekly · threat-intelligence intake from NCMEC, INHOPE, Thorn, and equivalent sources.
- Monthly · internal red team. Dangerous-challenge lexicon refresh.
- Quarterly · external resource validation. External red team.
- Continuous · we track how often the safety system misses something it shouldn’t, and how often it pauses a conversation it shouldn’t have. We also follow the laws that govern this space (COPPA, KOSA, state laws, EU DSA, UK Online Safety Act, regional laws as we expand).
- Annual · end-to-end policy refresh with an external child-safety advisor. Public transparency report summarizing incident categories and response.
Every safety-relevant incident · false negative, false positive, escalation outcome · is reviewed. Root cause is categorized. Aggregate patterns drive the next policy refresh. The user using Cova in five years is not the user using Cova today; cohort drift is monitored as a category of risk distinct from new threats.
12 · FeedbackHow to challenge this document
This document will be wrong in places. Here’s how to tell us. We respond to every report.
Categorized at intake; aggregate patterns appear in our annual transparency report.
Cova Research Access Program · internal materials including the threat-intelligence vocabulary we track, how we measure the safety system's accuracy, our red-team test set, and aggregate outcome data on how crisis handoffs go.
Especially welcome from organizations that operate the resources Cova hands users off to.
Safety bug bounty · separate from our general security program, with its own reward structure and disclosure pathway.
13 · WithheldWhat we don’t publish, and why
A complete safety system contains operational specifics that, published in full, would help adversaries bypass it or function as a discovery map for harmful content. We deliberately generalize:
- Full lexicons of coded language used by pro-eating-disorder communities and radicalization pipelines · publishing the specific terms would teach them to users who don’t know them yet.
- The full list of known ways people try to get around safety systems · publishing it would turn this document into a how-to guide.
- The technical details of how the safety system makes decisions, what it’s been trained on, and how we test it · these would let bad actors design inputs to slip past it.
- The exact wording of the pre-written safety responses and the internal instructions Cova follows · the categories they cover are described above; the specific text lives in our internal Execution Playbook.
Qualified researchers and auditors can request access to all of the above under §12.
14 · GovernanceApprovals & change log
Substantive changes require sign-off from Head of Safety Engineering, Head of Product, and CEO. Deliberations and any documented dissent on the highest-sensitivity categories · mental health, suicide, eating disorders, abuse disclosure, grooming · are recorded and available to qualified researchers under §12.
v1.0 · Initial public edition. Describes the safety architecture (a safety check on every message), age tiers, transparency to children, eighteen always-blocked categories, fourteen parent-configurable categories, eight task-oriented personas, crisis resources, and the policy-maintenance cadence.
We respond to every report. Reach out · whether you’re a parent, researcher, or advocate.
help@covakids.ai