Real-Time Chat & Messaging Moderation

The moderation problem

Chat moderation is a latency problem as much as an accuracy one. Users expect messages to send instantly, so you can't wait on a slow LLM round-trip for every line — but you still need to catch harassment, threats, and grooming in real time.

Why a word list isn't enough

Routing every message to an LLM adds hundreds of milliseconds and a per-call cost that doesn't scale to chat volume. Routing none of them misses the abuse that only intent classification can catch.

How The Profanity API handles it

Use context `chat` with `fast` mode: the fast layers (blocklist, static, semantic) clear the overwhelming majority of benign messages in single-digit milliseconds, and only genuine layer disagreement escalates to the LLM — so latency stays low without sacrificing the hard cases.

const res = await fetch("https://api.theprofanityapi.com/v1/check", {
  method: "POST",
  headers: {
    "Authorization": "Bearer " + process.env.PROFANITY_API_KEY,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ text, context: "chat", mode: "fast" }),
});

const { flagged, intent, score } = await res.json();