Content moderation

Content moderation helps you identify potentially unsafe material in video and audio before it is published, shared, or made available to viewers. FastPix analyzes the visual and spoken content in your media and returns the content moderation categories that may need attention, each with a confidence score and the relevant moments in the media.

Content moderation is useful for user-generated content, community platforms, education, internal communications, media libraries, and any workflow where content needs to be reviewed before publication. To run moderation, see Getting started with moderation. To build gating, review, and remediation flows, see Gate uploads before publishing, Review flagged media, and Fix flagged content and publish.

Why content moderation matters

User-generated video arrives faster than any team can watch it, and a single unsafe clip that reaches viewers can mean legal exposure, lost advertisers, and broken user trust. Reviewing every upload by hand doesn’t scale, and keyword or metadata filters miss what actually happens on screen and in the audio.

FastPix content moderation analyzes the video and audio directly and returns the unsafe categories it detects, each with a confidence score, so you can automate the decision to publish, block, or route for human review before the content reaches an audience. Because FastPix scores both visual and audio categories and lets you set your own thresholds, you decide how strict moderation is and act on the result through the dashboard or the API, with a webhook when results are ready.

What FastPix analyzes

FastPix content moderation screens two modalities:

• Visual moderation: sampled video frames are analyzed for potentially unsafe imagery.
• Audio moderation: spoken audio is transcribed and analyzed for potentially unsafe language.

Visual and audio are evaluated together when the media contains both, and the result reports which modalities produced findings.

The result is a set of moderation findings. A finding identifies a category, indicates how strongly the content matches that category, and points to the segment of the media, with a start and end time, where it was detected. Moderation is asynchronous, so your media can continue through the rest of your workflow while the analysis runs. When processing is complete, you can use the result to allow the content, send it for review, limit access, or take another action that matches your content policy.

Moderation categories

The categories depend on whether the finding comes from video or audio. The same content can match more than one category. For example, a spoken threat directed at a protected group can be returned as both hate and threats.

Video categories

Audio categories

Supported languages for audio moderation

Audio moderation, including profanity detection, runs on the transcript FastPix generates from the spoken audio, so it is available in the languages FastPix can transcribe. Visual (video) moderation does not depend on language.

FastPix supports audio moderation in the following languages. Accuracy varies by language, so start with English for the most reliable results and test other languages against your content before relying on them.

If a video’s spoken audio is in a language that is not listed, visual moderation still applies, but audio findings such as profanity may be limited or unavailable.

Understanding confidence scores

Each detected category includes a score between 0 and 1.

• A score closer to 1 means the moderation system is more confident that the category is present.
• A score closer to 0 means the match is weaker or less certain.

A score is a confidence measure, not a measure of how severe the content is. A score of 0.90 means the system is highly confident that the category is present. It does not mean the content is nine times more severe than content scored at 0.10.

Suggested score bands

These bands are starting points, not universal policy. Video and audio do not have to use the same thresholds.

Video findings. Video moderation evaluates sampled moments, so a brief or partially visible unsafe moment can receive a lower score than sustained or clearly visible content. Use a lower threshold for review while keeping automatic blocking more conservative:

Audio findings. Audio findings are based on transcript segments, so a higher band is a reasonable starting point:

The right threshold depends on your audience and risk tolerance. A children’s platform, a public social platform, and an internal training library can reasonably use different thresholds. Test the thresholds against a representative sample of your own content before applying an automatic action.

Understanding detected moments

Moderation returns time-stamped segments. Each segment identifies the modality (video or audio), a start and end time in seconds, an overall risk score, and the categories flagged in that moment. A segment that starts at 54 seconds and ends at 81 seconds applies to approximately that portion of the media, and multiple segments are returned when different categories appear at different points.

Time ranges represent the portion of the media covered by the analysis. Treat them as review guidance rather than a frame-perfect boundary, especially when the media is sampled at intervals or when a spoken finding covers multiple transcript segments.

What a moderation result looks like

A moderation result is a JSON object with an overall verdict, the modalities analyzed, the strongest score per category, and the time-stamped segments that triggered it:

{
  "moderation": {
    "verdict": "unsafe",
    "modalities": ["video", "audio"],
    "scores": { "violence": 0.8661, "profanity": 0.5 },
    "segments": [
      { "modality": "video", "start": 54, "end": 81, "riskScore": 0.8661, "categories": { "violence": 0.8661 } },
      { "modality": "audio", "start": 88.19, "end": 98.53, "riskScore": 0.5, "categories": { "profanity": 0.5 } }
    ]
  }
}

For a field-by-field breakdown, see Understand the moderation response.

How to use moderation results

Moderation results are most useful as part of a broader content policy:

• Allow low-risk content when all findings are below your review threshold.
• Queue uncertain content when scores fall in a range that needs human judgment.
• Restrict high-risk content when a category exceeds your automatic-action threshold.
• Edit or redact a flagged moment instead of removing the whole video, using the detected time range to target the change. See Fix flagged content and publish.
• Keep a human review path for appeals, borderline findings, and important publishing decisions.

Moderation can help prioritize review, but it should not replace human judgment. Context, intent, satire, news reporting, education, documentary footage, and fictional content can all affect how you choose to handle a finding. For end-to-end patterns, see Gate uploads before publishing.

Use cases

Video moderation fits any product where people upload or stream video that you are responsible for:

• UGC and social platforms: screen uploads and live recordings for violence, sexual content, and self-harm before they reach a feed.
• Online learning and kids’ content: hold submissions to a stricter threshold so unsafe material never reaches students or minors.
• Marketplaces and dating apps: check profile and listing videos to keep explicit or unsafe content off the platform.
• Advertising and brand safety: verify that creator or partner videos meet content standards before they run.
• Trust and safety teams: attach a confidence score to every asset so review queues focus on borderline cases instead of everything.

Detection accuracy

Evaluate whether your use case actually requires moderation on long videos. Most user-generated content platforms prioritize short-form video, where the current sampling model produces the strongest results. TikTok and Instagram Reels, for example, both operate well within the 3-minute range.

The average video on both platforms sits well below the 3-minute mark, which matches the shape of most UGC workloads that use moderation. Focusing moderation on short-form video gives you higher accuracy with lower processing cost.

Important considerations

• No automated moderation system detects every unsafe moment.
• Sampling and transcription can affect what is detected and how precisely it is located.
• A missing finding does not guarantee that the media is safe.
• A finding does not automatically mean that content violates your policy.
• Your thresholds should reflect your audience, geography, legal requirements, and publishing workflow.

For best results, combine automated moderation with clear policy definitions, a review queue for uncertain cases, and a way for users or reviewers to report missed content.

What’s next

• Getting started with moderation to run moderation from the dashboard or the API.
• Gate uploads before publishing to hold new uploads until moderation clears them.
• Review flagged media to send uncertain results to a reviewer.
• Fix flagged content and publish to repair only the flagged seconds and re-check.