AI moderation

Moderate content and detect profanity

Run content moderation with FastPix from the dashboard or the content moderation API, then read the categories and confidence scores in the response to act on the results.

Run content moderation on a media asset from the FastPix dashboard or programmatically through the content moderation API, then read the returned categories and confidence scores to decide what happens to the content. For what moderation detects and how the scores work, see Content moderation.

Note: You can generate AI features only for existing media. Upload and process your video first, then run moderation on the ready media.


Prerequisites

  • A FastPix account with an active workspace (Activate your account)
  • A ready video asset in your workspace
  • Your Access Token ID and Secret Key if you use the API

Run content moderation from the FastPix dashboard

To run moderation without calling the API:

  1. Open the media item: From the FastPix dashboard, go to Media > [your media item] and open the Media Details page.
FastPix Media Details page with the In-Video AI options in the left navigation
  1. Open the moderation tab: On the Media Details page, click the Moderation tab to view the moderation controls and history.
Moderation tab on the FastPix Media Details page

  1. Run the check: Click Generate to start moderation on the media.

  2. View the results: When moderation completes, the detected categories and their confidence scores appear in the tab.

Moderation results in the FastPix dashboard showing detected categories and confidence scores
  1. Take action: To block playback automatically, update the media accessPolicy or playback rules when a high-confidence detection appears. For manual review, forward the flagged categories to your review queue or ticketing system.

Run content moderation with the API

Run moderation and fetch the results programmatically with the In-Video AI API. The video moderation API is asynchronous, so you trigger it, wait for the webhook or poll, then read the result. Both endpoints use Basic authentication with your Access Token ID and Secret Key.

Step 1: Trigger moderation

Send a POST to the Detect NSFW content and profanity endpoint with the mediaId of a ready asset.

curl --request POST \
  --url https://api.fastpix.com/v1/ai/{mediaId}/moderation \
  --user '{Access_Token}:{Secret_Key}'

The response confirms that the job started:

{
  "success": true,
  "data": {
    "id": "c695988b-ff84-42ae-bb21-10f284fedb0e"
  }
}

Step 2: Wait for the result

Moderation runs in the background. Subscribe to the video.media.ai.moderation.ready webhook to be notified when the analysis completes, or poll the get-results endpoint.

Step 3: Get the moderation result

Send a GET to the Get video moderation results endpoint.

curl --request GET \
  --url https://api.fastpix.com/v1/ai/{mediaId}/moderation \
  --user '{Access_Token}:{Secret_Key}'

Understand the moderation response

The response contains a moderation object that summarizes the analysis and lists every flagged segment with its time range. A sample response for an unsafe video:

{
  "moderation": {
    "verdict": "unsafe",
    "modalities": ["video", "audio"],
    "scores": {
      "threats": 0.5,
      "violence": 0.8661,
      "profanity": 0.5
    },
    "segments": [
      { "modality": "audio", "start": 0.1, "end": 12.74, "riskScore": 0.3, "categories": { "violence": 0.3, "threats": 0.3 } },
      { "modality": "video", "start": 54, "end": 81, "riskScore": 0.8661, "categories": { "violence": 0.8661 } },
      { "modality": "video", "start": 81, "end": 108, "riskScore": 0.5197, "categories": { "violence": 0.5197 } },
      { "modality": "audio", "start": 88.19, "end": 98.53, "riskScore": 0.5, "categories": { "profanity": 0.5 } },
      { "modality": "audio", "start": 126.82, "end": 140.19, "riskScore": 0.3, "categories": { "profanity": 0.3 } },
      { "modality": "audio", "start": 140.19, "end": 153.6, "riskScore": 0.5, "categories": { "violence": 0.5, "threats": 0.5 } },
      { "modality": "audio", "start": 153.6, "end": 169.12, "riskScore": 0.5, "categories": { "profanity": 0.5 } },
      { "modality": "video", "start": 270, "end": 297, "riskScore": 0.7656, "categories": { "violence": 0.7656 } },
      { "modality": "audio", "start": 308.64, "end": 318.88, "riskScore": 0.5, "categories": { "profanity": 0.5, "threats": 0.5 } },
      { "modality": "audio", "start": 318.88, "end": 335.14, "riskScore": 0.5, "categories": { "threats": 0.5 } },
      { "modality": "audio", "start": 404.23, "end": 422.21, "riskScore": 0.4, "categories": { "violence": 0.4, "threats": 0.4 } },
      { "modality": "audio", "start": 476.87, "end": 497.31, "riskScore": 0.5, "categories": { "profanity": 0.5 } }
    ]
  }
}
FieldDescription
moderation.verdictOverall assessment of the media, such as unsafe, based on the flagged segments.
moderation.modalitiesThe tracks that produced findings: video, audio, or both.
moderation.scoresThe strongest confidence score per category across the whole media, from 0 to 1. Use these for a whole-media decision.
moderation.segmentsThe individual flagged moments, in time order. An empty array means nothing was flagged.
moderation.segments[].modalityWhether the moment was flagged in the video (visual) or audio (spoken) track.
moderation.segments[].start, moderation.segments[].endStart and end of the flagged moment, in seconds.
moderation.segments[].riskScoreConfidence for the segment overall, from 0 to 1.
moderation.segments[].categoriesThe categories flagged in the segment, each with its own confidence score from 0 to 1. See Moderation categories.

All scores are numbers from 0 to 1, where a higher value means stronger confidence that the category is present, not that the content is more severe.

Read the response at two levels:

  • Top level, for a quick decision. verdict and scores summarize the whole media. In this example, violence reaches 0.8661, so the media is marked unsafe. The top-level scores reflect the strongest finding for each category across all segments.
  • Segment level, to act precisely. Each entry in segments gives the exact start and end (in seconds) of a flagged moment, its modality, and the categories that triggered it. Use the timestamps to review, redact, or clip just that moment instead of the whole file.

Compare scores and each segment’s riskScore against the thresholds you set, then apply your content policy. For more context, see the blog post AI content moderation using NSFW and profanity filters.


Act on the result

Map the scores to an action that matches your content policy:

  • Below your review threshold: allow the content to publish.
  • In your review band: set the media to private and send it to a human review queue.
  • Above your automatic-action threshold: restrict playback by updating the media accessPolicy, and alert your trust and safety team.

Because each segment includes its start and end time, you can target a fix at the exact moment instead of the whole video, for example beep out profanity by replacing the audio track or cut a flagged moment by removing unwanted video segments.

To wire these decisions into an end-to-end flow, see Content moderation workflows.


Limits and considerations

  • Moderation runs on a ready video, so wait until the media has finished processing before you run it.
  • Sampling produces the strongest results on short-form video. Evaluate accuracy on your own content before relying on it for long videos.
  • Set a confidence threshold that matches your platform’s tolerance, and sample your own catalog before fixing the values.
  • Audio moderation, including profanity, runs on the transcript, so it is limited to the supported audio moderation languages.

Frequently asked questions

How do you moderate video content with an API?

Call the FastPix In-Video AI moderation endpoint with the mediaId of a ready asset. Because moderation is asynchronous, wait for the video.media.ai.moderation.ready webhook or poll the get-results endpoint, then read the returned categories and confidence scores.

What does the moderation response contain?

A moderation object with an overall verdict, the modalities analyzed, the top scores per category, and a segments array. Each segment gives its modality, the start and end time in seconds, a riskScore, and the categories flagged. All scores are numbers from 0 to 1, and an empty segments array means nothing was flagged. See the API reference for the full schema.

How do I choose a confidence threshold?

Start with a lower band for review and a higher band for automatic action, and use separate thresholds for video and audio. Sample your own catalog before fixing the values, since thresholds that work for short-form UGC may be too loose for kids or education platforms.

How do I block a video automatically when moderation flags it?

On the video.media.ai.moderation.ready webhook, if a category exceeds your threshold, update the media accessPolicy to restrict playback. Route borderline scores to a human review queue instead of blocking outright.


What’s next

Once moderation returns a finding, act on the flagged content: