AI captions
AI captions are on-screen text versions of a video's dialogue, generated automatically by speech recognition instead of typed by a person. The model transcribes the audio, a segmentation step splits the words into short timed lines, and the result is saved as a caption file such as WebVTT or SRT that the player loads beside the video. Closed AI captions can be switched on and off; open captions are burned into the picture. Either way, they make video usable for deaf and hard-of-hearing viewers and for anyone watching on mute.
How are AI captions generated?
The starting point is a speech-to-text transcript with a timestamp on every word. Turning that into captions is its own step. A segmentation pass groups words into cues, usually no more than two lines, breaking at natural phrase boundaries rather than mid-name or mid-number. Each cue gets a start and end time so it appears when the words are spoken and stays long enough to read.
Good caption systems also respect shot changes, so a line does not hang over a cut, and avoid flashing very short cues. Speaker changes may be marked with a dash or a name.
The cues are written to a caption file. WebVTT is the web standard and the format HLS uses for subtitle tracks; SRT is a simpler format many editors and social platforms accept. For multiple languages, the transcript is translated and re-segmented, because a translated sentence has a different length and rhythm from the original.
AI captions vs. human captions
AI captions are produced in minutes for any amount of video, which is why platforms use them to cover whole libraries. Human captioners are slower but catch what models miss: names spelled correctly, technical terms, who is speaking off screen, and sound cues such as music, laughter or a door slam that change the meaning of a scene.
Accessibility is where the difference shows. WCAG success criterion 1.2.2, Captions (Prerecorded), is a Level A requirement that captions are provided for all prerecorded audio in synchronized media. Captions full of wrong words technically exist but do not serve the viewer.
The common pattern is AI first, then review where it matters. A course platform might review captions on its flagship courses and leave AI captions on the long tail. A caption editor that lets someone fix a cue without re-timing the file keeps that review cheap.
How are AI captions delivered in a video player?
For streaming, captions usually travel as a separate text track. In HLS, the master playlist lists a subtitles group, and each language is its own playlist of WebVTT segments that line up with the video timeline. DASH does the same with text adaptation sets. The player reads the list, shows a captions menu and renders the selected track over the picture, applying its own font, size and background.
Broadcast and some TV devices still use CEA-608 and CEA-708, where caption data is embedded inside the video stream itself. For on-demand web and mobile playback, sidecar WebVTT is the norm.
Open captions are different: the text is rendered into the frames during encoding, so it cannot be turned off, restyled or translated without a new encode. Vertical feeds and social clips often use open captions, because clips get reposted to apps that drop separate tracks and many people watch on mute. Closed tracks are the choice for accessibility, multiple languages and viewer control.
How do you add AI captions to a whole video library?
Captioning has two jobs. The first is a one-off pass over the back catalogue, which is a batch problem: queue every existing video, run speech-to-text, store the tracks and attach them to each asset. The second is the steady flow of new uploads, where captioning should be a step in the ingest pipeline so no video goes live without a track.
Keeping captions next to the video matters more than it sounds. Caption files stored in a separate system drift: an editor trims the intro and every cue is now a few seconds late, or a migration leaves half the tracks behind. When captions are generated from the same asset that is streamed, timing stays aligned.
Plan languages early. Each language is another track in the manifest, and translated tracks inherit any errors in the source transcript, so fix the original before translating.
How does FastPix handle AI captions?
FastPix generates AI captions on every on-demand asset through in-video AI, which runs in parallel with encoding. The captions come from the same timestamped JSON as the transcript, with word-level timings and speaker diarization, so a new upload has its caption track when the video is ready rather than in a later job. Speech can be in any of 50+ transcription languages, and caption output is multi-language in WebVTT and SRT, with editable cue points for correcting names and terms.
The open-source FastPix Player renders them through native SDKs for web, Android (Media3-based, including Android TV), iOS (AVPlayer-based, including tvOS and visionOS) and Flutter, with a captions styling API and CSS variables for theming every control and caption without forking. The player meets WCAG 2.1 AA out of the box, with keyboard navigation and screen-reader labels. The WebVTT and SRT files also work in other players. On WordPress, including LearnDash, TutorLMS and LifterLMS sites, the ready-made FastPix WordPress plugin adds AI captions and chapters to uploaded videos.
For short-form output, the AI clipping agent renders clips with captions and a title, and can reframe them for vertical feeds. On live streams, real-time captions run on the stream itself.
Captioning is priced per minute beside encoding, with no tokens to count. Standard encoding is free up to 4K. Caption a sample of your library with the free plan's subtitle minutes, no card required, and review the tracks in the free player.
By the numbers
- WCAG success criterion 1.2.2, Captions (Prerecorded), is a Level A requirement: captions are provided for all prerecorded audio content in synchronized media. (Understanding SC 1.2.2: Captions (Prerecorded) (W3C))
