Build a course content effectiveness dashboard that finds where lessons lose learners

If you build an online-learning or training library, you have heard it from your content team. Half the video lessons finish under fifty percent, learners quit somewhere in the middle, and no one can say which video is the problem or where inside it. Completion as a single number does not tell you which video to fix. FastPix Video Data shows you the drop-off point inside each video, the concepts learners rewatch, and how each lesson maps to the quiz that follows it, so the edit list is ready.

The finished content dashboard, built from the sample export below. Every number comes from running this article's code on that file.

Content effectiveness is not one metric. A video lesson can have decent completion and still fail the quiz, or hold attention for four minutes and lose half the class at the fifth. This dashboard scores each lesson on completion, where it loses people, whether learners rewatch it, how it maps to the quiz, and its length, then rolls those into one content-health score you can rank a library by, the way you would build any analytics view into your product. The two lessons it flags to fix are the ones to edit first.

Why this is hard in video, and what Video Data gives you

Measuring content in a video is harder than in a quiz. A quiz returns a score. A page of text you can scan. A ten-minute video returns almost nothing on its own: it plays, and whether a learner watched all of it, skipped the middle, or replayed one confusing minute is invisible unless something captures it. That black box is why most platforms can see that completion is low but not which video is the problem, or where inside it learners leave.

Building that visibility yourself is not a feature, it is an analytics stack: a beacon in every player, a collection endpoint that survives traffic spikes, sessionization to stitch events into views, storage, and the dashboards on top. That is months of engineering with nothing to do with your courses, which is why most teams ship a play count and stop. If you own completion, retention, or the renewal number, that blind spot is the thing standing between you and the lever that moves them.

FastPix Video Data is that stack as infrastructure. It is first-party data you own, captured from inside the player: how much of each video learners finished, the exact point they stopped, which seconds they replayed or skipped, the playback quality they got, and more than fifty fields per view, ten of them yours to define (the docs list every field). You add a line to your player and read one clean row per view. There is no pipeline to build or run, and the data lands in your own product rather than a third-party portal, so you can put it in front of your team, or your customers, under your brand.

Video Data supplies the exact engagement signal. The outcome, the quiz score, the certificate, the pass, lives in your platform, joined on the video and learner ID. Here is each signal, what it tells you, and the decision it informs.

None of these is an opinion about the content. They are what learners did with the video, and each points to a decision the person who owns the roadmap can act on.

Get view data flowing from your player

Playback happens in your player. FastPix Video Data captures every interaction from inside it: it dedupes the beacons, stitches events into sessions, adds geo and device data, and computes quality scores, across millions of concurrent views when you need it. What you export is one clean row per view, with the lesson, the course, and the full playback event stream on every row.

You do not need to switch players to get this. FastPix Video Data works with the players you already run, hls.js, Shaka Player and Video.js on the web, and native AVPlayer on iOS and Media3 on Android, through the monitoring SDKs, or you can use the FastPix Player and get it built in. Either way, installation is a snippet per surface, and data is flowing in about thirty minutes.

Step 1: Add the player, or add monitoring to the one you have

<!-- FastPix Player -->
<script src="https://unpkg.com/@fastpix/[email protected]/dist/player.js"></script>
<fastpix-player playback-id="YOUR_PLAYBACK_ID" stream-type="on-demand"></fastpix-player>

Already have a player? The monitoring SDK captures the same data from hls.js, video.js, native mobile, and TV players:

import fastpixMetrix from "@fastpix/data-core";
fastpixMetrix.tracker(videoElement, {
  hlsjs: hls, Hls: Hls,          // both are required
  data: { workspace_id: "YOUR_WORKSPACE_KEY" },
});

Step 2: Tag each view with course, lesson and difficulty

fastpixMetrix.tracker(videoElement, {
  hlsjs: hls, Hls: Hls,
  data: {
    workspace_id: "YOUR_WORKSPACE_KEY",
    viewer_id:  learner.id,
    custom_1:   course.name,        // which course
    custom_2:   lesson.title,       // which lesson
    custom_3:   lesson.difficulty,  // Intro / Core / Hard
  },
});

Step 3: Export the views

Export from the Video Data dashboard (Views → Export CSV) or the export API. The columns this article uses: custom_course, video_title, difficulty, video_duration, view_max_playhead_position (how far they got), view_total_content_playback_time (watch time), and events (the playback stream, for the drop-off and rewatch signals). We joined lesson_quiz_score from the LMS on lesson and learner. Video Data measures the watching; your platform owns the quiz, and a join brings them together.

If a lesson shows almost no views, check the tag first. A lesson mislabelled in custom_2 splits into two titles and looks like two half-watched lessons. Fix the tag before you read the numbers.

Score each lesson

The stack is chdb (embedded ClickHouse, pip install chdb) with pandas. The same SQL runs later on a real ClickHouse.

Step 4: Load the views and score completion per lesson

import chdb, pandas as pd, numpy as np, json
v = chdb.query("""
  SELECT custom_course AS course, video_title AS lesson, difficulty,
         CAST(video_duration AS Int64) AS dur_ms,
         CAST(view_max_playhead_position AS Int64) AS reach_ms,
         CAST(lesson_quiz_score AS Int64) AS quiz, events
  FROM file('learner-analytics-sample-synthetic.csv', CSVWithNames)""", "DataFrame")
v["comp"] = v["reach_ms"] / v["dur_ms"]

Step 5: Build the retention curve and find the drop-off point

Completion is an average. The retention curve is the shape behind it: the share of learners still watching at each point of a lesson. The biggest single drop is where the lesson loses people.

def retention(d):
    return np.array([(d >= x/100).mean()*100 for x in range(0,101,5)])
def cliff(d):
    c = retention(d); drops = c[:-1] - c[1:]
    i = int(np.argmax(drops)); return i*5, round(float(drops.max()))   # position %, size

On the sample, the worst drop in a well-watched lesson is Endocrine and feedback loops, which loses 27 percent of its viewers around the 40 percent mark. That is the stretch to re-cut, and you would not have seen it in the completion average alone.

Start using FastPix for free. $25 credit, no credit card required. Score your own lessons this week. Create your workspace →

Step 6: Find the concepts learners rewatch, and the ones they skip

def replays(js):
    e=[x for x in json.loads(js) if x.get("pt") is not None and x.get("vt")]
    return sum(1 for a,b in zip(e,e[1:]) if b["vt"]-e[0]["vt"]>=2000 and (b["pt"]-a["pt"])<-1000)

Two lessons hold nearly every rewatch: Nervous system and action potentials (44) and Endocrine and feedback loops (44), both Hard science lessons. A rewatch cluster is not a bad lesson, it is a hard concept that needs a worked example or a slower explanation.

Step 7: Roll it into one content-health score

No single measure ranks a library. Blend completion, mid-lesson retention, the post-lesson quiz, and a gentle penalty for length, then band each lesson.

health = (0.40*completion + 0.25*retention_at_50 + 0.25*quiz + 0.10*length_ok) * 100
band   = "Healthy" if health>=71 else ("Review" if health>=58 else "Fix")

On the sample this gives 5 Healthy, 3 Review, 2 Fix, with a median completion of 63 percent. The two Fix lessons are the coloured red bars at the top of this page, and they are exactly the two hardest science lessons, low completion and heavy rewatch together.

Step 8: Check that content-health tracks the outcome

A content score is only worth editing against if it maps to learning. Correlate each lesson's completion with the post-lesson quiz average.

r = np.corrcoef(lesson.completion, lesson.quiz)[0,1]   # r = 0.92 on the sample

On the sample, lesson completion and the post-lesson quiz move together at r = 0.92, though with only ten lessons (one at seven views) treat that as the method, not a benchmark. On this sample the lessons learners finished were also the lessons they passed, which is what makes the Fix list worth working through.

What the dashboard tells you

Which lessons underperform: rank by content-health, not raw views. A popular lesson with low completion is worth less than a quiet one learners finish. The content-health score combines completion, mid-lesson retention, the quiz, and length into one number you can sort a library by, so the two lessons to fix rise to the top instead of hiding behind their view counts.

Where do learners drop off in a lesson: read the [retention curve](https://fastpix.com/blog/audience-retention-graph-your-own-videos). Completion hides the shape of the loss. Plotting the share of learners still watching across a lesson shows the exact point they leave. A healthy lesson holds most of the class to the end, while a lesson to fix sheds a quarter of its viewers at one spot, here around the 40 percent mark. That position is the edit.

Which concepts are hard: the ones learners rewatch. When learners jump back and replay the same stretch, the concept did not land the first time. Clustering rewatches per lesson surfaces the hard concepts without a survey, and on the sample the two Fix lessons are also the two most rewatched. The fix is targeted: a worked example on those concepts, not a rewrite of the course.

Does better content produce better outcomes: on this sample, they move together. Joining lesson completion to the post-lesson quiz shows the two rise together at r = 0.92, though with only ten lessons here that illustrates the method rather than proving a benchmark. Read that way, the Fix list is also the list most likely to lift the quiz scores those lessons feed.

Part 4 · Act

Turn the dashboard into an edit list

The dashboard earns its keep when it drives the content roadmap. FastPix surfaces the lessons and the exact stretch to change; your team makes the edit. Here is the rule that builds the list:

for L in lessons.itertuples():
    if L.band == "Fix":
        queue_edit(L.lesson, f"re-cut around {L.cliff_pos}%, add a worked example (rewatched {L.replays}x)")
    elif L.band == "Review" and L.mins >= 11:
        queue_edit(L.lesson, "long lesson with a mid drop-off, consider splitting")
# On the sample: 2 re-cuts queued, plus the long Review lessons to split.

None of this writes the lesson for you. It points at the exact lesson and the exact stretch to change, so your content team spends its time editing the two lessons that matter instead of guessing across ten.

Put this on a monthly content review

Run once, this is an audit. Run every month against a fresh export, and it is a content review with an owner: the Fix list is the edit queue, the Review list is the watch list, and re-cut lessons get a fresh cohort of views to prove the curve moved. Ship an edit, give it a month, and rerun the notebook; if completion and the quiz climbed, keep going.

Run this on your own data

Point Step 4 at your own export and set your bands for Healthy, Review and Fix. Re-weight the content-health score until it tracks your own quiz or assessment data, the way Step 8 does here. The retention curve, the drop-off finder and the rewatch signal run unchanged, and the same SQL scales from this laptop file to a real ClickHouse when your library grows.