> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://fastpix.com/docs/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://fastpix.com/docs/_mcp/server. # Export view data to Amazon Kinesis Stream every completed video view from FastPix into an Amazon Kinesis data stream in your own AWS account, then process the records with whatever pipeline you already run. Use streaming exports when you want view data in near real time rather than waiting for the [daily CSV export](/docs/video-data/export-raw-view-data), for example to feed a live alerting system, a warehouse, or a per-view workflow. \ ## Before you begin Make sure you have the following: * A FastPix account on a paid plan. Streaming exports aren't available on the free plan. * An AWS account, and permission in it to create a Kinesis data stream, attach a resource policy to that stream, and (if you encrypt the stream with your own KMS key) edit that key's policy. * The **FastPix role ARN** shown on your streaming exports settings page. You name this role in your stream policy so FastPix can write to your stream. \ ## How streaming exports work FastPix writes each completed view into your stream with a `PutRecord` call. You own the stream and whatever reads it. FastPix only writes, using a permission you grant and can revoke at any time. No credentials are exchanged. Only two values move between you and FastPix, and neither is a secret: | Direction | Value | | -------------- | ----------------------------- | | FastPix to you | An IAM role ARN (an identity) | | You to FastPix | Your stream ARN (an address) | FastPix can't read your stream, list your streams, delete anything, or change your retention. Writing to one stream is the only permission it holds. \ ## Set up the data stream Create an on-demand Kinesis data stream. On-demand capacity scales automatically, so you don't have to size shards. Provisioned mode also works, but if you under-provision, FastPix is throttled and views arrive late. Using the AWS console, go to **Kinesis > Data streams > Create data stream**, give the stream a name (for example `video-views`), select **On-demand** capacity mode, and create it. Using the AWS CLI: ```bash aws kinesis create-stream \ --stream-name video-views \ --stream-mode-details StreamMode=ON_DEMAND ``` Note the stream ARN. You paste it into FastPix later. In the console it's on the stream's **Details** tab with a copy icon; from the CLI it's in the `describe-stream-summary` response. ```text arn:aws:kinesis:eu-west-1:123456789012:stream/video-views ``` \ ## Grant FastPix write access Attach a resource policy to the stream that names the FastPix role from your export settings and allows it to write. ```json { "Version": "2012-10-17", "Statement": [{ "Sid": "FastPixStreamingExports", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam:::role/fastpix-export-prod-eu" }, "Action": ["kinesis:PutRecord", "kinesis:PutRecords"], "Resource": "arn:aws:kinesis:eu-west-1:123456789012:stream/video-views" }] } ``` Using the console, go to **Kinesis > Data streams >** your stream **> Data stream sharing > Create policy** (or **Edit** if one exists), paste the JSON, and save. Using the CLI, save the JSON as `grant-fastpix-write.json`, then apply and verify it: ```bash aws kinesis put-resource-policy \ --resource-arn arn:aws:kinesis:eu-west-1:123456789012:stream/video-views \ --policy file://grant-fastpix-write.json aws kinesis get-resource-policy \ --resource-arn arn:aws:kinesis:eu-west-1:123456789012:stream/video-views ``` `PutRecord` sends one record and `PutRecords` sends a batch. Both are listed so batching can be enabled later without every customer editing their policy again. The policy allows writing to this one stream and nothing else. > **NOTE:** The role ARN belongs to FastPix's AWS account, not yours. You aren't creating it, you're naming it in your policy. Copy it from your export settings page. A stream can hold only one sharing policy, so if yours already has one, add the statement above to it rather than replacing what's there. ### If your stream is encrypted with your own KMS key Skip this section if you didn't turn on server-side encryption, or if you use the AWS-managed key (`aws/kinesis`), whose policy you can't edit. With a customer-managed key, the stream policy alone isn't enough. The key's own policy must authorize the same FastPix role, or the write fails while your stream permission looks correct. Add this statement to the key policy: ```json { "Sid": "FastPixStreamingExports", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam:::role/fastpix-export-prod-eu" }, "Action": "kms:GenerateDataKey", "Resource": "*" } ``` Using the console, go to **KMS > Customer managed keys >** your key **> Key policy > Edit**, paste the statement into the `Statement` array, and save. Using the CLI, fetch the current policy, add the statement to the `Statement` array, and put it back: ```bash aws kms get-key-policy --key-id --policy-name default \ --query Policy --output text > key-policy.json # add the statement to the Statement array, then: aws kms put-key-policy --key-id --policy-name default \ --policy file://key-policy.json ``` Inside a key policy, `"Resource": "*"` means this key only. It doesn't widen access beyond it. > **IMPORTANT:** If a write fails with a KMS error, the stream permission isn't the problem, the key is. This is the most common way the setup goes wrong: the stream policy is correct, and the key policy is what blocks it. If you use the AWS-managed key and the connection test returns a KMS error, that key's policy does not allow a cross-account write. Recreate the stream with a customer-managed key and add the statement above. \ ## Set up a consumer Set up whatever reads the stream before you enable the export, or the first records age out of your retention window before anything reads them. FastPix never touches this side. Common choices: * **Kinesis Data Firehose** into S3 or Redshift, with no code, if you want the data in a warehouse. * **A Lambda trigger**, for per-view processing. * **The Kinesis Client Library**, for a long-running consumer that handles sharding and checkpointing. Whatever you build, it needs to do two things: process only view records, and upsert on `view_id`. The example below is deliberately plain Python to show the shape rather than a framework. ```python import json, boto3 kinesis = boto3.client("kinesis", region_name="eu-west-1") STREAM = "arn:aws:kinesis:eu-west-1:123456789012:stream/video-views" def handle(record): envelope = json.loads(record["Data"]) # Not every record is a view. A connection_test arrives when someone # validates the export, and new data_type values can appear without a # version bump. Skip anything you don't recognize. if envelope.get("data_type") != "video_view": return view = envelope["data"] # Delivery is at-least-once. view_id is the stable identity of a view; # event_id is unique per delivery attempt, so it won't deduplicate a # retry. Upsert on view_id or your view counts drift upward quietly. db.upsert("video_views", key="view_id", row=view) ``` Most consumers use Firehose, a Lambda trigger, or the KCL rather than reading shards directly, because those handle sharding and checkpointing for you. \ ## Finish setup in FastPix Copy the stream ARN from the console **Details** tab, or from the CLI: ```bash aws kinesis describe-stream-summary --stream-name video-views \ --query StreamDescriptionSummary.StreamARN --output text ``` Paste it into **Data Exports > Streaming exports** in FastPix and save. There's no separate region field, because the region is already in the ARN. ```text arn:aws:kinesis:eu-west-1:123456789012:stream/video-views ``` Saving stores the address and sends nothing yet. Select **Run connection test** to write one real record into your stream and start delivery. If it fails, you see the actual AWS error straight away. On the **Streaming exports** tab, open the **Service** menu and choose **Amazon Kinesis Data Streams**. FastPix then shows the role to grant, a ready-to-use resource policy, and the field where you paste your stream ARN. > **NOTE:** A workspace has one streaming destination at a time. To switch to a different stream or service, delete the current destination and set up the new one. The address can't be edited in place. \ ## What a view record looks like Each record is a partition key (the `view_id`) and a JSON body. The body is a stable envelope wrapping a per-type `data` object, so a new event type can be added later without changing anything you already parse. ```text partition key: 6db4638e-2e97-43bd-a4b1-d88c00a17c6e (the view_id) data: { "schema_version": "export_v1", "data_type": "video_view", "event_id": "evt_01K3T9WQ2M7YB4XC8HRN5ZQVD1", "workspace_id": "9e655f67-20b5-4963-8bc7-1058a716faff", "view_id": "6db4638e-2e97-43bd-a4b1-d88c00a17c6e", "emitted_at": "2026-09-02T10:14:02.318Z", "data": { "video_title": "test video", "view_start": "2026-09-01T10:51:14.965Z", "view_end": "2026-09-01T10:53:34.824Z", "watch_time_ms": 118568, "qoe_score": 0.9995538499937602, "city": "Hyderabad", "os_name": "MacOS", "cdn": "cloudflare" } } ``` This is byte-for-byte what a [Google Cloud Pub/Sub](/docs/video-data/export-to-google-cloud-pubsub) customer receives, so your parser doesn't change if you move between clouds. The full `data` object carries the same per-view fields as the daily CSV export. Timestamps are RFC 3339 in UTC, always. Viewer IP address and user agent aren't exported. ### What may change without a version bump | Change | Happens? | | -------------------------------------------- | -------------------------------- | | A new optional field appears inside `data` | Yes. Tolerate it. | | A new value appears in an existing field | Yes. Tolerate it. | | A new `data_type` appears | Yes. Ignore ones you don't know. | | A field is renamed, removed, or changes type | No. That would be `export_v2`. | | The timestamp format changes | No. RFC 3339, UTC, always. | \ ## Handle the data correctly Four rules cover the cases that trip up most integrations. **Filter on `data_type`.** When you save your configuration, FastPix writes a `connection_test` record into your stream. It's a real record your consumer receives. Process only records where `data_type` is `video_view`, and ignore any type you don't recognize, because new ones can be added without a version bump. **Read identifiers from the body.** A Kinesis record has no attribute mechanism, so every identifier lives in the JSON. `view_id` is both the partition key and a body field; `event_id`, `workspace_id`, and `data_type` are in the body only. Parse the record to route or deduplicate it. **Upsert on `view_id`, never insert.** Delivery is at-least-once, so you'll receive duplicates. If Kinesis redelivers a record your consumer didn't checkpoint, the `event_id` repeats. If FastPix retries after a transient failure, the `event_id` is new but the `view_id` is the same. `view_id` is the stable identity of a view, so upsert on it. Inserting produces duplicate rows. **Don't assume order.** FastPix guarantees no ordering. Kinesis orders records within a shard, and the partition key is `view_id`, so records may appear to arrive in a meaningful order, but that isn't a promise: it doesn't survive resharding, doesn't hold across shards, and can change without notice. Sort by `view_end` if you need chronological order. > **NOTE:** Watch retention and capacity. Kinesis keeps records for 24 hours by default. If your consumer falls behind your retention window, those records are gone, and draining that backlog is on your side. If you use provisioned mode and under-provision shards, FastPix is throttled and retries with backoff, so views arrive late rather than being lost. On-demand mode avoids this. \ ## If your export was paused FastPix pauses a destination when writing fails in a way retrying can't fix, such as a revoked policy, a deleted stream, or a KMS key that no longer admits FastPix. Views that arrive while the export is paused aren't delivered, and aren't sent later once it's fixed. Delivery resumes from the moment it's restored. FastPix records which views were missed, so support can tell you exactly what you didn't receive and over what window. The daily CSV export still covers those days, so running it alongside streaming gives you a backfill for any gap. \ ## If your plan changes Moving to a free plan stops delivery, but your destination configuration is kept. Returning to a paid plan resumes delivery automatically, with nothing to set up again. \ ## Troubleshooting | What you see | What it usually means | | ------------------------------------- | --------------------------------------------------------------------------------------------------------- | | Nothing arrives at all | The export isn't active yet, or your consumer started after the records aged out. | | Setup failed with a permission error | The resource policy is missing, or names the wrong role. | | Setup failed with an encryption error | The stream permission is fine. Your KMS key policy needs the same role. See the encryption section above. | | Setup failed with "not found" | The stream name or the region in your ARN is wrong. | | Views arrive late, in bursts | Your shards are throttling FastPix. Add shards, or switch to on-demand. | | The same view twice | Expected. Upsert on `view_id`. | | A record that isn't a view | Expected. Filter on `data_type`. | \ #### Frequently asked questions A streaming export delivers each view to your Kinesis stream in near real time as it completes. The [daily CSV export](/docs/video-data/export-raw-view-data) produces one file per UTC day that you pull on your own schedule. Streaming suits live pipelines and alerting; CSV suits batch loads and archival. No. You grant a FastPix IAM role write access through a resource policy on your stream, and you send FastPix your stream ARN. No access keys or secrets are exchanged, and you can revoke the grant at any time. When you save or re-test the export, FastPix writes a `connection_test` record to confirm the write permission works. Filter on `data_type == "video_view"` and ignore any type you don't recognize. Delivery is at-least-once, so duplicates are expected. Upsert on `view_id`, which is the stable identity of a view. Don't insert, and don't deduplicate on `event_id`, which changes on every delivery attempt. If the stream is encrypted with a customer-managed KMS key, that key's policy must also grant the FastPix role `kms:GenerateDataKey`. A KMS error means the key policy denied the write, not the stream policy. \ ## What's next * [Export view data to Google Cloud Pub/Sub](/docs/video-data/export-to-google-cloud-pubsub) to stream the same records into Google Cloud instead. * [Export raw view data](/docs/video-data/export-raw-view-data) to pull daily CSV files instead of streaming. * [What Video Data do we capture](/docs/video-data/what-video-data-do-we-capture) for the full per-view field reference.