> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://fastpix.com/docs/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://fastpix.com/docs/_mcp/server. # Export view data to Google Cloud Pub/Sub Stream every completed video view from FastPix into a Google Cloud Pub/Sub topic in your own project, then read the records into BigQuery, a push endpoint, or a client library. Use streaming exports when you want view data in near real time rather than waiting for the [daily CSV export](/docs/video-data/export-raw-view-data), for example to feed a live alerting system, a warehouse, or a per-view workflow. \ ## Before you begin Make sure you have the following: * A FastPix account on a paid plan. Streaming exports aren't available on the free plan. * A Google Cloud project with billing enabled. Pub/Sub doesn't work without it. * Permission to create topics and edit IAM in that project (`roles/pubsub.admin` or owner). * The **FastPix service account email** shown on your streaming exports settings page. It looks like `fastpix-export-prod-eu@fastpix-prod.iam.gserviceaccount.com`. This email isn't a secret. Knowing it grants no access; permission exists only because you grant it below. \ ## How streaming exports work FastPix publishes each completed view into your topic. You own the topic and the subscription. FastPix only publishes, using a permission you grant and can revoke at any time. No credentials are exchanged. Only two values move between you and FastPix, and neither is a secret: | Direction | Value | | -------------- | ------------------------------------- | | FastPix to you | A service account email (an identity) | | You to FastPix | Your topic path (an address) | FastPix never receives your password, key, or any credential, and it can't read your subscription. It can only write to your topic. \ ## Set up the topic Create a Pub/Sub topic. Using the Google Cloud console, go to **Pub/Sub > Topics > Create topic**, give the topic an ID (for example `fastpix-views`), and leave **Add a default subscription** ticked, which creates the subscription for you. Using the gcloud CLI: ```bash gcloud pubsub topics create fastpix-views --project=YOUR-PROJECT ``` Topic naming rules: 3 to 255 characters, starts with a letter, may contain `a-z A-Z 0-9 - _ . ~ + %`, no `/`, and can't start with `goog`. > **IMPORTANT:** Pick the name carefully. Once the export is live, renaming or deleting the topic stops delivery and requires FastPix to update your configuration by hand. \ ## Grant FastPix publish access Grant the FastPix service account the **Pub/Sub Publisher** role on the topic, not on the whole project. Using the console, go to **Pub/Sub > Topics >** your topic **> Permissions > Add principal**, paste the FastPix email, choose the role **Pub/Sub Publisher**, and save. Using the CLI: ```bash gcloud pubsub topics add-iam-policy-binding fastpix-views \ --project=YOUR-PROJECT \ --member="serviceAccount:fastpix-export-prod-eu@fastpix-prod.iam.gserviceaccount.com" \ --role="roles/pubsub.publisher" ``` Confirm the binding: ```bash gcloud pubsub topics get-iam-policy fastpix-views --project=YOUR-PROJECT ``` You should see: ```yaml bindings: - members: - serviceAccount:fastpix-export-prod-eu@fastpix-prod.iam.gserviceaccount.com role: roles/pubsub.publisher ``` Grant `roles/pubsub.publisher` and nothing broader. FastPix doesn't need, and shouldn't be given, any other permission. > **NOTE:** If the console rejects the email with a message that it must belong to an active Google account, the principal picker can't resolve a service account from a different organization. That's a limitation of the picker, not the permission. Use the `gcloud` command above instead, which makes the identical binding. If your organization enables Domain Restricted Sharing, you may see `One or more users named in the policy do not belong to a permitted customer`. Your org policy administrator must allow FastPix's organization before the export can be set up. Ask them about `constraints/iam.allowedPolicyMemberDomains`. \ ## Create the subscription Skip this if you left **Add a default subscription** ticked when you created the topic. Using the console, go to **Pub/Sub > Subscriptions > Create subscription**, set a subscription ID (for example `fastpix-views-sub`), select the topic `fastpix-views`, and choose **Pull** delivery. Using the CLI: ```bash gcloud pubsub subscriptions create fastpix-views-sub \ --topic=fastpix-views --project=YOUR-PROJECT ``` > **WARNING:** Create the subscription before you tell FastPix to enable the export. A Pub/Sub topic doesn't store anything. On publish it copies the message into every subscription that exists at that moment, then forgets it. With no subscription, the message is discarded immediately with no error: FastPix logs "published", you receive nothing, and the message can't be recovered. \ ## Finish setup in FastPix Copy the full topic path from the console **Topic name** column, or from the CLI: ```bash gcloud pubsub topics describe fastpix-views --project=YOUR-PROJECT \ --format="value(name)" ``` Either way you end up with: ```text projects/YOUR-PROJECT/topics/fastpix-views ``` Paste it into **Data Exports > Streaming exports** in FastPix and save. Saving stores the address and sends nothing yet. Select **Run connection test** to publish one real message to your topic and start delivery. On the **Streaming exports** tab, open the **Service** menu and choose **Google Cloud Pub/Sub**. FastPix then shows the service account to grant, a ready-to-use gcloud command, and the field where you paste your topic path. Until the connection test passes, the destination is saved but sends nothing, and there is no on/off switch to flip. After the connection test passes, the destination shows a **Delivering** status with the validated topic and schema version. > **NOTE:** A workspace has one streaming destination at a time. To switch to a different topic or service, delete the current destination and set up the new one. The address can't be edited in place. \ ## Read your data Set up a consumer before you enable the export, so the first messages have somewhere to go. There are three common approaches, easiest first. ### Stream into BigQuery with no code Best if you want the data in your warehouse. Create a BigQuery subscription on the topic: ```bash gcloud pubsub subscriptions create fastpix-views-bq \ --topic=fastpix-views \ --project=YOUR-PROJECT \ --bigquery-table=YOUR-PROJECT:analytics.fastpix_views \ --write-metadata ``` `--write-metadata` adds `subscription_name`, `message_id`, `publish_time`, and an `attributes` column alongside `data`. The five message attributes arrive together in `attributes` as JSON, so the identifiers are reachable without parsing the body: ```sql SELECT JSON_VALUE(attributes, '$.view_id') AS view_id, JSON_VALUE(attributes, '$.event_id') AS event_id, JSON_VALUE(data, '$.data.watch_time_ms') AS watch_time_ms FROM `YOUR-PROJECT.analytics.fastpix_views`; ``` Deduplicate on `view_id` and filter out `connection_test` messages. There's an equivalent for Cloud Storage using `--cloud-storage-bucket`. ### Push to your endpoint Pub/Sub POSTs each message to an HTTPS endpoint you own, with no client library and no long-running process: ```bash gcloud pubsub subscriptions create fastpix-views-push \ --topic=fastpix-views \ --project=YOUR-PROJECT \ --push-endpoint=https://your-app.example.com/fastpix-views ``` Return `2xx` to acknowledge a message. Any other status causes a retry. ### Pull with a client library Full control over how each message is handled. ```python from google.cloud import pubsub_v1 import json subscriber = pubsub_v1.SubscriberClient() path = "projects/YOUR-PROJECT/subscriptions/fastpix-views-sub" def callback(message): view_id = message.attributes["view_id"] envelope = json.loads(message.data) upsert_view(view_id, envelope["data"]) # upsert, never insert message.ack() subscriber.subscribe(path, callback=callback).result() ``` ```js const {PubSub} = require('@google-cloud/pubsub'); new PubSub().subscription('fastpix-views-sub').on('message', msg => { const viewId = msg.attributes.view_id; const envelope = JSON.parse(msg.data.toString()); upsertView(viewId, envelope.data); // upsert, never insert msg.ack(); }); ``` ```java Subscriber subscriber = Subscriber.newBuilder( ProjectSubscriptionName.of(projectId, "fastpix-views-sub"), (message, consumer) -> { String viewId = message.getAttributesMap().get("view_id"); JsonNode envelope = MAPPER.readTree(message.getData().toStringUtf8()); upsertView(viewId, envelope.get("data")); // upsert, never insert consumer.ack(); }).build(); subscriber.startAsync().awaitRunning(); ``` To inspect messages by hand, pull from the subscription in the console (**Pub/Sub > Subscriptions >** your subscription **> Messages > Pull**, with acknowledgement left unchecked so messages aren't deleted as you read them), or from the CLI: ```bash gcloud pubsub subscriptions pull fastpix-views-sub --project=YOUR-PROJECT --limit=5 ``` \ ## What a message looks like Each message carries attributes you can read without parsing the body, plus a JSON body. The attributes are useful for filtering and deduplication: | Attribute | Example | | ---------------- | ------------------------------------------------------ | | `workspace_id` | `9a20af9c-3556-4e26-a6ec-d269056d971a` | | `view_id` | `91cf737b-b684-44d2-8bfc-69f2058098ac` | | `event_id` | `evt_01M0SMDPZGAQEECSJCH895YMHD` | | `schema_version` | `export_v1` | | `data_type` | `video_view`, or `connection_test` for the setup probe | The body is UTF-8 JSON, a stable envelope wrapping a per-type `data` object. In the REST API and `gcloud --format=json` the body appears base64-encoded, which is encoding, not encryption, and the console decodes it for you. ```json { "schema_version": "export_v1", "data_type": "video_view", "event_id": "evt_01M0SMDPZGAQEECSJCH895YMHD", "workspace_id": "9a20af9c-3556-4e26-a6ec-d269056d971a", "view_id": "91cf737b-b684-44d2-8bfc-69f2058098ac", "emitted_at": "2026-08-24T10:18:05.412Z", "data": { "video_title": "Dental Billing 101: Mastering Efficiency and Profitability", "view_start": "2026-08-24T10:17:12.793Z", "view_end": "2026-08-24T10:17:40.136Z", "watch_time_ms": 27342, "qoe_score": 0.921535226295831, "country_code": "IN", "city": "Hyderabad", "os_name": "MacOS", "browser_name": "Chrome", "cdn": "cloudflare" } } ``` The example above is trimmed for readability. The full `data` object carries 117 fields, matching the columns of the daily CSV export. Fields with no value for a view are omitted, not sent as `null`. Two identifiers you might expect inside `data`, `workspace_id` and `view_id`, are one level up on the envelope instead. This is byte-for-byte what an [Amazon Kinesis](/docs/video-data/export-to-amazon-kinesis) customer receives, so your parser doesn't change if you move between clouds. Timestamps are RFC 3339 in UTC with three decimal places. Viewer IP address and user agent aren't exported. ### Field groups | Group | Examples | | --------------- | ------------------------------------------------------------------------------------- | | Identity | `fp_viewer_id` `session_id` `player_instance_id` `fp_playback_id` | | Timing | `view_start` `view_end` `watch_time_ms` `playing_time_ms` | | Startup | `video_startup_time_ms` `total_startup_time_ms` `page_load_time_ms` `startup_failed` | | Rebuffering | `rebuffer_count` `rebuffer_time_ms` `rebuffer_ratio` `rebuffer_frequency` | | Quality scores | `qoe_score` `playback_score` `startup_score` `stability_score` `render_quality_score` | | Delivery | `average_bitrate` `avg_upscaling` `avg_request_latency_ms` `dropped_frame_count` | | Engagement | `max_playhead_position_ms` `seeked_count` `used_full_screen` `has_ad` | | Errors | `has_error` `error_code` `error_message` `error_context` | | Geo and network | `country_code` `continent` `region` `city` `asn_name` `cdn` | | Client | `os_name` `browser_name` `device_type` `device_manufacturer` | | Player | `player_name` `player_version` `player_resolution` `player_autoplay_on` | | Video metadata | `video_title` `video_series` `video_language` `video_resolution` `drm_type` | | Video source | `video_source_duration_ms` `video_source_url` `video_source_stream_type` | | Your own | `custom_1` to `custom_10` | ### What may change without a version bump New optional fields may be added inside `data`, and new values may appear in existing fields, so tolerate both and ignore fields you don't recognize. A new `data_type` may appear, so ignore ones you don't know. Renames, removals, and type changes never happen silently: they require `export_v2` and a migration window where both versions are published. The envelope (`schema_version`, `data_type`, `event_id`, `workspace_id`, `view_id`, `emitted_at`) and the RFC 3339 UTC timestamp format don't change. \ ## Handle the data correctly Four rules cover the cases that trip up most integrations. **Create the subscription before the export is enabled.** A topic with no subscription discards messages silently, as noted above. This is the mistake people actually make. **Filter on `data_type`.** When you set up or re-test the export, FastPix publishes one `connection_test` message to prove the grant works before accepting your configuration. It arrives in the same envelope as a view, with a different type. Because `data_type` is a Pub/Sub attribute as well as a body field, you can drop it without deserializing: check `attributes.data_type == "video_view"`. A consumer that reads `data.view_id` unconditionally fails on this message, and it arrives before any real data. **Upsert on `view_id`, never insert.** Delivery is at-least-once, so you'll receive duplicates. If Pub/Sub redelivers an unacknowledged message, the `event_id` repeats. If FastPix re-sends a view, the `event_id` is new but the `view_id` is the same. Upsert on `view_id`, which is the stable identity of a view. Inserting produces duplicate rows. **Don't assume order.** FastPix guarantees no ordering, on every destination, so a pipeline written against that keeps working if you move. FastPix doesn't use Pub/Sub ordering keys, because they serialize publishing and slow delivery for everyone. Completed views are independent and carry their own timestamps, so sort by `view_end` if you need chronological order. > **NOTE:** Pub/Sub keeps unacknowledged messages for 7 days by default. If your consumer is down longer than that, the backlog is dropped. Raise it with `--message-retention-duration` if you need more. \ ## If your export was paused FastPix stops publishing to a topic when it gets a permanent error, such as `PERMISSION_DENIED`, `NOT_FOUND`, `UNAUTHENTICATED`, or `INVALID_ARGUMENT`. This protects other customers from a broken destination. The most common causes are the IAM binding from the grant step being removed, and the topic being renamed or deleted. Views published while the export is paused are lost, not queued, and not replayed once it resumes. To restart it, fix the cause (usually restore `roles/pubsub.publisher` on the topic), then open **Data Exports > Streaming exports** in your FastPix dashboard and select **Re-check and resume**. FastPix re-checks the grant before resuming, and delivery starts from the next view to complete. The daily CSV export still covers those days, so running it alongside streaming gives you a backfill for any gap. \ ## If your plan changes Moving to a free plan stops delivery, but your destination configuration is kept. Returning to a paid plan resumes delivery automatically, with nothing to set up again. \ ## Troubleshooting | Symptom | Cause | Fix | | ----------------------------------------- | -------------------------------------------------------------------------------- | --------------------------------------------------------- | | Console rejects the FastPix email | The console can't resolve external service accounts | Use `gcloud` to add the binding | | `do not belong to a permitted customer` | Domain Restricted Sharing org policy | Ask your org policy admin to allow FastPix's organization | | FastPix says "published", you see nothing | Subscription created after publishing | Create the subscription, then wait for the next view | | Pull returns nothing | No new view since your last pull, or a message is briefly leased by another pull | Wait and pull again | | Messages vanish after viewing | Acknowledgement was enabled in the console pull | Untick it while exploring | | Export stopped and never restarted | FastPix paused it after a permission error | See the paused export section above | \ #### Frequently asked questions A streaming export publishes each view to your Pub/Sub topic in near real time as it completes. The [daily CSV export](/docs/video-data/export-raw-view-data) produces one file per UTC day that you pull on your own schedule. Streaming suits live pipelines and alerting; CSV suits batch loads and archival. No. You grant the FastPix service account the `roles/pubsub.publisher` role on your topic, and you send FastPix your topic path. No keys or secrets are exchanged, and you can revoke the grant at any time. A Pub/Sub topic only delivers to subscriptions that exist at publish time. If the subscription was created after the message was published, the message was discarded with no error. Create the subscription first, then wait for the next view. Delivery is at-least-once, so duplicates are expected. Upsert on `view_id`, which is the stable identity of a view. Don't insert, and don't deduplicate on `event_id`, which changes on every delivery attempt. The console's principal picker can't always resolve a service account from another organization. Use the `gcloud pubsub topics add-iam-policy-binding` command instead, which makes the identical binding and isn't affected. \ ## What's next * [Export view data to Amazon Kinesis](/docs/video-data/export-to-amazon-kinesis) to stream the same records into AWS instead. * [Export raw view data](/docs/video-data/export-raw-view-data) to pull daily CSV files instead of streaming. * [What Video Data do we capture](/docs/video-data/what-video-data-do-we-capture) for the full per-view field reference.