Media Player Quickstart

Requirements

In addition to dbt being installed and a web events dataset being available in your database:

• A dataset of media-player web events from the Snowplow JavaScript tracker must be available in the database. In order for this to happen at least one of the JavaScript based media tracking plugins need to be enabled: Media Tracking plugin or YouTube Tracking plugin
• Have the webPage context enabled.
• Have the media-player event schema enabled.
• Have the media-player context schema enabled.
• Depending on the plugin / intention have all the relevant contexts from below enabled:
  • in case of embedded YouTube tracking: Have the YouTube specific context schema enabled.
  • in case of HTML5 audio or video tracking: Have the HTML5 media element context schema enabled.
  • in case of HTML5 video tracking: Have the HTML5 video element context schema enabled.

In addition to the standard privileges required by dbt, our packages by default write to additional schemas beyond just your profile schema. If your connected user does not have create schema privileges, you will need to ensure that the following schemas exist in your warehouse and the user can create tables in them:

• <profile_schema>_derived
• <profile_schema>_scratch
• <profile_schema>_snowplow_manifest

Alternatively, you can override the output schemas our models write to, see the relevant package configuration page for how to do this.

Snowflake

  grant create schema on database <database_name> to role <role_name>;

   --alternatively
   create schema <profile_schema>_derived;
   create schema <profile_schema>_scratch;
   create schema <profile_schema>_manifest;
   grant usage on schema <profile_schema>_derived to role <role_name>;
   grant usage on schema <profile_schema>_scratch to role <role_name>;
   grant usage on schema <profile_schema>_manifest to role <role_name>;

For more information, please refer to the Official Guide on setting up permissions.

BigQuery

Please refer to the Official Guide on setting up permissions.

Databricks

   -- user with "use catalog" privilege on the catalog
  grant create schema on catalog <catalog_name> to <principal_name>

   --alternatively
   create schema <profile_schema>_derived;
   create schema <profile_schema>_scratch;
   create schema <profile_schema>_manifest;
   grant usage on schema <profile_schema>_derived to <user_name>;
   grant usage on schema <profile_schema>_scratch to <user_name>;
   grant usage on schema <profile_schema>_manifest to <user_name>;

For more options (e.g.: granting to service principal, or group instead of users), please refer to the Official Guide on setting up permissions.

Redshift

   -- someone with superuser access
   create schema authorization <user_name>;

   --alternatively
   create schema <profile_schema>_derived;
   create schema <profile_schema>_scratch;
   create schema <profile_schema>_manifest;
   grant usage on schema <profile_schema>_derived to <user_name>;
   grant usage on schema <profile_schema>_scratch to <user_name>;
   grant usage on schema <profile_schema>_manifest to <user_name>;

For more options (e.g.: granting to role, or group instead of users), please refer to the Official Guide on setting up permissions.

Postgres

   -- someone with superuser access
   create schema authorization <user_name>;

   --alternatively
   create schema <profile_schema>_derived;
   create schema <profile_schema>_scratch;
   create schema <profile_schema>_manifest;
   grant usage on schema <profile_schema>_derived to <user_name>;
   grant usage on schema <profile_schema>_scratch to <user_name>;
   grant usage on schema <profile_schema>_manifest to <user_name>;

For more information, please refer to the Official Guide on setting up permissions.

Installation

Check dbt Hub for the latest installation instructions, or read the dbt docs for more information on installing packages. If you are using multiple packages you may need to up/downgrade a specific package to ensure compatibility.

note

Make sure to run the dbt deps command after updating your packages.yml to ensure you have the specified version of each package installed in your project.

Setup

caution

If you are not starting the media player package at the same time as the web package, see the media player package details for how to best sync them.

1. Override the dispatch order in your project

To take advantage of the optimized upsert that the Snowplow packages offer you need to ensure that certain macros are called from snowplow_utils first before dbt-core. This can be achieved by adding the following to the top level of your dbt_project.yml file:

dbt_project.yml

dispatch:
  - macro_namespace: dbt
    search_order: ['snowplow_utils', 'dbt']

If you do not do this the package will still work, but the incremental upserts will become more costly over time.

2. Adding the selectors.yml file

Within the packages we have provided a suite of suggested selectors to run and test the models within the package together with the web model. This leverages dbt's selector flag. You can find out more about each selector in the YAML Selectors section.

These are defined in the selectors.yml file (source) within the package, however in order to use these selections you will need to copy this file into your own dbt project directory. This is a top-level file and therefore should sit alongside your dbt_project.yml file. If you are using multiple packages in your project you will need to combine the contents of these into a single file.

3. Configuring the web model (in case it has not been run before)

Please refer to the Quick Start guide for the Snowplow Web package to make sure you configure the web model appropriately. (e.g. checking the source data or enabling desired contexts).

4. Enable desired contexts

If you have enabled a specific context you will need to enable it in your dbt_project.yml file:

dbt_project.yml

vars:
  snowplow_media_player:
    # set to true if the YouTube context schema is enabled
    snowplow__enable_youtube: true
    # set to true if the HTML5 media element context schema is enabled
    snowplow__enable_whatwg_media: true
    # set to true if the HTML5 video element context schema is enabled
    snowplow__enable_whatwg_video: true

For other variables you can configure please see the model configuration section.

5. Run your model

You can now run your models for the first time by running the below command (see the operation page for more information on operation of the package):

dbt run --selector snowplow_web