Operation

info

Due to its unique relationship with the web package, the media player package operates in a different way. More information can be found on the media player package section.

Required Privileges

In addition to the standard privileges required by dbt, our packages by default write to additional schemas beyond just your profile schema. If your connected user does not have create schema privileges, you will need to ensure that the following schemas exist in your warehouse and the user can create tables in them:

• <profile_schema>_derived
• <profile_schema>_scratch
• <profile_schema>_snowplow_manifest

Alternatively, you can override the output schemas our models write to, see the relevant package configuration page for how to do this.

Snowflake

  grant create schema on database <database_name> to role <role_name>;

   --alternatively
   create schema <profile_schema>_derived;
   create schema <profile_schema>_scratch;
   create schema <profile_schema>_manifest;
   grant usage on schema <profile_schema>_derived to role <role_name>;
   grant usage on schema <profile_schema>_scratch to role <role_name>;
   grant usage on schema <profile_schema>_manifest to role <role_name>;

For more information, please refer to the Official Guide on setting up permissions.

BigQuery

Please refer to the Official Guide on setting up permissions.

Databricks

   -- user with "use catalog" privilege on the catalog
  grant create schema on catalog <catalog_name> to <principal_name>

   --alternatively
   create schema <profile_schema>_derived;
   create schema <profile_schema>_scratch;
   create schema <profile_schema>_manifest;
   grant usage on schema <profile_schema>_derived to <user_name>;
   grant usage on schema <profile_schema>_scratch to <user_name>;
   grant usage on schema <profile_schema>_manifest to <user_name>;

For more options (e.g.: granting to service principal, or group instead of users), please refer to the Official Guide on setting up permissions.

Redshift

   -- someone with superuser access
   create schema authorization <user_name>;

   --alternatively
   create schema <profile_schema>_derived;
   create schema <profile_schema>_scratch;
   create schema <profile_schema>_manifest;
   grant usage on schema <profile_schema>_derived to <user_name>;
   grant usage on schema <profile_schema>_scratch to <user_name>;
   grant usage on schema <profile_schema>_manifest to <user_name>;

For more options (e.g.: granting to role, or group instead of users), please refer to the Official Guide on setting up permissions.

Postgres

   -- someone with superuser access
   create schema authorization <user_name>;

   --alternatively
   create schema <profile_schema>_derived;
   create schema <profile_schema>_scratch;
   create schema <profile_schema>_manifest;
   grant usage on schema <profile_schema>_derived to <user_name>;
   grant usage on schema <profile_schema>_scratch to <user_name>;
   grant usage on schema <profile_schema>_manifest to <user_name>;

For more information, please refer to the Official Guide on setting up permissions.

Manifest Tables

Each of our packages has a set of manifest tables that manage the Incremental Sessionization Logic logic of our package, as well as thinks such as quarantined sessions.

danger

These manifest tables are critical to the package and as such are protected from full refreshes, i.e. being dropped, when running in production by default. In development refreshes are enabled.

Snowplow Web

There are 3 manifest tables included in this package:

• snowplow_web_incremental_manifest: Records the current state of the package
• snowplow_web_base_sessions_lifecycle_manifest: Records the start & end timestamp of all sessions
• snowplow_web_base_quarantined_sessions: Records sessions that have exceeded the maximum allowed session length, defined by snowplow__max_session_days (default 3 days)

Snowplow Mobile

There are 2 manifest tables included in this package:

• snowplow_mobile_incremental_manifest: Records the current state of the package
• snowplow_mobile_base_sessions_lifecycle_manifest: Records the start & end timestamp of all sessions

Snowplow Normalize

There is 1 manifest table included in this package:

• snowplow_normalize_incremental_manifest: Records the current state of the package

Snowplow E-commerce

There are 3 manifest tables included in this package:

• snowplow_ecommerce_incremental_manifest: Records the current state of the package
• snowplow_ecommerce_base_sessions_lifecycle_manifest: Records the start & end timestamp of all sessions
• snowplow_ecommerce_base_quarantined_sessions: Records sessions that have exceeded the maximum allowed session length, defined by snowplow__max_session_days (default 3 days)

The allow_refresh() macro defines the protection behavior. As dbt recommends, target names are used here to differentiate between your prod and dev environment. By default, this macro assumes your dev target is named dev. This can be changed by setting the snowplow__dev_target_name var in your dbt_project.yml file.

To full refresh any of the manifest models in production, set the snowplow__allow_refresh to true at run time.

Alternatively, you can amend the behavior of this macro entirely by overwriting it. See the Overwriting Macros section for more details.