Consent Tracking Custom Module
This custom module is built as an extension of the dbt-snowplow-web package, it transforms raw
For the incremental logic to work within the module you must use at least
RDB Loader v4.0.0, as the custom module relies on the additional
load_tstamp field for dbt native incrementalisation.
Whenever a new consent version is added to be tracked, the model expects an
allow_all event in order to attribute the events to the full list of latest consent scopes. It is advisable to send a test event of that kind straight after deployment so that the model can process the data accurately.
To enable this optional module, the web package must be correctly configured. Please refer to the snowplow-web dbt quickstart guide for a full breakdown of how to set it up.
This custom module consists of a series of dbt models which produce the following aggregated models from the raw consent tracking events:
snowplow_web_consent_log: Snowplow incremental table showing the audit trail of consent and Consent Management Platform (cmp) events
snowplow_web_consent_users: Incremental table of user consent tracking stats
snowplow_web_consent_totals: Summary of the latest consent status, per consent version
snowplow_web_consent_scope_status: Aggregate of current number of users consented to each consent scope
snowplow_web_cmp_stats: Used for modeling cmp_visible events and related metrics
snowplow_web_consent_versions: Incremental table used to keep track of each consent version and its validity
It is assumed that the dbt_snowplow_web package is already installed and configured as per the Quick Start instructions.
Enable the module
You can enable the custom module through the
snowplow__enable_consent variable most conveniently set in your dbt_project.yml file:
Run the module
If you have previously run the web model without this optional module enabled, you can simply enable the module and run
dbt run --selector snowplow_web as many times as needed for this module to catch up with your other data. If you only wish to process this from a specific date, be sure to change your
snowplow__start_date, or refer to the Custom module section for a detailed guide on how to achieve this the most efficient way.
If you haven't run the web package before, then you can run it using
dbt run --selector snowplow_web either through your CLI, within dbt Cloud, or for Enterprise customers you can use the BDP console. In this situation, all models will start in-sync as no events have been processed.