dbt (data build tool) interview questions


11. How do I use the incremental materialization?
incremental models are defined with select statements, with the the materialization defined in a config block.
select …
To use incremental models, you also need to tell dbt, on how to filter the rows on an incremental run and the uniqueness constraint of the model (if any).

12. How to do Filtering rows on an incremental ?
To tell dbt which rows it should transform on an incremental run, wrap valid SQL that filters for these rows in the is_incremental() macro. Often, you’ll want to filter for “new” rows, as in, rows that have been created since the last time dbt ran this model. The best way to find the timestamp of the most recent run of this model is by checking the most recent timestamp in your target table. dbt makes it easy to query your target table by using the “{{ this }}” variable.

13. How do I rebuild an incremental model?
If your incremental model logic has changed, the transformations on your new rows of data may diverge from the historical transformations, which are stored in your target table. In this case, you should rebuild your incremental model. To force dbt to rebuild the entire incremental model from scratch, use the –full-refresh flag on the command line. This flag will cause dbt to drop the existing target table in the database before rebuilding it for all-time.
$ dbt run –full-refresh –models my_incremental_model+

14. What is the is_incremental() macro ?
The is_incremental() macro will return True if:
the destination table already exists in the database
dbt is not running in full-refresh mode
the running model is configured with materialized=’incremental’

15. What if the columns of my incremental model change?
If you add a column from your incremental model, and execute a dbt run, this column will not appear in your target table. Similarly, if you remove a column from your incremental model, and execute a dbt run, this column will not be removed from your target table. Instead, whenever the logic of your incremental changes, execute a full-refresh run of both your incremental model and any downstream models.

Author: user

Leave a Reply