Skip to content

Commit a871acd

Browse files
feature: add optimize() call to snapshot materialization (#1335) (#1341)
partially resolves #1335 ### Description 1. added `optimize()` call to the snapshot materialization, matching existing behavior in table and incremental materializations. when a snapshot is configured with `liquid_clustered_by`, `zorder`, or `auto_liquid_cluster`, the `OPTIMIZE` query will now execute automatically after each snapshot run. 2. the change is a single-line addition to `snapshot.sql`, placed after `persist_constraints` and before `post-hooks` (again, consistent with the pattern in `table.sql` and `incremental.sql`). **note**: the original issue also mentioned default clustering columns and `ALTER TABLE` support for config changes. these are intentionally excluded from this PR to keep scope minimal and can be addressed in follow-up PRs if needed. i believe they need a design decision that should be discussed by/with maintainers first due to having a larger scope. ### Checklist - [x] I have run this code in development and it appears to resolve the stated issue (**unit tests pass :** this change mirrors the identical optimize() call pattern already used in `table.sql` and `incremental.sql`.) - [x] This PR includes tests, or tests are not required/relevant for this PR - [x] I have updated the `CHANGELOG.md` and added information about my change to the "dbt-databricks next" section. --------- Signed-off-by: aarushisingh04 <aarushi07.singh@gmail.com>
1 parent b647999 commit a871acd

3 files changed

Lines changed: 10 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
gated behind the `use_concurrent_microbatch` behavior flag (default: `false`).
77
Opt in via `flags: {use_concurrent_microbatch: true}` in `dbt_project.yml`
88
([#914](https://github.com/databricks/dbt-databricks/issues/914))
9+
- Run `optimize` on snapshots when `liquid_clustered_by`, `zorder`, or `auto_liquid_cluster` config is set, matching existing behavior for tables and incremental models ([#1335](https://github.com/databricks/dbt-databricks/issues/1335))
910

1011
### Fixes
1112

dbt/include/databricks/macros/materializations/snapshot.sql

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,8 @@
124124

125125
{% do persist_constraints(target_relation, model) %}
126126

127+
{% do optimize(target_relation) %}
128+
127129
{{ run_hooks(post_hooks, inside_transaction=False) }}
128130

129131
{{ return({'relations': [target_relation]}) }}

tests/functional/adapter/simple_snapshot/test_snapshot.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,13 @@ class TestSnapshotCheck(BaseSnapshotCheck):
3232
pass
3333

3434

35+
@pytest.mark.skip_profile("databricks_uc_cluster", "databricks_cluster")
36+
class TestSnapshotLiquidClustering(TestSnapshot):
37+
@pytest.fixture(scope="class")
38+
def project_config_update(self):
39+
return {"snapshots": {"+liquid_clustered_by": "id"}}
40+
41+
3542
@pytest.mark.skip_profile("databricks_cluster")
3643
class TestSnapshotIceberg(BaseSnapshotCheck):
3744
@pytest.fixture(scope="class")

0 commit comments

Comments
 (0)