Skip to content

Commit e4a4826

Browse files
fix: copy dbt-origin dicts before mutation to fix fusion preview.134+ regression (#959)
* fix: copy dbt-origin dicts before mutation to fix fusion preview.134+ regression Hypothesis: dbt-fusion preview.134+ changed internal dict/map types to be immutable. Dicts originating from dbt internals (kwargs, model objects, graph nodes) no longer support .update(). Locally-created dicts are fine. Fix: wrap dbt-origin dicts with dict() to create mutable copies before calling .update() on them: - store_anomaly_test_results.sql: test_params from test_metadata.kwargs - python.sql: test_node from context.model, test_node.config - generate_json_schema_test.sql: node from get_node_by_name(), node.config CI pinned to preview.143 (known-broken) to validate the hypothesis. Co-Authored-By: Itamar Hartstein <haritamar@gmail.com> * ci: remove fusion version pin — hypothesis confirmed Fusion snowflake CI passed with preview.143 + dict() copies, confirming that dbt-origin dicts just need to be copied before mutation. Remove the version pin to use latest fusion. Co-Authored-By: Itamar Hartstein <haritamar@gmail.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Itamar Hartstein <haritamar@gmail.com>
1 parent f950ea8 commit e4a4826

4 files changed

Lines changed: 11 additions & 8 deletions

File tree

.github/workflows/test-warehouse.yml

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -141,9 +141,7 @@ jobs:
141141
- name: Install dbt-fusion
142142
if: inputs.dbt-version == 'fusion'
143143
run: |
144-
# Pin to preview.127 — later versions (134+) regress dict .update()/.pop().
145-
# See https://github.com/elementary-data/dbt-data-reliability/pull/956 for bisect details.
146-
curl -fsSL https://public.cdn.getdbt.com/fs/install/install.sh | sh -s -- --version 2.0.0-preview.127
144+
curl -fsSL https://public.cdn.getdbt.com/fs/install/install.sh | sh -s --
147145
148146
- name: Install Elementary
149147
run: pip install "./elementary[${{ (inputs.warehouse-type == 'databricks_catalog' && 'databricks') || inputs.warehouse-type }}]"

macros/commands/generate_json_schema_test.sql

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,11 @@
1515

1616
{% set elementary_database_name, elementary_schema_name = elementary.get_package_database_and_schema() %}
1717

18-
{% do node.config.update({"packages": ["genson"]}) %}
19-
{% do node.update({'database': elementary_database_name, 'schema': elementary_schema_name}) %}
18+
{# Copy node dict so we can mutate it — dbt graph nodes may be immutable maps in fusion #}
19+
{% set node = dict(node) %}
20+
{% set node_config = dict(node.get('config', {})) %}
21+
{% do node_config.update({"packages": ["genson"]}) %}
22+
{% do node.update({'config': node_config, 'database': elementary_database_name, 'schema': elementary_schema_name}) %}
2023
{% if node.resource_type == 'source' %}
2124
{# Source nodes don't have alias, and submit_python_job expects it #}
2225
{% do node.update({'alias': "jsonschemagen_{}_{}".format(node.source_name, node.name)}) %}

macros/edr/data_monitoring/anomaly_detection/store_anomaly_test_results.sql

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
{%- set full_table_name = elementary.insensitive_get_dict_value(latest_row, 'full_table_name') %}
2222
{%- set test_unique_id = flattened_test.unique_id %}
2323
{%- set test_configuration = elementary.get_cache(test_unique_id) %}
24-
{%- set test_params = elementary.insensitive_get_dict_value(flattened_test, 'test_params') %}
24+
{%- set test_params = dict(elementary.insensitive_get_dict_value(flattened_test, 'test_params')) %}
2525
{%- do test_params.update(test_configuration) %}
2626
{%- set parent_model_unique_id = elementary.insensitive_get_dict_value(flattened_test, 'parent_model_unique_id') %}
2727
{%- set column_name = elementary.insensitive_get_dict_value(latest_row, 'column_name') %}

macros/edr/tests/python.sql

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
{% endif %}
1818

1919
{% set test_args = kwargs %}
20-
{% set test_node = context.model %}
20+
{% set test_node = dict(context.model) %}
2121
{% set model_relation = model.quote(false, false, false) %}
2222
{% set elementary_database_name, elementary_schema_name = elementary.get_package_database_and_schema() %}
2323
{% set output_table = api.Relation.create(database=elementary_database_name, schema=elementary_schema_name,
@@ -30,7 +30,9 @@
3030
#}
3131
{% do test_node.update({'database': elementary_database_name, 'schema': elementary_schema_name}) %}
3232

33-
{% do test_node.config.update(test_args) %}
33+
{% set config_copy = dict(test_node.get('config', {})) %}
34+
{% do config_copy.update(test_args) %}
35+
{% do test_node.update({'config': config_copy}) %}
3436

3537
{% if code_macro is string %}
3638
{% set user_py_code_macro = context[code_macro] %}

0 commit comments

Comments
 (0)