Skip to content

Feature/sws2 18#583

Merged
duprijil merged 13 commits intodevelopfrom
feature/SWS2-18
May 27, 2025
Merged

Feature/sws2 18#583
duprijil merged 13 commits intodevelopfrom
feature/SWS2-18

Conversation

@duprijil
Copy link
Copy Markdown
Collaborator

Implements automatisation of daily view materialisation DAG.

  • Added mechanism to automatically trigger the daily_materialized_views_update DAG based on successful completion of the main DAG (notice_processing_pipeline);
  • Removed the current fixed scheduling and replacing it with event-driven execution (Datasets);
  • Introducing a semaphore mechanism to prevent concurrent materialisation runs and avoid conflicts (max_active_runs = 1);
  • Added DAGs testing and coverage.

@duprijil duprijil requested a review from Dragos0000 May 23, 2025 12:23
@duprijil duprijil self-assigned this May 23, 2025
@duprijil duprijil added the enhancement New feature or request label May 23, 2025
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 25, 2025

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (develop@366b5d4). Learn more about missing BASE report.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop     #583   +/-   ##
==========================================
  Coverage           ?   92.80%           
==========================================
  Files              ?      122           
  Lines              ?     6295           
  Branches           ?        0           
==========================================
  Hits               ?     5842           
  Misses             ?      453           
  Partials           ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread infra/airflow/Dockerfile Outdated
@duprijil duprijil merged commit 726ac53 into develop May 27, 2025
4 checks passed
@duprijil duprijil deleted the feature/SWS2-18 branch May 27, 2025 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants