Skip to content

TEDSWS-181 Parametrise notice reprocessing DAG#582

Merged
duprijil merged 8 commits intodevelopfrom
feature/SWS2-23
May 27, 2025
Merged

TEDSWS-181 Parametrise notice reprocessing DAG#582
duprijil merged 8 commits intodevelopfrom
feature/SWS2-23

Conversation

@Dragos0000
Copy link
Copy Markdown
Contributor

Description

This PR introduces two new Airflow DAGs to reprocess notices from the backlog, improving usability and flexibility while removing legacy implementations.


New

1. reprocess_notices_from_backlog_by_status

  • Reprocesses notices based on a configurable list of notice statuses.
  • Supports optional start_date and end_date as publication date filters.
  • Notice statuses are provided as a required parameter in the UI (multi-line array).

2. reprocess_notices_by_id_from_backlog

  • Reprocesses notices based on a list of explicit TED notice IDs.
  • Notice IDs are provided as a required parameter in the UI (multi-line array).

Cleanup

  • Removed old/legacy reprocessing DAGs that used hardcoded status filters or outdated logic.

UI Improvements

  • DAGs now have user-friendly names shown in the Airflow UI (e.g., Reprocess Notices From Backlog By Status).
  • Parameter fields include clear descriptions and input examples to help users configure DAGs easily.

@Dragos0000 Dragos0000 requested a review from duprijil May 23, 2025 12:10
@Dragos0000 Dragos0000 changed the base branch from main to develop May 26, 2025 06:54
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 26, 2025

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (develop@f012140). Learn more about missing BASE report.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop     #582   +/-   ##
==========================================
  Coverage           ?   96.38%           
==========================================
  Files              ?      105           
  Lines              ?     5609           
  Branches           ?        0           
==========================================
  Hits               ?     5406           
  Misses             ?      203           
  Partials           ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@duprijil duprijil merged commit 366b5d4 into develop May 27, 2025
4 checks passed
@duprijil duprijil deleted the feature/SWS2-23 branch May 27, 2025 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants