Skip to content

chore(api): migrate schedule tasks to use Session(db.engine)#35232

Open
jerryzai wants to merge 6 commits intolanggenius:mainfrom
jerryzai:chore/sqlalchemy-session-schedule-tasks
Open

chore(api): migrate schedule tasks to use Session(db.engine)#35232
jerryzai wants to merge 6 commits intolanggenius:mainfrom
jerryzai:chore/sqlalchemy-session-schedule-tasks

Conversation

@jerryzai
Copy link
Copy Markdown

@jerryzai jerryzai commented Apr 15, 2026

Summary

  • Replace db.session with with Session(db.engine) as session: in all Celery schedule task files under api/schedule/
  • Remove the stale finally: db.session.close() guard in queue_monitor_task.py (no longer needed with context-managed sessions)
  • Each task now owns its own session lifetime, scoped tightly to the DB work it performs

Files changed

  • api/schedule/clean_embedding_cache_task.py
  • api/schedule/check_upgradable_plugin_task.py
  • api/schedule/queue_monitor_task.py
  • api/schedule/update_tidb_serverless_status_task.py
  • api/schedule/create_tidb_serverless_task.py

Motivation

Part of the ongoing effort to remove the flask_sqlalchemy dependency (#24138#23647).
Schedule tasks run outside a Flask request context, so using the scoped db.session has always been fragile; explicit
Session(db.engine) blocks make the session lifetime obvious and safe.

Test plan

  • Existing unit/integration tests pass (uv run --project api pytest)
  • No regressions in Celery task behaviour — sessions are closed at the correct boundaries and commits are preserved.

@jerryzai jerryzai requested a review from QuantumGhost as a code owner April 15, 2026 01:52
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Apr 15, 2026
@asukaminato0721 asukaminato0721 requested a review from Copilot April 15, 2026 02:18
@github-actions
Copy link
Copy Markdown
Contributor

Pyrefly Diff

base → PR
--- /tmp/pyrefly_base.txt	2026-04-15 02:19:39.081559899 +0000
+++ /tmp/pyrefly_pr.txt	2026-04-15 02:19:29.802351074 +0000
@@ -579,7 +579,7 @@
 ERROR Argument `SimpleNamespace` is not assignable to parameter `dataset` with type `Dataset` in function `dify_vdb_weaviate.weaviate_vector.WeaviateVectorFactory.init_vector` [bad-argument-type]
    --> providers/vdb/vdb-weaviate/tests/unit_tests/test_weaviate_vector.py:912:42
 ERROR No matching overload found for function `redis.client.Redis.__init__` called with arguments: (host=int | str | Unknown, port=int | str | Unknown, password=int | str | Unknown | None, db=int, ssl=bool, ssl_ca_certs=str | None, ssl_cert_reqs=Any | None, ssl_certfile=Any | None, ssl_keyfile=Any | None, socket_timeout=Literal[5], socket_connect_timeout=Literal[5], health_check_interval=Literal[30]) [no-matching-overload]
-  --> schedule/queue_monitor_task.py:14:21
+  --> schedule/queue_monitor_task.py:13:21
 ERROR Object of class `Tenant` has no attribute `role` [missing-attribute]
     --> services/account_service.py:1154:13
 ERROR No matching overload found for function `flask.helpers.stream_with_context` called with arguments: (Generator[bytes]) [no-matching-overload]

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors Celery scheduled tasks under api/schedule/ to avoid using the Flask-SQLAlchemy scoped db.session by introducing explicit SQLAlchemy Session(db.engine) lifetimes, as part of the broader effort to remove flask_sqlalchemy usage in non-request contexts.

Changes:

  • Replaced db.session usage with context-managed Session(db.engine) blocks in several scheduled tasks.
  • Removed a stale finally: db.session.close() guard from queue_monitor_task.py.
  • Tightened session ownership so tasks control session lifetime explicitly.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
api/schedule/clean_embedding_cache_task.py Uses Session(db.engine) for batched embedding cleanup commits.
api/schedule/check_upgradable_plugin_task.py Loads auto-upgrade strategies via Session(db.engine) (but session scope is now too broad).
api/schedule/queue_monitor_task.py Removes db.session import/close guard; task is Redis/email only.
api/schedule/update_tidb_serverless_status_task.py Loads TiDB bindings via Session(db.engine) before calling cluster status updater.
api/schedule/create_tidb_serverless_task.py Uses Session(db.engine) for counting idle clusters and persisting newly created bindings.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread api/schedule/update_tidb_serverless_status_task.py Outdated
Comment thread api/schedule/check_upgradable_plugin_task.py Outdated
auto-merge was automatically disabled April 15, 2026 02:33

Head branch was pushed to by a user without write access

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants