make VideoFromFile.get_stream_source() re-entrant and BytesIO-compatible#11456
Open
bigcat88 wants to merge 1 commit intoComfy-Org:masterfrom
Open
make VideoFromFile.get_stream_source() re-entrant and BytesIO-compatible#11456bigcat88 wants to merge 1 commit intoComfy-Org:masterfrom
bigcat88 wants to merge 1 commit intoComfy-Org:masterfrom
Conversation
32fe5f4 to
ddb5dce
Compare
ddb5dce to
28db275
Compare
Member
Test Evidence CheckIf this PR modifies behavior that requires testing, a test explanation is required. PRs lacking applicable test explanations may not be reviewed until added. Please add test explanations to ensure code quality and prevent regressions. If this PR changes user-facing behavior, visual proof (screen recording or screenshot) is required. PRs without applicable visual documentation may not be reviewed until provided. You can add it by:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes a concurrency issue in ComfyUI video inputs when the same in-memory video is consumed by multiple parallel async nodes.
Previously VideoFromFile could expose a shared
BytesIOcursor, causing one consumer to advance the stream and the other to read truncated data or hit EOF.The solution introduces
_ReentrantBytesIO: a read-only, seekable BytesIO subclass that shares immutable bytes while providing an independent cursor per consumer.VideoFromFile.get_stream_source()now returns a fresh _ReentrantBytesIO instance for in-memory sources, preserving compatibility with downstream code that checksisinstance(x, BytesIO)Other options considered:
Return a new BytesIO on every call (copy each time)
Too expensive for large videos and parallel graphs.
Lock/serialize access to a shared BytesIO
Avoids copying but removes parallelism (and still relies on cursor rewinds)
Write to a temporary file and return a path
Compatible and re-entrant via independent file handles. The smallest implementation in size, can be found here
Currently chosen: return a lightweight BytesIO-compatible re-entrant view (shared bytes, independent cursor)
Minimal overhead, preserves parallelism, and keeps compatibility with node packs that expect a BytesIO.