Skip to content

gh-134262: Add retries to generate_sbom.py#134263

Merged
hugovk merged 6 commits intopython:mainfrom
emmatyping:retry-generate-sbom-dl
May 21, 2025
Merged

gh-134262: Add retries to generate_sbom.py#134263
hugovk merged 6 commits intopython:mainfrom
emmatyping:retry-generate-sbom-dl

Conversation

@emmatyping
Copy link
Copy Markdown
Member

@emmatyping emmatyping commented May 19, 2025 •
edited by bedevere-app Bot
Loading

Tools/build/generate_sbom.py downloads artifacts, which can flake. This commit adds simple exponential back-off retries to the download to avoid failures due to network flakes.

@emmatyping
Add retries to generate_sbom.py
6a0f84f 
generate_sbom.py downloads artifacts, which can
flake. This commit adds exponential backoff to the
download to avoid failures due to network flakes.
@emmatyping emmatyping requested a review from sethmlarson as a code owner May 19, 2025 18:08
@bedevere-app bedevere-app Bot mentioned this pull request May 19, 2025
Closed
sethmlarson
sethmlarson approved these changes May 19, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@sethmlarson sethmlarson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

donbarbos
donbarbos approved these changes May 19, 2025
View reviewed changes
Comment thread Tools/build/generate_sbom.py Outdated
Comment thread Tools/build/generate_sbom.py Outdated
jeanmarinho529
jeanmarinho529 reviewed May 19, 2025
View reviewed changes
Comment thread Tools/build/generate_sbom.py Outdated
emmatyping and others added 2 commits May 19, 2025 12:45
@emmatyping @donbarbos
Reformat function signature
dc830fe 
Co-authored-by: Semyon Moroz <donbarbos@proton.me>
@emmatyping
Apply suggestions from review
8ade104 
- Add `base_delay` for retries
- use a for loop instead of while loop
- rename retries to max_retries
@hugovk hugovk merged commit 0c5a8b0 into python:main May 21, 2025
46 checks passed
@hugovk
Copy link
Copy Markdown
Member

hugovk commented May 21, 2025

Thanks! And backport to 3.13?

@emmatyping
Copy link
Copy Markdown
Member Author

emmatyping commented May 21, 2025

Yeah backport to 3.14 and 3.13 is probably a good idea.

@hugovk
Copy link
Copy Markdown
Member

hugovk commented May 21, 2025

Would you like to add the "needs backport" labels?

@emmatyping emmatyping added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels May 21, 2025
@miss-islington-app
Copy link
Copy Markdown

miss-islington-app Bot commented May 21, 2025

Thanks @emmatyping for the PR, and @hugovk for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@miss-islington-app
Copy link
Copy Markdown

miss-islington-app Bot commented May 21, 2025

Thanks @emmatyping for the PR, and @hugovk for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 21, 2025
@emmatyping @donbarbos @miss-islington
pythongh-134262: Add retries to generate_sbom.py (pythonGH-134263)
d99a712 
(cherry picked from commit 0c5a8b0)

Co-authored-by: Emma Smith <emma@emmatyping.dev>
Co-authored-by: Semyon Moroz <donbarbos@proton.me>
@miss-islington-app
Copy link
Copy Markdown

miss-islington-app Bot commented May 21, 2025

Sorry, @emmatyping and @hugovk, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 0c5a8b0b55238a45b9073d06a10c3a59568cdf3c 3.13

@bedevere-app
Copy link
Copy Markdown

bedevere-app Bot commented May 21, 2025

GH-134460 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label May 21, 2025
@hugovk
Copy link
Copy Markdown
Member

hugovk commented May 21, 2025 •
edited
Loading

Here's a build that failed even with download_with_retries:

https://github.com/python/cpython/actions/runs/15169080044/job/42654367027?pr=134456

Shall we bump some numbers? And print out download_with_retries and the current retry/delay?

Would help us identify any flaky resources and possibly fetch from a reliable location.

@emmatyping
Copy link
Copy Markdown
Member Author

emmatyping commented May 21, 2025

Yeah, I can bump numbers (wanted to be conservative initially), note the location, and maybe add jitter?

@hugovk
Copy link
Copy Markdown
Member

hugovk commented May 21, 2025

Thanks, not sure how much jitter would help here, but no objection.

lkollar pushed a commit to lkollar/cpython that referenced this pull request May 26, 2025
@emmatyping @donbarbos @lkollar
pythongh-134262: Add retries to generate_sbom.py (python#134263)
2ce7e21 
Co-authored-by: Semyon Moroz <donbarbos@proton.me>
Pranjal095 pushed a commit to Pranjal095/cpython that referenced this pull request Jul 12, 2025
@emmatyping @donbarbos @Pranjal095
pythongh-134262: Add retries to generate_sbom.py (python#134263)
908bc6a 
Co-authored-by: Semyon Moroz <donbarbos@proton.me>
taegyunkim pushed a commit to taegyunkim/cpython that referenced this pull request Aug 4, 2025
@emmatyping @donbarbos @taegyunkim
pythongh-134262: Add retries to generate_sbom.py (python#134263)
3a9a5cd 
Co-authored-by: Semyon Moroz <donbarbos@proton.me>
emmatyping added a commit to emmatyping/cpython that referenced this pull request Aug 6, 2025
@emmatyping @donbarbos
[3.13] pythongh-134262: Add retries to generate_sbom.py (pythonGH-134263
f08fd10 
)

(cherry picked from commit 0c5a8b0)

Co-authored-by: Emma Smith <emma@emmatyping.dev>
Co-authored-by: Semyon Moroz <donbarbos@proton.me>
@bedevere-app
Copy link
Copy Markdown

bedevere-app Bot commented Aug 6, 2025

GH-137468 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Aug 6, 2025
@emmatyping emmatyping deleted the retry-generate-sbom-dl branch August 6, 2025 15:23
emmatyping added a commit that referenced this pull request Aug 6, 2025
@emmatyping @donbarbos
[3.13] gh-134262: Add retries to generate_sbom.py (GH-134263) (#137468)
7762de2 
(cherry picked from commit 0c5a8b0)

Co-authored-by: Semyon Moroz <donbarbos@proton.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sethmlarson sethmlarson sethmlarson approved these changes

+2 more reviewers

@jeanmarinho529 jeanmarinho529 jeanmarinho529 left review comments

@donbarbos donbarbos donbarbos approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@hugovk hugovk

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@emmatyping @hugovk @sethmlarson @donbarbos @jeanmarinho529