security: verify SHA256 of juliaup's own tarball before extraction#1456
Draft
IanButterworth wants to merge 8 commits intomainfrom
Draft
security: verify SHA256 of juliaup's own tarball before extraction#1456IanButterworth wants to merge 8 commits intomainfrom
IanButterworth wants to merge 8 commits intomainfrom
Conversation
Previously, non-Normal path components (e.g. '..' or absolute roots) were silently skipped, meaning a crafted archive could selectively omit files while appearing to succeed. Abort immediately instead so any unexpected archive structure is surfaced as an error. Co-authored-by: Claude <claude@anthropic.com>
…nd log when set Reject any non-HTTPS value supplied via these env vars so that a misconfiguration or malicious override cannot silently downgrade downloads to plain HTTP. Also print a one-time informational message when a custom server URL is in use, so users can see at a glance that downloads are not going to the official julialang.org servers. Co-authored-by: Claude <claude@anthropic.com>
- unpack_accepts_normal_paths: verify clean tarballs extract correctly - unpack_rejects_path_traversal_dotdot: verify crafted archives with '..' components are rejected with an error - juliaup_server_rejects_http / juliaup_nightly_server_rejects_http: verify non-HTTPS values for JULIAUP_SERVER / JULIAUP_NIGHTLY_SERVER are rejected - juliaup_server_accepts_https: verify valid HTTPS override is accepted Co-authored-by: Claude <claude@anthropic.com>
- unpack_rejects_path_traversal_dotdot: tar::Builder rejects '..' paths itself, so the test setup was failing before reaching unpack_sans_parent. Replace with a hand-rolled raw tar.gz that writes the path bytes directly into the header, bypassing the builder's check. The ParentDir component then reaches our own bail! as intended. - unpack_accepts_normal_paths: entry.unpack() requires the parent directory to already exist. Add a directory entry for 'top/bin' before the file entry so the extraction succeeds. Co-authored-by: Claude <claude@anthropic.com>
The tests asserted starts_with("1.10.10") but 1.10.11 has since been
released. Check that the channel was updated past 1.10.9 instead, which
is what the test semantically verifies.
These failures are unrelated to the security changes in this branch.
Co-authored-by: Claude <claude@anthropic.com>
The self-update and installer paths previously downloaded the juliaup tarball and unpacked it directly with no integrity check. A server-side compromise (CDN, S3 bucket) or MITM could substitute a malicious binary. Changes: - Release CI (package-unix): after creating all .tar.gz archives, run sha256sum to produce juliaup-<VERSION>-SHA256SUMS and upload it to S3 alongside the tarballs. - operations.rs: add download_sha256sums_entry (fetches and parses the SHA256SUMS text file) and download_extract_sans_parent_verified (buffers the download, verifies SHA-256 before touching disk, then extracts). - command_selfupdate.rs: fetch SHA256SUMS before downloading the tarball and verify the hash before passing bytes to unpack_sans_parent. - juliainstaller.rs: same. - Cargo.toml: add sha2 = "0.10" and hex = "0.4". This protects against accidental corruption and against scenarios where only the binary is replaced but not the hash file (e.g. partial access to the S3 bucket). Full protection against an attacker controlling the entire distribution requires signing the SHA256SUMS file; that is a follow-up. Co-authored-by: Claude <claude@anthropic.com>
IanButterworth
force-pushed
the
ib/security
branch
from
56cace5 to
eb12188
Compare
Member
|
has conflicts |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On top of #1455
Adds SHA256 verification to the juliaup self-update and first-install paths.
Developed with Claude:
What
Before extracting the juliaup tarball during self-update or first install, fetch a
SHA256SUMSfile from the same server, look up the expected hash for the specific tarball, and bail if the digest of the downloaded bytes does not match. The archive is never written to disk if the check fails.The release CI (
package-unix) now generates the SHA256SUMS file immediately after creating the archives and uploads it to S3 alongside them.Why
The previous flow trusted HTTPS alone:
A compromise of the S3 bucket, CDN, or anything in between was sufficient to push arbitrary code to every user who ran
juliaup self updateor ran the installer. There was no cryptographic check between "bytes received from the network" and "bytes executed on disk".What changes
release.yml(package-unix): add aGenerate SHA256SUMSstep that runssha256sum juliaup-<VERSION>-*.tar.gzand writes the result tojuliaup-<VERSION>-SHA256SUMS, which is uploaded to S3 with the rest.src/operations.rs: two new#[cfg(not(windows))]functions:download_sha256sums_entry— fetches and parses the SHA256SUMS file, returning the expected hex digest for a given filename.download_extract_sans_parent_verified— buffers the full download, verifies the SHA-256 before touching disk, then callsunpack_sans_parent.src/command_selfupdate.rs: use the verified path for self-update.src/bin/juliainstaller.rs: use the verified path for first install.Cargo.toml: addsha2 = "0.10"andhex = "0.4".Scope
This only covers juliaup's own binary (self-update and first install). Julia version tarballs are not yet verified — those downloads still use
download_extract_sans_parentdirectly and would require SHA256SUMS entries in the Julia release infrastructure to verify.Limitations / follow-ups
the next step and would close this remaining gap.
JULIAUP_SERVERmirrors must host a matchingjuliaup-<VERSION>-SHA256SUMSfile alongside the tarballs, otherwise self-update and first install will fail with an error.Rollout
The first release built from this commit will generate and upload its own SHA256SUMS. Existing installations (built before this change) are unaffected until they self-update to that release; after that, all subsequent juliaup-binary updates are verified.