From 4b4e89d496ce86d15401c8c3121723674aebc740 Mon Sep 17 00:00:00 2001 From: Marcos Nicolau Date: Thu, 5 Jun 2025 13:53:50 -0300 Subject: [PATCH 1/5] docs: benchmark results --- aggregation_mode/benchmarks.md | 55 ++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 aggregation_mode/benchmarks.md diff --git a/aggregation_mode/benchmarks.md b/aggregation_mode/benchmarks.md new file mode 100644 index 0000000000..cdff01f476 --- /dev/null +++ b/aggregation_mode/benchmarks.md @@ -0,0 +1,55 @@ +# Aggregation Mode Benchmark Results + +## Machine Specifications + +- **GPU**: NVIDIA RTX 3090 (24GB VRAM) +- **RAM**: 32GB +- **CPU**: AMD EPYC 7443 (24-Core) +- **Operating System**: Ubuntu + +## Benchmark Setup + +- **Total Proofs**: 3968 +- **Proofs per Chunk**: 128 +- **Total Chunks**: 32 + +--- + +## RISC Zero (Risc0) + +- **Start Time**: 14:28:36 +- **Aggregation Start**: 14:33:32 +- **End Time**: 16:46:28 + +**Performance Summary:** + +- **Verification Time (Start to Aggregation Start)**: **4 minutes 56 seconds** +- **Aggregation Time (Agg to End)**: **132.93 minutes** +- **Total Time (Start to End)**: **137.87 minutes** + +--- + +## SP1 + +- **Start Time**: 21:52:55 +- **Aggregation Start**: 22:26:40 +- **End Time**: 23:24:42 + +**Performance Summary:** + +- **Verification Time (Start to Aggregation Start)**: **33 minutes 45 seconds** +- **Aggregation Time (Agg to End)**: **31 minutes 47 seconds** +- **Total Time (Start to End)**: **92 minutes** + +--- + +## Comparison Summary + +| | Risc0 | SP1 | +| ----------------- | ------------ | ------------- | +| Verification Time | 4 min 56 sec | 33 min 45 sec | +| Aggregation Time | 132.93 min | 31 min 47 sec | +| Total Time | 137.87 min | 92 min | + +> **Note**: +> SP1 longer verification time is due to the big overhead in setting up the prover client to verify the proof when it is fetched. From 9490faaec5fc13d4746e3f5396a36eac389c38b1 Mon Sep 17 00:00:00 2001 From: Marcos Nicolau Date: Thu, 5 Jun 2025 13:58:47 -0300 Subject: [PATCH 2/5] docs: setup notes --- aggregation_mode/benchmarks.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/aggregation_mode/benchmarks.md b/aggregation_mode/benchmarks.md index cdff01f476..7e95c4da5d 100644 --- a/aggregation_mode/benchmarks.md +++ b/aggregation_mode/benchmarks.md @@ -13,6 +13,11 @@ - **Proofs per Chunk**: 128 - **Total Chunks**: 32 +### Notes + +- The total number of proofs (3968) is the **maximum that can be aggregated in a single run**, limited by blob capacity. +- Increasing the **proofs per chunk** generally improves performance, but requires **more powerful hardware** to avoid out of memory. + --- ## RISC Zero (Risc0) From 9f5ea707c7e4d9ace913a67a543533b20bcbbc28 Mon Sep 17 00:00:00 2001 From: Marcos Nicolau Date: Thu, 5 Jun 2025 14:08:36 -0300 Subject: [PATCH 3/5] docs: reproduce steps --- aggregation_mode/benchmarks.md | 49 ++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/aggregation_mode/benchmarks.md b/aggregation_mode/benchmarks.md index 7e95c4da5d..ef9dc3f3da 100644 --- a/aggregation_mode/benchmarks.md +++ b/aggregation_mode/benchmarks.md @@ -17,6 +17,55 @@ - The total number of proofs (3968) is the **maximum that can be aggregated in a single run**, limited by blob capacity. - Increasing the **proofs per chunk** generally improves performance, but requires **more powerful hardware** to avoid out of memory. +- In this benches the Aligned infrastructure was setup locally in the same machine using `ethereum-package`. + +## Reproduce it + +The step by step to run the benchmarks: + +1. Deploy aligned infrastructure locally with `ethereum-package`. +2. Fund a wallet on aligned, for example with rich account number 7: + +```shell +cargo run --release -- deposit-to-batcher \ + --network devnet \ + --private_key 0x4bbbf85ce3377467afe5d46f804f221813b2bb87f24d81f60f1fcdbf7cbf4356 \ + --amount 1ether +``` + +3. Send `3968` fibonacci proofs for `Risc0` and `Sp1`: + +```shell +cd batcher/aligned + +## Send SP1 Proofs +cargo run --release -- submit \ + --proving_system SP1 \ + --proof ../../scripts/test_files/sp1/sp1_fibonacci_5_0_0.proof \ + --vm_program ../../scripts/test_files/sp1/sp1_fibonacci_5_0_0.elf \ + --repetitions 1000 \ + --private_key 0x4bbbf85ce3377467afe5d46f804f221813b2bb87f24d81f60f1fcdbf7cbf4356 \ + --instant_fee_estimate \ + --network devnet \ + --random_address + +## Send Risc0 Proofs + cargo run --release -- submit \ + --proving_system Risc0 \ + --proof ../../scripts/test_files/risc_zero/fibonacci_proof_generator/risc_zero_fibonacci_2_0.proof \ - + --vm_program ../../scripts/test_files/risc_zero/fibonacci_proof_generator/fibonacci_id_2_0.bin \ + --public_input ../../scripts/test_files/risc_zero/fibonacci_proof_generator/risc_zero_fibonacci_2_0.pub \ + --repetitions 3968 \ + --private_key 0x4bbbf85ce3377467afe5d46f804f221813b2bb87f24d81f60f1fcdbf7cbf4356 +``` + +4. Modify the `proofs_per_chunk` in `config-files/config-proof-aggregator-ethereum-package.yaml` to `128`. +5. Run the aggregator with time: + +```shell +time make start_proof_aggregator_gpu_ethereum_package AGGREGATOR=sp1 +time make start_proof_aggregator_gpu_ethereum_package AGGREGATOR=risc0 +``` --- From ef650857021bf13442c7ca6f9989a4a714e8407d Mon Sep 17 00:00:00 2001 From: Marcos Nicolau <76252340+MarcosNicolau@users.noreply.github.com> Date: Fri, 6 Jun 2025 16:23:36 -0300 Subject: [PATCH 4/5] Update benchmarks.md --- aggregation_mode/benchmarks.md | 40 ++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/aggregation_mode/benchmarks.md b/aggregation_mode/benchmarks.md index ef9dc3f3da..ea7acc3448 100644 --- a/aggregation_mode/benchmarks.md +++ b/aggregation_mode/benchmarks.md @@ -4,7 +4,7 @@ - **GPU**: NVIDIA RTX 3090 (24GB VRAM) - **RAM**: 32GB -- **CPU**: AMD EPYC 7443 (24-Core) +- **CPU**: AMD EPYC 7443 (16-Core) - **Operating System**: Ubuntu ## Benchmark Setup @@ -23,11 +23,15 @@ The step by step to run the benchmarks: -1. Deploy aligned infrastructure locally with `ethereum-package`. +1. Deploy aligned infrastructure locally with `ethereum-package`, see the [guide here](https://github.com/yetanotherco/aligned_layer/blob/testnet/docs/0_internal/ethereum_package.md). 2. Fund a wallet on aligned, for example with rich account number 7: +```shell +# Install aligned cli +make aligned_install_compiling +``` ```shell -cargo run --release -- deposit-to-batcher \ +aligned deposit-to-batcher \ --network devnet \ --private_key 0x4bbbf85ce3377467afe5d46f804f221813b2bb87f24d81f60f1fcdbf7cbf4356 \ --amount 1ether @@ -36,25 +40,23 @@ cargo run --release -- deposit-to-batcher \ 3. Send `3968` fibonacci proofs for `Risc0` and `Sp1`: ```shell -cd batcher/aligned - ## Send SP1 Proofs -cargo run --release -- submit \ +aligned submit \ --proving_system SP1 \ - --proof ../../scripts/test_files/sp1/sp1_fibonacci_5_0_0.proof \ - --vm_program ../../scripts/test_files/sp1/sp1_fibonacci_5_0_0.elf \ - --repetitions 1000 \ + --proof scripts/test_files/sp1/sp1_fibonacci_5_0_0.proof \ + --vm_program scripts/test_files/sp1/sp1_fibonacci_5_0_0.elf \ + --repetitions 3968 \ --private_key 0x4bbbf85ce3377467afe5d46f804f221813b2bb87f24d81f60f1fcdbf7cbf4356 \ --instant_fee_estimate \ --network devnet \ --random_address ## Send Risc0 Proofs - cargo run --release -- submit \ +aligned submit \ --proving_system Risc0 \ - --proof ../../scripts/test_files/risc_zero/fibonacci_proof_generator/risc_zero_fibonacci_2_0.proof \ - - --vm_program ../../scripts/test_files/risc_zero/fibonacci_proof_generator/fibonacci_id_2_0.bin \ - --public_input ../../scripts/test_files/risc_zero/fibonacci_proof_generator/risc_zero_fibonacci_2_0.pub \ + --proof scripts/test_files/risc_zero/fibonacci_proof_generator/risc_zero_fibonacci_2_0.proof \ + --vm_program scripts/test_files/risc_zero/fibonacci_proof_generator/fibonacci_id_2_0.bin \ + --public_input scripts/test_files/risc_zero/fibonacci_proof_generator/risc_zero_fibonacci_2_0.pub \ --repetitions 3968 \ --private_key 0x4bbbf85ce3377467afe5d46f804f221813b2bb87f24d81f60f1fcdbf7cbf4356 ``` @@ -77,9 +79,9 @@ time make start_proof_aggregator_gpu_ethereum_package AGGREGATOR=risc0 **Performance Summary:** -- **Verification Time (Start to Aggregation Start)**: **4 minutes 56 seconds** -- **Aggregation Time (Agg to End)**: **132.93 minutes** -- **Total Time (Start to End)**: **137.87 minutes** +- **Verification Time**: **4 minutes 56 seconds** +- **Aggregation Time**: **132.93 minutes** +- **Total Time**: **137.87 minutes** --- @@ -91,9 +93,9 @@ time make start_proof_aggregator_gpu_ethereum_package AGGREGATOR=risc0 **Performance Summary:** -- **Verification Time (Start to Aggregation Start)**: **33 minutes 45 seconds** -- **Aggregation Time (Agg to End)**: **31 minutes 47 seconds** -- **Total Time (Start to End)**: **92 minutes** +- **Verification Time**: **33 minutes 45 seconds** +- **Aggregation Time**: **31 minutes 47 seconds** +- **Total Time**: **92 minutes** --- From 8e0137c888c7996a2b0e139e7d8a16717112f7ee Mon Sep 17 00:00:00 2001 From: Marcos Nicolau Date: Wed, 11 Jun 2025 11:53:18 -0300 Subject: [PATCH 5/5] docs: include time per proof --- aggregation_mode/benchmarks.md | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/aggregation_mode/benchmarks.md b/aggregation_mode/benchmarks.md index ea7acc3448..79d002a45e 100644 --- a/aggregation_mode/benchmarks.md +++ b/aggregation_mode/benchmarks.md @@ -25,6 +25,7 @@ The step by step to run the benchmarks: 1. Deploy aligned infrastructure locally with `ethereum-package`, see the [guide here](https://github.com/yetanotherco/aligned_layer/blob/testnet/docs/0_internal/ethereum_package.md). 2. Fund a wallet on aligned, for example with rich account number 7: + ```shell # Install aligned cli make aligned_install_compiling @@ -82,6 +83,8 @@ time make start_proof_aggregator_gpu_ethereum_package AGGREGATOR=risc0 - **Verification Time**: **4 minutes 56 seconds** - **Aggregation Time**: **132.93 minutes** - **Total Time**: **137.87 minutes** +- **Aggregation time per proof**: **2 seconds** +- **Total time per proof**: **2,1 seconds** --- @@ -96,16 +99,20 @@ time make start_proof_aggregator_gpu_ethereum_package AGGREGATOR=risc0 - **Verification Time**: **33 minutes 45 seconds** - **Aggregation Time**: **31 minutes 47 seconds** - **Total Time**: **92 minutes** +- **Aggregation time per proof**: **0,5 seconds** +- **Total time per proof**: **1,4 seconds** --- ## Comparison Summary -| | Risc0 | SP1 | -| ----------------- | ------------ | ------------- | -| Verification Time | 4 min 56 sec | 33 min 45 sec | -| Aggregation Time | 132.93 min | 31 min 47 sec | -| Total Time | 137.87 min | 92 min | +| | Risc0 | SP1 | +| -------------------------- | ------------ | ------------- | +| Verification Time | 4 min 56 sec | 33 min 45 sec | +| Aggregation Time | 132.93 min | 31 min 47 sec | +| Total Time | 137.87 min | 92 min | +| Aggregation Time per Proof | 2 seconds | 0,5 seconds | +| Total Time per Proof | 2,1 seconds | 1,4 seconds | -> **Note**: +> **Note**: > SP1 longer verification time is due to the big overhead in setting up the prover client to verify the proof when it is fetched.