feat: Poseidon2 gates for Ultra arithmetisation by maramihali · Pull Request #7494 · AztecProtocol/aztec-packages

maramihali · 2024-07-16T14:32:12Z

Add Poseidon2 gates to the UltraCircuitBuilder which now ensures that recursive verifier instantiated with the Ultra arithmetisation produce the correct number of constraints.

Updates required:

change verification key length and constant proof length constants across the codebase (two selectors from the new gate whose commitments need to be in the vk and the poseidon relation becomes the one with the highest degree); changes to Prover.toml accordingly
ensure the ultra recursive verifier still stays constant size now that hashing produces gates
small modification to solidity verifer to reflect the ones in cpp with the caveat that the UltraKeccak flavor still doesnt support Poseidon gate (changes coming in a followup PR)

Tube circuit changes in # of gates (post finalisation):

number of gates prior this change, in master: 13947018
number of gates post this change: 14038982

Closes AztecProtocol/barretenberg#1041

… IVC rec vec

…n-ultra

ghost · 2024-07-19T11:05:15Z

    void set_fixed_size(uint32_t size_in) { fixed_size = size_in; }
 };

-class TranslatorArith {


This could just be moved to Translator, randomly stumbled across it

…n-ultra

ghost

Nice work. Would like to see the before and after difference in the number of gates for the ultra recursive verifier as well if possible.

also a 650k gate difference in the tube circuit seems insane if its just from hashing. That's like 10000 calls to hash()

ghost · 2024-07-22T22:20:25Z

    using Verifier = UltraVerifier_<UltraFlavor>;
    Prover tube_prover{ *builder };
+    // Print the number of gates post finalisation for a precise result
+    info("num gates in tube circuit: ", builder->get_num_gates());


Erm I guess it's good info generally when running e2e tests

note that it calls get_num_gates() and not builder.num_gates. As a sidenote, maybe we can change the name of get_num_gates() to estimate_num_finalized_gates()

ghost · 2024-07-22T22:20:53Z

@@ -1 +1 @@
-barretenberg_module(commitment_schemes_recursion commitment_schemes stdlib_primitives)
+barretenberg_module(commitment_schemes_recursion commitment_schemes stdlib_poseidon2)


why do we need stdlib poseidon and not primitives for commitment schemes?

the module doesnt link without poseidon now that it's enabled and stdlib_poseidon2 already includes stdlib_primitives

ghost · 2024-07-22T22:22:59Z

 # np.set_printoptions(formatter={'int': hex})

-EXTENDED_RELATION_LENGTH = 13
+EXTENDED_RELATION_LENGTH = 12


interesting that this decreased

A prior line of work decreased it (using different etas rather than powers of eta) but because nothing in the logic changed, that PR could just remove one value in the combiner tests and this file wasnt modified. Now I had to rerun the script and realised we have one extra eval, but my PR didnt specifically decreased this

ghost · 2024-07-22T22:23:47Z

    EXPECT_TRUE(CircuitChecker::check(*builder));
+
+    // Print the number of gates post finalisation
+    info("Recursive Verifier: num gates = ", builder->num_gates);


maybe could have both actually and mark which one is pre/post finalization

ghost · 2024-07-22T22:24:24Z

        alpha[idx] = transcript->template get_challenge<FF>("alpha_" + std::to_string(idx));
    }

-    // TODO(https://github.com/AztecProtocol/barretenberg/issues/1041): Once hashing produces constraints for Ultra in


put this in the PR description

I'm not opposed to sprinkling TODOs in the code FWIW it helps keep them top of mind (as long as they link an issue that can then be used for cleanup)

this TODO is supposed to be resolved by this PR

ghost · 2024-07-22T22:25:39Z


        bool broke(false);
        auto check_eq = [&broke](auto& p1, auto& p2) {
+            EXPECT_TRUE(p1.size() == p2.size());


also could have checked that the circuit sizes of the vks were the same, but this works.

ghost · 2024-07-22T22:30:59Z

    if (sumcheck_verified.has_value() && !sumcheck_verified.value()) {
        info("Sumcheck verification failed.");
-        return false;
+        // return false;


why is this commented out?

wopsie, leftover from debugging, thanks for spotting

…n-ultra

…hing is still not working

…n-ultra' into mm/poseidon-relation-ultra

…n-ultra

ghost

Looks good in general with some more cleanup. Also not entirely sure what exactly needs to be done to get everything working after this work.

ghost · 2024-08-20T15:18:45Z

    builder->add_recursive_proof(current_aggregation_object);

-    info("num gates in tube circuit: ", builder->get_num_gates());
+    info("num gates in tube circuit: ", builder->num_gates);


maybe say that this is num prefinalized gates in tube circuit

will remove

ghost · 2024-08-20T15:43:49Z

    stdlib::field_t b(stdlib::witness_t(&builder, fr::random_element()));
    stdlib::field_t c(&builder);
-    size_t passes = (1UL << log2_num_gates) / 4 - 4;
+    size_t passes = (1UL << log2_num_gates) / 4 - 8;


I will add a comment, the function fills a circuit up ot the desired numbers of gates for the benchmark, ensuring finalisation and the add gates to ensure non zero polys method wont cause it to go to the next power of 2 (or at least that was my deduction). Because we enabled poseidon add gates to ensure non zero polys will add more gates so i need to lower a bit the number of passes

…n-ultra' into mm/poseidon-relation-ultra

…n-ultra

enabled poseidon gates in ultra

b1518fe

maramihali changed the title ~~Poseidon in UltraHonk~~ feat: Poseidon2 in UltraHonk Jul 16, 2024

maramihali added 3 commits July 16, 2024 16:25

add missing padding

49eca12

transcript produces constraints

a5ccefa

move print statement for number of gates post finalisation for client…

1d86341

… IVC rec vec

maramihali force-pushed the mm/poseidon-relation-ultra branch from 68bf98a to 1d86341 Compare July 17, 2024 16:33

maramihali added 3 commits July 18, 2024 10:23

stuff..

bb178f0

fixing tests, fishing for remaiining bug

269da29

fix test + cleanups

f1edf5f

maramihali force-pushed the mm/poseidon-relation-ultra branch from 5304c7e to f1edf5f Compare July 19, 2024 11:04

Merge remote-tracking branch 'origin/master' into mm/poseidon-relatio…

4b98f77

…n-ultra

maramihali commented Jul 19, 2024

View reviewed changes

Comment thread barretenberg/cpp/src/barretenberg/examples/join_split/join_split.test.cpp

preset cleanup

5058fa0

maramihali changed the title ~~feat: Poseidon2 in UltraHonk~~ feat: Poseidon2 gates for Ultra arithmetisation Jul 19, 2024

fixing tests

bf37f09

maramihali self-assigned this Jul 19, 2024

maramihali added the crypto cryptography label Jul 19, 2024

lucasxia01 self-requested a review July 19, 2024 15:50

maramihali added 3 commits July 22, 2024 10:20

Merge remote-tracking branch 'origin/master' into mm/poseidon-relatio…

b116e00

…n-ultra

fix stuff

e688422

Merge remote-tracking branch 'origin/master' into mm/poseidon-relatio…

3488ac4

…n-ultra

maramihali marked this pull request as ready for review July 22, 2024 16:49

remove prints

83ecd92

maramihali force-pushed the mm/poseidon-relation-ultra branch from fe9b300 to 83ecd92 Compare July 22, 2024 17:04

lucasxia01 approved these changes Jul 22, 2024

View reviewed changes

maramihali added 2 commits July 23, 2024 09:41

Merge remote-tracking branch 'origin/master' into mm/poseidon-relatio…

2bef500

…n-ultra

uncomment line

42bae7d

maramihali mentioned this pull request Jul 23, 2024

Investigate constraint count increase caused by enabling poseidon gate for the Tube circuit AztecProtocol/barretenberg#1058

Closed

added permutation that was missing and corrected check_circuit, somet…

191b587

…hing is still not working

maramihali added 4 commits August 20, 2024 14:43

Merge branch 'master' into mm/poseidon-relation-ultra

59d79b8

more cleanup

96e9e31

Merge remote-tracking branch 'refs/remotes/origin/mm/poseidon-relatio…

b7b0248

…n-ultra' into mm/poseidon-relation-ultra

Merge remote-tracking branch 'origin/master' into mm/poseidon-relatio…

25f4264

…n-ultra

lucasxia01 approved these changes Aug 20, 2024

View reviewed changes

maramihali added 5 commits August 20, 2024 17:12

Merge branch 'master' into mm/poseidon-relation-ultra

a8f5fdb

resolve PR review comments

f6521c6

Merge remote-tracking branch 'refs/remotes/origin/mm/poseidon-relatio…

a2544bc

…n-ultra' into mm/poseidon-relation-ultra

Merge remote-tracking branch 'origin/master' into mm/poseidon-relatio…

d355e00

…n-ultra

Merge remote-tracking branch 'origin/master' into mm/poseidon-relatio…

a3a2454

…n-ultra

maramihali enabled auto-merge (squash) August 21, 2024 09:55

maramihali merged commit d86577c into master Aug 21, 2024

maramihali deleted the mm/poseidon-relation-ultra branch August 21, 2024 10:54

This was referenced Aug 21, 2024

chore(master): Release 0.50.0 #8032

Merged

feat: Sync from aztec-packages noir-lang/noir#5790

Merged

This was referenced Sep 2, 2024

Stdlib transcript for Ultra has challenges generated out of thin air AztecProtocol/barretenberg#1035

Closed

Add poseidon2 hashing for stdlib transcript using UltraCircuitBuilder AztecProtocol/barretenberg#852

Closed

noirwhal mentioned this pull request Sep 3, 2024

chore: Release Noir(0.34.0) noir-lang/noir#5692

Merged

noirwhal mentioned this pull request Sep 13, 2024

chore: Release Noir(0.35.0) noir-lang/noir#6030

Merged

noirwhal mentioned this pull request Oct 3, 2024

chore: Release Noir(0.36.0) noir-lang/noir#6213

Merged

noirwhal mentioned this pull request Oct 23, 2024

chore: Release Noir(0.37.0) noir-lang/noir#6321

Merged

noirwhal mentioned this pull request Oct 31, 2024

chore: Release Noir(0.38.0) noir-lang/noir#6422

Merged

noirwhal mentioned this pull request Nov 8, 2024

chore: Release Noir(0.39.0) noir-lang/noir#6484

Merged

noirwhal mentioned this pull request Nov 19, 2024

chore: Release Noir(1.0.0-beta.0) noir-lang/noir#6559

Closed

		@@ -1 +1 @@
		barretenberg_module(commitment_schemes_recursion commitment_schemes stdlib_primitives) No newline at end of file
		barretenberg_module(commitment_schemes_recursion commitment_schemes stdlib_poseidon2) No newline at end of file

Uh oh!

Conversation

maramihali commented Jul 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost Jul 19, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ghost left a comment • edited by lucasxia01 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ghost Jul 22, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 22, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 22, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 22, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 22, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 25, 2024 • edited by ludamad Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ghost Jul 29, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 22, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 22, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Jul 23, 2024

Choose a reason for hiding this comment

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

ghost Aug 20, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Aug 20, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Aug 20, 2024

Choose a reason for hiding this comment

Uh oh!

ghost Aug 20, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

maramihali commented Jul 16, 2024 •

edited

Loading

ghost left a comment •

edited by lucasxia01

Loading

ghost Jul 25, 2024 •

edited by ludamad

Loading