refactor: offchain reception on entity store - #24142
Merged
mverzilli merged 165 commits intoJun 26, 2026
Merged
Conversation
Adds empty(), map, filter, any, all, find and read_as to EphemeralArray (spun off from #23928) and mirrors them on TransientArray.
Both types are now aliases of a single OracleArray<T, Oracle> generic, parameterized by an ArrayOracle backend trait whose impls wrap the existing #[oracle] declarations. Foreign-call names and the TS-side services are untouched; monomorphization emits the same calls as before. empty_at stays ephemeral-only via a specialized impl. Test suites are byte-for-byte unchanged. Expansion snapshots regenerated: nargo expand now prints the resolved OracleArray paths.
Each behavior check is written once in oracle_array/test_helpers.nr, generic over the ArrayOracle backend; the ephemeral and transient test modules keep one named #[test] wrapper per behavior so counts, names and CI filters are unchanged. Backend-specific tests (empty_at, store/load/delete) stay with their backend.
Moves empty_at from the ephemeral-only impl into the shared OracleArray impl, making it available on TransientArray as well. Its doc notes that for cross-frame backends it wipes data other frames may have written. The empty_at test moves to the shared suite and runs on both backends.
Generic store/load/delete cores live in oracle_array; ephemeral and transient expose them via thin module-level wrappers, so both backends now have identical public surfaces (alias + oracle marker + store/load/delete). The store/load test bodies move to the shared suite; each backend keeps a store_and_load smoke test through its own wrappers. Also fixes the stale test_helpers doc that still described empty_at and store/load as backend-specific.
Backend test modules no longer list one wrapper per shared check. Annotating a test module with ephemeral_oracle_array_tests / transient_oracle_array_tests reflects over test_helpers and emits one #[test] per unconstrained check with that backend's oracle swapped in, so new checks automatically run against every backend and the suites cannot drift. should_fail_with messages come from a single manifest in test_helpers; an unregistered failing check fails loudly rather than silently. Generated test names match the previous hand-written ones.
Replaces the two per-backend attribute functions with one oracle_array_tests(module, oracle) attribute that takes the backend's oracle as a quoted path at the annotation site.
…d-transient-array-apis
The module name collided with crate::ephemeral, which owns the EphemeralArray type. Name the oracle module after what it holds -- the #[oracle(...)] declarations -- so the array module keeps the plain ephemeral name and the two are unambiguous at use sites.
The module name collided with crate::transient, which owns the TransientArray type. Name the oracle module after what it holds -- the #[oracle(...)] declarations -- so the array module keeps the plain transient name and the two are unambiguous at use sites.
OracleArray and ArrayOracle were near-anagrams, forcing readers to disambiguate which was the struct and which the backend trait. Keep ArrayOracle (it names the set of oracles that implement an array) and rename the struct to UnconstrainedArray, after the property every backend shares: its operations are unconstrained foreign calls into PXE-side storage, so the data is host-provided and must be verified before being trusted in constrained code. The oracle_array module, the ArrayOracle trait, and the Oracle type parameter are unchanged.
Follows the struct rename: the module and its test macro were still named after the old OracleArray struct. Rename the module directory to unconstrained_array and the oracle_array_tests macro to unconstrained_array_tests so the module mirrors the type it houses. The ArrayOracle trait and the store/load/delete helpers keep their names.
…y rename UnconstrainedArray is longer than the old OracleArray, so several doc lines that referenced the type (or its method links) tipped past the 120-column limit. nargo fmt breaks an over-long comment line by pushing only the trailing word onto a new line, which left mid-paragraph orphans like '/// not' and '/// the'. Rejoin those words and rewrap the affected paragraphs at natural points, keeping every line within 120 columns.
…ntOracles Each of these names a set of oracle operations -- the full backend an array needs (ArrayOracles) and the concrete ephemeral/transient bundles that implement it -- so the plural reads truer and matches the ephemeral_oracles/transient_oracles module names. The singular Oracle remains as the type parameter: one backend slot filled by one bundle.
The explanatory paragraph on the ArrayOracles trait was dropped during the previous pluralize commit (a formatter re-stage race on the shared working tree); the rename itself only swapped the identifier. Restore the paragraph verbatim. nargo fmt run directly keeps it intact.
to EphemeralArray
…d-transient-array-apis
…suleArray for_each iterated in reverse so the callback could remove the current element without shifting unvisited ones, but nothing in production uses that capability and backward iteration is surprising both relative to general intuition and to the sibling methods (map/filter/find), which all read elements in order. Iterate forward instead, document that structural mutation from inside the callback is unsupported, and drop the tests that existed solely to exercise remove-during-iteration. The order test now asserts the exact forward visit order. Linear: F-729
The manual index loop in get_pending_partial_notes_completion_logs existed only because for_each had arbitrary iteration order. Now that for_each visits elements in order, the loop can use it directly while preserving the index alignment between the request array and the partial note array. Linear: F-729
OffchainMessage, OFFCHAIN_RECEPTION_TYPE and fact_collection_id (now a free function rather than a method) move into mod.nr; the near-empty message.nr and its round-trip test are removed.
mverzilli
commented
Jun 23, 2026
Comment on lines
+204
to
+226
| /// A message delivered via the `offchain_receive` utility function. | ||
| #[derive(Serialize, Deserialize)] | ||
| pub struct OffchainMessage { | ||
| /// The encrypted message payload. | ||
| pub ciphertext: BoundedVec<Field, MESSAGE_CIPHERTEXT_LEN>, | ||
| /// The intended recipient of the message. | ||
| pub recipient: AztecAddress, | ||
| /// The hash of the transaction that produced this message. `Option::none` indicates a tx-less message. | ||
| pub tx_hash: Option<Field>, | ||
| /// Anchor block timestamp at message emission. | ||
| pub anchor_block_timestamp: u64, | ||
| } | ||
|
|
||
| /// Fact-collection type id shared by every offchain message reception in the fact store. | ||
| global OFFCHAIN_RECEPTION_TYPE_ID: Field = sha256_to_field("AZTEC_NR::OFFCHAIN_RECEPTION_TYPE_ID".as_bytes()); | ||
|
|
||
| /// Computes the fact-collection id that identifies a message's reception workflow in the fact store. | ||
| /// | ||
| /// The id is a hash over the full serialized message. Re-delivering a byte-identical message therefore is idempotent, | ||
| /// and a message differing in any of its fields triggers a completely new reception workflow. | ||
| fn fact_collection_id(message: OffchainMessage) -> Field { | ||
| poseidon2_hash(message.serialize()) | ||
| } |
Contributor
Author
There was a problem hiding this comment.
Note: this is a bit of weird placement for these lines in the file. I chose to put them here so as not to scramble the previous review comments
Contributor
Author
There was a problem hiding this comment.
(I can move it upwards once we're done reviewing)
Collaborator
Flakey Tests🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry. |
nventuro
approved these changes
Jun 25, 2026
nventuro
left a comment
nventuro
left a comment
Contributor
There was a problem hiding this comment.
So much better. My only real question is re. state vs transitions for the fsms.
| })); | ||
|
|
||
| let processable_messages: EphemeralArray<OffchainMessageWithContext> = EphemeralArray::empty(); | ||
| let now = UtilityContext::new().timestamp(); |
Contributor
There was a problem hiding this comment.
Bit nasty to construct a context like this. Should this fn take a utility context instead?
Base automatically changed from
martin/expose-entity-store-oracles
to
merge-train/fairies-v5
June 26, 2026 14:51
mverzilli
enabled auto-merge (squash)
June 26, 2026 16:45
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-implementation of offchain reception workflows using the new EntityStore machinery
Note this exposed a problem in the EntityStore idempotency semantics: throwing on an attempt at creating an already existing entity is problematic, because Noir doesn't provide error handling mechanisms. In the end, I decided to make entity creation idempotent, with the first attempt at creating a given entity "winning" to avoid inadvertent overwrites. It is then user responsibility to model their workflows properly (there's plenty of alternatives: change how entity id's are derived, check for existence before attempting creation, using facts, etc).
Closes F-682