feat(platform)!: rules that read the referenced document (PV14)

book/src/data-model/documents.md

@@ -311,6 +311,8 @@ A `permanentDocument`, `moderatedDocument` or `deletableDocument` reference may
 
 reads: the proposal this join request names must be owned by its `recipientId`. A document found that fails an entry refuses the write, paid, with `ReferencedDocumentPropertyMismatchError` (40127); no document found is `ReferencedEntityNotFoundError` (40120). At registration both sides must exist, be single values and share one value kind, and the referenced side may not be transient (`ReferencedDocumentPropertyAgreementInvalidError`, 40126); a `where` holding `"."` or a function is refused by the parser, since both find the document and belong in `findBy`. In Rust the parser holds `where` as `property_agreement`, keyed by the referring property (`{referring property: referenced property}`), so the parsed model reads the opposite way round from the schema.
 
+An entry may instead hold an object with one key (protocol version 14): `{ "const": v }`, a literal the referenced side must equal; `{ "notEqual": b }`, a referring property, `"$ownerId"` or a `const` it must differ from; `{ "in": [...] }`, two to thirty literals it must be one of; or an ordering (`greaterThan`, `greaterThanOrEqual`, `lessThan`, `lessThanOrEqual`) of an integer by an integer property of the declaring type, an integer `const` or a time or height the declaring type records. A literal reads as the referenced property's type says (`WhereLiteralKind::of_referenced`): base58 for an identifier, hex for a byte array, at registration and when a document is written alike, since the type may be another contract's. The parser holds these as `where_conditions` (`{referenced property: WhereCondition}`), a field beside `property_agreement` on every document reference variant and on `ListElementReference`, and `DocumentReferenceDeclaration` carries both. They are checked with the equalities, against the same document, and refuse a write with 40127 naming the condition as written; an entry bounded by the writer or by `$updatedAt` or `$transferredAt` (or their heights) asks every replace about the reference (`DocumentReferenceDeclaration::where_is_rechecked_on_every_replace`), as a writer gate does. Registration checks them in contract create and update state validation with the equalities (40126).
+
 Read as SQL, a document reference is a query that must return a row: `documentType` is the `FROM`, `findBy` the part of the `WHERE` that finds the row through a unique index (without it, `$id = value`), `where` the rest of the `WHERE`, checked on the row found, `minimumAgeBlocks` an age condition and `consume` a delete of the row found.
 
 The spellings a protocol version 14 beta used, `lookup` (which named the index and mapped its properties under `keys`), `propertyAgreement` (pairs keyed by the referring property) and the type `listElement`, were replaced by `findBy`, `where` and `inList` on a `permanentDocument` before the version shipped. Meta-schema v3 does not admit them, and the parser refuses them on every parse (`refuse_replaced_reference_keywords`, `InvalidContractStructure`, 10231) with a message naming what replaced each, so a contract written with them never loads with another meaning.
@@ -865,6 +867,7 @@ Conditions nest: `{ "not": { "allOf": [{ "equal": ["price", 0] }, { "greaterThan
 - `{ "countPresent": [path, path, ...] }`, how many of two or more properties, no two alike, the document holds, each as `present` tests it (any type, null and an object with no member present counting as absent), so a rule bounds how many of a group are set: `{ "equal": [{ "countPresent": ["email", "phone", "handle"] }, 1] }` asks for exactly one, `{ "lessThanOrEqual": [..., 1] }` for at most one, and `{ "in": [..., [1, 2]] }` for one or two. It is one node plus one per path, where the same bound written with `present` takes an `anyOf` and a `not` of every pair;
 - a byte: `{ "byteAt": [path, index] }`, the byte, 0 to 255, at a literal index from 0 to 65535 of a byte array property, below its `maxItems` when it declares one. A byte the array does not hold, or one of an array the document leaves out, reads as 0, so `{ "in": [{ "byteAt": ["corePaymentAddress", 0] }, [0, 1]] }` holds a DIP-33 address to the P2PKH or P2SH type and lets a document without one through; `present` or `count` tells a 0 byte from a byte that is not there;
 - a system time or height: `"$createdAt"`, `"$updatedAt"` and `"$transferredAt"`, block times in milliseconds, and each with `BlockHeight` or `CoreBlockHeight` appended, the Platform and Core block heights: those of the document's creation, of its last create, replace or price update, and of its last create, transfer or purchase. A rule may read one only on a type that records it by listing it in `required`, so every stored document holds it; it takes no `ifAbsent`, and an indexOnly type, whose deletes carry none, reads none. `{ "lessThanOrEqual": [{ "subtract": ["endsAt", "$createdAt"] }, 604800000] }` keeps a listing to a week from its creation, and `{ "lessThanOrEqual": ["$updatedAt", "endsAt"] }` refuses a replace or a price update after it ends;
+- a value of the document a reference points at: `{ "referenced": [reference, field] }`, the `field` (a property path of the referenced type, its `$id`, `$ownerId` or `$creatorId`, or a time or height it records) of the document the top-level property `reference` finds through its `refersTo`, read as a path is, standing wherever a path stands. The reference holds one document, found by id or by a `findBy` without a function; consensus judges a rule reading one once the references hold, from the documents their checks read (see below);
 - a total read from state: `{ "countOf": [type] }` or `{ "countOf": [type, filter] }`, how many documents of a type of the same contract there are, or how many match the filter, and `{ "sumOf": [type, property] }` or `{ "sumOf": [type, property, filter] }`, the total of an integer property over them. A filter maps each key, a property of the counted type or `$ownerId`, to the value it must take, read from the document being written: one of its properties, `$ownerId`, `$id` (its id, so `{ "countOf": ["vote", { "pollId": "$id" }] }` counts the votes naming it), an integer or a `{ "const": ... }`. The total is the one a count or sum tree keeps, as it will be once the write is done (the document itself counted when the type is its own, at its new values, and moved to its new owner by a transfer or purchase), so `{ "lessThanOrEqual": [{ "countOf": ["listing", { "$ownerId": "$ownerId" }] }, 10] }` on `listing` keeps every owner at ten listings or fewer. A whole-type total needs `documentsCountable` or `documentsSummable`, and a filtered one an index of the counted type whose properties are exactly the filter's keys, countable or summing the property. A rule is judged only when its own type is written, so a fact about another type can go stale after the write.
 
 A JSON number is always a value and a string always a path, so a property named `100` is not confused with the number, and the rule is a tree the meta-schema can check rather than a string with precedence rules to parse. Consensus holds nothing but this tree; an SDK may offer an infix spelling that compiles to it.
@@ -877,6 +880,8 @@ The parser (generation 3, meta-schema v3) checks the keyword on every parse, sto
 
 Enforcement lives in `DataContract::validate_document_properties` (generation 0, extended in place: the call is inert before protocol version 14, where `validate_property_constraints` is `None`), after the schema validation. Document create and replace structure validation call it, so consensus applies the rules, and so does every client that validates a document before sending it. The rules are checked in name order against the document's properties, which for a replace is the whole document, and the first one broken fails with `DocumentPropertyConstraintViolatedError` (basic code 10422), naming the document type, the rule and why: the rule does not hold, or evaluating it overflowed, divided by zero, raised to a negative power or read a value that is not an integer. The check changes nothing stored. It reads state only for the `countOf` and `sumOf` totals, which the document batch transformer reads when it builds the action (`Drive::fetch_property_constraint_aggregate`, billed with the write) and hands the check in `DocumentSystemValues::aggregates`; the limits bound its cost. `$ownerId` and the system times and heights read the values of the document version being written (`validate_document_properties` takes them as `DocumentSystemValues`): on a create the writer and the block's time and heights, on a replace the writer, the stored creation and transfer values and the block's as the update. A client passes what it knows: an owner it does not know equals no identifier, and a rule reading a time, a height or a total it is not given is not judged. Consensus reads every total the rules it judges read (`DocumentSystemValues::aggregates` is `Some`), so one missing there is an error in the code building the write, never a skipped rule. The SDK pre-checks use the device clock for the times a write records, and leave a rule reading a block height unjudged, since the height is unknown until the block. Transfers and purchases change no property, only the owner and the transfer's time and heights, so only the rules reading those are judged again, with the new values (`DocumentTypeV0Methods::validate_property_constraints_for_system_change`, next to the `distinctFrom` check). Price updates change only the update's time and heights, and are judged against the rules reading those the same way.
 
+A rule reading a referenced document (protocol version 14) is not judged there: its values are read by the reference check, which runs in document create state validation 2 and replace state validation 1, after the structure validation. The parser turns a `referenced` operand into the path `$referenced.<reference>.<field>` (`REFERENCED_DOCUMENT_PREFIX`), so the evaluation reads it as any path; `DocumentSystemValues::referenced_documents` says whether the data holds those values, and `PropertyConstraint::violation` leaves a rule reading one unjudged without them, which is how the structure validation, transfers, purchases, price updates, moderator restores and changes of fields, and every SDK pre-check pass over it. The reference validation (`validate_document_references`, generation 0, extended in place) checks every reference a rule reads through on every replace (`referenced_fields` lists them), and, once each holds, records the values the rules read of the document it found, or of the removal record a kept `moderatedDocument` reference resolves to (`referenced_document_values`); the state validation then judges those rules alone (`DocumentTypeV0Methods::validate_referencing_property_constraints`) against the document's data with the values under `$referenced` (`data_with_referenced_documents`). No read is added: a create reads each referenced document once, as the existence check always did, and a replace makes the check's read again, billed the same way. Registration checks the declaring side when the type is parsed (`validate_referencing_rules`) and what a rule reads of a referenced type with that type in hand (`referenced_reads_error`): in the contract parse for a type of the same contract (10231), in contract create and update state validation for another contract's (`ReferencedDocumentRuleReadInvalidError`, 40148).
+
 In Rust the rules are `DocumentTypeV2Getters::property_constraints` (a map from name to `PropertyConstraint`: a comparison, an `in`, a string comparison (`TextCompare`, `TextIn`), a `present` or `absent`, or an `anyOf`, `allOf` or `not` of them, empty on types that predate the keyword; `property_reads` lists what a rule reads and how: by value, by presence, by size (`Length`, `Count`) or in a comparison of strings or identifiers, `system_reads` the system times and heights it reads, and `aggregate_reads` the totals, each an `AggregateRead`), each rule's `holds` and `violation` evaluate it against a document's data and `DocumentSystemValues`, and the document check is `DocumentTypeV0Methods::validate_property_constraints`.
 
 ### Delete rules (`deleteConstraints`)

book/src/error-handling/error-codes.md

@@ -107,7 +107,7 @@ The fee category currently has a single code. The 30000 range is reserved for fu
 | Range | Category | Examples |
 |-------|----------|----------|
 | 40000-40009 | Data Contract | `DataContractAlreadyPresentError` (40000), `DataContractIsReadonlyError` (40001), `DataContractNotFoundError` (40008) |
-| 40100-40147 | Documents | `DocumentAlreadyPresentError` (40100), `DocumentNotFoundError` (40101), `DuplicateUniqueIndexError` (40105), `DocumentActionFeeAgreementNotSetError` (40132), `DocumentActionFeeAgreementMismatchError` (40133), `DocumentActionFeeMultiplierNotToleratedError` (40134), `DocumentActionFeeModeratorsShareMismatchError` (40139), `DocumentExpiredError` (40140), `DocumentContestMaximumContendersReachedError` (40141), `ReferencedDocumentRequirementNotMetError` (40142), `ReferencedDocumentTypeNotModeratedError` (40143), `ReferencedDocumentTypeModeratedError` (40144), `ReferencedDocumentRemovedError` (40145), `ReferencedDocumentTypeIndexOnlyError` (40146), `DocumentDeleteConstraintViolatedError` (40147) |
+| 40100-40148 | Documents | `DocumentAlreadyPresentError` (40100), `DocumentNotFoundError` (40101), `DuplicateUniqueIndexError` (40105), `DocumentActionFeeAgreementNotSetError` (40132), `DocumentActionFeeAgreementMismatchError` (40133), `DocumentActionFeeMultiplierNotToleratedError` (40134), `DocumentActionFeeModeratorsShareMismatchError` (40139), `DocumentExpiredError` (40140), `DocumentContestMaximumContendersReachedError` (40141), `ReferencedDocumentRequirementNotMetError` (40142), `ReferencedDocumentTypeNotModeratedError` (40143), `ReferencedDocumentTypeModeratedError` (40144), `ReferencedDocumentRemovedError` (40145), `ReferencedDocumentTypeIndexOnlyError` (40146), `DocumentDeleteConstraintViolatedError` (40147), `ReferencedDocumentRuleReadInvalidError` (40148) |
 | 40200-40217 | Identity | `IdentityAlreadyExistsError` (40200), `InvalidIdentityRevisionError` (40203), `IdentityInsufficientBalanceError` (40210) |
 | 40300-40307 | Voting | `MasternodeNotFoundError` (40300), `MasternodeVoteAlreadyPresentError` (40304), `VoteChoiceNotAllowedForVotePollError` (40307) |
 | 40400-40401 | Prefunded Balances | `PrefundedSpecializedBalanceInsufficientError` (40400) |

packages/js-evo-sdk/README.md

@@ -206,6 +206,19 @@ for (const ref of contract.documentTypeReferences('note')) {
   // the writer's own `$ownerId`: a write gate such as
   // `{ '$ownerId': '$ownerId' }` lets only the referenced document's current
   // owner create or replace the referring document.
+  // A value may instead be a condition object with one key, as the schema
+  // writes it: `{ const: v }`, `{ notEqual: b }`, `{ in: [v1, v2] }`, or an
+  // ordering such as `{ lessThanOrEqual: 'amount' }` or
+  // `{ greaterThan: '$updatedAt' }` (a bound is a referring property,
+  // `'$ownerId'`, a time or height, or `{ const: v }`; an integer past
+  // Number.MAX_SAFE_INTEGER is a bigint). Tell the two apart by type:
+  for (const [referenced, value] of Object.entries(ref.where ?? {})) {
+    if (typeof value === 'string') {
+      // an equality with the referring property `value`
+    } else {
+      // a condition, e.g. { in: ['open', 'extended'] }
+    }
+  }
   console.log(ref.path, ref.type);
 }
 
@@ -467,13 +480,13 @@ try {
 }
 ```
 
-To find a broken rule before paying for a refused transition, a contract lists a document type's rules and checks a document against them with the code consensus runs. The check covers the rules alone, not the JSON schema. It reads the document's owner for `$ownerId`, and the device clock for the times the write will record (`readsSystem` lists the ones a rule reads); a rule reading a block height is not checked, since the height is unknown until the block, and neither is a rule reading a `countOf` or `sumOf` total, which only the platform reads from state (`readsTotals` lists the ones a rule reads, each with its `kind`, `documentType`, the summed `property` of a `sumOf` and the `filter` keys):
+To find a broken rule before paying for a refused transition, a contract lists a document type's rules and checks a document against them with the code consensus runs. The check covers the rules alone, not the JSON schema. It reads the document's owner for `$ownerId`, and the device clock for the times the write will record (`readsSystem` lists the ones a rule reads); a rule reading a block height is not checked, since the height is unknown until the block, and neither is a rule reading a `countOf` or `sumOf` total, which only the platform reads from state (`readsTotals` lists the ones a rule reads, each with its `kind`, `documentType`, the summed `property` of a `sumOf` and the `filter` keys), nor one reading a value of the document a reference points at, `{ referenced: [reference, field] }`, which the platform reads when it checks the reference (`readsReferenced` lists them, each with its `reference`, `path` and `kind`):
 
 ```ts
 contract.documentTypePropertyConstraints('offer');
 // [{ name: 'discountBelowPrice', rule: { lessThan: ['discount', 'price'] },
 //    reads: [{ path: 'discount', kind: 'value' }, { path: 'price', kind: 'value' }],
-//    readsOwner: false, readsSystem: [], readsTotals: [] }, ...]
+//    readsOwner: false, readsSystem: [], readsTotals: [], readsReferenced: [] }, ...]
 
 const broken = contract.checkDocumentPropertyConstraints(document);
 if (broken) {

packages/kotlin-sdk/KotlinExampleApp/app/src/main/java/org/dashfoundation/example/ui/contracts/DocumentTypeDetailsScreen.kt

@@ -404,8 +404,8 @@ private fun PropertyConstraintsFormSection(section: PropertyConstraintsSection)
 
 /**
  * One `propertyConstraints` rule: its name, the rule as declared, what it
- * reads, the system times and heights and the totals it reads, and whether an owner change
- * is judged against it too
+ * reads, the system times and heights, the totals and the values of referenced documents it
+ * reads, and whether an owner change is judged against it too
  * (← `PropertyConstraintRowView` in DocumentTypeDetailsView.swift).
  */
 @Composable
@@ -414,6 +414,7 @@ private fun PropertyConstraintRow(rule: DocumentPropertyConstraint) {
     val reads = remember(rule) { propertyConstraintReadsText(rule) }
     val systemReads = remember(rule) { propertyConstraintSystemReadsText(rule) }
     val totals = remember(rule) { propertyConstraintTotalsText(rule) }
+    val referenced = remember(rule) { propertyConstraintReferencedReadsText(rule) }
     Column(
         modifier = Modifier
             .fillMaxWidth()
@@ -475,6 +476,19 @@ private fun PropertyConstraintRow(rule: DocumentPropertyConstraint) {
                 color = MaterialTheme.colorScheme.tertiary,
             )
         }
+        if (referenced != null) {
+            Text(
+                referenced,
+                style = MaterialTheme.typography.labelSmall,
+                color = MaterialTheme.colorScheme.tertiary,
+                modifier = Modifier.testTag("documentType.propertyConstraint.${rule.name}.readsReferenced"),
+            )
+            Text(
+                PROPERTY_CONSTRAINT_REFERENCED_NOTE,
+                style = MaterialTheme.typography.labelSmall,
+                color = MaterialTheme.colorScheme.tertiary,
+            )
+        }
         if (rule.readsOwner) {
             Text(
                 "Reads \$ownerId, the document's owner: transfers and purchases are judged " +