Skip to content

Persist DashPay coreHeight backfill completion (rescan_triggered is in-memory only) #4302

Description

@bfoss765

Summary

The DIP-15 §12.6 coreHeight backfill is correct, but its "already did this" guard is in-memory only, so the backfill re-fires on every process start, forever. On a contact-heavy mainnet wallet this makes the initial sync impossible to complete.

The backfill itself is not in question: when DashPay receival contact accounts are provisioned, the SPV synced_height is lowered to the minimum core_height_created_at across established receival contacts so that payments which landed before those addresses were watched still get matched. Receival addresses enter the filter match set forward-only, so without the rewind those payments are never seen. That behaviour must stay.

What needs fixing is that completion of the backfill is never persisted.

Evidence

  • packages/rs-platform-wallet/src/wallet/identity/state/managed_identity/dashpay.rs — rescan_triggered is a plain in-memory field. It appears in no changeset, no FFI surface and no persistence path, so it resets to its default on every process start.
  • packages/rs-platform-wallet/src/wallet/identity/network/payments.rs — reconcile_dashpay_rescan() then calls the unconditional regressing setter update_synced_height(floor) again.
  • packages/dash-spv/src/sync/filters/sync_manager.rs — sees synced_height < committed_height and calls reset_for_rescan().

The watermark being lowered is the durable one: WalletEntity.syncedHeight (Kotlin persistence/entities/WalletEntity.kt), written back on every changeset header by PlatformWalletPersistenceHandler.onWalletChangesetHeader and rehydrated into WalletMetadata.synced_height on loadPersistedWallets. The filter scanner resumes at synced_height + 1 (packages/dash-spv/src/sync/filters/manager.rs). So the trigger state is ephemeral while the effect is durable — which is exactly the asymmetry that produces an infinite loop.

Impact on large wallets

A large mainnet wallet with roughly 145 DashPay contacts has a floor about 210,000 blocks below the tip. That is ~25 minutes of re-matching per launch, against app sessions of about 19 minutes. The scan is rewound at startup, never reaches the tip before the session ends, and is rewound again on the next launch. The wallet never completes an initial sync, and every launch throws away the progress the previous one made.

This is not self-correcting and does not depend on anything unusual in the wallet — any user with enough contacts for the floor to sit far enough below the tip will hit it, and the more contacts they have the more certain it becomes.

Proposed fix (SDK-side)

Persist backfill completion, not triggering.

Concretely, carry a per-identity (or per-wallet) record of the backfill through the existing persistence chain — DashPayState → changeset → FFI → Kotlin persistence — so it survives process death the same way synced_height already does. Suggested shape:

  • Record the floor the backfill was rewound to, and the height it was rewound from.
  • Treat the backfill as complete only once synced_height has climbed back to at least the rewound-from height. Everything in [floor, rewound_from] has then been re-matched against the contact addresses that were registered when the rewind fired.
  • reconcile_dashpay_rescan() consults the persisted record instead of rescan_triggered, and skips the regressing update_synced_height when the recorded range is already covered.
  • Re-arm whenever a receival contact account is registered that was not part of the covered set — a newly established contact's addresses were not in the filter match set during the covered scan regardless of its core_height_created_at, so it needs its own backfill.

The critical correctness rule: the record must mark completion only after the scan has climbed past the floor, never when the backfill is triggered or started. Marking on trigger converts an interrupted backfill into a permanently-missed set of historical payments — a silent funds-visibility bug, strictly worse than the current performance bug. Any ambiguity should fall back to re-running the backfill: slow, never lossy.

Doing this in the SDK means every consumer gets it, including iOS, rather than each platform re-deriving its own guard.

Interim app-side mitigation (already landed on Android)

To unblock affected wallets, dash-wallet has landed an app-side gate that does not touch the SDK or the AAR (DashPayBackfillGate, branch feat/kotlin-sdk-phase1). It works entirely by observation, since the app cannot compute the floor itself — app-side DashPay contact requests carry no core height at all:

  • Read the durable WalletEntity.syncedHeight either side of a dashPaySyncNow() pass. A regression is the rewind, directly observed, and the pre-rewind height becomes the completion target.
  • Persist that watch across launches, and skip forcing the pass while the scan climbs — otherwise the next pass re-lowers the watermark and the scan can never climb out.
  • Record completion only once the watermark reaches the target; invalidate on any change to the app's own contact set.
  • Believe "no rewind was needed" only on a process's first pass, precisely because rescan_triggered is per-process and would otherwise make a suppressed pass look like a quiet one.

This is a workaround for one consumer, and it costs the app a suppressed contact sweep while a backfill is in flight. Once completion is persisted SDK-side, the Android gate can and should be removed.

Activity

  1. bfoss765 commented on Aug 17, 2026

    @bfoss765
    CollaboratorAuthor

    Scoped for implementation (2026-08-17). The fix is exactly the "persisted breadcrumb" the field's own doc anticipates, and the natural home is IdentityEntry — rescan_triggered mirrors ignored_senders' shape and merge semantics precisely (a grow-only per-identity BTreeSet<Identifier>, union on merge, no tombstones: once a contact's backfill completed, it stays completed).

    Implementation map (why this is a standalone PR, not a drive-by): IdentityEntry crosses the FFI as a structured IdentityEntryFFI, not a serde blob, so the field touches every layer —

    1. IdentityEntry + from_managed capture + apply/restore into ManagedIdentity.dashpay + union merge (all modeled on ignored_senders);
    2. IdentityEntryFFI (ABI addition) + JNI marshal both directions;
    3. Kotlin PlatformWalletPersistenceHandler + a Room column/migration.

    Backward compat is benign: rows persisted before the migration load as empty → the backfill re-fires once per contact, persists, and never again — strictly better than today's every-launch refire, with no data risk.

    Field urgency note: this is the launch-time filter-rewind observed on the contact-heavy mainnet field wallet (hundreds of thousands of filters re-matched per start), so it should ride the next SDK integration fold.

  2. 9 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions