Repository navigation
feat: add support for D1 bookmarks to fight stale data returned from replicas - #813
Draft
kossnocorp wants to merge 3 commits into
Draft
kossnocorp wants to merge 3 commits into
kossnocorp wants to merge 3 commits into
Conversation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
🚀 Deploying Preview to Cloudflare 🚀Preview URL: https://fix-783-d1-bookmark-void-cloud.voidzero-docs.workers.dev (commit b8f635e)This URL reflects your latest Preview deploymentPreview Deployments by commit
|
kossnocorp
commented
Nov 13, 2026
| 'export declare function setSession(start: D1SessionConstraint | D1SessionBookmark): void;', | ||
| `export declare function getBookmark(client?: ${type}<${schemaType}>): D1SessionBookmark | null;`, | ||
| ] | ||
| : []), |
Contributor
Author
There was a problem hiding this comment.
This is clearly wrong, not all sqlite will have this API, but I didn't have time to figure out what is the best approach here.
Ideally we should know if we're running D1 or not and only then generate types, but maybe there's a cleaner solution like void/d1 or something like that.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR closes #783.
🚧 This is rough implementation with some questionable API decisions, but it works. Opening it in case someone can have some insight on the problem.
In cases where primary D1 is deployed to say
WEURbut the nearest edge isSINright after write, theSINreplica might not get the update fast enough and the user will see data before edits.Unless you're in Europe, where the primary D1 is deployed, you can see the problem in the playground (you might not have the access, ping me): https://voidzero-void-d1-bookmarks.skoss-individual-account.workers.dev/without-bookmark?experiment=immediate&trials=20&pressure=0.5&size=0.5&probes=20
To reproduce:
In "Read-only routing baseline" section click "Run probes" button. Repeat until you see the replica region closest to you (
replica / APAC / SINin my case:Then go a section below "Write/read experiment" and set:
Press "Run Trials" button. Unless you're unlucky or D1 got significant replication pressure and switched from replica to the primary D1, you'll see some of the requests getting stale data (stale data is only possible on a replica,
SINin my case):Depending on your region and current state of D1 instance you might not see stale data at all. If you see that read target is
primary / WEURwait for a 1-2 minutes and start from 1. You can also increase replication pressure a bit. A high pressure makes D1 switch to primary and only hit replicas occasionally. Most likely you will encounter a network error caused by D1 being overwhelmed.The working solution is on the "Manual bookmark" page where data is always fresh: https://voidzero-void-d1-bookmarks.skoss-individual-account.workers.dev/with-bookmark?experiment=immediate&trials=20&probes=20&pressure=0.5&size=0.5