Repository navigation
[BUG] S3 上传成功后立即 Head 返回 404（目录缓存）/ HeadObject 404 right after PutObject #3207
Description
Please confirm the following
-
I have read and agree to AGPL-3.0 Section 15 .
The program is provided "as is" without any warranties; you bear all risks of using it. -
I have read and agree to AGPL-3.0 Section 16 .
The copyright holders and distributors are not liable for any damages resulting from the use or inability to use the program. -
I confirm my description is clear, polite, helps developers quickly locate the issue, and complies with community rules.
-
I have read the OpenList documentation.
-
I confirm there are no duplicate issues or discussions.
-
I confirm this is an
OpenListissue, not caused by other reasons (such as network, dependencies, or operation). -
I believe this issue must be handled by
OpenListand not by a third party. -
I confirm this issue is not fixed in the latest version.
-
I have not read these checkboxes and therefore I just ticked them all, Please close this issue.
OpenList Version (required)
v4.2.7 (3b28270a)
Storage Driver Used (required)
百度网盘，通过内置 S3 服务访问。问题出在 internal/op，其他没有实现 driver.Getter 的驱动应该也会受影响。
English
BaiduNetdisk, accessed through the built-in S3 server. The cause is in internal/op, so other drivers without driver.Getter should be affected too.
Bug Description (required)
PutObject 返回 200 之后，立即对同一个 key 做 HeadObject / GetObject，有时会返回 404。之后会一直 404，直到目录缓存过期（cache_expiration）或手动刷新目录，而网盘里文件其实已经存在。写完立刻回读的备份工具（Velero/Kopia、检查自己刚上传的 complete.json 的脚本）会因此失败。
复现步骤
- 添加百度网盘存储，
cache_expiration> 0，开启 S3 服务。 - 往同一个前缀上传大量对象，每次
PutObject后立即对该 key 做HeadObject。同时 List 这个前缀更容易触发。 - 部分
HeadObject返回 404，但对应的PutObject是成功的。
预期： 通过 OpenList 刚写入的文件能立即读到。
实际： 偶发 404，一直持续到缓存过期。
原因（v4.2.7）
op.Put 只有在目录已经有缓存时，才会把新文件加进目录缓存（fs.go#L683-L697）；而 op.list 会用驱动返回的结果整体替换目录缓存（fs.go#L108）。所以，上传完成前就开始的 List，或者百度暂时还列不出新文件的 List，都会把新条目冲掉。百度网盘没有 Getter，op.Get（fs.go#L165-L213）只能查这份缓存列表，于是返回找不到。
相关：#1725（网页端同样的现象），#3086（类似的问题，只修了 Google Drive）。
修复：#3208
English
After a PutObject returns 200, a HeadObject / GetObject on the same key sometimes returns 404. The 404 then sticks until the directory cache expires (cache_expiration) or the folder is refreshed by hand. The file does exist on the netdisk. Backup tools that write a file and read it back right away (Velero/Kopia, a script checking its own complete.json) fail because of this.
Steps to reproduce
- Add a BaiduNetdisk storage with
cache_expiration> 0 and enable the S3 server. - Upload many objects to one prefix, and
HeadObjecteach key right after itsPutObject. Listing the same prefix at the same time makes it happen more often. - Some
HeadObjectcalls return 404 although theirPutObjectsucceeded.
Expected: a file just written through OpenList can be read back immediately.
Actual: occasional 404 that lasts until the cache expires.
Root cause (v4.2.7)
op.Put only adds the new file to the dir cache if that directory is already cached (fs.go#L683-L697), and op.list replaces the whole cached directory with whatever the driver returned (fs.go#L108). So a List that started before the upload finished, or a Baidu list that does not show the new file yet, wipes the new entry. Because BaiduNetdisk has no Getter, op.Get (fs.go#L165-L213) answers from that cached list and returns not found.
Related: #1725 (same symptom in the web UI), #3086 (similar gap, Google Drive only).
Fix: #3208
Logs (required)
在未修改的 v4.2.7 上跑单元测试（模拟一个没有 Getter、List 会滞后于 Put 的驱动）：
--- FAIL: TestReproA_RefreshListOverwritesPut (0.20s)
repro_put_get_test.go:105: BUG REPRODUCED: Put OK but immediate Get -> ObjectNotFound
--- FAIL: TestReproB_CacheMissDuringPut (0.00s)
repro_put_get_test.go:138: BUG REPRODUCED: Put OK but immediate Get -> ObjectNotFound
生产环境实际发生过（官方镜像，cache_expiration = 30 分钟），来自 OpenList 自己的访问日志：
- 2026-10-16：一个 Velero/Kopia blob 的 PUT 返回 200，之后 3 分钟内 7 次 GET 都返回 404，耗时 0.3–1.3 ms（直接由缓存应答）。之后的 GET 返回 200，对象确实存在。
- 2026-10-18：
complete.json的 PUT 返回 200，3 秒后 HEAD 返回 404，耗时 198 µs。当天晚些时候同一个 key 返回 200。
百度网盘 A/B 测试：cache_expiration = 1 分钟，14 个并发 worker 执行 Put 后立即 Head，同时在同一目录下 List 和 mkdir：
| 版本 | Put → Head 次数 | 缓存导致的 404 |
|---|---|---|
| v4.2.7 | 1459 | 1（30 s 后仍为 404，缓存过期后变为 200） |
| #3208 | 2057（2 轮） | 0 |
不属于本 issue：A/B 测试期间，有些 PUT 返回了 200，但百度同时返回 errno: 31500，这些文件最终没有出现在网盘里。这是驱动/后端的问题。
English
The log lines above are unit tests on unmodified v4.2.7, using a fake driver without Getter whose List lags behind Put.
Seen in production (official image, cache_expiration = 30 min), from OpenList's access log:
- 2026-10-16: a Velero/Kopia blob PUT returned 200, then 7 GETs over the next 3 minutes returned 404 in 0.3–1.3 ms (answered from cache). Later GETs returned 200 and the object exists.
- 2026-10-18:
complete.jsonPUT returned 200, HEAD 3 s later returned 404 in 198 µs. The same key returned 200 later that day.
The table above is an A/B run on BaiduNetdisk with cache_expiration = 1 min, 14 workers doing Put then Head, plus List and mkdir in the same folders.
Not part of this issue: during the A/B runs, some PUTs returned 200 while Baidu returned errno: 31500, and those files never appeared on the netdisk. That is a driver/backend problem.
Configuration File Content (required)
"s3": { "enable": true, "port": 5246, "ssl": false }存储：百度网盘，cache_expiration > 0，S3 bucket 映射到其中一个目录。密钥已省略。
English
Storage: BaiduNetdisk, cache_expiration > 0, S3 bucket mapped to a folder on it. Secrets omitted.
Reproduction Link (optional)
#3208（包含上述测试 / includes the tests）
AI Generated Content
- I used AI tools to generate this content
- I did not use AI tools to generate this content
AI model used
Grok Bot
Activity
zeroornull commentedon Oct 18, 2026
Updated the issue body with production evidence (Velero/Kopia and backup-script cases from OpenList's own access log) and an A/B run with the fix. / 已在正文补充生产环境证据（OpenList 访问日志中的 Velero/Kopia 与备份脚本案例）和修复前后的 A/B 测试结果。
[-][BUG] S3: object returns 404 right after a successful Put (dir cache overwritten by a lagging List, drivers without Getter e.g. BaiduNetdisk) / S3 上传成功后立即 Head 返回 404（目录缓存被滞后的 List 覆盖）[/-][+][BUG] S3 上传成功后立即 Head 返回 404（目录缓存）/ HeadObject 404 right after PutObject[/+]