Skip to content

[BUG] 百度网盘 31034 频控暂停结束后请求风暴导致熔断反复升级 / BaiduNetdisk 31034 pause stampede #3254

Open

Description

@zeroornull

Please confirm the following

  • I have read and agree to AGPL-3.0 Section 15 .
    The program is provided "as is" without any warranties; you bear all risks of using it.

  • I have read and agree to AGPL-3.0 Section 16 .
    The copyright holders and distributors are not liable for any damages resulting from the use or inability to use the program.

  • I confirm my description is clear, polite, helps developers quickly locate the issue, and complies with community rules.

  • I have read the OpenList documentation.

  • I confirm there are no duplicate issues or discussions.

  • I confirm this is an OpenList issue, not caused by other reasons (such as network, dependencies, or operation).

  • I believe this issue must be handled by OpenList and not by a third party.

  • I confirm this issue is not fixed in the latest version.

  • I have not read these checkboxes and therefore I just ticked them all, Please close this issue.

OpenList Version (required)

自建镜像(基于近期 main,含百度驱动观测日志);现象在官方驱动逻辑下可复现:冷却结束后等待请求一起放行。

English

Self-built image based on recent main with Baidu driver observation logs. The stampede after cooldown expiry is in the stock driver cooldown logic.

Storage Driver Used (required)

百度网盘(baidu_netdisk),经内置 S3 服务被 Velero/Kopia 等高并发删除/列举客户端访问。

English

BaiduNetdisk (baidu_netdisk), accessed through the built-in S3 server by high-concurrency delete/list clients such as Velero/Kopia.

Bug Description (required)

百度 filemanager 在突发请求时会返回外层 errno=12,info[] 里带 31034(频控)。驱动会进入暂停/冷却。原先冷却一结束,所有等待中的请求会一起发出;若百度仍在限流,它们再次全部收到 31034,暂停被反复拉长,吞吐长时间起不来。

复现步骤

  1. 添加百度网盘存储,开启 S3。
  2. 用备份工具或脚本对同一前缀做大量并发删除/列举(例如 Velero/Kopia 全量清理窗口)。
  3. 日志出现大量 31034;冷却结束后请求再次扎堆,熔断时间被反复拉长。

预期: 冷却结束时应半开探测(只放行少量请求),确认百度恢复后再放行其余等待者;限流错误应对客户端可见(S3 侧宜为 503 SlowDown),而不是静默成功或无限拉长暂停。
实际: 冷却到期后等待请求风暴式重放,31034 再次命中并升级暂停。

与 #3207 / #3208(写后 List 缓存可见性)正交:那边是 Put 后 Head 404;本 issue 是百度频控与冷却放行策略。

English

Under burst load Baidu filemanager returns outer errno=12 with per-item 31034 (frequency control). The driver enters a pause/cooldown. When that pause expires, every waiting request fires at once. If Baidu is still throttling, they all get 31034 again and the pause escalates, so throughput stays down for a long time.

Steps to reproduce

  1. Add BaiduNetdisk storage and enable the S3 server.
  2. Run a high-concurrency delete/list workload against one prefix (e.g. a Velero/Kopia full cleanup window).
  3. Logs show many 31034s; after each cooldown ends, waiting requests stampede and the pause keeps escalating.

Expected: when the pause ends, only a probe request goes out (half-open); release the rest only after Baidu answers something other than 31034. Rate-limit errors should be visible to clients (S3: 503 SlowDown).
Actual: on expiry all waiters fire together, hit 31034 again, and escalate the pause.

Orthogonal to #3207 / #3208 (post-Put List cache visibility).

Logs (required)

生产观察(自建镜像,经内置 S3 → 百度网盘,含多次 Velero/Kopia 全量备份窗口):

  • 约 2026-10-22~10-23 轮转日志:检出 31034 ×494;冷却/暂停多次升级。
  • 2026-10-24 日备窗口:kine-percona 经 OpenList→百度备份成功;Velero daily-full-clean-20261023190013 PartiallyFailed 的唯一错误是 Alertmanager PVB readiness 超时,与百度频控无关。
  • 冷却结束后若不做半开探测,等待请求会扎堆再次打出 31034。
English

Production (self-built image, built-in S3 → BaiduNetdisk, multiple Velero/Kopia full-backup windows):

  • Rotated logs ~2026-10-22–10-23: 31034 ×494; cooldown/pause escalated multiple times.
  • 2026-10-24 daily window: kine-percona backup via OpenList→Baidu succeeded; Velero daily-full-clean-20261023190013 PartiallyFailed solely on alertmanager PVB readiness timeout (not Baidu).
  • Without a half-open probe, waiters stampede on cooldown expiry and hit 31034 again.

Configuration File Content (required)

"s3": { "enable": true, "port": 5246, "ssl": false }

存储:百度网盘,S3 bucket 映射到其中一个目录。密钥已省略。

English

Storage: BaiduNetdisk, S3 bucket mapped to a folder on it. Secrets omitted.

Reproduction Link (optional)

(PR 将附单元测试:breaker open / probe success / probe still 31034 / probe handoff / wait budget)

AI Generated Content

  • I used AI tools to generate this content
  • I did not use AI tools to generate this content

AI model used

Grok Bot

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions