Skip to content

fix(drivers/139): improve mail login credential renewal - #3029

Merged
PIKACHUIM merged 2 commits into
OpenListTeam:mainfrom
UcnacDx2:fix/139-mail-login-pr
Sep 5, 2026
Merged

PIKACHUIM merged 2 commits into
OpenListTeam:mainfrom
UcnacDx2:fix/139-mail-login-pr

Conversation

@UcnacDx2

@UcnacDx2 UcnacDx2 commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

PR title: fix(139): improve mail login credential renewal

Summary / 摘要

This PR improves the 139Yun mail-login credential renewal flow so that password login no longer depends on a pre-login JSESSIONID, can start without pre-supplied mail cookies, and can refresh reusable mail cookies after password/SMS authentication.

此 PR 改进了 139Yun 邮箱登录凭据续期流程:账号密码登录不再依赖预登录阶段获取 JSESSIONID,首次配置时可不预先提供 MailCookies,并可在密码 / 短信验证后自动刷新可复用的 MailCookies。

  • Username + password can now be used without initial mail_cookies; existing cookie-only and Authorization flows remain supported.

  • Password login sends the current raw mail-cookie context, merges response cookies back into driver storage state, and reuses refreshed cookies on later logins.

  • When supported 139 Mail risk control requires SMS verification, the driver sends one verification request, keeps the temporary cookies needed for the follow-up save, verifies sms_code, then continues the normal token/authentication chain.

  • Login and SMS POST requests explicitly disable Resty retries to avoid duplicate non-idempotent authentication/SMS requests.

  • Password login stops at the first 302 response instead of following the redirect, allowing the driver to extract the returned sid directly.

  • Stale input MailCookies are allowed to fail fast-login when username/password fallback is available; MailCookies refreshed by a successful password login are expected to support subsequent fast login.

  • No public API or storage-format migration is introduced. The configuration schema is unchanged, but mail_cookies is now optional when username and password are provided.

  • 现在仅提供用户名和密码也可初始化登录;原有仅 Cookie 和 Authorization 登录方式仍保留。

  • 密码登录会保留现有原始 MailCookies,上游响应中的新 Cookie 会合并回驱动状态,并在后续登录中复用。

  • 遇到受支持的 139 邮箱短信风控时,会发送一次验证码请求、保留下一次保存所需的临时 Cookie,验证 sms_code 后继续完成 token / Authorization 登录链路。

  • 登录和短信 POST 请求显式禁用 Resty 自动重试,避免非幂等认证请求或短信请求被重复发送。

  • 密码登录不会继续跟随首个 302,而是直接从响应中提取 sid。

  • 用户手工提供的旧 MailCookies 失效时允许回退到账号密码登录;密码登录刷新得到的新 MailCookies 应可继续用于 fast login。

  • 未引入公开 API 或存储格式迁移;配置字段结构不变,但在提供用户名和密码时 mail_cookies 不再是必填项。

  • This PR has breaking changes.
    / 此 PR 包含破坏性变更。

  • This PR changes public API, config, storage format, or migration behavior.
    / 此 PR 修改了公开 API、配置、存储格式或迁移行为。

    • Backward-compatible config behavior change only: mail_cookies becomes optional for username/password login; no field or storage-format migration is required.
    • 仅包含向后兼容的配置行为变化:账号密码登录时 mail_cookies 改为可选,不需要字段或存储格式迁移。
  • This PR requires corresponding changes in related repositories.
    / 此 PR 需要关联仓库同步修改。

Related repository PRs / 关联仓库 PR:

  • OpenList-Frontend: N/A
  • OpenList-Docs: N/A

Testing / 测试

  • go test ./...
  • go test ./drivers/139 -count=1
  • Manual test / 手动测试:
    • Ran real-account GitHub Actions regression tests with credentials, MailCookies, and SMS code supplied only through masked repository secrets.
    • Verified password fallback with stale input MailCookies completes Step1 -> Step2 -> Step3 and produces Authorization.
    • Verified password/SMS authentication refreshes MailCookies and that the refreshed MailCookies can complete a subsequent fast-login Step2 -> Step3 chain.
    • Verified an already-stale input MailCookies set may fail fast login while password fallback and the refreshed-cookie fast login still succeed.
    • Verified zero-initial-cookie SMS bootstrap in earlier isolated live tests: risk detection -> SMS verification -> Step2 -> Step3 -> reusable device cookies.
    • No sensitive credentials, SMS codes, cookies, SID, RMKEY, or Authorization values are logged by the test workflows.

Deterministic tests additionally cover:

  • credential-state selection for Authorization, full login with/without initial MailCookies, cookies-only login, partial credentials, invalid cookies, and invalid Basic -prefixed Authorization;
  • Cookie merging and extraction of Os_SSo_Sid / RMKEY;
  • SMS risk-code-to-scene mapping, SMS request construction, picture-challenge rejection, SMS verification, Cookie merge, and sms_code clearing;
  • password login without initial cookies, single POST behavior, stopping at the first redirect, and persistence of returned cookies;
  • reuse of stale/raw device-cookie context and refreshed cookies across successive password logins.

Checklist / 检查清单

  • I have read CONTRIBUTING.
    / 我已阅读 CONTRIBUTING。
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
    / 我确认此贡献符合仓库许可证、贡献规范和行为准则。
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
    / 我已按适用情况使用 gofmt、go fmt 或 prettier 格式化变更代码。
  • I have requested review from relevant maintainers or code owners where applicable.
    / 我已在适用情况下请求相关维护者或代码所有者审查。

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.
    / 此 PR 包含 AI 辅助内容。

Tools used / 使用工具:

  • ChatGPT
  • Codex
  • GitHub Copilot
  • Claude
  • Gemini
  • Other (please specify) / 其他(请注明):

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Refactoring / 重构

  • Documentation / 文档

  • Tests / 测试

  • Translation / 翻译

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.
    / 我已审核并验证此 PR 中的所有 AI 辅助内容。

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.
    / 我已确保所有 AI 辅助提交都包含 Co-Authored-By 归属信息。

  • I can reproduce all AI-assisted content included in this PR without any AI tools.
    / 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。

Copilot AI lite review requested due to automatic review settings September 3, 2026 07:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Several new code paths call base.RestyClient.Clone() without guarding against base.RestyClient being nil, which can cause a runtime panic in uninitialized contexts (e.g., some tests/embedded usage).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR refactors and hardens the 139Yun mail-login renewal flow to support username/password login without requiring pre-supplied mail_cookies, refresh cookies after password/SMS auth, and avoid duplicate non-idempotent requests by disabling Resty retries for the login/SMS POSTs.

Changes:

  • Removes the pre-login JSESSIONID dependency and uses the existing raw cookie context (if any), while persisting cookies returned by password/SMS login back into driver state.
  • Updates credential validation to allow full login with just username + password (cookies optional), and avoids attempting cookie fast-login when mail_cookies is empty.
  • Updates tests and config help text to reflect the new credential-state behavior and cookie refresh semantics.
File summaries
File Description
drivers/139/util.go Removes pre-login cookie sanitization, disables retries for login/SMS POSTs, persists response cookies, and relaxes credential-state requirements.
drivers/139/util_test.go Reworks tests to cover password login without initial cookies, redirect stopping, and cookie persistence/reuse.
drivers/139/meta.go Updates configuration help strings to document mail_cookies as optional for username/password login.
Review details

Suppressed comments (2)

drivers/139/util.go:1303

  • base.RestyClient may be nil before base.InitClient() runs; base.RestyClient.Clone() will panic. Add a nil guard and use base.NewRestyClient() as a fallback.
	res, err := base.RestyClient.Clone().SetRetryCount(0).R().
		SetHeaders(mailXMLHeaders(d.MailCookies)).

drivers/139/util.go:1382

  • base.RestyClient.Clone() will panic if the global client hasn't been initialized (base.InitClient not called yet). Consider guarding nil and falling back to base.NewRestyClient() to keep this login path safe in tests/embedded usage.
	res, err := base.RestyClient.Clone().
		SetRetryCount(0).
		SetRedirectPolicy(resty.RedirectPolicyFunc(func(_ *http.Request, _ []*http.Request) error {
			return http.ErrUseLastResponse
		})).R().
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread drivers/139/util.go Resolved
Fall back to base.NewRestyClient() when base.RestyClient has not been initialized, while preserving cloned global-client behavior and the login/SMS retry and redirect policies.
@UcnacDx2
UcnacDx2 force-pushed the fix/139-mail-login-pr branch from 2a9e672 to 53347cf Compare September 5, 2026 03:02

@pikachuren pikachuren left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏 感谢 @UcnacDx2 提交!
🤖 AI 自动审核声明:本评审报告由 AI 自动生成,当前使用 Claude Opus 5 模型进行分析。
⚠️ AI 分析结果仅供参考,可能存在误判或遗漏。如您发现任何问题或有不同意见,欢迎随时提出讨论和纠正。
⚠️ 重要提醒:即使 AI 评审认为代码质量良好且建议合并,最终是否合并仍需由项目维护者进行人工判定。项目维护者会综合考虑代码质量、项目规划、技术方向、团队资源等多方面因素做出是否合并的决策。

🎯 结论

🔄 Request Changes — 存在 1 个 P1 问题需确认,1 个 P2 安全问题待评估

📖 概要

fix(drivers/139): improve mail login credential renewal · 改进 139Yun 邮箱登录凭据续期流程
核心改动:移除预登录 JSESSIONID 依赖,支持无初始 MailCookies 的密码登录,实现 Cookie 自动刷新和复用

🧭 整体方案

采用简化登录流程的技术路线:去掉预登录获取 JSESSIONID 步骤,直接使用现有 cookies 或空 cookies 进行密码登录;通过响应 cookies 合并实现自动刷新和复用。改动分为配置层(meta.go)、核心逻辑层(util.go)、测试层(util_test.go),层次清晰,方案合理。

📊 变更统计

3 个文件(+209 / -273 行) | 功能 ⭐⭐⭐⭐ | 最小改动 ⭐⭐⭐ | 前向兼容 ⭐⭐⭐⭐ | 方案设计 ⭐⭐⭐⭐

🚨 关键问题

P1(建议修复):

  • ⚠️ util.go:1256 / util.go:1309 禁用 Resty 自动重试可能导致网络不稳定时登录失败 → PR 说明是为了避免重复发送非幂等的短信验证码请求,这个出发点是对的。但是否考虑更优雅的方案?例如:
    • 对于短信发送请求,可以在请求中加入幂等性令牌(idempotency key),服务端保证相同令牌的请求只处理一次
    • 或者在客户端增加请求去重逻辑,记录最近发送的请求指纹,避免短时间内重复发送
    • 完全禁用重试可能在网络抖动时导致本可成功的请求失败,影响用户体验

P2(可选):

  • 💡 util.go:删除 sanitizeMailLoginCookies 和 mailLoginCookieExclusions → 原代码会过滤掉一些特定 cookie(如 Os_SSo_Sid、RMKEY 等),新代码直接使用 mergeMailCookieHeader 合并所有 cookies。请确认:
    • 这些被删除的过滤逻辑是否会导致某些敏感或临时 cookie 被意外保留?
    • 旧的 sanitizeMailLoginCookies 会替换旧的 JSESSIONID,新代码通过 mergeMailCookieHeader 覆盖同名 cookie,逻辑上等价,但请确认边界情况(如 cookie 顺序、重复名称)是否处理正确

📂 逐文件分析

meta.go

改动意图:更新配置字段的帮助文本,说明 mail_cookies 现在是可选的
代码逻辑:将 mail_cookies 从"必须与 username/password 一起提供"改为"可选,首次密码登录时可留空"
问题分析:✅ 帮助文本准确反映了新的配置行为,向后兼容性良好

util.go

改动意图:简化密码登录流程,移除预登录获取 JSESSIONID 的步骤,支持无初始 cookies 的密码登录
代码逻辑:

  • 删除了 fetchMailJSessionID、sanitizeMailLoginCookies 等辅助函数
  • step1_password_login 不再发起预登录请求,直接使用 d.MailCookies 发送密码登录请求
  • 登录成功后通过 mergeMailCookieHeader 将响应 cookies 合并回 d.MailCookies
  • sendSMSVerificationCode 和 verifySMSCode 禁用了 Resty 重试

问题分析:

  • ⚠️ [P1] 禁用重试(见上方关键问题)
  • ⚠️ [P2] 删除 cookie 过滤逻辑(见上方关键问题)
  • ✅ 登录流程简化合理,减少了一次不必要的 HTTP 请求
  • ✅ Cookie 合并和复用逻辑清晰,mergeMailCookieHeader 实现正确

详细建议:

  1. 对于 SetRetryCount(0),建议在代码注释中明确说明为何禁用重试(防止重复发送短信),并考虑未来是否可以通过幂等性令牌优化
  2. 对于删除的 sanitizeMailLoginCookies,建议在 PR 或代码注释中说明为何不再需要 cookie 过滤,以及新的 mergeMailCookieHeader 是否能正确处理所有边界情况

util_test.go

改动意图:更新测试用例以反映新的登录逻辑
代码逻辑:

  • 删除了 TestSanitizeLoginCookiesDropsStaleJSessionIDWhenFreshOneMissing 等旧测试
  • 新增了 TestPasswordLoginWithoutInitialCookiesStopsAtRedirectAndPersistsCookies 和 TestPasswordLoginReusesRawRefreshedCookies 测试无初始 cookies 的密码登录和 cookies 复用场景
  • 为需要重试的测试添加了 useRetryingTestClient 辅助函数

问题分析:✅ 测试覆盖较全面,新增的测试用例准确验证了核心改动的行为

✅ 待处理清单

  • [P1] util.go:1256/1309 评估是否需要为短信请求增加幂等性支持,或在注释中说明禁用重试的权衡
  • [P2] util.go 确认删除 sanitizeMailLoginCookies 后是否存在安全或逻辑问题

🎯 结论:🔄 Request Changes — 禁用重试可能影响网络不稳定时的用户体验,建议评估是否有更优方案或至少在注释中说明权衡;删除 cookie 过滤逻辑需确认安全性

@UcnacDx2

UcnacDx2 commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor Author

🙏 感谢 @UcnacDx2 提交! 🤖 AI 自动审核声明:本评审报告由 AI 自动生成,当前使用 Claude Opus 5 模型进行分析。 ⚠️ AI 分析结果仅供参考,可能存在误判或遗漏。如您发现任何问题或有不同意见,欢迎随时提出讨论和纠正。 ⚠️ 重要提醒:即使 AI 评审认为代码质量良好且建议合并,最终是否合并仍需由项目维护者进行人工判定。项目维护者会综合考虑代码质量、项目规划、技术方向、团队资源等多方面因素做出是否合并的决策。

🎯 结论

🔄 Request Changes — 存在 1 个 P1 问题需确认,1 个 P2 安全问题待评估

📖 概要

fix(drivers/139): improve mail login credential renewal · 改进 139Yun 邮箱登录凭据续期流程 核心改动:移除预登录 JSESSIONID 依赖,支持无初始 MailCookies 的密码登录,实现 Cookie 自动刷新和复用

🧭 整体方案

采用简化登录流程的技术路线:去掉预登录获取 JSESSIONID 步骤,直接使用现有 cookies 或空 cookies 进行密码登录;通过响应 cookies 合并实现自动刷新和复用。改动分为配置层(meta.go)、核心逻辑层(util.go)、测试层(util_test.go),层次清晰,方案合理。

📊 变更统计

3 个文件(+209 / -273 行) | 功能 ⭐⭐⭐⭐ | 最小改动 ⭐⭐⭐ | 前向兼容 ⭐⭐⭐⭐ | 方案设计 ⭐⭐⭐⭐

🚨 关键问题

P1(建议修复):

  • ⚠️ util.go:1256 / util.go:1309 禁用 Resty 自动重试可能导致网络不稳定时登录失败 → PR 说明是为了避免重复发送非幂等的短信验证码请求,这个出发点是对的。但是否考虑更优雅的方案?例如:

    • 对于短信发送请求,可以在请求中加入幂等性令牌(idempotency key),服务端保证相同令牌的请求只处理一次
    • 或者在客户端增加请求去重逻辑,记录最近发送的请求指纹,避免短时间内重复发送
    • 完全禁用重试可能在网络抖动时导致本可成功的请求失败,影响用户体验

P2(可选):

  • 💡 util.go:删除 sanitizeMailLoginCookies 和 mailLoginCookieExclusions → 原代码会过滤掉一些特定 cookie(如 Os_SSo_Sid、RMKEY 等),新代码直接使用 mergeMailCookieHeader 合并所有 cookies。请确认:

    • 这些被删除的过滤逻辑是否会导致某些敏感或临时 cookie 被意外保留?
    • 旧的 sanitizeMailLoginCookies 会替换旧的 JSESSIONID,新代码通过 mergeMailCookieHeader 覆盖同名 cookie,逻辑上等价,但请确认边界情况(如 cookie 顺序、重复名称)是否处理正确

📂 逐文件分析

meta.go

改动意图:更新配置字段的帮助文本,说明 mail_cookies 现在是可选的 代码逻辑:将 mail_cookies 从"必须与 username/password 一起提供"改为"可选,首次密码登录时可留空" 问题分析:✅ 帮助文本准确反映了新的配置行为,向后兼容性良好

util.go

改动意图:简化密码登录流程,移除预登录获取 JSESSIONID 的步骤,支持无初始 cookies 的密码登录 代码逻辑:

  • 删除了 fetchMailJSessionID、sanitizeMailLoginCookies 等辅助函数
  • step1_password_login 不再发起预登录请求,直接使用 d.MailCookies 发送密码登录请求
  • 登录成功后通过 mergeMailCookieHeader 将响应 cookies 合并回 d.MailCookies
  • sendSMSVerificationCode 和 verifySMSCode 禁用了 Resty 重试

问题分析:

  • ⚠️ [P1] 禁用重试(见上方关键问题)
  • ⚠️ [P2] 删除 cookie 过滤逻辑(见上方关键问题)
  • ✅ 登录流程简化合理,减少了一次不必要的 HTTP 请求
  • ✅ Cookie 合并和复用逻辑清晰,mergeMailCookieHeader 实现正确

详细建议:

  1. 对于 SetRetryCount(0),建议在代码注释中明确说明为何禁用重试(防止重复发送短信),并考虑未来是否可以通过幂等性令牌优化
  2. 对于删除的 sanitizeMailLoginCookies,建议在 PR 或代码注释中说明为何不再需要 cookie 过滤,以及新的 mergeMailCookieHeader 是否能正确处理所有边界情况

util_test.go

改动意图:更新测试用例以反映新的登录逻辑 代码逻辑:

  • 删除了 TestSanitizeLoginCookiesDropsStaleJSessionIDWhenFreshOneMissing 等旧测试
  • 新增了 TestPasswordLoginWithoutInitialCookiesStopsAtRedirectAndPersistsCookies 和 TestPasswordLoginReusesRawRefreshedCookies 测试无初始 cookies 的密码登录和 cookies 复用场景
  • 为需要重试的测试添加了 useRetryingTestClient 辅助函数

问题分析:✅ 测试覆盖较全面,新增的测试用例准确验证了核心改动的行为

✅ 待处理清单

  • [P1] util.go:1256/1309 评估是否需要为短信请求增加幂等性支持,或在注释中说明禁用重试的权衡
  • [P2] util.go 确认删除 sanitizeMailLoginCookies 后是否存在安全或逻辑问题

🎯 结论:🔄 Request Changes — 禁用重试可能影响网络不稳定时的用户体验,建议评估是否有更优方案或至少在注释中说明权衡;删除 cookie 过滤逻辑需确认安全性

p1预期行为,challenge令牌存在cookie里,无需额外变更
p2预期行为,去除了过度设计,解决无法触发authorization自动续期流程的问题

@PIKACHUIM
PIKACHUIM merged commit d90d849 into OpenListTeam:main Sep 5, 2026
12 checks passed
@UcnacDx2
UcnacDx2 deleted the fix/139-mail-login-pr branch September 5, 2026 03:58
@UcnacDx2
UcnacDx2 restored the fix/139-mail-login-pr branch September 5, 2026 05:36
@UcnacDx2
UcnacDx2 deleted the fix/139-mail-login-pr branch September 5, 2026 05:58
@hbqjren

hbqjren commented Sep 8, 2026

Copy link
Copy Markdown

移动云盘app,抓包cookie,几乎是没有到期时间。我去年上半年cookie还能正常使用。139邮箱app cookie估计也是没时间限制

@UcnacDx2

UcnacDx2 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

移动云盘app,抓包cookie,几乎是没有到期时间。我去年上半年cookie还能正常使用。139邮箱app cookie估计也是没时间限制

理论上都不能长期,我测试过的邮箱cookie有效期是14天,云盘authorization是30天,应该没有例外情况

@hbqjren

hbqjren commented Sep 10, 2026

Copy link
Copy Markdown

你抓是网页那个cookie,移动云盘网页cookie30天到期,而app上的cookie记录设备型号,操作系统版本,useragent等等。除非你在app内移除“信任设备

@UcnacDx2

Copy link
Copy Markdown
Contributor Author

你抓是网页那个cookie,移动云盘网页cookie30天到期,而app上的cookie记录设备型号,操作系统版本,useragent等等。除非你在app内移除“信任设备

authorization头部就有有效期,你自己看看就是了

cjd6568358 pushed a commit to cjd6568358/OpenList-zig that referenced this pull request Oct 18, 2026
…#3029)

* fix(drivers/139): improve mail login credential renewal

* fix(drivers/139): guard mail login client initialization

Fall back to base.NewRestyClient() when base.RestyClient has not been initialized, while preserving cloned global-client behavior and the login/SMS retry and redirect policies.
PIKACHUIM added a commit that referenced this pull request Oct 21, 2026
…3058)

* feat: add system initialization (setup wizard) support

* Squashed commit of the following:

commit 2d51c9a
Author: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
Date:   Mon Sep 7 14:14:22 2026 +0800

    feat!(init): add initialization wizard (#3041)

    feat: add system initialization (setup wizard) support

    Co-authored-by: PIKACHUIM <PIKACHUIM@users.noreply.github.com>

commit d9d8aa2
Author: ShenLin <773933146@qq.com>
Date:   Mon Sep 7 12:01:15 2026 +0800

    fix(s3): default upload content types and return partial content (#3053)

    - Default missing upload MIME types to application/octet-stream before passing streams to storage drivers.
    - Return HTTP 206 for successful ranged GET responses while preserving error statuses.
    - Add isolated response-status regression tests without database initialization.

    Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
    Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

commit 55530ff
Author: ShenLin <773933146@qq.com>
Date:   Mon Sep 7 12:00:50 2026 +0800

    fix(release): fetch frontend assets from edge (#3052)

    - Fetch frontend prerelease assets from edge after release immutability was accidentally enabled for rolling.

    Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
    Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

commit 6247cf7
Author: MadDogOwner <xiaoran@xrgzs.top>
Date:   Sat Sep 5 15:56:40 2026 +0800

    feat(server/s3): support multipart upload (#2813)

commit eee910b
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Sat Sep 5 12:22:53 2026 +0800

    fix(deps): update module github.com/rclone/rclone to v1.75.1 (#3035)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 6b55a82
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Sat Sep 5 12:14:09 2026 +0800

    chore(deps): update docker/setup-qemu-action digest to 1f40c72 (#3021)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 93dac16
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Sat Sep 5 12:12:51 2026 +0800

    chore(deps): update go toolchain directive to v1.27.1 (#3024)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 6ad4460
Author: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
Date:   Sat Sep 5 12:11:51 2026 +0800

    feat(drivers/guangyapan): add md5-based instant upload support (#3034)

    feat(guangyapan): add md5-based instant upload support

    Co-authored-by: PIKACHUIM <PIKACHUIM@users.noreply.github.com>

commit d90d849
Author: UcnacDx2 <127503808+UcnacDx2@users.noreply.github.com>
Date:   Sat Sep 5 11:55:41 2026 +0800

    fix(drivers/139): improve mail login credential renewal (#3029)

    * fix(drivers/139): improve mail login credential renewal

    * fix(drivers/139): guard mail login client initialization

    Fall back to base.NewRestyClient() when base.RestyClient has not been initialized, while preserving cloned global-client behavior and the login/SMS retry and redirect policies.

commit c3d3da9
Author: ShenLin <773933146@qq.com>
Date:   Sat Sep 5 00:12:20 2026 +0800

    fix(drivers/189): decode JSON strings before parsing timestamps (#3033)

    - Decode JSON time strings before normalizing Unicode spaces in both 189 drivers
    - Exercise escaped spaces and existing date formats through JSON unmarshalling
    - Cover invalid JSON input and XML time parsing

    Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
    Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* feat(seed): add advanced transfer seed support (OSS/torrent/CAS)

Add unified sharing-seed format library (openlist-sharing-seed v1), standard BT torrent v1 with x-openlist/x-cas extensions, and exact legacy-compatible CAS Base64 payload. Add /fs/seed/{capabilities,generate,parse,convert,rapid_upload,offline_download,update} APIs with hash-matrix driven generation, per-file comments, multi-format output, safe direct/share source embedding, rapid-upload and offline-download fallbacks, and seed sidecar lifecycle for upload/copy/move/rename/remove. Add global and per-storage (inherit/on/off) auto-generation policy, format policies, default hash matrix, site URL and single-file direct-preview settings. Includes security hardening: path traversal checks, SSRF-safe source validation restricted to the configured site, content-write permission checks, offline-download permission checks, and torrent/OSS/CAS parse limits.

* feat(seed): complete edit, recalculate and relayed transfer

Implement seed metadata editing (comment/trackers/channels/file comments/sources) and server-side hash recalculation with piece-size write-back and a bounded streaming reader. Add relayed transfer that saves synchronously into an intermediate storage then copies to the final destination. Add missing content-write and copy permission checks on the final relay target, source URL host validation against the configured site, and an io.LimitReader hard cap. Expose transfer/edit/recalculate in parse capabilities.

* feat(seed): channel update, share validity and CAS direct access

Record successful saves as channels and failures as missing_channels when update_channel is set. Return share_status during edit by validating openlist-share sources. Add seed_cas_direct_access setting for immediate single-file CAS restore. Rename and consume the default hash matrix setting (seed_default_matrix) with a whole/pieces JSON structure, returned via capabilities.

* fix(seed): allow capabilities preflight without seed_data

SeedCapabilityReq embeds SeedDataReq whose SeedData field was bound with required. The /fs/seed/capabilities preflight branch only needs paths, so the binding failed before the handler could branch. Drop the required tag and enforce non-empty seed_data inside decodeSeedData instead.

* feat(seed): support removing files from a seed via update

Add remove_files to SeedUpdateReq so the preview can drop individual files and re-encode the seed container.

* feat(seed): enrich capabilities and per-file source selection

Capabilities now report streamable/direct_source_available/share_available and the configured tracker list. Generate supports per-file share_files/direct_files with legacy global fallback. Add seed_default_trackers setting.

* feat(seed): BT always offline-downloadable and expose driver rapid capability

ParseSeed marks torrent seeds as offline_download capable even without sources (magnet/tracker). SeedCapabilities returns driver_supports so the frontend can show which rapid-transfer methods the destination driver accepts.

* feat(seed): asynchronous generation for large file sets

Extract seed generation into fs.GenerateSeedArtifacts and add a SeedGenerateTask manager. Requests over 1GB are queued as background tasks that write artifacts into the destination folder. Registers the manager in bootstrap and wires the handler to fall back to async.

* fix(seed): export shared seed helpers for upload sidecar

Export NormalizeSeedFormats and EncodeGeneratedSeed so fsup.go can reuse them after the generation logic moved into internal/fs. Drop the now-unused slices import.

* fix(seed): generate one CAS per file for multi-file selection

CAS is a single-file container, so multi-file generation now emits one .cas artifact per file instead of failing with 'CAS requires exactly one file'. Capabilities no longer gate cas on single-file selection.

* feat(seed): multi-file CAS and derive seed name

CAS now supports multiple files via a files array while keeping the legacy five-field single-file payload byte-compatible. Derive seed names from the selection (single file, common base, or folder) instead of hardcoding 'OpenList Seed'.

* fix(seed): preserve per-piece MD5 list in CAS payload

Encode the slice_md5s and slice_size fields in CAS (single-file and per-file), restore them into SeedFile.Hashes.Pieces.MD5 on decode, and hide the legacy warning when piece hashes are present. Relax the wire-format test to allow the optional extension fields while keeping the five required fields.

* feat(drivers): expose MD5/SHA1 hashes in misskey and quark_open file listing

* feat: implement seed-based rapid upload for 12 drivers with optimized hash calculation

- Add SeedRapidUpload interface and implementations for 12 cloud storage drivers:
  * 189pc, 189_tv (MD5-based)
  * 115 (SHA1-based)
  * 123, 123_open (SHA1/MD5)
  * baidu_netdisk (MD5-based)
  * aliyundrive_open (SHA1-based)
  * quark_open (MD5+SHA1)
  * pikpak, thunder, thunderx, thunder_browser (GCID-based)

- Enhance hash calculation engine with 4x performance improvement:
  * Add GCID hash support in hash_writer.go
  * Optimize to calculate MD5/SHA1/SHA256/GCID in single pass
  * Add file size context for proper hash generation
  * Improve torrent format to support GCID hashes

- Improve capability detection and error handling:
  * Add driver capability reporting (supported hash algorithms)
  * Detect available hashes from file metadata to avoid downloads
  * Add detailed error messages for unsupported operations

Performance: Reduces cross-storage transfer time by 92% and bandwidth by 50%

* fix(115): enhance Get() error handling for empty responses\n\n- Handle null/empty FileID in API response\n- Return ObjectNotFound when FileID is empty\n- Improve robustness for non-existent paths

* fix(seed): repair broken build of hash-driven rapid upload across 12 drivers

The previous commit (624fdd2) introduced the authoritative
driver.SeedRapidUploader interface, but left every driver's seed_rapid.go
on the older, incompatible API, so the branch did not compile at all.

Interface alignment (all 12 drivers):
- Migrate 189pc, 115, 123, 123_open, 189_tv, baidu_netdisk,
  aliyundrive_open, quark_open, pikpak, thunder, thunderx,
  thunder_browser to RapidUploadByHashes / RapidHashAlgos
  ([]utils.HashType, no longer []*utils.HashType) / RapidHashNeedsPieces
- Add shared driver.SeedHashStream as a complete model.FileStreamer that
  carries metadata and hashes only, replacing the duplicated, incomplete
  hashOnlyStream implementations

Fixes uncovered while aligning the interface:
- 123_open: response fields live under Data (Data.Reuse / Data.FileID)
- quark_open: pre.Data.FID -> pre.Data.Fid
- 123: type is Pan123 (not Yun123); FileId is int64 and needs formatting
- aliyundrive_open: CreateResp has no File field; use FileId plus
  completeUpload
- thunder/thunderx/thunder_browser: UploadTaskResponse.File is a Files
  value type, return &resp.File
- 189pc/189_tv: FamilyID is a string, use isFamily() instead
- 189pc: restore rapidUploadByCAS removed by the previous commit; it is
  still referenced by torrent.go. Reimplemented as the three-step CAS
  flow (initMultiUpload -> checkTransSecond -> commitMultiUploadFile)

Build and hashing fixes:
- hash_writer.go: HashType exposes NewFunc; GCID.New does not exist
- Add the missing fileSize argument to NewHashWriter at all call sites
  (pkg/torrent, drivers/189, drivers/189pc, internal/fs, server/handles)
- Add errs.ErrUnavailableHash / ErrEmptyHash / ErrHashMismatch /
  ErrRapidUploadFailed used by the rapid-upload implementations

Drivers whose protocol needs real content (115 pre_hash, aliyundrive_open
and quark_open proof_code) now open req.Open() lazily and degrade to
ErrUnavailableHash when no content source is available, so the caller can
fall back to a normal download.

Note: go vet warnings for non-constant format strings in 189pc/utils.go
are pre-existing and intentionally left untouched.

* fix(seed): harden seed source fetching and unify rapid-upload hash rules

Security fixes for the transfer-seed feature reviewed on
feat/advanced-transfer-seeds.

SSRF via redirect (torrent.go):
- Source validation only pinned the first hop while http.DefaultClient
  silently followed up to 10 redirects, so a benign-looking source could
  302 to a metadata or loopback endpoint. Fetching now goes through
  seedSourceHTTPClient, whose CheckRedirect re-validates every hop with the
  same rule, caps the hop count and forbids scheme downgrades.
- Host validation is collapsed into one validateSeedHost used by both the
  pre-flight check and the redirect guard, so the rules cannot drift apart.
- Requests stay anonymous by design: a seed has to remain usable from an
  instance that does not hold the originating session, so no credentials,
  cookies or signing parameters are ever attached.

Seed content fetching (torrent.go):
- Propagate the request context instead of context.Background(), so
  cancellation actually stops the download.
- Stream the body through an io.LimitReader instead of buffering up to 1GB
  in memory; only proof windows (quark/aliyun) and a 128KiB prefix (115)
  are read, so a full buffer was pure waste. Oversized responses are now
  rejected from Content-Length before any streaming starts.

Other correctness fixes:
- sameSeedHost compares hostname plus the effective port, so a configured
  "https://pan.example.com" and an embedded "...:443" are no longer treated
  as different origins (which silently dropped valid sources).
- buildSeedRapidUploadRequest rejects multi-file torrents, and single files
  whose metadata size disagrees with the torrent length, instead of sending
  the destination a size/hash pair that contradicts itself. Both call sites
  now handle the nil result instead of dereferencing it.
- SliceMD5FromPieces becomes the single implementation of the sliceMd5 rule,
  replacing five copies across hash_writer.go, torrent.go, generate.go,
  189/torrent.go and 189pc/torrent.go. Generation and CAS encoding compare
  this value against the remote provider, so drift silently degrades rapid
  uploads into hash mismatches.
- bencode string lengths are bounded by DefaultMaxSeedSize, matching the
  input limit that actually applies; the previous 100MB ceiling was
  unreachable and its comment claimed the wrong rationale.

The overwrite flag on TorrentRapidUpload stays true on purpose: rapid upload
semantically means mounting existing remote data into the target directory,
which is already an overwrite, so exposing it as an option adds no value.

Tests:
- pkg/torrent/seed_security_test.go: path traversal, file-count limit, the
  canonical sliceMd5 rule, agreement between GetSliceMD5 and
  BuildCASInfoFromMD5s, bencode length/depth/trailing-data rejection, and
  OSS -> torrent -> CAS -> OSS round trips.
- server/handles/torrent_seed_test.go: sameSeedHost port normalization
  (including look-alike domains), validateSeedHost rejections, the redirect
  guard blocking metadata/loopback/downgrade targets, hop limits, source path
  contracts, and rejection of multi-file or size-mismatched seeds.

go build ./... passes; go test ./pkg/torrent/... and
go test ./server/handles/... pass.

* refactor(seed): converge sliceMd5 rule on SliceMD5FromPieces

buildCASFileEntry previously reimplemented the sliceMd5 rule inline, which is the exact duplication that let the hash-generation and torrent/CAS encoding sides drift apart silently. Reuse the canonical SliceMD5FromPieces helper so both producers and consumers share one implementation.

* feat(ci): remove docs/PR-transfer-seeds.md

* ci(seeds): add standalone dev-seeds build and publish workflow

Builds linux amd64/arm64 (musl), windows amd64 and darwin arm64 from a chosen ref, embeds the rolling frontend by default, and force-publishes the compressed archives plus checksums to the dev-seeds branch.

* ci(seeds): add focused seed regression tests

Runs torrent codec/security tests, handler seed tests, go vet and a full build when seed sources change.

* chore(ci): drop seed test and dev-seeds build workflows

* fix(seed): address PR review findings

- drop the duplicate json format field from SeedConvertReq

- validate X-Seed-* headers before storing the uploaded file

- enforce read permission while sizing async seed generation

* fix(seed): refuse async offline downloads during relayed transfer

The relay path now tells saveSeedFilesToPath to reject offline-download tool tasks instead of enqueueing them into the transit storage and only rejecting the result afterwards.

* test(seed): cover hash-only stream contract for rapid upload

---------

Co-authored-by: PIKACHUIM <PIKACHUIM@users.noreply.github.com>
Co-authored-by: pikachuren <pikachuren@example.com>
liu200501 added a commit to liu200501/OpenList that referenced this pull request Oct 27, 2026
* fix: harden request handling for files, search, proxy and SSO

Co-authored-by: PIKACHUIM <PIKACHUIM@users.noreply.github.com>

* fix(drivers/lanzou): support new /fn download page structure (OpenListTeam#3157)

Lanzou recently changed the file share page: download params moved
into an iframe (/fn?<token>) inner page, and the download API is now
an absolute URL (https://apifile.woozooo.com/ajaxfile.php?file=N)
with new sign params (wp_sign/ajaxdata, action=downprocess).

The old findFileIDReg ('/ajaxm.php?file=N' relative path) no longer
matches, causing 'failed link: failed get link: not find file id'.

Fall back to parsing the new /fn page structure when the legacy
regex does not match. Legacy flow is kept untouched.

Signed-off-by: rzsgsfm <rzsgsfm@users.noreply.github.com>
Co-authored-by: rzsgsfm <rzsgsfm@users.noreply.github.com>
Co-authored-by: GLM (ZCode CLI) <noreply@z.ai>

* fix(drivers/189pc): align login with the official PC client (OpenListTeam#3105)

QR code scans were authorized on the phone but the storage stayed stuck on
the QR page, and token-only setups failed with "params is null".

The driver used appId 8025431004 while the official PC client uses
9317140619. Tokens, sessions and QR sessions are all scoped to an appId, so
the mismatch meant the QR poll never saw status:0 and an accessToken could
not be exchanged for a sessionSecret.

- Use appId 9317140619 and version 7.2.4.0. QR state polling uses
  clientType=1; password login keeps 10020.
- Send the QR poll parameters the official client sends (cb_SaveName,
  isOauth2, state, user-finger header, logbox Referer) and poll locally
  instead of only checking once per save.
- Parse lt/reqId from the logbox redirect and paramId from appConf.do. The
  new login page no longer embeds them as inline variables; the old inline
  format is still supported.
- Implement the -133 second device verification via
  sendSmsCodeForSecondAuth/submitForSecondAuth. That endpoint has no
  dedicated SMS field: the code goes into epd, encrypted with the same
  public key used for the password. Persist the DEVICEID cookie so the
  verification only happens once.
- Detect refreshToken.do failures. It reports them as HTTP 200 with a
  result field, so SetError never fired and a failed refresh was treated as
  success, surfacing later as a misleading "params is null".
- username/password are no longer required, so token-only storages save
  without placeholders. clientSn/jgOpenId are optional and only sent when
  configured; user-finger is generated once per storage.
- Return named errors instead of panicking when the login page changes shape.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(drivers/189pc): preserve file extension during family transfer (OpenListTeam#3083)

* fix(189pc): preserve family transfer extension

- Preserve the source extension in family-transfer temporary upload names.
- Fall back to .transfer for extensionless source files.
- Add unit coverage for extension handling.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* fix(189pc): include extension in upload progress key

- Separate resumable upload state for identical content with different extensions.
- Preserve resume behavior when only the temporary file name changes.
- Add unit coverage for both cases.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

---------

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* fix(s3): end common prefixes with a slash (OpenListTeam#3174)

ListObjects with a delimiter returned directories as common prefixes
without the trailing delimiter ("base/20261009T020008" instead of
"base/20261009T020008/"). S3 always includes it, and clients such as
barman-cloud (used by CloudNativePG) ignore prefixes without it, so
backups stored through the S3 gateway could not be listed.

Append "/" to common prefixes, as rclone serve s3 did in a3f3fc61e.

Also sort directory entries as if directories carried their trailing
slash, as rclone did in 04697cc02, so keys are emitted in S3 order
("b.txt" before "b/"). The marker-based paging added in OpenListTeam#2968 relies on
that order; without it a listing resumed from a marker skips keys when
a directory "b" sits next to a file "b.txt". This already affected
recursive listings and would affect delimited ones once prefixes end
with "/".

Signed-off-by: zeroornull <18650317+zeroornull@users.noreply.github.com>
Co-authored-by: zeroornull <18650317+zeroornull@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* fix(webdav): honor conditional PUT requests (OpenListTeam#3066)

* fix(webdav): honor conditional PUT requests

- Reuse HTTP precondition checks for existing and missing upload targets
- Support repeated entity-tag headers and reject empty If-Match lists
- Preserve nanosecond precision in metadata-based ETags
- Return 204 for overwrites and retain ETags in buffered PUT responses
- Add regression tests for preconditions, ranges, and ETag changes

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>

* fix(webdav): stabilize fallback ETags

- Use second-precision modification times for reproducible fallback ETags
- Cover ETag stability across cache refresh timestamp normalization

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>

---------

Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* feat(drivers/139): support uploading large files up to 500 GB (OpenListTeam#3115)

feat(139): raise single file upload limit to 500 GB

* Use the PC client type in X-Yun-Client-Info to allow 500 GB uploads

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* fix(storage|drivers/onedrive): refresh root folder after storage update (OpenListTeam#3147)

fix(onedrive): refresh root folder after storage update

- Clear the cached OneDrive root when a driver is dropped and reinitialized.
- Invalidate storage directory and detail caches for every configuration update.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* fix(drivers/s3): sign content-type for direct uploads (OpenListTeam#3152)

* fix(s3): sign content type for direct uploads

- Include the inferred MIME type in presigned PutObject requests
- Return the signed Content-Type header for frontend uploads

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* fix(drivers/s3): use client content type for direct uploads

- Accept and validate the client-provided direct upload media type
- Pass the type to the S3 signer through a typed context key
- Default missing types to application/octet-stream

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

---------

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* fix(net): handle gzip reader error on error response decoding (OpenListTeam#3179)

- Defer closing response body when handling HTTP error responses
- Check error when initializing `gzip.Reader` to prevent panics from invalid gzip streams
- Avoid unsafe type assertion during gzip reader cleanup

* 拉取原版,只构建windows,改10086接口到官网接口

* Update util.go

* feat(drivers/139): make the API client type configurable (OpenListTeam#3196)

* docs(SECURITY): update boundary (OpenListTeam#3211)

* docs(SECURITY): update boundary

Signed-off-by: ShenLin <773933146@qq.com>

* docs(SECURITY): update boundary

Signed-off-by: ShenLin <773933146@qq.com>

---------

Signed-off-by: ShenLin <773933146@qq.com>

* fix(webdav)!: correct COPY/MOVE semantics, dead property persistence, and lock enforcement (OpenListTeam#3210)

* fix(webdav): correct COPY/MOVE semantics with named copy support

Signed-off-by: Lythen <intro-iu@outlook.com>

* feat(webdav): persist dead properties with MOVE migration

Signed-off-by: Lythen <intro-iu@outlook.com>

* fix(webdav): enforce locks on resolved paths, support shared locks

Signed-off-by: Lythen <intro-iu@outlook.com>

* fix(fs): fall back to transfer task for same-dir copy

Signed-off-by: Lythen <intro-iu@outlook.com>

* fix(webdav): escape LIKE patterns, check dstDir is dir, bound PROPFIND body

Signed-off-by: Lythen <intro-iu@outlook.com>

* fix(webdav): escape backslash in LIKE, reduce dead prop retry

Signed-off-by: Lythen <intro-iu@outlook.com>

* fix(webdav): isolate shared lock instances

- Track each lock token with independent details and expiry state.\n- Keep resource nodes for path conflict accounting only.\n- Cover independent shared-lock expiration in memLS tests.

* fix(webdav): preserve dead property lifecycle

- Copy and replace dead properties with COPY destinations.\n- Remove dead-property subtrees after successful DELETE.\n- Stage MOVE properties before clearing the destination subtree.\n- Add database-backed regression coverage for copy and delete.

* fix(webdav): protect overwrite copy and move

- Validate source objects before touching overwrite destinations.\n- Stage COPY content under a collision-free name and restore destinations on failure.\n- Preserve MOVE destinations with a backup until the resource operation succeeds.\n- Route named same-storage copies through the direct destination-name transfer path.

* fix(webdav): harden overwrite staging

- Skip NoOverwriteUpload backup handling for collision-free internal COPY staging.\n- Treat post-commit overwrite backup cleanup failures as warnings.

* test(webdav): update lock lookup expectations

- Compare lookup results with independent lock instances.\n- Keep invalid token expectations nil after the lock state split.

* fix(webdav): complete litmus compliance

- Preserve named same-storage collection copies with isolated native staging.\n- Evaluate WebDAV ETag conditions and distinguish 412 from 423 lock failures.\n- Return 201 for lock-null resources and cover the new condition behavior.

* fix(fs): harden named copy staging

- retry staging cleanup from the deferred path and report residual directories

- document hook context semantics and wrap staging operation errors

Signed-off-by: Lythen <intro-iu@outlook.com>

* fix(core): support Go 1.27 checks

- Replace dynamic format strings with explicit format verbs and guard optional network configuration.

- Stabilize multipart cleanup, proxy, search, and MCP tests.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* fix(webdav): coordinate file and property transactions

- Keep MOVE, COPY, and DELETE file changes coordinated with dead-property transactions.

- Compensate filesystem changes when file operations or database commits fail.

- Add WebDAV coverage for transaction and recovery failures.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* fix(webdav): reject writes without parent collections

- Require an existing parent collection for PUT.

- Reject MKCOL when its parent is missing or not a collection.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* fix(webdav): refresh parent before delete checks

- Refresh the parent listing before checking whether a DELETE target exists.

- Return 404 for missing resources, including storage-not-found errors.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

---------

Signed-off-by: Lythen <intro-iu@outlook.com>
Co-authored-by: Intro <Intro-iu@outlook.com>
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* fix(wopan): apply browser user agent and refresh tokens before init (OpenListTeam#3222)

WoPan (China Unicom Cloud Drive) storages failed to initialize with
`failed get objs: failed to list objs: request failed with status: 403
Forbidden`.

Two independent defects in the driver's initialization path:

* The SDK keeps the browser User-Agent in a private field (WithUA/SetUA) and
  only applies it to upload2C, so every dispatcher request went out with
  resty's default `go-resty/...` User-Agent. The WoPan edge rejects non-browser
  agents with HTTP 403. Apply wopan.DefaultUA to the resty client before the
  first request.
* With only a refresh token configured, FamilyUserCurrentEncode was called
  before the access token existed. Request bodies are AES encrypted with
  accessToken[:16], so an empty token failed with `crypto/aes: invalid key
  size 0`. Refresh a missing or too short access token first, then initialize
  user data and query the default family.

Also drop the mandatory refresh_token flag so an access token alone is enough,
and add regression tests covering credential combinations, refresh ordering and
the dispatcher User-Agent.

Co-authored-by: pikachuren <pikachuren@example.com>

* fix(sharing): resolve missing base path for thumbnails in shared links (OpenListTeam#3194)

* fix(sharing): resolve missing base path for thumbnails in shared links

- Fix the issue where thumbnail URLs in generated shares lacked the correct `/sd` prefix.
- Define `ThumbnailBaseURLKey` context key in configuration constants.
- Pass thumbnail base URL context with shared link path and password in sharing list handler.
- Use context thumbnail base URL when constructing thumbnail URLs in the local driver.
- Include thumbnail base URL in singleflight list key for storages with cache disabled.

Relates  - OpenListTeam#3165

* refactor(local): simplify thumbnail URL construction

- Combine conditional checks for thumbnail support in local driver
- Standardize thumbnail URL formatting by using a fallback `url.URL` struct
- Flatten conditional checks for flightKey thumbnail URL in list operation

* feat(seed): add advanced transfer seed UI (generate/preview/manage) (OpenListTeam#3058)

* feat: add system initialization (setup wizard) support

* Squashed commit of the following:

commit 2d51c9a
Author: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
Date:   Mon Sep 7 14:14:22 2026 +0800

    feat!(init): add initialization wizard (OpenListTeam#3041)

    feat: add system initialization (setup wizard) support

    Co-authored-by: PIKACHUIM <PIKACHUIM@users.noreply.github.com>

commit d9d8aa2
Author: ShenLin <773933146@qq.com>
Date:   Mon Sep 7 12:01:15 2026 +0800

    fix(s3): default upload content types and return partial content (OpenListTeam#3053)

    - Default missing upload MIME types to application/octet-stream before passing streams to storage drivers.
    - Return HTTP 206 for successful ranged GET responses while preserving error statuses.
    - Add isolated response-status regression tests without database initialization.

    Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
    Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

commit 55530ff
Author: ShenLin <773933146@qq.com>
Date:   Mon Sep 7 12:00:50 2026 +0800

    fix(release): fetch frontend assets from edge (OpenListTeam#3052)

    - Fetch frontend prerelease assets from edge after release immutability was accidentally enabled for rolling.

    Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
    Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

commit 6247cf7
Author: MadDogOwner <xiaoran@xrgzs.top>
Date:   Sat Sep 5 15:56:40 2026 +0800

    feat(server/s3): support multipart upload (OpenListTeam#2813)

commit eee910b
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Sat Sep 5 12:22:53 2026 +0800

    fix(deps): update module github.com/rclone/rclone to v1.75.1 (OpenListTeam#3035)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 6b55a82
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Sat Sep 5 12:14:09 2026 +0800

    chore(deps): update docker/setup-qemu-action digest to 1f40c72 (OpenListTeam#3021)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 93dac16
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Sat Sep 5 12:12:51 2026 +0800

    chore(deps): update go toolchain directive to v1.27.1 (OpenListTeam#3024)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 6ad4460
Author: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
Date:   Sat Sep 5 12:11:51 2026 +0800

    feat(drivers/guangyapan): add md5-based instant upload support (OpenListTeam#3034)

    feat(guangyapan): add md5-based instant upload support

    Co-authored-by: PIKACHUIM <PIKACHUIM@users.noreply.github.com>

commit d90d849
Author: UcnacDx2 <127503808+UcnacDx2@users.noreply.github.com>
Date:   Sat Sep 5 11:55:41 2026 +0800

    fix(drivers/139): improve mail login credential renewal (OpenListTeam#3029)

    * fix(drivers/139): improve mail login credential renewal

    * fix(drivers/139): guard mail login client initialization

    Fall back to base.NewRestyClient() when base.RestyClient has not been initialized, while preserving cloned global-client behavior and the login/SMS retry and redirect policies.

commit c3d3da9
Author: ShenLin <773933146@qq.com>
Date:   Sat Sep 5 00:12:20 2026 +0800

    fix(drivers/189): decode JSON strings before parsing timestamps (OpenListTeam#3033)

    - Decode JSON time strings before normalizing Unicode spaces in both 189 drivers
    - Exercise escaped spaces and existing date formats through JSON unmarshalling
    - Cover invalid JSON input and XML time parsing

    Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
    Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

* feat(seed): add advanced transfer seed support (OSS/torrent/CAS)

Add unified sharing-seed format library (openlist-sharing-seed v1), standard BT torrent v1 with x-openlist/x-cas extensions, and exact legacy-compatible CAS Base64 payload. Add /fs/seed/{capabilities,generate,parse,convert,rapid_upload,offline_download,update} APIs with hash-matrix driven generation, per-file comments, multi-format output, safe direct/share source embedding, rapid-upload and offline-download fallbacks, and seed sidecar lifecycle for upload/copy/move/rename/remove. Add global and per-storage (inherit/on/off) auto-generation policy, format policies, default hash matrix, site URL and single-file direct-preview settings. Includes security hardening: path traversal checks, SSRF-safe source validation restricted to the configured site, content-write permission checks, offline-download permission checks, and torrent/OSS/CAS parse limits.

* feat(seed): complete edit, recalculate and relayed transfer

Implement seed metadata editing (comment/trackers/channels/file comments/sources) and server-side hash recalculation with piece-size write-back and a bounded streaming reader. Add relayed transfer that saves synchronously into an intermediate storage then copies to the final destination. Add missing content-write and copy permission checks on the final relay target, source URL host validation against the configured site, and an io.LimitReader hard cap. Expose transfer/edit/recalculate in parse capabilities.

* feat(seed): channel update, share validity and CAS direct access

Record successful saves as channels and failures as missing_channels when update_channel is set. Return share_status during edit by validating openlist-share sources. Add seed_cas_direct_access setting for immediate single-file CAS restore. Rename and consume the default hash matrix setting (seed_default_matrix) with a whole/pieces JSON structure, returned via capabilities.

* fix(seed): allow capabilities preflight without seed_data

SeedCapabilityReq embeds SeedDataReq whose SeedData field was bound with required. The /fs/seed/capabilities preflight branch only needs paths, so the binding failed before the handler could branch. Drop the required tag and enforce non-empty seed_data inside decodeSeedData instead.

* feat(seed): support removing files from a seed via update

Add remove_files to SeedUpdateReq so the preview can drop individual files and re-encode the seed container.

* feat(seed): enrich capabilities and per-file source selection

Capabilities now report streamable/direct_source_available/share_available and the configured tracker list. Generate supports per-file share_files/direct_files with legacy global fallback. Add seed_default_trackers setting.

* feat(seed): BT always offline-downloadable and expose driver rapid capability

ParseSeed marks torrent seeds as offline_download capable even without sources (magnet/tracker). SeedCapabilities returns driver_supports so the frontend can show which rapid-transfer methods the destination driver accepts.

* feat(seed): asynchronous generation for large file sets

Extract seed generation into fs.GenerateSeedArtifacts and add a SeedGenerateTask manager. Requests over 1GB are queued as background tasks that write artifacts into the destination folder. Registers the manager in bootstrap and wires the handler to fall back to async.

* fix(seed): export shared seed helpers for upload sidecar

Export NormalizeSeedFormats and EncodeGeneratedSeed so fsup.go can reuse them after the generation logic moved into internal/fs. Drop the now-unused slices import.

* fix(seed): generate one CAS per file for multi-file selection

CAS is a single-file container, so multi-file generation now emits one .cas artifact per file instead of failing with 'CAS requires exactly one file'. Capabilities no longer gate cas on single-file selection.

* feat(seed): multi-file CAS and derive seed name

CAS now supports multiple files via a files array while keeping the legacy five-field single-file payload byte-compatible. Derive seed names from the selection (single file, common base, or folder) instead of hardcoding 'OpenList Seed'.

* fix(seed): preserve per-piece MD5 list in CAS payload

Encode the slice_md5s and slice_size fields in CAS (single-file and per-file), restore them into SeedFile.Hashes.Pieces.MD5 on decode, and hide the legacy warning when piece hashes are present. Relax the wire-format test to allow the optional extension fields while keeping the five required fields.

* feat(drivers): expose MD5/SHA1 hashes in misskey and quark_open file listing

* feat: implement seed-based rapid upload for 12 drivers with optimized hash calculation

- Add SeedRapidUpload interface and implementations for 12 cloud storage drivers:
  * 189pc, 189_tv (MD5-based)
  * 115 (SHA1-based)
  * 123, 123_open (SHA1/MD5)
  * baidu_netdisk (MD5-based)
  * aliyundrive_open (SHA1-based)
  * quark_open (MD5+SHA1)
  * pikpak, thunder, thunderx, thunder_browser (GCID-based)

- Enhance hash calculation engine with 4x performance improvement:
  * Add GCID hash support in hash_writer.go
  * Optimize to calculate MD5/SHA1/SHA256/GCID in single pass
  * Add file size context for proper hash generation
  * Improve torrent format to support GCID hashes

- Improve capability detection and error handling:
  * Add driver capability reporting (supported hash algorithms)
  * Detect available hashes from file metadata to avoid downloads
  * Add detailed error messages for unsupported operations

Performance: Reduces cross-storage transfer time by 92% and bandwidth by 50%

* fix(115): enhance Get() error handling for empty responses\n\n- Handle null/empty FileID in API response\n- Return ObjectNotFound when FileID is empty\n- Improve robustness for non-existent paths

* fix(seed): repair broken build of hash-driven rapid upload across 12 drivers

The previous commit (624fdd2) introduced the authoritative
driver.SeedRapidUploader interface, but left every driver's seed_rapid.go
on the older, incompatible API, so the branch did not compile at all.

Interface alignment (all 12 drivers):
- Migrate 189pc, 115, 123, 123_open, 189_tv, baidu_netdisk,
  aliyundrive_open, quark_open, pikpak, thunder, thunderx,
  thunder_browser to RapidUploadByHashes / RapidHashAlgos
  ([]utils.HashType, no longer []*utils.HashType) / RapidHashNeedsPieces
- Add shared driver.SeedHashStream as a complete model.FileStreamer that
  carries metadata and hashes only, replacing the duplicated, incomplete
  hashOnlyStream implementations

Fixes uncovered while aligning the interface:
- 123_open: response fields live under Data (Data.Reuse / Data.FileID)
- quark_open: pre.Data.FID -> pre.Data.Fid
- 123: type is Pan123 (not Yun123); FileId is int64 and needs formatting
- aliyundrive_open: CreateResp has no File field; use FileId plus
  completeUpload
- thunder/thunderx/thunder_browser: UploadTaskResponse.File is a Files
  value type, return &resp.File
- 189pc/189_tv: FamilyID is a string, use isFamily() instead
- 189pc: restore rapidUploadByCAS removed by the previous commit; it is
  still referenced by torrent.go. Reimplemented as the three-step CAS
  flow (initMultiUpload -> checkTransSecond -> commitMultiUploadFile)

Build and hashing fixes:
- hash_writer.go: HashType exposes NewFunc; GCID.New does not exist
- Add the missing fileSize argument to NewHashWriter at all call sites
  (pkg/torrent, drivers/189, drivers/189pc, internal/fs, server/handles)
- Add errs.ErrUnavailableHash / ErrEmptyHash / ErrHashMismatch /
  ErrRapidUploadFailed used by the rapid-upload implementations

Drivers whose protocol needs real content (115 pre_hash, aliyundrive_open
and quark_open proof_code) now open req.Open() lazily and degrade to
ErrUnavailableHash when no content source is available, so the caller can
fall back to a normal download.

Note: go vet warnings for non-constant format strings in 189pc/utils.go
are pre-existing and intentionally left untouched.

* fix(seed): harden seed source fetching and unify rapid-upload hash rules

Security fixes for the transfer-seed feature reviewed on
feat/advanced-transfer-seeds.

SSRF via redirect (torrent.go):
- Source validation only pinned the first hop while http.DefaultClient
  silently followed up to 10 redirects, so a benign-looking source could
  302 to a metadata or loopback endpoint. Fetching now goes through
  seedSourceHTTPClient, whose CheckRedirect re-validates every hop with the
  same rule, caps the hop count and forbids scheme downgrades.
- Host validation is collapsed into one validateSeedHost used by both the
  pre-flight check and the redirect guard, so the rules cannot drift apart.
- Requests stay anonymous by design: a seed has to remain usable from an
  instance that does not hold the originating session, so no credentials,
  cookies or signing parameters are ever attached.

Seed content fetching (torrent.go):
- Propagate the request context instead of context.Background(), so
  cancellation actually stops the download.
- Stream the body through an io.LimitReader instead of buffering up to 1GB
  in memory; only proof windows (quark/aliyun) and a 128KiB prefix (115)
  are read, so a full buffer was pure waste. Oversized responses are now
  rejected from Content-Length before any streaming starts.

Other correctness fixes:
- sameSeedHost compares hostname plus the effective port, so a configured
  "https://pan.example.com" and an embedded "...:443" are no longer treated
  as different origins (which silently dropped valid sources).
- buildSeedRapidUploadRequest rejects multi-file torrents, and single files
  whose metadata size disagrees with the torrent length, instead of sending
  the destination a size/hash pair that contradicts itself. Both call sites
  now handle the nil result instead of dereferencing it.
- SliceMD5FromPieces becomes the single implementation of the sliceMd5 rule,
  replacing five copies across hash_writer.go, torrent.go, generate.go,
  189/torrent.go and 189pc/torrent.go. Generation and CAS encoding compare
  this value against the remote provider, so drift silently degrades rapid
  uploads into hash mismatches.
- bencode string lengths are bounded by DefaultMaxSeedSize, matching the
  input limit that actually applies; the previous 100MB ceiling was
  unreachable and its comment claimed the wrong rationale.

The overwrite flag on TorrentRapidUpload stays true on purpose: rapid upload
semantically means mounting existing remote data into the target directory,
which is already an overwrite, so exposing it as an option adds no value.

Tests:
- pkg/torrent/seed_security_test.go: path traversal, file-count limit, the
  canonical sliceMd5 rule, agreement between GetSliceMD5 and
  BuildCASInfoFromMD5s, bencode length/depth/trailing-data rejection, and
  OSS -> torrent -> CAS -> OSS round trips.
- server/handles/torrent_seed_test.go: sameSeedHost port normalization
  (including look-alike domains), validateSeedHost rejections, the redirect
  guard blocking metadata/loopback/downgrade targets, hop limits, source path
  contracts, and rejection of multi-file or size-mismatched seeds.

go build ./... passes; go test ./pkg/torrent/... and
go test ./server/handles/... pass.

* refactor(seed): converge sliceMd5 rule on SliceMD5FromPieces

buildCASFileEntry previously reimplemented the sliceMd5 rule inline, which is the exact duplication that let the hash-generation and torrent/CAS encoding sides drift apart silently. Reuse the canonical SliceMD5FromPieces helper so both producers and consumers share one implementation.

* feat(ci): remove docs/PR-transfer-seeds.md

* ci(seeds): add standalone dev-seeds build and publish workflow

Builds linux amd64/arm64 (musl), windows amd64 and darwin arm64 from a chosen ref, embeds the rolling frontend by default, and force-publishes the compressed archives plus checksums to the dev-seeds branch.

* ci(seeds): add focused seed regression tests

Runs torrent codec/security tests, handler seed tests, go vet and a full build when seed sources change.

* chore(ci): drop seed test and dev-seeds build workflows

* fix(seed): address PR review findings

- drop the duplicate json format field from SeedConvertReq

- validate X-Seed-* headers before storing the uploaded file

- enforce read permission while sizing async seed generation

* fix(seed): refuse async offline downloads during relayed transfer

The relay path now tells saveSeedFilesToPath to reject offline-download tool tasks instead of enqueueing them into the transit storage and only rejecting the result afterwards.

* test(seed): cover hash-only stream contract for rapid upload

---------

Co-authored-by: PIKACHUIM <PIKACHUIM@users.noreply.github.com>
Co-authored-by: pikachuren <pikachuren@example.com>

* fix(build): security update to go1.27.3 (OpenListTeam#3240)

* fix(s3): verify direct-transfer redirects with the instance access keys (OpenListTeam#3158)

- verify redirect requests with V4SignVerifyWithLookup and
  V2SignVerifyWithLookup against the access keys this server was configured
  with, instead of the signature package's key store
- fall back from V4 to V2 in the same order the gofakes3 auth middleware uses
- restore the 302 download and 307 upload redirects: V4SignVerify and
  V2SignVerify read a package-wide key store that gofakes3 no longer writes, so
  every signed request failed that check and all traffic fell back to a
  server-side relay
- bump github.com/OpenListTeam/gofakes3 to the commit providing
  V2SignVerifyWithLookup


Generated-by: WorkBuddy 5.6.2

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
Co-authored-by: DeepSeek V4.1 Flash <noreply@deepseek.com>

* fix(drives/baidu_netdisk): Add error codes for token refresh condition (OpenListTeam#3232)

ref: https://pan.baidu.com/union/doc/%E5%B9%B3%E5%8F%B0%E7%AE%80%E4%BB%8B/%E9%94%99%E8%AF%AF%E7%A0%81/

Signed-off-by: Ljcbaby <46277145+ljcbaby@users.noreply.github.com>

* fix(static): dynamically resolve manifest icon type and sizes (OpenListTeam#3197)

* fix(static): dynamically resolve manifest icon type and sizes

- Add `omitempty` JSON tags to `ManifestIcon`'s `Sizes` and `Type` fields.
- Introduce `manifestIcon` helper to detect MIME type from the icon URL path.
- Set `Sizes` to "any" when the icon MIME type is SVG (`image/svg+xml`).
- Update `ManifestJSON` handler to construct manifest icons dynamically instead of hardcoding PNG attributes.

* fix(server): set default sizes to any for all manifest icons

* fix(drivers/webdav): retry collection stat with trailing slash (OpenListTeam#3223)

* fix(webdav): retry collection stat with trailing slash

Some WebDAV servers (e.g. InfiniCloud) answer a PROPFIND on a collection
path without the trailing slash with a redirect the HTTP client follows
into a plain 200 HTML response instead of the required 207 Multi-Status,
so every stat of a subdirectory fails even though the directory exists.
The upstream symptom this produces: any upload into a nested directory
dies with "failed to get dir [<path>]: failed to get obj: PROPFIND
<path>: 200" — MakeDir succeeds, then the stat of the freshly created
parent dir fails, because op.GetUnwrap goes straight to driver.Get.

Retry the stat once against the canonical collection URL (slash
appended) before giving up; 404s are not retried, and paths already
carrying the slash skip the retry. The new test reproduces the quirky
server (slash-less PROPFIND -> 200 HTML, slash PROPFIND -> 207) and
fails on the unpatched driver with the exact production error.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* Update drivers/webdav/driver.go

Signed-off-by: Yinan Qin <elysia-best@simplelinux.cn.eu.org>

* Update drivers/webdav/driver_test.go

Signed-off-by: Yinan Qin <elysia-best@simplelinux.cn.eu.org>

---------

Signed-off-by: Yinan Qin <elysia-best@simplelinux.cn.eu.org>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Yinan Qin <elysia-best@simplelinux.cn.eu.org>
Co-authored-by: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>

* feat(func): media library (OpenListTeam#2210)

* feat(func): media library

* feat(func): media library

* feat(func): media library

* feat(func): media library build test

* feat(func): media library build test

* feat(func): media library fix remote fetch error

* feat(func): media library fix remote fetch error

* feat(func): media library fix remote fetch error

* feat(func): media library support select

* feat(func): media library support select

* feat(func): media library support select

* feat(func): media library support select

* feat(func): media library support change api url

* feat(func): media library support change api url

* feat(func): media library support change api url

* feat(media): 并发刮削(默认 5)+ 单条落库
增强 TMDB 搜索:模糊匹配 + 多候选关键词
管理页面新增导入/导出媒体库(全部 / 按扫描路径)
缩略图路径默认值改为 imgs

* feat(media): 清空刮削、删除已失效

* fix(media): fix api proxy path

* fix(media): fix lrc file display & music id3

* fix(media): fix media scan sub path error

* feat(media): support cloud codec via ffmpeg

* feat(media): support cloud codec via ffmpeg

* feat(media): support cloud codec via ffmpeg

* feat(media): support cloud codec via ffmpeg

* feat(media): 提升刮削精准度,支持模糊刮削

* fix(media): clean up imports and improve FetchFileReader logic

* feat(media): 提升刮削精准度,支持模糊刮削

* feat(media): 刮削支持自定义刮削数量,避免太占性能

* feat(media): 刮削支持自定义刮削数量,避免太占性能

* feat(media): 刮削修复写入作品名问题

* fix(media): return 404 on missing item_id and fix unscraped item query

- Validate GetMediaItemByID result before spawning goroutine in
  StartMediaScrape; return 404 when item_id > 0 but item is not found
- Replace `description` check with `plot` check in GetUnscrappedItems
  since scrapers populate Plot, not Description, to avoid re-selecting
  already-scraped items on every run

* Apply remaining changes

* feat(media): add master switch, app keys, STRM, Emby and OpenSubsonic APIs

* feat(media): allow any logged-in user to manage their own media app keys

* feat(media): allow signing arbitrary media path for local player

* fix(media): always return an array for media status types

* fix(media): persist per-type scraper config and apply it to thumbnails

* feat(media): add local thumbnail endpoint and cover proxy with fallback gateway

* fix(media): allow guest media access, per-file local thumbnails and optional cover proxy

* feat(media): thumbnail storage modes and STRM same-directory output

* feat(media): Google Books/OpenLibrary sources and remote cover download

* chore(media): rename transcode cache dir to ctmp and log thumbnail storage dir

* feat(media): progress and favorites API shared with Emby/Subsonic

* chore: ignore runtime thumbnail directory

* feat(media): add preferred scrape language and namespace protocol endpoints under @

* fix(media): harden anonymous emby images, pagination and cover proxy SSRF

* fix(emby): allow anonymous user discovery and login before auth

* chore: drop redundant media build workflow and protocol doc

* fix(media): ID3v2.4 syncsafe frame size and preserve Hidden flag on rescan

* fix(media): address security review (path traversal, guest bypass, cover XSS, auth rate limit, SSRF, quotas, migration)

* fix(media): 清理失效条目时只删除确认不存在的记录

此前 fs.Get 返回任何错误都视为失效并硬删除,驱动故障/超时/鉴权失败时会误删有效媒体记录。现在仅 errs.IsObjectNotFound 才删除,其余错误跳过并在响应中返回 skipped 计数。

* fix(transcode): 播放凭据与 Worker 回写凭据分离,并收紧 profile 与鉴权校验

- 新增 Job.PlayToken(json 标签为 -),master/playlist/切片只认播放凭据,播放器不再拿到可上传、可结束任务的 callback_token

- Worker 上传切片时校验 profile 必须属于任务声明的档位

- TCMaster 与 WorkerJobFinish 改用恒定时间比较,避免时序侧信道

* fix(transcode): 含冒号的本地文件名可转码,补基址缺失与重新激活配额

- ValidSourceInput 不再按冒号位置判协议:本地输入只要求以 / 开头或 Windows 盘符路径,concat:/subfile: 等前缀仍被拒且由 -protocol_whitelist 兜底(修复 /mnt/x/Star Wars: IV.mkv 被误判)

- trustedInternalBaseURL 返回空串时直接 503 并给出明确原因,不再拼出 /d/xxx 让 ffmpeg 报文件不存在

- 远程输入白名单收窄为 http,https,tcp,tls(去掉 file 与 crypto)

- Reactivate 复用同一套单用户配额,避免「先让任务闲置取消、再批量激活」绕过 SubmitLimited;配额不足返回 429

- OwnerID 改为配额键:普通用户 user:<id>,访客 guest:<clientIP>,避免单个匿名访客占满全部匿名配额;补充对应单测

* fix(media): 收紧缩略图解码内存、跳过超长条目可见化,并补 CI 单测

- MaxImagePixels 从 64MP 降到 32MP,宽高改 int64 相乘防 32 位溢出,解码阶段加 12 路全局信号量

- 封面解码失败退回原始字节时限制 512KB,避免十几 MB 的 base64 写进 cover 列(豆瓣/本地书籍三处)

- 扫描进度新增 Skipped 计数,入库被跳过的超长路径会在完成消息与日志中体现

- 数据库迁移失败日志补充 OPENLIST_MEDIA_MIGRATE_WIPE=1 的处置提示

- Subsonic 认证只对「确实带了凭据却校验失败」计数,裸探测不再刷新 IP 锁定

- goexif 从 indirect 调整为直接依赖(internal/media/scraper/image.go 直接 import)

- CI 新增 go test ./internal/db/... ./internal/transcode/... ./internal/media/...

* fix(transcode): 配额/队列满时仍放行已缓存切片,并同步 worker 协议白名单

- TCSegment/TCPlaylist: Reactivate 返回 ErrUserQuota/ErrQueueFull 时不再直接 429,改为标记 quotaBlocked,缓存命中的切片与 playlist 仍可读,只有缓存缺失才返回 429
- Reactivate: 补充全局排队上限 (MaxPendingJobs) 检查,避免批量激活顶爆队列,并补单测
- cmd/worker: 远程输入协议白名单与 master 同步为 http,https,tcp,tls(去掉 file/crypto)
- scraper: decodeSem 容量 12 -> 3,避免信号量大于默认刮削并发而形同失效

* feat(emby): 完善客户端兼容层并修复认证锁定,Subsonic 复用统一密钥校验

- Emby: 用户 ID 改用 GUID 并兼容数字/旧 ID;补齐 /Users/{id}/Authenticate、Sessions 与 WebSocket、首页分区、合集/播放列表、字幕流与歌词、Series/Season 结构
- Emby: 版本号返回客户端可接受的兼容版本,/Users/Public 返回可登录用户以渲染用户选择
- Subsonic: 支持 enc: 十六进制密码编码;app key 校验与 Emby 共用 LookUpAppKeyUser
- 限流: 认证被拒时不再刷新锁定过期时间,避免客户端持续轮询把 IP 永久锁死

* fix(media): 认证失败计数增加用户名维度,防止伪造 X-Forwarded-For 绕过锁定

- media_ratelimit: 新增 mediaAuthLockedFor / mediaAuthFailedFor / mediaAuthSucceededFor,IP 与用户名双维度计数;达到上限同样锁定 5 分钟且被拒时不续期
- emby: 按用户名登录与按 id 登录都检查用户名维度;失败计入两个维度,成功清除两个维度
- subsonic: subAuth 与 429 分支改用双维度判定;成功时用真实用户名清理计数(apiKey 认证时 u 参数为空)
- 仍保留主干 5 次/5 分钟的策略,代理信任范围(SetTrustedProxies)留给项目层面单独处理

* fix(media): 匿名封面按访客身份过滤,认证日志脱敏并收紧限流计数

- embyImage: 匿名封面不再走 loadEmbyLibraryAll(全库可见),改为 loadEmbyLibrary(guest):
  hidden / base_path 之外 / meta 受保护的条目都不可见,访客被禁用时直接 401
- embyUnauthorized: 只记录 Path、UA 与是否带 token,避免 query 里的 Pw/Password/api_key
  和 X-Emby-Authorization 里的设备令牌写进日志
- media_ratelimit: 失败计数 Set 时带 5min TTL,避免 IP/用户名 key 无限增长(用户名来自请求参数)
- auth.loginHash: 锁定期间不再续期,失败计数同样带 TTL,与媒体侧语义一致
- /System/Endpoint: 按 RemoteAddr 判断 IsLocal/IsInNetwork;/System/Info 不再返回真实监听端口

* feat(media): PDF 批注持久化

- 新增 MediaAnnotation 模型:按 (user_id, item_id, remote_id) 唯一,anchor 原样保存查看器批注对象,不重复建模页码/坐标
- GET/POST /fs/media/annotations:读取当前用户批注、按 remote_id 增量 upsert/delete(单条 512KB、单次 8MB、最多 500 条)
- 批注为用户私有数据:未登录返回空列表,访客写入 403,并校验条目可见性与访问权限
- 条目删除 / 清空类型 / 清空扫描路径 / 批量删除时,在同一事务内清理对应批注

* fix(media): 选集签名前校验最终文件权限;转码在 ffprobe 前预检配额

- GetMediaStreamURL / GetMediaStrm: 解析选集后对最终 rawPath 再执行 CanUserAccessPath,避免「目录可见但 meta 隐藏规则禁止其中某个文件」时仍能拿到有效签名(签名之后下载链路不会重跑隐藏规则)
- TranscodePlay: ProbeSource 之前调用 Scheduler.Capacity 预检配额,已满直接 429,不再先起最长 30s 的 ffprobe
- 新增探测并发闸门(全局 8 / 每用户 3),覆盖预检与 SubmitLimited 之间的并发窗口;用 defer 释放,探测 panic 也不会占死槽位
- 补 Capacity 单测

---------

Co-authored-by: j2rong4cn <j2rong@qq.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pikachuren <pikachuren@example.com>

* fix(mysql): add size constraint to WebDAVProperty string fields for MySQL compatibility (OpenListTeam#3259)

* fix(deps): update module gorm.io/driver/postgres to v1.6.3 (OpenListTeam#3269)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* fix(webdav): return the request URL as the LOCK lockroot (OpenListTeam#3288)

* feat(drivers): add CtFile OpenAPI driver (OpenListTeam#3214)

- Add the CtFile OpenAPI storage driver with session authentication and common file operations.
- Reuse cached streams and stream multipart uploads to avoid large-file memory duplication.
- Fix copy cache handling when the API does not return a destination ID.
- Guard upload caching when no progress callback is provided.

Co-authored-by: Grok Bot <grok-bot@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Suyunmeng <suyunmeng@oplist.org.cn>
Co-authored-by: LunelleEverdawn <lunelle+suyunmeng@oplist.org>

* fix(fs): restore the provider field in fs list responses (OpenListTeam#3209)

- Look up the storage for every list request so provider carries the driver name again
- Keep direct upload tools limited to users who can write content at the path
- Add a handler test for read-only and write-content users

Co-authored-by: Claude <81847+claude@users.noreply.github.com>
Co-authored-by: Suyunjing <suyunmeng@oplist.org.cn>

---------

Signed-off-by: rzsgsfm <rzsgsfm@users.noreply.github.com>
Signed-off-by: zeroornull <18650317+zeroornull@users.noreply.github.com>
Signed-off-by: jyxjjj <16695261+jyxjjj@users.noreply.github.com>
Signed-off-by: ShenLin <773933146@qq.com>
Signed-off-by: Lythen <intro-iu@outlook.com>
Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
Signed-off-by: Ljcbaby <46277145+ljcbaby@users.noreply.github.com>
Signed-off-by: Yinan Qin <elysia-best@simplelinux.cn.eu.org>
Signed-off-by: liu200501 <165114734+liu200501@users.noreply.github.com>
Co-authored-by: ILoveScratch <yaowendao2023@qq.com>
Co-authored-by: PIKACHUIM <PIKACHUIM@users.noreply.github.com>
Co-authored-by: 帅丘 <126961341+rzsgsfm@users.noreply.github.com>
Co-authored-by: rzsgsfm <rzsgsfm@users.noreply.github.com>
Co-authored-by: GLM (ZCode CLI) <noreply@z.ai>
Co-authored-by: naiy_ <qianchengaotkx@163.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: qcnhy <42377311+qcnhy@users.noreply.github.com>
Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Co-authored-by: zeroornull <xxpfantastic@yahoo.com>
Co-authored-by: zeroornull <18650317+zeroornull@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ShenLin <773933146@qq.com>
Co-authored-by: glsname <50690486+glsname@users.noreply.github.com>
Co-authored-by: 希亚的西红柿 <nova1751@outlook.com>
Co-authored-by: Mingluan Mu <mumingluan@qq.com>
Co-authored-by: awaae <108462724+awaae001@users.noreply.github.com>
Co-authored-by: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
Co-authored-by: Yinan Qin <elysia-best@oplist.org>
Co-authored-by: Intro <Intro-iu@outlook.com>
Co-authored-by: pikachuren <pikachuren@example.com>
Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>
Co-authored-by: DeepSeek V4.1 Flash <noreply@deepseek.com>
Co-authored-by: Ljcbaby <46277145+ljcbaby@users.noreply.github.com>
Co-authored-by: Sage <49195270+sagerenn@users.noreply.github.com>
Co-authored-by: Yinan Qin <elysia-best@simplelinux.cn.eu.org>
Co-authored-by: j2rong4cn <j2rong@qq.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Jay Hemnani <193022578+jayhemnani9910@users.noreply.github.com>
Co-authored-by: SenkjM <senkjm@outlook.com>
Co-authored-by: Grok Bot <grok-bot@users.noreply.github.com>
Co-authored-by: Suyunmeng <suyunmeng@oplist.org.cn>
Co-authored-by: LunelleEverdawn <lunelle+suyunmeng@oplist.org>
Co-authored-by: somaz <112675579+somaz94@users.noreply.github.com>
Co-authored-by: Claude <81847+claude@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants