Skip to content

fix(115_open): refresh expired OSS credentials during upload - #3063

Merged
PIKACHUIM merged 1 commit into
OpenListTeam:mainfrom
xqmmcqs:fix/115-open-upload-token-expired
Sep 21, 2026
Merged

PIKACHUIM merged 1 commit into
OpenListTeam:mainfrom
xqmmcqs:fix/115-open-upload-token-expired

Conversation

@xqmmcqs

@xqmmcqs xqmmcqs commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Summary / 摘要

Long-running 115 Open uploads fail when OSS credentials expire because retries keep using the original credentials. Refresh credentials on SecurityTokenExpired and retry the rejected multipart initiation, part upload, or completion once, preserving the upload ID, completed parts, and callbacks.

Part uploads retain their three outer attempts; each can make one additional attempt after refreshing expired credentials. Rewind the current part before resending it and pass the upload context to OSS requests. This PR changes only drivers/115_open/upload.go.

  • This PR has breaking changes.
  • This PR changes public API, config, storage format, or migration behavior.
  • This PR requires corresponding changes in related repositories.

Related repository PRs / 关联仓库 PR: None.

Related Issues / 关联 Issue

Fixes #1376

Testing / 测试

Linux amd64, Go 1.27.1 in Docker:

  • go test -tags=jsoniter ./drivers/115_open -count=1 passed with local, untracked tests. Those tests are not included in this PR. They exercise credential replacement and retries for multipart initiation, part upload, and completion, plus non-expiration errors and refresh failures.
  • git diff --check passed.
  • Before the refresh-closure refactor, driver race tests, a full application/image build, and container /ping and frontend checks passed. The image was not rebuilt after that refactor.
  • A related network-package test, TestNewOSSClientUsesEnvironmentHTTPSProxy, fails because it expects *http.Transport but receives *net.safeTransport. This was also reproduced on the unmodified base checkout.
  • go test ./... — not run; validation focused on the affected driver and related network package.
  • Manual test / 手动测试: Real 115 uploads spanning credential expiry have not been verified.

Checklist / 检查清单

  • I have read CONTRIBUTING.
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
  • I have requested review from relevant maintainers or code owners where applicable.

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.

Tools used / 使用工具:

  • Codex

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Refactoring / 重构

  • Documentation / 文档 — commit and PR descriptions

  • Tests / 测试 — local validation only; test files are not included

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.

  • I can reproduce all AI-assisted content included in this PR without any AI tools.

Codex implemented the change and performed the validation described above at the contributor's request. Human attestations not established during this work remain unchecked.

- Refresh OSS credentials and retry the rejected multipart operation
- Preserve upload IDs, completed parts, callbacks, and outer part retries
- Rewind parts before resending and pass the upload context to OSS requests

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Credential-refresh and multipart retry behavior is safety-sensitive, and real expiry scenarios were not verified.

Pull request overview

Refreshes expired OSS credentials during 115 Open multipart uploads while preserving upload state.

Changes:

  • Refreshes credentials on expiration and retries multipart operations.
  • Rewinds parts before retrying.
  • Preserves upload IDs, completed parts, and callbacks.
File summaries
File Description
drivers/115_open/upload.go Implements credential refresh and targeted multipart retries.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@PIKACHUIM
PIKACHUIM merged commit d894a39 into OpenListTeam:main Sep 21, 2026
12 checks passed
@CloudCauldron

Copy link
Copy Markdown

Thanks for the fix. I noticed a possible remaining credential-expiration case in retryExpiredToken: it refreshes credentials only for SecurityTokenExpired.

Alibaba Cloud's official Chinese OSS documentation, under InvalidAccessKeyId → "The OSS Access Key Id you provided does not exist in our records", identifies expired and invalidated temporary credentials as the cause. Its recommended solution is to obtain new temporary credentials.

My earlier PR #2575 documented SecurityTokenExpired followed by InvalidAccessKeyId during long Baidu → 115 uploads. If OSS returns InvalidAccessKeyId without a preceding SecurityTokenExpired, the current implementation does not trigger credential refresh. Subsequent part-upload retries would continue using the same credentials and could still fail.

Could this STS upload path also perform a bounded credential refresh and retry for the expiration-related InvalidAccessKeyId case? Since InvalidAccessKeyId can also indicate malformed or disabled keys, recovery should remain bounded and persistent errors should still be reported.

@jyxjjj

jyxjjj commented Sep 23, 2026

Copy link
Copy Markdown
Member

Please reference this in a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] 115open驱动上传大文件失败,提示token expired

5 participants