metrics: bump CSE meta cache metrics and fix prefetch cache hit ratio panel

[JaySon-Huang]

What problem does this PR solve?

Issue Number: close #11039

Problem Summary:

Bump cloud-storage-engine so TiFlash picks up the new columnar meta cache metrics, and fix TiFlash Summary which incorrectly plots tiflash_proxy_kv_engine_columnar_prefetch_cache_hit as a duration (seconds). That metric observes hit_ratio * 100 with percent buckets, so the panel should use a percent Y-axis and a ratio title.

What is changed and how it works?

metrics: bump CSE for columnar meta cache metrics and fix prefetch cache hit panel unit.

• Update contrib/cloud-storage-engine submodule (includes columnar meta cache metrics) and regenerate related Cargo.lock / proxy AGENTS notes for workspace-hack resolution.
• Change TiFlash Summary panel Columnar Prefetch Cache Hit Duration → Columnar Prefetch Cache Hit Ratio with unit="percent", then regenerate dashboard JSON/checksum.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Manual test:

1. Regenerate dashboard: cd metrics/grafana && ./generate_dashboard.sh
2. Confirm generated panel title is Columnar Prefetch Cache Hit Ratio and left Y-axis format is percent.
3. Import/update test Grafana dashboard and verify the panel reads as a hit-ratio distribution (0–100%), not seconds. Verified on http://10.2.12.81:21530/d/jmd6Payvz/jsonnet-tiflash-summary.
4. Confirm proxy/CSE build still works with the bumped submodule (prior sync commit notes verified libtiflash_proxy.so build / no jemalloc symbols).

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

TiFlash Summary: show columnar prefetch cache hit as percent ratio; bump CSE for columnar meta cache metrics.

Summary by CodeRabbit

• Monitoring

  • Updated the Columnar Storage dashboard to show prefetch cache hit ratio as a percentage, replacing the previous duration display.

• Documentation

  • Added troubleshooting guidance for lockfile updates after cloud-storage-engine revisions.
  • Clarified allocator-related build behavior.

• Maintenance

  • Updated the cloud-storage-engine revision.
  • Refined repository ignore patterns.
  • Refreshed the dashboard configuration checksum.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR updates the cloud-storage-engine submodule and related build guidance. It narrows contribution ignore rules. It also changes the prefetch cache Grafana panel from duration to percentage-based hit ratio display.

Changes

Cloud-storage-engine integration

Layer / File(s)	Summary
Revision and contribution scope .cursorignore, contrib/cloud-storage-engine	The submodule points to a newer revision. Ignore rules target AWS-related paths and contrib/grpc.
Build recovery guidance contrib/tiflash-columnar-hub/AGENTS.md	The guide documents Cargo.lock recovery and jemalloc behavior after cloud-storage-engine revisions.

Prefetch cache dashboard

Layer / File(s)	Summary
Prefetch cache ratio panel metrics/grafana/tiflash_summary.dashboard.py, metrics/grafana/tiflash_summary.json, metrics/grafana/tiflash_summary.json.sha256	The panel title now identifies cache hit ratio. The Y-axis uses percent units. The generated dashboard checksum is updated.

Estimated code review effort: 2 (Simple) | ~10 minutes

Mergeability Score: 🟡 Moderate · up to e6d54

The PR updates the cloud-storage-engine dependency and Grafana panel, but the dependency override and lockfile recovery path remain inconsistent; a future allocator reference could introduce conflicting malloc/free symbols and crash TiFlash, so merge should wait for this build/runtime risk to be corrected.

Possibly related PRs

• pingcap/tiflash#10988: Updates the related cloud-storage-engine revision.
• pingcap/tiflash#11022: Modifies the same Grafana dashboard file.
• pingcap/tiflash#11027: Updates the dashboard source and generated JSON for the prefetch cache hit-ratio panel.

Suggested reviewers: yongman, jinhelin, windtalker

Poem

A rabbit checks the cache at dawn,
Duration labels hop along.
Percent signs now mark the way,
A newer engine joins the day.
Cargo notes guide every hare.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Out of Scope Changes check	⚠️ Warning	The .cursorignore change narrows contrib exclusions but is not tied to the linked CSE or dashboard objectives.	Remove the .cursorignore change or document why these ignore patterns are required for the CSE bump and dashboard work.
Linked Issues check	❓ Inconclusive	The PR addresses the submodule and dashboard objectives, but Cargo.lock refresh cannot be verified because the file is excluded by !**/*.lock.	Include Cargo.lock in the review or provide generated-lock evidence, then confirm the bumped CSE exposes the required meta-cache metrics.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes both primary changes: the CSE metrics bump and the prefetch cache hit-ratio panel fix.
Description check	✅ Passed	The description includes the required problem, changes, issue, checklist, test steps, side effects, documentation, and release note sections.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@contrib/tiflash-columnar-hub/AGENTS.md`:
- Line 124: Update the error code fence in AGENTS.md around the documented
section to include a language identifier after the opening fence, using text or
console, so the markdownlint MD040 violation is resolved.
- Around line 132-137: Update the lockfile recovery guidance to use the existing
explicit list of nine cloud-storage-engine packages, followed by cargo check
--locked, instead of relying only on workspace-hack. Clarify that these packages
resolve from local paths, so git-source revision searches are invalid; document
workspace-hack under [[patch.unused]] and that hub-runtime does not use the
local replacement. Correct the jemalloc documentation and add the text language
tag to the error code fence.

Apply the same fix in `@contrib/tiflash-columnar-hub/AGENTS.md` at line 162.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: eea3c6f5-8f15-4b63-a5ac-32cae32be52c

📥 Commits

Reviewing files that changed from the base of the PR and between 5c72bfa and e6d5422.

⛔ Files ignored due to path filters (1)

• contrib/tiflash-columnar-hub/Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (6)

• .cursorignore
• contrib/cloud-storage-engine
• contrib/tiflash-columnar-hub/AGENTS.md
• metrics/grafana/tiflash_summary.dashboard.py
• metrics/grafana/tiflash_summary.json
• metrics/grafana/tiflash_summary.json.sha256

[JaySon-Huang]

/test pull-integration-test

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JinheLin, yongman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [JinheLin,yongman]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2026-08-13 07:18:56.68859431 +0000 UTC m=+3290122.724689366: ☑️ agreed by yongman.
• 2026-08-13 07:23:14.440518384 +0000 UTC m=+3290380.476613440: ☑️ agreed by JinheLin.

[JaySon-Huang]

/cherry-pick release-nextgen-202603

[ti-chi-bot]

@JaySon-Huang: new pull request created to branch release-nextgen-202603: #11045.

Details

In response to this:

/cherry-pick release-nextgen-202603

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.