Repository navigation
[mirror #238] cuda: grow the CUDA graph cache cap while it evicts graphs still in use - #10
LeoBorcherding wants to merge 2 commits into
Conversation
Tensor split needs ~2*n_layers+1 graphs per device per shape, so the fixed cap of 64 evicted every graph before its next use and decode ran without graph reuse. Start at 64 and double, up to 2048, whenever the LRU victim was used within the last second. Patch by floewe (unslothai/unsloth#12468).
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Something went wrong. Try again later by commenting “@codex review”. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review |
|
Codex Review: Didn't find any major issues. Swish! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Mirror of unslothai#238 at
5df4722d7on its merge base, for Codex review. Do not merge.