Skip to content

Fix(MInference): fix the search pattern feature - #156

Merged
Huiqiang Jiang (iofu728) merged 1 commit into
mainfrom
hjiang/resupport_search
Jun 17, 2025
Merged

Fix(MInference): fix the search pattern feature#156
Huiqiang Jiang (iofu728) merged 1 commit into
mainfrom
hjiang/resupport_search

Conversation

@iofu728

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes #110 resupport the search pattern feature.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Was this discussed/approved via a Github issue? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

Who can review?

Huiqiang Jiang (@iofu728)

@iofu728 Huiqiang Jiang (iofu728) added bug Something isn't working feature feature labels Jun 17, 2025

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR restores the “search pattern” workflow by introducing an is_search mode, wiring through new configuration flags, and adjusting model patching logic to skip loading existing patterns when searching.

  • Remove accelerate from core requirements.
  • Add is_search, starting_layer, and config_path to attention kwargs and propagate them through forward passes.
  • Adjust model patching to bypass loading best patterns in search mode and update experiment scripts and deps.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
setup.py Drop accelerate from INSTALL_REQUIRES.
minference/modules/minference_forward.py Implement search logic (is_search flag), load/save configs.
minference/modules/forward.py Pass num_hidden_layers into prefill_kwargs.
minference/models_patch.py Skip loading best_pattern when in search mode.
minference/minference_configuration.py Embed is_search, starting_layer, and config_path into attn_kwargs.
experiments/infinite_bench/run_infinitebench.py Standardize _attn_implementation usage.
experiments/infinite_bench/requirements.txt Add evaluation dependencies (jieba, nltk, rouge, etc.).
Comments suppressed due to low confidence (4)

setup.py:37

  • [nitpick] Removing accelerate from runtime requirements may affect users; please update the README or installation docs to explain whether accelerate is now optional or required via extras.
-    "accelerate",

minference/modules/minference_forward.py:636

  • Introduce a new is_search code path; add tests to cover both is_search=True (including config file creation and the error when the search is complete) and is_search=False to ensure backward compatibility.
+    is_search = prefill_kwargs["attn_forward_config"].get("is_search", False)

minference/minference_configuration.py:72

  • [nitpick] The new attn_kwargs now include is_search, starting_layer, and config_path; please update the class docstring or configuration reference to explain these parameters and their effects.
+        self.attn_kwargs = {

minference/models_patch.py:85

  • self.config does not define an is_search attribute; this should reference self.config.attn_kwargs["is_search"] (or add self.is_search = is_search in the config class) to avoid an AttributeError.
if not self.config.is_search:

config_list = json.load(f)
if len(config_list) == num_hidden_layers:
raise RuntimeError(f"Search already completed; see {config_path}.")
print(f"--- Searching layer {layer_idx} ---")

Copilot AI Jun 17, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Using print in a library module can be noisy and hard to control; consider using a logger (e.g., Python's logging module) at DEBUG or INFO level.

Suggested change
print(f"--- Searching layer {layer_idx} ---")
logging.info(f"--- Searching layer {layer_idx} ---")

Copilot uses AI. Check for mistakes.
@iofu728
Huiqiang Jiang (iofu728) deleted the hjiang/resupport_search branch June 17, 2025 02:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working feature feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Question] Pattern Search Not Working Anymore?

2 participants