Replace InspectAI with OpenAI proxy and use vector's proxy

[lotif]

PR Type

Feature

Short Description

Note: first time using agents to make the code for me using the Vibe Crafting skill.

Using OpenAI API instead of InspectAI to make LLM calls so we can use the Vector proxy (or any other proxy).

Additionally: Updated click v8.2.1 -> v8.4.2

Tests Added

Only updating existing tests

Summary by CodeRabbit

• New Features

  • Evaluation requests now use an OpenAI-compatible proxy with per-request API key authentication.
  • Supported model names and response handling have been updated, including improved support for structured responses.
  • Clearer errors are shown for unavailable models, empty responses, and invalid output formats.

• Documentation

  • Updated model lists and authentication guidance across the README, contribution guide, and API documentation.
  • Added guidance for using the OpenAI-compatible proxy.

• Maintenance

  • Updated dependency and security-audit configuration.

[lotif]

@coderabbitai full review

[coderabbitai]

📝 Walkthrough

Walkthrough

The evaluation implementation now uses the OpenAI SDK through the Vector proxy. Model metadata uses unprefixed proxy model IDs, and API keys pass to per-request clients. InspectAI execution and related dependencies were removed. Systematization and UI call sites now send prompt strings directly. Tests mock OpenAI clients and cover request handling, output parsing, and errors. Documentation and repository checks were updated.

Estimated code review effort: 4 (Complex) | ~45 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 30.23% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly identifies the primary change: replacing InspectAI with the OpenAI SDK and proxy-based requests.
Description check	✅ Passed	The description includes all required template sections and explains the implementation and test updates.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• ✅ Committed to branch successfully - (🔄 Check to regenerate)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch replace-inspectai-with-openai-proxy

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plans/2026-08-10-replace-inspectai-with-openai-proxy.md`:
- Around line 30-32: Update the plan step describing InspectAI Sample
replacement to state that Sample is removed and prompt strings are passed
directly to execute_samples_against_model; remove the inaccurate reference to
introducing a local type.

In `@src/aspis/inferencer.py`:
- Around line 97-103: Update evaluate_text and its evaluate endpoint boundary so
the synchronous OpenAI chat.completions.create call does not block the FastAPI
event loop: either use AsyncOpenAI or offload the call to a threadpool, and
configure an explicit request timeout for the model request.

In `@tests/aspis/ui/test_main.py`:
- Line 304: Update the test setup around make_openai_side_effect in test_main.py
so the answer-submission mock returns valid SystematizedConcept payloads, with
each concept dictionary containing title, body, and prompt_template. Keep the
test focused on verifying saved answers while preventing concept generation from
taking the error path.
- Line 175: Update the assertion in the test around the invalid and valid
submissions to verify the OpenAI client mock is called exactly once, replacing
the looser mock_openai.assert_called() check while preserving the existing
input-validation flow.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b9006b27-4570-44e0-a291-4c8a3e11faea

📥 Commits

Reviewing files that changed from the base of the PR and between ff97c2a and cc20028.

⛔ Files ignored due to path filters (1)

• uv.lock is excluded by !**/*.lock

📒 Files selected for processing (14)

• .github/workflows/code_checks.yml
• .gitignore
• CONTRIBUTING.md
• README.md
• plans/2026-08-10-replace-inspectai-with-openai-proxy.md
• pyproject.toml
• src/aspis/api/main.py
• src/aspis/inferencer.py
• src/aspis/systematization.py
• tests/aspis/api/test_main.py
• tests/aspis/manual_test_inferencer.py
• tests/aspis/test_inferencer.py
• tests/aspis/test_systematization.py
• tests/aspis/ui/test_main.py

💤 Files with no reviewable changes (1)

• .github/workflows/code_checks.yml

[coderabbitai]

Note

Docstrings generation - SUCCESS
Generated docstrings and committed to branch replace-inspectai-with-openai-proxy (commit: 40625e3223d986aebb873f7910ebdf6f1e029898)