Skip to content

SDK version is painfully out-of-date #186

Open

Description

@Stevie-O

The library hasn't been updated in over a year. There are about two dozen configs that are now missing, including:

  • enable_thinking to enable/disable thinking
  • reasoning budgests
  • over a dozen model loading options, such as CPU MoE offload, spec decoding, autofit, etc.

I'm currently working on bringing all of the missing configs over from the lmstudio-js SDK. There were a couple of things I had questions on, however.

For example, there's a llama.autoFit and an mlx.autoFit and (looking again) yuzu.autoFit and vllm.autoFit. for server->client that's easy (just consider both). what do I do on client->server? Send all of them?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions