server : add lora hotswap endpoint

[ngxson]

TODO:

• Update docs
• Add tests

New argument: --lora-init-without-apply

If --lora-init-without-apply is specified, lora adapter will be loaded but not being apply with llama_init_from_gpt_params.

User can apply it later with the POST /lora-adapters endpoint below

New endpoints

GET /lora-adapters

Get list of all adapters. If an adapter is disabled, the scale will be set to 0.

Response:

[
    {
        "id": 0,
        "path": "my_adapter_1.gguf",
        "scale": 0.0
    },
    {
        "id": 1,
        "path": "my_adapter_2.gguf",
        "scale": 0.0
    }
]

POST /lora-adapters

Set list of adapters. To disable an adapter, either remove it from the list below, or set scale to 0.

Request:

[
  {"id": 0, "scale": 0.2},
  {"id": 1, "scale": 0.8}
]

Response:

{ "success": true }

---

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[ngxson]

self note: maybe wait for changes from #8823 and add the list of loaded lora to struct

[Green-Sky]

--lora-no-apply sounds kind of contrived, maybe --lora-available or similar is better.

[ngxson]

I don't get what you mean. The option means "load the adapter to memory, but do not apply it right away"

probably something like --lora-apply-later or --lora-init-without-apply is more stupidly simple to understand?

[ngxson]

@ggerganov I added test and docs to this PR, plus adapt to change from #8823

Could you re-review this? Thank you.