Repository navigation
Support for local (self-hosted) LLMs #2
Description
Hi,
The project is very interesting and I love the idea of being able to ask questions about your data.
For more privacy focused people, is it possible to add support for local LLMs? Maybe Ollama support?
Thank you and keep you the great work you're doing! 👏
Reacted by Evan Gallis, Red5d, Rohan Verma, Famku, Dylan and jasonkaplan79
Activity
MODSetter commentedon Aug 23, 2024
Hi @danielnbalasoiu
In case you want to test with Local LLMs You can change the llm in backend code to https://python.langchain.com/v0.2/docs/integrations/providers/ollama/
It should be 1 liner change.
I started this project with Local Models but I faced issues with LLMGraphTransformer when using local models.
I will try to work on local llms..... lets see hopefully in a week or two I start getting good responses 🤞
danielnbalasoiu commentedon Aug 25, 2024
Hi @MODSetter,
Thanks for getting back to me.
When you're saying "change the llm in backend code", how the change should look like?
Change this to point to Ollama's endpoint?
MODSetter commentedon Aug 25, 2024
Hi @danielnbalasoiu
Ignore graph_config & llm_config. These fields doesn't store anything ATM.
Just go to server.py & change all llm (
SurfSense/backend/server.py
Line 67 in 789b483
from openai to Ollama like https://python.langchain.com/v0.2/docs/integrations/llms/ollama/
danielnbalasoiu commentedon Aug 26, 2024
I'll try it. Thanks!
MODSetter commentedon Sep 20, 2024
@danielnbalasoiu
SurfSense supports local LLM's now 👍
Merge pull request #2 from anshul7665/anshulss/buildimage
Merge pull request MODSetter#2 from anshul7665/anshulss/buildimage
Merge pull request #2 from anshul7665/anshulss/buildimage
fix: add Redis error handling to OAuth middleware and poll-token endp…
Merge pull request #1 from itsablabla/devin/1776196384-nextcloud-sso
feat(homepage): turn "Why it is different" into three preview cards