A console application demonstrating Retrieval Augmented Generation (RAG) using SQL Server 2025's vector features and local Ollama models.
- Document Processing: PDF, DOCX, TXT, MD files
- Web Scraping: Extract content from websites
- Local AI: Uses Ollama for embeddings and chat (no API keys needed!)
- Vector Search: SQL Server 2025 vector database with similarity search
- Interactive Chat: Ask questions about your documents
The application supports multiple SQL Server configurations with Windows authentication:
Option 1: SQL Server LocalDB (Recommended for development)
- Install SQL Server LocalDB from Microsoft Download Center
- Or install via Visual Studio installer (SQL Server Express LocalDB)
- No additional configuration needed - Windows authentication is default
Option 2: SQL Server Express
- Download and install SQL Server Express
- During installation, ensure Windows authentication is enabled
- Service will run as
MSSQL$SQLEXPRESS
Option 3: SQL Server 2025 Preview (Full)
- Download and install SQL Server 2025 Preview
- Configure Windows authentication mode during installation
- Service will run as
MSSQLSERVER
- Install Ollama
- Pull the required models:
ollama pull nomic-embed-text
ollama pull llama3.1:8b- Install .NET 10
-
Clone/Open the project
-
Check SQL Server setup (optional):
.\setup-sqlserver.ps1 -
Restore packages:
dotnet restore
-
Connection strings are pre-configured in
appsettings.jsonfor Windows authentication:{ "ConnectionStrings": { "DefaultConnection": "Server=(localdb)\\MSSQLLocalDB;Database=RagPocDb;Integrated Security=true;TrustServerCertificate=true;Connection Timeout=30;", "SqlServerExpress": "Server=.\\SQLEXPRESS;Database=RagPocDb;Integrated Security=true;TrustServerCertificate=true;Connection Timeout=30;", "LocalHost": "Server=localhost;Database=RagPocDb;Integrated Security=true;TrustServerCertificate=true;Connection Timeout=30;" } } -
Run the application:
dotnet run
If you encounter database connection issues:
-
Check SQL Server services are running:
Get-Service | Where-Object {$_.Name -like "*SQL*"}
-
For LocalDB issues:
sqllocaldb info sqllocaldb start MSSQLLocalDB
-
Verify Windows authentication:
- Ensure your Windows user has access to SQL Server
- Check SQL Server is configured for Windows authentication mode
-
Test connection manually:
sqlcmd -S (localdb)\MSSQLLocalDB -E # or sqlcmd -S .\SQLEXPRESS -E # or sqlcmd -S localhost -E
- The app will automatically create the database and tables
- Test Ollama connection (option 6) to ensure models are working
- Process this README.md file (option 1) to test the system:
- Choose option 1 (Process Document)
- Enter the path:
README.mdor the full path to this README file - This will allow you to ask questions about the project configuration
- Option 1: Process local files (PDF, DOCX, TXT, MD)
- Option 2: Process websites by URL
- Option 3: Ask questions about your processed documents
- The system will find relevant chunks and generate answers
- Option 4: List all processed documents
- Option 5: Delete documents from the database
Edit appsettings.json to customize:
{
"Ollama": {
"BaseUrl": "http://localhost:11434",
"EmbeddingModel": "nomic-embed-text",
"ChatModel": "llama3.1:8b"
},
"Rag": {
"ChunkSize": 1000,
"ChunkOverlap": 200,
"MaxContextChunks": 5,
"MaxTokens": 2000
}
}Your 3080 Ti (12GB VRAM) can handle:
- Embedding model:
nomic-embed-text(~140MB) - Chat models:
llama3.1:8b(~4.7GB) - Recommendedmistral:7b(~4.1GB) - Alternativeqwen2.5:7b(~4.4GB) - Another good option
For even better performance, try:
llama3.1:70bif you have enough system RAM to offloadqwen2.5:14bfor better quality with slightly more VRAM usage
Document/Web → Text Extraction → Chunking → Embeddings → SQL Server Vector DB
↓
User Question → Embedding → Vector Search → Context → LLM → Response
VECTORdata type for storing embeddingsVECTOR_DISTANCE()function for similarity search- Vector indexes for performance
- Ensure SQL Server 2025 is running
- Check connection string in appsettings.json
- Verify Windows Authentication or update to use SQL Auth
- Verify Ollama is running:
ollama list - Pull missing models:
ollama pull model-name - Check models are loaded: Use option 6 in the app
- If models don't fit in VRAM, Ollama will use system RAM
- Consider smaller models if needed
- Monitor GPU memory usage
- Research papers (PDF)
- Technical documentation (MD)
- News articles (web scraping)
- Company documents (DOCX)
This is a POC foundation. You can extend it with:
- Web UI (Blazor/ASP.NET Core)
- Better chunking strategies
- Multiple document formats
- Advanced search filtering
- User management
- API endpoints
Enjoy exploring RAG with local models! 🎉