Add grounded Ollama rewrite flow for chatbot
This commit is contained in:
@@ -57,6 +57,50 @@ Admin auth:
|
||||
|
||||
The app will be available at `http://localhost:4173`.
|
||||
|
||||
## Optional local LLM (Ollama) for grounded rewrites
|
||||
|
||||
You can keep deterministic retrieval as the source-of-truth and optionally rewrite responses with a local model.
|
||||
|
||||
1. Install and run Ollama on your host.
|
||||
2. Pull a small model suited to older hardware, for example:
|
||||
|
||||
```bash
|
||||
ollama pull qwen2.5:3b-instruct
|
||||
```
|
||||
|
||||
3. Start the API with these environment variables:
|
||||
|
||||
```bash
|
||||
CHATBOT_LLM_ENABLED=true
|
||||
CHATBOT_LLM_BASE_URL=http://127.0.0.1:11434
|
||||
CHATBOT_LLM_MODEL=qwen2.5:3b-instruct
|
||||
CHATBOT_LLM_TIMEOUT_MS=25000
|
||||
CHATBOT_LLM_NUM_CTX=2048
|
||||
```
|
||||
|
||||
4. Call the rewrite endpoint from your existing chat flow:
|
||||
|
||||
`POST /api/chatbot-grounded-rewrite`
|
||||
|
||||
Request payload shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"question": "Who was Titus?",
|
||||
"draftAnswer": "Deterministic answer produced by current retrieval/synthesis.",
|
||||
"sources": ["Episode 2 - Introduction to Titus"],
|
||||
"contextChunks": [
|
||||
{
|
||||
"title": "Episode 2 - Introduction to Titus",
|
||||
"sourceLabel": "Episode 2",
|
||||
"content": "Titus was a Gentile..."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
If the endpoint fails or is disabled, keep your deterministic answer and existing fallback behavior.
|
||||
|
||||
Persistent admin saves:
|
||||
|
||||
- Admin updates are written to `data/admin-content.json`.
|
||||
|
||||
Reference in New Issue
Block a user