LocalAI: Difference between revisions
From DWIKI
mNo edit summary |
mNo edit summary |
||
| Line 2: | Line 2: | ||
*[https://localai.io LocalAI homepage] | *[https://localai.io LocalAI homepage] | ||
=HOWTO= | |||
==List models= | |||
curl http://localhost:8080/v1/models | |||
==Audio to text== | ==Audio to text== | ||
*https://localai.io/features/audio-to-text/ | *https://localai.io/features/audio-to-text/ | ||
*https://docs.llamaindex.ai/en/stable/examples/llm/localai/ | *https://docs.llamaindex.ai/en/stable/examples/llm/localai/ | ||
==Apply model== | |||
curl http://localhost:8080/models/apply -H "Content-Type: application/json" -d '{ | |||
"id": "huggingface@TheBloke/Yarn-Mistral-7B-128k-GGUF/yarn-mistral-7b-128k.Q5_K_M.gguf" | |||
}' | |||
=Scripts= | =Scripts= | ||
Revision as of 19:36, 31 March 2024
Links
HOWTO
=List models
curl http://localhost:8080/v1/models
Audio to text
- https://localai.io/features/audio-to-text/
- https://docs.llamaindex.ai/en/stable/examples/llm/localai/
Apply model
curl http://localhost:8080/models/apply -H "Content-Type: application/json" -d '{ "id": "huggingface@TheBloke/Yarn-Mistral-7B-128k-GGUF/yarn-mistral-7b-128k.Q5_K_M.gguf" }'
Scripts
Talk to the chat interface
#!/bin/bash echo -n "Ask me anything: " read A curl -s http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4", "messages": [{"role": "user", "content": "'"$A"'", "temperature": 0.1}] }' |\ jq '.choices[].message.content' | sed 's/\\n/\n/g' | sed 's/\\"/"/g'
