LocalAI: Difference between revisions
From DWIKI
(Created page with "=Scripts= ==Talk to the chat interface== #!/bin/bash echo -n "Ask me anything: " read A curl -s http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4", "messages": [{"role": "user", "content": "'"$A"'", "temperature": 0.1}] }' |\ jq '.choices[].message.content' | sed 's/\\n/\n/g' | sed 's/\\"/"/g'") |
mNo edit summary |
||
| (3 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
=Links= | |||
*[https://localai.io LocalAI homepage] | |||
=HOWTO= | |||
==List models= | |||
curl http://localhost:8080/v1/models | |||
==Audio to text== | |||
*https://localai.io/features/audio-to-text/ | |||
*https://docs.llamaindex.ai/en/stable/examples/llm/localai/ | |||
==Apply model== | |||
curl http://localhost:8080/models/apply -H "Content-Type: application/json" -d '{ | |||
"id": "huggingface@TheBloke/Yarn-Mistral-7B-128k-GGUF/yarn-mistral-7b-128k.Q5_K_M.gguf" | |||
}' | |||
=Scripts= | =Scripts= | ||
==Talk to the chat interface== | ==Talk to the chat interface== | ||
| Line 4: | Line 21: | ||
echo -n "Ask me anything: " | echo -n "Ask me anything: " | ||
read A | read A | ||
curl -s http://localhost:8080/v1/chat/completions \ | curl -s http://localhost:8080/v1/chat/completions \ | ||
-H "Content-Type: application/json" \ | -H "Content-Type: application/json" \ | ||
-d '{ "model": "gpt-4", "messages": [{"role": "user", "content": "'"$A"'", "temperature": 0.1}] }' |\ | -d '{ "model": "gpt-4", "messages": [{"role": "user", "content": "'"$A"'", "temperature": 0.1}] }' |\ | ||
jq '.choices[].message.content' | sed 's/\\n/\n/g' | sed 's/\\"/"/g' | jq '.choices[].message.content' | sed 's/\\n/\n/g' | sed 's/\\"/"/g' | ||
Revision as of 19:36, 31 March 2024
Links
HOWTO
=List models
curl http://localhost:8080/v1/models
Audio to text
- https://localai.io/features/audio-to-text/
- https://docs.llamaindex.ai/en/stable/examples/llm/localai/
Apply model
curl http://localhost:8080/models/apply -H "Content-Type: application/json" -d '{ "id": "huggingface@TheBloke/Yarn-Mistral-7B-128k-GGUF/yarn-mistral-7b-128k.Q5_K_M.gguf" }'
Scripts
Talk to the chat interface
#!/bin/bash echo -n "Ask me anything: " read A curl -s http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4", "messages": [{"role": "user", "content": "'"$A"'", "temperature": 0.1}] }' |\ jq '.choices[].message.content' | sed 's/\\n/\n/g' | sed 's/\\"/"/g'
