self-hosted/dockerfiles
1
0
Fork 0
mirror of https://github.com/vimagick/dockerfiles.git synced 2025-01-02 03:37:40 +02:00
Code Issues Releases Activity

update litellm

Browse Source
This commit is contained in:
kev 2024-12-27 15:19:24 +08:00
parent 390955387f
commit e6e8aa7a1e
1 changed files with 30 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
litellm/README.md
View File

@ -1,11 +1,13 @@
litellm
=======
OpenAI Proxy Server (LLM Gateway) to call 100+ LLMs in a unified interface & track spend, set budgets per virtual key/user.
[OpenAI Proxy Server][1] (LLM Gateway) to call 100+ LLMs in a unified interface & track spend, set budgets per virtual key/user.
```bash
$ LITELLM_KEY=sk-xxxxxx
$ curl -H "Authorization: Bearer $LITELLM_KEY" http://127.0.0.1:4000/v1/models
$ curl -H "Authorization: Bearer $LITELLM_KEY" http://127.0.0.1:4000/model/info
$ curl http://127.0.0.1:4000/v1/chat/completions \
@ -26,3 +28,30 @@ $ curl http://127.0.0.1:4000/v1/chat/completions \
]
}'
```
## To create [virtual keys][2]
```bash
$ curl --location 'http://127.0.0.1:4000/user/new' \
--header "Authorization: Bearer $LITELLM_KEY" \
--header 'Content-Type: application/json' \
--data-raw '{"user_email": "username@example.com"}'
{
"expires": "2023-12-22T09:53:13.861000Z",
"user_id": "my-unique-id",
"max_budget": 0.0
}
$ curl 'http://127.0.0.1:4000/key/generate' \
--header "Authorization: Bearer $LITELLM_KEY" \
--header 'Content-Type: application/json' \
--data-raw '{"models": ["gpt-4o", "claude-3.5"], "user_id": "my-unique-id"}'
$ curl -H "Authorization: Bearer $LITELLM_KEY" 'http://127.0.0.1:4000/user/info?user_id=my-unique-id'
{
"spend": 0
}
```
[1]: https://docs.litellm.ai/docs/simple_proxy
[2]: https://docs.litellm.ai/docs/proxy/virtual_keys