Mert
7e18e69c1c
fix(ml): only use openvino if a gpu is available ( #7450 )
...
use `device_type`
2024-02-27 00:45:14 -05:00
Sourav Agrawal
2a75f884d9
Fix Smart Search when using OpenVINO ( #7389 )
...
* Fix external_path loading in OpenVINO EP
* Fix ruff lint
* Wrap block in try finally
* remove static input shape code
* add unit test
* remove unused imports
* remove repeat line
* linting
* formatting
---------
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2024-02-24 18:22:27 -05:00
renovate[bot]
2b8ec30767
chore(deps): update machine-learning (major) ( #6934 )
...
* chore(deps): update machine-learning
* formatting
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2024-02-12 22:29:15 -05:00
Mert
0c4df216d7
feat(ml): improve test coverage ( #7041 )
...
* update e2e
* tokenizer tests
* more tests, remove unnecessary code
* fix e2e setting
* add tests for loading model
* update workflow
* fixed test
2024-02-11 17:58:56 -05:00
Mert
c9b7f4e690
chore(ml): make execution provider log info-level ( #7024 )
...
* change debug log to info
* update test
2024-02-11 06:12:11 +00:00
Mert
79d3342c3d
fix(ml): openvino not working with dynamic axes ( #6871 )
...
* convert to static
* add comment about gross code
* formatting
* fixed test
* fix typing
* cleanup
* formatting
* Revert "formatting"
This reverts commit 073965c47ebb56bd3297
2024-02-03 05:35:26 +00:00
Mert
a84b6f5fb1
feat(ml): conditionally download .armnn models ( #6650 )
2024-01-28 10:31:59 -05:00
Mert
95cfe22866
feat(ml)!: cuda and openvino acceleration ( #5619 )
...
* cuda and openvino ep, refactor, update dockerfile
* updated workflow
* typing fixes
* added tests
* updated ml test gh action
* updated README
* updated docker-compose
* added compute to hwaccel.yml
* updated gh matrix
updated gh matrix
updated gh matrix
updated gh matrix
updated gh matrix
give up
* remove cuda/arm64 build
* add hwaccel image tags to docker-compose
* remove unnecessary quotes
* add suffix to git tag
* fixed kwargs in base model
* armnn ld_library_path
* update pyproject.toml
* add armnn workflow
* formatting
* consolidate hwaccel files, update docker compose
* update hw transcoding docs
* add ml hwaccel docs
* update dev and prod docker-compose
* added armnn prerequisite docs
* support 3.10
* updated docker-compose comments
* formatting
* test coverage
* don't set arena extend strategy for openvino
* working openvino
* formatting
* fix dockerfile
* added type annotation
* add wsl configuration for openvino
* updated lock file
* copy python3
* comment out extends section
* fix platforms
* simplify workflow suffix tagging
* simplify aio transcoding doc
* update docs and workflow for `hwaccel.yml` change
* revert docs
2024-01-21 18:22:39 -05:00
Fynn Petersen-Frey
753292956e
feat(ml): ARMNN acceleration ( #5667 )
...
* feat(ml): ARMNN acceleration for CLIP
* wrap ANN as ONNX-Session
* strict typing
* normalize ARMNN CLIP embedding
* mutex to handle concurrent execution
* make inputs contiguous
* fine-grained locking; concurrent network execution
---------
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2024-01-11 18:26:46 +01:00
Mert
092a23fd7f
feat(server,ml): remove image tagging ( #5903 )
...
* remove image tagging
* updated lock
* fixed tests, improved logging
* be nice
* fixed tests
2023-12-20 20:47:56 -05:00
Mert
935f471ccb
chore(ml): use strict mypy ( #5001 )
...
* improved typing
* improved export typing
* strict mypy & check export folder
* formatting
* add formatting checks for export folder
* re-added init call
2023-11-13 10:18:46 -06:00
Mert
328a58ac0d
feat(ml): add face models ( #4952 )
...
added models to config dropdown
fixed downloading
updated tests
use hf for face models
formatting
2023-11-11 19:04:49 -06:00
Mert
87a0ba3db3
feat(ml): export clip models to ONNX and host models on Hugging Face ( #4700 )
...
* export clip models
* export to hf
refactored export code
* export mclip, general refactoring
cleanup
* updated conda deps
* do transforms with pillow and numpy, add tokenization config to export, general refactoring
* moved conda dockerfile, re-added poetry
* minor fixes
* updated link
* updated tests
* removed `requirements.txt` from workflow
* fixed mimalloc path
* removed torchvision
* cleaner np typing
* review suggestions
* update default model name
* update test
2023-10-31 05:02:04 -05:00
Mert
258b98c262
fix(ml): load models in separate threads ( #4034 )
...
* load models in thread
* set clip mode logs to debug level
* updated tests
* made fixtures slightly less ugly
* moved responses to json file
* formatting
2023-09-09 16:02:44 +07:00
Mert
3cf0f5f11b
fix(ml): model downloading improvements ( #3988 )
...
* handle ort `NoSuchFile` error, stricter file check
* keep exception order
2023-09-06 08:48:40 +07:00
Mert
41461e0d5d
chore(ml): memory optimisations ( #3934 )
2023-08-31 18:30:53 -05:00
Mert
54b2779b79
chore(ml): improved logging ( #3918 )
...
* fixed `minScore` not being set correctly
* apply to init
* don't send `enabled`
* fix eslint warning
* added logger
* added logging
* refinements
* enable access log for info level
* formatting
* merged strings
---------
Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-08-30 08:22:01 +00:00
Mert
bcc36d14a1
feat(ml)!: customizable ML settings ( #3891 )
...
* consolidated endpoints, added live configuration
* added ml settings to server
* added settings dashboard
* updated deps, fixed typos
* simplified modelconfig
updated tests
* Added ml setting accordion for admin page
updated tests
* merge `clipText` and `clipVision`
* added face distance setting
clarified setting
* add clip mode in request, dropdown for face models
* polished ml settings
updated descriptions
* update clip field on error
* removed unused import
* add description for image classification threshold
* pin safetensors for arm wheel
updated poetry lock
* moved dto
* set model type only in ml repository
* revert form-data package install
use fetch instead of axios
* added slotted description with link
updated facial recognition description
clarified effect of disabling tasks
* validation before model load
* removed unnecessary getconfig call
* added migration
* updated api
updated api
updated api
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-08-29 08:58:00 -05:00
Mert
165b91b068
feat(ml)!: switch image classification and CLIP models to ONNX ( #3809 )
2023-08-25 06:28:51 +02:00
Mert
fa57853bd2
fix(ml): clear cache if bad zip file ( #3600 )
2023-08-08 12:33:13 -05:00
Mert
c73832bd9c
refactor(ml): model downloading ( #3545 )
...
* download facial recognition models
* download hf models
* simplified logic
* updated `predict` for facial recognition
* ensure download method is called
* fixed repo_id for clip
* fixed download destination
* use st's own `snapshot_download`
* conditional download
* fixed predict method
* check if loaded
* minor fixes
* updated mypy overrides
* added pytest-mock
* updated tests
* updated lock
2023-08-05 21:45:13 -05:00
Mert
df1e8679d9
chore(ml): added testing and github workflow ( #2969 )
...
* added testing
* github action for python, made mypy happy
* formatted with black
* minor fixes and styling
* test model cache
* cache test dependencies
* narrowed model cache tests
* moved endpoint tests to their own class
* cleaned up fixtures
* formatting
* removed unused dep
2023-06-27 18:21:33 -05:00
Mert
47982641b2
fix(ml): clear model cache on load error ( #2951 )
...
* clear model cache on load error
* updated caught exceptions
2023-06-27 16:01:24 -05:00
Mert
4d3ce0a65e
fixed setting different clip, removed unused stubs ( #2987 )
2023-06-27 12:21:50 -05:00
Mert
a2f5674bbb
refactor(ml): modularization and styling ( #2835 )
...
* basic refactor and styling
* removed batching
* module entrypoint
* removed unused imports
* model superclass, model cache now in app state
* fixed cache dir and enforced abstract method
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-06-24 22:18:09 -05:00
