2b37caba03
* feat(ml): introduce support of onnxruntime-rocm for AMD GPU
* try mutex for algo cache
use OrtMutex
* bump versions, run on mich
use 3.12
use 1.19.2
* acquire lock before any changes can be made
guard algo benchmark results
mark mutex as mutable
re-add /bin/sh (?)
use 3.10
use 6.1.2
* use composite cache key
1.19.2
fix variable name
fix variable reference
aaaaaaaaaaaaaaaaaaaa
* bump deps
* disable algo caching
* fix gha
* try ubuntu runner
* actually fix the gha
* update patch
* skip mimalloc preload for rocm
* increase build threads
* increase timeout for rocm
* Revert "increase timeout for rocm"
This reverts commit 2c4452f5d132198ed381a7b262b4a5cab5114b5f.
* attempt migraphx
* set migraphx_home
* Revert "set migraphx_home"
This reverts commit c121d3e48754b3bce100636f8d666deec58a44b7.
* Revert "attempt migraphx"
This reverts commit 521f9fb72dbe506dc6cb8faeb6494817d87265c6.
* migraphx, take two
* bump rocm
* allow cpu
* try only targeting migraphx
* skip tests
* migraph ❌
* known issues
* target gfx900 and gfx1102
* mention `HSA_USE_SVM`
* update lock
* set device id for rocm
---------
Co-authored-by: Mehdi GHESH <mehdi.ghesh@hotmail.fr>
2.8 KiB
Immich Machine Learning
- CLIP embeddings
- Facial recognition
Setup
This project uses uv, so be sure to install it first.
Running uv sync --extra cpu will install everything you need in an isolated virtual environment.
CUDA, ROCM and OpenVINO are supported as acceleration APIs. To use them, you can replace --extra cpu with either of --extra cuda, --extra rocm or --extra openvino. In the case of CUDA, a compute capability of 5.2 or higher is required.
To add or remove dependencies, you can use the commands uv add $PACKAGE_NAME and uv remove $PACKAGE_NAME, respectively.
Be sure to commit the uv.lock and pyproject.toml files with uv lock to reflect any changes in dependencies.
Load Testing
To measure inference throughput and latency, you can use Locust using the provided locustfile.py.
Locust works by querying the model endpoints and aggregating their statistics, meaning the app must be deployed.
You can change the models or adjust options like score thresholds through the Locust UI.
To get started, you can simply run locust --web-host 127.0.0.1 and open localhost:8089 in a browser to access the UI. See the Locust documentation for more info on running Locust.
Note that in Locust's jargon, concurrency is measured in users, and each user runs one task at a time. To achieve a particular per-endpoint concurrency, multiply that number by the number of endpoints to be queried. For example, if there are 3 endpoints and you want each of them to receive 8 requests at a time, you should set the number of users to 24.
Facial Recognition
Acknowledgements
This project utilizes facial recognition models from the InsightFace project. We appreciate the work put into developing these models, which have been beneficial to the machine learning part of this project.
Used Models
- antelopev2
- buffalo_l
- buffalo_m
- buffalo_s
License and Use Restrictions
We have received permission to use the InsightFace facial recognition models in our project, as granted via email by Jia Guo (guojia@insightface.ai) on 18th March 2023. However, it's important to note that this permission does not extend to the redistribution or commercial use of their models by third parties. Users and developers interested in using these models should review the licensing terms provided in the InsightFace GitHub repository.
For more information on the capabilities of the InsightFace models and to ensure compliance with their license, please refer to their official repository. Adhering to the specified licensing terms is crucial for the respectful and lawful use of their work.