self-hosted/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2024-12-27 10:58:13 +02:00
Code Issues Releases Activity
3,624 Commits 105 Branches 232 Tags 370 MiB
d67cc00e4e
Commit Graph

23 Commits

Author SHA1 Message Date
DawidPietrykowski
e8b001f62f
feat: preloading of machine learning models (#7540) 2024-03-03 19:48:56 -05:00
Mert
0c4df216d7
feat(ml): improve test coverage (#7041) 
* update e2e

* tokenizer tests

* more tests, remove unnecessary code

* fix e2e setting

* add tests for loading model

* update workflow

* fixed test
 2024-02-11 17:58:56 -05:00
Mert
95cfe22866
feat(ml)!: cuda and openvino acceleration (#5619) 
* cuda and openvino ep, refactor, update dockerfile

* updated workflow

* typing fixes

* added tests

* updated ml test gh action

* updated README

* updated docker-compose

* added compute to hwaccel.yml

* updated gh matrix

updated gh matrix

updated gh matrix

updated gh matrix

updated gh matrix

give up

* remove cuda/arm64 build

* add hwaccel image tags to docker-compose

* remove unnecessary quotes

* add suffix to git tag

* fixed kwargs in base model

* armnn ld_library_path

* update pyproject.toml

* add armnn workflow

* formatting

* consolidate hwaccel files, update docker compose

* update hw transcoding docs

* add ml hwaccel docs

* update dev and prod docker-compose

* added armnn prerequisite docs

* support 3.10

* updated docker-compose comments

* formatting

* test coverage

* don't set arena extend strategy for openvino

* working openvino

* formatting

* fix dockerfile

* added type annotation

* add wsl configuration for openvino

* updated lock file

* copy python3

* comment out extends section

* fix platforms

* simplify workflow suffix tagging

* simplify aio transcoding doc

* update docs and workflow for `hwaccel.yml` change

* revert docs
 2024-01-21 18:22:39 -05:00
Mert
68f52818ae
feat(server): separate face clustering job (#5598) 
* separate facial clustering job

* update api

* fixed some tests

* invert clustering

* hdbscan

* update api

* remove commented code

* wip dbscan

* cleanup

removed cluster endpoint

remove commented code

* fixes

updated tests

minor fixes and formatting

fixed queuing

refinements

* scale search range based on library size

* defer non-core faces

* optimizations

removed unused query option

* assign faces individually for correctness

fixed unit tests

remove unused method

* don't select face embedding

update sql

linting

fixed ml typing

* updated job mock

* paginate people query

* select face embeddings because typeorm

* fix setting face detection concurrency

* update sql

formatting

linting

* simplify logic

remove unused imports

* more specific delete signature

* more accurate typing for face stubs

* add migration

formatting

* chore: better typing

* don't select embedding by default

remove unused import

* updated sql

* use normal try/catch

* stricter concurrency typing and enforcement

* update api

* update job concurrency panel to show disabled queues

formatting

* check jobId in queueAll

fix tests

* remove outdated comment

* better facial recognition icon

* wording

wording

formatting

* fixed tests

* fix

* formatting & sql

* try to fix sql check

* more detailed description

* update sql

* formatting

* wording

* update `minFaces` description

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
 2024-01-18 00:08:48 -05:00
Mert
e2666f0e74
fix(ml): remove unused import (#6356) 
* remove unused import

* formatting
 2024-01-13 00:25:26 -05:00
renovate[bot]
20be42cec0
chore(deps): update machine-learning (#6302) 
* chore(deps): update machine-learning

* fix typing, use new lifespan syntax

* wrap in try / finally

* move log

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
 2024-01-13 05:00:09 +00:00
Mert
d729c863c8
chore(ml): improve shutdown (#5689) 2023-12-14 13:51:24 -06:00
Mert
ddf04a7eb4
chore(ml): increase spool threshold to 64MiB (#5176) 2023-11-20 09:05:35 -06:00
Mert
a6af4892e3
fix(ml): better model unloading (#3340) 
* restart process on inactivity

* formatting

* always update `last_called`

* load models sequentially

* renamed variable, updated docs

* formatting

* made poll env name consistent with model ttl env
 2023-11-16 20:42:44 -06:00
Mert
935f471ccb
chore(ml): use strict mypy (#5001) 
* improved typing

* improved export typing

* strict mypy & check export folder

* formatting

* add formatting checks for export folder

* re-added init call
 2023-11-13 10:18:46 -06:00
Mert
258b98c262
fix(ml): load models in separate threads (#4034) 
* load models in thread

* set clip mode logs to debug level

* updated tests

* made fixtures slightly less ugly

* moved responses to json file

* formatting
 2023-09-09 16:02:44 +07:00
Mert
41461e0d5d
chore(ml): memory optimisations (#3934) 2023-08-31 18:30:53 -05:00
Mert
54b2779b79
chore(ml): improved logging (#3918) 
* fixed `minScore` not being set correctly

* apply to init

* don't send `enabled`

* fix eslint warning

* added logger

* added logging

* refinements

* enable access log for info level

* formatting

* merged strings

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
 2023-08-30 08:22:01 +00:00
Mert
bcc36d14a1
feat(ml)!: customizable ML settings (#3891) 
* consolidated endpoints, added live configuration

* added ml settings to server

* added settings dashboard

* updated deps, fixed typos

* simplified modelconfig

updated tests

* Added ml setting accordion for admin page

updated tests

* merge `clipText` and `clipVision`

* added face distance setting

clarified setting

* add clip mode in request, dropdown for face models

* polished ml settings

updated descriptions

* update clip field on error

* removed unused import

* add description for image classification threshold

* pin safetensors for arm wheel

updated poetry lock

* moved dto

* set model type only in ml repository

* revert form-data package install

use fetch instead of axios

* added slotted description with link

updated facial recognition description

clarified effect of disabling tasks

* validation before model load

* removed unnecessary getconfig call

* added migration

* updated api

updated api

updated api

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
 2023-08-29 08:58:00 -05:00
Mert
165b91b068
feat(ml)!: switch image classification and CLIP models to ONNX (#3809) 2023-08-25 06:28:51 +02:00
Mert
c73832bd9c
refactor(ml): model downloading (#3545) 
* download facial recognition models

* download hf models

* simplified logic

* updated `predict` for facial recognition

* ensure download method is called

* fixed repo_id for clip

* fixed download destination

* use st's own `snapshot_download`

* conditional download

* fixed predict method

* check if loaded

* minor fixes

* updated mypy overrides

* added pytest-mock

* updated tests

* updated lock
 2023-08-05 21:45:13 -05:00
Mert
848ba685eb
fix(ml): race condition when loading models (#3207) 
* sync model loading, disabled model ttl by default

* disable revalidation if model unloading disabled

* moved lock
 2023-07-11 12:01:21 -05:00
Mert
df1e8679d9
chore(ml): added testing and github workflow (#2969) 
* added testing

* github action for python, made mypy happy

* formatted with black

* minor fixes and styling

* test model cache

* cache test dependencies

* narrowed model cache tests

* moved endpoint tests to their own class

* cleaned up fixtures

* formatting

* removed unused dep
 2023-06-27 18:21:33 -05:00
Mert
4d3ce0a65e
fixed setting different clip, removed unused stubs (#2987) 2023-06-27 12:21:50 -05:00
Mert
a2f5674bbb
refactor(ml): modularization and styling (#2835) 
* basic refactor and styling

* removed batching

* module entrypoint

* removed unused imports

* model superclass,  model cache now in app state

* fixed cache dir and enforced abstract method

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
 2023-06-24 22:18:09 -05:00
Zeeshan Khan
34201be74c
feat(ml) backend takes image over HTTP (#2783) 
* using pydantic BaseSetting

* ML API takes image file as input

* keeping image in memory

* reducing duplicate code

* using bytes instead of UploadFile & other small code improvements

* removed form-multipart, using HTTP body

* format code

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
 2023-06-17 22:49:19 -05:00
Mert
d0cc231782
feat(ml): model unloading (#2661) 
* model cache

* fixed revalidation when using cache namespace

* fixed ttl not being set, added lock
 2023-06-06 20:48:51 -05:00
Mert
1e748864c5
chore(ml): updated dockerfile, added typing, packaging (#2642) 
* updated dockerfile, added typing, packaging

apply env change

* added arm64 support

* added ml version pump, second try for arm64

* added linting config to pyproject.toml

* renamed ml input field

* fixed linter config

* fixed dev docker compose
 2023-06-05 09:40:48 -05:00