Jonathan Jogenfors
4cb0f37918
chore(server): Move library watcher to microservices ( #7533 )
...
* move watcher init to micro
* document watcher recovery
* chore: fix lint
* add try lock
* use global library watch lock
* fix: ensure lock stays on
* fix: mocks
* unit test for library watch lock
* move statement to correct test
* fix: correct return type of try lock
* fix: tests
* add library teardown
* add chokidar error handler
* make event strings an enum
* wait for event refactor
* refactor event type mocks
* expect correct error
* don't release lock in teardown
* chore: lint
* use enum
* fix mock
* fix lint
* fix watcher await
* remove await
* simplify typing
* remove async
* Revert "remove async"
This reverts commit 84ab5abac4
2024-03-07 11:36:53 -06:00
Jonathan Jogenfors
5d377e5b0f
chore(server): eslint await-thenable ( #7545 )
...
* await-thenable
* fix library watchers
* moar eslint
* fix test
* fix typo
* try to remove check void return
* fix checksVoidReturn
* move to domain utils
* remove eslint ignores
* chore: cleanup types
* chore: use logger
* fix: e2e
---------
Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-03-05 17:23:06 -05:00
Jason Rasmussen
3d25d91e77
refactor: library e2e ( #7538 )
...
* refactor: library e2e
* refactor: remove before each usages
2024-02-29 15:10:08 -05:00
Jonathan Jogenfors
efa6efd200
feat(server,web): remove external path nonsense and make libraries admin-only ( #7237 )
...
* remove external path
* open-api
* make sql
* move library settings to admin panel
* Add documentation
* show external libraries only
* fix library list
* make user library settings look good
* fix test
* fix tests
* fix tests
* can pick user for library
* fix tests
* fix e2e
* chore: make sql
* Use unauth exception
* delete user library list
* cleanup
* fix e2e
* fix await lint
* chore: remove unused code
* chore: cleanup
* revert docs
* fix: is admin stuff
* table alignment
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-02-29 12:35:37 -06:00
Jason Rasmussen
74d431f881
refactor(server): format and metadata e2e ( #7477 )
...
* refactor(server): format and metadata e2e
* refactor: on upload success waiting
2024-02-28 10:21:31 +01:00
Jason Rasmussen
807cd245f4
refactor(server): e2e ( #7462 )
...
* refactor: trash e2e
* refactor: asset e2e
2024-02-27 14:04:38 -05:00
Jason Rasmussen
2ebb57cbd4
refactor: e2e client ( #7324 )
2024-02-21 17:34:11 -05:00
Michel Heusschen
0730b54ca9
fix(server): empty/restore trash ( #7161 )
...
* fix(server): empty/restore trash
* add e2e tests
* add e2e tests - part 2
2024-02-17 14:05:43 -06:00
Jason Rasmussen
b648025e2f
refactor: library watching ( #7071 )
2024-02-13 08:48:47 -05:00
Mert
56b0643890
feat(server)!: pgvecto.rs 0.2 and pgvector compatibility ( #6785 )
...
* basic changes
update version check
set ef_search for clip
* pgvector compatibility
Revert "pgvector compatibility"
This reverts commit 2b66a52aa4097dd27da58138c5288fd87cb9b24a.
pgvector compatibility: minimal edition
pgvector startup check
* update extension at startup
* wording
shortened vector extension variable name
* nightly docker
* fixed version checks
* update tests
add tests for updating extension
remove unnecessary check
* simplify `getRuntimeConfig`
* wording
* reindex on minor version update
* 0.2 upgrade testing
update prod compose
* acquire lock for init
* wip vector down on shutdown
* use upgrade helper
* update image tag
* refine restart check
check error message
* test reindex
testing
upstream fix
formatting
fixed reindexing
* use enum in signature
* fix tests
remove unused code
* add reindexing tests
* update to official 0.2
remove alpha from version name
* add warning test if restart required
* update test image to 0.2.0
* linting and test cleanup
* formatting
* update sql
* wording
* handle setting search path for new and existing databases
* handle new db in reindex check
* fix post-update reindexing
* get dim size
* formatting
* use vbase
* handle different db name
* update sql
* linting
* fix suggested env
2024-02-06 21:46:38 -05:00
Jason Rasmussen
b0d7434e7b
fix: library watching ( #6802 )
2024-01-31 11:26:51 -05:00
Jonathan Jogenfors
068e703e88
feat(server): Automatic watching of library folders ( #6192 )
...
* feat: initial watch support
* allow offline files
* chore: ignore query errors when resetting e2e db
* revert db query
* add savepoint
* guard the user query
* chore: openapi and db migration
* wip
* support multiple libraries
* fix tests
* wip
* can now cleanup chokidar watchers
* fix unit tests
* add library watch queue
* add missing init from merge
* wip
* can now filter file extensions
* remove watch api from non job client
* Fix e2e test
* watch library with updated import path and exclusion pattern
* add library watch frontend ui
* case sensitive watching extensions
* can auto watch libraries
* move watcher e2e tests to separate file
* don't watch libraries from a queue
* use event emitters
* shorten e2e test timeout
* refactor chokidar code to filesystem provider
* expose chokidar parameters to config file
* fix storage mock
* set default config for library watching
* add fs provider mocks
* cleanup
* add more unit tests for watcher
* chore: fix format + sql
* add more tests
* move unwatch feature back to library service
* add file event unit tests
* chore: formatting
* add documentation
* fix e2e tests
* chore: fix e2e tests
* fix library updating
* test cleanup
* fix typo
* cleanup
* fixing as per pr comments
* reduce library watch config file
* update storage config and mocks
* move negative event tests to unit tests
* fix library watcher e2e
* make watch configuration global
* remove the feature flag
* refactor watcher teardown
* fix microservices init
* centralize asset scan job queue
* improve docs
* add more tests
* chore: open api
* initialize app service
* fix docs
* fix library watch feature flag
* Update docs/docs/features/libraries.md
Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
* fix: import right app service
* don't be truthy
* fix test speling
* stricter library update tests
* move fs watcher mock to external file
* subscribe to config changes
* docker does not need polling
* make library watch() private
* feat: add configuration ui
---------
Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-31 08:15:54 +00:00
Ben McCann
9c7dee8551
chore: migrate CLI to ESM and vitest ( #6777 )
...
* chore: migrate CLI to ESM and vitest
* fix lint
* update github workflow
* format
2024-01-30 18:23:33 -05:00
Jason Rasmussen
6dca47c629
refactor: no experimental vm modules ( #6719 )
2024-01-29 09:11:02 -06:00
Ben McCann
5cec8d1c63
chore: remove unused test util methods ( #6689 )
...
* chore: remove unused test util methods
* format
2024-01-27 12:27:55 -05:00
Jason Rasmussen
852effa998
refactor(server): e2e ( #6632 )
2024-01-24 17:24:53 -05:00
Aram Akhavan
a972dd4060
fix(server): extraction of Samsung Motionphoto videos ( #6337 )
...
* Fix extraction of samsung motionphoto videos
* Refactor binary tag extraction to the repository to consolidate exiftool usage
* format
* fix linting and swap argument orders
* Fix tag name and conditional order
* Add unit test
* Update server test assets submodule
* Remove old motion photo video assets when a new one is extracted
* delete first, then write
* Include motion photo asset uuid's in the filename
If the filenames are not uniquified, then we can't delete old/corrupt ones
* Fix formatting and fix/add tests
* chore: only use new uuid
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-22 13:04:45 -05:00
Mert
68f52818ae
feat(server): separate face clustering job ( #5598 )
...
* separate facial clustering job
* update api
* fixed some tests
* invert clustering
* hdbscan
* update api
* remove commented code
* wip dbscan
* cleanup
removed cluster endpoint
remove commented code
* fixes
updated tests
minor fixes and formatting
fixed queuing
refinements
* scale search range based on library size
* defer non-core faces
* optimizations
removed unused query option
* assign faces individually for correctness
fixed unit tests
remove unused method
* don't select face embedding
update sql
linting
fixed ml typing
* updated job mock
* paginate people query
* select face embeddings because typeorm
* fix setting face detection concurrency
* update sql
formatting
linting
* simplify logic
remove unused imports
* more specific delete signature
* more accurate typing for face stubs
* add migration
formatting
* chore: better typing
* don't select embedding by default
remove unused import
* updated sql
* use normal try/catch
* stricter concurrency typing and enforcement
* update api
* update job concurrency panel to show disabled queues
formatting
* check jobId in queueAll
fix tests
* remove outdated comment
* better facial recognition icon
* wording
wording
formatting
* fixed tests
* fix
* formatting & sql
* try to fix sql check
* more detailed description
* update sql
* formatting
* wording
* update `minFaces` description
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-18 00:08:48 -05:00
Tom Vincent
984feafb90
fix(server): extract image description ( #6344 )
2024-01-15 11:19:41 -05:00
Jason Rasmussen
bf1dd36fa9
refactor(server): split api and jobs into separate e2e suites ( #6307 )
...
* refactor: domain and infra modules
* refactor(server): e2e tests
2024-01-09 23:04:16 -05:00
