1
0
mirror of https://github.com/immich-app/immich.git synced 2024-12-23 02:06:15 +02:00
Commit Graph

1323 Commits

Author SHA1 Message Date
Alex
69983ff83a
feat: enhance search (#7127)
* feat: hybrid search

* fixing normal search

* building out the query

* okla

* filters

* date

* order by date

* Remove hybrid search endpoint

* remove search hybrid endpoint

* faces query

* search for person

* search and pagination

* with exif

* with exif

* justify gallery viewer

* memory view

* Fixed userId is null

* openapi and styling

* searchdto

* lint and format

* remove term

* generate sql

* fix test

* chips

* not showing true

* pr feedback

* pr feedback

* nit name

* linting

* pr feedback

* styling

* linting
2024-02-17 11:00:55 -06:00
Michel Heusschen
fab19a8583
fix(server): recognize faces when min. faces is set to 1 (#7144)
* fix(server): recognize face when min. faces is set to 1

* update logic

* clarified log

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2024-02-17 03:32:11 +00:00
Mert
5ff68d4cdb
feat(server): only transcode streams that require it (#7106) 2024-02-14 11:24:39 -05:00
Torbjorn Tyridal
6adff50f0a
feat(server, web): Include partner's photos on map (#7065)
* feat(server): Include partner's photos on map - if included in timeline

* depend on query parameter withPartners

instead of partners.inTimeline

* web: map option to include partners images

* make open-api
2024-02-14 10:07:00 -05:00
renovate[bot]
b2181ee6f1
chore(deps): update dependency @types/node to v20.11.17 (#7102)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-13 22:13:38 +00:00
Alex
4b3f8d1946
feat: Search filtering logic (#6968)
* commit

* controller/service/repository logic

* use enum

* openapi

* suggest people

* suggest place/camera

* cursor hover

* refactor

* Add try catch

* Remove get people with name service

* Remove deadcode

* people selection

* People placement

* sort people

* Update server/src/domain/repositories/metadata.repository.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* pr feedback

* styling

* done

* open api

* fix test

* use string type

* remmove bad merge

* use correct type

* fix test

* fix lint

* remove unused code

* remove unused code

* pr feedback

* pr feedback

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-13 19:54:58 +00:00
renovate[bot]
cd7a475578
fix(deps): update server (#7069)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-13 09:04:01 -05:00
renovate[bot]
338c9b0689
chore(deps): update base-image to v20240213 (major) (#7081)
chore(deps): update base-image to v20240213

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-13 09:02:15 -05:00
Jason Rasmussen
b648025e2f
refactor: library watching (#7071) 2024-02-13 08:48:47 -05:00
renovate[bot]
43e099fb56
fix(deps): update dependency picomatch to v4 (#7073)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-12 23:37:10 -05:00
Mert
e334443919
feat(server, web): smart search filtering and pagination (#6525)
* initial pagination impl

* use limit + offset instead of take + skip

* wip web pagination

* working infinite scroll

* update api

* formatting

* fix rebase

* search refactor

* re-add runtime config for vector search

* fix rebase

* fixes

* useless omitBy

* unnecessary handling

* add sql decorator for `searchAssets`

* fixed search builder

* fixed sql

* remove mock method

* linting

* fixed pagination

* fixed unit tests

* formatting

* fix e2e tests

* re-flatten search builder

* refactor endpoints

* clean up dto

* refinements

* don't break everything just yet

* update openapi spec & sql

* update api

* linting

* update sql

* fixes

* optimize web code

* fix typing

* add page limit

* make limit based on asset count

* increase limit

* simpler import
2024-02-12 20:50:47 -05:00
martin
f1e4fdf175
refactor: authentication on public routes (#6765)
* refactor: authentication on public routes

* fix: remove public user

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* remove unused method

* fix: tests

* fix: useless methods

* fix: tests

* pr feedback

* pr feedback

* chore: cleanup

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-13 01:47:26 +00:00
Alex
dca1bd22df
Change license to AGPLv3 (#7046)
* Change license to AGPLv3

* Readme and documentation

* Package.json

* More replacement

* Added License MIT for previous code

* NIT

* Update LICENSE

Co-authored-by: bo0tzz <git@bo0tzz.me>

* Update cli/LICENSE

Co-authored-by: bo0tzz <git@bo0tzz.me>

* remote mit

---------

Co-authored-by: bo0tzz <git@bo0tzz.me>
2024-02-12 15:31:59 +00:00
Trevor Jex
d7437d31d1
feat(server): restore modified at timestamp after upload, preserve when copying (#7010) 2024-02-11 22:40:34 -06:00
martin
4e31d82bac
feat(server): replace axios dependency by fetch (#7018)
* feat: replace axios dependency by fetch

* pr feedback
2024-02-11 11:15:06 -05:00
shenlong
ec5e4dbcc2
fix(mobile): stack entity handling (#6980)
Co-authored-by: shenlong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
2024-02-09 21:09:43 -06:00
Jonathan Jogenfors
954c1c2ef4
feat(server): server-side checking of duplicate import paths and exclusion patterns (#6993)
validate path and pattern
2024-02-09 13:02:11 -05:00
Jason Rasmussen
90a7f16817
chore(server): remove deprecated endpoints (#6984)
* chore: remove deprecated endpoints

* chore: open api
2024-02-08 16:57:54 -05:00
Manas Adepu
198e8517e5
fix(server): use luxon for deleted date calculation (#6958)
* luxon implementation

* chore: simplify

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-08 16:56:19 -05:00
Jason Rasmussen
bd1fa9377b
refactor(server): asset core (#6985)
refactor: asset core
2024-02-08 16:56:06 -05:00
Jan
4ea15a5b0c
fix(server): check if sidecarPath exists (#6293)
* check if sidecarPath exists

* Revert "check if sidecarPath exists"

This reverts commit 954a1097b870585afee34974d466e51c5172fed9.

* sidecar sync remove dead sidecarPaths and discover new ones

* tests and minor cleanup

* chore: linting

---------

Co-authored-by: Daniel Dietzler <mail@ddietzler.dev>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-07 17:30:38 +00:00
Mert
b31c7681ae
feat(server): optimize face re-queueing (#6961)
* do not defer faces with no matches

* move comment
2024-02-07 09:56:39 -06:00
renovate[bot]
479fca8f02
chore(deps): pin tensorchord/pgvecto-rs docker tag to 9072418 (#6960)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-07 02:51:22 +00:00
Mert
56b0643890
feat(server)!: pgvecto.rs 0.2 and pgvector compatibility (#6785)
* basic changes

update version check

set ef_search for clip

* pgvector compatibility

Revert "pgvector compatibility"

This reverts commit 2b66a52aa4097dd27da58138c5288fd87cb9b24a.

pgvector compatibility: minimal edition

pgvector startup check

* update extension at startup

* wording

shortened vector extension variable name

* nightly docker

* fixed version checks

* update tests

add tests for updating extension

remove unnecessary check

* simplify `getRuntimeConfig`

* wording

* reindex on minor version update

* 0.2 upgrade testing

update prod compose

* acquire lock for init

* wip vector down on shutdown

* use upgrade helper

* update image tag

* refine restart check

check error message

* test reindex

testing

upstream fix

formatting

fixed reindexing

* use enum in signature

* fix tests

remove unused code

* add reindexing tests

* update to official 0.2

remove alpha from version name

* add warning test if restart required

* update test image to 0.2.0

* linting and test cleanup

* formatting

* update sql

* wording

* handle setting search path for new and existing databases

* handle new db in reindex check

* fix post-update reindexing

* get dim size

* formatting

* use vbase

* handle different db name

* update sql

* linting

* fix suggested env
2024-02-06 21:46:38 -05:00
renovate[bot]
f80af067bf
chore(deps): update dependency eslint-plugin-unicorn to v51 (#6952)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-06 23:53:55 +01:00
renovate[bot]
d88ca5fb2a
chore(deps): update server (#6930)
* chore(deps): update server

* chore: generate sql

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Daniel Dietzler <mail@ddietzler.dev>
2024-02-06 22:06:02 +00:00
Michael Manganiello
4164bcfd0d
chore(server): Use ChunkedSet in Access repository (#6943)
This change simplifies the Access repository methods, by using the
`ChunkedSet` decorator. As the methods expect sets, the `chunks` util
needed to be fixed, so it returns chunks of the same type it received.

Now `chunks` is overloaded, to have proper typing based on the input
parameter.
2024-02-06 17:03:25 -05:00
renovate[bot]
61768ce89e
chore(deps): update base-image to v20240206 (major) (#6942)
chore(deps): update base-image to v20240206

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-06 17:01:07 -05:00
Alex
b768eef44d
fix(server): extract duration from video as ISO time (#6863)
* fix(server): extract duration from video as ISO time

* feedback and add test

* fix test
2024-02-02 14:58:13 -06:00
Daniel Dietzler
d3404f927c
feat(server)!: oauth encryption algorithm setting (#6818)
* feat: add oauth signing algorithm setting

* chore: open api

* chore: change default to RS256

* feat: test and clean up

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-02 00:27:54 -05:00
renovate[bot]
b87a6abb2c
fix(deps): update server (#6769)
fix(deps): update dependency @babel/runtime to v7.23.9

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-01 22:20:16 -05:00
Jonathan Jogenfors
f44fa45aa0
chore(server,cli,web): housekeeping and stricter code style (#6751)
* add unicorn to eslint

* fix lint errors for cli

* fix merge

* fix album name extraction

* Update cli/src/commands/upload.command.ts

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>

* es2k23

* use lowercase os

* return undefined album name

* fix bug in asset response dto

* auto fix issues

* fix server code style

* es2022 and formatting

* fix compilation error

* fix test

* fix config load

* fix last lint errors

* set string type

* bump ts

* start work on web

* web formatting

* Fix UUIDParamDto as UUIDParamDto

* fix library service lint

* fix web errors

* fix errors

* formatting

* wip

* lints fixed

* web can now start

* alphabetical package json

* rename error

* chore: clean up

---------

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-01 22:18:00 -05:00
Alex The Bot
07466fa7b7 Version v1.94.1 2024-01-31 19:21:00 +00:00
Alex The Bot
d4c000cb5a Version v1.94.0 2024-01-31 17:01:33 +00:00
Jason Rasmussen
b0d7434e7b
fix: library watching (#6802) 2024-01-31 11:26:51 -05:00
Jonathan Jogenfors
068e703e88
feat(server): Automatic watching of library folders (#6192)
* feat: initial watch support

* allow offline files

* chore: ignore query errors when resetting e2e db

* revert db query

* add savepoint

* guard the user query

* chore: openapi and db migration

* wip

* support multiple libraries

* fix tests

* wip

* can now cleanup chokidar watchers

* fix unit tests

* add library watch queue

* add missing init from merge

* wip

* can now filter file extensions

* remove watch api from non job client

* Fix e2e test

* watch library with updated import path and exclusion pattern

* add library watch frontend ui

* case sensitive watching extensions

* can auto watch libraries

* move watcher e2e tests to separate file

* don't watch libraries from a queue

* use event emitters

* shorten e2e test timeout

* refactor chokidar code to filesystem provider

* expose chokidar parameters to config file

* fix storage mock

* set default config for library watching

* add fs provider mocks

* cleanup

* add more unit tests for watcher

* chore: fix format + sql

* add more tests

* move unwatch feature back to library service

* add file event unit tests

* chore: formatting

* add documentation

* fix e2e tests

* chore: fix e2e tests

* fix library updating

* test cleanup

* fix typo

* cleanup

* fixing as per pr comments

* reduce library watch config file

* update storage config and mocks

* move negative event tests to unit tests

* fix library watcher e2e

* make watch configuration global

* remove the feature flag

* refactor watcher teardown

* fix microservices init

* centralize asset scan job queue

* improve docs

* add more tests

* chore: open api

* initialize app service

* fix docs

* fix library watch feature flag

* Update docs/docs/features/libraries.md

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* fix: import right app service

* don't be truthy

* fix test speling

* stricter library update tests

* move fs watcher mock to external file

* subscribe to config changes

* docker does not need polling

* make library watch() private

* feat: add configuration ui

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-31 08:15:54 +00:00
Markus
87c38d1832
feat(server, web): Added TranscodePolicy "Bitrate higher than max bitrate or not in accepted format" (#6479)
* chore: rebase

* chore: open api

* Add Database-Migration for setting targetCodec as acceptedCodec if it was set by admin

* Add TranscodePolicy setting, to only transcode files with a bitrate higher than set max bitrate

* Rename enum value of TranscodePolicy

* calculate max_bitrate according to "k" and "m" suffix for comparison

* remove migration

* minor changes

* UnitTest for Bitrate Policy

* Fix UnitTest

* Add missing output options

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2024-01-30 20:25:07 -05:00
Ben McCann
9c7dee8551
chore: migrate CLI to ESM and vitest (#6777)
* chore: migrate CLI to ESM and vitest

* fix lint

* update github workflow

* format
2024-01-30 18:23:33 -05:00
martin
1bfef200a5
fix(server): avoid leaking people data on shared links (#6779)
* fix: avoid leaking people data on shared links

* test: add e2e test
2024-01-30 15:34:00 -06:00
renovate[bot]
4290a29107
chore(deps): update base-image to v20240130 (major) (#6756)
chore(deps): update base-image to v20240130

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-30 09:24:44 -06:00
renovate[bot]
40e14fc272
fix(deps): update dependency geo-tz to v8.0.1 (#6750)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-30 08:03:23 -05:00
renovate[bot]
1231d621c4
chore(deps): update server (#6744)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-29 20:58:19 -05:00
t4keda
76f8d030ce
added a configuration option to select the dri node in transcoding (#6376)
* added a configuration option to select the dri node in transcoding

* chore: open api

* refactor: get hawrdware device

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-29 20:40:02 -05:00
Jason Rasmussen
6dca47c629
refactor: no experimental vm modules (#6719) 2024-01-29 09:11:02 -06:00
Mert
ae7f174948
refactor: rename clip -> smart search (#6713) 2024-01-29 09:51:22 -05:00
Mert
e5a70329c9
fix(server): HEVC videos not playing on Apple devices (#6712)
* add hvc1 tag

* formatting
2024-01-28 21:17:20 -05:00
Mert
0770ad15e5
fix(server): do not process faces of deleted assets (#6710)
* return if asset is not found

* add unit test

* formatting
2024-01-29 01:17:54 +00:00
Mert
28806d03e3
fix(server): fix person pagination when deleting (#6707)
fix person delete pagination
2024-01-28 17:02:01 -05:00
martin
fa0913120d
feat(web,server): search people (#5703)
* feat: search peoples

* fix: responsive design

* use existing count

* generate sql file

* fix: tests

* remove visible people

* fix: merge, hide...

* use component

* fix: linter

* chore: regenerate api

* fix: change name when searching for a face

* save search

* remove duplicate

* use enums for query parameters

* fix: increase to 20 for the local search

* use constants

* simplify

* fix: number of people more visible

* fix: merge

* fix: search

* fix: loading spinner position

* pr feedback
2024-01-27 19:54:31 -05:00
Daniel Dietzler
e9333837bf
chore(server): use fs to read package.json (#6692)
use fs readsync instead of js import for reading package json
2024-01-27 17:39:33 -05:00
Zack Pollard
25cad79657
refactor: move asset stacks to their own entity (#6353)
* feat: auto-stack burst photos

* feat: move stacks to asset stack entity

* chore: pin node version with volta in server

* chore: update e2e cases

* chore: cleanup

* feat: migrate existing stacks

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-27 18:52:14 +00:00
Ben McCann
5cec8d1c63
chore: remove unused test util methods (#6689)
* chore: remove unused test util methods

* format
2024-01-27 12:27:55 -05:00
Markus
8aef92affc
feat(server, web): accepted codecs (#6460)
* chore: rebase

* chore: open api

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-26 12:02:56 -05:00
Jason Rasmussen
96b7885583
refactor(server): trash endpoints (#6652)
* refactor(server): trash endpoints

* chore: open api

* chore: fix wrong rename
2024-01-26 11:48:37 -05:00
Jonathan Jogenfors
33757689fe
fix(deps) bump msgpackr (#6673)
bump msgpackr
2024-01-26 16:41:12 +00:00
Jason Rasmussen
7ea55c7236
refactor(server): download endpoints (#6653)
* refactor(server): download controller

* chore: open api

* chore: fix mobile references
2024-01-26 08:19:13 -06:00
renovate[bot]
ae4229b172
chore(deps): update base-image to v20240125 (major) (#6637)
chore(deps): update base-image to v20240125

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-25 20:59:47 -06:00
Jason Rasmussen
b306cf564e
refactor(server): move asset detail endpoint to new controller (#6636)
* refactor(server): move asset by id to new controller

* chore: open api

* refactor: more consolidation

* refactor: asset service
2024-01-25 12:52:21 -05:00
Jason Rasmussen
7fc4abba72
feat(server): sql access checks (#6635) 2024-01-25 10:14:38 -05:00
Mert
bd87eb309c
feat(server): optimize partial facial recognition (#6634)
* optimize partial facial recognition

* add tests

* use map

* bulk insert faces
2024-01-25 01:27:39 -05:00
Jason Rasmussen
852effa998
refactor(server): e2e (#6632) 2024-01-24 17:24:53 -05:00
renovate[bot]
4424f3cb13
fix(deps): update exiftool (#6586)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-24 11:01:46 -05:00
renovate[bot]
160366c5c1
fix(deps): update server (#6588)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-23 18:06:23 -05:00
Jason Rasmussen
bf64e64328
chore(server): remove unused dependency (#6606) 2024-01-23 17:50:39 -05:00
Jason Rasmussen
a00768c9e5
chore(server): remove old device id endpoint (#6578)
* chore: remove old endpoint

* chore: open api

* chore: remove old tests
2024-01-22 20:54:53 -06:00
renovate[bot]
773d093ace
fix(deps): update server (#6587)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-23 02:00:29 +00:00
Aram Akhavan
a972dd4060
fix(server): extraction of Samsung Motionphoto videos (#6337)
* Fix extraction of samsung motionphoto videos

* Refactor binary tag extraction to the repository to consolidate exiftool usage

* format

* fix linting and swap argument orders

* Fix tag name and conditional order

* Add unit test

* Update server test assets submodule

* Remove old motion photo video assets when a new one is extracted

* delete first, then write

* Include motion photo asset uuid's in the filename

If the filenames are not uniquified, then we can't delete old/corrupt ones

* Fix formatting and fix/add tests

* chore: only use new uuid

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-22 13:04:45 -05:00
Jason Rasmussen
7b314f9435
chore(server): sort open api params (#6484)
* chore: sort spec

* chore: open api

* chore(mobile): sort auditDeletes params

---------

Co-authored-by: shenlong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
2024-01-22 11:49:51 -05:00
Jason Rasmussen
bd2dbb4944
fix(web): always use websocket transport (#6564) 2024-01-22 11:37:00 -05:00
renovate[bot]
e6f260c70f
chore(deps): update base-image to v20240118 (major) (#6473)
chore(deps): update base-image to v20240118

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-22 11:07:35 -05:00
Jason Rasmussen
e4277128be
chore: remove unused files and references (#6562) 2024-01-21 22:57:37 -06:00
Daniel Dietzler
607fd39130
fix(server): only calculate quota usage for internal assets (#6556)
only calculate usage for internal assets
2024-01-21 15:48:29 -06:00
Mert
311261bd4e
fix(server): disable sharp file caching (#6542)
don't cache files
2024-01-20 23:10:14 -05:00
Mert
c8b33c00ec
fix(server): use crf-based two pass for vp9 if max bitrate is disabled (#6535)
use crf-based two pass for vp9 if max bitrate is disabled
2024-01-20 15:05:08 -05:00
Alex The Bot
4ebb9974ff Version v1.93.3 2024-01-20 16:03:18 +00:00
Alex The Bot
3dddc6b449 Version v1.93.2 2024-01-19 18:29:04 +00:00
martin
17eaeb695e
feat: smart merge (#6508)
* pr feedback

* fix: tests

* update assets statistics

* pr feedback

* pr feedback

* fix: linter

* pr feedback

* fix: don't limit the smart merge

* pr feedback

* fix: server code

* remove slider

* fix: tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-19 17:52:26 +00:00
Alex
07b874edda
fix(web): revert smart merge (#6504)
* revert smart merge

* fix test

* fix test

* Remove Slider file
2024-01-19 11:34:20 -06:00
Alex The Bot
d5af357992 Version v1.93.1 2024-01-19 15:01:17 +00:00
Alex The Bot
fa0b7c8563 Version v1.93.0 2024-01-19 02:14:46 +00:00
haosu
660b2e908d
feat(format): hif format (#6477) 2024-01-18 11:18:56 -06:00
Mert
68f52818ae
feat(server): separate face clustering job (#5598)
* separate facial clustering job

* update api

* fixed some tests

* invert clustering

* hdbscan

* update api

* remove commented code

* wip dbscan

* cleanup

removed cluster endpoint

remove commented code

* fixes

updated tests

minor fixes and formatting

fixed queuing

refinements

* scale search range based on library size

* defer non-core faces

* optimizations

removed unused query option

* assign faces individually for correctness

fixed unit tests

remove unused method

* don't select face embedding

update sql

linting

fixed ml typing

* updated job mock

* paginate people query

* select face embeddings because typeorm

* fix setting face detection concurrency

* update sql

formatting

linting

* simplify logic

remove unused imports

* more specific delete signature

* more accurate typing for face stubs

* add migration

formatting

* chore: better typing

* don't select embedding by default

remove unused import

* updated sql

* use normal try/catch

* stricter concurrency typing and enforcement

* update api

* update job concurrency panel to show disabled queues

formatting

* check jobId in queueAll

fix tests

* remove outdated comment

* better facial recognition icon

* wording

wording

formatting

* fixed tests

* fix

* formatting & sql

* try to fix sql check

* more detailed description

* update sql

* formatting

* wording

* update `minFaces` description

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-18 00:08:48 -05:00
Mert
9a2fa21b28
fix(server): scale transcoded videos if dimensions are odd (#6461)
scale if odd resolution
2024-01-17 22:16:44 -05:00
Steven Carter
d4146e3e6d
feat(server): provide the ability to search archived photos (#6332)
* Feat: provide the ability to search archived photos

Adds a query parameter (`searchArchived`) to the search URL parameters
to allow the results to contain archived photos.

* chore: rename includeArchived => withArchived

* chore: open api

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-18 02:08:00 +00:00
martin
f0b328fb6b
feat(server, web): smart merge (#5796)
* pr feedback

* fix: tests

* update assets statistics

* pr feedback

* pr feedback

* fix: linter

* pr feedback

* fix: don't limit the smart merge

* pr feedback

* fix: server code

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-18 01:52:11 +00:00
Alex
c55503496f
fix(server): set log level of immich-admin process in boostrap function (#6458)
* fix(server): set log level of immich-admin process in boostrap function

* Remove log
2024-01-17 16:47:46 -05:00
Jason Rasmussen
6f291006e4
fix(server): handle 5 digit years (#6457) 2024-01-17 16:08:38 -05:00
renovate[bot]
0350058689
fix(deps): update server (#6415)
* fix(deps): update server

* chore: fix tests

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-17 18:24:51 +00:00
Alex
78de4f1312
feat(mobile): quota (#6409)
* feat(mobile): quota

* openapi

* user entity update

* Render quota

* refresh usage upon opening the app bar

* stop backup when quota exceed
2024-01-16 20:08:31 -06:00
Jason Rasmussen
abce82e235
fix(server): enable/disable password login on truenas (#6433) 2024-01-16 16:40:09 -05:00
renovate[bot]
76b66e42e1
chore(deps): update base-image to v20240111 (major) (#6355)
chore(deps): update base-image to v20240111

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-16 11:09:59 -05:00
Sushain Cherivirala
7fc1954e2a
fix(server): add filename search (#6394)
Fixes https://github.com/immich-app/immich/issues/5982.

There are basically three options:

1. Search `originalFileName` by dropping a file extension from the query
(if present). Lower fidelity but very easy - just a standard index &
equality.
2. Search `originalPath` by adding an index on `reverse(originalPath)`
and using `starts_with(reverse(query) + "/", reverse(originalPath)`. A
weird index & query but high fidelity.
3. Add a new generated column called `originalFileNameWithExtension` or
something. More storage, kinda jank.

TBH, I think (1) is good enough and easy to make better in the future.
For example, if I search "DSC_4242.jpg", I don't really think it matters
if "DSC_4242.mov" also shows up.

edit: There's a fourth approach that we discussed a bit in Discord and
decided we could switch to it in the future: using a GIN. The minor
issue is that Postgres doesn't tokenize paths in a useful (they're a
single token and it won't match against partial components). We can
solve that by tokenizing it ourselves. For example:

```
immich=# with vecs as (select to_tsvector('simple', array_to_string(string_to_array('upload/library/sushain/2015/2015-08-09/IMG_275.JPG', '/'), ' ')) as vec)  select * from vecs where vec @@ phraseto_tsquery('simple', array_to_string(string_to_array('library/sushain', '/'), ' '));
                                      vec
-------------------------------------------------------------------------------
 '-08':6 '-09':7 '2015':4,5 'img_275.jpg':8 'library':2 'sushain':3 'upload':1
(1 row)
```

The query is also tokenized with the 'split-by-slash-join-with-space'
strategy. This strategy results in `IMG_275.JPG`, `2015`, `sushain` and
`library/sushain` matching. But, `08` and `IMG_275` do not match. The
former is because the token is `-08` and the latter because the
`img_275.jpg` token is matched against exactly.
2024-01-15 14:40:28 -06:00
Tom Vincent
984feafb90
fix(server): extract image description (#6344) 2024-01-15 11:19:41 -05:00
Alex
d096caccac
chore(web): quota enhancement (#6371)
* chore(web): quota enhancement

* show quota in user table

* update quota for single user ioption

* Add a note how to set unlimited storage

* fixed deletion doesn't update quota

* refactor relation

* fixed test

* re-refactor

* update sql

* fix e2e test

* Update server/src/domain/user/user.service.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* revert e2e test

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-15 09:04:29 -06:00
renovate[bot]
2a8cb70c98
fix(deps): update dependency geo-tz to v8 (#6388)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-15 08:27:59 -05:00
cfitzw
deb1f970a8
feat(server, web): quotas (#4471)
* feat: quotas

* chore: open api

* chore: update status box and upload error message

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-12 19:43:36 -05:00
Jason Rasmussen
f4edb6c4bd
feat(server): track metadata extracted at (#6352) 2024-01-12 19:39:45 -05:00
Jason Rasmussen
2439c5ab57
refactor: open api (#6334) 2024-01-12 07:36:27 -05:00
waclaw66
902977f165
fix(server): exif gps decoding (#6138) 2024-01-10 07:36:54 -05:00
Jason Rasmussen
bf1dd36fa9
refactor(server): split api and jobs into separate e2e suites (#6307)
* refactor: domain and infra modules

* refactor(server): e2e tests
2024-01-09 23:04:16 -05:00
Jason Rasmussen
12dc7c48c9
refactor(server): domain and infra modules (#6301) 2024-01-09 17:07:01 -05:00
Alex The Bot
df59b2099f Version v1.92.1 2024-01-08 15:24:38 +00:00
Daniel Dietzler
7cc0904273
feat(server): disable onboarding when config file is set (#6256) 2024-01-08 09:22:26 -06:00
Alex The Bot
319ddfda53 Version v1.92.0 2024-01-08 05:19:11 +00:00
Zack Pollard
5a66314ead
test: small improvements to database init tests (#6232) 2024-01-07 01:53:09 +00:00
Michael Manganiello
e262298090
fix(server): Split database queries based on PostgreSQL bound params limit (#6034)
* fix(server): Split database queries based on PostgreSQL bound params limit

PostgreSQL uses a 16-bit integer to indicate the number of bound
parameters.

This means that the maximum number of parameters for any query is 65535.
Any query that tries to bind more than that (e.g. searching by a list of
IDs) requires splitting the query into multiple chunks.

This change includes refactoring every Repository that runs queries
using a list of ids, and either flattening or merging results.

Fixes #5788, #5997.

Also, potentially a fix for #4648 (at least based on
[this comment](https://github.com/immich-app/immich/issues/4648#issuecomment-1826134027)).

References:

* https://github.com/typeorm/typeorm/issues/7565
* [PostgreSQL message format - Bind](https://www.postgresql.org/docs/15/protocol-message-formats.html#PROTOCOL-MESSAGE-FORMATS-BIND)

* misc: Create Chunked decorator to simplify implementation

* feat: Add ChunkedArray/ChunkedSet decorators
2024-01-06 20:36:12 -05:00
maxer137
6835d4519a
feat(server): add postgres major version check (#6213)
* feat(server): Throw error when PostgreSQL version is not within the supported versions
The pgvecto.rs extension, though not distributed, can be built for PostgreSQL 12 and 13.
An installation of PostgreSQL 12 with the pgvecto.rs extensions installed will not be caught by immich.
This causes immich to attempt to run the database migrations without having a proper environment.
With assertPostgresql the server will throw an error if the PostgreSQL version is not within the supported range.

* Replaced assertion with lesser than comparison
As requested by @zackpollard

* Changed the comparison to use the minPostgresVersion variable.
If we define one we might as well use it. makes changing the versioning later easier

* Added two new tests, modified two existing tests

`should return if minimum supported PostgreSQL and vectors version are installed`:
Check if init returns properly and that getPostgresVersion is called twice

`should thrown an error if PostgreSQL version is below minimum supported version`:
Checks if the init function correctly returns an error

`should suggest image with postgres ${major} if database is ${major}`:
Modified to set MockResolvedValue instead of MockResolvedValueOnce. With the new check we get the PostgreSQL version twice. So it needs to be set during the entire test.

`should not suggest image if postgres version is not in 14, 15 or 16`:
Modified the bounds to [14, 18]. Because values below 14 now will not get called.
Also Modified to call `getPostgresVersion.MockResolvedValueOnce` for twice, because it gets called twice.

* Fixed two mistakes in the jest functions from previous commit #2abcb60

`should thrown an error if PostgreSQL version is below minimum supported version`:
The regex function I wrote mistakingly used the negate function which check that the error *did not* contain the phrase "PostgreSQL". Which is the opposite

`should not suggest image if postgres version is not in 14, 15 or 16`:
confused bounds for a normal javascript array. Changed the test to only check for values above 16. As values below 14 will get thrown out by test `should return if minimum supported PostgreSQL and vectors version are installed`

I apologise for the mistakes in my previous commit.

* Format fix

---------

Co-authored-by: max <wak@vanling.net>
2024-01-06 19:24:09 -05:00
Mert
41a32b4e6b
feat(server): add rw2 support (#6231) 2024-01-06 23:58:04 +00:00
Alex
0750e13d3f
chore(server): set onboarding for existing instances (#6229)
* chore(server): set onboarding for existing instances

* down
2024-01-06 22:18:42 +00:00
Michael Manganiello
8921278447
fix(server): Execute query in AlbumRepository.removeAsset method (#6216)
The current `removeAsset` implementation just builds the query but does
not execute it. That also seems to be the reason the `@GenerateSql`
decorator was commented out.
2024-01-06 11:56:08 -06:00
Emanuel Bennici
d1e16025cf
fix(server): Fix incorrect sorting with search (#6031)
The search results are not sorted by date causing confision.

Fixes #5916
2024-01-04 14:47:09 -06:00
Zack Pollard
4cf1e553d2
feat(server): in upload folder, split the files into folders based on the first four of the files uuid (#6175) 2024-01-04 14:45:16 -06:00
Alex
13ba83dce6
chore(server): better ML error messages (#5914)
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-04 21:34:50 +01:00
martin
aefd93e43a
feat(server): add env for reverse geocoding path (#6163)
* feat: add env for reverse geocoding path

* fix: quote in doc
2024-01-04 13:36:52 +00:00
Alex
18f59f78e3
feat(web): onboarding (#6066)
* feat(web): onboarding

* feat: openapi

* feat: modulization

* feat: page advancing

* Animation

* Add storage templaete settings

* sql

* more style

* Theme

* information and styling

* hide/show table

* Styling

* Update user property

* fix test

* fix test:

* fix e2e

* test

* Update web/src/lib/components/onboarding-page/onboarding-hello.svelte

Co-authored-by: bo0tzz <git@bo0tzz.me>

* naming

* use System Metadata

* better return type

* onboarding using server metadata

* revert previous changes in user entity

* sql

* test web

* fix test server

* server/web test

* more test

* consolidate color theme change logic

* consolidate save button to storage template

* merge main

* fix web

---------

Co-authored-by: bo0tzz <git@bo0tzz.me>
2024-01-04 05:28:32 +00:00
Jason Rasmussen
f8d64be13c
feat(server)!: move welcome message to settings (#6157)
* feat(server): move welcome message to settings

* chore: open api
2024-01-04 05:00:17 +00:00
Jason Rasmussen
7b40c20ea5
chore(server): time bucket sql (#6156) 2024-01-03 22:32:52 -06:00
Jason Rasmussen
317adc5c28
feat(web,server): external domain setting (#6146)
* feat: external domain setting

* chore: open api

* mobile: handle serverconfig-externalDomain

---------

Co-authored-by: shenlong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
2024-01-03 21:54:48 -05:00
renovate[bot]
13fb32513c
fix(deps): update server (#6119)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-03 17:41:30 -05:00
renovate[bot]
23b38a0474
fix(deps): update dependency @nestjs/schedule to v4 (#5348)
* fix(deps): update dependency @nestjs/schedule to v4

* fix: change CronJob in addCronJob to match new type required by nestjs schedule module

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Zack Pollard <zackpollard@ymail.com>
2024-01-01 22:25:25 -05:00
Fynn Petersen-Frey
cc7ba3c21a
feat(server): search across own+partner assets (#5966)
* feat(server): search across own+partner assets

* generate sql

* fix sql parameter
2024-01-01 17:25:22 -05:00
Michael Manganiello
4a5b8c3770
feat(server): Enqueue jobs in bulk (#5974)
* feat(server): Enqueue jobs in bulk

The Job Repository now has a `queueAll` method, that enqueues messages
in bulk (using BullMQ's
[`addBulk`](https://docs.bullmq.io/guide/queues/adding-bulks)),
improving performance when many jobs must be enqueued within the same
operation.

Primary change is in `src/domain/job/job.service.ts`, and other services
have been refactored to use `queueAll` when useful.

As a simple local benchmark, triggering a full thumbnail generation
process over a library of ~1,200 assets and ~350 faces went from
**~600ms** to **~250ms**.

* fix: Review feedback
2024-01-01 15:45:42 -05:00
Jan
7dd88c4114
fix(server): sanitize storagelabel when creating a user #3346 (#5717)
* sanitize storagelabel when creating a user #3346

* code formatting
2024-01-01 15:37:39 -05:00
renovate[bot]
a53b2de3c4
chore(deps): update dependency @types/supertest to v6 (#6099)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-01 19:27:00 +00:00
renovate[bot]
cb3f18bb9f
chore(deps): update base-image to v20231228 (#6095)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-01 13:44:02 -05:00
Jason Rasmussen
03eb5ffc5c
refactor(server): simplify config init process (#5702) 2024-01-01 13:16:44 -05:00
martin
0ed89e61ec
fix: remove archived people from explore (#6091) 2024-01-01 11:07:42 -05:00
Zack Pollard
ca9cad20bc
feat: storage template locking + fix for database locks (#6054)
* fix: locks need to be acquired and released using the same session

* feat: only allow a single storage template operation at a single time

this has been added to avoid possible rare race conditions where two files are moved at once to the same path with the same name, causing our duplicate iterator to not detect them and therefore both files will have the same name and overwrite eachother
2023-12-30 09:09:33 -06:00
Zack Pollard
2e38fa73bf
feat: storage template file move hardening (#5917)
* fix: pgvecto.rs extension breaks typeorm schema:drop command

* fix: parse postgres bigints to javascript number types when selecting data

* feat: verify file size is the same as original asset after copying file for storage template job

* feat: allow disabling of storage template job, defaults to disabled for new instances

* fix: don't allow setting concurrency for storage template migration, can cause race conditions above 1

* feat: add checksum verification when file is copied for storage template job

* fix: extract metadata for assets that aren't visible on timeline
2023-12-29 18:41:33 +00:00
Michael Manganiello
5f6bd4ae7e
fix(server): Reduce number of bound parameters in Access queries (#6015)
* fix(server): Reduce number of bound parameters in Access queries

According to https://github.com/typeorm/typeorm/issues/7565, the
introduction of bulk queries for permission checks could quickly reach
the limit of 65536 bound parameters allowed by the PostgreSQL
connection.

To avoid reaching that limit, this first change refactors the Access
queries that are expanding the set of ids multiple times. For example,
`asset.checkSharedLinkAccess` expands the ids 4 times, so providing just
~16400 ids is enough to break the query.

Refactored queries:

* activity.checkCreateAccess

```sql
-- Before
SELECT "AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
    LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
        ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
    LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
        ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
        AND ("AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL)
WHERE
    (
        (
            "AlbumEntity"."id" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
            AND "AlbumEntity"."isActivityEnabled" = $11
            AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $12
        )
        OR (
            "AlbumEntity"."id" IN ($13, $14, $15, $16, $17, $18, $19, $20, $21, $22)
            AND "AlbumEntity"."isActivityEnabled" = $23
            AND "AlbumEntity"."ownerId" = $24
        )
    )
    AND "AlbumEntity"."deletedAt" IS NULL

-- After
SELECT "album"."id" AS "album_id"
FROM "albums" "album"
    LEFT JOIN "albums_shared_users_users" "album_sharedUsers"
        ON "album_sharedUsers"."albumsId"="album"."id"
    LEFT JOIN "users" "sharedUsers"
        ON "sharedUsers"."id"="album_sharedUsers"."usersId"
        AND "sharedUsers"."deletedAt" IS NULL
WHERE
    "album"."id" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
    AND "album"."isActivityEnabled" = true
    AND (
        "album"."ownerId" = $11
        OR "sharedUsers"."id" = $12
    )
    AND "album"."deletedAt" IS NULL
```

* asset.checkAlbumAccess

```sql
-- Before
SELECT
    "asset"."id" AS "assetId",
    "asset"."livePhotoVideoId" AS "livePhotoVideoId"
FROM "albums" "album"
    INNER JOIN "albums_assets_assets" "album_asset"
        ON "album_asset"."albumsId"="album"."id"
    INNER JOIN "assets" "asset"
        ON "asset"."id"="album_asset"."assetsId"
        AND "asset"."deletedAt" IS NULL
    LEFT JOIN "albums_shared_users_users" "album_sharedUsers"
        ON "album_sharedUsers"."albumsId"="album"."id"
    LEFT JOIN "users" "sharedUsers"
        ON "sharedUsers"."id"="album_sharedUsers"."usersId"
        AND "sharedUsers"."deletedAt" IS NULL
WHERE
    (
        "album"."ownerId" = $1
        OR "sharedUsers"."id" = $2
    )
    AND (
        "asset"."id" IN ($3, $4, $5, $6, $7, $8, $9, $10, $11, $12)
        OR "asset"."livePhotoVideoId" IN ($13, $14, $15, $16, $17, $18, $19, $20, $21, $22)
    )
    AND "album"."deletedAt" IS NULL

-- After
WITH "assetIds" AS (
    SELECT unnest(array[$1, $2, $3, $4, $5, $6, $7, $8, $9, $10])::uuid AS "id"
    FROM (SELECT 1 AS dummy_column) "dummy_table"
)
SELECT
    "asset"."id" AS "assetId",
    "asset"."livePhotoVideoId" AS "livePhotoVideoId"
FROM "albums" "album"
    INNER JOIN "albums_assets_assets" "album_asset"
        ON "album_asset"."albumsId"="album"."id"
    INNER JOIN "assets" "asset"
        ON "asset"."id"="album_asset"."assetsId"
        AND "asset"."deletedAt" IS NULL
    LEFT JOIN "albums_shared_users_users" "album_sharedUsers"
        ON "album_sharedUsers"."albumsId"="album"."id"
    LEFT JOIN "users" "sharedUsers"
        ON "sharedUsers"."id"="album_sharedUsers"."usersId"
        AND "sharedUsers"."deletedAt" IS NULL
WHERE
    (
        "album"."ownerId" = $11
        OR "sharedUsers"."id" = $12
    )
    AND (
        "asset"."id" IN (SELECT id FROM "assetIds")
        OR "asset"."livePhotoVideoId" IN (SELECT id FROM "assetIds")
    )
    AND "album"."deletedAt" IS NULL
```

* asset.checkSharedLinkAccess

```sql
-- Before
SELECT
    "assets"."id" AS "assetId",
    "assets"."livePhotoVideoId" AS "assetLivePhotoVideoId",
    "albumAssets"."id" AS "albumAssetId",
    "albumAssets"."livePhotoVideoId" AS "albumAssetLivePhotoVideoId"
FROM "shared_links" "sharedLink"
    LEFT JOIN "albums" "album"
        ON "album"."id"="sharedLink"."albumId"
        AND "album"."deletedAt" IS NULL
    LEFT JOIN "shared_link__asset" "assets_sharedLink"
        ON "assets_sharedLink"."sharedLinksId"="sharedLink"."id"
    LEFT JOIN "assets" "assets"
        ON "assets"."id"="assets_sharedLink"."assetsId"
        AND "assets"."deletedAt" IS NULL
    LEFT JOIN "albums_assets_assets" "album_albumAssets"
        ON "album_albumAssets"."albumsId"="album"."id"
    LEFT JOIN "assets" "albumAssets"
        ON "albumAssets"."id"="album_albumAssets"."assetsId"
        AND "albumAssets"."deletedAt" IS NULL
WHERE
    "sharedLink"."id" = $1
    AND (
        "assets"."id" IN ($2, $3, $4, $5, $6, $7, $8, $9, $10, $11)
        OR "albumAssets"."id" IN ($12, $13, $14, $15, $16, $17, $18, $19, $20, $21)
        OR "assets"."livePhotoVideoId" IN ($22, $23, $24, $25, $26, $27, $28, $29, $30, $31)
        OR "albumAssets"."livePhotoVideoId" IN ($32, $33, $34, $35, $36, $37, $38, $39, $40, $41)
    )

-- After
WITH "assetIds" AS (
    SELECT unnest(array[$1, $2, $3, $4, $5, $6, $7, $8, $9, $10])::uuid AS "id"
    FROM (SELECT 1 AS dummy_column) "dummy_table"
)
SELECT
    "assets"."id" AS "assetId",
    "assets"."livePhotoVideoId" AS "assetLivePhotoVideoId",
    "albumAssets"."id" AS "albumAssetId",
    "albumAssets"."livePhotoVideoId" AS "albumAssetLivePhotoVideoId"
FROM "shared_links" "sharedLink"
    LEFT JOIN "albums" "album"
        ON "album"."id"="sharedLink"."albumId"
        AND "album"."deletedAt" IS NULL
    LEFT JOIN "shared_link__asset" "assets_sharedLink"
        ON "assets_sharedLink"."sharedLinksId"="sharedLink"."id"
    LEFT JOIN "assets" "assets"
        ON "assets"."id"="assets_sharedLink"."assetsId"
        AND "assets"."deletedAt" IS NULL
    LEFT JOIN "albums_assets_assets" "album_albumAssets"
        ON "album_albumAssets"."albumsId"="album"."id"
    LEFT JOIN "assets" "albumAssets"
        ON "albumAssets"."id"="album_albumAssets"."assetsId"
        AND "albumAssets"."deletedAt" IS NULL
    WHERE
        "sharedLink"."id" = $11
        AND (
            "assets"."id" IN (SELECT id FROM "assetIds")
            OR "albumAssets"."id" IN (SELECT id FROM "assetIds")
            OR "assets"."livePhotoVideoId" IN (SELECT id FROM "assetIds")
            OR "albumAssets"."livePhotoVideoId" IN (SELECT id FROM "assetIds")
        )
```

* fix: Use array overlapping instead of CTEs
2023-12-27 23:50:54 -06:00
renovate[bot]
4fff2c75aa
chore(deps): update base-image to v20231221 (#6029)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-12-27 23:47:27 -06:00
Mert
a1e1f11399
feat(server): delete unnecessary encoded videos (#6027)
* delete unnecessary transcodes

* added test
2023-12-27 23:34:00 -06:00
Mert
8119d4bb26
chore(server): refactor locks (#5953)
* lock refactor

* add mocks

* add await

* move database repo injection to service

* update tests

* add mock implementation

* remove unused imports

* this
2023-12-27 18:36:51 -05:00
Thomas
c0ebc943d2
fix(server): strip metadata from thumbnails (#4438)
included all thumbnail metadata. It seems this has to be explicitly disabled.

Refs: #4382

feat. basic metadata e2e test

fix: use tiff thumbnails in first step + e2e fix

fix: revert switch to tiff

feat: test metadata of both webp and jpg

feat: use upload in e2e test

fix: lint

strip metadata with exiftool

use `withIccProfile`

fix e2e

formatting

run jobs in e2e
2023-12-26 16:27:51 -05:00
Daniel Dietzler
8fdd3aaed1
fix(server): init library scanning on start up (#5951) 2023-12-23 20:46:42 +00:00
Alex
e47e25e671
fix(server): access system config before database migration complete (#5912) 2023-12-21 12:52:49 -06:00
Mert
cc2dc12f6c
fix(server): run migrations after database checks (#5832)
* run migrations after checks

* optional migrations

* only run checks in server and e2e

* re-add migrations for microservices

* refactor

* move e2e init

* remove assert from migration

* update providers

* update microservices app service

* fixed logging

* refactored version check, added unit tests

* more version tests

* don't use mocks for sut

* refactor tests

* suggest image only if postgres is 14, 15 or 16

* review suggestions

* fixed regexp escape

* fix typing

* update migration
2023-12-21 10:06:26 -06:00
Mert
f602295bf9
chore(dev): move envs to image (#5906) 2023-12-21 09:28:23 -06:00
Mert
092a23fd7f
feat(server,ml): remove image tagging (#5903)
* remove image tagging

* updated lock

* fixed tests, improved logging

* be nice

* fixed tests
2023-12-20 20:47:56 -05:00
martin
7e216809f3
fix(server): remove shared link with removed asset (#5845) 2023-12-19 11:05:18 -06:00
Mohamed BOUSSAID
234449f3c6
fix(server, web): Prevent the user from setting a future date of birth (#5803)
* Hide the person age if it is negative

* Add validation to prevent future birth dates

* Add comment

* Add test, Add birth date validation and update birth date modal

* Add birthDate validation in PersonService and SetBirthDateModal

* Running npm run format:fix

* Generating the migration file propoerly, and Make the birthdate form logic simpler

* Make birthDate type only string

* Adding useLocationPin back
2023-12-19 10:07:38 -06:00
Alex The Bot
b40859551b Version v1.91.4 2023-12-19 03:34:19 +00:00
Jonathan Jogenfors
4e9b96ff1a
test(cli): e2e testing (#5101)
* Allow building and installing cli

* feat: add format fix

* docs: remove cli folder

* feat: use immich scoped package

* feat: rewrite cli readme

* docs: add info on running without building

* cleanup

* chore: remove import functionality from cli

* feat: add logout to cli

* docs: add todo for file format from server

* docs: add compilation step to cli

* fix: success message spacing

* feat: can create albums

* fix: add check step to cli

* fix: typos

* feat: pull file formats from server

* chore: use crawl service from server

* chore: fix lint

* docs: add cli documentation

* chore: rename ignore pattern

* chore: add version number to cli

* feat: use sdk

* fix: cleanup

* feat: album name on windows

* chore: remove skipped asset field

* feat: add more info to server-info command

* chore: cleanup

* wip

* chore: remove unneeded packages

* e2e test can start

* git ignore for geocode in cli

* add cli e2e to github actions

* can do e2e tests in the cli

* simplify e2e test

* cleanup

* set matrix strategy in workflow

* run npm ci in server

* choose different working directory

* check out submodules too

* increase test timeout

* set node version

* cli docker e2e tests

* fix cli docker file

* run cli e2e in correct folder

* set docker context

* correct docker build

* remove cli from dockerignore

* chore: fix docs links

* feat: add cli v2 milestone

* fix: set correct cli date

* remove submodule

* chore: add npmignore

* chore(cli): push to npm

* fix: server e2e

* run npm ci in server

* remove state from e2e

* run npm ci in server

* reshuffle docker compose files

* use new e2e composes in makefile

* increase test timeout to 10 minutes

* make github actions run makefile e2e tests

* cleanup github test names

* assert on server version

* chore: split cli e2e tests into one file per command

* chore: set cli release working dir

* chore: add repo url to npmjs

* chore: bump node setup to v4

* chore: normalize the github url

* check e2e code in lint

* fix lint

* test key login flow

* feat: allow configurable config dir

* fix session service tests

* create missing dir

* cleanup

* bump cli version to 2.0.4

* remove form-data

* feat: allow single files as argument

* add version option

* bump dependencies

* fix lint

* wip use axios as upload

* version bump

* cApiTALiZaTiON

* don't touch package lock

* wip: don't use job queues

* don't use make for cli e2e

* fix server e2e

* chore: remove old gha step

* add npm ci to server

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-12-18 20:29:26 -06:00
Mert
de1514a441
chore(server): startup check for pgvecto.rs (#5815)
* startup check for pgvecto.rs

* prefilter after assertion

* formatting

* add assert to migration

* more specific import

* use runner
2023-12-18 10:38:25 -06:00
Jason Rasmussen
d3e1572229
fix(server): file sending and cache control (#5829)
* fix: file sending

* fix: tests
2023-12-18 10:33:46 -06:00
Alex
3beeffaaf0
fix(server): metadata search does not return all EXIF info (#5810)
* docs: update default config content

* fix(server): metadata search does not return all EXIF info

* remove console log

* generate sql

* Correct sql generation
2023-12-18 07:13:36 -06:00
Mert
b520955d0e
fix(server): add more conditions to smart search (#5806)
* add more asset conditions

* udpate sql
2023-12-17 20:17:30 -06:00
Mert
6e7b3d6f24
fix(server): fix metadata search not working (#5800)
* don't require ml

* update e2e

* fixes

* fix e2e

* add additional conditions

* select all exif columns

* more fixes

* update sql
2023-12-17 20:16:08 -06:00
Michael Manganiello
c6f56d9591
chore(server): Check activity permissions in bulk (#5775)
Modify Access repository, to evaluate `asset` permissions in bulk.
This is the last set of permission changes, to migrate all of them to
run in bulk!
Queries have been validated to match what they currently generate for single ids.

Queries:

* `activity` owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "activity" "ActivityEntity"
  WHERE
    "ActivityEntity"."id" = $1
    AND "ActivityEntity"."userId" = $2
)
LIMIT 1

-- After
SELECT "ActivityEntity"."id" AS "ActivityEntity_id"
FROM "activity" "ActivityEntity"
WHERE
  "ActivityEntity"."id" IN ($1)
  AND "ActivityEntity"."userId" = $2
```

* `activity` album owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "activity" "ActivityEntity"
    LEFT JOIN "albums" "ActivityEntity__ActivityEntity_album"
      ON "ActivityEntity__ActivityEntity_album"."id"="ActivityEntity"."albumId"
      AND "ActivityEntity__ActivityEntity_album"."deletedAt" IS NULL
  WHERE
    "ActivityEntity"."id" = $1
    AND "ActivityEntity__ActivityEntity_album"."ownerId" = $2
)
LIMIT 1

-- After
SELECT "ActivityEntity"."id" AS "ActivityEntity_id"
FROM "activity" "ActivityEntity"
  LEFT JOIN "albums" "ActivityEntity__ActivityEntity_album"
    ON "ActivityEntity__ActivityEntity_album"."id"="ActivityEntity"."albumId"
    AND "ActivityEntity__ActivityEntity_album"."deletedAt" IS NULL
WHERE
  "ActivityEntity"."id" IN ($1)
  AND "ActivityEntity__ActivityEntity_album"."ownerId" = $2
```

* `activity` create access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "albums" "AlbumEntity"
    LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
    LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
      AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
  WHERE
    (
      (
        "AlbumEntity"."id" = $1
        AND "AlbumEntity"."isActivityEnabled" = $2
        AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
      )
      OR (
        "AlbumEntity"."id" = $4
        AND "AlbumEntity"."isActivityEnabled" = $5
        AND "AlbumEntity"."ownerId" = $6
      )
    )
    AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1

-- After
SELECT "AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
  LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
    ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
  LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
    ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
    AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
  (
    (
      "AlbumEntity"."id" IN ($1)
      AND "AlbumEntity"."isActivityEnabled" = $2
      AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
    )
    OR (
      "AlbumEntity"."id" IN ($4)
      AND "AlbumEntity"."isActivityEnabled" = $5
      AND "AlbumEntity"."ownerId" = $6
    )
  )
  AND "AlbumEntity"."deletedAt" IS NULL
```
2023-12-17 12:10:21 -06:00
Mert
e3fa32ad23
fix(server): fix inconsistent explore queries (#5774)
* remove limits

* update sql
2023-12-17 11:04:35 -06:00