1
0
mirror of https://github.com/immich-app/immich.git synced 2024-11-30 09:47:31 +02:00
Commit Graph

254 Commits

Author SHA1 Message Date
martin
fa0913120d
feat(web,server): search people (#5703)
* feat: search peoples

* fix: responsive design

* use existing count

* generate sql file

* fix: tests

* remove visible people

* fix: merge, hide...

* use component

* fix: linter

* chore: regenerate api

* fix: change name when searching for a face

* save search

* remove duplicate

* use enums for query parameters

* fix: increase to 20 for the local search

* use constants

* simplify

* fix: number of people more visible

* fix: merge

* fix: search

* fix: loading spinner position

* pr feedback
2024-01-27 19:54:31 -05:00
Zack Pollard
25cad79657
refactor: move asset stacks to their own entity (#6353)
* feat: auto-stack burst photos

* feat: move stacks to asset stack entity

* chore: pin node version with volta in server

* chore: update e2e cases

* chore: cleanup

* feat: migrate existing stacks

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-27 18:52:14 +00:00
Mert
bd87eb309c
feat(server): optimize partial facial recognition (#6634)
* optimize partial facial recognition

* add tests

* use map

* bulk insert faces
2024-01-25 01:27:39 -05:00
Aram Akhavan
a972dd4060
fix(server): extraction of Samsung Motionphoto videos (#6337)
* Fix extraction of samsung motionphoto videos

* Refactor binary tag extraction to the repository to consolidate exiftool usage

* format

* fix linting and swap argument orders

* Fix tag name and conditional order

* Add unit test

* Update server test assets submodule

* Remove old motion photo video assets when a new one is extracted

* delete first, then write

* Include motion photo asset uuid's in the filename

If the filenames are not uniquified, then we can't delete old/corrupt ones

* Fix formatting and fix/add tests

* chore: only use new uuid

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-22 13:04:45 -05:00
martin
17eaeb695e
feat: smart merge (#6508)
* pr feedback

* fix: tests

* update assets statistics

* pr feedback

* pr feedback

* fix: linter

* pr feedback

* fix: don't limit the smart merge

* pr feedback

* fix: server code

* remove slider

* fix: tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-19 17:52:26 +00:00
Alex
07b874edda
fix(web): revert smart merge (#6504)
* revert smart merge

* fix test

* fix test

* Remove Slider file
2024-01-19 11:34:20 -06:00
Mert
68f52818ae
feat(server): separate face clustering job (#5598)
* separate facial clustering job

* update api

* fixed some tests

* invert clustering

* hdbscan

* update api

* remove commented code

* wip dbscan

* cleanup

removed cluster endpoint

remove commented code

* fixes

updated tests

minor fixes and formatting

fixed queuing

refinements

* scale search range based on library size

* defer non-core faces

* optimizations

removed unused query option

* assign faces individually for correctness

fixed unit tests

remove unused method

* don't select face embedding

update sql

linting

fixed ml typing

* updated job mock

* paginate people query

* select face embeddings because typeorm

* fix setting face detection concurrency

* update sql

formatting

linting

* simplify logic

remove unused imports

* more specific delete signature

* more accurate typing for face stubs

* add migration

formatting

* chore: better typing

* don't select embedding by default

remove unused import

* updated sql

* use normal try/catch

* stricter concurrency typing and enforcement

* update api

* update job concurrency panel to show disabled queues

formatting

* check jobId in queueAll

fix tests

* remove outdated comment

* better facial recognition icon

* wording

wording

formatting

* fixed tests

* fix

* formatting & sql

* try to fix sql check

* more detailed description

* update sql

* formatting

* wording

* update `minFaces` description

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-18 00:08:48 -05:00
Mert
9a2fa21b28
fix(server): scale transcoded videos if dimensions are odd (#6461)
scale if odd resolution
2024-01-17 22:16:44 -05:00
martin
f0b328fb6b
feat(server, web): smart merge (#5796)
* pr feedback

* fix: tests

* update assets statistics

* pr feedback

* pr feedback

* fix: linter

* pr feedback

* fix: don't limit the smart merge

* pr feedback

* fix: server code

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-18 01:52:11 +00:00
cfitzw
deb1f970a8
feat(server, web): quotas (#4471)
* feat: quotas

* chore: open api

* chore: update status box and upload error message

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-12 19:43:36 -05:00
Jason Rasmussen
bf1dd36fa9
refactor(server): split api and jobs into separate e2e suites (#6307)
* refactor: domain and infra modules

* refactor(server): e2e tests
2024-01-09 23:04:16 -05:00
Jason Rasmussen
12dc7c48c9
refactor(server): domain and infra modules (#6301) 2024-01-09 17:07:01 -05:00
Zack Pollard
4cf1e553d2
feat(server): in upload folder, split the files into folders based on the first four of the files uuid (#6175) 2024-01-04 14:45:16 -06:00
Alex
18f59f78e3
feat(web): onboarding (#6066)
* feat(web): onboarding

* feat: openapi

* feat: modulization

* feat: page advancing

* Animation

* Add storage templaete settings

* sql

* more style

* Theme

* information and styling

* hide/show table

* Styling

* Update user property

* fix test

* fix test:

* fix e2e

* test

* Update web/src/lib/components/onboarding-page/onboarding-hello.svelte

Co-authored-by: bo0tzz <git@bo0tzz.me>

* naming

* use System Metadata

* better return type

* onboarding using server metadata

* revert previous changes in user entity

* sql

* test web

* fix test server

* server/web test

* more test

* consolidate color theme change logic

* consolidate save button to storage template

* merge main

* fix web

---------

Co-authored-by: bo0tzz <git@bo0tzz.me>
2024-01-04 05:28:32 +00:00
Jason Rasmussen
317adc5c28
feat(web,server): external domain setting (#6146)
* feat: external domain setting

* chore: open api

* mobile: handle serverconfig-externalDomain

---------

Co-authored-by: shenlong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
2024-01-03 21:54:48 -05:00
Michael Manganiello
4a5b8c3770
feat(server): Enqueue jobs in bulk (#5974)
* feat(server): Enqueue jobs in bulk

The Job Repository now has a `queueAll` method, that enqueues messages
in bulk (using BullMQ's
[`addBulk`](https://docs.bullmq.io/guide/queues/adding-bulks)),
improving performance when many jobs must be enqueued within the same
operation.

Primary change is in `src/domain/job/job.service.ts`, and other services
have been refactored to use `queueAll` when useful.

As a simple local benchmark, triggering a full thumbnail generation
process over a library of ~1,200 assets and ~350 faces went from
**~600ms** to **~250ms**.

* fix: Review feedback
2024-01-01 15:45:42 -05:00
Zack Pollard
2e38fa73bf
feat: storage template file move hardening (#5917)
* fix: pgvecto.rs extension breaks typeorm schema:drop command

* fix: parse postgres bigints to javascript number types when selecting data

* feat: verify file size is the same as original asset after copying file for storage template job

* feat: allow disabling of storage template job, defaults to disabled for new instances

* fix: don't allow setting concurrency for storage template migration, can cause race conditions above 1

* feat: add checksum verification when file is copied for storage template job

* fix: extract metadata for assets that aren't visible on timeline
2023-12-29 18:41:33 +00:00
Mert
a1e1f11399
feat(server): delete unnecessary encoded videos (#6027)
* delete unnecessary transcodes

* added test
2023-12-27 23:34:00 -06:00
Mert
8119d4bb26
chore(server): refactor locks (#5953)
* lock refactor

* add mocks

* add await

* move database repo injection to service

* update tests

* add mock implementation

* remove unused imports

* this
2023-12-27 18:36:51 -05:00
Thomas
c0ebc943d2
fix(server): strip metadata from thumbnails (#4438)
included all thumbnail metadata. It seems this has to be explicitly disabled.

Refs: #4382

feat. basic metadata e2e test

fix: use tiff thumbnails in first step + e2e fix

fix: revert switch to tiff

feat: test metadata of both webp and jpg

feat: use upload in e2e test

fix: lint

strip metadata with exiftool

use `withIccProfile`

fix e2e

formatting

run jobs in e2e
2023-12-26 16:27:51 -05:00
Mert
cc2dc12f6c
fix(server): run migrations after database checks (#5832)
* run migrations after checks

* optional migrations

* only run checks in server and e2e

* re-add migrations for microservices

* refactor

* move e2e init

* remove assert from migration

* update providers

* update microservices app service

* fixed logging

* refactored version check, added unit tests

* more version tests

* don't use mocks for sut

* refactor tests

* suggest image only if postgres is 14, 15 or 16

* review suggestions

* fixed regexp escape

* fix typing

* update migration
2023-12-21 10:06:26 -06:00
Mert
092a23fd7f
feat(server,ml): remove image tagging (#5903)
* remove image tagging

* updated lock

* fixed tests, improved logging

* be nice

* fixed tests
2023-12-20 20:47:56 -05:00
Jonathan Jogenfors
4e9b96ff1a
test(cli): e2e testing (#5101)
* Allow building and installing cli

* feat: add format fix

* docs: remove cli folder

* feat: use immich scoped package

* feat: rewrite cli readme

* docs: add info on running without building

* cleanup

* chore: remove import functionality from cli

* feat: add logout to cli

* docs: add todo for file format from server

* docs: add compilation step to cli

* fix: success message spacing

* feat: can create albums

* fix: add check step to cli

* fix: typos

* feat: pull file formats from server

* chore: use crawl service from server

* chore: fix lint

* docs: add cli documentation

* chore: rename ignore pattern

* chore: add version number to cli

* feat: use sdk

* fix: cleanup

* feat: album name on windows

* chore: remove skipped asset field

* feat: add more info to server-info command

* chore: cleanup

* wip

* chore: remove unneeded packages

* e2e test can start

* git ignore for geocode in cli

* add cli e2e to github actions

* can do e2e tests in the cli

* simplify e2e test

* cleanup

* set matrix strategy in workflow

* run npm ci in server

* choose different working directory

* check out submodules too

* increase test timeout

* set node version

* cli docker e2e tests

* fix cli docker file

* run cli e2e in correct folder

* set docker context

* correct docker build

* remove cli from dockerignore

* chore: fix docs links

* feat: add cli v2 milestone

* fix: set correct cli date

* remove submodule

* chore: add npmignore

* chore(cli): push to npm

* fix: server e2e

* run npm ci in server

* remove state from e2e

* run npm ci in server

* reshuffle docker compose files

* use new e2e composes in makefile

* increase test timeout to 10 minutes

* make github actions run makefile e2e tests

* cleanup github test names

* assert on server version

* chore: split cli e2e tests into one file per command

* chore: set cli release working dir

* chore: add repo url to npmjs

* chore: bump node setup to v4

* chore: normalize the github url

* check e2e code in lint

* fix lint

* test key login flow

* feat: allow configurable config dir

* fix session service tests

* create missing dir

* cleanup

* bump cli version to 2.0.4

* remove form-data

* feat: allow single files as argument

* add version option

* bump dependencies

* fix lint

* wip use axios as upload

* version bump

* cApiTALiZaTiON

* don't touch package lock

* wip: don't use job queues

* don't use make for cli e2e

* fix server e2e

* chore: remove old gha step

* add npm ci to server

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-12-18 20:29:26 -06:00
Mert
de1514a441
chore(server): startup check for pgvecto.rs (#5815)
* startup check for pgvecto.rs

* prefilter after assertion

* formatting

* add assert to migration

* more specific import

* use runner
2023-12-18 10:38:25 -06:00
Mert
6e7b3d6f24
fix(server): fix metadata search not working (#5800)
* don't require ml

* update e2e

* fixes

* fix e2e

* add additional conditions

* select all exif columns

* more fixes

* update sql
2023-12-17 20:16:08 -06:00
Michael Manganiello
c6f56d9591
chore(server): Check activity permissions in bulk (#5775)
Modify Access repository, to evaluate `asset` permissions in bulk.
This is the last set of permission changes, to migrate all of them to
run in bulk!
Queries have been validated to match what they currently generate for single ids.

Queries:

* `activity` owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "activity" "ActivityEntity"
  WHERE
    "ActivityEntity"."id" = $1
    AND "ActivityEntity"."userId" = $2
)
LIMIT 1

-- After
SELECT "ActivityEntity"."id" AS "ActivityEntity_id"
FROM "activity" "ActivityEntity"
WHERE
  "ActivityEntity"."id" IN ($1)
  AND "ActivityEntity"."userId" = $2
```

* `activity` album owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "activity" "ActivityEntity"
    LEFT JOIN "albums" "ActivityEntity__ActivityEntity_album"
      ON "ActivityEntity__ActivityEntity_album"."id"="ActivityEntity"."albumId"
      AND "ActivityEntity__ActivityEntity_album"."deletedAt" IS NULL
  WHERE
    "ActivityEntity"."id" = $1
    AND "ActivityEntity__ActivityEntity_album"."ownerId" = $2
)
LIMIT 1

-- After
SELECT "ActivityEntity"."id" AS "ActivityEntity_id"
FROM "activity" "ActivityEntity"
  LEFT JOIN "albums" "ActivityEntity__ActivityEntity_album"
    ON "ActivityEntity__ActivityEntity_album"."id"="ActivityEntity"."albumId"
    AND "ActivityEntity__ActivityEntity_album"."deletedAt" IS NULL
WHERE
  "ActivityEntity"."id" IN ($1)
  AND "ActivityEntity__ActivityEntity_album"."ownerId" = $2
```

* `activity` create access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "albums" "AlbumEntity"
    LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
    LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
      AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
  WHERE
    (
      (
        "AlbumEntity"."id" = $1
        AND "AlbumEntity"."isActivityEnabled" = $2
        AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
      )
      OR (
        "AlbumEntity"."id" = $4
        AND "AlbumEntity"."isActivityEnabled" = $5
        AND "AlbumEntity"."ownerId" = $6
      )
    )
    AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1

-- After
SELECT "AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
  LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
    ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
  LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
    ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
    AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
  (
    (
      "AlbumEntity"."id" IN ($1)
      AND "AlbumEntity"."isActivityEnabled" = $2
      AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
    )
    OR (
      "AlbumEntity"."id" IN ($4)
      AND "AlbumEntity"."isActivityEnabled" = $5
      AND "AlbumEntity"."ownerId" = $6
    )
  )
  AND "AlbumEntity"."deletedAt" IS NULL
```
2023-12-17 12:10:21 -06:00
Jason Rasmussen
9768931275
feat(web,server)!: runtime log level (#5672)
* feat: change log level at runtime

* chore: open api

* chore: prefer env over runtime

* chore: remove default env value
2023-12-14 16:55:40 +00:00
Jason Rasmussen
b34abf25f0
feat(server): server-side events (#5669) 2023-12-13 12:23:51 -05:00
Jason Rasmussen
ed4358741e
feat(web): re-add open graph tags for public share links (#5635)
* feat: re-add open graph tags for public share links

* fix: undefined in html

* chore: tests
2023-12-11 13:37:47 -06:00
Jason Rasmussen
33529d1d9b
refactor(server): auth dto (#5593)
* refactor: AuthUserDto => AuthDto

* refactor: reorganize auth-dto

* refactor: AuthUser() => Auth()
2023-12-09 23:34:12 -05:00
Jason Rasmussen
1e99ba8167
feat: use pgvecto.rs (#3605) 2023-12-08 11:15:46 -05:00
martin
7702560b12
feat(web): re-assign person faces (2) (#4949)
* feat: unassign person faces

* multiple improvements

* chore: regenerate api

* feat: improve face interactions in photos

* fix: tests

* fix: tests

* optimize

* fix: wrong assignment on complex-multiple re-assignments

* fix: thumbnails with large photos

* fix: complex reassign

* fix: don't send people with faces

* fix: person thumbnail generation

* chore: regenerate api

* add tess

* feat: face box even when zoomed

* fix: change feature photo

* feat: make the blue icon hoverable

* chore: regenerate api

* feat: use websocket

* fix: loading spinner when clicking on the done button

* fix: use the svelte way

* fix: tests

* simplify

* fix: unused vars

* fix: remove unused code

* fix: add migration

* chore: regenerate api

* ci: add unit tests

* chore: regenerate api

* feat: if a new person is created for a face and the server takes more than 15 seconds to generate the person thumbnail, don't wait for it

* reorganize

* chore: regenerate api

* feat: global edit

* pr feedback

* pr feedback

* simplify

* revert test

* fix: face generation

* fix: tests

* fix: face generation

* fix merge

* feat: search names in unmerge face selector modal

* fix: merge face selector

* simplify feature photo generation

* fix: change endpoint

* pr feedback

* chore: fix merge

* chore: fix merge

* fix: tests

* fix: edit & hide buttons

* fix: tests

* feat: show if person is hidden

* feat: rename face to person

* feat: split in new panel

* copy-paste-error

* pr feedback

* fix: feature photo

* do not leak faces

* fix: unmerge modal

* fix: merge modal event

* feat(server): remove duplicates

* fix: title for image thumbnails

* fix: disable side panel when there's no face until next PR

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-12-05 09:43:15 -06:00
Clement Ong
982183600d
feat(web): clear failed jobs (#5423)
* add clear failed jobs button

* refactor: clean up code

* chore: open api

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-12-05 02:07:20 +00:00
Michael Manganiello
5aa658de59
chore(server): Check asset permissions in bulk (#5329)
Modify Access repository, to evaluate `asset` permissions in bulk.
Queries have been validated to match what they currently generate for single ids.

Queries:

* `asset` album access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "albums" "AlbumEntity"
    LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets"
      ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id"
    LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets"
      ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId"
      AND "AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL
    LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
    LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
      AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
  WHERE
    (
      ("AlbumEntity"."ownerId" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2)
      OR ("AlbumEntity__AlbumEntity_sharedUsers"."id" = $3 AND "AlbumEntity__AlbumEntity_assets"."id" = $4)
      OR ("AlbumEntity"."ownerId" = $5 AND "AlbumEntity__AlbumEntity_assets"."livePhotoVideoId" = $6)
      OR ("AlbumEntity__AlbumEntity_sharedUsers"."id" = $7 AND "AlbumEntity__AlbumEntity_assets"."livePhotoVideoId" = $8)
    )
    AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1

-- After
SELECT
  "asset"."id" AS "assetId",
  "asset"."livePhotoVideoId" AS "livePhotoVideoId"
FROM "albums" "album"
  INNER JOIN "albums_assets_assets" "album_asset"
    ON "album_asset"."albumsId"="album"."id"
  INNER JOIN "assets" "asset"
    ON "asset"."id"="album_asset"."assetsId"
    AND "asset"."deletedAt" IS NULL
  LEFT JOIN "albums_shared_users_users" "album_sharedUsers"
    ON "album_sharedUsers"."albumsId"="album"."id"
  LEFT JOIN "users" "sharedUsers"
    ON "sharedUsers"."id"="album_sharedUsers"."usersId"
    AND "sharedUsers"."deletedAt" IS NULL
WHERE
  (
    "album"."ownerId" = $1
    OR "sharedUsers"."id" = $2
  )
  AND (
    "asset"."id" IN ($3, $4)
    OR "asset"."livePhotoVideoId" IN ($5, $6)
  )
  AND "album"."deletedAt" IS NULL
```

* `asset` owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "assets" "AssetEntity"
  WHERE
    "AssetEntity"."id" = $1
    AND "AssetEntity"."ownerId" = $2
)
LIMIT 1

-- After
SELECT
  "AssetEntity"."id" AS "AssetEntity_id"
FROM "assets" "AssetEntity"
WHERE
  "AssetEntity"."id" IN ($1, $2)
  AND "AssetEntity"."ownerId" = $3
```

* `asset` partner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "partners" "PartnerEntity"
    LEFT JOIN "users" "PartnerEntity__PartnerEntity_sharedWith"
      ON "PartnerEntity__PartnerEntity_sharedWith"."id"="PartnerEntity"."sharedWithId"
      AND "PartnerEntity__PartnerEntity_sharedWith"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__PartnerEntity_sharedBy"
      ON "PartnerEntity__PartnerEntity_sharedBy"."id"="PartnerEntity"."sharedById"
      AND "PartnerEntity__PartnerEntity_sharedBy"."deletedAt" IS NULL
    LEFT JOIN "assets" "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"
      ON "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."ownerId"="PartnerEntity__PartnerEntity_sharedBy"."id"
      AND "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__sharedBy"
      ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
      AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__sharedWith"
      ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
      AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
  WHERE
    "PartnerEntity__PartnerEntity_sharedWith"."id" = $1
    AND "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."id" = $2
)
LIMIT 1

-- After
SELECT
  "asset"."id" AS "assetId"
FROM "partners" "partner"
  INNER JOIN "users" "sharedBy"
    ON "sharedBy"."id"="partner"."sharedById"
    AND "sharedBy"."deletedAt" IS NULL
  INNER JOIN "assets" "asset"
    ON "asset"."ownerId"="sharedBy"."id"
    AND "asset"."deletedAt" IS NULL
WHERE
  "partner"."sharedWithId" = $1
  AND "asset"."id" IN ($2, $3)
```

* `asset` shared link access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "shared_links" "SharedLinkEntity"
    LEFT JOIN "albums" "SharedLinkEntity__SharedLinkEntity_album"
      ON "SharedLinkEntity__SharedLinkEntity_album"."id"="SharedLinkEntity"."albumId"
      AND "SharedLinkEntity__SharedLinkEntity_album"."deletedAt" IS NULL
    LEFT JOIN "albums_assets_assets" "760f12c00d97bdcec1ce224d1e3bf449859942b6"
      ON "760f12c00d97bdcec1ce224d1e3bf449859942b6"."albumsId"="SharedLinkEntity__SharedLinkEntity_album"."id"
    LEFT JOIN "assets" "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"
      ON "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."id"="760f12c00d97bdcec1ce224d1e3bf449859942b6"."assetsId"
      AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."deletedAt" IS NULL
    LEFT JOIN "shared_link__asset" "SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"
      ON "SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"."sharedLinksId"="SharedLinkEntity"."id"
    LEFT JOIN "assets" "SharedLinkEntity__SharedLinkEntity_assets"
      ON "SharedLinkEntity__SharedLinkEntity_assets"."id"="SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"."assetsId"
      AND "SharedLinkEntity__SharedLinkEntity_assets"."deletedAt" IS NULL
  WHERE (
    ("SharedLinkEntity"."id" = $1 AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."id" = $2)
    OR ("SharedLinkEntity"."id" = $3 AND "SharedLinkEntity__SharedLinkEntity_assets"."id" = $4)
    OR ("SharedLinkEntity"."id" = $5 AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."livePhotoVideoId" = $6)
    OR ("SharedLinkEntity"."id" = $7 AND "SharedLinkEntity__SharedLinkEntity_assets"."livePhotoVideoId" = $8)
  )
)
LIMIT 1

-- After
SELECT
  "assets"."id" AS "assetId",
  "assets"."livePhotoVideoId" AS "assetLivePhotoVideoId",
  "albumAssets"."id" AS "albumAssetId",
  "albumAssets"."livePhotoVideoId" AS "albumAssetLivePhotoVideoId"
FROM "shared_links" "sharedLink"
  LEFT JOIN "albums" "album"
    ON "album"."id"="sharedLink"."albumId"
    AND "album"."deletedAt" IS NULL
  LEFT JOIN "shared_link__asset" "assets_sharedLink"
    ON "assets_sharedLink"."sharedLinksId"="sharedLink"."id"
  LEFT JOIN "assets" "assets"
    ON "assets"."id"="assets_sharedLink"."assetsId"
    AND "assets"."deletedAt" IS NULL
  LEFT JOIN "albums_assets_assets" "album_albumAssets"
    ON "album_albumAssets"."albumsId"="album"."id"
  LEFT JOIN "assets" "albumAssets"
    ON "albumAssets"."id"="album_albumAssets"."assetsId"
    AND "albumAssets"."deletedAt" IS NULL
WHERE
  "sharedLink"."id" = $1
  AND (
    "assets"."id" IN ($2, $3)
    OR "albumAssets"."id" IN ($4, $5)
    OR "assets"."livePhotoVideoId" IN ($6, $7)
    OR "albumAssets"."livePhotoVideoId" IN ($8, $9)
  )
```
2023-12-02 02:56:41 +00:00
YFrendo
644e52b153
feat: Edit metadata (#5066)
* chore: rebase and clean-up

* feat: sync description, add e2e tests

* feat: simplify web code

* chore: unit tests

* fix: linting

* Bug fix with the arrows key

* timezone typeahead filter

timezone typeahead filter

* small stlying

* format fix

* Bug fix in the map selection

Bug fix in the map selection

* Websocket basic

Websocket basic

* Update metadata visualisation through the websocket

* Update timeline

* fix merge

* fix web

* fix web

* maplibre system

* format fix

* format fix

* refactor: clean up

* Fix small bug in the hour/timezone

* Don't diplay modify for readOnly asset

* Add log in case of failure

* Formater + try/catch error

* Remove everything related to websocket

* Revert "Remove everything related to websocket"

This reverts commit 14bcb9e1e4.

* remove notification

* fix test

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-11-30 03:52:28 +00:00
Alex
4e5eef129d
fix(server): get album's assets in getAlbumInfo route (#5325)
* fix(server): get album's assets in getAlbumInfo route

* unit test

* test: e2e tests
2023-11-26 21:21:04 -06:00
martin
3aa2927dae
fix(web): sorting options for albums (#5233)
* fix: albums

* pr feedback

* fix: current behavior

* rename

* fix: album metadatas

* fix: tests

* fix: e2e test

* simplify

* fix: cover shared links

* rename function

* merge main

* merge main

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-11-26 15:23:43 +00:00
Michael Manganiello
c04340c63e
chore(server): Check more permissions in bulk (#5315)
Modify Access repository, to evaluate `authDevice`, `library`, `partner`,
`person`, and `timeline` permissions in bulk.
Queries have been validated to match what they currently generate for
single ids.

As an extra performance improvement, we now use a custom QueryBuilder
for the Partners queries, to avoid the eager relationships that add
unneeded `LEFT JOIN` clauses. We only filter based on the ids present in
the `partners` table, so those joins can be avoided.

Queries:

* `library` owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "libraries" "LibraryEntity"
  WHERE
    "LibraryEntity"."id" = $1
    AND "LibraryEntity"."ownerId" = $2
    AND "LibraryEntity"."deletedAt" IS NULL
)
LIMIT 1

-- After
SELECT "LibraryEntity"."id" AS "LibraryEntity_id"
FROM "libraries" "LibraryEntity"
WHERE
  "LibraryEntity"."id" IN ($1, $2)
  AND "LibraryEntity"."ownerId" = $3
  AND "LibraryEntity"."deletedAt" IS NULL
```

* `library` partner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "partners" "PartnerEntity"
    LEFT JOIN "users" "PartnerEntity__sharedBy"
      ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
      AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__sharedWith"
      ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
      AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
  WHERE
    "PartnerEntity"."sharedWithId" = $1
    AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1

-- After
SELECT
  "partner"."sharedById" AS "partner_sharedById",
  "partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
  "partner"."sharedById" IN ($1, $2)
  AND "partner"."sharedWithId" = $3
```

* `authDevice` owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "user_token" "UserTokenEntity"
  WHERE
    "UserTokenEntity"."userId" = $1
    AND "UserTokenEntity"."id" = $2
)
LIMIT 1

-- After
SELECT "UserTokenEntity"."id" AS "UserTokenEntity_id"
FROM "user_token" "UserTokenEntity"
WHERE
  "UserTokenEntity"."userId" = $1
  AND "UserTokenEntity"."id" IN ($2, $3)
```

* `timeline` partner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "partners" "PartnerEntity"
    LEFT JOIN "users" "PartnerEntity__sharedBy"
      ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
      AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__sharedWith"
      ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
      AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
  WHERE
    "PartnerEntity"."sharedWithId" = $1
    AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1

-- After
SELECT
  "partner"."sharedById" AS "partner_sharedById",
  "partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
  "partner"."sharedById" IN ($1, $2)
  AND "partner"."sharedWithId" = $3
```

* `person` owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "person" "PersonEntity"
  WHERE
    "PersonEntity"."id" = $1
    AND "PersonEntity"."ownerId" = $2
)
LIMIT 1

-- After
SELECT "PersonEntity"."id" AS "PersonEntity_id"
FROM "person" "PersonEntity"
WHERE
  "PersonEntity"."id" IN ($1, $2)
  AND "PersonEntity"."ownerId" = $3
```

* `partner` update access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "partners" "PartnerEntity"
    LEFT JOIN "users" "PartnerEntity__sharedBy"
      ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
      AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__sharedWith"
      ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
      AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
  WHERE
    "PartnerEntity"."sharedWithId" = $1
    AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1

-- After
SELECT
  "partner"."sharedById" AS "partner_sharedById",
  "partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
  "partner"."sharedById" IN ($1, $2)
  AND "partner"."sharedWithId" = $3
```
2023-11-26 07:50:41 -05:00
Michael Manganiello
6d1b325b34
chore(server): Check album permissions in bulk (#5290)
* chore(server): Check album permissions in bulk

Modify Access repository, to evaluate `album` permissions in bulk.
Queries have been validated to match what they currently generate for
single ids.

Queries:

* Owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "albums" "AlbumEntity"
  WHERE
    "AlbumEntity"."id" = $1
    AND "AlbumEntity"."ownerId" = $2
    AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1

-- After
SELECT
  "AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
WHERE
  "AlbumEntity"."id" IN ($1, $2)
  AND "AlbumEntity"."ownerId" = $3
  AND "AlbumEntity"."deletedAt" IS NULL
```

* Shared link access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "shared_links" "SharedLinkEntity"
  WHERE
    "SharedLinkEntity"."id" = $1
    AND "SharedLinkEntity"."albumId" = $2
)
LIMIT 1

-- After
SELECT
  "SharedLinkEntity"."albumId" AS "SharedLinkEntity_albumId",
  "SharedLinkEntity"."id" AS "SharedLinkEntity_id"
FROM "shared_links" "SharedLinkEntity"
WHERE
  "SharedLinkEntity"."id" = $1
  AND "SharedLinkEntity"."albumId" IN ($2, $3)
```

* Shared album access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "albums" "AlbumEntity"
    LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
    LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
      AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
  WHERE
    "AlbumEntity"."id" = $1
    AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $2
    AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1

-- After
SELECT
  "AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
  LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
    ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
  LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
    ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
    AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
  "AlbumEntity"."id" IN ($1, $2)
  AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
  AND "AlbumEntity"."deletedAt" IS NULL
```

* chore(server): Add set utils, avoid double queries for same ids

* chore(server): Review feedback
2023-11-25 17:56:23 -05:00
Zack Pollard
698226634e
feat: postgres reverse geocoding (#5301)
* feat: add system metadata repository for storing key values for internal usage

* feat: add database entities for geodata

* feat: move reverse geocoding from local-reverse-geocoder to postgresql

* infra: disable synchronization for geodata_places table until typeorm supports earth column

* feat: remove cities override config as we will default all instances to cities500 now

* test: e2e tests don't clear geodata tables on reset
2023-11-25 18:53:30 +00:00
shenlong
0108211c0f
refactor: deprecate getUserAssetsByDeviceId (#5273)
* refactor: deprecated getUserAssetsByDeviceId

* prevent breaking changes

* chore: add deprecation

* prevent breaking changes

* prevent breaking changes

---------

Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-11-25 15:46:20 +00:00
Alex
f094ff2aa1
fix(server): album perf query (#5232)
* Revert "fix: album performances (#5224)"

This reverts commit c438e17954.

* Revert "fix: album sorting options (#5127)"

This reverts commit 725f30c494.
2023-11-21 10:07:49 -06:00
martin
725f30c494
fix: album sorting options (#5127)
* fix: album sort options

* fix: don't load assets

* pr feedback

* fix: albumStub

* fix(web): album shared without assets

* fix: tests

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-11-20 13:01:21 -06:00
Jason Rasmussen
87f02cc775
chore(server): shared link e2e for add/remove assets (#5052) 2023-11-15 17:50:55 -06:00
Jason Rasmussen
4b722517f0
chore: cleanup e2e (#5051) 2023-11-14 20:31:06 -05:00
Jason Rasmussen
6127fd4c5c
chore(server): improve e2e test speed (#5026)
* feat: improve shared link (41s to 8s)

* feat: improve activity (18s to 8s)

* feat: improve partner (20s to 10s)

* feat: improve server-info (10s to 6s)

* feat: improve system-config

* fix: e2e

* chore: linting
2023-11-14 20:08:22 -05:00
Jason Rasmussen
753dab8b3c
feat(server): asset search endpoint (#4931)
* feat(server): GET /assets endpoint

* chore: open api

* chore: use dumb name

* feat: search by make, model, lens, city, state, country

* chore: open api

* chore: pagination validation and tests

* chore: pr feedback
2023-11-14 22:47:15 +00:00
martin
d25a245049
feat(web,server): user avatar color (#4779) 2023-11-14 03:10:35 +00:00
Brian Austin
7fca0d8da5
fix: replace first and last name with single field (#4915) 2023-11-11 19:03:32 -06:00
Alex
35767591d2
feat(web): show partners assets on the main timeline (#4933) 2023-11-11 21:06:19 +00:00
martin
92ec1ce77f
fix(server,web): correctly show album level like (#4916)
* fix: like in global activity

* refactor: rename isGlobal to ReactionLevel.Album

* chore: open api

* chore: e2e test for album vs comment duplicate like checking

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-11-09 21:32:31 -05:00
Sushain Cherivirala
986bbfa831
feat(server): don't re-run face recognition on assets without any faces (#4854)
* Add AssetJobStatus

* fentity

* Add jobStatus field to AssetEntity

* Fix the migration doc paths

* Filter on facesRecognizedAt

* Set facesRecognizedAt field

* Test for facesRecognizedAt

* Done testing

* Adjust FK properties

* Add tests for WithoutProperty.FACES

* chore: non-nullable

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-11-09 20:55:00 -05:00
Jason Rasmussen
75c065c83a
chore(server): e2e test (#4941) 2023-11-09 20:23:03 -05:00
Daniel Dietzler
a147dee4b6
feat: Maplibre (#4294)
* maplibre on web, custom styles from server

Actually use new vector tile server, custom style.json

support multiple style files, light/dark mode

cleanup, use new map everywhere

send file directly instead of loading first

better light/dark mode switching

remove leaflet

fix mapstyles dto, first draft of map settings

delete and add styles

fix delete default styles

fix tests

only allow one light and one dark style url

revert config core changes

fix server config store

fix tests

move axios fetches to repo

fix package-lock

fix tests

* open api

* add assets to docker container

* web: use mapSettings color for style

* style: add unique ids to map styles

* mobile: use style json for vector / raster

* do not use svelte-material-icons

* add click events to markers, simplify asset detail map

* improve map performance by using asset thumbnails for markers instead of original file

* Remove custom attribution

(by request)

* mobile: update map attribution

* style: map dark mode

* style: map light mode

* zoom level for state

* styling

* overflow gradient

* Limit maxZoom to 14

* mobile: listen for mapStyle changes in MapThumbnail

* mobile: update concurrency

---------

Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: bo0tzz <git@bo0tzz.me>
Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-11-09 10:10:56 -06:00
Jason Rasmussen
5423f1c25b
refactor(server): auth dtos (#4881)
* refactor: auth dtos

* chore: open api
2023-11-09 10:14:15 -05:00
martin
9d01885b58
feat(server, web): Album's options (#4870)
* feat: disable activity

* fix: disable reactions

* fix: tests

* fix: tests

* fix: tests

* pr feedback

* pr feedback

* chore: styling & wording

* refactor component

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-11-07 04:37:21 +00:00
Jason Rasmussen
279481ad54
feat(server): make is favorite optional on asset upload (#4865)
* feat(server): make isFavorite optional

* chore: open api

* chore: e2e
2023-11-06 09:04:39 -05:00
Jason Rasmussen
2e424fe249
feat(server): better api error messages (for unhandled exceptions) (#4817)
* feat(server): better error messages

* chore: open api

* chore: remove debug log

* fix: syntax error

* fix: e2e test
2023-11-03 20:33:15 -05:00
Jason Rasmussen
0130591a0f
fix: show/set activity like per user (#4775)
* fix: like per user

* chore: open api

* chore: e2e test for userId filtering
2023-11-01 11:49:12 -04:00
martin
ce5966c23d
feat(web,server): activity (#4682)
* feat: activity

* regenerate api

* fix: make asset owner unable to delete comment

* fix: merge

* fix: tests

* feat: use textarea instead of input

* fix: do actions only if the album is shared

* fix: placeholder opacity

* fix(web): improve messages UI

* fix(web): improve input message UI

* pr feedback

* fix: tests

* pr feedback

* pr feedback

* pr feedback

* fix permissions

* regenerate api

* pr feedback

* pr feedback

* multiple improvements on web

* fix: ui colors

* WIP

* chore: open api

* pr feedback

* fix: add comment

* chore: clean up

* pr feedback

* refactor: endpoints

* chore: open api

* fix: filter by type

* fix: e2e

* feat: e2e remove own comment

* fix: web tests

* remove console.log

* chore: cleanup

* fix: ui tweaks

* pr feedback

* fix web test

* fix: unit tests

* chore: remove unused code

* revert useless changes

* fix: grouping messages

* fix: remove nullable on updatedAt

* fix: text overflow

* styling

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-31 22:13:34 -05:00
Daniel Dietzler
cd375a976e
feat(server): custom library scanning interval (#4390)
* add automatic library scan config options

* add validation

* open api

* use CronJob instead of cron-validator

* fix tests

* catch potential error of the library scan initialization

* better description for input field

* move library scan job initialization to server app service

* fix tests

* add comments to all parameters of cronjob contructor

* make scan a child of a more general library object

* open api

* chore: cleanup

* move cronjob handling to job repoistory

* web: select for common cron expressions

* fix open api

* fix tests

* put scanning settings in nested accordion

* fix system config validation

* refactor, tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-31 15:19:12 -05:00
Michael Manganiello
b6f18cbe81
fix(server): Correctly set album start and end dates (#4698)
* fix(server): Correctly set album start and end dates

Currently, the query that retrieves album assets uses
`ORDER BY assets.fileCreatedAt DESC`, which makes the existing logic
return the start/end dates reversed (with `startDate` being taken from
the first asset in the array).

Instead of using the index-based approach, this change iterates through
assets to get the min/max `fileCreatedAt`. This will avoid any future
issues, if the query ordering changes, or becomes customizable (e.g. in
case the user prefers to visualize older assets first).

* fix: Maintain constant cost and only swap variables if needed
2023-10-31 05:08:34 -05:00
Jason Rasmussen
3212a47720
refactor(server): user profile picture (#4728) 2023-10-30 19:38:34 -04:00
Jason Rasmussen
8dcd159bd6
chore(server): remove user count endpoint (#4724)
* chore: remove unused endpoint

* chore: open api
2023-10-30 19:29:18 +00:00
Jason Rasmussen
603b056512
refactor(server): auth delete device (#4720)
* refactor(server): auth delete device

* fix: person e2e
2023-10-30 11:48:38 -04:00
jarvis2f
8a6889529c
feat(server,web,mobile): Add optional password option for share links. (#4655)
* feat(server,web,mobile): Add optional password option for share links.

Signed-off-by: jarvis2f <137974272+jarvis2f@users.noreply.github.com>

* feat(server,web): Update shared-link.controller and page.svelte for improved cookie handling and metadata updates.

Signed-off-by: jarvis2f <137974272+jarvis2f@users.noreply.github.com>

---------

Signed-off-by: jarvis2f <137974272+jarvis2f@users.noreply.github.com>
2023-10-28 20:35:38 -05:00
Wingy
a08a687951
refactor(server, web): standardize theme api response (#4664)
* web: standardize theme api response

* revert makefile change that i didn't mean to commit
2023-10-27 02:32:33 +00:00
Wingy
cb0e37e76e
fix(web): fix Theme Custom CSS endpoint requiring the user to be logged in as the server admin (#4633)
* fix custom css requiring the user to be the admin and logged in

* move theme api to custom endpoint

* add e2e test
2023-10-25 22:13:05 +00:00
martin
3e3598fd92
fix: suggest people (#4566)
* fix: suggest people

* feat: remove hidden people

* add hidden people when merging faces

* pr feedback

* fix: don't use reactive statement

* fixed section height

* improve merging

* fix: migration

* fix migration

* feat: add asset count

* fix: test

* rename endpoint

* add server test

* improve responsive design

* fix: remove videos from live photos in the asset count

* pr feedback

* fix: rename asset count endpoint

* fix: return firstname and lastname

* fix: reset people only on error

* fix: search

* fix: responsive design & div flickering

* fix: cleanup

* chore: open api

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-24 15:53:49 +00:00
martin
1aae29a0b8
refactor(server, web)!: store latest immich version available on the server (#3565)
* refactor: store latest immich version available on the server

* don't store admins acknowledgement

* merge main

* fix: api

* feat: custom interval

* pr feedback

* remove unused code

* update environment-variables

* pr feedback

* ci: fix server tests

* fix: dart number

* pr feedback

* remove proxy

* pr feedback

* feat: make stringToVersion more flexible

* feat(web): disable check

* feat: working version

* remove env

* fix: check if interval exists when updating the interval

* feat: show last check

* fix: tests

* fix: remove availableVersion when updated

* fix merge

* fix: web

* fix e2e tests

* merge main

* merge main

* pr feedback

* pr feedback

* fix: tests

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* fix: migration

* regenerate api

* fix: typo

* fix: compare versions

* pr feedback

* fix

* pr feedback

* fix: checkIntervalTime on startup

* refactor: websockets and interval logic

* chore: open api

* chore: remove unused code

* fix: use interval instead of cron

* mobile: handle WS event data as json object

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
2023-10-24 11:05:42 -04:00
martin
99c6f8fb13
feat(server): allow unassigned asset-faces (#4474)
* feat: un-assign people

* regenerate api

* edit migration script

* fix: tests

* fix: typeorm

* fix: typo

* fix: type

* fix: migration

* fix: update

* fix: contraints

* fix: remove set

* feat: add assetId

* remove assetId

* remove unassignedFaces

* fix: migration

* regenerate api

* fix: tests

* remove changes to the api

* fix: migration

* fix migration

* pr feedback

* fix: revert change

* fix: tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-24 13:12:42 +00:00
Daniel Dietzler
6b25435b4f
refactor(server): make storage core singleton (#4608) 2023-10-23 11:52:21 -04:00
Michael Manganiello
2288b022bc
fix(server): Check album asset membership in bulk (#4603)
Add `AlbumRepository` method to retrieve an album's asset ids, with an
optional parameter to only filter by the provided asset ids. With this,
we can now check asset membership using a single query.

When adding or removing assets to an album, checking whether each asset
is already present in the album now requires a single query, instead of
one query per asset.

Related to #4539 performance improvements.

Before:
```
// Asset membership and permissions check (2 queries per asset)
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","b666ae6c-afa8-4d6f-a1ad-7091a0659320"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["b666ae6c-afa8-4d6f-a1ad-7091a0659320","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","c656ab1c-7775-4ff7-b56f-01308c072a76"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["c656ab1c-7775-4ff7-b56f-01308c072a76","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
```

After:
```
// Asset membership check (1 query for all assets)
immich_server            | query: SELECT "albums_assets"."assetsId" AS "assetId" FROM "albums_assets_assets" "albums_assets" WHERE "albums_assets"."albumsId" = $1 AND "albums_assets"."assetsId" IN ($2, $3, $4) -- PARAMETERS: ["ca870d76-6311-4e89-bf9a-f5b51ea2452c","b666ae6c-afa8-4d6f-a1ad-7091a0659320","c656ab1c-7775-4ff7-b56f-01308c072a76","cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9"]
// Permissions check (1 query per asset)
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["b666ae6c-afa8-4d6f-a1ad-7091a0659320","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["c656ab1c-7775-4ff7-b56f-01308c072a76","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
```
2023-10-23 09:02:27 -04:00
Daniel Dietzler
c6b4bc883b
refactor(server): make user core singleton (#4607) 2023-10-23 08:38:48 -04:00
Daniel Dietzler
50bc92aac0
refactor(server): make access core singleton (#4609) 2023-10-23 08:37:51 -04:00
shenlong
cf08ac7538
feat: manual stack assets (#4198) 2023-10-21 21:38:07 -05:00
Jason Rasmussen
4b59f83288
refactor: e2e tests (#4536) 2023-10-18 17:02:42 -05:00
Jason Rasmussen
0994575bf3
fix(server): album add/remove asset performance (#4516) 2023-10-18 10:56:00 -05:00
Jonathan Jogenfors
dadcf49eca
fix(server,web): correctly remove metadata from shared links (#4464)
* wip: strip metadata

* fix: authenticate time buckets

* hide detail panel

* fix tests

* fix lint

* add e2e tests

* chore: open api

* fix web compilation error

* feat: test with asset with gps position

* fix: only import fs.promises.cp

* fix: cleanup mapasset

* fix: format

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-14 01:46:30 +00:00
Jonathan Jogenfors
41befc0948
fix(server): don't publicly reveal user count (#4409)
* fix: don't reveal user count publicly

* fix: mobile and user controller

* fix: update other frontend endpoints

* fix: revert openapi change

* chore: open api

* fix: initialize

* openapi

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-11 02:37:13 +00:00
Daniel Dietzler
09bf1c9175
feat(server): harden move file (#4361)
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-10 21:14:44 -05:00
Jonathan Jogenfors
56eb7bf0fc
fix(server): improve library scan queuing performance (#4418)
* fix: inline mark asset as offline

* fix: improve log message

* chore: lint

* fix: offline asset algorithm

* fix: use set comparison to check what to import

* fix: only mark new offline files as offline

* fix: compare the correct array

* fix: set default library concurrency to 5

* fix: remove one db call when scanning new files

* chore: remove unused import

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-10 18:59:13 -04:00
martin
b8d6cc1e09
feat(server,web): improve performances in person page (1) (#4387)
* feat: improve performances in people page

* feat: add loadingspinner when searching

* fix: reset people on error

* fix: case insensitive

* feat: better sql query

* fix: reset people list before api request

* fix: format
2023-10-10 09:34:25 -05:00
Jason Rasmussen
ebb50476ac
fix(server): timeline bucket access for shared links (#4404) 2023-10-09 15:57:36 +00:00
Daniel Dietzler
0243570c0b
fix(server): make system config core singleton (#4392)
* make system config core singleton

* refactor

* fix tests

* chore: fix tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-09 00:51:03 +00:00
Jason Rasmussen
8d5bf93360
test(server): full backend end-to-end testing with microservices (#4225)
* feat: asset e2e with job option

* feat: checkout test assets

* feat: library e2e tests

* fix: use node 21 in e2e

* fix: tests

* fix: use normalized external path

* feat: more external path tests

* chore: use parametrized tests

* chore: remove unused test code

* chore: refactor test asset path

* feat: centralize test app creation

* fix: correct error message for missing assets

* feat: test file formats

* fix: don't compare checksum

* feat: build libvips

* fix: install meson

* fix: use immich test asset repo

* feat: test nikon raw files

* fix: set Z timezone

* feat: test offline library files

* feat: richer metadata tests

* feat: e2e tests in docker

* feat: e2e test with arm64 docker

* fix: manual docker compose run

* fix: remove metadata processor import

* fix: run e2e tests in test.yml

* fix: checkout e2e assets

* fix: typo

* fix: checkout files in app directory

* fix: increase e2e memory

* fix: rm submodules

* fix: revert action name

* test: mark file offline when external path changes

* feat: rename env var to TEST_ENV

* docs: new test procedures

* feat: can run docker e2e tests manually if needed

* chore: use new node 20.8 for e2e

* chore: bump exiftool-vendored

* feat: simplify test launching

* fix: rename env vars to use immich_ prefix

* feat: asset folder is submodule

* chore: cleanup after 20.8 upgrade

* fix: don't log postgres in e2e

* fix: better warning about not running all tests

---------

Co-authored-by: Jonathan Jogenfors <jonathan@jogenfors.se>
2023-10-06 23:32:28 +02:00
shenlong
4a8887f37b
feat(server): trash asset (#4015)
* refactor(server): delete assets endpoint

* fix: formatting

* chore: cleanup

* chore: open api

* chore(mobile): replace DeleteAssetDTO with BulkIdsDTOs

* feat: trash an asset

* chore(server): formatting

* chore: open api

* chore: wording

* chore: open-api

* feat(server): add withDeleted to getAssets queries

* WIP: mobile-recycle-bin

* feat(server): recycle-bin to system config

* feat(web): use recycle-bin system config

* chore(server): domain assetcore removed

* chore(server): rename recycle-bin to trash

* chore(web): rename recycle-bin to trash

* chore(server): always send soft deleted assets for getAllByUserId

* chore(web): formatting

* feat(server): permanent delete assets older than trashed period

* feat(web): trash empty placeholder image

* feat(server): empty trash

* feat(web): empty trash

* WIP: mobile-recycle-bin

* refactor(server): empty / restore trash to separate endpoint

* test(server): handle failures

* test(server): fix e2e server-info test

* test(server): deletion test refactor

* feat(mobile): use map settings from server-config to enable / disable map

* feat(mobile): trash asset

* fix(server): operations on assets in trash

* feat(web): show trash statistics

* fix(web): handle trash enabled

* fix(mobile): restore updates from trash

* fix(server): ignore trashed assets for person

* fix(server): add / remove search index when trashed / restored

* chore(web): format

* fix(server): asset service test

* fix(server): include trashed assts for duplicates from uploads

* feat(mobile): no dialog for trash, always dialog for permanent delete

* refactor(mobile): use isar where instead of dart filter

* refactor(mobile): asset provide - handle deletes in single db txn

* chore(mobile): review changes

* feat(web): confirmation before empty trash

* server: review changes

* fix(server): handle library changes

* fix: filter external assets from getting trashed / deleted

* fix(server): empty-bin

* feat: broadcast config update events through ws

* change order of trash button on mobile

* styling

* fix(mobile): do not show trashed toast for local only assets

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-06 02:01:14 -05:00
Jonathan Jogenfors
81009c17bf
fix(server): use fileCreatedAt in buckets (#4354) 2023-10-05 08:26:42 -05:00
Jason Rasmussen
192e950567
fix: use local time for time buckets and improve memories (#4072)
* fix: timezone bucket timezones

* chore: open api

* fix: interpret local time in utc

* fix: tests

* fix: refactor memory lane

* fix(web): use local date in memory viewer

* chore: set localDateTime non-null

* fix: filter out memories from the current year

* wip: move localDateTime to asset

* fix: correct sorting from db

* fix: migration

* fix: web typo

* fix: formatting

* fix: e2e

* chore: localDateTime is non-null

* chore: more non-nulliness

* fix: asset stub

* fix: tests

* fix: use extract and index for day of year

* fix: don't show memories before today

* fix: cleanup

* fix: tests

* fix: only use localtime for tz

* fix: display memories in client timezone

* fix: tests

* fix: svelte tests

* fix: bugs

* chore: open api

---------

Co-authored-by: Jonathan Jogenfors <jonathan@jogenfors.se>
2023-10-04 22:11:11 +00:00
Daniel Dietzler
ff331ffad9
fix(server): Offset of random endpoint could be higher than user's asset count (#4342)
* fix offset of all assets with correct ownerId

* (e2e): test if user does not have all assets
2023-10-04 12:51:44 -05:00
Daniel Dietzler
e571880c16
feat(web, mobile): Options to show archived assets in map (#4293)
* Add include archive setting to map on web

* open api

* better naming for web isArchived variable

* add withArchived setting to mobile

* (e2e): tests for mapMarker endpoint and isArchived

* isArchived to mobile

* chore: cleanup test

* chore: optimize e2e

---------

Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-04 09:51:07 -04:00
Daniel Dietzler
0a22e64799
refactor(server): merge facial-recognition and person (#4237)
* move facial recognition service into person service

* merge face repository and person repository

* fix imports
2023-09-27 16:46:46 -04:00
Daniel Dietzler
3a44e8f8d3
refactor(server): Move metadata extraction to domain (#4243)
* use storageRepository in metadata extraction

* move metadata extraction processor to domain

* cleanup infra/domain

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-09-27 18:44:51 +00:00
Daniel Dietzler
9bada51d56
feat(server, web)!: Move reverse geocoding settings to the UI (#4222)
* feat: reverse geocoding settings

* chore: open api

* re-init geocoder if precision has been updated

* update docs

* chore: update verbiage

* fix: re-init logic

* fix: reset to default

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-09-26 14:03:57 +07:00
Jason Rasmussen
7bc6e9ef64
refactor(server): person thumbnail job (#4233)
* refactor(server): person thumbnail job

* fix(server): set feature photo
2023-09-26 14:03:22 +07:00
Mert
56cf9464af
fix(server): use srgb pipeline for srgb images (#4101)
* added color-related exif fields

* remove metadata check, conditional pipe colorspace

* check exif metadata for srgb

* added migration

* updated e2e fixture

* uncased srgb check, search substrings

* extracted exif logic into separate function

* handle images with no bit depth or color metadata

* added unit tests
2023-09-25 19:18:47 -04:00
Daniele Ricci
014d164d99
feat(server): random assets API (#4184)
* feat(server): get random assets API

* Fix tests

* Use correct validation annotation

* Fix offset use in query

* Update API specs

* Fix typo

* Random assets e2e tests

* Improve e2e tests
2023-09-23 22:28:55 +07:00
Jonathan Jogenfors
acdc66413c
feat(server,web): libraries (#3124)
* feat: libraries

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-09-20 13:16:33 +02:00
Daniel Dietzler
1a633f3fca
chore(server): Use access core for person permissions (#4138)
* use access core for all person methods

* minor fixes, feedback

* reorder assignments

* remove unnecessary permission requirement

* unify naming of tests

* reorder variables
2023-09-18 21:22:44 +00:00
Daniel Dietzler
f1c98ac9e6
fix(server): Error when loading album with deleted owner (#4086)
* soft delete albums when user gets soft deleted

* fix wrong intl openapi version

* fix tests

* ability to restore albums, automatically restore when user restored

* (e2e) tests for shared albums via link and with user

* (e2e) test deletion of users and linked albums

* (e2e) fix share album with owner test

* fix: deletedAt

* chore: fix restore order

* fix: use timezone date column

* chore: cleanup e2e tests

* (e2e) fix user delete test

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-09-18 11:56:50 -04:00
Jason Rasmussen
74d34b4f6c
refactor(server): android motion photos (#3711) 2023-09-13 10:46:37 +07:00
Daniel Dietzler
7173af60e4
chore(server): Improve test coverage! (#3889)
* tests for person service

* tests for auth service

* tests for access core

* improve tests for album service

* fix missing brackets and remove comments

* tests for asset service

* tests for face recognition

* tests for job service

* feedback

* tests for search service (broken)

* fix: disabled search test

* tests for smart-info service

* tests for storage template service

* tests for user service

* fix formatting of untouched files LOL

* attempt to fix formatting

* streamline api utils, add asset api for uploading files

* test upload of assets

* fix formatting

* move test-utils to correct folder

* test add assets to album

* use random bytes instead of test image

* (e2e) test albums with assets

* (e2e) complete tests for album endpoints

* (e2e) tests for asset endpoint

* fix: asset upload/import dto validation

* (e2e) tests for statistics asset endpoint

* fix wrong describe text

* (e2e) tests for people with faces

* (e2e) clean up person tests

* (e2e) tests for partner sharing endpoints

* (e2e) tests for link sharing

* (e2e) tests for the asset time bucket endpoint

* fix minor issues

* remove access.core.spec.ts

* chore: wording

* chore: organize test api files

* chore: fix test describe

* implement feedback

* fix race condition in album tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-09-11 11:56:38 -04:00
Jason Rasmussen
f1db257628
feat(server,web): server config (#4006)
* feat: server config

* chore: open api

* fix: redirect /map to /photos when disabled
2023-09-08 22:51:46 -04:00
Daniele Ricci
3432b4625f
fix(server): regenerate missing person thumbnails (#3970)
* Regenerate missing person thumbnails

* Check for empty string instead of zero length

* Remember asset used as person face

* Define entity relation between person and asset via faceAssetId

* Typo

* Fix entity relation

* Tests

* Tests

* Fix code formatting

* Fix import path

* Fix migration

* format

* Fix entity and migration

* Linting

* Remove unneeded cast

* Conventions

* Simplify queries

* Simplify queries

* Remove unneeded typings from entity

* Remove unneeded cast

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-09-08 13:49:43 +07:00
Jason Rasmussen
15bfceb05a
fix(server): update asset with tagged people (#4000) 2023-09-07 03:51:09 +07:00
Maarten Rijke
b4fa60d4fd
feat(web): show original uploader in shared album photo details (#3977)
* feat(web): show original uploader in shared album photo details

* feat: send owner in asset by id response

* chore: open api

* fix: linting

* fix: change to Shared By

* openapi

* openapi

* api

* styling

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-09-06 10:14:44 +07:00
Jason Rasmussen
454737ca79
refactor(server): update asset endpoint (#3973)
* refactor(server): update asset

* chore: open api
2023-09-04 22:25:31 -04:00
Jason Rasmussen
816d040d81
fix(server): lint import order (#3974)
* fix: use prettier extension

* chore: format fix
2023-09-04 21:45:59 +02:00
Mert
f8ff342852
feat(server): advanced settings for transcoding (#3775)
* set stream with `-map` flag

* updated tests

* fixed audio stream mapping

* added bframe setting to config

* updated api

* added b-frame option in dashboard

* updated tests and formatting

* "Advanced" section for FFmpeg with extra options

* updated api

* updated tests and formatting

* styling

* made vp9 bitstream filters conditional on b-frames

* fixed gop size condition

* add cq override

* simplified isEdited conditions

* simplified conditional flow for cq mode

* fixed dto

* clarified cq mode in description

* formatting

* added npl setting

* Adjusted b-frame title and description

* fixed rebase

* changed defaults for pascal compatibility, added temporal aq setting

* updated api

* added temporal aq to ui

* polished dashboard

* formatting
2023-09-03 08:22:42 +07:00
Jason Rasmussen
4e5bf7ae2e
test: server-info e2e tests (#3948) 2023-09-01 22:01:54 -04:00
Mert
9539a361e4
fix(server): non-nullable IsOptional (#3939)
* custom `IsOptional`

* added link to source

* formatting

* Update server/src/domain/domain.util.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* nullable birth date endpoint

* made `nullable` a property

* formatting

* removed unused dto

* updated decorator arg

* fixed album e2e tests

* add null tests for auth e2e

* add null test for person e2e

* fixed tests

* added null test for user e2e

* removed unusued import

* log key in test name

* chore: add note about mobile not being able to use the endpoint

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-09-01 16:40:00 +00:00
Jason Rasmussen
a26ed3d1a6
refactor(web,server): use feature flags for oauth (#3928)
* refactor: oauth to use feature flags

* chore: open api

* chore: e2e test for authorize endpoint
2023-09-01 18:08:42 +07:00
Mert
bcc36d14a1
feat(ml)!: customizable ML settings (#3891)
* consolidated endpoints, added live configuration

* added ml settings to server

* added settings dashboard

* updated deps, fixed typos

* simplified modelconfig

updated tests

* Added ml setting accordion for admin page

updated tests

* merge `clipText` and `clipVision`

* added face distance setting

clarified setting

* add clip mode in request, dropdown for face models

* polished ml settings

updated descriptions

* update clip field on error

* removed unused import

* add description for image classification threshold

* pin safetensors for arm wheel

updated poetry lock

* moved dto

* set model type only in ml repository

* revert form-data package install

use fetch instead of axios

* added slotted description with link

updated facial recognition description

clarified effect of disabling tasks

* validation before model load

* removed unnecessary getconfig call

* added migration

* updated api

updated api

updated api

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-08-29 08:58:00 -05:00
Mert
e510e733cd
fix(server): extract motion photo android single frame (#3903) 2023-08-29 04:01:42 -05:00
Alex
d0a06739d8
chore(server): bump server dependencies (#3899)
* chore(server): bump server dependencies

* fix: test
2023-08-28 14:41:57 -05:00
Daniel Dietzler
59bb727636
feat(web, server): Ability to use config file instead of admin UI (#3836)
* implement method to read config file

* getConfig returns config file if present

* return isConfigFile for http requests

* disable elements if config file is used, show message if config file is set, copy existing config to clipboard

* fix allowing partial configuration files

* add new env variable to docs

* fix tests

* minor refactoring, address review

* adapt config type in frontend

* remove unnecessary imports

* move config file reading to system-config repo

* add documentation

* fix code formatting in system settings page

* add validator for config file

* fix formatting in docs

* update generated files

* throw error when trying to update config. e.g. via cli or api

* switch to feature flags for isConfigFile

* refactoring

* refactor: config file

* chore: open api

* feat: always show copy/export buttons

* fix: default flags

* refactor: copy to clipboard

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-08-25 13:44:52 -04:00
Fynn Petersen-Frey
cf9e04c8ec
feat(server): asset entity audit (#3824)
* feat(server): audit log

* feedback

* Insert to database

* migration

* test

* controller/repository/service

* test

* module

* feat(server): implement audit endpoint

* directly return changed assets

* add daily cleanup of audit table

* fix tests

* review feedback

* ci

* refactor(server): audit implementation

* chore: open api

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Co-authored-by: Fynn Petersen-Frey <zoodyy@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-08-24 15:28:50 -04:00
Daniele Ricci
98b72fdb9b
feat: set person birth date (web only) (#3721)
* Person birth date (data layer)

* Person birth date (data layer)

* Person birth date (service layer)

* Person birth date (service layer, API)

* Person birth date (service layer, API)

* Person birth date (UI) (wip)

* Person birth date (UI) (wip)

* Person birth date (UI) (wip)

* Person birth date (UI) (wip)

* UI: Use "date of birth" everywhere

* UI: better modal dialog

Similar to the API key modal.

* UI: set date of birth from people page

* Use typed events for modal dispatcher

* Date of birth tests (wip)

* Regenerate API

* Code formatting

* Fix Svelte typing

* Fix Svelte typing

* Fix person model [skip ci]

* Minor refactoring [skip ci]

* Typed event dispatcher [skip ci]

* Refactor typed event dispatcher [skip ci]

* Fix unchanged birthdate check [skip ci]

* Remove unnecessary custom transformer [skip ci]

* PersonUpdate: call search index update job only when needed

* Regenerate API

* Code formatting

* Fix tests

* Fix DTO

* Regenerate API

* chore: verbiage and view mode

* feat: show current age

* test: person e2e

* fix: show name for birth date selection

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-08-18 16:10:29 -04:00
Jason Rasmussen
bab739efbd
restore: bulk actions (#3730)
* feat: improve bulk isArchive and isFavorite updates

* chore: open api
2023-08-16 15:04:55 -05:00
Jason Rasmussen
4762fd83d4
fix(server): link live photos after metadata extraction finishes (#3702)
* fix(server): link live photos after metadata extraction finishes

* chore: fix test

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-08-15 20:34:57 -05:00
Jason Rasmussen
5cd13227ad
feat(web): timeline bucket for albums (4) (#3604)
* feat: server changes for album timeline

* feat(web): album timeline view

* chore: open api

* chore: remove archive action

* fix: favorite for non-owners
2023-08-11 11:00:51 -05:00
Jason Rasmussen
a6eb227330
feat(web,server): user memory settings (#3628)
* feat(web,server): user preference for time-based memories

* chore: open api

* dev: mobile

* fix: update

* mobile work

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: Alex Tran <Alex.Tran@conductix.com>
2023-08-09 21:01:16 -05:00
Mert
1d37d8cac0
feat(server): tone-mapping (#3512)
* added tonemapping

* check for hdr in transcode policy

* merged video thumbnail and transcoding logic

* updated tests

* removed log

* added dashboard option, updated api

* `out_color_matrix` for sdr video thumbs, cleanup

* updated tests & styling

* refactored tonemapping setting

* fixed tests

* formatting

* added tests

* updated api

* set target npl higher for mobius and reinhard

* convert to nv12 before nvenc

* fix test

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-08-07 15:35:25 -05:00
Jason Rasmussen
2f26a7edae
feat(server/web): album description (#3558)
* feat(server): add album description

* chore: open api

* fix: tests

* show and edit description on the web

* fix test

* remove unused code

* type event

* format fix

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-08-06 02:43:26 +00:00
Jason Rasmussen
c6abef186c
refactor(server,web): time buckets for main timeline, archived, and favorites (1) (#3537)
* refactor: time buckets

* feat(web): use new time bucket api

* feat(web): use asset grid in archive/favorites

* chore: open api

* chore: clean up uuid validation

* refactor(web): move memory lane to photos page

* Update web/src/routes/(user)/archive/+page.svelte

Co-authored-by: Sergey Kondrikov <sergey.kondrikov@gmail.com>

* fix: hide archived photos on main timeline

* fix: select exif info

---------

Co-authored-by: Sergey Kondrikov <sergey.kondrikov@gmail.com>
2023-08-04 17:07:15 -04:00
Mert
ee49f470b7
feat(server): transcoding hardware acceleration (#3171)
* added transcode configs for nvenc,qsv and vaapi

* updated dev docker compose

* added software fallback

* working vaapi

* minor fixes and added tests

* updated api

* compile libvips

* move hwaccel settings to `hwaccel.yml`

* changed default dockerfile, moved `readdir` call

* removed unused import

* minor cleanup

* fix for arm build

* added documentation, minor fixes

* added intel driver

* updated docs

styling

* uppercase codec and api names

* formatting

* added tests

* updated docs

* removed semicolons

* added link to `hwaccel.yml`

* added newlines

* added `hwaccel` section to docker-compose.prod.yml

* ensure mesa drivers are installed

* switch to mimalloc for sharp

* moved build version and sha256 to json

* let libmfx set the render device

* possible fix for vp9 on qsv

* updated tests

* formatting

* review suggestions

* semicolon

* moved `LD_PRELOAD` to start script

* switched to jellyfin's ffmpeg package

* fixed dockerfile

* use cqp instead of icq for qsv vp9

* updated dockerfile

* added sha256sum for other platforms

* fixtures
2023-08-01 20:56:10 -05:00
Jason Rasmussen
b9cda59172
refactor(server)!: add/remove album assets (#3109)
* refactor: add/remove album assets

* chore: open api

* feat: remove owned assets from album

* refactor: move to bulk id req/res dto

* chore: open api

* chore: merge main

* dev: mobile work

* fix: adding asset from web not sync with mobile

* remove print statement

---------

Co-authored-by: Alex Tran <Alex.Tran@conductix.com>
2023-08-01 20:29:14 -05:00
Jason Rasmussen
e53625b067
test(server): auth e2e (#3492)
* test(server): auth controller e2e test

* test(server): user e2e test

* refactor(server): album e2e

* fix: linting
2023-08-01 10:49:50 -05:00
Jason Rasmussen
9e085c1071
refactor(server): test fixtures (#3491) 2023-07-31 20:28:07 -05:00
Dmitry Brazhenko
e071b82e8a
feat (web/server) 360 degrees Web panoramas [attempt 2] (#3412)
* commit 1 (isPanorama: boolean)

* working solution for projectiontypeenum

* fix

* format fix

* fix

* fix

* fix

* fix

* enum projectiontype

* working solution with exif

* fix

* reverted >

* fix format

* reverted auto-magic api.ts prettification

* fix

* reverted api.ts autogenerated

* api ts regenerated

* Update web/src/lib/components/assets/thumbnail/thumbnail.svelte

Co-authored-by: Sergey Kondrikov <sergey.kondrikov@gmail.com>

* Update web/src/lib/components/asset-viewer/asset-viewer.svelte

Co-authored-by: Sergey Kondrikov <sergey.kondrikov@gmail.com>

* exifProjectionType

* Update server/src/microservices/processors/metadata-extraction.processor.ts

Co-authored-by: Sergey Kondrikov <sergey.kondrikov@gmail.com>

* projectionType?: string = ProjectionType.NONE;

* not null

* projectionType!: ProjectionType;

* opeapi generator fix

* fixes

* fix

* fix

* generate api

* asset.exifInifo?.projectionType

* Update server/src/domain/asset/response-dto/exif-response.dto.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* Update server/src/microservices/processors/metadata-extraction.processor.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* enum -> varchar;projectiontypeenum->projectiontype

* asset-viewer fixed prettiffier

* @Column({}) single line

* enum | string

* make api

* enum | string

* enum | str fix

* fix

* chore: use string instead of enum

* chore: open api

* fix: checks

---------

Co-authored-by: Sergey Kondrikov <sergey.kondrikov@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-07-27 23:29:09 -05:00
martin
f28fc8fa5c
feat(server,web): hide faces (#3262)
* feat: hide faces

* fix: types

* pr feedback

* fix: svelte checks

* feat: new server endpoint

* refactor: rename person count dto

* fix(server): linter

* fix: remove duplicate button

* docs: add comments

* pr feedback

* fix: get unhidden faces

* fix: do not use PersonCountResponseDto

* fix: transition

* pr feedback

* pr feedback

* fix: remove unused check

* add server tests

* rename persons to people

* feat: add exit button

* pr feedback

* add server tests

* pr feedback

* pr feedback

* fix: show & hide faces

* simplify

* fix: close button

* pr feeback

* pr feeback

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-07-18 13:09:43 -05:00
Jason Rasmussen
08c7054845
refactor(server): auth/oauth (#3242)
* refactor(server): auth/oauth

* fix: show server error message on login failure
2023-07-14 23:03:56 -05:00
Jason Rasmussen
f952bc0b64
refactor(server): asset stats (#3253)
* refactor(server): asset stats

* chore: open api
2023-07-14 08:30:17 -05:00
Alex
c86b2ae500
feat(web/server): merge faces (#3121)
* feat(server/web): Merge faces

* get parent id

* update

* query to get identical asset and change controller

* change delete asset signature

* delete identical assets

* gaming time

* delete merge person

* query

* query

* generate api

* pr feedback

* generate api

* naming

* remove unused method

* Update server/src/domain/person/person.service.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* Update server/src/domain/person/person.service.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* better method signature

* cleaning up

* fix bug

* added interfaces

* added tests

* merge main

* api

* build merge face interface

* api

* selector interface

* style

* more style

* clean up import

* styling

* styling

* better

* styling

* styling

* add merge face diablog

* finished

* refactor: merge person endpoint

* refactor: merge person component

* chore: open api

* fix: tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-07-11 16:52:41 -05:00
Jason Rasmussen
6180828ed2
refactor(server): mime types (#3197)
* refactor(server): mime type check

* chore: open api

* chore: remove duplicate test
2023-07-10 12:56:45 -05:00
Jason Rasmussen
398bd04ffd
refactor(server): upload config (#3148) 2023-07-08 23:37:40 -05:00
Mert
8349a28ed8
refactor(server): modularize getFfmpegOptions (#3138)
* refactored `getFfmpegOptions`

refactor transcoding, make separate service

* fixed enum casing

* use `Logger` instead of `console.log`

* review suggestions

* use enum for `getHandler`

* fixed formatting

* Update server/src/domain/media/media.util.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* Update server/src/domain/media/media.util.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* More specific imports, renamed codec classes

* simplified code

* removed unused import

* added tests

* added base implementation for bitrate and threads

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-07-08 21:43:11 -05:00
Alex
7947f4db4c
feat(web/server): Face thumbnail selection (#3081)
* add migration

* verify running migration populate new value

* implemented service

* generate api

* FE works

* FR Works

* fix test

* fix test fixture

* fix test

* fix test

* consolidate api

* fix test

* added test

* pr feedback

* refactor

* click ont humbnail to show feature selection as well
2023-07-02 17:46:20 -05:00
Jason Rasmussen
d69fa3ceae
refactor(server): guards, decorators, and utils (#3060) 2023-07-01 13:27:34 -05:00
Jason Rasmussen
ad343b7b32
refactor(server): download assets (#3032)
* refactor: download assets

* chore: open api

* chore: finish tests, make size configurable

* chore: defualt to 4GiB

* chore: open api

* fix: optional archive size

* fix: bugs

* chore: cleanup
2023-06-30 11:24:28 -05:00
Jason Rasmussen
e98398cab8
refactor(server): access permissions (#2910)
* refactor: access repo interface

* feat: access core

* fix: allow shared links to add to a shared link

* chore: comment out unused code

* fix: pr feedback

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-06-28 08:56:24 -05:00
Alex Phillips
e171fec5aa
feat(server): support for read-only assets and importing existing items in the filesystem (#2715)
* Added read-only flag for assets, endpoint to trigger file import vs upload

* updated fixtures with new property

* if upload is 'read-only', ensure there is no existing asset at the designated originalPath

* added test for file import as well as detecting existing image at read-only destination location

* Added storage service test for a case where it should not move read-only assets

* upload doesn't need the read-only flag available, just importing

* default isReadOnly on import endpoint to true

* formatting fixes

* create-asset dto needs isReadOnly, so set it to false by default on create, updated api generation

* updated code to reflect changes in MR

* fixed read stream promise return type

* new index for originalPath, check for existing path on import, reglardless of user, to prevent duplicates

* refactor: import asset

* chore: open api

* chore: tests

* Added externalPath support for individual users, updated UI to allow this to be set by admin

* added missing var for externalPath in ui

* chore: open api

* fix: compilation issues

* fix: server test

* built api, fixed user-response dto to include externalPath

* reverted accidental commit

* bad commit of duplicate externalPath in user response  dto

* fixed tests to include externalPath on expected result

* fix: unit tests

* centralized supported filetypes, perform file type checking of asset and sidecar during file import process

* centralized supported filetype check method to keep regex DRY

* fixed typo

* combined migrations into one

* update api

* Removed externalPath from shared-link code, added column to admin user page whether external paths / import is enabled or not

* update mimetype

* Fixed detect correct mimetype

* revert asset-upload config

* reverted domain.constant

* refactor

* fix mime-type issue

* fix format

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-06-21 21:33:20 -05:00
Jason Rasmussen
868f629f32
refactor(server, web): create shared link (#2879)
* refactor: shared links

* chore: open api

* fix: tsc error
2023-06-20 20:08:43 -05:00
Krisjanis Lejejs
746ca5d5ed
feat(web): Add album sorting to albums view (#2861)
* Add album sorting to web albums view

* generate api

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-06-20 20:00:59 -05:00
Covalent
3e804f16df
feat(web,server): add thumbhash support (#2649)
* add thumbhash: server generation and web impl

* move logic to infra & use byta in db

* remove unnecesary logs

* update generated API and simplify thumbhash gen

* fix check errors

* removed unnecessary library and css tag

* style edits

* syntax mistake

* update server test, change thumbhash job name

* fix tests

* Update server/src/domain/asset/response-dto/asset-response.dto.ts

Co-authored-by: Thomas <9749173+uhthomas@users.noreply.github.com>

* add unit test, change migration date

* change to official thumbhash impl

* update call method to not use eval

* "generate missing" looks for thumbhash

* improve queue & improve syntax

* update syntax again

* update tests

* fix thumbhash generation

* consolidate queueing to avoid duplication

* cover all types of incorrect thumbnail cases

* split out jest tasks

* put back thumbnail duration loading for images without thumbhash

* Remove stray package.json

---------

Co-authored-by: Luke McCarthy <mail@lukehmcc.com>
Co-authored-by: Thomas <9749173+uhthomas@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-06-17 22:22:31 -05:00
Jason Rasmussen
bff6914a73
chore(server): organize imports (#2779)
* feat: lint rule for organize imports

* chore: organize imports
2023-06-16 19:54:17 +00:00
Jason Rasmussen
f04e47803c
refactor(server): access checks (#2776)
* refactor(server): access checks

* chore: simply asset module
2023-06-16 14:01:34 -05:00
Thomas
3b4f6edbdb
don't fallback to exiftool for embedded image previews (#2747)
Given #2668 introduced support for imagemagick and libraw, this should no
longer be necessary which allow for reduced code footprint and complexity.

Fixes: #2744
2023-06-14 22:42:35 -05:00
Alex
43ec0b77a0
feat(web): Memory (#2759)
* Add on this day

* add query for x year

* dev: add query

* dev: front end

* dev: styling

* styling

* more styling

* add new page

* navigating

* navigate back and forth

* styling

* show gallery

* fix test

* fix test

* show previous and next title

* fix test

* show up down scrolling button

* more styling

* styling

* fix app bar

* fix height of next/previous

* autoplay

* auto play

* refactor

* refactor

* refactor

* show date

* Navigate

* finish

* pr feedback
2023-06-14 20:47:18 -05:00
Jason Rasmussen
8ebac41318
refactor(server)*: tsconfigs (#2689)
* refactor(server): tsconfigs

* chore: dummy commit

* fix: start.sh

* chore: restore original entry scripts
2023-06-08 10:01:07 -05:00