1
0
mirror of https://github.com/immich-app/immich.git synced 2025-08-08 23:07:06 +02:00

500 Commits

Author SHA1 Message Date
17eaeb695e feat: smart merge (#6508)
* pr feedback

* fix: tests

* update assets statistics

* pr feedback

* pr feedback

* fix: linter

* pr feedback

* fix: don't limit the smart merge

* pr feedback

* fix: server code

* remove slider

* fix: tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-19 17:52:26 +00:00
07b874edda fix(web): revert smart merge (#6504)
* revert smart merge

* fix test

* fix test

* Remove Slider file
2024-01-19 11:34:20 -06:00
68f52818ae feat(server): separate face clustering job (#5598)
* separate facial clustering job

* update api

* fixed some tests

* invert clustering

* hdbscan

* update api

* remove commented code

* wip dbscan

* cleanup

removed cluster endpoint

remove commented code

* fixes

updated tests

minor fixes and formatting

fixed queuing

refinements

* scale search range based on library size

* defer non-core faces

* optimizations

removed unused query option

* assign faces individually for correctness

fixed unit tests

remove unused method

* don't select face embedding

update sql

linting

fixed ml typing

* updated job mock

* paginate people query

* select face embeddings because typeorm

* fix setting face detection concurrency

* update sql

formatting

linting

* simplify logic

remove unused imports

* more specific delete signature

* more accurate typing for face stubs

* add migration

formatting

* chore: better typing

* don't select embedding by default

remove unused import

* updated sql

* use normal try/catch

* stricter concurrency typing and enforcement

* update api

* update job concurrency panel to show disabled queues

formatting

* check jobId in queueAll

fix tests

* remove outdated comment

* better facial recognition icon

* wording

wording

formatting

* fixed tests

* fix

* formatting & sql

* try to fix sql check

* more detailed description

* update sql

* formatting

* wording

* update `minFaces` description

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-18 00:08:48 -05:00
9a2fa21b28 fix(server): scale transcoded videos if dimensions are odd (#6461)
scale if odd resolution
2024-01-17 22:16:44 -05:00
f0b328fb6b feat(server, web): smart merge (#5796)
* pr feedback

* fix: tests

* update assets statistics

* pr feedback

* pr feedback

* fix: linter

* pr feedback

* fix: don't limit the smart merge

* pr feedback

* fix: server code

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-18 01:52:11 +00:00
deb1f970a8 feat(server, web): quotas (#4471)
* feat: quotas

* chore: open api

* chore: update status box and upload error message

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-12 19:43:36 -05:00
bf1dd36fa9 refactor(server): split api and jobs into separate e2e suites (#6307)
* refactor: domain and infra modules

* refactor(server): e2e tests
2024-01-09 23:04:16 -05:00
12dc7c48c9 refactor(server): domain and infra modules (#6301) 2024-01-09 17:07:01 -05:00
4cf1e553d2 feat(server): in upload folder, split the files into folders based on the first four of the files uuid (#6175) 2024-01-04 14:45:16 -06:00
18f59f78e3 feat(web): onboarding (#6066)
* feat(web): onboarding

* feat: openapi

* feat: modulization

* feat: page advancing

* Animation

* Add storage templaete settings

* sql

* more style

* Theme

* information and styling

* hide/show table

* Styling

* Update user property

* fix test

* fix test:

* fix e2e

* test

* Update web/src/lib/components/onboarding-page/onboarding-hello.svelte

Co-authored-by: bo0tzz <git@bo0tzz.me>

* naming

* use System Metadata

* better return type

* onboarding using server metadata

* revert previous changes in user entity

* sql

* test web

* fix test server

* server/web test

* more test

* consolidate color theme change logic

* consolidate save button to storage template

* merge main

* fix web

---------

Co-authored-by: bo0tzz <git@bo0tzz.me>
2024-01-04 05:28:32 +00:00
317adc5c28 feat(web,server): external domain setting (#6146)
* feat: external domain setting

* chore: open api

* mobile: handle serverconfig-externalDomain

---------

Co-authored-by: shenlong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
2024-01-03 21:54:48 -05:00
4a5b8c3770 feat(server): Enqueue jobs in bulk (#5974)
* feat(server): Enqueue jobs in bulk

The Job Repository now has a `queueAll` method, that enqueues messages
in bulk (using BullMQ's
[`addBulk`](https://docs.bullmq.io/guide/queues/adding-bulks)),
improving performance when many jobs must be enqueued within the same
operation.

Primary change is in `src/domain/job/job.service.ts`, and other services
have been refactored to use `queueAll` when useful.

As a simple local benchmark, triggering a full thumbnail generation
process over a library of ~1,200 assets and ~350 faces went from
**~600ms** to **~250ms**.

* fix: Review feedback
2024-01-01 15:45:42 -05:00
2e38fa73bf feat: storage template file move hardening (#5917)
* fix: pgvecto.rs extension breaks typeorm schema:drop command

* fix: parse postgres bigints to javascript number types when selecting data

* feat: verify file size is the same as original asset after copying file for storage template job

* feat: allow disabling of storage template job, defaults to disabled for new instances

* fix: don't allow setting concurrency for storage template migration, can cause race conditions above 1

* feat: add checksum verification when file is copied for storage template job

* fix: extract metadata for assets that aren't visible on timeline
2023-12-29 18:41:33 +00:00
a1e1f11399 feat(server): delete unnecessary encoded videos (#6027)
* delete unnecessary transcodes

* added test
2023-12-27 23:34:00 -06:00
8119d4bb26 chore(server): refactor locks (#5953)
* lock refactor

* add mocks

* add await

* move database repo injection to service

* update tests

* add mock implementation

* remove unused imports

* this
2023-12-27 18:36:51 -05:00
c0ebc943d2 fix(server): strip metadata from thumbnails (#4438)
included all thumbnail metadata. It seems this has to be explicitly disabled.

Refs: #4382

feat. basic metadata e2e test

fix: use tiff thumbnails in first step + e2e fix

fix: revert switch to tiff

feat: test metadata of both webp and jpg

feat: use upload in e2e test

fix: lint

strip metadata with exiftool

use `withIccProfile`

fix e2e

formatting

run jobs in e2e
2023-12-26 16:27:51 -05:00
cc2dc12f6c fix(server): run migrations after database checks (#5832)
* run migrations after checks

* optional migrations

* only run checks in server and e2e

* re-add migrations for microservices

* refactor

* move e2e init

* remove assert from migration

* update providers

* update microservices app service

* fixed logging

* refactored version check, added unit tests

* more version tests

* don't use mocks for sut

* refactor tests

* suggest image only if postgres is 14, 15 or 16

* review suggestions

* fixed regexp escape

* fix typing

* update migration
2023-12-21 10:06:26 -06:00
092a23fd7f feat(server,ml): remove image tagging (#5903)
* remove image tagging

* updated lock

* fixed tests, improved logging

* be nice

* fixed tests
2023-12-20 20:47:56 -05:00
4e9b96ff1a test(cli): e2e testing (#5101)
* Allow building and installing cli

* feat: add format fix

* docs: remove cli folder

* feat: use immich scoped package

* feat: rewrite cli readme

* docs: add info on running without building

* cleanup

* chore: remove import functionality from cli

* feat: add logout to cli

* docs: add todo for file format from server

* docs: add compilation step to cli

* fix: success message spacing

* feat: can create albums

* fix: add check step to cli

* fix: typos

* feat: pull file formats from server

* chore: use crawl service from server

* chore: fix lint

* docs: add cli documentation

* chore: rename ignore pattern

* chore: add version number to cli

* feat: use sdk

* fix: cleanup

* feat: album name on windows

* chore: remove skipped asset field

* feat: add more info to server-info command

* chore: cleanup

* wip

* chore: remove unneeded packages

* e2e test can start

* git ignore for geocode in cli

* add cli e2e to github actions

* can do e2e tests in the cli

* simplify e2e test

* cleanup

* set matrix strategy in workflow

* run npm ci in server

* choose different working directory

* check out submodules too

* increase test timeout

* set node version

* cli docker e2e tests

* fix cli docker file

* run cli e2e in correct folder

* set docker context

* correct docker build

* remove cli from dockerignore

* chore: fix docs links

* feat: add cli v2 milestone

* fix: set correct cli date

* remove submodule

* chore: add npmignore

* chore(cli): push to npm

* fix: server e2e

* run npm ci in server

* remove state from e2e

* run npm ci in server

* reshuffle docker compose files

* use new e2e composes in makefile

* increase test timeout to 10 minutes

* make github actions run makefile e2e tests

* cleanup github test names

* assert on server version

* chore: split cli e2e tests into one file per command

* chore: set cli release working dir

* chore: add repo url to npmjs

* chore: bump node setup to v4

* chore: normalize the github url

* check e2e code in lint

* fix lint

* test key login flow

* feat: allow configurable config dir

* fix session service tests

* create missing dir

* cleanup

* bump cli version to 2.0.4

* remove form-data

* feat: allow single files as argument

* add version option

* bump dependencies

* fix lint

* wip use axios as upload

* version bump

* cApiTALiZaTiON

* don't touch package lock

* wip: don't use job queues

* don't use make for cli e2e

* fix server e2e

* chore: remove old gha step

* add npm ci to server

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-12-18 20:29:26 -06:00
de1514a441 chore(server): startup check for pgvecto.rs (#5815)
* startup check for pgvecto.rs

* prefilter after assertion

* formatting

* add assert to migration

* more specific import

* use runner
2023-12-18 10:38:25 -06:00
6e7b3d6f24 fix(server): fix metadata search not working (#5800)
* don't require ml

* update e2e

* fixes

* fix e2e

* add additional conditions

* select all exif columns

* more fixes

* update sql
2023-12-17 20:16:08 -06:00
c6f56d9591 chore(server): Check activity permissions in bulk (#5775)
Modify Access repository, to evaluate `asset` permissions in bulk.
This is the last set of permission changes, to migrate all of them to
run in bulk!
Queries have been validated to match what they currently generate for single ids.

Queries:

* `activity` owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "activity" "ActivityEntity"
  WHERE
    "ActivityEntity"."id" = $1
    AND "ActivityEntity"."userId" = $2
)
LIMIT 1

-- After
SELECT "ActivityEntity"."id" AS "ActivityEntity_id"
FROM "activity" "ActivityEntity"
WHERE
  "ActivityEntity"."id" IN ($1)
  AND "ActivityEntity"."userId" = $2
```

* `activity` album owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "activity" "ActivityEntity"
    LEFT JOIN "albums" "ActivityEntity__ActivityEntity_album"
      ON "ActivityEntity__ActivityEntity_album"."id"="ActivityEntity"."albumId"
      AND "ActivityEntity__ActivityEntity_album"."deletedAt" IS NULL
  WHERE
    "ActivityEntity"."id" = $1
    AND "ActivityEntity__ActivityEntity_album"."ownerId" = $2
)
LIMIT 1

-- After
SELECT "ActivityEntity"."id" AS "ActivityEntity_id"
FROM "activity" "ActivityEntity"
  LEFT JOIN "albums" "ActivityEntity__ActivityEntity_album"
    ON "ActivityEntity__ActivityEntity_album"."id"="ActivityEntity"."albumId"
    AND "ActivityEntity__ActivityEntity_album"."deletedAt" IS NULL
WHERE
  "ActivityEntity"."id" IN ($1)
  AND "ActivityEntity__ActivityEntity_album"."ownerId" = $2
```

* `activity` create access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "albums" "AlbumEntity"
    LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
    LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
      AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
  WHERE
    (
      (
        "AlbumEntity"."id" = $1
        AND "AlbumEntity"."isActivityEnabled" = $2
        AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
      )
      OR (
        "AlbumEntity"."id" = $4
        AND "AlbumEntity"."isActivityEnabled" = $5
        AND "AlbumEntity"."ownerId" = $6
      )
    )
    AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1

-- After
SELECT "AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
  LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
    ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
  LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
    ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
    AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
  (
    (
      "AlbumEntity"."id" IN ($1)
      AND "AlbumEntity"."isActivityEnabled" = $2
      AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
    )
    OR (
      "AlbumEntity"."id" IN ($4)
      AND "AlbumEntity"."isActivityEnabled" = $5
      AND "AlbumEntity"."ownerId" = $6
    )
  )
  AND "AlbumEntity"."deletedAt" IS NULL
```
2023-12-17 12:10:21 -06:00
9768931275 feat(web,server)!: runtime log level (#5672)
* feat: change log level at runtime

* chore: open api

* chore: prefer env over runtime

* chore: remove default env value
2023-12-14 16:55:40 +00:00
b34abf25f0 feat(server): server-side events (#5669) 2023-12-13 12:23:51 -05:00
ed4358741e feat(web): re-add open graph tags for public share links (#5635)
* feat: re-add open graph tags for public share links

* fix: undefined in html

* chore: tests
2023-12-11 13:37:47 -06:00
33529d1d9b refactor(server): auth dto (#5593)
* refactor: AuthUserDto => AuthDto

* refactor: reorganize auth-dto

* refactor: AuthUser() => Auth()
2023-12-09 23:34:12 -05:00
1e99ba8167 feat: use pgvecto.rs (#3605) 2023-12-08 11:15:46 -05:00
7702560b12 feat(web): re-assign person faces (2) (#4949)
* feat: unassign person faces

* multiple improvements

* chore: regenerate api

* feat: improve face interactions in photos

* fix: tests

* fix: tests

* optimize

* fix: wrong assignment on complex-multiple re-assignments

* fix: thumbnails with large photos

* fix: complex reassign

* fix: don't send people with faces

* fix: person thumbnail generation

* chore: regenerate api

* add tess

* feat: face box even when zoomed

* fix: change feature photo

* feat: make the blue icon hoverable

* chore: regenerate api

* feat: use websocket

* fix: loading spinner when clicking on the done button

* fix: use the svelte way

* fix: tests

* simplify

* fix: unused vars

* fix: remove unused code

* fix: add migration

* chore: regenerate api

* ci: add unit tests

* chore: regenerate api

* feat: if a new person is created for a face and the server takes more than 15 seconds to generate the person thumbnail, don't wait for it

* reorganize

* chore: regenerate api

* feat: global edit

* pr feedback

* pr feedback

* simplify

* revert test

* fix: face generation

* fix: tests

* fix: face generation

* fix merge

* feat: search names in unmerge face selector modal

* fix: merge face selector

* simplify feature photo generation

* fix: change endpoint

* pr feedback

* chore: fix merge

* chore: fix merge

* fix: tests

* fix: edit & hide buttons

* fix: tests

* feat: show if person is hidden

* feat: rename face to person

* feat: split in new panel

* copy-paste-error

* pr feedback

* fix: feature photo

* do not leak faces

* fix: unmerge modal

* fix: merge modal event

* feat(server): remove duplicates

* fix: title for image thumbnails

* fix: disable side panel when there's no face until next PR

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-12-05 09:43:15 -06:00
982183600d feat(web): clear failed jobs (#5423)
* add clear failed jobs button

* refactor: clean up code

* chore: open api

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-12-05 02:07:20 +00:00
5aa658de59 chore(server): Check asset permissions in bulk (#5329)
Modify Access repository, to evaluate `asset` permissions in bulk.
Queries have been validated to match what they currently generate for single ids.

Queries:

* `asset` album access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "albums" "AlbumEntity"
    LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets"
      ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id"
    LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets"
      ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId"
      AND "AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL
    LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
    LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
      AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
  WHERE
    (
      ("AlbumEntity"."ownerId" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2)
      OR ("AlbumEntity__AlbumEntity_sharedUsers"."id" = $3 AND "AlbumEntity__AlbumEntity_assets"."id" = $4)
      OR ("AlbumEntity"."ownerId" = $5 AND "AlbumEntity__AlbumEntity_assets"."livePhotoVideoId" = $6)
      OR ("AlbumEntity__AlbumEntity_sharedUsers"."id" = $7 AND "AlbumEntity__AlbumEntity_assets"."livePhotoVideoId" = $8)
    )
    AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1

-- After
SELECT
  "asset"."id" AS "assetId",
  "asset"."livePhotoVideoId" AS "livePhotoVideoId"
FROM "albums" "album"
  INNER JOIN "albums_assets_assets" "album_asset"
    ON "album_asset"."albumsId"="album"."id"
  INNER JOIN "assets" "asset"
    ON "asset"."id"="album_asset"."assetsId"
    AND "asset"."deletedAt" IS NULL
  LEFT JOIN "albums_shared_users_users" "album_sharedUsers"
    ON "album_sharedUsers"."albumsId"="album"."id"
  LEFT JOIN "users" "sharedUsers"
    ON "sharedUsers"."id"="album_sharedUsers"."usersId"
    AND "sharedUsers"."deletedAt" IS NULL
WHERE
  (
    "album"."ownerId" = $1
    OR "sharedUsers"."id" = $2
  )
  AND (
    "asset"."id" IN ($3, $4)
    OR "asset"."livePhotoVideoId" IN ($5, $6)
  )
  AND "album"."deletedAt" IS NULL
```

* `asset` owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "assets" "AssetEntity"
  WHERE
    "AssetEntity"."id" = $1
    AND "AssetEntity"."ownerId" = $2
)
LIMIT 1

-- After
SELECT
  "AssetEntity"."id" AS "AssetEntity_id"
FROM "assets" "AssetEntity"
WHERE
  "AssetEntity"."id" IN ($1, $2)
  AND "AssetEntity"."ownerId" = $3
```

* `asset` partner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "partners" "PartnerEntity"
    LEFT JOIN "users" "PartnerEntity__PartnerEntity_sharedWith"
      ON "PartnerEntity__PartnerEntity_sharedWith"."id"="PartnerEntity"."sharedWithId"
      AND "PartnerEntity__PartnerEntity_sharedWith"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__PartnerEntity_sharedBy"
      ON "PartnerEntity__PartnerEntity_sharedBy"."id"="PartnerEntity"."sharedById"
      AND "PartnerEntity__PartnerEntity_sharedBy"."deletedAt" IS NULL
    LEFT JOIN "assets" "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"
      ON "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."ownerId"="PartnerEntity__PartnerEntity_sharedBy"."id"
      AND "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__sharedBy"
      ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
      AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__sharedWith"
      ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
      AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
  WHERE
    "PartnerEntity__PartnerEntity_sharedWith"."id" = $1
    AND "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."id" = $2
)
LIMIT 1

-- After
SELECT
  "asset"."id" AS "assetId"
FROM "partners" "partner"
  INNER JOIN "users" "sharedBy"
    ON "sharedBy"."id"="partner"."sharedById"
    AND "sharedBy"."deletedAt" IS NULL
  INNER JOIN "assets" "asset"
    ON "asset"."ownerId"="sharedBy"."id"
    AND "asset"."deletedAt" IS NULL
WHERE
  "partner"."sharedWithId" = $1
  AND "asset"."id" IN ($2, $3)
```

* `asset` shared link access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "shared_links" "SharedLinkEntity"
    LEFT JOIN "albums" "SharedLinkEntity__SharedLinkEntity_album"
      ON "SharedLinkEntity__SharedLinkEntity_album"."id"="SharedLinkEntity"."albumId"
      AND "SharedLinkEntity__SharedLinkEntity_album"."deletedAt" IS NULL
    LEFT JOIN "albums_assets_assets" "760f12c00d97bdcec1ce224d1e3bf449859942b6"
      ON "760f12c00d97bdcec1ce224d1e3bf449859942b6"."albumsId"="SharedLinkEntity__SharedLinkEntity_album"."id"
    LEFT JOIN "assets" "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"
      ON "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."id"="760f12c00d97bdcec1ce224d1e3bf449859942b6"."assetsId"
      AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."deletedAt" IS NULL
    LEFT JOIN "shared_link__asset" "SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"
      ON "SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"."sharedLinksId"="SharedLinkEntity"."id"
    LEFT JOIN "assets" "SharedLinkEntity__SharedLinkEntity_assets"
      ON "SharedLinkEntity__SharedLinkEntity_assets"."id"="SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"."assetsId"
      AND "SharedLinkEntity__SharedLinkEntity_assets"."deletedAt" IS NULL
  WHERE (
    ("SharedLinkEntity"."id" = $1 AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."id" = $2)
    OR ("SharedLinkEntity"."id" = $3 AND "SharedLinkEntity__SharedLinkEntity_assets"."id" = $4)
    OR ("SharedLinkEntity"."id" = $5 AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."livePhotoVideoId" = $6)
    OR ("SharedLinkEntity"."id" = $7 AND "SharedLinkEntity__SharedLinkEntity_assets"."livePhotoVideoId" = $8)
  )
)
LIMIT 1

-- After
SELECT
  "assets"."id" AS "assetId",
  "assets"."livePhotoVideoId" AS "assetLivePhotoVideoId",
  "albumAssets"."id" AS "albumAssetId",
  "albumAssets"."livePhotoVideoId" AS "albumAssetLivePhotoVideoId"
FROM "shared_links" "sharedLink"
  LEFT JOIN "albums" "album"
    ON "album"."id"="sharedLink"."albumId"
    AND "album"."deletedAt" IS NULL
  LEFT JOIN "shared_link__asset" "assets_sharedLink"
    ON "assets_sharedLink"."sharedLinksId"="sharedLink"."id"
  LEFT JOIN "assets" "assets"
    ON "assets"."id"="assets_sharedLink"."assetsId"
    AND "assets"."deletedAt" IS NULL
  LEFT JOIN "albums_assets_assets" "album_albumAssets"
    ON "album_albumAssets"."albumsId"="album"."id"
  LEFT JOIN "assets" "albumAssets"
    ON "albumAssets"."id"="album_albumAssets"."assetsId"
    AND "albumAssets"."deletedAt" IS NULL
WHERE
  "sharedLink"."id" = $1
  AND (
    "assets"."id" IN ($2, $3)
    OR "albumAssets"."id" IN ($4, $5)
    OR "assets"."livePhotoVideoId" IN ($6, $7)
    OR "albumAssets"."livePhotoVideoId" IN ($8, $9)
  )
```
2023-12-02 02:56:41 +00:00
644e52b153 feat: Edit metadata (#5066)
* chore: rebase and clean-up

* feat: sync description, add e2e tests

* feat: simplify web code

* chore: unit tests

* fix: linting

* Bug fix with the arrows key

* timezone typeahead filter

timezone typeahead filter

* small stlying

* format fix

* Bug fix in the map selection

Bug fix in the map selection

* Websocket basic

Websocket basic

* Update metadata visualisation through the websocket

* Update timeline

* fix merge

* fix web

* fix web

* maplibre system

* format fix

* format fix

* refactor: clean up

* Fix small bug in the hour/timezone

* Don't diplay modify for readOnly asset

* Add log in case of failure

* Formater + try/catch error

* Remove everything related to websocket

* Revert "Remove everything related to websocket"

This reverts commit 14bcb9e1e4.

* remove notification

* fix test

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-11-30 03:52:28 +00:00
4e5eef129d fix(server): get album's assets in getAlbumInfo route (#5325)
* fix(server): get album's assets in getAlbumInfo route

* unit test

* test: e2e tests
2023-11-26 21:21:04 -06:00
3aa2927dae fix(web): sorting options for albums (#5233)
* fix: albums

* pr feedback

* fix: current behavior

* rename

* fix: album metadatas

* fix: tests

* fix: e2e test

* simplify

* fix: cover shared links

* rename function

* merge main

* merge main

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-11-26 15:23:43 +00:00
c04340c63e chore(server): Check more permissions in bulk (#5315)
Modify Access repository, to evaluate `authDevice`, `library`, `partner`,
`person`, and `timeline` permissions in bulk.
Queries have been validated to match what they currently generate for
single ids.

As an extra performance improvement, we now use a custom QueryBuilder
for the Partners queries, to avoid the eager relationships that add
unneeded `LEFT JOIN` clauses. We only filter based on the ids present in
the `partners` table, so those joins can be avoided.

Queries:

* `library` owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "libraries" "LibraryEntity"
  WHERE
    "LibraryEntity"."id" = $1
    AND "LibraryEntity"."ownerId" = $2
    AND "LibraryEntity"."deletedAt" IS NULL
)
LIMIT 1

-- After
SELECT "LibraryEntity"."id" AS "LibraryEntity_id"
FROM "libraries" "LibraryEntity"
WHERE
  "LibraryEntity"."id" IN ($1, $2)
  AND "LibraryEntity"."ownerId" = $3
  AND "LibraryEntity"."deletedAt" IS NULL
```

* `library` partner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "partners" "PartnerEntity"
    LEFT JOIN "users" "PartnerEntity__sharedBy"
      ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
      AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__sharedWith"
      ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
      AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
  WHERE
    "PartnerEntity"."sharedWithId" = $1
    AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1

-- After
SELECT
  "partner"."sharedById" AS "partner_sharedById",
  "partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
  "partner"."sharedById" IN ($1, $2)
  AND "partner"."sharedWithId" = $3
```

* `authDevice` owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "user_token" "UserTokenEntity"
  WHERE
    "UserTokenEntity"."userId" = $1
    AND "UserTokenEntity"."id" = $2
)
LIMIT 1

-- After
SELECT "UserTokenEntity"."id" AS "UserTokenEntity_id"
FROM "user_token" "UserTokenEntity"
WHERE
  "UserTokenEntity"."userId" = $1
  AND "UserTokenEntity"."id" IN ($2, $3)
```

* `timeline` partner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "partners" "PartnerEntity"
    LEFT JOIN "users" "PartnerEntity__sharedBy"
      ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
      AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__sharedWith"
      ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
      AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
  WHERE
    "PartnerEntity"."sharedWithId" = $1
    AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1

-- After
SELECT
  "partner"."sharedById" AS "partner_sharedById",
  "partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
  "partner"."sharedById" IN ($1, $2)
  AND "partner"."sharedWithId" = $3
```

* `person` owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "person" "PersonEntity"
  WHERE
    "PersonEntity"."id" = $1
    AND "PersonEntity"."ownerId" = $2
)
LIMIT 1

-- After
SELECT "PersonEntity"."id" AS "PersonEntity_id"
FROM "person" "PersonEntity"
WHERE
  "PersonEntity"."id" IN ($1, $2)
  AND "PersonEntity"."ownerId" = $3
```

* `partner` update access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "partners" "PartnerEntity"
    LEFT JOIN "users" "PartnerEntity__sharedBy"
      ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
      AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
    LEFT JOIN "users" "PartnerEntity__sharedWith"
      ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
      AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
  WHERE
    "PartnerEntity"."sharedWithId" = $1
    AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1

-- After
SELECT
  "partner"."sharedById" AS "partner_sharedById",
  "partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
  "partner"."sharedById" IN ($1, $2)
  AND "partner"."sharedWithId" = $3
```
2023-11-26 07:50:41 -05:00
6d1b325b34 chore(server): Check album permissions in bulk (#5290)
* chore(server): Check album permissions in bulk

Modify Access repository, to evaluate `album` permissions in bulk.
Queries have been validated to match what they currently generate for
single ids.

Queries:

* Owner access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "albums" "AlbumEntity"
  WHERE
    "AlbumEntity"."id" = $1
    AND "AlbumEntity"."ownerId" = $2
    AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1

-- After
SELECT
  "AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
WHERE
  "AlbumEntity"."id" IN ($1, $2)
  AND "AlbumEntity"."ownerId" = $3
  AND "AlbumEntity"."deletedAt" IS NULL
```

* Shared link access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "shared_links" "SharedLinkEntity"
  WHERE
    "SharedLinkEntity"."id" = $1
    AND "SharedLinkEntity"."albumId" = $2
)
LIMIT 1

-- After
SELECT
  "SharedLinkEntity"."albumId" AS "SharedLinkEntity_albumId",
  "SharedLinkEntity"."id" AS "SharedLinkEntity_id"
FROM "shared_links" "SharedLinkEntity"
WHERE
  "SharedLinkEntity"."id" = $1
  AND "SharedLinkEntity"."albumId" IN ($2, $3)
```

* Shared album access:

```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
  SELECT 1
  FROM "albums" "AlbumEntity"
    LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
    LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
      ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
      AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
  WHERE
    "AlbumEntity"."id" = $1
    AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $2
    AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1

-- After
SELECT
  "AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
  LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
    ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
  LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
    ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
    AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
  "AlbumEntity"."id" IN ($1, $2)
  AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
  AND "AlbumEntity"."deletedAt" IS NULL
```

* chore(server): Add set utils, avoid double queries for same ids

* chore(server): Review feedback
2023-11-25 17:56:23 -05:00
698226634e feat: postgres reverse geocoding (#5301)
* feat: add system metadata repository for storing key values for internal usage

* feat: add database entities for geodata

* feat: move reverse geocoding from local-reverse-geocoder to postgresql

* infra: disable synchronization for geodata_places table until typeorm supports earth column

* feat: remove cities override config as we will default all instances to cities500 now

* test: e2e tests don't clear geodata tables on reset
2023-11-25 18:53:30 +00:00
0108211c0f refactor: deprecate getUserAssetsByDeviceId (#5273)
* refactor: deprecated getUserAssetsByDeviceId

* prevent breaking changes

* chore: add deprecation

* prevent breaking changes

* prevent breaking changes

---------

Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-11-25 15:46:20 +00:00
f094ff2aa1 fix(server): album perf query (#5232)
* Revert "fix: album performances (#5224)"

This reverts commit c438e17954.

* Revert "fix: album sorting options (#5127)"

This reverts commit 725f30c494.
2023-11-21 10:07:49 -06:00
725f30c494 fix: album sorting options (#5127)
* fix: album sort options

* fix: don't load assets

* pr feedback

* fix: albumStub

* fix(web): album shared without assets

* fix: tests

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-11-20 13:01:21 -06:00
87f02cc775 chore(server): shared link e2e for add/remove assets (#5052) 2023-11-15 17:50:55 -06:00
4b722517f0 chore: cleanup e2e (#5051) 2023-11-14 20:31:06 -05:00
6127fd4c5c chore(server): improve e2e test speed (#5026)
* feat: improve shared link (41s to 8s)

* feat: improve activity (18s to 8s)

* feat: improve partner (20s to 10s)

* feat: improve server-info (10s to 6s)

* feat: improve system-config

* fix: e2e

* chore: linting
2023-11-14 20:08:22 -05:00
753dab8b3c feat(server): asset search endpoint (#4931)
* feat(server): GET /assets endpoint

* chore: open api

* chore: use dumb name

* feat: search by make, model, lens, city, state, country

* chore: open api

* chore: pagination validation and tests

* chore: pr feedback
2023-11-14 22:47:15 +00:00
d25a245049 feat(web,server): user avatar color (#4779) 2023-11-14 03:10:35 +00:00
7fca0d8da5 fix: replace first and last name with single field (#4915) 2023-11-11 19:03:32 -06:00
35767591d2 feat(web): show partners assets on the main timeline (#4933) 2023-11-11 21:06:19 +00:00
92ec1ce77f fix(server,web): correctly show album level like (#4916)
* fix: like in global activity

* refactor: rename isGlobal to ReactionLevel.Album

* chore: open api

* chore: e2e test for album vs comment duplicate like checking

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-11-09 21:32:31 -05:00
986bbfa831 feat(server): don't re-run face recognition on assets without any faces (#4854)
* Add AssetJobStatus

* fentity

* Add jobStatus field to AssetEntity

* Fix the migration doc paths

* Filter on facesRecognizedAt

* Set facesRecognizedAt field

* Test for facesRecognizedAt

* Done testing

* Adjust FK properties

* Add tests for WithoutProperty.FACES

* chore: non-nullable

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-11-09 20:55:00 -05:00
75c065c83a chore(server): e2e test (#4941) 2023-11-09 20:23:03 -05:00
a147dee4b6 feat: Maplibre (#4294)
* maplibre on web, custom styles from server

Actually use new vector tile server, custom style.json

support multiple style files, light/dark mode

cleanup, use new map everywhere

send file directly instead of loading first

better light/dark mode switching

remove leaflet

fix mapstyles dto, first draft of map settings

delete and add styles

fix delete default styles

fix tests

only allow one light and one dark style url

revert config core changes

fix server config store

fix tests

move axios fetches to repo

fix package-lock

fix tests

* open api

* add assets to docker container

* web: use mapSettings color for style

* style: add unique ids to map styles

* mobile: use style json for vector / raster

* do not use svelte-material-icons

* add click events to markers, simplify asset detail map

* improve map performance by using asset thumbnails for markers instead of original file

* Remove custom attribution

(by request)

* mobile: update map attribution

* style: map dark mode

* style: map light mode

* zoom level for state

* styling

* overflow gradient

* Limit maxZoom to 14

* mobile: listen for mapStyle changes in MapThumbnail

* mobile: update concurrency

---------

Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: bo0tzz <git@bo0tzz.me>
Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-11-09 10:10:56 -06:00