1
0
mirror of https://github.com/immich-app/immich.git synced 2024-12-28 11:15:54 +02:00
Commit Graph

43 Commits

Author SHA1 Message Date
Jason Rasmussen
065fb166c2
refactor(server): bull jobs (#2569)
* refactor(server): bull jobs

* chore: add comment

* chore: metadata test coverage

* fix typo

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-05-26 07:52:52 -05:00
Alex Phillips
7c1dae918d
feat(server): xmp sidecar metadata (#2466)
* initial commit for XMP sidecar support

* Added support for 'missing' metadata files to include those without sidecar files, now detects sidecar files in the filesystem for media already ingested but the sidecar was created afterwards

* didn't mean to commit default log level during testing

* new sidecar logic for video metadata as well

* Added xml mimetype for sidecars only

* don't need capture group for this regex

* wrong default value reverted

* simplified the move here - keep it in the same try catch since the outcome is to move the media back anyway

* simplified setter logic

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* simplified logic per suggestions

* sidecar is now its own queue with a discover and sync, updated UI for the new job queueing

* queue a sidecar job for every asset based on discovery or sync, though the logic is almost identical aside from linking the sidecar

* now queue sidecar jobs for each assset, though logic is mostly the same between discovery and sync

* simplified logic of filename extraction and asset instantiation

* not sure how that got deleted..

* updated code per suggestions and comments in the PR

* stat was not being used, removed the variable set

* better type checking, using in-scope variables for exif getter instead of passing in every time

* removed commented out test

* ran and resolved all lints, formats, checks, and tests

* resolved suggested change in PR

* made getExifProperty more dynamic with multiple possible args for fallbacks, fixed typo, used generic in function  for better type checking

* better error handling and moving files back to positions on move or save failure

* regenerated api

* format fixes

* Added XMP documentation

* documentation typo

* Merged in main

* missed merge conflict

* more changes due to a merge

* Resolving conflicts

* added icon for sidecar jobs

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-05-24 20:59:30 -05:00
Jonathan Jogenfors
1b54c4f8e7
feat(server): Add support for client-side hashing (#2072)
* Modify controller DTOs

* Can check duplicates on server side

* Remove deviceassetid and deviceid

* Remove device ids from file uploader

* Add db migration for removed device ids

* Don't sanitize checksum

* Convert asset checksum to string

* Make checksum not optional for asset

* Use enums when rejecting duplicates

* Cleanup

* Return of the device id, but optional

* Don't use deviceId for upload folder

* Use checksum in thumb path

* Only use asset id in thumb path

* Openapi generation

* Put deviceAssetId back in asset response dto

* Add missing checksum in test fixture

* Add another missing checksum in test fixture

* Cleanup asset repository

* Add back previous /exists endpoint

* Require checksum to not be null

* Correctly set deviceId in db

* Remove index

* Fix compilation errors

* Make device id nullabel in asset response dto

* Reduce PR scope

* Revert asset service

* Reorder imports

* Reorder imports

* Reduce PR scope

* Reduce PR scope

* Reduce PR scope

* Reduce PR scope

* Reduce PR scope

* Update openapi

* Reduce PR scope

* refactor: asset bulk upload check

* chore: regenreate open-api

* chore: fix tests

* chore: tests

* update migrations and regenerate api

* Feat: use checksum in web file uploader

* Change to wasm-crypto

* Use crypto api for checksumming in web uploader

* Minor cleanup of file upload

* feat(web): pause and resume jobs

* Make device asset id not nullable again

* Cleanup

* Device id not nullable in response dto

* Update API specs

* Bump api specs

* Remove old TODO comment

* Remove NOT NULL constraint on checksum index

* Fix requested pubspec changes

* Remove unneeded import

* Update server/apps/immich/src/api-v1/asset/asset.service.ts

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>

* Update server/apps/immich/src/api-v1/asset/asset-repository.ts

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>

* Remove unneeded check

* Update server/apps/immich/src/api-v1/asset/asset-repository.ts

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>

* Remove hashing in the web uploader

* Cleanup file uploader

* Remove varchar from asset entity fields

* Return 200 from bulk upload check

* Put device asset id back into asset repository

* Merge migrations

* Revert pubspec lock

* Update openapi specs

* Merge upstream changes

* Fix failing asset service tests

* Fix formatting issue

* Cleanup migrations

* Remove newline from pubspec

* Revert newline

* Checkout main version

* Revert again

* Only return AssetCheck

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>
2023-05-24 16:08:21 -05:00
Mert
e9722710ac
feat(server): transcode bitrate and thread settings (#2488)
* support for two-pass transcoding

* added max bitrate and thread to transcode api

* admin page setting desc+bitrate and thread options

* Update web/src/lib/components/admin-page/settings/setting-input-field.svelte

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>

* Update web/src/lib/components/admin-page/settings/setting-input-field.svelte

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>

* two-pass slider, `crf` and `threads` as numbers

* updated and added transcode tests

* refactored `getFfmpegOptions`

* default `threads`, `maxBitrate` now 0, more tests

* vp9 constant quality mode

* fixed nullable `crf` and `threads`

* fixed two-pass slider, added apiproperty

* optional `desc` for `SettingSelect`

* disable two-pass if settings are incompatible

* fixed test

* transcode interface

---------

Co-authored-by: Michel Heusschen <59014050+michelheusschen@users.noreply.github.com>
2023-05-22 13:07:43 -05:00
Jason Rasmussen
74353193f8
feat(web,server): user storage label (#2418)
* feat: user storage label

* chore: open api

* fix: checks

* fix: api update validation and tests

* feat: default admin storage label

* fix: linting

* fix: user create/update dto

* fix: delete library with custom label
2023-05-21 23:18:10 -04:00
Michel Heusschen
a7b9adc692
feat(web+server): map improvements (#2498)
* feat(web+server): map improvements

* add number format double to fix mobile
2023-05-21 01:26:06 -05:00
Jason Rasmussen
93863b0629
feat: facial recognition (#2180) 2023-05-17 12:07:17 -05:00
Sergey Kondrikov
7f2fa23179
feat (server, web): Share with partner (#2388)
* feat(server, web): implement share with partner

* chore: regenerate api

* chore: regenerate api

* Pass userId to getAssetCountByTimeBucket and getAssetByTimeBucket

* chore: regenerate api

* Use AssetGrid to view partner's assets

* Remove disableNavBarActions flag

* Check access to buckets

* Apply suggestions from code review

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* Remove exception rethrowing

* Simplify partner access check

* Create new PartnerController

* chore api:generate

* Use partnerApi

* Remove id from PartnerResponseDto

* Refactor PartnerEntity

* Rename args

* Remove duplicate code in getAll

* Create composite primary keys for partners table

* Move asset access check into PartnerCore

* Remove redundant getUserAssets call

* Remove unused getUserAssets method

* chore: regenerate api

* Simplify getAll

* Replace ?? with ||

* Simplify PartnerRepository.create

* Introduce PartnerIds interface

* Replace two database migrations with one

* Simplify getAll

* Change PartnerResponseDto to include UserResponseDto

* Move partner sharing endpoints to PartnerController

* Rename ShareController to SharedLinkController

* chore: regenerate api after rebase

* refactor: shared link remove return type

* refactor: return user response dto

* chore: regenerate open api

* refactor: partner getAll

* refactor: partner settings event typing

* chore: remove unused code

* refactor: add partners modal trigger

* refactor: update url for viewing partner photos

* feat: update partner sharing title

* refactor: rename service method names

* refactor: http exception logic to service, PartnerIds interface

* chore: regenerate open api

* test: coverage for domain code

* fix: addPartner => createPartner

* fix: missed rename

* refactor: more code cleanup

* chore: alphabetize settings order

* feat: stop sharing confirmation modal

* Enhance contrast of the email in dark mode

* Replace button with CircleIconButton

* Fix linter warning

* Fix date types for PartnerEntity

* Fix PartnerEntity creation

* Reset assetStore state

* Change layout of the partner's assets page

* Add bulk download action for partner's assets

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-05-15 12:30:53 -05:00
Jason Rasmussen
b8313abfa8
feat(web,server): manage authorized devices (#2329)
* feat: manage authorized devices

* chore: open api

* get header from mobile app

* write header from mobile app

* styling

* fix unit test

* feat: use relative time

* feat: update access time

* fix: tests

* chore: confirm wording

* chore: bump test coverage thresholds

* feat: add some icons

* chore: icon tweaks

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-04-25 21:19:23 -05:00
Alex
a9859bc029
feat: Add description (#2237)
* Added dto, logic to insert description and web implementation

* create text field and update on remote database

* Update description and save changes

* styling

* fix web test

* fix server test

* preserve description on metadata extraction job run

* handle exif info is null situation

* pr feedback

* format openapi spec

* update createAssetDto

* refactor logic to service

* move files

* only owner can update description

* Render description correctly in shared album

* Render description correctly in shared link

* disable description edit for not owner of asset on mobile

* localization and clean up

* fix test

* Uses providers for description text (#2244)

* uses providers for description text

* comments

* fixes initial data setting

* fixes notifier

---------

Co-authored-by: martyfuhry <martyfuhry@gmail.com>
2023-04-13 10:22:06 -05:00
Sergey Kondrikov
d314805caf
feat (server, web): Implement Archive (#2225)
* feat (server, web): add archive

* chore: generate api

* feat (web): add empty placeholder for archive page

* chore: remove title on favorites page

Duplicates sidebar selection. Two pages (Archive and Favorites)
are consistent now

* refactor (web): create EmptyPlaceholder component for empty pages

* fixed menu close button not close:

* fix (web): remove not necessary store call

* test (web): simplify asset tests code

* test (web): simplify asset tests code

* chore (server): remove isArchived while uploading

* chore (server): remove isArchived from typesense schema

* chore: generate api

* fix (web): delete asset from archive page

* chore: change archive asset count endpoint

old endpoint: /asset/archived-count-by-user-id
new endpoint: /asset/stat/archive

* chore: generate api

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-04-12 10:37:52 -05:00
Alex
a1a62b00a0
feat(server): add originalFileName to asset table (#2231) 2023-04-11 05:23:39 -05:00
Zack Pollard
a5a6bebf0b
feat(all): transcoding improvements (#2171)
* test: rename some fixtures and add text for vertical video conversion

* feat: transcode video asset when audio or container don't match target

* chore: add niceness to the ffmpeg command to allow other processes to be prioritised

* chore: change video conversion queue to one concurrency

* feat: add transcode disabled preset to completely turn off transcoding

* linter

* Change log level and remove unused await

* opps forgot to save

* better logging

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-04-05 22:32:59 -05:00
Jason Rasmussen
48393c215b
refactor(server): video transcode processor (#2163)
* refactor(server): video transcode processor

* refactor: rename shouldRotate to isVideoVertical, remove unnecessary await

* refactor: rename getOptions to getFfmpegOptions to be clearer in that context

* fix: optimal preset converting vertical videos already smaller than target resolution

---------

Co-authored-by: Zack Pollard <zackpollard@ymail.com>
2023-04-04 10:48:02 -04:00
Zack Pollard
808d6423be
feat(all): ffmpeg quality options improvements (#2161)
* feat: change target scaling to resolution in ffmpeg config

* feat(microservices): scale vertical video correctly, only scale if video is larger than target
2023-04-03 20:42:53 -05:00
AndreAle94
94b2ea9b5f
Add timezone to exif entity (#1894)
* Add timezone to exif entity

* Refactor logging

---------

Co-authored-by: Andrea Alemani <andrea.alemani94@gmail.com>
2023-04-02 14:11:24 -05:00
Jason Rasmussen
34d300d1da
refactor(server): flatten infra folders (#2120)
* refactor: flatten infra folders

* fix: database migrations

* fix: test related import

* fix: github actions workflow

* chore: rename schemas to typesense-schemas
2023-03-30 14:38:55 -05:00
Sergey Kondrikov
2c67090e3c
feat(server): add transcode presets (#2084)
* feat: add transcode presets

* Add migration

* chore: generate api

* refactor: use enum type instead of string for transcode option

* chore: generate api

* refactor: enhance readability of runVideoEncode method

* refactor: reuse SettingSelect for transcoding presets

* refactor: simplify return statement

* chore: regenerate api

* fix: correct label attribute

* Update import

* fix test

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-03-28 14:03:43 -05:00
Michel Heusschen
c74fba483d
feat(server): improve and refactor get all albums (#2048)
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-03-25 21:46:48 -05:00
Jason Rasmussen
2400004f41
feat(server): split generated content into a separate folder (#2047)
* feat: organize media folders

* fix: tests
2023-03-25 09:50:57 -05:00
Jason Rasmussen
6745826f35
refactor(server): media service (#2051)
* refactor(server): media service

* merge main

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-03-23 21:40:46 -05:00
Alex
f56eaae019
feat(server): CLIP search integration (#1939) 2023-03-18 08:44:42 -05:00
bo0tzz
838ea56605
fix(server): Increase authentication cookie max-age (#1971)
This got missed in #1381.
2023-03-08 16:26:49 +00:00
Michel Heusschen
bdf35b6688
feat(server): improve thumbnail relation and updating (#1897)
* feat(server): improve thumbnail relation and updating

* improve query + update tests and migration

* make sure uuids are valid in migration

* fix unit test
2023-03-04 08:16:48 -06:00
Jason Rasmussen
0aaeab124d
feat(server)!: search via typesense (#1778)
* build: add typesense to docker

* feat(server): typesense search

* feat(web): search

* fix(web): show api error response message

* chore: search tests

* chore: regenerate open api

* fix: disable typesense on e2e

* fix: number properties for open api (dart)

* fix: e2e test

* fix: change lat/lng from floats to typesense geopoint

* dev: Add smartInfo relation to findAssetById to be able to query against it

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-03-02 20:47:08 -06:00
Michel Heusschen
368142e79b
feat(web): improved server stats (#1870)
* feat(web): improved server stats

* fix(web): don't log unauthorized errors

* Revert "fix(web): don't log unauthorized errors"

This reverts commit 7fc2987a77.
2023-02-26 13:57:34 -06:00
Jason Rasmussen
6c7679714b
refactor(server): jobs and processors (#1787)
* refactor: jobs and processors

* refactor: storage migration processor

* fix: tests

* fix: code warning

* chore: ignore coverage from infra

* fix: sync move asset logic between job core and asset core

* refactor: move error handling inside of catch

* refactor(server): job core into dedicated service calls

* refactor: smart info

* fix: tests

* chore: smart info tests

* refactor: use asset repository

* refactor: thumbnail processor

* chore: coverage reqs
2023-02-25 08:12:03 -06:00
Zack Pollard
d1ea6a897e
chore(server): typeorm definitions fix part 3 (#1796)
* chore(server): tidy up exif typeorm entity definition

* chore(server): tidy up shared link typeorm entity definition

* chore(server): tidy up smart info typeorm entity definition

* chore(server): tidy up tag typeorm entity definition

* ci: add job that checks typeorm migrations are correct and up-to-date
2023-02-20 01:50:27 +00:00
Zack Pollard
5ad4e5b614
infra(server)!: fix typeorm asset entity relations (#1782)
* fix: add correct relations to asset typeorm entity

* fix: add missing createdAt column to asset entity

* ci: run check to make sure generated API is up-to-date

* ci: cancel workflows that aren't for the latest commit in a branch

* chore: add fvm config for flutter
2023-02-19 16:44:53 +00:00
Zack Pollard
000d0a08f4
infra(server): fix Album TypeORM relations and change ids to uuids (#1582)
* infra: make api-key primary key column a UUID

* infra: move ManyToMany relations in album entity, make ownerId ManyToOne

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-02-18 14:58:55 -06:00
Jason Rasmussen
36197cca98
feat(server): auto-link live photos (#1761)
* feat(server): auto-link live photos

* fix: video extraction and linking
2023-02-16 01:41:51 -06:00
Alex
3cc4af5947
chore(server) Add user FK to album entity (#1569) 2023-02-06 20:47:06 -06:00
Alex
29bb1f7ef2
feat(server): add updatedAt to Asset, Album and User (#1566)
* feat: add updatedAt info to DTO and generate api

* chore: remove unsued file

* chore: Add update statement to add/remove asset/user to album

* fix: test
2023-02-06 10:24:58 -06:00
Alex
7dbddba757
chore(server): remove token when logged out (#1560)
* chore(mobile): invoke logout() on mobile app

* feat: add mechanism to delete token from logging out endpoint

* fix: set state after login sequence success

* fix: not removing token when logging out from OAuth

* fix: prettier

* refactor: using accessTokenId to delete

* chore: pr comments

* fix: test

* fix: test threshold
2023-02-05 23:31:16 -06:00
Jason Rasmussen
d2a9363fc5
refactor(server): auth guard (#1472)
* refactor: auth guard

* chore: move auth guard to middleware

* chore: tests

* chore: remove unused code

* fix: migration to uuid without dataloss

* chore: e2e tests

* chore: removed unused guards
2023-01-31 12:11:49 -06:00
Skyler Mäntysaari
fb408d7aa3
chore(server): cookie changes to SameSite=Lax (#1467)
* fix(server/cookie): cookie should have SameSite=Lax.

* Forgot to update tests.
2023-01-28 16:33:03 -06:00
Zack Pollard
3f2513a717
feat(server): move authentication to tokens stored in the database (#1381)
* chore: add typeorm commands to npm and set default database config values

* feat: move to server side authentication tokens

* fix: websocket should emit error and disconnect on error thrown by the server

* refactor: rename cookie-auth-strategy to user-auth-strategy

* feat: user tokens and API keys now use SHA256 hash for performance improvements

* test: album e2e test remove unneeded module import

* infra: truncate api key table as old keys will no longer work with new hash algorithm

* fix(server): e2e tests (#1435)

* fix: root module paths

* chore: linting

* chore: rename user-auth to strategy.ts and make validate return AuthUserDto

* fix: we should always send HttpOnly for our auth cookies

* chore: remove now unused crypto functions and jwt dependencies

* fix: return the extra fields for AuthUserDto in auth service validate

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-01-27 14:50:07 -06:00
Alex
8b73c2bf8a
fix(server): Handle exposure time correctly (#1432) 2023-01-26 13:14:05 -06:00
Jason Rasmussen
8f304b8157
refactor(server): shared links (#1385)
* refactor(server): shared links

* chore: tests

* fix: bugs and tests

* fix: missed one expired at

* fix: standardize file upload checks

* test: lower flutter version

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-01-25 10:35:28 -06:00
Jason Rasmussen
eade36ee82
refactor(server): auth service (#1383)
* refactor: auth

* chore: tests

* Remove await on non-async method

* refactor: constants

* chore: remove extra async

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-01-23 22:13:42 -06:00
Zack Pollard
4e0fe27de3
feat(server): transcoding improvements (#1370)
* feat: support isEdited flag for SettingSwitch

* feat: add transcodeAll ffmpeg settings for extra transcoding control

* refactor: tidy up and rename current video transcoding code + transcode everything

* feat: better video transcoding with ffprobe

analyses video files to see if they are already in the desired format
allows admin to choose to transcode all videos regardless of the current format

* fix: always serve encoded video if it exists

* feat: change video codec option to a select box, limit options

removed previous video codec config option as it's incompatible with new options
removed mapping for encoder to codec as we now store the codec in the config

* feat: add video conversion job for transcoding previously missed videos

* chore: fix spelling of job messages to pluralise assets

* chore: fix prettier/eslint warnings

* feat: force switch targetAudioCodec default to aac to avoid iOS incompatibility

* chore: lint issues after rebase
2023-01-21 20:09:02 -06:00
Jason Rasmussen
c0a6b3d5a3
refactor(server): system config (#1353)
* refactor(server): system config

* fix: jest circular import

* chore: ignore migrations in coverage report

* chore: tests

* chore: tests

* chore: todo note

* chore: remove vite config backup

* chore: fix redis hostname
2023-01-21 10:11:55 -06:00
Jason Rasmussen
92972ac776
refactor(server): api keys (#1339)
* refactor: api keys

* refactor: test module

* chore: tests

* chore: fix provider

* refactor: test mock repos
2023-01-18 08:40:15 -06:00