pocketbase/CHANGELOG.md

v0.10.2

• Fixed nested multiple expands with shared path (#586). A new helper method models.Record.MergeExpand(map[string]any) was also added to simplify the expand handling and unit testing.

v0.10.1

• Fixed nested transactions deadlock when authenticating with OAuth2 (#1291).

v0.10.0

• Added /api/health endpoint (thanks @MarvinJWendt).

• Added support for SMTP LOGIN auth for Microsoft/Outlook and other providers that don't support the PLAIN auth method (#1217).

• Reduced memory consumption (you can expect ~20% less allocated memory).

• Added support for split (concurrent and nonconcurrent) DB connections pool increasing even further the concurrent throughput without blocking reads on heavy write load.

• Improved record references delete performance.

• Removed the unnecessary parenthesis in the generated filter SQL query, reducing the "parse stack overflow" errors.

• Fixed ~ expressions backslash literal escaping (#1231).

• Refactored the core.app.Bootstrap() to be called before starting the cobra commands (#1267).

• ! Changed pocketbase.NewWithConfig(config Config) to pocketbase.NewWithConfig(config *Config) and added 4 new config settings:

  DataMaxOpenConns int // default to core.DefaultDataMaxOpenConns
  DataMaxIdleConns int // default to core.DefaultDataMaxIdleConns
  LogsMaxOpenConns int // default to core.DefaultLogsMaxOpenConns
  LogsMaxIdleConns int // default to core.DefaultLogsMaxIdleConns
  

• Added new helper method core.App.IsBootstrapped() to check the current app bootstrap state.

• ! Changed core.NewBaseApp(dir, encryptionEnv, isDebug) to NewBaseApp(config *BaseAppConfig).

• ! Removed rest.UploadedFile struct (see below filesystem.File).

• Added generic file resource struct that allows loading and uploading file content from different sources (at the moment multipart/form-data requests and from the local filesystem).

  filesystem.File{}
  filesystem.NewFileFromPath(path)
  filesystem.NewFileFromMultipart(multipartHeader)
  filesystem/System.UploadFile(file)
  

• Refactored forms.RecordUpsert to allow more easily loading and removing files programmatically.

  forms.RecordUpsert.AddFiles(key, filesystem.File...) // add new filesystem.File to the form for upload
  forms.RecordUpsert.RemoveFiles(key, filenames...)     // marks the filenames for deletion
  

• Trigger the password validators if any of the others password change fields is set.

v0.9.2

• Fixed field column name conflict on record deletion (#1220).

v0.9.1

• Moved the record file upload and delete out of the db transaction to minimize the locking times.

• Added Dao query semaphore and base fail/retry handling to improve the concurrent writes throughput (#1187).

• Fixed records cascade deletion when there are "A<->B" relation references.

• Replaced c.QueryString() with c.QueryParams().Encode() to allow loading middleware modified query parameters in the default crud actions (#1210).

• Fixed the datetime field not triggering the onChange event on manual field edit and added a "Clear" button (#1219).

• Updated the GitHub goreleaser action to use go 1.19.4 since it comes with some security fixes.

v0.9.0

• Fixed concurrent multi-relation cascade update/delete (#1138).

• Added the raw OAuth2 user data (meta.rawUser) and OAuth2 access token (meta.accessToken) to the auth response (#654).

• BaseModel.UnmarkAsNew() method was renamed to BaseModel.MarkAsNotNew(). Additionally, to simplify the insert model queries with custom IDs, it is no longer required to call MarkAsNew() for manually initialized models with set ID since now this is the default state. When the model is populated with values from the database (eg. after row Scan) it will be marked automatically as "not new".

• Added Record.OriginalCopy() method that returns a new Record copy populated with the initially loaded record data (useful if you want to compare old and new field values).

• Added new event hooks:

  app.OnBeforeBootstrap()
  app.OnAfterBootstrap()
  app.OnBeforeApiError()
  app.OnAfterApiError()
  app.OnRealtimeDisconnectRequest()
  app.OnRealtimeBeforeMessageSend()
  app.OnRealtimeAfterMessageSend()
  app.OnRecordBeforeRequestPasswordResetRequest()
  app.OnRecordAfterRequestPasswordResetRequest()
  app.OnRecordBeforeConfirmPasswordResetRequest()
  app.OnRecordAfterConfirmPasswordResetRequest()
  app.OnRecordBeforeRequestVerificationRequest()
  app.OnRecordAfterRequestVerificationRequest()
  app.OnRecordBeforeConfirmVerificationRequest()
  app.OnRecordAfterConfirmVerificationRequest()
  app.OnRecordBeforeRequestEmailChangeRequest()
  app.OnRecordAfterRequestEmailChangeRequest()
  app.OnRecordBeforeConfirmEmailChangeRequest()
  app.OnRecordAfterConfirmEmailChangeRequest()
  

• The original uploaded file name is now stored as metadata under the original_filename key. It could be accessed via:

  fs, _ := app.NewFilesystem()
  defer fs.Close()
  
  attrs, _ := fs.Attributes(fikeKey)
  attrs.Metadata["original_name"]
  

• Added support for Partial/Range file requests (#1125). This is a minor breaking change if you are using filesystem.Serve (eg. as part of a custom OnFileDownloadRequest hook):

  // old
  filesystem.Serve(res, e.ServedPath, e.ServedName)
  
  // new
  filesystem.Serve(res, req, e.ServedPath, e.ServedName)
  

• Refactored the migrate command to support external JavaScript migration files using an embedded JS interpreter (goja). This allow writing custom migration scripts such as programmatically creating collections, initializing default settings, running data imports, etc., with a JavaScript API very similar to the Go one (more documentation will be available soon).

  The migrate command is available by default for the prebuilt executable, but if you use PocketBase as framework you need register it manually:

  migrationsDir := "" // default to "pb_migrations" (for js) and "migrations" (for go)
  
  // load js files if you want to allow loading external JavaScript migrations
  jsvm.MustRegisterMigrations(app, &jsvm.MigrationsOptions{
    Dir: migrationsDir,
  })
  
  // register the `migrate` command
  migratecmd.MustRegister(app, app.RootCmd, &migratecmd.Options{
    TemplateLang: migratecmd.TemplateLangJS, // or migratecmd.TemplateLangGo (default)
    Dir:          migrationsDir,
    Automigrate:  true,
  })
  

  The refactoring also comes with automigrations support.

  If Automigrate is enabled (true by default for the prebuilt executable; can be disabled with --automigrate=0), PocketBase will generate seamlessly in the background JS (or Go) migration file with your collection changes. The directory with the JS migrations can be committed to your git repo. All migrations (Go and JS) are automatically executed on server start. Also note that the auto generated migrations are granural (in contrast to the migrate collections snapshot command) and allow multiple developers to do changes on the collections independently (even editing the same collection) miniziming the eventual merge conflicts. Here is a sample JS migration file that will be generated if you for example edit a single collection name:

  // pb_migrations/1669663597_updated_posts_old.js
  migrate((db) => {
    // up
    const dao = new Dao(db)
    const collection = dao.findCollectionByNameOrId("lngf8rb3dqu86r3")
    collection.name = "posts_new"
    return dao.saveCollection(collection)
  }, (db) => {
    // down
    const dao = new Dao(db)
    const collection = dao.findCollectionByNameOrId("lngf8rb3dqu86r3")
    collection.name = "posts_old"
    return dao.saveCollection(collection)
  })
  

• Added new Dao helpers to make it easier fetching and updating the app settings from a migration:

  dao.FindSettings([optEncryptionKey])
  dao.SaveSettings(newSettings, [optEncryptionKey])
  

• Moved core.Settings to models/settings.Settings:

  core.Settings{}           -> settings.Settings{}
  core.NewSettings()        -> settings.New()
  core.MetaConfig{}         -> settings.MetaConfig{}
  core.LogsConfig{}         -> settings.LogsConfig{}
  core.SmtpConfig{}         -> settings.SmtpConfig{}
  core.S3Config{}           -> settings.S3Config{}
  core.TokenConfig{}        -> settings.TokenConfig{}
  core.AuthProviderConfig{} -> settings.AuthProviderConfig{}
  

• Changed the mailer.Mailer interface (minor breaking if you are sending custom emails):

  // Old:
  app.NewMailClient().Send(from, to, subject, html, attachments?)
  
  // New:
  app.NewMailClient().Send(&mailer.Message{
    From: from,
    To: to,
    Subject: subject,
    HTML: html,
    Attachments: attachments,
    // new configurable fields
    Bcc: []string{"bcc1@example.com", "bcc2@example.com"},
    Cc: []string{"cc1@example.com", "cc2@example.com"},
    Headers: map[string]string{"Custom-Header": "test"},
    Text: "custom plain text version",
  })
  

  The new *mailer.Message struct is also now a member of the MailerRecordEvent and MailerAdminEvent events.

• Other minor UI fixes and improvements

v0.8.0

⚠️ This release contains breaking changes and requires some manual migration steps!

The biggest change is the merge of the User models and the profiles collection per #376. There is no longer user type field and the users are just an "auth" collection (we now support collection types, currently only "base" and "auth"). This should simplify the users management and at the same time allow us to have unlimited multiple "auth" collections each with their own custom fields and authentication options (eg. staff, client, etc.).

In addition to the Users and profiles merge, this release comes with several other improvements:

• Added indirect expand support #312.

• The json field type now supports filtering and sorting #423.

• The relation field now allows unlimited maxSelect (aka. without upper limit).

• Added support for combined email/username + password authentication (see below authWithPassword()).

• Added support for full "manager-subordinate" users management, including a special API rule to allow directly changing system fields like email, password, etc. without requiring oldPassword or other user verification.

• Enabled OAuth2 account linking on authorized request from the same auth collection (this is useful for example if the OAuth2 provider doesn't return an email and you want to associate it with the current logged in user).

• Added option to toggle the record columns visibility from the table listing.

• Added support for collection schema fields reordering.

• Added several new OAuth2 providers (Microsoft Azure AD, Spotify, Twitch, Kakao).

• Improved memory usage on large file uploads #835.

• More detailed API preview docs and site documentation (the repo is located at https://github.com/pocketbase/site).

• Other minor performance improvements (mostly related to the search apis).

Migrate from v0.7.x

• Data
• SDKs
• API
• Internals

Data

The merge of users and profiles comes with several required db changes. The easiest way to apply them is to use the new temporary upgrade command:

# make sure to have a copy of your pb_data in case something fails
cp -r ./pb_data ./pb_data_backup

# run the upgrade command
./pocketbase08 upgrade

# start the application as usual
./pocketbase08 serve

The upgrade command:

• Creates a new users collection with merged fields from the _users table and the profiles collection. The new user records will have the ids from the profiles collection.
• Changes all user type fields to relation and update the references to point to the new user ids.
• Renames all @collection.profiles.*, @request.user.* and @request.user.profile.* filters to @collection.users.* and @request.auth.*.
• Appends 2 to all schema field names and api filter rules that conflicts with the new system reserved ones:

  collectionId   => collectionId2
  collectionName => collectionName2
  expand         => expand2
  
  // only for the "profiles" collection fields:
  username               => username2
  email                  => email2
  emailVisibility        => emailVisibility2
  verified               => verified2
  tokenKey               => tokenKey2
  passwordHash           => passwordHash2
  lastResetSentAt        => lastResetSentAt2
  lastVerificationSentAt => lastVerificationSentAt2
  

SDKs

Please check the individual SDK package changelog and apply the necessary changes in your code:

• JavaScript SDK changelog

  npm install pocketbase@latest --save
  

• Dart SDK changelog

  dart pub add pocketbase:^0.5.0
  # or with Flutter:
  flutter pub add pocketbase:^0.5.0
  

API

You don't have to read this if you are using an official SDK.

• The authorization schema is no longer necessary. Now it is auto detected from the JWT token payload:

  Old	New
  Authorization: Admin TOKEN	Authorization: TOKEN
  Authorization: User TOKEN	Authorization: TOKEN

• All datetime stings are now returned in ISO8601 format - with Z suffix and space as separator between the date and time part:

  Old	New
  2022-01-02 03:04:05.678	2022-01-02 03:04:05.678Z

• Removed the @ prefix from the system record fields for easier json parsing:

  Old	New
  @collectionId	collectionId
  @collectionName	collectionName
  @expand	expand

• All users api handlers are moved under /api/collections/:collection/:

  Old	New
  GET /api/users/auth-methods	GET /api/collections/:collection/auth-methods
  POST /api/users/refresh	POST /api/collections/:collection/auth-refresh
  POST /api/users/auth-via-oauth2	POST /api/collections/:collection/auth-with-oauth2 You can now also pass optional createData object on OAuth2 sign-up. Also please note that now required user/profile fields are properly validated when creating new auth model on OAuth2 sign-up.
  POST /api/users/auth-via-email	POST /api/collections/:collection/auth-with-password Handles username/email + password authentication. {"identity": "usernameOrEmail", "password": "123456"}
  POST /api/users/request-password-reset	POST /api/collections/:collection/request-password-reset
  POST /api/users/confirm-password-reset	POST /api/collections/:collection/confirm-password-reset
  POST /api/users/request-verification	POST /api/collections/:collection/request-verification
  POST /api/users/confirm-verification	POST /api/collections/:collection/confirm-verification
  POST /api/users/request-email-change	POST /api/collections/:collection/request-email-change
  POST /api/users/confirm-email-change	POST /api/collections/:collection/confirm-email-change
  GET /api/users	GET /api/collections/:collection/records
  GET /api/users/:id	GET /api/collections/:collection/records/:id
  POST /api/users	POST /api/collections/:collection/records
  PATCH /api/users/:id	PATCH /api/collections/:collection/records/:id
  DELETE /api/users/:id	DELETE /api/collections/:collection/records/:id
  GET /api/users/:id/external-auths	GET /api/collections/:collection/records/:id/external-auths
  DELETE /api/users/:id/external-auths/:provider	DELETE /api/collections/:collection/records/:id/external-auths/:provider

  In relation to the above changes, the user property in the auth response is renamed to record.

• The admins api was also updated for consistency with the users api changes:

  Old	New
  POST /api/admins/refresh	POST /api/admins/auth-refresh
  POST /api/admins/auth-via-email	POST /api/admins/auth-with-password {"identity": "test@example.com", "password": "123456"} (notice that the email body field was renamed to identity)

• To prevent confusion with the auth method responses, the following endpoints now returns 204 with empty body (previously 200 with token and auth model):

  POST /api/admins/confirm-password-reset
  POST /api/collections/:collection/confirm-password-reset
  POST /api/collections/:collection/confirm-verification
  POST /api/collections/:collection/confirm-email-change
  

• Renamed the "user" related settings fields returned by GET /api/settings:

  Old	New
  userAuthToken	recordAuthToken
  userPasswordResetToken	recordPasswordResetToken
  userEmailChangeToken	recordEmailChangeToken
  userVerificationToken	recordVerificationToken

Internals

You don't have to read this if you are not using PocketBase as framework.

• Removed forms.New*WithConfig() factories to minimize ambiguities. If you need to pass a transaction Dao you can use the new SetDao(dao) method available to the form instances.

• forms.RecordUpsert.LoadData(data map[string]any) now can bulk load external data from a map. To load data from a request instance, you could use forms.RecordUpsert.LoadRequest(r, optKeysPrefix = "").

• schema.RelationOptions.MaxSelect has new type *int (you can use the new types.Pointer(123) helper to assign pointer values).

• Renamed the constant apis.ContextUserKey ("user") to apis.ContextAuthRecordKey ("authRecord").

• Replaced user related middlewares with their auth record alternative:

  Old	New
  apis.RequireUserAuth()	apis.RequireRecordAuth(optCollectionNames ...string)
  apis.RequireAdminOrUserAuth()	apis.RequireAdminOrRecordAuth(optCollectionNames ...string)
  N/A	RequireSameContextRecordAuth() (requires the auth record to be from the same context collection)

• The following record Dao helpers now uses the collection id or name instead of *models.Collection instance to reduce the verbosity when fetching records:

  Old	New
  dao.FindRecordById(collection, ...)	dao.FindRecordById(collectionNameOrId, ...)
  dao.FindRecordsByIds(collection, ...)	dao.FindRecordsByIds(collectionNameOrId, ...)
  dao.FindRecordsByExpr(collection, ...)	dao.FindRecordsByExpr(collectionNameOrId, ...)
  dao.FindFirstRecordByData(collection, ...)	dao.FindFirstRecordByData(collectionNameOrId, ...)
  dao.IsRecordValueUnique(collection, ...)	dao.IsRecordValueUnique(collectionNameOrId, ...)

• Replaced all User related Dao helpers with Record equivalents:

  Old	New
  dao.UserQuery()	dao.RecordQuery(collection)
  dao.FindUserById(id)	dao.FindRecordById(collectionNameOrId, id)
  dao.FindUserByToken(token, baseKey)	dao.FindAuthRecordByToken(token, baseKey)
  dao.FindUserByEmail(email)	dao.FindAuthRecordByEmail(collectionNameOrId, email)
  N/A	dao.FindAuthRecordByUsername(collectionNameOrId, username)

• Moved the formatted ApiError struct and factories to the github.com/pocketbase/pocketbase/apis subpackage:

  Old	New
  Import path
  github.com/pocketbase/pocketbase/tools/rest	github.com/pocketbase/pocketbase/apis
  Fields
  rest.ApiError{}	apis.ApiError{}
  rest.NewNotFoundError()	apis.NewNotFoundError()
  rest.NewBadRequestError()	apis.NewBadRequestError()
  rest.NewForbiddenError()	apis.NewForbiddenError()
  rest.NewUnauthorizedError()	apis.NewUnauthorizedError()
  rest.NewApiError()	apis.NewApiError()

• Renamed models.Record helper getters:

  Old	New
  SetDataValue	Set
  GetDataValue	Get
  GetBoolDataValue	GetBool
  GetStringDataValue	GetString
  GetIntDataValue	GetInt
  GetFloatDataValue	GetFloat
  GetTimeDataValue	GetTime
  GetDateTimeDataValue	GetDateTime
  GetStringSliceDataValue	GetStringSlice

• Added new auth collection models.Record helpers:

  func (m *Record) Username() string
  func (m *Record) SetUsername(username string) error
  func (m *Record) Email() string
  func (m *Record) SetEmail(email string) error
  func (m *Record) EmailVisibility() bool
  func (m *Record) SetEmailVisibility(visible bool) error
  func (m *Record) IgnoreEmailVisibility(state bool)
  func (m *Record) Verified() bool
  func (m *Record) SetVerified(verified bool) error
  func (m *Record) TokenKey() string
  func (m *Record) SetTokenKey(key string) error
  func (m *Record) RefreshTokenKey() error
  func (m *Record) LastResetSentAt() types.DateTime
  func (m *Record) SetLastResetSentAt(dateTime types.DateTime) error
  func (m *Record) LastVerificationSentAt() types.DateTime
  func (m *Record) SetLastVerificationSentAt(dateTime types.DateTime) error
  func (m *Record) ValidatePassword(password string) bool
  func (m *Record) SetPassword(password string) error
  

• Added option to return serialized custom models.Record fields data:

  func (m *Record) UnknownData() map[string]any
  func (m *Record) WithUnkownData(state bool)
  

• Deleted model.User. Now the user data is stored as an auth models.Record.

  Old	New
  User.Email	Record.Email()
  User.TokenKey	Record.TokenKey()
  User.Verified	Record.Verified()
  User.SetPassword()	Record.SetPassword()
  User.RefreshTokenKey()	Record.RefreshTokenKey()
  etc.

• Replaced User related event hooks with their Record alternative:

  Old	New
  OnMailerBeforeUserResetPasswordSend() *hook.Hook[*MailerUserEvent]	OnMailerBeforeRecordResetPasswordSend() *hook.Hook[*MailerRecordEvent]
  OnMailerAfterUserResetPasswordSend() *hook.Hook[*MailerUserEvent]	OnMailerAfterRecordResetPasswordSend() *hook.Hook[*MailerRecordEvent]
  OnMailerBeforeUserVerificationSend() *hook.Hook[*MailerUserEvent]	OnMailerBeforeRecordVerificationSend() *hook.Hook[*MailerRecordEvent]
  OnMailerAfterUserVerificationSend() *hook.Hook[*MailerUserEvent]	OnMailerAfterRecordVerificationSend() *hook.Hook[*MailerRecordEvent]
  OnMailerBeforeUserChangeEmailSend() *hook.Hook[*MailerUserEvent]	OnMailerBeforeRecordChangeEmailSend() *hook.Hook[*MailerRecordEvent]
  OnMailerAfterUserChangeEmailSend() *hook.Hook[*MailerUserEvent]	OnMailerAfterRecordChangeEmailSend() *hook.Hook[*MailerRecordEvent]
  OnUsersListRequest() *hook.Hook[*UserListEvent]	OnRecordsListRequest() *hook.Hook[*RecordsListEvent]
  OnUserViewRequest() *hook.Hook[*UserViewEvent]	OnRecordViewRequest() *hook.Hook[*RecordViewEvent]
  OnUserBeforeCreateRequest() *hook.Hook[*UserCreateEvent]	OnRecordBeforeCreateRequest() *hook.Hook[*RecordCreateEvent]
  OnUserAfterCreateRequest() *hook.Hook[*UserCreateEvent]	OnRecordAfterCreateRequest() *hook.Hook[*RecordCreateEvent]
  OnUserBeforeUpdateRequest() *hook.Hook[*UserUpdateEvent]	OnRecordBeforeUpdateRequest() *hook.Hook[*RecordUpdateEvent]
  OnUserAfterUpdateRequest() *hook.Hook[*UserUpdateEvent]	OnRecordAfterUpdateRequest() *hook.Hook[*RecordUpdateEvent]
  OnUserBeforeDeleteRequest() *hook.Hook[*UserDeleteEvent]	OnRecordBeforeDeleteRequest() *hook.Hook[*RecordDeleteEvent]
  OnUserAfterDeleteRequest() *hook.Hook[*UserDeleteEvent]	OnRecordAfterDeleteRequest() *hook.Hook[*RecordDeleteEvent]
  OnUserAuthRequest() *hook.Hook[*UserAuthEvent]	OnRecordAuthRequest() *hook.Hook[*RecordAuthEvent]
  OnUserListExternalAuths() *hook.Hook[*UserListExternalAuthsEvent]	OnRecordListExternalAuths() *hook.Hook[*RecordListExternalAuthsEvent]
  OnUserBeforeUnlinkExternalAuthRequest() *hook.Hook[*UserUnlinkExternalAuthEvent]	OnRecordBeforeUnlinkExternalAuthRequest() *hook.Hook[*RecordUnlinkExternalAuthEvent]
  OnUserAfterUnlinkExternalAuthRequest() *hook.Hook[*UserUnlinkExternalAuthEvent]	OnRecordAfterUnlinkExternalAuthRequest() *hook.Hook[*RecordUnlinkExternalAuthEvent]

• Replaced forms.UserEmailLogin{} with forms.RecordPasswordLogin{} (for both username and email depending on which is enabled for the collection).

• Renamed user related core.Settings fields:

  Old	New
  core.Settings.UserAuthToken{}	core.Settings.RecordAuthToken{}
  core.Settings.UserPasswordResetToken{}	core.Settings.RecordPasswordResetToken{}
  core.Settings.UserEmailChangeToken{}	core.Settings.RecordEmailChangeToken{}
  core.Settings.UserVerificationToken{}	core.Settings.RecordVerificationToken{}

• Marked as "Deprecated" and will be removed in v0.9+:

  core.Settings.EmailAuth{}
  core.EmailAuthConfig{}
  schema.FieldTypeUser
  schema.UserOptions{}
  

• The second argument of apis.StaticDirectoryHandler(fileSystem, enableIndexFallback) now is used to enable/disable index.html forwarding on missing file (eg. in case of SPA).