pocketbase/CHANGELOG.md at 08f2190ad1e732e12d5e31ff8b59a116b4ef2af1

go/pocketbase

Fork 0

mirror of https://github.com/pocketbase/pocketbase.git synced 2025-11-23 22:55:37 +02:00

Files

Gani Georgiev 37538f2a6d auto sort rate limit rules

2024-11-18 16:07:52 +02:00

22 KiB

Raw Blame History

v0.23.0-rc15 (WIP)

Fixed rate limiter rules matching to acount for the Audience field.
Minor UI fixes (fixed duplicate record control, removed duplicated id field in the record preview, hide Impersonate button for non-auth records, auto sort rate limit rules, etc.).

v0.23.0-rc14

Allow changing collate, sort and partial constraints for indexes on system fields.
Added system cron to run once a day PRAGMA wal_checkpoint(TRUNCATE) as a fallback to assist in high traffic applications where the autocheckpoint may not be able to take its turn. Similar to the PRAGMA optimize, its execution is optional and even if it is not supported by a custom driver it will result only in a WARN app log.

Unless something else show up, this is expected to be the last release candidate before the final v0.23.0, planned to be released in the next 1-2 weeks. Because there were several experiments and changes in the rc versions regarding the migrations, if you had a v0.22.x app that you've migrated already to some of the other rc versions and have a full collections snapshot generated for it, note that you may have to regenerate the snapshot again (hopefully this would be the last time).

v0.23.0-rc13

Added WakaTime OAuth2 provider (#5829; thanks @tigawanna).
Added superuser otp EMAIL command as fallback for generating superuser OTPs from the command line in case OTP has been enabled for the _superusers but the SMTP server has deliverability issues.
⚠️ Changed OnRecordRequestOTPRequest hook to be triggered even if there is no record matching the email (aka. e.Record could be nil), allowing you to manually create a new user with the OTP request and assigning it to e.Record.
Added new sentTo system field to the _otps collection (it is managed programmatically or by superusers; it could be anything - email, phone, messanger app id, etc.). By default when the OTP is submitted via email it is automatically populated with the user email address (via the OnMailerRecordOTPSend hook's finalizer). This allow us on valid auth-with-otp request to automatically mark the user email as verified.
Added RateLimitRule.Audience optional field for restricting a rate limit rule for "@guest"-only, "@auth"-only, ""-any (default).
Added default max limits for the expressions count and length of the search filter and sort params. This is just an extra measure mostly for the case when the filter and sort parameters are resolved outside of the request context since the request size limits won't apply.
Other minor changes (better error in case of duplicated rate limit rule, fixed typos, changed field id and name length validator to max 100, updated Go deps, etc.).

v0.23.0-rc12

Fixed validation error caused by the id change in v0.23.0-rc10 migration (#5820).

v0.23.0-rc11

Fixed JSVM types errors (#5797).
Skip the default loadAuthToken middleware if e.Auth is already loaded (#5800).
Restored the API rules for the default initial "users" collection migration to be the same as in <= v0.22.x.
⚠️ Replaced the old Instagram provider with a new one that is compatible with the new Instagram Login APIs (#5588; thanks @pnmcosta). The provider key is instagram2 to prevent conflicts with existing linked users.
⚠️ Changes regarding the first superuser creation based on #5814.
⚠️ Removed apis.RequireSuperuserAuthOnlyIfAny() middleware.
⚠️ Removed RequestEvent.UnsafeRealIP() to prevent misuse and confusion with RequestEvent.RealIP() (the latter is considered safer because it checks the trusted proxy headers settings).

v0.23.0-rc10

Restore the CRC32 checksum autogeneration for the collection/field ids in order to maintain deterministic default identifier value and minimize conflicts between custom migrations and full collections snapshots. There is a system migration that will attempt to normalize existing system collections ids, but if you already migrated to v0.23.0-rc and have generated a full collections snapshot migration, you have to delete it and regenerate a new one.
Change the behavior of the default generated collections snapshot migration to act as "extend" instead of "replace" to prevent accidental data deletion. I think this would be rare but if you want the old behaviour you can edit the generated snapshot file and replace the second argument (deleteMissing) of App.ImportCollection/App.ImportCollectionsByMarshaledJSON from false to true.
Added app.SubscriptionsBroker().TotalClients() helper method to return the total registered realtime clients (#5793).

v0.23.0-rc9

Fixed auto www. redirect due to missing URI schema.
Fixed collection and field renaming when reusing an old collection/field name (#5741).
Update the "API preview" section to include information about the batch api.
Exported core.DefaultDBConnect function that could be used as a fallback when initializing custom SQLite drivers and builds.

⚠️ No longer loads the mattn/go-sqlite3 driver by default when building with CGO_ENABLED=1 to avoid multiple definition ... linker errors in case different CGO SQLite drivers or builds are used. This means that no matter of the CGO_ENABLED value, now out of the box PocketBase will always use only the pure Go driver (modernc.org/sqlite). This will be documented properly in the new website but if you want to continue using mattn/go-sqlite3 (e.g. because of the icu or other builtin extension) you could register it as follow:

package main

import (
    "database/sql"
    "log"

    "github.com/mattn/go-sqlite3"
    "github.com/pocketbase/dbx"
    "github.com/pocketbase/pocketbase"
)

func init() {
    // initialize default PRAGMAs for each new connection
    sql.Register("pb_sqlite3",
        &sqlite3.SQLiteDriver{
            ConnectHook: func(conn *sqlite3.SQLiteConn) error {
                _, err := conn.Exec(`
                    PRAGMA busy_timeout       = 10000;
                    PRAGMA journal_mode       = WAL;
                    PRAGMA journal_size_limit = 200000000;
                    PRAGMA synchronous        = NORMAL;
                    PRAGMA foreign_keys       = ON;
                    PRAGMA temp_store         = MEMORY;
                    PRAGMA cache_size         = -16000;
                `, nil)

                return err
            },
        },
    )

    dbx.BuilderFuncMap["pb_sqlite3"] = dbx.BuilderFuncMap["sqlite3"]
}

func main() {
    app := pocketbase.NewWithConfig(pocketbase.Config{
        DBConnect: func(dbPath string) (*dbx.DB, error) {
            return dbx.Open("pb_sqlite3", dbPath)
        },
    })

    // custom hooks and plugins...

    if err := app.Start(); err != nil {
        log.Fatal(err)
    }
}

Also note that if you are not planning to use the core.DefaultDBConnect fallback as part of your custom driver registration you can exclude the default pure Go driver from the build with the build tag -tags no_default_driver to reduce the binary size a little.

⚠️ Removed JSVM BaseCollection(), AuthCollection(), ViewCollection() class aliases for simplicity and to avoid confusion with the accepted constructor arguments (you can simply use as before new Collection({ type: "base", ... }); this will also initialize the default type specific options).
Other minor improvements (added validator for duplicated index definitions, updated the impersonate popup styles, added query param support for loading a collection based on its name, etc.).

v0.23.0-rc8

Lock the _otps and _mfas system collections Delete API rule for superusers only.
Reassign in the JSVM executors the global $app variable with the hook scoped e.app value to minimize the risk of a deadlock when a hook or middleware is wrapped in a transaction.
Reuse the OAuth2 created user record pointer to ensure that all its following hooks operate on the same record instance.
Added tags support for the OnFileTokenRequest hook.
Other minor changes (added index for the _collections type column, added more detailed godoc for the collection fields and core.App methods, fixed flaky record enrich tests, etc.).

v0.23.0-rc7

Register the default panic-recover middleware after the activity logger so that we can log the error.
Updated the RequestEvent.BindBody FormData type inferring rules to convert numeric strings into float64 only if the resulting minimal number string representation matches the initial FormData string value (#5687).
Fixed the JSVM types to include properly generated function declarations when the related Go functions have shortened/combined return values.
Reorganized the record table fields<->columns syncing to remove the PRAGMA writable_schema usage.

v0.23.0-rc6

Fixed realtime 403 API error on resubscribe (#5674).
Fixed the auto OAuth2 avatar mapped field assignment when the OAuth2 provider doesn't return an avatar URL (#5673). In case the avatar retrieval fails and the mapped record field "Required" option is not set, the error is silenced and only logged with WARN level.
Added Router.SEARCH(path, action) helper method for registering SEARCH endpoints.
Changed all builtin middlewares to return *hook.Handler[*core.RequestEvent] with a default middleware id for consistency and to allow removal. Or in other words, replace .BindFunc(apis.Gzip()) with .Bind(apis.Gzip()).
Updated the JSVM types to reflect the recent changes.

v0.23.0-rc5

Added Notion OAuth2 provider (#4999; thanks @s-li1).
Added monday.com OAuth2 provider (#5346; thanks @Jaytpa01).
Added option to retrieve the OIDC OAuth2 user info from the id_token payload for the cases when the provider doesn't have a dedicated user info endpoint.
Fixed the relation record picker to sort by default by @rowid instead of the created field as the latter is optional (#5641).
Fixed the UI "Set Superusers only" button click not properly resetting the input state.
Fixed the OAuth2 providers logo path shown in the "Authorized providers" UI.
Fixed the single value UI for the select, file and relation fields (#5646)

v0.23.0-rc4

Fixed the UI settings update form to prevent sending empty string for the mail password or the S3 secret options on resave of the form.
⚠️ Added an exception for the OAuth2 field in the GO->JSVM name mapping rules:
```
// old              -> new
collection.oAuth2.* -> collection.oauth2.*
```
Added more user friendly view collection truncate error message.
Added an extra suffix character to the name of autogenerated template migration file for *test suffixed collections to prevent acidentally resulting in _test.go migration files.
Added FieldsList.AddMarshaledJSON([]byte) helper method to load a serialized json array of objects or a single json object into an existing collection fields list.

Fixed the autogenerated Go migration template when updating a collection (#5631).

⚠️ If you have already used a previous prerelease and have autogenerated Go migration files, please check the migration files named {timestamp}_updated_{collection}.go and manually change:

Old (broken) New

Old (broken)	New
// add field / update field if err := json.Unmarshal([]byte(`[{ ... }]`), &collection.Fields); err != nil { return err }	// add field / update field if err := collection.Fields.AddMarshaledJSON([]byte(`{ ... }`)); err != nil { return err }

// add field / update field
if err := json.Unmarshal([]byte(`[{
    ...
}]`), &collection.Fields); err != nil {
    return err
}

// add field / update field
if err := collection.Fields.AddMarshaledJSON([]byte(`{
    ...
}`)); err != nil {
    return err
}

To test that your Go migration files work correctly you can try to start PocketBase with a new temp pb_data, e.g.:

go run . serve --dir="pb_data_temp"

v0.23.0-rc3

Make PRAGMA optimize statement optional in case it is not supported by the driver (#5611).
Reapply the minimum required pb_data/auxiliary.db migrations if the db file was manually deleted (#5618).
To avoid confusion and unnecessary casting, the hook.HandlerFunc[T] type has been removed and instead everywhere we now use directly the underlying function definition, aka.:
```
func(T) error
```
Fixed the UI input field type of the OTP.length field (#5617).
Other minor fixes (fixed API preview and examples error message typos, better hint for combined/multi-spaced view query columns, fixed the path for the HTTPS green favicon path, etc.).

v0.23.0-rc2

Small update to the earlier v0.23.0-rc that uses pb_data/auxiliary.db instead of pb_data/aux.db because it seems that on Windows aux is disallowed as file name (#5607). If you have already upgraded to v0.23.0-rc please rename manually your pb_data/aux.db file to pb_data/auxiliary.db.

v0.23.0-rc

PocketBase v0.23.0-rc is a major refactor of the internals with the overall goal of making PocketBase an easier to use Go framework.

There are many changes but to highlight some of the most notable ones:

Replaced echo with a new router built on top of the Go 1.22 net/http mux enhancements.
Merged daos packages in core.App to simplify the DB operations (the models package structs are also migrated in core).
Option to specify custom DBConnect function as part of the app configuration to allow different database/sql SQLite drivers (turso/libsql, sqlcipher, etc.) and custom builds.
New hooks allowing better control over the execution chain and error handling (including wrapping an entire hook chain in a single DB transaction).
Various Record model improvements (support for get/set modifiers, simplfied file upload by treating the file(s) as regular field value like record.Set("document", file), etc.).
Dedicated fields structs with safer defaults to make it easier creating/updating collections programmatically.
Option to mark field as Private/Hidden, disallowing regular users to read or modify it (there is also a dedicated Record hook to hide/unhide Record fields programmatically from a single place).
Option to customize the default system collection fields (id, email, password, etc.).
Admins are now system _superusers auth records.
Builtin rate limiter (supports tags, wildcards and exact routes matching).
Batch/transactional Web API endpoint.
Impersonate Web API endpoint (it could be also used for generating fixed/non-refreshable superuser tokens, aka. "API keys").
Support for custom user request activity log attributes.
One-Time Password (OTP) auth method (via email code).
Multi-Factor Authentication (MFA) support (currently requires any 2 different auth methods to be used).
Support for Record "proxy/projection" in preparation for the planned autogeneration of typed Go record models.
Various minor UI improvements (recursive Presentable view, slightly different collection options organization, zoom/pan for the logs chart, etc.)
and many more...

In terms of performance, the Go standard router mux is known to be slightly slower compared to Gin, Echo, etc. implementations, but based on my local tests the difference is negliable. The benchmarks repo will be updated with the final v0.23.0 release (currently there seems to be ~10% memory consumption increase which I'll have to investigate to see whether it is from the router change or from the new hooks).

Go/JSVM APIs changes

For upgrading to PocketBase v0.23.0, please refer to:

Go: https://pocketbase.io/v023upgrade/go/.
JSVM: https://pocketbase.io/v023upgrade/jsvm/.

SDKs changes

Web APIs changes

New POST /api/batch endpoint.
New GET /api/collections/meta/scaffolds endpoint.
New DELETE /api/collections/{collection}/truncate endpoint.
New POST /api/collections/{collection}/request-otp endpoint.
New POST /api/collections/{collection}/auth-with-otp endpoint.
New POST /api/collections/{collection}/impersonate/{id} endpoint.
⚠️ Removed /api/admins/* endpoints because admins are converted to _superusers auth collection records.
⚠️ Previously when uploading new files to a multiple file field, new files were automatically appended to the existing field values. This behaviour has changed with v0.23+ and for consistency with the other multi-valued fields when uploading new files they will replace the old ones. If you want to prepend or append new files to an existing multiple file field value you can use the + prefix or suffix:
```
"documents": [file1, file2]  // => [file1_name, file2_name]
"+documents": [file1, file2] // => [file1_name, file2_name, old1_name, old2_name]
"documents+": [file1, file2] // => [old1_name, old2_name, file1_name, file2_name]
```
⚠️ Removed GET /records/{id}/external-auths and DELETE /records/{id}/external-auths/{provider} endpoints because this is now handled by sending list and delete requests to the _externalAuths collection.
⚠️ Changes to the app settings model fields and response (+new options such as trustedProxy, rateLimits, batch, etc.). The app settings Web APIs are mostly used by the Dashboard UI and rarely by the end users, but if you want to check all settings changes please refer to the Settings Go struct.
⚠️ New flatten Collection model and fields structure. The Collection model Web APIs are mostly used by the Dashboard UI and rarely by the end users, but if you want to check all changes please refer to the Collection Go struct.

⚠️ The top level error response code key was renamed to status for consistency with the Go APIs. The error field key remains code:

{
    "status": 400, // <-- old: "code"
    "message": "Failed to create record.",
    "data": {
        "title": {
            "code": "validation_required",
            "message": "Missing required value."
        }
    }
}

⚠️ New fields in the GET /api/collections/{collection}/auth-methods response. The old authProviders, usernamePassword, emailPassword fields are still returned in the response but are considered deprecated and will be removed in the future.

{
    "mfa": {
        "duration": 100,
        "enabled": true
    },
    "otp": {
        "duration": 0,
        "enabled": false
    },
    "password": {
        "enabled": true,
        "identityFields": ["email", "username"]
    },
    "oauth2": {
        "enabled": true,
        "providers": [{"name": "gitlab", ...}, {"name": "google", ...}]
    },
    // old fields...
}

⚠️ Soft-deprecated the OAuth2 auth success meta.avatarUrl field in favour of meta.avatarURL.

22 KiB Raw Blame History