diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 73e27c1ef..30753d50b 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -291,6 +291,8 @@ jobs: README.md RELEASE.md CODE_OF_CONDUCT.md + librclone\README.md + backend\s3\README.md docs/content/{authors,bugs,changelog,docs,downloads,faq,filtering,gui,install,licence,overview,privacy}.md - name: Scan edits of autogenerated files diff --git a/backend/s3/README.md b/backend/s3/README.md index d4ea43d9a..014218b09 100644 --- a/backend/s3/README.md +++ b/backend/s3/README.md @@ -1,4 +1,4 @@ -## Adding a new s3 provider +# Adding a new s3 provider It is quite easy to add a new S3 provider to rclone. @@ -12,179 +12,202 @@ All tags can be found in `backend/s3/providers.go` Provider Struct. Looking through a few of the yaml files as examples should make things clear. `AWS.yaml` as the most config. pasting. -### YAML +## YAML In `backend/s3/provider/YourProvider.yaml` - name - description - - More like the full name often "YourProvider + Object Storage" + - More like the full name often "YourProvider + Object Storage" - [Region] - - Any regions your provider supports or the defaults (use `region: {}` for this) - - Example from AWS.yaml: - ```yaml - region: - us-east-1: |- - The default endpoint - a good choice if you are unsure. - US Region, Northern Virginia, or Pacific Northwest. - Leave location constraint empty. - ``` - - The defaults (as seen in Rclone.yaml): - ```yaml - region: - "": |- - Use this if unsure. - Will use v4 signatures and an empty region. - other-v2-signature: |- - Use this only if v4 signatures don't work. - E.g. pre Jewel/v10 CEPH. - ``` + - Any regions your provider supports or the defaults (use `region: {}` for this) + - Example from AWS.yaml: + + ```yaml + region: + us-east-1: |- + The default endpoint - a good choice if you are unsure. + US Region, Northern Virginia, or Pacific Northwest. + Leave location constraint empty. + ``` + + - The defaults (as seen in Rclone.yaml): + + ```yaml + region: + "": |- + Use this if unsure. + Will use v4 signatures and an empty region. + other-v2-signature: |- + Use this only if v4 signatures don't work. + E.g. pre Jewel/v10 CEPH. + ``` + - [Endpoint] - - Any endpoints your provider supports - - Example from Mega.yaml - ```yaml - endpoint: - s3.eu-central-1.s4.mega.io: Mega S4 eu-central-1 (Amsterdam) - ``` + - Any endpoints your provider supports + + - Example from Mega.yaml + + ```yaml + endpoint: + s3.eu-central-1.s4.mega.io: Mega S4 eu-central-1 (Amsterdam) + ``` + - [Location Constraint] - - The Location Constraint of your remote, often same as region. - - Example from AWS.yaml - ```yaml - location_constraint: - "": Empty for US Region, Northern Virginia, or Pacific Northwest - us-east-2: US East (Ohio) Region - ``` + - The Location Constraint of your remote, often same as region. + - Example from AWS.yaml + + ```yaml + location_constraint: + "": Empty for US Region, Northern Virginia, or Pacific Northwest + us-east-2: US East (Ohio) Region + ``` + - [ACL] - - Identical across *most* providers. Select the default with `acl: {}` - - Example from AWS.yaml - ```yaml - acl: - private: |- - Owner gets FULL_CONTROL. - No one else has access rights (default). - public-read: |- - Owner gets FULL_CONTROL. - The AllUsers group gets READ access. - public-read-write: |- - Owner gets FULL_CONTROL. - The AllUsers group gets READ and WRITE access. - Granting this on a bucket is generally not recommended. - authenticated-read: |- - Owner gets FULL_CONTROL. - The AuthenticatedUsers group gets READ access. - bucket-owner-read: |- - Object owner gets FULL_CONTROL. - Bucket owner gets READ access. - If you specify this canned ACL when creating a bucket, Amazon S3 ignores it. - bucket-owner-full-control: |- - Both the object owner and the bucket owner get FULL_CONTROL over the object. - If you specify this canned ACL when creating a bucket, Amazon S3 ignores it. - ``` + - Identical across *most* providers. Select the default with `acl: {}` + - Example from AWS.yaml + + ```yaml + acl: + private: |- + Owner gets FULL_CONTROL. + No one else has access rights (default). + public-read: |- + Owner gets FULL_CONTROL. + The AllUsers group gets READ access. + public-read-write: |- + Owner gets FULL_CONTROL. + The AllUsers group gets READ and WRITE access. + Granting this on a bucket is generally not recommended. + authenticated-read: |- + Owner gets FULL_CONTROL. + The AuthenticatedUsers group gets READ access. + bucket-owner-read: |- + Object owner gets FULL_CONTROL. + Bucket owner gets READ access. + If you specify this canned ACL when creating a bucket, Amazon S3 ignores it. + bucket-owner-full-control: |- + Both the object owner and the bucket owner get FULL_CONTROL over the object. + If you specify this canned ACL when creating a bucket, Amazon S3 ignores it. + ``` + - [Storage Class] - - Identical across *most* providers. - - Defaults from AWS.yaml - ```yaml - storage_class: - "": Default - STANDARD: Standard storage class - REDUCED_REDUNDANCY: Reduced redundancy storage class - STANDARD_IA: Standard Infrequent Access storage class - ONEZONE_IA: One Zone Infrequent Access storage class - GLACIER: Glacier Flexible Retrieval storage class - DEEP_ARCHIVE: Glacier Deep Archive storage class - INTELLIGENT_TIERING: Intelligent-Tiering storage class - GLACIER_IR: Glacier Instant Retrieval storage class - ``` + - Identical across *most* providers. + - Defaults from AWS.yaml + + ```yaml + storage_class: + "": Default + STANDARD: Standard storage class + REDUCED_REDUNDANCY: Reduced redundancy storage class + STANDARD_IA: Standard Infrequent Access storage class + ONEZONE_IA: One Zone Infrequent Access storage class + GLACIER: Glacier Flexible Retrieval storage class + DEEP_ARCHIVE: Glacier Deep Archive storage class + INTELLIGENT_TIERING: Intelligent-Tiering storage class + GLACIER_IR: Glacier Instant Retrieval storage class + ``` + - [Server Side Encryption] - - Not common, identical across *most* providers. - - Defaults from AWS.yaml - ```yaml - server_side_encryption: - "": None - AES256: AES256 - aws:kms: aws:kms - ``` + - Not common, identical across *most* providers. + - Defaults from AWS.yaml + + ```yaml + server_side_encryption: + "": None + AES256: AES256 + aws:kms: aws:kms + ``` + - [Advanced Options] - - All advanced options are Boolean - if true the configurator asks about that value, if not it doesn't: - ```go - BucketACL bool `yaml:"bucket_acl,omitempty"` - DirectoryBucket bool `yaml:"directory_bucket,omitempty"` - LeavePartsOnError bool `yaml:"leave_parts_on_error,omitempty"` - RequesterPays bool `yaml:"requester_pays,omitempty"` - SSECustomerAlgorithm bool `yaml:"sse_customer_algorithm,omitempty"` - SSECustomerKey bool `yaml:"sse_customer_key,omitempty"` - SSECustomerKeyBase64 bool `yaml:"sse_customer_key_base64,omitempty"` - SSECustomerKeyMd5 bool `yaml:"sse_customer_key_md5,omitempty"` - SSEKmsKeyID bool `yaml:"sse_kms_key_id,omitempty"` - STSEndpoint bool `yaml:"sts_endpoint,omitempty"` - UseAccelerateEndpoint bool `yaml:"use_accelerate_endpoint,omitempty"` - ``` - - Example from AWS.yaml: - ```yaml - bucket_acl: true - directory_bucket: true - leave_parts_on_error: true - requester_pays: true - sse_customer_algorithm: true - sse_customer_key: true - sse_customer_key_base64: true - sse_customer_key_md5: true - sse_kms_key_id: true - sts_endpoint: true - use_accelerate_endpoint: true - ``` + - All advanced options are Boolean - if true the configurator asks about that + value, if not it doesn't: + + ```go + BucketACL bool `yaml:"bucket_acl,omitempty"` + DirectoryBucket bool `yaml:"directory_bucket,omitempty"` + LeavePartsOnError bool `yaml:"leave_parts_on_error,omitempty"` + RequesterPays bool `yaml:"requester_pays,omitempty"` + SSECustomerAlgorithm bool `yaml:"sse_customer_algorithm,omitempty"` + SSECustomerKey bool `yaml:"sse_customer_key,omitempty"` + SSECustomerKeyBase64 bool `yaml:"sse_customer_key_base64,omitempty"` + SSECustomerKeyMd5 bool `yaml:"sse_customer_key_md5,omitempty"` + SSEKmsKeyID bool `yaml:"sse_kms_key_id,omitempty"` + STSEndpoint bool `yaml:"sts_endpoint,omitempty"` + UseAccelerateEndpoint bool `yaml:"use_accelerate_endpoint,omitempty"` + ``` + + - Example from AWS.yaml: + + ```yaml + bucket_acl: true + directory_bucket: true + leave_parts_on_error: true + requester_pays: true + sse_customer_algorithm: true + sse_customer_key: true + sse_customer_key_base64: true + sse_customer_key_md5: true + sse_kms_key_id: true + sts_endpoint: true + use_accelerate_endpoint: true + ``` + - Quirks - - Quirks are discovered through documentation and running the tests as seen below. - - Most quirks are *bool as to have 3 values, `true`, `false` and `dont care`. - ```go - type Quirks struct { - ListVersion *int `yaml:"list_version,omitempty"` // 1 or 2 - ForcePathStyle *bool `yaml:"force_path_style,omitempty"` // true = path-style - ListURLEncode *bool `yaml:"list_url_encode,omitempty"` - UseMultipartEtag *bool `yaml:"use_multipart_etag,omitempty"` - UseAlreadyExists *bool `yaml:"use_already_exists,omitempty"` - UseAcceptEncodingGzip *bool `yaml:"use_accept_encoding_gzip,omitempty"` - MightGzip *bool `yaml:"might_gzip,omitempty"` - UseMultipartUploads *bool `yaml:"use_multipart_uploads,omitempty"` - UseUnsignedPayload *bool `yaml:"use_unsigned_payload,omitempty"` - UseXID *bool `yaml:"use_x_id,omitempty"` - SignAcceptEncoding *bool `yaml:"sign_accept_encoding,omitempty"` - CopyCutoff *int64 `yaml:"copy_cutoff,omitempty"` - MaxUploadParts *int `yaml:"max_upload_parts,omitempty"` - MinChunkSize *int64 `yaml:"min_chunk_size,omitempty"` - } + - Quirks are discovered through documentation and running the tests as seen below. + - Most quirks are *bool as to have 3 values, `true`, `false` and `dont care`. + + ```go + type Quirks struct { + ListVersion *int `yaml:"list_version,omitempty"` // 1 or 2 + ForcePathStyle *bool `yaml:"force_path_style,omitempty"` // true = path-style + ListURLEncode *bool `yaml:"list_url_encode,omitempty"` + UseMultipartEtag *bool `yaml:"use_multipart_etag,omitempty"` + UseAlreadyExists *bool `yaml:"use_already_exists,omitempty"` + UseAcceptEncodingGzip *bool `yaml:"use_accept_encoding_gzip,omitempty"` + MightGzip *bool `yaml:"might_gzip,omitempty"` + UseMultipartUploads *bool `yaml:"use_multipart_uploads,omitempty"` + UseUnsignedPayload *bool `yaml:"use_unsigned_payload,omitempty"` + UseXID *bool `yaml:"use_x_id,omitempty"` + SignAcceptEncoding *bool `yaml:"sign_accept_encoding,omitempty"` + CopyCutoff *int64 `yaml:"copy_cutoff,omitempty"` + MaxUploadParts *int `yaml:"max_upload_parts,omitempty"` + MinChunkSize *int64 `yaml:"min_chunk_size,omitempty"` + } ``` - - Example from AWS.yaml + + - Example from AWS.yaml + ```yaml - quirks: - might_gzip: false # Never auto gzips objects - use_unsigned_payload: false # AWS has trailer support + quirks: + might_gzip: false # Never auto gzips objects + use_unsigned_payload: false # AWS has trailer support ``` Note that if you omit a section, eg `region` then the user won't be asked that question, and if you add an empty section e.g. `region: {}` then the defaults from the `Other.yaml` will be used. -### DOCS +## DOCS - `docs/content/s3.md` - - Add the provider at the top of the page. - - Add a section about the provider linked from there. - - Make sure this is in alphabetical order in the `Providers` section. - - Add a transcript of a trial `rclone config` session - - Edit the transcript to remove things which might change in subsequent versions - - **Do not** alter or add to the autogenerated parts of `s3.md` - - Rule of thumb: don't edit anything not mentioned above. - - **Do not** run `make backenddocs` or `bin/make_backend_docs.py s3` - - This will make autogenerated changes! + - Add the provider at the top of the page. + - Add a section about the provider linked from there. + - Make sure this is in alphabetical order in the `Providers` section. + - Add a transcript of a trial `rclone config` session + - Edit the transcript to remove things which might change in subsequent versions + - **Do not** alter or add to the autogenerated parts of `s3.md` + - Rule of thumb: don't edit anything not mentioned above. + - **Do not** run `make backenddocs` or `bin/make_backend_docs.py s3` + - This will make autogenerated changes! - `README.md` - this is the home page in github - - Add the provider and a link to the section you wrote in `docs/contents/s3.md` + - Add the provider and a link to the section you wrote in `docs/contents/s3.md` - `docs/content/_index.md` - this is the home page of rclone.org - - Add the provider and a link to the section you wrote in `docs/contents/s3.md` + - Add the provider and a link to the section you wrote in `docs/contents/s3.md` - Once you've written the docs, run `make serve` and check they look OK - in the web browser and the links (internal and external) all work. + in the web browser and the links (internal and external) all work. -### TESTS +## TESTS Once you've written the code, test `rclone config` works to your satisfaction and looks correct, and check the integration tests work diff --git a/docs/content/b2.md b/docs/content/b2.md index 08ec37fe0..993f7b5d5 100644 --- a/docs/content/b2.md +++ b/docs/content/b2.md @@ -78,7 +78,6 @@ List the contents of a bucket rclone ls remote:bucket ``` - Sync `/home/local/directory` to the remote bucket, deleting any excess files in the bucket. @@ -98,7 +97,7 @@ Follow Backblaze's docs to create an Application Key with the required permission and add the `applicationKeyId` as the `account` and the `Application Key` itself as the `key`. -Note that you must put the _applicationKeyId_ as the `account` – you +Note that you must put the *applicationKeyId* as the `account` – you can't use the master Account ID. If you try then B2 will return 401 errors. @@ -192,7 +191,7 @@ You may opt in to a "hard delete" of files with the `--b2-hard-delete` flag which permanently removes files on deletion instead of hiding them. -Old versions of files, where available, are visible using the +Old versions of files, where available, are visible using the `--b2-versions` flag. It is also possible to view a bucket as it was at a certain point in time, diff --git a/docs/content/box.md b/docs/content/box.md index 397123c19..491849d2d 100644 --- a/docs/content/box.md +++ b/docs/content/box.md @@ -109,7 +109,9 @@ rclone ls remote: To copy a local directory to an Box directory called backup - rclone copy /home/source remote:backup +```console +rclone copy /home/source remote:backup +``` ### Using rclone with an Enterprise account with SSO @@ -249,8 +251,8 @@ either be actually deleted from Box or moved to the trash. Emptying the trash is supported via the rclone however cleanup command however this deletes every trashed file and folder individually so it -may take a very long time. -Emptying the trash via the WebUI does not have this limitation +may take a very long time. +Emptying the trash via the WebUI does not have this limitation so it is advised to empty the trash via the WebUI. ### Root folder ID diff --git a/docs/content/crypt.md b/docs/content/crypt.md index e65b5b846..b9acc1a5a 100644 --- a/docs/content/crypt.md +++ b/docs/content/crypt.md @@ -707,9 +707,9 @@ the same in the new encrypted remote. This will have the following advantages - * `rclone sync` will check the checksums while copying - * you can use `rclone check` between the encrypted remotes - * you don't decrypt and encrypt unnecessarily +- `rclone sync` will check the checksums while copying +- you can use `rclone check` between the encrypted remotes +- you don't decrypt and encrypt unnecessarily For example, let's say you have your original remote at `remote:` with the encrypted version at `eremote:` with path `remote:crypt`. You @@ -719,11 +719,15 @@ as `eremote:`. To sync the two remotes you would do - rclone sync --interactive remote:crypt remote2:crypt +```console +rclone sync --interactive remote:crypt remote2:crypt +``` And to check the integrity you would do - rclone check remote:crypt remote2:crypt +```console +rclone check remote:crypt remote2:crypt +``` ## File formats @@ -734,8 +738,8 @@ has a header and is divided into chunks. #### Header - * 8 bytes magic string `RCLONE\x00\x00` - * 24 bytes Nonce (IV) +- 8 bytes magic string `RCLONE\x00\x00` +- 24 bytes Nonce (IV) The initial nonce is generated from the operating systems crypto strong random number generator. The nonce is incremented for each @@ -753,8 +757,8 @@ authenticate messages. Each chunk contains: - * 16 Bytes of Poly1305 authenticator - * 1 - 65536 bytes XSalsa20 encrypted data +- 16 Bytes of Poly1305 authenticator +- 1 - 65536 bytes XSalsa20 encrypted data 64k chunk size was chosen as the best performing chunk size (the authenticator takes too much time below this and the performance drops @@ -767,15 +771,15 @@ This uses a 32 byte (256 bit key) key derived from the user password. 1 byte file will encrypt to - * 32 bytes header - * 17 bytes data chunk +- 32 bytes header +- 17 bytes data chunk 49 bytes total 1 MiB (1048576 bytes) file will encrypt to - * 32 bytes header - * 16 chunks of 65568 bytes +- 32 bytes header +- 16 chunks of 65568 bytes 1049120 bytes total (a 0.05% overhead). This is the overhead for big files. @@ -798,8 +802,8 @@ it on the cloud storage system. This means that - * filenames with the same name will encrypt the same - * filenames which start the same won't have a common prefix +- filenames with the same name will encrypt the same +- filenames which start the same won't have a common prefix This uses a 32 byte key (256 bits) and a 16 byte (128 bits) IV both of which are derived from the user password. @@ -808,8 +812,8 @@ After encryption they are written out using a modified version of standard `base32` encoding as described in RFC4648. The standard encoding is modified in two ways: - * it becomes lower case (no-one likes upper case filenames!) - * we strip the padding character `=` +- it becomes lower case (no-one likes upper case filenames!) +- we strip the padding character `=` `base32` is used rather than the more efficient `base64` so rclone can be used on case insensitive remotes (e.g. Windows, Box, Dropbox, Onedrive etc). @@ -825,6 +829,7 @@ then rclone uses an internal one. encrypted data. For full protection against this you should always use a salt. -## SEE ALSO +## See Also -* [rclone cryptdecode](/commands/rclone_cryptdecode/) - Show forward/reverse mapping of encrypted filenames +- [rclone cryptdecode](/commands/rclone_cryptdecode/) - Show forward/reverse +mapping of encrypted filenames. diff --git a/docs/content/dropbox.md b/docs/content/dropbox.md index c01bc2892..9c671e578 100644 --- a/docs/content/dropbox.md +++ b/docs/content/dropbox.md @@ -604,10 +604,9 @@ issue an error message `File name disallowed - not uploading` if it attempts to upload one of those file names, but the sync won't fail. Some errors may occur if you try to sync copyright-protected files -because Dropbox has its own [copyright detector](https://techcrunch.com/2014/03/30/how-dropbox-knows-when-youre-sharing-copyrighted-stuff-without-actually-looking-at-your-stuff/) that -prevents this sort of file being downloaded. This will return the error `ERROR : -/path/to/your/file: Failed to copy: failed to open source object: -path/restricted_content/.` +because Dropbox has its own [copyright detector](https://techcrunch.com/2014/03/30/how-dropbox-knows-when-youre-sharing-copyrighted-stuff-without-actually-looking-at-your-stuff/) +that prevents this sort of file being downloaded. This will return the error +`ERROR : /path/to/your/file: Failed to copy: failed to open source object: path/restricted_content/.` If you have more than 10,000 files in a directory then `rclone purge dropbox:dir` will return the error `Failed to purge: There are too @@ -617,7 +616,8 @@ many files involved in this operation`. As a work-around do an When using `rclone link` you'll need to set `--expire` if using a non-personal account otherwise the visibility may not be correct. (Note that `--expire` isn't supported on personal accounts). See the -[forum discussion](https://forum.rclone.org/t/rclone-link-dropbox-permissions/23211) and the +[forum discussion](https://forum.rclone.org/t/rclone-link-dropbox-permissions/23211) +and the [dropbox SDK issue](https://github.com/dropbox/dropbox-sdk-go-unofficial/issues/75). Modification times for Dropbox Paper documents are not exact, and @@ -627,23 +627,34 @@ or so, or use `--ignore-times` to force a full sync. ## Get your own Dropbox App ID -When you use rclone with Dropbox in its default configuration you are using rclone's App ID. This is shared between all the rclone users. +When you use rclone with Dropbox in its default configuration you are using +rclone's App ID. This is shared between all the rclone users. Here is how to create your own Dropbox App ID for rclone: -1. Log into the [Dropbox App console](https://www.dropbox.com/developers/apps/create) with your Dropbox Account (It need not -to be the same account as the Dropbox you want to access) +1. Log into the [Dropbox App console](https://www.dropbox.com/developers/apps/create) +with your Dropbox Account (It need not to be the same account as the Dropbox you +want to access) 2. Choose an API => Usually this should be `Dropbox API` -3. Choose the type of access you want to use => `Full Dropbox` or `App Folder`. If you want to use Team Folders, `Full Dropbox` is required ([see here](https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/How-to-create-team-folder-inside-my-app-s-folder/m-p/601005/highlight/true#M27911)). +3. Choose the type of access you want to use => `Full Dropbox` or `App Folder`. +If you want to use Team Folders, `Full Dropbox` is required +([see here](https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/How-to-create-team-folder-inside-my-app-s-folder/m-p/601005/highlight/true#M27911)). 4. Name your App. The app name is global, so you can't use `rclone` for example 5. Click the button `Create App` -6. Switch to the `Permissions` tab. Enable at least the following permissions: `account_info.read`, `files.metadata.write`, `files.content.write`, `files.content.read`, `sharing.write`. The `files.metadata.read` and `sharing.read` checkboxes will be marked too. Click `Submit` +6. Switch to the `Permissions` tab. Enable at least the following permissions: +`account_info.read`, `files.metadata.write`, `files.content.write`, `files.content.read`, +`sharing.write`. The `files.metadata.read` and `sharing.read` checkboxes will be +marked too. Click `Submit` -7. Switch to the `Settings` tab. Fill `OAuth2 - Redirect URIs` as `http://localhost:53682/` and click on `Add` +7. Switch to the `Settings` tab. Fill `OAuth2 - Redirect URIs` as `http://localhost:53682/` +and click on `Add` -8. Find the `App key` and `App secret` values on the `Settings` tab. Use these values in rclone config to add a new remote or edit an existing remote. The `App key` setting corresponds to `client_id` in rclone config, the `App secret` corresponds to `client_secret` +8. Find the `App key` and `App secret` values on the `Settings` tab. Use these +values in rclone config to add a new remote or edit an existing remote. +The `App key` setting corresponds to `client_id` in rclone config, the +`App secret` corresponds to `client_secret` diff --git a/docs/content/onedrive.md b/docs/content/onedrive.md index 7e450fd4a..f129e7c4c 100644 --- a/docs/content/onedrive.md +++ b/docs/content/onedrive.md @@ -147,7 +147,9 @@ To create your own Client ID, please follow these steps: - If you have not created an Azure account, you will be prompted to. This is free, but you need to provide a phone number, address, and credit card for identity verification. -2. Enter a name for your app, choose account type `Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)`, +2. Enter a name for your app, choose account type `Accounts in any organizational + directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts + (e.g. Skype, Xbox)`, select `Web` in `Redirect URI`, then type (do not copy and paste) `http://localhost:53682/` and click Register. Copy and keep the `Application (client) ID` under the app name for later use. @@ -170,7 +172,8 @@ The access_scopes option allows you to configure the permissions requested by rc See [Microsoft Docs](https://docs.microsoft.com/en-us/graph/permissions-reference#files-permissions) for more information about the different scopes. -The `Sites.Read.All` permission is required if you need to [search SharePoint sites when configuring the remote](https://github.com/rclone/rclone/pull/5883). +The `Sites.Read.All` permission is required if you need to +[search SharePoint sites when configuring the remote](https://github.com/rclone/rclone/pull/5883). However, if that permission is not assigned, you need to exclude `Sites.Read.All` from your access scopes or set `disable_site_permission` option to true in the advanced options. @@ -1109,7 +1112,8 @@ setting: 1. `Install-Module -Name Microsoft.Online.SharePoint.PowerShell` (in case you haven't installed this already) 2. `Import-Module Microsoft.Online.SharePoint.PowerShell -DisableNameChecking` -3. `Connect-SPOService -Url https://YOURSITE-admin.sharepoint.com -Credential YOU@YOURSITE.COM` (replacing `YOURSITE`, `YOU`, `YOURSITE.COM` with the actual values; this will +3. `Connect-SPOService -Url https://YOURSITE-admin.sharepoint.com -Credential YOU@YOURSITE.COM` + (replacing `YOURSITE`, `YOU`, `YOURSITE.COM` with the actual values; this will prompt for your credentials) 4. `Set-SPOTenant -EnableMinimumVersionRequirement $False` 5. `Disconnect-SPOService` (to disconnect from the server) diff --git a/docs/content/oracleobjectstorage/_index.md b/docs/content/oracleobjectstorage/_index.md index 849b89da5..1b8cba0a5 100644 --- a/docs/content/oracleobjectstorage/_index.md +++ b/docs/content/oracleobjectstorage/_index.md @@ -156,10 +156,10 @@ rclone ls remote:bucket rclone ls remote:bucket --max-depth 1 ``` -## Authentication Providers +## Authentication Providers -OCI has various authentication methods. To learn more about authentication methods please refer [oci authentication -methods](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdk_authentication_methods.htm) +OCI has various authentication methods. To learn more about authentication methods +please refer [oci authentication methods](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdk_authentication_methods.htm) These choices can be specified in the rclone config file. Rclone supports the following OCI authentication provider. @@ -900,4 +900,5 @@ Options: {{< rem autogenerated options stop >}} ## Tutorials + ### [Mounting Buckets](/oracleobjectstorage/tutorial_mount/) diff --git a/docs/content/oracleobjectstorage/tutorial_mount.md b/docs/content/oracleobjectstorage/tutorial_mount.md index c50937dfa..a2d315090 100644 --- a/docs/content/oracleobjectstorage/tutorial_mount.md +++ b/docs/content/oracleobjectstorage/tutorial_mount.md @@ -41,7 +41,7 @@ From repo : ol8_developer Summary : rsync for cloud storage URL : http://rclone.org/ License : MIT -Description : Rclone is a command line program to sync files and directories to and from various cloud services. +Description : Rclone is a command line program to sync files and directories to and from various cloud services. ``` To run it as a mount helper you should symlink rclone binary to /sbin/mount.rclone diff --git a/docs/content/pcloud.md b/docs/content/pcloud.md index d683e69ec..904bf81aa 100644 --- a/docs/content/pcloud.md +++ b/docs/content/pcloud.md @@ -161,15 +161,17 @@ The command will enumerate available directories, allowing you to locate the appropriate Folder ID for subsequent use. Example: -``` + +```console $ rclone lsf --dirs-only -Fip --csv TestPcloud: dxxxxxxxx2,My Music/ dxxxxxxxx3,My Pictures/ dxxxxxxxx4,My Videos/ ``` -So if the folder you want rclone to use your is "My Music/", then use the returned id from ```rclone lsf``` command (ex. `dxxxxxxxx2`) as -the `root_folder_id` variable value in the config file. +So if the folder you want rclone to use your is "My Music/", then use the returned +id from ```rclone lsf``` command (ex. `dxxxxxxxx2`) as the `root_folder_id` variable +value in the config file. {{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/pcloud/pcloud.go then run make backenddocs" >}} ### Standard options diff --git a/docs/content/remote_setup.md b/docs/content/remote_setup.md index 3e3a68c0b..bc709e18d 100644 --- a/docs/content/remote_setup.md +++ b/docs/content/remote_setup.md @@ -16,7 +16,8 @@ three ways of doing it, described below. ## Configuring using rclone authorize On the headless machine run [rclone config](/commands/rclone_config), but -answer `N` to the question `Use web browser to automatically authenticate rclone with remote?`. +answer `N` to the question `Use web browser to automatically authenticate +rclone with remote?`. ```text Use web browser to automatically authenticate rclone with remote? @@ -108,7 +109,8 @@ ssh -L localhost:53682:localhost:53682 username@remote_server ``` Then on the headless machine run [rclone config](/commands/rclone_config) and -answer `Y` to the question `Use web browser to automatically authenticate rclone with remote?`. +answer `Y` to the question `Use web browser to automatically authenticate rclone +with remote?`. ```text Use web browser to automatically authenticate rclone with remote? diff --git a/docs/content/webdav.md b/docs/content/webdav.md index c9f7ec39d..a8a692ed8 100644 --- a/docs/content/webdav.md +++ b/docs/content/webdav.md @@ -531,7 +531,7 @@ installed and configured, an access token is obtained by running the `oidc-token` command. The following example shows a (shortened) access token obtained from the *XDC* OIDC Provider. -``` +```text paul@celebrimbor:~$ oidc-token XDC eyJraWQ[...]QFXDt0 paul@celebrimbor:~$ @@ -555,7 +555,7 @@ edit the advanced config and enter the command to get a bearer token The following example config shows a WebDAV endpoint that uses oidc-agent to supply an access token from the *XDC* OIDC Provider. -``` +```ini [dcache] type = webdav url = https://dcache.example.org/ diff --git a/fs/rc/js/README.md b/fs/rc/js/README.md index 4a546d228..d258fc1a4 100644 --- a/fs/rc/js/README.md +++ b/fs/rc/js/README.md @@ -16,10 +16,11 @@ This contains the following files ## Compiling -This can be compiled by using `make` or alternatively `GOARCH=wasm GOOS=js go build -o rclone.wasm` +This can be compiled by using `make` or alternatively +`GOARCH=wasm GOOS=js go build -o rclone.wasm`. ## Running Run the test server with `make serve` and examine the page at -http://localhost:3000/ - look at the javascript console and look at + - look at the javascript console and look at the end of `loader.js` for how that works. diff --git a/librclone/README.md b/librclone/README.md index 7cdc4db6c..9ff580c31 100644 --- a/librclone/README.md +++ b/librclone/README.md @@ -13,18 +13,22 @@ The shims are a thin wrapper over the rclone RPC. The implementation is based on cgo; to build it you need Go and a GCC compatible C compiler (GCC or Clang). On Windows you can use the MinGW ports, e.g. by installing -in a [MSYS2](https://www.msys2.org) distribution (you may now install GCC in the newer -and recommended UCRT64 subsystem, however there were compatibility issues with previous -versions of cgo where, if not force rebuild with go build option `-a` helped, you had -to resort to the classic MINGW64 subsystem). +in a [MSYS2](https://www.msys2.org) distribution (you may now install GCC in the +newer and recommended UCRT64 subsystem, however there were compatibility issues +with previous versions of cgo where, if not force rebuild with go build option `-a` +helped, you had to resort to the classic MINGW64 subsystem). Build a shared library like this (change from .so to .dll on Windows): - go build --buildmode=c-shared -o librclone.so github.com/rclone/rclone/librclone +```console +go build --buildmode=c-shared -o librclone.so github.com/rclone/rclone/librclone +``` Build a static library like this (change from .a to .lib on Windows): - go build --buildmode=c-archive -o librclone.a github.com/rclone/rclone/librclone +```console +go build --buildmode=c-archive -o librclone.a github.com/rclone/rclone/librclone +``` Both the above commands will also generate `librclone.h` which should be `#include`d in `C` programs wishing to use the library (with some @@ -50,7 +54,7 @@ On Windows, when you build a shared library, you can embed version information as binary resource. To do that you need to run the following command **before** the build command. -``` +```console go run bin/resource_windows.go -binary librclone.dll -dir librclone ``` @@ -75,12 +79,19 @@ The official [C example](#linux-c-example) is targeting Linux/macOS, and will not work on Windows. It is very possible to use `librclone` from a C/C++ application on Windows, but there are some pitfalls that you can avoid by following these guidelines: -- Build `librclone` as shared library, and use run-time dynamic linking (see [linking](#linking)). -- Do not try to unload the library with `FreeLibrary` (see [unloading](#unloading)). -- Deallocate returned strings with API function `RcloneFreeString` (see [memory management](#memory-management)). -- Define struct `RcloneRPCResult`, instead of including `librclone.h` (see [include file](#include-file)). -- Use UTF-8 encoded strings (see [encoding](#encoding)). -- Properly escape JSON strings, beware of the native path separator (see [escaping](#escaping)). + +- Build `librclone` as shared library, and use run-time dynamic linking + (see [linking](#linking)). +- Do not try to unload the library with `FreeLibrary` + (see [unloading](#unloading)). +- Deallocate returned strings with API function `RcloneFreeString` + (see [memory management](#memory-management)). +- Define struct `RcloneRPCResult`, instead of including `librclone.h` + (see [include file](#include-file)). +- Use UTF-8 encoded strings + (see [encoding](#encoding)). +- Properly escape JSON strings, beware of the native path separator + (see [escaping](#escaping)). #### Linking @@ -133,7 +144,7 @@ small struct `RcloneRPCResult`, from [librclone.go](librclone.go): ```c++ struct RcloneRPCResult { char* Output; - int Status; + int Status; }; ``` @@ -174,7 +185,7 @@ constants, leaving escaping only necessary for the contained JSON. ## Example in golang -Here is a go example to help you move files : +Here is a go example to help you move files: ```go func main() { @@ -188,11 +199,10 @@ func main() { if err != nil { fmt.Println(err) } - + out, status: = librclone.RPC("sync/copy", string(syncRequestJSON)) fmt.Println("Got status : %d and output %q", status, out) } - ``` ## gomobile @@ -201,7 +211,9 @@ The `gomobile` subdirectory contains the equivalent of the C binding but suitable for using with [gomobile](https://pkg.go.dev/golang.org/x/mobile/cmd/gomobile) using something like this. - gomobile bind -v -target=android -javapkg=org.rclone github.com/rclone/rclone/librclone/gomobile +```console +gomobile bind -v -target=android -javapkg=org.rclone github.com/rclone/rclone/librclone/gomobile +``` The command generates an Android library (`aar`) that can be imported into an Android application project. Librclone will be contained @@ -233,8 +245,10 @@ Further docs: - [gomobile main website](https://pkg.go.dev/golang.org/x/mobile/cmd/gomobile) - [gomobile wiki](https://github.com/golang/go/wiki/Mobile) -- [go issue #16876](https://github.com/golang/go/issues/16876) where the feature was added -- [gomobile design doc](https://docs.google.com/document/d/1y9hStonl9wpj-5VM-xWrSTuEJFUAxGOXOhxvAs7GZHE/edit) for extra details not in the docs. +- [go issue #16876](https://github.com/golang/go/issues/16876) where the feature + was added +- [gomobile design doc](https://docs.google.com/document/d/1y9hStonl9wpj-5VM-xWrSTuEJFUAxGOXOhxvAs7GZHE/edit) + for extra details not in the docs. ## python @@ -247,19 +261,20 @@ This needs expanding and submitting to pypi... ## Rust -Rust bindings are available in the `librclone` crate: https://crates.io/crates/librclone +Rust bindings are available in the `librclone` crate: ## PHP -The `php` subdirectory contains how to use the C library librclone in php through foreign -function interface (FFI). +The `php` subdirectory contains how to use the C library librclone in php through +foreign function interface (FFI). Useful docs: + - [PHP / FFI](https://www.php.net/manual/en/book.ffi.php) ## TODO -- Async jobs must currently be cancelled manually at the moment - RcloneFinalize doesn't do it. +- Async jobs must currently be cancelled manually at the moment - RcloneFinalize + doesn't do it. - This will use the rclone config system and rclone logging system. - Need examples showing how to configure things, -