mirror of
https://github.com/rclone/rclone.git
synced 2025-03-17 20:27:52 +02:00
351fc609b1
Before this change when uploading files bigger than 1TiB, the chunk
calculator would work out that the chunk size needed to be bigger than
the default 100 MiB to fit within the 10,000 parts limit.
However the uploader was still using the memory pool for the old chunk
size and this caused errors like
panic: runtime error: slice bounds out of range [:122683392] with capacity 100663296
The fix for this is to make a temporary pool with the larger chunk
size and use it during the upload of the large file.
See: https://forum.rclone.org/t/rclone-cannot-complete-upload-to-b2-restarts-upload-frequently/35617/
Website | Documentation | Download | Contributing | Changelog | Installation | Forum
Rclone
Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers.
Storage providers
- 1Fichier 📄
- Akamai Netstorage 📄
- Alibaba Cloud (Aliyun) Object Storage System (OSS) 📄
- Amazon Drive 📄 (See note)
- Amazon S3 📄
- Backblaze B2 📄
- Box 📄
- Ceph 📄
- China Mobile Ecloud Elastic Object Storage (EOS) 📄
- Cloudflare R2 📄
- Arvan Cloud Object Storage (AOS) 📄
- Citrix ShareFile 📄
- DigitalOcean Spaces 📄
- Digi Storage 📄
- Dreamhost 📄
- Dropbox 📄
- Enterprise File Fabric 📄
- FTP 📄
- Google Cloud Storage 📄
- Google Drive 📄
- Google Photos 📄
- HDFS (Hadoop Distributed Filesystem) 📄
- HiDrive 📄
- HTTP 📄
- Huawei Cloud Object Storage Service(OBS) 📄
- Internet Archive 📄
- Jottacloud 📄
- IBM COS S3 📄
- IONOS Cloud 📄
- Koofr 📄
- Liara Object Storage 📄
- Mail.ru Cloud 📄
- Memset Memstore 📄
- Mega 📄
- Memory 📄
- Microsoft Azure Blob Storage 📄
- Microsoft OneDrive 📄
- Minio 📄
- Nextcloud 📄
- OVH 📄
- OpenDrive 📄
- OpenStack Swift 📄
- Oracle Cloud Storage 📄
- Oracle Object Storage 📄
- ownCloud 📄
- pCloud 📄
- premiumize.me 📄
- put.io 📄
- QingStor 📄
- Qiniu Cloud Object Storage (Kodo) 📄
- Rackspace Cloud Files 📄
- RackCorp Object Storage 📄
- Scaleway 📄
- Seafile 📄
- SeaweedFS 📄
- SFTP 📄
- SMB / CIFS 📄
- StackPath 📄
- Storj 📄
- SugarSync 📄
- Tencent Cloud Object Storage (COS) 📄
- Wasabi 📄
- WebDAV 📄
- Yandex Disk 📄
- Zoho WorkDrive 📄
- The local filesystem 📄
Please see the full list of all storage providers and their features
Virtual storage providers
These backends adapt or modify other storage providers
- Alias: rename existing remotes 📄
- Cache: cache remotes (DEPRECATED) 📄
- Chunker: split large files 📄
- Combine: combine multiple remotes into a directory tree 📄
- Compress: compress files 📄
- Crypt: encrypt files 📄
- Hasher: hash files 📄
- Union: join multiple remotes to work together 📄
Features
- MD5/SHA-1 hashes checked at all times for file integrity
- Timestamps preserved on files
- Partial syncs supported on a whole file basis
- Copy mode to just copy new/changed files
- Sync (one way) mode to make a directory identical
- Check mode to check for file hash equality
- Can sync to and from network, e.g. two different cloud accounts
- Optional large file chunking (Chunker)
- Optional transparent compression (Compress)
- Optional encryption (Crypt)
- Optional FUSE mount (rclone mount)
- Multi-threaded downloads to local disk
- Can serve local or remote files over HTTP/WebDAV/FTP/SFTP/DLNA
Installation & documentation
Please see the rclone website for:
Downloads
License
This is free software under the terms of the MIT license (check the COPYING file included in this package).