mirror of
https://github.com/rclone/rclone.git
synced 2025-01-24 12:56:36 +02:00
Nick Craig-Wood
f7e3115955
s3: fix Wasabi HEAD requests returning stale data by using only 1 transport
In this commit fc5b14b62046b240 s3: Added `--s3-disable-http2` to disable http/2 We created our own transport so we could disable http/2. However the added function is called twice meaning that we create two HTTP transports. This didn't happen with the original code because the default transport is cached by fshttp. Rclone normally does a PUT followed by a HEAD request to check an upload has been successful. With the two transports, the PUT and the HEAD were being done on different HTTP transports. This means that it wasn't re-using the same HTTP connection, so the HEAD request showed the previous object value. This caused rclone to declare the upload was corrupted, delete the object and try again. This patch makes sure we only create one transport and use it for both PUT and HEAD requests which fixes the problem with Wasabi. See: https://forum.rclone.org/t/each-time-rclone-is-run-1-3-fails-2-3-succeeds/22545
Website | Documentation | Download | Contributing | Changelog | Installation | Forum
Rclone
Rclone ("rsync for cloud storage") is a command line program to sync files and directories to and from different cloud storage providers.
Storage providers
- 1Fichier 📄
- Alibaba Cloud (Aliyun) Object Storage System (OSS) 📄
- Amazon Drive 📄 (See note)
- Amazon S3 📄
- Backblaze B2 📄
- Box 📄
- Ceph 📄
- Citrix ShareFile 📄
- DigitalOcean Spaces 📄
- Dreamhost 📄
- Dropbox 📄
- Enterprise File Fabric 📄
- FTP 📄
- GetSky 📄
- Google Cloud Storage 📄
- Google Drive 📄
- Google Photos 📄
- HDFS (Hadoop Distributed Filesystem) 📄
- HTTP 📄
- Hubic 📄
- Jottacloud 📄
- IBM COS S3 📄
- Koofr 📄
- Mail.ru Cloud 📄
- Memset Memstore 📄
- Mega 📄
- Memory 📄
- Microsoft Azure Blob Storage 📄
- Microsoft OneDrive 📄
- Minio 📄
- Nextcloud 📄
- OVH 📄
- OpenDrive 📄
- OpenStack Swift 📄
- Oracle Cloud Storage 📄
- ownCloud 📄
- pCloud 📄
- premiumize.me 📄
- put.io 📄
- QingStor 📄
- Rackspace Cloud Files 📄
- Scaleway 📄
- Seafile 📄
- SFTP 📄
- StackPath 📄
- SugarSync 📄
- Tardigrade 📄
- Tencent Cloud Object Storage (COS) 📄
- Wasabi 📄
- WebDAV 📄
- Yandex Disk 📄
- Zoho WorkDrive 📄
- The local filesystem 📄
Please see the full list of all storage providers and their features
Features
- MD5/SHA-1 hashes checked at all times for file integrity
- Timestamps preserved on files
- Partial syncs supported on a whole file basis
- Copy mode to just copy new/changed files
- Sync (one way) mode to make a directory identical
- Check mode to check for file hash equality
- Can sync to and from network, e.g. two different cloud accounts
- Optional large file chunking (Chunker)
- Optional transparent compression (Compress)
- Optional encryption (Crypt)
- Optional cache (Cache)
- Optional FUSE mount (rclone mount)
- Multi-threaded downloads to local disk
- Can serve local or remote files over HTTP/WebDav/FTP/SFTP/dlna
Installation & documentation
Please see the rclone website for:
Downloads
License
This is free software under the terms of MIT the license (check the COPYING file included in this package).
Description
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files
amazon-driveazure-blobbackblaze-b2cloud-storagedropboxencryptionftpfuse-filesystemgogolanggoogle-cloud-storagegoogle-drivehubiconedriveopenstack-swiftrclones3sftpsyncwebdav
Readme
344 MiB
Languages
Go
98.6%
Shell
0.4%
HTML
0.3%
Python
0.3%
JavaScript
0.2%
Other
0.1%