116 Commits

Author	SHA1	Message	Date
Josh Soref	d0888edc0a	Spelling fixes ... Fix spelling of: above, already, anonymous, associated, authentication, bandwidth, because, between, blocks, calculate, candidates, cautious, changelog, cleaner, clipboard, command, completely, concurrently, considered, constructs, corrupt, current, daemon, dependencies, deprecated, directory, dispatcher, download, eligible, ellipsis, encrypter, endpoint, entrieslist, essentially, existing writers, existing, expires, filesystem, flushing, frequently, hierarchy, however, implementation, implements, inaccurate, individually, insensitive, longer, maximum, metadata, modified, multipart, namedirfirst, nextcloud, obscured, opened, optional, owncloud, pacific, passphrase, password, permanently, persimmon, positive, potato, protocol, quota, receiving, recommends, referring, requires, revisited, satisfied, satisfies, satisfy, semver, serialized, session, storage, strategies, stringlist, successful, supported, surprise, temporarily, temporary, transactions, unneeded, update, uploads, wrapped Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>	2020-10-14 15:21:31 +01:00
Stephen Harris	bbddadbd04	sftp: remember entered password in AskPass mode ... As reported in https://github.com/rclone/rclone/issues/4660#issuecomment-705502792 After switching to a password callback function, if the ssh connection aborts and needs to be reconnected then the user is-reprompted for their password. Instead we now remember the password they entered and just give that back. We do lose the ability for them to correct mistakes, but that's the situation from before switching to callbacks. We keep the benefits of not asking for passwords until the SSH connection succeeds (right known_hosts entry, for example). This required a small refactor of how `f := &Fs{}` was built, so we can store the saved password in the Fs object	2020-10-13 16:53:11 +01:00
Stephen Harris	9e925becb6	sftp: defer asking for user passwords until the SSH connection succeeds ... Issue: 4660 https://github.com/rclone/rclone/issues/4660 Unexpected side effect: a wrong password allows for the user to retry!	2020-10-07 12:01:17 +01:00
Stephen Harris	6dc28ef50a	sftp: Allow user to optionally check server hosts key to add security ... Based on Issue 4087 https://github.com/rclone/rclone/issues/4087 Current behaviour is insecure. If the user specifies this value then we switch to validating the server hostkey and so can detect server changes or MITM-type attacks.	2020-10-06 16:27:42 +01:00
Nick Craig-Wood	41ec712aa9	ftp,sftp: fix docs for usernames ... - factor env.CurrentUser out of backend/sftp - Use env.CurrentUser in ftp and sftp - fix docs to have correct username	2020-09-27 11:44:05 +01:00
Stephen Harris	17acae2b00	sftp: allow cert based auth via optional pubkey ... Discussion at https://forum.rclone.org/t/ssh-certificate-based-authentication-does-not-work/19222 Basically we allow the user to specify their own public key cert rather than letting the SSH client extract the pubkey from the private key. This allows certificate based authentication to work.	2020-09-27 11:10:13 +01:00
buengese	60cc2cba1f	sftp: always convert the checksum to lower case - fixes #4518	2020-09-22 03:15:09 +02:00
aus	b6d3cad70e	sftp: add options for subsystem and server_command - fixes #1801	2020-08-25 21:38:13 +01:00
Nick Craig-Wood	fefcbf60fa	sftp: use the absolute path instead of the relative path ... Before this change rclone used the relative path from the current working directory. It appears that WS FTP doesn't like this and the openssh sftp tool also uses absolute paths which is a good reason for switching to absolute paths. This change reads the current working directory at startup and bases all file requests from there. See: https://forum.rclone.org/t/sftp-ssh-fx-failure-directory-not-found/17436	2020-06-30 16:07:23 +01:00
Nick Craig-Wood	c820576329	fs: define SlowModTime and SlowHash features in the relevant backends	2020-06-30 12:01:36 +01:00
Zac Rubin	f9306218f8	sftp: Fix SSH key PEM loading ... For SSH authentication, `key_pem` should both override `key_file` and not require other SSH authentication methods to be set. Prior to this fix, rclone would attempt to use an ssh-agent when `key_pem` was the only SSH authentication method set. Fixes #4240	2020-06-12 22:46:33 +01:00
Nick Craig-Wood	973e3d6a7b	backends: make sure backends expand ~ and environment vars in file names they use ... See: https://forum.rclone.org/t/relative-path-in-rclone-config-service-account-json/16693	2020-06-03 17:39:08 +01:00
Martin Michlmayr	041b201abd	doc: fix typos throughout docs and code	2020-05-25 11:23:58 +01:00
Martin Michlmayr	4aee962233	doc: fix typos throughout docs and code	2020-05-20 15:54:51 +01:00
calisro	bcbfad1482	sft[: added --sftp-pem-key to support inline key files	2020-05-19 11:55:38 +01:00
Nick Craig-Wood	e4f1e19127	sftp: fix post transfer copies failing with 0 size when using set_modtime=false ... Before this change we early exited the SetModTime call which means we skipped reading the info about the file. This change reads info about the file in the SetModTime call even if we are skipping setting the modtime. See: https://forum.rclone.org/t/sftp-and-set-modtime-false-error/16362	2020-05-14 17:30:01 +01:00
Nick Craig-Wood	db39adeb3e	sftp: open files for update write only to fix AWS SFTP interop - fixes #3776	2020-01-18 11:46:56 +00:00
Nick Craig-Wood	5f822f2660	sftp: fix "failed to parse private key file: ssh: not an encrypted key" error ... This error started happening after updating golang/x/crypto which was done as a side effect of: 3801b8109 vendor: update termbox-go to fix ncdu command on FreeBSD This turned out to be a deliberate policy of making ssh.ParsePrivateKeyWithPassphrase fail if the passphrase was empty. See: https://go-review.googlesource.com/c/crypto/+/207599 This fix calls ssh.ParsePrivateKey if the passphrase is empty and ssh.ParsePrivateKeyWithPassphrase otherwise which fixes the problem.	2020-01-13 11:05:16 +00:00
Nick Craig-Wood	a7d65bd519	sftp: add --sftp-skip-links to skip symlinks and non regular files - fixes #3716 ... This also corrects the symlink detection logic to only check symlink files. Previous to this it was checking all directories too which was making it do more stat calls than was necessary.	2019-11-24 16:10:53 +00:00
Sebastian Brandt	f158a398f3	sftp: Retry Creation of Connection - fixes #3656 ... Removes the existing rate limiter because it is implicitly included in the pacer.	2019-11-14 12:50:01 +00:00
Carlos Ferreyra	9cb549a227	sftp: include more ciphers with use_insecure_cipher	2019-10-17 14:58:31 +01:00
Nick Craig-Wood	b90e4a8769	sftp: fix hashes of files with backslashes	2019-09-30 22:00:24 +01:00
Nick Craig-Wood	3cff258577	sftp: fix --sftp-ask-password trying to contact the ssh agent ... See: https://forum.rclone.org/t/rclone-command-line/11766	2019-09-16 11:16:27 +01:00
Nick Craig-Wood	fa539b9d9b	sftp: save the md5/sha1 command in use to the config file	2019-08-14 20:54:08 +01:00
Nick Craig-Wood	3ea82032e7	sftp: support md5/sha1 with rsync.net #3254 ... rsync.net uses the freebsd equivalent of sha1sum and md5sum so adapt to that.	2019-08-14 20:54:08 +01:00
Nick Craig-Wood	57d5de6fba	build: fix up package paths after repo move ... git grep -l github.com/ncw/rclone | xargs -d'\n' perl -i~ -lpe 's|github.com/ncw/rclone|github.com/rclone/rclone|g' goimports -w `find . -name \*.go`	2019-07-28 18:47:38 +01:00
Yi FU	0a1169e659	ssh: opt-in support for diffie-hellman-group-exchange-sha256 diffie-hellman-group-exchange-sha1 - fixes #1810	2019-07-13 12:21:56 +02:00
Jon Fautley	cd762f04b8	sftp: Completely ignore all modtime checks if SetModTime=false	2019-06-28 10:33:14 +01:00
Nick Craig-Wood	a1840f6fc7	sftp: add missing interface check and fix About #3257 ... This bug was introduced as part of adding context to the backends and slipped through the net because the About call did not have an interface assertion in the sftp backend. I checked there were no other missing interface assertions on all the optional methods on all the backends.	2019-06-26 16:56:33 +01:00
Aleksandar Jankovic	f78cd1e043	Add context propagation to rclone ... - Change rclone/fs interfaces to accept context.Context - Update interface implementations to use context.Context - Change top level usage to propagate context to lover level functions Context propagation is needed for stopping transfers and passing other request-scoped values.	2019-06-19 11:59:46 +01:00
Nick Craig-Wood	5c0e5b85f7	Factor ShellExpand from sftp backend to lib/env	2019-05-11 23:39:04 +01:00
Nick Craig-Wood	edda6d91cd	Use go-homedir to read the home directory more reliably	2019-05-11 23:39:04 +01:00
Nick Craig-Wood	ada8c22a97	sftp: send custom client version and debug server version	2019-05-11 23:39:04 +01:00
Nick Craig-Wood	610466c18c	sftp: fix about parsing of df results so it can cope with -ve results ... This is useful when interacting with "serve sftp" which returns -ve results when the corresponding value is unknown.	2019-05-11 23:39:04 +01:00
Gary Kim	49d7b0d278	sftp: add About support - fixes #3107 ... This adds support for using About with SFTP remotes. This works by calling the df command remotely.	2019-04-25 10:51:15 +01:00
Nick Craig-Wood	16d8014cbb	build: drop support for go1.8	2019-04-15 21:49:58 +01:00
Alexandru Bumbacea	4fddec113c	sftp: allow custom ssh client config	2019-02-26 16:37:54 +01:00
Nick Craig-Wood	be643ecfbc	sftp: don't error on dangling symlinks	2019-02-01 16:43:26 +00:00
Fabian Möller	1433558c01	sftp: perform environment variable expansion on key-file	2019-01-09 10:11:33 +01:00
Fabian Möller	0458b961c5	sftp: add option to force the usage of an ssh-agent ... Also adds the possibility to specify a specific key to request from the ssh-agent.	2019-01-09 10:11:33 +01:00
Fabian Möller	c1998c4efe	sftp: add support for PEM encrypted private keys	2019-01-09 10:11:33 +01:00
Nick Craig-Wood	26b5f55cba	Update after goimports change	2018-12-04 10:11:57 +00:00
Nick Craig-Wood	0eba88bbfe	sftp: check directory is empty before issuing rmdir ... Some SFTP servers allow rmdir on full directories which is allowed according to the RFC so make sure we don't accidentally delete data here. See: https://forum.rclone.org/t/rmdir-and-delete-empty-src-dirs-file-does-not-exist/7737	2018-12-02 11:16:30 +00:00
Nick Craig-Wood	4b15c4215c	sftp: fix rmdir on Windows based servers (eg CrushFTP) ... Before this change we used Remove to remove directories. This works fine on Unix based systems but not so well on Windows based ones. Swap to using RemoveDirectory instead.	2018-11-29 21:34:37 +00:00
Jon Fautley	dedc7d885c	sftp: Ensure file hash checking is really disabled	2018-10-23 12:03:50 +01:00
Nick Craig-Wood	a9273c5da5	docs: move documentation for options from docs/content into backends ... In the following commit, the documentation will be autogenerated.	2018-10-06 11:47:46 +01:00
Nick Craig-Wood	f3f48d7d49	Implement new backend config system ... This unifies the 3 methods of reading config * command line * environment variable * config file And allows them all to be configured in all places. This is done by making the []fs.Option in the backend registration be the master source of what the backend options are. The backend changes are: * Use the new configmap.Mapper parameter * Use configstruct to parse it into an Options struct * Add all config to []fs.Option including defaults and help * Remove all uses of pflag * Remove all uses of config.FileGet	2018-07-16 21:20:47 +01:00
Jasper Lievisse Adriaanse	ed4f1b2936	sftp: fix typo in help text	2018-07-08 11:01:35 +01:00
Nick Craig-Wood	d4213c0ac5	sftp: Fix slow downloads for long latency connections - fixes #1158 ... This was caused by using the sftp.File.Read method which resets the streaming window after each call. Replacing it with sftp.File.WriteTo and an io.Pipe fixes the problem bringing the speed to the same as the sftp binary.	2018-05-24 15:10:28 +01:00
Nick Craig-Wood	512f4b4487	Update error checking on fmt.Fprint* after errcheck update ... Now we need to check or ignore errors on fmt.Fprint* explicitly - previously errcheck just ignored them for us.	2018-05-22 09:41:13 +01:00