Since all URLContexts have the same AVOptions, such AVOptions
will be applied on the outermost context only and removed from the
dict, while they probably make sense on all contexts.
This makes sure that rw_timeout gets propagated to the innermost
URLContext (to make sure it gets passed to the tcp protocol, when
opening a http connection for instance).
Alternatively, such matching options would be kept in the dict
and only removed after the ffurl_connect call.
Signed-off-by: Martin Storsjö <martin@martin.st>
Instead of a linked list constructed at av_register_all(), store them
in a constant array of pointers.
Since no registration is necessary now, this removes some global state
from lavf. This will also allow the urlprotocol layer caller to limit
the available protocols in a simple and flexible way in the following
commits.
The cur_*auth_type variables were set before the http_connect call
prior to 6a463e7fb - their sole purpose is to record the
authentication type used to do the latest request, since parsing
the http response sets the new type in the auth state.
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Previously this logic was only used if the server didn't
respond with Connection: close, but use it even for that case,
if the server response is non-chunked.
Originally the http code has relied on Connection: close to close
the socket when the file/stream is received - the http protocol
code just kept reading from the socket until the socket was closed.
In f240ed18 we added a check for the file size, because some
http servers didn't respond with Connection: close (and wouldn't
close the socket) even though we requested it, which meant that the
http protocol blocked for a long time at the end of files, waiting
for a socket level timeout.
When reading over tls, trying to read at the end of the connection,
when the peer has closed the connection, can produce spurious (but
harmless) warnings. Therefore always voluntarily stop reading when
the specified file size has been received, if not using a chunked
transfer encoding. (For chunked transfers, we already return 0
as soon as we get the chunk header indicating end of stream.)
Signed-off-by: Martin Storsjö <martin@martin.st>
Split return value handling from the actual opening.
Incidentally fixes the https -> http redirect issue reported by
Compn on behalf of rcombs.
CC: libav-stable@libav.org
The icy_metadata_headers string never gets initialized, so,
during the first call to av_strlcatf() in parse_icy(),
strlen() will be called on a pointer to uninitialized memory.
At best this causes some garbage data to be left at the
start of the string.
By initializing icy_metadata_headers to the empty string, the
first call to strlen() will always return 0, so that data is
appended from the start of the string.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Export the metadata as a icy_metadata_packet avoption.
Based on the work of wm4 and Alessandro Ghedini.
Bug-Id: https://bugs.debian.org/739936
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This makes sure we don't send the Except: 100-continue header
if no authentication credentials have been provided.
Signed-off-by: Martin Storsjö <martin@martin.st>
Inspired by a patch by Jakob van Bethlehem. But instead of doing
an empty POST first to trigger the WWW-Authenticate header (which
would succeed if no auth actually was required), add an Expect:
100-continue header, which is meant to be used exactly for
cases like this.
The header is added if doing a post, and the user has specified
authentication but we don't know the auth method yet.
Not all common HTTP servers support the Expect: 100-continue header,
though, so we only try to use it when it really is needed. The user
can request it to be added for other POST requests as well via
an option - which would allow the caller to know immediately that
the POST has failed (e.g. if no auth was provided but the server
required it, or if the target URL simply doesn't exist).
This is only done for write mode posts (e.g. posts without pre-set
post_data) - for posts with pre-set data, we can just redo the post
if it failed due to 401.
Signed-off-by: Martin Storsjö <martin@martin.st>
The default is to autodetect the auth method. This does require one
extra request (and also closing and reopening the http connection).
For some cases such as HTTP POST, the autodetection is not handled
properly (yet).
No option is added for digest, since this method requires getting
nonce parameters from the server first and can't be used straight
away like Basic.
Signed-off-by: Martin Storsjö <martin@martin.st>
When passing a dict to the nested protocol, it will consume
the used options from it, so a separate copy needs to be used
when reopening the connection multiple times.
Signed-off-by: Martin Storsjö <martin@martin.st>
Derived from VLC's http module.
Original authors:
Antoine Cellerier <dionoea@videolan.org>
Sébastien Escudier <sebastien-devel@celeos.eu>
Rémi Duraffort <ivoire@videolan.org>
Rémi Denis-Courmont <remi@remlab.net>
Francois Cartegnie <fcvlcdev@free.fr>
Normally, http servers shouldn't send this to us since we
don't advertise it with an Accept-Encoding header, but some
servers still do it anyway.
Signed-off-by: Martin Storsjö <martin@martin.st>
The handling of the environment variable no_proxy, present since
one of the initial commits (de6d9b6404), is inconsistent with
how many other applications and libraries interpret this
variable. Its bare presence does not indicate that the use of
proxies should be skipped, but it is some sort of pattern for
hosts that does not need using a proxy (e.g. for a local network).
As investigated by Rudolf Polzer, different libraries handle this
in different ways, some supporting IP address masks, some supporting
arbitrary globbing using *, some just checking that the pattern matches
the end of the hostname without regard for whether it actually is
the right domain or a domain that ends in the same string.
This simple logic should be pretty similar to the logic used by
lynx and curl.
Signed-off-by: Martin Storsjö <martin@martin.st>
Use the MAX_URL_SIZE define where applicable. Increase buffer
sizes for all buffers that need to fit a long pathname - buffers
that need to fit only the hostname (and other short strings, but
not the pathname - such as "headers" in http_connect) are kept
at 1024 bytes for now.
Also increase the max line length in http_read_header, since it
might need to contain a full url for Location: redirects.
Signed-off-by: Martin Storsjö <martin@martin.st>
This can happen if doing a new request using the same socket,
but the new request failed, which clears the urlcontext.
Signed-off-by: Martin Storsjö <martin@martin.st>
This fixes issues with opening http urls that have authentication
or redirects, introduced in commit e999b641.
Signed-off-by: Martin Storsjö <martin@martin.st>
