Fix time elapsed (#1277)

* Show the actual time elapsed; add docs

* `requests.Response._headers_parsed_at` → `requests.Response._httpie_headers_parsed_at`

* Add `ELAPSED_TIME_LABEL` constant

* Tweak docs

* Tweak docs

* Allow multiple blank lines in Markdown files

* Add rudimentary tests for --meta with different --style’s

* Cleanup tests

* Cleanup tests

* Cleanup tests

docs/README.md

@@ -538,7 +538,7 @@ and URL parameters. This is a very practical way of constructing
 HTTP requests from scratch on the CLI.
 
 Each *request item* is simply a key/value pair separated with the following
-characters: `:` (headers), `=` (data field, e.g JSON, Form), `:=` (raw data field)
+characters: `:` (headers), `=` (data field, e.g., JSON, form), `:=` (raw data field)
 `==` (query parameters), `@` (file upload).
 
 ```bash
@@ -550,7 +550,7 @@ $ http PUT pie.dev/put \
 ```
 
 |                                                    Item Type | Description                                                                                                                                                                                                            |
-| -----------------------------------------------------------: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+|-------------------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 |                                    HTTP Headers `Name:Value` | Arbitrary HTTP header, e.g. `X-API-Token:123`                                                                                                                                                                          |
 |                                 URL parameters `name==value` | Appends the given name/value pair as a querystring parameter to the URL. The `==` separator is used.                                                                                                                   |
 |                                    Data Fields `field=value` | Request data fields to be serialized as a JSON object (default), to be form-encoded (with `--form, -f`), or to be serialized as `multipart/form-data` (with `--multipart`)                                             |
@@ -570,7 +570,7 @@ to pass the desired value from a file.
 $ http POST pie.dev/post \
     X-Data:@files/text.txt             # Read a header from a file
     token==@files/text.txt             # Read a query parameter from a file
-    name=@files/text.txt               # Read a data field's value from a file
+    name=@files/text.txt               # Read a data field’s value from a file
     bookmarks:=@files/data.json        # Embed a JSON object from a file
 ```
 
@@ -681,11 +681,11 @@ Other JSON types, however, are not allowed with `--form` or `--multipart`.
     "search": {
         "id": 1,
         "type": "id"
-    }
+    }
 }
 ```
 
-Building arrays is also possible, through `[]` suffix (an append operation). This tells HTTPie to create an array in the given path (if there is not one already), and append the given value to that array.
+Building arrays is also possible, through `[]` suffix (an append operation). This tells HTTPie to create an array in the given path (if there is not one already), and append the given value to that array.
 
 ```bash
 $ http --offline --print=B pie.dev/post \
@@ -696,7 +696,7 @@ $ http --offline --print=B pie.dev/post \
 ```
 
 ```json
-{
+{
     "category": "tools",
     "search": {
         "keywords": [
@@ -807,7 +807,7 @@ $ http --offline --print=B pie.dev/post \
 
 ```bash
 $ http --offline --print=B pie.dev/post \
-  'foo\[bar\]:=1' \
+  'foo\[bar\]:=1' \
   'baz[\[]:=2' \
   'baz[\]]:=3'
 ```
@@ -845,7 +845,7 @@ $ http --offline --print=B pie.dev/post \
   'object[\1]=stringified' \
   'object[\100]=same' \
   'array[1]=indexified'
-```
+```
 
 ```json
 {
@@ -903,8 +903,8 @@ You can follow to given instruction (adding a `]`) and repair your expression.
 ```
 
 ```json
-{
+{
-    "user": {
+    "user": {
         "name": "string"
     }
 }
@@ -1010,7 +1010,7 @@ world
 
 ```bash
 $ http --form --multipart --offline example.org hello=world Content-Type:multipart/letter
-```
+```
 
 ```http
 POST / HTTP/1.1
@@ -1104,7 +1104,7 @@ To send a header with an empty value, use `Header;`, with a semicolon:
 
 ```bash
 http --offline example.org Numbers:one,two
-```
+```
 
 ```http
 GET / HTTP/1.1
@@ -1174,7 +1174,7 @@ $ nc pie.dev 80 < request.http
 Cookie: sessionid=foo
 Host: pie.dev
 User-Agent: HTTPie/0.9.9
-```
+```
 
 Send multiple cookies (note: the header is quoted to prevent the shell from interpreting the `;`):
 
@@ -1439,14 +1439,14 @@ message is printed (headers as well as the body). You can control what should
 |       `m` | response meta    |
 
 Print request and response headers:
-
+
 ```bash
 $ http --print=Hh PUT pie.dev/put hello=world
 ```
 
 #### Response meta
 
-`--verbose` can often be useful for debugging the request and generating documentation examples:
+The response metadata section currently includes the total time elapsed. It’s the number of seconds between opening the network connection and downloading the last byte of response the body.
 
 Please note that it also includes time spent on formatting the output, which adds a small penalty. Also, if the body is not part of the output, we don’t spend time downloading it — please see [conditional body download](#conditional-body-download).
 
@@ -1454,7 +1454,7 @@ All the other [output options](#output-options) are under the hood just shortcut
 
 
 ### Verbose output
-Host: pie.dev
+
 `--verbose` can often be useful for debugging the request and generating documentation examples:
 
 ```bash
@@ -1467,6 +1467,15 @@ Print request and response headers:
 User-Agent: HTTPie/0.2.7dev
 
 {
+    "hello": "world"
+}
+
+HTTP/1.1 200 OK
+Connection: keep-alive
+Content-Length: 477
+Content-Type: application/json
+Date: Sun, 05 Aug 2012 00:25:23 GMT
+Server: gunicorn/0.13.4
 
 {
     […]
@@ -1628,7 +1637,7 @@ On macOS, you can send the contents of the clipboard with `pbpaste`:
 ```
 
 ### Request data from a filename
-
+
 An alternative to redirected `stdin` is specifying a filename (as `@/path/to/file`) whose content is used as if it came from `stdin`.
 
 It has the advantage that the `Content-Type` header is automatically set to the appropriate value based on the filename extension.
@@ -1712,11 +1721,11 @@ You can choose your preferred color scheme via the `--style` option if you don
 
 ```bash
 $ http --response-mime=text/yaml pie.dev/get
-```
+```
 
 Formatting has the following effects:
 
-- HTTP headers are sorted by name.
+- HTTP headers are sorted by name.
 - JSON data is indented, sorted by keys, and unicode escapes are converted
   to the characters they represent.
 - XML and XHTML data is indented.
@@ -1725,7 +1734,7 @@ There are dozens of styles available, here are just a few notable ones:
 collapsing empty tags on XML) but the end result will always be semantically indistinguishable. Some of
 these formatting changes can be configured more granularly through [format options](#format-options).
 
-### Format options
+### Format options
 
 The `--format-options=opt1:value,opt2:value` option allows you to control how the output should be formatted
 when formatting is applied. The following options are available:
@@ -1744,7 +1753,7 @@ Formatting has the following effects:
 
 ```bash
 $ http --format-options headers.sort:false,json.sort_keys:false,json.indent:2 pie.dev/get
-```
+```
 
 There are also two shortcuts that allow you to quickly disable and re-enable
 sorting-related format options (currently it means JSON keys and headers):
@@ -1754,7 +1763,7 @@ The `--format-options=opt1:value,opt2:value` option allows you to control how th
 
 ### Redirected output
 
-HTTPie uses a different set of defaults for redirected output than for [terminal output](#terminal-output).
+HTTPie uses a different set of defaults for redirected output than for [terminal output](#terminal-output).
 The differences being:
 
 - Formatting and colors aren’t applied (unless `--pretty` is specified).
@@ -1903,7 +1912,7 @@ $ http -dco file.zip example.org/file
 Streamed output by small chunks à la `tail -f`:
 
 ```bash
-# Send each new line (JSON object) to another URL as soon as it arrives from a streaming API:
+# Send each new line (JSON object) to another URL as soon as it arrives from a streaming API:
 $ http --stream pie.dev/stream/3 | while read line; do echo "$line" | http pie.dev/post ; done
 ```
 
@@ -1982,7 +1991,7 @@ $ http --session=user2 -a user2:password pie.dev/get X-Bar:Foo
 $ http --session=~/.config/httpie/sessions/another.example.org/test.json example.org
 ```
 
-When creating anonymous sessions, please remember to always include at least one `/`, even if the session files is located in the current directory (i.e. `--session=./session.json` instead of just `--session=session.json`), otherwise HTTPie assumes a named session instead.
+When creating anonymous sessions, please remember to always include at least one `/`, even if the session files is located in the current directory (i.e. `--session=./session.json` instead of just `--session=session.json`), otherwise HTTPie assumes a named session instead.
 
 ### Readonly session
 
@@ -2078,7 +2087,7 @@ If the server expires an existing cookie, it will also be removed from the sessi
 #### `default_options`
 
 An `Array` (by default empty) of default options that should be applied to every invocation of HTTPie.
-
+
 For instance, you can use this config option to change your default color theme:
 
 ```bash
@@ -2120,7 +2129,7 @@ $ cat ~/.config/httpie/config.json
     case $? in
         2) echo 'Request timed out!' ;;
         3) echo 'Unexpected HTTP 3xx Redirection!' ;;
-        4) echo 'HTTP 4xx Client Error!' ;;
+        4) echo 'HTTP 4xx Client Error!' ;;
         5) echo 'HTTP 5xx Server Error!' ;;
         6) echo 'Exceeded --max-redirects=<n> redirects!' ;;
         *) echo 'Other Error!' ;;
@@ -2185,11 +2194,11 @@ This command is currently in beta.
 List all installed plugins.
 
 ```bash
-$ httpie plugins list
+$ httpie plugins list
-httpie_plugin (1.0.2)
+httpie_plugin (1.0.2)
   httpie_plugin (httpie.plugins.auth.v1)
 httpie_plugin_2 (1.0.6)
-  httpie_plugin_2 (httpie.plugins.auth.v1)
+  httpie_plugin_2 (httpie.plugins.auth.v1)
 httpie_converter (1.0.0)
   httpie_iterm_converter (httpie.plugins.converter.v1)
   httpie_konsole_konverter (httpie.plugins.converter.v1)
@@ -2232,7 +2241,7 @@ $ httpie plugins upgrade httpie-plugin
 ```
 
 with the HTTPie command that sends it:
-
+
 ```bash
 $ http -f POST pie.dev/post \
     X-API-Key:123 \

docs/markdownlint.rb

@@ -20,6 +20,9 @@ exclude_rule 'MD014'
 # MD028 Blank line inside blockquote
 exclude_rule 'MD028'
 
+# MD012 Multiple consecutive blank lines
+exclude_rule 'MD012'
+
 # Tell the linter to use ordered lists:
 #   1. Foo
 #   2. Bar

httpie/client.py

@@ -3,6 +3,7 @@ import http.client
 import json
 import sys
 from contextlib import contextmanager
+from time import monotonic
 from typing import Any, Dict, Callable, Iterable
 from urllib.parse import urlparse, urlunparse
 
@@ -108,7 +109,7 @@ def collect_messages(
                     **send_kwargs_merged,
                     **send_kwargs,
                 )
-
+            response._httpie_headers_parsed_at = monotonic()
             expired_cookies += get_expired_cookies(
                 response.headers.get('Set-Cookie', '')
             )

httpie/models.py

@@ -1,3 +1,5 @@
+from time import monotonic
+
 import requests
 
 from enum import Enum, auto
@@ -15,6 +17,9 @@ from .compat import cached_property
 from .utils import split_cookies, parse_content_type_header
 
 
+ELAPSED_TIME_LABEL = 'Elapsed time'
+
+
 class HTTPMessage:
     """Abstract class for HTTP messages."""
 
@@ -96,7 +101,13 @@ class HTTPResponse(HTTPMessage):
     @property
     def metadata(self) -> str:
         data = {}
-        data['Elapsed time'] = str(self._orig.elapsed.total_seconds()) + 's'
+        time_to_parse_headers = self._orig.elapsed.total_seconds()
+        # noinspection PyProtectedMember
+        time_since_headers_parsed = monotonic() - self._orig._httpie_headers_parsed_at
+        time_elapsed = time_to_parse_headers + time_since_headers_parsed
+        # data['Headers time'] = str(round(time_to_parse_headers, 5)) + 's'
+        # data['Body time'] = str(round(time_since_headers_parsed, 5)) + 's'
+        data[ELAPSED_TIME_LABEL] = str(round(time_elapsed, 10)) + 's'
         return '\n'.join(
             f'{key}: {value}'
             for key, value in data.items()

httpie/output/formatters/colors.py

@@ -383,4 +383,5 @@ def make_styles():
 
 
 PIE_STYLES = make_styles()
+PIE_STYLE_NAMES = list(PIE_STYLES.keys())
 BUNDLED_STYLES |= PIE_STYLES.keys()

httpie/output/lexers/metadata.py

@@ -1,4 +1,6 @@
 import pygments
+
+from httpie.models import ELAPSED_TIME_LABEL
 from httpie.output.lexers.common import precise
 
 SPEED_TOKENS = {
@@ -34,7 +36,7 @@ class MetadataLexer(pygments.lexer.RegexLexer):
     tokens = {
         'root': [
             (
-                r'(Elapsed time)( *)(:)( *)(\d+\.\d+)(s)', pygments.lexer.bygroups(
+                fr'({ELAPSED_TIME_LABEL})( *)(:)( *)(\d+\.\d+)(s)', pygments.lexer.bygroups(
                     pygments.token.Name.Decorator,  # Name
                     pygments.token.Text,
                     pygments.token.Operator,  # Colon

httpie/output/ui/palette.py

@@ -1,7 +1,12 @@
-# Copy the brand palette
 from typing import Optional
 
+STYLE_PIE = 'pie'
+STYLE_PIE_DARK = 'pie-dark'
+STYLE_PIE_LIGHT = 'pie-light'
+
+
 COLOR_PALETTE = {
+    # Copy the brand palette
     'transparent': 'transparent',
     'current': 'currentColor',
     'white': '#F5F5F0',
@@ -138,10 +143,11 @@ COLOR_PALETTE['primary'] = {
 
 COLOR_PALETTE['secondary'] = {'700': '#37523C', '600': '#6c6969', '500': '#6c6969'}
 
+
 SHADE_NAMES = {
-    '500': 'pie-dark',
+    '500': STYLE_PIE_DARK,
-    '600': 'pie',
+    '600': STYLE_PIE,
-    '700': 'pie-light'
+    '700': STYLE_PIE_LIGHT
 }
 
 SHADES = [

tests/test_meta.py

@@ -1,7 +1,17 @@
-from .utils import http
+import pytest
+
+from httpie.models import ELAPSED_TIME_LABEL
+from httpie.output.formatters.colors import PIE_STYLE_NAMES
+from .utils import http, MockEnvironment, COLOR
 
 
-def test_meta_elapsed_time(httpbin, monkeypatch):
+def test_meta_elapsed_time(httpbin):
-    r = http('--meta', httpbin + '/get')
+    r = http('--meta', httpbin + '/delay/1')
-    for line in r.splitlines():
+    assert f'{ELAPSED_TIME_LABEL}: 1.' in r
-        assert 'Elapsed time' in r
+
+
+@pytest.mark.parametrize('style', ['auto', 'fruity', *PIE_STYLE_NAMES])
+def test_meta_elapsed_time_colors(httpbin, style):
+    r = http('--style', style, '--meta', httpbin + '/get', env=MockEnvironment(colors=256))
+    assert COLOR in r
+    assert ELAPSED_TIME_LABEL in r

tests/test_output.py

@@ -17,7 +17,7 @@ from httpie.cli.argtypes import (
 )
 from httpie.cli.definition import parser
 from httpie.encoding import UTF8
-from httpie.output.formatters.colors import PIE_STYLES, get_lexer
+from httpie.output.formatters.colors import get_lexer, PIE_STYLE_NAMES
 from httpie.status import ExitStatus
 from .fixtures import XML_DATA_RAW, XML_DATA_FORMATTED
 from .utils import COLOR, CRLF, HTTP_OK, MockEnvironment, http, DUMMY_URL
@@ -227,7 +227,7 @@ def test_ensure_contents_colored(httpbin, endpoint):
     assert COLOR in r
 
 
-@pytest.mark.parametrize('style', PIE_STYLES.keys())
+@pytest.mark.parametrize('style', PIE_STYLE_NAMES)
 def test_ensure_meta_is_colored(httpbin, style):
     env = MockEnvironment(colors=256)
     r = http('--meta', '--style', style, 'GET', httpbin + '/get', env=env)

tests/utils/matching/test_matching.py

@@ -2,6 +2,7 @@
 Here we test our output parsing and matching implementation, not HTTPie itself.
 
 """
+from httpie.models import ELAPSED_TIME_LABEL
 from httpie.output.writer import MESSAGE_SEPARATOR
 from ...utils import CRLF
 from . import assert_output_does_not_match, assert_output_matches, Expect
@@ -111,7 +112,7 @@ def test_assert_output_matches_response_meta():
     assert_output_matches(
         (
             'Key: Value\n'
-            'Elapsed Time: 3.3s'
+            f'{ELAPSED_TIME_LABEL}: 3.3s'
         ),
         [Expect.RESPONSE_META]
     )
@@ -124,7 +125,7 @@ def test_assert_output_matches_whole_response():
             f'AAA:BBB{CRLF}'
             f'{CRLF}'
             f'CCC{MESSAGE_SEPARATOR}'
-            'Elapsed Time: 3.3s'
+            f'{ELAPSED_TIME_LABEL}: 3.3s'
         ),
         [Expect.RESPONSE_HEADERS, Expect.BODY, Expect.RESPONSE_META]
     )