1
0
mirror of https://github.com/BurntSushi/ripgrep.git synced 2024-12-12 19:18:24 +02:00
Commit Graph

854 Commits

Author SHA1 Message Date
Andrew Gallant
96ee4482cd globset: remove use of unsafe
This commit removes, in retrospect, a silly use of `unsafe`. In particular,
to extract a file name extension (distinct from how `std` implements it),
we were transmuting an OsStr to its underlying WTF-8 byte representation
and then searching that. This required `unsafe` and relied on an
undocumented std API, so it was a bad choice to make, but everything gets
sacrificed at the Alter of Performance.

The thing I didn't seem to realize at the time was that:

  1. On Unix, you can already get the raw byte representation in a manner
     that has zero cost.
  2. On Windows, paths are already being encoded and copied every which
     way. So doing a UTF-8 check and, in rare cases (for invalid UTF-8),
     an extra copy, doesn't seem like that much more of an added expense.

Thus, rewrite the extension extraction using safe APIs. On Unix, this
should have identical performance characteristics as the previous
implementation. On Windows, we do pay a higher cost in the UTF-8
check, but Windows is already paying a similar cost a few times over
anyway.
2018-02-10 22:28:12 -05:00
Andrew Gallant
3effea0b7c doc: add color FAQ entries
This commit adds FAQ entries about how to configure ripgrep's coloring,
and how to get true color support in Windows consoles.
2018-02-10 22:28:12 -05:00
Andrew Gallant
2d68054b1d termcolor: support ANSI in Windows terminals
This commit uses the new virtual terminal processing feature in Windows 10
to enable the use of ANSI escape codes to color ripgrep's output. This
technique is preferred over the console APIs because it seems like where
the future is heading, but also because it avoids needing to use an
intermediate buffer to deal with the Windows console in a multithreaded
environment.
2018-02-10 22:28:12 -05:00
Andrew Gallant
65a63788bc wincolor: add support for enabling VT100
This commit adds a new method to the Console type which permits toggling
the VIRTUAL_TERMINAL_PROCESSING mode on a console. Specifically, this
enables the use of ANSI escape sequences for color in Windows terminals.
2018-02-10 22:28:12 -05:00
Andrew Gallant
7e5589f07d termcolor: permit hex colors
This commit adds support for specifying Ansi256 or RGB colors using
hexadecimal notation.
2018-02-10 22:28:12 -05:00
Andrew Gallant
09c5b2c4ea ci: update deployment for doc rearrangement
This fixes CI to handle the new documentation files. We also continue to
do more cleanup. In particular, we devise a nicer way of detecting the
most recent Cargo OUT_DIR by writing a dummy file, and looking for the
most recently modified version of that file.
2018-02-10 12:12:47 -05:00
Andrew Gallant
904c75bd30 doc: overhaul documentation
This commit cleans up the README and splits portions of it out into
a user guide (GUIDE.md) and a FAQ (FAQ.md). The README now provides a
small list of documentation "quick" links to various parts of the docs.

This commit also does a few other minor touchups.
2018-02-10 12:12:47 -05:00
Andrew Gallant
ca3e0e8a49 doc: clarify --files-without-match
This adds a couple common keywords to the documentation.

Fixes #779
2018-02-10 12:12:47 -05:00
Andrew Gallant
ae2d036dd4 build: remove compile script
This script has only ever intended to be a convenience to me to compile
ripgrep. It is otherwise a distraction, so remove it.
2018-02-10 12:12:47 -05:00
Andrew Gallant
8e93fa0e7f
deps: update regex to 0.2.6
This regex update disabled the Tuned Boyer-Moore literal searcher which
has a bug in it that isn't straight-forward to fix. We bring that update
into ripgrep with this commit.

Fixes #780, Fixes #781
2018-02-08 18:25:55 -05:00
Andrew Gallant
7f5c07434b
argv: add several hidden flags
This adds hidden counter-flags for the following:

  -L/--follow [--no-follow]
  --hidden [--no-hidden]
  --no-ignore [--ignore]
  --no-ignore-parent [--ignore-parent]
  --no-ignore-vcs [--ignore-vcs]
  --no-messages [--messages]
  --search-zip [--no-search-zip]
  --sort-files [--no-sort-files]
  --text [--no-text]

In the above list, the counter-flags are in brackets.

While these flags are hidden, we document the counter-flags in the
documentation for the flags they are countering.
2018-02-06 18:28:39 -05:00
Andrew Gallant
874f0b96a6
argv: support hidden flags
This commit adds support for hidden flags. The purpose of hidden flags
is for things that end users likely won't need unless they have a
configuration file that disables ripgrep's defaults. These flags will
provide a way to re-enable ripgrep's defaults.
2018-02-06 18:26:23 -05:00
Behnam Esfahbod
706323ad8f
globset: add more tests for single-asterisk pattern
This adds a few tests that check for bugs reported here:
https://github.com/rust-lang/cargo/issues/4268

The bugs reported in the aforementioned issue are probably caused by not
enabling the `literal_separator` option in `GlobBuilder`. Enabling that
in the tests under question fixes the issue.

Closes #773
2018-02-06 17:42:22 -05:00
Andrew Gallant
8460d7fe3d
ci: fix deploy condition
The previous setting was for debugging that accidentally got merged.
2018-02-06 17:27:57 -05:00
Andrew Gallant
e1f1ede17d ci: test build outputs
This modifies CI to check that we generate completion files and a man
page.

We also enable tests on arm.
2018-02-06 17:24:31 -05:00
Andrew Gallant
6553940328 doc: generate man page
This commit uses the recent refactoring for defining flags to
automatically generate a man page. This finally allows us to define the
documentation for each flag in a single place.

The man page is generated on every build, if and only if `asciidoc` is
installed. When generated, it is placed in Cargo's `OUT_DIR` directory,
which is the same place that shell completions live.
2018-02-06 12:07:59 -05:00
Andrew Gallant
b50ae9a99c ci: cleanup
This cleans up our CI scripts but doesn't significantly change anything.
Mostly this is removing dead code and wrong comments, and making the style
a bit more consistent.
2018-02-06 12:07:59 -05:00
Andrew Gallant
224c112e05 argv: tweak the meaning of zero
This commit makes a small tweak to the --max-columns flag. Namely, if
the value of the flag is 0, then ripgrep behaves as-if the flag were
absent.

This is useful in the context of ripgrep reading configuration from the
environment. For example, an end user might set --max-columns=150, but we
should permit the user to disable this setting when needed. Using -M0 is
a nice way to do that.

We do this because a zero value for --max-columns isn't particularly
meaningful. We do leave the --max-count, --max-filesize and --maxdepth
flags alone though, since a zero value for those flags is potentially
meaningful. (--max-count even has tests for ripgrep's behavior when
given a value of 0.)
2018-02-06 12:07:59 -05:00
Andrew Gallant
8cb5833ef9 argv: update clap to 2.29.4
We use the new AppSettings::AllArgsOverrideSelf to permit all flags to
be specified multiple times. This removes the need for our previous
work-around where we would enable `multiple` for every flag and then
just extract the last value when consuming clap's matches.

We also add a couple regression tests that ensure repeated switches and
flags work as expected.
2018-02-06 12:07:59 -05:00
Kevin K
85cd3f0a6e argv: fix PATTERN typo
When referencing the PATTERN positional argument,
we should use `pattern` and not `PATTERN`. The former
is the clap identifier name while the latter is the argument
value name.
2018-02-05 11:37:17 -05:00
Andrew Gallant
c57d0fb4e8 config: add persistent configuration
This commit adds support for reading configuration files that change
ripgrep's default behavior. The format of the configuration file is an
"rc" style and is very simple. It is defined by two rules:

  1. Every line is a shell argument, after trimming ASCII whitespace.
  2. Lines starting with '#' (optionally preceded by any amount of
     ASCII whitespace) are ignored.

ripgrep will look for a single configuration file if and only if the
RIPGREP_CONFIG_PATH environment variable is set and is non-empty.
ripgrep will parse shell arguments from this file on startup and will
behave as if the arguments in this file were prepended to any explicit
arguments given to ripgrep on the command line.

For example, if your ripgreprc file contained a single line:

    --smart-case

then the following command

    RIPGREP_CONFIG_PATH=wherever/.ripgreprc rg foo

would behave identically to the following command

    rg --smart-case foo

This commit also adds a new flag, --no-config, that when present will
suppress any and all support for configuration. This includes any future
support for auto-loading configuration files from pre-determined paths
(which this commit does not add).

Conflicts between configuration files and explicit arguments are handled
exactly like conflicts in the same command line invocation. That is,
this command:

    RIPGREP_CONFIG_PATH=wherever/.ripgreprc rg foo --case-sensitive

is exactly equivalent to

    rg --smart-case foo --case-sensitive

in which case, the --case-sensitive flag would override the --smart-case
flag.

Closes #196
2018-02-04 10:40:20 -05:00
Andrew Gallant
d83bab4d3f argv: permit repeated flags
This commit builds on the previous argv refactor by being more principled
about how we declared our flags. In particular, we now require that every
clap argument is one of three things: a positional argument, a switch or
a flag that accepts exactly one value. The latter two are always permitted
to be repeated, and we modify the code that consumes a clap::ArgMatches to
always use the *last* value of an argument. (clap by default always uses
the first value of argument, if it has been repeated and is accessed via
one of the singleton accessors.)

Fixes #553
2018-02-04 10:40:20 -05:00
Andrew Gallant
ce84e1ef04 argv: refactor clap initialization
This commit refactors how we define flags. In theory, this commit
should not result in any behavioral changes (other than perhaps more
consistent rules for flag overrides).

There are two important changes:

Firstly, each flag (or tightly coupled group of flags) is defined in its
own function. This function includes the documentation for that flag. This
improves the locality for each flag; everything you need to know about it
is self-contained in one small region of code.

Secondly, each flag is defined in terms of a very small ripgrep-specific
layer above clap. This permits us to have a set of structured arguments
independent of clap. The intention here is that we can use this indirection
to generate other documentation such as man pages.

This commit lays the ground work for modifying our use of clap in
principled way such that flags can be specified multiple times without
conflict. This in turn will help us implement support for persistent
configuration.
2018-02-04 10:40:20 -05:00
Andrew Gallant
c8e755f11f deps: remove vec-map feature from clap
This removes the vec-map feature from clap. clap's README claims that
vec-map provides a small performance benefit, but I could observe any in
ripgrep workloads.

The benefit here is that it drops a dependency.

Amazingly, this drops whole release build times for ripgrep from 68s to
33s, and debug build time also decreases from 22s to 15.5s. This was
entirely unintentional but a welcome surprise.
2018-02-04 10:40:20 -05:00
Andrew Gallant
68dac7c4b0 build: add git hash
This commit makes the git hash ripgrep was built with available for use
in the version string.

We also do a few minor touchups in build.rs and src/app.rs.
2018-02-04 10:40:20 -05:00
Andrew Gallant
3535047094 logger: drop env_logger
This commit updates the `log` crate to 0.4 and drops the dependency on
env_logger. In particular, the latest version of env_logger brings in
additional non-optional dependencies such as chrono that I don't think is
worth including into ripgrep.

It turns out ripgrep doesn't need any fancy logging. We just need a concept
of log levels and the ability to print to stderr. Therefore, we just roll
our own super simple logger.

This update is motivated by the persistent configuration task. In
particular, we need the ability to toggle the global log level more than
once, and this doesn't appear to be possible with older versions of the
log crate.
2018-02-04 10:40:20 -05:00
Andrew Gallant
fe00255494
deps: bump wincolor 2018-02-03 20:38:03 -05:00
Andrew Gallant
07c837e740
wincolor-0.1.5 2018-02-03 20:34:08 -05:00
Andrew Gallant
cb0e693e31
docs: touch up issue template 2018-02-03 09:21:21 -05:00
Andrew Gallant
e9d448e93b
docs: add an issue template 2018-02-03 09:18:31 -05:00
Andrew Gallant
c7fc916e6b
deps: bump walkdir (again)
walkdir 2.1.2 introduced a subtle bug on Windows when dealing with
symlinks. We update to the latest to get the fix.
2018-02-01 22:51:34 -05:00
Andrew Gallant
e36b65a11a
windows: fix OneDrive traversals
This commit fixes a bug on Windows where directory traversals were
completely broken when attempting to scan OneDrive directories that use
the "file on demand" strategy.

The specific problem was that Rust's standard library treats OneDrive
directories as reparse points instead of directories, which causes
methods like `FileType::is_file` and `FileType::is_dir` to always return
false, even when retrieved via methods like `metadata` that purport to
follow symbolic links.

We fix this by peppering our code with checks on the underlying file
attributes exposed by Windows. We consider an entry a directory if and
only if the directory bit is set on the attributes. We are careful to
make sure that the code remains the same on non-Windows platforms.

Note that we also bump the dependency on `walkdir`, which contains a
similar fix for its traversals.

This bug is recorded upstream:
https://github.com/rust-lang/rust/issues/46484

Upstream also has a pending PR:
https://github.com/rust-lang/rust/pull/47956

Fixes #705
2018-02-01 21:11:02 -05:00
Andrew Gallant
11ad7ab204
ignore/deps: update walkdir
This commit updates to the latest walkdir release, which fixes a bug on
Windows where ripgrep would panic if it was told to traverse a directory
while following symlinks *and* if opening one of those symlinks failed.

Fixes #633
2018-02-01 17:09:06 -05:00
Andrew Gallant
93943793c3
worker: better error handling for memory maps
Previously, we would bail out of using memory maps if we could detect
ahead of time that opening a memory map would fail. The only case we
checked was whether the file size was 0 or not.

This is actually insufficient. The mmap call can return ENODEV errors
when a file doesn't support memory maps. This is the case for new files
exposed by Linux, for example,
/sys/devices/system/cpu/vulnerabilities/meltdown.

We fix this by checking the actual error codes returned by the mmap call.
If ENODEV (or EOVERFLOW) is returned, then we fall back to regular `read`
calls. If any other error occurs, we report it to the user.

Fixes #760
2018-01-31 19:20:36 -05:00
Andrew Gallant
0fedaa7d28
style: remove eprintln macro
The eprintln! macro was added to Rust's standard library in Rust 1.19.0,
which is below ripgrep's minimum Rust version. Therefore, we can rely on
the standard library variant now.
2018-01-31 19:17:51 -05:00
llogiq
e05023b406 deps: update bytecount
This improves performance with current nightly rustc.
2018-01-30 16:34:30 -05:00
Balaji Sivaraman
f007f940c5 search: add support for searching compressed files
This commit adds opt-in support for searching compressed files during
recursive search. This behavior is only enabled when the
`-z/--search-zip` flag is passed to ripgrep. When enabled, a limited set
of common compression formats are recognized via file extension, and a
new process is spawned to perform the decompression. ripgrep then
searches the stdout of that spawned process.

Closes #539
2018-01-30 09:13:53 -05:00
Marc Tiehuis
a8543f798d termcolor: add extended color support
This commit adds 256-color and 24-bit truecolor support to ripgrep.
This only provides output support on ANSI terminals. If the Windows
console is used for coloring, then 256-color and 24-bit color settings
are ignored.
2018-01-29 18:49:50 -05:00
Andrew Gallant
ef9e17d28a
deps: bump memmap to 0.6.2
This removes the last dependency that required winapi 0.2. ripgrep now
only depends on winapi 0.3.
2018-01-29 17:09:01 -05:00
ptzz
3cb4d1337e ignore: support custom file names
This commit adds support for ignore files with custom names. This
allows for application specific ignorefile names, e.g. using
`.fdignore` for `fd`.

See also: https://github.com/BurntSushi/ripgrep/issues/673

See also: https://github.com/sharkdp/fd/issues/156
2018-01-29 16:06:05 -05:00
kennytm
8514d4fbb4 termcolor: tweak reset escape
Write `Ansi::reset()` using `\x1b[0m` instead of `\x1b[m`.

This works around an AppVeyor bug: https://github.com/appveyor/ci/issues/1824
2018-01-29 14:14:55 -05:00
Michael Salihi
ed9150c9b4 types: add Smarty tpl
It is used by Prestashop CMS and more.
2018-01-29 14:12:02 -05:00
dana
51864c13fc ignore: fix handling of / in patterns
This commit makes handling of patterns containing a `/`
match actual git behaviour and the specification written
in `man gitignore`.

Fixes #761
2018-01-29 14:10:59 -05:00
Arvid Gerstmann
35f802166d types: add gn type
This is for Google's new build system.
2018-01-17 08:49:01 -05:00
Arvid Gerstmann
bba2d56292 types: add hxx to the cpp type 2018-01-17 08:47:23 -05:00
Hendrik Sollich
012880914b types: add webidl (*.webidl) 2018-01-14 11:20:21 -05:00
Hendrik Sollich
832f5baf1a types: add webidl 2018-01-12 19:48:50 -05:00
dana
a6d3a959eb types: yarn.lock is not YAML
Fixes #747
2018-01-12 19:35:46 -05:00
Sebastian Torres
f00625c3f4 readme: add Ubuntu install instructions 2018-01-12 18:44:28 -05:00
Igor Gnatenko
82d03b99cd readme: ripgrep is in Fedora 27
References: https://bodhi.fedoraproject.org/updates/FEDORA-2018-ca3c304458
2018-01-12 13:40:00 -05:00