1
0
mirror of https://github.com/BurntSushi/ripgrep.git synced 2024-12-12 19:18:24 +02:00
Commit Graph

134 Commits

Author SHA1 Message Date
Andrew Gallant
87b745454d
ripgrep: use 'ignore' for skipping stdout
This removes ripgrep-specific code for filtering files that correspond to
stdout and instead uses the 'ignore' crate's functionality for doing the
same.
2018-08-27 21:18:53 -04:00
Andrew Gallant
40e310a9f9 ripgrep: add --sort and --sortr flags
These flags each accept one of five choices: none, path, modified,
accessed or created. The value indicates how the results are sorted.
For --sort, results are sorted in ascending order where as for --sortr,
results are sorted in descending order.

Closes #404
2018-08-26 18:42:25 -04:00
Andrew Gallant
f9ce7a84a8 ignore: add 'same_file_system' option
This commit adds a 'same_file_system' option to the walk builder. For
single threaded walking, it defers to the walkdir crate, which has the
same option. The bulk of this commit implements this flag for the parallel
walker. We add one very feeble test for this.

The parallel walker is now officially a complete mess.

Closes #321
2018-08-26 18:42:25 -04:00
Andrew Gallant
05a0389555
ripgrep: use winapi-util for stdin_is_readable 2018-08-25 00:30:15 -04:00
Andrew Gallant
bb110c1ebe ripgrep: migrate to libripgrep
This commit does the work to delete the old `grep` crate and effectively
rewrite most of ripgrep core to use the new libripgrep crates. The new
`grep` crate is now a facade that collects the various crates that make
up libripgrep.

The most complex part of ripgrep core is now arguably the translation
between command line parameters and the library options, which is
ultimately where we want to be.
2018-08-20 07:10:19 -04:00
Andrew Gallant
dca8110da2
ripgrep: when given no patterns, don't match
Generally speaking, ripgrep prevents the case of not having any patterns
via its arg parsing. However, it is possible for users to provide a file
of patterns via the `-f` flag. If that file is empty, then ripgrep has
nothing to search for and therefore should not ever produce any match.

One way of fixing this might be to replace the absence of patterns with
a pattern that can never match, but this still requires opening and
searching through every file, which is quite a waste. Instead, we detect
this case explicitly and quit early.

Fixes #900
2018-07-22 12:07:18 -04:00
Andrew Gallant
560dffd247
ripgrep: add --no-ignore-global flag
This commit adds a new --no-ignore-global flag that permits disabling
the use of global gitignore filtering. Global gitignores are generally
found in `$HOME/.config/git/ignore`, but its location can be configured
via git's `core.excludesFile` option.

Closes #934
2018-07-22 10:42:32 -04:00
Andrew Gallant
b9c922be53
ripgrep: better --path-separator error message
This commit improves the error message when --path-separator fails. Namely,
it prints the separator it got and also prints a notice for Windows users
for common failure modes.

Fixes #957
2018-07-22 09:33:03 -04:00
Charles Blake
231456c409 ripgrep: add --pre flag
The preprocessor flag accepts a command program and executes this
program for every input file that is searched. Instead of searching the
file directly, ripgrep will instead search the stdout contents of the
program.

Closes #978, Closes #981
2018-07-21 17:25:12 -04:00
dana
b38b101c77 ripgrep: rename --maxdepth to --max-depth
We keep the old `--maxdepth` spelling to preserve backward
compatibility.

PR #967
2018-06-25 20:22:09 -04:00
Andrew Gallant
cd6c190967 ripgrep: use new BufferedStandardStream from termcolor
Specifically, this will use a buffered writer when not printing to a tty.
This fixes a long standing performance regression where ripgrep would
slow down dramatically if it needed to report a lot of matches.

Fixes #955
2018-06-23 20:49:05 -04:00
Andrew Gallant
ae6f871491
output: remove --line-number-width flag
This commit does what no software project has ever done before: we've
outright removed a flag with no possible way to recapture its
functionality.

This flag presents numerous problems in that it never really worked well
in the first place, and completely falls over when ripgrep uses the
--no-heading output format. Well meaning users want ripgrep to fix this
by getting into the alignment business by buffering all output, but that
is a line that I refuse to cross.

Fixes #795
2018-04-23 19:57:22 -04:00
Andrew Gallant
b75526bd7f
output: add --no-column flag
This disables columns in the output if they were otherwise enabled.

Fixes #880
2018-04-23 19:26:58 -04:00
Andrew Gallant
0ee0b160b5
logging: add new --no-ignore-messages flag
The new --no-ignore-messages flag permits suppressing errors related to
parsing .gitignore or .ignore files. These error messages can be somewhat
annoying since they can surface from repositories that one has no control
over.

Fixes #646
2018-04-23 18:18:44 -04:00
Andrew Gallant
cd08707c7c grep: upgrade to regex-syntax 0.5
This update brings with it many bug fixes:

  * Better error messages are printed overall. We also include
    explicit call out for unsupported features like backreferences
    and look-around.
  * Regexes like `\s*{` no longer emit incomprehensible errors.
  * Unicode escape sequences, such as `\u{..}` are now supported.

For the most part, this upgrade was done in a straight-forward way. We
resist the urge to refactor the `grep` crate, in anticipation of it
being rewritten anyway.

Note that we removed the `--fixed-strings` suggestion whenever a regex
syntax error occurs. In practice, I've found that it results in a lot of
false positives, and I believe that its use is not as paramount now that
regex parse errors are much more readable.

Closes #268, Closes #395, Closes #702, Closes #853
2018-03-13 22:55:39 -04:00
Andrew Gallant
1f70e9187c deps: update regex crate
This update brings with it a new feature of the regex crate which will
now use SIMD optimizations automatically at runtime with no necessary
compile time flags. All that's needed is to enable the `unstable` feature.

Other crates, such as bytecount and encoding_rs, are still using the
old-style SIMD support, so we leave the simd-accel and avx-accel features.
However, the binaries we distribute on Github no longer have those
features enabled, which makes them truly portable.

Fixes #135
2018-03-12 23:21:42 -04:00
Balaji Sivaraman
00520b30f5
output: add --stats flag
This commit provides basic support for a --stats flag, which will print
various aggregate statistics about a search after all of the results
have been printed. This is mostly intended to support a similar feature
found in the Silver Searcher. Note though that we don't emit the total
bytes searched; this is a first pass at an implementation and we can
improve upon it later.

Closes #411, Closes #799
2018-03-10 10:59:00 -05:00
Andrew Gallant
11a8f0eaf0
args: treat --count --only-matching as --count-matches
Namely, when ripgrep is asked to count things and is also asked to print
every match on its own line, then we should just automatically count the
matches and not the lines. This is a departure from how GNU grep behaves,
but there is a compelling argument to be made that GNU grep's behavior
doesn't make a lot of sense.

Note that since this changes the behavior of combining two existing
flags, this is a breaking change.
2018-03-10 10:38:34 -05:00
Balaji Sivaraman
27fc9f2fd3
search: add a --count-matches flag
This commit introduces a new flag, --count-matches, which will cause
ripgrep to report a total count of all matches instead of a count of
total lines matched.

Closes #566, Closes #814
2018-03-10 10:38:25 -05:00
Balaji Sivaraman
b006943c01
search: add -b/--byte-offset flag
This commit adds support for printing 0-based byte offset before each
line. We handle corner cases such as `-o/--only-matching` and
`-C/--context` as well.

Closes #812
2018-03-10 10:15:19 -05:00
Andrew Gallant
224c112e05 argv: tweak the meaning of zero
This commit makes a small tweak to the --max-columns flag. Namely, if
the value of the flag is 0, then ripgrep behaves as-if the flag were
absent.

This is useful in the context of ripgrep reading configuration from the
environment. For example, an end user might set --max-columns=150, but we
should permit the user to disable this setting when needed. Using -M0 is
a nice way to do that.

We do this because a zero value for --max-columns isn't particularly
meaningful. We do leave the --max-count, --max-filesize and --maxdepth
flags alone though, since a zero value for those flags is potentially
meaningful. (--max-count even has tests for ripgrep's behavior when
given a value of 0.)
2018-02-06 12:07:59 -05:00
Andrew Gallant
8cb5833ef9 argv: update clap to 2.29.4
We use the new AppSettings::AllArgsOverrideSelf to permit all flags to
be specified multiple times. This removes the need for our previous
work-around where we would enable `multiple` for every flag and then
just extract the last value when consuming clap's matches.

We also add a couple regression tests that ensure repeated switches and
flags work as expected.
2018-02-06 12:07:59 -05:00
Andrew Gallant
c57d0fb4e8 config: add persistent configuration
This commit adds support for reading configuration files that change
ripgrep's default behavior. The format of the configuration file is an
"rc" style and is very simple. It is defined by two rules:

  1. Every line is a shell argument, after trimming ASCII whitespace.
  2. Lines starting with '#' (optionally preceded by any amount of
     ASCII whitespace) are ignored.

ripgrep will look for a single configuration file if and only if the
RIPGREP_CONFIG_PATH environment variable is set and is non-empty.
ripgrep will parse shell arguments from this file on startup and will
behave as if the arguments in this file were prepended to any explicit
arguments given to ripgrep on the command line.

For example, if your ripgreprc file contained a single line:

    --smart-case

then the following command

    RIPGREP_CONFIG_PATH=wherever/.ripgreprc rg foo

would behave identically to the following command

    rg --smart-case foo

This commit also adds a new flag, --no-config, that when present will
suppress any and all support for configuration. This includes any future
support for auto-loading configuration files from pre-determined paths
(which this commit does not add).

Conflicts between configuration files and explicit arguments are handled
exactly like conflicts in the same command line invocation. That is,
this command:

    RIPGREP_CONFIG_PATH=wherever/.ripgreprc rg foo --case-sensitive

is exactly equivalent to

    rg --smart-case foo --case-sensitive

in which case, the --case-sensitive flag would override the --smart-case
flag.

Closes #196
2018-02-04 10:40:20 -05:00
Andrew Gallant
d83bab4d3f argv: permit repeated flags
This commit builds on the previous argv refactor by being more principled
about how we declared our flags. In particular, we now require that every
clap argument is one of three things: a positional argument, a switch or
a flag that accepts exactly one value. The latter two are always permitted
to be repeated, and we modify the code that consumes a clap::ArgMatches to
always use the *last* value of an argument. (clap by default always uses
the first value of argument, if it has been repeated and is accessed via
one of the singleton accessors.)

Fixes #553
2018-02-04 10:40:20 -05:00
Andrew Gallant
3535047094 logger: drop env_logger
This commit updates the `log` crate to 0.4 and drops the dependency on
env_logger. In particular, the latest version of env_logger brings in
additional non-optional dependencies such as chrono that I don't think is
worth including into ripgrep.

It turns out ripgrep doesn't need any fancy logging. We just need a concept
of log levels and the ability to print to stderr. Therefore, we just roll
our own super simple logger.

This update is motivated by the persistent configuration task. In
particular, we need the ability to toggle the global log level more than
once, and this doesn't appear to be possible with older versions of the
log crate.
2018-02-04 10:40:20 -05:00
Andrew Gallant
e36b65a11a
windows: fix OneDrive traversals
This commit fixes a bug on Windows where directory traversals were
completely broken when attempting to scan OneDrive directories that use
the "file on demand" strategy.

The specific problem was that Rust's standard library treats OneDrive
directories as reparse points instead of directories, which causes
methods like `FileType::is_file` and `FileType::is_dir` to always return
false, even when retrieved via methods like `metadata` that purport to
follow symbolic links.

We fix this by peppering our code with checks on the underlying file
attributes exposed by Windows. We consider an entry a directory if and
only if the directory bit is set on the attributes. We are careful to
make sure that the code remains the same on non-Windows platforms.

Note that we also bump the dependency on `walkdir`, which contains a
similar fix for its traversals.

This bug is recorded upstream:
https://github.com/rust-lang/rust/issues/46484

Upstream also has a pending PR:
https://github.com/rust-lang/rust/pull/47956

Fixes #705
2018-02-01 21:11:02 -05:00
Balaji Sivaraman
f007f940c5 search: add support for searching compressed files
This commit adds opt-in support for searching compressed files during
recursive search. This behavior is only enabled when the
`-z/--search-zip` flag is passed to ripgrep. When enabled, a limited set
of common compression formats are recognized via file extension, and a
new process is spawned to perform the decompression. ripgrep then
searches the stdout of that spawned process.

Closes #539
2018-01-30 09:13:53 -05:00
ptzz
3cb4d1337e ignore: support custom file names
This commit adds support for ignore files with custom names. This
allows for application specific ignorefile names, e.g. using
`.fdignore` for `fd`.

See also: https://github.com/BurntSushi/ripgrep/issues/673

See also: https://github.com/sharkdp/fd/issues/156
2018-01-29 16:06:05 -05:00
dana
58bdc366ec printer: add --passthru flag
The --passthru flag causes ripgrep to print every line,
even if the line does not contain a match. This is a
response to the common pattern of `^|foo` to match every
line, while still highlighting things like `foo`.

Fixes #740
2018-01-11 18:45:51 -05:00
dana
34c0b1bc70 doc: various updates
* Don't use 'smart typography' when generating man page
* Document PATTERN and PATH
* Capitalise place-holder names consistently
* Add note about PATH overriding glob/ignore rules
* Update args.rs for new PATH capitalisation

Fixes #725
2018-01-11 08:05:52 -05:00
Balaji Sivaraman
14779ed0ea ux: suggest --fixed-strings flag
If a regex syntax error occurs, then ripgrep will suggest
using the --fixed-strings flag.

Fixes #727
2018-01-01 11:24:46 -05:00
Balaji Sivaraman
b6177f0459 cleanup: replace try! with ? 2018-01-01 09:22:35 -05:00
Balaji Sivaraman
ba1023e1e4 printer: add support for line number alignment
Closes #544
2018-01-01 09:00:31 -05:00
dana
d73a75d6cd Omit context separators when using a contextless option like -c or -l
Fixes #693
2017-11-29 12:55:42 -05:00
Matthias Krüger
7ae1f373c2 clippy: fix warnings about useless format call and remove references that would be immediately dereferenced by the compiler. 2017-11-22 10:50:28 -05:00
Andrew Gallant
cd575d99f8
ignore: upgrade to walkdir 2
The uninteresting bits of this commit involve mechanical changes for
updates to walkdir 2. The more interesting bits of this commit are the
breaking changes, although none of them should require any significant
change on users of this library. The breaking changes are as follows:

* `DirEntry::path_is_symbolic_link` has been renamed to
  `DirEntry::path_is_symlink`. This matches the conventions in the
  standard library, and also the corresponding name change in walkdir.
* Removed the `From<walkdir::Error> for ignore::Error` impl. This was
  intended to only be used internally, but was the only thing that
  made `walkdir` a public dependency of `ignore`. Therefore, we remove
  it since it seems unnecessary.
* Renamed `WalkBuilder::sort_by` to `WalkBuilder::sort_by_file_name`,
  and changed the type of the comparator from

    Fn(&OsString, &OsString) -> cmp::Ordering + 'static

  to

    Fn(&OsStr, &OsStr) -> cmp::Ordering + Send + Sync + 'static

  The corresponding change in `walkdir` retains the `sort_by` name, but
  gives the comparator a pair of `&DirEntry` values instead of a pair
  of `&OsStr` values. Ideally, `ignore` would hand off its own pair of
  `&ignore::DirEntry` values, but this requires more design work. So for
  now, we retain previous functionality, but leave room to make a proper
  `sort_by` method.

[breaking-change]
2017-10-21 22:40:09 -04:00
dana
40bacbcd7c Add -x/--line-regexp (#520)
add -x/--line-regexp flag
2017-08-09 06:53:35 -04:00
Lincoln Atkinson
354a5cad97 Fix invisible file path text in PowerShell (#557)
change default path color on Windows

This avoids a conflict with a PowerShell configuration that causes text to be invisible.

Fixes #342
2017-07-17 18:01:13 -04:00
Peter S Panov
4047d9db71 add --iglob flag
Working with Chris Stadler, implemented
https://github.com/BurntSushi/ripgrep/issues/163#issuecomment-300012592
2017-07-03 06:52:52 -04:00
Evan.Mattiza
06393f888c fix word boundary w/ capture group
fixes BurntSushi/ripgrep#506. Word boundary search as arg had unexpected
behavior. added capture group to regex to encapsulate 'or' option search and
prevent expansion and partial boundary finds.

Signed-off-by: Evan.Mattiza <emattiza@gmail.com>
2017-06-15 06:55:55 -04:00
Eric Nielsen
f2d1c582a8 Use clap's overrides_with and default_value_if
to better organize options. These are the changes:
- color will have default value of "never" if --vimgrep is given,
  and only if no --color option is given
- last overrides previous: --line-number and --no-line-number, --heading
  and --no-heading, --with-filename and --no-filename, and --vimgrep and
  --count
- no heading will be show if --vimgrep is defined. This worked inside
  vim actually because heading is also only shown if tty is stdout
  (which is not the case when rg is called from vim).

Unfortunately, clap does not behave like a usual GNU/POSIX in some
cases, as reported in https://github.com/kbknapp/clap-rs/issues/970
and https://github.com/kbknapp/clap-rs/issues/976 (having all the bells
and whistles, on the other hand). So we still have issues like rg
failing when same argument is given more than once (unless for the few
ones marked with `multiple(true)`), or having unintuitive precedence
rules (and probably non-intentional, just there because of clap's
limitations) like:
- --no-filename over --vimgrep
- --no-line-number over --column, --pretty or --vimgrep
- --no-heading over --pretty
regardless of the order in which options where given, where the desired
behavior would be that the last option would override the previous ones
given.
2017-06-12 10:29:38 -04:00
Eric Nielsen
13235b596f Use uppercase for required argument names
This reverts a couple of changes introduced in 4c78ca8 and keeps the
`PATTERN` argument consistently uppercased, so error messages can look
like:

    error: The following required arguments were not provided:
        <PATTERN>
2017-06-01 20:41:04 -04:00
Eric Nielsen
ff898cd105 Remove vestigial color function from src/args.rs
It's usage was replaced by the `color_choice` function. Also, `color`
was outdated, as it didn't include testing for the new "ansi" option.
2017-05-26 07:00:58 -04:00
Eric Nielsen
1e3fc79949 Should show filename for one file with vimgrep
With vim configured with:

    set grepprg=rg\ --vimgrep
    set grepformat^=%f:%l:%c:%m

and running the command `:grep 'vimgrep' doc/rg.1`, the output should
be:

    doc/rg.1:446:8:.B \-\-vimgrep

but the actual output was:

    446:8:.B \-\-vimgrep

Same issue would happen if results only match one file. Ag behaves as
expected.
2017-05-24 23:12:16 -04:00
Andrew Gallant
7ad23e5565 Use for_label_no_replacement.
This will cause certain unsupported legacy encodings to act as if they
don't exist, in order to avoid using an unhelpful (in the context of
file searching) "replacement" encoding.

Kudos to @hsivonen for chirping about this!
2017-04-12 18:14:23 -04:00
Marc Tiehuis
66efbad871 Add dfa-size-limit and regex-size-limit arguments
Fixes #362.
2017-04-12 18:14:23 -04:00
Roman Proskuryakov
90a11dec5e Add -o/--only-matching flag.
Currently, the `--only-matching` flag conflicts with the `--replace`
flag. In the future, this restriction may be relaxed.

Fixes #34
2017-04-09 08:47:35 -04:00
Kevin K
0c298f60a6 updates clap and removes home rolled -h/--help distinction
This commit updates clap to v2.23.0

The update contained a bug fix in clap that results in broken code in
ripgrep. ripgrep was relying on the bug, but this commit fixes that
issue. The bug centered around not being able to override the
auto-generated help message by supplying a flag with a long of `help`.

Normally, supplying a flag with a long of `help` means whenever the user
passes `--help`, the consuming code (e.g. ripgrep) is responsible for
displaying the help message. However, due to the bug in clap this wasn't
necessary for ripgrep to do unless the user passed `-h`. With the bug
fixed, it meant the user passing `--help` and clap expected ripgrep to
display the help, yet ripgrep expected clap to display the help. This
has been fixed in this commit of ripgrep.

All well now!

v2.23.0 also brings the abilty to use `Arg::help` or `Arg::long_help`
allowing one to distinguish between `-h` and `--help`. This commit
leaves all doc strings in the `lazy_static!` hashmap however only for
aesthetic reasons.

This means all home rolled handling of `-h`/`--help` has been removed
from ripgrep, yet functionality *and* appearances are 100% the same.
2017-04-05 11:38:58 -04:00
Andrew Gallant
fc975af8e9 Enforce 79 column limit. Grr. 2017-03-31 15:59:04 -04:00
Andrew Gallant
95bc678403 Fix interaction with clap.
Previously, `get_matches` would return even if --help or --version was
given, and we could check for them manually. That behavior seems to have
changed. Instead, we must use get_matches_safe to inspect the error to
determine what happened.

We can't use the same process for -V/--version since clap will
unconditionally print its own version info. Instead, we rename (internally)
the version flag so that clap doesn't interfere.
2017-03-12 22:30:54 -04:00