1
0
mirror of https://github.com/BurntSushi/ripgrep.git synced 2024-12-12 19:18:24 +02:00
Commit Graph

113 Commits

Author SHA1 Message Date
Andrew Gallant
8cb5833ef9 argv: update clap to 2.29.4
We use the new AppSettings::AllArgsOverrideSelf to permit all flags to
be specified multiple times. This removes the need for our previous
work-around where we would enable `multiple` for every flag and then
just extract the last value when consuming clap's matches.

We also add a couple regression tests that ensure repeated switches and
flags work as expected.
2018-02-06 12:07:59 -05:00
Andrew Gallant
c57d0fb4e8 config: add persistent configuration
This commit adds support for reading configuration files that change
ripgrep's default behavior. The format of the configuration file is an
"rc" style and is very simple. It is defined by two rules:

  1. Every line is a shell argument, after trimming ASCII whitespace.
  2. Lines starting with '#' (optionally preceded by any amount of
     ASCII whitespace) are ignored.

ripgrep will look for a single configuration file if and only if the
RIPGREP_CONFIG_PATH environment variable is set and is non-empty.
ripgrep will parse shell arguments from this file on startup and will
behave as if the arguments in this file were prepended to any explicit
arguments given to ripgrep on the command line.

For example, if your ripgreprc file contained a single line:

    --smart-case

then the following command

    RIPGREP_CONFIG_PATH=wherever/.ripgreprc rg foo

would behave identically to the following command

    rg --smart-case foo

This commit also adds a new flag, --no-config, that when present will
suppress any and all support for configuration. This includes any future
support for auto-loading configuration files from pre-determined paths
(which this commit does not add).

Conflicts between configuration files and explicit arguments are handled
exactly like conflicts in the same command line invocation. That is,
this command:

    RIPGREP_CONFIG_PATH=wherever/.ripgreprc rg foo --case-sensitive

is exactly equivalent to

    rg --smart-case foo --case-sensitive

in which case, the --case-sensitive flag would override the --smart-case
flag.

Closes #196
2018-02-04 10:40:20 -05:00
Andrew Gallant
d83bab4d3f argv: permit repeated flags
This commit builds on the previous argv refactor by being more principled
about how we declared our flags. In particular, we now require that every
clap argument is one of three things: a positional argument, a switch or
a flag that accepts exactly one value. The latter two are always permitted
to be repeated, and we modify the code that consumes a clap::ArgMatches to
always use the *last* value of an argument. (clap by default always uses
the first value of argument, if it has been repeated and is accessed via
one of the singleton accessors.)

Fixes #553
2018-02-04 10:40:20 -05:00
Andrew Gallant
3535047094 logger: drop env_logger
This commit updates the `log` crate to 0.4 and drops the dependency on
env_logger. In particular, the latest version of env_logger brings in
additional non-optional dependencies such as chrono that I don't think is
worth including into ripgrep.

It turns out ripgrep doesn't need any fancy logging. We just need a concept
of log levels and the ability to print to stderr. Therefore, we just roll
our own super simple logger.

This update is motivated by the persistent configuration task. In
particular, we need the ability to toggle the global log level more than
once, and this doesn't appear to be possible with older versions of the
log crate.
2018-02-04 10:40:20 -05:00
Andrew Gallant
e36b65a11a
windows: fix OneDrive traversals
This commit fixes a bug on Windows where directory traversals were
completely broken when attempting to scan OneDrive directories that use
the "file on demand" strategy.

The specific problem was that Rust's standard library treats OneDrive
directories as reparse points instead of directories, which causes
methods like `FileType::is_file` and `FileType::is_dir` to always return
false, even when retrieved via methods like `metadata` that purport to
follow symbolic links.

We fix this by peppering our code with checks on the underlying file
attributes exposed by Windows. We consider an entry a directory if and
only if the directory bit is set on the attributes. We are careful to
make sure that the code remains the same on non-Windows platforms.

Note that we also bump the dependency on `walkdir`, which contains a
similar fix for its traversals.

This bug is recorded upstream:
https://github.com/rust-lang/rust/issues/46484

Upstream also has a pending PR:
https://github.com/rust-lang/rust/pull/47956

Fixes #705
2018-02-01 21:11:02 -05:00
Balaji Sivaraman
f007f940c5 search: add support for searching compressed files
This commit adds opt-in support for searching compressed files during
recursive search. This behavior is only enabled when the
`-z/--search-zip` flag is passed to ripgrep. When enabled, a limited set
of common compression formats are recognized via file extension, and a
new process is spawned to perform the decompression. ripgrep then
searches the stdout of that spawned process.

Closes #539
2018-01-30 09:13:53 -05:00
ptzz
3cb4d1337e ignore: support custom file names
This commit adds support for ignore files with custom names. This
allows for application specific ignorefile names, e.g. using
`.fdignore` for `fd`.

See also: https://github.com/BurntSushi/ripgrep/issues/673

See also: https://github.com/sharkdp/fd/issues/156
2018-01-29 16:06:05 -05:00
dana
58bdc366ec printer: add --passthru flag
The --passthru flag causes ripgrep to print every line,
even if the line does not contain a match. This is a
response to the common pattern of `^|foo` to match every
line, while still highlighting things like `foo`.

Fixes #740
2018-01-11 18:45:51 -05:00
dana
34c0b1bc70 doc: various updates
* Don't use 'smart typography' when generating man page
* Document PATTERN and PATH
* Capitalise place-holder names consistently
* Add note about PATH overriding glob/ignore rules
* Update args.rs for new PATH capitalisation

Fixes #725
2018-01-11 08:05:52 -05:00
Balaji Sivaraman
14779ed0ea ux: suggest --fixed-strings flag
If a regex syntax error occurs, then ripgrep will suggest
using the --fixed-strings flag.

Fixes #727
2018-01-01 11:24:46 -05:00
Balaji Sivaraman
b6177f0459 cleanup: replace try! with ? 2018-01-01 09:22:35 -05:00
Balaji Sivaraman
ba1023e1e4 printer: add support for line number alignment
Closes #544
2018-01-01 09:00:31 -05:00
dana
d73a75d6cd Omit context separators when using a contextless option like -c or -l
Fixes #693
2017-11-29 12:55:42 -05:00
Matthias Krüger
7ae1f373c2 clippy: fix warnings about useless format call and remove references that would be immediately dereferenced by the compiler. 2017-11-22 10:50:28 -05:00
Andrew Gallant
cd575d99f8
ignore: upgrade to walkdir 2
The uninteresting bits of this commit involve mechanical changes for
updates to walkdir 2. The more interesting bits of this commit are the
breaking changes, although none of them should require any significant
change on users of this library. The breaking changes are as follows:

* `DirEntry::path_is_symbolic_link` has been renamed to
  `DirEntry::path_is_symlink`. This matches the conventions in the
  standard library, and also the corresponding name change in walkdir.
* Removed the `From<walkdir::Error> for ignore::Error` impl. This was
  intended to only be used internally, but was the only thing that
  made `walkdir` a public dependency of `ignore`. Therefore, we remove
  it since it seems unnecessary.
* Renamed `WalkBuilder::sort_by` to `WalkBuilder::sort_by_file_name`,
  and changed the type of the comparator from

    Fn(&OsString, &OsString) -> cmp::Ordering + 'static

  to

    Fn(&OsStr, &OsStr) -> cmp::Ordering + Send + Sync + 'static

  The corresponding change in `walkdir` retains the `sort_by` name, but
  gives the comparator a pair of `&DirEntry` values instead of a pair
  of `&OsStr` values. Ideally, `ignore` would hand off its own pair of
  `&ignore::DirEntry` values, but this requires more design work. So for
  now, we retain previous functionality, but leave room to make a proper
  `sort_by` method.

[breaking-change]
2017-10-21 22:40:09 -04:00
dana
40bacbcd7c Add -x/--line-regexp (#520)
add -x/--line-regexp flag
2017-08-09 06:53:35 -04:00
Lincoln Atkinson
354a5cad97 Fix invisible file path text in PowerShell (#557)
change default path color on Windows

This avoids a conflict with a PowerShell configuration that causes text to be invisible.

Fixes #342
2017-07-17 18:01:13 -04:00
Peter S Panov
4047d9db71 add --iglob flag
Working with Chris Stadler, implemented
https://github.com/BurntSushi/ripgrep/issues/163#issuecomment-300012592
2017-07-03 06:52:52 -04:00
Evan.Mattiza
06393f888c fix word boundary w/ capture group
fixes BurntSushi/ripgrep#506. Word boundary search as arg had unexpected
behavior. added capture group to regex to encapsulate 'or' option search and
prevent expansion and partial boundary finds.

Signed-off-by: Evan.Mattiza <emattiza@gmail.com>
2017-06-15 06:55:55 -04:00
Eric Nielsen
f2d1c582a8 Use clap's overrides_with and default_value_if
to better organize options. These are the changes:
- color will have default value of "never" if --vimgrep is given,
  and only if no --color option is given
- last overrides previous: --line-number and --no-line-number, --heading
  and --no-heading, --with-filename and --no-filename, and --vimgrep and
  --count
- no heading will be show if --vimgrep is defined. This worked inside
  vim actually because heading is also only shown if tty is stdout
  (which is not the case when rg is called from vim).

Unfortunately, clap does not behave like a usual GNU/POSIX in some
cases, as reported in https://github.com/kbknapp/clap-rs/issues/970
and https://github.com/kbknapp/clap-rs/issues/976 (having all the bells
and whistles, on the other hand). So we still have issues like rg
failing when same argument is given more than once (unless for the few
ones marked with `multiple(true)`), or having unintuitive precedence
rules (and probably non-intentional, just there because of clap's
limitations) like:
- --no-filename over --vimgrep
- --no-line-number over --column, --pretty or --vimgrep
- --no-heading over --pretty
regardless of the order in which options where given, where the desired
behavior would be that the last option would override the previous ones
given.
2017-06-12 10:29:38 -04:00
Eric Nielsen
13235b596f Use uppercase for required argument names
This reverts a couple of changes introduced in 4c78ca8 and keeps the
`PATTERN` argument consistently uppercased, so error messages can look
like:

    error: The following required arguments were not provided:
        <PATTERN>
2017-06-01 20:41:04 -04:00
Eric Nielsen
ff898cd105 Remove vestigial color function from src/args.rs
It's usage was replaced by the `color_choice` function. Also, `color`
was outdated, as it didn't include testing for the new "ansi" option.
2017-05-26 07:00:58 -04:00
Eric Nielsen
1e3fc79949 Should show filename for one file with vimgrep
With vim configured with:

    set grepprg=rg\ --vimgrep
    set grepformat^=%f:%l:%c:%m

and running the command `:grep 'vimgrep' doc/rg.1`, the output should
be:

    doc/rg.1:446:8:.B \-\-vimgrep

but the actual output was:

    446:8:.B \-\-vimgrep

Same issue would happen if results only match one file. Ag behaves as
expected.
2017-05-24 23:12:16 -04:00
Andrew Gallant
7ad23e5565 Use for_label_no_replacement.
This will cause certain unsupported legacy encodings to act as if they
don't exist, in order to avoid using an unhelpful (in the context of
file searching) "replacement" encoding.

Kudos to @hsivonen for chirping about this!
2017-04-12 18:14:23 -04:00
Marc Tiehuis
66efbad871 Add dfa-size-limit and regex-size-limit arguments
Fixes #362.
2017-04-12 18:14:23 -04:00
Roman Proskuryakov
90a11dec5e Add -o/--only-matching flag.
Currently, the `--only-matching` flag conflicts with the `--replace`
flag. In the future, this restriction may be relaxed.

Fixes #34
2017-04-09 08:47:35 -04:00
Kevin K
0c298f60a6 updates clap and removes home rolled -h/--help distinction
This commit updates clap to v2.23.0

The update contained a bug fix in clap that results in broken code in
ripgrep. ripgrep was relying on the bug, but this commit fixes that
issue. The bug centered around not being able to override the
auto-generated help message by supplying a flag with a long of `help`.

Normally, supplying a flag with a long of `help` means whenever the user
passes `--help`, the consuming code (e.g. ripgrep) is responsible for
displaying the help message. However, due to the bug in clap this wasn't
necessary for ripgrep to do unless the user passed `-h`. With the bug
fixed, it meant the user passing `--help` and clap expected ripgrep to
display the help, yet ripgrep expected clap to display the help. This
has been fixed in this commit of ripgrep.

All well now!

v2.23.0 also brings the abilty to use `Arg::help` or `Arg::long_help`
allowing one to distinguish between `-h` and `--help`. This commit
leaves all doc strings in the `lazy_static!` hashmap however only for
aesthetic reasons.

This means all home rolled handling of `-h`/`--help` has been removed
from ripgrep, yet functionality *and* appearances are 100% the same.
2017-04-05 11:38:58 -04:00
Andrew Gallant
fc975af8e9 Enforce 79 column limit. Grr. 2017-03-31 15:59:04 -04:00
Andrew Gallant
95bc678403 Fix interaction with clap.
Previously, `get_matches` would return even if --help or --version was
given, and we could check for them manually. That behavior seems to have
changed. Instead, we must use get_matches_safe to inspect the error to
determine what happened.

We can't use the same process for -V/--version since clap will
unconditionally print its own version info. Instead, we rename (internally)
the version flag so that clap doesn't interfere.
2017-03-12 22:30:54 -04:00
Ralf Jung
d352b79294 Add new -M/--max-columns option.
This permits setting the maximum line width with respect to the number
of bytes in a line. Omitted lines (whether part of a match, replacement
or context) are replaced with a message stating that the line was
elided.

Fixes #129
2017-03-12 21:21:28 -04:00
Andrew Gallant
4ef4818130 No line numbers when searching only stdin.
This changes the default behavior of ripgrep to *not* show line numbers
when it is printing to a tty and is only searching stdin.

Fixes #380

[breaking-change]
2017-03-12 20:21:40 -04:00
Andrew Gallant
8bbe58d623 Add support for additional text encodings.
This includes, but is not limited to, UTF-16, latin-1, GBK, EUC-JP and
Shift_JIS. (Courtesy of the `encoding_rs` crate.)

Specifically, this feature enables ripgrep to search files that are
encoded in an encoding other than UTF-8. The list of available encodings
is tied directly to what the `encoding_rs` crate supports, which is in
turn tied to the Encoding Standard. The full list of available encodings
can be found here: https://encoding.spec.whatwg.org/#concept-encoding-get

This pull request also introduces the notion that text encodings can be
automatically detected on a best effort basis. Currently, the only
support for this is checking for a UTF-16 bom. In all other cases, a
text encoding of `auto` (the default) implies a UTF-8 or ASCII
compatible source encoding. When a text encoding is otherwise specified,
it is unconditionally used for all files searched.

Since ripgrep's regex engine is fundamentally built on top of UTF-8,
this feature works by transcoding the files to be searched from their
source encoding to UTF-8. This transcoding only happens when:

1. `auto` is specified and a non-UTF-8 encoding is detected.
2. A specific encoding is given by end users (including UTF-8).

When transcoding occurs, errors are handled by automatically inserting
the Unicode replacement character. In this case, ripgrep's output is
guaranteed to be valid UTF-8 (excluding non-UTF-8 file paths, if they
are printed).

In all other cases, the source text is searched directly, which implies
an assumption that it is at least ASCII compatible, but where UTF-8 is
most useful. In this scenario, encoding errors are not detected. In this
case, ripgrep's output will match the input exactly, byte-for-byte.

This design may not be optimal in all cases, but it has some advantages:

1. In the happy path ("UTF-8 everywhere") remains happy. I have not been
   able to witness any performance regressions.
2. In the non-UTF-8 path, implementation complexity is kept relatively
   low. The cost here is transcoding itself. A potentially superior
   implementation might build decoding of any encoding into the regex
   engine itself. In particular, the fundamental problem with
   transcoding everything first is that literal optimizations are nearly
   negated.

Future work should entail improving the user experience. For example, we
might want to auto-detect more text encodings. A more elaborate UX
experience might permit end users to specify multiple text encodings,
although this seems hard to pull off in an ergonomic way.

Fixes #1
2017-03-12 19:54:48 -04:00
Marc Tiehuis
adff43fbb4 Remove clap validator + add max-filesize integration tests 2017-03-08 10:17:18 -05:00
tiehuis
714ae82241 Add --max-filesize option to cli
The --max-filesize option allows filtering files which are larger than
the specified limit. This is potentially useful if one is attempting to
search a number of large files without common file-types/suffixes.

See #369.
2017-03-08 10:17:18 -05:00
Andrew Gallant
525b278049 Don't parses regexes with --files.
When the --files flag is given, ripgrep would still try to parse some of
the positional arguments as regexes. Don't do that.

Fixes #326
2017-02-18 15:34:54 -05:00
Peter Williams
22cb644eb6 termcolor: add support for output to standard error
This is essentially a rename of the existing `Stdout` type to `StandardStream`
and a change of its constructor from a single `new()` function to have two
`stdout()` and `stderr()` functions.

Under the hood, we add add internal IoStandardStream{,Lock} enums that allow
us to abstract between Stdout and Stderr conveniently. The rest of the needed
changes then fall out fairly naturally.

Fixes #324.

[breaking-change]
2017-02-09 20:57:23 -05:00
Andrew Gallant
f5a2d022ec Replace internal atty module with atty crate.
This removes all use of explicit unsafe in ripgrep proper except for
one: accessing the contents of a memory map. (Which may never go away.)
2017-01-15 16:32:30 -05:00
Andrew Gallant
a7d0e40668 Use basic SGR sequences when possible.
In Emacs, its terminal apparently doesn't support "extended" sets of
foreground/background colors. Unless we need to set an "intense" color,
we should instead use one of the eight basic color codes.

Also, remove the "intense" setting from the default set of colors. It
doesn't do much anyway and enables the default color settings to work
in Emacs out of the box.

Fixes #182 (again)
2017-01-13 19:03:03 -05:00
Andrew Gallant
7a951f103a Make --column imply --line-number.
Closes #243
2017-01-11 18:53:35 -05:00
Andrew Gallant
8751e55706 Add --path-separator flag.
This flag permits setting the path separator used for all file paths
printed by ripgrep in normal operation.

Fixes #275
2017-01-10 18:16:15 -05:00
Andrew Gallant
461e0c4e33 Don't search stdout redirected file.
When running ripgrep like this:

    rg foo > output

we must be careful not to search `output` since ripgrep is actively writing
to it. Searching it can cause massive blowups where the file grows without
bound.

While this is conceptually easy to fix (check the inode of the redirection
and the inode of the file you're about to search), there are a few problems
with it.

First, inodes are a Unix thing, so we need a Windows specific solution to
this as well. To resolve this concern, I created a new crate, `same-file`,
which provides a cross platform abstraction.

Second, stat'ing every file is costly. This is not avoidable on Windows,
but on Unix, we can get the inode number directly from directory traversal.
However, this information wasn't exposed, but now it is (through both the
ignore and walkdir crates).

Fixes #286
2017-01-09 16:12:08 -05:00
Andrew Gallant
b65a8c353b Add --sort-files flag.
When used, parallelism is disabled but the results are sorted by file
path.

Closes #263
2017-01-06 22:43:59 -05:00
Andrew Gallant
b187c1a817 Rejigger bold and intense settings.
Previously, ripgrep would only emit the 'bold' ANSI escape sequence if
no foreground or background color was set. Instead, it would convert colors
to their "intense" versions if bold was set. The intent was to do the same
thing on Windows and Unix. However, this had a few negative side effects:

  1. Omitting the 'bold' ANSI escape when 'bold' was set is surprising.
  2. Intense colors can look quite bad and be hard to read.

To fix this, we introduce a new setting called 'intense' in the --colors
flag, and thread that down through to the public API of the `termcolor`
crate. The 'intense' setting has environment specific behavior:

  1. In ANSI mode, it will convert the selected color to its "intense"
     variant.
  2. In the Windows console, it will make the text "intense."

There is no longer any "smart" handling of the 'bold' style. The 'bold'
ANSI escape is always emitted when it is selected. In the Windows
console, the 'bold' setting now has no effect. Note that this is a
breaking change.

Fixes #266, #293
2017-01-06 20:09:51 -05:00
Andrew Gallant
163e00677a Update to regex 0.2. 2017-01-01 01:03:21 -05:00
Andrew Gallant
de5cb7d22e Remove special ^C handling.
This means that ripgrep will no longer try to reset your colors in your
terminal if you kill it while searching. This could result in messing up
the colors in your terminal, and the fix is to simply run some other
command that resets them for you. For example:

    $ echo -ne "\033[0m"

The reason why the ^C handling was removed is because it is irrevocably
broken on Windows and is impossible to do correctly and efficiently in
ANSI terminals.

Fixes #281
2016-12-24 12:53:09 -05:00
Andrew Gallant
084d3f4911 Small code cleanups. 2016-12-24 10:06:37 -05:00
Leonardo Yvens
dd5ded2f78 fix some clippy lints (#288) 2016-12-23 14:53:35 -05:00
Andrew Gallant
d06f84ced3 Get rid of special mmap decision on Windows.
I spent some quality time on my Windows 10 laptop and it appears to
suffer from a similar trade-off as on Linux: mmaps are bad for large
directory traversals but good for single large files.

Darwin continues to reject memory maps in all cases (unless explicitly
requested), but more testing should be done there.
2016-11-20 15:32:50 -05:00
Andrew Gallant
9598331fa8 Propagate no_messages option to worker.
Fixes #241
2016-11-20 15:01:37 -05:00
Andrew Gallant
e8a30cb893 Completely re-work colored output and tty handling.
This commit completely guts all of the color handling code and replaces
most of it with two new crates: wincolor and termcolor. wincolor
provides a simple API to coloring using the Windows console and
termcolor provides a platform independent coloring API tuned for
multithreaded command line programs. This required a lot more
flexibility than what the `term` crate provided, so it was dropped.
We instead switch to writing ANSI escape sequences directly and ignore
the TERMINFO database.

In addition to fixing several bugs, this commit also permits end users
to customize colors to a certain extent. For example, this command will
set the match color to magenta and the line number background to yellow:

    rg --colors 'match:fg:magenta' --colors 'line:bg:yellow' foo

For tty handling, we've adopted a hack from `git` to do tty detection in
MSYS/mintty terminals. As a result, ripgrep should get both color
detection and piping correct on Windows regardless of which terminal you
use.

Finally, switch to line buffering. Performance doesn't seem to be
impacted and it's an otherwise more user friendly option.

Fixes #37, Fixes #51, Fixes #94, Fixes #117, Fixes #182, Fixes #231
2016-11-20 11:14:52 -05:00