1
0
mirror of https://github.com/BurntSushi/ripgrep.git synced 2024-12-12 19:18:24 +02:00
Commit Graph

1603 Commits

Author SHA1 Message Date
Andrew Gallant
29696d1455
deps/printer: update minimal versions 2021-06-12 08:08:18 -04:00
Andrew Gallant
57ce623a57
grep-printer-0.1.6 2021-06-12 08:07:46 -04:00
Andrew Gallant
f1c656de40
deps/searcher: update minimal versions 2021-06-12 08:07:28 -04:00
Andrew Gallant
dd47582619
grep-searcher-0.1.8 2021-06-12 08:06:58 -04:00
Andrew Gallant
9b88cf8b72
deps/pcre2: update minimal versions 2021-06-12 08:06:50 -04:00
Andrew Gallant
6668d7ba8a
grep-pcre2-0.1.5 2021-06-12 08:06:29 -04:00
Andrew Gallant
008da5dca4
pcre2: update minimal version to 0.2.3 2021-06-12 08:05:56 -04:00
Andrew Gallant
a34df1f690
deps/regex: update minimal versions 2021-06-12 08:05:36 -04:00
Andrew Gallant
7f3fd6f7ce
grep-regex-0.1.9 2021-06-12 08:03:56 -04:00
Andrew Gallant
6331a7ac18
deps/matcher: update minimal versions 2021-06-12 08:03:47 -04:00
Andrew Gallant
cd4386bd9b
grep-matcher-0.1.5 2021-06-12 08:02:30 -04:00
Andrew Gallant
cdc20c5685
deps/cli: update minimal versions 2021-06-12 08:02:18 -04:00
Andrew Gallant
0cf2b98df2
grep-cli-0.1.6 2021-06-12 08:01:22 -04:00
Andrew Gallant
9efdbf74a1
deps/ignore: update minimal versions 2021-06-12 08:01:13 -04:00
Andrew Gallant
53cb9a779e
release: add step about making sure 'master' is in sync
Otherwise, if we start doing crate releases from the local checkout
(with git tags) and it turns out that origin/master has newer commits,
rebasing local master will then invalidate those tags.
2021-06-12 07:59:47 -04:00
Andrew Gallant
14860b0f16
ignore-0.4.18 2021-06-12 07:59:07 -04:00
Andrew Gallant
0eb1a1e7c9
deps/globset: update minimal versions 2021-06-12 07:58:46 -04:00
Andrew Gallant
5631e5c7a0
globset-0.4.7 2021-06-12 07:56:56 -04:00
Andrew Gallant
21644408f2
release: tweak 'cargo outdated' advice
I do run --aggressive, although I've been ignoring the clap 3 update for
what seems like forever since it's still in beta.
2021-06-12 07:54:51 -04:00
Andrew Gallant
0ee85a89f5
deps: update to memmap2
Looking at the changelog for memmap2, the only breaking change was to
MmapOptions, which we don't use. So no migration is needed.
2021-06-12 07:53:42 -04:00
Andrew Gallant
ed9d37959f
deps: updates libc and syn 2021-06-12 07:52:04 -04:00
Andrew Gallant
9f924ee187 msrv: bump to Rust 1.52.1
This matches the latest stable release of Rust.
2021-06-01 21:07:37 -04:00
Andrew Gallant
35c5db6d1a deps: update everything
Removes two dependencies! autocfg and byteorder.
2021-06-01 21:07:37 -04:00
Andrew Gallant
e824531e38 edition: manual changes
This is mostly just about removing 'extern crate' everywhere and fixing
the fallout.
2021-06-01 21:07:37 -04:00
Andrew Gallant
af54069c51 edition: run 'cargo fix --edition --edition-idioms --all' 2021-06-01 21:07:37 -04:00
Andrew Gallant
77a9e99964 edition: set edition=2018 2021-06-01 21:07:37 -04:00
Andrew Gallant
459a9c5637 edition: initial 'cargo fix --edition' run 2021-06-01 21:07:37 -04:00
Andrew Gallant
e4c4540f6a
changelog: fix typo and add Ruby to type improvement list 2021-06-01 11:57:16 -04:00
Ulysse Buonomo
5d0f2b0fc0
ignore/types: config.ru and *.rbw Ruby
PR #1886
2021-06-01 10:57:09 -04:00
Andrew Gallant
079a23b515
changelog: a bit of polish
I think I'm just waiting on the CVE to be published at this point.
2021-06-01 06:59:06 -04:00
Andrew Gallant
6e27649af1
github: add note about file types 2021-06-01 06:26:13 -04:00
Andrew Gallant
df83b8b444 ci: re-work github actions release
This combines the tips from #1820 and the patch submitted in #1675.
The latter wasn't taken as-is because I didn't agree with some of the
changes, and in particular, it removed the ability to easily test the
release on a branch with a dummy tag name. I've tried to add that back
here with the 'rg_version' output. Overall though, using outputs is
indeed much simpler.

Closes #1675, Closes #1820
2021-05-31 21:51:18 -04:00
Andrew Gallant
e48a17e189 changelog: prep for ripgrep 13 release 2021-05-31 21:51:18 -04:00
Andrew Gallant
fbb2cfed28 printer: trim line terminator before doing replacements
This is basically the same bug as #1401, but applied to replacements
instead of --only-matching.

Fixes #1739
2021-05-31 21:51:18 -04:00
Andrew Gallant
af8b27ffae changelog: fish completions are staying
In a previous release, I announced that Fish completions were being
removed. But the Fish project decided to remove theirs and have
ripgrep's stay.

Closes #1577
2021-05-31 21:51:18 -04:00
Martin Pool
8a4071eea9 globset: expand docs and impl Default for GlobSet
Closes #1882, Closes #1883
2021-05-31 21:51:18 -04:00
Andrew Gallant
ee23ab5173 printer: trim line terminator before finding submatches
This fixes a bug where PCRE2 look-around could change the result of a
match if it observed a line terminator in the printer. And in
particular, this is precisely how the searcher operates: the line is
considered unto itself *without* the line terminator.

Fixes #1401
2021-05-31 21:51:18 -04:00
Andrew Gallant
efd9cfb2fc grep: fix bugs in handling multi-line look-around
This commit hacks in a bug fix for handling look-around across multiple
lines. The main problem is that by the time the matching lines are sent
to the printer, the surrounding context---which some look-behind or
look-ahead might have matched---could have been dropped if it wasn't
part of the set of matching lines. Therefore, when the printer re-runs
the regex engine in some cases (to do replacements, color matches, etc
etc), it won't be guaranteed to see the same matches that the searcher
found.

Overall, this is a giant clusterfuck and suggests that the way I divided
the abstraction boundary between the printer and the searcher is just
wrong. It's likely that the searcher needs to handle more of the work of
matching and pass that info on to the printer. The tricky part is that
this additional work isn't always needed. Ultimately, this means a
serious re-design of the interface between searching and printing. Sigh.

The way this fix works is to smuggle the underlying buffer used by the
searcher through into the printer. Since these bugs only impact
multi-line search (otherwise, searches are only limited to matches
across a single line), and since multi-line search always requires
having the entire file contents in a single contiguous slice (memory
mapped or on the heap), it follows that the buffer we pass through when
we need it is, in fact, the entire haystack. So this commit refactors
the printer's regex searching to use that buffer instead of the intended
bundle of bytes containing just the relevant matching portions of that
same buffer.

There is one last little hiccup: PCRE2 doesn't seem to have a way to
specify an ending position for a search. So when we re-run the search to
find matches, we can't say, "but don't search past here." Since the
buffer is likely to contain the entire file, we really cannot do
anything here other than specify a fixed upper bound on the number of
bytes to search. So if look-ahead goes more than N bytes beyond the
match, this code will break by simply being unable to find the match. In
practice, this is probably pretty rare. I believe that if we did a
better fix for this bug by fixing the interfaces, then we'd probably try
to have PCRE2 find the pertinent matches up front so that it never needs
to re-discover them.

Fixes #1412
2021-05-31 21:51:18 -04:00
Andrew Gallant
656aa12649 printer: fix multi-line replacement bug
This commit fixes a subtle bug in multi-line replacement of line
terminators.

The problem is that even though ripgrep supports multi-line searches, it
is *still* line oriented. It still needs to print line numbers, for
example. For this reason, there are various parts in the printer that
iterate over lines in order to format them into the desired output.

This turns out to be problematic in some cases. #1311 documents one of
those cases (with line numbers enabled to highlight a point later):

    $ printf "hello\nworld\n" | rg -n -U "\n" -r "?"
    1:hello?
    2:world?

But the desired output is this:

    $ printf "hello\nworld\n" | rg -n -U "\n" -r "?"
    1:hello?world?

At first I had thought that the main problem was that the printer was
taking ownership of writing line terminators, even if the input already
had them. But it's more subtle than that. If we fix that issue, we get
output like this instead:

    $ printf "hello\nworld\n" | rg -n -U "\n" -r "?"
    1:hello?2:world?

Notice how '2:' is printed before 'world?'. The reason it works this way
is because matches are reported to the printer in a line oriented way.
That is, the printer gets a block of lines. The searcher guarantees that
all matches that start or end in any of those lines also end or start in
another line in that same block. As a result, the printer uses this
assumption: once it has processed a block of lines, the next match will
begin on a new and distinct line. Thus, things like '2:' are printed.

This is generally all fine and good, but an impedance mismatch arises
when replacements are used. Because now, the replacement can be used to
change the "block of lines" approach. Now, in terms of the output, the
subsequent match might actually continue the current line since the
replacement might get rid of the concept of lines altogether.

We can sometimes work around this. For example:

    $ printf "hello\nworld\n" | rg -U "\n(.)?" -r '?$1'
    hello?world?

Why does this work? It's because the '(.)' after the '\n' causes the
match to overlap between lines. Thus, the searcher guarantees that the
block sent to the printer contains every line.

And there in lay the solution: all we need to do is tweak the multi-line
searcher so that it combines lines with matches that directly adjacent,
instead of requiring at least one byte of overlap. Fixing that solves
the issue above. It does cause some tests to fail:

* The binary3 test in the searcher crate fails because adjacent line
  matches are now one part of block, and that block is scanned for
  binary data. To preserve the essence of the test, we insert a couple
  dummy lines to split up the blocks.
* The JSON CRLF test. It was testing that we didn't output any messages
  with an empty 'submatches' array. That is indeed still the case. The
  difference is that the messages got combined because of the adjacent
  line merging behavior. This is a slight change to the output, but is
  still correct.

Fixes #1311
2021-05-31 21:51:18 -04:00
Andrew Gallant
fc31aedcf3 printer: vimgrep now only prints one line
It turns out that the vimgrep format really only wants one line per
match, even when that match spans multiple lines.

We continue to support the previous behavior (print all lines in a
match) in the `grep-printer` crate. We add a new option to enable the
"only print the first line" behavior, and unconditionally enable it in
ripgrep. We can do that because the option has no effect in single-line
mode, since, well, in that case matches are guaranteed to span one line
anyway.

Fixes #1866
2021-05-31 21:51:18 -04:00
Anthony Huang
578e1992fa cli: add --field-{context,match}-separator flags
These flags permit configuring the bytes used to delimit fields in match
or context lines, where "fields" are things like the file path, line
number, column number and the match/context itself.

Fixes #1842, Closes #1871
2021-05-31 21:51:18 -04:00
Austin Wise
46d0130597 cargo: statically link binary on Windows/MSVC
Before this change, rg.exe depended on vcruntime140.dll, which does not
exist on a fresh install of Windows.

Closes #1613
2021-05-31 21:51:18 -04:00
Andres Suarez
7534d5144f globset: fix recursive suffix over matching
Previous, 'foo/**' would match 'foo', but it shouldn't have. In this
case, not matching 'foo' is what is documented and also seems consistent
with other recursive globbing implementations (like that in zsh).

This also updates the prefix extractor to pull 'foo/' out of 'foo/**'.

Closes #1756
2021-05-31 21:51:18 -04:00
Richard Khoury
a28e664abd ignore: check ignore rules before issuing stat calls
This seems like an obvious optimization but becomes critical when
filesystem operations even as simple as stat can result in significant
overheads; an example of this was a bespoke filesystem layer in Windows
that hosted files remotely and would download them on-demand when
particular filesystem operations occurred. Users of this system who
ensured correct file-type fileters were being used could still get
unnecessary file access resulting in large downloads.

Fixes #1657, Closes #1660
2021-05-31 21:51:18 -04:00
Pen Tree
0ca96e004c printer: fix context bug when --max-count is used
In the case where after-context is requested with a match count limit,
we need to be careful not to reset the state tracking the remaining
context lines.

Fixes #1380, Closes #1642
2021-05-31 21:51:18 -04:00
Alessandro Menezes
2295061e80 searcher: do UTF-8 BOM sniffing like UTF-16
Previously, we were only looking for the UTF-16 BOM for determining
whether to do transcoding or not. But we should also look for the UTF-8
BOM as well.

Fixes #1638, Closes #1697
2021-05-31 21:51:18 -04:00
Raimon Grau
53c4855517 ignore/types: add red
See: https://www.red-lang.org/

Closes #1663
2021-05-31 21:51:18 -04:00
Simon Morgan
121e0135c1 ignore/types: replace duplicate glob with *.aspx.vb
*.aspx.cs was listed twice and the VB variant is missing.

Closes #1683
2021-05-31 21:51:18 -04:00
tillyboy
c53c4c0ade doc: explain ignore rules a bit more
Closes #1600
2021-05-31 21:51:18 -04:00
João Marcos
4566882521 cli: add -. as short option for --hidden
This is somewhat non-standard, but it seems nice on the surface: short
flag names are in short supply, --hidden is probably somewhat common and
-. has an obvious connection with how hidden files are named on Unix.

Closes #1680
2021-05-31 21:51:18 -04:00