mirror of
https://github.com/BurntSushi/ripgrep.git
synced 2025-01-19 05:49:14 +02:00
656aa12649
This commit fixes a subtle bug in multi-line replacement of line terminators. The problem is that even though ripgrep supports multi-line searches, it is *still* line oriented. It still needs to print line numbers, for example. For this reason, there are various parts in the printer that iterate over lines in order to format them into the desired output. This turns out to be problematic in some cases. #1311 documents one of those cases (with line numbers enabled to highlight a point later): $ printf "hello\nworld\n" | rg -n -U "\n" -r "?" 1:hello? 2:world? But the desired output is this: $ printf "hello\nworld\n" | rg -n -U "\n" -r "?" 1:hello?world? At first I had thought that the main problem was that the printer was taking ownership of writing line terminators, even if the input already had them. But it's more subtle than that. If we fix that issue, we get output like this instead: $ printf "hello\nworld\n" | rg -n -U "\n" -r "?" 1:hello?2:world? Notice how '2:' is printed before 'world?'. The reason it works this way is because matches are reported to the printer in a line oriented way. That is, the printer gets a block of lines. The searcher guarantees that all matches that start or end in any of those lines also end or start in another line in that same block. As a result, the printer uses this assumption: once it has processed a block of lines, the next match will begin on a new and distinct line. Thus, things like '2:' are printed. This is generally all fine and good, but an impedance mismatch arises when replacements are used. Because now, the replacement can be used to change the "block of lines" approach. Now, in terms of the output, the subsequent match might actually continue the current line since the replacement might get rid of the concept of lines altogether. We can sometimes work around this. For example: $ printf "hello\nworld\n" | rg -U "\n(.)?" -r '?$1' hello?world? Why does this work? It's because the '(.)' after the '\n' causes the match to overlap between lines. Thus, the searcher guarantees that the block sent to the printer contains every line. And there in lay the solution: all we need to do is tweak the multi-line searcher so that it combines lines with matches that directly adjacent, instead of requiring at least one byte of overlap. Fixing that solves the issue above. It does cause some tests to fail: * The binary3 test in the searcher crate fails because adjacent line matches are now one part of block, and that block is scanned for binary data. To preserve the essence of the test, we insert a couple dummy lines to split up the blocks. * The JSON CRLF test. It was testing that we didn't output any messages with an empty 'submatches' array. That is indeed still the case. The difference is that the messages got combined because of the adjacent line merging behavior. This is a slight change to the output, but is still correct. Fixes #1311
grep-searcher
A high level library for executing fast line oriented searches. This handles things like reporting contextual lines, counting lines, inverting a search, detecting binary data, automatic UTF-16 transcoding and deciding whether or not to use memory maps.
Dual-licensed under MIT or the UNLICENSE.
Documentation
NOTE: You probably don't want to use this crate directly. Instead, you
should prefer the facade defined in the
grep
crate.
Usage
Add this to your Cargo.toml:
[dependencies]
grep-searcher = "0.1"
and this to your crate root:
extern crate grep_searcher;