regex: push more pattern handling to matcher construction

Previously, ripgrep core was responsible for escaping regex patterns and
implementing the --line-regexp flag. This commit moves that
responsibility down into the matchers such that ripgrep just needs to
hand the patterns it gets off to the matcher builder. The builder will
then take care of escaping and all that.

This was done to make pattern construction completely owned by the
matcher builders. With the arrival regex-automata, this means we can
move to the HIR very quickly and then never move back to the concrete
syntax. We can then build our regex directly from the HIR. This overall
can save quite a bit of time, especially when searching for large
dictionaries.

We still aren't quite as fast as GNU grep when searching something on
the scale of /usr/share/dict/words, but we are basically within spitting
distance. Prior to this, we were about an order of magnitude slower.

This architecture in particular lets us write a pretty simple fast path
that avoids AST parsing and HIR translation entirely: the case where one
is just searching for a literal. In that case, we can hand construct the
HIR directly.

crates/pcre2/src/matcher.rs

@@ -11,6 +11,8 @@ pub struct RegexMatcherBuilder {
     builder: RegexBuilder,
     case_smart: bool,
     word: bool,
+    fixed_strings: bool,
+    whole_line: bool,
 }
 
 impl RegexMatcherBuilder {
@@ -20,6 +22,8 @@ impl RegexMatcherBuilder {
             builder: RegexBuilder::new(),
             case_smart: false,
             word: false,
+            fixed_strings: false,
+            whole_line: false,
         }
     }
 
@@ -29,17 +33,40 @@ impl RegexMatcherBuilder {
     /// If there was a problem compiling the pattern, then an error is
     /// returned.
     pub fn build(&self, pattern: &str) -> Result<RegexMatcher, Error> {
+        self.build_many(&[pattern])
+    }
+
+    /// Compile all of the given patterns into a single regex that matches when
+    /// at least one of the patterns matches.
+    ///
+    /// If there was a problem building the regex, then an error is returned.
+    pub fn build_many<P: AsRef<str>>(
+        &self,
+        patterns: &[P],
+    ) -> Result<RegexMatcher, Error> {
         let mut builder = self.builder.clone();
-        if self.case_smart && !has_uppercase_literal(pattern) {
+        let mut pats = Vec::with_capacity(patterns.len());
+        for p in patterns.iter() {
+            pats.push(if self.fixed_strings {
+                format!("(?:{})", pcre2::escape(p.as_ref()))
+            } else {
+                format!("(?:{})", p.as_ref())
+            });
+        }
+        let mut singlepat = pats.join("|");
+        if self.case_smart && !has_uppercase_literal(&singlepat) {
             builder.caseless(true);
         }
-        let res = if self.word {
-            let pattern = format!(r"(?<!\w)(?:{})(?!\w)", pattern);
-            builder.build(&pattern)
-        } else {
-            builder.build(pattern)
-        };
-        res.map_err(Error::regex).map(|regex| {
+        if self.whole_line {
+            singlepat = format!(r"(?m:^)(?:{})(?m:$)", singlepat);
+        } else if self.word {
+            // We make this option exclusive with whole_line because when
+            // whole_line is enabled, all matches necessary fall on word
+            // boundaries. So this extra goop is strictly redundant.
+            singlepat = format!(r"(?<!\w)(?:{})(?!\w)", singlepat);
+        }
+        log::trace!("final regex: {:?}", singlepat);
+        builder.build(&singlepat).map_err(Error::regex).map(|regex| {
             let mut names = HashMap::new();
             for (i, name) in regex.capture_names().iter().enumerate() {
                 if let Some(ref name) = *name {
@@ -144,6 +171,21 @@ impl RegexMatcherBuilder {
         self
     }
 
+    /// Whether the patterns should be treated as literal strings or not. When
+    /// this is active, all characters, including ones that would normally be
+    /// special regex meta characters, are matched literally.
+    pub fn fixed_strings(&mut self, yes: bool) -> &mut RegexMatcherBuilder {
+        self.fixed_strings = yes;
+        self
+    }
+
+    /// Whether each pattern should match the entire line or not. This is
+    /// equivalent to surrounding the pattern with `(?m:^)` and `(?m:$)`.
+    pub fn whole_line(&mut self, yes: bool) -> &mut RegexMatcherBuilder {
+        self.whole_line = yes;
+        self
+    }
+
     /// Enable Unicode matching mode.
     ///
     /// When enabled, the following patterns become Unicode aware: `\b`, `\B`,