From a6275648b38e9d8bee2e63130008a66d0d1ce98c Mon Sep 17 00:00:00 2001
From: Thomas Otto <th1000s@posteo.net>
Date: Wed, 3 Jul 2024 22:05:09 +0200
Subject: [PATCH] ignore: don't process command line arguments in reverse order
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When searching in parallel with many more arguments than threads, the
first arguments are searched last -- unlike in the -j1 case.

This is unexpected for users who know about the parallel nature of rg
and think they can give the scheduler a hint by positioning larger
input files (L1, L2, ..) before smaller ones (█, ██). Instead, this can
result in sub-optimal thread usage and thus longer runtime (simplified
example with 2 threads):

 T1:  █ ██ █ █ █ █ ██ █ █ █ █ █ ██ ╠═════════════L1════════════╣
 T2:  █ █ ██ █ █ ██ █ █ █ ██ █ █ ╠═════L2════╣

                                       ┏━━━━┳━━━━┳━━━━┳━━━━┓
This is caused by assigning work to    ┃ T1 ┃ T2 ┃ T3 ┃ T4 ┃
 per-thread stacks in a round-robin    ┡━━━━╇━━━━╇━━━━╇━━━━┩
              manner, starting here  → │ L1 │ L2 │ L3 │ L4 │ ↵
                                       ├────├────┼────┼────┤
                                       │ s5 │ s6 │ s7 │ s8 │ ↵
                                       ├────┼────┼────┼────┤
                                       ╷ .. ╷ .. ╷ .. ╷ .. ╷
                                       ├────┼────┼────┼────┤
                                       │ st │ su │ sv │ sw │ ↵
                                       ├────┼────┼────┼────┘
                                       │ sx │ sy │ sz │
                                       └────┴────┴────┘
   and then processing them bottom-up:   ↥    ↥    ↥    ↥

                                       ╷ .. ╷ .. ╷ .. ╷ .. ╷
This patch reverses the input order    ├────┼────┼────┼────┤
so the two reversals cancel each other │ s7 │ s6 │ s5 │ L4 │ ↵
out. Now at least the first N          ├────┼────┼────┼────┘
arguments, N=number-of-threads, are    │ L3 │ L2 │ L1 │
processed before any others (then      └────┴────┴────┘
work-stealing may happen):

 T1:  ╠═════════════L1════════════╣ █ ██ █ █ █ █ █ █ ██
 T2:  ╠═════L2════╣ █ █ ██ █ █ ██ █ █ █ ██ █ █ ██ █ █ █

(With some more shuffling T1 could always be assigned L1 etc., but
that would mostly be for optics).

Closes #2849
---
 CHANGELOG.md              | 2 ++
 crates/ignore/src/walk.rs | 5 ++++-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 891221d6..9ed29327 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,6 +23,8 @@ Feature enhancements:
   Add `italic` to the list of available style attributes in `--color`.
 * [FEATURE #2842](https://github.com/BurntSushi/ripgrep/pull/2842):
   Directories containing `.jj` are now treated as git repositories.
+* [FEATURE #2849](https://github.com/BurntSushi/ripgrep/pull/2849):
+  When using multithreading, schedule files to search in order given on CLI.
 
 
 14.1.1 (2024-09-08)
diff --git a/crates/ignore/src/walk.rs b/crates/ignore/src/walk.rs
index d6ea9c21..03a6e2d0 100644
--- a/crates/ignore/src/walk.rs
+++ b/crates/ignore/src/walk.rs
@@ -1420,8 +1420,11 @@ impl Stack {
                 stealers: stealers.clone(),
             })
             .collect();
-        // Distribute the initial messages.
+        // Distribute the initial messages, reverse the order to cancel out
+        // the other reversal caused by the inherent LIFO processing of the
+        // per-thread stacks which are filled here.
         init.into_iter()
+            .rev()
             .zip(stacks.iter().cycle())
             .for_each(|(m, s)| s.push(m));
         stacks