We can just ask the channel whether any work has been loaded. Normally
querying a channel for its length is a strong predictor of bugs, but in
this case, we do it before we ever attempt a `recv`, so it should work.
Kudos to @zsugabubus for suggesting this!
It turns out that the previous version wasn't quite correct. Namely, it
was possible for the following sequence to occur:
1. Consider that all workers, except for one, are `waiting`.
2. The last remaining worker finds one more job to do and sends it on
the channel.
3. One of the previously `waiting` workers wakes up from the job that
the last running worker sent, but `self.resume()` has not been
called yet.
4. The last worker, from (2), calls `get_work` and sees that the
channel has nothing on it, so it executes `self.waiting() ==
1`. Since the worker in (3) hasn't called `self.resume()` yet,
`self.waiting() == 1` evaluates to true.
5. This sets off a chain reaction that stops all workers, despite that
fact that (3) got more work (which could itself spawn more work).
The end result is that the traversal may terminate while their are still
outstanding work items to process. This problem was observed through
spurious failures in CI. I was not actually able to reproduce the bug
locally.
We fix this by changing our strategy to detect termination using a
counter. Namely, we increment the counter just before sending new work
and decrement the counter just after finishing work. In this way, we
guarantee that the counter only ever reaches 0 once there is no more
work to process.
See #1337 for more discussion. Many thanks to @zsugabubus for helping me
work through this.
The top-level listing was just getting a bit too long for my taste. So
put all of the code in one directory and shrink the large top-level mess
to a small top-level mess.
NOTE: This commit only contains renames. The subsequent commit will
actually make ripgrep build again. We do it this way with the naive hope
that this will make it easier for git history to track the renames.
Sigh.
