95cea77625
This commit fixes two issues. First, the iterator was executing the callback for every child of a directory in a single thread. Therefore, if the walker was run over a single directory, then no parallelism is used. We tweak the iterator slightly so that we don't fall into this trap. The second issue is a bit more subtle. In particular, we don't use the blocking semantics of MsQueue because we *don't know when iteration finishes*. This means that if there are a bunch of idle workers because there is no work available to them, then they will spin and burn the CPU. One case where this crops up is if you pipe the output of ripgrep into `less` *and* the total number of files to search is fewer than the number of threads ripgrep uses. We "fix" this with a very stupid heuristic: when the queue yields no work, we sleep the thread for 1ms. This still pegs the CPU, but not nearly as much as before. If one really want to avoid this behavior when using ripgrep, then `-j1` can be used to disable parallelism. Fixes #258 |
||
|---|---|---|
| .. | ||
| examples | ||
| src | ||
| Cargo.toml | ||
| README.md | ||
ignore
The ignore crate provides a fast recursive directory iterator that respects
various filters such as globs, file types and .gitignore files. This crate
also provides lower level direct access to gitignore and file type matchers.
Dual-licensed under MIT or the UNLICENSE.
Documentation
Usage
Add this to your Cargo.toml:
[dependencies]
ignore = "0.1"
and this to your crate root:
extern crate ignore;
Example
This example shows the most basic usage of this crate. This code will
recursively traverse the current directory while automatically filtering out
files and directories according to ignore globs found in files like
.ignore and .gitignore:
use ignore::Walk;
for result in Walk::new("./") {
// Each item yielded by the iterator is either a directory entry or an
// error, so either print the path or the error.
match result {
Ok(entry) => println!("{}", entry.path().display()),
Err(err) => println!("ERROR: {}", err),
}
}
Example: advanced
By default, the recursive directory iterator will ignore hidden files and
directories. This can be disabled by building the iterator with WalkBuilder:
use ignore::WalkBuilder;
for result in WalkBuilder::new("./").hidden(false).build() {
println!("{:?}", result);
}
See the documentation for WalkBuilder for many other options.