prek/docs/proposals/concurrency.md

Priority-based parallel hook execution

This document outlines the design for parallel hook execution using explicit priority levels.

Motivation

By default, prek executes hooks sequentially. While safe, this is inefficient for independent tasks (e.g., linting different languages). This proposal introduces per-hook priorities to allow safe, parallel execution of hooks.

Configuration

Hook Configuration: priority

A new optional field priority is added to the hook configuration.

- id: cargo-fmt
  priority: 10

• Type: u32
• Default: None (auto-populated by hook index)

When priority is omitted, the scheduler assigns the hook a priority equal to its index in the configuration file, starting at 0. This preserves the current sequential behavior by giving each hook a unique, increasing priority by default.

Execution Model

Execution is driven purely by priority numbers:

Scope

priority is global within a single configuration file. That is, priorities are compared across all hooks in the same .pre-commit-config.yaml, even if the hooks live under different repos: entries.

priority does not apply across different .pre-commit-config.yaml files (or separate prek runs with different configs). Each config file is scheduled independently.

1. Ordering: Hooks run from the lowest priority value to the highest.
2. Concurrency: Hooks that share the same priority execute concurrently, subject to the global concurrency limit (default: number of CPUs).
3. Defaults: Without explicit priorities, each hook receives a unique priority derived from its position, so execution remains sequential and backwards-compatible.
4. Conflicts: If two hooks intentionally share a priority, they will be run in parallel. Users are responsible for assigning priorities that match their desired grouping.

require_serial Clarification

The existing require_serial configuration key often causes confusion. In this design, its meaning is strictly scoped:

• require_serial: true: Controls invocation concurrency for that hook. When running a hook against files, prek limits that hook to a single in-flight invocation at a time. This effectively disables running multiple batches of the same hook concurrently.
  • prek will still try to pass all files in one invocation, but may split into multiple invocations if the OS command-line length limit would be exceeded.
• It does NOT imply exclusive execution. A hook with require_serial: true can still run in parallel with other hooks that share its priority.
• If a hook must run alone (e.g., it modifies global state), it should be assigned a unique priority value that no other hook uses.

Design Considerations

Mixing Explicit and Implicit Priorities

Implicit priorities are always derived from the hook's position in the configuration (0-based), regardless of any explicitly configured priorities on other hooks.

Positions are taken from the fully flattened hook list for the current .pre-commit-config.yaml, in the order hooks appear as repos: and hooks: are read. In other words, implicit priorities are assigned across the whole file, not per-repo.

Example:

• Hook at index 0 with no priority gets implicit priority 0.
• Hook at index 1 with priority: 10 keeps priority 10.
• Hook at index 2 with no priority gets implicit priority 2.

This means a later hook with an implicit priority can run before an earlier hook that was assigned a larger explicit priority.

If you want to avoid surprises when introducing explicit priorities, prefer setting priority on all hooks (or at least on every hook whose relative order matters).

Grouped Output

If files are modified during a parallel priority group, prek can only tell that one or more hooks in the group made changes (not which one). In this case, prek prints a grouped tree for the whole priority group and marks the group as failed.

Example:

  Files were modified by following hooks...................................Failed
    ┌ Modifies File........................................................Passed
    │ Prints Output........................................................Passed
    └ No Output............................................................Passed
  Later Hook...............................................................Passed

Fail Fast

If fail_fast is enabled:

• If a hook fails, prek should wait for currently running hooks with the current priority to finish, but abort the execution of higher-priority groups.

Example Configuration

repos:
  - repo: local
    hooks:
      - id: cargo-fmt
        name: Format Rust
        entry: cargo fmt
        language: system
        priority: 0  # Runs first

  # These hooks are in different repos, but share the same priority,
  # so they can run concurrently.
  - repo: local
    hooks:
      - id: ruff
        name: Lint Python
        entry: ruff check
        language: system
        priority: 10

  - repo: local
    hooks:
      - id: shellcheck
        name: Lint Shell
        entry: shellcheck
        language: system
        priority: 10

  - repo: local
    hooks:
      - id: integration-tests
        name: Integration Tests
        entry: just test
        language: system
        priority: 20 # Starts after priority=10 group completes