Skip to content

feat: Add new thread_pool component built on top of Task#673

Merged
guo-max merged 24 commits into
mainfrom
feat/thread_pool
Jul 24, 2026
Merged

feat: Add new thread_pool component built on top of Task#673
guo-max merged 24 commits into
mainfrom
feat/thread_pool

Conversation

@guo-max

@guo-max guo-max commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Description

add Thread_pool component

Motivation and Context

A thread pool that dispatches submitted jobs to a fixed set of worker threads.

How has this been tested?

an example is added and tested.
The example was tested on an esp32 board. it use a thread_pool with 2 works and finished 8 jobs.

Screenshots (if appropriate, e.g. schematic, board, console logs, lab pictures):

image

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update
  • Hardware (schematic, board, system design) change
  • Software change

Checklist:

  • My change requires a change to the documentation.
  • I have added / updated the documentation related to this change via either README or WIKI

Software

  • I have added tests to cover my changes.
  • I have updated the .github/workflows/build.yml file to add my new test to the automated cloud build github action.
  • All new and existing tests passed.
  • My code follows the code style of this project.

Hardware

  • I have updated the design files (schematic, board, libraries).
  • I have attached the PDFs of the SCH / BRD to this PR
  • I have updated the design output (GERBER, BOM) files.

Copilot AI review requested due to automatic review settings July 22, 2026 14:14
@github-actions

Copy link
Copy Markdown

✅Static analysis result - no issues found! ✅

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new thread_pool component to ESPP, providing an espp::ThreadPool API backed by espp::Task workers, plus an ESP-IDF example project demonstrating job submission and basic stats reporting.

Changes:

  • Introduces espp::ThreadPool implementation with a configurable worker count, optional bounded queue, and submission/statistics APIs.
  • Adds an ESP-IDF example (components/thread_pool/example) showing how to run multiple jobs and wait for completion.
  • Adds component packaging metadata (idf_component.yml) and CMake integration for the new component.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
components/thread_pool/src/thread_pool.cpp Implements worker lifecycle, queueing, and worker callback behavior.
components/thread_pool/include/thread_pool.hpp Public API, configuration, and documentation/comments for ThreadPool.
components/thread_pool/README.md Component-level README and feature list.
components/thread_pool/idf_component.yml Component Manager manifest (metadata, docs URL, dependencies).
components/thread_pool/CMakeLists.txt Registers the component with ESP-IDF build system.
components/thread_pool/example/CMakeLists.txt Example project CMake configuration and component selection.
components/thread_pool/example/main/CMakeLists.txt Registers the example’s main component.
components/thread_pool/example/main/thread_pool_example.cpp Example app demonstrating pool usage and completion waiting.
components/thread_pool/example/README.md Example build/run instructions.
components/thread_pool/example/sdkconfig.defaults Enables C++ exceptions for the example build.

Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/include/thread_pool.hpp
Comment thread components/thread_pool/include/thread_pool.hpp Outdated
Comment thread components/thread_pool/idf_component.yml
Comment thread components/thread_pool/README.md Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 22, 2026 15:18
guo-max and others added 2 commits July 22, 2026 10:19
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

components/thread_pool/README.md:12

  • README claims stop "drains queued jobs", but ThreadPool::stop() explicitly warns that queued jobs may not be executed (and the current implementation can leave queued jobs unprocessed). Please update this feature bullet to match the actual stop semantics so users aren’t misled.
- Graceful stop (stops workers; queued jobs may not be executed)

components/thread_pool/idf_component.yml:8

  • documentation points to core/thread_pool.html, but there are no corresponding doc/en/** pages and no doc/Doxyfile INPUT/EXAMPLE_PATH entries for this new component. This will produce a broken documentation link and omit the API/example from the published docs; please add the required doc pages and Doxyfile entries (kept in alphabetical order) so the link resolves.
documentation: "https://esp-cpp.github.io/espp/core/thread_pool.html"

Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/example/CMakeLists.txt Outdated
Comment thread components/thread_pool/CMakeLists.txt
Copilot AI review requested due to automatic review settings July 22, 2026 15:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 8 comments.

Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/include/thread_pool.hpp Outdated
Comment thread components/thread_pool/include/thread_pool.hpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/example/main/thread_pool_example.cpp Outdated
Copilot AI review requested due to automatic review settings July 22, 2026 15:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.

Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Copilot AI review requested due to automatic review settings July 22, 2026 16:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.

Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/idf_component.yml
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 22, 2026 16:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.

Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/include/thread_pool.hpp
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/idf_component.yml
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Copilot AI review requested due to automatic review settings July 22, 2026 16:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (1)

doc/en/core/thread_pool_example.md:3

  • The include path looks incorrect relative to doc/en/core/thread_pool_example.md. ../../components/... resolves under doc/components/..., not <repo_root>/components/.... Update the include path (likely to ../../../components/thread_pool/example/README.md) so docs builds can actually find the example README.

Comment thread components/thread_pool/example/main/thread_pool_example.cpp Outdated
Comment thread components/thread_pool/example/main/thread_pool_example.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Copilot AI review requested due to automatic review settings July 23, 2026 20:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (3)

components/thread_pool/src/thread_pool.cpp:60

  • Typos/grammar in comments: "thread should stopped" and "heres". These comments are user-facing in the source and should be corrected for clarity.
    queue_has_work_cv_.notify_all();
    queue_has_space_cv_.notify_all(); // thread should stopped after notify all.
    for (auto &worker : workers_) {
      worker->stop(); // stop and join heres
    }

components/thread_pool/include/thread_pool.hpp:62

  • Docstring grammar: "true if at all workers" / "and return true" reads incorrectly and is likely a typo.
  /// @brief Start all worker threads.
  /// @return true if at all workers were successfully started, false otherwise.
  /// @note No-op if the pool is already running and return true immediately.
  /// @note If any workers could not be started, the pool will roll back to the stopped state.

components/thread_pool/example/main/thread_pool_example.cpp:136

  • This test assumes max_queue_size includes the currently executing job ("1 executing + 2 queued"), but the implementation enforces the queue size only. As written, the first 3 try_submit() calls can be nondeterministically rejected depending on whether the worker has dequeued the first job yet.
    // 1 slow worker, capacity 2: 1 executing + 2 queued = 3 slots before rejection
    espp::ThreadPool pool({
        .worker_count = 1,

Comment thread doc/Doxyfile
Comment thread .github/workflows/upload_components.yml
Comment thread .github/workflows/build.yml
Copilot AI review requested due to automatic review settings July 23, 2026 20:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 8 comments.

Comments suppressed due to low confidence (1)

doc/en/core/thread_pool_example.md:3

  • The MyST {include} directive must start at column 0, otherwise Sphinx may treat it as a code block and the include won’t execute. Remove the leading indentation so the directive is parsed correctly.

Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/idf_component.yml
Comment thread components/thread_pool/include/thread_pool.hpp
Comment thread components/thread_pool/include/thread_pool.hpp
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Copilot AI review requested due to automatic review settings July 23, 2026 21:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

components/thread_pool/src/thread_pool.cpp:44

  • The warning message in start() is overly specific ("insufficient memory"), but Task::start() can also fail for other reasons (e.g., invalid core_id or too-small stack). This can mislead debugging; use a more general reason in the log message.
    if ((!workers_[i]->start()) && (!workers_[i]->is_running())) {
      logger_.warn("Failed to start worker {} (already started or insufficient memory)", i);

components/thread_pool/src/thread_pool.cpp:60

  • The inline comments here contain typos and are also a bit misleading (notify_all() does not itself stop threads). Clarify the intent so future changes don’t accidentally reorder the notify/stop sequence incorrectly.
    queue_has_space_cv_.notify_all(); // thread should stopped after notify all.
    for (auto &worker : workers_) {
      worker->stop(); // stop and join heres
    }

components/thread_pool/example/main/thread_pool_example.cpp:18

  • wait_for_jobs() uses std::condition_variable with an atomic predicate that is updated without holding the associated mutex. This can miss the final notification (lost wakeup) and block forever. Using a timed wait loop avoids deadlock without having to restructure all the producer-side increments.
static void wait_for_jobs(std::condition_variable &cv, std::mutex &mtx, std::atomic<int> &completed,
                          int expected) {
  std::unique_lock<std::mutex> lock(mtx);
  cv.wait(lock, [&]() { return completed.load() >= expected; });
}

Copilot AI review requested due to automatic review settings July 24, 2026 03:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

components/thread_pool/include/thread_pool.hpp:35

  • Stats::rejected currently documents both rejected submissions and jobs dropped from the queue during stop(), which makes the meaning ambiguous for API consumers reading the counters. Clarify that rejected includes both rejected submissions and dropped queued jobs on stop (or consider splitting into separate counters, but at least document it clearly).
  struct Stats {
    std::uint64_t submitted = 0; ///< Total jobs accepted into the queue.
    std::uint64_t executed = 0;  ///< Total jobs successfully executed.
    std::uint64_t rejected = 0;  ///< Total jobs rejected (invalid job, stopped/stopping, or queue full) or dropped (due to stop, the enqueued jobs were dropped).
  };

components/thread_pool/include/thread_pool.hpp:43

  • block_on_submit_when_full=true can deadlock if submit() is called from within a worker job on a bounded queue (e.g., worker_count==1 and queue full, or all workers block trying to submit). Please add an explicit warning in the config field docs so users know to prefer try_submit() (or an unbounded queue) from within jobs.
    std::size_t max_queue_size = 0;            ///< Maximum pending jobs (0 = unbounded).
    bool auto_start = true;                    ///< Start workers immediately on construction.
    bool block_on_submit_when_full = false;    ///< If true, submit() blocks when the queue is full instead of rejecting.
    espp::Task::BaseConfig worker_task_config = { ///< Base configuration applied to every worker task.

Comment thread components/thread_pool/README.md
Comment thread components/thread_pool/example/README.md
Copilot AI review requested due to automatic review settings July 24, 2026 14:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

doc/en/core/thread_pool_example.md:3

  • The include path looks incorrect relative to doc/en/core/thread_pool_example.md. From doc/en/core/, ../../components/... resolves to doc/components/... (likely non-existent). Adjust the relative path so it resolves to the repository root components/thread_pool/example/README.md (e.g., ../../../components/thread_pool/example/README.md) to prevent the docs build from failing.

Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/include/thread_pool.hpp
Comment thread components/thread_pool/include/thread_pool_format_helpers.hpp
Comment thread components/thread_pool/src/thread_pool.cpp
Comment thread components/thread_pool/src/thread_pool.cpp Outdated
Comment thread components/thread_pool/src/thread_pool.cpp
Copilot AI review requested due to automatic review settings July 24, 2026 15:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Comment thread components/thread_pool/include/thread_pool.hpp
Comment thread components/thread_pool/example/main/thread_pool_example.cpp
Comment thread components/thread_pool/src/thread_pool.cpp

@finger563 finger563 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, thanks for making this component! A minor update I might suggest is that the first / primary code snippet / example code for the class should be standard use (not one of the tests) so that it's easier for others to read / follow how they should expect to use it and it's easier for them to copy into their code as a starting point. For that purpose, I might suggest adding another section in the example which is simple use showing with straightforward code, and that section is the first snippet / example in the docstring.

@guo-max
guo-max merged commit 0a86553 into main Jul 24, 2026
132 checks passed
@guo-max
guo-max deleted the feat/thread_pool branch July 24, 2026 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request thread pool

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants