Repository navigation
Build tests prior to executing them - #4881
Conversation
…ting - BLOCKER: Replace path.relative() with executableTargets lookup to map test program paths to actual CMake target names. The previous approach produced file paths (e.g. 'tests/my_test') instead of target names (e.g. 'my_test'), causing cmake --build --target to fail. - BLOCKER: Fix accmulatedTestList.concat() no-op by using push(...) so tests are properly marked as errored on build failure. - MAJOR: Check return value of recursive getTestTargets() call so child failures propagate correctly to the caller. - MINOR: Add guard for empty testProgram() return to prevent nonsensical build targets from empty strings. - MINOR: Fix typo accmulatedTestList -> accumulatedTestList. - MINOR: Remove extra whitespace in for statement and redundant .toString() call. - NIT: Remove trailing semicolon after for...of block. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…or reporting" This reverts commit d548f90.
There was a problem hiding this comment.
Pull request overview
Updates CTest Explorer “build before run” behavior so running/debugging selected tests builds their required executables instead of defaulting to the ALL target, improving usability for partial builds and excluded-from-all tests.
Changes:
- Add logic in
CTestDriverto collect test executables perCMakeProjectand invoke targeted builds. - Map test executable paths back to CMake target names using
project.executableTargets(with a filename-based fallback). - Add a CHANGELOG entry describing the new build-before-test behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 13 comments.
| File | Description |
|---|---|
src/ctest.ts |
Adds target discovery and per-project targeted build step before running/debugging tests. |
CHANGELOG.md |
Documents the user-visible behavior change for build-before-run in the test workflow. |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…-cmake-tools into building-tests2
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…-cmake-tools into building-tests2
|
I have some feedback :) Here is what I'm thinking needs to be addressed! Full-project build still happensIn vscode-cmake-tools/src/ctest.ts Lines 1888 to 1907 in 3cf32d9 So "Run Test" ends up doing:
Lines ~1880–1905 should be removed so
|
… are built rather than the entire project
Running a single test from the inline Test CodeLens or the project outline "Run Test" went through preTest, which builds the default (or ALL) target. That is slower than necessary and, when a specific default build target is configured, could leave the test's executable unbuilt so the test could not run. This extends the targeted build-before-run added in #4881 (issue #4515) -- previously wired only into the Test Explorer run/debug handlers -- to the single-test entry points. - Add CTestDriver.getTestBuildTargets(), reusing the same test-program -> executable-target mapping as buildTestTargets (extracted into a shared executableTargetNamesByPath helper). It returns no targets for test commands that are not CMake executable targets (e.g. python, cmake -E, or a nested ctest build-and-test). - Thread an optional buildTargets list through CMakeProject.ctest() and preTest(); runTest() resolves the test's target and passes it, falling back to the default build when nothing resolves. All other ctest/preTest callers pass nothing and keep the existing default-build behavior.
* Fix Test Explorer and CodeLens test runs spinning forever
A test run launched from the Test Explorer or the inline Test CodeLens
created a vscode TestRun but only called run.end() on the happy path.
Any throw between createTestRun and run.end() (e.g. a build or ctest
failure) orphaned the run, so the progress spinner kept spinning
indefinitely even though the work had finished.
- Guarantee run.end() via try/finally at all three createTestRun sites
(runCTest, runTestHandler, debugTestHandler). runCTest keeps
propagating errors so task/workflow callers are unaffected; the two
run-profile handlers log and finalize.
- Remove the now-redundant run.end() in coverageCTestHelper (the caller
owns finalization) to avoid a double end().
- Await the run in the CodeLens handlers so failures are logged and
surfaced instead of escaping silently, and route to the project that
owns the test.
- Never collapse a single-test run to a match-everything regex when the
full set of tests is unknown (empty superset); use exact anchored
matches so one test can no longer trigger the whole suite.
- Log a warning when a requested test matches no discovered test.
* Report a clear reason when a CTest run produces no output
When a test was started but CTest returned no matching result -- for
example the test executable was not built (so CTest reports it as
"Not Run") or the run matched nothing -- the test was finalized with no
output attached, so VS Code showed a bare "The test case did not report
any output." This was especially confusing for projects like fmt whose
test executables are not built by the default target.
- Add reportMissingTestResult() to attach an actionable message ("the
test executable may not have been built -- build the test target,
then run again") to both the test output and its error state, in the
non-parallel path and for explicitly targeted tests in the parallel
path. Tests that genuinely ran and produced no output are unchanged.
- Harden CTest results parsing: decodeOutputMeasurement tolerates
missing/empty values and failed decompression, and cleanupResultsXml
no longer throws when a single test lacks Measurement/NamedMeasurement
(which previously discarded the results of the entire run). Compressed
(base64+gzip) test output continues to be decoded.
- Add a unit test and fixture covering compressed, missing and empty
measurements.
* Build only the test's own target for single-test runs
Running a single test from the inline Test CodeLens or the project
outline "Run Test" went through preTest, which builds the default (or
ALL) target. That is slower than necessary and, when a specific default
build target is configured, could leave the test's executable unbuilt so
the test could not run. This extends the targeted build-before-run added
in #4881 (issue #4515) -- previously wired only into the Test Explorer
run/debug handlers -- to the single-test entry points.
- Add CTestDriver.getTestBuildTargets(), reusing the same test-program ->
executable-target mapping as buildTestTargets (extracted into a shared
executableTargetNamesByPath helper). It returns no targets for test
commands that are not CMake executable targets (e.g. python, cmake -E,
or a nested ctest build-and-test).
- Thread an optional buildTargets list through CMakeProject.ctest() and
preTest(); runTest() resolves the test's target and passes it, falling
back to the default build when nothing resolves. All other ctest/preTest
callers pass nothing and keep the existing default-build behavior.
…rget (#5094) Building tests before running them (#4881, for #4515) replaced the previous per-folder default build with a targeted build of just the executable targets behind the selected tests. When a test's CTest command is a wrapper program rather than a CMake executable target -- "/usr/bin/bash" "-c" "mpiexec ...", mpiexec, a Python launcher, or "${CMAKE_COMMAND} -E env ..." -- the program cannot be mapped to a target, so it was dropped from the build list. If that left no targets at all, buildTestTargets skipped the build entirely and CTest ran against a stale or missing executable, which made cmake.buildBeforeRun ineffective for those tests. Such a test still normally runs something the project builds, so a selection that cannot be fully resolved now falls back to the project's default build instead of building nothing. Selections whose test programs all map to executable targets keep the targeted build introduced by #4881.
…rget (#5094) Building tests before running them (#4881, for #4515) replaced the previous per-folder default build with a targeted build of just the executable targets behind the selected tests. When a test's CTest command is a wrapper program rather than a CMake executable target -- "/usr/bin/bash" "-c" "mpiexec ...", mpiexec, a Python launcher, or "${CMAKE_COMMAND} -E env ..." -- the program cannot be mapped to a target, so it was dropped from the build list. If that left no targets at all, buildTestTargets skipped the build entirely and CTest ran against a stale or missing executable, which made cmake.buildBeforeRun ineffective for those tests. Such a test still normally runs something the project builds, so a selection that cannot be fully resolved now falls back to the project's default build instead of building nothing. Selections whose test programs all map to executable targets keep the targeted build introduced by #4881.
…rget (#5094) Building tests before running them (#4881, for #4515) replaced the previous per-folder default build with a targeted build of just the executable targets behind the selected tests. When a test's CTest command is a wrapper program rather than a CMake executable target -- "/usr/bin/bash" "-c" "mpiexec ...", mpiexec, a Python launcher, or "${CMAKE_COMMAND} -E env ..." -- the program cannot be mapped to a target, so it was dropped from the build list. If that left no targets at all, buildTestTargets skipped the build entirely and CTest ran against a stale or missing executable, which made cmake.buildBeforeRun ineffective for those tests. Such a test still normally runs something the project builds, so a selection that cannot be fully resolved now falls back to the project's default build instead of building nothing. Selections whose test programs all map to executable targets keep the targeted build introduced by #4881.
Original PR: #4523
Addresses the following issue: #4515
This PR
buildTestTargets()to correctly compute target name