Repository navigation
Make the Python unit-test job fail when a test fails - #4236
Merged
ekluzek merged 2 commits intoSep 30, 2026
Merged
Conversation
run_ctsm_py_tests.py called pytest.main() without passing its return code on, so ./run_ctsm_py_tests exited 0 even when tests failed or collection was interrupted, and the python-unit-tests job could not go red. That job also checked out without the cime submodule and without the Git LFS test inputs. Test modules that import CIME failed at collection, pytest stopped before running any test, and the netCDF inputs were LFS pointer files. Check out with lfs: true and run bin/git-fleximod update cime, as docs-common.yml does for the docs build. See ESCOMP#4230.
Member
|
@Santoshkumarpuppala Thanks for your contribution! Would you mind emailing ctsm-support@ucar.edu so we can discuss possible future contributions? |
12 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of changes
python/ctsm/run_ctsm_py_tests.pydropped the return value ofpytest.main(), so./run_ctsm_py_testsexited 0 on failures and thepython-unit-testsjob stayed green. The runner now callssys.exit(pytest.main(pytest_args)).That alone would turn the job red at collection. The checkout had no cime, so 8 test modules failed to import
CIMEand pytest stopped before running a test, and the.ncinputs were LFS pointers. The job now checks out withlfs: trueand runsbin/git-fleximod update cime, the same pattern asdocs-common.yml:24-32(which updates all submodules with-o). Background in #4230.Until #4231 is merged, this job fails on
test_inputdata_setup_files_gswp3_error(it looks for/glade). Merging #4231 first avoids that on b4b-dev.Specific notes
Contributors other than yourself, if any:
CTSM issues resolved or otherwise addressed, if any:
Any user interface changes (namelist or namelist defaults changes)?
No namelist changes.
./run_ctsm_py_testsandmake utestnow exit non-zero when a test fails.Testing planned or performed, if any:
Local only: a depth-1 checkout of this branch on b4b-dev
8e3f959, pip venv withctsm_pylib's test-dependency pins (Python 3.13, pytest 8.3.5), network blocked. No conda, and the workflow itself has not run.make utestwith the failing stub: 0 before, 2 after.collected 558 items / 8 errors, exit 0 before, 2 after.bin/git-fleximod update cimeas written: cime at92ae0fd(cime6.5.5), no other submodule checked out..ncinputs (sha256-checked against the pointers): 602 collected, 5 deselected (below), 596 passed, 1 failed, the gswp3 test. With test(subset_data): make unit test work outside NCAR cluster, like its siblings do #4231's line applied, 597 passed, exit 0. Without the LFS inputs, 35 tests fail on pointer files.--checkonrun_ctsm_py_tests.py.test_check_neon_time,test_get_neon,test_get_neon_false_site,test_check_neon_listing,test_check_neon_listing_misspelled).Requirements before merge:
doc/covers the Python test runner or this job.