ctsm5.4.065: Move CTSM tests from gx grids to t233 grid

[slevis-lmwg]

Description of changes

Similar to #4223 but for g16, g17, g37 grids

• testlist_clm.xml but left grids unchanged that did not have replacements yet, see next checkbox
• still to add to ccs_config
  • ne30_t233 and f19_t233_gris4
  • Erik to merge ccs_config_cesm1.0.93: Add 3 more t233/201 grids needed by CLM tests currently pointing to g17/37 ESMCI/ccs_config_cesm#307 and make new tag ccs_config_cesm1.0.93
  • Update to the new tag here or as separate PR, Erik is ok with either (same comment below for RTM/MOSART)
• anything g37 --> t201
  • check my work so far
  • update unchanged ones
• Sam R. says below that the RXCROPMATURITY* tests need the grids to match, so crop cal. files need updating; do this in this PR or open an issue? UPDATE: The crop cal. files probably do not need updating because they do not care about the ocean grid.
• Update RTM, MOSART grids in rtm1_0_90: Move g17/g37 grids to t233/t201 RTM#69 and mosart1.1.14: Replace g37 grid with t201 in mosart test-suite MOSART#131 and make new tags; see if I can fold other RTM/MOSART issues into this:
  • Remove CLM45 compset from testing as CTSM is deprecating and removing it from compsets RTM#68
  • Route new GLC runoff fluxes, or at least ensure that they're zero for now RTM#52 <-- not done
  • RTM does not have the history_tape_in_use functionality RTM#62 <-- not done
  • Fix logic for ctl%rof_from_glc MOSART#114 <-- not done
  • Remove SMS_D_Ld5.f10_f10_mt201.I1850Clm60Sp.derecho_nvhpc.mosart-default from expected fails MOSART#109 <-- not done
  • Update to the new tags: in .gitmodules update to rtm1_0_90 and mosart1.1.14

Specific notes

Contributors other than yourself, if any:

CTSM issues resolved or otherwise addressed, if any:
Contributes to #2540 (resolves?)

Testing planned or performed, if any:

• aux_clm
  • derecho tests_1012-181808de
  • izumi

Requirements before merge:

• I have followed the CTSM contribution guidelines.

• The code in this PR branch builds with no errors.
• The code in this PR branch runs with no errors. Briefly describe tested configuration(s):
• This does not change answers for unchanged tests; tests going from gx grids to t233 run on the new grid and have their names updated, so the test-suites will not compare them to the baseline
• This PR either (a) does not create a need to update the documentation or (b) includes required documentation updates (see guidelines for contributing documentation). Which?:

[ekluzek]

I'd call this as something that should go on master, because when compsets change you can't verify that things are b4b. And hence it could hide an accidental non-b4b change on the b4b-dev branch after this was merged.

[ekluzek]

We went over this together. And I noted the thing about g37 grids should be t201 as the low resolution MOM grid. And we realized that MOSART and RTM (and ccs_config) also need to be updated similarly.

[slevis-lmwg]

Marking "blocked" due to the checkbox above stating that we need to "add to ccs_config: ne30_t233, f19_t233_gris4 and make new tag"

[slevis-lmwg]

I started ./run_sys_tests -s aux_clm -c ctsm5.4.063 -g ctsm5.4.064_PR4234 on derecho ONLY for now. If all goes well, then we could consider this PR for the next merge/tag ctsm5.4.064.

For reference: tests_1012-181808de

[slevis-lmwg]

./cs.status.fails -c NLCOMP | grep -v 'TED FAILURE' | grep -v 'does not exist' failures to investigate:

FAIL SMS_D_Ld5_PS.f09_f09_mt233.I1850Clm60BgcCrop.derecho_gnu.clm-f09_ObscureStreamOpts BASELINE ctsm5.4.063: DIFF
FAIL LII2FINIDATAREAS_D_P256x2_Ld1.f09_t232.I1850Clm60BgcCrop.derecho_intel.clm-default--clm-matrixcnOn_ignore_warnings COMPARE_base_no_interp
FAIL LII2FINIDATAREAS_D_P256x2_Ld1.f09_t232.I1850Clm60BgcCrop.derecho_intel.clm-default COMPARE_base_no_interp
FAIL LII_D_Ld3_PS.f19_t233.I2000Clm50BgcCrop.derecho_intel.clm-default--clm-matrixcnOn RUN time=253
PEND LII_D_Ld3_PS.f19_t233.I2000Clm50BgcCrop.derecho_intel.clm-default--clm-matrixcnOn COMPARE_base_no_interp
FAIL LII_D_Ld3_PS.f19_t233.I2000Clm50BgcCrop.derecho_intel.clm-default RUN time=235
PEND LII_D_Ld3_PS.f19_t233.I2000Clm50BgcCrop.derecho_intel.clm-default COMPARE_base_no_interp
FAIL LVG_Ld5_D.f10_f10_mt201.I1850Clm60Bgc.derecho_intel.clm-no_vector_output COMPARE_base_more_virtual
FAIL PFS_Ld10_PS.f19_t233.I2000Clm50BgcCrop.derecho_intel.clm-default--clm-matrixcnOn RUN time=33
FAIL PFS_Ld10_PS.f19_t233.I2000Clm50BgcCrop.derecho_intel RUN time=38
FAIL SMS_D_Ld1.ne30pg3_t233.I1850Clm60BgcCropSpinup.derecho_intel.clm-crop BASELINE ctsm5.4.063: DIFF
FAIL SMS_D_Ld3_PS.f09_t233.I1850Clm60BgcNoAnthroRtm.derecho_intel.clm-decStart1851_noinitial--clm-matrixcnOn RUN

• For now revert the LII_* tests to their previous grid, as we did in ctsm5.4.063: Move CTSM tests from t232 to t233 #4223 for the LII2 tests (Update LII_ + PFS tests from g17 to t233 grid when finalizing our ICs #4245).
• I did not expect the LII2* tests to fail as they already point to the previous grid, but the LII2 tests with t232 now set use_init_interp = .true. and init_interp_method = 'use_finidat_areas', while running with the new grid now works!
• I did not expect DIFFs from the baseline
  • SMS_D_Ld5_PS.f09_f09_mt233.I1850Clm60BgcCrop.derecho_gnu.clm-f09_ObscureStreamOpts has many and large DIFFs, while it passes in vanilla ctsm5.4.063; reverting my mods to namelist_defaults_overall.xml STILL fails; the test is missing use_init_interp = .true. and init_interp_method = 'general' WHICH MAY ALSO EXPLAIN SOME OF THE OTHER FAILURES. Claude suggests accepting this DIFF because this test now finds a finidat that does not require interpolation.
  • SMS_D_Ld1.ne30pg3_t233.I1850Clm60BgcCropSpinup.derecho_intel.clm-crop passes in vanilla ctsm5.4.063, and the test DIFF comes from a single field PCT_GLC_MEC; this test is also missing use_init_interp = .true. and init_interp_method = 'general', so ok to accept this DIFF as it now does not require interpolation.
• LVG DIFF comes from a single field PCT_GLC_MEC, and I thought it might be a one-time DIFF due to the change in grid. I ran again, this time against the 064 baseline, but the test still fails in COMPARE_base_more_virtual, so I need to add it to Update LII_ + PFS tests from g17 to t233 grid when finalizing our ICs #4245.
• PFS tests need to revert to their prev. grid, as they do not set use_init_interp to true.

[slevis-lmwg]

@ekluzek in the above list of failures,
SMS_D_Ld3_PS.f09_t233.I1850Clm60BgcNoAnthroRtm.derecho_intel.clm-decStart1851_noinitial--clm-matrixcnOn

• fails consistently with "floating invalid" in line 764 CNSoilMatrixMod.F90, not line 793 of the tests that pass by resubmitting
• its old counterpart with f09_g17 passed in the baseline + again just now
• in rtm's testlist we use f09_f09_mt233 ONLY, but the test still fails with that
• in rtm's testlist we never test Bgc, how come? It's an omission, though we do in aux_clm, so ok
• testing rtm with matrixcnOn seems unnecessary since we use the latter ONLY in spin-up (rivers off), and the test without matrixcnOn passed just now, so I updated this test accordingly (I have not rerun to generate its baseline)
• document this failure in a ctsm issue for the benefit of paleo users
• create issue that we want a new SSP matrixcn test that checks that cpl_hist spinup works with NoAnthro (with and without Rtm)

[slevis-lmwg]

@billsacks I'm updating CTSM tests from the gx grids to t233. I want to ask you about this failure:
FAIL LVG_Ld5_D.f10_f10_mt201.I1850Clm60Bgc.derecho_intel.clm-no_vector_output COMPARE_base_more_virtual
I thought it might be a one-time failure, but rerunning against the new baseline failed the same way (path below). We will keep a short list of tests using the gx grids until we finalize initial conditions (#4245), and I now think that this test falls in that category. Do you agree?

/glade/derecho/scratch/slevis/LVG_Ld5_D.f10_f10_mt201.I1850Clm60Bgc.derecho_intel.clm-no_vector_output.C.20261013_183514_4298nm

[slevis-lmwg]

Since updating submodules... Consider updating CIME for #4239 here?

From meeting with Erik: No, because this PR here is effectively an answer-changing PR.

[slevis-lmwg]

• FAIL... Regarding the LVG test, Erik suggested pointing the test to the finidat generated previously by the test itself to see if it passes; I pointed to the finidat found in the test's case2 (manually reintroducing use_init_interp and related settings as found in the original test, but exice is still different) and rerunning cs.status.20261016_182626_h14wtz
• FAIL... Repeat with the finidat found in the test's case1 (manually reintroducing use_init_interp and related settings as found in the original test, but exice is still different) and rerunning cs.status.20261016_182626_h14wtz
• PASS Confirm that LVG passes vanilla ctsm5.4.063: ./create_test LVG_Ld5_D.f10_f10_mg37.I1850Clm60Bgc.derecho_intel.clm-no_vector_output -c /glade/campaign/cgd/tss/ctsm_baselines/ctsm5.4.063 with cs.status.20261016_183745_cle1ai

[billsacks]

Sorry that I didn't see the question about the LVG test earlier.

I do not expect changing finidat to resolve this.

I checked the test failure, and I see that the only field that differs is PCT_GLC_MEC. I think the right thing to do is to simply exclude this history field from the test, using hist_fexcl. You can do this via append_to_user_nl_files in lvg.py. (You can give a list of strings to append to the user_nl files; do this hist_fexcl for both case1 and case2.)

I think I understand why this is needed: There is a grid cell on the south tip of South America that has 0% glacier which must newly be present within the land mask (I haven't looked at the change in land mask, but I assume that's why it's a problem now). In the base case, this has 0% for each glacier column, but in the case with extra virtual glacier, it must fall within the Antarctica mask, and so it is given non-zero percentages in each column. This is a fine difference and should just be ignored.