error continuing a run / help me understand namelist changes

[jennykowalcz]

I ran into an error trying to continue a run from a few weeks ago, and as I looked into it, I realized there are fundamental things I don't understand. I'm hoping you all can enlighten me.

TLDR: when I submit a run, it appears that it's looking at whatever cime version I've currently checked out in my local E3SM repository, not the cime version I had checked out when I created the case. Is this how things are supposed to work?

The error continuing the run that ran successfully (including several restarts) as of early March was:

(lnd_init_mct) :ELM land model initialization
 524b75405c
 
 Attempting to initialize the land model .....
 
 Attempting to initialize run control settings .....
 Read in elm_inparm namelist from: lnd_in
 ENDRUN:
 ERROR reading elm_inparm namelistERROR in controlMod.F90 at line 370           
                                                                                
                                                                                
                                                                                
                                                                                
                                                                                
                                                                        
 ERROR: Unknown error submitted to shr_abort_abort.

Looking at lnd_in in the run directory, I saw that new parameters had been introduced when I tried to continue the run:

t/run> diff lnd_in lnd_in.prev
15d14
<  fluh_timeseries = '/global/cfs/cdirs/e3sm/inputdata/lnd/clm2/surfdata_map/fates-sci.1.68.3_api.31.0.0_tools.1.0.1/LUH2_states_transitions_management.timeseries_4x5_hist_simyr1850-2015_c231101.nc'
44d42
<  use_fates_luh = .false.

By switching between checking out different versions of E3SM/cime in my local repository and running preview_namelists for this case, it seems like the namelists are generated by whatever version I have currently checked out, not the version that was checked out when I created the case. Is this how it's supposed to work?

[glemieux]

I'm not well versed in manually restarting cases, but you've got the right of it @jennykowalcz. The land model will create symbolic links to the CIME directory inside the host land model when you create a case to access scripts like case.submit that have their home there instead of copying the scripts whole sale.

I feel like there has to be a way to take a case build and restart it without having to keep track of which commit your host land model repo is checkout to, but I'm not sure. I'll ask around.

Great question btw!

[jennykowalcz]

Thanks @glemieux! Sounds like it'd be a good idea to have different local HLM repositories for different CIME versions.