Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce isotropic surface stress to MPAS-Ocean #6917

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

proteanplanet
Copy link
Contributor

@proteanplanet proteanplanet commented Jan 19, 2025

Introduces 4-point averaging to ocean surface stress (wind and sea ice stress) around each vector point on the MPAS-Ocean C-grid, instead of averaging only across the two on-edge cell center surface stress points to the vector point. This affects coupling with both the atmosphere and sea ice models, which both couple surface stress through the scalar (cell center) grid point in MPAS-Ocean. In so doing, we ensure that surface stress is independent of the orientation of the mesh relative to the wind vectors or sea ice drift vector, thereby making this approach isotropic.

The code has been tested using both config_bulk_wind_stress_interp_isotropic = true (on) and config_bulk_wind_stress_interp_isotropic = false (off) in fully coupled E3SM pre-industrial 100 and 80 year B-cases, respectively. The change conserves mass and energy and has minimal impact on the overall coupled simulation, but provides more robust forcing numerics for the ocean, with feedbacks to sea ice. Figures supporting this are provided in comments attached to this pull request.

This pull request was authored by @xylar and @proteanplanet combined.

[NML]
[CC]

@proteanplanet
Copy link
Contributor Author

Introduces 4-point averaging to ocean surface stress (wind and sea ice stress) around each vector point in MPAS-Ocean, as per the right panel in the following figure:

Surface_Current_Coupling_MPAS-Ocean

The left panel illustrates the default up to the V3 LR and NARRM configurations, the right panel is intended for the V3 SORRM, HR and BGC configurations, and subsequent configurations of E3SM.

Two tests were conducted with this new code in B-cases, as documented here for the anisotropic and isotropic cases, respectively. Each case was run for at least 90 years with code stability for the changes introduced in this PR. A comparison of the results follows.

@proteanplanet
Copy link
Contributor Author

proteanplanet commented Jan 20, 2025

Impact of Isotropic Surface Stress on the Ocean

One of the largest impacts of introducing isotropic surface ocean stress is in sea ice extent and snow volume. In the lemnisc below, the difference between anisotropic (blue) and isotropic (red) 90-year PI spinups illustrates that both of these sea ice metrics are different at the 95% compatibility interval, where dotted red lines indicate no difference, and solid red lines illustrate a difference within each annual solar cycle. Even though the mean values appear identical, the sea ice climate is different due to variability.

E3SM_sea_ice_lemnisc 0001-0089 0001-0089 windstressold windstressnew icepack extent volume snow 3

There appears is no sizable change to AMOC for the simulations described above, comparing here also against the LR V3 Spinup:

MOC_Comparison_Windstress

However, it appears that the isotropic wind stress case may allow the model to spinup faster for key through-flow statistics:

Southern_Ocean_Transport

While the impact of this change is subtle, it offers more robust numerics in regions of heavily distorted grid cells, as illustrated in the mesh figure in the previous comment above. The change does not degrade the model, and E3SM runs stably with the improved coupling numerics when fully coupled, and it is therefore worth adopting.

@proteanplanet
Copy link
Contributor Author

@mark-petersen, @vanroekel, @cbegeman - this is now ready for your review.

@vanroekel
Copy link
Contributor

@proteanplanet -- the code looks great to me by inspection. Do you want me to do any testing?

@proteanplanet
Copy link
Contributor Author

@proteanplanet -- the code looks great to me by inspection. Do you want me to do any testing?

Thanks @vanroekel, not unless you see a gap in what I've presented. It's already extensively tested in B-cases, so I don't think it needs more testing.

@vanroekel
Copy link
Contributor

I think your B-case testing is great. Two little questions.

  1. did you do the standard developer suite testing (PEM, PET, etc...)?
  2. does the isotropic averaging add any noticeable increase in computational cost? I'm guessing not.

Copy link
Contributor

@mark-petersen mark-petersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked carefully at your results and the code. I agree that a larger stencil to compute surface stress is more robust, and would have little impact on the computational cost since it is only the surface. Thanks for your lengthy simulations and analysis above.

@proteanplanet
Copy link
Contributor Author

2. does the isotropic averaging add any noticeable increase in computational cost?  I'm guessing not.

As a rough comparison, cost of this code with the flag switched on for Chrysalis is 6204.43 pe-hrs/sim_year, 12.87 SYPD for a 25-year simulation, as compared to 6146.00 pe-hrs/sim_year at 13.00 SYPD for the anisotropic case, for an ocean model cost of 17.21 SYPD for the former and 17.43 SYPD for the latter.

In other words, the numerical change is in the noise relative to the climate changes caused by the different approach across the system.

@proteanplanet
Copy link
Contributor Author

proteanplanet commented Jan 23, 2025

1. did you do the standard developer suite testing (PEM, PET, etc...)?

The code passes the e3sm_cryo_developer test suite (PEM, PET, ERS, SMS with ice shelves) with config_bulk_wind_stress_interp_isotropic = true.

The code also passes this test suite when config_bulk_wind_stress_interp_isotropic = false.

Therefore both active and stealth modes pass.

@@ -332,6 +332,7 @@
<!-- forcing -->
<config_use_variable_drag>.false.</config_use_variable_drag>
<config_use_bulk_wind_stress>.true.</config_use_bulk_wind_stress>
<config_bulk_wind_stress_interp_isotropic>.false.</config_bulk_wind_stress_interp_isotropic>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A question for @cbegeman and @vanroekel: Given the extensive testing in B-cases, should we immediately just switch on isotropic wind stress averaging as the default in master, thus changing, this to true.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would support making isotropic default behavior based on your testing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with changing the default to true

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! Just left for @cbegeman to approve.

Copy link
Contributor

@vanroekel vanroekel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving by visual inspection of code and extensive testing from @proteanplanet

This non-BFB change to E3SM has been rigorously tested in an 1850 spinup B-case for more than 100 years, is stable, and produces a similar climate without the change, but offers more robust ocean numerics than the current default.
Copy link
Contributor

@cbegeman cbegeman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving on the basis of code inspection and @proteanplanet's testing. Thanks for identifying and prioritizing this, @proteanplanet!

@proteanplanet
Copy link
Contributor Author

Tagging @xuezhengllnl, @darincomeau and @njeffery - this is ready for HR, SORRM, and HES configs.

@njeffery
Copy link
Contributor

@proteanplanet : The icepack version on your branch is 8fad768] which is quite old and doesn't have the Icepack bgc. I'm planning on doing a coupled bgc run using the latest 22b8769 to compare against my baseline. Just want to confirm that that's correct.

I'm also planning on using config_bulk_wind_stress_interp_isotropic = true . Are there any other changes I should be aware of?

@proteanplanet
Copy link
Contributor Author

@proteanplanet : The icepack version on your branch is 8fad768] which is quite old and doesn't have the Icepack bgc. I'm planning on doing a coupled bgc run using the latest 22b8769 to compare against my baseline. Just want to confirm that that's correct.

I'm also planning on using config_bulk_wind_stress_interp_isotropic = true . Are there any other changes I should be aware of?

I'll take this discussion onto Slack as it isn't directly related to this PR.

@jonbob jonbob added NML non-BFB PR makes roundoff changes to answers. labels Jan 24, 2025
@jonbob
Copy link
Contributor

jonbob commented Jan 24, 2025

@proteanplanet -- do you consider this climate-changing? Or just non-BFB?

@jonbob
Copy link
Contributor

jonbob commented Jan 24, 2025

Link to mpas-analysis page comparing runs is here: intercomparison

@jonbob jonbob added CC PR is climate changing and removed non-BFB PR makes roundoff changes to answers. labels Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants