Estuarine Hypoxia model output

[marjy]

I see a long list of output from different models of Chesapeake Bay at: http://testbed.sura.org/datatable but all these various model runs don't seem to be available in the catalog: http://comt.sura.org/thredds/comt_1_archive_summary.html

Would it be possible for someone to add them to the thredds catalog, so that I could access the model output from there? Would it be possible to include the forcing files as well? (Those don't seem to be available from the data table.) Would it be possible to include the Chesapeake Bay Program data for the years for which data are available?

Thank you for the information!

[rsignell-usgs]

I see that datasets have been uploaded to the testbed server, and there are entries in the Google Drive Testbed Inventory Spreadsheet but the NcML files referenced in the spreadsheet are missing. I just created a 00_dir.ncml file for UMCES_ChesROMS-BGC
and sure enough, it now it appears in the CF-compliant Testbed1 THREDDS catalog.

I'll work on the other ones as time permits, or as we find another Corbitt. At least we know what the issue is.

[marjy]

That's great news. Thanks so much for looking into this for me!
Marjy

[rluettich]

Marjy,

I’ve spent some time looking through this as well. While the COMT_1_Archive_Summary.html doesn’t seem to be populated with all of the COMT_1 runs (for the reason Rich identified), it does appear that these runs are present under comt.sura.org/thredds/comt_1_archive_full.html and that both the “catalogue” and the “ftp” links from the datatable on the SURA COMT website work. Is this consistent with your observation?

Rick

[marjy]

Hi Rick,

Thanks again for looking into this some more!

Yes, the "ftp" site is now working! Thank you so much to whoever fixed that. I had thought some folks had uploaded their "forcing files", and those aren't appearing in the catalog or ftp sites, but perhaps I am wrong about that.

Thanks again!
Marjy

[rluettich]

It does seem to me that to make the data sets most useful for a future user, both the model inputs and outputs should be archived. They may all be there, however, I expect we will need one more scrub before all is said and done. We’ll get there eventually.

[rsignell-usgs]

Okay, I actually spent a few hours on this today and got more of the estuarine datasets aggregated and functioning in the COMT1 summary catalog .

Here's a screen grab from Godiva 2, just to show that the data is aggregated and browsible:

For those who are not familiar with how Godiva2 works, I made a little youtube video showing how to browse structured grid model data using godiva2, which is available on every THREDDS Data Server (if the WMS service is enabled).

So here's what I did. Starting with the testbed estuarine directory here:

@marjy requested 5 high-priority datasets:
"I would say that the highest priority could be Malcolm's 15 year one term model results. Also high priority would be the various 2004-2005 runs that went into our paper: CH3D, CBOFS, ChesROMS-constantR and ChesROMS-DepthDepR. All are two year runs. "

There were some problems, of course. We were never able to figure out the CH3D binary, non-netcdf format, so that was out. But I got the other ones going.

The VIMS CBOFS had some extra files (seg56-63) that dropped a lot of variables, and since it seemed seg15-55 contained two years of data, I just moved seg56-63 to a directory called dye_only, and then the data aggregated nicely. I added the one-term oxygen-proxy variable dye_01 to the NcML, gave it a scale_factor=0.032 as per the word doc file documentation so that it would have units of mg/l = kg m-3 and added standard_name="mass_concentration_of_oxygen_in_sea_water. I did this for all the 1-term runs.

The two VIMS ChesROMS 1Term simulations needed some rearranging of files to meet the conventions, so I made a new directory VIMS_ChesROMS and subdirs under them with names 1term_constR and 1term_ddR. I rearranged and renamed netcdf files where necessary so that they could be aggregated.

The 15 year data WHOI (Skully) Chesroms in the WHOI_ChesROMS-1Term directory didn't represent any challenges. Just standard aggregation and adding the dye_01 attributes. This simulation was run with ROMS 3.6.

I didn't add the 2004-2005_1termDO data from the same directory to the catalog because it was just two years of data run with ROMS 2.2. It think the 15 year run supercedes this.

I didn't include the UMCES_ChesROMS-BGC aggregation in the catalog because it contained no oxygen data.

I didn't include the UMCES_UMES-ROMS data in the catalog because there are four runs called:

• 2004_Case01_lres
• 2005_Case03_lres
• 2004_Case02_hres
• 2005_Case04_hres

and I don't know what they are or if we need them.

[kknee]

@rsignell-usgs having trouble looking at CBOFS data - is this the right URL for the DAP endpoint: http://comt.sura.org/thredds/dodsC/data/comt_1_archive/estuarine_hypoxia/VIMS_CBOFS/2004-2005.html

[rsignell-usgs]

@kknee, the CF-corrected datasets for COMT should be the ones on the "summary" catalog:
http://comt.sura.org/thredds/comt_1_archive_summary.html
so yes, that CBOFS DAP endpoint should be correct.

I'm not sure what you mean by "trouble looking". I was able to view the data in godiva2, but perhaps you mean the strange looking stuff that gets drawn by ncWMS because of the wacky lon/lat values that CBOFS grid has over land (see below). This is a plotting package problem, not with the actual data in the dataset.

[rsignell-usgs]

BTW, I awoke at 2am this morning thinking "wait, mg/l is not the same as kg/m3". The brain works in mysterious ways.... Anyway, I changed the units to mg/l this morning. At least I think I did. Please let me know if there are any remaining issues.

[kknee]

@rsignell-usgs last night the link was broken, seems okay this am.

[rsignell-usgs]

@marjy, what other estuarine hypoxia datasets do you want to see in the summary catalog at:
http://comt.sura.org/thredds/comt_1_archive_summary.html
(besides CH3D)

[marjy]

We have a new ~30 year time series of model output for the Chesapeake Bay (Malcolm’s ChesROMS-1term) that we would love to have archived. Would that be possible?

On Dec 10, 2014, at 10:27 AM, Rich Signell <[email protected]mailto:[email protected]> wrote:

@marjyhttps://github.com/marjy, what other estuarine hypoxia datasets do you want to see in the summary catalog at:
http://comt.sura.org/thredds/comt_1_archive_summary.html
(besides CH3D)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/28#issuecomment-66467947.

[brianmckenna]

@marjy should be able to upload it to the COMT server via FTP. Let me know if you no longer have the credentials

[rsignell-usgs]

@marjy, the instructions are here:
https://github.com/ioos/comt/wiki/COMT2%20Inventory%20and%20Upload%20Guidelines

[marjy]

We can upload the ROMS netcdf output files and a metadata document, but could you help us generate NcML files for our ROMS runs, since we haven’t done this before?

On Feb 24, 2015, at 9:12 AM, Rich Signell <[email protected]mailto:[email protected]> wrote:

@marjyhttps://github.com/marjy, the instructions are here:
https://github.com/ioos/comt/wiki/COMT2%20Inventory%20and%20Upload%20Guidelines

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/28#issuecomment-75762295.

[rsignell-usgs]

@marjy, would you be willing to try creating the NcML by using one of the existing NcML files as a template?

If you start here: https://github.com/ioos/comt_1_archive/tree/master/estuarine_hypoxia you can drill down to quite a few.

For example, this one:
https://github.com/ioos/comt_1_archive/blob/master/estuarine_hypoxia/WHOI_ChesROMS-1term/2004-2005_1termDO/00_dir.ncml