NOAA National Water Model Reanalysis Model Data on AWS

In the field of Earth System Modeling, a reanalysis is a model simulation that joins modern modeling technologies and newer, more complete, datasets to obtain a better understanding of past environments. For example, in weather forecasting, it can be thought of as running a weather forecast model in the past. In such a scenario it could mean the running of a weather model for a period when no data existed in order to get a better understanding of the state of the weather from that period. This same type of historical run can be conducted for other types of models as well, including water models.

The National Water Model Reanalysis dataset contains output from a A 25-year (January 1993 through December 2017) retrospective simulation using version 1.2 of the National Water Model, and a 26-year (January 1993 through December 2018) retrospective simulation using version 2.0 of the National Water Model (NWM). These simulations used observed rainfall as input and ingested other required meteorological input fields from a weather Reanalysis dataset. The output frequency and fields available in this historical NWM dataset differ from those contained in the real-time forecast model. One application of this dataset is to provide historical context to current real-time streamflow, soil moisture and snowpack NWM conditions. The Reanalysis data can be used to infer flow frequencies and perform temporal analyses with hourly streamflow output and 3-hourly land surface output. The long-term dataset can also be used in the development of end user applications which require a long baseline of data for system training or verification purposes.

Learn more about the National Water Model: http://water.noaa.gov/

Accessing NWM Reanalysis on AWS 🔗︎

The NWM Reanalysis version 1.2 is stored in the nwm-archive Amazon S3 bucket while the version 2.0 is stored int he noaa-nwm-retro-v2.0-pds bucket both in the us-east-1 AWS region.

This bucket contains the reanalysis archive organized by year starting in 1993. The files are an internally compressed NetCDF format. They do not need to be decompressed. Each file contains detailed metadata describing the data stored within it.

You can use the AWS Command Line Interface to list a particular year (2003 in this example) in the bucket like this:

aws s3 ls s3://noaa-nwm-retro-v2.0-pds/full_physics/2003/ --no-sign-request

aws s3 ls nwm-archive/2003/ --no-sign-request

or via a url constructed like this: https://noaa-nwm-retro-v2.0-pds.s3.amazonaws.com/?prefix=full_physics/2003

https://nwm-archive.s3.amazonaws.com/?prefix=2003

Each year contains files for each hour of the year iterated over different product types, not all products are published at hourly intervals. The products types are listed in the table below.

Product Description Cycle Example File Name

CHRTOUT Streamflow values at points associated with flow lines Every Hour 201701010000.CHRTOUT_DOMAIN1.comp LAKEOUT Output values at points associated with reservoirs (lakes) Every Hour 201701010000.LAKEOUT_DOMAIN1.comp LDASOUT Land surface model output Every 3rd Hour 201701010000.LDASOUT_DOMAIN1.comp RTOUT Ponded water and depth to soil saturation Every 3rd Hour 201701010000.RTOUT_DOMAIN1.comp

Each product (data file) uses the following naming convention: yyyy/yyyymmddhhhh.product_DOMAIN1.comp where:

  • yyyy year from 1993
  • mm month
  • dd day
  • hhhh hour of day
  • product (CHRTOUT, LAKEOUT, LDASOUT or RTOUT)
  • DOMAIN1.comp this is the same for all files

For example, to access the stream flow data for 12pm on May 1st 2003, the key would be:

2003/200305011200.CHRTOUT_DOMAIN1.comp

To access the data, combine the key with the bucket name like this:

s3://nwm-archive/2003/200305011200.CHRTOUT_DOMAIN1.comp

or via HTTPS like this:

https://nwm-archive.s3.amazonaws.com/2003/200305011200.CHRTOUT_DOMAIN1.com

Configuration of the Data 🔗︎

The dataset contains four products for each day. The four products are listed below along with some of the parameters and properties of the respective files. More details are available in the metadata contained within each file:

RTOUT: Geospatial, 250m Gridded NetCDF

  • sfcheadsubrt: Ponded water depth (mm)
  • zwattablrt: Water table depth (m)

LAKEOUT: Point Type (including Lake ID)

  • feature_id: Lake Common ID
  • elevation: Water surface elevation (m)
  • inflow: Reservoir inflow (m3 s-1)
  • outflow: Reservoir outflow (m3 s-1)

CHRTOUT: Point Type (including Reach ID)

  • feature_id: Reach ID
  • streamflow: River Flow (m3 s-1)
  • q_lateral: Runoff into channel reach (m3 s-1)
  • velocity: River Velocity (m s-1)
  • qSfcLatRunoff: Runoff from terrain routing (m3 s-1)
  • qBucket: Flux from ground water bucket (m3 s-1)
  • qBtmVertRunoff: Runoff from bottom of soil to ground water bucket (m3)

LDASOUT: Geospatial, 1Km Gridded NetCDF

  • ACCET: Accumulated evapotranspiration (mm)
  • FIRA: Total net long wave (LW) radiation to atmosphere (W m-2)
  • FSA: Total absorbed Short Wave (SW) radiation (W m-2)
  • FSNO: Snow cover fraction on the ground (fraction)
  • HFX: Total sensible heat to the atmosphere (W m-2)
  • LH: Latent heat to the atmosphere (W m-2)
  • SNEQV: Snow water equivalent (kg m-2)
  • SNOWH: Snow depth (m)
  • SOIL_M: Volumetric soil moisture (m3 m-3)
  • SOIL_W: Liquid volumetric soil moisture (m3 m-3)
  • TRAD: Surface radiative temperature (K)
  • UGDRNOFF: Accumulated underground runoff (mm)

Note: For the two accumulation variables, UGDRNOFF and ACCET, the accumulation takes place between pairs of dates: 1. 00Z January 1 – 21Z March 31 2. 00Z April 1 – 21Z June 30 3. 00Z July 1 – 21Z September 30 4. 00Z October 1 – 21Z December 31

Document Updates 🔗︎

27th July 2018. Corrected the units of the following paramters: - ACCET - from m to mm - qBucket - from m\^3\^ to m3 s-1 - qSfcLatRunoff - from m\^3\^ to m3 s-1

Please note that the metadata in the corresponding netCDF files may have incorrect units for: - qBucket and qSfcLatRunoff.

The units listed above are the correct units.