TUFLOW FV Get Atmos

From TUFLOW FV Wiki
Jump to navigation Jump to search

Introduction

This tutorial demonstrates how to run the Get Atmos python command line tool using conda. Get Atmos provides tools to download and process atmospheric data for usage with TUFLOW FV, including:

  • Mean sea level pressure
  • Wind speed and direction
  • Shortwave and longwave radiation
  • Air temperature
  • Relative humidity

This tool currently supports downloading atmospheric data from ECMWF's ERA5, NCAR's CFSR and (CFSv2), as well as the Australian Bureau of Meteorology's BARRA Reanalysis.

Prerequisites

1. IMPORTANT - To download the default data source ERA5, you are required to setup a CDSAPI key. For instructions on how to complete this, please visit the Copernicus API Webpage. You can alternatively download CFSR/CFSv2 or BARRA data without any additional account setup.

2. Install miniconda and setup the TUFLOW FV Python Environment by following the steps within the TUFLOW FV Python Environment page.

Licensing

Please ensure that your usage of data downloaded using this python library abides with the license requirements for the selected source.

Example - Downloading ERA5 Data For The Gulf Of Carpentaria

This example outlines the procedure used to download the ERA5 atmospheric data used within the Coastal Example Models.

1. Setup your Copernicus Account and CDSAPI key.

2. Ensure that you have successfully created a TUFLOW FV Python environment by following the instructions provided on the page TUFLOW FV Python Environment - TUFLOW FV Wiki.

3. Open Anaconda prompt, and activate your TUFLOW FV Python environment with the command: 

  conda activate tfv-workspace

Activate tfv-workspace.png


4. In this example, we have chosen to work in the “C:\TUFLOW\Get_Tools\my_atmos” directory, however you can select any location you like. Navigate to the directory you wish to work in through the command: 

  cd <directory>

GetAtmos cd my atmos.png

Please note: To successfully change your directory location, you must be working out of the same drive that you wish to access. To change drives, enter the command: 

  <drive>

GetAtmos change drive.png

5. Enter to command GetAtmos to see the program setup: 

  GetAtmos

This command will show the two sub-program options:

A) GetAtmos A Download raw data files

B) GetAtmos B Merge the raw data files

GetAtmos.png

6. Prior to downloading any data we'll first check the arguments by calling program A, and typing “-h” for help: 

  GetAtmos A -h

GetAtmos A -h.png

7. Call the program with your specifications for the required positional arguments, and optional arguments as desired. This example shows retrieval of data covering the Gulf of Carpentaria spanning 12:00 am on 01/03/2010 to 12:00 am on 01/05/2010 (dd/mm/yyyy). Please note the folder "raw_data" must exist before running. 

  GetAtmos A “2010-03-01 00:00” “2010-05-01 00:00” 135 143 -19 -7 --path raw_data

GetAtmos download data ERA5.png

GetAtmos download data complete ERA5.png

The downloaded ERA5 files should appear in your specified output directory as below:

GetAtmos downloaded files ERA5.png

8. Once all raw data files are downloaded, we can merge the datafiles into a single NetCDF suitable for direct usage with TUFLOW FV. To see the options for merging the data, enter the command: 

  GetAtmos B -h

GetAtmos B -h.png

9. Now submit the merge job. In this example, we use the command (Note: You need to create the folder "merged_data" before running): 

  GetAtmos B --in_path raw_data --out_path merged_data

GetAtmos merge data ERA5.png

To add on a timezone offset, the optional extra commands “tz” and “ltz” can be added. If the tz command is entered an additional time variable local_time will be added to the output NetCDF. The variable time will still be included in UTC. If the tlz command additional attribute metadata is added to the NetCDF local_time variable. If using a time offset, it’s good practice to include the ltz to avoid later confusion.

In this example, we will update our command to add both tz and ltz commands, and specify the filename “GoC_ERA5_20100301_20100531_AEST.nc” . 

  GetAtmos B --in_path raw_data --out_path merged_data -tz 10 -ltz AEST --file_name GoC_ERA5_20100301_20100531_AEST.nc

GetAtmos merge data rev ERA5.png


Please note: It is highly recommended to add the start, end, and timezone to your output file name. For example MyArea_20220201_20220401_UTC.nc, indicates that the data in this file is available from the 1st of February, 2022 to 1st of April, 2022 and is in the UTC timezone. If this convention is adhered to, it can save time and reduce errors and confusion when dealing with boundary conditions.

Review Get Atmos Outputs

Get Atmos produces a single NetCDF file that contains the merged downloaded data. Using xarray to inspect this NetCDF, we see it has the structure and properties shown below:

GetAtmos xarray netcdf ERA5.png

This NetCDF file can also be opened in QGIS using the TUFLOW Viewer Plugin to view the data variables as map outputs. The below image shows the map output for air temperature at a certain timestep. For guidance on plotting in QGIS using the TUFLOW Viewer Plugin, please visit: Viewing 3D TUFLOW FV Results in QGIS - TUFLOW FV Wiki.

GetAtmos QGIS NetCDF.png

The TUFLOW Viewer Plugin can also be used to generate a variety of plots of the data variables. The below figure shows creation of a timeseries plot of mean sea level pressure at a certain location.

GetAtmos QGIS timeseries.png


When Get Atmos merges the downloaded data into a single NetCDF file, it also creates an fvc file in the same location.

GetAtmos outputs ERA5.png

Within this fvc file, you will find that Get Atmos has written your atmospheric boundary conditions for you.

GetAtmos fvc ERA5.png

These atmospheric boundary conditions can then be incorporated into your TUFLOW FV model through an include statement, as shown below. This include statement will reference GoC_ERA5_20100301_20100531_AEST.fvc, so that the commands it contains will be read by TUFLOW FV.

GetAtmos include command ERA5.png

Downloading CFSR data for Gulf of Carpentaria

This example outlines the changes to the above procedure required to download CFSR/CFSv2 atmospheric data.

1. To download CFSR data, your command must include the optional argument --source CFSR. Call the program with your specifications for the required positional arguments, and further optional arguments as desired. This example shows retrieval of CFSR data covering the Gulf of Carpentaria spanning 12:00 am on 01/03/2010 to 12:00 am on 01/05/2010 (dd/mm/yyyy): 

  GetAtmos A “2010-03-01 00:00” “2010-05-01 00:00” 135 143 -19 -7 --path raw_data --source CFSR

2. To merge CFSR data, your command must include the optional argument --source CFSR. In this example, we use the command: 

  GetAtmos B --in_path raw_data --out_path merged_data --source CFSR

Downloading BARRA data for Gulf of Carpentaria

This example outlines the changes to the above procedure required to download BARRA atmospheric data.

1. To download BARRA data, your command must include the optional argument --source BARRA_<X>. Here, X represents your choice of the following sub-domains:

  • R (12km resolution over Australia, New-Zealand and the maritime continent) for the period 1990-01-01 to 2019-02-28 (yyyy-mm-dd)
  • AD (1.5km resolution over SA) for the period 1990-01-01 to 2019-02-28
  • PH (1.5km resolution over South-West WA) for the period 1990-01-01 to 2019-02-28
  • SY (1.5km resolution over Eastern NSW) for the period 1990-01-01 to 2019-02-28
  • TA (1.5km resolution over Tasmania) for the period 1990-01-01 to 2019-02-28

Call the program with your specifications for the required positional arguments, and further optional arguments as desired. This example shows retrieval of BARRA_R data covering the Gulf of Carpentaria spanning 12:00 am on 01/03/2010 to 12:00 am on 01/05/2010 (dd/mm/yyyy): 

  GetAtmos A “2010-03-01 00:00” “2010-05-01 00:00” 135 143 -19 -7 --path raw_data --source BARRA_R

2. To merge CFSR data, your command must include the optional argument --source BARRA_<X>. In this example, we use the command: 

  GetAtmos B --in_path raw_data --out_path merged_data --source BARRA_R

Additional Notes

  • To change the data source, add in “--source CFSR/BARRA_<x>/ERA5” to BOTH programs (A and B)
  • ERA5 will always download in monthly blocks, however the merged product can be cut down to exact dates using the “--time_start” and “--time_end” optional arguments.
  • GetAtmos B will by default generate an FVC file to use with TUFLOW FV.
  • Reprojection can be handled by supplying the “-rp / --reproject <EPSG_CODE>” optional argument. For example, “-rp 7856” will reproject the merged netcdf to GDA2020 MGA56 coordinates, and this will be reflected inside the FVC file.
  • Conversion to local time can be handled by providing the optional argument “-tz <hours>”. You can provide a label as well if you wish to add this to the metadata (e.g., “-ltz AEST” and “-tz 10”).

Conclusion

For further information on Get Atmos, please visit our GitLab page.

To learn how to use our other Python utilities, please visit our TUFLOW FV Python Toolbox, TUFLOW FV Get Tide and TUFLOW FV Get Ocean pages.

To see how to implement the most common TUFLOW FV model design features, please access our TUFLOW FV Example Models page.

To complete tutorials or learn more tips and tricks, please return to the TUFLOW FV Wiki Mainpage.

If you have any further queries, feedback or requests for new functionality, please feel free to get in contact with support@tuflow.com

Retrieved from "https://fvwiki.tuflow.com/w/index.php?title=TUFLOW_FV_Get_Atmos&oldid=12117"