In the TC PRIMED Amazon Web Services S3 bucket, the data directory structure is as follows

<TC PRIMED AWS URL>/<TC PRIMED Version>/<TC PRIMED Version Type>/<season>/<basin>/<annual number>/

• <TC PRIMED AWS URL> is the following URL https://noaa-nesdis-tcprimed-pds.s3.amazonaws.com
• <TC PRIMED Version> is the version number, currently v01r01
• <TC PRIMED Type> is the final or preliminary type
• <season> is the four digit season.
  • This is the calendar year for the Northern Hemisphere.
  • For the Southern Hemisphere, the year begins July 1, with the calendar year plus one.
• <basin> is the ocean basin.
  • AL – North Atlantic basin, north of the Equator;
  • SL – South Atlantic basin, south of the Equator;
  • EP – North East Pacific basin, eastward of 140 degrees west longitude;
  • CP – North Central Pacific basin, between the dateline and 140 degrees west longitude;
  • WP – North West Pacific basin, westward of the dateline;
  • IO – North Indian Ocean basin, north of the Equator between 40 and 100 degrees east longitude;
  • SH – South Pacific Ocean basin and South Indian Ocean basin.
• <annual number> is the annual cyclone number from 01 to 49.

Here are a few specific sample files near the time of the lifetime maximum intensity of Hurricane Florence (2018).

• An overpass file per satellite
  • TCPRIMED_v01r01-final_AL062018_AMSR2_GCOMW1_033628_20180912181212.nc (13 MB)
  • TCPRIMED_v01r01-final_AL062018_ATMS_NOAA20_004227_20180912070612.nc (4 MB)
  • TCPRIMED_v01r01-final_AL062018_ATMS_NPP_035622_20180912061712.nc (4 MB)
  • TCPRIMED_v01r01-final_AL062018_GMI_GPM_025795_20180912184512.nc (14 MB)
  • TCPRIMED_v01r01-final_AL062018_MHS_METOPA_061733_20180912004612.nc (3 MB)
  • TCPRIMED_v01r01-final_AL062018_MHS_NOAA18_068603_20180912004912.nc (3 MB)
  • TCPRIMED_v01r01-final_AL062018_MHS_NOAA19_049447_20180912090812.nc (3 MB)
  • TCPRIMED_v01r01-final_AL062018_SSMIS_F16_076888_20180912093012.nc (6 MB)
• ERA5 environment and storm metadata file (env)
  • TCPRIMED_v01r01-final_AL062018_env_s20180830060000_e20180918120000.nc (118 MB)

You can download TC PRIMED data in bulk from the AWS S3 bucket using a UNIX-like terminal.

1. Download and install the AWS Command Line Interface (https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html)
2. After installing AWS CLI, you can pull a directory using the following

   aws s3 cp s3://noaa-nesdis-tcprimed-pds/<path>/<to>/<aws>/<directory>/ /<path>/<to>/<local>/<directory>/ --recursive --no-sign-request

   Since TC PRIMED is a public dataset, you do not need an AWS credential.

If you just want 2015 tropical cyclones in the North Atlantic, you would replace /<path>/<to>/<aws>/<directory>/ with

aws s3 cp s3://noaa-nesdis-tcprimed-pds/v01r01/final/2015/AL/ /<path>/<to>/<local>/<directory>/ --recursive --no-sign-request

Note: TC PRIMED is big (1.6 TB and growing) so you may not want to pull the entire dataset. The example above lets you pull a subset of files.

The NOAA Center for Artificial Intelligence (NCAI) is exploring the creation of an AI-ready data standard to assist in facilitating the use of NOAA data in artificial intelligence (AI) and machine learning. NCAI is working with stakeholders to identify requirements for an AI-ready dataset standard using guidelines provided by the Earth Science Information Partners Data Readiness Cluster. The cluster has produced a Checklist to Examine AI-readiness for Open Environmental Datasets and an AI-ready assessment matrix. These tools and guidelines are designed to help data providers and data users determine if the dataset is ready and applicable to their AI applications.

The questions and answers below provide information on how TC PRIMED addresses relevant AI-ready elements.

TC PRIMED is in the machine-readable Network Common Data Form (NetCDF) file format. While this is the only file format available, NetCDF is easily converted to other file formats.

Once the data is available through NCEI, TC PRIMED users can use the NCEI search functionality to find subsets of the dataset. Users can interact with the data through the application programming interface via Thematic Real-time Environmental Distributed Data Services (THREDDS) Data Server (TDS). Both options allow users to avoid downloading the entire dataset.

No, there are no restrictions or restricted data in TC PRIMED.

Yes, TC PRIMED uses several machine-readable standards for metadata and naming conventions. These conventions, standard naming, and keywords provide information on the spatial and temporal extent, names for variables, standardized variable attributes, and instruments & platforms.

• Climate and Forecast (CF) Metadata Conventions version 1-11
• CF Standard Name Table v85
• Earth Science Information Partners Attribute Convention for Data Discovery 1-3
• Global Change Master Directory (GCMD) Keywords

TC PRIMED is a global dataset where subsets are provided for single tropical cyclones. For each file type, we provide geospatial and temporal bound information using standardized global and group-level attributes.

Each group provides provenance information through the source attribute. For detailed provenance information, see the Data Source.

Yes, the TC PRIMED team has taken steps to quality control the data sources while creating the dataset. For uncertainty quantification, we do not provide new uncertainty quantification information beyond what is available from our upstream data providers.

Yes. Different parts of the dataset include different processing, derivation, and regridding steps and procedures. See the Data Source page for information.

Yes. See the Publications page for publications using TC PRIMED. NOAA Center for Satellite Applications and Research and CIRA/CSU projects are actively using TC PRIMED in AI applications. More details forthcoming.

TC PRIMED is a general dataset that is suitable for multiple AI/ML applications. As a result, TC PRIMED does not contain one set of targets/labels. However, it is common to use the Naval Research Laboratory Automated Tropical Cyclone Forecast (ATCF) system best-track database information from NOAA/NWS National Hurricane Center, NOAA/NWS Central Pacific Hurricane Center, and DOD Joint Typhoon Warning Center that is in the storm metadata group as targets/labels for tropical cyclone applications. But, other projects may end up using infrared or microwave brightness temperatures as the targets/labels.

No.

Files generated prior to 19 September contain information about the update in the global history attribute. Since TC PRIMED files are continually being created, files with the date_created attribute with dates after 19 September are unaffected.

IO052023 and IO082023

The tropical cyclone center latitude and longitude changes can differ relative to the previous version anywhere from 0.1 degrees to 0.8 degrees. Whereas the biggest difference in intensity from the previous version is 15 knots for SH192021. For EP082023, the National Hurricane Center updated the tropical cyclone position information from the Central Pacific Hurricane center, which adds two additional best-track entries on top of the described changes.

The changes result in new values for the central pressure and storm type in both the TC PRIMED overpass files and environmental files.

API users would have seen an error setting the platform to F08 or F10. But, if they didn't specify these in file_type, then they would have received these overpasses. Version 1.0b of the API accepted the F08 or F10 platforms as the file_type value.

The NCEI Archive rejected overpasses from the F08 or F10 platforms. These data have been backfilled.

No. If you pulled the files from NODD directly, you would not have experienced a problem, and no files were updated with this issue.

WP092019, IO012020, SH062020, SH242020, SH262020, WP132020, SH012021, SH052021, SH122021, SH132021, SH152021, SH172021, SH192021, SH242021, WP062021, WP182021, WP222021, WP232021, and EP082023

The tropical cyclone center latitude and longitude changes can differ relative to the previous version anywhere from 0.1 degrees to 0.8 degrees. Whereas the biggest difference in intensity from the previous version is 15 knots for SH192021. For EP082023, the National Hurricane Center updated the tropical cyclone position information from the Central Pacific Hurricane center, which adds two additional best-track entries on top of the described changes.

Environmental files: The changes in the tropical cyclone best-track position may affect the calculation of the environmental diagnostics.

Overpass files: The changes in the tropical cyclone best-track position affect the calculation of the overpass coverage fraction, and result in the inclusion of new overpasses and the exclusion of old overpasses, most prominently evident in EP082023.

Both environmental and overpass files: The differences in tropical cyclone best-track intensity results in new values for the tropical cyclone intensity and intensity change.

Users can find a list of detailed changes in the global “history” attribute in both the environmental and overpass files for the affected storms.

No. You are not impacted.

If you don't want to redownload and need overpass_storm_metadata/storm_heading, you can calculate the storm_heading yourself by using the storm_speed_meridional_component and storm_speed_zonal_component with something like

If you want to redownload all files, see details on this page for using the AWS Command Line Interface. If you only want a subset of files, check out our tcprimedapi Python API.

No. storm_metadata/storm_heading is unaffected. Only overpass_storm_metadata/storm_heading contained incorrect values.

Files generated prior to 19 November contain information about the update in the global history attribute as well as the date_modified and date_metadata_modified global/group attributes. Since TC PRIMED files are continually being created, files with the date_created attribute with dates after 19 November are unaffected.

We have uploaded the fixed files onto the AWS S3 bucket. We are also pushing the fixed files to NCEI. However, due to procedures internal to NCEI in making files available, we do not know when the files on NCEI will be updated. Again, please check the file metadata, as described above, to know if you have the latest update.