Dataset products and access information

What are the data products?

In TC PRIMED, low-Earth-orbiting satellite observations of tropical cyclones serve as the anchor of the dataset. As a whole, TC PRIMED includes the following data:

For specific information about the source for each data product, see the dataset documentation Tropical Cyclone Precipitation, Infrared, Microwave, and Environmental Dataset (TC PRIMED) Dataset Documentation for Version v01r01 and the data sources page.

Where can I access the products?

TC PRIMED is currently available through the NOAA Open Data Dissemination and the NOAA National Centers for Environmental Information (NCEI) at https://doi.org/10.25921/dmy1-0595.

See information in the Registry of Open Data on AWS for TC PRIMED and the associated Amazon Web Services S3 bucket at https://noaa-nesdis-tcprimed-pds.s3.amazonaws.com/index.html.

How are the data products structured on AWS?

In the TC PRIMED Amazon Web Services S3 bucket, the data directory structure is as follows

<TC PRIMED AWS URL>/<TC PRIMED Version>/<TC PRIMED Version Type>/<season>/<basin>/<annual number>/

where

<TC PRIMED AWS URL> is the following URL https://noaa-nesdis-tcprimed-pds.s3.amazonaws.com
<TC PRIMED Version> is the version number, currently v01r01
<TC PRIMED Type> is the final or preliminary type
<season> is the four digit season.
- This is the calendar year for the Northern Hemisphere.
- For the Southern Hemisphere, the year begins July 1, with the calendar year plus one.
<basin> is the ocean basin.
- AL – North Atlantic basin, north of the Equator;
- SL – South Atlantic basin, south of the Equator;
- EP – North East Pacific basin, eastward of 140 degrees west longitude;
- CP – North Central Pacific basin, between the dateline and 140 degrees west longitude;
- WP – North West Pacific basin, westward of the dateline;
- IO – North Indian Ocean basin, north of the Equator between 40 and 100 degrees east longitude;
- SH – South Pacific Ocean basin and South Indian Ocean basin.
<annual number> is the annual cyclone number from 01 to 49.

Are there sample files that I could look at?

Here are a few specific sample files near the time of the lifetime maximum intensity of Hurricane Florence (2018).

How do I pull an entire storm or all of TC PRIMED?

You can download TC PRIMED data in bulk from the AWS S3 bucket using a UNIX-like terminal.

Download and install the AWS Command Line Interface (https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html)
After installing AWS CLI, you can pull a directory using the following
aws s3 cp s3://noaa-nesdis-tcprimed-pds/<path>/<to>/<aws>/<directory>/ /<path>/<to>/<local>/<directory>/ --recursive --no-sign-request
Since TC PRIMED is a public dataset, you do not need an AWS credential.

If you just want 2015 tropical cyclones in the North Atlantic, you would replace /<path>/<to>/<aws>/<directory>/ with

aws s3 cp s3://noaa-nesdis-tcprimed-pds/v01r01/final/2015/AL/ /<path>/<to>/<local>/<directory>/ --recursive --no-sign-request

Note: TC PRIMED is big (1.6 TB and growing) so you may not want to pull the entire dataset. The example above lets you pull a subset of files.

How do I interact with TC PRIMED files?

To help you learn how to interact with TC PRIMED files, the TC PRIMED development team has created a Learning Journey in partnership with the NOAA Center for Artificial Intelligence (NCAI). See the tools and tutorials page to learn more.

Is TC PRIMED ready for my machine learning (ML) application?

What is ML-ready data?

The NOAA Center for Artificial Intelligence (NCAI) is exploring the creation of an ML-ready data standard to assist in facilitating the use of NOAA data in machine learning. NCAI is working with stakeholders to identify requirements for an ML-ready dataset standard using guidelines provided by the Earth Science Information Partners Data Readiness Cluster. The cluster has produced a Checklist to Examine AI-readiness for Open Environmental Datasets and a ML-ready assessment matrix. These tools and guidelines are designed to help data providers and data users determine if the dataset is ready and applicable to their machine learning applications.

The questions and answers below provide information on how TC PRIMED addresses relevant ML-ready elements.

What is the data access for TC PRIMED?

TC PRIMED is in the machine-readable Network Common Data Form (NetCDF) file format. While this is the only file format available, NetCDF is easily converted to other file formats.

Once the data is available through NCEI, TC PRIMED users can use the NCEI search functionality to find subsets of the dataset. Users can interact with the data through the application programming interface via Thematic Real-time Environmental Distributed Data Services (THREDDS) Data Server (TDS). Both options allow users to avoid downloading the entire dataset.

Are there restrictions or restricted data in TC PRIMED?

No, there are no restrictions or restricted data in TC PRIMED.

Does TC PRIMED have standardized metadata?

Yes, TC PRIMED uses several machine-readable standards for metadata and naming conventions. These conventions, standard naming, and keywords provide information on the spatial and temporal extent, names for variables, standardized variable attributes, and instruments & platforms.

Does TC PRIMED have geospatial and temporal bound information?

TC PRIMED is a global dataset where subsets are provided for single tropical cyclones. For each file type, we provide geospatial and temporal bound information using standardized global and group-level attributes.

Where is TC PRIMED's data provenance information?

Each group provides provenance information through the source attribute. For detailed provenance information, see the Data Source.

Does TC PRIMED have quality control monitoring and uncertainty quantification?

Yes, the TC PRIMED team has taken steps to quality control the data sources while creating the dataset. For uncertainty quantification, we do not provide new uncertainty quantification information beyond what is available from our upstream data providers.

Is TC PRIMED data processed/derived/regridded?

Yes. Different parts of the dataset include different processing, derivation, and regridding steps and procedures. See the Data Source page for information.

Is TC PRIMED being used in other machine learning applications?

Yes. See the Publications page for publications using TC PRIMED. NOAA Center for Satellite Applications and Research and CIRA/CSU projects are actively using TC PRIMED in machine learning applications. More details forthcoming.

Does TC PRIMED have targets/labels?

TC PRIMED is a general dataset that is suitable for multiple machine learning applications. As a result, TC PRIMED does not contain one set of targets/labels. However, it is common to use the Naval Research Laboratory Automated Tropical Cyclone Forecast (ATCF) system best-track database information from NOAA/NWS National Hurricane Center, NOAA/NWS Central Pacific Hurricane Center, and DOD Joint Typhoon Warning Center that is in the storm metadata group as targets/labels for tropical cyclone applications. But, other projects may end up using infrared or microwave brightness temperatures as the targets/labels.

If you have specific questions, contact the TC PRIMED development team via .