Automatic Identification System (AIS) vessel traffic data plays a crucial role in maritime operations and navigation safety, including various oceanic and coastal management activities. It is transmitted and received by vessels and collected by receivers on shore stations, providing real-time information about the location and characteristics of vessels in U.S. and international waters. This data is vital for supporting environmental and marine planning, ship routing, ecological research, and efforts such as oil spill monitoring or fishing density mapping. Notably, AIS data has been instrumental in the strategic planning of offshore wind farms and to reduce whale fatalities along the California coast.

The U.S. Coast Guard (USCG) collects AIS data through its Nationwide Automatic Identification System (NAIS), utilizing a national network of AIS receivers. To make this data accessible to the public, the National Oceanic and Atmospheric Administration (NOAA) and Bureau of Ocean Energy Management (BOEM) have collaborated to collect and freely distribute the USCG data through the Marine Cadastre Project.

Traditionally, the data was available through NOAA’s previous data provisioning system and interfacing, which had been developed over many years. Accessing AIS datasets required using outdated FTP downloads and bulk CD mailing methods. After receiving a year’s worth of data, NOAA’s Office for Coastal Management (OCM) would have to process, organize, and upload it, resulting in a delay of more than a year before it was available.

Upgrading to Next-Generation Geospatial Technology

To meet the growing demand for AIS data and expedite delivery to the public, NOAA recognized the need to modernize its data provisioning system. With these considerations in mind, NOAA engaged our team to reimagine and build a new data provisioning platform, using the latest cloud and geospatial technologies.

Employing a cloud-native design approach, we successfully built an updated AIS data provisioning system that met each of NOAA’s goals, enabling new data to be brought online up to four times faster. " Ken Logsdon

The project encompassed five primary goals:

1. Develop an updated provisioning system capable of processing AIS data for quarterly dissemination
2. Enhance the user experience by designing an intuitive interface with improved functionality
3. Create a simpler system architecture that is easy to maintain, enabling future enhancements and rapid response to security issues and concerns
4. Transition the system to a state where it requires minimal specialized services to keep it up to date, potentially allowing NOAA to modify and maintain it on site
5. Improve the responsiveness and performance of the provisioning system

One of the major challenges was the immense scale of existing data, consisting of more than 30 billion data points captured from 60 million ship tracklines covering more than 10 million square miles. Spanning a 12-year collection period resulted in over six terabytes of data that had to be archived and easily accessible in the cloud. This required a database structure that could both store a massive amount of information and ingest more frequent data inputs, and still be quick and responsive to user requests.

A Vanguard Prototype

Employing a cloud-native design approach, we successfully built an updated AIS data provisioning system that met each of NOAA’s goals, enabling new data to be brought online up to four times faster. The system features a self-serve geospatial map interface, allowing users to select their region of interest and time period to download historical AIS data directly to their inbox. The user experience is optimized for efficient data retrieval. Within three clicks users can begin downloading—despite the amount of data that the application must query through. The project incorporates a multitiered architecture and serves as a vanguard prototype that can be replicated across NOAA’s portfolio, paving the way to move other resources to the cloud. The provisioning system adheres to necessary federal procedures and security protocols, establishing a pathway for future cloud migrations. This was a project that went through thoughtful technology envisioning and architecture planning, including rigorous development and deployment. It spanned several task orders—we broke the massive project into bite-size pieces so we could collaborate actively with stakeholders to achieve an accurate outcome. The redesigned provisioning system received several accolades, including the 2021 MAPPS Geospatial Products and Services Excellence Award.