Harmonized Landsat Sentinel-2 (HLS)

HLS Cloud usage exploited:

• Ability to quickly setup clusters of servers
• Access Cloud resident data
• Vary system configurations as well as processors
• Easily vary system storage needs and cluster sizes

• Data harmonization allows users to channel data from different sources into a consistent, standardized, and comprehensive format for analysis.
• HLS takes data from the joint NASA and USGS Landsat and the ESA Sentinel-2 satellites to create a harmonized dataset available for research.
• HLS began in 2017 at NASA Ames on the NASA Earth Exchange (NEX) computer cluster.
• The SMCE was used by the HLS effort to test, improve, tune, and perform quality control on the methods employed to harmonize the data.
• SMCE was chosen due to:
  • The very high compute node storage demands (16TB EBS required on a cluster’s headnode), and up to 400TB of S3 storage.
  • Easy access to Sentinel-2 data sets, available from Google, but difficult to get direct from ESA.
  • Parallel clusters with 100’s of cpus to be deployed daily to process new daily data loads, but no need to pay for this compute between these cron job runs.
• In early FY23 the HLS effort on the SMCE had completed processing 9 years of Landsat data and 7 years of Sentinel-2 data over 30% of the globe to verify and finalize the methods for deployment in production within MSFC to be stored in publicly accessible DACCS.