EarthAlign: Learning Language-Aligned Representations of Earth System Data

Fadillah Adamsyah Maani1, Numan Saeed1, Jiaqi Ma1, Dat Tien Nguyen1, Youssef Wehbe2, Duncan Watson-Parris3, Salman Khan1,2
1 Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · 2 ADIA Lab · 3 University of California, San Diego (UC San Diego)

Paper Code Dataset Weights

Overview

EarthAlign is a large-scale Earth system-language alignment dataset and benchmark for learning and evaluating multimodal representations of Earth system data. It contains 553,738 samples spanning 2000-2024, with broad geographic coverage and diverse natural phenomena and normal weather conditions. Each sample includes temporal and geospatial information, multivariate ERA5 data, and four metadata-grounded caption styles that describe the same Earth system condition from different perspectives. Building on EarthAlign, MeteoLIP is a dual-encoder framework for aligning meteorological data with natural language. Its meteorology tokenizer, MetTok, aggregates variable-specific representations into compact meteorology tokens while incorporating temporal and geospatial information. MeteoLIP supports cross-modal retrieval, weather-condition classification, and physical variable estimation, providing a benchmark for evaluating language-aligned Earth system representations.

EarthAlign overview figure 1
EarthAlign overview figure 2

🏆 Highlights

  1. EarthAlign Dataset. 553,738 samples spanning 2000-2024, with broad geographic coverage and diverse natural phenomena and normal weather conditions. Four metadata-grounded caption styles provide complementary descriptions of each Earth system condition.
  2. MeteoLIP. A CLIP/SigLIP-based dual-encoder framework for meteorology-language representation learning, with MetTok for meteorological tokenization and temporal/geospatial conditioning.
  3. Three Evaluation Tasks. Cross-modal retrieval, weather-condition classification, and physical variable estimation.
  4. Open Release. Dataset, training and evaluation code, and pretrained model weights are publicly available.

EarthAlign Data Construction

EarthAlign data construction pipeline

EarthAlign combines expert-curated natural-event records with climatology-based normal-weather samples. The resulting records contain standardized spatial and temporal information, labels, four grounded caption styles for alignment with Earth system data. Each record is paired with 63-variable ERA5 snapshots.

MeteoLIP Architecture and Training

MeteoLIP architecture and training

MeteoLIP consists of transformer-based meteorological and text encoders. MetTok converts multivariate meteorological input into a compact token sequence. During training, we randomly sample a caption style for each meteorological input and optimize a contrastive objective to align meteorological and textual representations in a shared embedding space.

Quantitative Results

EarthAlign quantitative results table

Comparison of CLIP and SigLIP baselines with their MeteoLIP counterparts. (a) Cross-modal retrieval. (b) Classification under text-prototype inference and linear probing. (c) Physical variable estimation of cyclone maximum sustained wind speed and minimum central pressure under the same two protocols. Bold and underline indicate the best and second-best results.

Qualitative Results

EarthAlign qualitative results

Qualitative results using the MeteoLIP meteorology and text encoders. (a) Meteorology-to-text retrieval retrieves captions describing the same or closely related event across different caption styles. (b) The cold-wave example assigns the highest probability to cold wave, followed by winter weather and frost. (c) The similarity profiles over candidate physical values show peaks around the predicted minimum central pressure and maximum sustained wind speed.

BibTeX

@misc{maani2026earthalignlearninglanguagealignedrepresentations,
      title={EarthAlign: Learning Language-Aligned Representations of Earth System Data}, 
      author={Fadillah Adamsyah Maani and Numan Saeed and Jiaqi Ma and Dat Tien Nguyen and Youssef Wehbe and Duncan Watson-Parris and Salman Khan},
      year={2026},
      eprint={2610.14624},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.14624}, 
}

Acknowledgement

This material is based on work supported by ADIA Lab, Abu Dhabi under a collaborative research project titled Climate Co-Pilot: A Multimodal AI Agent for Climate Resilience in the MENA Region.

Our codebase is built upon the OpenCLIP repository, and we thank sincerely its developers and contributors for making their work publicly available.

IVAL logo MBZUAI logo UC San Diego logo ADIA Lab logo