Skip to main content
← Back to Thoughts

Crop Classification with Synthetic Multitemporal Satellite Images

Guillermo Siesto••7 min read
EN/ES

Published in: Remote Sensing 2021, 13(17), 3378
Authors: Guillermo Siesto, Marcos Fernández-Sellers, Adolfo Lozano-Tello
Institution: Quercus Software Engineering Group, Universidad de Extremadura
DOI: 10.3390/rs13173378 • Open Access • MDPI Full Paper

Abstract: The demand for new tools for mass remote sensing of crops, combined with the open availability of satellite imagery, has prompted the development of new methods for crop classification. Because this classification is frequently required to be completed within a specific time frame, performance is essential. In this work, we propose a new method that creates synthetic images by extracting satellite data at the pixel level, processing all available bands, as well as their data distributed over time considering images from multiple dates. With this approach, Sentinel-2 imagery is ingested by a deep convolutional network to discern crop classes over an entire agricultural year after being trained on previous years. A software system implementing this method was deployed across Extremadura (Spain) as a monitoring tool for Common Agricultural Policy (CAP) subsidies of the European Union.

1. The Real-World Challenge: European CAP Monitoring

In the European Union, the Common Agricultural Policy (CAP) allocates subsidies to encourage sustainable agriculture and land stewardship. In 2014, the European Commission introduced new regulations permitting satellite-driven Earth observation to monitor eligibility, verify crop declarations, and flag anomalies automatically.

For the CAP Paying Agency of the Regional Government of Extremadura (Spain), manual field inspections across hundreds of thousands of hectares were labor-intensive and incomplete. The agency needed an automated machine learning tool capable of:

  1. Predicting the current year knowing only labels from preceding years (coping with changing weather and climatic variation).
  2. Operating before the payment deadline at the end of the summer, requiring rapid data ingestion and model inference.
  3. Covering regional scale: Over 510,000 hectares (204 geographic zones of 5 × 5 km) in the agricultural heartland of Mérida and Don Benito, containing 34,135 hectares of declared crop plots.

2. The Core Innovation: Pixel-Level Synthetic Matrices

Traditional deep learning approaches for satellite imagery either feed single-date multispectral scenes into 2D segmentation networks (losing seasonal crop growth patterns) or stack multi-date rasters into massive 3D spatio-temporal volumes (extremely heavy on RAM and slow to train).

We developed a different paradigm: transforming the temporal phenology of individual pixels into compact 2D synthetic images.

Figure 4: Pipeline of synthetic matrix generation Figure 1 — Extraction pipeline: for every 10 × 10 m pixel within a parcel, all 12 Sentinel-2 spectral bands across 21 five-day intervals (April 1 to July 15) are mapped into a 21 × 12 synthetic matrix.

How the Synthetic Input Works:

  • Dimension 1 (Time): 21 five-day time periods (pentads), spanning April 1 to July 15. This specific window was selected because it captures the critical vegetative and reproductive cycles of summer crops while avoiding uninformative early spring rainfall.
  • Dimension 2 (Spectrum): All 12 usable Sentinel-2 bands (B1, B2, B3, B4, B5, B6, B7, B8, B8A, B9, B11, B12), resampled to a uniform 10-meter spatial resolution (B10 is dropped during atmospheric correction).
  • The Result: Every pixel becomes a clean 21 × 12 matrix where pixel values represent normalized reflectance. This encodes the complete temporal and spectral evolution of the crop without storing unwanted non-agricultural background terrain.

3. Data Processing & Atmospheric Cleaning

To convert raw satellite passes into clean training data, we addressed two challenges: isolating agricultural parcel boundaries and eliminating atmospheric noise.

Geometry & Parcel Extraction

Plot boundaries registered by farmers for the CAP were matched against Sentinel-2 UTM grids (Zones 29N and 30N). Using polygon rasterization algorithms, we extracted all enclosed 10 × 10 m pixels.

Figure 2: Parcel pixel grid extraction Figure 2 — Parcel boundary extraction: (a) Sentinel-2 satellite capture; (b) Extracted 10 × 10 m pixel mesh within registered parcel boundaries.

Atmospheric Conditions & Cloud Masking

Satellite imagery is frequently contaminated by clouds, cirrus haze, and cast shadows. We incorporated the European Space Agency (ESA) atmospheric classification model, which categorizes pixels into vegetation, water, clouds, and dark shadow areas:

Figure 3: Cloud detection and atmospheric mask Figure 3 — Atmospheric variable detection: (a) Sentinel-2 RGB image sample showing cloud coverage (8 August 2019); (b) Classification mask produced by the ESA atmospheric model identifying clouds, cirrus, shadows, and vegetation.

Because optical cloud-masking models are not 100% reliable—especially when catching subtle cloud shadows—we applied a secondary Hampel Filter across the temporal series of each band. This prunes aberrant spikes, removes remaining outliers, and smoothly interpolates values across consecutive pentads.

4. Distinctive Crop Phenology Signatures

By averaging the synthetic matrices across entire crops for the 2019 season, each crop group displayed a distinct, machine-distinguishable "fingerprint":

Figure 4: Average synthetic inputs per crop Figure 4 — 2019 average synthetic input matrices across the studied crop classes. Yellow represents values close to 1.0; dark blue represents values close to 0.0.

The model was tasked with classifying seven primary commercial summer crop groups:

  • Corn (corn, sweet corn)
  • Tomato (fresh tomato, tomato for industrial processing)
  • Rice
  • Sunflower
  • Potato
  • Onion / Garlic / Leek
  • Asparagus

Notice how crops like rice (which begins flooded in early pentads) and corn (which surges in near-infrared reflectance during peak vegetative growth) produce radically different visual patterns across the 21 × 12 matrices.

5. Lightweight CNN Architecture

Because the synthetic matrices condense complex spatio-temporal data into a compact 21 × 12 input, the Convolutional Neural Network did not require massive 3D kernels or millions of parameters.

Figure 5: CNN Model Architecture Figure 5 — Convolutional Neural Network architecture developed for the 21 × 12 synthetic input matrix.

Model Characteristics:

  • Input Layer: 21 × 12 × 1 single-channel synthetic matrix.
  • Feature Extraction: Two successive convolution stages (first expanding to 128 filters, then contracting back to 32 filters) using ReLU activations.
  • Regularization: Batch Normalization and Dropout between layers to eliminate overfitting.
  • Classification Head: Flatten layer connected to a Dense layer with Softmax output predicting the 7 crop categories.
  • Training Efficiency: Models trained in a fraction of the time needed by standard volumetric architectures, running with automated Early Stopping based on validation loss.

6. Field Validation & Real-World Results

To simulate realistic operations, the system was trained exclusively on historical data from 2017, 2018, and 2019 (85% train, 15% validation) and evaluated on the entire unseen 2020 campaign (100% test set).

Figure 6: Prediction validation in Zone 30-581 Figure 6 — Validation over Zone 30-581 (Accuracy: 97.37%): (a) Desired ground truth; (b) Prediction on ground-truth plots; (c) Full-raster inference across all pixels; (d) Sentinel-2 summer RGB.

Key Outcomes:

  • High Predictive Accuracy: The model achieved an average F-score of 96% across dominant crops (corn, tomato, rice, sunflower).
  • Adjoining Plot Discovery: When inferred over the entire raster (Figure 5c), the network successfully identified crops on adjoining, undeclared fields—providing agricultural authorities with visibility beyond official declarations.
  • Low Computational Overhead: By performing pixel extraction once as a preprocessing step, neural network training and hyperparameter tuning executed rapidly, enabling fast seasonal recalibration.
  • Minority Crop Nuances: Ground-level crops with low leaf coverage (such as asparagus) presented greater confusion due to the 10 m satellite ground resolution, highlighting specific areas for complementary radar (Sentinel-1) fusion.

7. Citation & Paper Links

If you use this methodology or find it useful in your remote sensing research, please cite the original article:

@article{siesto2021crop,
  title={Crop Classification of Satellite Imagery Using Synthetic Multitemporal and Multispectral Images in Convolutional Neural Networks},
  author={Siesto, Guillermo and Fern{\'a}ndez-Sellers, Marcos and Lozano-Tello, Adolfo},
  journal={Remote Sensing},
  volume={13},
  number={17},
  pages={3378},
  year={2021},
  publisher={MDPI},
  doi={10.3390/rs13173378}
}