Stage 2: data and output presentation

Agro-climate, soil and crop yield in 16 major palay and corn provinces, 2015–2024

Ten years of daily weather, soil properties and PSA harvest statistics, drawn four ways in ggplot2 (R) and Seaborn (Python) from the same processed files.

Members: [fill in] · Instructor: [fill in] · Presented: [fill in]

What we set out to show

Problem

Weather, soil and crop records for Philippine provinces sit with different agencies in different formats, so they are rarely read together. We merged them into one dataset and charted it to see how growing conditions differ between provinces, how they changed over ten years, and how they line up with palay and corn yield.

Research questions

  1. How do seasonal rainfall, temperature and evapotranspiration relate to palay and corn yield?
  2. What trends show in rainfall, temperature and yield from 2015 to 2024, especially in El Niño years?
  3. Which weather, soil and yield variables move together?
  4. Which provinces group together on standardized climate, soil and yield?
  5. How do the ggplot2 and Seaborn versions compare?

Three public sources, one row per province and year

Sources
SourceVariablesRows × columnsCoverage
Open-Meteo Historical Weather API Mean, max and min temperature, rainfall, FAO reference evapotranspiration, solar radiation, relative humidity, soil moisture 0–7 cm (all numeric) 58,448 × 10 Daily, 2015–2024, one point per province
ISRIC SoilGrids 2.0 pH, clay, sand, silt, organic carbon, nitrogen, CEC, bulk density at 0–5, 5–15 and 15–30 cm (numeric) 384 × 6 Static, one point per province
PSA OpenSTAT, tables 0012E4EVCP0 and 0022E4EAHC0 Palay and corn production (t) and area harvested (ha) (numeric); province and crop (text) 4,360 × 5 Annual, all provinces, 2015–2024

The merged file crop_panel.csv has 320 rows × 25 columns: province, region, island group, crop and year, plus area, production, yield, 8 weather and 8 soil columns.

What we did to the data

Acquisition

A Python script queried each API once and saved the replies as CSV, so every chart is reproducible without calling an API again. Each province is represented by one hand-picked point in its main cropland. Two points (Isabela, Sultan Kudarat) were moved after SoilGrids returned no data at the first spot.

Missing values and duplicates

Weather: 0 missing, 0 duplicate days. Soil: 0 missing after the two moves. Yield: 0 missing for the 16 provinces.

Outliers

Yields outside 1.5 × IQR per crop were flagged, not removed. The five flagged rows are all Nueva Ecija palay from 2020 to 2024 (5.9–6.3 t/ha), a real result of the country's most irrigated rice province.

Transformation

  • PSA's 2022 split of Maguindanao into del Norte and del Sur was summed back into one province. Negros Occidental's 2024 row under the new Negros Island Region was joined to its earlier series.
  • SoilGrids integers converted to units (pH ÷ 10, g/kg ÷ 10 and so on), then combined into a 0–30 cm mean weighted by layer thickness.
  • Daily weather summed or averaged to months and years. Wet-season rainfall is June to November.
  • Yield = production ÷ area harvested, in t/ha.

Standardization

For the cluster maps each variable was converted to a z-score (mean 0, SD 1) across the 16 province averages, so millimetres of rain and pH units weigh the same.

Descriptive statistics

Province-years, n = 160 (soil values are fixed per province)
VariableUnitMeanSDMinMedianMax
Annual rainfallmm2,8758831,3822,7145,773
Wet-season rainfallmm1,8584881,0211,7983,331
Mean temperature°C26.201.3122.2426.4628.01
Mean daily maximum°C30.271.7725.1730.5233.05
Reference evapotranspirationmm/yr1,4201241,1041,4141,728
Solar radiationMJ/m²/day18.170.8615.9418.2220.10
Relative humidity%82.073.8573.9982.7189.42
Soil moisture, 0–7 cmm³/m³0.350.050.260.350.47
Palay yieldt/ha4.090.752.474.056.28
Corn yieldt/ha3.921.441.323.967.00
Soil pH (water)pH5.750.475.055.616.53
Clay, 0–30 cm%38.645.4225.1240.0750.65
Organic carbon, 0–30 cmg/kg31.6123.0712.9721.96104.68
CEC, 0–30 cmcmol(c)/kg21.246.7711.6519.4834.62

Initial observations

Wet-season rainfall barely moves yield

Variables: wet-season rainfall, June–November (x) against yield in t/ha (y), one point per province, crop and year (n = 320), with a linear fit and 95% band per crop.

R · ggplot2 geom_point + geom_smooth
R scatterplot of wet-season rainfall from 1,000 to 3,300 mm against yield from 1.3 to 7 t/ha. Points are spread widely; the palay fit line is flat near 4.1 t/ha and the corn line falls slightly from 4.1 to 3.5 t/ha.

Not rendered yet. Run Rscript r/charts.R.

Python · sns.scatterplot + sns.regplot
Seaborn scatterplot of the same data: palay as green crosses with a flat fit line, corn as gold dots with a slightly falling fit line and a wide confidence band.

Not rendered yet. Run python python/charts.py.

Interpretation

Neither crop has a meaningful linear link to wet-season rainfall. For palay r = 0.03 (p = 0.71); for corn r = −0.09 (p = 0.27), about 0.26 t/ha less per extra 1,000 mm. The vertical spread at any rainfall level is much larger than the trend, which points to differences between provinces (irrigation, soil, variety) rather than rain itself. The wettest points, above 2,400 mm, come from six provinces including Lanao del Sur and Negros Occidental, which also have low corn yields.

Comparison

Both show the same points and fits. ggplot2 maps colour and shape in one aes() call and draws the fits with one geom_smooth(). Seaborn needs a separate regplot() per crop to colour each fit, but its bands take the crop colour by default, which makes them easier to tell apart than ggplot2's grey ones.

Yields rose over the decade and dipped in El Niño years

Variables: year (x) against mean yield in t/ha (y) for each island group, faceted by crop. Shaded bands mark the El Niño years 2015–16, 2019 and 2023–24.

R · ggplot2 geom_line + facet_wrap
R line plot in two panels, palay and corn. Luzon is highest in both and rises from 4.4 to 4.8 t/ha for palay and 4.9 to 5.3 for corn. Visayas and Mindanao sit lower. Gold bands mark 2015-16, 2019 and 2023-24.

Not rendered yet. Run Rscript r/charts.R.

Python · sns.lineplot + axvspan
Seaborn line plot of the same means, with markers on each year and the same gold El Niño bands. Luzon leads both crops; Visayas corn is lowest at 2.3 to 2.8 t/ha.

Not rendered yet. Run python python/charts.py.

Trend analysis

Across the 16 provinces, mean palay yield rose from 3.93 t/ha in 2015 to 4.17 in 2024 and corn from 3.77 to 3.93. Luzon stays on top every year (palay 4.6, corn 5.1 t/ha on average); Visayas corn is lowest (2.5 t/ha). Luzon palay gained the most (4.37 to 4.72 t/ha); Visayas palay stayed flat near 3.7.

El Niño years. 2016 was the low point for Luzon (palay 4.35, corn 4.76 t/ha) and Visayas corn (2.31 t/ha). Luzon corn also fell from 5.31 in 2023 to 5.05 t/ha in 2024. The 2019 event left no clear mark. These dips line up with the drought years but do not prove the drought caused them.

Comparison

ggplot2 needs the means computed first (aggregate()) and the El Niño bands as a data frame for geom_rect(); facet_wrap() then splits the crops in one line. Seaborn averages inside lineplot(), but the facets and bands need a matplotlib loop with axvspan().

Soil chemistry tracks yield more closely than rainfall does

Variables: Pearson correlation between 13 weather, soil and yield variables over the 160 province-years.

R · ggplot2 geom_tile
R correlation heatmap, 13 by 13, brown for negative and teal for positive. Strongest pairs: corn yield with soil pH 0.85, evapotranspiration with relative humidity minus 0.81.

Not rendered yet. Run Rscript r/charts.R.

Python · sns.heatmap
Seaborn correlation heatmap with the same 13 variables and identical values, annotated to two decimals.

Not rendered yet. Run python python/charts.py.

Colour scale

A diverging brown-to-teal scale (ColorBrewer BrBG) fixed at −1 to 1: teal cells move together, brown cells move in opposite directions, and near-white cells are unrelated. The diverging scale keeps zero in the middle, so sign is readable at a glance.

Interpretation

Comparison

The numbers match exactly. Seaborn draws the annotated heatmap in one call from the correlation matrix. ggplot2 has no heatmap function, so the matrix is reshaped to long form and drawn with geom_tile() plus geom_text(), which is more code but gives full control over each layer.

Central Luzon stands apart; the rest split into upland and lowland groups

Variables: 13 province averages (climate, soil, palay and corn yield), each z-scored. Method: hierarchical clustering, Euclidean distance, Ward linkage, on rows and columns.

R · pheatmap, ward.D2
R cluster map of 16 provinces by 13 variables with dendrograms. Nueva Ecija, Pampanga, Pangasinan and Tarlac form the top cluster with high yield and pH and low humidity.

Not rendered yet. Run Rscript r/charts.R.

Python · sns.clustermap, ward
Seaborn cluster map with the same province order and dendrogram as the R version.

Not rendered yet. Run python python/charts.py.

Groups (cut at four clusters)

GroupProvincesWhat sets it apart
Central Luzon plainNueva Ecija, Pampanga, Pangasinan, TarlacHighest yields, highest pH and CEC, low humidity and soil moisture
Cool uplandsBukidnon, Lanao del SurLowest temperature and evapotranspiration; Lanao del Sur has the most rain
Mindanao lowlands + LeyteMaguindanao, Cotabato, South Cotabato, Sultan Kudarat, LeyteLow soil pH (5.05–5.50) and below-average corn yield
Mixed lowlandsIsabela, Cagayan, Iloilo, Camarines Sur, Negros OccidentalNear-average on most variables; Negros Occidental is sandy and wet

The groups follow landform more than island group: Mindanao provinces fall into two different clusters, and Visayas provinces into two. On the column side, yield, pH, CEC and evapotranspiration cluster together against humidity, soil moisture and rainfall, the same split the heatmap shows.

Comparison

Both tools produced the same tree, since pheatmap's ward.D2 and SciPy's ward are the same method. pheatmap draws the whole figure in one call but is a separate package outside ggplot2. Seaborn's clustermap() is built in, but placing the colour bar and fitting long labels took manual layout code.

ggplot2 and Seaborn side by side

Aspectggplot2 (R)Seaborn (Python)
Code structureLayers added with +; one grammar for all four charts except the cluster map (pheatmap)One function per chart type; matplotlib needed for bands, facets and layout
Data shapePrefers long data; the correlation matrix had to be reshapedTakes wide matrices directly for heatmap and clustermap
DefaultsClean minimal theme; grey confidence bands; legend outside the plotGrid theme; bands in the series colour; legend inside the plot
CustomizationFacets, scales and labels in one chainFine control through matplotlib, at the cost of more lines
Lines of chart code72, including the pheatmap call94, including layout fixes for the cluster map
Best fit hereFaceted trend charts and layered annotationsCorrelation heatmaps and cluster maps in one call

Both read the same CSVs and produce the same numbers, so the choice comes down to workflow. R needed fewer lines overall. R was quicker for the line plot, Python for the heatmap.

What the charts show so far

  1. Rain alone does not explain yield. Wet-season rainfall has no significant linear link to palay (r = 0.03) or corn yield (r = −0.09). (Scatterplot)
  2. Yields rose over the decade. Mean palay yield went from 3.93 to 4.17 t/ha and corn from 3.77 to 3.93 t/ha between 2015 and 2024. (Line plot)
  3. El Niño years coincide with dips. 2016 was the lowest year for Luzon palay and corn and for Visayas corn, and Luzon corn fell again in 2024. (Line plot)
  4. Soil and dryness go with high yield. Corn yield correlates with soil pH (0.85) and CEC (0.74) and negatively with humidity (−0.66). (Heatmap)
  5. Provinces group by landform. The four Central Luzon plain provinces form their own cluster with the highest yields; Bukidnon and Lanao del Sur form a cool upland pair. (Cluster map)

Associations here are descriptive. Province-level soil and one weather point per province cannot separate the effect of irrigation, variety or farm practice.