Meteo Product Chain

How the thirteen meteo product notebooks (10_METEO/30_PRODUCTS) turn screened measurements into the finished variables, and which of them depend on each other.

The numbering is a run order: a notebook reads only products with a lower number, so running 01 to 13 and then the merge 99 satisfies every dependency. Five of those reads change an exported value. 02 and 03 use 01’s gap-filled SW_IN as a gap-filling driver, 04 uses 02’s measured and homogenised TA for its homogenised RH, and 07 computes VPD from the measured and the homogenised columns of 02 and 04. All other reads are diagnostic: 06 reads 01 and 02; 08 reads 02, 05 and 06; 09 reads 02 and 08; 10 reads 02 and 09; 11 reads 09 and 10; 12 reads 01; 13 reads 02 and 06. A diagnostic read changes what a notebook concludes, not what it writes.

01 to 07 and 99 run to 30 September 2026. 08 to 13 still end on 31 December 2025.

flowchart LR
    %% ---------- inputs ----------
    DB[("InfluxDB<br/>20_SCREENING output<br/>meteoscreening_mst + _diive")]
    MSLAE["10_REFERENCE<br/>MeteoSwiss_LAE_30MIN_2004-2026.09<br/>TA / RH / PA / SW_IN as *_LAE_MS"]
    MSOED["10_REFERENCE<br/>PREC_MeteoSwiss_OED_DAILY_2004-2014<br/>PREC_MeteoSwiss_OED_30MIN_2014-2025"]
    MSREG["10_REFERENCE<br/>PREC_MeteoSwiss_REGIONAL_DAILY_2004-2025<br/>13 stations within 21 km"]
    TSSCR["TS_FF1_SCREENED_30MIN_2004-2025<br/>DIIVE + MST download, 34 channels"]

    %% ---------- notebooks ----------
    N01["01 SW_IN"]
    N02["02 TA"]
    N03["03 PPFD_IN"]
    N04["04 RH"]
    N05["05 PA"]
    N06["06 LW_IN"]
    N07["07 VPD"]
    N08["08 PREC"]
    N09["09 SWC"]
    N10["10 TS"]
    N11["11 G"]
    N12["12 SW_OUT"]
    N13["13 LW_OUT"]
    N99["99 MERGED"]

    %% ---------- products ----------
    F01["01_METEO_SW_IN_GAPFILLED_2004-2026.09<br/>SW_IN_T1_47_1_gfXG<br/>FLAG_..._ISFILLED"]
    F02["02_METEO_TA_GAPFILLED_2004-2026.09<br/>TA_T1_47_1_gfXG<br/>TA_T1_47_1_HOMOGENIZED_gfXG<br/>FLAG_..._ISFILLED<br/>FLAG_..._SOURCE"]
    F03["03_METEO_PPFD_IN_GAPFILLED_2004-2026.09<br/>PPFD_IN_T1_47_1_gfXG<br/>FLAG_..._ISFILLED"]
    F04["04_METEO_RH_CORRECTED_2004-2026.09<br/>RH_T1_47_1<br/>RH_T1_47_1_HOMOGENIZED<br/>FLAG_..._MISSING<br/>FLAG_..._SOURCE"]
    F05["05_METEO_PA_2005-2026.09<br/>PA_T1_47_1<br/>no flag"]
    F06["06_METEO_LW_IN_2005-2026.09<br/>LW_IN_T1_47_1<br/>FLAG_..._SOURCE"]
    F07["07_METEO_VPD_2004-2026.09<br/>VPD_T1_47_1<br/>VPD_T1_47_1_HOMOGENIZED<br/>FLAG_..._ISFILLED<br/>FLAG_..._SOURCE"]
    F08["08_METEO_PREC_GAPFILLED_2004-2025<br/>PREC_TOT_T1_47_1<br/>PREC_TOT_T1_47_1_HOMOGENIZED<br/>FLAG_..._SOURCE<br/>FLAG_..._ISFILLED"]
    F09["09_METEO_SWC_FF1_2004-2025<br/>SWC_FF1_*_1, 5 depths<br/>..._HOMOGENIZED, 4 of 5<br/>FLAG_..._SOURCE"]
    F10["10_METEO_TS_FF1_2004-2025<br/>18 individual channels<br/>TS_FF1_*_HOMOGENIZED_GAPFILLED, 7 depths<br/>FLAG_..._METHOD, FLAG_..._SUSPECT"]
    F11["11_METEO_G_FF1_2004-2025<br/>G_FF1_0.05_1 to _3, 3 plates<br/>G_FF1_0.05_HOMOGENIZED<br/>FLAG_..._SOURCE, FLAG_..._NPLATES"]
    F12["12_METEO_SW_OUT_2005-2025<br/>SW_OUT_T1_47_1<br/>FLAG_..._SOURCE<br/>FLAG_..._INSTRUMENT"]
    F13["13_METEO_LW_OUT_2005-2025<br/>LW_OUT_T1_47_1<br/>FLAG_..._SOURCE"]
    F99["99_METEO_MERGED_2004-2026.09<br/>31 series + 30 flags of 01-13<br/>08-13 end Dec 2025"]

    %% ---------- database reads ----------
    DB --> N01 & N02 & N03 & N04 & N05 & N06 & N08 & N09 & N11 & N12 & N13
    DB --> TSSCR --> N10

    %% ---------- reference reads ----------
    MSLAE -->|SW_IN_LAE_MS| N01
    MSLAE -->|TA_LAE_MS| N02
    MSLAE -->|SW_IN_LAE_MS| N03
    MSLAE -->|RH_LAE_MS, fills gaps| N04
    MSLAE -.->|PA_LAE_MS, diagnostic| N05
    MSLAE -.->|TA_LAE_MS, RH_LAE_MS, diagnostic| N07
    MSLAE -.->|SW_IN_LAE_MS, diagnostic| N12
    MSOED -->|fills gaps| N08
    MSREG -->|homogenisation factors| N08
    MSREG -.->|regional rain, second check| N09
    TSSCR -.->|early TS channels| N09

    %% ---------- notebook -> product ----------
    N01 --> F01
    N02 --> F02
    N03 --> F03
    N04 --> F04
    N05 --> F05
    N06 --> F06
    N07 --> F07
    N08 --> F08
    N09 --> F09
    N10 --> F10
    N11 --> F11
    N12 --> F12
    N13 --> F13
    N99 --> F99

    %% ---------- product -> product, changes exported values ----------
    F01 -->|SW_IN_gfXG as gap-fill driver| N02
    F01 -->|SW_IN_gfXG as gap-fill driver| N03
    F02 -->|TA_gfXG, TA_HOMOGENIZED_gfXG| N04
    F02 -->|TA_gfXG, TA_HOMOGENIZED_gfXG| N07
    F04 -->|RH, RH_HOMOGENIZED| N07

    %% ---------- product -> product, diagnostic only ----------
    F01 -.->|SW_IN, same radiometer| N06
    F02 -.->|TA, emission check| N06
    F02 -.->|TA, rain or snow| N08
    F05 -.->|PA, shared outage| N08
    F06 -.->|LW_IN, shared outage| N08
    F02 -.->|TA, detector covariate| N09
    F08 -.->|PREC, detector covariate| N09
    F02 -.->|TA_gfXG, diagnostic| N10
    F09 -.->|SWC at the 2015 break| N10
    F09 -.->|SWC, heat capacity| N11
    F10 -.->|TS 0.05 m, depth and storage tests| N11
    F01 -.->|SW_IN, albedo| N12
    F02 -.->|TA, emission check| N13
    F06 -.->|LW_IN, same instrument| N13

    %% ---------- the merge ----------
    F01 & F02 & F03 & F04 & F05 & F06 & F07 --> N99
    F08 & F09 & F10 & F11 & F12 & F13 --> N99

    classDef gf fill:#E3F2FD,stroke:#1976D2,color:#000
    classDef meas fill:#F1F8E9,stroke:#689F38,color:#000
    classDef calc fill:#FFF3E0,stroke:#F57C00,color:#000
    classDef src fill:#ECEFF1,stroke:#607D8B,color:#000
    class F01,F02,F03,F08,F10 gf
    class F04,F05,F06,F09,F11,F12,F13 meas
    class F07 calc
    class DB,MSLAE,MSOED,MSREG,TSSCR src

Figure 1: Inputs, notebooks and products of stage 10_METEO/30_PRODUCTS. Solid arrows are reads that change the exported values; dashed arrows are diagnostic-only reads.

Reading the diagram

Blue products are gap-filled or reference-filled (SW_IN, TA, PPFD_IN, PREC, TS), green products ship as measured (RH, PA, LW_IN, SWC, G, SW_OUT, LW_OUT), and the orange product is computed from other products (VPD).

• Two notebooks do not read the database. 07 reads two finished product files and applies a formula. 10 reads the screened TS channels through TS_FF1_SCREENED_30MIN_2004-2025, which downloads the 34 channels once. 09 reads the early half of that download as its witness for the June 2015 wiring break.
• 01 feeds the gap-filling of 02 and 03. Its gap-filled SW_IN_T1_47_1_gfXG is a driver in both models, so 01 must run first. It is also the diagnostic partner of 06 and 12, which describe the other channels of the same radiometer.
• 02 is read by seven notebooks, more than any other product, and two of those reads change a value: 04 and 07. Air temperature separates rain from snow in 08, excludes snowmelt in 09, and gives the Stefan-Boltzmann reference emission in 06 and 13. Diagnostic reads across January 2016 use the HOMOGENIZED column, because the measured one carries the step of that date. 07 reads both columns and computes a VPD from each.
• 08 (PREC) reads three reference files, for two purposes. The Ehrendingen (OED) record is daily to September 2014 and 30MIN after; the 30MIN part fills the gaps. The regional ensemble supplies the homogenisation factors: one nearby gauge can show that the tower changed in 2018, but only a dozen of them can show which side of that break is wrong. 09 reads the same ensemble so that its jump detector does not read a gap in the tower gauge as a dry spell.
• 05 reads the MeteoSwiss pressure as a diagnostic only. It checks the tower record but never writes a value into it. The NABEL hut barometer is also read as a diagnostic and is not exported.
• Six products have no external reference, and a second channel takes its place. No weather service measures this plot’s soil, and no station near the site measures the radiation this canopy emits and reflects. 09 (SWC), 10 (TS) and 11 (G) are checked against the other depths and plates of their own profile. 06 (LW_IN) and 13 (LW_OUT) are checked against the Stefan-Boltzmann emission at 02’s air temperature, and 13 also against 06, the other pyrgeometer of the same instrument. 12 (SW_OUT) is checked against 01: SW_OUT / SW_IN is the surface albedo, and anything that scales both channels of one radiometer cancels in that ratio. These checks catch a whole-record error such as a wrong file, unit or year, but cannot validate a single measurement.

What 99 merges

Notebook 99 joins the value and flag columns of all thirteen products into one half-hourly table, 99_METEO_MERGED_2004-2026.09, and checks the joined index. It holds 61 columns: 31 data series and 30 flag columns. Products 08 to 13 end on 31 December 2025, so their columns are NaN from January to September 2026 until their notebooks are extended.

• TA, RH, VPD and PREC each contribute two value columns, the measured and the homogenised series, and two flags.
• 09 contributes five soil-moisture depths and the homogenised series of four of them.
• 10 and 11 contribute only their reconciled series: seven soil-temperature depths and the soil heat flux plate mean. The 18 soil-temperature channels and the three plates are one sensor each and stay in their product files.
• 12 contributes SW_OUT with two flags, one naming the instrument and one the screening.

Eight of the 31 series are complete over the whole period. For every series that may contain NaN, the merge records the reason and checks it; the reasons are listed in the information file 99_METEO_MERGED_2004-2026.09_INFO.md written beside the data.

Precipitation aggregates differently from the other variables: rainfall is summed, not averaged. Each entry in the merge registry names its aggregation. A daily mean of 30-minute rainfall (about 0.05 mm) is not an interpretable quantity; the daily total is.