Level-3.2: Outlier flagging
Five tests remove outliers from NEE_L3.1. Each writes a flag column FLAG_L3.2_NEE_L3.1_QCF_OUTLIER_<TEST>_TEST (0 kept, 2 removed); a record counts for the first test that removed it.
Method, evidence and checks:
01_FPPC_IRGA72_NEE, Step 4: one plot per test and an overview of all removals.- diive: outlier tests: how the tests work.
Tests
| Flag | Test | Settings |
|---|---|---|
MANUAL |
Periods removed by hand, see below | Whole days, both ends included |
ABSLIM |
Absolute limits | ±50 µmol m-2 s-1 |
HAMPEL |
Hampel filter on double differences: single spikes | 13 days, 5.5 sigma, day and night separately |
HAMPEL_2 |
Hampel filter on the values per time of day: values wrong for their time of day | 13 days, 5.5 sigma, the half-hour and one on either side, at least 10 values |
NEGNIGHT |
Nighttime values clearly below zero | Below −4 times the robust spread of all nighttime values within ±15 days |
The nighttime test uses the spread of all nighttime values rather than the uncertainty of each record. The per-record uncertainty grows with bad values in a noisy stretch: on 12 December 2021, while the LI-7200 was failing, night values of −41, −20 and −13 µmol m-2 s-1 passed a test based on it. High nighttime values are real respiration and are not tested.
Compared with the 2016-2024 processing, the absolute limits are kept, and the trimming of low values and the local standard deviation test are replaced by the two Hampel filters and the nighttime test.
Instrument problems and periods removed by hand
- Start-up of the LI-7200 in 2016. During the installation (11 to 22 January) and a period with too low a flow rate (1 March to 25 May), water in the filter lowered the fluxes. Both periods are removed by hand.
- 19 May to 7 June 2018. There is no
FC: the flow module of the analyser had failed, so this period was processed with the sonic anemometer only. - 30 December 2018 to 10 January 2019. The LI-7200 (IRGA72-B) was failing: low signal strength from December, an iced inlet filter, error codes and repeated stops of the data acquisition. It was replaced on 11 January 2019. Removed by hand.
- 8 November to 6 December 2020. A fault of the gas analyser: the scatter of the flux rose to many times its usual level, with half-hourly values from −190 to +190 µmol m-2 s-1, while the flow rate and the sonic stayed normal. The signal strength fell from 100 to about 60 on 12 November and recovered after the LI-7200 was replaced on 10 December. 8 to 27 November and 4 to 6 December are removed by hand; the days between them are within the usual scatter.
- 15 January to 4 February 2026. The nighttime scatter was about twice its seasonal level, also after the outlier tests, with winter daytime values above 20 µmol m-2 s-1. The signal strength fell from 93 to 87; the scatter ended when the LI-7200 was cleaned on 4 February. Removed by hand.
- Wrong calibration gas, 14 December 2017 to 15 March 2019. The CO2 concentrations of this period were calibrated against the wrong reference gas. The correction, a factor of 0.974 on the concentration, was applied when the raw data were converted with bico, so
FCis corrected.
The long gaps without any FC are listed on Level-4.1.
Latent and sensible heat flux
The notebooks 11_FPPC_IRGA72_LE and 15_FPPC_IRGA72_H run the first four tests. Negative nighttime values are not tested: LE can be negative at night (dew), and a negative H at night is physical.
| Flux | Absolute limits | Why |
|---|---|---|
LE |
−50 to 800 W m-2 | Above 800 W m-2 most values, and above 900 W m-2 all, exceed the incoming shortwave radiation of the same half-hour; below −50 W m-2, LE would mean condensation of more than 0.07 mm h-1. |
H |
−200 to 800 W m-2 | The upper limit keeps the largest daytime values that agree with the incoming radiation, in spring before leaf-out; the lower limit lies below the nighttime values of summer and removes the large negative values of wet and freezing episodes in winter. |
Periods removed by hand:
LE: the same windows asFC. The LI-7200 measures H2O and CO2 in the same cell, so its faults affect both fluxes; each window was checked inLE.Hcomes from the sonic anemometer alone, so the faults of the gas analyser do not apply:Hstays normal during the 2016 start-up and the November 2020 fault. Only 20 to 26 December 2018 is removed: after work at the LI-7200 the instrument boom was not fixed in its position, so the sonic could move with it.- Sonic electronics, 19 December 2019 to 10 January 2020. After the change from the HS-100 to the HS-50 (see Instrumentation) the electronics unit of the HS-100 stayed in use, so the fieldbook rates these data as possibly not entirely accurate.
Hof this period does not differ from the weeks around it and is kept. - Wet or freezing weather. The sonic can give
Hof several hundred W m-2 of either sign, while the sonic temperature falls several kelvin below its usual offset from the air temperature: water or ice on the transducers. These episodes occur every winter, most often in 2016 to 2019, before the sonic head was replaced. The absolute limits and the Hampel filters remove their values.