{ "cells": [ { "cell_type": "markdown", "id": "88ced42a-31f3-4e28-a89a-fb18ab7730be", "metadata": {}, "source": [ "# **Collect partitioning results**" ] }, { "cell_type": "markdown", "id": "17b60a39-7749-461c-ac85-f2c0376a8302", "metadata": {}, "source": [ "**Author**: Lukas Hörtnagl (holukas@ethz.ch)" ] }, { "cell_type": "markdown", "id": "792e7b63-708d-4320-b4bf-a112864a0c96", "metadata": {}, "source": [ "# Info" ] }, { "cell_type": "markdown", "id": "36287f71-51af-42d0-ad3b-e8630ccaa40f", "metadata": {}, "source": [ "- Previously gap-filled TA, SW_IN (Rg) and VPD were used to partition NEE (daytime and nighttime method)\n", "- Using NEE fluxes gap-filled with random forest\n", "- Using the partitioning algorithm implementations in `REddyProc`" ] }, { "cell_type": "markdown", "id": "171e068a-deba-45e2-bcfa-642e77375e9f", "metadata": {}, "source": [ "# Imports" ] }, { "cell_type": "code", "execution_count": 1, "id": "f8fb99de-1f34-4fba-901b-59dc7c110ea8", "metadata": {}, "outputs": [], "source": [ "from pathlib import Path\n", "import pandas as pd\n", "from diive.core.io.files import save_parquet, load_parquet\n", "from diive.core.times.times import TimestampSanitizer" ] }, { "cell_type": "markdown", "id": "0e02b0f0-a1e6-4985-a3e8-80359dd5ef4f", "metadata": {}, "source": [ "# Load partitioning results for fluxes gap-filled with random forest" ] }, { "cell_type": "code", "execution_count": 2, "id": "0d4204db-4f5a-40c7-9107-c9c1e83204de", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Tair_origTair_fTair_fqcTair_fallTair_fall_qcTair_fnumTair_fsdTair_fmethTair_fwinRg_origRg_fRg_fqcRg_fallRg_fall_qcRg_fnum...FP_GPP2000FP_kFP_betaFP_alphaFP_RRefFP_E0FP_k_sdFP_beta_sdFP_alpha_sdFP_RRef_sdFP_E0_sdReco_DT_U50GPP_DT_U50Reco_DT_U50_SDGPP_DT_U50_SD
TIMESTAMP_MIDDLE
2005-01-01 00:15:001.5666671.56666701.566667NaNNaNNaNNaNNaN0.00.000.0NaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN0.0930710.00.0800160.0
2005-01-01 00:45:001.5333331.53333301.533333NaNNaNNaNNaNNaN0.00.000.0NaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN0.0926820.00.0796880.0
2005-01-01 01:15:001.5666671.56666701.566667NaNNaNNaNNaNNaN0.00.000.0NaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN0.0930710.00.0800160.0
2005-01-01 01:45:001.5666671.56666701.566667NaNNaNNaNNaNNaN0.00.000.0NaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN0.0930710.00.0800160.0
2005-01-01 02:15:001.5000001.50000001.500000NaNNaNNaNNaNNaN0.00.000.0NaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN0.0922950.00.0793610.0
................................................................................................
2024-12-31 21:45:00-1.919472-1.9194720-1.919472NaNNaNNaNNaNNaN0.00.000.0NaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN1.0910280.00.2658080.0
2024-12-31 22:15:00-2.104678-2.1046780-2.104678NaNNaNNaNNaNNaN0.00.000.0NaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN1.0787510.00.2643270.0
2024-12-31 22:45:00-2.089444-2.0894440-2.089444NaNNaNNaNNaNNaN0.00.000.0NaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN1.0797590.00.2644470.0
2024-12-31 23:15:00-2.355761-2.3557610-2.355761NaNNaNNaNNaNNaN0.00.000.0NaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN1.0621640.00.2623730.0
2024-12-31 23:45:00-2.578839-2.5788390-2.578839NaNNaNNaNNaNNaN0.00.000.0NaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaNNaN1.0474830.00.2606880.0
\n", "

350640 rows × 129 columns

\n", "
" ], "text/plain": [ " Tair_orig Tair_f Tair_fqc Tair_fall Tair_fall_qc Tair_fnum Tair_fsd Tair_fmeth Tair_fwin Rg_orig Rg_f Rg_fqc Rg_fall Rg_fall_qc Rg_fnum ... FP_GPP2000 FP_k FP_beta FP_alpha FP_RRef FP_E0 FP_k_sd FP_beta_sd FP_alpha_sd FP_RRef_sd FP_E0_sd Reco_DT_U50 GPP_DT_U50 Reco_DT_U50_SD GPP_DT_U50_SD\n", "TIMESTAMP_MIDDLE ... \n", "2005-01-01 00:15:00 1.566667 1.566667 0 1.566667 NaN NaN NaN NaN NaN 0.0 0.0 0 0.0 NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 0.093071 0.0 0.080016 0.0\n", "2005-01-01 00:45:00 1.533333 1.533333 0 1.533333 NaN NaN NaN NaN NaN 0.0 0.0 0 0.0 NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 0.092682 0.0 0.079688 0.0\n", "2005-01-01 01:15:00 1.566667 1.566667 0 1.566667 NaN NaN NaN NaN NaN 0.0 0.0 0 0.0 NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 0.093071 0.0 0.080016 0.0\n", "2005-01-01 01:45:00 1.566667 1.566667 0 1.566667 NaN NaN NaN NaN NaN 0.0 0.0 0 0.0 NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 0.093071 0.0 0.080016 0.0\n", "2005-01-01 02:15:00 1.500000 1.500000 0 1.500000 NaN NaN NaN NaN NaN 0.0 0.0 0 0.0 NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 0.092295 0.0 0.079361 0.0\n", "... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...\n", "2024-12-31 21:45:00 -1.919472 -1.919472 0 -1.919472 NaN NaN NaN NaN NaN 0.0 0.0 0 0.0 NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 1.091028 0.0 0.265808 0.0\n", "2024-12-31 22:15:00 -2.104678 -2.104678 0 -2.104678 NaN NaN NaN NaN NaN 0.0 0.0 0 0.0 NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 1.078751 0.0 0.264327 0.0\n", "2024-12-31 22:45:00 -2.089444 -2.089444 0 -2.089444 NaN NaN NaN NaN NaN 0.0 0.0 0 0.0 NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 1.079759 0.0 0.264447 0.0\n", "2024-12-31 23:15:00 -2.355761 -2.355761 0 -2.355761 NaN NaN NaN NaN NaN 0.0 0.0 0 0.0 NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 1.062164 0.0 0.262373 0.0\n", "2024-12-31 23:45:00 -2.578839 -2.578839 0 -2.578839 NaN NaN NaN NaN NaN 0.0 0.0 0 0.0 NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 1.047483 0.0 0.260688 0.0\n", "\n", "[350640 rows x 129 columns]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "partitioning_results = \"61.2_CH-CHA_NEE_RF-GAPF_PART_RP-20250319215835.csv\"\n", "results = pd.read_csv(partitioning_results)\n", "results = results.set_index(\"TIMESTAMP\")\n", "results.index.name = \"TIMESTAMP_END\"\n", "results = TimestampSanitizer(data=results).get()\n", "results" ] }, { "cell_type": "markdown", "id": "38845c9e-b231-4967-a2d8-4547e698f546", "metadata": {}, "source": [ "# Identify GPP and RECO columns" ] }, { "cell_type": "code", "execution_count": 3, "id": "cb8b7d58-f86e-40ea-8a4b-db41383cb0db", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Reco_U16',\n", " 'GPP_U16_f',\n", " 'Reco_DT_U16',\n", " 'GPP_DT_U16',\n", " 'Reco_DT_U16_SD',\n", " 'GPP_DT_U16_SD',\n", " 'Reco_U84',\n", " 'GPP_U84_f',\n", " 'Reco_DT_U84',\n", " 'GPP_DT_U84',\n", " 'Reco_DT_U84_SD',\n", " 'GPP_DT_U84_SD',\n", " 'Reco_U50',\n", " 'GPP_U50_f',\n", " 'Reco_DT_U50',\n", " 'GPP_DT_U50',\n", " 'Reco_DT_U50_SD',\n", " 'GPP_DT_U50_SD']" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "partcols = [c for c in results.columns if any(substring in c for substring in [\"GPP\", \"Reco\"])];\n", "partcols = [c for c in partcols if not str(c).endswith(\"_fqc\")] # These are from REddyProc's MDS gap-filling, but data were already gap-filled, therefore not needed\n", "partcols.remove(\"FP_GPP2000\") # Not needed\n", "partcols" ] }, { "cell_type": "markdown", "id": "ee014d2e-63c5-4e6f-a23a-842879718f62", "metadata": {}, "source": [ "# Create subset with GPP and RECO columns" ] }, { "cell_type": "code", "execution_count": 4, "id": "059994b4-b415-4f21-8981-8f5423f73fa2", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Reco_U16GPP_U16_fReco_DT_U16GPP_DT_U16Reco_DT_U16_SDGPP_DT_U16_SDReco_U84GPP_U84_fReco_DT_U84GPP_DT_U84Reco_DT_U84_SDGPP_DT_U84_SDReco_U50GPP_U50_fReco_DT_U50GPP_DT_U50Reco_DT_U50_SDGPP_DT_U50_SD
TIMESTAMP_MIDDLE
2005-01-01 00:15:001.8007480.5629110.4769220.00.2938650.01.7468951.1058250.0892050.00.1229120.01.8305430.9185530.0930710.00.0800160.0
2005-01-01 00:45:001.7993030.5753360.4751550.00.2928170.01.7441071.1014360.0888430.00.1224130.01.8288980.9179720.0926820.00.0796880.0
2005-01-01 01:15:001.8007480.1703410.4769220.00.2938650.01.7468950.4621040.0892050.00.1229120.01.8305430.1630010.0930710.00.0800160.0
2005-01-01 01:45:001.8007480.2772980.4769220.00.2938650.01.7468950.4608660.0892050.00.1229120.01.8305430.1908900.0930710.00.0800160.0
2005-01-01 02:15:001.7978560.1893330.4733920.00.2917720.01.7413200.4028700.0884820.00.1219160.01.8272530.1670420.0922950.00.0793610.0
.........................................................
2024-12-31 21:45:000.763007-0.4011740.7437880.00.3650330.00.946163-0.2643341.5002570.00.3212810.00.825725-0.3349961.0910280.00.2658080.0
2024-12-31 22:15:000.759039-0.3699660.7321020.00.3591920.00.936404-0.2616331.4823910.00.3216970.00.820921-0.3105331.0787510.00.2643270.0
2024-12-31 22:45:000.759366-0.2876010.7330600.00.3596700.00.937206-0.1097611.4838580.00.3216630.00.821317-0.2256511.0797590.00.2644470.0
2024-12-31 23:15:000.753639-0.6190350.7163990.00.3514040.00.923204-0.4494701.4582740.00.3222790.00.814389-0.5582851.0621640.00.2623730.0
2024-12-31 23:45:000.748823-0.2870360.7025810.00.3446080.00.911507-0.2341881.4369470.00.3228070.00.808567-0.3175431.0474830.00.2606880.0
\n", "

350640 rows × 18 columns

\n", "
" ], "text/plain": [ " Reco_U16 GPP_U16_f Reco_DT_U16 GPP_DT_U16 Reco_DT_U16_SD GPP_DT_U16_SD Reco_U84 GPP_U84_f Reco_DT_U84 GPP_DT_U84 Reco_DT_U84_SD GPP_DT_U84_SD Reco_U50 GPP_U50_f Reco_DT_U50 GPP_DT_U50 Reco_DT_U50_SD GPP_DT_U50_SD\n", "TIMESTAMP_MIDDLE \n", "2005-01-01 00:15:00 1.800748 0.562911 0.476922 0.0 0.293865 0.0 1.746895 1.105825 0.089205 0.0 0.122912 0.0 1.830543 0.918553 0.093071 0.0 0.080016 0.0\n", "2005-01-01 00:45:00 1.799303 0.575336 0.475155 0.0 0.292817 0.0 1.744107 1.101436 0.088843 0.0 0.122413 0.0 1.828898 0.917972 0.092682 0.0 0.079688 0.0\n", "2005-01-01 01:15:00 1.800748 0.170341 0.476922 0.0 0.293865 0.0 1.746895 0.462104 0.089205 0.0 0.122912 0.0 1.830543 0.163001 0.093071 0.0 0.080016 0.0\n", "2005-01-01 01:45:00 1.800748 0.277298 0.476922 0.0 0.293865 0.0 1.746895 0.460866 0.089205 0.0 0.122912 0.0 1.830543 0.190890 0.093071 0.0 0.080016 0.0\n", "2005-01-01 02:15:00 1.797856 0.189333 0.473392 0.0 0.291772 0.0 1.741320 0.402870 0.088482 0.0 0.121916 0.0 1.827253 0.167042 0.092295 0.0 0.079361 0.0\n", "... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...\n", "2024-12-31 21:45:00 0.763007 -0.401174 0.743788 0.0 0.365033 0.0 0.946163 -0.264334 1.500257 0.0 0.321281 0.0 0.825725 -0.334996 1.091028 0.0 0.265808 0.0\n", "2024-12-31 22:15:00 0.759039 -0.369966 0.732102 0.0 0.359192 0.0 0.936404 -0.261633 1.482391 0.0 0.321697 0.0 0.820921 -0.310533 1.078751 0.0 0.264327 0.0\n", "2024-12-31 22:45:00 0.759366 -0.287601 0.733060 0.0 0.359670 0.0 0.937206 -0.109761 1.483858 0.0 0.321663 0.0 0.821317 -0.225651 1.079759 0.0 0.264447 0.0\n", "2024-12-31 23:15:00 0.753639 -0.619035 0.716399 0.0 0.351404 0.0 0.923204 -0.449470 1.458274 0.0 0.322279 0.0 0.814389 -0.558285 1.062164 0.0 0.262373 0.0\n", "2024-12-31 23:45:00 0.748823 -0.287036 0.702581 0.0 0.344608 0.0 0.911507 -0.234188 1.436947 0.0 0.322807 0.0 0.808567 -0.317543 1.047483 0.0 0.260688 0.0\n", "\n", "[350640 rows x 18 columns]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "subset_partcols = results[partcols].copy()\n", "subset_partcols" ] }, { "cell_type": "markdown", "id": "1d483617-368f-4310-8ad1-225803d156ab", "metadata": {}, "source": [ "# Rename partitioning variables" ] }, { "cell_type": "markdown", "id": "070ec11e-1045-4437-bb70-45d60efc4f60", "metadata": {}, "source": [ "These original NEE flux columns were renamed and then used during partitioning:\n", "- NEE_L3.1_L3.3_CUT_16_QCF_gfRF\n", "- NEE_L3.1_L3.3_CUT_50_QCF_gfRF\n", "- NEE_L3.1_L3.3_CUT_84_QCF_gfRF" ] }, { "cell_type": "code", "execution_count": 5, "id": "49e1844b-a2e3-4603-ba77-7b358998874b", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
RECO_NT_CUT_16_gfRFGPP_NT_CUT_16_gfRFRECO_DT_CUT_16_gfRFGPP_DT_CUT_16_gfRFRECO_DT_CUT_16_gfRF_SDGPP_DT_CUT_16_gfRF_SDRECO_NT_CUT_84_gfRFGPP_NT_CUT_84_gfRFRECO_DT_CUT_84_gfRFGPP_DT_CUT_84_gfRFRECO_DT_CUT_84_gfRF_SDGPP_DT_CUT_84_gfRF_SDRECO_NT_CUT_50_gfRFGPP_NT_CUT_50_gfRFRECO_DT_CUT_50_gfRFGPP_DT_CUT_50_gfRFRECO_DT_CUT_50_gfRF_SDGPP_DT_CUT_50_gfRF_SD
TIMESTAMP_MIDDLE
2005-01-01 00:15:001.8007480.5629110.4769220.00.2938650.01.7468951.1058250.0892050.00.1229120.01.8305430.9185530.0930710.00.0800160.0
2005-01-01 00:45:001.7993030.5753360.4751550.00.2928170.01.7441071.1014360.0888430.00.1224130.01.8288980.9179720.0926820.00.0796880.0
2005-01-01 01:15:001.8007480.1703410.4769220.00.2938650.01.7468950.4621040.0892050.00.1229120.01.8305430.1630010.0930710.00.0800160.0
2005-01-01 01:45:001.8007480.2772980.4769220.00.2938650.01.7468950.4608660.0892050.00.1229120.01.8305430.1908900.0930710.00.0800160.0
2005-01-01 02:15:001.7978560.1893330.4733920.00.2917720.01.7413200.4028700.0884820.00.1219160.01.8272530.1670420.0922950.00.0793610.0
.........................................................
2024-12-31 21:45:000.763007-0.4011740.7437880.00.3650330.00.946163-0.2643341.5002570.00.3212810.00.825725-0.3349961.0910280.00.2658080.0
2024-12-31 22:15:000.759039-0.3699660.7321020.00.3591920.00.936404-0.2616331.4823910.00.3216970.00.820921-0.3105331.0787510.00.2643270.0
2024-12-31 22:45:000.759366-0.2876010.7330600.00.3596700.00.937206-0.1097611.4838580.00.3216630.00.821317-0.2256511.0797590.00.2644470.0
2024-12-31 23:15:000.753639-0.6190350.7163990.00.3514040.00.923204-0.4494701.4582740.00.3222790.00.814389-0.5582851.0621640.00.2623730.0
2024-12-31 23:45:000.748823-0.2870360.7025810.00.3446080.00.911507-0.2341881.4369470.00.3228070.00.808567-0.3175431.0474830.00.2606880.0
\n", "

350640 rows × 18 columns

\n", "
" ], "text/plain": [ " RECO_NT_CUT_16_gfRF GPP_NT_CUT_16_gfRF RECO_DT_CUT_16_gfRF GPP_DT_CUT_16_gfRF RECO_DT_CUT_16_gfRF_SD GPP_DT_CUT_16_gfRF_SD RECO_NT_CUT_84_gfRF GPP_NT_CUT_84_gfRF RECO_DT_CUT_84_gfRF GPP_DT_CUT_84_gfRF RECO_DT_CUT_84_gfRF_SD GPP_DT_CUT_84_gfRF_SD RECO_NT_CUT_50_gfRF GPP_NT_CUT_50_gfRF RECO_DT_CUT_50_gfRF GPP_DT_CUT_50_gfRF RECO_DT_CUT_50_gfRF_SD GPP_DT_CUT_50_gfRF_SD\n", "TIMESTAMP_MIDDLE \n", "2005-01-01 00:15:00 1.800748 0.562911 0.476922 0.0 0.293865 0.0 1.746895 1.105825 0.089205 0.0 0.122912 0.0 1.830543 0.918553 0.093071 0.0 0.080016 0.0\n", "2005-01-01 00:45:00 1.799303 0.575336 0.475155 0.0 0.292817 0.0 1.744107 1.101436 0.088843 0.0 0.122413 0.0 1.828898 0.917972 0.092682 0.0 0.079688 0.0\n", "2005-01-01 01:15:00 1.800748 0.170341 0.476922 0.0 0.293865 0.0 1.746895 0.462104 0.089205 0.0 0.122912 0.0 1.830543 0.163001 0.093071 0.0 0.080016 0.0\n", "2005-01-01 01:45:00 1.800748 0.277298 0.476922 0.0 0.293865 0.0 1.746895 0.460866 0.089205 0.0 0.122912 0.0 1.830543 0.190890 0.093071 0.0 0.080016 0.0\n", "2005-01-01 02:15:00 1.797856 0.189333 0.473392 0.0 0.291772 0.0 1.741320 0.402870 0.088482 0.0 0.121916 0.0 1.827253 0.167042 0.092295 0.0 0.079361 0.0\n", "... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...\n", "2024-12-31 21:45:00 0.763007 -0.401174 0.743788 0.0 0.365033 0.0 0.946163 -0.264334 1.500257 0.0 0.321281 0.0 0.825725 -0.334996 1.091028 0.0 0.265808 0.0\n", "2024-12-31 22:15:00 0.759039 -0.369966 0.732102 0.0 0.359192 0.0 0.936404 -0.261633 1.482391 0.0 0.321697 0.0 0.820921 -0.310533 1.078751 0.0 0.264327 0.0\n", "2024-12-31 22:45:00 0.759366 -0.287601 0.733060 0.0 0.359670 0.0 0.937206 -0.109761 1.483858 0.0 0.321663 0.0 0.821317 -0.225651 1.079759 0.0 0.264447 0.0\n", "2024-12-31 23:15:00 0.753639 -0.619035 0.716399 0.0 0.351404 0.0 0.923204 -0.449470 1.458274 0.0 0.322279 0.0 0.814389 -0.558285 1.062164 0.0 0.262373 0.0\n", "2024-12-31 23:45:00 0.748823 -0.287036 0.702581 0.0 0.344608 0.0 0.911507 -0.234188 1.436947 0.0 0.322807 0.0 0.808567 -0.317543 1.047483 0.0 0.260688 0.0\n", "\n", "[350640 rows x 18 columns]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "renaming_dict = {\n", " \n", " 'GPP_DT_U16': 'GPP_DT_CUT_16_gfRF',\n", " 'GPP_DT_U16_SD': 'GPP_DT_CUT_16_gfRF_SD',\n", " 'GPP_DT_U50': 'GPP_DT_CUT_50_gfRF',\n", " 'GPP_DT_U50_SD': 'GPP_DT_CUT_50_gfRF_SD',\n", " 'GPP_DT_U84': 'GPP_DT_CUT_84_gfRF',\n", " 'GPP_DT_U84_SD': 'GPP_DT_CUT_84_gfRF_SD',\n", "\n", " 'GPP_U16_f': 'GPP_NT_CUT_16_gfRF', \n", " 'GPP_U50_f': 'GPP_NT_CUT_50_gfRF', \n", " 'GPP_U84_f': 'GPP_NT_CUT_84_gfRF', \n", "\n", " 'Reco_DT_U16': 'RECO_DT_CUT_16_gfRF',\n", " 'Reco_DT_U16_SD': 'RECO_DT_CUT_16_gfRF_SD',\n", " 'Reco_DT_U50': 'RECO_DT_CUT_50_gfRF',\n", " 'Reco_DT_U50_SD': 'RECO_DT_CUT_50_gfRF_SD',\n", " 'Reco_DT_U84': 'RECO_DT_CUT_84_gfRF',\n", " 'Reco_DT_U84_SD': 'RECO_DT_CUT_84_gfRF_SD',\n", "\n", " 'Reco_U16': 'RECO_NT_CUT_16_gfRF',\n", " 'Reco_U50': 'RECO_NT_CUT_50_gfRF',\n", " 'Reco_U84': 'RECO_NT_CUT_84_gfRF',\n", "}\n", "subset_partcols = subset_partcols.rename(columns=renaming_dict, inplace=False)\n", "subset_partcols" ] }, { "cell_type": "markdown", "id": "a71fbd41-675c-4ca9-983c-f49d4229ef4f", "metadata": {}, "source": [ "# Export " ] }, { "cell_type": "markdown", "id": "c144fe4a-0d34-4956-81e6-1b26c2922208", "metadata": {}, "source": [ "## Export all data" ] }, { "cell_type": "code", "execution_count": 6, "id": "97948e54-2782-4b01-8656-3d1846616e24", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Saved file 61.4_PARTITIONED_FLUXES_GPP_RECO.parquet (0.456 seconds).\n" ] }, { "data": { "text/plain": [ "'61.4_PARTITIONED_FLUXES_GPP_RECO.parquet'" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "filename = \"61.4_PARTITIONED_FLUXES_GPP_RECO\"\n", "# subset_partcols.to_csv(f\"{filename}.csv\", index=True)\n", "save_parquet(data=subset_partcols, filename=filename)" ] }, { "cell_type": "markdown", "id": "46451b7f-afa2-4a15-b41e-7e96166f1d64", "metadata": {}, "source": [ "# **End of notebook**" ] }, { "cell_type": "markdown", "id": "6784eedd-cdbf-4af3-8c37-622ea78db69f", "metadata": {}, "source": [ "Congratulations, you reached the end of this notebook! Before you go let's store your finish time." ] }, { "cell_type": "code", "execution_count": 7, "id": "65dafb8d-15c8-465b-915e-8f22c5adcce3", "metadata": { "ExecuteTime": { "end_time": "2024-10-04T12:05:15.229626Z", "start_time": "2024-10-04T12:05:15.220627Z" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Finished. 2025-05-14 12:14:22\n" ] } ], "source": [ "from datetime import datetime\n", "dt_string = datetime.now().strftime(\"%Y-%m-%d %H:%M:%S\")\n", "print(f\"Finished. {dt_string}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "3f489141-150c-4c4d-8d93-e0291cb64478", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 5 }