ecpet.ecpack module¶
EC-PeT Core Processing Module¶
Comprehensive eddy-covariance flux calculation engine implementing the complete processing chain from raw measurements to final surface exchange fluxes. Provides instrument calibration, coordinate transformations, atmospheric corrections, and flux calculations following established micrometeorological principles and best practices for eddy-covariance methodology.
- The module performs:
Multi-sensor calibration (sonic anemometers, gas analyzers, thermocouples)
Statistical analysis with uncertainty quantification
Coordinate system transformations and tilt corrections
Atmospheric density and frequency response corrections
Surface flux calculations with error propagation
Quality assessment and data validation
- ecpet.ecpack.averag(df)¶
Calculate mean values, covariances, and their tolerances from calibrated time series.
- Parameters:
df (pandas.DataFrame) – DataFrame containing calibrated measurement samples
- Returns:
Tuple of (means, tolerance_means, covariances, tolerance_covariances, independent_samples_means, independent_samples_covariances)
- Return type:
Estimates tolerances as 2 * sigma / sqrt(N_independent) where N_independent is estimated from sign changes in fluctuations. Only uses samples with valid data.
Name in ECPACK: EC_M_Averag
- ecpet.ecpack.calibrat(conf, dat, pres, cormean=None, badtc=False, tref=nan)¶
Apply instrument-specific calibrations to raw measurement data.
- Parameters:
conf (object) – Configuration object with calibration parameters
dat (pandas.DataFrame) – DataFrame containing raw measurement data
pres (float) – Atmospheric pressure [Pa]
cormean (dict, optional) – Correction terms for calibrated quantities, defaults to None
badtc (bool, optional) – Flag indicating bad thermocouple, defaults to False
tref (float, optional) – Reference temperature [K], defaults to np.nan
- Returns:
DataFrame with calibrated measurement data
- Return type:
- Applies sonic anemometer calibration, coordinate rotations,
Schotanus corrections, and sensor-specific polynomial calibrations. Calculates derived quantities like specific humidity and CO2 mixing ratios.
Name in ECPACK: Calibrat
- ecpet.ecpack.convert(conf, dat)¶
Apply gain and offset corrections for unit conversion.
- Parameters:
conf (object) – Configuration object with gain/offset parameters
dat (pandas.DataFrame) – DataFrame containing measurement data
- Returns:
DataFrame with converted data including time delays
- Return type:
Applies mapping x → (x/Gain) + Offset and time delays specified in configuration. Delays converted from milliseconds to sample counts.
- ecpet.ecpack.detrend(x)¶
Construct linearly detrended dataset while preserving mean values.
- Parameters:
x (pandas.DataFrame or pandas.Series) – DataFrame or Series with measurement samples
- Returns:
Detrended data with original mean added back
- Return type:
- Uses linear regression to remove trends while handling NaN values.
Preserves data structure (DataFrame vs Series) of input.
- ecpet.ecpack.crosscorr(datax, datay, lag=0)¶
Calculate lag-N cross correlation between two time series.
- Parameters:
datax (pandas.Series) – First time series
datay (pandas.Series) – Second time series
lag (int, optional) – Time lag in samples, defaults to 0
- Returns:
Cross correlation coefficient
- Return type:
- Based on pandas correlation with lag shifting.
Used for sensor synchronization analysis.
- ecpet.ecpack.errfisi(val)¶
Calculate covariance error matrix using Finkelstein-Sims method.
- Parameters:
val (pandas.DataFrame) – DataFrame with calibrated measurement samples
- Returns:
Covariance error matrix
- Return type:
- Uses FFT-based high-pass filtering and correlation analysis.
Provides alternative error estimation to traditional methods. Returns NaN matrix if insufficient data (< 8 samples).
- ecpet.ecpack.freqcorr(conf, Means, Covs, TolCov, WhichTemp)¶
Calculate frequency response corrections for sensor and path effects.
- Parameters:
conf (object) – Configuration object with sensor parameters
Means (pandas.Series) – Mean values of measured quantities
Covs (pandas.DataFrame) – Covariance matrix of fluctuations
TolCov (pandas.DataFrame) – Tolerance matrix for covariances
WhichTemp (str) – Temperature variable to use (ts or tcoup)
- Returns:
Tuple of (corrected_covariances, corrected_tolerances)
- Return type:
Corrects for sensor response, path averaging, separation effects, and signal processing. Based on [Moore, 1986] and others. Only corrects variances and covariances involving vertical velocity.
- See also
- ecpet.ecpack.mapmtx(a, b)¶
Calculate matrix-vector product y = A·b for coordinate transformations.
- Parameters:
a (pandas.DataFrame) – Transformation matrix
b (pandas.DataFrame) – Vector or matrix to be transformed
- Returns:
Transformed vector/matrix
- Return type:
- Applies transformation to each row using matrix multiplication.
Used extensively in coordinate system rotations.
Name in ECPACK: EC_M_MapMtx
- ecpet.ecpack.flux(means, tolmean, covs, tolcov, badtc, p, webvel, diryaw)¶
Calculate surface fluxes from mean values and covariances.
- Parameters:
means (pandas.Series) – Mean values of all variables
tolmean (pandas.Series) – Tolerances in mean values
covs (pandas.DataFrame) – Covariance matrix of all variables
tolcov (pandas.DataFrame) – Tolerances in covariances
badtc (bool) – Flag for corrupt thermocouple temperature
p (float) – Atmospheric pressure [Pa]
webvel (float) – Webb correction velocity [m/s]
diryaw (float) – Yaw rotation angle [degrees]
- Returns:
Tuple of (physical_quantities, uncertainties)
- Return type:
- Calculates friction velocity, sensible/latent heat fluxes,
CO2 fluxes, wind statistics, and their uncertainties. Handles both sonic and thermocouple temperature options.
Name in ECPACK: EC_Ph_Flux
- ecpet.ecpack.obukhov(Ustar, Tstar, Qstar, MeanT)¶
Calculate Obukhov length including water vapor buoyancy effects.
- Parameters:
- Returns:
Obukhov length [m]
- Return type:
- Uses standard Monin-Obukhov similarity theory formulation
with humidity correction for buoyancy effects.
name in ECPACK: EC_Ph_Obukhov
- ecpet.ecpack.oxygen(MeanT, Covin, P, HygType, WhichTemp)¶
Correct hygrometer measurements for oxygen absorption effects.
- Parameters:
MeanT (float) – Mean temperature [K]
Covin (pandas.DataFrame) – Input covariance matrix
P (float) – Atmospheric pressure [Pa]
HygType (str) – Hygrometer type code
WhichTemp (str) – Temperature variable used
- Returns:
Corrected covariance matrix
- Return type:
- Applies corrections for KH20, Lyman-alpha, and LiCor hygrometers.
LiCor IR hygrometers are not sensitive to oxygen.
Names in ECPACK: EC_C_Oxygen1 and EC_C_Oxygen2
- ecpet.ecpack.scal(conf, raw)¶
Apply wind tunnel calibration to sonic anemometer data.
- Parameters:
conf (object) – Configuration object with calibration coefficients
raw (pandas.DataFrame) – DataFrame with raw sonic measurements
- Returns:
DataFrame with calibrated sonic data
- Return type:
The method is based on a wind tunnel calibration of the sonic The real velocity components can be derived from the measured components and the real azimuth (\(\varphi\)) and elevation angle (\(\theta\)). But the latter are not known and have to be determined iteratively from the measured components. The relationship between the real components and the measured components is:
\[ \begin{align}\begin{aligned}U_\mathrm{real} &= \frac{U_\mathrm{meas}} {U_{C1}\left(1 - 0.5 \left( (\varphi + (\theta/0.5236) \cdot U_{C2}) \cdot (1 - U_{C3} \cdot |\theta/0.5236|)\right)^2 \right)}\\V_\mathrm{real} &= V_\mathrm{meas} \cdot (1 - V_{C1} \cdot |\theta/0.5236|)\\W_\mathrm{real} &= \frac{W_\mathrm{meas}} {W_{C1} \left(1 - 0.5 (\varphi \cdot W_{C2})^2\right)}\\\varphi &= \arctan\left(\frac{V}{U}\right)\\\theta &= \arctan\left(\frac{W}{\sqrt{U^2 + V^2}}\right)\end{aligned}\end{align} \]where \(U_{C1}\), \(U_{C2}\), \(U_{C3}\), \(V_{C1}\), \(W_{C1}\), \(W_{C2}\) are fitting coefficients. An azimuth angle of zero is supposed to refer to a wind direction from the most optimal direction (i.e. the ‘open’ side of a sonic). Samples with an absolute azimuth angle of more than 40 degrees are rejected.
- ecpet.ecpack.schotanus1(MeanQ, MeanTSon, Covin)¶
Compute Schotanus correction factors for sonic temperature and humidity effects.
- Parameters:
MeanQ (float) – Mean specific humidity [kg/kg]
MeanTSon (float) – Mean sonic temperature [K]
Covin (pandas.DataFrame) – Input covariance matrix
- Returns:
Tuple of (correction_factors, temperature_factor)
- Return type:
First step of Schotanus correction accounting for humidity effects on sonic temperature measurements. Sidewind correction applied separately.
Name in ECPACK: EC_C_Schot1
- ecpet.ecpack.schotanus2(factor, TSonFact, MeanTSon, Covin)¶
Apply Schotanus correction factors to covariances and temperature.
- Parameters:
factor (dict) – Correction factors from schotanus1
TSonFact (float) – Temperature correction factor
MeanTSon (float) – Mean sonic temperature [K]
Covin (pandas.DataFrame) – Input covariance matrix
- Returns:
Tuple of (corrected_covariances, corrected_temperature)
- Return type:
Second step of Schotanus correction applying factors computed in schotanus1 to covariance matrix and mean temperature.
Name in ECPACK: EC_C_Schot2
- ecpet.ecpack.schotanus3(temp, rhov, press)¶
Apply humidity correction to sonic temperature iteratively.
- Parameters:
- Returns:
Humidity-corrected sonic temperature [K]
- Return type:
Iterative solution for humidity-corrected temperature when thermocouple unavailable. Convergence criterion: 0.01% change.
- ecpet.ecpack.tilt_matrix_pitch(Direction)¶
Generate rotation matrix for pitch angle correction.
- Parameters:
Direction (float) – Pitch angle in degrees
- Returns:
3x3 pitch rotation matrix
- Return type:
Rotation about y-axis (0,1,0). Positive angles rotate x-axis toward z-axis.
Name in ECPACK: EC_C_T08
- ecpet.ecpack.tilt_matrix_roll(Direction)¶
Generate rotation matrix for roll angle correction.
- Parameters:
Direction (float) – Roll angle in degrees
- Returns:
3x3 roll rotation matrix
- Return type:
Rotation about x-axis (1,0,0). Positive angles rotate y-axis toward z-axis.
Name in ECPACK: EC_C_T10
- ecpet.ecpack.tilt_matrix_yaw(Direction)¶
Generate rotation matrix for yaw angle correction.
- Parameters:
Direction (float) – Yaw angle in degrees
- Returns:
3x3 yaw rotation matrix
- Return type:
Rotation about z-axis (0,0,1). Positive angles rotate x-axis toward y-axis.
Name in ECPACK: EC_C_T06
- ecpet.ecpack.tilt_mean_dir(MeanU, MeanV)¶
Calculate yaw angle to eliminate mean lateral wind component.
- Parameters:
- Returns:
Yaw angle in degrees
- Return type:
Rotates coordinate system so mean v-velocity = 0. Aligns x-axis with mean horizontal wind direction.
Name in ECPACK: EC_C_T07
- ecpet.ecpack.tilt_mean_roll(CovVV, CovVW, CovWW)¶
Calculate roll angle to eliminate v-w covariance.
- Parameters:
- Returns:
Roll angle in degrees
- Return type:
Rotates coordinate system so Cov(v,w) = 0. Makes vertical and lateral wind fluctuations independent.
Name in ECPACK: EC_C_T11
- ecpet.ecpack.tilt_mean_vert(MeanU, MeanW)¶
Calculate pitch angle to eliminate mean vertical wind component.
- Parameters:
- Returns:
Pitch angle in degrees
- Return type:
Rotates coordinate system so mean w-velocity = 0. Aligns measurement plane with mean streamline.
Name in ECPACK: EC_C_T09
- ecpet.ecpack.tilt_rot_mean(Meani, Covi, Map)¶
Apply coordinate transformation to mean values and covariances.
- Parameters:
Meani (pandas.Series) – Input mean values
Covi (pandas.DataFrame) – Input covariance matrix
Map (pandas.DataFrame) – 3x3 rotation matrix
- Returns:
Tuple of (rotated_means, rotated_covariances)
- Return type:
Rotates velocity components and all covariances involving velocity. Used by all tilt correction procedures.
Names in ECPACK: EC_C_T05, EC_C_T03, and EC_C_T04
- ecpet.ecpack.tilt_rot_speed(Sample, Map)¶
Apply coordinate transformation to raw velocity samples.
- Parameters:
Sample (pandas.DataFrame) – DataFrame with velocity measurements
Map (pandas.DataFrame) – 3x3 rotation matrix
- Returns:
DataFrame with rotated velocities
- Return type:
Applies rotation only to velocity components (ux, uy, uz). Other variables remain unchanged.
- ecpet.ecpack.calcstruct(conf, sample, cindep)¶
- ecpet.ecpack.webb(Means, Covs, P, WhichTemp)¶
Calculate Webb correction velocity for density effects.
- Parameters:
Means (pandas.Series) – Mean values of all variables
Covs (pandas.DataFrame) – Covariance matrix
P (float) – Atmospheric pressure [Pa]
WhichTemp (str) – Temperature variable to use
- Returns:
Webb correction velocity [m/s]
- Return type:
Corrects scalar fluxes for air density fluctuations following [Webb et al., 1980]. Accounts for temperature and humidity effects on air density.
Name in ECPACK: EC_C_Webb
- ecpet.ecpack.flux_to_file(conf, intervals)¶
Write flux calculation results to formatted output file.
- Parameters:
conf (object) – Configuration object with output settings
intervals (pandas.DataFrame) – DataFrame containing flux results
Creates comprehensive output file with fluxes, meteorology, quality measures, and covariances in EC-PACK format. Handles NaN values and unit conversions for output.
- ecpet.ecpack.cmain(conf, interval, Means, TolMean, Covs, TolCov)¶
Apply all flux corrections with optional iteration for interdependence.
- Parameters:
conf (object) – Configuration object with correction settings
interval (dict) – Single interval data record
Means (pandas.Series) – Mean values of calibrated signals
TolMean (pandas.Series) – Tolerances in mean values
Covs (pandas.DataFrame) – Covariance matrix
TolCov (pandas.DataFrame) – Covariance tolerances
- Returns:
Tuple of corrected means, tolerances, covariances, and Webb velocity
- Return type:
Orchestrates Schotanus, oxygen, frequency response, and Webb corrections. Supports iteration to account for interdependence following :cite`ofv_bm07`
Name in ECPACK: EC_C_MAIN
- ecpet.ecpack.process_flux_interval(args)¶
Process single averaging interval for flux calculations.
- Parameters:
args (tuple) – Tuple of (configuration, interval_data) for multiprocessing
- Returns:
Dictionary with processed interval results
- Return type:
Complete flux processing workflow including data retrieval, calibration, corrections, tilt correction, and flux calculation. Designed for parallel execution.
- ecpet.ecpack.process_flux(conf, intervals)¶
Process all flux intervals with optional parallel execution.
- Parameters:
conf (object) – Configuration object with processing parameters
intervals (pandas.DataFrame) – DataFrame with processing intervals
- Returns:
DataFrame with flux calculation results
- Return type:
Coordinates parallel processing of flux intervals using multiprocessing Pool. Number of processes controlled by conf[‘nproc’].