Configuration

Configuration File

EC-PeT uses a single configuration file. The tokens are mostly identical to the ones used by EC-frame and modified EC-PACK. The tokens are organized in groups (corresponding to the different configuration files used by EC-PACK). The tokens in each group begin with a specific prefix, followed by a period and the actual token name.

Comments: In each line, any characters behind a hash (#) are ignored as comments (the sequence //$ used by EC-PACK is not supported).

Main Group

Files and Directories

Token

Meaning

OutDir

Directory for output files (flux files, intermediate result file, flags files), will be overridden by command line arguments

DatDir

Directory for data files

Parmdir

Directory for parameter file (processing parameters, calibration files, interval files)

InterName

Name of interval file

FluxName

Name of ECPACK flux output file

Variable Names

Token

Meaning

U_var

Name of u-velocity variable in TOA5 column definition, will be used also for the NetCDF file

V_var

Name of v-velocity variable in TOA5 column definition, will be used also for the NetCDF file

W_var

Name of w-velocity variable in TOA5 column definition, will be used also for the NetCDF file

CO2_var

Name of CO₂ variables in TOA5 column definition, will be used also for the NetCDF file (CO₂ concentration should be in kg/m³ or the calibration file should take care of that)

Humidity_var

Name of humidity variable in TOA5 column definition, will be used also for the NetCDF file (H₂O concentration should be in kg/m³ or the calibration file should take care of that)

Tsonic_var

Name of sonic temperature variable in TOA5 column definition, will be used also for the NetCDF file

diag_var

Name of diagnostic variable TOA5 column definition, will be used also for the NetCDF file

Doy_var

Name of day of year variable in NetCDF file

Hourmin_var

Name of hour/minute variable in NetCDF file (presently this should be a la the Campbell hour/minutes: e.g. 1845 means 18:45)

sec_var

Name of seconds variable in NetCDF

Device Calibration Groups

Supported Devices

EC-PeT supports up to four different devices taking measurements at high frequency (referenced as ‘fast data’):

  • a three-dimensional sonic anemometer,

  • a separate fast thermometer,

  • a fast Hygrometer, and

  • a fast CO₂ sensor,

as well as up tho three conventional sensors that can be used to provide potentially more stable reference values for mean values (referenced as ‘slow data’):

  • a thermometer,

  • a barometer, and

  • a hygrometer.

Each sesnsor is optional, except for the anemometer. If mulitple of these measurements are taken by the same device (for exapmle: humidity and CO₂), they are configured as if the were taken by indiviual devices.

Information about the four fast sensors is provided in separate groups:

Prefix

Device

SonCal

Sonic anemometer

TcoupCal

Fast thermometer (also referred to as thermocouple)

HygCal

Fast hygrometer

Co2Cal

CO₂ sensor

Device Type Codes

A total of 12 types of sensors is known:

Code

Device

1

Campbell Sonic (30 degree angle with vertical)

2

Thermo couple (undefined type)

3

Krypton hygrometer

4

PT100 thermometer

5

Psychrometer (used for what?)

6

Wind tunnel calibrated sonic anemometer

7

Mierij Lyman-alpha

8

LiCor7500 IR H₂O/CO₂ sensor

9

Kaijo Denki TR90 3-D sonic

10

Kaijo Denki TR61 3-D sonic

11

Gill Solent anemometer (45 degree angle with vertical, wind direction might still be incorrect: not tested)

12

Generic sonic (specific parameters are passed as extra calibration coefficients)

Calibration Parameters

Each device group supports the following parameters (<prefix> is one of the group prefixex listed above):

Parameter

Description
<prefix>.QQX
<prefix>.QQY
<prefix>.QQZ

Position coordinates (m) and height above ground (or better: above the displacement height) for frequency response corrections

<prefix>.QQType

Device type code (see table above)

<prefix>.QQGain

Calibration gain factor

<prefix>.QQOffset

Calibration offset

<prefix>.QQFunc

Calibration function type (1=polynomial, 2=logarithmic polynomial), only evaluated with QQType = 2 (thermocouple), 3 and 7 (hygrometer), and 8 (H₂O/CO₂ sensor)
<prefix>.QQC0
<prefix>.QQC1
<prefix>.QQOrder

Calibration curve coefficients
<prefix>.QQExt1
…
<prefix>.QQExt5

Calibration factors specific to a type of sensor
<prefix>.QQExt5
…
<prefix>.QQExt9

Additional calibration factors specific to sensor type 12
<prefix>.QQExt6
…
<prefix>.QQExt11

Additional calibration factors specific to sensor type 6

Calibration Order

The order in which the various gains, offsets and calibrations are applied to the data is:

  1. If QQGain present: sample = sample / QQGain

  2. If QQOffset present: sample = sample + QQOffset

  3. If QQOrder present: sample = calibration_function(sample)

Time Selection

Token

Meaning

DateBegin

Beginning time of the period to process, format: yyyy/mm/dd-hh:mm:ss.ss, will be overridden by command line arguments

DateEnd

End time of the period to process, format: yyyy/mm/dd-hh:mm:ss.ss, will be overridden by command line arguments

AvgInterval

Averaging interval in seconds, will be overridden by command line arguments

Site-specific Information

Token

Meaning

ExcludeSector

Definition of one or more excluded sectors (for example, where the sensors are downwind from a tower). This directive has to be repeated for each sector. Start and end of each sector should be supplied as space-separated list in degrees from north (0-360). Sectors including north have to be split at 0°.

Displacement

Aerodynamic displacement height at the instrument site

InstLatLon

Position of the Instrument: space-separated list of latitude (-90° … 90°) and longitude (-180° … 180°) in degrees.

SourceArea

Name (including path) of area-of-interest file, the file containing the definition of the targeted source area.

Selection of Tests and Corrections

Token

Meaning

Despiking

Select despiking method:

  • spk = despiking by [Vickers and Mahrt, 1997] based on standard deviation over a moving window,

  • chr = despiking based on value-to-value change rate,

  • mad = despiking based on median absolute deviation (MAD),

  • (empty) = do not apply despiking

QCdisable

Space-separated list of quality control tests to disable for faster processing, use three-letter codes from tables Preprocessor Quality Control Tests and Postprocessor Quality Control Tests.

Qflags

Select special ways to treat diagnostic words: noirga = simulate missing IRGA diag word, Campbell = rebuild CSAT diagnostic flag if scrambled by Campbell-supplied example program

FlagRules

Code for the flux the rule is valid for, followed by space-separated list of rules describing how results of individual tests should be combined to final output flag. See Flag Rules Syntax for syntax.

InterRules

Space-separated list of rules describing interdependence of final flags associated with different flux values. See Inter Rules Syntax for syntax.

Fast-data Input File Format

Token

Column containig …

FastFmt.U_col

Wind along device x axis in m/s

FastFmt.V_col

Wind along device y axis in m/s

FastFmt.W_col

Wind along device z axis in m/s

FastFmt.Humidity_col

Water vapor density in g/m³

FastFmt.TSonic_col

Sonic temperature in °C

FastFmt.Press_col

Air pressure in kPa

FastFmt.diag_col

CSAT3 diagnostic word

FastFmt.agc_col

LiCor diagnostic word

Slow-data Input File Format

Token

Column containig …

SlowFmt.TRef_col

Air temperature in °C

SlowFmt.RelHum_col

Relative humidity in percent

SlowFmt.PRef_col

Air pressure in hPa

Quality Control Configuration

Preprocessor tests

Instrument flags

Error flags raised by the instruments usually void usability of the data. Values not meeting these criteria are treated as missing.

Parameter

Default

Meaning

qcconf.csatmask

61440

Hi bits must be lo in CSAT diagnostic word

qcconf.irgamask

240

Hi bits must be hi in LiCor diagnostic word

qcconf.agclimit

Max. accepted value for LiCor auto gain control

NaN values

Value representing nans in despiked intermediate output:

Parameter

Default

Meaning

qcconf.doutnan

NaN

produces “Nan” in TOA5 and missing value in NetCDF. Other values may be selected if needed.

qcconf.spfill

0

If \(= 0\): spikes are repplacd by “Nan”, if \(\neq 0\): spikes are interpolated

1. spikes (Vickers & Mahrt)

qcconf.Parameter

Default

Meaning

qcconf.L1

Averaging subrecord width in s

qcconf.spth

3.5

Threshold in standard deviations for first pass

qcconf.spin

0.2

Threshold increment per pass in standard deviations

qcconf.spco

3

Max. number of consecutive values regarded as spike

qcconf.spcr

0.01

Max. acceptable fraction of values removed as spikes

2. Amplitude Resolution (Vickers & Mahrt)

Parameter

Default

Meaning

qcconf.widampres

1000

Averaging subrecord width in data points

qcconf.mxem

0.5

Max. acceptable fraction of empty cells in histogram

3. Dropouts (Vickers & Mahrt)

Parameter

Default

Meaning

qcconf.widdropout

1000

Averaging subrecord width in data points

qcconf.maxcon1

0.10

Max. acceptable fraction of consecutive values per subrecord within 10%-90% of value range

qcconf.maxcon2

0.06

Max. acceptable fraction of consecutive values per subrecord outside 10%-90% of value range

4. Absolute Limits

Parameter

Default

Meaning

qcconf.limu

Mean wind limit in m/s

qcconf.limw

Vertical wind limit in m/s

qcconf.limtl

-30.

Lower temperature limit in °C

qcconf.limth

Upper temperature limit in °C

qcconf.limql

Lower specific humidity limit in g/kg

qcconf.limqh

Upper specific humidity limit in g/kg

qcconf.limcl

Lower CO₂ concentration in ppm

qcconf.limch

Upper CO₂ concentration in ppm

5. Higher Moments (Vickers & Mahrt)

Parameter

Default

Meaning

qcconf.maxskew_1

1

Skewness threshold for soft flag

qcconf.maxskew_2

2

Skewness threshold for hard flag

qcconf.minkurt_1

2

Lower kurtosis threshold for soft flag

qcconf.maxkurt_1

5

Upper kurtosis threshold for soft flag

qcconf.minkurt_2

1

Lower kurtosis threshold for hard flag

qcconf.maxkurt_2

8

Upper kurtosis threshold for hard flag

6. discontinuities (Vickers & Mahrt)

Parameter

Default

Meaning

qcconf.L1

Averaging subrecord width in s

7. Nonstationarity (Vickers & Mahrt)

Parameter

Default

Meaning

qcconf.minred

0.9

Min. acceptable reduction of mean wind vector amount compared to mean wind speed

qcconf.maxrn

0.5

Max. acceptable mean, cross, and vector wind relative nonstationarity

8. Lag correlation (Vickers & Mahrt)

Parameter

Default

Meaning

qcconf.L2

Lag search window in s

9. Change Rate Spikes

Parameter

Default

Meaning

qcconf.chr_u

Max. horizontal wind change in m/s

qcconf.chr_w

Max. vertical wind change in m/s

qcconf.chr_t

Max. temperature change in K

qcconf.chrh2o

3.6

Max. specific humidity change in g/m³

qcconf.chrco2

Max. CO₂ concentration change in mg/m³

qcconf.chrcr

0.01

Max. acceptable fraction of values removed as spikes

10. MAD Spikes

Parameter

Default

Meaning

qcconf.madth

5.18

Threshold for spike detection in median absolute deviations (MAD). Note: for normal distribution 1.48 MAD correspond to 1 stdev.

qcconf.madcr

0.01

Max. acceptable fraction of values removed as spikes

11. Stationarity after Foken & Wichura

Parameter

Default

Meaning

qcconf.fwsub

6

Number of subrecords

qcconf.fwlimit

0.30

Max. acceptable relative difference between subrecord and full record variances

12. Stationarity from flux by linear trend

Parameter

Default

Meaning

qcconf.cotlimit

0.30

Max. acceptable relative difference between covariances with and without detrending

13. Higher moments compared to beta distribution

Parameter

Default

Meaning

qcconf.bettol

Max. tolerated deviation from borderline beta / gamma distribution

qcconf.betdet

Apply detrending, 0 = no , 1 = yes

14. Stationarity of variance

Parameter

Default

Meaning

qcconf.L1

averaging subrecord width in s

qcconf.betdet

Max. variance change between left and right side of a subrecord

15. Fraction of Turbulence

Parameter

Default

Meaning

qcconf.Lf

75

Subwindow length in seconds

qcconf.ftmin1

0.7

Min. acceptable fraction before raising a soft flag

qcconf.ftmin2

0.5

Min. acceptable fraction before raising a hard flag

16. Surviving values

Parameter

Default

Meaning

qcconf.msurv1

0.99

Minimum acceptable fraction of surviving values before raising a soft flag (1)

qcconf.msurv2

0.90

Minimum acceptable fraction of surviving values before raising a hard flag (2)

Turbulence processing settings

Most parameters in this group are boolean variables (i.e. true ore false).

Values recognized as “true” are : 1, Y, YES, T, TRUE, .TRUE., y, yes, t, true, .true.

Values recognized as “false” are : 0, N, NO, F, FALSE, .FALSE., n, no, f, false, .false.

Parameter

Default

Meaning

Par.DoIterate

true

Iterate the Schotanus correction, the oxygen correction and the frequency correction due to their interdependence [Oncley et al., 2007]. See Iterative Correction Process for details.

Par.MaxIter

10

The maximum number of iterations to be performed.

Par.FREQ

Samplinf frequency of the fast data in Hz

Par.PitchLim

Limit when Mean(W) is turned to zero in degree

Par.RollLim

Limit when Cov(V,W) is turned to zero in degree

Par.PreYaw

Fixed pitch angle for known-tilt correction

Par.PrePitch

Fixed yaw angle for known-tilt correction

Par.PreRoll

Fixed roll angle for known-tilt correction

Par.LLimit

0.85

Smallest acceptable freq-response corr. factor

Par.ULimit

1.15

Largest acceptable freq-response corr. factor

Par.DoCrMean

true

Replace mean quantities by better estimates

Par.DoDetrend

false

Correct data for linear trend

Par.DoSonic

true

Correct sonic temperature for humidity

Par.DoTilt

false

Perform true knwon-tilt correction with fixed angles

Par.DoYaw

true

Turn system such that Mean(V) is minimized.

Par.DoPitch

false

Turn system such that Mean(W) is minimized.

Par.DoRoll

false

Turn System such that Cov(W,V) is minimized.

Par.DoFreq

true

Correct for limited frequency response of the instruments.

Par.DoO2

true

Correct hygrometer for oxygen-sensitivity.

Par.DoWebb

true

Calculate mean velocity according to Webb.

Par.DoStruct

true

Calculate structure coefficients.

Par.StructSep

0.15

separation used for calculation of structure parameters in m.

Par.DoPF

true

Apply untilting by the planar-fit method

Par.PFValid

0.99

Fraction of valid samples required to include interval in planar fit

Par.DoErrFiSi

true

Calculate alternate random error of covariances after [Finkelstein and Sims, 2001] instead of after [Van Dijk et al., 2004]. See Alternative Stochastic Error Calculation for details.

Postprocessor tests

i. Mean vertical wind

Parameter

Default

Meaning

qcconf.wlimit

Max. acceptable magnitude of mean vertical wind

ii. Integrated turb. characteristics (Foken & Wichura)

Parameter

Default

Meaning

qcconf.wlimit

0.3

Max. acceptable relative difference between measured and modeled integr. turb. characteristics (ITC)

iii. Excluded sector wind direction

no configurable parameters (sectors are defined in main configuration file).

iv. Footprint model after Kormann & Meixner

Parameter

Default

Meaning

qcconf.wlimit

0.75

Min. acceptable fraction of footprint in the area of interest

iv. Excessive Error

Parameter

Default

Meaning

qcconf.Herrmin

Max. acceptable absolute error in sensible heat flux in W/m²

qcconf.Herrfac

0.3

Max. acceptable relative error in sensible heat flux

qcconf.Eerrmin

Max. acceptable absolute error in latent heat flux in W/m²

qcconf.Eerrfac

0.3

Max. acceptable relative error in latent heat flux

qcconf.Cerrmin

6.5E-8

Max. acceptable absolute error in CO₂ flux in kg/m²/s

qcconf.Cerrfac

0.3

Max. acceptable relative error in CO₂ flux

qcconf.terrmin

0.05

Max. acceptable absolute error in momentum flux in N/m²

qcconf.terrfac

0.12

Max. acceptable relative error in momentum flux

Flagging Rules

Flag Rules Syntax

Each rule is of the form vvv_ttt[al]:

  • vvv_ttt denotes one individual test result flag of test ttt applied to input variable vvv

  • Available actions are:

    • u (use): take the maximum value of “used” flags

    • i (increase): increase, take the maximum value of “used” flags and increase by one (0→1, 1→2, 2→2)

    • s (soft): set flag to 1 (suspicious), if no other condition implies a higher value

    • h (hard): set flag to 2 (bad), if no other condition implies a higher value

  • If no action is specified, action u is assumed as default

  • l denotes the minimum value of the flag vvv_ttt that triggers action a

Inter Rules Syntax

Each rule is of the form fffaccc[ccc[ccc]]:

  • fff denotes the target flux, of which the flag should be changed

  • a is the action to be taken:

    • i (increase): raise the flag of fff by 1 (0→1, 1→2, 2→2)

    • d (delete): set the flag of fff to 2 (bad)

  • ccc denote(s) the condition fluxes

  • Action a is executed if all condition fluxes are 2 (bad)

Area-of-interest File

The area is given as a polygon. It is assumed that the polygonal path is a loop, or there is an arc from the last to the first position. The positions are to be given in meters, relative to the position of the measurements. The name of this file is given under the configuration parameter “SourceArea”.

Comments: Lines starting with a hash sign (#) or any characters following a hash sign are ignored as comments.

Example:

-999999.  -999999.
 999999.  -999999.
 999999.   999999.
-999999.   999999.

Configuration Examples

Basic Configuration

Here’s a minimal configuration file example:

# Basic EC-PeT Configuration
# Data directories
DatDir = /data/raw/
OutDir = /data/processed/
Parmdir = /data/config/

# Variable names in TOA5 files
U_var = u
V_var = v
W_var = w
Tsonic_var = Ts

# Time period
DateBegin = 2023/06/01-00:00:00
DateEnd = 2023/06/30-23:59:59
AvgInterval = 1800

# Basic sonic configuration
SonCal.QQType = 1
SonCal.QQZ = 3.0
SonCal.QQGain = 1.0
SonCal.QQOffset = 0.0

Advanced Configuration

For a complete setup with gas analyzer:

# Advanced EC-PeT Configuration
# Data and output directories
DatDir = /data/tower1/raw/
OutDir = /data/tower1/processed/
Parmdir = /data/tower1/config/

# File names
RawFastData = tower1_fast.dat
RawSlowData = tower1_slow.dat
FluxName = tower1_fluxes.dat

# Variable mappings
U_var = Ux
V_var = Uy
W_var = Uz
Tsonic_var = Ts
CO2_var = CO2
Humidity_var = H2O
diag_var = CSAT_DIAG
agc_var = IRGA_AGC

# Processing period
DateBegin = 2023/01/01-00:00:00
DateEnd = 2023/12/31-23:59:59
AvgInterval = 1800

# Site information
Displacement = 0.8
ExcludeSector = 170 190
ExcludeSector = 200 220

# Sonic anemometer (CSAT3)
SonCal.QQType = 1
SonCal.QQX = 0.0
SonCal.QQY = 0.0
SonCal.QQZ = 3.0
SonCal.QQYaw = 0.0
SonCal.QQGain = 1.0
SonCal.QQOffset = 0.0

# Gas analyzer (LI-7500)
Co2Cal.QQType = 8
Co2Cal.QQX = 0.15
Co2Cal.QQY = 0.0
Co2Cal.QQZ = 3.0
Co2Cal.QQGain = 1.0
Co2Cal.QQOffset = 0.0

HygCal.QQType = 8
HygCal.QQX = 0.15
HygCal.QQY = 0.0
HygCal.QQZ = 3.0
HygCal.QQGain = 1.0
HygCal.QQOffset = 0.0

# Quality control settings
Despiking = mad
QCdisable =

# Custom QC parameters
qcconf.madth = 4.0
qcconf.ftmin1 = 0.8
qcconf.ftmin2 = 0.6
qcconf.minfoot = 0.8

# Flag rules for momentum flux
FlagRules = tau ux__spks uy__spks uz__spkh

# Inter-dependencies
InterRules = h_0itau e_0itau