ecpet.ecfile module

EC-PeT File Operations Module

File handling utilities for Campbell Scientific TOA5 data logger files and eddy-covariance data processing workflows. Provides efficient reading, writing, and validation of TOA5 format files commonly used in meteorological and flux tower measurements.

The module handles:

  • File discovery with pattern matching and glob support

  • TOA5 format validation and header parsing

  • Efficient timestamp extraction from large files

  • File integrity verification with MD5 checksums

  • Campbell Scientific formatting conventions

  • Time range analysis across multiple files

ecpet.ecfile.find(path, names)

Find files matching patterns in specified directory.

Parameters:

  • path (str) – Directory path to search in

  • names (str, list, or None) – File patterns or names to search for

Returns:

List of filenames in TOA5 format

Return type:

list

Supports direct filenames, glob patterns, and directory listings. Returns only files that pass TOA5 format validation.

ecpet.ecfile.toa5_check(filename)

Check if file is in Campbell Scientific TOA5 format.

Parameters:

filename (str) – Path to file to check

Returns:

True if file is TOA5 format

Return type:

bool

ecpet.ecfile.toa5_get_header(filename)

Extract header information from TOA5 file.

Parameters:

filename (str) – Path to TOA5 file

Returns:

Dictionary containing header information

Return type:

dict

Returns station name, logger info, column names, units, and sampling info. Column names have parentheses replaced with underscores.

ecpet.ecfile.toa5_get_hash(filename)

Calculate MD5 hash of TOA5 file.

Parameters:

filename (str) – Path to TOA5 file

Returns:

MD5 hash as hexadecimal string

Return type:

str

ecpet.ecfile.toa5_get_times(filename, count=False)

Extract first and last timestamps from TOA5 file.

Parameters:

  • filename (str) – Path to TOA5 file

  • count (bool, optional) – Whether to count total records, defaults to False

Returns:

List with [first_time, last_time] or [first_time, last_time, count]

Return type:

list

Uses efficient file seeking to find last timestamp without reading entire file.

ecpet.ecfile.get_time_range(d, fl)

Determine overall time range from list of TOA5 files.

Parameters:

  • d (str) – Directory containing files

  • fl (list) – List of filenames

Returns:

List with [earliest_time, latest_time] as pandas Timestamps

Return type:

list

ecpet.ecfile.toa5_put_header(file, header)

Write TOA5 header information to file.

Parameters:

  • file (str) – Path to output file

  • header (dict) – Dictionary containing header information

Creates 4-line TOA5 header with station info, column names, units, and sampling.

ecpet.ecfile.cs_style(v)

Format values in Campbell Scientific TOA5 style.

Parameters:

v (int, float, str, bytes, or None) – Value to format

Returns:

Formatted string representation

Return type:

str

Applies precision rules based on magnitude and strips trailing zeros.