Package noaa.coastwatch.tools

Class cwcomposite

java.lang.Object
noaa.coastwatch.tools.cwcomposite
public final class cwcomposite extends Object

The composite tool combines a time series of earth data.

Name

cwcomposite - combines a time series of earth data.

Synopsis

cwcomposite [OPTIONS] input [input2 ...] output
cwcomposite [OPTIONS] --inputs=FILE output

Options:

-c, --coherent=VARIABLE1[/VARIABLE2[...]]
-h, --help
-k, --keephistory
-m, --match=PATTERN
-M, --method=TYPE
-o, --optimal=VARIABLE/TYPE
-p, --pedantic
-S, --savemap
--serial
--threads=MAX
-t, --collapsetime
-v, --verbose
-V, --valid=COUNT
--version

Description

The composite tool combines a time series of earth data. Data variables are combined on a pixel-by-pixel basis using one of several statistical or temporal methods: mean, geometric mean, median, minimum, maximum, explicit or latest. The input files must have matching earth transforms but may have different dates. The composite tool may be used, for example, to combine a number of sea-surface-temperature datasets into one in order to obtain a mean SST for a certain region and help eliminate cloud. Another use is to combine datasets from different regions that are registered to the same earth transform to create a mosaic. The output dataset is constructed using metadata from each input dataset so that it properly reflects the different input dataset dates and other metadata.

Parameters

Main parameters:

input [input2 ...]
The input data file names. At least one input file is required, unless the --inputs option is used. If multiple files are specified, they must have matching earth transforms.
--inputs=FILE
The file name containing a list of input data files. The file must be an ASCII text file containing input file names, one per line. If multiple files are listed, they must have matching earth transforms. If the inputs file name is '-', input is read from standard input.
output
The output data file name.

Options:

-c, --coherent=VARIABLE1[/VARIABLE2[...]]
Turns on coherent mode (can only be used with --method latest, --method explicit, or --method optimal). In coherent mode, the output values for all variables at a given pixel location are guaranteed to originate from the same input file. The specified variable list is used to prioritize variables to check for a valid latest/last value. If there are no valid values for the first variable at a given location, then the next variable is checked and so on until the latest/last valid value is found. This mode is useful for when data variables and their respective quality flags should be kept together during a composite operation. Without this option, the 'latest' and 'explicit' composite methods may select the latest/last valid data value from one input file, and the latest/last valid quality flag from another input file for a given location.
-h, --help
Prints a brief help message.
-k, --keephistory
Turns keep history mode on. In keep history mode, the processing commands used to create each input file are combined together, along with the composite command. The combined history of all processing commands from all input files can be very large and in many cases overflows the maximum length for history metadata in the output file, so by default only the composite command is written to the output file history.
-m, --match=PATTERN
The variable name matching pattern. If specified, the pattern is used as a regular expression to match variable names. Only variables matching the pattern will be present in the output. By default, no pattern matching is performed and all variables are combined.
-M, --method=TYPE
The composite method. Valid methods are:
  • mean - Computes the arithmetic mean or average value (sum of values over n)
  • geomean - Computes the geometric mean (nth root of product of values)
  • median - Finds the median value (middle value of n values)
  • min - Finds the minimum value
  • max - Finds the maximum value
  • latest - Finds the most recent valid value (latest in time according to the data time stamp)
  • explicit - Finds the last valid value in the set of input files, according to the explicit order given on the command line. This would yield the same results as the 'latest' method if the files were listed in chronological order on the command line.
  • optimal - Finds the best pixel from the set of input files based on the minimum or maximum of an optimization variable. See the --optimal option for more details. This method turns on coherent mode as specified by the --coherent option.
The default is to compute the mean value.
-o, --optimal=VARIABLE/TYPE
The optimization variable and type to use for optimal compositing (see --method optimal above). The type may be either 'min' or 'max'. Optimal compositing is done by searching for the input file whose data value is either minimum or maximum. The output values for all variables at a given pixel location are then guaranteed to originate from this same input file. By default, the input files are searched for a satellite zenith angle variable and optimized by selecting the minimum satellite zenith angle.
-p, --pedantic
Turns pedantic mode on. In pedantic mode, metadata from each input file is combined exactly such that composite attributes in the output file may contain repeated values. When pedantic mode is off, composite attributes are collapsed so that only unique values appear. By default, pedantic mode is off.
-S, --savemap
Saves the source index mapping to the output file in coherent mode. The variable 'source_index' is written as a 16-bit signed integer, and for each destination pixel in the output, represents the 0-based index of the input file used to copy variable values into the output file. An attribute is written to the mapping variable specifying, in index order, the names of the input files used. See the --coherent option above for details on coherent mode.
--serial
Turns on serial processing mode. By default the program will use multiple processors in parallel to process chunks of data.
--threads=MAX
Specifies the maximum number of threads for parallel processing. By default the program will automatically detect the maximum number of threads possible.
-t, --collapsetime
Specifies that the time metadata in the output file should be simplified and collapsed into a single period with start date/time, and end date/time. By default the output file contains the full time period metadata with the time periods from all of the input files preserved.
-v, --verbose
Turns verbose mode on. The current status of data combination is printed periodically. The default is to run quietly.
-V, --valid=COUNT
The minimum number of valid values required to form an aggregate function. By default, only one value per pixel is required. If the actual number of valid values is below this threshold, the output value is set to invalid.
--version
Prints the software version.

Exit status

0 on success, > 0 on failure. Possible causes of errors:

  • Invalid command line option
  • Invalid input or output file names
  • Unsupported input file format
  • Input file earth transforms do not match
  • No matching variables found
  • Unsupported composite method

Examples

The following shows the combination of 24 hours of Himawari-8 NetCDF SST data files into one median composite file, using a maximum heap size of 8 Gb and at least 5 valid pixels per median calculation:

   phollema$ cwcomposite -J-Xmx8g -v --method median --valid=5
     --match sea_surface_temperature 2019*.nc composite.hdf
   [INFO] Checking input file information
   [INFO] Checking input file [1/24] 20190410000000-AHI_H08.nc
   [INFO] Adding sea_surface_temperature to composite output variables
   [INFO] Checking input file [2/24] 20190410010000-AHI_H08.nc
   [INFO] Checking input file [3/24] 20190410020000-AHI_H08.nc
   [INFO] Checking input file [4/24] 20190410030000-AHI_H08.nc
   [INFO] Checking input file [5/24] 20190410040000-AHI_H08.nc
   [INFO] Checking input file [6/24] 20190410050000-AHI_H08.nc
   [INFO] Checking input file [7/24] 20190410060000-AHI_H08.nc
   [INFO] Checking input file [8/24] 20190410070000-AHI_H08.nc
   [INFO] Checking input file [9/24] 20190410080000-AHI_H08.nc
   [INFO] Checking input file [10/24] 20190410090000-AHI_H08.nc
   [INFO] Checking input file [11/24] 20190410100000-AHI_H08.nc
   [INFO] Checking input file [12/24] 20190410110000-AHI_H08.nc
   [INFO] Checking input file [13/24] 20190410120000-AHI_H08.nc
   [INFO] Checking input file [14/24] 20190410130000-AHI_H08.nc
   [INFO] Checking input file [15/24] 20190410140000-AHI_H08.nc
   [INFO] Checking input file [16/24] 20190410150000-AHI_H08.nc
   [INFO] Checking input file [17/24] 20190410160000-AHI_H08.nc
   [INFO] Checking input file [18/24] 20190410170000-AHI_H08.nc
   [INFO] Checking input file [19/24] 20190410180000-AHI_H08.nc
   [INFO] Checking input file [20/24] 20190410190000-AHI_H08.nc
   [INFO] Checking input file [21/24] 20190410200000-AHI_H08.nc
   [INFO] Checking input file [22/24] 20190410210000-AHI_H08.nc
   [INFO] Checking input file [23/24] 20190410220000-AHI_H08.nc
   [INFO] Checking input file [24/24] 20190410230000-AHI_H08.nc
   [INFO] Creating output file composite.hdf
   [INFO] Total grid size is 5500x5500
   [INFO] Found 8 processor(s) to use
   [INFO] Creating sea_surface_temperature variable with chunk size 1834x1834
Since:
3.1.4
Author:
Peter Hollemans

  • Method Details

    • main

      public static void main(String[] argv)
      Performs the main function.
      Parameters:
      argv - the list of command line parameters.