Package noaa.coastwatch.tools

Class cwmath

java.lang.Object
noaa.coastwatch.tools.cwmath
public final class cwmath extends Object

The math tool combines earth data using a mathematical expression.

Name

cwmath - combines earth data using a mathematical expression.

Synopsis

cwmath [OPTIONS] input
cwmath [OPTIONS] input1 [input2 ...] output

Options:

-c, --scale=FACTOR/OFFSET | none
-e, --expr=EXPRESSION
-h, --help
-k, --skip-missing
-l, --longname=STRING
-m, --missing=VALUE
-p, --parser=TYPE
-s, --size=TYPE
--serial
--threads=MAX
-t, --template=VARIABLE
-f, --full-template
-u, --units=STRING
-v, --verbose
--version

Description

The math tool combines earth data using a mathematical expression. The expression takes the form:

   variable = formula

where the variable is the output variable to create and the formula is a mathematical combination of input variables. The formula may contain a number of standard operators, for example addition and subtraction, as well as functions such as sine and cosine and numerical and symbolic constants. The formula syntax can follow one of two standards specified by the --parser option: the legacy syntax as originally supported by the math tool, or Java syntax that takes advantage of the full java.lang.Math function library and in some cases allows for simpler expressions. See the Java API documentation for a full list of java.lang.Math functions. A comparison of the most common language features is as follows:

Parser Expressions
Language Feature Legacy Parser Syntax Java Parser Syntax
Unary plus +x Unsupported
Unary minus -x -x
Addition x + y x + y
Subtraction x - y x - y
Multiplication x * y x * y
Division x / y x / y
Modulus x % y or mod (x, y) x % y
Exponentiation x ^ y pow (x, y)
Bitwise AND and (x, y) x & y
Bitwise OR or (x, y) x | y
Bitwise XOR xor (x, y) x ^ y
Bitwise NOT not (x) ~x
Less than x < y x < y
Greater than x > y x > y
Less than or equal to x <= y x <= y
Greater than or equal to x >= y x >= y
Not equal to x != y x != y
Equal to x == y x == y
Boolean conjunction x && y x && y
Boolean disjunction x || y x || y
Boolean negation !x !x
Sine sin (x) sin (x)
Cosine cos (x) cos (x)
Tangent tan (x) tan (x)
Arcsine asin (x) asin (x)
Arccosine acos (x) acos (x)
Arctangent atan (x) atan (x)
Hyperbolic sine sinh (x) sinh (x)
Hyperbolic cosine cosh (x) cosh (x)
Hyperbolic tangent tanh (x) tanh (x)
Inverse hyperbolic sine asinh (x) asinh (x)
Inverse hyperbolic cosine acosh (x) acosh (x)
Inverse hyperbolic tangent atanh (x) atanh (x)
Natural log ln (x) log (x)
Base 10 log log (x) log10 (x)
Polar coordinate angle angle (y, x) atan2 (y, x)
Absolute value abs (x) abs (x)
Random value [0..1] rand() random()
Square root sqrt (x) sqrt (x)
Sum of values sum (x1, x2, ...) sum (x1, x2, ...)
Conditional operator select (condition, x, y) (condition ? x : y)
Hexadecimal constant hex ("0xffff") 0xffff
Data value masking mask (x, flag, bitmask) ((flag & bitmask) == 0 ? x : NaN)
Value of e (2.71828...) e E
Value Pi (3.14159...) pi PI
Not-a-Number as 64-bit nan NaN
Test for Not-a-Number Unsupported isNaN (x)
Haversine distance (km) Unsupported dist (lat1, lon1, lat2, lon2)

Note that in legacy parser expressions, boolean result values from operators (==, !=, >, <, >=, <=, &&, ||, !) evaluated to either 1.0 (true) or 0.0 (false). As a result, legacy parser expressions could treat boolean values as numbers in arithmatic expressions such as addition, subtraction, etc. This is not the case with the Java parser, and even when emulating the legacy parser, arithmatic operations on boolean values generate a parsing error. The only solution for this is to modify the expression. For example:

   result = (var1 != 0) + (var2 != 0)

should be modified to:

   result = (var1 != 0 ? 1 : 0) + (var2 != 0 ? 1 : 0)

and the new Java parser used (see the --parser option below).

Parameters

Main parameters:

input
The single input and output data file. In the case that a single input file is specified and no output file, data is read from and written to the same file. The new variable created by the expression must not already exist in the input file.
input1 [input2...]
The input data file name(s). If multiple input files are specified, variables on the right hand side of the expression (or used in the --template option) must be prefixed by the string 'file<N>_' where <N> is replaced with the index of the input file which contains the variable and input file indexing starts at 1. For example, to reference the variable 'avhrr_ch4' in the second input file, use 'file2_avhrr_ch4' in the expression.
output
The output data file name. If specified and the file does not already exist, it will be created using metadata from the first input file. If it does exist, it will be opened and checked for a compatible earth transform and the new variable data will be added to the file. The new variable created by the expression must not already exist in the output file. The output file can be one of the input files if needed.

Options:

-c, --scale=FACTOR/OFFSET | none
The output variable scale and offset. The scaling is effectively a packing scheme to reduce data file size by scaling floating-point values to integers using the equation: 
     integer = value/factor + offset
The default is '0.01/0'. If 'none' is specified, the expression result is considered to be an integer and stored directly in the output value. This option is ignored for floating-point storage types, namely if --size is 'float' or 'double' or if the template variable type is floating-point.
-e, --expr=EXPRESSION
The mathematical expression. See above for the expression syntax and supported operators and functions. If no expression is specified, the user will be prompted to enter an expression at the keyboard. The latter method is recommended for operating systems such as Microsoft Windows in which the command line shell can mangle some expression characters such as the equals sign.
-h, --help
Prints a brief help message.
-k, --skip-missing
Turns on skip missing mode. When on, the computation skips any locations where one or more variable values are set to the missing value. This is an optimization for when the expression does not contain any tests for NaN values, and data values flagged as missing would lead to in an invalid computation result.
-l, --longname=STRING
The output variable long name. The long name is a verbose string to describe the variable in common terms. For example, the variable named 'sst' might have the long name 'sea surface temperature'. The default is to use the output variable name as the long name.
-m, --missing=VALUE
The missing output value. The missing value is used to mark output locations in the variable where there is no valid value, either because the computation of the expression failed, or no value was ever or should ever be written to the location (possibly, it falls outside some geographic domain). By default, the missing value is type-dependent and is set to the minimum representable value for integer types. For floating point types, this option is ignored and the missing value is set to the NaN value or inherited from the template variable.
-p, --parser=TYPE
The expression parser type. Valid choices are 'java' or 'emulated':
  • Java (DEFAULT) - The Java expression parser accepts expressions written in the Java Language Specification, with access to the full java.lang.Math class static methods, in addition to the constants E, PI, NaN, and the functions isNaN(x), asinh(x), acosh(x), atanh(x), and sum(x1,x2,...). The Java parser compiles expressions to Java byte code and runs the code natively for high speed expression evaluation. The Java parser is more strict with type checking than the legacy or emulated legacy parsers. For example, operands of the bitwise operators must be an integer type (or cast to an integer type) in order to pass the parsing phase without a type error.
  • Legacy emulated - The emulated expression parser emulates the legacy parser (the original parser used in the math tool) by internally converting the legacy expression syntax, operators, functions, and constants to Java Language Specification. The converted expression is then parsed and evaluated by the high speed Java expression parser.
-s, --size=TYPE
The output variable type. Valid choices include integer data in both signed and unsigned types, and floating-point data as follows:
  • 8-bit byte: 'byte' or 'ubyte'
  • 16-bit short integer: 'short' or 'ushort'
  • 32-bit integer: 'int' or 'uint'
  • 64-bit long integer: 'long' or 'ulong'
  • 32-bit floating-point: 'float'
  • 64-bit floating-point: 'double'
Integer output types can be used to represent floating-point values by specifying the --scale option. The default variable type is 'short', and with the default scaling factor of 0.01, floating-point values are packed into 16-bit signed integers with a range of [-327.68 ... 327.67] and two decimals of accuracy.
--serial
Turns on serial processing mode. By default the program will use multiple processors in parallel to process chunks of data.
--threads=MAX
Specifies the maximum number of threads for parallel processing. By default the program will automatically detect the maximum number of threads possible.
-t, --template=VARIABLE
The output template variable. When a template is used, the output variable size, scaling, units, long name, and missing value are all determined from the template variable. Any of these properties set from the command line using --size, --scale, --units, --longname or --missing override the corresponding template property. There is no template variable by default, rather the math routine outputs 16-bit scaled integer values. See the --size and --scale options for details.
-f, --full-template
Turns on full template attribute mode. All attributes from the template variable (except for those overridden at the command line) are copied to the output variable. By default only the minimal set of attributes is written.
-u, --units=STRING
The output variable units. For example if the output variable data is based on temperature in Celsius, the variable units might be 'celsius'. There is no default units value.
-v, --verbose
Turns verbose mode on. The current status of computation is printed periodically. The default is to run quietly.
--version
Prints the software version.

Exit status

0 on success, > 0 on failure. Possible causes of errors:

  • Invalid command line option
  • Invalid input or output file names
  • Unsupported input file format
  • Invalid mathematical expression
  • Output variable already exists in input file
  • Invalid scale or size specified
  • Unsupported variable rank detected
  • Invalid expression variable name

Examples

The following shows the correction of AVHRR channel 2 data for solar zenith angle. The output variable is named 'avhrr_ch2_corr' and is written to the input file:

   phollema$ cwmath -v --units percent --longname "AVHRR channel 2 corrected"
     --expr "avhrr_ch2_corr = avhrr_ch2 / cos (toRadians (sun_zenith))"
     2019_250_2241_n19_mr.hdf

   [INFO] Opening input/output 2019_250_2241_n19_mr.hdf
   [INFO] Creating avhrr_ch2_corr variable
   [INFO] Total grid size is 1101x1401
   [INFO] Found 8 processor(s) to use
   [INFO] Processing 9 data chunks of size 512x512

Another example below shows the computation of Normalized Difference Vegetation Index (NDVI):

   phollema$ cwmath -v --longname "Normalized Difference Vegetation Index"
     --expr "ndvi = (avhrr_ch2 - avhrr_ch1)/(avhrr_ch2 + avhrr_ch1)" 
     2019_015_2121_n19_er.hdf

   [INFO] Opening input/output 2019_015_2121_n19_er.hdf
   [INFO] Creating ndvi variable
   [INFO] Total grid size is 1401x1302
   [INFO] Found 8 processor(s) to use
   [INFO] Processing 9 data chunks of size 512x512

To show how to mark certain data values in a variable as invalid, the example below masks the 'sst' variable using data from the 'cloud' variable:

   phollema$ cwmath -v --template sst
     --expr 'sst_masked = ((cloud & 0x6f) == 0 ? sst : NaN)'
     2019_015_2121_n19_er.hdf

   [INFO] Opening input/output 2019_015_2121_n19_er.hdf
   [INFO] Creating sst_masked variable
   [INFO] Total grid size is 1401x1302
   [INFO] Found 8 processor(s) to use
   [INFO] Processing 9 data chunks of size 512x512

We use a hexadecimal value to set which bits from the 8-bit cloud mask to use for masking. In this case the value '0x6f' tests the cloud mask bits 1, 2, 3, 4, 6, and 7 -- to use all cloud mask bits, the value would be '0xff'. The output value 'NaN' is a special number that marks the SST as invalid where the cloud mask has detected cloud (ie: at least one of the cloud mask bits specified in the mask is set to 1). The conditional operator is also used: (condition ? x : y) means 'if condition is true, the value of x, otherwise the value of y'.

A final example below shows how the tool may be used to compute complex formulas using a Unix Bourne shell script. The example computes the theoretical AVHRR channel 3b albedo at night for NOAA-17 using actual channel 3b temperatures and channel 3b emission temperatures estimated from channel 4 and 5:

   #!/bin/sh
   
   input=$1
   T3E_A=6.82947
   T3E_B=0.97232
   T3E_C=1.66366
   ZERO_C=273.15
   t3="(avhrr_ch3 + $ZERO_C)"
   t4="(avhrr_ch4 + $ZERO_C)"
   t5="(avhrr_ch5 + $ZERO_C)"
   t3e="($T3E_A + $T3E_B*$t4 + $T3E_C*($t4 - $t5))"
   planck_c1=1.1910427e-5
   planck_c2=1.4387752
   w3=2669.3554
   c3b_a=1.702380
   c3b_b=0.997378
   rad3="(($planck_c1*pow ($w3, 3)) / (pow (E, ($planck_c2*$w3)/($c3b_a + $c3b_b*$t3)) - 1.0))"
   rad3e="(($planck_c1*pow ($w3, 3)) / (pow (E, ($planck_c2*$w3)/($c3b_a + $c3b_b*$t3e)) - 1.0))"
   alb3="(100*(1 - $rad3/$rad3e))"
   cwmath -v --longname "AVHRR channel 3 albedo" --units "percent" \
     --expr "avhrr_ch3_albedo=$alb3" $input
Since:
3.1.4
Author:
Peter Hollemans

  • Method Details

    • main

      public static void main(String[] argv)
      Performs the main function.
      Parameters:
      argv - the list of command line parameters.