Package noaa.coastwatch.io

Class CachedGrid

java.lang.Object
noaa.coastwatch.util.MetadataContainer
noaa.coastwatch.util.DataVariable
noaa.coastwatch.util.Grid
noaa.coastwatch.io.CachedGrid
All Implemented Interfaces:
Cloneable, ValueSource
Direct Known Subclasses:
HDFCachedGrid, NCCachedGrid, NOAA1bCachedGrid
public abstract class CachedGrid extends Grid

A CachedGrid is a Grid that uses temporary caching to reduce the overall memory footprint of gridded data. The cache uses a similar strategy to how modern operating systems cache memory pages. A data stream is kept open, and a number of tiles (rectangular sections of data) are brought into memory as they are needed. The number of tiles in memory (the cache size) and the size of each tile may be set by the user. Tiles that are no longer needed are swapped out to make room for new tiles in the cache. In order to avoid excessive data I/O, a least-recently-used rule is used to determine which tile to remove from the cache when the cache reaches its maximum capacity.

The standard DataVariable.setValue(noaa.coastwatch.util.DataLocation, double) method is supported by keeping a dirty flag for each tile. If the tile has been written to, it is kept in the cache until, upon removal, it is written to the data stream.

The only methods that subclasses need to implement are readTile(noaa.coastwatch.io.tile.TilingScheme.TilePosition) to retrieve tiles from the data source, writeTile(noaa.coastwatch.io.tile.TilingScheme.Tile) to update the data with any changes made to tiles, and getDataStream() to retrieve the object used to read and write data (only used to check for equality).

Since:
3.1.0
Author:
Peter Hollemans

  • Field Details

    • DEFAULT_MAX_TILES

      public static final int DEFAULT_MAX_TILES
      Default cache tiles.
      See Also:

    • DEFAULT_TILE_DIMS

      public static final int DEFAULT_TILE_DIMS
      Default tile dimensions.
      See Also:

    • READ_ONLY

      protected static final int READ_ONLY
      Read-only mode.
      See Also:

    • READ_WRITE

      protected static final int READ_WRITE
      Read-write mode.
      See Also:

    • tiling

      protected TilingScheme tiling
      The tiling scheme.

    • accessMode

      protected int accessMode
      The access mode.

  • Constructor Details

    • CachedGrid

      protected CachedGrid(Grid grid, int accessMode)
      Constructs a new cached grid with the specified properties. The cache is created with a default size and maximum number of tiles.
      Parameters:
      grid - the grid to use for attributes.
      accessMode - the access mode, either READ_ONLY or READ_WRITE.

  • Method Details

    • getTilingScheme

      public TilingScheme getTilingScheme()
      Description copied from class: Grid
      Gets the tiling scheme for this grid if one is available.
      Overrides:
      getTilingScheme in class Grid
      Returns:
      the tiling scheme or null for none.

    • timeAccess

      public long timeAccess(int row, int col)
      Times an access to the cache at the specified coordinates.
      Parameters:
      row - the data row to access.
      col - the data column to access.
      Returns:
      the access time in nanoseconds (ns).

    • setDynamic

      public void setDynamic(boolean flag)
      Turns on or off dynamic optimization mode. In dynamic optimization, the cache is monitored for access events and cache miss events and the number of tiles in the cache is changed dynamically to maintain an acceptable range of the cache miss rate.
      Parameters:
      flag - the mode flag, true to perform dynamic optimization or false to not.

    • setOptimizedCacheSize

      public void setOptimizedCacheSize(int cacheSize)
      Creates a new cache with the specified size. The cache size is optimized so that the cache can contain a set of tiles which span at least the entire width of the grid for more efficient caching. Thus, the actual cache size may be slightly larger than that requested. As an alternative to this method, consider using dynamic caching instead by calling setDynamic(boolean).
      Parameters:
      cacheSize - the cache memory size in bytes.
      See Also:

    • getDataStream

      public abstract Object getDataStream()
      Gets the cache data stream as an object. This is useful for classes that read and write cached grids.
      Returns:
      the data stream used for reading and writing data.

    • clearCache

      public void clearCache()
      Clears the existing cache. All tiles are removed and the cache set to empty. Use this method to release the memory used by the cache when it won't be needed again. The same notes in resetCache() on using flush() apply here as well.
      Since:
      3.5.0

    • resetCache

      protected void resetCache()
      Resets the tile cache to be empty. This method should be used with caution, since it does not perform a flush() prior to resetting the cache. Any tiles whose contents have been modified but not written will lose those modifications. Therefore, users of read/write mode tiles should explicitly call flush() before this method.

    • setCacheSize

      public void setCacheSize(int cacheSize)
      Creates a new cache with the specified size.
      Parameters:
      cacheSize - the cache memory size in bytes.
      See Also:

    • getMaxTiles

      public int getMaxTiles()
      Gets the current maximum number of tiles allowed in the cache.
      Returns:
      the maximum tiles allowed.

    • setMaxTiles

      public void setMaxTiles(int tiles)
      Creates a new cache with a maximum number of tiles.
      Parameters:
      tiles - the maximum number of tiles.
      See Also:

    • setTileSize

      public void setTileSize(int tileSize)
      Creates a new cache where each tile has the specified size.
      Parameters:
      tileSize - the tile memory size in bytes.

    • setTileDims

      public void setTileDims(int[] tileDims)
      Creates a new cache where each tile has the specified dimensions.
      Parameters:
      tileDims - the tile dimensions as [rows, columns].

    • getTileSize

      public static int getTileSize(int[] tileDims, Grid grid)
      Calculates the tile memory size based on dimensions.
      Parameters:
      tileDims - the tile dimensions as [rows, columns].
      grid - the grid for calculation.
      Returns:
      the memory size used by each tile in bytes.

    • getTileDims

      public static int[] getTileDims(int tileSize, Grid grid)
      Calculates the tile dimensions required to fill the specified memory.
      Parameters:
      tileSize - the tile memory size in bytes.
      grid - the grid for calculation.
      Returns:
      the dimensions of the largest tile that fills the memory without being larger than the grid dimensions.

    • readTile

      protected abstract TilingScheme.Tile readTile(TilingScheme.TilePosition pos) throws IOException
      Reads the specified tile.
      Parameters:
      pos - the tile position to read.
      Returns:
      the tile at the specified position.
      Throws:
      IOException - if an error occurred reading the tile data.

    • writeTile

      protected abstract void writeTile(TilingScheme.Tile tile) throws IOException
      Writes the specified tile. The implementation of this method must set the tile to be not dirty upon successfully writing it, or throw an error.
      Parameters:
      tile - the tile to write.
      Throws:
      IOException - if an error occurred writing the tile data.

    • flush

      public void flush() throws IOException
      Writes any unwritten tiles to the data stream.
      Throws:
      IOException - if an error occurred writing the tile data.

    • setValue

      public void setValue(int index, double val)
      Description copied from class: DataVariable
      Writes a scaled data value. The data value is scaled according to the scaling factor and offset and written to the data array.
      Overrides:
      setValue in class DataVariable
      Parameters:
      index - the index into the data array.
      val - the data value as a double. If the data value is Double.NaN and the missing value is non-null, the missing value is written to the array.
      See Also:

    • setValue

      public void setValue(int row, int col, double val)
      Description copied from class: Grid
      Writes a scaled data value with no navigation. The data value is scaled according to the scaling factor and offset and written to the data array. No navigation transform is applied to correct the data location.
      Overrides:
      setValue in class Grid
      Parameters:
      row - the data location row.
      col - the data location column.
      val - the data value as a double. If the data value is Double.NaN and the missing value is non-null, the missing value is written to the array.
      See Also:

    • getValue

      public double getValue(int index)
      Description copied from class: DataVariable
      Reads a scaled data value. The data value is read from the data array and scaled according to the scaling factor and offset.
      Overrides:
      getValue in class DataVariable
      Parameters:
      index - the index into the data array.
      Returns:
      the scaled data value as a double. The Double.NaN value is used if the data value is missing.
      See Also:

    • getValue

      public double getValue(int row, int col)
      Description copied from class: Grid
      Reads a scaled data value with no navigation. The data value is read from the data array and scaled according to the scaling factor and offset. No navigation transform is applied to correct the data location.
      Overrides:
      getValue in class Grid
      Parameters:
      row - the data location row.
      col - the data location column.
      Returns:
      the scaled data value as a double. The Double.NaN value is used if the data value is missing or data coordinate is not valid.
      See Also:

    • setData

      public void setData(Object data)
      Description copied from class: DataVariable
      Sets the variable data array.
      Overrides:
      setData in class DataVariable

    • getData

      public Object getData()
      Description copied from class: DataVariable
      Gets the variable data array. The returned data is read-only -- the result of attempting to set values in the data is undefined.
      Overrides:
      getData in class DataVariable

    • setAccessType

      public void setAccessType(DataVariable.AccessType type)
      Description copied from class: DataVariable
      Sets a hint to the variable that subsequent data access will be of the specified access type. This is an opportunity for the underlying data structures to be rearranged to better accomodate the specified access type.
      Overrides:
      setAccessType in class DataVariable
      Parameters:
      type - the access type hint.

    • setData

      public void setData(Object subset, int[] start, int[] count)
      Description copied from class: Grid
      Sets a subset of grid data values. This method is similar to DataVariable.setData(Object), but sets only a subset of data values in the raw, unscaled form.
      Overrides:
      setData in class Grid
      Parameters:
      subset - the subset array of unscaled data values.
      start - the subset starting [row, column].
      count - the subset dimensions [rows, columns].

    • getData

      public Object getData(int[] start, int[] count)
      Description copied from class: Grid
      Gets a subset of grid data values. This method is similar to Grid.getData(int[], int[]), but retrieves only a subset of data values in the raw, unscaled form.
      Overrides:
      getData in class Grid
      Parameters:
      start - the subset starting [row, column].
      count - the subset dimension [rows, columns].
      Returns:
      an array containing the unscaled data values.
      See Also:

    • main

      public static void main(String[] argv) throws Exception
      Tests this class.
      Parameters:
      argv - the array of command line parameters.
      Throws:
      Exception