Isis 3 Programmer Reference
Isis::CubeBsqHandler Class Reference

IO Handler for Isis Cubes using the BSQ format. More...

#include <CubeBsqHandler.h>

Inheritance diagram for Isis::CubeBsqHandler:
Inheritance graph
Collaboration diagram for Isis::CubeBsqHandler:
Collaboration graph

Public Member Functions

 CubeBsqHandler (QFile *dataFile, const QList< int > *virtualBandList, const Pvl &label, bool alreadyOnDisk)
 Construct a BSQ IO handler.
 
 ~CubeBsqHandler ()
 The destructor writes all cached data to disk.
 
void updateLabels (Pvl &labels)
 Function to update the labels with a Pvl object.
 
void read (Buffer &bufferToFill) const
 Read cube data from disk into the buffer.
 
void write (const Buffer &bufferToWrite)
 Write buffer data into the cube data on disk.
 
void addCachingAlgorithm (CubeCachingAlgorithm *algorithm)
 This will add the given caching algorithm to the list of attempted caching algorithms.
 
void clearCache (bool blockForWriteCache=true) const
 Free all cube chunks (cached cube data) from memory and write them to disk.
 
BigInt getDataSize () const
 
void setVirtualBands (const QList< int > *virtualBandList)
 This changes the virtual band list.
 
QMutex * dataFileMutex ()
 Get the mutex that this IO handler is using around I/Os on the given data file.
 

Protected Member Functions

virtual void readRaw (RawCubeChunk &chunkToFill)
 This needs to populate the chunkToFill with unswapped raw bytes from the disk.
 
virtual void writeRaw (const RawCubeChunk &chunkToWrite)
 This needs to write the chunkToWrite directly to disk with no modifications to the data itself.
 
int bandCount () const
 
int getBandCountInChunk () const
 
BigInt getBytesPerChunk () const
 
int getChunkCountInBandDimension () const
 
int getChunkCountInLineDimension () const
 
int getChunkCountInSampleDimension () const
 
int getChunkIndex (const RawCubeChunk &) const
 Given a chunk, what's its index in the file.
 
BigInt getDataStartByte () const
 
QFile * getDataFile ()
 
int lineCount () const
 
int getLineCountInChunk () const
 
PixelType pixelType () const
 
int sampleCount () const
 
int getSampleCountInChunk () const
 
void setChunkSizes (int numSamples, int numLines, int numBands)
 This should be called once from the child constructor.
 

Private Member Functions

 CubeBsqHandler (const CubeBsqHandler &other)
 Disallow copying of this object.
 
CubeBsqHandleroperator= (const CubeBsqHandler &other)
 Disallow assignments of this object.
 
int findGoodSize (int maxSize, int dimensionSize) const
 This method attempts to compute a good chunk line size.
 
BigInt getChunkStartByte (const RawCubeChunk &chunk) const
 This is a helper method that goes from chunk to file position.
 
void blockUntilThreadPoolEmpty () const
 This blocks (doesn't return) until the number of active runnables in the thread pool goes to 0.
 
QPair< QList< RawCubeChunk * >, QList< int > > findCubeChunks (int startSample, int numSamples, int startLine, int numLines, int startBand, int numBands) const
 Get the cube chunks that correspond to the given cube area.
 
void findIntersection (const RawCubeChunk &cube1, const Buffer &cube2, int &startX, int &startY, int &startZ, int &endX, int &endY, int &endZ) const
 Find the intersection between the buffer area and the cube chunk.
 
void flushWriteCache (bool force=false) const
 This attempts to write the so-far-unwritten buffers from m_writeCache into the cube's RawCubeChunk cache.
 
void freeChunk (RawCubeChunk *chunkToFree) const
 If the chunk is dirty, then we write it to disk.
 
RawCubeChunkgetChunk (int chunkIndex, bool allocateIfNecessary) const
 Retrieve the cached chunk at the given chunk index, if there is one.
 
int getChunkCount () const
 
void getChunkPlacement (int chunkIndex, int &startSample, int &startLine, int &startBand, int &endSample, int &endLine, int &endBand) const
 Get the X/Y/Z (Sample,Line,Band) range of the chunk at the given index.
 
RawCubeChunkgetNullChunk (int chunkIndex) const
 This creates a chunk filled with NULLs whose placement is at chunkIndex's position.
 
void minimizeCache (const QList< RawCubeChunk * > &justUsed, const Buffer &justRequested) const
 Apply the caching algorithms and get rid of excess cube data in memory.
 
void synchronousWrite (const Buffer &bufferToWrite)
 This method takes the given buffer and synchronously puts it into the Cube's cache.
 
void writeIntoDouble (const RawCubeChunk &chunk, Buffer &output, int startIndex) const
 Write the intersecting area of the chunk into the buffer.
 
void writeIntoRaw (const Buffer &buffer, RawCubeChunk &output, int index) const
 Write the intersecting area of the buffer into the chunk.
 
void writeNullDataToDisk () const
 Write all NULL cube chunks that have not yet been accessed to disk.
 

Static Private Member Functions

static bool bufferLessThan (Buffer *const &lhs, Buffer *const &rhs)
 This is used for sorting buffers into the most efficient write order.
 

Private Attributes

QFile * m_dataFile
 The file containing cube data.
 
BigInt m_startByte
 The start byte of the cube data.
 
PixelType m_pixelType
 The format of each DN in the cube.
 
double m_base
 The additive offset of the data on disk.
 
double m_multiplier
 The multiplicative factor of the data on disk.
 
ByteOrder m_byteOrder
 The byte order (endianness) of the data on disk.
 
EndianSwapperm_byteSwapper
 A helper that swaps byte order to and from file order.
 
int m_numSamples
 The number of samples in the cube.
 
int m_numLines
 The number of lines in the cube.
 
int m_numBands
 The number of physical bands in the cube.
 
QList< CubeCachingAlgorithm * > * m_cachingAlgorithms
 The caching algorithms to use, in order of priority.
 
QMap< int, RawCubeChunk * > * m_rawData
 The map from chunk index to chunk for cached data.
 
QMap< int, bool > * m_dataIsOnDiskMap
 The map from chunk index to on-disk status, all true if not allocated.
 
QList< int > * m_virtualBands
 Converts from virtual band to physical band.
 
int m_samplesInChunk
 The number of samples in a cube chunk.
 
int m_linesInChunk
 The number of lines in a cube chunk.
 
int m_bandsInChunk
 The number of physical bands in a cube chunk.
 
QList< RawCubeChunk * > * m_lastProcessByLineChunks
 This is an optimization for process by line.
 
QByteArray * m_nullChunkData
 A raw cube chunk's data when it was all NULL's. Used for speed.
 
QPair< QMutex *, QList< Buffer * > > * m_writeCache
 These are the buffers we need to write to raw cube chunks.
 
QThreadPool * m_ioThreadPool
 This contains threads for doing cube writes (and later maybe cube reads).
 
bool m_lastOperationWasWrite
 If the last operation was a write then we need to flush the cache when reading.
 
bool m_useOptimizedCubeWrite
 This is true if the Isis preference for the cube write thread is optimized.
 
QMutex * m_writeThreadMutex
 This enables us to block while the write thread is working.
 
volatile int m_idealFlushSize
 Ideal write cache flush size.
 
int m_consecutiveOverflowCount
 How many times the write cache has overflown in a row.
 

Detailed Description

IO Handler for Isis Cubes using the BSQ format.

This class is used to open, create, read, and write data from Isis cube files.

Author
2003-02-14 Jeff Anderson
History

2007-09-14 Stuart Sides - Fixed bug where pixels from a buffer outside the ns/nl were being transfered to the right most and bottom most tiles

2007-10-11 Stuart Sides - Fixed bug introduced with previous bug fix.

2008-09-03 Steven Lambright - Fixed MSB/LSB problem with the Move() method

2011-06-16 Jai Rideout and Steven Lambright - Refactored to work with the new parent.

2011-07-18 Jai Rideout and Steven Lambright - Added unimplemented copy constructor and assignment operator.

2014-09-16 Ian Humphrey - Increased size limit for determining chunk sizes. Added findGoodSize method to better calculate number of lines in chunks for bsq cubes. References #1689.

2017-09-22 Cole Neubauer - Fixed documentation. References #4807

Definition at line 41 of file CubeBsqHandler.h.

Constructor & Destructor Documentation

◆ CubeBsqHandler() [1/2]

Isis::CubeBsqHandler::CubeBsqHandler ( QFile * dataFile,
const QList< int > * virtualBandList,
const Pvl & labels,
bool alreadyOnDisk )

Construct a BSQ IO handler.

This will determine a good chunk size to use that does not result in the cube being enlarged or misordered.

Parameters
dataFileThe file with cube DN data in it
virtualBandListThe mapping from virtual band to physical band, see CubeIoHandler's description.
labelsThe Pvl labels for the cube
alreadyOnDiskTrue if the cube is allocated on the disk, false otherwise

Definition at line 34 of file CubeBsqHandler.cpp.

References findGoodSize(), Isis::CubeIoHandler::lineCount(), Isis::CubeIoHandler::pixelType(), Isis::CubeIoHandler::sampleCount(), Isis::CubeIoHandler::setChunkSizes(), and Isis::SizeOf().

◆ ~CubeBsqHandler()

Isis::CubeBsqHandler::~CubeBsqHandler ( )

The destructor writes all cached data to disk.

Definition at line 58 of file CubeBsqHandler.cpp.

References Isis::CubeIoHandler::clearCache().

◆ CubeBsqHandler() [2/2]

Isis::CubeBsqHandler::CubeBsqHandler ( const CubeBsqHandler & other)
private

Disallow copying of this object.

Parameters
otherThe object to copy.

Member Function Documentation

◆ addCachingAlgorithm()

void Isis::CubeIoHandler::addCachingAlgorithm ( CubeCachingAlgorithm * algorithm)
inherited

This will add the given caching algorithm to the list of attempted caching algorithms.

The algorithms are tried in the opposite order that they were added - the first algorithm added is the last algorithm tried.

This method takes ownership of algorithm.

Parameters
algorithmThe caching algorithm to add to the Cube for I/O

Definition at line 364 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_cachingAlgorithms.

Referenced by Isis::Cube::addCachingAlgorithm().

◆ bandCount()

int Isis::CubeIoHandler::bandCount ( ) const
protectedinherited
Returns
the number of physical bands in the cube.

Definition at line 462 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_numBands.

Referenced by Isis::CubeIoHandler::findCubeChunks().

◆ blockUntilThreadPoolEmpty()

void Isis::CubeIoHandler::blockUntilThreadPoolEmpty ( ) const
privateinherited

This blocks (doesn't return) until the number of active runnables in the thread pool goes to 0.

This uses the m_writeThreadMutex, because the thread pool doesn't delete threads immediately, so runnables might actually still exist but are inconsequential/being deleted already.

Definition at line 667 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_ioThreadPool, and Isis::CubeIoHandler::m_writeThreadMutex.

Referenced by Isis::CubeIoHandler::flushWriteCache().

◆ bufferLessThan()

bool Isis::CubeIoHandler::bufferLessThan ( Buffer *const & lhs,
Buffer *const & rhs )
staticprivateinherited

This is used for sorting buffers into the most efficient write order.

Parameters
lhsThe left hand side of the '<' operator
rhsThe right hand side of the '<' operator
Returns
True if lhs is obviously earlier in a cube than rhs.

Definition at line 681 of file CubeIoHandler.cpp.

References Isis::Displacement::Pixels, and Isis::Distance::Pixels.

◆ clearCache()

void Isis::CubeIoHandler::clearCache ( bool blockForWriteCache = true) const
inherited

Free all cube chunks (cached cube data) from memory and write them to disk.

Child destructors need to call this method.

This method should only be called otherwise when lots of cubes are in memory and the many caches cause problems with system RAM limitations.

Parameters
blockForWriteCacheThis should be true unless this method is called from the write thread.

Definition at line 379 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::flushWriteCache(), Isis::CubeIoHandler::m_dataIsOnDiskMap, Isis::CubeIoHandler::m_lastProcessByLineChunks, Isis::CubeIoHandler::m_rawData, and Isis::CubeIoHandler::writeNullDataToDisk().

Referenced by Isis::Cube::clearIoCache(), Isis::Cube::copy(), Isis::CubeIoHandler::minimizeCache(), ~CubeBsqHandler(), and Isis::CubeTileHandler::~CubeTileHandler().

◆ dataFileMutex()

QMutex * Isis::CubeIoHandler::dataFileMutex ( )
inherited

Get the mutex that this IO handler is using around I/Os on the given data file.

A lock should be acquired before doing any reads/writes on the data file externally.

Returns
A mutex that can guarantee exclusive access to the data file

Definition at line 455 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_writeThreadMutex.

Referenced by Isis::Cube::read(), Isis::Cube::write(), and Isis::Cube::writeLabels().

◆ findCubeChunks()

QPair< QList< RawCubeChunk * >, QList< int > > Isis::CubeIoHandler::findCubeChunks ( int startSample,
int numSamples,
int startLine,
int numLines,
int startBand,
int numBands ) const
privateinherited

Get the cube chunks that correspond to the given cube area.

This will create and initialize the chunks if they are not already in memory.

Parameters
startSampleThe starting sample of the cube data
numSamplesThe number of samples of cube data
startLineThe starting line of the cube data
numLinesThe number of lines of cube data
startBandThe starting band of the cube data
numBandsThe number of bands of cube data
Returns
The cube chunks that correspond to the given cube area

The user has requested a slice of a cube (a rectangle):

Cube
-----—

| | | <---------— Requested Area | | | | | -----— |

| |

But our cube is split into "chunks" which may not line up to what was requested:


| | | || |------------------—|| | -----— | || |—|---—|-------—|| (User requested area intersected with | | | | | || cube chunks) |------------------—||

| | | ||


| 1 | 2 | 3 || |------------------—|| | 4 | 5 | 6 || |------------------—|| (User requested area intersected with | 7 | 8 | 9 || cube chunks) |------------------—||

| 10 | 11 | 12 ||

(In this case we want chunks 4, 5, 7, and 8)

So we want to find which cube chunks line up to the requested area. To do this, we find which chunk intersects the top left of the requested cube area. That chunk is a given that we need it. Then we search in the positive X direction for more chunks that intersect the user-requested area until we no longer do. Then we know all of the chunks that intersect a whole y-range of the requested area. We shrink the requested area (remove the found y-range) and repeat.

Definition at line 730 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::bandCount(), Isis::CubeIoHandler::getChunk(), Isis::CubeIoHandler::getChunkCountInLineDimension(), Isis::CubeIoHandler::getChunkCountInSampleDimension(), Isis::CubeIoHandler::lineCount(), Isis::CubeIoHandler::m_bandsInChunk, Isis::CubeIoHandler::m_linesInChunk, Isis::CubeIoHandler::m_samplesInChunk, Isis::CubeIoHandler::m_virtualBands, and Isis::CubeIoHandler::sampleCount().

Referenced by Isis::CubeIoHandler::read(), and Isis::CubeIoHandler::synchronousWrite().

◆ findGoodSize()

int Isis::CubeBsqHandler::findGoodSize ( int maxSize,
int dimensionSize ) const
private

This method attempts to compute a good chunk line size.

Chunk band size is always 1 and chunk sample size is always number of samples in the cube for this format.

Parameters
maxSizeThe largest allowed size of a chunk dimension
dimensionSizeThe cube's size for the chunk size we are trying to calculate (number of lines)
Returns
The calculated chunk size for the dimension given

Definition at line 133 of file CubeBsqHandler.cpp.

Referenced by CubeBsqHandler().

◆ findIntersection()

void Isis::CubeIoHandler::findIntersection ( const RawCubeChunk & cube1,
const Buffer & cube2,
int & startX,
int & startY,
int & startZ,
int & endX,
int & endY,
int & endZ ) const
privateinherited

Find the intersection between the buffer area and the cube chunk.

This accounts for virtual bands when considering the buffer and may return an area where all of the bands don't actually intersect due to virtual bands. This is implemented the way it is for performance reasons - so much as returning an Area3D or a QList<int> caused undesirable slowdowns.

Parameters
cube1The cube chunk to intersect
cube2The buffer (in virtual band space) to intersect
startX(output) The leftmost sample position (inclusive)
startY(output) The topmost line position (inclusive)
startZ(output) The frontmost band position (inclusive)
endX(output) The rightmost sample position (inclusive)
endY(output) The bottommost line position (inclusive)
endZ(output) The backmost band position (inclusive)

Definition at line 875 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_virtualBands.

Referenced by Isis::CubeIoHandler::writeIntoDouble(), and Isis::CubeIoHandler::writeIntoRaw().

◆ flushWriteCache()

void Isis::CubeIoHandler::flushWriteCache ( bool force = false) const
privateinherited

This attempts to write the so-far-unwritten buffers from m_writeCache into the cube's RawCubeChunk cache.

This will not do anything if there is a runnable and will block if the write cache grows to be too large.

Parameters
forceSet to true to force start a flush

Definition at line 940 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::blockUntilThreadPoolEmpty(), Isis::CubeIoHandler::m_consecutiveOverflowCount, Isis::CubeIoHandler::m_idealFlushSize, Isis::CubeIoHandler::m_ioThreadPool, Isis::CubeIoHandler::m_lastOperationWasWrite, Isis::CubeIoHandler::m_useOptimizedCubeWrite, Isis::CubeIoHandler::m_writeCache, Isis::CubeIoHandler::m_writeThreadMutex, and Isis::CubeIoHandler::synchronousWrite().

Referenced by Isis::CubeIoHandler::clearCache(), Isis::CubeIoHandler::read(), and Isis::CubeIoHandler::write().

◆ freeChunk()

void Isis::CubeIoHandler::freeChunk ( RawCubeChunk * chunkToFree) const
privateinherited

If the chunk is dirty, then we write it to disk.

Regardless, we then free it from memory.

Parameters
chunkToFreeThe chunk we're removing from memory

Definition at line 1017 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::getChunkIndex(), Isis::CubeIoHandler::m_lastProcessByLineChunks, and Isis::CubeIoHandler::m_rawData.

Referenced by Isis::CubeIoHandler::minimizeCache().

◆ getBandCountInChunk()

int Isis::CubeIoHandler::getBandCountInChunk ( ) const
protectedinherited

◆ getBytesPerChunk()

BigInt Isis::CubeIoHandler::getBytesPerChunk ( ) const
protectedinherited
Returns
the byte size of each chunk in the cube. Currently they must be constant size, but this is planned to be changed at some point in time.
Number of bytes in a cube chunk

Definition at line 482 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_bandsInChunk, Isis::CubeIoHandler::m_linesInChunk, Isis::CubeIoHandler::m_pixelType, Isis::CubeIoHandler::m_samplesInChunk, and Isis::SizeOf().

Referenced by Isis::CubeIoHandler::getChunk(), getChunkStartByte(), Isis::CubeIoHandler::getDataSize(), Isis::CubeIoHandler::getNullChunk(), Isis::CubeTileHandler::getTileStartByte(), and Isis::CubeIoHandler::minimizeCache().

◆ getChunk()

RawCubeChunk * Isis::CubeIoHandler::getChunk ( int chunkIndex,
bool allocateIfNecessary ) const
privateinherited

Retrieve the cached chunk at the given chunk index, if there is one.

Parameters
chunkIndexThe position of the chunk in the cube
allocateIfNecessaryIf true, the chunk will be read into cache when necessary and this method will not return NULL.
Returns
NULL if data is not cached, otherwise the cube file data

Definition at line 1045 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::getBytesPerChunk(), Isis::CubeIoHandler::getChunkPlacement(), Isis::CubeIoHandler::getNullChunk(), Isis::CubeIoHandler::m_dataIsOnDiskMap, and Isis::CubeIoHandler::m_rawData.

Referenced by Isis::CubeIoHandler::findCubeChunks(), Isis::CubeIoHandler::read(), and Isis::CubeIoHandler::synchronousWrite().

◆ getChunkCount()

int Isis::CubeIoHandler::getChunkCount ( ) const
privateinherited
Returns
The number of chunks that are required to encapsulate all of the cube data

Definition at line 1086 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::getChunkCountInBandDimension(), Isis::CubeIoHandler::getChunkCountInLineDimension(), and Isis::CubeIoHandler::getChunkCountInSampleDimension().

Referenced by Isis::CubeIoHandler::writeNullDataToDisk().

◆ getChunkCountInBandDimension()

int Isis::CubeIoHandler::getChunkCountInBandDimension ( ) const
protectedinherited
Returns
the total number of chunks in the band (Z) dimension. This is always enough to contain every band in the cube.

Definition at line 492 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_bandsInChunk, and Isis::CubeIoHandler::m_numBands.

Referenced by Isis::CubeIoHandler::getChunkCount(), and Isis::CubeIoHandler::getDataSize().

◆ getChunkCountInLineDimension()

int Isis::CubeIoHandler::getChunkCountInLineDimension ( ) const
protectedinherited

◆ getChunkCountInSampleDimension()

int Isis::CubeIoHandler::getChunkCountInSampleDimension ( ) const
protectedinherited

◆ getChunkIndex()

int Isis::CubeIoHandler::getChunkIndex ( const RawCubeChunk & chunk) const
protectedinherited

Given a chunk, what's its index in the file.

Chunks are ordered from left to right, then top to bottom, then front to back (BSQ). The first chunk is at the top left of band 1 and is index 0, for example. In other words, this is going from the value of m_rawData to the key.

Chunks which sit outside of the cube entirely must not be passed into this method; the results will be wrong.

Returns
The chunk's index into the file

Definition at line 526 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::getBandCountInChunk(), Isis::CubeIoHandler::getChunkCountInLineDimension(), Isis::CubeIoHandler::getChunkCountInSampleDimension(), Isis::CubeIoHandler::getLineCountInChunk(), and Isis::CubeIoHandler::getSampleCountInChunk().

Referenced by Isis::CubeIoHandler::freeChunk(), getChunkStartByte(), and Isis::CubeTileHandler::getTileStartByte().

◆ getChunkPlacement()

void Isis::CubeIoHandler::getChunkPlacement ( int chunkIndex,
int & startSample,
int & startLine,
int & startBand,
int & endSample,
int & endLine,
int & endBand ) const
privateinherited

Get the X/Y/Z (Sample,Line,Band) range of the chunk at the given index.

Parameters
chunkIndexThe chunk number in the cube file
startSample(output) The leftmost sample position (inclusive)
startLine(output) The topmost line position (inclusive)
startBand(output) The frontmost band position (inclusive)
endSample(output) The rightmost sample position (inclusive)
endLine(output) The bottommost line position (inclusive)
endBand(output) The backmost band position (inclusive)

Definition at line 1104 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::getBandCountInChunk(), Isis::CubeIoHandler::getChunkCountInLineDimension(), Isis::CubeIoHandler::getChunkCountInSampleDimension(), Isis::CubeIoHandler::getLineCountInChunk(), and Isis::CubeIoHandler::getSampleCountInChunk().

Referenced by Isis::CubeIoHandler::getChunk(), Isis::CubeIoHandler::getNullChunk(), Isis::CubeIoHandler::read(), and Isis::CubeIoHandler::synchronousWrite().

◆ getChunkStartByte()

BigInt Isis::CubeBsqHandler::getChunkStartByte ( const RawCubeChunk & chunk) const
private

This is a helper method that goes from chunk to file position.

Parameters
chunkThe chunk to locate in the file.
Returns
The file position to start reading or writing at

Definition at line 157 of file CubeBsqHandler.cpp.

References Isis::CubeIoHandler::getBytesPerChunk(), Isis::CubeIoHandler::getChunkIndex(), and Isis::CubeIoHandler::getDataStartByte().

Referenced by readRaw(), and writeRaw().

◆ getDataFile()

QFile * Isis::CubeIoHandler::getDataFile ( )
protectedinherited
Returns
the QFile containing cube data. This is what should be read from and written to.
The data file for I/O

Definition at line 555 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_dataFile.

Referenced by readRaw(), Isis::CubeTileHandler::readRaw(), writeRaw(), and Isis::CubeTileHandler::writeRaw().

◆ getDataSize()

BigInt Isis::CubeIoHandler::getDataSize ( ) const
inherited
Returns
the number of bytes that the cube DNs will take up. This includes padding caused by the cube chunks not aligning with the cube dimensions.

Definition at line 421 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::getBytesPerChunk(), Isis::CubeIoHandler::getChunkCountInBandDimension(), Isis::CubeIoHandler::getChunkCountInLineDimension(), and Isis::CubeIoHandler::getChunkCountInSampleDimension().

Referenced by Isis::CubeIoHandler::setChunkSizes(), and Isis::Cube::write().

◆ getDataStartByte()

BigInt Isis::CubeIoHandler::getDataStartByte ( ) const
protectedinherited
Returns
the byte offset to the beginning of the cube data.

Definition at line 544 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_startByte.

Referenced by getChunkStartByte(), Isis::CubeTileHandler::getTileStartByte(), and Isis::CubeIoHandler::setChunkSizes().

◆ getLineCountInChunk()

int Isis::CubeIoHandler::getLineCountInChunk ( ) const
protectedinherited

◆ getNullChunk()

RawCubeChunk * Isis::CubeIoHandler::getNullChunk ( int chunkIndex) const
privateinherited

This creates a chunk filled with NULLs whose placement is at chunkIndex's position.

This is used for getting NULL-filled chunks the first time the chunk is requested.

Ownership of the return value is given to the caller.

Parameters
chunkIndexThe chunk's index which provides it's positioning.
Returns
A chunk filled with nulls at the chunkIndex position

Definition at line 1136 of file CubeIoHandler.cpp.

References Isis::RawCubeChunk::bandCount(), Isis::CubeIoHandler::getBytesPerChunk(), Isis::CubeIoHandler::getChunkPlacement(), Isis::RawCubeChunk::getRawData(), Isis::RawCubeChunk::getStartBand(), Isis::RawCubeChunk::getStartLine(), Isis::RawCubeChunk::getStartSample(), Isis::RawCubeChunk::lineCount(), Isis::CubeIoHandler::m_nullChunkData, Isis::Null, Isis::RawCubeChunk::sampleCount(), Isis::RawCubeChunk::setDirty(), Isis::RawCubeChunk::setRawData(), and Isis::CubeIoHandler::writeIntoRaw().

Referenced by Isis::CubeIoHandler::getChunk(), and Isis::CubeIoHandler::writeNullDataToDisk().

◆ getSampleCountInChunk()

int Isis::CubeIoHandler::getSampleCountInChunk ( ) const
protectedinherited

◆ lineCount()

int Isis::CubeIoHandler::lineCount ( ) const
protectedinherited
Returns
the number of lines in the cube. This does not include lines created by the chunk overflowing the line dimension.

Definition at line 564 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_numLines.

Referenced by CubeBsqHandler(), Isis::CubeTileHandler::CubeTileHandler(), and Isis::CubeIoHandler::findCubeChunks().

◆ minimizeCache()

void Isis::CubeIoHandler::minimizeCache ( const QList< RawCubeChunk * > & justUsed,
const Buffer & justRequested ) const
privateinherited

Apply the caching algorithms and get rid of excess cube data in memory.

This is intended to be called after every IO operation.

Parameters
justUsedThe cube chunks that were used in the IO operation that is calling this method.
justRequestedThe buffer that was used in the IO operation that is calling this method.

Definition at line 1188 of file CubeIoHandler.cpp.

References Isis::CubeCachingAlgorithm::CacheResult::algorithmUnderstoodData(), Isis::CubeIoHandler::clearCache(), Isis::CubeIoHandler::freeChunk(), Isis::CubeIoHandler::getBytesPerChunk(), Isis::CubeCachingAlgorithm::CacheResult::getChunksToFree(), Isis::CubeIoHandler::m_cachingAlgorithms, Isis::CubeIoHandler::m_rawData, and Isis::CubeCachingAlgorithm::recommendChunksToFree().

Referenced by Isis::CubeIoHandler::read(), and Isis::CubeIoHandler::synchronousWrite().

◆ operator=()

CubeBsqHandler & Isis::CubeBsqHandler::operator= ( const CubeBsqHandler & other)
private

Disallow assignments of this object.

Parameters
otherThe CubeBsqHandler on the right-hand side of the assignment that we are copying into *this.
Returns
A reference to *this.

◆ pixelType()

PixelType Isis::CubeIoHandler::pixelType ( ) const
protectedinherited
Returns
the physical cube DN format.

Definition at line 580 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_pixelType.

Referenced by CubeBsqHandler(), and Isis::CubeTileHandler::CubeTileHandler().

◆ read()

◆ readRaw()

void Isis::CubeBsqHandler::readRaw ( RawCubeChunk & chunkToFill)
protectedvirtual

This needs to populate the chunkToFill with unswapped raw bytes from the disk.

See also
CubeTileHandler::readRaw()
Parameters
chunkToFillThe container that needs to be filled with cube data.

Implements Isis::CubeIoHandler.

Definition at line 75 of file CubeBsqHandler.cpp.

References getChunkStartByte(), Isis::CubeIoHandler::getDataFile(), and Isis::IException::Io.

◆ sampleCount()

int Isis::CubeIoHandler::sampleCount ( ) const
protectedinherited
Returns
the number of samples in the cube. This does not include samples created by the chunk overflowing the sample dimension.

Definition at line 589 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_numSamples.

Referenced by CubeBsqHandler(), Isis::CubeTileHandler::CubeTileHandler(), Isis::CubeIoHandler::findCubeChunks(), and Isis::CubeIoHandler::synchronousWrite().

◆ setChunkSizes()

void Isis::CubeIoHandler::setChunkSizes ( int numSamples,
int numLines,
int numBands )
protectedinherited

This should be called once from the child constructor.

This determines the chunk sizes used for the cube and often should remain constant for a particular cube (BSQ, for example, doesn't need it to be constant but Tile does). These being large can cause excessive use of disk space that stores no cube data. These being large can also drastically increase the amount of RAM consumed.

Parameters
numSamplesThe chunk size in the sample dimension
numLinesThe chunk size in the line dimension
numBandsThe chunk size in the band dimension

Definition at line 614 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::getDataSize(), Isis::CubeIoHandler::getDataStartByte(), Isis::CubeIoHandler::m_bandsInChunk, Isis::CubeIoHandler::m_dataFile, Isis::CubeIoHandler::m_dataIsOnDiskMap, Isis::CubeIoHandler::m_linesInChunk, Isis::CubeIoHandler::m_samplesInChunk, and Isis::IException::Programmer.

Referenced by CubeBsqHandler(), and Isis::CubeTileHandler::CubeTileHandler().

◆ setVirtualBands()

void Isis::CubeIoHandler::setVirtualBands ( const QList< int > * virtualBandList)
inherited

This changes the virtual band list.

Parameters
virtualBandListA list where the indices are the vbands and the values are the physical bands. The values are 1-based. This can be specified as NULL, in which case the vbands are the physical bands. The virtual band list is copied (the pointer provided isn't remembered).

Definition at line 438 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::m_virtualBands.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), and Isis::Cube::setVirtualBands().

◆ synchronousWrite()

◆ updateLabels()

void Isis::CubeBsqHandler::updateLabels ( Pvl & label)
virtual

Function to update the labels with a Pvl object.

Parameters
labelPvl object to update with

Implements Isis::CubeIoHandler.

Definition at line 68 of file CubeBsqHandler.cpp.

References Isis::PvlObject::findObject().

◆ write()

void Isis::CubeIoHandler::write ( const Buffer & bufferToWrite)
inherited

Write buffer data into the cube data on disk.

This could do no IO if the cube chunks required are already in memory and our caching algorithms say to not free any of the cube chunks afterwards.

Parameters
bufferToWriteThe buffer to get cube data from.

Definition at line 334 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::flushWriteCache(), Isis::CubeIoHandler::m_ioThreadPool, Isis::CubeIoHandler::m_lastOperationWasWrite, Isis::CubeIoHandler::m_writeCache, Isis::CubeIoHandler::m_writeThreadMutex, and Isis::CubeIoHandler::synchronousWrite().

Referenced by Isis::Cube::write().

◆ writeIntoDouble()

void Isis::CubeIoHandler::writeIntoDouble ( const RawCubeChunk & chunk,
Buffer & output,
int index ) const
privateinherited

◆ writeIntoRaw()

◆ writeNullDataToDisk()

void Isis::CubeIoHandler::writeNullDataToDisk ( ) const
privateinherited

Write all NULL cube chunks that have not yet been accessed to disk.

Definition at line 1831 of file CubeIoHandler.cpp.

References Isis::CubeIoHandler::getChunkCount(), Isis::CubeIoHandler::getNullChunk(), Isis::CubeIoHandler::m_dataIsOnDiskMap, and Isis::IException::Programmer.

Referenced by Isis::CubeIoHandler::clearCache().

◆ writeRaw()

void Isis::CubeBsqHandler::writeRaw ( const RawCubeChunk & chunkToWrite)
protectedvirtual

This needs to write the chunkToWrite directly to disk with no modifications to the data itself.

See also
CubeTileHandler::writeRaw()
Parameters
chunkToWriteThe container that needs to be put on disk.

Implements Isis::CubeIoHandler.

Definition at line 100 of file CubeBsqHandler.cpp.

References getChunkStartByte(), Isis::CubeIoHandler::getDataFile(), and Isis::IException::Io.

Member Data Documentation

◆ m_bandsInChunk

◆ m_base

double Isis::CubeIoHandler::m_base
privateinherited

The additive offset of the data on disk.

Definition at line 284 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), Isis::CubeIoHandler::writeIntoDouble(), and Isis::CubeIoHandler::writeIntoRaw().

◆ m_byteOrder

ByteOrder Isis::CubeIoHandler::m_byteOrder
privateinherited

The byte order (endianness) of the data on disk.

Definition at line 290 of file CubeIoHandler.h.

◆ m_byteSwapper

EndianSwapper* Isis::CubeIoHandler::m_byteSwapper
privateinherited

A helper that swaps byte order to and from file order.

Definition at line 293 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), Isis::CubeIoHandler::writeIntoDouble(), Isis::CubeIoHandler::writeIntoRaw(), and Isis::CubeIoHandler::~CubeIoHandler().

◆ m_cachingAlgorithms

QList<CubeCachingAlgorithm *>* Isis::CubeIoHandler::m_cachingAlgorithms
privateinherited

◆ m_consecutiveOverflowCount

int Isis::CubeIoHandler::m_consecutiveOverflowCount
mutableprivateinherited

How many times the write cache has overflown in a row.

Definition at line 362 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), and Isis::CubeIoHandler::flushWriteCache().

◆ m_dataFile

QFile* Isis::CubeIoHandler::m_dataFile
privateinherited

The file containing cube data.

Definition at line 271 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), Isis::CubeIoHandler::getDataFile(), and Isis::CubeIoHandler::setChunkSizes().

◆ m_dataIsOnDiskMap

QMap<int, bool>* Isis::CubeIoHandler::m_dataIsOnDiskMap
mutableprivateinherited

◆ m_idealFlushSize

volatile int Isis::CubeIoHandler::m_idealFlushSize
mutableprivateinherited

Ideal write cache flush size.

Definition at line 359 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), and Isis::CubeIoHandler::flushWriteCache().

◆ m_ioThreadPool

QThreadPool* Isis::CubeIoHandler::m_ioThreadPool
mutableprivateinherited

This contains threads for doing cube writes (and later maybe cube reads).

We're using a thread pool so that the same QThread is reused for performance reasons.

Definition at line 342 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::blockUntilThreadPoolEmpty(), Isis::CubeIoHandler::CubeIoHandler(), Isis::CubeIoHandler::flushWriteCache(), Isis::CubeIoHandler::read(), Isis::CubeIoHandler::write(), and Isis::CubeIoHandler::~CubeIoHandler().

◆ m_lastOperationWasWrite

bool Isis::CubeIoHandler::m_lastOperationWasWrite
mutableprivateinherited

If the last operation was a write then we need to flush the cache when reading.

Keep track of this.

Definition at line 348 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), Isis::CubeIoHandler::flushWriteCache(), Isis::CubeIoHandler::read(), and Isis::CubeIoHandler::write().

◆ m_lastProcessByLineChunks

QList<RawCubeChunk *>* Isis::CubeIoHandler::m_lastProcessByLineChunks
mutableprivateinherited

This is an optimization for process by line.

It relies on chunks found often being the exact same between multiple reads or multiple writes.

Definition at line 329 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::clearCache(), Isis::CubeIoHandler::CubeIoHandler(), Isis::CubeIoHandler::freeChunk(), Isis::CubeIoHandler::synchronousWrite(), and Isis::CubeIoHandler::~CubeIoHandler().

◆ m_linesInChunk

◆ m_multiplier

double Isis::CubeIoHandler::m_multiplier
privateinherited

The multiplicative factor of the data on disk.

Definition at line 287 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), Isis::CubeIoHandler::writeIntoDouble(), and Isis::CubeIoHandler::writeIntoRaw().

◆ m_nullChunkData

QByteArray* Isis::CubeIoHandler::m_nullChunkData
mutableprivateinherited

A raw cube chunk's data when it was all NULL's. Used for speed.

Definition at line 332 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), Isis::CubeIoHandler::getNullChunk(), and Isis::CubeIoHandler::~CubeIoHandler().

◆ m_numBands

int Isis::CubeIoHandler::m_numBands
privateinherited

The number of physical bands in the cube.

Definition at line 302 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::bandCount(), Isis::CubeIoHandler::CubeIoHandler(), and Isis::CubeIoHandler::getChunkCountInBandDimension().

◆ m_numLines

int Isis::CubeIoHandler::m_numLines
privateinherited

◆ m_numSamples

int Isis::CubeIoHandler::m_numSamples
privateinherited

◆ m_pixelType

PixelType Isis::CubeIoHandler::m_pixelType
privateinherited

◆ m_rawData

QMap<int, RawCubeChunk *>* Isis::CubeIoHandler::m_rawData
mutableprivateinherited

◆ m_samplesInChunk

◆ m_startByte

BigInt Isis::CubeIoHandler::m_startByte
privateinherited

The start byte of the cube data.

This is 0-based (i.e. a value of 0 means write data into the first byte of the file). Usually the label is between 0 and m_startByte.

Definition at line 278 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), and Isis::CubeIoHandler::getDataStartByte().

◆ m_useOptimizedCubeWrite

bool Isis::CubeIoHandler::m_useOptimizedCubeWrite
privateinherited

This is true if the Isis preference for the cube write thread is optimized.

Definition at line 353 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), Isis::CubeIoHandler::flushWriteCache(), and Isis::CubeIoHandler::read().

◆ m_virtualBands

◆ m_writeCache

QPair< QMutex *, QList<Buffer *> >* Isis::CubeIoHandler::m_writeCache
privateinherited

These are the buffers we need to write to raw cube chunks.

Definition at line 335 of file CubeIoHandler.h.

Referenced by Isis::CubeIoHandler::CubeIoHandler(), Isis::CubeIoHandler::flushWriteCache(), Isis::CubeIoHandler::write(), and Isis::CubeIoHandler::~CubeIoHandler().

◆ m_writeThreadMutex


The documentation for this class was generated from the following files: