Isis 3 Programmer Reference
Isis::StatCumProbDistDynCalc Class Reference

This class is used to approximate cumulative probibility distributions of a stream of observations without storing the observations or having any apriori knowlege of the range of the data. More...

#include <StatCumProbDistDynCalc.h>

Inheritance diagram for Isis::StatCumProbDistDynCalc:
Inheritance graph
Collaboration diagram for Isis::StatCumProbDistDynCalc:
Collaboration graph

Classes

class  XmlHandler
 

Public Member Functions

 StatCumProbDistDynCalc (unsigned int nodes=20, QObject *parent=0)
 Construtor sets up the class to start recieving data. More...
 
 StatCumProbDistDynCalc (Project *project, XmlStackedHandlerReader *xmlReader, QObject *parent=0)
 
 StatCumProbDistDynCalc (const StatCumProbDistDynCalc &other)
 
 ~StatCumProbDistDynCalc ()
 Destroys StatCumProbDistDynCalc object. More...
 
StatCumProbDistDynCalcoperator= (const StatCumProbDistDynCalc &other)
 
void initialize ()
 Inializer, resets the class to start its dynamic calculation anew. More...
 
void setQuantiles (unsigned int nodes)
 
void validate ()
 
void addObs (double obs)
 Values for the estimated quantile positions are update as observations are added. More...
 
double cumProb (double value)
 Provides the cumulative probility, that is, the proportion of the distribution that is less than or equal to the value given (according the current estimate of cumulative probility function). More...
 
double value (double cumProb)
 Provides the value of the variable that has the given cumulative probility (according the current estimate of cumulative probility function) More...
 
double max ()
 Returns the maximum observation so far included in the dynamic calculation. More...
 
double min ()
 Returns the maximum observation so far included in the dynamic calculation. More...
 
void save (QXmlStreamWriter &stream, const Project *project) const
 
QDataStream & write (QDataStream &stream) const
 
QDataStream & read (QDataStream &stream)
 

Private Attributes

unsigned int m_numberCells
 The number of cells or histogram bins that are being used to model the probility density function. More...
 
unsigned int m_numberQuantiles
 The number of quantiles being used to model the probility density function. More...
 
unsigned int m_numberObservations
 The number of observations, note this is dynamically changing as observations are added. More...
 
QList< double > m_quantiles
 The target quantiles being modeled, between 0 and 1. More...
 
QList< double > m_observationValues
 The calculated values of the quantiles, note this is dynamically changing as observations are added. More...
 
QList< double > m_idealNumObsBelowQuantile
 The ideal number of observations that should be less than or equal to the value of the corresponding quantiles, note this is dynamically changing as observations are added. More...
 
QList< int > m_numObsBelowQuantile
 The actual number of observations that are less than or equal to the value of the corresponding quantiles, note this is dynamically changing as observations are added. More...
 

Detailed Description

This class is used to approximate cumulative probibility distributions of a stream of observations without storing the observations or having any apriori knowlege of the range of the data.

This class is used to approximate cumulative probibility distributions of a stream of observations without storing the observations or having any apriori knowlege of the range of the data. "The P^2 algorithim for dynamic calculation of Quantiles and Histograms without storing Observations" Raj Jain and Imrich Chlamtac, Communication of the ACM Oct 1985, is used. A finite set of evenly spaced qunatiles are dynamically updated as more observations are added. The number of quantiles is set in the construtor, and has a defualt of 20. After sufficient data points (number of observations >> number of quantiles to track) the class provides cumulative probility as a function of value or vice versa. Thus it can be used to build histograms or find any number of discrete quantiles. Specific points on the function are evaluated by fiting piece wise parabolic functions to the three nearest adjacent nodes. Preformance of algorithim is within a few percent error for most of the distribution (given sufficient data), however care should be taken if the points to be querried are within 200/(numberOfQuantiles-1)% of the edges of the distributions. Near the edges the individual quantiles are still well calculated, but the piece wise parabolic function doesn't always fit the tails well, so interpolated points are more unrealiable. Developement note: Two possible ways to improve the fitting of the tails: caculate more densely place quantiles near the edges, use exponential regression (or some other alternative–perhaps adaptively selected).

Author
2012-03-23 Orrin Thomas
History:

2012-03-23 Orrin Thomas - Original Version

2014-07-19 Jeannie Backer - Added QDataStream >> and << operator methods. Brought code closer to ISIS standards. Updated unitTest to include these methods.

2014-09-11 Jeannie Backer - Added xml write/read capabilities. Fixed bug in cumPro() method for previously untested lines (case where the given value is closest to the last quantile value). Renamed member variables for clarity.

Definition at line 79 of file StatCumProbDistDynCalc.h.

Constructor & Destructor Documentation

◆ StatCumProbDistDynCalc()

Isis::StatCumProbDistDynCalc::StatCumProbDistDynCalc ( unsigned int  nodes = 20,
QObject parent = 0 
)

Construtor sets up the class to start recieving data.

Parameters
[in]unsignedint nodes – this is the number of specific evenly spaced quantiles that will be dynamically tracked

Definition at line 50 of file StatCumProbDistDynCalc.cpp.

References initialize().

◆ ~StatCumProbDistDynCalc()

Isis::StatCumProbDistDynCalc::~StatCumProbDistDynCalc ( )

Destroys StatCumProbDistDynCalc object.

Definition at line 82 of file StatCumProbDistDynCalc.cpp.

Member Function Documentation

◆ addObs()

void Isis::StatCumProbDistDynCalc::addObs ( double  obs)

Values for the estimated quantile positions are update as observations are added.

Parameters
[in]doubleobs – the individual observation to be used to dynamically readjust the cumulative probility distribution

Definition at line 407 of file StatCumProbDistDynCalc.cpp.

References m_idealNumObsBelowQuantile, m_numberCells, m_numberObservations, m_numberQuantiles, m_numObsBelowQuantile, m_observationValues, and m_quantiles.

Referenced by Isis::BundleResults::addProbabilityDistributionObservation(), and Isis::BundleResults::addResidualsProbabilityDistributionObservation().

◆ cumProb()

double Isis::StatCumProbDistDynCalc::cumProb ( double  value)

Provides the cumulative probility, that is, the proportion of the distribution that is less than or equal to the value given (according the current estimate of cumulative probility function).

Parameters
[in]value– the upper bound of values considered in the cumlative probility calculation
Returns
double – the cumulative probility, that is, the proportion of the distribution that is less than or equal to the value given (according the current estimate of cumulative probility function).
Exceptions
IsisProgrammerErrorStatCumProbDistDynCalc will return no data until there has been at least m_numberQuantiles observations added

Definition at line 312 of file StatCumProbDistDynCalc.cpp.

References m_numberCells, m_numberQuantiles, m_observationValues, m_quantiles, and value().

Referenced by value().

◆ initialize()

void Isis::StatCumProbDistDynCalc::initialize ( )

Inializer, resets the class to start its dynamic calculation anew.

Parameters
[in]unsignedint nodes – this is the number of specific evenly spaced quantiles that will be dynamically tracked

Definition at line 110 of file StatCumProbDistDynCalc.cpp.

References m_idealNumObsBelowQuantile, m_numberCells, m_numberObservations, m_numberQuantiles, m_numObsBelowQuantile, m_observationValues, and m_quantiles.

Referenced by StatCumProbDistDynCalc().

◆ max()

double Isis::StatCumProbDistDynCalc::max ( )

Returns the maximum observation so far included in the dynamic calculation.

Returns
double – the maximum observation so far included in the dynamic calculation
Exceptions
IsisProgrammerErrorStatCumProbDistDynCalc will return no data until the number of observations added matches the number of quantiles (i.e. number of nodes) selected.

Definition at line 156 of file StatCumProbDistDynCalc.cpp.

References m_numberCells, and m_observationValues.

Referenced by Isis::BundleSolutionInfo::outputHeader().

◆ min()

double Isis::StatCumProbDistDynCalc::min ( )

Returns the maximum observation so far included in the dynamic calculation.

Returns
double – the maximum observation so far included in the dynamic calculation
Exceptions
IsisProgrammerErrorStatCumProbDistDynCalc will return no data until the number of observations added matches the number of quantiles (i.e. number of nodes) selected.

Definition at line 173 of file StatCumProbDistDynCalc.cpp.

References m_observationValues.

Referenced by Isis::BundleSolutionInfo::outputHeader().

◆ value()

double Isis::StatCumProbDistDynCalc::value ( double  cumProb)

Provides the value of the variable that has the given cumulative probility (according the current estimate of cumulative probility function)

Parameters
[in]cumProb– cumlative probability, domain [0, 1]
Returns
double – the vaule of the variable that has the cumulative probility (according the current estimate of cumulative probility function)
Exceptions
IsisProgrammerErrorStatCumProbDistDynCalc will return no data until the number of observations added matches the number of quantiles (i.e. number of nodes) selected.
IsisProgrammerError– Invalid cumulative probability passed in to StatCumProbDistDynCalc::value(double cumProb). Must be on the domain [0, 1].

Definition at line 195 of file StatCumProbDistDynCalc.cpp.

References _FILEINFO_, cumProb(), m_numberCells, m_numberQuantiles, m_observationValues, m_quantiles, Isis::IException::Programmer, and Isis::toString().

Referenced by cumProb(), Isis::BundleSolutionInfo::outputHeader(), and Isis::BundleResults::printMaximumLikelihoodTierInformation().

Member Data Documentation

◆ m_idealNumObsBelowQuantile

QList<double> Isis::StatCumProbDistDynCalc::m_idealNumObsBelowQuantile
private

The ideal number of observations that should be less than or equal to the value of the corresponding quantiles, note this is dynamically changing as observations are added.

Definition at line 150 of file StatCumProbDistDynCalc.h.

Referenced by addObs(), and initialize().

◆ m_numberCells

unsigned int Isis::StatCumProbDistDynCalc::m_numberCells
private

The number of cells or histogram bins that are being used to model the probility density function.

Definition at line 135 of file StatCumProbDistDynCalc.h.

Referenced by addObs(), cumProb(), initialize(), max(), and value().

◆ m_numberObservations

unsigned int Isis::StatCumProbDistDynCalc::m_numberObservations
private

The number of observations, note this is dynamically changing as observations are added.

Definition at line 142 of file StatCumProbDistDynCalc.h.

Referenced by addObs(), and initialize().

◆ m_numberQuantiles

unsigned int Isis::StatCumProbDistDynCalc::m_numberQuantiles
private

The number of quantiles being used to model the probility density function.

This value is one more than the number of cells, (i.e. m_numberQuantiles=m_cells+1).

Definition at line 138 of file StatCumProbDistDynCalc.h.

Referenced by addObs(), cumProb(), initialize(), and value().

◆ m_numObsBelowQuantile

QList<int> Isis::StatCumProbDistDynCalc::m_numObsBelowQuantile
private

The actual number of observations that are less than or equal to the value of the corresponding quantiles, note this is dynamically changing as observations are added.

Definition at line 155 of file StatCumProbDistDynCalc.h.

Referenced by addObs(), and initialize().

◆ m_observationValues

QList<double> Isis::StatCumProbDistDynCalc::m_observationValues
private

The calculated values of the quantiles, note this is dynamically changing as observations are added.

Definition at line 147 of file StatCumProbDistDynCalc.h.

Referenced by addObs(), cumProb(), initialize(), max(), min(), and value().

◆ m_quantiles

QList<double> Isis::StatCumProbDistDynCalc::m_quantiles
private

The target quantiles being modeled, between 0 and 1.

Definition at line 145 of file StatCumProbDistDynCalc.h.

Referenced by addObs(), cumProb(), initialize(), and value().


The documentation for this class was generated from the following files: