 |
Isis 3 Programmer Reference
|
|
Isis 3 Programmer Reference
|
This class is used to approximate cumulative probibility distributions of a stream of observations without storing the observations or having any apriori knowlege of the range of the data. More...
#include <StatCumProbDistDynCalc.h>
Classes | |
| class | XmlHandler |
Public Member Functions | |
| StatCumProbDistDynCalc (unsigned int nodes=20, QObject *parent=0) | |
| Construtor sets up the class to start recieving data. More... | |
| StatCumProbDistDynCalc (Project *project, XmlStackedHandlerReader *xmlReader, QObject *parent=0) | |
| StatCumProbDistDynCalc (const StatCumProbDistDynCalc &other) | |
| ~StatCumProbDistDynCalc () | |
| Destroys StatCumProbDistDynCalc object. More... | |
| StatCumProbDistDynCalc & | operator= (const StatCumProbDistDynCalc &other) |
| void | initialize () |
| Inializer, resets the class to start its dynamic calculation anew. More... | |
| void | setQuantiles (unsigned int nodes) |
| void | validate () |
| void | addObs (double obs) |
| Values for the estimated quantile positions are update as observations are added. More... | |
| double | cumProb (double value) |
| Provides the cumulative probility, that is, the proportion of the distribution that is less than or equal to the value given (according the current estimate of cumulative probility function). More... | |
| double | value (double cumProb) |
| Provides the value of the variable that has the given cumulative probility (according the current estimate of cumulative probility function) More... | |
| double | max () |
| Returns the maximum observation so far included in the dynamic calculation. More... | |
| double | min () |
| Returns the maximum observation so far included in the dynamic calculation. More... | |
| void | save (QXmlStreamWriter &stream, const Project *project) const |
| QDataStream & | write (QDataStream &stream) const |
| QDataStream & | read (QDataStream &stream) |
Private Attributes | |
| unsigned int | m_numberCells |
| The number of cells or histogram bins that are being used to model the probility density function. More... | |
| unsigned int | m_numberQuantiles |
| The number of quantiles being used to model the probility density function. More... | |
| unsigned int | m_numberObservations |
| The number of observations, note this is dynamically changing as observations are added. More... | |
| QList< double > | m_quantiles |
| The target quantiles being modeled, between 0 and 1. More... | |
| QList< double > | m_observationValues |
| The calculated values of the quantiles, note this is dynamically changing as observations are added. More... | |
| QList< double > | m_idealNumObsBelowQuantile |
| The ideal number of observations that should be less than or equal to the value of the corresponding quantiles, note this is dynamically changing as observations are added. More... | |
| QList< int > | m_numObsBelowQuantile |
| The actual number of observations that are less than or equal to the value of the corresponding quantiles, note this is dynamically changing as observations are added. More... | |
This class is used to approximate cumulative probibility distributions of a stream of observations without storing the observations or having any apriori knowlege of the range of the data.
This class is used to approximate cumulative probibility distributions of a stream of observations without storing the observations or having any apriori knowlege of the range of the data. "The P^2 algorithim for dynamic calculation of Quantiles and Histograms without storing Observations" Raj Jain and Imrich Chlamtac, Communication of the ACM Oct 1985, is used. A finite set of evenly spaced qunatiles are dynamically updated as more observations are added. The number of quantiles is set in the construtor, and has a defualt of 20. After sufficient data points (number of observations >> number of quantiles to track) the class provides cumulative probility as a function of value or vice versa. Thus it can be used to build histograms or find any number of discrete quantiles. Specific points on the function are evaluated by fiting piece wise parabolic functions to the three nearest adjacent nodes. Preformance of algorithim is within a few percent error for most of the distribution (given sufficient data), however care should be taken if the points to be querried are within 200/(numberOfQuantiles-1)% of the edges of the distributions. Near the edges the individual quantiles are still well calculated, but the piece wise parabolic function doesn't always fit the tails well, so interpolated points are more unrealiable. Developement note: Two possible ways to improve the fitting of the tails: caculate more densely place quantiles near the edges, use exponential regression (or some other alternative–perhaps adaptively selected).
2012-03-23 Orrin Thomas - Original Version
2014-07-19 Jeannie Backer - Added QDataStream >> and << operator methods. Brought code closer to ISIS standards. Updated unitTest to include these methods.
2014-09-11 Jeannie Backer - Added xml write/read capabilities. Fixed bug in cumPro() method for previously untested lines (case where the given value is closest to the last quantile value). Renamed member variables for clarity.
Definition at line 79 of file StatCumProbDistDynCalc.h.
| Isis::StatCumProbDistDynCalc::StatCumProbDistDynCalc | ( | unsigned int | nodes = 20, |
| QObject * | parent = 0 |
||
| ) |
Construtor sets up the class to start recieving data.
| [in] | unsigned | int nodes – this is the number of specific evenly spaced quantiles that will be dynamically tracked |
Definition at line 50 of file StatCumProbDistDynCalc.cpp.
References initialize().
| Isis::StatCumProbDistDynCalc::~StatCumProbDistDynCalc | ( | ) |
Destroys StatCumProbDistDynCalc object.
Definition at line 82 of file StatCumProbDistDynCalc.cpp.
| void Isis::StatCumProbDistDynCalc::addObs | ( | double | obs | ) |
Values for the estimated quantile positions are update as observations are added.
| [in] | double | obs – the individual observation to be used to dynamically readjust the cumulative probility distribution |
Definition at line 407 of file StatCumProbDistDynCalc.cpp.
References m_idealNumObsBelowQuantile, m_numberCells, m_numberObservations, m_numberQuantiles, m_numObsBelowQuantile, m_observationValues, and m_quantiles.
Referenced by Isis::BundleResults::addProbabilityDistributionObservation(), and Isis::BundleResults::addResidualsProbabilityDistributionObservation().
| double Isis::StatCumProbDistDynCalc::cumProb | ( | double | value | ) |
Provides the cumulative probility, that is, the proportion of the distribution that is less than or equal to the value given (according the current estimate of cumulative probility function).
| [in] | value | – the upper bound of values considered in the cumlative probility calculation |
| IsisProgrammerError | – StatCumProbDistDynCalc will return no data until there has been at least m_numberQuantiles observations added |
Definition at line 312 of file StatCumProbDistDynCalc.cpp.
References m_numberCells, m_numberQuantiles, m_observationValues, m_quantiles, and value().
Referenced by value().
| void Isis::StatCumProbDistDynCalc::initialize | ( | ) |
Inializer, resets the class to start its dynamic calculation anew.
| [in] | unsigned | int nodes – this is the number of specific evenly spaced quantiles that will be dynamically tracked |
Definition at line 110 of file StatCumProbDistDynCalc.cpp.
References m_idealNumObsBelowQuantile, m_numberCells, m_numberObservations, m_numberQuantiles, m_numObsBelowQuantile, m_observationValues, and m_quantiles.
Referenced by StatCumProbDistDynCalc().
| double Isis::StatCumProbDistDynCalc::max | ( | ) |
Returns the maximum observation so far included in the dynamic calculation.
| IsisProgrammerError | – StatCumProbDistDynCalc will return no data until the number of observations added matches the number of quantiles (i.e. number of nodes) selected. |
Definition at line 156 of file StatCumProbDistDynCalc.cpp.
References m_numberCells, and m_observationValues.
Referenced by Isis::BundleSolutionInfo::outputHeader().
| double Isis::StatCumProbDistDynCalc::min | ( | ) |
Returns the maximum observation so far included in the dynamic calculation.
| IsisProgrammerError | – StatCumProbDistDynCalc will return no data until the number of observations added matches the number of quantiles (i.e. number of nodes) selected. |
Definition at line 173 of file StatCumProbDistDynCalc.cpp.
References m_observationValues.
Referenced by Isis::BundleSolutionInfo::outputHeader().
| double Isis::StatCumProbDistDynCalc::value | ( | double | cumProb | ) |
Provides the value of the variable that has the given cumulative probility (according the current estimate of cumulative probility function)
| [in] | cumProb | – cumlative probability, domain [0, 1] |
| IsisProgrammerError | – StatCumProbDistDynCalc will return no data until the number of observations added matches the number of quantiles (i.e. number of nodes) selected. |
| IsisProgrammerError | – Invalid cumulative probability passed in to StatCumProbDistDynCalc::value(double cumProb). Must be on the domain [0, 1]. |
Definition at line 195 of file StatCumProbDistDynCalc.cpp.
References _FILEINFO_, cumProb(), m_numberCells, m_numberQuantiles, m_observationValues, m_quantiles, Isis::IException::Programmer, and Isis::toString().
Referenced by cumProb(), Isis::BundleSolutionInfo::outputHeader(), and Isis::BundleResults::printMaximumLikelihoodTierInformation().
|
private |
The ideal number of observations that should be less than or equal to the value of the corresponding quantiles, note this is dynamically changing as observations are added.
Definition at line 150 of file StatCumProbDistDynCalc.h.
Referenced by addObs(), and initialize().
|
private |
The number of cells or histogram bins that are being used to model the probility density function.
Definition at line 135 of file StatCumProbDistDynCalc.h.
Referenced by addObs(), cumProb(), initialize(), max(), and value().
|
private |
The number of observations, note this is dynamically changing as observations are added.
Definition at line 142 of file StatCumProbDistDynCalc.h.
Referenced by addObs(), and initialize().
|
private |
The number of quantiles being used to model the probility density function.
This value is one more than the number of cells, (i.e. m_numberQuantiles=m_cells+1).
Definition at line 138 of file StatCumProbDistDynCalc.h.
Referenced by addObs(), cumProb(), initialize(), and value().
|
private |
The actual number of observations that are less than or equal to the value of the corresponding quantiles, note this is dynamically changing as observations are added.
Definition at line 155 of file StatCumProbDistDynCalc.h.
Referenced by addObs(), and initialize().
|
private |
The calculated values of the quantiles, note this is dynamically changing as observations are added.
Definition at line 147 of file StatCumProbDistDynCalc.h.
Referenced by addObs(), cumProb(), initialize(), max(), min(), and value().
|
private |
The target quantiles being modeled, between 0 and 1.
Definition at line 145 of file StatCumProbDistDynCalc.h.
Referenced by addObs(), cumProb(), initialize(), and value().