Isis 3 Programmer Reference
Isis::PvlTokenizer Class Reference

Construct Token list from a stream. More...

#include <PvlTokenizer.h>

Collaboration diagram for Isis::PvlTokenizer:
Collaboration graph

Public Member Functions

 PvlTokenizer ()
 Constructs a Tokenizer with an empty token list. More...
 
 ~PvlTokenizer ()
 Destroys the Tokenizer object and token list. More...
 
void Load (std::istream &stream, const QString &terminator="END")
 Loads the Token list from a stream. More...
 
void Clear ()
 Empties the token list. More...
 
std::vector< Isis::PvlToken > & GetTokenList ()
 

Protected Member Functions

QString ReadComment (std::istream &stream)
 Reads and returns a comment from the stream. More...
 
QString ReadToken (std::istream &stream)
 Reads and returns a token from the stream. More...
 
bool SkipWhiteSpace (std::istream &stream)
 Skips over whitespace so long as it is not inside quotes. More...
 
QString ReadToSingleQuote (std::istream &stream)
 
QString ReadToDoubleQuote (std::istream &stream)
 
QString ReadToParen (std::istream &stream)
 
QString ReadToBrace (std::istream &stream)
 
void ParseCommaList (Isis::PvlToken &t, const QString &cl)
 This routine parses a QString containing a comma separated list. More...
 
void ValidateCharacter (int c)
 Make sure a character is valid printable (non-control) character. More...
 

Protected Attributes

std::vector< Isis::PvlTokentokens
 The array of Tokens parse out of the stream. More...
 

Detailed Description

Construct Token list from a stream.

This class tokenizes a stream. That is, it will take a stream, from a file or a string, and break the contents of the stream into keyword-value pairs. Examples of such are PDS labels, the standard Isis command line, and VICAR labels. Note that this does not validate the stream to ensure it is of PDS- or VICAR-type. It simply creates a list of keyword-value pairs which can be parsed by another object. The ruleset for tokenizing is straightforward. Consider SPACECRAFT=MARS_GLOBAL_SURVEYOR. The keyword would be SPACECRAFT and the value would be MARS_GLOBAL_SURVEYOR. Other valid examples include: LINES=5, FOCAL_LENGTH=12.4, INSTRUMENT="CAMERA_A", LIST=(0,1,5), and DOGS=("LAB","PUG","BULL"). The later examples, are considered arrays and therefore, will have multiple values associated with the keyword. Comments are allowed in the stream and are indicated by either "#" or "/ *" as the first character on the line.

Author
2002-02-15 Jeff Anderson
History:
2003-02-25 Stuart Sides - Modified the way END is checked for. It was not working for embedded labels.
History:
2003-05-16 Stuart Sides - Modified schema from astrogeology... isis.astrogeology...
History:
2003-05-28 Stuart Sides - Modified so single quotes work the same as double quotes
History:
2003-10-28 Jeff Anderson - Fixed bug in order to allow for whitespace inside of arrays of strings
History:
2004-01-22 Jeff Anderson - Removed single quotes when the occur as delimeters inside of array keywords.
History:
2005-02-16 Jeff Anderson - Modified parsing of comma separated lists to not remove parens or squiggly brackets.
History:
2005-02-18 Elizabeth Ribelin - Modified file to support Doxygen documentation
History:
2006-05-31 Elizabeth Miller - Fixed bug in Load method when a keyword is loaded without a value
Todo:
2005-02-14 Jeff Anderson - finish class documentation and add coded and implementation examples
History:
2007-04-13 Stuart Sides - Fixed bug where quoted strings broken over more than two lines were not being read correctly.
History:
2009-03-13 Steven Lambright - Inline comments now correctly correlate to the keywords before them on the same line.
History:
2010-01-08 Eric Hyer - PvlTokenizer.cpp used EOF without including fstream, breaking this class on recent compilers (I added #include <fstream>).

Definition at line 92 of file PvlTokenizer.h.

Constructor & Destructor Documentation

◆ PvlTokenizer()

Isis::PvlTokenizer::PvlTokenizer ( )

Constructs a Tokenizer with an empty token list.

Definition at line 37 of file PvlTokenizer.cpp.

◆ ~PvlTokenizer()

Isis::PvlTokenizer::~PvlTokenizer ( )

Destroys the Tokenizer object and token list.

Definition at line 42 of file PvlTokenizer.cpp.

Member Function Documentation

◆ Clear()

void Isis::PvlTokenizer::Clear ( )

Empties the token list.

Definition at line 47 of file PvlTokenizer.cpp.

◆ Load()

void Isis::PvlTokenizer::Load ( std::istream &  stream,
const QString &  terminator = "END" 
)

Loads the Token list from a stream.

The loading will be terminated upon reaching either 1) end-of-stream, or 2) a programmer specified terminator QString

Parameters
streamThe input stream to tokenize
terminatorIf the tokenizer see's this QString as a token in the input stream it will cease tokenizing. Defaults to "END"
Exceptions
Isis::iException::Parse

Definition at line 63 of file PvlTokenizer.cpp.

References _FILEINFO_, Isis::PvlToken::addValue(), Isis::PvlToken::key(), Isis::PvlToken::keyUpper(), and Isis::Message::KeywordValueBad().

◆ ParseCommaList()

void Isis::PvlTokenizer::ParseCommaList ( Isis::PvlToken t,
const QString &  cl 
)
protected

This routine parses a QString containing a comma separated list.

Each of the items in the list is stored as a value in the Token.

Parameters
tToken to load the comma separated list
clQString containing comma separated list

Definition at line 464 of file PvlTokenizer.cpp.

References Isis::PvlToken::addValue().

◆ ReadComment()

QString Isis::PvlTokenizer::ReadComment ( std::istream &  stream)
protected

Reads and returns a comment from the stream.

Parameters
streamInput stream to read from
Returns
QString

Definition at line 220 of file PvlTokenizer.cpp.

◆ ReadToken()

QString Isis::PvlTokenizer::ReadToken ( std::istream &  stream)
protected

Reads and returns a token from the stream.

A token is delimited by either whitespace or an equal sign. In the case of whitespace the token will be considered valueless. That is, there will be no value in the value side of the token (e.g., KEYWORD=).

Parameters
streamInput stream to read from
Returns
QString

Definition at line 248 of file PvlTokenizer.cpp.

◆ SkipWhiteSpace()

bool Isis::PvlTokenizer::SkipWhiteSpace ( std::istream &  stream)
protected

Skips over whitespace so long as it is not inside quotes.

Whitespace is tabs, blanks, line feeds, carriage returns, and NULLs.

Parameters
streamInput stream to read from

Definition at line 272 of file PvlTokenizer.cpp.

◆ ValidateCharacter()

void Isis::PvlTokenizer::ValidateCharacter ( int  c)
protected

Make sure a character is valid printable (non-control) character.

Parameters
cCharacter to be validated

Definition at line 511 of file PvlTokenizer.cpp.

References _FILEINFO_.

Member Data Documentation

◆ tokens

std::vector<Isis::PvlToken> Isis::PvlTokenizer::tokens
protected

The array of Tokens parse out of the stream.

Definition at line 95 of file PvlTokenizer.h.


The documentation for this class was generated from the following files: