Construct Token list from a stream. More...

#include <PvlTokenizer.h>

Collaboration diagram for Isis::PvlTokenizer:

Public Member Functions
	PvlTokenizer ()
	Constructs a Tokenizer with an empty token list.

	~PvlTokenizer ()
	Destroys the Tokenizer object and token list.

void	Load (std::istream &stream, const QString &terminator="END")
	Loads the Token list from a stream.

void	Clear ()
	Empties the token list.

std::vector< Isis::PvlToken > &	GetTokenList ()

Protected Member Functions
QString	ReadComment (std::istream &stream)
	Reads and returns a comment from the stream.

QString	ReadToken (std::istream &stream)
	Reads and returns a token from the stream.

bool	SkipWhiteSpace (std::istream &stream)
	Skips over whitespace so long as it is not inside quotes.

QString	ReadToSingleQuote (std::istream &stream)

QString	ReadToDoubleQuote (std::istream &stream)

QString	ReadToParen (std::istream &stream)

QString	ReadToBrace (std::istream &stream)

void	ParseCommaList (Isis::PvlToken &t, const QString &cl)
	This routine parses a QString containing a comma separated list.

void	ValidateCharacter (int c)
	Make sure a character is valid printable (non-control) character.

Protected Attributes
std::vector< Isis::PvlToken >	tokens
	The array of Tokens parse out of the stream.

Detailed Description

Construct Token list from a stream.

This class tokenizes a stream. That is, it will take a stream, from a file or a string, and break the contents of the stream into keyword-value pairs. Examples of such are PDS labels, the standard Isis command line, and VICAR labels. Note that this does not validate the stream to ensure it is of PDS- or VICAR-type. It simply creates a list of keyword-value pairs which can be parsed by another object. The ruleset for tokenizing is straightforward. Consider SPACECRAFT=MARS_GLOBAL_SURVEYOR. The keyword would be SPACECRAFT and the value would be MARS_GLOBAL_SURVEYOR. Other valid examples include: LINES=5, FOCAL_LENGTH=12.4, INSTRUMENT="CAMERA_A", LIST=(0,1,5), and DOGS=("LAB","PUG","BULL"). The later examples, are considered arrays and therefore, will have multiple values associated with the keyword. Comments are allowed in the stream and are indicated by either "#" or "/ *" as the first character on the line.

Author: 2002-02-15 Jeff Anderson

History: 2003-02-25 Stuart Sides - Modified the way END is checked for. It was not working for embedded labels.

History: 2003-05-16 Stuart Sides - Modified schema from astrogeology... isis.astrogeology...

History: 2003-05-28 Stuart Sides - Modified so single quotes work the same as double quotes

History: 2003-10-28 Jeff Anderson - Fixed bug in order to allow for whitespace inside of arrays of strings

History: 2004-01-22 Jeff Anderson - Removed single quotes when the occur as delimeters inside of array keywords.

History: 2005-02-16 Jeff Anderson - Modified parsing of comma separated lists to not remove parens or squiggly brackets.

History: 2005-02-18 Elizabeth Ribelin - Modified file to support Doxygen documentation

History: 2006-05-31 Elizabeth Miller - Fixed bug in Load method when a keyword is loaded without a value

Todo: 2005-02-14 Jeff Anderson - finish class documentation and add coded and implementation examples

History: 2007-04-13 Stuart Sides - Fixed bug where quoted strings broken over more than two lines were not being read correctly.

History: 2009-03-13 Steven Lambright - Inline comments now correctly correlate to the keywords before them on the same line.

History: 2010-01-08 Eric Hyer - PvlTokenizer.cpp used EOF without including fstream, breaking this class on recent compilers (I added #include <fstream>).

Definition at line 76 of file PvlTokenizer.h.

Constructor & Destructor Documentation

◆ PvlTokenizer()

Isis::PvlTokenizer::PvlTokenizer ( )

Constructs a Tokenizer with an empty token list.

Definition at line 22 of file PvlTokenizer.cpp.

References Clear().

◆ ~PvlTokenizer()

Isis::PvlTokenizer::~PvlTokenizer ( )

Destroys the Tokenizer object and token list.

Definition at line 27 of file PvlTokenizer.cpp.

References Clear().

Member Function Documentation

◆ Clear()

void Isis::PvlTokenizer::Clear ( )

Empties the token list.

Definition at line 32 of file PvlTokenizer.cpp.

References tokens.

Referenced by PvlTokenizer(), and ~PvlTokenizer().

◆ GetTokenList()

vector< Isis::PvlToken > & Isis::PvlTokenizer::GetTokenList ( )

Definition at line 487 of file PvlTokenizer.cpp.

◆ Load()

void Isis::PvlTokenizer::Load	(	std::istream &	stream,
		const QString &	terminator = "END" )

Loads the Token list from a stream.

The loading will be terminated upon reaching either 1) end-of-stream, or 2) a programmer specified terminator QString

Parameters

stream	The input stream to tokenize
terminator	If the tokenizer see's this QString as a token in the input stream it will cease tokenizing. Defaults to "END"

Exceptions

Isis::iException::Parse

Definition at line 48 of file PvlTokenizer.cpp.

References Isis::Message::KeywordValueBad(), ParseCommaList(), ReadComment(), ReadToken(), SkipWhiteSpace(), tokens, Isis::IException::Unknown, and ValidateCharacter().

◆ ParseCommaList()

void Isis::PvlTokenizer::ParseCommaList	(	Isis::PvlToken &	t,
		const QString &	cl )

protected

This routine parses a QString containing a comma separated list.

Each of the items in the list is stored as a value in the Token.

Parameters

t	Token to load the comma separated list
cl	QString containing comma separated list

Definition at line 449 of file PvlTokenizer.cpp.

References SkipWhiteSpace().

Referenced by Load().

◆ ReadComment()

QString Isis::PvlTokenizer::ReadComment ( std::istream & stream )

protected

Reads and returns a comment from the stream.

Parameters

stream Input stream to read from

Returns: QString

Definition at line 205 of file PvlTokenizer.cpp.

References ValidateCharacter().

Referenced by Load().

◆ ReadToBrace()

QString Isis::PvlTokenizer::ReadToBrace ( std::istream & stream )

protected

Definition at line 397 of file PvlTokenizer.cpp.

◆ ReadToDoubleQuote()

QString Isis::PvlTokenizer::ReadToDoubleQuote ( std::istream & stream )

protected

Definition at line 277 of file PvlTokenizer.cpp.

◆ ReadToken()

QString Isis::PvlTokenizer::ReadToken ( std::istream & stream )

protected

Reads and returns a token from the stream.

A token is delimited by either whitespace or an equal sign. In the case of whitespace the token will be considered valueless. That is, there will be no value in the value side of the token (e.g., KEYWORD=).

Parameters

stream Input stream to read from

Returns: QString

Definition at line 233 of file PvlTokenizer.cpp.

References ValidateCharacter().

Referenced by Load().

◆ ReadToParen()

QString Isis::PvlTokenizer::ReadToParen ( std::istream & stream )

protected

Definition at line 353 of file PvlTokenizer.cpp.

◆ ReadToSingleQuote()

QString Isis::PvlTokenizer::ReadToSingleQuote ( std::istream & stream )

protected

Definition at line 315 of file PvlTokenizer.cpp.

◆ SkipWhiteSpace()

bool Isis::PvlTokenizer::SkipWhiteSpace ( std::istream & stream )

protected

Skips over whitespace so long as it is not inside quotes.

Whitespace is tabs, blanks, line feeds, carriage returns, and NULLs.

Parameters

stream Input stream to read from

Definition at line 257 of file PvlTokenizer.cpp.

References ValidateCharacter().

Referenced by Load(), and ParseCommaList().

◆ ValidateCharacter()

void Isis::PvlTokenizer::ValidateCharacter ( int c )

protected

Make sure a character is valid printable (non-control) character.

Parameters

c	Character to be validated

Definition at line 496 of file PvlTokenizer.cpp.

References Isis::IException::Unknown.

Referenced by Load(), ReadComment(), ReadToken(), and SkipWhiteSpace().

Member Data Documentation

◆ tokens

std::vector<Isis::PvlToken> Isis::PvlTokenizer::tokens

protected

The array of Tokens parse out of the stream.

Definition at line 79 of file PvlTokenizer.h.

Referenced by Clear(), and Load().

The documentation for this class was generated from the following files:

src/base/objs/PvlTokenizer/PvlTokenizer.h
src/base/objs/PvlTokenizer/PvlTokenizer.cpp

Public Member Functions

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ PvlTokenizer()

◆ ~PvlTokenizer()

Member Function Documentation

◆ Clear()

◆ GetTokenList()

◆ Load()

◆ ParseCommaList()

◆ ReadComment()

◆ ReadToBrace()

◆ ReadToDoubleQuote()

◆ ReadToken()

◆ ReadToParen()

◆ ReadToSingleQuote()

◆ SkipWhiteSpace()

◆ ValidateCharacter()

Member Data Documentation

◆ tokens