Ogre::Compiler2Pass Class Reference

Compiler2Pass is a generic 2 pass compiler/assembler. More...

#include <OgreCompiler2Pass.h>

Inheritance diagram for Ogre::Compiler2Pass:

[legend]

List of all members.

Public Member Functions
	Compiler2Pass ()
	constructor
virtual	~Compiler2Pass ()
bool	compile (const String &source, const String &sourceName)
	compile the source - performs 2 passes.
virtual const String &	getClientBNFGrammer (void) const =0
	gets BNF Grammar.
virtual const String &	getClientGrammerName (void) const =0
	get the name of the BNF grammar.
void *	operator new (size_t sz, const char *file, int line, const char *func)
	operator new, with debug line info
void *	operator new (size_t sz)
void *	operator new (size_t sz, void *ptr)
	placement operator new
void *	operator new[] (size_t sz, const char *file, int line, const char *func)
	array operator new, with debug line info
void *	operator new[] (size_t sz)
void	operator delete (void *ptr)
void	operator delete (void *ptr, const char *, int, const char *)
void	operator delete[] (void *ptr)
void	operator delete[] (void *ptr, const char *, int, const char *)
Protected Types
enum	OperationType {   otUNKNOWN, otRULE, otAND, otOR,   otOPTIONAL, otREPEAT, otDATA, otNOT_TEST,   otINSERT_TOKEN, otEND }
enum	SystemRuleToken { _no_token_ = SystemTokenBase, _character_, _value_, _no_space_skip_ }
enum	BNF_ID {   BNF_UNKOWN = 0, BNF_SYNTAX, BNF_RULE, BNF_IDENTIFIER,   BNF_IDENTIFIER_RIGHT, BNF_IDENTIFIER_CHARACTERS, BNF_ID_BEGIN, BNF_ID_END,   BNF_CONSTANT_BEGIN, BNF_SET_RULE, BNF_EXPRESSION, BNF_AND_TERM,   BNF_OR_TERM, BNF_TERM, BNF_TERM_ID, BNF_CONSTANT,   BNF_OR, BNF_TERMINAL_SYMBOL, BNF_TERMINAL_START, BNF_REPEAT_EXPRESSION,   BNF_REPEAT_BEGIN, BNF_REPEAT_END, BNF_SET, BNF_SET_BEGIN,   BNF_SET_END, BNF_NOT_TEST, BNF_NOT_TEST_BEGIN, BNF_CONDITIONAL_TOKEN_INSERT,   BNF_OPTIONAL_EXPRESSION, BNF_NOT_EXPRESSION, BNF_NOT_CHK, BNF_OPTIONAL_BEGIN,   BNF_OPTIONAL_END, BNF_NO_TOKEN_START, BNF_SINGLEQUOTE, BNF_SINGLE_QUOTE_EXC,   BNF_SET_END_EXC, BNF_ANY_CHARACTER, BNF_SPECIAL_CHARACTERS1, BNF_SPECIAL_CHARACTERS2,   BNF_WHITE_SPACE_CHK, BNF_LETTER, BNF_LETTER_DIGIT, BNF_DIGIT,   BNF_WHITE_SPACE, BNF_ALPHA_SET, BNF_NUMBER_SET, BNF_SPECIAL_CHARACTER_SET1,   BNF_SPECIAL_CHARACTER_SET2, BNF_SPECIAL_CHARACTER_SET3, BNF_NOT_CHARS, BNF_AUTOTOKENSTART }
typedef std::vector < TokenRule >	TokenRuleContainer
typedef TokenRuleContainer::iterator	TokenRuleIterator
typedef std::vector < LexemeTokenDef >	LexemeTokenDefContainer
typedef LexemeTokenDefContainer::iterator	LexemeTokenDefIterator
typedef std::map < String, size_t >	LexemeTokenMap
typedef LexemeTokenMap::iterator	TokenKeyIterator
typedef std::vector < TokenInst >	TokenInstContainer
typedef TokenInstContainer::iterator	TokenInstIterator
typedef std::map < size_t, String >	LabelContainer
	storage container for string labels defined in source container uses Token index as a key associated with a label
Protected Member Functions
bool	doPass1 ()
	perform pass 1 of compile process scans source for lexemes that can be tokenized and then performs general semantic and context verification on each lexeme before it is tokenized.
bool	doPass2 ()
	performs Pass 2 of compile process which is execution of the tokens
virtual void	executeTokenAction (const size_t tokenID)=0
	execute the action associated with the token pointed to by the Pass 2 token instruction position.
virtual size_t	getAutoTokenIDStart () const =0
	Get the start ID for auto generated token IDs.
virtual void	setupTokenDefinitions (void)=0
	setup client token definitions.
const TokenInst &	getNextToken (const size_t expectedTokenID=0) const
	Gets the next token from the instruction que.
const TokenInst &	getCurrentToken (const size_t expectedTokenID=0) const
	Gets the current token from the instruction que.
bool	testNextTokenID (const size_t expectedTokenID) const
	If a next token instruction exist then test if its token ID matches.
bool	testCurrentTokenID (const size_t expectedTokenID) const
	If a current token instruction exist then test if its token ID matches.
void	skipToken (void) const
	skip to the next token in the pass2 queue.
void	replaceToken (void)
	go back to the previous token in the pass2 queue.
float	getNextTokenValue (void) const
	Gets the next token's associated floating point value in the instruction que that was parsed from the text source.
float	getCurrentTokenValue (void) const
	Gets the current token's associated floating point value in the instruction que that was parsed from the text source.
const String &	getNextTokenLabel (void) const
	Gets the next token's associated text label in the instruction que that was parsed from the text source.
const String &	getCurrentTokenLabel (void) const
	Gets the next token's associated text label in the instruction que that was parsed from the text source.
size_t	getNextTokenID (void) const
	Get the next token's ID value.
size_t	getCurrentTokenID (void) const
	Get the current token's ID value.
const String &	getNextTokenLexeme (void) const
	Get the next token's lexeme string.
const String &	getCurrentTokenLexeme (void) const
	Get the current token's lexeme string.
size_t	getPass2TokenQueCount (void) const
	Gets the number of tokens waiting in the instruction que that need to be processed by an token action in pass 2.
size_t	getRemainingTokensForAction (void) const
	Get the number of tokens not processed by action token.
void	setPass2TokenQuePosition (size_t pos, const bool activateAction=false)
	Manually set the Pass2 Token que position.
size_t	getPass2TokenQuePosition (void) const
	Get the current position in the Pass2 Token Que.
bool	setNextActionQuePosition (size_t pos, const bool search=false)
	Set the position of the next token action in the Pass2 Token Que.
size_t	addLexemeToken (const String &lexeme, const size_t token, const bool hasAction=false, const bool caseSensitive=false)
	Add a lexeme token association.
void	setClientBNFGrammer (void)
	Sets up the parser rules for the client based on the BNF Grammar text passed in.
void	findEOL ()
	find the eol character
bool	isFloatValue (float &fvalue, size_t &charsize) const
	check to see if the text at the present position in the source is a numerical constant
bool	isCharacterLabel (const size_t rulepathIDX)
	Check if source at current position is supposed to be a user defined character label.
bool	isLexemeMatch (const String &lexeme, const bool caseSensitive) const
	check to see if the text is in the lexeme text library
bool	isEndOfSource () const
	Check if pass 1 has parsed to the end of the source.
bool	positionToNextLexeme ()
	position to the next possible valid symbol
bool	processRulePath (size_t rulepathIDX)
	process input source text using rulepath to determine allowed tokens
void	setActiveContexts (const uint contexts)
	setup ActiveContexts - should be called by subclass to setup initial language contexts
void	skipComments ()
	comment specifiers are hard coded
void	skipEOL ()
	find end of line marker and move past it
void	skipWhiteSpace ()
	skip all the white space which includes spaces and tabs
bool	ValidateToken (const size_t rulepathIDX, const size_t activeRuleID)
	check if current position in source has the lexeme text equivalent to the TokenID
void	verifyTokenRuleLinks (const String &grammerName)
	scan through all the rules and initialize token definition with index to rules for non-terminal tokens.
void	checkTokenActionTrigger (void)
	Checks the last token instruction and if it has an action then it triggers the action of the previously found token having an action.
String	getBNFGrammerTextFromRulePath (size_t ruleID, const size_t level=0)
	Get the text representation of the rule path.
Protected Attributes
TokenState *	mClientTokenState
TokenState *	mActiveTokenState
	Active token que, definitions, rules currently being used by parser.
size_t	mPass2TokenQuePosition
	the location within the token instruction container where pass 2 is
size_t	mPreviousActionQuePosition
	the que position of the previous token that had an action.
size_t	mNextActionQuePosition
	the que position for the next token that has an action.
const String *	mSource
	pointer to the source to be compiled
String	mSourceName
	name of the source to be compiled
size_t	mEndOfSource
size_t	mCurrentLine
size_t	mCharPos
	current line number in source being tokenized
size_t	mErrorCharPos
	position in current line in source being tokenized
std::map< size_t, float >	mConstants
	character position in source where last error occurred
LabelContainer	mLabels
bool	mLabelIsActive
	flag indicates when a label is being parsed.
size_t	mActiveLabelKey
	the key of the active label being built during pass 1.
String *	mActiveLabel
	The active label that is receiving characters during pass 1.
bool	mNoSpaceSkip
	flag being true indicates that spaces are not to be skipped automatically gets set to false when mLabelIsActive goes to false
bool	mNoTerminalToken
	if flag is true then next terminal token is not added to token que if found but does effect rule path flow
size_t	mInsertTokenID
	TokenID to insert if next rule finds a terminal token if zero then no token inserted.
uint	mActiveContexts
	Active Contexts pattern used in pass 1 to determine which tokens are valid for a certain context.
Static Protected Attributes
static const size_t	SystemTokenBase = 1000
Private Types
typedef std::map < String, TokenState >	TokenStateContainer
Private Member Functions
void	activatePreviousTokenAction (void)
	if a previous token action was setup then activate it now
void	initBNFCompiler (void)
	initialize token definitions and rule paths
void	buildClientBNFRulePaths (void)
	Convert BNF grammar token que created in pass 1 into a BNF rule path.
void	modifyLastRule (const OperationType pendingRuleOp, const size_t tokenID)
	modify the last rule in the container. An end operation is added to the rule path.
size_t	getClientLexemeTokenID (const String &lexeme, const bool isCaseSensitive=false)
	get the token ID for a lexeme in the client state.
void	extractNonTerminal (const OperationType pendingRuleOp)
	Extract a Non Terminal identifier from the token que.
void	extractTerminal (const OperationType pendingRuleOp, const bool notoken=false)
	Extract a Terminal lexeme from the token que and add to current rule expression.
void	extractSet (const OperationType pendingRuleOp)
	Extract a set from the token que and add to current rule expression.
void	extractNumericConstant (const OperationType pendingRuleOp)
	Extract a numeric constant from the token que and add it to the current rule expression.
void	setConditionalTokenInsert (void)
	changes previous terminal token rule into a conditional terminal token insert rule
String	getLexemeText (size_t &ruleID, const size_t level=0)
	get the lexeme text of a rule.
Static Private Attributes
static TokenState	mBNFTokenState
static TokenStateContainer	mClientTokenStates
Classes
struct	LexemeTokenDef
	structure used to build lexeme Type library More...
struct	TokenInst
	map used to lookup client token based on previously defined lexeme More...
struct	TokenRule
	structure used to build rule paths More...
struct	TokenState

---

Detailed Description

Compiler2Pass is a generic 2 pass compiler/assembler.

Remarks:

provides a tokenizer in pass 1 and relies on the subclass to provide the virtual method for pass 2

PASS 1 - tokenize source: this is a simple brute force lexical scanner/analyzer that also parses the formed token for proper semantics and context in one pass it uses top down (recursive descent) ruling based on Backus - Naur Form (BNF) notation for semantic checking.

During Pass1, if a terminal token is identified as having an action then that action gets triggered when the next terminal token is encountered that has an action.

PASS 2 - generate application specific instructions i.e. native instructions based on the tokens in the instruction container.

this class must be subclassed with the subclass providing some implementation details for Pass 2. The subclass is responsible for setting up the token libraries along with defining the language syntax and handling token actions during the second pass.

The sub class normally supplies a simplified BNF text description in its constructor prior to doing any parsing/tokenizing of source. The simplified BNF text description defines the language syntax and rule structure. The meta-symbols used in the BNF text description are:

::= meaning "is defined as". "::=" starts the definition of a rule. The left side of ::= must contain an <identifier>

<> angle brackets are used to surround syntax rule names. A syntax rule name is also called a non-terminal in that it does not generate a terminal token in the instruction container for pass 2 processing.

| meaning "or". if the item on the left of the | fails then the item on the right is tested. Example: <true_false> ::= 'true' | 'false'; whitespace is used to imply AND operation between left and right items. Example: <terrain_shadaws> ::= 'terrain_shadows' <true_false> the 'terrain_shadows' terminal token must be found and <true_false> rule must pass in order for <terrain_shadows> rule to pass.

[] optional rule identifier is enclosed in meta symbols [ and ]. Note that only one identifier or terminal token can take [] modifier.

{} repetitive identifier (zero or more times) is enclosed in meta symbols { and } Note that only one identifier or terminal token can take {} modifier.

'' terminal tokens are surrounded by single quotes. A terminal token is always one or more characters. For example: 'Colour' defines a character sequence that must be matched in whole. Note that matching is case sensitive.

@ turn on single character scanning and don't skip white space. Mainly used for label processing that allow white space. Example: '@ ' prevents the white space between the quotes from being skipped

-'' no terminal token is generated when a - precedes the first single quote but the text in between the quotes is still tested against the characters in the source being parsed.

(?! ) negative lookahead (not test) inspired by Perl 5. Scans ahead for a non-terminal or terminal expression that should fail in order to make the rule production pass. Does not generate a token or advance the cursur. If the lookahead result fails ie token is found, then the current rule fails and rollback occurs. Mainly used to solve multiple contexts of a token. An Example of where not test is used to solve multiple contexts:

<rule> ::= <identifier> "::=" <expression>
<expression> ::= <and_term> { <or_term> }
<or_term> ::= "|" <and_term>
<and_term> ::= { }
::= <identifier_right> | <terminal_symbol> | <repeat_expression> | <optional_expression>
<identifier_right> ::= <identifier> (?!"::=")

<identiefier> appears on both sides of the ::= so (?!"::=") test to make sure that ::= is not on the right which would indicate that a new rule was being formed.

Works on both terminals and non-terminals. Note: lookahead failure causes the whole rule to fail and rollback to occur

<name> # indicates that a numerical value is to be parsed to form a terminal token. Name is optional and is just a descriptor to help with understanding what the value will be used for. Example: <Colour> ::= <red> <green> <blue>

() parentheses enclose a set of characters that can be used to generate a user identifier. for example: (0123456789) matches a single character found in that set. An example of a user identifier:

<Label> ::= <Character> {<Character>}
<Character> ::= (abcdefghijklmnopqrstuvwxyz)

This will generate a rule that accepts one or more lowercase letters to make up the Label. The User identifier stops collecting the characters into a string when a match cannot be found in the rule.

(! ) if the first character in the set is a ! then any input character not found in the set will be accepted. An example:

<Label> ::= <AnyCharacter_NoLineBreak> {<AnyCharacter_NoLineBreak>}
<AnyCharacter_NoLineBreak> ::= (!
)

any character but
or is accepted in the input.

: Insert the terminal token on the left before the next terminal token on the right if the next terminal token on right parses. Usefull for when terminal tokens don't have a definate text state but only context state based on another terminal or character token. An example:

<Last_Resort> ::= 'external_command' : <Special_Label>
<Special_Label> ::= (!
)

In the example, <Last_Resort> gets processed when all other rules fail to parse. if <Special_Label> parses (reads in any character but
) then the terminal token 'external_command' is inserted prior to the Special_Label for pass 2 processing. 'external_command' does not have have an explicit text representation but based on the context of no other rules matching and <Special_Label> parsing, 'external_command' is considered parsed.

Definition at line 151 of file OgreCompiler2Pass.h.

---

Member Typedef Documentation

typedef std::vector<TokenRule> Ogre::Compiler2Pass::TokenRuleContainer [protected]

Definition at line 173 of file OgreCompiler2Pass.h.

typedef TokenRuleContainer::iterator Ogre::Compiler2Pass::TokenRuleIterator [protected]

Definition at line 174 of file OgreCompiler2Pass.h.

typedef std::vector<LexemeTokenDef> Ogre::Compiler2Pass::LexemeTokenDefContainer [protected]

Definition at line 227 of file OgreCompiler2Pass.h.

typedef LexemeTokenDefContainer::iterator Ogre::Compiler2Pass::LexemeTokenDefIterator [protected]

Definition at line 228 of file OgreCompiler2Pass.h.

typedef std::map<String, size_t> Ogre::Compiler2Pass::LexemeTokenMap [protected]

Definition at line 230 of file OgreCompiler2Pass.h.

typedef LexemeTokenMap::iterator Ogre::Compiler2Pass::TokenKeyIterator [protected]

Definition at line 231 of file OgreCompiler2Pass.h.

typedef std::vector<TokenInst> Ogre::Compiler2Pass::TokenInstContainer [protected]

Definition at line 245 of file OgreCompiler2Pass.h.

typedef TokenInstContainer::iterator Ogre::Compiler2Pass::TokenInstIterator [protected]

Definition at line 246 of file OgreCompiler2Pass.h.

typedef std::map<size_t, String> Ogre::Compiler2Pass::LabelContainer [protected]

storage container for string labels defined in source container uses Token index as a key associated with a label

Definition at line 286 of file OgreCompiler2Pass.h.

typedef std::map<String, TokenState> Ogre::Compiler2Pass::TokenStateContainer [private]

Definition at line 606 of file OgreCompiler2Pass.h.

---

Member Enumeration Documentation

enum Ogre::Compiler2Pass::OperationType [protected]

Enumerator:

otUNKNOWN
otRULE
otAND
otOR
otOPTIONAL
otREPEAT
otDATA
otNOT_TEST
otINSERT_TOKEN
otEND

Definition at line 157 of file OgreCompiler2Pass.h.

enum Ogre::Compiler2Pass::SystemRuleToken [protected]

Enumerator:

_no_token_
_character_
_value_
_no_space_skip_

Definition at line 177 of file OgreCompiler2Pass.h.

enum Ogre::Compiler2Pass::BNF_ID [protected]

Enumerator:

BNF_UNKOWN
BNF_SYNTAX
BNF_RULE
BNF_IDENTIFIER
BNF_IDENTIFIER_RIGHT
BNF_IDENTIFIER_CHARACTERS
BNF_ID_BEGIN
BNF_ID_END
BNF_CONSTANT_BEGIN
BNF_SET_RULE
BNF_EXPRESSION
BNF_AND_TERM
BNF_OR_TERM
BNF_TERM
BNF_TERM_ID
BNF_CONSTANT
BNF_OR
BNF_TERMINAL_SYMBOL
BNF_TERMINAL_START
BNF_REPEAT_EXPRESSION
BNF_REPEAT_BEGIN
BNF_REPEAT_END
BNF_SET
BNF_SET_BEGIN
BNF_SET_END
BNF_NOT_TEST
BNF_NOT_TEST_BEGIN
BNF_CONDITIONAL_TOKEN_INSERT
BNF_OPTIONAL_EXPRESSION
BNF_NOT_EXPRESSION
BNF_NOT_CHK
BNF_OPTIONAL_BEGIN
BNF_OPTIONAL_END
BNF_NO_TOKEN_START
BNF_SINGLEQUOTE
BNF_SINGLE_QUOTE_EXC
BNF_SET_END_EXC
BNF_ANY_CHARACTER
BNF_SPECIAL_CHARACTERS1
BNF_SPECIAL_CHARACTERS2
BNF_WHITE_SPACE_CHK
BNF_LETTER
BNF_LETTER_DIGIT
BNF_DIGIT
BNF_WHITE_SPACE
BNF_ALPHA_SET
BNF_NUMBER_SET
BNF_SPECIAL_CHARACTER_SET1
BNF_SPECIAL_CHARACTER_SET2
BNF_SPECIAL_CHARACTER_SET3
BNF_NOT_CHARS
BNF_AUTOTOKENSTART

Definition at line 184 of file OgreCompiler2Pass.h.

---

Constructor & Destructor Documentation

Ogre::Compiler2Pass::Compiler2Pass

(

)

constructor

virtual Ogre::Compiler2Pass::~Compiler2Pass

(

)

[virtual]

Definition at line 639 of file OgreCompiler2Pass.h.

---

Member Function Documentation

bool Ogre::Compiler2Pass::doPass1

(

)

[protected]

perform pass 1 of compile process scans source for lexemes that can be tokenized and then performs general semantic and context verification on each lexeme before it is tokenized.

A tokenized instruction list is built to be used by Pass 2. A rule path can trigger Pass 2 execution if enough tokens have been generated in Pass 1. Pass 1 will then pass control to pass 2 temporarily until the current tokens have been consumed.

bool Ogre::Compiler2Pass::doPass2

(

)

[protected]

performs Pass 2 of compile process which is execution of the tokens

Remarks:

Pass 2 takes the token instructions generated in Pass 1 and builds the application specific instructions along with verifying semantic and context rules that could not be checked in Pass 1.

Pass 2 execution consumes tokens and moves the Pass 2 token instruction position towards the end of the token container. Token execution can insert new tokens into the token container.

virtual void Ogre::Compiler2Pass::executeTokenAction

(

const size_t

tokenID

)

[protected, pure virtual]

execute the action associated with the token pointed to by the Pass 2 token instruction position.

Remarks:

It's up to the child class to implement how it will associate a token key with and action. Actions should be placed at positions withing the BNF grammar (instruction que) that indicate enough tokens exist for pass 2 processing to take place.

Implemented in Ogre::CompositorScriptCompiler.

virtual size_t Ogre::Compiler2Pass::getAutoTokenIDStart

(

)

const [protected, pure virtual]

Get the start ID for auto generated token IDs.

This is also one pass the end of manually set token IDs. Manually set Token ID are usually setup in the client code through an enum type so its best to make the last entry the auto ID start position and return this enum value. This method gets called automatically just prior to setupTokenDefinitions() to ensure that any tokens that are auto generated are placed after the manually set ones.

Implemented in Ogre::CompositorScriptCompiler.

virtual void Ogre::Compiler2Pass::setupTokenDefinitions

(

void

)

[protected, pure virtual]

setup client token definitions.

Gets called when BNF grammar is being setup.

Implemented in Ogre::CompositorScriptCompiler.

const TokenInst& Ogre::Compiler2Pass::getNextToken

(

const size_t

expectedTokenID = 0

)

const [protected]

Gets the next token from the instruction que.

Remarks:

If an unknown token is found then an exception is raised but the instruction pointer is still moved passed the unknown token. The subclass should catch the exception, provide an error message, and attempt recovery.

Parameters:

expectedTokenID

if greater than 0 then an exception is raised if tokenID does not match.

Definition at line 355 of file OgreCompiler2Pass.h.

const TokenInst& Ogre::Compiler2Pass::getCurrentToken

(

const size_t

expectedTokenID = 0

)

const [protected]

Gets the current token from the instruction que.

Remarks:

If an unknown token is found then an exception is raised. The subclass should catch the exception, provide an error message, and attempt recovery.

Parameters:

expectedTokenID

if greater than 0 then an exception is raised if tokenID does not match.

bool Ogre::Compiler2Pass::testNextTokenID

(

const size_t

expectedTokenID

)

const [protected]

If a next token instruction exist then test if its token ID matches.

Remarks:

This method is usefull for peeking ahead during pass 2 to see if a certain token exists. If the tokens don't match or there is no next token (end of que) then false is returned.

Parameters:

expectedTokenID

is the ID of the token to match.

bool Ogre::Compiler2Pass::testCurrentTokenID

(

const size_t

expectedTokenID

)

const [protected]

If a current token instruction exist then test if its token ID matches.

Parameters:

expectedTokenID

is the ID of the token to match.

Definition at line 381 of file OgreCompiler2Pass.h.

void Ogre::Compiler2Pass::skipToken

(

void

)

const [protected]

skip to the next token in the pass2 queue.

void Ogre::Compiler2Pass::replaceToken

(

void

)

[protected]

go back to the previous token in the pass2 queue.

float Ogre::Compiler2Pass::getNextTokenValue

(

void

)

const [protected]

Gets the next token's associated floating point value in the instruction que that was parsed from the text source.

If an unknown token is found or no associated value was found then an exception is raised but the instruction pointer is still moved passed the unknown token. The subclass should catch the exception, provide an error message, and attempt recovery.

Definition at line 396 of file OgreCompiler2Pass.h.

float Ogre::Compiler2Pass::getCurrentTokenValue

(

void

)

const [protected]

Gets the current token's associated floating point value in the instruction que that was parsed from the text source.

Remarks:

If an unknown token is found or no associated value was found then an exception is raised. The subclass should catch the exception, provide an error message, and attempt recovery.

const String& Ogre::Compiler2Pass::getNextTokenLabel

(

void

)

const [protected]

Gets the next token's associated text label in the instruction que that was parsed from the text source.

Remarks:

If an unknown token is found or no associated label was found then an exception is raised but the instruction pointer is still moved passed the unknown token. The subclass should catch the exception, provide an error message, and attempt recovery.

Definition at line 415 of file OgreCompiler2Pass.h.

const String& Ogre::Compiler2Pass::getCurrentTokenLabel

(

void

)

const [protected]

Gets the next token's associated text label in the instruction que that was parsed from the text source.

If an unknown token is found or no associated label was found then an exception is raised. The subclass should catch the exception, provide an error message, and attempt recovery.

size_t Ogre::Compiler2Pass::getNextTokenID

(

void

)

const [protected]

Get the next token's ID value.

Definition at line 427 of file OgreCompiler2Pass.h.

size_t Ogre::Compiler2Pass::getCurrentTokenID

(

void

)

const [protected]

Get the current token's ID value.

Definition at line 430 of file OgreCompiler2Pass.h.

const String& Ogre::Compiler2Pass::getNextTokenLexeme

(

void

)

const [protected]

Get the next token's lexeme string.

Handy when you don't want the ID but want the string representation.

Definition at line 434 of file OgreCompiler2Pass.h.

const String& Ogre::Compiler2Pass::getCurrentTokenLexeme

(

void

)

const [protected]

Get the current token's lexeme string.

Handy when you don't want the ID but want the string representation.

size_t Ogre::Compiler2Pass::getPass2TokenQueCount

(

void

)

const [protected]

Gets the number of tokens waiting in the instruction que that need to be processed by an token action in pass 2.

size_t Ogre::Compiler2Pass::getRemainingTokensForAction

(

void

)

const [protected]

Get the number of tokens not processed by action token.

Client Actions should use this method to retrieve the number of parameters(tokens) remaining to be processed in the action.

void Ogre::Compiler2Pass::setPass2TokenQuePosition	(	size_t	pos,
		const bool	activateAction = false
	)			[protected]

Manually set the Pass2 Token que position.

Remarks:

This method will also set the position of the next token in the pass2 que that has an action ensuring that getRemainingTokensForAction works correctly. This method is useful for when the token que must be reprocessed after pass1 and the position in the que must be changed so that an action will be triggered.

Parameters:

	pos	is the new position within the Pass2 que
	activateAction	if set true and the token at the new position has an action then the action is activated.

size_t Ogre::Compiler2Pass::getPass2TokenQuePosition

(

void

)

const [protected]

Get the current position in the Pass2 Token Que.

Definition at line 464 of file OgreCompiler2Pass.h.

bool Ogre::Compiler2Pass::setNextActionQuePosition	(	size_t	pos,
		const bool	search = false
	)			[protected]

Set the position of the next token action in the Pass2 Token Que.

Remarks:

If the position is not within the que or there is no action associated with the token at the position in the que then NextActionQuePosition is not set.

Parameters:

	pos	is the position in the Pass2 Token Que where the next action is.
	search	if true then the que is searched from pos until an action is found. If the end of the que is reached and no action has been found then NextActionQuePosition is set to the end of the que and false is returned.

size_t Ogre::Compiler2Pass::addLexemeToken	(	const String &	lexeme,
		const size_t	token,
		const bool	hasAction = false,
		const bool	caseSensitive = false
	)			[protected]

Add a lexeme token association.

Remarks:

The backend compiler uses the associations between lexeme and token when building the rule base from the BNF script so all associations must be done prior to compiling a source.

Parameters:

	lexeme	is the name of the token and use when parsing the source to determine a match for a token.
	token	is the ID associated with the lexeme. If token is 0 then the token ID is auto generated and returned.
	hasAction	must be set true if the client wants an action triggered when this token is generated
	caseSensitive	should be set true if lexeme match should use case sensitivity

Returns:

the ID of the token. Useful when auto generating token IDs.

void Ogre::Compiler2Pass::setClientBNFGrammer

(

void

)

[protected]

Sets up the parser rules for the client based on the BNF Grammar text passed in.

Remarks:

Raises an exception if the grammar did not compile successfully. This method gets called when a call to compile occurs and no compiled BNF grammar exists, otherwise nothing will happen since the compiler has no rules to work with. The grammar only needs to be set once during the lifetime of the compiler unless the grammar changes.

Note:

BNF Grammar rules are cached once the BNF grammar source is compiled. The client should never have to call this method directly.

void Ogre::Compiler2Pass::findEOL

(

)

[protected]

find the eol character

bool Ogre::Compiler2Pass::isFloatValue	(	float &	fvalue,
		size_t &	charsize
	)			const [protected]

check to see if the text at the present position in the source is a n