org.modeshape.common.text

Class TokenStream.TokenFactory

java.lang.Object

org.modeshape.common.text.TokenStream.TokenFactory

All Implemented Interfaces:

TokenStream.Tokens

Direct Known Subclasses:

TokenStream.CaseInsensitiveTokenFactory, TokenStream.CaseSensitiveTokenFactory

Enclosing class:

TokenStream

protected abstract class TokenStream.TokenFactory
extends Object
implements TokenStream.Tokens

Field Summary

Fields

Modifier and Type	Field and Description
protected List<TokenStream.Token>	tokens

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	TokenStream.TokenFactory()

Method Summary

Methods

Modifier and Type	Method and Description
void	addToken(Position position, int index) Create a single-character token at the supplied index in the character stream.
void	addToken(Position position, int startIndex, int endIndex) Create a single- or multi-character token with the characters in the range given by the starting and ending index in the character stream.
List<TokenStream.Token>	getTokens()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.modeshape.common.text.TokenStream.Tokens

addToken

Field Detail

tokens

protected final List<TokenStream.Token> tokens

Constructor Detail

TokenStream.TokenFactory

protected TokenStream.TokenFactory()

Method Detail

addToken

public void addToken(Position position,
            int index)

Create a single-character token at the supplied index in the character stream. The token type is set to 0, meaning this is equivalent to calling addToken(index,index+1) or addToken(index,index+1,0).

Specified by:

addToken in interface TokenStream.Tokens

Parameters:

position - the position (line and column numbers) of this new token; may not be null

index - the index of the character to appear in the token; must be a valid index in the stream

See Also:

TokenStream.Tokens.addToken(Position, int)

addToken

public final void addToken(Position position,
            int startIndex,
            int endIndex)

Create a single- or multi-character token with the characters in the range given by the starting and ending index in the character stream. The character at the ending index is not included in the token (as this is standard practice when using 0-based indexes). The token type is set to 0, meaning this is equivalent to calling addToken(startIndex,endIndex,0) .

Specified by:

addToken in interface TokenStream.Tokens

Parameters:

position - the position (line and column numbers) of this new token; may not be null

startIndex - the index of the first character to appear in the token; must be a valid index in the stream

endIndex - the index just past the last character to appear in the token; must be a valid index in the stream

See Also:

TokenStream.Tokens.addToken(Position, int, int)

getTokens

public List<TokenStream.Token> getTokens()

Returns:

tokens