Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.modeshape.common.text
Interface TokenStream.Tokens

All Known Implementing Classes:
  TokenStream.CaseInsensitiveTokenFactory, TokenStream.CaseSensitiveTokenFactory, TokenStream.TokenFactory

Enclosing class:
  TokenStream

    public static interface TokenStream.Tokens

A factory for Token objects, used by a `TokenStream.Tokenizer` to create tokens in the correct order.

Method Summary
`void` | `addToken(Position position, int index)` Create a single-character token at the supplied index in the character stream.
`void` | `addToken(Position position, int startIndex, int endIndex)` Create a single- or multi-character token with the characters in the range given by the starting and ending index in the character stream.
`void` | `addToken(Position position, int startIndex, int endIndex, int type)` Create a single- or multi-character token with the supplied type and with the characters in the range given by the starting and ending index in the character stream.

Method Detail

### addToken

    void addToken(Position position,
                  int index)

  Create a single-character token at the supplied index in the character stream. The token type is set to 0, meaning this is equivalent to calling `addToken(index,index+1)` or `addToken(index,index+1,0)`.

  Parameters:
    `position` - the position (line and column numbers) of this new token; may not be null
    `index` - the index of the character to appear in the token; must be a valid index in the stream

### addToken

    void addToken(Position position,
                  int startIndex,
                  int endIndex)

  Create a single- or multi-character token with the characters in the range given by the starting and ending index in the character stream. The character at the ending index is not included in the token (as this is standard practice when using 0-based indexes). The token type is set to 0, meaning this is equivalent to calling `addToken(startIndex,endIndex,0)` .

  Parameters:
    `position` - the position (line and column numbers) of this new token; may not be null
    `startIndex` - the index of the first character to appear in the token; must be a valid index in the stream
    `endIndex` - the index just past the last character to appear in the token; must be a valid index in the stream

### addToken

    void addToken(Position position,
                  int startIndex,
                  int endIndex,
                  int type)

  Create a single- or multi-character token with the supplied type and with the characters in the range given by the starting and ending index in the character stream. The character at the ending index is not included in the token (as this is standard practice when using 0-based indexes).

  Parameters:
    `position` - the position (line and column numbers) of this new token; may not be null
    `startIndex` - the index of the first character to appear in the token; must be a valid index in the stream
    `endIndex` - the index just past the last character to appear in the token; must be a valid index in the stream
    `type` - the type of the token

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2008-2011 JBoss, a division of Red Hat. All Rights Reserved.