org.modeshape.sequencer.ddl

## Class DdlTokenStream.DdlTokenizer

- java.lang.Object
- - org.modeshape.sequencer.ddl.DdlTokenStream.DdlTokenizer

- All Implemented Interfaces:
    TokenStream.Tokenizer
  Enclosing class:
    DdlTokenStream

      public static class DdlTokenStream.DdlTokenizer
      extends Object
      implements TokenStream.Tokenizer

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static int` | `COMMENT` The `token type` for tokens that consist of all the characters between "/*" and "*/", between "//" and the next line terminator (e.g., '\n', '\r' or "\r\n"), or between "--" and the next line terminator (e.g., '\n', '\r' or "\r\n").
    `static int` | `DECIMAL` The `token type` for tokens that consist of an individual '.'
    `static int` | `DOUBLE_QUOTED_STRING` The `token type` for tokens that consist of all the characters within double-quotes.
    `static int` | `KEYWORD` The `token type` for tokens that represent key words or reserved words for a given DDL dialect.
    `static String` | `PARSER_ID`
    `static int` | `SINGLE_QUOTED_STRING` The `token type` for tokens that consist of all the characters within single-quotes.
    `static int` | `STATEMENT_KEY` The `token type` for tokens that represent the start of a DDL statement.
    `static int` | `SYMBOL` The `token type` for tokens that consist of an individual "symbol" character.
    `static int` | `WORD` The `token type` for tokens that represent an unquoted string containing a character sequence made up of non-whitespace and non-symbol characters.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `DdlTokenStream.DdlTokenizer(boolean useComments)`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `boolean` | `includeComments()`
    `void` | `tokenize(TokenStream.CharacterStream input, TokenStream.Tokens tokens)` Process the supplied characters and construct the appropriate `TokenStream.Token` objects.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### PARSER_ID

          public static final String PARSER_ID

      See Also:
        Constant Field Values

    - #### WORD

          public static final int WORD

      The `token type` for tokens that represent an unquoted string containing a character sequence made up of non-whitespace and non-symbol characters.
      See Also:
        Constant Field Values

    - #### SYMBOL

          public static final int SYMBOL

      The `token type` for tokens that consist of an individual "symbol" character. The set of characters includes: `-(){}*,;+%?$[]!<>|=:`
      See Also:
        Constant Field Values

    - #### DECIMAL

          public static final int DECIMAL

      The `token type` for tokens that consist of an individual '.' character.
      See Also:
        Constant Field Values

    - #### SINGLE_QUOTED_STRING

          public static final int SINGLE_QUOTED_STRING

      The `token type` for tokens that consist of all the characters within single-quotes. Single quote characters are included if they are preceded (escaped) by a '\' character.
      See Also:
        Constant Field Values

    - #### DOUBLE_QUOTED_STRING

          public static final int DOUBLE_QUOTED_STRING

      The `token type` for tokens that consist of all the characters within double-quotes. Double quote characters are included if they are preceded (escaped) by a '\' character.
      See Also:
        Constant Field Values

    - #### COMMENT

          public static final int COMMENT

      The `token type` for tokens that consist of all the characters between "/*" and "*/", between "//" and the next line terminator (e.g., '\n', '\r' or "\r\n"), or between "--" and the next line terminator (e.g., '\n', '\r' or "\r\n").
      See Also:
        Constant Field Values

    - #### KEYWORD

          public static final int KEYWORD

      The `token type` for tokens that represent key words or reserved words for a given DDL dialect.

      Examples would be: "CREATE", "TABLE", "ALTER", "SCHEMA", "DROP", etc...

      see `DdlConstants` for the default SQL 92 representations.
      See Also:
        Constant Field Values

    - #### STATEMENT_KEY

          public static final int STATEMENT_KEY

      The `token type` for tokens that represent the start of a DDL statement.

      Examples would be: {"CREATE", "TABLE"} {"CREATE", "OR", "REPLACE", "VIEW"}

      see `DdlConstants` for the default SQL 92 representations.
      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### DdlTokenStream.DdlTokenizer

          public DdlTokenStream.DdlTokenizer(boolean useComments)

  - ### Method Detail

    - #### includeComments

          public boolean includeComments()

      Returns:
        useComments

    - #### tokenize

          public void tokenize(TokenStream.CharacterStream input,
                      TokenStream.Tokens tokens)
                        throws ParsingException

      Process the supplied characters and construct the appropriate `TokenStream.Token` objects.
      Specified by:
        `tokenize` in interface `TokenStream.Tokenizer`
      Parameters:
        `input` - the character input stream; never null
        `tokens` - the factory for `TokenStream.Token` objects, which records the order in which the tokens are created
      Throws:
        `ParsingException` - if there is an error while processing the character stream (e.g., a quote is not closed, etc.)
      See Also:
        `org.modeshape.common.text.TokenStream.Tokenizer#tokenize(CharacterStream, Tokens)`

Copyright © 2008-2014 JBoss, a division of Red Hat. All Rights Reserved.