## Uses of Interface
org.modeshape.common.text.TokenStream.Tokenizer

- Packages that use TokenStream.Tokenizer
  Package | Description
  --- | ---
  org.modeshape.common.text | A set of utilities for working with text.
  org.modeshape.jcr |
  org.modeshape.jcr.query.parse | This package defines the `QueryParser` interface, which defines a component that can parse a query represented in a specific language and produce the corresponding `abstract query model` representation.
  org.modeshape.jcr.query.xpath | The components used to parse JCR XPath queries and translate them into equivalent JCR-SQL2 queries.
  org.modeshape.sequencer.ddl | The classes that make up the DDL sequencer, which is capable of parsing the more important DDL statements from SQL-92, Oracle, Derby, and PostgreSQL, and constructing a graph structure containing a structured representation of these statements.

- - ### Uses of TokenStream.Tokenizer in org.modeshape.common.text

    Classes in org.modeshape.common.text that implement TokenStream.Tokenizer
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `TokenStream.BasicTokenizer` A basic `TokenStream.Tokenizer` implementation that ignores whitespace but includes tokens for individual symbols, the period ('.'), single-quoted strings, double-quoted strings, whitespace-delimited words, and optionally comments.

    Constructors in org.modeshape.common.text with parameters of type TokenStream.Tokenizer
    Constructor and Description
    `TokenStream(String content, TokenStream.Tokenizer tokenizer, boolean caseSensitive)`

  - ### Uses of TokenStream.Tokenizer in org.modeshape.jcr

    Classes in org.modeshape.jcr that implement TokenStream.Tokenizer
    Modifier and Type | Class and Description
    --- | ---
    `class` | `CndTokenizer` A `TokenStream.Tokenizer` implementation that adheres to the CND format by ignoring whitespace while including tokens for individual symbols, the period ('.'), single-quoted strings, double-quoted strings, whitespace-delimited words, and optionally comments.

  - ### Uses of TokenStream.Tokenizer in org.modeshape.jcr.query.parse

    Classes in org.modeshape.jcr.query.parse that implement TokenStream.Tokenizer
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `BasicSqlQueryParser.SqlTokenizer` A `TokenStream.Tokenizer` implementation that parses words, quoted phrases, comments, and symbols.
    `static class` | `FullTextSearchParser.TermTokenizer` A basic `TokenStream.Tokenizer` implementation that ignores whitespace but includes tokens for individual symbols, the period ('.'), single-quoted strings, double-quoted strings, whitespace-delimited words, and optionally comments.

  - ### Uses of TokenStream.Tokenizer in org.modeshape.jcr.query.xpath

    Classes in org.modeshape.jcr.query.xpath that implement TokenStream.Tokenizer
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `XPathParser.XPathTokenizer` A `TokenStream.Tokenizer` implementation that parses single- and double-quoted strings, symbols, words consisting of `NCName`s (as defined by the Namespaces in XML 1.0 specification), XPath comments,and `other` single-character tokens.

  - ### Uses of TokenStream.Tokenizer in org.modeshape.sequencer.ddl

    Classes in org.modeshape.sequencer.ddl that implement TokenStream.Tokenizer
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `DdlTokenStream.DdlTokenizer`

    Constructors in org.modeshape.sequencer.ddl with parameters of type TokenStream.Tokenizer
    Constructor and Description
    `DdlTokenStream(String content, TokenStream.Tokenizer tokenizer, boolean caseSensitive)`

Copyright © 2008-2014 JBoss, a division of Red Hat. All Rights Reserved.