## Uses of Class
org.modeshape.common.text.TokenStream.BasicTokenizer

- Packages that use TokenStream.BasicTokenizer
  Package | Description
  --- | ---
  org.modeshape.common.text | A set of utilities for working with text.

- - ### Uses of TokenStream.BasicTokenizer in org.modeshape.common.text

    Methods in org.modeshape.common.text that return TokenStream.BasicTokenizer
    Modifier and Type | Method and Description
    --- | ---
    `static TokenStream.BasicTokenizer` | TokenStream.`basicTokenizer(boolean includeComments)` Obtain a basic `TokenStream.Tokenizer` implementation that ignores whitespace but includes tokens for individual symbols, the period ('.'), single-quoted strings, double-quoted strings, whitespace-delimited words, and optionally comments.

Copyright © 2008-2014 JBoss, a division of Red Hat. All Rights Reserved.