org.openkoreantext.processor

Class OpenKoreanTextProcessorJava

java.lang.Object

org.openkoreantext.processor.OpenKoreanTextProcessorJava

public final class OpenKoreanTextProcessorJava
extends Object

Java wrapper for TwitterKoreanProcessor using Builder pattern

Constructor Summary

Constructors

Constructor and Description
OpenKoreanTextProcessorJava()

Method Summary

Modifier and Type	Method and Description
static void	addNounsToDictionary(List<String> words) Add user-defined words to the noun dictionary.
static void	addWordsToDictionary(KoreanPosJava pos, List<String> words) Add user-defined word List to the dictionary for the specified KoreanPos.
static String	detokenize(List<String> tokens) Detokenize the input list of words.
static List<KoreanPhraseExtractor.KoreanPhrase>	extractPhrases(scala.collection.Seq<KoreanToken> tokens, boolean filterSpam, boolean includeHashtags) Extract phrases from Korean input text
static void	loadResources() Load the dictionary resources into memory.
static CharSequence	normalize(CharSequence text) Normalize Korean text 그랰ㅋㅋㅋㅋㅋㅋ -> 그래ㅋㅋ
static void	removeWordFromDictionary(KoreanPosJava pos, List<String> words) Remove user-defined word List from the dictionary for the specified KoreanPos.
static List<Sentence>	splitSentences(CharSequence text) Split input text into sentences.
static scala.collection.Seq<KoreanToken>	tokenize(CharSequence text) Tokenize with the builder options.
static List<KoreanTokenJava>	tokensToJavaKoreanTokenList(scala.collection.Seq<KoreanToken> tokens)
static List<KoreanTokenJava>	tokensToJavaKoreanTokenList(scala.collection.Seq<KoreanToken> tokens, boolean keepSpace) Transforms the tokenization output to List
static List<String>	tokensToJavaStringList(scala.collection.Seq<KoreanToken> tokens)
static List<String>	tokensToJavaStringList(scala.collection.Seq<KoreanToken> tokens, boolean keepSpace) Tokenize with the builder options into a String Iterable.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OpenKoreanTextProcessorJava

public OpenKoreanTextProcessorJava()

Method Detail

normalize

public static CharSequence normalize(CharSequence text)

Normalize Korean text 그랰ㅋㅋㅋㅋㅋㅋ -> 그래ㅋㅋ

Parameters:

text - Input text.

Returns:

Normalized text.

tokenize

public static scala.collection.Seq<KoreanToken> tokenize(CharSequence text)

Tokenize with the builder options.

Parameters:

text - Input text.

Returns:

A list of Korean Tokens (run tokensToJavaList to transform to Java List)

addNounsToDictionary

public static void addNounsToDictionary(List<String> words)

Add user-defined words to the noun dictionary. Spaced words are ignored.

Parameters:

words - List of user nouns.

addWordsToDictionary

public static void addWordsToDictionary(KoreanPosJava pos,
                                        List<String> words)

Add user-defined word List to the dictionary for the specified KoreanPos.

Parameters:

pos - KoreanPos of words to add.

words - Sequence of words to add.

removeWordFromDictionary

public static void removeWordFromDictionary(KoreanPosJava pos,
                                            List<String> words)

Remove user-defined word List from the dictionary for the specified KoreanPos.

Parameters:

pos - KoreanPos of words to add.

words - Sequence of words to add.

tokensToJavaKoreanTokenList

public static List<KoreanTokenJava> tokensToJavaKoreanTokenList(scala.collection.Seq<KoreanToken> tokens,
                                                                boolean keepSpace)

Transforms the tokenization output to List

Parameters:

tokens - Korean tokens (output of tokenize(CharSequence text)).

Returns:

List of KoreanTokenJava.

tokensToJavaKoreanTokenList

public static List<KoreanTokenJava> tokensToJavaKoreanTokenList(scala.collection.Seq<KoreanToken> tokens)

tokensToJavaStringList

public static List<String> tokensToJavaStringList(scala.collection.Seq<KoreanToken> tokens,
                                                  boolean keepSpace)

Tokenize with the builder options into a String Iterable.

Parameters:

tokens - Korean tokens (output of tokenize(CharSequence text)).

Returns:

List of token strings.

tokensToJavaStringList

public static List<String> tokensToJavaStringList(scala.collection.Seq<KoreanToken> tokens)

splitSentences

public static List<Sentence> splitSentences(CharSequence text)

Split input text into sentences.

Parameters:

text - Input text.

Returns:

List of Sentence objects.

extractPhrases

public static List<KoreanPhraseExtractor.KoreanPhrase> extractPhrases(scala.collection.Seq<KoreanToken> tokens,
                                                                      boolean filterSpam,
                                                                      boolean includeHashtags)

Extract phrases from Korean input text

Parameters:

tokens - Korean tokens (output of tokenize(CharSequence text)).

Returns:

List of phrase CharSequences.

detokenize

public static String detokenize(List<String> tokens)

Detokenize the input list of words.

Parameters:

tokens - List of words.

Returns:

Detokenized string.

loadResources

public static void loadResources()

Load the dictionary resources into memory.

Dictionary resources are loaded lazily by most API methods, however in some instances it is useful to be able to trigger this loading manually.