Module org.apache.lucene.analysis.common

Package org.apache.lucene.analysis.wikipedia

Class WikipediaTokenizerFactory

java.lang.Object

org.apache.lucene.analysis.AbstractAnalysisFactory

org.apache.lucene.analysis.TokenizerFactory

org.apache.lucene.analysis.wikipedia.WikipediaTokenizerFactory

public class WikipediaTokenizerFactory
extends TokenizerFactory

Factory for WikipediaTokenizer.

 <fieldType name="text_wiki" class="solr.TextField" positionIncrementGap="100">
   <analyzer>
     <tokenizer class="solr.WikipediaTokenizerFactory"/>
   </analyzer>
 </fieldType>

Since:

3.1

Field Summary

Fields

Modifier and Type	Field	Description
static final String	NAME	SPI name
static final String	TOKEN_OUTPUT
protected final int	tokenOutput
static final String	UNTOKENIZED_TYPES
protected Set<String>	untokenizedTypes

Fields inherited from class org.apache.lucene.analysis.AbstractAnalysisFactory

LUCENE_MATCH_VERSION_PARAM, luceneMatchVersion

Constructor Summary

Constructors

Constructor	Description
WikipediaTokenizerFactory()	Default ctor for compatibility with SPI
WikipediaTokenizerFactory(Map<String,String> args)	Creates a new WikipediaTokenizerFactory

Method Summary

Modifier and Type	Method	Description
WikipediaTokenizer	create(AttributeFactory factory)	Creates a TokenStream of the specified input using the given AttributeFactory

Methods inherited from class org.apache.lucene.analysis.TokenizerFactory

availableTokenizers, create, findSPIName, forName, lookupClass, reloadTokenizers

Methods inherited from class org.apache.lucene.analysis.AbstractAnalysisFactory

defaultCtorException, get, get, get, get, get, getBoolean, getChar, getClassArg, getFloat, getInt, getLines, getLuceneMatchVersion, getOriginalArgs, getPattern, getSet, getSnowballWordSet, getWordSet, isExplicitLuceneMatchVersion, require, require, require, requireBoolean, requireChar, requireFloat, requireInt, setExplicitLuceneMatchVersion, splitAt, splitFileNames

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

NAME

public static final String NAME

SPI name

See Also:

• Constant Field Values

TOKEN_OUTPUT

public static final String TOKEN_OUTPUT

See Also:

• Constant Field Values

UNTOKENIZED_TYPES

public static final String UNTOKENIZED_TYPES

See Also:

• Constant Field Values

tokenOutput

protected final int tokenOutput

untokenizedTypes

protected Set<String> untokenizedTypes

Constructor Details

WikipediaTokenizerFactory

public WikipediaTokenizerFactory(Map<String,String> args)

Creates a new WikipediaTokenizerFactory

WikipediaTokenizerFactory

public WikipediaTokenizerFactory()

Default ctor for compatibility with SPI

Method Details

create

public WikipediaTokenizer create(AttributeFactory factory)

Description copied from class: TokenizerFactory

Creates a TokenStream of the specified input using the given AttributeFactory

Specified by:

create in class TokenizerFactory