Uses of Class
org.apache.lucene.analysis.TokenStream

Packages that use TokenStream

Package	Description
org.apache.lucene.analysis	Text analysis.
org.apache.lucene.analysis.ar	Analyzer for Arabic.
org.apache.lucene.analysis.bg	Analyzer for Bulgarian.
org.apache.lucene.analysis.bn	Analyzer for Bengali Language.
org.apache.lucene.analysis.boost	Provides various convenience classes for creating boosts on Tokens.
org.apache.lucene.analysis.br	Analyzer for Brazilian Portuguese.
org.apache.lucene.analysis.ca	Analyzer for Catalan.
org.apache.lucene.analysis.cjk	Analyzer for Chinese, Japanese, and Korean, which indexes bigrams.
org.apache.lucene.analysis.ckb	Analyzer for Sorani Kurdish.
org.apache.lucene.analysis.classic	Fast, general-purpose grammar-based tokenizers.
org.apache.lucene.analysis.cn.smart	Analyzer for Simplified Chinese, which indexes words.
org.apache.lucene.analysis.commongrams	Construct n-grams for frequently occurring terms and phrases.
org.apache.lucene.analysis.compound	A filter that decomposes compound words you find in many Germanic languages into the word parts.
org.apache.lucene.analysis.core	Basic, general-purpose analysis components.
org.apache.lucene.analysis.custom	A general-purpose Analyzer that can be created with a builder-style API.
org.apache.lucene.analysis.cz	Analyzer for Czech.
org.apache.lucene.analysis.da	Analyzer for Danish.
org.apache.lucene.analysis.de	Analyzer for German.
org.apache.lucene.analysis.el	Analyzer for Greek.
org.apache.lucene.analysis.email	Fast, general-purpose URLs and email addresses tokenizers.
org.apache.lucene.analysis.en	Analyzer for English.
org.apache.lucene.analysis.es	Analyzer for Spanish.
org.apache.lucene.analysis.et	Analyzer for Estonian.
org.apache.lucene.analysis.eu	Analyzer for Basque.
org.apache.lucene.analysis.fa	Analyzer for Persian.
org.apache.lucene.analysis.fi	Analyzer for Finnish.
org.apache.lucene.analysis.fr	Analyzer for French.
org.apache.lucene.analysis.ga	Analyzer for Irish.
org.apache.lucene.analysis.gl	Analyzer for Galician.
org.apache.lucene.analysis.hi	Analyzer for Hindi.
org.apache.lucene.analysis.hu	Analyzer for Hungarian.
org.apache.lucene.analysis.hunspell	A Java implementation of Hunspell stemming and spell-checking algorithms (Hunspell), and a stemming TokenFilter (HunspellStemFilter) based on it.
org.apache.lucene.analysis.hy	Analyzer for Armenian.
org.apache.lucene.analysis.icu	Analysis components based on ICU
org.apache.lucene.analysis.icu.segmentation	Tokenizer that breaks text into words with the Unicode Text Segmentation algorithm.
org.apache.lucene.analysis.id	Analyzer for Indonesian.
org.apache.lucene.analysis.in	Analyzer for Indian languages.
org.apache.lucene.analysis.it	Analyzer for Italian.
org.apache.lucene.analysis.ja	Analyzer for Japanese.
org.apache.lucene.analysis.ko	Analyzer for Korean.
org.apache.lucene.analysis.lt	Analyzer for Lithuanian.
org.apache.lucene.analysis.lv	Analyzer for Latvian.
org.apache.lucene.analysis.minhash	MinHash filtering (for LSH).
org.apache.lucene.analysis.miscellaneous	Miscellaneous Tokenstreams.
org.apache.lucene.analysis.ne	Analyzer for Nepali.
org.apache.lucene.analysis.ngram	Character n-gram tokenizers and filters.
org.apache.lucene.analysis.nl	Analyzer for Dutch.
org.apache.lucene.analysis.no	Analyzer for Norwegian.
org.apache.lucene.analysis.path	Analysis components for path-like strings such as filenames.
org.apache.lucene.analysis.pattern	Set of components for pattern-based (regex) analysis.
org.apache.lucene.analysis.payloads	Provides various convenience classes for creating payloads on Tokens.
org.apache.lucene.analysis.phonetic	Analysis components for phonetic search.
org.apache.lucene.analysis.pl	Analyzer for Polish.
org.apache.lucene.analysis.pt	Analyzer for Portuguese.
org.apache.lucene.analysis.reverse	Filter to reverse token text.
org.apache.lucene.analysis.ro	Analyzer for Romanian.
org.apache.lucene.analysis.ru	Analyzer for Russian.
org.apache.lucene.analysis.shingle	Word n-gram filters.
org.apache.lucene.analysis.sinks	TeeSinkTokenFilter.
org.apache.lucene.analysis.snowball	TokenFilter and Analyzer implementations that use a modified version of Snowball stemmers.
org.apache.lucene.analysis.sr	Analyzer for Serbian.
org.apache.lucene.analysis.standard	Fast, general-purpose grammar-based tokenizer StandardTokenizer implements the Word Break rules from the Unicode Text Segmentation algorithm, as specified in Unicode Standard Annex #29.
org.apache.lucene.analysis.stempel	Stempel: Algorithmic Stemmer
org.apache.lucene.analysis.sv	Analyzer for Swedish.
org.apache.lucene.analysis.synonym	Analysis components for Synonyms.
org.apache.lucene.analysis.synonym.word2vec	Analysis components for Synonyms using Word2Vec model.
org.apache.lucene.analysis.ta	Analyzer for Tamil.
org.apache.lucene.analysis.te	Analyzer for Telugu Language.
org.apache.lucene.analysis.th	Analyzer for Thai.
org.apache.lucene.analysis.tr	Analyzer for Turkish.
org.apache.lucene.analysis.util	Utility functions for text analysis.
org.apache.lucene.analysis.wikipedia	Tokenizer that is aware of Wikipedia syntax.
org.apache.lucene.classification.document	Uses already seen data (the indexed documents) to classify new documents.
org.apache.lucene.document	The logical representation of a Document for indexing and searching.
org.apache.lucene.index	Code to maintain and access indices.
org.apache.lucene.index.memory	High-performance single-document main memory Apache Lucene fulltext search index.
org.apache.lucene.misc.document	Misc extensions of the Document/Field API.
org.apache.lucene.monitor	Monitoring framework
org.apache.lucene.queries.intervals	Intervals queries
org.apache.lucene.sandbox.search	This package contains a flexible graph-based proximity query, TermAutomatonQuery, and geospatial queries.
org.apache.lucene.search.highlight	Highlighting search terms.
org.apache.lucene.search.suggest.analyzing	Analyzer based autosuggest.
org.apache.lucene.search.suggest.document	Support for document suggestion
org.apache.lucene.search.uhighlight	The UnifiedHighlighter -- a flexible highlighter that can get offsets from postings, term vectors, or analysis.
org.apache.lucene.util	Some utility classes.
org.apache.lucene.util.graph	Utility classes for working with token streams as graphs.

Uses of TokenStream in org.apache.lucene.analysis

Subclasses of TokenStream in org.apache.lucene.analysis

Modifier and Type	Class	Description
private static final class	Analyzer.StringTokenStream
private static class	AutomatonToTokenStream.TopoTokenStream	Token Stream that outputs tokens from a topo sorted graph.
final class	CachingTokenFilter	This class can be used if the token attributes of a TokenStream are intended to be consumed more than once.
class	FilteringTokenFilter	Abstract base class for TokenFilters that may remove tokens.
class	GraphTokenFilter	An abstract TokenFilter that exposes its input stream as a graph
class	LowerCaseFilter	Normalizes token text to lower case.
class	StopFilter	Removes stop words from a token stream.
class	TokenFilter	A TokenFilter is a TokenStream whose input is another TokenStream.
class	Tokenizer	A Tokenizer is a TokenStream whose input is a Reader.

Fields in org.apache.lucene.analysis declared as TokenStream

Modifier and Type	Field	Description
protected final TokenStream	TokenFilter.input	The source of tokens for this filter.
protected final TokenStream	Analyzer.TokenStreamComponents.sink	Sink tokenstream, such as the outer tokenfilter decorating the chain.

Methods in org.apache.lucene.analysis that return TokenStream

Modifier and Type	Method	Description
abstract TokenStream	TokenFilterFactory.create(TokenStream input)	Transform the specified input TokenStream
TokenStream	Analyzer.TokenStreamComponents.getTokenStream()	Returns the sink TokenStream
protected TokenStream	Analyzer.normalize(String fieldName, TokenStream in)	Wrap the given TokenStream in order to apply normalization filters.
protected final TokenStream	AnalyzerWrapper.normalize(String fieldName, TokenStream in)
TokenStream	TokenFilterFactory.normalize(TokenStream input)	Normalize the specified input TokenStream While the default implementation returns input unchanged, filters that should be applied at normalization time can delegate to create method.
final TokenStream	Analyzer.tokenStream(String fieldName, Reader reader)	Returns a TokenStream suitable for fieldName, tokenizing the contents of reader.
final TokenStream	Analyzer.tokenStream(String fieldName, String text)	Returns a TokenStream suitable for fieldName, tokenizing the contents of text.
static TokenStream	AutomatonToTokenStream.toTokenStream(Automaton automaton)	converts an automaton into a TokenStream.
TokenStream	TokenFilter.unwrap()
protected TokenStream	AnalyzerWrapper.wrapTokenStreamForNormalization(String fieldName, TokenStream in)	Wraps / alters the given TokenStream for normalization purposes, taken from the wrapped Analyzer, to form new components.
protected final TokenStream	DelegatingAnalyzerWrapper.wrapTokenStreamForNormalization(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis with parameters of type TokenStream

Modifier and Type	Method	Description
abstract TokenStream	TokenFilterFactory.create(TokenStream input)	Transform the specified input TokenStream
protected TokenStream	Analyzer.normalize(String fieldName, TokenStream in)	Wrap the given TokenStream in order to apply normalization filters.
protected final TokenStream	AnalyzerWrapper.normalize(String fieldName, TokenStream in)
TokenStream	TokenFilterFactory.normalize(TokenStream input)	Normalize the specified input TokenStream While the default implementation returns input unchanged, filters that should be applied at normalization time can delegate to create method.
Automaton	TokenStreamToAutomaton.toAutomaton(TokenStream in)	Pulls the graph (including PositionLengthAttribute) from the provided TokenStream, and creates the corresponding automaton where arcs are bytes (or Unicode code points if unicodeArcs = true) from each term.
protected TokenStream	AnalyzerWrapper.wrapTokenStreamForNormalization(String fieldName, TokenStream in)	Wraps / alters the given TokenStream for normalization purposes, taken from the wrapped Analyzer, to form new components.
protected final TokenStream	DelegatingAnalyzerWrapper.wrapTokenStreamForNormalization(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis with parameters of type TokenStream

Modifier	Constructor	Description
	CachingTokenFilter(TokenStream input)	Create a new CachingTokenFilter around input.
	FilteringTokenFilter(TokenStream in)	Create a new FilteringTokenFilter.
	GraphTokenFilter(TokenStream input)	Create a new GraphTokenFilter
	LowerCaseFilter(TokenStream in)	Create a new LowerCaseFilter, that normalizes token text to lower case.
	StopFilter(TokenStream in, CharArraySet stopWords)	Constructs a filter which removes words from the input TokenStream that are named in the Set.
protected	TokenFilter(TokenStream input)	Construct a token stream filtering the given input.
	TokenStreamComponents(Consumer<Reader> source, TokenStream result)	Creates a new Analyzer.TokenStreamComponents instance.
	TokenStreamComponents(Tokenizer tokenizer, TokenStream result)	Creates a new Analyzer.TokenStreamComponents instance

Uses of TokenStream in org.apache.lucene.analysis.ar

Subclasses of TokenStream in org.apache.lucene.analysis.ar

Modifier and Type	Class	Description
final class	ArabicNormalizationFilter	A TokenFilter that applies ArabicNormalizer to normalize the orthography.
final class	ArabicStemFilter	A TokenFilter that applies ArabicStemmer to stem Arabic words..

Methods in org.apache.lucene.analysis.ar that return TokenStream

Modifier and Type	Method	Description
TokenStream	ArabicNormalizationFilterFactory.create(TokenStream input)
protected TokenStream	ArabicAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	ArabicNormalizationFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.ar with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	ArabicNormalizationFilterFactory.create(TokenStream input)
ArabicStemFilter	ArabicStemFilterFactory.create(TokenStream input)
protected TokenStream	ArabicAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	ArabicNormalizationFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.ar with parameters of type TokenStream

Modifier	Constructor	Description
	ArabicNormalizationFilter(TokenStream input)
	ArabicStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.bg

Subclasses of TokenStream in org.apache.lucene.analysis.bg

Modifier and Type	Class	Description
final class	BulgarianStemFilter	A TokenFilter that applies BulgarianStemmer to stem Bulgarian words.

Methods in org.apache.lucene.analysis.bg that return TokenStream

Modifier and Type	Method	Description
TokenStream	BulgarianStemFilterFactory.create(TokenStream input)
protected TokenStream	BulgarianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.bg with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	BulgarianStemFilterFactory.create(TokenStream input)
protected TokenStream	BulgarianAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.bg with parameters of type TokenStream

Modifier	Constructor	Description
	BulgarianStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.bn

Subclasses of TokenStream in org.apache.lucene.analysis.bn

Modifier and Type	Class	Description
final class	BengaliNormalizationFilter	A TokenFilter that applies BengaliNormalizer to normalize the orthography.
final class	BengaliStemFilter	A TokenFilter that applies BengaliStemmer to stem Bengali words.

Methods in org.apache.lucene.analysis.bn that return TokenStream

Modifier and Type	Method	Description
TokenStream	BengaliNormalizationFilterFactory.create(TokenStream input)
TokenStream	BengaliStemFilterFactory.create(TokenStream input)
protected TokenStream	BengaliAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	BengaliNormalizationFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.bn with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	BengaliNormalizationFilterFactory.create(TokenStream input)
TokenStream	BengaliStemFilterFactory.create(TokenStream input)
protected TokenStream	BengaliAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	BengaliNormalizationFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.bn with parameters of type TokenStream

Modifier	Constructor	Description
	BengaliNormalizationFilter(TokenStream input)
	BengaliStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.boost

Subclasses of TokenStream in org.apache.lucene.analysis.boost

Modifier and Type	Class	Description
final class	DelimitedBoostTokenFilter	Characters before the delimiter are the "token", those after are the boost.

Methods in org.apache.lucene.analysis.boost with parameters of type TokenStream

Modifier and Type	Method	Description
DelimitedBoostTokenFilter	DelimitedBoostTokenFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.boost with parameters of type TokenStream

Modifier	Constructor	Description
	DelimitedBoostTokenFilter(TokenStream input, char delimiter)

Uses of TokenStream in org.apache.lucene.analysis.br

Subclasses of TokenStream in org.apache.lucene.analysis.br

Modifier and Type	Class	Description
final class	BrazilianStemFilter	A TokenFilter that applies BrazilianStemmer.

Methods in org.apache.lucene.analysis.br that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	BrazilianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.br with parameters of type TokenStream

Modifier and Type	Method	Description
BrazilianStemFilter	BrazilianStemFilterFactory.create(TokenStream in)
protected TokenStream	BrazilianAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.br with parameters of type TokenStream

Modifier	Constructor	Description
	BrazilianStemFilter(TokenStream in)	Creates a new BrazilianStemFilter

Uses of TokenStream in org.apache.lucene.analysis.ca

Methods in org.apache.lucene.analysis.ca that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	CatalanAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.ca with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	CatalanAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.cjk

Subclasses of TokenStream in org.apache.lucene.analysis.cjk

Modifier and Type	Class	Description
final class	CJKBigramFilter	Forms bigrams of CJK terms that are generated from StandardTokenizer or ICUTokenizer.
final class	CJKWidthFilter	A TokenFilter that normalizes CJK width differences: Folds fullwidth ASCII variants into the equivalent basic latin Folds halfwidth Katakana variants into the equivalent kana

Methods in org.apache.lucene.analysis.cjk that return TokenStream

Modifier and Type	Method	Description
TokenStream	CJKBigramFilterFactory.create(TokenStream input)
TokenStream	CJKWidthFilterFactory.create(TokenStream input)
protected TokenStream	CJKAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	CJKWidthFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.cjk with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	CJKBigramFilterFactory.create(TokenStream input)
TokenStream	CJKWidthFilterFactory.create(TokenStream input)
protected TokenStream	CJKAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	CJKWidthFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.cjk with parameters of type TokenStream

Modifier	Constructor	Description
	CJKBigramFilter(TokenStream in)	Calls CJKBigramFilter(in, HAN | HIRAGANA | KATAKANA | HANGUL)
	CJKBigramFilter(TokenStream in, int flags)	Calls CJKBigramFilter(in, flags, false)
	CJKBigramFilter(TokenStream in, int flags, boolean outputUnigrams)	Create a new CJKBigramFilter, specifying which writing systems should be bigrammed, and whether or not unigrams should also be output.
	CJKWidthFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.ckb

Subclasses of TokenStream in org.apache.lucene.analysis.ckb

Modifier and Type	Class	Description
final class	SoraniNormalizationFilter	A TokenFilter that applies SoraniNormalizer to normalize the orthography.
final class	SoraniStemFilter	A TokenFilter that applies SoraniStemmer to stem Sorani words.

Methods in org.apache.lucene.analysis.ckb that return TokenStream

Modifier and Type	Method	Description
TokenStream	SoraniNormalizationFilterFactory.create(TokenStream input)
protected TokenStream	SoraniAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	SoraniNormalizationFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.ckb with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	SoraniNormalizationFilterFactory.create(TokenStream input)
SoraniStemFilter	SoraniStemFilterFactory.create(TokenStream input)
protected TokenStream	SoraniAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	SoraniNormalizationFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.ckb with parameters of type TokenStream

Modifier	Constructor	Description
	SoraniNormalizationFilter(TokenStream input)
	SoraniStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.classic

Subclasses of TokenStream in org.apache.lucene.analysis.classic

Modifier and Type	Class	Description
class	ClassicFilter	Normalizes tokens extracted with ClassicTokenizer.
final class	ClassicTokenizer	A grammar-based tokenizer constructed with JFlex

Methods in org.apache.lucene.analysis.classic that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	ClassicAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.classic with parameters of type TokenStream

Modifier and Type	Method	Description
TokenFilter	ClassicFilterFactory.create(TokenStream input)
protected TokenStream	ClassicAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.classic with parameters of type TokenStream

Modifier	Constructor	Description
	ClassicFilter(TokenStream in)	Construct filtering in.

Uses of TokenStream in org.apache.lucene.analysis.cn.smart

Subclasses of TokenStream in org.apache.lucene.analysis.cn.smart

Modifier and Type	Class	Description
class	HMMChineseTokenizer	Tokenizer for Chinese or mixed Chinese-English text.

Methods in org.apache.lucene.analysis.cn.smart that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	SmartChineseAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.cn.smart with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	SmartChineseAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.commongrams

Subclasses of TokenStream in org.apache.lucene.analysis.commongrams

Modifier and Type	Class	Description
final class	CommonGramsFilter	Construct bigrams for frequently occurring terms while indexing.
final class	CommonGramsQueryFilter	Wrap a CommonGramsFilter optimizing phrase queries by only returning single words when they are not a member of a bigram.

Methods in org.apache.lucene.analysis.commongrams with parameters of type TokenStream

Modifier and Type	Method	Description
TokenFilter	CommonGramsFilterFactory.create(TokenStream input)
TokenFilter	CommonGramsQueryFilterFactory.create(TokenStream input)	Create a CommonGramsFilter and wrap it with a CommonGramsQueryFilter

Constructors in org.apache.lucene.analysis.commongrams with parameters of type TokenStream

Modifier	Constructor	Description
	CommonGramsFilter(TokenStream input, CharArraySet commonWords)	Construct a token stream filtering the given input using a Set of common words to create bigrams.

Uses of TokenStream in org.apache.lucene.analysis.compound

Subclasses of TokenStream in org.apache.lucene.analysis.compound

Modifier and Type	Class	Description
class	CompoundWordTokenFilterBase	Base class for decomposition token filters.
class	DictionaryCompoundWordTokenFilter	A TokenFilter that decomposes compound words found in many Germanic languages.
class	HyphenationCompoundWordTokenFilter	A TokenFilter that decomposes compound words found in many Germanic languages.

Methods in org.apache.lucene.analysis.compound that return TokenStream

Modifier and Type	Method	Description
TokenStream	DictionaryCompoundWordTokenFilterFactory.create(TokenStream input)

Methods in org.apache.lucene.analysis.compound with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	DictionaryCompoundWordTokenFilterFactory.create(TokenStream input)
TokenFilter	HyphenationCompoundWordTokenFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.compound with parameters of type TokenStream

Modifier	Constructor	Description
protected	CompoundWordTokenFilterBase(TokenStream input, CharArraySet dictionary)
protected	CompoundWordTokenFilterBase(TokenStream input, CharArraySet dictionary, boolean onlyLongestMatch)
protected	CompoundWordTokenFilterBase(TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch)
	DictionaryCompoundWordTokenFilter(TokenStream input, CharArraySet dictionary)	Creates a new DictionaryCompoundWordTokenFilter
	DictionaryCompoundWordTokenFilter(TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch)	Creates a new DictionaryCompoundWordTokenFilter
	HyphenationCompoundWordTokenFilter(TokenStream input, HyphenationTree hyphenator)	Create a HyphenationCompoundWordTokenFilter with no dictionary.
	HyphenationCompoundWordTokenFilter(TokenStream input, HyphenationTree hyphenator, int minWordSize, int minSubwordSize, int maxSubwordSize)	Create a HyphenationCompoundWordTokenFilter with no dictionary.
	HyphenationCompoundWordTokenFilter(TokenStream input, HyphenationTree hyphenator, CharArraySet dictionary)	Creates a new HyphenationCompoundWordTokenFilter instance.
	HyphenationCompoundWordTokenFilter(TokenStream input, HyphenationTree hyphenator, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch)	Creates a new HyphenationCompoundWordTokenFilter instance.
	HyphenationCompoundWordTokenFilter(TokenStream input, HyphenationTree hyphenator, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch, boolean noSubMatches, boolean noOverlappingMatches)	Creates a new HyphenationCompoundWordTokenFilter instance.

Uses of TokenStream in org.apache.lucene.analysis.core

Subclasses of TokenStream in org.apache.lucene.analysis.core

Modifier and Type	Class	Description
final class	DecimalDigitFilter	Folds all Unicode digits in [:General_Category=Decimal_Number:] to Basic Latin digits (0-9).
final class	FlattenGraphFilter	Converts an incoming graph token stream, such as one from SynonymGraphFilter, into a flat form so that all nodes form a single linear chain with no side paths.
final class	KeywordTokenizer	Emits the entire input as a single token.
class	LetterTokenizer	A LetterTokenizer is a tokenizer that divides text at non-letters.
final class	LowerCaseFilter	Normalizes token text to lower case.
final class	StopFilter	Removes stop words from a token stream.
final class	TypeTokenFilter	Removes tokens whose types appear in a set of blocked types from a token stream.
final class	UnicodeWhitespaceTokenizer	A UnicodeWhitespaceTokenizer is a tokenizer that divides text at whitespace.
final class	UpperCaseFilter	Normalizes token text to UPPER CASE.
final class	WhitespaceTokenizer	A tokenizer that divides text at whitespace characters as defined by Character.isWhitespace(int).

Methods in org.apache.lucene.analysis.core that return TokenStream

Modifier and Type	Method	Description
TokenStream	DecimalDigitFilterFactory.create(TokenStream input)
TokenStream	FlattenGraphFilterFactory.create(TokenStream input)
TokenStream	LowerCaseFilterFactory.create(TokenStream input)
TokenStream	StopFilterFactory.create(TokenStream input)
TokenStream	TypeTokenFilterFactory.create(TokenStream input)
TokenStream	UpperCaseFilterFactory.create(TokenStream input)
TokenStream	DecimalDigitFilterFactory.normalize(TokenStream input)
TokenStream	LowerCaseFilterFactory.normalize(TokenStream input)
protected TokenStream	SimpleAnalyzer.normalize(String fieldName, TokenStream in)
protected TokenStream	StopAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	UpperCaseFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.core with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	DecimalDigitFilterFactory.create(TokenStream input)
TokenStream	FlattenGraphFilterFactory.create(TokenStream input)
TokenStream	LowerCaseFilterFactory.create(TokenStream input)
TokenStream	StopFilterFactory.create(TokenStream input)
TokenStream	TypeTokenFilterFactory.create(TokenStream input)
TokenStream	UpperCaseFilterFactory.create(TokenStream input)
TokenStream	DecimalDigitFilterFactory.normalize(TokenStream input)
TokenStream	LowerCaseFilterFactory.normalize(TokenStream input)
protected TokenStream	SimpleAnalyzer.normalize(String fieldName, TokenStream in)
protected TokenStream	StopAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	UpperCaseFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.core with parameters of type TokenStream

Modifier	Constructor	Description
	DecimalDigitFilter(TokenStream input)	Creates a new DecimalDigitFilter over input
	FlattenGraphFilter(TokenStream in)
	LowerCaseFilter(TokenStream in)	Create a new LowerCaseFilter, that normalizes token text to lower case.
	StopFilter(TokenStream in, CharArraySet stopWords)	Constructs a filter which removes words from the input TokenStream that are named in the Set.
	TypeTokenFilter(TokenStream input, Set<String> stopTypes)	Create a new TypeTokenFilter that filters tokens out (useWhiteList=false).
	TypeTokenFilter(TokenStream input, Set<String> stopTypes, boolean useWhiteList)	Create a new TypeTokenFilter.
	UpperCaseFilter(TokenStream in)	Create a new UpperCaseFilter, that normalizes token text to upper case.

Uses of TokenStream in org.apache.lucene.analysis.custom

Methods in org.apache.lucene.analysis.custom that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	CustomAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.custom with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	CustomAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.cz

Subclasses of TokenStream in org.apache.lucene.analysis.cz

Modifier and Type	Class	Description
final class	CzechStemFilter	A TokenFilter that applies CzechStemmer to stem Czech words.

Methods in org.apache.lucene.analysis.cz that return TokenStream

Modifier and Type	Method	Description
TokenStream	CzechStemFilterFactory.create(TokenStream input)
protected TokenStream	CzechAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.cz with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	CzechStemFilterFactory.create(TokenStream input)
protected TokenStream	CzechAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.cz with parameters of type TokenStream

Modifier	Constructor	Description
	CzechStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.da

Methods in org.apache.lucene.analysis.da that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	DanishAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.da with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	DanishAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.de

Subclasses of TokenStream in org.apache.lucene.analysis.de

Modifier and Type	Class	Description
final class	GermanLightStemFilter	A TokenFilter that applies GermanLightStemmer to stem German words.
final class	GermanMinimalStemFilter	A TokenFilter that applies GermanMinimalStemmer to stem German words.
final class	GermanNormalizationFilter	Normalizes German characters according to the heuristics of the German2 snowball algorithm.
final class	GermanStemFilter	A TokenFilter that stems German words.

Methods in org.apache.lucene.analysis.de that return TokenStream

Modifier and Type	Method	Description
TokenStream	GermanLightStemFilterFactory.create(TokenStream input)
TokenStream	GermanMinimalStemFilterFactory.create(TokenStream input)
TokenStream	GermanNormalizationFilterFactory.create(TokenStream input)
protected TokenStream	GermanAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	GermanNormalizationFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.de with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	GermanLightStemFilterFactory.create(TokenStream input)
TokenStream	GermanMinimalStemFilterFactory.create(TokenStream input)
TokenStream	GermanNormalizationFilterFactory.create(TokenStream input)
GermanStemFilter	GermanStemFilterFactory.create(TokenStream in)
protected TokenStream	GermanAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	GermanNormalizationFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.de with parameters of type TokenStream

Modifier	Constructor	Description
	GermanLightStemFilter(TokenStream input)
	GermanMinimalStemFilter(TokenStream input)
	GermanNormalizationFilter(TokenStream input)
	GermanStemFilter(TokenStream in)	Creates a GermanStemFilter instance

Uses of TokenStream in org.apache.lucene.analysis.el

Subclasses of TokenStream in org.apache.lucene.analysis.el

Modifier and Type	Class	Description
final class	GreekLowerCaseFilter	Normalizes token text to lower case, removes some Greek diacritics, and standardizes final sigma to sigma.
final class	GreekStemFilter	A TokenFilter that applies GreekStemmer to stem Greek words.

Methods in org.apache.lucene.analysis.el that return TokenStream

Modifier and Type	Method	Description
TokenStream	GreekLowerCaseFilterFactory.create(TokenStream in)
TokenStream	GreekStemFilterFactory.create(TokenStream input)
protected TokenStream	GreekAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	GreekLowerCaseFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.el with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	GreekLowerCaseFilterFactory.create(TokenStream in)
TokenStream	GreekStemFilterFactory.create(TokenStream input)
protected TokenStream	GreekAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	GreekLowerCaseFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.el with parameters of type TokenStream

Modifier	Constructor	Description
	GreekLowerCaseFilter(TokenStream in)	Create a GreekLowerCaseFilter that normalizes Greek token text.
	GreekStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.email

Subclasses of TokenStream in org.apache.lucene.analysis.email

Modifier and Type	Class	Description
final class	UAX29URLEmailTokenizer	This class implements Word Break rules from the Unicode Text Segmentation algorithm, as specified in Unicode Standard Annex #29 URLs and email addresses are also tokenized according to the relevant RFCs.

Methods in org.apache.lucene.analysis.email that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	UAX29URLEmailAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.email with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	UAX29URLEmailAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.en

Subclasses of TokenStream in org.apache.lucene.analysis.en

Modifier and Type	Class	Description
final class	EnglishMinimalStemFilter	A TokenFilter that applies EnglishMinimalStemmer to stem English words.
final class	EnglishPossessiveFilter	TokenFilter that removes possessives (trailing 's) from words.
final class	KStemFilter	A high-performance kstem filter for english.
final class	PorterStemFilter	Transforms the token stream as per the Porter stemming algorithm.

Methods in org.apache.lucene.analysis.en that return TokenStream

Modifier and Type	Method	Description
TokenStream	EnglishMinimalStemFilterFactory.create(TokenStream input)
TokenStream	EnglishPossessiveFilterFactory.create(TokenStream input)
protected TokenStream	EnglishAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.en with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	EnglishMinimalStemFilterFactory.create(TokenStream input)
TokenStream	EnglishPossessiveFilterFactory.create(TokenStream input)
TokenFilter	KStemFilterFactory.create(TokenStream input)
PorterStemFilter	PorterStemFilterFactory.create(TokenStream input)
protected TokenStream	EnglishAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.en with parameters of type TokenStream

Modifier	Constructor	Description
	EnglishMinimalStemFilter(TokenStream input)
	EnglishPossessiveFilter(TokenStream input)
	KStemFilter(TokenStream in)
	PorterStemFilter(TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.es

Subclasses of TokenStream in org.apache.lucene.analysis.es

Modifier and Type	Class	Description
final class	SpanishLightStemFilter	A TokenFilter that applies SpanishLightStemmer to stem Spanish words.
final class	SpanishMinimalStemFilter	Deprecated. Use SpanishPluralStemFilter instead.
final class	SpanishPluralStemFilter	A TokenFilter that applies SpanishPluralStemmer to stem Spanish words.

Methods in org.apache.lucene.analysis.es that return TokenStream

Modifier and Type	Method	Description
TokenStream	SpanishLightStemFilterFactory.create(TokenStream input)
TokenStream	SpanishMinimalStemFilterFactory.create(TokenStream input)	Deprecated.
TokenStream	SpanishPluralStemFilterFactory.create(TokenStream input)
protected TokenStream	SpanishAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.es with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	SpanishLightStemFilterFactory.create(TokenStream input)
TokenStream	SpanishMinimalStemFilterFactory.create(TokenStream input)	Deprecated.
TokenStream	SpanishPluralStemFilterFactory.create(TokenStream input)
protected TokenStream	SpanishAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.es with parameters of type TokenStream

Modifier	Constructor	Description
	SpanishLightStemFilter(TokenStream input)
	SpanishMinimalStemFilter(TokenStream input)	Deprecated.
	SpanishPluralStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.et

Methods in org.apache.lucene.analysis.et that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	EstonianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.et with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	EstonianAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.eu

Methods in org.apache.lucene.analysis.eu that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	BasqueAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.eu with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	BasqueAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.fa

Subclasses of TokenStream in org.apache.lucene.analysis.fa

Modifier and Type	Class	Description
final class	PersianNormalizationFilter	A TokenFilter that applies PersianNormalizer to normalize the orthography.
final class	PersianStemFilter	A TokenFilter that applies PersianStemmer to stem Persian words.

Methods in org.apache.lucene.analysis.fa that return TokenStream

Modifier and Type	Method	Description
TokenStream	PersianNormalizationFilterFactory.create(TokenStream input)
protected TokenStream	PersianAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	PersianNormalizationFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.fa with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	PersianNormalizationFilterFactory.create(TokenStream input)
PersianStemFilter	PersianStemFilterFactory.create(TokenStream input)
protected TokenStream	PersianAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	PersianNormalizationFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.fa with parameters of type TokenStream

Modifier	Constructor	Description
	PersianNormalizationFilter(TokenStream input)
	PersianStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.fi

Subclasses of TokenStream in org.apache.lucene.analysis.fi

Modifier and Type	Class	Description
final class	FinnishLightStemFilter	A TokenFilter that applies FinnishLightStemmer to stem Finnish words.

Methods in org.apache.lucene.analysis.fi that return TokenStream

Modifier and Type	Method	Description
TokenStream	FinnishLightStemFilterFactory.create(TokenStream input)
protected TokenStream	FinnishAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.fi with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	FinnishLightStemFilterFactory.create(TokenStream input)
protected TokenStream	FinnishAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.fi with parameters of type TokenStream

Modifier	Constructor	Description
	FinnishLightStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.fr

Subclasses of TokenStream in org.apache.lucene.analysis.fr

Modifier and Type	Class	Description
final class	FrenchLightStemFilter	A TokenFilter that applies FrenchLightStemmer to stem French words.
final class	FrenchMinimalStemFilter	A TokenFilter that applies FrenchMinimalStemmer to stem French words.

Methods in org.apache.lucene.analysis.fr that return TokenStream

Modifier and Type	Method	Description
TokenStream	FrenchLightStemFilterFactory.create(TokenStream input)
TokenStream	FrenchMinimalStemFilterFactory.create(TokenStream input)
protected TokenStream	FrenchAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.fr with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	FrenchLightStemFilterFactory.create(TokenStream input)
TokenStream	FrenchMinimalStemFilterFactory.create(TokenStream input)
protected TokenStream	FrenchAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.fr with parameters of type TokenStream

Modifier	Constructor	Description
	FrenchLightStemFilter(TokenStream input)
	FrenchMinimalStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.ga

Subclasses of TokenStream in org.apache.lucene.analysis.ga

Modifier and Type	Class	Description
final class	IrishLowerCaseFilter	Normalises token text to lower case, handling t-prothesis and n-eclipsis (i.e., that 'nAthair' should become 'n-athair')

Methods in org.apache.lucene.analysis.ga that return TokenStream

Modifier and Type	Method	Description
TokenStream	IrishLowerCaseFilterFactory.create(TokenStream input)
protected TokenStream	IrishAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	IrishLowerCaseFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.ga with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	IrishLowerCaseFilterFactory.create(TokenStream input)
protected TokenStream	IrishAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	IrishLowerCaseFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.ga with parameters of type TokenStream

Modifier	Constructor	Description
	IrishLowerCaseFilter(TokenStream in)	Create an IrishLowerCaseFilter that normalises Irish token text.

Uses of TokenStream in org.apache.lucene.analysis.gl

Subclasses of TokenStream in org.apache.lucene.analysis.gl

Modifier and Type	Class	Description
final class	GalicianMinimalStemFilter	A TokenFilter that applies GalicianMinimalStemmer to stem Galician words.
final class	GalicianStemFilter	A TokenFilter that applies GalicianStemmer to stem Galician words.

Methods in org.apache.lucene.analysis.gl that return TokenStream

Modifier and Type	Method	Description
TokenStream	GalicianMinimalStemFilterFactory.create(TokenStream input)
TokenStream	GalicianStemFilterFactory.create(TokenStream input)
protected TokenStream	GalicianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.gl with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	GalicianMinimalStemFilterFactory.create(TokenStream input)
TokenStream	GalicianStemFilterFactory.create(TokenStream input)
protected TokenStream	GalicianAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.gl with parameters of type TokenStream

Modifier	Constructor	Description
	GalicianMinimalStemFilter(TokenStream input)
	GalicianStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.hi

Subclasses of TokenStream in org.apache.lucene.analysis.hi

Modifier and Type	Class	Description
final class	HindiNormalizationFilter	A TokenFilter that applies HindiNormalizer to normalize the orthography.
final class	HindiStemFilter	A TokenFilter that applies HindiStemmer to stem Hindi words.

Methods in org.apache.lucene.analysis.hi that return TokenStream

Modifier and Type	Method	Description
TokenStream	HindiNormalizationFilterFactory.create(TokenStream input)
TokenStream	HindiStemFilterFactory.create(TokenStream input)
protected TokenStream	HindiAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	HindiNormalizationFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.hi with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	HindiNormalizationFilterFactory.create(TokenStream input)
TokenStream	HindiStemFilterFactory.create(TokenStream input)
protected TokenStream	HindiAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	HindiNormalizationFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.hi with parameters of type TokenStream

Modifier	Constructor	Description
	HindiNormalizationFilter(TokenStream input)
	HindiStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.hu

Subclasses of TokenStream in org.apache.lucene.analysis.hu

Modifier and Type	Class	Description
final class	HungarianLightStemFilter	A TokenFilter that applies HungarianLightStemmer to stem Hungarian words.

Methods in org.apache.lucene.analysis.hu that return TokenStream

Modifier and Type	Method	Description
TokenStream	HungarianLightStemFilterFactory.create(TokenStream input)
protected TokenStream	HungarianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.hu with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	HungarianLightStemFilterFactory.create(TokenStream input)
protected TokenStream	HungarianAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.hu with parameters of type TokenStream

Modifier	Constructor	Description
	HungarianLightStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.hunspell

Subclasses of TokenStream in org.apache.lucene.analysis.hunspell

Modifier and Type	Class	Description
final class	HunspellStemFilter	TokenFilter that uses hunspell affix rules and words to stem tokens.

Methods in org.apache.lucene.analysis.hunspell that return TokenStream

Modifier and Type	Method	Description
TokenStream	HunspellStemFilterFactory.create(TokenStream tokenStream)

Methods in org.apache.lucene.analysis.hunspell with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	HunspellStemFilterFactory.create(TokenStream tokenStream)

Constructors in org.apache.lucene.analysis.hunspell with parameters of type TokenStream

Modifier	Constructor	Description
	HunspellStemFilter(TokenStream input, Dictionary dictionary)	Create a HunspellStemFilter outputting all possible stems.
	HunspellStemFilter(TokenStream input, Dictionary dictionary, boolean dedup)	Create a HunspellStemFilter outputting all possible stems.
	HunspellStemFilter(TokenStream input, Dictionary dictionary, boolean dedup, boolean longestOnly)	Creates a new HunspellStemFilter that will stem tokens from the given TokenStream using affix rules in the provided Dictionary

Uses of TokenStream in org.apache.lucene.analysis.hy

Methods in org.apache.lucene.analysis.hy that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	ArmenianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.hy with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	ArmenianAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.icu

Subclasses of TokenStream in org.apache.lucene.analysis.icu

Modifier and Type	Class	Description
final class	ICUFoldingFilter	A TokenFilter that applies search term folding to Unicode text, applying foldings from UTR#30 Character Foldings.
class	ICUNormalizer2Filter	Normalize token text with ICU's Normalizer2
final class	ICUTransformFilter	A TokenFilter that transforms text with ICU.

Methods in org.apache.lucene.analysis.icu that return TokenStream

Modifier and Type	Method	Description
TokenStream	ICUFoldingFilterFactory.create(TokenStream input)
TokenStream	ICUNormalizer2FilterFactory.create(TokenStream input)
TokenStream	ICUTransformFilterFactory.create(TokenStream input)
TokenStream	ICUFoldingFilterFactory.normalize(TokenStream input)
TokenStream	ICUNormalizer2FilterFactory.normalize(TokenStream input)
TokenStream	ICUTransformFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.icu with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	ICUFoldingFilterFactory.create(TokenStream input)
TokenStream	ICUNormalizer2FilterFactory.create(TokenStream input)
TokenStream	ICUTransformFilterFactory.create(TokenStream input)
TokenStream	ICUFoldingFilterFactory.normalize(TokenStream input)
TokenStream	ICUNormalizer2FilterFactory.normalize(TokenStream input)
TokenStream	ICUTransformFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.icu with parameters of type TokenStream

Modifier	Constructor	Description
	ICUFoldingFilter(TokenStream input)	Create a new ICUFoldingFilter on the specified input
	ICUFoldingFilter(TokenStream input, com.ibm.icu.text.Normalizer2 normalizer)	Create a new ICUFoldingFilter on the specified input with the specified normalizer
	ICUNormalizer2Filter(TokenStream input)	Create a new Normalizer2Filter that combines NFKC normalization, Case Folding, and removes Default Ignorables (NFKC_Casefold)
	ICUNormalizer2Filter(TokenStream input, com.ibm.icu.text.Normalizer2 normalizer)	Create a new Normalizer2Filter with the specified Normalizer2
	ICUTransformFilter(TokenStream input, com.ibm.icu.text.Transliterator transform)	Create a new ICUTransformFilter that transforms text on the given stream.

Uses of TokenStream in org.apache.lucene.analysis.icu.segmentation

Subclasses of TokenStream in org.apache.lucene.analysis.icu.segmentation

Modifier and Type	Class	Description
final class	ICUTokenizer	Breaks text into words according to UAX #29: Unicode Text Segmentation (http://www.unicode.org/reports/tr29/)

Uses of TokenStream in org.apache.lucene.analysis.id

Subclasses of TokenStream in org.apache.lucene.analysis.id

Modifier and Type	Class	Description
final class	IndonesianStemFilter	A TokenFilter that applies IndonesianStemmer to stem Indonesian words.

Methods in org.apache.lucene.analysis.id that return TokenStream

Modifier and Type	Method	Description
TokenStream	IndonesianStemFilterFactory.create(TokenStream input)
protected TokenStream	IndonesianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.id with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	IndonesianStemFilterFactory.create(TokenStream input)
protected TokenStream	IndonesianAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.id with parameters of type TokenStream

Modifier	Constructor	Description
	IndonesianStemFilter(TokenStream input)	Calls IndonesianStemFilter(input, true)
	IndonesianStemFilter(TokenStream input, boolean stemDerivational)	Create a new IndonesianStemFilter.

Uses of TokenStream in org.apache.lucene.analysis.in

Subclasses of TokenStream in org.apache.lucene.analysis.in

Modifier and Type	Class	Description
final class	IndicNormalizationFilter	A TokenFilter that applies IndicNormalizer to normalize text in Indian Languages.

Methods in org.apache.lucene.analysis.in that return TokenStream

Modifier and Type	Method	Description
TokenStream	IndicNormalizationFilterFactory.create(TokenStream input)
TokenStream	IndicNormalizationFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.in with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	IndicNormalizationFilterFactory.create(TokenStream input)
TokenStream	IndicNormalizationFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.in with parameters of type TokenStream

Modifier	Constructor	Description
	IndicNormalizationFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.it

Subclasses of TokenStream in org.apache.lucene.analysis.it

Modifier and Type	Class	Description
final class	ItalianLightStemFilter	A TokenFilter that applies ItalianLightStemmer to stem Italian words.

Methods in org.apache.lucene.analysis.it that return TokenStream

Modifier and Type	Method	Description
TokenStream	ItalianLightStemFilterFactory.create(TokenStream input)
protected TokenStream	ItalianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.it with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	ItalianLightStemFilterFactory.create(TokenStream input)
protected TokenStream	ItalianAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.it with parameters of type TokenStream

Modifier	Constructor	Description
	ItalianLightStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.ja

Subclasses of TokenStream in org.apache.lucene.analysis.ja

Modifier and Type	Class	Description
final class	JapaneseBaseFormFilter	Replaces term text with the BaseFormAttribute.
final class	JapaneseCompletionFilter	A TokenFilter that adds Japanese romanized tokens to the term attribute.
final class	JapaneseHiraganaUppercaseFilter	A TokenFilter that normalizes small letters (捨て仮名) in hiragana into normal letters.
final class	JapaneseKatakanaStemFilter	A TokenFilter that normalizes common katakana spelling variations ending in a long sound character by removing this character (U+30FC).
final class	JapaneseKatakanaUppercaseFilter	A TokenFilter that normalizes small letters (捨て仮名) in katakana into normal letters.
class	JapaneseNumberFilter	A TokenFilter that normalizes Japanese numbers (kansūji) to regular Arabic decimal numbers in half-width characters.
final class	JapanesePartOfSpeechStopFilter	Removes tokens that match a set of part-of-speech tags.
final class	JapaneseReadingFormFilter	A TokenFilter that replaces the term attribute with the reading of a token in either katakana or romaji form.
final class	JapaneseTokenizer	Tokenizer for Japanese that uses morphological analysis.

Methods in org.apache.lucene.analysis.ja that return TokenStream

Modifier and Type	Method	Description
TokenStream	JapaneseBaseFormFilterFactory.create(TokenStream input)
TokenStream	JapaneseCompletionFilterFactory.create(TokenStream input)
TokenStream	JapaneseHiraganaUppercaseFilterFactory.create(TokenStream input)
TokenStream	JapaneseKatakanaStemFilterFactory.create(TokenStream input)
TokenStream	JapaneseKatakanaUppercaseFilterFactory.create(TokenStream input)
TokenStream	JapaneseNumberFilterFactory.create(TokenStream input)
TokenStream	JapanesePartOfSpeechStopFilterFactory.create(TokenStream stream)
TokenStream	JapaneseReadingFormFilterFactory.create(TokenStream input)
protected TokenStream	JapaneseAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.ja with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	JapaneseBaseFormFilterFactory.create(TokenStream input)
TokenStream	JapaneseCompletionFilterFactory.create(TokenStream input)
TokenStream	JapaneseHiraganaUppercaseFilterFactory.create(TokenStream input)
TokenStream	JapaneseKatakanaStemFilterFactory.create(TokenStream input)
TokenStream	JapaneseKatakanaUppercaseFilterFactory.create(TokenStream input)
TokenStream	JapaneseNumberFilterFactory.create(TokenStream input)
TokenStream	JapanesePartOfSpeechStopFilterFactory.create(TokenStream stream)
TokenStream	JapaneseReadingFormFilterFactory.create(TokenStream input)
protected TokenStream	JapaneseAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.ja with parameters of type TokenStream

Modifier	Constructor	Description
	JapaneseBaseFormFilter(TokenStream input)
	JapaneseCompletionFilter(TokenStream input)	Creates a new JapaneseCompletionFilter with default configurations
	JapaneseCompletionFilter(TokenStream input, JapaneseCompletionFilter.Mode mode)	Creates a new JapaneseCompletionFilter
	JapaneseHiraganaUppercaseFilter(TokenStream input)
	JapaneseKatakanaStemFilter(TokenStream input)
	JapaneseKatakanaStemFilter(TokenStream input, int minimumLength)
	JapaneseKatakanaUppercaseFilter(TokenStream input)
	JapaneseNumberFilter(TokenStream input)
	JapanesePartOfSpeechStopFilter(TokenStream input, Set<String> stopTags)	Create a new JapanesePartOfSpeechStopFilter.
	JapaneseReadingFormFilter(TokenStream input)
	JapaneseReadingFormFilter(TokenStream input, boolean useRomaji)

Uses of TokenStream in org.apache.lucene.analysis.ko

Subclasses of TokenStream in org.apache.lucene.analysis.ko

Modifier and Type	Class	Description
class	KoreanNumberFilter	A TokenFilter that normalizes Korean numbers to regular Arabic decimal numbers in half-width characters.
final class	KoreanPartOfSpeechStopFilter	Removes tokens that match a set of part-of-speech tags.
final class	KoreanReadingFormFilter	Replaces term text with the ReadingAttribute which is the Hangul transcription of Hanja characters.
final class	KoreanTokenizer	Tokenizer for Korean that uses morphological analysis.

Methods in org.apache.lucene.analysis.ko that return TokenStream

Modifier and Type	Method	Description
TokenStream	KoreanNumberFilterFactory.create(TokenStream input)
TokenStream	KoreanPartOfSpeechStopFilterFactory.create(TokenStream stream)
TokenStream	KoreanReadingFormFilterFactory.create(TokenStream input)
protected TokenStream	KoreanAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.ko with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	KoreanNumberFilterFactory.create(TokenStream input)
TokenStream	KoreanPartOfSpeechStopFilterFactory.create(TokenStream stream)
TokenStream	KoreanReadingFormFilterFactory.create(TokenStream input)
protected TokenStream	KoreanAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.ko with parameters of type TokenStream

Modifier	Constructor	Description
	KoreanNumberFilter(TokenStream input)
	KoreanPartOfSpeechStopFilter(TokenStream input)	Create a new KoreanPartOfSpeechStopFilter with the default list of stop tags KoreanPartOfSpeechStopFilter.DEFAULT_STOP_TAGS.
	KoreanPartOfSpeechStopFilter(TokenStream input, Set<POS.Tag> stopTags)	Create a new KoreanPartOfSpeechStopFilter.
	KoreanReadingFormFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.lt

Methods in org.apache.lucene.analysis.lt that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	LithuanianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.lt with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	LithuanianAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.lv

Subclasses of TokenStream in org.apache.lucene.analysis.lv

Modifier and Type	Class	Description
final class	LatvianStemFilter	A TokenFilter that applies LatvianStemmer to stem Latvian words.

Methods in org.apache.lucene.analysis.lv that return TokenStream

Modifier and Type	Method	Description
TokenStream	LatvianStemFilterFactory.create(TokenStream input)
protected TokenStream	LatvianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.lv with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	LatvianStemFilterFactory.create(TokenStream input)
protected TokenStream	LatvianAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.lv with parameters of type TokenStream

Modifier	Constructor	Description
	LatvianStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.minhash

Subclasses of TokenStream in org.apache.lucene.analysis.minhash

Modifier and Type	Class	Description
class	MinHashFilter	Generate min hash tokens from an incoming stream of tokens.

Methods in org.apache.lucene.analysis.minhash that return TokenStream

Modifier and Type	Method	Description
TokenStream	MinHashFilterFactory.create(TokenStream input)

Methods in org.apache.lucene.analysis.minhash with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	MinHashFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.minhash with parameters of type TokenStream

Modifier	Constructor	Description
	MinHashFilter(TokenStream input, int hashCount, int bucketCount, int hashSetSize, boolean withRotation)	create a MinHash filter

Uses of TokenStream in org.apache.lucene.analysis.miscellaneous

Subclasses of TokenStream in org.apache.lucene.analysis.miscellaneous

Modifier and Type	Class	Description
final class	ASCIIFoldingFilter	This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists.
final class	CapitalizationFilter	A filter to apply normal capitalization rules to Tokens.
final class	CodepointCountFilter	Removes words that are too long or too short from the stream.
final class	ConcatenateGraphFilter	Concatenates/Joins every incoming token with a separator into one output token for every path through the token stream (which is a graph).
final class	ConcatenatingTokenStream	A TokenStream that takes an array of input TokenStreams as sources, and concatenates them together.
class	ConditionalTokenFilter	Allows skipping TokenFilters based on the current set of attributes.
private final class	ConditionalTokenFilter.OneTimeWrapper
class	DateRecognizerFilter	Filters all tokens that cannot be parsed to a date, using the provided DateFormat.
final class	DelimitedTermFrequencyTokenFilter	Characters before the delimiter are the "token", the textual integer after is the term frequency.
final class	DropIfFlaggedFilter	Allows Tokens with a given combination of flags to be dropped.
final class	EmptyTokenStream	An always exhausted token stream.
class	FingerprintFilter	Filter outputs a single token which is a concatenation of the sorted and de-duplicated set of input tokens.
final class	FixBrokenOffsetsFilter	Deprecated. Fix the token filters that create broken offsets in the first place.
final class	HyphenatedWordsFilter	When the plain text is extracted from documents, we will often have many words hyphenated and broken into two lines.
final class	KeepWordFilter	A TokenFilter that only keeps tokens with text contained in the required words.
class	KeywordMarkerFilter	Marks terms as keywords via the KeywordAttribute.
final class	KeywordRepeatFilter	This TokenFilter emits each incoming token twice once as keyword and once non-keyword, in other words once with KeywordAttribute.setKeyword(boolean) set to true and once set to false.
final class	LengthFilter	Removes words that are too long or too short from the stream.
final class	LimitTokenCountFilter	This TokenFilter limits the number of tokens while indexing.
final class	LimitTokenOffsetFilter	Lets all tokens pass through until it sees one with a start offset <= a configured limit, which won't pass and ends the stream.
final class	LimitTokenPositionFilter	This TokenFilter limits its emitted tokens to those with positions that are not greater than the configured limit.
final class	PatternKeywordMarkerFilter	Marks terms as keywords via the KeywordAttribute.
class	ProtectedTermFilter	A ConditionalTokenFilter that only applies its wrapped filters to tokens that are not contained in a protected set.
final class	RemoveDuplicatesTokenFilter	A TokenFilter which filters out Tokens at the same position and Term text as the previous token in the stream.
final class	ScandinavianFoldingFilter	This filter folds Scandinavian characters åÅäæÄÆ->a and öÖøØ->o.
final class	ScandinavianNormalizationFilter	This filter normalize use of the interchangeable Scandinavian characters æÆäÄöÖøØ and folded variants (aa, ao, ae, oe and oo) by transforming them to åÅæÆøØ.
final class	SetKeywordMarkerFilter	Marks terms as keywords via the KeywordAttribute.
final class	StemmerOverrideFilter	Provides the ability to override any KeywordAttribute aware stemmer with custom dictionary-based stemming.
final class	TrimFilter	Trims leading and trailing whitespace from Tokens in the stream.
final class	TruncateTokenFilter	A token filter for truncating the terms into a specific length.
final class	TypeAsSynonymFilter	Adds the TypeAttribute.type() as a synonym, i.e.
final class	WordDelimiterFilter	Deprecated. Use WordDelimiterGraphFilter instead: it produces a correct token graph so that e.g.
final class	WordDelimiterGraphFilter	Splits words into subwords and performs optional transformations on subword groups, producing a correct token graph so that e.g.

Fields in org.apache.lucene.analysis.miscellaneous declared as TokenStream

Modifier and Type	Field	Description
private final TokenStream	ConditionalTokenFilter.delegate
private final TokenStream	ConcatenateGraphFilter.inputTokenStream
private final TokenStream[]	ConcatenatingTokenStream.sources

Methods in org.apache.lucene.analysis.miscellaneous that return TokenStream

Modifier and Type	Method	Description
TokenStream	ASCIIFoldingFilterFactory.create(TokenStream input)
TokenStream	ConcatenateGraphFilterFactory.create(TokenStream input)
TokenStream	ConditionalTokenFilterFactory.create(TokenStream input)
TokenStream	DateRecognizerFilterFactory.create(TokenStream input)
TokenStream	DropIfFlaggedFilterFactory.create(TokenStream input)
TokenStream	FingerprintFilterFactory.create(TokenStream input)
TokenStream	FixBrokenOffsetsFilterFactory.create(TokenStream input)	Deprecated.
TokenStream	KeepWordFilterFactory.create(TokenStream input)
TokenStream	KeywordMarkerFilterFactory.create(TokenStream input)
TokenStream	KeywordRepeatFilterFactory.create(TokenStream input)
TokenStream	LimitTokenCountFilterFactory.create(TokenStream input)
TokenStream	LimitTokenOffsetFilterFactory.create(TokenStream input)
TokenStream	LimitTokenPositionFilterFactory.create(TokenStream input)
TokenStream	ScandinavianFoldingFilterFactory.create(TokenStream input)
TokenStream	StemmerOverrideFilterFactory.create(TokenStream input)
TokenStream	TrimFilterFactory.create(TokenStream input)
TokenStream	TruncateTokenFilterFactory.create(TokenStream input)
TokenStream	TypeAsSynonymFilterFactory.create(TokenStream input)
TokenStream	ASCIIFoldingFilterFactory.normalize(TokenStream input)
TokenStream	ScandinavianFoldingFilterFactory.normalize(TokenStream input)
TokenStream	ScandinavianNormalizationFilterFactory.normalize(TokenStream input)
TokenStream	TrimFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.miscellaneous with parameters of type TokenStream

Modifier and Type	Method	Description
private static AttributeSource	ConcatenatingTokenStream.combineSources(TokenStream... sources)
TokenStream	ASCIIFoldingFilterFactory.create(TokenStream input)
CapitalizationFilter	CapitalizationFilterFactory.create(TokenStream input)
CodepointCountFilter	CodepointCountFilterFactory.create(TokenStream input)
TokenStream	ConcatenateGraphFilterFactory.create(TokenStream input)
TokenStream	ConditionalTokenFilterFactory.create(TokenStream input)
protected abstract ConditionalTokenFilter	ConditionalTokenFilterFactory.create(TokenStream input, Function<TokenStream,TokenStream> inner)	Modify the incoming TokenStream with a ConditionalTokenFilter
TokenStream	DateRecognizerFilterFactory.create(TokenStream input)
DelimitedTermFrequencyTokenFilter	DelimitedTermFrequencyTokenFilterFactory.create(TokenStream input)
TokenStream	DropIfFlaggedFilterFactory.create(TokenStream input)
TokenStream	FingerprintFilterFactory.create(TokenStream input)
TokenStream	FixBrokenOffsetsFilterFactory.create(TokenStream input)	Deprecated.
HyphenatedWordsFilter	HyphenatedWordsFilterFactory.create(TokenStream input)
TokenStream	KeepWordFilterFactory.create(TokenStream input)
TokenStream	KeywordMarkerFilterFactory.create(TokenStream input)
TokenStream	KeywordRepeatFilterFactory.create(TokenStream input)
LengthFilter	LengthFilterFactory.create(TokenStream input)
TokenStream	LimitTokenCountFilterFactory.create(TokenStream input)
TokenStream	LimitTokenOffsetFilterFactory.create(TokenStream input)
TokenStream	LimitTokenPositionFilterFactory.create(TokenStream input)
protected ConditionalTokenFilter	ProtectedTermFilterFactory.create(TokenStream input, Function<TokenStream,TokenStream> inner)
RemoveDuplicatesTokenFilter	RemoveDuplicatesTokenFilterFactory.create(TokenStream input)
TokenStream	ScandinavianFoldingFilterFactory.create(TokenStream input)
ScandinavianNormalizationFilter	ScandinavianNormalizationFilterFactory.create(TokenStream input)
TokenStream	StemmerOverrideFilterFactory.create(TokenStream input)
TokenStream	TrimFilterFactory.create(TokenStream input)
TokenStream	TruncateTokenFilterFactory.create(TokenStream input)
TokenStream	TypeAsSynonymFilterFactory.create(TokenStream input)
TokenFilter	WordDelimiterFilterFactory.create(TokenStream input)	Deprecated.
TokenFilter	WordDelimiterGraphFilterFactory.create(TokenStream input)
TokenStream	ASCIIFoldingFilterFactory.normalize(TokenStream input)
TokenStream	ScandinavianFoldingFilterFactory.normalize(TokenStream input)
TokenStream	ScandinavianNormalizationFilterFactory.normalize(TokenStream input)
TokenStream	TrimFilterFactory.normalize(TokenStream input)

Method parameters in org.apache.lucene.analysis.miscellaneous with type arguments of type TokenStream

Modifier and Type	Method	Description
protected abstract ConditionalTokenFilter	ConditionalTokenFilterFactory.create(TokenStream input, Function<TokenStream,TokenStream> inner)	Modify the incoming TokenStream with a ConditionalTokenFilter
protected abstract ConditionalTokenFilter	ConditionalTokenFilterFactory.create(TokenStream input, Function<TokenStream,TokenStream> inner)	Modify the incoming TokenStream with a ConditionalTokenFilter
protected ConditionalTokenFilter	ProtectedTermFilterFactory.create(TokenStream input, Function<TokenStream,TokenStream> inner)
protected ConditionalTokenFilter	ProtectedTermFilterFactory.create(TokenStream input, Function<TokenStream,TokenStream> inner)

Constructors in org.apache.lucene.analysis.miscellaneous with parameters of type TokenStream

Modifier	Constructor	Description
	ASCIIFoldingFilter(TokenStream input)
	ASCIIFoldingFilter(TokenStream input, boolean preserveOriginal)	Create a new ASCIIFoldingFilter.
	CapitalizationFilter(TokenStream in)	Creates a CapitalizationFilter with the default parameters.
	CapitalizationFilter(TokenStream in, boolean onlyFirstWord, CharArraySet keep, boolean forceFirstLetter, Collection<char[]> okPrefix, int minWordLength, int maxWordCount, int maxTokenLength)	Creates a CapitalizationFilter with the specified parameters.
	CodepointCountFilter(TokenStream in, int min, int max)	Create a new CodepointCountFilter.
	ConcatenateGraphFilter(TokenStream inputTokenStream)	Creates a token stream to convert input to a token stream of accepted strings by its token stream graph.
	ConcatenateGraphFilter(TokenStream inputTokenStream, boolean preserveSep, boolean preservePositionIncrements, int maxGraphExpansions)	Calls ConcatenateGraphFilter(org.apache.lucene.analysis.TokenStream, java.lang.Character, boolean, int)
	ConcatenateGraphFilter(TokenStream inputTokenStream, Character tokenSeparator, boolean preservePositionIncrements, int maxGraphExpansions)	Creates a token stream to convert input to a token stream of accepted strings by its token stream graph.
	ConcatenatingTokenStream(TokenStream... sources)	Create a new ConcatenatingTokenStream from a set of inputs
protected	ConditionalTokenFilter(TokenStream input, Function<TokenStream,TokenStream> inputFactory)	Create a new ConditionalTokenFilter
	DateRecognizerFilter(TokenStream input)	Uses DateFormat.DEFAULT and Locale.ENGLISH to create a DateFormat instance.
	DateRecognizerFilter(TokenStream input, DateFormat dateFormat)
	DelimitedTermFrequencyTokenFilter(TokenStream input)
	DelimitedTermFrequencyTokenFilter(TokenStream input, char delimiter)
	DropIfFlaggedFilter(TokenStream input, int dropFlags)	Construct a token stream filtering the given input.
	FingerprintFilter(TokenStream input)	Create a new FingerprintFilter with default settings
	FingerprintFilter(TokenStream input, int maxOutputTokenSize, char separator)	Create a new FingerprintFilter with control over all settings
	FixBrokenOffsetsFilter(TokenStream in)	Deprecated.
	HyphenatedWordsFilter(TokenStream in)	Creates a new HyphenatedWordsFilter
	KeepWordFilter(TokenStream in, CharArraySet words)	Create a new KeepWordFilter.
protected	KeywordMarkerFilter(TokenStream in)	Creates a new KeywordMarkerFilter
	KeywordRepeatFilter(TokenStream input)	Construct a token stream filtering the given input.
	LengthFilter(TokenStream in, int min, int max)	Create a new LengthFilter.
	LimitTokenCountFilter(TokenStream in, int maxTokenCount)	Build a filter that only accepts tokens up to a maximum number.
	LimitTokenCountFilter(TokenStream in, int maxTokenCount, boolean consumeAllTokens)	Build an filter that limits the maximum number of tokens per field.
	LimitTokenOffsetFilter(TokenStream input, int maxStartOffset)	Lets all tokens pass through until it sees one with a start offset <= maxStartOffset which won't pass and ends the stream.
	LimitTokenOffsetFilter(TokenStream input, int maxStartOffset, boolean consumeAllTokens)
	LimitTokenPositionFilter(TokenStream in, int maxTokenPosition)	Build a filter that only accepts tokens up to and including the given maximum position.
	LimitTokenPositionFilter(TokenStream in, int maxTokenPosition, boolean consumeAllTokens)	Build a filter that limits the maximum position of tokens to emit.
	PatternKeywordMarkerFilter(TokenStream in, Pattern pattern)	Create a new PatternKeywordMarkerFilter, that marks the current token as a keyword if the tokens term buffer matches the provided Pattern via the KeywordAttribute.
	ProtectedTermFilter(CharArraySet protectedTerms, TokenStream input, Function<TokenStream,TokenStream> inputFactory)	Creates a new ProtectedTermFilter
	RemoveDuplicatesTokenFilter(TokenStream in)	Creates a new RemoveDuplicatesTokenFilter
	ScandinavianFoldingFilter(TokenStream input)
	ScandinavianNormalizationFilter(TokenStream input)
	SetKeywordMarkerFilter(TokenStream in, CharArraySet keywordSet)	Create a new KeywordSetMarkerFilter, that marks the current token as a keyword if the tokens term buffer is contained in the given set via the KeywordAttribute.
	StemmerOverrideFilter(TokenStream input, StemmerOverrideFilter.StemmerOverrideMap stemmerOverrideMap)	Create a new StemmerOverrideFilter, performing dictionary-based stemming with the provided dictionary.
	TrimFilter(TokenStream in)	Create a new TrimFilter.
	TruncateTokenFilter(TokenStream input, int length)
	TypeAsSynonymFilter(TokenStream input)
	TypeAsSynonymFilter(TokenStream input, String prefix)
	TypeAsSynonymFilter(TokenStream input, String prefix, Set<String> ignore, int synFlagsMask)
	WordDelimiterFilter(TokenStream in, byte[] charTypeTable, int configurationFlags, CharArraySet protWords)	Deprecated. Creates a new WordDelimiterFilter
	WordDelimiterFilter(TokenStream in, int configurationFlags, CharArraySet protWords)	Deprecated. Creates a new WordDelimiterFilter using WordDelimiterIterator.DEFAULT_WORD_DELIM_TABLE as its charTypeTable
	WordDelimiterGraphFilter(TokenStream in, boolean adjustInternalOffsets, byte[] charTypeTable, int configurationFlags, CharArraySet protWords)	Creates a new WordDelimiterGraphFilter
	WordDelimiterGraphFilter(TokenStream in, int configurationFlags, CharArraySet protWords)	Creates a new WordDelimiterGraphFilter using WordDelimiterIterator.DEFAULT_WORD_DELIM_TABLE as its charTypeTable

Constructor parameters in org.apache.lucene.analysis.miscellaneous with type arguments of type TokenStream

Modifier	Constructor	Description
protected	ConditionalTokenFilter(TokenStream input, Function<TokenStream,TokenStream> inputFactory)	Create a new ConditionalTokenFilter
protected	ConditionalTokenFilter(TokenStream input, Function<TokenStream,TokenStream> inputFactory)	Create a new ConditionalTokenFilter
	ProtectedTermFilter(CharArraySet protectedTerms, TokenStream input, Function<TokenStream,TokenStream> inputFactory)	Creates a new ProtectedTermFilter
	ProtectedTermFilter(CharArraySet protectedTerms, TokenStream input, Function<TokenStream,TokenStream> inputFactory)	Creates a new ProtectedTermFilter

Uses of TokenStream in org.apache.lucene.analysis.ne

Methods in org.apache.lucene.analysis.ne that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	NepaliAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.ne with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	NepaliAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.ngram

Subclasses of TokenStream in org.apache.lucene.analysis.ngram

Modifier and Type	Class	Description
final class	EdgeNGramTokenFilter	Tokenizes the given token into n-grams of given size(s).
class	EdgeNGramTokenizer	Tokenizes the input from an edge into n-grams of given size(s).
final class	NGramTokenFilter	Tokenizes the input into n-grams of the given size(s).
class	NGramTokenizer	Tokenizes the input into n-grams of the given size(s).

Methods in org.apache.lucene.analysis.ngram with parameters of type TokenStream

Modifier and Type	Method	Description
TokenFilter	EdgeNGramFilterFactory.create(TokenStream input)
TokenFilter	NGramFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.ngram with parameters of type TokenStream

Modifier	Constructor	Description
	EdgeNGramTokenFilter(TokenStream input, int gramSize)	Creates an EdgeNGramTokenFilter that produces edge n-grams of the given size.
	EdgeNGramTokenFilter(TokenStream input, int minGram, int maxGram, boolean preserveOriginal)	Creates an EdgeNGramTokenFilter that, for a given input term, produces all edge n-grams with lengths >= minGram and <= maxGram.
	NGramTokenFilter(TokenStream input, int gramSize)	Creates an NGramTokenFilter that produces n-grams of the indicated size.
	NGramTokenFilter(TokenStream input, int minGram, int maxGram, boolean preserveOriginal)	Creates an NGramTokenFilter that, for a given input term, produces all contained n-grams with lengths >= minGram and <= maxGram.

Uses of TokenStream in org.apache.lucene.analysis.nl

Methods in org.apache.lucene.analysis.nl that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	DutchAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.nl with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	DutchAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.no

Subclasses of TokenStream in org.apache.lucene.analysis.no

Modifier and Type	Class	Description
final class	NorwegianLightStemFilter	A TokenFilter that applies NorwegianLightStemmer to stem Norwegian words.
final class	NorwegianMinimalStemFilter	A TokenFilter that applies NorwegianMinimalStemmer to stem Norwegian words.
final class	NorwegianNormalizationFilter	This filter normalize use of the interchangeable Scandinavian characters æÆäÄöÖøØ and folded variants (ae, oe, aa) by transforming them to åÅæÆøØ.

Methods in org.apache.lucene.analysis.no that return TokenStream

Modifier and Type	Method	Description
TokenStream	NorwegianLightStemFilterFactory.create(TokenStream input)
TokenStream	NorwegianMinimalStemFilterFactory.create(TokenStream input)
protected TokenStream	NorwegianAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	NorwegianNormalizationFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.no with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	NorwegianLightStemFilterFactory.create(TokenStream input)
TokenStream	NorwegianMinimalStemFilterFactory.create(TokenStream input)
NorwegianNormalizationFilter	NorwegianNormalizationFilterFactory.create(TokenStream input)
protected TokenStream	NorwegianAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	NorwegianNormalizationFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.no with parameters of type TokenStream

Modifier	Constructor	Description
	NorwegianLightStemFilter(TokenStream input)	Calls NorwegianLightStemFilter(input, BOKMAAL)
	NorwegianLightStemFilter(TokenStream input, int flags)	Creates a new NorwegianLightStemFilter
	NorwegianMinimalStemFilter(TokenStream input)	Calls NorwegianMinimalStemFilter(input, BOKMAAL)
	NorwegianMinimalStemFilter(TokenStream input, int flags)	Creates a new NorwegianLightStemFilter
	NorwegianNormalizationFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.path

Subclasses of TokenStream in org.apache.lucene.analysis.path

Modifier and Type	Class	Description
class	PathHierarchyTokenizer	Tokenizer for path-like hierarchies.
class	ReversePathHierarchyTokenizer	Tokenizer for domain-like hierarchies.

Uses of TokenStream in org.apache.lucene.analysis.pattern

Subclasses of TokenStream in org.apache.lucene.analysis.pattern

Modifier and Type	Class	Description
final class	PatternCaptureGroupTokenFilter	CaptureGroup uses Java regexes to emit multiple tokens - one for each capture group in one or more patterns.
final class	PatternReplaceFilter	A TokenFilter which applies a Pattern to each token in the stream, replacing match occurrences with the specified replacement string.
final class	PatternTokenizer	This tokenizer uses regex pattern matching to construct distinct tokens for the input stream.
class	PatternTypingFilter	Set a type attribute to a parameterized value when tokens are matched by any of a several regex patterns.
final class	SimplePatternSplitTokenizer	This tokenizer uses a Lucene RegExp or (expert usage) a pre-built determinized Automaton, to locate tokens.
final class	SimplePatternTokenizer	This tokenizer uses a Lucene RegExp or (expert usage) a pre-built determinized Automaton, to locate tokens.

Methods in org.apache.lucene.analysis.pattern that return TokenStream

Modifier and Type	Method	Description
TokenStream	PatternTypingFilterFactory.create(TokenStream input)

Methods in org.apache.lucene.analysis.pattern with parameters of type TokenStream

Modifier and Type	Method	Description
PatternCaptureGroupTokenFilter	PatternCaptureGroupFilterFactory.create(TokenStream input)
PatternReplaceFilter	PatternReplaceFilterFactory.create(TokenStream input)
TokenStream	PatternTypingFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.pattern with parameters of type TokenStream

Modifier	Constructor	Description
	PatternCaptureGroupTokenFilter(TokenStream input, boolean preserveOriginal, Pattern... patterns)
	PatternReplaceFilter(TokenStream in, Pattern p, String replacement, boolean all)	Constructs an instance to replace either the first, or all occurrences
	PatternTypingFilter(TokenStream input, PatternTypingFilter.PatternTypingRule... replacementAndFlagByPattern)

Uses of TokenStream in org.apache.lucene.analysis.payloads

Subclasses of TokenStream in org.apache.lucene.analysis.payloads

Modifier and Type	Class	Description
final class	DelimitedPayloadTokenFilter	Characters before the delimiter are the "token", those after are the payload.
class	NumericPayloadTokenFilter	Assigns a payload to a token based on the TypeAttribute
class	TokenOffsetPayloadTokenFilter	Adds the OffsetAttribute.startOffset() and OffsetAttribute.endOffset() First 4 bytes are the start
class	TypeAsPayloadTokenFilter	Makes the TypeAttribute a payload.

Methods in org.apache.lucene.analysis.payloads with parameters of type TokenStream

Modifier and Type	Method	Description
DelimitedPayloadTokenFilter	DelimitedPayloadTokenFilterFactory.create(TokenStream input)
NumericPayloadTokenFilter	NumericPayloadTokenFilterFactory.create(TokenStream input)
TokenOffsetPayloadTokenFilter	TokenOffsetPayloadTokenFilterFactory.create(TokenStream input)
TypeAsPayloadTokenFilter	TypeAsPayloadTokenFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.payloads with parameters of type TokenStream

Modifier	Constructor	Description
	DelimitedPayloadTokenFilter(TokenStream input, char delimiter, PayloadEncoder encoder)
	NumericPayloadTokenFilter(TokenStream input, float payload, String typeMatch)
	TokenOffsetPayloadTokenFilter(TokenStream input)
	TypeAsPayloadTokenFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.phonetic

Subclasses of TokenStream in org.apache.lucene.analysis.phonetic

Modifier and Type	Class	Description
final class	BeiderMorseFilter	TokenFilter for Beider-Morse phonetic encoding.
final class	DaitchMokotoffSoundexFilter	Create tokens for phonetic matches based on Daitch–Mokotoff Soundex.
final class	DoubleMetaphoneFilter	Filter for DoubleMetaphone (supporting secondary codes)
final class	PhoneticFilter	Create tokens for phonetic matches.

Methods in org.apache.lucene.analysis.phonetic that return TokenStream

Modifier and Type	Method	Description
TokenStream	BeiderMorseFilterFactory.create(TokenStream input)

Methods in org.apache.lucene.analysis.phonetic with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	BeiderMorseFilterFactory.create(TokenStream input)
DaitchMokotoffSoundexFilter	DaitchMokotoffSoundexFilterFactory.create(TokenStream input)
DoubleMetaphoneFilter	DoubleMetaphoneFilterFactory.create(TokenStream input)
PhoneticFilter	PhoneticFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.phonetic with parameters of type TokenStream

Modifier	Constructor	Description
	BeiderMorseFilter(TokenStream input, org.apache.commons.codec.language.bm.PhoneticEngine engine)	Calls BeiderMorseFilter(TokenStream, PhoneticEngine, org.apache.commons.codec.language.bm.Languages.LanguageSet)
	BeiderMorseFilter(TokenStream input, org.apache.commons.codec.language.bm.PhoneticEngine engine, org.apache.commons.codec.language.bm.Languages.LanguageSet languages)	Create a new BeiderMorseFilter
	DaitchMokotoffSoundexFilter(TokenStream in, boolean inject)	Creates a DaitchMokotoffSoundexFilter by either adding encoded forms as synonyms ( inject=true) or replacing them.
	DoubleMetaphoneFilter(TokenStream input, int maxCodeLength, boolean inject)	Creates a DoubleMetaphoneFilter with the specified maximum code length, and either adding encoded forms as synonyms (inject=true) or replacing them.
	PhoneticFilter(TokenStream in, org.apache.commons.codec.Encoder encoder, boolean inject)	Creates a PhoneticFilter with the specified encoder, and either adding encoded forms as synonyms (inject=true) or replacing them.

Uses of TokenStream in org.apache.lucene.analysis.pl

Methods in org.apache.lucene.analysis.pl that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	PolishAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.pl with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	PolishAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.pt

Subclasses of TokenStream in org.apache.lucene.analysis.pt

Modifier and Type	Class	Description
final class	PortugueseLightStemFilter	A TokenFilter that applies PortugueseLightStemmer to stem Portuguese words.
final class	PortugueseMinimalStemFilter	A TokenFilter that applies PortugueseMinimalStemmer to stem Portuguese words.
final class	PortugueseStemFilter	A TokenFilter that applies PortugueseStemmer to stem Portuguese words.

Methods in org.apache.lucene.analysis.pt that return TokenStream

Modifier and Type	Method	Description
TokenStream	PortugueseLightStemFilterFactory.create(TokenStream input)
TokenStream	PortugueseMinimalStemFilterFactory.create(TokenStream input)
TokenStream	PortugueseStemFilterFactory.create(TokenStream input)
protected TokenStream	PortugueseAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.pt with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	PortugueseLightStemFilterFactory.create(TokenStream input)
TokenStream	PortugueseMinimalStemFilterFactory.create(TokenStream input)
TokenStream	PortugueseStemFilterFactory.create(TokenStream input)
protected TokenStream	PortugueseAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.pt with parameters of type TokenStream

Modifier	Constructor	Description
	PortugueseLightStemFilter(TokenStream input)
	PortugueseMinimalStemFilter(TokenStream input)
	PortugueseStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.reverse

Subclasses of TokenStream in org.apache.lucene.analysis.reverse

Modifier and Type	Class	Description
final class	ReverseStringFilter	Reverse token string, for example "country" => "yrtnuoc".

Methods in org.apache.lucene.analysis.reverse with parameters of type TokenStream

Modifier and Type	Method	Description
ReverseStringFilter	ReverseStringFilterFactory.create(TokenStream in)

Constructors in org.apache.lucene.analysis.reverse with parameters of type TokenStream

Modifier	Constructor	Description
	ReverseStringFilter(TokenStream in)	Create a new ReverseStringFilter that reverses all tokens in the supplied TokenStream.
	ReverseStringFilter(TokenStream in, char marker)	Create a new ReverseStringFilter that reverses and marks all tokens in the supplied TokenStream.

Uses of TokenStream in org.apache.lucene.analysis.ro

Methods in org.apache.lucene.analysis.ro that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	RomanianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.ro with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	RomanianAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.ru

Subclasses of TokenStream in org.apache.lucene.analysis.ru

Modifier and Type	Class	Description
final class	RussianLightStemFilter	A TokenFilter that applies RussianLightStemmer to stem Russian words.

Methods in org.apache.lucene.analysis.ru that return TokenStream

Modifier and Type	Method	Description
TokenStream	RussianLightStemFilterFactory.create(TokenStream input)
protected TokenStream	RussianAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.ru with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	RussianLightStemFilterFactory.create(TokenStream input)
protected TokenStream	RussianAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.ru with parameters of type TokenStream

Modifier	Constructor	Description
	RussianLightStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.shingle

Subclasses of TokenStream in org.apache.lucene.analysis.shingle

Modifier and Type	Class	Description
final class	FixedShingleFilter	A FixedShingleFilter constructs shingles (token n-grams) from a token stream.
final class	ShingleFilter	A ShingleFilter constructs shingles (token n-grams) from a token stream.

Methods in org.apache.lucene.analysis.shingle that return TokenStream

Modifier and Type	Method	Description
TokenStream	FixedShingleFilterFactory.create(TokenStream input)

Methods in org.apache.lucene.analysis.shingle with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	FixedShingleFilterFactory.create(TokenStream input)
ShingleFilter	ShingleFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.shingle with parameters of type TokenStream

Modifier	Constructor	Description
	FixedShingleFilter(TokenStream input, int shingleSize)	Creates a FixedShingleFilter over an input token stream
	FixedShingleFilter(TokenStream input, int shingleSize, String tokenSeparator, String fillerToken)	Creates a FixedShingleFilter over an input token stream
	ShingleFilter(TokenStream input)	Construct a ShingleFilter with default shingle size: 2.
	ShingleFilter(TokenStream input, int maxShingleSize)	Constructs a ShingleFilter with the specified shingle size from the TokenStream input
	ShingleFilter(TokenStream input, int minShingleSize, int maxShingleSize)	Constructs a ShingleFilter with the specified shingle size from the TokenStream input
	ShingleFilter(TokenStream input, String tokenType)	Construct a ShingleFilter with the specified token type for shingle tokens and the default shingle size: 2

Uses of TokenStream in org.apache.lucene.analysis.sinks

Subclasses of TokenStream in org.apache.lucene.analysis.sinks

Modifier and Type	Class	Description
final class	TeeSinkTokenFilter	This TokenFilter provides the ability to set aside attribute states that have already been analyzed.
static final class	TeeSinkTokenFilter.SinkTokenStream	TokenStream output from a tee.

Methods in org.apache.lucene.analysis.sinks that return TokenStream

Modifier and Type	Method	Description
TokenStream	TeeSinkTokenFilter.newSinkTokenStream()	Returns a new TeeSinkTokenFilter.SinkTokenStream that receives all tokens consumed by this stream.

Constructors in org.apache.lucene.analysis.sinks with parameters of type TokenStream

Modifier	Constructor	Description
	TeeSinkTokenFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.snowball

Subclasses of TokenStream in org.apache.lucene.analysis.snowball

Modifier and Type	Class	Description
final class	SnowballFilter	A filter that stems words using a Snowball-generated stemmer.

Methods in org.apache.lucene.analysis.snowball with parameters of type TokenStream

Modifier and Type	Method	Description
TokenFilter	SnowballPorterFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.snowball with parameters of type TokenStream

Modifier	Constructor	Description
	SnowballFilter(TokenStream in, String name)	Construct the named stemming filter.
	SnowballFilter(TokenStream input, SnowballStemmer stemmer)

Uses of TokenStream in org.apache.lucene.analysis.sr

Subclasses of TokenStream in org.apache.lucene.analysis.sr

Modifier and Type	Class	Description
final class	SerbianNormalizationFilter	Normalizes Serbian Cyrillic and Latin characters to "bald" Latin.
final class	SerbianNormalizationRegularFilter	Normalizes Serbian Cyrillic to Latin.

Methods in org.apache.lucene.analysis.sr that return TokenStream

Modifier and Type	Method	Description
TokenStream	SerbianNormalizationFilterFactory.create(TokenStream input)
protected TokenStream	SerbianAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	SerbianNormalizationFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.sr with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	SerbianNormalizationFilterFactory.create(TokenStream input)
protected TokenStream	SerbianAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	SerbianNormalizationFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.sr with parameters of type TokenStream

Modifier	Constructor	Description
	SerbianNormalizationFilter(TokenStream input)
	SerbianNormalizationRegularFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.standard

Subclasses of TokenStream in org.apache.lucene.analysis.standard

Modifier and Type	Class	Description
final class	StandardTokenizer	A grammar-based tokenizer constructed with JFlex.

Methods in org.apache.lucene.analysis.standard that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	StandardAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.standard with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	StandardAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.stempel

Subclasses of TokenStream in org.apache.lucene.analysis.stempel

Modifier and Type	Class	Description
final class	StempelFilter	Transforms the token stream as per the stemming algorithm.

Methods in org.apache.lucene.analysis.stempel that return TokenStream

Modifier and Type	Method	Description
TokenStream	StempelPolishStemFilterFactory.create(TokenStream input)

Methods in org.apache.lucene.analysis.stempel with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	StempelPolishStemFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.stempel with parameters of type TokenStream

Modifier	Constructor	Description
	StempelFilter(TokenStream in, StempelStemmer stemmer)	Create filter using the supplied stemming table.
	StempelFilter(TokenStream in, StempelStemmer stemmer, int minLength)	Create filter using the supplied stemming table.

Uses of TokenStream in org.apache.lucene.analysis.sv

Subclasses of TokenStream in org.apache.lucene.analysis.sv

Modifier and Type	Class	Description
final class	SwedishLightStemFilter	A TokenFilter that applies SwedishLightStemmer to stem Swedish words.
final class	SwedishMinimalStemFilter	A TokenFilter that applies SwedishMinimalStemmer to stem Swedish words.

Methods in org.apache.lucene.analysis.sv that return TokenStream

Modifier and Type	Method	Description
TokenStream	SwedishLightStemFilterFactory.create(TokenStream input)
TokenStream	SwedishMinimalStemFilterFactory.create(TokenStream input)
protected TokenStream	SwedishAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.sv with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	SwedishLightStemFilterFactory.create(TokenStream input)
TokenStream	SwedishMinimalStemFilterFactory.create(TokenStream input)
protected TokenStream	SwedishAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.sv with parameters of type TokenStream

Modifier	Constructor	Description
	SwedishLightStemFilter(TokenStream input)
	SwedishMinimalStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.synonym

Subclasses of TokenStream in org.apache.lucene.analysis.synonym

Modifier and Type	Class	Description
final class	SynonymFilter	Deprecated. Use SynonymGraphFilter instead, but be sure to also use FlattenGraphFilter at index time (not at search time) as well.
final class	SynonymGraphFilter	Applies single- or multi-token synonyms from a SynonymMap to an incoming TokenStream, producing a fully correct graph output.

Methods in org.apache.lucene.analysis.synonym that return TokenStream

Modifier and Type	Method	Description
TokenStream	SynonymFilterFactory.create(TokenStream input)	Deprecated.
TokenStream	SynonymGraphFilterFactory.create(TokenStream input)

Methods in org.apache.lucene.analysis.synonym with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	SynonymFilterFactory.create(TokenStream input)	Deprecated.
TokenStream	SynonymGraphFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.synonym with parameters of type TokenStream

Modifier	Constructor	Description
	SynonymFilter(TokenStream input, SynonymMap synonyms, boolean ignoreCase)	Deprecated.
	SynonymGraphFilter(TokenStream input, SynonymMap synonyms, boolean ignoreCase)	Apply previously built synonyms to incoming tokens.

Uses of TokenStream in org.apache.lucene.analysis.synonym.word2vec

Subclasses of TokenStream in org.apache.lucene.analysis.synonym.word2vec

Modifier and Type	Class	Description
final class	Word2VecSynonymFilter	Applies single-token synonyms from a Word2Vec trained network to an incoming TokenStream.

Methods in org.apache.lucene.analysis.synonym.word2vec that return TokenStream

Modifier and Type	Method	Description
TokenStream	Word2VecSynonymFilterFactory.create(TokenStream input)

Methods in org.apache.lucene.analysis.synonym.word2vec with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	Word2VecSynonymFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.analysis.synonym.word2vec with parameters of type TokenStream

Modifier	Constructor	Description
	Word2VecSynonymFilter(TokenStream input, Word2VecSynonymProvider synonymProvider, int maxSynonymsPerTerm, float minAcceptedSimilarity)	Apply previously built synonymProvider to incoming tokens.

Uses of TokenStream in org.apache.lucene.analysis.ta

Methods in org.apache.lucene.analysis.ta that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	TamilAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.ta with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	TamilAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.te

Subclasses of TokenStream in org.apache.lucene.analysis.te

Modifier and Type	Class	Description
final class	TeluguNormalizationFilter	A TokenFilter that applies TeluguNormalizer to normalize the orthography.
final class	TeluguStemFilter	A TokenFilter that applies TeluguStemmer to stem Telugu words.

Methods in org.apache.lucene.analysis.te that return TokenStream

Modifier and Type	Method	Description
TokenStream	TeluguNormalizationFilterFactory.create(TokenStream input)
TokenStream	TeluguStemFilterFactory.create(TokenStream input)
protected TokenStream	TeluguAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	TeluguNormalizationFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.te with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	TeluguNormalizationFilterFactory.create(TokenStream input)
TokenStream	TeluguStemFilterFactory.create(TokenStream input)
protected TokenStream	TeluguAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	TeluguNormalizationFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.te with parameters of type TokenStream

Modifier	Constructor	Description
	TeluguNormalizationFilter(TokenStream input)
	TeluguStemFilter(TokenStream input)

Uses of TokenStream in org.apache.lucene.analysis.th

Subclasses of TokenStream in org.apache.lucene.analysis.th

Modifier and Type	Class	Description
class	ThaiTokenizer	Tokenizer that use BreakIterator to tokenize Thai text.

Methods in org.apache.lucene.analysis.th that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	ThaiAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.th with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	ThaiAnalyzer.normalize(String fieldName, TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.tr

Subclasses of TokenStream in org.apache.lucene.analysis.tr

Modifier and Type	Class	Description
final class	ApostropheFilter	Strips all characters after an apostrophe (including the apostrophe itself).
final class	TurkishLowerCaseFilter	Normalizes Turkish token text to lower case.

Methods in org.apache.lucene.analysis.tr that return TokenStream

Modifier and Type	Method	Description
TokenStream	ApostropheFilterFactory.create(TokenStream input)
TokenStream	TurkishLowerCaseFilterFactory.create(TokenStream input)
protected TokenStream	TurkishAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	TurkishLowerCaseFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.tr with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	ApostropheFilterFactory.create(TokenStream input)
TokenStream	TurkishLowerCaseFilterFactory.create(TokenStream input)
protected TokenStream	TurkishAnalyzer.normalize(String fieldName, TokenStream in)
TokenStream	TurkishLowerCaseFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.tr with parameters of type TokenStream

Modifier	Constructor	Description
	ApostropheFilter(TokenStream in)
	TurkishLowerCaseFilter(TokenStream in)	Create a new TurkishLowerCaseFilter, that normalizes Turkish token text to lower case.

Uses of TokenStream in org.apache.lucene.analysis.util

Subclasses of TokenStream in org.apache.lucene.analysis.util

Modifier and Type	Class	Description
class	CharTokenizer	An abstract base class for simple, character-oriented tokenizers.
final class	ElisionFilter	Removes elisions from a TokenStream.
class	SegmentingTokenizerBase	Breaks text into sentences with a BreakIterator and allows subclasses to decompose these sentences into words.

Methods in org.apache.lucene.analysis.util that return TokenStream

Modifier and Type	Method	Description
TokenStream	ElisionFilterFactory.create(TokenStream input)
TokenStream	ElisionFilterFactory.normalize(TokenStream input)

Methods in org.apache.lucene.analysis.util with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	ElisionFilterFactory.create(TokenStream input)
TokenStream	ElisionFilterFactory.normalize(TokenStream input)

Constructors in org.apache.lucene.analysis.util with parameters of type TokenStream

Modifier	Constructor	Description
	ElisionFilter(TokenStream input, CharArraySet articles)	Constructs an elision filter with a Set of stop words

Uses of TokenStream in org.apache.lucene.analysis.wikipedia

Subclasses of TokenStream in org.apache.lucene.analysis.wikipedia

Modifier and Type	Class	Description
final class	WikipediaTokenizer	Extension of StandardTokenizer that is aware of Wikipedia syntax.

Uses of TokenStream in org.apache.lucene.classification.document

Methods in org.apache.lucene.classification.document with parameters of type TokenStream

Modifier and Type	Method	Description
protected String[]	SimpleNaiveBayesDocumentClassifier.getTokenArray(TokenStream tokenizedText)	Returns a token array from the TokenStream in input

Uses of TokenStream in org.apache.lucene.document

Subclasses of TokenStream in org.apache.lucene.document

Modifier and Type	Class	Description
private static final class	FeatureField.FeatureTokenStream
private static final class	Field.BinaryTokenStream
private static final class	Field.StringTokenStream

Fields in org.apache.lucene.document declared as TokenStream

Modifier and Type	Field	Description
protected TokenStream	Field.tokenStream	Pre-analyzed tokenStream for indexed fields; this is separate from fieldsData because you are allowed to have both; eg maybe field has a String value but you customize how it's tokenized

Methods in org.apache.lucene.document that return TokenStream

Modifier and Type	Method	Description
TokenStream	FeatureField.tokenStream(Analyzer analyzer, TokenStream reuse)
TokenStream	Field.tokenStream(Analyzer analyzer, TokenStream reuse)
TokenStream	ShapeDocValuesField.tokenStream(Analyzer analyzer, TokenStream reuse)	TokenStreams are not yet supported
TokenStream	Field.tokenStreamValue()	The TokenStream for this field to be used when indexing, or null.

Methods in org.apache.lucene.document with parameters of type TokenStream

Modifier and Type	Method	Description
void	Field.setTokenStream(TokenStream tokenStream)	Expert: sets the token stream to be used for indexing and causes isIndexed() and isTokenized() to return true.
TokenStream	FeatureField.tokenStream(Analyzer analyzer, TokenStream reuse)
TokenStream	Field.tokenStream(Analyzer analyzer, TokenStream reuse)
TokenStream	ShapeDocValuesField.tokenStream(Analyzer analyzer, TokenStream reuse)	TokenStreams are not yet supported

Constructors in org.apache.lucene.document with parameters of type TokenStream

Modifier	Constructor	Description
	Field(String name, TokenStream tokenStream, IndexableFieldType type)	Create field with TokenStream value.
	TextField(String name, TokenStream stream)	Creates a new un-stored TextField with TokenStream value.

Uses of TokenStream in org.apache.lucene.index

Fields in org.apache.lucene.index declared as TokenStream

Modifier and Type	Field	Description
(package private) TokenStream	IndexingChain.PerField.tokenStream

Methods in org.apache.lucene.index that return TokenStream

Modifier and Type	Method	Description
TokenStream	IndexableField.tokenStream(Analyzer analyzer, TokenStream reuse)	Creates the TokenStream used for indexing this field.
TokenStream	IndexingChain.ReservedField.tokenStream(Analyzer analyzer, TokenStream reuse)

Methods in org.apache.lucene.index with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	IndexableField.tokenStream(Analyzer analyzer, TokenStream reuse)	Creates the TokenStream used for indexing this field.
TokenStream	IndexingChain.ReservedField.tokenStream(Analyzer analyzer, TokenStream reuse)

Uses of TokenStream in org.apache.lucene.index.memory

Methods in org.apache.lucene.index.memory that return TokenStream

Modifier and Type	Method	Description
<T> TokenStream	MemoryIndex.keywordTokenStream(Collection<T> keywords)	Convenience method; Creates and returns a token stream that generates a token for each keyword in the given collection, "as is", without any transforming text analysis.

Methods in org.apache.lucene.index.memory with parameters of type TokenStream

Modifier and Type	Method	Description
void	MemoryIndex.addField(String fieldName, TokenStream stream)	Iterates over the given token stream and adds the resulting terms to the index; Equivalent to adding a tokenized, indexed, termVectorStored, unstored, Lucene Field.
void	MemoryIndex.addField(String fieldName, TokenStream stream, int positionIncrementGap)	Iterates over the given token stream and adds the resulting terms to the index; Equivalent to adding a tokenized, indexed, termVectorStored, unstored, Lucene Field.
void	MemoryIndex.addField(String fieldName, TokenStream tokenStream, int positionIncrementGap, int offsetGap)	Iterates over the given token stream and adds the resulting terms to the index; Equivalent to adding a tokenized, indexed, termVectorStored, unstored, Lucene Field.
private void	MemoryIndex.storeTerms(MemoryIndex.Info info, TokenStream tokenStream, int positionIncrementGap, int offsetGap)

Uses of TokenStream in org.apache.lucene.misc.document

Methods in org.apache.lucene.misc.document that return TokenStream

Modifier and Type	Method	Description
TokenStream	LazyDocument.LazyField.tokenStream(Analyzer analyzer, TokenStream reuse)

Methods in org.apache.lucene.misc.document with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	LazyDocument.LazyField.tokenStream(Analyzer analyzer, TokenStream reuse)

Uses of TokenStream in org.apache.lucene.monitor

Subclasses of TokenStream in org.apache.lucene.monitor

Modifier and Type	Class	Description
(package private) final class	SuffixingNGramTokenFilter
(package private) class	TermsEnumTokenStream	A TokenStream created from a TermsEnum

Methods in org.apache.lucene.monitor that return TokenStream

Modifier and Type	Method	Description
default TokenStream	CustomQueryHandler.wrapTermStream(String field, TokenStream in)	Adds additional processing to the TokenStream over a document's terms index
TokenStream	RegexpQueryHandler.wrapTermStream(String field, TokenStream ts)

Methods in org.apache.lucene.monitor with parameters of type TokenStream

Modifier and Type	Method	Description
default TokenStream	CustomQueryHandler.wrapTermStream(String field, TokenStream in)	Adds additional processing to the TokenStream over a document's terms index
TokenStream	RegexpQueryHandler.wrapTermStream(String field, TokenStream ts)

Constructors in org.apache.lucene.monitor with parameters of type TokenStream

Modifier	Constructor	Description
	SuffixingNGramTokenFilter(TokenStream input, String suffix, String wildcardToken, int maxTokenLength)	Creates SuffixingNGramTokenFilter.

Uses of TokenStream in org.apache.lucene.queries.intervals

Methods in org.apache.lucene.queries.intervals with parameters of type TokenStream

Modifier and Type	Method	Description
static IntervalsSource	Intervals.analyzedText(TokenStream tokenStream, int maxGaps, boolean ordered)	Returns intervals that correspond to tokens from the provided TokenStream.
private static List<IntervalsSource>	IntervalBuilder.analyzeGraph(TokenStream source)
private static IntervalsSource	IntervalBuilder.analyzeSynonyms(TokenStream ts, int maxGaps, boolean ordered)
private static IntervalsSource	IntervalBuilder.analyzeTerm(TokenStream ts)
private static List<IntervalsSource>	IntervalBuilder.analyzeTerms(TokenStream ts)

Uses of TokenStream in org.apache.lucene.sandbox.search

Methods in org.apache.lucene.sandbox.search with parameters of type TokenStream

Modifier and Type	Method	Description
TermAutomatonQuery	TokenStreamToTermAutomatonQuery.toQuery(String field, TokenStream in)	Pulls the graph (including PositionLengthAttribute) from the provided TokenStream, and creates the corresponding automaton where arcs are bytes (or Unicode code points if unicodeArcs = true) from each term.

Uses of TokenStream in org.apache.lucene.search.highlight

Subclasses of TokenStream in org.apache.lucene.search.highlight

Modifier and Type	Class	Description
(package private) final class	LimitTokenOffsetFilter	This is a simplified version of org.apache.lucene.analysis.miscellaneous.LimitTokenOffsetFilter to prevent a dependency on analysis-common.jar.
final class	OffsetLimitTokenFilter	This TokenFilter limits the number of tokens while indexing by adding up the current offset.
final class	TokenStreamFromTermVector	TokenStream created from a term vector field.

Fields in org.apache.lucene.search.highlight declared as TokenStream

Modifier and Type	Field	Description
private TokenStream	WeightedSpanTermExtractor.tokenStream

Methods in org.apache.lucene.search.highlight that return TokenStream

Modifier and Type	Method	Description
static TokenStream	TokenSources.getAnyTokenStream(IndexReader reader, int docId, String field, Analyzer analyzer)	Deprecated.
static TokenStream	TokenSources.getAnyTokenStream(IndexReader reader, int docId, String field, Document document, Analyzer analyzer)	Deprecated.
static TokenStream	TokenSources.getTermVectorTokenStreamOrNull(String field, Fields tvFields, int maxStartOffset)	Get a token stream by un-inverting the term vector.
static TokenStream	TokenSources.getTokenStream(String field, String contents, Analyzer analyzer)	Deprecated.
static TokenStream	TokenSources.getTokenStream(String field, Fields tvFields, String text, Analyzer analyzer, int maxStartOffset)	Get a token stream from either un-inverting a term vector if possible, or by analyzing the text.
static TokenStream	TokenSources.getTokenStream(Document doc, String field, Analyzer analyzer)	Deprecated.
static TokenStream	TokenSources.getTokenStream(IndexReader reader, int docId, String field, Analyzer analyzer)	Deprecated.
static TokenStream	TokenSources.getTokenStream(Terms tpv)	Deprecated.
static TokenStream	TokenSources.getTokenStream(Terms vector, boolean tokenPositionsGuaranteedContiguous)	Deprecated.
TokenStream	WeightedSpanTermExtractor.getTokenStream()	Returns the tokenStream which may have been wrapped in a CachingTokenFilter.
static TokenStream	TokenSources.getTokenStreamWithOffsets(IndexReader reader, int docId, String field)	Deprecated.
TokenStream	QueryScorer.init(TokenStream tokenStream)
TokenStream	QueryTermScorer.init(TokenStream tokenStream)
TokenStream	Scorer.init(TokenStream tokenStream)	Called to init the Scorer with a TokenStream.
private TokenStream	QueryScorer.initExtractor(TokenStream tokenStream)

Methods in org.apache.lucene.search.highlight with parameters of type TokenStream

Modifier and Type	Method	Description
final String	Highlighter.getBestFragment(TokenStream tokenStream, String text)	Highlights chosen terms in a text, extracting the most relevant section.
final String[]	Highlighter.getBestFragments(TokenStream tokenStream, String text, int maxNumFragments)	Highlights chosen terms in a text, extracting the most relevant sections.
final String	Highlighter.getBestFragments(TokenStream tokenStream, String text, int maxNumFragments, String separator)	Highlights terms in the text , extracting the most relevant sections and concatenating the chosen fragments with a separator (typically "...").
final TextFragment[]	Highlighter.getBestTextFragments(TokenStream tokenStream, String text, boolean mergeContiguousFragments, int maxNumFragments)	Low level api to get the most relevant (formatted) sections of the document.
Map<String,WeightedSpanTerm>	WeightedSpanTermExtractor.getWeightedSpanTerms(Query query, float boost, TokenStream tokenStream)	Creates a Map of WeightedSpanTerms from the given Query and TokenStream.
Map<String,WeightedSpanTerm>	WeightedSpanTermExtractor.getWeightedSpanTerms(Query query, float boost, TokenStream tokenStream, String fieldName)	Creates a Map of WeightedSpanTerms from the given Query and TokenStream.
Map<String,WeightedSpanTerm>	WeightedSpanTermExtractor.getWeightedSpanTermsWithScores(Query query, float boost, TokenStream tokenStream, String fieldName, IndexReader reader)	Creates a Map of WeightedSpanTerms from the given Query and TokenStream.
TokenStream	QueryScorer.init(TokenStream tokenStream)
TokenStream	QueryTermScorer.init(TokenStream tokenStream)
TokenStream	Scorer.init(TokenStream tokenStream)	Called to init the Scorer with a TokenStream.
private TokenStream	QueryScorer.initExtractor(TokenStream tokenStream)
void	Fragmenter.start(String originalText, TokenStream tokenStream)	Initializes the Fragmenter.
void	NullFragmenter.start(String s, TokenStream tokenStream)
void	SimpleFragmenter.start(String originalText, TokenStream stream)
void	SimpleSpanFragmenter.start(String originalText, TokenStream tokenStream)

Constructors in org.apache.lucene.search.highlight with parameters of type TokenStream

Modifier	Constructor	Description
(package private)	LimitTokenOffsetFilter(TokenStream input, int maxStartOffset)
	OffsetLimitTokenFilter(TokenStream input, int offsetLimit)
	TokenGroup(TokenStream tokenStream)

Uses of TokenStream in org.apache.lucene.search.suggest.analyzing

Subclasses of TokenStream in org.apache.lucene.search.suggest.analyzing

Modifier and Type	Class	Description
final class	SuggestStopFilter	Like StopFilter except it will not remove the last token if that token was not followed by some token separator.

Methods in org.apache.lucene.search.suggest.analyzing that return TokenStream

Modifier and Type	Method	Description
TokenStream	SuggestStopFilterFactory.create(TokenStream input)

Methods in org.apache.lucene.search.suggest.analyzing with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	SuggestStopFilterFactory.create(TokenStream input)

Constructors in org.apache.lucene.search.suggest.analyzing with parameters of type TokenStream

Modifier	Constructor	Description
	SuggestStopFilter(TokenStream input, CharArraySet stopWords)	Sole constructor.

Uses of TokenStream in org.apache.lucene.search.suggest.document

Subclasses of TokenStream in org.apache.lucene.search.suggest.document

Modifier and Type	Class	Description
final class	CompletionTokenStream	A ConcatenateGraphFilter but we can set the payload and provide access to config options.
private static final class	ContextSuggestField.PrefixTokenFilter	The ContextSuggestField.PrefixTokenFilter wraps a TokenStream and adds a set prefixes ahead.

Fields in org.apache.lucene.search.suggest.document declared as TokenStream

Modifier and Type	Field	Description
(package private) final TokenStream	CompletionTokenStream.inputTokenStream

Methods in org.apache.lucene.search.suggest.document that return TokenStream

Modifier and Type	Method	Description
TokenStream	SuggestField.tokenStream(Analyzer analyzer, TokenStream reuse)

Methods in org.apache.lucene.search.suggest.document with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	SuggestField.tokenStream(Analyzer analyzer, TokenStream reuse)
protected CompletionTokenStream	ContextSuggestField.wrapTokenStream(TokenStream stream)
protected CompletionTokenStream	SuggestField.wrapTokenStream(TokenStream stream)	Wraps a stream with a CompletionTokenStream.

Constructors in org.apache.lucene.search.suggest.document with parameters of type TokenStream

Modifier	Constructor	Description
(package private)	CompletionTokenStream(TokenStream inputTokenStream)
(package private)	CompletionTokenStream(TokenStream inputTokenStream, boolean preserveSep, boolean preservePositionIncrements, int maxGraphExpansions)
	PrefixTokenFilter(TokenStream input, char separator, Iterable<CharSequence> prefixes)	Create a new ContextSuggestField.PrefixTokenFilter

Uses of TokenStream in org.apache.lucene.search.uhighlight

Subclasses of TokenStream in org.apache.lucene.search.uhighlight

Modifier and Type	Class	Description
private static final class	AnalysisOffsetStrategy.MultiValueTokenStream	Wraps an Analyzer and string text that represents multiple values delimited by a specified character.

Fields in org.apache.lucene.search.uhighlight declared as TokenStream

Modifier and Type	Field	Description
(package private) TokenStream	TokenStreamOffsetStrategy.TokenStreamOffsetsEnum.stream

Methods in org.apache.lucene.search.uhighlight that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	AnalysisOffsetStrategy.tokenStream(String content)

Methods in org.apache.lucene.search.uhighlight with parameters of type TokenStream

Modifier and Type	Method	Description
private static FilteringTokenFilter	MemoryIndexOffsetStrategy.newKeepWordFilter(TokenStream tokenStream, CharArrayMatcher matcher)

Constructors in org.apache.lucene.search.uhighlight with parameters of type TokenStream

Modifier	Constructor	Description
private	MultiValueTokenStream(TokenStream subTokenStream, String fieldName, Analyzer indexAnalyzer, String content, char splitChar, int splitCharIdx)
(package private)	TokenStreamOffsetsEnum(TokenStream ts, CharArrayMatcher[] matchers)

Uses of TokenStream in org.apache.lucene.util

Methods in org.apache.lucene.util with parameters of type TokenStream

Modifier and Type	Method	Description
protected Query	QueryBuilder.analyzeBoolean(String field, TokenStream stream)	Creates simple boolean query from the cached tokenstream contents
protected Query	QueryBuilder.analyzeGraphBoolean(String field, TokenStream source, BooleanClause.Occur operator)	Creates a boolean query from a graph token stream.
protected Query	QueryBuilder.analyzeGraphPhrase(TokenStream source, String field, int phraseSlop)	Creates graph phrase query from the tokenstream contents
protected Query	QueryBuilder.analyzeMultiBoolean(String field, TokenStream stream, BooleanClause.Occur operator)	Creates complex boolean query from the cached tokenstream contents
protected Query	QueryBuilder.analyzeMultiPhrase(String field, TokenStream stream, int slop)	Creates complex phrase query from the cached tokenstream contents
protected Query	QueryBuilder.analyzePhrase(String field, TokenStream stream, int slop)	Creates simple phrase query from the cached tokenstream contents
protected Query	QueryBuilder.analyzeTerm(String field, TokenStream stream)	Creates simple term query from the cached tokenstream contents
protected Query	QueryBuilder.createFieldQuery(TokenStream source, BooleanClause.Occur operator, String field, boolean quoted, int phraseSlop)	Creates a query from a token stream.

Uses of TokenStream in org.apache.lucene.util.graph

Subclasses of TokenStream in org.apache.lucene.util.graph

Modifier and Type	Class	Description
private class	GraphTokenStreamFiniteStrings.FiniteStringsTokenStream

Methods in org.apache.lucene.util.graph that return types with arguments of type TokenStream

Modifier and Type	Method	Description
Iterator<TokenStream>	GraphTokenStreamFiniteStrings.getFiniteStrings()	Get all finite strings from the automaton.
Iterator<TokenStream>	GraphTokenStreamFiniteStrings.getFiniteStrings(int startState, int endState)	Get all finite strings that start at startState and end at endState.

Methods in org.apache.lucene.util.graph with parameters of type TokenStream

Modifier and Type	Method	Description
private Automaton	GraphTokenStreamFiniteStrings.build(TokenStream in)	Build an automaton from the provided TokenStream.

Constructors in org.apache.lucene.util.graph with parameters of type TokenStream

Modifier	Constructor	Description
	GraphTokenStreamFiniteStrings(TokenStream in)