Module org.apache.lucene.analysis.common

Package org.apache.lucene.analysis.tr

Class TurkishLowerCaseFilter

java.lang.Object

org.apache.lucene.util.AttributeSource

org.apache.lucene.analysis.TokenStream

org.apache.lucene.analysis.TokenFilter

org.apache.lucene.analysis.tr.TurkishLowerCaseFilter

All Implemented Interfaces:

Closeable, AutoCloseable, Unwrappable<TokenStream>

public final class TurkishLowerCaseFilter
extends TokenFilter

Normalizes Turkish token text to lower case.

Turkish and Azeri have unique casing behavior for some characters. This filter applies Turkish lowercase rules. For more information, see http://en.wikipedia.org/wiki/Turkish_dotted_and_dotless_I

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

AttributeSource.State

Field Summary

Fields

Modifier and Type	Field	Description
private static final int	COMBINING_DOT_ABOVE
private static final int	LATIN_CAPITAL_LETTER_I
private static final int	LATIN_SMALL_LETTER_DOTLESS_I
private static final int	LATIN_SMALL_LETTER_I
private final CharTermAttribute	termAtt

Fields inherited from class org.apache.lucene.analysis.TokenFilter

input

Fields inherited from class org.apache.lucene.analysis.TokenStream

DEFAULT_TOKEN_ATTRIBUTE_FACTORY

Constructor Summary

Constructors

Constructor	Description
TurkishLowerCaseFilter(TokenStream in)	Create a new TurkishLowerCaseFilter, that normalizes Turkish token text to lower case.

Method Summary

Modifier and Type	Method	Description
private int	delete(char[] s, int pos, int len)	delete a character in-place.
final boolean	incrementToken()	Consumers (i.e., IndexWriter) use this method to advance the stream to the next token.
private boolean	isBeforeDot(char[] s, int pos, int len)	lookahead for a combining dot above.

Methods inherited from class org.apache.lucene.analysis.TokenFilter

close, end, reset, unwrap

Methods inherited from class org.apache.lucene.util.AttributeSource

addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Details

LATIN_CAPITAL_LETTER_I

private static final int LATIN_CAPITAL_LETTER_I

See Also:

• Constant Field Values

LATIN_SMALL_LETTER_I

private static final int LATIN_SMALL_LETTER_I

See Also:

• Constant Field Values

LATIN_SMALL_LETTER_DOTLESS_I

private static final int LATIN_SMALL_LETTER_DOTLESS_I

See Also:

• Constant Field Values

COMBINING_DOT_ABOVE

private static final int COMBINING_DOT_ABOVE

See Also:

• Constant Field Values

termAtt

private final CharTermAttribute termAtt

Constructor Details

TurkishLowerCaseFilter

public TurkishLowerCaseFilter(TokenStream in)

Create a new TurkishLowerCaseFilter, that normalizes Turkish token text to lower case.

Parameters:

in - TokenStream to filter

Method Details

incrementToken

public final boolean incrementToken()
                             throws IOException

Description copied from class: TokenStream

Consumers (i.e., IndexWriter) use this method to advance the stream to the next token. Implementing classes must implement this method and update the appropriate AttributeImpls with the attributes of the next token.

The producer must make no assumptions about the attributes after the method has been returned: the caller may arbitrarily change it. If the producer needs to preserve the state for subsequent calls, it can use AttributeSource.captureState() to create a copy of the current attribute state.

This method is called for every token of a document, so an efficient implementation is crucial for good performance. To avoid calls to AttributeSource.addAttribute(Class) and AttributeSource.getAttribute(Class), references to all AttributeImpls that this stream uses should be retrieved during instantiation.

To ensure that filters and consumers know which attributes are available, the attributes must be added during instantiation. Filters and consumers are not required to check for availability of attributes in TokenStream.incrementToken().

Specified by:

incrementToken in class TokenStream

Returns:

false for end of stream; true otherwise

Throws:

IOException

isBeforeDot

private boolean isBeforeDot(char[] s,
 int pos,
 int len)

lookahead for a combining dot above. other NSMs may be in between.

delete

private int delete(char[] s,
 int pos,
 int len)

delete a character in-place. rarely happens, only if COMBINING_DOT_ABOVE is found after an i