Module org.apache.lucene.core

Package org.apache.lucene.codecs.lucene90.blocktree

Class Lucene90BlockTreeTermsReader

java.lang.Object

org.apache.lucene.index.Fields

org.apache.lucene.codecs.FieldsProducer

org.apache.lucene.codecs.lucene90.blocktree.Lucene90BlockTreeTermsReader

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<String>

public final class Lucene90BlockTreeTermsReader
extends FieldsProducer

A block-based terms index and dictionary that assigns terms to variable length blocks according to how they share prefixes. The terms index is a prefix trie whose leaves are term blocks. The advantage of this approach is that seekExact is often able to determine a term cannot exist without doing any IO, and intersection with Automata is very fast. Note that this terms dictionary has its own fixed terms index (ie, it does not support a pluggable terms index implementation).

NOTE: this terms dictionary supports min/maxItemsPerBlock during indexing to control how much memory the terms index uses.

The data structure used by this implementation is very similar to a burst trie (http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.18.3499), but with added logic to break up too-large blocks of all terms sharing a given prefix into smaller ones.

Use CheckIndex with the -verbose option to see summary statistics on the blocks in the dictionary.

See Lucene90BlockTreeTermsWriter.

Field Summary

Fields

Modifier and Type	Field	Description
private final FieldInfos	fieldInfos
private final List<String>	fieldList
private final IntObjectHashMap<FieldReader>	fieldMap
(package private) static final Outputs<BytesRef>	FST_OUTPUTS
(package private) final IndexInput	indexIn
(package private) static final BytesRef	NO_OUTPUT
(package private) static final int	OUTPUT_FLAG_HAS_TERMS
(package private) static final int	OUTPUT_FLAG_IS_FLOOR
(package private) static final int	OUTPUT_FLAGS_MASK
(package private) static final int	OUTPUT_FLAGS_NUM_BITS
(package private) final PostingsReaderBase	postingsReader
(package private) final String	segment
(package private) static final String	TERMS_CODEC_NAME
(package private) static final String	TERMS_EXTENSION	Extension of terms file
(package private) static final String	TERMS_INDEX_CODEC_NAME
(package private) static final String	TERMS_INDEX_EXTENSION	Extension of terms index file
(package private) static final String	TERMS_META_CODEC_NAME
(package private) static final String	TERMS_META_EXTENSION	Extension of terms meta file
(package private) final IndexInput	termsIn
(package private) final int	version
static final int	VERSION_CURRENT	Current terms format.
static final int	VERSION_FST_CONTINUOUS_ARCS	The version that specialize arc store for continuous label in FST.
static final int	VERSION_MSB_VLONG_OUTPUT	Version that encode output as MSB VLong for better outputs sharing in FST, see GITHUB#12620.
static final int	VERSION_START	Initial terms format.

Fields inherited from class org.apache.lucene.index.Fields

EMPTY_ARRAY

Constructor Summary

Constructors

Constructor	Description
Lucene90BlockTreeTermsReader(PostingsReaderBase postingsReader, SegmentReadState state)	Sole constructor.

Method Summary

Modifier and Type	Method	Description
void	checkIntegrity()	Checks consistency of this reader.
void	close()
Iterator<String>	iterator()	Returns an iterator that will step through all fields names.
private static BytesRef	readBytesRef(IndexInput in)
int	size()	Returns the number of fields or -1 if the number of distinct field names is unknown.
private static List<String>	sortFieldNames(IntObjectHashMap<FieldReader> fieldMap, FieldInfos fieldInfos)
Terms	terms(String field)	Get the Terms for this field.
String	toString()

Methods inherited from class org.apache.lucene.codecs.FieldsProducer

getMergeInstance

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Field Details

FST_OUTPUTS

static final Outputs<BytesRef> FST_OUTPUTS

NO_OUTPUT

static final BytesRef NO_OUTPUT

OUTPUT_FLAGS_NUM_BITS

static final int OUTPUT_FLAGS_NUM_BITS

See Also:

• Constant Field Values

OUTPUT_FLAGS_MASK

static final int OUTPUT_FLAGS_MASK

See Also:

• Constant Field Values

OUTPUT_FLAG_IS_FLOOR

static final int OUTPUT_FLAG_IS_FLOOR

See Also:

• Constant Field Values

OUTPUT_FLAG_HAS_TERMS

static final int OUTPUT_FLAG_HAS_TERMS

See Also:

• Constant Field Values

TERMS_EXTENSION

static final String TERMS_EXTENSION

Extension of terms file

See Also:

• Constant Field Values

TERMS_CODEC_NAME

static final String TERMS_CODEC_NAME

See Also:

• Constant Field Values

VERSION_START

public static final int VERSION_START

Initial terms format.

See Also:

• Constant Field Values

VERSION_MSB_VLONG_OUTPUT

public static final int VERSION_MSB_VLONG_OUTPUT

Version that encode output as MSB VLong for better outputs sharing in FST, see GITHUB#12620.

See Also:

• Constant Field Values

VERSION_FST_CONTINUOUS_ARCS

public static final int VERSION_FST_CONTINUOUS_ARCS

The version that specialize arc store for continuous label in FST.

See Also:

• Constant Field Values

VERSION_CURRENT

public static final int VERSION_CURRENT

Current terms format.

See Also:

• Constant Field Values

TERMS_INDEX_EXTENSION

static final String TERMS_INDEX_EXTENSION

Extension of terms index file

See Also:

• Constant Field Values

TERMS_INDEX_CODEC_NAME

static final String TERMS_INDEX_CODEC_NAME

See Also:

• Constant Field Values

TERMS_META_EXTENSION

static final String TERMS_META_EXTENSION

Extension of terms meta file

See Also:

• Constant Field Values

TERMS_META_CODEC_NAME

static final String TERMS_META_CODEC_NAME

See Also:

• Constant Field Values

termsIn

final IndexInput termsIn

indexIn

final IndexInput indexIn

postingsReader

final PostingsReaderBase postingsReader

fieldInfos

private final FieldInfos fieldInfos

fieldMap

private final IntObjectHashMap<FieldReader> fieldMap

fieldList

private final List<String> fieldList

segment

final String segment

version

final int version

Constructor Details

Lucene90BlockTreeTermsReader

public Lucene90BlockTreeTermsReader(PostingsReaderBase postingsReader,
 SegmentReadState state)
                             throws IOException

Sole constructor.

Throws:

IOException

Method Details

readBytesRef

private static BytesRef readBytesRef(IndexInput in)
                              throws IOException

Throws:

IOException

sortFieldNames

private static List<String> sortFieldNames(IntObjectHashMap<FieldReader> fieldMap,
 FieldInfos fieldInfos)

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in class FieldsProducer

Throws:

IOException

iterator

public Iterator<String> iterator()

Description copied from class: Fields

Returns an iterator that will step through all fields names. This will not return null.

Specified by:

iterator in interface Iterable<String>

Specified by:

iterator in class Fields

terms

public Terms terms(String field)
            throws IOException

Description copied from class: Fields

Get the Terms for this field. This will return null if the field does not exist.

Specified by:

terms in class Fields

Throws:

IOException

size

public int size()

Description copied from class: Fields

Returns the number of fields or -1 if the number of distinct field names is unknown. If >= 0, Fields.iterator() will return as many field names.

Specified by:

size in class Fields

checkIntegrity

public void checkIntegrity()
                    throws IOException

Description copied from class: FieldsProducer

Checks consistency of this reader.

Note that this may be costly in terms of I/O, e.g. may involve computing a checksum value against large data files.

Specified by:

checkIntegrity in class FieldsProducer

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object