Module org.apache.lucene.core

Package org.apache.lucene.util.hnsw

Class HnswConcurrentMergeBuilder.ConcurrentMergeWorker

java.lang.Object

org.apache.lucene.util.hnsw.HnswGraphBuilder

org.apache.lucene.util.hnsw.HnswConcurrentMergeBuilder.ConcurrentMergeWorker

All Implemented Interfaces:

HnswBuilder

Enclosing class:

HnswConcurrentMergeBuilder

private static final class HnswConcurrentMergeBuilder.ConcurrentMergeWorker
extends HnswGraphBuilder

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.util.hnsw.HnswGraphBuilder

HnswGraphBuilder.GraphBuilderKnnCollector

Field Summary

Fields

Modifier and Type	Field	Description
private int	batchSize
private final BitSet	initializedNodes
private final AtomicInteger	workProgress	A common AtomicInteger shared among all workers, used for tracking what's the next vector to be added to the graph.

Fields inherited from class org.apache.lucene.util.hnsw.HnswGraphBuilder

DEFAULT_BEAM_WIDTH, DEFAULT_MAX_CONN, hnsw, HNSW_COMPONENT, randSeed

Constructor Summary

Constructors

Modifier	Constructor	Description
private	ConcurrentMergeWorker(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, OnHeapHnswGraph hnsw, BitSet initializedNodes, AtomicInteger workProgress)

Method Summary

Modifier and Type	Method	Description
void	addGraphNode(int node)	Inserts a doc with vector value to the graph
private int	getStartPos(int maxOrd)	Reserve the work by atomically increment the workProgress
private void	run(int maxOrd)	This method first try to "reserve" part of work by calling getStartPos(int) and then calling HnswGraphBuilder.addVectors(int, int) to actually add the nodes to the graph.

Methods inherited from class org.apache.lucene.util.hnsw.HnswGraphBuilder

addVectors, build, create, create, getGraph, setInfoStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

workProgress

private final AtomicInteger workProgress

A common AtomicInteger shared among all workers, used for tracking what's the next vector to be added to the graph.

initializedNodes

private final BitSet initializedNodes

batchSize

private int batchSize

Constructor Details

ConcurrentMergeWorker

private ConcurrentMergeWorker(RandomVectorScorerSupplier scorerSupplier,
 int M,
 int beamWidth,
 long seed,
 OnHeapHnswGraph hnsw,
 BitSet initializedNodes,
 AtomicInteger workProgress)
                       throws IOException

Throws:

IOException

Method Details

run

private void run(int maxOrd)
          throws IOException

This method first try to "reserve" part of work by calling getStartPos(int) and then calling HnswGraphBuilder.addVectors(int, int) to actually add the nodes to the graph. By doing this we are able to dynamically allocate the work to multiple workers and try to make all of them finishing around the same time.

Throws:

IOException

getStartPos

private int getStartPos(int maxOrd)

Reserve the work by atomically increment the workProgress

addGraphNode

public void addGraphNode(int node)
                  throws IOException

Description copied from interface: HnswBuilder

Inserts a doc with vector value to the graph

Specified by:

addGraphNode in interface HnswBuilder

Overrides:

addGraphNode in class HnswGraphBuilder

Throws:

IOException