Module org.apache.lucene.analysis.kuromoji
Package org.apache.lucene.analysis.ja.util

Class BinaryDictionaryWriter

java.lang.Object
org.apache.lucene.analysis.ja.util.BinaryDictionaryWriter
Direct Known Subclasses:
TokenInfoDictionaryWriter, UnknownDictionaryWriter
abstract class BinaryDictionaryWriter extends Object

  • Field Details

    • ID_LIMIT

      private static final int ID_LIMIT
      See Also:

    • implClazz

      private final Class<? extends BinaryDictionary> implClazz

    • buffer

      protected ByteBuffer buffer

    • targetMapEndOffset

      private int targetMapEndOffset

    • lastWordId

      private int lastWordId

    • lastSourceId

      private int lastSourceId

    • targetMap

      private int[] targetMap

    • targetMapOffsets

      private int[] targetMapOffsets

    • posDict

      private final ArrayList<String> posDict

  • Constructor Details

  • Method Details

    • put

      public int put(String[] entry)
      put the entry in map
      Returns:
      current position of buffer, which will be wordId of next entry

    • isKatakana

      private boolean isKatakana(String s)

    • writeKatakana

      private void writeKatakana(String s)

    • toKatakana

      private String toKatakana(String s)

    • sharedPrefix

      private static int sharedPrefix(String left, String right)

    • addMapping

      void addMapping(int sourceId, int wordId)

    • getBaseFileName

      final String getBaseFileName()

    • write

      public void write(Path baseDir) throws IOException
      Write dictionary in file Dictionary format is: [Size of dictionary(int)], [entry:{left id(short)}{right id(short)}{word cost(short)}{length of pos info(short)}{pos info(char)}], [entry...], [entry...].....
      Throws:
      IOException - if an I/O error occurs writing the dictionary files

    • writeTargetMap

      private void writeTargetMap(Path path) throws IOException
      Throws:
      IOException

    • writePosDict

      private void writePosDict(Path path) throws IOException
      Throws:
      IOException

    • writeDictionary

      private void writeDictionary(Path path) throws IOException
      Throws:
      IOException