Popular GitHub Repositories for Elasticsearch Tokenizer

Unsupervised text tokenizer for Neural Network-based text generation.

Chinese version of GPT2 training code, using BERT tokenizer.

精选机器学习，NLP，图像识别， 深度学习等人工智能领域学习资料，搜索，推荐，广告系统架构及算法技术资料整理。算法大牛笔记汇总

Represent, send, store and search multimodal data

Stock market analyzer and predictor using Elasticsearch, Twitter, News headlines and Python natural language processing and sentiment analysis

?基于SpringCloud 2023.x + Dubbo 3.x + AI构建的模拟秒杀微服务项目，集成了Elasticsearch?、Gateway、Mybatis-Plus、Sharding-JDBC等常用开源组件

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

中文对话0.2B小模型（ChatLM-Chinese-0.2B），开源所有数据集来源、数据清洗、tokenizer训练、模型预训练、SFT指令微调、RLHF优化等流程的全部代码。支持下游任务sft微调，给出三元组信息抽取微调示例。

Underthesea - Vietnamese NLP Toolkit

Persian NLP Toolkit

GNES is Generic Neural Elastic Search, a cloud-native semantic search system based on deep neural network.

Solves basic Russian NLP tasks, API for lower level Natasha projects

Unsupervised text tokenizer focused on computational efficiency

Notebooks & Example Apps for Search & AI Applications with Elasticsearch

Jcseg is a light weight NLP framework developed with Java. Provide CJK and English segmentation based on MMSEG algorithm, With also keywords extraction, key sentence extraction, summary extraction implemented based on TEXTRANK algorithm. Jcseg had a build-in http server and search modules for lucene,solr,elasticsearch,opensearch

Elasticsearch with BERT for advanced document search.

iBOT :robot:: Image BERT Pre-Training with Online Tokenizer (ICLR 2022)

JTokkit is a Java tokenizer library designed for use with OpenAI models.

NBoost is a scalable, search-api-boosting platform for deploying transformer models to improve the relevance of search results on different platforms (i.e. Elasticsearch)

A Vietnamese natural language processing toolkit (NAACL 2018)