存储、索引和管理由深度神经网络和其他机器学习 (ML) 模型生成的海量嵌入向量数据库。

smellthemoon 6bedc7e8c8 fix: not set valid_data in bitmap index when mmap (#37023) 15 小时之前
.github 00e7e8c661 enhance: enable macos checker (#36833) 1 周之前
build ac736ad472 enhance: friendly for docker cache (#35789) 1 月之前
ci 0be81d95aa enhance: ut for cpp code (#36941) 1 天之前
client e6cb7d6695 enhance: [GoSDK] Add collection alias & rename API (#36990) 1 天之前
cmd 3de57ec4fa enhance: add vector index mgr to remove vector index type dependency (#36843) 5 天之前
configs 50da48a30d enhance: adding mix compaction first prioritizer (#36956) 4 天之前
deployments a05a37a583 enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214) 3 周之前
docs 35cc7dde63 enhance: Add clustering compaction user guide doc (#35427) 2 月之前
githooks 12f50cb22c [skip ci]Update OWNERS files (#11898) 2 年之前
internal 6bedc7e8c8 fix: not set valid_data in bitmap index when mmap (#37023) 15 小时之前
pkg 50da48a30d enhance: adding mix compaction first prioritizer (#36956) 4 天之前
scripts aee61973fb [automated] Bump milvus version to v2.4.13-hotfix (#36947) 5 天之前
tests 36147b10f5 test: Add insert and upsert related cases for null and default value support (#36932) 17 小时之前
tools aab060aeff upgrade Go 1.18 in go.mod (#18165) 2 年之前
.clang-format bdd6bc7695 Re-format cpp code (#22513) 1 年之前
.clang-tidy 48422dd4c5 Fix spawn too many threads (#26293) 1 年之前
.clang-tidy-ignore a13638b172 Add cpplint and optimize the check of c++ code 3 年之前
.contributors 9a5b23fa4f [skip ci]Update all-contributors (#6358) 3 年之前
.devcontainer.json da628863bc Update main.yaml for Github Action (#6381) 3 年之前
.dockerignore 755a592b08 Add dockerignore for building milvus image (#28102) 11 月之前
.env a05a37a583 enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214) 3 周之前
.gitignore 39c7e06bc5 enhance: add message and msgstream msgpack adaptor (#34874) 3 月之前
.golangci.yml dfda9f0478 enhance: Add depguard rules to ban deprecated proto lib (#35140) 2 月之前
.pre-commit-config.yaml 4c6aa95ec8 enhance: [skip e2e]remove slow pre commit (#31397) 7 月之前
CODE_OF_CONDUCT.md 3655792fac [skip ci]Format markdown doc for CODE_OF_CONDUCT.md (#10042) 3 年之前
CODE_REVIEW.md 531a51bd75 [skip e2e]Fix a grammar issue (#15086) 2 年之前
COMMAND_HELP.md 91ab8373b3 Update docs for run-cpu-e2e (#18520) 2 年之前
COMMITTERS d011f750d8 Update COMMITTERS 1 年之前
CONTRIBUTING.md 901e98a868 Update CONTRIBUTING.md for mockery generation (#26842) 1 年之前
DEVELOPMENT.md bfe5c651d6 doc: Update go version in DEVELOPMENT.md (#36796) 1 周之前
LICENSE 0d4daa22e0 Update README 3 年之前
MAINTAINERS 92fa4df9c5 Update maintainers list 1 年之前
Makefile d51a808851 fix: Rootcoord stuck at graceful stop progress (#36880) 5 天之前
OWNERS 91554e3548 [skip ci]Add GitHub label for compilation changes (#12894) 2 年之前
OWNERS_ALIASES c54de88e2b [skip e2e] Update OWNERS_ALIASES for approve label (#22223) 1 年之前
README.md 4d08eec1af Update all contributors 6 天之前
README_CN.md 4d08eec1af Update all contributors 6 天之前
codecov.yml 060649b8aa Refactor task scheduler and executor (#20828) 1 年之前
docker-compose.yml a05a37a583 enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214) 3 周之前
go.mod 903450f5c6 enhance: add ts support for iterator(#22718) (#36572) 6 天之前
go.sum 903450f5c6 enhance: add ts support for iterator(#22718) (#36572) 6 天之前
milvus20vs1x.md 6d1a8666c4 docs: update table, show feat status with badge (#19624) (#19624) 2 年之前
rules.go 26f06dd732 Format the code (#27275) 1 年之前

README.md

milvus banner

license license docker-pull-count

What is Milvus?

milvus-logo

Milvus is an open-source vector database built to power embedding similarity search and AI applications. Milvus makes unstructured data search more accessible, and provides a consistent user experience regardless of the deployment environment.

Milvus 2.0 is a cloud-native vector database with storage and computation separated by design. All components in this refactored version of Milvus are stateless to enhance elasticity and flexibility. For more architecture details, see Milvus Architecture Overview.

Milvus was released under the open-source Apache License 2.0 in October 2019. It is currently a graduate project under LF AI & Data Foundation.

Key features

Millisecond search on trillion vector datasets Average latency measured in milliseconds on trillion vector datasets.

Simplified unstructured data management
  • Rich APIs designed for data science workflows.
  • Consistent user experience across laptop, local cluster, and cloud.
  • Embed real-time search and analytics into virtually any application.
  • Reliable, always on vector database Milvus’ built-in replication and failover/failback features ensure data and applications can maintain business continuity in the event of a disruption.

    Highly scalable and elastic Component-level scalability makes it possible to scale up and down on demand. Milvus can autoscale at a component level according to the load type, making resource scheduling much more efficient.

    Hybrid search Since Milvus 2.4, we introduced multi-vector support and a hybrid search framework, which means users can bring in several vector fields (up to 10) into a single collection. These vectors in different columns represent diverse facets of data, originating from different embedding models or undergoing distinct processing methods. The results of hybrid searches are integrated using reranking strategies, such as Reciprocal Rank Fusion (RRF) and Weighted Scoring.

    This feature is particularly useful in comprehensive search scenarios, such as identifying the most similar person in a vector library based on various attributes like pictures, voice, fingerprints, etc. For details, refer to Hybrid Search for more.

    Unified Lambda structure Milvus combines stream and batch processing for data storage to balance timeliness and efficiency. Its unified interface makes vector similarity search a breeze.

    Community supported, industry recognized With over 1,000 enterprise users, 27,000+ stars on GitHub, and an active open-source community, you’re not alone when you use Milvus. As a graduate project under the LF AI & Data Foundation, Milvus has institutional support.

    Quick start

    Start with Zilliz Cloud

    Zilliz Cloud is a fully managed service on cloud and the simplest way to deploy LF AI Milvus®, See Zilliz Cloud and start your free trial.

    Install Milvus

    Build Milvus from source code

    Check the requirements first.

    Linux systems (Ubuntu 20.04 or later recommended):

    go: >= 1.21
    cmake: >= 3.26.4
    gcc: 9.5
    python: > 3.8 and  <= 3.11
    

    MacOS systems with x86_64 (Big Sur 11.5 or later recommended):

    go: >= 1.21
    cmake: >= 3.26.4
    llvm: >= 15
    python: > 3.8 and  <= 3.11
    

    MacOS systems with Apple Silicon (Monterey 12.0.1 or later recommended):

    go: >= 1.21 (Arch=ARM64)
    cmake: >= 3.26.4
    llvm: >= 15
    python: > 3.8 and  <= 3.11
    

    Clone Milvus repo and build.

    # Clone github repository.
    $ git clone https://github.com/milvus-io/milvus.git
    
    # Install third-party dependencies.
    $ cd milvus/
    $ ./scripts/install_deps.sh
    
    # Compile Milvus.
    $ make
    

    For the full story, see developer's documentation.

    IMPORTANT The master branch is for the development of Milvus v2.0. On March 9th, 2021, we released Milvus v1.0, the first stable version of Milvus with long-term support. To use Milvus v1.0, switch to branch 1.0.

    Milvus 2.0 vs. 1.x: Cloud-native, distributed architecture, highly scalable, and more

    See Milvus 2.0 vs. 1.x for more information.

    Real world demos

    Image search Chatbots Chemical structure search

    Image Search

    Images made searchable. Instantaneously return the most similar images from a massive database.

    Chatbots

    Interactive digital customer service that saves users time and businesses money.

    Chemical Structure Search

    Blazing fast similarity search, substructure search, or superstructure search for a specified molecule.

    Bootcamps

    Milvus bootcamp is designed to expose users to both the simplicity and depth of the vector database. Discover how to run benchmark tests as well as build similarity search applications spanning chatbots, recommendation systems, reverse image search, molecular search, and much more.

    Contributing

    Contributions to Milvus are welcome from everyone. See Guidelines for Contributing for details on submitting patches and the contribution workflow. See our community repository to learn about our governance and access more community resources.

    All contributors




    Documentation

    For guidance on installation, development, deployment, and administration, check out Milvus Docs. For technical milestones and enhancement proposals, check out milvus confluence

    SDK

    The implemented SDK and its API documentation are listed below:

    Attu

    Attu provides an intuitive and efficient GUI for Milvus.

    Community

    Join the Milvus community on Discord to share your suggestions, advice, and questions with our engineering team.

    You can also check out our FAQ page to discover solutions or answers to your issues or questions.

    Subscribe to Milvus mailing lists:

    Follow Milvus on social media:

    Reference

    Reference to cite when you use Milvus in a research paper:

    @inproceedings{2021milvus,
      title={Milvus: A Purpose-Built Vector Data Management System},
      author={Wang, Jianguo and Yi, Xiaomeng and Guo, Rentong and Jin, Hai and Xu, Peng and Li, Shengjun and Wang, Xiangyu and Guo, Xiangzhou and Li, Chengming and Xu, Xiaohai and others},
      booktitle={Proceedings of the 2021 International Conference on Management of Data},
      pages={2614--2627},
      year={2021}
    }
    
    @article{2022manu,
      title={Manu: a cloud native vector database management system},
      author={Guo, Rentong and Luan, Xiaofan and Xiang, Long and Yan, Xiao and Yi, Xiaomeng and Luo, Jigao and Cheng, Qianya and Xu, Weizhi and Luo, Jiarui and Liu, Frank and others},
      journal={Proceedings of the VLDB Endowment},
      volume={15},
      number={12},
      pages={3548--3561},
      year={2022},
      publisher={VLDB Endowment}
    }
    

    Acknowledgments

    Milvus adopts dependencies from the following:

    • Thanks to FAISS for the excellent search library.
    • Thanks to etcd for providing great open-source key-value store tools.
    • Thanks to Pulsar for its wonderful distributed pub-sub messaging system.
    • Thanks to Tantivy for its full-text search engine library written in Rust.
    • Thanks to RocksDB for the powerful storage engines.

    Milvus is adopted by following opensource project:

    • Towhee a flexible, application-oriented framework for computing embedding vectors over unstructured data.
    • Haystack an open source NLP framework that leverages Transformer models
    • Langchain Building applications with LLMs through composability
    • LLamaIndex a data framework for your LLM applications
    • GPTCache a library for creating semantic cache to store responses from LLM queries.