Ted Xu 50da48a30d enhance: adding mix compaction first prioritizer (#36956) 4 天之前
..
allocator 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
broker 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
session f8ac91f1db fix: datacoord stuck at stopping progress (#36852) 5 天之前
OWNERS 12f50cb22c [skip ci]Update OWNERS files (#11898) 2 年之前
README.md 08ca551a75 [skip ci]Change etcd to lowercase (#9983) 3 年之前
analyze_meta.go 7bf40694fd fix: Record the nodeID before assigning tasks (#36371) 3 周之前
analyze_meta_test.go 7bf40694fd fix: Record the nodeID before assigning tasks (#36371) 3 周之前
build_index_policy.go 26f06dd732 Format the code (#27275) 1 年之前
channel.go f12e368a76 fix: Fill nil schema so that Milvus can watch channel for those upgraded from 2.2 to 2.4 #35695 (#35694) 1 月之前
channel_manager.go d51a808851 fix: Rootcoord stuck at graceful stop progress (#36880) 5 天之前
channel_manager_factory.go 314f4d995b enhance: Tidy dc channel manager (#34515) 3 月之前
channel_manager_test.go d51a808851 fix: Rootcoord stuck at graceful stop progress (#36880) 5 天之前
channel_store.go c45f38aa61 enhance: Update protobuf-go to protobuf-go v2 (#34394) 2 月之前
channel_store_test.go f12e368a76 fix: Fill nil schema so that Milvus can watch channel for those upgraded from 2.2 to 2.4 #35695 (#35694) 1 月之前
cluster.go 582d2eec79 enhance: Move datanode/indexnode manager to session pkg (#35634) 2 月之前
cluster_test.go 582d2eec79 enhance: Move datanode/indexnode manager to session pkg (#35634) 2 月之前
compaction.go 605e0890fb enhance: enable parallel execution of L0 compactions (#36816) 5 天之前
compaction_l0_view.go 04edb07d82 enhance: Add deltaRowCount in l0 compaction (#33997) 4 月之前
compaction_l0_view_test.go 04edb07d82 enhance: Add deltaRowCount in l0 compaction (#33997) 4 月之前
compaction_policy_clustering.go c992a61a23 enhance: Separate allocator pkg in datacoord (#35622) 2 月之前
compaction_policy_clustering_test.go c992a61a23 enhance: Separate allocator pkg in datacoord (#35622) 2 月之前
compaction_policy_l0.go b172ea1093 fix: Remove enableLevelZeroSegment config (#36535) 5 天之前
compaction_policy_l0_test.go c992a61a23 enhance: Separate allocator pkg in datacoord (#35622) 2 月之前
compaction_policy_single.go ef1832ff9c enhance: enable manual compaction for collections without indexes (#36577) 2 周之前
compaction_policy_single_test.go c992a61a23 enhance: Separate allocator pkg in datacoord (#35622) 2 月之前
compaction_queue.go 50da48a30d enhance: adding mix compaction first prioritizer (#36956) 4 天之前
compaction_queue_test.go 50da48a30d enhance: adding mix compaction first prioritizer (#36956) 4 天之前
compaction_task.go 7ff41697f9 enhance: make slotUsage a field of CompactionTask (#36510) 3 周之前
compaction_task_clustering.go 04c306e63f fix: Fix clustering compaction task leak (#36800) 5 天之前
compaction_task_clustering_test.go 04c306e63f fix: Fix clustering compaction task leak (#36800) 5 天之前
compaction_task_l0.go 277c50fd31 fix:add log in mixCompactionTask and set fail/timeout task to clean (#35970) 5 天之前
compaction_task_l0_test.go 277c50fd31 fix:add log in mixCompactionTask and set fail/timeout task to clean (#35970) 5 天之前
compaction_task_meta.go d230b91bd1 enhance: Add PreallocatedSegmentIDs for the compaction task (#36734) 1 周之前
compaction_task_meta_test.go 52fa668f5d fix: fix collection is compacting logic (#34855) 3 月之前
compaction_task_mix.go 277c50fd31 fix:add log in mixCompactionTask and set fail/timeout task to clean (#35970) 5 天之前
compaction_task_mix_test.go 277c50fd31 fix:add log in mixCompactionTask and set fail/timeout task to clean (#35970) 5 天之前
compaction_task_test.go 582d2eec79 enhance: Move datanode/indexnode manager to session pkg (#35634) 2 月之前
compaction_test.go 605e0890fb enhance: enable parallel execution of L0 compactions (#36816) 5 天之前
compaction_trigger.go d230b91bd1 enhance: Add PreallocatedSegmentIDs for the compaction task (#36734) 1 周之前
compaction_trigger_test.go d230b91bd1 enhance: Add PreallocatedSegmentIDs for the compaction task (#36734) 1 周之前
compaction_trigger_v2.go d230b91bd1 enhance: Add PreallocatedSegmentIDs for the compaction task (#36734) 1 周之前
compaction_trigger_v2_test.go c992a61a23 enhance: Separate allocator pkg in datacoord (#35622) 2 月之前
compaction_view.go 04edb07d82 enhance: Add deltaRowCount in l0 compaction (#33997) 4 月之前
const.go 0907d76253 enhance: pass partition key scalar info if enabled when build vector index (#29931) 9 月之前
errors.go 6539a5ae2c Refine DataCoord status (#27262) 1 年之前
errors_test.go 26f06dd732 Format the code (#27275) 1 年之前
garbage_collector.go ef1832ff9c enhance: enable manual compaction for collections without indexes (#36577) 2 周之前
garbage_collector_test.go 2c9bb4dfa3 feat: Support stats task to sort segment by PK (#35054) 1 月之前
go_channel_singleton.go 9d8d332c88 fix: Fix improper use of offset in HybridSearch (#36244) 1 月之前
handler.go ef1832ff9c enhance: enable manual compaction for collections without indexes (#36577) 2 周之前
handler_test.go 4e0ea39235 fix: Remove neighbors if compactTo is unindexed (#36503) 2 周之前
import_checker.go 0fc2a4aa53 enhance: Optimize import scheduling and add time cost metric (#36601) 1 周之前
import_checker_test.go 0fc2a4aa53 enhance: Optimize import scheduling and add time cost metric (#36601) 1 周之前
import_job.go 0fc2a4aa53 enhance: Optimize import scheduling and add time cost metric (#36601) 1 周之前
import_meta.go 0fc2a4aa53 enhance: Optimize import scheduling and add time cost metric (#36601) 1 周之前
import_meta_test.go a434d33e75 feat: Add import scheduler and manager (#29367) 7 月之前
import_scheduler.go 0fc2a4aa53 enhance: Optimize import scheduling and add time cost metric (#36601) 1 周之前
import_scheduler_test.go 0fc2a4aa53 enhance: Optimize import scheduling and add time cost metric (#36601) 1 周之前
import_task.go 5ec4163d0f feat: support bm25 logs mixcompaction (#36072) 1 周之前
import_util.go 0fc2a4aa53 enhance: Optimize import scheduling and add time cost metric (#36601) 1 周之前
import_util_test.go 1f47d5510b fix: Fix import segments leak in segment manager (#36602) 2 周之前
index_engine_version_manager.go 8cf2cf5c94 enhance: Add `go-deadlock` as unittest only dependency (#33063) 5 月之前
index_engine_version_manager_test.go 0f14d18201 Optimize the codec code of session (#27360) 1 年之前
index_meta.go 3de57ec4fa enhance: add vector index mgr to remove vector index type dependency (#36843) 5 天之前
index_meta_test.go 7bf40694fd fix: Record the nodeID before assigning tasks (#36371) 3 周之前
index_service.go 3de57ec4fa enhance: add vector index mgr to remove vector index type dependency (#36843) 5 天之前
index_service_test.go d2c774fb6d fix: Return all compactTo segments after support split (#36361) 1 月之前
job_manager.go a61668c77e feat: Introduce stats task for import (#35868) 1 月之前
job_manager_test.go f7b811450d feat: add enable_tokenizer params to VarChar field (#36480) 1 周之前
meta.go f3b6792a25 enhance: Enhance segment log (#36848) 1 周之前
meta_test.go ecb2b242e2 enhance: Add sorted for segment info (#36469) 3 周之前
meta_util.go 43ae6791e6 Make VChannelInfo read compatible with 2.0.2 (#17995) 2 年之前
metrics_info.go 22cce44afc fix: metrics stored_index_files_size is never cleared (#36160) 1 月之前
metrics_info_test.go 582d2eec79 enhance: Move datanode/indexnode manager to session pkg (#35634) 2 月之前
mock_channel_store.go 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
mock_channelmanager.go 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
mock_cluster.go 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
mock_compaction_meta.go 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
mock_compaction_plan_context.go 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
mock_handler.go 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
mock_index_engine_version_manager.go 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
mock_job_manager.go 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
mock_segment_manager.go 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
mock_subcluster.go 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
mock_test.go d230b91bd1 enhance: Add PreallocatedSegmentIDs for the compaction task (#36734) 1 周之前
mock_trigger_manager.go 6851738fd1 fix: fix `make generate-mockery` panic with go1.22 (#36830) 5 天之前
partition_stats_meta.go 81773bfadf enhance: add commit time in partitionStats proto (#35125) 2 月之前
partition_stats_meta_test.go 81773bfadf enhance: add commit time in partitionStats proto (#35125) 2 月之前
policy.go 6e9fbd1630 enhance: Solve channel unbalance on datanode (#34984) 2 月之前
policy_test.go f12e368a76 fix: Fill nil schema so that Milvus can watch channel for those upgraded from 2.2 to 2.4 #35695 (#35694) 1 月之前
segment_allocation_policy.go 196a7986b3 enhance: Change the fixed value to a ratio for clustering segment size (#35076) 2 月之前
segment_allocation_policy_test.go 196a7986b3 enhance: Change the fixed value to a ratio for clustering segment size (#35076) 2 月之前
segment_info.go 4e0ea39235 fix: Remove neighbors if compactTo is unindexed (#36503) 2 周之前
segment_info_test.go d2c774fb6d fix: Return all compactTo segments after support split (#36361) 1 月之前
segment_manager.go 1f47d5510b fix: Fix import segments leak in segment manager (#36602) 2 周之前
segment_manager_test.go 1f47d5510b fix: Fix import segments leak in segment manager (#36602) 2 周之前
segment_operator.go 8395c8a8db enhance: Update stats task to optional (#35947) 1 月之前
segment_operator_test.go 6387403639 fix: Prevent clone when selecting segments from meta (#30928) 7 月之前
server.go d51a808851 fix: Rootcoord stuck at graceful stop progress (#36880) 5 天之前
server_test.go d51a808851 fix: Rootcoord stuck at graceful stop progress (#36880) 5 天之前
services.go 0fc2a4aa53 enhance: Optimize import scheduling and add time cost metric (#36601) 1 周之前
services_test.go 2c9bb4dfa3 feat: Support stats task to sort segment by PK (#35054) 1 月之前
stats_task_meta.go 7bf40694fd fix: Record the nodeID before assigning tasks (#36371) 3 周之前
stats_task_meta_test.go 7bf40694fd fix: Record the nodeID before assigning tasks (#36371) 3 周之前
sync_segments_scheduler.go f8ac91f1db fix: datacoord stuck at stopping progress (#36852) 5 天之前
sync_segments_scheduler_test.go f8ac91f1db fix: datacoord stuck at stopping progress (#36852) 5 天之前
task_analyze.go 7bf40694fd fix: Record the nodeID before assigning tasks (#36371) 3 周之前
task_index.go 3de57ec4fa enhance: add vector index mgr to remove vector index type dependency (#36843) 5 天之前
task_scheduler.go 7bf40694fd fix: Record the nodeID before assigning tasks (#36371) 3 周之前
task_scheduler_test.go 3de57ec4fa enhance: add vector index mgr to remove vector index type dependency (#36843) 5 天之前
task_stats.go 5ec4163d0f feat: support bm25 logs mixcompaction (#36072) 1 周之前
task_stats_test.go 7bf40694fd fix: Record the nodeID before assigning tasks (#36371) 3 周之前
types.go 7bf40694fd fix: Record the nodeID before assigning tasks (#36371) 3 周之前
util.go 3de57ec4fa enhance: add vector index mgr to remove vector index type dependency (#36843) 5 天之前
util_test.go c992a61a23 enhance: Separate allocator pkg in datacoord (#35622) 2 月之前

README.md

Data Coordinator

Data cooridnator(datacoord for short) is the component to organize DataNodes and segments allocations.

Dependency

  • KV store: a kv store has all the meta info datacoord needs to operate. (etcd)
  • Message stream: a message stream to communicate statistics information with data nodes. (Pulsar)
  • Root Coordinator: timestamp, id and meta source.
  • Data Node(s): could be an instance or a cluster, actual worker group handles data modification operations.