test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 11:59:00 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
b5c67948b7
milvus/internal/datacoord
History
yihao.dai b5c67948b7
enhance: Enhance and modify the return content of ImportV2 (#31192) 
1. The Import APIs now provide detailed progress information for each
imported file, including details such as file name, file size, progress,
and more.
2. The APIs now return the collection name and the completion time.
3. Other modifications include changing jobID to jobId and other similar
adjustments.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-03-13 19:51:03 +08:00
..
broker enhance: index meta use independent rather than global meta lock (#30869) 2024-03-04 16:56:59 +08:00
allocator_test.go Decoupling client and server API in types interface (#27186) 2023-09-26 09:57:25 +08:00
allocator.go feat: Add import scheduler and manager (#29367) 2024-03-01 18:31:02 +08:00
build_index_policy.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
channel_checker_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
channel_checker.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
channel_manager_factory.go enhance: Remove not in use policies (#29448) 2023-12-28 10:38:46 +08:00
channel_manager_test.go enhance: Change ChannelManager to interface (#29300) 2023-12-25 19:24:46 +08:00
channel_manager.go enhance:change some logs (#29579) 2024-01-05 16:12:48 +08:00
channel_store_test.go Change channel to Interface (#27839) 2023-11-13 11:16:18 +08:00
channel_store.go Change channel to Interface (#27839) 2023-11-13 11:16:18 +08:00
channel.go Change channel to Interface (#27839) 2023-11-13 11:16:18 +08:00
cluster_test.go enhance: Change ChannelManager to interface (#29300) 2023-12-25 19:24:46 +08:00
cluster.go feat: Add import scheduler and manager (#29367) 2024-03-01 18:31:02 +08:00
compaction_l0_view_test.go enhance: Add force trigger (#30641) 2024-02-19 18:40:50 +08:00
compaction_l0_view.go fix: Trigger l0 compaction when l0 views don't change (#30729) 2024-03-05 16:37:00 +08:00
compaction_scheduler_test.go enhance: Add CompactionTaskNum metrics (#29518) 2023-12-28 15:46:55 +08:00
compaction_scheduler.go fix: Trigger l0 compaction when l0 views don't change (#30729) 2024-03-05 16:37:00 +08:00
compaction_test.go fix: Skip invalid compaction plan (#31045) 2024-03-06 21:36:59 +08:00
compaction_trigger_test.go enhance: Prevent the backlog of channelCP update tasks, perform batch updates of channelCPs (#30941) 2024-03-07 20:39:02 +08:00
compaction_trigger_v2_test.go fix: Trigger l0 compaction when l0 views don't change (#30729) 2024-03-05 16:37:00 +08:00
compaction_trigger_v2.go fix: Trigger l0 compaction when l0 views don't change (#30729) 2024-03-05 16:37:00 +08:00
compaction_trigger.go enhance: Check channel cp lag before generate compaction task (#30997) 2024-03-05 13:39:01 +08:00
compaction_view_manager_test.go fix: Trigger l0 compaction when l0 views don't change (#30729) 2024-03-05 16:37:00 +08:00
compaction_view_manager.go fix: Trigger l0 compaction when l0 views don't change (#30729) 2024-03-05 16:37:00 +08:00
compaction_view.go fix: Trigger l0 compaction when l0 views don't change (#30729) 2024-03-05 16:37:00 +08:00
compaction.go fix: Skip invalid compaction plan (#31045) 2024-03-06 21:36:59 +08:00
const.go enhance: pass partition key scalar info if enabled when build vector index (#29931) 2024-01-24 00:04:55 +08:00
errors_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
errors.go Refine DataCoord status (#27262) 2023-09-26 17:15:27 +08:00
garbage_collector_test.go enhance: Prevent the backlog of channelCP update tasks, perform batch updates of channelCPs (#30941) 2024-03-07 20:39:02 +08:00
garbage_collector.go enhance: index meta use independent rather than global meta lock (#30869) 2024-03-04 16:56:59 +08:00
handler.go enhance: Explicitly pass LevelZero segment ids in vchan info (#29612) 2024-01-04 16:46:45 +08:00
import_checker_test.go feat: Add import scheduler and manager (#29367) 2024-03-01 18:31:02 +08:00
import_checker.go enhance: Enhance and modify the return content of ImportV2 (#31192) 2024-03-13 19:51:03 +08:00
import_job.go enhance: Enhance and modify the return content of ImportV2 (#31192) 2024-03-13 19:51:03 +08:00
import_meta_test.go feat: Add import scheduler and manager (#29367) 2024-03-01 18:31:02 +08:00
import_meta.go feat: Add import scheduler and manager (#29367) 2024-03-01 18:31:02 +08:00
import_scheduler_test.go feat: Add import scheduler and manager (#29367) 2024-03-01 18:31:02 +08:00
import_scheduler.go enhance: Enhance and modify the return content of ImportV2 (#31192) 2024-03-13 19:51:03 +08:00
import_task.go enhance: Enhance and modify the return content of ImportV2 (#31192) 2024-03-13 19:51:03 +08:00
import_util_test.go enhance: Enhance and modify the return content of ImportV2 (#31192) 2024-03-13 19:51:03 +08:00
import_util.go enhance: Enhance and modify the return content of ImportV2 (#31192) 2024-03-13 19:51:03 +08:00
index_builder_test.go enhance: index meta use independent rather than global meta lock (#30869) 2024-03-04 16:56:59 +08:00
index_builder.go enhance: index meta use independent rather than global meta lock (#30869) 2024-03-04 16:56:59 +08:00
index_engine_version_manager_test.go Optimize the codec code of session (#27360) 2023-10-01 10:33:30 +08:00
index_engine_version_manager.go feat: trigger compaction to handle index version (#28442) 2023-11-21 09:26:22 +08:00
index_meta_test.go enhance: index meta use independent rather than global meta lock (#30869) 2024-03-04 16:56:59 +08:00
index_meta.go fix: filter mmap key when checking index params (#31030) 2024-03-06 16:03:00 +08:00
index_service_test.go enhance: Add ListIndexes API from datacoord (#31104) 2024-03-07 17:37:01 +08:00
index_service.go fix: merge index params when creating index (#31127) 2024-03-11 17:31:03 +08:00
indexnode_manager_test.go Refine state check (#27541) 2023-10-11 21:01:35 +08:00
indexnode_manager.go Decoupling client and server API in types interface (#27186) 2023-09-26 09:57:25 +08:00
meta_test.go enhance: Prevent the backlog of channelCP update tasks, perform batch updates of channelCPs (#30941) 2024-03-07 20:39:02 +08:00
meta_util.go
meta.go enhance: Prevent the backlog of channelCP update tasks, perform batch updates of channelCPs (#30941) 2024-03-07 20:39:02 +08:00
metrics_info_test.go Refine state check (#27541) 2023-10-11 21:01:35 +08:00
metrics_info.go disable auto balance when old node exists (#28191) 2023-11-07 14:02:20 +08:00
mock_allocator_test.go enhance: index meta use independent rather than global meta lock (#30869) 2024-03-04 16:56:59 +08:00
mock_channel_store.go Change channel to Interface (#27839) 2023-11-13 11:16:18 +08:00
mock_channelmanager.go enhance: Add CompactionTaskNum metrics (#29518) 2023-12-28 15:46:55 +08:00
mock_cluster.go enhance: index meta use independent rather than global meta lock (#30869) 2024-03-04 16:56:59 +08:00
mock_compaction_meta.go enhance: Refine compaction interfaces to support major compaction (#30632) 2024-02-19 20:52:50 +08:00
mock_compaction_plan_context.go fix: Separate schedule and check results loop (#28692) 2023-11-29 10:50:29 +08:00
mock_handler.go Change channel to Interface (#27839) 2023-11-13 11:16:18 +08:00
mock_index_engine_version_manager.go feat: trigger compaction to handle index version (#28442) 2023-11-21 09:26:22 +08:00
mock_scheduler.go enhance: Add CompactionTaskNum metrics (#29518) 2023-12-28 15:46:55 +08:00
mock_segment_manager.go enhance: index meta use independent rather than global meta lock (#30869) 2024-03-04 16:56:59 +08:00
mock_session_manager.go enhance: index meta use independent rather than global meta lock (#30869) 2024-03-04 16:56:59 +08:00
mock_test.go feat: Add import scheduler and manager (#29367) 2024-03-01 18:31:02 +08:00
mock_trigger_manager.go feat: Add Compaction views and triggers (#27906) 2023-11-23 17:30:25 +08:00
OWNERS
policy_test.go enhance: Remove not in use policies (#29448) 2023-12-28 10:38:46 +08:00
policy.go enhance: Remove not in use policies (#29448) 2023-12-28 10:38:46 +08:00
README.md
segment_allocation_policy_test.go Adjust datacoord for L0 Delta (#28021) 2023-11-06 15:26:16 +08:00
segment_allocation_policy.go fix: Decoupling importing segment from flush process (#30402) 2024-02-03 13:01:12 +08:00
segment_info_test.go fix: starve lock caused by slow GetCompactionTo method when too much segments (#30963) 2024-03-05 10:04:59 +08:00
segment_info.go fix: starve lock caused by slow GetCompactionTo method when too much segments (#30963) 2024-03-05 10:04:59 +08:00
segment_manager_test.go feat: Add import scheduler and manager (#29367) 2024-03-01 18:31:02 +08:00
segment_manager.go fix: add GetSegments optimization to avoid meta mutex competition (#31025) 2024-03-05 14:47:00 +08:00
segment_operator_test.go fix: Prevent clone when selecting segments from meta (#30928) 2024-03-01 13:59:02 +08:00
segment_operator.go fix: Prevent clone when selecting segments from meta (#30928) 2024-03-01 13:59:02 +08:00
server_test.go enhance: Prevent the backlog of channelCP update tasks, perform batch updates of channelCPs (#30941) 2024-03-07 20:39:02 +08:00
server.go fix: Trigger l0 compaction when l0 views don't change (#30729) 2024-03-05 16:37:00 +08:00
services_test.go enhance: index meta use independent rather than global meta lock (#30869) 2024-03-04 16:56:59 +08:00
services.go enhance: Enhance and modify the return content of ImportV2 (#31192) 2024-03-13 19:51:03 +08:00
session_manager_test.go feat: Add import scheduler and manager (#29367) 2024-03-01 18:31:02 +08:00
session_manager.go feat: Add import scheduler and manager (#29367) 2024-03-01 18:31:02 +08:00
session.go Decoupling client and server API in types interface (#27186) 2023-09-26 09:57:25 +08:00
util_test.go feat: Add import scheduler and manager (#29367) 2024-03-01 18:31:02 +08:00
util.go enhance: index meta use independent rather than global meta lock (#30869) 2024-03-04 16:56:59 +08:00

README.md

Data Coordinator

Data cooridnator(datacoord for short) is the component to organize DataNodes and segments allocations.

Dependency

  • KV store: a kv store has all the meta info datacoord needs to operate. (etcd)
  • Message stream: a message stream to communicate statistics information with data nodes. (Pulsar)
  • Root Coordinator: timestamp, id and meta source.
  • Data Node(s): could be an instance or a cluster, actual worker group handles data modification operations.