Commit Graph

14587 Commits

Author SHA1 Message Date
Jade Flute
9537394971
Add Milvus Log to issue template (#16425) (#16676)
Signed-off-by: zhangdiandian <1635468471@qq.com>
2022-04-29 11:34:32 +08:00
zhuwenxing
61f50122ca
[skip e2e]Add tasks to check compatibility (#16710)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2022-04-29 09:17:48 +08:00
binbin
222112d98a
Enable and modify test cases after code fix (#16720)
Signed-off-by: Binbin Lv <binbin.lv@zilliz.com>
2022-04-29 09:11:47 +08:00
jaime
68b1b82faf
Remove DataKV interface (#16692)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-28 21:03:47 +08:00
groot
21b5973253
Refine list import task api (#16690)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-28 17:21:47 +08:00
jaime
2205b878b1
Refine log in query_shard.go (#16693)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-28 17:13:48 +08:00
jingkl
f07e0eb8da
[test]Modify the search testcase (#16709)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-04-28 17:09:48 +08:00
yah01
206b554d49
Sync the distribution of segments to shard leaders (#16685)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-28 16:01:48 +08:00
Xiaofan
ed51ffd50f
Fix Compact Entity ttl param (#16665)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-28 11:51:47 +08:00
jaime
c0a73dbe98
Using a temporary dir instead of a specified dir within embed etcd test (#16694)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-28 11:17:47 +08:00
yah01
7017756601
Validate the number of replicas for load collection/partitions request (#16697)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-28 10:54:00 +08:00
zhenshan.cao
0a953948af
Simplify monitoring metrics (#16687)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-27 23:03:47 +08:00
congqixia
3a6db2faeb
Fix handling segment change logic (#16695)
Dispatch segmentChangeInfo to ShardCluster leader
Hold segment remove before search is done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-27 22:23:46 +08:00
yah01
bb6cd4b484
Disable the unit test for the case of filter out unavailable leaders of GetShardLeaders (#16688)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-27 22:15:47 +08:00
Xiaofan
be5d1f1243
Fix Datacoord ut when etcd is slow (#16689)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-27 21:49:47 +08:00
ThreadDao
aed819d06d
[test] Update load multi replicas cases (#16681)
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2022-04-27 21:23:46 +08:00
xiyichan
056fd9aa17
Correct wrong words (#16490)
Signed-off-by: chenlifeng <chenlifeng0995@wezhuiyi.com>

Co-authored-by: chenlifeng <chenlifeng0995@wezhuiyi.com>
2022-04-27 19:51:46 +08:00
Letian Jiang
2879796b42
Validate partitionIDs & segmentIDs in query request (#16684)
* check partitions if they are released or unloaded
    * check segments if their collection/partition(s) are released or unloaded

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-04-27 19:23:48 +08:00
XuanYang-cn
35b7267edb
Add ut for task_policies (#16663)
See also: #16652

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-27 16:55:46 +08:00
yah01
68ef720d6b
Filter out the unavailable shard leaders (#16677)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-27 16:27:46 +08:00
congqixia
0bf176a7e1
Add SyncReplicaSegments interface for QueryNode (#16672)
Add SyncReplicaSegments for LoadCollection/LoadPartition child task
This API notifies related ShardCluster to record segment Loaded
Preventing the Etcd events is delayed, which will cause some segment not search

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-27 10:41:46 +08:00
Xiaofan
b657b10eb5
Remove useless log (#16624)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-27 10:01:47 +08:00
zhuwenxing
337338685a
[test]Add test case for multi replicase (#16668)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2022-04-26 18:35:45 +08:00
yah01
60f7fef3b3
Remove offline nodes from replica info after LoadBalanceTask done (#16653)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-26 18:23:46 +08:00
zhenshan.cao
a3f7bb5f2d
Remove the collectionID label from metrics (#16571)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-26 17:29:45 +08:00
Ten Thousand Leaves
4855ba6fa3
Optimize Mac M1 support some tweaks to embedded Milvus (#16650)
issue: #16500
issue: #16621

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-26 14:21:46 +08:00
Letian Jiang
f75dedb317
Validate partitionIDs & segmentIDs in search request (#16639)
* check partitions if they are released or unloaded
* check segments if their collection/partition(s) are released or unloaded

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-04-26 14:19:45 +08:00
yah01
80ae6de323
Add comments for concurrency package (#16654)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-26 11:33:45 +08:00
ThreadDao
955b9a31d5
[test] Add multi memory replicas test cases (#16649)
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2022-04-26 11:31:49 +08:00
Xiaofan
c1ff0cec8f
Change QueryCoord Log level (#16590)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-26 11:29:54 +08:00
XuanYang-cn
fbc7fe1cdc
Fix proxy unable to update cache (#16646)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-26 11:27:53 +08:00
groot
bc5e9ec929
Add list import tasks api (#16605)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-25 17:37:46 +08:00
congqixia
bb69e09736
Use fmt.Errorf instead of string concat in querynode/segment.go (#16422)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-25 16:11:48 +08:00
yah01
7af02fa531
Improve load performance, load binlogs concurrently per file, deserialize binlogs concurrently per field/segment (#16514)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-25 15:57:47 +08:00
Bingyi Sun
1682dc49fc
Fix unit test timeout (#16537)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-04-25 14:49:46 +08:00
Ten Thousand Leaves
14724013ba
[skip e2e]Update config for embedded Milvus (#16614)
issue: #15711

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-25 14:17:45 +08:00
congqixia
b99b65c26e
Add segment reference count and handles change info in ShardCluster (#16620)
Resolves #16619
Add reference count for each search/query request
For SegmentChangeInfo
- Wait all segments in OnlineList to be loaded
- Add handoff event into pending list
- Wait all segments in OfflineList is not used (reference count = 0)

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-25 11:51:46 +08:00
Ten Thousand Leaves
4ef2df8cb9
Fix an issue where query coord calls MinioChunkManager during init (#16629)
It causes Milvus not runnable with local storage.

issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-25 11:11:46 +08:00
Ten Thousand Leaves
e66ac6a77c
Resolve an issue with bulk load where segments in flow graph replica are not updated (#16609)
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-25 11:07:47 +08:00
zhenshan.cao
65a9e01a8f
Use AssertInfo instead of assert to ensure compiling (#16533)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-04-25 10:27:46 +08:00
Xiaofan
89b4a34892
Fix Data race in NodeID (#16603)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-24 22:03:44 +08:00
sre-ci-robot
0667b50bb4 Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2022-04-24 12:00:34 +00:00
zhuwenxing
a88e4a90ae
[skip e2e]Fix deployment step in deploy test (#16582)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2022-04-24 18:11:45 +08:00
yah01
5c66300408
Always add the shard leader into the nodes list of shard (#16616)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-24 17:37:46 +08:00
Aivin V. Solatorio
04fffb08ef
Support arithmetic operations on numerical fields for scalar filtering (#16520)
Signed-off-by: Aivin V. Solatorio <avsolatorio@gmail.com>
2022-04-24 16:43:45 +08:00
yah01
b82f6a1a0a
Fix GetReplicas() returns all nodes in ShardReplicas (#16597)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-24 11:35:45 +08:00
Ten Thousand Leaves
242f08fb67
Refine import task related logic (#16601)
issue: #16600

/kind bug

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-24 11:29:45 +08:00
yah01
2022a361c0
Enable ZSTD compression for pulsar (#16014)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-24 10:25:43 +08:00
groot
0cec1b9448
Fix bulkload bug (#16591)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-22 18:43:43 +08:00
yah01
3f5733b738
Return the local variable directly, make RVO enabled (#16201)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-22 17:43:42 +08:00