Commit Graph

14473 Commits

Author SHA1 Message Date
edward.zeng
086e8968ef
Upgrade minio version to RELEASE.2022-03-17T06-34-49Z (#16393)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-04-06 15:47:32 +08:00
congqixia
18d76261a0
Add ShardNodeDetector implementation (#16373)
Use etcd as a ShardNodeDetector
Unit tests is based on embedded etcd

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-06 15:43:31 +08:00
Ten Thousand Leaves
1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. (#16374)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-06 15:33:32 +08:00
XuanYang-cn
78200009a3
Impl GetReplicas in Proxy (#16390)
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-06 14:57:31 +08:00
bigsheeper
2b27fa4609
Print numTimestamps instead of Timestamps (#16369)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-04-06 14:47:31 +08:00
cai.zhang
4267dc2efc
Handle error and oragnize log (#16367)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 14:33:30 +08:00
bigsheeper
7161a7760c
Fix segIDAssigner spelling mistake (#16370)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-04-06 14:11:31 +08:00
bigsheeper
4f4a417c8f
Unify in log print (#16368)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-04-06 14:09:35 +08:00
cai.zhang
e73139deeb
Compressed IndexNode log (#16365)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 11:33:31 +08:00
cai.zhang
04076393c5
Fix log error (#16363)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 10:13:32 +08:00
cai.zhang
f9c6b3980f
Compressed log (#16354)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 10:11:36 +08:00
Cai Yudong
543ec4a209
Rename log to milvus_log (#16383)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-04-05 22:55:30 +08:00
Cai Yudong
7323180861
Move bench to directory unittest (#16386)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-04-05 12:43:30 +08:00
Cai Yudong
0c53f06a33
Fix build error on mac (#16380)
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2022-04-04 21:09:32 +08:00
Letian Jiang
09b90fd8bf
Fix QueryCollection retrieve unit test (#16376)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-04-04 21:07:30 +08:00
Ten Thousand Leaves
2bd0a9526a
Implement task expiring logic and make all deadlines configurable. (#16355)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-03 11:37:29 +08:00
groot
be8d9a8b6b
Support import numpy file (#16348)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-03 11:27:29 +08:00
sre-ci-robot
6682d1b635 Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2022-04-02 12:00:51 +00:00
xige-16
99984b88e1
Support delete varChar value (#16229)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-04-02 17:43:29 +08:00
congqixia
aa1c26de77
Add ShardCluster implementation (#16360)
ShardCluster maintains shard replica meta information
It watches node & segment change events
Provides shard replica search/query services

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-02 17:35:29 +08:00
edward.zeng
573eed5bd3
Bump milvus version to v2.0.2 (#16358)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-04-02 16:00:31 +08:00
Letian Jiang
028c5cb882
Modify grpc interface for replica Search/Query in QueryNode (#16326)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-04-02 14:15:31 +08:00
Xiaofan
7f7379d55f
Fix DML channel AsProducer after Close (#16352)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-02 13:09:29 +08:00
XuanYang-cn
5494f3c318
Remove offLineNodes from watch states check (#16347)
This PR also
- adds more information in log
- makes Session able to logged by zap.Any/zap.String

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-01 22:33:29 +08:00
Jenny Li
7303036c19
[skip e2e]Update issue example into #<xyx> (#16350)
Use real issue example will combine too many pr metioned make the pr author annoying

Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-04-01 21:33:29 +08:00
congqixia
6111dcde52
Fix ticker leakage in datanode flowgraph (#16346)
Using  in case ticker need to be closed may cause leakage
Use time.NewTicker with deferred Stop instead

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-01 19:35:29 +08:00
XuanYang-cn
e9090a62ab
Rearrange search/queryTask for readability (#16325)
`searchTask` has 683 lines of code, `queryTask` has 485 lines of code.
`task.go` contains 4k+ codes including `searchTask` and `queryTask`.

It was so much pain navigating codes of searchTask and queryTask though task.go,
task.go and task_test.go are literaly unreadable.

This PR moves
1. 650+ lines of code of `searchTask` from `task.go` to `task_search.go`.
2. 1.4k+ lines of test code of `searchTask` from `task_test.go` to
   `task_search_test.go`.
3. 450+ lines of code of `queryTask` from `task.go` to `task_query.go`.
4. 200+ lines of test code of `queryTask from `task_test.go to
   `task_query_test.go`.

This PR also rearrange methods positions of `searchTask` and
`queryTask`:
-  Putting the most important methods `PreExecute`, `Execute`, and
   `PosExecute` at the beginning of the file.
-  Moves interface methods `ID`, `SetID`, `Type`, `BeginTs`, etc.
   that nobody cares about to the bottom of the file.

See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-01 18:59:29 +08:00
congqixia
7a44fff8cd
Use fmt.Errorf instead of string concat in tsafe_replica (#16344)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-01 18:37:28 +08:00
yah01
c005f07ccc
Modify proto to add GetReplicas() interface (#16328)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-01 16:15:29 +08:00
Jiquan Long
fd589baca7
Integrates marisa trie index (#16192)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-04-01 15:31:29 +08:00
Jenny Li
8b0f260b05
[skip e2e] Add commit-id after date (#16316)
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-04-01 14:25:29 +08:00
Letian Jiang
cb65860e22
Add implementation for QueryShardService (#16317)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-04-01 14:19:29 +08:00
zhuwenxing
0c2fc2721e
[skip e2e]Update deploy test github action (#16340)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2022-04-01 14:09:29 +08:00
Ten Thousand Leaves
45be3deb3f
Implement RC, DC, DN calling path for import. (#16321)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-01 11:33:28 +08:00
congqixia
376a7f6567
Fix Datacoord unsubAttempt using pchannel (#16337)
Fix the channel name used when Datacoord unsub for DataNode
This channel shall always be pChannel instead of vChannel

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-01 11:25:28 +08:00
groot
bd241cd28b
Add performance test case for import (#16296)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-04-01 10:07:28 +08:00
zhenshan.cao
b03da87df5
Accelerate query speed in sealed segment (#16287)
Fix delete duplicate key

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-31 17:33:28 +08:00
XuanYang-cn
58addbf9f4
Add GetShardLeaders rpc in querycoord proto (#16299)
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-31 16:39:29 +08:00
Jenny Li
e20385c817
[skip e2e] Update mergify rule for adding centos7 (#16319)
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-03-31 15:19:28 +08:00
congqixia
f529f8c1fc
Use fmt.Errorf instead of str concat in func.go (#16315)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-31 15:03:28 +08:00
Jenny Li
56c0ea9b1d
[skip e2e] Add centos build (#16264)
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-03-31 14:49:28 +08:00
Letian Jiang
ad9a2217ae
Add interface for QueryShardService and QueryShard (#16309)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-31 14:05:28 +08:00
Ten Thousand Leaves
0d54697d42
Implement ReportImport related logic. (#16312)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-31 13:51:28 +08:00
Bingyi Sun
baa745b83e
Improve log (#16307)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-31 11:33:28 +08:00
aakejiang
30be20e357
correct DEVELOPMENT.md (#16302)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-03-31 09:21:28 +08:00
zhuwenxing
fd3ef9b11b
[skip e2e]Update deploy test script (#16280)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2022-03-31 09:19:32 +08:00
Bingyi Sun
40fab91448
improve log (#16236)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-30 23:05:28 +08:00
XuanYang-cn
a0bb758be5
Refine log in proxy searchTask (#16263)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-30 22:09:28 +08:00
congqixia
899638810a
Add error log when tsafe register watcher more than once (#16294)
Add some test case to cover the error case
Add log in error branch

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-30 22:07:28 +08:00
Jiquan Long
ba37531456
Add support for loading multiple indexes (#16138)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-30 21:11:28 +08:00