test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-03 04:19:18 +08:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity
15,540 Commits 95 Branches 105 Tags 184 MiB
5f02f0975c
Commit Graph

320 Commits

Author SHA1 Message Date
Bennu
373290d351
[Skip ci] Add comments for milvus.yaml (#12381) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-30 14:17:12 +08:00
Bennu
f034f893c9
[Skip ci] Add comments for milvus.yaml (#12428) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-30 14:05:57 +08:00
congqixia
b99164f43d
Add datacoord GC configuration (#12061) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-18 22:29:40 +08:00
Xiaofan
d37fdf9379
Remove component.yaml (#11959) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-17 14:35:10 +08:00
Xiangyu Wang
12f50cb22c
[skip ci]Update OWNERS files (#11898) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-16 15:41:11 +08:00
shaoyue
5b50731ff4
Fix searchResultBufFlags & queryResultBufFlags memory leak (#11585) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-16 14:13:12 +08:00
Bingyi Sun
b8c5239784
Add compaction retention config (#11793) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-16 11:47:10 +08:00
Xiaofan
3cd47cd89e
Remove common.yaml and component.yaml (#11662) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-12 18:53:11 +08:00
xige-16
d857577a7b
Postpone the execution of handoff until index creation is complete (#11648) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-11-12 18:49:10 +08:00
Bingyi Sun
002b854590
Move compaction and garbage switch to milvus.yaml (#11654) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-11 23:38:42 +08:00
Cai Yudong
77b6439c65
[skip ci] Update configuration name (#11656) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-11 17:24:41 +08:00
yukun
f51ef71b51
Change default retentionSizeInMB to 8192(8GB) (#11436) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-11 17:18:43 +08:00
Xiaofan
5f08b67156
Move node.yaml to component.yaml (#11363) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-10 14:27:37 +08:00
yanliang567
a5305a482e
[skip ci]Update milvus yaml config (#11361) 
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2021-11-08 10:21:04 +08:00
yukun
8ec040a35c
Add rocksmqPageSize to config (#11113) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-08 10:17:18 +08:00
Bennu
dcf4cecea4
Add comments for milvus.yaml (#11268) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-06 14:32:56 +08:00
sunby
c26cd79532
Add compaction (#9906) 
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-11-05 22:25:00 +08:00
Xiaofan
d81b15163f
Seek Query/Query Result channel to Latest (#10985) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-05 14:57:44 +08:00
Bennu
9ece3b65e4
Add comments for milvus.yaml (#11269) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-04 19:19:21 +08:00
Bennu
bb16d14ba5
Add comments for milvus.yaml (#11267) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-11-04 19:17:30 +08:00
Cai Yudong
a2ad88237d
Add deltaChannels in rootcoord core (#11097) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-03 21:04:14 +08:00
Xiaofan
385eed1d17
Remove knowhere config (#10614) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-01 10:26:03 +08:00
Xiaofan
93f205b49e
Rename param names (#10853) 
* Rename param names

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

* Rename configs

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-28 23:16:40 +08:00
Bennu
2c9776feaa
[skip ci] Add comments for milvus.yaml (#10825) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-28 20:22:59 +08:00
Bennu
997fed0ddd
[skip ci] Add comments for milvus.yaml (#10827) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-28 19:51:03 +08:00
Xiaofan
cf574b34a0
[skip ci]Add etcd config comment (#10617) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-25 22:54:21 +08:00
xige-16
9ebabdac79
Add handoffTask (#10330) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-10-24 22:39:09 +08:00
bigsheeper
132f57f5b7
Set gracefulTime to 0 (#10114) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-10-21 10:55:12 +08:00
Bennu
b57d7a7508
[skip ci] Add comments for milvus.yaml (#10061) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-18 18:02:57 +08:00
Bennu
b9a904a1d3
[skip ci] Add comments for Milvus.yaml (#9565) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-12 08:20:33 +08:00
XuanYang-cn
532470a607
[skip ci]Update license for all configs (#9644) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-11 21:36:55 +08:00
XuanYang-cn
43da3e3b1e
[skip ci]Add license for configs (#9643) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-11 21:35:03 +08:00
Bennu
e2638a3232
[skip ci] Add comments for Milvus.yaml (#9611) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-11 21:31:16 +08:00
Bennu
ab72753c4b
[skip ci] Add comments for Milvus.yaml (#9609) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-11 21:29:27 +08:00
Bennu
ce4e848663
Add comments for Milvus.yaml (#9553) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-11 21:21:47 +08:00
Bennu
24de3fa76d
Add comments for Milvus.yaml (#9555) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-11 21:19:40 +08:00
XuanYang-cn
8d8d53beeb
[skip ci]Change license on config files (#9642) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-11 21:15:00 +08:00
Bennu
8f3b8ed1da
[skip ci] Add comments for Milvus.yaml (#9610) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-10-11 16:28:05 +08:00
shiyu22
7caccd36ce
[skip ci]Update the comment of yaml to uppercase (#9502) 
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
 2021-10-09 10:55:14 +08:00
Xiangyu Wang
270cdbd4e2
[skip ci]Update reviewers in OWNERS files (#9316) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-10-06 07:06:08 +08:00
XuanYang-cn
491744b954
[skip ci]Update comments for configs in datanode (#8976) 
Signed-off-by: Yang Xuan <xuan.yang@zilliz.com>
 2021-10-01 10:51:15 +08:00
bigsheeper
9d6c95bf25
Add cacheSize to prevent OOM in query node (#8765) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-09-28 22:24:03 +08:00
Xiaofan
93fc10f8a6
Remove channel&&common.yaml (#8716) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-27 22:32:02 +08:00
XuanYang-cn
ebd894d8b7
Refactor buffer size calculation (#8565) 
This PR changes the datanode insertBufSize config
from 32000 number of rows to 16MB

Resolves: #7741

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-09-26 20:55:59 +08:00
Xiaofan
31ec399edb
Move channel configuration from channel.yaml to milvus.yaml (#8420) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-26 19:46:17 +08:00
Bennu
607e16f60f
[skip ci] add comments in query_node.yaml (#8585) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-26 16:53:58 +08:00
Xiaofan
dbcacec0a7
[skip ci]Add comments for common conf (#8531) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-25 11:17:56 +08:00
Xiaofan
d7f98ca4bb
[skip ci]Add comments for proxy conf (#8530) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-25 11:16:02 +08:00
Bennu
aec4c98363
[skip ci] add comments in root_coord.yaml (#8507) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-24 20:55:56 +08:00
sunby
816bac82ec
Add root path in minio and replace etcd.rootPath in datanode (#8272) 
issue: #7905
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-09-24 20:50:23 +08:00
shiyu22
8cc78b2a97
[skip ci] Add comments about minio in yaml (#8493) 
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
 2021-09-24 18:40:13 +08:00
Bennu
270200b93c
[skip ci] add comments in root_coord.yaml (#8478) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-24 16:57:55 +08:00
Bennu
865c67efdb
[skip ci] add comments in proxy.yaml (#8481) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-24 16:03:55 +08:00
shiyu22
11de20c8e5
[skip ci] Add comments for etcd (#8366) 
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
 2021-09-23 16:38:24 +08:00
Xiaofan
f5173b595f
Add a channel prefix for all channels (#8166) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-23 10:53:53 +08:00
shiyu22
51aa4f0254
Add comments with localStorage in yaml (#7892) 
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>

Add comments with localStorage in yaml

Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
 2021-09-22 19:23:59 +08:00
dragondriver
a3215ceaab
[skip ci] Explain maxShardNum in proxy.yaml (#8236) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-18 18:26:08 +08:00
dragondriver
3738655ed0
[skip ci] Explain maxTaskNum in proxy.yaml (#8235) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-18 18:24:08 +08:00
dragondriver
cd367c83d9
[skip ci] Explain timeTickInterval in proxy.yaml (#8233) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-18 18:21:57 +08:00
XuanYang-cn
46c72c57c2
Add BufferData in insertbufferNode (#8138) 
Optimize bufferInsertMsg param to avoid a second transfer
of endposition

See also: #8058, #7741

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-09-18 14:25:50 +08:00
Cai Yudong
83bf9c5efb
Add knowhere.yaml to support knowhere configuration (#7967) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-18 10:37:51 +08:00
Cai Yudong
d10c6abc9e
Apply for msgstream from pool when creating collection (#7738) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-18 09:13:50 +08:00
dragondriver
7f048ea8c8
Support to set max task number of task queue (#7974) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-15 19:09:50 +08:00
Bennu
511612d3bf
[skip ci] Add comments to the file query_node.yaml (#7916) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-15 16:05:48 +08:00
Cai Yudong
14ef405a6b
Rename chunk_size to chunk_rows for better code readability (#7953) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-15 15:15:52 +08:00
Cai Yudong
a0fd2707cb
Support config SIMD type (#7942) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-15 12:57:48 +08:00
Cai Yudong
e6c384b4d4
Move segcore chunk_size configuration to querynode.yaml (#7913) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-15 10:35:52 +08:00
Bennu
e28db5600d
[skip ci] Add comments to the file data_node.yaml (#7780) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-14 16:11:48 +08:00
ryjiang
9a9a1492d0
[skip ci]add comment for segcore config (#7840) 
Signed-off-by: ruiyi.jiang <ruiyi.jiang@zilliz.com>
 2021-09-13 20:52:58 +08:00
Bennu
888c7d7bfb
[skip ci] Add comments to the file query_node.yaml (#7707) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-11 14:52:01 +08:00
Cai Yudong
27dcf698d3
Support set segcore chunk_size via config file (#7635) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-11 14:40:01 +08:00
Bennu
04b8160cac
[skip ci]Add comments to the file data_coord.yaml (#7611) 
Signed-off-by: Bennu <yunmei.li@zilliz.com>
 2021-09-10 23:18:01 +08:00
Cai Yudong
4e23ed9a18
Revert apply physical channels from pool (d5f7e358 and 3b960969) (#7687) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-10 13:00:00 +08:00
shiyu22
fc962d856a
Add comments to log in yaml (#7614) 
Signed-off-by: shiyu22 <shiyu.chen@zilliz.com>
 2021-09-09 18:48:02 +08:00
zhenshan.cao
b0524adbd8
Support specified shard number when create collection (#7482) 
Add shards_num to CollectionInfo

Modify dmlChannelNum config

Compatible with old meta

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-09-08 15:00:00 +08:00
Cai Yudong
3b9609692b
Apply for msgstream from pool when creating collection (#7377) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-08-31 18:35:58 +08:00
Cai Yudong
f4b53df727
Revert "Apply for msgstream from pool when creating collection (#7323)" (#7376) 
This reverts commit e6de86a433.
 2021-08-31 11:07:58 +08:00
Cai Yudong
e6de86a433
Apply for msgstream from pool when creating collection (#7323) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-08-31 09:47:58 +08:00
zhenshan.cao
7c60e6aa00
Avoid port conflicts between embedded etcd and external etcd (#7320) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-08-27 10:13:57 +08:00
Xiangyu Wang
c62f23e201
[skip ci]Remove neza2017 from reviewers (#7324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-08-27 00:27:36 +08:00
Xiaofan
58b2089692
Import Embedded Etcd Kv Part 1(#7112) (#7174) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-08-24 09:45:51 +08:00
godchen
8cc24e09c9
Make users get vectors without using local storage (#7134) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-08-18 16:30:11 +08:00
yukun
847586eb95
Add rocksmq retention (#6617) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-08-16 18:46:10 +08:00
Xiangyu Wang
5e78fd6a97
[skip ci]Update approvers (#7063) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-08-12 19:36:09 +08:00
dragondriver
ad77a6e0d4
Fix #6859, increase the MaxCallRecvMsgSize and MaxCallSendMsgSize of grpc client (#6861) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-08-04 13:03:24 +08:00
godchen
346e9cba4a
Add minio FGet method (#6386) 
* Add minio FGet method

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* go fmt

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change el append

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-07-09 11:44:40 +08:00
yukun
39614aa8eb
Add rocksmq_path in config and paramtable (#6099) 
* Add rocksmq_path in config and paramtable

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add rocksdbPath in NewRmsFactory

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change rdb default path to /vat/lib/milvus/rdb_data

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-25 19:44:11 +08:00
Cai Yudong
a09a3a1905
rename service to coord (#6020) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-23 16:14:08 +08:00
Cai Yudong
7dbf33b788
Update component names in config files (#5989) 
* rename master/indexService in .yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxyNode to proxy in config files

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move master.yaml to root_coord.yaml, move proxy_node.yaml to proxy.yaml

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proto/query_service.proto to proto/query_coord.proto

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move proxy_node_manager.go to proxy_manager.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add query_coord.pb.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-22 19:08:03 +08:00
xige-16
2af1c27811
Rename queryservice to querycoordinator (#5931) 
* Rename queryservice to querycoordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix param table

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix unittest

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* service to coordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* set querycoord in ci

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-22 16:44:09 +08:00
sunby
d2c8462dac
Rename params in data_coord.yaml (#5968) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-22 16:18:21 +08:00
sunby
7f2d7c1da3
Rename dataservice to datacoordinator (#5912) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-21 18:22:13 +08:00
sunby
da1f9f9241
Rename dataservice to datacoordinator (#5904) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-21 11:40:15 +08:00
XuanYang-cn
ab7f642740
Add alias in paramtable (#5878) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-19 12:38:06 +08:00
yukun
e9a8d1c404
Add vChannels in proxy for query results (#5802) 
* Fix proxynode for new retrieve logic

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove querynodenum from proxynode and querynode

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove QueryNodeIDList from proxy

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-16 20:15:59 +08:00
XuanYang-cn
75f2b3c2ba
Add Unittest for allocator (#5781) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-06-15 19:11:55 +08:00
Cai Yudong
0f4bd50ce3 Support ETCD endpoints array (#5755) 2021-06-15 16:36:36 +08:00
congqixia
ef7339ceda DataService stores flush msgstream pos at VChannel granularity (#5397) 
SaveBinLogPaths saves stream positions at VChannel granularity

resolves #5396

Signed-off-by: Congqi Xia congqi.xia@zilliz.com
 2021-06-15 15:54:57 +08:00
Xiangyu Wang
ca0993c274 Update OWNERS 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-05-30 18:58:47 +08:00
congqixia
4aa6a8fe18
Dataservice use lastest grpc design and saves ddl/dml pos pair (#5348) 
Dataservice use lastest grpc design and saves ddl/dml pos pair

* Update SaveBinlogPaths grpc definition
* Saves segment ddl & dml flush pos pair(start, end)

See also #5220 

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-21 11:38:58 +00:00
neza2017
c0daf8e41d
set timestamp on dd requests (#5298) 
- set dd request's time tick on master
- send time tick into dd channel

See also: #5291

Signed-off-by: yefu.chen yefu.chen@zilliz.com
 2021-05-20 06:14:14 +00:00
congqixia
38f5b0826f
Add SaveBinlogPaths grpc in dataservice (#5277) 
* Add SaveBinlogPaths grpc in dataservice

* fix merged compile error

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-20 11:34:45 +08:00
congqixia
d5ab8ac345
Restore flush and stats stream pos (#5284) 
Restore segment flush stream & statistic stream to last success pos

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-05-19 06:13:53 +00:00
XuanYang-cn
cebdfb34f7
Enable multiple datanode (#5068) 
Now it's able to start multiple datanode, however, they will process the same insert data.
Further changes are needed to truly enable multiple datanode:

While registering, dataservice need to return different insert channels for different datanode.

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-04-28 18:43:48 +08:00
bigsheeper
c4b379be8b
Set search graceful time from config (#5014) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-24 07:53:37 +00:00
bigsheeper
dadb02db75
Remove unused params in query node (#4987) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-23 10:07:45 +08:00
XuanYang-cn
5645718311
edit datanode config files (#4992) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-04-22 11:19:48 +08:00
sunby
fad94fb576
Remove unsed configs (#4990) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-22 03:04:59 +00:00
sunby
b79a408491 Remove unused config item in dataservice and datanode 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-19 19:28:45 +08:00
XuanYang-cn
e281f6a4a1 Remove useless configs in config files 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-04-13 10:04:39 +08:00
zhenshan.cao
c68bcfea16 Increase insert buff size 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-19 11:51:16 +08:00
zhenshan.cao
ab200e5598 Change querynode num to one 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-16 21:30:44 +08:00
dragondriver
af32f442bb Split big insert message into serveral smaller 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-09 16:18:58 +08:00
Cai Yudong
c795cdbe2a Change minSegmentSizeToEnableIndex default to 1024 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-03-07 13:37:25 -06:00
sunby
76480fcefe Change default log output to stdout 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-08 16:24:55 +08:00
neza2017
014c4fe8ce Add config of minSegmentSizeToEnableIndex 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-08 15:46:51 +08:00
dragondriver
801396e3ef Disable register link logic, use proxy node directly 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-04 22:27:12 +08:00
bigsheeper
d3db2b0d82 Add zap log in query service 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-27 10:11:52 +08:00
cai.zhang
1d899513dd Unify service name in config file 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-24 18:46:19 +08:00
zhenshan.cao
7c5e2d89f6 Read node port from config 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 18:08:17 +08:00
sunby
5769c000a1 Find failed python tests and add skip mark "r0.3-test" 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-19 15:37:04 +08:00
bigsheeper
d8f41db823 Fix search crash error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-19 13:56:01 +08:00
sunby
5b61182140 Add zap log 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-18 19:53:55 +08:00
neza2017
85c1404393 Refactor master:index ids 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-11 08:41:59 +08:00
godchen
c2ca2c276f Fix response check error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-04 19:34:35 +08:00
Xiangyu Wang
fd9e42d3ed Update msgstream/unmarshal interface 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-03 17:30:10 +08:00
bigsheeper
abffdbff62 Remove partition tag 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 16:44:38 +08:00
sunby
208700b0e7 Set fixed insert channel number, remove channel manager 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-03 15:07:48 +08:00
neza2017
854accf95b Remove master and writenode 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-30 16:41:47 +08:00
bigsheeper
68cd15af63 Get index info from master 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-27 14:41:56 +08:00
dragondriver
a0226c09f0 Change the logic node id allocator in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-27 13:52:01 +08:00
sunby
b4bcaaf847 Fix data service startup bug 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-27 11:40:10 +08:00
xige-16
7c210310dc Add load function for query service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-26 15:13:20 +08:00
XuanYang-cn
b445587e11 Add datanode main 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-26 14:46:54 +08:00
dragondriver
c6a6b1436c Add soft time tick in proxy service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-26 13:41:41 +08:00
neza2017
5af23cf018 Init params and add segment start/end position 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-26 09:43:41 +08:00
cai.zhang
f940cc455a Add dockerfile for index 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-26 09:38:40 +08:00
bigsheeper
7554246ace Add segment seeking and use real client 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-25 18:33:10 +08:00
sunby
7c0c835b49 Init params and add segment start/end position 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-25 15:16:28 +08:00
XuanYang-cn
5015a1035d Add logic of get meta from MasterService for datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-25 17:10:32 +08:00
neza2017
1fe099775d Refactor data service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-25 15:17:17 +08:00
neza2017
e89e89b894 Add main for masterservice 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-25 12:03:44 +08:00
XuanYang-cn
9fe672ab86 Enchance datanode interface 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-24 21:20:11 +08:00
neza2017
e2b45f9866 Init param table and open unittest at ci 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-23 17:56:57 +08:00
xige-16
511aa419fa Refactor query service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-22 14:28:06 +08:00
dragondriver
cd52adc18b Add proxy service to ci workflow 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-22 12:57:23 +08:00
dragondriver
eba6bb288c Refactor the sructure of proto file 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-22 09:36:18 +08:00
neza2017
9828a52999 Add create index 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-21 10:01:29 +08:00
cai.zhang
83c60fd84f Update proto about index 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-21 15:20:23 +08:00
XuanYang-cn
ec27788628 Add segment statistics updates logic for datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-21 09:55:25 +08:00