test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-05 05:18:52 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
16,074 Commits 96 Branches 105 Tags 186 MiB
8da44638ac
Commit Graph

95 Commits

Author SHA1 Message Date
bigsheeper
cd19d99ad7
Add channel level checkpoint (#20350) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-10 22:13:04 +08:00
Xiaofan
2bfecf5b4e
Refine bloomfilter and memory usage (#20168) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-31 17:41:34 +08:00
XuanYang-cn
d5bc4e2585
Make DN able to process syncing request in batch (#20126) 
See also: #19834

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-27 21:25:33 +08:00
XuanYang-cn
e081eb287d
Refine flowgraph nodes for readability (#19864) 
- delete compaction tasks from executor while dropping

See also: #19780

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-20 16:39:29 +08:00
XuanYang-cn
2785f98de6
Refactor segment replica in DN (#19815) 
- Replace isNew and isFlushed with sType
- Add compacted into proto
- Rename replica to channel
- Rename segmentReplica to channelMeta
- Replace 4 maps with 1 map
- Add segment.go and segment_test.go

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-18 15:33:26 +08:00
congqixia
a1cdc55bcb
Make newSegment transfer state after SaveBinlogPath succuess (#19858) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-18 14:07:25 +08:00
SimFG
a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
congqixia
04e1333552
Fix insert buffer node time range uint64 conversion error (#19725) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-12 17:19:23 +08:00
congqixia
d64f318a80
Add collection schema in watch channel info (#19663) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-10 01:37:21 +08:00
congqixia
838a633584
Add ctx parameter for ChunkManager methods (#19546) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-29 16:18:56 +08:00
Ten Thousand Leaves
b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-09-26 18:06:54 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
congqixia
91663a6603
Fix bloom filter size not match (#19140) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-13 14:12:31 +08:00
congqixia
68a6587374
Set insert&stats binlog timestamp range (#19005) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-04 09:05:09 +08:00
congqixia
f0846fb79b
Handles DropPartitionMsg in datanode flowgraph (#18292) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-15 17:12:27 +08:00
Ten Thousand Leaves
5e1e7a6896
Fix an issue that causes flush hanging (#17169) 
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-25 14:34:00 +08:00
bigsheeper
a4ea2fb18a
Handle errors in DataNode and QueryNode flow graph (#17096) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-24 21:11:59 +08:00
godchen
bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
xige-16
99984b88e1
Support delete varChar value (#16229) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-04-02 17:43:29 +08:00
xige-16
205c92e54b
Support insert string data (#15993) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-03-25 14:27:25 +08:00
xiyichan
ca129d4308
Support configurable msgstream (#16131) 
Signed-off-by: xiyichan <2863768433@qq.com>
 2022-03-24 10:15:25 +08:00
godchen
78557ca6d7
Replace minio kv with minio chunk manager (#15936) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-17 18:03:23 +08:00
jaime
7a3a721380
Reconstruct mqstream module (#15784) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-03 21:57:56 +08:00
bigsheeper
13177a90af
Add prometheus metrics for DataNode (#15650) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
 2022-02-28 19:11:55 +08:00
Cai Yudong
f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-07 10:09:45 +08:00
Cai Yudong
7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-09 23:45:33 +08:00
Cai Yudong
f8f61fb116
Use BaseParams etcd configuration for all components (#15012) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-07 16:15:26 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
Bingyi Sun
0c39e0370c
Move segment statistics update to datanode tt loop (#13233) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-15 10:53:16 +08:00
XuanYang-cn
dbf757e08b
Change genKey into JoinIDPath (#12959) 
1. Simplify genKey(), make genKey a rpc call
2. Unify the usage of JoinIDPath

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-09 11:09:06 +08:00
congqixia
8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-02 16:39:33 +08:00
congqixia
e813867652
Implement Drop mode for flush manager (#12469) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-01 10:11:39 +08:00
congqixia
579416e3d3
Apply retry for flush manager execution (#11808) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-15 17:19:10 +08:00
Bingyi Sun
5c8978e5d6
Fix drop segment bug (#11709) 
issue: #11708
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-12 17:31:11 +08:00
XuanYang-cn
c4bb73dc74
Complete drop collection procedure in DN (#11552) 
drop, cancel compaction, flush all buffers and release fg

See also: #11426

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-11 20:56:49 +08:00
congqixia
0e5931a3da
Add mergedTimetickSender for datanode (#11297) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-05 14:59:32 +08:00
congqixia
0ef95c5df1
Add timeticker logger for insert buffer node (#11216) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-04 15:40:14 +08:00
godchen
b63a9521c2
Make gen msg private (#10115) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-04 15:36:19 +08:00
XuanYang-cn
21d98c2825
Add binlog io for compaction (#10536) 
See also: #9530

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-25 20:13:51 +08:00
congqixia
2713988ff2
Merge auto manual flush with same segment id (#10550) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-25 18:03:42 +08:00
congqixia
5737e0075f
Apply flush manager logic in datanode (#10142) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-19 11:04:34 +08:00
congqixia
0438ed2bfe
Use single signal chan to notify flush (#10075) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-18 12:34:34 +08:00
XuanYang-cn
dcd56a575d
[skip ci]Update license for flowgraph of dn (#9962) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-15 18:07:09 +08:00
godchen
a2dc0d8808
Init segment replica bloom filter (#9770) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-14 10:24:33 +08:00
XuanYang-cn
aac6323cff
Use config as parameter (#9707) 
Make functions in flowgraph lesser parameters

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-13 11:16:32 +08:00
godchen
59ab0e441c
Add bloom filter for stats (#9630) 
* Add bloom filter for stats

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* trigger GitHub actions

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-13 10:22:33 +08:00
Cai Yudong
84e568ee32
Support Delete in datanode (#8505) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-11 16:31:44 +08:00
Cai Yudong
b9f5a6bb1e
Rename some variables in unittest (#9484) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-08 17:47:25 +08:00
Cai Yudong
ec9ec97cec
Fix typo (#9482) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-08 17:45:35 +08:00
XuanYang-cn
ebd894d8b7
Refactor buffer size calculation (#8565) 
This PR changes the datanode insertBufSize config
from 32000 number of rows to 16MB

Resolves: #7741

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-09-26 20:55:59 +08:00