Commit Graph

54 Commits

Author SHA1 Message Date
Ten Thousand Leaves
1acd256481
Add DataQueryable and DataIndexed states for bulk load tasks (#16725)
issue: #16607

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-05 21:17:50 +08:00
yah01
804fbb6e27
Fix panic: Set Base of GetReplicas request (#16578)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-22 14:47:42 +08:00
congqixia
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470)
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
Ten Thousand Leaves
442c73a544
Support passing channel names to DataNode (#16424)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-12 13:25:34 +08:00
codeman
4e12271d79
Support login with username and password (#15656) (#16341)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-04-11 19:49:34 +08:00
Ten Thousand Leaves
0d54697d42
Implement ReportImport related logic. (#16312)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-31 13:51:28 +08:00
Jiquan Long
ba37531456
Add support for loading multiple indexes (#16138)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-30 21:11:28 +08:00
groot
478890a700
modify import proto (#16155)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-23 19:29:25 +08:00
groot
1c4b949a1d
modify import rpc interface (#16129)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-22 15:11:24 +08:00
groot
2078b24dca
Prepare for import (#16083)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-21 15:47:23 +08:00
groot
89356ebcb9
Modify import rpc interfaces (#16006)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-11 17:13:59 +08:00
groot
5fdef607d5
Add rpc interfaces for import (#15930)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
xige-16
cc5bee4684
Stop handoff if segment has been compacted (#15181)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-13 21:49:35 +08:00
Jiquan Long
8a63dba149
Add consistency level field in collection meta (#13766)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-21 19:49:02 +08:00
xige-16
6f8ea8cc7b
Fix querycoord panic at getRecoveryInfo (#12983)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-13 10:29:27 +08:00
congqixia
85af0b571d
Add dropped segment state (#11499)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-09 18:41:37 +08:00
Bingyi Sun
c3f0c5a3ff
Add compaction interfaces in proxy (#11431)
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-09 14:47:02 +08:00
godchen
09f8fa1040
Add query coord watch delta channel (#11218)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-05 14:47:19 +08:00
xige-16
b567a8e5b5
Add SystemInfo msgType for getMetricRequest (#11151)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-04 10:58:11 +08:00
bigsheeper
3a4df03b70
Add SealedSegmentsChangeInfo and changeInfoMsg (#9479)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-09 14:45:00 +08:00
congqixia
c17f845ab3
Add StartPositions fields in proto (#8657)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-27 14:13:58 +08:00
zhenshan.cao
b71bffe26f
Replace ChannelID with ShardName in DML Request (#8539)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-27 10:01:59 +08:00
Yusup
ccb57ba4b2
add collection alias (#7255)
Signed-off-by: Yusup <yusup@lsgrep.com>
2021-09-18 11:13:51 +08:00
yukun
eb818d05ab
Remove vector fields from output fields (#6112)
* Remove vector fields from output fields

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add empty_collection error_code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-25 16:52:11 +08:00
sunby
189ac881f3 Fix bugs (#5676)
* Remove redundant session startup

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Register datanode after start success

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix meta snap shot

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix datanode message stream channel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix bugs when drop empty collection

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix bug of getting pchan statistics from task scheduler

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix i/dist/dataservice test code

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix epoch lifetime not applied

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* fix datanode flowgraph dd node

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix handle datanode timetick bug

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Apply extended seal policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add check for time tick

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix check

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix the repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the bug when send statistics of pchan

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the repack function when craete dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix bugs

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix describe collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix bug when send timestamp statistics

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add length check before flush request

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix SaveBinlog bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Add more log in datanode

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Put SegmentState.Flushing as the last one in enum to fit the client

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix params in GetInsertBinlogPaths

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Rename policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove unused ddl functions and fields

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove pchan when drop collection

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Add balanced assignment policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix master ut

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add lock in session manager

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for debug

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix some logic bug and typo

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix recover bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Get collection scheme of a specific timestamp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Change CheckPoint to SegmentInfo in VchannelInfo

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Recover Unflushed segment numOfRows

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix dataservice unit tests

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:11 +08:00
sunby
3662b3f0e5 Refactor dataservice (#5400)
Pick up grpc calls to grpc_handler.go and refactor code format

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:01 +08:00
bigsheeper
cdbc6d2c94
Refactor query node and query service (#5751)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 12:41:40 +08:00
yukun
f18dfb4ff6
Add RetrieveTask implementation (#5313)
Resolves: #5257 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-20 07:02:31 +00:00
FluorineDog
88f5642603
Add plan proto and support basic boolean expr parser (#5088)
**What type of PR is this?**
- [x] Feature

**What this PR does / why we need it:**
This PR supports boolean expression as DSL.
1. The goal of this PR is to support predicates
    like `A > 3 && not B < 5 or C in [1, 2, 3]`. 
2. Defines `plan.proto`, as Intermediate Representation (IR) 
    used between go and cpp. 
3. Support expr parser, convert predicate expr to IR
    in proxynode, while doing static check there
4. Support IR to AST in cpp, enable the execution
2021-04-29 08:48:06 +00:00
godchen
060f7ca0d2
Add get index build progress proto (#5049)
Add get index build progress proto.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-27 07:46:45 +00:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
cai.zhang
a54097708d Adjust the DropIndex logic of the indexservice
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-04-16 15:37:13 +08:00
godchen
f3649f0419 Refactor interface and proto
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-03-12 14:22:09 +08:00
ThreadDao
ca916a5c47 Add ci nightly
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2021-03-11 14:14:29 +08:00
xige-16
09ae985daa Fix wrong error code in master_service_test.go
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-10 22:06:22 +08:00
neza2017
7b2d67242d Fix describe segment if index not exist
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-10 14:45:35 +08:00
quicksilver
455cc59dbb Update publish-test-images.yaml
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-03-09 13:55:35 +08:00
Cai Yudong
c795cdbe2a Change minSegmentSizeToEnableIndex default to 1024
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2021-03-07 13:37:25 -06:00
neza2017
2835bcdc4e Update masterservice unittest
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-05 20:41:34 +08:00
neza2017
1d3fdebcf4 Add drop index for proxy
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-20 15:38:44 +08:00
xige-16
f62078c027 Fix crash error when search
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-02-20 10:14:03 +08:00
cai.zhang
55634dc5f3 Add interface of load and release collection and partition
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-02-04 15:31:02 +08:00
zhenshan.cao
1b616fb874 Add persistent_segmentinfo
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-03 18:55:00 +08:00
neza2017
5aec8bc5e6 Refactor masterservice
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-24 20:26:35 +08:00
XuanYang-cn
50e2369000 Change name and tag to ID
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-22 19:36:09 +08:00
neza2017
90a0b8b2f5 Update proto
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-22 11:44:02 +08:00
XuanYang-cn
ec27788628 Add segment statistics updates logic for datanode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-21 09:55:25 +08:00
sunby
76a7684d96 Add data service
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-19 12:10:49 +08:00
zhenshan.cao
76d92e73d1 Replace old proto with new proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-18 21:39:56 +08:00
xige-16
d22c2cd203 Add proto for query service loading collection
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-18 19:32:08 +08:00