Commit Graph

276 Commits

Author SHA1 Message Date
Ten Thousand Leaves
e66ac6a77c
Resolve an issue with bulk load where segments in flow graph replica are not updated (#16609)
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-25 11:07:47 +08:00
Xiaofan
89b4a34892
Fix Data race in NodeID (#16603)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-24 22:03:44 +08:00
jaime
ed3c29eae7
Support Kafka (#16149)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-12 19:47:33 +08:00
godchen
bb7a0766fe
Add dependency factory (#16204)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-07 22:05:32 +08:00
Ten Thousand Leaves
1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. (#16374)
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-06 15:33:32 +08:00
XuanYang-cn
a77dd1034c
Adopt the extended state in DataCoord (#16200)
ChannelWatchInfo with ToWatch, ToRelease will trigger a timer.
ChannelManager now reacts to different ChannelWatch states.

- WatchSuccess > log this info

- WatchFailure/WatchTimeout > ToRelease

- ReleaseSuccess > Delete, reassign if not from DropCollection

- ReleaseFailure/ReleaseTimeout > Cleanup subscription and Delete,
  reassgin if not from DropCollection.

Some Notes:
1. Reassignment will add this channel to buffer if there's only one node.

See also: #15846

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-28 22:33:27 +08:00
xiyichan
ca129d4308
Support configurable msgstream (#16131)
Signed-off-by: xiyichan <2863768433@qq.com>
2022-03-24 10:15:25 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
jaime
29975a7a26
Remove Consume() API from mqstream interface (#15886)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-11 20:09:59 +08:00
Cai Yudong
3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
jaime
7a3a721380
Reconstruct mqstream module (#15784)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-03 21:57:56 +08:00
Xiaofan
bd7d294c6d
Refine log level in data (#15821)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-02 15:35:55 +08:00
congqixia
b0923f1299
Fix DataCoord panics if message queue service quits before it (#15702)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-23 11:09:52 +08:00
Nemo
3eb074a9b0
Update some comments for data coord with some minor code refactoring. (#15686)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-02-22 13:15:51 +08:00
congqixia
c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted (#15497)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-15 15:07:48 +08:00
Xiaofan
be42627ab2
Move Logutil to Util dir (#15447)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-09 18:55:46 +08:00
Cai Yudong
1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
Cai Yudong
f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
Cai Yudong
4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-02 00:35:43 +08:00
Xiaofan
885ecaf222
Refine time sync logic log (#15251)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-01-27 11:25:41 +08:00
Bingyi Sun
3987574448
Unsubscribe topic in DataCoord (#15353)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-24 19:47:38 +08:00
Cai Yudong
b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 17:29:34 +08:00
Cai Yudong
7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-09 23:45:33 +08:00
Cai Yudong
f8f61fb116
Use BaseParams etcd configuration for all components (#15012)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-07 16:15:26 +08:00
godchen
b0357360d2
[skip e2e]Sort imports (#14807)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 14:04:46 +08:00
Bingyi Sun
37c9419707
Use logutil in DataCoord (#14737)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-04 17:31:23 +08:00
Bingyi Sun
75b0e9935b
[skip e2e]Use logutil in DataCoord (#14739)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-04 13:53:39 +08:00
Jiquan Long
00b3fcb98b
Add log for flowgraph (#14441)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-29 14:55:21 +08:00
Xiaofan
69087ff8dd
Support embedded etcd (#14333)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 14:35:21 +08:00
Bingyi Sun
02e568603b
Add log with context in DataCoord (#14431)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-28 19:42:19 +08:00
Bingyi Sun
098516f3d7
Improve log (#13696)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-27 16:52:06 +08:00
congqixia
7c83b52de0
Add retry logic for DataCoord GC minio bucket (#14095)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-23 21:48:25 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components (#13885)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
Bingyi Sun
ca71e134d1
[skip e2e]Improve log (#13700)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-22 15:15:05 +08:00
Xiangyu Wang
1341dcd3f8
Improve import statement (#13730)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-21 15:21:25 +08:00
Bingyi Sun
9d44ea034c
[skip e2e]Improve log (#13704)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-21 09:17:00 +08:00
edward.zeng
38add97685
[skip e2e] Refine datacoord server comment (#13797)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-20 19:54:58 +08:00
Bingyi Sun
7ee0a2bf9f
[skip e2e]Improve log (#13707)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 11:18:49 +08:00
Bingyi Sun
229a727b3a
[skip e2e]Improve log (#13706)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 11:16:52 +08:00
Bingyi Sun
9e0219ec3f
[skip e2e]Improve log (#13702)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 11:14:51 +08:00
Bingyi Sun
06c4662e97
[skip e2e]Improve log (#13705)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 11:06:43 +08:00
Bingyi Sun
279316e09b
[skip e2e]Improve log (#13701)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 11:02:45 +08:00
cai.zhang
9f23fc7f2a
Register the service when the component state is healthy (#13248)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-15 11:47:10 +08:00
Bingyi Sun
0c39e0370c
Move segment statistics update to datanode tt loop (#13233)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:53:16 +08:00
congqixia
3b4d114b71
Add comment for DataCoord startFlushLoop (#13379)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-15 09:45:09 +08:00
Bingyi Sun
a78b2010e9
[skip ci]Improve log style (#13008)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-08 20:27:06 +08:00
Bingyi Sun
48c5632277
[skip ci]Improve log style (#13007)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-08 20:25:15 +08:00
Bingyi Sun
320cfbb606
[skip ci]Improve log style (#13006)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-08 20:19:26 +08:00
congqixia
de454956fe
Refactor session WatchSessions to allow rewatch when Rev Compacted (#12880)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-08 10:11:04 +08:00
congqixia
4121e31df1
Send SIGINT to runner goroutine after etcd disconnects (#12163)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-22 16:23:17 +08:00
congqixia
b99164f43d
Add datacoord GC configuration (#12061)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-18 22:29:40 +08:00
Bingyi Sun
175a656ff1
Unwatch dropped channel when init channel manager (#11986)
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-17 23:25:12 +08:00
congqixia
5edbb82610
Add session revoke (#11908)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-16 22:31:14 +08:00
Bingyi Sun
e3e4dc7153
Improve GetVChanPosistion and fix unit tests (#11888)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-16 21:41:50 +08:00
congqixia
d38043f235
Filter with partition id in GetVChanPosition (#11746)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-13 08:45:09 +08:00
Xiaofan
71e814f796
Remove common.yaml and component.yaml (#11661)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-11-12 21:25:08 +08:00
bigsheeper
93149c5ad9
Load growing segment in query node (#11664)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-12 18:27:10 +08:00
Bingyi Sun
54b40da4c5
Add segment drop logic (#11621)
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-12 00:22:42 +08:00
Bingyi Sun
7a3873960b
Fix watch channel with nil seek position (#11650)
issue: #11649
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-11 19:32:43 +08:00
congqixia
3c9610cb1b
Add garbage collector for DataCoord (#11554)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-10 19:03:38 +08:00
congqixia
0c4e09e341
Add comment for datacoord Server.startDataNodeTtLoop (#11033)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-05 23:01:02 +08:00
sunby
c26cd79532
Add compaction (#9906)
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-05 22:25:00 +08:00
sunby
a55019825b
Fix stale segments selector (#11051)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-01 22:30:04 +08:00
Xiaofan
2df2d488e1
Fix Unit test fatal fail (#10933)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-30 10:24:38 +08:00
congqixia
390fad4eb7
Fix LatestPosition option conflict with earliest patch (#10907)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-29 21:30:49 +08:00
congqixia
c019e80dbb
Change SeekPosition to earliest of all segments (#10771)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-28 14:30:26 +08:00
Xiaofan
6fddcb3925
Fix Session checker for proxy (#10737)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-27 21:58:33 +08:00
XuanYang-cn
548634ff05
[skip ci]Update license for dc segmentinfo (#10588)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-25 19:44:37 +08:00
congqixia
285c3f63e3
Trim segmentinfo binlog for VChaninfo usage (#10425)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-22 14:35:13 +08:00
congqixia
7ce7cb7a5e
Add a timed flush trigger mechanism (#10197)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-20 15:02:36 +08:00
groot
76b053a11c
Fix a typo (#10192)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-10-20 12:46:42 +08:00
Xiaofan
a0b98dacda
Skip reconsume tt msgstream when datacoord restart (#9684)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-15 11:46:33 +08:00
congqixia
6b8ff60de3
Close event channel when watch event found error (#9819)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-14 19:20:35 +08:00
congqixia
ec6e8e4127
Hide liveChn into sessionutil and fix liveness init order (#9860)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-14 16:40:35 +08:00
sunby
044b06421a
Use etcd to watch channel on datanode (#7115)
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-14 15:44:34 +08:00
congqixia
02dc2a2453
Add GetVChanPosition new cases ut (#9544)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-09 20:57:33 +08:00
godchen
937dbf4279
Add flushed segments (#9476)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-08 19:09:12 +08:00
congqixia
2711b53d57
Apply collection start position (#9459)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-08 14:51:13 +08:00
sunby
e7e481444b
Check error returned by server stop (#9015)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-08 10:09:02 +08:00
sunby
f705281884
Check error returned by ExpireAllocations (#9016)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-08 10:07:02 +08:00
congqixia
768535f31c
Add positionProvider implementation assertion (#9261)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-05 20:58:04 +08:00
godchen
2792f85211
Handle server error (#9032)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-03 22:17:56 +08:00
congqixia
e20445987e
Add Datacoord Server implementation Assertion (#9143)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-03 19:44:11 +08:00
godchen
f18e0628ae
Make NewMeta private (#9057)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-02 23:09:56 +08:00
Xiaofan
2c2b800ff4
Change Log Name with NodeId (#8721)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-01 08:52:50 +08:00
zhenshan.cao
d1d5f9e409
Replace proto.MarshalTextString with proto.Marshal (#8542)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-29 20:26:00 +08:00
congqixia
d374ea71ba
Unify liveness check in datacoord and datanode (#8480)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-28 16:10:04 +08:00
congqixia
ffbfaf7f76
Apply collection start position when open segment (#8697)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-27 20:34:03 +08:00
dragondriver
019f18ddf9
Expose created time and updated time to datacoord metrics (#8172)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-26 17:50:07 +08:00
congqixia
21f15d66bb
Add option for DataNode creation (#8240)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-18 19:10:07 +08:00
congqixia
c76f91233f
Put errors together (#7958)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-15 21:05:52 +08:00
congqixia
85d73358cc
Remove ctx from LongTermChecker (#7792)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-13 16:50:57 +08:00
sunby
49b6c3101d
Remove datacoord useless code (#7673)
Issue: #7624
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-09-09 19:08:03 +08:00
dragondriver
9b21b3b5fd
Initialize test framework of proxy (#7563)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-09 10:06:29 +08:00
congqixia
664aeb09c6
Add ut and comments for datacoord meta (#7560)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-08 11:35:59 +08:00
congqixia
fb4e23bc79
Add datacoord server unit tests (#7499)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-06 17:02:41 +08:00
dragondriver
1db573731b
Cache the system information metrics (#7436)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-03 17:15:26 +08:00
sunby
4d56d53c0b
Add a tool to move binlog path to SegmentInfo (#7243)
issue: #7214
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-24 15:51:51 +08:00
congqixia
f5451d12f3
Refactor datacoord allocator (#7226)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-08-23 17:59:51 +08:00
sunby
3cc210da26
Add flush interval (#6996)
Time tick message is sent about per 0.2 seconds. DataCoord will call
`FlushSegment` too frequently. We add a min interval for segment flush. A
segment will called `FlushSegment` only once between the interval.

issue: #6995
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-19 14:08:10 +08:00
sunby
4f7c3e7d3a
Save binlog path in SegmentInfo (#6590)
issue: #6589
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-19 13:00:12 +08:00
sunby
9551dc3b7e
Fix segment auto flush bug (#7085)
issue: #7084
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-16 11:00:09 +08:00
XuanYang-cn
6c5be83ea1
Refactor NewEtcdKV API (#7075)
This PR mainly did the following 3 things:

- Refactor NewEtcdKV API
- Add unittests for `kv/etcd/`, raise the coverage
up to 94%
- Correct some APIs from version to revision

The old NewEtcdKV() API took `clientv3.Client` as
one of the input, making everyone who using this API
had to create a new `clientv3.Client`. So there're a lot
of repeated codes everywhere.

Meanwhile, someplace didn't even check whether the
`clientv3.New()` returned error or not, and the old API
also didn't check whether the clientv3.Client was Nil.
This caused issue#6955.

Ectd's version and revision have nothing alike, they
represent different concepts, but our APIs kind of
mixed them all up. So I correct those APIs' names.

Resolves: #6955

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-08-13 11:04:09 +08:00
sunby
5c383d1d02
Add a checker to listen for the timetick. (#7061)
issue: #7060
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-08-12 19:58:08 +08:00
sunby
23bffe399d
Enhance log print in data coordinator (#6681)
issue: #6680
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-28 11:43:22 +08:00
zhenshan.cao
8a1a841011
FixBug: Incorrect handling of time synchronization (#6814)
* FixBug: proxy send wrong sync timetick and DataCoord did not filter channel

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-07-27 14:34:48 +08:00
sunby
aa20d88f6f
Fix datacoord stop bug (#6642)
issue: #6641
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-24 09:25:14 +08:00
sunby
fc74a0f578
Auto create new segments when allocating rows more than (#6665)
max number of rows per segment

If user insert too much rows in a request. Now we will return a failed
response. Maybe auto creating new segments to hold that much rows is a
better way.

issue: #6664
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-23 21:58:33 +08:00
sunby
b98b30b96f
Fix datacoord_test.go failed (#6558)
Create a mock rootcoord when testing datacoord to hide the network layer
issue: #6557
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-19 10:41:15 +08:00
sunby
83be910baa
Refactor segment manager (#6441)
issue: #6440
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-12 17:24:25 +08:00
sunby
a8e5fd2024
Refactor cluster in dataservice (#6356)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-12 11:03:52 +08:00
sunby
017484e093
Refactor meta in data coord (#6300)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-07 14:02:01 +08:00
sunby
836a45ec26
Remove sending newSegmentMsg to rootcoord (#6301)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-06 09:24:05 +08:00
sunby
07b0989628
Add GetFlushedSegments in data coordinator (#6253)
Send segment flush message by grpc and add a new interface
`GetFlushedSegments`

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-07-02 11:16:20 +08:00
sunby
dd3ba9b2b3
Add rev by one when start watch event (#6152)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-29 14:52:14 +08:00
congqixia
c49ce4ddc2
Add datacoord Server comment and improve para (#6182)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-29 10:46:13 +08:00
congqixia
b27e6b52bf
Force cluster refresh for each dn change event (#6161)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-28 13:28:14 +08:00
godchen
3785bb6330
Change retry times (#6083)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-24 19:05:06 +08:00
congqixia
f6761bbc33
Refactor datacoord segment status (#6039)
* Refactor datacoord segment status

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove lastExpireTime

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-24 14:20:10 +08:00
yukun
c8bf9078ae
Change pchannels to vchannel in retrieveResults (#6054)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-24 13:40:03 +08:00
sunby
fe8432016d
Refactor data coordinator (#6008)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-23 12:10:12 +08:00
godchen
fd07a44dba
Fix error in retry refactor (#6000)
* Fix error in retry refactor

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-23 11:48:06 +08:00
godchen
99be4c09a5
change retry (#5996)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-23 09:24:10 +08:00
sunby
0978b93a21
Refactor data coordinator (#5982)
Rename variable name and make error msg more clear

Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-22 18:24:08 +08:00
sunby
c863a193de
Rename filename from dataservice to datacoordinator (#5946)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-22 10:42:07 +08:00