Commit Graph

27 Commits

Author SHA1 Message Date
yiwangdr
b1eacb2ae8
feat: datacoord/node watch based on rpc (#32036)
issue: https://github.com/milvus-io/milvus/issues/25309

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-05-07 15:49:30 +08:00
XuanYang-cn
a52a52064d
fix: Use lock and map instead of concurrentMap (#31212)
See also: #31209

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-03-14 18:39:04 +08:00
yiwangdr
c6665c2a4c
test: support multiple data/querynodes in integration test (#30618)
issue: https://github.com/milvus-io/milvus/issues/29507

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-02-21 11:54:53 +08:00
chyezh
6d63fb5d3f
fix: panic with datanode negetive wait group counter (#30135)
issue: #29170

Signed-off-by: chyezh <chyezh@outlook.com>
2024-01-30 18:15:04 +08:00
XuanYang-cn
9c8fd5e51d
fix: Save lite WatchInfo into etcd in DataNode (#29687)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-01-10 21:18:49 +08:00
smellthemoon
1c1f2a1371
enhance:change some logs (#29579)
related #29588

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-01-05 16:12:48 +08:00
XuanYang-cn
e62edb991a
enhance: Add FlowgraphManager interface (#28852)
- Change flowgraphManager to fgManagerImpl
- Change close to stop
- change execute to controlMemWaterLevel
- Change method name of fgManager for readability
- Add mockery for fgmanager

Issue: #28853

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-30 18:42:32 +08:00
smellthemoon
0aa90de141
Reduce the goroutine in flowgraph to 2 (#28233)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-13 10:50:17 +08:00
XuanYang-cn
5c5f9aa05e
Enhance newDataSyncService (#27277)
- Add flowgraph.Assemble assembles nodes in flowgraph.go
- remove fgCtx in newDataSyncService
- Add newServiceWithEtcdTickler func, reduce param numbers to 3
- Remove unnecessary params
  - config.maxQueueLength, config.maxParallelish

See also: #27207

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-27 11:07:25 +08:00
SimFG
26f06dd732
Format the code (#27275)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
XuanYang-cn
09505ea78e
Move etcd watch related code into eventmanager (#27192)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-19 15:27:38 +08:00
yiwangdr
337edc321b
tikv integration (#26246)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-09-07 07:25:14 +08:00
SimFG
b9bc6681ae
Use atomic.bool for the isWatchFailed param in the tickler (#26558)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-23 13:22:27 +08:00
SimFG
3be4ac4022
Fix datanode/datacoord continuous restart (#26470)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-08-20 21:20:24 +08:00
Enwei Jiao
66fdc71479
Refactor logs in DataCoord & DataNode (#25574)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-14 15:56:31 +08:00
SimFG
f9e2d00f91
Prevent exclusive consumer exception in pulsar (#25376)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-07-12 17:26:30 +08:00
congqixia
03297a7ef7
Fix event manager not closed after datanode stop (#24206)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-18 11:43:23 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
zhenshan.cao
e768437681
Correct usage of Timer and Ticker (#22228)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-02-23 18:59:45 +08:00
aoiasd
148a024e05
Add tickle for datacoord watch event (#21193)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-15 16:20:34 +08:00
zhenshan.cao
c15b880f0e
Remove redundant logs (#19412)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-09-26 15:40:53 +08:00
Xiaofan
a9b1d71a8f
change the etcd compareAndSwap interface (#18068)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-07-06 13:54:21 +08:00
XuanYang-cn
93777e0c9d
Handle flowgraph releasing properly (#16169)
See also: #15846

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-04-20 16:45:41 +08:00
congqixia
6513be5c44
Extend watch/release channel logic in DataNode (#15925)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-09 15:39:58 +08:00
Xiaofan
bd7d294c6d
Refine log level in data (#15821)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-02 15:35:55 +08:00
XuanYang-cn
23dfbf1f34
Fix eventmanager retry interval (#15631)
See also: #15608

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-02-18 17:31:49 +08:00
XuanYang-cn
008e08a996
Add flowgraph manager and event manager (#15102)
1. Add flowgraph manager to manager start and release of dataSyncService
2. Add event manager to manager etcd watchInfo event
  - Make put event able to retry
  - Make delete event able to terminate the retry loop

See also: #14604, #14300

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-13 14:21:34 +08:00