Commit Graph

28 Commits

Author SHA1 Message Date
Enwei Jiao
697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
aoiasd
cc5d72a9f2
add search option to ignore ignore segment (#21994)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-22 17:31:46 +08:00
yah01
47b76e13be
Fix shard leader misses to fill target ID while delegates query/search (#20973)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-12-05 14:09:17 +08:00
yah01
2d806788fb
Add version for subscribed channel (#20585)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-15 13:21:07 +08:00
congqixia
3de74165ac
Fix SyncSegment generate source (#19929)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-20 16:35:28 +08:00
SimFG
a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
congqixia
8c1c8000ac
Add TargetID in MsgBase and querynode v2 API check (#19627)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-08 20:26:57 +08:00
yah01
377f856833
Fix balance may confuse leader observer (#19435)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-28 12:10:54 +08:00
congqixia
384e7c3169
Fix force release segment logic (#19471)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-27 19:22:54 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
Bingyi Sun
626854cf0c
Refactor QueryCoord (#18836)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-15 18:48:32 +08:00
congqixia
6ea0fb82ab
Add InvalidNodeID logic to remove segments from offline segments (#18975)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-02 10:42:59 +08:00
congqixia
f0fe8dae0b
Close Node/Segment detector when close ShardCluster (#18476)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-01 13:50:33 +08:00
Jeng.Gwan
638f6c36e9
Support to get real row count of segment (#18115)
Signed-off-by: xaxys <zheng.guan@zilliz.com>
2022-07-18 09:58:28 +08:00
congqixia
543197c2be
Fix handoff segment unit test uses wrong node id (#18137)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-07 15:22:21 +08:00
Zach
1c9647ff31
Merging multiple shard query/search requests into one rpc request if QueryNode is the leader of multiple shards of Collection (#17588)
Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
2022-07-06 15:06:21 +08:00
yah01
797218a8ad
Fix handoff isn't processed correctly (#17738)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-27 22:08:17 +08:00
congqixia
5471e35ccc
Wait last version search to release growing safely (#17713)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-25 18:48:15 +08:00
congqixia
785a5a757f
Use segment version instead of ref cnt (#17609)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 17:38:12 +08:00
congqixia
a0614a3ff8
Use NodeIds instead of NodeID in segment info (#16912)
The NodeID field is deprecated in current design
Query Node segment detector shall read NodeIds instead of NodeID

Fix #16911

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-11 17:39:53 +08:00
congqixia
c7935d1a86
Add legacySegments for load balance segments rc track (#16831)
- Add legacySegments in ShardCluster tracking old segment inUse
- Apply node id check for segment online/inUse check
- Add unit test simulating load balance case

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-07 18:25:52 +08:00
yah01
415a0e5978
Fix miss segments/DmChannel when query by nodeID (#16719)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-29 14:11:47 +08:00
congqixia
3a6db2faeb
Fix handling segment change logic (#16695)
Dispatch segmentChangeInfo to ShardCluster leader
Hold segment remove before search is done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-27 22:23:46 +08:00
congqixia
0bf176a7e1
Add SyncReplicaSegments interface for QueryNode (#16672)
Add SyncReplicaSegments for LoadCollection/LoadPartition child task
This API notifies related ShardCluster to record segment Loaded
Preventing the Etcd events is delayed, which will cause some segment not search

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-27 10:41:46 +08:00
congqixia
b99b65c26e
Add segment reference count and handles change info in ShardCluster (#16620)
Resolves #16619
Add reference count for each search/query request
For SegmentChangeInfo
- Wait all segments in OnlineList to be loaded
- Add handoff event into pending list
- Wait all segments in OfflineList is not used (reference count = 0)

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-25 11:51:46 +08:00
congqixia
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470)
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
congqixia
18d76261a0
Add ShardNodeDetector implementation (#16373)
Use etcd as a ShardNodeDetector
Unit tests is based on embedded etcd

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-06 15:43:31 +08:00
congqixia
aa1c26de77
Add ShardCluster implementation (#16360)
ShardCluster maintains shard replica meta information
It watches node & segment change events
Provides shard replica search/query services

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-02 17:35:29 +08:00