test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 21:09:06 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
20,240 Commits 96 Branches 105 Tags 186 MiB
eeb03a0d6a
Commit Graph

52 Commits

Author SHA1 Message Date
yihao.dai
3540eee977
enhance: Support L0 import (#33514) 
issue: https://github.com/milvus-io/milvus/issues/33157

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-06-07 14:17:20 +08:00
smellthemoon
c61fb1eff5
enhance: do check when add not empty logpath (#33640) 
meta only store logid

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-06-07 10:19:51 +08:00
SimFG
b1a1cca10b
feat: add more operation detail info for better allocation (#30438) 
issue: #30436

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-03-28 06:33:11 +08:00
yihao.dai
0fe5e90e8b
enhance: Remove import v1 (#31403) 
Remove all code and logic related to import v1.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-22 15:29:09 +08:00
yihao.dai
a434d33e75
feat: Add import scheduler and manager (#29367) 
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-03-01 18:31:02 +08:00
yihao.dai
7ce876a072
fix: Decoupling importing segment from flush process (#30402) 
This pr decoups importing segment from flush process by:
1. Exclude the importing segment from the flush policy, this approch
avoids notifying the datanode to flush the importing segment, which may
not exist.
2. When RootCoord call Flush, DataCoord directly set the importing
segment state to `Flushed`.

issue: https://github.com/milvus-io/milvus/issues/30359

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-02-03 13:01:12 +08:00
aoiasd
1d4be0d257
Adjust datacoord for L0 Delta (#28021) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-06 15:26:16 +08:00
jaime
7f7c71ea7d
Decoupling client and server API in types interface (#27186) 
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-09-26 09:57:25 +08:00
SimFG
26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
MrPresent-Han
4f6122dd4a
enable ctx traceId for assignsegment on dc(#26972) (#27108) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-09-18 09:53:22 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam (#26725) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-05 10:31:48 +08:00
MrPresent-Han
f4e72cb170
remove sync segmentLastExpire every time when assigning(#25271) (#25316) (#25557) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-07-24 14:11:07 +08:00
Bingyi Sun
268368031c
Remove redundant kv mocks (#25420) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-07-07 19:38:26 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message (#24736) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-08 15:36:36 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
yah01
319f1773af
Fix segment not found if forward delete to empty segment (#22528) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-03 14:13:49 +08:00
jaime
748a935a11
Modify segment seal policy with number of statslog files (#21957) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-03 17:07:52 +08:00
Xiaofan
f1daef22a1
Fix failed bulkload segment marked as sealed (#21799) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-31 12:41:53 +08:00
jaime
73d8509600
Improve ListSegments performance for datacoord catalog (#21477) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-06 14:33:36 +08:00
jaime
d401608899
Add a segment seal policy by number of binlog files (#21263) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-20 14:09:25 +08:00
Bingyi Sun
2390095232
Fix load uses compacted segments' binlogs (#20655) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-11-17 20:37:10 +08:00
XuanYang-cn
22fb1d3b93
Remove dup mock_kv.go and mock_txn_kv.go (#20267) 
Replace them with mockery generated mocks in kv/mocks

See also: #19289

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-03 14:41:35 +08:00
SimFG
a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
jaime
31db01b6ae
Support TTL for collection level (#19383) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-10 20:31:22 +08:00
jaime
91df8f2d6a
Use log id instead of log path of the binlog in metastore (#19123) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-09-25 15:56:51 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
codeman
70254c3ec5
Unified catalog interface for segment (#18289) (#18290) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-08-20 10:24:51 +08:00
Ten Thousand Leaves
f0b036a35a
Move bulk load segment lock happen early (#17612) 
issue: #17600
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-27 13:56:17 +08:00
Bingyi Sun
05273309e8
Drop empty sealed segment (#17708) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-06-24 15:16:14 +08:00
groot
f31be941bb
Fix bulkload row count bug for multi datanodes (#17384) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-06-07 16:56:07 +08:00
Ten Thousand Leaves
e66ac6a77c
Resolve an issue with bulk load where segments in flow graph replica are not updated (#16609) 
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-25 11:07:47 +08:00
godchen
f4caa94ea8
[skip e2e]Fix segment manager typo (#13644) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-17 22:34:42 +08:00
Bingyi Sun
0c39e0370c
Move segment statistics update to datanode tt loop (#13233) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-15 10:53:16 +08:00
Xiangyu Wang
405b3cd932
Use the same lint rules with golint in revive (#13324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-14 15:31:07 +08:00
congqixia
06ced4d1d1
Add DropVirtualChannel for DataCoord (#12361) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-29 22:35:41 +08:00
Bingyi Sun
5c8978e5d6
Fix drop segment bug (#11709) 
issue: #11708
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-12 17:31:11 +08:00
Bingyi Sun
54b40da4c5
Add segment drop logic (#11621) 
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-12 00:22:42 +08:00
XuanYang-cn
548634ff05
[skip ci]Update license for dc segmentinfo (#10588) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-25 19:44:37 +08:00
godchen
f18e0628ae
Make NewMeta private (#9057) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-02 23:09:56 +08:00
sunby
c152cf2b6b
Check error in segment_manager_test.go (#8706) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-09-27 22:42:02 +08:00
congqixia
664aeb09c6
Add ut and comments for datacoord meta (#7560) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-08 11:35:59 +08:00
congqixia
5b22697ec4
Add segment manager unit tests (#7525) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-07 13:59:58 +08:00
congqixia
f5451d12f3
Refactor datacoord allocator (#7226) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-08-23 17:59:51 +08:00
sunby
3cc210da26
Add flush interval (#6996) 
Time tick message is sent about per 0.2 seconds. DataCoord will call
`FlushSegment` too frequently. We add a min interval for segment flush. A
segment will called `FlushSegment` only once between the interval.

issue: #6995
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-08-19 14:08:10 +08:00
sunby
31182e52a9
Fix expire allocation bug (#7053) 
issue: #7052
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-08-12 10:48:08 +08:00
sunby
fc74a0f578
Auto create new segments when allocating rows more than (#6665) 
max number of rows per segment

If user insert too much rows in a request. Now we will return a failed
response. Maybe auto creating new segments to hold that much rows is a
better way.

issue: #6664
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-23 21:58:33 +08:00
sunby
83be910baa
Refactor segment manager (#6441) 
issue: #6440
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-12 17:24:25 +08:00
sunby
017484e093
Refactor meta in data coord (#6300) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-07 14:02:01 +08:00
congqixia
f6761bbc33
Refactor datacoord segment status (#6039) 
* Refactor datacoord segment status

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove lastExpireTime

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-24 14:20:10 +08:00