Commit Graph

976 Commits

Author SHA1 Message Date
jaime
29975a7a26
Remove Consume() API from mqstream interface (#15886)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-11 20:09:59 +08:00
groot
89356ebcb9
Modify import rpc interfaces (#16006)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-11 17:13:59 +08:00
yah01
7d2934e4c2
Add methods to compress/decompress small blocks (#15980)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-11 14:55:59 +08:00
Ten Thousand Leaves
d5bae6710f
Make all logs configurable for standalone/embedded Milvus. (#15926)
Resolves: #15919, #15708
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-10 10:35:59 +08:00
groot
5fdef607d5
Add rpc interfaces for import (#15930)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
yah01
565524ed3c
Add zstd compressor in util (#15779)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-08 18:17:58 +08:00
Jiquan Long
f71651e294
Support column-based insert data in message stream (#15802)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-04 15:09:56 +08:00
Cai Yudong
3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
jaime
7a3a721380
Reconstruct mqstream module (#15784)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-03 21:57:56 +08:00
Cai Yudong
92c8e32ebd
Let MemoryKV.Load return error when key not exist (#15814)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-02 18:51:55 +08:00
XuanYang-cn
e394ba8c88
Refactor rocksmq comments and logs for readability (#15801)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-01 18:55:55 +08:00
Jiquan Long
001567640e
Fix shallow copy of typeutil.AppendFieldData (#15790)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-01 15:13:54 +08:00
Bingyi Sun
d6857f4b24
Remove globalZapWrapper in logutil.go (#15726)
issue: #15720

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-02-28 17:23:54 +08:00
yah01
867cf620ee
Add Extract() method to take out C pointer from BytesConverter without releasing it (#15614)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2022-02-27 09:47:53 +08:00
Nemo
cdcb3627b4
Make Etcd log level configurable. (#15737)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-02-25 15:03:53 +08:00
godchen
fcf0887d92
Add and implement chunk manager interface. (#15541)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-02-24 23:53:53 +08:00
shaoyue
83fdc1da7c
Add web server in proxy with sample handler (#15647)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-02-23 14:37:52 +08:00
congqixia
c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted (#15497)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-15 15:07:48 +08:00
yah01
684110bc9a
Avoid to copy during converting C bytes to Go bytes (#15551)
* Avoid to copy during converting C bytes to Go bytes

Signed-off-by: yah01 <yah2er0ne@outlook.com>

* Add BytesConverter to achieve no-copy converting C bytes to Go bytes

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2022-02-15 14:17:48 +08:00
Xiaofan
be42627ab2
Move Logutil to Util dir (#15447)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-09 18:55:46 +08:00
Cai Yudong
8255b713ac
Move ProxySubName/QueryNodeSubName/DataNodeSubName to msgChannelConfig (#15501)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-09 14:41:45 +08:00
jaime
307a8ce535
Support compile and run on Mac (#15491)
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
2022-02-09 14:27:46 +08:00
xige-16
55cdc5aa35
Get indexInfo and segment size by queryCoord (#14207)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-08 21:57:46 +08:00
Cai Yudong
1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
JunSung Park
f6873d3dc1
Implement automatic item expiration on compaction (#14418)
* Implement automatic item expiration on compaction

Signed-off-by: JunSung Park <writtic@gmail.com>

* Change entityExpiration value comparison logic

Signed-off-by: JunSung Park <writtic@gmail.com>

* Add logging for the number of expired entity

Signed-off-by: JunSung Park <writtic@gmail.com>

* Set default value of CompactionEntityExpiration to off

Signed-off-by: JunSung Park <writtic@gmail.com>

* Fix errors for code checker

Signed-off-by: JunSung Park <writtic@gmail.com>

* Set default entityExpiration to max value

Signed-off-by: JunSung Park <writtic@gmail.com>
2022-02-07 22:45:46 +08:00
Cai Yudong
f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
Cai Yudong
4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-02 00:35:43 +08:00
Bingyi Sun
515fe962d5
Fix subscription path (#15373)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-25 13:27:44 +08:00
Bingyi Sun
3987574448
Unsubscribe topic in DataCoord (#15353)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-24 19:47:38 +08:00
xige-16
1cd2363bd8
Delete usless params SkipQueryChannelRecovery (#15289)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-21 21:43:40 +08:00
Bingyi Sun
c877f33084
Downgrade pulsar client (#15325)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-20 22:25:38 +08:00
congqixia
1e61112b5a
Add retry logic in pulsar consumer unsubscribe (#15284)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-20 01:23:37 +08:00
bigsheeper
7f775096c5
Unsubscribe when close pulsar consumer (#15221)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-01-17 18:53:36 +08:00
xige-16
dfc6670f0f
Unsubscribe channel after query node down (#15230)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-17 17:37:37 +08:00
Jiquan Long
675e6d352b
QueryNodes send search/query results by rpc (#15223)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-17 14:41:35 +08:00
zhenshan.cao
1601a61b78
Move logic of checking available port to service (#15222)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-14 21:17:34 +08:00
zhenshan.cao
f14df1d2db
Initilize listener in service instead of ParamTable (#15220)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-14 18:29:34 +08:00
Cai Yudong
951249c6c1
Use KnowhereConfig for indexnode and querynode (#15112)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-11 09:35:34 +08:00
zhenshan.cao
543e348730
[skip e2e]Update license (#15131)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-11 00:09:34 +08:00
zhenshan.cao
0f7a04fc8c
[skip e2e]Update license (#15129)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-11 00:03:38 +08:00
zhenshan.cao
6d36fc60cf
[skip e2e]Update license (#15132)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-11 00:01:36 +08:00
zhenshan.cao
e5e739ec26
[skip e2e]Update license (#15127)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-10 23:59:43 +08:00
zhenshan.cao
2962ceeb65
[skip e2e]Update license (#15126)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-10 23:57:40 +08:00
jaime
828b9d399f
[skip e2e] Update license (#15123)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2022-01-10 22:45:37 +08:00
jaime
96dde53240
[skip e2e] Update license (#15122)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2022-01-10 22:43:41 +08:00
jaime
fca29c6db0
[skip e2e] Update license (#15121)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2022-01-10 22:23:46 +08:00
zhenshan.cao
1aafb56694
Clarify the index building process (#15044)
Reduce peak memory usage

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-10 20:51:34 +08:00
Cai Yudong
6323907566
Use CommonConfig in GlobalParams for all components (#15106)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 19:03:35 +08:00
jingkl
b2cb15798d
[skip e2e] Add comments to conf adapter (#15097)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-01-10 18:37:37 +08:00
jingkl
d3c6d47537
[skip e2e] Add comments to conf adapter (#15096)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-01-10 18:35:40 +08:00
Cai Yudong
b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 17:29:34 +08:00
jingkl
d719f1edb7
[skip e2e] Add comments to conf adapter (#15095)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-01-10 16:49:34 +08:00
Cai Yudong
ee60e95310
Use RocksmqConfig in GlobalParams (#15083)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 14:51:34 +08:00
edward.zeng
08bb5b3b75
[skip e2e] Update mqclient rmq id test license (#15079)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-01-10 13:35:55 +08:00
edward.zeng
bf804d74bf
[skip e2e] Update mqclient rmq id license (#15078)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-01-10 13:34:04 +08:00
edward.zeng
1b96086285
[skip e2e] Update mqclient rmq consumer license (#15077)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-01-10 13:32:04 +08:00
Xieql
f2c01e81c3
[skip e2e] Improve annotation (#15069)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2022-01-10 13:22:32 +08:00
XuanYang-cn
d80e61e26c
[skip e2e]Update license for trace stack trace (#15057)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-10 13:05:20 +08:00
XuanYang-cn
25efd97f2a
[skip e2e]Update license for trace interceptor suite (#15056)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-10 13:03:23 +08:00
XuanYang-cn
94750960c4
[skip e2e]Update license for tsoutil tso (#15055)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-10 12:58:49 +08:00
XuanYang-cn
f4b5a6733e
[skip e2e]Update license for typeutil hash (#15035)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-10 10:47:34 +08:00
Cai Yudong
7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-09 23:45:33 +08:00
XuanYang-cn
e2d16ed84d
[skip e2e]Update license for type (#15037)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-07 23:21:34 +08:00
XuanYang-cn
504550df73
[skip e2e]Update license for typeutil time (#15036)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-07 23:19:38 +08:00
XuanYang-cn
df179aa09b
[skip e2e]Update license for typeutil index (#15034)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-07 23:17:44 +08:00
XuanYang-cn
9ba61d7223
[skip e2e]Update license for typeutil schema (#15033)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-07 23:15:45 +08:00
XuanYang-cn
ef21e41175
[skip e2e]Update license for typeutil string util (#15032)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-07 23:13:50 +08:00
jaime
24b48743e5
[skip e2e] Add license (#15031)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2022-01-07 23:11:56 +08:00
jaime
def0dc33a5
[skip e2e] Add license (#15030)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2022-01-07 23:09:50 +08:00
jaime
701cbf3284
[skip e2e] Update license (#15029)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2022-01-07 23:07:42 +08:00
congqixia
eb1db3360a
[skip e2e] Improve log for rmq retention DeleteMessages (#15025)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-07 21:15:34 +08:00
jingkl
6616dbe6dd
[skip e2e]Add comments to conf adapter file (#15017)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-01-07 19:11:23 +08:00
jingkl
e108a64135
[skip e2e]Add comments to conf adapter file (#15015)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-01-07 19:07:32 +08:00
jingkl
6d526fbb3d
[skip e2e]Add comments to conf adapter file (#15014)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-01-07 19:05:41 +08:00
edward.zeng
6d86d80dcb
[skip e2e] Update mqclient rmq client license (#15007)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-01-07 18:49:25 +08:00
edward.zeng
2e012fa99f
[skip e2e] Update mqclient reader license (#15006)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-01-07 18:45:28 +08:00
edward.zeng
718564927d
[skip e2e] Update mqclient rmq client test license (#15008)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-01-07 18:43:21 +08:00
Jiquan Long
7f47ef0244
Optimize workflow of parallel (#15001)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-01-07 18:07:22 +08:00
Cai Yudong
f8f61fb116
Use BaseParams etcd configuration for all components (#15012)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-07 16:15:26 +08:00
congqixia
dec68280dc
Use ElementMatch instead of Equal in unit test (#14997)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-07 15:17:22 +08:00
Xieql
5109b88552
[skip e2e] Improve annotation (#14979)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2022-01-07 14:01:50 +08:00
Xieql
0700312a4e
[skip e2e] Improve annotation (#14977)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2022-01-07 13:59:58 +08:00
shaoyue
19bb516aab
[skip e2e] Fix comment typo for mqclient message (#14988)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-01-07 13:29:28 +08:00
shaoyue
0b74c62aed
[skip e2e] Add comment for global param (#14987)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-01-07 13:27:38 +08:00
Xieql
cc66847f9a
[skip e2e] Improve annotation (#14983)
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2022-01-07 13:25:42 +08:00
yukun
ae27be637d
[skip e2e]Add comment for pulsar producer (#14945)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2022-01-07 09:51:28 +08:00
yukun
7b873602c1
[skip e2e]Add comment for pulsar client (#14944)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2022-01-07 09:49:34 +08:00
jaime
29e37e3379
[skip e2e] Update license (#14961)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2022-01-06 23:39:34 +08:00
jaime
2751d1abbb
[skip e2e] Update license (#14959)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2022-01-06 23:37:39 +08:00
jaime
b88c313424
[skip e2e] Update license (#14960)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2022-01-06 23:19:41 +08:00
congqixia
da4182a90f
Use GroupChecker for flowgraph node input check (#14916)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-06 19:27:25 +08:00
edward.zeng
7bda81d050
[skip e2e] Update mqclient pulsar reader test license (#14933)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-01-06 18:17:54 +08:00
edward.zeng
81ccade3f3
[skip e2e] Add license for mqclient pulsar reader (#14932)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-01-06 18:16:03 +08:00
edward.zeng
04f7f7ce2a
[skip e2e] Update mqclient pulsar producer test license (#14931)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-01-06 18:14:07 +08:00
jingkl
13e27c9803
[skip e2e]Add comments to conf adapter file (#14930)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-01-06 18:12:15 +08:00
jingkl
078a1e0a42
[skip e2e]Add comments to conf adapter file (#14929)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-01-06 18:08:18 +08:00
jingkl
f68c1a7a74
[skip e2e]Add comments to conf adapter file (#14927)
Signed-off-by: jingkl <jingjing.jia@zilliz.com>
2022-01-06 18:06:15 +08:00
XuanYang-cn
92b8eddb5c
[skip e2e]Update license for util uniquegenerator (#14786)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-06 14:53:20 +08:00
XuanYang-cn
06ae5b19a3
Remove not inuse configs (#14836)
- segmentBinlgoSubPath
- collectionBinlogSubPath
- flushStreamPosSubPath
- statsStreamPosSubPath

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-06 14:49:20 +08:00
shaoyue
8b70ae0507
[skip e2e] Add comment for global param (#14926)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-01-06 13:51:21 +08:00