test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-11-29 18:38:44 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
20,408 Commits 94 Branches 105 Tags 184 MiB
aa247f192d
Commit Graph

20408 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
zhenshan.cao
aa247f192d
enhance: remove unused code for StorageV2 (#35132) 
issue: https://github.com/milvus-io/milvus/issues/34168

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2024-08-01 12:08:13 +08:00
cai.zhang
9412002d7d
fix: Fix data race for clustering buffer writer (#35145) 
issue: #34495

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-01 11:20:13 +08:00
cai.zhang
089fb2b5fb
enhance: Skip updating the level that are already at the specified level (#35094) 
issue: #34495

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-08-01 11:18:12 +08:00
Bingyi Sun
f229f244d2
enhance: add chunk basic impl (#34634) 
https://github.com/milvus-io/milvus/issues/35112
This pr would not affect milvus functionality by now.
It implments a Chunk memory layout that looks like 
```
VariableColumn
|offset|offset|offset|
|data|data|data|

```
We maybe move offsets to the beginning and add null bitmaps later but
not in this PR.
And mmap test will also be added in another PR.

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-08-01 10:29:51 +08:00
wei liu
e9d61daa3f
enhance: Reduce delegator memory overloaded factor to 0.1 (#35092) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-08-01 10:21:50 +08:00
congqixia
dfda9f0478
enhance: Add depguard rules to ban deprecated proto lib (#35140) 
See also #34394 #34252

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-08-01 10:01:49 +08:00
sre-ci-robot
c15d074b21
[automated] Update Knowhere Commit (#35157) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-08-01 01:17:49 +08:00
Gao
6695c6d0a3
enhance: add channel num for queryHook optimization (#35104) 
At most cases, data in each channel is almost evenly distributed, we
could utilize the channel num info to optimize searh param in queryHook

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-07-31 18:59:49 +08:00
wei liu
03912a8788
enhance: Avoid balance stuck after segment list become stable (#34728) 
issue: #34715
if collection's segment list doesn't changes anymore, then the next
target will be empty at most time, and balance segment will check
whether segment exist in both current and next target, so the balance
cloud be blocked due to next target is empty.

This PR permit segment to be moved if next target is empty, to avoid
balance stuck.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-31 18:09:48 +08:00
Patrick Weizhi Xu
a9744d62b1
enhance: improve error msg when expr invalid during isolation (#35127) 
issue: #34336 

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit b1d7b24972e9414ab9488c7611aca9975e674cd4)
 2024-07-31 17:23:51 +08:00
smellthemoon
6106a48acb
fix: upsert result use the previous pk (#34672) 
#34668

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-07-31 15:25:51 +08:00
Patrick Weizhi Xu
acaa78db58
fix: disable sparse and binary in MV (#35126) 
issue: #35120 

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 0d340ebe7412cd85963825bc8abd3c874e339679)
 2024-07-31 15:02:01 +08:00
congqixia
f7f9a729c9
enhance: Pre-allocate space for reduce data structure (#35118) 
Grow slice & map.growWork may cause a lot when segment number is large
for big K query. This PR pre-allocate space for reduce methods to avoid
this cost.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-31 10:35:49 +08:00
foxspy
48b14c56f6
fix: add prometheus dependency for milvus_monitor (#35078) 
add prometheus dependency for monitor module. Or else Some compilers may
report a compilation failure.
issue: #35077

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
 2024-07-31 10:15:48 +08:00
wayblink
f919b6e1b8
enhance: refine error handle in clustering compaction task (#34956) 
related: #30633 #34792

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-30 20:37:57 +08:00
wayblink
ce3f836876
fix: compaction task not be cleaned correctly (#34765) 
1.fix compaction task not be cleaned correctly
2.add a new parameter to control compaction gc loop interval
3.remove some useless configs of clustering compaction

bug: #34764

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-07-30 20:21:56 +08:00
yihao.dai
39d25938c5
enhance: Avoid panic due to nil schema (#35063) 
/kind improvement

issue: https://github.com/milvus-io/milvus/discussions/25620

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-30 20:18:32 +08:00
zhagnlu
dd0c26cf58
enhance: redefine variable column block size (#35040) 
#35013

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-30 19:23:50 +08:00
yellow-shine
273e9203f4
feat: [skip e2e] effective way to use cache (#34774) (#35109) 
See also #34483

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-07-30 18:41:51 +08:00
congqixia
b787bb4783
enhance: [skip e2e] Skip Amazon&Rockylinux instead of commenting job (#35108) 
Related to #35084

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 16:23:49 +08:00
congqixia
de8a266d8a
enhance: Enable linux code checker (#35084) 
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 15:53:51 +08:00
wei liu
4bb969ef8f
enhance: Skip manual stopped component during health check (#34953) 
after manual stop component by management restful api, `healthz` may
return unhealthy state. k8s may restart the pod to save the unhealthy
sate, and the manual stop operation will got unexpected result.

to solve this, we make `healthz` API skip the manual stopped component.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-30 14:37:51 +08:00
congqixia
cabb200498
enhance: [GoSDK] Expose indexing progress in DescribeIndex result (#35066) 
See also #31293

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 10:33:49 +08:00
zhagnlu
a8a4779749
fix: fix parse plan proto failed for search type (#34944) 
#25848

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-29 21:19:49 +08:00
congqixia
d36cfe71e5
enhance: Refine protobuf dependency installation in Makefile (#35072) 
Related to #34394

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-29 21:15:48 +08:00
zhagnlu
86322e0468
fix: fix query count(*) concurrently (#35007) 
#34778
#34849
fix two problems:
1. count(*) incorrect, if growing insert duplicated (pk, timestamp)
pairs that pk and timestamp all same, need to keep just one pair.
2. count(*) may core dump, if get_real_count interface get snapshot and
do mvcc at not consistency status, mainly happens under concurrency.

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-29 19:53:50 +08:00
congqixia
972752258a
enhance: Support otlp http exporter (#35053) 
See also #35052

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-29 17:43:49 +08:00
chyezh
a495754034
enhance: the datacoord gc should fast quitable (#35050) 
issue: #35049

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-07-29 17:03:48 +08:00
yellow-shine
e62f6b2f4e
enhance: [skip e2e] enlarge timeout of UT for go up to 90 mins (#35035) 
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-07-29 15:51:49 +08:00
Patrick Weizhi Xu
3c7f73137e
fix: disallow expr when partition key isolation is enabled (#35031) 
issue: #34336 

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 2889481ce9a14230e9c7f1c8f9c3c1decde77e03)
 2024-07-29 14:21:50 +08:00
SimFG
0e41f104c5
enhance: use the collection id to group msg pack in the msg dispatcher (#34871) 
- issue: #34870

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-07-29 11:45:49 +08:00
wei liu
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394) 
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-29 11:31:51 +08:00
Chun Han
e2e38e98df
fix: nil part stats without l2 compaction(#34923) (#34992) 
related: #34923

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-07-29 11:07:48 +08:00
XuanYang-cn
9a07790215
enhance: Add ut for l0CompactionTask processExecuting (#34800) 
See also: #34796

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-29 10:53:48 +08:00
yihao.dai
4a3d98d88c
enhance: Fast close msgdispatcher target (#34803) 
/kind improvement

issue: https://github.com/milvus-io/milvus/issues/34075

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-29 00:25:49 +08:00
zhagnlu
f77f5364b2
fix: disable use_index when some array expr (#34894) 
#34797

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-29 00:17:46 +08:00
PowderLi
5584ae2e14
enhance: add metrics for restful apis (#34969) 
issue: #34968

Signed-off-by: PowderLi <min.li@zilliz.com>
 2024-07-29 00:13:47 +08:00
cai.zhang
497afcb897
enhance: Refine code for GetRecoveryInfo (#34973) 
issue: #34495

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-29 00:01:46 +08:00
cai.zhang
2372452fac
enhance: Optimized the GC logic to ensure that memory is released in time (#34949) 
issue: #34703

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-28 23:53:47 +08:00
jaime
08fa51d4f4
fix: memory leak while parsing query plan (#34931) 
issue: #34930

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-28 21:50:20 +08:00
Aldrin
9463eeef2b
fix: Avoided dereferencing NULL pointer (#34836) 
issue : https://github.com/milvus-io/milvus/issues/34835

Signed-off-by: Ald392 <imagesai32@gmail.com>
 2024-07-27 17:27:52 +08:00
zhagnlu
804ec24c02
fix:fix retrieve raw data from bitmap array index (#34848) 
#34795

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-07-27 01:53:47 +08:00
sre-ci-robot
e93687cdd3
[automated] Update Knowhere Commit (#35039) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-07-27 00:59:46 +08:00
congqixia
b86d795d6d
enhance: [GoSDK]Support specifying field schema to be cluster key (#35027) 
See also #32939 milvus-io/milvus-sdk-go#794

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 20:01:45 +08:00
congqixia
783f9d9c33
fix: Unify hook singleton implementation in proxy (#34887) 
Related to #34885

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 18:07:53 +08:00
wei liu
6e9fbd1630
enhance: Solve channel unbalance on datanode (#34984) 
issue: #33583
the old policy permit datanode has at most 2 more channels than other
datanode. so if milvus has 2 datanode and 2 channels, both 2 channels
will be assign to 1 datanode, left another datanode empty.

This PR refine the balance policy to solve channel unbalance on datanode

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-26 17:35:47 +08:00
congqixia
94d8f48b85
enhance: Use auto detection proto interface to fetch grpc size (#35024) 
Related to #34763

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 14:23:47 +08:00
cai.zhang
ea13b97b6a
fix: Fix bug for block clustering compaction (#35019) 
issue: #34703

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-07-26 11:33:58 +08:00
chyezh
1cff55381d
enhance: add manual alloc segment rpc for datacoord (#35002) 
issue: #33285

- segment allocation will move to streamingnode, so a manual alloc
segment rpc is required

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-07-26 10:15:46 +08:00
congqixia
4ee6c69217
enhance: Add Segment Level in milvus segment info APIs (#34763) 
See also #34746

This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-26 10:01:46 +08:00