test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-11-30 10:59:32 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
19,824 Commits 94 Branches 105 Tags 184 MiB
b560602885
Commit Graph

19824 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Cai Yudong
b560602885
enhance: Store SparseFloatVector into parquet as JSON string (#33101) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-05-17 15:01:37 +08:00
Yinzuo Jiang
2cc50d80a3
fix: add openblas in install_deps.sh (#33065) 
Install openblas using apt or yum in scripts/install_deps.sh, update
documentations and fix some typos related to build and installation.

issue: #33056, #33066

Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
 2024-05-17 14:53:37 +08:00
smellthemoon
225f4a6134
enhance: use the only MaxEtcdTxnNum (#33070) 
#33071

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-05-17 14:27:42 +08:00
congqixia
1ef975d327
enhance: Update latest sdk update to client pkg (#33105) 
Related to #31293
See also milvus-io/milvus-sdk-go#704 milvus-io/milvus-sdk-go#711 
milvus-io/milvus-sdk-go#713 milvus-io/milvus-sdk-go#721
milvus-io/milvus-sdk-go#732 milvus-io/milvus-sdk-go#739 
milvus-io/milvus-sdk-go#748

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-17 10:39:37 +08:00
wei liu
f1c9986974
enhance: Skip return data distribution if no change happen (#32814) 
issue: #32813

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-05-17 10:11:37 +08:00
sre-ci-robot
5b27a0cd2c Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-05-16 12:01:03 +00:00
smellthemoon
1671c7898a
enhance: sync milvus proto version (#33094) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-05-16 18:51:34 +08:00
binbin
be87ae6424
test: update pymilvus version (#33079) 
issue: #17614

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
 2024-05-16 14:53:34 +08:00
Cai Yudong
4ef163fb70
enhance: Support readable JSON file import for Float16/BFloat16/SparseFloat (#33064) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-05-16 14:47:35 +08:00
congqixia
892fe66b57
enhance: Refine channelCpUpdater field & test (#33083) 
Avoid passing datanode around preparing datanode code directory
refactory.

Also refine unit test code for same component. The `Await` shall return
first before checking the counter number since when lock cost is heavy
(using deadlock.RWMutex See PR #33069.) case may fail due to long
running time submitting tasks.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-16 14:19:34 +08:00
smellthemoon
b45798107a
enhance: add nullable in Field, check valid_data and fill data (#32086) 
1. add nullable in model.Field
   help to read nullable accurately.
2. check valid_data
a. if user pass default_value or the field is nullable, the length of
valid_data must be num_rows.
b. if passed valid_data, the length of passed field data must equal to
the number of 'true' in valid_data.
c. after fill default_value, only nullable field will still has
valid_data.
3. fill data in two situation
    a. has no default_value, if nullable,
will fill nullValue when passed num_rows not equal to expected num_rows.
    b. has default_value,
will fill default_value when passed num_rows not equal to expected
num_rows.
c. after fill data, the length of all field will equal to passed
num_rows.
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-05-16 11:57:35 +08:00
aoiasd
875ad88d84
enhance: Support access log use stdout with write cache and auto flush as time (#32213) 
relate: https://github.com/milvus-io/milvus/issues/28948

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-05-16 10:05:34 +08:00
sre-ci-robot
9a56cba7af Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-05-15 12:00:50 +00:00
Jiquan Long
dd9919a7dc
fix: two-phase retrieval on lru-segment (#32945) 
issue: #31822

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-05-15 17:53:34 +08:00
congqixia
8cf2cf5c94
enhance: Add go-deadlock as unittest only dependency (#33063) 
See also #33062

This PR:

- Add `lock.RWMutex` & `lock.Mutex` alias to switch implementation based
  on build flags
- When build flags has `test` in it, use `go-deadlock` to detect
  possible deadlocks
- Replace all `sync.RWMutex` & `sync.Mutex` in datacoord pkg

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-15 16:33:34 +08:00
Buqian Zheng
7c60d725cc
fix: validate sparse vector in search request (#32856) 
issue: #32368

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-05-15 15:39:33 +08:00
cai.zhang
4382cf5283
enhance: Reduce the frequency of index ralted logs (#33050) 
issue: #33001

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-05-15 14:37:34 +08:00
congqixia
b2d83d3354
enhance: Bump milvus version to v2.4.2 (#33048) 
Bumping version to v2.4.2. Also bump milvus-proto version to v2.4.3.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-15 13:57:35 +08:00
wayblink
259bc97a2b
fix: Fix segments lost in flush response (#33061) 
#33055

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2024-05-15 13:49:34 +08:00
cai.zhang
6ea7633bd5
enhance: Add memory size for binlog (#33025) 
issue: #33005
1. add `MemorySize` field for insert binlog.
2. `LogSize` means the file size in the storage object.
3. `MemorySize` means the size of the data in the memory.

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2024-05-15 12:59:34 +08:00
XuanYang-cn
d4837307b3
fix: Make submit idempotent (#33053) 
issue: #33054

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-14 22:09:34 +08:00
yihao.dai
6fc00e42d7
fix: Fix binlog import (#33027) 
binlog import should only import into one partition.

issue: https://github.com/milvus-io/milvus/issues/32807

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-14 21:45:32 +08:00
Gao
3777eb2f68
enhance: update knowhere new version (#33043) 
for milvus new release, knowhere update to latest release version

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2024-05-14 20:45:32 +08:00
jaime
f48a7ff8ff
enhance: use Delete instead of DeletePartialMatch to remove metrics (#33029) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-05-14 18:49:33 +08:00
smellthemoon
89a7c34c7a
fix: exceed etcd limit (#33041) 
#32974

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-05-14 18:45:32 +08:00
congqixia
dea7faf394
fix: Remove RLock re-entry in GetNumRowsOfPartition (#33045) 
Resolves #33044

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-14 17:51:32 +08:00
Jiquan Long
3d85e6e028
fix: etcd txn exceeds limit due to too many fields (#33040) 
fix: #33038

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-05-14 17:05:33 +08:00
yanliang567
ba3b2a91a0
test: Remove useless common types and refine error assert in negative cases (#33023) 
Related issue: #32653 

1. Remove some meaningless common types
2. Refine error assertion in negative cases
3. Remove some dup tests

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
 2024-05-14 15:03:33 +08:00
SimFG
1d48d0aeb2
enhance: use different value to get related data size according to segment type (#33017) 
issue: #30436

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-05-14 14:59:33 +08:00
Cai Yudong
4fc7915c70
enhance: unify data generation test APIs (#32955) 
Issue: #22837

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2024-05-14 14:33:33 +08:00
chyezh
96489b814d
fix: remove busy log (#33042) 
issue: #32963

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-05-14 14:20:32 +08:00
congqixia
861977ab60
fix: Start LeaderCacheObserver before SyncAll (#33035) 
Related to #33033

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-14 13:25:32 +08:00
XuanYang-cn
40e5c0be70
fix: Enable to balance released standby channel (#32986) 
See also: #32879

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-14 11:51:32 +08:00
aoiasd
dc058eaf61
fix: should init access logger before init server (#32976) 
relate: https://github.com/milvus-io/milvus/issues/32968

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-05-14 10:47:32 +08:00
zhuwenxing
3871009cf5
test: add timeout for GetMetrics grpc interface (#33024) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-05-14 10:29:32 +08:00
congqixia
4ae7cabb04
fix: Remove channel when create flowgraph timeout (#33014) 
See also #33013

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-14 10:07:33 +08:00
wei liu
cba2c7a3be
enhance: clean channel node info in meta store (#32988) 
issue: #32910
see also: #32911
when channel exclusive mode is enabled, replica will record channel node
info in meta store, and if the balance policy changes, which means
channel exclusive mode is disabled, we should clean up the channel node
info in meta store, and stop to balance node between channels.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-05-14 10:05:40 +08:00
yihao.dai
a984e46a29
enhance: Remove rootcoord from datanode broker (#32818) 
issue: https://github.com/milvus-io/milvus/issues/32827

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-05-14 10:03:32 +08:00
sre-ci-robot
09cca1fb51
[automated] Update Knowhere Commit (#33030) 
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-05-14 01:47:32 +08:00
zhuwenxing
9e3f3d99f7
test: add timeout for import testcases (#33006) 
add timeout for import testcases

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-05-13 19:35:32 +08:00
XuanYang-cn
efdbd8e7c1
enhance: Enable to upload by batch (#32788) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-13 19:07:32 +08:00
XuanYang-cn
29b621f759
fix: Make compactor able to clear empty segments (#32821) 
See also: #32553

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-13 18:21:32 +08:00
jaime
ba625835bc
enhance: Add metrics for segment index files size (#32979) 
issue:#32980

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-05-13 17:59:32 +08:00
chyezh
293f14a8b9
fix: remove redundant replica recover (#32985) 
issue: #22288 

- replica recover should be only triggered by replica recover

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-05-13 15:25:32 +08:00
foxspy
f6777267e3
enhance: add score compute consistency config for knowhere (#32997) 
issue: https://github.com/milvus-io/milvus/issues/32583
related: #32584

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
 2024-05-13 14:21:31 +08:00
SimFG
4031abd2fa
enhance: change default partition num to 16 when using partition key (#32950) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-05-13 14:19:31 +08:00
congqixia
12ec3d61d9
fix: Fill deltalog entry num & time range in L0 compactions (#33004) 
Resolves #33003

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-13 14:13:31 +08:00
Bingyi Sun
4724779b3b
enhance: remove fallback keys for config generator (#32946) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-05-13 13:33:31 +08:00
Xiaofan
230c88b1de
enhance: improve reduce performance on very large topk (#32871) 
fix #32870
reduce cpu usage for reduce on large k

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-05-13 11:55:31 +08:00
congqixia
ef41f80bd0
enhance: Sync go.mod version & dep to main pkg (#32973) 
Related to #31293
See also #32951 #32952 #32953 #32954

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-05-13 10:25:32 +08:00