chyezh
cc8f7aa110
fix: streaming service related fix patch ( #34696 )
...
issue: #33285
- add idAlloc interface
- fix binary unsafe bug for message
- fix service discovery lost when repeated address with different server
id
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-16 15:49:38 +08:00
cai.zhang
fbc10ac695
fix:Reset flushed row num after pack segment for clustering compaction ( #34702 )
...
issue: #34703
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-16 15:37:38 +08:00
zhagnlu
804dd5409a
enhance: mark duplicated pk as deleted ( #34586 )
...
fix #34247
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-16 14:25:39 +08:00
Xiaofan
be7760a9ab
fix: CVE by upgrading some dependencies. ( #34462 )
...
fix #34434 and #34456
upgrade otelgrpc to fix CVE
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-07-16 11:55:36 +08:00
smellthemoon
b5b5afb63f
fix: [skip e2e]golang lint check fail ( #34682 )
...
#34680
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-16 10:47:36 +08:00
congqixia
ceb138da6d
enhance: [GoSDK] Add release methods & GPU indexes ( #34690 )
...
Related to #31293
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-16 10:27:38 +08:00
balloon1995
7306d2d115
fix: fix metaCache cleanup issue when listPolicy failed ( #34449 )
...
issue: #34667
---------
Signed-off-by: balloon1995 <hszoe1995@outlook.com>
Co-authored-by: congqixia <congqi.xia@zilliz.com>
2024-07-16 10:03:38 +08:00
congqixia
b5225df21c
enhance: Use unified requery flag when set metrics ( #34632 )
...
See also #34601
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-15 23:03:38 +08:00
congqixia
eb4bfa3281
fix: Revert reuse deserialize result to fix data overwritten ( #34683 )
...
See also #34637
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-15 22:31:38 +08:00
wei liu
acb33bba4d
enhance: Preserve fixed-size memory in delegator node for growing segment. ( #34596 )
...
issue: #34595
When consuming insert data on the delegator node, QueryCoord will move
out some sealed segments to manage its memory usage. After the growing
segment gets flushed, some sealed segments from other workers will be
moved back to the delegator node. To avoid the frequent movement of
segments, we estimate the maximum growing row count and preserve a
fixed-size memory in the delegator node.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-15 20:51:46 +08:00
chyezh
fda720b880
enhance: streaming service grpc utilities ( #34436 )
...
issue: #33285
- add two grpc resolver (by session and by streaming coord assignment
service)
- add one grpc balancer (by serverID and roundrobin)
- add lazy conn to avoid block by first service discovery
- add some utility function for streaming service
Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-15 20:49:38 +08:00
XuanYang-cn
eb472b7f08
enhance: [skip e2e]Enable compaction it test ( #34526 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-07-15 20:45:39 +08:00
yihao.dai
1495b7a2c9
fix: Fix no L0 compaction tasks are executed ( #34537 )
...
The nodeID for compaction task initialization is 0. This PR adjusts the
task reassignment conditions to allow new compaction tasks to be
reassigned and executed.
issue: https://github.com/milvus-io/milvus/issues/34460
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-15 17:01:37 +08:00
congqixia
8b5754f7fe
enhance: Add segment seal proportion jitter ( #34636 )
...
See also #34574
Add jitter for segment seal proportion to avoid seal operation burst in
short period of time.
This PR also fix license header in paramtable pkg.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-15 14:47:39 +08:00
wayblink
48898cefea
fix: fix meta revert bug when clustering compaction task fail ( #34604 )
...
#34603
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-14 22:29:38 +08:00
zhagnlu
bd9727a1f7
fix: fix bug that set incorrect info to columnbase ( #34428 )
...
#34427
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-14 22:27:46 +08:00
zhagnlu
18c83c6466
fix: fix auto merge error ( #34661 )
...
#33704
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-13 13:07:37 +08:00
elstic
2c462d387b
test: Fix index creation mismatch for sparse vector columns in multivector case ( #34618 )
...
If `is_all_data_type` is true, the case will add float32, f16, bf16, and
sparse vectors, but the created indexes are all `flat` indexes by
default. The sparse type cannot create a flat index. Fix the test code
to create a `SPARSE_INVERTED_INDEX` index for the sparse vector when
is_all_data_type is true
Signed-off-by: elstic <hao.wang@zilliz.com>
2024-07-13 10:45:37 +08:00
cai.zhang
93a500676b
fix: Remove flushed segment in segment manager generated through import ( #34649 )
...
issue: #34648
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-12 23:15:22 +08:00
congqixia
1a248f2668
enhance: Add param item for segmentFlushInterval ( #34629 )
...
See also #28817
Add paramitem for segment flush interval
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-12 18:59:35 +08:00
SimFG
d8e68cbb97
enhance: use the insert rate limit for the upsert request ( #34615 )
...
- issue: #34614
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-12 16:17:35 +08:00
congqixia
531092c031
enhance: Add lint rule to forbid gogo protobuf ( #34594 )
...
github.com/gogo/protobuf is deprecated and could be error prune after
upgrade protobuf message to v2.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-12 10:19:35 +08:00
Chun Han
f00c529aea
feat: support group_size for search_group_by( #33544 ) ( #33720 )
...
related: #33544
mainly changes in three aspects:
1. enable setting group_size for group by function
2. separate normal reduce and group by reduce
3. eleminate uncessary padding in search result for reducing
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-12 10:17:36 +08:00
aoiasd
5bb0d21e32
fix: Restful API use deprecate error code cause access log panic. ( #34576 )
...
relate: https://github.com/milvus-io/milvus/issues/34578
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-12 10:13:35 +08:00
wayblink
358e9a10d2
enhance: Alter compactTo segments before compactFrom to avoid data loss if crash ( #34513 )
...
#34512
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-12 00:55:34 +08:00
jaime
a08a0c831f
fix: encountering orphan channel-cp meta after DataCoord GC ( #34612 )
...
issue: #34545
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-11 23:01:35 +08:00
XuanYang-cn
d7966f46ad
fix: Compaction task l0 state transfer wrong ( #34597 )
...
This bug caused failed L0 compaction tasks never end.
See also: #34460
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-07-11 21:39:36 +08:00
Patrick Weizhi Xu
104d0966b7
feat: support partition key isolation ( #34336 )
...
issue: #34332
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-07-11 19:01:35 +08:00
XuanYang-cn
d7a3697fb5
enhance: Add back compactionTaskNum metrics ( #34583 )
...
Fix L0 compaction task recover unable to set segment not isCompacting
See also: #34460
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-07-11 17:45:37 +08:00
wayblink
fd3da90089
enhance: Refine clustering compaction latency metrics ( #34528 )
...
#34527
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-11 17:43:43 +08:00
smellthemoon
07b94b4615
enhance: support upsert autoid==true ( #30342 )
...
related with: #29258
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-11 16:53:35 +08:00
ThreadDao
e4115485b6
test: add cases for gosdk v2 partitions ( #34593 )
...
issue: #33419 #34426
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-07-11 16:49:36 +08:00
XuanYang-cn
e0b39d8bf4
fix: Milvus panic when compaction disabled and dropping a collection ( #34103 )
...
See also: #31059
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-07-11 14:44:52 +08:00
congqixia
be92147723
enhance: Print reason when seal segments ( #34575 )
...
See also #34574
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-11 11:40:51 +08:00
jaime
3b62138c5c
fix: unstable UT for level0 deletion ( #34524 )
...
issue: #34533
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-11 10:02:56 +08:00
chyezh
1bc3c0b925
enhance: implement balancer at streaming coord ( #34435 )
...
issue: #33285
- add balancer implementation
- add channel count fair balance policy
- add channel assignment discover grpc service
Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-11 09:58:48 +08:00
jaime
c332f69dec
enhance: skip orphan channel cp meta when checking cp lag ( #34555 )
...
issue: # #34545
Print warn log instead of check health fail if orphan channel cp meta is
found in health check request.
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-11 09:36:56 +08:00
yihao.dai
aef7664fc6
fix: Prevent dispatcher merging if curTs is 0 ( #34562 )
...
When the main dispatcher has not yet consumed data, curTs is 0. During
this time, merging dispatchers should not be allowed; otherwise, the
data of the solo dispatcher will be skipped.
issue: https://github.com/milvus-io/milvus/issues/34255
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-11 00:02:50 +08:00
SimFG
b58a5617ef
enhance: add the seal segment when dispatch delete msgs ( #34565 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-10 19:14:51 +08:00
wei liu
ebc68d2774
fix: Indexnode stuck at stopping progress cause by wrong lifetime control ( #34558 )
...
issue: #34557
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-10 15:12:13 +08:00
congqixia
4850336ca3
fix: Write padding at end of mmap file not chunk ( #34529 )
...
Related to #34508
The padding bytes shall be written only at the end of the mmap file not
the chunk of each field data file.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-10 11:12:14 +08:00
wei liu
9b37d3f517
enhance: Enable setting the replica number and resource group during collection creation ( #34403 )
...
issue: #30040
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-10 10:20:13 +08:00
congqixia
d60e628aed
enhance: Avoid use concrete segment type in segments interfaces ( #34521 )
...
See also #34519
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-10 10:18:12 +08:00
wei liu
3c7046353a
fix: RootCoord panic during check health ( #34541 )
...
issue: #34540
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-10 09:51:44 +08:00
sre-ci-robot
2f70bbed88
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-07-10 00:01:19 +00:00
sre-ci-robot
0b3f317b14
[automated] Update Knowhere Commit ( #34549 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-07-10 01:26:14 +08:00
XuanYang-cn
314f4d995b
enhance: Tidy dc channel manager ( #34515 )
...
See also: #34518
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-07-09 18:26:12 +08:00
aoiasd
c1e04534c3
enhance:change access log write cache default config ( #34354 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-09 18:18:12 +08:00
jaime
4365308241
enhance: support setting properties in create database request ( #34510 )
...
issue: #34493
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-09 18:16:18 +08:00
SimFG
5016038781
enhance: release the record in delete codec and add some log for compaction ( #34454 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-09 15:40:17 +08:00
