Commit Graph

20684 Commits

Author SHA1 Message Date
chyezh
16b0aee97f
enhance: timetick interceptor optimization (#35287)
issue: #33285

- remove redundant goroutine by using insepctor.
- remove the coutinous non-message timetick persistence
- periodically push the time tick forward without persistent timetick
message.
- add 'message type filter' deliver filter.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-12 18:58:25 +08:00
XuanYang-cn
06f9ba294e
enhance: Add more ut for l0CompactionTask (#35100)
See also: #34796

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-09 14:24:18 +08:00
smellthemoon
fa41c7960e
fix:use echo instead of print in Makefile (#35373)
#35372

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-09 14:16:17 +08:00
smellthemoon
c4a240acb1
enhance: memory value are displayed in MB (#35292)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-09 14:14:25 +08:00
Patrick Weizhi Xu
e1d725b938
fix: rollback grpc to 1.50.1 (#35385)
issue: #35298

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-08-09 14:08:17 +08:00
XuanYang-cn
61a426bc6f
enhance: Init channel cp metric when DataCoord recover (#35353)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-09 10:08:18 +08:00
congqixia
c0ee25afd8
fix: Use k locations only for basic BF test location (#35380)
Related to #35379

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-09 07:52:22 +08:00
congqixia
143502e708
enhance: Fix progress always zero due to integer divide (#35374)
See also #35183

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-08 22:26:16 +08:00
cai.zhang
aaab827a16
fix: Fix the issue of missing stats log after clustering compaction (#35266)
issue: #35265

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-08 14:24:17 +08:00
zhagnlu
626b1b2f5e
fix:redefine hybrid internal index type (#35314)
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-08 10:32:16 +08:00
yellow-shine
7420115b5e
enhance: pinpoint revision of 3thirdparty package (#35325)
issue: https://github.com/milvus-io/milvus/issues/35340

---------

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-08 09:46:22 +08:00
Bingyi Sun
ae1b81ac1a
fix: fix panic when generating plans (#35309)
issue: https://github.com/milvus-io/milvus/issues/35335

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-08-07 18:14:16 +08:00
yihao.dai
72a175478f
enhance: Append drop partition msg to wal (#35326)
issue: https://github.com/milvus-io/milvus/issues/33285

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-07 17:28:16 +08:00
wei liu
838f06323f
fix: datacoord binlog size metrics include dropped segment (#35338)
issue: #35337

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-07 16:24:16 +08:00
wayblink
c6253f9c9b
enhance: improve ut cov of clustering compaction task (#35242)
#34792

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-08-07 14:44:21 +08:00
wayblink
bb15ecdc13
enhance: improve ut for compaction_policy_clustering (#35205)
#34792

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-08-07 14:42:17 +08:00
congqixia
634eadd0a1
enhance: Remove duplicated generate-mockery-flushcommon target (#35331)
Makefile contains two `generate-mockery-flushcommon` targets and cause
unwanted warning during building. This PR removes one with less commands
to fix the problem.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-07 14:32:22 +08:00
Patrick Weizhi Xu
1d5bf34e38
fix: init telemetry fails when set otlp as trace exporter (#35301)
issue: #35298

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-08-07 14:20:21 +08:00
wei liu
344dc6a9f8
enhance: enable to set load config in cluster level (#35169)
issue: #35170
This PR enable to set load configs in cluster level, such as replicas
and resource groups. then when load collections will use the load
config.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-07 12:38:21 +08:00
chyezh
c725416288
enhance: move streaming proto into pkg (#35284)
issue: #33285

- move streaming related proto into pkg.
- add v2 message type and change flush message into v2 message.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-07 10:34:16 +08:00
Jiquan Long
b5022b0647
enhance: add metric to record disk quota (#35305)
https://github.com/milvus-io/milvus/issues/35177

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-07 10:26:16 +08:00
Jiquan Long
976ceb4a46
fix: comparision operations between incompatible operands (#35264)
fix: #34139

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-07 10:24:27 +08:00
Xiaofan
885d891106
Change minio use virtual config to not panic (#35327)
minio use virtual should not panic if empty. User may use other object
storage instead

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-08-07 09:58:15 +08:00
Jiquan Long
f13b21c041
fix: error message when input is not aligned (#35322)
fix: https://github.com/milvus-io/milvus/issues/35321

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-07 09:56:28 +08:00
zhagnlu
c19fe95154
fix: support string match for hybrid and bitmap index (#35294)
#34841

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-07 09:54:22 +08:00
sre-ci-robot
e75efa7359
[automated] Update Knowhere Commit (#35328)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-07 01:00:16 +08:00
congqixia
6ff238e88a
fix: Set corresponding DataScope for loadStreamDelete (#35312)
Related to #35311

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-06 22:32:23 +08:00
sre-ci-robot
e937197270
[automated] Update gpu Builder image changes (#35300)
Update gpu Builder image changes
See changes:
d8668fe405
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-06 17:38:16 +08:00
Cai Yudong
3c9a47c8db
feat: Encode traceID and spanID as hex string (#34807)
Issue: https://github.com/zilliztech/knowhere/pull/714

Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-08-06 15:20:16 +08:00
Jiquan Long
91df03afe8
feat: put inverted index into ram (#35222)
fix: https://github.com/milvus-io/milvus/issues/35224

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-06 11:54:16 +08:00
yihao.dai
678018d9ca
enhance: Avoid unnecessary compaction (#35148)
Estimate the import segment size based on DiskSegmentMaxSize(2G) to
avoid unnecessary compaction after import completed.

issue: https://github.com/milvus-io/milvus/issues/35147

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-06 10:30:21 +08:00
presburger
a9352a073b
enhance: change gpu default mem pool size (#34159)
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2024-08-06 10:24:15 +08:00
sre-ci-robot
dea73ef288
[automated] Update cpu Builder image changes (#35283)
Update cpu Builder image changes
See changes:
d8668fe405
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-06 10:16:16 +08:00
chyezh
14051fed7d
enhance: streaming service client (#34656)
issue: #33285

- implement streaming service client.
- implement producing and consuming service client by streaming coord
client and streaming node client.

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-05 21:38:15 +08:00
congqixia
a2b517523d
enhance: Add in-memory cache for casbin enforcer result (#35271)
See also #35270

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-05 18:48:15 +08:00
ThreadDao
a62118ca33
test: update cases for index describe and upsert with autoID (#35191)
issue: #33419

---------

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-08-05 18:00:16 +08:00
shaoyue
1db099973f
enhance: Gin & restfulv1 handler use sonic json (#35020)
/cc @czs007 @congqixia 
/kind enhancement

Signed-off-by: haorenfsa <haorenfsa@gmail.com>
2024-08-05 16:44:17 +08:00
yihao.dai
b71e058bc5
enhance: Add import option to skip disk quota check (#35274)
Add an option to skip the disk quota check for backup-restore import.

issue: https://github.com/milvus-io/milvus/issues/33775

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-05 16:40:16 +08:00
cai.zhang
6542c1ab0e
enhance: Add monitoring metrics for task execution time in datacoord (#35139)
issue: #35138

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-05 16:26:17 +08:00
cai.zhang
d905713724
fix: Maintain load idempotency even when building new indexes (#35178)
issue: #34404

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-05 16:22:17 +08:00
sthuang
d8668fe405
enhance: upgrade go version to 1.21.11 (#35257)
issue: #35142

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-08-05 16:12:15 +08:00
yihao.dai
678c0a708b
enhance: Refine import error log (#35067)
issue: https://github.com/milvus-io/milvus/issues/35060

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-05 15:30:16 +08:00
congqixia
a6903701a4
enhance: Remove stackerror imported unexpected (#35256)
`stackerror` pkg is imported by accident and could be replaced by
cockroachdb errors lib. This PR removes import line and fix this
problem.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-05 14:48:16 +08:00
wei liu
c22b0a636e
fix: delete by expr failed at retry progress (#35241)
issue: #35240
delete by expr shard the same err object between channels, so if one
channel's query failed, it will fail all channel, which will break
channel level retry policy, and make delete operation failed.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-05 11:18:16 +08:00
zhagnlu
4b553b0333
enhance: revert remove duplicated pk function (#35103)
issue: #34778
 Revert "fix: fix query count(*) concurrently"
 Revert "enhance: mark duplicated pk as deleted "

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-05 10:48:17 +08:00
sre-ci-robot
e6640346d0
[automated] Update gpu Builder image changes (#35246)
Update gpu Builder image changes
See changes:
7bf68d8657
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-05 09:26:15 +08:00
sre-ci-robot
eb5461905b
[automated] Update Pytest image changes (#35244)
Update Pytest image changes
See changes:
241c71fdde
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-05 09:24:33 +08:00
sre-ci-robot
e201110fbd
[automated] Update cpu Builder image changes (#35248)
Update cpu Builder image changes
See changes:
7bf68d8657
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-05 09:22:21 +08:00
chyezh
9871966415
enhance: segment alloc interceptor (#34996)
#33285

- add segment alloc interceptor for streamingnode.
- add add manual alloc segment rpc for datacoord.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-04 07:40:15 +08:00
sre-ci-robot
a7f7d911cf Update all contributors
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-08-02 12:00:48 +00:00