Zhen Ye
b2eb9fe2a7
fix: memory leak in unittest and open the USE_ASAN option when build unittest ( #35855 )
...
issue: #35854
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-02 15:59:04 +08:00
yihao.dai
6fd33285e1
fix: Fix compile error ( #35901 )
...
/kind improvement
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-02 14:50:35 +08:00
cai.zhang
2c9bb4dfa3
feat: Support stats task to sort segment by PK ( #35054 )
...
issue: #33744
This PR includes the following changes:
1. Added a new task type to the task scheduler in datacoord: stats task,
which sorts segments by primary key.
2. Implemented segment sorting in indexnode.
3. Added a new field `FieldStatsLog` to SegmentInfo to store token index
information.
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-02 14:19:03 +08:00
congqixia
9d80137698
fix: Check clustering key skip load behavior ( #35865 )
...
feature issue: #35415
See also #35861
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-02 11:17:02 +08:00
zhagnlu
576ac2bbed
fix: Fix the reference to a variable after it has been moved ( #35875 )
...
#35607
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-02 10:05:02 +08:00
Jiquan Long
5ea2454fdf
feat: tantivy tokenizer binding ( #35801 )
...
fix : #35800
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-09-01 17:13:03 +08:00
yihao.dai
1413ffe9b1
enhance: Rename preAllocatedSegments ( #35871 )
...
Rename `preAllocatedSegments` to `preAllocatedSegmentIDs` to avoid
confusion.
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-01 17:09:01 +08:00
yihao.dai
2e090b2426
enhance: Refine flusher channel management ( #35870 )
...
Change the ChannelTask to ChannelLifetime, only removing it upon
unregistering.
issue: https://github.com/milvus-io/milvus/issues/33285
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-01 17:07:10 +08:00
yihao.dai
ef451f5e1f
enhance: Add build-go target ( #35844 )
...
Add a `build-go` target to the Makefile that only compiles Go.
issue: https://github.com/milvus-io/milvus/issues/35611
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-01 17:05:09 +08:00
Yinzuo Jiang
407fc933e7
fix: bump bytedance/sonic to v1.12.2 to fix compilation error with go 1.23.0 ( #35879 )
...
fixes : #35878
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-09-01 13:43:01 +08:00
SimFG
311f860676
enhance: support to drop the role which is related the privilege list ( #35727 )
...
- issue: #35545
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-30 15:17:00 +08:00
yellow-shine
1e3e34d149
enhance: [ci] organize values file in the form of milvus deployment options ( #35832 )
...
https://github.com/milvus-io/milvus/issues/35842
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-30 13:07:15 +08:00
yellow-shine
ac736ad472
enhance: friendly for docker cache ( #35789 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-30 13:05:34 +08:00
XuanYang-cn
323400c190
enhance: Enable to write multiple segments in mix compactor ( #35705 )
...
Prevent segments to be written larger than maxSize * expansionRate
See also: #35584
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-30 11:29:01 +08:00
yihao.dai
1c3abd1aea
fix: Fix unstable TestFlusher ut ( #35831 )
...
issue: https://github.com/milvus-io/milvus/issues/35830
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-30 10:25:01 +08:00
Zhen Ye
f9be01ce89
fix: mockery config ( #35834 )
...
issue: #33285
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-30 09:49:00 +08:00
SimFG
07bb071aaf
enhance: add the unmarshal method for the rbac msg ( #35692 )
...
/kind improvement
- issue: #35391
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-30 07:23:00 +08:00
zhagnlu
671112d17b
enhance: add more info to hybrid index log ( #35808 )
...
#32900
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-29 21:07:04 +08:00
smellthemoon
a3f2f044d6
fix: not set nullable when stream writer write headers ( #35799 )
...
#35802
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-29 20:59:00 +08:00
zhenshan.cao
d10aa4626f
enhance: [skip e2e] add make run-test-cpp with support for filter gtest ( #35829 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-08-29 20:03:02 +08:00
congqixia
4373c6994f
fix: [GoSDK] reset cache after UsingDatabase ( #35638 )
...
Related to milvus-io/milvus-sdk-go#809
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 19:41:02 +08:00
yihao.dai
69265978bf
fix: Fix arrow go client ( #35819 )
...
issue: https://github.com/milvus-io/milvus/issues/35662
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-29 17:53:01 +08:00
congqixia
3c8941a88e
enhance: Print log only when rate limit updates ( #35806 )
...
The debug log for "RateLimiter register for rateType" is too frequent
and in e2e cases, the may print 18M times in one run.
This PR make the log be printed only when rate limit is updated.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 15:07:00 +08:00
congqixia
09ef3f1b4f
fix: Make sure querycoord observers started once ( #35811 )
...
Related to #35809
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 14:45:00 +08:00
Zhen Ye
70bc0e4538
enhance: add skip 3rdparty env var ( #35736 )
...
issue: #35611
- call `conan install` is too slow for developer, add option to skip it.
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-29 14:21:01 +08:00
Patrick Weizhi Xu
b3089b5bdc
feat: support range search pagination retains order ( #35738 )
...
issue: #35464
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-08-29 14:09:00 +08:00
wei liu
73be0ba941
enhance: avoid the coexistence of the old coordinator and the new node/proxy ( #35720 )
...
issue: #35719
In standalone mode, block the start process until the new coordinator is
active to avoid the coexistence of the old coordinator and the new
node/proxy
1. In the start/restart process, the new coordinator will become active
immediately and will not be blocked
2. In the rolling upgrade process, the new coordinator will not be
active until the old coordinator is down, and it will be blocked
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-29 13:55:01 +08:00
zhuwenxing
e34b0c43a1
enhance: add GitHub Actions to delete spam issues comments ( #35807 )
...
/kind improvement
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-08-29 13:19:00 +08:00
jaime
b0ac04d104
enhance: set database properties to restrict read access ( #35745 )
...
issue: #35744
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-29 13:17:01 +08:00
smellthemoon
b51b4a2838
fix: try get not exist file after upgrade ( #35740 )
...
https://github.com/milvus-io/milvus/issues/35741
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-29 11:09:01 +08:00
congqixia
86691656f3
enhance: Change frequent balancer debug log to rated one ( #35749 )
...
"skip balance" log is too frequent in debug level. This PR changes it
into rated on.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 10:07:00 +08:00
congqixia
985d84d3ee
enhance: Add skip load validation for create collection task ( #35737 )
...
Related to #35415
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 10:05:08 +08:00
Zhen Ye
99dff06391
enhance: using streaming service in insert/upsert/flush/delete/querynode ( #35406 )
...
issue: #33285
- using streaming service in insert/upsert/flush/delete/querynode
- fixup flusher bugs and refactor the flush operation
- enable streaming service for dml and ddl
- pass the e2e when enabling streaming service
- pass the integration tst when enabling streaming service
---------
Signed-off-by: chyezh <chyezh@outlook.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-29 10:03:08 +08:00
yellow-shine
ae53ce381b
delete pvc when finished ( #35783 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-29 09:57:06 +08:00
congqixia
c5c9ef9876
fix: Check response size before add to counter ( #35778 )
...
Related to #35767
prometheus counter cannot add negative value
when response is not written(say timeout/network broken) panicking may
happen if not check
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 00:05:00 +08:00
sre-ci-robot
1cd9123a6c
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-08-28 12:00:44 +00:00
Zhen Ye
9b96841ae9
fix: wrong construction in evalctx ( #35772 )
...
issue: #35771
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-28 19:21:00 +08:00
congqixia
91223deb91
fix: Use SafeChan as TxnManager.closed ( #35763 )
...
Resolves : #35762
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-28 19:06:59 +08:00
congqixia
b75b25794b
enhance: [skip e2e] Fix test script always skip pip install ( #35766 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-28 17:01:00 +08:00
yellow-shine
709666b88f
enhance: fix new nightly pipeline ( #35747 )
...
https://github.com/milvus-io/milvus/issues/35704
---------
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-28 16:56:59 +08:00
yihao.dai
56b3490a98
fix: Fix rate wasn't limited to the expected value ( #35699 )
...
Each time the rate is reset, the token bucket is fully refilled, causing
the rate wasn't limited to the expected value. This PR addresses the
issue by preventing the token reset.
issue: https://github.com/milvus-io/milvus/issues/35675 ,
https://github.com/milvus-io/milvus/issues/35702
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-28 14:13:00 +08:00
Jiquan Long
a52ba3d09d
enhance: allow many segments for inverted index ( #35616 )
...
fix: https://github.com/milvus-io/milvus/issues/35615
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-28 11:30:59 +08:00
jaime
d8aa01bc1a
fix: inconsistent meta view causes rate limit invalid ( #35665 )
...
issue: #35663
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-28 11:21:04 +08:00
jaime
b7ea1defd3
fix: mistaken deletions may occur during GC channel checkpoints ( #35707 )
...
issue: #35706
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-28 10:11:05 +08:00
yellow-shine
7de210ff28
enhance: nightly pipeline ( #35730 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-27 17:51:05 +08:00
Zhen Ye
98866205fa
fix: munmap deallocate too much memory ( #35725 )
...
issue: #35693
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-27 17:18:59 +08:00
yihao.dai
9868fe4e6c
fix: Fix panic due to empty candidate import segments ( #35673 )
...
issue: https://github.com/milvus-io/milvus/issues/35662
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-27 17:08:59 +08:00
zhagnlu
4d2f96c760
enhance: support bitmap mmap ( #35399 )
...
#32900
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-27 16:34:59 +08:00
yellow-shine
3a0c61a455
fix: [skip e2e]change conflict key ( #35733 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-27 15:40:59 +08:00
yellow-shine
bb9bed64a3
enhance: new nightly ci to support testing distributed-streaming ( #35672 )
...
issue: https://github.com/milvus-io/milvus/issues/35704
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-27 14:24:59 +08:00
