yihao.dai
b6612e02b4
enhance: Reduce GetIndexInfos calls ( #37695 )
...
Batch `GetIndexInfos` calls for segments to reduce RPC calls.
issue: https://github.com/milvus-io/milvus/issues/37634
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-19 14:24:31 +08:00
jaime
1d06d4324b
fix: Int64 overflow in JSON encoding ( #37657 )
...
issue: ##36621
- For simple types in a struct, add "string" to the JSON tag for
automatic string conversion during JSON encoding.
- For complex types in a struct, replace "int64" with "string."
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-14 22:52:30 +08:00
foxspy
d7b2ffe5aa
enhance: add an unify vector index config checker ( #36844 )
...
issue: #34298
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-28 10:11:37 +08:00
foxspy
3de57ec4fa
enhance: add vector index mgr to remove vector index type dependency ( #36843 )
...
issue: #34298
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-17 22:15:25 +08:00
cai.zhang
8395c8a8db
enhance: Update stats task to optional ( #35947 )
...
issue: #33744
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-12 20:37:08 +08:00
zhagnlu
208c8a2328
fix:support config index offsetcache and fix create same index again ( #35985 )
...
#35971
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-08 18:23:05 +08:00
cai.zhang
2c9bb4dfa3
feat: Support stats task to sort segment by PK ( #35054 )
...
issue: #33744
This PR includes the following changes:
1. Added a new task type to the task scheduler in datacoord: stats task,
which sorts segments by primary key.
2. Implemented segment sorting in indexnode.
3. Added a new field `FieldStatsLog` to SegmentInfo to store token index
information.
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-02 14:19:03 +08:00
zhagnlu
42f7800b5b
enhance: add bitmap offset cache to speed up retrieve raw data ( #35498 )
...
#35458
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-24 01:40:58 +08:00
congqixia
c992a61a23
enhance: Separate allocator pkg in datacoord ( #35622 )
...
Related to #28861
Move allocator interface and implementation into separate package. Also
update some unittest logic.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 10:06:56 +08:00
SimFG
731d45abbe
enhance: provide more general configuration to control mmap behavior ( #35359 )
...
- issue: #35273
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-21 00:22:54 +08:00
cai.zhang
feb13cdf07
enhance: Skip pick worker when task doesn't need to execute actually ( #34348 )
...
issue: #34347
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-03 15:52:09 +08:00
cai.zhang
27cc9f2630
enhance: Support analyze data ( #33651 )
...
issue: #30633
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
2024-06-06 17:37:51 +08:00
congqixia
54797b4286
enhance: Refine frequent log in datacoord ( #33449 )
...
This PR changes:
- Frequent `ListIndexes` success log to debug level
- Aggregate collection missing log after collection dropped in
`meta.GetCollectionIndexFilesSize`
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-29 19:15:43 +08:00
cai.zhang
ed39a38953
enhance: Reduce the frequency of logs describing indexing failures ( #33212 )
...
issue: #33001 #33102
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-05-21 19:27:39 +08:00
cai.zhang
4382cf5283
enhance: Reduce the frequency of index ralted logs ( #33050 )
...
issue: #33001
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-05-15 14:37:34 +08:00
congqixia
cedb33ceec
enhance: Improve datacoord segment filtering with collection ( #32831 )
...
See also #32165
This PR modify the `SelectSegments` interface to utilizing collection id
information when selecting segment with provided collection
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-08 21:37:29 +08:00
congqixia
6af5275066
fix: Refine index param check for Create/Alter Index ( #32712 )
...
See also #32711
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-06 15:43:29 +08:00
Bingyi Sun
bdc70dfc6a
feat: Add global mmap enable configuration ( #31267 )
...
https://github.com/milvus-io/milvus/issues/31279
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-18 15:17:10 +08:00
cai.zhang
4871786a7b
enhance: When describing an index, fetch the index info in batches ( #31238 )
...
issue: #29313
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-15 16:37:09 +08:00
Bingyi Sun
5c0bb40549
fix: merge index params when creating index ( #31127 )
...
issue: https://github.com/milvus-io/milvus/issues/31102
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-11 17:31:03 +08:00
Bingyi Sun
425da78b38
fix: alter index request's index name can not be empty ( #31128 )
...
issue: https://github.com/milvus-io/milvus/issues/31138
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-11 13:05:02 +08:00
congqixia
d81ba164c8
enhance: Add ListIndexes API from datacoord ( #31104 )
...
See also #31103
This PR add `listIndexes` API for datacoor server to list all indexes
for provided collection.
Comparing to the existing `DescribeIndex` API, the new one does NOT
check the segment index building progress to ease the burden when
invoking it
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-07 17:37:01 +08:00
jaime
4b0c3dd377
enhance: index meta use independent rather than global meta lock ( #30869 )
...
issue: https://github.com/milvus-io/milvus/issues/30837
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-03-04 16:56:59 +08:00
cai.zhang
f6ff2588cd
enhance: Optimize DescribeIndex to reduce lock contention ( #30939 )
...
issue: #29313
issue: #30443
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-03-03 19:00:59 +08:00
Bingyi Sun
816ed671aa
fix: alter_index should return error if index not found ( #30786 )
...
issue: https://github.com/milvus-io/milvus/issues/30932
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-02-29 15:55:01 +08:00
Patrick Weizhi Xu
0907d76253
enhance: pass partition key scalar info if enabled when build vector index ( #29931 )
...
issue: #29892
Pass optional scalar IVF offsets to Cardinal
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-01-24 00:04:55 +08:00
yah01
1c8ce33eea
fix: report error if the altering index doesn't support mmap ( #29832 )
...
this also checks the param value
fix #29909
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2024-01-17 16:40:54 +08:00
xige-16
02673914a0
feat: Support multiple vector indexes in a collection ( #27700 )
...
issue: #25639
/kind improvement
Signed-off-by: xige-16 <xi.ge@zilliz.com>
---------
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-12-29 11:44:45 +08:00
yah01
a0e1a1eb31
feat: support enable/disable mmap for index ( #29005 )
...
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 18:07:24 +08:00
cai.zhang
1b4d2674b3
fix: Set the default index name to the name of the existing index ( #29275 )
...
issue: #29269
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-12-20 17:22:41 +08:00
wei liu
5b45a138b1
disable auto balance when old node exists ( #28191 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-07 14:02:20 +08:00
yah01
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
yah01
41495ed266
Improve the error message for getting all indexes of collection ( #27389 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-08 21:23:32 +08:00
Jiquan Long
370fdaf50d
Record engine version for segment index ( #27384 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-09-28 18:03:28 +08:00
cai.zhang
dedb90f85f
Fix error message for creating scalar index ( #27382 )
...
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-09-27 10:33:27 +08:00
yah01
6539a5ae2c
Refine DataCoord status ( #27262 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
foxspy
5db4a0489e
dynamic index version control ( #27335 )
...
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-25 21:39:27 +08:00
foxspy
370b6fde58
milvus support multi index engine ( #27178 )
...
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-22 09:59:26 +08:00
cai.zhang
22d7fa4e1c
Use GetXXX instead of directly using dots to get value ( #27045 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-15 10:07:19 +08:00
SimFG
0901b76732
Avoid the panic when the status of rpc response is nil ( #26910 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-07 19:23:15 +08:00
cai.zhang
d7cd1f2a6d
Fix bug for get index state with compaction segment ( #26822 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-04 17:29:48 +08:00
yah01
3349db4aa7
Refine errors to remove changes breaking design ( #26521 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-04 09:57:09 +08:00
cai.zhang
6e1f284485
Refine index log ( #26191 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-08-11 11:14:39 +08:00
cai.zhang
eaf53c7b39
Refine describe index that doesn't wait compacted segment indexed ( #25899 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-28 11:15:02 +08:00
cai.zhang
760a2d9aa7
Support AllocTimestamp api for Milvus ( #25784 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-07-25 10:05:00 +08:00
Enwei Jiao
66fdc71479
Refactor logs in DataCoord & DataNode ( #25574 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-14 15:56:31 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp ( #22984 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
cai.zhang
93ea9c4925
Support return pending index rows when describe index ( #24588 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-06-01 18:14:32 +08:00
wayblink
899702f13c
Implement GetIndexStatistics interface ( #23603 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-05-06 10:34:39 +08:00
