Commit Graph

232 Commits

Author SHA1 Message Date
zhenshan.cao
c15b880f0e
Remove redundant logs (#19412)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-09-26 15:40:53 +08:00
cai.zhang
9e18645363
Optimize IndexCoord scheduler loop (#19375)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-24 13:40:52 +08:00
Xiaofan
928a213e31
Fix Chunk Manager Path mis use (#19141)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-09-23 14:40:51 +08:00
cai.zhang
1c489a85b2
Add ut for IndexCoord (#19349)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-23 09:58:51 +08:00
cai.zhang
17f5e3c7fe
Drop segment indexes when drop partition (#19022)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-23 09:36:51 +08:00
xige-16
428840178c
Support diskann index for vector field (#19093)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-21 20:16:51 +08:00
cai.zhang
fc3309b8b8
All indexes segments both write handoff (#19230)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-21 16:34:51 +08:00
SimFG
d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
cai.zhang
525201decc
Mark index meta as deleted when segment is compacted (#19163)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-15 15:44:31 +08:00
cai.zhang
e65ba6331e
Only store segmentID for flushed segments event (#19075)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-09 15:52:35 +08:00
cai.zhang
d6b0fdc944
Fill index size and fail reason when query index job (#19026)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-06 17:19:11 +08:00
cai.zhang
11b4f4e901
Fix typo for index code (#18971)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-05 17:37:11 +08:00
cai.zhang
6d37b1db17
Fix bug for umarshal index model (#18959)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-01 17:18:57 +08:00
yah01
03a59ccbd4
Fix IndexCoord forget to fill DmChannel (#18914)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-08-31 11:28:58 +08:00
cai.zhang
34c9eae16f
Add IndexCoord ut and fix data race (#18897)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-08-31 10:23:04 +08:00
congqixia
c91bb0b015
Unify rootPath in configs and ChunkManager (#18808)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-25 19:32:53 +08:00
cai.zhang
c924f73105
Refactor for IndexCoord to support cloud (#18643)
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
aoiasd
8a70f2badb
Add ShowConfigurations for all remaining session (#18594)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-12 13:20:39 +08:00
jaime
0e95f8a690
Refine indexcoord logs (#18589)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-08-11 10:00:38 +08:00
xing.zhao
d90313a1a8
Use chunkManager rather than minio.Client in datanode garbage collection (#18092)
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: xingzhao <xing.zhao@zilliz.com>
2022-07-22 22:10:28 +08:00
cai.zhang
315df1ff15
Reduce the timer time of assigning index task scheduler (#18319)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-19 14:24:29 +08:00
cai.zhang
a47a414dce
Fix bug for IndexNode pod kill test (#18177)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-13 16:16:25 +08:00
cai.zhang
90faf4a212
Fix bug for IndexCoord recycles index files (#18168)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-08 16:54:22 +08:00
cai.zhang
0c883b215b
Fix bug for IndexCoord ut (#18148)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-07 18:52:21 +08:00
cai.zhang
58a9a0b79a
Add license for new files (#18138)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-07 17:44:21 +08:00
cai.zhang
c1a47c8941
Use struct channel instead of bool chan (#18140)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-07 16:20:22 +08:00
cai.zhang
0ad15a22c8
IndexCoord handle events correctly (#17878)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-07 14:44:21 +08:00
Xiaofan
a9b1d71a8f
change the etcd compareAndSwap interface (#18068)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-07-06 13:54:21 +08:00
cai.zhang
01fc411566
Handle etcd compacted error (#17886)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-30 18:54:19 +08:00
cai.zhang
899a9bf86d
IndexNode still writes finished state to ETCD when task was deleted (#17761)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-28 12:08:17 +08:00
cai.zhang
ef0fc37ad7
Fixed the bug that IndexCoord lost some meta information (#17818)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-27 21:52:17 +08:00
cai.zhang
e8f53af749
When the task is canceled, reduce the load of node (#17735)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-23 19:22:15 +08:00
cai.zhang
1fbdafc943
IndexCoord does not assign task to IndexNode prematurely (#17717)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-23 10:40:13 +08:00
cai.zhang
141a3bbe00
Mark deleted before drop index (#17490)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-17 18:08:12 +08:00
cai.zhang
ea5041aec2
Acquiring the segment reference lock on task level (#17544)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-15 21:38:10 +08:00
Xiaofan
af994b5e1c
Fix etcd watch error (#17364)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-06 16:26:06 +08:00
cai.zhang
bcf3b7426a
Add distributed lock for segment refgerence (#16782)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
cai.zhang
f1bad98f00
Fix indexcoord ut (#17282)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 12:00:09 +08:00
cai.zhang
b37b87eb97
The GetIndexFilePaths interface judges whether the index is complete (#17052)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-17 20:41:56 +08:00
cai.zhang
b0c9f25352
Sort out the log of IndexCoord (#16810)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-09 12:05:52 +08:00
cai.zhang
2be46a01c0
Reduce the number of retries and add error log (#16754)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-05-05 09:31:51 +08:00
cai.zhang
6440c0dd98
Add log for getting IndexNode's mertrics (#16747)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-29 19:07:47 +08:00
godchen
bb7a0766fe
Add dependency factory (#16204)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-07 22:05:32 +08:00
cai.zhang
4267dc2efc
Handle error and oragnize log (#16367)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 14:33:30 +08:00
cai.zhang
f9c6b3980f
Compressed log (#16354)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-04-06 10:11:36 +08:00
cai.zhang
e2e345079f
Add indexMeta version for IndexCoord log (#16266)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-29 21:03:27 +08:00
Jiquan Long
3219b4e392
Skip dimension check for scalar index (#16257)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-29 15:35:27 +08:00
cai.zhang
c8688a6275
Organize log for GetIndexFilePaths (#16230)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-28 21:27:27 +08:00
Jiquan Long
f8d9bc919d
Unify interface of vector index & scalar index. (#15959)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-21 14:23:24 +08:00
godchen
78557ca6d7
Replace minio kv with minio chunk manager (#15936)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-17 18:03:23 +08:00