Commit Graph

184 Commits

Author SHA1 Message Date
bigsheeper
01e9dc8e3f Remove collection name
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 11:52:19 +08:00
sunby
f3aad3a71c Change SegmentInfo
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-02 18:53:10 +08:00
bigsheeper
5e781b9370 Remove field name in query node and segCore
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 10:10:07 +08:00
cai.zhang
f940cc455a Add dockerfile for index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-26 09:38:40 +08:00
neza2017
7f3aa92d10 Add docker file for masterservice
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-25 15:58:04 +08:00
FluorineDog
a8fd6e7686 Enable drop index/field_data and extract info visitor
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-25 18:59:44 +08:00
BossZou
01267776d2 Fix rocksmq consume invalid currendID
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
2021-01-23 10:15:57 +08:00
XuanYang-cn
50e2369000 Change name and tag to ID
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-22 19:36:09 +08:00
FluorineDog
bff208d78c Add sealedSegment (go&c) unittest, fix growingSegment field id check
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-22 17:40:58 +08:00
bigsheeper
067c30c422 Add sealedSegment cgo unittest, fix growingSegment field id check
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-22 15:41:54 +08:00
dragondriver
eba6bb288c Refactor the sructure of proto file
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-22 09:36:18 +08:00
FluorineDog
7f044fff82 Enable search for sealed
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-21 15:29:52 +08:00
cai.zhang
83c60fd84f Update proto about index
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-21 15:20:23 +08:00
FluorineDog
5e06dc1732 Enable segment sealed, add support for loadXXX
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-20 17:33:31 +08:00
cai.zhang
c35079d7e7 Update registerNode in indexservice
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-20 10:15:43 +08:00
zhenshan.cao
2ec573d2f9 Recover delete docs
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-20 09:36:50 +08:00
cai.zhang
fb493cc235 Update interface name of GetIndexStates
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-19 18:32:57 +08:00
neza2017
918e279869 Set msg stream
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 16:14:13 +08:00
neza2017
d5e53aa28a Add Meta table implementation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 14:13:49 +08:00
bigsheeper
2064b014a1 Add loadFieldData, use partition id instead of tag
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-19 14:44:03 +08:00
XuanYang-cn
6a13386393 Add DataNode package
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-19 11:37:16 +08:00
neza2017
1ce32d8775 Add master service
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-19 09:45:42 +08:00
bigsheeper
aaf8b0ad40 Add sealed segment, segment type and manager, send loadIndexReq to service's buffer
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-19 10:29:16 +08:00
FluorineDog
9ff1bc0f99 Adapt Expr Executor to SegmentInternalInterface, fix row_count consistency
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-16 18:12:58 +08:00
XuanYang-cn
37688559f1 Fix multi rowdata writenode crash bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-16 15:06:19 +08:00
bigsheeper
46a14da3eb Implement segment manager and loadSegment
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-16 15:04:32 +08:00
FluorineDog
16ff16b212 Renaming and add load index api beta
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-16 10:48:01 +08:00
quicksilver
7fe061b846 Raneme init_devcontainer.sh to devcontainer.sh
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-15 14:38:36 +08:00
neza2017
56ab2bda2d Update proto:add partition id
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-15 11:53:22 +08:00
FluorineDog
8fccf7e630 Use bitset_view at search
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-15 13:40:51 +08:00
FluorineDog
7d81222550 Support span in SegmentGrowing, refine vector_trait
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-15 10:32:50 +08:00
FluorineDog
63ca5f8031 Add span and segment interface, rename segment_growing and use interface as base
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-13 18:46:25 +08:00
GuoRentong
1104f059ee Update doc:module interfaces
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-13 11:08:03 +08:00
quicksilver
8e9d8e36e1 Update run_go_unittest.sh
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-13 10:40:46 +08:00
FluorineDog
4cd42c553f Rename field_name, make field_id strongly typed, skip multithread test
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-12 18:31:52 +08:00
sunby
3ae04794f6 Fix bug in describeIndexProgressTask
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-07 15:39:20 +08:00
FluorineDog
34a00aacfa Enable binary index in segcore
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-07 15:00:42 +08:00
dragondriver
ced56c84f8 Add support to build binary vector index
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-07 14:56:17 +08:00
godchen
8f1f118b26 Add query opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-07 14:24:35 +08:00
FluorineDog
3bf205d9a8 Fix inner product
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-07 09:32:17 +08:00
xige-16
8715cd1f0c Fix key error when loading index
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-06 18:19:44 +08:00
bigsheeper
a1e2ff3c26 Add gcNode to write node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-06 16:44:12 +08:00
neza2017
4015d7245d Merge operation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-06 14:45:50 +08:00
cai.zhang
1cfc2ff0a5 Add make clean in Makefile
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-06 13:50:48 +08:00
sunby
da6eeddbf8 Add flush timestamp
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-06 13:14:38 +08:00
FluorineDog
5a26f6ef21 Enable sub_query_result
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-06 12:01:13 +08:00
godchen
1c87faf8b3 Add insert request opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-06 11:17:35 +08:00
FluorineDog
e16a7478da Add verifier
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-05 20:13:54 +08:00
dragondriver
bccf8d2098 Add query support for ivf-flat, nsg
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-05 19:20:55 +08:00
bigsheeper
1ba47ac433 Check system field in segCore
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-04 10:13:01 +08:00
quicksilver
1927eaf88b Add revive linter
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-12-31 16:38:37 +08:00
zhenshan.cao
59878b1dbb Add lazy connect logic for indexbuilder
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-31 15:44:06 +08:00
dragondriver
6cd8c2528d Add parameter check for almost all index type
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-31 15:16:03 +08:00
xige-16
29b21b4bf2 Refactor reduce strategy in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-31 14:49:27 +08:00
dragondriver
2e198e2783 Add support for idmap, nsg, sq8
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-30 19:37:45 +08:00
zhenshan.cao
adf77c423c Update knowhere
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-30 18:30:51 +08:00
bigsheeper
4835d82060 Add index builder test cases
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-30 18:16:34 +08:00
dragondriver
243346247b Add unittest case which used in regression to index builder
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-29 16:31:03 +08:00
xige-16
be32a33eff Refactor loadIndexTest to test diffrent index params
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-29 14:43:40 +08:00
cai.zhang
f80536ed77 Update the logic of running go unittest
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-29 11:56:37 +08:00
XuanYang-cn
9577da9fce Add writenode main
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-28 18:52:34 +08:00
FluorineDog
eb8717188e Enable load index
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-28 17:55:48 +08:00
dragondriver
f67aae9596 Fix the codec bug of cgo wrapper of index builder
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-28 16:55:50 +08:00
dragondriver
11cef6e978 Add support to build binary index
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-28 10:00:02 +08:00
xige-16
d599407e2b Add unittest for loadIndex service
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-26 14:16:51 +08:00
dragondriver
84a0cf7097 Add unittest to c index wrapper
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-12-25 11:10:31 +08:00
XuanYang-cn
c0797333ce Add ddl binlog metaTable in writenode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017
4dbf4ade41 Add test for master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00
xige-16
f5198bcaee Add LoadIndexService in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-24 14:51:55 +08:00
FluorineDog
342f4cb741 Support dynamic chunk size
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-23 19:02:37 +08:00
FluorineDog
d5daa18392 Enable most DSL-related cases
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-21 14:45:00 +08:00
sunby
c71cd40f68 Refactor segment manager
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-19 12:55:24 +08:00
FluorineDog
e65cfe1e3d Enable complex dsl parser
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-19 10:36:49 +08:00
zhenshan.cao
0d75840ed6 Fix missing file
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-18 18:35:03 +08:00
FluorineDog
1540cffbb1 Add executor for expression
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-12 18:17:53 +08:00
FluorineDog
02f855359a Fix term bug and add todo note
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-10 20:13:37 +08:00
bigsheeper
762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-10 16:31:09 +08:00
sunby
ed54d3e26c Fix binlog reader bug
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-09 09:55:56 +08:00
quicksilver
d09ad77fb2 Add archiving file workflow for cache thirdparty
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-12-08 19:53:22 +08:00
sunby
8ace1c3837 Add binlog serialization and deserialization
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-08 19:13:39 +08:00
XuanYang-cn
e6f726e73a Add cache for thirdparty files cache
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-08 18:51:07 +08:00
neza2017
70710dee47 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-05 16:11:03 +08:00
FluorineDog
63c8f60c6e Enable term parser and executor
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-05 09:51:27 +08:00
FluorineDog
6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-05 06:46:01 +08:00
godchen
7ab5b5d80d Add MinIO kv implements
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-03 19:00:11 +08:00
neza2017
cec903da19 Add parquet payload
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-02 18:31:56 +08:00
xige-16
36cf8a8ea7 Support return primary key when doing search
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2020-12-03 17:18:06 +08:00
zhenshan.cao
23aebe7e2a Fix proxy unittest
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-01 19:53:53 +08:00
cai.zhang
c1fbc4f8fa Open the tests for insert
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-12-01 15:18:57 +08:00
FluorineDog
6b74d82259 Support some validation for dsl parser
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-01 02:16:53 +08:00
zhenshan.cao
fda8d62b38 Fix make error
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-30 22:14:19 +08:00
bigsheeper
0cd3e8d86c Use exception instead of runtime error
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-11-30 17:58:23 +08:00
quicksilver
86573d0053 Create Jenkins pipeline
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-11-30 10:07:04 +08:00
FluorineDog
77fa75b1ec Add binary insert and warper of binary search, rename vector
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-30 05:18:44 +08:00
cai.zhang
85a544b79b Add dockerfile for each component
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-28 19:06:48 +08:00
dragondriver
568cef0730 Fix InsertTask and SearchTask
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2020-11-28 10:48:29 +08:00
FluorineDog
c9fb34142c Enable primary_key switch
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-11-28 09:16:00 +08:00
neza2017
32445dd02b Add cpu profile
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-11-26 16:01:31 +08:00
cai.zhang
57da66ba72 Update read config logic in docker
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2020-11-26 15:18:36 +08:00
zhenshan.cao
045fc3d808 Fix segment fault
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-11-26 14:29:01 +08:00