Commit Graph

61 Commits

Author SHA1 Message Date
congqixia
94f0951f0d
Fix query lock logic in query shard (#17034)
Previously query shard locks the querylock in collectionReplica before any search/query
The lock range is too large and easy to cause dead lock

This PR makes following changes:
- Rename collectionReplica to metaReplica which is more reasonable
- Make release collection operation cancels waiting search/query request
- Reduce the queryLock to collection level
- Add some unit tests for timeout & released case

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-16 18:23:55 +08:00
Xiaofan
92b6293be4
Fix QueryNode log level (#16604)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-07 10:27:51 +08:00
xige-16
515d0369de
Support string type in segcore (#16546)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
2022-04-29 13:35:49 +08:00
Xiaofan
89b4a34892
Fix Data race in NodeID (#16603)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-24 22:03:44 +08:00
congqixia
91d5436807
Improve log level and param function in QueryCollection (#16254)
Use debug level for all debug information
Use specific parameter function for appliable ones

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-29 12:31:27 +08:00
Ji Bin
3cd28420f1
Support compile under windows (#15786)
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
bigsheeper
caf9cbfcd4
Fix load failure and remove parition release related code (#16038)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-14 23:18:02 +08:00
bigsheeper
12eb8c3da5
Add prometheus metrics for QueryNode (#15649)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-02 14:49:55 +08:00
jaime
307a8ce535
Support compile and run on Mac (#15491)
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
2022-02-09 14:27:46 +08:00
godchen
7bead10aa8
[skip e2e]sort imports (#14183)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-24 17:08:42 +08:00
edward.zeng
1af317e3a3
[skip e2e] Refine querynode collection comment (#14136)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-24 13:23:51 +08:00
XuanYang-cn
84f7f90f06
[skip ci]Update license for qn collection (#12396)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-29 20:13:50 +08:00
bigsheeper
65608836e6
Remove vChannel for collection in historical (#12359)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-29 20:11:49 +08:00
bigsheeper
58964621a5
Remove tSafeWatcher and vChannel when release (#12264)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-26 01:33:16 +08:00
bigsheeper
152d55dbfa
Fix the deadlock of the querycoord cluster (#11796)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Co-authored-by: xige-16 <xi.ge@zilliz.com>
2021-11-17 23:37:11 +08:00
godchen
639800241d
Make flow graph for delta channel (#11379)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-09 09:27:04 +08:00
zhenshan.cao
9eacfe719a
[skip ci]Fix golint error in querynode (#9849)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-14 14:02:41 +08:00
bigsheeper
8ab25a1bb1
[skip ci] Add comment for deleteCollection (#9367)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-06 23:38:00 +08:00
bigsheeper
c86193fb7b
[skip ci] Add comment for newCollection (#9366)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-06 23:36:13 +08:00
bigsheeper
0e4ee6d420
[skip ci] Add comment for getLoadType (#9365)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-06 23:34:24 +08:00
bigsheeper
12dcbe3d78
[skip ci] Add comment for setLoadType (#9299)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-05 22:44:00 +08:00
bigsheeper
eab57af117
[skip ci] Add comment for checkReleasedPartitions (#9297)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-05 22:40:34 +08:00
bigsheeper
8b30d9aeb5
[skip ci] Add comment for deleteReleasedPartition (#9296)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-05 22:38:44 +08:00
bigsheeper
040618187b
[skip ci] Add comment for addReleasedPartition (#9244)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-05 11:19:23 +08:00
bigsheeper
4427b24171
[skip ci] Add comment for setReleaseTime (#9242)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-05 11:17:29 +08:00
bigsheeper
c1b9f4303f
[skip ci] Add comment for getReleaseTime (#9243)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-05 11:15:14 +08:00
bigsheeper
be9ab19b7e
[skip ci] Add comment for getPChannels (#9174)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-04 08:22:51 +08:00
bigsheeper
91e4adff15
[skip ci] Add comment for addPChannels (#9173)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-04 08:18:59 +08:00
bigsheeper
358d2e1e07
[skip ci] Add comment for getVChannels (#9172)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-04 08:16:44 +08:00
bigsheeper
02b2f3c88b
[skip ci] Add comment for addVChannels (#9103)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-03 00:52:02 +08:00
bigsheeper
0b545efa4a
[skip ci] Add comment for removePartitionID (#9102)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-03 00:40:03 +08:00
bigsheeper
5b074732f5
[skip ci] Add comment for addPartitionID (#9101)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-03 00:38:13 +08:00
bigsheeper
a9a4b89b0e
[skip ci] Add comment for Schema (#9045)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-01 23:44:33 +08:00
bigsheeper
243ee06660
[skip ci] Add comment for collection id (#8806)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-28 22:40:05 +08:00
bigsheeper
bcccd767ac
Add unittests for querynode (#7510)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-07 15:45:59 +08:00
xige-16
6b65529086
Fix search error after releasePartition (#6135)
* fix search error after releasePartition

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-26 16:08:11 +08:00
bigsheeper
bbd8a7e13a
Add released partitions and fix search error in empty partition (#5893)
* rename flowgraphType to loadType

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* add load type and released partitions

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* filter released partition

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-19 18:38:07 +08:00
bigsheeper
58bf92a407
Add vChannel,pChannel in query node and get vChannel from master in query service (#5784)
* add vChannel,pChannel in query node and get vChannel from master in query service

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-15 20:06:10 +08:00
bigsheeper
cdbc6d2c94
Refactor query node and query service (#5751)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 12:41:40 +08:00
bigsheeper
8aae0f7cc9
Refactor flow graph and load/watchDML in query node (#5682)
* Refactor flow graph and load/watchDML in query node

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-06-09 11:37:55 +08:00
bigsheeper
03f90ff28c
refactor collection replica and add tsafe replica in querynode (#5475)
* refactor collection replica and add tsafe replica in querynode

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* static check

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* static check2

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* code format

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-05-28 15:40:32 +08:00
bigsheeper
88e1d651cb
Remove partition and segment ids properly in query node (#5090)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

**What type of PR is this?**

- [ ] API-change
- [x] BUG
- [ ] Improvement
- [ ] Documentation
- [ ] Feature
- [ ] Test and CI
- [ ] Code Refactoring

**Which issue(s) this PR fixes:**

Fixes #

**What this PR does / why we need it:**

Not Available
2021-04-29 07:59:08 +00:00
Xiangyu Wang
82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
bigsheeper
71b8f309b2 Add licence to query node and query service
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-19 13:47:10 +08:00
xige-16
7e60460aee close timeTickStream in flow graph when the collection is released
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-26 18:40:04 +08:00
bigsheeper
447a15207e Add zap log to query node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-03-05 09:21:35 +08:00
bigsheeper
1578c13224 Refactor collection replica
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-05 10:53:11 +08:00
bigsheeper
f12a0490bb Do not drop collections or partitions in flow graph
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 18:12:48 +08:00
Xiangyu Wang
fd9e42d3ed Update msgstream/unmarshal interface
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-02-03 17:30:10 +08:00