Before this PR, DataNode considered auto-flush a valid flush
complete. It's wrong. So I open this PR to correct this behaviour
in DataNode.
Now binlog paths from auto-flush will be buffered in replica,
waiting until the manul flush to save into etcd all together.
See also: #5220, #5268
A follow up job of #5271
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
query base on timeline, add timestamp on these queries:
- ShowCollections
- HasCollection
- DescribeCollection
Resolves: #5219
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
Auto-flush of a segment is not considered flush-completed.
So we need to buffer binlog paths generated by auto-flush.
See also: #5268#5220
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Updata PhysicalChannelNames and VirtualChannelNames based
on ShardsNum when create collection, and update collVChan2Chan
map.
Resolves: #5240
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
GetEntityByID needs the flushed segment to be sorted by RowID field,
then do binary search to get the target id and entities.
See also: #5177
Signed-off-by: fishpenguin kun.yu@zilliz.com
Save ddmsg info and type into ETCD to support re-send ddmsg
when system restart or recover from failure (#5172)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Pass a parameter `IsExpr` from proxy to query node, so query
node can create search plan according to the variable `IsExpr`.
It makes `segcore` unnecessary to distinguish `dsl` or `expr`
using dynamic exception. When `IsExpr` is set to true, query
node will call `CreatePlanByExpr` according to the passed
information about expression. Otherwise query node will keep
still to use `CreatePlan` according to the `dsl` information. At
the same time, this pr adds some unittests to `CreatePlanByExpr`,
these unittests translate already exist case with `dsl` to case with
`expr`.
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
1. Support Term, like `A in [1, 2, 3]`
2. Support Not, like `! A < 3`
3. Support logical combination, like `A < 3 && B > 5 or C == 0`
Type: Feature
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
When the remaining space is empty, the zero size allocation will
get the pointer `end_`. However, `cudafree` the pointer `end_` will
cause to crash.
Fix: #5078#4770#4412#4340#3646
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
**What type of PR is this?**
- [x] Feature
**What this PR does / why we need it:**
This PR supports boolean expression as DSL.
1. The goal of this PR is to support predicates
like `A > 3 && not B < 5 or C in [1, 2, 3]`.
2. Defines `plan.proto`, as Intermediate Representation (IR)
used between go and cpp.
3. Support expr parser, convert predicate expr to IR
in proxynode, while doing static check there
4. Support IR to AST in cpp, enable the execution
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
**What type of PR is this?**
- [ ] API-change
- [x] BUG
- [ ] Improvement
- [ ] Documentation
- [ ] Feature
- [ ] Test and CI
- [ ] Code Refactoring
**Which issue(s) this PR fixes:**
Fixes #
**What this PR does / why we need it:**
Not Available
Now it's able to start multiple datanode, however, they will process the same insert data.
Further changes are needed to truly enable multiple datanode:
While registering, dataservice need to return different insert channels for different datanode.
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Segment should be sealed once it exceeds the limited size. This operation
should be done in segment allocator. Watcher only check the status to
decide whether the segment will be flushed.
Signed-off-by: sunby <bingyi.sun@zilliz.com>
