Commit Graph

13 Commits

Author SHA1 Message Date
zhuwenxing
b348827102
test: add array data type and parquet file type for bulk insert case (#29030)
add array data type and parquet file type for the bulk insert case

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2023-12-13 19:56:38 +08:00
cai.zhang
1b7a503f89
enhance: Revert import support csv format (#28760)
Revert import support csv format.
issue: #28778

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-28 14:32:27 +08:00
groot
9c9ab08f54
test: Avoid unstable case of bulkinsert (#28679)
test: There are too many test cases for bulkinsert+partition_key. Each
case creates 10 bulkinsert tasks to import a file with 100~200 rows. The
default num_partitions is 64 for partition_key. So, each task will
generate 64 tiny segments. There are 10 cases, each case 10 tasks, each
task 64 tiny segment, totally there are 6400 tiny segments generated.
And all these segment row count is less than 1024, no need to build
index, and take part in compaction. There will be lots of compaction
tasks generated. It costs too much time to process these compaction
tasks. Eventually, some cases are timeout after waiting 5 minutes for
their segments to be ready and cases fail.

Specifying the num_partitions to a small value can avoid this problem.

```
[2023-11-21T03:41:16.187Z] testcases/test_bulk_insert.py::TestBulkInsert::test_partition_key_on_json_file[int_scalar-True-True] PASSED [ 54%]
[2023-11-21T03:41:42.796Z] testcases/test_bulk_insert.py::TestBulkInsert::test_partition_key_on_json_file[int_scalar-False-True] PASSED [ 57%]
[2023-11-21T03:42:04.694Z] testcases/test_bulk_insert.py::TestBulkInsert::test_partition_key_on_json_file[string_scalar-True-True] PASSED [ 60%]
[2023-11-21T03:42:31.205Z] testcases/test_bulk_insert.py::TestBulkInsert::test_partition_key_on_json_file[string_scalar-False-True] PASSED [ 63%]
[2023-11-21T03:43:38.876Z] testcases/test_bulk_insert.py::TestBulkInsert::test_partition_key_on_multi_numpy_files[10-150-13-True] XPASS [ 66%]
[2023-11-21T03:49:00.357Z] testcases/test_bulk_insert.py::TestBulkInsert::test_partition_key_on_multi_numpy_files[10-150-13-False] XFAIL [ 69%]
[2023-11-21T03:53:51.811Z] testcases/test_bulk_insert.py::TestBulkInsert::test_partition_key_on_csv_file[int_scalar-True] FAILED [ 72%]
[2023-11-21T03:58:58.283Z] testcases/test_bulk_insert.py::TestBulkInsert::test_partition_key_on_csv_file[int_scalar-False] FAILED [ 75%]
[2023-11-21T04:02:04.696Z] testcases/test_bulk_insert.py::TestBulkInsert::test_partition_key_on_csv_file[string_scalar-True] PASSED [ 78%]
[2023-11-21T04:02:26.608Z] testcases/test_bulk_insert.py::TestBulkInsert::test_partition_key_on_csv_file[string_scalar-False] PASSED [ 81%]
```

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-11-28 10:34:31 +08:00
zhuwenxing
9fd81025ea
test:skip a bulk insert testcase temporarily (#28619)
skip a bulk insert test case temporarily.
It is a known issue but needs more time to solve. skip the test case is
for not blocking other PR

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2023-11-22 10:14:22 +08:00
KumaJie
939ee0c257
Add csv bulk insert test (#28189)
Signed-off-by: kuma <675613722@qq.com>
Co-authored-by: kuma <675613722@qq.com>
2023-11-07 20:42:22 +08:00
zhuwenxing
356af86cba
[skip e2e]Mark a case as xfail (#28245)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2023-11-07 18:12:22 +08:00
zhuwenxing
fc429cabba
[test]Fix timeout in create index (#27437)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2023-10-08 15:09:31 +08:00
yanliang567
0bad7a32d2
Imporve bulk insert test in timeout (#27500)
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2023-10-07 17:43:33 +08:00
Jiquan Long
ace352aa95
Fix case test_partition_key_on_multi_numpy_files (#27421)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-09-28 18:15:27 +08:00
yanliang567
3e864d4210
Merge bulkinsert partition key enable tests from 2.2.0 branch (#27372)
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2023-09-26 18:43:32 +08:00
zhuwenxing
ee5da73fae
[test]Add bulk insert for test and refactoring the checker function (#25997)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2023-07-31 12:45:03 +08:00
binbin
4ba922876e
Add test cases support for random primary keys (#25840)
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2023-07-25 19:39:12 +08:00
zhuwenxing
b06e88153c
[test] Add bulk insert test in ci (#24329)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2023-05-24 15:27:26 +08:00