test/fastNLP
1
0
Fork 0
mirror of https://gitee.com/fastnlp/fastNLP.git synced 2024-12-03 12:47:35 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

check if data_dev.pkl exists

Browse Source 
In line 100, add a if statement to check whether there exists a file named "data_dev.pkl" in the pickle path. If not, the file won't be loaded.
This commit is contained in:
Coet 2018-08-29 17:03:26 +08:00 committed by GitHub
parent 18586c9c6d
commit 50b5345539
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
fastNLP/core/preprocess.py
View File

@ -97,7 +97,8 @@ class BasePreprocess(object):
save_pickle(data_train, pickle_path, "data_train.pkl")
else:
data_train = load_pickle(pickle_path, "data_train.pkl")
data_dev = load_pickle(pickle_path, "data_dev.pkl")
if pickle_exist(pickle_path, "data_dev.pkl"):
data_dev = load_pickle(pickle_path, "data_dev.pkl")
else:
# cross_val is True
if not pickle_exist(pickle_path, "data_train_0.pkl"):
@ -307,4 +308,4 @@ def infer_preprocess(pickle_path, data):
data_index = []
for example in data:
data_index.append([word2index.get(w, DEFAULT_UNKNOWN_LABEL) for w in example])
return data_index
return data_index