出现非法格式catch住的时候,应该将sample清空

出现非法格式catch住的时候,应该将sample清空,不然会一直报warning,sample也会一直累积新的内容得不到正确的更新。
This commit is contained in:
Gosicfly 2019-12-17 17:29:22 +08:00 committed by GitHub
parent 03e9ca2147
commit baf009fe5c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -117,6 +117,7 @@ def _read_conll(path, encoding='utf-8', indexes=None, dropna=True):
except Exception as e:
if dropna:
logger.warning('Invalid instance which ends at line: {} has been dropped.'.format(line_idx))
sample = []
continue
raise ValueError('Invalid instance which ends at line: {}'.format(line_idx))
elif line.startswith('#'):