- add character vocab in preprocessor
- add dataset loader for language model dataset
- other minor adjustments
- preserve only a little example data for language model
2. changes of names:
aggregation ----> aggregator
interaction ----> interactor
action.py ----> sampler.py
BasePreprocess ---> Preprocessor
BaseTester ----> Tester
BaseTrainer ----> Trainer
3. add more code comments
4. fix bugs in predictor's data_forward
5. in sampler.py, remove Bachifier, fix some codes. but not test
6. remove unused codes in other_modules.py & utils.py
7. update fastnlp.py with new config file names and code comments
8. add data examples in data_for_tests/
- apply DataSet in Predictor; remove sub-predictors; add "task" argument to specify which task to predict, as how Trainer/Tester did.
- remove Action class
- add helper function for DataSet, to create DataSet easily
- more code comments
- clean up unnecessary codes
- add unit tests for Batch, Predictor, Preprocessor, Trainer, Tester
- update LabelField's to_tensor method to support int & str single label
- update preprocessor's convert_to_dataset method to support single label inputs
- introduce "task" in Trainer/Tester's data_forward, Tester's evaluate and metrics methods
- in cnn_text_classification.py, change the name of the argument of forward
- in sequence_modeling.py, change the name of the argument of forward
- minor adjustments in test codes
- text_classify.py works
- add DataSet, Instance, Field to represent data in different levels
- encapsulate batching method in Batch class
- modify samplers in action.py to fit Batch
- preprocessor.run returns DataSet, instead of list
- Use Batch in Trainer/Tester
- add required_arg "task" in Trainer/Tester
- remove SeqLabelTrainer/SeqLabelTester dependencies successfully. They empty classes to deprecate.
- modify SeqLabeling model, add another argument in forward, in order to compute mask inside model
- test\model\seq_labeling.py works
