mirror of https://gitee.com/fastnlp/fastNLP.git synced 2024-12-02 04:07:35 +08:00

History

FengZiYjun fb806163c3 remove unused codes; add more tests		2018-10-07 15:03:00 +08:00
..
__init__.py	- add validation loss into trainer.train	2018-07-11 21:51:35 +08:00
LICENSE	- add validation loss into trainer.train	2018-07-11 21:51:35 +08:00
main.py	remove unused codes; add more tests	2018-10-07 15:03:00 +08:00
model.py	- add validation loss into trainer.train	2018-07-11 21:51:35 +08:00
README.md	- add validation loss into trainer.train	2018-07-11 21:51:35 +08:00
test.py	- add validation loss into trainer.train	2018-07-11 21:51:35 +08:00
test.txt	- analyze codes for language model, unable to run yet	2018-09-23 16:03:20 +08:00
train.py	- add validation loss into trainer.train	2018-07-11 21:51:35 +08:00
train.txt	- analyze codes for language model, unable to run yet	2018-09-23 16:03:20 +08:00
utilities.py	- add validation loss into trainer.train	2018-07-11 21:51:35 +08:00
valid.txt	- analyze codes for language model, unable to run yet	2018-09-23 16:03:20 +08:00

PyTorch-Character-Aware-Neural-Language-Model

This is the PyTorch implementation of character-aware neural language model proposed in this paper by Yoon Kim.

Requiredments

The code is run and tested with Python 3.5.2 and PyTorch 0.3.1.

HyperParam	value
LSTM batch size	20
LSTM sequence length	35
LSTM hidden units	300
epochs	35
initial learning rate	1.0
character embedding dimension	15

Train the model with split train/valid/test data.

python train.py

The trained model will saved in cache/net.pkl. Test the model.

python test.py

Best result on test set: PPl=127.2163 cross entropy loss=4.8459

This implementation borrowed ideas from