Commit Graph

29 Commits

Author SHA1 Message Date
gitee_code_template
6c8a4f3dc4 修改 baichuan readme 2023-09-28 19:59:36 +08:00
gitee_code_template
36ce0d2c37 add readme for baichua 2023-09-28 18:07:38 +08:00
i-robot
20e090b93d
!91 change llama65B batch size
Merge pull request !91 from Jializheng/master
2023-09-28 09:21:19 +00:00
gitee_code_template
6eb65d6dc9 baichuan13B模型 2023-09-28 10:30:20 +08:00
jializheng
b1de673886 change llama65B batch size 2023-09-28 09:35:30 +08:00
fengliangjun
7a21f0bf58 up 2023-09-26 14:30:47 +08:00
machangjun
8e436e3a9a add ffts mode
del torch_trans

del torch_trans and resove bloom ckpt and add bloom ffts+

add ffts mode

del torch_trans

del torch_trans and resove bloom ckpt and add bloom ffts+

replace fused_adam to adam

del unused code
2023-07-25 14:14:28 +08:00
Mrtutu
4532812837 更新bloom README: bloom7b在osacr-1G单机8卡训练 2023-07-26 14:10:51 +08:00
fengliangjun
260e8eea8f create megatron core 2023-07-24 15:00:57 +08:00
chenzomi
92c27d5e2a add a llama2 brach. 2023-07-21 15:20:25 +08:00
fengliangjun
db9c25bdd9 llama modify 2023-07-19 10:20:40 +08:00
liulinfeng
f6d7982b02 处理review意见 2023-07-14 15:34:07 +08:00
liulinfeng
243bfe5cfa Bloom适配SP代码 2023-07-14 14:18:07 +08:00
chenzomi
4455b80650 change the readme format. 2023-07-14 10:54:42 +08:00
liulinfeng
36f787bc89 Author:刘林峰
修改说明:
1、提交权重加载、推理生成文本的代码实现
2、修改codecheck问题
3、修复断点续训卡死的问题
2023-07-07 15:03:48 +08:00
kingsleyandher
3afb525a97 提交SP算法 2023-07-10 14:44:42 +08:00
kingsleyandher
21609f3083 llama模型zeroshot 33B/65B适配代码提交;提交README.md文件 2023-07-05 14:25:29 +08:00
wiyr
d87e921410 added trick 2023-06-30 11:00:38 +08:00
kingsleyandher
2c104a087e llama-zeroshot任务精度适配,对齐源论文中的效果。 2023-06-25 09:34:42 +08:00
wiyr
6304cab765 remove useless code 2023-06-20 16:54:12 +08:00
machangjun
2d8c6fee9d add bloom st and adapt new data load method
modify bloom st run

modify bloom st run

modify times

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add st
2023-06-17 17:36:17 +08:00
kingsleyandher
4e3b7cd992 LlamaTokenizer适配及预训练脚本更改 2023-06-13 12:33:33 +08:00
wiyr
2f826f7351 can run with bloom7b and pass ci 2023-06-12 14:42:29 +08:00
fengliangjun
37ba281c40 readme update 2023-06-10 11:26:55 +08:00
chenzomi
37cc0b949d change megatron to ascendspeed 2023-06-10 21:26:01 +08:00
fengliangjun
106a415556 inital AscendSpeed 2023-06-09 16:15:23 +08:00
wangyixian
d55d341fe1 Adapt the bloom 7.1b model to the AscendSpeed framework, which is jointly completed by liulinfeng and wangyixian 2023-06-06 22:30:19 +08:00
chenzomi
ce6af59f73 remove unused paraemter and models. 2023-05-26 10:53:07 +08:00
chenzomi
e4a120a662 fork megatron-deepspeed code. 2023-05-25 14:49:59 +08:00