11 Commits

Author	SHA1	Message	Date
fengliangjun	7a21f0bf58	up	2023-09-26 14:30:47 +08:00
machangjun	8e436e3a9a	add ffts mode ... del torch_trans del torch_trans and resove bloom ckpt and add bloom ffts+ add ffts mode del torch_trans del torch_trans and resove bloom ckpt and add bloom ffts+ replace fused_adam to adam del unused code	2023-07-25 14:14:28 +08:00
fengliangjun	260e8eea8f	create megatron core	2023-07-24 15:00:57 +08:00
kingsleyandher	3afb525a97	提交SP算法	2023-07-10 14:44:42 +08:00
kingsleyandher	b5a0fc04a2	Optimizer Pipeline parallel ... Author: 李冰聪	2023-07-07 11:41:45 +08:00
kingsleyandher	bc2a4a33d5	提交VP算法 ... Author: 李冰聪/张梦阳	2023-06-29 10:22:17 +08:00
machangjun	2d8c6fee9d	add bloom st and adapt new data load method ... modify bloom st run modify bloom st run modify times add new pretrain_bloom.py add new pretrain_bloom.py add new pretrain_bloom.py add new pretrain_bloom.py add new pretrain_bloom.py add new pretrain_bloom.py add new pretrain_bloom.py add new pretrain_bloom.py add new pretrain_bloom.py add st	2023-06-17 17:36:17 +08:00
chenzomi	37cc0b949d	change megatron to ascendspeed	2023-06-10 21:26:01 +08:00
fengliangjun	106a415556	inital AscendSpeed	2023-06-09 16:15:23 +08:00
chenzomi	ce6af59f73	remove unused paraemter and models.	2023-05-26 10:53:07 +08:00
chenzomi	e4a120a662	fork megatron-deepspeed code.	2023-05-25 14:49:59 +08:00