fengliangjun
e32e0c22f9
update ci/access_control_test.py.
...
Signed-off-by: fengliangjun <fengliangjun@huawei.com>
2023-09-26 08:42:32 +00:00
fengliangjun
30a6713f76
删除文件 tests/ut/test_preprocessing.py
2023-09-26 08:30:47 +00:00
fengliangjun
f8cebebd4e
update .gitignore.
...
Signed-off-by: fengliangjun <fengliangjun@huawei.com>
2023-09-26 08:02:39 +00:00
fengliangjun
314e852a0a
update .gitignore.
...
Signed-off-by: fengliangjun <fengliangjun@huawei.com>
2023-09-26 07:47:34 +00:00
fengliangjun
de42a65b6b
update .gitignore.
...
Signed-off-by: fengliangjun <fengliangjun@huawei.com>
2023-09-26 07:37:14 +00:00
fengliangjun
9b87852c90
update ci/access_control_test.py.
...
Signed-off-by: fengliangjun <fengliangjun@huawei.com>
2023-09-26 07:20:59 +00:00
fengliangjun
964cc96582
update README.md.
...
Signed-off-by: fengliangjun <fengliangjun@huawei.com>
2023-09-26 06:50:01 +00:00
fengliangjun
7a21f0bf58
up
2023-09-26 14:30:47 +08:00
yangyuan667
be5d413ec6
Won't Import ascendspeed.ops if unneccssary
2023-08-16 09:35:10 +08:00
xuqiang
8e0f2e9b1a
sync parallel_state.py file
2023-08-15 10:45:26 +08:00
yangyuan667
ff9d62bcc7
Add gcc compiler args
2023-08-11 11:44:28 +08:00
machangjun
8e436e3a9a
add ffts mode
...
del torch_trans
del torch_trans and resove bloom ckpt and add bloom ffts+
add ffts mode
del torch_trans
del torch_trans and resove bloom ckpt and add bloom ffts+
replace fused_adam to adam
del unused code
2023-07-25 14:14:28 +08:00
yangyuan667
2ae07e63ee
[New]add FlashAttention adpater
2023-08-10 11:11:24 +08:00
xuqiang
eb3dcf3f02
update OWNERS
2023-08-10 09:40:28 +08:00
kingsleyandher
e4bee6c48a
Layer Fusion for LLama
2023-07-25 15:25:55 +08:00
xuqiang
49f7bc726e
update OWNERS
2023-08-07 09:46:16 +08:00
Mrtutu
4532812837
更新bloom README: bloom7b在osacr-1G单机8卡训练
2023-07-26 14:10:51 +08:00
fengliangjun
260e8eea8f
create megatron core
2023-07-24 15:00:57 +08:00
fengliangjun
b559dc6385
set log level
2023-07-20 22:23:33 +08:00
chenzomi
92c27d5e2a
add a llama2 brach.
2023-07-21 15:20:25 +08:00
fengliangjun
db9c25bdd9
llama modify
2023-07-19 10:20:40 +08:00
machangjun
de85201818
modify baddbmm to bmm to accelerate
2023-07-14 15:41:00 +08:00
simon717
3a7d87c2b8
1. llama_model.py attention实现回退
...
2. huggingface llama权重转换脚本
3. llama并行训练策略改变后权重转换脚本
4. codecheck解决
2023-07-06 10:33:21 +08:00
simon717
fedb2127c0
1. llama_model.py attention实现回退
...
2. huggingface llama权重转换脚本
3. llama并行训练策略改变后权重转换脚本
2023-07-05 15:30:44 +08:00
liulinfeng
f6d7982b02
处理review意见
2023-07-14 15:34:07 +08:00
liulinfeng
243bfe5cfa
Bloom适配SP代码
2023-07-14 14:18:07 +08:00
chenzomi
937791fa6d
format some code.
2023-07-21 01:42:26 +08:00
chenzomi
4455b80650
change the readme format.
2023-07-14 10:54:42 +08:00
kingsleyandher
31cf1ecdd0
merge code
2023-07-11 20:12:39 +08:00
liulinfeng
85e23be9f2
删除保存初始权重的代码
2023-07-10 11:16:43 +08:00
liulinfeng
8aa62e1049
修复codecheck问题,删除多余的空行
2023-07-10 10:54:13 +08:00
liulinfeng
36f787bc89
Author:刘林峰
...
修改说明:
1、提交权重加载、推理生成文本的代码实现
2、修改codecheck问题
3、修复断点续训卡死的问题
2023-07-07 15:03:48 +08:00
kingsleyandher
3afb525a97
提交SP算法
2023-07-10 14:44:42 +08:00
kingsleyandher
b5a0fc04a2
Optimizer Pipeline parallel
...
Author: 李冰聪
2023-07-07 11:41:45 +08:00
kingsleyandher
21609f3083
llama模型zeroshot 33B/65B适配代码提交;提交README.md文件
2023-07-05 14:25:29 +08:00
wiyr
d87e921410
added trick
2023-06-30 11:00:38 +08:00
kingsleyandher
bc2a4a33d5
提交VP算法
...
Author: 李冰聪/张梦阳
2023-06-29 10:22:17 +08:00
kingsleyandher
2c104a087e
llama-zeroshot任务精度适配,对齐源论文中的效果。
2023-06-25 09:34:42 +08:00
wiyr
6304cab765
remove useless code
2023-06-20 16:54:12 +08:00
machangjun
2d8c6fee9d
add bloom st and adapt new data load method
...
modify bloom st run
modify bloom st run
modify times
add new pretrain_bloom.py
add new pretrain_bloom.py
add new pretrain_bloom.py
add new pretrain_bloom.py
add new pretrain_bloom.py
add new pretrain_bloom.py
add new pretrain_bloom.py
add new pretrain_bloom.py
add new pretrain_bloom.py
add st
2023-06-17 17:36:17 +08:00
simon717
7047d75663
Llama模型结构与huggingface对齐, 前向推理结果与huggingface对齐
2023-06-20 11:27:15 +08:00
kingsleyandher
4e3b7cd992
LlamaTokenizer适配及预训练脚本更改
2023-06-13 12:33:33 +08:00
wiyr
2f826f7351
can run with bloom7b and pass ci
2023-06-12 14:42:29 +08:00
wiyr
89bfaf64c2
added fused softmax, tokenizer, language, utils for bloom
2023-06-12 11:50:46 +08:00
wiyr
6d07c029fe
added func code
2023-06-12 10:39:51 +08:00
fengliangjun
37ba281c40
readme update
2023-06-10 11:26:55 +08:00
chenzomi
37cc0b949d
change megatron to ascendspeed
2023-06-10 21:26:01 +08:00
fengliangjun
106a415556
inital AscendSpeed
2023-06-09 16:15:23 +08:00
wangyixian
d55d341fe1
Adapt the bloom 7.1b model to the AscendSpeed framework, which is jointly completed by liulinfeng and wangyixian
2023-06-06 22:30:19 +08:00
chenzomi
ce6af59f73
remove unused paraemter and models.
2023-05-26 10:53:07 +08:00