test/ModelLink2
1
0
Fork 0
mirror of https://gitee.com/ascend/ModelLink.git synced 2024-12-05 05:17:40 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
986 Commits 6 Branches 6 Tags 57 MiB
289ad342f8
Commit Graph

485 Commits

Author SHA1 Message Date
shengjiayi@huawei.com
620dd31e4f llama33B适配更新 2023-12-04 19:06:47 +08:00
wenjiang2357
de248d084c Merge branch 'master' of https://gitee.com/ascend/AscendSpeed into master 2023-12-04 17:24:22 +08:00
wenjiang2357
9c594efe8e llama2-7B readme更新 2023-12-04 17:23:55 +08:00
i-robot
96f103fcda
!352 Aquila7B模型精度达标，性能达标，已修改AscendSpeed的master README增加Aquila相关信息 
Merge pull request !352 from yaojia2021/master
 2023-12-04 06:57:38 +00:00
i-robot
f18cbbb0d9
!334 add function of save checkpoint to cpu tensor 
Merge pull request !334 from xqiangx1991/cpu-checkpoint
 2023-12-04 06:19:18 +00:00
19952409173
476131b265 modify llama-7B/13B readme for Q4 
modify llama-7B/13B readme for Q4

modify llama-7B/13B readme for Q4

modify llama-7B/13B readme for Q4

modify llama-7B/13B readme for Q4
 2023-12-04 11:29:28 +08:00
i-robot
f1a7f4daa5
!351 FineTune Internlm 
Merge pull request !351 from l00636998/master
 2023-12-04 02:58:33 +00:00
xqiangx1991
0ef6c33cda Merge branch 'master' of gitee.com:ascend/AscendSpeed into cpu-checkpoint 
Signed-off-by: xqiangx1991 <joeltsui@gmail.com>
 2023-12-04 02:07:32 +00:00
i-robot
f10cdf1267
!343 修复合并脚本后的lora功能 
Merge pull request !343 from zhangshengdong29/master
 2023-12-04 02:05:15 +00:00
y00546703
3ba91a9d58 add aquila-7B model, modified AscendSpeed master README accordingly. 2023-12-04 09:59:20 +08:00
lizekai
99af579fe5 FineTune Internlm 2023-12-02 18:59:14 +08:00
liuyanghan
fe1c7d7869 书生65B模型合入 2023-12-02 10:39:47 +08:00
zhansghengdong
2c516db866 修复合并脚本后的lora功能 2023-12-01 16:16:01 +08:00
i-robot
680c0fbe65
!333 Finetune Internlm 
Merge pull request !333 from l00636998/master
 2023-12-01 06:38:03 +00:00
lizekai
9415637bed FineTune Internlm 2023-12-01 10:50:46 +08:00
xqiangx1991
79ee48d108 add function of save checkpoint to cpu tensor 2023-12-01 10:15:26 +08:00
i-robot
a5d76b70b1
!330 支持自适应选择重计算-优化重计算策略选择算法，加上UT 
Merge pull request !330 from Gou Jicheng/dev_from_master_only_recompute
 2023-11-30 11:15:49 +00:00
goujicheng
8ca3cfe00a 支持自适应选择重计算-优化重计算策略选择算法，加上UT 2023-11-30 14:12:08 +08:00
mhh001
2e6003abf3 合并脚本后，一些问题修改 2023-11-30 09:41:55 +08:00
i-robot
49aac6e5c4
!305 删除llama_model.py，llama模型合并到transformer.py 
Merge pull request !305 from mhh001/master
 2023-11-28 11:19:01 +00:00
mhh001
1454fa076d 删除llama_model.py，llama模型合并到transformer.py 2023-11-28 17:26:17 +08:00
fengliangjun
a71f138128 update me 2023-11-28 11:50:59 +08:00
i-robot
cc8f80bda7
!315 新增bloom模型评测脚本 
Merge pull request !315 from huangyiming/master
 2023-11-24 09:18:57 +00:00
matrixssy
eb8973ce0b llama2-7b全流程脚本及readme 2023-11-24 16:52:16 +08:00
huangyiming
b9f2d96a5d 1. 增加bloom模型评测脚本 
2. 修复Human_eval代码路径问题
 2023-11-24 16:19:18 +08:00
ningbenzhe1
f801b32e15 llama2 uploads the training inference script and the README. 2023-11-24 12:15:46 +08:00
i-robot
569e1cc7b3
!299 bloom适配lora以及bloom7B全流程打通 
Merge pull request !299 from ningbenzhe1/master
 2023-11-24 01:20:30 +00:00
ningbenzhe1
8a282348ca Adapt to lora and get through the whole process. 2023-11-24 09:16:02 +08:00
19952409173
f36b2414dc 更新llama-7B、llama-13B README.md;新增llama-13B 16p参数脚本 
更新llama-7B、llama-13B README.md;新增llama-13B 16p参数脚本

更新llama-7B、llama-13B README.md;新增llama-13B 16p参数脚本

更新llama-7B、llama-13B README.md;新增llama-13B 16p参数脚本
 2023-11-23 20:40:36 +08:00
fengliangjun
0b79c57e1b add chinese read me 2023-11-22 20:17:28 +08:00
i-robot
b04f782575
!288 修改README文件单词拼写错误 
Merge pull request !288 from xiongliangcheng/master
 2023-11-22 06:15:25 +00:00
liulinfeng
e57e6e0b2b Bloom模型适配FA及打通基于megatron框架预训练 2023-11-22 10:39:42 +08:00
i-robot
8757d302a5
!291 增加bloom中文readme 
Merge pull request !291 from fengliangjun/master
 2023-11-22 01:29:17 +00:00
i-robot
90bcaf0e48
!285 自适应选择重计算特性开发 
Merge pull request !285 from 刘蛟龙/dev_from_master_only_recompute
 2023-11-21 12:09:05 +00:00
goujicheng
fcbc94aa51 支持自适应选择重计算 2023-11-21 19:18:30 +08:00
i-robot
3dbf8233cb
!278 混精训练内存复用O1（FP32梯度副本去冗余）算法提交 
Merge pull request !278 from Kingsleyandher/master
 2023-11-20 11:54:20 +00:00
fengliangjun
540651a4fa add chinese bloom readme 2023-11-20 19:33:09 +08:00
l00619700
70412752da FP32梯度副本去冗余等价算法提交 2023-11-20 16:40:23 +08:00
fengliangjun
eea45231c2 add baichuan chinese readme 2023-11-17 13:58:19 +08:00
xiongliangcheng
1ce6eee7de modified: examples/baichuan/README.md 2023-11-17 13:56:37 +08:00
i-robot
7067138248
!274 添加alpaca中文readme 
Merge pull request !274 from fengliangjun/master
 2023-11-16 01:28:45 +00:00
chantcalf
b6d0d67f3c 由于transformer框架 v3升级，更新LLama2-70B脚本和readme 2023-11-15 18:40:06 +08:00
fengliangjun
79eb5e89ef add alpaca chinese read 2023-11-14 21:59:02 +08:00
mhh001
ad6f867d94 transformer框架升级 2023-11-14 20:44:46 +08:00
i-robot
711911f9fc
!259 LLama2-70B 添加readme 
Merge pull request !259 from Liuchang/master
 2023-11-13 08:33:08 +00:00
chantcalf
eb38c34a8c LLama2-70B 添加readme 2023-11-13 16:13:29 +08:00
i-robot
63ee9d5ca3
!253 在线推理接口支持多batch输入 
Merge pull request !253 from shishaoyu/master
 2023-11-13 08:04:26 +00:00
matrixssy
934614b039 在线推理支持batch size >= 1 2023-11-10 15:58:07 +08:00
robert
a14f76b18e 修复在INF_NAN模式下拉起176B OOM问题 2023-11-07 22:32:57 +08:00
i-robot
ac98107b42
!248 LLama model权重转换支持自定义和GQA 
Merge pull request !248 from Liuchang/master
 2023-11-07 13:31:04 +00:00
chantcalf
df8ab5e07c LLama model权重转换支持自定义、GQA和MLP融合 
1.支持自定义模型参数：type设为非预制模型，可自定义num_layers、num_heads、hidden_size
2.支持GQA：增加num_kv_heads参数
3.支持merge-mlp，开启则将gate_proj和up_proj权重合并
4.修复arguments中关于位置编码判断的bug
5.修复convert_weights_when_tp_pp_change中的若干bug
 2023-11-07 11:36:07 +08:00
i-robot
5ff0d1d4b9
!247 送检整改： README中补充加固说明 
Merge pull request !247 from 郭鑫杰1/master
 2023-11-06 11:04:12 +00:00
xqiangx1991
3588767539 !250 fix pip install megatron 
* fix pip install megatron
 2023-11-06 03:16:21 +00:00
g00841271
877ade85dd README security hardening description and Clean code 2023-11-04 18:03:37 +08:00
i-robot
8e06ed062f
!223 (PART 3)在线推理框架ST用例与样例补充 
Merge pull request !223 from shishaoyu/master
 2023-11-03 02:52:56 +00:00
i-robot
f2928ea58a
!246 修改baichuan13B权重链接 
Merge pull request !246 from xiongliangcheng/master
 2023-11-02 13:47:47 +00:00
i-robot
a7ef12d257
!210 LLama2 model添加GQA和GQA FA功能 
Merge pull request !210 from Liuchang/master
 2023-11-02 02:13:17 +00:00
matrixssy
b8f718cc3d 1.补充样例代码与ST用例 2023-11-01 11:58:52 +08:00
liuyanghan
d56c249a43 更新README中失效huggingface链接 2023-11-01 11:47:40 +08:00
xiongliangcheng
146bfd7012 modify examples/baichuan/README.md 2023-11-01 10:07:20 +08:00
i-robot
1aefef58e0
!218 指令数据集增加动态padding功能 
Merge pull request !218 from ningbenzhe1/master
 2023-11-01 01:04:16 +00:00
chantcalf
38c83b2a74 LLama model添加GQA和GQA-FA功能 
LLama model rotray分支增加GQA功能
LLama model 增加FlashAttention 接口
添加llama2-70B 启动脚本
 2023-10-31 22:17:02 +08:00
i-robot
02cb18d70d
!236 修改LLama7B/13B 书生模型超参，提升性能 
Merge pull request !236 from Kingsleyandher/master
 2023-10-31 08:15:54 +00:00
ningbenzhe1
a8c0549085 Added dynamic padding function. 2023-10-31 11:59:22 +08:00
i-robot
b5dc9c8b58
!241 修改baichuan7B权重转换脚本和readme 
Merge pull request !241 from xiongliangcheng/master
 2023-10-30 15:00:13 +00:00
i-robot
d6e6722f6a
!227 add llama-33b script 
Merge pull request !227 from chenzhihong/llama-33b
 2023-10-30 07:19:58 +00:00
xiongliangcheng
97de2c58f5 修改Baichuan模型的README文件 2023-10-28 17:51:20 +08:00
l00619700
5ec2c196d4 修改模型超参，提升性能 2023-10-28 17:37:06 +08:00
liuyanghan
adb069216d 修改版本包链接 2023-10-28 09:12:27 +08:00
liuyanghan
6801afd180 README添加提示语; 修复评估代码中bug 2023-10-27 15:11:00 +08:00
Zhihong Chen
8e6a9c0574 add llama-33b script and readme update 2023-10-26 15:38:32 +08:00
l00619700
a511221b5c llama 13b增加自定义选择重计算，兼容之前所有逻辑。 2023-10-24 19:59:03 +08:00
i-robot
5e27f2e9b6
!157 Add alpaca 13B performance and device info to README.md 
Merge pull request !157 from stacey/master
 2023-10-20 01:41:09 +00:00
19952409173
a4367fb11b add performance and device information 
add performance and device information

add performance and device information

add performance and device information

add performance and device information

add performance and device information

move images to source/images/alpaca, add inference

move images to source/images/alpaca, add inference

move images to source/images/alpaca, add inference

move images to source/images/alpaca, add inference

move images to source/images/alpaca, add inference
 2023-10-20 09:31:36 +08:00
liuyanghan
c703b968e3 1 ./examples/bloom/images 文件夹移动到 ./sources/images/bloom 
2 ./examples/baichuan/images 文件夹移动到 ./sources/images/baichuan
3 修改评估代码bug
4 删除reanMe中冗余操作步骤
 2023-10-19 15:36:39 +08:00
i-robot
16baa028f4
!176 llama-65b readme update 
Merge pull request !176 from chenzhihong/master
 2023-10-19 06:19:30 +00:00
l00619700
76609dec8c 提交llama13b脚本与readme文件 2023-10-18 14:23:53 +08:00
Zhihong Chen
e6b3b937b4 llama-65b readme update 2023-10-18 12:11:13 +08:00
i-robot
45434eead6
!167 新增bloom权重转换脚本以及推理脚本，完善bloom readme 
Merge pull request !167 from ningbenzhe1/master
 2023-10-18 01:38:15 +00:00
ningbenzhe1
010747ebb8 Provide bloom reasoning scripts and refine the bloom readme. 2023-10-17 19:06:12 +08:00
i-robot
602eb623e3
!149 llama model加入选择性重计算，更新脚本 
Merge pull request !149 from 胡晓波/master
 2023-10-17 09:56:10 +00:00
胡晓波
5bd2363a7b llama model支持选择性重计算；修改训练脚本 2023-10-17 15:57:16 +08:00
mhh001
6fe54d95d5 目前llama2 model均使用llama model相关代码，删除llama2无关的代码。 2023-10-17 15:13:37 +08:00
l00562083
1609a0fa25 add llama-2-7b model 2023-10-14 04:11:12 +08:00
l00619700
58a3865fc9 InternLM readme提交 2023-10-13 14:04:42 +08:00
liulinfeng
c747d4ceeb 修复Bloom的权重转换脚本无法正确拉起的错误及修改Bloom README 2023-10-12 17:07:53 +08:00
i-robot
b9b1feadc1
!158 ./README.md.修改 
Merge pull request !158 from liuyanghan/master
 2023-10-11 09:47:40 +00:00
i-robot
bb09ab2962
!155 送检整改： codecheck空行缩进修改 
Merge pull request !155 from chenzhihong/codecheck
 2023-10-11 08:21:53 +00:00
Zhihong Chen
9d5c53521f codecheck update 2023-10-11 11:19:20 +08:00
i-robot
a22bd383ca
!148 修改baichuan权重转换脚本和readme 
Merge pull request !148 from 高楼大熊/master
 2023-10-11 02:01:45 +00:00
liuyanghan
d19f47f4be ./README.md.修改: 
1、明确链接标志 如Baichuan-7B/13B -> Baichuan-[README:7B/13B]
2、ARM环境通过下载troch安装包安装torch(直接pip安装会导致torch_npu import失败)
3、更正错误链接; 修改链接排版方式

examples/alpaca/README.md.修改:
1、ARM环境通过下载troch安装包安装torch(直接pip安装会导致torch_npu import失败)

examples/llama/pretrain_llama_7B_zero_8p.sh.
1、解决混合精度下溢出问题
 2023-10-10 21:05:41 +08:00
xiongliangcheng
4946f364a6 add readme 2023-10-10 17:40:33 +08:00
xqiangx1991
393f78a3a3 remove unused functions and files. 2023-10-10 10:40:00 +08:00
i-robot
4a42cf21f2
!120 add readme for llama-65b 
Merge pull request !120 from chenzhihong/master
 2023-10-09 11:34:21 +00:00
Zhihong Chen
9efb449178 add readme for llama-65b 2023-10-09 18:33:33 +08:00
i-robot
a9d3f05550
!98 add finetune chinese llama alpaca scripts and readme file 
Merge pull request !98 from stacey/master
 2023-10-08 09:10:24 +00:00
19952409173
fb12bc05ee add finetune chinese llama alpaca scripts and readme file 
modify readme bugs

modify readme bugs

modify readme

modify readme

modify readme

modify readme

modify readme

modify readme

modify readme

modify readme
 2023-10-08 15:38:53 +08:00
gitee_code_template
db2ef8b3fc add baichuan readme 2023-10-08 11:20:27 +08:00
gitee_code_template
23e0424ea8 add baichuan readme 2023-10-08 09:17:14 +08:00
gitee_code_template
e46a197248 add baichuan readme 2023-10-08 09:07:13 +08:00
gitee_code_template
88af8f0d1d add baichuan readme 2023-10-07 23:20:26 +08:00
gitee_code_template
4a04dc7355 add baichuan readme 2023-10-07 22:54:15 +08:00
gitee_code_template
d1d42a7fc9 add baichuan readme 2023-10-07 22:43:16 +08:00
gitee_code_template
b2159ae221 add readme baichuan 2023-10-07 21:20:00 +08:00
gitee_code_template
1ad58c3e2d add readme baichuan 2023-10-07 20:48:05 +08:00
gitee_code_template
062a639d1d add baichuan readme 2023-10-07 19:37:43 +08:00
gitee_code_template
6c8a4f3dc4 修改 baichuan readme 2023-09-28 19:59:36 +08:00
gitee_code_template
36ce0d2c37 add readme for baichua 2023-09-28 18:07:38 +08:00
i-robot
20e090b93d
!91 change llama65B batch size 
Merge pull request !91 from Jializheng/master
 2023-09-28 09:21:19 +00:00
gitee_code_template
6eb65d6dc9 baichuan13B模型 2023-09-28 10:30:20 +08:00
jializheng
b1de673886 change llama65B batch size 2023-09-28 09:35:30 +08:00
fengliangjun
7a21f0bf58 up 2023-09-26 14:30:47 +08:00
machangjun
8e436e3a9a add ffts mode 
del torch_trans

del torch_trans and resove bloom ckpt and add bloom ffts+

add ffts mode

del torch_trans

del torch_trans and resove bloom ckpt and add bloom ffts+

replace fused_adam to adam

del unused code
 2023-07-25 14:14:28 +08:00
Mrtutu
4532812837 更新bloom README: bloom7b在osacr-1G单机8卡训练 2023-07-26 14:10:51 +08:00
fengliangjun
260e8eea8f create megatron core 2023-07-24 15:00:57 +08:00
chenzomi
92c27d5e2a add a llama2 brach. 2023-07-21 15:20:25 +08:00
fengliangjun
db9c25bdd9 llama modify 2023-07-19 10:20:40 +08:00
liulinfeng
f6d7982b02 处理review意见 2023-07-14 15:34:07 +08:00
liulinfeng
243bfe5cfa Bloom适配SP代码 2023-07-14 14:18:07 +08:00
chenzomi
4455b80650 change the readme format. 2023-07-14 10:54:42 +08:00
liulinfeng
36f787bc89 Author：刘林峰 
修改说明：
1、提交权重加载、推理生成文本的代码实现
2、修改codecheck问题
3、修复断点续训卡死的问题
 2023-07-07 15:03:48 +08:00
kingsleyandher
3afb525a97 提交SP算法 2023-07-10 14:44:42 +08:00
kingsleyandher
21609f3083 llama模型zeroshot 33B/65B适配代码提交；提交README.md文件 2023-07-05 14:25:29 +08:00
wiyr
d87e921410 added trick 2023-06-30 11:00:38 +08:00
kingsleyandher
2c104a087e llama-zeroshot任务精度适配，对齐源论文中的效果。 2023-06-25 09:34:42 +08:00
wiyr
6304cab765 remove useless code 2023-06-20 16:54:12 +08:00
machangjun
2d8c6fee9d add bloom st and adapt new data load method 
modify bloom st run

modify bloom st run

modify times

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add new pretrain_bloom.py

add st
 2023-06-17 17:36:17 +08:00
kingsleyandher
4e3b7cd992 LlamaTokenizer适配及预训练脚本更改 2023-06-13 12:33:33 +08:00
wiyr
2f826f7351 can run with bloom7b and pass ci 2023-06-12 14:42:29 +08:00
fengliangjun
37ba281c40 readme update 2023-06-10 11:26:55 +08:00
chenzomi
37cc0b949d change megatron to ascendspeed 2023-06-10 21:26:01 +08:00
fengliangjun
106a415556 inital AscendSpeed 2023-06-09 16:15:23 +08:00
wangyixian
d55d341fe1 Adapt the bloom 7.1b model to the AscendSpeed framework, which is jointly completed by liulinfeng and wangyixian 2023-06-06 22:30:19 +08:00
chenzomi
ce6af59f73 remove unused paraemter and models. 2023-05-26 10:53:07 +08:00
chenzomi
e4a120a662 fork megatron-deepspeed code. 2023-05-25 14:49:59 +08:00