| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/text_generation/beam_utils.py |
megatron/text_generation/beam_utils.py:9 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/text_generation/sampling.py |
megatron/text_generation/sampling.py:5 |
https://github.com/ari-holtzman/degen/blob/master/gen.py |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/text_generation/sampling.py |
megatron/text_generation/sampling.py:6 |
https://huggingface.co/transformers/_modules/transformers/generation_logits_process.html |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/text_generation/sampling.py |
megatron/text_generation/sampling.py:33 |
https://github.com/ari-holtzman/degen/blob/master/gen.py |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/theoretical_memory_usage.py |
megatron/theoretical_memory_usage.py:73 |
https://arxiv.org/pdf/2205.05198.pdf |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/fused_kernels/compat.h |
megatron/fused_kernels/compat.h:4 |
https://github.com/NVIDIA/apex |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/static/index.html |
megatron/static/index.html:86 |
https://cdnjs.cloudflare.com/ajax/libs/jquery/3.5.1/jquery.min.js |
前端代码库地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/datasets/blended_megatron_dataset_config.py |
megatron/core/datasets/blended_megatron_dataset_config.py:66 |
https://docs.python.org/3/library/dataclasses.html#post-init-processing |
详情地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/models/retro/encoder_attention.py |
megatron/core/models/retro/encoder_attention.py:22 |
https://arxiv.org/abs/2112.04426 |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/models/retro/decoder_attention.py |
megatron/core/models/retro/decoder_attention.py:27 |
https://arxiv.org/abs/2112.04426 |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/models/common/embeddings/rotary_pos_embedding.py |
megatron/core/models/common/embeddings/rotary_pos_embedding.py:147 |
https://kexue.fm/archives/8265 |
详情地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/pipeline_parallel/schedules.py |
megatron/core/pipeline_parallel/schedules.py:493 |
https://arxiv.org/pdf/2205.05198.pdf |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/transformer/transformer_config.py |
megatron/core/transformer/transformer_config.py:87 |
https://arxiv.org/abs/2205.05198 |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/transformer/transformer_config.py |
megatron/core/transformer/transformer_config.py:107 |
https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/api/common.html |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/transformer/mlp.py |
megatron/core/transformer/mlp.py:49 |
https://arxiv.org/pdf/2002.05202.pdf |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/transformer/transformer_config.py |
megatron/core/transformer/transformer_config.py:196 |
https://docs.python.org/3/library/dataclasses.html#post-init-processing |
详情地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/transformer/dot_product_attention.py |
megatron/core/transformer/dot_product_attention.py:23 |
https://arxiv.org/abs/2205.05198 |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/pipeline_parallel/schedules.py |
megatron/core/pipeline_parallel/schedules.py:1122 |
https://arxiv.org/pdf/2205.05198.pdf |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/model_parallel_config.py |
megatron/core/model_parallel_config.py:26 |
https://arxiv.org/pdf/2104.04473.pdf |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/model_parallel_config.py |
megatron/core/model_parallel_config.py:31 |
https://arxiv.org/abs/2205.05198 |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/tensor_parallel/random.py |
megatron/core/tensor_parallel/random.py:4 |
https://github.com/pytorch/pytorch |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/tensor_parallel/layers.py |
megatron/core/tensor_parallel/layers.py:4 |
https://github.com/pytorch/pytorch |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/model_parallel_config.py |
megatron/core/model_parallel_config.py:200 |
https://docs.python.org/3/library/dataclasses.html#post-init-processing |
详情地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/tensor_parallel/cross_entropy.py |
megatron/core/tensor_parallel/cross_entropy.py:80 |
https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/common/losses/smoothed_cross_entropy.py |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/tensor_parallel/layers.py |
megatron/core/tensor_parallel/layers.py:370 |
c47cf9bc7f/torch/_refs/init.py |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/package_info.py |
megatron/core/package_info.py:17 |
nemo-toolkit@nvidia.com |
源码作者邮箱地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/package_info.py |
megatron/core/package_info.py:19 |
https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/ |
详情地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/package_info.py |
megatron/core/package_info.py:21 |
https://github.com/NVIDIA/Megatron-LM/megatron/core |
源码仓库地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/package_info.py |
megatron/core/package_info.py:22 |
https://github.com/NVIDIA/Megatron-LM/releases |
源码仓库地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/tokenizer/bert_tokenization.py |
megatron/tokenizer/bert_tokenization.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/tokenizer/gpt2_tokenization.py |
megatron/tokenizer/gpt2_tokenization.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/tokenizer/gpt2_tokenization.py |
megatron/tokenizer/gpt2_tokenization.py:41 |
https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-vocab.json |
预训练文件地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/tokenizer/gpt2_tokenization.py |
megatron/tokenizer/gpt2_tokenization.py:44 |
https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-merges.txt |
预训练文件地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/utils.py |
megatron/utils.py:69 |
https://github.com/NVIDIA/apex |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/tokenizer/bert_tokenization.py |
megatron/tokenizer/bert_tokenization.py:299 |
https://en.wikipedia.org/wiki/CJK_Unified_Ideographs_(Unicode_block) |
详情地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/tokenizer/tokenizer.py |
megatron/tokenizer/tokenizer.py:389 |
c8fa217e81/nemo/collections/common/tokenizers/sentencepiece_tokenizer.py |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/tokenizer/tokenizer.py |
megatron/tokenizer/tokenizer.py:415 |
c8fa217e81/nemo/collections/common/tokenizers/sentencepiece_tokenizer.py |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/model/fused_layer_norm.py |
megatron/model/fused_layer_norm.py:4 |
https://github.com/NVIDIA/apex |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/model/fused_layer_norm.py |
megatron/model/fused_layer_norm.py:83 |
https://github.com/NVIDIA/apex |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/model/vision/esvit_swin_backbone.py |
megatron/model/vision/esvit_swin_backbone.py:6 |
chunyl@microsoft.com |
源码作者邮箱地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/model/vision/esvit_swin_backbone.py |
megatron/model/vision/esvit_swin_backbone.py:509 |
https://arxiv.org/pdf/2103.14030 |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/model/vision/dino.py |
megatron/model/vision/dino.py:6 |
https://github.com/facebookresearch/dino/blob/main/main_dino.py |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/model/vision/knn_monitor.py |
megatron/model/vision/knn_monitor.py:101 |
https://arxiv.org/abs/1805.01978 |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/model/vision/knn_monitor.py |
megatron/model/vision/knn_monitor.py:102 |
http://github.com/zhirongw/lemniscate.pytorch |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/model/vision/knn_monitor.py |
megatron/model/vision/knn_monitor.py:103 |
https://github.com/leftthomas/SimCLR |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/model/vision/swin_backbone.py |
megatron/model/vision/swin_backbone.py:474 |
https://arxiv.org/pdf/2103.14030 |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/model/transformer.py |
megatron/model/transformer.py:96 |
https://arxiv.org/pdf/2002.05202.pdf |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/model/language_model.py |
megatron/model/language_model.py:377 |
https://github.com/kingoflolz/mesh-transformer-jax/ |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/optimizer_param_scheduler.py |
megatron/optimizer_param_scheduler.py:81 |
https://openreview.net/pdf?id=BJYwwY9ll |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/data/autoaugment.py |
megatron/data/autoaugment.py:29 |
https://github.com/DeepVoltaire/AutoAugment |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/data/autoaugment.py |
megatron/data/autoaugment.py:36 |
https://arxiv.org/abs/1805.09501 |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/data/dataset_utils.py |
megatron/data/dataset_utils.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/data/dataset_utils.py |
megatron/data/dataset_utils.py:18 |
https://github.com/google-research/albert/blob/master/create_pretraining_data.py |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/arguments.py |
megatron/arguments.py:889 |
https://arxiv.org/abs/2205.14135 |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/data/dataset_utils.py |
megatron/data/dataset_utils.py:265 |
https://arxiv.org/pdf/1907.10529.pdf |
参考论文地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/data/image_folder.py |
megatron/data/image_folder.py:32 |
https://github.com/pytorch/vision/blob/main/torchvision/datasets/folder.py |
参考代码地址 |
| 开源代码引入 |
https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/data/image_folder.py |
megatron/data/image_folder.py:238 |
https://github.com/python-pillow/Pillow/issues/835 |
详情地址 |
| 开源代码引入 |
不涉及 |
tests/ut/module/test_fold_schedules.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tests/ut/module/test_auto_recomputing.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tests/ut/module/test_triangle_attn.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tests/st/test_bloom/run_bloom_ptd.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tests/st/test_bloom/run_llama_ptd.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tests/st/test_bloom/run_gpt_ptd.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
setup.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
setup.py:85 |
https://packaging.python.org/en/latest/single_source_version.html |
详情地址 |
| 开源代码引入 |
不涉及 |
tools/retro/utils.py:6 |
https://github.com/NVIDIA/Megatron-LM/blob/main/tools/retro/utils.py |
源代码地址 |
| 开源代码引入 |
不涉及 |
tools/preprocess_data.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tools/checkpoint/saver_megatron.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tools/checkpoint/util.py:8 |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tools/checkpoint/loader_llama2_hf.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
modellink/text_generation/beam_utils.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
modellink/text_generation/utils.py |
https://medium.com/huggingface/how-to-build-a-state-of-the-art-conversational-ai-with-transfer |
源代码地址 |
| 开源代码引入 |
不涉及 |
modellink/init.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
modellink/adapter_lora/init.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
modellink/tokenizer/init.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
modellink/tokenizer/tokenizer.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
modellink/utils.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
modellink/model/module.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
modellink/error_utils.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
modellink/data/prompter.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
modellink/data/data_handler.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tasks/evaluation/eval_api/chat.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tasks/evaluation/eval_api/dataset_eval.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tasks/evaluation/evaluation_llama.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tasks/evaluation/eval_impl/agi_eval.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tasks/evaluation/eval_impl/bbh_eval.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tasks/evaluation/eval_impl/boolq_eval.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tasks/evaluation/eval_impl/ceval_exam.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tasks/evaluation/eval_impl/gsm8k_eval.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tasks/evaluation/eval_impl/human_eval.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tasks/evaluation/eval_impl/mmlu_eval.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
| 开源代码引入 |
不涉及 |
tasks/evaluation/eval_impl/template.py |
http://www.apache.org/licenses/LICENSE-2.0 |
License地址 |
