🏆 #5,000 overall#1822 of 2,559 in Natural Language Processing

guotong1988 /BERT-pre-training

multi-gpu pre-training in one machine for BERT without horovod (Data Parallelism)

★ View on GitHub ↗

$ git clone https://github.com/guotong1988/BERT-pre-training.git

GitHub social preview for guotong1988/BERT-pre-training

Stars

171

171

This week

—

Forks

52

52

Language

Python

License

Apache-2.0

Created

Dec 25, 2018

7.9 years old

Last push

Dec 27, 2025

Categories

🧠 Natural Language Processing

GitHub topics

#bert #multi-gpu #nlp #tensorflow

More in Natural Language Processing

Natural Language Processing

graykode/nlp-tutorial

Natural Language Processing Tutorial for Deep Learning Researchers

Jupyter NotebookMITupdated Feb 21, 2024

GitHub ↗★ 14.9K⑂ 3.9K

Natural Language Processing

ymcui/Chinese-BERT-wwm

▲ +1 ★ this week

Pre-Training with Whole Word Masking for Chinese BERT（中文BERT-wwm系列模型）

PythonApache-2.0updated Apr 19, 2026

GitHub ↗★ 10.2K⑂ 1.4K

Natural Language Processing

imcaspar/gpt2-ml

GPT2 for Multiple Languages, including pretrained models. GPT2 多语言支持, 15亿参数中文预训练模型

🤖 Language Models

PythonApache-2.0updated May 22, 2023

GitHub ↗★ 1.7K⑂ 323

Natural Language Processing

ymcui/Chinese-ELECTRA

Pre-trained Chinese ELECTRA（中文ELECTRA预训练模型）

🤖 Language Models

PythonApache-2.0updated Apr 19, 2026

GitHub ↗★ 1.4K⑂ 166

Natural Language Processing

ymcui/MacBERT

▲ +1 ★ this week

Revisiting Pre-trained Models for Chinese Natural Language Processing (MacBERT)

🤖 Language Models

OtherApache-2.0updated Apr 19, 2026

GitHub ↗★ 728⑂ 61

Natural Language Processing

re-search/DocProduct

Medical Q&A with Deep Language Models

Jupyter NotebookMITupdated Mar 25, 2023

GitHub ↗★ 574⑂ 152

See all 2,559 in Natural Language Processing →

Data from GitHub · snapshot Nov 13, 2026