🏆 #3,922 overall#854 of 1,378 in Language Models

guolinke /TUPE

Transformer with Untied Positional Encoding (TUPE). Code of paper "Rethinking Positional Encoding in Language Pre-training". Improve existing models like BERT.

$ git clone https://github.com/guolinke/TUPE.git
GitHub social preview for guolinke/TUPE
Stars
250
250
This week
0 ★
0 this month
Forks
27
27
Language
Python
License
MIT
Created
Jun 24, 2020
6.4 years old
Last push
Nov 8, 2021

Stars, last 30 days

Oct 15, 2026 → Nov 16, 2026
★ 250▲ +0★ 250

Categories

GitHub topics

More in Language Models

Language Models

codertimo/BERT-pytorch

▲ +1 ★ this week

Google AI 2018 BERT pytorch implementation

🧠 Natural Language Processing
PythonApache-2.0updated Sep 15, 2023
GitHub ↗★ 6.5K⑂ 1.3K
Language Models

extreme-bert/extreme-bert

ExtremeBERT is a toolkit that accelerates the pretraining of customized language models on customized datasets, described in the paper “ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT”.

🧠 Natural Language Processing
PythonApache-2.0updated Mar 5, 2023
GitHub ↗★ 268⑂ 14
Language Models

alisafaya/Arabic-BERT

Arabic edition of BERT pretrained language models

🧠 Natural Language Processing
OtherMITupdated Dec 5, 2020
GitHub ↗★ 133⑂ 20
Language Models

michiyasunaga/LinkBERT

▲ +1 ★ this week

[ACL 2022] LinkBERT: A Knowledgeable Language Model 😎 Pretrained with Document Links

❓ Question Answering
PythonApache-2.0updated Apr 5, 2022
GitHub ↗★ 462⑂ 43
Language Models

michiyasunaga/dragon

[NeurIPS 2022] DRAGON 🐲: Deep Bidirectional Language-Knowledge Graph Pretraining

❓ Question Answering
PythonApache-2.0updated May 10, 2023
GitHub ↗★ 336⑂ 46

Data from GitHub · snapshot Nov 16, 2026