Taykhoom/ModernGENA-large

This repository contains the model presented in D3LM: A Discrete DNA Diffusion Language Model for Bidirectional DNA Understanding and Generation.

Minimal HuggingFace port of the 6-mer variant of DNABERT -- a BERT-base masked language model pre-trained on the human reference genome using overlapping 6-mer tokenization.

The Nucleotide Transformers are a collection of foundational language models that were pre-trained on DNA sequences from whole-genomes. Compared to other approaches, our models do not only integrate information from single reference genomes, but leverage DNA sequences from over 3,200 diverse human…