TibyrIA v2.1

1. BERT (Google, 2018)
BERT (Bidirectional Encoder Representations from Transformers) was created by Google as a general-purpose language model, trained on English. It revolutionized natural language processing by understanding bidirectional context, but was developed for English and does not capture Brazilian Portuguese specifics.

2. Tupi-BERT (FpOliveira, Brazil — now a partner of Código Não Binário)
Brazilian researchers adapted BERT for Brazilian Portuguese, creating Tupi-BERT. This model was pre-trained specifically on Brazilian Portuguese and received a "warm-start" in hate speech detection — i.e., it already had some knowledge of hate speech before our fine-tuning. This makes it better suited for content moderation tasks in Portuguese.

3. TibyrIA v2.1 (Código Não Binário)
We perform specialized fine-tuning of Tupi-BERT on our unique anti-LGBTQIA+ hate dataset. We trained the model on 1,891 comments manually annotated by our community, using Focal Loss to handle class imbalance. The result is a model that not only understands Brazilian Portuguese but specifically recognizes the nuances of LGBT+ phobia — transphobia, lesbophobia, homophobia, biphobia — in its intersections with racism, misogyny, and other power structures.

Why this architecture?
Small models (SLMs) like Tupi-BERT (~110M parameters, 12 layers) are better suited for community use: they require less infrastructure, are faster, can run on your own servers, and do not depend on Big Tech. TibyrIA shows that specialized, contextualized models can outperform generic giants on specific tasks.