Transformers Library
The standard library from Hugging Face for accessing and fine-tuning Transformer models like BERT and GPT.
Transformers API (by Hugging Face)
The transformers Python library has become the undisputed global standard mechanism for implementing Deep Learning into NLP. It completely abstracts away the terrifying calculus equations of attention matrices, offering thousands of diverse model architectures through unified Python classes.
Level 1 — The 'Pipeline' Abstraction
The pipeline() object is the fastest way to use a pre-trained model. By simply telling the function what task you want to do (e.g., "sentiment-analysis", "summarization", "translation", "ner", "question-answering"), the library automatically contacts the Hugging Face hub, downloads an appropriate default tokenizer, downloads standard weights, and handles the tensor math—all in the background.
from transformers import pipeline
# 1. Create an English-to-French Translation Pipeline
translator = pipeline("translation_en_to_fr")
result1 = translator("The quick brown fox jumps over the lazy dog.")
print("Translation:", result1[0]['translation_text'])
# Le rapide renard brun saute sur le chien paresseux.
# 2. Change the underlying default model dynamically
# Here we force a specific tiny model called 'bert-tiny' fine-tuned for emotion
classifier = pipeline("text-classification", model="bhadresh-savani/distilbert-base-uncased-emotion")
result2 = classifier("I am so frustrated that the code threw an error again!")
print("Classifier Output:", result2)
# [{'label': 'anger', 'score': 0.9921}]
Level 2 — Peeking under the Hood: Tokenizers and Models
If you want to train (fine-tune) a model, or understand how the pipeline works, you must separate it into its two massive components: the Tokenizer (which turns text strings into math IDs) and the Model (which processes those IDs).
PyTorch vs TensorFlow
Hugging Face uniquely operates across BOTH deep learning frameworks! A model starting with AutoModel... executes in PyTorch. A model starting with TFAutoModel... executes natively in TensorFlow/Keras!
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_name = "distilbert-base-uncased-finetuned-sst-2-english"
# Download the Tokenizer associated with the DistilBERT checkpoint
tokenizer = AutoTokenizer.from_pretrained(model_name)
# Download the actual Neural Network Architecture and Weights
# The "ForSequenceClassification" explicitly attaches a linear classification head onto the transformer body
model = AutoModelForSequenceClassification.from_pretrained(model_name)
# 1. Tokenize Input Data
text = "The graphics are stunning but the storyline is incredibly weak."
# return_tensors='pt' = Return PyTorch Tensors. (Use 'tf' for TensorFlow)
inputs = tokenizer(text, return_tensors="pt")
print("Input IDs (Math):", inputs['input_ids'])
# tensor([[ 101, 1996, 5143, 2024, ... 102]]) # Note the [CLS] and [SEP] tokens!
# 2. Feed-Forward Neural Network Pass (No Gradient Tracking needed for Inference)
with torch.no_grad():
outputs = model(**inputs)
# 3. Analyze Output Tensors
# The model outputs un-normalized raw scores (Logits)
logits = outputs.logits
print("Raw Logits:", logits)
# Convert logits to percentages using Softmax
probabilities = torch.nn.functional.softmax(logits, dim=-1)
predicted_class = torch.argmax(probabilities).item()
# Assuming LABEL_0 is Negative and LABEL_1 is Positive
print(f"Prediction Class: {predicted_class}") # 0