Hugging Face
Access 500,000+ pre-trained models via the Hub, run inference with Transformers pipelines, and fine-tune or deploy models with Inference Endpoints.
skill: Hugging Face category: AI/ML tags: [Hugging Face, Transformers, Model Hub, Python, NLP] readingTime: 6
Hugging Face is the default platform for open-source AI - use it when you need model transparency, privacy, customization, or want to avoid API costs at scale. For production serving of large models without managing infrastructure, Hugging Face Inference Endpoints are easier than building your own GPU cluster.
Workflow
- Run a model with pipeline()pipeline() is the highest-level API - it handles tokenization, inference, and post-processing in one call.
- Browse and download models from the HubThe Hub has 500K+ models across every task - search by task, dataset, license, and benchmark scores.
- Load a model and tokenizer directlyFor fine-grained control - custom generation config, device placement, quantization - load the model and tokenizer separately.
- Fine-tune with TrainerThe Trainer API handles training loops, gradient accumulation, mixed precision, evaluation, and checkpointing - removing ~200 lines of boilerplate.
- Deploy with Inference EndpointsInference Endpoints provision a dedicated GPU container for your model on AWS/Azure/GCP - autoscaling, private, zero infrastructure management.
Output Format
Hugging Face Transformers pipeline or model loaded locally, fine-tuned with Trainer, deployed on Inference Endpoints.
from transformers import pipeline
pipe = pipeline('task', model='org/model-name')
result = pipe('input text')Examples
Input
Output
Edge Cases & Gotchas
- Gated models require access request - Llama 3, Gemma, and other models require accepting a license on the Hub page. Access is granted within minutes but the huggingface-cli login token must match your account.
- Model card evaluation != your use case - Benchmark scores on a model card reflect the evaluation dataset, not your task. Always run the model on your own examples before committing to it.
- Disk space - 7B models are 14GB in fp16; 70B models are 140GB. Check available disk space before downloading. Use TRANSFORMERS_CACHE to specify a location with sufficient space.
- Tokenizer chat template - Each instruction-tuned model has its own chat template (Llama uses [INST], Mistral uses different tokens). Use tokenizer.apply_chat_template() - don't hardcode.
When to Break These Rules
- Use the OpenAI API when you don't want to manage GPU infrastructure and the task doesn't require open-source models.
- Use Ollama for local development - it handles model downloading and serving with a simple API, without needing to write Transformers code.
Reference Files
https://huggingface.co/docs/transformershttps://huggingface.co/docs/hub