Back to Best

Best list

Best Small Models for Consumer GPUs

Find small open models for consumer GPUs and local AI workflows, including Phi, Gemma, Mistral, Qwen, and embedding/reranking models.

Disclosure: Some links may be affiliate links. We may earn a commission if you buy through them, at no extra cost to you.

Top picks

  1. Phi-4 Mini
  2. Mistral Small 3.1
  3. Gemma 3 27B
  4. Multilingual E5 Large
  5. BGE Reranker v2 M3

Grouped recommendations

Best tiny/edge candidate

Phi-4 Mini

Best medium chat candidates

Mistral Small 3.1, Gemma 3 27B

Best retrieval helpers

Multilingual E5 Large, BGE Reranker v2 M3

How to choose

For consumer GPUs, start smaller than you think. Reliable speed beats a huge model that barely fits.

Related links

/models/phi-4-mini//models/mistral-small-3-1//models/gemma-3-27b//guides/how-much-vram-local-ai/

Sources

Microsoft PhiMistral AI

Sponsorship note

Built an AI tool or open-source project? Submit it for review or sponsor a featured placement on OpenSourcesAI.

Sponsor or submit