Model field guide
Qwen2.5-3B-Instruct
42 of 44 hardware configurations have a supported memory fit at one or more recorded quantizations. Runtime compatibility still needs verification.
Total parameters3.09B
Active parameters3.09B
Recorded context limit32,768
Architecturedense
Verified 2026-10-22 · license metadata: other · commercial use: unknown (review the terms).
Actual files, not a size shortcut.
| Quantization | Total file size | Source |
|---|---|---|
| Q4_K_M | 1.960 GiB2,104,932,768 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| Q5_K_M | 2.271 GiB2,438,740,384 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| Q8_0 | 3.368 GiB3,616,088,480 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| FP16 | 6.334 GiB6,800,646,784 bytes; 2 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
The hardware picture.
Q4_K_M: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 3.10 GiB | fits | 29.1–58.1 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 3.10 GiB | fits | 32.3–64.6 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 3.10 GiB | fits | 32.3–64.6 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 3.10 GiB | fits | 58.3–116.7 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 3.10 GiB | fits | 58.3–116.7 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 3.10 GiB | fits | 58.3–116.7 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 3.10 GiB | fits | 87.4–174.8 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 3.10 GiB | fits | 116.7–233.4 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 3.10 GiB | fits | 116.7–233.4 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 3.10 GiB | fits | 228.0–456.1 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 3.10 GiB | fits | 228.0–456.1 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 3.10 GiB | fits | 228.0–456.1 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 3.10 GiB | fits | 29.1–58.1 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 3.10 GiB | fits | 58.3–116.7 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 3.10 GiB | fits | 116.7–233.4 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 3.10 GiB | fits | 22.8–45.6 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 3.10 GiB | fits | 22.8–45.6 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 3.10 GiB | fits | 22.8–45.6 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 3.10 GiB | fits | 51.9–103.8 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 3.10 GiB | fits | 51.9–103.8 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 3.10 GiB | fits | 51.9–103.8 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 3.10 GiB | fits | 77.9–155.8 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 3.10 GiB | fits | 103.8–207.5 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 3.10 GiB | fits | 103.8–207.5 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 3.10 GiB | fits | 155.6–311.3 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 3.10 GiB | fits | 155.6–311.3 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 3.10 GiB | fits | 68.4–136.8 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 3.10 GiB | fits | 177.9–355.7 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 3.10 GiB | fits | 54.7–109.5 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 3.10 GiB | fits | 95.8–191.6 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 3.10 GiB | fits | 136.2–272.4 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 3.10 GiB | fits | 191.6–383.1 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 3.10 GiB | fits | 85.1–170.3 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 3.10 GiB | fits | 127.7–255.4 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 3.10 GiB | fits | 170.3–340.5 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 3.10 GiB | fits | 182.4–364.9 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 3.10 GiB | fits | 340.5–681.1 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 3.10 GiB | fits | 340.5–681.1 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 3.10 GiB | fits | 182.4–364.9 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 3.10 GiB | fits | 86.7–173.3 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 3.10 GiB | fits | 48.6–97.3 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 3.10 GiB | fits | 51.9–103.8 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 3.10 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 3.10 GiB | Needs verification | Estimated speed unavailable |
Q5_K_M: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 3.41 GiB | fits | 25.1–50.2 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 3.41 GiB | fits | 27.9–55.8 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 3.41 GiB | fits | 27.9–55.8 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 3.41 GiB | fits | 50.4–100.7 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 3.41 GiB | fits | 50.4–100.7 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 3.41 GiB | fits | 50.4–100.7 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 3.41 GiB | fits | 75.4–150.9 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 3.41 GiB | fits | 100.7–201.4 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 3.41 GiB | fits | 100.7–201.4 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 3.41 GiB | fits | 196.8–393.6 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 3.41 GiB | fits | 196.8–393.6 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 3.41 GiB | fits | 196.8–393.6 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 3.41 GiB | fits | 25.1–50.2 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 3.41 GiB | fits | 50.4–100.7 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 3.41 GiB | fits | 100.7–201.4 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 3.41 GiB | fits | 19.7–39.4 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 3.41 GiB | fits | 19.7–39.4 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 3.41 GiB | fits | 19.7–39.4 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 3.41 GiB | fits | 44.8–89.6 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 3.41 GiB | fits | 44.8–89.6 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 3.41 GiB | fits | 44.8–89.6 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 3.41 GiB | fits | 67.2–134.5 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 3.41 GiB | fits | 89.6–179.1 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 3.41 GiB | fits | 89.6–179.1 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 3.41 GiB | fits | 134.3–268.7 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 3.41 GiB | fits | 134.3–268.7 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 3.41 GiB | fits | 59.0–118.1 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 3.41 GiB | fits | 153.5–307.0 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 3.41 GiB | fits | 47.2–94.5 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 3.41 GiB | fits | 82.7–165.3 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 3.41 GiB | fits | 117.6–235.1 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 3.41 GiB | fits | 165.3–330.7 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 3.41 GiB | fits | 73.5–147.0 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 3.41 GiB | fits | 110.2–220.4 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 3.41 GiB | fits | 147.0–293.9 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 3.41 GiB | fits | 157.5–314.9 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 3.41 GiB | fits | 293.9–587.8 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 3.41 GiB | fits | 293.9–587.8 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 3.41 GiB | fits | 157.5–314.9 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 3.41 GiB | fits | 74.8–149.6 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 3.41 GiB | fits | 42.0–84.0 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 3.41 GiB | fits | 44.8–89.6 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 3.41 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 3.41 GiB | Needs verification | Estimated speed unavailable |
Q8_0: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 4.51 GiB | fits | 16.9–33.8 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 4.51 GiB | fits | 18.8–37.6 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 4.51 GiB | fits | 18.8–37.6 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 4.51 GiB | fits | 34.0–67.9 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 4.51 GiB | fits | 34.0–67.9 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 4.51 GiB | fits | 34.0–67.9 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 4.51 GiB | fits | 50.9–101.8 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 4.51 GiB | fits | 67.9–135.8 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 4.51 GiB | fits | 67.9–135.8 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 4.51 GiB | fits | 132.7–265.5 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 4.51 GiB | fits | 132.7–265.5 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 4.51 GiB | fits | 132.7–265.5 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 4.51 GiB | fits | 16.9–33.8 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 4.51 GiB | fits | 34.0–67.9 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 4.51 GiB | fits | 67.9–135.8 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 4.51 GiB | fits | 13.3–26.5 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 4.51 GiB | fits | 13.3–26.5 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 4.51 GiB | fits | 13.3–26.5 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 4.51 GiB | fits | 30.2–60.4 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 4.51 GiB | fits | 30.2–60.4 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 4.51 GiB | fits | 30.2–60.4 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 4.51 GiB | fits | 45.4–90.7 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 4.51 GiB | fits | 60.4–120.8 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 4.51 GiB | fits | 60.4–120.8 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 4.51 GiB | fits | 90.6–181.2 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 4.51 GiB | fits | 90.6–181.2 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 4.51 GiB | fits | 39.8–79.6 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 4.51 GiB | fits | 103.5–207.1 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 4.51 GiB | fits | 31.9–63.7 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 4.51 GiB | fits | 55.8–111.5 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 4.51 GiB | fits | 79.3–158.6 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 4.51 GiB | fits | 111.5–223.0 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 4.51 GiB | fits | 49.6–99.1 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 4.51 GiB | fits | 74.3–148.7 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 4.51 GiB | fits | 99.1–198.2 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 4.51 GiB | fits | 106.2–212.4 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 4.51 GiB | fits | 198.2–396.5 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 4.51 GiB | fits | 198.2–396.5 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 4.51 GiB | fits | 106.2–212.4 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 4.51 GiB | fits | 50.4–100.9 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 4.51 GiB | fits | 28.3–56.6 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 4.51 GiB | fits | 30.2–60.4 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 4.51 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 4.51 GiB | Needs verification | Estimated speed unavailable |
FP16: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 7.47 GiB | fits | 9.0–18.0 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 7.47 GiB | fits | 10.0–20.0 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 7.47 GiB | fits | 10.0–20.0 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 7.47 GiB | fits | 18.1–36.1 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 7.47 GiB | fits | 18.1–36.1 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 7.47 GiB | fits | 18.1–36.1 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 7.47 GiB | fits | 27.1–54.1 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 7.47 GiB | fits | 36.1–72.2 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 7.47 GiB | fits | 36.1–72.2 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 7.47 GiB | fits | 70.6–141.2 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 7.47 GiB | fits | 70.6–141.2 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 7.47 GiB | fits | 70.6–141.2 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 7.47 GiB | fits | 9.0–18.0 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 7.47 GiB | fits | 18.1–36.1 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 7.47 GiB | fits | 36.1–72.2 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 7.47 GiB | fits | 7.1–14.1 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 7.47 GiB | fits | 7.1–14.1 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 7.47 GiB | fits | 7.1–14.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 7.47 GiB | fits | 16.1–32.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 7.47 GiB | fits | 16.1–32.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 7.47 GiB | fits | 16.1–32.1 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 7.47 GiB | fits | 24.1–48.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 7.47 GiB | fits | 32.1–64.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 7.47 GiB | fits | 32.1–64.2 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 7.47 GiB | fits | 48.2–96.3 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 7.47 GiB | fits | 48.2–96.3 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 7.47 GiB | fits | 21.2–42.3 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 7.47 GiB | fits | 55.1–110.1 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 7.47 GiB | fits | 16.9–33.9 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 7.47 GiB | fits | 29.6–59.3 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 7.47 GiB | fits | 42.2–84.3 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 7.47 GiB | fits | 59.3–118.6 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 7.47 GiB | fits | 26.4–52.7 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 7.47 GiB | fits | 39.5–79.1 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 7.47 GiB | fits | 52.7–105.4 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 7.47 GiB | fits | 56.5–112.9 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 7.47 GiB | fits | 105.4–210.8 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 7.47 GiB | fits | 105.4–210.8 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 7.47 GiB | fits | 56.5–112.9 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 7.47 GiB | fits | 26.8–53.6 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 7.47 GiB | fits | 15.1–30.1 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 7.47 GiB | fits | 16.1–32.1 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 7.47 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 7.47 GiB | Needs verification | Estimated speed unavailable |
A “fits” result checks estimated memory only. “Needs verification” preserves missing cache, vision, or placement information. All speed figures are estimated from the published bandwidth formula.
Field-level sources and verification dates
- id: Official source · verified 2026-10-22. Lowercase URL slug derived from the upstream repository ID; baseModel retains the exact Hugging Face identifier.
- name: Official source · verified 2026-10-22
- baseModel: Official source · verified 2026-10-22
- downloads: Official source · verified 2026-10-22
- tasks: Official source · verified 2026-10-22. Chat denotes a text-generating model, including base models that may require a chat template or instruction tuning. Code label is inferred from the repository model-family name. Vision uses pipeline metadata or vision_config.
- license: Official source · verified 2026-10-22
- commercialUse: Official source · verified 2026-10-22. Unknown or custom license, conflicting metadata, or unverified publisher. Review the linked license and applicable terms.
- license_url: Official source · verified 2026-10-22. Pinned license file confirmed in repository listing.
- architecture: Official source · verified 2026-10-22. Classification from model_type and expert/state-space configuration fields (qwen2); hybrid classification takes precedence over expert count.
- parametersB: Official source · verified 2026-10-22. Exact safetensors parameter total reported by the Hugging Face API, divided by 1e9.
- activeParametersB: Official source · verified 2026-10-22. Dense model: all parameters are included. This is not a runtime FLOP measurement.
- contextLength: Official source · verified 2026-10-22. Configured maximum token positions; not a guarantee of runtime support or quality at that length. No speculative RoPE scaling applied.
- kvCache: Official source · verified 2026-10-22. Standard attention KV shape from num_hidden_layers, num_key_value_heads, and head_dim or hidden_size / num_attention_heads; FP16 cache bytes are a separate estimator choice.