ToshLLM
Run local LLMs on Intel Macs with AMD GPUs
Free LLM ServerChat Open Source
Overview
ToshLLM is a native macOS app that runs large language models entirely on-device on Intel Macs equipped with discrete AMD GPUs, filling a gap left by tools that only target Apple Silicon. It offers a native SwiftUI chat interface with persistent conversations and full Markdown, a model manager with per-hardware VRAM/RAM estimates, and an OpenAI-compatible API endpoint at localhost:8080 for connecting external clients. Under the hood it manages llama-server as a child process with AMD-specific optimizations, delivering dramatic speedups (e.g. Qwen3-8B from under 3 t/s to ~57 t/s). No accounts, no cloud, no per-token fees.
Minimum macOS: 14.0 (Sonoma)
Architecture: Intel
Key Features
- Native SwiftUI chat interface with persistent conversations and Markdown
- Local inference on Intel Macs with AMD GPUs via Metal
- Model manager with curated catalog and per-hardware VRAM/RAM estimates
- OpenAI-compatible API endpoint at localhost:8080
- Dual inference engines: official llama.cpp and experimental TurboQuant
- MTP speculative decoding for ~34% faster generation
- Automatic --n-cpu-moe calculation for Mixture-of-Experts models
- Built-in benchmarking with performance comparison charts
- Multiple independent server instances
- 100% offline with no accounts or per-token fees
