Use case ยท Needs verification
Local LLM Tools
Local model runners, inference servers, gateways, and self-hosted chat or agent tools.
Verification status
This page lists recommended resources for this use case and is marked Needs verification. Community contributions and verification status are updated regularly.
Start here
Browse recommended tools and libraries to speed up your build.
Skills
11 mapped skills
Tools
12 candidate tools
Repos
15 GitHub records
Required and useful skills
API Literacy
EngineeringUnderstanding authentication, rate limits, request shapes, errors, and source attribution.
Skill linked from curated resource requirements.
Backend API Design
BackendDesigning HTTP APIs, data contracts, auth boundaries, and error handling.
Skill linked from curated resource requirements.
Command Line
EngineeringUsing terminal commands, package managers, environment variables, and logs.
Skill linked from curated resource requirements.
Data Privacy
GovernanceHandling private code, prompts, logs, documents, and user data safely.
Skill linked from curated resource requirements.
Deployment Basics
OpsShipping static sites, APIs, and background jobs with clear environment boundaries.
Skill linked from curated resource requirements.
Docker
OpsContainerizing and running tools consistently across local and server environments.
Skill linked from curated resource requirements.
IDE Workflow
EngineeringWorking efficiently with editors, extensions, terminals, diffs, and code navigation.
Skill linked from curated resource requirements.
Local Model Ops
AIRunning local models, choosing model sizes, managing inference servers, and monitoring resources.
Skill linked from curated resource requirements.
Prompt Debugging
AIDiagnosing poor AI outputs by adjusting instructions, context, examples, and constraints.
Skill linked from curated resource requirements.
Retrieval-Augmented Generation
AIConnecting LLMs to external knowledge with retrieval, ranking, and grounded responses.
Skill linked from curated resource requirements.
Self-Hosting Basics
OpsRunning open-source tools locally or on controlled infrastructure.
Skill linked from curated resource requirements.
Recommended tools
AnythingLLM
RAG App PlatformsSelf-hosted workspace and RAG interface for documents and LLM workflows.
Curated tool relationship for future one-stop directory pages.
Continue
AI Coding AssistantsOpen-source AI code assistant framework for custom IDE-based coding workflows.
Curated tool relationship for future one-stop directory pages.
Jan
Local LLM ToolsOpen-source desktop AI assistant for local model workflows.
Curated tool relationship for future one-stop directory pages.
LiteLLM
LLM GatewaysOpen-source LLM gateway for routing requests across model providers.
Curated tool relationship for future one-stop directory pages.
llama.cpp
Local LLM ToolsCore local inference project for running LLMs efficiently on local hardware.
Curated tool relationship for future one-stop directory pages.
LM Studio
Local LLM ToolsDesktop tool for running local language models and experimenting with local inference.
Curated tool relationship for future one-stop directory pages.
LocalAI
Local LLM ToolsOpen-source local inference server compatible with OpenAI-style APIs.
Curated tool relationship for future one-stop directory pages.
Open WebUI
Local LLM ToolsSelf-hosted AI chat interface for local and remote models.
Curated tool relationship for future one-stop directory pages.
vLLM
Local LLM ToolsHigh-throughput inference engine for serving large language models.
Curated tool relationship for future one-stop directory pages.
GPT4All
Local LLM ToolsLocal AI model ecosystem and desktop tooling for private model experimentation.
Curated tool relationship for future one-stop directory pages.
LibreChat
Local LLM ToolsOpen-source AI chat platform with multi-provider support.
Curated tool relationship for future one-stop directory pages.
LobeChat
Local LLM ToolsOpen-source AI chat and agent interface for model and plugin workflows.
Curated tool relationship for future one-stop directory pages.
Repository candidates
affaan-m/ECC
JavascriptThe agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
NousResearch/hermes-agent
PythonThe agent that grows with you
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
Significant-Gravitas/AutoGPT
PythonAutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
ollama/ollama
GoGet up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
f/prompts.chat
HTMLf.k.a. Awesome ChatGPT Prompts. Share, discover, and collect prompts from the community. Free and open source โ self-host for your organization with complete privacy.
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
huggingface/transformers
Python๐ค Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
langgenius/dify
TypescriptProduction-ready platform for agentic workflow development.
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
open-webui/open-webui
PythonUser-friendly AI Interface (Supports Ollama, OpenAI API, ...)
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
langchain-ai/langchain
PythonThe agent engineering platform.
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
firecrawl/firecrawl
TypescriptThe API to search, scrape, and interact with the web at scale. ๐ฅ
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
browser-use/browser-use
Python๐ Make websites accessible for AI agents. Automate tasks online with ease.
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
rasbt/LLMs-from-scratch
Jupyter NotebookImplement a ChatGPT-like LLM in PyTorch from scratch, step by step
High-star GitHub discovery seed from query `topic:llm stars:>10000`.
Caveats and failure modes
Resource verification status and scoring are reviewed regularly to maintain directory accuracy.