Use case · Needs verification
Self-Hosted AI Tools
Open-source and locally controllable AI tools for builders who need privacy, portability, or reduced vendor lock-in.
Verification status
This page lists recommended resources for this use case and is marked Needs verification. Community contributions and verification status are updated regularly.
Start here
Browse recommended tools and libraries to speed up your build.
Skills
12 mapped skills
Tools
13 candidate tools
Repos
3 GitHub records
Required and useful skills
API Literacy
EngineeringUnderstanding authentication, rate limits, request shapes, errors, and source attribution.
Skill linked from curated resource requirements.
Command Line
EngineeringUsing terminal commands, package managers, environment variables, and logs.
Skill linked from curated resource requirements.
Data Ingestion
DataLoading, cleaning, chunking, and normalizing documents or structured data.
Skill linked from curated resource requirements.
Data Privacy
GovernanceHandling private code, prompts, logs, documents, and user data safely.
Skill linked from curated resource requirements.
Deployment Basics
OpsShipping static sites, APIs, and background jobs with clear environment boundaries.
Required or useful skill for resources in this use case.
Docker
OpsContainerizing and running tools consistently across local and server environments.
Skill linked from curated resource requirements.
Evaluation
QualityTesting LLM quality, retrieval quality, task completion, and regression behavior.
Skill linked from curated resource requirements.
Local Model Ops
AIRunning local models, choosing model sizes, managing inference servers, and monitoring resources.
Skill linked from curated resource requirements.
Python
ProgrammingGeneral Python programming for automation, data, AI, and backend scripts.
Required or useful skill for resources in this use case.
Retrieval-Augmented Generation
AIConnecting LLMs to external knowledge with retrieval, ranking, and grounded responses.
Required or useful skill for resources in this use case.
Self-Hosting Basics
OpsRunning open-source tools locally or on controlled infrastructure.
Skill linked from curated resource requirements.
Vector Databases
DataIndexing and retrieving embeddings for semantic search and RAG applications.
Required or useful skill for resources in this use case.
Recommended tools
AnythingLLM
RAG App PlatformsSelf-hosted workspace and RAG interface for documents and LLM workflows.
Curated tool relationship for future one-stop directory pages.
Chroma
Vector DatabasesEmbedding database commonly used for local and application-level RAG prototypes.
Seeded from manual curation; metadata enriched where possible.
Jan
Local LLM ToolsOpen-source desktop AI assistant for local model workflows.
Curated tool relationship for future one-stop directory pages.
llama.cpp
Local LLM ToolsCore local inference project for running LLMs efficiently on local hardware.
Curated tool relationship for future one-stop directory pages.
LocalAI
Local LLM ToolsOpen-source local inference server compatible with OpenAI-style APIs.
Curated tool relationship for future one-stop directory pages.
Open WebUI
Local LLM ToolsSelf-hosted AI chat interface for local and remote models.
Curated tool relationship for future one-stop directory pages.
Qdrant
Vector DatabasesVector database for semantic search and retrieval-heavy AI applications.
Seeded from manual curation; metadata enriched where possible.
RAGFlow
RAG App PlatformsOpen-source RAG engine focused on document understanding and retrieval workflows.
Curated tool relationship for future one-stop directory pages.
vLLM
Local LLM ToolsHigh-throughput inference engine for serving large language models.
Curated tool relationship for future one-stop directory pages.
LibreChat
Local LLM ToolsOpen-source AI chat platform with multi-provider support.
Curated tool relationship for future one-stop directory pages.
LobeChat
Local LLM ToolsOpen-source AI chat and agent interface for model and plugin workflows.
Curated tool relationship for future one-stop directory pages.
Milvus
Vector DatabasesOpen-source vector database for large-scale similarity search and AI retrieval systems.
Seeded from manual curation; metadata enriched where possible.
Repository candidates
qdrant/qdrant
RustQdrant - High-performance, massive-scale Vector Database and Vector Search Engine for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/
Official GitHub repository metadata fetched where API allowed.
chroma-core/chroma
RustSearch infrastructure for AI
Official GitHub repository metadata fetched where API allowed.
milvus-io/milvus
GoMilvus is a high-performance, cloud-native vector database built for scalable vector ANN search
Official GitHub repository metadata fetched where API allowed.
Caveats and failure modes
Resource verification status and scoring are reviewed regularly to maintain directory accuracy.