AgentToolDrop

61 of 71

LLM

Runs prompts against remote and local LLMs from the command line or Python, logging calls to SQLite

Install with pip

pip install llm

Copied from the project's docs at github.com/simonw/llm

Why you'd want it

LLM is Simon Willison's CLI tool and Python library for OpenAI, Claude, Gemini and dozens of other models, via remote APIs or models installed on your own machine. Plugins such as llm-anthropic, llm-gemini and llm-ollama add providers, and llm openai endpoint targets any OpenAI-compatible Chat Completions server without configuration. Prompts and responses are stored in SQLite, and it also generates embeddings, extracts structured content with schemas and lets models execute tools.

Best for

  • Scripting LLM calls in the shell
  • Comparing models from one CLI
  • Keeping a searchable log of prompts

Use it when

You want quick model access from the terminal or a small Python library, with every prompt and response logged locally.

Skip it if

You need a shared gateway with keys and spend tracking for a team; this is a CLI and Python library. Providers other than OpenAI need a plugin installed.

Related tools

Same category or shared tags, by drop score.

  1. Rank 6

    Ollama

    Model/API

    Runs open models locally behind a CLI and a REST API, with an OpenAI-compatible endpoint

    • score 83
    • 183k stars
    • pushed
    • v0.40.4 on
    • MIT

    Install on macOS or Linux

    curl -fsSL https://ollama.com/install.sh | sh
    Use it when, skip it if, stats for Ollama

    Why you'd want it

    Ollama downloads and runs open models such as Gemma with one command (ollama run gemma4) and serves them on a local REST API at localhost:11434. It supports a subset of the OpenAI API, so OpenAI clients work by changing the base URL, and it ships official Python and JavaScript libraries. ollama launch connects local models to coding agents including Claude Code, Codex, Copilot CLI and OpenCode, and the same API can reach Ollama's cloud models with an API key.

    Best for

    • Running open models on your own machine
    • Offline or private agent development
    • Pointing coding agents at local models

    Use it when

    You want a local model server that existing OpenAI-style clients and coding agents can talk to with minimal setup.

    Skip it if

    You only use hosted models from providers like Anthropic or OpenAI. Its OpenAI compatibility covers a subset of the OpenAI API, so check the docs if you depend on less common endpoints.

    Stats

    Stars
    183,183
    Last push
    Release
    v0.40.4,
    License
    MIT
    Pricing
    Open source
    Works with
    macOS, Linux, Windows, Docker, Claude Code, Codex
    Drop score
    83 of 100

    Full stat block for Ollama

  2. Rank 12

    LiteLLM

    Model/API

    Calls 100+ LLM providers in the OpenAI format, as a Python SDK or a self-hosted AI gateway

    • score 77
    • 61.3k stars
    • pushed
    • v1.105.1 on
    • MIT

    Install with uv

    uv add litellm
    Use it when, skip it if, stats for LiteLLM

    Why you'd want it

    LiteLLM gives one interface to OpenAI, Anthropic, Gemini, Bedrock, Azure and many other providers using the OpenAI request format, so you can swap providers without rewriting code. The Python SDK includes a Router with retry and fallback logic across deployments. The proxy server adds virtual keys, per-project spend tracking, guardrails, load balancing and an admin dashboard, and it can also act as an MCP gateway and an A2A agent gateway.

    Best for

    • Platform teams centralizing LLM access
    • Multi-provider apps with fallbacks
    • Tracking spend per project or key

    Use it when

    You call several model providers and want one OpenAI-format API, or a central gateway with keys, budgets and logging for a team.

    Skip it if

    You use a single provider and its own SDK covers your needs. Features such as SSO sit under the commercial enterprise license.

    Stats

    Stars
    61,264
    Last push
    Release
    v1.105.1,
    License
    MIT
    Pricing
    Open source
    Works with
    Docker
    Drop score
    77 of 100

    Full stat block for LiteLLM

  3. Rank 31

    llama.cpp

    Model/API

    Runs LLM and VLM inference locally in plain C/C++, with a CLI and an OpenAI-compatible server

    • score 61
    • 131k stars
    • pushed
    • v0.6.0 on
    • MIT

    Install with the install script

    curl -LsSf https://llama.app/install.sh | sh
    Use it when, skip it if, stats for llama.cpp

    Why you'd want it

    llama.cpp is a dependency-free C/C++ inference engine that runs on a wide range of hardware: Metal on Apple silicon, CUDA, HIP, Vulkan, SYCL, WebGPU and CPU builds, with CPU+GPU hybrid inference for models larger than your VRAM. It supports 1.5-bit to 8-bit quantization to cut memory use. llama cli -hf <repo> downloads and runs a model straight from Hugging Face, and llama serve starts an OpenAI-compatible API server with a built-in web UI that agents and editors can call.

    Best for

    • Local inference on laptops and mixed hardware
    • Quantized models on limited memory
    • Self-hosted OpenAI-compatible endpoints

    Use it when

    You want direct control over local inference, quantization and hardware backends, or a lightweight OpenAI-compatible server.

    Skip it if

    You only use hosted models, or you would rather not manage model files, quantization choices and hardware backends yourself.

    Stats

    Stars
    131,189
    Last push
    Release
    v0.6.0,
    License
    MIT
    Pricing
    Open source
    Works with
    macOS, Windows, Linux, Docker
    Drop score
    61 of 100

    Full stat block for llama.cpp

For agents

The same entry, as files an agent can read without scraping this page. 

# read one entry
curl -s https://agenttooldrop.com/tools/llm-cli.md

# list the MCP servers in the catalog
curl -s https://agenttooldrop.com/tools.json \
  | jq -r '.tools[] | select(.category=="mcp-server") | .slug'