Endpoint Lock: lock down self-hosted LLMs before attack agents run up your bill

The auth proxy is close to crowded. LiteLLM already has per-key budgets and rate limits for Ollama and vLLM, vLLM and llama.cpp have built-in keys, and there are many small open-source Ollama proxies with keys, rate limits or token caps. None of them is polished, cross-server, and built around spike alerts. A new entrant could still stand out with a very easy single binary that covers Ollama, vLLM and llama.cpp, has token caps measured in tokens rather than dollars, sends usage-spike alerts, and ships with secure defaults. That is a thin advantage, and hobbyists are used to free tools. The exposure self-check is less served. People use curl, Shodan InternetDB, command-line probes like ollama-security, or paid scanners like SurfaceScan. No hosted probe was found that only tests IPs whose owner has proved control with a token file, and none ties the probe to the proxy to confirm the lock actually works. That includes vLLM paths the built-in key may miss, such as /invocations, and ports Docker has published through the firewall. The most defensible paid product is probably the pairing of proxy and outside check, plus ongoing monitoring that warns when your IP appears in scanner data, rather than either piece alone.

• LiteLLM Proxy

  Open-source gateway that already gives each key its own budget and requests-per-minute and tokens-per-minute limits, and works in front of Ollama and vLLM. It is the main existing answer to the proxy idea, but it is heavier than a single binary and its budgets are in dollars, which fit poorly with local models that have no per-token price.

• vLLM and llama.cpp built-in --api-key

  Both servers have built-in bearer-token keys, and llama.cpp can read several keys from a file. Neither gives per-key rate limits, token caps or alerts. One source says vLLM's key leaves some paths such as /invocations unprotected.

• 1lcb/ollama-gateway

  Go reverse proxy for Ollama with API keys, per-client rate limits, an endpoint for creating keys and load balancing. It has about 1 star and no pushes since January 2025.

• hofmannol/llmproxy

  Docker proxy with per-key daily and monthly token or request caps that accepts OpenAI-style requests and, in 1.0.0, Anthropic-style requests. Its admin API is protected by a single password.

• ollama_proxy_2 (Ollama Proxy Server)

  Python/FastAPI load-balancing proxy that stores keys hashed with bcrypt and caps how many requests run at once.

• smol-llm-proxy

  Proxy for llama.cpp with per-key requests-per-minute and tokens-per-minute limits that returns 429 with Retry-After.

• ParisNeo lollms_server_proxy

  Lightweight reverse proxy for load balancing and rate limiting that checks user:key pairs from an authorized-users file.

• Perufitlife/ollama-security

  Read-only command-line tool run with npx that sends anonymous requests to an Ollama URL to see whether it answers without authentication. Created June 2026, 0 stars. You run it yourself; it is not a hosted probe.

• SurfaceScan

  Commercial outside scanner that checks your IPs for exposed Ollama and LLM endpoints, including non-standard ports. It is the closest thing to the hosted self-check.

• Ollama Scanner

  Python tool with a local web UI that scans one host or a range of addresses on common LLM ports, including vLLM's 8000. It runs on your own machine, not as a hosted service.

• Shodan InternetDB

  Free lookup by IP that shows whether Shodan has already indexed port 11434 for you. Guides recommend it as the quick external check, though it shows past scan data rather than a live test.

Checked 2026-10-23