Your Internal AI Gateway Is on the Internet (LiteLLM Lesson)
What LiteLLM said about its own exposed deployment on October 21, 2026, and a check for any app builder who runs an AI gateway or proxy.
Founder, Newly
Published Last updated
Keep your AI gateway on a private network, turn off its admin UI unless you use it, and give each app its own limited key. LiteLLM's October 21, 2026 incident post says its own internal deployment was exposed to the public internet. A gateway holds your AI provider keys, so check today who can reach yours.
What did LiteLLM report on October 21, 2026?
Two separate things appear in the same post. One is a malicious release of the LiteLLM Python package. The other is a finding about LiteLLM's own infrastructure. The table keeps them apart. The post was last updated on October 21 at 20:30 UTC, and we checked it on October 24, 2026.
| Topic | What LiteLLM states |
|---|---|
| Affected release | litellm 1.107.0 on PyPI, live from 11:39:51 to 13:23 UTC. Docker images v1.107.0, latest, main-latest and main-stable, from 11:46:56 to 14:20 UTC. The post names no other version. |
| Internal deployment | The team found "an unexpected, innocuously named user account in our internal LiteLLM deployment, which was exposed to the public internet." |
| Logs | Claude Code will embed a user's GitHub CLI OAuth token in a response body. Where response bodies are logged, these tokens may appear in LiteLLM logs. |
| What to rotate | The instance or task IAM role, every provider key stored in the proxy database, the master and virtual keys, and the DATABASE_URL password. |
| Status | "Incident contained, investigation in progress." A fix for a separate vulnerability is scheduled for disclosure on October 27, 2026. |
Datadog Security Labs adds that the malicious files run every time Python starts, and that it found no established link to LiteLLM's earlier March 2026 incident, which the PyPI incident report describes as compromised litellm releases. The post does not say how the exposed deployment relates to the package release, and this page does not claim a link either. What it offers is a reminder: an internal tool with real credentials was reachable from the internet.
Why does an AI gateway matter more than the app that calls it?
A gateway, also called an LLM proxy, is a server between your app and the AI providers. Your code sends it a request with one key, and it forwards the request using the provider key it stores. That is useful for budgets and logging. It also makes the gateway the place where every key meets.
LiteLLM's rotation list shows what one gateway can hold: provider keys, a master key, virtual keys, a database password and a cloud role. Its security and encryption FAQ describes encryption for stored credentials, while spend logs and request payloads are outside that scope. A stolen provider key is a bill you pay, and OWASP's 2026 list for LLM applications covers both sensitive information disclosure and unbounded consumption.
Exposure of this kind is common. A Cloud Security Alliance research note from September 15, 2026 counted 36,769 self-hosted AI endpoints reachable from the internet and reported that about 2% asked for any login. CISA applied the same logic to federal agencies in BOD 23-02: remove management interfaces from the internet or put them behind access control.
How do I check whether my gateway is reachable from the internet?
Start from where you run it. Open your cloud console and note the address, the port and the firewall or security group rule in front of it. LiteLLM's admin UI docs show port 4000 and the path /ui as the defaults. Then test from a place that is not your network, such as a phone on mobile data, and match what you see to the table.
| Check | What you see | State |
|---|---|---|
| Gateway address from mobile data | Times out or refuses the connection | Unknown. It covers one address and one port from one place. |
/ui on that address | A login page appears | Detected. The admin UI is reachable from the internet. |
/health/liveliness | A short text reply, no key needed | Detected as reachable. The docs list this endpoint as open by design, so it proves the port is open and nothing more. |
| Firewall rule source | 0.0.0.0/0 on the gateway port | Detected. Anyone can attempt a connection. |
| Firewall rule source | Your app server's address, or a private network only | Confirmed by owner evidence once you keep the rule export or a screenshot. |
The health endpoints page lists which health routes need a key and which do not. If your load balancer needs the open ones, allow only the load balancer to call them. A reply from outside tells you the door is open. A timeout from one place never tells you the door is shut everywhere.
Which gateway settings should I look for?
These are the settings LiteLLM documents for a production proxy. Open your config file and environment, find each one and write down its value.
| Setting | What it does | Look for |
|---|---|---|
DISABLE_ADMIN_UI | Turns the admin UI off when set to True. | True if nobody on your team uses the UI. |
LITELLM_MASTER_KEY | The admin credential. It must start with sk- and belongs in a secret manager. | A long random value that no app or script uses. |
UI_USERNAME | Bootstrap login. The username defaults to admin and the password falls back to the master key. | Personal admin accounts, with environment login turned off after setup. |
allowed_ips | Returns 403 to addresses not on the list. The docs mark it as an enterprise feature. | A list of your own server addresses, or a network rule that does the same job. |
| Virtual keys | Per-key budgets, tpm_limit and rpm_limit, so one leaked key has a ceiling. | One key per app, each with a budget and a rate limit. |
turn_off_message_logging | Stops request and response bodies from being logged. | True unless you need bodies and have decided who may read them. |
LITELLM_MODE | Set to PRODUCTION, it stops the proxy from loading credentials out of a local .env file. | PRODUCTION on any internet-facing host. |
A setting in a file is a claim
Sources for this table are the production docs, the IP filtering page and the virtual keys page. Reading a value in a file shows what you asked for. The test from outside in the previous section shows what the running service does. You need both before an item moves to confirmed.
Do my gateway logs contain secrets?
They can. LiteLLM's post describes one route: a coding tool puts a GitHub token in a response, the gateway logs response bodies, and the token sits in the log. Any chatbot or agent that handles pasted text, files or tool output can do the same with whatever the user pasted.
- Find out whether bodies are stored. The logging docs cover redaction and the spend log storage options.
- Search a day of recent logs and the spend log table for words like token, authorization and key, and for long random strings. Treat any hit as a credential to rotate.
- Limit who can read logs to the people who debug the gateway, and keep them out of shared chat threads and tickets.
If you already know a credential was readable, the rotation guide on red.run.cloud covers the order to rotate in. The chat feature in front of your gateway has its own checks in the 15-minute chatbot self-test.
What if I ran LiteLLM 1.107.0 or a Docker tag from that window?
Check each place LiteLLM was installed, because a laptop, a CI runner and a container can each differ.
- Search lockfiles,
pip freezeoutput and image digests for 1.107.0. For Docker, the tags in the post moved during the window, so a pull oflatestormain-stablein those hours matters. - The post lists these indicators:
cook.pth,cook.py,sitecustomize.pyand alitellm-1.107.0.dist-infofolder. Finding none is a reading from one host, so mark it unknown until you have a rebuild. - Datadog notes the files run at every Python start. That is why rebuilding the image or virtual environment beats patching it in place, and why running
pipon an affected host is a poor idea. - Rotate the credentials in the table above. A rotation counts as confirmed when you can show the old key refused and the new key in use.
To limit the next bad release, install from a lockfile with hashes. pip's hash-checking mode refuses any package whose hash is not on your list. Replace floating image tags with a version or digest, and update on a day you chose.
Does my mobile app need to talk to the gateway at all?
Usually not directly. A React Native and Expo app compiles into something people download, and Expo's environment variable docs say not to store private keys in EXPO_PUBLIC_ variables, because they are readable in the compiled app. OpenAI's production guide also tells you to keep keys out of your code and to rotate them.
- Let the app call your own server code. That code checks the signed-in user, applies a per-user limit and then calls the gateway or the provider.
- Set a spend ceiling at the provider as well. Anthropic's workspaces support custom spend and rate limits.
- Remember that AI inside an app built with Newly uses credits and some features have limits. A gateway you run yourself has its own bill, and you set its limits.
If you have not yet looked at the rest of the app, start with the 15-minute app check. The full list of guides is on the security hub.
Where do you stand?
Every item starts as unknown because an unchecked item is unknown. Move one to confirmed only when you hold the evidence listed under it.
Your gateway check
- Detected: You found the problem.
- Unknown: Not checked yet, or the check could not tell.
- Confirmed by owner evidence: You checked and kept the proof.
Your answers stay in this browser tab. Nothing is saved or sent.
Frequently asked questions
Is every LiteLLM install affected?
No. LiteLLM's incident post names one release, litellm 1.107.0, on PyPI and in the Docker tags v1.107.0, latest, main-latest and main-stable, inside the time windows it lists. Other versions are not named. Separately, any gateway reachable from the internet deserves the checks on this page, whatever version it runs.
Does turning off the admin UI close the gateway?
No. The admin UI is one page of the proxy. The API that your app calls stays reachable at the same address unless a firewall, private network or access rule keeps outsiders away. Check the network first and treat the UI setting as a second layer.
Can I just keep the gateway behind my app's login?
The gateway is a separate server with its own address. Your app's login does not sit in front of it. Either keep it on a private network that only your server code can reach, or make sure every request to it needs a key you issued and can limit.
Why do request logs matter if the gateway is private?
Logs outlive the request. LiteLLM's post says a coding tool placed a GitHub token in a response body and that these tokens may appear in LiteLLM logs wherever response bodies are logged. Anyone who can read those logs can read what they hold, so decide what you store before you need it.
Will the free scan on red.run.cloud find my gateway?
Only if it answers on a domain you own and prove. The free scan reads the stack a site serves and also looks for panels, consoles and debug pages left online. A gateway on a bare IP address, a private network or a port the scan does not look at stays unknown.
Should I use a gateway at all?
It depends on what you need. A gateway helps when several apps or teams share provider keys and you want one place for budgets and limits. A single app with one provider can call a server function that holds the key. Fewer moving parts means fewer things to expose.
Sources
- Security incident, October 21, 2026 (litellm 1.107.0), LiteLLM
- LiteLLM compromised on PyPI and Docker Hub, Datadog Security Labs
- Production best practices, LiteLLM docs
- Admin UI and authentication, LiteLLM docs
- IP address filtering, LiteLLM docs
- Logging and redaction, LiteLLM docs
- Virtual keys, budgets and rate limits, LiteLLM docs
- Security and encryption FAQ, LiteLLM docs
- Health endpoints, LiteLLM docs
- Incident report: LiteLLM and Telnyx supply chain attack, PyPI blog, April 2, 2026
- Secure installs: hash-checking mode, pip documentation
- BOD 23-02: Mitigating the risk from internet-exposed management interfaces, CISA
- OWASP Top 10 for LLM Applications 2026, OWASP GenAI Security Project
- Environment variables in Expo, Expo docs
- Production best practices: API keys, OpenAI docs
- Workspaces and spend limits, Anthropic docs
- 36,769 exposed AI endpoints, research note, September 15, 2026, Cloud Security Alliance