How I let an agent inspect the lab without handing it the credentials
The lab MCP server gives an assistant named, read-only operations for Vault metadata, monitoring, and Jenkins state. It fetches each credential inside the server, uses it for the request, and checks the response before returning anything. There is no tool that returns a secret value and no tool that changes infrastructure.
I built it after three different credentials reached agent transcripts through general-purpose shell output. The common problem was not one unsafe command. It was that the assistant held a shell in the same context as plaintext credentials. The new path moves the operation behind a narrow tool and keeps the credential inside that process.
Built from the stdio MCP server, response-egress guards, read-method allowlist, management-path exclusions, and offline secret-leak regression tests in the repository.
Three leaks showed that command rules were guarding the wrong boundary
MCP is a way to give an AI assistant a set of tools it can call. Instead of the model guessing about the lab from training data, it calls a tool that returns the real answer: what alerts are firing, what a host looks like in inventory, whether a backup succeeded. The lab MCP server is the program that answers those calls. It sits between the assistant and the lab's systems and translates a question into a specific, read-only lookup against the right system.
The reason to build it carefully is the same reason it is useful. A tool that can read your whole estate is one prompt away from being asked to change your whole estate. So the design started at the dangerous end and worked backwards.
In one day, a git token, a secrets-server token, and a database password reached transcripts through three different shell commands. The database case made the design problem clear. The assistant asked for a username, and an environment search returned the password beside it. A longer denylist of dangerous commands would not have predicted that output.
Read-only by construction
Version one exposes reads and only reads. There is no tool that creates, edits, restarts, or deletes anything, so there is no mutating tool to misfire, be tricked into calling, or be reached by a bug. This is a deliberate first version, not a limitation waiting to be lifted casually. When a write capability is ever added, it will be its own reviewed change with its own gates, not a flag flipped on the read server. The safe version ships first and earns trust before anything with teeth is considered.
Exclusions live in code, not config
Some things must never be exposed even for reading, and some systems must never be reachable at all. The list of what is off limits lives in the server's own source code, not in a configuration file. The difference matters more than it sounds. A config file is data that ships alongside the program and can be edited, overridden per environment, or left out of a deploy by mistake. Code that enforces an exclusion is reviewed like any other change, cannot be quietly overridden at runtime, and travels with the program everywhere it runs. Putting the boundary in code is how you make it a property of the system rather than a setting that depends on everyone remembering to set it.
On top of that, individual tools are gated by configuration for what is merely optional, so the hard floor and the soft switches are two separate layers. The hard floor cannot be moved by the soft layer. This mirrors a rule the rest of the lab already runs on: the most important machines are excluded from automated action permanently and in the floor, never by a toggle that can be flipped.
The protocol makes each operation explicit; durable audit is still owed
Each request names one registered tool and supplies structured arguments. The response can therefore be tied to a concrete operation such as listing Vault field names, reading an allowlisted monitoring method, or fetching a safe Jenkins REST path. Unknown tools, monitoring methods outside the allowlist, and Jenkins management paths are refused.
The current server returns those results over standard input and output. It does not yet write a separate append-only audit log. A client transcript can show which call was made, but that is not the same as a server-owned record that survives the client. I would add that record before considering any wider deployment, and well before designing a write-capable sibling.
The surface that is not there
The strongest security is the capability you did not build. Version one has no deploy path, no way to mint or hand out a token, no general mutation surface, and no ad-hoc transport for a caller to reach past the defined tools into the systems underneath. Every one of those is a door, and every door is something to defend. By not building them into the first version, there is simply less to get wrong. The page for the guardrails around the AI stack goes into the budget caps and the operator approvals; this page is about the one design choice under all of it, which is to make the unsafe thing impossible rather than merely discouraged.
How it runs now
| When | What runs | What happens next |
|---|---|---|
| Client startup | The MCP client starts the Python server as a local stdio process with its own scoped environment. | The assistant receives the registered tool descriptions, not the server's credentials. |
| Vault metadata question | The server lists path names, field names, or a record version. | Values never enter the response surface. |
| Monitoring question | The server accepts only methods in the read allowlist and fetches its token internally. | An unknown or mutating method is refused before the API call. |
| Jenkins question | The server makes a GET request to a non-management REST path. | Credential, script-console, configuration, and management paths remain unreachable. |
| Every credential-backed response | The server scans the result and error detail for the raw and encoded credential it used. | A match returns a refusal instead of the upstream payload. |
What broke before this design existed
General-purpose output exposed data nobody asked to see
The three transcript leaks came from different commands, so blocking those exact commands would have left the underlying problem intact. A shell can return neighboring environment values, command output, error bodies, or configuration fields that contain a credential.
The server now fetches a credential only for the operation that needs it and refuses a response that echoes it. This does not make a general shell safe. Plaintext credentials still have to disappear from any filesystem or environment a general agent session can reach.
What I would do differently
I would begin with the credential boundary, not the list of useful tools. The original AIOps tool surface was organized around which systems an analyst wanted to query. The leak review showed that custody and response filtering had to be the first design.
I would also make the append-only audit sink part of version one. Structured calls improve the record, but the server should own a value-free event trail independent of whichever client started it.