Secrets Management for Agent Chains: Vaults, Rotation, and Scoping

Every agent chain that does something useful eventually needs a secret. An API key for OpenAI. A database connection string. An SSH key for a remote server. AWS credentials for S3. Slack tokens for notifications. The moment your agents interact with external systems, secrets management becomes the thing that determines whether your pipeline is secure or a breach waiting to happen.

Most teams start by hardcoding secrets in chain definitions. It works until it doesn't -- someone commits the chain config to git, a log captures the prompt that includes the API key, or a junior dev copies a chain template without realizing there's a production database password embedded in the prompt text. The fix isn't discipline. The fix is architecture.

Why agent chains make secrets harder

Traditional applications have a relatively simple secrets story. The app starts, reads secrets from environment variables or a vault, and uses them for the lifetime of the process. Agent chains introduce complications:

Multiple agents, different credentials. A 5-agent chain might need OpenAI keys for two agents, a GitHub token for one, a database password for another, and Slack credentials for the notification agent. Each agent should only see the secrets it needs.

Dynamic execution environments. Agents might run locally, in Docker containers, or over SSH on remote machines. The secret delivery mechanism needs to work across all of these.

Long-running chains. A chain that runs for 30 minutes might outlive a short-lived token. A chain that runs on a schedule needs credentials that are valid at execution time, not at definition time.

Output leakage. Agents produce logs, events, and output files. Any of these could inadvertently contain secrets that were part of the agent's context. You need to prevent secrets from appearing in output, not just input.

Shared chains. When teams share chain templates through a marketplace or internal library, secrets must be completely decoupled from the chain definition. A chain template should be portable without carrying any credentials with it.

Vault architecture for agent chains

A secrets vault for agent orchestration needs four capabilities: encrypted storage, scoped access, runtime injection, and audit logging.

Encrypted storage

Secrets should be encrypted at rest with AES-256-GCM or equivalent. The encryption key itself should be derived from a master key that's stored separately from the encrypted secrets -- ideally in a hardware security module or a cloud KMS. Mentiko uses AES-256-GCM with per-secret initialization vectors, so compromising one secret's ciphertext doesn't help decrypt another.

The vault should store secrets as key-value pairs with metadata:

{
  "key": "OPENAI_API_KEY",
  "value": "[encrypted]",
  "scope": "chain:content-pipeline",
  "created_at": "2026-03-15T10:00:00Z",
  "rotated_at": "2026-03-15T10:00:00Z",
  "expires_at": "2026-06-15T10:00:00Z",
  "created_by": "user:marco",
  "access_log_enabled": true
}

Scoped access

Not every agent needs every secret. Scoping controls which chains, agents, and users can access which secrets. There are three natural scope levels:

Organization scope. Secrets available to all chains in the org. Use this sparingly -- maybe for the LLM API key that every chain needs.

Chain scope. Secrets available only to a specific chain. A chain that processes financial data gets the database credentials; other chains don't.

Agent scope. Secrets available only to a specific agent within a chain. The deployment agent gets the SSH key; the analysis agent in the same chain doesn't.

In Mentiko's chain definition, scoping looks like this:

{
  "chain": "deploy-pipeline",
  "agents": [
    {
      "name": "CodeAnalyzer",
      "secrets": ["OPENAI_API_KEY"]
    },
    {
      "name": "TestRunner",
      "secrets": ["OPENAI_API_KEY", "DATABASE_URL"]
    },
    {
      "name": "Deployer",
      "secrets": ["OPENAI_API_KEY", "SSH_DEPLOY_KEY", "AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY"]
    }
  ]
}

The CodeAnalyzer never sees the SSH key. The TestRunner never sees the AWS credentials. Each agent operates with minimum necessary access.

Runtime injection

Secrets are injected into the agent's execution environment at runtime, not baked into the chain definition. The flow:

1. Chain execution starts.
2. For each agent, the orchestrator resolves the agent's secret references against the vault.
3. Access control checks run: does this chain/agent have permission to access this secret?
4. Secrets are decrypted and injected as environment variables into the agent's workspace.
5. The agent runs with secrets available in its environment.
6. When the agent completes, the environment is destroyed. Secrets don't persist.

This means your chain JSON never contains actual secret values. It contains references -- names that map to vault entries. You can commit your chain definitions to git, share them with teammates, publish them to a marketplace, and there's zero risk of secret exposure.

Audit logging

Every secret access should produce an audit record: who accessed what, when, from which chain, for which agent. This isn't optional for teams in regulated industries, but it's good practice for everyone.

{
  "event": "secret_accessed",
  "secret_key": "DATABASE_URL",
  "chain": "financial-reconciliation",
  "agent": "DataFetcher",
  "user": "marco",
  "timestamp": "2026-03-19T14:30:22Z",
  "workspace": "docker:finance-sandbox"
}

When something goes wrong -- and eventually something will -- you need to know exactly which secrets were accessed and by whom.

Rotation without downtime

Secrets expire. API keys get compromised. Compliance policies require periodic rotation. The question is whether rotation disrupts your running chains.

The dual-version pattern

The safest rotation strategy keeps two versions of a secret active simultaneously during the transition window:

1. Create the new secret version in your vault. Mark it as pending.
2. Update the vault entry so both current and pending versions are valid.
3. New chain executions pick up the pending version.
4. Wait for all in-flight chains using the current version to complete.
5. Promote pending to current. Delete the old version.

This means zero downtime. No chain execution fails because a secret changed mid-run. No race condition between rotation and execution.

{
  "key": "OPENAI_API_KEY",
  "versions": {
    "current": {
      "value": "[encrypted-old]",
      "valid_until": "2026-03-20T00:00:00Z"
    },
    "pending": {
      "value": "[encrypted-new]",
      "valid_from": "2026-03-19T12:00:00Z"
    }
  }
}

Automated rotation schedules

For secrets that support programmatic rotation (AWS IAM keys, database passwords with ALTER USER, OAuth client secrets), automate the rotation on a schedule:

{
  "rotation_policy": {
    "OPENAI_API_KEY": {
      "rotate_every_days": 90,
      "notify_before_days": 7,
      "auto_rotate": false
    },
    "AWS_ACCESS_KEY_ID": {
      "rotate_every_days": 30,
      "notify_before_days": 3,
      "auto_rotate": true,
      "rotation_chain": "infra/rotate-aws-keys"
    }
  }
}

Yes, you can use a Mentiko chain to rotate the secrets that other Mentiko chains use. The rotation chain calls the AWS IAM API, generates new keys, updates the vault, and verifies the new keys work -- all automated.

Emergency rotation

When a secret is compromised, you need to rotate immediately and invalidate the old version. No grace period, no dual-version window. The vault should support an emergency_revoke operation that:

1. Immediately invalidates the old secret.
2. Activates the new secret.
3. Kills any in-flight chain executions that had the old secret injected.
4. Sends an alert to the security team.
5. Produces an audit trail of which executions were terminated and why.

Running chains will fail. That's acceptable when the alternative is a compromised credential continuing to be used.

Per-chain scoping patterns

Beyond basic scope levels, there are patterns that help organize secrets for complex multi-chain setups.

Environment-based scoping

The same chain often runs in different environments: dev, staging, production. Each environment has its own secrets. Rather than maintaining separate chain definitions, use environment prefixes:

{
  "secrets": {
    "dev/DATABASE_URL": "postgres://localhost:5432/dev",
    "staging/DATABASE_URL": "postgres://staging.internal:5432/app",
    "prod/DATABASE_URL": "postgres://prod.internal:5432/app"
  }
}

The chain definition references DATABASE_URL. The orchestrator resolves it based on the execution environment. Same chain, different credentials, zero configuration changes.

Secret groups

When multiple secrets are always used together (like AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY), group them:

{
  "secret_groups": {
    "aws-deploy": {
      "AWS_ACCESS_KEY_ID": "[encrypted]",
      "AWS_SECRET_ACCESS_KEY": "[encrypted]",
      "AWS_DEFAULT_REGION": "us-west-2"
    }
  }
}

Agents reference the group, not individual secrets. When you rotate AWS credentials, you update one group and every agent that uses it gets the new credentials.

Cross-chain secret sharing

Some secrets are shared across chains but shouldn't be organization-wide. A "project" scope sits between org and chain:

{
  "key": "STRIPE_SECRET_KEY",
  "scope": "project:billing-system",
  "chains_allowed": [
    "billing/invoice-generator",
    "billing/payment-processor",
    "billing/reconciliation"
  ]
}

Three chains share one Stripe key, but the marketing team's chains can't access it.

Preventing secret leakage in output

Injecting secrets securely is half the battle. The other half is preventing agents from leaking secrets in their output.

Prompt engineering. Instruct agents explicitly: "Never include API keys, passwords, or tokens in your output. If you need to reference a credential, say 'using configured credentials' instead of echoing the value."

Output scanning. After each agent completes, scan the output for patterns that look like secrets: long base64 strings, strings matching known key formats (sk-..., AKIA..., ghp_...), or any value that matches a secret currently in the vault.

Redaction. If a secret is detected in output, replace it with a placeholder like [REDACTED:OPENAI_API_KEY] before the output is logged or passed to the next agent.

Event file sanitization. Mentiko's file-based event system means agent output is written to disk. The orchestrator should sanitize event files before writing, stripping any detected secrets from the content.

The implementation checklist

If you're setting up secrets management for agent chains:

• [ ] Secrets stored in encrypted vault, never in chain definitions
• [ ] Per-agent scoping so each agent sees only what it needs
• [ ] Runtime injection via environment variables, not prompt text
• [ ] Audit logging on every secret access
• [ ] Dual-version rotation for zero-downtime updates
• [ ] Automated rotation schedules for programmatically rotatable secrets
• [ ] Emergency revocation procedure documented and tested
• [ ] Output scanning for accidental secret leakage
• [ ] Environment-based scoping for dev/staging/prod separation
• [ ] Secret groups for credentials that travel together

Secrets management isn't glamorous. It's plumbing. But it's the plumbing that determines whether your agent chains are a productivity multiplier or a security incident waiting for a trigger.

---

For a broader look at agent security, see our security practices guide. Ready to build? Start your first chain in five minutes.