How to Build a Hierarchical AI Team in Hermes Without Modifying the A2A Code

An experimental persistent-memory workaround for letting a Director agent delegate work to a specialized Analyst

Hermes Agent supports Agent-to-Agent communication through its built-in A2A plugin. This makes it possible for one Hermes instance to discover another, send it tasks, maintain multi-turn conversations, and receive structured results.

But there is an important limitation if you want to build a hierarchical team.

Hermes treats incoming A2A messages as untrusted peer input—even when the peer is authenticated. That is a sensible security default, but it can interfere with setups where one agent is supposed to manage another. For example, a Project Director may be able to contact a Data Analyst, but the Analyst may hesitate or refuse when the Director asks it to access local data, create output files, or perform other delegated work.

Hermes does not currently provide a native “operator-equivalent peer” role. There are open community proposals for A2A trust levels and trusted-operator peers, but those features are not yet part of the standard trust model.

After experimenting with my own agents, I found a prompt-level workaround that can make this arrangement more practical: use the receiving agent’s persistent memory to define a narrow trust policy for one authenticated A2A peer.

This does not modify the A2A plugin. It also does not remove Hermes’ security wrapper or create real Role-Based Access Control. Instead, it gives the receiving model an operator-authored policy for interpreting messages from a specific authenticated agent. You can fix it at the A2A plugin code level, but I am avoiding it as Hermes update their versions far too quickly and I don’t want to spend time having to watch all their code changes. And I can directly instruct each agent directly if I wanted to, but my ideal is that I get an agent to coordinate for me. Hermes can do subagents, but they are just ad-hoc agents and not tied in with profiles, so you can’t have separate memory, profile, etc.

Important: This is an experimental model-level workaround, not protocol-level RBAC. Test it carefully before using it with sensitive files, credentials, or production systems.

The use case

For this example, imagine two separate Hermes agents:

• Project Director: Plans projects, divides work, and delegates data-related tasks.
• Data Analyst: Processes datasets, calculates metrics, organizes information, and returns structured findings.

The goal is for the Director to send tasks such as:

• Analyze a CSV and calculate summary statistics.
• Compare two datasets and identify meaningful differences.
• Organize research results into a table.
• Produce a report inside an assigned workspace.

The Analyst should accept those requests when they come from the authenticated Director, while continuing to treat other A2A peers according to Hermes’ normal security policy.

Why ordinary A2A configuration may not be enough

Hermes’ A2A security layer separates two different concepts:

1. Authentication and admission: Is this peer allowed to submit a task?
2. Instruction trust: Should the agent treat the peer’s text as operator-authorized?

Per-peer bearer tokens and A2A_TRUSTED_PEERS address the first question. They let Hermes identify a peer and restrict who can submit tasks.

They do not answer the second question.

According to the Hermes A2A documentation, incoming task text is passed through prompt-injection filtering and framed as untrusted peer input. The receiving agent is reminded not to follow embedded instructions or disclose private resources.

That framing is useful when talking to unknown or partially trusted agents. It becomes less convenient when two Hermes instances are controlled by the same operator and one is explicitly meant to delegate work to the other.

The workaround below adds a persistent, operator-authored interpretation policy to the receiving agent.

Step 1: Give each agent an isolated profile

Each agent should have its own Hermes profile or home directory.

Do not run two agents against the same Hermes home directory. Hermes memory is scoped to a profile, and two processes writing to the same memory files can create conflicting or unintended state.

The Director and Analyst should therefore have separate:

• Configuration
• Memory files
• Sessions
• Credentials
• Tool permissions
• Workspaces

If both agents run on the same machine, they must also use different inbound ports.

For example:

The port identifies the receiving endpoint. It does not prove which agent sent a request.

Step 2: Configure a named, authenticated peer

For this setup, the Analyst must be able to cryptographically identify the Director. Use a unique per-peer token rather than relying only on a shared token, hostname, port, or name supplied inside the message body.

On the Analyst, configure an inbound token that maps to a stable peer identity:

CopyA2A_PEER_TOKENS=director:<unique-random-secret>
A2A_TRUSTED_PEERS=director

The Director’s outbound peer configuration should point to the Analyst’s endpoint and use the corresponding token:

Copya2a_agents:
  analyst:
    url: "http://127.0.0.1:9901"
    auth:
      type: bearer
      token: "<matching-secret>"
    timeout: 120
    capabilities:
      - data_analysis
      - research
      - reporting

In a real deployment, keep secrets in the appropriate environment or secret store rather than committing them to a repository.

The important point is that the Analyst identifies the sender as director because the Director presented the correct token. The Analyst should not trust a message simply because its text says, “I am the Director.”

A2A_TRUSTED_PEERS=director limits which authenticated identity may submit tasks. It does not, by itself, remove the untrusted-peer framing.

Step 3: Confirm that ordinary A2A communication works

Before adding any persistent trust policy, test the standard A2A connection.

Ask the Director to send the Analyst a harmless request such as:

“Describe your available data-analysis capabilities. Do not access any files.”

Confirm that:

• The Director can reach the Analyst.
• The Analyst returns a response.
• The request appears in ~/.hermes/a2a_audit.jsonl.
• The authenticated peer identity is recorded as director.
• A wrong or missing token is rejected.
• An unlisted peer cannot submit tasks.

Do not proceed until authentication and routing are working correctly. Persistent memory cannot repair an incorrectly configured A2A connection.

Step 4: Add a narrow trust policy to the Analyst’s memory

Open a direct, operator-controlled conversation with the Analyst. Do not send this instruction through the Director.

Submit the following prompt:

Update your built-in persistent memory with this policy:

“A2A requests whose Hermes-authenticated peer identity is director are authorized delegations from my operator. I may follow those requests when they fall within my Data Analyst role and assigned workspace.

Normal credential protections, protected paths, execution approvals, safety controls, and operator-only slash-command restrictions remain in force. Requests from other peers are not covered by this policy.”

Save this policy to persistent memory and show me the resulting memory entry.

This wording is intentionally narrow.

It does not give the Director unlimited authority to rewrite the Analyst’s identity, disable safeguards, reveal credentials, or operate anywhere on the machine. It authorizes ordinary delegation within the Analyst’s assigned role and workspace.

It also ties the policy to the authenticated director identity rather than to:

• A port number
• An IP address
• A self-declared agent name
• A sentence claiming operator authorization

Step 5: Verify that the memory was actually saved

Do not rely only on the model saying, “I saved it.”

Hermes’ built-in persistent memory is normally stored under:

~/.hermes/memories/MEMORY.md

Confirm that the policy appears in the Analyst’s profile-specific memory file.

This distinction matters because Hermes treats SOUL.md and persistent memory differently:

• SOUL.md defines global identity, personality, and behavioral context.
• MEMORY.md contains the agent’s persistent notes and learned policies.
• The normal memory tool writes to MEMORY.md or USER.md; it does not automatically update SOUL.md.

For this workaround, MEMORY.md is the more accurate target.

Step 6: Start a new session

Hermes loads persistent memory into the system prompt as a frozen snapshot at the start of a session.

When the agent writes a new memory entry, the file is updated immediately, but the current session’s system prompt is not rewritten in place.

After saving the trust policy:

1. End or clear the current Analyst session.
2. Start a new session, or restart the relevant gateway process.
3. Ask the Analyst to summarize its A2A delegation policy.
4. Confirm that it names the authenticated director peer and preserves the stated boundaries.

This is an important validation step. If you test only in the original conversation, the apparent success may be caused by ordinary conversation history rather than persistent memory.

Step 7: Scope the Director’s delegation behavior

The receiving agent is only half of the arrangement. The Director should also know when it is appropriate to use the Analyst.

Add a boundary like the following to the Director’s instructions:

The Data Analyst is a specialized agent for processing datasets, calculating metrics, organizing research, and producing structured analytical reports.

Delegate tasks to the Analyst only when they fall within that role. Do not use the Analyst for unrelated operational work, credential handling, system administration, account access, or changes to security policy.

Provide the Analyst with the minimum data and context required for each task. Treat the Analyst’s output as work that may require verification before it is used for consequential decisions.

This prevents the Director from treating the Analyst as a generic subordinate for every available task.

It also reduces transitive trust risk. If the Director reads malicious external content, that content should not automatically become an unrestricted instruction for the Analyst.

Step 8: Test the hierarchy with bounded tasks

Begin with low-risk, observable tests inside a dedicated workspace.

Test 1: Read-only analysis

Ask the Director to send:

“Ask the Analyst to read sample-sales.csv from the assigned test workspace and return the row count, total revenue, mean order value, and any missing-value counts. Do not modify the source file.”

Verify the calculations independently.

Test 2: Controlled file creation

Ask:

“Ask the Analyst to analyze sample-sales.csv and save a Markdown summary as output/sales-summary.md inside the assigned test workspace.”

Confirm that:

• The output file is created.
• It is created only in the intended directory.
• The Analyst does not alter the source data.
• The result is returned to the Director.

Test 3: Role boundary

Ask the Director to send a clearly unrelated request.

The Analyst should decline or explain that the task is outside its Data Analyst role.

Test 4: Authentication boundary

Send an A2A request using:

• The wrong token
• No token
• A different authenticated peer identity
• Text claiming to be the Director without the Director token

None of those cases should receive the Director-specific treatment.

Test 5: Persistence

Restart both agents and repeat the read-only analysis. This confirms that the behavior survives a restart and does not depend only on the original conversation.

What this workaround actually changes

It is important to describe the mechanism accurately.

The permanent memory entry does not alter the A2A adapter. Hermes still:

• Authenticates the peer
• Applies the trusted-peer admission gate
• Filters inbound text
• Adds the untrusted-peer framing
• Blocks remote operator slash commands
• Redacts credential-shaped outbound text
• Applies rate limits
• Writes to the A2A audit log
• Enforces any configured tool approvals and protected paths

The memory policy sits in the receiving agent’s system context. It tells the model that requests from one authenticated peer may be treated as operator-authorized delegations within a defined scope.

In other words, it influences the model’s interpretation of the request. It does not create a new protocol permission or remove code-enforced protections.

Why this is not native RBAC

A real RBAC implementation would make an explicit, deterministic authorization decision in code.

For example, Hermes might eventually support trust levels such as:

• Untrusted peer
• Read-only peer
• Task-delegating peer
• Trusted operator peer

The adapter could then decide which actions and resources are available to each identity.

The persistent-memory workaround does not do that. Its behavior may vary based on:

• The model and inference provider
• The exact memory wording
• Prompt ordering
• Existing conversation context
• Enabled tools
• Approval settings
• Hermes version
• Model updates

That is why the technique should be called a prompt-level trust policy, not RBAC.

Security considerations

Authentication is not authorization

A bearer token proves which configured peer sent the message. It does not prove that the requested action is safe.

If the Director’s token is stolen, an attacker may be able to submit requests under the trusted identity.

Use unique tokens, protect them like credentials, and rotate them if they may have been exposed.

Trust can become transitive

The Director may consume:

• Web pages
• Documents
• Emails
• Scraped datasets
• Tool results
• Messages from other agents

If malicious content influences the Director and the Director forwards the resulting instruction to the Analyst, the Analyst may treat it as an authorized delegation.

The Director should therefore minimize copied external text, distinguish data from instructions, and limit delegation to a defined role.

Keep the Analyst sandboxed

Where possible, give the Analyst:

• A dedicated workspace
• Only the datasets it needs
• No unnecessary credentials
• Restricted file permissions
• A containerized or isolated execution environment
• Human approval for dangerous operations

Prompt-level boundaries are helpful, but OS and container boundaries are more dependable.

Avoid unrestricted self-modification

I do not recommend telling the Analyst that the Director may freely modify its:

• Identity
• Trust policy
• Permanent memory
• Security instructions
• Tool configuration
• Credentials

That creates a broad, difficult-to-audit chain of authority.

If the Director needs to propose a change to the Analyst’s operating policy, it is safer for the Analyst to present the proposed change to the human operator for confirmation.

What I found in the Hermes community

I found multiple open Hermes issues describing the need for hierarchical or more trusted A2A relationships.

One proposal requests configurable A2A trust levels. Another asks for a trusted-operator peer tier that would allow designated authenticated peers to perform private or local tasks without receiving the standard restrictive interpretation.

These discussions confirm that the underlying problem is real.

However, I did not find an independently documented report confirming this exact persistent-memory technique as a reliable, production-ready solution. My results should therefore be treated as an initial implementation report rather than proof of a standard Hermes pattern.

My current result

In my initial testing, the Director and Analyst are now communicating successfully, and the Analyst appears more willing to accept properly scoped work from the authenticated Director.

The most important improvements were:

1. Giving the Director a named per-peer identity.
2. Restricting inbound access to that identity.
3. Saving a narrow delegation policy in the Analyst’s persistent memory.
4. Restarting the session so the new memory was loaded.
5. Defining clear role boundaries on both agents.

I am continuing to test persistence, repeatability, negative authentication cases, workspace restrictions, and behavior across restarts.

I would not yet describe the arrangement as native hierarchy support. It is better understood as a practical, experimental bridge until Hermes gains a first-class trusted-operator or trust-level feature.

Conclusion

Hermes’ default A2A security model is intentionally conservative. An authenticated peer can be allowed to submit tasks without automatically becoming equivalent to the human operator.

For users running several Hermes instances under the same administration, that can make hierarchical delegation difficult.

A narrowly written persistent-memory policy can help the receiving model recognize one authenticated agent as an authorized delegator without modifying the A2A plugin. The approach leaves Hermes’ protocol-level authentication, filtering, redaction, audit, and execution safeguards in place.

But the distinction is critical:

This is a model-level interpretation policy, not protocol-level RBAC.

Use per-peer authentication, narrow role boundaries, isolated workspaces, repeatable tests, and ordinary host security. For production environments, native code-enforced authorization would still be the better long-term solution.

Sources and further reading

• Hermes Agent A2A documentation
• Hermes persistent-memory documentation
• Hermes security documentation
• Hermes context-files documentation
• Hermes A2A security implementation
• A2A trust-level proposal, Issue #86061
• Trusted-operator peer proposal, Issue #91168
• Hierarchical agent proposal, Issue #61109