觀音AI : On-Device Multi-Agent Architecture
The architecture of 觀音AI is built around three requirements:
- multiple AI agents with different responsibilities must collaborate on the user's problem;
- the collaboration must appear to the user as one coherent 觀音AI conversation;
- private conversations and AI inference should remain on the user's device.
The application itself is the multi-agent AI system.
┌─────────────────────────────────────────────────────────────────┐
│ iPhone │
│ │
│ ┌─────────────┐ │
│ │ User │ │
│ └──────┬──────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────┐ │
│ │ 觀音AI │ │
│ │ Conversation / UI │ │
│ └──────────┬───────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Multi-Agent AI System │ │
│ │ │ │
│ │ Fortune / Symbolic Perspective │ │
│ │ ↕ │ │
│ │ Rational / Objective Analysis │ │
│ │ ↕ │ │
│ │ Emotional Context │ │
│ │ ↕ │ │
│ │ Practical Action │ │
│ │ ↕ │ │
│ │ Response Synthesis │ │
│ └──────────────────────┬───────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ On-Device LLM │ │
│ │ Inference │ │
│ └──────────────────┘ │
│ │
│ Conversation history + model + inference remain on device │
│ │
└─────────────────────────────────────────────────────────────────┘
The device boundary is also the primary privacy boundary.
1. Product Responsibilities Before Components
The architecture begins with a separation that is more important than the choice of model or runtime.
觀音靈簽 and AI do not have the same responsibility.
USER'S PROBLEM
│
▼
┌───────────────────┐
│ 觀音靈簽 │
│ │
│ • intuition │
│ • symbolism │
│ • insight │
└─────────┬─────────┘
│
reflective context
│
▼
┌───────────────────┐
│ AI │
│ │
│ • reasoning │
│ • objectivity │
│ • emotional │
│ steadiness │
│ • practical │
│ next steps │
└─────────┬─────────┘
│
▼
USER DECIDES
The fortune stick is therefore not treated as a prediction produced by the AI.
It is contextual input into the reasoning process.
This prevents the LLM from simply becoming an electronic fortune teller and provides an architectural boundary between symbolic interpretation and real-world reasoning.
2. Multi-Agent Reasoning
A user's question can require several kinds of reasoning simultaneously.
Consider:
My girlfriend said she needs some space.
It's been almost two days since she replied.
Should I send another message, or should I leave her alone?
This apparently simple question contains several separate problems.
There may be a relevant fortune-stick interpretation.
There are known facts.
There are unknown facts.
There are assumptions about what silence means.
There is anxiety caused by uncertainty.
There is also an immediate decision.
Instead of treating all of these as one undifferentiated reasoning responsibility, 觀音AI decomposes the problem across specialized AI roles.
USER
│
│ message
▼
┌───────────────────┐
│ Conversation │
│ Context │
└─────────┬─────────┘
│
▼
┌────────────────────────┐
│ Multi-Agent Reasoning │
└────────────┬───────────┘
│
┌───────────────┼───────────────┐
│ │ │
▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Fortune / │ │ Rational / │ │ Emotional │
│ Symbolic │ │ Objective │ │ Context │
│ Perspective │ │ Analysis │ │ │
└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
│ │ │
└──────────┬──────┴─────────┬───────┘
│ │
▼ ▼
┌──────────────┐ ┌──────────────┐
│ Practical │ │ Additional │
│ Action │ │ Context / │
│ Reasoning │ │ Reflection │
└──────┬───────┘ └──────┬───────┘
│ │
└────────┬───────┘
▼
┌─────────────────┐
│ Synthesis │
└────────┬────────┘
│
▼
ONE USER-FACING
RESPONSE
The important property is not the number of boxes.
The important property is separation of responsibility followed by collaboration.
3. Agent Responsibilities
Fortune / Symbolic Perspective
This responsibility examines the relevant 觀音靈簽 context.
Its purpose is to extract useful themes, symbolism or alternative perspectives.
It must not convert symbolic interpretation into factual evidence.
Fortune Context
│
▼
┌──────────────────────┐
│ Symbolic Reasoning │
│ │
│ What perspective │
│ does the fortune │
│ introduce? │
└──────────┬───────────┘
│
▼
Reflective Perspective
The output is a perspective for consideration, not a prediction.
Rational / Objective Analysis
This responsibility deliberately examines the user's situation independently of fortune-telling claims.
Its reasoning includes questions such as:
What happened?
│
├──► What is known?
│
├──► What is assumed?
│
├──► What is unknown?
│
├──► What alternative explanations exist?
│
└──► What conclusions are unsupported?
This role acts as an important counterweight to both symbolic interpretation and emotionally driven assumptions.
For example:
Observed fact:
"She has not replied for almost two days."
≠
Unsupported conclusion:
"She doesn't love me anymore."
The system should preserve that distinction.
Emotional Context
The literal question is not always the complete problem.
The emotional-context responsibility examines what may be driving the immediate decision.
User Message
│
▼
┌──────────────────────┐
│ Emotional Context │
│ │
│ anxiety? │
│ fear? │
│ frustration? │
│ uncertainty? │
│ urgency? │
└──────────┬───────────┘
│
▼
How should this affect
the timing and form
of the response?
The objective is not simply to generate comforting language.
Emotional state can affect decision quality.
The architecture therefore treats emotional context as an input into reasoning rather than merely a tone-setting mechanism.
Practical Action
The action responsibility asks what the user can realistically do next.
Insights
+
Facts
+
Uncertainty
+
Emotional Context
│
▼
┌──────────────────────┐
│ Action Reasoning │
│ │
│ What can the user │
│ actually do next? │
└──────────┬───────────┘
│
▼
Small, realistic,
low-regret action
The objective is not to produce a long generic checklist.
The action should follow from the reasoning performed by the other roles.
4. Collaboration Rather Than Concatenation
A naive multi-agent implementation could ask several agents for independent answers and concatenate the results.
That is not the architecture 觀音AI is trying to achieve.
NOT:
Agent A ──► Answer A ─┐
Agent B ──► Answer B ─┼──► concatenate ──► user
Agent C ──► Answer C ─┤
Agent D ──► Answer D ─┘
The intended architecture is collaborative:
┌────────────────┐
│ Fortune │
│ Perspective │
└───────┬────────┘
│
▼
┌────────────────┐ challenge / ┌────────────────┐
│ Emotional │◄── constrain ────►│ Rational │
│ Context │ │ Analysis │
└───────┬────────┘ └───────┬────────┘
│ │
│ influence │
└──────────────┬─────────────────────┘
▼
┌───────────────┐
│ Practical │
│ Action │
└───────┬───────┘
│
▼
┌───────────────┐
│ Synthesis │
└───────┬───────┘
│
▼
Final Response
A symbolic interpretation can therefore be challenged by objective reasoning.
Emotional context can alter the timing of an otherwise rational recommendation.
Practical-action reasoning can reject an interpretation that produces no useful next step.
Synthesis resolves those contributions into one response.
5. System Instructions as Architecture
In conventional software, product behavior is primarily encoded in source code.
LLM applications introduce another architectural artifact:
instructions.
The system instructions define responsibilities such as:
┌──────────────────────────────────────────────┐
│ SYSTEM INSTRUCTIONS │
├──────────────────────────────────────────────┤
│ │
│ Product identity │
│ │
│ Role of 觀音靈簽 │
│ │
│ Role of AI │
│ │
│ Agent responsibilities │
│ │
│ Collaboration rules │
│ │
│ Reasoning boundaries │
│ │
│ Safety / unsupported-claim boundaries │
│ │
│ Response behavior │
│ │
└──────────────────────────────────────────────┘
This makes prompt engineering part of architecture rather than merely copywriting.
Changing an instruction can alter system behavior as materially as changing application code.
Instructions therefore need to be treated as versioned product artifacts.
6. Runtime Reasoning Flow
At runtime, the architecture can be understood as a pipeline of context construction, collaborative reasoning and synthesis.
┌──────────────────┐
│ Current Message │
└────────┬─────────┘
│
│
┌────────▼─────────┐
│ Relevant Fortune│
│ Context │
└────────┬─────────┘
│
│
┌────────▼─────────┐
│ Conversation │
│ Context │
└────────┬─────────┘
│
▼
┌─────────────────────────────┐
│ Construct Reasoning Context │
└──────────────┬──────────────┘
│
▼
┌─────────────────────────────┐
│ Multi-Agent Collaboration │
│ │
│ symbolic │
│ rational │
│ emotional │
│ action │
└──────────────┬──────────────┘
│
▼
┌─────────────────────────────┐
│ Response Synthesis │
└──────────────┬──────────────┘
│
▼
┌─────────────────────────────┐
│ Final User Response │
└──────────────┬──────────────┘
│
▼
┌─────────────────────────────┐
│ Local Conversation History │
└─────────────────────────────┘
The final response becomes part of the local conversation context available to future interactions.
7. Conversation Continuity
The architecture does not treat each message as an isolated transaction.
Previous conversations can affect what becomes useful next.
┌────────────────────┐
│ Conversation │
│ at Time T │
└─────────┬──────────┘
│
▼
┌────────────────────┐
│ Reasoning + │
│ Suggested Actions │
└─────────┬──────────┘
│
▼
┌────────────────────┐
│ Local History │
└─────────┬──────────┘
│
user returns
│
▼
┌────────────────────┐
│ Relevant Previous │
│ Context │
└─────────┬──────────┘
│
┌──────────┼───────────┐
│ │ │
▼ ▼ ▼
New update Action Emotional
question follow-up follow-up
│ │ │
└──────────┼───────────┘
▼
┌────────────────────┐
│ Conversation │
│ at Time T+1 │
└────────────────────┘
This architecture allows starter prompts to become contextual follow-up prompts.
Instead of:
Ask me about your career.
the system can generate a continuation such as:
Were you able to speak with your manager?
The value comes from continuity, not simply from retaining transcripts.
8. Privacy Boundary
The privacy architecture follows directly from the type of conversations the system is designed to support.
The strongest boundary is the physical device.
INTERNET
│
│
──────────────┼──────────────
│
PRIVACY BOUNDARY
│
┌───────────────────────────┴────────────────────────────┐
│ iPhone │
│ │
│ ┌──────────────┐ │
│ │ User Message │ │
│ └──────┬───────┘ │
│ ▼ │
│ ┌──────────────┐ │
│ │ Conversation │ │
│ │ Context │ │
│ └──────┬───────┘ │
│ ▼ │
│ ┌──────────────────────┐ │
│ │ Multi-Agent System │ │
│ └──────────┬───────────┘ │
│ ▼ │
│ ┌──────────────────────┐ │
│ │ On-Device Inference │ │
│ └──────────┬───────────┘ │
│ ▼ │
│ ┌──────────────────────┐ │
│ │ Local LLM │ │
│ └──────────┬───────────┘ │
│ ▼ │
│ ┌──────────────────────┐ │
│ │ Final Response │ │
│ └──────────┬───────────┘ │
│ ▼ │
│ ┌──────────────────────┐ │
│ │ Local History │ │
│ └──────────────────────┘ │
│ │
└───────────────────────────────────────────────────────┘
The architectural goal is straightforward:
Private conversational content should not need to cross the device boundary for AI inference.
This is stronger than relying exclusively on transport encryption or server-side retention policies because the architecture removes the remote inference hop itself.
9. Cloud AI vs On-Device AI
The difference is easiest to see in the data path.
Conventional Cloud Inference
User
│
▼
iPhone
│
│ private prompt
▼
Network
│
▼
Cloud AI Infrastructure
│
├── inference
│
└── response
│
▼
Network
│
▼
iPhone
The prompt must leave the device for inference to occur.
觀音AI On-Device Inference
┌────────────────────── iPhone ──────────────────────┐
│ │
│ User │
│ │ │
│ ▼ │
│ Conversation │
│ │ │
│ ▼ │
│ Multi-Agent Reasoning │
│ │ │
│ ▼ │
│ Local Inference │
│ │ │
│ ▼ │
│ Response │
│ │
└────────────────────────────────────────────────────┘
The architectural difference is not merely where computation occurs.
It changes which party needs access to the conversation.
10. Local Model Deployment
On-device inference turns the model into part of the application's physical deployment architecture.
A representative deployment footprint is approximately:
Gemma on-device model
│
├── model file ~2.5–2.6 GB
│
└── XNNPACK/runtime cache ~0.75 GB
─────────
effective footprint ~3.2 GB
This creates requirements that do not exist when an application simply calls a remote API.
The application has to consider:
- model acquisition;
- local storage;
- runtime initialization;
- cache generation;
- available device memory;
- model replacement;
- application lifecycle;
- device compatibility.
The model is therefore not simply a dependency identified by a string.
It is a large deployed asset.
11. Runtime Stack
The on-device architecture separates the application from the model runtime.
┌───────────────────────────────────┐
│ Swift / iOS │
│ │
│ UI │
│ Conversation management │
│ Context management │
│ Agent orchestration │
│ Local persistence │
└────────────────┬──────────────────┘
│
▼
┌───────────────────────────────────┐
│ On-Device AI Runtime │
│ │
│ Model loading │
│ Tokenization │
│ Inference │
│ Runtime cache │
└────────────────┬──────────────────┘
│
▼
┌───────────────────────────────────┐
│ Local LLM Model │
│ │
│ Gemma / LiteRT-LM model asset │
└───────────────────────────────────┘
This separation allows product logic and orchestration to evolve independently from the underlying model.
It also leaves open the possibility of replacing the model or runtime without redesigning the entire application.
12. Memory Is the Hard Constraint
Model file size affects storage.
Runtime memory affects whether the system can execute at all.
This is especially important when supporting devices such as an iPhone with 4 GB of physical RAM.
The model does not have exclusive access to that memory.
Physical RAM
│
├── iOS
│
├── system services
│
├── application UI
│
├── application data
│
├── AI runtime
│
├── model state
│
├── context / KV cache
│
└── temporary allocations
Therefore:
Model fits on disk
≠
Model can run reliably
Storage capacity and runtime viability are separate engineering questions.
This becomes particularly important for a multi-agent system because additional reasoning passes can increase inference workload even when all agents share the same underlying model.
13. One Model, Multiple Responsibilities
A multi-agent architecture does not necessarily require multiple model files.
Agent identity is primarily a reasoning responsibility.
Multiple agents can share one local model while operating with different instructions and context.
┌──────────────────┐
│ Local LLM Model │
└────────┬─────────┘
│
┌────────────────┼─────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Fortune │ │ Rational │ │ Emotional │
│ Role │ │ Role │ │ Role │
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
└────────────┬───┴──────────┬──────┘
│ │
▼ ▼
┌─────────────┐ ┌─────────────┐
│ Action │ │ Synthesis │
│ Role │ │ Role │
└─────────────┘ └─────────────┘
End of Architecture Notebook