A Conversation About Building 觀音AI

A Conversation About Building 觀音AI

Why rebuild a fortune-stick application around AI?

I would describe it differently.

I did not rebuild the fortune-stick application around AI. The original fortune-stick system already did its job well. A user draws or enters a stick number, and the application retrieves the corresponding interpretation. Generative AI is not inherently better at that.

The opportunity appeared after the interpretation.

A traditional interpretation cannot know why this particular person asked the question, what happened yesterday, what they are afraid of, what they have already tried, or what decision they are actually facing.

AI makes that second part possible.

So the product evolved from delivering an interpretation to helping the user think about what that interpretation might mean in the context of their actual situation.

Why not simply build an AI fortune teller?

Because I think that confuses two very different responsibilities.

I wanted 觀音靈簽 to remain responsible for intuition, symbolism and insight.

AI should contribute reasoning, objectivity, emotional steadiness and practical next steps.

If the AI simply says, "This is a good fortune, therefore your relationship will succeed," it has not added intelligence. It has converted symbolic material into an unsupported prediction.

That is exactly what I wanted to avoid.

The fortune can introduce a perspective.

AI can help examine that perspective.

The user still makes the decision.

Why did that require a multi-agent system?

Because a seemingly simple user question can contain several different problems.

Take this example:

My girlfriend said she needs some space.

It's been almost two days since she replied.

Should I send another message, or should I leave her alone?

There is potentially a fortune-stick context.

There are facts.

There are assumptions.

There is uncertainty.

There is an emotional response to that uncertainty.

And there is a practical decision.

Those are different reasoning responsibilities.

Instead of pretending that one undifferentiated AI role is equally responsible for all of them, 觀音AI separates the work across multiple agents.

The important idea is not "more agents."

It is separation of responsibility.

What do the different agents contribute?

One responsibility is the fortune or symbolic perspective.

It asks what insight the relevant 觀音靈簽 introduces without treating the fortune as factual evidence.

Another responsibility is rational analysis.

It asks what is actually known, what is assumed, what remains unknown, and whether alternative explanations exist.

Another looks at emotional context.

It considers whether anxiety, fear, frustration or urgency may be affecting how the user is interpreting the situation.

Another concentrates on practical action.

It asks what the user can realistically do next.

Then those contributions have to be synthesized.

The user does not need four or five separate answers. The user needs one useful response.

So are the agents basically several chatbots talking to each other?

That description is too simplistic.

The agents represent different reasoning responsibilities inside the same product.

What matters is that their contributions can constrain and improve one another.

Suppose the fortune perspective identifies patience as an important theme.

The rational perspective should be able to prevent that from becoming: "The fortune proves you should wait and everything will work out."

The emotional perspective might recognize that the urge to send another message is being driven by anxiety.

The practical perspective can then consider whether sending a message would improve the relationship or merely make the user feel less anxious for a few minutes.

That is collaboration.

The goal is not to simulate a room full of AI personalities having a conversation.

The goal is to improve the quality of reasoning by making different responsibilities explicit.

Why not put all those responsibilities into one very detailed prompt?

You can.

Modern language models can follow surprisingly complicated instructions.

A single prompt could tell the model to consider the fortune, remain objective, identify assumptions, recognize emotional context, propose actions and synthesize everything.

The problem is not that this is impossible.

The problem is that the responsibilities become harder to inspect.

When an answer is weak, what failed?

Did the model misunderstand the symbolic context?

Did it accept an assumption as fact?

Did it miss the emotional state?

Was the analysis good but the suggested action poor?

Did one instruction interfere with another?

Decomposing responsibilities gives the system more structure.

It also makes the architecture easier to evolve.

Does every agent need its own language model?

No.

An agent is a responsibility, not necessarily a separate model.

Multiple agents can use the same underlying on-device model with different instructions and context.

That distinction matters enormously on a phone.

If the model itself occupies gigabytes, duplicating a model for every conceptual agent would make no sense.

The logical architecture can contain several specialized agents while the physical implementation shares the same inference infrastructure.

Why hide the agents from the user?

Because exposing them would make the product harder to use without necessarily creating any user value.

I don't want someone with a relationship problem to first decide:

"Should I ask the emotional agent or the rational agent?"

That is the system's problem.

The user should simply talk to 觀音AI .

Internally, the architecture can be sophisticated.

Externally, the interaction should remain simple.

I think that distinction is important in AI product design. Internal complexity should not automatically become interface complexity.

Isn't there a danger that multiple agents simply make the system slower?

Absolutely.

Multi-agent architecture is not free.

Every additional reasoning pass can increase latency, compute usage and energy consumption.

That matters in any AI system, but it matters even more when inference happens on a mobile device.

This is why I don't think "more agents" is a useful goal.

An agent should exist because separating that responsibility improves the final result enough to justify its cost.

The architecture should be as complicated as necessary, but no more complicated than that.

Why run the AI on the device at all?

Because of the kinds of conversations I want the product to support.

The useful questions are often personal.

Relationships.

Career uncertainty.

Family problems.

Fear.

Regret.

Things someone may not want to say publicly—or even to people they know.

If the product encourages people to speak honestly, I have to think seriously about where those words go.

Running inference on the device changes the answer.

The conversation does not need to leave the phone simply to obtain an AI response.

That is a fundamentally different privacy architecture from sending every message to a cloud model.

But cloud AI can also be secure.

Of course.

This is not an argument that cloud AI is inherently insecure.

Cloud systems can use strong transport security, access controls, retention policies and many other protections.

The distinction I care about is architectural.

If a conversation does not need to leave the device, then a remote AI provider does not need access to that conversation in the first place.

For this particular product, that property is valuable.

Why does privacy matter to the quality of the AI response?

Because AI reasons from what the user tells it.

If someone censors the most important part of the story because they are uncomfortable sending it somewhere else, the system receives incomplete context.

Incomplete context can produce poorer reasoning.

So privacy is not only about what happens to data after a conversation.

Privacy can affect whether the honest conversation happens at all.

That was an important change in how I thought about the product.

Privacy stopped being something represented only by a privacy policy.

It became part of the experience.

What is difficult about running the model locally?

Almost everything becomes a constraint.

The model has to physically fit on the device.

It has to fit into available runtime memory.

Inference has to be fast enough to remain usable.

The runtime has to coexist with iOS, the application, conversation context and everything else consuming memory.

There are storage costs.

There are runtime caches.

There is battery consumption.

There is thermal behavior.

And different generations of iPhone have very different practical limits.

With a cloud model, much of that infrastructure is somebody else's problem.

With on-device AI, it becomes application engineering again.

How do you choose the "best" model under those constraints?

I don't think there is a universally best model.

There is a best trade-off for a particular product.

A larger model may produce stronger reasoning but require more memory and storage.

A smaller model may be faster but fail on nuanced conversations.

A model that performs beautifully on a recent high-end phone may be useless if the product needs to support devices with much less memory.

So the question is not:

Which model scores highest?

It is:

Which model provides enough intelligence to deliver the product experience within the hardware constraints we actually have?

That is a product-engineering decision as much as an AI decision.

Doesn't multi-agent reasoning make that hardware problem worse?

Yes.

That is one of the most important trade-offs.

Even if several agents share the same model, multiple reasoning passes still require computation.

Logical parallelism also does not mean I necessarily want several large inference operations executing simultaneously on a phone.

Parallel execution can increase peak memory pressure, thermal load and contention.

So there is a distinction between the logical architecture and physical execution.

Logically, several agents may have independent responsibilities.

Physically, their work may need to be scheduled much more conservatively against one shared local model.

The architecture has to respect the hardware.

How important is conversation history?

Very important, but not because I want people spending more time in the app.

The interesting value is continuity.

Suppose the system suggested that a user speak with a manager before resigning.

When that person returns three days later, a useful system can ask:

Were you able to speak with your manager?

That is more valuable than pretending the previous conversation never happened.

The AI can help with a process rather than only an isolated question.

Does that mean sending the entire conversation history back into the model every time?

No.

Stored history and active model context are different things.

A conversation can become very long.

Passing everything into every reasoning cycle would eventually become inefficient and may exceed practical context limits.

The system needs to determine what previous information is relevant to the current problem.

That might include an unresolved issue, a previous action, an important change in circumstances, or the relevant fortune context.

The full history can remain stored locally while the active reasoning context contains only what is useful now.

How did this change starter prompts?

They stopped being merely examples.

A typical AI product might show:

Ask me about relationships.

That is useful when the system knows nothing about you.

But if yesterday you discussed whether to contact your girlfriend again, a much better prompt might be:

Has she replied since we last talked?

Or:

Do you still feel like sending another message?

Or:

Would you like to think through what to say if you decide to contact her?

Starter prompts can become continuation prompts.

That turns conversation history into something operationally useful.

What surprised you most while building the system?

How quickly apparently separate product decisions became architecture decisions.

Conversation sounds like a UX feature.

Then conversation creates context.

Context creates memory.

Memory creates privacy concerns.

Privacy affects where inference should happen.

On-device inference creates model and hardware constraints.

Multi-agent reasoning creates orchestration and performance constraints.

Those constraints then affect what kind of conversational experience is practical.

The layers are much more connected than they initially appear.

Did AI also change the way you developed the application?

Yes.

AI became part of both the product and the engineering workflow.

I increasingly work through explicit artifacts: Markdown specifications, architecture notes, screenshots, visual references and implementation instructions.

Those artifacts provide context to coding agents working against the codebase.

That changes the bottleneck.

Generating code becomes cheaper.

Defining what should be built becomes more important.

Does that reduce the importance of being able to code?

It changes where the leverage is.

You still need to understand software well enough to know whether the generated implementation makes sense.

You need to recognize architectural problems.

You need to understand platform constraints.

You need to review what was produced.

You need to know when an AI agent has implemented the wrong interpretation of a requirement.

The valuable skill moves from producing every line manually toward directing a much faster implementation process without losing technical coherence.

What is the biggest danger of AI-assisted development?

Confusing generated possibilities with reality.

AI can generate ideas incredibly quickly.

It can generate a design.

Then another design.

Then an architecture.

Then another architecture.

Then code for a feature that was only being discussed.

If project discipline is weak, it becomes surprisingly easy to lose track of what was actually decided and what was merely generated.

That makes sources of truth more important, not less.

A proposed feature is not a shipped feature.

A generated screenshot is not an approved design.

A plausible architecture document is not necessarily the implementation.

AI increases the speed of exploration.

Engineering discipline has to keep up.

What would you do differently if you started again?

I would establish the responsibility boundaries even earlier.

It is tempting to begin an AI project by experimenting with models and prompts because those experiments produce visible results quickly.

But the more important questions are upstream.

What job is the traditional fortune stick performing?

What job should AI perform?

Which responsibilities need independent reasoning?

Which information needs to persist?

What absolutely should not leave the device?

What hardware must the product support?

Once those boundaries are explicit, technology choices become easier to evaluate.

What would you tell another technical leader building an agent-based product?

Do not start by asking how many agents you need.

Start by asking what responsibilities exist.

If two responsibilities have different objectives, different constraints or need to challenge one another, separating them may be useful.

If two agents would simply produce almost the same reasoning, they probably should not be two agents.

Agents are a means of structuring intelligence.

They are not the product.

And what would you tell someone building an AI feature into an existing product?

First ask whether AI should replace anything at all.

In 觀音AI , the deterministic fortune-stick system remained useful.

There was no reason to replace reliable retrieval with probabilistic generation.

AI was valuable somewhere else: understanding context and continuing the conversation.

Existing systems often contain parts that already work extremely well.

The goal should not be to maximize the amount of AI in the product.

The goal should be to place AI where its capabilities actually change what the product can do.

After all of this, is 觀音AI still a fortune-telling application?

Its roots are certainly there.

But I think the more interesting product is about reflection.

The fortune stick introduces a perspective.

The agents examine the situation from different directions.

The AI helps distinguish facts from assumptions.

It recognizes emotional context.

It helps identify practical next steps.

Then the user decides.

That last part is essential.

The purpose of the system is not to make decisions for people.

It is not to predict their future.

It is to help them think more clearly about what they might do next.

What is the most important technical lesson from the project?

Architecture should follow responsibility.

I could describe 觀音AI using models, runtimes, prompts, agents, orchestration, local storage and inference pipelines.

Those things matter.

But they became coherent only after the responsibilities became clear.

觀音靈簽 provides intuition, symbolism and insight.

AI provides reasoning, objectivity, emotional steadiness and practical next steps.

Different AI responsibilities can then be separated into collaborating agents.

The sensitivity of the conversation makes privacy important.

Privacy makes on-device inference valuable.

On-device inference creates hardware constraints.

Those constraints shape the implementation.

That sequence matters.

Start with the problem.

Define the responsibilities.

Then choose the architecture.