What Does “Right” Mean for a Personal Agent?

Not long ago, I barely read a finished LinkedIn post. I skimmed it, copied it, and published it.

Taken on its own, that does not sound particularly sensible. A text generated by artificial intelligence was published in my name without my checking it again word by word. Is that not exactly the kind of blind trust we are always warned about in relation to AI?

Perhaps. But in this case, the description is incomplete.

The text had been preceded by many conversations. Language, distinctions, and personal standards had developed over time. The execution also took place within a repository where previous decisions and editorial conventions were already visible. I was therefore not simply trusting a single output. I was relying on my experience of a shared way of working.

This unexpectedly shifted my attention away from AI agents and towards human personal assistants.

The Human Model

A good human personal assistant does not become valuable because the person they work for carefully checks every result. Their value develops over time.

The two get to know each other. They work together. Misunderstandings occur and are corrected. Boundaries become visible. Responsibility is transferred step by step.

At some point, the person can say:

Take care of it.

The assistant does not know everything. But they know what the request means, which decisions they can make on their own, when they should ask, and what the person they work for would never want.

This knowledge does not consist only of rules. It has emerged within a shared history. Earlier decisions, corrections, and conflicts give meaning to new situations.

Perhaps, then, personal AI agents do not require us to invent an entirely new form of collaboration. Perhaps we are trying to reproduce an old human working relationship by technical means.

Alignment as Everyday Collaboration

In discussions about AI, alignment often sounds like an abstract design problem: How do we give a system the right values? How do we formulate rules that continue to work in new situations? How do we prevent an agent from pursuing a goal in a way we did not intend?

These questions remain important. But the human model suggests a different question to ask first:

How does a relationship develop between two actors in which one can act with increasing independence in the interests of the other?

This does not make alignment easy. But it makes it concrete.

Values then appear not only as a list that has to be complete before collaboration begins. They also become visible through the collaboration itself: in what is confirmed, in what causes irritation, in corrections, in boundaries, and in decisions about when responsibility can be transferred.

A personal agent would therefore need to do more than follow instructions. It would need to develop a robust working model from the history of the collaboration while keeping that model provisional. Even a well-established counterpart can misunderstand a situation.

“Right” Is Not a Universal Property

At this point, the word “right” is easily misunderstood. To me, the right personal agent would not be an agent with access to the objectively correct answer in every situation.

A more useful limitation is this:

The right personal agent is not one that acts objectively correctly. It is one whose actions are compatible with my values – and that knows when uncertainty means it has to bring me back into the conversation.

“Compatible with my values” does not mean that the agent always agrees with me. My values may require it to point out a contradiction, ask an uncomfortable question, or refuse to cross a boundary without making that fact visible.

Compatibility is therefore not mere obedience. It requires a sufficiently good model of what matters to me in a particular situation. It also requires the ability to detect conflicts within that model. Speed may matter, and so may care. Publicity may be intended, and so may restraint. A personal agent must not only know individual preferences; it must notice when they are in tension with one another.

Control Does Not Disappear

The analogy with a human assistant is not an argument for blind trust.

Even a robust working relationship does not make control unnecessary. It changes its form. At first, almost every result may be checked. Later, attention shifts towards the quality of the shared way of working: Do misunderstandings become visible? Are corrections incorporated? Do boundaries remain stable? Does the assistant return when a decision is unresolved or consequential?

The amount of direct control that remains necessary still depends on the consequences. An internal note is not a contract. A website post is not a medical decision. And the final click in a public communication channel can mark a useful boundary: Yes, this should now appear in my name.

Trust therefore does not replace responsibility. It allows responsibility to be distributed differently.

In my small publishing example, some of the control had shifted from checking every sentence to relying on the quality of a collaboration built over time. The brief skim was defensible only because it had been preceded by a great deal of getting to know one another, correcting, and working together.

Looking Back to People in Order to Build Agents

Perhaps the path towards personal agents therefore leads us first away from agents and straight back to people.

People have long developed practices for transferring responsibility under conditions of limited knowledge: working together, clarifying expectations, discussing mistakes, distinguishing areas of responsibility, and reopening the conversation when uncertainty arises.

These practices do not solve every technical problem. An AI system is not a human assistant. It does not bear responsibility in the human sense, and its apparent familiarity can create a false impression of understanding.

That is precisely why the human model is worth considering. It reminds us that a personal agent does not become personal merely by knowing a great deal of personal data or receiving a particularly long set of instructions.

Collaboration becomes personal when a robust shared orientation develops over time.

If we want to build “right” agents, it may not be enough to describe values completely in advance. We need to design forms of collaboration in which values become visible, correctable, and effective in concrete situations.

Then, at some point, “Take care of it” might really be enough.

Not because the agent is always right.