If responsibility is distributed, fragmented, and time-dependent,
then evaluation cannot remain the same.

Most current approaches still focus on evaluating:

• individual agents

• isolated outputs

• performance at a specific moment in time

This works when systems are stable,
and when outcomes can be attributed to discrete components.

But in multi-agent systems, those conditions no longer hold.

2. The problem with individual evaluation

When responsibility fragments, individual evaluation becomes misleading.

An agent may appear high-performing,
while systematically degrading the performance of others.

Another agent may appear weak,
while enabling capabilities that only become visible later.

In such systems, performance is not only about what an agent does,
but about how it shapes the system around it.

Evaluation that focuses only on individuals
misses this entirely.

3. A shift in what we evaluate

This suggests a shift in perspective.

From:

• evaluating agents

• ranking outputs

• optimizing for immediate performance

To:

• evaluating the system as a whole

• observing interaction patterns

• understanding how capabilities evolve over time

In other words:

The primary object of evaluation is no longer the agent,
but the ecosystem.

4. What ecosystem-level evaluation means

This does not mean abandoning individual metrics.

But it does mean interpreting them differently.

What starts to matter is not only:

• how well an agent performs
  but also:

• how it interacts

• how it contributes

• how it affects the system’s ability to grow and adapt

Some questions begin to replace others:

• Is the system becoming more capable over time?

• Is diversity being preserved or suppressed?

• Can the system recover from disruption?

• Are new types of tasks becoming possible?

These are not properties of individual agents.
They are properties of the system.

5. Why time matters

One of the most significant shifts is temporal.

In many current frameworks, evaluation happens at a fixed point:

after a task is completed,
after an output is produced,
after a decision is made.

But in complex systems, value does not always appear immediately.

Some contributions:

• enable future capabilities

• become relevant only in different contexts

• are recognized only when extended by others

This creates a gap:

between when something is done,
and when its value becomes visible.

6. From point-in-time to multi-horizon evaluation

If value unfolds over time, evaluation must do the same.

Instead of a single moment, evaluation becomes distributed across horizons:

• immediate signals (what is happening now)

• short-term outcomes (what worked in this context)

• medium-term adoption (what gets reused or extended)

• long-term impact (what changes the system’s capabilities)

This does not eliminate uncertainty.
But it makes space for it.

7. A different kind of signal

This also changes what we look for.

Not only correctness or performance,
but patterns:

• convergence that may indicate lock-in

• amplification that may signal feedback loops

• suppression that may reduce diversity

• unexpected reuse that signals generative value

Evaluation becomes less about scoring,
and more about sensing.

8. Why this matters

If evaluation remains tied to individuals and immediate outcomes,
systems will optimize for what is easy to measure.

• simple outputs will be rewarded

• complex contributions will be ignored

• diversity will collapse into uniformity

Over time, this does not make systems more capable.
It makes them more fragile.

Ecosystem-level evaluation is not just a refinement.
It is necessary for sustaining capability in adaptive systems.

9. What this changes

Taken together, this leads to a different role for evaluation.

Not as a mechanism to rank or judge,
but as a way for the system to understand itself.

Evaluation becomes:

• distributed rather than centralized

• continuous rather than episodic

• interpretive rather than purely quantitative

It does not produce a final answer.
It shapes the next iteration.

10. Where this leaves us

Across these pieces, a pattern begins to emerge:

• responsibility no longer aligns

• governance cannot rely on assignment

• systems must be designed differently

• evaluation must move beyond individuals

This is not a small adjustment.

It is a shift in how we think about
responsibility, governance, and intelligence itself.

If responsibility no longer aligns—and cannot be cleanly assigned—then governance cannot rely on assignment alone.

Defining roles more precisely, tightening controls, or enforcing accountability more strictly may still be necessary.
But they are no longer sufficient.

The problem is not simply that responsibility is unclear.
It is that the system itself no longer supports a stable point where responsibility can sit.

This shifts the question.

Not how to assign responsibility more effectively,
but how to design systems where responsibility can remain meaningful at all.

2. The limits of control

Most governance approaches are built on a control logic:

• define what agents should do

• restrict what they must not do

• monitor compliance

• intervene when necessary

This works when systems are predictable and boundaries are stable.

But in multi-agent environments, systems evolve through interaction.
They adapt, reconfigure, and generate outcomes that are not fully specified in advance.

In such systems, tighter control does not necessarily produce better governance.

In some cases, it does the opposite:
It reduces diversity, suppresses useful deviations, and obscures the very signals that indicate something is going wrong.

Control does not fail because it is weak.
It fails because it is applied to the wrong structure.

3. A different starting point

If responsibility is no longer something that can be located at a single point,
then governance cannot be designed around that assumption.

Instead, governance needs to start from a different premise:

Responsibility is not only assigned.
It is made possible—or constrained—by the structure of the system itself.

This reframes governance as a design problem.

Not only about rules,
but about the conditions under which those rules operate.

4. Designing for governed autonomy

One way to approach this is to separate what must remain fixed from what must remain adaptive.

Not everything in the system should be tightly controlled.
But not everything should be left open either.

A useful distinction begins to emerge:

• There are elements that must always hold, regardless of context

• There are elements that should adapt dynamically to the task

• There are patterns that indicate when the system is behaving pathologically

• There are feedback processes through which the system learns over time

These are not layers in a strict architectural sense,
but they point to different roles that governance needs to play within the system.

5. What this looks like in practice

Without going into a full framework, this distinction already begins to shape design choices.

If certain principles must always hold, then they need to be:

• simple enough to be consistently enforced

• fundamental enough to apply across contexts

If behavior must adapt dynamically, then:

• agents need the ability to interpret context

• governance cannot be fully pre-specified

If system failure is not always visible through performance, then:

• we need signals that detect patterns, not just outcomes

• monitoring shifts from evaluation to anomaly detection

If value emerges over time, then:

• evaluation cannot be limited to a single moment

• systems need ways to revisit and reinterpret past actions

These are not implementation details.
They are consequences of how responsibility behaves in the system.

6. What changes

Taken together, this leads to a shift in how governance is understood.

From:

• assigning responsibility to components

• enforcing compliance at defined points

• evaluating outcomes in isolation

To:

• shaping conditions under which responsibility remains traceable

• enabling autonomy within shared constraints

• observing patterns across interactions and over time

This is less about governing agents,
and more about governing the space in which agents operate.

7. Not a complete model

This is not a complete framework.

It is an attempt to move away from a model that no longer holds,
toward one that better reflects how these systems actually behave.

There are still open questions:

• How minimal can shared constraints be without losing reliability?

• How do we prevent systems from exploiting those constraints?

• How do we intervene without collapsing autonomy into control?

These are design questions, not just governance questions.

8. What this sets up

Once governance is understood in these terms,
another challenge becomes visible.

If responsibility is distributed, adaptive, and time-dependent,
then how should it be evaluated?

Not at the level of individual agents,
but at the level of the system itself.

9. What comes next

In the next piece, I’ll explore how this changes the way we think about evaluation—
and why measuring individual performance may no longer be enough.

In the previous piece, I argued that responsibility in multi-agent systems does not disappear.
It becomes distributed across the system.

But “distributed” is still too vague.

What actually happens is more specific—and more problematic.

Responsibility does not just spread.
It fragments into different layers that no longer align.

2. What we assume (without noticing)

Most governance models rely—often implicitly—on a simple expectation:

That different aspects of responsibility point to the same place.

The one who knew,
the one who caused,
the one who decided,
and the one who is held accountable—

in many traditional settings, these tend to overlap enough for governance to function.
But that alignment was never guaranteed.
It was a property of the system.

3. A provisional way to see it

This is not meant to be a definitive model of responsibility.

If anything, it is a provisional lens—a way to make visible something that tends to remain collapsed in current discussions.

I suspect that in multi-agent systems, responsibility may need to be understood less as something assigned after the fact, and more as something that is structurally enabled—or constrained—by the system itself.

But before going there, it is useful to at least surface how different aspects of responsibility already begin to diverge.

4. Where responsibility starts to split

One way to make this visible is to look at responsibility along multiple dimensions.

For example:

• Epistemic — who knew, or should have known
  This shapes how information is captured, shared, and made visible across agents.

• Causal — who influenced or contributed to the outcome
  This determines whether interaction chains can be reconstructed, or remain opaque.

• Decision — who enacted or triggered an action
  This affects where execution authority sits—and how actions are attributed in the system.

• Normative — who is expected to answer for it
  This defines how responsibility is surfaced externally—to users, organizations, or regulators.

These are not abstract distinctions.
They map to different parts of system design.

5. A simple way to see the problem

Consider an outcome that emerges from a sequence of interactions:

One agent processes partial information.
Another transforms or amplifies it.
A third executes an action based on that transformed input.
A human operator may or may not be aware of the chain.

If something goes wrong, which layer do we follow?

If we follow knowledge, we trace who had access to relevant information.
If we follow causality, we reconstruct the chain of influence.
If we follow decisions, we look at the point of execution.
If we follow norms, we ask who is expected to be accountable.

These paths do not necessarily converge.

6. Misalignment is not an exception

It is tempting to treat this as an edge case—something that happens only in complex or poorly designed systems.

But in multi-agent environments, this misalignment is not accidental.

It is structural.

As systems become more distributed, more interactive, and more adaptive over time, the alignment between these layers becomes harder to maintain.

At some point, it breaks.

7. Why this matters for governance

Most governance frameworks still assume that responsibility can be stabilized.

That if we define roles clearly enough,
or trace decisions precisely enough,
we can map responsibility back to a coherent point.

But when these layers diverge, that mapping becomes unstable.

We can still assign responsibility.
But the assignment no longer reflects how the system actually works.

This is where governance begins to lose its grip—not because it is absent,
but because it is misaligned with the structure of the system.

8. A shift in focus

If alignment cannot be assumed, then the goal cannot simply be to restore it.

Instead, the question becomes:

How do we design systems where different layers of responsibility
remain traceable—even when they no longer coincide?

This shifts the focus.

From locating responsibility,
to preserving its structure.

Not as a single point,
but as a set of relationships across the system.

Responsibility may not be something we locate—
but something we either make possible, or fail to.

9. What this sets up

This fragmentation is not just a conceptual observation.

It has direct implications for system design:

• how information flows are structured

• how interactions are recorded

• where authority is placed

• and how accountability is surfaced

These questions lead to a different design space—one that moves beyond assigning responsibility, toward structuring it.

10. What comes next

In the next piece, I’ll explore what this shift looks like in practice—
and how it changes the way we think about governance itself.

There is a growing sense that AI governance is not keeping up.

Organizations are investing more than ever in compliance frameworks, oversight mechanisms, and risk controls. The structures are there. The effort is there.

And yet, when something goes wrong, the same question keeps surfacing:

Who is actually responsible?

The uncomfortable reality is that, in many cases, no one can clearly answer that question.

This is often interpreted as a failure of governance. But that interpretation may be missing something more fundamental.

2. It’s not that governance is missing

The default explanation is familiar. Governance fails, we assume, because rules are insufficient, controls are weak, or accountability has not been clearly defined.

So the response is predictable: more structure, clearer roles, tighter enforcement.

But this response rests on a deeper assumption—one that is rarely questioned:

That responsibility can be clearly located within the system.

I’ve noticed a subtle shift in how organizations respond when things go wrong.
The conversation is no longer only about fixing the issue.
It’s also about understanding how it happened — across systems, teams, and increasingly, across AI components.

In some cases, the answer is not a single mistake, but a chain of interactions that no one fully owns. This is still manageable in simpler systems. But as systems become more distributed and agentic, this pattern starts to break down.

3. That assumption no longer holds

That assumption made sense in systems that were relatively stable—systems where decision-making followed identifiable paths, and where outcomes could be traced back to discrete actors.

Multi-agent systems operate differently.

They are dynamic rather than static. Distributed rather than centralized. Outcomes are not the result of a single decision, but of ongoing interactions—between agents, across layers, and over time.

In such environments, what we call a “decision” is often less a moment and more a process.

4. Responsibility hasn’t disappeared

It is tempting to conclude that responsibility has simply become unclear, or worse, that it has disappeared.

But neither is quite right.

Responsibility is still there. What has changed is its form.

It no longer resides in a single place. It no longer aligns neatly with a role, a function, or a moment of decision.

Instead, it becomes distributed across the system itself—embedded in interactions, dependencies, and evolving states.

5. A structural mismatch

This is where the tension emerges.

Most governance models are still designed around stability. They assume that there is a definable decision point, an identifiable decision-maker, and a clear path for assigning and enforcing responsibility.

But in multi-agent systems, these assumptions begin to break down.

Decisions emerge. Influence is shared. System behavior evolves in ways that are not reducible to any single component.

The issue, then, is not simply ambiguity.

It is a structural mismatch between how governance is designed and how these systems actually operate.

6. A shift in the question

If responsibility cannot be cleanly assigned, then perhaps the problem is not how we assign it, but how we think about it.

The question is no longer:

Who is responsible?

But rather:

How should responsibility be designed in systems where no single actor fully holds it?

This is not just a governance problem. It is a system design problem.

7. What this opens up

Taking this seriously shifts the focus.

Instead of asking how to tighten control, we begin to ask:

How can responsibility remain traceable even when it is distributed?

How do we avoid systems where responsibility is formally assigned but practically absent?

What kinds of structures make responsibility legible—not just at a single moment, but over time?

These are not questions that can be resolved through more rules alone.

They point toward a different way of thinking about governance—one that is less about control, and more about how systems are shaped.

8. What comes next

In the next piece, I’ll look more closely at how responsibility actually fragments in multi-agent systems—and why those fragments don’t align in the way we expect.

When digital tools entered sales organizations in the 1990s, veterans pushed back hard. “Sales is human connection,” they said. “You can’t replace that with a bulk email.” When computers entered engineering design floors, senior engineers warned that the tacit knowledge embedded in hand-drawn schematics would vanish. When digital audio emerged, music educators insisted that children’s ears — and souls — would be damaged by something artificial.

They were not entirely wrong. Something did change. But what changed was not the disappearance of human value. What changed was the structure of how humans think — and therefore what kinds of thinking became most valuable.

We are in the middle of that same transition again. This time, the technology is AI. And this time, the shift goes deeper than any before it.

Part I: When Knowledge Was the Asset

For most of human history, the person who knew the most held the most power. The scholar, the elder, the craftsperson who had accumulated decades of experience — their value was inseparable from the knowledge stored in their memory.

Thinking in this era was inherently contextual. Decisions were made based on accumulated wisdom, direct observation, and situational judgment. There were no systems to enforce consistency. Intelligence meant knowing — and knowing deeply.

Then came writing. Then printing. Then the age of recording and transmission. Knowledge could be captured, copied, shared. And with that, the definition of intelligence quietly began to shift.

Each shift in how we record and process information reshapes how we think, decide, and create value.

Part II: When Access Became the Advantage

The digital era did not simply add more knowledge. It reorganized the entire architecture of how information works.

Search engines, databases, enterprise systems — these tools made retrieval fast and retrieval consistent. The competitive advantage moved from who knew the most to who could access, structure, and apply information most efficiently. Rules became central: if you could encode knowledge into systems, you could scale it. Consistency replaced context as the dominant value.

In parallel, something subtler happened. The way organizations rewarded people shifted. Deep contextual judgment — the kind that resists standardization — became harder to measure and therefore harder to value. Efficiency, reproducibility, and rule-following rose. Situational wisdom receded.

The digital era delivered clarity and scale. But it quietly narrowed what we recognized as intelligence.

This was not a conspiracy. It was a natural consequence of designing systems optimized for consistency. When your environment rewards rule-following, rule-following is what you optimize for.

Part III: When Questions Become the Scarcest Resource

Now comes AI — and with it, a fundamental inversion.

For the first time in history, answers are cheap. Not just fast: cheap. Any question you can articulate clearly enough will yield a fluent, confident, often accurate response within seconds. The marginal cost of an answer is approaching zero.

This changes everything about where value lives.

I have watched this play out in consulting work. When a junior colleague asks AI to help build a marketing website, they typically ask: “How do I make a site like this?” or “Give me a table of contents for a company website.” The result is technically complete — and almost entirely misses the point. It answers a procedural question without ever engaging with the strategic one.

A more experienced practitioner asks differently. Not “how do I build this” but “who is this for, and what do we want them to feel?” Not a complete brief delivered upfront, but an open-ended conversation: “What am I missing? What assumptions am I making that I shouldn’t be?” They use vague questions deliberately — because vagueness creates space for the AI to surface what the human has not yet thought to ask.

Give the same prompt to ten people, and you will get ten different answers — not because the AI changed, but because the questions were different.

The difference is not technical skill. It is the capacity to design a question — and then to interrogate the answer.

This second step matters as much as the first. AI produces responses that are fluent and confident. That fluency is dangerous. A response that sounds right is rarely questioned. But an answer that fits your existing assumptions perfectly should raise a flag, not lower your guard.

The ability to ask a second question — and a third — is where the real thinking happens. “Is this actually what I was looking for? What’s missing here? What would someone with a completely different frame say about this?”

The Mirror Problem

There is a pattern I have observed consistently: when people use AI, they tend to get back a refined version of what they already believed.

A client who suspects the answer is X will frame their question in a way that produces X. A junior analyst who thinks in terms of process will generate process-heavy outputs. The AI is not biased toward their view — it is responding to the shape of the question they asked. The question is a mirror. And most people do not realize they are looking into one.

This is why the capacity to question is not simply a skill. It requires something prior: awareness of your own assumptions. In physics, the first move of rigorous thinking is to question your premises. What are you taking for granted? What is the invisible frame inside which your thinking is happening?

Without that self-awareness, no amount of prompt engineering will help. You will simply get more confident-sounding reflections of your own blind spots.

Rethinking What Makes a Person Valuable

If the above is true, then the basis of human value at work is shifting — and it is shifting faster than most organizations have recognized.

In the age of knowledge, value was stored in memory. In the age of access, value was in efficiency and structure. In the age of questions, value lies in judgment: the ability to see clearly, to challenge assumptions, to design questions that open new possibilities rather than confirm existing ones.

This is not abstract. It has immediate implications for how we hire, how we train, how we evaluate contribution, and how we build organizations. The person who can ask the right question — and then challenge the answer — is not doing less than the person who used to generate the answer manually. They are doing something harder and rarer.

Knowledge can be stored. Access can be automated. Judgment cannot be outsourced.

History Is Not Repeating — But It Rhymes

Every major technological transition has been accompanied by moral panic about the loss of something irreplaceable. And in every case, what was lost was real — but what emerged was also real.

Digital tools did diminish certain forms of memory and craft. They also enabled forms of coordination and creativity that were previously impossible. The question was never whether to resist or adopt. The question was always: what do we carry forward, and what do we consciously redesign?

We are at that question again. AI will commoditize answers. What it cannot commoditize is the human capacity to ask questions that matter — questions that arise from genuine curiosity, from ethical responsibility, from the willingness to challenge what seems obvious.

That capacity is not new. It is, in many ways, a return. It is what Socrates was doing. It is what good scientists, good designers, and good leaders have always done. What is new is that this capacity is now the primary source of human advantage — not a supplement to it.

We are not at a turning point in technology.

We are at a turning point in how humans think.

The shift from knowledge to access took decades. The shift from access to questions is happening now — faster, and with higher stakes. The organizations, communities, and individuals who understand this will not simply adapt. They will define what comes next.

If thinking itself is changing — what assumptions about our society are we still taking for granted?

— Mari Sekino