| Both sides previous revision Previous revision Next revision | Previous revision |
| bmas [2026/06/19 15:41] – pedroortega | bmas [2026/06/19 15:44] (current) – [Introduction] pedroortega |
|---|
| //Keywords: AI safety, bounded agency, multi-agent systems, modularity, verification, governance. // | //Keywords: AI safety, bounded agency, multi-agent systems, modularity, verification, governance. // |
| |
| Position Paper, June 2026 | {{ bmas_position.pdf | Position Paper, }} June 2026 |
| ===== Abstract ===== | ===== Abstract ===== |
| |
| The scaling paradigm treats bounds on compute, memory, information, authority, and affordances as obstacles. This paper argues that these bounds are also design variables. A **bounded agent** is an agent whose information, architecture, resources, or affordances limit its ability to optimise its objective. A **bounded multi-agent system** (BMAS) composes such agents through explicit interfaces so that system-level capability is obtained through decomposition, delegation, verification, and coordination. | The scaling paradigm treats bounds on compute, memory, information, authority, and affordances as obstacles. This paper argues that these bounds are also design variables. A **bounded agent** is an agent whose information, architecture, resources, or affordances limit its ability to optimise its objective. A **bounded multi-agent system** (BMAS) composes such agents through explicit interfaces so that system-level capability is obtained through decomposition, delegation, verification, and coordination. |
| |
| The safety claim is architectural. If unsafe behaviour requires a conjunction of capabilities, then risk can be reduced by separating those capabilities across components and controlling the interfaces that recombine them. BMAS therefore turns some global safety problems into local specification, monitoring, and verification problems. This does not eliminate multi-agent risk; it makes the relevant risk-bearing structures explicit. | The safety claim is architectural. If unsafe behaviour requires a conjunction of capabilities, then risk can be reduced by separating those capabilities across components and controlling the interfaces that recombine them. BMAS therefore turns some global safety problems into local specification, monitoring, and verification problems. The approach does not eliminate multi-agent risk, but it makes the relevant risk-bearing structures explicit. |
| |
| ===== Introduction ===== | ===== Introduction ===== |
| BMAS starts from a different decomposition. A task induces a capability profile: reasoning, knowledge, coding, planning, tool use, verification, and communication may all be required in different proportions. Unsafe behaviour also has a capability profile. Risk is highest when the capabilities required for task completion and the capabilities required for harm are colocated in one agent. | BMAS starts from a different decomposition. A task induces a capability profile: reasoning, knowledge, coding, planning, tool use, verification, and communication may all be required in different proportions. Unsafe behaviour also has a capability profile. Risk is highest when the capabilities required for task completion and the capabilities required for harm are colocated in one agent. |
| |
| The BMAS proposal is to design systems in which useful capability is distributed across bounded components. A planner plans. A retriever retrieves. A coder writes code. A verifier checks outputs. A monitor inspects interactions. An executor acts under controlled authority. The system becomes capable through composition, while each component has a restricted scope. | The BMAS proposal is to design systems in which useful capability is distributed across bounded components. A planner plans. A retriever retrieves. A coder writes code. A verifier checks outputs. A monitor inspects interactions. An executor acts under controlled authority. And so forth. The system becomes capable through composition, while each component has a restricted scope. |
| |
| ===== Bounded agents ===== | ===== Bounded agents ===== |