User Pods

Context and problem statement

Currently, harnesi provisions a dedicated Kubernetes pod and dynamic PersistentVolumeClaim (PVC) for every single chat conversation (k8s_runtime.py).

This causes or can cause:

1. Cold-start latency (15–30+ seconds): Dynamic cloud volume attachment and pod scheduling block the user on their first prompt
2. Fragility and timeouts: Streaming stdio over Kubernetes API server SPDY attach connections (connect_get_namespaced_pod_attach) frequently drops or times out
3. Resource Inefficiency: A user with 4 open tabs consumes 4 separate Pods and PVCs

We need to decouple the chat session lifecycle from the pod lifecycle so that 1 chat session = 1 pi process. Two architectures are being proposed.

Decision drivers

• Security and multi-tenancy: The agent executes untrusted code via bash and file tools
• Performance: Sub-second response times for starting new conversations
• Operational simplicity: Low architectural complexity and failure modes
• Credential isolation: Safe handling of user-specific OAuth tokens (Gmail, HubSpot, Sheets)

Considered options

• Option 1: Multi-User shared pods (True Multi-Tenancy) A shared pool of worker pods. Any Pod can run pi sessions for any user concurrently

• Option 2: User-Scoped pods (1 pod per active user, Multi-Session) Each active user gets one warm pod with their own persistent PVC. That single pod hosts all concurrent chat sessions for that user via an in-pod session daemon

Options Comparison

Option 1: Multi-User Shared Pods

Pros:

• Maximum Hardware Density: Best CPU/RAM bin-packing; fewer total pods needed in the cluster
• Predictable cluster sizing: Pool size scales with overall cluster load, not distinct active users Cons and risks:
• Breaks the Primary security perimeter: Pod/Linux namespaces no longer isolate users. Users share the same UID, /proc table, and loopback network. A user running bash("ps aux") or bash("ls/workspaces") can snoop on other users
• pi-permissions is not an OS Sandbox: Relying on application-level tool allowlists (pi-permissions.jsonc) to prevent cross-user tampering is fundamentally unsafe against shell evasion
• Credential bleed: In-pod OAuth proxies cannot easily distinguish which process owns which vendor token without complex per-session network namespace virtualisation
• Noisy neighbors: One user running a runaway loop or build can trigger the Linux OOM-killer and kill other users' active sessions

Option 2: User-Scoped Pods (1 Pod Per Active User)

Pros:

• Preserves OS-Level tenant isolation: Re-enforces agent-sandboxing.md. The pod remains the hard security perimeter; a compromised agent can only touch that user's own data
• Solves 95% of the Performance Bottleneck: Opening new chat sessions or tabs takes ~50ms (a local process fork inside the user's running pod) rather than 20 seconds
• Trivial Storage Model: Replaces dynamic per-conversation cloud PVCs with a single, stable PVC per user (pi-user-{user_id})
• Uniform Credentials & Permissions: All sessions in the Pod share the same user identity, OAuth credentials, and MCP server entitlements Cons:
• Higher Memory Footprint: Memory scales with active concurrent users rather than packed sessions (mitigated by idle-reaping Pods after 10–15 minutes of inactivity)

Comparison Matrix

Criteria	Current	Option 1 (Multi-User Pods)	Option 2 (1 Pod / User)
New session latency	15–30s (Slow)	~50ms (Instant)	~50ms (Instant)
Connection stability	Fragile (pod_attach)	Robust (HTTP/WS Daemon)	Robust (HTTP/WS Daemon)
Tenant isolation	Strong (Pod-level)	Severely Degraded (Shared OS)	Strong (Pod-level)
OAuth / Credential safety	Isolated	High Leakage Risk	Fully Isolated
Implementation complexity	Baseline	Extremely High	Moderate
Resource efficiency	Very Poor	Highest	Efficient

Recommendation & Advice

Option 2 is what the Kubernetes backend does: one User Pod per user, many Warm Sessions inside it (chat/k8s_runtime.py). The Docker backend used by ./dev.sh is still one container per conversation. See Warm pi Sessions.

Advised Decision: Go with Option 2 (1 Pod Per User)

Rationale:

1. Security cannot be compromised for coding agents: Because pi possesses raw shell execution capabilities (bash), true multi-tenancy inside a single container creates an unacceptable attack surface between untrusted users. Application-level guards like pi-permissions are insufficient to replace container boundaries
2. Delivers the intended performance gains: The user pain point (slow starts, timeouts) is eliminated. Users pay the pod startup cost only once per active day/login; all subsequent conversations and tabs start instantaneously
3. Much faster time-to-ship: Option 2 solves the storage and process multiplexing bottlenecks without requiring complex Linux namespace isolation, dynamic Landlock generation, or cross-user credential routers