@servicedesk please investigate and provide options.

Don't assume. Ask any multiple choice questions if required in a numbered list with the options being a letter-based selection and provide the pros and cons and your recommendation.

@zaakiy Investigation findings and options below. Please pick a direction so we can proceed.

What happens today

When a Mac sleeps, the agent websocket drops. The server then marks the workspace inactive with Workspace stopped: Agent disconnected. The Docker container on the agent host is not stopped in that path (the agent is unreachable, so docker stop never runs).

Two cleanup/recovery gaps follow from that:

1. Delete while agent is offline — server-side cleanup explicitly skips deleting workspace storage when the pinned agent has no session (Workspace agent offline. Skipping deleting workspace directory...). So the container/dir can remain until someone removes them manually.

2. Agent DELETE_WORKSPACE only removes the workspace directory — even when the agent is online, it does not remove the matching Docker container (workspace-<provisioner>-<projectId>-<workspaceNumber>). Normal stop uses docker stop via the process killer; delete does not do docker rm. Orphans are especially likely after an abrupt disconnect (sleep), when stop never ran.

3. Reprovision does not reattach — current design always provisions a new container. On agent Docker it first runs docker rm -f for the same container name, then starts a new one. There is no “reconnect to the still-running container” path. Repo/workdir under the agent workspace dir can still be reused, but process state inside the old container is not.

So the reporter’s workaround (delete workspace in UI + manually remove the container) matches current behavior, especially when the agent was offline at delete time.

Options

Q1. Primary goal for this issue

A. Reliable cleanup only — after disconnect/delete, never leave orphan containers/dirs; reprovision stays “create fresh”.

• Pros: Smaller change; fixes the painful manual Docker cleanup; aligns with how reprovision already works.
• Cons: Does not restore an in-container session after sleep; user still loses process state and must reprovision.

B. Reattach/recover existing container after agent comes back — when workspace is inactive due to agent disconnect and the named container is still running, resume it instead of recreating.

• Pros: Best UX for laptop sleep; preserves running processes/terminals when possible.
• Cons: Much larger design (token/runtime rebinding, port mappings, shells, partial agent state); edge cases when container exited or image/spec changed.

C. Both — cleanup first (A), then reattach (B) as a follow-up.

• Pros: Stops orphan accumulation immediately; recovery can be staged.
• Cons: Longer overall delivery; need clear sequencing.

Recommendation: C, implement A first. Cleanup is clearly broken for the offline-delete case and incomplete even online; reattach is valuable but higher risk/scope.

Q2. Cleanup strategy when agent is offline at delete time (for A / C)

A. Queue cleanup until the agent reconnects — persist pending deletes; on agent online, remove workspace dir + matching container(s).

• Pros: Correct eventual cleanup without requiring the agent at delete time.
• Cons: Orphans linger until reconnect; needs durable pending-delete tracking.

B. Best-effort only when agent is online; document manual cleanup when offline — plus fix online path to also docker rm containers.

• Pros: Minimal server changes.
• Cons: Sleep/offline delete still leaves orphans (the reported case).

C. Agent-side sweep — on agent startup/reconnect, remove containers/dirs for workspaces the server says are gone (or unknown).

• Pros: Self-healing after sleep/restart; covers crash paths.
• Cons: Needs a server API or sync protocol so the agent does not delete still-valid workspaces.

Recommendation: A + C together if we take cleanup seriously: queue delete for known workspace removals, and add a reconnect sweep as a safety net. If we want the smallest fix first, do online docker rm + queued delete (A), sweep later.

Q3. Scope for the first implementation PR

A. Agent delete removes containers + queue offline deletes (no reattach).

B. Reattach only (cleanup unchanged beyond what reattach implies).

C. Full A+B in one change.

Recommendation: A.

Please reply with choices like Q1=C, Q2=A, Q3=A (or your variants). If you want different product behavior than recommended, say so and we will follow that.

@jakeceballos over to you.

@jakeceballos @zaakiy no more info needed on this issue. I will investigate and choose approach best suited for current implementation.