Unfortunately setting up workspace via remote docker via agents or something similar is not supported due to the fact that workspace files need to be persisted unless workspace is deleted. Putting it on agents or transient pods means that workspace files get lost if agent is disconnected or pod is moved/terminated.

The reason this cannot be supported on OneDev server running on k8s is the same reason: we cannot find an approach to start pod with volume mapping directly to server workspace storage area.

Thanks for the explanation. That makes sense.

Maybe there are two separate but related approaches that could solve this in a Kubernetes-compatible way:

1. Kubernetes Workspace Provisioner

This provisioner would create a dedicated PVC per workspace and start a workspace pod mounting this PVC as /onedev-workspace.

Lifecycle idea:

• create PVC when the workspace is provisioned
• prepare repository/config/auth files either via init container or helper job
• start workspace pod with the PVC mounted as /onedev-workspace
• keep the PVC while the workspace exists
• delete the pod when the workspace is stopped/reprovisioned
• delete the PVC only when the workspace is deleted
• use Kubernetes exec/attach for terminal and command execution

This would avoid the need to mount the OneDev server workspace storage into arbitrary pods. The workspace storage would be Kubernetes-native and tied to the workspace lifecycle.

2. Remote Docker Workspace Provisioner with a persistent agent

For environments where workspaces should run outside the OneDev Kubernetes cluster, a remote Docker workspace provisioner could use a small persistent agent running on a dedicated Docker host.

The important difference from a plain remote Docker socket would be that the workspace storage is owned and persisted on the remote Docker host, not on the OneDev server pod.

Possible architecture:

• OneDev acts as control plane
• remote workspace agent runs on a hardened Docker host
• agent has a persistent workspace root, for example /var/lib/onedev-workspaces
• each workspace gets its own directory, for example /var/lib/onedev-workspaces/
• agent starts Docker containers with that directory mounted as /onedev-workspace
• agent keeps the directory until OneDev explicitly deletes the workspace

Initial provisioning flow:

• OneDev asks the agent to create/provision workspace
• agent creates the workspace directory locally
• agent initializes/clones the repository into /onedev-workspace/work
  • using the clone URL and temporary workspace token provided by OneDev
  • or via a streamed repository/archive from OneDev if direct Git access is not desired
• agent writes required config/auth/cert/setup files into the workspace directory
• agent pulls the configured image, using registry credentials provided by OneDev if necessary
• agent starts the container with:
  • /var/lib/onedev-workspaces/:/onedev-workspace
  • environment variables such as ONEDEV_SERVER_URL and ONEDEV_WORKSPACE_TOKEN
  • configured resource limits, user, env vars, config files, user data, caches, etc.
• agent reports container status and port mappings back to OneDev

Runtime flow:

• opening a terminal would be forwarded from OneDev to the agent
• the agent performs docker exec / attach locally and streams stdin/stdout/stderr back to OneDev
• git commands requested by OneDev would be executed by the agent inside the workspace container or in the workspace directory
• logs/status would be streamed or periodically reported back to OneDev

Stop/restart flow:

• stopping the workspace removes/stops the container only
• the workspace directory remains on the Docker host
• reprovisioning starts a new container using the same persistent directory
• if needed, the agent can perform cache/user-data synchronization before stopping

Delete flow:

• when the workspace is deleted in OneDev, OneDev sends a delete request to the agent
• agent stops/removes the container
• agent deletes /var/lib/onedev-workspaces/
• agent deletes any associated cache/user-data directories for that workspace if applicable

Providing files back to OneDev:

Depending on OneDev’s desired ownership model, there could be two modes:

A) Runtime-owned storage

• workspace files remain on the remote Docker host for the lifetime of the workspace
• OneDev only requests operations through the agent
• on delete, files are removed
• on backup, the Docker host workspace root needs to be included in backup strategy

B) Sync-back mode

• agent keeps files locally during runtime for performance
• on workspace stop/delete/reprovision, agent can package selected state and upload it back to OneDev
• for example: git worktree state, user data, caches, logs, metadata
• this is more complex but keeps OneDev as the central persistence owner

For our use case, mode A would already be very useful. We want to place the Docker host in a locked-down network segment and only allow very limited traffic:

• OneDev Kubernetes network -> agent or Docker host control port
• Docker host/workspace containers -> OneDev HTTPS for API, registry, and possibly Git
• Docker host/workspace containers -> OneDev SSH only if Git over SSH is used
• admin VDI -> Docker host SSH
• no dynamically exposed workspace ports initially

This would allow OneDev running in Kubernetes to use a hardened external Docker host without needing to mount the OneDev server’s workspace storage into that host or into transient pods.

The key design principle in both approaches is that the workspace runtime owns persistent workspace storage and keeps it until the workspace is deleted. This avoids the transient-agent/pod problem while still making the feature usable for Kubernetes installations.

Also, if this is something that would fit the long-term architecture of OneDev, we would be happy to explore contributing such a provisioner/agent implementation ourselves and upstreaming it to the project.

We believe this could make the workspace feature much more attractive for Kubernetes-based installations and for organizations that need stronger isolation between the OneDev control plane and the actual workspace runtime environment.

If this direction sounds reasonable to you, we would appreciate any guidance regarding:

• which extension points or abstractions would be the best starting point
• whether there are architectural concerns we should be aware of before investing time into a prototype

We would be very interested in collaborating on this.

Thanks for your time for such a detailed proposal. We will investigate the PVC approach in future versions, and as with current docker and shell provisioner, the workspace pod must be on same node as OneDev server.

The remote docker host approach is too complicated and needs big changes to OneDev workspace architecture and agent management, and I think this great complication is simply not worthwhile for the workspace feature.

Also want to mention that in future versions, AI user will be able to use current CI/CD facility to work on assigned issues and submit PR automatically. In this mode, code is considered cheap and it can be created again easily by AI user even if CI/CD job pod crashes. This feature will make the manual workspace mode less important.

Hi,

I'm a user with an enterprise subscription. I just experimented with the new Workspaces feature. Thank you so much. This is going in the right direction. I have exactly the same problem: workspaces should run on different machine(s) from the production environment where the OneDev server is running (there are multiple reasons for that; no need to mention them here, I believe).

So, even if it's not possible to change whatever the current approach is, I believe we could access a third option to use SSH or some other way to access remotely and deploy workspaces there (it would be incredible if we could use Docker containers remotely using DOCKER_HOST, for instance, or something else).

Thanks for your feedback. Running workspace remotely means that OneDev server should take care of workspace storage remotely in case the remote docker container or pod dies, and this can complicate things a lot. We will explore that in future versions, but no promise that this will be implemented.

build OD-7442 is able to run workspaces both on Kubernetes and on remote agents.