OneDev currently does not support rootless container.

@robin Note that user name space remaps is somewhat similar, but different than rootless docker.

With user namespace remap, the code in the docker container is still running as root from the perspective of that container.

Changing this to an improvement request to support either or both. I think it should be high priority due to the security issues associated with allowing execution in docker containers with root access.

Thank you!

Thanks for the info. Just have time to look into this. When run a docker based job, OneDev create an unique workspace for the build, and clone the repository into it. This is done outside of container, and this is the reason why command running in container has permission issue creating directories in the workspace (set as working directory automatically). The workaround is to edit /etct/subuid to make the mapped id starting from id of the host user running OneDev process. This way, root user in container will be mapped to host user running OneDev process. Make sure to restart docker daemon after changing /etc/subuid

Also you may run docker in rootless mode (yes, I verified that OneDev works with rootless mode). If OneDev is running in service mode, make sure to use docker context use rootless to instruct docker client to use user docker sock.

As to security issues for all onedev-build* directories, I do not think this will be a problem if mount docker sock option is not enabled. In this case, build script runs in container can not escape from container, and can only access its own onedev-build* directory mounted into the container. However if mount docker sock is enabled, you should also configure authorized jobs of the executor to make sure the executor can only be used by trust jobs.

Thank you very much @robin, I will give all that another try.

Wouldn't it enhance security to specify a user that the Server Docker Executor should use for that workspace and clone the repository etc.? This way different projects / jobs could be more isolated, and from the start the docker executor would not have the full range of permissions as OneDev itself?

There is some info on container escapes here that I will try to understand better into exactly what the requirements are .

At minimum we want to avoid --privileged, SYS_ADMIN capabilities, add --security-opt=no-new-privileges:true.

Perhaps we can add these options like --cap-drop ALL under More Settings / Run Options to make it more secure.

Thank you!

Running as root inside container will greatly simplify things. As long as docker sock mount and privilege is not enabled, I do not think there is any chance of container escape except for container bugs. I'd glad to know if I am wrong on this.

Also preparing workspace is quite involved, including setup credentials, symbol-linking caches, etc. These tasks are much easier to be handled out side of container.

@robin My suggestion was not to not allow root inside container or to prepare workspace inside container, but rather to allow the option of selecting a user in the ServerDockerExecutor setup for the workspace preparation separate from the user running OneDev as a whole. That user could then be remapped to a docker daemon for that particular docker executor. Multiple docker daemons with different --userns-remap options could also be set up to use one or another of these users to better isolate things.

Thanks for the suggestion. This will complicate things quite a lot, and even impossible. As mentioned previously OneDev has to handle caches and other complicate things outside of container, map container user to a different user than OneDev process owner will make OneDev impossible to clear cache, clear workspace, etc.

Unless there is an obvious security issue, this mode will not be changed.

Thanks @robin .

This will complicate things quite a lot, and even impossible. As mentioned previously OneDev has to handle caches and other complicate things outside of container, map container user to a different user than OneDev process owner will make OneDev impossible to clear cache, clear workspace, etc.

Well that should at least still be possible if OneDev runs as root. There might not be any way to make it work without OneDev being ran as root... we would need 3 different user tiers, where the workspace and the remapped docker user are still fully under the control of the OneDev user.

Unless there is an obvious security issue, this mode will not be changed.

I was thinking that there could be, but I am not an expert on any of this. Of course if at some point in the future when vulnerabilities are pointed out, then this issue might be worth re-considering.

The idea was that to separate the docker executor which can easily bring in arbitrary containers and code as soon as you have Code Writer role on any project and a Can be used by any job Server Docker Executor (by pushing a .onedev-buildspec.yml), from having the same full access as OneDev itself to the system (e.g., all projects hosted on a OneDev instance). I felt that restrictions on the user to which the executed docker is remapped could mitigate potential security issues that could result from a malicious Code Writer managing to set up an escaping container.

But that is just my two cents as someone quite new and unfamiliar with containers :)

Even if OneDev runs as root, as long as container is not mapped to same host user as OneDev, there still exist permission issue. Assume OneDev sets up workspace, and your code insides container wants to modify checked out files and commit, it will not be possible...

For job executors can be used by any jobs, you absolutely should not enable mount docker sock and add --privilege option. With this in mind, malicious users should not be able to do nasty things even if they are running as same OneDev user.

there still exist permission issue. Assume OneDev sets up workspace, and your code insides container wants to modify checked out files and commit, it will not be possible...

The idea was that OneDev would have chown'ed the whole job workspace to that remapped docker user before running the docker.

For job executors can be used by any jobs, you absolutely should not enable mount docker sock and add --privilege option. With this in mind, malicious users should not be able to do nasty things even if they are running as same OneDev user.

I hear that, but on one hand I am not convinced, and on the other I am not experienced/knowledgeable enough with docker and security to know for sure :) So I will cautiously take your word for it, and throw in --cap-drop ALL and --security-opt=no-new-privileges:true as a bonus hoping things are secure enough.

Thanks again! :)

It might not be obvious which host user OneDev should chown the workspace to. Also requiring OneDev running as root is somewhat too strict.

Thanks for this discussion. I am taking security seriously, let me know if there is real security issue here, and I will be open to change.

It might not be obvious which host user OneDev should chown the workspace to.

That is the option I was suggesting could be configured in the Docker Server Executor.

Also requiring OneDev running as root is somewhat too strict.

Indeed. I wish there were another way... but the chown job workspace option could have been only enabled when running OneDev as root.

Thanks for this discussion. I am taking security seriously, let me know if there is real security issue here, and I will be open to change.

Thank you very much for listening to my concerns and providing guidance. I will try to investigate this more when I have time and discuss the issue with people more knowledgeable about dockers and security than myself, and come back to you if there is a real issue.

That is the option I was suggesting could be configured in the Docker Server Executor.

I see. I was thinking you are mentioning the uid remap approach. So with -u option, user inside container will no longer be root and this will be very limited in containers.

@robin Yes still with the user namespace remap approach (but not -u for docker run, --userns-remap on dockerd, or { "userns-remap": "testuser" } in /etc/docker/daemon.json).

If OneDev runs as root, no issue to clean up caches etc. Server Docker Executor option to select a user to chown the job workspace to. I would set that chown user to the same user as the --userns-remap user option used to run the docker daemon, and that remapped user is root inside the container, but outside the container it is the same user owning the workspace that OneDev prepared (and then chowned to that user, while retaining control over it if OneDev is running as root).

The only issue with this I think is if you cannot or don't want to run OneDev as root: how to regain ownership after having chowned to the docker remapped user, to clean up caches etc.?

Perhaps one potential solution is something like a bind mount, which would allow to chown the bind mount while retaining ownership of the true job workspace being mounted there?

This option is used to start docker daemon instead of docker container. Also docker container will not be mapped to this user, it looks up /etc/subuid for entry of that user to determine which host user id to use. Or I am misunderstanding something here?

It could very well be me who is misunderstanding things.

How I understand users-remap is that a range of users IDs (specified in /etc/subuid) that will be used inside the container (including the 'root' user inside the container), will be remapped to the user namespace specified by --userns-remap when starting the docker daemon.

e.g., if /etc/subuid has testuser:231072:65536 and the docker daemon is started with --userns-remap testuser

Then this means 'root' inside the container is actually testuser / 231072 in the host, and all the other users inside the container (mapping to 231073..65536 in the host) are also in the testuser namespace.

That is the user namespace (testuser in this example) that I would put in the Server Docker Executor option and to which I would chown the workspace before running the docker (the first testuser entry 231072), so that the 'root' user inside the container in effect owns the job workspace (or the bind mount of the job workspace, while OneDev keeps control of the real thing to clean it up).

NOTE: I also noticed that the temporary job workspace have read permissions for all users, so other users on the system running OneDev have full visibility into all of them, and I think that is also an issue. I think this bind mount idea could allow OneDev only to have permissions on the job workspace, while the bind mounts reflect the different permissions and ownership that you want the docker to have.

Related: https://github.com/docker/roadmap/issues/398

Confirmed that you've misundertood users-remap here, 😉

/etc/subuid has one entry per user, when determine host user of a docker container user, docker will look for the entry of the user specified via option --userns-remap to get the starting uid, the host uid will be calcualted by adding container uid to this starting uid. In your example of testuser, the host uid will be 231072 + container uid, which will be 231072 for root container user whose uid is 0. Note that 231072 is normally not the same uid as testuser, instead it is normally a non-existant user in host system.

As to temporarity job workspace read permission, you may just change OneDev installation directory to make it less permissive (chmod o-x) if you do not trust other users in the machine running OneDev. Different systems (Windows, Linux) have different approach of restricting access, and OneDev leaves this to OneDev administrator.