#400  Details on 1dev on k8s
Closed
Artur opened 3 years ago

Hi,

I am trying to understand how 1dev uses k8s environment and what would be advantages of running 1dev on k8s over simple Docker + external DB setup.

So, I have a few questions, assuming I would be using AWS EKS:

  1. Where does the 1dev runs build while runinng on k8s? Are they run on the same machine as the main 1dev server or on a separate instances/workers?
  2. If 1dev starts new workers, how many of them would be run concurrently? Do we have a control over it?
  3. How to adjust configuration for the k8s setup, like select certain EC2 type for the main 1dev server and/or workers?
  4. I know there are 2 persistent volumes created, one for MySQL database and another for 1dev server data. What would be the best way to run periodic backups on this data?
  5. Is there any special procedure to upgrade 1dev on k8s to a new version without a data loss?

My goal is to run a reliable and good performance 1dev service for our development team. With a decent 1dev main server and up to 3 build workers.

Any recommendations or suggestions would be very much appreciated.

Robin Shen commented 3 years ago

Where does the 1dev runs build while runinng on k8s? Are they run on the same machine as the main 1dev server or on a separate instances/workers?

This is determined by node selector of the k8s executor. To use k8s executor, delete the default auto-discover executor from menu administration/job executors and add executor of type Kubernetes Executor

If 1dev starts new workers, how many of them would be run concurrently? Do we have a control over it?

This is controlled by job cpu/memory requirement and node cpu/memory availability. If no available nodes satisfy job requirement, the job may be put into waiting state, or the cluster may spawn new nodes to run jobs if your cluster is configured to auto-scale. In auto-scale mode, you can normally control max nodes to be spawned.

How to adjust configuration for the k8s setup, like select certain EC2 type for the main 1dev server and/or workers?

The node for 1dev and database can be controlled by node-selector.yaml in production folder of OneDev's k8s deployment descriptors. Workers are controlled via job executor as explained above

I know there are 2 persistent volumes created, one for MySQL database and another for 1dev server data. What would be the best way to run periodic backups on this data?

EC2 has volume snapshot strategies which can be used to backup your persistent volumes.

Is there any special procedure to upgrade 1dev on k8s to a new version without a data loss?

Take a snapshot of your persistent volumes before upgrading 1dev is enough

Robin Shen changed state to 'Closed' 3 years ago
Previous Value Current Value
Open
Closed
Artur changed state to 'Open' 3 years ago
Previous Value Current Value
Closed
Open
Artur commented 3 years ago

Thank you for all the answers. I am currently trying to set it up on AWS EKS. The very basic setup is relatively simple but making it a real production system is not as easy.

I am trying to get SSL to work. And there seems to be a few ways to do that.

AWS recommended way is to setup additional load balancer and terminate SSL there. But there is no good documentation on doing it and connect all to k8s.

I noticed there is ssl folder in OneDev k8s package. It looks like OneDev can be setup to terminate ssl traffic. However, I am not certain on how to use it in connection with 'production' folder and I am also not certain whether OneDev would automatically renew the SSL certificate from letsencrypt.

Please advise.

Robin Shen commented 3 years ago

You may run a Nginx pod with letsencrypt on your cluster. This part does not relate to OneDev and I guess there are many posts on the web regarding this. Then you can configure your Nginx to forward traffic to OneDev server following instructions here:

https://code.onedev.io/projects/onedev-manual/blob/main/pages/reverse-proxy-setup.md (need to change localhost to host name of OneDev server pod)

Artur commented 3 years ago

Thank you for response.

I know that I can setup a separate proxy. I was curious however, if there is any "built in" support for this.

I am especially referring to ssl folder in the k8s folder of OneDev package.

Robin Shen commented 3 years ago

Check this for various options of configuring SSL certificate for OneDev:

https://code.onedev.io/projects/onedev-manual/blob/main/pages/https-setup.md

Robin Shen changed state to 'Closed' 3 years ago
Previous Value Current Value
Open
Closed
issue 1 of 1
Type
Question
Priority
Normal
Assignee
Issue Votes (0)
Watchers (3)
Reference
onedev/server#400
Please wait...
Page is in error, reload to recover