#32  Helm: improve service by spilitting HTTP and SSH services and adding many settings
Merged
Commits were merged into target branch
nex opened 2 years ago

I only saw #30 a similar PR after I've done this modification. I saw that one merged but didn't find the code in main branch, strange. I will open it anyway since I've done more than that PR.

Firstly it's better spilitting HTTP and SSH services. In my case for example, I need the HTTP service to be of type ClusterIP to use it with my ingress controller, for the SSH though, I use the type LoadBalancer and add annotations to create a load balancer on Alibaba Cloud.

I also needed the externalTrafficPolicy field on the SSH service to be set to Local to preserve source IP. But I've add more settings that could be useful in some use cases:

service:
  http:
    # type: ClusterIP
    port: 80
    # clusterIP: None
    #loadBalancerIP:
    #nodePort:
    #externalTrafficPolicy:
    #externalIPs:
    loadBalancerSourceRanges: []
    annotations:
  ssh:
    # type: LoadBalancer
    port: 22
    # clusterIP: None
    #loadBalancerIP:
    #nodePort:
    #externalTrafficPolicy:
    #externalIPs:
    loadBalancerSourceRanges: []
    annotations:

Mostly adapted from Gitea's chart here.

The default behavior and the default settings of the chart remains unchanged.

New commits added 2 years ago
Robin Shen commented 2 years ago

Thanks for the suggestion. Greatly appreciated! Will look into this.

As to pull request #30: before that, OneDev does not have helm deployment yet (only has k8s resource based deployment). So I merged it, but found that it needs many tweaks for production use. So I decided to work out a minimum usable but robust deployment based off that contribution.

Robin Shen commented 2 years ago

@nex I started a new gke cluster to run below command to deploy the chart:

helm install onedev ./chart --namespace onedev --create-namespace

After deployment, service onedev-http and onedev-ssh are using ClusterIP by default, and no longer has external ip assigned. I hope that LoadBalancer is used by default, which is the easiest to get started for new users, and this can give them confidence to move on.

Also for default set up, I hope to use same service for ssh and http, for below reasons:

  1. Only a single load balancer is allocated. Load balancer is quite expensive.
  2. You have only one external ip address assigned for http and ssh service, and you can use a single dns name for them

I understand that separating these two services is necessary for some teams. Maybe we can add an option in values.yaml to switch to this behavior if necessary.

In general, I hope that things start simple, and flexibility is achieved with additional options. This is what helm deployment works currently (progressively support ingress and letsencrypt): https://code.onedev.io/projects/162/blob/main/pages/deploy-into-k8s.md

nex commented 2 years ago

@robin

Did you set .Values.ingress.host ? If that one is set, service type will be set to LoadBalancer otherwise ClusterIP, that's the original behavior of the chart, which I meant to preserve.

What I've changed was that if you explicitly set .Values.service.http/ssh.type, it wil override the above original behavior and use whatever type you set, but in the default values.yaml I left the options empty so it shouldn't affect the default behavior.

nex commented 2 years ago

This is the original service.yaml, you have to set ingress.host option for the service to be LoadBalancer:

spec:
  {{- if not .Values.ingress.host }}
  type: LoadBalancer
  {{- end }}
nex commented 2 years ago
  1. Only a single load balancer is allocated. Load balancer is quite expensive.

Multiple services can use the same LoadBalancer, most cloud providers should support this by specifying the same instance ID in the annotations.

You have only one external ip address assigned for http and ssh service, and you can use a single dns name for them

I see, in my case I've configured my load balacner to forward port 22 to the pod, and forward port 80 and 443 to my ingress controller, so I can use only one external IP address and a single DNS name, but generally I got your point.

Robin Shen commented 2 years ago

spec: {{- if not .Values.ingress.host }} type: LoadBalancer {{- end }}

Doesn't this mean: use LoadBalancer if ingress host is NOT specified?

nex commented 2 years ago

Yeah my bad, I missed the not..., I will update the PR to stick to the original behavior.

{{ $type := $http.type | default (ternary "LoadBalancer" "ClusterIP" (not .Values.ingress.host)) }}
New commits added 2 years ago
Robin Shen commented 2 years ago

This is great. Thank you 👍

nex commented 2 years ago

You're welcome. I'll think about how to structure the template to support both single/spilit service use case.

PS 中文通吗?

Robin Shen commented 2 years ago

太棒了,非常感谢。说实话,我对Kubernetes/Helm了解的也不是很深入,只是了解到够我完成应该做的功能而已。

New commits added 2 years ago
nex commented 2 years ago

I've updated the PR, added an option service.ssh.separateService to control whether to use a single service or separate ones, the default behavior is false (single service):

service:
  main:
    port: 80
    #nodePort:
    # type: ClusterIP
    # clusterIP: None
    #loadBalancerIP:
    #externalTrafficPolicy:
    #externalIPs:
    loadBalancerSourceRanges: []
    annotations:
  ssh:
    # If separateService is enabled, a separate service is created for SSH with the following options,
    # otherwise, a port for SSH will be added in the main service with the following options.
    separateService: false
    port: 22
    #nodePort:
    ## The following options are only useful when separateService is set to true
    # type: LoadBalancer
    # clusterIP: None
    #loadBalancerIP:
    #externalTrafficPolicy:
    #externalIPs:
    loadBalancerSourceRanges: []
    annotations:
nex commented 2 years ago

PS 不客气,最近刚发现这个项目想要部署到自己集群尝试一下,very impressed你一个人开发出这么完整的平台,好奇你的主业做什么的 😁

Robin Shen commented 2 years ago

感谢贡献 ??,下班后我看下。我工作上主要是负责配置管理方面的事情。动作也不算快,日积月累吧。

Robin Shen merged 2 years ago
Robin Shen commented 2 years ago

我这边先合并,然后一些小的问题我这边来改。

pull request 1 of 1
Submitter nex
Target main
Source daot/onedev-server:feat/helm-improve-service
Assignees
Merge Strategy
Create Merge Commit
Watchers (2)
Reference
pull request onedev/server#32
Please wait...
Page is in error, reload to recover