Set up Test Server
- Launch a AWS EC2 c5a.xlarge instance with Ubuntu Server 18.04 image and 50G GP2 disk. Configure security group to allow all incoming TCP ports for our test convenience.
- Login to launched instance as user
ubuntu
, edit/etc/hostname
to set host name asserver
, and then reboot - Install GitLab EE follow its installation guide (this installs
14.10.2-ee
at the time of testing). Specify environment variableEXTERNAL_URL
ashttp://server
during install - After install, login to GitLab and disable gravatar to avoid out-of-server resource loading from
Admin/Settings/General/Account and Limit
. Also disable auto devops to save cpu cycles fromAdmin/Settings/CICD/Continuous Integration and Deployment
. - Add a blank public project
kubernetes
underroot
user, without ticking the optionInitialize repository with a README
- Install OneDev by running below commands
ubuntu@server:~$ sudo apt update
ubuntu@server:~$ sudo apt install -y unzip openjdk-8-jdk
ubuntu@server:~$ wget https://code.onedev.io/downloads/projects/160/builds/2730/artifacts/onedev-7.3.2.zip
ubuntu@server:~$ unzip onedev-7.3.2.zip
ubuntu@server:~$ mv onedev-7.3.2 onedev
- Continue to edit file
/home/ubuntu/onedev/conf/wrapper.conf
to changewrapper.java.maxmemory.percent
to20
, which means to use 20% of physical memory as the default 50% is too deluxe for our tests - Start OneDev from terminal by running
/home/ubuntu/onedev/bin/server.sh console
- Login to OneDev to set it up. Disable gravatar from menu
Administration / System Setting
- Add a new project
kubernetes
. Edit general setting of the new project to changedefault role
asCode Read
. Also switch to menuAdministration / Security Setting
to enable anonymous access
Test Repository Push/Clone Performance
- Launch another EC2 instance with same subnet, hardware spec and OS as test server. Configure security group to allow all incoming TCP ports for our test convenience.
- Login to new launched instance, name it
test
by editing/etc/hostname
and reboot - Edit
/etc/hosts
to add an entryserver
pointing to private ip address of server machine - Login to
test
as userubuntu
and get kubernetes source from GitHub:
ubuntu@test:~$ git clone https://github.com/kubernetes/kubernetes
ubuntu@test:~$ cd kubernetes
ubuntu@test:~/kubernetes$ git reset --hard 92285fd74e8906bc4910d6fb6729b7ec0e74f427
Here reset master branch of the repository to same commit as I am doing the test
3. Stop GitLab on server
by running sudo gitlab-ctl stop
), and keep OneDev running
4. Measure repository push time to OneDev by running:
ubuntu@test:~$ cd ~/kubernetes
ubuntu@test:~/kubernetes$ time git push http://<user>:<pass>@server:6610/kubernetes master:master
Here <user>
and <pass>
should be replaced with OneDev administrator user/pass
5. Continue to measure repository clone time from OneDev by running:
ubuntu@test:~/kubernetes$ cd ~
ubuntu@test:~$ time git clone http://server:6610/kubernetes kubernetes-onedev
- Now stop OneDev on
server
by pressingctrl-c
from its console window, and start GitLab again by runningsudo gitlab-ctl start
- Measure repository push time to GitLab by running:
ubuntu@test:~$ cd ~/kubernetes
ubuntu@test:~/kubernetes$ time git push http://<user>:<pass>@server/root/kubernetes.git master:master
Here <user>
and <pass>
should be replaced with GitLab administrator user/pass
5. Continue to measure repository clone time from GitLab by running:
ubuntu@test:~/kubernetes$ cd ~
ubuntu@test:~$ time git clone http://server/root/kubernetes.git kubernetes-gitlab
Test Web UI Performance
- Login to
server
machine to start both OneDev and GitLab - Login to
test
machine to run below commands:
ubuntu@test:~$ cd kubernetes
ubuntu@test:~/kubernetes$ git push http://<OneDev user>:<OneDev pass>@server:6610/kubernetes eb88daeeae4a53a20579620e1791e30416517223:refs/heads/prod
ubuntu@test:~/kubernetes$ git push http://<GitLab user>:<GitLab pass>@server/root/kubernetes.git eb88daeeae4a53a20579620e1791e30416517223:refs/heads/prod
Here <OneDev user>
, <OneDev pass>
, <GitLab user>
and <GitLab pass>
should be replaced by OneDev/GitLab administrator user/pass respectively
- Login to OneDev to create a pull request from branch
master
toprod
for kubernetes project. Do the same for GitLab - Login to
test
machine, run below command to install chrome and jmeter:
ubuntu@test:~$ sudo apt update
ubuntu@test:~$ sudo apt install -y openjdk-8-jdk
ubuntu@test:~$ wget https://code.onedev.io/projects/406/raw/master/jmeter.tar.gz
ubuntu@test:~$ tar zxvf jmeter.tar.gz
ubuntu@test:~$ wget https://code.onedev.io/projects/406/raw/master/google-chrome-stable_current_amd64.deb
ubuntu@test:~$ sudo apt install ./google-chrome-stable_current_amd64.deb
Here we download a pre-configured JMeter package from code.onedev.io
. If you want to start from scratch with official JMeter package, you will need to:
- Install JMeter plugin manager
- Install Selenium webdrive plugin
- Install chrome webdrive for Linux AMD 64
- Generate rmi keystore
- Copy folder
bin/tests
from pre-configured JMeter package to your own JMeter directory
We also use Chrome package from code.onedev.io
as its version is 101
, which is the same as the webdrive executable. You may download from Google directly, but make sure the version matches.
- Run below command to make sure our tests can run without any errors:
ubuntu@test:~$ cd apache-jmeter-5.4.3/bin
ubuntu@test:~/apache-jmeter-5.4.3/bin$ ./jmeter -n -t tests/onedev/tree.jmx
- Press
ctrl-c
to stop tests. Now we need to launch more EC2 instances to run chrome as a single test machine is not powerful enough to drive OneDev to use nearly 100% cpu - Launch a AWS EC2 c5a.2xlarge instance with Ubuntu Server 18.04 image and 16G GP2 disk. Configure security group to allow all incoming TCP ports for our test convenience
- Login to newly launched instance, edit
/etc/hosts
to map nameserver
to private ip address of the server running OneDev and GitLab - Install JMeter and Chrome on this instance following instructions in step 4 above
- Run below command to run JMeter server:
ubuntu@ip-xxx-xxx-xxx-xxx:~$ cd apache-jmeter-5.4.3/bin
ubuntu@ip-xxx-xxx-xxx-xxx:~/apache-jmeter-5.4.3/bin$ ./jmeter-server
- Repeat step 7~10 to run JMeter server on another three instances. You may consider creating AWS image to start JMeter server instances easier
- Login to
test
machine, edit propertyremote_hosts
of fileapache-jmeter-5.4.3/bin/jmeter.properties
to include private addresses (separated by comma) of all JMeter server instances launched above - Now you can run distributed tests from the
test
machine with below command:
ubuntu@test:~$ cd apache-jmeter-5.4.3/bin
ubuntu@test:~/apache-jmeter-5.4.3/bin$ ./jmeter -n -t <path to JMeter test plan>
Here <path to JMeter test plan>
should be replaced by path to various .jmx file under tests directory. Make sure to stop GitLab when testing OneDev, and vice versa. Each test plan will last for 10 minutes, and will print test summary when it finishes. Make sure to run each test plan for 1 minute to warm up the server before formal test. Running test can be stopped by executing ~/apache-jmeter-5.4.3/bin/stoptest.sh
.
While test is running, you may run command htop
on the server
machine to monitor cpu and memory usage. For test plan tests/onedev/tree.jmx
, one more instance needs to be launched to run JMeter server (and remember to add its private ip address to jmeter.properties
on test
machine as explained above), otherwise server cpu can not reach 100%
Test Result
https://robinshen.medium.com/performance-compasion-of-onedev-and-gitlab-c11fc27b25be
Any question, please contact me at [robin AT onedev DOT io]