This is great! Thanks for your contribution 👍

It might not be very valueable to build package for head commit, as that is often a work-in-progress. What do you think?

As to documentation, please submit pull request to this project:

https://code.onedev.io/onedev/docs

It is a docusaurus project, and is published at:

https://docs.onedev.io

Thanks for clarification. It makes sense.

To get notification for releases, just subscribe to the release query of builds:

https://code.onedev.io/onedev/server/~builds

Thank you, I have added notifications for the builds.

Just to make sure, when a build does occur (release) it will email me a notification, correct?

Thanks for the support.

Yes that is true. Only when the release is successful, as the query requires the build to be successful. Release fails occasionally and in that case you will not be bothered.

Out of curiosity, when are builds triggered?

Is it when there is a new commit to the master branch, or when a new tag (release) is pushed to master?

@robin How would you sum up OneDev in one line, this is for the AUR package description :)

When a new commit is pushed, CI build will be running automatically. At some point, if I feel it is ready for a new release, I just pick up relevant CI build and promote it to release manually.

The one line description can be "Self-hosted Git Server with CI/CD and Kanban". Not very sexy, but I can not think of a better description...

The description sounds decent, better than anything I could think of xD

I am currently having two worries about this:

• Currently the only place to get the source code of OneDev is the builds sections, I worry that if lots of people build this it will kill the bandwidth of the server and will possibly cost you a lot of money.
• I do not see a way to fetch the build using the package name ("onedev") and the package version ("7.8.12"), this means it is a lot more annoying to maintain as every new release I will have to change the source of the package to the specific build. Is there any way to fetch the source by substituting in these two variables into the http link, makes maintaining the package more conventional and easier

Also what Java version would you recommend to run OneDev with, is there any flags you would feel that would improve the performance of OneDev, what Xmx and Xms values for example.

So after inspecting the zip which is created by the builds, I have noticed a few things:

• 3rdparty-licenses, I assume these are licences for the dependencies of onedev, however I am not sure which goes with which depedencies.
• lib and boot directories, seem to contain a large number of libraries, it would be unconventional for me to install them within a package meant to ONLY install onedev, I need a list of each dependency, its licence and what is used for (whether it is a hard dependency or a soft dependency). Any dependencies which are not in the AUR or arch repositories I need to package and then depend on, this is because of conventions which I must conform to when packaging.

I do not know where the final onedev server jar is, I can't find it anywhere within the build. I will check the docs now to see if they clarify anything.

You may call RESTful api to get download url of OneDev:

https://code.onedev.io/~help/api/io.onedev.server.rest.BuildResource/queryBasicInfo

Basically, you need to query builds using this query: "Project" is "onedev/server" and "Job" is "Release" and "Version" is "7.8.12"

From returned build object to get build number which can be used to construct the download url.

As to 3rd-party license, I guess it is sufficient to include these licenses required by 3rd party libraries, instead of declaring them one by one, which will be overly complicated and difficult to maintain, considering OneDev have hundreds of dependencies.

Lib and boot directory mainly contains JSW (https://wrapper.tanukisoftware.com/doc/english/home.html) which licenses to OneDev open source project for free distribution. This library is vital to wrap OneDev as OS services. If you only want to include OneDev jars itself without JSW, check the docker script for some reference:

https://code.onedev.io/onedev/server/~files/main/server-product/docker/entrypoint.sh

You also need to handle upgrade yourself by checking the docker upgrade instructions.

Before that, you may need to get familar with OneDev installation and upgrade first:

https://docs.onedev.io/category/installation-guide https://docs.onedev.io/upgrade-guide

The docker entrypoint also contains suggested JVM version and options to run OneDev.

@robin this makes deploying OneDev very difficult, especially how arch linux users will not want to install a proprietary wrapper for a java codebase when there is already open source alternatives which can be added as dependencies already.

Instead I might clone the master branch at the release tag, and then build it to a jar. This then allows execution with a JRE which can be depended on, which is not only more maintainable for me, but it also means OneDev would be more appealing for arch linux users, considering a lot of them would reject the idea of proprietary wrapper for an open source project.

In the future I recommend setting up a second job on the CI/CD to build to a jar as well as being wrapped. People who know what they are doing will benefit from configuring the JRE to their liking when running OneDev, and jar's are easier to deploy (preferably light jars without the dependencies included, because otherwise duplicate dependencies could be existant, for example if two codebases use similar dependencies and both compile them into the jar, it wastes storage space instead of sourcing the jar locally).

Upgrading OneDev might be an issue, however as far as I am aware all it does is upgrade the database version to the latest schema and then replace the current executables with the new ones. Therefore all I need to do (hopefully) is when the version is bumped in the package, that I set the installation to always execute the update script to update the database, but this could also be problematic depending on which database OneDev uses (as there is a selection), any ideas on how to do this wold be appreciated.

Just as mentioned above, you can build your package without JSW. The docker container distribution does that exactly. However for OneDev official distribution, JSW will be included as it will be very easy to run OneDev servivce on different platforms, either Ubuntu or RedHat, either Windows or Linux.

I'd suggest only to support one database for simplicity, and it is up to you to select which database to support (internal hsqldb, or postgresql etc).

@robin Unfortunately, I am not the kind of person which takes things the easy way, I intend to support every database when I finish the package, I will take the advice however and focus on supporting one database first, which will most likely be postgresql cause it has good support and also it is a solid choice.

Can I download the jar used for the docker container distribution somewhere, or am I going to need to build this manually? If so do you provide any compile guide for OneDev?

By the way I am aware I am eating up a lot of your time, there is no rush in replying to this issue (it is the reason I set it as minor priority), I just want to have all the info I need so the package I produce is to the highest quality possible. At the same time I am currently in a mailing list conversation with other AUR packagers to decide the best course of action for dealing with java dependencies within packages, because it is a large unsolved topic.

Thanks for giving your time to support me however :)

No problem for my time at all. Just feel a bit of tied of coding today and it is good to talk to OneDev users, 😃

To get OneDev jar, just get the source, enter into the top directory, and run mvn clean package. Then from within server-product/docker directory, you can run ./build.sh to build the docker container from jar files. This serves as a good reference on how you can run OneDev jar as well as how to do the upgrade.

@robin thanks I will read the wiki page about this

Does this produce an uberjar or just the jar for OneDev, preferably a jar without the dependencies is what would be more useful, I can add the dependencies to the classpath externally (which is a bit of a pain) but it provides more flexibility than a HUGE jar.

It does not produce uberjar. Instead multiple jars are produced, with dependencies remain as separate files.

Oki thanks

Hello!

I think I might've just beaten you to putting a package on the AUR - Ive been deploying OneDev at work so decided to whip up a PKGBUILD yesterday afternoon and committed it just now without looking at open issues here first.

Sorry for sniping you at the last minute, it definitely wasn't intentional. Let me know your AUR username and I'd be more than happy to add you as a maintainer if you'd like - I'm not the most experienced packager so any improvements are most definitely welcome.

@bengcooper I was half way through my AUR package, dammit!

For that I flagged your package as out of date, cause there is a new build :P

@bengcooper Your package does not affect mine, you are using the bundled version of OneDev, I plan to deploy without bundled JRE, therefore are packages do not conflict, I will rename my package to opendev-oss so that it is more clear.

Godamnit :P

I'm glad there's no conflict - I just went for the download and extract approach as I'm not a Java dev :)

Yeah, I plan to package mine without the proprietary dependencies.

However @bencinn you need to check your AUR package because you have not listed all the licenses, therefore you are breaking copyright laws, make sure to revise your package to update the licenses.

I have left a comment on your AUR package explaining what you will need to change :)

Good luck :)

Will do, thanks for the feedback :D

No problem 😃

@polarian Do you have any pointers r.e. using the bundled JRE rather than the system one? I'm not too familiar with how Java applications are packaged.

@bengcooper The JRE is bundled within the .zip package you download from OneDev, you do not need to worry about it, the script you run to start OneDev does all the hard work. Just remove the JRE from the package and it should still function :)

Good luck!

So I tried this when I first tried deploying it, but I got the following error/log output:

Running OneDev...
--> Wrapper Started as Console
Java Service Wrapper Standard Edition 64-bit 3.5.48
  Copyright (C) 1999-2021 Tanuki Software, Ltd. All Rights Reserved.
    http://wrapper.tanukisoftware.com
  Licensed to OneDev for Service Wrapping

The configured wrapper.java.command could not be found, attempting to launch anyway: java
Unable to start JVM: No such file or directory (2)

------------------------------------------------------------------------
Advice:
Usually when the Wrapper fails to start the JVM process, it is because
of a problem with the value of the configured Java command.  Currently:
wrapper.java.command=java
Please make sure that the PATH or any other referenced environment
variables are correctly defined for the current environment.
------------------------------------------------------------------------

Failed to launch the Java command.
<-- Wrapper Stopped

I wasn't able to locate another suitable executable in the archive (again this could be from total lack of experience) to set the wrapper command to, hence my inclusion of the system Java package. Am I missing something obvious?

Sorry I was mistaken, a JRE is still needed:

https://docs.onedev.io/installation-guide/run-on-bare-metal

Ignore my previous comments.

No problem at all 🙂 thanks for your help and support!

No problem, this has taken the pressure of me a little to produce a package for OneDev, I will be able to spend more time on mine to perfect it.

@robin why did you close it then reopen this? It is confusing :/

@polarian I am reopening as I feel that there might more needs to be done for your AUR package, especially upgrade.

OneDev upgrade currently works this way:

1. Take a backup of old program and database
2. Call old program to export data from database as XML
3. Replace old program with new program, and upgrade exported XML and then import upgraded data into database
4. If any of above failed, old program and database is restored to make sure old version still works.

OneDev upgrades exported XML as the same logic can be applied to all database types supported by OneDev, and I do not need to write database specific sql to perform upgrade.

Also XML should be exported by old program, as it may contain binary data only readable by old program. Once exported as XML, new program can handle the upgrade.

@bengcooper Please take note of this seen as you are the primary maintainer for the onedev package, I gotta worry about the -oss version without the proprietary wrapper :P

I am afraid this will not work for most package managers, as if I am not wrong, package managers only updates program files, and new program files are expected to perform in-place upgrade of old data.

I am afraid this will not work for most package managers, as if I am not wrong, package managers only updates program files, and new program files are expected to perform in-place upgrade of old data.

We can find methods of solving this issue, don't worry about it :)

I do not believe Ben is active on OneDev, I will inform him of the issue of the updating of the database, we might need to write a script for a user to execute to update the schema on every package update (it is a little annoying but I can't see another method of doing it).

I will discuss this with Ben to improve the AUR package, before working on the -oss version

Hi both,

I've been looking into how other packages manage this - and it appears that the consensus is that packages don't automatically run database migrations etc on package upgrades. I imagine this might be because there could other steps to upgrading as well, especially across major versions (e.g. config files, web proxy configuration, etc).

We could definitely automate running of upgrades via pacman hooks and a small shell script - similar to how the GitLab Arch Wiki page suggests - but I'm not sure if including it in the package by default is a typical thing to do?

In case you want to check current upgrade logic:

https://code.onedev.io/onedev/server/~files/main/server-core/src/main/java/io/onedev/server/commandhandler/Upgrade.java?position=source-37.14-37.21-1

It runs from new OneDev installation, and expects a param pointing to an old OneDev installation to be upgraded.

@bengcooper Yes, you are not meant to migrate, the package manager is designed to update the package, not to update the database, this can be solved by two ways:

OneDev detects outdated schema and updates the schema for the latest version (this is what is recommended, and what most codebases do)

User has to manually update it using the OneDev script, see the wiki page: https://docs.onedev.io/upgrade-guide/

@robin As discussed in issue #1099, is there any way to update the database AFTER installing the update?

The recommended method for packages is:

• Stop the service
• Update the package
• Run some script which migrates the database
• Start the service

But currently in your update guide it follows the current order

• Stop the service
• Get the updated source
• Execute the update script on the old source (and data)
• Replace the old data?!?!?
• Start the service

Any advice on how to make it comply with packaging guidelines would be useful :)

No it can not. The old program must be available to do the upgrade. It is because database has many binary serialized data which can only be loaded by old program.

It is the old program file, not the source code. But it means the same for package managers...

This mode saves me tons of time supporting multiple databases. I do not need to write migration sql for each of them. Also you can put freeform binary data in database and it can simplify things a lot.

So OneDev is not very suitable to be installed via package managers, but it is very consistent and simple using JSW.

Also according to my stats, most user are running OneDev as docker container, and I can put the upgrade logic into docker container transparently, so that user only needs to change the image tag to upgrade to new version.

To be efficient as one developer, this is one of the secret weapons I have, :😉

Upgrade can also be performed without JSW. Check the docker entrypoint shell script: https://code.onedev.io/onedev/server/~files/main/server-product/docker/entrypoint.sh

Not just system admins are using docker images. Most OneDev users here are using docker images according to my observation. Performance is not a problem at all.

Even with bare metal insallation, I disagree that this makes things hard for end users, as long as they follow official installation guide.

@polarian sorry I have to remove ArchLinux installation guide as this only cause confusions if it is not useable now. Thanks to git, your work is not lost, we can revert from history once ArchLinux package manager installation is complete.

Thanks for understanding that OneDev is a production ready software, and hope this will not hurt you.

Even without JSW, the old program still expects to be existed for upgrade to work. Sorry that this might not fit your flavor. But it works for users.

Upgrade script will update program files including all plugins. However do not remove these plugins as OneDev will almost be useless without these plugins.

Inside docker container, onedev works like below when upgrade:

1. /app will be updated to contain new program files
2. when start onedev, the start script always call below command first:

/app/bin/upgrade.sh /opt/onedev

3. then the start script calls /opt/onedev/bin/server.sh console to start onedev

I guess you should be able to do something similar. That is, the package always update /app. Once updated, packaging updating is considered successful. Step 2 and 3 is actually not part of packaging update, it is part of onedev start up process instead. This should hugely simplifies things.

Need to mention that once /opt/onedev is updated, calling /app/bin/upgrade.sh /opt/onedev again will not do any harm. So the docker start up script always call this as first command in onedev startup script.

That is true. /app will not be discarded even after upgrade, as it is always needed in the startup script. Only this way, you can consider running upgrade.sh part of onedev startup.

Please check docker entrypoint.sh like I mentioned before, and you will get some idea.

I guess package manager has some form of staging area where you can put new program files into that temporarily? If so, it will be simper as you may just use upgrade.sh in staging area to upgrade existing installation.

Also upgrade.sh is smart enough that if it found target directory does not get populated, it will populate automatically.

2 hours later, after a lot of documentation reading, I think I got a solution which will work.

Now time to test, fingers crossed it works!

What really slows me down is pulling builds from onedev.io

Good news? there is no good news

bad news? It didn't work

I am continuing to discuss this issue on the AUR mailing list, there is 2 promising solutions which I can use, which will actually work really well (if it does actually work).

I guess Ben (and me included) thought packaging onedev for the aur would be a simple task, we were so wrong xD

@robin Just as a little heads up, a little while back I created the ArchWiki page for OneDev, today it was added to two categories by the ArchWiki adminstrators:

https://wiki.archlinux.org/title/Category:Git_web_interfaces

https://wiki.archlinux.org/title/Category:Servers

You are now in line with other git instances, which is cool!

Great! Thank you for the effort!

@robin @bengcooper I have good news and bad news:

Good news, I have found a solution to the upgrade package problem.

Bads news, its a very complex solution and also needs a complete rework on the arch package we got so far :/

@polarian just realized there is another trouble: currently onedev agent is downloaded from server, and server expects JSW files to be exist and will use them to package agent.

Once agent is installed, it will be update itself automatically when server upgrades, and it relies on JSW to restart itself once program is updated.

Since many sides of OneDev assumes existance of JSW, it will be error prone to remove that part. Considering this situation, I'd suggest to discontinue the AUR packaging.

Sorry if this wastes too much of your time...

@robin is it possible to disable this automatic updating, self-updating software is against linux ideology

Yes this is possible, but it is very cubersome. You will need to maintain a server/agent version compatibility matrix, and check it to decide whether or not to upgrade the agent when server upgrades.

Once again, OneDev aims at easy maintenance and user friendly, and linux ideology is not its purpose.

OK OK, can you disable the automatic updates... if so how?

I am so close to actually getting this to work and I don't want to have to just give up

Also @robin where is the data for onedev stored?

It is stored under "site" sub-directory under OneDev' installation directory.

I mean like the database, issues... comments etc

It is stored under "sampledb" sub folder which is database files of HSQLDB, an embedded Java database.

Since many sides of OneDev assumes existance of JSW, it will be error prone to remove that part. Considering this situation, I'd suggest to discontinue the AUR packaging.

Firstly the AUR package onedev, not the oss version I am working on, uses JSW and does not modify it at all.

Secondly, suggestion denied, I started this and I must finish it

It is stored under "sampledb" sub folder which is database files of HSQLDB, an embedded Java database.

Why is it called sampledb, makes no sense :/

Since this is for demonstration purpose, and user is expected to switch to a production database such as MySQL, PostgreSQL for real project usages.

I understand that you do not want to discard things. But as I emphasized and warned previously, OneDev uses some special components and relies on it to accompolish many features. It has its own philosophy for doing things across all platforms. Trying to break this philosopy means difficulty and loss of functionality.

I am very grateful for your recognizing of OneDev and trying to bring it into ArchLinux community. Just that it works very different from Linux ideology. Not following its way of doing things means we will get bite from time to time.

Since this is for demonstration purpose, and user is expected to switch to a production database such as MySQL, PostgreSQL for real project usages.

Meh, integrated database isn't terrible man

I understand that you do not want to discard things. But as I emphasized and warned previously, OneDev uses some special components and relies on it to accompolish many features. It has its own philosophy for doing things across all platforms. Trying to break this philosopy means difficulty and loss of functionality.

Not completely, if you can get it working fluently on docker, I am more than able to get it to work on Arch Linux, I just need to iron out the issues it has, it has been working fine so far, it currently only has issues with updating, once I fix that it should be all fine!

As long as JSW is preserved, things will be easier.

I do appreciate your value of OSS. However since OneDev aims at being easy to use and maintain, and even targeting for commercial users in future, it may not adopt changes facilitating OSS but harms its primary purpose, or making things unnecessarily complicated.

I think I need to clarify this so that we can understand each other.

An example of such change is to disable agent auto-update when JSW is not available, as I do not want to consider this case everywhere in the code base to complicate things.

Although server does not run on JSW inside docker, it does ship JSW for agent packaging.

I do not use any chating software. Mainly because that I do not want to be interrupted while at work.

Exactly

As for the package status, I have ALMOST got it, it should migrate on every single package bump, if not I have another idea how to do this.

@robin As for this issue, turns out there is other software which has similar issues, so its not actually as exotic as I thought it would be.

However this "automatic" updates you talk about, will it be an issue?

It will be an issue as I am updating agent from time to time.

So could you explain what the agent does, and how it is implemented into OneDev :)

You may get into agent with this tutorial:

https://docs.onedev.io/tutorials/cicd/agent-farm/

It connects to OneDev server via websocket to get workloads.

@robin So how does bumping the agent affect OneDev, does JSW check for dependency updates and pull the libraries or something?

Server contains agent package, and running agent will check with server to see if it contains a new version of agent package. If there is, it downloads the new version and asks JSW to restart it to take new code into effect. Detailed logic here: https://code.onedev.io/onedev/agent/~files/39d5337fc2a176932f828413850d71a295b0c6fe/src/main/java/io/onedev/agent/AgentSocket.java?position=source-89.1-89.19-1

Something I have always been fascinated about is modularisation of projects, for example the Linux kernel you pick which components you want compiled in. I believe you do this with the plugins by just compiling separate jars an then load them at runtime.

Would be cool (obviously it would be too high maintenance unless someone PR's it and maintains it) to be able to select OneDev features to compile in.

@robin Just letting you know, I have added Onedev to the huge list of applications which the ArchWiki maintains: https://wiki.archlinux.org/title/List_of_applications/Utilities#Code_forges

You the list looked a bit bare with only gitea and gitlab there, so I decided to add you in, lets hope no admins get mad and remove it hehe

although, I have spent a lot of time working on the ArchWiki page, so if they did remove it, it would be a little sad

@polarian thanks a lot for your effort on this.

Just a little update, so this is mainly for privacy reasons, but also because not everyone has internet connection 24/7

https://aur.archlinux.org/packages/onedev-docs

Now provides the documentation for onedev, it is added to the /usr/share/doc/onedev and then it can be accessed locally through the use of a http server, such as python's http server as added by one of the ArchWiki admins, see the arch wiki for more info about this:

https://wiki.archlinux.org/title/Onedev

This is just so people can read the docs without an internet connection, its pretty simple and seen often within the AUR, so it shouldn't be a surprise :)

The onedev AUR package is still awaiting patching, which is why it is still on version 7.9.4 (I am aware there has been many bug fixes) I have not yet got around to the solution suggested by another AUR packager on the aur-general mailing list.

After I implement this suggestion, Onedev SHOULD be fully compatible with the AUR (until another issue is found :P)

Of course the manual install takes preference due to stability, but having an AUR package makes setup 100x easier (and updates)

Man I wish I had issue #1105 right now xD

@polarian if you follow the link in email, it will bring you directly to the comment and you do not need to scroll the page all way down.

I see k9-mail always attaches the signature to the email :/

Anyways, update: onedev-docs aur package has been packaged within an unofficial arch repository (to aid in installation), yes its my arch repository, for more info see: PolarRepo.

Once I fix onedev package I will deploy it there too so its even easier for users to install xD, still not officially supported and probably going to be buggy but meh!

Just as a sidenote: I know it was not intended to ever be packaged, but it is almost like Onedev intentionally tries to make packaging it stupidly hard xD

I see k9-mail always attaches the signature to the email :/

Hmm... Need to find a Android phone to test

@robin Bad news:

Currently, I am having issues with the Arch Community, some people want me gone and thus I am demanding a vote for my termination (sounds a little weird asking for people to vote to terminate you), if enough members are against me, I will be terminated, and thus will no longer be able to work on this.

@bengcooper has been inactive, I have emailed him I think a week ago about the updates to the Onedev AUR package and what to do moving forward, but I do not believe he has read it, and he did not reply to me. So most likely, the termination of me will result in onedev being abandoned within the Arch Community. I am the sole maintainer of the ArchWiki page on Onedev, and the ArchWiki is used throughout the linux community as it is the goto source of information and software.

A large number of users have complained about the following:

• Me speaking too much/being to active on mailing list and IRC channels
• Me being "annoying"
• Me being "too young" and "inexperienced"
• Me being "narrow minded" Now I am not sure which of these are exaggerations or maybe they are all facts, but I get mixed responses from different people, some aggressive some compliments.

I will still be around the onedev community, unless of course you request for me to leave, I still do intend to implement issue #972 when I get the time to, and I am working on a java opendev API wrapper, do not know why, I just randomly got the urge to do it.

Sorry that this entire issue might be completely irrelevant in the long run, its completely my fault!

Polarian

@polarian sorry to hear about this. It does not matter that OneDev ArchWiki will no nonger be maintained, or even this feature will be abandoned, as I do not think it is a good idea in the first place to wrap OneDev as OS native package. But I do respect your effort on this. Even without this, your other suggestions do benefit OneDev. Thanks!