This is by design that the same commit and params will only result in a single build. This is necessary to make sure that downstream job in the pipeline can depend on existing upstream build to use possibly verified artifacts instead of re-run upstream job again.

To work around this issue, you may define a groovy script, say get-timestamp with below content:

new Date().getTime().toString()

And then add a parameter of text type to the job, and set its default value to evaluate groovy script get-timestamp

This is by design that the same commit and params will only result in a single build. This is necessary to make sure that downstream job in the pipeline can depend on existing upstream build to use possibly verified artifacts instead of re-run upstream job again.

While I understand the motivation, unfortunately it breaks a bit when there are a lot of projects producing snapshots and whatnot using different build tools. While there is "Project Dependencies" and "Job dependencies" it's somewhat problematic as it seems to rely on the retrieval of the artifacts directly from the job/project.

I liked how TC handled that - one could specify other job/project dependencies and if they were set then TC checked if the project had any changes (to be build but not yet run for any reason) it would trigger the build of the dependency (which would publish snapshot to maven repository) and then main build would run (and retrieve dependency artifact from the maven repository).

To work around this issue, you may define a groovy script, say get-timestamp with below content:

new Date().getTime().toString()

And then add a parameter of text type to the job, and set its default value to evaluate groovy script get-timestamp

Will that work with templates (so I can configure it once for all projects)? I tried that and when updating the build job with new template I go "item #1->templateName: Error validating step template parameters (Missing job parameter (timestamp-to-make-distinct))" (we do have a couple of dozens of projects/components so updating them all instead of just importing new template would be way more problematic)

Would you consider adding a global option "Treat all builds as distinct"?

I'd argue that (and many of our users do) when downstream job runs, it should get artifacts from existing upstream build directly, instead of running upstream all over again, which is waste of resources, and sometime may even produce undesired result (when some unepxected external environment changes) compared to verified artifacts of upstream.

Also I think the snapshot approach is not optimal, as you can not get what has been changed between two builds resulting different artifacts.

The job parameter approach works for template also. Just edit the step using the template and you will be prompted to sepcify the parameter, and you can use same groovy script to provide a timestamp in the value field.

In case of step template, you may define the parameter at template level instead of job level.

Sorry I was wrong on template. The timestamp has to be defined for all the jobs. Defining it at template level does not work.

I'd argue that (and many of our users do) when downstream job runs, it should get artifacts from existing upstream build directly, instead of running upstream all over again, which is waste of resources, and sometime may even produce undesired result (when some unepxected external environment changes) compared to verified artifacts of upstream.

Agreed that running it again is not needed. In such case the trigger should be smart (i.e. check if the build is required: build commit diff from last commit).

And I'd still argue that being able to trigger re-build for snapshot as separate build would be handy/desired - it may happen (and it's quite possible) that we depend on external snapshots to which we have no control.

When to think about it, I think only 1dev replaces existing builds when re-run/re-executed.

Also I think the snapshot approach is not optimal, as you can not get what has been changed between two builds resulting different artifacts.

Could you explain?

In general snapshot/development dependencies cause certain issues and by design they are not reproducible and hence unstable…

The job parameter approach works for template also. Just edit the step using the template and you will be prompted to sepcify the parameter, and you can use same groovy script to provide a timestamp in the value field. Sorry I was wrong on template. The timestamp has to be defined for all the jobs. Defining it at template level does not work.

I was toying with it and each time with script defined in template I was getting exact resulting value in edited template step/job (i.e. "1728302612704"). Thus my understanding is I would have to edit each job and add the call to script as default value still?

I was playing with it and only at the end figured out that it has to be done outside of the template.

I would ask again for a global option affecting this behaviour - either current "smart" system which would replace build or treat all builds as distinct? Or at least make it a template option so updating it for multiple (all) projects would be simpler?

Agreed that running it again is not needed. In such case the trigger should be smart (i.e. check if the build is required: build commit diff from last commit).

Making such smart will result in inconsistent behaviors and cause troubles.

Could you explain?

I mean two builds can produce different artifacts (due to different SNAPSHOT), yet the changes does not disclose anything. So I avoid using SNAPSHOT versions in OneDev development.

I was toying with it and each time with script defined in template I was getting exact resulting value in edited template step/job (i.e. "1728302612704"). Thus my understanding is I would have to edit each job and add the call to script as default value still?

Please define param at job level instead of template level.

I would ask again for a global option affecting this behaviour - either current "smart" system which would replace build or treat all builds as distinct? Or at least make it a template option so updating it for multiple (all) projects would be simpler?

Sorry that this global option will not be added as it breaks basic assumption of OneDev.

Agreed that running it again is not needed. In such case the trigger should be smart (i.e. check if the build is required: build commit diff from last commit).

Making such smart will result in inconsistent behaviors and cause troubles.

Could you explain?

I mean two builds can produce different artifacts (due to different SNAPSHOT), yet the changes does not disclose anything. So I avoid using SNAPSHOT versions in OneDev development.

We are still discussing SNAPSHOTs, which are by definition inconsistent and volatile (and it's more than OK during development cycle till the release). Not every build has to be reproducible.

Right now we are (quite often) hitting the issue that one build is success and the next day it has failed just because dependency change and it wasn't picked up upstream (and this is the job of the CI to pick those up in the end)

I was toying with it and each time with script defined in template I was getting exact resulting value in edited template step/job (i.e. "1728302612704"). Thus my understanding is I would have to edit each job and add the call to script as default value still?

Please define param at job level instead of template level.

I'll try that.

I would ask again for a global option affecting this behaviour - either current "smart" system which would replace build or treat all builds as distinct? Or at least make it a template option so updating it for multiple (all) projects would be simpler?

Sorry that this global option will not be added as it breaks basic assumption of OneDev.

That's unfortunate.

Could you at least reconsider it at the template level (assuming it's possible to adjust the build replacement strategy via parameters) or make the parameter inheritance from template work?

because dependency change and it wasn't picked up upstream (and this is the job of the CI to pick those up in the end)

What do you mean by this?

Could you at least reconsider it at the template level (assuming it's possible to adjust the build replacement strategy via parameters) or make the parameter inheritance from template work?

Adding to template does not work as the template works at a different level as job, it is basically a step composition.

because dependency change and it wasn't picked up upstream (and this is the job of the CI to pick those up in the end)

What do you mean by this?

Imagine you have 50 modules/components/projects. You adjust code/api in some projects but you forget that it's used in yet another project. Failing CI build serves as a reminder to adjust code everywhere.

Having distinct builds could be helpful to identify when the failure happened. This is mostly visible with bigger codebases with more people working on the code and requiring more coordination.

Could you at least reconsider it at the template level (assuming it's possible to adjust the build replacement strategy via parameters) or make the parameter inheritance from template work?

Adding to template does not work as the template works at a different level as job, it is basically a step composition.

I (I think) understand what it does and it's useful but again - it would be helpful to control the behaviour via template. If it can't be done via template step configuration (and script) then maybe template parameter/configuration/option in a way? It would just give the flexibility - if someone doesn't want distinct builds then it wouldn't enable that :)

Having distinct builds could be helpful to identify when the failure happened. This is mostly visible with bigger codebases with more people working on the code and requiring more coordination.

The non-distinct build approach can still serve the purpose. Just that the old build running against old snapshot is not available. Even if old build exists, it does not help much to find out which dependency change caused the build failure, as dependency version is not changed (assuming the failure is caused by SNAPSHOT dependency), you still need to check the build log to find out why.

That was only one example.

Another, that was more recent - we had an application build on top of another application (sources inlined via github submodule) which had dependency. As it's in rapid development cycle everything is in SNAPSHOT most of the time. We provided a build number X for test of the check (library got build, the main app got build, the final app got built and was green). Due to unexpected timing (a push to the library and the timer build on the last app) the final app build X was re-run but because there was a change in the underlying library the build failed and replaced same build that was correct. You could argue - "but then don't use timer to run build!" but because not all changes require changes in the final app an only in the middle repo then there is no commit in final app repo to trigger the build and it would require to always remember to trigger it manually.

Is it ideal? Far from it. Could it be improved? Probably by doing a "final" release of the library and always use it but that would mean a release basically every day which is just... terrible and we have SNAPSHOTS for that.

Imagine you have 50 modules/components/projects. You adjust code/api in some projects but you forget that it's used in yet another project. Failing CI build serves as a reminder to adjust code everywhere.

Sounds like what you really want is an artifact dependency view in OneDev, similar to what maven repository manager show you if you take a look at a published artifact (e.g. https://central.sonatype.com/artifact/com.google.guava/guava/33.3.1-jre/dependents)

Personally I would never ever define SNAPSHOT dependencies in a project and then push it on the main branch. All my 60+ projects use versioned dependencies. The reason is that if a developer pulls the project and it has SNAPSHOT dependencies defined then the developer cannot work properly if the dependency is unstable and can change at any time. If it is a central core library which affects many projects that can be a real nightmare.

So all projects are updated over time to the new version of a dependency and if a project requires running many/slow tests to verify that the upgraded dependency does work as intended in the project then a dedicated branch can be used and CI then checks that branch.

"but then don't use timer to run build!" but because not all changes require changes in the final app an only in the middle repo then there is no commit in final app repo to trigger the build and it would require to always remember to trigger it manually.

@robin Maybe the post build actions of a build spec should allow starting jobs in different projects as well. Currently it only shows jobs in the current project.

That way @wojtek could change the build logic for this CI requirement and let lower projects trigger builds of higher projects, e.g. successful library build triggers build of middle app which in turn triggers build of main app if successful.

Then there is no need to a timer based build anymore.

You may add a command step at end of upstream job to trigger downstream job via restful api. This way timer based triggering can be avoided to verify dependency changes.

Imagine you have 50 modules/components/projects. You adjust code/api in some projects but you forget that it's used in yet another project. Failing CI build serves as a reminder to adjust code everywhere.

Sounds like what you really want is an artifact dependency view in OneDev, similar to what maven repository manager show you if you take a look at a published artifact (e.g. https://central.sonatype.com/artifact/com.google.guava/guava/33.3.1-jre/dependents)

No, I need that each build is distinct, as simple as that :)

Btw. I prefer simpler https://mvnrepository.com/artifact/com.google.guava/guava/33.3.1-jre :)

Personally I would never ever define SNAPSHOT dependencies in a project and then push it on the main branch. All my 60+ projects use versioned dependencies. The reason is that if a developer pulls the project and it has SNAPSHOT dependencies defined then the developer cannot work properly if the dependency is unstable and can change at any time. If it is a central core library which affects many projects that can be a real nightmare.

We do have stable branches for released versions. It's just semantics and how you define your project: you can have super-stable master branch with latest GA and development branch or flip that around and have stable branches with fixed versions and use master for develpment...

Triggers may work but again - defining all the structure is time consuming and doesn't relly solve the issue at hand :)

At any rate. I tried to setup parameter with script evaluated default value and while trying to save the config I got:

Error validating job parameters (item: #1, error message: Missing job parameter (timestamp-to-make-distinct))

The error is quite cryptic...

All triggers should also configure the job parameter. Please edit the trigger "when update branch 'master'" and specify value of the job parameter (can use output of same groovy script).

Want to mention that default value specified when define the param only tells OneDev to prepopulate the param when trigger it manually.

OK, that worked. Somewhat confusing. Maybe the error be a bit more clear?