Usage Scenario

Generate files in one job, process each file concurrently in the second job, and collect processed results in the third job

How to Set Up

1. Define a groovy script to get list of published artifact names from upstream build whose number is passed via parameter upstream-build-number:

Content of the script:

import io.onedev.server.OneDev
import io.onedev.server.entitymanager.BuildManager

// Variable "build" represents the build being triggered currently
// For single value parameter, just use the first one
def upstreamBuildNumber = build.paramMap["upstream-build-number"][0]
def upstreamBuild = OneDev.getInstance(BuildManager.class).find(build.project, upstreamBuildNumber as Long)
return upstreamBuild.artifactsDir.listFiles().collect {it.name}

1. Edit build spec to add job Generate Files with below command:

Command to generate files:

echo 1 > file1
echo 2 > file2
echo 3 > file3

1. Continue to edit job Generate Files. In Artifacts & Reports section, publish generated files:

1. Add job Process File with parameter file which will be used to hold name of the file being processed

For demonstration purpose, the command prepend message processed to the file content:

echo "processed $(cat @params:file@)" > @params:file@

1. Continue to edit job Process File. In Dependencies & Services section, add a job dependency to retrieve published file with name equal to parameter file:

1. Continue to edit job Process File. In Artifacts & Reports section, publish processed file:

1. Add job Collect Files with parameter upstream-build-number which will be used to hold number of the build generating files

The command cat files* simply prints contents of all collected files

1. Continue to edit job Collect Files. In Dependencies & Services section, add a job dependency to Process File with param file set to evaluated result of the groovy script we added in the first step. For each of the name returned from the script, the job will be triggered once and the processed file will be collected back:

1. Go back to edit job Generate Files. In More Settings section, add a post-action to trigger job Collect Files, with param upstream-build-number set to number of current build:

1. Done! Now run job Generate Files. After it succeeds, three builds of job Process File will run to process generated files. After all files are processed, job Collect Files will be running to collect the results