Error on Build-Pipeline (OD-2173)
crome696 opened 1 year ago

I´m currently using OneDev 11.5.2 Community to host my projects on a local server.

When I generate a maven Build Job such as "maven ci", only my first project can be build. All other projects fail with following error:

22:17:39 Step "detect build version" is successful (5 seconds)
22:17:39 Running step "set build version"...
22:17:39 javax.persistence.PersistenceException: org.hibernate.exception.DataException: could not execute statement
	at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:154)
	at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:181)
	at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:188)
	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1366)
	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:453)
	at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3212)
	at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2380)
	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:447)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:183)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$300(JdbcResourceLocalTransactionCoordinatorImpl.java:40)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:281)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
	at io.onedev.server.persistence.DefaultTransactionManager.lambda$call$0(DefaultTransactionManager.java:67)
	at io.onedev.server.persistence.DefaultSessionManager.call(DefaultSessionManager.java:90)
	at io.onedev.server.persistence.DefaultTransactionManager.call(DefaultTransactionManager.java:57)
	at io.onedev.server.buildspec.step.SetBuildVersionStep.run(SetBuildVersionStep.java:49)
	at io.onedev.server.job.DefaultJobManager.runServerStep(DefaultJobManager.java:1489)
	at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor$1$1$1.run(ServerDockerExecutor.java:534)
	at io.onedev.k8shelper.ServerSideFacade.execute(ServerSideFacade.java:91)
	at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor$1$1.doExecute(ServerDockerExecutor.java:530)
	at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor$1$1.lambda$execute$1(ServerDockerExecutor.java:406)
	at io.onedev.agent.ExecutorUtils.runStep(ExecutorUtils.java:100)
	at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor$1$1.execute(ServerDockerExecutor.java:403)
	at io.onedev.k8shelper.LeafFacade.execute(LeafFacade.java:12)
	at io.onedev.k8shelper.CompositeFacade.execute(CompositeFacade.java:35)
	at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor$1.run(ServerDockerExecutor.java:319)
	at io.onedev.server.job.DefaultJobManager.runJob(DefaultJobManager.java:1362)
	at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor.lambda$execute$fee504de$1(ServerDockerExecutor.java:252)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at com.hazelcast.executor.impl.DistributedExecutorService$Processor.run(DistributedExecutorService.java:276)
	at com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:217)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76)
	at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:111)
Caused by: org.hibernate.exception.DataException: could not execute statement
	at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:52)
	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:99)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:200)
	at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3496)
	at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:3358)
	at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3772)
	at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:201)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:604)
	at org.hibernate.engine.spi.ActionQueue.lambda$executeActions$1(ActionQueue.java:478)
	at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:475)
	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:348)
	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:40)
	at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:102)
	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1362)
	... 32 more
Caused by: java.sql.SQLDataException: data exception: string data, right truncation
	at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
	at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
	at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(Unknown Source)
	at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(Unknown Source)
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:197)
	... 44 more
Caused by: org.hsqldb.HsqlException: data exception: string data, right truncation
	at org.hsqldb.error.Error.error(Unknown Source)
	at org.hsqldb.error.Error.error(Unknown Source)
	at org.hsqldb.types.CharacterType.castOrConvertToType(Unknown Source)
	at org.hsqldb.types.CharacterType.convertToType(Unknown Source)
	at org.hsqldb.StatementDML.getUpdatedData(Unknown Source)
	at org.hsqldb.StatementDML.executeUpdateStatement(Unknown Source)
	at org.hsqldb.StatementDML.getResult(Unknown Source)
	at org.hsqldb.StatementDMQL.execute(Unknown Source)
	at org.hsqldb.Session.executeCompiledStatement(Unknown Source)
	at org.hsqldb.Session.execute(Unknown Source)
	... 49 more
22:17:39 Step "run tests" is skipped

If i outcomment the following steps:

  #- !CommandStep
  #  name: detect build version
  #  runInContainer: true
  #  image: maven
  #  interpreter: !DefaultInterpreter
  #    commands: |
  #      echo "Detecting project version (may require some time while downloading maven dependencies)..."
  #      echo $(mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate -Dexpression=project.version -q -DforceStdout) > buildVersion
  #  useTTY: true
  #  condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
  #- !SetBuildVersionStep
  #  name: set build version
  #  buildVersion: '@file:buildVersion@'
  #  condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL

It works but I think this is still not corrent since "maven ci" is a generated example from OneDev.

I had this issue first time when i fresh started with OneDev 11.2.12 and i tried to fix it by clean reinstall (and clean configuration) on version 11.5.2

Might be important: My Pipelines are a collection of packages required to work together. So I have a parent pom.xml in one project and other projects are using it. I deploy the pom.xml all the time to maven package registry. The only project which works with get\set version is the parent.. Others seem to fail.

As we speak i just disabled those two steps and it works so far...

  • Robin Shen commented 1 year ago

    Please run command docker run -it --rm -v /path/to/your/project:/test maven bash to start container from your terminal, and then run below inside container to see what happens:

    mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate -Dexpression=project.version -q -DforceStdout
    
  • Robin Shen commented 1 year ago

    Run below commands inside container:

    cd /test
    mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate -Dexpression=project.version -q -DforceStdout
    
  • crome696 commented 1 year ago

    Hi, I´m not that kind of CI/CD -expert, so i might need more insight 😟

    /path/to/your/project
    

    Does this means my path what docker uses for OneDev Installation?

    What i tried to do:

    version: 37
    jobs:
    - name: maven ci
      steps:
      - !CheckoutStep
        name: checkout code
        cloneCredential: !DefaultCredential {}
        withLfs: false
        withSubmodules: false
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !GenerateChecksumStep
        name: generate pom checksum
        files: '**/pom.xml'
        targetFile: checksum
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !SetupCacheStep
        name: set up maven cache
        key: maven_repository_@file:checksum@
        loadKeys:
        - maven_repository
        paths:
        - /root/.m2/repository
        uploadStrategy: UPLOAD_IF_NOT_HIT
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CommandStep
        name: run tests
        runInContainer: true
        image: maven bash
        interpreter: !DefaultInterpreter
          commands: |
            mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate -Dexpression=project.version -q -DforceStdout
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      triggers:
      - !BranchUpdateTrigger {}
      - !PullRequestUpdateTrigger {}
      retryCondition: never
      maxRetries: 3
      retryDelay: 30
      timeout: 14400
    
    

    And i get following result:

    image.png

    So it seems the buildsystem itself can catch the version.

    What might be the problem: Maven packages can be snapshots So it could build the same package over and over and upload it to repository. If OneDev always try to insert to database it might be the troublesome part.

    Edit: The more i think about it might be an intentional approach by maven to do this but not useful for snapshot builds. Maybe it could be considered as enhancement to add a check to default maven-ci, if snapshot is involved , to try an update?

  • Robin Shen commented 1 year ago

    I want to test how it behaves outside of OneDev. So /path/to/project is clone path of your own project that is causing build failure. Then mount the project into maven container and run the command to see if it can resolves the version.

  • crome696 commented 1 year ago

    First of all, this error happens to all my maven builds except the first project I had added.

    So i tried three things:

    1. Run command on projects within IntelliJ:
    "C:\Program Files\Eclipse Adoptium\jdk-21.0.4.7-hotspot\bin\java.exe" -Dmaven.multiModuleProjectDirectory=C:\Users\USER\IdeaProjects\PROJECT\spring-boot-starter -Djansi.passthrough=true "-Dmaven.home=C:\Users\USER\AppData\Local\Programs\IntelliJ IDEA Ultimate\plugins\maven\lib\maven3" "-Dclassworlds.conf=C:\Users\USER\AppData\Local\Programs\IntelliJ IDEA Ultimate\plugins\maven\lib\maven3\bin\m2.conf" "-Dmaven.ext.class.path=C:\Users\USER\AppData\Local\Programs\IntelliJ IDEA Ultimate\plugins\maven\lib\maven-event-listener.jar" "-javaagent:C:\Users\USER\AppData\Local\Programs\IntelliJ IDEA Ultimate\lib\idea_rt.jar=52589:C:\Users\USER\AppData\Local\Programs\IntelliJ IDEA Ultimate\bin" -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8 -Dsun.stderr.encoding=UTF-8 -classpath "C:\Users\USER\AppData\Local\Programs\IntelliJ IDEA Ultimate\plugins\maven\lib\maven3\boot\plexus-classworlds-2.8.0.jar;C:\Users\USER\AppData\Local\Programs\IntelliJ IDEA Ultimate\plugins\maven\lib\maven3\boot\plexus-classworlds.license" org.codehaus.classworlds.Launcher -Didea.version=2024.3 org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate -Dexpression=project.version -q -DforceStdout
    1.0.0-SNAPSHOT
    Process finished with exit code 0
    
    1. Run commands through docker terminal:
    Windows PowerShell
    Copyright (C) Microsoft Corporation. Alle Rechte vorbehalten.
    
    Installieren Sie die neueste PowerShell für neue Funktionen und Verbesserungen! https://aka.ms/PSWindows
    
    PS C:\Users\USER> docker run -it --rm -v C:\Users\USER\Downloads\spring-boot-starter\spring-boot-starter\test:/test maven bash
    root@62f7553893e4:/# cd /test
    root@62f7553893e4:/test#mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate -Dexpression=project.version -q -DforceStdout
    1root@62f7553893e4:/test# 
    
    1. Run commands through Build-Pipeline -> see comment above :-)

    After doing more tests it seems following from maven ci seem to fail:

      - !SetBuildVersionStep
        name: set build version
        buildVersion: '@file:buildVersion@'
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
    

    Edit:

    I changed all my pipeline settings to test it more. GetVersion always works. For now i just skip that step.

  • crome696 removed comment 1 year ago
  • crome696 commented 1 year ago

    Another update:

    image_2.png

    I tried it now with angular as build and setFileVersion does the same trouble if the version is same as another build.

  • Robin Shen commented 1 year ago

    I am a bit overwhelmed by all the updates. Let's focus on the initial error before trying others:

    22:17:39 Step "detect build version" is successful (5 seconds)
    22:17:39 Running step "set build version"...
    22:17:39 javax.persistence.PersistenceException: org.hibernate.exception.DataException: could not execute statement
        at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:154)
        at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:181)
        at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:188)
        at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1366)
        at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:453)
        at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3212)
        at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2380)
        at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:447)
        at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:183)
        at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$300(JdbcResourceLocalTransactionCoordinatorImpl.java:40)
        at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:281)
        at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
        at io.onedev.server.persistence.DefaultTransactionManager.lambda$call$0(DefaultTransactionManager.java:67)
        at io.onedev.server.persistence.DefaultSessionManager.call(DefaultSessionManager.java:90)
        at io.onedev.server.persistence.DefaultTransactionManager.call(DefaultTransactionManager.java:57)
        at io.onedev.server.buildspec.step.SetBuildVersionStep.run(SetBuildVersionStep.java:49)
        at io.onedev.server.job.DefaultJobManager.runServerStep(DefaultJobManager.java:1489)
        at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor$1$1$1.run(ServerDockerExecutor.java:534)
        at io.onedev.k8shelper.ServerSideFacade.execute(ServerSideFacade.java:91)
        at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor$1$1.doExecute(ServerDockerExecutor.java:530)
        at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor$1$1.lambda$execute$1(ServerDockerExecutor.java:406)
        at io.onedev.agent.ExecutorUtils.runStep(ExecutorUtils.java:100)
        at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor$1$1.execute(ServerDockerExecutor.java:403)
        at io.onedev.k8shelper.LeafFacade.execute(LeafFacade.java:12)
        at io.onedev.k8shelper.CompositeFacade.execute(CompositeFacade.java:35)
        at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor$1.run(ServerDockerExecutor.java:319)
        at io.onedev.server.job.DefaultJobManager.runJob(DefaultJobManager.java:1362)
        at io.onedev.server.plugin.executor.serverdocker.ServerDockerExecutor.lambda$execute$fee504de$1(ServerDockerExecutor.java:252)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at com.hazelcast.executor.impl.DistributedExecutorService$Processor.run(DistributedExecutorService.java:276)
        at com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:217)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
        at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76)
        at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:111)
    Caused by: org.hibernate.exception.DataException: could not execute statement
        at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:52)
        at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
        at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
        at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:99)
        at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:200)
        at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3496)
        at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:3358)
        at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3772)
        at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:201)
        at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:604)
        at org.hibernate.engine.spi.ActionQueue.lambda$executeActions$1(ActionQueue.java:478)
        at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
        at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:475)
        at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:348)
        at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:40)
        at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:102)
        at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1362)
        ... 32 more
    Caused by: java.sql.SQLDataException: data exception: string data, right truncation
        at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
        at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
        at org.hsqldb.jdbc.JDBCPreparedStatement.fetchResult(Unknown Source)
        at org.hsqldb.jdbc.JDBCPreparedStatement.executeUpdate(Unknown Source)
        at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
        at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
        at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:197)
        ... 44 more
    Caused by: org.hsqldb.HsqlException: data exception: string data, right truncation
        at org.hsqldb.error.Error.error(Unknown Source)
        at org.hsqldb.error.Error.error(Unknown Source)
        at org.hsqldb.types.CharacterType.castOrConvertToType(Unknown Source)
        at org.hsqldb.types.CharacterType.convertToType(Unknown Source)
        at org.hsqldb.StatementDML.getUpdatedData(Unknown Source)
        at org.hsqldb.StatementDML.executeUpdateStatement(Unknown Source)
        at org.hsqldb.StatementDML.getResult(Unknown Source)
        at org.hsqldb.StatementDMQL.execute(Unknown Source)
        at org.hsqldb.Session.executeCompiledStatement(Unknown Source)
        at org.hsqldb.Session.execute(Unknown Source)
        ... 49 more
    22:17:39 Step "run tests" is skipped
    

    Here the set version step fails because the detect version step captures incorrect version from POM, which might be a long error message, and it exceeds version column in database.

    For the detect version step, OneDev simply mounts your project into "maven" container and run maven helper plugin to do the job. This can be easily tested manually on your terminal. Just login to the machine where you are running OneDev container, and mount your own project into maven container and run the maven helper plugin command mentioned before.

    I see that you are doing that against a test sprintboot project as below:

    Windows PowerShell
    Copyright (C) Microsoft Corporation. Alle Rechte vorbehalten.
    
    Installieren Sie die neueste PowerShell für neue Funktionen und Verbesserungen! https://aka.ms/PSWindows
    
    PS C:\Users\USER> docker run -it --rm -v C:\Users\USER\Downloads\spring-boot-starter\spring-boot-starter\test:/test maven bash
    root@62f7553893e4:/# cd /test
    root@62f7553893e4:/test#mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate -Dexpression=project.version -q -DforceStdout
    1root@62f7553893e4:/test# 
    

    The output 1 will be taken as version, and it seems incorrect. Can you please remove the -q option and run command again to see what it outputs?

    Also please test with your own project instead of springboot example on same envionrment as OneDev (OneDev does not support to run Linux containers on Windows).

  • Robin Shen commented 1 year ago

    Also want to mention that using same version for multiple builds is not a problem. OneDev does not enforce different version for different builds.

  • crome696 commented 1 year ago

    Okay lets take a few steps back.

    1. The first occurance happend for me when i tried to build a maven based project. In this case it doesnt matter if its spring boot (maven ci) or java (maven ci). They are all Java 21 & maven based projects.

    2. Regardless if i use Terminal and bash (I made an error on trying your example) or directly in IntelliJ or if i use OneDev generated command. It always generates me 1.0.0-SNAPSHOT. I also have the same problem (together with other problems) when building a new angular project without any changes. In that case my Angular project also prints 0.0.0 and then crashes.

    All my Java projects are Java 21 based maven projects All my Angular projects are Angular 18.2.11 based projects using nodejs v22.11.0

    My first conclusion was, its based on my infrastructure where i deploy maven packages and have to access to it by using an access-token and a custom settings.xml. But i Have the same issue also with new generated projects in maven and with angular. SetVersion fails always. If i only do GetVersion and print it it works in console (but then not writes it to the job description).

    • All my Java Projects are : Maven projects with Java 21
    • All my Angular Projects are: Angular 18.2.11 projects with nodejs v22.11.0
  • Robin Shen commented 1 year ago

    My first conclusion was, its based on my infrastructure where i deploy maven packages and have to access to it by using an access-token and a custom settings.xml. But i Have the same issue also with new generated projects in maven and with angular. SetVersion fails always.

    If some maven packages need to be accessed with access token, the command to print maven project version will fail and print error messages into buildVersion file. This causes subsequent set version step fails as OneDev simply set the error message of previous step as build version and it causes data truncation error in database. To work around this issue, you need to set up repository credentials via maven settings.xml in the detect version step, or simply move the detect version and set version step to be executed after the test step (your test step still needs to set up appropriate settings.xml for authentication).

    However if the maven build does not need to access private registry for dependency retrieval, the detect version step should work just fine. If not, please create an example project at this site demonstrating the issue.

  • crome696 commented 1 year ago

    My first conclusion was, its based on my infrastructure where i deploy maven packages and have to access to it by using an access-token and a custom settings.xml. But i Have the same issue also with new generated projects in maven and with angular. SetVersion fails always.

    If some maven packages need to be accessed with access token, the command to print maven project version will fail and print error messages into buildVersion file. This causes subsequent set version step fails as OneDev simply set the error message of previous step as build version and it causes data truncation error in database. To work around this issue, you need to set up repository credentials via maven settings.xml in the detect version step, or simply move the detect version and set version step to be executed after the test step (your test step still needs to set up appropriate settings.xml for authentication).

    However if the maven build does not need to access private registry for dependency retrieval, the detect version step should work just fine. If not, please create an example project at this site demonstrating the issue.

    I had the same thought before i posted first time here. So i changed my job to the following:

    version: 37
    jobs:
    - name: maven ci
      steps:
      - !CommandStep
        name: set secrets & validate
        runInContainer: true
        image: alpine
        interpreter: !DefaultInterpreter
          commands: |
            echo @secret:access-token@
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CheckoutStep
        name: checkout code
        cloneCredential: !DefaultCredential {}
        withLfs: false
        withSubmodules: false
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !GenerateChecksumStep
        name: generate pom checksum
        files: '**/pom.xml'
        targetFile: checksum
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !SetupCacheStep
        name: set up maven cache
        key: maven_repository_@file:checksum@
        loadKeys:
        - maven_repository
        paths:
        - /root/.m2/repository
        uploadStrategy: UPLOAD_IF_NOT_HIT
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CommandStep
        name: set settings.xml
        runInContainer: true
        image: alpine
        interpreter: !DefaultInterpreter
          commands: |
            cat << EOF > settings.xml
            <settings>
              <servers>
                <server>
                  <id>onedev</id>
                  <username>@job_token@</username>
                  <password>@secret:access-token@</password>
                </server>
              </servers>
            </settings>
            EOF
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CommandStep
        name: detect build version
        runInContainer: true
        image: maven
        interpreter: !DefaultInterpreter
          commands: |
            echo "Detecting project version (may require some time while downloading maven dependencies)..."
            echo $(mvn org.apache.maven.plugins:maven-help-plugin:3.1.0:evaluate -Dexpression=project.version -q -DforceStdout) > buildVersion
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !SetBuildVersionStep
        name: set build version
        buildVersion: '@file:buildVersion@'
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CommandStep
        name: run tests
        runInContainer: true
        image: maven
        interpreter: !DefaultInterpreter
          commands: |
            mvn clean test --settings settings.xml
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CommandStep
        name: run deployment
        runInContainer: true
        image: maven
        interpreter: !DefaultInterpreter
          commands: |
            mvn deploy -DskipTests --settings settings.xml
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !PublishJUnitReportStep
        name: publish unit test report
        reportName: Unit Test
        filePatterns: '**/TEST-*.xml'
        condition: ALWAYS
      triggers:
      - !BranchUpdateTrigger {}
      - !PullRequestUpdateTrigger {}
      retryCondition: never
      maxRetries: 3
      retryDelay: 30
      timeout: 14400
    

    I tried to reproduce it today and now a few builds seem to work.. Let me dig deeper into the topic before we watch deeper into it :)

  • Robin Shen commented 1 year ago

    Want to mention that the cache can affect the reproducibility. For instance if the first build fails due to unresolved artifacts, and subsequent builds succeed by removing the set version step. When you add back the set version step later, the build still succeeds, as unresolved artifacts are cached now. You may disable the set up cache step for build to have same running environment.

  • crome696 commented 1 year ago

    A few more insights... I´ve added --settings settings.xml at all maven commands now and reworking the whole scripts and my pipelines. The part of cache might be also troublesome. I will give you a feedback once I´m done starting from scratch

    @robin : https://code.onedev.io/OD-2173-example I set it up as i did on my personal space. Exact same configs etc.. and you are right, it doesnt print the file version.

  • crome696 commented 1 year ago

    Finally

    I´ve got it working and its a mixture of everything we discussed 😟

    1. When reading the FileVersion it remained blank. This is due the cache and it doesnt update the dependencies
    2. When using a parent package which is hosted on oneDev or any other private repository it needs to obtain the informations or version reading fails
    3. it might be trouble that maven ci adds a fixed maven version due my setup to ban or block certain dependencies. I had to remove the version in order to get it work
    4. Whenever maven is involved i have to link to the settings.xml
    version: 37
    properties:
    - name: autodeployment
      value: 'true'
      archived: false
    jobs:
    - name: maven ci
      steps:
      - !CommandStep
        name: set secrets & validate
        runInContainer: true
        image: alpine
        interpreter: !DefaultInterpreter
          commands: |
            echo @secret:access-token@
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CheckoutStep
        name: checkout code
        cloneCredential: !DefaultCredential {}
        withLfs: false
        withSubmodules: false
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !GenerateChecksumStep
        name: generate pom checksum
        files: '**/pom.xml'
        targetFile: checksum
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !SetupCacheStep
        name: set up maven cache
        key: maven_repository_@file:checksum@
        loadKeys:
        - maven_repository
        paths:
        - /root/.m2/repository
        uploadStrategy: UPLOAD_IF_NOT_HIT
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CommandStep
        name: set settings.xml
        runInContainer: true
        image: alpine
        interpreter: !DefaultInterpreter
          commands: |
            cat << EOF > settings.xml
            <settings>
              <servers>
                <server>
                  <id>onedev</id>
                  <username>@job_token@</username>
                  <password>@secret:access-token@</password>
                </server>
              </servers>
            </settings>
            EOF
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CommandStep
        name: clean cache
        runInContainer: true
        image: maven
        interpreter: !DefaultInterpreter
          commands: |
            mvn clean install -U -DskipTests --settings settings.xml
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CommandStep
        name: detect build version
        runInContainer: true
        image: maven
        interpreter: !DefaultInterpreter
          commands: |
            echo "Detecting project version (may require some time while downloading maven dependencies)..."
            echo $(mvn org.apache.maven.plugins:maven-help-plugin:evaluate -Dexpression=project.version -q -DforceStdout) > buildVersion
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !SetBuildVersionStep
        name: set build version
        buildVersion: '@file:buildVersion@'
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CommandStep
        name: run tests
        runInContainer: true
        image: maven
        interpreter: !DefaultInterpreter
          commands: |
            mvn clean test --settings settings.xml
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CommandStep
        name: get deployment options
        runInContainer: true
        image: alpine
        interpreter: !DefaultInterpreter
          commands: |
            echo $(mvn help:evaluate -Dexpression=onedev.autodeployment -q -DforceStdout  --settings settings.xml) > @property:autodeployment@
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !CommandStep
        name: run deployment
        runInContainer: true
        image: maven
        interpreter: !DefaultInterpreter
          commands: |
            if [ "@property:autodeployment@" = true ]; then
              echo "property:autodeployment is true. Running Maven command..."
              mvn deploy -DskipTests --settings settings.xml
            else
              echo "property:autodeployment is not true. Skipping Maven command."
            fi
        useTTY: true
        condition: ALL_PREVIOUS_STEPS_WERE_SUCCESSFUL
      - !PublishJUnitReportStep
        name: publish unit test report
        reportName: Unit Test
        filePatterns: '**/TEST-*.xml'
        condition: ALWAYS
      triggers:
      - !BranchUpdateTrigger {}
      - !PullRequestUpdateTrigger {}
      retryCondition: never
      maxRetries: 3
      retryDelay: 30
      timeout: 14400
    
    

    This is my final configuration and it works on all projects now.

  • Robin Shen changed state to 'Closed' 1 year ago
    Previous Value Current Value
    Open
    Closed
  • Robin Shen commented 1 year ago

    Thanks for the info. OneDev will be using latest maven image. Also the detect version step will be moved after test step. Closing this now.

issue 1/1
Type
Bug
Priority
Normal
Assignee
Affected Versions
11.2.12 11.5.2
Labels
No labels
Issue Votes (0)
Watchers (2)
Reference
OD-2173
Please wait...
Connection lost or session expired, reload to recover
Page is in error, reload to recover