[Question] Docker swarm and onedev update flow #3101
Raphaël Thémans opened 3 days ago

Hello,

Thanks for this great application. You really have a great tool and I will never thank you enough for the work you do! I hope I can soon pay for its usage, truly!

I’m using docker swarm to manage my applications. I’m using onedev as the main tool for devsecops.

So, I have a project with the infrastructure files (docker-swarm yamls, config files and so on). onedev-docker-swarm.yml is the definition for onedev and I’m using it with infra-onedev-buildspec.yml Note that this file is way bigger and I’ve removed a few "sensitive" data. Along with it, in the "Settings/Build/Job Properties", I have a property "ONEDEV_VERSION" which I update when I’m upgrading onedev. We are notified of the build status through webhooks.

Now that you have the context, here is my question: When I run that build, It will always go in "Cancelled" is there something I could/should do so that it will provide the correct status? or is this out of scope for the moment? any feedback on this procedure? Just to be clear. This works, onedev is updated properly and everything is quickly back to working as it should. Just that we are notified of a Cancelled build and the build will always show as "Cancelled" while it went through properly.

Btw, I tried this setup with docker-compose and it was mainly a bad experience as the "runner" will just stop when taking down onedev and I needed to restart the container so that it would go through statring onedev back up… I think, might be that I was manually running the docker-compose file myself. Not sure.

  • Service Desk commented 3 days ago

    I could not start implementation because the Assignee account does not have write-code permission for the issue project. tod issue checkout --for-write 3101 failed with HTTP 403: No permission to write code in issue project.

  • Robin Shen commented 3 days ago

    @servicedesk answer user's question

  • Service Desk commented 3 days ago

    @rthemans This is expected when a OneDev build deploys/restarts the same OneDev instance that is coordinating the build. During graceful shutdown, OneDev cancels its active job executions, so this build cannot remain alive long enough to record a successful result and send a success webhook. The Swarm rollout can still complete independently, which explains why the deployment succeeds while the build is shown as cancelled.

    In the attached build spec, docker stack deploy --detach=false waits for the rollout to converge. The rollout stops the current OneDev container while that command is still running, so self-cancellation is unavoidable.

    There are two practical options:

    • Remove --detach=false (or explicitly use detached mode). This lets the command return after Swarm accepts the update, and the build may finish before OneDev is restarted. However, a successful build then only means that Swarm accepted the deployment request; it does not verify that the upgraded OneDev became healthy, and there is still an inherent timing race.
    • For a reliable end-to-end result, perform the OneDev upgrade from an external control plane that is not itself being upgraded. Let that process deploy the stack, wait for the new OneDev service to become healthy, and send the final notification. The same OneDev instance cannot reliably certify its own successful shutdown and restart.

    The Docker Compose behavior has the same underlying cause: stopping OneDev also stops/cancels the build coordination, and stopping the runner as part of the same Compose deployment makes that coupling even stronger.

    Does this answer your question? If so, I will close the issue.

  • Raphaël Thémans commented 3 days ago

    This answers my question

  • Service Desk changed state to 'Closed' 3 days ago
    Previous Value Current Value
    Open
    Closed
  • Robin Shen commented 3 days ago

    @rthemans the AI bot closes the issue way fast. Thank you for your kind words and your willingness to support OneDev! It means a lot to hear that it’s become such a useful part of your workflow.

1/1
Type
Question
Priority
Normal
Assignee
Labels
No labels
Issue Votes (0)
Watchers (3)
Reference
OD-3101
Please wait...
Connection lost or session expired, reload to recover
Page is in error, reload to recover