Failure to restart after unclean shutdown while builds running (OD-1055)
Released
Jerome St-Louis opened 1 year ago

The server would not start after I had to forcefully restart the system. Clearing the /temp/server directory allowed it to start, but it is a major issue to require manual maintenance for this and not be able to come back to life by itself.

Launching a JVM... WrapperManager: Initializing... 04:08:35 INFO i.onedev.commons.bootstrap.Bootstrap - Launching application from '/onedev'... 04:08:35 INFO i.onedev.commons.bootstrap.Bootstrap - Cleaning temp directory... 04:08:35 ERROR i.onedev.commons.bootstrap.Bootstrap - Error booting application java.io.UncheckedIOException: java.nio.file.AccessDeniedException: /onedev/temp/server/onedev-build12442802571581486111/workspace/[** build directories here **] at java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:87) at java.base/java.nio.file.FileTreeIterator.hasNext(FileTreeIterator.java:103) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132) at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) at io.onedev.commons.bootstrap.Bootstrap.main(Bootstrap.java:129) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:349) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.nio.file.AccessDeniedException: /onedev/temp/server/onedev-build12442802571581486111/workspace/[** build directories here **] at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) at java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:432) at java.base/java.nio.file.Files.newDirectoryStream(Files.java:472) at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:300) at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:373) at java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:83) ... 16 common frames omitted

Robin Shen commented 1 year ago

The build has something left uncleaned upon forcibly restart, and manually cleaning them in such rare cases does not seem like a big deal. Auto healing in every edge cases will greatly improves the complexiity.

Robin Shen changed state to 'Closed' 1 year ago
Previous Value Current Value
Open
Closed
Jerome St-Louis commented 1 year ago

It makes no sense to me that the whole system refuses to start because there is some left over files from a build. This is not a rare case, it would happen anytime a machine need to be forcefully restarted. The whole point of CI is to automate things. Requiring manual intervention for something like this is a huge deal.

Please keep open as a future improvement suggestion if this is not considered a bug.

Robin Shen changed state to 'Open' 1 year ago
Previous Value Current Value
Closed
Open
OneDev changed state to 'Closed' 1 year ago
Previous Value Current Value
Open
Closed
OneDev commented 1 year ago

State changed as code fixing the issue is committed

Jerome St-Louis commented 1 year ago

Thank you so much @robin ! You rock :)

OneDev changed state to 'Released' 1 year ago
Previous Value Current Value
Closed
Released
OneDev commented 1 year ago

State changed as build #3255 is successful

issue 1 of 1
Type
Bug
Priority
Normal
Assignee
Affected Versions
Not Found
Issue Votes (0)
Watchers (4)
Reference
OD-1055
Please wait...
Page is in error, reload to recover