#1035  OneDev fails to start due to hard shutdown
Closed
Andrew Lalis opened 1 year ago

Similar issue as issue #663, where an unexpected EOF causes startup to fail:

18:41:15 ERROR i.onedev.commons.bootstrap.Bootstrap - Error booting application
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at io.onedev.commons.loader.Listener.notify(Listener.java:21)
	at io.onedev.commons.loader.DefaultListenerRegistry.post(DefaultListenerRegistry.java:72)
	at io.onedev.server.OneDev.postStart(OneDev.java:142)
	at io.onedev.server.persistence.SessionInterceptor$1.call(SessionInterceptor.java:23)
	at io.onedev.server.persistence.DefaultSessionManager.call(DefaultSessionManager.java:85)
	at io.onedev.server.persistence.SessionInterceptor.invoke(SessionInterceptor.java:18)
	at io.onedev.commons.loader.DefaultPluginManager.start(DefaultPluginManager.java:48)
	at io.onedev.commons.loader.AppLoader.start(AppLoader.java:73)
	at io.onedev.commons.bootstrap.Bootstrap.main(Bootstrap.java:189)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:349)
	at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.lang.reflect.InvocationTargetException: null
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at io.onedev.commons.loader.Listener.notify(Listener.java:19)
	... 14 common frames omitted
Caused by: java.lang.RuntimeException: java.io.EOFException: Short read of block.
	at io.onedev.server.infomanager.DefaultCommitInfoManager.collect(DefaultCommitInfoManager.java:1077)
	at io.onedev.server.infomanager.DefaultCommitInfoManager.on(DefaultCommitInfoManager.java:1091)
	at io.onedev.server.persistence.SessionInterceptor$1.call(SessionInterceptor.java:23)
	at io.onedev.server.persistence.DefaultSessionManager.call(DefaultSessionManager.java:85)
	at io.onedev.server.persistence.SessionInterceptor.invoke(SessionInterceptor.java:18)
	... 19 common frames omitted
Caused by: java.io.EOFException: Short read of block.
	at org.eclipse.jgit.util.IO.readFully(IO.java:203)
	at org.eclipse.jgit.internal.storage.file.UnpackedObject.open(UnpackedObject.java:72)
	at org.eclipse.jgit.internal.storage.file.LooseObjects.getObjectLoader(LooseObjects.java:188)
	at org.eclipse.jgit.internal.storage.file.LooseObjects.open(LooseObjects.java:149)
	at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openLooseObject(ObjectDirectory.java:396)
	at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openLooseFromSelfOrAlternate(ObjectDirectory.java:373)
	at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openObjectWithoutRestoring(ObjectDirectory.java:349)
	at org.eclipse.jgit.internal.storage.file.ObjectDirectory.openObject(ObjectDirectory.java:330)
	at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:132)
	at org.eclipse.jgit.lib.ObjectReader.open(ObjectReader.java:212)
	at org.eclipse.jgit.revwalk.RevWalk.parseAny(RevWalk.java:1069)
	at io.onedev.server.infomanager.DefaultCommitInfoManager.collect(DefaultCommitInfoManager.java:1072)
	... 23 common frames omitted
<-- Wrapper Stopped

I know that in this case I can recover because I still have my own local version, but can anything be done to avoid this issue in the future? Like periodically keeping a backup of the last working version of each project, even as a default without an administrator explicitly activating any backup settings.

In an ideal world, the server should still start, but report an issue to administrators that one of its repositories is in a damaged state.

Robin Shen commented 1 year ago

Seems like many other projects also suffer from this using jgit. In future, OneDev high availability where a project has one or more active backups should be able to solve this type of failure nicely.

Robin Shen commented 1 year ago

It might be more of an issue of git instead of jgit in case of hard shutdown...

Robin Shen changed state to 'Closed' 1 year ago
Previous Value Current Value
Open
Closed
Robin Shen commented 1 year ago

This will be addressed with HA feature (issue #290)

issue 1 of 1
Type
Bug
Priority
Normal
Assignee
Affected Versions
Not Found
Issue Votes (0)
Watchers (4)
Reference
onedev/server#1035
Please wait...
Page is in error, reload to recover