OneDev
Dashboards
Projects
Global Views
Code Search
server
Overview
Code
Pull Requests
Issues
List
Boards
Iterations
Builds
Packages
Statistics
Audit Log
OneDev 16.2.0
Documentation Support & Bug Report RESTful API Incompatibilities License Agreement Check Update
Projects onedev server Issues OD-2862
OneDev git SSH Connections get stuck #2862
مِيزَانُ الْدِّينْ عَبْدُ ذِيْ الْجَلَالِ وَ الْإِكْرَامِ opened 3 days ago

Every few days my SSH connectivity to OneDev just stops. I have to restart it for it start responding.

The logs show line items like the following:

app    | 07:49:58 ERROR io.onedev.server.git.SshCommand - Error executing git command
app    | java.lang.RuntimeException: java.lang.InterruptedException
app    | 	at io.onedev.commons.utils.command.Commandline.execute(Commandline.java:399)
app    | 	at io.onedev.commons.utils.command.Commandline.execute(Commandline.java:203)
app    | 	at io.onedev.server.git.command.UploadPackCommand.run(UploadPackCommand.java:65)
app    | 	at io.onedev.server.git.SshCommand$1.run(SshCommand.java:154)
app    | 	at io.onedev.server.util.concurrent.DefaultWorkExecutionService$1.call(DefaultWorkExecutionService.java:198)
app    | 	at io.onedev.server.util.concurrent.DefaultWorkExecutionService$1.call(DefaultWorkExecutionService.java:194)
app    | 	at io.onedev.server.security.SecurityUtils$1.call(SecurityUtils.java:691)
app    | 	at io.onedev.server.util.concurrent.DefaultWorkExecutionService.lambda$call$0(DefaultWorkExecutionService.java:63)
app    | 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
app    | 	at io.onedev.server.security.SecurityUtils.lambda$inheritSubject$1(SecurityUtils.java:680)
app    | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
app    | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
app    | 	at java.base/java.lang.Thread.run(Thread.java:840)
app    | Caused by: java.lang.InterruptedException: null
app    | 	at java.base/java.lang.Object.wait(Native Method)
app    | 	at java.base/java.lang.Object.wait(Object.java:338)
app    | 	at java.base/java.lang.ProcessImpl.waitFor(ProcessImpl.java:434)
app    | 	at io.onedev.commons.utils.command.Commandline.execute(Commandline.java:396)
app    | 	... 12 common frames omitted @
app    | 07:54:37 ERROR i.o.s.w.c.c.list.CommitListPanel - Error calculating commits:
app    | io.onedev.commons.utils.ExplicitException: Please login to perform this query
app    | 	at io.onedev.server.search.commit.PersonCriteria.fill(PersonCriteria.java:56)
app    | 	at io.onedev.server.search.commit.CommitterCriteria.fill(CommitterCriteria.java:20)
app    | 	at io.onedev.server.search.commit.CommitQuery.lambda$fill$13(CommitQuery.java:169)
app    | 	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
app    | 	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
app    | 	at io.onedev.server.search.commit.CommitQuery.fill(CommitQuery.java:169)
app    | 	at io.onedev.server.web.component.commit.list.CommitListPanel$2.load(CommitListPanel.java:153)
app    | 	at io.onedev.server.web.component.commit.list.CommitListPanel$2.load(CommitListPanel.java:122)
app    | 	at org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachableModel.java:137)
app    | 	at io.onedev.server.web.component.commit.list.CommitListPanel.newCommitsView(CommitListPanel.java:557)
app    | 	at io.onedev.server.web.component.commit.list.CommitListPanel$11.onBeforeRender(CommitListPanel.java:434)
app    | 	at org.apache.wicket.Component.internalBeforeRender(Component.java:957)
app    | 	at org.apache.wicket.Component.beforeRender(Component.java:1027)
app    | 	at org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1826)
app    | 	at org.apache.wicket.Component.onBeforeRender(Component.java:3933)
app    | 	at io.onedev.server.web.component.commit.list.CommitListPanel.onBeforeRender(CommitListPanel.java:764)
app    | 	at org.apache.wicket.Component.internalBeforeRender(Component.java:957)
app    | 	at org.apache.wicket.Component.beforeRender(Component.java:1027)
app    | 	at org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1826)
app    | 	at org.apache.wicket.Component.onBeforeRender(Component.java:3933)
app    | 	at org.apache.wicket.Page.onBeforeRender(Page.java:801)
app    | 	at io.onedev.server.web.page.base.BasePage.onBeforeRender(BasePage.java:396)
app    | 	at org.apache.wicket.Component.internalBeforeRender(Component.java:957)
app    | 	at org.apache.wicket.Component.beforeRender(Component.java:1027)
app    | 	at org.apache.wicket.Component.internalPrepareForRender(Component.java:2248)
app    | 	at org.apache.wicket.Page.internalPrepareForRender(Page.java:242)
app    | 	at org.apache.wicket.Component.render(Component.java:2341)
app    | 	at org.apache.wicket.Page.renderPage(Page.java:1018)
app    | 	at org.apache.wicket.request.handler.render.WebPageRenderer.renderPage(WebPageRenderer.java:124)
app    | 	at org.apache.wicket.request.handler.render.WebPageRenderer.respond(WebPageRenderer.java:236)
app    | 	at org.apache.wicket.core.request.handler.RenderPageRequestHandler.respond(RenderPageRequestHandler.java:175)
app    | 	at org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor.respond(RequestCycle.java:892)

And then an interesting EatWhatYouKill:

app    | 07:39:56 ERROR io.onedev.server.git.SshCommand - Error executing git command
app    | java.lang.RuntimeException: java.lang.InterruptedException
app    | 	at io.onedev.commons.utils.command.Commandline.execute(Commandline.java:399)
app    | 	at io.onedev.commons.utils.command.Commandline.execute(Commandline.java:203)
app    | 	at io.onedev.server.git.command.UploadPackCommand.run(UploadPackCommand.java:65)
app    | 	at io.onedev.server.git.SshCommand$1.run(SshCommand.java:154)
app    | 	at io.onedev.server.util.concurrent.DefaultWorkExecutionService$1.call(DefaultWorkExecutionService.java:198)
app    | 	at io.onedev.server.util.concurrent.DefaultWorkExecutionService$1.call(DefaultWorkExecutionService.java:194)
app    | 	at io.onedev.server.security.SecurityUtils$1.call(SecurityUtils.java:691)
app    | 	at io.onedev.server.util.concurrent.DefaultWorkExecutionService.lambda$call$0(DefaultWorkExecutionService.java:63)
app    | 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
app    | 	at io.onedev.server.security.SecurityUtils.lambda$inheritSubject$1(SecurityUtils.java:680)
app    | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
app    | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
app    | 	at java.base/java.lang.Thread.run(Thread.java:840)
app    | Caused by: java.lang.InterruptedException: null
app    | 	at java.base/java.lang.Object.wait(Native Method)
app    | 	at java.base/java.lang.Object.wait(Object.java:338)
app    | 	at java.base/java.lang.ProcessImpl.waitFor(ProcessImpl.java:434)
app    | 	at io.onedev.commons.utils.command.Commandline.execute(Commandline.java:396)
app    | 	... 12 common frames omitted
app    | 07:48:15 ERROR i.o.s.w.c.c.list.CommitListPanel - Error calculating commits:
app    | io.onedev.commons.utils.ExplicitException: Please login to perform this query
app    | 	at io.onedev.server.search.commit.PersonCriteria.fill(PersonCriteria.java:56)
app    | 	at io.onedev.server.search.commit.CommitterCriteria.fill(CommitterCriteria.java:20)
app    | 	at io.onedev.server.search.commit.CommitQuery.lambda$fill$13(CommitQuery.java:169)
app    | 	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
app    | 	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
app    | 	at io.onedev.server.search.commit.CommitQuery.fill(CommitQuery.java:169)
app    | 	at io.onedev.server.web.component.commit.list.CommitListPanel$2.load(CommitListPanel.java:153)
app    | 	at io.onedev.server.web.component.commit.list.CommitListPanel$2.load(CommitListPanel.java:122)
app    | 	at org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachableModel.java:137)
app    | 	at io.onedev.server.web.component.commit.list.CommitListPanel.newCommitsView(CommitListPanel.java:557)
app    | 	at io.onedev.server.web.component.commit.list.CommitListPanel$11.onBeforeRender(CommitListPanel.java:434)
app    | 	at org.apache.wicket.Component.internalBeforeRender(Component.java:957)
app    | 	at org.apache.wicket.Component.beforeRender(Component.java:1027)
app    | 	at org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1826)
app    | 	at org.apache.wicket.Component.onBeforeRender(Component.java:3933)
app    | 	at io.onedev.server.web.component.commit.list.CommitListPanel.onBeforeRender(CommitListPanel.java:764)
app    | 	at org.apache.wicket.Component.internalBeforeRender(Component.java:957)
app    | 	at org.apache.wicket.Component.beforeRender(Component.java:1027)
app    | 	at org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1826)
app    | 	at org.apache.wicket.Component.onBeforeRender(Component.java:3933)
app    | 	at org.apache.wicket.Page.onBeforeRender(Page.java:801)
app    | 	at io.onedev.server.web.page.base.BasePage.onBeforeRender(BasePage.java:396)
app    | 	at org.apache.wicket.Component.internalBeforeRender(Component.java:957)
app    | 	at org.apache.wicket.Component.beforeRender(Component.java:1027)
app    | 	at org.apache.wicket.Component.internalPrepareForRender(Component.java:2248)
app    | 	at org.apache.wicket.Page.internalPrepareForRender(Page.java:242)
app    | 	at org.apache.wicket.Component.render(Component.java:2341)
app    | 	at org.apache.wicket.Page.renderPage(Page.java:1018)
app    | 	at org.apache.wicket.request.handler.render.WebPageRenderer.renderPage(WebPageRenderer.java:124)
app    | 	at org.apache.wicket.request.handler.render.WebPageRenderer.respond(WebPageRenderer.java:236)
app    | 	at org.apache.wicket.core.request.handler.RenderPageRequestHandler.respond(RenderPageRequestHandler.java:175)
app    | 	at org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor.respond(RequestCycle.java:892)
app    | 	at org.apache.wicket.request.RequestHandlerStack.execute(RequestHandlerStack.java:64)
app    | 	at org.apache.wicket.request.cycle.RequestCycle.execute(RequestCycle.java:261)
app    | 	at org.apache.wicket.request.cycle.RequestCycle.processRequest(RequestCycle.java:218)
app    | 	at org.apache.wicket.request.cycle.RequestCycle.processRequestAndDetach(RequestCycle.java:289)
app    | 	at org.apache.wicket.protocol.ws.AbstractUpgradeFilter.processRequestCycle(AbstractUpgradeFilter.java:70)
app    | 	at org.apache.wicket.protocol.http.WicketFilter.processRequest(WicketFilter.java:203)
app    | 	at org.apache.wicket.protocol.http.WicketServlet.doGet(WicketServlet.java:137)
app    | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
app    | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
app    | 	at io.onedev.server.web.DefaultWicketServlet.access$001(DefaultWicketServlet.java:18)
app    | 	at io.onedev.server.web.DefaultWicketServlet.lambda$service$0(DefaultWicketServlet.java:50)
app    | 	at io.onedev.server.persistence.DefaultSessionService.lambda$run$0(DefaultSessionService.java:103)
app    | 	at io.onedev.server.persistence.DefaultSessionService.call(DefaultSessionService.java:85)
app    | 	at io.onedev.server.persistence.DefaultSessionService.run(DefaultSessionService.java:102)
app    | 	at io.onedev.server.web.DefaultWicketServlet.service(DefaultWicketServlet.java:44)
app    | 	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
app    | 	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
app    | 	at com.google.inject.servlet.DefaultFilterPipeline.dispatch(DefaultFilterPipeline.java:47)
app    | 	at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133)
app    | 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
app    | 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
app    | 	at io.onedev.server.git.GoGetFilter.doFilter(GoGetFilter.java:90)
app    | 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
app    | 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
app    | 	at io.onedev.server.git.GitLfsFilter.doFilter(GitLfsFilter.java:494)
app    | 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
app    | 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
app    | 	at io.onedev.server.git.GitFilter.doFilter(GitFilter.java:396)
app    | 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
app    | 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
app    | 	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
app    | 	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
app    | 	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
app    | 	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154)
app    | 	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
app    | 	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
app    | 	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
app    | 	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154)
app    | 	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
app    | 	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
app    | 	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
app    | 	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154)
app    | 	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
app    | 	at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:458)
app    | 	at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:373)
app    | 	at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
app    | 	at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
app    | 	at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387)
app    | 	at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:370)
app    | 	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154)
app    | 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
app    | 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
app    | 	at io.onedev.server.security.CorsFilter.doFilter(CorsFilter.java:47)
app    | 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
app    | 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
app    | 	at io.onedev.server.jetty.DisableTraceFilter.doFilter(DisableTraceFilter.java:28)
app    | 	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
app    | 	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
app    | 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
app    | 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
app    | 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
app    | 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
app    | 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
app    | 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
app    | 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
app    | 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
app    | 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
app    | 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
app    | 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
app    | 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
app    | 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
app    | 	at org.eclipse.jetty.server.Server.handle(Server.java:516)
app    | 	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
app    | 	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
app    | 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
app    | 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
app    | 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
app    | 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
app    | 	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
app    | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
app    | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
app    | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
app    | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
app    | 	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
app    | 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
app    | 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
app    | 	at java.base/java.lang.Thread.run(Thread.java:840)

The server resources do not show any cpu or memory issues at all..

1/1
Type
Bug
Priority
Normal
Assignee
Robin Shen
Affected Versions
Been seeing this for many versions - today I just found the time to report it. Currently running 16.0.8.
Labels
No labels
Issue Votes (0)
Login to vote
Watchers (2)
Reference
OD-2862
Please wait...
Connection lost or session expired, reload to recover
Page is in error, reload to recover