15.1.2 update causes agents using self-signed certificate to no longer connect (OD-2798)

15.1.2 update causes agents using self-signed certificate to no longer connect #2798

Alejandro Oton Garcia opened 2 weeks ago

I recently updated to OneDev server to 15.1.2 and after the update neither of my agents connect back to OneDev server. I have my own PKI and have my certs in conf/trust-certs. The logs show the following:

Launching a JVM...
01:26:13 INFO  io.onedev.agent.Agent - Connecting to https://git.domain.com...
01:26:13 INFO  io.onedev.agent.AgentSocket - Connected to server
01:26:13 INFO  io.onedev.agent.AgentSocket - Updating agent to version 3.0.9...
01:26:13 ERROR io.onedev.agent.AgentSocket - Error processing websocket message
javax.ws.rs.ProcessingException: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:270)
        at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:300)
        at org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$0(JerseyInvocation.java:662)
        at org.glassfish.jersey.client.JerseyInvocation.call(JerseyInvocation.java:697)
        at org.glassfish.jersey.client.JerseyInvocation.lambda$runInScope$3(JerseyInvocation.java:691)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:205)
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
        at org.glassfish.jersey.client.JerseyInvocation.runInScope(JerseyInvocation.java:691)
        at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:661)
        at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:413)
        at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:313)
        at io.onedev.agent.AgentSocket.onMessage(AgentSocket.java:160)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.eclipse.jetty.websocket.common.events.annotated.CallableMethod.call(CallableMethod.java:70)
        at org.eclipse.jetty.websocket.common.events.annotated.OptionalSessionCallableMethod.call(OptionalSessionCallableMethod.java:72)
        at org.eclipse.jetty.websocket.common.events.JettyAnnotatedEventDriver.onBinaryMessage(JettyAnnotatedEventDriver.java:143)
        at org.eclipse.jetty.websocket.common.message.SimpleBinaryMessage.messageComplete(SimpleBinaryMessage.java:75)
        at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:67)
        at org.eclipse.jetty.websocket.common.events.JettyAnnotatedEventDriver.onBinaryFrame(JettyAnnotatedEventDriver.java:130)
        at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:147)
        at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:326)
        at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:202)
        at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:225)
        at org.eclipse.jetty.websocket.common.Parser.parseSingleFrame(Parser.java:259)
        at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:459)
        at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:440)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
        at org.eclipse.jetty.io.ssl.SslConnection$1.run(SslConnection.java:149)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:383)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1294)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1169)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1112)
        at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:481)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:459)
        at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:206)
        at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
        at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1510)
        at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1425)
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455)
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:426)
        at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:589)
        at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:187)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1717)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1641)
        at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
        at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:380)
        at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:268)
        ... 36 common frames omitted
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
        at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
        at java.base/sun.security.validator.Validator.validate(Validator.java:264)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1278)
        ... 55 common frames omitted
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:148)
        at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:129)
        at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
        at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
        ... 60 common frames omitted

Activities

Alejandro Oton Garcia changed fields 2 weeks ago
Name Previous Value Current Value
Type

Security Vulnerability

Bug

Affected Versions
empty
15.1.2
Robin Shen commented 2 weeks ago

Are you running agents via docker, or on the machine directly?

Name	Previous Value	Current Value
Type	Security Vulnerability	Bug
Affected Versions	empty	15.1.2

Alejandro Oton Garcia commented 2 weeks ago

I am running one agent as a container and one agent on bare metal (Windows). Both agents are showing the same errors. It seems that they are not picking up the certs in conf\trust-certs when updating to 3.0.9.

20:36:12 INFO  io.onedev.agent.AgentSocket - Connected to server
20:36:12 INFO  io.onedev.agent.AgentSocket - Updating agent to version 3.0.9...
20:36:12 ERROR io.onedev.agent.AgentSocket - Error processing websocket message
javax.ws.rs.ProcessingException: javax.net.ssl.SSLHandshakeException: (certificate_unknown) PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Robin Shen commented 2 weeks ago

This issue is actually fixed since 15.1. However the bug in 15.0.x prevents agent from auto-updating itself when using a self-signed certificate. Please re-install new agents by downloading from 15.1.2 agents page. Subsequent agent upgrade will no longer experience this issue.
Robin Shen commented 2 weeks ago

For agents running with docker, just re-pull the image is enough.

docker pull 1dev/agent

Robin Shen changed title 2 weeks ago

Previous Value	Current Value
15.1.2 update causes agents to no longer connect	15.1.2 update causes agents using self-signed certificate to no longer connect

Alejandro Oton Garcia commented 2 weeks ago

Thanks Robin. Image pull did the trick for my Docker agent and for Windows, I did create a new agent like you suggested and it connected right away.
Alejandro Oton Garcia changed state to 'Closed' 2 weeks ago
Previous Value Current Value
Open

Closed
Login to comment

Previous Value	Current Value
Open	Closed

Type	Bug
Priority	Major
Assignee	Robin Shen
Affected Versions	15.1.2
Labels	No labels

Issue Votes (0)

Watchers (4)

Reference

OD-2798