#1835  Problem using built image from registry hosted in the same kubernetes cluster
Closed
Martin Langer opened 3 weeks ago

Hi,

I hit a roadblock after getting on board with OneDev quite fast:

I built a couple of images with kaniko and got kaniko to push them into my registry which I deployed in the same kubernetes cluster. It is the normal registry implementation from docker without credentials or encryption configured.

Here the log from the kaniko job:

19:04:57 Step "Compute libs hash" is successful
19:04:57 Running step "Build tests image"...
19:04:57 INFO[0000] Retrieving image manifest registry.ci.svc.cluster.local:5000/optics/phaseopt-libs-unit:d68d25a39270 
19:04:57 INFO[0000] Retrieving image registry.ci.svc.cluster.local:5000/optics/phaseopt-libs-unit:d68d25a39270 from registry registry.ci.svc.cluster.local:5000 
19:04:57 INFO[0000] Built cross stage deps: map[]                
19:04:57 INFO[0000] Retrieving image manifest registry.ci.svc.cluster.local:5000/optics/phaseopt-libs-unit:d68d25a39270 
19:04:57 INFO[0000] Returning cached image manifest              
19:04:57 INFO[0000] Executing 0 build triggers                   
19:04:57 INFO[0000] Building stage 'registry.ci.svc.cluster.local:5000/optics/phaseopt-libs-unit:d68d25a39270' [idx: '0', base-idx: '-1'] 
19:04:57 INFO[0000] Unpacking rootfs as cmd COPY ../../opm /app/opm requires it. 
...           
19:05:03 INFO[0007] Taking snapshot of files...                  
19:05:03 INFO[0007] CMD ["/bin/bash"]                            
19:05:03 INFO[0007] Pushing image to registry.ci.svc.cluster.local:5000/optics/phaseopt-tests:a3e787b250be16645cab6c6fb589910968f9b1a0 
19:05:03 INFO[0007] Pushed registry.ci.svc.cluster.local:5000/optics/phaseopt-tests@sha256:5b0ab23ae6cc7903c7148e1fb9246588d22198ce4031634e465adc8cabf393e3 
19:05:03 Step "Build tests image" is successful

Now it is very strange that kaniko is able to pull and push from my registry while I am not able to use the image in another job: screen.png

results to:

19:20:44 No job executor defined, auto-discovering...
19:20:44 Discovered job executor type: Kubernetes Executor
19:20:44 Checking cluster access...
19:20:44 Preparing job (executor: auto-discovered, namespace: auto-discovered-5-34-0)...
19:20:44 Running job on node lima-rancher-desktop...
19:20:44 Starting job containers...
19:20:45 Retrieving job data from http://192.168.64.2:6610...
19:20:45 Generating command scripts...
19:20:45 Downloading job dependencies from http://192.168.64.2:6610...
19:20:45 Job workspace initialized
19:21:11 Step "Unittests": failed to pull and unpack image "registry.ci.svc.cluster.local:5000/optics/phaseopt-tests:a3e787b250be16645cab6c6fb589910968f9b1a0": failed to resolve reference "registry.ci.svc.cluster.local:5000/optics/phaseopt-tests:a3e787b250be16645cab6c6fb589910968f9b1a0": failed to do request: Head "https://registry.ci.svc.cluster.local:5000/v2/optics/phaseopt-tests/manifests/a3e787b250be16645cab6c6fb589910968f9b1a0": dial tcp: lookup registry.ci.svc.cluster.local: no such host

This blocks me from switching to OneDev for a project, any hints what I did wrong? Or is this maybe a bug?

Robin Shen commented 3 weeks ago

Kaniko step resolves the image inside the pod container, while the command step relies on Kubernetes pod scheduler to resolve image (configured as container image of job pod). Seems to me that it is a problem of k8s setup. You may create a simple pod with a container using your custom image to see if it works.

Martin Langer commented 3 weeks ago

Thanks for the response!

A test pod also makes problems, seems like a http vs https problem.

Martin Langer commented 3 weeks ago

Well I have a really hard time setting this stuff up.

Is there somewhere an example in the documentation where (via kubernetes) kaniko builds an image which gets executed in following job? And/or something which demonstrates the use of private registries?

Robin Shen commented 3 weeks ago

This is no longer a OneDev issue and I can not help much here. Maybe k8s forum can help on this.

Martin Langer commented 3 weeks ago

Ok, I'll close the issue.

Martin Langer changed state to 'Closed' 3 weeks ago
Previous Value Current Value
Open
Closed
issue 1 of 1
Type
Bug
Priority
Normal
Assignee
Affected Versions
OneDev 10.4.0
Labels
No labels
Issue Votes (0)
Watchers (3)
Reference
onedev/server#1835
Please wait...
Page is in error, reload to recover