Problem using built image from registry hosted in the same kubernetes cluster (OD-1835)

Martin Langer opened 2 years ago

Hi,

I hit a roadblock after getting on board with OneDev quite fast:

I built a couple of images with kaniko and got kaniko to push them into my registry which I deployed in the same kubernetes cluster. It is the normal registry implementation from docker without credentials or encryption configured.

Here the log from the kaniko job:

19:04:57 Step "Compute libs hash" is successful
19:04:57 Running step "Build tests image"...
19:04:57 INFO[0000] Retrieving image manifest registry.ci.svc.cluster.local:5000/optics/phaseopt-libs-unit:d68d25a39270 
19:04:57 INFO[0000] Retrieving image registry.ci.svc.cluster.local:5000/optics/phaseopt-libs-unit:d68d25a39270 from registry registry.ci.svc.cluster.local:5000 
19:04:57 INFO[0000] Built cross stage deps: map[]                
19:04:57 INFO[0000] Retrieving image manifest registry.ci.svc.cluster.local:5000/optics/phaseopt-libs-unit:d68d25a39270 
19:04:57 INFO[0000] Returning cached image manifest              
19:04:57 INFO[0000] Executing 0 build triggers                   
19:04:57 INFO[0000] Building stage 'registry.ci.svc.cluster.local:5000/optics/phaseopt-libs-unit:d68d25a39270' [idx: '0', base-idx: '-1'] 
19:04:57 INFO[0000] Unpacking rootfs as cmd COPY ../../opm /app/opm requires it. 
...           
19:05:03 INFO[0007] Taking snapshot of files...                  
19:05:03 INFO[0007] CMD ["/bin/bash"]                            
19:05:03 INFO[0007] Pushing image to registry.ci.svc.cluster.local:5000/optics/phaseopt-tests:a3e787b250be16645cab6c6fb589910968f9b1a0 
19:05:03 INFO[0007] Pushed registry.ci.svc.cluster.local:5000/optics/phaseopt-tests@sha256:5b0ab23ae6cc7903c7148e1fb9246588d22198ce4031634e465adc8cabf393e3 
19:05:03 Step "Build tests image" is successful

Now it is very strange that kaniko is able to pull and push from my registry while I am not able to use the image in another job:

results to:

19:20:44 No job executor defined, auto-discovering...
19:20:44 Discovered job executor type: Kubernetes Executor
19:20:44 Checking cluster access...
19:20:44 Preparing job (executor: auto-discovered, namespace: auto-discovered-5-34-0)...
19:20:44 Running job on node lima-rancher-desktop...
19:20:44 Starting job containers...
19:20:45 Retrieving job data from http://192.168.64.2:6610...
19:20:45 Generating command scripts...
19:20:45 Downloading job dependencies from http://192.168.64.2:6610...
19:20:45 Job workspace initialized
19:21:11 Step "Unittests": failed to pull and unpack image "registry.ci.svc.cluster.local:5000/optics/phaseopt-tests:a3e787b250be16645cab6c6fb589910968f9b1a0": failed to resolve reference "registry.ci.svc.cluster.local:5000/optics/phaseopt-tests:a3e787b250be16645cab6c6fb589910968f9b1a0": failed to do request: Head "https://registry.ci.svc.cluster.local:5000/v2/optics/phaseopt-tests/manifests/a3e787b250be16645cab6c6fb589910968f9b1a0": dial tcp: lookup registry.ci.svc.cluster.local: no such host

This blocks me from switching to OneDev for a project, any hints what I did wrong? Or is this maybe a bug?

Activities

Robin Shen commented 2 years ago

Kaniko step resolves the image inside the pod container, while the command step relies on Kubernetes pod scheduler to resolve image (configured as container image of job pod). Seems to me that it is a problem of k8s setup. You may create a simple pod with a container using your custom image to see if it works.
Martin Langer commented 2 years ago

Thanks for the response!

A test pod also makes problems, seems like a http vs https problem.
Martin Langer commented 2 years ago

Well I have a really hard time setting this stuff up.

Is there somewhere an example in the documentation where (via kubernetes) kaniko builds an image which gets executed in following job? And/or something which demonstrates the use of private registries?
Robin Shen commented 2 years ago

This is no longer a OneDev issue and I can not help much here. Maybe k8s forum can help on this.
Martin Langer commented 2 years ago

Ok, I'll close the issue.
Martin Langer changed state to 'Closed' 2 years ago
Previous Value Current Value
Open

Closed
Login to comment

Previous Value	Current Value
Open	Closed

Type	Bug
Priority	Normal
Assignee	Robin Shen
Affected Versions	OneDev 10.4.0
Labels	No labels

Issue Votes (0)

Watchers (3)

Reference

OD-1835