OneDev
Dashboards
Projects
Global Views
Code Search
server
Code
Pull Requests
Issues
List
Boards
Iterations
Builds
Packages
Statistics
Child Projects
Audit Log
OneDev 14.0.6
Documentation Support & Bug Report RESTful API Incompatibilities License Agreement Check Update
Projects onedev server Issues OD-2449
Using tmp in COPY command in Containerfile fails (OD-2449)
daniel-developer opened 7 months ago

I tried to build this container image. It fails with the following error:

07:45:06 #10 [4/4] COPY ./tmp/superset/docker/docker-*.sh /usr/local/bin/
07:45:06 #10 ERROR: lstat /tmp/superset/docker: no such file or directory

This is the full log:

07:33:04 Pending resource allocation...
07:33:04 Executing job (executor: remote-docker-amd64, agent: xxxxx)...
07:33:04 Creating docker network 'remote-docker-amd64-19-35-1'...
07:33:04 Downloading job dependencies...
07:33:04 Running step "checkout code"...
07:33:04 Checking out code...
07:33:04 Updated Git hooks.
07:33:04 Git LFS initialized.
07:33:04 Switched to branch 'main'
07:33:04 branch 'main' set up to track 'origin/main'.
07:33:04 Step "checkout code" is successful (0 seconds)
07:33:04 Running step "download repo from github"...
07:33:06 afc384f871be: Already exists
07:33:06 114b400bc236: Already exists
07:33:06 fe07684b16b8: Already exists
07:33:06 d415a033d488: Pulling fs layer
07:33:06 d415a033d488: Download complete
07:33:06 d415a033d488: Download complete
07:33:06 Cloning into 'tmp/superset'...
07:33:07 remote: Enumerating objects: 6746, done.
07:33:07 remote: Counting objects: 100% (6746/6746), done.
07:33:09 remote: Compressing objects: 100% (5725/5725), done.
07:33:23 remote: Total 6746 (delta 2092), reused 2805 (delta 877), pack-reused 0 (from 0)
07:33:23 Receiving objects: 100% (6746/6746), 98.33 MiB | 7.37 MiB/s, done.
07:33:23 Resolving deltas: 100% (2092/2092), done.
07:33:26 Updating files: 100% (5840/5840), done.
07:33:26 before pause
07:45:02 after pause
07:45:02 Step "download repo from github" is successful (11 minutes 57 seconds)
07:45:02 Running step "store secrets"...
07:45:04 fe07684b16b8: Already exists
07:45:04 cea2ff433c61: Pulling fs layer
07:45:04 cea2ff433c61: Download complete
07:45:04 cea2ff433c61: Download complete
07:45:04 Step "store secrets" is successful (2 seconds)
07:45:04 Running step "build container image"...
07:45:05 #0 building with "onedev" instance using docker-container driver
07:45:05 
07:45:05 #1 [internal] load build definition from Containerfile
07:45:05 #1 transferring dockerfile: 331B done
07:45:05 #1 DONE 0.1s
07:45:05 
07:45:05 #2 [internal] load metadata for docker.io/apache/superset:latest
07:45:06 #2 DONE 0.9s
07:45:06 
07:45:06 #3 [internal] load .dockerignore
07:45:06 #3 transferring context: 45B done
07:45:06 #3 DONE 0.1s
07:45:06 
07:45:06 #4 [internal] load build context
07:45:06 #4 DONE 0.0s
07:45:06 
07:45:06 #5 [1/4] FROM docker.io/apache/superset:latest@sha256:15e110b8533d3cb6a0d529512ea71252b0ac62e3f72b1f7a5000f1361822ac26
07:45:06 #5 resolve docker.io/apache/superset:latest@sha256:15e110b8533d3cb6a0d529512ea71252b0ac62e3f72b1f7a5000f1361822ac26
07:45:06 #5 resolve docker.io/apache/superset:latest@sha256:15e110b8533d3cb6a0d529512ea71252b0ac62e3f72b1f7a5000f1361822ac26 0.0s done
07:45:06 #5 DONE 0.0s
07:45:06 
07:45:06 #6 [auth] *:: xxx/duckdb-superset/cache:pull token for xxxx
07:45:06 #6 DONE 0.0s
07:45:06 
07:45:06 #7 importing cache manifest from xxxx/xxxx/duckdb-superset/cache:latest
07:45:06 #7 inferred cache manifest type: application/vnd.oci.image.manifest.v1+json done
07:45:06 #7 DONE 0.1s
07:45:06 
07:45:06 #8 [2/4] RUN apt-get update -y && apt-get install -y libldap2-dev gcc
07:45:06 #8 CACHED
07:45:06 
07:45:06 #9 [3/4] RUN pip install --no-cache-dir duckdb duckdb-engine python-ldap
07:45:06 #9 CACHED
07:45:06 
07:45:06 #4 [internal] load build context
07:45:06 #4 transferring context: 2B done
07:45:06 #4 DONE 0.1s
07:45:06 
07:45:06 #10 [4/4] COPY ./tmp/superset/docker/docker-*.sh /usr/local/bin/
07:45:06 #10 ERROR: lstat /tmp/superset/docker: no such file or directory
07:45:06 ------
07:45:06  > [4/4] COPY ./tmp/superset/docker/docker-*.sh /usr/local/bin/:
07:45:06 ------
07:45:06 Containerfile:8
07:45:06 --------------------
07:45:06    6 |     RUN pip install --no-cache-dir duckdb duckdb-engine python-ldap
07:45:06    7 |     
07:45:06    8 | >>> COPY ./tmp/superset/docker/docker-*.sh /usr/local/bin/
07:45:06    9 |     
07:45:06   10 |     ENV SUPERSET_CONFIG_PATH=/config/superset_config.py
07:45:06 --------------------
07:45:06 ERROR: failed to solve: lstat /tmp/superset/docker: no such file or directory
07:45:06 Command execution failed (command: docker buildx build --builder onedev --pull --cache-to type=registry,ref=xxxx/xxxx/duckdb-superset/cache:latest --cache-from type=registry,ref=xxxx/xxxx/duckdb-superset/cache:latest . -f containers/latest/Containerfile "-o type=oci,dest=-", exit code: 1)
07:45:06 Step "push to internal registry" is skipped
07:45:06 Step "remove secret" is skipped
07:45:09 Job finished

Minimal failing Containerfile:

FROM apache/superset:latest

USER root

COPY tmp/superset/docker/docker-*.sh /usr/local/bin/

USER superset

This command is run within a container cloning the git repository:

image.png

git clone --depth=1  https://github.com/apache/superset.git superset

These are the container image build settings:

image_2.png image_3.png

--cache-to=type=registry,ref=@server@/@project_path@/cache:latest --cache-from=type=registry,ref=@server@/@project_path@/cache:latest

I'm not sure if this is a configuration issue. I ran the clone command and build command locally. They succeeded.

git clone --depth=1 https://github.com/apache/superset.git tmp/superset

docker buildx build --pull --cache-to type=registry,ref=xxx/xxx/duckdb-superset/cache:latest --cache-from type=registry,ref=xxxx/xxxx/duckdb-superset/cache:latest . -f containers/latest/Containerfile

Changing the path from tmp/superset to superset makes the build succeed.

  • Robin Shen commented 7 months ago

    Seems that the job workspace does not have tmp directory. You are cloning the repository into superset directory.

  • daniel-developer commented 7 months ago

    Damn it. I took the screenshot from the fixed build pipeline. 🤦‍♂️

    Here are the logs from the failing job:

    09:15:31 Cloning into 'tmp/superset'...
09:15:32 remote: Enumerating objects: 6746, done.
09:15:32 remote: Counting objects: 100% (6746/6746), done.
09:15:34 remote: Compressing objects: 100% (5725/5725), done.
09:15:48 remote: Total 6746 (delta 2092), reused 2805 (delta 877), pack-reused 0 (from 0)
09:15:48 Receiving objects: 100% (6746/6746), 98.33 MiB | 7.37 MiB/s, done.
09:15:48 Resolving deltas: 100% (2092/2092), done.
09:15:51 Updating files: 100% (5840/5840), done.

    ls -al 

    09:15:51 total 336
09:15:51 drwxr-xr-x   18 root     root          4096 Jun  1 07:15 .
09:15:51 drwxr-xr-x    3 root     root          4096 Jun  1 07:15 ..
09:15:51 -rw-r--r--    1 root     root          3236 Jun  1 07:15 .asf.yaml
09:15:51 -rw-r--r--    1 root     root           911 Jun  1 07:15 .codecov.yml
09:15:51 -rw-r--r--    1 root     root          1240 Jun  1 07:15 .dockerignore
09:15:51 -rw-r--r--    1 root     root          1365 Jun  1 07:15 .editorconfig
09:15:51 -rw-r--r--    1 root     root           839 Jun  1 07:15 .flaskenv
09:15:51 -rwxr-xr-x    1 root     root          1176 Jun  1 07:15 .fossa.yml
09:15:51 drwxr-xr-x    7 root     root          4096 Jun  1 07:15 .git
09:15:51 -rw-r--r--    1 root     root            55 Jun  1 07:15 .gitattributes
09:15:51 drwxr-xr-x    5 root     root          4096 Jun  1 07:15 .github
09:15:51 -rw-r--r--    1 root     root          2495 Jun  1 07:15 .gitignore
09:15:51 -rw-r--r--    1 root     root          1985 Jun  1 07:15 .gitmodules
09:15:51 -rw-r--r--    1 root     root            52 Jun  1 07:15 .markdownlint.json
09:15:51 -rw-r--r--    1 root     root          3746 Jun  1 07:15 .pre-commit-config.yaml
09:15:51 -rw-r--r--    1 root     root          1097 Jun  1 07:15 .rat-excludes
09:15:51 drwxr-xr-x    2 root     root          4096 Jun  1 07:15 ASF
09:15:51 drwxr-xr-x    2 root     root          4096 Jun  1 07:15 CHANGELOG
09:15:51 -rw-r--r--    1 root     root          1580 Jun  1 07:15 CHANGELOG.md
09:15:51 -rw-r--r--    1 root     root         13012 Jun  1 07:15 CODE_OF_CONDUCT.md
09:15:51 -rw-r--r--    1 root     root          1151 Jun  1 07:15 CONTRIBUTING.md
09:15:51 -rw-r--r--    1 root     root          9633 Jun  1 07:15 Dockerfile
09:15:51 -rw-r--r--    1 root     root           978 Jun  1 07:15 INSTALL.md
09:15:51 -rw-r--r--    1 root     root         11585 Jun  1 07:15 LICENSE.txt
09:15:51 -rw-r--r--    1 root     root          1113 Jun  1 07:15 MANIFEST.in
09:15:51 -rw-r--r--    1 root     root          3232 Jun  1 07:15 Makefile
09:15:51 -rw-r--r--    1 root     root           169 Jun  1 07:15 NOTICE
09:15:51 -rw-r--r--    1 root     root         14891 Jun  1 07:15 README.md
09:15:51 drwxr-xr-x   14 root     root          4096 Jun  1 07:15 RELEASING
09:15:51 drwxr-xr-x    2 root     root          4096 Jun  1 07:15 RESOURCES
09:15:51 -rw-r--r--    1 root     root         76290 Jun  1 07:15 UPDATING.md
09:15:51 drwxr-xr-x    7 root     root          4096 Jun  1 07:15 docker
09:15:51 -rw-r--r--    1 root     root          4478 Jun  1 07:15 docker-compose-image-tag.yml
09:15:51 -rw-r--r--    1 root     root          4529 Jun  1 07:15 docker-compose-non-dev.yml
09:15:51 -rw-r--r--    1 root     root          9201 Jun  1 07:15 docker-compose.yml
09:15:51 -rw-r--r--    1 root     root          1229 Jun  1 07:15 dockerize.Dockerfile
09:15:51 drwxr-xr-x    7 root     root          4096 Jun  1 07:15 docs
09:15:51 drwxr-xr-x    3 root     root          4096 Jun  1 07:15 helm
09:15:51 -rw-r--r--    1 root     root          1793 Jun  1 07:15 lintconf.yaml
09:15:51 -rw-r--r--    1 root     root         11854 Jun  1 07:15 pyproject.toml
09:15:51 -rw-r--r--    1 root     root           900 Jun  1 07:15 pytest.ini
09:15:51 drwxr-xr-x    2 root     root          4096 Jun  1 07:15 requirements
09:15:51 drwxr-xr-x    7 root     root          4096 Jun  1 07:15 scripts
09:15:51 -rw-r--r--    1 root     root          2699 Jun  1 07:15 setup.py
09:15:51 drwxr-xr-x   48 root     root          4096 Jun  1 07:15 superset
09:15:51 drwxr-xr-x    3 root     root          4096 Jun  1 07:15 superset-embedded-sdk
09:15:51 drwxr-xr-x   10 root     root          4096 Jun  1 07:15 superset-frontend
09:15:51 drwxr-xr-x    5 root     root          4096 Jun  1 07:15 superset-websocket
09:15:51 -rw-r--r--    1 root     root          1151 Jun  1 07:15 superset_text.yml
09:15:51 drwxr-xr-x    8 root     root          4096 Jun  1 07:15 tests
    History
  • Robin Shen commented 7 months ago

    Once cloning into tmp/superset, it works at my side. So I guess there are some misconfigured path at your side. Please attach an example repository to demonstrate this issue if you believe this is a bug.

  • daniel-developer commented 7 months ago

    Ok. Thanks for looking into this.

  • daniel-developer commented 7 months ago

    I don't want to invest more time into this.

  • daniel-developer changed state to 'Closed' 7 months ago
    Previous Value Current Value 
    Open
    		 
    Closed
issue 1/1
Type
Bug
Priority
Normal
Assignee
Robin Shen
Affected Versions
11.9.9
Labels
No labels
Issue Votes (0)
Login to vote
Watchers (2)
Reference
OD-2449
Please wait...
Connection lost or session expired, reload to recover
Page is in error, reload to recover