How to pass variable while running pipeline (OD-2021)
Sagar Bud opened 1 year ago

Hello @robin,

I've run the npm install command in the Dockerfile. However, I want to avoid running npm install every time the Docker build job is executed. To optimize this, I've stored the node_modules directory on a VM and have been copying it over using scp instead of running npm install.

However, when a developer updates the package.json file, I want the npm install command to run during the Docker build job.

Is there a way to set up a variable in the pipeline that developers can use to trigger npm install only when package.json changes? If there's no change, the scp command should run instead.

Thanks, Sagar

  • Sagar Bud changed fields 1 year ago
    Name Previous Value Current Value
    Assignee
    crobinson
    robin
  • Robin Shen changed state to 'Closed' 1 year ago
    Previous Value Current Value
    Open
    Closed
  • Robin Shen commented 1 year ago

    This can be achieved by caching the node_modules. For details, please check: https://docs.onedev.io/tutorials/cicd/job-cache

  • Sagar Bud changed state to 'Open' 1 year ago
    Previous Value Current Value
    Closed
    Open
  • Sagar Bud commented 1 year ago

    Hello @robin

    We have refereed your provided URL and implement it on pipeline and it is working fine. We have attached the screen capture for your reference.

    But our problem is when we change in package.json file at that time we do not require node_module cache and we want to run the command npm install for latest node modules.

    In current scenario we have to run npm install command every time, but we want to run npm install command when package.json file change. Is there any way to pass variable, when we want to run npm install command?

    Thanks

    cache.png

  • Robin Shen commented 1 year ago

    Your implementation is not the same as in tutorial. The tutorial generates cache key based on package-lock.json (which should be committed to git repository together with package.json). Please do follow the tutorial and set upload strategy as upload if not hit.

    npm install should always be called, and in case there is a cache hit when package-lock.json is not changed, it will run very fast as all npm modules are already there.

  • Robin Shen changed state to 'Closed' 1 year ago
    Previous Value Current Value
    Open
    Closed
  • Sagar Bud removed comment 1 year ago
  • Sagar Bud changed state to 'Open' 1 year ago
    Previous Value Current Value
    Closed
    Open
  • Sagar Bud commented 1 year ago

    Hello @robin

    I hope you're doing well.

    I have refereed your provided URL and implement it on pipeline. I have attached the screen capture for your reference.

    I have registered multiple docker executor. If pipeline run in same docker executor node-module cache is working but when pipeline goes to different docker executor node-module cache is not working and it will install node module again. It's taking too much time.

    cache_3.png

    cache2_2.png

    Thanks, Sagar

  • Robin Shen commented 1 year ago

    Works at my side even for different job executors. Please check if generated checksum is the same by checking below message:

    Generated checksum: <checksum of package-lock.json>
    
  • Robin Shen changed state to 'Closed' 1 year ago
    Previous Value Current Value
    Open
    Closed
  • Sagar Bud changed state to 'Open' 1 year ago
    Previous Value Current Value
    Closed
    Open
  • Sagar Bud commented 1 year ago

    Hi @robin

    I have figured out Why node-modules caching is not working? I have compared the onedev-buildspec.yaml file with the below URL and found that the in below file npm install command using in onedev-buildspec yaml file and I have used the npm install command in dockerfile. Hence, node-module cache mechanism is not working for me. https://docs.onedev.io/tutorials/cicd/job-cache

    I have attached dockerfile with the ticket. Can you please help me if i use npm imstall command in dockerfile, how we can store npm cache in that case?

    FROM us-central1-docker.pkg.dev/test-dev/test-dev/test-base:v1
    ARG UTILITY_BRANCH
    WORKDIR /home/nodeapp
    ADD ./package.json .
    ADD ./updateDependencies.js .
    RUN source ./.bash_profile && nvm install 12 && nvm use 12 &&  UTILITY_BRANCH=$UTILITY_BRANCH npm install --unsafe-perm --production
    ADD ./src ./
    EXPOSE 8080
    CMD ["npm start"]
    
  • Robin Shen commented 1 year ago

    I guess you need multi-stage docker build, or docker build cache in this case, which is not OneDev related anymore.

  • Robin Shen changed state to 'Closed' 1 year ago
    Previous Value Current Value
    Open
    Closed
issue 1/1
Type
Question
Priority
Normal
Assignee
Labels
No labels
Issue Votes (0)
Watchers (2)
Reference
OD-2021
Please wait...
Connection lost or session expired, reload to recover
Page is in error, reload to recover