#984  Cache not working as expected for apt packages
Closed
Jerome St-Louis opened 1 year ago

I have the following cache set up:

  caches:
  - key: '@project_name@Cache'
    path: /var/cache/apt/archives

Yet I don't notice any improvement speeding up the builds and see outputs like Get:1 http://archive.ubuntu.com/ubuntu. It takes a good 6 minute for the actual building to start.

Am I doing something wrong? Thank you.

Robin Shen commented 1 year ago

I am not familar with apt cache. Just started a ubuntu container from command line, and run apt update, but nothing gets cached in /var/cache/apt/archives (it only contains file lock and directory partial). What command are you using to populate the cache?

Marcos de Oliveira commented 1 year ago

I think you also need /var/lib/apt/lists to cache indexes. /Var/cache/apt/archives are for packages, iirc...

Jerome St-Louis commented 1 year ago

@robin @markkrj Thank you for the help, I tried also caching /var/lib/apt/lists, as well as the whole /var/cache/apt directory. I am installing packages with apt install. I realized that apt does not keep the packages by default, while apt-get does, so I tried applying the following fix:

      - echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/98keep-debs
      - echo 'APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/99keep-debs

Somehow I still get Get: requests and it does not seem any faster, it still takes a good 6 minutes for the actual building to start.

Full build spec here.

Not sure what is going on...

Robin Shen commented 1 year ago

Thanks for the info @markkrj. Just cached /var/lib/apt/lists, and run apt-get update, the cache works as expected. The second run finishes very fast with all cache hit message.

Not sure why apt cache does not work. But at OneDev side, it is doing the job as expected (remount cache to specified directory in container). Also please note that cache may gets cleared from time to time (one week by default, and can be tuned in job executor). So do not assume it should contain any particular files.

Robin Shen changed state to 'Closed' 1 year ago
Previous Value Current Value
Open
Closed
Jerome St-Louis commented 1 year ago

@robin @markkrj Re-investigating this, my issue was that in the ubuntu:18.04 Docker image that I am using, /etc/apt/apt.conf.d/docker-clean contains the following:

DPkg::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; };
APT::Update::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; };
Dir::Cache::pkgcache ""; Dir::Cache::srcpkgcache "";

Removing that file at the beginning of my build spec solved the issue.

issue 1 of 1
Type
Question
Priority
Normal
Assignee
Issue Votes (0)
Watchers (4)
Reference
onedev/server#984
Please wait...
Page is in error, reload to recover