S3 for artifacts storage (OD-2047)

S3 for artifacts storage #2047

Artur opened 2 years ago

Related to OD-2046 Another idea which would solve the problem or make it less significant would be functionality to store artifacts on S3. S3 storage is usually much cheaper and is virtually unlimited in capacity so there is no danger that the system would run out of disk space and affect all services on the machine.

Copy

Activities

Robin Shen commented 2 years ago

You may mount S3 as a drive via s3fs, and configure OneDev to store artifacts on that drive. A tutorial for LFS files, but the same for artifacts:

https://docs.onedev.io/tutorials/misc/s3-lfs

Copy
Artur commented 2 years ago

Thank you, would this work if 1dev runs in k8s?

Copy
Robin Shen commented 2 years ago

For k8s, artifacts are stored on PV. I guess S3 will not work here.

Copy
Artur commented 2 years ago

Would you consider a direct S3 support for artifacts and LFS?

Copy
Robin Shen commented 2 years ago

Direct S3 support for artifacts should be possible. For LFS, it might not be possible, as OneDev relies on git built-in support for LFS which targets ordinary files.

Copy
Artur commented 2 years ago

Understand. For us S3 for LFS is not important but for build artifacts would be very useful.

Copy
Artur commented 1 year ago

Hi again,

We have recently started to use Docker registry feature built-in to OneDev. Here it is where S3 support is even more critical. Regular volumens on rented machines are much more expensive than S3 space. Please consider implementing S3, maybe, initially for packages only.

Copy
hichemmejri commented 1 year ago

can you publish the steps of publishing artifact on s3 , in pipeline it ignores the command s3 sync

Copy
Login to comment

Type	New Feature
Priority	Normal
Assignee	Robin Shen
Labels	No labels

Issue Votes (0)

Watchers (3)

Reference

OD-2047