Hi, right now COPR keeps all builds. This is not sustainable. Especially right now, when we lack of disk space.
So we need to have some policy for removing old builds.
What we have on input are: * successful builds * failed builds * builds which succeed in one chroot, but fail in another chroot
Variables for equation are: * date of the build * is it last build? * is it last successful build?
Where we will draw the line? I would suggest:
* Keep everything which was built between NOW() and X days ago. We can tune up X when we will see how fast we consume disk space. In ideal world I would imagine X as 14 days. But I can imagine that even one day would be enough.
* If build is older, keep that build only if it is last successful build in chroot.
* anything else is going to be deleted. Question: should we keep logs?
This policy can however result in having package foo-1.0 in chroot fedora-19-i386 and package foo-2.0 in chroot fedora-19-x86_64 (because foo-2.0 would fail in fedora-19-i386). Is this what we want?
Ideas?
I would just add that once we migrate to koji as backend (~7 months) this policy may become obsolete as the retention will be handled by koji.
Mirek
Is it possible to copy logs to fpaste.org? If koji doesn't want to keep, we can send logs to fpaste. (crazy idea? )
I think fpaste have time limitation selection. But will you add an option to let people set if they want to keep or not?
On 09/27/2013 12:40 AM, Christopher Meng wrote:
Is it possible to copy logs to fpaste.org http://fpaste.org? If koji doesn't want to keep, we can send logs to fpaste. (crazy idea? )
I think fpaste have time limitation selection. But will you add an option to let people set if they want to keep or not?
I did some calculation: logs are about ~2% of all data in copr-repo - right now it is 156 MB. If Copr become so popular as OBS it will mean we will have terabytes of data, which will mean that logs sent to fpaste can have tens of gigabytes. Can fpaste handle this?
On 09/26/2013 11:06 PM, Miroslav Suchy wrote:
Hi, right now COPR keeps all builds. This is not sustainable. Especially right now, when we lack of disk space.
So we need to have some policy for removing old builds.
What we have on input are:
- successful builds
- failed builds
- builds which succeed in one chroot, but fail in another chroot
Variables for equation are:
- date of the build
- is it last build?
- is it last successful build?
Where we will draw the line? I would suggest:
- Keep everything which was built between NOW() and X days ago. We can tune up X when we will see how fast we consume disk space. In ideal world I would imagine X as 14 days. But I
can imagine that even one day would be enough.
If build is older, keep that build only if it is last successful build in chroot.
anything else is going to be deleted. Question: should we keep logs?
I implemented this ^^^ as command "copr-prune-repo" and set it as cronjob.
I ran it on current copr-be and it removed 12 GB out of 20 GB.
copr-devel@lists.fedorahosted.org
-
Christopher Meng -
Miroslav Suchý