Greetings.
We saw two issues in composes this morning (an ostree that failed to download things and a server dvd compose that failed to download a package).
looking at the time this happened, the apache on kojipkgs02 hit maxrequestworkers.
So, I think when this happens, varnish sends back a 502 to new requests for a few seconds while the queue processes down.
I would like to increase the limit from 256 (the default) to 512.
+1s?
kevin -- From 6733ba83e129157eb93e2ac5c24b79aebc2ac0b8 Mon Sep 17 00:00:00 2001 From: Kevin Fenzi kevin@scrye.com Date: Wed, 28 Jun 2017 20:00:45 +0000 Subject: [PATCH] up the apache limit on kojipgks from default 256 to 512. This should hopefully stop issues with 256 requests at a time causing download errors for composes
Signed-off-by: Kevin Fenzi kevin@scrye.com --- roles/kojipkgs/files/kojipkgs.conf | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/roles/kojipkgs/files/kojipkgs.conf b/roles/kojipkgs/files/kojipkgs.conf index cf8ea56..73174bd 100644 --- a/roles/kojipkgs/files/kojipkgs.conf +++ b/roles/kojipkgs/files/kojipkgs.conf @@ -1,4 +1,6 @@ ServerName https://kojipkgs.fedoraproject.org +MaxRequestWorkers 512 +ServerLimit 512
RequestHeader unset Accept-Encoding early
On Wed, Jun 28, 2017 at 10:07 PM, Kevin Fenzi kevin@scrye.com wrote:
Greetings.
We saw two issues in composes this morning (an ostree that failed to download things and a server dvd compose that failed to download a package).
looking at the time this happened, the apache on kojipkgs02 hit maxrequestworkers.
So, I think when this happens, varnish sends back a 502 to new requests for a few seconds while the queue processes down.
I would like to increase the limit from 256 (the default) to 512.
Assuming you checked we have the resources (RAM) to deal with this increase, +1.
+1s?
kevin
From 6733ba83e129157eb93e2ac5c24b79aebc2ac0b8 Mon Sep 17 00:00:00 2001 From: Kevin Fenzi kevin@scrye.com Date: Wed, 28 Jun 2017 20:00:45 +0000 Subject: [PATCH] up the apache limit on kojipgks from default 256 to 512. This should hopefully stop issues with 256 requests at a time causing download errors for composes
Signed-off-by: Kevin Fenzi kevin@scrye.com
roles/kojipkgs/files/kojipkgs.conf | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/roles/kojipkgs/files/kojipkgs.conf b/roles/kojipkgs/files/kojipkgs.conf index cf8ea56..73174bd 100644 --- a/roles/kojipkgs/files/kojipkgs.conf +++ b/roles/kojipkgs/files/kojipkgs.conf @@ -1,4 +1,6 @@ ServerName https://kojipkgs.fedoraproject.org +MaxRequestWorkers 512 +ServerLimit 512
RequestHeader unset Accept-Encoding early
-- 1.8.3.1
infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-leave@lists.fedoraproject.org
Same caveat as Patrick.. do we need to add more ram to the host? +!
On 28 June 2017 at 16:12, Patrick Uiterwijk puiterwijk@redhat.com wrote:
On Wed, Jun 28, 2017 at 10:07 PM, Kevin Fenzi kevin@scrye.com wrote:
Greetings.
We saw two issues in composes this morning (an ostree that failed to download things and a server dvd compose that failed to download a package).
looking at the time this happened, the apache on kojipkgs02 hit maxrequestworkers.
So, I think when this happens, varnish sends back a 502 to new requests for a few seconds while the queue processes down.
I would like to increase the limit from 256 (the default) to 512.
Assuming you checked we have the resources (RAM) to deal with this increase, +1.
+1s?
kevin
From 6733ba83e129157eb93e2ac5c24b79aebc2ac0b8 Mon Sep 17 00:00:00 2001 From: Kevin Fenzi kevin@scrye.com Date: Wed, 28 Jun 2017 20:00:45 +0000 Subject: [PATCH] up the apache limit on kojipgks from default 256 to 512. This should hopefully stop issues with 256 requests at a time causing download errors for composes
Signed-off-by: Kevin Fenzi kevin@scrye.com
roles/kojipkgs/files/kojipkgs.conf | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/roles/kojipkgs/files/kojipkgs.conf b/roles/kojipkgs/files/kojipkgs.conf index cf8ea56..73174bd 100644 --- a/roles/kojipkgs/files/kojipkgs.conf +++ b/roles/kojipkgs/files/kojipkgs.conf @@ -1,4 +1,6 @@ ServerName https://kojipkgs.fedoraproject.org +MaxRequestWorkers 512 +ServerLimit 512
RequestHeader unset Accept-Encoding early
-- 1.8.3.1
infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-leave@lists.fedoraproject.org
infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-leave@lists.fedoraproject.org
On 06/28/2017 02:35 PM, Stephen John Smoogen wrote:
Same caveat as Patrick.. do we need to add more ram to the host? +!
Thats a good question.
The hosts have 96GB currently with 72GB for varnish, and they are pretty full memory wise.
so, to be safe, I think we should cut varnish down to 48G to leave room for overhead and apache processes.
kevin
On Wed, Jun 28, 2017 at 02:07:24PM -0600, Kevin Fenzi wrote:
Greetings.
We saw two issues in composes this morning (an ostree that failed to download things and a server dvd compose that failed to download a package).
looking at the time this happened, the apache on kojipkgs02 hit maxrequestworkers.
So, I think when this happens, varnish sends back a 502 to new requests for a few seconds while the queue processes down.
I would like to increase the limit from 256 (the default) to 512.
+1s?
kevin
From 6733ba83e129157eb93e2ac5c24b79aebc2ac0b8 Mon Sep 17 00:00:00 2001 From: Kevin Fenzi kevin@scrye.com Date: Wed, 28 Jun 2017 20:00:45 +0000 Subject: [PATCH] up the apache limit on kojipgks from default 256 to 512. This should hopefully stop issues with 256 requests at a time causing download errors for composes
Signed-off-by: Kevin Fenzi kevin@scrye.com
roles/kojipkgs/files/kojipkgs.conf | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/roles/kojipkgs/files/kojipkgs.conf b/roles/kojipkgs/files/kojipkgs.conf index cf8ea56..73174bd 100644 --- a/roles/kojipkgs/files/kojipkgs.conf +++ b/roles/kojipkgs/files/kojipkgs.conf @@ -1,4 +1,6 @@ ServerName https://kojipkgs.fedoraproject.org +MaxRequestWorkers 512 +ServerLimit 512
RequestHeader unset Accept-Encoding early
+1 for me as well
Pierre
infrastructure@lists.fedoraproject.org