-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Hi,
Earlier today, the riddler.io bot pulled Bodhi under by performing a lot of heavy requests.
I know from experience that this bot does NOT listen to robots.txt, even though the website says it does (it hit my personal webserver as well at an earlier time), so I have currently blocked it by user agent.
We probably need to fix this better soon, but for the time being I put this in to make Bodhi behave again.
Any comments?
1 files changed, 3 insertions(+), 1 deletions(-) [puiterwijk@lockbox01 files]$ git show HEAD commit a635e91849f29dd3c146b8067f602871ff959f00 Author: Patrick Uiterwijk puiterwijk@redhat.com Date: Fri Apr 17 11:29:48 2015 +0000
Block the Riddler.io bot from accessing bodhi
This bot does NOT follow robots.txt, even though it announces that it does, and it hits Bodhi so much that it pulls the bodhi servers under.
An email has been sent to the maintainer of the bot, but for the time being, let's block it from using any bodhi resources.
Signed-off-by: Patrick Uiterwijk puiterwijk@redhat.com
diff --git a/roles/bodhi/base/files/bodhi-app.conf b/roles/bodhi/base/files/bodhi-app.conf index 3e10a59..3f44964 100644 - --- a/roles/bodhi/base/files/bodhi-app.conf +++ b/roles/bodhi/base/files/bodhi-app.conf @@ -13,6 +13,8 @@ Alias /updates/tg_widgets/tgmochikit/packed/MochiKit/MochiKit.js /usr/lib/python
<Directory /usr/share/bodhi> WSGIProcessGroup bodhi - - Order deny,allow + SetEnvIf User-Agent Riddler GoAway=1 + Order allow,deny Allow from all + Deny from env=GoAway </Directory>
On Fri, Apr 17, 2015 at 01:36:50PM +0200, Patrick Uiterwijk wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Hi,
Earlier today, the riddler.io bot pulled Bodhi under by performing a lot of heavy requests.
I know from experience that this bot does NOT listen to robots.txt, even though the website says it does (it hit my personal webserver as well at an earlier time), so I have currently blocked it by user agent.
Is this something we could report to them?
1 files changed, 3 insertions(+), 1 deletions(-) [puiterwijk@lockbox01 files]$ git show HEAD commit a635e91849f29dd3c146b8067f602871ff959f00 Author: Patrick Uiterwijk puiterwijk@redhat.com Date: Fri Apr 17 11:29:48 2015 +0000
Block the Riddler.io bot from accessing bodhi This bot does NOT follow robots.txt, even though it announces that it does, and it hits Bodhi so much that it pulls the bodhi servers under. An email has been sent to the maintainer of the bot, but for the time being, let's block it from using any bodhi resources. Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
diff --git a/roles/bodhi/base/files/bodhi-app.conf b/roles/bodhi/base/files/bodhi-app.conf index 3e10a59..3f44964 100644
- --- a/roles/bodhi/base/files/bodhi-app.conf
+++ b/roles/bodhi/base/files/bodhi-app.conf @@ -13,6 +13,8 @@ Alias /updates/tg_widgets/tgmochikit/packed/MochiKit/MochiKit.js /usr/lib/python
<Directory /usr/share/bodhi> WSGIProcessGroup bodhi
- Order deny,allow
- SetEnvIf User-Agent Riddler GoAway=1
- Order allow,deny Allow from all
- Deny from env=GoAway
</Directory>
+1 for me, thanks!
Pierre
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Is this something we could report to them?
Yup, we can, and I have. I will post to the list if I get a reply.
+1 for me, thanks!
Thanks!
Pierre
Patrick
infrastructure@lists.fedoraproject.org