From: Mathieu Bridon bochecha@daitauha.fr
With the current code, trying to SSL-login with a bad certificate will just make it look like the client code is hanging.
That's because it tries and tries again, silently, until it reaches it's maximum retry limit.
But in the case of an SSL error, such as an expired client cert, there's really no point in retrying. --- koji/__init__.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/koji/__init__.py b/koji/__init__.py index 58971da..9ebe83a 100644 --- a/koji/__init__.py +++ b/koji/__init__.py @@ -57,7 +57,7 @@ import xmlrpclib import xml.sax import xml.sax.handler from xmlrpclib import loads, dumps, Fault -#import OpenSSL.SSL +import OpenSSL import zipfile
def _(args): @@ -1938,6 +1938,9 @@ class ClientSession(object): except (SystemExit, KeyboardInterrupt): #(depending on the python version, these may or may not be subclasses of Exception) raise + except OpenSSL.SSL.Error as e: + # There's no point in retrying this + raise except Exception, e: self._close_connection() if not self.logged_in:
This appears to work as you intend.
I restored a very old and expired backup copy of my .fedora.cert file. Then attempted to scratch build an srpm:
$ koji build --scratch --nowait f23 /home/jdisnard/fedora-scm/glmark2/glmark2-2014.03-3.fc23.src.rpm Error: [('SSL routines', 'SSL3_READ_BYTES', 'sslv3 alert certificate revoked'), ('SSL routines', 'SSL3_READ_BYTES', 'ssl handshake failure')]
My only question is why the previous OpenSSL import line was commented-out ? Care to speculate? I'm guessing the SSLCommon was enough?
Regardless the patch looks good.
ACK
On Fri, Jun 12, 2015 at 4:26 PM, Mathieu Bridon bochecha@fedoraproject.org wrote:
From: Mathieu Bridon bochecha@daitauha.fr
With the current code, trying to SSL-login with a bad certificate will just make it look like the client code is hanging.
That's because it tries and tries again, silently, until it reaches it's maximum retry limit.
But in the case of an SSL error, such as an expired client cert, there's really no point in retrying.
koji/__init__.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/koji/__init__.py b/koji/__init__.py index 58971da..9ebe83a 100644 --- a/koji/__init__.py +++ b/koji/__init__.py @@ -57,7 +57,7 @@ import xmlrpclib import xml.sax import xml.sax.handler from xmlrpclib import loads, dumps, Fault -#import OpenSSL.SSL +import OpenSSL import zipfile
def _(args): @@ -1938,6 +1938,9 @@ class ClientSession(object): except (SystemExit, KeyboardInterrupt): #(depending on the python version, these may or may not be subclasses of Exception) raise
except OpenSSL.SSL.Error as e:# There's no point in retrying thisraise except Exception, e: self._close_connection() if not self.logged_in:-- 2.4.3
-- buildsys mailing list buildsys@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/buildsys
On Fri, 2015-06-12 at 22:06 -0500, Jon wrote:
This appears to work as you intend.
I restored a very old and expired backup copy of my .fedora.cert file. Then attempted to scratch build an srpm:
$ koji build --scratch --nowait f23 /home/jdisnard/fedora -scm/glmark2/glmark2-2014.03-3.fc23.src.rpm Error: [('SSL routines', 'SSL3_READ_BYTES', 'sslv3 alert certificate revoked'), ('SSL routines', 'SSL3_READ_BYTES', 'ssl handshake failure')]
My only question is why the previous OpenSSL import line was commented-out ? Care to speculate? I'm guessing the SSLCommon was enough?
It was commented out in commit 9e9549d994d750e5eca0729afd30eef794e129fc. At that point, it hadn't been needed for a while, so I'm not sure why it wasn't just removed.
The import hadn't been needed since commit 54f79ff665fd4147b889b1e18e5846de3476b4e4, which is the one that introduced the retry mechanism.
Before this commit, there was a similar code to the one I'm introducing in this patch: the code would just reraise the exception if it was an SSL-related error.
My guess is that when the code was made to retry a few times on failures, it was omitted that there isn't a need to retry if the problem is with the SSL certs.
My patch just reintroduces that, as IMHO it shouldn't have been removed in the first place.
Regardless the patch looks good.
ACK
Thanks. Could this be merged, then?
On 06/28/2015 01:23 PM, Mathieu Bridon wrote:
On Fri, 2015-06-12 at 22:06 -0500, Jon wrote:
This appears to work as you intend.
I restored a very old and expired backup copy of my .fedora.cert file. Then attempted to scratch build an srpm:
$ koji build --scratch --nowait f23 /home/jdisnard/fedora -scm/glmark2/glmark2-2014.03-3.fc23.src.rpm Error: [('SSL routines', 'SSL3_READ_BYTES', 'sslv3 alert certificate revoked'), ('SSL routines', 'SSL3_READ_BYTES', 'ssl handshake failure')]
My only question is why the previous OpenSSL import line was commented-out ? Care to speculate? I'm guessing the SSLCommon was enough?
It was commented out in commit 9e9549d994d750e5eca0729afd30eef794e129fc. At that point, it hadn't been needed for a while, so I'm not sure why it wasn't just removed.
The import hadn't been needed since commit 54f79ff665fd4147b889b1e18e5846de3476b4e4, which is the one that introduced the retry mechanism.
Before this commit, there was a similar code to the one I'm introducing in this patch: the code would just reraise the exception if it was an SSL-related error.
My guess is that when the code was made to retry a few times on failures, it was omitted that there isn't a need to retry if the problem is with the SSL certs.
My patch just reintroduces that, as IMHO it shouldn't have been removed in the first place.
Regardless the patch looks good.
ACK
Thanks. Could this be merged, then?
Btw. this patch may fix the bug filed against fedkg too: https://bugzilla.redhat.com/show_bug.cgi?id=1207178
On Mon, 2015-06-29 at 10:42 +0200, Pavol Babincak wrote:
Btw. this patch may fix the bug filed against fedkg too: https://bugzilla.redhat.com/show_bug.cgi?id=1207178
Not "may", it does fix this bug. That's how I encountered it, and how I tested that my patch fixes it.
I didn't know about the BZ, though.
From: Mathieu Bridon bochecha@daitauha.fr
With the current code, trying to SSL-login with a bad certificate will just make it look like the client code is hanging.
That's because it tries and tries again, silently, until it reaches it's maximum retry limit.
But in the case of an SSL error, such as an expired client cert, there's really no point in retrying.
koji/__init__.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/koji/__init__.py b/koji/__init__.py index 58971da..9ebe83a 100644 --- a/koji/__init__.py +++ b/koji/__init__.py @@ -57,7 +57,7 @@ import xmlrpclib import xml.sax import xml.sax.handler from xmlrpclib import loads, dumps, Fault -#import OpenSSL.SSL +import OpenSSL import zipfile
def _(args): @@ -1938,6 +1938,9 @@ class ClientSession(object): except (SystemExit, KeyboardInterrupt): #(depending on the python version, these may or may not be subclasses of Exception) raise
except OpenSSL.SSL.Error as e:# There's no point in retrying thisraise except Exception, e: self._close_connection() if not self.logged_in:-- 2.4.3
This has actually caused me issues with newRepo on el6. I get the following traceback from kojid:
Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/koji/daemon.py", line 1161, in runTask response = (handler.run(),) File "/usr/lib/python2.6/site-packages/koji/tasks.py", line 158, in run return koji.util.call_with_argcheck(self.handler, self.params, self.opts) File "/usr/lib/python2.6/site-packages/koji/util.py", line 154, in call_with_argcheck return func(*args, **kwargs) File "/usr/sbin/kojid", line 4243, in handler results = self.wait(subtasks.values(), all=True, failany=True) File "/usr/lib/python2.6/site-packages/koji/tasks.py", line 215, in wait finished, unfinished = self.session.host.taskWait(self.id) File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1577, in __call__ return self.__func(self.__name,args,opts) File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1920, in _callMethod return self._sendCall(handler, headers, request) File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1831, in _sendCall return self._sendOneCall(handler, headers, request) File "/usr/lib/python2.6/site-packages/koji/__init__.py", line 1850, in _sendOneCall cnx.send(request) File "/usr/lib64/python2.6/httplib.py", line 759, in send self.sock.sendall(str) File "/usr/lib/python2.6/site-packages/koji/ssl/SSLConnection.py", line 111, in sendall self.close() File "/usr/lib/python2.6/site-packages/koji/ssl/SSLConnection.py", line 82, in close self.shutdown() File "/usr/lib/python2.6/site-packages/koji/ssl/SSLConnection.py", line 53, in shutdown self.__dict__["conn"].shutdown() Error: []
http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=3087577
On Wed, 2015-07-22 at 00:45 +0100, Peter Robinson wrote:
This has actually caused me issues with newRepo on el6. I get the following traceback from kojid:
Are you sure this is due to this patch? Looking at the traceback, it doesn't seem to be passing through the lines I added. :-/
Also, all the patch did, is to stop retrying on an SSL error, because no matter how many times Koji retries, you'd still get the same SSL error.
Which means if you're getting an error because of this patch, then you'd have had it without the patch as well, only after a few retries.
Which means if this patch is responsible for the error you got, then I'd say there already is a problem with your SSL setup.
But in any case, I really don't see how this patch could have caused your problem. :-/
On Wed, Jul 22, 2015 at 8:28 AM, Mathieu Bridon bochecha@daitauha.fr wrote:
On Wed, 2015-07-22 at 00:45 +0100, Peter Robinson wrote:
This has actually caused me issues with newRepo on el6. I get the following traceback from kojid:
Are you sure this is due to this patch? Looking at the traceback, it doesn't seem to be passing through the lines I added. :-/
Also, all the patch did, is to stop retrying on an SSL error, because no matter how many times Koji retries, you'd still get the same SSL error.
Which means if you're getting an error because of this patch, then you'd have had it without the patch as well, only after a few retries.
Which means if this patch is responsible for the error you got, then I'd say there already is a problem with your SSL setup.
But in any case, I really don't see how this patch could have caused your problem. :-/
Well as far as I could tell it was the only recent change to that code and commenting it out everything started working again. I might have another one that seems related, might not be. This time when signing updates using secondary-sigul, network connection was contrained (ISP having traffic issues) with low bandwidth. Again only change around this was the update to koji 1.10 client.
Passphrase for fedora-22-secondary: 2015-07-22 11:26:30,139 INFO: Setting up koji session 2015-07-22 11:26:34,605 INFO: Getting builds from f22-updates 2015-07-22 11:35:45,683 INFO: Got 1712 builds 2015-07-22 11:35:45,684 INFO: Getting build IDs from Koji Traceback (most recent call last): File "./sigulsign_unsigned.py", line 534, in <module> build_ids, buildID_errors = kojihelper.get_build_ids(buildNVRs) File "./sigulsign_unsigned.py", line 214, in get_build_ids for build, result in zip(nvrs, self.kojisession.multiCall()): File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1986, in multiCall ret = self._callMethod('multiCall', (calls,), {}) File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1920, in _callMethod return self._sendCall(handler, headers, request) File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1831, in _sendCall return self._sendOneCall(handler, headers, request) File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1853, in _sendOneCall ret = self._read_xmlrpc_response(response, handler) File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1888, in _read_xmlrpc_response chunk = response.read(8192) File "/usr/lib64/python2.7/httplib.py", line 612, in read s = self.fp.read(amt) File "/usr/lib64/python2.7/socket.py", line 384, in read data = self._sock.recv(left) File "/usr/lib/python2.7/site-packages/koji/ssl/SSLConnection.py", line 140, in recv return con.recv(bufsize, flags) File "/usr/lib/python2.7/site-packages/OpenSSL/SSL.py", line 995, in recv self._raise_ssl_error(self._ssl, result) File "/usr/lib/python2.7/site-packages/OpenSSL/SSL.py", line 864, in _raise_ssl_error raise SysCallError(-1, "Unexpected EOF") OpenSSL.SSL.SysCallError: (-1, 'Unexpected EOF')
On 2015-07-22 02:28, Mathieu Bridon wrote:
On Wed, 2015-07-22 at 00:45 +0100, Peter Robinson wrote:
This has actually caused me issues with newRepo on el6. I get the following traceback from kojid:
Are you sure this is due to this patch? Looking at the traceback, it doesn't seem to be passing through the lines I added. :-/
That's because a plain `raise` doesn't modify the traceback.
Also, all the patch did, is to stop retrying on an SSL error, because no matter how many times Koji retries, you'd still get the same SSL error.
Not necessarily. The EOF error is pretty commonly transient, and retrying once or twice is typically sufficient to work around it.
On Friday, June 12, 2015 11:26:34 PM Mathieu Bridon wrote:
From: Mathieu Bridon bochecha@daitauha.fr
With the current code, trying to SSL-login with a bad certificate will just make it look like the client code is hanging.
That's because it tries and tries again, silently, until it reaches it's maximum retry limit.
But in the case of an SSL error, such as an expired client cert, there's really no point in retrying.
koji/__init__.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/koji/__init__.py b/koji/__init__.py index 58971da..9ebe83a 100644 --- a/koji/__init__.py +++ b/koji/__init__.py @@ -57,7 +57,7 @@ import xmlrpclib import xml.sax import xml.sax.handler from xmlrpclib import loads, dumps, Fault -#import OpenSSL.SSL +import OpenSSL import zipfile
def _(args): @@ -1938,6 +1938,9 @@ class ClientSession(object): except (SystemExit, KeyboardInterrupt): #(depending on the python version, these may or may not be subclasses of Exception) raise
except OpenSSL.SSL.Error as e:# There's no point in retrying thisraise except Exception, e: self._close_connection() if not self.logged_in:
Applied, thanks
Dennis
On Fri, 12 Jun 2015 23:26:34 +0200 Mathieu Bridon bochecha@fedoraproject.org wrote:
From: Mathieu Bridon bochecha@daitauha.fr
With the current code, trying to SSL-login with a bad certificate will just make it look like the client code is hanging.
That's because it tries and tries again, silently, until it reaches it's maximum retry limit.
But in the case of an SSL error, such as an expired client cert, there's really no point in retrying.
this change caused us troubles with the new s390 hub and had to be reverted at the end
http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=1919798 aka "SysCallError: (-1, 'Unexpected EOF')" with koji 1.10 (from Fedora Infra repos)
same result with the proposed patch (paste.fedoraproject.org/247278/63976214/)
only after reverting 4de27c52de I got successful newRepo
Dan
koji/__init__.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/koji/__init__.py b/koji/__init__.py index 58971da..9ebe83a 100644 --- a/koji/__init__.py +++ b/koji/__init__.py @@ -57,7 +57,7 @@ import xmlrpclib import xml.sax import xml.sax.handler from xmlrpclib import loads, dumps, Fault -#import OpenSSL.SSL +import OpenSSL import zipfile
def _(args): @@ -1938,6 +1938,9 @@ class ClientSession(object): except (SystemExit, KeyboardInterrupt): #(depending on the python version, these may or #may not be subclasses of Exception) raise
except OpenSSL.SSL.Error as e:# There's no point in retrying thisraise except Exception, e: self._close_connection() if not self.logged_in:-- 2.4.3
-- buildsys mailing list buildsys@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/buildsys
On Fri, 2015-07-24 at 22:32 +0200, Dan Horák wrote:
On Fri, 12 Jun 2015 23:26:34 +0200 Mathieu Bridon bochecha@fedoraproject.org wrote:
From: Mathieu Bridon bochecha@daitauha.fr
With the current code, trying to SSL-login with a bad certificate will just make it look like the client code is hanging.
That's because it tries and tries again, silently, until it reaches it's maximum retry limit.
But in the case of an SSL error, such as an expired client cert, there's really no point in retrying.
this change caused us troubles with the new s390 hub and had to be reverted at the end
http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=1919798 aka "SysCallError: (-1, 'Unexpected EOF')" with koji 1.10 (from Fedora Infra repos)
same result with the proposed patch (paste.fedoraproject.org/247278/63976214/)
only after reverting 4de27c52de I got successful newRepo
Well, yeah, the patchthat paste link **does** remove the 3 lines introduced in 4de27c52de, so in effect it reverts it.
So I'm a bit puzzled as to how you could get the "same effect until reverting 4de27c52de", because if you had applied that patch, it would already have reverted 4de27c52de. :-/
buildsys@lists.fedoraproject.org
-
Dan Horák -
Dennis Gilmore -
Dustin C. Hatch -
Jon -
Mathieu Bridon -
Mathieu Bridon -
Pavol Babincak -
Peter Robinson