Greetings,
While performing a package review, I noticed that rpmlint doesn't like Source URL's that point to github. Rpmlint emits a 'file-size-mismatch' warning for the package I am reviewing, and a quick google search highlights additional package reviews with github source URL's having the same issue. I've not yet seen a recommended resolution in the other package reviews, so I thought this might be appropriate for the list.
I see the following warning ...
$ rpmlint -i nagios-plugins-rhev-1.0.0-2.fc16.src.rpm nagios-plugins-rhev.src: W: file-size-mismatch nagios-plugins-rhev-1.0.0.tar.gz = 9941, https://github.com/dougsland/nagios-plugins-rhev/raw/master/nagios-plugins-r... = 1 The size of the file in the package does not match the size indicated by peeking at its URL. Verify that the file in the package has the intended contents.
A quick inspection of --verbose output from curl made me believe that a redirect might be confusing rpmlint ...
< HTTP/1.1 302 Found < Server: nginx/1.0.4 < Date: Wed, 24 Aug 2011 12:10:07 GMT < Content-Type: text/html; charset=utf-8 < Connection: keep-alive < Status: 302 Found < X-RateLimit-Limit: 100 < Location: https://raw.github.com/dougsland/nagios-plugins-rhev/master/nagios-plugins-r... < X-RateLimit-Remaining: 100 < X-Runtime: 108ms < Content-Length: 158
I adjusted the SourceURL in the spec file to the redirected URL, but the original problem remains.
I'm confused, curl seems to think everything matches (size=9941)...
$ curl -v https://raw.github.com/dougsland/nagios-plugins-rhev/master/nagios-plugins-r... > /dev/null
<snip> < HTTP/1.1 200 OK <snip> < Content-Length: 9941
But rpmlint is still upset ...
nagios-plugins-rhev.src: W: file-size-mismatch nagios-plugins-rhev-1.0.0.tar.gz = 9941, https://raw.github.com/dougsland/nagios-plugins-rhev/master/nagios-plugins-r... = 1
Looking into rpmlint a bit further, abstractCheck::check_url() is discovering a content-length of 1. When I check the value of 'Content-Length' using urllib2.urlopen ... I get the expected value.
urllib2.urlopen('https://raw.github.com/dougsland/nagios-plugins-rhev/master/nagios-plugins-r...')
'9941'
I suspect something is amiss with rpmlint abstractCheck and the custom _HeadRequest.get_method(). Removing it, resolves the problem. Perhaps I should take this to the rpmlint development list as it seems related to an intentional upstream fix (http://rpmlint.zarb.org/cgi-bin/trac.cgi/ticket/165).
Any thoughts/suggestions?
Thanks, James
Greetings,
While performing a package review, I noticed that rpmlint doesn't like Source URL's that point to github. Rpmlint emits a 'file-size-mismatch' warning for the package I am reviewing, and a quick google search highlights additional package reviews with github source URL's having the same issue. I've not yet seen a recommended resolution in the other package reviews, so I thought this might be appropriate for the list.
I see the following warning ...
$ rpmlint -i nagios-plugins-rhev-1.0.0-2.fc16.src.rpm nagios-plugins-rhev.src: W: file-size-mismatch nagios-plugins-rhev-1.0.0.tar.gz = 9941, https://github.com/dougsland/nagios-plugins-rhev/raw/master/nagios-plugins-r... = 1 The size of the file in the package does not match the size indicated by peeking at its URL. Verify that the file in the package has the intended contents.
A quick inspection of --verbose output from curl made me believe that a redirect might be confusing rpmlint ...
< HTTP/1.1 302 Found < Server: nginx/1.0.4 < Date: Wed, 24 Aug 2011 12:10:07 GMT < Content-Type: text/html; charset=utf-8 < Connection: keep-alive < Status: 302 Found < X-RateLimit-Limit: 100 < Location: https://raw.github.com/dougsland/nagios-plugins-rhev/master/nagios-plugins-r... < X-RateLimit-Remaining: 100 < X-Runtime: 108ms < Content-Length: 158
I adjusted the SourceURL in the spec file to the redirected URL, but the original problem remains.
I'm confused, curl seems to think everything matches (size=9941)...
$ curl -v https://raw.github.com/dougsland/nagios-plugins-rhev/master/nagios-plugins-r...
/dev/null
<snip> < HTTP/1.1 200 OK <snip> < Content-Length: 9941
But rpmlint is still upset ...
nagios-plugins-rhev.src: W: file-size-mismatch nagios-plugins-rhev-1.0.0.tar.gz = 9941, https://raw.github.com/dougsland/nagios-plugins-rhev/master/nagios-plugins-r... = 1
Looking into rpmlint a bit further, abstractCheck::check_url() is discovering a content-length of 1. When I check the value of 'Content-Length' using urllib2.urlopen ... I get the expected value.
urllib2.urlopen('https://raw.github.com/dougsland/nagios-plugins-rhev/master/nagios-plugins-r...')
'9941'
I suspect something is amiss with rpmlint abstractCheck and the custom _HeadRequest.get_method(). Removing it, resolves the problem. Perhaps I should take this to the rpmlint development list as it seems related to an intentional upstream fix (http://rpmlint.zarb.org/cgi-bin/trac.cgi/ticket/165).
Any thoughts/suggestions?
Thanks, James -- packaging mailing list packaging@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/packaging
Do the hashes match?
-J
On Wed, 2011-08-24 at 07:59 -0500, Jon Ciesla wrote:
Do the hashes match?
They do. I should have been more clear on that in the initial email.
$ md5sum nagios-plugins-rhev-1.0.0.tar.gz* cb19d63804fca11c99996d354bae4d5a nagios-plugins-rhev-1.0.0.tar.gz cb19d63804fca11c99996d354bae4d5a nagios-plugins-rhev-1.0.0.tar.gz.upstream
Thanks, James
On Wed, 2011-08-24 at 07:59 -0500, Jon Ciesla wrote:
Do the hashes match?
They do. I should have been more clear on that in the initial email.
$ md5sum nagios-plugins-rhev-1.0.0.tar.gz* cb19d63804fca11c99996d354bae4d5a nagios-plugins-rhev-1.0.0.tar.gz cb19d63804fca11c99996d354bae4d5a nagios-plugins-rhev-1.0.0.tar.gz.upstream
No worries. Given that they match, the warning is bogus and should then be fixed by rpmlint, IMHO, unless I'm missing something.
-J
Thanks, James -- packaging mailing list packaging@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/packaging
On Wed, Aug 24, 2011 at 08:45:20AM -0400, James Laska wrote:
< Location: https://raw.github.com/dougsland/nagios-plugins-rhev/master/nagios-plugins-r...
Side comment to your main issue: How is this tarball being generated? I see in the review request that the md5sum of the file at that URL has changed over time. If it's just the upstream not officially releasing this tarball until the Fedora RPM is out and therefore making changes to the tarball to address review criteria it's not standard practice but okay. If the tarball is going to continue to evolve with this same name after the Fedora RPM is reviewed, then it's probably better to generate a git snapshot.
The aim is to make things reproducible. If we can't count on getting the same tarball once the rpm is built, we'd rather have instructions on making a snapshot that has a revision id that we can count on pulling to get the same set of files at a later date.
Thanks for the conscientious review! -Toshio
On Wed, 24 Aug 2011 07:23:30 -0700 Toshio Kuratomi a.badger@gmail.com wrote:
On Wed, Aug 24, 2011 at 08:45:20AM -0400, James Laska wrote:
< Location: https://raw.github.com/dougsland/nagios-plugins-rhev/master/nagios-plugins-r...
Side comment to your main issue: How is this tarball being generated? I see in the review request that the md5sum of the file at that URL has changed over time. If it's just the upstream not officially releasing this tarball until the Fedora RPM is out and therefore making changes to the tarball to address review criteria it's not standard practice but okay. If the tarball is going to continue to evolve with this same name after the Fedora RPM is reviewed, then it's probably better to generate a git snapshot.
The aim is to make things reproducible. If we can't count on getting the same tarball once the rpm is built, we'd rather have instructions on making a snapshot that has a revision id that we can count on pulling to get the same set of files at a later date.
I've done a few reviews on github packages. Even if you download a stable tag tarball from the project in github (which in theory should be equivalent to using a stable release tarball), it turns out that the checksums might not match a few days after.
I think github caches the tarballs it generates for a few days, so if you grab the same tarball repeatedly, you'll get the same md5sum. If you wait a longer time, you will get a different result. But even though the md5sums won't match, the contents will still be the same.
On 08/24/2011 03:45 PM, James Laska wrote:
I suspect something is amiss with rpmlint abstractCheck and the custom _HeadRequest.get_method(). Removing it, resolves the problem.
Removing it also causes rpmlint to actually fetch the content for redirected requests (HTTP HEAD vs GET) which is not what we want.
This is not a rpmlint but a github bug; github does not respond to HEAD requests as it should per the HTTP spec -- the headers to HEAD requests (including Content-Length) should be identical to the headers for the corresponding GET ones, but they're not.
If someone feels sufficiently competent with github and how packagers should interact with it, please consider helping us revise http://fedoraproject.org/wiki/Packaging:SourceURL to add a section which covers its idiosyncrasies. It needs to be done, but so far nobody on FPC feels sufficiently familiar with github to do it properly.
- J<
packaging@lists.fedoraproject.org
-
James Laska -
Jason L Tibbitts III -
Jon Ciesla -
Susi Lehtola -
Toshio Kuratomi -
Ville Skyttä