Adam Williamson awilliam@redhat.com writes:

On Wed, 2011-08-31 at 08:50 +0200, Andreas Schwab wrote:

...

Please wait until I am finished working on it. This is not a bug that can be easily reproduced.

The Arch report claims a fully reliable reproducer:

https://bugs.archlinux.org/task/24615

"I can 100% reliably reproduce it by creating an iptables reject rule for DNS packets: # iptables -A OUTPUT -p udp --dport 53 -j REJECT --reject-with icmp-admin-prohibited # wget http://google.com/ --2011-06-08 12:13:58-- http://google.com/ Resolving google.com... zsh: segmentation fault (core dumped)"

I cannot find any of that in the bug report.

Andreas.

Zoltan Boszormenyi zboszor@freemail.hu writes:

The "I can 100%..." is not the first sentence of the comment but it's all in there.

I'm taking about the redhat bug. How do I get to know about all this if nobody tells me?

Andreas.

On Wed, 31 Aug 2011 12:11:54 +0200 Thomas Spura wrote:

On Wed, 31 Aug 2011 12:00:43 +0200 Andreas Schwab wrote:

...

Zoltan Boszormenyi zboszor@freemail.hu writes:

...

The "I can 100%..." is not the first sentence of the comment but it's all in there.

I'm taking about the redhat bug. How do I get to know about all this if nobody tells me?

Yep...

If this would be triaged before, then you would know of it: https://bugzilla.redhat.com/show_bug.cgi?id=710697

But with component = pidgin not...

Sorry, my bad. Not the same, on the second look...

Adam Williamson awilliam@redhat.com writes:

But I did mention all the various bug reports - Arch and upstream - in my ML post on the topic: subject "glibc causing crashes in most anything that does DNS lookups in F16".

That is useless. Please always put such important information in the bug report, so that it can be found.

Andreas.

Jakub Jelinek jakub@redhat.com writes:

It is also in bugzilla, just not in https://bugzilla.redhat.com/show_bug.cgi?id=730856 but in https://bugzilla.redhat.com/show_bug.cgi?id=732857 which has been marked as duplicate of that.

There should have been a comment pointing out this important information by the person who added the duplicate.

Andreas.

On Thu, 2011-09-01 at 09:48 -0700, Adam Williamson wrote:

and I'm _not_ being paid to maintain gedit.

Er...glibc. though I'm not paid to maintain gedit either. =)

Dne 31.8.2011 08:50, Andreas Schwab napsal(a):

Please wait until I am finished working on it. This is not a bug that can be easily reproduced.

I don't have a good reproducer, but I believe this one

firefox -g run<ENTER>

{ and then run Firefox for couple of hours it fails }

is pretty certain way how to get it. Happens to me couple of times a day. Would backtrace help?

Matěj

On Fri, Sep 02, 2011 at 01:20:19PM -0700, Adam Williamson wrote:

Is there a specific reason glibc does this?

Yes.

Can it not have a set of patches, one per change, as is usual practice?

Fedora glibc sources are from git, and the bit diff is just generated diff between the upstream snapshot and corresponding Fedora snapshot, sans a few Fedora-only directories (which are packaged as extra tarball).

Jakub

On Fri, 2011-09-02 at 13:43 -0700, Adam Williamson wrote:

On Fri, 2011-09-02 at 22:28 +0200, Jakub Jelinek wrote:

...

On Fri, Sep 02, 2011 at 01:20:19PM -0700, Adam Williamson wrote:

...

Is there a specific reason glibc does this?

Yes.

...

Can it not have a set of patches, one per change, as is usual practice?

Fedora glibc sources are from git, and the bit diff is just generated diff between the upstream snapshot and corresponding Fedora snapshot, sans a few Fedora-only directories (which are packaged as extra tarball).

What is the 'corresponding Fedora snapshot'? What git tree is that a snapshot of? A Fedora downstream branch of glibc, a different upstream branch...?

Looking at http://repo.or.cz/w/glibc.git , it seems we're simply talking about the 'fedora' branch of upstream glibc. Given that this is an upstream branch anyway, it would seem simpler to make our Source0 a snapshot of the 'fedora' branch upstream, rather than starting with the upstream 'master' and then adding an ugly patch and a mystery tarball to turn upstream 'master' into the 'fedora' branch. I just don't see that doing it the second way adds anything but confusion...

So this:

Source0: %{?glibc_release_url}%{glibcsrcdir}.tar.xz Source1: %{?glibc_release_url}%{glibcportsdir}.tar.xz Source2: %{glibcsrcdir}-fedora.tar.xz Patch0: %{name}-fedora.patch ... %setup -q -n %{glibcsrcdir} -b1 -b2 %patch0 -E -p1

would simply become:

# Fedora changes are an upstream git branch # git clone -b fedora git://sources.redhat.com/git/glibc.git # tar cvJf glibc-%{git}.tar.xz glibc/ --exclude=".git*" Source0: glibc-%{git}.tar.xz .... %setup -q -n glibc-%{git}

is there a problem with doing it that way? AFAIK, any upstream git branch is a legitimate 'clean source', not just master.

On Sat, 2011-09-03 at 00:50 +0200, Jan Kratochvil wrote:

On Fri, 02 Sep 2011 23:02:04 +0200, Adam Williamson wrote:

...

about the 'fedora' branch of upstream glibc.

GDB uses a similar style for the merged patchsets in the Archer repository: http://pkgs.fedoraproject.org/gitweb/?p=gdb.git;a=blob_plain;f=gdb-archer.pa...

...

Given that this is an upstream branch anyway, it would seem simpler to make our Source0 a snapshot of the 'fedora' branch upstream, rather than starting with the upstream 'master' and then adding an ugly patch and a mystery tarball to turn upstream 'master' into the 'fedora' branch. I just don't see that doing it the second way adds anything but confusion...

The suggested way is made gcc: -rw-r--r-- 1 jkratoch jkratoch 54082276 Sep 7 2010 fedora/gcc/f14/gcc-4.5.1-20100907.tar.bz2 -rw-r--r-- 1 jkratoch jkratoch 59121454 Jun 3 14:45 fedora/gcc/f15/gcc-4.6.0-20110603.tar.bz2

and I find it as difficult to manage updates over my 2Mbit ADSL (that is downloading the whole tarball again and again, instead of downloading just smaller differences); plus I was and sometimes I may be developing on GPRS.

I don't see why it would make any difference to download sizes. In fact, they should get slightly smaller. When we bump glibc from master right now, you have to download a new master tarball, plus new fedora tarball and fedora patch. If we used the fedora branch as our upstream source, you'd only have to download a new fedora branch tarball, which would be slightly smaller than those three things combined.

I guess it'd cause a bigger download when Fedora changes are made but master branch is not, but then that seems more a consequence of the Fedora changes being in upstream git than anything else, which in itself is somewhat odd.

On Sep 2, 2011, at 3:39 PM, Simo Sorce wrote:

On Fri, 2011-09-02 at 22:28 +0200, Jakub Jelinek wrote:

...

On Fri, Sep 02, 2011 at 01:20:19PM -0700, Adam Williamson wrote:

...

Is there a specific reason glibc does this?

Yes.

...

Can it not have a set of patches, one per change, as is usual practice?

Fedora glibc sources are from git, and the bit diff is just generated diff between the upstream snapshot and corresponding Fedora snapshot, sans a few Fedora-only directories (which are packaged as extra tarball).

Jakub I guess this would be some more work but why don't you just use git format-patch to get all the patches for the commits between the baseline and the top of the tree ?

That would give you a set of discrete patches that mirror the commits you have in the git tree.

Simo.

There are also ways to automate applying those patches in the spec file using git am and the %{patches} macro.

- jlk

Dne 3.9.2011 10:38, Richard W.M. Jones napsal(a):

https://rwmj.wordpress.com/2011/08/09/nice-rpm-git-patch-management-trick/

This method is quite probably simpler than the one you're using now.

I am in the process of pushing our less interesting Xorg patches upstream, and I had a great experience with quilt and managing patches as patches. Not sure how it scales to the size glibc/gdb/guestfs patches but it worked great for me.

(and of course, I wish Fedora left all the silly patch business and used git only as $DEITY intended).

Matěj

On Sat, 2011-09-03 at 13:43 +0200, Jakub Jelinek wrote:

On Sat, Sep 03, 2011 at 09:38:46AM +0100, Richard W.M. Jones wrote:

...

...

Fedora glibc sources are from git, and the bit diff is just generated diff between the upstream snapshot and corresponding Fedora snapshot, sans a few Fedora-only directories (which are packaged as extra tarball).

That's not a reason.

Why not keep the Fedora branch in git and make patches from it:

https://rwmj.wordpress.com/2011/08/09/nice-rpm-git-patch-management-trick/

This method is quite probably simpler than the one you're using now.

glibc is doing it that way for many years, even when it was diffing upstream CVS trunk vs. Fedora CVS branch, not sure if the Fedora changes from CVS era would result in something reasonable with your tricks, but moreover what glibc does now is fully scripted in the Makefiles and it would be extra work to change it to something different. I'd say it is up to the Fedora glibc maintainer to do it the way he prefers to (which is not me for a couple of years now).

Well, not entirely. There are the packaging guidelines. I took a look, and interestingly I can't find anything in there specific about how to format patches: seems that 'one-patch-for-one-change' is just a convention, albeit a very deeply-embedded one with most packagers. However, there's this, on sources:

"There are several cases where upstream is not providing the source to you in an upstream tarball. In these cases you must document how to generate the tarball used in the rpm either through a spec file comment or a script included as a separate SourceX:. "

There is no indication in the glibc spec of what the Fedora 'source' is or where it comes from; anyone coming to the spec without prior knowledge will be confused, as I was, as to the nature of this tarball. Its nature and method of generation should be documented in the spec. I'm not sure if the 'Makefiles' used to generate glibc.spec are part of upstream glibc source - if so, a simple comment which says 'look at foo/bar/Makefile in source0 for details' would be fine, if not, the Makefile should probably be included as another Source, as suggested in the guideline.

There's also this, for patches:

"All patches in Fedora spec files SHOULD have a comment above them about their upstream status. Any time you create a patch, it is best practice to file it in an upstream bug tracker, and include a link to that in the comment above the patch."

This is a SHOULD not a MUST, but really, it's pretty basic - I'd say it's more important in this case as the glibc patch is not a typical one, and I think would leave most readers of the spec file confused. Its nature and source really should be indicated by a comment. Too, there's no indication of the 'upstream status' of the Fedora changes; once you know they form a git branch, you could go upstream and look at the git commit logs to discover the *nature* of the change, I suppose, but not necessarily its *upstream status* - i.e. whether a change made in the Fedora branch of git is Fedora specific, or is planned to be merged into master, or what.

To look at things at a higher level: it's clearly the goal of the guidelines that any interested party (with sufficient basic knowledge) who comes along and checks a Fedora package out of git should be able to _understand it_, and this includes finding out where all the stuff that goes to make up the package actually comes from. glibc spec clearly doesn't achieve this goal; the casual browser is left looking at a gigantic patch and a mystery tarball and wondering where they came from and what they do, as I was. This makes glibc not at all raptor-proof, and not very amenable to outside review or improvements, which is rather against the spirit of an open source, community project.

On Sat, 2011-09-03 at 20:56 +0200, Roberto Ragusa wrote:

On 09/03/2011 07:31 PM, Adam Williamson wrote:

...

To look at things at a higher level: it's clearly the goal of the guidelines that any interested party (with sufficient basic knowledge) who comes along and checks a Fedora package out of git should be able to _understand it_, and this includes finding out where all the stuff that goes to make up the package actually comes from. glibc spec clearly doesn't achieve this goal; the casual browser is left looking at a gigantic patch and a mystery tarball and wondering where they came from and what they do, as I was. This makes glibc not at all raptor-proof, and not very amenable to outside review or improvements, which is rather against the spirit of an open source, community project.

And the mind goes to a recent case of "obfuscation by merging patches".

http://lwn.net/Articles/432012/

In that case RedHat acknowledged that a single giant patch is more difficult to understand and it confirmed that this was considered a feature (for commercial reasons); someone even started to debate if that could be considered a GPL violation, on the "source in preferred form" criteria.

Well, yes, that parallel came up in my mind too, but really, the two aren't particularly similar. I don't think there's any intent to obfuscate in the case of the glibc spec, it's simply done the way that seemed convenient to its maintainers at the time. Note the Fedora kernel package is a normal source / split out patches set. I'm not sure that whole kerfuffle is particularly relevant to Fedora.

On Tue, Sep 06, 2011 at 03:27:15PM -0800, Jef Spaleta wrote:

On Tue, Sep 6, 2011 at 3:09 PM, Adam Williamson awilliam@redhat.com wrote:

...

Well, yes, that parallel came up in my mind too, but really, the two aren't particularly similar. I don't think there's any intent to obfuscate in the case of the glibc spec, it's simply done the way that seemed convenient to its maintainers at the time. Note the Fedora kernel package is a normal source / split out patches set. I'm not sure that whole kerfuffle is particularly relevant to Fedora.

Let me turn that on its head.

As more projects become git based over time, the preferred form for code development might actually be a bisectable git checkout and not broken out patchsets for some projects. I'm not sure the distribution and packaging model that we collectively understand now and which grew up in the cvs and svn dominated era fits really well in the git dominated era. I think we are still groping around trying to figure out what the "preferred form" really is in the git dominated era. I'm not sure the broken out patchset will be it. It might soon be considered a legacy format in some situations.

While I agree with you, the glibc "big blob of patch" approach isn't in either of the preferred forms.

Wishlist item:

At the same time that RPM allows you to bundle a git repo, perhaps we can finally get rid of %changelog?

Rich.

On Wed, Sep 7, 2011 at 8:05 AM, Richard W.M. Jones rjones@redhat.com wrote:

On Tue, Sep 06, 2011 at 03:27:15PM -0800, Jef Spaleta wrote:

...

On Tue, Sep 6, 2011 at 3:09 PM, Adam Williamson awilliam@redhat.com wrote:

...

Well, yes, that parallel came up in my mind too, but really, the two aren't particularly similar. I don't think there's any intent to obfuscate in the case of the glibc spec, it's simply done the way that seemed convenient to its maintainers at the time. Note the Fedora kernel package is a normal source / split out patches set. I'm not sure that whole kerfuffle is particularly relevant to Fedora.

Let me turn that on its head.

As more projects become git based over time, the preferred form for code development might actually be a bisectable git checkout and not broken out patchsets for some projects. I'm not sure the distribution and packaging model that we collectively understand now and which grew up in the cvs and svn dominated era fits really well in the git dominated era.  I think we are still groping around trying to figure out what the "preferred form" really is in the git dominated era. I'm not sure the broken out patchset will be it. It might soon be considered a legacy format in some situations.

While I agree with you, the glibc "big blob of patch" approach isn't in either of the preferred forms.

Wishlist item:

At the same time that RPM allows you to bundle a git repo, perhaps we can finally get rid of %changelog?

%changelog isn't for developers. It's for users to see what the developers changed in the package.

josh

On Wed, Sep 7, 2011 at 12:32 PM, Genes MailLists lists@sapience.com wrote:

On 09/07/2011 09:57 AM, Josh Boyer wrote:

...

%changelog isn't for developers.  It's for users to see what the developers changed in the package.

Would a git-shortlog suffice for %changelog ? Assuming appropriate comments are required for fedora's git repo.

No... users can access %changelog for the RPMs on their system via rpm. Making them go to a git repository elsewhere to figure out what changed isn't exactly friendly.

Unless of course you meant "have fedpkg automatically stick a git-shortlog into the %changelog section of the spec file on commit" or something. Then.. maybe.

And yes, this assumes in all cases that developers are actually putting useful information in both %changelog and commit logs.

josh

On 09/07/2011 12:42 PM, Josh Boyer wrote:

On Wed, Sep 7, 2011 at 12:32 PM, Genes MailListslists@sapience.com wrote:

...

On 09/07/2011 09:57 AM, Josh Boyer wrote:

...

%changelog isn't for developers. It's for users to see what the developers changed in the package.

Would a git-shortlog suffice for %changelog ? Assuming appropriate comments are required for fedora's git repo.

No... users can access %changelog for the RPMs on their system via rpm. Making them go to a git repository elsewhere to figure out what changed isn't exactly friendly.

Unless of course you meant "have fedpkg automatically stick a git-shortlog into the %changelog section of the spec file on commit" or something. Then.. maybe.

The installer team has been doing this for anaconda for a while now. The RPM changelog block is automatically generated for us when we make a new release.

When we do a "make release", this script (among other things) is run:

http://git.fedorahosted.org/git/?p=anaconda.git;a=blob_plain;f=scripts/makeb...

And we have a commit log format we follow:

http://git.fedorahosted.org/git/?p=anaconda.git;a=blob_plain;f=docs/commit-l...

It works well for us.

And yes, this assumes in all cases that developers are actually putting useful information in both %changelog and commit logs.

On 09/07/2011 11:12 AM, Michael Cronenworth wrote:

Genes MailLists wrote:

...

Would a git-shortlog suffice for %changelog ?

It would need to be "git-short-shortlog" (hypothetically) as filling a rpm changelog with hundreds of lines of commits is not very helpful.

I've always considered the rpm changelog to be a changelog of the spec itself and a very brief summary of any upstream changes.

git log --oneline TAG-OF-PREVIOUS-RELEASE.. | cat

the | cat (or | more) is needed because git log will truncate lines

On 09/07/2011 01:50 PM, Michael Cronenworth wrote:

Rich Megginson on 09/07/2011 12:44 PM wrote:

...

git log --oneline TAG-OF-PREVIOUS-RELEASE.. | cat

the | cat (or | more) is needed because git log will truncate lines

This is not what I meant.

Upstream may have had 20-30 commits inbetween tags. I wouldn't want to see 20-30 lines of RPM changelog.

Seems pretty useful for users to see what changed - curious why not?

Am 07.09.2011 20:00, schrieb Michael Cronenworth:

Genes MailLists on 09/07/2011 12:57 PM wrote:

...

Seems pretty useful for users to see what changed - curious why not?

Users are not programmers. Commits may range from "merge from branch such-n-such" to "ran indent to clean up formatting" which has extremely little value to users.

+1 from me. Well, it would be convenient to automate the rpm changelog creation in some way. But we need *our* changelog for *our* changes to the package. Most packages ship a NEWS file anyway, which includes the changes to the software itself.

Best Regards, Mario