First off I would like to say that I'm very impressed by the effort you have put into java packaging over the last few years. Getting a usable environment for offline maven execution is no small feat.
One thing I noted, however, was that the unit tests for the guava library was not run on package build. Which seems like a reasonable compromise but not ideal. I decided to look into what would be needed dependency wise for that to happen, and I came up with the following:
* updates-testing needs to be activated for the auto-value dependency.
* packages truth, allocation-instrumenter and caliper needs to be introduced. I have created those packages and built them with guidance from the Java Packaging HOWTO https://fedorahosted.org/released/javapackages/doc/, the packages (SRPMS and mock built noarch.rpms) are available from https://resare.com/noa-fedora-playground/repo/22 but please beware of newbie packaging mistakes :)
* With the dependencies handled I updated guava.spec to re-enable guava-testlib and guava-tests. One test was consistently failing due to new java8 behaviour, and I back ported a fix from guava master. The changes needed can be viewed in my fork of git://pkgs.fedoraproject.org/guava.git at https://github.com/nresare/fedora-guava in the testlib-tests branch. The actual commit can be viewed here: https://github.com/nresare/fedora-guava/commit/c1a8a831b1ef0ec91509fea806625...
I would love for my work to be of use in the fedora project and I'm willing to spend some time to land my contributions at this time. I will now read up on what I need to to contribute more formally.
/noa
On 4 June 2015 at 09:19, Noa Resare noa@spotify.com wrote:
First off I would like to say that I'm very impressed by the effort you have put into java packaging over the last few years. Getting a usable environment for offline maven execution is no small feat.
One thing I noted, however, was that the unit tests for the guava library was not run on package build. Which seems like a reasonable compromise but not ideal. I decided to look into what would be needed dependency wise for that to happen, and I came up with the following:
updates-testing needs to be activated for the auto-value dependency.
packages truth, allocation-instrumenter and caliper needs to be
introduced. I have created those packages and built them with guidance from the Java Packaging HOWTO https://fedorahosted.org/released/javapackages/doc/, the packages (SRPMS and mock built noarch.rpms) are available from https://resare.com/noa-fedora-playground/repo/22 but please beware of newbie packaging mistakes :)
- With the dependencies handled I updated guava.spec to re-enable
guava-testlib and guava-tests. One test was consistently failing due to new java8 behaviour, and I back ported a fix from guava master. The changes needed can be viewed in my fork of git://pkgs.fedoraproject.org/guava.git at https://github.com/nresare/fedora-guava in the testlib-tests branch. The actual commit can be viewed here: https://github.com/nresare/fedora-guava/commit/c1a8a831b1ef0ec91509fea806625...
I would love for my work to be of use in the fedora project and I'm willing to spend some time to land my contributions at this time. I will now read up on what I need to to contribute more formally.
/noa
-- Spotify Free Software ombudsman. I/O Tribe.
-- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
This is great to hear -- it sounds like useful work you have done and I look forward to seeing your new packages appear in Fedora :-)
I assume you have seen this document already: https://fedoraproject.org/wiki/Join_the_package_collection_maintainers?rd=Pa...
On Thu, Jun 4, 2015 at 11:33 AM, Mat Booth fedora@matbooth.co.uk wrote:
On 4 June 2015 at 09:19, Noa Resare noa@spotify.com wrote:
First off I would like to say that I'm very impressed by the effort you have put into java packaging over the last few years. Getting a usable environment for offline maven execution is no small feat.
One thing I noted, however, was that the unit tests for the guava library was not run on package build. Which seems like a reasonable compromise but not ideal. I decided to look into what would be needed dependency wise for that to happen, and I came up with the following:
updates-testing needs to be activated for the auto-value dependency.
packages truth, allocation-instrumenter and caliper needs to be
introduced. I have created those packages and built them with guidance from the Java Packaging HOWTO https://fedorahosted.org/released/javapackages/doc/, the packages (SRPMS and mock built noarch.rpms) are available from https://resare.com/noa-fedora-playground/repo/22 but please beware of newbie packaging mistakes :)
- With the dependencies handled I updated guava.spec to re-enable
guava-testlib and guava-tests. One test was consistently failing due to new java8 behaviour, and I back ported a fix from guava master. The changes needed can be viewed in my fork of git://pkgs.fedoraproject.org/guava.git at https://github.com/nresare/fedora-guava in the testlib-tests branch. The actual commit can be viewed here: https://github.com/nresare/fedora-guava/commit/c1a8a831b1ef0ec91509fea806625...
I would love for my work to be of use in the fedora project and I'm willing to spend some time to land my contributions at this time. I will now read up on what I need to to contribute more formally.
/noa
-- Spotify Free Software ombudsman. I/O Tribe.
-- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
This is great to hear -- it sounds like useful work you have done and I look forward to seeing your new packages appear in Fedora :-)
I assume you have seen this document already: https://fedoraproject.org/wiki/Join_the_package_collection_maintainers?rd=Pa...
Thanks!
I hit a slight snag in my progression to publish the depenencies. It turns out that one of the dependencies, caliper doesn't build in rawhide because of a dependency on jersey-client:1.11. There is a compatibility package in rawhide named jersey1 which provides jersey-client:1.19 but it seems like the "fuzzy artefact resolution" going on works differently when depending on compat packages and there needs to be an exact version match (updating the dependency version to something jersey1 provides, such as 1.19 or 1 makes the package build.)
What is the best cause of action here? A, flip the dependency from 1.11 to 1.19. B, flip the dependency from 1.11 to 1. Any of those seems to have potential to be "interesting" when some other code has it's more specific dependency on jersey fulfilled transitively.
Another quick question: attempting to use the same copr project for both a released version of fedora and rawhide doesn't make a lot of sense, in the case of java with fast moving and complex dependencies, right? I assume after having run into this that I would want one copr project per target so that I can let the packages diverge.
/noa
Il 04/06/2015 23:15, Noa Resare ha scritto:
Thanks!
I hit a slight snag in my progression to publish the depenencies. It turns out that one of the dependencies, caliper doesn't build in rawhide because of a dependency on jersey-client:1.11. There is a compatibility package in rawhide named jersey1 which provides jersey-client:1.19 but it seems like the "fuzzy artefact resolution" going on works differently when depending on compat packages and there needs to be an exact version match (updating the dependency version to something jersey1 provides, such as 1.19 or 1 makes the package build.)
What is the best cause of action here? A, flip the dependency from 1.11 to 1.19. B, flip the dependency from 1.11 to 1. Any of those seems to have potential to be "interesting" when some other code has it's more specific dependency on jersey fulfilled transitively.
Another quick question: attempting to use the same copr project for both a released version of fedora and rawhide doesn't make a lot of sense, in the case of java with fast moving and complex dependencies, right? I assume after having run into this that I would want one copr project per target so that I can let the packages diverge.
/noa
hi
it should work both ways. but in case of an upgrade of jersey1 package
you must change the version references an example ho manage a compat package: %pom_xpath_set "pom:dependency[pom:groupId = 'com.sun.jersey']/pom:version" 1 or %pom_xpath_set "pom:dependency[pom:groupId = 'com.sun.jersey']/pom:version" 1.19 the same should be done for hibernate-core
regards gil
On 06/04/2015 11:15 PM, Noa Resare wrote:
I hit a slight snag in my progression to publish the depenencies. It turns out that one of the dependencies, caliper doesn't build in rawhide because of a dependency on jersey-client:1.11. There is a compatibility package in rawhide named jersey1 which provides jersey-client:1.19 but it seems like the "fuzzy artefact resolution" going on works differently when depending on compat packages and there needs to be an exact version match (updating the dependency version to something jersey1 provides, such as 1.19 or 1 makes the package build.)
What is the best cause of action here? A, flip the dependency from 1.11 to 1.19. B, flip the dependency from 1.11 to 1. Any of those seems to have potential to be "interesting" when some other code has it's more specific dependency on jersey fulfilled transitively.
IMHO compat packages should be used or introduced only as last resort, mostly because they prevent innovation and force us to waste our precious time on maintaining legacy stuff. See also [1].
In general, the workflow I use:
Try to build the package with latest dependency version; if (there are compilation or other problems) { Check latest upstream development branch; if (upstream already migrated to later dep version) { Backport upstream patch and include in the spec; } else { try { Port package to newer dep version; Include patch in spec file; Forward patch upstream; } catch (porting not feasible) { Ask upstream about updating to new dep version; Give them some time to react; if (upstream not responding || package is needed urgently) { Use compat package as dependency; Periodically check upstream status of dependency; Try to remove dep on compat package as soon as it is possible; } } } }
Sorry for pseudo-code, I hope that this is readable :)
If after reading the above you really want to use compat package, then indeed you need to use exact dependency version. Better version matching for compat packages has been on my todo [2] for long time, but as you probably noticed, compat packages are not top priority for me :)
Another quick question: attempting to use the same copr project for both a released version of fedora and rawhide doesn't make a lot of sense, in the case of java with fast moving and complex dependencies, right? I assume after having run into this that I would want one copr project per target so that I can let the packages diverge.
Assuming that the goal is to eventually contribute these packages to Fedora, I would advice to focus on rawhide only for development. Once rawhide package is done you can consider porting it to older Fedora (or EPEL) versions.
You don't need to install rawhide to target it. If used correctly, mock tool can be very useful in building rawhide packages on other systems.
[1] https://lists.fedoraproject.org/pipermail/java-devel/2013-October/005000.htm... [2] https://github.com/mizdebsk/xmvn/blob/master/TODO#L50-L51
Thank you for all the useful feedback. In the latest few days, I have looked quite closely at the packages I prepared and concluded that the guava change to enable testlib and unit tests can be made much smaller if the caliper dependency is simply disabled.
The dependency is not used by the default maven build (caliper is a benchmarking framework and benchmarking, while useful, is not part of the code shipping or testing)
This means that we are down to a single extra dependency to be able to enable guava testing, the truth package.
I have submitted a review request and I'm looking for sponsors: https://bugzilla.redhat.com/show_bug.cgi?id=1229704
On Fri, Jun 5, 2015 at 8:30 AM, Mikolaj Izdebski mizdebsk@redhat.com wrote:
On 06/04/2015 11:15 PM, Noa Resare wrote:
I hit a slight snag in my progression to publish the depenencies. It
turns
out that one of the dependencies, caliper doesn't build in rawhide
because
of a dependency on jersey-client:1.11. There is a compatibility package
in
rawhide named jersey1 which provides jersey-client:1.19 but it seems like the "fuzzy artefact resolution" going on works differently when depending on compat packages and there needs to be an exact version match (updating the dependency version to something jersey1 provides, such as 1.19 or 1 makes the package build.)
What is the best cause of action here? A, flip the dependency from 1.11
to
1.19. B, flip the dependency from 1.11 to 1. Any of those seems to have potential to be "interesting" when some other code has it's more specific dependency on jersey fulfilled transitively.
IMHO compat packages should be used or introduced only as last resort, mostly because they prevent innovation and force us to waste our precious time on maintaining legacy stuff. See also [1].
In general, the workflow I use:
Try to build the package with latest dependency version; if (there are compilation or other problems) { Check latest upstream development branch; if (upstream already migrated to later dep version) { Backport upstream patch and include in the spec; } else { try { Port package to newer dep version; Include patch in spec file; Forward patch upstream; } catch (porting not feasible) { Ask upstream about updating to new dep version; Give them some time to react; if (upstream not responding || package is needed urgently) { Use compat package as dependency; Periodically check upstream status of dependency; Try to remove dep on compat package as soon as it is possible; } } } }
Sorry for pseudo-code, I hope that this is readable :)
If after reading the above you really want to use compat package, then indeed you need to use exact dependency version. Better version matching for compat packages has been on my todo [2] for long time, but as you probably noticed, compat packages are not top priority for me :)
Another quick question: attempting to use the same copr project for both
a
released version of fedora and rawhide doesn't make a lot of sense, in
the
case of java with fast moving and complex dependencies, right? I assume after having run into this that I would want one copr project per target
so
that I can let the packages diverge.
Assuming that the goal is to eventually contribute these packages to Fedora, I would advice to focus on rawhide only for development. Once rawhide package is done you can consider porting it to older Fedora (or EPEL) versions.
You don't need to install rawhide to target it. If used correctly, mock tool can be very useful in building rawhide packages on other systems.
[1]
https://lists.fedoraproject.org/pipermail/java-devel/2013-October/005000.htm... [2] https://github.com/mizdebsk/xmvn/blob/master/TODO#L50-L51
-- Mikolaj Izdebski Software Engineer, Red Hat IRC: mizdebsk -- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
On 06/09/2015 03:27 PM, Noa Resare wrote:
Thank you for all the useful feedback. In the latest few days, I have looked quite closely at the packages I prepared and concluded that the guava change to enable testlib and unit tests can be made much smaller if the caliper dependency is simply disabled.
The dependency is not used by the default maven build (caliper is a benchmarking framework and benchmarking, while useful, is not part of the code shipping or testing)
This means that we are down to a single extra dependency to be able to enable guava testing, the truth package.
That's nice.
I have submitted a review request and I'm looking for sponsors: https://bugzilla.redhat.com/show_bug.cgi?id=1229704
I've taken the review.
Since the truth package landed in rawhide a few days ago now (and I have been busy moving cross Sweden) I thought it might be time to revisit the guava change. I have updated the https://github.com/nresare/fedora-guava/tree/testlib-tests branch with your feedback.
Somewhat troubling, trying to create a koji scratch build ( http://koji.fedoraproject.org/koji/taskinfo?taskID=10133700) was really slow and now seem to have stalled indefinitely since a thread died in TreeMultisetTest with OutOfMemoryError. I guess the build server farm is somewhat overloaded at the moment, but it is unfortunate if unit tests die when low on memory.
I hope someone else has some insight into this as the whole build infrastructure is pretty opaque to me at the moment.
n
On Tue, Jun 9, 2015 at 3:40 PM, Mikolaj Izdebski mizdebsk@redhat.com wrote:
On 06/09/2015 03:27 PM, Noa Resare wrote:
Thank you for all the useful feedback. In the latest few days, I have looked quite closely at the packages I prepared and concluded that the guava change to enable testlib and unit tests can be made much smaller if the caliper dependency is simply disabled.
The dependency is not used by the default maven build (caliper is a benchmarking framework and benchmarking, while useful, is not part of the code shipping or testing)
This means that we are down to a single extra dependency to be able to enable guava testing, the truth package.
That's nice.
I have submitted a review request and I'm looking for sponsors: https://bugzilla.redhat.com/show_bug.cgi?id=1229704
I've taken the review.
-- Mikolaj Izdebski Software Engineer, Red Hat IRC: mizdebsk -- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
Il 18/06/2015 17:29, Noa Resare ha scritto:
Since the truth package landed in rawhide a few days ago now (and I have been busy moving cross Sweden) I thought it might be time to revisit the guava change. I have updated the https://github.com/nresare/fedora-guava/tree/testlib-tests branch with your feedback.
Somewhat troubling, trying to create a koji scratch build (http://koji.fedoraproject.org/koji/taskinfo?taskID=10133700) was really slow and now seem to have stalled indefinitely since a thread died in TreeMultisetTest with OutOfMemoryError. I guess the build server farm is somewhat overloaded at the moment, but it is unfortunate if unit tests die when low on memory.
hi can you try with other arch builder (maybe is only a problem with ARM ... builder) ? koji build --scratch --arch-override [preferred arch x86_64 i386] rawhide [srpm] regards - gil
I hope someone else has some insight into this as the whole build infrastructure is pretty opaque to me at the moment.
n
On Tue, Jun 9, 2015 at 3:40 PM, Mikolaj Izdebski <mizdebsk@redhat.com mailto:mizdebsk@redhat.com> wrote:
On 06/09/2015 03:27 PM, Noa Resare wrote: > Thank you for all the useful feedback. In the latest few days, I have > looked quite closely at the packages I prepared and concluded that the > guava change to enable testlib and unit tests can be made much smaller if > the caliper dependency is simply disabled. > > The dependency is not used by the default maven build (caliper is a > benchmarking framework and benchmarking, while useful, is not part of the > code shipping or testing) > > This means that we are down to a single extra dependency to be able to > enable guava testing, the truth package. That's nice. > I have submitted a review request and I'm looking for sponsors: > https://bugzilla.redhat.com/show_bug.cgi?id=1229704 I've taken the review. -- Mikolaj Izdebski Software Engineer, Red Hat IRC: mizdebsk -- java-devel mailing list java-devel@lists.fedoraproject.org <mailto:java-devel@lists.fedoraproject.org> https://admin.fedoraproject.org/mailman/listinfo/java-devel-- Spotify Free Software ombudsman. I/O Tribe.
-- java-devel mailing list java-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/java-devel
On 06/18/2015 05:29 PM, Noa Resare wrote:
Since the truth package landed in rawhide a few days ago now (and I have been busy moving cross Sweden) I thought it might be time to revisit the guava change. I have updated the https://github.com/nresare/fedora-guava/tree/testlib-tests branch with your feedback.
I've merged your commits. Thank you for contribution.
Somewhat troubling, trying to create a koji scratch build ( http://koji.fedoraproject.org/koji/taskinfo?taskID=10133700) was really slow and now seem to have stalled indefinitely since a thread died in TreeMultisetTest with OutOfMemoryError. I guess the build server farm is somewhat overloaded at the moment, but it is unfortunate if unit tests die when low on memory.
You've most likely hit this because your build was picked up by armv7hl builder. These builders are pretty slow compared to x86_64 ones. Moreover, we don't have JIT enabled on this architecture, so everything runs in interpreted mode. Package builds can be one to three orders of magnitude slower on armv7hl compared to x86_64 and often fail due to timeouts or OOM.
For scratch builds you can workaround this by using "--arch-override x86_64" option. For "real" builds the only known workaround is resubmitting the build until you get non-ARM builder.
On 06/04/2015 10:19 AM, Noa Resare wrote:
First off I would like to say that I'm very impressed by the effort you have put into java packaging over the last few years. Getting a usable environment for offline maven execution is no small feat.
Thank you.
One thing I noted, however, was that the unit tests for the guava library was not run on package build. Which seems like a reasonable compromise but not ideal. I decided to look into what would be needed dependency wise for that to happen, and I came up with the following:
- updates-testing needs to be activated for the auto-value dependency.
Buildroot overrides are useful in such cases. See: https://fedoraproject.org/wiki/Bodhi/BuildRootOverrides
- packages truth, allocation-instrumenter and caliper needs to be
introduced. I have created those packages and built them with guidance from the Java Packaging HOWTO https://fedorahosted.org/released/javapackages/doc/, the packages (SRPMS and mock built noarch.rpms) are available from https://resare.com/noa-fedora-playground/repo/22 but please beware of newbie packaging mistakes :)
Great. We will review them once you submit review requests.
- With the dependencies handled I updated guava.spec to re-enable
guava-testlib and guava-tests. One test was consistently failing due to new java8 behaviour, and I back ported a fix from guava master. The changes needed can be viewed in my fork of git://pkgs.fedoraproject.org/guava.git at https://github.com/nresare/fedora-guava in the testlib-tests branch. The actual commit can be viewed here: https://github.com/nresare/fedora-guava/commit/c1a8a831b1ef0ec91509fea806625...
Overally it looks good. I've added a few comments at Github. I'll be happy to merge this into Fedora once all dependencies are available in rawhide.
I would love for my work to be of use in the fedora project and I'm willing to spend some time to land my contributions at this time. I will now read up on what I need to to contribute more formally.
It's really good to see a new contributor. Don't hesitate to ask if you have any questions or need help.
java-devel@lists.fedoraproject.org
-
gil -
Mat Booth -
Mikolaj Izdebski -
Noa Resare