bigdata May 2014

bigdata@lists.fedoraproject.org

9 participants
16 discussions

Re: [curator] Update for log4j12
by Tim St. Clair 28 May '14

28 May '14

I'm digging. I see an error that only exists in rawhide, and I can't yet explain. ERROR - package junit.framework does not exist Yet: jar -tf /usr/share/java/junit.jar | grep -i assert junit/framework/Assert.class Cheers, Tim ----- Original Message ----- > From: "tstclair" <tstclair(a)fedoraproject.org> > To: curator-owner(a)fedoraproject.org, scm-commits(a)lists.fedoraproject.org > Sent: Wednesday, May 28, 2014 8:58:59 PM > Subject: [curator] Update for log4j12 > > commit 54698725a96240030ca4c221092be0b88c578928 > Author: Timothy St. Clair <tstclair(a)redhat.com> > Date: Wed May 28 20:58:55 2014 -0500 > > Update for log4j12 > > curator.spec | 6 +++++- > 1 files changed, 5 insertions(+), 1 deletions(-) > --- > diff --git a/curator.spec b/curator.spec > index 177221d..1e67443 100644 > --- a/curator.spec > +++ b/curator.spec > @@ -2,7 +2,7 @@ > %global namedversion %{version}%{?namedreltag} > Name: curator > Version: 2.2.0 > -Release: 1%{?dist} > +Release: 2%{?dist} > Summary: A set of Java libraries that make using Apache ZooKeeper much > easier > License: ASL 2.0 > URL: http://curator.incubator.apache.org/ > @@ -91,5 +91,9 @@ sed -i 's|1.2.16|1.2.17|' pom.xml > %doc LICENSE NOTICE > > %changelog > +* Wed May 28 2014 Timothy St. Clair <tstclair(a)redhat.com> 2.2.0-2 > +- Rebuild for log4j12 > + > * Thu Oct 10 2013 gil cattaneo <puntogil(a)libero.it> 2.2.0-1 > - initial rpm > +- initial rpm > -- Cheers, Tim Freedom, Features, Friends, First -> Fedora https://fedoraproject.org/wiki/SIGs/bigdata

1 0

log4j12 (1.2.17) is available for review
by gil 22 May '14

22 May '14

https://bugzilla.redhat.com/show_bug.cgi?id=1100323 hi - gil

1 0

Re: [fedora-java] Hadoop + log4j2 = fullstop
by Robert Rati 22 May '14

22 May '14

So, who needed log4j2? It is massively incompatible with log4j1.2 and isn't a simple port job. I would argue if log4j2 was actually needed, it should have been introduced as a separate log4j2 package and allow projects to port to it as they have time/need. This update log4j to an incompatible version with no compat package provided at the same time is not the way to handle such an upgrade. Giving advanced notice that the world will come crumbling down and you'll have to deal with it is not enough. Rob On 05/21/2014 02:01 PM, Aleksandar Kurtakov wrote: > > > Alexander Kurtakov > Red Hat Eclipse team > > ----- Original Message ----- >> From: "Robert Rati" <rrati(a)redhat.com> >> To: "Fedora Big Data SIG" <bigdata(a)lists.fedoraproject.org>, "Development discussions related to Fedora" >> <devel(a)lists.fedoraproject.org>, "Fedora Java Development List" <java-devel(a)lists.fedoraproject.org> >> Sent: Wednesday, May 21, 2014 7:03:42 PM >> Subject: Hadoop + log4j2 = fullstop >> >> I've been working on updating the hadoop package to the latest 2.4.0 >> release and at this point I've resolved all the issues but I'm now >> blocked by the log4j2 update. log4j2 breaks the hadoop build pretty >> severely, and it doesn't seem the log4j2 team has spent much time >> thinking about how to provide backwards compatibility to existing >> log4j1.2 users. From my investigation of log4j2: >> >> log4j.properties file is no longer read at all >> configuration file is now in XML or JSON >> configuration file name is log4j2.[xml|json|jsn] >> V2 isn't backwards compatible with V1. There's a shim for v1 api but it >> will only work for a limited set of cases, and for some cases it does >> work for it turns some operations into noop calls. >> >> This is a pretty major change and even the compatibility layer, if it >> will work for a project, does not seem to guarantee like functionality >> and minimally will require a re-do of all log4j configuration files a >> project ships. I'm not sure many sizable upstream projects would >> undertake/accept such a drastic change very quickly. >> >> The list of projects currently blocked by this update are: >> hadoop >> hbase >> oozie >> hive >> apache-log4j-extras >> amplab-tachyon >> >> I would be surprised if there aren't a lot more. I understand Fedora is >> always pushing for the latest versions, but for some fundamental >> packages can there be compatibility packages introduced at the same time >> as an incompatible update? > > Nothing stops compatibility packages from appearing. But it's the people that need it that have to drive it. Everyone is time constrained so whoever needs something must do it. > It's really is as simple as that. > > Alexander Kurtakov > Red Hat Eclipse team > >> Package maintainers of dependent packages >> will still need to touch their packages and determine if the new version >> will work for them. Providing a compat package will also allow packages >> to update to their newer versions while not held up on trying to >> integrate changes from a compatibility breaking dependency update. >> >> Rob >> -- >> devel mailing list >> devel(a)lists.fedoraproject.org >> https://admin.fedoraproject.org/mailman/listinfo/devel >> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct > -- > java-devel mailing list > java-devel(a)lists.fedoraproject.org > https://admin.fedoraproject.org/mailman/listinfo/java-devel >

3 2

Hadoop + log4j2 = fullstop
by Robert Rati 21 May '14

21 May '14

I've been working on updating the hadoop package to the latest 2.4.0 release and at this point I've resolved all the issues but I'm now blocked by the log4j2 update. log4j2 breaks the hadoop build pretty severely, and it doesn't seem the log4j2 team has spent much time thinking about how to provide backwards compatibility to existing log4j1.2 users. From my investigation of log4j2: log4j.properties file is no longer read at all configuration file is now in XML or JSON configuration file name is log4j2.[xml|json|jsn] V2 isn't backwards compatible with V1. There's a shim for v1 api but it will only work for a limited set of cases, and for some cases it does work for it turns some operations into noop calls. This is a pretty major change and even the compatibility layer, if it will work for a project, does not seem to guarantee like functionality and minimally will require a re-do of all log4j configuration files a project ships. I'm not sure many sizable upstream projects would undertake/accept such a drastic change very quickly. The list of projects currently blocked by this update are: hadoop hbase oozie hive apache-log4j-extras amplab-tachyon I would be surprised if there aren't a lot more. I understand Fedora is always pushing for the latest versions, but for some fundamental packages can there be compatibility packages introduced at the same time as an incompatible update? Package maintainers of dependent packages will still need to touch their packages and determine if the new version will work for them. Providing a compat package will also allow packages to update to their newer versions while not held up on trying to integrate changes from a compatibility breaking dependency update. Rob

3 2

FTBFS: hadoop
by NoReply 09 May '14

09 May '14

The following package(s) failed to build from source: Package hadoop (6829528): x86_64 (6829550): root.log : http://kojipkgs.fedoraproject.org/work/tasks/9550/6829550/root.log build.log : http://kojipkgs.fedoraproject.org/work/tasks/9550/6829550/build.log state.log : http://kojipkgs.fedoraproject.org/work/tasks/9550/6829550/state.log mock_output.log : http://kojipkgs.fedoraproject.org/work/tasks/9550/6829550/mock_output.log -- This is an automated message. To submit feedback or report any problems, write to <java-maint at redhat>.

2 2

FTBFS: hadoop
by NoReply 07 May '14

07 May '14

The following package(s) failed to build from source: Package hadoop (6824371): x86_64 (6824398): root.log : http://kojipkgs.fedoraproject.org/work/tasks/4398/6824398/root.log build.log : http://kojipkgs.fedoraproject.org/work/tasks/4398/6824398/build.log state.log : http://kojipkgs.fedoraproject.org/work/tasks/4398/6824398/state.log mock_output.log : http://kojipkgs.fedoraproject.org/work/tasks/4398/6824398/mock_output.log -- This is an automated message. To submit feedback or report any problems, write to <java-maint at redhat>.

1 0

FTBFS: hadoop
by NoReply 06 May '14

06 May '14

The following package(s) failed to build from source: Package hadoop (6820648): i686 (6820661): root.log : http://kojipkgs.fedoraproject.org/work/tasks/661/6820661/root.log build.log : http://kojipkgs.fedoraproject.org/work/tasks/661/6820661/build.log state.log : http://kojipkgs.fedoraproject.org/work/tasks/661/6820661/state.log mock_output.log : http://kojipkgs.fedoraproject.org/work/tasks/661/6820661/mock_output.log -- This is an automated message. To submit feedback or report any problems, write to <java-maint at redhat>.

1 0

Ambari in F20 and rawhide
by Peter MacKinnon 06 May '14

06 May '14

Greetings, http://ambari.apache.org/ Apache Ambari 1.5.1 (latest release) is available in F20 and rawhide. It is a Hadoop cluster management project with a centralized server, web console and agent framework. The project originated from Hortonworks and to-date features stack definitions and bits from their repos, the latest being HDP 2.1. Supported agent OS include the RHEL/CentOS family (6 and 5). These agents can be installed manually and self-register, or be provisioned to an "empty" node by the server via scp and bootstrap Python scripts. Interoperability between the Fedora Ambari server and Hortonworks Ambari agents from RHEL6 and CentOS6 appear to work correctly for the manual registration use case. The server provisioning use case needs some more work and likely patches upstream due to some lingering inflexibility with their software requirements. Currently, they enforce a strict version alignment of the agents to server sprinkled in various pockets of their configuration and Python code base. The Fedora agent is packaged but has unresolved interop issues when deploying even a basic stack of HDFS and Zookeeper. Beyond these agent issues, there will be other technical considerations which may lead me to attempt to define and implement a Fedora stack obviously based on our expanding set of Big Data packaged components. Stay tuned but in the meantime feel free to tinker if you like (and raise BZ) as I iterate on the 1.5.x package some more. http://koji.fedoraproject.org/koji/packageinfo?packageID=18392 \Pete -- Peter MacKinnon CTO Office: Big Data Red Hat Inc. Raleigh, NC

1 0

FTBFS: hadoop
by NoReply 05 May '14

05 May '14

The following package(s) failed to build from source: Package hadoop (6816651): i686 (6816673): root.log : http://kojipkgs.fedoraproject.org/work/tasks/6673/6816673/root.log build.log : http://kojipkgs.fedoraproject.org/work/tasks/6673/6816673/build.log state.log : http://kojipkgs.fedoraproject.org/work/tasks/6673/6816673/state.log mock_output.log : http://kojipkgs.fedoraproject.org/work/tasks/6673/6816673/mock_output.log -- This is an automated message. To submit feedback or report any problems, write to <java-maint at redhat>.

1 0

Release Notes
by Zach Oglesby 05 May '14

05 May '14

I am working on the databases and big data beats for the release notes and trying to figure out the best way to orginize them. I would like the input of the SIG to figure out if projects like Hbase, and Accumulo should be in the database or big data section of the release notes. They could go in either, but what is the prefrence of the SIG, and where do you think that people would look for them. Zach Oglesby

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

bigdata May 2014