bigdata August 2013

bigdata@lists.fedoraproject.org

9 participants
10 discussions

Packaging ambari for Fedora
by Robert Rati 03 Sep '13

03 Sep '13

I've created a wiki detailing the issues that need to be addressed in order to get ambari packaged in Fedora. Comments and discussions about these issues are welcome. https://fedoraproject.org/wiki/Ambari_packaging Rob

2 1

Packaging GlusterFS Hadoop
by Matthew Farrellee 28 Aug '13

28 Aug '13

I'm starting to package glusterfs-hadoop, a shim on top of GlusterFS that allows applications written for HDFS to run on GlusterFS. The SRPM and spec are - http://matt.fedorapeople.org/pkg/0/glusterfs-hadoop-2.1.2-1.src.rpm http://matt.fedorapeople.org/pkg/0/glusterfs-hadoop.spec I'm trying to get it to build in an F20 mock environment before submitting a review request. However, I'm running into a problem... [ERROR] Failed to execute goal on project glusterfs: Could not resolve dependencies for project org.apache.hadoop.fs.glusterfs:glusterfs:jar:2.0-SNAPSHOT: Could not find artifact com.sun:tools:jar:1.6 at specified path /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.25-2.3.12.3.fc20.x86_64/jre/../lib/tools.jar -> [Help 1] com.sun:tools is coming in from... [DEBUG] Loading depmap file: /usr/share/maven-fragments/javapackages-depmap.xml [DEBUG] Added mapping [com.sun:tools] => [JAVA_HOME:../lib/tools] ... [DEBUG] org.apache.hadoop:hadoop-annotations:jar:2.0.5-alpha:compile [DEBUG] com.sun:tools:jar:1.6:system The javapackages tools is 2.0.1-1.fc20 and xmvn is 0.5.1-4.fc20. Has anyone seen this before, or have suggestions on getting past it? Best, matt

4 12

Review Request: glusterfs-hadoop - GlusterFS Hadoop Compatible File System (HCFS) plugin
by Matthew Farrellee 28 Aug '13

28 Aug '13

https://bugzilla.redhat.com/show_bug.cgi?id=1002101 This is my first java library package, so I probably got a few things wrong. Also, I was not able to fedora-review ahead of time. Will someone do a review of it for me? Best, matt

1 0

Fwd: Broken dependencies: stout
by Tim St. Clair 27 Aug '13

27 Aug '13

Is there any reason to get this into the epel channels? I know some folks had expressed interest, but currently I have no interest in maintaining for those channels. -Tim ----- Forwarded Message ----- > From: buildsys(a)fedoraproject.org > To: stout-owner(a)fedoraproject.org > Sent: Saturday, August 17, 2013 11:32:29 PM > Subject: Broken dependencies: stout > > > > stout has broken dependencies in the epel-5 tree: > On ppc: > stout-devel-0.1.2-1.099483f.el5.ppc requires gmock-devel > stout-devel-0.1.2-1.099483f.el5.ppc requires glog-devel > On x86_64: > stout-devel-0.1.2-1.099483f.el5.x86_64 requires gmock-devel > stout-devel-0.1.2-1.099483f.el5.x86_64 requires glog-devel > On i386: > stout-devel-0.1.2-1.099483f.el5.i386 requires gmock-devel > stout-devel-0.1.2-1.099483f.el5.i386 requires glog-devel > Please resolve this as soon as possible. > > > -- Cheers, Tim

1 0

Fwd: [VOTE] Release Apache Hadoop 2.1.0-beta
by Tim St. Clair 27 Aug '13

27 Aug '13

FYI. It may make sense just to track and update once the release comes out. ----- Forwarded Message ----- > From: "Arun C Murthy" <acm(a)hortonworks.com> > To: common-dev(a)hadoop.apache.org, hdfs-dev(a)hadoop.apache.org, mapreduce-dev(a)hadoop.apache.org, > yarn-dev(a)hadoop.apache.org > Sent: Sunday, August 25, 2013 2:38:50 PM > Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta > > With 9 +1s including mine (6 binding) and no -1s, the vote passes. Thanks to > all who voted. > > Arun > > On Aug 15, 2013, at 2:15 PM, Arun C Murthy <acm(a)hortonworks.com> wrote: > > > Folks, > > > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > > like to get released - this fixes the bugs we saw since the last go-around > > (rc1). > > > > The RC is available at: > > http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/ > > The RC tag in svn is here: > > http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2 > > > > The maven artifacts are available via repository.apache.org. > > > > Please try the release and vote; the vote will run for the usual 7 days. > > > > thanks, > > Arun > > > > -- > > Arun C. Murthy > > Hortonworks Inc. > > http://hortonworks.com/ > > > > > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. > -- Cheers, Tim

1 0

Big Data packaging wiki
by Matthew Farrellee 27 Aug '13

27 Aug '13

Hey fellow big data fans, https://fedoraproject.org/wiki/Big_data_packaging I just updated the packaging wiki page to include some of the things that have been done, a few that are in progress, and a bit of a wishlist I've heard over the past year. Feel free to edit the page at will. Especially if some existing packages aren't listed, or if you are working on something that isn't listed. This entire ecosystem is quite unruly when it comes to massaging into Fedora-acceptable packages, and it turns out there is often overlap under the covers (e.g. thrift for spark and hive). So, if you're looking to start a new package, let's us know on the mailing list. Best, matt

1 0

Plans for Apache Mahout?
by Björn Esser 27 Aug '13

27 Aug '13

Hi folks! Are there currently any plans to get Apache Mahout [1] into Fedora? If there are, I'd be glad to join in. If not I'd volunteer to get it packaged. :) Cheers, Björn [1] http://mahout.apache.org/

3 4

Savanna - Hadoop on OpenStack
by Matthew Farrellee 26 Aug '13

26 Aug '13

Hello Big Data SIG folks, If you aren't familiar, Savanna is an OpenStack project that provides Hadoop cluster and workload management. Cluster - construct and manage the lifecycle of Hadoop clusters. Workload - workflow for big data processing with Hadoop (similar to AWS EMR). The project home page is https://launchpad.net/savanna Savanna is made up of 4 sub-projects - . savanna, the main services . savannadashboard, web UI integration with OpenStack Horizon . python-savannaclient, python bindings for the REST API . savanna-extra - diskimage-builder elements for...image building As of today all those are available in F19, F20, and EL6* openstack-savanna - https://bugzilla.redhat.com/show_bug.cgi?id=986615 python-django-savanna - https://bugzilla.redhat.com/show_bug.cgi?id=998123 python-savannaclient - https://bugzilla.redhat.com/show_bug.cgi?id=998701 savanna-image-elements - https://bugzilla.redhat.com/show_bug.cgi?id=998702 Thanks for all the community help getting Savanna into Fedora, especially the #rdo folks. With any luck the project will have Fedora based cloud images with its next release. Right now all the images are Ubuntu based. Best, matt * openstack-savanna (package for savanna sub-project) has a dep on pycrypto and isn't available on EL6 yet, savanna-image-elements depends on diskimage-builder which isn't included in EL6 yet - Install - # yum --enablerepo=updates-testing install openstack-savanna python-django-savanna (make sure you get python-django-savanna-0.2-2) Setup & start savanna-api service - # sed -i "s/^#os_admin_password=/os_admin_password=$OS_PASSWORD/" /etc/savanna/savanna.conf # systemctl start openstack-savanna-api Setup and load Dashboard plugin - # echo "SAVANNA_URL = 'http://localhost:8386/v1.0'" >> /etc/openstack-dashboard/local_settings Edit /usr/share/openstack-dashboard/openstack_dashboard/settings.py - . HORIZON_CONFIG = { 'dashboards': ('project', 'admin', 'settings', 'savanna'), . INSTALLED_APPS = ( 'savannadashboard', 'openstack_dashboard', # systemctl reload httpd

1 0

Hadoop is in Fedora
by Robert Rati 22 Aug '13

22 Aug '13

The hadoop package has passed review and has been built for Fedora 20. Rob

5 4

zookeeper - FindClass issue on 3.4.5, works fine in 3.3.6
by Tim St. Clair 14 Aug '13

14 Aug '13

Folks - I've created a very simple JNI test which accesses a utility class in the Jar, and it worked fine in 3.3.X, but fails on 3.4.5 - http://paste.fedoraproject.org/32073/76490572/ I can't tell if this is a bug or an expected behavior.?.? -- Cheers, Tim

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

bigdata August 2013