bigdata November 2013

bigdata@lists.fedoraproject.org

7 participants
17 discussions

flume spec file (work in progress)
by gil 24 Nov '13

24 Nov '13

hi i started flume spec file there are again a lot of missing build deps .again... (also in my system F19: hadoop, avro*:1.7.3, ...) regards gil Name: flume Version: 1.4.0 Release: 1%{?dist} Summary: Log collection application License: ASL 2.0 URL: http://flume.apache.org/ Source0: http://www.apache.org/dist/flume/stable/apache-%{name}-%{version}-src.tar.gz BuildRequires: java-devel BuildRequires: mvn(com.google.guava:guava) BuildRequires: mvn(com.google.code.gson:gson) BuildRequires: mvn(com.google.protobuf:protobuf-java) BuildRequires: mvn(commons-cli:commons-cli) BuildRequires: mvn(commons-codec:commons-codec) BuildRequires: mvn(commons-collections:commons-collections) BuildRequires: mvn(commons-dbcp:commons-dbcp) BuildRequires: mvn(commons-io:commons-io) BuildRequires: mvn(commons-lang:commons-lang) BuildRequires: mvn(io.netty:netty) BuildRequires: mvn(joda-time:joda-time) BuildRequires: mvn(log4j:log4j) BuildRequires: mvn(org.apache.activemq:activemq-core) BuildRequires: mvn(org.apache.avro:avro:1.7.3) BuildRequires: mvn(org.apache.avro:avro-ipc:1.7.3) BuildRequires: mvn(org.apache.derby:derby) BuildRequires: mvn(org.apache.hadoop:hadoop-auth) BuildRequires: mvn(org.apache.hadoop:hadoop-common) BuildRequires: mvn(org.apache.hadoop:hadoop-hdfs) BuildRequires: mvn(org.apache.mina:mina-core) # https://bugzilla.redhat.com/show_bug.cgi?id=1025904 BuildRequires: mvn(org.apache.solr:solr-core) # libthrift Hadoop 2 profile BuildRequires: mvn(org.apache.thrift:libthrift:0.8.0) BuildRequires: mvn(org.apache.tika:tika-xmp) BuildRequires: mvn(org.codehaus.jackson:jackson-core-asl) BuildRequires: mvn(org.codehaus.jackson:jackson-mapper-asl) # https://bugzilla.redhat.com/show_bug.cgi?id=976049 BuildRequires: mvn(org.schwering:irclib) BuildRequires: mvn(org.slf4j:jcl-over-slf4j) BuildRequires: mvn(org.slf4j:slf4j-api) BuildRequires: mvn(org.slf4j:slf4j-log4j12) %if 0 # Not availables build deps # https://github.com/cloudera/cdk/ BuildRequires: mvn(com.cloudera.cdk:cdk-morphlines-avro:0.4.0) BuildRequires: mvn(com.cloudera.cdk:cdk-morphlines-solr-cell:0.4.0) BuildRequires: mvn(com.cloudera.cdk:cdk-morphlines-tika-core:0.4.0) BuildRequires: mvn(com.cloudera.cdk:cdk-morphlines-tika-decompress:0.4.0) BuildRequires: mvn(com.cloudera.cdk:cdk-morphlines-twitter:0.4.0) # http://gil.fedorapeople.org/apache-log4j-extras-1.2.17-1.fc19.src.rpm BuildRequires: mvn(log4j:apache-log4j-extras:1.1) BuildRequires: mvn(org.apache.hbase:hbase:0.94.2) BuildRequires: mvn(org.easytesting:fest-reflect:1.4) # http://www.elasticsearch.org/ http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current… https://github.com/elasticsearch/elasticsearch/ BuildRequires: mvn(org.elasticsearch:elasticsearch:0.90.1) # https://github.com/OpenTSDB/asynchbase/ depend on com.stumbleupon:async:1.3.1 https://github.com/stumbleupon/async/ BuildRequires: mvn(org.hbase:asynchbase:1.4.1) BuildRequires: mvn(org.mortbay.jetty:jetty:6.1.26) BuildRequires: mvn(org.mortbay.jetty:jetty-util:6.1.26) BuildRequires: mvn(org.mortbay.jetty:servlet-api:2.5-20110124) # Test deps BuildRequires: mvn(junit:junit) BuildRequires: mvn(org.apache.hadoop:hadoop-minicluster) BuildRequires: mvn(org.apache.httpcomponents:httpclient) BuildRequires: mvn(org.apache.solr:solr-test-framework) BuildRequires: mvn(org.mockito:mockito-all) # Not availables test deps BuildRequires: mvn(org.apache.solr:solr-test-framework) BuildRequires: mvn(org.easytesting:fest-reflect) %endif BuildRequires: maven-local BuildRequires: mvn(org.apache.maven.plugins:maven-antrun-plugin) BuildRequires: mvn(org.apache.maven.plugins:maven-remote-resources-plugin) BuildRequires: mvn(org.apache.maven.plugins:maven-site-plugin) BuildRequires: mvn(com.thoughtworks.paranamer:paranamer-maven-plugin) BuildRequires: mvn(org.apache.avro:avro-maven-plugin:1.7.3) BuildRequires: mvn(org.codehaus.mojo:build-helper-maven-plugin) BuildRequires: thrift BuildArch: noarch %description Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application. %package javadoc Summary: Javadoc for %{name} %description javadoc This package contains javadoc for %{name}. %prep %setup -q -n apache-%{name}-%{version}-src %pom_remove_plugin :apache-rat-plugin %build # Not available test deps: solr-test-framework, fest-reflect %mvn_build -f

1 0

Re: Introduction
by Tim St. Clair 22 Nov '13

22 Nov '13

Feel free to ping me on freenode #fedora-devel, or #fedora-bigdata. We've been doing a fair amount of packaging lately, and could offer up tips where needed. I often find the best starting point is here: https://fedoraproject.org/wiki/Join_the_package_collection_maintainers You will want to also read: https://fedoraproject.org/wiki/Packaging:Java Cheers, Tim ----- Original Message ----- > From: "Tako Schotanus" <tschotan(a)redhat.com> > To: devel(a)lists.fedoraproject.org > Sent: Friday, November 22, 2013 6:58:03 AM > Subject: Introduction > > > Hi, > > I read that somebody wanting to be a package maintainer should introduce > themselves, well here goes. > > I'm Tako, a Software Engineer for Red Hat working on a new language, Ceylon > (http://ceylon-lang.org), that runs on top of the Java and Javascript VMs. > We just published out first 1.0.0 and although we already had RPMs (and > ZIPs and DEBs) that could be downloaded from our website for a while now > we'd like to add the package to the official Fedora repositories now that > we're at 1.0.0 (also because Ubuntu is working on getting the package into > their repos and as a RedHat sponsored project it would be weird if we didn't > do the same for Fedora, right?) > > So I'm completely new to this packaging business, I managed to piece together > a SPEC file that results in an actually working RPM for our project and even > Koji seems to accept it, but there's so much information to absorb that I'm > feeling a bit out of my depth. (Our project being a programming language > we're dealing with some difficult issues with respect to versioning and > such, for now I've copied Java's with alternatives and such which might or > not be a good idea). So if there are some friendly people here that can > guide me through my first real submission that would be great! > > Cheers, > > Tako Schotanus > Senior Software Developer > Ceylon Language Project > www.ceylon-lang.org > RedHat > > > -- > devel mailing list > devel(a)lists.fedoraproject.org > https://admin.fedoraproject.org/mailman/listinfo/devel > Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct -- Cheers, Tim

1 0

BIG DATA SIG irc meeting
by Tim St. Clair 21 Nov '13

21 Nov '13

The following is a new meeting request: Subject: BIG DATA SIG irc meeting Organizer: "Tim St Clair" <tstclair(a)redhat.com> Location: freenode #fedora-meeting-2 Time: Thursday, November 21, 2013, 9:00:00 AM - 10:00:00 AM GMT -06:00 US/Canada Central Invitees: bigdata(a)lists.fedoraproject.org *~*~*~*~*~*~*~*~*~* Folks - Let's follow up from our last conversations and get an update on the current state of the Big Data SIG. ---------------------------------------------------- freenode #fedora-meeting-2 Big Data SIG meeting details below You are kindly invited to join to the following meeting: Meeting: Big Data SIG Starting time: 14:00:00 UTC Ending time: 15:00:00 UTC ---------------------------------------------------- Agenda: - Current development(s) - Current issue(s) - Any new package(s) to add to the list - Any "new" takers for the current list (https://fedoraproject.org/wiki/SIGs/bigdata/packaging) ... Feel free to add to the agenda.

1 1

Reviews for sale
by Peter MacKinnon 20 Nov '13

20 Nov '13

I've got a family of Hive deps that are available for review. Brag to your friends that you've done Fedora package reviews! Girls will swoon and boys will be jealous! The time-api dep is now in for OJ7 and datanucleus-core: http://koji.fedoraproject.org/koji/packageinfo?packageID=17363 1011962 <https://bugzilla.redhat.com/show_bug.cgi?id=1011962> nobody(a)fedoraproject.org Package Review --- NEW Review Request: datanucleus-api-jdo - plugin for JDO support <https://bugzilla.redhat.com/show_bug.cgi?id=1011962> unspecified unspecified 1011705 <https://bugzilla.redhat.com/show_bug.cgi?id=1011705> nobody(a)fedoraproject.org Package Review --- NEW Review Request: datanucleus-core - Java persistence abstraction layer <https://bugzilla.redhat.com/show_bug.cgi?id=1011705> unspecified unspecified 1011960 <https://bugzilla.redhat.com/show_bug.cgi?id=1011960> nobody(a)fedoraproject.org Package Review --- NEW Review Request: datanucleus-rdbms - plugin for RDBMS storage <https://bugzilla.redhat.com/show_bug.cgi?id=1011960> unspecified unspecified \Pete -- Peter MacKinnon MRG Grid/Big Data Red Hat Inc. Raleigh, NC

1 0

BIG DATA SIG irc meeting
by Tim St. Clair 19 Nov '13

19 Nov '13

The following meeting has been modified: Subject: BIG DATA SIG irc meeting Organizer: "Tim St Clair" <tstclair(a)redhat.com> Location: freenode #fedora-meeting-2 Time: Thursday, November 21, 2013, 9:00:00 AM - 10:00:00 AM GMT -06:00 US/Canada Central Invitees: bigdata(a)lists.fedoraproject.org *~*~*~*~*~*~*~*~*~* Folks - Let's follow up from our last conversations and get an update on the current state of the Big Data SIG. ---------------------------------------------------- freenode #fedora-meeting-2 Big Data SIG meeting details below You are kindly invited to join to the following meeting: Meeting: Big Data SIG Starting time: 15:00:00 UTC Ending time: 16:00:00 UTC ---------------------------------------------------- Agenda: - Current development(s) - Current issue(s) - Any new package(s) to add to the list - Any "new" takers for the current list (https://fedoraproject.org/wiki/SIGs/bigdata/packaging) ... Feel free to add to the agenda.

1 0

Re: Installing Apache Hadoop 2.2 on fedora-19
by Matthew Farrellee 18 Nov '13

18 Nov '13

On 11/10/2013 08:22 PM, Husam Suleiman wrote: > Dear Matthew, > > I am a student interested in working on Hadoop-2.2 as a case study for my > PhD thesis and would appreciate your invaluable feedback! > > Currently, I am just a beginner and trying to install Hadoop-2.2 on > fedora-19. I've already downloaded the Hadoop-2.2 files from this website: > http://www.apache.org/dyn/closer.cgi/hadoop/common/ and especially all the > files from this one: > http://mirrors.sonic.net/apache/hadoop/common/hadoop-2.2.0/ > > Do you please have a clear and easy tutorial (step-by-step) to make the > installation of Hadoop-2.2 on fedora-19? > > Your quick feedback is really appreciated and many thanks in advance. > > Best Regards, > Husam Husam, You can follow https://fedoraproject.org/wiki/Changes/Hadoop#YUM_repositories However, I strongly recommend you spin up an F20 VM or just upgrade to F20 and use Hadoop from the standard yum repositories. Best, matt

2 1

Re: Interested in contributing to Hadoop - a Beginner
by Matthew Farrellee 18 Nov '13

18 Nov '13

On 10/15/2013 02:33 PM, IAmNot ThatGuy wrote: > Hi Matt, > > I am Mohan from India. I am interested in contributing to Fedora Hadoop. > I am getting trained as a developer in Hadoop in the company I work for. > > I recently switched from Ubuntu to Fedora and I am a beginner in terms > of contribution through code. > > I am interested to learn packaging and bug fixing. If you can guide me > in pointing out things I have to learn and mentor me, it will be > helpful for me to learn and do my part for the community. > > Awaiting your response. > > > Thanks, > -- > Mohi > > My IRC nick: IAmNotThatGuy Mohi, I hope you're still interested in Hadoop and Fedora! You should join us on #fedora-bigdata and the Big Data SIG mailing list. There are a number of people around that can help you get started small tasks as you get trained up. Best, matt

1 0

F21 System Wide Change: Headless Java
by Peter MacKinnon 18 Nov '13

18 Nov '13

FYI https://lists.fedoraproject.org/pipermail/devel-announce/2013-November/0012… -- Peter MacKinnon MRG Grid/Big Data Red Hat Inc. Raleigh, NC

1 0

review swap: amplab-tachyon
by Tim St. Clair 15 Nov '13

15 Nov '13

Folks - I've put amplab-tachyon up for review (https://bugzilla.redhat.com/show_bug.cgi?id=1029142) and would be happy to review in exchange. -- Cheers, Tim

1 0

Packaging:No Bundled Libraries - FedoraProject
by Peter MacKinnon 13 Nov '13

13 Nov '13

Recent policy update that provides grounds for exception, such as with the javax time-api. https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries#Reverse_Bundl… \Pete -- Peter MacKinnon MRG Grid/Big Data Red Hat Inc. Raleigh, NC

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

bigdata November 2013