r-devel September 2009

r-devel@lists.fedoraproject.org

9 participants
10 discussions

R package dependencies, translated into RPM land.
by Allen S. Rout 30 Sep '09

30 Sep '09

OK; I've been chewing and debating on this with several folks, in several fora, for some time. Here are the facts as I've been able to determine them so far; at least some of them will be stupidly obvious to most of you. Please tear them apart as indicated. + The R community is much sysadmin-y and progammer-y than other language communities. Several positions about correctness which lots of admins take as Truth (i.e. dependency cycle == BAD) they find to be more of an aesthetic call. This is reasonable. + Different repositories in the R community have independant lives and attitudes. There is modest competition and grumbling between maintainers associated with different repos. + Package dependencies cross repo boundaries; sticking with the 'Better' repositories just won't work, and discussion of these variations tends to make R folks testy. conclusion: The goal of evolving the R packages into a DAG is a non-starter. + There are four classes of dependency in R-package land: Requires, Imports, Suggests, and Enhances. + Requires and Imports are required to load the package. [1] + Suggests may be required to fully CMD CHECK the package [1] + The need for suggests at CMD CHECK can be deactivated by build config file. [2] + Many of the dependency cycles can be avoided if we ignore Suggests as an RPM dependency. Now, on to opinion: + We would like all official packages to have passed a full R CMD CHECK + We would like an absolute minimum of manual special case handling. It may not be possible to make that amount zero. So: Here's my suggested procedure for building any single package, gangked from a message I sent to R-core: 1) Express binary package dependencies according to Depends and Imports. I'll call this the 'narrow dependency graph'. 2) As part of the binary package build process, run CHECK with R_CHECK_FORCE_SUGGESTS = false. I'll pull nomenclature out of my ear and call these "built" but not "checked". 3) Build all binary packages which are downstream according to all of Depends, Imports, Suggests, and Extends. I'll call this the 'broad dependency graph'. 4) Install all the packages in the broad dependency graph. 5) for each package in the broad graph, run CHECK with R_CHECK_FORCE_SUGGESTS=true. Then the affected packages are "checked". Perhaps this can be noted with a signature. .... Whew! - Allen S. Rout [1] http://cran.r-project.org/doc/manuals/R-exts.html#The-DESCRIPTION-file [2] http://cran.r-project.org/doc/manuals/R-exts.html#Customizing-checking-and-…

2 1

r2spec
by Allen S. Rout 30 Sep '09

30 Sep '09

So, I've modified r2spec fairly heavily on my own. Pierre-Yves suggested I bring conversation straight here, so I'm doing that. I'll put a SRPM of my r2spec up at the next point when it works. It worked earlier today, but of course I'm doing more surgery right now. Here are the most visible changes I've made so far; Please hear them as suggestions, not demands. :) + Takes R package metadata from repositories directly through available.packages. Can load from many repos at once. + Cleanly handles multi-line fields in package metadata + Recursively chases Depends + including across repositories (i.e. CRAN depends on BioC) + Or Depends and Suggests, and Imports and Extends. There's a lot more to this. Separate topic, separate post. + Uses a single template for specfile, instead of two long functions with literal strings. + Permits template to be specified on the command line + String manipulation by regexp instead of iterated 'replace' and such. + Maintains a dependency graph to permit building an automated rpmbuild order. [ toy additions ] outputs GraphViz (dot format) illustrations of dependency graphs. I know this is lots and lots of change. I hope you don't see it as encroaching. - Allen S. Rout

4 13

Any feedback? any at all?
by Allen S. Rout 24 Sep '09

24 Sep '09

Hi, folks. I've now got my script in such a form that I can generate, build, and check most of the 'broad' dependency tree for ggplot2. There are several package build errors which I haven't started running down; I'm not sure if they're relevant to my case. Do any of you have any interest in this? I was thinking it might help accellerate new R packages' introduction to e.g. EPEL, but my sense now is that you'd much rather hand-craft every one. If our goals are just skew, I'm fine with that. I've put nearly a work week into this over the last few months, and am very near the end of what I need for my local concerns. I can happily put a bit more time to address things you-all want done, if your perspective is something like "We could use this, if he'd just [x, y, z]". Getting stuff accepted 'upstream' is worthwhile in my work environment. But if you're never going to be interested in the mass maintenance of these packages, I'll just stop bothering you. - Allen S. Rout

4 6

Should rpy require R or R-core?
by José Abílio Matos 22 Sep '09

22 Sep '09

Hi all, I got a feature request in bugzilla where the user request that rpy requires R-core and not R. I don't have any good reason to choose one or the other, what do others think? Would anyone oppose to such a move? Regards, -- José Abílio

3 4

Reflections on R...
by Allen S. Rout 18 Sep '09

18 Sep '09

I've completed the "OK, now try to CMD CHECK all these" script, and seeing how well it functions. If you want to be impressed with what you can do with R, all out of proportion with what you thought was keen yesterday, run R CMD check on rgl. Dang. - Allen S. Rout

1 0

R2spec kibitzing.
by Allen S. Rout 18 Sep '09

18 Sep '09

Sorry, those messages were supposed to have come out periodically over the last week or two. I've got the next version ready, tested as I had alluded to in my previous. http://nersp.osg.ufl.edu/~asr/media/R2spec-2.6.1-asr.src.rpm So, here's what I did to get more or less useful results out of it: After installing, I made a temp directory, /var/tmp/Rstuff. There, I ran (as root): # clear; R2spec -n "Allen S. Rout" -e "asr(a)ufl.edu" -c --force --suggests --verbose --package ggplot2 This downloaded the CRAN-like lists of packages, downloaded source tarballs, and built (and copied) the specfiles and source. It also generated a few dot files, including the broad and narrow dependency graphs for the requested package. It also generates a shell file: 'rpmbuild.sh'. This is intended to make all the RPMs you need for the broad graph. I haven't decided what the sane options are in terms of 'pwd' throughout all this process. At the moment, the next step is: # cd /usr/src/redhat/SPECS # sh /var/tmp/Rstuff/rpmbuild.sh This will build and install all of the RPMs in the broad graph. ... If the individual builds work. This script uses a tempdir extensively, again I'm not sure what the Right Way is to do this, I haven't thought about it too much yet. It's statically defined at the beginning of the script to be /var/tmp/Rstuff that's obviously inappropriate for release, but will do for the moment. The script drops a build log and an install log for every package addressed. Right now, for me to get the sysdeps to build and install all (most) of ggplot2's broad depgraph, I must: yum install openmpi openmpi-devel openmpi-libs unixODBC-devel pvm tcl tcl-devel perl-DateManip perl-Date-Calc perl-version and then install from EPEL: environment-modules-3.2.6-4.el5.x86_64.rpm mpich2-1.1.1-1.el5.x86_64.rpm perl-IO-stringy-2.110-5.el5.noarch.rpm perl-Jcode-2.06-6.el5.noarch.rpm perl-OLE-Storage_Lite-0.14-9.el5.noarch.rpm perl-Parse-RecDescent-1.96-1.el5.noarch.rpm perl-Spreadsheet-WriteExcel-2.18-1.el5.noarch.rpm perl-Unicode-Map-0.112-12.el5.x86_64.rpm and then install from the graphviz people: graphviz-2.24.0-1.el5.x86_64.rpm graphviz-devel-2.24.0-1.el5.x86_64.rpm With these things done, I still fail to build (and haven't investigated much yet): tcltk2: Fails to install: need tclsh8.3, not 8.4 ? multcomp: requires Survival >= 2.35.7 ? ... flexmix: dep on multcomp R-RandomFields, R-MCMCpack: uncaught debug files in the build tree. MPI, sprng As threatened, my next step will be a similar shell script to R CMD CHECK all the packages, and populate yet another family of logfiles. Once that is complete, I will re-announce and go back to try to figure out why individual packages fail. - Allen S. Rout

2 1

GMANE postings held in limbo?
by Allen S. Rout 15 Sep '09

15 Sep '09

I got this: > Your mail to 'Fedora-r-devel-list' with the subject > Re: r2spec > Is being held until the list moderator can review it for approval. > The reason it is being held: > Message has implicit destination when I posted from GMANE. Would it be possible to set the mailing list to permit these messages? Here's another example of a similar config change. http://www.mail-archive.com/tortoisehg-discuss@lists.sourceforge.net/msg009… - Allen S. Rout

2 1

collaborating on ggplot2 ?
by Jack Tanner 07 Sep '09

07 Sep '09

5 15

Review request: R-plyr
by Tom Moertel 07 Sep '09

07 Sep '09

Fedora R hackers, I have submitted a review request for R-plyr: https://bugzilla.redhat.com/show_bug.cgi?id=521671 This package is a prerequisite of the much-desired ggplot2 package, so if anybody has a few moments to review it, I would be grateful. Thanks! Cheers, Tom

1 0

Introduction; was Re: collaborating on ggplot2 ?
by Allen S. Rout 03 Sep '09

03 Sep '09

Well, it didn't get through GMANE. So I'm re-sending directly. Pierre-Yves <pingou-E11Oz7VxvVOXCRStZZN3OA(a)public.gmane.org> writes: > Did you look at the subdependencies ? I mean that for example quantreg > requires SparseM tripack, akima, MASS, survival, rgl, logspline > (Which already points out a circular dependency on MASS)... I'm also working on this problem, albeit from a slightly different perspective. I started out wanting to just get ggplot2 installed. :) I found r2spec, and have been folding, spindling, and mutilating it for somewhere under a month. Pierre-Yves, somewhere in the ether between me and you are several messages along the lines of "I've got some suggestions, will you entertain them?" I've also been talking with other R developer types; I came up with a similar chunk of code to make "Ebuilds" (Gentoo artifact mostly like a specfile) a few years back. So, Hi, there! I've got several different topics queued up, I figure I'll wait to see if this gets through GMANE before I start on them. - Allen S. Rout

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

r-devel September 2009