Are there any ongoing plans for being able to browse/link/search Fedora SRPM contents online?
One obvious use case is automatically generating links for FAF stack traces to omnigrok(etc) SRPM:file:lines. Having permalink URLS would also be useful for ongoing bugfix/feature discussions.
E.g. lines in this https://retrace.fedoraproject.org/faf/problems/1378013/ could link to the cgit,opengrok view of the file:line of its original SRPM (for a selected release) http://koji.fedoraproject.org/koji/buildinfo?buildID=742571
Also, in general having some reusable functionality to bidirectionally map between these... RPM binary/script line/debugsymbol <-> SRPM NVR file:line <-> pkgs.fedoraproject.org commit/srcfile-or-patch:line <-> upstream commit/file:line ... would be useful for many use cases considering the low complexity of the problem (E.g. examples above, tagging, upstream auto-mining of FAF, future https://fedoraproject.org/wiki/StaticAnalysis LXR style browser etc.)
Previous efforts/discussions in this area: This was promising but looks dead: https://fedoraproject.org/wiki/LubomirKundrak/OpenGrok Previous discussion: https://lists.fedoraproject.org/pipermail/users/2010-November/387149.html
I think you're out of luck. It would be a huge amount of resources for fedora to host the exploded contents of all 10k+ packages (let along keeping multiple versions or revision control history for those upstream projects)
This is a critical point, but maybe a workaround is to dynamicaly extract the existing SRPMs on request along with some simple caching.
(PS Just curious about this area, I don't think I'd be able to do much more than a hacky PoC:)
Thanks! - Joe
On Tue, 14 Jun 2016 14:59:16 -0000 "Joseph Mullally" jwmullally@fedoraproject.org wrote:
Are there any ongoing plans for being able to browse/link/search Fedora SRPM contents online?
nope. Not that I am aware of.
One obvious use case is automatically generating links for FAF stack traces to omnigrok(etc) SRPM:file:lines. Having permalink URLS would also be useful for ongoing bugfix/feature discussions.
E.g. lines in this https://retrace.fedoraproject.org/faf/problems/1378013/ could link to the cgit,opengrok view of the file:line of its original SRPM (for a selected release) http://koji.fedoraproject.org/koji/buildinfo?buildID=742571
Also, in general having some reusable functionality to bidirectionally map between these... RPM binary/script line/debugsymbol <-> SRPM NVR file:line <-> pkgs.fedoraproject.org commit/srcfile-or-patch:line <-> upstream commit/file:line ... would be useful for many use cases considering the low complexity of the problem (E.g. examples above, tagging, upstream auto-mining of FAF, future https://fedoraproject.org/wiki/StaticAnalysis LXR style browser etc.)
Previous efforts/discussions in this area: This was promising but looks dead: https://fedoraproject.org/wiki/LubomirKundrak/OpenGrok Previous discussion: https://lists.fedoraproject.org/pipermail/users/2010-November/387149.html
I think you're out of luck. It would be a huge amount of resources for fedora to host the exploded contents of all 10k+ packages (let along keeping multiple versions or revision control history for those upstream projects)
This is a critical point, but maybe a workaround is to dynamicaly extract the existing SRPMs on request along with some simple caching.
(PS Just curious about this area, I don't think I'd be able to do much more than a hacky PoC:)
Right, there's potentially a ton of resources needed for something like this.
we do have ability to view spec and patches in packages app: https://apps.fedoraproject.org/packages/limnoria/sources/ but that doesn't help with src.rpms.
There was someone we were talking with a while back that ran the debian source indexer and wanted to know if we were interested in such a thing in Fedora. We are/were, but again resources were not there. https://codesearch.debian.net/about
kevin
Hi Kevin, thanks for your reply.
I am guessing one factor against easy linking to source is how its archived in the source control Lookaside Cache.
Just for kicks I threw together a small proof of concept RPM content browser web service. It pulls the packages on-demand, and works surprisingly well. The debuginfo RPM contents seem like ideal targets to surface for stack traces, as they have a nice 1:1 relationship with binaries.
https://github.com/jwmullally/test_rpmbrowser/blob/master/README.md http://rpmbrowser-jwmullally.rhcloud.com/
https://retrace.fedoraproject.org/faf/problems/1354441/ http://rpmbrowser-jwmullally.rhcloud.com/rpm/nautilus-debuginfo-3.18.1-1.fc2...
I'll probably take that site down in a few weeks. SRPM extraction doesn't work due to "rpm" not being executable on the old OpenShift containers. It gets the RPMs from http://kojipkgs.fedoraproject.org , if that is an issue let me know, but I couldn't see a quick way of using mirrors without having to do index lookups.
- Joe
On Mon, 20 Jun 2016 09:15:56 -0000 "Joseph Mullally" jwmullally@fedoraproject.org wrote:
Hi Kevin, thanks for your reply.
I am guessing one factor against easy linking to source is how its archived in the source control Lookaside Cache.
Yeah, we would need to at least temporarily unpack it, which is a lot of cpu and disk space.
Just for kicks I threw together a small proof of concept RPM content browser web service. It pulls the packages on-demand, and works surprisingly well. The debuginfo RPM contents seem like ideal targets to surface for stack traces, as they have a nice 1:1 relationship with binaries.
https://github.com/jwmullally/test_rpmbrowser/blob/master/README.md http://rpmbrowser-jwmullally.rhcloud.com/
https://retrace.fedoraproject.org/faf/problems/1354441/ http://rpmbrowser-jwmullally.rhcloud.com/rpm/nautilus-debuginfo-3.18.1-1.fc2...
I'll probably take that site down in a few weeks. SRPM extraction doesn't work due to "rpm" not being executable on the old OpenShift containers. It gets the RPMs from http://kojipkgs.fedoraproject.org , if that is an issue let me know, but I couldn't see a quick way of using mirrors without having to do index lookups.
Nice. ;) Once we get our openshift setup you could spin it up there again.
kevin
"KF" == Kevin Fenzi kevin@scrye.com writes:
KF> Yeah, we would need to at least temporarily unpack it, which is a KF> lot of cpu and disk space.
I thought that there was some service which already unpacked tarballs in the lookaside at the time they were uploaded, for the purpose of checksumming all of the files inside. I can never remember what it's called, though.
Anyway, if that still happens then it's the perfect place to do this kind of thing on a grand scale.
- J<
On Wed, 22 Jun 2016 11:51:03 -0500 Jason L Tibbitts III tibbs@math.uh.edu wrote:
"KF" == Kevin Fenzi kevin@scrye.com writes:
KF> Yeah, we would need to at least temporarily unpack it, which is a KF> lot of cpu and disk space.
I thought that there was some service which already unpacked tarballs in the lookaside at the time they were uploaded, for the purpose of checksumming all of the files inside. I can never remember what it's called, though.
Anyway, if that still happens then it's the perfect place to do this kind of thing on a grand scale.
You're thinking of summershum. https://github.com/fedora-infra/summershum
However, it has:
* Nothing using it currently. * No UI/interface to it. * Only records the checksums of each file.
kevin
infrastructure@lists.fedoraproject.org
-
Jason L Tibbitts III -
Joseph Mullally -
Kevin Fenzi