Argh. Can't get the mailing list address right.
Begin forwarded message:
Date: Fri, 27 Jun 2008 17:01:22 -0400
From: Robin Norwood <rnorwood(a)redhat.com>
To: amber-devel(a)lists.fedoraproject.org
Cc: toshio(a)fedoraproject.org, jrb(a)redhat.com, Richard Hughes
<hughsient(a)gmail.com> Subject: Amber status as of now
Hi,
Here's the status of the Amber project, as of now.
TRAC instance: https://fedorahosted.org/amber/wiki
Data import
-----------
What I've got right now works, mostly. See the 'util' directory in git.
fedora_to_amber.py is the 'driver' script. It works like this:
1. It takes a collection name (F9), a package # limit (So it doesn't
try to eat all the fedora packages in one bite), and a path to a
comps.xml file (to get the groups info from comps).
2. It maps the collection name from package db collection name, to koji
tag. This is kind of ugly. I can't find a cross-tool label for 'this
is Fedora 9'.
3. It gets a list of package names for the given collection from the
package db.
4. It makes a temp dir, and changes to it.
5. It starts to look through the package list, and for each package:
a. Checks to see if we've see the package before (by name)
b. Checks to see if we've seen this particular version of the package.
- If we've seen both the name and version before, skips to the next
package.
- If the package is a new version, we try to update the existing
info we have about it. If it's completely new, we try to insert
new records.
c. It insert the package (name,version,collection) into the db.
d. It downloads the rpm (i386, ignoring other arches for now) from
koji.
** koji seems to fairly frequently give bogus 404s and whatnot.
For now I'm ignoring this problem.
e. It extracts the rpm header, and gets some information from that.
f. It extracts the contents of the rpm, slurping up any .desktop files
g. For each .desktop file in the package (That has a '[Desktop
Entry]' section with a 'name' field - some don't....):
- Put all of the data collected from various sources into a
DOAP .rdf file.
- Turn around and parse the doap file into the DB.
(Very much extending/abusing the DOAP format - but I think it's
still useful as a starting point, and perhaps as an interchange
format 'later')
This works ok for now, but has to go through every package in the dist
(up to the limit) for each run. It needs to be fixed to be more
robust. I think something like:
o Initial run for a collection (f9, f10, etc...):
- Slurp up all the package names, put them in a process queue (in the
DB)
- Work through the process queue in a similar manner to what's
described above (with some additional robustness).
Then:
o Periodically poll bodhi watching for package updates to that
collection.
- When an update rolls through, update the package data if needed,
and perhaps notify the amber admin for hand-tweaking.
Next!
Data model
----------
We have a data model for applications, packages, and collections (aka
distribution, e.g. Fedora 9), as well as the relationship to say 'This
application is provided by this package in this collection.'
We have a data model for Suites of applications (eg: The Open
Office.org suite).
The data model is ready to be extended for other relationships, such as:
"This application has plugins available: (eg: flash plugin, pidgin
plugins, etc)."
We have users (basically just an openid URL (I think the format I have
for these is wrong, need to fix that)).
We have screenshots! This worked out pretty nicely, I think, though we
may need to move them out of the DB for performance reasons.
We have a wiki-like long description for applications.
We have tags and categories for apps. Right now we get a bunch of
reduntant/useless categories - need to add editing/hiding of cats.
* We need to add user roles:
o Not logged in user can:
- See everything
- Install applications
o Normal user can:
- Edit certain metadata (the long description, tags, upload
screenshots, reviews)
- Flag content as useless, bogus, inappropriate.
o A normal user who the packageDB tells us 'owns' a package can:
- Edit other metadata about the app(s) provided by the package (like
the fixed description, add 'plugin' relationships, maybe add app
Suites tying apps together...maybe other stuff), add/remove
categories.
- Delete/hide useless, bogus, or inappropriate content for those
apps.
- Create new apps from packages that don't already provide an app
(ie, for things that should be apps but don't have a desktop file)
o A site admin can:
- Do all that stuff for any app ^
- Create, hide, update, modify categories.
* Need to add other app/package relationships.
* Need to add reviews/suitability to task data model stuff.
Controller
----------
* Need to add some more controllers for getting at:
- App suites
- Editing history
- Some sort of 'breadcrumbs' or 'trail' type navigation:
Games -> GNOME Games -> Blackjack -> Screenshots -> blackjack_win
- Better defined normal user/owner/admin interfaces - right now
everything is kind of splatted together.
* Some of my code is clearly not very turbogears/pythonic. I'd love,
love, love a code review along the lines of "Wow, that's the craziest,
most non-pythonic way to do this. Do it this way instead." from Toshio
and/or lmacken. (Or just fix it and commit, I watch the git commits.)
* Need search
View
----
* Lots and lots and lots of design/html/css work still needs to be
done. I'm basically just splatting everything on the screen in <div>
tags. We're supposed to get some help from Tim Allen's group on this.
* Better app 'summary' and 'details' view (Summary view for showing a
lot of apps on the screen, details view for a single app)
* Front page
* App list (per category)
* Better screenshot navigation
* Different/dynamic views based upon user role (for editing/creating
content, etc).
Other
-----
* Need the 'install me' button - everything else is pretty much
pointless without this. Owen says he can probably have someone look at
it after GUADEC. The more I think about it, the more I think that
maybe we should strongly consider using the 'one click install' format,
minus the crazy 'install from some untrusted repo' business. The OCI
guys have actively asked for PackageKit to use their stuff on the pk
mailing list.
* I'd like just a general sanity check of the whole darn thing, design,
python, and turbogears-wise, as well as how I'm interacting with the
rest of the Fedora infrastructure (FAS, packageDB, koji, bodhi soon).
Things seem to work really, really nicely, but I'm sure that are many,
many design improvements that could happen to the whole thing.
* Need to find a box somewhere in fedora infrastructure to host a
pre-alpha style thing to show people the basic concept. I had this as
a milestone for July 1, but that's not going to happen.
* At a bare minimum it needs some of the design/html/css love I
talked about above. I can do some of this if we're not going to get
time from a real designer anytime soon. Right now it's pretty much as
intentionally ugly as I can make it.
* Other contributors/testers/etc. This should come from the public
alpha mentioned above. We have a mailing list, an unofficial channel
on FreeNode (#amber)
* Stuff I'm not thinking of.
-RN
--
Robin Norwood
Red Hat, Inc.
"The Sage does nothing, yet nothing remains undone."
-Lao Tzu, Te Tao Ching
--
Robin Norwood
Red Hat, Inc.
"The Sage does nothing, yet nothing remains undone."
-Lao Tzu, Te Tao Ching
