There are several things happening with the package DB this week.
1) Jesse Keating reported that the Brew buildsystem has a packagedb implementation. If approval goes through to open source brew, we'll get the schema for that packageDB as part of the package. I know of a few areas where we'll have to enhance the schema to support all the things we want to do (ACLs for building, committing, and modifying packageDB records) but am unclear on others (Does the brew packageDB have a web interface? Does it tie into Core's CVS?)
2) I've finished an importer for owners.list and some information from the CVS repository for the current schema. I haven't tested it extensively but it has imported data into my local test DB. I'll have to try it on the xen test server next. The importer separates the exporting functionality from the importing so it shouldn't be too hard to switch to a new packagedb schema later.
The importer is owners.py on test3: test3.fedora.phx.redhat.com:/var/www/repo/fedora-packagedb/owners.py
Or available from a Bazaar repository: bzr branch http://www.tiki-lounge.com/~toshio/fedora/fedora-packagedb
3) I've started a redesign of the schema to address ACLs and collection inheritance. I've some questions about this that I'll try to take up with Jesse this week. I'm writing up the schema in SQL now and will post it this week for review.
4) While working on the schema, I've become a bit less enamoured with SQLObject. It seems to make easy things easy and hard things difficult to impossible. For instance, there doesn't seem to be a way to specify multi-field primary keys (or multi-field unique constraints which would do almost as well.) The latest TurboGears beta has preliminary support for a second ORM, SQLAlchemy. I've installed the 0.3.1 of SQLAlchemy here and it is much more flexible. SQLAlchemy is newer than SQLObject, has excelent documentation, and a very active upstream. This is good in that bugs are fixed quickly and features are often added once a user requests them. It's downside is the potential that the API could change and we'd have to port our applications or stay with an older version. Any opinions on this?
-Toshio
On Nov 26, 2006, at 5:20 PM, Toshio Kuratomi wrote:
- While working on the schema, I've become a bit less enamoured with
SQLObject. It seems to make easy things easy and hard things difficult to impossible. For instance, there doesn't seem to be a way to specify multi-field primary keys (or multi-field unique constraints which would do almost as well.)
From the docs: "SQLObject does not support primary keys made up of multiple columns (that probably won't change). It does not generally support tables with primary keys with business meaning -- i.e., primary keys are assumed to be immutable (that won't change)."
(And I happen to agree with their reasoning behind this decision...) Doing unique constraints is easy enough, though:
class Foo: firstName = StringCol(length=30) lastName = StringCol(length=40) firstLastIndex = DatabaseIndex('firstName', 'lastName', unique=1)
Are there any other specific issues that you had with SQLObject?
I don't know SQLAlchemy, but I suspect that for now the "devil you know" is better than the "devil you don't". Or maybe I haven't run into the real-world problems that you have with your schema :)
I think calling Brew a packageDB is a bit of a stretch. From what I recall, yes, it can track who owns what packages (and the packageDB schema I originally suggested was based in part on the brew-style schema :). However, the packageDB has a lot more community-only stuff that just wasn't thought of in the brew world. The best end result would probably be had by continuing the packageDB work, and merging that functionality onto brew when appropriate.
Best, -- Elliot
On Mon, 2006-11-27 at 21:29 -0500, Elliot Lee wrote:
On Nov 26, 2006, at 5:20 PM, Toshio Kuratomi wrote:
- While working on the schema, I've become a bit less enamoured
with SQLObject. It seems to make easy things easy and hard things difficult to impossible. For instance, there doesn't seem to be a way to specify multi-field primary keys (or multi-field unique constraints which would do almost as well.)
From the docs: "SQLObject does not support primary keys made up of multiple columns (that probably won't change). It does not generally support tables with primary keys with business meaning -- i.e., primary keys are assumed to be immutable (that won't change)."
(And I happen to agree with their reasoning behind this decision...) Doing unique constraints is easy enough, though:
class Foo: firstName = StringCol(length=30) lastName = StringCol(length=40) firstLastIndex = DatabaseIndex('firstName', 'lastName', unique=1)
Cool. Just a documentation issue then. (Should have remembered to look under the Index documentation).
Are there any other specific issues that you had with SQLObject?
It's possible that it's just documentation disorganization. Since I've started working with SQLAlchemy I've forgotten details of other problems I had with SQLObject. I'll have to go back and try to work with SQLObject again.
I don't know SQLAlchemy, but I suspect that for now the "devil you know" is better than the "devil you don't". Or maybe I haven't run into the real-world problems that you have with your schema :)
Not sure :-) SQLAlchemy made it quick to figure out what I wanted to do. Coupled with the remarks on the TurboGears mailing list about SQLAlchemy vs SQLObject, I've gotten the impression that SQLObject isn't flexible enough to solve all our problems. I'm not sure that SQLObject is truly not up to handling what I want to do or if I'm just not reading the SQLObject documentation carefully enough, though. The packageDB schema is complex enough to be a good test of both ORMs so I'll test the new TurboGears with both SQLObject and SQLAlchemy on a subset of the packageDB to see what turns up.
I think calling Brew a packageDB is a bit of a stretch. From what I recall, yes, it can track who owns what packages (and the packageDB schema I originally suggested was based in part on the brew-style schema :). However, the packageDB has a lot more community-only stuff that just wasn't thought of in the brew world. The best end result would probably be had by continuing the packageDB work, and merging that functionality onto brew when appropriate.
k. Jesse let FESCo know that if Brew became opensource, it would come with its own packageDB but we didn't get full details of what's in the schema. I'd love to see what brew has to see how it addresses the new requirements (like subcollections). But I can continue to work on the standalone packageDB and then we can merge and modify things as appropriate.
-Toshio
infrastructure@lists.fedoraproject.org