On Sun, 25 Oct 2009, Ben Boeckel wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Mike McGrath wrote:
On Sun, 25 Oct 2009, Ben Boeckel wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Hi,
I am working on a project designed to mirror large, changing archives of software in a manner that ensures data integrity and atomic updates using a peer-to-peer protocol.
My team and I would like to design this software such that it provides a suitable replacement for rsync. As Fedora is among the leading distributions, I would like to solicit your opinions on the implementation. We have designed much of the architecture, but have not implemented anything yet. One issue we wish to address currently is that of interpreted vs. compiled languages.
My team and I would like to know whether the community would be accepting of such a project (which includes a daemon) if it were written in Python rather than C or C++. If so, it would greatly simplify the implementation and allow it to be more robust. Python's built-in libraries and facilities provide much of the path and network manipulation that the daemon requires. Using the Python standard libraries allows us to rely upon a well-tested base and focus on higher-level issues.
What are your opinions, as system administrators, on using Python for long-running daemon processes when the developers are explicitly mindful of memory considerations?
I've generally had better luck with C/C++ based daemons from a systems admin point of view. But we've used plenty of python based daemons that worked just fine. I think the problem is the lower barrier to using python means it's easier for less experienced developers to create python daemons.
Really though, if it's done right, python daemons can be just as good as C daemons. And extending Python is very easy as you mentioned with using pythons built in libraries. If you're concerned about performance though it shouldn't take much to do the basics of what you want and compare C to python. Sometimes they're identical, sometimes C wins but I don't think I've seen python win yet (with performance). But it is much easier to work with python :)
-Mike
Thanks for your reply.
Seeing as it is a mirroring daemon, the network is the bottleneck. If it isn't then either you're sitting next door, our implementation is bad, or the hardware shouldn't be a mirror in the first place.
With all my babbling I forgot to mention we do already run python daemons in Fedora Infrastructure. Func is one, TurboGears (though it's wrapped in mod_wsgi) and one that we wrote ourselves[1] is our mirrorlist server. It's the backend that powers:
http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-11&arch=i386
-Mike
[1] By 'ourselves' I mean Matt Domsch who, quite honestly, has designed the most robust, fault tolerant dynamic content system we run.