I'd love to play with it. Is systemd stable enough to try it on Fedora 13?

-Ilyes Gouta

On Sat, May 22, 2010 at 10:10 PM, Rahul Sundaram metherid@gmail.com wrote:

On 05/23/2010 02:36 AM, Lennart Poettering wrote:

...

(And as a side note: systemd by default puts a tmpfs to both /var/run and /var/lock).

So do you plan on getting it by default for Fedora 14?   :-)

Rahul

devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel

On Sun, 23.05.10 02:40, Rahul Sundaram (metherid@gmail.com) wrote:

On 05/23/2010 02:36 AM, Lennart Poettering wrote:

...

(And as a side note: systemd by default puts a tmpfs to both /var/run and /var/lock).

So do you plan on getting it by default for Fedora 14? :-)

Well, I am certainly planning to have a package for it in F14. But whether we can have it as default is to be seen.

ATM everything looks rosy. I just finished porting over all F13 installed-by-default daemons to socket activation, and a few more (and the patches are good enough to be upstreamable). I'll publish the numbers of a 100% socket-activated boot soon. Codewise only very little is missing to be fully ready for Fedora (in fact selinux support is the only item really mattering on my todo list).

So far response from the community has been very positive, but we didn't have a fedora-devel discussion about this yet, so we'll have to see whether we can somehow make the broader community as enthusiastic about the whole idea as I am. ;-)

(some of the systemd-related work in other projects already hit rawhide btw, e.g. in udev)

A little bit down the road I'll make my mind up whether I plan to suggest as a default for F14, and then propose that on fedora-devel and FESCO. Stay tuned.

It would certainly be a shame though if other distros would ship systemd by default before we do.

Lennart

On Sun, 23.05.10 04:10, Rahul Sundaram (metherid@gmail.com) wrote:

On 05/23/2010 04:04 AM, Lennart Poettering wrote:

...

ATM everything looks rosy. I just finished porting over all F13 installed-by-default daemons to socket activation, and a few more (and the patches are good enough to be upstreamable). I'll publish the numbers of a 100% socket-activated boot soon. Codewise only very little is missing to be fully ready for Fedora (in fact selinux support is the only item really mattering on my todo list).

It would great to have a repo or something to try it out quickly and provide some feedback.

hughsie prepped some packages for systemd, but I cannot really tell you right now where they are available.

Richard?

And I am not sure if my own yum-fu is good enough to provide you with a repo which you can just install everything from, including the patched daemons...

Lennart

Hi,

systemd package available here: http://people.freedesktop.org/~hughsient/fedora/

Thanks!

Please be patient. To quote myself: "I'll publish the numbers of a 100% socket-activated boot soon."

I won't miss that.

Regards, Ilyes Gouta.

On Sun, May 23, 2010 at 7:32 AM, Richard Hughes hughsient@gmail.com wrote:

On 22 May 2010 23:51, Lennart Poettering mzerqung@0pointer.de wrote:

...

hughsie prepped some packages for systemd, but I cannot really tell you right now where they are available.

systemd package available here: http://people.freedesktop.org/~hughsient/fedora/

The patched daemons are not available yet, so it doesn't yet work very well. Caveat emptor.

Richard.

devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel

On Sun, 23.05.10 00:04, Ilyes Gouta (ilyes.gouta@gmail.com) wrote:

Heya,

So how fast is a systemd boot (with all the changes to the scripts) compared to the current F13 setup? How about a ratio?

Please be patient. To quote myself: "I'll publish the numbers of a 100% socket-activated boot soon."

Lennart

On Sat, May 22, 2010 at 11:34 PM, Lennart Poettering mzerqung@0pointer.de wrote:

On Sun, 23.05.10 02:40, Rahul Sundaram (metherid@gmail.com) wrote:

...

On 05/23/2010 02:36 AM, Lennart Poettering wrote:

...

(And as a side note: systemd by default puts a tmpfs to both /var/run and /var/lock).

So do you plan on getting it by default for Fedora 14?   :-)

Well, I am certainly planning to have a package for it in F14. But whether we can have it as default is to be seen.

ATM everything looks rosy. I just finished porting over all F13 installed-by-default daemons to socket activation, and a few more (and the patches are good enough to be upstreamable). I'll publish the numbers of a 100% socket-activated boot soon. Codewise only very little is missing to be fully ready for Fedora (in fact selinux support is the only item really mattering on my todo list).

So far response from the community has been very positive, but we didn't have a fedora-devel discussion about this yet, so we'll have to see whether we can somehow make the broader community as enthusiastic about the whole idea as I am. ;-)

(some of the systemd-related work in other projects already hit rawhide btw, e.g. in udev)

A little bit down the road I'll make my mind up whether I plan to suggest as a default for F14, and then propose that on fedora-devel and FESCO. Stay tuned.

If its possible to turn it on in a spin for testing I would quite happily volunteer the Moblin/Meego spin to be a test platform for F-14 if its decided its too risky to do for the main platform. In terms of services its very minimal so it might be a good starter.

Peter

On 05/23/2010 04:19 AM, Kevin Kofler wrote:

Lennart Poettering wrote:

...

So far response from the community has been very positive, but we didn't have a fedora-devel discussion about this yet, so we'll have to see whether we can somehow make the broader community as enthusiastic about the whole idea as I am. ;-)

I, for one, think systemd should be the default ASAP.

Perhaps the first time I can recall that we have agreed. ;)

~spot

On Tue, 25.05.10 10:21, Casey Dahlin (cdahlin@redhat.com) wrote:

On Mon, May 24, 2010 at 09:05:31AM -0400, Tom spot Callaway wrote:

...

On 05/23/2010 04:19 AM, Kevin Kofler wrote:

...

Lennart Poettering wrote:

...

So far response from the community has been very positive, but we didn't have a fedora-devel discussion about this yet, so we'll have to see whether we can somehow make the broader community as enthusiastic about the whole idea as I am. ;-)

I, for one, think systemd should be the default ASAP.

Perhaps the first time I can recall that we have agreed. ;)

~spot

Any particular reason on either of your parts?

Just about everything in systemd is either set to be in upstart (simpler dependency syntax, xinetd-style service activation) or was discarded by it years ago (cgroups are a dead end).

Oh, is that so?

Have you actually read the blog story I put together?

Why do you say "cgroups are a dead end"? Sure, Scott claims that, but uh, it's not the only place where he is simply wrong and his claims baseless. In fact it works really well, and is one of the strong points in systemd. I simply see no alternative for it. The points Scott raised kinda showed that he never really played around with them.

Please read up on cgroups, before you just dismiss technology like that as "dead end".

The assumption that just because its new means its more advanced is in this case a bit misguided.

Well, please read the blog story. I know it's long, but it should be an interesting read. The whole point of the blog story is to make clear the reason systemd exists is purely technical, and that we think our design is simply the better one.

We didn't do systemd for political reasons.

We didn't do systemd just to do things differently.

We did systemd because we thought that technically Upstart is fundamentally flawed and misses out on so many opportunities.

And if you cannot see that then I can only beg you to read the blog story, because it goes into much detail about all the technical features.

Please, judge systemd on technical grounds, don't judge it on political, or emotional grounds.

Thank you,

Lennart

On Tue, 2010-05-25 at 09:00 -0800, Jeff Spaleta wrote:

On Tue, May 25, 2010 at 7:45 AM, Lennart Poettering mzerqung@0pointer.de wrote:

...

Please, judge systemd on technical grounds, don't judge it on political, or emotional grounds.

"I'll publish the numbers of a 100% socket-activated boot soon."

I would love to have the necessary data to have an informed public discussion on the technical merits which include some benchmarking and some cookie cutter packages to play around with. I want t be given a chance to rig up some pathological poor performance systemd benchmarks that you can publicly cut to shreds as part of the educational process.

My one concern about the current discussion is that the decision to move to systemd in the timeframe of F14 is being rushed ahead of the aforementioned technical judgement. I can understand your personal enthusiasm for the move as you are intimately familiar with systemd. But I'm not sure its adequate to rush ahead until we all have a better feel on how this works in practise... and how much work its going to take to convert all our existing packages to systemd styled startup. The back and forth on your blog between yourself and Tim Waugh about cups service activation, for example, is something I think we all need to understand. I think there is a lot buried in that discussion about how complex some of our existing services are going to look under systemd styled configuration.

-jef

Right now, it's not about speed. Speed is one thing, and somewhat important. But doing it right is also important. Make it right, then make it fast, because if you try to make it fast first, you'll often be doing it wrong.

On Tue, 25.05.10 09:42, Jeff Spaleta (jspaleta@gmail.com) wrote:

Like I said... we need to have more discussion a long the lines of what I saw concerning how cups and other services will work in a systemd setting. We have to have a feel for the complexity of configuration.

We'll also need to have good examples showing the sysV and inetd compatibility at work. Not that I don't trust what was written in the blog story, but having tangible example of a parallel installable systemd that used the existing SySV styled init scripts as a precursor for a serious discussion is going to go a long way at helping the signal to noise ratio. I didn't see the systemd packages in the people space Richard pointed to. Can someone point me to an updated url?

OK, here's an example, for a relatively modern and clean daemon, the dbus system bus. Its SysV init script is relatively simple (see /etc/init.d/messagebus). And the configuration to make it socket-bus activatable would be even simpler and look like this:

You'd have two units, one for the socket:

http://0pointer.de/public/dbus.socket

The contents should be pretty self-explanatory.

And one for the service itself, that is activated when the socket units receives traffic:

http://0pointer.de/public/dbus.service

This one is pretty self-explanatory too I guess. It defines some requirement and ordering dependencies, to make sure that dbus is started after the initial base system is set up and after the socket itself is set up. (Later on we might even drop part of those ordering dependencies, since dbus could run much earlier at boot-up if it was simply using an abstract namespace socket, instead of relying on a writable /var. But that's another discussion)

(Oh, don't try to run actually use that .socket and .service file, you need my patches that add --address to dbus daemon, and the special "systemd:" address)

Lennart

On Tue, 25.05.10 13:15, Simo Sorce (ssorce@redhat.com) wrote:

...

My one concern about the current discussion is that the decision to move to systemd in the timeframe of F14 is being rushed ahead of the aforementioned technical judgement. I can understand your personal enthusiasm for the move as you are intimately familiar with systemd. But I'm not sure its adequate to rush ahead until we all have a better feel on how this works in practise... and how much work its going to take to convert all our existing packages to systemd styled startup. The back and forth on your blog between yourself and Tim Waugh about cups service activation, for example, is something I think we all need to understand. I think there is a lot buried in that discussion about how complex some of our existing services are going to look under systemd styled configuration.

I second that.

I'd like to remember that there are *many* upstreams are going to resistant to this change. A lot of upstream projects need to be compatible to a lot of Linux and Unix systems, even old ones, so before they move they want guarantee that this is really going to be the next thing. That means that until most distributions start using systemd they are not going to do the work.

They don't *need* to move. It's not an exclusive disjunction, that you'd have to support either systemd or not systemd. You can easily support both systemd activation and classic sysv from the same binary. The 10 daemons we have patched so far only needed about 10 lines of changes each to become socket-activatable. And those changes are nops unless $LISTEN_FDS is set.

Or in other words: moving to systemd is a gradual process, where for every service we can decide indivdually how we want to do it:

1) continue use a classic sysv init script 2) provide a native unit file 3) provide a native unit file and do socket/bus based activation

And all that for the same binary.

As mentioned I have now patched the 10 daemons or so we ship by default (and a few more) to become socket activatable. If we can merge those upstream and into our packages we already are a big step ahead and everything else can come later.

Lennart

On Tue, 25 May 2010, Simo Sorce wrote:

I'd like to remember that there are *many* upstreams are going to resistant to this change. A lot of upstream projects need to be compatible to a lot of Linux and Unix systems, even old ones, so before they move they want guarantee that this is really going to be the next thing. That means that until most distributions start using systemd they are not going to do the work.

So before it is rushed in it is paramount that tests are done with those application s that will not have systemd support and make sure there is not going to be regressions there.

Even bigger question: The architecture has to be correct.

Systemd is trying to solve dependencies between applications by considering simple FD activity (by acting as a proxy for applications). However, there almost certainly are more complex, logical dependencies which are not visible to systemd, and hence which apps will still have to deal with and resolve between themselves.

In short, we will still need that app-layer. We need to consider how that impacts architecturally on the worth of solving a subset of problems in a lower layer.

regards,

tis 2010-05-25 klockan 20:30 +0200 skrev Lennart Poettering:

/etc/systemd/system. We could for example choose to not have that symlink by default, but ask the administrator to create it if he sets up a multiplexing print server the way Tim suggested.

Wouldn't it be started automatically anyway when something connects to systemd's *:631 socket?

/Alexander

Lennart Poettering wrote:

If we chose to adopt it for F14 it would probably a good idea to have native files for everything that is installed by default at least. I have mostly written those files now, so this goal should not be too hard to reach, if we can convince all the maintainers involved to include them in their packages.

I think that if the maintainers don't do it, it should just be done by a provenpackager. This is global distro integration and needs to be done at global distro level.

Kevin Kofler

On 05/25/2010 08:44 PM, Jon Masters wrote:

On Tue, 2010-05-25 at 17:45 +0200, Lennart Poettering wrote:

...

Please, judge systemd on technical grounds, don't judge it on political, or emotional grounds.

I'm not trying to "judge" it at all. But when the first thing I really hear about a project is "we should replace our init system", my ears prick up and I start to dig to find out what the new thing is (which I admit I had never heard of) and why the rush. I'll read your blog again, and I'm sure you have good ideas, I'm just not keen to see us rush into replacing the init system again without some good co-ordination (I'm not just thinking with my RHEL hat on, but I admit to also thinking about other users of the base bits in the future).

I'll shutup now and read your blog.

Jon.

It doesn't have to be a "rush". systemd can easily be an alternative init system in F14, if the services provide LSB initscripts. Our goal for F14 should be to try to get the socket activation patches upstream, which are really not obtrusive. The decision if and when to switch to systemd can be made later on. We are not in a hurry. Even after some releases with upstart our initscripts have not yet been ported to upstart event files, which says a lot IMHO :-)

On 05/26/2010 08:32 AM, Casey Dahlin wrote:

I'd be happy to firm up the design details with you if you wanted to contribute patches.

I would recommend that noone do that unless Canonical's drops it's flawed copyright license agreement.

Rahul

On 05/26/2010 05:02 AM, Casey Dahlin wrote:

On Tue, May 25, 2010 at 05:45:07PM +0200, Lennart Poettering wrote:

...

We did systemd because we thought that technically Upstart is fundamentally flawed and misses out on so many opportunities.

And we think the same of systemd.

Who is "we" and which opportunity is systemd missing?

James Findley wrote:

Modern systems just don't take very long to spawn awk. Or sed. Or cut. Or bash. IMO this sort of tradeoff between speed and ease of use hasn't been appropriate in 20 years.

It's really not at all uncommon for me to need to modify an init script. There would be much rage if in order to do this I had to download the SRPM, extract the init code, figure out what I needed to change, modify it, recompile then install.

Absolutely.

I remember something similar with hal-disk-something, related to mount options. One day everything got switched to compiled code and manageability approached zero.

On 26/05/10 11:12, Bastien Nocera wrote:

On Wed, 2010-05-26 at 10:01 +0100, James Findley wrote:

...

On 26/05/10 04:02, Casey Dahlin wrote:

...

On Tue, May 25, 2010 at 05:45:07PM +0200, Lennart Poettering wrote:

...

On Tue, 25.05.10 10:21, Casey Dahlin (cdahlin@redhat.com) wrote:

...

On Mon, May 24, 2010 at 09:05:31AM -0400, Tom spot Callaway wrote:

...

On 05/23/2010 04:19 AM, Kevin Kofler wrote: > Lennart Poettering wrote: >> So far response from the community has been very positive, but we didn't >> have a fedora-devel discussion about this yet, so we'll have to see >> whether we can somehow make the broader community as enthusiastic about >> the whole idea as I am. ;-) > > I, for one, think systemd should be the default ASAP.

Perhaps the first time I can recall that we have agreed. ;)

~spot

Any particular reason on either of your parts?

Just about everything in systemd is either set to be in upstart (simpler dependency syntax, xinetd-style service activation) or was discarded by it years ago (cgroups are a dead end).

Oh, is that so?

Have you actually read the blog story I put together?

Yes, but I'm going to go read it again just to be sure.

...

Why do you say "cgroups are a dead end"? Sure, Scott claims that, but uh, it's not the only place where he is simply wrong and his claims baseless. In fact it works really well, and is one of the strong points in systemd. I simply see no alternative for it. The points Scott raised kinda showed that he never really played around with them.

Please read up on cgroups, before you just dismiss technology like that as "dead end".

I did. When upstart was about to use them. 2 years ago. We chucked them by the following LPC.

The problem we've found is that cgroups are too aggressive. They don't have a notion of sessions and count too much as being part of your service, so you end up with your screen session being counted as part of gdm.

This is why setsid was added to the netlink connector.

...

...

The assumption that just because its new means its more advanced is in this case a bit misguided.

Well, please read the blog story. I know it's long, but it should be an interesting read. The whole point of the blog story is to make clear the reason systemd exists is purely technical, and that we think our design is simply the better one.

So you have:

1. Socket activation. Part of Upstart's roadmap. Would happen sooner if you

cared to submit the patch. We don't think its good enough by itself, hence the rest of Upstart, but a socket activation subsystem that could reach as far as systemd and even work standalone in settings where systemd can work is perfectly within Upstart's scope. I'd be happy to firm up the design details with you if you wanted to contribute patches.

2. Bus activation. Missed opportunity here to actually become the launchpoint

for activated services. I won't criticize that too much though, as its usefulness is largely dependent on kernelspace DBus, which I've been trying to bludgeon Marcel Holtmann into turning over to the public for a year now.

3. Cutting down on the forking by replacing some of the shell scripts... cool 3a) With C code... really?

Yeah. I think this is odd too. The blog complains about how many awk spawns there are - but this looks like a complaint that belongs in the 70's.

It really doesn't take long to launch awk. at all. And most of the things people are asking awk to do in shell scripts are very trivial, requiring very little processing.

using a simple for i in {1..1000}; do ... loop I can launch on this rubbish old laptop a thousand awk processes in 3.35 seconds. By comparison, it takes 2.24 seconds to launch a thousand C hello world programs. So C is faster. Just about.

Now run a loop, in C, which does the same thing.

$ time ./foo> /dev/null

real 0m0.002s user 0m0.001s sys 0m0.001s

You're comparing the wrong thing here - I was demonstrating that it doesn't take noticeably longer to spawn awk than a small C app on modern systems. thus using: for i in {1..1000}; do awk 'BEGIN{print "Hello World"}' > /dev/null; done for i in {1..1000}; do ./helloworld > /dev/null; done we compare how long it takes to start a trivial C program 1000 times to a trivial awk program 1000 times. The bash for loop overhead is present in both cases, and since we are only interested in the difference in speed, we can ignore it. (I actually ran this comparison a number of times, and used the mean value)

You're comparing how long it takes to launch an awk program 1000 times to how long it takes to run 1000 iterations of a loop in C. This is not an especially useful thing to do.

...

But the complaint in the blog is that awk is called 92 times. By this metric that costs the user 0.3 seconds of CPU time. To get rid of this horrible waste of resources we are compiling all our startup scripts.... wait. Something is wrong with that picture ;)

There's no compiling of startup scripts, read the blog again.

"On my system the scripts in /etc/init.d call grep at least 77 times. awk is called 92 times, cut 23 and sed 74. [...] Of course, that has to be incredibly slow" #from the blog.

"So, let's get rid of shell scripts in the boot process" #from the blog.

I'm afraid it appears to be you that needs to re-read. I suggest that you read the section entitled "Keeping the First User PID Small".

I'm sorry to say that while it was slow in the 70's, and it would be too slow for a kernel function, on modern hardware all this completes in well under a second, even on this old and rubbish laptop. that's not really slow by any useful *user* definition.

...

Modern systems just don't take very long to spawn awk. Or sed. Or cut. Or bash. IMO this sort of tradeoff between speed and ease of use hasn't been appropriate in 20 years.

Not very long multiplied by cold caches, and memory allocations and frees, etc. The problem isn't that awk is launched, is that it's launched a 100 times along with grep, cut, whatever kind of sticks and strings you can get in shell text processing. That's hardly something a modern init system should be doing.

What. Are you seriously suggesting that launching a 340k binary, which links against a grand total of 4 libs, all of which will be in FS cache anyway as pretty much _everything_ uses them, is an issue because of "cold caches" and "memory allocations"? /really/? If saving a couple of kilobytes of memory, total, across the entire boot process is this much of an issue for you, I rather think there are easier ways to achieve that. Or, you could note that most modern systems appear to be sold with between 2 and 4 gigabytes of ram, and that it is rather hard to see any noticeable difference.

Sorry. I like fast boots as much as the next person - this is just a bad trade. We only save ourselves (much) less than a second of boot time, but turn a common and simple practice into a major headache.

If anyone has a bash/awk/whatever initscript that actually takes a user-noticeable amount of time to run, this means that you've done something badly wrong - it isn't the language at fault. I would be happy to help you fix this - just poke me on IRC.

-siXy

On 26/05/10 12:03, drago01 wrote:

On Wed, May 26, 2010 at 12:53 PM, James Findleysixy@gmx.com wrote:

...

[...] Sorry. I like fast boots as much as the next person - this is just a bad trade. We only save ourselves (much) less than a second of boot time, but turn a common and simple practice into a major headache.

If anyone has a bash/awk/whatever initscript that actually takes a user-noticeable amount of time to run, this means that you've done something badly wrong - it isn't the language at fault. I would be happy to help you fix this - just poke me on IRC.

But the speed ups _add_ up ... lets say you save x, y, z time uints for script A, B and C ... you end up saving x+y+z.

Oh, I agree that they add up completely. It's just that they don't add up to very much. 0.3 seconds for 93 spawns of awk, remember?

In terms of a helpful comparison let's assume you run 50 initscripts on boot (unlikely on a desktop - but lets go for worst case)

We will compare 3 things, and run each comparison 3 times to get a mean result.

1) a poorly written bash script, that does horrible things like piping grep to awk (sidenote: this is always wrong, don't do it), spawning subshells, sourcing stuff when not required, etc

2) a clean bash script, that still calls awk but gets rid of unnecessary sources, subshells, etc.

3) an incredibly lightweight C program that just execv()'s the program

in all cases the program we will use is /bin/true - we just want something consistant and small. Each is run 50 times in a loop.

Result:

1: 0.57 seconds 2: 0.20 seconds 3: 0.15 seconds

cleaning up the initscripts - which are often fairly poorly written - saves us nearly as much as writing them in C. And even the worst case is only ~ half a second total across the entire boot process.

I just don't see any evidence at all that writing out initscripts in C really gets us anywhere useful in the real world, and I am extremely sure that it would be a significant pain in the ass to many, many people.

Yu want the initscripts to run faster? Sure. lets clean them up. For a start /etc/init.d/functions is a mess that takes way to many CPU cycles for what it does - and includes a lot of functions that really aren't widely used enough to justify sourcing by _every_ initscript. Massively obfuscating our boot process, however, is not the answer.

-siXy

Once upon a time, Kevin Kofler kevin.kofler@chello.at said:

IMHO replacing slow interpreted code by fast compiled code is always a good idea, especially so if the interpreted code is shell code with its massive abuse of process spawning.

No, it isn't. It makes the process much more opaque to system administrators and makes it harder to make quick fixes and simple local changes.

If there isn't a measurable and significant speed improvement, it is definately a waste of time to replace working scripts.

Once upon a time, Kevin Kofler kevin.kofler@chello.at said:

Editing the code is the wrong way to make "simple local changes".

That is only true if you can anticipate every task every system admin will ever need to do with your tools.

I've edited init scripts for multiple reasons. For example, I have multiple instances of some daemons running, so I'll make multiple copies of the init script and edit as needed for config/PID file locations, etc.

On Thu, 27.05.10 10:13, Chris Adams (cmadams@hiwaay.net) wrote:

Once upon a time, Kevin Kofler kevin.kofler@chello.at said:

...

Editing the code is the wrong way to make "simple local changes".

That is only true if you can anticipate every task every system admin will ever need to do with your tools.

I've edited init scripts for multiple reasons. For example, I have multiple instances of some daemons running, so I'll make multiple copies of the init script and edit as needed for config/PID file locations, etc.

Just because we want to do more stuff in C then in shell it doesnt mean things become less editable or configurable. So, when you previously edited the init script, you can now edit the (much shorter and simpler) .service file, adjusting the various options it offers you. If you previously maintained a modified copy of an existing init script, you can now just have a modified copy of a .service file. There's really nothing that changes here.

Lennart

On Wed, 26.05.10 10:01, James Findley (sixy@gmx.com) wrote:

...

3. Cutting down on the forking by replacing some of the shell scripts... cool 3a) With C code... really?

Yeah. I think this is odd too. The blog complains about how many awk spawns there are - but this looks like a complaint that belongs in the 70's.

It really doesn't take long to launch awk. at all. And most of the things people are asking awk to do in shell scripts are very trivial, requiring very little processing.

Maybe individually its not a problem. But doing that all the time (> 1000x or so during boot) is just slow.

using a simple for i in {1..1000}; do ... loop I can launch on this rubbish old laptop a thousand awk processes in 3.35 seconds. By

It's no only about launching it. Take a couple of tools, pipe them together, and actually execute some code with it.

comparison, it takes 2.24 seconds to launch a thousand C hello world programs. So C is faster. Just about. But the complaint in the blog is that awk is called 92 times. By this metric that costs the user 0.3 seconds of CPU time. To get rid of this horrible waste of resources we are compiling all our startup scripts.... wait. Something is wrong with that picture ;)

Dude, this is not what I am suggesting, I want no "compiled" startup scripts.

All I am saying that the majority of stuff should just be done in the init system or the daemons themselves. And if there is something to configure then do so in the .service file or in the daemon configuration. But there is no point in keeping all in shell.

Also 2.24s is already quite a bit, if I may say so.

Modern systems just don't take very long to spawn awk. Or sed. Or cut. Or bash. IMO this sort of tradeoff between speed and ease of use hasn't been appropriate in 20 years.

Well, they actually do if you do that a thousand times.

It's really not at all uncommon for me to need to modify an init script. There would be much rage if in order to do this I had to download the SRPM, extract the init code, figure out what I needed to change, modify it, recompile then install.

We are not taking control away from you: you can still configure the .service file to your needs, and you have much more high-level controls in there, and if you really want to you can still plug in a shell script instead of the daemon binary itself and things will work just fine.

Also, much the code in the init scripts simply goes away without any need for replacement if the init system is just smarter. You will have to debug less, you will have to fix less, because there is less code, and less buggy code, and less fragile shell code. You also will have less duplication in each and every init script.

Lennart

Le mercredi 26 mai 2010 à 19:39 +0200, Alexander Boström a écrit :

ons 2010-05-26 klockan 10:01 +0100 skrev James Findley:

...

It's really not at all uncommon for me to need to modify an init script. There would be much rage if in order to do this I had to download the SRPM, extract the init code, figure out what I needed to change, modify it, recompile then install.

Various ways to deal with that:

1. Change the Exec=/usr/libexec/food to

ExecStart=/usr/local/sbin/foodwrapper

Won't work since one of the main things current scripts do is run some code as root, and some other code as the target user. Here you are forcing all the shell part to run as either the target user (in which case the root parts won't work) or as root (in which case you have to reimplement all the shell user changing logic systemd is supposed to deprecate)

On Wed, 26.05.10 19:39, Alexander Boström (abo@root.snowtree.se) wrote:

ons 2010-05-26 klockan 10:01 +0100 skrev James Findley:

...

It's really not at all uncommon for me to need to modify an init script. There would be much rage if in order to do this I had to download the SRPM, extract the init code, figure out what I needed to change, modify it, recompile then install.

Various ways to deal with that:

1. Change the Exec=/usr/libexec/food to

ExecStart=/usr/local/sbin/foodwrapper

Yes, that works well.

2. Add some hook support in the systemd config files.

We actually have that in place already. You can hook in as many shell scripts as you want, like this:

ExecStartPre=/some/shell/script/to/run/before/the/daemon ExecStartPost=/some/shell/script/to/run/after/the/daemon ExecStopPre=... ExecStopPost=...

3. Add support for switching specific services back to using the

initscript even when booting with systemd.

You can easily do that, simply by renaming the .service file.

Lennart

On Wed, 26 May 2010 12:42:13 +0200 drago01 drago01@gmail.com wrote:

On Wed, May 26, 2010 at 5:02 AM, Casey Dahlin cdahlin@redhat.com wrote:

...

On Tue, May 25, 2010 at 05:45:07PM +0200, Lennart Poettering wrote:

...

On Tue, 25.05.10 10:21, Casey Dahlin (cdahlin@redhat.com) wrote:

[...] 3) Cutting down on the forking by replacing some of the shell scripts... cool 3a) With C code... really?

This does make a lot of sense to me, initscripts being scripts is a major slowdown factor by itself.

It is not like you want to edit the scripts all the time, so there is no reason for them being scripts.

While you don't edit them *all* the time, it is something that is done regularly, and it is something most admins can do with ease. Turn them in a C program and you left admins out in the cold, most of them.

I would be very, very wary of accepting a C "init script". An unmanageable system is a useless system.

Simo.

* Ola Thoresen [26/05/2010 14:39] :

Would it not be more fruitful to discuss _why_ you (we?) need to edit the initscripts? Describe what functionality is missing or wrong in the default ones?

Editing environnement variables and indicating which specific interfaces I want the daemon to listen on for requests are the only real-world cases that come to my mind straight away.

Emmanuel

On 05/26/2010 10:20 PM, Nathanael D. Noblet wrote:

On 05/26/2010 06:57 AM, Emmanuel Seyman wrote:

...

• Ola Thoresen [26/05/2010 14:39] :

...

Would it not be more fruitful to discuss _why_ you (we?) need to edit the initscripts? Describe what functionality is missing or wrong in the default ones?

Editing environnement variables and indicating which specific interfaces I want the daemon to listen on for requests are the only real-world cases that come to my mind straight away.

For most of the scripts I've seen this used, that is changed in /etc/sysconfig/daemonname. Like dhcpd etc, which isn't in the initscript itself. Just pointing it out, not taking sides...

And this could easily be specified in the daemon config itsself. No need for a shell wrapper, which only sets the command line arguments.

On Wed, 26 May 2010, Simo Sorce wrote:

While you don't edit them *all* the time, it is something that is done regularly, and it is something most admins can do with ease. Turn them in a C program and you left admins out in the cold, most of them.

I would be very, very wary of accepting a C "init script". An unmanageable system is a useless system.

+20 million.

I couldn't agree more. They need to be scripts, considering how seldom they actually run it makes even less sense to chase down optimization in them by making them compiled.

-sv

On Wed, May 26, 2010 at 08:54:23AM -0400, Seth Vidal wrote:

On Wed, 26 May 2010, Simo Sorce wrote:

...

While you don't edit them *all* the time, it is something that is done regularly, and it is something most admins can do with ease. Turn them in a C program and you left admins out in the cold, most of them.

I would be very, very wary of accepting a C "init script". An unmanageable system is a useless system.

+20 million.

I couldn't agree more. They need to be scripts, considering how seldom they actually run it makes even less sense to chase down optimization in them by making them compiled.

-21 million.

Scripts are a crutch to avoid properly designed daemons and configuration systems. I never edit initscripts to "configure" daemons, because they would just be overwritten at the next package upgrade. Configuration should be separate from code.

On Wed, 26 May 2010 09:08:09 -0400 (EDT) Seth Vidal skvidal@fedoraproject.org wrote:

On Wed, 26 May 2010, Chuck Anderson wrote:

...

-21 million.

Scripts are a crutch to avoid properly designed daemons and configuration systems. I never edit initscripts to "configure" daemons, because they would just be overwritten at the next package upgrade. Configuration should be separate from code.

And given my experience dealing with actual real-world designed daemons and other such things, I'd say we've shown no ability to 'properly' design them and won't be showing that anytime soon. So since we have to live in the world, let's not go shooting our feet off.

-sv

If people are really keen on "C init scripts", then a compromise could be to make it optional to use a bash script when the admin wants to do something. It would be a change and require re-training to understand how to do that, but it will be at least possible.

Simo.

I got a BZ for a package I maintain from someone who needs multilib support without using Mock:

https://bugzilla.redhat.com/show_bug.cgi?id=595923

I found myself asking what the requirements are for a -devel package. In general, do we support this in -devel libs or not? On IRC, I think I've learned that the answer is "yes", but ....

.... this also made me realize that I really don't know what other requirements we may have for -devel packages, I wondered if everyone building -devel packages is providing multilib support, and I wondered if we need some way to detect conflicts among headers for multiple architectures. In this case, the problem was a conflict in the size of a long:

#define XERCES_SIZEOF_LONG 4 vs. #define XERCES_SIZEOF_LONG 8

Are there other requirements for -devel packages that I should be aware of? Is there something I should read?

Jonathan

On Wed, 26.05.10 09:08, Seth Vidal (skvidal@fedoraproject.org) wrote:

...

Scripts are a crutch to avoid properly designed daemons and configuration systems. I never edit initscripts to "configure" daemons, because they would just be overwritten at the next package upgrade. Configuration should be separate from code.

And given my experience dealing with actual real-world designed daemons and other such things, I'd say we've shown no ability to 'properly' design them and won't be showing that anytime soon. So since we have to live in the world, let's not go shooting our feet off.

Well, this sounds like an ad for systemd:

since it is hard to get writing daemons right, we have tought systemd a lot of tricks so that what you need to do for proper daemonizations is near to zero. systemd can do a lot of stuff for you, the forking, the supervising, the dropping of privs, the setting up of a clean working environment and more. Also, PID files are unnecessary with systemd. In effect writing a daemon becomes very easy then, because the environment will be set up completely for you right-away.

Lennart

On Wed, 26.05.10 09:53, Andrew Parker (andrewparker@bigfoot.com) wrote:

...

...

I couldn't agree more. They need to be scripts, considering how seldom they actually run it makes even less sense to chase down optimization in them by making them compiled.

-21 million.

Scripts are a crutch to avoid properly designed daemons and configuration systems.  I never edit initscripts to "configure" daemons, because they would just be overwritten at the next package upgrade.  Configuration should be separate from code.

I don't edit them, but I do frequently look at them to see what they're doing/why they aren't doing something/what config files i can add/edit to change behaviour etc.

actually, i do edit them sometimes to add a temporary "-x" to them. sure as heck beats gdb.

But the question is whether it beats systemd's kernel opts such as "systemd.confirm_spawn=" or "systemd.log_level=", which are much more useful to debug or trace the start-up of services.

And again, nobody said anything of replacing the current shell scripts with identical equivalents written in C. There will be no shell-to-C compiler or any such madness.

Lennart

On Wed, May 26, 2010 at 4:07 PM, Jeremy Sanders jeremy@jeremysanders.net wrote:

Seth Vidal wrote:

...

+20 million.

I couldn't agree more. They need to be scripts, considering how seldom they actually run it makes even less sense to chase down optimization in them by making them compiled.

Absolutely. I have no idea why you shouldn't use a small and light interpreted language rather than C. You would have a standard library of useful init related functions, so you wouldn't have to fork awk, etc. The actual init scripts would be very small then. C is also missing useful datatypes such as maps, which would require libraries to load.

Something like Lua would be very good. The overheads over C would be minimal, and it would have the advantage of being editable.

Well spawing your logic further means we should avoid compiled programs at all, what if I want configure $app by editing its source code? Oh wait it is written in C ...

If it goes down to "want to edit scripts for configuration reasons" (which isn't sane anyway) compared to a faster an cleaner boot process I'd opt for the later.

Seriously source code is NOT and never was intended to be a configuration system period.

Jeremy Sanders (jeremy@jeremysanders.net) said:

Something like Lua would be very good. The overheads over C would be minimal, and it would have the advantage of being editable.

I've had to edit an init script to get something working properly many times.

If you're going to want them to be editable to pass the lowest-common-denominator test of whatever admins might be editing them, I think bash is probably the only reasonable choice.

Bill

On Wed, 2010-05-26 at 08:54 -0400, Seth Vidal wrote:

On Wed, 26 May 2010, Simo Sorce wrote:

...

While you don't edit them *all* the time, it is something that is done regularly, and it is something most admins can do with ease. Turn them in a C program and you left admins out in the cold, most of them.

I would be very, very wary of accepting a C "init script". An unmanageable system is a useless system.

+20 million.

I couldn't agree more. They need to be scripts, considering how seldom they actually run it makes even less sense to chase down optimization in them by making them compiled.

This is a nice example of how discussions on this list go off into the weeds in no time.

'compiling initscripts' ? come on, that is just silly. Nobody is proposing such a thing.

It is completely clear that there needs to be some flexibility in any init system to cater to real-world services.

But it is also clear that there is a difference between gobs of shell script and nice and clean files like the ones you find in /etc/init.

On 26/05/10 15:20, Matthias Clasen wrote:

On Wed, 2010-05-26 at 08:54 -0400, Seth Vidal wrote:

...

On Wed, 26 May 2010, Simo Sorce wrote:

...

While you don't edit them *all* the time, it is something that is done regularly, and it is something most admins can do with ease. Turn them in a C program and you left admins out in the cold, most of them.

I would be very, very wary of accepting a C "init script". An unmanageable system is a useless system.

+20 million.

I couldn't agree more. They need to be scripts, considering how seldom they actually run it makes even less sense to chase down optimization in them by making them compiled.

This is a nice example of how discussions on this list go off into the weeds in no time.

'compiling initscripts' ? come on, that is just silly. Nobody is proposing such a thing.

It is completely clear that there needs to be some flexibility in any init system to cater to real-world services.

But it is also clear that there is a difference between gobs of shell script and nice and clean files like the ones you find in /etc/init.

Actually the blog post is proposing exactly that, as I read it. And it seems not only that lots of other people read it the same way, but some even agree with it.

So I'm not sure I see how this is going off into the weeds - if transitioning some/all initscripts to C is a genuine goal of the author of the project, on which he is not prepared to budge, then that is likely to have bearing on its adoption into fedora, and rightly so.

-siXy

On Wed, 26.05.10 17:41, Bill Nottingham (notting@redhat.com) wrote:

James Findley (sixy@gmx.com) said:

...

Actually the blog post is proposing exactly that, as I read it. And it seems not only that lots of other people read it the same way, but some even agree with it.

So I'm not sure I see how this is going off into the weeds - if transitioning some/all initscripts to C is a genuine goal of the author of the project, on which he is not prepared to budge, then that is likely to have bearing on its adoption into fedora, and rightly so.

What I took from it is that code like this from rc.sysinit:

... if [ ! -e /proc/mounts ]; then mount -n -t proc /proc /proc mount -n -t sysfs /sys /sys >/dev/null 2>&1 fi ...

really doesn't need to be there. If you're redesigning the init system, rather than keeping compatibility with ancient stupid sysvinit, there's no reason that shell code is needed for this; init should Just Do It when it starts. In fact, I'm pretty sure recent upstart releases do the same thing. I'm sure we can find more examples if we look.

Yes, this is exactly what I meant.

Lennart

On Wed, 2010-05-26 at 08:54 -0400, Seth Vidal wrote:

On Wed, 26 May 2010, Simo Sorce wrote:

...

While you don't edit them *all* the time, it is something that is done regularly, and it is something most admins can do with ease. Turn them in a C program and you left admins out in the cold, most of them.

I would be very, very wary of accepting a C "init script". An unmanageable system is a useless system.

+20 million.

I couldn't agree more. They need to be scripts, considering how seldom they actually run it makes even less sense to chase down optimization in them by making them compiled.

I *very* strongly agree also. I do change init scripts, but even more than this, I see a growing trend for Linux systems to be less friendly to user modification. We are not so smart that we should do this.

Jon.

On Wed, 26.05.10 14:08, Jon Masters (jonathan@jonmasters.org) wrote:

...

I couldn't agree more. They need to be scripts, considering how seldom they actually run it makes even less sense to chase down optimization in them by making them compiled.

I *very* strongly agree also. I do change init scripts, but even more than this, I see a growing trend for Linux systems to be less friendly to user modification. We are not so smart that we should do this.

Well, we won't take anything away from you:

if we add an option to systemd itself, for something that was previously done manually in the init script, then this is just that: an option you can configure as much as you want, simply by editing the .service file.

Lennart

On Thu, May 27, 2010 at 10:10:35AM +0200, Harald Hoyer wrote:

We mainly speek about rc.sysinit, which most of you don't touch, because it would be overwritten the next initscripts update anyway.

I would never touch it with a permanent change, but I definitely make debugging changes to it often enough. That's not down to it being a shell script though; it's because it's a crufty mess.

On Thu, May 27, 2010 at 03:14:54PM +0200, Kevin Kofler wrote:

...

That's not down to it being a shell script though; it's because it's a crufty mess.

… which is exactly why it should be handled inside the init system instead of being that mess of a shell script. :-)

Maybe. That cruft is all there for a reason, and it'll take a lot of work to disentangle the good reasons from the ones that should be left behind. Putting it all in an opaque binary seems _less_ conducive to that rather than more.

Once upon a time, drago01 drago01@gmail.com said:

This does make a lot of sense to me, initscripts being scripts is a major slowdown factor by itself.

But they aren't a major slowdown factor (see the example numbers in this thread).

And, if they were, any init scripts that are a problem could probably be optimized and still be shell scripts (a number of sed/awk/grep calls could probably be rewritten as pure bash for example).

Once upon a time, drago01 drago01@gmail.com said:

On Wed, May 26, 2010 at 4:16 PM, Chris Adams cmadams@hiwaay.net wrote:

...

Once upon a time, drago01 drago01@gmail.com said:

...

This does make a lot of sense to me, initscripts being scripts is a major slowdown factor by itself.

But they aren't a major slowdown factor (see the example numbers in this thread).

They are flawed, simply spawning awk multiple times and measure the time is not a test to compare bash to C.

...

And, if they were, any init scripts that are a problem could probably be optimized and still be shell scripts (a number of sed/awk/grep calls could probably be rewritten as pure bash for example).

Which would be faster than spawing random process but still orders of magnitudes slower than a C program.

Where are your numbers proving this is a measurable impact?

Adam Williamson wrote:

I beg to differ. I've had to create or modify initscripts quite often, either as a sysadmin or a packager. If this is now going to require C coding skills, I'm not going to be able to do it. I don't think it's safe to assume that everyone who needs to write or modify an initscript is going to know C. What about people who write apps that need initscripts in some other language?

What about a compromise? The initscripts are plain text files that get compiled after you edit them.

Solution 1 Example: viinit postgresql Make your changes :wq viinit compiles your script into a binary.

Solution 2: Give Bash JIT.

OTOH, why is this even a sub-topic in this sub-topic of a thread? I'd love to see some numbers from the complainers about scripting being slow. I have a normal Fedora 13 x86_64 system that boots through initscripts in under 10 seconds. Normal services are starting as I have not "tweaked" my service list. Unless Fedora needs a 1 second boot time (hey I wouldn't complain) do we really need to spend time on 100+ email threads and jump through multiple init systems to find that perfect solution? I've read similar claims of salvation when upstart was being marketed to replace SysVinit. "Everyone will switch to native scripts and everything will be better!" Has everyone switched to native upstart scripts? AFAIK - No. Will everyone switch to systemd native scripts? I'm betting - no.

On Wed, May 26, 2010 at 6:07 PM, Adam Williamson awilliam@redhat.com wrote:

On Wed, 2010-05-26 at 12:42 +0200, drago01 wrote:

...

On Wed, May 26, 2010 at 5:02 AM, Casey Dahlin cdahlin@redhat.com wrote:

...

On Tue, May 25, 2010 at 05:45:07PM +0200, Lennart Poettering wrote:

...

On Tue, 25.05.10 10:21, Casey Dahlin (cdahlin@redhat.com) wrote:

[...] 3) Cutting down on the forking by replacing some of the shell scripts... cool   3a) With C code... really?

This does make a lot of sense to me, initscripts being scripts is a major slowdown factor by itself.

It is not like you want to edit the scripts all the time, so there is no reason for them being scripts.

I beg to differ. I've had to create or modify initscripts quite often, either as a sysadmin or a packager.

Again the sysadmin case just implies that something *else* is broken. Well if changing over to C does only get rid of this "disease" it would be enough of a gain. It would force broken apps to be fixed, and let admins edit *configuration* files and not source code.

Why don't people try to configure lets say X by editing its code'? Does this sound wrong to you? If yes than why would initscripts be different?

If this is now going to require C coding skills, I'm not going to be able to do it. I don't think it's safe to assume that everyone who needs to write or modify an initscript is going to know C

Well if you want to write and modify something you have to know the language it is written in, in case you don't it isn't a problem either just ask for help. It is not rocket science or something that would required hundreds of man hours.

What about people who write apps that need initscripts in some other language?

See above.

On Wed, 26.05.10 12:49, Matthew Miller (mattdm@mattdm.org) wrote:

On Wed, May 26, 2010 at 06:39:43PM +0200, drago01 wrote:

...

Again the sysadmin case just implies that something *else* is broken.

Sure. As a distribution, we don't have control over upstream projects and their assumptions for daemon startup, shutdown, status, etc. Sometimes, they want odd things.

...

Well if changing over to C does only get rid of this "disease" it would be enough of a gain. It would force broken apps to be fixed, and let admins edit *configuration* files and not source code.

If you think you can get every open source / free software project to agree on completely consistent behavior, or if you can create a text-format config file for your compiled daemon handler which handles every unanticipated case, well, okay. But it seems unlikely. (And that's not even considering running non-free software, which, while something I try to avoid, is a reasonable real-world use.)

Well, if we cannot get rid of all shell scripts that is fine. If we can get rid of 90% of them a lot is achieved already. And to me it looks much more like > 98% of the scripts that can be removed without ill-effect if systemd is used.

Lennart

Jeremy Sanders wrote:

What rubbish. For instance, we customised an early Fedora to work on a diskless NFS rooted system, mainly by mucking around with init scripts. Being able to easily edit these files made this project work and work quickly. Trying to get upstream to special case our particular setup wouldn't have happened.

You're suggesting hammering shut the insides of linux to stop people playing around and reducing freedom - sounds like you want Fedora to be like the products of other large propitiatory systems.

C code can be changed too, if really needed. But the point is that it should not be necessary to edit code at all to get what's needed.

There is a clear case for having an open and completely configurable system. It's not going to cost 1 sec of bootup either.

I'm not so sure there. The overhead adds up.

Kevin Kofler

On Fri, May 28, 2010 at 2:06 PM, Jon Masters jonathan@jonmasters.org wrote:

Didn't we "decide" that Fedora was intended for more technical users? I don't see many technical users crying out for a hammered shut init system where they feel like they have to go to their auto mechanic to attach the right magic dongle to fix it when it breaks.

I don't think this kind of rhetoric is useful. Once an effort is made to switch to systemd, if *specific* real world examples of things that are reasonable to configure are no longer so, we can treat that as a bug.

Kevin Kofler wrote:

Jon Masters wrote:

...

Didn't we "decide" that Fedora was intended for more technical users? I don't see many technical users crying out for a hammered shut init system where they feel like they have to go to their auto mechanic to attach the right magic dongle to fix it when it breaks.

A really TECHNICAL user has no trouble editing a C program. :-)

On a router? With no gcc installed? And with no disk space to install gcc? And with no network available? (because, you know, I'm fixing the router which is failing to boot properly)

I hope your smiley face means you are joking. :-)

On Sat, 29.05.10 19:27, Roberto Ragusa (mail@robertoragusa.it) wrote:

Kevin Kofler wrote:

...

Jon Masters wrote:

...

Didn't we "decide" that Fedora was intended for more technical users? I don't see many technical users crying out for a hammered shut init system where they feel like they have to go to their auto mechanic to attach the right magic dongle to fix it when it breaks.

A really TECHNICAL user has no trouble editing a C program. :-)

On a router? With no gcc installed? And with no disk space to install gcc? And with no network available? (because, you know, I'm fixing the router which is failing to boot properly)

Guys, nobody wants to take away configuration options. You can edit the .service files too, and readjust things. You can even plug in shell scripts here and there and wherever it suits you.

There are not plans to make configuration of systemd depndent on gcc or gdb. That is completely made up.

Lennart

Kevin Kofler wrote:

Jeremy Sanders wrote:

...

You're suggesting hammering shut the insides of linux to stop people playing around and reducing freedom - sounds like you want Fedora to be like the products of other large propitiatory systems.

C code can be changed too, if really needed. But the point is that it should not be necessary to edit code at all to get what's needed.

Well, I really do not want to flame anyone, but please consider that the guy proposing the change already gave us pulseaudio, which promised the "it will do anything you do now, just easier" feature too. We then discovered that some _trivial_ things where lost: - having multiple independent sound cards - having control of the mixer - having sound with no user logged in ... should I also mention latency, CPU usage, stability,...? Come on, it took a couple of years to have a decently working audio system back.

Now, should I really believe that the new "compiled but it will do anything" init system will be able to help me when ...don't know... I need to change firewalls rules and routing rules in the middle of the init scripts, because I have a multihomed internet connection and remote filesystems and I need the firewall closed and then opened in a way which is dependent on the IP I got from the ISP and I need a "sleep 5" in a proper place because the ADSL modem is still booting its firmware and I want some output printed to /dev/tty1 during boot and I want some filesystems to be mounted with "mount &" because they take time but only contain data not needed to boot and .... ?

This stuff can be trivially done by a competent sysadmin with shell-based init scripts and vi.

Linux must NOT be Windows. Linux must NOT be OS X.

Kevin Kofler wrote:

Roberto Ragusa wrote:

...

I need to change firewalls rules and routing rules in the middle of the init scripts, because I have a multihomed internet connection and remote filesystems and I need the firewall closed and then opened in a way which is dependent on the IP I got from the ISP

Ugh! Are you sure there's no better solution?!

Kevin, this is not an exactly real scenario, but I wanted to say that sometimes it is useful to *hack* something in the middle of the init script for some reason.

I know that no "sleep 5" should be there to wait for the firmware of the ADSL modem and so on... what I'm trying to say is that in some cases you need to do something dirty to work around a bug or to confirm a bug. For example, I had really bad luck with the rt2500 driver having weird crashes with the ordering and relative timing of "modprobe", "ifconfig up", "iwconfig". Understanding what the hell was happening during boot (and only during boot) made me add strange hacks into the scripts so to be able to describe the bug.

On Sat, 29.05.10 19:48, Roberto Ragusa (mail@robertoragusa.it) wrote:

Kevin Kofler wrote:

...

Jeremy Sanders wrote:

...

You're suggesting hammering shut the insides of linux to stop people playing around and reducing freedom - sounds like you want Fedora to be like the products of other large propitiatory systems.

C code can be changed too, if really needed. But the point is that it should not be necessary to edit code at all to get what's needed.

Well, I really do not want to flame anyone, but please consider that the guy proposing the change already gave us pulseaudio, which promised the "it will do anything you do now, just easier" feature too.

Ah, turning this into something personal. Love you too.

We then discovered that some _trivial_ things where lost:

• having multiple independent sound cards
• having control of the mixer
• having sound with no user logged in

... should I also mention latency, CPU usage, stability,...?

You seem to have no idea what you are talking about. But anyway, let's not turn this into a discussion about PA. Don't need another one of those.

Linux must NOT be Windows. Linux must NOT be OS X.

Well I for one think we can learn a lot from the competition. Open your eyes. There's a lot of good stuff in those other OS worlds, particularly in the designs of MacOS X. There's still so much they do better than we do.

Lennart

Roberto Ragusa wrote:

In recent times some stupid (IMHO) ideas have been adopted in Linux just to copy what others do. Just as examples: the control of desktop widgets in KDE4 (functional GUI elements modified by a mouse-over???),

I only know of 2 plasmoids triggering actions on mouse-over: * the folder view's popping up of subfolders. This is not an active action, it's just displaying stuff, I don't see how this is fundamentally different from a tooltip. You don't actually CHANGE anything in the folders or files without a mouse click, it just shows you stuff. * Lancelot's clickless mode. While those are true actions (i.e. they're active), this mode is a deliberate OPTION (i.e. it can be turned off) in a plasmoid which is not shown by default, nor even INSTALLED by default (it's in kdeplasma-addons).

I don't think either is a real problem.

the fast-user-switching approach (the Unix way is to have multiple X servers).

FWIW, KDE/KDM uses the "multiple X servers" approach. There are advantages and drawbacks to either method. Unfortunately, the different approaches mean user switching doesn't work with KDE under GDM nor with GNOME under KDM, which is the only real problem I see in that area (and one of my semi-secret plans is to try to fix this at some point one way or the other, but I'm way too busy).

Kevin Kofler

Dave Airlie wrote:

So does gdm use multiple X servers, I wasn't aware there was any other way.

So what does it do exactly? Spawn a new X server on the same vterm? Or on a different vterm? What KDM does is to spawn a new X server on a different vterm.

Kevin Kofler

Kevin Kofler wrote:

Roberto Ragusa wrote:

...

In recent times some stupid (IMHO) ideas have been adopted in Linux just to copy what others do. Just as examples: the control of desktop widgets in KDE4 (functional GUI elements modified by a mouse-over???),

I only know of 2 plasmoids triggering actions on mouse-over:

Sorry, I didn't manage to explain me well. I'm referring to the vertical bar which popups at the left of right of _every_ plasmoid. The thing with the close button and which you can drag around to move the plasmoid. It is basically a window title bar done vertically. The bad part is that it popups on mouse movement and creates active elements (buttons) just under your mouse by guessing that you want them. If you have a crowded screen with overlapping windows and plasmoids, you get this popups in your way when you just want to click on a near window. Tooltips in general have this problem (uncontrollable creation of obstructions), but in this case the tooltip even contains clickable elements, so you have to be careful to not click them by error. Be_careful => be_slow_and_think => bad_GUI_concept.

The folder view popping up you cited is another bad example (but I was not thinking about that). You say it's only visualization, but it changes the context, in the sense that now my dropping the icon has a new meaning (something randomly changed under my pointer). If the change happens just an instant before I'm releasing the button, my jpg will not go into Friends, but into Friends/DiscardedPhotos. So what should I do? Be careful to not park my pointer on any active area while I'm thinking about where to place my jpg; and every folder is now an active area; my desktop has turned into minesweeper. :-) I think the original idea from Apple was to use the spacebar to enter the directory. That is perfectly acceptable.

In my opinion every "automatic" element (popups and tooltips) should only be allowed to show inactive things (better yet if the obstruction is not complete, for example, by using partial opacity and, if we want to be smart, increasing the opacity when the mouse is near, for legibility). In no case this thing must have clickable items. In no case this thing must intercept clicks: if I click on it the click should go to the elements it is covering (that is where transparency helps), because those elements have earned my attention for some time, before the appearance of the little intruder. The mouse pointer should be blended _below_ the tooltip. Tooltips should just be semitransparent post-it notes attached to my monitor.

On Tue, Jun 01, 2010 at 09:58:37PM +0200, Roberto Ragusa wrote:

Lennart Poettering wrote:

...

On Sat, 29.05.10 19:48, Roberto Ragusa (mail@robertoragusa.it) wrote:

...

Well, I really do not want to flame anyone, but please consider that the guy proposing the change already gave us pulseaudio, which promised the "it will do anything you do now, just easier" feature too.

Ah, turning this into something personal. Love you too.

This is not a personal attack and I want to explain. You will agree with me that pulseaudio caused a lot of complaints. (we do not discuss if motivated or unmotivated, here) That bad reputation unavoidably leads to weaker trust in your promised guarantees.

The most painful parts of PA were mainly the underestimation of both "peculiar" use cases and impact of issues in related software (e.g. bugged alsa drivers). It is only natural to be worried about the same things (like lack of customizability) for this new init system.

You *are* in a worse position than someone else when proposing a revolution in some critical part of the system. That's no personal offense. On the positive side you are now well aware of the risks you face, so you will probably play it better than someone else.

Regardless of how this applies in this case, don't forget that digging your way out of a big mess is a different way to build trust. Sooner or later we all screw up, knowing how a person handles screwups can be quite valuable.

I hope your new init system will be a great success and help you clean your name from the PA mess. I honestly hope so.

...

...

We then discovered that some _trivial_ things where lost:

• having multiple independent sound cards
• having control of the mixer
• having sound with no user logged in

... should I also mention latency, CPU usage, stability,...?

You seem to have no idea what you are talking about. But anyway, let's not turn this into a discussion about PA. Don't need another one of those.

I've been personally been burned by these issues, to the point of going 90% of the way of removing PA from the system (I'm currently running the unrecommended system-wide instance, I manually restart it in some cases, I use often pasuspender and for some things I know I have to turn if off completely).

But I'm still on F10 and I read that pulseaudio has become better. Maybe I will be positively surprised when upgrading to F13.

for better or worse, released Fedora versions, especially <= N-1 don't tend to see a lot of updates. Developer manpower seems to shift quite early to rawhide and/or upstream and there's a few packages that only see token efforts once the next release is out.

Staying on a particular release is unfortunately not the solution to get real fixes into your system.

...

...

Linux must NOT be Windows. Linux must NOT be OS X.

Well I for one think we can learn a lot from the competition. Open your eyes. There's a lot of good stuff in those other OS worlds, particularly in the designs of MacOS X. There's still so much they do better than we do.

With "must NOT" I do not mean "we have to be far", I mean "we are not obligated to be near".

In recent times some stupid (IMHO) ideas have been adopted in Linux just to copy what others do. Just as examples: the control of desktop widgets in KDE4 (functional GUI elements modified by a mouse-over???), the fast-user-switching approach (the Unix way is to have multiple X servers).

For every "stupid (IMHO) idea" there's people who disagree with the "stupid", "IMHO" and sometimes even the "idea" part. Long term, all this is evolutionary and some ideas die off, others prevail. Unlike in nature, it is steered evolution though and you can help steer away from ideas you deem bad by talking to the upstream developers. Venting on a distribution list won't do a lot of difference.

Cheers, Peter

On Wed, 26.05.10 09:07, Adam Williamson (awilliam@redhat.com) wrote:

On Wed, 2010-05-26 at 12:42 +0200, drago01 wrote:

...

On Wed, May 26, 2010 at 5:02 AM, Casey Dahlin cdahlin@redhat.com wrote:

...

On Tue, May 25, 2010 at 05:45:07PM +0200, Lennart Poettering wrote:

...

On Tue, 25.05.10 10:21, Casey Dahlin (cdahlin@redhat.com) wrote:

[...] 3) Cutting down on the forking by replacing some of the shell scripts... cool 3a) With C code... really?

This does make a lot of sense to me, initscripts being scripts is a major slowdown factor by itself.

It is not like you want to edit the scripts all the time, so there is no reason for them being scripts.

I beg to differ. I've had to create or modify initscripts quite often, either as a sysadmin or a packager. If this is now going to require C coding skills, I'm not going to be able to do it. I don't think it's safe to assume that everyone who needs to write or modify an initscript is going to know C. What about people who write apps that need initscripts in some other language?

THERE ARE NO PLANS TO SHIP COMPILED INIT SCRIPTS OR ANYTHING LIKE THAT!

The plan is to reduce what is currenlty done in files like /etc/init.d/messagebus to files like http://0pointer.de/public/dbus.service.

And those files you can edit just fine, and reconfigure.

Lennart

On Wed, 26.05.10 12:57, Colin Walters (walters@verbum.org) wrote:

On Wed, May 26, 2010 at 12:50 PM, Lennart Poettering mzerqung@0pointer.de wrote:

...

http://0pointer.de/public/dbus.service.

Note the ExecStartPre here, like most daemons, is conceptually busted. There's no reason we shouldn't lay that file down once when the OS is installed, and not check it every boot. Or alternatively, just move the uuid generation into the daemon itself.

I think it kinda makes sense to to leave the uuid generating for the first boot, simply to make it easy to generate a hdd image during installation which can tzhen be booted from various machines, where each of them will then get its own uuid.

But yes, I too think that dbus itself should just create that uuid on startup.

Lennart

On Wed, 26.05.10 13:08, Colin Walters (walters@verbum.org) wrote:

...

...

I beg to differ. I've had to create or modify initscripts quite often, either as a sysadmin or a packager. If this is now going to require C coding skills, I'm not going to be able to do it. I don't think it's safe to assume that everyone who needs to write or modify an initscript is going to know C. What about people who write apps that need initscripts in some other language?

THERE ARE NO PLANS TO SHIP COMPILED INIT SCRIPTS OR ANYTHING LIKE THAT!

The plan is to reduce what is currenlty done in files like /etc/init.d/messagebus to files like http://0pointer.de/public/dbus.service.

Also:

...

Description=D-Bus System Bus

This seems unnecessary. Can we default to the name of the script? If this isn't translated, I don't see how it's more interesting than just "dbus".

Later on this shall be translatable, the same way as the Comment field of .desktop files.

And if you omit this option we'll already default to the unit name, the way you suggsted.

...

Requires=basic.target sockets.target dbus.socket After=basic.target sockets.target dbus.socket

What does this goop mean and why is it necessary?

basic.target encapsulates the early boot process (kinda the same stuff rc.sysinit currently does). The Requires= make sure that this is pulled in by dbus.service. The After= makes sure that it has finished before we fork dbus.

And ignore the sockets.target/dbus.socket stuff, that I had in there for debugging only. It's not necessary, and completely redundant. It will not be in the final version of this .service file.

The reason why basic.target is explicitly mentioned in the .service file is that we want to handle early boot and normal daemon start-up with the same .service files. This is different from launchd for example, where early boot is basically a shell script handled completely independently and differently from normal services. I think we need more flexibility there, simply because we have to cover way more than Apple has to when it comes to plugging something into early boot. The price for that is that normal daemons that are started in "late boot", have to add these Requires= and After= lines for basic.target.

This felxible design has various other advantages too, for example we can blur the destinction between early and late boot a bit and remove the synchronization point that seperates them, for selected services.

Lennart

On Tue, 01.06.10 15:25, Bill Nottingham (notting@redhat.com) wrote:

Lennart Poettering (mzerqung@0pointer.de) said:

...

...

...

Requires=basic.target sockets.target dbus.socket After=basic.target sockets.target dbus.socket

What does this goop mean and why is it necessary?

basic.target encapsulates the early boot process (kinda the same stuff rc.sysinit currently does). The Requires= make sure that this is pulled in by dbus.service. The After= makes sure that it has finished before we fork dbus.

How are these evaluated differently that you'd need to list both 'Requires' and 'After', as opposed to just one? (I would think Requires would imply After.)

Well, in systemd ordering dependencies are independent of requirement independencies.

Explanation:

A requires B means that when A is started B is started too, at the same time.

A after B means that when A is started and B is started as well, then B is started first and only when that finished A is started too.

Now, if you combine both, then you get A requires B + A after B, which means that when A is started we first start B and when that is finished wwe load A too.

There are use-cases for all three cases listed above, so we thought we cover this in just two dependency types, instead of three. (And in fact actually, since we have "requires" and "requisite" too, this saves us quite a few additional types)

Now, since base.target is pulled in by the default boot target anyway, one could argue that in this case an After= would suffice, and the requires= is redundant. But uh, on a logical level base.target is actually required, so we should list both elements I think.

Lennart

On Wed, 26.05.10 12:27, Adam Williamson (awilliam@redhat.com) wrote:

...

The plan is to reduce what is currenlty done in files like /etc/init.d/messagebus to files like http://0pointer.de/public/dbus.service.

And those files you can edit just fine, and reconfigure.

There's no need to yell, you only explained that a couple of minutes ago, before I wrote the above mail. From your blog post, it sounded very much like compiled initscripts were what you were proposing.

This seems perfectly reasonable, but at the same time, when you explain it this way, it doesn't seem like anything that needs a new init system to happen. You can move stuff into the daemons themselves no matter what init system you're using, and it seems like upstart would be open to moving appropriate things into the init system. That does seem like a perfectly sensible thing to do, though, so I hope it happens, whatever init system we wind up with.

Well. Sure. Upstart can copy every feature we have. But while all of this is still in the future for Upstart we already have implemented a fair bit of it in systemd.

Lennart

On Wed, 26.05.10 13:37, Przemek Klosowski (przemek.klosowski@nist.gov) wrote:

On 05/26/2010 12:07 PM, Adam Williamson wrote:

...

...

It is not like you want to edit the scripts all the time, so there is no reason for them being scripts.

I beg to differ. I've had to create or modify initscripts quite often, either as a sysadmin or a packager. If this is now going to require C coding skills, I'm not going to be able to do it. I don't think it's safe to assume that everyone who needs to write or modify an initscript is going to know C. What about people who write apps that need initscripts in some other language?

It could work out if systemd provided access to a system() equivalent which could then execute an arbitrary script.

You can easily hook shell scripts into .service files via stuff like ExecStartPre=/some/shell/script which will then be executed before the actual daemon is forked off.

I think one good argument for redoing initscripts is that they are so repetitive: most of the content is fairly standard: initialization, argument parsing, case $1 in start) stop), etc etc. ; stuff that might as well be done in the common framework.

That is true. Most scripts are 1:1 copies of the init script template of our guidelines.

Lennart

On Tue, 25.05.10 23:02, Casey Dahlin (cdahlin@redhat.com) wrote:

...

Why do you say "cgroups are a dead end"? Sure, Scott claims that, but uh, it's not the only place where he is simply wrong and his claims baseless. In fact it works really well, and is one of the strong points in systemd. I simply see no alternative for it. The points Scott raised kinda showed that he never really played around with them.

Please read up on cgroups, before you just dismiss technology like that as "dead end".

I did. When upstart was about to use them. 2 years ago. We chucked them by the following LPC.

Who is "we"?

The problem we've found is that cgroups are too aggressive. They don't have a notion of sessions and count too much as being part of your service, so you end up with your screen session being counted as part of gdm.

Well, how exactly you set up the groups is up to you, but the way we do it in systemd is stick every service in a seperate cgroup, plus every user in a seperate one, too. Some examples:

/systemd-1/avahi.service /systemd-1/getty@tty1.service /systemd-1/gdm.service /systemd-1/apache.service /user/lennart /user/cjd

The per-user cgroups are controlled via a PAM module. That way there's finally a nice way how we can reliably clean up behind a user when he logs out: we just kill his complete cgroup and he's gone.

In addition we can easily set all kinds of cgroup-based resource enforcement to these groups, i.e. force user "lennart" to CPU 1, and say that apache and all the cgi scripts it creates can only get up to 20% CPU. And avahi-daemon could be forced to get a quarter of the available RAM at max -- with all its processes summed up, regardless how often it might fork.

And the whole thing is even recursive. If you run a per-user systemd as user lennart, you will end up with a sub-hierarchy like this:

/user/lennart/systemd-4711/dbus-daemon.service /user/lennart/systemd-4711/dconfd.service

And so on.

And the nice thing is that these cgroups are shown when you do "ps -eo cgroup,...". You can always figure out from "ps" to which service a process belongs, even if if it fork()ed a gazillions of times. And all the keeping track is done by the kernel, basically for free. No involvement from userspace.

This is why setsid was added to the netlink connector.

Well, this is just flawed, on so many levels, that it hurts. Asynchronously trying to follow how daemons fork/exit is just inherently broken because they can do an exponential amount of forks in the time you can (realisticly) collect them linearly only. Also, if a daemon forks too often, netlink willl drop your messages, which makes an easy-peasy way for processes to escape your supervision, using only inprivileged operations. You have constant userspace wakeups. Everything you apply on the processes is done asynchronously and hence is racy (killing, renice, yadda yadda). The problem is simply that your grip on the processes can never work, because you are scheduled at the same priority as the daemons you supervise and you get all notifications asynchronously. You *really* want to leave process tracking to the kernel, and not try to emulate that in userspace. Everything else is just unsafe and hence a joke in the context of process baby sitting. It's like if you'd employ a babysitter and give the kid a bike it can escape on while chaining your babysitter to the couch.

So, it's just not safe, processes can *easily* escape your supervision. On top of that it is just ugly. And finally, you cannot nicely show the service something belongs to in "ps" the way cgroups give it to you for free. Then, you cannot set cgroup resource enforcement for your services, since you simply have no cgroups. And no nice interfacing with the other libcgroup tools either.

Meh, and this lists gets on like this.

People should not use cn_proc. It's evil. And if you are using for anything except logging you are doomed. And even for logging it isnt really useful.

1. Socket activation. Part of Upstart's roadmap. Would happen sooner if you

cared to submit the patch. We don't think its good enough by itself, hence the rest of Upstart, but a socket activation subsystem that could reach as far as systemd and even work standalone in settings where systemd can work is perfectly within Upstart's scope. I'd be happy to firm up the design details with you if you wanted to contribute patches.

Well, for once, it would be nice to judge things due to actually existign features, not of big plans nobody is working on as you apparently admit outright.

And then, the socket activtion is nice for various reasons, and lazy-loading is just one of them. The bigger advantage is that it does automatic dependency handling -- which of course is nothign that really fits into upstart's design, since that is based on "events" not dependencies -- events are just broken, as I might note. And adding dependency would turn around upstart's design, making it a completely different beast.

I mean, you called socket activation "xinetd-style activation" in the earlier mail of yours -- that is just completely besides the point, because this all is not so much about doing on-demand starting of internet services. systemd-style activation is about parallelizing startup of (mostly) local services and making fully written dependencies obsolete. And that's what is so magic about it. And an init systemd should be designed around this at its core! And systemd is.

2. Bus activation. Missed opportunity here to actually become the launchpoint

for activated services. I won't criticize that too much though, as its usefulness is largely dependent on kernelspace DBus, which I've been trying to bludgeon Marcel Holtmann into turning over to the public for a year now.

Not sure what this has to do with kernel space D-Bus, except that that is practically dead.

If people want to reinvestigate the kernel/dbus issue they should not focus on an AF_DBUS, but instead just use netlink and use BSD socket filters for minimizing wakeups, plus come up with something inspired by iptables/ebtables to filter netlink traffic, for the permission problems. But that's a completely different story.

3. Cutting down on the forking by replacing some of the shell scripts... cool 3a) With C code... really?

Yes, really. MacOS could do it, and so can we. Its not that hard. And as I my add here I already hacked up a big part of it now for the servcies we start by default.

4. Process environment control. No complaints, and also nothing Upstart doesn't

want to do.

Well, have you seen the functionality we provide thre, and the limited stuff upstart has there?

And what about the other features? the automounter, a full blown dependency system, depedencies between mounts, automounts, devices, sockets, services, timers, paths, swaps? The snapshot system? The fact that we take advantage of the LSB stuff right now, to paralleize the boot, without changing a single file? All the process control stuff, i.e. the syslog/kmsg hookup of stdout/stderr of every process we spawn? The FS namespace stuff we do? The TTY stuff we do? the IO/CPU scheduler tricks we do? The capability stuff we do? The CPU affinity stuff we do? the full integration of /etc/fstab and friends? The transaction logic? The crosslinking between systemd and syslog/abrt? the almost complete dbus coverage? all the small btis we already took out of the shell boot rpocess and already moved into proper C code in system (and some of it into udevd)? the fact that systemd works for normal users/session supervisor too? The fact that we support different boot targets? The fact we have a UI? the sane copyright? The fact that we don't use bloody bzr? And so on and so forth, i could repeat the whole blog story here. Please just read that instead.

...

We did systemd because we thought that technically Upstart is fundamentally flawed and misses out on so many opportunities.

And we think the same of systemd.

We?

OK, I am listening. What's flawed about systemd? I wrote very detailed explanation why Upstart is just wrong in its core design. Please be so kind and be more specific if you claim the same abotu systemd, because otherwise all you are doing is spreading FUD.

And mail it to upstart-devel-list. I'm being a bit hyperbolic, but actually not much. That would have been a valid move.

Please read my blog story. Thank you. It actually answers you that very question:

"Why didn't you just add this to Upstart, why did you invent something new?

Well, the point of the part about Upstart above was to show that the core design of Upstart is flawed, in our opinion. Starting completely from scratch suggests itself if the existing solution appears flawed in its core. However, note that we took a lot of inspiration from Upstart's code-base otherwise."

If there is a valid reason for starting anew if something exists already, then it is that the basic design of the existing system is just flawed. And hence we started anew.

You should really come work with us. We're fun guys.

Man, you are suggesting I didn't talk to the Upstart folks. That's a ridiculous claim, and quite insulting that you imply that. Kay and I have had long discussions with Scott, during the last three years or so, at various confernces. And as far as I can see Upstart is Scott and Scott is Upstart, and the bzr repo commits underline that. And I am sorry if weren't part of those discussions.

You should really come work with us. We're even funnier guys.

Lennart

On Wed, May 26, 2010 at 02:01:35PM +0200, Tomasz Torcz wrote:

On Wed, May 26, 2010 at 01:26:48PM +0200, Lennart Poettering wrote:

...

...

The problem we've found is that cgroups are too aggressive. They don't have a notion of sessions and count too much as being part of your service, so you end up with your screen session being counted as part of gdm.

The per-user cgroups are controlled via a PAM module. That way there's finally a nice way how we can reliably clean up behind a user when he logs out: we just kill his complete cgroup and he's gone.

You are avoiding the question: what about screen sessions? Whole point of screen is to stay after logout, and by killing cgroup you nullify it.

Precisely my point.

--CJD

On Wed, May 26, 2010 at 5:55 AM, Lennart Poettering mzerqung@0pointer.de wrote:

In systemd you can choose individually for each unit whether you want to allow it to continue run processes on shut down, whether you want the main process killed, the process group to be killed or the cgroup to be killed.

So in other words... we are going to have a knockdown dragout over default unit config for a systemd enabled gdm and for anything that allows shell logins....just not today.

-jef

On Wed, 26.05.10 08:19, Jeff Spaleta (jspaleta@gmail.com) wrote:

On Wed, May 26, 2010 at 5:55 AM, Lennart Poettering mzerqung@0pointer.de wrote:

...

In systemd you can choose individually for each unit whether you want to allow it to continue run processes on shut down, whether you want the main process killed, the process group to be killed or the cgroup to be killed.

So in other words... we are going to have a knockdown dragout over default unit config for a systemd enabled gdm and for anything that allows shell logins....just not today.

I don't think the will be much fuss about this, as I personally think that we should continue with the current behaviour here, and support screen out-of-the-box.

Lennart

On Wed, 26.05.10 09:01, Jeff Spaleta (jspaleta@gmail.com) wrote:

On Wed, May 26, 2010 at 5:55 AM, Lennart Poettering mzerqung@0pointer.de wrote:

...

Well, that depends on configuration.

...

In systemd you can choose individually for each unit whether you want to allow it to continue run processes on shut down, whether you want the main process killed, the process group to be killed or the cgroup to be killed.

Do you have a service file example yet in systemd git that can be used to get an understanding of the various configuration options which determine what gets killed and what doesn't?

Nope, not really.

But it is as easy as this: when we stop a service the "KillMode=" option controls whether and how any processes remaining after the "ExecStop=" command (if there is such a command, and if there are any processes left) is killed.

KillMode=control-group → the entire cgroup is shot down KillMode=process-group → only the process group of the process we forked is shot down KillMode=process → only the process we forked is shot down KillMode=none → nothing is shot down

Lennart

On Tue, 01.06.10 01:56, Mike Fedyk (mfedyk@mikefedyk.com) wrote:

...

KillMode=control-group → the entire cgroup is shot down KillMode=process-group → only the process group of the process we forked is shot down KillMode=process → only the process we forked is shot down KillMode=none → nothing is shot down

And what is the default?

KillMode=control-group for native services and KillMode=process-group for SysV services.

Lennart

What type of cgroup are you using?  Does it impede the use of lxc for containers?  Ie, the cgroup type that systemd needs to be able to be nested inside a container in a whole OS virtualization (think VPS / Virtual Private Server where each VPS has root in its container).

systemd uses a named hierarchy without mounting any other subsystem. So no, it does not impede the use of lxc for containers.

Dhaval

On Wed, May 26, 2010 at 01:26:48PM +0200, Lennart Poettering wrote:

On Tue, 25.05.10 23:02, Casey Dahlin (cdahlin@redhat.com) wrote:

...

...

Why do you say "cgroups are a dead end"? Sure, Scott claims that, but uh, it's not the only place where he is simply wrong and his claims baseless. In fact it works really well, and is one of the strong points in systemd. I simply see no alternative for it. The points Scott raised kinda showed that he never really played around with them.

Please read up on cgroups, before you just dismiss technology like that as "dead end".

I did. When upstart was about to use them. 2 years ago. We chucked them by the following LPC.

Who is "we"?

Upstart has about 1.7 developers. The 1 is Scott, myself and Johann Kiviniemi fight over the .7.

...

The problem we've found is that cgroups are too aggressive. They don't have a notion of sessions and count too much as being part of your service, so you end up with your screen session being counted as part of gdm.

Well, how exactly you set up the groups is up to you, but the way we do it in systemd is stick every service in a seperate cgroup, plus every user in a seperate one, too. Some examples:

<snip>

And so on.

Someone else already pointed out the issues here.

And the nice thing is that these cgroups are shown when you do "ps -eo cgroup,...". You can always figure out from "ps" to which service a process belongs, even if if it fork()ed a gazillions of times. And all the keeping track is done by the kernel, basically for free. No involvement from userspace.

...

This is why setsid was added to the netlink connector.

Well, this is just flawed, on so many levels, that it

<snip>

enforcement for your services, since you simply have no cgroups. And no nice interfacing with the other libcgroup tools either.

There's no reason Upstart couldn't offer cgroups for resource limits, its just passed them up as a process supervision mechanism. And they are broken for the screen case, though perhaps that could be fixed in the kernel.

Meh, and this lists gets on like this.

People should not use cn_proc. It's evil. And if you are using for anything except logging you are doomed. And even for logging it isnt really useful.

...

1. Socket activation. Part of Upstart's roadmap. Would happen sooner if you

cared to submit the patch. We don't think its good enough by itself, hence the rest of Upstart, but a socket activation subsystem that could reach as far as systemd and even work standalone in settings where systemd can work is perfectly within Upstart's scope. I'd be happy to firm up the design details with you if you wanted to contribute patches.

Well, for once, it would be nice to judge things due to actually existign features, not of big plans nobody is working on as you apparently admit outright.

You worked on it, and finished it. You just didn't do it within Upstart. The manpower was there it was just misapplied.

And then, the socket activtion is nice for various reasons, and lazy-loading is just one of them. The bigger advantage is that it does automatic dependency handling -- which of course is nothign that really fits into upstart's design, since that is based on "events" not dependencies -- events are just broken, as I might note. And adding dependency would turn around upstart's design, making it a completely different beast.

Event-driven in Upstart's case just talks about the internal operation of Upstart. Its rather exposed in current iterations but its just a design philosophy. You can implement "dependencies" within it, though in the past Upstart has been hesitant with the term because it tends to imply a dependency-solving algorithm, where as Upstart provides its dependencies reactively, in response to change.

I mean, you called socket activation "xinetd-style activation" in the earlier mail of yours -- that is just completely besides the point, because this all is not so much about doing on-demand starting of internet services. systemd-style activation is about parallelizing startup of (mostly) local services and making fully written dependencies obsolete. And that's what is so magic about it. And an init systemd should be designed around this at its core! And systemd is.

Yes. I get it.

...

2. Bus activation. Missed opportunity here to actually become the launchpoint

for activated services. I won't criticize that too much though, as its usefulness is largely dependent on kernelspace DBus, which I've been trying to bludgeon Marcel Holtmann into turning over to the public for a year now.

Not sure what this has to do with kernel space D-Bus, except that that is practically dead.

If people want to reinvestigate the kernel/dbus issue they should not focus on an AF_DBUS, but instead just use netlink and use BSD socket filters for minimizing wakeups, plus come up with something inspired by iptables/ebtables to filter netlink traffic, for the permission problems. But that's a completely different story.

...

3. Cutting down on the forking by replacing some of the shell scripts... cool 3a) With C code... really?

Yes, really. MacOS could do it, and so can we. Its not that hard. And as I my add here I already hacked up a big part of it now for the servcies we start by default.

Good for Apple? Maybe now they can design a better kernel? This is linux. We configure our systems here.

...

4. Process environment control. No complaints, and also nothing Upstart doesn't

want to do.

Well, have you seen the functionality we provide thre, and the limited stuff upstart has there?

<snip>

supervisor too? The fact that we support different boot targets? The fact we have a UI? the sane copyright? The fact that we don't use bloody bzr? And so on and so forth, i could repeat the whole blog story here. Please just read that instead.

Again, could have been Upstart patches (except for the last couple).

...

...

We did systemd because we thought that technically Upstart is fundamentally flawed and misses out on so many opportunities.

And we think the same of systemd.

We?

OK, I am listening. What's flawed about systemd? I wrote very detailed explanation why Upstart is just wrong in its core design. Please be so kind and be more specific if you claim the same abotu systemd, because otherwise all you are doing is spreading FUD.

Apart from the issue of circular dependencies, which I know you've already had explained to you, I'd like to see how you intend to manage, say Veritas clustering products. Like it or not there are systems out there running software that's unstable, not open source, written by people who don't like you, will /never/ take your socket passing patch and would probably find clever ways to break or do it wrong if they did (yes, because of a 10-line patch).

Upstart doesn't ask anything of the userspace it intends to manage, and that userspace is a harsher place than you seem to realize.

...

And mail it to upstart-devel-list. I'm being a bit hyperbolic, but actually not much. That would have been a valid move.

Please read my blog story. Thank you. It actually answers you that very question:

"Why didn't you just add this to Upstart, why did you invent something new?

Well, the point of the part about Upstart above was to show that the
core design of Upstart is flawed, in our opinion. Starting
completely from scratch suggests itself if the existing solution
appears flawed in its core. However, note that we took a lot of
inspiration from Upstart's code-base otherwise."

If there is a valid reason for starting anew if something exists already, then it is that the basic design of the existing system is just flawed. And hence we started anew.

...

You should really come work with us. We're fun guys.

Man, you are suggesting I didn't talk to the Upstart folks. That's a ridiculous claim, and quite insulting that you imply that. Kay and I have had long discussions with Scott, during the last three years or so, at various confernces. And as far as I can see Upstart is Scott and Scott is Upstart, and the bzr repo commits underline that. And I am sorry if weren't part of those discussions.

See Scott's reply a few mails down.

--CJD