On Fri, Apr 13, 2018 at 9:39 AM, Peter Robinson pbrobinson@gmail.com wrote:

...

As some of you may know, one of the upstream projects I work with is the Snappy/Snapcraft system.

Recently, the Snappy system gained full support for having alternative base/runtimes used for the basis of snaps[1]. Given the design of the Snappy system, it makes it very easy for us to provide a model in which people can build applications and services using Fedora in a manner where it's available in a cross-distribution fashion. One of the key advantages here is that we do not need to rebuild RPMs to retrofit them for the snap model, they can be directly sourced and installed as components to build a snap.

However, to be able to do this, Fedora needs to provide what is known as a "base snap". The base snap provides the foundation for applications and devices to be built using Fedora software with snaps. Historically, the only "base" supported for general application use has been the Ubuntu one shipped by Canonical, but snapd recently learned how to use different "base" runtimes for different snaps.

I believe we have an opportunity to be a great enabler for offering the latest and greatest technologies for developers to consume for their software if they want to distribute it as a snap. While the Fedora Workstation WG has been primarily focused on Flatpaks for GUI applications, snaps are useful for more than desktop applications. Indeed, some of the most interesting uses are involved with server and embedded systems software.

You can also use other container formats in the Server and Embedded space like OCI or docker style containers.

Yes, but those are centered around fully separating the application from the host, rather than offering flexible points of integration between the application and the OS, similar to how Flatpaks do for desktop applications. For example, while it is possible to make OCI-style containers run "like" services, they typically don't behave that way. The base runtime must be "booted up" and then the service inside can be executed. It's opaque from the host, appearing as a single container thing, rather than the service inside. In contrast, snaps appear and function more or less like regular software packaged via RPMs do. They interact with system services directly (through the mediation provided by snap-confine), and their processes are fully transparent and show up normally in standard process monitoring tools.

In this model, you get the benefits of independent life cycling while still being able to do integration with the host system without bending over backwards with weird hacks to connect the two OS environments.

...

I've worked closely with the upstream developers (of which Zygmunt Krynicki is the main point of contact for Fedora with the Snappy team and is CC'd to this email) to ensure that this functionality is available to us in a way that allows us to leverage our software as effectively as possible. In addition, I've built a relatively trivial application[2] to build proof of concept core base snaps with Fedora, Mageia, and openSUSE packages.

At least initially, some of these first application snaps will be produced in a rather simple manner as we would work with upstream on figuring out how to adapt Snapcraft to handle alternative bases. My eventual goal is to be able to produce a Modular Fedora variant that works off snaps, using our software packaged as RPMs as inputs, as a variation of the modularity work going on in Fedora as a whole. This work would be more or less similar to the efforts for producing the Cloud Base images that are used in cloud providers and for container systems to use (e.g. Docker/OpenShift/Kubernetes).

I discussed this with Stephen Gallagher, and he seemed amenable to the idea of the Server WG producing the base snap as a deliverable and publishing it to the snap store so that it can be used to build applications. He suggested that I bring this to broader WG mailing list to solicit some feedback on this.

So what do you all think?

Who's going to do the engineering needed to be able to build the base snap in the Fedora infrastructure like all the other Fedora produced artifacts?

I've already started on this particular effort, but I hope to leverage the work done for the Fedora Cloud Base images to simplify things on the whole. The base snap is a subset of what we ship for Fedora container base images as well, as the goal is to provide enough of Fedora to function as a base runtime environment for Fedora-based snaps.

Do you still need to disable selinux or set it to permissive (I don't remember what the exact requirement was there)?

That hasn't been necessary for the last two years, since I contributed and help maintain an SELinux module for snapd.

On Thu, May 10, 2018 at 01:44:13PM +0000, Stephen Gallagher wrote:

The proposal will be considered accepted as long as it receives three +1 votes and zero -1 votes within the next week.

I am not a voting member of the SIG, but as FPL I support this proposal.

On Thu, May 10, 2018 at 6:08 PM Peter Robinson pbrobinson@gmail.com wrote:

...

...

As some of you may know, one of the upstream projects I work with is the Snappy/Snapcraft system.

Recently, the Snappy system gained full support for having alternative base/runtimes used for the basis of snaps[1]. Given the design of the Snappy system, it makes it very easy for us to provide a model in which people can build applications and services using Fedora in a manner where it's available in a cross-distribution fashion. One of the key advantages here is that we do not need to rebuild RPMs to retrofit them for the snap model, they can be directly sourced and installed as components to build a snap.

However, to be able to do this, Fedora needs to provide what is known as a "base snap". The base snap provides the foundation for applications and devices to be built using Fedora software with snaps. Historically, the only "base" supported for general application use has been the Ubuntu one shipped by Canonical, but snapd recently learned how to use different "base" runtimes for different snaps.

I believe we have an opportunity to be a great enabler for offering the latest and greatest technologies for developers to consume for their software if they want to distribute it as a snap. While the Fedora Workstation WG has been primarily focused on Flatpaks for GUI applications, snaps are useful for more than desktop applications. Indeed, some of the most interesting uses are involved with server and embedded systems software.

I've worked closely with the upstream developers (of which Zygmunt Krynicki is the main point of contact for Fedora with the Snappy team and is CC'd to this email) to ensure that this functionality is available to us in a way that allows us to leverage our software as effectively as possible. In addition, I've built a relatively trivial application[2] to build proof of concept core base snaps with Fedora, Mageia, and openSUSE packages.

At least initially, some of these first application snaps will be produced in a rather simple manner as we would work with upstream on figuring out how to adapt Snapcraft to handle alternative bases. My eventual goal is to be able to produce a Modular Fedora variant that works off snaps, using our software packaged as RPMs as inputs, as a variation of the modularity work going on in Fedora as a whole. This work would be more or less similar to the efforts for producing the Cloud Base images that are used in cloud providers and for container systems to use (e.g. Docker/OpenShift/Kubernetes).

I discussed this with Stephen Gallagher, and he seemed amenable to the idea of the Server WG producing the base snap as a deliverable and publishing it to the snap store so that it can be used to build applications. He suggested that I bring this to broader WG mailing list to solicit some feedback on this.

So what do you all think?

Best regards, Neal

I had a conversation with Neal and Zygmunt this morning in

#fedora-server.

...

We went over the necessary steps for creating a base snap and such and

I'm

...

pretty confident that they have a good handle on it.

What they would like to do (and I support this) is get the

proof-of-concept

...

base snap creation scripts cleaned up and added to the standard Fedora compose, under the Fedora Server umbrella. I'm CCing Mohan Boddu on this message to get him involved. Neal and Zygmunt are volunteering to do

most of

...

the work, with assistance provided where needed on integrating with the compose process.

So I have a problem with the fact that you appear to be answering for rel-eng (yet again) and appearing to commit them to doing something that hasn't going through the standard Change process.

Where is the the Fedora change? Where is the assessment of the changes that are needed here for FESCo to approve?

Peter, the Change Process requires rel-eng signoff to be submitted to FESCo. So this is the part where they approach rel-eng and ask for that, assuming Server SIG gives its blessing to the project. If rel-eng says "no", there's no point to filing the Change.

With my old rel-eng hat on the needed integration with the compose process would be:

• ability to build the artifacts in koji, I suspect this will be image

build via imagefactry with appropriate reporting of the contents into PDC

• pungi so that the compose process can build the above images
• bodhi integration so as to be able to compose updates when things

like CVEs occur

Yes, these came up in a conversation on #fedora-releng yesterday. Neal and Zygmunt are planning to provide the necessary patches to do this, with some guidance from rel-eng.

The above it needed so that it can integrated in the compose process, but also so we know what's in them so we know if they need to be regenerated in case of CVEs etc.

Probably also need some ability to push the base image to what ever repository snaps have so people that use them can choose them to build their apps on (else what is the point?)

Yes, though this will be secondary to producing them. I think the short-term plan was to do the uploads manually and work on having a service for it a little further down the road, but I'll let Neal speak to that.

...

As we're looking at treating this as a Fedora Server deliverable (with

the

...

associated branding), we need to open a vote.

Proposal: Beginning with Fedora 29, the Fedora Server SIG will produce a base snap image for Snappy. This image will not be blocking for the

Fedora

...

29 release. Upon approval of this proposal, Neal Gompa and Zygmunt

Krynicki

...

will discuss the plan with rel-eng and upon rel-eng approval will submit

a

...

Fedora Change Proposal announcing their intention.

Well the change process actually has the rel-eng sign off so it's sort of putting the cart before the horse, but while there's snap support I've not seen any patches around the integration into imagefactory/koji/pungi which is the core compose process. I feel these patches should be posted and accepted before the process goes forward, or at least be documenting and have the teams that would need to provide them agree. Traditionally these sorts of things have generally come from those wanting the feature and aren't the job of rel-eng to code and deliver (they seriously have more than enough to deal with on a day-to-day basis) and rely on tools that are maintained by a number of teams outside of rel-eng.

Yes, that's what I'm saying: Neal and Zygmunt are offering to do the legwork here. What they want is a sign-off from us that if they do the work, we'll include it. I think there's value in inclusion, so I proposed it to the Server SIG for approval.

...

The proposal will be considered accepted as long as it receives three +1 votes and zero -1 votes within the next week.

I've no idea if I have a vote, which no doubt means I don't, but I'm -1

You do, in fact, get a vote. Our policy is that anyone who takes the time to become non-trivially involved in the discussion is eligible to cast a vote. If we don't have lazy consensus on the mailing list, it will move to a majority vote at the next Server SIG meeting that reaches a quorum of WG members. So if your -1 stands after my reply, we'll put it on the agenda for Tuesday.

I feel this is a diversion of other things that the Server Working Group should be focusing on like making Modularity amazing.

Adding two people who were not otherwise working on Modularity doesn't seem to me like a diversion, other than the approval process and other administrivia.

I'm also very unclear as to what value a base Snap provides to the Server WG. I mean sure it provides the ability for someone that wishes to use the Snap delivery mechanism to use Fedora with in that "base container" to build their application on to run in the, IMO rather small, eco system that is the Snap ecosystem, but to run those on the Server WG deliverable is shipping the appropriate runtimes part of the proposal? I think it should be a similar but related change. If it's not part of the proposal why is it useful to the Server Working Group?

Well, even without the snap core bits on the Server deliverable, I think there's value to having Fedora provide a base for generating snaps that can run on other systems. If nothing else, it will help reduce friction for people moving from one system to another.

It's also useful for avoiding the "all eggs in one basket" situation. I think we've got a lot of good ideas and good work in the pipeline, but I'm also not one for blocking other folks from experimenting with their own ideas. As long as their efforts don't cause a significant workload increase on the rest of Server or Rel Eng (which has been communicated to them; I've told Mohan that if he thinks it's going to eat into his ability to get other things done, he needs to let us know so we can defer or reject this), I'd like to see where they take it.

Ultimately I feel it adds a diversion and/or distraction to other things the Server WG should be focusing on like the widely open and already adopted OCI container mechanism, modularity, orchestration through tooling and other such things.

That's a fair opinion and I respect it. Mine differs; I think if someone else is coming in and offering to do the work to add a new feature, I'd like to encourage them to do so, as long as it doesn't become a drain on our core work. I don't currently feel like this will do that, but I'm certainly going to take under advisement any new information that suggests otherwise.