We've been working on the idea that we should have a way to take Fedora RPMs and convert them into xdg-app runtimes and applications; by doing this inside of Koji we would be able to pretty transparently provide a set of xdg-apps for many existing applications in Fedora.
(https://fedoraproject.org/wiki/Workstation/BuildingXdgApps tries to work out possible details.)
I have an initial prototype of this working with a branch of Koji (https://pagure.io/fork/otaylor/koji/commits/xdg-app, still rough), and that raised some questions to me about around versioning:
* How should the fedora runtime be versioned? Is each Fedora release (23,24,25) a separate branch in the repository? (I'm not entirely sure if the "branch" of the runtime the same as "runtime-version" that is part of the xdg-app app metadata?)
* What happens if we evolve the Fedora runtime during the development release - say we add a new package to it that we find we are repeatedly bundling into applications. Can we make the behavior for the user better than simply having xdg-apps that rely on the new package in the runtime die with obscure error messages?
* How should apps generated from Fedora packages be versioned? Presumably we don’t want each Fedora version to be a branch in this case, since we want automatic updates. Are refs pointing to devel/testing/stable versions are something added post-build into the repository?
* We probably will be creating some sort of bundles when we build apps and runtimes in Koji. (This is what my prototype does using the existing xdg-app ostree delta format.) What do we use for filenames for these bundles? The RPM nvr of the application itself doesn't tell the story because of the bundling of dependencies in the application. Is the filename just based off the hash of the xdg-app like ‘eog-f23-d3dafc9cf48.xdgapp’? Do we add a build-serial to make it clear what bundle is newer - e.g. ‘eog-f23-14-d3dafc9cf48.xdgapp’ ?
Thanks, Owen
----- Original Message -----
We've been working on the idea that we should have a way to take Fedora RPMs and convert them into xdg-app runtimes and applications; by doing this inside of Koji we would be able to pretty transparently provide a set of xdg-apps for many existing applications in Fedora.
(https://fedoraproject.org/wiki/Workstation/BuildingXdgApps tries to work out possible details.)
I have an initial prototype of this working with a branch of Koji (https://pagure.io/fork/otaylor/koji/commits/xdg-app, still rough), and that raised some questions to me about around versioning:
- How should the fedora runtime be versioned? Is each Fedora release
(23,24,25) a separate branch in the repository? (I'm not entirely sure if the "branch" of the runtime the same as "runtime-version" that is part of the xdg-app app metadata?)
- What happens if we evolve the Fedora runtime during the development release
- say we add a new package to it that we find we are repeatedly bundling into applications. Can we make the behavior for the user better than simply having xdg-apps that rely on the new package in the runtime die with obscure error messages?
- How should apps generated from Fedora packages be versioned? Presumably we
don’t want each Fedora version to be a branch in this case, since we want automatic updates. Are refs pointing to devel/testing/stable versions are something added post-build into the repository?
- We probably will be creating some sort of bundles when we build apps and
runtimes in Koji. (This is what my prototype does using the existing xdg-app ostree delta format.) What do we use for filenames for these bundles? The RPM nvr of the application itself doesn't tell the story because of the bundling of dependencies in the application. Is the filename just based off the hash of the xdg-app like ‘eog-f23-d3dafc9cf48.xdgapp’? Do we add a build-serial to make it clear what bundle is newer - e.g. ‘eog-f23-14-d3dafc9cf48.xdgapp’ ?
* How do we know what authorisations each application needs to be able to run properly? How do we avoid applications overreaching and asking for access for things they don't need, or silently failing to work because the xdg-app bundle didn't offer enough?
If we're going to have to manually create the manifests to list those, what difference would it make compared to doing those in their respective upstreams?
On Thu, 2016-04-21 at 07:32 -0400, Bastien Nocera wrote:
- How do we know what authorisations each application needs to be
able to run properly? How do we avoid applications overreaching and asking for access for things they don't need, or silently failing to work because the xdg-app bundle didn't offer enough?
If we're going to have to manually create the manifests to list those, what difference would it make compared to doing those in their respective upstreams?
In general, the desired end-game is that application authors take responsibility for making xdg-apps available for their own applications. These active and interested application authors are also the ones I expect to take interest in modifying applications to take advantage of sandboxing.
But what we're really targeting here is all the other applications - how do we get an xdg-app available for bzflag or exmh without creating a whole new packaging, security updating, etc, infrastructure going?
I expect mostly that the metadata will be pretty permissive - that most of these apps will need access to $HOME, to the X socket, but yes, we do need it there.
The approach I'm taking to metadata is to have a json file is to have a stripped-down version of the xdg-app-builder json file, that can live in dist-git along side the spec file. Attached is the one I used for an initial build of eog; it would be nice if we could get it even smaller without complicated boilerplate for DConf, etc.
- Owen
On Wed, 2016-04-20 at 21:24 -0400, Owen Taylor wrote:
We've been working on the idea that we should have a way to take Fedora RPMs and convert them into xdg-app runtimes and applications; by doing this inside of Koji we would be able to pretty transparently provide a set of xdg-apps for many existing applications in Fedora.
(https://fedoraproject.org/wiki/Workstation/BuildingXdgApps tries to work out possible details.)
I have an initial prototype of this working with a branch of Koji (https://pagure.io/fork/otaylor/koji/commits/xdg-app, still rough), and that raised some questions to me about around versioning:
- How should the fedora runtime be versioned? Is each Fedora release
(23,24,25) a separate branch in the repository? (I'm not entirely sure if the "branch" of the runtime the same as "runtime-version" that is part of the xdg-app app metadata?)
The branch should be "24", etc. "branch" and "runtime-version" do indeed refer to the same kind of thing (one is the branch to build for and the other is the branch to use for the runtime it depends on). This discrepancy in terminology is a bit unfortunate, and i think we should make it use "version" instead of "branch".
- What happens if we evolve the Fedora runtime during the development
release - say we add a new package to it that we find we are repeatedly bundling into applications. Can we make the behavior for the user better than simply having xdg-apps that rely on the new package in the runtime die with obscure error messages?
The idea with xdg-app runtimes is that they define a stable ABI, so that old apps keep working. This means you should avoid to add things to them (as then new apps may break on older runtime versions), but additionally you should mainly do bugfix updates and not version bumps in the runtime. However, xdg-app doesn't enforce this any way, and it can't in general because things like unstable nightly snapshots of runtimes are an interesting and useful thing. Basically, it is up to the producer of a runtime to define (and document) the stability of a runtime and for producers of apps to pick a runtime that has a stability guarantee that works for them.
This is a bit problematic for fedora, as historically we've been pretty gung-ho about version updates in the middle of a stable release. The runtime will be smaller and more core though, so perhaps it will see less of such updates.
At the end of the day it depends on the goal of the fedora runtime. If we want it to mainly be a vehicle for running xdg-apps created from existing fedora rpms, which we're in full control off, then having the runtime be more "slushy" is an advantage to us, but if we want third party developers to do release binaries that rely on the fedora runtimes, then we can't break backwards compat in them at all, and even adding stuff is risky.
- How should apps generated from Fedora packages be versioned?
Presumably we don’t want each Fedora version to be a branch in this case, since we want automatic updates. Are refs pointing to devel/testing/stable versions are something added post-build into the repository?
For the gnome runtime and app releases I'm using two branches. One is "master" which gets a new build from git every day, and the other is "stable" which gets builds of the latest released version. That way, if you install the stable one you will automatically get updates.
ostree doesn't currently support tags, so there is no clean way to get a particular (old) version of an app. Technically one could make a branch for each release, but that is a bit unclean imho (and it explode the size of the repo lists).
- We probably will be creating some sort of bundles when we build
apps and runtimes in Koji. (This is what my prototype does using the existing xdg-app ostree delta format.) What do we use for filenames for these bundles? The RPM nvr of the application itself doesn't tell the story because of the bundling of dependencies in the application. Is the filename just based off the hash of the xdg-app like ‘eog-f23-d3dafc9cf48.xdgapp’? Do we add a build-serial to make it clear what bundle is newer - e.g. ‘eog-f23-14-d3dafc9cf48.xdgapp’ ?
Having a build serial seems useful, but that implies some kind of permanent storage of how many builds each app has. Do we really have that?
On Thu, 2016-04-21 at 16:55 +0200, Alexander Larsson wrote:
[...]
- What happens if we evolve the Fedora runtime during the
development release - say we add a new package to it that we find we are repeatedly bundling into applications. Can we make the behavior for the user better than simply having xdg-apps that rely on the new package in the runtime die with obscure error messages?
The idea with xdg-app runtimes is that they define a stable ABI, so that old apps keep working. This means you should avoid to add things to them (as then new apps may break on older runtime versions), but additionally you should mainly do bugfix updates and not version bumps in the runtime. However, xdg-app doesn't enforce this any way, and it can't in general because things like unstable nightly snapshots of runtimes are an interesting and useful thing. Basically, it is up to the producer of a runtime to define (and document) the stability of a runtime and for producers of apps to pick a runtime that has a stability guarantee that works for them.
This is a bit problematic for fedora, as historically we've been pretty gung-ho about version updates in the middle of a stable release. The runtime will be smaller and more core though, so perhaps it will see less of such updates.
Without any actual data to back it up, I don't think the types of libraries that appear in the runtime actually get non-bug-fix updated during a stable release very often.
What I'm mostly concerned about is the devel phase where libraries *should* be getting updated, dependencies added, and so forth. Is the user experience going to be very bad here if the apps get out of sync with the runtime, with apps refusing to start or crashing obscurely?
Maybe gnome-software simply needs to check for a newer available version of a runtime when installing an application that uses a runtime.
For runtimes and applications created from RPMs, we have some idea that we should be shipping the RPM database information in the xdg-app. There would also be a possibility to check that the RPMs bundled in the runtime satisfy the dependencies of the RPMs bundled in the application, but that doesn't work in the general case, and a failure could only realistically could be used to trigger for a check for a newer runtime. And then why not just do that to start with? - it's a single http roundtrip.
[...]
- How should apps generated from Fedora packages be versioned?
Presumably we don’t want each Fedora version to be a branch in this case, since we want automatic updates. Are refs pointing to devel/testing/stable versions are something added post-build into the repository?
For the gnome runtime and app releases I'm using two branches. One is "master" which gets a new build from git every day, and the other is "stable" which gets builds of the latest released version. That way, if you install the stable one you will automatically get updates.
ostree doesn't currently support tags, so there is no clean way to get a particular (old) version of an app. Technically one could make a branch for each release, but that is a bit unclean imho (and it explode the size of the repo lists).
Do you think there should be a field in the metadata that is a version string provided at build time (not implying any comparison), so that a tool could look at a repo and show the old versions on a branch in some human readable form? Or should that be in the ostree commit message?
- We probably will be creating some sort of bundles when we build
apps and runtimes in Koji. (This is what my prototype does using the existing xdg-app ostree delta format.) What do we use for filenames for these bundles? The RPM nvr of the application itself doesn't tell the story because of the bundling of dependencies in the application. Is the filename just based off the hash of the xdg-app like ‘eog-f23-d3dafc9cf48.xdgapp’? Do we add a build- serial to make it clear what bundle is newer - e.g. ‘eog-f23-14- d3dafc9cf48.xdgapp’ ?
Having a build serial seems useful, but that implies some kind of permanent storage of how many builds each app has. Do we really have that?
Something could be added to the koji database, or we could even just use the koji task ID - that would be more like:
eog-f23-13799291-d3dafc9cf48.xdgapp
But still would allow comparison.
- Owen
On mån, 2016-04-25 at 15:45 -0400, Owen Taylor wrote:
On Thu, 2016-04-21 at 16:55 +0200, Alexander Larsson wrote:
This is a bit problematic for fedora, as historically we've been pretty gung-ho about version updates in the middle of a stable release. The runtime will be smaller and more core though, so perhaps it will see less of such updates.
Without any actual data to back it up, I don't think the types of libraries that appear in the runtime actually get non-bug-fix updated during a stable release very often.
True.
What I'm mostly concerned about is the devel phase where libraries *should* be getting updated, dependencies added, and so forth. Is the user experience going to be very bad here if the apps get out of sync with the runtime, with apps refusing to start or crashing obscurely?
This is similar to the "nightly build" of the gnome runtime and apps, which builds nightly from git master. If the app and the runtime go out of sync there is no guarantees (not that git master has any guarantees anyway).
Maybe gnome-software simply needs to check for a newer available version of a runtime when installing an application that uses a runtime.
So, yeah, its probably always best if gnome-software ensure that whenever it updates an app it updates the runtime to the latest version.
For the gnome runtime and app releases I'm using two branches. One is "master" which gets a new build from git every day, and the other is "stable" which gets builds of the latest released version. That way, if you install the stable one you will automatically get updates.
ostree doesn't currently support tags, so there is no clean way to get a particular (old) version of an app. Technically one could make a branch for each release, but that is a bit unclean imho (and it explode the size of the repo lists).
Do you think there should be a field in the metadata that is a version string provided at build time (not implying any comparison), so that a tool could look at a repo and show the old versions on a branch in some human readable form? Or should that be in the ostree commit message?
I dunno if this should go in the metadata. General user readable version information go in the appdata file, and it seems putting things into the ostree commit is a better approach.
I'm not sure using the ostree commit message is a great idea though, its freeform and we should keep it so. Instead we could use a custom metadata field in the commit. Ostree has a generic extension mechanism for this.
Basically, you'd do: ostree commit --add-metadata-string=xa.build_name=gedit-3.20.1-1fc24 or something like that. Of course, we'd have to plumb that into xdg- app, maybe as a generic metadata thing or a specific thing for this (or both).
- We probably will be creating some sort of bundles when we build
apps and runtimes in Koji. (This is what my prototype does using the existing xdg-app ostree delta format.) What do we use for filenames for these bundles? The RPM nvr of the application itself doesn't tell the story because of the bundling of dependencies in the application. Is the filename just based off the hash of the xdg-app like ‘eog-f23-d3dafc9cf48.xdgapp’? Do we add a build- serial to make it clear what bundle is newer - e.g. ‘eog-f23-14- d3dafc9cf48.xdgapp’ ?
Having a build serial seems useful, but that implies some kind of permanent storage of how many builds each app has. Do we really have that?
Something could be added to the koji database, or we could even just use the koji task ID - that would be more like:
eog-f23-13799291-d3dafc9cf48.xdgapp
But still would allow comparison.
Ah, globally unique via task id rather than an incremental per-app number. Seems good to me.
On Thu, Apr 21, 2016 at 11:24 AM, Owen Taylor otaylor@redhat.com wrote:
We've been working on the idea that we should have a way to take Fedora RPMs and convert them into xdg-app runtimes and applications; by doing this inside of Koji we would be able to pretty transparently provide a set of xdg-apps for many existing applications in Fedora.
(https://fedoraproject.org/wiki/Workstation/BuildingXdgApps tries to work out possible details.)
I have an initial prototype of this working with a branch of Koji (https://pagure.io/fork/otaylor/koji/commits/xdg-app, still rough), and that raised some questions to me about around versioning:
- How should the fedora runtime be versioned? Is each Fedora release
(23,24,25) a separate branch in the repository? (I'm not entirely sure if the "branch" of the runtime the same as "runtime-version" that is part of the xdg-app app metadata?)
- What happens if we evolve the Fedora runtime during the development
release - say we add a new package to it that we find we are repeatedly bundling into applications. Can we make the behavior for the user better than simply having xdg-apps that rely on the new package in the runtime die with obscure error messages?
- How should apps generated from Fedora packages be versioned? Presumably
we don’t want each Fedora version to be a branch in this case, since we want automatic updates. Are refs pointing to devel/testing/stable versions are something added post-build into the repository?
- We probably will be creating some sort of bundles when we build apps and
runtimes in Koji. (This is what my prototype does using the existing xdg-app ostree delta format.) What do we use for filenames for these bundles? The RPM nvr of the application itself doesn't tell the story because of the bundling of dependencies in the application. Is the filename just based off the hash of the xdg-app like ‘eog-f23-d3dafc9cf48.xdgapp’? Do we add a build-serial to make it clear what bundle is newer - e.g. ‘eog-f23-14-d3dafc9cf48.xdgapp’ ?
Thanks, Owen
A few questions about this
How will the Fedora versions of the runtimes be delivered to users? will Fedora have a ostree/xdg-app repository similar tohttps:// sdk.gnome.org/repo/ ? and will this be configured and ready-to-go for users when they install an xdg-app?
If i try to install an xdg-app that uses a particular runtime, but i don't have that runtime installed, will this just be pulled down as a dependency? what happens if i try to install an xdg-app when the runtime isnt available on my system or in one of the repos i have installed?
cheers, ryanlerch
-- desktop mailing list desktop@lists.fedoraproject.org http://lists.fedoraproject.org/admin/lists/desktop@lists.fedoraproject.org
----- Original Message -----
From: "Ryan Lerch" rlerch@redhat.com To: "Discussions about development for the Fedora desktop" desktop@lists.fedoraproject.org Sent: Thursday, April 21, 2016 7:14:04 PM Subject: Re: Versioning Fedora-built xdg-apps and runtimes
On Thu, Apr 21, 2016 at 11:24 AM, Owen Taylor < otaylor@redhat.com > wrote:
We've been working on the idea that we should have a way to take Fedora RPMs and convert them into xdg-app runtimes and applications; by doing this inside of Koji we would be able to pretty transparently provide a set of xdg-apps for many existing applications in Fedora.
( https://fedoraproject.org/wiki/Workstation/BuildingXdgApps tries to work out possible details.)
I have an initial prototype of this working with a branch of Koji ( https://pagure.io/fork/otaylor/koji/commits/xdg-app , still rough), and that raised some questions to me about around versioning:
- How should the fedora runtime be versioned? Is each Fedora release
(23,24,25) a separate branch in the repository? (I'm not entirely sure if the "branch" of the runtime the same as "runtime-version" that is part of the xdg-app app metadata?)
- What happens if we evolve the Fedora runtime during the development release
say we add a new package to it that we find we are repeatedly bundling into applications. Can we make the behavior for the user better than simply having xdg-apps that rely on the new package in the runtime die with obscure error messages?
- How should apps generated from Fedora packages be versioned? Presumably we
don’t want each Fedora version to be a branch in this case, since we want automatic updates. Are refs pointing to devel/testing/stable versions are something added post-build into the repository?
- We probably will be creating some sort of bundles when we build apps and
runtimes in Koji. (This is what my prototype does using the existing xdg-app ostree delta format.) What do we use for filenames for these bundles? The RPM nvr of the application itself doesn't tell the story because of the bundling of dependencies in the application. Is the filename just based off the hash of the xdg-app like ‘eog-f23-d3dafc9cf48.xdgapp’? Do we add a build-serial to make it clear what bundle is newer - e.g. ‘eog-f23-14-d3dafc9cf48.xdgapp’ ?
Thanks, Owen
A few questions about this
How will the Fedora versions of the runtimes be delivered to users? will Fedora have a ostree/xdg-app repository similar tohttps:// sdk.gnome.org/repo/ ? and will this be configured and ready-to-go for users when they install an xdg-app?
If i try to install an xdg-app that uses a particular runtime, but i don't have that runtime installed, will this just be pulled down as a dependency? what happens if i try to install an xdg-app when the runtime isnt available on my system or in one of the repos i have installed?
I think the plan is for every xdg-app to contain a pointer to where one can find its runtime. So if you don't have it the xdg-app should be able to tell your system where to get it from. I guess there might be a chance with old xdg-apps that the runtime has disappeared from the upstream, but that is where being a credible longitime runtime provider comes in.
Christian
On Fri, 2016-04-22 at 09:14 +1000, Ryan Lerch wrote:
A few questions about this
How will the Fedora versions of the runtimes be delivered to users? will Fedora have a ostree/xdg-app repository similar tohttps://sdk.gn ome.org/repo/ ? and will this be configured and ready-to-go for users when they install an xdg-app?
Yes, that would be the plan.
If i try to install an xdg-app that uses a particular runtime, but i don't have that runtime installed, will this just be pulled down as a dependency? what happens if i try to install an xdg-app when the runtime isnt available on my system or in one of the repos i have installed?
Runtimes are identified by name (org.fedoraproject.Platform) and not by URL (http://repo.fedoraproject.org/runtime/platform%C2%A0or whatever), so the runtimes do have to be configured somewhere. I would certainly expect that for the Fedora runtimes and also other well-known runtimes this configuration would ship as part of the Fedora repo and installation would occur seamlessly. Runtimes can be big - the draft Fedora 24 runtime is 200MB download/860MB installed - so perhaps there would be a prompt.
- Owen
desktop@lists.fedoraproject.org