(Pulled out of the meeting thread)
We've been talking about extended lifecycles as an admin-friendly direction to take Fedora Server. But, I think we should consider spending more resources better-supporting migrations to new servers, including OS version bumps. Administrators often favor long lifecycles because this process is usually tedious and error-prone. The more we make this part of normal operations, the more we make super-long lifecycles unnecessary.
It's probably ambitious for a first release, but it would be amazing to have good tools for vacuuming over services, containers, and VMs. The latter two already have tooling, but it would be super cool to abstract it into a general utility for migrations. It could enumerate migration-enabled resources. Integration with RPM verification could list configuration that's not possible to automatically migrate.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon 25 Nov 2013 11:17:57 PM EST, David Timothy Strauss wrote:
(Pulled out of the meeting thread)
We've been talking about extended lifecycles as an admin-friendly direction to take Fedora Server. But, I think we should consider spending more resources better-supporting migrations to new servers, including OS version bumps. Administrators often favor long lifecycles because this process is usually tedious and error-prone. The more we make this part of normal operations, the more we make super-long lifecycles unnecessary.
It's probably ambitious for a first release, but it would be amazing to have good tools for vacuuming over services, containers, and VMs. The latter two already have tooling, but it would be super cool to abstract it into a general utility for migrations. It could enumerate migration-enabled resources. Integration with RPM verification could list configuration that's not possible to automatically migrate.
Migrations are a really difficult problem to solve, particularly when dealing with Linux, since all of the subsystems on the OS are developed more or less independently of each other. Not every project agrees on the degree of backwards-compatibility or migration path that they need to support.
Now, with that being said, I think we can use the "Roles" proposals here to improve on this somewhat. We can try to set some rules for compatibility and migration path on those roles that are declared "Featured" in our parlance.
I'm not sure we want to "vacuum" these services up, exactly. But any services that are registered with our (to be designed) role interface should be possible to *reproduce* on the new system. I think we need to keep this use-case in mind as we design the role API, so that it's possible to use it to pull all of the necessary information back out of an installed role to be used to seed a newly-deployed role. Then we can focus our migration efforts on handling changes in the needs of the API input instead of necessarily dealing with every service's particular configurations directly.
There are two primary parts to any service: configuration and state. Configuration is, more or less, straightforward with the limitations you've mentioned.
State is the complex one. I've worked with all the following relationships of state between servers:
(a) Replication (master/slave and ring-based) (b) Serialization/archive for backup purposes (c) Serialization/archive for server migration (d) Serialization/archive for replica-bootstrapping purposes (often a special case of b or c)
This is based on work with Cassandra, Redis, MariaDB, Solr, PostgreSQL, and I'm sure a few others. It would be really neat for our featured roles to come pre-packaged with consistent access to facilities like these.
I see this potentially happening in several stages:
(1) Dump and load for configuration and state. Support using the archive for backup or server-migration purposes. (2) Support replica creation, including whatever's required to bootstrap replicas and having a secure relationship between the servers. (3) Use replicas as the server migration facility. At my company, we call this "mutiny." It's where a replica gets created, caught up, and then takes over the master role (however necessary). (4) Support HA tools using replicas for fail-over.
Some of these facilities may be free with container tools like Docker.
On Wed, 2013-11-27 at 02:34 +1000, David Timothy Strauss wrote:
There are two primary parts to any service: configuration and state. Configuration is, more or less, straightforward with the limitations you've mentioned.
State is the complex one. I've worked with all the following relationships of state between servers:
(a) Replication (master/slave and ring-based) (b) Serialization/archive for backup purposes (c) Serialization/archive for server migration (d) Serialization/archive for replica-bootstrapping purposes (often a special case of b or c)
This is based on work with Cassandra, Redis, MariaDB, Solr, PostgreSQL, and I'm sure a few others. It would be really neat for our featured roles to come pre-packaged with consistent access to facilities like these.
I see this potentially happening in several stages:
(1) Dump and load for configuration and state. Support using the archive for backup or server-migration purposes. (2) Support replica creation, including whatever's required to bootstrap replicas and having a secure relationship between the servers. (3) Use replicas as the server migration facility. At my company, we call this "mutiny." It's where a replica gets created, caught up, and then takes over the master role (however necessary). (4) Support HA tools using replicas for fail-over.
Some of these facilities may be free with container tools like Docker.
Are you doing these "replicas" despite what the tools offer ? Or do you integrate with them ?
I am fairly familiar with LDAP replication for example, and that will work only with the cooperation of the old and new LDAP servers.
Simo.
On Tue, Dec 3, 2013 at 6:45 AM, Simo Sorce simo@redhat.com wrote:
Are you doing these "replicas" despite what the tools offer ? Or do you integrate with them ?
I am fairly familiar with LDAP replication for example, and that will work only with the cooperation of the old and new LDAP servers.
Natively using the built-in daemon support: MySQL master/slave, LDAP multi-master, Cassandra multi-master, etc.
server@lists.fedoraproject.org
-
David Timothy Strauss -
Simo Sorce -
Stephen Gallagher