-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
During this week's Fedora Server Working Group meeting[1], we started planning out how we want the public D-BUS API to look. I volunteered to take the discussion to the list so we could start hashing it out with a wider audience. I'm CC-ing the cockpit-devel list since a lot of this will affect that project, but I'd prefer to keep replies on server@lists.fedoraproject.org please (easier to track on one list).
Some pre-conditions/goals (many of these are reminders from earlier discussions):
1) We want to be able to support two initial clients of this API: the Cockpit management console and a local command-line tool (I've been tentatively calling it 'fedora-role-manage') A third, future option will be to also support remote access to this API via the OpenLMI Project.
2) We do not need to have a single generic API that will work for any conceivable Role. We've identified in the past that the selection of Featured roles will (for the conceivable future) remain a small, manageable list. For this reason, it is acceptable for the deployment and post-deployment configuration APIs to be entirely different between two roles. However, some functionality such as the backup and firewall queries should be stable between all roles.
3) The supported clients should be able to determine if a target system supports the deployment of a particular role (for backwards- compatibility and architecture differences), but the Role API should *NOT* be attempting to dictate the UI experience. UIs should merely check whether the role is available and then use a previously- developed experience. We don't want to be reimplementing PAM or debconf here.
4) The D-BUS API must be stable. If it is likely that the set of input required to deploy or configure will change, this should be planned for in the design.
5) As it is not possible to plan for all contingencies, the API must be versioned such that the clients can determine whether they can properly perform the requested actions.
Some initial thoughts on the matter (all are open to discussion):
Standard Objects:
/org/fedoraproject/server/ServerRoleManager: * properties: * all_roles: list of object references to Roles that can be installed and deployed * staging_roles: list of object references to Roles that have been pre-loaded but not yet configured * active_roles: list of object references to deployed Roles * version: API version * methods:
# These properties and methods should be available and identical on # all Roles
/org/fedoraproject/server/role/$ROLE: * properties: * version: API version * loaded: Bool: whether the packages are installed * deployed: Bool: whether the Role has been deployed * methods: * PreloadRole: Install all necessary software packages for default operation * GetFirewallPorts: list of port/protocol pairs that the Role needs open in the firewall * PrepBackup: method to invoke any necessary pre-backup steps (such as running a database dump to a file) * GetBackupFiles: list of filesystem path objects that identifies all data that should be copied by backup software
# Individual Roles should have unique implementations of installation # and deployment
e.g. /org/fedoraproject/server/role/DomainController * methods: * DeployPrimaryDomainController: Set up the first domain controller in the domain. * DeployReplicaDomainController: Add a new replica domain controller to the domain * AddCertificateAuthority: Add a certificate authority to this domain controller * EnableDisableDNS: Enable or disable the DNS server on this server
== Open questions ==
1) How do we handle configuring the Firewall? I think we want to have a Firewall object (/org/fedoraproject/server/RoleFirewallManager) available to query the firewall as a whole, but we may also want to be able to view and apply firewall rules from the Role objects directly. (Note: I'm not suggesting we need a complete firewall solution here. This should be a wrapper that deals only with the Roles). Firewalld already provides a more comprehensive firewall interface for the general case.
I'm sure there are other questions we'll discover out there, but this email is already running long for an "introduction" to the topic.
[1] http://meetbot.fedoraproject.org/fedora-meeting-1/2014-03-18/fedora-meeting-...
On Wed, 19 Mar 2014 09:47:20 -0400 Stephen Gallagher sgallagh@redhat.com wrote:
...snip...
- The D-BUS API must be stable. If it is likely that the set of
input required to deploy or configure will change, this should be planned for in the design.
- As it is not possible to plan for all contingencies, the API must
be versioned such that the clients can determine whether they can properly perform the requested actions.
I agree with these, but we may not want to '1.0' the api until we are further along?
Some initial thoughts on the matter (all are open to discussion):
Standard Objects:
/org/fedoraproject/server/ServerRoleManager:
- properties:
- all_roles: list of object references to Roles that can be installed and deployed
It gets this from the fedora-release-server? or ?
- staging_roles: list of object references to Roles that have been pre-loaded but not yet configured
This it can get from installed packages or a file.
- active_roles: list of object references to deployed Roles
- version: API version
- methods:
Ditto.
...snip...
== Open questions ==
- How do we handle configuring the Firewall? I think we want to have
a Firewall object (/org/fedoraproject/server/RoleFirewallManager) available to query the firewall as a whole, but we may also want to be able to view and apply firewall rules from the Role objects directly. (Note: I'm not suggesting we need a complete firewall solution here. This should be a wrapper that deals only with the Roles). Firewalld already provides a more comprehensive firewall interface for the general case.
Should we try and rope in the firewalld maintainer(s) here? They may have some ideas on how to do this so it causes the least amount of pain.
I'm sure there are other questions we'll discover out there, but this email is already running long for an "introduction" to the topic.
I think it's a good start! :)
kevin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/19/2014 06:10 PM, Kevin Fenzi wrote:
On Wed, 19 Mar 2014 09:47:20 -0400 Stephen Gallagher sgallagh@redhat.com wrote:
...snip...
- The D-BUS API must be stable. If it is likely that the set of
input required to deploy or configure will change, this should be planned for in the design.
- As it is not possible to plan for all contingencies, the API
must be versioned such that the clients can determine whether they can properly perform the requested actions.
I agree with these, but we may not want to '1.0' the api until we are further along?
I'd like for clients to be backwards and forwards-compatible. So new versions of clients should be able to fall back to older, deprecated versions of the API and newer versions of the server should be able to support any version of the API that is still meaningful (such as we've added a new version of the API that gives more control, but the old version could still produce a usable system compatible with older releases).
...snip...
== Open questions ==
- How do we handle configuring the Firewall? I think we want to
have a Firewall object (/org/fedoraproject/server/RoleFirewallManager) available to query the firewall as a whole, but we may also want to be able to view and apply firewall rules from the Role objects directly. (Note: I'm not suggesting we need a complete firewall solution here. This should be a wrapper that deals only with the Roles). Firewalld already provides a more comprehensive firewall interface for the general case.
Should we try and rope in the firewalld maintainer(s) here? They may have some ideas on how to do this so it causes the least amount of pain.
Yes, good idea. I'm CCing Thomas Woerner and will politely ask him to review this thread.[1]
[1] https://lists.fedoraproject.org/pipermail/server/2014-March/000999.html
2014-03-20 14:00 GMT+01:00 Stephen Gallagher sgallagh@redhat.com:
I agree with these, but we may not want to '1.0' the api until we are further along?
I'd like for clients to be backwards and forwards-compatible. So new versions of clients should be able to fall back to older, deprecated versions of the API and newer versions of the server should be able to support any version of the API that is still meaningful (such as we've added a new version of the API that gives more control, but the old version could still produce a usable system compatible with older releases).
That requires us to know and specify in advance how would the server know which version of the API the client is indenting to use. So, for the record, do we plan to just add methods (e.g. DeployV2) to the existing interfaces? Mirek
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/24/2014 05:04 PM, Miloslav Trmač wrote:
2014-03-20 14:00 GMT+01:00 Stephen Gallagher <sgallagh@redhat.com mailto:sgallagh@redhat.com>:
I agree with these, but we may not want to '1.0' the api until we are further along?
I'd like for clients to be backwards and forwards-compatible. So new versions of clients should be able to fall back to older, deprecated versions of the API and newer versions of the server should be able to support any version of the API that is still meaningful (such as we've added a new version of the API that gives more control, but the old version could still produce a usable system compatible with older releases).
That requires us to know and specify in advance how would the server know which version of the API the client is indenting to use. So, for the record, do we plan to just add methods (e.g. DeployV2) to the existing interfaces?
That's certainly the easiest (and most D-BUS-esque) way to do it. I'm open to other suggestions, of course.
Hello,
On 03/20/2014 02:00 PM, Stephen Gallagher wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/19/2014 06:10 PM, Kevin Fenzi wrote:
On Wed, 19 Mar 2014 09:47:20 -0400 Stephen Gallagher sgallagh@redhat.com wrote:
...snip...
- The D-BUS API must be stable. If it is likely that the set of
input required to deploy or configure will change, this should be planned for in the design.
- As it is not possible to plan for all contingencies, the API
must be versioned such that the clients can determine whether they can properly perform the requested actions.
I agree with these, but we may not want to '1.0' the api until we are further along?
I'd like for clients to be backwards and forwards-compatible. So new versions of clients should be able to fall back to older, deprecated versions of the API and newer versions of the server should be able to support any version of the API that is still meaningful (such as we've added a new version of the API that gives more control, but the old version could still produce a usable system compatible with older releases).
...snip...
== Open questions ==
- How do we handle configuring the Firewall? I think we want to
have a Firewall object (/org/fedoraproject/server/RoleFirewallManager) available to query the firewall as a whole, but we may also want to be able to view and apply firewall rules from the Role objects directly. (Note: I'm not suggesting we need a complete firewall solution here. This should be a wrapper that deals only with the Roles). Firewalld already provides a more comprehensive firewall interface for the general case.
Should we try and rope in the firewalld maintainer(s) here? They may have some ideas on how to do this so it causes the least amount of pain.
Yes, good idea. I'm CCing Thomas Woerner and will politely ask him to review this thread.[1]
I have been on vacation and are in the office since today again.
What exactly do you want to be able to configure? Do you think of something like firewall zones or more fine grained configuration options like for example ports or direct rules?
[1] https://lists.fedoraproject.org/pipermail/server/2014-March/000999.html -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlMq5osACgkQeiVVYja6o6P6AwCeJV91PZDeY1mdRB4GlP+zLOVy ztcAnjg4/7Ta8MibVwMJKEyWIbAu467I =lgUF -----END PGP SIGNATURE-----
Regards, Thomas
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/25/2014 11:00 AM, Thomas Woerner wrote:
Hello,
On 03/20/2014 02:00 PM, Stephen Gallagher wrote:
== Open questions ==
- How do we handle configuring the Firewall? I think we
want to have a Firewall object (/org/fedoraproject/server/RoleFirewallManager) available to query the firewall as a whole, but we may also want to be able to view and apply firewall rules from the Role objects directly. (Note: I'm not suggesting we need a complete firewall solution here. This should be a wrapper that deals only with the Roles). Firewalld already provides a more comprehensive firewall interface for the general case.
Should we try and rope in the firewalld maintainer(s) here? They may have some ideas on how to do this so it causes the least amount of pain.
Yes, good idea. I'm CCing Thomas Woerner and will politely ask him to review this thread.[1]
I have been on vacation and are in the office since today again.
What exactly do you want to be able to configure? Do you think of something like firewall zones or more fine grained configuration options like for example ports or direct rules?
I'm not going to attempt to specify an implementation right now. I'd rather state the problem space as I see it and have you recommend an approach instead.
So to restate the problem: One of the primary goals of the Fedora Server product is to be able to package up some common IT infrastructure components and provide an easy way to deploy them on a physical or virtual machine[1]. Some representative examples of this are a Domain Controller (FreeIPA) and a Database Server (PostgreSQL).
We are working on building a D-BUS API and listener on Fedora Server systems whose duty it is to install the requisite packages and deploy a fully-functional Server Role onto the system.
Part of being a fully-functional part of infrastructure is having those services be accessible by the appropriate clients. The Role itself will "know" what ports it needs open, and this will be possible to query through the D-BUS API in some manner TBD. We need a way in the D-BUS API for an administrator to grant access to those requested ports on some or all network interfaces attached to the machine. We also want this interface to have an association with the Role object in the system, so that a client such as Cockpit can easily query a Role for "What ports do you need and on which interfaces can that port be reached?" Furthermore, we want there to be a mechanism to apply a set of very simple changes.
For a real-world example: I have decided to stand up a database server. While I'm setting it up, managing permissions and filling it with data, I want to keep this private, so I only want access to be allowed on the 'lo' interface. I want this to be done atomically as part of the initial deployment, so I don't have any race-conditions to worry about where my database might have been visible externally.
Once that is prepared, my tests are all complete and I know it's safe to expose the database to the external interfaces. I have two external ethernet interfaces. One is to the public network and the other is a private internal connection (a storage network or management plane, etc.) At this time, I want to update the firewall so that only 'lo' and the public network have access to the set of ports required by my database server.
Also, in cases where a Role might require more than one port (such as the Domain Controller) I might also want to only allow a subset of the Role's ports access on a particular interface.
Finally, I want this ability to allow access to some or all required ports on some or all available interfaces to be offered by the Deployment phase as well (in the same atomic manner as my example above).
I hope that describes the problem space sufficiently well. My colleagues in the Server SIG can correct me if I got any of the details wrong.
[1] I'm intentionally leaving containers out of the discussion for the moment.
2014-03-25 18:29 GMT+01:00 Stephen Gallagher sgallagh@redhat.com:
We also want this interface to have an association with the Role object in the system, so that a client such as Cockpit can easily query a Role for "What ports do you need and on which interfaces can that port be reached?" Furthermore, we want there to be a mechanism to apply a set of very simple changes.
<snip>
Also, in cases where a Role might require more than one port (such as the Domain Controller) I might also want to only allow a subset of the Role's ports access on a particular interface.
I think this really should be "a pre-designed subset", not "arbitrary subset"; once the user starts listing port numbers, the connection with the role starts becoming tenuous. It would be reasonable for some roles to provide consistent set of ports (e.g. "company-visible public data" vs. "DHCP and PXE and tftp server to be restricted to a specific interface"), but we shouldn't need arbitrary subsets that don't make sense (say, enabling cups announcing itself over avai while preventing access to the IPP service).
IPA might be a "worst-case" situation; if IPA can only live with pre-designed subsets of ports, probably every thing can. Mirek
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/25/2014 06:54 PM, Miloslav Trmač wrote:
2014-03-25 18:29 GMT+01:00 Stephen Gallagher sgallagh@redhat.com:
We also want this interface to have an association with the Role object in the system, so that a client such as Cockpit can easily query a Role for "What ports do you need and on which interfaces can that port be reached?" Furthermore, we want there to be a mechanism to apply a set of very simple changes.
<snip> > Also, in cases where a Role might require more than one port > (such as the Domain Controller) I might also want to only allow a > subset of the Role's ports access on a particular interface.
I think this really should be "a pre-designed subset", not "arbitrary subset"; once the user starts listing port numbers, the connection with the role starts becoming tenuous. It would be reasonable for some roles to provide consistent set of ports (e.g. "company-visible public data" vs. "DHCP and PXE and tftp server to be restricted to a specific interface"), but we shouldn't need arbitrary subsets that don't make sense (say, enabling cups announcing itself over avai while preventing access to the IPP service).
IPA might be a "worst-case" situation; if IPA can only live with pre-designed subsets of ports, probably every thing can.
Good point. I could get on board with that.
Hello everyone.
On Wed, 19 Mar 2014 09:47:20 -0400 Stephen Gallagher sgallagh@redhat.com wrote:
- We want to be able to support two initial clients of this API: the Cockpit management console and a local command-line tool (I've been tentatively calling it 'fedora-role-manage') A third, future option will be to also support remote access to this API via the OpenLMI Project.
Why do we need to separate the local and remote access in the CLI tool? You know where I'm heading to... There's the LMIShell: interactive Python shell-like environment allowing one to treat the remote CIM objects as local Python objects, then there is the LMI metacommand on top of it wrapping the direct CIM manipulations in a human-readable form since knowing the details of the API itself is something we simply don't want the users to bother with.
Now replace "CIM" with "D-Bus"... And at the "metacommand" level you may not even spot a difference. When you write about discoverability, API versioning, etc., I can see so many similarities with OpenLMI server side that I really think we don't need a brand new CLI tool to expose this new API.
Regards,
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/20/2014 08:38 AM, Tomáš Smetana wrote:
Hello everyone.
On Wed, 19 Mar 2014 09:47:20 -0400 Stephen Gallagher sgallagh@redhat.com wrote:
- We want to be able to support two initial clients of this API:
the Cockpit management console and a local command-line tool (I've been tentatively calling it 'fedora-role-manage') A third, future option will be to also support remote access to this API via the OpenLMI Project.
Why do we need to separate the local and remote access in the CLI tool? You know where I'm heading to... There's the LMIShell: interactive Python shell-like environment allowing one to treat the remote CIM objects as local Python objects, then there is the LMI metacommand on top of it wrapping the direct CIM manipulations in a human-readable form since knowing the details of the API itself is something we simply don't want the users to bother with.
Now replace "CIM" with "D-Bus"... And at the "metacommand" level you may not even spot a difference. When you write about discoverability, API versioning, etc., I can see so many similarities with OpenLMI server side that I really think we don't need a brand new CLI tool to expose this new API.
If the OpenLMI project would like to volunteer to implement this provider and metacommand during the Fedora 21 schedule, I'd be all for it. I was mostly assuming that the timing of things would be difficult to manage and that we might want to hack up a simple local command for the first pass.
I definitely see 'lmi role deploy' as a better solution (particularly since the same syntax can be used locally or remotely) in the long term. The provisional schedule for Fedora 21 means that the Alpha change deadline (when all major features need to work) is July 22nd. Are you willing to commit to being feature-complete by that date?
Dne Thu, 20 Mar 2014 08:55:58 -0400 Stephen Gallagher sgallagh@redhat.com napsal(a):
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/20/2014 08:38 AM, Tomáš Smetana wrote:
Hello everyone.
On Wed, 19 Mar 2014 09:47:20 -0400 Stephen Gallagher sgallagh@redhat.com wrote:
- We want to be able to support two initial clients of this API:
the Cockpit management console and a local command-line tool (I've been tentatively calling it 'fedora-role-manage') A third, future option will be to also support remote access to this API via the OpenLMI Project.
<...>
If the OpenLMI project would like to volunteer to implement this provider and metacommand during the Fedora 21 schedule, I'd be all for it. I was mostly assuming that the timing of things would be difficult to manage and that we might want to hack up a simple local command for the first pass.
It is difficult to manage not impossible though and it was generally admitted that it makes sense for OpenLMI to participate in this effort.
I definitely see 'lmi role deploy' as a better solution (particularly since the same syntax can be used locally or remotely) in the long term. The provisional schedule for Fedora 21 means that the Alpha change deadline (when all major features need to work) is July 22nd. Are you willing to commit to being feature-complete by that date?
We need to have some discussion about technical issues. OpenLMI is now trying to figure out how to do HW RAID management which probably means using libstoragemgmt and this would lead to: no CIM in OpenLMI this time. So we are already moving to "non-CIM solutions" for certain tasks and have to find out how to do that anyway.
I should know better on Monday (when the more knowledgeable people would be done with other work). I'll let you know. OpenLMI is definitely interested.
Thanks and regards,
On Fri, 21 Mar 2014 09:33:21 +0100 Tomáš Smetana tsmetana@redhat.com wrote:
If the OpenLMI project would like to volunteer to implement this provider and metacommand during the Fedora 21 schedule, I'd be all for it. I was mostly assuming that the timing of things would be difficult to manage and that we might want to hack up a simple local command for the first pass.
<...>
I should know better on Monday (when the more knowledgeable people would be done with other work). I'll let you know. OpenLMI is definitely interested.
OK. We will implement the Server role provider and the metacommand and I'm quite confident about these two to be ready in F21. We will also take a look at patching Pegasus for the socket activation which would obviously make things look much better when no remote management is required.
Regards,
2014-03-19 14:47 GMT+01:00 Stephen Gallagher sgallagh@redhat.com:
- We do not need to have a single generic API that will work for any conceivable Role. We've identified in the past that the selection of Featured roles will (for the conceivable future) remain a small, manageable list. For this reason, it is acceptable for the deployment and post-deployment configuration APIs to be entirely different between two roles. However, some functionality such as the backup and firewall queries should be stable between all roles.
(I.e. the universal Role in the proposal doesn't even have a DeployRole method.)
I don't think this is efficient--every "universal" client (and there will be at least one or perhaps two, fedora-role-manage, anaconda/initial-setup kickstart processing) would have to build essentially the same abstraction over these role differences.
Wouldn't it be cleaner to have generic ValidateSettings(settings) DeployRole(settings) as the abstraction used by the "universal" clients (with "settings" possibly having some universal settings, like our favorite firewall_enabled_after_deploy, but mostly role-specific, only using a standard format, perhaps JSON or some object graph with similar capabilities)?
The kickstart processor would only decode JSON and pass it to the API without understanding the contents; the Role-specific GUIs would have specific knowledge of settings to set--or specific Roles could even provide helper APIs, say AutodetectDefaultReplicaControllerSettings() to query the network and give the GUI information about what the role thinks is appropriate for the specific situation.)
/org/fedoraproject/server/ServerRoleManager:
- properties:
- all_roles
- staging_roles
- active_roles
Isn't this redundant with the per-role properties specifying the role's state? (Even if it were redundant, it wouldn't be a huge deal--just a little work that we might be able to avoid.)
/org/fedoraproject/server/role/$ROLE:
- properties:
- loaded: Bool: whether the packages are installed
- deployed: Bool: whether the Role has been deployed
(Nitpicking? Rather an enum to eliminate possibilities of paradoxes like deployed & ~loaded.)
* methods:
- GetFirewallPorts: list of port/protocol pairs that the Role needs
I'd much rather have FirewallRestriction enum (LocalhostOnly, Unrestricted, Complex) or something like that, or, if we decided to depend on firewalld, an object reference to a firewalld service. Roles will be asking for ICMP, new IP protocols, and whatnot, and we don't need to start with that complexity in the API and the clients.
e.g. /org/fedoraproject/server/role/DomainController
- methods:
- DeployPrimaryDomainController: Set up the first domain controller in the domain.
- DeployReplicaDomainController: Add a new replica domain controller to the domain
See above.
- AddCertificateAuthority: Add a certificate authority to this domain controller
- EnableDisableDNS: Enable or disable the DNS server on this server
This would of course work. A philosophical question, though: would we rather move to closer to the "cattle" model, i.e.
settings = role.GetCurrentSettings() settings.enableDNS = False
role.Redeploy(settings)
?
A "redeploy" is more work for the role, having to compare the settings with current deployment and apply them, but it also ensure that that code path works; with the individual configuration modification operations, the roles could still provide a GetCurrentSettings method, but we would have two completely separate paths (EnableDisableDNS vs. Deploy) to test, and the risk of them diverging.
1) How do we handle configuring the Firewall? I think we want to have
a Firewall object (/org/fedoraproject/server/RoleFirewallManager) available to query the firewall as a whole, but we may also want to be able to view and apply firewall rules from the Role objects directly. (Note: I'm not suggesting we need a complete firewall solution here. This should be a wrapper that deals only with the Roles). Firewalld already provides a more comprehensive firewall interface for the general case.
If we provide a limited firewall facade so that Fedora Server can work without firewalld, IMHO it should literally be the simplest possible "role.FirewallPreventsNonLocalhostAccess" value; not even listing the ports involved. If we provide something closer to a full-featured D-Bus API firewall, we might just as well require firewalld directly instead of writing a firewalld competitor. Mirek
2014-03-24 22:02 GMT+01:00 Miloslav Trmač mitr@volny.cz:
... with the individual configuration modification operations, the roles could still provide a GetCurrentSettings method
BTW the roles *should* provide a GetCurrentSettings in any case, to make it as easy as possible to go from an experimental manually-configured server to a kickstart for mass deployment. Mirek
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/24/2014 05:02 PM, Miloslav Trmač wrote:
2014-03-19 14:47 GMT+01:00 Stephen Gallagher <sgallagh@redhat.com mailto:sgallagh@redhat.com>:
- We do not need to have a single generic API that will work for
any conceivable Role. We've identified in the past that the selection of Featured roles will (for the conceivable future) remain a small, manageable list. For this reason, it is acceptable for the deployment and post-deployment configuration APIs to be entirely different between two roles. However, some functionality such as the backup and firewall queries should be stable between all roles.
(I.e. the universal Role in the proposal doesn't even have a DeployRole method.)
I don't think this is efficient—every "universal" client (and there will be at least one or perhaps two, fedora-role-manage, anaconda/initial-setup kickstart processing) would have to build essentially the same abstraction over these role differences.
Wouldn't it be cleaner to have generic ValidateSettings(settings) DeployRole(settings) as the abstraction used by the "universal" clients (with "settings" possibly having some universal settings, like our favorite firewall_enabled_after_deploy, but mostly role-specific, only using a standard format, perhaps JSON or some object graph with similar capabilities)?
That's an idea. I'm not sure it's doing anything other than moving the problem to the settings object instead of the method call, but *shrug*.
The kickstart processor would only decode JSON and pass it to the API without understanding the contents; the Role-specific GUIs would have specific knowledge of settings to set—or specific Roles could even provide helper APIs, say AutodetectDefaultReplicaControllerSettings() to query the network and give the GUI information about what the role thinks is appropriate for the specific situation.)
*nod*
/org/fedoraproject/server/ServerRoleManager: * properties: * all_roles * staging_roles * active_roles
Isn't this redundant with the per-role properties specifying the role's state? (Even if it were redundant, it wouldn't be a huge deal—just a little work that we might be able to avoid.)
Yes, see Jan's response as well. Better to leave that to the D-BUS object manager.
/org/fedoraproject/server/role/$ROLE: * properties: * loaded: Bool: whether the packages are installed * deployed: Bool: whether the Role has been deployed
(Nitpicking? Rather an enum to eliminate possibilities of paradoxes like deployed & ~loaded.)
See also Jan's reply. We also need a "deploying" state, so maybe an enum is the right idea here.
- methods: * GetFirewallPorts: list of port/protocol pairs that the
Role needs
I'd much rather have FirewallRestriction enum (LocalhostOnly, Unrestricted, Complex) or something like that, or, if we decided to depend on firewalld, an object reference to a firewalld service. Roles will be asking for ICMP, new IP protocols, and whatnot, and we don't need to start with that complexity in the API and the clients.
Sorry, that was a little thin. Could you go into a bit more detail here, please?
e.g. /org/fedoraproject/server/role/DomainController * methods: * DeployPrimaryDomainController: Set up the first domain controller in the domain. * DeployReplicaDomainController: Add a new replica domain controller to the domain
See above.
- AddCertificateAuthority: Add a certificate authority to
this domain controller * EnableDisableDNS: Enable or disable the DNS server on this server
This would of course work. A philosophical question, though: would we rather move to closer to the "cattle" model, i.e.
settings = role.GetCurrentSettings() settings.enableDNS = False
role.Redeploy(settings)
?
A "redeploy" is more work for the role, having to compare the settings with current deployment and apply them, but it also ensure that that code path works; with the individual configuration modification operations, the roles could still provide a GetCurrentSettings method, but we would have two completely separate paths (EnableDisableDNS vs. Deploy) to test, and the risk of them diverging.
Hmm, but redeploy may not be possible (or idempotent) for all possible setting changes. For example, if we deploy FreeIPA with a built-in Certificate Authority, we can't redeploy without it, since the whole system will have been predicated on it. (Not without having a mechanism for redeploy being effectively a complete migration... possibly between VMs or containers?)
- How do we handle configuring the Firewall? I think we want to
have a Firewall object (/org/fedoraproject/server/RoleFirewallManager) available to query the firewall as a whole, but we may also want to be able to view and apply firewall rules from the Role objects directly. (Note: I'm not suggesting we need a complete firewall solution here. This should be a wrapper that deals only with the Roles). Firewalld already provides a more comprehensive firewall interface for the general case.
If we provide a limited firewall facade so that Fedora Server can work without firewalld, IMHO it should literally be the simplest possible "role.FirewallPreventsNonLocalhostAccess" value; not even listing the ports involved. If we provide something closer to a full-featured D-Bus API firewall, we might just as well require firewalld directly instead of writing a firewalld competitor.
I think my original statement failed to get my point across, but I think I was pretty much saying the same thing you are. First of all, we *did* agree to use firewalld as the proper solution. I just meant that in terms of defining the Role-firewall interop, it should *not* try to be a complete set of firewall rules. For that, we should allow a Role to have a 'managed_firewall=False' setting[1] or something like it, and that admins would be expected to handle the firewall appropriately through firewalld/iptables. But if we *are* managing it, it's probably acceptable for it to be essentially either blocked or allowed on a whitelist of interfaces ('lo' being just a special case). Anyone requiring more control than that should set managed_firewall=False and do it themselves.
[1] The reason for this being so that UIs can note that this is under manual control and also so that later Role upgrades don't attempt to open or close ports.
2014-03-25 18:14 GMT+01:00 Stephen Gallagher sgallagh@redhat.com:
On 03/24/2014 05:02 PM, Miloslav Trmač wrote:
2014-03-19 14:47 GMT+01:00 Stephen Gallagher <sgallagh@redhat.com Wouldn't it be cleaner to have generic ValidateSettings(settings) DeployRole(settings) as the abstraction used by the "universal" clients (with "settings" possibly having some universal settings, like our favorite firewall_enabled_after_deploy, but mostly role-specific, only using a standard format, perhaps JSON or some object graph with similar capabilities)?
That's an idea. I'm not sure it's doing anything other than moving the problem to the settings object instead of the method call, but *shrug*.
Right, it would be even better to provide role-specific APIs (or a generic API with schema enforcement) to change values within the settings object :) in order to make typos in settings names obvious, and that point it is really similar.
It does only one structurally significant thing: encode the "settings format" (JSON/whatever) as a canonical way to describe/dump/persistently store role deployment configuration, giving the API users precisely the same facilities they have during installation.
- methods: * GetFirewallPorts: list of port/protocol pairs that the
Role needs
I'd much rather have FirewallRestriction enum (LocalhostOnly, Unrestricted, Complex) or something like that, or, if we decided to depend on firewalld, an object reference to a firewalld service. Roles will be asking for ICMP, new IP protocols, and whatnot, and we don't need to start with that complexity in the API and the clients.
Sorry, that was a little thin. Could you go into a bit more detail here, please?
Stepping back a little, the role *implementation* and *external API* should be different. The implementation obviously does need to actually contain port numbers and the like somewhere, but I don't think we want them in the role's public API.
The reason is that this introduces too much complexity into the API, at a level that isn't interesting to many important users of the API. The complexity is in all the things a role might need (TCP ports, UDP ports, IP protocols, specific multicast addresses, perhaps even conntrack modules), and the presumed callers of a "Role deploy API" just don't care, they want the role deployed firewalled/not firewalled, and a service dashboard needs to to know whether it has been firewalled. Receiving a list of half-a-dozen different individual firewall concepts to check isn't all that helpful, the public API should be able to give a simple yes/no answer (well, a "yes/no/"it's complicated" answer, which are the badly-explained LocalhostOnly/Unrestricted/Complex values above).
Depending on firewalld, it seems simplest for roles to provide firewalld services, and refer to callers to them. We might still want a simple facade above, providing only the ability set LocalhostOnly/Unrestricted, and to receive current status as one of the three values. That would be appropriate for a simple dashboard, more detailed management would be done on the firewalld service object directly using the firewalld API.
- AddCertificateAuthority: Add a certificate authority to
this domain controller * EnableDisableDNS: Enable or disable the DNS server on this server
This would of course work. A philosophical question, though: would we rather move to closer to the "cattle" model, i.e.
settings = role.GetCurrentSettings() settings.enableDNS = False role.Redeploy(settings) ?
A "redeploy" is more work for the role, having to compare the settings with current deployment and apply them, but it also ensure that that code path works; with the individual configuration modification operations, the roles could still provide a GetCurrentSettings method, but we would have two completely separate paths (EnableDisableDNS vs. Deploy) to test, and the risk of them diverging.
Hmm, but redeploy may not be possible (or idempotent) for all possible setting changes.
It's always possible to delete all data and start fresh :) An API to redeploy that doesn't delete data needs the option to fail, sure--but any functionality that can be done using a GetValue/SetValue-like API should be equally possible to do using a Redeploy model; in the role-already-deployed case, it's just a series of
if (settings.option_name != role.GetValue(option_name)) role.setValue(option_name, settings.option_name)
and
if (settings.immutable_option_name != role.GetValue(immutable_option_name) fail;
snippets.
So semantically the two are sort of equivalent, this is primarily a matter of how we want to encourage the users to work with the system: keep a pet and modify it over time with the API, or have a settings file managed in a VCS somewhere and use the API only to sync the VCS and the deployment?
If we provide a limited firewall facade so that Fedora Server can work without firewalld, IMHO it should literally be the simplest possible "role.FirewallPreventsNonLocalhostAccess" value; not even listing the ports involved. If we provide something closer to a full-featured D-Bus API firewall, we might just as well require firewalld directly instead of writing a firewalld competitor.
I think my original statement failed to get my point across, but I think I was pretty much saying the same thing you are.
I think so too.
First of all, we *did* agree to use firewalld as the proper solution. I just meant that in terms of defining the Role-firewall interop, it should *not* try to be a complete set of firewall rules. For that, we should allow a Role to have a 'managed_firewall=False' setting[1] or something like it, and that admins would be expected to handle the firewall appropriately through firewalld/iptables. But if we *are* managing it, it's probably acceptable for it to be essentially either blocked or allowed on a whitelist of interfaces ('lo' being just a special case). Anyone requiring more control than that should set managed_firewall=False and do it themselves.
We could even have the managed_firewall=False implicit, as "service is not assigned to the Public or Drop zones" (... which would leave even the "allow on a whitelist of interfaces" case up to non-Role-API firewalld calls; I have no idea whether this would be appropriate). Mirek
On 03/26/2014 12:17 AM, Miloslav Trmač wrote:
2014-03-25 18:14 GMT+01:00 Stephen Gallagher <sgallagh@redhat.com mailto:sgallagh@redhat.com>:
On 03/24/2014 05:02 PM, Miloslav Trmač wrote:
2014-03-19 14:47 GMT+01:00 Stephen Gallagher <sgallagh@redhat.com
Wouldn't it be cleaner to have generic ValidateSettings(settings) DeployRole(settings) as the abstraction used by the "universal" clients (with "settings" possibly having some universal settings, like our favorite firewall_enabled_after_deploy, but mostly role-specific, only using a standard format, perhaps JSON or some object graph with similar capabilities)?
That's an idea. I'm not sure it's doing anything other than moving the problem to the settings object instead of the method call, but *shrug*.
Right, it would be even better to provide role-specific APIs (or a generic API with schema enforcement) to change values within the settings object :) in order to make typos in settings names obvious, and that point it is really similar.
It does only one structurally significant thing: encode the "settings format" (JSON/whatever) as a canonical way to describe/dump/persistently store role deployment configuration, giving the API users precisely the same facilities they have during installation.
What UDisks and storaged do, they have 'options' parameter of nearly all methods, e.g. BlockDevice.Format(fstype, options).
The options parameter is just dictionary string -> variant, where variant can be literally anything, int, string, array, dictionary of arrays, etc. I think you could represent it as JSON or XML to users.
Supported keys and allowed values are usually documented somewhere and adding new key is not considered as API change, while adding a new method parameter is.
See http://udisks.freedesktop.org/docs/latest/gdbus-org.freedesktop.UDisks2.Bloc... for example.
Also, UDisks has a 'Configuration' property for a block device, which basically sets fstab / crypttab properties + it has methods to manipulate it. I don't know how common is this approach.
See http://udisks.freedesktop.org/docs/latest/gdbus-org.freedesktop.UDisks2.Bloc...
Jan
2014-03-26 8:57 GMT+01:00 Jan Safranek jsafrane@redhat.com:
On 03/26/2014 12:17 AM, Miloslav Trmač wrote:
2014-03-25 18:14 GMT+01:00 Stephen Gallagher <sgallagh@redhat.com mailto:sgallagh@redhat.com>:
On 03/24/2014 05:02 PM, Miloslav Trmač wrote:
2014-03-19 14:47 GMT+01:00 Stephen Gallagher <sgallagh@redhat.com
Wouldn't it be cleaner to have generic ValidateSettings(settings) DeployRole(settings) as the abstraction used by the "universal" clients (with "settings" possibly having some universal settings, like our favorite firewall_enabled_after_deploy, but mostly role-specific, only using a standard format, perhaps JSON or some object graph with similar capabilities)?
That's an idea. I'm not sure it's doing anything other than moving the problem to the settings object instead of the method call, but *shrug*.
Right, it would be even better to provide role-specific APIs (or a generic API with schema enforcement) to change values within the settings object :) in order to make typos in settings names obvious, and that point it is really similar.
It does only one structurally significant thing: encode the "settings format" (JSON/whatever) as a canonical way to describe/dump/persistently store role deployment configuration, giving the API users precisely the same facilities they have during installation.
What UDisks and storaged do, they have 'options' parameter of nearly all methods, e.g. BlockDevice.Format(fstype, options).
The options parameter is just dictionary string -> variant, where variant can be literally anything, int, string, array, dictionary of arrays, etc. I think you could represent it as JSON or XML to users.
Actually a flat dictionary might well be sufficient. I was just mentioning JSON as a way to avoid inventing a plain text representation.
(And, to somewhat contradict the gist of my comments, an explicit API on a settings object, say SetDHCPEnabled(bool) is IMHO generally preferable to passing a dictionary and relying only on non-automatable key name conventions within a dictionary. But starting with the requirement to have the generic API define a plaintext format, the plaintext format is already a way to bypass any explicit API in favor of name conventions, so... ... so you need to keep pushing back against me complicating the API I think :) )
Also, UDisks has a 'Configuration' property for a block device, which basically sets fstab / crypttab properties + it has methods to manipulate it. I don't know how common is this approach.
See http://udisks.freedesktop.org/docs/latest/gdbus-org.freedesktop.UDisks2.Bloc...
I intuitively really don't like this; that's freezing the existing not-allways-that-well-designed configuration as a stable API, while at the same time making the API volatile and changed by any upstream update to the upstream file formats. Any caller willing to deal with the upstream file formats could just as well use them directly.
But it's you who has actual experience with defining and implementing system management APIs, my intuition shouldn't have that much weight. Mirek
server@lists.fedoraproject.org
-
Jan Safranek -
Kevin Fenzi -
Miloslav Trmač -
Stephen Gallagher -
Thomas Woerner -
Tomáš Smetana