Good Morning Everyone,
So yesterday we held a meeting on #fedora-apps about the future of PDC in Fedora. We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora Here is the gist of it:
What do we currently store in PDC: * modules data - the list of what modules have been built, what rpms are in them, and which ones are active or not. * Stream branches, branch ownership, retirement dates (EOL/SLE) * release/life-circle tracking * product and product versions (fedpkg gets active Fedora and EPEL releases from product versions endpont) * critpath boolean on packages (bodhi uses this) * metadata from composes (RPMS/images) * "dependency" data about which rpms depend on which other rpms and which containers include other rpms. This is not used by anything and can be de-prioritized, dropped. (It was originally going to be used in Freshmaker.) Endpoints: * release-component-relationships * release-components * List of all packages: fedora-packages uses this list to know what to index. It sets up the "for loop" from which fedora-packages pulls data from all our other systems. Endpoints: * global-components
After some discussion the decision we reached was: * The "dependency" data in its current form can be dropped. It's something we'll need in the future but the data structure will most likely need to be adjusted so let's just drop the existing data and worry about this later. * The critpath boolean can just be imported into bodhi itself, especially considering that bodhi is the only tool using this flag. * The modules data is entirely going to move into MBS
Everything else (ie: composes metadata, release/life-circle tracking, package/branches information) will need to find a new home. This new home will be a new django app using the Django Rest Framework (DRF) in which we will import the PDC apps as needed (potentially adjusting them where desired).
- kellin has agreed to be the project leader for this work but needs a team to do the actual work. - pingou has agreed to look for said team
If you have any worries, questions or suggestions or if you want to join the effort, please let us know :)
Finally some links: Minutes: https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018... Minutes (text): https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018... Log: https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018...
Have a nice day, Pierre
On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote:
Good Morning Everyone,
So yesterday we held a meeting on #fedora-apps about the future of PDC in Fedora. We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora Here is the gist of it:
What do we currently store in PDC:
- modules data - the list of what modules have been built, what rpms are in them, and which ones are active or not.
- Stream branches, branch ownership, retirement dates (EOL/SLE)
- release/life-circle tracking
from product versions endpont)
- product and product versions (fedpkg gets active Fedora and EPEL releases
As an aside did we ever get that 'in development' addition we wanted so we could point everything still using pkgdb to pdc for release status?
- critpath boolean on packages (bodhi uses this)
- metadata from composes (RPMS/images)
- "dependency" data about which rpms depend on which other rpms and which containers include other rpms. This is not used by anything and can be de-prioritized, dropped. (It was originally going to be used in Freshmaker.) Endpoints:
- release-component-relationships
- release-components
- List of all packages: fedora-packages uses this list to know what to index. It sets up the "for loop" from which fedora-packages pulls data from all our other systems. Endpoints: * global-components
After some discussion the decision we reached was:
- The "dependency" data in its current form can be dropped. It's something we'll need in the future but the data structure will most likely need to be adjusted so let's just drop the existing data and worry about this later.
- The critpath boolean can just be imported into bodhi itself, especially considering that bodhi is the only tool using this flag.
ok. Note that this data changes over time, and releng needs a script to update it (or better a automatic updating of it).
We might want to bring up the bigger topic of if we want to bother keeping this. The only current use of it is some rules about going stable in bodhi... are those actually useful?
- The modules data is entirely going to move into MBS
Everything else (ie: composes metadata, release/life-circle tracking, package/branches information) will need to find a new home. This new home will be a new django app using the Django Rest Framework (DRF) in which we will import the PDC apps as needed (potentially adjusting them where desired).
- kellin has agreed to be the project leader for this work but needs a team to do the actual work.
- pingou has agreed to look for said team
If you have any worries, questions or suggestions or if you want to join the effort, please let us know :)
I'm a little worried about Django. True, we have to maintain a version for mailman3, but it's rhel7/python3. Is this new app going to use that? Alternately if we use Fedora, we need to adjust to new Django versions pretty often (one problem we already hit with PDC).
Since this is just a simple api, could we do something more simple?
Thanks for leading this effort pingou!!!
kevin
On 06/13/2018 03:03 PM, Kevin Fenzi wrote:
ok. Note that this data changes over time, and releng needs a script to update it (or better a automatic updating of it).
Yeah there is a Bodhi ticket about this and I noted that we will need to make sure we still work with releng's script if we make the change:
https://github.com/fedora-infra/bodhi/issues/2433
We might want to bring up the bigger topic of if we want to bother keeping this. The only current use of it is some rules about going stable in bodhi... are those actually useful?
This sounds like a policy decision, so perhaps the question could be posed to FESCo or possibly the packaging committee. It does make some intuitive sense to me that we would treat certain packages more stringently than we do others, but I don't have data to say one way or the other whether the current policies are beneficial. I will say that I would feel uncomfortable changing the policy without the blessing of a governing body.
On 06/13/2018 02:50 PM, Randy Barlow wrote:
On 06/13/2018 03:03 PM, Kevin Fenzi wrote:
ok. Note that this data changes over time, and releng needs a script to update it (or better a automatic updating of it).
Yeah there is a Bodhi ticket about this and I noted that we will need to make sure we still work with releng's script if we make the change:
Yeah, it sure would be nice if we could just have something run in cron once a day or so and just update it. Releng has not been good about running this script often.
We might want to bring up the bigger topic of if we want to bother keeping this. The only current use of it is some rules about going stable in bodhi... are those actually useful?
This sounds like a policy decision, so perhaps the question could be posed to FESCo or possibly the packaging committee. It does make some intuitive sense to me that we would treat certain packages more stringently than we do others, but I don't have data to say one way or the other whether the current policies are beneficial. I will say that I would feel uncomfortable changing the policy without the blessing of a governing body.
Yes, this is definitely something for FESCo. They made the update policy that uses it.
I can take it to them, I just wanted to see if there was a general sense that it was useful and we should keep it, or it was pointless and we should drop it. Perhaps I'll post to devel about it to see what the general feeling is.
kevin
On 06/13/2018 08:04 PM, Kevin Fenzi wrote:
I can take it to them, I just wanted to see if there was a general sense that it was useful and we should keep it, or it was pointless and we should drop it. Perhaps I'll post to devel about it to see what the general feeling is.
I only have a slight "intuition" that it seems useful to me, but that's a rather weak statement and I certainly wouldn't argue about it. If you feel more strongly about it, +1 to raising it for discussion!
On Wed, 2018-06-13 at 17:04 -0700, Kevin Fenzi wrote:
On 06/13/2018 02:50 PM, Randy Barlow wrote:
On 06/13/2018 03:03 PM, Kevin Fenzi wrote:
ok. Note that this data changes over time, and releng needs a script to update it (or better a automatic updating of it).
Yeah there is a Bodhi ticket about this and I noted that we will need to make sure we still work with releng's script if we make the change:
Yeah, it sure would be nice if we could just have something run in cron once a day or so and just update it. Releng has not been good about running this script often.
We might want to bring up the bigger topic of if we want to bother keeping this. The only current use of it is some rules about going stable in bodhi... are those actually useful?
This sounds like a policy decision, so perhaps the question could be posed to FESCo or possibly the packaging committee. It does make some intuitive sense to me that we would treat certain packages more stringently than we do others, but I don't have data to say one way or the other whether the current policies are beneficial. I will say that I would feel uncomfortable changing the policy without the blessing of a governing body.
Yes, this is definitely something for FESCo. They made the update policy that uses it.
I can take it to them, I just wanted to see if there was a general sense that it was useful and we should keep it, or it was pointless and we should drop it. Perhaps I'll post to devel about it to see what the general feeling is.
FWIW I certainly think we should keep it. If anything, with the few folks we currently have who seem to +1 almost any update two seconds after it lands, we might want to make the barrier a bit higher.
On Wed, 2018-06-13 at 12:03 -0700, Kevin Fenzi wrote:
On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote:
Good Morning Everyone,
So yesterday we held a meeting on #fedora-apps about the future of PDC in Fedora. We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora Here is the gist of it:
What do we currently store in PDC:
- modules data - the list of what modules have been built, what rpms are in them, and which ones are active or not.
- Stream branches, branch ownership, retirement dates (EOL/SLE)
- release/life-circle tracking
from product versions endpont)
- product and product versions (fedpkg gets active Fedora and EPEL releases
As an aside did we ever get that 'in development' addition we wanted so we could point everything still using pkgdb to pdc for release status?
All the code got done but no-one has yet/ever put a process in place to actually populate Fedora's PDC with the appropriate data, so it's not there. Things (fedfind, gnome-software...) are still using the 'collections' JSON for now.
On 06/13/2018 03:19 PM, Adam Williamson wrote:
On Wed, 2018-06-13 at 12:03 -0700, Kevin Fenzi wrote:
On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote:
Good Morning Everyone,
So yesterday we held a meeting on #fedora-apps about the future of PDC in Fedora. We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora Here is the gist of it:
What do we currently store in PDC:
- modules data - the list of what modules have been built, what rpms are in them, and which ones are active or not.
- Stream branches, branch ownership, retirement dates (EOL/SLE)
- release/life-circle tracking
from product versions endpont)
- product and product versions (fedpkg gets active Fedora and EPEL releases
As an aside did we ever get that 'in development' addition we wanted so we could point everything still using pkgdb to pdc for release status?
All the code got done but no-one has yet/ever put a process in place to actually populate Fedora's PDC with the appropriate data, so it's not there. Things (fedfind, gnome-software...) are still using the 'collections' JSON for now.
Is that the last thing using pkgdb that we know of?
It would be pretty sad to keep running pkgdb, and pdc and the new thing all at the same time. :(
kevin
On Wed, 2018-06-13 at 17:06 -0700, Kevin Fenzi wrote:
On 06/13/2018 03:19 PM, Adam Williamson wrote:
On Wed, 2018-06-13 at 12:03 -0700, Kevin Fenzi wrote:
On 06/12/2018 07:41 AM, Pierre-Yves Chibon wrote:
Good Morning Everyone,
So yesterday we held a meeting on #fedora-apps about the future of PDC in Fedora. We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora Here is the gist of it:
What do we currently store in PDC:
- modules data - the list of what modules have been built, what rpms are in them, and which ones are active or not.
- Stream branches, branch ownership, retirement dates (EOL/SLE)
- release/life-circle tracking
from product versions endpont)
- product and product versions (fedpkg gets active Fedora and EPEL releases
As an aside did we ever get that 'in development' addition we wanted so we could point everything still using pkgdb to pdc for release status?
All the code got done but no-one has yet/ever put a process in place to actually populate Fedora's PDC with the appropriate data, so it's not there. Things (fedfind, gnome-software...) are still using the 'collections' JSON for now.
Is that the last thing using pkgdb that we know of?
It would be pretty sad to keep running pkgdb, and pdc and the new thing all at the same time. :(
It's the last thing *I* know of.
I'm a little worried about Django. True, we have to maintain a version for mailman3, but it's rhel7/python3. Is this new app going to use that?
Actually, HyperKitty and Postorius are using Django on Python 2.7. The Django version is 1.8 and it's pretty old now. I would recommend against starting a new app on Python 2 today and it does not look like we have a Python 3 package of Django in EPEL yet.
Alternately if we use Fedora, we need to adjust to new Django versions pretty often (one problem we already hit with PDC).
Would it make sense to run it in OpenShift? I'd think so. Then we could build it with whichever version works, right?
Since this is just a simple api, could we do something more simple?
The thing is that the Django REST Framework library is really wonderful and there is no Flask equivalent that I know of. It would save us handling of a lot of corner cases, and it has built-in tools for versionning the API and thus preserving API compatibility. Authentication is also very flexible, etc. It's nice.
That said, nothing impossible to do in Flask, just longer and possibly more error-prone.
Aurélien
On 06/18/2018 01:25 PM, Aurelien Bompard wrote:
I'm a little worried about Django. True, we have to maintain a version for mailman3, but it's rhel7/python3. Is this new app going to use that?
Actually, HyperKitty and Postorius are using Django on Python 2.7. The Django version is 1.8 and it's pretty old now. I would recommend against starting a new app on Python 2 today and it does not look like we have a Python 3 package of Django in EPEL yet.
Yeah, this is the part that worries me... Django 1.8 went out of support on April 1st (no joke!).
Alternately if we use Fedora, we need to adjust to new Django versions pretty often (one problem we already hit with PDC).
Would it make sense to run it in OpenShift? I'd think so. Then we could build it with whichever version works, right?
Within limits. It should be a version thats supported and gets at least security updates. Hopefully the one(s) in Fedora follow this.
Since this is just a simple api, could we do something more simple?
The thing is that the Django REST Framework library is really wonderful and there is no Flask equivalent that I know of. It would save us handling of a lot of corner cases, and it has built-in tools for versionning the API and thus preserving API compatibility. Authentication is also very flexible, etc. It's nice.
That said, nothing impossible to do in Flask, just longer and possibly more error-prone.
Yeah, understood. I'd just like to make sure we have security support and aren't leaving ourselves exposed. :(
There are a few flask rest frameworks, but I have not much idea how well they are supported or work.
https://github.com/flask-restful/flask-restful
https://github.com/noirbizarre/flask-restplus
kevin
Within limits. It should be a version thats supported and gets at least security updates. Hopefully the one(s) in Fedora follow this.
Yeah it's 1.11 now which is LTS, since it'll be the last version to support Python 2
There are a few flask rest frameworks, but I have not much idea how well they are supported or work. https://github.com/flask-restful/flask-restful https://github.com/noirbizarre/flask-restplus
Yeah I tried those in my search for something like DRF in the Flask world. They are decent, but far from DRF feature-wise.
A.
On Tue, 19 Jun 2018 at 01:13, Aurelien Bompard abompard@fedoraproject.org wrote:
Within limits. It should be a version thats supported and gets at least security updates. Hopefully the one(s) in Fedora follow this.
Yeah it's 1.11 now which is LTS, since it'll be the last version to support Python 2
There are a few flask rest frameworks, but I have not much idea how well they are supported or work. https://github.com/flask-restful/flask-restful https://github.com/noirbizarre/flask-restplus
Yeah I tried those in my search for something like DRF in the Flask world. They are decent, but far from DRF feature-wise.
I still think we should keep Django, but I came across this tutorial [0] yesterday, and it looks like this is a neat way of dealing with REST API using Flask and connexion [1]
[0] - https://realpython.com/flask-connexion-rest-api/ [1] - https://github.com/zalando/connexion
A. _______________________________________________ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedorapro...
On Tue, Jun 19, 2018 at 08:35:54AM +0200, Clement Verna wrote:
On Tue, 19 Jun 2018 at 01:13, Aurelien Bompard abompard@fedoraproject.org wrote:
Within limits. It should be a version thats supported and gets at least security updates. Hopefully the one(s) in Fedora follow this.
Yeah it's 1.11 now which is LTS, since it'll be the last version to support Python 2
There are a few flask rest frameworks, but I have not much idea how well they are supported or work. https://github.com/flask-restful/flask-restful https://github.com/noirbizarre/flask-restplus
Yeah I tried those in my search for something like DRF in the Flask world. They are decent, but far from DRF feature-wise.
I still think we should keep Django, but I came across this tutorial [0] yesterday, and it looks like this is a neat way of dealing with REST API using Flask and connexion [1]
[0] - https://realpython.com/flask-connexion-rest-api/ [1] - https://github.com/zalando/connexion
The tutorial is quite tempting, it looks quite nice, but the entire auth aspect isn't covered (though we do have flask-oidc).
Pierre
Actually, I haven't tried that one. It seems pretty good (from the docs), has anybody tried it?
On Tue, Jun 19, 2018 at 09:39:25AM +0200, Aurelien Bompard wrote:
Actually, I haven't tried that one. It seems pretty good (from the docs), has anybody tried it?
Seen it but not tried it :(
Pierre
On Tue, Jun 19, 2018 at 09:39:25AM +0200, Aurelien Bompard wrote:
Actually, I haven't tried that one. It seems pretty good (from the docs), has anybody tried it?
It does expose API doc using swagger as does the Connexion extension Clément just pointed to.
Pierre
On Tue, 12 Jun 2018 at 16:48, Pierre-Yves Chibon pingou@pingoured.fr wrote:
Good Morning Everyone,
So yesterday we held a meeting on #fedora-apps about the future of PDC in Fedora. We kept notes in etherpad at: http://etherpad.osuosl.org/pdc_fedora Here is the gist of it:
What do we currently store in PDC:
- modules data - the list of what modules have been built, what rpms are in them, and which ones are active or not.
- Stream branches, branch ownership, retirement dates (EOL/SLE)
- release/life-circle tracking
from product versions endpont)
- product and product versions (fedpkg gets active Fedora and EPEL releases
- critpath boolean on packages (bodhi uses this)
- metadata from composes (RPMS/images)
- "dependency" data about which rpms depend on which other rpms and which containers include other rpms. This is not used by anything and can be de-prioritized, dropped. (It was originally going to be used in Freshmaker.) Endpoints:
- release-component-relationships
- release-components
- List of all packages: fedora-packages uses this list to know what to index. It sets up the "for loop" from which fedora-packages pulls data from all our other systems. Endpoints: * global-components
After some discussion the decision we reached was:
- The "dependency" data in its current form can be dropped. It's something we'll need in the future but the data structure will most likely need to be adjusted so let's just drop the existing data and worry about this later.
- The critpath boolean can just be imported into bodhi itself, especially considering that bodhi is the only tool using this flag.
During our last infrastructure meeting [0], tflink mentioned that the critpath boolean was also used by taskotron. Currently taskotron still uses pkgdb to get this info, so taskotron will need to be updated to use the new source of truth for the critpathwe might reconsider moving this to bodhi and leave in the PDC replacement.
[0] - https://meetbot.fedoraproject.org/teams/infrastructure/infrastructure.2018-0...
- The modules data is entirely going to move into MBS
Everything else (ie: composes metadata, release/life-circle tracking, package/branches information) will need to find a new home. This new home will be a new django app using the Django Rest Framework (DRF) in which we will import the PDC apps as needed (potentially adjusting them where desired).
- kellin has agreed to be the project leader for this work but needs a team to do the actual work.
- pingou has agreed to look for said team
If you have any worries, questions or suggestions or if you want to join the effort, please let us know :)
Finally some links: Minutes: https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018... Minutes (text): https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018... Log: https://meetbot.fedoraproject.org/fedora-apps/2018-06-11/pdc_and_fedora.2018...
Have a nice day, Pierre _______________________________________________ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-leave@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedorapro...
infrastructure@lists.fedoraproject.org
-
Adam Williamson -
Aurelien Bompard -
Clement Verna -
Kevin Fenzi -
Pierre-Yves Chibon -
Randy Barlow