devel March 2020

devel@lists.stg.fedoraproject.org

235 participants
484 discussions

HEADS UP: repoquery --whatrequires yields incomplete results when run from Fedora 32+
by Miro Hrončok 27 Apr '20

27 Apr '20

I have upgraded to Fedora 32 today and after a while I have noticed that `repoquery --whatrequires` yields incomplete results. From Fedora 31: $ dnf --refresh repoquery --repo=rawhide{,-source} --whatrequires python3-flaky pipenv-0:2018.11.26-13.fc32.src python-ipykernel-0:5.1.4-1.fc33.src From Fedora 32/33: $ dnf --refresh repoquery --repo=rawhide{,-source} --whatrequires python3-flaky python-ipykernel-0:5.1.4-1.fc33.src I have reported this here: https://bugzilla.redhat.com/show_bug.cgi?id=1812596 Be extra careful when taking decisions based on `repoquery --whatrequires` (such as: I can safely retire this or upgrade that, nothing else requires it). -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

3 4

Fedora 32 Self-Contained Change proposal: Build Python with -fno-semantic-interposition for better performance
by Ben Cotton 23 Apr '20

23 Apr '20

https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup Simplified version of another change proposal|This change was originally proposed for [[Releases/32|Fedora 32]] as [[Changes/PythonStaticSpeedup]], however based on [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… community feedback], it has been significantly reduced. == Summary == We add the <code>-fno-semantic-interposition</code> compiler/linker flag when building Python interpreters, as it provides significant performance improvement, up to 27% depending on the workload. Users will no longer be able to use LD_PRELOAD to override a symbol from libpython, which we consider a good trade off for the speedup. == Owner == * Name: [[User:Cstratak| Charalampos Stratakis]], [[User:Vstinner| Victor Stinner]], [[User:Churchyard| Miro Hrončok]] * Email: python-maint(a)redhat.com * Shout-out: [[User:Jankratochvil|Jan Kratochvíl]] for first suggesting this instead of the original proposal, followed by [[User:Kkofler|Kevin Kofler]]. [[User:Fweimer|Florian Weimer]] for providing answers to our questions. David Gray for originally suggesting to link Python statically to gain performance. == Detailed Description == When we build the Python interpreter with the <code>-fno-semantic-interposition</code> compiler/linker flag, we can achieve a performance gain of 5% to 27% depending on the workload. Link time optimizations and profile guided optimizations also have a greater impact when python3 is built this way. As a negative side effect, it disables the LD_PRELOAD feature: it's no longer possible to override symbols in libpython with LD_PRELOAD. Interposition is enabled by default in compilers like GCC: function calls to a library goes through a "Procedure Linkage Table" (PLT). This indirection is required to allow a library loaded by LD_PRELOAD environment variable to override a function. The indirection puts more pressure on the CPU level 1 cache (instruction cache). In term of performance, the main drawback is that function calls from a library to the same library cannot be inlined, to respect the interposition semantics. Inlining is usually a big win in term of performance. Disabling interposition for libpython removes the overhead on function calls by avoiding the PLT indirection, and allows to inline more function calls. We're describing function calls from libpython to libpython, something which is very common in Python: almost all function calls are calls from libpython to libpython. If Fedora users need to use LD_PRELOAD to override symbols in libpython, the recommend way is to build a custom Python without <code>-fno-semantic-interposition</code>. It is still possible to use LD_PRELOAD to override symbols in other libraries (for example in glibc). === Affected Pythons === Primarily, we will change the interpreter in the {{package|python3}} package, that is Python 3.8 in Fedora 32 and any later version of Python in future Fedora releases. Impact on other Python packages (and generally software using Python) is not anticipated (other than the possible speedup). We will also change the [https://developer.fedoraproject.org/tech/languages/python/multiple-pythons.… alternate Python interpreters] where possible and useful, primarily the upstream supported versions of CPython, such as {{package|python39}} (if already packaged), {{package|python37}} and {{package|python36}}. === Affected Fedora releases === This is a Fedora 32 change and it will be implemented in Rawhide (Fedora 32) only. Any future versions of Fedora will inherit the change until it is reverted for some reason. If it turns out that there are absolutely no issues, we might consider backporting the speedup to already released Fedora versions (for example Fedora 31). Such action would be separately coordinated with [https://docs.fedoraproject.org/en-US/fesco/ FESCo]. == Benefit to Fedora == Python's performance will increase significantly depending on the workload. Since many core components of the OS also depend on Python this could lead to an increase in their performance as well, however individual benchmarks will need to be conducted to verify the performance gain for those components. [https://pyperformance.readthedocs.io/ pyperformance] results, ignoring differences smaller than 5%: (See change proposal) == Scope == * Proposal owners: ** Review and merge the [https://src.fedoraproject.org/rpms/python3/pull-request/151 pull request with the implementation]. ** Monitor Koschei for significant problems. ** Backport the change to alternate Python versions. * Other developers are encouraged to check if their package works as expected * Release engineering: N/A (not needed for this Change) -- this change does not require a mass rebuild nor any other special releng work * Policies and guidelines: N/A (not needed for this Change) * Trademark approval: N/A (not needed for this Change) == Upgrade/compatibility impact == Python package maintainers should verify that their packages work as expected and the only impact the end users should see is a performance increase for workloads relying on Python. == How To Test == Test that everything Python related in Fedora works as usual. === Was the flag applied test === You can test whether the <code>-fno-semantic-interposition</code> flag was applied for your Python build: <pre> >>> import sysconfig >>> '-fno-semantic-interposition' in (sysconfig.get_config_var('PY_CFLAGS') + sysconfig.get_config_var('PY_CFLAGS_NODIST')) True >>> '-fno-semantic-interposition' in (sysconfig.get_config_var('PY_LDFLAGS') + sysconfig.get_config_var('PY_LDFLAGS_NODIST')) True </pre> Before the change, you would see <code>False</code>, <code>False</code>. === Performance test === The performance speedup can be measured using the official Python benchmark suite [https://pyperformance.readthedocs.io/ pyperformance]: see [https://pyperformance.readthedocs.io/usage.html#run-benchmarks Run benchmarks]. == User Experience == Python based workloads should see a performance gain of up to 27%. == Dependencies == This change is not dependent on anything else. == Contingency Plan == * Contingency mechanism: If issues appear that cannot be fixed in a timely manner the change can be easily reverted and will be considered again for the next fedora release. * Contingency deadline: Before the beta freeze of Fedora 32 (2020-02-25) * Blocks release? Yes * Blocks product? None == Documentation == This change proposal has all the documentation. See the [[Changes/PythonStaticSpeedup|previous change proposal]] and the [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… thread about it on the devel mailing list] for more relevant information about what we are not doing -- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis

2 1

fedpkg clone fails with Permission denied (publickey).
by Richard Shaw 20 Apr '20

20 Apr '20

Long story short I lost my home directory where I do all of my packager activities (separate from my main user) so I'm setting things up from scratch. I created new ssh keys and uploaded the public key to admin.fedoraproject.org and pasted into pagure.io. It's been over an hour and I'm still getting: $ fedpkg clone hamlib Cloning into 'hamlib'... hobbes1069(a)pkgs.fedoraproject.org: Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. Could not execute clone: Failed to execute command. I've also updated my API tokens, which is STILL not well documented. I pasted them in the appropriate spot in "/etc/rpkg/fedpkg.conf" which isn't real intuitive. Thanks, Richard

3 8

Announcing new anitya integration and de-orphaning process
by Pierre-Yves Chibon 16 Apr '20

16 Apr '20

Good Morning Everyone, Tomorrow we are planning on deploying a new version of pagure and pagure-dist-git on production. These changes come with two changes to the packager workflow: * Anitya integration in dist-git Something we lost when loosing pkgdb was the easy integration to anitya (https://release-monitoring.org) With the coming changes we are getting them back. On the left hand-side column, there will be a drop-down button allowing to change the settings for anitya for the project. Existing status will be migrated from the fedora-scm-requests repo on pagure to use this drop-down. Using the fedora-scm-requests repo for the anitya integration will no longer be supported. * Change in the de-orphaning process Currently if a package is orphaned, one has to open a ticket against the releng project to adopt it. With these changes, anyone will be able to adopt orphaned projects (not retired on master) directly from dist-git's UI. If the project is retired or has been orphaned for too long, a ticket on the releng project will still be required though. Both of these changes can already be reviewed in staging at: https://src.stg.fedoraproject.org Looking forward for your feedback! Pierre _______________________________________________ devel-announce mailing list -- devel-announce(a)lists.fedoraproject.org To unsubscribe send an email to devel-announce-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedorapr…

13 17

Outage: Migration of Copr servers - 2020-02-23 06:00 UTC
by Jakub Kadlcik 13 Apr '20

13 Apr '20

There will be an outage starting at 2020-02-23 06:00 UTC, which will last approximately 7 hours (but can take more if something goes wrong). To convert UTC to your local time, take a look at https://fedoraproject.org/wiki/UTCHowto or run: date -d '2020-02-23 06:00 UTC' Reason for outage: Moving Copr servers from OpenStack to Amazon AWS because OpenStack is going to be shut down soon. Affected Services: copr-frontend - https://copr.fedorainfracloud.org copr-backend - https://copr-be.cloud.fedoraproject.org/ copr-distgit copr-keygen Backend repositories should remain available during the outage. Ticket Link: https://pagure.io/fedora-infrastructure/issue/8668 Contact Information: Please join #fedora-admin in irc.freenode.net or add comments to the ticket for this outage above. Jakub

5 11

python-msgpack
by Fabian Affolter 11 Apr '20

11 Apr '20

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, A month ago was msgpack 1.0.0 released. I'm planning to update the package in rawhide in the near future. This should start a conversation about a coordinated approach to keep the update smooth for all packages which depends on python-msgpack. Especially borgbackup as there is a restriction in place that will break installations. Kind regards, Fabian -----BEGIN PGP SIGNATURE----- iF0EARECAB0WIQQvbJMP08R+OGr6TrTiPNLdNqQ5fwUCXnnltQAKCRDiPNLdNqQ5 f7t6AJ9j7JdEqUqp3YXfPB4PGX04HunqqACgjhzp4EmVy644q+nzcSTP1J4W+mY= =D1+a -----END PGP SIGNATURE-----

2 2

Fedora 33 System-Wide Change proposal: Introduce module Obsoletes and EOL
by Ben Cotton 11 Apr '20

11 Apr '20

https://fedoraproject.org/wiki/Changes/Module_Obsoletes_and_EOL == Summary == Fix Fedora upgradability issues when upgrading systems with module streams enabled. == Owner == * Name: [[User:dmach| Daniel Mach]] * Email: dmach(a)redhat.com == Detailed Description == DNF currently doesn't have sufficient information to perform system upgrades of systems with enabled module streams correctly. To solve upgradability problems, we will add additional modulemd metadata. This includes information about a stream obsoleting another one or a stream being EOLed. Use Cases: * A module (all its streams) is removed from a distribution. * A stream has reached its EOL and must no longer be available. ** The stream packages have no replacements and must be removed from the system (an extension of fedora-obsolete-packages?) ** The stream packages get replaced with non-modular packages * A stream has reached its EOL and is Obsoleted by another stream. * A stream has reached its EOL and a user wants to continue using it regardless of EOL/Obsoletes. * A stream has reached its EOL and a user wants to install a new host with the EOLed stream enabled. There is going to be a new DNF config option: * (name: TBD) - automatically follow stream EOL/Obsoletes; only print a warning if turned off Proposed new modulemd document: <pre> document: TBD version: 1 data: # A string representing UTC date in ISO 8601 format: YYYY-MM-DD[T ]HH:MMZ # When merging, entries with a newer 'modified' value will override any earlier values. # MANDATORY modified: 2020-03-19T23:26Z # A boolean option to cancel/reset all previously specified obsoletes # Example: repo 'fedora' has Obsoletes:nodejs:12; we want to bring nodejs:12 back in 'updates' # If used, following options will be ignored: eol_date, obsoleted_by # OPTIONAL <TBD>: <bool> # A string representing a Name of a module that is EOLed # MANDATORY module: nodejs # A string representing a Stream of a module that is EOLed # MANDATORY stream: 11 # A string representing a Context of a module that is EOLed # If not specified, all contexts get EOLed. # NOTE: consider specifying a list of contexts # OPTIONAL context: aabbccddee # A string representing UTC date in ISO 8601 format: YYYY-MM-DD[T ]HH:MMZ # It is strongly recommended to keep HH:MM to 00:00. # If not specified, the module is EOLed immediately. OPTIONAL eol_date: 2020-03-19T00:00Z # A string describing the change, reason, etc. # MANDATORY message: "Module stream nodejs:11 is no longer supported. It is recommended to switch to nodejs:12" # If a stream is not EOLed but Obsoleted, provide details about the obsoleting stream: # OPTIONAL obsoleted_by: module: nodejs stream: 12 </pre> == Benefit to Fedora == Seamless system upgrades of systems with module streams enabled. == Scope == * Proposal owners: ** Introduce modularity features to: ** Obsolete a module stream ** Stream EOL ** Design a new modulemd documents for Obsoletes and EOL ** Get the new documents supported by libmodulemd ** Implement the new functionality in the DNF stack (libdnf, dnf, microdnf) and PackageKit * Other developers: ** Follow updated packaging policy. See the "Policies and guidelines" section. * Release engineering: [https://pagure.io/releng/issue/9359] (a check of an impact with Release Engineering is needed)  Maintain and distribute new module metadata. * Policies and guidelines: ** Packaging policy and modularity documents require a change: *** When a module stream gets removed from a Fedora release, the maintainer of the module stream must provide a modulemd document with Obsoletes or EOL data. * Trademark approval: N/A (not needed for this Change) == Upgrade/compatibility impact == This feature solves upgradability problems of systems with module streams enabled. == How To Test == Stream Obsoletes: * Enable a stream that is not available in the next version of Fedora and is obsoleted by another stream in the metadata * Run system upgrade * Verify that the obsoleting stream is enabled instead of the obsoleted one Stream EOL: * Enable a stream that is not available in the next version of Fedora and is obsoleted in its module metadata * Run system upgrade * Verify that the EOLed stream is no longer enabled and does not show up in `dnf module list` Fedora upgradability: * Install Fedora 31 or 32 * For each $stream in available streams ** dnf module reset '*' ** dnf module enable $stream ** dnf distro-sync --releasever=33 --assumeno ** # the command must not throw any errors related to modularity, especially related to the enabled stream == User Experience == Fedora users experience upgradability problems when upgrading to Fedora 32 when they have any module streams enabled. If a stream no longer exists on the version of Fedora they are upgrading to, DNF used to error out on resolving modular dependencies which was a clear release blocker. To workaround this case, all module streams are reset during the system upgrade. By doing this, modularity users lose information about enabled streams and they need to re-enable them by hand. This Change aims at removing the upgradability problems and allowing Fedora modularity users to upgrade their systems while keeping their streams enabled, reset or switched (obsoleted) according to newly provided modular metadata. == Dependencies == * libmodulemd * libdnf * dnf * microdnf * PackageKit == Contingency Plan == * Contingency mechanism: Remove the new metadata from Fedora composes and repositories. Fedora will stay at the current behavior. * Contingency deadline: Beta Freeze * Blocks release? Yes == Documentation == TBD == Release Notes == Modularity supports Obsoleting streams and stream EOL (end-of-life). Enabled streams are no longer reset during the system upgrade. When a stream reaches its EOL, it's reset. When a stream is Obsoleted, it's switched to the obsoleting stream. -- Ben Cotton He / Him / His Senior Program Manager, Fedora & CentOS Stream Red Hat TZ=America/Indiana/Indianapolis

8 17

Schedule for Thursday's FPC Meeting (2019-08-15 16:00 UTC)
by James Antill 09 Apr '20

09 Apr '20

Following is the list of topics that will be discussed in the FPC meeting Thursday at 2019-08-15 16:00 UTC in #fedora-meeting-1 on irc.freenode.net. Local time information (via. uitime): ================= Day: Thursday ================== 2019-08-15 09:00 PDT US/Pacific 2019-08-15 12:00 EDT --> US/Eastern <-- 2019-08-15 16:00 UTC UTC 2019-08-15 17:00 BST Europe/London 2019-08-15 18:00 CEST Europe/Berlin 2019-08-15 18:00 CEST Europe/Paris 2019-08-15 21:30 IST Asia/Calcutta ---------------- New Day: Friday ----------------- 2019-08-16 00:00 HKT Asia/Hong_Kong 2019-08-16 00:00 +08 Asia/Singapore 2019-08-16 01:00 JST Asia/Tokyo 2019-08-16 02:00 AEST Australia/Brisbane Links to all tickets below can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting = Followups = #topic #902 Cleanup & enhance spec files .fpc 902 https://pagure.io/packaging-committee/issue/902 #topic #904 Caret versioning .fpc 904 https://pagure.io/packaging-committee/issue/904 #topic #907 Which %__foo macros for executables are acceptable? .fpc 907 https://pagure.io/packaging-committee/issue/907 #topic #909 Suggest that linting/measuring-coverage is not for %check .fpc 909 https://pagure.io/packaging-committee/issue/909 #topic #914 Automatic R runtime dependencies .fpc 914 https://pagure.io/packaging-committee/issue/914 = Open Floor = For more complete details, please visit each individual ticket. The report of the agenda items can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting If you would like to add something to this agenda, you can: * Reply to this e-mail * File a new ticket at: https://pagure.io/packaging-committee * E-mail me directly * Bring it up at the end of the meeting, during the open floor topic. Note that added topics may be deferred until the following meeting.

2 1

CPE Git Forge Decision
by Leigh Griffin 08 Apr '20

08 Apr '20

Hi everyone, Thank you for your patience while the CPE Team worked through an incredible number of requirements from multiple stakeholder sources. On Friday evening we announced on the Community Blog <https://communityblog.fedoraproject.org/making-a-git-forge-decision/> our decision to adopt Gitlab as our Git Forge and to retain pagure.io to ultimately hand over to the Community to maintain. It wasn't an easy decision by any stretch of the imagination and we hope that the compromise that we are striking will help to allow Pagure flourish and to give a choice of Forges for your usage. I'm happy to field any questions or comments about this decision. Kind regards, Leigh, on behalf of the CPE Team -- Leigh Griffin Engineering Manager Red Hat Waterford <https://www.redhat.com/> Communications House Cork Road, Waterford City lgriffin(a)redhat.com M: +353877545162 IM: lgriffin @redhatjobs <https://twitter.com/redhatjobs> redhatjobs <https://www.facebook.com/redhatjobs> @redhatjobs <https://instagram.com/redhatjobs> <https://red.ht/sig>

61 277

libcdio soname bump in rawhide
by Adrian Reber 07 Apr '20

07 Apr '20

I will update libcdio to 2.1.0 next week in rawhide. As always, libcdio comes with a new SO version. The following packages have to be rebuilt and I will do the necessary rebuilds: cantata cdw clementine gstreamer1-plugins-ugly-free gvfs kover libcddb libcdio-0:2.0.0-6.fc32.src libcdio-paranoia pcsxr pragha python-pycdio qmmp strawberry tellico whipper xmms2 Adrian

1 1

← Newer
1
2
3
4
5
6
7
...
49
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

devel March 2020