On 01/23/2014 05:02 AM, Siddhesh Poyarekar wrote:

Hi Carlos,

I wrote up a script that automates some bits of the process of importing tarballs from upstream master into rawhide. I've attached the script for your review. Do you think this would be useful to have in the fedora rawhide branch? With its presence, the workflow for resyncing rawhide would simply be:

Great job!

We should absolutely put this into the Fedora rawhide branch.

We want all of our tools easily accessible on checkout, either from that branch or some Fedora glibc-releng repository.

Honestly I think you should check this into master as an example distribution script for RPM systems. That way others can look at what you've done without needing to go into a distribution specific branch.

It's glibc's responsibility to help downstream distributions use the project effectively.

0. Check out the master branch on the local glibc repo

1. Check out the master branch on the fedora glibc repo

2. In fedora glibc, run:

   ./sync-upstream.sh <path-to-local-glibc-repo>

3.1. If the script succeeds:

fedpkg push
fedpkg build

3.2. If the script fails, analyze the reason for failure and fall back to manual process. The most likely failure is that a patch refuses to apply and one may have to adjust the patch before committing the changes.

Let me know what you think about this.

Sounds good to me.

It's a step in the right direction.

Siddhesh

#!/bin/sh # Given a git source repo, generate a tarball from master, modify the spec file # and upload it to lookaside cache if the tarball unpacks and gets patched # cleanly. This currently needs the source repo to be on the master branch. # XXX In future, we could just do the git-archive ourselves instead of relying

s/XXX/TODO:/g

Prefer TODO over XXX only because it's clear this is an RFE.

I generally reserve XXX and FIXME for something that are wrong.

# on the makefile bits. That way we don't have to waste time waiting for the # bzip and xz archives that we don't use.

if [ $# -ne 1 ]; then echo "Usage: $0 <path-to-upstream-repo>" 1>&2 exit 1 fi

upstream=$1

tmpdir=/tmp/tmp-rawhide-gzbuild

Can't run two on the same machine.

Use mktemp please.

srcdir=$(git --git-dir=$upstream/.git describe) cursrc=$(sed -ne 's/^%define glibcsrcdir (.*)/\1/p' glibc.spec)

These are automatically escaped by the sub-shell invocation.

Assignment also preserves escaping e.g. cursrc2=$cursrc is OK.

if [ "$cursrc" = "$srcdir" ]; then echo Already in sync with upstream exit 0 fi

echo "+ Generating tarballs..." mkdir $tmpdir pushd $tmpdir

These are not and and need escaping now.

# We don't need any options here since we only want the makefile to be # generated so that we can call make dist $upstream/configure --prefix=/usr

Needs escaping e.g. "$upstream/configure"

# HACK! We don't want to regenerate the INSTALL file, so regenerate the file # explicitly, undo its changes and then make dist. That way make thinks that # INSTALL is up-to-date. make INSTALL pushd $upstream git diff INSTALL | patch -p1 -R popd make dist popd rm -rf $tmpdir

Needs escaping e.g. "$tmpdir".

if ! [ -f $upstream/$srcdir.tar.gz ]; then

Needs escaping.

echo "Failed to create the tarball" exit 2 fi

# Copy over the tarball. echo "+ Copying over generated tarball..." cp $upstream/$srcdir.tar.gz ./

Needs escaping.

# Our changelog header cldate=$(date +'%a %b %d %Y') clname=$(git config --get user.name) clemail=$(git config --get user.email | sed 's/@/\@/')

OK.

# Getting our version and release number from the spec file. nv=$(perl -ne 's/^%define glibcversion (.+)/printf("%s-", $1)/e;' \ -e 's/^%define glibcrelease ([0-9]+).*/printf("%d\n", $1 + 1)/e;' \ glibc.spec)

# Our changelog entry. changelog="* $cldate $clname <$clemail> - $nv\n- Sync with upstream master.\n"

# Change the glibcsrcdir variable, bump up the release number and add an extra # entry to the changelog. echo "+ Updating spec file..." perl -pi \ -e "s/^(%define glibcsrcdir ).+/$1$srcdir/; s/^(%define glibcrelease )(\d+)/print($1); print($2 + 1);'';/e; s/^(%changelog)$/$1\n$changelog/" \ glibc.spec

echo "Testing if fedpkg prep works..." if fedpkg prep; then

Run `fedpkg prep' as a distinct step, save the output, and then check the return code.

echo "Source prep is clean, so we're good to go." fedpkg new-sources $srcdir.tar.gz

Needs escaping.

git commit -a -m "Sync with upstream master" # XXX I don't have the courage yet to do a fedpkg push && fedpkg build # from the script.

Do it.

But do three things.

(1) Use `set -e' immediately after #!/bin/sh to ensure that any non-zero exit from any command, builtin, or sub-shell exits the shell.

(2) Make sure you have no compound steps that could hide an exit code. e.g.

[carlos@koi ~]$ cat test.sh #!/bin/sh # Oh, oh, typo! It should be carlos not carlso dir=$(ls /home/carlso | sed -e "s,a,b,g") echo $? # We didn't notice though... [carlos@koi ~]$ ./test.sh ls: cannot access /home/carlso: No such file or directory 0 [carlos@koi ~]$

(3) Parse the fedpkg prep log output and verify that the expected tarball name was used.

else echo "Source prep failed." echo "Check the output above and fix things before committing." fi

echo "+ Done!"

Tada!

Cheers, Carlos.