Michael E Brown wrote:

On Fri, Nov 30, 2007 at 04:17:52PM +0000, Paul Howarth wrote:

...

Michael E Brown wrote:

...

I need a little bit of help understanding what the 'libselinux-mock.so' LD_PRELOAD was supposed to be doing. I released and pushed out mock-0.8 without this. I have rebuilt most of rawhide with this new mock version and have not seen anything that I directly can say was a failure due to this being missing, so I am sort of not seeing the point.

I have searched around as much as I can to try to understand why this was put into place. From what I can understand, it was only put in in the FC2 timeframe to fix some problems with the selinux policy on the host machine. These *appear* to have been fixed in the host selinux policy, so again, i dont see a reason to keep this around.

Jesse mentioned on IRC, though, that this might be needed, so I pose this question. I've a local branch set up with the 0.8.x code and the LD_PRELOAD put back in. So, I can quickly spin a new release with this back in if it is actually needed. So far, I havent convinced myself it is needed, though...

Could somebody please enlighten me?

I suspect it was there to convince rpm that selinux was disabled and hence prevent it from trying to set the contexts of installed files. If rpm doesn't set the context, the files are installed as mock_var_lib_t (assuming my policy module from the wiki is loaded), and this is a "neutral" type that doesn't cause any problems. However, if rpm sets the contexts of files it installs, it'll set things like /sbin/ldconfig to ldconfig_exec_t, and there is now a transition defined in policy from unconfined_t to ldconfig_exec_t. The result of this is that some processes such as ldconfig get run "confined" in the chroot and fail as a result because not *all* file contexts are set up as they would be on the host system, which is what the policy is set up for.

I've not quite figured out all te ins and outs of it but a tell-tale sign of there being a problem is to look (ls -lZ) at the root directories of your chroots and see if they look like this:

/var/lib/mock/fedora-2-i386-core/root: drwxr-xr-x root root system_u:object_r:bin_t:s0 bin drwxr-xr-x root root system_u:object_r:boot_t:s0 boot drwx------ root mock system_u:object_r:mock_var_lib_t:s0 builddir drwxrwsr-x root mock system_u:object_r:mock_var_lib_t:s0 dev drwxr-xr-x root root system_u:object_r:etc_t:s0 etc drwxr-xr-x root root system_u:object_r:home_root_t:s0 home drwxr-xr-x root root system_u:object_r:root_t:s0 initrd drwxr-xr-x root root system_u:object_r:lib_t:s0 lib drwxr-xr-x root root system_u:object_r:mnt_t:s0 mnt drwxr-xr-x root root system_u:object_r:usr_t:s0 opt drwxrwsr-x root mock system_u:object_r:mock_var_lib_t:s0 proc drwxr-x--- root root system_u:object_r:user_home_dir_t:s0 root drwxr-xr-x root root system_u:object_r:bin_t:s0 sbin drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 selinux drwxrwsr-x root mock system_u:object_r:mock_var_lib_t:s0 sys drwxrwxrwt root root system_u:object_r:tmp_t:s0 tmp drwxr-xr-x root root system_u:object_r:usr_t:s0 usr drwxr-xr-x root root system_u:object_r:var_t:s0 var

rather than this: /var/lib/mock/fedora-3-i386-core/root: drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 bin drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 boot drwx------ paul mock system_u:object_r:mock_var_lib_t:s0 builddir drwxrwsr-x root mock system_u:object_r:mock_var_lib_t:s0 dev drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 etc drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 home drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 initrd drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 lib drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 media drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 mnt drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 opt drwxrwsr-x root mock system_u:object_r:mock_var_lib_t:s0 proc drwxr-x--- root root system_u:object_r:mock_var_lib_t:s0 root drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 sbin drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 selinux drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 srv drwxrwsr-x root mock system_u:object_r:mock_var_lib_t:s0 sys drwxrwxrwt root root system_u:object_r:mock_var_lib_t:s0 tmp drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 usr drwxr-xr-x root root system_u:object_r:mock_var_lib_t:s0 var

(it's best to remove any cache or existing root and generate a fresh one)

I'd like to try a version with the LD_PRELOAD back in if possible.

Well you can look at current fedora mock and run mock -r fedora-8-x86_64 init

On my local machine (with selinux enabled), I see that everything is labelled with "var_lib_t", like this:

$ ls -lZ drwxr-xr-x root root user_u:object_r:var_lib_t bin drwxr-xr-x root root user_u:object_r:var_lib_t boot drwx------ michael_e_brown mock user_u:object_r:var_lib_t builddir drwxrwxr-x root root user_u:object_r:var_lib_t dev ... cut ...

If you're not using the policy module, I'd expect you to have problems building packages that run mono and/or java code at build time as described at http://fedoraproject.org/wiki/PackageMaintainers/MockTricks

The package I came across that exhibited this problem and led me to write the policy module was "lat", a mono-based package.

There is a current git branch with the selinux changes ported at:

git clone http://linux.dell.com/git/mock.git
git checkout -b selinux origin/selinux

I built an RPM from this, and my first try at running it resulted in:

Traceback (most recent call last): File "/usr/libexec/mock.py", line 430, in <module> os.environ["LD_PRELOAD"] = LIBDIR+"/libselinux-mock.so" NameError: name 'LIBDIR' is not defined

I added a manual definition of LIBDIR into /usr/libexec/mock.py and tt ran a bit further. I now get the sane file contexts that I was expecting but none of the scriptlets run in the target chroot because the object pointed to by the LD_PRELOAD isn't available in the chroot:

/bin/sh: error while loading shared libraries: /usr/lib64/libselinux-mock.so: cannot open shared object file: No such file or directory error: %post(bash-2.05b-38.i386) scriptlet failed, exit status 127 /bin/sh: error while loading shared libraries: /usr/lib64/libselinux-mock.so: cannot open shared object file: No such file or directory error: %post(libselinux-1.11.4-1.mockfix.fc2.i386) scriptlet failed, exit status 127 /bin/sh: error while loading shared libraries: /usr/lib64/libselinux-mock.so: cannot open shared object file: No such file or directory error: %post(info-4.7-4.i386) scriptlet failed, exit status 127 /bin/sh: error while loading shared libraries: /usr/lib64/libselinux-mock.so: cannot open shared object file: No such file or directory error: %post(sed-4.0.8-4.i386) scriptlet failed, exit status 127 /bin/sh: error while loading shared libraries: /usr/lib64/libselinux-mock.so: cannot open shared object file: No such file or directory

... and so on.

Don't know what to do about that.

Paul.

On Mon, Dec 03, 2007 at 04:39:26PM -0600, Michael E Brown wrote:

On Mon, Dec 03, 2007 at 04:49:41PM +0000, Paul Howarth wrote:

...

Michael E Brown wrote: If you're not using the policy module, I'd expect you to have problems building packages that run mono and/or java code at build time as described at http://fedoraproject.org/wiki/PackageMaintainers/MockTricks

Can you explain to me what you mean by "if you're not using the policy module"? I'm sorta-slow when it comes to selinux (as evidenced by this thread...)

...

The package I came across that exhibited this problem and led me to write the policy module was "lat", a mono-based package.

Using unmodified current mock (0.8.12) on Fedora 8 with selinux enforcing, I was able to compile current F8 lat:

$ mock -r fedora-8-x86_64 --rebuild --resultdir=./try/out ./try/lat-1.2.3-1.fc8.src.rpm INFO: mock.py version 0.8.12 starting... State Changed: init plugins State Changed: start State Changed: lock buildroot State Changed: clean INFO: Start(./try/lat-1.2.3-1.fc8.src.rpm) Config(fedora-8-x86_64) State Changed: init State Changed: lock buildroot INFO: enabled yum cache State Changed: cleaning yum metadata INFO: enabled root cache State Changed: unpacking cache State Changed: running yum State Changed: setup State Changed: build INFO: Done(./try/lat-1.2.3-1.fc8.src.rpm) Config(fedora-8-x86_64) 9 minutes 42 seconds INFO: Results and/or logs in: ./try/out INFO: Cleaning up build root ('clean_on_success=True') State Changed: lock buildroot State Changed: clean

-- Michael

Michael E Brown wrote:

On Mon, Dec 03, 2007 at 04:49:41PM +0000, Paul Howarth wrote:

...

Michael E Brown wrote: If you're not using the policy module, I'd expect you to have problems building packages that run mono and/or java code at build time as described at http://fedoraproject.org/wiki/PackageMaintainers/MockTricks

The package I came across that exhibited this problem and led me to write the policy module was "lat", a mono-based package.

...

There is a current git branch with the selinux changes ported at:

git clone http://linux.dell.com/git/mock.git git checkout -b selinux origin/selinux

I built an RPM from this, and my first try at running it resulted in:

Traceback (most recent call last): File "/usr/libexec/mock.py", line 430, in <module> os.environ["LD_PRELOAD"] = LIBDIR+"/libselinux-mock.so" NameError: name 'LIBDIR' is not defined

I added a manual definition of LIBDIR into /usr/libexec/mock.py and tt ran a bit further. I now get the sane file contexts that I was expecting but none of the scriptlets run in the target chroot because the object pointed to by the LD_PRELOAD isn't available in the chroot:

/bin/sh: error while loading shared libraries: /usr/lib64/libselinux-mock.so: cannot open shared object file: No such file or directory

... and so on.

Don't know what to do about that.

The selinux stuff was only lightly tested. Looks like it still needs work. Looks like I missed something when I rebased to HEAD.

The way I *think* it used to work was that mock-helper would set the LD_PRELOAD and then exec() the required program (rpm, yum, whatever). When it came to running yum, it didn't exec() yum directly, it exec()-ed mock-yum instead, which was a simple wrapper that removed the LD_PRELOAD from the environment (the libselinux-mock already being in place from the exec() that called it). The result of this was that child processes of mock-yum (e.g. rpm, package scriptlets running in the chroot) got the fake libselinux without the LD_PRELOAD being visible.

The more integrated architecture of mock now may make this sort of hack quite difficult to implement.

Paul.

On Fri, Dec 07, 2007 at 02:40:44PM -0600, Michael E Brown wrote:

On Wed, Dec 05, 2007 at 12:35:46PM +0000, Paul Howarth wrote:

...

The way I *think* it used to work was that mock-helper would set the LD_PRELOAD and then exec() the required program (rpm, yum, whatever). When it came to running yum, it didn't exec() yum directly, it exec()-ed mock-yum instead, which was a simple wrapper that removed the LD_PRELOAD from the environment (the libselinux-mock already being in place from the exec() that called it). The result of this was that child processes of mock-yum (e.g. rpm, package scriptlets running in the chroot) got the fake libselinux without the LD_PRELOAD being visible.

The more integrated architecture of mock now may make this sort of hack quite difficult to implement.

s/difficult/easy/g;

It should be extremely easy to do this, *if* it is necessary. We just need to set/unset the variable as necessary around all calls to external programs. Like this: os.environ['LD_PRELOAD'] = "..."; or del(os.environ["LD_PRELOAD"]);

Luckily, we have *one* entry point to call all external programs, atm, which is mock.util.do(). We just need to decide before each external call if we need to set the variable or not.

We also have *one* wrapper for running yum, which then calls down to mock.util.do(). If necessary, we could easily set/unset this variable in that call and insulate all other callers from this knowledge.

All-in-all, if we can come up with a test case for why we would still need the preload, I could quite easily add this functionality back. So far, though, I'm not seeing a lot of evidence of what is broken, and I'm the sort that likes to see the broken pieces before I implement the fix.

Paul, I have recreated the git selinux branch at http://linux.dell.com/git/mock.git (if you have previously cloned, please re-clone.) It is based on the current 0.8 codebase.

This version passes all unit tests with no messages about missing libraries, and also unsets LD_PRELOAD prior to running mock.

Could you please give it a runthrough and see if you notice it doing anything? I get the same (nonfatal) AVC denials that I get without the patch when running in enforcing mode. -- Michael

Paul Howarth wrote:

Michael E Brown wrote:

...

On Fri, Dec 07, 2007 at 02:40:44PM -0600, Michael E Brown wrote:

...

On Wed, Dec 05, 2007 at 12:35:46PM +0000, Paul Howarth wrote:

...

The way I *think* it used to work was that mock-helper would set the LD_PRELOAD and then exec() the required program (rpm, yum, whatever). When it came to running yum, it didn't exec() yum directly, it exec()-ed mock-yum instead, which was a simple wrapper that removed the LD_PRELOAD from the environment (the libselinux-mock already being in place from the exec() that called it). The result of this was that child processes of mock-yum (e.g. rpm, package scriptlets running in the chroot) got the fake libselinux without the LD_PRELOAD being visible.

The more integrated architecture of mock now may make this sort of hack quite difficult to implement.

s/difficult/easy/g;

It should be extremely easy to do this, *if* it is necessary. We just need to set/unset the variable as necessary around all calls to external programs. Like this: os.environ['LD_PRELOAD'] = "..."; or del(os.environ["LD_PRELOAD"]);

Luckily, we have *one* entry point to call all external programs, atm, which is mock.util.do(). We just need to decide before each external call if we need to set the variable or not.

We also have *one* wrapper for running yum, which then calls down to mock.util.do(). If necessary, we could easily set/unset this variable in that call and insulate all other callers from this knowledge.

All-in-all, if we can come up with a test case for why we would still need the preload, I could quite easily add this functionality back. So far, though, I'm not seeing a lot of evidence of what is broken, and I'm the sort that likes to see the broken pieces before I implement the fix.

Paul, I have recreated the git selinux branch at http://linux.dell.com/git/mock.git (if you have previously cloned, please re-clone.) It is based on the current 0.8 codebase.

OK, I'll give that a go as soon as I can find a reasonable amount of time.

Just tried it, seems to have the same LIBDIR problem as last time:

$ mock -r fedora-8-x86_64 rebuild mock-0.8.17-0.se.fc8.src.rpm INFO: mock.py version 0.8.17 starting... State Changed: init plugins State Changed: start ERROR: global name 'LIBDIR' is not defined Traceback (most recent call last): File "/usr/libexec/mock.py", line 529, in <module> main(retParams) File "/usr/libexec/mock.py", line 512, in main do_rebuild(config_opts, chroot, args) File "<peak.util.decorators.rewrap wrapping __main__.do_rebuild at 0x008BA668>", line 3, in do_rebuild def do_rebuild(config_opts, chroot, srpms): return __decorated(config_opts, chroot, srpms) File "/usr/lib/python2.5/site-packages/mock/trace_decorator.py", line 70, in trace result = func(*args, **kw) File "/usr/libexec/mock.py", line 312, in do_rebuild os.environ["LD_PRELOAD"] = LIBDIR+"/libselinux-mock.so" NameError: global name 'LIBDIR' is not defined

Having fixed that, I found that some packages I tried built OK but there were still lots of messages about the LD_PRELOAD in the root and build logs. The visibility of LD_PRELOAD in the build phase caused builds of proftpd to fail on f8.x86_64 with a linker error (complaint about a relocation).

So I added:

del(os.environ["LD_PRELOAD"])

between the calls to chroot.init() and chroot.build() in do_rebuild, and that fixed that problem (there are still the messages in the root logs though.

The resulting target root directories now contain a mix of var_lib_t and rpm_var_lib_t files, and apparently nothing else. That should fix pretty well all of the selinux denials now.

I still need to contrive an example of a package that needs the mock selinux policy module from the wiki page; I'll keep trying.

Paul.

Michael E Brown wrote:

On Thu, Dec 13, 2007 at 12:01:47PM +0000, Paul Howarth wrote:

...

Paul Howarth wrote:

Just tried it, seems to have the same LIBDIR problem as last time:

$ mock -r fedora-8-x86_64 rebuild mock-0.8.17-0.se.fc8.src.rpm INFO: mock.py version 0.8.17 starting... State Changed: init plugins State Changed: start ERROR: global name 'LIBDIR' is not defined Traceback (most recent call last): File "/usr/libexec/mock.py", line 529, in <module> main(retParams) File "/usr/libexec/mock.py", line 512, in main do_rebuild(config_opts, chroot, args) File "<peak.util.decorators.rewrap wrapping __main__.do_rebuild at 0x008BA668>", line 3, in do_rebuild def do_rebuild(config_opts, chroot, srpms): return __decorated(config_opts, chroot, srpms) File "/usr/lib/python2.5/site-packages/mock/trace_decorator.py", line 70, in trace result = func(*args, **kw) File "/usr/libexec/mock.py", line 312, in do_rebuild os.environ["LD_PRELOAD"] = LIBDIR+"/libselinux-mock.so" NameError: global name 'LIBDIR' is not defined

This is odd. I ran a full unit test until I didnt see this message at all. Might be having git sync issues with our public mirror, I'll check.

I don't think this stuff is necessary any more. Since selinux-policy 3.0.8-67 in Fedora 8, /usr/bin/mock is labelled unconfined_notrans_exec_t. So mock doesn't transition into other domains and it doesn't matter that rpm labels files in the chroot with context types that would normally cause the problematic transitions (into useradd_t, ldconfig_t etc.). The result is nice, clean, denial-free builds with SELinux in enforcing mode.

This fix also renders the mock policy module as described on the wiki (the MockTricks page) largely redundant. The only exception case I can see is if some task needing to run as part of a build requires execheap permission, which might happen for some mono/java-based packages but I don't know of any problem packages right now. That bridge can no doubt be crossed when someone comes tp it.

Not sure if this fix has been applied in F-7 or if it will ever make it into RHEL/CentOS though.

Paul.