Hi

I found a bug with flexmmap on x86-64 (kernel-2.6.8-1.521).

A 32bit process only get the new vm layout when it's started from a 32bit process. When it's started from a 64bit process it's get the legacy layout:

A 32bit cat started from a 32bit shell:

cat32 /proc/self/maps

00111000-00126000 r-xp 00000000 03:01 342737 /lib/ld-2.3.3.so 00126000-00127000 r-xp 00014000 03:01 342737 /lib/ld-2.3.3.so 00127000-00128000 rwxp 00015000 03:01 342737 /lib/ld-2.3.3.so 00128000-0012e000 r-xp 00da2000 03:01 1066271 /usr/lib/locale/locale-archive 0012e000-0012f000 r-xp 02153000 03:01 1066271 /usr/lib/locale/locale-archive 00136000-00137000 rwxp 00136000 00:00 0 00137000-0024c000 r-xp 00000000 03:01 342750 /lib/tls/libc-2.3.3.so 0024c000-0024e000 r-xp 00115000 03:01 342750 /lib/tls/libc-2.3.3.so 0024e000-00250000 rwxp 00117000 03:01 342750 /lib/tls/libc-2.3.3.so 00250000-00252000 rwxp 00250000 00:00 0 00252000-00452000 r-xp 00000000 03:01 1066271 /usr/lib/locale/locale-archive 00452000-00486000 r-xp 00da9000 03:01 1066271 /usr/lib/locale/locale-archive 08048000-0804c000 r-xp 00000000 03:01 146973 /bin/cat32 0804c000-0804d000 rwxp 00003000 03:01 146973 /bin/cat32 0804d000-0806e000 rwxp 0804d000 00:00 0 ffffc000-ffffe000 rwxp ffffc000 00:00 0 ffffe000-fffff000 ---p 00000000 00:00 0

The same 32bit cat started from a 64bit shell:

cat32 /proc/self/maps

08048000-0804c000 r-xp 00000000 03:01 146973 /bin/cat32 0804c000-0804d000 rwxp 00003000 03:01 146973 /bin/cat32 0804d000-0806e000 rwxp 0804d000 00:00 0 55555000-5556a000 r-xp 00000000 03:01 342737 /lib/ld-2.3.3.so 5556a000-5556b000 r-xp 00014000 03:01 342737 /lib/ld-2.3.3.so 5556b000-5556c000 rwxp 00015000 03:01 342737 /lib/ld-2.3.3.so 5557a000-5557b000 rwxp 5557a000 00:00 0 5557b000-55690000 r-xp 00000000 03:01 342750 /lib/tls/libc-2.3.3.so 55690000-55692000 r-xp 00115000 03:01 342750 /lib/tls/libc-2.3.3.so 55692000-55694000 rwxp 00117000 03:01 342750 /lib/tls/libc-2.3.3.so 55694000-55696000 rwxp 55694000 00:00 0 55696000-55896000 r-xp 00000000 03:01 1066271 /usr/lib/locale/locale-archive 55896000-5589c000 r-xp 00da2000 03:01 1066271 /usr/lib/locale/locale-archive 5589c000-558d0000 r-xp 00da9000 03:01 1066271 /usr/lib/locale/locale-archive 558d0000-558d1000 r-xp 02153000 03:01 1066271 /usr/lib/locale/locale-archive ffffc000-ffffe000 rwxp ffffc000 00:00 0 ffffe000-fffff000 ---p 00000000 00:00 0

I think when arch_pick_mmap_layout() is called in fs/exec.c::exec_mmap() the TIF_IA32 flag is not setuped yet for the new process. So it's really the flag from the parent.

Adding a additional arch_pick_mmap_layout() in fs/binfmt_elf.c works for me (only tested on x86-64):

diff -Nrup linux-2.6.8-1.521/fs/binfmt_elf.c linux-2.6.8-1.521-fix-flexmm1/fs/binfmt_elf.c --- linux-2.6.8-1.521/fs/binfmt_elf.c 2004-08-16 14:58:43.000000000 +0200 +++ linux-2.6.8-1.521-fix-flexmm1/fs/binfmt_elf.c 2004-08-18 16:28:27.000000000 +0200 @@ -769,6 +769,8 @@ static int load_elf_binary(struct linux_ if (elf_read_implies_exec(elf_ex, have_pt_gnu_stack)) current->personality |= READ_IMPLIES_EXEC;

+ arch_pick_mmap_layout(current->mm); + /* Do this so that we can load the interpreter, if need be. We will change some of these later */ current->mm->rss = 0;

utz