Bug 462542 – dev-python/matplotlib-1.2.0-r2 - ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.

• Status: RESOLVED INVALID
• Product: Gentoo Linux

[Description W. Trevor King 2013-03-21 00:47:50 UTC]

When I try to emerge the package, it fails with:

  …
  /build/lib/matplotlib/tests/baseline_images/test_delaunay
  Traceback (most recent call last):
    File "/usr/lib/python3.2/multiprocessing/synchronize.py", line 54, in <module>
      from _multiprocessing import SemLock
  ImportError: cannot import name SemLock
  …
  ImportError: This platform lacks a functioning sem_open implementation,
   therefore, the required synchronization primitives needed will not function,
   see issue 3770.
  …


Reproducible: Always

Steps to Reproduce:
  # emerge matplotlib




I'm building a live cd in a ca talyst chroot on an Ubuntu system, so this may be due to something wonky in my setup.  I can't *think* of why that would matter, but maybe it does.

[Comment 1 W. Trevor King 2013-03-21 00:48:16 UTC]

Created attachment 342796 [details]
emerge --info

[Comment 2 W. Trevor King 2013-03-21 00:48:59 UTC]

Created attachment 342798 [details]
Log from python-2.7.3-r3

[Comment 3 W. Trevor King 2013-03-21 00:50:41 UTC]

Created attachment 342800 [details]
Second log from Python 2.7.3-r3

I don't know why there are two separate logs, but I set PORT_LOGDIR and got two files for each Python version, so here they are ;).

[Comment 4 W. Trevor King 2013-03-21 00:52:09 UTC]

Created attachment 342802 [details]
First log from Python 3.2.3-r2 (gzipped for size)

[Comment 5 W. Trevor King 2013-03-21 00:53:17 UTC]

Created attachment 342804 [details]
Second log for Python 3.2.3-r2

[Comment 6 W. Trevor King 2013-03-21 00:53:51 UTC]

Post about this on the catalyst@ list:

http://article.gmane.org/gmane.linux.gentoo.catalyst/2207

[Comment 7 Jeroen Roovers (RETIRED) 2013-03-21 15:41:36 UTC]

Comment on attachment 342804 [details]
Second log for Python 3.2.3-r2

That's an "unmerge log".

[Comment 8 Jeroen Roovers (RETIRED) 2013-03-21 15:42:21 UTC]

Comment on attachment 342800 [details]
Second log from Python 2.7.3-r3

Ditto.

[Comment 9 Jeroen Roovers (RETIRED) 2013-03-21 15:43:24 UTC]

Neither build log shows the error you put in the Summary and in comment #0.

[Comment 10 W. Trevor King 2013-03-21 15:45:10 UTC]

(In reply to comment #9)
> Neither build log shows the error you put in the Summary and in comment #0.

Ah, sorry.  I thought I'd attached the matplotlib log too…

Thanks for fixing my MIME error and explaining the other two Python logs.

[Comment 11 W. Trevor King 2013-03-21 15:46:46 UTC]

Created attachment 342846 [details]
Matplotlib log with the build failure

[Comment 12 W. Trevor King 2013-03-23 19:42:44 UTC]

It looks like the difference may be in the Python 3.2 log:

  checking whether POSIX semaphores are enabled... no

the example C program in Python's `configure` responsible for the test fails on the system in question:

  $ cat conftest.c
  #include <unistd.h>
  #include <fcntl.h>
  #include <stdio.h>
  #include <semaphore.h>
  #include <sys/stat.h>

  int main(void) {
    sem_t *a = sem_open("/autoconf", O_CREAT, S_IRUSR|S_IWUSR, 0);
    if (a == SEM_FAILED) {
      perror("sem_open");
      return 1;
    }
    sem_close(a);
    return 0;
  }
  $ cc -o conftest conftest.c -lpthread
  $ ./conftest
  sem_open: Function not implemented
  $ echo $?
  1

But passes on my local machine:

  $ ./conftest
  $ echo $?
  0

The patch that added this check [1] is about FreeBSD (the failing system is running Linux 3.2), but it mentions kernel support.  The BSD issues are also mentioned in the issue referenced in the traceback [2].

[1]: http://hg.python.org/cpython/rev/3d77d7c16c90
[2]: http://bugs.python.org/issue3770

[Comment 13 W. Trevor King 2013-03-23 20:13:36 UTC]

Possibly related, sem_overview(7) says:

  On  Linux,  named semaphores are created in a virtual file system, normally mounted under /dev/shm, …

on the failing system, it looks like /dev/shm is a symlink /run/shm.  I'm building in a chroot (since the failing system is running Ubuntu, and setting up a catalyst build environment without mucking around in the main system was easier in a chroot.  However, when I mount /dev in the chroot, I didn't realize I'd also need to mount /run, so I have a broken symlink in my chroot:

  $ ls -l /dev/shm
  lrwxrwxrwx 1 root root 8 Mar 20 17:45 /dev/shm -> /run/shm

If I mount a new tmpfs at the target location:

  # mkdir /tmp/shm
  # mount -t tmpfs none /tmp/shm

the test program succeeds:

  $ ./conftest
  $ echo $?
  0

Rebuilding Python picks up the enabled semaphores, which were not picked up in the older builds:

  # grep 'POSIX semaphores' /tmp/dev-lang\:python-*
  /tmp/dev-lang:python-2.7.3-r3:20130321-003541.log:checking whether POSIX semaphores are enabled... no
  /tmp/dev-lang:python-2.7.3-r3:20130323-195748.log:checking whether POSIX semaphores are enabled... yes
  /tmp/dev-lang:python-3.2.3-r2:20130321-003804.log:checking whether POSIX semaphores are enabled... no
  /tmp/dev-lang:python-3.2.3-r2:20130323-200011.log:checking whether POSIX semaphores are enabled... yes

With the new semaphore-enabled Pythons, the Matplotlib emerge succeeds, so the problem was just the missing /dev/shm mount.  This is either a problem with catalyst, or a problem with my catalyst configuration ;).

[Comment 14 W. Trevor King 2013-03-23 20:15:44 UTC]

(In reply to comment #13)
>   # mkdir /tmp/shm
>   # mount -t tmpfs none /tmp/shm

Oops, should be /run/shm.

[Comment 15 W. Trevor King 2013-03-23 20:31:21 UTC]

> With the new semaphore-enabled Pythons, the Matplotlib emerge succeeds, so
> the problem was just the missing /dev/shm mount.  This is either a problem
> with catalyst, or a problem with my catalyst configuration ;).

It looks like mount points are not currently configurable in catalyst, so this error is due to an odd /dev/shm mounting that catalyst doesn't (yet) support.  I'm closing this as resolved/invalid.  I may open a new bug about catalyst mount configuration, or just try and work a fix in through the catalyst mailing list and releng IRC channel.

[Comment 16 W. Trevor King 2013-03-28 11:56:08 UTC]

Created attachment 343522 [details, diff]
Patch against catalyst 2.0.12.2 with a hard-coded fix

If anyone else bumps into this problem, this patch has a hard-coded fix that works for me.  I'll post again once catalyst supports configurable mount points.