This is pretty gross. [11/6730] python ../../tools/protoc_wrapper/protoc_wrapper.py test_sample.proto --protoc ./x64/protoc --proto-in-dir ../../third_party/dom_distiller_js --cc-out-dir gen/third_party/dom_distiller_js --py-out-dir pyproto/third_party/dom_distiller_js --plugin ../../third_party/dom_distiller_js/protoc_plugins/json_values_converter.py --plugin-out-dir gen/third_party/dom_distiller_js --plugin-options output_dir=: FAILED: gen/third_party/dom_distiller_js/test_sample.pb.h gen/third_party/dom_distiller_js/test_sample.pb.cc pyproto/third_party/dom_distiller_js/test_sample_pb2.py gen/third_party/dom_distiller_js/test_sample_json_converter.h python ../../tools/protoc_wrapper/protoc_wrapper.py test_sample.proto --protoc ./x64/protoc --proto-in-dir ../../third_party/dom_distiller_js --cc-out-dir gen/third_party/dom_distiller_js --py-out-dir pyproto/third_party/dom_distiller_js --plugin ../../third_party/dom_distiller_js/protoc_plugins/json_values_converter.py --plugin-out-dir gen/third_party/dom_distiller_js --plugin-options output_dir=: Traceback (most recent call last): File "../../third_party/dom_distiller_js/protoc_plugins/json_values_converter.py", line 14, in <module> from util import plugin_protos, types, writer File "/var/tmp/portage/www-client/chromium-58.0.3029.19/work/chromium-58.0.3029.19/third_party/dom_distiller_js/protoc_plugins/util/plugin_protos.py", line 22, in <module> from google.protobuf.descriptor_pb2 import FieldDescriptorProto ImportError: No module named protobuf.descriptor_pb2 --plugin_out: protoc-gen-plugin: Plugin failed with status code 1. Protoc has returned non-zero status: 1 . [12/6730] python ../../tools/protoc_wrapper/protoc_wrapper.py dom_distiller.proto --protoc ./x64/protoc --proto-in-dir ../../third_party/dom_distiller_js/dist/proto --cc-out-dir gen/third_party/dom_distiller_js --py-out-dir pyproto/third_party/dom_distiller_js --plugin ../../third_party/dom_distiller_js/protoc_plugins/json_values_converter.py --plugin-out-dir gen/third_party/dom_distiller_js --plugin-options output_dir=: FAILED: gen/third_party/dom_distiller_js/dom_distiller.pb.h gen/third_party/dom_distiller_js/dom_distiller.pb.cc pyproto/third_party/dom_distiller_js/dom_distiller_pb2.py gen/third_party/dom_distiller_js/dom_distiller_json_converter.h python ../../tools/protoc_wrapper/protoc_wrapper.py dom_distiller.proto --protoc ./x64/protoc --proto-in-dir ../../third_party/dom_distiller_js/dist/proto --cc-out-dir gen/third_party/dom_distiller_js --py-out-dir pyproto/third_party/dom_distiller_js --plugin ../../third_party/dom_distiller_js/protoc_plugins/json_values_converter.py --plugin-out-dir gen/third_party/dom_distiller_js --plugin-options output_dir=: Traceback (most recent call last): File "../../third_party/dom_distiller_js/protoc_plugins/json_values_converter.py", line 14, in <module> from util import plugin_protos, types, writer File "/var/tmp/portage/www-client/chromium-58.0.3029.19/work/chromium-58.0.3029.19/third_party/dom_distiller_js/protoc_plugins/util/plugin_protos.py", line 22, in <module> from google.protobuf.descriptor_pb2 import FieldDescriptorProto ImportError: No module named protobuf.descriptor_pb2 --plugin_out: protoc-gen-plugin: Plugin failed with status code 1. Protoc has returned non-zero status: 1 . This brought me to stumble onto this delightful bug: https://bugs.chromium.org/p/chromium/issues/detail?id=638488 Which, in a nutshell, says that although protobuf-python is "vendored" in chromium sources, the a non-vendored protobuf-python must be installed on the build machine, or, put another way, out-of-tree protobuf python is required to successfully build anything depending on the in-tree protobuf-python in order, presumably, to make up for the fact that protoc is not smart enough to handle the just-in-time PYTHONPATH hacking that occurs in the dependent units such as json_values_converter.py above. So, my chromium-maintaining friends, your options are apparently: (1, the ugly) fix and/or augment the protoc-wrapper to ensure the vendored protobuf-python dependencies are respected by the toolchain (2, the bad) make dev-python/protobuf-python a DEPEND of affected chromium drops Sounds like good times either way! Thanks for your tireless efforts!
Actually it seems Google solved this the "ugly" (but way better) way: https://codereview.chromium.org/2250333002 -> https://codereview.chromium.org/download/issue2250333002_20001_30001.diff https://codereview.chromium.org/download/issue2250333002_20001_30002.diff Which helps restore my faith in humanity a little bit, is up to Gentoo dependency management standards, and also presents an alternative approach to fixing this bug by ignoring it until it goes away.
I was unable to reproduce this with chromium-58.0.3029.110 (made sure protobuf-python is not installed).
I think upstream's fix has trickled down to us -- guess I should have closed this at some point, thanks for noticing.
Something changed recently so this is happening again. I had to rebuild www-client/chromium-58.0.3029.110 because of some subslot upgrade and now its failing to build. The exact same version built before tho so not sure why its failing. I can reliably make it fail (although takes many hours :() and cding into $S and running the ninja command manually also fails. And qlop -l says ive never ever had dev-python/protobuf-python installed on this machine so not sure why im only now hitting this.
This is still happening on 59.0.3071.86 :( I dont get why this machine is suddenly not building it when ive been building chromium for years before without protobuf installed. i guess i'll install it and rebuild chromium and see if it works then. [18503/26379] python ../../tools/protoc_wrapper/protoc_wrapper.py dom_distiller.proto --protoc ./x64/protoc --proto-in-dir ../../third_party/dom_distiller_js/dist/proto --cc-out-dir gen/third_party/dom_distiller_js --py-out-dir pyproto/third_party/dom_distiller_js --plugin ../../third_party/dom_distiller_js/protoc_plugins/json_values_converter.py --plugin-out-dir gen/third_party/dom_distiller_js --plugin-options output_dir=: FAILED: gen/third_party/dom_distiller_js/dom_distiller.pb.h gen/third_party/dom_distiller_js/dom_distiller.pb.cc pyproto/third_party/dom_distiller_js/dom_distiller_pb2.py gen/third_party/dom_distiller_js/dom_distiller_json_converter.h python ../../tools/protoc_wrapper/protoc_wrapper.py dom_distiller.proto --protoc ./x64/protoc --proto-in-dir ../../third_party/dom_distiller_js/dist/proto --cc-out-dir gen/third_party/dom_distiller_js --py-out-dir pyproto/third_party/dom_distiller_js --plugin ../../third_party/dom_distiller_js/protoc_plugins/json_values_converter.py --plugin-out-dir gen/third_party/dom_distiller_js --plugin-options output_dir=: Traceback (most recent call last): File "../../third_party/dom_distiller_js/protoc_plugins/json_values_converter.py", line 14, in <module> from util import plugin_protos, types, writer File "/var/tmp/portage/www-client/chromium-59.0.3071.86/work/chromium-59.0.3071.86/third_party/dom_distiller_js/protoc_plugins/util/plugin_protos.py", line 22, in <module> from google.protobuf.descriptor_pb2 import FieldDescriptorProto ImportError: No module named protobuf.descriptor_pb2 --plugin_out: protoc-gen-plugin: Plugin failed with status code 1. Protoc has returned non-zero status: 1 .
(In reply to Jason Zaman from comment #5) Before you go and install protobuf-python, please check a couple of things: Does /usr/lib64/python2.7/site-packages/google exist on your system? If so, what package owns it? Also, grep for the string "google" in /usr/lib64/python2.7/site-packages/*.pth. I'm thinking this may be an issue caused by an improperly installed namespace package. https://wiki.gentoo.org/wiki/Project:Python/Namespace_packages
I am able to reproduce this build failure by installing dev-python/google-apputils-0.4.2-r1. This package has a broken python namespace. Upgrading to dev-python/google-apputils-0.4.2-r2 fixes the namespace support and resolves the chromium build issue.
(In reply to Mike Gilbert from comment #7) > I am able to reproduce this build failure by installing > dev-python/google-apputils-0.4.2-r1. This package has a broken python > namespace. > > Upgrading to dev-python/google-apputils-0.4.2-r2 fixes the namespace support > and resolves the chromium build issue. Looks like I have dev-python/google-apputils-0.4.2-r1 installed. I recently installed net-misc/gsutil which pulled in apputils so that would explain why it suddenly broke. I'll upgrade to -r2 and try it again. how do you know that the namespace is broken? # qlist -Iv google dev-python/google-apitools-0.5.4 dev-python/google-apputils-0.4.0-r1 dev-util/google-perftools-2.0-r2 www-client/google-chrome-59.0.3071.86 # ls -al /usr/lib64/python*/site-packages/google* -rw-r--r--. 1 root root 555 May 29 22:55 /usr/lib64/python2.7/site-packages/google_apputils-0.4.0-py2.7-nspkg.pth /usr/lib64/python2.7/site-packages/google: total 90 drwxr-xr-x. 3 root root 3 May 29 22:55 ./ drwxr-xr-x. 268 root root 537 Jun 13 00:44 ../ drwxr-xr-x. 2 root root 41 May 29 22:55 apputils/ /usr/lib64/python2.7/site-packages/google_apitools-0.5.4-py2.7.egg-info: total 104 drwxr-xr-x. 2 root root 8 May 29 22:55 ./ drwxr-xr-x. 268 root root 537 Jun 13 00:44 ../ -rw-r--r--. 1 root root 2142 May 29 22:55 PKG-INFO -rw-r--r--. 1 root root 2746 May 29 22:55 SOURCES.txt -rw-r--r--. 1 root root 1 May 29 22:55 dependency_links.txt -rw-r--r--. 1 root root 61 May 29 22:55 entry_points.txt -rw-r--r--. 1 root root 174 May 29 22:55 requires.txt -rw-r--r--. 1 root root 17 May 29 22:55 top_level.txt /usr/lib64/python2.7/site-packages/google_apputils-0.4.0-py2.7.egg-info: total 105 drwxr-xr-x. 2 root root 9 May 29 22:55 ./ drwxr-xr-x. 268 root root 537 Jun 13 00:44 ../ -rw-r--r--. 1 root root 247 May 29 22:55 PKG-INFO -rw-r--r--. 1 root root 1111 May 29 22:55 SOURCES.txt -rw-r--r--. 1 root root 1 May 29 22:55 dependency_links.txt -rw-r--r--. 1 root root 175 May 29 22:55 entry_points.txt -rw-r--r--. 1 root root 7 May 29 22:55 namespace_packages.txt -rw-r--r--. 1 root root 51 May 29 22:55 requires.txt -rw-r--r--. 1 root root 7 May 29 22:55 top_level.txt # grep google /usr/lib64/python2.7/site-packages/*.pth /usr/lib64/python2.7/site-packages/google_apputils-0.4.0-py2.7-nspkg.pth:import sys, types, os;has_mfs = sys.version_info > (3, 5);p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('google',));importlib = has_mfs and __import__('importlib.util');has_mfs and __import__('importlib.machinery');m = has_mfs and sys.modules.setdefault('google', importlib.util.module_from_spec(importlib.machinery.PathFinder.find_spec('google', [os.path.dirname(p)])));m = m or not has_mfs and sys.modules.setdefault('google', types.ModuleType('google'));mp = (m or []) and m.__dict__.setdefault('__path__',[]);(p not in mp) and mp.append(p)
(In reply to Jason Zaman from comment #8) > how do you know that the namespace is broken? The commit log for dev-python/google-apputils says so. ;) Basically, there are 2 problems: 1. site-packages/google/__init__.py is missing. 2. site-packages/google_apputils-0.4.0-py2.7-nspkg.pth exists. Read the wiki page I linked in comment 6 for additional details.
(In reply to Mike Gilbert from comment #7) > Upgrading to dev-python/google-apputils-0.4.2-r2 fixes the namespace support > and resolves the chromium build issue. Confirming this fixes it for me too. thanks!
