Summary: | sys-apps/portage traceback on read-only rootfs | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Duncan <1i5t5.duncan> |
Component: | Core | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | hlein |
Priority: | Normal | Keywords: | InVCS |
Version: | 2.2 | ||
Hardware: | All | ||
OS: | Linux | ||
See Also: | https://bugs.gentoo.org/show_bug.cgi?id=712222 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 484436 |
Here's a second instance of the bug, this time with an emerge --pretend. Why should an emerge --pretend need a read-write filesystem -- it's *PRETEND*!? In this case the OSError relates to the lockfile for (what would be) the news file for my local overlay, jed-local (jed being my initials). Except, being the local overlay, why would I ever HAVE any news to read at all? If I know it to put it in some news file, I know it and don't NEED to read about it in a news file. =:^) Between that and the fact that I'm getting this with a --pretend, which shouldn't need a writable filesystem in the first place as it IS pretend, it's a rather frustrating bug. Luckily portage spits out the information from the pretend that I need, before it does the traceback. Here's an example. It appears to happen with any emerge --pretend I try, however. $ emerge --pretend systemd These are the packages that would be merged, in order: Calculating dependencies ..... done! [ebuild R ] sys-apps/systemd-212-r2 Traceback (most recent call last): File "/usr/bin/emerge", line 50, in <module> retval = emerge_main() File "/lib64/portage/pym/_emerge/main.py", line 1037, in emerge_main return run_action(emerge_config) File "/lib64/portage/pym/_emerge/actions.py", line 4064, in run_action emerge_config.trees, emerge_config.target_config.mtimedb, retval) File "/lib64/portage/pym/portage/proxy/objectproxy.py", line 31, in __call__ return result(*args, **kwargs) File "/lib64/portage/pym/_emerge/post_emerge.py", line 110, in post_emerge display_news_notification(root_config, myopts) File "/lib64/portage/pym/_emerge/post_emerge.py", line 43, in display_news_notification news_counts = count_unread_news(portdb, vardb) File "/lib64/portage/pym/portage/news.py", line 388, in count_unread_news count = manager.getUnreadItems(repo, update=True) File "/lib64/portage/pym/portage/news.py", line 180, in getUnreadItems unread_lock = lockfile(unread_filename, wantnewlockfile=1) File "/lib64/portage/pym/portage/locks.py", line 106, in lockfile myfd = os.open(lockfilename, os.O_CREAT|os.O_RDWR, 0o660) File "/lib64/portage/pym/portage/__init__.py", line 259, in __call__ rval = self._func(*wrapped_args, **wrapped_kwargs) OSError: [Errno 30] Read-only file system: '/var/lib/gentoo/news/.news-jed-local.unread.portage_lockfile' FWIW I'm now on portage-2.2.10. (In reply to Duncan from comment #1) > OSError: [Errno 30] Read-only file system: > '/var/lib/gentoo/news/.news-jed-local.unread.portage_lockfile' I have a patch for this in the following branch: https://github.com/zmedico/portage/tree/bug_490732 I've posted the patch for review here: http://thread.gmane.org/gmane.linux.gentoo.portage.devel/4766 (In reply to Duncan from comment #0) > OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile' We could handle this similarly to the PermissionDenied exception, which can be raised from anywhere and results in a short error message like "Permission Denied: /var/db/.pkg.portage_lockfile". (In reply to Zac Medico from comment #2) > (In reply to Duncan from comment #0) > > OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile' > > We could handle this similarly to the PermissionDenied exception, which can > be raised from anywhere and results in a short error message like > "Permission Denied: /var/db/.pkg.portage_lockfile". Actually, I'm not so sure about hiding the traceback now, since it could make it difficult to diagnose bugs involving unhandled ReadOnlyFileSystem exceptions (like the one reported in comment #1). (In reply to Zac Medico from comment #2) > (In reply to Duncan from comment #1) > > OSError: [Errno 30] Read-only file system: > > '/var/lib/gentoo/news/.news-jed-local.unread.portage_lockfile' > > I have a patch for this in the following branch: > > https://github.com/zmedico/portage/tree/bug_490732 This one is in the master branch now: https://github.com/gentoo/portage/commit/e6121d92e586c0f9b75998eca2d1fbb46db7c821 (In reply to Duncan from comment #0) > OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile' For this one, I'm going to make it bail out (without traceback) after the dependency calculation if there are packages to be installed and /var/db is mounted readonly. If you re-mount it readonly after it has passed this check, then it will still show a traceback as before, since hiding the traceback at any point after that is likely to make bugs more difficult to diagnose. (In reply to Zac Medico from comment #4) > (In reply to Duncan from comment #0) > > OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile' > > For this one, I'm going to make it bail out (without traceback) after the > dependency calculation if there are packages to be installed and /var/db is > mounted readonly. There's a patch for this in my branch: https://github.com/zmedico/portage/tree/bug_490732 I've posted it for review here: http://thread.gmane.org/gmane.linux.gentoo.portage.devel/4816 (In reply to Zac Medico from comment #5) > (In reply to Zac Medico from comment #4) > > (In reply to Duncan from comment #0) > > > OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile' > > > > For this one, I'm going to make it bail out (without traceback) after the > > dependency calculation if there are packages to be installed and /var/db is > > mounted readonly. > > There's a patch for this in my branch: > > https://github.com/zmedico/portage/tree/bug_490732 It's in the master branch now: https://github.com/gentoo/portage/commit/103d27694197893badb3b5b5fb2e476e04f88033 We could do something similar to check for write access to $PKGDIR when appropriate. (In reply to Zac Medico from comment #6) > We could do something similar to check for write access to $PKGDIR when > appropriate. There's a patch for this in my branch. I've posted it here for review: http://gentoo.2317880.n4.nabble.com/PATCH-emerge-check-for-writable-PKGDIR-490732-td288311.html Just saying thanks. And thanks for the profiling related patches too. I'm sure I'm not the only gentooer and portage user glad you're back... with a better bus-factor now. =:^) (In reply to Duncan from comment #8) > Just saying thanks. And thanks for the profiling related patches too. I'm > sure I'm not the only gentooer and portage user glad you're back... with a > better bus-factor now. =:^) You're welcome. (In reply to Zac Medico from comment #7) > http://gentoo.2317880.n4.nabble.com/PATCH-emerge-check-for-writable-PKGDIR- > 490732-td288311.html This one's in the master branch now: https://github.com/gentoo/portage/commit/206efe5f6341bce99a5e9994a0458c304513b2c3 This is in the portage-2.2.15 release. This change caused new breakage for me, created bug #541302. |
Some months ago I switched to a read-only rootfs, which includes (almost) everything portage touches when trying to install anything. In my sync-script I,'ve long taken care of mounting the filesystem with the gentoo tree and overlays, sources and binpkgs, so it was simple enough to add a remount / rw to it, and since I normally sync as my first order of business, everything appeared to work fine. Today, while investigating a different bug, I manually mounted my package-tree filesystem and started my investigation. Then I decided to try a package remerge, forgetting that my rootfs was still readonly mounted and the actual qmerge would obviously fail. This was the result: >>> Installing (1 of 1) app-arch/ekpar2-0.6.4 Traceback (most recent call last): File "/usr/bin/emerge", line 50, in <module> retval = emerge_main() File "/usr/lib64/portage/pym/_emerge/main.py", line 1031, in emerge_main return run_action(emerge_config) File "/usr/lib64/portage/pym/_emerge/actions.py", line 4062, in run_action emerge_config.args, spinner) File "/usr/lib64/portage/pym/_emerge/actions.py", line 453, in action_build retval = mergetask.merge() File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1019, in merge rval = self._merge() File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1408, in _merge self._main_loop() File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1385, in _main_loop self._event_loop.iteration() File "/usr/lib64/portage/pym/portage/util/_eventloop/EventLoop.py", line 268, in iteration if not x.callback(f, event, *x.args): File "/usr/lib64/portage/pym/portage/util/_async/PipeLogger.py", line 133, in _output_handler self._unregister_if_appropriate(event) File "/usr/lib64/portage/pym/_emerge/AbstractPollTask.py", line 129, in _unregister_if_appropriate self.wait() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait self._wait_hook() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook self._exit_listener_stack.pop()(self) File "/usr/lib64/portage/pym/_emerge/SpawnProcess.py", line 168, in _pipe_logger_exit self.wait() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait self._wait_hook() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook self._exit_listener_stack.pop()(self) File "/usr/lib64/portage/pym/_emerge/EbuildPhase.py", line 266, in _ebuild_exit self.wait() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait self._wait_hook() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook self._exit_listener_stack.pop()(self) File "/usr/lib64/portage/pym/_emerge/EbuildBinpkg.py", line 50, in _package_phase_exit self.wait() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait self._wait_hook() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook self._exit_listener_stack.pop()(self) File "/usr/lib64/portage/pym/_emerge/TaskSequence.py", line 43, in _task_exit_handler self.wait() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait self._wait_hook() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook self._exit_listener_stack.pop()(self) File "/usr/lib64/portage/pym/_emerge/EbuildBuild.py", line 352, in _buildpkg_exit self.wait() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait self._wait_hook() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook self._exit_listener_stack.pop()(self) File "/usr/lib64/portage/pym/_emerge/CompositeTask.py", line 134, in _default_final_exit return self.wait() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait self._wait_hook() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook self._exit_listener_stack.pop()(self) File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1330, in _build_exit self._task_queues.merge.add(merge) File "/usr/lib64/portage/pym/_emerge/SequentialTaskQueue.py", line 23, in add self.schedule() File "/usr/lib64/portage/pym/_emerge/SequentialTaskQueue.py", line 45, in schedule task.start() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 30, in start self._start() File "/usr/lib64/portage/pym/_emerge/PackageMerge.py", line 43, in _start self._start_task(task, self._default_final_exit) File "/usr/lib64/portage/pym/_emerge/CompositeTask.py", line 151, in _start_task task.start() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 30, in start self._start() File "/usr/lib64/portage/pym/_emerge/EbuildMerge.py", line 34, in _start self._start_task(merge_task, self._merge_exit) File "/usr/lib64/portage/pym/_emerge/CompositeTask.py", line 151, in _start_task task.start() File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 30, in start self._start() File "/usr/lib64/portage/pym/portage/dbapi/_MergeProcess.py", line 60, in _start super(MergeProcess, self)._start() File "/usr/lib64/portage/pym/_emerge/SpawnProcess.py", line 106, in _start retval = self._spawn(self.args, **kwargs) File "/usr/lib64/portage/pym/portage/dbapi/_MergeProcess.py", line 156, in _spawn counter = self.vartree.dbapi.counter_tick() File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 869, in counter_tick return self.counter_tick_core(incrementing=1, mycpv=mycpv) File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 952, in counter_tick_core self.lock() File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 220, in lock self._lock = lockdir(self._dbroot) File "/usr/lib64/portage/pym/portage/locks.py", line 52, in lockdir return lockfile(mydir, wantnewlockfile=1, flags=flags) File "/usr/lib64/portage/pym/portage/locks.py", line 105, in lockfile myfd = os.open(lockfilename, os.O_CREAT|os.O_RDWR, 0o660) File "/usr/lib64/portage/pym/portage/__init__.py", line 259, in __call__ rval = self._func(*wrapped_args, **wrapped_kwargs) OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile' While the OSError at the end was quite enough to tell me what was wrong and it was clearly user error that triggered the problem, I've observed general portage policy to be that such catchable user errors should produce nice error messages, not ugly tracebacks. Thus this bug. FWIW, I'm currently on portage-2.2.7, but decided not to put that in the summary since this bug very likely applies to pretty much any near-current portage, and may well apply to all portage versions to-date. If you believe the version should be there anyway, no prob, either change it or ask me to do so. Similarly if you believe emerge --info portage would be helpful. Thanks, Duncan