Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 490732

Summary: sys-apps/portage traceback on read-only rootfs
Product: Portage Development Reporter: Duncan <1i5t5.duncan>
Component: CoreAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: normal CC: hlein
Priority: Normal Keywords: InVCS
Version: 2.2   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=712222
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 484436    

Description Duncan 2013-11-08 08:03:21 UTC
Some months ago I switched to a read-only rootfs, which includes (almost) everything portage touches when trying to install anything.  In my sync-script I,'ve long taken care of mounting the filesystem with the gentoo tree and overlays, sources and binpkgs, so it was simple enough to add a remount / rw to it, and since I normally sync as my first order of business, everything appeared to work fine.

Today, while investigating a different bug, I manually mounted my package-tree filesystem and started my investigation.  Then I decided to try a package remerge, forgetting that my rootfs was still readonly mounted and the actual qmerge would obviously fail.  This was the result:

>>> Installing (1 of 1) app-arch/ekpar2-0.6.4
Traceback (most recent call last):
  File "/usr/bin/emerge", line 50, in <module>
    retval = emerge_main()
  File "/usr/lib64/portage/pym/_emerge/main.py", line 1031, in emerge_main
    return run_action(emerge_config)
  File "/usr/lib64/portage/pym/_emerge/actions.py", line 4062, in run_action
    emerge_config.args, spinner)
  File "/usr/lib64/portage/pym/_emerge/actions.py", line 453, in action_build
    retval = mergetask.merge()
  File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1019, in merge
    rval = self._merge()
  File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1408, in _merge
    self._main_loop()
  File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1385, in _main_loop
    self._event_loop.iteration()
  File "/usr/lib64/portage/pym/portage/util/_eventloop/EventLoop.py", line 268, in iteration
    if not x.callback(f, event, *x.args):
  File "/usr/lib64/portage/pym/portage/util/_async/PipeLogger.py", line 133, in _output_handler
    self._unregister_if_appropriate(event)
  File "/usr/lib64/portage/pym/_emerge/AbstractPollTask.py", line 129, in _unregister_if_appropriate
    self.wait()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib64/portage/pym/_emerge/SpawnProcess.py", line 168, in _pipe_logger_exit
    self.wait()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib64/portage/pym/_emerge/EbuildPhase.py", line 266, in _ebuild_exit
    self.wait()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib64/portage/pym/_emerge/EbuildBinpkg.py", line 50, in _package_phase_exit
    self.wait()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib64/portage/pym/_emerge/TaskSequence.py", line 43, in _task_exit_handler
    self.wait()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib64/portage/pym/_emerge/EbuildBuild.py", line 352, in _buildpkg_exit
    self.wait()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib64/portage/pym/_emerge/CompositeTask.py", line 134, in _default_final_exit
    return self.wait()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 57, in wait
    self._wait_hook()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 175, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib64/portage/pym/_emerge/Scheduler.py", line 1330, in _build_exit
    self._task_queues.merge.add(merge)
  File "/usr/lib64/portage/pym/_emerge/SequentialTaskQueue.py", line 23, in add
    self.schedule()
  File "/usr/lib64/portage/pym/_emerge/SequentialTaskQueue.py", line 45, in schedule
    task.start()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 30, in start
    self._start()
  File "/usr/lib64/portage/pym/_emerge/PackageMerge.py", line 43, in _start
    self._start_task(task, self._default_final_exit)
  File "/usr/lib64/portage/pym/_emerge/CompositeTask.py", line 151, in _start_task
    task.start()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 30, in start
    self._start()
  File "/usr/lib64/portage/pym/_emerge/EbuildMerge.py", line 34, in _start
    self._start_task(merge_task, self._merge_exit)
  File "/usr/lib64/portage/pym/_emerge/CompositeTask.py", line 151, in _start_task
    task.start()
  File "/usr/lib64/portage/pym/_emerge/AsynchronousTask.py", line 30, in start
    self._start()
  File "/usr/lib64/portage/pym/portage/dbapi/_MergeProcess.py", line 60, in _start
    super(MergeProcess, self)._start()
  File "/usr/lib64/portage/pym/_emerge/SpawnProcess.py", line 106, in _start
    retval = self._spawn(self.args, **kwargs)
  File "/usr/lib64/portage/pym/portage/dbapi/_MergeProcess.py", line 156, in _spawn
    counter = self.vartree.dbapi.counter_tick()
  File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 869, in counter_tick
    return self.counter_tick_core(incrementing=1, mycpv=mycpv)
  File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 952, in counter_tick_core
    self.lock()
  File "/usr/lib64/portage/pym/portage/dbapi/vartree.py", line 220, in lock
    self._lock = lockdir(self._dbroot)
  File "/usr/lib64/portage/pym/portage/locks.py", line 52, in lockdir
    return lockfile(mydir, wantnewlockfile=1, flags=flags)
  File "/usr/lib64/portage/pym/portage/locks.py", line 105, in lockfile
    myfd = os.open(lockfilename, os.O_CREAT|os.O_RDWR, 0o660)
  File "/usr/lib64/portage/pym/portage/__init__.py", line 259, in __call__
    rval = self._func(*wrapped_args, **wrapped_kwargs)
OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile'


While the OSError at the end was quite enough to tell me what was wrong and it was clearly user error that triggered the problem, I've observed general portage policy to be that such catchable user errors should produce nice error messages, not ugly tracebacks.  Thus this bug.

FWIW, I'm currently on portage-2.2.7, but decided not to put that in the summary since this bug very likely applies to pretty much any near-current portage, and may well apply to all portage versions to-date.  If you believe the version should be there anyway, no prob, either change it or ask me to do so.  Similarly if you believe emerge --info portage would be helpful.

Thanks,
Duncan
Comment 1 Duncan 2014-04-16 03:39:29 UTC
Here's a second instance of the bug, this time with an emerge --pretend.  Why should an emerge --pretend need a read-write filesystem -- it's *PRETEND*!?

In this case the OSError relates to the lockfile for (what would be) the news file for my local overlay, jed-local (jed being my initials).  Except, being the local overlay, why would I ever HAVE any news to read at all?  If I know it to put it in some news file, I know it and don't NEED to read about it in a news file. =:^)

Between that and the fact that I'm getting this with a --pretend, which shouldn't need a writable filesystem in the first place as it IS pretend, it's a rather frustrating bug.  Luckily portage spits out the information from the pretend that I need, before it does the traceback.

Here's an example.  It appears to happen with any emerge --pretend I try, however.

$ emerge --pretend systemd

These are the packages that would be merged, in order:

Calculating dependencies  ..... done!
[ebuild   R    ] sys-apps/systemd-212-r2 
Traceback (most recent call last):
  File "/usr/bin/emerge", line 50, in <module>
    retval = emerge_main()
  File "/lib64/portage/pym/_emerge/main.py", line 1037, in emerge_main
    return run_action(emerge_config)
  File "/lib64/portage/pym/_emerge/actions.py", line 4064, in run_action
    emerge_config.trees, emerge_config.target_config.mtimedb, retval)
  File "/lib64/portage/pym/portage/proxy/objectproxy.py", line 31, in __call__
    return result(*args, **kwargs)
  File "/lib64/portage/pym/_emerge/post_emerge.py", line 110, in post_emerge
    display_news_notification(root_config, myopts)
  File "/lib64/portage/pym/_emerge/post_emerge.py", line 43, in display_news_notification
    news_counts = count_unread_news(portdb, vardb)
  File "/lib64/portage/pym/portage/news.py", line 388, in count_unread_news
    count = manager.getUnreadItems(repo, update=True)
  File "/lib64/portage/pym/portage/news.py", line 180, in getUnreadItems
    unread_lock = lockfile(unread_filename, wantnewlockfile=1)
  File "/lib64/portage/pym/portage/locks.py", line 106, in lockfile
    myfd = os.open(lockfilename, os.O_CREAT|os.O_RDWR, 0o660)
  File "/lib64/portage/pym/portage/__init__.py", line 259, in __call__
    rval = self._func(*wrapped_args, **wrapped_kwargs)
OSError: [Errno 30] Read-only file system: '/var/lib/gentoo/news/.news-jed-local.unread.portage_lockfile'

FWIW I'm now on portage-2.2.10.
Comment 2 Zac Medico gentoo-dev 2014-11-10 06:47:18 UTC
(In reply to Duncan from comment #1)
> OSError: [Errno 30] Read-only file system:
> '/var/lib/gentoo/news/.news-jed-local.unread.portage_lockfile'

I have a patch for this in the following branch:

	https://github.com/zmedico/portage/tree/bug_490732

I've posted the patch for review here:

	http://thread.gmane.org/gmane.linux.gentoo.portage.devel/4766

(In reply to Duncan from comment #0)
> OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile'

We could handle this similarly to the PermissionDenied exception, which can be raised from anywhere and results in a short error message like "Permission Denied: /var/db/.pkg.portage_lockfile".
Comment 3 Zac Medico gentoo-dev 2014-11-10 07:27:40 UTC
(In reply to Zac Medico from comment #2)
> (In reply to Duncan from comment #0)
> > OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile'
> 
> We could handle this similarly to the PermissionDenied exception, which can
> be raised from anywhere and results in a short error message like
> "Permission Denied: /var/db/.pkg.portage_lockfile".

Actually, I'm not so sure about hiding the traceback now, since it could make it difficult to  diagnose bugs involving unhandled ReadOnlyFileSystem exceptions (like the one reported in comment #1).
Comment 4 Zac Medico gentoo-dev 2014-11-18 01:34:11 UTC
(In reply to Zac Medico from comment #2)
> (In reply to Duncan from comment #1)
> > OSError: [Errno 30] Read-only file system:
> > '/var/lib/gentoo/news/.news-jed-local.unread.portage_lockfile'
> 
> I have a patch for this in the following branch:
> 
> 	https://github.com/zmedico/portage/tree/bug_490732

This one is in the master branch now:

https://github.com/gentoo/portage/commit/e6121d92e586c0f9b75998eca2d1fbb46db7c821

(In reply to Duncan from comment #0)
> OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile'

For this one, I'm going to make it bail out (without traceback) after the dependency calculation if there are packages to be installed and /var/db is mounted readonly. If you re-mount it readonly after it has passed this check, then it will still show a traceback as before, since hiding the traceback at any point after that is likely to make bugs more difficult to diagnose.
Comment 5 Zac Medico gentoo-dev 2014-11-18 09:22:37 UTC
(In reply to Zac Medico from comment #4)
> (In reply to Duncan from comment #0)
> > OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile'
> 
> For this one, I'm going to make it bail out (without traceback) after the
> dependency calculation if there are packages to be installed and /var/db is
> mounted readonly.

There's a patch for this in my branch:

	https://github.com/zmedico/portage/tree/bug_490732

I've posted it for review here:

	http://thread.gmane.org/gmane.linux.gentoo.portage.devel/4816
Comment 6 Zac Medico gentoo-dev 2014-11-20 04:11:42 UTC
(In reply to Zac Medico from comment #5)
> (In reply to Zac Medico from comment #4)
> > (In reply to Duncan from comment #0)
> > > OSError: [Errno 30] Read-only file system: '/var/db/.pkg.portage_lockfile'
> > 
> > For this one, I'm going to make it bail out (without traceback) after the
> > dependency calculation if there are packages to be installed and /var/db is
> > mounted readonly.
> 
> There's a patch for this in my branch:
> 
> 	https://github.com/zmedico/portage/tree/bug_490732

It's in the master branch now:

https://github.com/gentoo/portage/commit/103d27694197893badb3b5b5fb2e476e04f88033

We could do something similar to check for write access to $PKGDIR when appropriate.
Comment 7 Zac Medico gentoo-dev 2014-11-20 12:56:21 UTC
(In reply to Zac Medico from comment #6)
> We could do something similar to check for write access to $PKGDIR when
> appropriate.

There's a patch for this in my branch. I've posted it here for review:

http://gentoo.2317880.n4.nabble.com/PATCH-emerge-check-for-writable-PKGDIR-490732-td288311.html
Comment 8 Duncan 2014-11-21 07:20:05 UTC
Just saying thanks.  And thanks for the profiling related patches too.  I'm sure I'm not the only gentooer and portage user glad you're back... with a better bus-factor now. =:^)
Comment 9 Zac Medico gentoo-dev 2014-11-24 08:18:37 UTC
(In reply to Duncan from comment #8)
> Just saying thanks.  And thanks for the profiling related patches too.  I'm
> sure I'm not the only gentooer and portage user glad you're back... with a
> better bus-factor now. =:^)

You're welcome.

(In reply to Zac Medico from comment #7)
> http://gentoo.2317880.n4.nabble.com/PATCH-emerge-check-for-writable-PKGDIR-
> 490732-td288311.html

This one's in the master branch now:

https://github.com/gentoo/portage/commit/206efe5f6341bce99a5e9994a0458c304513b2c3
Comment 10 Brian Dolbec (RETIRED) gentoo-dev 2014-12-04 05:50:20 UTC
This is in the portage-2.2.15 release.
Comment 11 Hank Leininger 2015-02-25 03:59:51 UTC
This change caused new breakage for me, created bug #541302.