Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 485308 - sys-apps/portage-2.2.6 handling of file's mtime property is not correct - lead to problems
Summary: sys-apps/portage-2.2.6 handling of file's mtime property is not correct - lea...
Status: UNCONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Interface (emerge) (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-09-18 14:29 UTC by zack24
Modified: 2013-09-20 12:47 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description zack24 2013-09-18 14:29:32 UTC
If unmerging certain packages some files are not deleted because of assuming manual modification which is wrong.
As an example we take dev-texlive/texlive-basic to demonstrate the issue.

1.
emerge --unmerge dev-texlive/texlive-basic
gives us:

-cut-
--- !mtime   obj /var/lib/texmf/web2c/tex/tex.log
--- !mtime   obj /var/lib/texmf/web2c/tex/tex.fmt
--- !mtime   obj /var/lib/texmf/web2c/pdftex/pdftex.log
--- !mtime   obj /var/lib/texmf/web2c/pdftex/pdftex.fmt
--- !mtime   obj /var/lib/texmf/web2c/pdftex/pdfetex.log
--- !mtime   obj /var/lib/texmf/web2c/pdftex/pdfetex.fmt
--- !mtime   obj /var/lib/texmf/web2c/pdftex/etex.log
--- !mtime   obj /var/lib/texmf/web2c/pdftex/etex.fmt
--- !mtime   obj /var/lib/texmf/web2c/metafont/mf.log
--- !mtime   obj /var/lib/texmf/web2c/metafont/mf.base
--- !mtime   obj /var/lib/texmf/web2c/luatex/luatex.log
--- !mtime   obj /var/lib/texmf/web2c/luatex/luatex.fmt
--- !mtime   obj /var/lib/texmf/web2c/luatex/dviluatex.log
--- !mtime   obj /var/lib/texmf/web2c/luatex/dviluatex.fmt
<<<          obj /usr/share/texmf-dist/web2c/texmfcnf.lua
-cut-

2. emerge dev-texlive/texlive-basic 
gives us:

-cut-
 * Detected file collision(s):
 * 
 * 	/var/lib/texmf/web2c/pdftex/pdftex.fmt
 * 	/var/lib/texmf/web2c/pdftex/pdftex.log
 * 	/var/lib/texmf/web2c/pdftex/pdfetex.fmt
 * 	/var/lib/texmf/web2c/pdftex/etex.fmt
 * 	/var/lib/texmf/web2c/pdftex/pdfetex.log
 * 	/var/lib/texmf/web2c/pdftex/etex.log
 * 	/var/lib/texmf/web2c/luatex/luatex.log
 *  	/var/lib/texmf/web2c/luatex/luatex.fmt
 * 	/var/lib/texmf/web2c/luatex/dviluatex.fmt
 * 	/var/lib/texmf/web2c/luatex/dviluatex.log
 * 	/var/lib/texmf/web2c/tex/tex.fmt
 *  	/var/lib/texmf/web2c/tex/tex.log
 * 	/var/lib/texmf/web2c/metafont/mf.log
 * 	/var/lib/texmf/web2c/metafont/mf.base
 * Package 'dev-texlive/texlive-basic-2013' merged despite file
 * collisions. If necessary, refer to your elog messages for the whole
-cut-

Let us compare two files: texmfcnf.lua, causing no problem and
tex.log which one from the bad list

# stat tex.log 
  File: ‘tex.log’
  Size: 2454      	Blocks: 8          IO Block: 4096   regular file
Device: 801h/2049d	Inode: 131283      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2013-09-18 14:31:11.767481683 +0200
Modify: 2013-09-18 14:31:11.807481682 +0200
Change: 2013-09-18 14:31:11.807481682 +0200
 Birth: -
Press any key to continue...

# stat texmfcnf.lua 
  File: ‘texmfcnf.lua’
  Size: 8288      	Blocks: 24         IO Block: 4096   regular file
Device: 801h/2049d	Inode: 573743      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2013-09-18 14:30:44.000000000 +0200
Modify: 2013-09-18 14:30:44.000000000 +0200
Change: 2013-09-18 14:31:08.867481731 +0200

The timestamp is different but it should not be a reason of considering
one of them as manually modified.

The sequence of merge and unmerge can be repeated as many times as desired
with the same result. Previous versions of portage have the same issue.

Link to initial reporting of the problem:
http://forums.gentoo.org/viewtopic.php?p=7398620#7398620

Reproducible: Always

Steps to Reproduce:
1. see description
2.
3.
Actual Results:  
see description

Expected Results:  
see description
Comment 1 zack24 2013-09-18 14:32:42 UTC
Please understand the given package is not a problem and was chosen as an example only! Many other packages affected, but probably not all.
Comment 2 Zac Medico gentoo-dev 2013-09-18 18:20:19 UTC
Please post output of `emerge --info`.
Comment 3 zack24 2013-09-19 05:17:31 UTC
Portage 2.2.6 (default/linux/x86/13.0, gcc-4.8.1, glibc-2.18, 3.11.0-rc7 i686)
=================================================================
System uname: Linux-3.11.0-rc7-i686-Intel-R-_Core-TM-_i5_CPU_M_460_@_2.53GHz-with-gentoo-2.2
KiB Mem:     3932428 total,    272124 free
KiB Swap:          0 total,         0 free
Timestamp of tree: Wed, 18 Sep 2013 17:00:01 +0000
ld GNU ld (GNU Binutils) 2.23.2
app-shells/bash:          4.2_p45
dev-lang/python:          2.7.5-r2
dev-util/cmake:           2.8.11.2
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6, 1.13.4, 1.14
sys-devel/binutils:       2.23.2
sys-devel/gcc:            4.8.1
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.11 (virtual/os-headers)
sys-libs/glibc:           2.18

ABI_X86="32"
ACCEPT_KEYWORDS="x86 ~x86"
ACCEPT_LICENSE="*"
ACCEPT_PROPERTIES="*"
ACCEPT_RESTRICT="*"
ALSA_CARDS=""
APACHE2_MODULES=""
ARCH="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=native -mtune=generic -Os -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CLEAN_DELAY="5"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CPPFLAGS="-march=native -mtune=generic -Os -fomit-frame-pointer"
CURL_SSL="openssl"
CVS_RSH="ssh"
CXXFLAGS="-march=native -mtune=generic -Os -fomit-frame-pointer"
ELIBC="glibc"
FCFLAGS="-march=native -mtune=generic -Os -fomit-frame-pointer"

FEATURES="assume-digests buildpkg downgrade-backup nodoc noinfo noman"

LCD_DEVICES=""
LC_COLLATE="C"
LC_MESSAGES="C"
LDFLAGS="-Wl,-O2 -Wl,--as-needed -Wl,--enable-new-dtags -Wl,--sort-common -s"
LESS="-R -M --shift 5"
LESSOPEN="|lesspipe %s"
LIBREOFFICE_EXTENSIONS=""
LINGUAS="en"

MAKEOPTS="-j1 -s"
MANPATH="/usr/local/share/man:/usr/share/man:/usr/share/gcc-data/i686-pc-linux-gnu/4.8.1/man:/usr/share/binutils-data/i686-pc-linux-gnu/2.23.2/man"
MC_SID="1856"
MC_TMPDIR="/tmp/mc-root"
MMGT_CLEAR="1"
NETBEANS="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml"
OFFICE_IMPLEMENTATION=""
OLDPWD="/portage/gentoo"
PAGER="/usr/bin/less"
PATH="/sbin:/bin:/usr/sbin:/usr/bin"
PHP_TARGETS=""
PKGDIR="/home/portage/packages"
PKG_DBDIR="/home/portage/pkg"
PORT="/home/portage"
PORTAGE_ARCHLIST="ppc sparc64-freebsd ppc-openbsd x86-openbsd ppc64 x86-winnt x86-fbsd ppc-aix alpha arm x86-freebsd s390 amd64 arm-linux x86-macos x64-openbsd ia64-hpux hppa x86-netbsd ppc64-linux x86-cygwin amd64-linux ia64-linux x86 sparc-solaris x64-freebsd sparc64-solaris x86-linux x64-macos sparc m68k-mint ia64 mips ppc-macos x86-interix hppa-hpux amd64-fbsd x64-solaris m68k sh x86-solaris sparc-fbsd"
PORTAGE_BIN_PATH="/usr/lib/portage/bin"
PORTAGE_COMPRESS="xz"
PORTAGE_COMPRESS_EXCLUDE_SUFFIXES="css gif htm[l]? jp[e]?g js pdf png"
PORTAGE_COMPRESS_FLAGS="-z -e -9 -v"
PORTAGE_CONFIGROOT="/"
PORTAGE_DEBUG="0"
PORTAGE_DEPCACHEDIR="/var/cache/edb/dep"
PORTAGE_ELOG_CLASSES="log warn error"
PORTAGE_ELOG_MAILFROM="portage@localhost"
PORTAGE_ELOG_MAILSUBJECT="[portage] ebuild log for ${PACKAGE} on ${HOST}"
PORTAGE_ELOG_MAILURI="root"
PORTAGE_ELOG_SYSTEM="save_summary:log,warn,error,qa echo"
PORTAGE_FETCH_CHECKSUM_TRY_MIRRORS="5"
PORTAGE_FETCH_RESUME_MIN_SIZE="350K"
PORTAGE_GID="250"
PORTAGE_GPG_SIGNING_COMMAND="gpg --sign --digest-algo SHA256 --clearsign --yes --default-key "${PORTAGE_GPG_KEY}" --homedir "${PORTAGE_GPG_DIR}" "${FILE}""
PORTAGE_INST_GID="0"
PORTAGE_INST_UID="0"
PORTAGE_INTERNAL_CALLER="1"
PORTAGE_OVERRIDE_EPREFIX=""
PORTAGE_PYM_PATH="/usr/lib/portage/pym"
PORTAGE_PYTHONPATH="/usr/lib/portage/pym"
PORTAGE_REPOSITORIES="[DEFAULT]
-cut-
Comment 4 Zac Medico gentoo-dev 2013-09-19 05:34:48 UTC
(In reply to zack24 from comment #3)
> FEATURES="assume-digests buildpkg downgrade-backup nodoc noinfo noman"

It looks like you've removed most of the default FEATURES settings. For the issue that you have reported, adding unmerge-orphans to FEATURES will restore the default behavior. You really should not remove default settings from FEATURES unless you know what you are doing.
Comment 5 zack24 2013-09-19 13:55:36 UTC
Zac! Please be so kind and read the manual !

unmerge-orphans
    If a file is not claimed by another package in the same slot and it is not protected by CONFIG_PROTECT, unmerge it even if the modification time or checksum differs from the file that was originally installed.


And here we go:

#:qfile /var/lib/texmf/web2c/tex/tex.log
dev-texlive/texlive-basic (/var/lib/texmf/web2c/tex/tex.log)


It means in other words, that the file tex.log BELONGS to the given package
and MUST be removed during unmerge independent or any settings any user might have.

Tell me what's the difference between 2 files I took as an example?
Why one is successfully deleted and another one not? Both files located
in CONTENTS of corresponding package and have to be removed from the system
during uninstall. In case it's control sum has been changed (edited) one can consider to skip it.

I insist, that it's a bug and not matter of unmerge-orphans flag.
I have many files in /usr/bin/* which are program files and remained in the system after deleting corresponding package. It is unrealistic to consider some one has edited them!
Comment 6 Zac Medico gentoo-dev 2013-09-19 16:58:02 UTC
(In reply to zack24 from comment #5)
> Zac! Please be so kind and read the manual !
> 
> unmerge-orphans
>     If a file is not claimed by another package in the same slot and it is
> not protected by CONFIG_PROTECT, unmerge it even if the modification time or
> checksum differs from the file that was originally installed.
> 
> 
> And here we go:
> 
> #:qfile /var/lib/texmf/web2c/tex/tex.log
> dev-texlive/texlive-basic (/var/lib/texmf/web2c/tex/tex.log)
> 
> 
> It means in other words, that the file tex.log BELONGS to the given package
> and MUST be removed during unmerge independent or any settings any user
> might have.

Prior to addition of the unmerge-orphans feature, the default portage behavior was to _not_ unmerge files that had modification times differing from those recorded in CONTENTS. The unmerge-orphans feature was added to change the behavior as described in the man page. The unmerge-orphans feature has since been enabled be default. If you want some slightly different behavior with respect to mtimes, then we could add a new feature to trigger that.

> Tell me what's the difference between 2 files I took as an example?
> Why one is successfully deleted and another one not? Both files located
> in CONTENTS of corresponding package and have to be removed from the system
> during uninstall. In case it's control sum has been changed (edited) one can
> consider to skip it.

The mtime of one file matches the mtime recorded in CONTENTS, while the other one does not.

> I insist, that it's a bug and not matter of unmerge-orphans flag.

You're talking about changing the long-standing behavior. As said, we can add a new features setting to make it behave like you want.
Comment 7 zack24 2013-09-19 20:24:25 UTC
Zac,

It doesn't look like you understand what I am writing here.

Please check carefully again what I have done.

1. I have unmerged intentionally certain package and then merged it back again.
2. After that I repeat the same procedure again.

There was nothing in between. No file edit, no any other operation.

It means, that this:
"that had modification times differing from those recorded in CONTENTS."

can not be valid. Rather mtime recordered in CONTENTS IS WRONG!

That is a bug and that is what I am talking about.


"The mtime of one file matches the mtime recorded in CONTENTS, while the other one does not."

Exactly. And that is problem of portage! No one else has modified timestamp of the files. Only the installer could do that.

I'm guessing that recording of the mtime into CONTENTS is happening
in the wrong time or with not real folder - still in /something/tmp
this I didn't check.

Well in overall that conception is wrong and silly. Not timestamp, but
control sum must be used instead. Anyway even that solution must work properly
although it is error prone.

I am just asking to confirm the bug. Please do. Thx.
Comment 8 Zac Medico gentoo-dev 2013-09-19 22:19:12 UTC
(In reply to zack24 from comment #7)
> "The mtime of one file matches the mtime recorded in CONTENTS, while the
> other one does not."
> 
> Exactly. And that is problem of portage! No one else has modified timestamp
> of the files. Only the installer could do that.
> 
> I'm guessing that recording of the mtime into CONTENTS is happening
> in the wrong time or with not real folder - still in /something/tmp
> this I didn't check.

It's not portage's fault. The ebuild is probably modifying the files in pkg_postinst or some similar phase.
Comment 9 zack24 2013-09-20 11:16:08 UTC
Hm. Do you still think, that presented behavior is desired? If so, then we have different opinions and there is nothing we can discuss. close the bug
and I will report NO bugs anymore. bye.
Comment 10 Zac Medico gentoo-dev 2013-09-20 12:47:25 UTC
The historical reason for protecting files with changed mtimes was that older versions of portage relied on the mtime differences to protect new files during upgrades. However, portage no longer relies on mtimes like this, so it should be safe to change the default unmerge behavior.