Bug 257660 – sys-apps/portage-2.2_rc23: repoman commit fails to commit files in CVS subdirectories when checked out without -P

• Product: Portage Development
• Status: RESOLVED FIXED

[Description Harald van Dijk (RETIRED) 2009-02-04 19:24:29 UTC]

$ cvs -d [e-mail]:/var/cvsroot co -d gcc-without-P gentoo-x86/sys-devel/gcc
Enter passphrase for key '/home/harald/.ssh/id_dsa':                                                         
cvs checkout: Updating gcc-without-P                                                                         
U gcc-without-P/ChangeLog                                                                                    
[...]
$ cvs -d [e-mail]:/var/cvsroot co -P -d gcc-with-P gentoo-x86/sys-devel/gcc
Enter passphrase for key '/home/harald/.ssh/id_dsa':                                                         
cvs checkout: Updating gcc-with-P                                                                            
U gcc-with-P/ChangeLog                                                                                       
[...]
$ echo abc >>gcc-without-P/files/c89
$ echo abc >>gcc-with-P/files/c89
$ ln -nsfv /home/harald/gcc-without-P /etc/portage/overlay/sys-devel/gcc
`/etc/portage/overlay/sys-devel/gcc' -> `/home/harald/gcc-without-P'
$ (cd /etc/portage/overlay/sys-devel/gcc && repoman commit)
Enter passphrase for key '/home/harald/.ssh/id_dsa':

RepoMan scours the neighborhood...
>>> Creating Manifest for /etc/portage/overlay/sys-devel/gcc
[...]
* 0 files being committed... 0 have headers that will change.
* Files with headers will cause the manifests to be made and recommited.
Please enter a commit message. Use Ctrl-d to finish or Ctrl-c to abort.
^C[CRITICAL] Interrupted; exiting...
$ ln -nsfv /home/harald/gcc-with-P /etc/portage/overlay/sys-devel/gcc
`/etc/portage/overlay/sys-devel/gcc' -> `/home/harald/gcc-with-P'
$ (cd /etc/portage/overlay/sys-devel/gcc && repoman commit)
Enter passphrase for key '/home/harald/.ssh/id_dsa':

RepoMan scours the neighborhood...
>>> Creating Manifest for /etc/portage/overlay/sys-devel/gcc
[...]
* 1 files being committed... 0 have headers that will change.
* Files with headers will cause the manifests to be made and recommited.
Please enter a commit message. Use Ctrl-d to finish or Ctrl-c to abort.
^C[CRITICAL] Interrupted; exiting...

cvs diff shows c89 as modified in both gcc-without-P and gcc-with-P.

[ebuild   R   ] sys-apps/portage-2.2_rc23  USE="-build -doc -epydoc (-selinux)" LINGUAS="pl" 0 kB
[ebuild   R   ] dev-util/cvs-1.12.12-r6  USE="crypt nls pam -doc -kerberos -server" 0 kB

Portage 2.2_rc23 (default/linux/amd64/2008.0/developer, gcc-4.3.3, glibc-2.9_p20081201-r1, 2.6.28-gentoo-r1 x86_64)
=================================================================
System uname: Linux-2.6.28-gentoo-r1-x86_64-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_4200+-with-glibc2.2.5
Timestamp of tree: Tue, 03 Feb 2009 17:15:02 +0000
app-shells/bash:     3.2_p48
dev-lang/python:     2.5.4-r2
dev-util/cmake:      2.6.2-r1
sys-apps/baselayout: 2.0.0
sys-apps/openrc:     0.4.2
sys-apps/sandbox:    1.3.3
sys-devel/autoconf:  2.13, 2.63
sys-devel/automake:  1.5, 1.7.9-r1, 1.9.6-r2, 1.10.2
sys-devel/binutils:  2.19
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6a
virtual/os-headers:  2.6.28-r1
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O2 -g"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=athlon64 -O2 -g"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps=y --noconfmem --verbose"
FEATURES="assume-digests autopatch collision-protect cvs digest distlocks fixpackages installsources multilib-strict notitles parallel-fetch preserve-libs protect-owned sandbox sfperms sign splitdebug strict unmerge-orphans userfetch userpriv usersandbox"
GENTOO_MIRRORS="http://ftp.snt.utwente.nl/pub/os/linux/gentoo "
INSTALL_MASK="/var/www"
LANG="en_GB.UTF-8"
LDFLAGS="-Xlinker --as-needed"
LINGUAS="en *"
MAKEOPTS="-j1"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/cvs/gentoo-x86 /etc/portage/overlay/linamh /etc/portage/overlay"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext X a52 aac accessibility acl acpi ada agg alsa amd64 aspell async bidi bindist bzip2 cairo cdda cdparanoia cdr chm cjk cli cracklib crypt curl custom-cflags custom-optimization cvs dbus debugger demo dhcp divx dri dvd dvdnav dvdr dvdread eds emacs emboss esd evo fam ffmpeg filepicker flac ftp gif git gmp gnustep gpg gstreamer gtk gucharmap hal haskell htmlhandbook iconv idn inotify inquisitio ipv6 isdnlog ithreads jpeg kde kqemu latex ldap lesstif libffi mad mbox mdnsresponder-compat midi mikmod mmx mmxext mozdevelop mp3 mpeg mudflap multilib ncurses nls nocxx nptl nptlonly nsplugin ntfs objc objc++ ogg opengl openmp oss pam pango pascal pch pcre pdf perl pg-intdatetime pic png portage ppds pppd qa qt3support qt4 quicktime readline realmedia redland reflection rtc samba sdl session snmp songs spell spl sql sqlite sqlite3 sse sse2 ssl startup-notification subversion svg swat sysfs tcl tcpd teletext tetex theora threads tidy tiff timidity tk toolkit-scroll-bars tordns truetype unicode urandom usb v4l v4l2 vim-syntax visibility vlm vorbis wmp xcomposite xim xine xml xorg xscreensaver xslt xulrunner zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cachefilter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en *" USERLAND="GNU" VIDEO_CARDS="vesa"
Unset:  CPPFLAGS, CTARGET, FFLAGS, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

[Comment 1 Harald van Dijk (RETIRED) 2009-02-04 19:26:54 UTC]

This caused bug #257538 while trying to fix #257299, by the way.

[Comment 2 Peter Alfredsen (RETIRED) 2009-02-11 13:26:42 UTC]

Bug 258584 seems to be another case.

[Comment 3 Jory A. Pratt 2009-02-12 03:48:03 UTC]

http://bugs.gentoo.org/show_bug.cgi?id=258670 looks yet like another bug due to portage 2.2_rcX

[Comment 4 Zac Medico 2009-02-12 04:30:46 UTC]

Repoman's code hasn't changed, and it just calls `cvs commit` (relies on cvs to decide which files to send). Adding cvs-utils@ to CC in case they have any insight.

[Comment 5 Robin Johnson 2009-02-12 05:40:55 UTC]

Can you please show me the output of 'cvs up' in both variants of your directories?
Also include your ~/.cvsrc file.

[Comment 6 Harald van Dijk (RETIRED) 2009-02-12 18:30:23 UTC]

(In reply to comment #5)
> Can you please show me the output of 'cvs up' in both variants of your
> directories?

[gcc-without-P]
Enter passphrase for key '/home/harald/.ssh/id_dsa':
cvs update: Updating .
cvs update: Updating files
M files/c89
cvs update: Updating files/2.95.3
cvs update: Updating files/3.0
cvs update: Updating files/3.1
cvs update: Updating files/3.1.1
cvs update: Updating files/3.2
cvs update: Updating files/3.2.1
cvs update: Updating files/3.2.2
cvs update: Updating files/3.2.3
cvs update: Updating files/3.3
cvs update: Updating files/3.3.1
cvs update: Updating files/3.3.2
cvs update: Updating files/3.3.3
cvs update: Updating files/3.3.4
cvs update: Updating files/3.3.5
cvs update: Updating files/3.3.6
cvs update: Updating files/3.4.0
cvs update: Updating files/3.4.1
cvs update: Updating files/3.4.2
cvs update: Updating files/3.4.3
cvs update: Updating files/3.4.4
cvs update: Updating files/4.0.2
cvs update: Updating files/4.1.0
cvs update: Updating files/4.3.2
cvs update: Updating files/awk
cvs update: Updating files/gcc-2.96.20000731
cvs update: Updating files/stubs

[gcc-with-P]
Enter passphrase for key '/home/harald/.ssh/id_dsa':
cvs update: Updating .
cvs update: Updating files
M files/c89
cvs update: Updating files/3.2.1
cvs update: Updating files/3.2.2
cvs update: Updating files/3.2.3
cvs update: Updating files/3.3.4
cvs update: Updating files/3.3.6
cvs update: Updating files/3.4.0
cvs update: Updating files/3.4.1
cvs update: Updating files/3.4.2
cvs update: Updating files/3.4.3
cvs update: Updating files/3.4.4
cvs update: Updating files/4.0.2
cvs update: Updating files/4.1.0
cvs update: Updating files/4.3.2
cvs update: Updating files/awk
cvs update: Updating files/stubs

> Also include your ~/.cvsrc file.

ls: cannot access /home/harald/.cvsrc: No such file or directory

:-)

(In reply to comment #4)
> Repoman's code hasn't changed, and it just calls `cvs commit` (relies on cvs to
> decide which files to send).

repoman decides which files to send by scanning CVS/*; see the call to cvstree.getentries and findchanged. The list of files is kept, and eventually passed to cvs commit, and I suppose if you made no other changes, cvs commit by itself will figure out what to commit, but I also updated the ChangeLog, so repoman passed only that.

[Comment 7 Serkan Kaba (RETIRED) 2009-03-29 08:45:59 UTC]

Another problem with it. Bug #262692 c#7

[Comment 8 Zac Medico 2009-03-29 09:04:11 UTC]

(In reply to comment #6)
> repoman decides which files to send by scanning CVS/*; see the call to
> cvstree.getentries and findchanged. The list of files is kept, and eventually
> passed to cvs commit, and I suppose if you made no other changes, cvs commit by
> itself will figure out what to commit, but I also updated the ChangeLog, so
> repoman passed only that.

It seems like repoman makes unreliable assumptions about CVS/Entries, which causes it to generate an incorrect list of files to commit. So, it's not really a fault of cvs. We'll just have to find some other way for repoman to identify which files need to be committed.

[Comment 9 Harald van Dijk (RETIRED) 2009-06-25 20:42:56 UTC]

I'm wondering, is there any case where you want to skip a subdirectory? If the directory is empty, it doesn't matter either way. If the directory is not empty, you will want to report any files in it that have not been added to CVS, even if the directory itself hasn't been either. So I'm thinking you could simply check all subdirectories, and pay attention to CVS/Entries only for files.

[Comment 10 Christian Ruppert (idl0r) 2009-09-10 21:52:04 UTC]

Created attachment 203733 [details, diff]
cvstree.py.patch

Guys, please test :)

[Comment 11 Christian Ruppert (idl0r) 2009-09-11 18:24:14 UTC]

Steps to reproduce:

Ensure that your ~/.cvsrc will not be used.

cvs -d :pserver:[e-mail]:/var/cvsroot co gentoo-x86/app-accessibility/SphinxTrain
find gentoo-x86 -type f -name 'Entries' -exec grep '^D' {} ';'

rm -rf gentoo-x86
cvs -d :pserver:[e-mail]:/var/cvsroot co -P gentoo-x86/app-accessibility/SphinxTrain
find gentoo-x86 -type f -name 'Entries' -exec grep '^D' {} ';'

[Comment 12 Zac Medico 2009-09-12 01:37:17 UTC]

(In reply to comment #10)
> Created an attachment (id=203733) [edit]
> cvstree.py.patch

Thanks, that's in svn r14222.

[Comment 13 Zac Medico 2009-09-20 00:21:24 UTC]

This is fixed in 2.2_rc41.

[Comment 14 Zac Medico 2009-10-11 00:45:37 UTC]

This is fixed in 2.1.7.