Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 632317 - cdrom.eclass does not work with multiple CDs
Summary: cdrom.eclass does not work with multiple CDs
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Games
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-09-28 21:33 UTC by Gary Macindoe
Modified: 2017-12-23 23:05 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Gary Macindoe 2017-09-28 21:33:24 UTC
cdrom.eclass fails to find the second and subsequent disks when installing a package that requires multiple CDs.

Example output:
 * UT2003CrashFix.zip SHA256 SHA512 WHIRLPOOL size ;-) ...               [ ok ]
 * The installed game takes about 2.7GB of space!
>>> Unpacking source...
 * Found CD #1 root at /mnt/usb/ut2003-data
Archive:  /var/tmp/portage/games-fps/ut2003-data-2107/distdir/UT2003CrashFix.zip
   creating: CrashFix/CrashFix/
   creating: CrashFix/CrashFix/classes/
  inflating: CrashFix/CrashFix/classes/iAccessControl.uc  
  inflating: CrashFix/CrashFix/classes/iAccessControlIni.uc  
   creating: CrashFix/Help/
  inflating: CrashFix/Help/readme.txt  
   creating: CrashFix/System/
  inflating: CrashFix/System/crashfix.u  
>>> Source unpacked in /var/tmp/portage/games-fps/ut2003-data-2107/work
>>> Preparing source in /var/tmp/portage/games-fps/ut2003-data-2107/work ...
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/games-fps/ut2003-data-2107/work ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/games-fps/ut2003-data-2107/work ...
>>> Source compiled.
>>> Test phase [not enabled]: games-fps/ut2003-data-2107

>>> Install ut2003-data-2107 into /var/tmp/portage/games-fps/ut2003-data-2107/image/ category games-fps
 * Copying files from Disk 1...
 * ERROR: games-fps/ut2003-data-2107::gentoo failed (install phase):
 *   unable to locate CD #2 root at /mnt/usb/ut2003-data
 * 
 * Call stack:
 *     ebuild.sh, line  115:  Called src_install
 *   environment, line 2954:  Called cdrom_load_next_cd
 *   environment, line  735:  Called die
 * The specific snippet of code:
 *               die "unable to locate CD #${CDROM_CURRENT_CD} root at ${CDROM_ROOT}";
 * 
 * If you need support, post the output of `emerge --info '=games-fps/ut2003-data-2107::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=games-fps/ut2003-data-2107::gentoo'`.
 * The complete build log is located at '/var/log/portage/games-fps:ut2003-data-2107:20170928-203012.log.gz'.
 * For convenience, a symlink to the build log is located at '/var/tmp/portage/games-fps/ut2003-data-2107/temp/build.log.gz'.
 * The ebuild environment file is located at '/var/tmp/portage/games-fps/ut2003-data-2107/temp/environment'.
 * Working directory: '/var/tmp/portage/games-fps/ut2003-data-2107/work'
 * S: '/var/tmp/portage/games-fps/ut2003-data-2107/work'
Comment 1 Gary Macindoe 2017-09-28 21:34:19 UTC
emerge --info

Portage 2.3.8 (python 3.6.1-final-0, default/linux/amd64/13.0/desktop/plasma, gcc-6.4.0, glibc-2.25-r4, 4.5.4-tuxonice x86_64)
=================================================================
System uname: Linux-4.5.4-tuxonice-x86_64-Intel-R-_Core-TM-_i7-3630QM_CPU_@_2.40GHz-with-gentoo-2.3
KiB Mem:    32716636 total,  21188144 free
KiB Swap:    1028092 total,   1028092 free
Timestamp of repository gentoo: Fri, 01 Sep 2017 10:45:02 +0000
Head commit of repository gentoo: a01ed4b99a6a266217bea712a70cdc87268c00eb
Head commit of repository Local-Overlay: bd3812431156c307d0609fcd5e400408b562bf9b

sh bash 4.3_p48-r1
ld GNU ld (Gentoo 2.28.1 p1.0) 2.28.1
app-shells/bash:          4.3_p48-r1::gentoo
dev-java/java-config:     2.2.0-r3::gentoo
dev-lang/perl:            5.24.1-r2::gentoo
dev-lang/python:          2.7.13::gentoo, 3.5.3::gentoo, 3.6.1-r1::gentoo
dev-util/cmake:           3.9.1::gentoo
dev-util/pkgconfig:       0.28-r2::gentoo
sys-apps/baselayout:      2.3::gentoo
sys-apps/openrc:          0.28::gentoo
sys-apps/sandbox:         2.10-r3::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69::gentoo
sys-devel/automake:       1.15-r2::gentoo
sys-devel/binutils:       2.28.1::gentoo
sys-devel/gcc:            5.4.0-r3::gentoo, 6.4.0::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r3::gentoo
sys-devel/make:           4.2.1::gentoo
sys-kernel/linux-headers: 4.11::gentoo (virtual/os-headers)
sys-libs/glibc:           2.25-r4::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: ssh://raspberrypi.garymacindoe.co.uk/gentoo-portage
    sync-user: portage:portage
    priority: -1000
    sync-rsync-extra-opts: -e 'ssh -i /var/lib/portage/id_ed25519 -p 2222 -o UserKnownHostsFile=/var/lib/portage/known_hosts'

Local-Overlay
    location: /usr/local/portage
    sync-type: git
    sync-uri: https://github.com/garymacindoe/gentoo-overlay.git
    masters: gentoo

crossdev
    location: /usr/local/crossdev
    masters: gentoo

bumblebee
    location: /var/lib/layman/bumblebee
    sync-type: laymansync
    sync-uri: git://github.com/Bumblebee-Project/bumblebee-gentoo
    masters: gentoo
    priority: 50

dotnet
    location: /var/lib/layman/dotnet
    sync-type: laymansync
    sync-uri: git://anongit.gentoo.org/proj/dotnet.git
    masters: gentoo
    priority: 50

java
    location: /var/lib/layman/java
    sync-type: laymansync
    sync-uri: git://anongit.gentoo.org/proj/java.git
    masters: gentoo
    priority: 50

proaudio
    location: /var/lib/layman/proaudio
    sync-type: laymansync
    sync-uri: svn://svn.tuxfamily.org/svnroot/proaudio/proaudio/trunk/overlays/proaudio
    masters: gentoo
    priority: 50

science
    location: /var/lib/layman/science
    sync-type: laymansync
    sync-uri: git://anongit.gentoo.org/proj/sci.git
    masters: gentoo
    priority: 50

Installed sets: @steam
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.3/conf"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs candy clean-logs compress-build-logs config-protect-if-modified distlocks ebuild-locks fail-clean fixlafiles merge-sync multilib-strict news parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="ftp://mirror.bytemark.co.uk/gentoo/ http://mirror.bytemark.co.uk/gentoo/ http://mirror.qubenet.net/mirror/gentoo/ http://www.mirrorservice.org/sites/www.ibiblio.org/gentoo/ ftp://ftp.mirrorservice.org/sites/www.ibiblio.org/gentoo/"
LANG="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j9"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac accessibility acl acpi activities alsa amd64 berkdb bluetooth branding bzip2 cairo cdda cdr cli consolekit cracklib crypt cups cxx dbus declarative dri dts dvd dvdr emboss encode exif fam ffmpeg fftw firefox flac fortran gdbm gif git glamor gpm gtk iconv icu ipv6 java jpeg kde kipi kwallet lapack lcms ldap libnotify lm_sensors mad mng modules mp3 mp4 mpeg mpi mtp multilib mysql ncurses nls nptl ogg opengl openmp pam pango pcre pdf phonon plasma png policykit ppds python qml qt3support qt4 qt5 readline scanner sdl seccomp semantic-desktop session spell sqlite ssl startup-notification subversion svg tcpd threads tiff truetype udev udisks unicode upower usb vaapi vdpau vorbis wayland widgets wxwidgets x264 xattr xcb xcomposite xinerama xml xscreensaver xv xvid zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="evdev synaptics" KERNEL="linux" L10N="en-GB" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en_GB" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_5" PYTHON_TARGETS="python2_7 python3_5 python3_6" QEMU_SOFTMMU_TARGETS="aarch64 arm i386 x86_64" QEMU_USER_TARGETS="aarch64 arm" RUBY_TARGETS="ruby22" USERLAND="GNU" VIDEO_CARDS="intel i965 nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 2 Gary Macindoe 2017-09-28 21:38:32 UTC
Ebuilds that use cdrom.eclass must call cdrom_get_cds passing a file to look for on each CD, e.g. cdrom_get_cds cdrom_get_cds System/Packages.md5 StaticMeshes/AWHardware.usx.uz2 Extras/MayaPLE/Maya4PersonalLearningEditionEpic.exe for 3 CDs.

cdrom_get_cds calls cdrom_load_next_cd and this call is successful in finding disk 1.  Subsequent calls to cdrom_load_next_cd from the ebuild fail to find subsequent disks.

cdrom_get_cds exports an array, CDROM_CHECKS, containing the files to look for on each CD (basically, what it was called with).  It is not possible, in Bash, to export an array to the calling environment as there is no POSIX representation of storing arrays in the environment and the calling shell may not be Bash.  This is why the first call to cdrom_load_next_cd works (as it is invoked from Bash), but subsequent calls from the ebuild fail (as CDROM_CHECKS is no longer defined).

I have fixed this in https://github.com/gentoo/gentoo/pull/5818.
Comment 3 James Le Cuirot gentoo-dev 2017-10-22 20:11:58 UTC
Sorry for the delay, I was without my desktop for a few weeks.

I have managed to reproduce this but while I'm still unsure about the precise cause, I think your reasoning is a little off.

Portage (et al) will always use Bash so we don't have to worry about other shells. The Bash version is significant though. It's guaranteed to be at least 3.2 under EAPI 5, which this package uses.

Unfortunately I have UT2004 but not UT2003 so I wasn't able to try this package but I did reproduce it with games-rpg/comi by moving some of src_unpack into src_install.

Initially it worked fine for me but I noticed that you have Bash 4.3 while I have 4.4. I then pointed /bin/bash to 4.3 and that did the trick.

I thought that the environment file was able to faithfully represent arrays so I compared the file between the two Bash versions. 4.4 includes the following line but it is missing under 4.3.

declare -ax CDROM_CHECKS='([0]="comi.la1" [1]="comi.la2")'

I checked the release notes for 4.4 but couldn't find anything that seemed to relate to this. If I fire up 4.3, declare the array, source Portage's save-ebuild-env.sh script, and call __save_ebuild_env then the above line is actually included. I don't yet know why it's only missing under Portage.
Comment 4 Gary Macindoe 2017-11-04 22:13:32 UTC
> Portage (et al) will always use Bash so we don't have to worry about other
> shells. The Bash version is significant though. It's guaranteed to be at least 
> 3.2 under EAPI 5, which this package uses.

It may be the case that Portage will always use Bash but Bash itself is unaware that it is being used in Portage and cannot assume anything about its calling environment (other than it being POSIX).  This is a bug in the interaction between Portage and Bash - i.e. Portage is expecting Bash to have a feature it doesn't, specifically the ability to export arrays (or save/restore arrays) to the calling environment.

> I checked the release notes for 4.4 but couldn't find anything that seemed to 
> relate to this. If I fire up 4.3, declare the array, source Portage's save-
> ebuild-env.sh script, and call __save_ebuild_env then the above line is 
> actually included. I don't yet know why it's only missing under Portage.

Are these functions part of Portage?  Where can I find them to debug the problem further?

I couldn't find anything in the Bash Changelogs either but did find this post about exporting arrays from Bash on StackOverflow: https://stackoverflow.com/questions/5564418/exporting-an-array-in-bash-script

Two links from answers to that post are interesting: https://www.mail-archive.com/bug-bash@gnu.org/msg01774.html and http://www.openwall.com/lists/oss-security/2014/09/30/1.  Particularly the author states that exporting arrays is incomplete in 4.3 and has no plans to complete that part of the code.  Maybe it has been completed in 4.4 but 4.3 is latest version of Bash available in the stable amd64 arch.

From one of the answers:
"The first thing to understand is that the bash environment (more properly command execution environment) is different to the POSIX concept of an environment. The POSIX environment is a collection of un-typed name=value pairs, and can be passed from a process to its children in various ways (effectively a limited form of IPC)."