I've just switched my dom0 to a hardened profile and (after recompiling toolchain and everything else) ran into the same problem (RPC Protocol Error when attempting to create domUs) as efphe reported on xen-users (http://tinyurl.com/lg7re). Brad's solution (http://tinyurl.com/q5bya) of compiling python and xen-tools with gcc's hardened-nossp profile worked for me. Portage 2.1-r1 (hardened/x86/2.6, gcc-3.4.6, glibc-2.3.6-r4, 2.6.16.18-xen i686) ================================================================= System uname: 2.6.16.18-xen i686 AMD Athlon(tm) Gentoo Base System version 1.6.15 app-admin/eselect-compiler: [Not Present] dev-lang/python: 2.4.3-r1 dev-python/pycrypto: 2.0.1-r5 dev-util/ccache: [Not Present] dev-util/confcache: [Not Present] sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2 sys-devel/binutils: 2.16.1-r3 sys-devel/gcc-config: 1.3.13-r3 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="x86" ACCEPT_LICENSE="" ARCH="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-mtune=athlon-xp -O2 -pipe -mno-tls-direct-seg-refs" CHOST="i686-pc-linux-gnu" CLEAN_DELAY="5" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo" CVS_RSH="ssh" CXXFLAGS="-mtune=athlon-xp -O2 -pipe -mno-tls-direct-seg-refs" DISTDIR="/usr/portage/distfiles" EDITOR="/usr/bin/vim" ELIBC="glibc" EMERGE_DEFAULT_OPTS="--ask --verbose" EMERGE_WARNING_DELAY="10" FEATURES="autoconfig collision-protect distlocks metadata-transfer sandbox sfperms strict test userfetch" FETCHCOMMAND="/usr/bin/wget -t 5 -T 60 --passive-ftp -P ${DISTDIR} ${URI}" GCC_SPECS="" GENTOO_MIRRORS="http://mirror.aarnet.edu.au/pub/gentoo http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo" GRP_STAGE23_USE="x86 x86 berkdb crypt dlloader hardened nls pam pic readline ssl tcpd userlocales zlib" HOME="/home/aross" HOSTNAME="oak" INFOPATH="/usr/share/info:/usr/share/binutils-data/i686-pc-linux-gnu/2.16.1/info:/usr/share/gcc-data/i686-pc-linux-gnu/3.4.6/info" INPUT_DEVICES="mouse keyboard" KERNEL="linux" LESS="-R -M --shift 5" LESSOPEN="|lesspipe.sh %s" LOGNAME="aross" LS_COLORS="no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:su=37;41:sg=30;43:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.qt=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.pdf=00;32:*.ps=00;32:*.txt=00;32:*.patch=00;32:*.diff=00;32:*.log=00;32:*.tex=00;32:*.doc=00;32:*.flac=01;35:*.mp3=01;35:*.mpc=00;36:*.ogg=00;36:*.wav=00;36:*.mid=00;36:*.midi=00;36:*.au=00;36:*.flac=00;36:*.aac=00;36:" MAIL="/var/mail/aross" MANPATH="/usr/local/share/man:/usr/share/man:/usr/share/binutils-data/i686-pc-linux-gnu/2.16.1/man:/usr/share/gcc-data/i686-pc-linux-gnu/3.4.6/man" PAGER="/usr/bin/less" PATH="/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/3.4.6" PKGDIR="/usr/portage//packages/x86/" PORTAGE_ARCHLIST="ppc s390 amd64 ppc64 x86-fbsd m68k arm sparc sh mips ia64 alpha ppc-macos hppa x86" PORTAGE_BINHOST_CHUNKSIZE="3000" PORTAGE_BIN_PATH="/usr/lib/portage/bin" PORTAGE_CALLER="emerge" PORTAGE_CONFIGROOT="/" PORTAGE_ELOG_CLASSES="warn error log" PORTAGE_ELOG_MAILFROM="portage" PORTAGE_ELOG_MAILSUBJECT="[portage] ebuild log for ${PACKAGE} on ${HOST}" PORTAGE_ELOG_MAILURI="root" PORTAGE_GID="250" PORTAGE_INST_GID="0" PORTAGE_INST_UID="0" PORTAGE_NICENESS="5" PORTAGE_PYM_PATH="/usr/lib/portage/pym" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'" PORTAGE_RSYNC_RETRIES="3" PORTAGE_TMPDIR="/var/tmp" PORTAGE_WORKDIR_MODE="0700" PORTDIR="/usr/portage/" PORTDIR_OVERLAY="/usr/local/portage" PRELINK_PATH="" PRELINK_PATH_MASK="" PWD="/home/aross" PYTHONPATH="/usr/lib/portage/pym" RESUMECOMMAND="/usr/bin/wget -c -t 5 -T 60 --passive-ftp -P ${DISTDIR} ${URI}" ROOT="/" RPMDIR="/usr/portage/rpm" SHELL="/bin/bash" SHLVL="1" SSH_AUTH_SOCK="/tmp/ssh-loctl21401/agent.21401" SSH_CLIENT="10.2.0.1 51349 22" SSH_CONNECTION="10.2.0.1 51349 10.2.1.143 22" SSH_TTY="/dev/pts/3" STAGE1_USE="hardened pic userlocales" SYNC="rsync://gentoo.whitley.unimelb.edu.au/gentoo-portage" TERM="xterm-color" USE="acl bash-completion berkdb bzip2 crypt dlloader hardened ncurses nls nptl pam pic python readline ssl symlink sysfs tcpd unicode userlocales x86 xorg zlib elibc_glibc input_devices_mouse input_devices_keyboard kernel_linux userland_GNU" USER="aross" USERLAND="GNU" USE_EXPAND="DVB_CARDS ELIBC FCDSL_CARDS FRITZCAPI_CARDS INPUT_DEVICES KERNEL LINGUAS LIRC_DEVICES USERLAND VIDEO_CARDS" USE_EXPAND_HIDDEN="ELIBC KERNEL USERLAND" USE_ORDER="env:pkg:conf:defaults" XARGS="xargs -r" _="/usr/bin/emerge"
Any idea how this can be fixed in the ebuild?
I won't be in a position to test this for a few days, but using "append-flags -fno-stack-protector" should have the same benefit as using gcc's hardened-nossp profile. However, we can't force this in the python ebuild - perhaps pkg_setup can check if python was built with ssp until a real fix can be found.
Thanks to solar on IRC for suggesting "scanelf -s __guard -q `which python`" to check if python was built using ssp.
Fixed in xen-tools-3.0.2-r3
I have been using hardened profile for a long time without this issue. I really don't like this "workaround" (reemerge python with -fno-stack-protector) . I'll reemerge my system with "-mno-tls-direct-seg-refs" to see if I can reproduce this bug.
I also dont like this "workaround". I solved this problem by 'custom-cflags' USE option. My cflags in make.conf: CFLAGS="-pipe -march=prescott -Os -march=prescott -fomit-frame-pointer -frename-registers -fweb -freorder-blocks -mno-tls-direct-seg-refs"
USE=custom-cflags works for me as well - thanks, Raimonds. Brad, I take it you're not using custom-cflags?
(In reply to comment #7) > USE=custom-cflags works for me as well - thanks, Raimonds. > > Brad, I take it you're not using custom-cflags? > No, I wasn't using the custon-cflags USE flag. I just recompiled python and xen-tools using the hardened compiler and I seem to be able to create domains without any problems. I'm not aware of why this would have started working suddenly. Python was definitely compiled with stack smashing: xen2 ~ # scanelf -s __guard /usr/bin/python TYPE SYM FILE ET_DYN __guard /usr/bin/python And I definitely compiled xen-tools without the custom-cflags USE flag: xen2 ~ # emerge -pv xen-tools These are the packages that I would merge, in order: Calculating dependencies ...done! [ebuild R ] app-emulation/xen-tools-3.0.2-r3 -custom-cflags -debug -doc +hardened +screen +sdl +vnc 0 kB I was able to successfully create guest domains with both xen-tools 3.0.2-r1 (what I was originally using) and with 3.0.2-r3 which I only updated to today. I did however have to comment out the section that checked to see if python was compiled with stack smashing. Cheers, Brad
On hardened amd64 xend is also running fine with ssp: xen1 ~ # scanelf -s __guard /usr/bin/python TYPE SYM FILE ET_DYN __guard /usr/bin/python
I did some tests today, and just to summarize current status on hardened x86: - the RPC Protocol Error is caused by python being compiled in the 'normal way', eg. with hardened compiler, usual CFLAGS. - recompiling python-2.4.3-r4 with i386-pc-linux-gnu-3.4.6-hardenednossp does solve the problem (no need to use vanilla, nossp is ok) - recompiling xen-tools with other (nossp, vanilla, anything) compilers or CFLAGS does not solve the problem. If python is compiled without ssp, xen-tools will work ok even if compiled with the usual hardened compiler. - don't know about xen-tools-3.0.2-r3 (see <a href="#c4">comment #4</a>), but xen-tools-3.0.2-r4 does have this problem From my make.conf: CFLAGS="-O2 -march=i686 -pipe -mno-tls-direct-seg-refs" USE="userlocales sysfs screen nptl ncurses snmp" Ebuilds used: python-2.4.3-r4 xen-sources-2.6.16.28 xen-tools-3.0.2-r4 xen-3.0.2 Everything compiled with hardened compiler except python, this way the system works fine with hardened gentoo linux domU's (still looking at why Yuan Jue's freebsd domU does not work, but that's another story). Hope this helps.
Is there any progress on this problem? Just to confuse things a bit, I have the same problem as described here, but without using hardened.
I can report that this problem disappeared using xen-tools-3.0.4_p1, xen-3.0.4_p1 on hardened/amd64 using default (hardened PIE+SSP) x86_64-pc-linux-gnu-3.4.6.
Xen 3.0.2 has been dropped from portage and both 3.0.4 and 3.1.0 seem to be working fine on amd64 and x86 hardened. Closing bug, if anyone still hits this problem using >=3.0.4 please re-open.
