Bugzilla Bug 141866

I've just switched my dom0 to a hardened profile and (after recompiling
toolchain and everything else) ran into the same problem (RPC Protocol Error
when attempting to create domUs) as efphe reported on xen-users
(http://tinyurl.com/lg7re).

Brad's solution (http://tinyurl.com/q5bya) of compiling python and xen-tools
with gcc's hardened-nossp profile worked for me.

Portage 2.1-r1 (hardened/x86/2.6, gcc-3.4.6, glibc-2.3.6-r4, 2.6.16.18-xen
i686)
=================================================================
System uname: 2.6.16.18-xen i686 AMD Athlon(tm)
Gentoo Base System version 1.6.15
app-admin/eselect-compiler: [Not Present]
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.13-r3
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE=""
ARCH="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-mtune=athlon-xp -O2 -pipe -mno-tls-direct-seg-refs"
CHOST="i686-pc-linux-gnu"
CLEAN_DELAY="5"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo"
CVS_RSH="ssh"
CXXFLAGS="-mtune=athlon-xp -O2 -pipe -mno-tls-direct-seg-refs"
DISTDIR="/usr/portage/distfiles"
EDITOR="/usr/bin/vim"
ELIBC="glibc"
EMERGE_DEFAULT_OPTS="--ask --verbose"
EMERGE_WARNING_DELAY="10"
FEATURES="autoconfig collision-protect distlocks metadata-transfer sandbox
sfperms strict test userfetch"
FETCHCOMMAND="/usr/bin/wget -t 5 -T 60 --passive-ftp -P ${DISTDIR} ${URI}"
GCC_SPECS=""
GENTOO_MIRRORS="http://mirror.aarnet.edu.au/pub/gentoo
http://distfiles.gentoo.org
http://www.ibiblio.org/pub/Linux/distributions/gentoo"
GRP_STAGE23_USE="x86 x86 berkdb crypt dlloader hardened nls pam pic readline
ssl tcpd userlocales zlib"
HOME="/home/aross"
HOSTNAME="oak"
INFOPATH="/usr/share/info:/usr/share/binutils-data/i686-pc-linux-gnu/2.16.1/info:/usr/share/gcc-data/i686-pc-linux-gnu/3.4.6/info"
INPUT_DEVICES="mouse keyboard"
KERNEL="linux"
LESS="-R -M --shift 5"
LESSOPEN="|lesspipe.sh %s"
LOGNAME="aross"
LS_COLORS="no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:su=37;41:sg=30;43:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.qt=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.pdf=00;32:*.ps=00;32:*.txt=00;32:*.patch=00;32:*.diff=00;32:*.log=00;32:*.tex=00;32:*.doc=00;32:*.flac=01;35:*.mp3=01;35:*.mpc=00;36:*.ogg=00;36:*.wav=00;36:*.mid=00;36:*.midi=00;36:*.au=00;36:*.flac=00;36:*.aac=00;36:"
MAIL="/var/mail/aross"
MANPATH="/usr/local/share/man:/usr/share/man:/usr/share/binutils-data/i686-pc-linux-gnu/2.16.1/man:/usr/share/gcc-data/i686-pc-linux-gnu/3.4.6/man"
PAGER="/usr/bin/less"
PATH="/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/3.4.6"
PKGDIR="/usr/portage//packages/x86/"
PORTAGE_ARCHLIST="ppc s390 amd64 ppc64 x86-fbsd m68k arm sparc sh mips ia64
alpha ppc-macos hppa x86"
PORTAGE_BINHOST_CHUNKSIZE="3000"
PORTAGE_BIN_PATH="/usr/lib/portage/bin"
PORTAGE_CALLER="emerge"
PORTAGE_CONFIGROOT="/"
PORTAGE_ELOG_CLASSES="warn error log"
PORTAGE_ELOG_MAILFROM="portage"
PORTAGE_ELOG_MAILSUBJECT="[portage] ebuild log for ${PACKAGE} on ${HOST}"
PORTAGE_ELOG_MAILURI="root"
PORTAGE_GID="250"
PORTAGE_INST_GID="0"
PORTAGE_INST_UID="0"
PORTAGE_NICENESS="5"
PORTAGE_PYM_PATH="/usr/lib/portage/pym"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_RSYNC_RETRIES="3"
PORTAGE_TMPDIR="/var/tmp"
PORTAGE_WORKDIR_MODE="0700"
PORTDIR="/usr/portage/"
PORTDIR_OVERLAY="/usr/local/portage"
PRELINK_PATH=""
PRELINK_PATH_MASK=""
PWD="/home/aross"
PYTHONPATH="/usr/lib/portage/pym"
RESUMECOMMAND="/usr/bin/wget -c -t 5 -T 60 --passive-ftp -P ${DISTDIR} ${URI}"
ROOT="/"
RPMDIR="/usr/portage/rpm"
SHELL="/bin/bash"
SHLVL="1"
SSH_AUTH_SOCK="/tmp/ssh-loctl21401/agent.21401"
SSH_CLIENT="10.2.0.1 51349 22"
SSH_CONNECTION="10.2.0.1 51349 10.2.1.143 22"
SSH_TTY="/dev/pts/3"
STAGE1_USE="hardened pic userlocales"
SYNC="rsync://gentoo.whitley.unimelb.edu.au/gentoo-portage"
TERM="xterm-color"
USE="acl bash-completion berkdb bzip2 crypt dlloader hardened ncurses nls nptl
pam pic python readline ssl symlink sysfs tcpd unicode userlocales x86 xorg
zlib elibc_glibc input_devices_mouse input_devices_keyboard kernel_linux
userland_GNU"
USER="aross"
USERLAND="GNU"
USE_EXPAND="DVB_CARDS ELIBC FCDSL_CARDS FRITZCAPI_CARDS INPUT_DEVICES KERNEL
LINGUAS LIRC_DEVICES USERLAND VIDEO_CARDS"
USE_EXPAND_HIDDEN="ELIBC KERNEL USERLAND"
USE_ORDER="env:pkg:conf:defaults"
XARGS="xargs -r"
_="/usr/bin/emerge"

Any idea how this can be fixed in the ebuild?

I won't be in a position to test this for a few days, but using "append-flags
-fno-stack-protector" should have the same benefit as using gcc's
hardened-nossp profile.

However, we can't force this in the python ebuild - perhaps pkg_setup can check
if python was built with ssp until a real fix can be found.

Thanks to solar on IRC for suggesting "scanelf -s __guard -q `which python`" to
check if python was built using ssp.

Fixed in xen-tools-3.0.2-r3

I have been using hardened profile for a long time without this issue. I really
don't like this "workaround" (reemerge python with -fno-stack-protector) . I'll
reemerge my system with "-mno-tls-direct-seg-refs" to see if I can reproduce
this bug.

I also dont like this "workaround".
I solved this problem by 'custom-cflags' USE option.
My cflags in make.conf:
CFLAGS="-pipe -march=prescott -Os -march=prescott -fomit-frame-pointer
-frename-registers -fweb -freorder-blocks
-mno-tls-direct-seg-refs"

USE=custom-cflags works for me as well - thanks, Raimonds.

Brad, I take it you're not using custom-cflags?

(In reply to comment #7)
> USE=custom-cflags works for me as well - thanks, Raimonds.
> 
> Brad, I take it you're not using custom-cflags?
> 

No, I wasn't using the custon-cflags USE flag. I just recompiled python and
xen-tools using the hardened compiler and I seem to be able to create domains
without any problems. I'm not aware of why this would have started working
suddenly.

Python was definitely compiled with stack smashing:

xen2 ~ # scanelf -s __guard /usr/bin/python
 TYPE   SYM FILE
ET_DYN __guard /usr/bin/python

And I definitely compiled xen-tools without the custom-cflags USE flag:

xen2 ~ # emerge -pv xen-tools

These are the packages that I would merge, in order:

Calculating dependencies ...done!
[ebuild   R   ] app-emulation/xen-tools-3.0.2-r3  -custom-cflags -debug -doc
+hardened +screen +sdl +vnc 0 kB

I was able to successfully create guest domains with both xen-tools 3.0.2-r1
(what I was originally using) and with 3.0.2-r3 which I only updated to today.
I did however have to comment out the section that checked to see if python was
compiled with stack smashing.

Cheers,

Brad

On hardened amd64 xend is also running fine with ssp:

xen1 ~ # scanelf -s __guard /usr/bin/python
 TYPE   SYM FILE
ET_DYN __guard /usr/bin/python

I did some tests today, and just to summarize current status on hardened x86:

- the RPC Protocol Error is caused by python being compiled in the 'normal
way', eg. with hardened compiler, usual CFLAGS.
- recompiling python-2.4.3-r4 with i386-pc-linux-gnu-3.4.6-hardenednossp does
solve the problem (no need to use vanilla, nossp is ok)
- recompiling xen-tools with other (nossp, vanilla, anything) compilers or
CFLAGS does not solve the problem. If python is compiled without ssp, xen-tools
 will work ok even if compiled with the usual hardened compiler.
- don't know about xen-tools-3.0.2-r3 (see <a href="#c4">comment #4</a>), but
xen-tools-3.0.2-r4 does have this problem

From my make.conf:
CFLAGS="-O2 -march=i686 -pipe -mno-tls-direct-seg-refs"
USE="userlocales sysfs screen nptl ncurses snmp"

Ebuilds used:
python-2.4.3-r4
xen-sources-2.6.16.28
xen-tools-3.0.2-r4
xen-3.0.2

Everything compiled with hardened compiler except python, this way the system
works fine with hardened gentoo linux domU's (still looking at why Yuan Jue's
freebsd domU does not work, but that's another story).

Hope this helps.

Is there any progress on this problem?

Just to confuse things a bit, I have the same problem as described here, but
without using hardened.

I can report that this problem disappeared using xen-tools-3.0.4_p1,
xen-3.0.4_p1 on hardened/amd64 using default (hardened PIE+SSP)
x86_64-pc-linux-gnu-3.4.6.

Xen 3.0.2 has been dropped from portage and both 3.0.4 and 3.1.0 seem to be
working fine on amd64 and x86 hardened. Closing bug, if anyone still hits this
problem using >=3.0.4 please re-open.