dev-libs/crypto++-5.4-r1 breaks on non x86/amd64 architectures since it's forcing the -mno-sse2 CFLAG. gcc4 seems to handle this graciously on non-x86/amd64 archs, but gcc3 does not. Even if gcc4 does this behaviour could change since it's really invalid.
OK, removed the sse stuff, the package autodetect it from compiler settings. Thanks.