Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 568912 - sys-kernel/mips-sources: IP30/SMP support might be broken
Summary: sys-kernel/mips-sources: IP30/SMP support might be broken
Status: RESOLVED WORKSFORME
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Unspecified (show other bugs)
Hardware: MIPS Linux
: Normal normal (vote)
Assignee: MIPS Porters
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-12-20 23:11 UTC by Carlo
Modified: 2017-01-05 23:27 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
kernel config (ip30-config.txt,46.07 KB, text/plain)
2015-12-20 23:11 UTC, Carlo
Details
4.3.3-mipsgit-20151126 IP30 .config (ip30-433.config,54.50 KB, text/plain)
2015-12-26 22:07 UTC, Joshua Kinard
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Carlo 2015-12-20 23:11:42 UTC
Created attachment 419950 [details]
kernel config

hi
I am experiencing troubles with linux kernel v4.3.0 rev2015112 and 4.2.6-20151126 on Octane/IP30. I enabled the flag "IP30" and then I emerged them from the portage. 

Disabling SMP everything is fine, while rnabling the SMP randomly causes "Instruction bus error" on CPU1. "randomly" means that the same kernel executed on the same machine sometimes has troubles, while sometimes it works properly, with an uptime of 2 days without issues!


I am using a mips-3-be-uclibc ram-rootfs , successfully tested with the kernel 2.6.17-SMP

the final size, kernel+ramrootfs, is ~12Mbyte


>> hinv
                   System: IP30
                Processor: 400 Mhz R12000, 2M secondary cache
                Processor: 400 Mhz R12000, 2M secondary cache, (cpu 1)
              Memory size: 1792 Mbytes
                  Network: ef0 ethernet (100/10 base-T)
                    Audio: RAD Audio Processor
                  PCI Bus: XTALKPCI (13)
               PCI Device: slot 1 vendor 0x1033 part 0x35 rev 67


in order to compile the IP30 kernel, I am using kgcc v4.9.3

 [5] mips64-unknown-linux-gnu-4.3.5
 [6] mips64-unknown-linux-gnu-4.9.3 *

attached my .config

let me know if you need additional information
my email is carlojpisani at gmail dot com

Carlo

------
Comment 1 Carlo 2015-12-21 19:21:31 UTC
Support initial ramdisks compressed using bzip2

I have noted that using bzip2 does not work at all, the machine loads the boot file from the lan, but something bad happens and it resets into the monitor

>> bootp():
Setting $netaddr to 192.168.1.5 (from server )
Obtaining  from server 
9123968+228256 entry: 0xa8000000203b9790



                         Running power-on diagnostics...




while gzip2 works
Comment 2 Carlo 2015-12-22 08:59:31 UTC
found a good article(1) which summaries the IP30 context of IP30_SMP with linux

~ ~ ~

yesterday I wanted to recompile the kernel removing the ramrootfs, so, the kernel is now forced to mount the root onto an external SCSI hard drive which contains the same rootfs I used in the previous experiment

I have copied the ramrootfs contents into the hard drive, and it seems that "Instruction bus error" does never happen

I wonder if there is something wrong with the ramfs support, decompressor, …

(1) http://stackoverflow.com/questions/27099008/implementing-smp-properly-on-a-linux-mips-platform
Comment 3 Joshua Kinard gentoo-dev 2015-12-23 06:47:23 UTC
I haven't forgotten about you.  The problem is likely your ramrootfs is just too old to play nicely with the new kernel.  I'm running SMP fine on my Octane, but that machine has been tied up trying to compile new uClibc-based root filesystems to create new netboots from.

Try to unpack one of these stage3 tarballs from here:
http://distfiles.gentoo.org/experimental/mips/stages/mips4/2015/

- stage3-mips4_r12k_n32-20150205.tar.bz2: N32 rootfs (-march=r12000)
- stage3-mips4-20150205.tar.bz2 O32 rootfs (-march=mips4)

Do so under non-SMP, then switch to SMP and try to chroot in from your ramrootfs and see if the IBE's can still be reliably triggered.  Make sure you trim your ramrootfs down to the bare minimum needed to get networking up and chroot into the stage3.  If the IBE's can no longer be produced, then it suggests that your ramrootfs is the source of the problems.  If they can be produced, then something else is not right, and I'll have to re-evaluate it after the holidays over here in the US.

Hopefully, I'll coax this uclibc stage4 into a reliable seed stage to get a new install netboot working, and that will be able to install new stageballs I hope to create once the netboot works.  But I've only got one working/reliable SGI machine right now, so it takes time to build everything.
Comment 4 Carlo 2015-12-24 20:14:08 UTC
can you check my .config, please ?
can you send me your .config ?

the one with SMP!
Comment 5 Carlo 2015-12-24 20:22:27 UTC
can't you send me your kernel ? the binary !

if we work in parallel the solution will needs less time
currently here kernel 4.2 is NOT working at all!
Comment 6 Carlo 2015-12-24 20:28:29 UTC
e.g. python is not working properly over 4.2 and 4.3
mips4-stage3

root@ip30-octane2-2xR12K / # emerge --sync  
Traceback (most recent call last):
  File "/usr/lib/python-exec/python2.7/emerge", line 44, in <module>
    from _emerge.main import emerge_main
  File "/usr/lib/python2.7/site-packages/_emerge/main.py", line 23, in <module>
    from portage.sync import _SUBMODULE_PATH_MAP
  File "/usr/lib/python2.7/site-packages/portage/sync/__init__.py", line 8, in <module>
    from portage.sync.controller import SyncManager
  File "/usr/lib/python2.7/site-packages/portage/sync/controller.py", line 22, in <module>
    from portage.package.ebuild.doebuild import _check_temp_dir
  File "/usr/lib/python2.7/site-packages/portage/package/ebuild/doebuild.py", line 51, in <module>
    from portage.dbapi.porttree import _parse_uri_map
  File "/usr/lib/python2.7/site-packages/portage/dbapi/porttree.py", line 88, in <module>
    portage.process.atexit_register(close_portdbapi_caches)
  File "/usr/lib/python2.7/site-packages/portage/proxy/objectproxy.py", line 22, in __getattribute__
    result = object.__getattribute__(self, '_get_target')()
  File "/usr/lib/python2.7/site-packages/portage/proxy/lazyimport.py", line 107, in _get_target
    __import__(name)
  File "/usr/lib/python2.7/site-packages/portage/process.py", line 31, in <module>
    max_fd_limit = resource.getrlimit(resource.RLIMIT_NOFILE)[0]
resource.error: (-89, 'Unknown error -89')
root@ip30-octane2-2xR12K / #
Comment 7 Carlo 2015-12-24 21:46:38 UTC
I saved my old stage4, 2008,2009, it's MIPS4-BE-GLIBC
I have resurrected it from an old DVD, and installed into the hard drive

emerge and other things are perfectly working (as expected)

I am a bit confused about what is happening here!

# mount -t ext3 /dev/sda1 /mnt/disk0
# chroot /mnt/disk0/gentoo-2008
root@ip30-octane2-2xR12K / # ls
bin         lib    root                                  tmp
boot        mnt    sbin                                  usr
cavia-here  my     source.local--updated-to-21-05-2008   var
dev         opt    src
etc         proc   stage4--updated-to-2009-10-20--16-13
home        proof  sys
root@ip30-octane2-2xR12K / # gcc -v
Using built-in specs.
Target: mips-unknown-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-4.1.2/work/gcc-4.1.2/configure --prefix=/usr --bindir=/usr/mips-unknown-linux-gnu/gcc-bin/4.1.2 --includedir=/usr/lib/gcc/mips-un
known-linux-gnu/4.1.2/include --datadir=/usr/share/gcc-data/mips-unknown-linux-gnu/4.1.2 --mandir=/usr/share/gcc-data/mips-unknown-linux-gnu/4.1.2/man --infodir=/usr/share/gcc-d
ata/mips-unknown-linux-gnu/4.1.2/info --with-gxx-include-dir=/usr/lib/gcc/mips-unknown-linux-gnu/4.1.2/include/g++-v4 --host=mips-unknown-linux-gnu --build=mips-unknown-linux-gn
u --disable-altivec --enable-nls --without-included-gettext --with-system-zlib --disable-checking --disable-werror --enable-secureplt --disable-multilib --enable-libmudflap --di
sable-libssp --disable-libgcj --enable-languages=c,c++,treelang,fortran --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu
Thread model: posix
gcc version 4.1.2 (Gentoo 4.1.2 p1.1)
root@ip30-octane2-2xR12K / # gcc-config -l
 [1] mips-unknown-linux-gnu-3.4.6
 [2] mips-unknown-linux-gnu-4.1.2 *
 [3] mips64-unknown-linux-gnu-3.4.6
 [4] mips64-unknown-linux-gnu-4.1.2 *
root@ip30-octane2-2xR12K / # emerge --search bash
Searching...   
[ Results for search key : bash ]
[ Applications found : 11 ]
 
*  app-shells/bash
      Latest version available: 4.0_p28
      Latest version installed: 4.0_p28
      Size of files: 6,154 kB
      Homepage:      http://cnswww.cns.cwru.edu/~chet/bash/bashtop.html
      Description:   The standard GNU Bourne again shell
      License:       GPL-3



I am trying to rebuild the kernel from x86, by a fresh stage3 & crossdev!
Comment 8 Carlo 2015-12-25 13:34:38 UTC
kernel 2.6.17 over rootfs-2008: no issues!
mips / # emerge --search bash

*  app-shells/bash
      Latest version available: 4.0_p28
      Latest version installed: 4.0_p28
      Size of files: 6,154 kB
      Homepage:      http://cnswww.cns.cwru.edu/~chet/bash/bashtop.html
      Description:   The standard GNU Bourne again shell
      License:       GPL-3

kenrel 4.2.6 over rootfs-2008: strange behavior!

mips / # emerge --search bash
/usr/bin/emerge: line 6: import: command not found
/usr/bin/emerge: line 8: try:: command not found
/usr/bin/emerge: line 9: import: command not found
/usr/bin/emerge: line 11: syntax error near unexpected token `('
/usr/bin/emerge: line 11: `     def exithandler(signum,frame):'
Comment 9 Carlo 2015-12-25 14:49:08 UTC
mips /stage-2015 # tar -xjf stage3-mips4_r12k_n32-20150205.tar.bz2

mips /stage-2015 # ls
bin   etc   lib32  net.sh  root     sbin                                    tmp
boot  home  media  opt     rules.d  stage3-mips4_r12k_n32-20150205.tar.bz2  usr
dev   lib   mnt    proc    run      sys   

mips / # cd /
mips / # uname -r
4.2.6-SMP-ip30

mips / # chroot stage-2015/ /bin/bash
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
Instruction bus error, epc == 0000000010025a80, ra == 0000000010025e7c

mips / # ls
bin   etc   lib32  net.sh  root     sbin                                    tmp
boot  home  media  opt     rules.d  stage3-mips4_r12k_n32-20150205.tar.bz2  usr
dev   lib   mnt    proc    run      sys                                     var

mips / # emerge --info
/usr/bin/emerge: line 11: from: command not found
/usr/bin/emerge: line 12: import: command not found
/usr/bin/emerge: line 14: try:: command not found
/usr/bin/emerge: line 15: from: command not found
/usr/bin/emerge: line 16: except: command not found
/usr/bin/emerge: line 17: syntax error near unexpected token `('
/usr/bin/emerge: line 17: `     EPYTHON = os.path.basename(sys.executable)'
mips / # 




I give up, my time is over
Comment 10 Joshua Kinard gentoo-dev 2015-12-26 22:07:49 UTC
Created attachment 420914 [details]
4.3.3-mipsgit-20151126 IP30 .config

Try this .config with the 4.3.3 ebuild I added to the tree a day or so ago.  It's virtually the same one currently running on my Octane.  I only dropped IPX & NCP support, changed the RTC driver to a DS1687-5 (I use a DS17887-5), and added EXT4 support back in (I use XFS).  This build a monolithic kernel only, no modules.  This also builds a PAGE_SIZE=64KB kernel, so you'll need to run 'swapon --fix-pgsz <swapdev>' to get working swap.

Try to minimize making any changes to it.  Obviously, add your ramrootfs into it, but leave as much else alone and let me know if the IBE's still occur or not.  If you still get them, then I will have to assume there's something really weird about your userland, and you'll probably just have to wait until I can get a semi-working uclibc netboot put together.

I can't rule out that really old userlands simply cause problems.  When I resurrected my machine, it had a ~2009 userland still on it, but I wasn't using SMP at the time, so it wasn't too difficult to get it upgraded.  By the time I got SMP figured back out, the userland was fairly current, so this IBE problem may have not been triggered.
Comment 11 Carlo 2015-12-26 23:10:33 UTC
> Try to 

why don't you send me a working kernel ???

> ~2009 userland

stage3-mips4_r12k_n32-20150205.tar.bz2 fails with the last kernel I have compiled !
Comment 12 Carlo 2015-12-26 23:30:12 UTC
I suspect the problem is with the kgcc I have emerged on a native mip32-r2 system!

I guess my 2009 stage3 is fine, the kernel is NOT working as expected, it may be because of the compiler!

kika / # cat /proc/cpuinfo | grep "cpu model"
cpu model		: MIPS 24Kc V7.4

kika / # gcc-config -l
 [1] mips-unknown-linux-gnu-4.1.2 
 [2] mips-unknown-linux-gnu-4.5.2
 [3] mips-unknown-linux-gnu-4.6.3
 [4] mips-unknown-linux-gnu-4.9.3 *

 [5] mips64-unknown-linux-gnu-4.3.5
 [6] mips64-unknown-linux-gnu-4.9.3 *

kika / # file /usr/mips-unknown-linux-gnu/mips64-unknown-linux-gnu/gcc-bin/4.9.3/mips64-unknown-linux-gnu-gcc
/usr/mips-unknown-linux-gnu/mips64-unknown-linux-gnu/gcc-bin/4.9.3/mips64-unknown-linux-gnu-gcc: ELF 32-bit MSB executable, MIPS, MIPS-I version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.16, with unknown capability 0x41000000 = 0xf676e75, with unknown capability 0x10000 = 0x70401, stripped

~ ~ ~ ~ ~ ~ ~ 

I can't emerge kgcc on my IP30 because I have a dated 2009 stage4 and I can't sync the portage! Emerge is not compatible with the new SLOTS &C

I can't use 4.* kernel because they are not working (emerge does not work, and I got a lot of strange errors and bad behaviors), and my old 2.6.17 can't be used with a 2015 stage3, it says "too old kernel", so I am stopped !


If you send me (carlojpisani at gmail dot com) a working-kernel, with SMP and ext3 support, I can boot it with root=/dev/sdc3 and try my 2009-rootfs (already copied into the hard drive)

/dev/sdc3: contains my 2009 uclibc rootfs
/dev/sdc5: contains stage3-mips4_r12k_n32-20150205

Which compiler version have you used?

Unfortunately here crossdev -v --target mips64-unknown-linux-gnu is not working on my x86 box, even if its stage3 is fresh brand new

I have followed an online manual but I still get the following 

checking for mips64-unknown-linux-gnu-xlC... no
checking for g++... g++
configure: WARNING: using cross tools not prefixed with host triplet
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking whether g++ can link programs... yes
checking for sysdeps preconfigure fragments... aarch64 alpha arm hppa i386 m68k microblaze mips configure: error: Unable to determine ABI.
 * ERROR: cross-mips64-unknown-linux-gnu/glibc-9999::gentoo-b3 failed (configure phase):
 *   failed to configure glibc
 * 
 * Call stack:
 *             ebuild.sh, line   93:  Called src_configure
 *           environment, line 3539:  Called eblit-run 'src_configure'
 *           environment, line  949:  Called eblit-glibc-src_configure
 *   src_configure.eblit, line  258:  Called foreach_abi 'toolchain-glibc_src_configure'
 *          common.eblit, line  343:  Called toolchain-glibc_src_configure
 *   src_configure.eblit, line  250:  Called toolchain-glibc_headers_configure
 *   src_configure.eblit, line  245:  Called die
 * The specific snippet of code:
 *   	CC="$(tc-getBUILD_CC)" \
 *   	CFLAGS="-O1 -pipe" \
 *   	CPPFLAGS="-U_FORTIFY_SOURCE" \
 *   	LDFLAGS="" \
 *   	"$@" || die "failed to configure glibc"
 * 

so I do not have any other changes to compile the kernel.
Comment 13 Carlo 2015-12-27 00:20:08 UTC
kika, the MIPS32R2 machine I used to the compile the IP30 kernel has the following

  sys-devel/binutils
      Latest version available: 2.25.1-r1
      Latest version installed: 2.23.1

I am going to update binutils
Comment 14 Joshua Kinard gentoo-dev 2015-12-27 00:38:25 UTC
(In reply to Carlo from comment #11)
> > Try to 
> 
> why don't you send me a working kernel ???
> 

One step at a time.  If I send you the currently-running kernel on my Octane, then it limits what you're able to test with it.  I've got that machine in the middle of recompiling gcc-5.3 inside of a mips3-uclibc chroot right now, and it probably won't finish for another 10+ hours, nor am I going to interrupt it to reboot and test slightly different .config parameters.

So by attaching the .config from that machine with the few minor changes I made (because I doubt you use Novell IPX, you need ext4/3/2, and you're very likely still on the original RTC chip), that lets you build it using your local compiler.  If you still have problems, it's probably going to eliminate the kernel as the cause (for now).

Which leaves either the compiler as your problem, or your ramrootfs...


(In reply to Carlo from comment #12)
> I suspect the problem is with the kgcc I have emerged on a native mip32-r2
> system!
> 
> I guess my 2009 stage3 is fine, the kernel is NOT working as expected, it
> may be because of the compiler!
> 
> kika / # cat /proc/cpuinfo | grep "cpu model"
> cpu model		: MIPS 24Kc V7.4
> 
> kika / # gcc-config -l
>  [1] mips-unknown-linux-gnu-4.1.2 
>  [2] mips-unknown-linux-gnu-4.5.2
>  [3] mips-unknown-linux-gnu-4.6.3
>  [4] mips-unknown-linux-gnu-4.9.3 *
> 
>  [5] mips64-unknown-linux-gnu-4.3.5
>  [6] mips64-unknown-linux-gnu-4.9.3 *
> 
> kika / # file
> /usr/mips-unknown-linux-gnu/mips64-unknown-linux-gnu/gcc-bin/4.9.3/mips64-
> unknown-linux-gnu-gcc
> /usr/mips-unknown-linux-gnu/mips64-unknown-linux-gnu/gcc-bin/4.9.3/mips64-
> unknown-linux-gnu-gcc: ELF 32-bit MSB executable, MIPS, MIPS-I version 1
> (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.16, with
> unknown capability 0x41000000 = 0xf676e75, with unknown capability 0x10000 =
> 0x70401, stripped

If you're running a 32bit compiler, but still somehow built a 64bit kernel binary...then I don't know.  I know binutils can install one time and output stuff for multiple architectures and ISAs, but I didn't think gcc was there just yet.

That said, 4.9.3 should work fine.  I've built & booted several IP30 and IP27 kernels using it, before I switched to 5.x for kernel compiles.


> I can't emerge kgcc on my IP30 because I have a dated 2009 stage4 and I
> can't sync the portage! Emerge is not compatible with the new SLOTS &C
> 
> I can't use 4.* kernel because they are not working (emerge does not work,
> and I got a lot of strange errors and bad behaviors), and my old 2.6.17
> can't be used with a 2015 stage3, it says "too old kernel", so I am stopped !

The "too old kernel" is due to current glibc requiring a minimium of 2.6.32 kernel headers.  This happened within the last year (I forget when -- a news item was issued, but you probably can't access it if the IBE's are interrupting 'eselect news').


> If you send me (carlojpisani at gmail dot com) a working-kernel, with SMP
> and ext3 support, I can boot it with root=/dev/sdc3 and try my 2009-rootfs
> (already copied into the hard drive)
> 
> /dev/sdc3: contains my 2009 uclibc rootfs
> /dev/sdc5: contains stage3-mips4_r12k_n32-20150205

Let's save this idea for last.  I want to try to figure out what's causing your problems locally, first.  The fact that you're getting IBE's on SMP only is a sign of either odd hardware problems or something with the compiler setup you have.

Do you know what your machine's HEART revision is?  Booting that 4.3.x kernel you built, it should be in dmesg, near the top.  It'll look like this:

[    0.543236] xtalk:0 xbow widget (rev 2.0) registered as a platform device.
[    0.543416] xtalk:8 heart widget (rev F) registered as a platform device.
[    0.543597] xtalk:12 impact widget (rev B) registered as a platform device.
[    0.543775] xtalk:13 bridge widget (rev C) registered as a platform device.
[    0.543954] xtalk:15 bridge widget (rev D) registered as a platform device.

Rev F is the latest/final HEART rev, as far as I know.  I did boot my test SMP kernels on an older system board that had a Rev D HEART, I think, so I don't think this is the problem, but I didn't test that instance for very long.  I also can't test older spins of the IP30 frontplane, which had XBOW 1.3 and earlier.  My current frontplane is XBOW 1.4 (self-reports as 2.0).


> Which compiler version have you used?

# gcc-config -l
 [1] mips64-unknown-linux-gnu-5.3.0 *
 [2] mips64-unknown-linux-gnu-5.3.0-hardenednopie
 [3] mips64-unknown-linux-gnu-5.3.0-hardenednopiessp
 [4] mips64-unknown-linux-gnu-5.3.0-hardenednossp
 [5] mips64-unknown-linux-gnu-5.3.0-vanilla

5.3.0, currently, as a cross-compiler on my x64_64 box.


> Unfortunately here crossdev -v --target mips64-unknown-linux-gnu is not
> working on my x86 box, even if its stage3 is fresh brand new

crossdev on x86_64 should work perfectly fine, so that means your existing cross-compiler install is screwed up, and you need to blow it away and start over.

Run these commands in the following order:

# First, build a stage2 mips64 cross-compiler (this is minimum for kernel builds):

crossdev --b 2.25.1-r1 --g 5.3.0 --l 2.22-r1 --k 4.3 -s2 -A n32 -t mips64-unknown-linux-gnu --without-headers -v

# Second, build mips64 cross-glibc separately, making sure to override several compiler flags because of an odd bug in glibc's build system:

ASFLAGS="-march=mips4 -mtune=mips4" LDFLAGS="-Wl,-z,now -Wl,-z,relro" CFLAGS="-O2 -pipe -march=mips4 -mtune=mips4 -mplt" CPPFLAGS="-O2 -pipe -march=mips4 -mtune=mips4 -mplt" crossdev --b 2.25.1-r1 --g 5.3.0 --l 2.22-r1 --k 4.3 -s3 -A n32 -t mips64-unknown-linux-gnu --without-headers -v

# Third, build a full mips64 cross-compiler toolchain with GDB, excluding the sanitizer (because of another bug, but it might be fixed by now):

USE="-sanitize" crossdev --b 2.25.1-r1 --g 5.3.0 --l 2.22-r1 --k 4.3 -s4 -A n32 --ex-gdb -t mips64-unknown-linux-gnu --without-headers -v

I run these exact commands every time I rebuild my mips64 cross-toolchain (making sure to update the versions as needed).  Obviously, if you're using a different libc, you'll have to adjust the parameters for the three crossdev invocations as appropriate.

Do NOT use -9999 packages for cross-toolchains.  That's just asking for trouble.  Stick with released versions and periodically uninstall the toolchain and rebuild it as needed when new versions of gcc, binutils, glibc, or the kernel headers are released.

Last, create an alias like this for corss-compiling mips64 kernels:
alias mips64make='make ARCH=mips CROSS_COMPILE=mips64-unknown-linux-gnu-'

Then you can issue make commands like this to build mips64 kernels.
# mips64make -j10 nconfig
# mips64make -j10 vmlinux

If you have other MIPS CHOST targets you'd like to work on, you can repeat the above instructions, changing versions, ABI's, and CHOST parameters as appropriate.
Comment 15 Joshua Kinard gentoo-dev 2015-12-27 01:10:05 UTC
(In reply to Joshua Kinard from comment #14)
> crossdev on x86_64 should work perfectly fine, so that means your existing
> cross-compiler install is screwed up, and you need to blow it away and start
> over.
> 
> Run these commands in the following order:

If you use gcc-5.3 for a cross-compiler target, refer to this comment on Bug #547754:
https://bugs.gentoo.org/show_bug.cgi?id=547754#c7

I haven't seen any side-effects of it yet, but FYI.
Comment 16 Joshua Kinard gentoo-dev 2015-12-27 01:15:02 UTC
I've put new stages that I built earlier in December here:
http://dev.gentoo.org/~kumba/mips/stages/

They're mips4_r12k and mips3, both N32.  I meant to do more, but decided to pursue the uclibc stuff to try and get a more modern netboot created.  I'll be restarting those builds again once I finish the uclibc stuff.
Comment 17 Carlo 2015-12-27 01:41:18 UTC
> If you're running a 32bit compiler, but still somehow built a 64bit kernel
> binary...then I don't know.  I know binutils can install one time and output
> stuff for multiple architectures and ISAs, but I didn't think gcc was there
> just yet.

I have emerged kgcc in order to compile a 64bit kernel. It's a common practice for a 32 bit host and it works perfectly with kernel 2.6.17!

btw, I need 15 hours to rebuild kgcc && binutils, and then the whole 4.3 kernel with your /config

> My current frontplane is XBOW 1.4 (self-reports as 2.0)

as here on both machines: my IP30 & my friend's IP30, they are both XBOX v1.4 with modern motherboard (we can install 8Gbyte of ram and use V12 gfx). I will post a dmesg soon, I do not know their revision.
Comment 18 Joshua Kinard gentoo-dev 2015-12-27 01:54:57 UTC
(In reply to Carlo from comment #17)
> > If you're running a 32bit compiler, but still somehow built a 64bit kernel
> > binary...then I don't know.  I know binutils can install one time and output
> > stuff for multiple architectures and ISAs, but I didn't think gcc was there
> > just yet.
> 
> I have emerged kgcc in order to compile a 64bit kernel. It's a common
> practice for a 32 bit host and it works perfectly with kernel 2.6.17!

What worked for 2.6.17 isn't likely going to work with modern kernels.  In fact, kgcc64 is supposed to be masked on mips, because it's redundant these days, but I unmasked it for the time being.  If building on a local 32-bit system, mips-unknown-linux-gnu-gcc can output 64bit code by specifying the -mabi switch, I believe.  I haven't tried it yet (no 32bit installs left), but kgcc is slated for removal down the road once a few more arches get off of it.


> btw, I need 15 hours to rebuild kgcc && binutils, and then the whole 4.3
> kernel with your /config

Use the instructions in my previous post and build a new cross-compiler on a faster x86_64 box.  I don't build any of my IP30's kernels locally on that machine due to the compile time (and because I'm stuck on gcc-4.9.x in the n32 root due to gcc PR66038).


> > My current frontplane is XBOW 1.4 (self-reports as 2.0)
> as here on both machines: my IP30 & my friend's IP30, they are both XBOX
> v1.4 with modern motherboard (we can install 8Gbyte of ram and use V12 gfx).
> I will post a dmesg soon, I do not know their revision.

Keep your IP30's RAM at 2GB or less.  Anything greater than that is known to be broken because of the DMA problems yet to be worked out.
Comment 19 Carlo 2015-12-27 14:41:03 UTC
> Keep your IP30's RAM at 2GB or less.  Anything greater than that is known to
> be broken because of the DMA problems yet to be worked out.

yes, I know, our machines have 1.5Gbyte and 1.7Gbyte of ram.

I have a 32bit x86 machine, your instruction about cross-mips64 are successfully working! I am building a 4.3 kernel on x86 with gcc 5.3, then I will test it on octane!
Comment 20 Carlo 2015-12-27 15:49:15 UTC
how big is the kernel in size once compile ?
Comment 21 Carlo 2015-12-27 16:03:23 UTC
I have cross-emerged gcc-mips64 v5.2 
I emerge mips-sources, v4.3.0-20151126, with USE="ip30"
I added your .config
I compiled
I got a 22Mbyte file !

# lsprettysize vmlinux
22. 6 Mbyte

once booted I got this:

Option? 5
Command Monitor.  Type "exit" to return to the menu.
>> bootp():
Setting $netaddr to 192.168.1.5 (from server )
Obtaining  from server 
8274432+499232 entry: 0xa800000020614380

*** PROM write error on cacheline 0x1fcc1100 at PC=0xa8000000207c0b14 RA=0xa8000000207a4cf4


it happens on both Octanes here, I am removing features from your .config, and I am removing debug symbols
Comment 22 Carlo 2015-12-27 19:11:28 UTC
I have modified the .config, now it boots

see the dmseg

I am going to test my rootfs, and 2015-stage3


>> bootp():
Setting $netaddr to 192.168.1.5 (from server )
Obtaining  from server 
6851328+212144 entry: 0xa800000020490640
Initializing cgroup subsys cpu
Initializing cgroup subsys cpuacct
Linux version 4.3.0-mipsgit-20151126-ip30 (root@x60) (gcc version 5.3.0 (Gentoo 5.3.0 p1.0, pie-0.6.5) ) #4 SMP Sun Dec 27 19:52:07 Local time zone must be set--see zic 
ARCH: SGI-IP30
PROMLIB: ARC firmware Version 64 Revision 0
CPU0 revision is: 00000e35 (R12000)
FPU revision is: 00000900
Checking for the multiply/shift bug... no.
Checking for the daddiu bug... no.
Detected 1792MB of physical memory.
Determined physical RAM map:
 memory: 0000000000004000 @ 0000000000000000 (reserved)
 memory: 00000000006bd000 @ 0000000020004000 (reserved)
 memory: 000000000083f000 @ 00000000206c1000 (usable)
 memory: 0000000000100000 @ 0000000020f00000 (ROM data)
 memory: 000000006f000000 @ 0000000021000000 (usable)
Wasting 8499264 bytes for tracking 132801 unused pages
Zone ranges:
  DMA      [mem 0x0000000000000000-0x000000009fffffff]
  Normal   empty
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000000000-0x0000000000003fff]
  node   0: [mem 0x0000000020004000-0x000000008fffffff]
Initmem setup node 0 [mem 0x0000000000000000-0x000000008fffffff]
IP30: Slot: 0, PrID: 00000e35, PhyID: 0, VirtID: 0
IP30: Slot: 1, PrID: 00000e35, PhyID: 1, VirtID: 1
IP30: Detected 2 CPU(s) present.
Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
Unified secondary cache 2048kB 2-way, linesize 128 bytes.
PERCPU: Embedded 14 pages/cpu @a80000002341b000 s19200 r8192 d29952 u57344
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 449536
Kernel command line: ip=off console=ttyS0,9600 rdinit=/sbin/init init=/bin/bash root=/dev/sdc5
PID hash table entries: 4096 (order: 3, 32768 bytes)
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Memory: 1786872K/1835008K available (4691K kernel code, 241K rwdata, 1500K rodata, 244K init, 193K bss, 48136K reserved, 0K cma-reserved)
Hierarchical RCU implementation.
        Build-time adjustment of leaf fanout to 64.
NR_IRQS:128
IP30: HEART interrupt controller initialized.
IP30: CPU0: 400 MHz CPU detected
clocksource: HEART: mask: 0xfffffffffffff max_cycles: 0x2e2049cda, max_idle_ns: 440795202628 ns
sched_clock: 52 bits at 12MHz, resolution 80ns, wraps every 4398046511080ns
clocksource: MIPS: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 9532684529 ns
sched_clock: 32 bits at 200MHz, resolution 4ns, wraps every 10710881277ns
Console: colour dummy device 80x25
Calibrating delay loop... 600.47 BogoMIPS (lpj=3002368)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 4096 (order: 3, 32768 bytes)
Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes)
Initializing cgroup subsys io
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys net_prio
Checking for the daddi bug... no.
Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
Unified secondary cache 2048kB 2-way, linesize 128 bytes.
CPU1 revision is: 00000e35 (R12000)
FPU revision is: 00000900
Synchronize counters for CPU 1: done.
Brought up 2 CPUs
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
NET: Registered protocol family 16
xtalk:0 xbow widget (rev 2.0) registered as a platform device.
xtalk:8 heart widget (rev F) registered as a platform device.
xtalk:13 bridge widget (rev C) registered as a platform device.
xtalk:15 bridge widget (rev D) registered as a platform device.
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
PTP clock support registered
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0x1d200000-0x1d9fffff]
pci_bus 0000:00: root bus resource [io  0x1da00000-0x1dbfffff]
pci_bus 0000:00: root bus resource [bus 00-ff]
PCI host bridge to bus 0001:00
pci_bus 0001:00: root bus resource [mem 0x1f200000-0x1f9fffff]
pci_bus 0001:00: root bus resource [io  0x1fa00000-0x1fbfffff]
pci_bus 0001:00: root bus resource [bus 01-ff]
clocksource: Switched to clocksource HEART
NET: Registered protocol family 2
TCP established hash table entries: 16384 (order: 5, 131072 bytes)
TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
TCP: Hash tables configured (established 16384 bind 16384)
UDP hash table entries: 1024 (order: 3, 32768 bytes)
UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
NET: Registered protocol family 1
pci-ip30: 0000:00:01.0 Bar 0 with size 0x00001000 at bus 0x01000000 vma 0x000000d041000000 is Direct 32-bit.
pci-ip30: 0000:00:01.1 Bar 0 with size 0x00001000 at bus 0x01001000 vma 0x000000d041001000 is Direct 32-bit.
PCI: Enabling device 0000:00:01.1 (0000 -> 0002)
pci-ip30: 0000:00:01.2 Bar 0 with size 0x00000100 at bus 0x01002000 vma 0x000000d041002000 is Direct 32-bit.
PCI: Enabling device 0000:00:01.2 (0000 -> 0002)
futex hash table entries: 512 (order: 4, 65536 bytes)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
brd: module loaded
loop: module loaded
pci-ip30: 0001:00:02.0 Bar 0 with size 0x00100000 at bus 0x01000000 vma 0x000000f041000000 is Direct 32-bit.
ioc3: part: [030-0891-003], serial: [KZG030] => class IP30 System Board
console [ttyS0] disabled
0001:00:02.0: ttyS0 at IOC3 0xf041020178 (irq = 0, base_baud = 458333) is a 16550A
console [ttyS0] enabled
0001:00:02.0: ttyS1 at IOC3 0xf041020170 (irq = 0, base_baud = 458333) is a 16550A
IOC3 Master Driver loaded for 0001:00:02.0
qla1280: QLA1040 found on PCI bus 0, dev 3
pci-ip30: 0000:00:03.0 Bar 0 with size 0x00000100 at bus 0x00700000 vma 0x000000001d700000 is Window.
pci-ip30: 0000:00:03.0 Bar 1 with size 0x00001000 at bus 0x01003000 vma 0x000000d041003000 is Direct 32-bit.
pci-ip30: 0000:00:03.0 Bar 6 with size 0x00010000 at bus 0x01010000 vma 0x000000d041010000 is Direct 32-bit.
PCI: Enabling device 0000:00:03.0 (0006 -> 0007)
random: nonblocking pool is initialized
scsi(0:0): Resetting SCSI BUS
scsi host0: QLogic QLA1040 PCI to SCSI Host Adapter
       Firmware version:  7.65.06, Driver version 3.27.1
blk-mq: reduced tag depth to 10240
qla1280: QLA1040 found on PCI bus 0, dev 0
pci-ip30: 0001:00:00.0 Bar 0 with size 0x00000100 at bus 0x01000000 vma 0x000000f101000000 is Direct I/O.
pci-ip30: 0001:00:00.0 Bar 1 with size 0x00001000 at bus 0x01100000 vma 0x000000f041100000 is Direct 32-bit.
pci-ip30: 0001:00:00.0 Bar 6 with size 0x00010000 at bus 0x01110000 vma 0x000000f041110000 is Direct 32-bit.
PCI: Enabling device 0001:00:00.0 (0006 -> 0007)
scsi(1:0): Resetting SCSI BUS
scsi host1: QLogic QLA1040 PCI to SCSI Host Adapter
       Firmware version:  7.65.06, Driver version 3.27.1
blk-mq: reduced tag depth to 10240
qla1280: QLA1040 found on PCI bus 0, dev 1
pci-ip30: 0001:00:01.0 Bar 0 with size 0x00000100 at bus 0x01000100 vma 0x000000f101000100 is Direct I/O.
pci-ip30: 0001:00:01.0 Bar 1 with size 0x00001000 at bus 0x01120000 vma 0x000000f041120000 is Direct 32-bit.
pci-ip30: 0001:00:01.0 Bar 6 with size 0x00010000 at bus 0x01130000 vma 0x000000f041130000 is Direct 32-bit.
PCI: Enabling device 0001:00:01.0 (0006 -> 0007)
scsi 1:0:1:0: Direct-Access     FUJITSU  MAH3091MC        0115 PQ: 0 ANSI: 4
scsi(1:0:1:0): Sync: period 10, offset 12, Tagged queuing: depth 31
scsi 1:0:2:0: Direct-Access     FUJITSU  MAH3091MC        0115 PQ: 0 ANSI: 4
scsi(1:0:2:0): Sync: period 10, offset 12, Tagged queuing: depth 31
scsi(2:0): Resetting SCSI BUS
scsi host2: QLogic QLA1040 PCI to SCSI Host Adapter
       Firmware version:  7.65.06, Driver version 3.27.1
blk-mq: reduced tag depth to 10240
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
sd 1:0:1:0: [sda] 17850264 512-byte logical blocks: (9.13 GB/8.51 GiB)
ioc3-eth: Ethernet address is 08:00:69:13:66:d4.
IOC3 0001:00:02.0 eth0: link up, 100Mbps, full-duplex, lpa 0x05E1
eth0: Using PHY 1, vendor 0x15f42, model 2, rev 3.
eth0: IOC3 SSRAM has 128 kbyte.
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
pci-ip30: 0000:00:01.2 Bar 0 with size 0x00000100 at bus 0x01020000 vma 0x000000d041020000 is Direct 32-bit.
ehci-pci 0000:00:01.2: EHCI Host Controller
ehci-pci 0000:00:01.2: new USB bus registered, assigned bus number 1
ehci-pci 0000:00:01.2: irq 9, io mem 0xd041020000
ehci-pci 0000:00:01.2: USB 2.0 started, EHCI 1.00
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 4.3.0-mipsgit-20151126-ip30 ehci_hcd
usb usb1: SerialNumber: 0000:00:01.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
usbcore: registered new interface driver uas
usbcore: registered new interface driver usb-storage
usbcore: registered new interface driver usbserial
usbcore: registered new interface driver usbserial_generic
usbserial: USB Serial support registered for generic
usbcore: registered new interface driver ftdi_sio
usbserial: USB Serial support registered for FTDI USB Serial Device
rtc-ds1685 rtc-ds1685: rtc core: registered rtc-ds1685 as rtc0
NET: Registered protocol family 17
sd 1:0:2:0: [sdb] 17850264 512-byte logical blocks: (9.13 GB/8.51 GiB)
rtc-ds1685 rtc-ds1685: setting system clock to 2015-12-27 19:09:27 UTC (1451243367)
sd 1:0:2:0: [sdb] Write Protect is off
sd 1:0:2:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
sd 1:0:2:0: [sdb] Attached SCSI disk
sd 1:0:1:0: [sda] Write Protect is off
sd 1:0:1:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
 sda: sda1 sda2 sda3 sda9 sda11
sd 1:0:1:0: [sda] Attached SCSI disk
scsi 2:0:4:0: Direct-Access     SEAGATE  ST19101N         0014 PQ: 0 ANSI: 2
scsi(2:0:4:0): Sync: period 10, offset 12, Tagged queuing: depth 31
sd 2:0:4:0: [sdc] 17821600 512-byte logical blocks: (9.12 GB/8.49 GiB)
sd 2:0:4:0: [sdc] Write Protect is off
sd 2:0:4:0: [sdc] Write cache: disabled, read cache: disabled, supports DPO and FUA
 sdc: [mac] sdc1 sdc2 sdc3 sdc4 sdc5
sd 2:0:4:0: [sdc] Attached SCSI disk
EXT2-fs (sdc5): warning: mounting ext3 filesystem as ext2
VFS: Mounted root (ext2 filesystem) readonly on device 8:37.
devtmpfs: mounted
Freeing unused kernel memory: 244K (a800000020653000 - a800000020690000)
root #
Comment 23 Carlo 2015-12-27 19:33:51 UTC
 
> *** PROM write error on cacheline 0x1fcc1100 at PC=0xa8000000207c0b14
> RA=0xa8000000207a4cf4

as far as I have understood (studying "See MIPS Run", book) it means: 
-       cause: an unhandled exception happened due to using a mapped address
- consequence: the PROM caught it and reported

the problem is WHY happens!
Comment 24 Joshua Kinard gentoo-dev 2015-12-27 20:03:37 UTC
(In reply to Carlo from comment #23)
>  
> > *** PROM write error on cacheline 0x1fcc1100 at PC=0xa8000000207c0b14
> > RA=0xa8000000207a4cf4
> 
> as far as I have understood (studying "See MIPS Run", book) it means: 
> -       cause: an unhandled exception happened due to using a mapped address
> - consequence: the PROM caught it and reported
> 
> the problem is WHY happens!

What config items did you remove to get around this?

And what version is your PROM on both Octanes?
Comment 25 Carlo 2015-12-27 20:24:12 UTC
> What config items did you remove to get around this?

i can upload the .config, basically I have removed debug symbols, gfx, sound, kb*mouse, hid, XFS, and things like these, getting vmlinux within 9Mbyte of size
 
> And what version is your PROM on both Octanes?

>> version
PROM Monitor SGI Version 6.5 Rev 4.17 IP30 Jan 20, 2004 - 64 Bit

which is your version ?
Comment 26 Carlo 2015-12-27 21:36:49 UTC
I switched from sys-kernel/mips-sources-4.3.0 to sys-kernel/mips-sources-4.3.3
I copied your config, and compile
I got vmlinux, 23. 8 Mbyte 
start address 0xa8000000205ef6c0


>> version
PROM Monitor SGI Version 6.5 Rev 4.17 IP30 Jan 20, 2004 - 64 Bit


Usage: list files:       ls DEVICE
>> bootp():
Setting $netaddr to 192.168.1.5 (from server )
Obtaining  from server 
8328064+646672 entry: 0xa8000000205ef6c0

*** PROM write error on cacheline 0x1fcc1100 at PC=0xa8000000207cd568 RA=0xa8000000207b1eb4


I am lost =(
Comment 27 Carlo 2015-12-27 21:41:02 UTC
objdump -x vmlinux | grep 'start address'

start address 0xa8000000205ef6c0 not booting kernel
start address 0xa800000020490640 booting kernel
Comment 28 Joshua Kinard gentoo-dev 2015-12-28 03:20:14 UTC
The output size of vmlinux is generally not a problem on the ARCS64 systems, like IP30 or IP27.  The only issue I've ever had where the PROM rejects the kernel is when using CONFIG_DEBUG_LOCK_ALLOC.  That seems to pad one of the ELF sections out the wrong way, causing the PROM to throw a "not enough space in a FreeMemory area" error.

That said, I've built and booted kernels up to 40MB in size on IP30, with most debugging options turned on *and* a ramdisk included, so size isn't your problem.  I've searched Google, and it seems the cacheline error is common to both Linux and IRIX.  No real solutions, though.

One thing does come to mind, though.  In the .config I sent you, I have Impact Early Printk support enabled, and that makes an attempt to write directly to address space where the Impact hardware is expected to be mapped to by the PROM.  I believe you said you run headless, so start again with the .config I attached to this bug, and disable ONLY Early Printk support in the Kernel debugging menu.  Then try and boot with that kernel.  If that fails, remove the entire framebuffer subsystem, rebuild and try to boot again.

I have a sneaking suspicion it's one of those two drivers, because Stan's original Impact driver does use some hardcoded hardware addresses.  The early printk stuff started out as a hack that Stan sent me and I had to manually add to key spots in the code.  After buffered printk() support was added back in 3.6, I converted the early printk into an early_console driver.  But it still relies on scribbling to fixed hardware addresses VERY early in the boot cycle, and that's probably what's causing the PROM to throw a write error.
Comment 29 Carlo 2015-12-28 11:08:27 UTC
> I believe you said you run headless

yes, there is NO gfx in both machines!

my carrier is empty in machine1 (dual R12K)
while machine2 (dual R14K) has MENET & ShoeHorn+PCI_TG1

~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 

> so start again with the .config I attached to this bug, 
> and disable ONLY Early Printk support in the Kernel debugging menu.  

I will!

I have built a bash-script that keeps track of every config differences, so I can proceed step by step and give you a dot diff

I have also forked 4.3.3 sources into two branches: your configured, and mine.

Mine has the purpose to give me a working machine, your-configured has the purpese to sync differences between me and you


~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 

talking about my-bransh: 

even if the kernel supports PAGE_SIZE[]={ 4Kbyte, 16Kbyte, 64Kbyte }, i have a problem using uclibc with PAGE_SIZE=64Kbyte, and a problem with glibc with PAGE_SIZE=64Kbyte, I haven't tried 16Kbyte yet.

stage3-uclibc can run well with 4KB, while stage3-glibc reports errors

mips / # emerge --search bash
/usr/bin/emerge: line 6: import: command not found
/usr/bin/emerge: line 8: try:: command not found
/usr/bin/emerge: line 9: import: command not found
/usr/bin/emerge: line 11: syntax error near unexpected token `('
/usr/bin/emerge: line 11: `     def exithandler(signum,frame):'

stage3-glibc is fine with 64Kbyte, emerge works, while stage3-uclibc reports errors

mips mnt # chroot /mnt/gentoo/ /bin/bash
/bin/bash: can't map '/lib/libdl.so.0'
/bin/bash: can't map '/lib/libdl.so.0'
/bin/bash: can't map '/lib/libdl.so.0'
/bin/bash: can't load library 'libdl.so.0


to summarize
============

with 4Kbyte
     - stage3-uclibc seems to be working
     - stage3-glibc has no working emerge and python
with 16Kbyte
     - stage3-uclibc untested
     - stage3-glibc untested
with 64Kbyte
     - stage3-uclibc is not working
     - stage3-glibc seems to have a working emerge and python


what is your PROM revision ?
Comment 30 Carlo 2015-12-28 12:11:03 UTC
with PAGE_SIZE=16Kbyte

mips / # mount -t ext3 /dev/sda3 /mnt/gentoo/
EXT4-fs (sda3): mounting ext3 file system using the ext4 subsystem
EXT4-fs (sda3): recovery complete
EXT4-fs (sda3): mounted filesystem with ordered data mode. Opts: (null)

mips / # chroot /mnt/gentoo/ /bin/bash
/bin/bash: can't map '/lib/libdl.so.0'
/bin/bash: can't map '/lib/libdl.so.0'
/bin/bash: can't map '/lib/libdl.so.0'
/bin/bash: can't load library 'libdl.so.0'

mips / # emerge --search bash
Instruction bus error, epc == 0000000000418c98, ra == 0000000000418c2c
/usr/bin/emerge: line 6: import: command not found
/usr/bin/emerge: line 8: try:: command not found
/usr/bin/emerge: line 9: import: command not found
/usr/bin/emerge: line 11: syntax error near unexpected token `('
/usr/bin/emerge: line 11: `     def exithandler(signum,frame):'


so uclibc does not work, && glibc does not work

I will set PAGE_SIZE=64Kbyte
Comment 31 Carlo 2015-12-28 12:39:44 UTC
differences with your .config, so it uses PAGE_SIZE=64Kbyte

diff .config myconfig/ip30-433.config.txt | grep "<"

< CONFIG_MAC_PARTITION=y
< CONFIG_EXT2_FS=y
< # CONFIG_EXT2_FS_XATTR is not set
< CONFIG_EXT3_FS=y
< # CONFIG_EXT3_FS_POSIX_ACL is not set
< # CONFIG_EXT3_FS_SECURITY is not set
< # CONFIG_ISO9660_FS is not set
< # CONFIG_VFAT_FS is not set
< # CONFIG_EFS_FS is not set
< # CONFIG_SQUASHFS is not set
< # CONFIG_PRINTK_TIME is not set
< CONFIG_PANIC_TIMEOUT=3
< # CONFIG_EARLY_PRINTK is not set
< CONFIG_CMDLINE_BOOL=y
< CONFIG_CMDLINE="ip=off console=ttyS0,9600 init=/bin/bash root=/dev/sda5"
< CONFIG_CMDLINE_OVERRIDE=y

NAME = Blurry Fish Butt
       27. 2 Mbyte vmlinux
start address 0xa800000020763190

(none) / # emerge --search bash
..
it works!
..

stage3-uclibc3

(none) / # mount -t ext3 /dev/sda3 /mnt/gentoo/
EXT4-fs (sda3): mounting ext3 file system using the ext4 subsystem
EXT4-fs (sda3): mounted filesystem with ordered data mode. Opts: (null)
(none) / # chroot /mnt/gentoo/ /bin/bash
/bin/bash: can't map '/lib/libdl.so.0'
/bin/bash: can't map '/lib/libdl.so.0'
/bin/bash: can't map '/lib/libdl.so.0'
/bin/bash: can't load library 'libdl.so.0'
(none) / #
Comment 32 Carlo 2015-12-28 16:26:05 UTC
googling for kernel pagesize I have found the following 

>> Why should the application or the toolchains depend on pagesize? I am
>> not very clear on this. Can someone explain it?
>
>To allow loading directly with mmap the executable file's layout must
>be such that it's it's segments are on offsets that are a multiple of
>the page size so in turn the linker must know that alignment.
Comment 33 Joshua Kinard gentoo-dev 2015-12-28 18:51:42 UTC
Where did you get a stage3-uclibc from?  Is it one of the old ones from 2012?  I haven't fully tested one of those in anything other than 64KB PAGE_SIZE, so I cannot attest to how well they work or not.  Glibc-based userlands should work regardless, but again, older userlands are likely to have problems because older binutils may have not handled PAGE_SIZE correctly.

Generally speaking, larger PAGE_SIZE values are more efficient for these CPUs.  The R10000-family can actually support up to 16MB PAGE_SIZE, but the ELF specification realistically limits Linux and IRIX to ~64KB max.  IRIX normally ran on 16KB PAGE_SIZE.  Under Linux, 4KB and 16KB PAGE_SIZE requires a three-level page table setup, but 64KB uses a two-level, so it's a bit quicker to find things in memory via the TLB, and thus, reduces TLB pressure and cache misses.

The downside is, under 64KB PAGE_SIZE, problems with DMA manifest themselves quicker if your RAM is >2GB.  And, >2GB RAM + Impact will crash on boot because the DMA allocation will get messed up when Impact goes to reserve the first 64kb kmempool.  Hence why you want to stay under 2GB RAM.

And I don't know my PROM version offhand.  I'll have to check next time I cycle the machine.  It's the latest from IRIX, though, so likely 4.17.
Comment 34 Carlo 2015-12-28 19:18:16 UTC
> Where did you get a stage3-uclibc from?  

I built it in 2009, but I am going to use this(1). That dude has a few uclibc-rootfs (for X86, PPC, MIPS, etc), and a great builder (QEMU based).

(1)http://landley.net/aboriginal/downloads/binaries/root-filesystem/

> Glibc-based userlands should work regardless

I have tested { 2007, 2008, 2015 } stage4: they seem working BUT only with PAGE_SIZE=64Kbyte! Emerge is working and I do not see any strange behavior with bash and python

> but again, older userlands are likely to have problems 
> because older binutils may have not handled PAGE_SIZE correctly

here I can't use stage3 2015 with PAGE_SIZE[]={ 4Kbyte, 16Kbyte }
- emerge does not work correctly 
- python does not work correctly 
- bash does not work correctly 

to have all of the above working, I MUST use PAGE_SIZE=64Kbyte


> Generally speaking, larger PAGE_SIZE values are more efficient for these
> CPUs.  The R10000-family can actually support up to 16MB PAGE_SIZE, but the
> ELF specification realistically limits Linux and IRIX to ~64KB max.  IRIX
> normally ran on 16KB PAGE_SIZE.  Under Linux, 4KB and 16KB PAGE_SIZE
> requires a three-level page table setup, but 64KB uses a two-level, so it's
> a bit quicker to find things in memory via the TLB, and thus, reduces TLB
> pressure and cache misses.
> 
> The downside is, under 64KB PAGE_SIZE, problems with DMA manifest themselves
> quicker if your RAM is >2GB.  And, >2GB RAM + Impact will crash on boot
> because the DMA allocation will get messed up when Impact goes to reserve
> the first 64kb kmempool.  Hence why you want to stay under 2GB RAM.

I am 1.7Gbyte, also with exactly 2Gbyte I have a LOT of problems with PCI_USB
Comment 35 Carlo 2015-12-28 19:34:37 UTC
the MENET board has only 2 NICs recognized X____X
there is a name inversion with the built-in lan

Eth0 <------ MENET.NIC0
Eth1 <------ MENET.NIC1
Eth2 <------ built-in lan (on the IP30's motherboard)

MENET.NIC2 ?
MENET.NIC3 ?

the built-in lan should be called "eth0"

is it possible to fix it ?
Comment 36 Carlo 2015-12-28 19:42:51 UTC
PageSize, goot lecture (for me): https://www.linux-mips.org/wiki/Page_size
Comment 37 Carlo 2015-12-28 19:50:14 UTC
corollary: stage3-mips1 only supporta 4K pagesize
so, it can't be used with kernel 4.* on IP30!
Comment 38 Joshua Kinard gentoo-dev 2016-01-01 01:53:55 UTC
(In reply to Carlo from comment #34)
> > Where did you get a stage3-uclibc from?  
> 
> I built it in 2009, but I am going to use this(1). That dude has a few
> uclibc-rootfs (for X86, PPC, MIPS, etc), and a great builder (QEMU based).
> 
> (1)http://landley.net/aboriginal/downloads/binaries/root-filesystem/

I am making good progress on creating a MIPS-II, stage3, uClibc-based rootfs.  Hopefully catalyst will finish up tonight and I'll start assembling a netboot image by hand to test stuff with over this weekend.


> > Glibc-based userlands should work regardless
> 
> I have tested { 2007, 2008, 2015 } stage4: they seem working BUT only with
> PAGE_SIZE=64Kbyte! Emerge is working and I do not see any strange behavior
> with bash and python

I'll have to test different PAGE_SIZE values again once I cobble a netboot together.  The problem with those IBEs and DBEs, is you usually have to cold reset the machine, including pulling the power cord for a good 30 seconds to let it fully de-energize.  That seems to clear everything out.  Otherwise, when you boot back up, something is still stuck in a cache somewhere that seems to bring them right back.  This has made testing stuff really annoying, especially on the Onyx2, which can hold a lot of power in that PSU it has.


> > but again, older userlands are likely to have problems 
> > because older binutils may have not handled PAGE_SIZE correctly
> 
> here I can't use stage3 2015 with PAGE_SIZE[]={ 4Kbyte, 16Kbyte }
> - emerge does not work correctly 
> - python does not work correctly 
> - bash does not work correctly 
> 
> to have all of the above working, I MUST use PAGE_SIZE=64Kbyte

64KB PAGE_SIZE is better anyways.  My binutils compile on the Octane went from 1hr 15mins to 45mins under both 16KB and 64KB from 4KB.

We just need to work all of the bugs out of userland apps that have problems with it.  I'll have to play around some more to see if building stages under a 64KB PAGE_SIZE is hardcoding it somewhere and causing problems for 4KB and 16KB PAGE_SIZEs.  That would suggest a bug somewhere, possibly in the libc or such, because if everything is working correctly, you should be able to change PAGE_SIZE around with no side-effects.

It could also be the R1x000 family itself.  My O2 boots fine under either 16KB or 64KB.  But, more experimentation is needed and the machine(s) cold restarted when side-effects are seen to eliminate spurious variables.


> 
> I am 1.7Gbyte, also with exactly 2Gbyte I have a LOT of problems with PCI_USB

Yeah, I forgot this about Octane.  USB/PCI stuff starts having problems above 1.7GB, while almost all add-on PCI has problems beyond 2GB.  The built-in PCI devices still appear to function fine.  We'll get it worked out, eventually...

--------

(In reply to Carlo from comment #35)
> the MENET board has only 2 NICs recognized X____X
> there is a name inversion with the built-in lan
> 
> Eth0 <------ MENET.NIC0
> Eth1 <------ MENET.NIC1
> Eth2 <------ built-in lan (on the IP30's motherboard)
> 
> MENET.NIC2 ?
> MENET.NIC3 ?
> 
> the built-in lan should be called "eth0"
> 
> is it possible to fix it ?

Possible, yes.  We had to do this on the O2 a while back, to force the kernel to always enumerate the Meth interface as eth0.  I don't know if we can do it on the Octane, though.  When I was testing my MENET card out for a while, it found all four NICs, so I had to make my primary ethernet interface eth5.
Comment 39 Carlo 2016-01-01 02:26:21 UTC
here I have a working net boot!
a working uclibc-mips3 stage4
(including a native uclibc-gcc)

let me know if you need!
Comment 40 Carlo 2016-01-01 02:32:04 UTC
> We just need to work all of the bugs out of userland apps that have problems
> with it.  I'll have to play around some more to see if building stages under
> a 64KB PAGE_SIZE is hardcoding it somewhere and causing problems for 4KB and
> 16KB PAGE_SIZEs.  That would suggest a bug somewhere, possibly in the libc
> or such, because if everything is working correctly, you should be able to
> change PAGE_SIZE around with no side-effects.

interesting, here I have MIPS32-R2 machines (and I am temped to buy (1)), I can use them to check if the problem is related to userland(glibc? uclibc?) or kernel 


(1) https://wiki.gentoo.org/wiki/MIPS/ERLite-3
it's MIPS64, claimed to be compatible with ISA { MIPS1, MIPS2, MIPS3, MIPS4 }
Comment 41 Carlo 2016-12-11 13:12:01 UTC
any news ?
Comment 42 Joshua Kinard gentoo-dev 2017-01-05 23:27:32 UTC
I recently added new mips-sources for 4.4.x and 4.8.x.  SMP works fine on IP30 for both kernels.  Please try those out and if you still have problems, open a new bug.  I am going to close this one.