Hello again, :) There is a bug when compiling par2cmdline with "-O3 -mcpu=pentium4" in CFLAGS/CXXFLAGS (gcc-3.3.3-r5). This is of course probably a bug in the compiler/optimizer itself, but finding this bug in the compiler/optizer is probably highly non-trivial since I cannot seem to even trigger the bug with datasets smaller than about 3GB. What happens is that a recovery data set can be created, but does not verify properly immediately after creation. Changing "-O3" to "-O2" fixes the problem for me, so replace-flags -O3 -O2 should probably be added to src_compile(). I don't have access to other architectures so I cannot check whether this only applies to Pentium4, or if it also applies to other architectures. Cheers,
It turns out that the verify phase still fails under some (even rarer) circumstances if the "-mcpu=pentium4" is present, so replace-flags -O3 -O2 replace-flags -mcpu=pentium4 "" should be used.
Just some more input from testing: The bug is also present with -mcpu=pentium3 and -march=pentium3 (gcc-3.3.3-r6). (Even at -O2)
I hacked up a sample 6gb dataset myself, on a dual P4 xeon machine, and it seems to work perfectly fine for me. So I wonder if your hardware isn't just slightly off? Try some runs of cpuburn and memtest86.
Well, the thing is that this is completely reproducible for me, that is, the "checksum" will always fail at precisely the same spot given the same input files, so I shouldn't think that it was being caused by flaky hardware. I've also never, ever had any h/w-related problems with my current hardware in the 2-3 years I've had it. I would also be skeptical that flaky hardware problems would only show up when using "-O2 -mcpu=pentium3" and not when using "-O2". (I mean, yes, there are obviously differences in the code ordering, but you'd think hardware problems would show up in any case...) I've previously run memtest86 (I let it loop about 7 times, IIRC), so the memory is probably fine (btw, I've never had any problems with gcc itself -- just running gcc on major code bases tends to reveal problems with flaky memory). I'll try running cpuburn for a couple of hours, but I seriously doubt anything will show up. Anyway, will report back in a few hours. :)
Well, I've had burnMMX running for about 3 hours, and... nothing. It doesn't show any errors, and there have been no crashes or anything like that.
I still can't reproduce this, with -O3 and -march=pentium4 -mcpu=pentium4
If you're up for downloading it (it may take a while, it's 2GB), I've found a particular randomly generated file which causes problems for me (with the -O3 -mcpu=pentium3 combo... this time on 3.3.4 20040623). I can put probably put up a .torrent of it later if you're interested in downloading it to see if you can reproduce the problem.
2Gb is a bit much of a download, but lets try a different approach first. using your 2gb problem inputfile, do: 'split -b 100M inputfile' then run par2 over each segment, and see if you can isolate it to one of them. if so, repeat as: 'split -b 5M isolated100Mproblemfile' and see if you can get it down to one of those.
Bah! I thought it was reproducible on this particular file, but it turns out it wasn't (although I'm pretty sure I tried ran "verify" on it twice initially). I think something more subtle is going on my particular machine, so I'll just close the bug so you don't have to waste any more time on it. I can always reopen it if I ever find a reliable test case. Sorry to have wasted your time.
