The manual for gcc points out thot -fprofile-arcs can be used to generate .da files for each source file and will cause a program to generate a profile of its own exectuion. These profiling features have further been enhanced in GCC 3.3 to give extra functionality in comparison with 3.2. After exiting, a program dumps into its local directory a .da file for each source file. This file contains profiling info for the functions in the source which can then be used to recompile the program with the same options AND with -fbranch-probabilities to output code optimized for the program's execution patterns, instead of ordered randomly. I propose a patch on GCC be written to make the program spit out a single .da file, preferably in a directory determined at compile time (/var/cache/profile/) and to be able to command GCC to look for this output file when compiling a source file. This output file would contain a tree-like structure to trace down to each source file, and to each function, via hash. This would allow a single output file to be read without much of a speed trade-off. This patch should also allow the program to augment the output file with the statistics, instead of replace it. This would allow profiling to occur over a number of runs. IN ADDITION, THIS PATCH SHOULD BE SUBMITTED TO THE GCC PROJECT ITSELF. This file would be generated if the portage variable "PORTAGE_USE" included "+profiling", or if the --profile option was given on the emerge command line. When the package was merged, portage would store the exact command line passed to gcc and would delete any existing .da files for the package. When portage is run again, if all of the merged executables have .da files and the emerge option --profile-evaluate is given, or the --force-profile-evaluate option is specified for a package and it was compiled with profiling, the package is remerged with the same USE flags and gcc options, except with the -fbranch-probablilities option and the .da file given. This would be extremely advantageous for packages that don't get updated much but does get used a lot, such as GCC itself. GCC gets updated/recompiled on a system infrequently, but used quite a lot. The speed increase would be substantial. The same could be said about the Gnome packages or Mozilla. Reproducible: Always Steps to Reproduce: 1. 2. 3.
Right, so how far are you with a patch ? Serious now, I do not have time for a big project like this, so feel free, or if anybody else feels up to it ...
On the portage end, this looks more like a project for portage-ng, if that plig-in capability works out well. This "profiling" feature could simply be a C/Python plugin. I think this bug needs to be saved for review at a later date once something functional on the portage-ng project gets outputted.
I can't rewrite GCC to handle it, unfortunately. If I had the time and skill I would have before sending this in. I see it's been assigned to the GCC porting team and left open for review though, so at least that's something. Maybe someone will in the future. As it is now, the .da files would overlap and flood out the /usr/bin directory and become altogether useless.
The biggers issue in my opinion will be the gcc side. Just getting portage to see if there is a foo file, and setting some variable or cp'ing something somewhere is not that a major issue.
Definitely. Everything hinges on patching the profiling code to spit out per-binary .da files. Without this, there is no way to make this work due to the inevitable <source file>.da filename overlap. What else would be nice would be to be able to combine multiple profiles and decide what would in general be the best for a library or executable, and then spit out a small file explaining the block ordering, which portage could download with the builds to use for non-profiling builds (GCC by default uses a random model, so sampled models would of course be better in some or most cases). Still, it would rely on sampled user feedback.
i think it would be alot quicker and easier if portage was changed to allow profiling to happen easier i did some playing with gcc 3.4 and its profiling a while back all thats really needed is first run, portage told via some command line option, to add -fprofile-generate to cflags then after installing the program keep the source directorys around and change perms so any user can write to the directorys to generate the profiling info, keeping the source files around isnt nessary just the directory structure of the source, then user does some profiling of the apps then remerges the program with another command line option to emerge which adds -fprofile-use to cflags this time then it rebuilds the program using the generated profile data and on this run it can delete the source left over if you wanted to take this farther you would make it so portage would keep the profiling info around between -r versions of the ebuilds
original gcc enhancment idea is beyond the Gentoo project if gcc gets said enhancment, we can review again how best to incorporate it into portage
