Compilation fails with "ACCESS VIOLATION SUMMARY" just after ">>> Source compiled.". There are some "ACCESS DENIED" errors around 49% of the build.log (it is probably between the first and the second phase of the PGO compilation). Reproducible: Always Steps to Reproduce: 1. emerge -av1 firefox Actual Results: Compilation fails with "ACCESS VIOLATION SUMMARY" just after ">>> Source compiled.". There are some "ACCESS DENIED" errors around 49% of the build.log Expected Results: Firefox gets compiled and installed successfully. Without "pgo" use flag firefox compiles and installs successfully. emerge --info Portage 2.1.10.11 (default/linux/amd64/10.0/desktop/kde, gcc-4.5.3, glibc-2.13-r4, 2.6.39-gentoo-r3km x86_64) ================================================================= System uname: Linux-2.6.39-gentoo-r3km-x86_64-Intel-R-_Core-TM-_i5_CPU_M_430_@_2.27GHz-with-gentoo-2.0.3 Timestamp of tree: Wed, 14 Dec 2011 10:45:01 +0000 app-shells/bash: 4.1_p9 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.7.2-r3, 3.1.4-r3 dev-util/cmake: 2.8.6-r4 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.0.3 sys-apps/openrc: 0.9.4 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1 sys-devel/binutils: 2.21.1-r1 sys-devel/gcc: 4.5.3-r1 sys-devel/gcc-config: 1.4.1-r1 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r1 sys-kernel/linux-headers: 2.6.39 (virtual/os-headers) sys-libs/glibc: 2.13-r4 Repositories: gentoo x-portage ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa /var/bind /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps y --keep-going" FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="" GENTOO_MIRRORS="http://gentoo.po.opole.pl ftp://gentoo.po.opole.pl rsync://gentoo.prz.rzeszow.pl/gentoo http://gentoo.prz.rzeszow.pl/ rsync://ftp.vectranet.pl/gentoo/ http://ftp.vectranet.pl/gentoo/ ftp://ftp.vectranet.pl/gentoo/ http://gentoo.mirror.pw.edu.pl/" LANG="pl_PL.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="pl en" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.pl.gentoo.org/gentoo-portage" USE="3ds 64bit R X a52 aac aacplus aacs aalib abiword acl acpi activefilter addressbook airplay akonadi alac allegro alsa amd64 amr apache2 apng archive aspell assistant atm atmo attica autoipd automap autotrace avahi avx bash-completion battery berkdb bidi binary-drivers bineditor binfilter bittorrent blas blender blender-game bluetooth bluray bookmarks boost branding bzip2 cairo cdda cddax cddb cdinstall cdio cdparanoia cdr cdrdao clamdtop cli clucene collab compress compressed-lumas consolekit context contrast contrib coverage cpio cracklib crypt css cue cups curl cxx cyrillic d daap dbus declarative designer designer-plugin desktopglobe detex device-mapper devil dewarping dga dhcp dia dirac directv disk-partition djvu dot downloadorder drawspaces dri drm dts dv dvb dvd dvdr dvi dvi2tty dvipdfm eap-sim eap-tls ebook eclipse editor eigen embedded emboss emf emovix encode epos espeak exceptions exif expoblending extensions extra extra-algorithms extrafilters extras facedetect fam fat fax fbcon festival ffmpeg fftw firefox firefox3 fits flac flash flite fltk fluidsynth fontconfig foomaticdb fortran fping fpx freetts frei0r fuse gadu galago gallium game-engine games garmin gcj gd gdal gdbm gdml gdu gecko gedit geoip geolocation geos ggi ggz gif gimp ginac glep gles glew glgd glib glibc-omitfp glsl glu gmail gmath gme gmm gmp gnome gnome-keyring gnome-print gnuplot gnutls google-gadgets gpac gphoto2 gpm gps gpu grammar graph graphics graphicsmagick graphite graphviz grass groupwise gs gsl gsm gsmloc gssapi gstreamer gtk gtkstyle gts guile hbci hddtemp hdf5 hdri hfs hipe hotpixels howl-compat hpijs httpd humanities hunspell hybrid-auth hyperestraier icecast iconv icq icu id3tag idn ilbc imagemagick imap imlib indi inkjar inotify int64 ios ipfilter iphone iplayer ipod ipv6 irc irrlicht jabber jack jadetex java javascript jbig jce jfs jingle john joinlines jpeg jpeg2k kate kdcraw kde kdecards kdepim kdm kdrive kerberos kernel-patch kexiv2 keyboard kig-scripting kino kipi kismet kmail kolab konqueror kontact kpathsea kpoll kqemu kross kvm ladspa lame lapack laptop lash lastfm latex latex3 lcms ldap lensfun libass libburn libcaca libextractor libmms libnl libnotify libproxy libsamplerate libsysfs libtiger libv4l libv4l2 libvisual libwww live llvm llvm-gcc lm_sensors lock logviewer lqr lua luatex lucene lv2 lzma lzo mad madwifi maemo6 magnetgenerator map mapnik math mathml matplotlib matroska mbrola mdadm mdnsresponder-compat meanwhile mediaplayer melt metric midi mikmod milter mime mjpeg mkl mktemp mms mmx mng modplug modules mono monolithic-build mozdom mozembed mp3 mp3tunes mp4 mpeg mpi mpi-threads mplayer msn mtp mudflap multilib multimedia musepack music musicbrainz nano-syntax nas natspec nautilus ncurses netcdf netpbm network networking networkmanager new-login nls nmap nokia nova nowlistening npp nptl nptlonly nsplugin ntfs ntlm ntp numpy nut nuv obex objc objc++ objc-gc ocaml ocean ocr ocrad octave odbc offensive ofx ogdi ogg ogm okular omega openal openctl opencv openexr opengl openmp openvg openxml optimization optimized-qmake oscar otr ots pam pango pcm pcmcia pcre pda pdf pdfannotextractor perl pgf pgo phonon pidgin plasma player plotutils pmu png policykit portaudio postgis postproc povray ppds pppd private-headers projectm psf pstoedit pstricks publishers pulseaudio pvr python qalculate qq qt-dbus qt-faststart qt-webkit qt3support qt4 qthelp qtscript quicktime quotes qwt radio radius rar raw readline realtime redeyes reiser4 reiserfs remoteosd reports resolvconf roaming romio rss rtf rtmp rtsp ruby samba scale0tilt scanfolder scanner schroedinger science scim script scripts sctp sdl sdl-image semantic-desktop session shm shout sigsegv sip sipim skins skype smp sms sndfile snmp socks socks5 solver soundex soundtouch soup sox sparse speech speex spell sqlite srtp sse sse2 ssl ssse3 startup-notification stream strigi stroke subtitles subversion svg swat swig sysfs t1lib taglib tcl tcpd telepathy teletext templates testbed tex4ht texmacs tga themedesigner theora thesaurus threads threadsafe thumbnail tiff timezone timidity tk tlen tomsfastmath tonal toolame toolbar tools totem truetype twitter twolame type3 udev udev-acl umfpack unicode unlock-notify unzip upnp usb utils uuencode v4l v4l2 vaapi vamp vboxwebsrv vcd vcdinfo vcdx vdr verse video virtualbox vlm vnc vorbis vpx wav wavpack webcam webinterface webkit webp webpresence wicd wifi wikipedia win32 win64 windeco winetriks winpopup wireshark wma wma-fixed wmf word-perfect wordperfect wpd wpg wps wv2 wxwidgets x264 xanim xattr xbase xcb xcf xcomposite xetex xext xface xfs xindy xine xinerama xml xmlpatterns xmp xorg xpm xscreensaver xulrunner xv xvfb xvid xvmc yahoo yaz youtube yv12 zeroconf zip zlib zvbi" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="spca50x canon fuji konica mustek samsung ricoh pccam300 pccam600 ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev synaptics wacom" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="pl en" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fglrx" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS emerge -pv firefox These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] www-client/firefox-8.0 USE="alsa crashreporter dbus ipc libnotify pgo* startup-notification webm wifi -bindist -custom-cflags -custom-optimization -debug -system-sqlite" LINGUAS="en pl -af -ak -ar -ast -be -bg -bn -bn_BD -bn_IN -br -bs -ca -cs -cy -da -de -el -en_GB -en_US -en_ZA -eo -es -es_AR -es_CL -es_ES -es_MX -et -eu -fa -fi -fr -fy -fy_NL -ga -ga_IE -gd -gl -gu -gu_IN -he -hi -hi_IN -hr -hu -hy -hy_AM -id -is -it -ja -kk -kn -ko -ku -lg -lt -lv -mai -mk -ml -mr -nb -nb_NO -nl -nn -nn_NO -nso -or -pa -pa_IN -pt -pt_BR -pt_PT -rm -ro -ru -si -sk -sl -son -sq -sr -sv -sv_SE -ta -ta_LK -te -th -tr -uk -vi -zh -zh_CN -zh_TW -zu" 0 kB
Created attachment 295833 [details] build log
I have my portage temporary directory mounted as tmpfs: tmpfs /var/tmp/portage tmpfs noauto 0 0 but for firefox (due to its size) I use another direcory (/home/kornel/tmp/portage/).
Why did you put a single file in a tar archive?
(In reply to comment #3) > Why did you put a single file in a tar archive? Sorry, I was in a hurry and didn't notice it.
Created attachment 296401 [details] emerge_info.txt exactly the same here.
Created attachment 296685 [details] ff9.0 - same problem The same problem with www-client/firefox-9.0
Created attachment 299009 [details, diff] move addpredicts to pkg_setup() This simply moves the addpredicts to pkg_setup so they are carried threw the entire build process, give it a spin, if it fails we will drop pgo support in gentoo.
The problem is that pgo is trying to mknod the special files for some reason. Maybe it can't access /dev/ati/card0 or something? I guess the solution is either to: a) Put addpredict for /dev/ati/card{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,5} or b) Remove the code that tries to do this stupidity
Third solution: export LIBGL_ALWAYS_INDIRECT=1 or LIBGL_ALWAYS_SOFTWARE=1 so that the code doesn't try direct rendering at all.
I've just successfully compiled firefox 9: emerge -pv firefox These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] www-client/firefox-9.0 USE="alsa crashreporter dbus ipc libnotify pgo startup-notification webm wifi -bindist -custom-cflags -custom-optimization -debug -system-sqlite" LINGUAS="en pl -af -ak -ar -ast -be -bg -bn -bn_BD -bn_IN -br -bs -ca -cs -cy -da -de -el -en_GB -en_US -en_ZA -eo -es -es_AR -es_CL -es_ES -es_MX -et -eu -fa -fi -fr -fy -fy_NL -ga -ga_IE -gd -gl -gu -gu_IN -he -hi -hi_IN -hr -hu -hy -hy_AM -id -is -it -ja -kk -kn -ko -ku -lg -lt -lv -mai -mk -ml -mr -nb -nb_NO -nl -nn -nn_NO -nso -or -pa -pa_IN -pt -pt_BR -pt_PT -rm -ro -ru -si -sk -sl -son -sq -sr -sv -sv_SE -ta -ta_LK -te -th -tr -uk -vi -zh -zh_CN -zh_TW -zu" 0 kB Total: 1 package (1 reinstall), Size of downloads: 0 kB
(In reply to comment #10) > I've just successfully compiled firefox 9: > > emerge -pv firefox > > These are the packages that would be merged, in order: > > Calculating dependencies... done! > [ebuild R ] www-client/firefox-9.0 USE="alsa crashreporter dbus ipc > libnotify pgo startup-notification webm wifi -bindist -custom-cflags > -custom-optimization -debug -system-sqlite" LINGUAS="en pl -af -ak -ar -ast -be > -bg -bn -bn_BD -bn_IN -br -bs -ca -cs -cy -da -de -el -en_GB -en_US -en_ZA -eo > -es -es_AR -es_CL -es_ES -es_MX -et -eu -fa -fi -fr -fy -fy_NL -ga -ga_IE -gd > -gl -gu -gu_IN -he -hi -hi_IN -hr -hu -hy -hy_AM -id -is -it -ja -kk -kn -ko > -ku -lg -lt -lv -mai -mk -ml -mr -nb -nb_NO -nl -nn -nn_NO -nso -or -pa -pa_IN > -pt -pt_BR -pt_PT -rm -ro -ru -si -sk -sl -son -sq -sr -sv -sv_SE -ta -ta_LK > -te -th -tr -uk -vi -zh -zh_CN -zh_TW -zu" 0 kB > > Total: 1 package (1 reinstall), Size of downloads: 0 kB We would need to know what actually fixed it for you if you would like it fixed in the tree.
(In reply to comment #11) > (In reply to comment #10) > > I've just successfully compiled firefox 9: > > > > emerge -pv firefox > > > > These are the packages that would be merged, in order: > > > > Calculating dependencies... done! > > [ebuild R ] www-client/firefox-9.0 USE="alsa crashreporter dbus ipc > > libnotify pgo startup-notification webm wifi -bindist -custom-cflags > > -custom-optimization -debug -system-sqlite" LINGUAS="en pl -af -ak -ar -ast -be > > -bg -bn -bn_BD -bn_IN -br -bs -ca -cs -cy -da -de -el -en_GB -en_US -en_ZA -eo > > -es -es_AR -es_CL -es_ES -es_MX -et -eu -fa -fi -fr -fy -fy_NL -ga -ga_IE -gd > > -gl -gu -gu_IN -he -hi -hi_IN -hr -hu -hy -hy_AM -id -is -it -ja -kk -kn -ko > > -ku -lg -lt -lv -mai -mk -ml -mr -nb -nb_NO -nl -nn -nn_NO -nso -or -pa -pa_IN > > -pt -pt_BR -pt_PT -rm -ro -ru -si -sk -sl -son -sq -sr -sv -sv_SE -ta -ta_LK > > -te -th -tr -uk -vi -zh -zh_CN -zh_TW -zu" 0 kB > > > > Total: 1 package (1 reinstall), Size of downloads: 0 kB > > We would need to know what actually fixed it for you if you would like it fixed > in the tree. It is the ebuild from the main tree! Kornel
I've edited the ebuild to export LIBGL_ALWAYS_INDIRECT=1 if it detects that fglrx/nvidia are in use. This should solve the problem, and still allow *some* WebGL PGO for binary driver users. If the bug persists, we'll just disable the webgl bits in the PGO suite.
I suggest to switch eselect opengl to xorg-x11 during build.
*** Bug 402519 has been marked as a duplicate of this bug. ***
Still seeing symptoms described by OP with most recent versions of the following ebuilds and "eselect opengl set nvidia". =x11-drivers/nvidia-drivers-290.10:0::portage =www-client/firefox-9.0:0::portage[pgo] One thing that's odd to me is that src_compile() completes (emerge says 'source compiled.') , and then emerge bails with the error message described by kornel. emerge.log looks like an unsuccessful build: 1328906260: Started emerge on: Feb 10, 2012 21:37:39 1328906260: *** emerge --keep-going --load-average=10.0 --jobs=10 --autounmask=n firefox 1328906327: >>> emerge (1 of 1) www-client/firefox-9.0 to / 1328906327: === (1 of 1) Cleaning (www-client/firefox-9.0::/usr/portage/www-client/firefox/firefox-9.0.ebuild) 1328906421: === (1 of 1) Compiling/Merging (www-client/firefox-9.0::/usr/portage/www-client/firefox/firefox-9.0.ebuild) 1328909953: *** Finished. Cleaning up... 1328909958: *** exiting unsuccessfully with status '1'. After that, though, I can successfully run the install and qmerge phases manually using the ebuild command. Me confused.
same here as Carsten Milkau described 'export LIBGL_ALWAYS_INDIRECT=1' and 'export LIBGL_ALWAYS_SOFTWARE=1' both did not the trick had to add addpredict /dev/nvidiactl to pkg_setup
forgot to mention: firefox-10.0.1 and nvidia-drivers-290.10
Sorry, bug NOT fixed. Look at https://bugs.gentoo.org/show_bug.cgi?id=382753 its same bug.
--------------------------- ACCESS VIOLATION SUMMARY --------------------------- LOG FILE "/var/log/sandbox/sandbox-1729.log" VERSION 1.0 FORMAT: F - Function called FORMAT: S - Access Status FORMAT: P - Path as passed to function FORMAT: A - Absolute Path (not canonical) FORMAT: R - Canonical Path FORMAT: C - Command Line F: open_wr S: deny P: /dev/nvidiactl A: /dev/nvidiactl R: /dev/nvidiactl C: /var/portagetmp/portage/www-client/firefox-11.0-r1/work/mozilla-release/obj-x86_64-unknown-linux-gnu/dist/firefox/firefox-bin -no-remote -profile /var/portagetmp/portage/www-client/firefox-11.0-r1/work/mozilla-release/obj-x86_64-unknown-linux-gnu/_profile/pgo/pgoprofile/ http://localhost:8888/index.html F: open_wr S: deny P: /dev/nvidiactl A: /dev/nvidiactl R: /dev/nvidiactl C: /var/portagetmp/portage/www-client/firefox-11.0-r1/work/mozilla-release/obj-x86_64-unknown-linux-gnu/dist/firefox/firefox-bin --version --------------------------------------------------------------------------------
Created attachment 309205 [details, diff] proposed fix for pgo ff emerge
i guess recently attached patch fixes ff emerge issue with +pgo: the idea is obvious - move X11 environment change from pkg_setup ebuild section to pkg_compile one this will force emerge to use Xvfb instead of configured graphic card (with appropriate proprietary video driver) i created that patch for recent firefox-11.0-r1.ebuild, however the same technique (not the patch) might be applied to earlier releases
apparently the patch is relevant to bugs 382753 and 388585
reopened, because people (including me) still seem to encounter this issue. There have been various suggestions on how to fix it, but this needs a definite solution.
(In reply to comment #24) > reopened, because people (including me) still seem to encounter this issue. > > There have been various suggestions on how to fix it, but this needs a > definite solution. The definite solution is to p.use.mask pgo and remove all support from the tree as far as pgo builds are concerned. Nirbheek if you would please p.use.mask pgo for all archs, if at a later time we rework the ebuild to support pgo builds for all we can unmask.
Yeah, just was gonna add that whatever was added to try and support PGO with binary drivers still doesn't work (I use NVIDIA), not that big of a deal considering the easy workarounds, but just letting it be known.
Another easy Workaround is: emerge firefox from tty Terminal This build and install for me fine.
(In reply to comment #27) > Another easy Workaround is: emerge firefox from tty Terminal > This build and install for me fine. You can't emerge with +pgo from tty because Firefox need a running X server for testing build before optimization phase. The solution for me is adding permission to access device file if on nvidia or ati card.
Created attachment 311715 [details, diff] proposed fix for pgo ff ebuild (12.0) attached snippet patches firefox-12.0.ebuild to support pgo the code profiling is performed on separate xorg-server (mind xvfb use flag for x11-base/xorg-server) so the access to real frame buffer device(s) is actually not needed complete explanation could be found in here: http://forums.gentoo.org/viewtopic-t-887296-start-0.html
(In reply to comment #29) > Created attachment 311715 [details, diff] [details, diff] > proposed fix for pgo ff ebuild (12.0) > > attached snippet patches firefox-12.0.ebuild to support pgo > > the code profiling is performed on separate xorg-server (mind xvfb use flag > for x11-base/xorg-server) so the access to real frame buffer device(s) is > actually not needed > > complete explanation could be found in here: > http://forums.gentoo.org/viewtopic-t-887296-start-0.html Moving the exports is not the answer, if exported in pkg_setup() they are suppose to be carried threw all phases of the ebuild.
(In reply to comment #30) > > Moving the exports is not the answer, if exported in pkg_setup() they are > suppose to be carried threw all phases of the ebuild. i just wanted to say it works for me (the original ebuild does not). definitely i'm not familiar with all portage rules and hints
Please don't just add the x86@g.o alias unless you know why. In this case you do, please leave a comment why x86 would need to follow here. We do keywording.
*** Bug 420149 has been marked as a duplicate of this bug. ***
*** Bug 421125 has been marked as a duplicate of this bug. ***
PGO useflag has been masked until such a time we can properly support it in gentoo. Soon as we can we will unmask and support pgo builds for all users. At this time even upstream is having problems supporting pgo builds in Linux. I will continue to track there upstream to determine when best we can come back and review all pgo build failures. Please ignore resolution we no longer have an option for later.
