I've just noticed that both my desktop and laptop get stuck at 100% CPU at exactly 00:00:00 until the display updates, which is a full minute when seconds display is turned off. Can reproduce this on both machines by setting the clock back to 23:59:55. 50-60% of the CPU time is in X and the rest is the xfce4-panel process - for some reason it doesn't go above 100% on SMP though.
Try xfce4-panel-4.7.4 (and finally 4.7.6) with rest of ~arch version of Xfce packages. If it's still a problem, reopen this bug and report it also to: http://bugzilla.xfce.org/ Xfce 4.6 is no longer maintained (neither by upstream or Gentoo). Everyones working on getting Xfce 4.8 ready to be released (and finally stabilized) January/Febuary.