After running into trouble getting Greyhole (http://www.greyhole.net/) working on Gentoo, I bit the bullet and brought up an Ubuntu server to replace a failing NAS box. It's a 12.04 LTS Server box, with Samba 3.6.3 and Greyhole 0.9.26 installed. I pushed a backup of my files from one of my Gentoo boxes to the new server and thought all was good, but I noticed when browsing files on the Ubuntu box that any file with a non-ASCII character in the name was mangled. For instance, in the music directory, names like Björk or Mötley Crüe were had the ö and ü replaced with two-character pairs (for instance, "Björk" would show up as "Björk"). It appeared that "iocharset=utf8" needed to be added to the mount options. I fixed the filenames on the Ubuntu server by mounting the shares at another mount point with "iocharset=iso8859-1" and moving them to the usual mount. Now, though, whenever the Gentoo box tries to enumerate directories in the shares provided by the Ubuntu box, if a directory name has a non-ASCII character in it, it gets duplicated. If you happen to know where a file is located, you can still access it, but it appears that the directory structure is somehow mangled. Yesterday, I Reproducible: Always Steps to Reproduce: 1. mkdir -p /mnt/media/itunes-media/Music/tmp/Björk/Debut 2. touch /mnt/media/itunes-media/Music/tmp/Björk/Debut/foo 3. ls /mnt/media/itunes-media/Music/tmp/Björk/Debut/foo 4. find /mnt/media/itunes-media/Music/tmp 5. find /mnt/media/itunes-media/Music/tmp/Björk 6. find /mnt/media/itunes-media/Music/tmp/Björk/Debut Actual Results: $ ls /mnt/media/itunes-media/Music/tmp/Björk/Debut/foo /mnt/media/itunes-media/Music/tmp/Björk/Debut/foo $ find /mnt/media/itunes-media/Music/tmp /mnt/media/itunes-media/Music/tmp/ /mnt/media/itunes-media/Music/tmp/Björk /mnt/media/itunes-media/Music/tmp/Björk/Björk $ find /mnt/media/itunes-media/Music/tmp/Björk /mnt/media/itunes-media/Music/tmp/Björk /mnt/media/itunes-media/Music/tmp/Björk/Björk $ find /mnt/media/itunes-media/Music/tmp/Björk/Debut /mnt/media/itunes-media/Music/tmp/Björk/Debut /mnt/media/itunes-media/Music/tmp/Björk/Debut/Debut Expected Results: $ ls /mnt/media/itunes-media/Music/tmp/Björk/Debut/foo /mnt/media/itunes-media/Music/tmp/Björk/Debut/foo $ find /mnt/media/itunes-media/Music/tmp /mnt/media/itunes-media/Music/tmp/ /mnt/media/itunes-media/Music/tmp/Björk /mnt/media/itunes-media/Music/tmp/Björk/Debut /mnt/media/itunes-media/Music/tmp/Björk/Debut/foo $ find /mnt/media/itunes-media/Music/tmp/Björk /mnt/media/itunes-media/Music/tmp/Björk /mnt/media/itunes-media/Music/tmp/Björk/Debut /mnt/media/itunes-media/Music/tmp/Björk/Debut/foo $ find /mnt/media/itunes-media/Music/tmp/Björk/Debut /mnt/media/itunes-media/Music/tmp/Björk/Debut /mnt/media/itunes-media/Music/tmp/Björk/Debut/foo
Portage 2.2.0_alpha149 (default/linux/amd64/10.0, gcc-4.5.3, glibc-2.14.1-r3, 3.3.3-gentoo x86_64) ================================================================= System uname: Linux-3.3.3-gentoo-x86_64-Intel-R-_Core-TM-2_Quad_CPU_Q6600_@_2.40GHz-with-gentoo-2.0.3 Timestamp of tree: Thu, 03 Jan 2013 16:30:01 +0000 ld GNU ld (GNU Binutils) 2.21.1 ccache version 3.1.6 [enabled] app-shells/bash: 4.2_p37 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.7.3-r2, 3.2.3-r1 dev-util/ccache: 3.1.6 dev-util/cmake: 2.8.9 dev-util/pkgconfig: 0.27.1 sys-apps/baselayout: 2.0.3 sys-apps/openrc: 0.9.8.4 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.11.1 sys-devel/binutils: 2.21.1-r1 sys-devel/gcc: 4.5.3-r2 sys-devel/gcc-config: 1.5-r2 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r1 sys-kernel/linux-headers: 3.3 (virtual/os-headers) sys-libs/glibc: 2.14.1-r3 Repositories: gentoo local-ebuilds multimedia hamsterrific ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=native -mno-tls-direct-seg-refs" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.4/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe -march=native -mno-tls-direct-seg-refs" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--autounmask-write --quiet-build=y" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs ccache config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://mirrors.tds.net/gentoo/" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j5" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage /var/lib/layman/multimedia /var/lib/layman/hamsterrific" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X a52 aac ac3 acl airplay alsa amd64 apache2 avahi bcmath berkdb bluetooth bluray bzip2 cairo ccache cdda cdparanoia cdr cleartype cli compressed-lumas consolekit corefonts cracklib crypt css cups cupsddk cxx dbus djvu dri dts dv dvb dvd dvdnav dvdr dvdread eac3 ebook encode exif faac faad ffmpeg flac foomaticdb fortran frei0r g3dvl gd gdbm geoip gif gimp git gmp gpm gstreamer gtk hpijs iconv id3tag ieee1394 ilbc ipv6 ivtv java javascript jpeg jpeg2k kde lame lcd libsamplerate lirc logrotate lzma lzo mad matroska melt mjpeg mmx mmxext mng modules mp2 mp3 mp4 mpeg mudflap multilib musicbrainz mysql mysqli ncurses netpbm network nftf nls nptl nsplugin ntfsprogs nvidia odbc offensive ogg ogm opengl openmp pam pcre pdf png pnm policykit posix pppd python qt3support qt4 quicktime readline rplay samba scanner sdl server session slp snmp sox speex spell sql sqlite srt sse sse2 sse3 ssl ssse3 subversion svg system-libvncserver taglib tcpd theora threads threadsafe tiff tk truetype udev unicode upnp usb v4l v4l2 vcd vdpau vlc vorbis webkit wxwidgets x264 xcomposite xinerama xml xterm xv xvid zeroconf zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Should've previewed my original comment, and there appears to be no way to edit it...the final, trailing "Yesterday, I" at the end doesn't belong in there. <strike>/etc/samba.conf from both the Ubuntu server and Gentoo client will be attached shortly. The relevant shares are mounted on both machines with the iocharset=utf8 and mapchars options.</strike> Hmm...googling "cifs mapchars" turned up this: https://bugs.launchpad.net/ubuntu/+source/cifs-utils/+bug/260396 I think I had added the mapchars option in the past to deal with filenames with certain characters in them. Removing this option causes directories to be enumerated properly. The only side effect is that filenames with characters such as : and ? in them get mangled when browsed on Windows hosts, but that would be a deficiency with that platform. I'll see if I can close this issue as invalid.
On second thought, Samba on Ubuntu doesn't exhibit the same directory-enumeration error with mapchars and UTF8 names that Gentoo does. It probably has the patch at https://bugs.launchpad.net/ubuntu/+source/cifs-utils/+bug/260396/comments/12 applied. Will look into applying this patch to Gentoo Samba.
(In reply to comment #3) > On second thought, Samba on Ubuntu doesn't exhibit the same > directory-enumeration error with mapchars and UTF8 names that Gentoo does. > It probably has the patch at > https://bugs.launchpad.net/ubuntu/+source/cifs-utils/+bug/260396/comments/12 > applied. Will look into applying this patch to Gentoo Samba. ...and it appears to actually be a kernel patch, not a Samba patch. Further, it's a patch against 2.6.32. Looking into the kernel source Ubuntu uses now.
Created attachment 334232 [details, diff] bugfix for sys-kernel/gentoo-sources-3.3.3 The return value of cifsConvertToUTF16() is incorrect. It should return j, not i. It's basically the same fix as in https://bugs.launchpad.net/ubuntu/+source/cifs-utils/+bug/260396/comments/12, applied to a newer kernel.
Just upgraded from gentoo-sources 3.3.3 to 3.7.1...looks like somewhere in between, this bug got fixed. Closing this issue again.
