Bug 171123 - [2.6.20 regression] tifm sd/mmc driver broken
Bug#: 171123 Product:  Gentoo Linux Version: unspecified Platform: AMD64
OS/Version: Linux Status: RESOLVED Severity: normal Priority: P2
Resolution: FIXED Assigned To: dsd@gentoo.org Reported By: gentoo_bugzilla@kbc.net.au
Component: Core system
URL:  http://bugzilla.kernel.org/show_bug.cgi?id=8052
Summary: [2.6.20 regression] tifm sd/mmc driver broken
Keywords:  
Status Whiteboard: linux-2.6.20-regression
Opened: 2007-03-16 05:17 0000
Description:   Opened: 2007-03-16 05:17 0000
Using identical kernel configuration for MMC/SD support, the upgrade to the
2.6.60 series kernels stops a listing appearing in /dev for the MMC/SD reader.

Boot back into the 2.6.19 kernel, it works again.

Reproducible: Always

Steps to Reproduce:
1. Boot with 2.6.20 kernel.
2. Plug in MMC/SD card
2. Look in /dev

Actual Results:  
No mention of MMC/SD device in /dev

Expected Results:  
MMC/SD device should appear in /dev.

Kernel config files:

Working:
http://www.smiffysplace.com/files/kernel_config_2.6.19_20070316

Non-working:
http://www.smiffysplace.com/files/kernel_config_2.6.20_20070316

Dmesg output:
http://www.smiffysplace.com/files/dmesg_20070316

The latter lines show me inserting and removing the card a couple of times,
with 2.6.20-gentoo-r2, nothing appearing in /dev.

------- Comment #1 From Jakub Moc (RETIRED) 2007-03-16 08:16:54 0000 -------
Please, attach all the files here; third party links render bugs useless in
days.

------- Comment #2 From Smiffy 2007-03-16 08:19:02 0000 -------
Created an attachment (id=113442) [details]
Kernel config, 2.6.19

------- Comment #3 From Smiffy 2007-03-16 08:19:45 0000 -------
Created an attachment (id=113443) [details]
Kernel config, 2.6.20

------- Comment #4 From Smiffy 2007-03-16 08:20:06 0000 -------
Created an attachment (id=113445) [details]
Dmesg output

------- Comment #5 From Smiffy 2007-03-16 08:21:10 0000 -------
Files attached as requested.

------- Comment #6 From Daniel Drake 2007-03-24 23:59:12 0000 -------
Which device node are you looking for?

You seem to have both sdhci and tifm hardware. Do you know which one you are
trying to use?

Can you boot 2.6.19, and run "dmesg". Then plug a card in, ensure the device
node is created, run "dmesg" again and paste here which messages have been
added to the end since you plugged the card in. Then do the same for 2.6.20.

------- Comment #7 From Smiffy 2007-03-25 00:28:09 0000 -------
(In reply to comment #6)
> Which device node are you looking for?
These are what appear with 2.6.19:

brw-rw---- 1 root disk  254,    0 Mar 23 15:50 mmcblk0
brw-rw---- 1 root disk  254,    1 Mar 23 15:50 mmcblk0p1

> You seem to have both sdhci and tifm hardware. Do you know which one you are
> trying to use?
Sorry, I wouldn't know.  I just enabled it in the kernel and it just worked in
2.6.19.

> Can you boot 2.6.19, and run "dmesg". Then plug a card in, ensure the device
> node is created, run "dmesg" again and paste here which messages have been
> added to the end since you plugged the card in. Then do the same for 2.6.20.

I have done this, diff'ing the results, and am submitting attachments
dmesg_19.diff and dmesg_20.diff, respectively.

------- Comment #8 From Smiffy 2007-03-25 00:30:04 0000 -------
Created an attachment (id=114308) [details]
Booted to 2.6.19, difference between dmesg without card and dmesg with card.

Note: Although this shows errors, card can be mounted and accessed without
problems.

------- Comment #9 From Smiffy 2007-03-25 00:30:57 0000 -------
Created an attachment (id=114309) [details]
Booted to 2.6.20, difference between dmesg without card and dmesg with card.

------- Comment #10 From Daniel Drake 2007-03-25 00:49:38 0000 -------
Nothing obvious there. Can you reproduce this on the latest development kernel,
currently 2.6.21-rc4?

------- Comment #11 From Smiffy 2007-03-25 01:48:13 0000 -------
(In reply to comment #10)
> Nothing obvious there. Can you reproduce this on the latest development kernel,
> currently 2.6.21-rc4?

No gentoo-sources package available for amd64 post 2.6.20-r3.  

Looks like I'll have to pull down source from kernel.org to do this, so may
take a while to get it configured and working on this laptop.  I will report
back when I have been able to do this.

------- Comment #12 From Daniel Drake 2007-03-25 02:04:12 0000 -------
gentoo-sources doesn't do -rc releases but switching to a -rc is generally easy
as gentoo-sources doesn't include much.

emerge -n ketchup
cd /usr/src
mkdir -p linux-2.6.21-rc4
cd linux-2.6.21-rc4
ketchup 2.6.21-rc4

------- Comment #13 From Smiffy 2007-03-25 05:37:16 0000 -------
Thanks for the guidance on ketchup.  I have now built 2.6.21-rc4, the problem
persists: no device nodes are created when the card is inserted.  Full dmesg
and post-insertion diff to be attached.

------- Comment #14 From Smiffy 2007-03-25 05:38:02 0000 -------
Created an attachment (id=114318) [details]
2.6.21-rc4 dmesg, card not inserted

------- Comment #15 From Smiffy 2007-03-25 05:39:04 0000 -------
Created an attachment (id=114320) [details]
diff between dmesg before and after card insertion, kernel 2.6.21-rc4

------- Comment #16 From Daniel Drake 2007-03-25 19:25:03 0000 -------
Thanks for the quick testing. The next step is to file a bug upstream against
the tifm driver at http://bugzilla.kernel.org.

Mention that 2.6.19 works, 2.6.20 breaks, and 2.6.21-rc4 is still broken.

As this is a regression you can track down the exact buggy commit which
introduced the bug, however it is a fairly time consuming process. If you're
keen, see
http://www.reactivated.net/weblog/archives/2006/01/using-git-bisect-to-find-buggy-kernel-patches/
You might want to wait for 2-3 days after filing the bug before going down this
route, in case it is not needed.

------- Comment #17 From Daniel Drake 2007-03-25 19:25:25 0000 -------
please post the new bug URL here when you have filed it upstream

------- Comment #18 From Smiffy 2007-03-25 21:23:30 0000 -------
I did a quick check on the kernel Bugzilla before opening a new one and found
one open (but nothing happening) which I guess is the same. I have, therefore,
posted against: http://bugzilla.kernel.org/show_bug.cgi?id=8052

Thanks for your help, Daniel, I just hope someone up there knows how to fix
this otherwise I'm going to be trying to shoehorn 2.6.19 code back into the new
kernel.

------- Comment #19 From Daniel Drake 2007-03-25 22:14:10 0000 -------
Your 2.6.21 logs show that mmcblk0 and mmcblk0p1 are being detected, although
there are a fair number of errors too. Do those device nodes really not appear
under 2.6.21 or has the problem changed with this kernel release?

------- Comment #20 From Smiffy 2007-03-25 22:59:17 0000 -------
(In reply to comment #19)
> Your 2.6.21 logs show that mmcblk0 and mmcblk0p1 are being detected, although
> there are a fair number of errors too. Do those device nodes really not appear
> under 2.6.21 or has the problem changed with this kernel release?

I have now re-emerged all necessary packages so that I can run 2.6.21-r4 as my
default, to save all the re-booting.

Whilst dmesg shows mmc stuff when card inserted (it did on 2.6.20, too), I can
confirm that nodes are still NOT found in /dev. 

------- Comment #21 From Daniel Drake 2007-04-27 22:58:29 0000 -------
Created an attachment (id=117456) [details]
tifm v0.8d for 2.6.21

Please test gentoo-sources-2.6.21. If it's still broken there, apply this patch
on top of gentoo-sources-2.6.21 and see if it helps.

------- Comment #22 From Smiffy 2007-04-28 05:48:52 0000 -------
Device node is not created with linux-2.6.21-gentoo either before or after
patching.

Eject and insert events can be seen on dmesg - this is very similar to the way
it was on 2.6.20 prior to using the external module build from the bug on
kernel.org

------- Comment #23 From Daniel Drake 2007-04-29 14:31:37 0000 -------
Strange, I thought that patch should bring it in line with the external
version. I'll check this soon.

In the mean time, could you double check that you applied the patch correctly
and installed the new kernel/module successfully?

If you compile the driver as a module, you can check with "modinfo
/lib/modules/<VERSION>/kernel/drivers/mmc/tifm_sd.ko" -- the version field
should read 0.8, not 0.6

If you compile the driver into the kernel, check the last modified time of
drivers/mmc/tifm_sd.c in the kernel directory -- it should be equal to when you
applied the patch. If so, while booted into the kernel you believe is patched,
run "uname -v". The value there is the time/date that kernel was compiled --
mentally check this is when you recompiled the kernel, and also check that it
is newer than the modified time for the tifm_sd.c file.

------- Comment #24 From Smiffy 2007-04-30 04:58:11 0000 -------
Yep, this is version 0.8.  Just for reference, here's the modinfo for all in
that directory:

filename:       /lib/modules/2.6.21-gentoo/kernel/drivers/mmc/mmc_block.ko
license:        GPL
description:    Multimedia Card (MMC) block device driver
vermagic:       2.6.21-gentoo SMP mod_unload
depends:        mmc_core
parm:           major:specify the major device number for MMC block driver
(int)

filename:       /lib/modules/2.6.21-gentoo/kernel/drivers/mmc/mmc_core.ko
license:        GPL
vermagic:       2.6.21-gentoo SMP mod_unload
depends:

filename:       /lib/modules/2.6.21-gentoo/kernel/drivers/mmc/tifm_sd.ko
author:         Alex Dubov
description:    TI FlashMedia SD driver
license:        GPL
version:        0.8
vermagic:       2.6.21-gentoo SMP mod_unload
depends:        mmc_core,tifm_core
srcversion:     C11C3EBB8F4E8ECA33554AC
parm:           fixed_timeout:bool
parm:           no_dma:bool

And this is the one that lurks in ../misc:

filename:       /lib/modules/2.6.21-gentoo/kernel/drivers/misc/tifm_7xx1.ko
author:         Alex Dubov
description:    TI FlashMedia host driver
license:        GPL
version:        0.8
vermagic:       2.6.21-gentoo SMP mod_unload
depends:        tifm_core
alias:          pci:v0000104Cd00008033sv*sd*bc*sc*i*
alias:          pci:v0000104Cd0000803Bsv*sd*bc*sc*i*
alias:          pci:v0000104Cd0000AC8Fsv*sd*bc*sc*i*
srcversion:     D736152CA27D661613954A9
filename:       /lib/modules/2.6.21-gentoo/kernel/drivers/misc/tifm_core.ko
license:        GPL
author:         Alex Dubov
description:    TI FlashMedia core driver
license:        GPL
version:        0.8
vermagic:       2.6.21-gentoo SMP mod_unload
depends:
srcversion:     BE1DF8B52CD61BF90C0A9A2

------- Comment #25 From Mike Pagano 2007-05-01 19:14:54 0000 -------
Created an attachment (id=117883) [details]
diff of comment 21 patch against upstream .08d

patched 2.6.21 (with patch from comment #21) and then diff'ed them against
tifm_7xx1.c tifm_core.c tifm_sd.c and tifm.h from external version 0.8d from
upstream

------- Comment #26 From Mike Pagano 2007-05-01 19:55:04 0000 -------
Created an attachment (id=117885) [details]
Patch for 2.6.21 to upstream version 08d

Smiffy,

Can you please apply this patch to your 2.6.21 source and test?

Thanks,
Mike

------- Comment #27 From Mike Pagano 2007-05-01 21:09:43 0000 -------
Created an attachment (id=117890) [details]
Patch for 2.6.21 to upstream version 08d

Smiffy,

Can you please ignore the last patch and instead apply this patch to your
2.6.21 source and test?

Thanks,
Mike

------- Comment #28 From Mike Pagano 2007-05-02 00:13:15 0000 -------
Created an attachment (id=117908) [details]
Patch for 2.6.21 to upstream version 08d

This one can be properly applied with -p1 

------- Comment #29 From Smiffy 2007-05-02 09:18:29 0000 -------
I applied the patch to a fresh emerge by entering /usr/src/linux-2.6.21-gentoo
and running:

patch -p1 tifm-update-to-0.8d.patch

This reported that the files had been patched successfully.

However, when I came to make the kernel, I got the following error:

  CC [M]  drivers/misc/tifm_core.o
  CC [M]  drivers/misc/tifm_7xx1.o
  CC [M]  drivers/mmc/mmc.o
  CC [M]  drivers/mmc/mmc_sysfs.o
  CC [M]  drivers/mmc/mmc_queue.o
  LD [M]  drivers/mmc/mmc_core.o
  CC [M]  drivers/mmc/mmc_block.o
  CC [M]  drivers/mmc/tifm_sd.o
drivers/mmc/tifm_sd.c: In function 'tifm_sd_probe':
drivers/mmc/tifm_sd.c:987: error: 'struct mmc_host' has no member named
'max_sectors'
make[2]: *** [drivers/mmc/tifm_sd.o] Error 1
make[1]: *** [drivers/mmc] Error 2
make: *** [drivers] Error 2

I will attach my .config, in case this has any bearing on the matter, although
I'm sure that I have seen this error before, when I tried to apply the 2.6.20
patch to 2.6.21.  (I may be wrong, but I have seen it before somewhere.)

------- Comment #30 From Smiffy 2007-05-02 09:20:11 0000 -------
Created an attachment (id=117931) [details]
Kernel .config to go with report on latest compliation attempt.

------- Comment #31 From Mike Pagano 2007-05-02 11:56:55 0000 -------
thanks for the .config

It also failed for me. I am going to try applying this patch to 2.6.21.1 and
see what happens.

------- Comment #32 From Mike Pagano 2007-05-02 12:26:00 0000 -------
Created an attachment (id=117948) [details]
Patch for 2.6.21.1 to upstream version 08e

Here's the patch to upstream version 0.8e

This does compile when applied to 2.6.21.1, but not 2.6.21

------- Comment #33 From Smiffy 2007-05-02 22:23:18 0000 -------
Will 2.6.21.1 be released as a 2.6.21-gentoo-rx ebuild?  If so, would this
patch be already applied?

------- Comment #34 From Mike Pagano 2007-05-03 00:07:25 0000 -------
Smitty,

I was incorrect when I said the version 08e patch doesn't compile against
linux-2.6.21-gentoo. It did compile for me.

Can you apply the patch "Patch for 2.6.21.1 to upstream version 08e" to
linux-2.6.21, and test the result?

Thanks

------- Comment #35 From Smiffy 2007-05-03 00:32:32 0000 -------
gentoo-sources now re-emerged and patched with tifm-update-to-0.8e.patch

The kernel build was successful, but the device node still does not appear. 
Modules appear to be loaded (from lsmod):

tifm_sd                13320  0
mmc_core               27208  1 tifm_sd
tifm_7xx1               8576  0
tifm_core              11144  2 tifm_sd,tifm_7xx1

However, if I run modinfo, version only comes back as 0.8 - is this correct or
should it say 0.8e?

------- Comment #36 From Daniel Drake 2007-05-03 01:34:40 0000 -------
That's correct. The plot thickens...

Can you revert that patch, remove tifm from your kernel config, remove the
modules from /lib/modules, and then install and test the external tifm-0.8e
against 2.6.21?

Direct link to download is
http://prdownload.berlios.de/tifmxx/tifm-0.8e.tar.bz2

Basically we need to try and figure out what's going on.. the patch I posted
(against 2.6.20) should have brought things just about inline with the external
0.8d (which you said worked fine), and Mike's patch above did similar, so its a
bit of a mystery why these patches aren't working if the external versions
are...

------- Comment #37 From Smiffy 2007-05-03 02:52:31 0000 -------
External module for 08e works fine, just like in the case of 2.6.20 and its
external module.

Whether it is relevant or not, the Makefile is broken in that it does not
create /lib/modules/2.6.21-gentoo/kernel/drivers/misc/ and
/lib/modules/2.6.21-gentoo/kernel/drivers/mmc which breaks make install.  I
have to create these by hand first, then run make install.

------- Comment #38 From Nick Devito 2007-05-13 01:58:38 0000 -------
I've also been having issues with tifm (mainly buffer and I/O errors in dmesg,
corrupt files, etc). I tried applying the 2.6.21.1 patch posted by Mike to
gentoo-sources-2.6.21 and it has fixed my issue. If this could get into
gentoo-sources-2.6.21-r1, that would be great. (It does appear that they are
planning to merge it into 2.6.22, and I think tifm 0.8 is in the kernel git
tree, I'd have to check though).

------- Comment #39 From Nick Devito 2007-05-13 05:38:10 0000 -------
(In reply to comment #38)
> I've also been having issues with tifm (mainly buffer and I/O errors in dmesg,
> corrupt files, etc). I tried applying the 2.6.21.1 patch posted by Mike to
> gentoo-sources-2.6.21 and it has fixed my issue. If this could get into
> gentoo-sources-2.6.21-r1, that would be great. (It does appear that they are
> planning to merge it into 2.6.22, and I think tifm 0.8 is in the kernel git
> tree, I'd have to check though).
> 

Err, I should clarify which patch I'm talking about: Patch for 2.6.21.1 to
upstream version 08e (attachment #117948 [details])

------- Comment #40 From Nick Devito 2007-05-13 06:30:00 0000 -------
It seems that tifm 0.8e has been merged into 2.6.22-rc1. I gave it a try here,
and it seems to have resolved the issue. 

------- Comment #41 From Daniel Drake 2007-05-13 15:05:08 0000 -------
Smiffy, any chance you could test 2.6.22-rc1?

------- Comment #42 From Daniel Drake 2007-05-13 23:52:59 0000 -------
Yes, vanilla kernel with the appropriate options selected

------- Comment #43 From Smiffy 2007-05-14 00:03:39 0000 -------
I should be able to test 2.6.22-rc1 in the next couple of days - will advise
results.

------- Comment #44 From Smiffy 2007-05-14 00:25:47 0000 -------
I've started the configuration of the new kernel, but there appears to be a
problem in make menuconfig:

<*> MMC/SD card support  --->

This does not expand to offer the driver options as it does in the previous
kernels, so I am unable to fulfill:

"If you want MMC support, you should say Y here and also to the specific driver
for your MMC interface."

------- Comment #45 From Smiffy 2007-05-14 00:31:19 0000 -------
Ignore previous, this has been hidden under Misc Devices for some reason.

------- Comment #46 From Nick Devito 2007-05-14 00:33:49 0000 -------
(In reply to comment #45)
> Ignore previous, this has been hidden under Misc Devices for some reason.
> 

I think it was in Misc devices in 2.6.19 as well ;-)

------- Comment #47 From Smiffy 2007-05-14 02:42:49 0000 -------
OK, firstly, 2.6.22-rc1 works.  However, to get it to work, I had to modprobe
something called mmc_block, which I had not had to do before.  (I only found it
whilst I was having a nose around /lib/modules/----/kernel/drivers/; I
modprobed it to see what it did and all of a sudden it worked...)

Having discovered this, I rebooted to my (patched) 2.6.21-gentoo, did the same
modprobe and found it worked there too.  Please note, I did not have to do this
modprobe with the external module in 2.6.21 or 2.6.20.  Could this be the
missing link?

NOTE:  make menuconfig on 2.6.22-rc1 doesn't set CONFIG_MMC_TIFM_SD - don't
know if this is another parameter hidden in some daft place or whether it has
been missed.  I had to go back and set this manually to build the module and
get this working.

@nick - there were three things I spotted that had been moved around
menuconfig; made it a helluva job to copy my .21 to my .22 configuration. 
Something's also been broken in the SATA AHCI driver so that I have to change
the BIOS if I want to swap between 2.6.22 and Solaris (2.6.21 and Solaris, no
problems).  When someone puts GNU+Solaris kernel+Portage, I'll be there like a
shot ;-)

------- Comment #48 From Nick Devito 2007-05-14 04:54:21 0000 -------
Just a quick question, what do you have to change in the BIOS when you switch
between 2.6.22 and Solaris? (I'm just kind of curious)

------- Comment #49 From Smiffy 2007-05-14 06:42:39 0000 -------
Comment #48 answered at length, off-list.

------- Comment #50 From Daniel Drake 2007-05-18 22:50:15 0000 -------
OK, thanks for the testing. Time to backport the patches from 2.6.22. This
won't make it into the gentoo-sources-2.6.21 release being generated right now,
but hopefully will make it into the next one.

------- Comment #51 From Smiffy 2007-05-19 23:39:13 0000 -------
I have just upgraded to kernel 2.6.21-gentoo-r1 (x86_64) and have applied patch
tifm-update-to-0.8e.patch.  I can confirm that my card reader is WORKING.

------- Comment #52 From Daniel Drake 2007-05-25 02:41:20 0000 -------
Fixed in gentoo-sources-2.6.21-r2 (genpatches-2.6.21-3), thanks for testing.