Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 929240 - >sys-kernel/gentoo-sources-6.6.30 .. 6.9.0 : reboot and shutdown freeze
Summary: >sys-kernel/gentoo-sources-6.6.30 .. 6.9.0 : reboot and shutdown freeze
Status: RESOLVED UPSTREAM
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Kernel Bug Wranglers and Kernel Maintainers
URL: https://gitlab.freedesktop.org/drm/i9...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-04-12 20:04 UTC by mrpingo
Modified: 2024-06-30 19:49 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
dmesg log (dmesg,21.04 KB, text/plain)
2024-04-12 21:23 UTC, mrpingo
Details
kernel config (config-6.8.5,162.71 KB, text/plain)
2024-04-12 21:24 UTC, mrpingo
Details
dmesg log from kernel 6.8.5 (dmesg-6.8.5,65.13 KB, text/plain)
2024-04-12 21:34 UTC, mrpingo
Details
x390 rc.log (rc.log,542.17 KB, text/plain)
2024-04-13 09:43 UTC, mrpingo
Details
x390 dmesg (dmesg,73.65 KB, text/plain)
2024-04-13 09:44 UTC, mrpingo
Details
x390 syslog (syslog,40.48 KB, text/plain)
2024-04-13 09:45 UTC, mrpingo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description mrpingo 2024-04-12 20:04:11 UTC
Hello,

I am having an issue while executing suspend-to-ram since sys-kernel/gentoo-sources-6.6.23 till 6.8.5 with serial8250 that is not configured as a module, it is included as "*" in the kernel config:

[  128.464762] port serial8250:0.0: PM: dpm_run_callback(): pm_runtime_force_suspend+0x0/0x140 returns -16
[  128.464770] port serial8250:0.0: PM: failed to suspend: error -16
[  131.131373] port serial8250:0.0: PM: dpm_run_callback(): pm_runtime_force_suspend+0x0/0x140 returns -16
[  131.131381] port serial8250:0.0: PM: failed to suspend: error -16

All kernels before "6.6.23" work as expected, none of the mayor versions work suspend-to-ram at all with the same config, I mean "6.7.*" and "6.8.*".

Dont know if something changed in the kernel past this version, didn't change any config so far related to this issue and also cant find anything on the web related to this issue timed recently.

I have 2 thinkpads, one x270 and x390, this happens on both of them but the upper version of the x390 that works suspend-to-ram is 6.6.26. Noone seems to have been reported this issue so my guess is that is in my side.

I guess this is a needed module and should not disable it.

Attached dmesg log, ask for anything you want me to share/try.


Many thanks.
Comment 1 Jonas Stein gentoo-dev 2024-04-12 20:40:59 UTC
I could not see your attachments (dmesg log).
https://wiki.gentoo.org/wiki/Attach_the_logs_to_the_bug_ticket

Lets see if the kernel team can help please attach your kernel config in the mean time.
Comment 2 mrpingo 2024-04-12 21:23:21 UTC
Created attachment 890580 [details]
dmesg log
Comment 3 mrpingo 2024-04-12 21:24:17 UTC
Created attachment 890581 [details]
kernel config
Comment 4 mrpingo 2024-04-12 21:24:49 UTC
sorry for that.

done.
Comment 5 mrpingo 2024-04-12 21:26:38 UTC
(In reply to mrpingo from comment #0)
> Hello,
> 
> I am having an issue while executing suspend-to-ram since
> sys-kernel/gentoo-sources-6.6.23 till 6.8.5 with serial8250 that is not
> configured as a module, it is included as "*" in the kernel config:
> 
> [  128.464762] port serial8250:0.0: PM: dpm_run_callback():
> pm_runtime_force_suspend+0x0/0x140 returns -16
> [  128.464770] port serial8250:0.0: PM: failed to suspend: error -16
> [  131.131373] port serial8250:0.0: PM: dpm_run_callback():
> pm_runtime_force_suspend+0x0/0x140 returns -16
> [  131.131381] port serial8250:0.0: PM: failed to suspend: error -16
> 
> All kernels before "6.6.23" work as expected, none of the mayor versions
> work suspend-to-ram at all with the same config, I mean "6.7.*" and "6.8.*".
> 
> Dont know if something changed in the kernel past this version, didn't
> change any config so far related to this issue and also cant find anything
> on the web related to this issue timed recently.
> 
> I have 2 thinkpads, one x270 and x390, this happens on both of them but the
> upper version of the x390 that works suspend-to-ram is 6.6.26. Noone seems
> to have been reported this issue so my guess is that is in my side.
> 
> I guess this is a needed module and should not disable it.
> 
> Attached dmesg log, ask for anything you want me to share/try.
> 
> 
> Many thanks.

6.6.23 is working as expected, all of them after this one come with this issue.
Comment 6 mrpingo 2024-04-12 21:33:29 UTC
first dmesg is form kernel 6.6.26.
I have now attached dmesg from 6.8.5

Many thanks
Comment 7 mrpingo 2024-04-12 21:34:10 UTC
Created attachment 890582 [details]
dmesg log from kernel 6.8.5
Comment 8 mrpingo 2024-04-13 09:41:01 UTC
I forgot to mention that both laptops hangs when shutting down with those kernels, although nothing related is logged on dmesg, syslog, rc.log, etc.

sharing those logs for completion.
Comment 9 mrpingo 2024-04-13 09:43:20 UTC
Created attachment 890722 [details]
x390 rc.log

x390 rc.log after hang when shutting down
Comment 10 mrpingo 2024-04-13 09:44:02 UTC
Created attachment 890723 [details]
x390 dmesg

x390 dmesg after hang when shutting down
Comment 11 mrpingo 2024-04-13 09:45:09 UTC
Created attachment 890724 [details]
x390 syslog

x390 syslog after hang when shutting down
Comment 12 mrpingo 2024-04-13 09:48:48 UTC
By hang I mean it shuts-down included the screen (which is turned off) but the power button has the light on and the block mayus key is blinking indefinitely.

That happens on both laptops x270 and x390.
Comment 13 mrpingo 2024-04-13 11:35:52 UTC
I removed:

Symbol: SERIAL_8250 [=n]

seems that everything that usually need works on both laptops, and probably I do not need that module for anything, also suspend-to-ram now works on gentoo-sources-6.8.5 on both computers, although shutdown still hangs on final stage on both.

How can I log everything while shutting down especially kernel messages? seems that nothing is recorded/shown on logs/tty12 between services shutting down and kernel final messages while shutting down hardware, because still hangs when screen is completely off and I am not able to see anything.

Shall I edit the description of the bug?


Regards
Comment 14 mrpingo 2024-04-13 11:59:33 UTC
Just found on ubuntu same behaviour that I am experiencing:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2059738

looks like it is a kernel bug(?) and not specifically gentoo kernel bug(?).

Still able to do tests if you want some debug.


Regards
Comment 15 mrpingo 2024-05-15 11:37:47 UTC
Today and again, compiled gentoo-sources-6.9.0 and both laptops x270 and x390 still hangs on the last point of shutdown.

Everything seems to properly shuts down (screen, keyboard, ...) but in the end both laptops still powered; x270 have power light switched on, x390 bloq mayus light is blinking. I still have to press both laptops power button to do a completely poweroff.

Should I report this upstream to bugs.kernel.org?

I found people on internet with same problem but different distributions like fedora, ubuntu, ...


Best
Comment 16 Mike Pagano gentoo-dev 2024-05-15 16:16:13 UTC
Is Wake-On-LAN was enabled in the BIOS?
Comment 17 mrpingo 2024-05-16 13:36:13 UTC
Thanks for your quick answer,

I thought it was disabled im x270 from the last time I checked, I think it was something related with the BIOS upgrade thingy but I disabled it and still have the same issue. x390 has nothing enabled related with WoL. So most probably is not WoL.

Upgraded kernel to 6.6.30 that is the last one that I dont have this freezing in the last part of the shutdown/reboot process on both computers. Both (x270 and x390 thinkpads) have the same issue with kernels from 6.7.x -> 6.9. I then think that something has changed on those range of kernel versions in the shutdown process.

I dont know if I am explaining the issue correctly, hope you understand.

I am still open to do test and ideas.

Best
Comment 18 Mike Pagano gentoo-dev 2024-05-16 19:07:25 UTC
(In reply to mrpingo from comment #17)
> Thanks for your quick answer,
> 
> I thought it was disabled im x270 from the last time I checked, I think it
> was something related with the BIOS upgrade thingy but I disabled it and
> still have the same issue. x390 has nothing enabled related with WoL. So
> most probably is not WoL.
> 
> Upgraded kernel to 6.6.30 that is the last one that I dont have this
> freezing in the last part of the shutdown/reboot process on both computers.
> Both (x270 and x390 thinkpads) have the same issue with kernels from 6.7.x
> -> 6.9. I then think that something has changed on those range of kernel
> versions in the shutdown process.
> 
> I dont know if I am explaining the issue correctly, hope you understand.
> 
> I am still open to do test and ideas.
> 
> Best

That helps.  

Can you do a git bisect between 6.6.30 and a failing kernel ?  The closer to 6.6.30 the easier your life will be.
Comment 19 mrpingo 2024-05-17 18:01:18 UTC
Aha!

Thanks Mike! You just told me how to get there.

I dressed as a surgeon, did a master on git, took a scalpel  and started to dig. After 15 iterations (config, compile, install and reboot the kernel) I got this:
_______________________________________________________________________________
❯ git bisect good
e046d156249140711a08ff27473591c74ef90814 is the first bad commit
commit e046d156249140711a08ff27473591c74ef90814
Author: Ville Syrjälä <ville.syrjala@linux.intel.com>
Date:   Tue Aug 29 14:39:17 2023 +0300

    drm/i915/hdmi: Use connector->ddc everwhere
    
    We already populate connector->ddc for HDMI ports, but
    so far we've not taken full advantage of it. Do that by
    eliminating a bunch of intel_gmbus_get_adapter() lookups.
    
    Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
    Link: https://patchwork.freedesktop.org/patch/msgid/20230829113920.13713-10-ville.syrjala@linux.intel.com
    Reviewed-by: Jani Nikula <jani.nikula@intel.com>

 drivers/gpu/drm/i915/display/intel_ddi.c  |  3 +--
 drivers/gpu/drm/i915/display/intel_hdmi.c | 37 +++++++++++--------------------
 2 files changed, 14 insertions(+), 26 deletions(-)'
_____________________________________________________________________________


 Now, what I should do? I have no idea, and never got this far looking for a "bad commit", but I think this is just related to me? Nobody had this issue? What module should I disable (please, not Intel GPU) to see if the laptops shutdown/reboot without issues?

Many many thanks already!
Comment 20 mrpingo 2024-05-26 19:18:42 UTC
I would report it upstream if you consider it. The sooner we take action the sooner they patch it.

Best
Comment 21 mrpingo 2024-05-31 11:13:20 UTC
For completeness you can find the upstream issue here:

https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/11244

Many thanks to everyone.


Regards
Comment 22 Mike Pagano gentoo-dev 2024-06-30 19:49:58 UTC
We'll follow the upstream bug and backport any fixes identified.