Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 575292 - sys-apps/smartmontools: update-smart-drivedb: drop install of script that overwrites db at runtime
Summary: sys-apps/smartmontools: update-smart-drivedb: drop install of script that ove...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-02-21 13:20 UTC by Sergey S. Starikoff
Modified: 2017-10-31 21:53 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sergey S. Starikoff 2016-02-21 13:20:07 UTC
AFAIR /usr/ subdirectories should not contain files, not registered in portage base.

=sys-apps/smartmontools-6.4 package installs file /usr/share/smartmontools/drivedb.h and /usr/sbin/update-smart-drivedb script, which updates that file to the latest one from SVN-repository.

This script doesn't checks file (neither checksums nor sign) that is already a potential security hole.
If succeed, it updates /usr/share/smartmontools/drivedb.h file breaking digest for files of =sys-apps/smartmontools-6.4 package. That is not rigt.
If failed, AFAIR script lives temporary (unregistered) files. That's also wrong.

I don't see quick and easy solution.
For first time I suggest to exclude updater tool and ask upstream about enchancing update mechanism.
Later — to move /usr/share/smartmontools/drivedb.h into separate package with emerge-time update/validation (something like 6.4 slot for release version of file and 6.4.9999 for latest from SVN).
Comment 1 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2016-02-23 15:35:10 UTC
I'd suggest to entirely remove this script in a revision bump.
After that we can still develop a way to get this functionality integrated in a way that is reasonable for Gentoo.
Comment 2 Sergey S. Starikoff 2016-02-24 13:46:13 UTC
(In reply to Lars Wendler (Polynomial-C) from comment #1)
> I'd suggest to entirely remove this script in a revision bump.
It is what I've suggested in initial comment.
Although, I think it will be incorrect to perform silent change in original package file list. We should install a custom documentation file (for example something like README.Gentoo) with description which utility (script) is skipped and why it was done.

> After that we can still develop a way to get this functionality integrated
> in a way that is reasonable for Gentoo.
AFAIK sys-apps/smartmontools is the not only package, which installs self-update tools. At least net-misc/geoipupdate was mentioned.
Also in my system I see the following suspicious file:
# equery b /usr/sbin/geoipupdate.sh 
 * Searching for /usr/sbin/geoipupdate.sh ... 
dev-libs/geoip-1.6.6 (/usr/sbin/geoipupdate.sh)
(probably all or at least some geoip packages should use the same database, with single update tool)

So, I suggest to use this bug to discuss about Gentoo-way task solution.
After getting some solution — create a tracker bug to list packages, installing self-updater tools.
And a bug for each listed package to monitor communication with upstream.
Comment 3 Joe Kappus 2016-02-24 18:27:24 UTC
It seems broken at the moment as well, probably due to sourceforge shaking things up post-ownership transfer.

update-smart-drivedb -v
Download from branches/RELEASE_6_4_DRIVEDB
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   419  100   419    0     0   4360      0 --:--:-- --:--:-- --:--:--  4364
/usr/share/smartmontools/drivedb.h.error: rejected by /usr/sbin/smartctl, probably no longer compatible

That error file shows a 302 redirect: 

  <h1>302 Found</h1>
  The resource was found at <a href="https://sourceforge.net/p/smartmontools/code/HEAD/tree/branches/RELEASE_6_4_DRIVEDB/smartmontools/drivedb.h?format=raw">https://sourceforge.net/p/smartmontools/code/HEAD/tree/branches/RELEASE_6_4_DRIVEDB/smartmontools/drivedb.h?format=raw</a>;
you should be redirected automatically.
Comment 4 Sergey S. Starikoff 2016-02-25 11:47:44 UTC
(In reply to Joe Kappus from comment #3)
> It seems broken at the moment as well…

It _not_ seems, it is.
But if script does something wrong, it is NOT interesting: does it works as expected or not.

Following quoted error I think about feature request to portage: it would be nice to see a check /usr content for not registered in portage files or files not match with registered digest. Executed at post-merge time, like "metadata-transfer".
Comment 5 SpanKY gentoo-dev 2016-02-25 23:31:15 UTC
(In reply to Sergey S. Starikoff from comment #4)

emerge provides no value in doing content checks in the normal case when overwriting/update packages.  if you want that, use something like `qcheck` instead.
Comment 6 Sergey S. Starikoff 2016-02-26 11:48:06 UTC
(In reply to SpanKY from comment #5)
> emerge provides no value in doing content checks in the normal case when
> overwriting/update packages.

It may be reasonable.
But as for me, looking from common solution it isn't right.

> if you want that, use something like `qcheck` instead.

Thank you.


You've incorrectly changed bug's description.
This bug is not about removing script for runtime update package files.
It is incorrect just to remove upstream-provided feature.
We should develop some mechanism for handling such (by-design updateable) files.
sys-apps/smartmontools is not the only package, which installs such tools.
Comment 7 SpanKY gentoo-dev 2016-02-27 17:20:37 UTC
(In reply to Sergey S. Starikoff from comment #6)

we're well aware of this mechanism and we've opted to not support it.
this is why the hwids package exists.
Comment 8 Sergey S. Starikoff 2016-02-29 12:05:45 UTC
(In reply to SpanKY from comment #7)
> we're well aware of this mechanism and we've opted to not support it.
> this is why the hwids package exists.

Smartmontools' drivedb.h database seems to differ from present content of sys-apps/hwids package
So, we should add it to sys-apps/hwids (possibly, USE-conditionally)?

The next step we could simply skip update-smart-drivedb installation and creating symbolic link instead of installation of /usr/share/smartmontools/drivedb.h file.
Or it will be more correct to ask upstream about adding a build options to use external drivedb instead of bundled?


About the second example: is there similiad database-package for geoips?
Comment 9 SpanKY gentoo-dev 2016-02-29 16:16:32 UTC
(In reply to Sergey S. Starikoff from comment #8)

if the smartdb sees lots of updates and is used by more than this one package, we can consider integrating it.  if neither is true, then we'd just leave it as-is.
Comment 10 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2016-04-18 13:56:16 UTC
commit 1a1f6c1d1327664ccf81805b01cc435a4cd9fc69
Author: Lars Wendler <polynomial-c@gentoo.org>
Date:   Mon Apr 18 15:44:31 2016

    sys-apps/smartmontools: Revbump to fix bugs #575292 and #580424
    
    Don't install drivedb.h into /usr and don't add it to the PM's file
    database (#575292). Fix download location of latest drivedb.h file
    (#580424).
    
    Package-Manager: portage-2.2.28
    Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>


Debian removed this tool (See "See Also" link).

I went a different approach with =sys-apps/smartmontools-6.4-r1 and moved the database file to /var/db/smartmontools. 
pkg_postinst() takes care of installation of the drivedb.h file and thus the package manager doesn't know anything about the file.
That way users can update the file whenever they wish. Furthermore a script in /etc/cron.monthly updates the file every month.

The update script does a syntax check of the new file before it replaces the old file with the new one but I agree that a better check (by checksum or signature) would be preferrable.
Comment 11 Philipp 2016-04-18 15:39:00 UTC
With the new revision, using paludis, I get

>>> Starting src_install
mv: cannot stat '/var/tmp/paludis/sys-apps-smartmontools-6.4-r1/image//var/db/smartmontools/drivedb.h': No such file or directory

!!! ERROR in sys-apps/smartmontools-6.4-r1::gentoo:
!!! In src_install at line 3942
!!! (no message provided)

!!! Call stack:
!!!    * src_install (/var/tmp/paludis/sys-apps-smartmontools-6.4-r1/temp/loadsaveenv:3942)
!!!    * ebuild_f_install (/usr/libexec/paludis/4/src_install.bash:66)
!!!    * ebuild_main (/usr/libexec/paludis/ebuild.bash:675)
!!!    * main (/usr/libexec/paludis/ebuild.bash:698)
Comment 12 Nick Wallingford 2016-04-18 17:03:01 UTC
(In reply to Philipp from comment #11)
> With the new revision, using paludis, I get
> 
> >>> Starting src_install
> mv: cannot stat
> '/var/tmp/paludis/sys-apps-smartmontools-6.4-r1/image//var/db/smartmontools/
> drivedb.h': No such file or directory

I get the same error using portage.
Comment 13 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2016-04-18 17:31:57 UTC
commit 73c52170e46f94aa54947291e1c31249b32326f5
Author: Lars Wendler <polynomial-c@gentoo.org>
Date:   Mon Apr 18 19:25:38 2016

    sys-apps/smartmontools: Keep temporarily empty /var/db/smartmontools directory.

    This should fix isntallation with paludis.

    Package-Manager: portage-2.2.28
    Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>


Please re-sync and try again.
Comment 14 Perfect Gentleman 2016-04-19 00:11:56 UTC
>>> Install smartmontools-6.4-r1 into /tmp/portage/sys-apps/smartmontools-6.4-r1/image/ category sys-apps
mv: cannot stat '/tmp/portage/sys-apps/smartmontools-6.4-r1/image//var/db/smartmontools/drivedb.h': No such file or directory
 * ERROR: sys-apps/smartmontools-6.4-r1::gentoo failed (install phase):
 *   (no error message)
 * 
 * Call stack:
 *     ebuild.sh, line 133:  Called src_install
 *   environment, line 2296:  Called die
 * The specific snippet of code:
 *       mv "${ED}"${db_path}/drivedb.h "${T}" || die;
Comment 15 Tomohide Ishimaru 2016-04-19 04:54:06 UTC
(In reply to Perfect Gentleman from comment #14)
> >>> Install smartmontools-6.4-r1 into /tmp/portage/sys-apps/smartmontools-6.4-r1/image/ category sys-apps
> mv: cannot stat
> '/tmp/portage/sys-apps/smartmontools-6.4-r1/image//var/db/smartmontools/
> drivedb.h': No such file or directory
>  * ERROR: sys-apps/smartmontools-6.4-r1::gentoo failed (install phase):
>  *   (no error message)
>  * 
>  * Call stack:
>  *     ebuild.sh, line 133:  Called src_install
>  *   environment, line 2296:  Called die
>  * The specific snippet of code:
>  *       mv "${ED}"${db_path}/drivedb.h "${T}" || die;

Same error occurred with use flag 'minimal'.
Comment 16 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2016-04-19 06:04:43 UTC
commit 3c73a8c4682e8dc02d69cbbfe073af19da744d4b
Author: Lars Wendler <polynomial-c@gentoo.org>
Date:   Tue Apr 19 08:01:56 2016

    sys-apps/smartmontools: Fixed installation with USE=minimal.
    
    Package-Manager: portage-2.2.28
    Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>


Sorry for all the trouble :-(
Comment 17 Oleh 2016-05-14 07:56:16 UTC
this seems not really fixing things, because now you hiting file collssion with updates:

1. have a "fixed" sartmontools-6.4-r1
2. try update to 6.5
3. get a collision:


>>> Installing (5 of 17) sys-apps/smartmontools-6.5::gentoo
 * checking 28 files for package collisions
 * This package will overwrite one or more files that may belong to other
 * packages (see list below). You can use a command such as `portageq
 * owners / <filename>` to identify the installed package that owns a
 * file. If portageq reports that only one package owns a file then do
 * NOT file a bug report. A bug report is only useful if it identifies at
 * least two or more packages that are known to install the same file(s).
 * If a collision occurs and you can not explain where the file came from
 * then you should simply ignore the collision since there is not enough
 * information to determine if a real problem exists. Please do NOT file
 * a bug report at http://bugs.gentoo.org unless you report exactly which
 * two packages install the same file(s). See
 * http://wiki.gentoo.org/wiki/Knowledge_Base:Blockers for tips on how to
 * solve the problem. And once again, please do NOT file a bug report
 * unless you have completely understood the above message.
 * 
 * package sys-apps/smartmontools-6.5 NOT merged
 * 
 * Detected file collision(s):
 * 
 * 	/var/db/smartmontools/drivedb.h
 * 
 * Searching all installed packages for file collisions...
 * 
 * Press Ctrl-C to Stop
 * 
 * None of the installed packages claim the file(s).
 * 
 * Package 'sys-apps/smartmontools-6.5' NOT merged due to file
 * collisions. If necessary, refer to your elog messages for the whole
 * content of the above message.
Comment 18 Oleh 2016-05-14 08:00:22 UTC
src_install and pkg_postinst logic with moving drivedb.h is broken
Comment 19 Larry the Git Cow gentoo-dev 2017-10-31 21:53:07 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ad8950f967e37dcff1fb1ab9fcd2e0b4001b0a69

commit ad8950f967e37dcff1fb1ab9fcd2e0b4001b0a69
Author:     Thomas Deutschmann <whissi@gentoo.org>
AuthorDate: 2017-10-31 21:51:15 +0000
Commit:     Thomas Deutschmann <whissi@gentoo.org>
CommitDate: 2017-10-31 21:52:56 +0000

    sys-apps/smartmontools: Rev bump to address several problems
    
    - EAPI updated to EAPI 6.
    
    - Depend on sys-freebsd/freebsd-lib[usb] on FreeBSD. (bug #552054)
    
    - Update smartd systemd service to reflect that we don't provide a general
      config file for the service. Users are supposed to overwrite the
      provided service if they want to pass additional
      parameters. (bug #527648)
    
    - We no longer install executable files in /usr/share/{PN}. (bug #622072)
    
    - "minimal" USE flag was replaced by "daemon" USE flag to better indicate
      what the USE flag does.
    
    - "caps" USE flag now requires "daemon" USE flag.
    
    - It is now possible to use the "update_drivedb" USE flag
      alone (i.e. without the "minimal", now "daemon" USE flag). (bug #635700)
    
    - The drive database logic was rewritten: Per default we try to install
      the drive database shipped with the package. If there's already a drive
      database installed we will not replace the database anymore in
      assumption that the user has updated the database in the meantime and
      want to preserve the changes.
      A warning will be displayed in this case with an instruction how to
      update or restore the default drive database shipped with the package.
      The new logic now also works when using binary packages. (bug #575292)
    
    - RDEPENDs on net-misc/curl, net-misc/wget, www-client/lynx and
      dev-vcs/subversion (just one of them is required!) were added when using
      "update_drivedb" USE flag.
    
    Closes: https://bugs.gentoo.org/552054
    Closes: https://bugs.gentoo.org/527648
    Closes: https://bugs.gentoo.org/622072
    Closes: https://bugs.gentoo.org/635700
    Closes: https://bugs.gentoo.org/575292
    Package-Manager: Portage-2.3.13, Repoman-2.3.4

 sys-apps/smartmontools/files/smartd.systemd        |  11 ++
 sys-apps/smartmontools/metadata.xml                |   2 +
 sys-apps/smartmontools/smartmontools-6.5-r1.ebuild | 148 +++++++++++++++++++++
 3 files changed, 161 insertions(+)