Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 299222 - Please mark media-tv/mythtv-0.22-p23069 stable
Summary: Please mark media-tv/mythtv-0.22-p23069 stable
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Highest normal (vote)
Assignee: Doug Goldstein (RETIRED)
URL:
Whiteboard:
Keywords: STABLEREQ
Depends on: 298439
Blocks: 280303 283429
  Show dependency tree
 
Reported: 2010-01-01 17:28 UTC by Richard Freeman
Modified: 2010-05-11 19:26 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Freeman gentoo-dev 2010-01-01 17:28:14 UTC
Just logging this as a blocker for the qt-3 removal.  Looks like there are lots of possible stable candidates out there, but I'll let the maintainer pick one appropriate for a STABLEREQ.
Comment 1 Markos Chandras (RETIRED) gentoo-dev 2010-01-15 19:27:19 UTC
Ping here
Comment 2 Richard Freeman gentoo-dev 2010-01-21 17:07:38 UTC
I'm happy to test/keyword on amd64, but I need some guidance on what version will be the stable candidate.  Also - you might consider a news item on the database changes unless that has been mitigated somehow.
Comment 3 Jesse Adelman 2010-01-22 03:59:42 UTC
Been running 0.22_p23069 and some extras:
mythtvbox recordings # eix -I mythtv
[I] media-tv/mythtv
     Available versions:  0.21_p19961-r2 0.21_p20877 (~)0.22_p22778 (~)0.22_p22811 (~)0.22_p22824 (~)0.22_p22824-r1 (~)0.22_p22860 (~)0.22_p23069 [M](~)0.23_alpha22834 [M](~)0.23_alpha22857 {aac alsa altivec autostart css debug directv dvb faad fftw ieee1394 jack lcd lirc mmx opengl perl pulseaudio python tiff vdpau video_cards_nvidia video_cards_via xvmc}
     Installed versions:  0.22_p23069(15:47:31 01/06/10)(alsa css debug faad fftw lcd lirc mmx perl python tiff -altivec -autostart -directv -dvb -ieee1394 -jack -pulseaudio -vdpau -video_cards_nvidia -video_cards_via -xvmc)
     Homepage:            http://www.mythtv.org
     Description:         Homebrew PVR project

[I] x11-themes/mythtv-themes
     Available versions:  0.21_p16505 (~)0.22_p22783 (~)0.22_p22793 (~)0.22_p22869
     Installed versions:  0.22_p22869(17:14:14 01/06/10)
     Homepage:            http://www.mythtv.org
     Description:         A collection of themes for the MythTV project.

[I] x11-themes/mythtv-themes-extra
     Available versions:  0.21_p18657 (~)0.22_p22492
     Installed versions:  0.22_p22492(17:14:42 01/06/10)
     Homepage:            http://www.mythtv.org
     Description:         A collection of themes for the MythTV project.

on (mostly) x86-stable. I'm stuck at gentoo-sources-2.6.30-r8 (not a big deal) until LIRC works with 2.6.32. I run a mixed FE/BE with a PVR-350 doing the encoding and decoding. Cheers!
Comment 4 Ben de Groot (RETIRED) gentoo-dev 2010-01-26 02:55:35 UTC
As the maintainer has so far not specified a version, I am assuming we are going for the latest version in testing:

media-tv/mythtv-0.22_p23069
x11-themes/mythtv-themes-0.22_p22869
x11-themes/mythtv-themes-extra-0.22_p22492

target KEYWORDS="amd64 ppc x86"

Doug, can you please ACK and add arches?
Comment 5 Richard Freeman gentoo-dev 2010-01-26 14:13:28 UTC
Before any archs mark this stable we should REALLY consider having a news item or documentation page about the mysql UTF-8 issues.  That is, unless we can confirm that this is no longer an issue.  The write-up on Doug's blog wasn't bad as a start if it is still accurate.  There is also some info on the mythtv wiki I think.

I've been running the stable mythtv for 1-2 years now, so if the issue exists I should be hit by it.  I can see how it goes once we confirm the stable candidate.
Comment 6 Dale Pontius 2010-01-26 17:08:26 UTC
I'll second the request for careful explanation of how to manage the mysql utf8/latin1 issues.  I've read the various posts and blogs until my head is spinning.  I'm particularly concerned because my mostly-Gentoo myth cluster has one Ubuntu client, I've diddled (dropped/reloaded) my database in the past as suggested in some Gentoo (forums?) source, and I've wandered the various mailing lists and blogs on the whole corrupt database issue.

This needs to be carefully managed, in order to make sure we know that we need to fix our databases, how (concisely explained, preferably in one spot, not 5 contradictory ones) to do it, and all before mythtv suddenly gets upgraded to 0.22 without our being prepared.
Comment 7 Ben de Groot (RETIRED) gentoo-dev 2010-02-08 18:42:15 UTC
As Doug gave me the go ahead on IRC, I am requesting arches to stable the ebuilds mentioned in my previous comment (#4 above). 

And if anyone wants to write a howto about the migration, you are very welcome, as the maintainer currently doesn't have the time.
Comment 8 Doug Goldstein (RETIRED) gentoo-dev 2010-02-09 04:11:28 UTC
works for me. To be honest I'll probably mark it stable on all arches cause the arch teams rarely respond within any reasonable time. I just need the time...

So beandog or tanderson... wanna stable the packages and all the plugins?
Comment 9 Thomas Anderson (tanderson) (RETIRED) gentoo-dev 2010-02-09 13:18:27 UTC
Sure Doug.

It seems the new mythzoneminder is not compatible with the latest in-tree zoneminder, but we can keep mythzoneminder keyworded for now.
Comment 10 Richard Freeman gentoo-dev 2010-02-09 14:18:08 UTC
I HIGHLY recommend publishing a HOWTO or a news item regarding the utf8 issues, unless they no longer exist.  Otherwise everybody who just does an emerge -u world is going to end up with a ruined mythtv database with no easy recovery path.  

Does the utf8 issue even exist any longer?  There wasn't a great deal of documentation about the issue.  If there is a document already out there that indicates what needs to be done I'm happy to try it out to make sure it works.

I won't get burned any which way - I'll be doing a quickpkg and a full database backup and plan for some testing before allowing this package to get updated regardless.  However, others may not do the same and the whole point of running a stable system is so that you don't get burned by stuff like this...
Comment 11 Tom Dexter 2010-02-09 14:50:40 UTC
When upgrading from 0.21 to 0.22 under Gentoo when you've been running with the default Gentoo configuration (with utf8) you _will_ need to fix the database first using the steps described here:

http://wiki.mythtv.org/wiki/Fixing_Corrupt_Database_Encoding

It is possible to do this ahead of time, which I did (I haven't upgraded to 0.22 yet), but that's not the recommended approach.  The reason is that, if you fix the database and plan to continue running 0.21 at all, you MUST change your my.cnf on BOTH the front and backends to latin1, or you can cause the "partial corruption" described in the wiki.  However if you do it when you're ready to upgrade, 0.22 will work fine with your mysql configured for utf8.
Comment 12 Christian Faulhammer (RETIRED) gentoo-dev 2010-02-09 14:51:56 UTC
So this definitely needs a news item and maybe an upgrade guide.
Comment 13 Joe Jezak (RETIRED) gentoo-dev 2010-02-09 20:37:50 UTC
I'm fine with it going stable on ppc once there is an upgrade guide. I use it on ppc as a frontend with no real issues. If you need us to test any specific libs, please let us know.
Comment 14 Dale Pontius 2010-02-11 20:30:05 UTC
There is discussion in the forum about this, particularly database corruption issues.  Near the end of the thread, tld is working on a script to detect corruption issues prior to attempting the upgrade.  It's still a work in progress, but will be incredibly valuable for reducing the problems with this upgrade.  (Asking for trouble-free is probably going to be too much, on this one.)

Here's the thread: http://forums.gentoo.org/viewtopic-p-6168337.html
Comment 15 Doug Goldstein (RETIRED) gentoo-dev 2010-02-12 06:45:59 UTC
(In reply to comment #14)
> There is discussion in the forum about this, particularly database corruption
> issues.  Near the end of the thread, tld is working on a script to detect
> corruption issues prior to attempting the upgrade.  It's still a work in
> progress, but will be incredibly valuable for reducing the problems with this
> upgrade.  (Asking for trouble-free is probably going to be too much, on this
> one.)
> 
> Here's the thread: http://forums.gentoo.org/viewtopic-p-6168337.html
> 

mythtv-setup, which you must run before starting the mythbackend when upgrading (unless of course you don't follow the instructions and don't care and want to corrupt your data... but hey.. that has been the standard operating procedure since MythTV 0.9), will detect the situation in which the database will be corrupted and throw up an error. It will also point you to the correct instructions to follow to fix the situation. It was really quite easy and have had a few people walk through it.

As I've said before, I don't have time to write up an entire upgrade document. If anyone wants to write it, they can feel free. Just note, x11-libs/qt-3 is being removed in 9 days.. which means Gentoo will no longer have a stable MythTV.

Comment 16 Christian Faulhammer (RETIRED) gentoo-dev 2010-02-12 07:23:35 UTC
If nothing unrecoverable happens, then you can go on with stabilisation for the arches you have, cardoe.  Maybe a news item is still a good idea (no full guide, but a pointer to an external source).
Comment 17 Richard Freeman gentoo-dev 2010-02-12 10:40:21 UTC
My understanding is that if you DON'T run the manual database fixes (which will be required for anybody who has been running stable on gentoo for more than a year or two), you end up with a mix of text encodings in the database, and that is pretty-much unrecoverable (without a lot of manual cleanup).

If the init.d for mythbackend did some kind of check that mythtv-setup was run then I'd say this is a non-issue, but I don't believe that it does.  If somebody just does an emerge -u world and reboots or whatever they're going to end up messing up their database.  Anybody running mythtv is going to have it in their runlevels and that means that an after-the-fact warning could easily be a warning-too-late.  Plus, the database cleanup isn't a 3-minute fix so people might want to plan ahead.

I think that a news item BEFORE keywording would be sensible, unless we really think that anybody running mythtv would be following this stuff.  Otherwise I'd say that having no stable mythtv is preferable to potential breakage.  No stable mythtv just means that people are going to get lots of portage error messages - I'd think that most would prefer that to a partial database corruption.  

However, a news item in 9 days shouldn't be to big of a deal - especially since we apparently have reasonable documentation online.
Comment 18 Tom Dexter 2010-02-12 14:43:06 UTC
The instructions on the MythTV wiki about fixing the database are as detailed as you're going to get.  There's certainly no need to reinvent that wheel.

One thing that I think there's a misunderstanding on here however:  As I understand it, MythTV will attempt to run the database schema update (if needed) in either mythtv-setup _or_ mythtvbackend.  It's just recommended to use mythtv-setup.  There is no situation where the backend will start without a necessary upgrade.

In either case the fix instructions should be followed first.  However I believe (though I'm not sure) that anyone whose been running with utf8 client connections will most likely have a database that will fail the upgrade tests if they don't do the backup/restore fix first.  I have backups from before I did the backup/restore fix and after, and in my case the tests fail prior to the fix and succeed after.
Comment 19 Dale Pontius 2010-02-15 21:21:22 UTC
I might suggest as a precaution that people who have had long-running myth systems begin with backups of data, obviously, but also code.  I would also suggest that long-time users might mask 0.22 for a while, in order to take this slowly and carefully, searching and reading available instructions.

There have in the past been suggestions in either news or forums to do a "backup/drop/latin1/reload" which I have followed, and then seen my.cnf go back to utf8 later.  I've also had an Ubuntu client connected to my Gentoo server.  Between these two items, it's very possible that I have a partially corrupted or perhaps unfixable database.  Neither of these are unusual scenarios, and both may create database problems.

With that in mind I'm busy transcoding everything I can, getting it safe before even attempting the upgrade.  I still want to upgrade my database, but my expectations of getting there are not very high.  With a qpkg of 0.21, I could go back there for a bit, if needed.

Obviously there's no reason for new installs not to be 0.22, but some people face possible loss of large amounts of data.  This is aggravated with Gentoo in particular because of the attempt to handle my.cnf "correctly".  (My impression is that other distributions tweaked their installs away from the the upstream utf8 to latin1, as a matter of course.)
Comment 20 Ben de Groot (RETIRED) gentoo-dev 2010-02-16 16:37:19 UTC
CC'ing QA

QA: with the looming Qt3 mask it looks like mythtv users will be left without a stable version. The consensus seems to be that a news item is needed wrt the database migration for 0.22 as a prerequisite for stabling this (Qt4) version. Please advise on the path forward.
Comment 21 Christian Faulhammer (RETIRED) gentoo-dev 2010-02-17 09:05:44 UTC
Please prepare the news item, a little guide is found here: http://www.faulhammer.org/archiv-mainmenu-31/35-gentoo/282-writing-a-news-item
Comment 22 Tom Dexter 2010-02-17 14:47:10 UTC
I upgraded to the versions of mythtv, mythtv-themes, and mythtv-themes-extra above and also media-plugins/mythvideo-0.22_p22991 and www-apps/mythweb-0.22_p22763-r1 on Sunday (on stable x86 backend and frontend), and used it extensively since...all seems just great.

On thing I did notice is that mythweb requires dev-perl/Net-UPnP which is still ~x86.

In addition to the warnings about the database issues, it might be worth noting in the news item that mythweb now requires that php be compiled with the json use flag, which it didn't in 0.21.  That aborted my first upgrade attempt.
Comment 23 Christian Faulhammer (RETIRED) gentoo-dev 2010-02-22 14:16:02 UTC
Dale, can you provide a short announcement with a link to the upgrade guide?  I will then take it through the news item process within Gentoo.
Comment 24 Dale Pontius 2010-02-22 14:49:02 UTC
(In reply to comment #23)
> Dale, can you provide a short announcement with a link to the upgrade guide?  I
> will then take it through the news item process within Gentoo.
> 
This morning I put a post onto this thread, where I attempted to balance between following official directions and putting my own windage and experience into it.  Is this or a pointer to it something like what you want?  I need to go back and read my old news items to get an idea of the wording, etc.
Comment 25 Richard Freeman gentoo-dev 2010-02-22 14:58:36 UTC
Here is my proposal (also posted to -dev/pr per the glep):

Title: MythTV 0.22 Upgrade Database Corruption
Author: Richard Freeman <rich0@gentoo.org>
Content-Type: text/plain
Posted: <date>
Revision: 1
News-Item-Format: 1.0
Display-If-Installed: <media-tv/mythtv-0.22

Due to an incompatibility between MythTV 0.21 and the default Gentoo MySQL configuration, it is likely that long-time MythTV users will have databases with a mixture of locale encodings.  If you upgrade to 0.22 without following these directions carefully, you could end up with a database that contains errors that are extremely difficult to fix.

Please see the MythTV Upgrade Guide for instructions:

    http://wiki.mythtv.org/wiki/Fixing_Corrupt_Database_Encoding

Be sure to save a database backup before upgrading.  Also, be sure to upgrade any other clients/backends you are using to 0.22 at the same time.  The upgrade instructions need to be followed once per database - individual client/backend upgrades do not require these steps.
Comment 26 Dale Pontius 2010-02-22 15:29:09 UTC
I like what you've said.  I have to add that I found a problem with the "partial corruption upgrade" instructions from the list archives.  I've highlighted this problem here : http://forums.gentoo.org/viewtopic-p-6181527.html#6181527 and here : http://forums.gentoo.org/viewtopic-p-6181183.html#6181183

Basically, Step #4 of the "partial corruption" instructions on the mailing list archives omits the "--partial_restore" flag, where the previous backup is loaded overtop of the "blank" 1214 database snapshot.  My first upgrade attempt, where I omitted this flag, failed.  I found out about the flag while reading the "database backup and restore" document, added it to Step #4 on my second upgrade attempt, and it worked.

Now that I think of it, it's worth mentioning that the plugins separately upgrade their sections of the database, which has 2 problems of its own.  First, that the user will see the "Upgrading" panel a second time, which is normal.  Second on my system I had some permissions problems on my /media partition, giving problems when running the upgrade process as non-root tried to make a backup of the database.

I'm not sure how to incorporate this into the news item, other than perhaps as a pointer into the forums.  I see the addition of the "--partial_restore" flag to Step #4 as being critical.
Comment 27 Richard Freeman gentoo-dev 2010-02-22 16:20:42 UTC
Do we really want to try to post a guide for partial database corruptions?  I suspect that the typical stable user would not have one of those.  

Right now the news item does not link to any of the list postings that contain the error you mention, so I'm not sure it is appropriate to bring those into the news item.

My concern is that there is a great wiki page that exists RIGHT NOW that will fix the issue for 98% of users.  If we don't post a news item and stable a particular release, then in a few days the whole package will get masked.  Then users might end up doing their own upgrades without any guidance and that will mean that nearly 100% of them will end up with issues (unless they run into the upgrade guide).

If a clear howto exists on fixing partial corruptions, it probably would make sense just to add it to the MythTV wiki, and then link it off of the article I put in the news item.  However, that sounds like a bit of a daunting task to try to accomplish in a few days.
Comment 28 Ben de Groot (RETIRED) gentoo-dev 2010-02-22 16:49:33 UTC
(In reply to comment #27)
> However, that sounds like a bit of a daunting task to
> try to accomplish in a few days.

As long as we are assured that somebody is actively working on this, the Qt team is willing to postpone the mask for a reasonable time.

Comment 29 Dale Pontius 2010-02-22 17:58:39 UTC
(In reply to comment #28)
> (In reply to comment #27)
> > However, that sounds like a bit of a daunting task to
> > try to accomplish in a few days.
> 
> As long as we are assured that somebody is actively working on this, the Qt
> team is willing to postpone the mask for a reasonable time.
> 
I can go with this.  My only concerns are threefold:

1 - The existing guide is quite clear for the regular case, but I feel it's a bit murky for the partially corrupt case.  Please read it yourself and see if you agree.

2 - I believe that the mailing-list directions for fixing the partially corrupt case are missing the "--partial_restore" flag, that the omission is critical, and without that flag for me the upgrade failed.

3 - The longer a Gentoo user has been running MythTV, (0.19 for me, iirc) the more likely he is to have run across the "database fix" instructions I followed, and after following those instructions, partial corruption becomes much more likely.  I don't think it's going to be a rare event for longtime Gentoo mythtv users.

I posted my findings on the mythtv list about the missing flag on Step 4, but the guy who was helping hasn't answered me.  I need to repost, asking to verify that the flag is indeed necessary.

Nothing like completing the upgrade and finding 150GB of video missing.
Comment 30 Richard Freeman gentoo-dev 2010-02-22 18:10:20 UTC
(In reply to comment #29)
> 1 - The existing guide is quite clear for the regular case, but I feel it's a
> bit murky for the partially corrupt case.  Please read it yourself and see if
> you agree.

Agreed - but it does clearly instruct on how to distinguish between the two, and it recommends seeking advice on-list for partial corruptions. 

> 
> 2 - I believe that the mailing-list directions for fixing the partially corrupt
> case are missing the "--partial_restore" flag, that the omission is critical,
> and without that flag for me the upgrade failed.

Is there a single link that is formated clearly for somebody new to the situation to read that explains what to do?  I don't mind adding to the end of the news item "If while following the guide it becomes clear that you have a partial corruption you should consider the following link in addition to the mailing lists..."

> 
> 3 - The longer a Gentoo user has been running MythTV, (0.19 for me, iirc) the
> more likely he is to have run across the "database fix" instructions I
> followed, and after following those instructions, partial corruption becomes
> much more likely.  I don't think it's going to be a rare event for longtime
> Gentoo mythtv users.

If a user has actually followed instructions from the lists I don't think they're really the target of this news item, which is users that would otherwise have no idea that they're about to be in for a ride.

In any case, the qt team really needs a commitment to action.  I can commit to revising and getting the news item out, but I can't commit to howtos or anything more serious than what already exists.  So what we need is:

1.  To move forward as-is.
2.  To have a link to an already-existing howto or other guide for the partial corruption case.
3.  To have a serious commitment by somebody willing to create such a guide within a reasonable period of time (days to maybe a week or two it sounds - with demonstrable progress if we really want 1-2 weeks).
Comment 31 Dale Pontius 2010-02-22 18:53:38 UTC
(In reply to comment #30)
> Is there a single link that is formated clearly for somebody new to the
> situation to read that explains what to do?  I don't mind adding to the end of
> the news item "If while following the guide it becomes clear that you have a
> partial corruption you should consider the following link in addition to the
> mailing lists..."

In the case where it has been established that the database has partial corruption, does my second post on this forum thread look like an acceptable HowTo, or starting point for one?  I agree that if all goes well, they're better off sticking to the official document.  If they've already failed to upgrade because of partial corruption, the extra "normal" steps at the front of my instructions shouldn't faze them.  I will however agree that sometimes my prose leaves something to be desired.

http://forums.gentoo.org/viewtopic-t-816566-highlight-.html
Comment 32 Richard Freeman gentoo-dev 2010-02-22 19:53:53 UTC
Ok, how about I add the following to the news item.

Additionally, a gentoo forum thread exists with useful information about this upgrade, including tips for addressing partial database corruptions should you encounter one:

   http://forums.gentoo.org/viewtopic-t-816566-highlight-.html

Comment 33 Dale Pontius 2010-02-22 20:24:58 UTC
(In reply to comment #32)
> Ok, how about I add the following to the news item.
> 
> Additionally, a gentoo forum thread exists with useful information about this
> upgrade, including tips for addressing partial database corruptions should you
> encounter one:
> 
>    http://forums.gentoo.org/viewtopic-t-816566-highlight-.html
> 

That works for me.  Besides, I'm on notification for that thread, and I'll be checking in there, helping people out as time permits.
Comment 34 Dale Pontius 2010-02-23 13:17:11 UTC
Just when we thought it was safe to go back in the water, and this whole mythtv upgrade issue was handled, Gentoo news handed me something this morning.

mysql-5.1 upgrade is imminent.  It mentions "database conversions" as perhaps being necessary.

I also took a few moments to do some searches, and find that if "-O3" is used when compiling mysql-5.1.40 (and thereabouts) it will fail with mythtv, as well as some other clients.  I don't think this is a common problem, since Gentoo generally uses "-O2", but it might be worth a warning for the ricers.

Anyway, the presence of these two significant and interrelated upgrades at about the same time is, at the very least, annoying.  I'll try to find a block of time to test the mysql-5.1 upgrade.  If there's no hurry to push mysql-5.0 off to the bit-bucket it might be good to advise mythtv users to mask the upgrade.
Comment 35 Tom Dexter 2010-02-23 15:06:16 UTC
Just so others reading this don't get as confused as i did about your "mysql-5.1 upgrade is imminent" comment...mysql-5.1 was NOT marked stable, but simply had the hard mask removed.
Comment 36 Dale Pontius 2010-02-23 15:33:23 UTC
Thanks for the clarification.  I guess I was confused, too.  That gives us a few months for mythtv-0.22 to settle down before we have to worry about the database, again.
Comment 37 Doug Goldstein (RETIRED) gentoo-dev 2010-02-23 16:07:52 UTC
(In reply to comment #25)
> Here is my proposal (also posted to -dev/pr per the glep):
> 
> Title: MythTV 0.22 Upgrade Database Corruption
> Author: Richard Freeman <rich0@gentoo.org>
> Content-Type: text/plain
> Posted: <date>
> Revision: 1
> News-Item-Format: 1.0
> Display-If-Installed: <media-tv/mythtv-0.22
> 
> Due to an incompatibility between MythTV 0.21 and the default Gentoo MySQL
> configuration, it is likely that long-time MythTV users will have databases
> with a mixture of locale encodings.  If you upgrade to 0.22 without following
> these directions carefully, you could end up with a database that contains
> errors that are extremely difficult to fix.
> 
> Please see the MythTV Upgrade Guide for instructions:
> 
>     http://wiki.mythtv.org/wiki/Fixing_Corrupt_Database_Encoding
> 
> Be sure to save a database backup before upgrading.  Also, be sure to upgrade
> any other clients/backends you are using to 0.22 at the same time.  The upgrade
> instructions need to be followed once per database - individual client/backend
> upgrades do not require these steps.
> 

As far as I know, this is wrong because if you attempt to "convert" on an already converted database, you will corrupt your data. So really, like I've said before... run mythtv-setup or mythbackend and if it tells you to convert then convert. Otherwise do nothing. mythbackend will simply fail to start and print a message in its log file while mythtv-setup will print out a message the user will easily see as to why things are going wrong. mythtv-setup is the required step to be run before starting mythbackend.

If you can't follow instructions then I really don't have sympathy for you.
Comment 38 Tom Dexter 2010-02-23 21:44:36 UTC
(In reply to comment #37)
> (In reply to comment #25)
> 
> 
> As far as I know, this is wrong because if you attempt to "convert" on an
> already converted database, you will corrupt your data. So really, like I've
> said before... run mythtv-setup or mythbackend and if it tells you to convert
> then convert. Otherwise do nothing. mythbackend will simply fail to start and
> print a message in its log file while mythtv-setup will print out a message the
> user will easily see as to why things are going wrong. mythtv-setup is the
> required step to be run before starting mythbackend.
> 
> If you can't follow instructions then I really don't have sympathy for you.
>
 
I'm a bit unclear as to what you mean by an "already converted database".  If by that you mean someone like myself, who converted the database ahead of time (while still running 0.21) and switched mysql to latin1 (as per the wiki), then it's impossible to run the fix procedure again, as your backups will already have "SET NAMES latin1".

To further complicate matters however, it was my experience that running the fix ahead of time and continuing to use 0.21 in that manner, causes corruption in the database.  I've tested copies of my database from backups before running the fix, after running the fix, and after running 0.21 with the fixed database (with latin1 connections) for a few months.  Of those three the only one that would pass the upgrade test was the one immediately after the fix.  The pre-fix database would not, and running the fixed database with 0.21 exactly as the wiki recommends caused me significant partial corruption.
Comment 39 Richard Freeman gentoo-dev 2010-02-23 23:41:51 UTC
(In reply to comment #37)
> As far as I know, this is wrong because if you attempt to "convert" on an
> already converted database, you will corrupt your data. 

Why would you attempt to convert an already-converted database?  Section 2 in the linked guide is titled "Determining if your database is misconfigured" - if it isn't misconfigured the start of the next section clearly indicates that following the remaining instructions will corrupt your database.

> So really, like I've
> said before... run mythtv-setup or mythbackend and if it tells you to convert
> then convert. Otherwise do nothing. 

Perhaps you should convince upstream to update their wiki if it is incorrect?

Several here have posted that they've followed the wiki instructions and it has worked for them.  I'm not comfortable committing a news item that hasn't been tested and which conflicts with the upstream instructions.

Do you have proposed wording on a replacement news item?  I don't want to make the wording unnecessarily complicated, but if you feel it should be clarified I'm certainly open to it.  
Comment 40 Doug Goldstein (RETIRED) gentoo-dev 2010-02-24 06:10:55 UTC
(In reply to comment #39)
> (In reply to comment #37)
> > As far as I know, this is wrong because if you attempt to "convert" on an
> > already converted database, you will corrupt your data. 
> 
> Why would you attempt to convert an already-converted database?  Section 2 in
> the linked guide is titled "Determining if your database is misconfigured" - if
> it isn't misconfigured the start of the next section clearly indicates that
> following the remaining instructions will corrupt your database.
> 
> > So really, like I've
> > said before... run mythtv-setup or mythbackend and if it tells you to convert
> > then convert. Otherwise do nothing. 
> 
> Perhaps you should convince upstream to update their wiki if it is incorrect?
> 
> Several here have posted that they've followed the wiki instructions and it has
> worked for them.  I'm not comfortable committing a news item that hasn't been
> tested and which conflicts with the upstream instructions.
> 
> Do you have proposed wording on a replacement news item?  I don't want to make
> the wording unnecessarily complicated, but if you feel it should be clarified
> I'm certainly open to it.  
> 

TBH, you're more than welcome to join the MythTV herd and take over maintenance of MythTV. Like I've said over a dozen times now it seems, I do not have the time right now and will not for the next few months. 
Comment 41 Richard Freeman gentoo-dev 2010-02-24 16:08:46 UTC
(In reply to comment #40)
> TBH, you're more than welcome to join the MythTV herd and take over maintenance
> of MythTV. Like I've said over a dozen times now it seems, I do not have the
> time right now and will not for the next few months. 
> 

Short term - lets just do whatever we need to do to resolve the immediate crisis.

Longer term - I don't personally feel comfortable taking over sole maintainership of myth, but I do run it on a multi-system setup and I can certainly try to help out.  If there are others interested in taking on maintaining this (gentoo devs, or non-devs willing to help out somewhat consistently) we might be able to form a small team to do so.  My main concern is that I couldn't be as active in maintaining experimental versions as you have been.  If somebody else wants to take the lead role by all means do so, but if not we can at least solicit those interested in helping out and see what we can do to organize ourselves.
Comment 42 Markos Chandras (RETIRED) gentoo-dev 2010-02-24 16:12:58 UTC
I am currently setting up a multimedia server running mythtv and stuff, so I might be able to help as well maintaining this package
Comment 43 Richard Freeman gentoo-dev 2010-02-26 03:06:15 UTC
See gentoo-dev for the latest iteration of the news item.  My plan is to do a round of testing on Sat, and send out the news item if there are no serious objections and all goes well.  Then on Sunday I plan to stabilize on amd64.
Comment 44 Jesse Adelman 2010-02-26 04:45:42 UTC
Just a thumbs-up status report here. From my FE/BE x86-stable (mostly) box, with PVR-350 endocer/decoder (SD Only system), X11 XV output, LIRC hauppague remote, and MyBlaster serial IR Blaster, 897 GB used storage.

     Wed Jan  6 15:47:43 2010 >>> media-tv/mythtv-0.22_p23069
       merge time: 15 minutes and 12 seconds.

mythtvbox src # uname -a
Linux mythtvbox 2.6.30-gentoo-r8 #1 Mon Feb 1 22:24:35 PST 2010 i686 AMD Athlon(tm) XP 2800+ AuthenticAMD GNU/Linux

(Holding back kernel update until LIRC works with 2.6.32-*)
Running well since then. No upgrade issues. However, I noticed that there have been updates to 22-fixes since this release, and I hope that either that goes stable, or is marked ~x86, along with the plugins and mythweb. Thanks!
Comment 45 Richard Freeman gentoo-dev 2010-02-27 20:13:12 UTC
Ok, testing went reasonably well (although I might explicitly state in the news item to save a backup USING MYSQLDUMP as the mythtv-provided script didn't save my entire database - that could have been a real mess).  Only took me two hours (mostly spent compiling and then figuring out that half my tables were missing).  The built-in sanity checks did seem to work.

Plan is to send out the news item in a day or two, and then stabilize a day or two after that on amd64.  
Comment 46 Tom Dexter 2010-02-27 21:58:43 UTC
(In reply to comment #45)
> Ok, testing went reasonably well (although I might explicitly state in the news
> item to save a backup USING MYSQLDUMP as the mythtv-provided script didn't save
> my entire database - that could have been a real mess).
Are you sure about that?  What is it that you think it missed?  The MythTV backup script is simply doing a mysqldump of the database.  It's very widely used...as a matter of fact, I used it when I did my backup/restore fix, and I'd certainly know if it missed anything.  the only thing I can see in the backup script that would ever be selective in any way is the --backup_xmltvids option.
Comment 47 Richard Freeman gentoo-dev 2010-02-27 22:08:56 UTC
(In reply to comment #46)
> Are you sure about that?  

Absolutely.  

> What is it that you think it missed?  

Looks like it stopped somewhere in the p's, backing up tables alphabetically.  The missing recorded table was a good tipoff.

>  The MythTV
> backup script is simply doing a mysqldump of the database.  It's very widely
> used...as a matter of fact, I used it when I did my backup/restore fix, and I'd
> certainly know if it missed anything.  the only thing I can see in the backup
> script that would ever be selective in any way is the --backup_xmltvids option.
> 

Yup - not sure why it died.  No errors were generated, but I did notice that it didn't compress the output.  I didn't pick up on that since I've never used that script before.  

When I ran mythtv-setup it couldn't figure out the database schema version, and it couldn't update the schema.  I couldn't figure out what the problem was until I ran a test perl script posted in the partial corruption thread.  It happened to noticed that I didn't have a recorded table.  

Apparently their script died partway through the backup, and then when I wiped the database and reloaded it I was missing half of my tables.  

Once I realized what was going on I pulled out my manual mysqldump backup (no parameters - just a default dump), and ran the latin1 sed on it and used that to do the restore.  It worked like a charm.  If I hadn't done a manual backup I'd have been hosed.

Maybe there was something in a table that caused mysqldump to abort with all the options the script uses.  No idea, and it wouldn't surprise me if it works fine for most others.  Bottom line is to make sure that the backup is complete.  Greping and counting for the CREATE TABLE lines in the dump is probably an easy way to tell.

Bottom line is that you can never have too many backups...
Comment 48 Richard Freeman gentoo-dev 2010-03-01 16:12:05 UTC
News item committed.  I plan to commit stable on amd64 on Wed, Mar 3rd.
Comment 49 Christian Faulhammer (RETIRED) gentoo-dev 2010-03-03 09:00:46 UTC
Who has a mythtv installation on x86 and can do the stabilisation?
Comment 50 Richard Freeman gentoo-dev 2010-03-03 15:40:35 UTC
Finally!

amd64 stable

also stabilized media-plugins/mytharchive-0.22_p22763
Comment 51 Richard Freeman gentoo-dev 2010-03-03 15:42:17 UTC
(In reply to comment #49)
> Who has a mythtv installation on x86 and can do the stabilisation?
> 

If you're desperate I might be able to test this in a chroot (certainly the frontend won't be hard to test).  I probably won't be able to test the full database migration on x86, however.  Obviously it would be better for somebody on the x86 arch team to handle this - just let me know if you end up needing a hand and I'll do what I can.
Comment 52 Christian Faulhammer (RETIRED) gentoo-dev 2010-03-03 15:51:55 UTC
(In reply to comment #51)
> (In reply to comment #49)
> > Who has a mythtv installation on x86 and can do the stabilisation?
> > 
> 
> If you're desperate I might be able to test this in a chroot (certainly the
> frontend won't be hard to test).  I probably won't be able to test the full
> database migration on x86, however.  Obviously it would be better for somebody
> on the x86 arch team to handle this - just let me know if you end up needing a
> hand and I'll do what I can.

 Cardoe mentioned, that he tested it...so another test from you and it is good to go.  Please do.
Comment 53 Tom Dexter 2010-03-04 13:57:53 UTC
(In reply to comment #52)
>  Cardoe mentioned, that he tested it...so another test from you and it is good
> to go.  Please do.
> 

Sorry I didn't see this until today.  I'm running x86 and have migrated.  In my case I had developed mixed corruption from running 0.21 for several months after running the backup/restore latin1 conversion (even though my my.cnf was set to all latin1).  I had to correct that myself.  Other than that, the upgrade went flawlessly and everything's been running great.  Also, I've been able to determine that a copy of the database from immediately after the latin1 fix in fact would have upgraded flawlessly...so had I have waited until the upgrade to do the fix I'd have had no issues at all.

Just note however that I don't use vdpau (my card doesn't support it) so I can't speak for that, if it makes any difference.
Comment 54 Richard Freeman gentoo-dev 2010-03-04 14:57:19 UTC
x86 stable on:
media-tv/mythtv-0.22_p23069
x11-themes/mythtv-themes-0.22_p22869
x11-themes/mythtv-themes-extra-0.22_p22492

I was able to test that the front-end works - between that, Doug's testing, and Christian's blessing I'm comfortable marking stable.
Comment 55 Christian Faulhammer (RETIRED) gentoo-dev 2010-03-04 15:14:59 UTC
Thanks.
Comment 56 Joe Jezak (RETIRED) gentoo-dev 2010-05-11 19:26:42 UTC
Marked ppc stable, tested as a front end only. Closing since we're the last arch.