124440 2006-02-28 10:27 0000 webapp.eclass calls emerge 2006-12-31 11:29:09 0000 1 1 1 Unclassified Gentoo Linux Eclasses and Profiles unspecified All Other RESOLVED FIXED P1 critical --- 127144 1 antarus@gentoo.org web-apps@gentoo.org anakin.skyw@gmx.de beu@gentoo.org dev-portage@gentoo.org jakub@gentoo.org qa@gentoo.org sgtphou@fire-eyes.org vericgar@gentoo.org wrobel@gentoo.org antarus@gentoo.org 2006-02-28 10:27:45 0000 You cannot call emege in any ebuild or eclass. It is not safe as locking is not used everywhere and there are races in certain areas still. rl03@gentoo.org 2006-02-28 10:28:40 0000 thanks for reporting vapier@gentoo.org 2006-02-28 20:55:32 0000 > You cannot call emege in any ebuild or eclass. It is not safe as locking is > not used everywhere and there are races in certain areas still. not only that, but emerge also utilizes environment pollution to pass information, and the sandbox gets very very pissed stuart@gentoo.org 2006-03-01 01:44:25 0000 (In reply to comment #2) > not only that, but emerge also utilizes environment pollution to pass > information, and the sandbox gets very very pissed Do you have an example demonstrating how this breaks the sandbox? I'm not aware of anyone ever having a problem installing a web-based app because of the interaction between the environment pollution and the sandbox. Best regards, Stu stuart@gentoo.org 2006-03-01 01:47:09 0000 (In reply to comment #0) > You cannot call emege in any ebuild or eclass. It is not safe as locking is > not used everywhere and there are races in certain areas still. I believe we've been able to get away with this so far because webapps are at the bottom of the resolver tree, so to speak. Can you help me out by providing an example of where a missing lock or an existing race condition causes a problem with the way we're doing this? Do you have any suggestions on how we can switch away from calling emerge? Best regards, Stu stuart@gentoo.org 2006-03-01 05:02:27 0000 I have a suggestion that might solve this. We can introduce a new tool called webapp-cleaner. Instead of calling emerge -C from the eclass, we'll call 'webapp-cleaner --check-if-clean-required' instead. This mode will output a message if the user has web-based applications that need removing, and will prompt the user to run 'webapp-cleaner --clean ${PN}' for themselves. webapp-cleaner --clean will remove all installed web-based apps that have *not* been installed using webapp-config, by building up a call to emerge -C. This solution moves the emerge -C call out from the eclass into a tool that the user runs outside Portage. I believe that would satisfy all parties? Best regards, Stu solar@gentoo.org 2006-03-01 05:07:49 0000 I think you can safely unmerge files without the need of using portage at all. It would take a little effort. But not so much. I've been over this with the portage team before about calling emerge from within an ebuild. You are the first guy to take the stance of saying it's a QA problem. While it's not ideal it can be done correctly by exporting the right variables. ciaran.mccreesh@googlemail.com 2006-03-01 09:53:24 0000 (In reply to comment #5) > This solution moves the emerge -C call out from the eclass into a tool that the > user runs outside Portage. I believe that would satisfy all parties? Works for me. wrobel@gentoo.org 2006-04-20 16:03:55 0000 *** Bug 127144 has been marked as a duplicate of this bug. *** rl03@gentoo.org 2006-07-03 13:24:54 0000 Created an attachment (id=90810) webapp-cleaner rl03@gentoo.org 2006-07-03 13:26:17 0000 Created an attachment (id=90811) webapp.eclass_emerge.patch rl03@gentoo.org 2006-07-03 13:28:13 0000 I've attached webapp-cleaner and a corresponding patch to webapp.eclass. If this looks sane, I'd like to commit both soon. rl03@gentoo.org 2006-07-19 15:38:48 0000 webapp-cleaner is part of webapp-config-1.50.15 beu@gentoo.org 2006-11-20 04:48:34 0000 Ping? Any progress on this? beu@gentoo.org 2006-11-20 04:52:01 0000 What I should have noted in comment #13 is that this is still causing problems as the eclass is still calling emerge... stuart@gentoo.org 2006-11-20 05:33:03 0000 We are currently waiting for a new version of webapp-config to be released (plus the 30 days required to get it stable). Best regards, Stu antarus@gentoo.org 2006-11-30 17:53:32 0000 The ~arch version of portage causes locking issues, since the parent emerge locks the vdb, and then in the eclass another emerge is executed, which then blocks waiting for the parent. This is a deadlock condition. This bug blocks the stabling of that version of portage and breaks webapps for ~arch users. Adding dev-portage to the CC Webapp team, how is your replacement coming along? wrobel@gentoo.org 2006-12-05 08:52:52 0000 I restarted webapp-config development now and will try to fix this asap. zmedico@gentoo.org 2006-12-23 03:55:34 0000 (In reply to comment #16) > The ~arch version of portage causes locking issues, since the parent emerge > locks the vdb, and then in the eclass another emerge is executed, which then > blocks waiting for the parent. This is a deadlock condition. As a workaround this, the eclass can remove ${ROOT}/var/db/.pkg.portage_lockfile before calling emerge. Yes, it defeats the whole purpose of the lock, but it's probably a better alternative than the deadlock that users are currently exposed to. ciaran.mccreesh@googlemail.com 2006-12-23 04:14:29 0000 What? That's a horrible idea. This thing needs fixed, not worked around. Are you seriously suggesting adding yet another hack to webapp.eclass??? zmedico@gentoo.org 2006-12-24 18:16:36 0000 (In reply to comment #19) > What? That's a horrible idea. This thing needs fixed, not worked around. Are > you seriously suggesting adding yet another hack to webapp.eclass??? The current behavior is broken, and removing the lock file won't really make it it any worse than it already is. I just want the webapp.eclass maintainers to know all possible options. zmedico@gentoo.org 2006-12-25 00:09:04 0000 In cvs I've added an ewarn message so that users experiencing a deadlock won't have to search in order to find out what went wrong: * This action may result in a deadlock. Please refer to * http://bugs.gentoo.org/show_bug.cgi?id=124440 for more information. rl03@gentoo.org 2006-12-31 11:29:09 0000 done 90810 2006-07-03 13:24 0000 webapp-cleaner webapp-cleaner text/plain IyEvYmluL2Jhc2gKIyBEaXN0cmlidXRlZCB1bmRlciB0aGUgdGVybXMgb2YgdGhlIEdOVSBHZW5l cmFsIFB1YmxpYyBMaWNlbnNlIHYyCiMgJEhlYWRlcjogJAoKUE49CgpBQ1RJT049ClBSRVRFTkQ9 CgpDTUQ9ImVtZXJnZSAtQ2F2IgpXRUJBUFBfRElSPSIvdXNyL3NoYXJlL3dlYmFwcHMiCldFQkFQ UF9DT05GSUc9CgpbWyAteiAke1JDX0dPVF9GVU5DVElPTlN9IF1dICYmIHNvdXJjZSAvc2Jpbi9m dW5jdGlvbnMuc2gKCmZ1bmN0aW9uIGhlbHAoKSB7CgllY2hvICJSZW1vdmUgb2Jzb2xldGUgYW5k IHVudXNlZCB2ZXJzaW9ucyBvZiB3ZWIgYXBwbGljYXRpb25zIgoJZWNobwoJZWNobyAiVXNhZ2U6 IgoJZWNobyAiICAgJDAgW29wdGlvbnNdIFthY3Rpb25dIGFwcF9uYW1lIgoJZWNobwoJZWNobyAi T3B0aW9uczoiCgllY2hvICIgICAtcCwgLS1wcmV0ZW5kIgoJZWNobyAiICAgICAgIEluc3RlYWQg b2YgY2xlYW5pbmcsIHNpbXBseSBzaG93IHRoZSBjb21tYW5kcyB0byBiZSBleGVjdXRlZCIKCWVj aG8KCWVjaG8gIkFjdGlvbnM6IgoJZWNobyAiICAgLVAsIC0tcHJ1bmUiCgllY2hvICIgICAgICAg UmVtb3ZlcyBhbGwgYnV0IHRoZSBsYXRlc3QgdmVyc2lvbiBvZiBhIHBhY2thZ2UuIFNpbWlsYXIg dG8iCgllY2hvICIgICAgICAgZW1lcmdlIC0tcHJ1bmUiCgllY2hvCgllY2hvICIgICAtQywgLS1j bGVhbi11bnVzZWQiCgllY2hvICIgICAgICAgUmVtb3ZlcyBhbGwgdmVyc2lvbnMgb2YgYSB3ZWIg YXBwbGljYXRpb24gdGhhdCBoYXZlIG5vdCBiZWVuIgoJZWNobyAiICAgICAgIGluc3RhbGxlZCBp bnRvIGEgdmlydHVhbCBob3N0IgoJZWNobwoJZWNobyAiICAgLWgsIC0taGVscCIKCWVjaG8gIiAg ICAgICBEaXNwbGF5cyB0aGlzIGhlbHAgbWVzc2FnZSIKCWVjaG8KCWVjaG8gIklmIG11bHRpcGxl IGFjdGlvbnMgYXJlIGdpdmVuLCBvbmx5IHRoZSBhY3Rpb24gc3BlY2lmaWVkIGxhc3Qgd2lsbCIK CWVjaG8gImJlIGV4ZWN1dGVkIgp9CgpmdW5jdGlvbiBzYW5pdHlfY2hlY2tzKCkgewoJV0VCQVBQ X0NPTkZJRz0kKHdoaWNoIHdlYmFwcC1jb25maWcpCglpZiBbICIke1dFQkFQUF9DT05GSUd9eCIg PT0gIngiIF07IHRoZW4KCQllZXJyb3IgIndlYmFwcC1jb25maWcgbm90IGZvdW5kIgoJCWV4aXQg MQoJZmkKCglpZiBbWyAteiAiJHtBQ1RJT059IiBdXTsgdGhlbgoJCWVlcnJvciAiUGxlYXNlIHNw ZWNpZnkgYSB2YWxpZCBhY3Rpb24iCgkJZXhpdCAxCglmaQoKCWlmIFtbIC16ICR7UE59ICYmICR7 QUNUSU9OfSAhPSAiaGVscCIgXV07IHRoZW4KCQllZXJyb3IgIlBsZWFzZSBzcGVjaWZ5IGEgd2Vi IGFwcGxpY2F0aW9uIHRvIGJlIGNsZWFuZWQiCgkJZXhpdCAxCglmaQoKCWlmIFtbICEgLWQgIiR7 V0VCQVBQX0RJUn0vJHtQTn0iIF1dOyB0aGVuCgkJZWVycm9yICIke1BOfSBub3QgZm91bmQiCgkJ ZXhpdCAxCglmaQoKfQoKCmZ1bmN0aW9uIHBydW5lKCkgewoKCWlmIFtbICQobHMgLTEgIiR7V0VC QVBQX0RJUn0vJHtQTn0iIHwgd2MgLWwpID09ICIxIiBdXTsgdGhlbgoJCWVpbmZvICJOb3RoaW5n IHRvIGNsZWFuIgoJCXJldHVybgoJZWxzZQoJCWxvY2FsIEJFU1RfVkVSU0lPTj0kKGxzICR7V0VC QVBQX0RJUn0vJHtQTn0gfCBzb3J0IC1uciB8IGhlYWQgLW4gMSkKCQlmb3IgeCBpbiAkKGxzICR7 V0VCQVBQX0RJUn0vJHtQTn0pOyBkbwoJCQlpZiBbWyAke0JFU1RfVkVSU0lPTn0gIT0gJHt4fSBd XTsgdGhlbgoJCQkJQ01EPSIke0NNRH0gPSR7UE59LSR7eH0iCgkJCWZpCgkJZG9uZQoKCQllaW5m byAiTXVsdGlwbGUgdmVyc2lvbnMgb2YgJHtQTn0gZGV0ZWN0ZWQuIgoJCWlmIFtbIC16ICR7UFJF VEVORH0gXV07IHRoZW4KCQkJZWluZm8gIlJ1bm5pbmcgJHtDTUR9IgoJCQkke0NNRH0KCQllbHNl CgkJCWVpbmZvICJUbyBwcnVuZSwgcnVuIHRoZSBmb2xsb3dpbmcgY29tbWFuZDoiCgkJCWVpbmZv ICIke0NNRH0iCgkJZmkKCWZpCn0KCmZ1bmN0aW9uIGNsZWFuX3VudXNlZCgpIHsKCWxvY2FsIG91 dHB1dD0kKCR7V0VCQVBQX0NPTkZJR30gLS1sdWkgJHtQTn0pCgoJaWYgW1sgLXogJHtvdXRwdXR9 IF1dIDsgdGhlbgoJCWVpbmZvICJOb3RoaW5nIHRvIGNsZWFuIgoJCXJldHVybgoJZWxzZQoJCUNN RD0iJHtDTUR9ID0ke291dHB1dH0iCgkJZWluZm8gIlVudXNlZCB2ZXJzaW9ucyBvZiAke1BOfSBk ZXRlY3RlZC4iCgkJaWYgW1sgLXogJHtQUkVURU5EfSBdXTsgdGhlbgoJCQllaW5mbyAiUnVubmlu ZyAke0NNRH0iCgkJCSR7Q01EfQoJCWVsc2UKCQkJZWluZm8gIlRvIGNsZWFuLCBydW4gdGhlIGZv bGxvd2luZyBjb21tYW5kOiIKCQkJZWluZm8gIiR7Q01EfSIKCQlmaQoJZmkKfQoKZnVuY3Rpb24g cHJvY2Vzc19vcHRzKCkgewoKCWlmIFtbIC16ICQxIF1dOyB0aGVuCgkJaGVscAoJCWV4aXQgMAoJ ZmkKCgl3aGlsZSBbICIkMSsiICE9ICIrIiBdOyBkbwoJCWNhc2UgIiQxIiBpbgoJCQktcHwtLXBy ZXRlbmQpCgkJCQlQUkVURU5EPSJ5ZXMiCgkJCQk7OwoJCQktUHwtLXBydW5lKQoJCQkJQUNUSU9O PSJwcnVuZSIKCQkJCTs7CgkJCS1DfC0tY2xlYW4tdW51c2VkKQoJCQkJQUNUSU9OPSJjbGVhbl91 bnVzZWQiCgkJCQk7OwoJCQktaHwtLWhlbHApCgkJCQlBQ1RJT049ImhlbHAiCgkJCQk7OwoJCQkq KQoJCQkJUE49IiQxIgoJCQkJOzsKCQllc2FjCgoJCXNoaWZ0Cglkb25lCn0KCnByb2Nlc3Nfb3B0 cyAkQAoKc2FuaXR5X2NoZWNrcwoKJHtBQ1RJT059Cg== 90811 2006-07-03 13:26 0000 webapp.eclass_emerge.patch webapp.eclass_emerge.patch text/plain LS0tIC9ob21lL3JsMDMvZ2VudG9vLXg4Ni9lY2xhc3Mvd2ViYXBwLmVjbGFzcwkyMDA2LTA2LTI1 IDAzOjM0OjM2LjAwMDAwMDAwMCArMDAwMAorKysgd2ViYXBwLmVjbGFzcwkyMDA2LTA2LTI1IDAz OjMzOjQ1LjAwMDAwMDAwMCArMDAwMApAQCAtMzcsNiArMzcsNyBAQAogCiBFVENfQ09ORklHPSIk e1JPT1R9L2V0Yy92aG9zdHMvd2ViYXBwLWNvbmZpZyIKIFdFQkFQUF9DT05GSUc9IiR7Uk9PVH0v dXNyL3NiaW4vd2ViYXBwLWNvbmZpZyIKK1dFQkFQUF9DTEVBTkVSPSIke1JPT1R9L3Vzci9iaW4v d2ViYXBwLWNsZWFuZXIiCiAKICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCiAjIElOVEVSTkFMIEZVTkNUSU9O IC0gVVNFRCBCWSBUSElTIEVDTEFTUyBPTkxZCkBAIC00OTYsMjIgKzQ5NywxMCBAQAogCQllaW5m byAiUnVubmluZyAke215X2NtZH0iCiAJCSR7bXlfY21kfQogCi0JCSMgcmVtb3ZlIHRoZSBvbGQg dmVyc2lvbgotCQkjCi0JCSMgd2h5IGRvIHdlIGRvIHRoaXM/ICB3ZWxsIC4uLgotCQkjCi0JCSMg bm9ybWFsbHksIGVtZXJnZSAtdSBpbnN0YWxscyBhIG5ldyB2ZXJzaW9uIGFuZCB0aGVuIHJlbW92 ZXMgdGhlCi0JCSMgb2xkIHZlcnNpb24uICBob3dldmVyLCBpZiB0aGUgbmV3IHZlcnNpb24gZ29l cyBpbnRvIGEgZGlmZmVyZW50Ci0JCSMgc2xvdCB0byB0aGUgb2xkIHZlcnNpb24sIHRoZW4gdGhl IG9sZCB2ZXJzaW9uIGdldHMgbGVmdCBiZWhpbmQKLQkJIwotCQkjIGlmIFVTRT0tdmhvc3RzLCB0 aGVuIHdlIHdhbnQgdG8gcmVtb3ZlIHRoZSBvbGQgdmVyc2lvbiwgYmVjYXVzZQotCQkjIHRoZSB1 c2VyIGlzIHJlbHlpbmcgb24gcG9ydGFnZSB0byBkbyB0aGUgbWFnaWNhbCB0aGluZyBmb3IgaXQK LQotCQlpZiBbICIke0lTX1VQR1JBREV9IiA9ICIxIiBdIDsgdGhlbgotCQkJZWluZm8gIlJlbW92 aW5nIG9sZCB2ZXJzaW9uICR7UkVNT1ZFX1BLR30iCisJCSMgcnVuIHdlYmFwcC1jbGVhbmVyIGlu c3RlYWQgb2YgZW1lcmdlCisJCWVjaG8KKwkJJHtXRUJBUFBfQ0xFQU5FUn0gLXAgLUMgJHtQTn0K IAotCQkJZW1lcmdlIC1DICIke1JFTU9WRV9QS0d9IgotCQlmaQogCWVsc2UKIAkJIyB2aG9zdHMg ZmxhZyBpcyBvbgogCQkjCg==