Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 410675 - dev-python/unittest2 should not install for =dev-lang/python-2.7 or >=dev-lang/python-3.2
Summary: dev-python/unittest2 should not install for =dev-lang/python-2.7 or >=dev-lan...
Status: RESOLVED INVALID
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Development (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-04-03 17:09 UTC by Mike Nerone
Modified: 2018-04-20 21:44 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mike Nerone 2012-04-03 17:09:29 UTC
Python 2.7 got a revamp of the unittest module. As described in various unittest documentation [1, 2, 3], the thing called "unittest2" is a backport of those features for Python 2.4-2.6, and should only be installed for those ABIs. For 2.7+, it's redundant at best, and at worst can cause unexpected behavior due to import interactions, including deployment cases where tests that appear to run in a development environment turn out not to even be able to execute in a different staging or production environment.

For Python 2.7+, including Python 3, "import unittest2" should cause an ImportError to be raised.

[1] http://docs.python.org/library/unittest.html
[2] http://pypi.python.org/pypi/unittest2
[3] http://www.voidspace.org.uk/python/articles/unittest2.shtml
Comment 1 Mike Nerone 2012-04-05 06:17:39 UTC
Sorry, need to clarify:

I'm referring to the Python 2 line in this bug, and my mention of Python 3 was oversimplified to the point of being wrong. Python 3 gets the revamped unittest in Python 3.2, so the current behavior of substituting unittest2py3k is a good one, but should be restricted to only 3.0 and 3.1.

If I understand the eclass correctly, I think what I'm suggesting is something like:

RESTRICT_PYTHON_ABIS="2.7 3.[23456789]"
Comment 2 Mike Gilbert gentoo-dev 2012-04-05 21:38:13 UTC
That sounds reasonable. I don't really like that 3.[23456789] mask, but I can't think of a better way.
Comment 3 Arfrever Frehtes Taifersar Arahesis 2012-04-05 22:13:17 UTC
Some packages might unconditionally import unittest2. There is nothing technically wrong in it.
Comment 4 Mike Nerone 2012-04-06 04:27:04 UTC
(In reply to comment #3)
> Some packages might unconditionally import unittest2. There is nothing
> technically wrong in it.

That's one of the bad cases I'm talking about. Such a package would appear to support Python 2.7 fine as long as the dev is using Gentoo, but (s)he'll get a rude awakening after releasing it when someone attempts to test it in preparation of a deployment on any platform *other* than Gentoo. As much as it pains me, "not Gentoo" is unfortunately still a significantly larger segment.

Anybody importing unittest2 unconditionally is "doing it wrong" (TM), and the current ebuild does them the disservice of hiding that fact.
Comment 5 Mike Nerone 2012-04-06 04:39:02 UTC
Hint (from someone who always wants the new features):

tests/__init__.py:

import sys
if sys.version_info < (2, 7) or (sys.version_info.major == 3 and sys.version_info.minor < 2):
    sys.modules['unittest'] = __import__('unittest2')

Now all of your test modules can "import unittest" and you can assume it's always the new unittest, but now done the right way.
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-01-01 12:26:44 UTC
(In reply to comment #4)
> (In reply to comment #3)
> > Some packages might unconditionally import unittest2. There is nothing
> > technically wrong in it.
> 
> That's one of the bad cases I'm talking about. Such a package would appear
> to support Python 2.7 fine as long as the dev is using Gentoo, but (s)he'll
> get a rude awakening after releasing it when someone attempts to test it in
> preparation of a deployment on any platform *other* than Gentoo. As much as
> it pains me, "not Gentoo" is unfortunately still a significantly larger
> segment.
> 
> Anybody importing unittest2 unconditionally is "doing it wrong" (TM), and
> the current ebuild does them the disservice of hiding that fact.

Did anyone perform a research on what other distros are actually doing? Unless I'm reading their page wrong, it seems that Debian sid has unittest2 [1] and py2.7 [2].

[1]:http://packages.debian.org/sid/python-unittest2
[2]:http://packages.debian.org/sid/python
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-01-01 12:28:47 UTC
(In reply to comment #5)
> Hint (from someone who always wants the new features):
> 
> tests/__init__.py:
> 
> import sys
> if sys.version_info < (2, 7) or (sys.version_info.major == 3 and
> sys.version_info.minor < 2):
>     sys.modules['unittest'] = __import__('unittest2')
> 
> Now all of your test modules can "import unittest" and you can assume it's
> always the new unittest, but now done the right way.

This is really a bad solution. I'd rather suggest something like:

  try:
    import unittest2 as unittest
  except ImportError:
    import unittest

or the other way around. ImportError is much cleaner than relying on version numbers.
Comment 8 Mike Nerone 2013-01-03 08:43:32 UTC
(In reply to comment #7)
> (In reply to comment #5)
> This is really a bad solution. I'd rather suggest something like:
> 
>   try:
>     import unittest2 as unittest
>   except ImportError:
>     import unittest
> 
> or the other way around. ImportError is much cleaner than relying on version
> numbers.

No, it's not "really bad". It's just "not the first choice" for most cases like this, and I agree with you. Unfortunately, the "first choice" practice of using ImportError doesn't work well in this particular case. Let me explain:

Put simply, the logic is backwards: we want unittest2's features - they are builtin for Python 2.7 and 3.2+. What we want is _not_ "try to import 'unittest2' as first choice, otherwise fall back to builtin 'unittest'", it's "use builtin 'unittest' as first choice _if_ it's already unittest2ish (i.e. Python 2.7 or 3.2+), otherwise fall back to actually importing 'unittest2'".

This reversed logic has a bad result: for Python 2.7 or 3.2+, you end up using unittest2 if it's available instead of the builtin unittest. One reason this is bad: the builtin one is better integrated as part of the standard library instead of being a bolt-on - a symptom of this is that the builtin unittest(2) allows "python -m unittest ..." to work, where the external "unittest2" can not (providing a separate "unit2.py" script as a hackaround). Another, related, bad result is that if you are on 2.7 or 3.2+, and deign to actually use "python -m unittest", then you're using the builtin unittest test runner, but the test modules themselves are using the external unittest2's facilities. I shudder to think how many WTFs might result from the side effects of such interactions.

Finally, it's also potentially wrong for Python <2.7 or <3.2: the ImportError solution does nothing to prevent third-party libraries from incorrectly importing the builtin unittest. Libraries don't do this often, but it happens. More potential WTFs there's no reason to risk.

As I said, I agree with you for most cases. Version-checking is akin to type-checking, and it's something we actively try to avoid in Python. In this case, though, it's the lesser of the evils. I also thought about introspecting the builtin and proceeding based on that, but this would require a wasteful import and potential side-effects (I vaguely recall that I actually confirmed such side effects, but don't hold me to that). Believe me, I thought long and hard before my grudging acceptance of the technique I gave in comment #5. :P

Regarding "the other way around", as you mentioned, that doesn't work either, because for Python <2.7 or <3.2, is simply fails to import unittest2 at all.

Note: I realize your recipe comes from the unittest2 docs [1], but that bit is addressing a different use case: "how to use unittest2 with the specific Python versions 2.4, 2.5, and 2.6 *only if it happens to be available*". Thus, it only applies if A. you _know_for_a_fact_ that those are your only deployment targets and B. you're not actually using any features of the new unittest. My solution is for the use case "_ensure_ you're using the new unittest so you can reliably use its features regardless of Python version". Further, the same docs say just above that recipe that unittest2 is only _tested_ with those specific versions - the author (the quite brilliant Michael Foord) obviously doesn't intend for us to use "unittest2" _at_all_ with Python 2.7 or 3.2+, but that's exactly what the "ImportError" solution does if you run it under those versions. My solution keeps your code compatible across _all_ versions, and addresses the additional problems I outlined above, as well.

[1] http://www.voidspace.org.uk/python/articles/unittest2.shtml#the-unittest2-package
Comment 9 Mike Gilbert gentoo-dev 2013-01-07 06:05:54 UTC
After reviewing this a bit, I don't really think we should enforce this by not installing the code for certain Python versions.

The fact is, although it was intended as a backport, there are certain differences[1]. At least one package relies on this for its test suite (see bug 450620).

[1] http://pypi.python.org/pypi/unittest2#differences
Comment 10 Pacho Ramos gentoo-dev 2013-09-28 15:07:46 UTC
In most cases looks like the import is wrong and, taking care only a few packages are affected, maybe we could even patch them to use "unittest" and make them need python >= 2.7
Comment 11 Justin Lecher (RETIRED) gentoo-dev 2015-11-23 09:23:07 UTC
What is the situation here nowadays? Are packages directly using unittest2 instead of unittest?
Comment 12 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-04-20 21:44:24 UTC
Some packages actually require it, so we shouldn't block installing it.