Discussion:
[Scikit-learn-general] How do I run the the unit tests?
Keith Goodman
2010-09-10 18:46:41 UTC
Permalink
I can't figure out how to run the unit tests. There's no
scikits.learn.test() so I tried nosetests:

$ nosetests scikits.learn

----------------------------------------------------------------------
Ran 0 tests in 0.002s

OK

$ nosetests scikits.learn.tests

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

$ nosetests --dammit scikits.learn
Usage: nosetests [options]

nosetests: error: no such option: --dammit
Alexandre Gramfort
2010-09-10 18:49:29 UTC
Permalink
Hi Keith,

if you just type "nosetests" in /path/to/scikit-learn/
it should work

Alex
Post by Keith Goodman
I can't figure out how to run the unit tests. There's no
$ nosetests scikits.learn
----------------------------------------------------------------------
Ran 0 tests in 0.002s
OK
$ nosetests scikits.learn.tests
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
$ nosetests --dammit scikits.learn
Usage: nosetests [options]
nosetests: error: no such option: --dammit
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing
http://p.sf.net/sfu/novell-sfdev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Keith Goodman
2010-09-10 18:56:47 UTC
Permalink
On Fri, Sep 10, 2010 at 11:49 AM, Alexandre Gramfort
Post by Alexandre Gramfort
Hi Keith,
if you just type "nosetests" in /path/to/scikit-learn/
it should work
Sorry to need so much hand holding. I can't get that to work:

***@kg:/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg$
nosetests

----------------------------------------------------------------------
Ran 0 tests in 0.682s

OK
***@kg:/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg$
cd scikits/
***@kg:/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits$
nosetests

----------------------------------------------------------------------
Ran 0 tests in 0.002s

OK
***@kg:/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits$
ls
__init__.py __init__.pyc learn
***@kg:/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits$
cd learn
***@kg:/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn$
nosetests

----------------------------------------------------------------------
Ran 0 tests in 0.002s

OK
***@kg:/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn$
cd /ba/devel/scikit-learn/
***@kg:/ba/devel/scikit-learn$ nosetests

Name Stmts Exec Cover Missing
---------------------------------------------------------------------------
scikits.learn 14 14 100%
scikits.learn.base 73 15 20%
22-29, 47-73, 93-106, 117-124, 134-155, 158-159, 167-168, 197, 219
scikits.learn.cluster 4 4 100%
scikits.learn.cluster.affinity_propagation_ 74 6 8%
48-148, 199-202, 221-225
scikits.learn.cluster.k_means_ 86 10 11%
44-67, 125-171, 193-200, 222-231, 312-315, 319-324
scikits.learn.cluster.mean_shift_ 75 9 12%
29-43, 57-61, 96-177, 222, 233-235
scikits.learn.cluster.spectral 56 17 30%
15, 49-95, 125-128, 161-162, 179-182
scikits.learn.covariance 2 2 100%
scikits.learn.covariance.covariance 41 14 34%
38, 42-46, 50-54, 58-60, 64, 89-96, 130-131, 135-139
scikits.learn.covariance.ledoit_wolf 27 6 22%
44-63, 103, 107-110
scikits.learn.cross_val 156 38 24%
52, 55-60, 63, 69, 112-113, 116-123, 126, 134, 177-181, 184-195, 198,
206, 255-263, 266-290, 293, 301, 347-348, 352-357, 360, 367, 414-417,
421-432, 435, 443, 452-456, 486-504, 514-524
scikits.learn.externals 0 0 100%
scikits.learn.externals.joblib 6 6 100%
scikits.learn.externals.joblib.format_stack 223 21 9%
41-64, 67, 83-89, 99-110, 114-146, 150-175, 180-319, 334-386, 392-427
scikits.learn.externals.joblib.func_inspect 102 7 6%
35-54, 68-114, 140-227
scikits.learn.externals.joblib.hashing 37 11 29%
22-25, 28-32, 49-53, 60-85, 101-105
scikits.learn.externals.joblib.logger 63 16 25%
20, 23-26, 42, 45, 49, 54-63, 74-103, 117-135
scikits.learn.externals.joblib.memory 199 41 20%
19-20, 28-33, 63-69, 132-153, 159-176, 186-197, 206-212, 218-219,
230-280, 286-294, 301-313, 320-323, 330-352, 359-368, 375-378,
388-394, 403, 449-455, 470-476, 486-489, 501-503, 510
scikits.learn.externals.joblib.numpy_pickle 60 15 25%
23, 35-44, 51-65, 75-81, 92-103, 121-128, 144-150
scikits.learn.externals.joblib.parallel 84 23 27%
14, 18-19, 31, 35, 38, 55, 59-65, 70-77, 92-97, 199-200, 207-265, 269
scikits.learn.features 0 0 100%
scikits.learn.glm 5 5 100%
scikits.learn.glm.base 36 11 30%
42-43, 48, 58-66, 72-75, 79-84, 107, 129-139
scikits.learn.glm.bayes 112 13 11%
129-136, 151-216, 233-243, 372-380, 400-420, 442-489, 507-515
scikits.learn.glm.coordinate_descent 119 21 17%
55-57, 82-110, 137-140, 144-173, 213-231, 268-283, 290-292, 315-350,
409-412
scikits.learn.glm.lars 172 17 9%
65-206, 251-253, 256-275, 323-326, 331-354, 387-389, 421-466, 482-486
scikits.learn.glm.ridge 18 6 33% 33-34, 52-72
scikits.learn.gmm 256 42 16%
36-46, 50-58, 89-93, 119-134, 245-265, 274, 279, 284, 288-295,
298-300, 306, 309-312, 318, 321-326, 351-356, 372-373, 391-392,
405-406, 421-433, 472-503, 506-519, 528-536, 540-543, 547-554,
559-572, 576-600, 605-616, 628-631, 635, 639-651, 655, 659-663,
667-687
scikits.learn.lda 90 11 12%
70-73, 90-172, 176-183, 198-207, 224-226, 241-246
scikits.learn.logistic 10 7 70% 61, 65-66
scikits.learn.metrics 53 8 15%
36-48, 78-90, 111-115, 145-160, 171, 178, 185
scikits.learn.svm 124 39 31%
23-40, 46-55, 78-120, 142-145, 179-194, 212-214, 227-229, 251-258,
264-269, 289-298, 320-321, 330-332, 336-338, 342, 352, 437, 537, 606,
677, 741, 755
scikits.learn.utils 0 0 100%
scikits.learn.utils._csgraph 21 9 42% 59-76
scikits.learn.utils.extmath 38 13 34%
15-21, 30-33, 38, 44-47, 53-65, 77-78
scikits.learn.utils.fixes 48 6 12%
13-47, 54, 59-75, 81-82
scikits.learn.utils.graph 73 8 10%
46-66, 70, 78-107, 111-126, 130-138
scikits.learn.utils.sparsetools 1 1 100%
scikits.learn.utils.sparsetools.csgraph 54 26 48%
16-18, 28, 32-33, 35-45, 48, 51-54, 57-59, 64-66, 72
---------------------------------------------------------------------------
TOTAL 2612 518 19%
----------------------------------------------------------------------
Ran 0 tests in 1.369s
Alexandre Gramfort
2010-09-10 18:57:54 UTC
Permalink
ok give a try to :

nosetests /path/to/scikits/learn

Alex
Post by Keith Goodman
On Fri, Sep 10, 2010 at 11:49 AM, Alexandre Gramfort
Post by Alexandre Gramfort
Hi Keith,
if you just type "nosetests" in /path/to/scikit-learn/
it should work
nosetests
----------------------------------------------------------------------
Ran 0 tests in 0.682s
OK
cd scikits/
nosetests
----------------------------------------------------------------------
Ran 0 tests in 0.002s
OK
ls
__init__.py  __init__.pyc  learn
cd learn
nosetests
----------------------------------------------------------------------
Ran 0 tests in 0.002s
OK
cd /ba/devel/scikit-learn/
Name                                          Stmts   Exec  Cover   Missing
---------------------------------------------------------------------------
scikits.learn                                    14     14   100%
scikits.learn.base                               73     15    20%
22-29, 47-73, 93-106, 117-124, 134-155, 158-159, 167-168, 197, 219
scikits.learn.cluster                             4      4   100%
scikits.learn.cluster.affinity_propagation_      74      6     8%
48-148, 199-202, 221-225
scikits.learn.cluster.k_means_                   86     10    11%
44-67, 125-171, 193-200, 222-231, 312-315, 319-324
scikits.learn.cluster.mean_shift_                75      9    12%
29-43, 57-61, 96-177, 222, 233-235
scikits.learn.cluster.spectral                   56     17    30%
15, 49-95, 125-128, 161-162, 179-182
scikits.learn.covariance                          2      2   100%
scikits.learn.covariance.covariance              41     14    34%
38, 42-46, 50-54, 58-60, 64, 89-96, 130-131, 135-139
scikits.learn.covariance.ledoit_wolf             27      6    22%
44-63, 103, 107-110
scikits.learn.cross_val                         156     38    24%
52, 55-60, 63, 69, 112-113, 116-123, 126, 134, 177-181, 184-195, 198,
206, 255-263, 266-290, 293, 301, 347-348, 352-357, 360, 367, 414-417,
421-432, 435, 443, 452-456, 486-504, 514-524
scikits.learn.externals                           0      0   100%
scikits.learn.externals.joblib                    6      6   100%
scikits.learn.externals.joblib.format_stack     223     21     9%
41-64, 67, 83-89, 99-110, 114-146, 150-175, 180-319, 334-386, 392-427
scikits.learn.externals.joblib.func_inspect     102      7     6%
35-54, 68-114, 140-227
scikits.learn.externals.joblib.hashing           37     11    29%
22-25, 28-32, 49-53, 60-85, 101-105
scikits.learn.externals.joblib.logger            63     16    25%
20, 23-26, 42, 45, 49, 54-63, 74-103, 117-135
scikits.learn.externals.joblib.memory           199     41    20%
19-20, 28-33, 63-69, 132-153, 159-176, 186-197, 206-212, 218-219,
230-280, 286-294, 301-313, 320-323, 330-352, 359-368, 375-378,
388-394, 403, 449-455, 470-476, 486-489, 501-503, 510
scikits.learn.externals.joblib.numpy_pickle      60     15    25%
23, 35-44, 51-65, 75-81, 92-103, 121-128, 144-150
scikits.learn.externals.joblib.parallel          84     23    27%
14, 18-19, 31, 35, 38, 55, 59-65, 70-77, 92-97, 199-200, 207-265, 269
scikits.learn.features                            0      0   100%
scikits.learn.glm                                 5      5   100%
scikits.learn.glm.base                           36     11    30%
42-43, 48, 58-66, 72-75, 79-84, 107, 129-139
scikits.learn.glm.bayes                         112     13    11%
129-136, 151-216, 233-243, 372-380, 400-420, 442-489, 507-515
scikits.learn.glm.coordinate_descent            119     21    17%
55-57, 82-110, 137-140, 144-173, 213-231, 268-283, 290-292, 315-350,
409-412
scikits.learn.glm.lars                          172     17     9%
65-206, 251-253, 256-275, 323-326, 331-354, 387-389, 421-466, 482-486
scikits.learn.glm.ridge                          18      6    33%   33-34, 52-72
scikits.learn.gmm                               256     42    16%
36-46, 50-58, 89-93, 119-134, 245-265, 274, 279, 284, 288-295,
298-300, 306, 309-312, 318, 321-326, 351-356, 372-373, 391-392,
405-406, 421-433, 472-503, 506-519, 528-536, 540-543, 547-554,
559-572, 576-600, 605-616, 628-631, 635, 639-651, 655, 659-663,
667-687
scikits.learn.lda                                90     11    12%
70-73, 90-172, 176-183, 198-207, 224-226, 241-246
scikits.learn.logistic                           10      7    70%   61, 65-66
scikits.learn.metrics                            53      8    15%
36-48, 78-90, 111-115, 145-160, 171, 178, 185
scikits.learn.svm                               124     39    31%
23-40, 46-55, 78-120, 142-145, 179-194, 212-214, 227-229, 251-258,
264-269, 289-298, 320-321, 330-332, 336-338, 342, 352, 437, 537, 606,
677, 741, 755
scikits.learn.utils                               0      0   100%
scikits.learn.utils._csgraph                     21      9    42%   59-76
scikits.learn.utils.extmath                      38     13    34%
15-21, 30-33, 38, 44-47, 53-65, 77-78
scikits.learn.utils.fixes                        48      6    12%
13-47, 54, 59-75, 81-82
scikits.learn.utils.graph                        73      8    10%
46-66, 70, 78-107, 111-126, 130-138
scikits.learn.utils.sparsetools                   1      1   100%
scikits.learn.utils.sparsetools.csgraph          54     26    48%
16-18, 28, 32-33, 35-45, 48, 51-54, 57-59, 64-66, 72
---------------------------------------------------------------------------
TOTAL                                          2612    518    19%
----------------------------------------------------------------------
Ran 0 tests in 1.369s
Keith Goodman
2010-09-10 19:07:06 UTC
Permalink
On Fri, Sep 10, 2010 at 11:57 AM, Alexandre Gramfort
Post by Alexandre Gramfort
nosetests /path/to/scikits/learn
It just runs the code coverage. OK, I'll move on to try some examples.
No need to worry about unit tests for that.
Alexandre Gramfort
2010-09-10 19:10:17 UTC
Permalink
my last attempt:

nosetests scikits/learn/tests/*

it won't run all tests but at least you should see something

Alex
Post by Keith Goodman
On Fri, Sep 10, 2010 at 11:57 AM, Alexandre Gramfort
Post by Alexandre Gramfort
nosetests /path/to/scikits/learn
It just runs the code coverage. OK, I'll move on to try some examples.
No need to worry about unit tests for that.
j***@gmail.com
2010-09-10 19:15:48 UTC
Permalink
On Fri, Sep 10, 2010 at 3:10 PM, Alexandre Gramfort
Post by Alexandre Gramfort
nosetests scikits/learn/tests/*
it won't run all tests but at least you should see something
Alex
Post by Keith Goodman
On Fri, Sep 10, 2010 at 11:57 AM, Alexandre Gramfort
Post by Alexandre Gramfort
nosetests /path/to/scikits/learn
this worked for me, that's how I ran the tests

in general it should also work using a specific file

nosetests /path/to/scikits/learn/tests/testfilename

Keith, did you try to import scikits.learn, maybe nosetests doesn't
find it on the python path ?

Josef
Post by Alexandre Gramfort
Post by Keith Goodman
It just runs the code coverage. OK, I'll move on to try some examples.
No need to worry about unit tests for that.
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing
http://p.sf.net/sfu/novell-sfdev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Keith Goodman
2010-09-10 19:33:51 UTC
Permalink
Post by j***@gmail.com
Keith, did you try to import scikits.learn, maybe nosetests doesn't
find it on the python path ?
Yes, I ran the lasso CV and digit recognition examples, no problem.

I should probably try building a stable version and then run the unit tests.

Would it be helpful to wire up scikits.learn.test() using numpy's
Tester? Something like this in the top level __init__:

try:
from numpy.testing import Tester
test = Tester().test
del Tester
except (ImportError, ValueError):
print "No la unit testing available."
Fabian Pedregosa
2010-09-10 19:46:10 UTC
Permalink
Post by Keith Goodman
Post by j***@gmail.com
Keith, did you try to import scikits.learn, maybe nosetests doesn't
find it on the python path ?
Yes, I ran the lasso CV and digit recognition examples, no problem.
I should probably try building a stable version and then run the unit tests.
Would it be helpful to wire up scikits.learn.test() using numpy's
from numpy.testing import Tester
test = Tester().test
del Tester
print "No la unit testing available."
Yeah, that would be useful also for testing it on an installed location (it
has happened that we forgot to add a whole directory to setup.py and didn't
notice because we were all running it inplace).

Can you submit this in form of a working patch ?

Thanks,
Post by Keith Goodman
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing
http://p.sf.net/sfu/novell-sfdev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Keith Goodman
2010-09-10 19:59:46 UTC
Permalink
On Fri, Sep 10, 2010 at 12:46 PM, Fabian Pedregosa
Post by Fabian Pedregosa
Post by Keith Goodman
Would it be helpful to wire up scikits.learn.test() using numpy's
   from numpy.testing import Tester
   test = Tester().test
   del Tester
   print "No la unit testing available."
Yeah, that would be useful also for testing it on an installed location (it
has happened that we forgot to add a whole directory to setup.py and didn't
notice because we were all running it inplace).
Can you submit this in form of a working patch ?
I'd be happy to submit a patch when/if I get all the unit tests running.

If anyone wants to try it, add to scikits/learn/__init__.py:

try:
from numpy.testing import Tester
test = Tester().test
del Tester
except (ImportError, ValueError):
print "No scikits.learn unit testing available."

That should run all the unit tests but I get:

$ ipython
Post by Fabian Pedregosa
Post by Keith Goodman
import scikits.learn
scikits.learn.test()
Running unit tests for scikits.learn
NumPy version 1.4.1
NumPy is installed in /usr/local/lib/python2.6/site-packages/numpy
Python version 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3]
nose version 0.11.1

----------------------------------------------------------------------
Ran 0 tests in 0.002s

OK
<nose.result.TextTestResult run=0 errors=0 failures=0>
Fabian Pedregosa
2010-09-11 07:10:39 UTC
Permalink
Post by Keith Goodman
On Fri, Sep 10, 2010 at 12:46 PM, Fabian Pedregosa
Post by Fabian Pedregosa
Post by Keith Goodman
Would it be helpful to wire up scikits.learn.test() using numpy's
from numpy.testing import Tester
test = Tester().test
del Tester
print "No la unit testing available."
Yeah, that would be useful also for testing it on an installed location
(it
Post by Fabian Pedregosa
has happened that we forgot to add a whole directory to setup.py and
didn't
Post by Fabian Pedregosa
notice because we were all running it inplace).
Can you submit this in form of a working patch ?
I'd be happy to submit a patch when/if I get all the unit tests running.
from numpy.testing import Tester
test = Tester().test
del Tester
print "No scikits.learn unit testing available."
$ ipython
Post by Fabian Pedregosa
Post by Keith Goodman
import scikits.learn
scikits.learn.test()
Running unit tests for scikits.learn
NumPy version 1.4.1
NumPy is installed in /usr/local/lib/python2.6/site-packages/numpy
Python version 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3]
nose version 0.11.1
----------------------------------------------------------------------
Ran 0 tests in 0.002s
OK
<nose.result.TextTestResult run=0 errors=0 failures=0>
That's odd since the same code does run the test suite for me. I'm running
almost the same versions (just nose 0.11.3 differ but not much). It would be
helpful if you could verify that the following does run numpy test suite so
that we can be sure it's an issue in scikits.learn:

import numpy
numpy.test()


Thanks,
Post by Keith Goodman
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing
http://p.sf.net/sfu/novell-sfdev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Gael Varoquaux
2010-09-11 07:55:34 UTC
Permalink
Post by Fabian Pedregosa
That's odd since the same code does run the test suite for me. I'm running
almost the same versions (just nose 0.11.3 differ but not much). It would
be helpful if you could verify that the following does run numpy test
import numpy
numpy.test()
Maybe nose 0.11.3 fixed the bug with namespace packages?

Ga�l
Gael Varoquaux
2010-09-11 07:58:42 UTC
Permalink
Post by Keith Goodman
I'd be happy to submit a patch when/if I get all the unit tests running.
from numpy.testing import Tester
test = Tester().test
del Tester
print "No scikits.learn unit testing available."
Keith, if you can upgrade your nosetests install, and check that it does
fix the problem for you, I will add such lines, as I believe that they
are indeed valuable.

Thanks,

Gaël
j***@gmail.com
2010-09-11 14:20:20 UTC
Permalink
On Sat, Sep 11, 2010 at 3:58 AM, Gael Varoquaux
Post by Gael Varoquaux
Post by Keith Goodman
I'd be happy to submit a patch when/if I get all the unit tests running.
    from numpy.testing import Tester
    test = Tester().test
    del Tester
    print "No scikits.learn unit testing available."
Keith, if you can upgrade your nosetests install, and check that it does
fix the problem for you, I will add such lines, as I believe that they
are indeed valuable.
Post by Keith Goodman
nose.__version__
'0.11.1'

I can run the nosetests and have scikits installed in different location.
Every once in a while, I have a problem with it, expecially when the
egg-info or *.pth file is missing

for example setup.py bdist created the file

scikits.learn-0.5_git-py2.5-nspkg.pth
with one-line content:
-------------
import sys,new,os; p =
os.path.join(sys._getframe(1).f_locals['sitedir'], *('scikits',)); ie
= os.path.exists(os.path.join(p,'__init__.py')); m = not ie and
sys.modules.setdefault('scikits',new.module('scikits')); mp = (m or
[]) and m.__dict__.setdefault('__path__',[]); (p not in mp) and
mp.append(p)
---------------

the file is designated for site-packages, (but I was working in the
local install directory when I tried out scikits learn)


Also my first instinct is always
import package
package.test()

Josef
Post by Gael Varoquaux
Thanks,
Gaël
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing
http://p.sf.net/sfu/novell-sfdev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Gael Varoquaux
2010-09-11 07:54:23 UTC
Permalink
Post by j***@gmail.com
Post by Alexandre Gramfort
nosetests /path/to/scikits/learn
this worked for me, that's how I ran the tests
in general it should also work using a specific file
nosetests /path/to/scikits/learn/tests/testfilename
Keith, did you try to import scikits.learn, maybe nosetests doesn't
find it on the python path ?
I believe that the problem is that Keith has other scikits installed that
define other namespace packages. Nosetest clearly doesn't handle
namespace package, I have had the problem too. I can't blame nosetests
for that, I believe that namespace packages are really an ugly hack.
Unfortunately, they are a standard in the Python world, and nosetests
should fix their code.

Keith, one option is probably to go to the root of the scikit-learn code
(where you have your git checkout) and, after building it inplace, cd in
the scikits/learn directory, where you can find files such as base.py or
pipeline.py. In this directory, you can run::

nosetests --with-doctest

This should give you are decent coverage of the tests.

Sorry for the inconvenience.

Gaël

PS: do people believe that we can aleviate this problem by providing a
'test()' command in the scikit, a la numpy? I haven't looked in detail at
how this 'test()' command works.
Keith Goodman
2010-09-11 15:29:03 UTC
Permalink
On Sat, Sep 11, 2010 at 12:54 AM, Gael Varoquaux
Post by Gael Varoquaux
I believe that the problem is that Keith has other scikits installed that
define other namespace packages. Nosetest clearly doesn't handle
namespace package, I have had the problem too. I can't blame nosetests
for that, I believe that namespace packages are really an ugly hack.
Unfortunately, they are a standard in the Python world, and nosetests
should fix their code.
I don't understand any of this but here's what I did. I removed
statsmodels and learn, the only scikits on my computer. Then

$ git clone http://github.com/scikit-learn/scikit-learn.git
$ cd scikit-learn
$ make

which gives:

Ran 523 tests in 28.703s
FAILED (SKIP=1, failures=3)
make: *** [test] Error 1

I then ran (are the failures know?) nosetests --with-doctest (see below)

Next I installed learn and statsmodels and added my
scikits.learn.test() code to learn's __init__.py. That ran 0 tests. I
then replaced my code with the .test() code from statemodels 0.2; it
ran 489 tests:

Ran 489 tests in 13.167s
FAILED (SKIP=1, failures=2)
<nose.result.TextTestResult run=489 errors=0 failures=2>

---

$ nosetests --with-doctest
.../ba/devel/scikit-learn/scikits/learn/cluster/spectral.py:52:
UserWarning: pyamg not available, using scipy.sparse
warnings.warn('pyamg not available, using scipy.sparse')
..../ba/devel/scikit-learn/scikits/learn/cross_val.py:515:
UserWarning: split is deprecated and will be removed, please use
indexing instead
warnings.warn('split is deprecated and will be removed, '
...../usr/local/lib/python2.6/site-packages/numpy/lib/utils.py:140:
DeprecationWarning: `unique1d` is deprecated!
warnings.warn(depdoc, DeprecationWarning)
.............................................................................................................................................................................................................................................................................................................F.............................................................F.........F....................................................................../usr/local/lib/python2.6/site-packages/scipy/cluster/vq.py:570:
UserWarning: One of the clusters is empty. Re-run kmean with a
different initialization.
warnings.warn("One of the clusters is empty. "
.................................S.................................
======================================================================
FAIL: Test BayesianRidge on diabetes
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.6/nose/case.py", line 183, in runTest
self.test(*self.arg)
File "/ba/devel/scikit-learn/scikits/learn/glm/tests/test_bayes.py",
line 27, in XFailed_test_bayesian_on_diabetes
assert_array_equal(np.diff(clf.all_score_) > 0, True)
File "/usr/local/lib/python2.6/site-packages/numpy/testing/utils.py",
line 677, in assert_array_equal
verbose=verbose, header='Arrays are not equal')
File "/usr/local/lib/python2.6/site-packages/numpy/testing/utils.py",
line 609, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not equal

(mismatch 66.6666666667%)
x: array([False, True, True, True, False, False, False, False,
False], dtype=bool)
y: array(True, dtype=bool)
Post by Gael Varoquaux
raise AssertionError('\nArrays are not equal\n\n(mismatch 66.6666666667%)\n x: array([False, True, True, True, False, False, False, False, False], dtype=bool)\n y: array(True, dtype=bool)')
-------------------- >> begin captured stdout << ---------------------
Convergence after 9 iterations

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: test_train (scikits.learn.tests.test_gmm.TestGMMWithFullCovars)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/ba/devel/scikit-learn/scikits/learn/tests/test_gmm.py", line
384, in test_train
self.assertTrue(post_testll >= init_testll)
AssertionError:
"""Fail the test unless the expression is true."""
Post by Gael Varoquaux
if not False: raise self.failureException, None
======================================================================
FAIL: test_train (scikits.learn.tests.test_gmm.TestGMMWithTiedCovars)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/ba/devel/scikit-learn/scikits/learn/tests/test_gmm.py", line
380, in test_train
self.assertTrue(np.all(np.diff(trainll) > -0.5))
AssertionError:
"""Fail the test unless the expression is true."""
Post by Gael Varoquaux
if not False: raise self.failureException, None
Name Stmts Exec
Cover Missing
------------------------------------------------------------------------------------
scikits.learn 14 14 100%
scikits.learn.ann 0 0 100%
scikits.learn.ann.mlp 76 14
18% 25-48, 54-57, 63, 76-92, 97-98, 111, 121, 126-143, 146
scikits.learn.ann.rbf 68 15
22% 25-27, 33-34, 40, 51-69, 75-76, 85-98, 108, 113-130, 134
scikits.learn.ann.srn 85 14
16% 25-55, 61-65, 71, 85-113, 118-119, 129, 139, 145-164, 167
scikits.learn.base 73 70
95% 26, 103-105
scikits.learn.cluster 4 4 100%
scikits.learn.cluster.affinity_propagation_ 74 68
91% 57, 60, 123, 144-146
scikits.learn.cluster.k_means_ 86 74
86% 46-47, 130-134, 142-145, 155, 158, 197
scikits.learn.cluster.mean_shift_ 75 56
74% 29-43, 57-61, 97
scikits.learn.cluster.spectral 56 45
80% 15, 81-94
scikits.learn.covariance 2 2 100%
scikits.learn.covariance.covariance 41 39
95% 45, 91
scikits.learn.covariance.ledoit_wolf 27 23
85% 46-48, 61
scikits.learn.cross_val 156 154
98% 455, 491
scikits.learn.datasets 2 2 100%
scikits.learn.datasets.base 33 26
78% 91-99
scikits.learn.datasets.mlcomp 63 8
12% 14-42, 90-137
scikits.learn.datasets.samples_generator 38 30
78% 14, 133-136, 161-164
scikits.learn.externals 0 0 100%
scikits.learn.externals.joblib 6 6 100%
scikits.learn.externals.joblib.format_stack 223 176
78% 44, 49, 57, 61-64, 125-127, 138, 140, 160-165, 186-190, 198,
201, 208-217, 239-240, 274-275, 291, 314, 336-337, 355-357, 364-365,
369-383, 412, 419
scikits.learn.externals.joblib.func_inspect 102 81
79% 73-77, 82-90, 111-113, 143, 148, 159, 200, 204, 218
scikits.learn.externals.joblib.hashing 37 34
91% 64-68, 104
scikits.learn.externals.joblib.logger 63 54
85% 23-26, 49, 59, 75, 80, 122-123
scikits.learn.externals.joblib.memory 199 178
89% 19-20, 28-33, 68, 192, 249, 288, 342, 344, 366-368, 393-394,
487, 502, 510
scikits.learn.externals.joblib.numpy_pickle 60 59 98% 101
scikits.learn.externals.joblib.parallel 84 72
85% 14, 18-19, 70-77, 210, 229, 241
scikits.learn.fastica 128 111
86% 194, 217-226, 233-234, 250, 256-258, 277-278, 342
scikits.learn.feature_selection 1 1 100%
scikits.learn.feature_selection.univariate_selection 94 93 98% 161
scikits.learn.features 0 0 100%
scikits.learn.features.image 40 39 97% 97
scikits.learn.features.text 151 128
84% 98, 104, 138, 209, 225-232, 235-238, 279, 313-320, 323-326
scikits.learn.glm 5 5 100%
scikits.learn.glm.base 36 33
91% 79-84
scikits.learn.glm.bayes 112 106
94% 179-186
scikits.learn.glm.benchmarks 0 0 100%
scikits.learn.glm.benchmarks.bench_bayes 17 6
35% 17-32
scikits.learn.glm.benchmarks.bench_glm 36 5
13% 15-60
scikits.learn.glm.benchmarks.bench_lars 11 7
63% 17-20
scikits.learn.glm.coordinate_descent 119 116
97% 103, 166, 228
scikits.learn.glm.lars 172 172 100%
scikits.learn.glm.ridge 18 18 100%
scikits.learn.gmm 256 230
89% 289, 292-295, 580, 586, 592, 600, 614, 639-651, 655, 659-663,
680, 682
scikits.learn.grid_search 71 53
74% 18-24, 71-72, 183, 205-216
scikits.learn.hmm 394 386
97% 237-238, 343, 477, 480, 682, 733-734
scikits.learn.lda 90 68
75% 71, 94, 101, 108, 115, 124, 126-127, 143, 147, 163, 176-183,
241-246
scikits.learn.logistic 10 8
80% 65-66
scikits.learn.metrics 53 53 100%
scikits.learn.naive_bayes 33 33 100%
scikits.learn.neighbors 25 25 100%
scikits.learn.pca 87 83
95% 33, 44-45, 183
scikits.learn.pipeline 37 37 100%
scikits.learn.preprocessing 21 21 100%
scikits.learn.qda 75 63
84% 72, 94, 96, 102, 109, 113, 126, 130, 134, 197-202
scikits.learn.rfe 47 47 100%
scikits.learn.sparse 1 1 100%
scikits.learn.sparse.svm 85 77
90% 181, 201, 221, 239, 244-247
scikits.learn.svm 124 116
93% 180, 267, 332, 336-338, 342, 677
scikits.learn.utils 0 0 100%
scikits.learn.utils._csgraph 21 17
80% 61-62, 65, 70
scikits.learn.utils.bench 3 1
33% 13-15
scikits.learn.utils.extmath 38 19
50% 20, 30-33, 38, 44-47, 53-65, 77-78
scikits.learn.utils.fixes 48 30
62% 15-20, 23-28, 38, 42-47, 73, 81-82
scikits.learn.utils.graph 73 70
95% 47, 64, 70
scikits.learn.utils.sparsetools 1 1 100%
scikits.learn.utils.sparsetools.csgraph 54 27
50% 16-18, 28, 32-33, 35-45, 48, 51-54, 57-59, 64-66
------------------------------------------------------------------------------------
TOTAL 4334 3624 83%
----------------------------------------------------------------------
Ran 523 tests in 31.811s

FAILED (SKIP=1, failures=3)
Olivier Grisel
2010-09-11 15:39:13 UTC
Permalink
Perfect this is the "expected" results. Fixes for the remaining broken
tests are underway AFAIK.
--
Olivier
Gael Varoquaux
2010-09-11 15:52:42 UTC
Permalink
Post by Keith Goodman
I don't understand any of this but here's what I did. I removed
statsmodels and learn, the only scikits on my computer.
YEs, this coincides with what I had noticed: differents 'scikits'
installed means different namespace packages with the same prefix, and
noes gets lost.
Post by Keith Goodman
Next I installed learn and statsmodels and added my
scikits.learn.test() code to learn's __init__.py. That ran 0 tests. I
then replaced my code with the .test() code from statemodels 0.2; it
Ran 489 tests in 13.167s
FAILED (SKIP=1, failures=2)
<nose.result.TextTestResult run=489 errors=0 failures=2>
That seems that the tests from statsmodel get run. I remembered that the
numpy 'test()' trick did not work with namespace packages and that was
one of the reasons that we were not using it.

I'll give it a second look.
Post by Keith Goodman
$ nosetests --with-doctest
UserWarning: pyamg not available, using scipy.sparse
warnings.warn('pyamg not available, using scipy.sparse')
All this is (unfortunately) normal. Mostly spurious erros because the
methods are slightly unstable.

However, we need to fix these failures, via better tests, and more stable
methods.

Gaël
Keith Goodman
2010-09-11 15:58:27 UTC
Permalink
On Sat, Sep 11, 2010 at 8:52 AM, Gael Varoquaux
Post by Gael Varoquaux
Post by Keith Goodman
I don't understand any of this but here's what I did. I removed
statsmodels and learn, the only scikits on my computer.
YEs, this coincides with what I had noticed: differents 'scikits'
installed means different namespace packages with the same prefix, and
noes gets lost.
Post by Keith Goodman
Next I installed learn and statsmodels and added my
scikits.learn.test() code to learn's __init__.py. That ran 0 tests. I
then replaced my code with the .test() code from statemodels 0.2; it
Ran 489 tests in 13.167s
FAILED (SKIP=1, failures=2)
   <nose.result.TextTestResult run=489 errors=0 failures=2>
That seems that the tests from statsmodel get run. I remembered that the
numpy 'test()' trick did not work with namespace packages and that was
one of the reasons that we were not using it.
It runs learn tests:

======================================================================
FAIL: Test BayesianRidge on diabetes
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.6/nose/case.py", line 183, in runTest
self.test(*self.arg)
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/glm/tests/test_bayes.py",
line 27, in XFailed_test_bayesian_on_diabetes
assert_array_equal(np.diff(clf.all_score_) > 0, True)
File "/usr/local/lib/python2.6/site-packages/numpy/testing/utils.py",
line 677, in assert_array_equal
verbose=verbose, header='Arrays are not equal')
File "/usr/local/lib/python2.6/site-packages/numpy/testing/utils.py",
line 609, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not equal

(mismatch 66.6666666667%)
x: array([False, True, True, True, False, False, False, False,
False], dtype=bool)
y: array(True, dtype=bool)

======================================================================
FAIL: test_train (scikits.learn.tests.test_gmm.TestGMMWithDiagonalCovars)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/tests/test_gmm.py",
line 384, in test_train
self.assertTrue(post_testll >= init_testll)
AssertionError

----------------------------------------------------------------------
Ran 489 tests in 13.167s

FAILED (SKIP=1, failures=2)
<nose.result.TextTestResult run=489 errors=0 failures=2>
Post by Gael Varoquaux
I'll give it a second look.
Post by Keith Goodman
$ nosetests --with-doctest
UserWarning: pyamg not available, using scipy.sparse
  warnings.warn('pyamg not available, using scipy.sparse')
All this is (unfortunately) normal. Mostly spurious erros because the
methods are slightly unstable.
However, we need to fix these failures, via better tests, and more stable
methods.
Gaël
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing
http://p.sf.net/sfu/novell-sfdev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Gael Varoquaux
2010-09-11 16:00:15 UTC
Permalink
Post by Keith Goodman
Post by Gael Varoquaux
Post by Keith Goodman
Next I installed learn and statsmodels and added my
scikits.learn.test() code to learn's __init__.py. That ran 0 tests. I
then replaced my code with the .test() code from statemodels 0.2; it
Ran 489 tests in 13.167s
FAILED (SKIP=1, failures=2)
   <nose.result.TextTestResult run=489 errors=0 failures=2>
That seems that the tests from statsmodel get run. I remembered that the
numpy 'test()' trick did not work with namespace packages and that was
one of the reasons that we were not using it.
======================================================================
FAIL: Test BayesianRidge on diabetes
----------------------------------------------------------------------
File "/usr/lib/pymodules/python2.6/nose/case.py", line 183, in runTest
self.test(*self.arg)
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/glm/tests/test_bayes.py",
line 27, in XFailed_test_bayesian_on_diabetes
assert_array_equal(np.diff(clf.all_score_) > 0, True)
File "/usr/local/lib/python2.6/site-packages/numpy/testing/utils.py",
line 677, in assert_array_equal
verbose=verbose, header='Arrays are not equal')
File "/usr/local/lib/python2.6/site-packages/numpy/testing/utils.py",
line 609, in assert_array_compare
raise AssertionError(msg)
Arrays are not equal
Ha! That's the scikit-learn, quite clearly. Looking at that right now.

Gaël
Gael Varoquaux
2010-09-11 19:09:14 UTC
Permalink
Post by Gael Varoquaux
Ha! That's the scikit-learn, quite clearly. Looking at that right now.
OK, so I added a 'test' function that runs all our tests, including the
doctests.

It was a bit nasty, because numpy's testing architecture is very fussy
about details. In particuliar, we now need to specify imports explicitely
in the doctests (that might actually be a good thing), we can't use
'print' in the doctests, and we need to make sure that there and some
things printed via the repr after a '>>>'.

Gaël
Keith Goodman
2010-09-11 19:19:45 UTC
Permalink
On Sat, Sep 11, 2010 at 12:09 PM, Gael Varoquaux
Post by Gael Varoquaux
OK, so I added a 'test' function that runs all our tests, including the
doctests.
It was a bit nasty, because numpy's testing architecture is very fussy
about details. In particuliar, we now need to specify imports explicitely
in the doctests (that might actually be a good thing), we can't use
'print' in the doctests, and we need to make sure that there and some
things printed via the repr after a '>>>'.
Here's what I get on Ubuntu 10.04:

$ git clone http://github.com/scikit-learn/scikit-learn.git
$ cd scikit-learn/
$ python setup.py build
$ sudo python setup.py install --prefix=/usr/local
$ cd
$ ipython
Post by Gael Varoquaux
Post by j***@gmail.com
import scikits.learn
scikits.learn.test()
Running unit tests and doctests for scikits.learn
NumPy version 1.4.1
NumPy is installed in /usr/local/lib/python2.6/site-packages/numpy
Python version 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3]
nose version 0.11.1
FE
======================================================================
ERROR: Doctest: scikits.learn.externals.joblib
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.6/nose/plugins/doctests.py", line
395, in tearDown
delattr(__builtin__, self._result_var)
AttributeError: _

======================================================================
FAIL: Doctest: scikits.learn.externals.joblib
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.6/doctest.py", line 2152, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for scikits.learn.externals.joblib
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/externals/joblib/__init__.py",
line 0, in joblib

----------------------------------------------------------------------
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/externals/joblib/__init__.py",
line 64, in scikits.learn.externals.joblib
Failed example:
from joblib import Memory
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.6/doctest.py", line 1248, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.externals.joblib[0]>", line 1, in <module>
from joblib import Memory
ImportError: No module named joblib
----------------------------------------------------------------------
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/externals/joblib/__init__.py",
line 65, in scikits.learn.externals.joblib
Failed example:
mem = Memory(cachedir='/tmp/joblib')
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.6/doctest.py", line 1248, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.externals.joblib[1]>", line 1, in <module>
mem = Memory(cachedir='/tmp/joblib')
NameError: name 'Memory' is not defined
----------------------------------------------------------------------
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/externals/joblib/__init__.py",
line 68, in scikits.learn.externals.joblib
Failed example:
square = mem.cache(np.square)
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.6/doctest.py", line 1248, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.externals.joblib[4]>", line 1, in <module>
square = mem.cache(np.square)
NameError: name 'mem' is not defined
----------------------------------------------------------------------
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/externals/joblib/__init__.py",
line 69, in scikits.learn.externals.joblib
Failed example:
b = square(a)
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.6/doctest.py", line 1248, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.externals.joblib[5]>", line 1, in <module>
b = square(a)
NameError: name 'square' is not defined
----------------------------------------------------------------------
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/externals/joblib/__init__.py",
line 77, in scikits.learn.externals.joblib
Failed example:
c = square(a)
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.6/doctest.py", line 1248, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.externals.joblib[6]>", line 1, in <module>
c = square(a)
NameError: name 'square' is not defined
----------------------------------------------------------------------
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/externals/joblib/__init__.py",
line 83, in scikits.learn.externals.joblib
Failed example:
from joblib import Parallel, delayed
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.6/doctest.py", line 1248, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.externals.joblib[7]>", line 1, in <module>
from joblib import Parallel, delayed
ImportError: No module named joblib
----------------------------------------------------------------------
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/externals/joblib/__init__.py",
line 85, in scikits.learn.externals.joblib
Failed example:
Parallel(n_jobs=1)(delayed(sqrt)(i**2) for i in range(10))
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.6/doctest.py", line 1248, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.externals.joblib[9]>", line 1, in <module>
Parallel(n_jobs=1)(delayed(sqrt)(i**2) for i in range(10))
NameError: name 'Parallel' is not defined
----------------------------------------------------------------------
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/externals/joblib/__init__.py",
line 105, in scikits.learn.externals.joblib
Failed example:
import shutil ; shutil.rmtree('/tmp/joblib/')
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.6/doctest.py", line 1248, in __run
compileflags, 1) in test.globs
File "<doctest scikits.learn.externals.joblib[10]>", line 1, in <module>
import shutil ; shutil.rmtree('/tmp/joblib/')
File "/usr/lib/python2.6/shutil.py", line 208, in rmtree
onerror(os.listdir, path, sys.exc_info())
File "/usr/lib/python2.6/shutil.py", line 206, in rmtree
names = os.listdir(path)
OSError: [Errno 2] No such file or directory: '/tmp/joblib/'


----------------------------------------------------------------------
Ran 1 test in 0.025s

FAILED (errors=1, failures=1)
<nose.result.TextTestResult run=1 errors=1 failures=1>
Gael Varoquaux
2010-09-11 22:17:43 UTC
Permalink
Post by Olivier Grisel
$ git clone http://github.com/scikit-learn/scikit-learn.git
$ cd scikit-learn/
$ python setup.py build
$ sudo python setup.py install --prefix=/usr/local
$ cd
$ ipython
Post by j***@gmail.com
import scikits.learn
scikits.learn.test()
Running unit tests and doctests for scikits.learn
NumPy version 1.4.1
NumPy is installed in /usr/local/lib/python2.6/site-packages/numpy
Python version 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3]
nose version 0.11.1
FE
[...]
FAILED (errors=1, failures=1)
<nose.result.TextTestResult run=1 errors=1 failures=1>
That's just completely broken. The tests did not run.

That test function hack definitely is fragile :). The very exact same
thing works for me, and I have:

Running unit tests and doctests for scikits.learn
NumPy version 2.0.0.dev
NumPy is installed in
/home/varoquau/usr/lib/python2.6/site-packages/numpy
SciPy version 0.9.0.dev
SciPy is installed in
/home/varoquau/usr/lib/python2.6/site-packages/scipy
Python version 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) [GCC 4.4.3]
nose version 0.11.1
PyAMG version 1.0.0.dev
PyAMG is installed in /home/varoquau/dev/pyamg/pyamg

I just tried with versions of scipy and numpy a bit older:

NumPy version 1.3.0
NumPy is installed in /usr/lib/python2.6/dist-packages/numpy
SciPy version 0.7.0
SciPy is installed in /usr/lib/python2.6/dist-packages/scipy
Python version 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) [GCC 4.4.3]
nose version 0.11.1
PyAMG version 1.0.0.dev
PyAMG is installed in /home/varoquau/dev/pyamg/pyamg

and it works also!

_____________

How crazy is that: it seems that I can replicate your problem is I
install to '/usr/local', but not if I install to '~/usr'.

Looking into that.

Gaël
Gael Varoquaux
2010-09-11 22:25:05 UTC
Permalink
Post by Gael Varoquaux
How crazy is that: it seems that I can replicate your problem is I
install to '/usr/local', but not if I install to '~/usr'.
That's because when it is install to '/usr/local', all files are turned
executable.

That's outrageously broken, I have added a fix to workaround the problem.
It should now work, and you should get around 519 test ran, with 3 to 4
broken.

Thanks for your feedback,

Gaël
Keith Goodman
2010-09-12 00:01:53 UTC
Permalink
On Sat, Sep 11, 2010 at 3:25 PM, Gael Varoquaux
Post by Gael Varoquaux
Post by Gael Varoquaux
How crazy is that: it seems that I can replicate your problem is I
install to '/usr/local', but not if I install to '~/usr'.
That's because when it is install to '/usr/local', all files are turned
executable.
That's outrageously broken, I have added a fix to workaround the problem.
It should now work, and you should get around 519 test ran, with 3 to 4
broken.
Thanks for your feedback,
Nice work!

Ran 520 tests in 13.597s
FAILED (SKIP=1, failures=3)
<nose.result.TextTestResult run=520 errors=0 failures=3>
Gael Varoquaux
2010-09-12 08:16:08 UTC
Permalink
Post by Keith Goodman
Ran 520 tests in 13.597s
FAILED (SKIP=1, failures=3)
<nose.result.TextTestResult run=520 errors=0 failures=3>
Yey! Now we have to fix the 3 failing tests.

Skipper Seabold
2010-09-12 00:10:02 UTC
Permalink
On Sat, Sep 11, 2010 at 6:25 PM, Gael Varoquaux
Post by Gael Varoquaux
Post by Gael Varoquaux
How crazy is that: it seems that I can replicate your problem is I
install to '/usr/local', but not if I install to '~/usr'.
That's because when it is install to '/usr/local', all files are turned
executable.
That's outrageously broken, I have added a fix to workaround the problem.
It should now work, and you should get around 519 test ran, with 3 to 4
broken.
How did you fix this out of curiosity? I had to monkeypatch
numpy.testing.Tester to run executables.

Skipper
Gael Varoquaux
2010-09-12 08:15:29 UTC
Permalink
Post by Skipper Seabold
Post by Gael Varoquaux
That's outrageously broken, I have added a fix to workaround the problem.
It should now work, and you should get around 519 test ran, with 3 to 4
broken.
How did you fix this out of curiosity? I had to monkeypatch
numpy.testing.Tester to run executables.
I had a look at how you had done it, and did it in a slightly less
invasive way, but with the same idea:
http://github.com/scikit-learn/scikit-learn/blob/master/scikits/learn/__init__.py#L30

Gaël
j***@gmail.com
2010-09-11 17:23:45 UTC
Permalink
Post by Keith Goodman
On Sat, Sep 11, 2010 at 12:54 AM, Gael Varoquaux
Post by Gael Varoquaux
I believe that the problem is that Keith has other scikits installed that
define other namespace packages. Nosetest clearly doesn't handle
namespace package, I have had the problem too. I can't blame nosetests
for that, I believe that namespace packages are really an ugly hack.
Unfortunately, they are a standard in the Python world, and nosetests
should fix their code.
I don't understand any of this but here's what I did. I removed
statsmodels and learn, the only scikits on my computer. Then
$ git clone http://github.com/scikit-learn/scikit-learn.git
$ cd scikit-learn
$ make
Ran 523 tests in 28.703s
FAILED (SKIP=1, failures=3)
make: *** [test] Error 1
I then ran (are the failures know?) nosetests --with-doctest (see below)
Next I installed learn and statsmodels and added my
scikits.learn.test() code to learn's __init__.py. That ran 0 tests. I
then replaced my code with the .test() code from statemodels 0.2; it
Ran 489 tests in 13.167s
FAILED (SKIP=1, failures=2)
  <nose.result.TextTestResult run=489 errors=0 failures=2>
I think then, this is the problem that Skipper found, when testfiles
are not executable on Linux, or something like this.
One solution is to change the file properties/permissions for the test
files. Or a test files on unix supposed to have a shebang ?

I don't remember the details since this problem doesn't show up on Windows.

Josef
Keith Goodman
2010-09-10 19:19:20 UTC
Permalink
On Fri, Sep 10, 2010 at 12:10 PM, Alexandre Gramfort
Post by Alexandre Gramfort
nosetests scikits/learn/tests/*
it won't run all tests but at least you should see something
$ nosetests /usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/tests

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
$ nosetests /usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/tests/*
........../usr/local/lib/python2.6/site-packages/numpy/lib/utils.py:140:
DeprecationWarning: `unique1d` is deprecated!
warnings.warn(depdoc, DeprecationWarning)
...................F........................................................................................................................................../usr/local/lib/python2.6/site-packages/scipy/cluster/vq.py:570:
UserWarning: One of the clusters is empty. Re-run kmean with a
different initialization.
warnings.warn("One of the clusters is empty. "
..............................................................................................S....S..................................................................
======================================================================
FAIL: test_train (scikits.learn.tests.test_gmm.TestGMMWithFullCovars)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/tests/test_gmm.py",
line 384, in test_train
self.assertTrue(post_testll >= init_testll)
AssertionError

----------------------------------------------------------------------
Ran 334 tests in 20.159s

FAILED (SKIP=2, failures=1)
Alexandre Gramfort
2010-09-10 19:21:25 UTC
Permalink
ok that's better. Unfortunately you should have 2/3 tests failing
since the last days sprint. It will be fixed soon.

Alex
Post by Keith Goodman
On Fri, Sep 10, 2010 at 12:10 PM, Alexandre Gramfort
Post by Alexandre Gramfort
nosetests scikits/learn/tests/*
it won't run all tests but at least you should see something
$ nosetests /usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/tests
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
$ nosetests /usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/tests/*
DeprecationWarning: `unique1d` is deprecated!
 warnings.warn(depdoc, DeprecationWarning)
UserWarning: One of the clusters is empty. Re-run kmean with a
different initialization.
 warnings.warn("One of the clusters is empty. "
..............................................................................................S....S..................................................................
======================================================================
FAIL: test_train (scikits.learn.tests.test_gmm.TestGMMWithFullCovars)
----------------------------------------------------------------------
 File "/usr/local/lib/python2.6/dist-packages/scikits.learn-0.5_git-py2.6-linux-x86_64.egg/scikits/learn/tests/test_gmm.py",
line 384, in test_train
   self.assertTrue(post_testll >= init_testll)
AssertionError
----------------------------------------------------------------------
Ran 334 tests in 20.159s
FAILED (SKIP=2, failures=1)
Olivier Grisel
2010-09-10 19:14:23 UTC
Permalink
Post by Keith Goodman
On Fri, Sep 10, 2010 at 11:57 AM, Alexandre Gramfort
Post by Alexandre Gramfort
nosetests /path/to/scikits/learn
Try:

$ git clone http://github.com/scikit-learn/scikit-learn.git
$ cd scikit-learn
$ make

It should build the C extensions inplace and launch nosetests from the
right position.
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
Keith Goodman
2010-09-10 19:26:45 UTC
Permalink
On Fri, Sep 10, 2010 at 12:14 PM, Olivier Grisel
Post by Olivier Grisel
Post by Keith Goodman
On Fri, Sep 10, 2010 at 11:57 AM, Alexandre Gramfort
Post by Alexandre Gramfort
nosetests /path/to/scikits/learn
$ git clone http://github.com/scikit-learn/scikit-learn.git
$ cd scikit-learn
$ make
It should build the C extensions inplace and launch nosetests from the
right position.
It only runs the coverage tests for me. I do get a lot of warnings like this:

Ignoring attempt to set 'name' (from 'scikits.learn' to
'scikits.learn.features/tests')
Warning: Assuming default configuration
(scikits/learn/cluster/{setup_cluster,setup}.py was not found)
Loading...