Thursday, October 15, 2009

When your Django doctests never fail...

Grr... don't believe the Django test runner when it claims that your doctests pass! I was eagerly working on issue #1 and wrote a new test application called 20091014, (the lino.test_apps package is a convenient way to write such quick standalone Django tests: I just change INSTALLED_APPS = ( 'lino.test_apps.20091014',  ) in my settings.py and then I can do "python manage.py test"). This usually works fine, but now I noticed that after the last changes it says:
Lino version 0.0.1
Creating test database...
Creating table 20091014_a
Creating table 20091014_b
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
Destroying test database...


Which means that it actually ignored the whole doctest string, as if there were no test at all. I read here and there about other people who had had similar problems. It still took me some time to discover my mistake. Afterwards I made the following experiment in a file tmp.py:
"""
First test:

  >>> 9+2
  11
 
  >>> foo() #doctest: +IGNORE_EXCEPTION_DETAIL
  Traceback (most recent call last):
  (...)
  NameError: name 'foo' is not defined 
 
"""

if __name__ == '__main__':
    import doctest
    doctest.testmod()

python tmp.py says nothing, which means that all tests passed. Then I removed one space in front of the word "Traceback". This is in fact the syntax error I had introduced by mistak into my lino.test_app docstring. Here is how Python reacts in such a case:

Traceback (most recent call last):
  File "tmp.py", line 16, in
    doctest.testmod()
  File "c:\Python25\lib\doctest.py", line 1814, in testmod
    for test in finder.find(m, name, globs=globs, extraglobs=extraglobs):
  File "c:\Python25\lib\doctest.py", line 839, in find
    self._find(tests, obj, name, module, source_lines, globs, {})
  File "c:\Python25\lib\doctest.py", line 881, in _find
    test = self._get_test(obj, name, module, globs, source_lines)
  File "c:\Python25\lib\doctest.py", line 965, in _get_test
    filename, lineno)
  File "c:\Python25\lib\doctest.py", line 594, in get_doctest
    return DocTest(self.get_examples(string, name), globs,
  File "c:\Python25\lib\doctest.py", line 608, in get_examples
    return [x for x in self.parse(string, name)
  File "c:\Python25\lib\doctest.py", line 570, in parse
    self._parse_example(m, name, lineno)
  File "c:\Python25\lib\doctest.py", line 640, in _parse_example
    lineno + len(source_lines))
  File "c:\Python25\lib\doctest.py", line 726, in _check_prefix
    (lineno+i+1, name, line))
ValueError: line 8 of the docstring for __main__ has inconsistent leading whitespace: ' Traceback (most recent call last
):'


The Django test runner ignores such docstring syntax errors without any warning.
I'm using today's Django development version (revision 11624).

Django in fact has already a ticket #11436 which reveals that they are waiting for Python issue 1521051 to be fixed... for me that's enough to forgive them.

No comments:

Post a Comment