We’re using a mix of Django’s Unit Testing and pytest with pytest-django for our automated testing. This gives us a lot of power and flexibility to test all aspects of the site.
Configuration for your unit tests is handled automatically. The only
thing you’ll need to ensure is that the database credentials in your settings
has full permissions to modify a database with
test_ prepended to it. By
default the database name is
olympia, so the test database is
Optionally, in particular if the code you are working on is related to search,
you’ll want to run Elasticsearch tests. Obviously, you need Elasticsearch to be
installed. See Elasticsearch page for details.
If you don’t want to run the Elasticsearch tests, you can use the
test_no_es target in the Makefile:
On the contrary, if you only want to run Elasticsearch tests, use the
To run the whole test suite use:
There are a lot of options you can pass to adjust the output. Read pytest and pytest-django docs for the full set, but some common ones are:
-vto provide more verbose information about the test run
-stells pytest to not capture the logging output
--create-dbtells pytest-django to recreate the database instead of reusing the one from the previous run
-x --pdbto stop on the first failure, and drop into a python debugger
--lfto re-run the last test failed
-m test_eswill only run tests that are marked with the
-k foobarwill only run tests that contain
foobarin their name
There are a few useful makefile targets that you can use:
Run all the tests:
If you need to rebuild the database:
To fail and stop running tests on the first failure:
If you wish to add arguments, or run a specific test, overload the variables (check the Makefile for more information):
make test ARGS='-v src/olympia/amo/tests/test_url_prefix.py::MiddlewareTest::test_get_app'
If you wish to re-run only the tests failed from the previous run:
Our test runner is configured by default to reuse the database between each
test run. If you really want to make a new database (e.g. when models have
changed), use the
We support two types of automated tests right now and there are some details below but remember, if you’re confused look at existing tests for examples.
Also, take some time to get familiar with pytest way of dealing with dependency injection, which they call fixtures (which should not be confused with Django’s fixtures). They are very powerful, and can make your tests much more independent, cleaner, shorter, and more readable.
Most tests are in this category. Our test classes extend
django.test.TestCase and follow the standard rules for unit tests.
We’re using JSON fixtures for the data.
Connecting to remote services in tests is not recommended, developers should mock out those calls instead.
Why Tests Fail
Tests usually fail for one of two reasons: The code has changed or the data has changed. An third reason is time. Some tests have time-dependent data usually in the fixtures. For example, some featured items have expiration dates.
We can usually save our future-selves time by setting these expirations far in the future.
If you want test that your localization works then you can add in locales
in the test directory. For an example see
devhub/tests/locale. These locales
are not in the normal path so should not show up unless you add them to the
LOCALE_PATH. If you change the .po files for these test locales, you will
need to recompile the .mo files manually, for example:
msgfmt --check-format -o django.mo django.po