xpcshell tests are quick-to-run tests, that are generally used to write unit tests. They do not have access to the full browser chrome like browser chrome tests, and so have much lower overhead. They are typical run by using ./mach xpcshell-test which initiates a new xpcshell session with the xpcshell testing harness. Anything available to the XPCOM layer (through scriptable interfaces) can be tested with xpcshell. See Mozilla automated testing and pages tagged "automated testing" for more information.
xpcshell test filenames must start with test_.
If you need to create a new test directory, then follow the steps here. The test runner needs to know about the existence of the tests and how to configure them through the use of the xpcshell.ini manifest file.
First add a XPCSHELL_TESTS_MANIFESTS += ['xpcshell.ini'] declaration (with the correct relative xpcshell.ini path) to the moz.build file located in or above the directory.
Then create an empty xpcshell.ini file to tell the build system about the individual tests, and provide any additional configuration options.
If you're creating a new test in an existing directory, you can simply run:
$ ./mach addtest path/to/test/test_example.js $ hg add path/to/test/test_example.js
This will automatically create the test file and add it to xpcshell.ini, the second line adds it to your commit.
The test file contains an empty test which will give you an idea of how to write a test. There are plenty more examples throughout mozilla-central.
To run the test, execute it by running the mach command from the root of the Gecko source code directory.
# Run a single test: $ ./mach xpcshell-test path/to/tests/test_example.js # Test an entire test suite in a folder: $ ./mach xpcshell-test path/to/tests/ # Or run any type of test, including both xpcshell and browser chrome tests: $ ./mach test path/to/tests/test_example.js
The test is executed by the testing harness. It will call in turn:
run_test (if it exists).add_task or add_test in the order they were defined in the file.See also the notes below around add_task and add_test.
xpcshell tests have access to the following functions. They are defined in testing/xpcshell/head.js and testing/modules/Assert.jsm.
Assert.ok(truthyOrFalsy[, message])Assert.equal(actual, expected[, message])Assert.notEqual(actual, expected[, message])Assert.deepEqual(actual, expected[, message])Assert.notDeepEqual(actual, expected[, message])Assert.strictEqual(actual, expected[, message])Assert.notStrictEqual(actual, expected[, message])Assert.rejects(actual, expected[, message])Assert.greater(actual, expected[, message])Assert.greaterOrEqual(actual, expected[, message])Assert.less(actual, expected[, message])Assert.lessOrEqual(actual, expected[, message])Assert. from the beginning of the name, e.g. ok(true) rather than Assert.ok(true), however keeping the Assert. prefix may be seen as more descriptive and easier to spot where the tests are.Assert.throws(callback, expectedException[, message])Assert.throws(callback[, message])expectedException argument can be an Error instance, or a regular expression matching part of the error message (like in Assert.throws(() => a.b, /is not defined/).Assert.rejects(promise, expectedException[, message])await. The expectedException argument can be an Error instance, or a regular expression matching part of the error message. Example: await Assert.rejects(myPromise, /bad response/);add_task([condition, ]testFunc)awaits a Promise, the test runner waits until the promise is resolved or rejected before proceeding. Rejected promises are converted into exceptions, and resolved promises are converted into values.add_task(), the run_test() function is optional, but if present, it should also call run_next_test() to start execution of all asynchronous test functions. The test cases must not call run_next_test(), it is called automatically when the task finishes. See Async tests, below, for more information.add_test([condition, ]testFunction)run_next_test() when it's done. For tests that use add_test(), the run_test() function is optional, but if present, it should also call run_next_test() to start execution of all asynchronous test functions. In most cases, you should rather use the more readable variant add_task(). See Async tests, below, for more information.run_next_test()run_next_test() when it's done. run_test() should also call run_next_test() to start execution of all asynchronous test functions. See Async tests, below, for more information.registerCleanupFunction(callback)callback after the current JS test file has finished running, regardless of whether the tests inside it pass or fail. You can use this to clean up anything that might otherwise cause problems between test runs.callback returns a Promise, the test will not finish until the promise is fulfilled or rejected (making the termination function asynchronous).do_test_pending()do_test_finished()do_get_file(testdirRelativePath, allowNonexistent)nsILocalFile object representing the given file (or directory) in the test directory. For example, if your test is unit/test_something.js, and you need to access unit/data/somefile, you would call do_get_file('data/somefile'). The given path must be delimited with forward slashes. You can use this to access test-specific auxiliary files if your test requires access to external files. Note that you can also use this function to get directories.
XPCSHELL_TESTS. For an example, see netwerk/test/Makefile.in#117.do_get_profile()nsILocalFile object representing that directory. It also makes sure that the profile-change-net-teardown, profile-change-teardown, and profile-before-change observer notifications are sent before the test finishes. This is useful if the components loaded in the test observe them to do cleanup on shutdown (e.g., places).
do_register_cleanup will perform any cleanup operation before the profile and the network is shut down by the observer notifications.do_get_idle()do_get_cwd()nsILocalFile object representing the test directory. This is the directory containing the test file when it is currently being run. Your test can write to this directory as well as read any files located alongside your test. Your test should be careful to ensure that it will not fail if a file it intends to write already exists, however.load(testdirRelativePath)testdirRelativePath into the global script context, executing the code inside it. The file specified is a file within the test directory. For example, if your test is unit/test_something.js and you have another file unit/extra_helpers.js, you can load the second file from the first by calling load('extra_helpers.js').do_parse_document(path, type)executeSoon(callback)callback on a later pass through the event loop. Use this when you want some code to execute after the current function has finished executing, but you don't care about a specific time delay. This function will automatically insert a do_test_pending / do_test_finished pair for you.do_timeout(delay, fun)do_test_pending so that the test isn't completed before your timer fires, and you must call do_test_finished when the actions you perform in the timeout complete, if you have no other functionality to test. (Note: the function argument used to be a string argument to be passed to eval, and some older branches support only a string argument or support both string and function.)do_send_remote_message(name, optionalData)do_await_remote_message or equivalent ProcessMessageManager listeners.do_await_remote_message(name, optionalCallback) do_send_remote_message or equivalent ProcessMessageManager calls. If optionalCallback is provided, the callback must call do_test_finished. If optionalData is passed to do_send_remote_message then that data is the first argument to optionalCallback or the value to which the promise resolves.The manifest controls what tests are included in a test suite, and the configuration of the tests. It is loaded via the `moz.build` property configuration proprety.
The following are all of the configuration options for a test suite as listed under the [DEFAULT] section of the manifest.
tags./mach xpcshell-test --tag TAGNAMEheadfirefox-appdirskip-ifrun-iffail-ifsupport-filesresource://test/[filename] path to the tests. The path can be relative to other directories, but it will be served only with the filename. See Test head and support files for more information and usage.[test_*]test_ and are listed in square bracketsWhen creating a new directory and new xpcshell.ini manifest file, the following must be added to a moz.build file near that file in the directory hierarchy:
XPCSHELL_TESTS_MANIFESTS += ['path/to/xpcshell.ini']
Typically, the moz.build containing XPCSHELL_TESTS_MANIFESTS is not in the same directory as xpcshell.ini, but rather in a parent directory. Common directory structures look like:
feature
├──moz.build
└──tests/xpcshell
└──xpcshell.ini
# or
feature
├──moz.build
└──tests
├──moz.build
└──xpcshell
└──xpcshell.ini
Typically in a test suite, similar setup code and dependencies will need to be loaded in across each test. This can be done through the test head, which is the file declared in the xpcshell.ini manifest file under the head property. The file itself is typically called head.js. Any variable declared in the test head will be in the global scope of each test in that test suite.
In addition to the test head, other support files can be declared in the xpcshell.ini manifest file. This is done through the support-files declaration. These files will be made available through the url resource://test plus the name of the file. These files can then be loaded in using the Components.utils.import function or other loaders. The support files can be located in other directory as well, and they will be made available by their filename.
# File structure: path/to/tests ├──head.js ├──module.jsm ├──moz.build ├──test_example.js └──xpcshell.ini
# xpcshell.ini [DEFAULT] head = head.js support-files = ./module.jsm ../../some/other/file.js [test_component_state.js]
// head.js
var globalValue = "A global value.";
// Import support-files.
Components.utils.import("resource://test/module.jsm");
Components.utils.import("resource://test/file.jsm");
// test_example.js
function run_test() {
equal(globalValue, "A global value.", "Declarations in head.js can be accessed");
}
Asynchronous tests (that is, those whose success cannot be determined until after run_test finishes) can be written in a variety of ways.
The easiest is using the add_task helper. add_task can take an asynchronous function as a parameter. add_task tests are run automatically if you don't have a run_test function.
add_task(async function test_foo() {
let foo = await makeFoo(); // makeFoo() returns a Promise<foo>
equal(foo, expectedFoo, "Should have received the expected object");
});
add_task(async function test_bar() {
let foo = await makeBar(); // makeBar() returns a Promise<bar>
Assert.equal(bar, expectedBar, "Should have received the expected object");
});
You can also use add_test, which takes a function and adds it to the list of asynchronously-run functions. Each function given to add_test must also call run_next_test at its end. You should normally use add_task instead of add_test, but you may see add_test in existing tests.
add_test(function test_foo() {
makeFoo(function callback(foo) { // makeFoo invokes a callback<foo> once completed
equal(foo, expectedFoo);
run_next_test();
});
});
add_test(function test_bar() {
makeBar(function callback(bar) {
equal(bar, expectedBar);
run_next_test();
});
});
We can also tell the test harness not to kill the test process once run_test() is finished, but to keep spinning the event loop until our callbacks have been called and our test has completed. Newer tests prefer the use of add_task rather than this method. This can be achieved with do_test_pending() and do_test_finished():
function run_test() {
// Tell the harness to keep spinning the event loop at least
// until the next do_test_finished() call.
do_test_pending();
someAsyncProcess(function callback(result) {
equal(result, expectedResult);
// Close previous do_test_pending() call.
do_test_finished();
});
}
By default xpcshell tests run in the parent process. If you wish to run test logic in the child, you have several ways to do it:
run_test_in_child() function to run an entire script file in the child. This is an easy way to arrange for a test to be run twice, once in chrome and then later (via the _wrap.js file) in content. See /network/test/unit_ipc for examples. The run_test_in_child() function takes a callback, so you should be able to call it multiple times with different files, if that's useful.sendCommand() function, which takes a code string to be executed on the child, and a callback function to be run on the parent when it has completed. You will want to first call do_load_child_test_harness() to set up a reasonable test environment on the child. sendCommand returns immediately, so you will generally want to use do_test_pending/do_test_finished with it. NOTE: this method of test has not been used much, and your level of pain may be significant. Consider option #1 if possible.See the documentation for run_test_in_child() and do_load_child_test_harness() in testing/xpcshell/head.js for more information.
Sometimes you might want a test to know what platform it's running on (to test platform-specific features, or allow different behaviors). Unit tests are not normally invoked from a Makefile (unlike Mochitests), or preprocessed (so not #ifdefs), so platform detection with those methods isn't trivial.
Some tests will want to only execute certain portions on specific platforms. Use AppConstants.jsm for determing the platform, for example:
ChromeUtils.import("resource://gre/modules/AppConstants.jsm");
let isMac = AppConstants.platform == "macosx";
There are two different ways to conditional skip a test, either through
add_task or add_test functionYou can use conditionals on individual test functions instead of entire files. The condition is provided as an optional first parameter passed into add_task() or add_test(). The condition is an object which contains a function named skip_if(), which is an arrow function returning a boolean value which is true if the test should be skipped.
For example, you can provide a test which only runs on Mac OS X like this:
ChromeUtils.import("resource://gre/modules/AppConstants.jsm");
add_task({
skip_if: () => AppConstants.platform != "mac"
}, async function some_test() {
// Test code goes here
});
Since AppConstants.platform != "mac" is true only when testing on Mac OS X, the test will be skipped on all other platforms.
Note: Arrow functions are ideal here because if your condition compares constants, it will already have been evaluated before the test is even run, meaning your output will not be able to show the specifics of what the condition is.
Sometimes you may want to add conditions to specify that a test should be skipped in certain configurations, or that a test is known to fail on certain platforms. You can do this in xpcshell manifests by adding annotations below the test file entry in the manifest, for example:
[test_example.js] skip-if = os == 'win'
This example would skip running test_example.js on Windows.
Note: Starting with Gecko (Firefox 40 / Thunderbird 40 / SeaMonkey 2.37), you can use conditionals on individual test functions instead of on entire files. See Adding conditions through the add_task or add_test function above for details.
There are currently four conditionals you can specify:
skip-if tells the harness to skip running this test if the condition evaluates to true. You should use this only if the test has no meaning on a certain platform, or causes undue problems like hanging the test suite for a long time.
run-if tells the harness to only run this test if the condition evaluates to true. It functions as the inverse of skip-if.
fail-if tells the harness that this test is expected to fail if the condition is true. If you add this to a test, make sure you file a bug on the failure and include the bug number in a comment in the manifest, like:
[test_example.js] # bug xxxxxx fail-if = os == 'linux'
run-sequentially basically tells the harness to run the respective test in isolation. This is required for tests that are not "thread-safe". You should do all you can to avoid using this option, since this will kill performance. However, we understand that there are some cases where this is imperative, so we made this option available. If you add this to a test, make sure you specify a reason and possibly even a bug number, like:
[test_example.js] run-sequentially = Has to launch Firefox binary, bug 123456.
For a more detailed description of the syntax of the conditional expressions, as well as what variables are available, see this page.
When working on a specific feature or issue, it is convenient to only run a specific task from a whole test suite. Use .only() for that purpose:
add_task(async function some_test() {
// Some test.
});
add_task(async function some_interesting_test() {
// Only this test will be executed.
}).only();
Events are not processed during test execution if not explicitly triggered. This sometimes causes issues during shutdown, when code is run that expects previously created events to have been already processed. In such cases, this code at the end of a test can help:
let thread = gThreadManager.currentThread; while (thread.hasPendingEvents()) thread.processNextEvent(true);
You can specify flags when issuing the xpcshell-test command that will cause your test to stop right before running so you can attach the javascript debugger.
Example:
$ ./mach xpcshell-test --jsdebugger browser/components/tests/unit/test_browserGlue_pingcentre.js 0:00.50 INFO Running tests sequentially. ... 0:00.68 INFO "" 0:00.68 INFO "*******************************************************************" 0:00.68 INFO "Waiting for the debugger to connect on port 6000" 0:00.68 INFO "" 0:00.68 INFO "To connect the debugger, open a Firefox instance, select 'Connect'" 0:00.68 INFO "from the Developer menu and specify the port as 6000" 0:00.68 INFO "*******************************************************************" 0:00.68 INFO "" 0:00.71 INFO "Still waiting for debugger to connect..." ...
At this stage in a running Firefox instance:
Web Developer -> Remote Debugginglocalhost:6000 and select ConnectMain Process, click it and the Developer Tools debugger window will open.If you get a message such as:
0:00.62 ERROR Failed to initialize debugging: Error: resource://devtools appears to be inaccessible from the xpcshell environment. This can usually be resolved by adding: firefox-appdir = browser to the xpcshell.ini manifest. It is possible for this to alter test behevior by triggering additional browser code to run, so check test behavior after making this change.
This is typically a test in core code. You can attempt to add that to the xpcshell.ini, however as it says, it might affect how the test runs and cause failures. Generally the firefox-appdir should only be left in xpcshell.ini for tests that are in the browser/ directory, or are Firefox-only.
--debugger and -debugger-interactiveYou can specify flags when issuing the xpcshell-test command that will launch xpcshell in the specified debugger (implemented in bug 382682). Provide the full path to the debugger, or ensure that the named debugger is in your system PATH.
Example:
$ ./mach xpcshell-test --debugger gdb --debugger-interactive netwerk/test/unit/test_resumable_channel.js # js>_execute_test(); ...failure or success messages are printed to the console... # js>quit();
On Windows with the VS debugger:
$ ./mach xpcshell-test --debugger devenv --debugger-interactive netwerk/test/test_resumable_channel.js
Or with WinDBG:
$ ./mach xpcshell-test --debugger windbg --debugger-interactive netwerk/test/test_resumable_channel.js
Or with modern WinDbg (WinDbg Preview as of April 2020):
$ ./mach xpcshell-test --debugger WinDbgX --debugger-interactive netwerk/test/test_resumable_channel.js
To debug the child process, where code is often being run in a project, set MOZ_DEBUG_CHILD_PROCESS=1 in your environment (or on the command line) and run the test. You will see the child process emit a printf with its process ID, then sleep. Attach a debugger to the child's pid, and when it wakes up you can debug it:
$ MOZ_DEBUG_CHILD_PROCESS=1 ./mach xpcshell-test test_simple_wrap.js CHILDCHILDCHILDCHILD debug me @13476
Use MOZ_DEBUG_CHILD_PROCESS=1 to attach debuggers to each process. (For gdb at least, this means running separate copies of gdb, one for each process.)