• Simon Glass's avatar
    doc: test: Explain how to run pytests in parallel · 9666220d
    Simon Glass authored
    
    
    Add documentation for this so people can try it out. At present it does
    not fully work.
    
    Series-to: u-boot
    Series-cc: heinrich, stephen, trini
    Series-links: 254822
    Series-prefix: RESEND
    Series-version: 2
    Series-changes: 2
    - Add documentation for how to run parallel tests
    
    Cover-letter:
    test: Try to deal with some co-dependent tests
    Tests are supposed to be independent. With driver model tests, the
    environment is reset before each test, which ensures that.
    
    With Python tests there is no reset of the board between tests, since we
    want to run all the tests as quickly as possible and without needing the
    external scripts running constantly.
    
    In principle the Python tests can be independent if they each put the
    world back the way they found it, but it turns out that some are not.
    This means that some tests cannot be run unless another test is run
    first. It also means that tests cannot be run in parallel, e.g. on
    sandbox.
    
    This series fixes some of them. Those that remain:
    
       test_gpt_swap_partitions - not sure?
       test_pinmux_status - not sure?
       test_sqfs_load - cannot be run more than once!
       test_bind_unbind_with_uclass - relies on previous test
    
    The last one would be much better done as a C test, so it doesn't have
    to deal with the changing driver tree. There isn't a lot of value in
    running the test on a real board, since sandbox should find any bugs
    in driver model or the 'bind' command.
    
    If the above can be resolved we can enable parallel tests. On my test
    machine (32 threads) it reduces the time from 38 seconds to 7.5s
    
    To use this feature, see the documentaiton added, or:
    
       pip3 install pytest-xdist
    
       test/py/test.py -B sandbox --build-dir /tmp/xx -q -k 'not slow' -n32
    END
    Signed-off-by: Simon Glass's avatarSimon Glass <sjg@chromium.org>
    Change-Id: I4b868f615036d4b8bc4d9f6a8555db49be9f6009
    9666220d
py_testing.rst 17.5 KB