pytest API Manual
Guide
- April 2015 is “adopt pytest month”
- Release announcements
- py.test 2.0.0: asserts++, unittest++, reporting++, config++, docs++
- py.test 2.0.1: bug fixes
- py.test 2.0.2: bug fixes, improved xfail/skip expressions, speed ups
- py.test 2.0.3: bug fixes and speed ups
- py.test 2.1.0: perfected assertions and bug fixes
- py.test 2.1.1: assertion fixes and improved junitxml output
- py.test 2.1.2: bug fixes and fixes for jython
- py.test 2.1.3: just some more fixes
- py.test 2.2.0: test marking++, parametrization++ and duration profiling
- pytest-2.2.1: bug fixes, perfect teardowns
- pytest-2.2.2: bug fixes
- pytest-2.2.4: bug fixes, better junitxml/unittest/python3 compat
- pytest-2.3: improved fixtures / better unittest integration
- pytest-2.3.1: fix regression with factory functions
- pytest-2.3.2: some fixes and more traceback-printing speed
- pytest-2.3.3: integration fixes, py24 support, */** shown in traceback
- pytest-2.3.4: stabilization, more flexible selection via “-k expr”
- pytest-2.3.5: bug fixes and little improvements
- pytest-2.4.0: new fixture features/hooks and bug fixes
- pytest-2.4.1: fixing three regressions compared to 2.3.5
- pytest-2.4.2: colorama on windows, plugin/tmpdir fixes
- pytest-2.5.0: now down to ZERO reported bugs!
- pytest-2.5.1: fixes and new home page styling
- pytest-2.5.2: fixes
- pytest-2.6.0: shorter tracebacks, new warning system, test runner compat
- pytest-2.6.1: fixes and new xfail feature
- Changes 2.6.1
- pytest-2.6.2: few fixes and cx_freeze support
- pytest-2.6.3: fixes and little improvements
- Changes 2.6.3
- pytest-2.7.0: fixes, features, speed improvements
- pytest-2.7.1: bug fixes
- pytest-2.7.2: bug fixes
- pytest-2.8.2: bug fixes
- pytest-2.8.3: bug fixes
- pytest-2.8.4
- pytest-2.8.5
- pytest-2.8.6
- pytest-2.8.7
- pytest-2.9.0
- pytest-2.9.1
- pytest-2.9.2
- pytest-3.0.0
- pytest-3.0.1
- pytest-3.0.2
- pytest-3.0.3
- pytest-3.0.4
- pytest-3.0.5
- pytest-3.0.6
- pytest-3.0.7
- pytest-3.1.0
- pytest-3.1.1
- pytest-3.1.2
- pytest-3.1.3
- pytest-3.2.0
- pytest-3.2.1
- pytest-3.2.2
- pytest-3.2.3
- pytest-3.2.4
- pytest-3.2.5
- pytest-3.3.0
- pytest-3.3.1
- pytest-3.3.2
- pytest-3.4.0
- pytest-3.4.1
- pytest-3.4.2
- pytest-3.5.0
- pytest-3.5.1
- pytest-3.6.0
- pytest-3.6.1
- pytest-3.6.2
- pytest-3.6.3
- pytest-3.6.4
- pytest-3.7.0
- pytest-3.7.1
- pytest-3.7.2
- python testing sprint June 20th-26th 2016
- The writing and reporting of assertions in tests
- Backwards Compatibility Policy
- Setting up bash completion
- Pytest API and builtin fixtures
- Cache: working with cross-testrun state
- Capturing of the stdout/stderr output
- Changelog history
- Contact channels
- Full pytest documentation
- Contribution getting started
- Configuration
- Development Guide
- Doctest integration for modules and test files
- Examples and customization tricks
- Working with custom markers
- Working with non-python tests
- Parametrizing tests
- Changing standard (Python) test discovery
- Demo of Python failure reports with pytest
- Basic patterns and examples
- A session-fixture which can look at all collected tests
- Using pytest with an existing test suite
- Some Issues and Questions
- pytest fixtures: explicit, modular, scalable
- pytest-2.3: reasoning for fixture/funcarg evolution
- Installation and Getting Started
- Good Integration Practices
- Historical Notes
- pytest: helps you write better programs
- License
- Logging
- Marking test functions with attributes
- Monkeypatching/mocking modules and environments
- Running tests written for nose
- Parametrizing fixtures and test functions
- Installing and Using plugins
- Project examples
- PROPOSAL: Parametrize with fixtures
- pytest import mechanisms and sys.path/PYTHONPATH
- Reference
- Skip and xfail: dealing with tests that cannot succeed
- Talks and Tutorials
- Temporary directories and files
- unittest.TestCase Support
- Usage and Invocations
- Warnings Capture
- Writing plugins
- Writing hook functions
- classic xunit-style setup
- “yield_fixture” functions
Section
- The ideal pytest helper
- The ideal partner project
- What does it mean to “adopt pytest”?
- Other ways to help
- New Features
- Fixes
- Important Notes
- (Incompatible) Removals
- Changes between 2.0.0 and 2.0.1
- Changes between 2.0.1 and 2.0.2
- Changes between 2.0.2 and 2.0.3
- Changes between 2.0.3 and 2.1.0
- Changes between 2.1.0 and 2.1.1
- Changes between 2.1.1 and 2.1.2
- Changes between 2.1.2 and 2.1.3
- notes on incompatibility
- Changes between 2.1.3 and 2.2.0
- Changes between 2.2.0 and 2.2.1
- Changes between 2.2.1 and 2.2.2
- Changes between 2.2.3 and 2.2.4
- Changes between 2.2.4 and 2.3.0
- Changes between 2.3.0 and 2.3.1
- Changes between 2.3.1 and 2.3.2
- Changes between 2.3.2 and 2.3.3
- Changes between 2.3.4 and 2.3.5
- Changes between 2.3.5 and 2.4
- 2.5.0
- 2.5.1
- 2.5.2
- 2.6.0
- 2.6.2
- 2.7.0 (compared to 2.6.4)
- 2.7.1 (compared to 2.7.0)
- 2.7.2 (compared to 2.7.1)
- 2.8.2 (compared to 2.7.2)
- 2.8.3 (compared to 2.8.2)
- 2.8.4 (compared to 2.8.3)
- 2.8.5 (compared to 2.8.4)
- 2.8.6 (compared to 2.8.5)
- 2.8.7 (compared to 2.8.6)
- 2.9.0 (compared to 2.8.7)
- 2.9.1 (compared to 2.9.0)
- 2.9.2 (compared to 2.9.1)
- Participants
- Sprint organisation, schedule
- Money / funding
- Asserting with the assert statement
- Assertions about expected exceptions
- Assertions about expected warnings
- Making use of context-sensitive comparisons
- Defining your own assertion comparison
- Advanced assertion introspection
- Deprecation Roadmap
- Usage
- Rerunning only failures or failures first
- Behavior when no tests failed in the last run
- The new config.cache object
- Inspecting Cache content
- Clearing Cache content
- Default stdout/stderr/stdin capturing behaviour
- Setting capturing methods or disabling capturing
- Using print statements for debugging
- Accessing captured output from a test function
- pytest 3.7.2 (2018-08-16)
- pytest 3.7.1 (2018-08-02)
- pytest 3.7.0 (2018-07-30)
- pytest 3.6.4 (2018-07-28)
- pytest 3.6.3 (2018-07-04)
- pytest 3.6.2 (2018-06-20)
- pytest 3.6.1 (2018-06-05)
- pytest 3.6.0 (2018-05-23)
- pytest 3.5.1 (2018-04-23)
- pytest 3.5.0 (2018-03-21)
- pytest 3.4.2 (2018-03-04)
- pytest 3.4.1 (2018-02-20)
- pytest 3.4.0 (2018-01-30)
- pytest 3.3.2 (2017-12-25)
- pytest 3.3.1 (2017-12-05)
- pytest 3.3.0 (2017-11-23)
- pytest 3.2.5 (2017-11-15)
- pytest 3.2.4 (2017-11-13)
- pytest 3.2.3 (2017-10-03)
- pytest 3.2.2 (2017-09-06)
- pytest 3.2.1 (2017-08-08)
- pytest 3.2.0 (2017-07-30)
- pytest 3.1.3 (2017-07-03)
- pytest 3.1.2 (2017-06-08)
- pytest 3.1.1 (2017-05-30)
- 3.1.0 (2017-05-22)
- 3.0.7 (2017-03-14)
- 3.0.6 (2017-01-22)
- 3.0.5 (2016-12-05)
- 3.0.4 (2016-11-09)
- 3.0.3 (2016-09-28)
- 3.0.2 (2016-09-01)
- 3.0.1 (2016-08-23)
- 3.0.0 (2016-08-18)
- 2.9.2 (2016-05-31)
- 2.9.1 (2016-03-17)
- 2.9.0 (2016-02-29)
- 2.8.7 (2016-01-24)
- 2.8.6 (2016-01-21)
- 2.8.5 (2015-12-11)
- 2.8.4 (2015-12-06)
- 2.8.3 (2015-11-18)
- 2.8.2 (2015-10-07)
- 2.8.1 (2015-09-29)
- 2.8.0 (2015-09-18)
- 2.7.3 (2015-09-15)
- 2.7.2 (2015-06-23)
- 2.7.1 (2015-05-19)
- 2.7.0 (2015-03-26)
- 2.6.4 (2014-10-24)
- 2.6.3 (2014-09-24)
- 2.6.2 (2014-09-05)
- 2.6.1 (2014-08-07)
- 2.6
- 2.5.2 (2014-01-29)
- 2.5.1 (2013-12-17)
- 2.5.0 (2013-12-12)
- 2.4.2 (2013-10-04)
- 2.4.1 (2013-10-02)
- 2.4
- 2.3.5 (2013-04-30)
- 2.3.4 (2012-11-20)
- 2.3.3 (2012-11-06)
- 2.3.2 (2012-10-25)
- 2.3.1 (2012-10-20)
- 2.3.0 (2012-10-19)
- 2.2.4 (2012-05-22)
- 2.2.3 (2012-02-05)
- 2.2.2 (2012-02-05)
- 2.2.1 (2011-12-16)
- 2.2.0 (2011-11-18)
- 2.1.3 (2011-10-18)
- 2.1.2 (2011-09-24)
- 2.1.1
- 2.1.0 (2011-07-09)
- 2.0.3 (2011-05-11)
- 2.0.2 (2011-03-09)
- 2.0.1 (2011-02-07)
- 2.0.0 (2010-11-25)
- 1.3.4 (2010-09-14)
- 1.3.3 (2010-07-30)
- 1.3.2 (2010-07-08)
- 1.3.1 (2010-05-25)
- 1.3.0 (2010-05-05)
- 1.2.0 (2010-01-18)
- 1.1.1 (2009-11-24)
- 1.1.0 (2009-11-05)
- 1.0.3
- 1.0.2 (2009-08-27)
- 1.0.1 (2009-08-19)
- 1.0.0 (2009-08-04)
- 1.0.0b9 (2009-07-31)
- 1.0.0b8 (2009-07-22)
- 1.0.0b7
- 1.0.0b3 (2009-06-19)
- 1.0.0b1
- 0.9.2
- 0.9.1
- Feature requests and feedback
- Report bugs
- Fix bugs
- Implement features
- Write documentation
- Submitting Plugins to pytest-dev
- Preparing Pull Requests
- Joining the Development Team
- Command line options and configuration file settings
- Initialization: determining rootdir and inifile
- How to change command line options defaults
- Builtin configuration file options
- Code Style
- Branches
- Issues
- Release Procedure
- The ‘doctest_namespace’ fixture
- Output format
- Marking test functions and selecting them for a run
- Selecting tests based on their node ID
- Using -k expr to select tests based on their name
- Registering markers
- Marking whole classes or modules
- Marking individual tests when using parametrize
- Custom marker and command line option to control test runs
- Passing a callable to custom markers
- Reading markers which were set from multiple places
- marking platform specific tests with pytest
- Automatically adding markers based on test names
- A basic example for specifying tests in Yaml files
- Generating parameters combinations, depending on command line
- Different options for test IDs
- A quick port of “testscenarios”
- Deferring the setup of parametrized resources
- Apply indirect on particular arguments
- Parametrizing test methods through per-class configuration
- Indirect parametrization with multiple fixtures
- Indirect parametrization of optional implementations/imports
- Set marks or test ID for individual parametrized test
- Ignore paths during test collection
- Deselect tests during test collection
- Keeping duplicate paths specified from command line
- Changing directory recursion
- Changing naming conventions
- Interpreting cmdline arguments as Python packages
- Finding out what is collected
- Customizing test collection
- Pass different values to a test function, depending on command line options
- Dynamically adding command line options
- Control skipping of tests according to command line option
- Writing well integrated assertion helpers
- Detect if running from within a pytest run
- Adding info to test report header
- profiling test duration
- incremental testing - test steps
- Package/Directory-level fixtures (setups)
- post-process test reports / failures
- Making test result information available in fixtures
- PYTEST_CURRENT_TEST environment variable
- Freezing pytest
- Running an existing test suite with pytest
- On naming, nosetests, licensing and magic
- pytest fixtures, parametrized tests
- pytest interaction with other packages
- Fixtures as Function arguments
- Fixtures: a prime example of dependency injection
- conftest.py: sharing fixture functions
- Sharing test data
- Scope: sharing a fixture instance across tests in a class, module or session
- Higher-scoped fixtures are instantiated first
- Fixture finalization / executing teardown code
- Fixtures can introspect the requesting test context
- Factories as fixtures
- Parametrizing fixtures
- Using marks with parametrized fixtures
- Modularity: using fixtures from a fixture function
- Automatic grouping of tests by fixture instances
- Using fixtures from classes, modules or projects
- Autouse fixtures (xUnit setup on steroids)
- Overriding fixtures on various levels
- Shortcomings of the previous pytest_funcarg__ mechanism
- Direct scoping of fixture/funcarg factories
- Direct parametrization of funcarg resource factories
- No pytest_funcarg__ prefix when using @fixture decorator
- solving per-session setup / autouse fixtures
- funcargs/fixture discovery now happens at collection time
- Conclusion and compatibility notes
- Install pytest
- Create your first test
- Run multiple tests
- Assert that a certain exception is raised
- Group multiple tests in a class
- Request a unique temporary directory for functional tests
- Continue reading
- Install package with pip
- Conventions for Python test discovery
- Choosing a test layout / import rules
- tox
- Integrating with setuptools / python setup.py test / pytest-runner
- cache plugin integrated into the core
- funcargs and pytest_funcarg__
- @pytest.yield_fixture decorator
- [pytest] header in setup.cfg
- Applying marks to @pytest.mark.parametrize parameters
- @pytest.mark.parametrize argument names as a tuple
- setup: is now an “autouse fixture”
- Conditions as strings instead of booleans
- pytest.set_trace()
- Features
- Documentation
- Bugs/Requests
- Changelog
- License
- caplog fixture
- Live Logs
- Release notes
- Incompatible changes in pytest 3.4
- Raising errors on unknown marks: –strict
- Marker revamp and iteration
- Simple example: monkeypatching functions
- example: preventing “requests” from remote operations
- API Reference
- Usage
- Supported nose Idioms
- Unsupported idioms / known issues
- @pytest.mark.parametrize: parametrizing test functions
- Basic pytest_generate_tests example
- More examples
- Requiring/Loading plugins in a test module or conftest file
- Finding out which plugins are active
- Deactivating / unregistering a plugin by name
- Some organisations using pytest
- Problem
- Issues
- Proposed solution
- Alternative approach
- Test modules / conftest.py files inside packages
- Standalone test modules / conftest.py files
- Invoking pytest versus python -m pytest
- Functions
- Marks
- Fixtures
- Hooks
- Objects
- Special Variables
- Environment Variables
- Configuration Options
- Skipping test functions
- XFail: mark test functions as expected to fail
- Skip/xfail with parametrize
- Books
- Talks and blog postings
- The ‘tmpdir’ fixture
- The ‘tmpdir_factory’ fixture
- The default base temporary directory
- Benefits out of the box
- pytest features in unittest.TestCase subclasses
- Mixing pytest fixtures into unittest.TestCase subclasses using marks
- Using autouse fixtures and accessing other fixtures
- Calling pytest through python -m pytest
- Possible exit codes
- Getting help on version, option names, environment variables
- Stopping after the first (or N) failures
- Specifying tests / selecting tests
- Modifying Python traceback printing
- Dropping to PDB (Python Debugger) on failures
- Dropping to PDB (Python Debugger) at the start of a test
- Setting breakpoints
- Using the builtin breakpoint function
- Profiling test execution duration
- Creating JUnitXML format files
- Creating resultlog format files
- Sending test report to online pastebin service
- Disabling plugins
- Calling pytest from Python code
- @pytest.mark.filterwarnings
- Disabling warning capture
- Asserting warnings with the warns function
- Recording warnings
- Ensuring a function triggers a deprecation warning
- Plugin discovery order at tool startup
- conftest.py: local per-directory plugins
- Writing your own plugin
- Making your plugin installable by others
- Assertion Rewriting
- Requiring/Loading plugins in a test module or conftest file
- Accessing another plugin by name
- Testing plugins
- hook function validation and execution
- firstresult: stop at first non-None result
- hookwrapper: executing around other hooks
- Hook function ordering / call example
- Declaring new hooks
- Optionally using hooks from 3rd party plugins
- Module level setup/teardown
- Class level setup/teardown
- Method and function level setup/teardown
Function
Exception
Option
Class
Attribute
Method
Guide(129)
Section(358)
Function(74)
Exception(5)
Option(29)
Class(244)
Attribute(67)
Method(147)