BDD library for the pytest runner
=================================

.. image:: https://img.shields.io/pypi/v/pytest-bdd.svg
   :target: https://pypi.python.org/pypi/pytest-bdd
.. image:: https://codecov.io/gh/pytest-dev/pytest-bdd/branch/master/graph/badge.svg
   :target: https://codecov.io/gh/pytest-dev/pytest-bdd
.. image:: https://github.com/pytest-dev/pytest-bdd/actions/workflows/main.yml/badge.svg
   :target: https://github.com/pytest-dev/pytest-bdd/actions/workflows/main.yml
.. image:: https://readthedocs.org/projects/pytest-bdd/badge/?version=stable
   :target: https://readthedocs.org/projects/pytest-bdd/
   :alt: Documentation Status

pytest-bdd implements a subset of the Gherkin language to enable automating project
requirements testing and to facilitate behavioral driven development.

Unlike many other BDD tools, it does not require a separate runner and benefits from
the power and flexibility of pytest. It enables unifying unit and functional
tests, reduces the burden of continuous integration server configuration and allows the reuse of
test setups.

Pytest fixtures written for unit tests can be reused for setup and actions
mentioned in feature steps with dependency injection. This allows a true BDD
just-enough specification of the requirements without maintaining any context object
containing the side effects of Gherkin imperative declarations.

.. _behave: https://pypi.python.org/pypi/behave
.. _pytest-splinter: https://github.com/pytest-dev/pytest-splinter

Install pytest-bdd
------------------

::

    pip install pytest-bdd

Example
-------

An example test for a blog hosting software could look like this.
Note that pytest-splinter_ is used to get the browser fixture.

.. code-block:: gherkin

    # content of publish_article.feature

    Feature: Blog
        A site where you can publish your articles.

        Scenario: Publishing the article
            Given I'm an author user
            And I have an article

            When I go to the article page
            And I press the publish button

            Then I should not see the error message
            And the article should be published  # Note: will query the database

Note that only one feature is allowed per feature file.

.. code-block:: python

    # content of test_publish_article.py

    from pytest_bdd import scenario, given, when, then

    @scenario('publish_article.feature', 'Publishing the article')
    def test_publish():
        pass

    @given("I'm an author user")
    def author_user(auth, author):
        auth['user'] = author.user

    @given("I have an article", target_fixture="article")
    def article(author):
        return create_test_article(author=author)

    @when("I go to the article page")
    def go_to_article(article, browser):
        browser.visit(urljoin(browser.url, '/manage/articles/{0}/'.format(article.id)))

    @when("I press the publish button")
    def publish_article(browser):
        browser.find_by_css('button[name=publish]').first.click()

    @then("I should not see the error message")
    def no_error_message(browser):
        with pytest.raises(ElementDoesNotExist):
            browser.find_by_css('.message.error').first

    @then("the article should be published")
    def article_is_published(article):
        article.refresh()  # Refresh the object in the SQLAlchemy session
        assert article.is_published

Scenario decorator
------------------

Functions decorated with the `scenario` decorator behave like a normal test function,
and they will be executed after all scenario steps.

.. code-block:: python

    from pytest_bdd import scenario, given, when, then

    @scenario('publish_article.feature', 'Publishing the article')
    def test_publish(browser):
        assert article.title in browser.html

.. NOTE:: It is however encouraged to try as much as possible to have your logic only inside the Given, When, Then steps.

Step aliases
------------

Sometimes, one has to declare the same fixtures or steps with
different names for better readability. In order to use the same step
function with multiple step names simply decorate it multiple times:

.. code-block:: python

    @given("I have an article")
    @given("there's an article")
    def article(author, target_fixture="article"):
        return create_test_article(author=author)

Note that the given step aliases are independent and will be executed
when mentioned.

For example if you associate your resource to some owner or not. Admin
user can’t be an author of the article, but articles should have a
default author.

.. code-block:: gherkin

    Feature: Resource owner
        Scenario: I'm the author
            Given I'm an author
            And I have an article

        Scenario: I'm the admin
            Given I'm the admin
            And there's an article

Step arguments
--------------

Often it's possible to reuse steps giving them a parameter(s).
This allows to have single implementation and multiple use, so less code.
Also opens the possibility to use same step twice in single scenario and with different arguments!
And even more, there are several types of step parameter parsers at your disposal
(idea taken from behave_ implementation):

.. _pypi_parse: http://pypi.python.org/pypi/parse
.. _pypi_parse_type: http://pypi.python.org/pypi/parse_type

**string** (the default)
    This is the default and can be considered as a `null` or `exact` parser. It parses no parameters
    and matches the step name by equality of strings.
**parse** (based on: pypi_parse_)
    Provides a simple parser that replaces regular expressions for
    step parameters with a readable syntax like ``{param:Type}``.
    The syntax is inspired by the Python builtin ``string.format()``
    function.
    Step parameters must use the named fields syntax of pypi_parse_
    in step definitions. The named fields are extracted,
    optionally type converted and then used as step function arguments.
    Supports type conversions by using type converters passed via `extra_types`
**cfparse** (extends: pypi_parse_, based on: pypi_parse_type_)
    Provides an extended parser with "Cardinality Field" (CF) support.
    Automatically creates missing type converters for related cardinality
    as long as a type converter for cardinality=1 is provided.
    Supports parse expressions like:
    * ``{values:Type+}`` (cardinality=1..N, many)
    * ``{values:Type*}`` (cardinality=0..N, many0)
    * ``{value:Type?}``  (cardinality=0..1, optional)
    Supports type conversions (as above).
**re**
    This uses full regular expressions to parse the clause text. You will
    need to use named groups "(?P<name>...)" to define the variables pulled
    from the text and passed to your ``step()`` function.
    Type conversion can only be done via `converters` step decorator argument (see example below).

The default parser is `string`, so just plain one-to-one match to the keyword definition.
Parsers except `string`, as well as their optional arguments are specified like:

for `cfparse` parser

.. code-block:: python

    from pytest_bdd import parsers

    @given(
        parsers.cfparse("there are {start:Number} cucumbers", extra_types={"Number": int}),
        target_fixture="cucumbers",
    )
    def given_cucumbers(start):
        return {"start": start, "eat": 0}

for `re` parser

.. code-block:: python

    from pytest_bdd import parsers

    @given(
        parsers.re(r"there are (?P<start>\d+) cucumbers"),
        converters={"start": int},
        target_fixture="cucumbers",
    )
    def given_cucumbers(start):
        return {"start": start, "eat": 0}

Example:

.. code-block:: gherkin

    Feature: Step arguments
        Scenario: Arguments for given, when, then
            Given there are 5 cucumbers

            When I eat 3 cucumbers
            And I eat 2 cucumbers

            Then I should have 0 cucumbers

The code will look like:

.. code-block:: python

    from pytest_bdd import scenarios, given, when, then, parsers

    scenarios("arguments.feature")

    @given(parsers.parse("there are {start:d} cucumbers"), target_fixture="cucumbers")
    def given_cucumbers(start):
        return {"start": start, "eat": 0}

    @when(parsers.parse("I eat {eat:d} cucumbers"))
    def eat_cucumbers(cucumbers, eat):
        cucumbers["eat"] += eat

    @then(parsers.parse("I should have {left:d} cucumbers"))
    def should_have_left_cucumbers(cucumbers, left):
        assert cucumbers["start"] - cucumbers["eat"] == left

Example code also shows possibility to pass argument converters which may be useful if you need to postprocess step
arguments after the parser.

You can implement your own step parser. It's interface is quite simple. The code can look like:

.. code-block:: python

    import re
    from pytest_bdd import given, parsers

    class MyParser(parsers.StepParser):
        """Custom parser."""

        def __init__(self, name, **kwargs):
            """Compile regex."""
            super().__init__(name)
            self.regex = re.compile(re.sub("%(.+)%", "(?P<\1>.+)", self.name), **kwargs)

        def parse_arguments(self, name):
            """Get step arguments.

            :return: `dict` of step arguments
            """
            return self.regex.match(name).groupdict()

        def is_matching(self, name):
            """Match given name with the step name."""
            return bool(self.regex.match(name))

    @given(parsers.parse("there are %start% cucumbers"), target_fixture="cucumbers")
    def given_cucumbers(start):
        return {"start": start, "eat": 0}

Override fixtures via given steps
---------------------------------

Dependency injection is not a panacea if you have complex structure of your test setup data. Sometimes there's a need
such a given step which would imperatively change the fixture only for certain test (scenario), while for other tests
it will stay untouched. To allow this, special parameter `target_fixture` exists in the `given` decorator:

.. code-block:: python

    from pytest_bdd import given

    @pytest.fixture
    def foo():
        return "foo"

    @given("I have injecting given", target_fixture="foo")
    def injecting_given():
        return "injected foo"

    @then('foo should be "injected foo"')
    def foo_is_foo(foo):
        assert foo == 'injected foo'

.. code-block:: gherkin

    Feature: Target fixture
        Scenario: Test given fixture injection
            Given I have injecting given
            Then foo should be "injected foo"

In this example, the existing fixture `foo` will be overridden by given step `I have injecting given` only for the scenario it's
used in.

Sometimes it is also useful to let `when` and `then` steps provide a fixture as well.
A common use case is when we have to assert the outcome of an HTTP request:

.. code-block:: python

    # content of test_blog.py

    from pytest_bdd import scenarios, given, when, then

    from my_app.models import Article

    scenarios("blog.feature")

    @given("there is an article", target_fixture="article")
    def there_is_an_article():
        return Article()

    @when("I request the deletion of the article", target_fixture="request_result")
    def there_should_be_a_new_article(article, http_client):
        return http_client.delete(f"/articles/{article.uid}")

    @then("the request should be successful")
    def article_is_published(request_result):
        assert request_result.status_code == 200

.. code-block:: gherkin

    # content of blog.feature

    Feature: Blog
        Scenario: Deleting the article
            Given there is an article

            When I request the deletion of the article

            Then the request should be successful

Multiline steps
---------------

As Gherkin, pytest-bdd supports multiline steps
(a.k.a. `Doc Strings <https://cucumber.io/docs/gherkin/reference/#doc-strings>`_).
But in much cleaner and powerful way:

.. code-block:: gherkin

    Feature: Multiline steps
        Scenario: Multiline step using sub indentation
            Given I have a step with:
                Some
                Extra
                Lines
            Then the text should be parsed with correct indentation

A step is considered as a multiline one, if the **next** line(s) after it's first line is indented relatively
to the first line. The step name is then simply extended by adding further lines with newlines.
In the example above, the Given step name will be:

.. code-block:: python

    'I have a step with:\nSome\nExtra\nLines'

You can of course register a step using the full name (including the newlines), but it seems more practical to use
step arguments and capture lines after first line (or some subset of them) into the argument:

.. code-block:: python

    from pytest_bdd import given, then, scenario, parsers

    scenarios("multiline.feature")

    @given(parsers.parse("I have a step with:\n{content}"), target_fixture="text")
    def given_text(content):
        return content

    @then("the text should be parsed with correct indentation")
    def text_should_be_correct(text):
        assert text == "Some\nExtra\nLines"

Scenarios shortcut
------------------

If you have a relatively large set of feature files, it's boring to manually bind scenarios to the tests using the scenario decorator. Of course with the manual approach you get all the power to be able to additionally parametrize the test, give the test function a nice name, document it, etc, but in the majority of the cases you don't need that.
Instead, you want to bind all the scenarios found in the ``features`` folder(s) recursively automatically, by using the ``scenarios`` helper.

.. code-block:: python

    from pytest_bdd import scenarios

    # assume 'features' subfolder is in this file's directory
    scenarios('features')

That's all you need to do to bind all scenarios found in the ``features`` folder!
Note that you can pass multiple paths, and those paths can be either feature files or feature folders.

.. code-block:: python

    from pytest_bdd import scenarios

    # pass multiple paths/files
    scenarios('features', 'other_features/some.feature', 'some_other_features')

But what if you need to manually bind a certain scenario, leaving others to be automatically bound?
Just write your scenario in a "normal" way, but ensure you do it **before** the call of ``scenarios`` helper.

.. code-block:: python

    from pytest_bdd import scenario, scenarios

    @scenario('features/some.feature', 'Test something')
    def test_something():
        pass

    # assume 'features' subfolder is in this file's directory
    scenarios('features')

In the example above, the ``test_something`` scenario binding will be kept manual, other scenarios found in the ``features`` folder will be bound automatically.

Scenario outlines
-----------------

Scenarios can be parametrized to cover multiple cases. These are called `Scenario Outlines <https://cucumber.io/docs/gherkin/reference/#scenario-outline>`_ in Gherkin, and the variable templates are written using angular brackets (e.g. ``<var_name>``).

Example:

.. code-block:: gherkin

    # content of scenario_outlines.feature

    Feature: Scenario outlines
        Scenario Outline: Outlined given, when, then
            Given there are <start> cucumbers
            When I eat <eat> cucumbers
            Then I should have <left> cucumbers

            Examples:
            | start | eat | left |
            |  12   |  5  |  7   |

.. code-block:: python

    from pytest_bdd import scenarios, given, when, then, parsers

    scenarios("scenario_outlines.feature")

    @given(parsers.parse("there are {start:d} cucumbers"), target_fixture="cucumbers")
    def given_cucumbers(start):
        return {"start": start, "eat": 0}

    @when(parsers.parse("I eat {eat:d} cucumbers"))
    def eat_cucumbers(cucumbers, eat):
        cucumbers["eat"] += eat

    @then(parsers.parse("I should have {left:d} cucumbers"))
    def should_have_left_cucumbers(cucumbers, left):
        assert cucumbers["start"] - cucumbers["eat"] == left

Organizing your scenarios
-------------------------

The more features and scenarios you have, the more important the question of their organization becomes.
The things you can do (and that is also a recommended way):

* organize your feature files in the folders by semantic groups:

::

    features
    │
    ├──frontend
    │  │
    │  └──auth
    │     │
    │     └──login.feature
    └──backend
       │
       └──auth
          │
          └──login.feature

This looks fine, but how do you run tests only for a certain feature?
As pytest-bdd uses pytest, and bdd scenarios are actually normal tests. But test files
are separate from the feature files, the mapping is up to developers, so the test files structure can look
completely different:

::

    tests
    │
    └──functional
       │
       └──test_auth.py
          │
          └ """Authentication tests."""
            from pytest_bdd import scenario

            @scenario('frontend/auth/login.feature')
            def test_logging_in_frontend():
                pass

            @scenario('backend/auth/login.feature')
            def test_logging_in_backend():
                pass

For picking up tests to run we can use the
`tests selection <https://pytest.org/en/7.1.x/how-to/usage.html#specifying-which-tests-to-run>`_ technique. The problem is that
you have to know how your tests are organized, knowing only the feature files organization is not enough.
Cucumber uses `tags <https://cucumber.io/docs/cucumber/api/#tags>`_ as a way of categorizing your features
and scenarios, which pytest-bdd supports. For example, we could have:

.. code-block:: gherkin

    @login @backend
    Feature: Login

      @successful
      Scenario: Successful login

pytest-bdd uses `pytest markers <http://pytest.org/latest/mark.html>`_ as a `storage` of the tags for the given
scenario test, so we can use standard test selection:

.. code-block:: bash

    pytest -m "backend and login and successful"

The feature and scenario markers are not different from standard pytest markers, and the ``@`` symbol is stripped out automatically to allow test selector expressions. If you want to have bdd-related tags to be distinguishable from the other test markers, use a prefix like ``bdd``.
Note that if you use pytest with the ``--strict`` option, all bdd tags mentioned in the feature files should be also in the ``markers`` setting of the ``pytest.ini`` config. Also for tags please use names which are python-compatible variable names, i.e. start with a non-number, only underscores or alphanumeric characters, etc. That way you can safely use tags for tests filtering.

You can customize how tags are converted to pytest marks by implementing the
``pytest_bdd_apply_tag`` hook and returning ``True`` from it:

.. code-block:: python

   def pytest_bdd_apply_tag(tag, function):
       if tag == 'todo':
           marker = pytest.mark.skip(reason="Not implemented yet")
           marker(function)
           return True
       else:
           # Fall back to the default behavior of pytest-bdd
           return None

Test setup
----------

Test setup is implemented within the Given section. Even though these steps
are executed imperatively to apply possible side-effects, pytest-bdd is trying
to benefit of the PyTest fixtures which is based on the dependency injection
and makes the setup more declarative style.

.. code-block:: python

    @given("I have a beautiful article", target_fixture="article")
    def article():
        return Article(is_beautiful=True)

The target PyTest fixture "article" gets the return value and any other step can depend on it.

.. code-block:: gherkin

    Feature: The power of PyTest
        Scenario: Symbolic name across steps
            Given I have a beautiful article
            When I publish this article

The When step is referencing the ``article`` to publish it.

.. code-block:: python

    @when("I publish this article")
    def publish_article(article):
        article.publish()

Many other BDD toolkits operate on a global context and put the side effects there.
This makes it very difficult to implement the steps, because the dependencies
appear only as the side-effects during run-time and not declared in the code.
The "publish article" step has to trust that the article is already in the context,
has to know the name of the attribute it is stored there, the type etc.

In pytest-bdd you just declare an argument of the step function that it depends on
and the PyTest will make sure to provide it.

Still side effects can be applied in the imperative style by design of the BDD.

.. code-block:: gherkin

    Feature: News website
        Scenario: Publishing an article
            Given I have a beautiful article
            And my article is published

Functional tests can reuse your fixture libraries created for the unit-tests and upgrade
them by applying the side effects.

.. code-block:: python

    @pytest.fixture
    def article():
        return Article(is_beautiful=True)

    @given("I have a beautiful article")
    def i_have_a_beautiful_article(article):
        pass

    @given("my article is published")
    def published_article(article):
        article.publish()
        return article

This way side-effects were applied to our article and PyTest makes sure that all
steps that require the "article" fixture will receive the same object. The value
of the "published_article" and the "article" fixtures is the same object.

Fixtures are evaluated **only once** within the PyTest scope and their values are cached.

Backgrounds
-----------

It's often the case that to cover certain feature, you'll need multiple scenarios. And it's logical that the
setup for those scenarios will have some common parts (if not equal). For this, there are `backgrounds`.
pytest-bdd implements `Gherkin backgrounds <https://cucumber.io/docs/gherkin/reference/#background>`_ for
features.

.. code-block:: gherkin

    Feature: Multiple site support

      Background:
        Given a global administrator named "Greg"
        And a blog named "Greg's anti-tax rants"
        And a customer named "Wilson"
        And a blog named "Expensive Therapy" owned by "Wilson"

      Scenario: Wilson posts to his own blog
        Given I am logged in as Wilson
        When I try to post to "Expensive Therapy"
        Then I should see "Your article was published."

      Scenario: Greg posts to a client's blog
        Given I am logged in as Greg
        When I try to post to "Expensive Therapy"
        Then I should see "Your article was published."

In this example, all steps from the background will be executed before all the scenario's own given
steps, adding a possibility to prepare some common setup for multiple scenarios in a single feature.
About best practices for Background, please read Gherkin's
`Tips for using Background <https://cucumber.io/docs/gherkin/reference/#tips-for-using-background>`_.

.. NOTE:: Only "Given" steps should be used in "Background" section.
          Steps "When" and "Then" are prohibited, because their purposes are
          related to actions and consuming outcomes; that is in conflict with the
          aim of "Background" - to prepare the system for tests or "put the system
          in a known state" as "Given" does it.
          The statement above applies to strict Gherkin mode, which is
          enabled by default.

Reusing fixtures
----------------

Sometimes scenarios define new names for an existing fixture that can be
inherited (reused). For example, if we have the pytest fixture:

.. code-block:: python

    @pytest.fixture
    def article():
       """Test article."""
       return Article()

Then this fixture can be reused with other names using given():

.. code-block:: python

    @given('I have a beautiful article')
    def i_have_an_article(article):
       """I have an article."""

Reusing steps
-------------

It is possible to define some common steps in the parent ``conftest.py`` and
simply expect them in the child test file.

.. code-block:: gherkin

    # content of common_steps.feature

    Scenario: All steps are declared in the conftest
        Given I have a bar
        Then bar should have value "bar"

.. code-block:: python

    # content of conftest.py

    from pytest_bdd import given, then

    @given("I have a bar", target_fixture="bar")
    def bar():
        return "bar"

    @then('bar should have value "bar"')
    def bar_is_bar(bar):
        assert bar == "bar"

.. code-block:: python

    # content of test_common.py

    @scenario("common_steps.feature", "All steps are declared in the conftest")
    def test_conftest():
        pass

There are no definitions of steps in the test file. They were
collected from the parent conftest.py.

Default steps
-------------

Here is the list of steps that are implemented inside pytest-bdd:

given
    * trace - enters the `pdb` debugger via `pytest.set_trace()`
when
    * trace - enters the `pdb` debugger via `pytest.set_trace()`
then
    * trace - enters the `pdb` debugger via `pytest.set_trace()`

Feature file paths
------------------

By default, pytest-bdd will use the current module's path as the base path for finding feature files, but this behaviour can be changed in the pytest configuration file (i.e. `pytest.ini`, `tox.ini` or `setup.cfg`) by declaring the new base path in the `bdd_features_base_dir` key. The path is interpreted as relative to the `pytest root directory <https://docs.pytest.org/en/latest/reference/customize.html#rootdir>`__.
You can also override the features base path on a per-scenario basis, in order to override the path for specific tests.

pytest.ini:

.. code-block:: ini

    [pytest]
    bdd_features_base_dir = features/

tests/test_publish_article.py:

.. code-block:: python

    from pytest_bdd import scenario

    @scenario("foo.feature", "Foo feature in features/foo.feature")
    def test_foo():
        pass

    @scenario(
        "foo.feature",
        "Foo feature in tests/local-features/foo.feature",
        features_base_dir="./local-features/",
    )
    def test_foo_local():
        pass

The `features_base_dir` parameter can also be passed to the `@scenario` decorator.

Avoid retyping the feature file name
------------------------------------

If you want to avoid retyping the feature file name when defining your scenarios in a test file, use ``functools.partial``.
This will make your life much easier when defining multiple scenarios in a test file. For example:

.. code-block:: python

    # content of test_publish_article.py

    from functools import partial

    import pytest_bdd

    scenario = partial(pytest_bdd.scenario, "/path/to/publish_article.feature")

    @scenario("Publishing the article")
    def test_publish():
        pass

    @scenario("Publishing the article as unprivileged user")
    def test_publish_unprivileged():
        pass

You can learn more about `functools.partial <https://docs.python.org/3/library/functools.html#functools.partial>`_
in the Python docs.

Programmatic step generation
----------------------------
Sometimes you have step definitions that would be much easier to automate rather than writing them manually over and over again.
This is common, for example, when using libraries like `pytest-factoryboy <https://pytest-factoryboy.readthedocs.io/>`_ that automatically creates fixtures.
Writing step definitions for every model can become a tedious task.

For this reason, pytest-bdd provides a way to generate step definitions automatically.

The trick is to pass the ``stacklevel`` parameter to the ``given``, ``when``, ``then``, ``step`` decorators. This will instruct them to inject the step fixtures in the appropriate module, rather than just injecting them in the caller frame.

Let's look at a concrete example; let's say you have a class ``Wallet`` that has some amount of each currency:

.. code-block:: python

    # contents of wallet.py

    import dataclass

    @dataclass
    class Wallet:
        verified: bool

        amount_eur: int
        amount_usd: int
        amount_gbp: int
        amount_jpy: int

You can use pytest-factoryboy to automatically create model fixtures for this class:

.. code-block:: python

    # contents of wallet_factory.py

    from wallet import Wallet

    import factory
    from pytest_factoryboy import register

    class WalletFactory(factory.Factory):
        class Meta:
            model = Wallet

        amount_eur = 0
        amount_usd = 0
        amount_gbp = 0
        amount_jpy = 0

    register(Wallet)  # creates the "wallet" fixture
    register(Wallet, "second_wallet")  # creates the "second_wallet" fixture

Now we can define a function ``generate_wallet_steps(...)`` that creates the steps for any wallet fixture (in our case, it will be ``wallet`` and ``second_wallet``):

.. code-block:: python

    # contents of wallet_steps.py

    import re
    from dataclasses import fields

    import factory
    import pytest
    from pytest_bdd import given, when, then, scenarios, parsers

    def generate_wallet_steps(model_name="wallet", stacklevel=1):
        stacklevel += 1

        human_name = model_name.replace("_", " ")  # "second_wallet" -> "second wallet"

        @given(f"I have a {human_name}", target_fixture=model_name, stacklevel=stacklevel)
        def _(request):
            return request.getfixturevalue(model_name)

        # Generate steps for currency fields:
        for field in fields(Wallet):
            match = re.fullmatch(r"amount_(?P<currency>[a-z]{3})", field.name)
            if not match:
                continue
            currency = match["currency"]

            @given(
                parsers.parse(f"I have {{value:d}} {currency.upper()} in my {human_name}"),
                target_fixture=f"{model_name}__amount_{currency}",
                stacklevel=stacklevel,
            )
            def _(value: int) -> int:
                return value

            @then(
                parsers.parse(f"I should have {{value:d}} {currency.upper()} in my {human_name}"),
                stacklevel=stacklevel,
            )
            def _(value: int, _currency=currency, _model_name=model_name) -> None:
                wallet = request.getfixturevalue(_model_name)
                assert getattr(wallet, f"amount_{_currency}") == value

    # Inject the steps into the current module
    generate_wallet_steps("wallet")
    generate_wallet_steps("second_wallet")

This last file, ``wallet_steps.py``, now contains all the step definitions for our "wallet" and "second_wallet" fixtures.

We can now define a scenario like this:

.. code-block:: gherkin

    # contents of wallet.feature
    Feature: A feature

        Scenario: Wallet EUR amount stays constant
            Given I have 10 EUR in my wallet
            And I have a wallet
            Then I should have 10 EUR in my wallet

        Scenario: Second wallet JPY amount stays constant
            Given I have 100 JPY in my second wallet
            And I have a second wallet
            Then I should have 100 JPY in my second wallet

and finally a test file that puts it all together and run the scenarios:

.. code-block:: python

    # contents of test_wallet.py

    from pytest_factoryboy import scenarios

    from wallet_factory import *  # import the registered fixtures "wallet" and "second_wallet"
    from wallet_steps import *  # import all the step definitions into this test file

    scenarios("wallet.feature")

Hooks
-----

pytest-bdd exposes several `pytest hooks <https://docs.pytest.org/en/7.1.x/reference/reference.html#hooks>`_
which might be helpful building useful reporting, visualization, etc. on top of it:

* `pytest_bdd_before_scenario(request, feature, scenario)` - Called before scenario is executed

* `pytest_bdd_after_scenario(request, feature, scenario)` - Called after scenario is executed
  (even if one of steps has failed)

* `pytest_bdd_before_step(request, feature, scenario, step, step_func)` - Called before step function
  is executed and it's arguments evaluated

* `pytest_bdd_before_step_call(request, feature, scenario, step, step_func, step_func_args)` - Called before step
  function is executed with evaluated arguments

* `pytest_bdd_after_step(request, feature, scenario, step, step_func, step_func_args)` - Called after step function
  is successfully executed

* `pytest_bdd_step_error(request, feature, scenario, step, step_func, step_func_args, exception)` - Called when step
  function failed to execute

* `pytest_bdd_step_func_lookup_error(request, feature, scenario, step, exception)` - Called when step lookup failed

Browser testing
---------------

Tools recommended to use for browser testing:

* pytest-splinter_ - pytest `splinter <https://splinter.readthedocs.io/>`_ integration for the real browser testing

Reporting
---------

It's important to have nice reporting out of your bdd tests. Cucumber introduced some kind of standard for
`json format <https://www.relishapp.com/cucumber/cucumber/docs/json-output-formatter>`_
which can be used for, for example, by `this <https://plugins.jenkins.io/cucumber-testresult-plugin/>`_ Jenkins
plugin.

To have an output in json format:

::

    pytest --cucumberjson=<path to json report>

This will output an expanded (meaning scenario outlines will be expanded to several scenarios) Cucumber format.

To enable gherkin-formatted output on terminal, use

::

    pytest --gherkin-terminal-reporter

Test code generation helpers
----------------------------

For newcomers it's sometimes hard to write all needed test code without being frustrated.
To simplify their life, a simple code generator was implemented. It allows to create fully functional
(but of course empty) tests and step definitions for a given feature file.
It's done as a separate console script provided by pytest-bdd package:

::

    pytest-bdd generate <feature file name> .. <feature file nameN>

It will print the generated code to the standard output so you can easily redirect it to the file:

::

    pytest-bdd generate features/some.feature > tests/functional/test_some.py

Advanced code generation
------------------------

For more experienced users, there's a smart code generation/suggestion feature. It will only generate the
test code which is not yet there, checking existing tests and step definitions the same way it's done during the
test execution. The code suggestion tool is called via passing additional pytest arguments:

::

    pytest --generate-missing --feature features tests/functional

The output will be like:

::

    ============================= test session starts ==============================
    platform linux2 -- Python 2.7.6 -- py-1.4.24 -- pytest-2.6.2
    plugins: xdist, pep8, cov, cache, bdd, bdd, bdd
    collected 2 items

    Scenario is not bound to any test: "Code is generated for scenarios which are not bound to any tests" in feature "Missing code generation" in /tmp/pytest-552/testdir/test_generate_missing0/tests/generation.feature
    --------------------------------------------------------------------------------

    Step is not defined: "I have a custom bar" in scenario: "Code is generated for scenario steps which are not yet defined(implemented)" in feature "Missing code generation" in /tmp/pytest-552/testdir/test_generate_missing0/tests/generation.feature
    --------------------------------------------------------------------------------
    Please place the code above to the test file(s):

    @scenario('tests/generation.feature', 'Code is generated for scenarios which are not bound to any tests')
    def test_Code_is_generated_for_scenarios_which_are_not_bound_to_any_tests():
        """Code is generated for scenarios which are not bound to any tests."""

    @given("I have a custom bar")
    def I_have_a_custom_bar():
        """I have a custom bar."""

As as side effect, the tool will validate the files for format errors, also some of the logic bugs, for example the
ordering of the types of the steps.

.. _Migration from 5.x.x:

Migration of your tests from versions 5.x.x
-------------------------------------------

The primary focus of the pytest-bdd is the compatibility with the latest gherkin developments
e.g. multiple scenario outline example tables with tags support etc.

In order to provide the best compatibility, it is best to support the features described in the official
gherkin reference. This means deprecation of some non-standard features that were implemented in pytest-bdd.

Removal of the feature examples
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The example tables on the feature level are no longer supported. If you had examples on the feature level, you should copy them to each individual scenario.

Removal of the vertical examples
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Vertical example tables are no longer supported since the official gherkin doesn't support them.
The example tables should have horizontal orientation.

Step arguments are no longer fixtures
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Step parsed arguments conflicted with the fixtures. Now they no longer define fixture.
If the fixture has to be defined by the step, the target_fixture param should be used.

Variable templates in steps are only parsed for Scenario Outlines
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In previous versions of pytest, steps containing ``<variable>`` would be parsed both by ``Scenario`` and ``Scenario Outline``.
Now they are only parsed within a ``Scenario Outline``.

.. _Migration from 4.x.x:

Migration of your tests from versions 4.x.x
-------------------------------------------

Replace usage of <parameter> inside step definitions with parsed {parameter}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Templated steps (e.g. ``@given("there are <start> cucumbers")``) should now the use step argument parsers in order to match the scenario outlines and get the values from the example tables. The values from the example tables are no longer passed as fixtures, although if you define your step to use a parser, the parameters will be still provided as fixtures.

.. code-block:: python

    # Old step definition:
    @given("there are <start> cucumbers")
    def given_cucumbers(start):
        pass

    # New step definition:
    @given(parsers.parse("there are {start} cucumbers"))
    def given_cucumbers(start):
        pass

Scenario `example_converters` are removed in favor of the converters provided on the step level:

.. code-block:: python

    # Old code:
    @given("there are <start> cucumbers")
    def given_cucumbers(start):
        return {"start": start}

    @scenario("outline.feature", "Outlined", example_converters={"start": float})
    def test_outline():
        pass

    # New code:
    @given(parsers.parse("there are {start} cucumbers"), converters={"start": float})
    def given_cucumbers(start):
        return {"start": start}

    @scenario("outline.feature", "Outlined")
    def test_outline():
        pass

Refuse combining scenario outline and pytest parametrization
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The significant downside of combining scenario outline and pytest parametrization approach was an inability to see the
test table from the feature file.

.. _Migration from 3.x.x:

Migration of your tests from versions 3.x.x
-------------------------------------------

Given steps are no longer fixtures. In case it is needed to make given step setup a fixture,
the target_fixture parameter should be used.

.. code-block:: python

    @given("there's an article", target_fixture="article")
    def there_is_an_article():
        return Article()

Given steps no longer have the `fixture` parameter. In fact the step may depend on multiple fixtures.
Just normal step declaration with the dependency injection should be used.

.. code-block:: python

    @given("there's an article")
    def there_is_an_article(article):
        pass

Strict gherkin option is removed, so the ``strict_gherkin`` parameter can be removed from the scenario decorators
as well as ``bdd_strict_gherkin`` from the ini files.

Step validation handlers for the hook ``pytest_bdd_step_validation_error`` should be removed.

License
-------

This software is licensed under the `MIT License <https://opensource.org/licenses/MIT>`_.

© 2013 Oleg Pidsadnyi, Anatoly Bubenkov and others


# libtmux

`libtmux` is a [typed](https://docs.python.org/3/library/typing.html) Python library that provides a wrapper for interacting programmatically with tmux, a terminal multiplexer. You can use it to manage tmux servers,
sessions, windows, and panes. Additionally, `libtmux` powers [tmuxp], a tmux workspace manager.

[![Python Package](https://img.shields.io/pypi/v/libtmux.svg)](https://pypi.org/project/libtmux/)
[![Docs](https://github.com/tmux-python/libtmux/workflows/docs/badge.svg)](https://libtmux.git-pull.com/)
[![Build Status](https://github.com/tmux-python/libtmux/workflows/tests/badge.svg)](https://github.com/tmux-python/tmux-python/actions?query=workflow%3A%22tests%22)
[![Code Coverage](https://codecov.io/gh/tmux-python/libtmux/branch/master/graph/badge.svg)](https://codecov.io/gh/tmux-python/libtmux)
[![License](https://img.shields.io/github/license/tmux-python/libtmux.svg)](https://github.com/tmux-python/libtmux/blob/master/LICENSE)

libtmux builds upon tmux's
[target](http://man.openbsd.org/OpenBSD-5.9/man1/tmux.1#COMMANDS) and
[formats](http://man.openbsd.org/OpenBSD-5.9/man1/tmux.1#FORMATS) to
create an object mapping to traverse, inspect and interact with live
tmux sessions.

View the [documentation](https://libtmux.git-pull.com/),
[API](https://libtmux.git-pull.com/api.html) information and
[architectural details](https://libtmux.git-pull.com/about.html).

# Install

```console
$ pip install --user libtmux
```

# Open a tmux session

Session name `foo`, window name `bar`

```console
$ tmux new-session -s foo -n bar
```

# Pilot your tmux session via python

```console
$ python
```

Use [ptpython], [ipython], etc. for a nice shell with autocompletions:

```console
$ pip install --user ptpython
```

```console
$ ptpython
```

Connect to a live tmux session:

```python
>>> import libtmux
>>> svr = libtmux.Server()
>>> svr
Server(socket_path=/tmp/tmux-.../default)
```

Tip: You can also use [tmuxp]'s [`tmuxp shell`] to drop straight into your
current tmux server / session / window pane.

[tmuxp]: https://tmuxp.git-pull.com/
[`tmuxp shell`]: https://tmuxp.git-pull.com/cli/shell.html
[ptpython]: https://github.com/prompt-toolkit/ptpython
[ipython]: https://ipython.org/

Run any tmux command, respective of context:

Honors tmux socket name and path:

```python
>>> server = Server(socket_name='libtmux_doctest')
>>> server.cmd('display-message', 'hello world')
<libtmux...>
```

New session:

```python
>>> server.cmd('new-session', '-d', '-P', '-F#{session_id}').stdout[0]
'$2'
```

```python
>>> session.cmd('new-window', '-P').stdout[0]
'libtmux...:2.0'
```

From raw command output, to a rich `Window` object (in practice and as shown
later, you'd use `Session.new_window()`):

```python
>>> Window.from_window_id(window_id=session.cmd('new-window', '-P', '-F#{window_id}').stdout[0], server=session.server)
Window(@2 2:..., Session($1 libtmux_...))
```

Create a pane from a window:

```python
>>> window.cmd('split-window', '-P', '-F#{pane_id}').stdout[0]
'%2'
```

Raw output directly to a `Pane`:

```python
>>> Pane.from_pane_id(pane_id=window.cmd('split-window', '-P', '-F#{pane_id}').stdout[0], server=window.server)
Pane(%... Window(@1 1:..., Session($1 libtmux_...)))
```

List sessions:

```python
>>> server.sessions
[Session($1 ...), Session($0 ...)]
```

Filter sessions by attribute:

```python
>>> server.sessions.filter(history_limit='2000')
[Session($1 ...), Session($0 ...)]
```

Direct lookup:

```python
>>> server.sessions.get(session_id="$1")
Session($1 ...)
```

Filter sessions:

```python
>>> server.sessions[0].rename_session('foo')
Session($1 foo)
>>> server.sessions.filter(session_name="foo")
[Session($1 foo)]
>>> server.sessions.get(session_name="foo")
Session($1 foo)
```

Control your session:

```python
>>> session
Session($1 ...)

>>> session.rename_session('my-session')
Session($1 my-session)
```

Create new window in the background (don't switch to it):

```python
>>> bg_window = session.new_window(attach=False, window_name="ha in the bg")
>>> bg_window
Window(@... 2:ha in the bg, Session($1 ...))

# Session can search the window
>>> session.windows.filter(window_name__startswith="ha")
[Window(@... 2:ha in the bg, Session($1 ...))]

# Directly
>>> session.windows.get(window_name__startswith="ha")
Window(@... 2:ha in the bg, Session($1 ...))

# Clean up
>>> bg_window.kill()
```

Close window:

```python
>>> w = session.active_window
>>> w.kill()
```

Grab remaining tmux window:

```python
>>> window = session.active_window
>>> window.split(attach=False)
Pane(%2 Window(@1 1:... Session($1 ...)))
```

Rename window:

```python
>>> window.rename_window('libtmuxower')
Window(@1 1:libtmuxower, Session($1 ...))
```

Split window (create a new pane):

```python
>>> pane = window.split()
>>> pane = window.split(attach=False)
>>> pane.select()
Pane(%3 Window(@1 1:..., Session($1 ...)))
>>> window = session.new_window(attach=False, window_name="test")
>>> window
Window(@2 2:test, Session($1 ...))
>>> pane = window.split(attach=False)
>>> pane
Pane(%5 Window(@2 2:test, Session($1 ...)))
```

Type inside the pane (send key strokes):

```python
>>> pane.send_keys('echo hey send now')

>>> pane.send_keys('echo hey', enter=False)
>>> pane.enter()
Pane(%1 ...)
```

Grab the output of pane:

```python
>>> pane.clear()  # clear the pane
Pane(%1 ...)
>>> pane.send_keys("cowsay 'hello'", enter=True)
>>> print('\n'.join(pane.cmd('capture-pane', '-p').stdout))  # doctest: +SKIP
$ cowsay 'hello'
 _______
< hello >
 -------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
...
```

Traverse and navigate:

```python
>>> pane.window
Window(@1 1:..., Session($1 ...))
>>> pane.window.session
Session($1 ...)
```

# Python support

Unsupported / no security releases or bug fixes:

- Python 2.x: The backports branch is
  [`v0.8.x`](https://github.com/tmux-python/libtmux/tree/v0.8.x).

# Donations

Your donations fund development of new features, testing and support.
Your money will go directly to maintenance and development of the
project. If you are an individual, feel free to give whatever feels
right for the value you get out of the project.

See donation options at <https://git-pull.com/support.html>.

# Project details

- tmux support: 1.8+
- python support: >= 3.8, pypy, pypy3
- Source: <https://github.com/tmux-python/libtmux>
- Docs: <https://libtmux.git-pull.com>
- API: <https://libtmux.git-pull.com/api.html>
- Changelog: <https://libtmux.git-pull.com/history.html>
- Issues: <https://github.com/tmux-python/libtmux/issues>
- Test Coverage: <https://codecov.io/gh/tmux-python/libtmux>
- pypi: <https://pypi.python.org/pypi/libtmux>
- Open Hub: <https://www.openhub.net/p/libtmux-python>
- Repology: <https://repology.org/project/python:libtmux/versions>
- License: [MIT](http://opensource.org/licenses/MIT).


natsort
=======

.. image:: https://img.shields.io/pypi/v/natsort.svg
    :target: https://pypi.org/project/natsort/

.. image:: https://img.shields.io/pypi/pyversions/natsort.svg
    :target: https://pypi.org/project/natsort/

.. image:: https://img.shields.io/pypi/l/natsort.svg
    :target: https://github.com/SethMMorton/natsort/blob/main/LICENSE

.. image:: https://github.com/SethMMorton/natsort/workflows/Tests/badge.svg
    :target: https://github.com/SethMMorton/natsort/actions

.. image:: https://codecov.io/gh/SethMMorton/natsort/branch/main/graph/badge.svg
    :target: https://codecov.io/gh/SethMMorton/natsort

.. image:: https://img.shields.io/pypi/dw/natsort.svg
    :target: https://pypi.org/project/natsort/

Simple yet flexible natural sorting in Python.

    - Source Code: https://github.com/SethMMorton/natsort
    - Downloads: https://pypi.org/project/natsort/
    - Documentation: https://natsort.readthedocs.io/

      - `Examples and Recipes`_
      - `How Does Natsort Work?`_
      - `API`_

    - `Quick Description`_
    - `Quick Examples`_
    - `FAQ`_
    - `Requirements`_
    - `Optional Dependencies`_
    - `Installation`_
    - `How to Run Tests`_
    - `How to Build Documentation`_
    - `Dropped Deprecated APIs`_
    - `History`_

**NOTE**: Please see the `Dropped Deprecated APIs`_ section for changes.

Quick Description
-----------------

When you try to sort a list of strings that contain numbers, the normal python
sort algorithm sorts lexicographically, so you might not get the results that
you expect:

.. code-block:: pycon

    >>> a = ['2 ft 7 in', '1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '7 ft 6 in']
    >>> sorted(a)
    ['1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '2 ft 7 in', '7 ft 6 in']

Notice that it has the order ('1', '10', '2') - this is because the list is
being sorted in lexicographical order, which sorts numbers like you would
letters (i.e. 'b', 'ba', 'c').

`natsort`_ provides a function `natsorted()`_ that helps sort lists
"naturally" ("naturally" is rather ill-defined, but in general it means
sorting based on meaning and not computer code point).
Using `natsorted()`_ is simple:

.. code-block:: pycon

    >>> from natsort import natsorted
    >>> a = ['2 ft 7 in', '1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '7 ft 6 in']
    >>> natsorted(a)
    ['1 ft 5 in', '2 ft 7 in', '2 ft 11 in', '7 ft 6 in', '10 ft 2 in']

`natsorted()`_ identifies numbers anywhere in a string and sorts them
naturally. Below are some other things you can do with `natsort`_
(also see the `Examples and Recipes`_ for a quick start guide, or the
`API`_ for complete details).

**Note**: `natsorted()`_ is designed to be a drop-in replacement for the
built-in `sorted()`_ function. Like `sorted()`_, `natsorted()`_
`does not sort in-place`. To sort a list and assign the output to the same
variable, you must explicitly assign the output to a variable:

.. code-block:: pycon

    >>> a = ['2 ft 7 in', '1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '7 ft 6 in']
    >>> natsorted(a)
    ['1 ft 5 in', '2 ft 7 in', '2 ft 11 in', '7 ft 6 in', '10 ft 2 in']
    >>> print(a)  # 'a' was not sorted; "natsorted" simply returned a sorted list
    ['2 ft 7 in', '1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '7 ft 6 in']
    >>> a = natsorted(a)  # Now 'a' will be sorted because the sorted list was assigned to 'a'
    >>> print(a)
    ['1 ft 5 in', '2 ft 7 in', '2 ft 11 in', '7 ft 6 in', '10 ft 2 in']

Please see `Generating a Reusable Sorting Key and Sorting In-Place`_ for
an alternate way to sort in-place naturally.

Quick Examples
--------------

- `Sorting Versions`_
- `Sort Paths Like My File Browser (e.g. Windows Explorer on Windows)`_
- `Sorting by Real Numbers (i.e. Signed Floats)`_
- `Locale-Aware Sorting (or "Human Sorting")`_
- `Further Customizing Natsort`_
- `Sorting Mixed Types`_
- `Handling Bytes`_
- `Generating a Reusable Sorting Key and Sorting In-Place`_
- `Other Useful Things`_

Sorting Versions
++++++++++++++++

`natsort`_ does not actually *comprehend* version numbers.
It just so happens that the most common versioning schemes are designed to
work with standard natural sorting techniques; these schemes include
``MAJOR.MINOR``, ``MAJOR.MINOR.PATCH``, ``YEAR.MONTH.DAY``. If your data
conforms to a scheme like this, then it will work out-of-the-box with
`natsorted()`_ (as of `natsort`_ version >= 4.0.0):

.. code-block:: pycon

    >>> a = ['version-1.9', 'version-2.0', 'version-1.11', 'version-1.10']
    >>> natsorted(a)
    ['version-1.9', 'version-1.10', 'version-1.11', 'version-2.0']

If you need to versions that use a more complicated scheme, please see
`these version sorting examples`_.

Sort Paths Like My File Browser (e.g. Windows Explorer on Windows)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Prior to `natsort`_ version 7.1.0, it was a common request to be able to
sort paths like Windows Explorer. As of `natsort`_ 7.1.0, the function
`os_sorted()`_ has been added to provide users the ability to sort
in the order that their file browser might sort (e.g Windows Explorer on
Windows, Finder on MacOS, Dolphin/Nautilus/Thunar/etc. on Linux).

.. code-block:: python

    import os
    from natsort import os_sorted
    print(os_sorted(os.listdir()))
    # The directory sorted like your file browser might show

Output will be different depending on the operating system you are on.

For users **not** on Windows (e.g. MacOS/Linux) it is **strongly** recommended
to also install `PyICU`_, which will help
`natsort`_ give results that match most file browsers. If this is not installed,
it will fall back on Python's built-in `locale`_ module and will give good
results for most input, but will give poor results for special characters.

Sorting by Real Numbers (i.e. Signed Floats)
++++++++++++++++++++++++++++++++++++++++++++

This is useful in scientific data analysis (and was the default behavior
of `natsorted()`_ for `natsort`_ version < 4.0.0). Use the `realsorted()`_
function:

.. code-block:: pycon

    >>> from natsort import realsorted, ns
    >>> # Note that when interpreting as signed floats, the below numbers are
    >>> #            +5.10,                -3.00,            +5.30,              +2.00
    >>> a = ['position5.10.data', 'position-3.data', 'position5.3.data', 'position2.data']
    >>> natsorted(a)
    ['position2.data', 'position5.3.data', 'position5.10.data', 'position-3.data']
    >>> natsorted(a, alg=ns.REAL)
    ['position-3.data', 'position2.data', 'position5.10.data', 'position5.3.data']
    >>> realsorted(a)  # shortcut for natsorted with alg=ns.REAL
    ['position-3.data', 'position2.data', 'position5.10.data', 'position5.3.data']

Locale-Aware Sorting (or "Human Sorting")
+++++++++++++++++++++++++++++++++++++++++

This is where the non-numeric characters are also ordered based on their
meaning, not on their ordinal value, and a locale-dependent thousands
separator and decimal separator is accounted for in the number.
This can be achieved with the `humansorted()`_ function:

.. code-block:: pycon

    >>> a = ['Apple', 'apple15', 'Banana', 'apple14,689', 'banana']
    >>> natsorted(a)
    ['Apple', 'Banana', 'apple14,689', 'apple15', 'banana']
    >>> import locale
    >>> locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')
    'en_US.UTF-8'
    >>> natsorted(a, alg=ns.LOCALE)
    ['apple15', 'apple14,689', 'Apple', 'banana', 'Banana']
    >>> from natsort import humansorted
    >>> humansorted(a)  # shortcut for natsorted with alg=ns.LOCALE
    ['apple15', 'apple14,689', 'Apple', 'banana', 'Banana']

You may find you need to explicitly set the locale to get this to work
(as shown in the example). Please see `locale issues`_ and the
`Optional Dependencies`_ section below before using the `humansorted()`_ function.

Further Customizing Natsort
+++++++++++++++++++++++++++

If you need to combine multiple algorithm modifiers (such as ``ns.REAL``,
``ns.LOCALE``, and ``ns.IGNORECASE``), you can combine the options using the
bitwise OR operator (``|``). For example,

.. code-block:: pycon

    >>> a = ['Apple', 'apple15', 'Banana', 'apple14,689', 'banana']
    >>> natsorted(a, alg=ns.REAL | ns.LOCALE | ns.IGNORECASE)
    ['Apple', 'apple15', 'apple14,689', 'Banana', 'banana']
    >>> # The ns enum provides long and short forms for each option.
    >>> ns.LOCALE == ns.L
    True
    >>> # You can also customize the convenience functions, too.
    >>> natsorted(a, alg=ns.REAL | ns.LOCALE | ns.IGNORECASE) == realsorted(a, alg=ns.L | ns.IC)
    True
    >>> natsorted(a, alg=ns.REAL | ns.LOCALE | ns.IGNORECASE) == humansorted(a, alg=ns.R | ns.IC)
    True

All of the available customizations can be found in the documentation for
`the ns enum`_.

You can also add your own custom transformation functions with the ``key``
argument. These can be used with ``alg`` if you wish.

.. code-block:: pycon

    >>> a = ['apple2.50', '2.3apple']
    >>> natsorted(a, key=lambda x: x.replace('apple', ''), alg=ns.REAL)
    ['2.3apple', 'apple2.50']

Sorting Mixed Types
+++++++++++++++++++

You can mix and match `int`_, `float`_, and `str`_ types when you sort:

.. code-block:: pycon

    >>> a = ['4.5', 6, 2.0, '5', 'a']
    >>> natsorted(a)
    [2.0, '4.5', '5', 6, 'a']
    >>> # sorted(a) would raise an "unorderable types" TypeError

Handling Bytes
++++++++++++++

`natsort`_ does not officially support the `bytes`_ type, but
convenience functions are provided that help you decode to `str`_ first:

.. code-block:: pycon

    >>> from natsort import as_utf8
    >>> a = [b'a', 14.0, 'b']
    >>> # natsorted(a) would raise a TypeError (bytes() < str())
    >>> natsorted(a, key=as_utf8) == [14.0, b'a', 'b']
    True
    >>> a = [b'a56', b'a5', b'a6', b'a40']
    >>> # natsorted(a) would return the same results as sorted(a)
    >>> natsorted(a, key=as_utf8) == [b'a5', b'a6', b'a40', b'a56']
    True

Generating a Reusable Sorting Key and Sorting In-Place
++++++++++++++++++++++++++++++++++++++++++++++++++++++

Under the hood, `natsorted()`_ works by generating a custom sorting
key using `natsort_keygen()`_ and then passes that to the built-in
`sorted()`_. You can use the `natsort_keygen()`_ function yourself to
generate a custom sorting key to sort in-place using the `list.sort()`_
method.

.. code-block:: pycon

    >>> from natsort import natsort_keygen
    >>> natsort_key = natsort_keygen()
    >>> a = ['2 ft 7 in', '1 ft 5 in', '10 ft 2 in', '2 ft 11 in', '7 ft 6 in']
    >>> natsorted(a) == sorted(a, key=natsort_key)
    True
    >>> a.sort(key=natsort_key)
    >>> a
    ['1 ft 5 in', '2 ft 7 in', '2 ft 11 in', '7 ft 6 in', '10 ft 2 in']

All of the algorithm customizations mentioned in the
`Further Customizing Natsort`_ section can also be applied to
`natsort_keygen()`_ through the *alg* keyword option.

Other Useful Things
+++++++++++++++++++

 - recursively descend into lists of lists
 - automatic unicode normalization of input data
 - `controlling the case-sensitivity`_
 - `sorting file paths correctly`_
 - `allow custom sorting keys`_
 - `accounting for units`_

FAQ
---

How do I debug `natsorted()`_?
    The best way to debug `natsorted()`_ is to generate a key using `natsort_keygen()`_
    with the same options being passed to `natsorted()`_. One can take a look at
    exactly what is being done with their input using this key - it is highly
    recommended to `look at this issue describing how to debug`_ for *how* to debug,
    and also to review the `How Does Natsort Work?`_ page for *why* `natsort`_ is
    doing that to your data.

    If you are trying to sort custom classes and running into trouble, please
    take a look at https://github.com/SethMMorton/natsort/issues/60. In short,
    custom classes are not likely to be sorted correctly if one relies
    on the behavior of ``__lt__`` and the other rich comparison operators in
    their custom class - it is better to use a ``key`` function with
    `natsort`_, or use the `natsort`_ key as part of your rich comparison
    operator definition.

`natsort`_ gave me results I didn't expect, and it's a terrible library!
    Did you try to debug using the above advice? If so, and you still cannot figure out
    the error, then please `file an issue`_.

How *does* `natsort`_ work?
    If you don't want to read `How Does Natsort Work?`_,
    here is a quick primer.

    `natsort`_ provides a `key function`_ that can be passed to `list.sort()`_
    or `sorted()`_ in order to modify the default sorting behavior. This key
    is generated on-demand with the key generator `natsort_keygen()`_.
    `natsorted()`_ is essentially a wrapper for the following code:

    .. code-block:: pycon

        >>> from natsort import natsort_keygen
        >>> natsort_key = natsort_keygen()
        >>> sorted(['1', '10', '2'], key=natsort_key)
        ['1', '2', '10']

    Users can further customize `natsort`_ sorting behavior with the ``key``
    and/or ``alg`` options (see details in the `Further Customizing Natsort`_
    section).

    The key generated by `natsort_keygen()`_ *always* returns a `tuple`_. It
    does so in the following way (*some details omitted for clarity*):

      1. Assume the input is a string, and attempt to split it into numbers and
         non-numbers using regular expressions. Numbers are then converted into
         either `int`_ or `float`_.
      2. If the above fails because the input is not a string, assume the input
         is some other sequence (e.g. `list`_ or `tuple`_), and recursively
         apply the key to each element of the sequence.
      3. If the above fails because the input is not iterable, assume the input
         is an `int`_ or `float`_, and just return the input in a `tuple`_.

    Because a `tuple`_ is always returned, a `TypeError`_ should not be common
    unless one tries to do something odd like sort an `int`_ against a `list`_.

Shell script
------------

`natsort`_ comes with a shell script called `natsort`_, or can also be called
from the command line with ``python -m natsort``.  Check out the
`shell script wiki documentation`_ for more details.

Requirements
------------

`natsort`_ requires Python 3.7 or greater.

Optional Dependencies
---------------------

fastnumbers
+++++++++++

The most efficient sorting can occur if you install the
`fastnumbers`_ package
(version >=2.0.0); it helps with the string to number conversions.
`natsort`_ will still run (efficiently) without the package, but if you need
to squeeze out that extra juice it is recommended you include this as a
dependency. `natsort`_ will not require (or check) that
`fastnumbers`_ is installed at installation.

PyICU
+++++

It is recommended that you install `PyICU`_ if you wish to sort in a
locale-dependent manner, see this page on `locale issues`_ for an explanation why.

Installation
------------

Use ``pip``!

.. code-block:: console

    $ pip install natsort

If you want to install the `Optional Dependencies`_, you can use the
`"extras" notation`_ at installation time to install those dependencies as
well - use ``fast`` for `fastnumbers`_ and ``icu`` for `PyICU`_.

.. code-block:: console

    # Install both optional dependencies.
    $ pip install natsort[fast,icu]
    # Install just fastnumbers
    $ pip install natsort[fast]

How to Run Tests
----------------

Please note that `natsort`_ is NOT set-up to support ``python setup.py test``.

The recommended way to run tests is with `tox`_. After installing ``tox``,
running tests is as simple as executing the following in the `natsort`_ directory:

.. code-block:: console

    $ tox

``tox`` will create virtual a virtual environment for your tests and install
all the needed testing requirements for you.  You can specify a particular
python version with the ``-e`` flag, e.g. ``tox -e py36``. Static analysis
is done with ``tox -e flake8``. You can see all available testing environments
with ``tox --listenvs``.

How to Build Documentation
--------------------------

If you want to build the documentation for `natsort`_, it is recommended to
use ``tox``:

.. code-block:: console

    $ tox -e docs

This will place the documentation in ``build/sphinx/html``.

Dropped Deprecated APIs
-----------------------

In `natsort`_ version 6.0.0, the following APIs and functions were removed

 - ``number_type`` keyword argument (deprecated since 3.4.0)
 - ``signed`` keyword argument (deprecated since 3.4.0)
 - ``exp`` keyword argument (deprecated since 3.4.0)
 - ``as_path`` keyword argument (deprecated since 3.4.0)
 - ``py3_safe`` keyword argument (deprecated since 3.4.0)
 - ``ns.TYPESAFE`` (deprecated since version 5.0.0)
 - ``ns.DIGIT`` (deprecated since version 5.0.0)
 - ``ns.VERSION`` (deprecated since version 5.0.0)
 - ``versorted()`` (discouraged since version 4.0.0,
   officially deprecated since version 5.5.0)
 - ``index_versorted()`` (discouraged since version 4.0.0,
   officially deprecated since version 5.5.0)

In general, if you want to determine if you are using deprecated APIs you
can run your code with the following flag

.. code-block:: console

    $ python -Wdefault::DeprecationWarning my-code.py

By default `DeprecationWarnings`_ are not shown, but this will cause them
to be shown. Alternatively, you can just set the environment variable
``PYTHONWARNINGS`` to "default::DeprecationWarning" and then run your code.

Author
------

Seth M. Morton

History
-------

Please visit the changelog `on GitHub`_.

.. _natsort: https://natsort.readthedocs.io/en/stable/index.html
.. _natsorted(): https://natsort.readthedocs.io/en/stable/api.html#natsort.natsorted
.. _natsort_keygen(): https://natsort.readthedocs.io/en/stable/api.html#natsort.natsort_keygen
.. _realsorted(): https://natsort.readthedocs.io/en/stable/api.html#natsort.realsorted
.. _humansorted(): https://natsort.readthedocs.io/en/stable/api.html#natsort.humansorted
.. _os_sorted(): https://natsort.readthedocs.io/en/stable/api.html#natsort.os_sorted
.. _the ns enum: https://natsort.readthedocs.io/en/stable/api.html#natsort.ns
.. _fastnumbers: https://github.com/SethMMorton/fastnumbers
.. _sorted(): https://docs.python.org/3/library/functions.html#sorted
.. _list.sort(): https://docs.python.org/3/library/stdtypes.html#list.sort
.. _key function: https://docs.python.org/3/howto/sorting.html#key-functions
.. _locale: https://docs.python.org/3/library/locale.html
.. _int: https://docs.python.org/3/library/functions.html#int
.. _float: https://docs.python.org/3/library/functions.html#float
.. _str: https://docs.python.org/3/library/stdtypes.html#str
.. _bytes: https://docs.python.org/3/library/stdtypes.html#bytes
.. _list: https://docs.python.org/3/library/stdtypes.html#list
.. _tuple: https://docs.python.org/3/library/stdtypes.html#tuple
.. _TypeError: https://docs.python.org/3/library/exceptions.html#TypeError
.. _DeprecationWarnings: https://docs.python.org/3/library/exceptions.html#DeprecationWarning
.. _"extras" notation: https://packaging.python.org/tutorials/installing-packages/#installing-setuptools-extras
.. _PyICU: https://pypi.org/project/PyICU
.. _tox: https://tox.readthedocs.io/en/latest/
.. _Examples and Recipes: https://github.com/SethMMorton/natsort/wiki/Examples-and-Recipes
.. _How Does Natsort Work?: https://github.com/SethMMorton/natsort/wiki/How-Does-Natsort-Work%3F
.. _API: https://natsort.readthedocs.io/en/stable/api.html
.. _on GitHub: https://github.com/SethMMorton/natsort/blob/main/CHANGELOG.md
.. _file an issue: https://github.com/SethMMorton/natsort/issues/new
.. _look at this issue describing how to debug: https://github.com/SethMMorton/natsort/issues/13#issuecomment-50422375
.. _controlling the case-sensitivity: https://github.com/SethMMorton/natsort/wiki/Examples-and-Recipes#controlling-case-when-sorting
.. _sorting file paths correctly: https://github.com/SethMMorton/natsort/wiki/Examples-and-Recipes#sort-os-generated-paths
.. _allow custom sorting keys: https://github.com/SethMMorton/natsort/wiki/Examples-and-Recipes#using-a-custom-sorting-key
.. _accounting for units: https://github.com/SethMMorton/natsort/wiki/Examples-and-Recipes#accounting-for-units-when-sorting
.. _these version sorting examples: https://github.com/SethMMorton/natsort/wiki/Examples-and-Recipes#sorting-more-expressive-versioning-schemes
.. _locale issues: https://github.com/SethMMorton/natsort/wiki/Possible-Issues-with-natsort.humansorted-or-ns.LOCALE
.. _shell script wiki documentation: https://github.com/SethMMorton/natsort/wiki/Shell-Script


.. image:: https://raw.githubusercontent.com/SALib/SALib/main/docs/assets/logo.png
    :width: 232px
    :align: center

Sensitivity Analysis Library (SALib)
====================================

Python implementations of commonly used sensitivity analysis methods.
Useful in systems modeling to calculate the effects of model inputs or
exogenous factors on outputs of interest.

**Documentation:** `ReadTheDocs <http://salib.readthedocs.org>`__

**Requirements:** `NumPy <http://www.numpy.org/>`__,
`SciPy <http://www.scipy.org/>`__,
`matplotlib <http://matplotlib.org/>`__,
`pandas <http://https://pandas.pydata.org/>`__,
Python 3 (from SALib v1.2 onwards SALib does not officially support Python 2)

**Installation:** ``pip install SALib`` or ``pip install .`` or ``conda install SALib``

**Build Status:** |Build Status| **Test Coverage:** |Coverage Status|

Included methods
----------------

* Sobol Sensitivity Analysis (`Sobol 2001 <http://www.sciencedirect.com/science/article/pii/S0378475400002706>`__,
  `Saltelli 2002 <http://www.sciencedirect.com/science/article/pii/S0010465502002801>`__,
  `Saltelli et al. 2010 <http://www.sciencedirect.com/science/article/pii/S0010465509003087>`__)

* Method of Morris, including groups and optimal trajectories (`Morris
  1991 <http://www.tandfonline.com/doi/abs/10.1080/00401706.1991.10484804>`__,
  `Campolongo et al. 2007 <http://www.sciencedirect.com/science/article/pii/S1364815206002805>`__,
  `Ruano et al. 2012 <https://doi.org/10.1016/j.envsoft.2012.03.008>`__)

* extended Fourier Amplitude Sensitivity Test (eFAST) (`Cukier et al. 1973 <http://scitation.aip.org/content/aip/journal/jcp/59/8/10.1063/1.1680571>`__,
  `Saltelli et al. 1999 <http://amstat.tandfonline.com/doi/abs/10.1080/00401706.1999.10485594>`__, `Pujol (2006) in Iooss et al., (2021) <http://scitation.aip.org/content/aip/journal/jcp/59/8/10.1063/1.1680571>`__)

* Random Balance Designs - Fourier Amplitude Sensitivity Test (RBD-FAST) (`Tarantola et al. 2006 <https://hal.archives-ouvertes.fr/hal-01065897/file/Tarantola06RESS_HAL.pdf>`__,
  `Plischke 2010 <https://doi.org/10.1016/j.ress.2009.11.005>`__,
  `Tissot et al. 2012 <https://doi.org/10.1016/j.ress.2012.06.010>`__)

* Delta
  Moment-Independent Measure (`Borgonovo 2007 <http://www.sciencedirect.com/science/article/pii/S0951832006000883>`__,
  `Plischke et al. 2013 <http://www.sciencedirect.com/science/article/pii/S0377221712008995>`__)

* Derivative-based Global Sensitivity Measure (DGSM) (`Sobol and
  Kucherenko 2009 <http://www.sciencedirect.com/science/article/pii/S0378475409000354>`__)

* Fractional Factorial Sensitivity Analysis
  (`Saltelli et al. 2008 <http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470059974.html>`__)

* High-Dimensional Model Representation (HDMR)
  (`Rabitz et al. 1999 <https://doi.org/10.1016/S0010-4655(98)00152-0>`__, `Li et al. 2010 <https://doi.org/10.1021/jp9096919>`__)

* PAWN (`Pianosi and Wagener 2018 <https://dx.doi.org/10.1016/j.envsoft.2018.07.019>`__, `Pianosi and Wagener 2015 <https://doi.org/10.1016/j.envsoft.2015.01.004>`__)

* Regional Sensitivity Analysis (based on `Hornberger and Spear, 1981 <https://www.osti.gov/biblio/6396608-approach-preliminary-analysis-environmental-systems>`__, `Saltelli et al. 2008 <https://dx.doi.org/10.1002/9780470725184>`__, `Pianosi et al., 2016 <https://dx.doi.org/10.1016/j.envsoft.2016.02.008>`__)

**Contributing:** see `here <CONTRIBUTING.md>`__

Quick Start
-----------

Procedural approach
~~~~~~~~~~~~~~~~~~~

.. code:: python

    from SALib.sample import saltelli
    from SALib.analyze import sobol
    from SALib.test_functions import Ishigami
    import numpy as np

    problem = {
      'num_vars': 3,
      'names': ['x1', 'x2', 'x3'],
      'bounds': [[-np.pi, np.pi]]*3
    }

    # Generate samples
    param_values = saltelli.sample(problem, 1024)

    # Run model (example)
    Y = Ishigami.evaluate(param_values)

    # Perform analysis
    Si = sobol.analyze(problem, Y, print_to_console=True)
    # Returns a dictionary with keys 'S1', 'S1_conf', 'ST', and 'ST_conf'
    # (first and total-order indices with bootstrap confidence intervals)

It's also possible to specify the parameter bounds in a file with 3
columns:

::

    # name lower_bound upper_bound
    P1 0.0 1.0
    P2 0.0 5.0
    ...etc.

Then the ``problem`` dictionary above can be created from the
``read_param_file`` function:

.. code:: python

    from SALib.util import read_param_file
    problem = read_param_file('/path/to/file.txt')
    # ... same as above

Lots of other options are included for parameter files, as well as a
command-line interface. See the `advanced
section in the documentation <https://salib.readthedocs.io/en/latest/advanced.html>`__.

Method chaining approach
~~~~~~~~~~~~~~~~~~~~~~~~

Chaining calls is supported from SALib v1.4

.. code:: python

    from SALib import ProblemSpec
    from SALib.test_functions import Ishigami

    import numpy as np

    # By convention, we assign to "sp" (for "SALib Problem")
    sp = ProblemSpec({
      'names': ['x1', 'x2', 'x3'],   # Name of each parameter
      'bounds': [[-np.pi, np.pi]]*3,  # bounds of each parameter
      'outputs': ['Y']               # name of outputs in expected order
    })

    (sp.sample_saltelli(1024, calc_second_order=True)
       .evaluate(Ishigami.evaluate)
       .analyze_sobol(print_to_console=True))

    print(sp)

    # Samples, model results and analyses can be extracted:
    print(sp.samples)
    print(sp.results)
    print(sp.analysis)

    # Basic plotting functionality is also provided
    sp.plot()

The above is equivalent to the procedural approach shown previously.

Also check out the `FAQ <https://github.com/SALib/SALib/tree/main/FAQ.md>`__ and
`examples <https://github.com/SALib/SALib/tree/main/examples>`__ for a
full description of options for each method.

How to cite SALib
-----------------

If you would like to use our software, please cite it using the following:

    Iwanaga, T., Usher, W., & Herman, J. (2022).
    Toward SALib 2.0: Advancing the accessibility and interpretability of global sensitivity analyses.
    Socio-Environmental Systems Modelling, 4, 18155.
    doi:10.18174/sesmo.18155

    Herman, J. and Usher, W. (2017) SALib: An open-source Python library for
    sensitivity analysis. Journal of Open Source Software, 2(9).
    doi:10.21105/joss.00097

|paper status|

If you use BibTeX, cite using the following entries::

    @article{Iwanaga2022,
      title = {Toward {SALib} 2.0: {Advancing} the accessibility and interpretability of global sensitivity analyses},
      volume = {4},
      url = {https://sesmo.org/article/view/18155},
      doi = {10.18174/sesmo.18155},
      journal = {Socio-Environmental Systems Modelling},
      author = {Iwanaga, Takuya and Usher, William and Herman, Jonathan},
      month = may,
      year = {2022},
      pages = {18155},
    }

    @article{Herman2017,
      doi = {10.21105/joss.00097},
      url = {https://doi.org/10.21105/joss.00097},
      year  = {2017},
      month = {jan},
      publisher = {The Open Journal},
      volume = {2},
      number = {9},
      author = {Jon Herman and Will Usher},
      title = {{SALib}: An open-source Python library for Sensitivity Analysis},
      journal = {The Journal of Open Source Software}
    }

Projects that use SALib
-----------------------

Many projects now use the Global Sensitivity Analysis features provided by
SALib. Here is a selection:

Software
~~~~~~~~

* `The City Energy Analyst <https://github.com/architecture-building-systems/CEAforArcGIS>`_
* `pynoddy <https://github.com/flohorovicic/pynoddy>`_
* `savvy <https://github.com/houghb/savvy>`_
* `rhodium <https://github.com/Project-Platypus/Rhodium>`_
* `pySur <https://github.com/MastenSpace/pysur>`_
* `EMA workbench <https://github.com/quaquel/EMAworkbench>`_
* `Brain/Circulation Model Developer <https://github.com/bcmd/BCMD>`_
* `DAE Tools <http://daetools.com/>`_
* `agentpy <https://github.com/JoelForamitti/agentpy>`_
* `uncertainpy <https://github.com/simetenn/uncertainpy>`_
* `CLIMADA <https://github.com/CLIMADA-project/climada_python>`_

Blogs
~~~~~

* `Sensitivity Analysis in Python <http://www.perrygeo.com/sensitivity-analysis-in-python.html>`_
* `Sensitivity Analysis with SALib <http://keyboardscientist.weebly.com/blog/sensitivity-analysis-with-salib>`_
* `Running Sobol using SALib <https://waterprogramming.wordpress.com/2013/08/05/running-sobol-sensitivity-analysis-using-salib/>`_
* `Extensions of SALib for more complex sensitivity analyses <https://waterprogramming.wordpress.com/2014/02/11/extensions-of-salib-for-more-complex-sensitivity-analyses/>`_

Videos
~~~~~~

* `PyData Presentation on SALib <https://youtu.be/gkR_lz5OptU>`_

If you would like to be added to this list, please submit a pull request,
or create an issue.

Many thanks for using SALib.

How to contribute
-----------------

See `here <CONTRIBUTING.md>`__ for how to contribute to SALib.

License
-------

Copyright (C) 2012-2019 Jon Herman, Will Usher, and others. Versions v0.5 and
later are released under the `MIT license <LICENSE.md>`__.

.. |Build Status| image:: https://travis-ci.com/SALib/SALib.svg?branch=master
   :target: https://travis-ci.com/SALib/SALib
.. |Coverage Status| image:: https://img.shields.io/coveralls/SALib/SALib.svg
   :target: https://coveralls.io/r/SALib/SALib
.. |Code Issues| image:: https://www.quantifiedcode.com/api/v1/project/ed62e70f899e4ec8af4ea6b2212d4b30/badge.svg
   :target: https://www.quantifiedcode.com/app/project/ed62e70f899e4ec8af4ea6b2212d4b30
.. |paper status| image:: http://joss.theoj.org/papers/431262803744581c1d4b6a95892d3343/status.svg
   :target: http://joss.theoj.org/papers/431262803744581c1d4b6a95892d3343


Pyrollbar

  Proactively discover, predict, and resolve errors in real-time with Rollbar’s error monitoring platform. Start tracking errors today!

![Build Status](https://github.com/rollbar/pyrollbar/workflows/Pyrollbar%20CI/badge.svg?tag=latest)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/rollbar)

Python notifier for reporting exceptions, errors, and log messages to [Rollbar](https://rollbar.com).

## Key benefits of using Pyrollbar are:
- **Frameworks:** Pyrollbar supports popular Python frameworks such as Django, Flask, FastAPI, AWS Lambda and more!
- **Automatic error grouping:** Rollbar aggregates Occurrences caused by the same error into Items that represent application issues. Learn more about reducing log noise.
- **Advanced search:** Filter items by many different properties. Learn more about search.
- **Customizable notifications:** Rollbar supports several messaging and incident management tools where your team can get notified about errors and important events by real-time alerts. Learn more about Rollbar notifications.

## Versions Supported

| PyRollbar Version | Python Version Compatibility                  | Support Level       |
|-------------------|-----------------------------------------------|---------------------|
| 1.0.0             | 3.6, 3.7. 3.8, 3.9, 3.10, 3.11                | Full                |
| 0.16.3            | 2.7, 3.4, 3.5, 3.6, 3.7. 3.8, 3.9, 3.10, 3.11 | Security Fixes Only |

#### Support Level Definitions

**Full** - We will support new features of the library and test against all supported versions.

**Security Fixes Only** - We will only provide critical security fixes for the library.

## Setup Instructions

1. [Sign up for a Rollbar account](https://rollbar.com/signup)
2. Follow the [Quick Start](https://docs.rollbar.com/docs/python#section-quick-start) instructions in our [Python SDK docs](https://docs.rollbar.com/docs/python) to install pyrollbar and configure it for your platform.

## Usage and Reference

For complete usage instructions and configuration reference, see our [Python SDK docs](https://docs.rollbar.com/docs/python).

## Release History & Changelog

See our [Releases](https://github.com/rollbar/pyrollbar/releases) page for a list of all releases, including changes.

## Help / Support

If you run into any issues, please email us at [support@rollbar.com](mailto:support@rollbar.com)

For bug reports, please [open an issue on GitHub](https://github.com/rollbar/pyrollbar/issues/new).

## Contributing

1. Fork it
2. Create your feature branch (```git checkout -b my-new-feature```).
3. Commit your changes (```git commit -am 'Added some feature'```)
4. Push to the branch (```git push origin my-new-feature```)
5. Create new Pull Request

Tests are in `rollbar/test`. To run the tests: `python setup.py test`


# brainrender

*A user-friendly python library to create high-quality, 3D neuro-anatomical renderings combining data from publicly available brain atlases with user-generated experimental data.*

[![Python Version](https://img.shields.io/pypi/pyversions/brainrender.svg)](https://pypi.org/project/brainrender)
[![PyPI](https://img.shields.io/pypi/v/brainrender.svg)](https://pypi.org/project/brainrender)
[![tests](https://github.com/brainglobe/brainrender/workflows/tests/badge.svg)](https://github.com/brainglobe/brainrender/actions)
[![codecov](https://codecov.io/gh/brainglobe/brainrender/graph/badge.svg)](https://codecov.io/gh/brainglobe/brainrender)
[![Downloads](https://static.pepy.tech/badge/brainrender)](https://pepy.tech/project/brainrender)

![Example gallery](https://iiif.elifesciences.org/lax/65751%2Felife-65751-fig3-v3.tif/full/,1500/0/default.jpg)

From: Claudi et al. (2021) Visualizing anatomically registered data with brainrender. eLife

## Documentation

brainrender is a project of the BrainGlobe Initiative, which is a collaborative effort to develop a suite of Python-based software tools for computational neuroanatomy. A comprehensive online documentation for brainrender can be found on the BrainGlobe website [here](https://brainglobe.info/documentation/brainrender/index.html).

Furthermore, an open-access journal article describing BrainRender has been published in eLife, available [here](https://doi.org/10.7554/eLife.65751).

## Installation

From PyPI:

```
pip install brainrender
```

If you encounter any issues, please ask a question on the [image.sc forum](https://forum.image.sc/tag/brainglobe) tagging your question with `brainglobe`.

## Quickstart

``` python
import random

import numpy as np

from brainrender import Scene
from brainrender.actors import Points

def get_n_random_points_in_region(region, N):
    """
    Gets N random points inside (or on the surface) of a mesh
    """

    region_bounds = region.mesh.bounds()
    X = np.random.randint(region_bounds[0], region_bounds[1], size=10000)
    Y = np.random.randint(region_bounds[2], region_bounds[3], size=10000)
    Z = np.random.randint(region_bounds[4], region_bounds[5], size=10000)
    pts = [[x, y, z] for x, y, z in zip(X, Y, Z)]

    ipts = region.mesh.inside_points(pts).coordinates
    return np.vstack(random.choices(ipts, k=N))

# Display the Allen Brain mouse atlas.
scene = Scene(atlas_name="allen_mouse_25um", title="Cells in primary visual cortex")

# Display a brain region
primary_visual = scene.add_brain_region("VISp", alpha=0.2)

# Get a numpy array with (fake) coordinates of some labelled cells
coordinates = get_n_random_points_in_region(primary_visual, 2000)

# Create a Points actor
cells = Points(coordinates)

# Add to scene
scene.add(cells)

# Add label to the brain region
scene.add_label(primary_visual, "Primary visual cortex")

# Display the figure.
scene.render()

```

## Citing brainrender

If you use brainrender in your scientific work, please cite:
```
Claudi, F., Tyson, A. L., Petrucco, L., Margrie, T.W., Portugues, R.,  Branco, T. (2021) "Visualizing anatomically registered data with Brainrender" <i>eLife</i> 2021;10:e65751 [doi.org/10.7554/eLife.65751](https://doi.org/10.7554/eLife.65751)
```

BibTeX:

``` bibtex
@article{Claudi2021,
author = {Claudi, Federico and Tyson, Adam L. and Petrucco, Luigi and Margrie, Troy W. and Portugues, Ruben and Branco, Tiago},
doi = {10.7554/eLife.65751},
issn = {2050084X},
journal = {eLife},
pages = {1--16},
pmid = {33739286},
title = {{Visualizing anatomically registered data with brainrender}},
volume = {10},
year = {2021}
}

```

## Contributing

Contributions to brainrender are more than welcome. Please see the [developers guide](https://brainglobe.info/community/developers/index.html).


[![License](https://img.shields.io/github/license/analysiscenter/batchflow.svg)](https://www.apache.org/licenses/LICENSE-2.0)
[![Python](https://img.shields.io/badge/python-3.8-blue.svg)](https://python.org)
[![PyTorch](https://img.shields.io/badge/PyTorch-2.0-orange.svg)](https://pytorch.org)
[![codecov](https://codecov.io/gh/analysiscenter/batchflow/branch/master/graph/badge.svg)](https://codecov.io/gh/analysiscenter/batchflow)
[![PyPI](https://badge.fury.io/py/batchflow.svg)](https://badge.fury.io/py/batchflow)
[![Status](https://github.com/analysiscenter/batchflow/workflows/status/badge.svg)](https://github.com/analysiscenter/batchflow/actions?query=workflow%3Astatus)

# BatchFlow

`BatchFlow` helps you conveniently work with random or sequential batches of your data
and define data processing and machine learning workflows even for datasets that do not fit into memory.

For more details see [the documentation and tutorials](https://analysiscenter.github.io/batchflow/).

Main features:
- flexible batch generaton
- deterministic and stochastic pipelines
- datasets and pipelines joins and merges
- data processing actions
- flexible model configuration
- within batch parallelism
- batch prefetching
- ready to use ML models and proven NN architectures
- convenient layers and helper functions to build custom models
- a powerful research engine with parallel model training and extended experiment logging.

## Basic usage

```python
my_workflow = my_dataset.pipeline()
              .load('/some/path')
              .do_something()
              .do_something_else()
              .some_additional_action()
              .save('/to/other/path')
```
The trick here is that all the processing actions are lazy. They are not executed until their results are needed, e.g. when you request a preprocessed batch:
```python
my_workflow.run(BATCH_SIZE, shuffle=True, n_epochs=5)
```
or
```python
for batch in my_workflow.gen_batch(BATCH_SIZE, shuffle=True, n_epochs=5):
    # only now the actions are fired and data is being changed with the workflow defined earlier
    # actions are executed one by one and here you get a fully processed batch
```
or
```python
NUM_ITERS = 1000
for i in range(NUM_ITERS):
    processed_batch = my_workflow.next_batch(BATCH_SIZE, shuffle=True, n_epochs=None)
    # only now the actions are fired and data is changed with the workflow defined earlier
    # actions are executed one by one and here you get a fully processed batch
```

## Train a neural network
`BatchFlow` includes ready-to-use proven architectures like VGG, Inception, ResNet and many others.
To apply them to your data just choose a model, specify the inputs (like the number of classes or images shape)
and call `train_model`. Of course, you can also choose a loss function, an optimizer and many other parameters, if you want.
```python
from batchflow.models.torch import ResNet34

my_workflow = my_dataset.pipeline()
              .init_model('model', ResNet34, config={'loss': 'ce', 'classes': 10})
              .load('/some/path')
              .some_transform()
              .another_transform()
              .train_model('ResNet34', inputs=B.images, targets=B.labels)
              .run(BATCH_SIZE, shuffle=True)
```

For more advanced cases and detailed API see [the documentation](https://analysiscenter.github.io/batchflow/).

## Installation

> `BatchFlow` module is in the beta stage. Your suggestions and improvements are very welcome.

> `BatchFlow` supports python 3.6 or higher.

### Stable python package

With [poetry](https://python-poetry.org/)
```
poetry add batchflow
```

With old-fashioned [pip](https://pip.pypa.io/en/stable/)
```
pip3 install batchflow
```

### Development version

With [poetry](https://python-poetry.org/)
```
poetry add --editable git+https://github.com/analysiscenter/batchflow
```

With old-fashioned [pip](https://pip.pypa.io/en/stable/)
```
pip install --editable git+https://github.com/analysiscenter/batchflow
```

### Extras
Some `batchflow` functions and classed require additional dependencies.
In order to use that functionality you might need to install `batchflow` with extras (e.g. `batchflow[nn]`):

- image - working with image datasets and plotting
- nn - for neural networks (includes torch, torchvision, ...)
- datasets - loading standard datasets (MNIST, CIFAR, ...)
- profile - performance profiling
- jupyter - utility functions for notebooks
- research - multiprocess research
- telegram - for monitoring pipelines via a telegram bot
- dev - batchflow development (pylint, pytest, ...)

You can install several extras at once, like `batchflow[image,nn,research]`.

## Projects based on BatchFlow
- [SeismiQB](https://github.com/GeoscienceML/seismiqb) - ML for seismic interpretation
- [SeismicPro](https://github.com/GeoscienceML/SeismicPro) - ML for seismic processing
- [PyDEns](https://github.com/analysiscenter/pydens) - DL Solver for ODE and PDE
- [RadIO](https://github.com/analysiscenter/radio) - ML for CT imaging
- [CardIO](https://github.com/analysiscenter/cardio) - ML for heart signals

## Citing BatchFlow
Please cite BatchFlow in your publications if it helps your research.

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.1041203.svg)](https://doi.org/10.5281/zenodo.1041203)

```
Roman Khudorozhkov et al. BatchFlow library for fast ML workflows. 2017. doi:10.5281/zenodo.1041203
```

```
@misc{roman_kh_2017_1041203,
  author       = {Khudorozhkov, Roman and others},
  title        = {BatchFlow library for fast ML workflows},
  year         = 2017,
  doi          = {10.5281/zenodo.1041203},
  url          = {https://doi.org/10.5281/zenodo.1041203}
}
```