From fc93e481a99552e6142d148ebdf90ebcd5b76632 Mon Sep 17 00:00:00 2001 From: David Lambauer Date: Mon, 21 Aug 2023 13:06:38 +0200 Subject: [PATCH] Updated the docs according to feedback --- integration-testing.md | 357 ++++++++++++++++++++++++++++++++--------- 1 file changed, 284 insertions(+), 73 deletions(-) diff --git a/integration-testing.md b/integration-testing.md index 80474d1..0edc32f 100644 --- a/integration-testing.md +++ b/integration-testing.md @@ -4,48 +4,46 @@ - [Overview](#overview) - [Running Integration Tests in Magento 2](#running-integration-tests-in-magento-2) - [Setting Up the Integration Test Framework](#setting-up-the-integration-test-framework) - - [Command Line Interface (CLI)](#command-line-interface) - - [PhpStorm IDE](#phpstorm-ide) + - [Command Line Interface (CLI)](#command-line-interface) + - [PhpStorm IDE](#phpstorm-ide) - [Preparing Integration Test Execution](#preparing-integration-test-execution) - - [Integration Test Database](#integration-test-database) - - [Configuring the Framework for Test Environment](#configuring-the-framework-for-test-environment) - - [Adjusting the PHPUnit Configuration File](#adjusting-the-phpunit-configuration-file) - - [TESTS_CLEANUP Constant](#tests-cleanup-constant) - - [RABBITMQ_MANAGEMENT_PORT Constant](#rabbitmq-management-port-constant) - - [PHP memory_limit](#php-memory-limit) + - [Integration Test Database](#integration-test-database) + - [Configuring the Framework for Test Environment](#configuring-the-framework-for-test-environment) + - [Adjusting the PHPUnit Configuration File](#adjusting-the-phpunit-configuration-file) + - [TESTS_CLEANUP Constant](#tests-cleanup-constant) - [Executing Third-Party Integration Tests](#executing-third-party-integration-tests) - [Running Integration Tests in the CLI](#running-integration-tests-in-the-cli) - - [Running All Integration Tests](#running-all-integration-tests) - - [Running Only a Custom Test Suite](#running-only-a-custom-test-suite) - - [Running Tests from a Specific Directory Tree](#running-tests-from-a-specific-directory-tree) - - [Running a Single Test Class](#running-a-single-test-class) - - [Running a Single Test within a Test Class](#running-a-single-test-within-a-test-class) + - [Running All Integration Tests](#running-all-integration-tests) + - [Running Only a Custom Test Suite](#running-only-a-custom-test-suite) + - [Running Tests from a Specific Directory Tree](#running-tests-from-a-specific-directory-tree) + - [Running a Single Test Class](#running-a-single-test-class) + - [Running a Single Test within a Test Class](#running-a-single-test-within-a-test-class) - [Common Mistakes](#common-mistakes) - - [Could Not Read Files Specified as Arguments](#could-not-read-files-specified-as-arguments) - - [Could Not Read “dev/tests/integration/phpunit.xml”](#could-not-read-dev-tests-integration-phpunit-xml) - - [Unable to Connect to MySQL](#unable-to-connect-to-mysql) + - [Could Not Read Files Specified as Arguments](#could-not-read-files-specified-as-arguments) + - [Could Not Read “dev/tests/integration/phpunit.xml”](#could-not-read-dev-tests-integration-phpunit-xml) + - [Unable to Connect to MySQL](#unable-to-connect-to-mysql) - [Running Integration Tests in PhpStorm](#running-integration-tests-in-phpstorm) - - [Creating an Integration Test Run Configuration](#creating-an-integration-test-run-configuration) - - [Using the Integration Test Configuration File](#using-the-integration-test-configuration-file) -- [Integration Tests File Structure](#integration-tests-file-structure) + - [Creating an Integration Test Run Configuration](#creating-an-integration-test-run-configuration) + - [Using the Integration Test Configuration File](#using-the-integration-test-configuration-file) - [Using Data Providers in Integration Tests](#using-data-providers-in-integration-tests) - - [Creating a Data Provider](#creating-a-data-provider) - - [Using a Data Provider](#using-a-data-provider) - - [Benefits of Using Data Providers](#benefits-of-using-data-providers) + - [Creating a Data Provider](#creating-a-data-provider) + - [Using a Data Provider](#using-a-data-provider) + - [Benefits of Using Data Providers](#benefits-of-using-data-providers) - [Using Fixtures in Integration Tests](#using-fixtures-in-integration-tests) - - [Creating Fixtures](#creating-fixtures) - - [Using Fixtures](#using-fixtures) - - [Fixture Rollback Scripts](#fixture-rollback-scripts) - - [Benefits of Using Fixtures](#benefits-of-using-fixtures) + - [Creating Fixtures](#creating-fixtures) + - [Using Fixtures](#using-fixtures) + - [Fixture Rollback Scripts](#fixture-rollback-scripts) + - [Benefits of Using Fixtures](#benefits-of-using-fixtures) - [Understanding DocBlock Annotations in Magento 2 Integration Tests](#understanding-docblock-annotations-in-magento-2-integration-tests) - - [@magentoDataFixture](#magentodatafixture) - - [@magentoDbIsolation](#magentodbisolation) - - [@magentoAppIsolation](#magentoappisolation) - - [@magentoConfigFixture](#magentoconfigfixture) - - [@magentoAppArea](#magentoapparea) - - [@magentoCache](#magentocache) + - [@magentoDataFixture](#magentodatafixture) + - [@magentoDbIsolation](#magentodbisolation) + - [@magentoAppIsolation](#magentoappisolation) + - [@magentoConfigFixture](#magentoconfigfixture) + - [@magentoAppArea](#magentoapparea) + - [@magentoCache](#magentocache) + ## Overview Integration testing is a crucial phase in the software development lifecycle that involves testing individual software @@ -68,6 +66,7 @@ verify that the system meets its overall functional and performance requirements Now, let's dive into the specifics of running integration tests in Magento 2. + ## Running Integration Tests in Magento 2 Integration tests in Magento 2 require a runtime environment, necessitating some preparation before execution. These @@ -75,6 +74,7 @@ tests can be run using the command line interface (CLI) or within an Integrated PhpStorm. + ## Setting Up the Integration Test Framework To execute integration tests, you need to create and configure a test database. Depending on your requirements, you @@ -82,12 +82,14 @@ might also want to adjust the PHPUnit configuration. For more details, refer to Test Execution. + ### Command Line Interface (CLI) The CLI is a suitable option for running tests locally during development or on remote servers during continuous integration. For more information, refer to the section on Running Integration Tests in the CLI. + ### PhpStorm IDE Running integration tests within an IDE like PhpStorm is convenient during development, especially when writing a new @@ -95,6 +97,7 @@ integration test. However, apart from convenience, there are no additional benef console. For more information, refer to the section on Running Integration Tests in PhpStorm. + ## Preparing Integration Test Execution Before using the Magento integration test framework, you need to prepare the test environment. This involves setting up @@ -102,6 +105,7 @@ a dedicated integration test database, configuring the test framework database, aligns with the purpose of the integration test execution. + ### Integration Test Database By default, the test framework installs a fresh Magento test database for every integration test run. It's crucial not @@ -121,12 +125,13 @@ ON magento_integration_tests.* TO 'magento2_test_user'@'localhost' IDENTIFIED BY Remember to replace the example database, username, and password with ones that suit your requirements and conventions. + ### Configuring the Framework for Test Environment The Magento 2 integration test framework provides a configuration file template located -at `mage2ce/dev/tests/integration/etc/install-config-mysql.php.dist`. +at `dev/tests/integration/etc/install-config-mysql.php.dist`. -Copy this file to `mage2ce/dev/tests/integration/etc/install-config-mysql.php` (without the `.dist` suffix) and add your +Copy this file to `dev/tests/integration/etc/install-config-mysql.php` (without the `.dist` suffix) and add your test database access credentials. The contents will look similar to the following: ```php @@ -155,6 +160,7 @@ Leave all the settings that do not start with `db-` and `amqp-` at their default setup options available to the `setup:install` command in the test configuration file. + ### Adjusting the PHPUnit Configuration File The default integration test configuration is located in the `dev/tests/integration/phpunit.xml.dist` file. Without @@ -168,6 +174,7 @@ the available configuration settings, refer to the [PHPUnit documentation](https://phpunit.readthedocs.io/en/9.5/configuration.html) and comments in the default file. + #### TESTS_CLEANUP Constant The default value for the `TESTS_CLEANUP` constant is `enabled`. If set to `enabled`, the integration test framework @@ -191,30 +198,8 @@ the cache and the other files, remove the directory: rm -r dev/tests/integration/tmp/sandbox-* ``` - -#### RABBITMQ_MANAGEMENT_PORT Constant - -The default value for the `RABBITMQ_MANAGEMENT_PORT` constant is `15672`. The RabbitMQ Management Plugin APIs are -accessible on port 15672 by default. This constant provides the configuration value when the test environment is not -using the default. Tests depending on `Magento\TestFramework\Helper\Amqp`, often used for validating queue and exchange -status, depend on the management plugin being enabled. Beginning in 2.3.1, the helper no longer assumes RabbitMQ is -running on localhost and will use the connection configuration defined in `env.php`. - - -#### PHP memory_limit - -The default `phpunit.xml.dist` file does not contain any PHP `memory_limit` settings. However, sometimes the PHP -configuration restricts the amount of memory PHP may consume, which can make it impossible to run the integration tests. - -You can turn off the PHP memory limit by adding the following configuration to the `` section of the integration -test `phpunit.xml` file: - -```xml - - -``` - + ## Executing Third-Party Integration Tests Magento code integration tests reside in the `dev/tests/integration/testsuite` directory. For core tests, it makes sense @@ -249,6 +234,7 @@ php ../../../vendor/bin/phpunit --testsuite "Third Party Integration Tests" ``` + ## Running Integration Tests in the CLI The most common way to execute integration tests is using the CLI. Ensure you have prepared the integration test @@ -259,6 +245,7 @@ that directory and will be picked up by `phpunit` automatically, without the nee option. + ### Running All Integration Tests By default, if no additional arguments are specified, the test configuration executes all integration tests in @@ -270,6 +257,7 @@ cd dev/tests/integration ``` + ### Running Only a Custom Test Suite PHPUnit offers several ways to only execute a subset of tests. For example, it is common to only execute a single test @@ -281,6 +269,7 @@ cd dev/tests/integration ``` + ### Running Tests from a Specific Directory Tree To execute only the tests within a specific directory (for example an extension), pass the path to that directory as an @@ -292,6 +281,7 @@ cd dev/tests/integration ``` + ### Running a Single Test Class When developing a new integration test class, it is common to run only that single test many times. Pass the path to the @@ -303,6 +293,7 @@ cd dev/tests/integration ``` + ### Running a Single Test within a Test Class You can run only a single test within a test class by specifying the test class together with the `--filter` argument @@ -314,14 +305,17 @@ cd dev/tests/integration ``` + ## Common Mistakes + ### Could Not Read Files Specified as Arguments This error occurs if the integration tests are executed from the wrong directory. + ### Could Not Read “dev/tests/integration/phpunit.xml” This error occurs if the integration tests are executed from a different directory than `dev/tests/integration`. To fix @@ -329,6 +323,7 @@ the issue, change to the `dev/tests/integration` directory, adjust any relative again. + ### Unable to Connect to MySQL The PHP interpreter must be able to connect to the test database. By default, this means the tests have to run on the @@ -349,12 +344,14 @@ exception 'PDOException' with message 'SQLSTATE[HY000] [2002] No such file or di There are many ways this problem can be resolved, but the easiest is to run the tests in the virtual machine as well. + ## Running Integration Tests in PhpStorm When writing new integration tests or during debugging, it is convenient to execute tests from within the PhpStorm IDE. Ensure you have prepared the integration test environment before starting. + ### Creating an Integration Test Run Configuration Setting up a run configuration for integration tests is very similar to creating a run configuration for unit tests. See @@ -362,28 +359,14 @@ Running Unit Tests in PhpStorm for instructions on creating a basic run configur test to use the configuration file. + ### Using the Integration Test Configuration File The only difference in the run configuration is that the integration test `phpunit.xml.dist` or `phpunit.xml` configuration file from the `dev/tests/integration` directory must be selected. - -## Integration Tests File Structure - -The root folder for the Magento integration tests suite — `/dev/tests/integration` — contains the -following sub-folders and files: - -- `framework/` – Integration testing framework scripts, configuration files, and classes. -- `Magento/` – A set of classes that implement the Magento integration tests framework. -- `bootstrap.php` – The PHPUnit bootstrap script. -- `etc/install-config-.php` – A configuration file that provides values for installing the Magento - application. -- `testsuite/` – The test suite. -- `tmp/` – A writable directory for storing temporary data during test execution. -- `sandbox-/` – The folder where each Magento instance stores temporary and configuration data. -- `phpunit.xml.dist` – A PHPUnit configuration file. - + ## Using Data Providers in Integration Tests Data providers are a powerful feature of PHPUnit that allow you to run a test multiple times with different data sets. @@ -391,6 +374,7 @@ They are methods that return an array of arrays, where each array is the set of of the test. + ### Creating a Data Provider A data provider method must be public and either return an array of arrays or an object that implements the `Iterator` @@ -407,3 +391,230 @@ public function provider() ]; } ``` + + + +### Using a Data Provider + +To use a data provider, add the `@dataProvider` annotation to your test method and specify the name of the data provider +method. The test method will then be run once for each set of parameters returned by the data provider. + +```php +/** + * @dataProvider provider + */ +public function testMethod($param1, $param2) +{ + // Test code here +} +``` + +In this example, `testMethod` would be run twice: once with `$param1` set to `'value1'` and `$param2` set to `'value2'`, +and once with `$param1` set to `'value3'` and `$param2` set to `'value4'`. + + + +### Benefits of Using Data Providers + +Data providers can help you write more efficient and effective tests. They allow you to easily test a method with a +variety of data, which can help you ensure that your code works correctly in different scenarios. They also make your +tests more readable and maintainable, as you can easily see the data that is being used for each test and add or remove +data sets as needed. + + + +## Using Fixtures in Integration Tests + +In the context of testing, a fixture is a fixed state of a set of objects used as a baseline for running tests. The +purpose of a test fixture is to ensure that there is a well-known and fixed environment in which tests are run so that +results are repeatable. Examples of fixtures: + +- Preparation of input data and setup/creation of fake or mock objects +- Loading a database with a specific, known set of data +- Copying a specific known set of files creating a test fixture will create a set of objects initialized to certain + states. + + + +### Creating Fixtures + +In Magento 2, fixtures are PHP scripts that set required preconditions. Here is an example of a fixture script: + +```php + + +### Using Fixtures + +To use a fixture in your test, you need to add a special annotation to your test method: + +```php +/** + * @magentoDataFixture Magento/Catalog/_files/categories.php + */ +public function testMethod() +{ + // Test code here +} +``` + +In this example, before `testMethod` is run, the `categories.php` fixture script will be executed. This script will set +up the necessary preconditions for the test. The `@magentoDataFixture Magento/Catalog/_files/categories.php` annotation +will resolve to `dev/tests/integration/testsuite/Magento/Catalog/_files/categories.php`. + + +### Fixture Rollback Scripts + +Magento 2 also allows you to create rollback scripts for your fixtures. These scripts are used to return the system to +its previous state after the test is run. This is useful for cleaning up any changes that were made during the test. + +Rollback scripts are named the same as the fixture script, but with `_rollback` appended to the name. For example, the +rollback script for `categories.php` would be `categories_rollback.php`. + +To use a rollback script, you don't need to do anything special in your test method. Magento 2 will automatically run +the appropriate rollback script after each test. + + + +### Benefits of Using Fixtures + +Fixtures allow you to create consistent, repeatable environments for your tests. This can help you ensure that your +tests are reliable and that your code works correctly under different conditions. Fixtures also make your tests easier +to write and understand, as you can clearly see the preconditions for each test. + + + +## Understanding DocBlock Annotations in Magento 2 Integration Tests + +In Magento 2 integration tests, DocBlock annotations are used to provide metadata about the test, such as preconditions, +expectations, and dependencies. Here are some of the most commonly used annotations: + + + +### @magentoDataFixture + +This annotation is used to specify a fixture script that sets up the necessary preconditions for the test. The script is +run before the test is executed. + +```php +/** + * @magentoDataFixture Magento/Catalog/_files/categories.php + */ +public function testMethod() +{ + // Test code here +} +``` + + + +### @magentoDbIsolation + +This annotation is used to enable or disable database isolation for the test. If database isolation is enabled, any +changes made to the database in the test will be rolled back after the test is run. This helps to ensure test isolation. + +```php +/** + * @magentoDbIsolation enabled + */ +public function testMethod() +{ + // Test code here +} +``` + + + +### @magentoAppIsolation + +This annotation is used to enable or disable application isolation for the test. If application isolation is enabled, +the Magento application is reinitialized after the test is run. This helps to ensure that the state of the application +does not affect other tests. + +```php +/** + * @magentoAppIsolation enabled + */ +public function testMethod() +{ + // Test code here +} +``` + + + +### @magentoConfigFixture + +This annotation is used to set a configuration value for the test. The configuration value is set before the test is run +and is restored to its original value after the test is run. + +```php +/** + * @magentoConfigFixture current_store web/unsecure/base_url http://example.com/ + */ +public function testMethod() +{ + // Test code here +} +``` + + + +### @magentoAppArea + +This annotation is used to set the application area for the test, such as 'frontend', 'adminhtml', 'global', etc. The +application area affects the behavior of the Magento application. + +```php +/** + * @magentoAppArea frontend + */ +public function testMethod() +{ + // Test code here +} +``` + + + +### @magentoCache + +This annotation is used to enable or disable cache types for the test. The cache types are set before the test is run +and are restored to their original state after the test is run. + +```php +/** + * @magentoCache full_page enabled + */ +public function testMethod() +{ + // Test code here +} +``` + +These annotations provide a powerful way to control the environment in which your tests are run, helping to ensure that +your tests are reliable, repeatable, and isolated from each other. \ No newline at end of file