The Goal of Testing
The goal of all testing is to find bugs before the users do—preferably as soon in the development cycle as possible when it's cheapest to fix. A bug in a critical system might mean orders aren't placed or critical third-party services are unavailable.
While end-to-end testing can't completely replace lower-level testing such as unit, functional, or integration testing, because it tests the final product it's often the perfect place to start a new testing program when those lower levels are missing. If the final product isn't working correctly, a developer can look at the code and pinpoint the error.
What is End-to-End Testing?
Unlike other testing types, end-to-end testing works with the final website by performing the tasks a real user performs work. For instance, a test may include logging in as a specific user with certain permissions and testing that access to the correct locations of the site are granted. Another test may confirm that a certain image appears in the footer only when the site is viewed in the Spanish language.
A Testing Vision for Your Site
Ideally, it's best to create tests while a Drupal site is being created. However, many Drupal sites are made without constructing lower-level tests such as unit tests. Although a site can still launch with high quality in this way, often changes over time introduce bugs that aren't caught without the strict quality scrutiny a site about to launch often gets. That's where end-to-end Cypress testing comes in.
A typical testing regime wants to assure all stakeholders that the site is working properly. Start small and increase the depth of testing as the project continues:
- Smoke tests. Start by making a small number of tests that run quickly. Within 15 minutes, the tests should determine that most if not all of the critical functions are working, such as logging in, having the ability to perform critical searches, or retrieving data from an external service.
- Component testing. Parts of the site (components) have specific functions such as displaying data from the database in a user-friendly way or submitting a form. A large site will have dozens of components of varying complexity and importance. Component testing will take (often much) longer to perform than smoke tests.
- Compatibility testing. These tests ensure the site works correctly across multiple browsers and devices. For compatibility testing to be performed alongside component testing—and to get a result in a reasonable time frame—requires many test runners operating in parallel.
Note that performance testing is not typically something that Cypress would perform. It's not set up to run the multiple thousands of test runners in parallel (and often from different parts of the world) required to simulate a fully-loaded website.
Target Any Destination Server
Test scripts should be designed to run against any target server (local, stage, and production, for instance) and be smart about how to interact with that destination. For instance, running the entire test suite may exclude a few tests on production because they may leave lasting database changes. Or tests targeting a staging server may use different accounts than those targeting production.
When Do the Tests Run?
Typical times to run tests are:
- during the sprint to find errors as early as possible; this can be overnight or immediately before or after a merge
- before the release against staging servers (servers that are set up as close as possible to the production environment)
- immediately after release to ensure all parts of the site on production are working
Testing Best Practices
We've got a list of best practices for Cypress and Drupal tests that we use while writing tests.
One of the most important best practices is to write tests that are resilient to changes. For instance, often it's not critical that a menu item is located in a particular order. If the test is written that looks for the login item in the fifth position, what if a menu item is inserted or deleted above it? The login will fail—and so will perhaps dozens of tests that require a login.
One way to make the test more resilient is to scan the menu items until the login is found. Or, even better, a login can often be executed without going through the menu by submitting a form.
Many Drupal sites have multiple languages. A particular challenge is creating tests for multilingual sites in a way that is clean and easy to maintain. One definitely doesn't want to simply copy and paste a test for each language. The resulting code bloat simply creates a maintenance problem in the future.
Who Writes the Tests?
For example, typically you want to run a routine on the target environment to set it up for testing. The target environment may be the developer's machine, a feature preview instance (such as what Tugboat provides), a staging environment, or the production environment. These setup routines may require that Drush commands be executed on the target environment. Enabling QA test accounts (that have specific roles assigned to them expected by the tests) is one type of setup task. These sorts of things will need the help of a Drupal developer to accomplish.
However, ideally, developers should submit new tests or changes to existing tests along with their new code. They should work closely with the QA team to ensure duplicate tests are not being written.
Here is a summary.
Stay connected with our latest news, articles, webinars, and more.