Engineering practices for test automation process

Photo by Josue Isai Ramos Figueroa on Unsplash

List of an ideas concerning test automation. Based on my and other people’s experiences.

Engineering practices for test automation

• Use tools to speed up testing
• Always pay technical dent
  Self-service infrastructure — everything must start (tests, or environments) with one button
• Self documented tests — everything has to be in the code, documentation becomes obsolete
• Postmortem should be conducted and documented after each incident

Testing automation framework

• Use VCS to store a code, use GitFlow to provide changes safely
• Automatic rollout of a new version
• Unified style of code and tests
• Tests can be executed in any environment without complicated preparations
• Test automation framework and test scripts also should be tested
  — Unit tests for the framework
  — Test scope with the smoke tests
• Clean and useful logs
• Use profiler to find and optimize bottlenecks
• Rerun failed tests

Test scripts

• Test scripts it’s a common property — they should be clear for everyone, and everyone can maintain it.
• Self documented tests — test documentation should be generated
• Have a set of stable tests

Metrics

• Use metrics to improve testing process
  — How stable are the tests?
  — How fast are the tests?
  — How fast are we writing tests?
  — What are the most common problems?
• Visualize metrics

Feedback

• Generate coverage report
• Integrate tests into CI/CD
• Test results should be visible for everyone
• Cause of test failure should be clear for everyone
• Collect feedback to improve test framework
  — End users of a tests framework not only QA team, but all project members

Flexible test control

• Use tags to include and exclude tests from a test execution
  — Positive / Negative
  — Stable / Unstable / Smoke

Health check for the automation test

• Is it still valid?
• Is it hard to maintain?
• How much fail causes this test has?
• What will be if we don’t catch this bug

Increase trust to test automation

• Run only stable tests
• Every test should have only one reason for failure
• Understand the cause of each and every test failure
• Weed out flaky tests like weeds

Reduce test execution time

• Parallelism
• Cache code dependencies
• Reduce the number of tests
• Set a time limit (what I can test in 10/30/60 minutes)

Cross-functional teams

• Know what the other teams are doing
• Other teams should known what QA is doing

Share in the comments which of the items is most useful in your project or what you would like to add to this list.