Engineering practices for test automation process

Photo by Josue Isai Ramos Figueroa on Unsplash

List of an ideas concerning test automation. Based on my and other people’s experiences.

Engineering practices for test automation

  • Use tools to speed up testing
  • Always pay technical dent
    Self-service infrastructure — everything must start (tests, or environments) with one button
  • Self documented tests — everything has to be in the code, documentation becomes obsolete
  • Postmortem should be conducted and documented after each incident

Testing automation framework

  • Use VCS to store a code, use GitFlow to provide changes safely
  • Automatic rollout of a new version
  • Unified style of code and tests
  • Tests can be executed in any environment without complicated preparations
  • Test automation framework and test scripts also should be tested
    — Unit tests for the framework
    — Test scope with the smoke tests
  • Clean and useful logs
  • Use profiler to find and optimize bottlenecks
  • Rerun failed tests

Test scripts

  • Test scripts it’s a common property — they should be clear for everyone, and everyone can maintain it.
  • Self documented tests — test documentation should be generated
  • Have a set of stable tests

Metrics

  • Use metrics to improve testing process
    — How stable are the tests?
    — How fast are the tests?
    — How fast are we writing tests?
    — What are the most common problems?
  • Visualize metrics

Feedback

  • Generate coverage report
  • Integrate tests into CI/CD
  • Test results should be visible for everyone
  • Cause of test failure should be clear for everyone
  • Collect feedback to improve test framework
    — End users of a tests framework not only QA team, but all project members

Flexible test control

  • Use tags to include and exclude tests from a test execution
    — Positive / Negative
    — Stable / Unstable / Smoke

Health check for the automation test

  • Is it still valid?
  • Is it hard to maintain?
  • How much fail causes this test has?
  • What will be if we don’t catch this bug

Increase trust to test automation

  • Run only stable tests
  • Every test should have only one reason for failure
  • Understand the cause of each and every test failure
  • Weed out flaky tests like weeds

Reduce test execution time

  • Parallelism
  • Cache code dependencies
  • Reduce the number of tests
  • Set a time limit (what I can test in 10/30/60 minutes)

Cross-functional teams

  • Know what the other teams are doing
  • Other teams should known what QA is doing

Share in the comments which of the items is most useful in your project or what you would like to add to this list.

--

--

--

Test automation engineer, writer

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Flutter Animation in ListView

Editing text without arrow keys on MacOS — Salman AlSaigal

DKP Open Beta Reward Update

Does Your Database/ORM Make You Do Stupid Things Too?

Take Self-Signed Certificates to the next level

Kubernetes and its architecture (part 1)

Modify Title and Header Django Admin Interface

Creating a basic Wox launcher plugin

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Artur Mishustin

Artur Mishustin

Test automation engineer, writer

More from Medium

How to refactor legacy test automation project, save code and your time

Why is software testing so hard?

Stages in the Software Testing Process

Test Story