f 612-605-1978

Quality Assurance

Quality Assurance at Kettle River

Many developers have a dislike for Quality Assurance.  They think it is a pain in the neck that doesn’t add value to the software development process.  In our opinion, nothing could be further from the truth.  Quality Assurance is one piece of the software quality process.  In order to create “boring” software, you have to have a good QA techniques in your toolbox.  Tools in the QA toolbox include continuous integration, automated unit testing, automated integration testing, code coverage analysis, and automated UI testing.

What is automated UI testing?

Automated UI testing is tool in the Quality Assurance toolbox.  It offers another way to ensure that your applications are robust and free of nasty surprises . Automated UI tests are just like the ones the manual tester would run against the website.  However, automated UI tests can run in a fraction of the time it takes a person to run the tests manually.  They can be run repeatedly without changing how the test is run.  Also, automated tests for Web applications can be run using different web browsers to ensure that functional elements of the UI work properly across all browsers.

One thing that automated UI testing won’t do is verify that the UI appears correctly.  Different browsers interpret the HTML and CSS standards differently.  We have yet to find a tool that can verify that something “looks” correct.

Why Automated UI testing?

Automated UI Testing allows you to quickly and efficiently test sections of your site that you are not currently developing.  If sections of your site are linked or dependent on other sections of the website for information, automated UI testing can make sure that the sections are still working together.

Automated UI testing can save you time.  It takes time for someone to manually type and click all the buttons. It takes time for someone to run different scenarios.  With the Automated UI testing you control what is tested.  An example of one test is it will take 30 seconds – 1 minute to run something that takes a person 5 minutes to enter.

  • Automated UI testing allows you too quickly and efficiently test sections of your site that you are not currently developing to make sure any new issues were introduced.
  • Automated UI testing runs faster than a manual test.
  • Automated UI tests can run tests for previously fixed issue to make sure they do not show up again.
  • Automated UI testing can check to make sure interconnected parts of the website are still working together.
  • Automated UI testing can help warn you of problems before a new release.
  • Automated UI tests can be designed to mimic a user in the system.

What is Selenium?

Selenium is our preferred tool for automated UI testing.  Selenium is a suite of open-source products that allow you to generate and create automated tests. These tests can be  run individually or as a group against your UI.  Selenium tests can be saved as Java jUnit tests or as XML files for later playback.  Elements of Selenium are the IDE and the WebDriver.

Selenium IDE:

  • Runs with Mozilla Firefox using JavaScript (other languages are available).
  • It allows you to record what the manual tester is doing on the UI.
  • User can add asserts to the script from the UI to tell the test where and what to check.
  • User can add code or adjust the information in the test as it is recording.
  • User can replay the tests to ensure accurate recording.
  • Test(s) can be grouped into suites and replayed.
  • Test(s) can be exported for use in or by a different program such as jUnit.

Selenium Webdriver:

  • Runs separate browser for tests.
  • Can use jUnit4 for testing.
  • Can be brought into the project to help keep all the code together.
  • Allows for methods to be created and used by some or all of the tests.
  • Allows for debugging of errors in the program you are using such as Eclipse or IntelliJ
  • Not limited to how the tests are built like you are in IDE.
  • Easier to manipulate code across many tests.

Continuous Integration and Automated UI testing

Continuous integration and automated UI testing allows you to be able to run the automated tests at anytime day or night.  The tests run faster than a person and can be done just before a release.  If automated tests are set up within your project then anyone with access to the code can run the tests including the developers.

  • Regression testing against the UI.
  • Can be used to test ‘happy’ path or verify that all the error messages still trigger.
  • Tests for specific areas can be run separately.

Sometimes tests can take too long to run when using the full UI especially when you are just repeating things you have already tested but need the data in all the rest of the sections.  To resolve this problem, we use our knowledge of databases to set up the test data before running Selenium.  Examples of scripts that we write include.

  • SQL scripts can put in the balance of information in larger sections where you need all the data.
  • SQL scripts can help to offset system integration by providing an option to upload data from previous tests to be able to run this one.  Instead of having to run the tests all over again from the beginning to get back to that point.
  • SQL scripts can in a QA test location clean up information that will allow you to run the tests again without interference from other data.  Example: If the QA test location uses a copy of the production database you might need to remove all time entry that is currently in process to make sure your tests run.

Running automated tests using Jenkins is possible if the Selenium tests are exported as jUnit4 test cases.   The tests can be set up in Jenkins and run  the tests against the code that has not been pushed out anywhere.  Or run at a time that is convenient for your system.

Automated UI testing can be a powerful tool in the toolkit. It can quickly run through tests that were originally run against the UI and give your team a “sanity check” on the application.  However, it is not a “silver bullet” for finding all user experience or UI problems.  Also, like all tests, automated UI tests must be updated as the software it tests is changed.  This update process can affect the velocity of your team as they work through changes and enhancements.  But the cost of not testing can be significant in terms of bugs found after release, data quality issues, and other problems.  Since we believe that testing is better than not testing, the teams at Kettle River have included it in our toolbox of best practices to help keep our software boring.