Ex Machina

Singularity—is a hypothetical point in time at which technological growth becomes uncontrollable and irreversible, resulting in unforeseeable changes to human civilization

Wikipedia.

The Unexpected Drawback of Shifting Testing to the Left

Back in 2015, we helped coin the term Continuous TestingBuilt on the premise of having developers testing software often, and in every phase of the software life cycle. Continuous Testing entails the automation of the following testing activities: Unit tests, Pre-integration, component level tests that run following every code commit, Integration tests, Acceptance tests, Post production tests, Type of tests can include, depending on availability and phase the following: Functional, Regression, Performance, Load, Stress, Soak and Security. Continuous Testing is often part of Continuous Integration/Delivery. It is used to identify failures early in the life cycle, allowing engineering teams time to prevent them from reaching production.. Continuous Testing is synonymous with the term ‘testing shift-left’. My company back then, BlazeMeter, was among the first to shift testing to developers.

Shifting testing to the left is a great idea, but it can also adversely impact engineering productivity. Testing is a time-consuming task that requires specific skills. Developers don’t always have the time or required skills to make excellent testers, and they have many other things to do with their time - like building kick-ass applications.

If not done right, when you shift testing to the left, you may end up with one of two results:

  1. More work for already very busy developers, which leads to subpar results in terms of testing
  2. Developers who neglect testing altogether because of the poor ROI

If done the right way, engineering teams can maintain excellent test coverage without impacting engineering productivity or developer velocity.

Characteristics of a Modern Quality Organization

Quality was always important. However, with the onset of new technologies and the increasing pace of software development, testing modern systems is becoming more complex and time-consuming. 

The Goals of a Modern QA Organization

When organizations consider their quality strategies, the following goals come to mind:

Accountability for Quality

Developers need to be accountable for the quality of their code, and must avoid simply handing off accountability for quality to someone else (e.g. QA).

Engineering Productivity

Engineering productivity is very high on the corporate priority list. Therefore, any solution must not hinder developer velocity.

Automation

Any investment in testing needs to be automated, as opposed to any one-off activity or manual task. Otherwise, such testing can’t scale.

Strategy (SDET + Automation)

Following a similar trend over the last several decades, many organizations now hire a group of Software Development Engineers in Tests (SDETs) and make them a part of the engineering team.

An SDET has both testing and programming skills, and can apply them to real-world situations. Their work has an emphasis on automation tools for functional, API, and performance testing.

SDET also known as Software Development Engineer in Test, is a job role within the Software Testing and Quality Assurance Domain. The term was originally used by Microsoft and then Google with a view of replacing mundane and repetitive manual testing tasks with automation.

devqa.io .

The SDET can become part of the engineering team and focus on various testing-related activities like test creation, updates and automation. This way, engineering teams are still accountable for the quality of their code. Developers can focus on coding, and SDETs on the testing activities.

SDET Job Requirements

Here are a few of the SDET job requirements, their day-to-day operations, and how UP9 can help offload the heavy lifting:

  • 2+ years of work experience in software testing/development or a related field
  • Experience with a coding language
  • Capacity to write, maintain, and execute test cases
  • Assurance of the proper documentation for software defects through a comprehensive defect tracking system

Cost

The average base pay of such an engineer is around $83K/yr1. The salary of an SDET Mind you, Google pays an average of $150K/yr per SDET, and you must admit, they do produce high-quality products.

Scale

Even when we move the workload from QA testers to SDETs, we can still expect a ratio of 1:4 (e.g. hiring one SDET for every 3-4 developers)2. As mentioned, scaling the SDET group can be as challenging as scaling any engineering group. While the addition of SDETs to your team won’t make hiring easier, it will make your developers more productive and save them precious time.

Challenges of Hiring SDETs

While hiring SDETs can be somewhat easier than hiring developers, hiring and scaling the group of SDETs all together can still be challenging.

Hiring SDET Ex Machina

UP9 can help the SDET group scale by automating most of the time-consuming activities an SDET would undertake, thereby allowing the organization to do with a far lower SDET headcount.

Codified API Test-cases Creation and Update

UP9 automatically generates and continuously updates codified API test-cases and commits them to a GIT repository on every change. Developers can choose the coding language and the framework through which the test code is generated. UP9 currently supports:

Years of Experience as Developers and Testers

The code generated by UP9 is based on best practices from decades of experience in the fields of testing by people like myself and our Chief Scientist, Andrey Pokhilko.

Defect Documentation

When UP9 detects a regression, it automatically provides a root cause analysis to the regression, coupled with an abundance of debug and log information. As such, it will always be clear why the test failed and what contributed to that failure. Root-cause-analysis

Scale

UP9 uses scalable Contract Testing and can instantly support any number of services with the same level of effort. The contract testing pyramid At the service level, Contract Testing includes the service contract, the contract test-suite and the service mock. As long as providers and consumers adhere to the service contract, there will be fewer regressions. This concept is infinitely scalable, and works whether you have five, 500 or 5,000 services.

The entire operation is fully automated. UP9 includes:

  • Automatic contract discovery for all services via API traffic observation
  • Automatic generation of and continuous updates to codified API tests and developer-friendly mocks for all services

10x Efficiency

UP9 helps transform QA organizations into development teams focused on TEST. If you had a team of developers writing code, they would run in pairs where one developer writes the code and the other reviews the code. From my experience, it takes about 10% of the time to review the code than it does to write. With UP9, developers would only need to review the code generated by UP9, thus saving a significant portion of time spent on writing the code.

Conclusion

The pace of software development is increasing exponentially. Developer time is becoming more and more precious. For the past several years, testing has been the number one reason releases are delayed, as tests consume around 25% of a developer’s time. Developers need to be accountable for the code they generate. Hiring a group of SDETs can help developers focus on coding while the SDETs focus on the time consuming tasks of test creation, update and automation. Hiring and scaling a group of SDETs, however, is challenging on its own. UP9 greatly eases this challenge, offloading most of an SDET’s heavy lifting. It does so by automatically generating and continuously updating codified API test-cases, which are committed to Git on every change with an MIT license.

ABOUT UP9

UP9 is a microservice testing platform for Cloud Native systems that can scale across any number of services. UP9 helps developers prevent software regressions and increase engineering productivity through effortless testing and instant virtualization.

  • Understand how services interact with each other with automatic contract discovery.
  • Ensure system-wide contract adherence with tests that write themselves.
  • Start testing early with test environments as code, automatically mocking all service dependencies.

UP9 offloads the microservice testing workload from developers, giving them precious time back.

You can sign up now for free!


  1. SDET salary https://www.glassdoor.com/Salaries/software-development-engineer-in-test-sdet-salary-SRCH_KO0,42.htm

  2. Ratio of 1:4 is taken from the Workld Quality Report 2019-2020