Testing with UP9

Overview

When UP9 is installed in a Kubernetes, Docker, or ECS cluster, UP9 observes the service traffic and begins building tests from service traffic. Run these tests from your terminal or in the cloud, and schedule reoccuring tests, export version-controlled test artifacts, and compare results from old test versions.

Tests

Terminal Testing

Use this command to test from the Terminal window:

Copy to clipboard
up9 test:run-local my.new.workspace all

The 'run-local' option will cause the test to run from where you installed the UP9 CLI. If the UP9 CLI is installed on your laptop, it will run from your laptop.

Once the test run is complete, test results are available in the Workspace Summary tab in the UP9 browser.

Cloud Testing

UP9 can also test with a cloud runner. This method only works when the web app being tested is publicly available. If the webapp is local, please use the local method detailed above.

On UP9's Tests page, select a test collection and choose Run Now.

Test_Tools

When the test is finished, UP9 will generate a report.

Reports

Your test reports reliability and coverage of services, and it can go even deeper than that.

The Reliability Score is a success ratio of the recent test Service Endpoint calls. The Reliability Score histogram is available for each test execution, so you can view and compare old test results with new test results. If you see a low reliability score, it means that UP9 wasn't able to learn your architecture from the first testing pass, and needs more information.

The Coverage Score represents the ratio of active to inactive test-spans.

Test_Reliability

You can also view the actual response from the service for troubleshooting.

Test_Card

Scheduling Tests, Exporting Artifacts, and Run in CLI

You probably noticed some other available options when we ran a test in UP9 above. These are additional test tools that deserve a mention!

  • Schedule presents a test scheduler for this test collection.
  • Export downloads a ZIP archive of all tests, mocks, and contracts in this test collection, which you can run in other languages and frameworks. For more information on this archive, see Exporting Artifacts.
  • Run with CLI generates copyable test code.

CLI

Test Artifacts

After exporting the tests, mocks, and contracts archive, you can run tests with the language and framework of your choice. These instructions are also available within their respective folders.

Postman

Install Postman on your machine: https://www.postman.com/downloads/

Running the tests:

Open Postman, click "Import" and select UP9.postman_collection.json from your file system.

Differences from the full Postman tests:

  • No Data Driven Tests
  • No chained requests
  • No assertions / test scripts
  • It's a plain Postman collection

Pytest

Install dependencies with: pip install requests ddt grpc

Running the tests:

pytest tests.py

Running a single test:

pytest tests.py::TestClassName::test_method_name

Jest

Install the dependencies with: npm install

Running the tests:

npm test

Running a single test:

node_modules/.bin/jest -i tests_script_name.test.js -t "test_method_name"

Junit

Install Apache Maven on your machine: https://maven.apache.org/install.html

Running the tests:

mvn clean test -DtrimStackTrace=false

Running a single test:

mvn clean test -DtrimStackTrace=false -Dtest=TestsClassNameTest#testMethodName

Generating the Surefire HTML report:

mvn surefire-report:report-only

Testify

There is a go.mod file in the root directory that will automatically install the dependencies when you run the tests. To update go.mod:

go get -v -t ./...

Running the tests:

go test -v

Running a single test:

go test module_name_test.go -v -run TestFunctionName

to have JUnit.xml output:

gotestsum --junitfile unit-tests.xml -- carts_test.go -run TestGetCartsIdItems13

Support

For support, feel free to use any one of the three: