Smoke Testing

A test suite that covers the main functionality of a component or system to determine whether it works properly before planned testing begins.[Definition as per ISTQB]

Amazon CloudWatch Synthetics Canaries

You can use Amazon CloudWatch Synthetics to create canaries, which are configurable scripts that run on a schedule. Canaries offer programmatic access to a headless Google Chrome Browser via Puppeteer or Selenium Webdriver.

Now, We are going to create a canary to monitor the webpage "ReverseString Checker" which we have deployed using amplify using Selenium python test scripts to monitor our web page

We are going to do a kind of smoke testing to verify the following

• Whether the webpage has loaded successfully
• verify the title of the webpage
• Verify if the string is reversed successfully and printed on the webpage once the user enters the input to the text field and clicks the submit button

Ingredients

• The single page web application ReverseString Checker
  Recipe: Cook a recipe with AWS: A simple Web application which calls a webservice and displays result, using AWS Amplify

• Selenium Libraries

• Python

Prerequisites

• Log into AWS account with a valid user credentials (Root user not preferred)

Lets Jump to the Recipe

Create CloudWatch Canaries using Selenium Python test scripts for monitoring the health of the Website

• Open the CloudWatch from aws console

• Click on Synthetics Canaries link which is displayed below Application Monitoring

• Click Create Canary button

• You can use a blueprint provided by CloudWatch or use your own script by uploading it or importing it from Amazon S3.

• By default, Use a blueprint option will be selected. We are going to proceed with this

• On Blueprints section, Heartbeat monitoring option will be selected by default. Leave it as it is

• Now, Scroll down and specify a Name to the canary. I am naming it as "reversestringwebtest"

• In Application or endpoint URL field, Copy and paste the application URL (which you will find the amplify console)

• Tick the Take screenshots check box so that Screenshots will be visible on the canary detail screen for each canary run

• Now Scroll down, if necessary and you will see a Script editor section.

A Synthetics runtime version is a combination of the Synthetics code that handles calling your script handler and the Lambda layers of bundled dependencies.

Selenium is an open-source browser automation tool.

Major dependencies in syn-python-selenium-1.3

• Python 3.8
• Selenium 3.141.0
• Chromium version 92.0.4512.0

New features in syn-python-selenium-1.3

• More precise timestamps— The start time and stop time of canary runs are now precise to the millisecond.

• Choose Runtime version as syn-python-selenium-1.3
• Once you make the selection, You can see the following script getting generated on script editor

Note: Make sure that the url is still present in the Application or endpoint URL field, if not, add the url once more.

• Once you have the url on the field, you can see the same url getting updated and stored on url variable of script editor

Now, let me explain the auto populated script which you can find below,

• The chrome browser will hit the url and a screenshot of the loaded page will be saved

• The response code is also validated to make sure that it falls between 200 and 299 to make sure that a success response is generated

• Now we are going to edit the above code so as to add few more smoke tests. The final code below

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import selenium.common.exceptions

#  Add the required imports
from aws_synthetics.selenium import (
    synthetics_webdriver as webdriver,
)

# To access the logging capabability
from aws_synthetics.common import (
    synthetics_logger as logger,
)

# Configuration parameters
from aws_synthetics.common import synthetics_configuration  # Configuration parameters


def main():

    url = "<URL OF THE APPLICATION PASTED FROM AMPLIFY CONSOLE>"    
    # Set screenshot option
    takeScreenshot = True
    # Set the browser as chrome
    browser = webdriver.Chrome()
    # Set the implicit wait to 10 Seconds
    browser.implicitly_wait(10)
    # Hit the url
    browser.get(url)
    # Get the title of the launched webpage and store it on pagetitle variable
    pagetitle = browser.title
    # Verifying if page title is "ReverseString Checker"
    assert pagetitle == "ReverseString Checker"
    # logging the info - Canary title assertion passed
    logger.info("Canary title assertion passed.")
    # locating the input field by using name locator
    input = browser.find_element_by_name("first_name")
    # locating the submit button using tagname
    submit = browser.find_element(By.TAG_NAME, "button")
    # Entering input string
    input.send_keys("Cloud Tuner")
    # Clicking on submit button
    submit.click()
    # logging the info - Submit button clicked
    logger.info("Submit button clicked")
    # locating the result element using id locator
    result = browser.find_element(By.ID, "demo")
    # Fetching the text from result element and storing it in text variable
    actualText = result.text
    # logging the content stored in actualText  variable
    logger.info(actualText)
    # Storing the expected text in expectedText variable
    expectedText = "renuT duolC"
    logger.info(expectedText)
    # Verifying if content stored in expectedText variable is present in actualText variable content
    assert expectedText in actualText
    # logging the info - Reverse String Assertion Passed
    logger.info("Reverse String Assertion Passed")

    if takeScreenshot:
        browser.save_screenshot("loaded.png")

    response_code = webdriver.get_http_response(url)
    if not response_code or response_code < 200 or response_code > 299:
        raise Exception("Failed to load page!")
    logger.info("Canary successfully executed.")


def handler(event, context):
    # user defined log statements using synthetics_logger
    logger.info("Selenium Python heartbeat canary.")
    return main()

Now, let’s walk through the above code

• First import the WebDriver and Wait classes from Selenium.
• Next, the instance of Chrome WebDriver is created.
• Now, we are setting the implicit wait for 10 seconds

Implicit wait in selenium
When searching for any element (or elements) that aren't immediately available, WebDriver is instructed to poll the DOM for a predetermined amount of time by an implicit wait.
The default setting is 0 (zero). Once set, the implicit wait is set for the life of the WebDriver object.

• The driver.get method will navigate to a page given by the URL. WebDriver will wait until the page has fully loaded (that is, the “onload” event has fired) before returning control to your test or script.
• We are storing the title of the webpage in variable page title
• The next line is an assertion to confirm that title of the webpage is “ReverseString Checker”

• We are logging the message

• Now, we are going to verify whether the input string is getting reversed and displayed on the webpage once submit button is clicked

• In order to locate the web elements like input text field, submit button, Locators are used

Locators in Selenium
There are several methods for locating elements on a page. You can select the most appropriate one for your scenario. To locate elements on a page, Selenium provides the following methods

• find_element
• find_elements (To find multiple elements (these methods will return a list) The 'By' class specifies which attribute will be used to locate elements on a page. The attributes are used to locate elements on a page in the following ways:

find_element(By.ID, "id")
find_element(By.NAME, "name")
find_element(By.XPATH, "xpath")
find_element(By.LINK_TEXT, "link text")
find_element(By.PARTIAL_LINK_TEXT, "partial link text")
find_element(By.TAG_NAME, "tag name")
find_element(By.CLASS_NAME, "class name")
find_element(By.CSS_SELECTOR, "css selector")

If you want to locate several elements with the same attribute replace find_element with find_elements

Let me quickly dive through web-element location process

I have copied the snippet of index.html (Copied from: Cook a recipe with AWS: A simple Web application which calls a webservice and displays result, using AWS Amplify)

<!--- HTML form data --->
<html>
<head> 
<title> ReverseString Checker</title>
<!--- Linking the css file  --->
<link rel="stylesheet" href="main.css">

</head>

<body>
<form id='form'>
<label for="Input">Enter the String to be reversed</label>
    <input name="first_name" type="text">
    <button type="submit">Submit</button>
</form> 
<p id= "demo"></p>

<!--- linking the js file  --->
<script src="main.js"></script>

</body>

</html>

Locating Element by Name
I have used the Name locator for locating the input text field. As you see here, the first element with a matching name attribute will be returned. If no element has a matching name attribute, a NoSuchElementException will be raised.

input = browser.find_element_by_name("first_name")

Locating Elements by Tag Name

I have used the tag locator for locating the submit button. With this locator, the first element with the given tag name will be returned. If no element has a matching tag name, a NoSuchElementException will be raised.

submit= browser.find_element(By.TAG_NAME, 'button')

Locating Element by ID
Now, I have used ID locator for locating the <p id= "demo"></p> element where the reversed string is getting generated

When you have the id attribute for an element, Go with ID locator. With this strategy, the first element with a matching id attribute will be returned. If no element has a matching id attribute, a NoSuchElementException will be raised.

result=browser.find_element(By.ID,'demo')

Actions on Web elements

• After locating the web elements and storing the web elements in appropriate variables, Actions such as sendKeys() , click() are performed on the web elements

input.send_keys("Cloud Tuner")
sendkeys() is a method in Selenium that allows to enter content automatically into an editable field.
Here we are entering the Text CloudTuner to the Text field

submit.click()
The click() command in Selenium simulates a mouse click operation for any interactive UI element (links, buttons, checkboxes, radio buttons).
Here, We are clicking the submit button

actualText = result.text
Now one can get text of the result element with result.text and store the content in text variable

Now, the code assert expectedText in actualText verifies that if the content stored in expectedText variable is present in actualText variable

Just keep in mind that the text saved in the expectedText variable is the reverse of the string we entered for Cloud Tuner, which is "renuT duolC."

The content stored in actual result is the text obtained from webpage after clicking submit button which is "Reversed String is: renuT duolC"

The assert statement verifies that if renuT duolC is present in Reversed String is: renuT duolC", which is true and hence the test script passes

• Now, You have the script on Script Editor, Lets proceed with remaining configurations for creating the canary

• You can edit this canary and change run schedule at any time on the schedule section. By default, Run continuously option would be selected

• Run the canary every 5 minutes is auto selected (You can make the frequency changes as required). Now I am leaving it to default

• The canary data retention period can also be configured on Data retention section. For now, I am leaving the default values

• Canary run data is stored in an Amazon S3 storage resource bucket. A default S3 bucket will be used or created, You can also select an existing S3 bucket from your AWS account. Now, I am leaving it to default value

• We need your permission to put artifacts into S3, and to store logs and publish metrics to Cloudwatch. An IAM role with those permissions will be created by default or You can choose an existing role. Now I am proceeding with the default selection

• There are certain optional configurations available such as

  • CloudWatch alarms : You can let Synthetics create alarms for your canary automatically, and customize these later. (No alarms will be configured by default)
  • VPC settings : Use this if your endpoints are under your network (No vpc will be selected by default)
  • Add tags: to canaries to help set permissions, organize, and search for them later (blueprint tag with value _API canary _ will be added by default as per our selections and you can also additional tags as required)
  • Active tracing: For active tracing with AWS X-Ray to help troubleshoot and reduce the mean time to resolution.

AWS X-Ray and Synthetics help you analyze and debug to find the root cause of ongoing failures, identify performance bottlenecks and trends, compare latency rates, and identify if you have enough canary coverage for your APIs and URLs.

Trace my service with AWS X-Ray. checkbox will be enabled by default which will allow you to

• View canaries in AWS X-Ray and CloudWatch ServiceLens service maps.
• View traces and segments for each canary run.
• View trends using AWS X-Ray analytics.

• Click Create canary button

You can see a message as below. Please wait for a couple of minutes until the canary is created for you

• You will be navigated to Canaries page once the canary is successfully created

• Wait for a couple of minutes and You can see the Heartbeat canary which we configured getting executed and test results will be displayed

• As per run schedule configuration, The canary will be running every 5 minutes and thus Web page will be monitored

View the Canary and the results

• Click on the Canary link to view the canary

• You will be navigated to Canary page, Now scroll down to see the screenshots on Availability Tab

In the Availability tab, check the SuccessPercent metric to see whether the problem is constant or intermittent.

While still in the Availability tab, choose a failed data point to see screenshots, logs, and step reports (if available) for that failed run.

• For each canary run you can see the steps, screenshots, logs, and HAR file.

• Screenshots let you see visually what the canary received for each request

• Remember, We have the code to take the screenshot on the script editor. Please find the code snippet below

if takeScreenshot:
        browser.save_screenshot("loaded.png")

    response_code = webdriver.get_http_response(url)
    if not response_code or response_code < 200 or response_code > 299:
        raise Exception("Failed to load page!")
    logger.info("Canary successfully executed.")

• We can also view the logs that are logged using the code such as logger.info("submit button clicked") by navigating to logs tab

• The HAR file shows the HTTP requests which were made, along with their size and duration.

Okay, Now we have successfully created a canary and viewed the results

Note:

• Delete the resources after use
• Canary should be stopped to enable Delete option