Why you should care about Robot Framework – Part II

Have you read the first part of this series yet? If not, you should take a moment and do that now. You can find it here: Why you should care about Robot Framework – Part I. If you are up to date, just continue reading.

The examples used in this article are from the official Robot Framework web demo project, but DON’T click that just yet. Read on and I will show you what test cases and resource files are and how they are executed with the Robot Framework executor. And more importantly, see just how the test report quickly tell you if your test suite passed or failed, and the log file reveals details about every keyword executed.

Let us start with…

Test cases

Test cases in Robot Framework are written using keywords. To gain the full benefit of using keywords, they should be written in a domain specific vocabulary. They will then become your own DSL (domain specific language).

Let us start by looking at the “Valid Login” test case from the web demo:

*** Test Cases ***
Valid Login
    Open Browser To Login Page
    Input Username    demo
    Input Password    mode
    Submit Credentials
    Welcome Page Should Be Open
    [Teardown]    Close Browser

Source: https://github.com/robotframework/WebDemo/blob/master/login_tests/valid_login.robot

This test case displays the clean and simple syntax that Robot Framework uses.
The intention of the test is very easy to understand, which is the trademark of good test case design.

Test case files, keyword files and other files like resource files are saved with the .robot extension. They are however plain text files. This means that you can edit them in any text editor of choice.

There are of course advandtages of using and IDE, which you will see in the Using an IDE section.

Resource files

Each keyword has an implementation that handles the actual interactions with the system under test. The implementation keywords are imported in the Settings sections of the file, with the Resource keyword:

*** Settings ***
Documentation     A test suite with a single test for valid login.
...
...               This test has a workflow that is created using keywords in
...               the imported resource file.
Resource          resource.robot

Source: https://github.com/robotframework/WebDemo/blob/master/login_tests/valid_login.robot

The contents of the resource file resource.robot shown below, contains all the keyword implementations and in this case, uses an external library to handle user interactions with the system under test.

*** Settings ***
Documentation     A resource file with reusable keywords and variables.
...
...               The system specific keywords created here form our own
...               domain specific language. They utilize keywords provided
...               by the imported SeleniumLibrary.
Library           SeleniumLibrary

*** Variables ***
${SERVER}         localhost:7272
${BROWSER}        Firefox
${DELAY}          0
${VALID USER}     demo
${VALID PASSWORD}    mode
${LOGIN URL}      http://${SERVER}/
${WELCOME URL}    http://${SERVER}/welcome.html
${ERROR URL}      http://${SERVER}/error.html

*** Keywords ***
Open Browser To Login Page
    Open Browser    ${LOGIN URL}    ${BROWSER}
    Maximize Browser Window
    Set Selenium Speed    ${DELAY}
    Login Page Should Be Open

Login Page Should Be Open
    Title Should Be    Login Page

Go To Login Page
    Go To    ${LOGIN URL}
    Login Page Should Be Open

Input Username
    [Arguments]    ${username}
    Input Text    username_field    ${username}

Input Password
    [Arguments]    ${password}
    Input Text    password_field    ${password}

Submit Credentials
    Click Button    login_button

Welcome Page Should Be Open
    Location Should Be    ${WELCOME URL}
    Title Should Be    Welcome Page

Source: https://github.com/robotframework/WebDemo/blob/master/login_tests/resource.robot

As you can see in the Settings section, this keyword file imports the SeleniumLibrary, since the demo is a web application. You are of course not limited to using Selenium for testing web applications, but it is still the most commonly used library for this purpose. In fact, you should check out the new Robot Framework Browser, which is using Playwright instead of Selenium.

Also pay attention to the Variables section, where you can declare your own variables for use.

From this file, let’s look at the keyword Open Browser To Login Page:

*** Keyword ***
Open Browser To Login Page
    Open Browser    ${LOGIN URL}    ${BROWSER}
    Maximize Browser Window
    Set Selenium Speed    ${DELAY}
    Login Page Should Be Open

This consists of four keywords. The first Open Browser ${LOGIN URL} ${BROWSER} is a keyword from the Selenium library. The Open Browser keyword opens the specified browser with the url passed in.

The following two keywords Maximize Browser Window and Set Selenium Speed ${DELAY} are also from the Selenium library. Maximize Browser Window does exactly as promised, while Set Selenium Speed is mainly a keyword used for debugging, since it adds a pause after each Selenium command is executed, allowing you to see what is being executed.

The last keyword Login Page Should Be Open is defined in the same file, and uses Selenium library to check that the title of the page is as expected with the Title Should be keyword.

So keywords can, and often do, call other keywords. This helps promote reuse instead of duplication. Pretty straightforward, but very powerful.

Test execution

Test cases in Robot Framework are executed easily using the command line. This of course requires you to have installed the necessary libraries and drivers. The Robot Framework installation guide will help you get going.

If you want to execute the test case above, you would simply run this command from the root directory:

robot login_tests/valid_login.robot

In this example, when running the web demo server, you will successfully login to a simple page and receive confirmation. The output looks like this:

But, as introduced in the reporting section of Why you should care about Robot Framework – Part I you also get three test report files as output. The are by default placed in directory from where you started the execution.

Report

The report is the overview, where you quickly see the status, the number of tests run and the test run time. You also have access to view tests by tags, suite or do a search for a specific test. In Why you should care about Robot Framework – Part I you saw an example of a report of a successful test run. Here you now see what happens, when a test fails.

Log

When all the tests in your test suite passes, you are typically either satisfied or surprised. If you are surprised, the first thing you’ll want to do, is find out what happend. For this task, the Robot Framework log is the best tool. This will show you in detail, the execution of each keyword in your test case and help you understand the unexpected behaviour in the execution of your test case.

It shows you detailed information about each test and each keyword executed. You can see the value of arguments passed into a keyword, and error output when something failed.

In this example, you can see that it fails verifying that the welcome page should be open. Instead of showing the welcome.html page, you see the error.html page. The cause of the failure in this case, is an invalid password mode_INVALID. When reverted back to mode, the login will be successful again.

For more detailed information, you have the option of adjusting log levels to more verbose output.

Using an IDE

As you have seen in the examples above, Robot Framework saves test cases in plain text files. This makes them easy to edit in your favorite text editor, whether that is Notepad++, Sublime Text, BBEdit or even… Vim.

They will all work out-of-the-box.

But, honestly we are used to much more than just basic editing capabilities in our tool.

Fortunately, you can find plugins, extensions or whatever they might be named, that bring syntax highlighting and “code completion” to your Robot Framework files, for a growing number of tools.

Or you can jump into a full fledged IDE. And you should start by checking out RIDE, the Robot Framework IDE. Go ahead and take the development version, which will become version 2.0 at some point in the near future.

Plugins for Robot Framework likely exist for your favorite IDE. So here are 3 examples of some widely used IDE’s and plugins:

Conclusion

Test cases in Robot Framework are created from high level keywords, written in a domain specific language. Keywords can also be more technical keywords from other libraries both internal and external. Keywords are implemented in resource files, so test case files are kept clean from the implementation details.

Executing test cases is done from the command line using the robot command, and the result is presented in an HTML report for easily getting the overview of the test run. A detailed log is produced, which shows details about each keyword executed, for quick analysis of possible test failures.

While both test case files and keyword files are plain text files and can be edited in your favorite text editor, there are several advantages of using on of the many IDE’s that with plugins support Robot Framework and adds helpful features for editing and running tests.

So now, go check out Robot Framework at https://robotframework.org/ and try it out for yourself. Start with running the web demo, and see all the things we talked about in this article in action.

---