Skip to content

Calliope Gitlab Integration

Our Gitlab platform provides you with an easy way to manage your test scripts and execution. To make use of this feature, you first have to register on our environment. After registration, you have to setup your testsuite as a Gitlab project. We suggest starting out with forking our Cucumber boilerplate by pressing the fork button right under the repository name. Once you've done that you can then download the repository using the git clone command in combination with the URL that you can find on your newly forked repository page. The resulting command should look similar to this, but using your gitlab account name instead of.

$ git clone<GITLAB_ACCOUNT_NAME>/ecommerce-cucumber-web-test-automation-suite.git

If you do not have Git installed you can read how to install Git for your respective operating system over here. We also recommend taking a look at this Git tutorial if you are unfamiliar with Git and wish to learn more about it before using it.

When we open the project that we just cloned there are two files that we're interested in for setting up our testsuite so it can export the results to Calliope.


The '' file is where we tell gitlab what command to run when a job gets triggered. When you're using our boilerplate you should see something like this:

  SELENIUM_URL: "http://selenium__standalone-chrome:4444/wd/hub/"

    - bundle exec cucumber -f json -o ./results/latest_results.json && true
    - curl -X POST -H "x-api-key:$API_KEY" -H "Content-Type:application/*" --data "@results/latest_results.json"$PROFILE_ID/testreport?os=myos&platform=myplatform&build=mybuild
  when: manual

The following variables are defined:

  • SELENIUM_URL: Is only required for this specific test suite when its run from gitlab and using the selenium/standalone-chrome service
  • API_KEY: Required to import tests to Can be found on your account page
  • PROFILE_ID: Tests will be imported in one of your profiles on you can find the ID in the curl command (see below).

The 'script:' part is the command that is going to run when the job gets triggered.

  • bundle exec cucumber -f json -o ./results/latest_results.json && true runs all the cucumber tests and stores the results in './results/latest_results.json'. && true makes sure the next commands are also executed.
  • curl -X POST ... will import the tests into the platform.

Make sure the file @results/latest_results.json points to the results file you generated.

You can find and copy your specific API import curl command on your project overview page that contains the profile.

Calliope api import

After you've updated the .gitlab-ci.yml file and committed the changes to your repository, your test suite is ready to import results to Calliope, and you can configure calliope to use the gitlab jobs you've created.

Configuring gitlab in

We're ready to configure the profile to use gitlab! Make sure you have set your account gitlab settings on your account page.

Now you can select your gitlab test suite, when we go to our profile that we set our test suite to export to and click 'edit profile' we can select a 'repository' and a 'job' from a list.

Calliope gitlab settings

Select the repository you created on our Gitlab environment and select the job you created in the '' file ('curl-upload-example' if you used our boilerplate). Finally we can save the profile by clicking 'save'.

When we return to our dashboard we can see that something has changed, the disabled play button is now active and we can click it to start a testrun. Also our profile's page has a two new tabs, 'log' and 'schedules'. In log we can see the logs of our current or latest test run and in 'schedules' we can let our test suite run according to a schedule that we set.