A-synchronous import processing

What does it mean?

In the past we would process an uploaded file directly, while the client is waiting for the response. While Calliope Pro is growing, we see more (very) large reports to be processed and saved. This can sometimes cause a response time longer than 60 seconds.

The long response time can cause a time out of the user importing, but can also make the server respond slow for a while.

To prevent this we have introduced a-synchronous import processing, which means that after your API call, we will store all the information given with the API call and let a separate worker process and save everything.

Breaking changes

This does mean that directly after the /import and /import/{formatter} API call, we cannot return the details of the report yet.

Old response from /import:

{
  "report_id": 43,
  "report_url": "https://app.calliope.pro/reports/43",
  "status": "success",
  "report": {
    "origin": "junit_xml",
    "os": "My OS",
    "build": "#1234",
    "platform": "Windows 10",
    "status_hash": "a9fa127d63eab16a86cddd6b0b26019adc16a739",
    "cached_duration": "260.682",
    "cached_total": 8,
    "cached_passed": 7,
    "cached_failed": 1,
    "cached_broken": 0,
    "cached_skipped": 0,
    "cached_unknown": 0,
    "id": 43,
    "profile_id": 1,
    "date": "2021-06-15T11:38:17.000Z",
    "created_at": "2021-06-15T11:38:17.000Z",
    "updated_at": "2021-06-15T11:38:18.000Z",
    "history_hash": "356a192b7913b04c54574d18c28d46e6395428ab",
    "is_moved": false,
    "cached_json_asset_id": null,
    "most_recent": true,
    "children_count": 0
  }
}

New response from /import:

{
    "report_id": 41,
    "report_url": "https://app.calliope.pro/reports/41",
    "report_status_url": "https://app.calliope.pro/api/v2/reports/41/status",
    "import_status": "queued",
    "import_progress": "Import has been put in the queue.",
    "message": "Go to the report url to follow the progress and see the results of your report."
}

Workaround (DEPRECATED)

You can still receive the old response by adding the parameter process_async=false. This will process your report inline and produce the old response.

Note: If you use this, let us know why, so we can improve Calliope in a way that you don't need this anymore. If we don't know, we might make this workaround obsolete without fixing your problem.

Pro's & Con's

Con's

• You don't get your result directly in the API response.
• Might be more difficult to write your own custom code on a response.

Pro's

• The API call barely has any delay.
• We can process multiple reports at the same time.
• Faster server response times.
• We cache your report right after processing instead of when visiting the report.
• Max file/report size is significantly higher.
• When there is a high load, it will automatically build up a queue.