Remote service request and response format

This feature is only enabled on certain environments and tenants. Contact support for more information about this feature.

In order for a remote service to communicate with Spark, it needs to use a standard request and response data format. If you are using a proxy service to intermediate the XConnector, it may need to be responsible for the data format.

Remote service request format

This section describes the format of the HTTP request that is sent from the Spark XConnector to the remote service. The HTTP request from Spark takes the form of a POST request containing headers and a request body.

Request headers

The header information typically appears to the remote service as a collection of key-value pairs. The HTTP request header should incorporate the Authorization key with a Bearer token.

Request body

The request body is a JSON object, presenting a set of name/value pairs (also termed a "dictionary"). A JSON array called inputs denotes the input records to pass to the remote service.

The JSON should follow the format prescribed in inputs definitions "Synchronous request of inputs using JSON arrays". Data type serialization is described in request_data.

{
  "inputs": [
    // ... array of the request data
  ]
}

Example 1: Single value inputs

Submit details for a life insurance quotation.

age 40
sex M
smoking false

The first array in inputs defines the key names.
The second array in inputs defines the value pairs.

{
  "inputs": [
    [ "age", "sex", "smoking" ],
    [ 40, "M", false ]
  ]
}

Example 2: Table of inputs

Submit a table of vehicles for insurance quotes.

vehicles
Make
Model
Year
Dodge
Viper
2017
Ford
F-150
2005
GMC
Blazer
2026

{
   "inputs": [
    [ "vehicles" ],
    [ 
      [ [ "Make",  "Model",  "Year" ], 
        [ "Dodge", "Viper",  2017   ],
        [ "Ford",  "F-150",  2005   ],
        [ "GMC",   "Blazer", 2026   ] ]
    ]
  ]
}

Example 3: Single and table inputs

Submit a table of vehicles and state for insurance quotes.

state NY
vehicles
Make
Model
Year
Dodge
Viper
2017
Ford
F-150
2005
GMC
Blazer
2026

{
  "inputs": [
    [ "state", "vehicles" ],
    [ 
      "NY", [ [ "Make",  "Model",  "Year" ], 
              [ "Dodge", "Viper",  2017   ],
              [ "Ford",  "F-150",  2005   ],
              [ "GMC",   "Blazer", 2026   ] ]
    ]
  ]
}

Remote service response format

This section describes the format of the HTTP response that is returned from the remote service to the Spark XConnector. The HTTP response from the remote service should take the form of a POST response containing headers and a response body.

Response body

This is very similar to the format described for the Request body. The JSON should follow the format prescribed in inputs definitions "Synchronous request of inputs using JSON arrays". Data type serialization is described in request_data.

The response body from the remote service encapsulates the serialized data, adhering to the following JSON structure:

{
  "outputs": [
    // ... array of the returned results
  ],
  "warnings": [
    // ... array of warning arrays
  ],
  "errors": [
    // ... array of error arrays
  ],
  "process_time": [
    // ... array of single values of the computation time for each record
  ]
}

outputs is a JSON array, where each element is a JSON array that represents one record of the response data. Outputs should follow the same structure denoted above for the Request body.

The first record defines the string keys.
Subsequent arrays can be returned corresponding to the processing of multiple records.

{
  "outputs": [
    [
      "spider", // returns string
      "Australia", // returns string
      "specimen" // returns table
    ],
    [
      "Spiders, with eight legs, spin silk webs to catch prey",
      "Australia is a vast island continent renowned for its diverse landscapes, unique wildlife, and vibrant cities.",
      [ // table rows
        [ "name", "poisonous"],
        [ "Sydney Funnel Web Spider", true],
        [ "Redback Spider", true]
      ]
    ]
  ]
}

Warning messages can be returned from the remote service. It should be constructed with a JSON array containing a JSON array for every record of data with a warning. If a record does not have any warnings, return null.

{
  "warnings": [
    null,
    [
      {
        "source_path": "type",
        "message": "Unknown specimen type spider, skept type filter"
      }
    ],
    [
      {
        "source_path": "main_country",
        "message": "Unknown country Astralaya, skept country filter"
      }
    ]
  ]
}

Error messages can be returned from the remote service. They should be constructed with a JSON array containing a JSON array for every record of data with an error. If a record does not have any error, return null.

{
  "errors": [
    null,
    [
      {
        "error_category": "timeout",
        "error_type": "500",
        "additional_details": "500 Internal Server Error",
        "source_path": "database",
        "message": "Connection timeout"
      }
    ],
    [
      {
        "error_category": "unauthorized",
        "error_type": "401",
        "additional_details": "401 Unauthorized",
        "source_path": "access_management",
        "message": "Unauthorized token"
      }
    ]
  ]
}

The process time is used to return the amount of calculation time needed for each row of data in milliseconds. Every successful execution to take at least 1 millisecond.

{
  "process_time": [
    10,
    7,
    5
  ]
}

Example responses

HTTP 200 OK Content-Type: application/json

{
  "outputs": [
    [
      "spider",
      "Australia",
      "specimen"
    ],
    [
      "Spiders, with eight legs, spin silk webs to catch prey",
      "Australia is a vast island continent renowned for its diverse landscapes, unique wildlife, and vibrant cities.",
      [
        [ "name", "poison"],
        [ "Sydney Funnel Web Spider", true],
        [ "Redback Spider", true]
      ]
    ]
  ],
  "warnings": [ null ],
  "errors": [ null ],
  "process_time": [ 1 ]
}

HTTP 200 OK Content-Type: application/json

{
  "outputs": [
    [
      "specimen"
    ],
    [
      [
        [ "name", "poisonous"],
        [ "Cobra", true],
        [ "Black Mamba", true],
        [ "Python", false],
        [ "Redback Spider", true],
        [ "Tarantula", false],
        [ "Viper", true],
        [ "Black Widow", true],
        [ "Boa Constrictor", false],
        [ "Wolf Spider", false],
        [ "King Cobra", true],
        [ "Sydney Funnel Web Spider", true]
      ]
    ]
  ],
  "warnings": [
    [
      {
        "source_path": "type",
        "message": "Unknown specimen type spader, skept type filter"
      },
      {
        "source_path": "main_country",
        "message": "Unknown country Astralaya, skept country filter"
      }
    ]
  ],
  "errors": [ null ],
  "process_time": [ 1 ]
}

HTTP 500 INTERNAL SERVER ERROR Content-Type: application/json

{
  "outputs": [],
  "warnings": [],
  "errors": [
    [
      {
        "error_category": "validation_error",
        "error_type": "500",
        "additional_details": "500 Internal Server Error",
        "source_path": "database",
        "message": "Connection timeout"
      } 
    ]
  ],
  "process_time": [ 1 ]
}

Last updated 11 days ago