> For the complete documentation index, see [llms.txt](https://docs.coherent.global/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.coherent.global/xconnector/introduction-to-xconnector.md).

# Introduction to XConnector

{% hint style="info" %}
This feature is only enabled on certain environments and tenants. [Contact support](/support/contact-support.md) for more information about this feature.
{% endhint %}

`XConnector` is an integration pattern that allows Spark services to interact with external applications or services.

Unlike traditional Spark services, where execution logic is derived directly from an Excel model and compiled into a WebAssembly module, the `XConnector` pattern enables Spark to invoke externally hosted code outside of the Spark platform. This allows organizations to integrate Spark calculations with existing systems, custom logic, and external data sources while maintaining a consistent API interface through Spark. Because execution occurs outside the Spark platform, ownership and operational responsibility for that external code lies with the customer.

Spark acts as an integration layer that forwards requests to an external application.

## `XConnector` as an integration pattern

Traditional Spark services operate as follows:

```mermaid
flowchart LR
  A["Excel workbook"] --> B["WebAssembly module"] --> C["Spark API endpoint"] --> D["Execution within Spark runtime"]
```

With the `XConnector` integration pattern Spark:

* Defines the API structure.
* Maps inputs and outputs.
* Forwards requests to an external system.

```mermaid
flowchart LR
  A["Excel workbook"] --> B["WebAssembly module"] --> C["Spark API endpoint"] --> D["Proxy / API gateway"] --> E["Customer-hosted remote application"]
```

The actual execution logic runs outside of the Spark platform in a remote application. 

## Use cases

`XConnector` is an extremely powerful capability when combined with existing Spark services. Code written in programming languages (such as [C#](https://learn.microsoft.com/en-us/dotnet/csharp/), [Node.js](https://nodejs.org/docs/), or [Python](https://docs.python.org/)) can be executed, with results returned to Spark.

Recommended use cases:

* Provide consistent access to internal and external calculations and/or data for Spark users in Excel via [Coherent Assistant](https://docs.coherent.global/assistant/).
* Deploy knowledge produced in Excel and Python using consistent API structures.
* Incorporate external data as part of the logic for an existing Spark service.
* Centralize a conditional multiple step process using Spark services that orchestrate other Spark services and external data.

## Architecture and information flow

```mermaid
sequenceDiagram

    participant Client as Client application
    participant Spark as Spark XConnector service
    participant Proxy as Proxy service
    participant Remote as Remote service
    
    Note over Spark: Interface<br>request to downstream
    Note over Proxy: Authenticate and relay
    Note over Remote: Process inputs<br>Return output
    
    Client ->> Spark: HTTP request
    Spark -->> Proxy: HTTP request
    Proxy ->> Remote: HTTP request
    Remote ->> Proxy: HTTP response
    Proxy -->> Spark: HTTP response
    Spark ->> Client: HTTP response
    
    Note over Spark: Compose<br>response to client application
    
```

1. Client application invokes a Spark API endpoint and sends input data.
2. Spark evaluates the [#spark-xconnector-service](#spark-xconnector-service "mention") configuration associated with the service.
3. Spark constructs an HTTP request containing the input payload and authentication token.
4. Request is sent to the proxy service.
5. [#proxy-service](#proxy-service "mention") validates the request and forwards to the [#remote-application](#remote-application "mention").
6. [#remote-application](#remote-application "mention") processes the request and returns a response.
7. Response is returned through the [#proxy-service](#proxy-service "mention") and Spark back to the client application.

Spark acts as an orchestration layer, while the remote application performs the actual computation.

### Spark `XConnector` service

An `XConnector` service in Spark resembles a standard Spark service but has a different purpose. Instead of executing a WebAssembly module derived from an Excel model, the service is used to:

* Define the input and output schema.
* Configure the remote service endpoint.
* Forward requests to the external service.

The Excel workbook associated with the `XConnector` service is typically used only to define input and output mappings (see [Tutorial: Map inputs and outputs](/build-spark-services/tutorial-map-inputs-and-outputs.md)) as well as `XConnector` configurations. No computational logic needs to exist in the workbook. From the perspective of a client application, the `XConnector` service behaves like a normal Spark API endpoint.

### Remote application

The remote application is the system that performs the actual computation. It must expose an `HTTPS` endpoint that can:

* Accept JSON input.
* Execute business logic.
* Return JSON output.

<details>

<summary>Examples of software for remote service implementations</summary>

* [AWS Lambda](https://aws.amazon.com/lambda/)
* [Azure Functions](https://azure.microsoft.com/en-us/products/functions)
* [Node.js](https://nodejs.org/) or [Python](https://www.python.org/) web services
* Container-based microservices
* On-premises HTTP services

</details>

### Proxy service

Place a proxy or API gateway between Spark and the [#remote-application](#remote-application "mention"). The proxy service acts as a secure relay between Spark and the remote application.

* Authentication and authorization validation.
* Token issuer validation.
* Rate limiting and traffic control.
* Request transformation.
* Monitoring and logging.
* Protection of internal infrastructure.

<details>

<summary>Common proxy implementations</summary>

* [Amazon API Gateway](https://aws.amazon.com/api-gateway/)
* [Azure API Management](https://azure.microsoft.com/en-us/products/api-management)
* [Google Cloud Apigee API Management](https://cloud.google.com/apigee)
* [Kong Gateway](https://konghq.com/products/kong-gateway)

</details>

## Shared responsibility model

Because the `XConnector` architecture involves systems outside the Spark platform, responsibilities are divided between Coherent and the customer. Spark provides the integration mechanism, but external systems remain under the customer’s operational control.

| Responsibility area                                                            | Coherent                         | Customer                         |
| ------------------------------------------------------------------------------ | -------------------------------- | -------------------------------- |
| Spark platform                                                                 | :white\_check\_mark: Responsible |                                  |
| Spark API infrastructure                                                       | :white\_check\_mark: Responsible |                                  |
| [#spark-xconnector-service](#spark-xconnector-service "mention") configuration | :white\_check\_mark: Responsible | :white\_check\_mark: Responsible |
| [#proxy-service](#proxy-service "mention")                                     |                                  | :white\_check\_mark: Responsible |
| [#remote-application](#remote-application "mention")                           |                                  | :white\_check\_mark: Responsible |
| Infrastructure hosting remote services                                         |                                  | :white\_check\_mark: Responsible |
| Secrets, API keys, and credentials                                             |                                  | :white\_check\_mark: Responsible |
| Compliance and regulatory controls                                             |                                  | :white\_check\_mark: Responsible |