How to: Deploy Spark services with Azure DevOps

Use PowerShell scripting to integrate Spark service promotion with the ImpEx APIs and Azure DevOps Pipelines.

Prerequisites

• Client credentials configured in both the source and destination tenants with access to the source and destination targets.

• All referenced services must have a compiled Neuron WebAssembly module.

• Any destination folders need to already exist and be accessible with the provided authorization.

Information flow

1. Azure DevOps Pipelines to executes the PowerShell script.

2. The PowerShell script performs the following steps:

   1. Acquires client access tokens for both the source and destination tenants using Client credentials grant (OAuth 2.0).

   2. Initiates the Export API package generation job in the source tenant.

   3. Checks the Export API for the completed job.

   4. Once completed, it downloads the package.

   5. The package is then uploaded to the destination tenant and deployed using the Import API.

ImpEx.ps1

6KB

Open

PowerShell script parameters

The parameters for the PowerShell script are aligned with those from the Export and Import APIs. Make sure you understand these APIs before defining the parameters of the script.

Key	Value
exportEndpoint	Reference: POST export job https://excel.{environment}.coherent.global/{tenant}/api/v4/export
importEndpoint	Reference: POST import job https://excel.{environment}.coherent.global/{tenant}/api/v4/import
serviceUriSource	Reference: Request body inputs.services Example: myfolder/myservice
serviceUriDestination	Reference: Request body inputs.services and Request body inputs.services_modify Example: myfolder/myservice
sourceSystem	Tag API call. Example: mycicd
correlationId	Tag API call. Example: 456
servicesExisting	Reference: Import Request body servicesExisting Example: add_version
sourceKeycloakMetadataUri and destinationKeycloakMetadataUri	Source of the Keycloak OpenID endpoint configuration. https://keycloak.{environment}.coherent.global/auth/realms/{tenant}/protocol/openid-connect/token
sourceKeycloakClientId and destinationKeycloakClientId	The client ID for the source Keycloak client. Example: client-credentials
sourceKeycloakClientSecret and destinationKeycloakClientSecret	The client secret for the source Keycloak client. Example: 9Oh1nscDtPzMehwA3ejKJFAar3RRktHY

Set up the pipeline in Azure DevOps

1. Read the article PowerShell scripts for pipelines and follow the appropriate instructions for your Azure instance.

2. Push the PowerShell script ImpEx.ps1 to the root of the project repository.

3. Add the completed YAML code to the pipeline file.

Empty template

- powershell: .\ImpEx.ps1 `
    -exportEndpoint " exportEndpoint " `
    -importEndpoint " importEndpoint " `
    -serviceUriSource " serviceUriSource " `
    -serviceUriDestination " serviceUriDestination " `
    -sourceSystem " sourceSystem" `
    -correlationId " correlationId " `
    -servicesExisting " servicesExisting " `
    -sourceKeycloakMetadataUri " sourceKeycloakMetadataUri " `
    -sourceKeycloakClientId " sourceKeycloakClientId " `
    -sourceKeycloakClientSecret " sourceKeycloakClientSecret " `
    -destinationKeycloakMetadataUri " destinationKeycloakMetadataUri " `
    -destinationKeycloakClientId " destinationKeycloakClientId " `
    -destinationKeycloakClientSecret " destinationKeycloakClientSecret "

Sample

- powershell: .\ImpEx.ps1 `
    -exportEndpoint "https://excel.my-source-environment.coherent.global/my-source-tenant/api/v4/export" `
    -importEndpoint "https://excel.my-destination-environment.coherent.global/my-destination-tenant/api/v4/import" `
    -serviceUriSource "CICD-SOURCE/Service2" `
    -serviceUriDestination "CICD-DESTINATION/Service2" `
    -sourceSystem "mycicd" `
    -correlationId "456" `
    -servicesExisting "add_version" `
    -sourceKeycloakMetadataUri "https://keycloak.my-source-environment.coherent.global/auth/realms/my-source-tenant/protocol/openid-connect/token" `
    -sourceKeycloakClientId "client-credentials" `
    -sourceKeycloakClientSecret "9Oh1nscDtPzMehwA3ejKJFAar3RRktHY" `
    -destinationKeycloakMetadataUri "https://keycloak.my-destination-environment.coherent.global/auth/realms/my-destination-tenant/protocol/openid-connect/token" `
    -destinationKeycloakClientId "client-credentials" `
    -destinationKeycloakClientSecret "lZai8ARrLUBpLBLbueHusWUfCoYZ6DxN"