Setup the Hybrid Runner

nodegen-server acts as a worker and does not handle user authentication or authorization. It does not store any sensitive data during its runtime, including when executing models or retrieving Wasm modules. It is the responsibility of the end user to secure the server and implement necessary authentication and authorization mechanisms such as a proxy or API gateway.

There are multiple ways to setup the hybrid runner:

1. Hybrid method 1: Automatic Wasm pull: With this option, once nodegen-server is running, it can automatically fetch the necessary Wasm modules from the Spark SaaS.

2. Hybrid method 2: Manual Wasm mount: This alternate method involves downloading the required Wasm modules using the Export API and utilizing Docker's bind-mount feature to execute them.

3. Hybrid method 3: Custom Docker image: This is a variation of method 2 where the end result is a Docker image that can be deployed which includes the runner and Wasm files.

We always recommend using the latest version of the runner available. If you are not using the latest version of the runner, you can potentially encounter issues when used in conjunction with newer versions of Neuron. More information is described in Manage compatible Neuron versions.

Information about OpenTelemetry can be found in Configure OpenTelemetry.

The following steps assume that you have already created and tested a Spark service in the Spark SaaS environment. If you haven't done so, please refer to the guides such as Get started in 5 minutes or How to: Create a Spark service.