Advanced Reference Guide

• 1: Advanced CLI
• 2: Naming Limits
• 3: Rest API
• 4: Scheduler
• 5: Runtimes under the hood
• 6: Building your runtime

1 - Advanced CLI

OpenServerless Cli

OpenServerless offers a powerful command line interface named ops which extends and embeds the OpenWhisk wsk.

Download instructions are available here.

Let’s see some advanced uses of ops.

OpenServerless access is usually configured logging into the platform with the ops -login command.

You can also configure access directly using the ops -wsk command.

There are two required properties to configure:

1. API host (name or IP address) for the OpenWhisk and OpenServerless deployment you want to use.

2. Authorization key (username and password) which grants you access to the OpenWhisk and OpenServerless API.

The API host is the installation host, the one you configure in installation with ops config apihost

ops -wsk property set --apihost <openwhisk_baseurl>

If you know your authorization key, you can configure the CLI to use it. Otherwise, you will need to obtain an authorization key for most CLI operations. The API key is visible in the file ~/.wskprops after you perform a ops -login. This file can be sourced to be read as environment variables.

source ~/.wskprops
ops -wsk property set --auth $AUTH

Tip: The OpenWhisk and OpenServerless CLI stores properties in the ~/.wskprops configuration file by default. The location of this file can be altered by setting the WSK_CONFIG_FILE environment variable.

The required properties described above have the following keys in the .wskprops file:

• APIHOST - Required key for the API host value.

• AUTH - Required key for the Authorization key.

To verify your CLI setup, try ops action list.

Configure the CLI to use an HTTPS proxy

The CLI can be setup to use an HTTPS proxy. To setup an HTTPS proxy, an environment variable called HTTPS_PROXY must be created. The variable must be set to the address of the HTTPS proxy, and its port using the following format: {PROXY IP}:{PROXY PORT}.

Configure the CLI to use client certificate

The CLI has an extra level of security from client to apihost, system provides default client certificate configuration which deployment process generated, then you can refer to below steps to use client certificate:

ops -wsk property set --cert <client_cert_path> --key <client_key_path>

2 - Naming Limits

The following sections provide more details about the OpenWhisk and OpenServerless system.

Entities

Namespaces and packages

OpenWhisk and OpenServerless actions, triggers, and rules belong in a namespace, and optionally a package.

Packages can contain actions and feeds. A package cannot contain another package, so package nesting is not allowed. Also, entities do not have to be contained in a package.

In OpenServerless a namespace corresponds to an user. You can create users with the admin subcommand of the CLI.

The fully qualified name of an entity is /namespaceName[/packageName]/entityName. Notice that / is used to delimit namespaces, packages, and entities.

If the fully qualified name has three parts: /namespaceName/packageName/entityName, then the namespace can be entered without a prefixed /; otherwise, namespaces must be prefixed with a /.

For convenience, the namespace can be left off if it is the user’s default namespace.

For example, consider a user whose default namespace is /myOrg. Following are examples of the fully qualified names of a number of entities and their aliases.

You will be using this naming scheme when you use the OpenWhisk and OpenServerless CLI, among other places.

Entity names

The names of all entities, including actions, triggers, rules, packages, and namespaces, are a sequence of characters that follow the following format:

• The first character must be an alphanumeric character, or an underscore.

• The subsequent characters can be alphanumeric, spaces, or any of the following: _, @, ., -.

• The last character can’t be a space.

More precisely, a name must match the following regular expression (expressed with Java metacharacter syntax): \A([\w]|[\w][\w@ .-]*[\w@.-]+)\z.

System limits

Actions

OpenWhisk and OpenServerless has a few system limits, including how much memory an action can use and how many action invocations are allowed per minute.

Note: On Openwhisk 2.0 with the scheduler service, concurrent in the table below really means the max containers that can be provisioned at once for a namespace. The api may be able to accept more activations than this number at once depending on a number of factors.

The following table lists the default limits for actions.

Per action timeout (ms) (Default: 60s)

• The timeout limit N is in the range [100ms..300000ms] and is set per action in milliseconds.

• A user can change the limit when creating the action.

• A container that runs longer than N milliseconds is terminated.

Per action memory (MB) (Default: 256MB)

• The memory limit M is in the range from [128MB..512MB] and is set per action in MB.

• A user can change the limit when creating the action.

• A container cannot have more memory allocated than the limit.

Per action max instance concurrency (Default: namespace limit for concurrent invocations) Only applicable using new scheduler

• The max containers that will be created for an action before throttling in the range from [1..concurrentInvocations limit for namespace]

• By default the max allowed containers / server instances for an action is equal to the namespace limit.

• A user can change the limit when creating the action.

• Defining a lower limit than the namespace limit means your max container concurrency will be the action defined limit.

• If using actionConcurrency > 1 such that your action can handle multiple requests per instance, your true concurrency limit is actionContainerConcurrency * actionConcurrency.

• The actions within a namespaces containerConcurrency total do not have to add up to the namespace limit though you can configure it that way to guarantee an action will get exactly the action container concurrency.

• For example with a namespace limit of 30 with 2 actions each with a container limit of 20; if the first action is using 20, there will still be space for 10 for the other.

Per action logs (MB) (Default: 10MB)

• The log limit N is in the range [0MB..10MB] and is set per action.

• A user can change the limit when creating or updating the action.

• Logs that exceed the set limit are truncated and a warning is added as the last output of the activation to indicate that the activation exceeded the set log limit.

Per action artifact (MB) (Default: 48MB)

• The maximum code size for the action is 48MB.

• It is recommended for a JavaScript action to use a tool to concatenate all source code including dependencies into a single bundled file.

Per activation payload size (MB) (Fixed: 1MB)

• The maximum POST content size plus any curried parameters for an action invocation or trigger firing is 1MB.

Per activation result size (MB) (Fixed: 1MB)

• The maximum size of a result returned from an action is 1MB.

Per namespace concurrent invocation (Default: 100)

• The number of activations that are either executing or queued for execution for a namespace cannot exceed 100.

• A user is currently not able to change the limits.

Invocations per minute (Fixed: 120)

• The rate limit N is set to 120 and limits the number of action invocations in one minute windows.

• A user cannot change this limit when creating the action.

• A CLI or API call that exceeds this limit receives an error code corresponding to HTTP status code 429: TOO MANY REQUESTS.

Size of the parameters (Fixed: 1MB)

• The size limit for the parameters on creating or updating of an action/package/trigger is 1MB.

• The limit cannot be changed by the user.

• An entity with too big parameters will be rejected on trying to create or update it.

Per Docker action open files ulimit (Fixed: 1024:1024)

• The maximum number of open files is 1024 (for both hard and soft limits).

• The docker run command use the argument --ulimit nofile=1024:1024.

• For more information about the ulimit for open files see the docker run documentation.

Per Docker action processes ulimit (Fixed: 1024)

• The maximum number of processes available to the action container is 1024.

• The docker run command use the argument --pids-limit 1024.

• For more information about the ulimit for maximum number of processes see the docker run documentation.

Triggers

Triggers are subject to a firing rate per minute as documented in the table below.

Triggers per minute (Fixed: 60)

• The rate limit N is set to 60 and limits the number of triggers that may be fired in one minute windows.

• A user cannot change this limit when creating the trigger.

• A CLI or API call that exceeds this limit receives an error code corresponding to HTTP status code 429: TOO MANY REQUESTS.

3 - Rest API

Using REST APIs with OpenWhisk and OpenServerless

After your OpenWhisk and OpenServerlesss environment is enabled, you can use it with your web apps or mobile apps with REST API calls.

For more details about the APIs for actions, activations, packages, rules, and triggers, see the OpenWhisk and OpenServerless API documentation.

All the capabilities in the system are available through a REST API. There are collection and entity endpoints for actions, triggers, rules, packages, activations, and namespaces.

These are the collection endpoints:

https://$APIHOST/api/v1/namespaces
https://$APIHOST/api/v1/namespaces/{namespace}/actions
https://$APIHOST/api/v1/namespaces/{namespace}/triggers
https://$APIHOST/api/v1/namespaces/{namespace}/rules
https://$APIHOST/api/v1/namespaces/{namespace}/packages
https://$APIHOST/api/v1/namespaces/{namespace}/activations
https://$APIHOST/api/v1/namespaces/{namespace}/limits

The $APIHOST is the OpenWhisk and OpenServerless API hostname (for example, localhost, 172.17.0.1, and so on). For the {namespace}, the character _ can be used to specify the user’s default namespace.

You can perform a GET request on the collection endpoints to fetch a list of entities in the collection.

There are entity endpoints for each type of entity:

https://$APIHOST/api/v1/namespaces/{namespace}
https://$APIHOST/api/v1/namespaces/{namespace}/actions/[{packageName}/]{actionName}
https://$APIHOST/api/v1/namespaces/{namespace}/triggers/{triggerName}
https://$APIHOST/api/v1/namespaces/{namespace}/rules/{ruleName}
https://$APIHOST/api/v1/namespaces/{namespace}/packages/{packageName}
https://$APIHOST/api/v1/namespaces/{namespace}/activations/{activationName}

The namespace and activation endpoints support only GET requests. The actions, triggers, rules, and packages endpoints support GET, PUT, and DELETE requests. The endpoints of actions, triggers, and rules also support POST requests, which are used to invoke actions and triggers and enable or disable rules.

All APIs are protected with HTTP Basic authentication. You can use the ops admin tool to generate a new namespace and authentication. The Basic authentication credentials are in the AUTH property in your ~/.nuvprops file, delimited by a colon. You can also retrieve these credentials using the CLI running ops property get --auth.

The following is an example that uses the cURL command tool to get the list of all packages in the whisk.system namespace:

curl -u USERNAME:PASSWORD https://$APIHOST/api/v1/namespaces/whisk.system/packages

[
  {
    "name": "slack",
    "binding": false,
    "publish": true,
    "annotations": [
      {
        "key": "description",
        "value": "Package that contains actions to interact with the Slack messaging service"
      }
    ],
    "version": "0.0.1",
    "namespace": "whisk.system"
  }
]

In this example the authentication was passed using the -u flag, you can pass this value also as part of the URL as https://USERNAME:PASSWORD@$APIHOST.

The OpenWhisk API supports request-response calls from web clients. OpenWhisk responds to OPTIONS requests with Cross-Origin Resource Sharing headers. Currently, all origins are allowed (that is, Access-Control-Allow-Origin is “*”), the standard set of methods are allowed (that is, Access-Control-Allow-Methods is GET, DELETE, POST, PUT, HEAD), and Access-Control-Allow-Headers yields Authorization, Origin, X-Requested-With, Content-Type, Accept, User-Agent.

Attention: Because OpenWhisk and OpenServerless currently supports only one key per namespace, it is not recommended to use CORS beyond simple experiments. Use Web Actions to expose your actions to the public and not use the OpenWhisk and OpenServerless authorization key for client applications that require CORS.

Using the CLI verbose mode

The OpenWhisk and OpenServerless CLI is an interface to the OpenWhisk and OpenServerless REST API. You can run the CLI in verbose mode with the flag -v, this will print truncated information about the HTTP request and response. To print all information use the flag -d for debug.

Note: HTTP request and response bodies will only be truncated if they exceed 1000 bytes.

Let’s try getting the namespace value for the current user.

ops namespace list -v

REQUEST:
[GET]  https://$APIHOST/api/v1/namespaces
Req Headers
{
  "Authorization": [
    "Basic XXXYYYY"
  ],
  "User-Agent": [
    "OpenWhisk and OpenServerless-CLI/1.0 (2017-08-10T20:09:30+00:00)"
  ]
}
RESPONSE:Got response with code 200
Resp Headers
{
  "Content-Type": [
    "application/json; charset=UTF-8"
  ]
}
Response body size is 28 bytes
Response body received:
["john@example.com_dev"]

As you can see you the printed information provides the properties of the HTTP request, it performs a HTTP method GET on the URL https://$APIHOST/api/v1/namespaces using a User-Agent header OpenWhisk and OpenServerless-CLI/1.0 (<CLI-Build-version>) and Basic Authorization header Basic XXXYYYY. Notice that the authorization value is your base64-encoded OpenWhisk and OpenServerless authorization string. The response is of content type application/json.

Actions

Note: In the examples that follow, $AUTH and $APIHOST represent environment variables set respectively to your OpenWhisk and OpenServerless authorization key and API host.

To create or update an action send a HTTP request with method PUT on the the actions collection. For example, to create a nodejs:6 action with the name hello using a single file content use the following:

curl -u $AUTH -d '{"namespace":"_","name":"hello","exec":{"kind":"nodejs:6","code":"function main(params) { return {payload:\"Hello \"+params.name}}"}}' -X PUT -H "Content-Type: application/json" https://$APIHOST/api/v1/namespaces/_/actions/hello?overwrite=true

To perform a blocking invocation on an action, send a HTTP request with a method POST and body containing the input parameter name use the following:

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/actions/hello?blocking=true \
-X POST -H "Content-Type: application/json" \
-d '{"name":"John"}'

You get the following response:

{
  "duration": 2,
  "name": "hello",
  "subject": "john@example.com_dev",
  "activationId": "c7bb1339cb4f40e3a6ccead6c99f804e",
  "publish": false,
  "annotations": [{
    "key": "limits",
    "value": {
      "timeout": 60000,
      "memory": 256,
      "logs": 10
    }
  }, {
    "key": "path",
    "value": "john@example.com_dev/hello"
  }],
  "version": "0.0.1",
  "response": {
    "result": {
      "payload": "Hello John"
    },
    "success": true,
    "status": "success"
  },
  "end": 1493327653769,
  "logs": [],
  "start": 1493327653767,
  "namespace": "john@example.com_dev"
}

If you just want to get the response.result, run the command again with the query parameter result=true

curl -u $AUTH "https://$APIHOST/api/v1/namespaces/_/actions/hello?blocking=true&result=true" \
-X POST -H "Content-Type: application/json" \
-d '{"name":"John"}'

You get the following response:

{
  "payload": "hello John"
}

Annotations and Web Actions

To create an action as a web action, you need to add an annotation of web-export=true for web actions. Since web-actions are publicly accessible, you should protect pre-defined parameters (i.e., treat them as final) using the annotation final=true. If you create or update an action using the CLI flag --web true this command will add both annotations web-export=true and final=true.

Run the curl command providing the complete list of annotations to set on the action

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/actions/hello?overwrite=true \
-X PUT -H "Content-Type: application/json" \
-d '{"namespace":"_","name":"hello","exec":{"kind":"nodejs:6","code":"function main(params) { return {payload:\"Hello \"+params.name}}"},"annotations":[{"key":"web-export","value":true},{"key":"raw-http","value":false},{"key":"final","value":true}]}'

You can now invoke this action as a public URL with no OpenWhisk and OpenServerless authorization. Try invoking using the web action public URL including an optional extension such as .json or .http for example at the end of the URL.

curl https://$APIHOST/api/v1/web/john@example.com_dev/default/hello.json?name=John

{
  "payload": "Hello John"
}

Note that this example source code will not work with .http, see web actions documentation on how to modify.

Sequences

To create an action sequence, you need to create it by providing the names of the actions that compose the sequence in the desired order, so the output from the first action is passed as input to the next action.

$ ops action create sequenceAction –sequence /whisk-system/utils/split,/whisk-system/utils/sort

Create a sequence with the actions /whisk-system/utils/split and /whisk-system/utils/sort.

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/actions/sequenceAction?overwrite=true \
-X PUT -H "Content-Type: application/json" \
-d '{"namespace":"_","name":"sequenceAction","exec":{"kind":"sequence","components":["/whisk.system/utils/split","/whisk.system/utils/sort"]},"annotations":[{"key":"web-export","value":true},{"key":"raw-http","value":false},{"key":"final","value":true}]}'

Take into account when specifying the names of the actions, they have to be full qualified.

Triggers

To create a trigger, the minimum information you need is a name for the trigger. You could also include default parameters that get passed to the action through a rule when the trigger gets fired.

Create a trigger with name events with a default parameter type with value webhook set.

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/triggers/events?overwrite=true \
-X PUT -H "Content-Type: application/json" \
-d '{"name":"events","parameters":[{"key":"type","value":"webhook"}]}'

Now whenever you have an event that needs to fire this trigger it just takes an HTTP request with a method POST using the OpenWhisk and OpenServerless Authorization key.

To fire the trigger events with a parameter temperature, send the following HTTP request.

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/triggers/events \
-X POST -H "Content-Type: application/json" \
-d '{"temperature":60}'

Rules

To create a rule that associates a trigger with an action, send a HTTP request with a PUT method providing the trigger and action in the body of the request.

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/rules/t2a?overwrite=true \
-X PUT -H "Content-Type: application/json" \
-d '{"name":"t2a","status":"","trigger":"/_/events","action":"/_/hello"}'

Rules can be enabled or disabled, and you can change the status of the rule by updating its status property. For example, to disable the rule t2a send in the body of the request status: "inactive" with a POST method.

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/rules/t2a?overwrite=true \
-X POST -H "Content-Type: application/json" \
-d '{"status":"inactive","trigger":null,"action":null}'

Packages

To create an action in a package you have to create a package first, to create a package with name iot send an HTTP request with a PUT method

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/packages/iot?overwrite=true \
-X PUT -H "Content-Type: application/json" \
-d '{"namespace":"_","name":"iot"}'

To force delete a package that contains entities, set the force parameter to true. Failure will return an error either for failure to delete an action within the package or the package itself. The package will not be attempted to be deleted until all actions are successfully deleted.

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/packages/iot?force=true \
-X DELETE

Activations

To get the list of the last 3 activations use a HTTP request with a GET method, passing the query parameter limit=3

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/activations?limit=3

To get all the details of an activation including results and logs, send a HTTP request with a GET method passing the activation identifier as a path parameter

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/activations/f81dfddd7156401a8a6497f2724fec7b

Limits

To get the limits set for a namespace (i.e. invocationsPerMinute, concurrentInvocations, firesPerMinute, actionMemoryMax, actionLogsMax…)

curl -u $AUTH https://$APIHOST/api/v1/namespaces/_/limits

Note that the default system values are returned if no specific limits are set for the user corresponding to the authenticated identity.

4 - Scheduler

OpenServerless Operator offers the possibility to deploy a simple “scheduler” to invoke repetitive or one-shot OpenWhisk actions. For example, an action executing a SQL script to create a PostgreSQL Database or inserting reference data, or simply an action that sends notifications with an API call every day at the same time.

How to Activate the Scheduler

Using the ops CLI, you can enable the scheduler with the following command:

ops config enable --cron

# if OpenServerless is not yet deployed
ops setup devcluster

# alternatively if OpenServerless is already deployed
ops update apply

By default, the internal scheduler executes a job every minute that starts searching for OpenWhisk actions with special annotations.

How to Deploy a Repetitive Action

Let’s assume we want to deploy an OpenWhisk action to be executed every 30 minutes. Suppose it’s an action that simply prints something, like this:

def main(args): 
    print('Hello from a repeated action')
    return {
        'body': 'action invoked'
    }

abd save it to a file called scheduled-action.py

To deploy the action and instruct OpenServerless to execute it every 30 minutes, issue the following command:

ops action create scheduled-action scheduled-action.py -a cron "*/30 * * * *"

So you can create the action in the usual way and at the end add -a cron yourCronExpression.

How to Deploy a One-Shot Execution Action

Now suppose we want to execute the same action scheduled-action.py only once.

To deploy an action and request a single execution automatically via the OpenServerless Scheduler, issue the following command:

ops action create scheduled-action scheduled-action.py -a autoexec true

If we now print activation logs with ops activation poll, we will see our action execution log:

Activation: 'scheduled' (ebd532139a464e9d9532139a46ae9d8a)
[
    "2024-03-08T07:28:02.050739962Z stdout: Hello from a scheduled action"
]

Remarks

The Scheduler executes the action according to the following rules:

Actions are called in a non-blocking fashion. To verify execution and logs, use the command ops activation list. Actions are invoked without any parameters. It is advised to deploy actions with self-contained parameters.

5 - Runtimes under the hood

Adding Action Language Runtimes

OpenWhisk and OpenServerless supports several languages and runtimes but there may be other languages or runtimes that are important for your organization, and for which you want tighter integration with the platform.

The platform is extensible and you can add new languages or runtimes (with custom packages and third-party dependencies)

💡 NOTE

This guide describes the contract a runtime must satisfy. However all the OpenServerless runtimes are implemented the using the ActionLoop Proxy. This proxy is implemented in Go, already satifies the semantic of a runtime ands makes very easy to build a new runtime. You just need to provide “launcher code” in your favorite programming language and a compilation script (generally written in python) for the initialization of an action. You are advised to use it for your own runtimes and use the material of this document as reference for the behaviour of a runtime.

Runtime general requirements

The unit of execution for all functions is a Docker container which must implement a specific Action interface that, in general performs:

1. Initialization - accepts an initialization payload (the code) and prepared for execution,

2. Activation - accepts a runtime payload (the input parameters) and

   • prepares the activation context,

   • runs the function,

   • returns the function result,

3. Logging - flushes all stdout and stderr logs and adds a frame marker at the end of the activation.

The specifics of the Action interface and its functions are shown below.

The runtimes manifest

Actions when created specify the desired runtime for the function via a property called kind. When using the nuv CLI, this is specified as --kind <runtime-kind>. The value is typically a string describing the language (e.g., nodejs) followed by a colon and the version for the runtime as in nodejs:20 or php:8.1.

The manifest is a map of runtime family names to an array of specific kinds. As an example, the following entry add a new runtime family called nodejs with a single kind nodejs:20.

{
  "nodejs": [{
    "kind": "nodejs:20",
    "default": true,
    "image": {
      "prefix": "openwhisk",
      "name": "action-nodejs-v20",
      "tag": "latest"
    }
  }]
}

The default property indicates if the corresponding kind should be treated as the default for the runtime family. The JSON image structure defines the Docker image name that is used for actions of this kind (e.g., openwhisk/nodejs10action:latest for the JSON example above).

The test action

The standard test action is shown below in JavaScript. It should be adapted for the new language and added to the test artifacts directory with the name <runtime-kind>.txt for plain text file or <runtime-kind>.bin for a a binary file. The <runtime-kind> must match the value used for kind in the corresponding runtime manifest entry, replacing : in the kind with a -. For example, a plain text function for nodejs:20 becomes nodejs-20.txt.

function main(args) {
    var str = args.delimiter + " ☃ " + args.delimiter;
    console.log(str);
    return { "winter": str };
}

Action Interface

An action consists of the user function (and its dependencies) along with a proxy that implements a canonical protocol to integrate with the OpenWhisk and OpenServerless platform.

The proxy is a web server with two endpoints.

• It listens on port 8080.

• It implements /init to initialize the container.

• It also implements /run to activate the function.

The proxy also prepares the execution context, and flushes the logs produced by the function to stdout and stderr.

Initialization

The initialization route is /init. It must accept a POST request with a JSON object as follows:

{
  "value": {
    "name" : String,
    "main" : String,
    "code" : String,
    "binary": Boolean,
    "env": Map[String, String]
  }
}

• name is the name of the action.

• main is the name of the function to execute.

• code is either plain text or a base64 encoded string for binary functions (i.e., a compiled executable).

• binary is false if code is in plain text, and true if code is base64 encoded.

• env is a map of key-value pairs of properties to export to the environment. And contains several properties starting with the __OW_ prefix that are specific to the running action.

  • __OW_API_KEY the API key for the subject invoking the action, this key may be a restricted API key. This property is absent unless explicitly requested.

  • __OW_NAMESPACE the namespace for the activation (this may not be the same as the namespace for the action).

  • __OW_ACTION_NAME the fully qualified name of the running action.

  • __OW_ACTION_VERSION the internal version number of the running action.

  • __OW_ACTIVATION_ID the activation id for this running action instance.

  • __OW_DEADLINE the approximate time when this initializer will have consumed its entire duration quota (measured in epoch milliseconds).

The initialization route is called exactly once by the OpenWhisk and OpenServerless platform, before executing a function. The route should report an error if called more than once. It is possible however that a single initialization will be followed by many activations (via /run). If an env property is provided, the corresponding environment variables should be defined before the action code is initialized.

Successful initialization: The route should respond with 200 OK if the initialization is successful and the function is ready to execute. Any content provided in the response is ignored.

Failures to initialize: Any response other than 200 OK is treated as an error to initialize. The response from the handler if provided must be a JSON object with a single field called error describing the failure. The value of the error field may be any valid JSON value. The proxy should make sure to generate meaningful log message on failure to aid the end user in understanding the failure.

Time limit: Every action in OpenWhisk and OpenServerless has a defined time limit (e.g., 60 seconds). The initialization must complete within the allowed duration. Failure to complete initialization within the allowed time frame will destroy the container.

Limitation: The proxy does not currently receive any of the activation context at initialization time. There are scenarios where the context is convenient if present during initialization. This will require a change in the OpenWhisk and OpenServerless platform itself. Note that even if the context is available during initialization, it must be reset with every new activation since the information will change with every execution.

Activation

The proxy is ready to execute a function once it has successfully completed initialization. The OpenWhisk and OpenServerless platform will invoke the function by posting an HTTP request to /run with a JSON object providing a new activation context and the input parameters for the function. There may be many activations of the same function against the same proxy (viz. container). Currently, the activations are guaranteed not to overlap — that is, at any given time, there is at most one request to /run from the OpenWhisk and OpenServerless platform.

The route must accept a JSON object and respond with a JSON object, otherwise the OpenWhisk and OpenServerless platform will treat the activation as a failure and proceed to destroy the container. The JSON object provided by the platform follows the following schema:

{
  "value": JSON,
  "namespace": String,
  "action_name": String,
  "api_host": String,
  "api_key": String,
  "activation_id": String,
  "transaction_id": String,
  "deadline": Number
}

• value is a JSON object and contains all the parameters for the function activation.

• namespace is the OpenWhisk and OpenServerless namespace for the action (e.g., whisk-system).

• action_name is the fully qualified name of the action.

• activation_id is a unique ID for this activation.

• transaction_id is a unique ID for the request of which this activation is part of.

• deadline is the deadline for the function.

• api_key is the API key used to invoke the action.

The value is the function parameters. The rest of the properties become part of the activation context which is a set of environment variables constructed by capitalizing each of the property names, and prefixing the result with __OW_. Additionally, the context must define __OW_API_HOST whose value is the OpenWhisk and OpenServerless API host. This value is currently provided as an environment variable defined at container startup time and hence already available in the context.

Successful activation: The route must respond with 200 OK if the activation is successful and the function has produced a JSON object as its result. The response body is recorded as the result of the activation.

Failed activation: Any response other than 200 OK is treated as an activation error. The response from the handler must be a JSON object with a single field called error describing the failure. The value of the error field may be any valid JSON value. Should the proxy fail to respond with a JSON object, the OpenWhisk and OpenServerless platform will treat the failure as an uncaught exception. These two failures modes are distinguished by the value of the response.status in the activation record which is application error if the proxy returned an error object, and action developer error otherwise.

Time limit: Every action in OpenWhisk and OpenServerless has a defined time limit (e.g., 60 seconds). The activation must complete within the allowed duration. Failure to complete activation within the allowed time frame will destroy the container.

Logging

The proxy must flush all the logs produced during initialization and execution and add a frame marker to denote the end of the log stream for an activation. This is done by emitting the token XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX as the last log line for the stdout and stderr streams. Failure to emit this marker will cause delayed or truncated activation logs.

Testing

Action Interface tests

The Action interface is enforced via a canonical test suite which validates the initialization protocol, the runtime protocol, ensures the activation context is correctly prepared, and that the logs are properly framed. Your runtime should extend this test suite, and of course include additional tests as needed.

Runtime proxy tests

The tests verify that the proxy can handle the following scenarios:

• Test the proxy can handle the identity functions (initialize and run).

• Test the proxy can handle pre-defined environment variables as well as initialization parameters.

• Test the proxy properly constructs the activation context.

• Test the proxy can properly handle functions with Unicode characters.

• Test the proxy can handle large payloads (more than 1MB).

• Test the proxy can handle an entry point other than main.

• Test the proxy does not permit re-initialization.

• Test the error handling for an action returning an invalid response.

• Test the proxy when initialized with no content.

The canonical test suite should be extended by the new runtime tests. Additional tests will be required depending on the feature set provided by the runtime.

Since the OpenWhisk and OpenServerless platform is language and runtime agnostic, it is generally not necessary to add integration tests. That is the unit tests verifying the protocol are sufficient. However, it may be necessary in some cases to modify the ops CLI or other OpenWhisk and OpenServerless clients. In which case, appropriate tests should be added as necessary. The OpenWhisk and OpenServerless platform will perform a generic integration test as part of its basic system tests. This integration test will require a test function to be available so that the test harness can create, invoke, and delete the action.

6 - Building your runtime

Developing a new Runtime with the ActionLoop proxy

The OpenWhisk and OpenServerless runtime specification defines the expected behavior of an OpenWhisk and OpenServerless runtime; you can choose to implement a new runtime from scratch by just following this specification. However, the fastest way to develop a new, compliant runtime is by reusing the ActionLoop proxy which already implements most of the specification and requires you to write code for just a few hooks to get a fully functional (and fast) runtime in a few hours or less.

What is the ActionLoop proxy

The ActionLoop proxy is a runtime “engine”, written in the Go programming language, originally developed specifically to support the OpenWhisk and OpenServerless Go language runtime. However, it was written in a generic way such that it has since been adopted to implement OpenWhisk and OpenServerless runtimes for Swift, PHP, Python, Rust, Java, Ruby and Crystal. Even though it was developed with compiled languages in mind it works equally well with scripting languages.

Using it, you can develop a new runtime in a fraction of the time needed for authoring a full-fledged runtime from scratch. This is due to the fact that you have only to write a command line protocol and not a fully-featured web server (with a small amount of corner cases to consider). The results should also produce a runtime that is fairly fast and responsive. In fact, the ActionLoop proxy has also been adopted to improve the performance of existing runtimes like Python, Ruby, PHP, and Java where performance has improved by a factor between 2x to 20x.

Precompilation of OpenWhisk and OpenServerless Actions

In addition to being the basis for new runtime development, ActionLoop runtimes can also support offline “precompilation” of OpenWhisk and OpenServerless Action source files into a ZIP file that contains only the compiled binaries which are very fast to start once deployed. More information on this approach can be found here: Precompiling Go Sources Offline which describes how to do this for the Go language, but the approach applies to any language supported by ActionLoop.

Tutorial - How to write a new runtime with the ActionLoop Proxy

This section contains a stepwise tutorial which will take you through the process of developing a new ActionLoop runtime using the Ruby language as the example.

General development process

The general procedure for authoring a runtime with the ActionLoop proxy requires the following steps:

• building a docker image containing your target language compiler and the ActionLoop runtime.

• writing a simple line-oriented protocol in your target language.

• writing a compilation script for your target language.

• writing some mandatory tests for your language.

ActionLoop Starter Kit

To facilitate the process, there is an actionloop-starter-kit in the openwhisk-devtools GitHub repository, that implements a fully working runtime for Python. It contains a stripped-down version of the real Python runtime (with some advanced features removed) along with guided, step-by-step instructions on how to translate it to a different target runtime language using Ruby as an example.

In short, the starter kit provides templates you can adapt in creating an ActionLoop runtime for each of the steps listed above, these include :

-checking out the actionloop-starter-kit from the openwhisk-devtools repository -editing the Dockerfile to create the target environment for your target language. -converting (rewrite) the launcher.py script to an equivalent for script for your target language. -editing the compile script to compile your action in your target language. -writing the mandatory tests for your target language, by adapting the ActionLoopPythonBasicTests.scala file.

As a starting language, we chose Python since it is one of the more human-readable languages (can be treated as pseudo-code). Do not worry, you should only need just enough Python knowledge to be able to rewrite launcher.py and edit the compile script for your target language.

Finally, you will need to update the ActionLoopPythonBasicTests.scala test file which, although written in the Scala language, only serves as a wrapper that you will use to embed your target language tests into.

Notation

In each step of this tutorial, we typically show snippets of either terminal transcripts (i.e., commands and results) or “diffs” of changes to existing code files.

Within terminal transcript snippets, comments are prefixed with # character and commands are prefixed by the $ character. Lines that follow commands may include sample output (from their execution) which can be used to verify against results in your local environment.

When snippets show changes to existing source files, lines without a prefix should be left “as is”, lines with - should be removed and lines with + should be added.

Prerequisites

• Docker engine - please have a valid docker engine installed that supports multi-stage builds (i.e., Docker 17.05 or higher) and assure the Docker daemon is running.

# Verify docker version
$ docker --version
Docker version 18.09.3

# Verify docker is running
$ docker ps

# The result should be a valid response listing running processes

Setup the development directory

So let’s start to create our own actionloop-demo-ruby-2.6 runtime. First, check out the devtools repository to access the starter kit, then move it in your home directory to work on it.

git clone https://github.com/apache/openwhisk-devtools
mv openwhisk-devtools/actionloop-starter-kit ~/actionloop-demo-ruby-v2.6

Now, take the directory python3.7 and rename it to ruby2.6 and use sed to fix the directory name references in the Gradle build files.

cd ~/actionloop-demo-ruby-v2.6
mv python3.7 ruby2.6
sed -i.bak -e 's/python3.7/ruby2.6/' settings.gradle
sed -i.bak -e 's/actionloop-demo-python-v3.7/actionloop-demo-ruby-v2.6/' ruby2.6/build.gradle

Let’s check everything is fine building the image.

# building the image
$ ./gradlew distDocker
# ... intermediate output omitted ...
BUILD SUCCESSFUL in 1s
2 actionable tasks: 2 executed
# checking the image is available
$ docker images actionloop-demo-ruby-v2.6
REPOSITORY                  TAG                 IMAGE ID            CREATED             SIZE
actionloop-demo-ruby-v2.6   latest              df3e77c9cd8f        2 minutes ago          94.3MB

At this point, we have built a new image named actionloop-demo-ruby-v2.6. However, despite having Ruby in the name, internally it still is a Python language runtime which we will need to change to one supporting Ruby as we continue in this tutorial.

Preparing the Docker environment

Our language runtime’s Dockerfile has the task of preparing an environment for executing OpenWhisk and OpenServerless Actions. Using the ActionLoop approach, we use a multistage Docker build to

1. derive our OpenWhisk and OpenServerless language runtime from an existing Docker image that has all the target language’s tools and libraries for running functions authored in that language.

   • In our case, we will reference the ruby:2.6.2-alpine3.9 image from the Official Docker Images for Ruby on Docker Hub.

2. leverage the existing openwhisk/actionlooop-v2 image on Docker Hub from which we will “extract” the ActionLoop proxy (i.e. copy /bin/proxy binary) our runtime will use to process Activation requests from the OpenWhisk and OpenServerless platform and execute Actions by using the language’s tools and libraries from step #1.

Repurpose the renamed Python Dockerfile for Ruby builds

Let’s edit the ruby2.6/Dockerfile to use the official Ruby image on Docker Hub as our base image, instead of a Python image, and add our our Ruby launcher script:

 FROM openwhisk/actionloop-v2:latest as builder
-FROM python:3.7-alpine
+FROM ruby:2.6.2-alpine3.9
 RUN mkdir -p /proxy/bin /proxy/lib /proxy/action
 WORKDIR /proxy
 COPY --from=builder /bin/proxy /bin/proxy
-ADD lib/launcher.py /proxy/lib/launcher.py
+ADD lib/launcher.rb /proxy/lib/launcher.rb
 ADD bin/compile /proxy/bin/compile
+RUN apk update && apk add python3
 ENV OW_COMPILER=/proxy/bin/compile
 ENTRYPOINT ["/bin/proxy"]

Next, let’s rename the launcher.py (a Python script) to one that indicates it is a Ruby script named launcher.rb.

mv ruby2.6/lib/launcher.py ruby2.6/lib/launcher.rb

Note that:

1. You changed the base Docker image to use a Ruby language image.

2. You changed the launcher script from Python to Ruby.

3. We had to add a python3 package to our Ruby image since our compile script will be written in Python for this tutorial. Of course, you may choose to rewrite the compile script in Ruby if you wish to as your own exercise.

Implementing the ActionLoop protocol

This section will take you through how to convert the contents of launcher.rb (formerly launcher.py) to the target Ruby programming language and implement the ActionLoop protocol.

What the launcher needs to do

Let’s recap the steps the launcher must accomplish to implement the ActionLoop protocol :

1. import the Action function’s main method for execution.

   • Note: the compile script will make the function available to the launcher.

2. open the system’s file descriptor 3 which will be used to output the functions response.

3. read the system’s standard input, stdin, line-by-line. Each line is parsed as a JSON string and produces a JSON object (not an array nor a scalar) to be passed as the input arg to the function.

   • Note: within the JSON object, the value key contains the user parameter data to be passed to your functions. All the other keys are made available as process environment variables to the function; these need to be uppercased and prefixed with "__OW_".

4. invoke the main function with the JSON object payload.

5. encode the result of the function in JSON (ensuring it is only one line and it is terminated with one newline) and write it to file descriptor 3.

6. Once the function returns the result, flush the contents of stdout, stderr and file descriptor 3 (FD 3).

7. Finally, include the above steps in a loop so that it continually looks for Activations. That’s it.

Converting launcher script to Ruby

Now, let’s look at the protocol described above, codified within the launcher script launcher.rb, and work to convert its contents from Python to Ruby.

Import the function code

Skipping the first few library import statements within launcer.rb, which we will have to resolve later after we determine which ones Ruby may need, we see the first significant line of code importing the actual Action function.

# now import the action as process input/output
from main__ import main as main

In Ruby, this can be rewritten as:

# requiring user's action code
require "./main__"

Note that you are free to decide the path and filename for the function’s source code. In our examples, we chose a base filename that includes the word "main" (since it is OpenWhisk and OpenServerless default function name) and append two underscores to better assure uniqueness.

Open File Descriptor (FD) 3 for function results output

The ActionLoop proxy expects to read the results of invoking the Action function from File Descriptor (FD) 3.

The existing Python:

out = fdopen(3, "wb")

would be rewritten in Ruby as:

out = IO.new(3)

Process Action’s arguments from STDIN

Each time the function is invoked via an HTTP request, the ActionLoop proxy passes the message contents to the launcher via STDIN. The launcher must read STDIN line-by-line and parse it as JSON.

The launcher’s existing Python code reads STDIN line-by-line as follows:

while True:
  line = stdin.readline()
  if not line: break
  # ...continue...

would be translated to Ruby as follows:

while true
  # JSON arguments get passed via STDIN
  line = STDIN.gets()
  break unless line
  # ...continue...
end

Each line is parsed in JSON, where the payload is extracted from contents of the "value" key. Other keys and their values are as uppercased, "__OW_" prefixed environment variables:

The existing Python code for this is:

  # ... continuing ...
  args = json.loads(line)
  payload = {}
  for key in args:
    if key == "value":
      payload = args["value"]
    else:
      os.environ["__OW_%s" % key.upper()]= args[key]
  # ... continue ...

would be translated to Ruby:

  # ... continuing ...
  args = JSON.parse(line)
  payload = {}
  args.each do |key, value|
    if key == "value"
      payload = value
    else
      # set environment variables for other keys
      ENV["__OW_#{key.upcase}"] = value
    end
  end
  # ... continue ...

Invoking the Action function

We are now at the point of invoking the Action function and producing its result. Note we must also capture exceptions and produce an {"error": <result> } if anything goes wrong during execution.

The existing Python code for this is:

  # ... continuing ...
  res = {}
  try:
    res = main(payload)
  except Exception as ex:
    print(traceback.format_exc(), file=stderr)
    res = {"error": str(ex)}
  # ... continue ...

would be translated to Ruby:

  # ... continuing ...
  res = {}
  begin
    res = main(payload)
  rescue Exception => e
    puts "exception: #{e}"
    res ["error"] = "#{e}"
  end
  # ... continue ...

Finalize File Descriptor (FD) 3, STDOUT and STDERR

Finally, we need to write the function’s result to File Descriptor (FD) 3 and “flush” standard out (stdout), standard error (stderr) and FD 3.

The existing Python code for this is:

  out.write(json.dumps(res, ensure_ascii=False).encode('utf-8'))
  out.write(b'\n')
  stdout.flush()
  stderr.flush()
  out.flush()

would be translated to Ruby:

  STDOUT.flush()
  STDERR.flush()
  out.puts(res.to_json)
  out.flush()

Congratulations! You just completed your ActionLoop request handler.

Writing the compilation script

Now, we need to write the compilation script. It is basically a script that will prepare the uploaded sources for execution, adding the launcher code and generate the final executable.

For interpreted languages, the compilation script will only “prepare” the sources for execution. The executable is simply a shell script to invoke the interpreter.

For compiled languages, like Go it will actually invoke a compiler in order to produce the final executable. There are also cases like Java where we still need to execute the compilation step that produces intermediate code, but the executable is just a shell script that will launch the Java runtime.

How the ActionLoop proxy handles action uploads

The OpenWhisk and OpenServerless user can upload actions with the ops Command Line Interface (CLI) tool as a single file.

This single file can be:

• a source file

• an executable file

• a ZIP file containing sources

• a ZIP file containing an executable and other support files

Important: an executable for ActionLoop is either a Linux binary (an ELF executable) or a script. A script is, using Linux conventions, is anything starting with #!. The first line is interpreted as the command to use to launch the script: #!/bin/bash, #!/usr/bin/python etc.

The ActionLoop proxy accepts any file, prepares a work folder, with two folders in it named "src" and "bin". Then it detects the format of the uploaded file. For each case, the behavior is different.

• If the uploaded file is an executable, it is stored as bin/exec and executed.

• If the uploaded file is not an executable and not a zip file, it is stored as src/exec then the compilation script is invoked.

• If the uploaded file is a zip file, it is unzipped in the src folder, then the src/exec file is checked.

• If it exists and it is an executable, the folder src is renamed to bin and then again the bin/exec is executed.

• If the src/exec is missing or is not an executable, then the compiler script is invoked.

Compiling an action in source format

The compilation script is invoked only when the upload contains sources. According to the description in the past paragraph, if the upload is a single file, we can expect the file is in src/exec, without any prefix. Otherwise, sources are spread the src folder and it is the task of the compiler script to find the sources. A runtime may impose that when a zip file is uploaded, then there should be a fixed file with the main function. For example, the Python runtime expects the file __main__.py. However, it is not a rule: the Go runtime does not require any specific file as it compiles everything. It only requires a function with the name specified.

The compiler script goal is ultimately to leave in bin/exec an executable (implementing the ActionLoop protocol) that the proxy can launch. Also, if the executable is not standalone, other files must be stored in this folder, since the proxy can also zip all of them and send to the user when using the pre-compilation feature.

The compilation script is a script pointed by the OW_COMPILER environment variable (you may have noticed it in the Dockerfile) that will be invoked with 3 parameters:

1. <main> is the name of the main function specified by the user on the ops command line

2. <src> is the absolute directory with the sources already unzipped

3. an empty <bin> directory where we are expected to place our final executables

Note that both the <src> and <bin> are disposable, so we can do things like removing the <bin> folder and rename the <src>.

Since the user generally only sends a function specified by the <main> parameter, we have to add the launcher we wrote and adapt it to execute the function.

Implementing the compile for Ruby

This is the algorithm that the compile script in the kit follows for Python:

1. if there is a <src>/exec it must rename to the main file; I use the name main__.py

2. if there is a <src>/__main__.py it will rename to the main file main__.py

3. copy the launcher.py to exec__.py, replacing the main(arg) with <main>(arg); this file imports the main__.py and invokes the function <main>

4. add a launcher script <src>/exec

5. finally it removes the <bin> folder and rename <src> to <bin>

We can adapt this algorithm easily to Ruby with just a few changes.

The script defines the functions sources and build then starts the execution, at the end of the script.

Start from the end of the script, where the script collect parameters from the command line. Instead of launcher.py, use launcher.rb:

- launcher = "%s/lib/launcher.py" % dirname(dirname(sys.argv[0]))
+ launcher = "%s/lib/launcher.rb" % dirname(dirname(sys.argv[0]))

Then the script invokes the source function. This function renames the exec file to main__.py, you will rename it instead to main__.rb:

- copy_replace(src_file, "%s/main__.py" % src_dir)
+ copy_replace(src_file, "%s/main__.rb" % src_dir)

If instead there is a __main__.py the function will rename to main__.py (the launcher invokes this file always). The Ruby runtime will use a main.rb as starting point. So the next change is:

- # move __main__ in the right place if it exists
- src_file = "%s/__main__.py" % src_dir
+ # move main.rb in the right place if it exists
+ src_file = "%s/main.rb" % src_dir

Now, the source function copies the launcher as exec__.py, replacing the line from main__ import main as main (invoking the main function) with from main__ import <main> as main. In Ruby you may want to replace the line res = main(payload) with res = <main>(payload). In code it is:

- copy_replace(launcher, "%s/exec__.py" % src_dir,
-   "from main__ import main as main",
-    "from main__ import %s as main" % main )
+ copy_replace(launcher, "%s/exec__.rb" % src_dir,
+    "res = main(payload)",
+     "res = %s(payload)" % main )

We are almost done. We just need the startup script that instead of invoking python will invoke Ruby. So in the build function do this change:

 write_file("%s/exec" % tgt_dir, """#!/bin/sh
 cd "$(dirname $0)"
-exec /usr/local/bin/python exec__.py
+exec ruby exec__.rb
 """)

For an interpreted language that is all. We move the src folder in the bin. For a compiled language instead, we may want to actually invoke the compiler to produce the executable.

Debugging

Now that we have completed both the launcher and compile scripts, it is time to test them.

Here we will learn how to:

1. enter in a test environment

2. simple smoke tests to check things work

3. writing the validation tests

4. testing the image in an actual OpenWhisk and OpenServerless environment

Entering in the test environment

In the starter kit, there is a Makefile that can help with our development efforts.

We can build the Dockerfile using the provided Makefile. Since it has a reference to the image we are building, let’s change it:

sed -i.bak -e 's/actionloop-demo-python-v3.7/actionloop-demo-ruby-v2.6/' ruby2.6/Makefile

We should be now able to build the image and enter in it with make debug. It will rebuild the image for us and put us into a shell so we can enter access the image environment for testing and debugging:

$ cd ruby2.6
$ make debug
# results omitted for brevity ...

Let’s start with a couple of notes about this test environment.

First, use --entrypoint=/bin/sh when starting the image to have a shell available at our image entrypoint. Generally, this is true by default; however, in some stripped down base images a shell may not be available.

Second, the /proxy folder is mounted in our local directory, so that we can edit the bin/compile and the lib/launcher.rb using our editor outside the Docker image

NOTE It is not necessary to rebuild the Docker image with every change when using make debug since directories and environment variables used by the proxy indicate where the code outside the Docker container is located.

Once at the shell prompt that we will use for development, we will have to start and stop the proxy. The shell will help us to inspect what happened inside the container.

A simple smoke test

It is time to test. Let’s write a very simple test first, converting the example\hello.py in example\hello.rb to appear as follows:

def hello(args)
  name = args["name"] || "stranger"
  greeting = "Hello #{name}!"
  puts greeting
  { "greeting" => greeting }
end

Now change into the ruby2.6 subdirectory of our runtime project and in one terminal type:

$ cd <projectdir>/ruby2.6
$ make debug
# results omitted for brevity ...
# (you should see a shell prompt of your image)
$ /bin/proxy -debug
2019/04/08 07:47:36 OpenWhisk and OpenServerless ActionLoop Proxy 2: starting

Now the runtime is started in debug mode, listening on port 8080, and ready to accept Action deployments.

Open another terminal (while leaving the first one running the proxy) and go into the top-level directory of our project to test the Action by executing an init and then a couple of run requests using the tools/invoke.py test script.

These steps should look something like this in the second terminal:

$ cd <projectdir>
$ python tools/invoke.py init hello example/hello.rb
{"ok":true}
$ python tools/invoke.py run '{}'
{"greeting":"Hello stranger!"}
$ python tools/invoke.py run  '{"name":"Mike"}'
{"greeting":"Hello Mike!"}

We should also see debug output from the first terminal running the proxy (with the debug flag) which should have successfully processed the init and run requests above.

The proxy’s debug output should appear something like:

/proxy # /bin/proxy -debug
2019/04/08 07:54:57 OpenWhisk and OpenServerless ActionLoop Proxy 2: starting
2019/04/08 07:58:00 compiler: /proxy/bin/compile
2019/04/08 07:58:00 it is source code
2019/04/08 07:58:00 compiling: ./action/16/src/exec main: hello
2019/04/08 07:58:00 compiling: /proxy/bin/compile hello action/16/src action/16/bin
2019/04/08 07:58:00 compiler out: , <nil>
2019/04/08 07:58:00 env: [__OW_API_HOST=]
2019/04/08 07:58:00 starting ./action/16/bin/exec
2019/04/08 07:58:00 Start:
2019/04/08 07:58:00 pid: 13
2019/04/08 07:58:24 done reading 13 bytes
Hello stranger!
XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
2019/04/08 07:58:24 received::{"greeting":"Hello stranger!"}
2019/04/08 07:58:54 done reading 27 bytes
Hello Mike!
XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
2019/04/08 07:58:54 received::{"greeting":"Hello Mike!"}

Hints and tips for debugging

Of course, it is very possible something went wrong. Here a few debugging suggestions:

The ActionLoop runtime (proxy) can only be initialized once using the init command from the invoke.py script. If we need to re-initialize the runtime, we need to stop the runtime (i.e., with Control-C) and restart it.

We can also check what is in the action folder. The proxy creates a numbered folder under action and then a src and bin folder.

For example, using a terminal window, we would would see a directory and file structure created by a single action:

$ find
action/
action/1
action/1/bin
action/1/bin/exec__.rb
action/1/bin/exec
action/1/bin/main__.rb

Note that the exec starter, exec__.rb launcher and main__.rb action code are have all been copied under a directory numbered`1`.

In addition, we can try to run the action directly and see if it behaves properly:

$ cd action/1/bin
$ ./exec 3>&1
$ {"value":{"name":"Mike"}}
Hello Mike!
{"greeting":"Hello Mike!"}

Note we redirected the file descriptor 3 in stdout to check what is happening, and note that logs appear in stdout too.

Also, we can test the compiler invoking it directly.

First let’s prepare the environment as it appears when we just uploaded the action:

$ cd /proxy
$ mkdir -p action/2/src action/2/bin
$ cp action/1/bin/main__.rb action/2/src/exec
$ find action/2
action/2
action/2/bin
action/2/src
action/2/src/exec

Now compile and examine the results again:

$ /proxy/bin/compile main action/2/src action/2/bin
$ find action/2
action/2/
action/2/bin
action/2/bin/exec__.rb
action/2/bin/exec
action/2/bin/main__.rb

Testing

If we have reached this point in the tutorial, the runtime is able to run and execute a simple test action. Now we need to validate the runtime against a set of mandatory tests both locally and within an OpenWhisk and OpenServerless staging environment. Additionally, we should author and automate additional tests for language specific features and styles.

The starter kit includes two handy makefiles that we can leverage for some additional tests. In the next sections, we will show how to update them for testing our Ruby runtime.

Testing multi-file Actions

So far we tested a only an Action comprised of a single file. We should also test multi-file Actions (i.e., those with relative imports) sent to the runtime in both source and binary formats.

First, let’s try a multi-file Action by creating a Ruby Action script named example/main.rb that invokes our hello.rb as follows:

require "./hello"
def main(args)
    hello(args)
end

Within the example/Makefile makefile:

• update the name of the image to ruby-v2.6" as well as the name of the main action.

• update the PREFIX with your DockerHub username.

-IMG=actionloop-demo-python-v3.7:latest
-ACT=hello-demo-python
-PREFIX=docker.io/openwhisk
+IMG=actionloop-demo-ruby-v2.6:latest
+ACT=hello-demo-ruby
+PREFIX=docker.io/<docker username>

Now, we are ready to test the various cases. Again, start the runtime proxy in debug mode:

cd ruby2.6
make debug
/bin/proxy -debug

On another terminal, try to deploy a single file:

$ make test-single
python ../tools/invoke.py init hello ../example/hello.rb
{"ok":true}
python ../tools/invoke.py run '{}'
{"greeting":"Hello stranger!"}
python ../tools/invoke.py run '{"name":"Mike"}'
{"greeting":"Hello Mike!"}

Now, stop and restart the proxy and try to send a ZIP file with the sources:

$ make test-src-zip
zip src.zip main.rb hello.rb
  adding: main.rb (deflated 42%)
  adding: hello.rb (deflated 42%)
python ../tools/invoke.py init ../example/src.zip
{"ok":true}
python ../tools/invoke.py run '{}'
{"greeting":"Hello stranger!"}
python ../tools/invoke.py run '{"name":"Mike"}'
{"greeting":"Hello Mike!"}

Finally, test the pre-compilation: the runtime builds a zip file with the sources ready to be deployed. Again, stop and restart the proxy then:

$ make test-bin-zip
docker run -i actionloop-demo-ruby-v2.6:latest -compile main <src.zip >bin.zip
python ../tools/invoke.py init ../example/bin.zip
{"ok":true}

python ../tools/invoke.py run '{}'
{"greeting":"Hello stranger!"}

python ../tools/invoke.py run '{"name":"Mike"}'
{"greeting":"Hello Mike!"}

Congratulations! The runtime works locally! Time to test it on the public cloud. So as the last step before moving forward, let’s push the image to Docker Hub with make push.

Testing on OpenWhisk and OpenServerless

To run this test you need to configure access to OpenWhisk and OpenServerless with ops. A simple way is to get access is to register a free account in the IBM Cloud but this works also with our own deployment of OpenWhisk and OpenServerless.

Edit the Makefile as we did previously:

IMG=actionloop-demo-ruby-v2.6:latest
ACT=hello-demo-ruby
PREFIX=docker.io/<docker username>

Also, change any reference to hello.py and main.py to hello.rb and main.rb.

Once this is done, we can re-run the tests we executed locally on “the real thing”.

Test single:

$ make test-single
ops action update hello-demo-ruby hello.rb --docker docker.io/linus/actionloop-demo-ruby-v2.6:latest --main hello
ok: updated action hello-demo-ruby
ops action invoke hello-demo-ruby -r
{
    "greeting": "Hello stranger!"
}
ops action invoke hello-demo-ruby -p name Mike -r
{
    "greeting": "Hello Mike!"
}

Test source zip:

$ make test-src-zip
zip src.zip main.rb hello.rb
  adding: main.rb (deflated 42%)
  adding: hello.rb (deflated 42%)
ops action update hello-demo-ruby src.zip --docker docker.io/linus/actionloop-demo-ruby-v2.6:latest
ok: updated action hello-demo-ruby
ops action invoke hello-demo-ruby -r
{
    "greeting": "Hello stranger!"
}
ops action invoke hello-demo-ruby -p name Mike -r
{
    "greeting": "Hello Mike!"
}

Test binary ZIP:

$ make test-bin-zip
docker run -i actionloop-demo-ruby-v2.6:latest -compile main <src.zip >bin.zip
ops action update hello-demo-ruby bin.zip --docker docker.io/actionloop/actionloop-demo-ruby-v2.6:latest
ok: updated action hello-demo-ruby
ops action invoke hello-demo-ruby -r
{
    "greeting": "Hello stranger!"
}
ops action invoke hello-demo-ruby -p name Mike -r
{
    "greeting": "Hello Mike!"
}

Congratulations! Your runtime works also in the real world.

Writing the validation tests

Before you can submit your runtime you should ensure your runtime pass the validation tests.

Under tests/src/test/scala/runtime/actionContainers/ActionLoopPythonBasicTests.scala there is the template for the test.

Rename to tests/src/test/scala/runtime/actionContainers/ActionLoopRubyBasicTests.scala, change internally the class name to class ActionLoopRubyBasicTests and implement the following test cases:

• testNotReturningJson

• testUnicode

• testEnv

• testInitCannotBeCalledMoreThanOnce

• testEntryPointOtherThanMain

• testLargeInput

You should convert Python code to Ruby code. We do not do go into the details of each test, as they are pretty simple and obvious. You can check the source code for the real test here.

You can verify tests are running properly with:

$ ./gradlew test

Starting a Gradle Daemon, 1 busy Daemon could not be reused, use --status for details

> Task :tests:test

runtime.actionContainers.ActionLoopPythoRubyTests > runtime proxy should handle initialization with no code PASSED

runtime.actionContainers.ActionLoopPythoRubyTests > runtime proxy should handle initialization with no content PASSED

runtime.actionContainers.ActionLoopPythoRubyTests > runtime proxy should run and report an error for function not returning a json object PASSED

runtime.actionContainers.ActionLoopPythoRubyTests > runtime proxy should fail to initialize a second time PASSED

runtime.actionContainers.ActionLoopPythoRubyTests > runtime proxy should invoke non-standard entry point PASSED

runtime.actionContainers.ActionLoopPythoRubyTests > runtime proxy should echo arguments and print message to stdout/stderr PASSED

runtime.actionContainers.ActionLoopPythoRubyTests > runtime proxy should handle unicode in source, input params, logs, and result PASSED

runtime.actionContainers.ActionLoopPythoRubyTests > runtime proxy should confirm expected environment variables PASSED

runtime.actionContainers.ActionLoopPythoRubyTests > runtime proxy should echo a large input PASSED

BUILD SUCCESSFUL in 55s

Big congratulations are in order having reached this point successfully. At this point, our runtime should be ready to run on any OpenWhisk and OpenServerless platform and also can be submitted for consideration to be included in the Apache OpenWhisk and OpenServerless project.