Google Apps Script - Webhook Processing Template with ScriptLab Logging

Overview

This quickstart template allows you to get started processing Shipwell webhooks quickly using Google Apps Script. The data, processes, and script execution status all integrate with the Shipwell platform and the Shipwell UI. This template allows you to clone/copy to get start quickly and add your own custom logic. The template builds on the concepts from Processing Webhooks with Custom Logic.

Webhook Script Template Organization

The project is organized into multiple files and each file contains related functions. Descriptions of the files are listed below:

• InboundWebhooks
  • Receives the webhook HTTP POST and adds the webhook body/payload to the WebhooksIn queue.
• Main
  • Pulls webhook events from the webhooks queue. Main runs every minute and creates a trigger to schedule the appropriate async process for the webhook event pulled from the queue. It removes the webhook event from the queue and places it in WebhooksProcessed queue. It stops pulling from the webhooks queue when the 20 trigger limit is reached.
• ProcessWebhooks
  • An async process scheduled by Main . The webhook body retrieved from the webhooks queue is passed and this is where code is added for processing the webhook.
• Async
  • Used by Main to schedule an async process.
• ScriptLab
  • Creates a class with methods to interface with Shipwell ScriptLab service which allow script execution monitoring and logging in the Shipwell platform.

Webhook Script Template Process Diagram

Process flow diagram for the Shipwell ScriptLab Template in this example. This utilizes Google Apps Script, allows processing webhooks without standing up your own infrastructure, and it enables monitoring and logging of your webhook processing and script execution from within the Shipwell platform.

Webhook Script Template Concepts

Concepts of how the Shipwell Webhook Processing Template and the scripts work are outlined below. Learn more about webhooks as a concept here.

Event Names

The list of webhook event names/types are available here.

Use of Custom Data

The Shipwell platform enables use of custom_data key-value pairs of data on many objects including webhook configurations. Learn more about custom_data here and more about how custom_data is supported in webhooks here

The Shipwell ScriptLab webhook template uses the following custom_data as a base:

{
    "name": "<user defined>", 
    "token": "<user-token>"
}

• The name custom_data property is used for documentation purposes only and will be used by `ScriptLab when displaying execution status in the Shipwell platform.
• The token custom_data property (learn more here ) prevents hard coding the token in the web app or script directly and uses the custom_data of the webhook configuration

Multi-Environment Support

The source environment where the webhook originated and the token in the custom_data property of the delivered webhook allow the single deployed endpoint or web app to communicate with multiple Shipwell environments. The function below demonstrates how the API token is created and the env (aka environment) URL is constructed for use in subsequent Shipwell API calls.

function setEnvVariables(webhookBody) {
    env = JSON.parse(webhookBody).source.environment;
    token = `Token ` + JSON.parse(webhookBody).custom_data.token;

    if (env == 'dev') {
        env = env + '-';
    }
    if (env == 'sandbox') {
        env = env + '-';
    }
    if (env == 'prod') {
        env = '';
    }
}

Use of Queues

Inbound webhooks (webhooks sent from Shipwell and received "inbound" by your webhook endpoint URL) are processed using the Google Apps Script built-in function doPost() which extracts the request body from the webhook delivery and places it in the WebhooksIn queue. The Shipwell ScriptLab template uses a Google Sheet to implement queues and each queue is a separate tab in the Webhook Capture and Management Template sheet.

Asynchronous Processing

Google Apps Script uses triggers to schedule execution of the deployed web app but there is a limit of only 20 active triggers. The main process will pull webhook bodies/payloads from the WebhooksIn queue until the 20 trigger limit is reached.

Webhook Capture and Management Template Sheet Tab Overview

The provided sheet not only provides queues for processing webhooks, but also includes webhook creation and management features. The sheet has tabs that provide specific functionality which is described below:

Tokens Tab

Used to store API tokens for the environments the user wants to manage.

Webhooks Tab

This tab uses the tokens provided in the Tokens Tab when a user selects an environment to manage from the navigation window. Selecting an environment will refresh the sheet with all the webhooks created for that environment (if the token is valid for that environment). The user can update the status of a specific webhook configuration to DELETE (this will delete the webhook configuration), ENABLED, or DISABLED by selecting the dropdown in the Status column of the sheet.

Create Webhook Tab

This tab is used to create a new webhook for the currently selected environment. The user provides which events are enabled, error email, url, event version, and custom_data. Selecting POST /webhooks/ from the navigation menu will create the webhook.

Webhook Script Template Getting Started Steps

Copy Template Files

Make a copy of the Google Apps Script project and the Google Sheet using the link below:

• WebhookProcessingTemplate
  • https://drive.google.com/drive/folders/15pQvK9VhA5gZKLe5BnBd9KxKSx8IERNT?usp=sharing

Update Apps Script to Use Copied Google Sheet

Copy the ID of the Google sheet (found in the url of the sheet) and paste it into the SpreadsheetApp references in Main.gs around line 9 and InboundWebhooks.gs around line 6.

Deploy Webhook Processing Web App

Select Deploy -> New Deployment and deploy the Google Apps Script as web app.

After deployment, the user may need to grant access to run the script. When deployment completes the user should copy the URL of the deployed web app to their clipboard and to cloud or filesystem storage. The URL will be used as the webhook endpoint that webhook events are sent to from Shipwell.

Create Webhook

For this example, a webhook will be created using the shipment.updated event (see the webhook event types here). In the Google Sheet tab Create Webhook, paste the url of the web app and update the enabled_events column with the event name. Also update the custom_data field with the name and token.

From within the Google Sheet navigation menu, select Shipwell Webhooks -> POST /webhooks/. A notification of a script running indicates that the webhook configuration is being created in the actual Shipwell environment and account associated with the token. After the script execution completes, the newly created webhook configuration may be viewed in the Webhooks Tab.

Trigger Webhook

Login to the Shipwell platform and update the references of an existing shipment so the shipment.updated event will trigger the webhook (any kind of saved updated to shipment will trigger the webhook). The web app will receive the webhook delivery and place the webhook body/payload in the WebhooksIn queue. The webhook body can be viewed in the WebhooksIn tab of the Google Sheet.

Enable Queue Processing

Navigate to the Triggers section in the Apps Script editor and select Add Trigger.

Set the parameters of the trigger to execute the doQueue function and use the version of the web app deployed. Also set the event source to be time-based using a minute timer to execute every minute. Then save the trigger.

Verify Webhook Queue Processed

Return to the Google sheet and note that the WebhooksIn queue is now empty and the WebhooksProcessed queue has the processed webhook data.

View ScriptLab Workflows On Shipment Detail Within Shipwell Platform

The status of the script execution is available in the Scripts section of the Workflows tab of the shipment details UI within the Shipwell platform.

The name provided in the webhook custom_data is displayed here and selecting View Log will show the execution logs of the script.

Adding Your Custom Code and Logic To The Webhook Processing Template

Add your code to the ProcessWebhook.gs file in the processWebhook() function.

In this example of adding custom logic, we're going to use the self_link in the webhook to GET the shipment, update the Pickup # reference data, and then PUT the shipment back.

Replace the code in the file with the code below to perform the update. See Processing Webhooks with Custom Logic - Define Processing Logic for additional details regarding the callShipwellAPI() function.

function processWebhook(webhookBody) {
    let url = 'https://' + env + 'api.shipwell.com' + JSON.parse(webhookBody).details.self_link;
    let shipmentData = callShipwellAPI(url, 'get');
    shipmentData.pickup_number = "12345"
    shipmentData = callShipwellAPI(url, 'put', shipmentData);

    logEntry = {
        "level": "TRACE",
        "timestamp": '',
        "message": "pickup_number updated to " + shipmentData.pickup_number,
        "stack_trace": [
            "doPostAsync",
            "processWebhook"
        ]
    };
    sw.pushLog(logEntry);

    console.info("pickup_number updated to " + shipmentData.pickup_number);

    return;
}

Deploy Changes To Webhook Processing Logic

Save the changes and select Deploy. Copy the url provided after the deploy completes and update the webhook created earlier with the new url. Delete the previously created trigger and create a new one using the newly deployed version of the web app.

While this is a trivial example, it demonstrates the process to call the Shipwell API and update a shipment object. After updating a shipment the user will note that the Pickup # has been updated to 12345 and the script logs show the updated ScriptLab messaging.