Use Webhooks

Now that you know how webhooks work, you can now manage Webhooks through the webhooks API.

Prerequisites

To properly configure a webhook, you need one webhook URL with the following:

• Available on the public internet
• Valid SSL certificate that is served over https
• Supports POST calls from Shipwell
• Attaches to the software or server from which you parse or transform Shipwell data

Webhook Configuration Concepts

A webhook configuration is an instance of webhook configuration details such as:

• url: Your publicly accessible https webhook handler URL endpoint with a valid SSL certificate
• enabled_events: A list of the Shipwell events for which you want the webhook to receive
• error_contact_email: An email address to contact if your webhook fails to receive every payload 5 times
• event_version: The version of the event type configured to your webhook. Refer to Types of events to see the event version. Event types may have different versions. You may have to create multiple webhook configurations for the same event type if you want to the same webhook URL to listen for events with different versions. We recommend using one event type per webhook endpoint URL (even if this URL is used across different webhook configurations). If you choose not to indicate an event version, the event payload will default to the latest version.
• custom_data: Optional key-value string pairs that will be stored with the webhook configuration and added to each webhook payloads triggered for the webhook configuration

We recommend that you create webhook configurations in a manner that makes maintenance and deployment processes for your webhook handlers easy. You may have multiple webhook configurations per environment in your account.

Here is a diagram that shows approaches to webhook configurations and how events interact with different webhook configurations (click for a larger version of the image):

Security & Verification

• SSL Certificates and HTTPS
  • Ensure that your webhook URLs have a valid SSL certificate and begin with https:// as the URL scheme.
• IP Addresses and Verifying Webhook Requests/Payloads
  • Webhooks are delivered from dynamic IP addresses and not delivered from a static list of IPs.
  • The verification of the payloads of webhooks as being from Shipwell and valid may be performed with this cryptographically secure verification method.

Add the webhook URL

curl --request POST \
  --url https://sandbox-api.shipwell.com/webhooks/ \
  --header 'accept: application/json' \
  --header 'content-type: application/json' \
  --data '{"enabled_events":["shipment.created","shipment.updated"]
,"status":"ENABLED","include_updated_object":false
,"event_version":1, "error_contact_email":"test@shipwell.com"
,"url":"https://webhook.site/f4b4f289-9a4f-41ba-a0cb-47f72fb12b53"
,"custom_data":{"your_custom_key1":"your_custom_value1"}}'

import requests

base_path = ""
host_path = "sandbox-api.shipwell.com"
target_url = (
    "https://"
    + host_path
    + base_path
    + "/webhooks/"
)

headers = {
    "Authorization": "YOUR_AUTHORIZATION_HEADER",
    "Content-Type": "application/json",
}

payload = {
    'enabled_events': [
        'shipment.created',
        'shipment.updated'
    ],
    'status': 'ENABLED',
    'include_updated_object': False,
    'event_version': 1,
    'error_contact_email': 'test@shipwell.com',
    'url': 'https://webhook.site/f4b4f289-9a4f-41ba-a0cb-47f72fb12b53',
    'custom_data':
        {
            'your_custom_key1':'your_custom_value1',
        }
}

response = requests.post(target_url, headers=headers, json=payload)
data = response.json()
print(data)

Update the webhook URL

If details about your webhook URL need to be updated, you may change the webhook configuration or webhook URL.

We recommend updating webhooks using HTTP PATCH which accepts a partial payload of only the elements that you wish to change or update.

The following example shows updating the following data for an existing webhook configuration:

• enabled_events
  • The list of events that the webhook URL will receive
• error_contact_email
  • The optional email address that receives notification of errors

Partial PATCH Update

The following is an example of utilizing a HTTP PATCH request to partially update the details of an existing webhook configuration.

curl -i -X PATCH \
  'https://sandbox-api.shipwell.com/webhooks/{webhookId}' \
  -H 'Authorization: YOUR_AUTHORIZATION_HEADER' \
  -H 'Content-Type: application/json' \
  -d '{
    "enabled_events": [
        "shipment.created"
    ],
    "error_contact_email": "error-distribution-list@example.com"
}'

import requests

webhook_id = "YOUR_webhookId_PARAMETER"

base_path = ""
host = "sandbox-api.shipwell.com"
target_url = "https://" + host + base_path + "/webhooks/" + webhook_id

headers = {
    "Content-Type": "application/json",
    "Authorization": "YOUR_AUTHORIZATION_HEADER",
}

payload = {
    "enabled_events": [
        "shipment.created"
    ],
    "error_contact_email": "error-distribution-list@example.com"
}
response = requests.patch(target_url, headers=headers, json=payload)
data = response.json()
print(data)

Full PUT Update

The following is an example of a full HTTP PUT request for updating the full details of an existing webhook configuration.

curl -i -X PUT \
  'https://sandbox-api.shipwell.com/webhooks/{webhookId}' \
  -H 'Authorization: YOUR_AUTHORIZATION_HEADER' \
  -H 'Content-Type: application/json' \
  -d '{
      "id": "{webhookId}",
      "resource_type": "webhook",
      "created_at": "2024-07-18T14:15:22Z",
      "enabled_events": [
        "shipment.created",
        "shipment.updated"
      ],
      "error_contact_email": "user@example.com",
      "username_password_authentication": {
        "username": "johndoe",
        "password": "Yga90E)a23z!"
      },
      "status": "ENABLED",
      "include_updated_object": false,
      "url": "https://api.yourcompany.com/webhook_url",
      "event_version": 1,
      "custom_data": {
        "your_custom_key1": "your_custom_value1",
        "your_custom_key2": "your_custom_value2"
      }
}'

import requests

webhook_id = "YOUR_webhookId_PARAMETER"

base_path = ""
host = "sandbox-api.shipwell.com"
target_url = "https://" + host + base_path + "/webhooks/" + webhook_id

headers = {
    "Content-Type": "application/json",
    "Authorization": "YOUR_AUTHORIZATION_HEADER",
}

payload = {
  "id": f"{webhook_id}",
  "resource_type": "webhook",
  "created_at": "2014-07-18T14:15:22Z",
  "enabled_events": [
    "shipment.created",
    "shipment.updated"
  ],
  "error_contact_email": "user@example.com",
  "username_password_authentication": {
    "username": "johndoe",
    "password": "Yga90E)a23z!"
  },
  "status": "ENABLED",
  "include_updated_object": False,
  "url": "https://api.yourcompany.com/webhook_url",
  "event_version": 1,
  "custom_data":
    {
        "your_custom_key1":"your_custom_value1",
        "your_custom_key2":"your_custom_value2"
    }
}

response = requests.put(target_url, headers=headers, json=payload)
data = response.json()
print(data)

Basic Auth Support

curl -i -X PATCH \
  'https://sandbox-api.shipwell.com/webhooks/{webhookId}' \
  -H 'Authorization: YOUR_AUTHORIZATION_HEADER' \
  -H 'Content-Type: application/json' \
  -d '{
    "username_password_authentication": {
        "username": "johndoe",
        "password": "Yga90E)a23z!"
    }
}'

import requests

webhook_id = "YOUR_webhookId_PARAMETER"

base_path = ""
host = "sandbox-api.shipwell.com"
target_url = "https://" + host + base_path + "/webhooks/" + webhook_id

headers = {
    "Content-Type": "application/json",
    "Authorization": "YOUR_AUTHORIZATION_HEADER",
}

payload = {
    "username_password_authentication": {
        "username": "johndoe",
        "password": "Yga90E)a23z!"
    }
}
response = requests.patch(target_url, headers=headers, json=payload)
data = response.json()
print(data)

Custom Data Support

• For example, if you specify "abc_webhook_environment": "development-west-2" for a particular webhook configuration, then every webhook payload for that configuration and delivered to the url will also contain that data in the custom_data section of the webhook payload (it is echoed into the payload).

curl -i -X PATCH \
  'https://sandbox-api.shipwell.com/webhooks/{webhookId}' \
  -H 'Authorization: YOUR_AUTHORIZATION_HEADER' \
  -H 'Content-Type: application/json' \
  -d '{
    "custom_data": {
        "your_custom_key1": "your_custom_value1",
        "your_custom_key2": "your_custom_value2"
    }
}'

import requests

webhook_id = "YOUR_webhookId_PARAMETER"

base_path = ""
host = "sandbox-api.shipwell.com"
target_url = "https://" + host + base_path + "/webhooks/" + webhook_id

headers = {
    "Content-Type": "application/json",
    "Authorization": "YOUR_AUTHORIZATION_HEADER",
}

payload = {
    "custom_data": {
        "your_custom_key1": "your_custom_value1",
        "your_custom_key2": "your_custom_value2"
    }
}
response = requests.patch(target_url, headers=headers, json=payload)
data = response.json()
print(data)

Test your webhook

When configuring a webhook through the sandbox environment in Shipwell, you can test your webhook to make sure it receives event payloads. Using the example from "Add the webhook URL".:

Trigger an event

Verify payload

Upon creation of the event payload, your webhook receives two additional custom headers used to verify that the payload was sent to the webhook URL:

• X-Shipwell-Signature - a unique signature generated from the body, X-Shipwell-Timestamp and the Webhook secret
• X-Shipwell-Timestamp - an ISO-8601 timestamp indicating when the signature was generated, used for signature verification

These 2 headers can be used to verify that you are receiving payloads from Shipwell.

To verify a payload:

1. Generate a signature_payload by concatenating:
   1. X-Shipwell-Timestamp header
   2. A semicolon, ;
   3. The JSON payload you received (use the raw request body without formatting)
2. Using the signature_payload, generate a SHA-256 signature using the Webhook's secret as the key.
3. The computed signature should match the X-Shipwell-Signature. Don't worry, sandbox data is not real data for customers, bills, invoices, or shipments or any other objects in your account. If your endpoint is working successfully, you’ll see the webhook get executed on your server. Additionally, you can call the Events API and look for the specific event you received.

The following is a flask example of verifying a webhook signature:

import hashlib
import hmac
from flask import Flask, request, abort

app = Flask(__name__)
webhook_secret = ... # the Webhook secret is generated and returned when creating a Webhook

@app.route('/webhook-handler', methods=['POST'])
def webhook_handler():
    if request.method == 'POST':
        received_signature = request.headers.get('X-Shipwell-Signature')
        timestamp = request.headers.get('X-Shipwell-Timestamp')
        if not received_signature and not timestamp:
            abort(400)

        signature_payload = timestamp + ";" + request.get_data() # need the raw String to compute the signature
        signature = hmac.new(key=webhook_secret.encode(),
                             msg=signature_payload.encode(),
                             digestmod=hashlib.sha256).hexdigest()

        if signature != received_signature:
            abort(400)

        print(f'Received data from Shipwell! {request.json}')
        # This must return in less than 10 seconds
        return '', 200

    abort(400)


if __name__ == '__main__':
    app.run()

We recommend using a cryptographically secure signing key when configuring your endpoints as well as having all endpoints verify the signature matches the payload to ensure the request is authentically from Shipwell.

Example payload

All events received by your webhook use a custom JSON payload depending on the event type, according to the OpenAPI specification, the following is an example of a generalized event payload:

{
  "id": "545b40eb-1adf-43a2-9159-64190400900b",
  "occurred_at": "2020-04-20",
  "source": {
    "user_id": "fdc6a67b-0533-485c-8e96-ee98d703b8bb",
    "company_id": "f75099e9-a057-46a5-9fc6-490332a110f6",
    "request_id": "919386f2-c8cf-4670-a1ee-1a5e09b6e1f9",
    "publishing_system": "backend",
    "environment": "sandbox"
  },
  "event_name": "shipment.created",
  "custom_data": {
    "your_custom_key1": "your_custom_value1",
    "your_custom_key2": "your_custom_value2"
  },
  "details": {
    // ... details varies by event
  }
}

The Event object schema uses the OpenAPI 3 for the event body:

openapi: 3.0.2
info:
  version: "1.0.0"
  title: Webhook payload
components:

  schemas:
    Source:
      type: object
      required:
        - publishing_system
      description: >-
        Object containing data associated with the source of the event.
      properties:
        user_id:
          type: string
          nullable: true
          description: "The user that originated the event, if applicable."
        company_id:
          type: string
          nullable: true
          description: "The company that originated the event, if applicable."
        request_id:
          type: string
          nullable: true
          description: If the event came from an API request, the ID of that request is null unless system generated.
        publishing_system:
          type: string
          description: "The system name where the event originated"
          example: "shipwell.rating"

    WebhookPayload:
      type: object
      description: |
        What the external endpoint (specified by the Webhook `url`) receives as a `POST` body.
      readOnly: true
      required:
        - id
        - occurred_at
        - source
        - event_name
        - details
      properties:
        id:
          type: string
          description: Unique ID of the event.
        occurred_at:
          type: string
          format: 'date-time'
          description: When the event occurred.
        source:
          allOf:
            - $ref: '#/components/schemas/Source'
            - type: object
              required:
                - environment
              properties:
                environment:
                  type: string
                  description: >-
                    The environment from which the event was generated. Useful in scenarios where the same hook subscribes to events in multiple environments.
                  enum: [dev, sandbox, prod]
        event_name:
          type: string
          description: Which particular event is being received. This will always be a subset of what was specified in enabled_events
        webhook_id:
          type: string
          description: Unique id of the Webhook that prompted delivery
        custom_data:
          type: object
          nullable: true
          description: Custom data registered with the Webhook
        details:
          type: object
          description: The event payload that was subscribed to. The contents differ both on the event name as well as the event_version that was subscribed to
        new_object:
          type: object
          description: If the original Webhook `include_updated_object` was true, this becomes the new object. Otherwise the object does not appear.

Respond to webhook events

Your endpoint must return a 2xx HTTP status code within 10 seconds when it has successfully received an event. This might mean you need to run business logic as a background job if it cannot be completed within the 10-second response window.

Recommended webhook processing approach

The recommended three-step approach to implement for webhook handlers and processing endpoints is to:

1. Verify and save the webhook request payload data in a database, queue, or file system.
2. Respond to the webhook request with an HTTP 200 (or HTTP 202) status code (or an appropriate HTTP status code) within the hard timeout threshold of 10 seconds.
3. Perform additional processing logic using the saved webhook data in the background and out-of-band.

This allows you to acknowledge or ACK the HTTP POST (delivered over HTTPS) containing the webhook data payload, save the data within the timeout threshold, and helps ensure an architecture that scales as you add processing logic and capabilities.

The following diagram illustrates the recommended webhook processing approach (click for a larger version of the image):

Event delivery

Timeouts and retries

When a webhook is triggered, Shipwell queues the event and webhook to be published. If the webhook is unsuccessful in delivering the event (the remote server was unreachable, returned a non-2xx HTTP response, or the took longer than 10s to respond), the webhook handler will attempt to deliver the payload to the webhook again after a 20-minute wait. We'll try to deliver the webhook up to 5 times, doubling the delay between each attempt. If the webhook handler gives up, Shipwell will send an automatic email to the email address submitted with the webhook configuration.

Shipwell will automatically follow all redirects returned by the server; however, more than 30 redirections are considered a failure. Additionally, the status code found to determine if the webhook request succeeded or not is the last one received in the chain of redirections. Shipwell ignores any other information returned in the request headers or request body.

The majority of webhook issues result in 4xx and 5xx HTTP status codes.

• 4xx indicates Shipwell was able to contact your server, but the endpoint was not valid. Double-check that the endpoint URL is correct and ready to receive POST requests.
• 5xx usually indicates an exception on your server.

Timeout threshold

Retry timing

The following is a table of the retry timing calculations. If you experience a delay in your webhook processing, this is often due to the webhook endpoint URL erroring or timing out then the system retrying in N minutes. To debug or investigate errors in webhook delivery and to see the request and response details of the webhook delivery attempt, see Debugging Webhooks.

The following is an example of how retries are attempted for a case where the webhook endpoint errors or has a timeout for 4 out of 5 attempts and then succeeds on the final attempt. If the webhook endpoint URL returns a 200 series (i.e. 200-299) HTTP status code within the 10-second webhook response timeout threshold for any attempt before the 5th and final retry then the webhook delivery will be considered successful without additional retries.

Event ordering

• shipment.created
• purchase_order.created

Handling duplicate events

Shipwell does not guarantee exactly once event delivery. This is common for event delivery systems that support scale and speed and there are common ways to handle if you receive a duplicate event.

• Ensure that your event processing is idempotent
  • You may log or save events processed, after creation/save of an item look that up in a database/cache/data store, etc., and then do not process those events
  • You may utilize the event id, event name, etc. when determining if you have already logged/processed/recorded an event in your system

Debugging webhooks

Sometimes you need to debug or investigate webhook endpoint issue(s) for a new or existing webhook configuration. For example:

• Shipwell may have a delay delivering events to your webhook endpoint
• Your webhook endpoint URL is not the URL you expect and the webhook configuration needs to be updated
• Your webhook endpoint may have an SSL issue
• Your webhook endpoint or server times out
• Your webhook endpoint is not returning a 200 series (i.e. 200-299) HTTP status code
• Your webhook endpoint is not receiving the events that you expect
• Your HTTP basic auth information has changed

Webhook Delivery Attempts API Endpoint

You may view the webhook delivery attempts and responses by querying the Webhook Delivery Attempts API Endpoint.

• The webhook attempts are sorted in descending order by the webhook attempt date/time so the most recent attempts are returned at the top of the response list. You may also page through the results using date/time windows if needed.
• Similar to other Shipwell endpoints that support filtering, the filter criteria is ANDed together and not ORed (i.e. a=1 and b=2).

• GET /webhooks/attempts?response_code.in=400,401,403,404,408,500,501,502,503,504,null
  • Retrieves webhook attempts where the response contains specific HTTP status codes or the endpoint or server timed out
  • Specifying the string null for a response code query indicates no response code (i.e. the server timed out)
• GET /webhooks/attempts?response_code=null
  • Retrieves webhook attempts that timed out where your endpoint or server did not respond within the 10-second threshold
• GET /webhooks/attempts?response_code=null&webhook_config_id=YOUR_WEBHOOK_CONFIG_ID
  • Retrieves webhook attempts that timed out for a particular webhook endpoint or webhook_config_id
• GET /webhooks/attempts?response_code.gte=200&response_code.lte=299
  • Retrieves webhook attempts that were successful with any HTTP 200-series status code (200-299 (server success))

curl --location --request GET 'https://sandbox-api.shipwell.com/webhooks/attempts?response_code.gte=400' \
    --header 'Authorization: YOUR_AUTHORIZATION_HEADER'

import requests

base_path = ""
host = "sandbox-api.shipwell.com"
target_url = "https://" + host + base_path + "/webhooks/attempts"

headers = {"Authorization": "YOUR_AUTHORIZATION_HEADER"}

query_params = {
    "response_code.gte": 400
}

# Specific Error Codes Example:

# response_code_list = [400, 401, 403, 404, 408, 500, 501, 502, 503, 504, "null"]
# response_codes_delimited = ",".join([str(item) for item in response_code_list])
# query_params = {
#     "response_code.in": response_codes_delimited
# }

response = requests.get(target_url, headers=headers, params=query_params)
data = response.json()
print(data)

To view events in your account (in case you need to import events after a webhook handler or endpoint downtime, etc.), utilize the Events API Endpoint

Go live