Configuration Reference

This page describes the customer-configurable fields used in pipeline_config.

Pipeline configuration

Field	Type	Default	Description
`schemas`	array	`[]`	Structured values to extract.
`split`	boolean	`false`	Detect multiple logical documents in an eligible PDF and return separate result slices.
`embed`	boolean	`true`	Generate searchable representations for semantic search.
`describe`	object	Disabled	Configure automatic document descriptions.

Only the fields documented here should be configured by customer applications. Model and provider selection are managed by Shipwell.

Description configuration

Field	Type	Default	Description
`enabled`	boolean	`false`	Generate a natural-language document description.
`prompt`	string or `null`	`null`	Optional instructions for the generated description.
`key_facts`	boolean	`true`	Include a compact set of important facts when description generation is enabled.

Keep description prompts focused on purpose and audience. Put fields that must be consumed programmatically in schemas instead.

Extraction schema

Every item in schemas has the following customer-configurable fields:

Field	Type	Required	Default	Description
`name`	string	Yes	—	Stable key used to identify the result.
`prompt`	string	Yes	—	Instructions describing what to extract.
`json_schema`	object	Yes	—	JSON Schema describing the expected value.
`scope`	`segment` or `document`	No	`segment`	Whether extraction runs for each segment or once for the whole document.
`result_cardinality`	`single` or `multiple`	No	`single`	Whether the schema returns one result or a list of matching results.
`include_source_regions`	boolean	No	`false`	Include normalized source locations when available.

Unknown fields are rejected.

Name

Choose a stable snake_case name such as:

purchase_order_number
invoice_total
line_items
delivery_date

Treat the name as part of your integration contract. Changing it changes the key consumers use to find the extracted result.

Prompt

A good prompt:

Names the exact value to extract
Explains how to handle ambiguity
Defines behavior when the value is absent
Avoids asking for a different shape than json_schema

Example:

Copy

Copied

Extract the bill of lading number from the document header.
Return null when no bill of lading number is present.
Do not return shipment or purchase order numbers.

JSON Schema

Use JSON Schema to define the response type. Prefer the narrowest shape that supports your workflow.

String or null:

Copy

Copied

{
  "type": ["string", "null"]
}

Structured object:

Copy

Copied

{
  "type": ["object", "null"],
  "properties": {
    "amount": {"type": "number"},
    "currency": {"type": "string"}
  },
  "required": ["amount", "currency"],
  "additionalProperties": false
}

List of line items:

Copy

Copied

{
  "type": "array",
  "items": {
    "type": "object",
    "properties": {
      "description": {"type": "string"},
      "quantity": {"type": ["number", "null"]},
      "unit_price": {"type": ["number", "null"]}
    },
    "required": ["description"],
    "additionalProperties": false
  }
}

Scope

Use document when the value should be resolved once from the entire logical document, such as:

Document number
Total amount
Shipper or consignee

Use segment when each independently processed segment can produce its own value. This is especially useful when split processing is enabled.

Result cardinality

Use single when one answer is expected for the selected scope.

Use multiple when several independent matches are expected. Define json_schema for one match; the result contains multiple schema-result entries.

Do not use multiple only because the value itself is an array. A single extracted line_items value can use result_cardinality: "single" with an array-shaped json_schema.

Source regions

Set include_source_regions to true when a user should be able to trace an extracted value to its location.

Regions use normalized coordinates:

Copy

Copied

{
  "left": 0.12,
  "top": 0.41,
  "width": 0.28,
  "height": 0.04
}

Coordinates range from 0 to 1, with the origin at the top-left. Source regions are available for supported PDFs and images. They can be null for text and spreadsheet formats.

Complete example

Copy

Copied

{
  "schemas": [
    {
      "name": "bill_of_lading_number",
      "prompt": "Extract the bill of lading number. Return null when absent.",
      "json_schema": {"type": ["string", "null"]},
      "scope": "document",
      "result_cardinality": "single",
      "include_source_regions": true
    },
    {
      "name": "reference_numbers",
      "prompt": "Extract each labeled reference number.",
      "json_schema": {
        "type": "object",
        "properties": {
          "label": {"type": "string"},
          "value": {"type": "string"}
        },
        "required": ["label", "value"],
        "additionalProperties": false
      },
      "scope": "document",
      "result_cardinality": "multiple",
      "include_source_regions": true
    }
  ],
  "split": false,
  "embed": true,
  "describe": {
    "enabled": true,
    "prompt": "Summarize the document for an operations user.",
    "key_facts": true
  }
}

Validation guidance

Before using a configuration in production:

Test representative layouts from every expected document source.
Test missing, malformed, and repeated values.
Validate result values against json_schema .
Review source regions when enabled.
Version changes in the consuming integration before renaming or reshaping schemas.