File Uploads

Shipwell's API endpoints that accept file uploads do so in a consistent manner with support for multipart/form-data uploads.

The API utilizes multipart form data for file uploads since this is type of file upload foundational to the internet, is the most common file upload format for APIs and the web, and it has wide support in all known web-accessible programming languages (i.e. cURL, Python, Java, JavaScript, PHP, etc.).

Multipart Form Data File Uploads

As described in many examples elsewhere on the web, multipart requests combine one or more sets (and potentially types) of data into one request.

The different parts of the data are separated or delimited by a specified boundary marker (i.e. this can be anything, is set in the request, and is often handled in the background by programming languages and libraries without the developer or integrator needing to consider this).

We recommend using libraries to help with your programming and integration (as you are likely already doing) as they make tasks like file uploads simpler and essentially remove the potential errors from reinventing the wheel.

For example, in python's requests library when uploading a file or making an HTTP request with a POST that has a body (aka payload), it will calculate and set the Content-Length header (this is an integer of the size in bytes of a request)

multipart/form-data is most often used for file uploads with accompanying data/metadata and details in one single request, so it is extremely useful as a format and was built to be efficient.

Note

When sending the bytes of a file, ensure that the raw binary data or bytes of the file are sent and that the file data is not base64 encoded. If you send base64 encoded file data as the bytes then you may receive errors similar to, "The submitted data was not a file. Check the encoding type on the form."

Below is an example of what occurs at the HTTP level (which all HTTP requests are in the end) and what the programming languages and libraries are building when you send a multipart request:

Copy

Copied

POST /example/upload-endpoint HTTP/1.1

Content-Length: 1234
Content-Type: multipart/form-data; boundary=---abc12345678

---abc12345678
Content-Disposition: form-data; name="description"

Testing. 123. Testing.
---abc12345678
Content-Disposition: form-data; name="type"

INVOICE
---abc12345678
Content-Disposition: form-data; name="file"; filename="example.pdf"
Content-Type: application/pdf

(content of the uploaded file example.pdf as a string of bytes)
---abc12345678

Below are some examples showing how to perform file uploads simply using a multipart/form-data in various programming languages. If you see \r\n in examples like cURL and you are not familiar, this is the computer equivalent of a new line and a carriage return (yes, like in the days of typewriters).

Upload a File with Multipart Form Data

If a file is located on a computer's file system or accessible via a file share or drive from that computer, then you may send the file directly using various helpers in programming languages.

Since a file is a set of bytes or data, you may also send the bytes of the file or with the knowledge that files are just bytes and data then you may save the bytes of a file (a byte array, byte stream, byte string, etc.) to a filesystem then send the file that way.

Uploading File Via Multipart Form Data

curlpythonjava

Copy

Copied

curl --location 'https://example.com/example/upload-endpoint' \
--header 'Content-Type: multipart/form-data' \
--header 'Authorization: YOUR_API_KEY_OR_AUTH_VALUE_HERE' \
--form 'file=@"/C:/Users/jdoe/temp/example.pdf"' \
--form 'type="INVOICE"' \
--form 'description="Testing 123."'

Copy

Copied

import requests

url = "https://example.com/example/upload-endpoint"
headers = {
    "Authorization": "YOUR_API_KEY_OR_AUTH_VALUE_HERE"
}
payload = {
    "description": "Testing 123.",
    "type": "INVOICE"
}

bytes_of_file_or_file_like_object = open("path/to/file/example.pdf", "rb")
files = [
    ("file", ("example.pdf", bytes_of_file_or_file_like_object, "application/pdf"))
]

response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.json())

Copy

Copied

import java.io.IOException;
import okhttp3.MultipartBody;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;

OkHttpClient client = new OkHttpClient();

File file; // this is your file object
RequestBody requestBody = new MultipartBody.Builder()
    .setType(MultipartBody.FORM)
    .addFormDataPart("description", "Testing 123.")
    .addFormDataPart("file", 
        file.getName(),
        RequestBody.create(MediaType.parse("application/pdf"), file))
    .addFormDataPart("type", "INVOICE")
    .build();

Request request = new Request.Builder()
    .url("https://example.com/example/upload-endpoint")
    .post(requestBody)
    .header("Authorization", "YOUR_API_KEY_OR_AUTH_VALUE_HERE")
    .header("Content-Type", "multipart/form-data")
    .build();

try (Response response = client.newCall(request).execute()) {
    if (!response.isSuccessful()) throw new IOException("Unexpected code " + response);
    response.body().string();
}

Tip

The essence of a file is the bytes or data of the file, so if you have access to the bytes or data of a file you may write or persist that file to a location that you have write permissions to such as the local filesystem of a computer or server (i.e. you can save files on a machine at a particular location). For some use cases, saving file bytes to disk (such as in a temp directory) and sending from there may be easier than sending the bytes directly.

Uploading File Bytes Via Multipart Form Data

Since files are just their bytes and data, they may also be uploaded without direct access to the file or need to re-read bytes of the file if you already have those bytes and data in memory.

Below are examples of sending multipart/form-data data with a file upload using the file's bytes or data as part of the request.

curlpythonjava

Copy

Copied

curl 'https://example.com/example/upload-endpoint ' \
  -H 'authorization: Token {replace_with_token}' \
  -H 'content-type: multipart/form-data; boundary=---abc12345678' \
  --data-raw $'---abc12345678\r\nContent-Disposition: form-data; name="description"\r\n\r\nTesting. 123.\r\n---abc12345678\r\nContent-Disposition: form-data; name="type"\r\n\r\nINVOICE\r\n---abc12345678\r\nContent-Disposition: form-data; name="file"; filename="example.pdf"\r\nContent-Type: application/pdf\r\n\r\n\(content of the uploaded file example.pdf as a string of bytes)\r\n---abc12345678--\r\n' \
  --compressed

Copy

Copied

import requests

url = "https://example.com/example/upload-endpoint"
headers = {
    "Authorization": "YOUR_API_KEY_OR_AUTH_VALUE_HERE"
}
payload = {
    "description": "Testing 123.",
    "type": "INVOICE"
}

# requests handles creating the file boundaries
#   Also see the following if sending bytes: https://stackoverflow.com/a/74389385
# Sending a file or file bytes as part of a multipart form data request
# is the same for python. You may need to base64 decode a byte string 
# from another system, decode a byte string into bytes, etc. depending on
# the format of how you received the bytes of the file  
bytes_of_file_or_file_like_object = open("path/to/file/example.pdf", "rb")
files = [
    ("file", ("example.pdf", bytes_of_file_or_file_like_object, "application/pdf"))
]

response = requests.request("POST", url, headers=headers, data=payload, files=files)
print(response.json())

Copy

Copied

import java.io.IOException;
import okhttp3.MultipartBody;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;

OkHttpClient client = new OkHttpClient();

byte [] fileData;
// byte [] fileData =  FileUtils.readFileToByteArray(file);

RequestBody requestBody = new MultipartBody.Builder()
    .setType(MultipartBody.FORM)
    .addFormDataPart("description", "Testing 123.")
    .addFormDataPart("file", 
        "example.pdf",
        RequestBody.create(MediaType.parse("application/pdf"), fileData))
    .addFormDataPart("type", "INVOICE")
    .build();

Request request = new Request.Builder()
    .url("https://example.com/example/upload-endpoint")
    .post(requestBody)
    .header("Authorization", "YOUR_API_KEY_OR_AUTH_VALUE_HERE")
    .header("Content-Type", "multipart/form-data")
    .build();

try (Response response = client.newCall(request).execute()) {
    if (!response.isSuccessful()) throw new IOException("Unexpected code " + response);
    response.body().string();
}