Blob Support

CrateDB includes support to store binary large objects. By utilizing CrateDB’s cluster features the files can be replicated and sharded just like regular data.

Creating a Table for Blobs

Before adding blobs a blob table must be created. Lets use the CrateDB shell crash to issue the SQL statement:

sh$ crash -c "create blob table myblobs clustered into 3 shards with (number_of_replicas=0)"
CREATE OK, 1 row affected (... sec)

Now CrateDB is configured to allow blobs to be management under the /_blobs/myblobs endpoint.

Custom Location for Storing Blob Data

It is possible to define a custom directory path for storing blob data which can be completely different than the normal data path. Best use case for this is storing normal data on a fast SSD and blob data on a large cheap spinning disk.

The custom blob data path can be set either globally by config or while creating a blob table. The path can be either absolute or relative and must be creatable/writable by the user CrateDB is running as. A relative path value is relative to CRATE_HOME.

Blob data will be stored under this path with the following layout:


Global by Config

Just uncomment or add following entry at the CrateDB config in order to define a custom path globally for all blob tables:

blobs.path: /path/to/blob/data

Also see Configuration.

Per Blob Table Setting

It is also possible to define a custom blob data path per table instead of global by config. Also per table setting take precedence over the config setting.

See CREATE BLOB TABLE for details.

Creating a blob table with a custom blob data path:

sh$ crash -c "create blob table myblobs clustered into 3 shards with (blobs_path='/tmp/crate_blob_data')" # doctest: +SKIP
CREATE OK, 1 row affected (... sec)

Altering a Blob Table

The number of replicas a blob table has can be changed using the ALTER BLOB TABLE clause:

sh$ crash -c "alter blob table myblobs set (number_of_replicas=0)"
ALTER OK, 1 row affected (... sec)


To upload a blob the sha1 hash of the blob has to be known upfront since this has to be used as the ID of the new blob. For this example we use a fancy Python one-liner to compute the shasum:

sh$ python -c 'import hashlib;print(hashlib.sha1("contents".encode("utf-8")).hexdigest())'

The blob can now be uploaded by issuing a PUT request:

sh$ curl -isSX PUT '' -d 'contents'
HTTP/1.1 201 Created
content-length: 0

If a blob already exists with the given hash a 409 Conflict is returned:

sh$ curl -isSX PUT '' -d 'contents'
HTTP/1.1 409 Conflict
content-length: 0


To list all blobs inside a blob table a SELECT statement can be used:

sh$ crash -c "select digest, last_modified from blob.myblobs"
| digest                                   | last_modified |
| 4a756ca07e9487f482465a99e8286abc86ba4dc7 | ...           |
SELECT 1 row in set (... sec)


To query blob tables it is necessary to always specify the schema name blob.


To download a blob simply use a GET request:

sh$ curl -sS ''


Since the blobs are sharded throughout the cluster not every node has all the blobs. In case that the GET request has been sent to a node that doesn’t contain the requested file it will respond with a 307 Temporary Redirect which will lead to a node that does contain the file.

If the blob doesn’t exist a 404 Not Found error is returned:

sh$ curl -isS ''
HTTP/1.1 404 Not Found
content-length: 0

To determine if a blob exists without downloading it, a HEAD request can be used:

sh$ curl -sS -I ''
HTTP/1.1 200 OK
content-length: 8
accept-ranges: bytes
expires: Thu, 31 Dec 2037 23:59:59 GMT
cache-control: max-age=315360000


The cache headers for blobs are static and basically allows clients to cache the response forever since the blob is immutable.


To delete a blob simply use a DELETE request:

sh$ curl -isS -XDELETE ''
HTTP/1.1 204 No Content
content-length: 0

If the blob doesn’t exist a 404 Not Found error is returned:

sh$ curl -isS -XDELETE ''
HTTP/1.1 404 Not Found
content-length: 0

Deleting a Blob Table

Blob tables can be deleted similar to normal tables (again using the CrateDB shell here):

sh$ crash -c "drop blob table myblobs"
DROP OK, 1 row affected (... sec)