File Library

Overview

The library can hold an unlimited number of files, but a maximum of 1GB of files.

File metadata

Files support the following metadata:

Show properties

fileId · uuid

The unique identifier of the file, generated by AI21.

name · string

The name of the file specified by you.

path · string

An arbitrary file-path-like string you can assign to indicate the content of a file.

fileType · string

The type of the file.

sizeBytes · int

The size of the file in bytes.

labels · string[]

The labels associated with the file. You can apply arbitrary string labels to your files and limit queries to files with one or more labels. Similar to paths, but labels do not prefix match. Labels are case-sensitive.

publicUrl · string

The public URL of the file, specified by you. This URL is not validated by AI21 or used in any way. It is strictly a piece of metadata that you can optionally attach to a file.

createdBy · uuid

The identifier of the user who uploaded the file.

creationDate · date

The date when the file was uploaded.

lastUpdated · date

The last update date of the file in the library.

dataSource · string

Where was the file uploaded from.

status · string

The status of the file in the library.

Show supported values

Value	Description
DB_RECORD_CREATED	The file record has been created in the database.
UPLOADED	The file has been uploaded and is being processed.
UPLOAD_FAILED	The file upload has failed.
PROCESSING	The file is currently being processed.
PROCESSED	The file has been processed and is ready for use.
FILE_DOWNLOAD_FAILED	The file download has failed.
PARSING_FAILED	Parsing the file has failed.
SEGMENTATION_FAILED	File segmentation has failed.
EMBEDDING_FAILED	Embedding the file segments has failed.
VECTORS_DB_UPSERT_FAILED	Updating the vector database has failed.
VECTOR_DATA_UPLOAD_TO_STORAGE_FAILED	Uploading the file’s embedding vectors to the storage has failed.
SEGMENTS_UPLOAD_TO_DB_FAILED	Uploading the file’s segments to the database has failed.
PROCESSING_FAILED	The processing of the file has failed.
DELETION_FAILED	The deletion of the file has failed.
RE_PROCESS_CANDIDATE	The file is a candidate for re-processing.
RE_PROCESSING	The file is currently being re-processed.
DELETING	The file is in the process of being deleted.
DELETED_FROM_VECTOR_DB	The file has been deleted from the vector database.
DELETE_REQUEST_DURING_PROCESSING	A delete request was received during processing.
MAX_RETRIES_REACHED	The maximum number of retries has been reached, the operation has failed.

Filtering file results

When you create or update a file, you can optionally apply path and/or label values to a file. You can later search, query, or list files based on matching paths or labels. This can be useful to query subsets of documents.

For example, you can assign a path of either /financial/USA or /financial/UK to certain documents and later query only US financial documents, only UK financial documents, or query all financial documents by specifying /financial/ as the path.

Assistant

Responses are generated using AI and may contain mistakes.