Docs Dashboard

JSON Models

Model Description
Category Category of the object. The set of potential categories depends on the model. For instance, with object recognition, a category may be a "chair" or "dog". For visual intent, it may be "food_or_drink".
CategoryList A list of categories that an image matches.
ImageInfo Information about the image. May be omitted if image binary is provided in POST request. Otherwise, either url or imageLookupToken is mandatory.
ImageRequestInfo Base-64 encoded image and optional coordinates of crop area (if specified by user).
Entity Describes an entity.
IUFeature Defines a feature that was detected on the image. For instance, detected OCR or objects.
IUModelResults Output of a model run on the image. The name of the object gives context into the model run, and a list of results is generated for the model.
NormalizedBox Similar to NormalizedRectangle, a box that defines a portion of a larger rectangle. In most cases, NormalizedBox will define a portion of the input image. It contains a set of NormalizedPoints that define a box.
NormalizedPoint An (x,y) coordinate normalized to the size of the image. It has values between 0 and 1. The values are relative to the width and height of the image (or other containing rectangle) respectively. For instance a normalized point defined as (0.3, 0.5) would describe a point that is 30% across the image (from the left) and 50% down the image (from the top).
NormalizedRectangle A rectangle specified as a portion of a larger rectangle - generally the input image. For instance, { left: 0, top: 0, right: 1, bottom: 1 } would be a rectangle that fills the containing rectangle. This is commonly used to describe a cropped area of an image.
OcrRegion A grouping of text elements detected within an image.
OcrText A block of text detected in an image.

Category

Category of the object. The set of potential categories depends on the model. For instance, with object recognition, a category may be a "chair" or "dog". For visual intent, it may be "food_or_drink".

Name Type Description
name string
confidence number

CategoryList (extends IUFeature)

A list of categories that an image matches.

Name Type Description
categories array[Category]
featureType string Always 'Categories'.

ImageInfo

Information about the image. May be omitted if image binary is provided in POST request. Otherwise, either url or imageLookupToken is mandatory.

Name Type Description
recognitionModels array[IUModelResults] Objects (faces, etc.) and other features found by client-side recognition engines. One item per model.
url string
ImageRequestInfo ImageRequestInfo

ImageRequestInfo

Base-64 encoded image and optional coordinates of crop area (if specified by user).

Name Type Description
blob string Base-64 encoded image
cropArea NormalizedRectangle

Entity (extends IUFeature)

Describes an entity.

Name Type Description
name string The name of the entity. For example, this could be a dog breed or plant name.
description string A description of the entity.
featureType string Always 'Entity'.

IUFeature

Defines a feature that was detected on the image. For instance, detected OCR or objects.

Name Type Description
featureType string
score number Model's condifence score for this feature
area NormalizedBox
text string for MSOCR featuretype
categories array[Category] for Categories featureType

IUModelResults

Output of a model run on the image. The name of the object gives context into the model run, and a list of results is generated for the model.

Name Type Description
modelName string Name of the recognition engine
results array[IUFeature] The list of results of the recognition model. If the array is empty, it means that the model was run, but there were no results.

NormalizedBox

Similar to NormalizedRectangle, a box that defines a portion of a larger rectangle. In most cases, NormalizedBox will define a portion of the input image. It contains a set of NormalizedPoints that define a box.

Name Type Description
topleft NormalizedPoint
topright NormalizedPoint
bottomleft NormalizedPoint
bottomright NormalizedPoint

NormalizedPoint

An (x,y) coordinate normalized to the size of the image. It has values between 0 and 1. The values are relative to the width and height of the image (or other containing rectangle) respectively. For instance a normalized point defined as (0.3, 0.5) would describe a point that is 30% across the image (from the left) and 50% down the image (from the top).

Name Type Description
x number
y number

NormalizedRectangle

A rectangle specified as a portion of a larger rectangle - generally the input image. For instance, { left: 0, top: 0, right: 1, bottom: 1 } would be a rectangle that fills the containing rectangle. This is commonly used to describe a cropped area of an image.

Name Type Description
top number
left number
right number
bottom number

OcrRegion (extends IUFeature)

A grouping of text elements detected within an image.

Name Type Description
ocrLines array[OcrText] Lines of text within the region.
featureType string Always 'OcrRegion'.

OcrText (extends IUFeature)

A block of text detected in an image.

Name Type Description
text string The entire text detected in the image.
embeddedTexts array[OcrText] (Optional) Sub-regions of text within the given block. For instance, this could be an array of lines in a paragraph.
featureType string Always 'OcrText'.