Docs Dashboard

Image Metadata

Annotations

Terms that describe the image content

JSON Models

Example

Image JSON
{
  "imageInfo": {
    "imageRequestInfo": {
      "blob": ""
    },
    "recognitionModels": [
      {
        "modelName": "ImageAnnotationV2",
        "results": [
          {
            "categories": [
              {
                "name": "rice"
              },
              {
                "name": "fried rice"
              },
              {
                "name": "chicken rice"
              },
              {
                "name": "vegetable recipes"
              },
              {
                "name": "vegetable stir fries"
              },
              {
                "name": "chicken"
              }
            ],
            "featureType": "Categories"
          }
        ]
      }
    ]
  }
}

Celebrity Entities

Information about celebrities detected in an image.

JSON Models

Example

Image JSON
{
  "imageInfo": {
    "recognitionModels": [
      {
        "modelName": "EntityRecognition.Image.Person",
        "results": [
          {
            "name": "Satya Nadella",
            "description": "Satya Narayana Nadella is an Indian-American business executive. He currently serves as the Chief Executive Officer of Microsoft, succeeding Steve Ballmer in 2014. Before becoming chief executive, he was Executive Vice President of Microsoft's Cloud and Enterprise Group, responsible for building and running the company's computing platforms.",
            "featureType": "Entity",
            "score": 0.999947548,
            "area": {
              "topLeft": {
                "x": 0.306640625,
                "y": 0.170087978
              },
              "topRight": {
                "x": 0.55859375,
                "y": 0.170087978
              },
              "bottomLeft": {
                "x": 0.306640625,
                "y": 0.5483871
              },
              "bottomRight": {
                "x": 0.55859375,
                "y": 0.5483871
              }
            }
          }
        ]
      }
    ]
  }
}

Dog Entities

Information about dogs detected in an image.

JSON Models

Example

Image JSON
{
  "imageInfo": {
    "imageRequestInfo": {
      "blob": ""
    },
    "recognitionModels": [
      {
        "modelName": "EntityRecognition.Image.Dog",
        "results": [
          {
            "name": "Chesapeake Bay Retriever",
            "description": "The Chesapeake Bay Retriever is a large-sized breed of dog belonging to the Retriever, Gundog, and Sporting breed groups. Members of the breed may also be referred to as a Chessie, CBR, or Chesapeake. The breed was developed in the United States Chesapeake Bay area during the 19th century. Historically used by area market hunters to retrieve waterfowl, pull fishing nets, and rescue fishermen, it is primarily a family pet and hunting companion. They are often known for their love of water and their ability to hunt. It is a medium to large sized dog similar in appearance to the Labrador Retriever. The Chesapeake has a wavy coat, rather than the Labrador's smooth coat. They are described as having a bright and happy disposition, courage, willingness to work, alertness, intelligence, and love of water as some of their characteristics.",
            "featureType": "Entity",
            "score": 0.956623554,
            "area": {
              "topLeft": {
                "x": 0,
                "y": 0
              },
              "topRight": {
                "x": 1,
                "y": 0
              },
              "bottomLeft": {
                "x": 0,
                "y": 1
              },
              "bottomRight": {
                "x": 1,
                "y": 1
              }
            }
          }
        ]
      }
    ]
  }
}

Faces

Faces with bounding box coordinates

JSON Models

Example

Image JSON
{
  "imageInfo": {
    "recognitionModels": [
      {
        "modelName": "FaceDetection",
        "results": [
          {
            "annotations": [],
            "featureType": "FaceDetectRegion",
            "score": 0,
            "area": {
              "topLeft": {
                "x": 0.310546875,
                "y": 0.173020527
              },
              "topRight": {
                "x": 0.55859375,
                "y": 0.173020527
              },
              "bottomLeft": {
                "x": 0.310546875,
                "y": 0.545454562
              },
              "bottomRight": {
                "x": 0.55859375,
                "y": 0.545454562
              }
            }
          }
        ]
      }
    ]
  }
}

Fashion-related objects

Clothing, shoes & accessories detected

JSON Models

Example

Image JSON
{
  "imageInfo": {
    "modelName": "ObjectDetection",
    "results": [
      {
        "hotspot": {
          "topLeft": {
            "x": 0.61368835,
            "y": 0.490951449
          },
          "topRight": {
            "x": 0.61368835,
            "y": 0.490951449
          },
          "bottomLeft": {
            "x": 0.61368835,
            "y": 0.490951449
          },
          "bottomRight": {
            "x": 0.61368835,
            "y": 0.490951449
          }
        },
        "featureType": "Object",
        "area": {
          "topLeft": {
            "x": 0.021561414,
            "y": 0.0277318656
          },
          "topRight": {
            "x": 0.9689646,
            "y": 0.0277318656
          },
          "bottomLeft": {
            "x": 0.021561414,
            "y": 0.954171062
          },
          "bottomRight": {
            "x": 0.9689646,
            "y": 0.954171062
          }
        }
      }
    ]
  }
}

Image

Binary image or URL (Base64)

JSON Models

Example

Image JSON
{
  "imageInfo": {
    "imageRequestInfo": {
      "blob": "iVBORw0KGgoAAAANSUhEUgAAAKAAAABQCAYAAACeXX40AAAABGdBTUEAALGPC/xhBQAAQABJREFUeAFMvVmQ5Nl1n3dy3ytrX7q7eu+efcEMSGAIQADFRZRJSyGadpiSw5ZDCvlB4Qg/6VHBB+uFr36xXyzLYTvMsPVAyjLpsE2RImAQ2wAYzGC23qu7qmuvzMqs3DP9fTd7aFdPTlZl/pd7z/md/dz7zwQ// …"
    }
  }
}

Landmark Entities

Landmarks detected

JSON Models

Example

Image JSON
{
  "imageInfo": {
    "recognitionModels": [
      {
        "modelName": "EntityRecognition.Image.Landmark",
        "results": [
          {
            "name": "Cairo Tower",
            "description": "The Cairo Tower is a free-standing concrete tower in Cairo, Egypt. At 187 m, it has been the tallest structure in Egypt and North Africa for about 50 years. It was the tallest structure in Africa for ten years until 1971, when it was surpassed by Hillbrow Tower in South Africa.",
            "featureType": "Entity",
            "score": 0.9434098,
            "area": {
              "topLeft": {
                "x": 0,
                "y": 0
              },
              "topRight": {
                "x": 1,
                "y": 0
              },
              "bottomLeft": {
                "x": 0,
                "y": 1
              },
              "bottomRight": {
                "x": 1,
                "y": 1
              }
            }
          },
          {
            "name": "Cairo",
            "description": "Cairo is the capital of Egypt. The city's metropolitan area is one of the largest in Africa, the largest in the Middle East, and the 15th-largest in the world, and is associated with ancient Egypt, as the famous Giza pyramid complex and the ancient city of Memphis are located in its geographical area. Located near the Nile Delta, modern Cairo was founded in 969 AD by the Fatimid dynasty, but the land composing the present-day city was the site of ancient national capitals whose remnants remain visible in parts of Old Cairo. Cairo has long been a centre of the region's political and cultural life, and is titled \"the city of a thousand minarets\" for its preponderance of Islamic architecture. Cairo is considered a World City with a \"Beta +\" classification according to GaWC.",
            "featureType": "Entity",
            "score": 0.7346044,
            "area": {
              "topLeft": {
                "x": 0,
                "y": 0
              },
              "topRight": {
                "x": 1,
                "y": 0
              },
              "bottomLeft": {
                "x": 0,
                "y": 1
              },
              "bottomRight": {
                "x": 1,
                "y": 1
              }
            }
          }
        ]
      }
    ]
  }
}

Optical Character Recognition

Detected text and symbols

JSON Models

Example

Image JSON
{
  "imageInfo": {
    "imageRequestInfo": {
      "blob": ""
    },
    "recognitionModels": [
      {
        "modelName": "MSROCR",
        "results": [
          {
            "ocrLines": [
              {
                "text": "Margie's Travel",
                "embeddedTexts": [
                  {
                    "text": "Margie's",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.04095238,
                        "y": 0.41
                      },
                      "topRight": {
                        "x": 0.306666672,
                        "y": 0.406666666
                      },
                      "bottomLeft": {
                        "x": 0.0390476175,
                        "y": 0.53
                      },
                      "bottomRight": {
                        "x": 0.303809524,
                        "y": 0.54
                      }
                    }
                  },
                  {
                    "text": "Travel",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.311428577,
                        "y": 0.406666666
                      },
                      "topRight": {
                        "x": 0.5,
                        "y": 0.405
                      },
                      "bottomLeft": {
                        "x": 0.307619035,
                        "y": 0.54
                      },
                      "bottomRight": {
                        "x": 0.4961905,
                        "y": 0.521666646
                      }
                    }
                  }
                ],
                "featureType": "OcrText",
                "score": 1,
                "area": {
                  "topLeft": {
                    "x": 0.0390476175,
                    "y": 0.4
                  },
                  "topRight": {
                    "x": 0.5,
                    "y": 0.405
                  },
                  "bottomLeft": {
                    "x": 0.0390476175,
                    "y": 0.5366667
                  },
                  "bottomRight": {
                    "x": 0.5,
                    "y": 0.5416667
                  }
                }
              }
            ],
            "featureType": "OcrRegion",
            "score": 1,
            "area": {
              "topLeft": {
                "x": 0.03904762,
                "y": 0.400000036
              },
              "topRight": {
                "x": 0.501482,
                "y": 0.405015022
              },
              "bottomLeft": {
                "x": 0.0375656672,
                "y": 0.536651731
              },
              "bottomRight": {
                "x": 0.50000006,
                "y": 0.5416667
              }
            }
          },
          {
            "ocrLines": [
              {
                "text": "Phone: 716 555 0123",
                "embeddedTexts": [
                  {
                    "text": "Phone:",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.0466666669,
                        "y": 0.665
                      },
                      "topRight": {
                        "x": 0.155238092,
                        "y": 0.6666667
                      },
                      "bottomLeft": {
                        "x": 0.0466666669,
                        "y": 0.728333354
                      },
                      "bottomRight": {
                        "x": 0.155238092,
                        "y": 0.7266667
                      }
                    }
                  },
                  {
                    "text": "716",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.152380958,
                        "y": 0.6666667
                      },
                      "topRight": {
                        "x": 0.208571434,
                        "y": 0.6666667
                      },
                      "bottomLeft": {
                        "x": 0.153333336,
                        "y": 0.7266667
                      },
                      "bottomRight": {
                        "x": 0.209523812,
                        "y": 0.725
                      }
                    }
                  },
                  {
                    "text": "555",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.211428568,
                        "y": 0.6666667
                      },
                      "topRight": {
                        "x": 0.2704762,
                        "y": 0.665
                      },
                      "bottomLeft": {
                        "x": 0.211428568,
                        "y": 0.725
                      },
                      "bottomRight": {
                        "x": 0.2704762,
                        "y": 0.725
                      }
                    }
                  },
                  {
                    "text": "0123",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.272380948,
                        "y": 0.665
                      },
                      "topRight": {
                        "x": 0.347619057,
                        "y": 0.6616667
                      },
                      "bottomLeft": {
                        "x": 0.273333341,
                        "y": 0.725
                      },
                      "bottomRight": {
                        "x": 0.34857142,
                        "y": 0.725
                      }
                    }
                  }
                ],
                "featureType": "OcrText",
                "score": 1,
                "area": {
                  "topLeft": {
                    "x": 0.0466666669,
                    "y": 0.665
                  },
                  "topRight": {
                    "x": 0.347619057,
                    "y": 0.6616667
                  },
                  "bottomLeft": {
                    "x": 0.0466666669,
                    "y": 0.728333354
                  },
                  "bottomRight": {
                    "x": 0.34857142,
                    "y": 0.725
                  }
                }
              },
              {
                "text": "4567 Main Street, Buffalo, NY 98052",
                "embeddedTexts": [
                  {
                    "text": "4567",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.04761905,
                        "y": 0.733333349
                      },
                      "topRight": {
                        "x": 0.121904761,
                        "y": 0.735
                      },
                      "bottomLeft": {
                        "x": 0.05047619,
                        "y": 0.785
                      },
                      "bottomRight": {
                        "x": 0.123809524,
                        "y": 0.79
                      }
                    }
                  },
                  {
                    "text": "Main",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.123809524,
                        "y": 0.735
                      },
                      "topRight": {
                        "x": 0.1952381,
                        "y": 0.735
                      },
                      "bottomLeft": {
                        "x": 0.125714287,
                        "y": 0.79
                      },
                      "bottomRight": {
                        "x": 0.197142854,
                        "y": 0.7916667
                      }
                    }
                  },
                  {
                    "text": "Street,",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.207619041,
                        "y": 0.735
                      },
                      "topRight": {
                        "x": 0.300952375,
                        "y": 0.735
                      },
                      "bottomLeft": {
                        "x": 0.209523812,
                        "y": 0.793333352
                      },
                      "bottomRight": {
                        "x": 0.302857131,
                        "y": 0.793333352
                      }
                    }
                  },
                  {
                    "text": "Buffalo,",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.300952375,
                        "y": 0.735
                      },
                      "topRight": {
                        "x": 0.416190475,
                        "y": 0.7316667
                      },
                      "bottomLeft": {
                        "x": 0.302857131,
                        "y": 0.793333352
                      },
                      "bottomRight": {
                        "x": 0.418095231,
                        "y": 0.7916667
                      }
                    }
                  },
                  {
                    "text": "NY",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.414285719,
                        "y": 0.7316667
                      },
                      "topRight": {
                        "x": 0.46,
                        "y": 0.7316667
                      },
                      "bottomLeft": {
                        "x": 0.416190475,
                        "y": 0.7916667
                      },
                      "bottomRight": {
                        "x": 0.461904764,
                        "y": 0.79
                      }
                    }
                  },
                  {
                    "text": "98052",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.466666669,
                        "y": 0.73
                      },
                      "topRight": {
                        "x": 0.5542857,
                        "y": 0.7266667
                      },
                      "bottomLeft": {
                        "x": 0.467619061,
                        "y": 0.79
                      },
                      "bottomRight": {
                        "x": 0.555238068,
                        "y": 0.7866667
                      }
                    }
                  }
                ],
                "featureType": "OcrText",
                "score": 1,
                "area": {
                  "topLeft": {
                    "x": 0.04857143,
                    "y": 0.725
                  },
                  "topRight": {
                    "x": 0.555238068,
                    "y": 0.7266667
                  },
                  "bottomLeft": {
                    "x": 0.04761905,
                    "y": 0.793333352
                  },
                  "bottomRight": {
                    "x": 0.555238068,
                    "y": 0.795
                  }
                }
              },
              {
                "text": "margie@margiestravel.com",
                "embeddedTexts": [
                  {
                    "text": "margie@margiestravel.com",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.0457142852,
                        "y": 0.805
                      },
                      "topRight": {
                        "x": 0.425714284,
                        "y": 0.8016667
                      },
                      "bottomLeft": {
                        "x": 0.0466666669,
                        "y": 0.8616667
                      },
                      "bottomRight": {
                        "x": 0.4247619,
                        "y": 0.858333349
                      }
                    }
                  }
                ],
                "featureType": "OcrText",
                "score": 1,
                "area": {
                  "topLeft": {
                    "x": 0.0457142852,
                    "y": 0.805
                  },
                  "topRight": {
                    "x": 0.425714284,
                    "y": 0.8016667
                  },
                  "bottomLeft": {
                    "x": 0.0466666669,
                    "y": 0.8616667
                  },
                  "bottomRight": {
                    "x": 0.425714284,
                    "y": 0.858333349
                  }
                }
              },
              {
                "text": "www.margiestravel.com",
                "embeddedTexts": [
                  {
                    "text": "www.margiestravel.com",
                    "embeddedTexts": [],
                    "featureType": "OcrText",
                    "score": 1,
                    "area": {
                      "topLeft": {
                        "x": 0.0495238081,
                        "y": 0.873333335
                      },
                      "topRight": {
                        "x": 0.369523823,
                        "y": 0.86833334
                      },
                      "bottomLeft": {
                        "x": 0.05142857,
                        "y": 0.92
                      },
                      "bottomRight": {
                        "x": 0.369523823,
                        "y": 0.9216667
                      }
                    }
                  }
                ],
                "featureType": "OcrText",
                "score": 1,
                "area": {
                  "topLeft": {
                    "x": 0.0495238081,
                    "y": 0.87
                  },
                  "topRight": {
                    "x": 0.369523823,
                    "y": 0.86833334
                  },
                  "bottomLeft": {
                    "x": 0.0495238081,
                    "y": 0.923333347
                  },
                  "bottomRight": {
                    "x": 0.369523823,
                    "y": 0.9216667
                  }
                }
              }
            ],
            "featureType": "OcrRegion",
            "score": 1,
            "area": {
              "topLeft": {
                "x": 0.0449762,
                "y": 0.663242459
              },
              "topRight": {
                "x": 0.5548941,
                "y": 0.6605875
              },
              "bottomLeft": {
                "x": 0.0463305,
                "y": 0.92335
              },
              "bottomRight": {
                "x": 0.556248367,
                "y": 0.920695
              }
            }
          }
        ]
      }
    ]
  }
}

Barcode (coming soon)

Decoded value of a barcode

QR Code (coming soon)

Decoded value of a QR code