Data formats
============

This document describes the various data formats that are used in the system.

Raw annotation data
-------------------

The client sends raw data for image annotations in a JSON format which is a list
of shapes. Each shape is a dictionary with the following keys:

* `type`: The type of the shape which can be:
  * `bbox` (bounding box, rectangle)
  * `polygon`
  * `polyline`
  * `point`
* `shape`: The shape data. Its format depends on the shape `type`:
  * For `bbox` it is a dictionary with keys x, y, w, h:
    ~~~json
    {"x": x, "y": y, "w": w, "h": h}
    ~~~
  * For `polygon` and `polyline` it is a list of points; each point is a
    dictionary with keys x and y:
    ~~~json
    [{"x": x1, "y": y1}, {"x": x2, "y": y2}, ...]
    ~~~
    The only difference between `polygon` and `polyline` is that the former is
    supposed to be closed so the last point is connected to the first one.
  * For `point` it is a dictionary with keys x and y:
    ~~~json
    {"x": x, "y": y}
    ~~~
  * All coordinates are floating-point numbers in the range [0, 1] and relative
    to the image size, with the origin in the top-left corner.
* `object`: The ID of the type of object (label) depicted in the shape. This ID
  is a human-readable string that must be registered in the system before
  being used on shapes.

The server sends the same data back to the client, to use to show the existing
annotations for an image.

### Example

~~~json
[
    {
        "type": "bbox",
        "shape": {"x": 0.1, "y": 0.1, "w": 0.5, "h": 0.5},
        "object": "Cat (Felis catus)"
    },
    {
        "type": "polygon",
        "shape": [{"x": 0, "y": 0}, {"x": 1, "y": 0}, {"x": 0, "y": 1}],
        "object": "Slice of pizza margherita"
    },
    {
        "type": "point",
        "shape": {"x": 0.5, "y": 0.5},
        "object": "Cat (Felis catus) - left eye"
    }
]
~~~

Query format
------------

The query format is based on YAML and used to query for pictures in the system.



### Example
~~~yaml
# Restrictions for queried images
want:
    # This means that the image must contain both rules, so both a cat and a dog
    - has_object: ["Cat (Felis catus)"]
    - has_object: ["Dog (Canis lupus familiaris)"]
    # Or we can put them in a list to mean that the image can contain any of the
    # objects in the list
    - has_object: ["Grass", "Flower"]
    # So the image must contain a cat and a dog, as well as either grass or
    # a flower
    # The following rule restricts the images to those with a certain source,
    # like a camera or a drawing; omitting this rule means that the images can
    # be of any type
    - nature: ["photo", "drawing"]
    # The following rule restricts the images to those with a certain licence
    - licence: ["CC-BY-1.0", "CC-BY-2.0", "CC-BY-3.0", "CC-BY-4.0", "CC0-1.0",
                "Unlicense", "WTFPL", "MIT", "BSD-2-Clause", "BSD-3-Clause",
                "Apache-2.0", "Informal-attribution", "Informal-do-anything",
                "Public-domain-old", "Public-domain"]
# Prohibitions for queried images
exclude:
    # This means that the image must not contain any of the objects in the list
    - has_object: ["Human"]
    # This excludes images uploaded before the given date
    - before_date: 1546300800
    # This requires images to have a minimum resolution
    - below_width: 800
    - below_height: 600
# In summary, we want images that contain both a cat and a dog, either a grass
# or a flower, but not a human, taken after 2019-01-01, must be a photo or a
# drawing, must carry one of certain permissive licences and have a resolution
# of at least 800x600 pixels.
~~~