Data formats
============

This document describes the various data formats that are used in the system.

Raw annotation data (client → server)
-------------------------------------

The client sends raw data for image annotations in a JSON format which is a list
of shapes. Each shape is a dictionary with the following keys:

* `type`: The type of the shape which can be:
  * `bbox` (bounding box, rectangle)
  * `polygon`
  * `polyline`
  * `point`
* `shape`: The shape data. Its format depends on the shape `type`:
  * For `bbox` it is a dictionary with keys x, y, w, h:
    ~~~json
    {"x": x, "y": y, "w": w, "h": h}
    ~~~
  * For `polygon` and `polyline` it is a list of points; each point is a
    dictionary with keys x and y:
    ~~~json
    [{"x": x1, "y": y1}, {"x": x2, "y": y2}, ...]
    ~~~
    The only difference between `polygon` and `polyline` is that the former is
    supposed to be closed so the last point is connected to the first one.
  * For `point` it is a dictionary with keys x and y:
    ~~~json
    {"x": x, "y": y}
    ~~~
  * All coordinates are floating-point numbers in the range [0, 1] and relative
    to the image size, with the origin in the top-left corner.
* `object`: The ID of the type of object (label) depicted in the shape. This ID
  is a human-readable string that must be registered in the system before
  being used on shapes.

### Example

~~~json
[
    {
        "type": "bbox",
        "shape": {"x": 0.1, "y": 0.1, "w": 0.5, "h": 0.5},
        "object": "Cat (Felis catus)"
    },
    {
        "type": "polygon",
        "shape": [{"x": 0, "y": 0}, {"x": 1, "y": 0}, {"x": 0, "y": 1}],
        "object": "Slice of pizza margherita"
    },
    {
        "type": "point",
        "shape": {"x": 0.5, "y": 0.5},
        "object": "Cat (Felis catus) - left eye"
    }
]
~~~

Query format
------------

The query format is based on YAML and used to query for pictures in the system.



### Example
~~~yaml
# Restrictions for queried images
- want:
    # This means that the image must contain both rules, so both a cat and a dog
    - has_object: ["Cat (Felis catus)"]
    - has_object: ["Dog (Canis lupus familiaris)"]
    # Or we can put them in a list to mean that the image can contain any of the
    # objects in the list
    - has_object: ["Grass", "Flower"]
    # So the image must contain a cat and a dog, as well as either grass or
    # a flower
    # The following rule restricts the images to those with a certain source,
    # like a camera or a drawing; omitting this rule means that the images can
    # be of any type
    - nature: ["photo", "drawing"]
    # The following rule restricts the images to those with a certain licence
    - licence: ["CC-BY-1.0", "CC-BY-2.0", "CC-BY-3.0", "CC-BY-4.0", "CC0-1.0",
                "Unlicense", "WTFPL", "MIT", "BSD-2-Clause", "BSD-3-Clause",
                "Apache-2.0", "Informal-attribution", "Informal-do-anything",
                "Public-domain-old", "Public-domain"]
# Prohibitions for queried images
- exclude:
    # This means that the image must not contain any of the objects in the list
    - has_object: ["Human"]
    # This excludes images taken before the given date
    - before_date: 2019-01-01
    # This requires images to have a minimum resolution
    - below_width: 800
    - below_height: 600
# Pagination
- limit: 32
- offset: 0
# Sorting
- sort_by: "date-uploaded-recent"
# Format
- format: "jpg"
- max_resolution: [800, 800]  # resizes
# In summary, we want the 32 most recent images that contain both a cat and
# a dog, either a grass or a flower, but not a human, taken after 2019-01-01,
# must be a photo or a drawing, must carry one of certain permissive licences
# and have a resolution of at least 800x600 pixels.
~~~