Skip to content

Broox Vision Node

The Broox Vision Node is a general-purpose vision pipeline building kit. It is built with Intel Openvino.

Architecture

A VideoCapture device (webcam, file, IP camera) generates a starting Snap(shot) object.

Snaps hold the picture(s) and metadata from the different Processors.

Each Processor analyzes and/or transforms the Snap contents, either modifying the images or adding extra data.

flowchart LR VideoCapture -- Snap --> Pipeline --> Processor subgraph Processor Processor1 -- Snap --> Processor2 -- Snap --> ... -- Snap --> ProcessorN end

The main property of every Snap is the boxes list. Contains Box objects that represent a rectangular section over the image. That might be faces, bodies, blobs or any item detected.

Boxes hold extra features of the detection (data pairs).

The output of this processing can be delivered by any custom output Processor to the network, via OSC, MQTT, REST or whatever.

Video Sources

Video sources can be webcams, RTSP or HTTP video streams, 3D cameras...

Video Source Format explains the video source definition.

Processors

Processor are separate threads that receive a Snap and operate upon it, and return it. Each Processor has a list of tunable Parameters.

Some Processors generate output Events. Those can be emitted as MQTT messages, trigger messages, or via OSC/TUIO UDP messages.

See Processors for details, and an API list of available processors.

Pipelines

A pipeline is a concatenation of processors, executing sequentially over the input image.

Pipelines are defined as .yml files inside the pipelines/ directory.

Standard pipelines included are:

  • Analytics oriented pipelines, of different processing power needs:

    • audience-analytics-high: Detect people, faces, age, gender, accessories, attention and dwell time.
    • audience-analytics-medium: Detect people, faces, age, gender, attention and dwell time.
    • audience-analytics-low: Detect people, faces age and gender.

  • Application oriented pipelines:

    • photobooth: Presence detection and interaction triggering.
    • interactive-world: Presence, skeleton tracking and gestural interaction.

Inside the installation folder there is an extra custom folder containing more pipeline examples for other scenarios.

Features

Attributes (features) depend on the processors in the pipeline.

  • AgeGenderProcessor:
    • age: integer 0-99
    • gender: string either "M" (male) or "F" (female)
  • FaceAttentionProcessor:
    • attention: integer percent 0-100.
    • attention_time: float, in seconds (only on action: out messages).
  • FaceEmotionProcessor:
    • dominant: string, One of calm, happy, sad, anger or surprise.
  • StageProcessor:
    • dwell: integer, only on action:out, seconds the box has been present.
  • AreaHitProcesor:
    • tags: list of strings, only on action:out, of areas entered.