Skip to content

Format of Video Sources (Cameras)

Video Sources (a.k.a "cameras") are defined in the visionnode.ini file under the [camera] header.

[camera]
id=orbecc
type=usb-orbecc
name=Orbecc 3D Camera
resolution=640x480
...

Note: Multiple cameras can be defined in the .ini as [camera:0], [camera:1] sections after the base [camera] section, though multicamera support is still experimental.

A [camera] definition section can split into:

  1. Base Definition
  2. Frame properties and Transforms
  3. Camera specific properties

Base Definition

  • id: Short id for the camera.
  • name: Descriptive name for the camera (user facing).
  • type: Driver the camera uses.
    • opencv-nt uses OpenCV2 VideoCaptureOpenCV.
    • picamera uses VideoCaptureRaspberry.
    • usb-realsense for Intel Realsense, uses VideoCaptureRealSense, requires pyrealsense2.
    • usb-orbbec for Orbbec Astra, uses VideoCaptureOrbbec, requires OpenNI2.
    • usb-k4a for Kinect 4, requires the Kinekt SDK (experimental).
  • calibration: Optional pointer to a .json file containing camera calibration parameters.

Capture Properties

  • resolution: In the form Width x Height, with no spaces: e.g. 640x480.
  • If the device supports capturing in this resolution, it will be set so. If not, frames will be rescaled to match.
  • fps: Frames per second, a decimal number, e.g. 15 or 24.5.
  • For higher framerates see fourcc setting in OpenCV Driver
  • rate_limit_fps: Rate limiting by discarding captured frames, useful if the device doesn't support a lower framerate.

Frame Transforms

  • rotation: One of none, 90, 180 or -90 degrees.
  • flip: One of none, horizontal, vertical or both axis.
  • clipping: A comma-separated coordinate values list of x,y,w,h defining a clipping frame (resulting frames will be w x h sized).

OpenCV/OpenCV-nt Driver

  • backend: String of the form CAP_XXXXX.
    • Default is CAP_ANY (0).
    • Others: CAP_FFMPEG, CAP_GSTREAMER, CAP_OPENNI2...
  • index: For webcams, an integer index for the camera.
  • fourcc: If the camera can offer video over MJPEG with a higher framerate, you can set to MJPG to select this framerate.
  • url: For network cameras, video files, etc:
    • http://... or rtsp://... URLs
    • A file path (e.g. /home/user/movie.mp4)
  • buffer: Number of frames for OpenCV to buffer (might cause lag).

Calibration Files

Calibration json files contain intrinsic and extrinsic calibration data for the given camera, as used by opencv and related toolkits.

{
  "sensor_size_mm": [ 3.6, 2.7 ],
  "intrinsic": {
    "width": 640,
    "height": 480,
    "matrix": [
      [ 779.5962524414062, 0.0, 511.0530090332031, 0.0 ],
      [ 0.0, 1008.8184204101562, 382.3992614746094, 0.0 ],
      [ 0.0, 0.0, 1.0, 0.0 ],
      [ 0.0, 0.0, 0.0, 1.0 ]
    ]
  },
  "extrinsic": {
    "matrix": [
      [ 0.9928808212280273, 0.09882735460996628, -0.06648891419172287, -0.19881358742713928 ],
      [ -0.06223934516310692, 0.9063953757286072, 0.41782012581825256, -0.9262909889221191 ],
      [ 0.10155729949474335, -0.41070738434791565, 0.9060935974121094, 36.0 ],
      [ 0.0, 0.0, 0.0, 1.0 ]
    ]
  }
}