RITMO

Video Visualization

Alexander Refsum Jensenius


Based on: Jensenius, Alexander Refsum. Some Video Abstraction Techniques for Displaying Body Movement in Analysis and Performance. Leonardo 46, no. 1 (2013): 53–60.

Working with Video

Recording video for analysis

  • Foreground/background
  • Lighting
  • Camera placement
  • Image quality

Spatiotemporal manipulation

  • Trimming: cut out part of the timeline
  • Cropping: cut out part of the image (space)

What is a video file?

Video file information

Video as a stream of numbers

A video file is just a collection of numbers

Compression

  • Container formats (AVI, MOV, MPEG)
  • Compression formats (H.264, MJPEG)

Preparing video for analysis

Qualitative:

  • Video: H.264
  • Audio: AAC

Quantitative:

  • Video: MJPEG
  • Audio: WAV/AIFF

Video visualisation

Why visualise something visual?

Different types of visualisation

  • dynamic visualisations (video files)
    • motion video
    • motion history video
  • static visualisations (images)
    • motion average image
    • motiongrams
    • videograms
  • motion data (csv files)
    • quantity of motion
    • centroid of motion
    • area of motion

Visualising Motion

Motion video

A *motion image* is created by subtracting subsequent frames in a video file (Frame(2) - Frame(1)).

Based on "frame-differencing"

Original video

Motion video

Motion video (inverse)

Filtering

Combining Displays

Visualising Time

Motion history video

Motion history video settings

Silhouette motion video

Optical Flow

Visualising Space

Motion Average Image

MAI

Motion Average Image with overlay

MAI

Combinations

Percussion

Visualising Time and Space

Motiongrams

Sketch of the calculation of a motiongram.

Greyscale (inverse)

Sketch of the calculation of a motiongram.

In colour

Sketch of the calculation of a motiongram.

Long time stretches

Sketch of the calculation of a motiongram.

Motiongram with spectrogram

Sketch of the calculation of a motiongram.

Comparative

Sketch of the calculation of a motiongram.

Quantity of Motion

1.2162e+05,259.1,223.2
97576,256.26,210.92
88415,250.04,214.91
76769,249.61,221.22
81350,242.81,218.21
77598,239.7,208.27
70974,256.93,219.15

A plot of the quantity of motion for a 5-minute long dance sequence. The grey line is a plot of the tracked data, and the black line is a filtered version of the same data set.

Centroid of motion (CoM) and Area of motion (AoM)

Illustrations of the area and centroid of body and motion.

Software

VideoAnalysis tutorial

Applications

CIMA

Sonifications based on motiongrams