Lab Resources: Video Processing

A very useful technique for measurement of mechanical systems is simply to record the motion using a camera. Cameras have become very good at taking somewhat high speed video (120 FPS is common, with some phones even being able to record at 1000FPS), and very interesting phenomena can be investigated using the footage.

Remember to double check the camera is actually recording before starting the experiment. It is good practice to start recording the video at least a few seconds before and after the actual event. The event will be cut from the longer video later.

Some advice:

• Remember to place reference measurement devices (a meter stick for example) in the frame of the video. Two references should be used if the motion is two dimensional.
• Set the shutter speed to double the frame rate (240 FPS -> 1/480 second shutter speed)
• Film the event in very bright conditions (direct sunlight), so you can lower the shutter speed on very high frame rates. Avoid using LED lights, as the PWM will create variance in brigness of the individual frames.
• Do not record for a long period of time. High speed video will create large files very quickly. A 2 minute video at 240 FPS creates a 1GB file with h264 compression.

After you record the raw footage, move the video file to a computer. Use a media player to find the time range which includes the relevant parts of the motion. Be sure to get all of the motion you think is important, add a second to the beginning if you are unsure (you can further trim the video in step 2). Then use Handbreak to cut the important motion our of the raw video:

1. Open file -> select the original (long) video file.
2. Select “Preset”: Fast 1080p30.
3. Select “Range”: Seconds (input your time range).
4. Select “Framerate”: Same as source (under the video tab).
5. Select “Browse” in the bottom right to save the new short video.
6. Click “Start Encode” to create the shortened video of the event.

Please review the output video to see if the entire event is captured.

Result from this step: A short M4V video file (a couple seconds) showing the entire event. I will call this file “event.m4v”

This is an optional but encouraged step. Here, you take the rough trimmed video (event.m4v) and isolate only the frames which you find important. To accomplish this, I find it easiest to add the frame number to the video with ffmpeg, and then use handbrake to reprocess the video to create a new video file. This will help keep your file size to a minimum in the later analysis and let you focus only on the important part of the motion.

1. Create a new notebook in Google Colaboratory named “TrimVideo.ipynb”.
2. Upload the rough cut event.m4v to Colab.

3. Add frame numbers to the video with ffmpeg

      !ffmpeg -i event.m4v -vf "drawtext=fontfile=Arial.ttf: text='%{pts\:flt} \#%{frame_num}': start_number=1: x=500: y=900: fontcolor=white: fontsize=60: " -y -c:a copy event_frames.m4v
   

4. Download the newly created event_frames.m4v file to your computer.
5. View this file with a media player and identify which frames you want to keep. In VLC Media Player, click View->Advanced Controls to show a button for single frame advance (looks similar to the play button). In SMPlayer, simply press the comma and period button to move frames.
6. Use Handbrake to trim the original file (event.m4v) based on frame number. Follow the instructions above (Step 1) to encode the new file, which I will call “event_trim.m4v”, except select “Range: Frame” for the third step. Be sure to load the original file (without frame numbers) into handbrake, as you will reprint the frame numbers with the new trimmed video in the next step.

Result from this step: A short M4V video file (a couple seconds) showing the event. This file is now trimmed to the correct starting and ending frame. I will call this file “event_trim.m4v”.

In this step, you will take your raw video and cut out the important parts in preparation for further analysis. ImageJ requires a somewhat primitive video format called mjpeg, which is essentially a series of jpeg images all glued together. This file format will create files which are much larger than a normal mp4 or m4v file (format which youtube uses). You will be uploading your file to Google Colab and doing the video processing using a command line transcoding software called ffmpeg.

1. Open a new notebook in Google Colaboratory named “TrimVideo.ipynb”.
2. Upload the trimmed video event_trim.m4v to colab.

3. Add frame numbers to the video with ffmpeg and convert to an mjpeg (avi) format (Be careful, this makes LARGE FILES!):

      !ffmpeg -i event_trim.m4v -vf "drawtext=fontfile=Arial.ttf: text='t\=%{pts\:flt}s    Frame\:%{frame_num}': start_number=0: x=600: y=900: fontcolor=white: fontsize=60, crop=w=1920:h=1080:x=0:y=0" -an -vcodec mjpeg -y -q:v 4 event_imagej.avi
   

   Options for ffmpeg:

   • drawtext is used to insert text on top of the frames. Options for text location, color and size are shown.
   • drawtext is used to crop area Example: crop to size 100x100 at position (12,34). crop=w=100:h=100:x=12:y=34
   • video codec must be specified with “-vcodec mjpeg” for use with ImageJ
   • audio is dropped with the -an flag
   • image quality is specified with the -q:v flag. Highest quality and largest size is 2, lowest quality and smallest size is 31 (linear compression ratio).
4. download the new “/event_imagej.avi” file.

Success! Now you should be left with a clip of your event which can be imported into ImageJ!

1. Download ImageJ and extract the folder. Double click on the ImageJ applicaiton to start. You should see a horizontal bar with a bunch of buttons and no words…
2. Click File->Open and select the mjpeg file. Please be sure to select the mjpeg file, as ImageJ cannot open M4V or MP4 file types.

3. A window named “AVI Reader” should open. Select OK and your video should be displayed.

   Now we can move on to analysis: Lab Resources - Data Extraction with ImageJ