DV Analyzer Case Study: STTS Fluctuation

“Sample to Time” (STTS) is a table in a Quicktime wrapper that represents how much time is allotted to each sample of a given track, i.e. the length of the sample’s duration.

In a typical DV video track every sample — in this case the sample would be a video frame — is allotted exactly the same amount of time. This sample/frame duration can be used to calculate the frame rate of a DV file, most typically 29.97 (or 25) frames per second. The audio track is also assigned a regular STTS number. The STTS Fluctuation error arises when one particular frame of a file includes a sample-to-time value in consistent with the other frames in the file.

The sample file was recorded from a half inch video tape run through a time base corrector to an I/O box, then out as a DV stream, and into a Mac running Final Cut. During this process 8 video frames were dropped. In the resulting file the frame before the drop (in the example presented frame #43) is assigned a timeline duration (the STTS value) that is 9 times higher than the other frames, thus frame 43 is presented for 9 times longer than any other frame. This results in a loss of visual data where frame 43 is held through the time represented by the dropped frames, and then a visual stutter where the image jumps to original frame 52. As a result of this irregularity, the sample file now has a variable frame rate and averages 27.36 frames per second instead of 29.97.

By adjusting the sample to time duration the playback system is trying to compensate for the 8 dropped frames. This keeps the timecode track and video track in sync following the event, thus preventing further visual errors that would affect much more of the video image. However the audio track is also missing 8 frames worth of audio which is not compensated for within the STTS associated with the audio track. The result is that the audio falls 8 frames out of sync following the event.

In the two hours of video captured from this particular setup the fluctuation causing error occurred 3 times. Each time the error manifested in the same manner with exactly 8 frames lost, an inconsistent sample-to-time value on the frame prior to the drop to cover the loss, and an increased loss of synchronization with the audio track.

Within the data of the DV stream at the point of this error the frame proceeding the drop and the one following it have sequential numbering, meaning that the video was lost before the DV stream was
encoded. The clues to discover this error (loss of video frames and loss of sync) are only discernible by identifying unexpected sample-to-time ratios associated with codecs that shouldn’t normally allow them, not within the DV stream itself. However, through use of DV stream analysis we were able to narrow down possible error points within the transfer and playback chain so that we could more quickly discover the cause and potential solution.

Thanks to Angelo Sacerdote of Bay Area Video Coalition
File Credit: Thomas F. Barrett, Ph.D., Hanna Perkins Center for Child Development

The original DV file referenced in this article can be downloaded here.