Movie Playback

From Derivative
Revision as of 23:12, 3 February 2019 by Greg (talk | contribs)
Jump to navigation Jump to search


This article will describe the stages of data transfer and processing required to playback a movie. It describes more in depth what the Movie File In TOP does to get a frame on the GPU in a TOP.

There are three stages: The first stage of playing back a movie is reading the file data from disk, unmodified, into CPU memory. Once the file data is in CPU memory it needs to be decoded into a raster image. The final stage of playback occurs when the frame is uploaded from the pre-read cache of CPU memory to GPU memory where it can be used by all TOPs downstream on the GPU.

Reading from Disk[edit]

The first stage of playing back a movie is reading the file data from disk. This data needs to be read into CPU memory so it can be decompressed into uncompressed RGBA or YUV image data.


The bit-rate of a movie is the amount of data per second that needs to be read from disk to play the file. This number is given in bits-per-second usually. Drive speed is usually given in bytes-per-second though, so be sure you are using the correct units when determining if you have the required read speed to playback your movie. Bytes are usually denoted with a capital 'B', while bits are denoted with a lower case 'b'. There are 8 bits in 1 byte, so a file that has a bit-rate of 80Mbits/sec will consume disk read bandwidth of 10MBytes/s.

Buffered/Cached Reading[edit]

By default all files read in the operating system are read using what is called Buffered/Cached reading. What this means is that data from the file is first copied to CPU memory owned by the OS, and then copied over to TouchDesigner's CPU memory. The OS is free to keep its copy of the file in its memory for as long as it wants. The OS will only discard that file data if it needs to, for example if that CPU memory is needed for other reasons. The data kept by the OS in it's memory is the compressed file data, not uncompressed RGB/YUV frames.

If you are playing back files that total 4GB in size, and you have 16GB of CPU RAM. Unless you have applications eat up that other RAM it's entirely possible the files will only be read from disk once and the rest of the time the data in the OS's CPU memory will be read from instead. File data is cached as it gets read so a file won't be entirely cached until you've played through the entire file once. Check your disk usage to know if your playback is reading from the disk or using cached data.

Note - If you have content that plays back poorly the first run through but fast the second time, this likely means your disk cannot deliver the data at the required rate, and playback is only possible from the cached data. This can be a dangerous state to be in unless you have enough RAM, because you have no control of when/if the OS will drop cached data out of it's RAM.

Unless you are playing back files that have very high data-rate (uncompressed data, HAP, lossless H264 etc.), this is usually the mode you want to playback in.

High-Performance Read (Unbuffered Reading)[edit]

The High-Performance Read parameter in the Movie File In TOP turned off buffered/cached file reading. Data is read directly from the disk into TouchDesigner's CPU memory. This is faster because it avoids a memory copy operation. There is also other algorithmic optimizations done specific to TouchDesigner in this mode that allows for extremely high data-rate playback. When using this mode the disk will be constantly in use during playback and it will show the true rate playback of your content.

Decoding Frames[edit]

Once the file data is in CPU memory it needs to be decoded into an image. This image is usually uncompressed RGB or YUV data. Some codecs will make use of multiple threads to decode the data. This is controllable by the 'Max Decode CPUs' parameter in the Movie File In TOP. Codecs such as HAP have a very light CPU decompression stage, while codecs like H264 have a very heavy CPU decompression stage.

Although a portion of Hap is decoded on the GPU, there is still a CPU cost. By default it has a lossless CPU compression that is applied which will take time. If this compression is disabled when encoding the cost will be greatly reduces, but there is still memory copies that will take some time.

Decompressed frames are placed into to the Pre-Read cache of the Movie File In TOP. The size of this cache is controlled by the 'Pre-Read Frames' parameter. The preload() method on the Movie File In TOP is a way to fill the 'Pre-Read Frames' with minimal impact on playback of the rest of the file.

Movie In CPU Cache Size[edit]

Legacy Feature - In the Preferences for TouchDesigner on the TOPs page, there is a preference called 'Movie In CPU Cache Size'. This is a block of memory where decompressed frames will be placed to be re-used next time they are required. This feature is legacy because it only really works if you have short video loops that will fit into this cache as uncompressed frames. Due to the size of uncompressed data it is rare that this feature is useful and should generally be left at the default 0.

Uploading to the GPU[edit]

The final stage of playback occurs when the Movie File In TOP needs to output a particular frame. The frame is uploaded from the pre-read cache to GPU memory where it can be used by all TOPs downstream on the GPU. If the frame isn't already in the pre-read cache, possibly due to cueing the movie to a new index, a pre-read miss will occur which will force the Movie File In TOP to wait until that frame is read and decompressed.