Experimental:WrnchAI CHOP

From TouchDesigner Documentation
Jump to: navigation, search

Summary
[edit]

Note: Only available in TouchDesigner Pro.

Using the wrnchAI engine to track human motion in any video stream, this CHOP will output channels for Body, Face and Hand positions. Multiple people can be tracked at the same time with low-latency, and the system compatible with all cameras and video feeds. TouchDesigner used the wrnchAI Edge SDK and all computation is done locally in realtime utilizing Nvidia GPUs and the NVIDIA CUDA® Deep Neural Network library (cuDNN). This can offer skeletal tracking of multiple people at 60fps or higher (90-120fps etc).

Requires a separate license from wrnchAI to run in TouchDesigner. Requires Nvidia GPU on Windows OS and TouchDesigner Pro. 
Make sure to install TouchDesigner from the Full Installer, the Lite Installer does not include wrnchAI libraries.
Quickstart Guide

Once you have a wrnchAI license (purchased or trial), you will have access to your wrnchAI licenses page and the wrnchAI Edge Releases page.

  1. Go to your wrnchAI licenses page. Click on the 'key' icon and copy the Edge License Key into the CHOP's 'wrnchAI License' parameter.
  2. Go to wrnchAI Edge Releases page and download the wrnchAI Engine for Win10. The file will be something like wrnchAI-Engine-1.17.0-GPU-Win10.zip
  3. Once downloaded, extract the .zip file and locate the wrnchAI trained models folder for use in the CHOP's 'Model Folder' parameter. It should be located here: wrnchAI-engine-GPU-1.17.0-Windows-amd64/bin/wrModels

The wrnchAI CHOP will take a few moments to initialize and then will be ready for tracking. Initializing is only required when a new Model Folder is introduced, it will not need to re-initialize each time the project is started.

PythonIcon.pngwrnchaiCHOP_Class


Parameters - Setup Page

wrnchAI License license - Add the Edge License Key from your wrnchAI account here. Requires a separate license from wrnchAI.  

Model Folder modelfolder - Specify the path the the folder that contains the trained models. See Quickstart Guide above for details.  

GPU Device gpu - - A menu of available GPU(s) to run wrnchAI on. Selecting 'Default' uses the same GPU TouchDesigner is currently running on.

  • Default default - Uses the same GPU that TouchDesigner is currently assigned to.

TOP top - Specify the TOP to process for tracking.  

Body 3D body3d - Enable the channels (tx, ty, tz) for body tracking points in 3D space.  

Body 2D body2d - Enable the channels (u, v) for body tracking points in 2D.  

Body 3D IK body3dik - Enable the IK channels (tx, ty, tz, rx, ry, rz and associated roll channels) for body tracking points in 3D space.  

Face Bounds facebounds - Enable the position and bounds channels (u, v, width, height) for faces.  

Face face - Enable the position channels (u, v) for all face tracking points.  

Hands Bounds handsbounds - Enable the position and bounds channels (u, v, width, height) for hands.  

Hands hands - Enable the position channels (u, v) for all hand tracking points.  

Max Players maxplayers - Sets the maximum number of people that can be tracked. For each person/player a new set of channels is created prefixed p1, p2, p3, ... etc.

NOTE: The parameter slider goes to 10 but higher numbers of players can be tracked by setting a higher Max Player value.  

Aspect Correct UVs aspectcorrectuv - Adjusts the values of u and v channels to take the TOP's aspect ratio into account. When using non-square input TOP, turn this on to line up u and v position with the image.  


Parameters - Common Page

Time Slice timeslice - Turning this on forces the channels to be "Time Sliced". A Time Slice is the time between the last cook frame and the current cook frame.  

Scope scope - To determine which channels get affected, some CHOPs use a Scope string on the Common page.  

Sample Rate Match srselect - - Handle cases where multiple input CHOPs' sample rates are different. When Resampling occurs, the curves are interpolated according to the Interpolation Method Option, or "Linear" if the Interpolate Options are not available.

  • Resample At First Input's Rate first - Use rate of first input to resample others.
  • Resample At Maximum Rate max - Resample to the highest sample rate.
  • Resample At Minimum Rate min - Resample to the lowest sample rate.
  • Error If Rates Differ err - Doesn't accept conflicting sample rates.

Export Method exportmethod - - This will determine how to connect the CHOP channel to the parameter. Refer to the Export article for more information.

  • DAT Table by Index datindex - Uses the docked DAT table and references the channel via the index of the channel in the CHOP.
  • DAT Table by Name datname - Uses the docked DAT table and references the channel via the name of the channel in the CHOP.
  • Channel Name is Path:Parameter autoname - The channel is the full destination of where to export to, such has geo1/transform1:tx.

Export Root autoexportroot - This path points to the root node where all of the paths that exporting by Channel Name is Path:Parameter are relative to.  

Export Table exporttable - The DAT used to hold the export information when using the DAT Table Export Methods (See above).  


CHOPs
Ableton Link • Analyze • Angle • Attribute • Audio Band EQ • Audio Device In • Audio Device Out • Audio Dynamics • Audio File In • Audio File Out • Audio Filter • Audio Movie • Audio NDI • Audio Oscillator • Audio Para EQ • Audio Play • Audio Render • Audio Spectrum • Audio Stream In • Experimental:Audio Stream In • Audio Stream Out • Beat • Bind • BlackTrax • Blend • Blob Track • Bullet Solver • Clip Blender • Clip • Clock • Composite • Constant • Copy • Count • CPlusPlus • Cross • Cycle • DAT to • Delay • Delete • DMX In • DMX Out • Envelope • EtherDream • Event • Expression • Extend • Fan • Feedback • File In • File Out • Filter • Experimental:Filter • Experimental:FreeD • Function • Gesture • Handle • Helios DAC • Hog • Hokuyo • Hold • Import Select • In • Info • Interpolate • Introduction To CHOPs id • Inverse Curve • Inverse Kin • Join • Joystick • Keyboard In • Keyframe • Kinect Azure • Kinect • Lag • Laser • Leap Motion • Leuze ROD4 • LFO • Limit • Logic • Lookup • LTC In • LTC Out • Math • Merge • MIDI In • MIDI In Map • MIDI Out • Mouse In • Mouse Out • NatNet In • Noise • Null • Object • Oculus Audio • Oculus Rift • OpenVR • OSC In • OSC Out • Out • Override • Panel • Pangolin • Parameter • Pattern • Perform • Experimental:Phaser • Pipe In • Pipe Out • PosiStageNet • Pulse • RealSense • Record • Rename • Render Pick • Reorder • Replace • Resample • S Curve • Scan • Script • Select • Sequencer • Serial • Shared Mem In • Shared Mem Out • Shift • Shuffle • Slope • SOP to • Sort • Speed • Splice • Spring • Stretch • Stype • Switch • Sync In • Sync Out • Tablet • Time Slice • Timeline • Timer • TOP to • Touch In • Touch Out • Trail • Transform • Transform XYZ • Trigger • Trim • Warp • Wave • Experimental:WrnchAI • ZED

An Operator Family which operate on Channels (a series of numbers) which are used for animation, audio, mathematics, simulation, logic, UI construction, and many other applications.

The Graphics Processing Unit. This is the high-speed, many-core processor of the graphics card/chip that takes geometry, images and data from the CPU and creates images and processed data.

The operating system's holder of files and other folders (directories). It does not refer to operators within TouchDesigner. See Network Path.

An Operator Family that creates, composites and modifies images, and reads/writes images and movies to/from files and the network. TOPs run on the graphics card's GPU.

A Time Slice is the time from the last cook frame to the current cook frame. In CHOPs it is the set of short channels that only contain the CHOP channels' samples between the last and the current cook frame.

A parameter in most CHOPs that restricts which channels of that CHOP will be affected. Normally all channels of a CHOP are affected by the operator.

samples-per-second of a CHOP. Each CHOP in your network has a sample rate, whether it is used or not. The overall timeline has a "frame rate", which is the number of animation frames per second, generally your monitor display frequency.

Exporting is the connection of CHOP channels to parameters of operators. The output of each exporting CHOP is one or more channels, active only while the CHOP Viewer is on. The current value of a channel can be exported to a parameter of any operator, overriding that parameter's value. See Parameter.

An Operator Family that manipulates text strings: multi-line text or tables. Multi-line text is often a command Script, but can be any multi-line text. Tables are rows and columns of cells, each containing a text string.

The location of an operator within the TouchDesigner environment, for example, /geo1/torus1, a node called torus1 in a component called geo1. The path / is called Root. To refer instead to a filesystem folder, directory, disk file or http: address, see Folder.

Parameters in TouchDesigner are an operator's settings (numbers, menus, flags, strings, paths) which the user can alter. Parameters for any operator can be opened by right-clicking on the operator and selecting "Parameters..." A currently selected operator's parameters can be viewed in a Network Editor by pressing the keyboard shortcut 'p'.

TouchDesigner is a hierarchy of components. "root" is the top-most network in the hierarchy. The Path is simply /. A typical path is /project1/moviein1.