Timeline
Timeline
First this happens, than that, and then this other thing… You’re going to need Timeline.
If at some point things / characters other than your player are having an interaction, Timeline can help you out. If you want to have your character be part of a scripted interaction, Timeline can help you out (after some extending, see below). If you want to have a whole bunch of objects move in a specific way at a specific time... Timeline.
Changing sections of these stories is easy too. For example: you want to show your participants a conversation between two virtual people. You can put the animations and audio-clips for both characters on two Timeline tracks inside one Timeline Playable. If you now want to have another version of this same conversation (for instance in another condition), you can simply duplicate the entire Timeline track, and change only the parts you want. (Keep in mind that the 'binding' done by the Playable Director is limited to the Scene they're in, so that if you want to re-use these completely, they should both be in the same Scene, or the Scene should be direct duplicates of one another.)
The basics of Timeline can be learned here, while a more in-depth tutorial will help you further.
Timeline is great, but fairly limited. That is, until you get to extending its features.
Extending Timeline functionality
Adding custom tracks to Timeline allows this part of Unity to truly shine. Out of the box Timeline is not very versatile, and even the things that you are allowed to do are done in a limited way. You can for example use Animation clips on Timeline, but you cannot control a specific Animator Controller using this system... at least not yet.
Luckily you can customize Timeline. By extending Timeline you're creating your own custom tracks, and your own custom Playables.
Follow the above video, and this tutorial for a comprehensive guide on how to extend Timeline in a meaningful way.
In this repo you can find some custom Playables. Feel free to use these as a starting point for your work. Also, make sure to copy them to a separate folder, since any updates to the repo might change or break existing functionality!
In the image below you can see a combination of Playables being used: some are included in Unity's implementation of Timeline, while others can be found in the above repo.
General points to consider
In no particular order, here are a few things to consider when working with custom playables:
- The creators of Timeline have taken steps to encapsulate and decouple the various systems that comprise the Custom Playable. In large part they have done an excellent job. However, in some cases they have been a little overzealous, creating needless barriers between sections of code that do really depend on each other. In practice, this means that in some cases you have to get access to components and values in creative ways:
- For example: find the bound Rigidbody in the custom 'RigidbodyPlayable' from the repo above. (The bound object is the component you see linked in the left-hand side of the Track). You can easily access this object... if you're willing to detour a bit. The bound object in this case references a Rigidbody in the Scene. You want to be able to mutate its values from the Clip in the Timeline window. However, to get access to the bound object, you first need to access it from the CreatePlayable method on the RigidbodyTrack class, to then pass it into the RididbodyClip class, from where you can pipe it into the RigidbodyBehaviour class. Once there you can access this useful component.
- Playables are based on the Scriptable Object system. This means that in some capacity, the Clips and Tracks are not linked to the Scene they govern. This is especially important to remember when you're referencing an object or component in the scene. If you want to reference an object in the scene which is not the bound object, you need to get it via an 'ExposedReference<>' call. If you would like to access this component often, I'd suggest to cache it into a private variable (e.g. using the PlayableGraph's Resolver to resolve a 'public ExposedReference<Light> lightReference;' into a 'private Light light' variable).
- Even though Playables are not really part of the Scene, this doesn't mean you can create PlayableDirectors and Timelines outside of a Scene, or move them between Scenes. They retain references to the Scene objects, and can not be easily moved between multiple Scenes.
- Mark the Behaviour class as [Serializable] if you want to be able to create custom inspectors for it.
- A lot of Playables in the repo above allow for easing to change the values gradually (see the slanted lines in the image above) using 'easing'. This is basically the difference between a light-switch and a dimmer-switch. Check the 'FrameData' or 'Playable' variables on the ProcessFrame method, which can both provide you with the inputWeight. Multiply the inputWeight by your desired float/double value to get ease in and ease out.
- Some Playables have an implementation of 'AnimationCurves' (as showcased here). Even though they might not be the most intuitive way to access through code, AnimationCurves are very useful in the Inspector. In the above repo, AnimationCurves are usually multiplied by the inputWeight mentioned before. You could remove the inputWeight multiplication, if you only want to rely on the AnimationCurve values.
- Some Playables allow you to force the clip duration. This is useful if you want to show in the Timeline window exactly how long a behaviour actually lasts. See for a good example the 'ToTargetPlayable' from the above repo. Once you have set the start-point, the end-point, and the move-speed (including any easing), you can see that you cannot drag the duration of the clip manually if 'Force Clip Duration' is enabled. The length of the Clip is the exact amount of time it takes for your bound GameObject to move from the start-position to the end-position.
- See the Animator Playable for a different implementation of Animations in Timeline. In the standard Unity version, you select Animations to be played directly from Timeline, by dragging them into your Timeline window. This however does not allow you to access the Animation system as it is created in the AnimatorController. This does provide issues however, such as when you've already created a character with complex animations and animation transitions, and you want to access this using Timeline. The AnimatorPlayable implementation included in the above repo solves this problem. Instead of forcing a single Animation to be played, you're manipulating the various Parameters (float, bool, trigger or int) in the AnimatorController itself. This way, any transition effects (both between Animations played in Timeline, and between onset and offset of using Timeline) are preserved. It also allows you to use Timeline to set in motion an Animation, which then proceeds on it's own without being influenced by the timescale of the Timeline.
- Two of the custom Playables (Looper and TimelineSpeed) allow you to control what Timeline does in response to events from outside. In these Clips you can tell Timeline to either repeat a specific section, or to pause at a specific point. You can then tell Timeline that another class has the power to break the loop / continue the Timeline. This is an immensely powerful feature, which allows you to have your participants interact with the events on Timeline. For instance, you can get a virtual character (VC) to walk up to an object and point at it. At this point, Timeline pauses (using the TimelineSpeed track which tels the ExecutiveDirector to set this Timeline's speed to 0). Timeline hands control over to another class (a TimelineResumerBase derived class). You could attach this class to the 'OnSelect' call of your XR Interaction system of that object your VC is pointing to. Then, once your participant picks up this item (and thus triggers the OnSelect), Timeline will continue, and your VC will play the animation in which she's asking your participant for said item.
Why do this?
The main benefit of Timeline, and thereby of customising Timeline, is that you can easily create and modify complex scenario's. Once the systems are in place, all you need to do is drag blocks around on your Timeline, and entire scenario's are played out. This makes it a flexible system, and allows you to prototype and create complex interactions with minimal work.
Need help?
If you need help creating Playables, don't hesitate to contact Maarten at SOLO for more information: m.r.struijk@FSW.leidenuniv.nl