Using DirectManipulation with WPF

Or said differently, how to enjoy smooth pinch-to-zoom gestures on a legacy platform that’s not UWP (no pun intended).

If you have done any UI developement on Windows, chances are you have used a ScrollViewer at some point. If you use the UWP version of ScrollViewer you will notice that in addition to, well, scroll events the control also natively handles zooming. In the case of WPF however, this capability is completely absent.

The main reason for this is that WPF was created at a time where neither precision trackpad nor touchscreen were ubiquitous (touch was just starting with Windows 8 after all). Rather it was designed with stylus support in mind and some initial iteration for inking recognition. Since WPF has been put on ice until recently, it never got a fresh coat of paint on its input APIs and things have indeed quite changed since then.

For starter, the low-level input strategy on Windows is now different. Under the hood of Win32, input events coming from trackpad, touch, stylus and others are now all unified under a single WM_POINTER event type where previously they were separate. If the app doesn’t have support for it however, the OS will try to convert those events and gestures into a somewhat equivalent combination that the app might recognize.

For our case (zooming and more precisely pinch to zoom gestures), that compatibility combination is a “Ctrl + Mouse Wheel Up/Down” event which is a standard pattern that apps could implement before. The trouble is that the mouse wheel event that’s sent is only in increment/decrement of a default value which means you can only implement “jagged” zoom behaviors (least you feel like animating everything yourself). This would look a little bit like this:

container.PreviewMouseWheel += HandleZoomGesture;

void HandleZoomGesture (object sender, MouseWheelEventArgs e)
{
    // Input emulation will make it look like the Ctrl key is pressed
    // when converting the pinch-to-zoom gesture
    if ((Keyboard.Modifiers & ModifierKeys.Control) == 0)
        return;

    var zoomDelta = ZoomGestureDelta * (((double)e.Delta) / Mouse.MouseWheelDeltaForOneLine);
    SetZoom (currentZoom + zoomDelta);
    e.Handled = true;
}

This compatibility behavior might work for your case and, as an example, is what some browsers still implement (at least Firefox as I’m writing this) or also the Visual Studio text editor. If you would like something better however read on.

As it turns out, WPF did have its input subsystem updated to use this new unified WM_POINTER event type but it was never enabled by default and still today sits under an appcontext switch (Switch.System.Windows.Input.Stylus.EnablePointerSupport). Despite using the new events, that updated stack also willingly filters out scenario that are not strictly coming from touch or stylus thus completely bypassing the added support for precision trackpad.

Fortunately, there is a way out. Likely to allow external apps that are not UWP based to still benefit from this gesture support (aka browsers), a generic low-level API does exists going by the name of DirectManipulation. This API operates at the HWND level and works by having you pass on those WM_POINTER messages who then get processed on a different thread with DirectManipulation handling all the heavy lifting of recognizing gestures and calculating the right transformation to apply.

It’s normally supposed to be used together with DirectComposition which is another low-level API to interact with Windows compositor but you can still make it work manually by simply extracting the values you need (aka content scale).

The DirectManipulation API is not provided out of the box on the framework but you can use SharpDX which has bindings for it or a tool like tlbimp on the IDL of the library. Or you can trust me with this binding I made.

For your convenience, here is a “minimal” wrapper class, inspired from Chrome’s usage, that implements and use a subset of DirectManipulation so that you can plug it into your WPF application (including the support to understand WM_POINTER events):

using System;
using System.Runtime.InteropServices;
using System.Windows;
using System.Windows.Interop;
using DirectManipulation;

class PointerBasedManipulationHandler : IDirectManipulationViewportEventHandler, IDisposable
{
    const int ContentMatrixSize = 6;
    // Actual values don't matter
    static tagRECT DefaultViewport = new tagRECT { top = 0, left = 0, right = 1000, bottom = 1000 };

    HwndSource hwndSource;
    IDirectManipulationManager manager;
    IDirectManipulationViewport viewport;
    Size viewportSize;

    uint viewportEventHandlerRegistration;
    float lastScale;
    float[] matrix = new float[ContentMatrixSize];
    IntPtr matrixContent;

    float lastTranslationX, lastTranslationY;

    public PointerBasedManipulationHandler ()
    {
        matrixContent = Marshal.AllocCoTaskMem (sizeof (float) * ContentMatrixSize);
    }

    public event Action<float> ScaleUpdated;
    public event Action<float, float> TranslationUpdated;

    public HwndSource HwndSource {
        get => hwndSource;
        set {
            var first = hwndSource == null && value != null;
            var oldHwndSource = hwndSource;
            if (oldHwndSource != null)
                oldHwndSource.RemoveHook (WndProcHook);
            if (value != null)
                value.AddHook (WndProcHook);
            this.hwndSource = value;
            if (first && value != null)
                InitializeDirectManipulation ();
        }
    }

    IntPtr Window => hwndSource.Handle;

    void InitializeDirectManipulation ()
    {
        this.manager = (IDirectManipulationManager)Activator.CreateInstance (typeof (DirectManipulationManagerClass));
        var riid = typeof (IDirectManipulationUpdateManager).GUID;
        riid = typeof (IDirectManipulationViewport).GUID;
        this.viewport = manager.CreateViewport (null, Window, ref riid) as IDirectManipulationViewport;

        var configuration = DIRECTMANIPULATION_CONFIGURATION.DIRECTMANIPULATION_CONFIGURATION_INTERACTION
            | DIRECTMANIPULATION_CONFIGURATION.DIRECTMANIPULATION_CONFIGURATION_SCALING
            | DIRECTMANIPULATION_CONFIGURATION.DIRECTMANIPULATION_CONFIGURATION_TRANSLATION_X
            | DIRECTMANIPULATION_CONFIGURATION.DIRECTMANIPULATION_CONFIGURATION_TRANSLATION_Y
            | DIRECTMANIPULATION_CONFIGURATION.DIRECTMANIPULATION_CONFIGURATION_TRANSLATION_INERTIA;
        viewport.ActivateConfiguration (configuration);
        viewportEventHandlerRegistration = viewport.AddEventHandler (Window, this);
        viewport.SetViewportRect (ref DefaultViewport);
        viewport.Enable ();
    }

    public void Dispose ()
    {
        viewport.RemoveEventHandler (viewportEventHandlerRegistration);
        Marshal.FreeCoTaskMem (matrixContent);
        HwndSource = null;
    }

    public void SetSize (Size size)
    {
        this.viewportSize = size;
        viewport.Stop ();
        var rect = new tagRECT {
            left = 0,
            top = 0,
            right = (int)size.Width,
            bottom = (int)size.Height
        };
        viewport.SetViewportRect (ref rect);
    }

    // Our custom hook to process WM_POINTER event
    IntPtr WndProcHook (IntPtr hwnd, int msg, IntPtr wParam, IntPtr lParam, ref bool handled)
    {
        if (msg == WM_POINTERDOWN || msg == DM_POINTERHITTEST) {
            var pointerID = GetPointerId (wParam);
            var pointerInfo = default (POINTER_INFO);
            if (!GetPointerInfo (pointerID, ref pointerInfo))
                return IntPtr.Zero;
            if (pointerInfo.pointerType != POINTER_INPUT_TYPE.PT_TOUCHPAD &&
                pointerInfo.pointerType != POINTER_INPUT_TYPE.PT_TOUCH)
                return IntPtr.Zero;

            viewport.SetContact (pointerID);
        } else if (msg == WM_SIZE && manager != null) {
            if (wParam == SIZE_MAXHIDE
                || wParam == SIZE_MINIMIZED)
                manager.Deactivate (Window);
            else
                manager.Activate (Window);
        }
        return IntPtr.Zero;
    }

    void ResetViewport (IDirectManipulationViewport viewport)
    {
        viewport.ZoomToRect (0, 0, (float)viewportSize.Width, (float)viewportSize.Height, 0);
        lastScale = 1.0f;
        lastTranslationX = lastTranslationY = 0;
    }

    public void OnViewportStatusChanged ([In, MarshalAs (UnmanagedType.Interface)] IDirectManipulationViewport viewport, [In] DIRECTMANIPULATION_STATUS current, [In] DIRECTMANIPULATION_STATUS previous)
    {
        if (previous == current)
            return;

        if (current == DIRECTMANIPULATION_STATUS.DIRECTMANIPULATION_READY)
            ResetViewport (viewport);
    }

    public void OnViewportUpdated ([In, MarshalAs (UnmanagedType.Interface)] IDirectManipulationViewport viewport)
    {
    }

    public void OnContentUpdated ([In, MarshalAs (UnmanagedType.Interface)] IDirectManipulationViewport viewport, [In, MarshalAs (UnmanagedType.Interface)] IDirectManipulationContent content)
    {
        content.GetContentTransform (matrixContent, ContentMatrixSize);
        Marshal.Copy (matrixContent, matrix, 0, ContentMatrixSize);

        float scale = matrix[0];
        float newX = matrix[4];
        float newY = matrix[5];

        if (scale == 0.0f)
            return;

        var deltaX = (newX - lastTranslationX);
        var deltaY = (newY - lastTranslationY);

        bool ShallowFloatEquals (float f1, float f2)
            => Math.Abs (f2 - f1) < float.Epsilon;

        if ((ShallowFloatEquals (scale, 1.0f) || ShallowFloatEquals (scale, lastScale))
            && (Math.Abs (deltaX) > 1.0f || Math.Abs (deltaY) > 1.0f)) {
            TranslationUpdated?.Invoke (-deltaX, -deltaY);
        } else if (!ShallowFloatEquals (lastScale, scale)) {
            ScaleUpdated?.Invoke (scale);
        }

        lastScale = scale;
        lastTranslationX = newX;
        lastTranslationY = newY;
    }

    #region Native methods
    const int WM_POINTERDOWN = 0x0246;
    const int DM_POINTERHITTEST = 0x0250;
    const int WM_SIZE = 0x0005;

    static readonly IntPtr SIZE_MAXHIDE = new IntPtr (4);
    static readonly IntPtr SIZE_MINIMIZED = new IntPtr (1);

    uint GetPointerId (IntPtr wParam) => (uint)(unchecked((int)wParam.ToInt64 ()) & 0xFFFF);

    [DllImport ("user32.dll", EntryPoint = "GetPointerInfo", SetLastError = true)]
    internal static extern bool GetPointerInfo ([In] uint pointerId, [In, Out] ref POINTER_INFO pointerInfo);

    [StructLayout (LayoutKind.Sequential, CharSet = CharSet.Unicode)]
    internal struct POINTER_INFO
    {
        internal POINTER_INPUT_TYPE pointerType;
        internal uint pointerId;
        internal uint frameId;
        internal POINTER_FLAGS pointerFlags;
        internal IntPtr sourceDevice;
        internal IntPtr hwndTarget;
        internal POINT ptPixelLocation;
        internal POINT ptHimetricLocation;
        internal POINT ptPixelLocationRaw;
        internal POINT ptHimetricLocationRaw;
        internal uint dwTime;
        internal uint historyCount;
        internal int inputData;
        internal uint dwKeyStates;
        internal ulong PerformanceCount;
        internal POINTER_BUTTON_CHANGE_TYPE ButtonChangeType;
    }

    [StructLayout (LayoutKind.Sequential, CharSet = CharSet.Unicode)]
    internal struct POINT
    {
        internal int X;
        internal int Y;
    }

    internal enum POINTER_INPUT_TYPE : uint
    {
        PT_POINTER = 0x00000001,
        PT_TOUCH = 0x00000002,
        PT_PEN = 0x00000003,
        PT_MOUSE = 0x00000004,
        PT_TOUCHPAD = 0x00000005
    }

    [Flags]
    internal enum POINTER_FLAGS : uint
    {
        POINTER_FLAG_NONE = 0x00000000,
        POINTER_FLAG_NEW = 0x00000001,
        POINTER_FLAG_INRANGE = 0x00000002,
        POINTER_FLAG_INCONTACT = 0x00000004,
        POINTER_FLAG_FIRSTBUTTON = 0x00000010,
        POINTER_FLAG_SECONDBUTTON = 0x00000020,
        POINTER_FLAG_THIRDBUTTON = 0x00000040,
        POINTER_FLAG_FOURTHBUTTON = 0x00000080,
        POINTER_FLAG_FIFTHBUTTON = 0x00000100,
        POINTER_FLAG_PRIMARY = 0x00002000,
        POINTER_FLAG_CONFIDENCE = 0x000004000,
        POINTER_FLAG_CANCELED = 0x000008000,
        POINTER_FLAG_DOWN = 0x00010000,
        POINTER_FLAG_UPDATE = 0x00020000,
        POINTER_FLAG_UP = 0x00040000,
        POINTER_FLAG_WHEEL = 0x00080000,
        POINTER_FLAG_HWHEEL = 0x00100000,
        POINTER_FLAG_CAPTURECHANGED = 0x00200000,
        POINTER_FLAG_HASTRANSFORM = 0x00400000,
    }

    internal enum POINTER_BUTTON_CHANGE_TYPE : uint
    {
        POINTER_CHANGE_NONE,
        POINTER_CHANGE_FIRSTBUTTON_DOWN,
        POINTER_CHANGE_FIRSTBUTTON_UP,
        POINTER_CHANGE_SECONDBUTTON_DOWN,
        POINTER_CHANGE_SECONDBUTTON_UP,
        POINTER_CHANGE_THIRDBUTTON_DOWN,
        POINTER_CHANGE_THIRDBUTTON_UP,
        POINTER_CHANGE_FOURTHBUTTON_DOWN,
        POINTER_CHANGE_FOURTHBUTTON_UP,
        POINTER_CHANGE_FIFTHBUTTON_DOWN,
        POINTER_CHANGE_FIFTHBUTTON_UP
    }
    #endregion
}

To use the helper, instantiate it somewhere and pass it the HwndSource where your zoomable content is hosted. You can do so using the PresentationSource.AddSourceChangedHandler method like so:

PointerBasedManipulationHandler manipulationHandler = new PointerBasedManipulationHandler ();

PresentationSource.AddSourceChangedHandler (container, HandleSourceUpdated);

void HandleSourceUpdated (object sender, SourceChangedEventArgs e)
{
    if (manipulationHandler != null && e.NewSource is System.Windows.Interop.HwndSource newHwnd)
        manipulationHandler.HwndSource = newHwnd;
}

Finally, subscribe to the handler’s ScaleUpdated / TranslationUpdated events to react to DirectManipulation changes. For instance you can use a ScaleTransform as your container RenderTransform and set the corresponding values when you receive the event. Also don’t forget to set the size of your canvas by calling the SetSize method (for instance wired up from a SizeChanged event).

Now (at the end of the post) here is the final important caveat. If you want any of this DirectManipulation integration to work you are going to have to do one of two things. Because of the way DirectManipulation employs input delegation, you cannot have the legacy WPF stylus input stack run inside your application (which is the default). As such you have two options:

Disable native stylus/touch support and instead only rely on DirectManipulation: set the Switch.System.Windows.Input.Stylus.DisableStylusAndTouchSupport=true switch in you app.config or programmatically.
Enable the pointer-based stylus/touch input stack which is compatible with DirectManipulation: set the Switch.System.Windows.Input.Stylus.EnablePointerSupport=true switch in your app.config or programmatically (early on).

Blog

Using DirectManipulation with WPF

Jérémie Laval

Join Our Newsletter. No Spam, Only the good stuff.

Related