Kinecting the Dots: Adding Buttons to your Kinect Application

Introduction

The official Kinect SDK has been out for a while now and I haven’t seen many people actually doing “how-to” post to get others started developing for it. I decided that I would help kick-start the movement by creating a series called, “Kinecting the Dots”. This is going to be a series of blog posts covering questions or concerns that I’ve seen in the community.  I am planning on answering questions so if  you have one that you want answered then please contact me by using the form above.

First a few post by me that will help you get started developing for Kinect. I highly recommend the first one.

The busy developers guide to the Kinect SDK Beta
3 Incredibly Useful Projects to jump-start your Kinect Development.

Question:

How do you get create a button that can be used in your Kinect Applications? I would like my users to be able to interact with it inside of the application.

Answer:

There are a lot of ways you can implement a custom button that your user can interact with in a Kinect Application. The one that we are going to do today is part of the Coding4Fun Kinect Toolkit. This toolkit provides a button called “Hover Button”. The only problem is that some people can’t figure it out because it does not have any documentation.

Let’s get started:

Download and install the following software.

The  official Kinect for Windows SDK beta – You can either download a 32 or 64 bit SDK depending on your OS.
Coding4Fun Kinect Toolkit – Lots of extension methods and controls for WPF and WinForms – we are only interested in the Hover Button at this point.
KinectContrib is a set of VS2010 Templates that will help you get started building a Kinect project very quickly.
GUI Buttons for Active/Hover – Not required but some great sample buttons with a free license on them.

Source Code to Sample Application

If you just want the source code to this application then it is located here.

Fire up Visual Studio

Start Visual Studio 2010 SP1 and go to File-> New Project and select Windows->Kinect->KinectSkeletonApplication. You can go ahead and give it a name if you wish.

As soon as the project loads, we are going to add a reference to add a reference to the Coding4Fun Kinect Toolkit file named Coding4Fun.Kinect.Wpf.dll. You should have these files if you downloaded the Coding4Fun Kinect Toolkit mentioned earlier.

Since we are going to add some buttons to our project. Let’s go ahead and download some premade button images that have a active and inactive state. I used the pack from here.

Create a folder in your project named “Resource” and copy all of the buttons to that folder. We are only going to use one set, but you may find the others useful later. Your solution should look like this:

XAML

This template comes with a lot of code to get started already. We are simply going to add the Hover Button and it’s namespace. Our MainPage.xaml should look like the following once complete:

<Window x:Class="KinectingTheDotsUserControl.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:Controls="clr-namespace:Coding4Fun.Kinect.Wpf.Controls;assembly=Coding4Fun.Kinect.Wpf" Title="MainWindow" Height="600" Width="800">
    <Grid x:Name="theGrid">
        <Image Name="videoImage"></Image>
        <Canvas Background="Transparent">
            <Ellipse Fill="Red" Height="20" Width="20" Name="leftHand" Stroke="White" />
            <Ellipse Fill="Red" Height="20" Width="20" Name="rightHand" Stroke="White" />
            <Ellipse Fill="Red" Height="20" Width="20" Name="head" Stroke="White" />
            <Controls:HoverButton x:Name="kinectButton" ImageSize="100" ImageSource="/Resources/RedButton-Hover.png"  ActiveImageSource="/Resources/RedButton-Active.png" IsTriggeredOnHover="True" Canvas.Left="631" Canvas.Top="34" />
        </Canvas>
    </Grid>
</Window>

We set the initial ImageSource and the ActiveImageSource on the Hover Button. We also set the IsTriggeredOnHover to true to Trigger the button when something is hovered onto it.

CODE BEHIND

Now let’s add some code behind. I am only going to display the code that I added. This template comes with the skeleton code already in place.

Please NOTE that I grabbed a chunk of code out of the Jellybean project located here for this demo.

First things First

I added a Click Event handler for the Hover Button.

 kinectButton.Click += new RoutedEventHandler(kinectButton_Clicked);

Then I wired up my event handler as shown below.

void kinectButton_Clicked(object sender, RoutedEventArgs e)
{
   TextBlock textBlock = new TextBlock();
    textBlock.Text = "Triggered";
    textBlock.FontSize = 42;
    theGrid.Children.Add(textBlock);
}

This is simply going to throw a TextBlock onto the Grid and display the word “Triggered”.

I then took JellyBean’s code for “CheckButton” and eliminated the UserControl and used the Ellipse that is included in the KinectContrib Skeleton Template.

private static void CheckButton(HoverButton button, Ellipse thumbStick)
{
    if (IsItemMidpointInContainer(button, thumbStick))
    {
        button.Hovering();
    }
    else
    {
        button.Release();
    }
}

I then took the next two methods directly out of the JellyBean project as they will determine if an Item is MidPoint in a Container.

public static bool IsItemMidpointInContainer(FrameworkElement container, FrameworkElement target)
{
    FindValues(container, target);

    if (_itemTop < _topBoundary || _bottomBoundary < _itemTop)
    {
        //Midpoint of target is outside of top or bottom
        return false;
    }

    if (_itemLeft < _leftBoundary || _rightBoundary < _itemLeft)
    {
        //Midpoint of target is outside of left or right
        return false;
    }

    return true;
}


private static void FindValues(FrameworkElement container, FrameworkElement target)
{
    var containerTopLeft = container.PointToScreen(new Point());
    var itemTopLeft = target.PointToScreen(new Point());

    _topBoundary = containerTopLeft.Y;
    _bottomBoundary = _topBoundary + container.ActualHeight;
    _leftBoundary = containerTopLeft.X;
    _rightBoundary = _leftBoundary + container.ActualWidth;

    //use midpoint of item (width or height divided by 2)
    _itemLeft = itemTopLeft.X + (target.ActualWidth / 2);
    _itemTop = itemTopLeft.Y + (target.ActualHeight / 2);
}

The very last thing to do was wire CheckButton inside of the SkeletonFrameReady event.

CheckButton(kinectButton, rightHand);

This will simply allow me to tie the rightHand Ellipse to the kinectButton that I placed on the Canvas earlier.

THE APPLICATION

You will see we have a Kinect Application with three red ellipse.

1. Head
2. Left Hand
3. Right Hand

When my right hand moved the Red Ellipse over the Hover Button it triggered the click event handler and displayed the TextBlock with the text “Triggered” in it.

I can now make a button more similar to the official ones used in the XBOX Kinect Games by using the following XAML.

 <Controls:HoverButton x:Name="kinectButton" ImageSize="100" ImageSource="/Resources/RedButton-Hover.png"  ActiveImageSource="/Resources/RedButton-Active.png" TimeInterval="3000"  Canvas.Left="631" Canvas.Top="34" />

As the user hovers over the button the white bar will begin to fill. Once it hits 100% then your “Click” event will fire.

Conclusion

It really is amazing what you can do with the Kinect SDK so far. I believe in a few months things will get even easier. I hope this helps kick-start your Kinect Development.

Full Source Code is provided here.

TweetFollow mbcrump