Physical Interaction Design

From Notebook
Jump to: navigation, search

Learning Diary

Day one

input --> process --> output --> input --> process --> output..........

Bret victor - Brief rant on the future of physical interaction design

I/o of the human


  • 5 senses
  • balance, temp, body, time, etc.


  • voluntary: movements, sound, facial, expression,
  • involuntary: physiology, reflexes

Computer I/o


  • Low level sensors: movement, distance, force/bend, environment, light, sound etc.
  • devices: kinect, leap, webcam, biosignals etc.


  • anything.

Machine Learning

Computer system learns from example data (training data) to predict new data.

We will mainly be using supervised learning -

  • training data includes information about inputs and outputs
  • as opposed to unsupervised learning, where we want to know more about unlabelled data
  • other types learning like - reinforcement learning - ants navigate a colony by finding more food.

Regression is fitting mathematical equations to the data set. (on wekinator set classifier to all continuous)

To keep in mind -

  • what kind of task: Classification vs regression
  • what kind of relationship in data: Choice of model

Wekinator Basics

The Wekinator is a free software package developed by Rebecca Fiebrink to facilitate rapid development of and experimentation with machine learning in live music performance and other real-time domains. The Wekinator allows users to build interactive systems by demonstrating human actions and computer responses, rather than by programming. It can be downloaded from

Use unique port numbers. Match no. of inputs and outputs.

Day 2

Arduino connected to pd connected to wekinator to train sensor responses.

Using this patch we can set up serial communication between the arduino and pd.

Serial to pd.png

Connecting 2 sensors to pd and training them with wekinator.

Wekinator 2sensors pd.png

Day 3

Experimenting with LEAP and Kinect.

For kinect, download delicode NImate. For older macs, synapse also works.

Day 4

Put that there MIT 1980,

Dynamic time warping for gestures.