NOMNOM 2: The Video Machine – The Physical Computing Aspects of the Project

 

NOMNOM: The Video Machine
NOMNOM: The Video Machine

Intent

The purpose of this project was to allow users to play music (or a DJ set) using videos from YouTube.

NOMNOM is an advanced version of The Video Machine presented for the mid-term. It controls the playback of videos presented on a web browser.
By pressing a button on the controller, the correlated video is being played on the screen and heard through the speakers. The videos are being played in sync with one another. Only the videos that are being played, are being heard.

On the new version, The Video Machine controller offers four functions that allow making changes to the way the videos are being played:

  • Repetition – Affects The number of times a video is being played during a single loop (1-4 times).
  • Volume – Affects the volume of the selected video.
  • Speed – Changes the speed of the selected video.
  • Trim – Trims the length of the selected video.

The first prototype of the new version in action –

NomNom: The Video Machine

Main Objectives

The goal was to gain a few critical improvements from the previous versions of the product. After brainstorming for possible improvements, and reviewing the feedback we had received, the following objectives were chosen:

  • To keep it simple, while introducing more functionality – One of the major strength of the original version was its simplicity. We were able to achieve a design that allowed simple and self-explanatory interaction, that was enjoyable for both experienced DJs and users with zero experience.
    For the new version, new features, such as a consistent and predictable playback sequence, an automatic beat-sync between the played videos, the ability to change the number of times a video will be played over a single loop, and the ability to change the playback properties for each one of the videos while it is being played.
    The new features of the new version allow the user the achieve great results more easily, by using the same simple controls of the old version. A total of 6 new features and improvements were added to the product while adding only a single rotary switch to the previous layout.

    The new features of the new version allow the user the achieve great results more easily, by using the same simple controls of the old version.

  • To make it feel solid – The first impression the user has on a product comes from looking at it. NOMNOM was built from solid materials in order to allow the user to feel free to physically interact with it. The solidity of the controls freed up users from thinking about the physical interaction and concentrating on the content (the video and the sound).

NOMNOM: The new version

  • To smoothen the controls – Enjoyable interaction cannot be achieved only by providing a fast and easy way to complete a task. The time the user spends using the product should be enjoyable as well.
    Is order to build a smooth and fun tangible interaction, a research was done around different potentiometers, buttons, and switches. Eventually, the controls that provide the best ‘feel’, and that were the most accurate, were chosen.

  • To take further development in current considerations – In most cases, the ability to innovate comes from deep understanding of the way a certain system works. To allow further development, there was a need to build the product in a way that will make it be easy to learn and to understand, to both for us and for other future contributors. Therefore, an effort was done to design and build the inner parts of the box in a way that will be very understandable for anyone who reveals it.
    The design of the structure of the internal electronic parts, not only allowed clarity on the debugging stages, but also fast analysis and understanding of the implications of any change or addition.
NOMNOM: Designing the inner structure
NOMNOM: Designing the inner structure

There was a need to build the product in a way that will make it be easy to learn and to understand, to both for us and for other future contributors. Therefore, an effort was done to design and build the inner parts of the box in a way that will be very understandable for anyone who reveals it.

NOMNOM: In the making of
NOMNOM: In the making of

Decision-Making and challenges

Design Overview

Leaning on the design of the previous version, we made a few improvements to our electric circuits, and a few major improvements to our physical interface design.

NOMNOM: Schematic
NOMNOM: Schematic

Doing More With the Same Buttons

On of the major limitations of the first version was that in order change the playback mode (properties / attributes) of a video, the user had to stop the playback, make the changes using the knobs, and start the playback again. Therefore, one of the most important features of the new version, was the ability to change the playback mode (properties / attributes) of a single video while the video is being played.

To avoid adding a series of knobs for each on of the videos, the existing buttons are being used for two functions:

NOMNOM: A single press to start / stop
A single press to start / stop
NOMNOM: Press & hold to make changes to the video playback
Press & hold to make changes to the video playback

 

 

 

 

 

 

The component that was used for the buttons is the Adafruit Trellis, a single PCB that connects 16 press buttons.

The Trellis PCB and its Arduino library support two modes:

MOMENTARY – A mode on which button press event is detected only a buttons is being held down.
LATCHING – A mode on which button press event changes the state of the button (e.g. from ON to OFF).

NOMNOM: One of the challenges was to make the Trellis PCB support both of its different modes at the same time
NOMNOM: One of the challenges was to make the Trellis PCB support both of its different modes at the same time

One problem was that by default, the Trellis can operate on only one of these modes at the time.
Another challenge was to find an efficient way (in terms of performance) to read the button states, so the controller will be very responsive to the user actions — the changes on the screen, and on the LEDs on the controller should be immediate.

After 3-4 weeks of research on the way the Trellis PCB works and coding different experiments, the following Arduino code allowed the support of the two modes simultaneously.


#include 
#include "Adafruit_Trellis.h"

Adafruit_Trellis matrix0 = Adafruit_Trellis();
Adafruit_TrellisSet trellis =  Adafruit_TrellisSet(&matrix0);

#define NUMTRELLIS 1
#define numKeys (NUMTRELLIS * 16)
#define INTPIN A2

int LEDstatus[16] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
int blinkStatus = 1;
int blinkTime = 0;
int buttonPress[16] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
int oldStatus[16] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};

void setup() {
  Serial.begin(9600);
  pinMode(INTPIN, INPUT);
  pinMode(5, INPUT);
  pinMode(6, INPUT);
  pinMode(7, INPUT);
  pinMode(8, INPUT);
  digitalWrite(INTPIN, HIGH);

  trellis.begin(0x70);  // only one trellis is connected

  // light up all the LEDs in order
   for (uint8_t i = 0; i < numKeys; i++) {
     trellis.setLED(i);
     trellis.writeDisplay();
     delay(50);
   }

  // then turn them off
  for (uint8_t i = 0; i < numKeys; i++) {
    trellis.clrLED(i);
    trellis.writeDisplay();
    delay(50);
  }
  while (Serial.available() <= 0) {
    Serial.println("hello"); // send a starting message
    delay(300);              // wait 1/3 second
  }
}

void loop() {
  delay(80); // 30ms delay is required, don't remove me!


  /*************************************
  // SENDING DATA TO P5.JS
  *************************************/
  if (Serial.available() > 0) {

      // reading serial from p5.js
      int incoming = Serial.read();

      // print current status
      for (int i = 0; i < 16; i++) {
        Serial.print(LEDstatus[i]);
        Serial.print(",");
      }

      // step knob
      int pot1Value = 0;
      if (digitalRead(5) == HIGH) {
        pot1Value = 4;
      } else if (digitalRead(6) == HIGH) {
        pot1Value = 3;
      } else if (digitalRead(7) == HIGH) {
        pot1Value = 2;
      } else if (digitalRead(8) == HIGH) {
        pot1Value = 1;
      }
      Serial.print(pot1Value);
      Serial.print(",");

      // volume knob
      int pot2Value = analogRead(A1);
      int pot2ValueMapped = map(pot2Value, 0, 1020, 0, 100);
      Serial.print(pot2ValueMapped);
      Serial.print(",");

      // speed knob
      int pot3Value = analogRead(A0);
      int pot3ValueMapped = map(pot3Value, 0, 1020, 0, 100);
      Serial.print(pot3ValueMapped);
      Serial.print(",");

      // cut knob
      int pot4Value = analogRead(A3);
      int pot4ValueMapped = map(pot4Value, 0, 1020, 0, 100);
      Serial.print(pot4ValueMapped);
      Serial.print(",");

      // blink data
      Serial.print(blinkTime);

      Serial.println("");
  }

  /*************************************************
  // CHANGING BUTTON STATES BASED ON BUTTON PRESSES
  **************************************************/
  blinkTime = blinkTime + 1;
  if (blinkTime == 5) {
    blinkTime = 0;
  }

  trellis.readSwitches();
  for (uint8_t n = 0; n < numKeys; n++) {
    if (trellis.justPressed(n)) {
      LEDstatus[n] = 3;

      continue;
    }

      if (LEDstatus[n] == 3) {
        buttonPress[n]++;
        if (blinkTime >= 4) {
          if (trellis.isLED(n)) {
            trellis.clrLED(n);
            trellis.writeDisplay();
            } else {
              trellis.setLED(n);
              trellis.writeDisplay();
            }
        }
      }

    if (trellis.justReleased(n)) {
      if (buttonPress[n] > 8) {
        LEDstatus[n] = 1;
        oldStatus[n] = 1;
        buttonPress[n] = 0;
        trellis.setLED(n);
        trellis.writeDisplay();
      } else {
        buttonPress[n] = 0;
        if (oldStatus[n] == 1) {
          LEDstatus[n] = 0;
          oldStatus[n] = 0;
          trellis.clrLED(n);
          trellis.writeDisplay();
        } else {
          LEDstatus[n] = 1;
          oldStatus[n] = 1;
          trellis.setLED(n);
          trellis.writeDisplay();
        }
      }
    }
  }
}

This code includes a fast and efficient protocol to read the different states from the Trellis board using a single read command, and to communicate them to the web browser using ‘handshaking’.

At a first glance, this code looks simple, but it includes a fast and efficient protocol to read the different states (“ON”, “OFF”, and “Being pressed”, a state that was used to make changes to the video playback) from the Trellis board using a single read command (trellis.readSwitches()), and to communicate them to the web browser using ‘handshaking’.

More about the programming behind NOMNOM can be found on this blog post, and on the project’s GitHub repository.

Finding the Right Potentiometers

As much as the Trellis board was satisfying as our press buttons, the movement of the potentiometers needed an upgrade. A long research and multiple experiments with different types of potentiometers and knobs (mostly from Adafruit, DigiKey) were made. It appeared that the knobs and potentiometers offered by Mammoth Electronics were the smoothest to turn, most built using high-quality materials, and fit best with our design vision.

Fabrications

One of the major objective for the new version was to make the physical interface feel as stable as the software that supports it. The desire was to build the box from more solid materials, which do not feel breakable like wood or delicate like thin acrylic. Therefore, a solid metal enclosure was used to add sense of strength and stability to the overall interaction.

To avoid any ‘shaky’ feeling when interacting with the product, the design of the drilled holes on the enclosure had to be very accurate and tight to the size of the electronic components.

NOMNOM: Design sketch before the drilling process
NOMNOM: Design sketch before the drilling process

User Testing

After building the first fully functional prototype, a user testing phase some light on the strength and weaknesses of the product.

Luckily, the physical interaction worked well and was largely understood by the users. A few changes were done to the terminology – The term “Steps”, which described the number of times a video will be played within a single loop, was changed to “Repetitions”, and the term “Cut”, which described the ability to trim the video, was changed to the term “Trim”.

The rest of the changes, based on the users’ feedback, were done on the graphical user interface, which now includes a much simpler and straight forward indications for each and every video status.

Presenting the Project to New Audience

As part of the process, I presented the product in front of a new audience, outside of the ITP community. This experience allowed us to get feedback from an audience that is closer to our target audience, and helped us to be more prepared for the (intense) presentation at the ITP Winter Show.

\

The ITP Winter Show

NOMNOM: The Video Machine was presented at the ITP Winter Show 2016.

NOMNOM: The Video Machine @ ITP Winter Show 2016
NOMNOM: The Video Machine @ ITP Winter Show 2016

Published by

Dror Ayalon

@drorayalon

Leave a Reply

Your email address will not be published. Required fields are marked *