Ghosts of the Present


For this performance, I was interested in exploring the body of the individual as a nexus point for receiving and transmitting history. The idea came out of an oral history recording I made with my grandmother and father. In reviewing the recording, I was struck by how different their styles of storytelling were. Each evoked a completely different mood in me as a listener even when talking about the same people and events. In particular, I was drawn to their stories about my grandfather, whom I never met. I realized that my entire understanding of whom he was and how he fits into my understanding of my own history was based on just few stories and photographs.


My grandmother and grandfather on their wedding day.


My grandfather is uniform during the Korean war.


I wanted to create a piece that expressed how historical identity is formed and re-presented. Early in the process I was thinking about the contrast between the two narrators. I drew on Akira Kurosawa’s film Rashomon as inspiration for how to portray a story from multiple perspectives.

Akira Kurosawa, Roshomon

Akira Kurosawa, Roshomon

Later, I started thinking about story telling as a form of compression – condensing complex events and even entire human lives into a few anecdotes. One piece I looked to as an example of how to deal with the concepts of memory and history in a new media format was Timo Wright’s Our Memories are Tomorrow. In this piece the viewer is photographed upon entering the gallery. The image is then sent to a computer that uses a photo destruction program to degrade and eventually destroy the images over the course of the exhibition.


Timo Wright, Our Memories are Tomorrow


My performance consists of two pre-recorded audio narratives and two projected images which are manipulated using flex sensors on the garment that I wear. The recordings and images represent the historical input that the individual receives. The movement of the performer’s body acts as a form of interpretation, which then affects the audio and visual output that the audience perceives.


Natasha Lewandrowski, Still from Ghosts of the Present.

I created the two narratives using segments from the oral history recordings, one told by my grandmother, the other by my father, both focusing on my grandfather’s experiences before, during, and after the Korean War, where he served as a doctor in the army.

My grandmother’s story describes their experiences leading up to and during the war. It climaxes in a battle that my grandfather narrowly survived. My father’s story begins by recounting events that occurred during this battle. He goes on to speculate about how these experiences contributed to my grandfather’s alcoholism and, ultimately, his death.

During the performance, the two recordings play simultaneously. Two images of my grandfather are overlaid and projected onto the wall.  In one image he is in a military setting. In the other he is with family on his wedding day. The movement of the performer affects the volume of each story, and the opacity of each image, such that one story and image become more or less dominant than the other at any given moment.

Technical Description

This design for this project came out of an earlier experiment I did controlling sound with flex sensors on a glove.

For this project, I created a garment that fully covers the entire body including the head, hands, and feet. Two flex sensors are positioned at the elbows of the garment. These are connected to a Lily Pad Arduino located on the chest. The Arduino is loaded with an SD card containing the audio file and an audio output connection. The stories play on two speakers located off-screen. The volume of the left speaker is controlled by one sensor, and the other sensor controls the volume of the right speaker.


Garment in progress.

The image manipulation is done in another program called Processing. The Arduino sends the sensor values to Processing via a USB cable connect the garment to a computer. The altered image is projected onto the wall.



When I created this piece I wanted it to be about how the individual interprets the available influences to construct a self-history. It was not until I actually performed the piece, that I realized that the most important theme might actually be the individual’s desire to have a physical connection with the past at all and the inability to ever really make that connection. As I reach my hand out toward the image of my grandfather the act of reaching changes what is seen. We are bound to our past in ways we cannot always see. What we do in the present moment changes how we interpret history.

Arduino Code
//MP3 player libraries
#include <SPI.h>
#include <SdFat.h>
#include <SdFatUtil.h>
#include <SFEMP3Shield.h>

SdFat sd;
SFEMP3Shield MP3player;

//Set constants for the trigger input pins
const int TRIG1 = A0;
const int TRIG2 = A4;
const int EN_GPIO1 = A2; // Amp enable + MIDI/MP3 mode select

//storage variables to be used for flex sensor values
int val1 = 0;
int val2 = 0;

//Serial communication stuff
//This section is only necessary if information is being received from the serial port

char val; // Data received from the serial port
//int ledPin = 13; // Set the pin to digital I/O 13
//boolean ledState = LOW; //to toggle our LED

void setup()

//initialize serial communications at a 9600 baud rate

//Set pinMode
//pinMode(ledPin, OUTPUT);
pinMode(TRIG1, INPUT);
pinMode(TRIG2, INPUT);

//start the shield

//Establish contact with Processing
//The establishContact function is declared below.
//Here we are just calling it.
//NOTE If the serial monitor is opened communication with the other program will be garbled
establishContact(); // send a byte to establish contact until receiver responds

//should this go in the loop section?
//Should this go in an if statement? (if Serial.available)?
//start playing track 1, only seems to work with this file name.

//Turn the amplifier chip on

} //end setup

//void loop=====================================================================
void loop()

//MP3 player stuff==============================================================
//set the storage variables equal to the sensor value
val1 = analogRead(TRIG1);
val2 = analogRead(TRIG2);

//Map the sensor values to a range of 0-255
//0 is loudest, 255 is lowest (off)
//The map values of the sensor will change depending on the sensor readings
int val1Map;
int val2Map;

//these are the two values I want to send to Processing
val1Map = map(val1, 600, 1023, 255, 0);
val2Map = map(val2, 700, 1023, 255, 0);

//For debugging=========================================================================
//Don’t open the serial monitor while communication is in effect or it will be garbled
//print the sensor and map values to the serial monitor to check values
Serial.print(“val1: “);
Serial.print(“val1map: “);
Serial.print(“val2: “);
Serial.print(“val2Map: “);
Serial.println(“hello world”);
//set MP3 volume using the mapped sensor values==========================================
MP3player.setVolume(val1Map, val2Map);

//For debugging

//Set both channels to maximum volume

//set one channel to on and control the other channel using the flex sensor
//MP3player.setVolume(0, val2Map);
//serial communication stuff====================================================

//This part is for two way communication.
if (Serial.available() > 0) { // If data is available to read,
val =; // read it and store it in val

//this part is only necessary if we are reading a value from serial
if(val == ‘1’) //if we get a 1
ledState = !ledState; //flip the ledState
digitalWrite(ledPin, ledState);

//If no data is received, send the flex sensor data to Processing.
else {

//convert val1 and val2 to a new ints called ‘r’ and ‘t’
int r = val1;
int t = val2;

//send the data to processing
//separate values with commas
//this will be used to separate out the values on the other end
//use println after the last item
//Processing will be looking for ‘\n’ to know when the packet is over

//set a delay to avoid garbled communication

}//end loop

//void establishContact=========================================================
//this function is called in void setup
void establishContact() {
while (Serial.available() <= 0) {

//check this if there is an error. It might have to be (“0,0,0”);
Serial.println(“A”); // send a capital A
}//end establishContanct

Processing Code

//In this sketch I am controlling the opacity of two images based on
//two sensor values I receive from Arduino.

import processing.serial.*; //import the Serial library

//Image variables=======================================
//declare a variable of type PImage for each image
PImage myImg1;
PImage myImg2;

//make variables for tint and alpha
//these will be filled with the sensor valrues from Arduino
int myAlpha1;
int myAlpha2;

//Serial stuff========================================
Serial myPort; //the Serial port object

// since we’re doing serial handshaking,
// we need to check if we’ve heard from the microcontroller
boolean firstContact = false;

void setup() {

//make our canvas the size of the display
size(displayWidth, displayHeight);

//load the images in setup
myImg1 = loadImage(“img1.jpg”);
myImg2 = loadImage(“img2.jpg”);

//print the available serial ports to the console
println(“Available serial ports: “);

// initialize your serial port and set the baud rate to 9600
myPort = new Serial(this, Serial.list()[4], 9600);
myPort.bufferUntil(‘\n’); //”n” because the last character of println is “n” ?
void draw() {

background(0); //set the background to black
imageMode(CENTER); //position images from center of image

//this is where I use the sensor values to control the alpha of the images
tint(myAlpha2, myAlpha2);
image(myImg2, displayWidth/2, displayHeight/2); //draw image 2, center image on screen

tint(myAlpha1, myAlpha1/2);
image(myImg1, displayWidth/2, displayHeight/2); //draw image 1, center image on screen
//This is where the serial communication programing happens
void serialEvent( Serial myPort) {

//read the serial port
String myString = myPort.readStringUntil(‘\n’);

//trim the data
myString = trim(myString);

//so we can see the length of the array
import java.util.Arrays;

//convert the sensor values to ints and split the string at the commas

String[] sensorsStr = split(myString, ‘,’);
//println (sensorsStr.length);

if (sensorsStr.length>1){
int r = int(sensorsStr[0]);
int t = int(sensorsStr[1]);

println(“r: ” + r);
println(“t: ” + t);
myAlpha1 = int(map(r, 0, 1023, 0, 255));
myAlpha2 = int(map(t, 0, 1023, 0, 255));
println(“myAlpha1: ” + myAlpha1);
println(“myAlpha2: ” + myAlpha2);


//look for our ‘A’ string to start the handshake
//if it’s there, clear the buffer, and send a request for data
if (firstContact == false) {

//if getting one variable
//if (valr.equals(“A”)) {

//if getting multiple values
if (myString.equals(“A”)){

firstContact = true;
else { //if we’ve already established contact, keep getting and parsing data

if (mousePressed == true)
{ //if we clicked in the window
myPort.write(‘1’); //send a 1

// when you’ve parsed the data you have, ask for more:
} //end else
} //end serialEvent

Circuit Diagram



One thought on “Ghosts of the Present

  1. Pingback: Ghosts of the Present | Major Studio 1 - PGTE 5200 - E

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s