JESSIE ZHAI

Digital artist

A documentation page of rambling and experimenting.

@jessiehaseyes
jessiejiranzhai@gmail.com
05 TIME
ICM WEEK5



GO TO:
SIMULATION
P5.JS SKETCH



For week 6’s creative assignment SIMULATION, I created an aduio reactive webcam that display pixelizated dots.



PROCESS


This week I decided to explore the webcam fucntion on P5.JS, after following a quick turtorial, I was able to display the webcam in pixelated dots using createCapture function.

By creating a variable called vid, I used the createCapture function to turn on the webcam and drew it on my canvas.
The original video ratio is 640x480 so I resized the canvas to that ratio, while keeping the original vdieo ten times smaller.

In order to access each of the pixels on my canvas, I first used the get() function to get the pixels values in canvas. Variables I and J each access the horizontal and vertical values, and they are represented by rows and columns of ellipses. I accessed the value of each of the ellipse using the get function. Then by mapping the brightness value from 0-100 to 0-255, I used the brightness of the video as a greyscale fill for each of the ellipse.



I switched to a different function called loadPixels(which I’ve used in my CHOICES assignment), however, the pixels array is a 1D array (horizonal values or vertical values) and I needed both the X and Y access the pixels. The tutorial used the equation index = (y *width) + x, (which I don’t fully understand), but it allowed both I and J values tp access the pixel array. Since I am using color, which has four values RGB and Alpha, we also needed to multiply the equation by 4. 

I also create a variable “c” for brightness which is set to the average number of the R,G,B values.






After figuring out how to access the live webcam and its values, I really want to experiment more with animation, or simply giving the ellispe a bit more motion. So I looked into how to import audio, and connect teh audio to the visual. 

It sounds like a very simple idea, and I’ve done something similiar in touchdesigner, but in P5 it is a completely different process. 
I watched the Coding Train’s tutorial on the object in P5 called FFT(Fast Fourier Transform) (Frequency Analysis) to get the amplitude for low and high frequency sounds. The official definition of it is that it returns the frequency specturm or time domain(no idea what that means).

To give the ellipses a bit more changes, I added a few outer circles around them. I mapped the frequency specturm to the canvas size, and mapped that value to the actual canvas divided by 2. 




Now the ellipse sizes are changing based on the frequency values, but I didn’t love how they were showing up from the bottom right of my canvas. I really wanted it to change the entire canvas based on both x and y axis.




This took me a while to figure out, but I eventually created two variables and mapped them separately to the width and height, and now the changing of the ellipse sizes are happening across the whole canvas. 

This week’s project for me is more so to udnerstand using webcam and audio in P5. There’s a lot more that I would like to explore in the future, for example using a set of color palette instead of taking RGBA values from the webcam, as well as using other shapes to detect my motion and create different reactive animation.


REFERENCE


Webcam - Patt Vira

FFT - The Coding Train


SOLVED PROBLEMS

  • using live webcam and mapping shapes to pixels using loadPixels()
  • upload sound and use FTT to create audio reactive visuals
  • Incorporating index
 


THINGS TO IMPROVE

  • How to create more dynamic visuals? Using only brightness to change the direction of the shapes
  • Something to analyze later: Can this be done in p5?


INDEX  NEXT
JESSIEZHAI