← return to jerome.gangneux.net
Fork me on GitHub

Live Image Mix Experiments


This experiment needs the MIDI API and WebAudio API / Google Chrome only

Step 1: Glitch images

The idea was to mix this project: glitch-canvas
with a MIDI keyboard binding (including a classic keyboard fallback).
K1 K2 to see it working
Check the result: Glitch image

Step 2: Basic images-list player

Then I wanted to be able to play and control GIF images, but it is not that easy. I decided to split GIF or movies into a list of images.
K1 K2 to see it working
Check the result: Basic player

Step 3: Glitch player (images-list)

I mixed the first Step and the second Step so I can glitch a list of playing images!
This was quite ok but it could make my computer laggy sometime.
I decided to go for video instead of images and optimized the code.
K1 K2 to see it working
Check the result: Glitch player

Step 4: Glitch player 2 (video frames)

This is the same concept as Step 3 but with video instead of images.
The problem is that video can not be played in reverse with Google Chrome, so I decided to extract images from video and use that instead.
One limitation is that the first time a given video loads you can not do anything (black screen). It is by design to allow the code to stay easy to understand.
K1 K2 to see it working
Check the result: Glitch player 2

Step 5: Microphone input analyser

Then the next step was to get audio input to automate effect later.
I used the WebAudio API to get the microphone input and analyse the data.
K1 K2 to see it working
Check the result: Micro+oscillator

Step 6: Auto-glitch Player

Step 5 and Step 4 getting mixed gives an automated video glitcher that react to audio input.
K1 K2 to see it working
Check the result: Auto-glitch Player

How to use

Server

You have to use these experiments locally (localhost or 127.0.0.1) or on an HTTPS connexion (required to access MIDI/Audio Web API)

MIDI bindings


Midi binding are for the MPK Mini II (above)
Use K1 - K2 - K3 - K4 - K5 - K6 - K7 - K8 to change parameters (yellow)
Use Pads (bank A and B) to change the current image/video (purple)
Use the 3 last black keys to change binary parameters (red)

Keyboard fallback


Keyboard binding is related to the MPK Mini II and layout agnostic
The first and second 4 letters row (usualy QWER/ASDF) are for K1-K8 increase (use Ctrl to descrease) (yellow)
The number row (first row) are for Pad (use Shift to access pad bank B) (purple)
The last letters on the first letters row are for the last 3 black keys (red)