Welcome to 2017

I haven't written a blog post in nearly a year; hoping to change that in the new year. To get the ball rolling here are a few recent beginnings of new projects and updates on old ones.

Donald Trump was elected president and was sworn into office last Friday. Since then I have been using GitHub to keep a running list of what happens in government everyday.

Another project you can find on GitHub is my first attempt at a chat-bot, CocoaEliza. This is of course the beginning of yet another Eliza clone. We all have to start somewhere learning something new and why not with a classic? Hopefully this effort will eventually evolve into a new project in development, bot.dating. For a historical reference, the original Eliza chat-bot was by Joseph Weizenbaum who also wrote an excellent book titled “Computer Power and Human Reason” which covers his philosophy on what AI should and shouldn't do; an important read for anyone working in this field. You can also easily find an online version of Eliza to talk to.

Also on the topic of bots, though I did not take time to write about it at the time, last year I gave a talk about TVCommentBot and the future of television at Bot Summit in London. It was a wonderful conference and you can watch all of the presentations online. One of my related spin-off bots, TVFaceBot has recently reached 3000 followers. Though I've made several other small bots the only other one worth mention here was TheSoundOfBot created in memory of Pauline Oliveros, one of my teachers from RPI who passed away recently.

More to follow in the coming days, stay tuned!

Tuxedo Charlie Cat RIP

Last night my best friend Charlie the cat passed away after a bout with cancer. He was a special little buddy who won over the hearts of many humans, even several who were allergic to him. Words can't describe how much I'm going to miss seeing his face and hearing his purrs everyday.

Here are a few photos and videos of his adventures and fun times with friends. If you have any to add to the collection please send them my way.

Charlie loved to get out of the house, for a romp in the yard or around the block on his leash.

Charlie loved to eat good food, especially his greens including dill, spinach and of course wheat grass. Sadly he never managed to catch the most delicious thing in the world, the taunting birds on the fire escape.

Like all cats Charlie was on a never ending quest to find the perfect place to nap.

Most of all Charlie loved a good visit or cuddle with one of the many humans who loved him...

Doctor Who / Orphan Black Fan Parties At Comic Con San Diego 2015

Quick recap from last weekend – Just got back from filling in as DJ / VJ with the Eclectic Method at Comic Con San Diego at the fan meet ups for Orphan Black and Doctor Who. As a huge fan of both shows it was a special honor to be up on stage doing my thing alongside the stars and writers from the shows.

BBC America has official run downs of each event, keep a look out for a certain nerd behind a laptop in these videos,

I also took a couple of photos of random people who had some awesome (assorted) costumes,

And these were two of my favorite moments from the meet ups that I managed to catch on video,

That's about it! Hope the fans who made it out had a great time just like me!

Randomized Life Safari

On my flight home from Eyeo Festival this year, inspired as usual, I revisited a recent code sketch for creating randomized shapes to include a few new degrees of freedom.

The Random Shape generator can be played with interactively in a web browser by following the link above, but as I was going along I kept taking still image screen grabs as I twisted the shapes to look like birds, fauna, landscapes and other creatures.

At the end the series seemed rather like a slideshow from a safari to an odd digital world and I've collected some of my favorites below.

On a technical note, these are all drawn as layered 2D triangles, no 3D fanciness involved.

Birds and Creatures

Flowers and Terrain

TV Comment Bot

Last weekend I participated in an event here in Brooklyn called Art Hack Day where a bunch of artists and technologists with different skill sets all got together to implement new projects in 36 hours and finish with a gallery opening. I teamed up with David Newbury and Blair Neal on an installation we called “TV Helper” which is now also running as a Twitter-bot called TV Comment Bot.

The TV Helper is designed to make new TV shows out of existing ones by inserting new subtitle text over any video feed, replacing the original plot and dialogue of the show being watched. When possible the TV Helper algorithms will reference specific objects it “sees” in the video stream in the plot lines being created.

Additionally the genre of the new show can be a customized by the viewer using the set of provided knobs to be a mix of 8 different styles: Sports, News, Politics, Humor, Crime, NASCAR, Religion and Horror.

Here are some of my favorite results of feed from the home edition that is now running 24/7 since the Super Bowl on Feb 1st. You can find hundreds more on the Twitter feed.

If you're interested in hacking together a bit of this yourself, the two main tools used were the DeepBeliefSDK based analysis app that I wrote recently and this MadLibs generator created by David Newbury. We combined these using Quartz Composer / Syphon / OSC to create the simply text overlays for display.

The whole thing looked a bit like this behind the scenes:

Early display prototype.

Signal path sketch.

Software screenshot.

Testing, testing.

Me with the project!

Home edition.

Since this can run in real-time I'm also curious to try this out as an additional overlay during a VJ set in the future. We'll see how that goes.

Lastly a huge shout out to the Art Hack Day organizers for baby sitting all us artists while we worked. If you'd like to see more of the results from the event my friend Alejandro blogged about the awesome Oculus / Unity drowning game they made. You can find additional photos of the action (several of which are on this page!) on Margarida Malarkey's site.

[Update] Also on Art Hack Day: Deluge, read the details on the cool WiFi Taser by Max Henstell and another write up in this Makezine post.

[Update 2] Below are some more recent pictures from the Apple WWDC event, political press events and of course TV.

How to Escape The Twilight Zone

Submitted for your approval,

While most encounters with The Twilight Zone result in terrible death, loss of sanity or other horrible misfortune, a few lucky souls manage to survive their brush with the unknown and come out on the other side. 

Embedded inside every episode of the Twilight Zone is a nugget of truth about human nature or society. While these stories did not take place in the real world, they often contained real world truths about the dangers of bigotry, superstition, censorship, fear mongering, and emphasized the need to cling on to the best parts of our humanity as we become more technologically advanced.

In an effort to learn some life lessons about how to survive one of the darker places of the human imagination, I’ve started to put together this list of special episodes where things don't go completely terribly wrong.

As a note, before you get started in here, there are several spoilers. Those who aren't already avid fans of the Twilight Zone will probably want to watch all 156 episodes of the original series before reading any further in this post.


The Escape Artists

We begin by highlighting the rare cases of people who managed to enter and exit the Twilight Zone.

Having Confidence In Your Own Future

Sometimes the key to success is finding the right balance of superstition and making your own fate. Don't dwell on the past, work towards making a brighter tomorrow.

Nick of Time – Watch On Hulu

While honeymooning William Shatner almost gets trapped in a paranoid frenzy by a penny operated predication machine that bosses him around. As they leave, we see another couple who didn't make it out. 

Shatner vs Supersition

Counterbalance in the little town of Ridgeview, Ohio. Two people permanently enslaved by the tyranny of fear and superstitution, facing the future with a kind of helpless dread. Two others facing the future with confidence - having escaped one of the darker places of the Twilight Zone.
— Rod Serling, Nick of Time

The Trouble With Templeton – Watch On Hulu

Dealing with troubles in his personal and professional life, an aging actor revisits his glory days through his memories, only to be rejected by his former friends. Upon returning to the present he discovers that his recollections had merely put on a play of their own to trick him into not living in the past, giving him renewed strength to take on his current life challenges.

Mr. Booth Templeton, who shared with most human beings the hunger to recapture the past moments, the ones that soften with the years. But in his case, the characters of his past blocked him out and sent him back to his own time, which is where we find him now. Mr. Booth Templeton, who had a round-trip ticket - into The Twilight Zone.
— Rod Serling, The Trouble With Templeton

No Time Like The Past – Watch On Hulu

Disgusted with the present day, a scientist uses a time machine and tries to alter key historical events (such as killing Hitler) but after multiple failures he finds that time can not be altered. Learning his lesson, he resolves to work towards positively impacting the future instead of dwelling on the mistakes of the past.

Incident on a July afternoon, 1881. A man named Driscoll who came and went and, in the process, learned a simple lesson, perhaps best said by a poet named Lathbury, who wrote, ‘Children of yesterday, heirs of tomorrow, what are you weaving? Labor and sorrow? Look to your looms again, faster and faster fly the great shuttles prepared by the master. Life’s in the loom, room for it. Room.’ Tonight’s tale of clocks and calendars in the Twilight Zone.
— Rod Serling, No Time Like The Past

A Penny For Your ThoughtsWatch On Hulu

When a coin lands on its side, Dick York  temporarily gains the ability to read minds and though there are some misunderstandings, tries to use the new power responsibly. This leads to finding a previously unknown confidence in himself that continues after things return to normal.

One time in a million, a coin will land on its edge, but all it takes to knock it over is a vagrant breeze, a vibration, or a slight blow. Hector B. Poole, a human coin, on edge for a brief time - in the Twilight Zone.
— Rod Serling, A Penny For Your Thoughts

Encounters with the After Life

Though we've learned a lot about the universe in the 50 years since the Twilight Zone originally aired we still aren't any closer to understanding death and the afterlife, nor have we gained new mechanisms for coping with loss. In these episodes we're shown that life is worth living even when things look bleak or near the end, but that it also does eventually come time to say goodbye.

A Passage for Trumpet – Watch On Hulu

Jack Klugman plays a down and out drunkard of a trumpet player who tries to give up on life. While stuck in limbo he has a chance to talk things through with a peculiar trumpeter named Gabe who convinces him to give living another chance.

Joey Crown, who makes music, and who discovered something about life; that it can be rich and rewarding and full of beauty, just like the music he played, if a person would only pause to look and to listen. Joey Crown, who got his clue in the Twilight Zone
— Rod Serling, A Passage for Trumpet

Changing of the GuardWatch On Hulu

A professor forced into retirement is on the verge of suicide but decides to not kill himself after a supernatural encounter with former students who were inspired by his teachings.

Professor Ellis Fowler, teacher, who discovered rather belatedly something of his own value. A very small scholastic lesson, from the campus of the Twilight Zone.
— Rod Serling, Changing of the Guard

Long Distance Call – Watch On Hulu

A young boy tries to drown himself in an attempt to be reunited with his beloved dead grandmother and is allowed to live after his fathers pleads with her spirit.

Playing telephone with the dead is standard fare in the Twilight Zone.

A toy telephone, an act of faith, a set of improbable circumstances, all combine to probe a mystery, to fathom a depth, to send a facet of light into a dark after-region, to be believed or disbelieved, depending on your frame of reference. A fact or a fantasy, a substance or a shadow - but all of it very much a part of The Twilight Zone.
— Rod Serling, Long Distance Call

Nothing in the Dark – Watch On Hulu

The infamous Death shows up for an elderly woman as Robert Redford and shows her that the unknown isn’t always to be feared.

There was an old woman who lived in a room. And, like all of us, was frightened of the dark. But who discovered in a minute last fragment of her life that there was nothing in the dark that wasn’t there when the lights were on. Object lesson for the more frightened amongst us, in or out, of the Twilight Zone.
— Rod Serling, Nothing in the Dark

Looks Can Be Deceiving

When things around you don't make sense, that doesn't always mean you're crazy. On the other hand, that may in fact be a clear sign that you've completely lost it.

Where Is Everybody? – Watch On Hulu

Also notable as the first episode of the Twilight Zone, a mysterious stranger in an empty town turns out to be a potential astronaut slowly losing his mind during an experiment to test the limits of isolation in space travel.

Up there, up there in the vastness of space, in the void that is sky, up there is an enemy known as isolation. It sits there in the stars waiting, waiting with the patience of eons, forever waiting... in the Twilight Zone.
— Rod Serling, Where Is Everybody?

I Sing The Body Electric – Watch On Hulu

Written by Ray Bradbury (and later the name of a short story of his by the same name), an android housemaid is brought into a home to help care of three children after the death of their mother. Despite not being human she is eventually loved as one of the family and everyone lives happily ever after.

A fable? Most assuredly. But who’s to say at some distant moment there might be an assembly line producing a gentle product in the form of a grandmother whose stock in trade is love. Fable, sure, but who’s to say?
— Rod Serling, I Sing The Body Electric

Twenty Two – Watch On Hulu

A stressed out Barbara Nichols is in the hospital suffering from a mental breakdown caused by recurring nightmare. Despite everyone thinking she's insane the dream ultimately contains clues that later save her life.

Those nightmares are trying to tell you something.

This is Miss Liz Powell. She’s a professional dancer and she’s in the hospital as a result of overwork and nervous fatigue. And at this moment we have just finished walking with her in a nightmare. In a moment she’ll wake up and we’ll remain at her side. The problem here is that both Miss Powell and you will reach a point where it might be difficult to decide which is reality and which is nightmare, a problem uncommon perhaps but rather peculiar to the Twilight Zone.
— Rod Serling, Twenty Two

Making the Best of a Bad Situation

While not everyone makes it out of the Twilight Zone, there are some noteworthy cases for what to do when escape is not an option.

Two Enter, One Leaves

A Game of PoolWatch On Hulu

Wanting to be known the greatest pool shark of all time, Jack Klugman has a chance encounter with a previous, now deceased, legend. Upon winning the game he gets the title but finds himself stuck with defending his name in the after life for all time.

Mr. Jesse Cardiff, who became a legend by beating one, but who has found out after his funeral that being the best of anything carries with it a special obligation to keep on proving it. Mr. Fats Brown, on the other hand, having relinquished the champion’s mantle, has gone fishing. These are the ground rules in the Twilight Zone.
— Rod Serling, A Game of Pool

The Jeopardy Room – Watch On Hulu

Attempting to defect to a new life, a former KGB agent must outwit a sadistic enemy assassin in a game of cat and mouse where he is all but guaranteed to lose.

The cast of characters- a cat and a mouse, this is the latter. The intended victim who may or may not know that he is to die, be it by butchery or ballet. His name is Major Ivan Kuchenko. He has, if events go according to certain plans, perhaps three or four more hours of living. But an ignorance shared by both himself and his executioner, is that both of them have taken the first step into the twilight zone.
— Rod Serling, The Jeopardy Room

Starting Over In the Unknown

Third From The Sun – Watch On Hulu 

With nuclear war on the horizon a scientist realizes the only chance of survival for his family is to travel to an unknown planet in a stolen experimental rocket ship. Now on their way to another world, like all refugees of war, it is unclear if they are coming or going from the Twilight Zone.

Behind a tiny ship heading into space is a doomed planet on the verge of suicide. Ahead lies a place called Earth, the third planet from the sun. And for William Sturka and the men and women with him, it’s the eve of the beginning - in the Twilight Zone.
— Rod Serling, Third From The Sun

TwoWatch On Hulu

Elizabeth Montgomery and Charles Bronson find out the hard way that when you're the last two people on Earth after an apocalyptic war, you were probably made for each other, even if you previously fought for opposing armies.

Romance at the end of the world.

The time, perhaps 100 years from now or sooner; or perhaps it’s already happened 2 million years ago. The place, the signposts are in English so that we may read them more easily, but the place is the Twilight Zone
— Rod Serling, Two

Probe 7, Over and OutWatch On Hulu

The last survivors of mankind stranded on an alien world might just be the next Adam and Eve.

Do you know these people? Names familiar, are they? They lived a long time ago. Perhaps they’re part fable, perhaps they’re part fantasy. And perhaps the place they’re walking to now is not really called ‘Eden.’ We offer it only as a presumption. This has been the Twilight Zone.
— Rod Serling, Probe 7, Over and Out

Kick the Can – Watch On Hulu

An old man in a retirement community believes acting young will make him young. Though everyone thinks he's crazy, he succeeds when playing the game ‘kick the can’ and escapes into the Twilight Zone where he becomes a child again.

Sunnyvale Rest, a dying place for ancient people, who have forgotten the fragile magic of youth. A dying place for those who have forgotten that childhood, maturity, and old age are curiously intertwined and not separate. A dying place for those who have grown too stiff in their thinking - to visit - The Twilight Zone.
— Rod Serling, Kick the Can

“When the fall's all that's left, it matters a great deal”

In some cases the best you can hope for is a last chance to go out in style.

The Obsolete Man – Watch On Hulu

A librarian in a totalitarian society is sentenced to death by the state for having an obsolete profession. Taking advantage of a loophole in the system allowing him to choose the method of his execution Burgess Meredith gets the last word by revealing the true nature of the Chancellor (and those like him) on television.

You’re a librarian, Mr. Wordsworth! You’re a dealer in books and two cent fines and pamphlets and closed stacks and the musty insides of a language factory that spews out meaningless words on an assembly line. Words, Mr. Wordsworth, that have no substance and no dimension, like air, like the wind, like a vacuum that you make-believe has an existence by scribbling index numbers on little cards.
— The Chancellor

One for the Angels – Watch On Hulu

Attempting to outsmart Death, a kind hearted salesman accidentally places the life of a child in danger. Doing the right thing, he earns his place “Up there” in the After Life.

Lewis J. Bookman, age sixtyish. Occupation: pitchman. Formerly a fixture of the summer, formerly a rather minor component to a hot July. But, throughout his life, a man beloved by the children, and therefore, a most important man. Couldn’t happen, you say? Probably not in most places - but it did happen in the Twilight Zone.
— Rod Serling, One for the Angels

So there you have it friends – study these episodes carefully and you too may be one of the few to escape, or at least survive in, the Twilight Zone!

[Update: For further reading, check out this recently dug up Mike Wallace interview with Rod Serling from 1959]

VJing – WTF is it?

This blog post is adapted from a presentation I gave at Decibel Festival 2014. Links / images / embedded video materials are copyright of respective owners, presented here for educational purposes, no implication of endorsements in either direction. Another recap from the workshop can be found on Geek Wire. Also note that the wikipedia article on VJing is a great resource for further reading on this topic.


Video Jockey – in the 80's and 90's, the term “VJ” was popularly considered the video version of radio “Disc Jockeys,” the person who introduced the next song on television.

In the late 90's and early 2000's, as the world of DJing evolved beyond simple curation of music with specializations like scratching, sampling, remixing and the like, so have VJs expanded their set of tools and techniques for live performance and production.

Today someone who is a VJ might appear to be something more akin to a video instrumentalist or visualistsomeone who creates and manipulates images in ways similar to how a modern musician works with sound.

 
VJing in the year 1981 (MTV)

VJing in the year 1981 (MTV)

VJing in the year 3001? (Futurama)

VJing in the year 3001? (Futurama)

 

Personally I think of a pro VJ as a video art ninja and rockstar in one; capable of bridging the worlds of preproduction, live performance and post production. The methodologies involved are evolutions of those used for early film special effects and avant garde video artists from the 60's and 70's, well before the introduction of the MTV VJ era.

 

Genres of Live Visuals

Like the world of professional music, modern live video art has developed several distinct sub-genres of VJing. Some of the most commonly encountered today include:

  • Show Visuals
  • Media Remixing
  • Live Cinema
  • Set Design
  • Interactive Installations
  • Film and Music Video Production
  • Art Objects

While these are very different fields, there is often a large overlap in the skill sets involved. Many VJs eventually find themselves specializing in one or more of these areas, or at least assisting on projects that fall outside of the scope of traditional production techniques. Likewise many people who start in these fields may find VJing to be something they want to learn more about.

To get a sense of what each of these fields looks like, let's look at some examples:

 

Show Visuals

Typically part of concert tours, in night clubs, churches and at other live events, show visuals are mainly found backing up or enhancing other elements of a live performance.

Sufjan Stevens / Age of Adz in Prospect Park with visuals by CandyStations.

 

Media Remixer

Covered by Mashable in its “10 Crazy Jobs That Will Exist in the Future” article, the “Media Remixer takes the job of DJ or VJ to another level by remixing various forms of media into one cohesive new project. These remixers will bring together audio, video, images and augmented reality to create projects ranging from marketing campaigns to wedding entertainment to installation art.”

We don't have to wait for the future for this job, with artists like Eclectic Method taking on jobs like remixing 50 years of Doctor Who for the BBC.

 

Live Cinema

The live cinema artist mixes stories that have narrative elements. If there is a musical score, it is usually background to the visual performance and plot.

My first introduction to live cinema, The Light Surgeons, have been performing for nearly two decades.

 

 

Set Design

For many decades, designers have used video elements as backdrops or scenery on stage. While it is by no means a new idea, these virtual elements are increasingly becoming capable of interacting with actors as part of the performance.

New Creatures - Red Bull Murals: "A Heros Journey"

 

Digital Signage and Interactive Installations

Usually seen in storefronts, museums, art galleries, and festivals, interactive installations use similar techniques to show visuals and set design, but run off of input from the audience instead of performers. They can also include elements of narrative structures or data visualizations.

Syfy Upfront event by LEADDOG CREATØRS

 

Film and Music Video Production

The use of the VJ skills on a film set and music video production comes in many different forms.

In one potential use, similar to with set design, visuals are projected or otherwise incorporated into the scenery that is captured by a traditional camera setup.

Death Cab for Cutie, You Are a Tourist, a live shot, scripted, one-take music video shoot. Production by GoodCompany w/ Nicholas Gould.

 

An even more modern technique is using VJ software to capture in studio video performances, then mastering with traditional non-linear editing software to make a music video that mixes shots from real and virtual worlds.

DATA by Wolfshirt, made by recording the output of “shots” in a 3D game world running in sync with music production software.

 

Art Objects

As the technology used for making real-time video becomes smaller, another creative outlet for VJs has been the building of one-off art objects that include integrated displays or projectors.

Compendium by Will Reardon,

“Found deep under the sea, this object carbon-dated to around 12,000BC contains the sum total of past and future human technological knowledge...”

 

A Common Tool and Skill Set

While the final result of each of these examples varies, many of the processes and technologies used by the artists were similar. For a VJ, it's about finding the combinations to create a unique visual style or figuring out an innovative way to get a job done.

 

Hardware – Displays

Ultimately the final place where the visuals appear is one of the most important aspects of a project. At large events, projectorsplasma screens, and LED walls are found. In the case of videos that are uploaded on the web or traditional broadcast, viewers might watch on a computer monitor or television

While typically in the domain of the lighting designer, at times the VJ can control them as part of the video rig.

Modern micro projectors and small low power LCD screens have also opened a lot of new possibilities for installations and art objects. Though it is becoming rare, some VJs still make use physical media film and photographic slide projectors at part of their setups.

 

Hardware – Playback

Standard physical media player devices that have video outputs such VCRs, DVD and Blu-Ray Players, media servers and video iPods. These can either be connected directly to displays, routed through other systems or used simply for backup feeds.

 

Hardware – Cameras

Whether shots of the performers, the audience or something else altogether, incorporating live camera feeds is a common task in event production.

In recent years camera technologies have themselves become more specialized for different use cases and visual artists have taken advantage of this. Webcams, night vision security cameras, depth sensing and even high end pro video cameras can all be found being used by VJs at live events. On the preproduction side, the ability to shoot high frame rate video on GoPro and newer mobile phones has made slow motion video capture both affordable and easier to use.

 

Hardware – Physical Controllers, Instruments and Sensors

One of the corner stones of visual performances is using a physical interface that allows for instrumental and gestural control over the output images. Standard protocols like MIDI, OSC and DMX make it possible for all kinds of gear to work together seamlessly together whether you use software or hardware tools.

 

Hardware – Mixers and Routers

When more than a basic connection is needed between a source and display, video mixers and routers are used by video techs to quickly switch or combine multiple images into a single output. These devices often also have features for cropping, resizing and applying other visual effects. Some mixers have been designed in a DJ style, complete with a sideways crossfader, whereas others are laid out for a standard live television production workflow.

 

Hardware – Cables and Converters and Power

The physical connections between each device in a chain are in the form of cables. Though wireless and network based options are becoming more prevalent, the majority of situations still use wired connections which come in two main varieties; analog and digital cables. When needed special converter boxes or adapter cables can usually be found to change from one format to another.

Analog Video Cable Types:

Coaxial (often requires RF converter)

Composite (1x RCA or analog over BNC)

Component (3x RCA or analog over BNC)

S-Video

VGA

 

Digital Video Cable Types:

HDMI

DVI

Display Port / Mini Display Port

SDI / HD-SDI (digital over BNC)

Ethernet / LAN (streaming video)

Along with video signals the modern VJ also typically has to connect hard drives, webcams, sensors, MIDI instruments and other devices to computers or other media players. USB, Firewire and Thunderbolt are fairly common on current systems. Older gear may require the use of serial (rs232) or MIDI cables. When working with lighting control systems individual DMX cables may be needed for each fixture.

For many video performers it is also necessary to integrate with the audio world. Whether you are receiving feeds for analysis and capture, or playing back soundtracks in movie files, it can help to be familiar with the different types of cables that will be encountered when talking with audio engineers: 1/8" (headphone jack), 1/4" (big headphone jack, can be mono or stereo), RCA, XLR and optical are the most common. Though many computers have built in ports for sound you may need to get additional hardware that connects over a digital connection such as USB or Firewire.

Though some things are powered by battery or as part of the connection type (such as USB or Firewire), another important note is to keep track of the proper power cables for each device. Even if the port is the same size be sure to check the amperage and wattage if you aren't sure it is the original adapter. When traveling to another country you may need to use additional special converters to plug into wall sockets to work with different electrical standards.

 

Hardware – Analog Synthesizers and Other Weird Stuff

Long before computers became the tool of choice for live visualists, other amazing tools were used to generate video signals or project images with light.

While much of the technology involved is either considered obsolete or nearly impossible to come by, there are modern day analog video synthesizers such as the LZX Industries synth that carry on the tradition. Additionally there is a continued movement to replicate these styles and techniques digitally with software, though some processes require very powerful computers to accurately emulate.

 

Software – Mac / Windows

VJ Software usually includes a suite of what you get from dedicated playback and mixing hardware, plus image processing FX, control data generators like audio analysis and LFOs, along with basic configuration settings for outputting to displays or recording to disk. Popular options include ArKaos, modul8, Resolume and VDMX. Some music production and DJ software packages are now starting to include their own add ons for video, though usually in a manor focused to the workflow that already exists within the application.

 

Software – iOS / Android

Simple tablet based mixing and FX apps can be fun starting points for new VJs to learn the basic concepts before moving on to professional level desktop software. Video synths and samplers can be used to create sample movies for playback or as live camera inputs for use in larger projects alongside desktop VJ software or hardware.

 

Software – Specialized Inputs and Outputs

Following new protocols like Syphon (Mac) and Spout (Win) for routing video between applications along with standards like MIDI / OSC / DMX for control information has come a new breed of programs that are designed to run alongside existing desktop VJ apps to add new input, output and processing features. One of the most popular use cases is projection mapping where tools like Mad Mapper, VPT, IR Mapio, MWM and Blendy Dome each have unique workflows designed for different kinds of mappings.

These protocols also make it possible for less mainstream software features to integrate easily into larger projects, such as interfacing with depth sensing Kinect cameras and other high end video capture devices, or playback of esoteric file formats.


Software – Pre and Post Production

In addition to real-time software, VJs also use the same tools as motion graphics designers and movie editors such as After Effects, Motion, C4D, iMovie and Final Cut Pro when creating video loops and polished demo reels for the web.


Software – Programming Environments and Creative Coding

For many people, using VJ software is a gateway drug of sorts into writing their own software tools. In some cases this can be in the form of plugins that load into existing software or connect using established protocols as described above, and for others it may mean the creation of a completely new application that runs independently. Quartz Composer, GLSL / WebGL, Processing, Max/MSP/Jitter, OpenFrameworks, Cinder, Unity, Touch Designer, vvvv and PD, are all excellent examples of established languages and frameworks being used by creative coding communities. This is where you'll be happy you paid a little attention in high school geometry and algebra classes.


Pioneers of VJing – from Ancient Greece to the 1990's

Though the technologies used have changed greatly over the years, the visual language of light, film and video has been a constantly growing part of human society. To become an expert VJ, it helps to have knowledge of the history and learned theories of the past, not just mastery of the modern technical aspects.

The use of light and images as part of live performances and pre/post production have been around for at two thousands of years with Aristotle's noted use of the infamous camera obscura technique to focus light through a small hole to create light images on walls in darkened rooms. Zoetropes, magic lanterns and pepper's ghost holograms are further examples of pre-cinema forms of live visual technologies that are still found being used today.

Though it would be some time before the idea of video instrumentalism would come around, the advent of film in the later 1800's is what set off the revolution that would eventually turn into VJing. The practices of cinematic language and montage editing developed by early experimental film artists like Sergei Eisenstein are as relevant in the clubs as they are in the art house movie theater. Color theory, the study of how colors mix and play together, has also been evolving for hundreds of years, and something that many visual artists will want to take into account when creating their works.

Film was the precursor to the technology that VJs get their name from: Video.

Though there were early experiments with video in the earlier part of the 20th century, it was first really introduced to society starting in the 1950's. Almost immediately artists began to find ways to include video beyond standard commercial use. The founder of the video art movement is often considered to be Nam June Paik who started to hack the electron tubes on televisions in the early 60's. Lillian Schwartz, Steina and Woody Vasulka, Dave Jones and countless other artists worked out of places like Bell Labs and the Experimental Television Center to do things with video signals and early computers outside of the mainstream.

Around the side time period, other non-video light projection techniques were being implemented by The Joshua Light Show to create a new form of psychedelic live concert visuals. These shows introduced some of the foundations of instrumental visual performance.

Another pre-VJ pioneer that paved the way was Laurie Anderson who along with being a prominent musician and visual artist spent a lot of time inventing fantastic new interfaces for interacting with these mediums:

On a technical level, early cinematic special effects and mixing was in some ways closer to the real-time methods used by VJs. As there was no concept of a non-realtime renderer, everything went straight to output. Even as it became possible to create rudimentary digital graphics on a computer, the use of “rescanning” material was a powerful way for production studios to get visual effects that would otherwise not be possible.

While much of this seems primitive by current standards, studios and VJs alike still take advantage of these techniques. Perhaps one of the most well known systems of the time was the Scanimate which was used to make a lot of the now classic commercials and animations seen on television in the 70's and early 80's. The styles of other esoteric video processors and synthesizers like the Rutt Etra from the time period are still replicated today.

As they became more powerful, computers of course came and once again changed what was possible when it came to manipulating images. However it wouldn't be until the late 1990's that they would be fast enough for anything beyond basic real-time video processing, leaving VJing to slightly lag behind advances in graphics used in television and movie production where most things could be rendered in advance. When it did become possible early VJs who had come from a musical background like EBN, Hexstatic and Coldcut were quick to make the connection between the performative and sequencing capabilities of MIDI instruments for creating groundbreaking new A/V pieces.

By the early 2000's modern VJing as we know it began to mature into the fields described above.


Getting Started With VJing

If something on this page inspires you to want to make some artwork of your own, my first suggestion is to find the easiest path to dive right in and figure it out as you go. Start by downloading sample loops and shoot footage of your friends to play with when trying out different software and hardware for remixing. Watch technical tutorial videos while reading about film editing and music theory. Find people in your local and online communities who are already in these fields and learn from them.

And most importantly, keep looking at how all of the pieces available can work together to make something amazing that comes from your own creativity. Plug everything into everything else and see what happens, even if it is just for fun.

Limits, Calculus and How Things Change (Explained Without Numbers)

Preface

Put your name in the hat to tell a story at the Ale House on Front Parlor nights. Math is sometimes best discussed with beer.

Put your name in the hat to tell a story at the Ale House on Front Parlor nights. Math is sometimes best discussed with beer.

A few years ago for a story night on the theme of “limits” the first definition that came to mind for me as a math nerd was not “a restriction on the size or amount of something permissible or possible,” nor “a point or level beyond which something does not or may not extend or pass,” or any other definition that carried a negative association that most people have with the word. Instead, my mind wandered back to the definition I was taught in September of 1998 during my first days of Calculus at Brooklyn Technical High School.

While the idea of Mathematics is often confused with numbers and equations, at its heart it is a language like any other – we use it to formulate, transcribe, communicate and prove our ideas. It can be used to describe the world around us, either artistically or in explicit detail. Like other languages it contains forms of verbs, nouns, proper nouns, syntax, and other grammatical rules and conventions to be followed. Though it is particularly efficient for expressing and calculating numerical data, it can be equally as powerful in the hands a poet. Further discussion of this topic of this will remain for another day...

The Language of Change

In the language of Mathematics, Calculus contains the words we use to talk about change. Typically when we think about something changing, whether explicitly or not, it happens in relation, or with respect to, one or more other things.

Here are some gifs of things changing:

Like all languages, new words and concepts in mathematics evolve as people need to share new ideas with other people, whether in spoken or written form. In the case of limits, a big part of the backstory is actually already very well known by most elementary school students – when Isaac Newton famously needed to explain his theory of how gravity works. In particular it isn't just that gravity pulls on objects causing them to move towards the ground, it is the subtle detail of what happens when a force (visible or invisible) is applied continuously to an object. When something is falling, it doesn't just fall at a constant speed, it begins to fall at a faster and faster rate the longer it falls.

Common math notation and grammar

Newton wasn't the only mathematician of the time period who needed these kinds of words in their dictionary to describe the world in detail. It took over 200 years, from Descartes (noted with bridging algebraic words with geometric pictures) to Leibniz (who came up with much of the notation we still use today) to Cauchy and Riemann (who refined the definition and grammar of limits) for the new branch to develop and become a formalized addition to the language of Mathematics. Before this you can imagine a world full of mathematicians trying to talk about how things change while speaking slightly different dialects of the same tongue.

When change happens, it's either very small, very big, or somewhere in the middle. Calculus is the mathematical grammar we use when talking about change and it's broken into two main types: Differential Calculus introduces the idea of taking these very small, infinitesimal steps to accurately approximate instantaneous rates of change (like how fast a file is downloading from the Internet right now), whereas Integral Calculus introduces the idea of taking infinitesimally small steps to approximate large sums. Whether you are consciously aware of it or not, when making predications about the future or imagining the past, the thought process is similar to this branch of math and we can use it to express and understand the results of our thinking.

Regardless of the amount of the change, we can also talk about the direction. Is it increasing, decreasing or staying the same? Did the price go up or down? When did it happen? Like Newton and his peers, when we start to ask these questions, we quickly run into one the most important words in all of modern Mathematics: limits.

Approaching The Beginning and Ends of Time and Space

There's actually a bit more imagination involved with understanding limits in the mathematical definition than typically encountered in other branches of math. Unlike the natural language versions, in Mathematics we use limits to talk about what happens as we approach a given point on a path from all possible directions at the same time. There's a bit of poetry to the idea. As we get closer and closer, traveling from any possible direction, if we always arrive on the at the same finite point in space, a limit is said to exist at that point.

Everything is nothing,
with a twist.
— Kurt Vonnegut, Slaughter-house Five

Why is it important that these limits exist on our paths going from one place to another? It's actually more about of what happens when they don't – it's difficult to have a discussion about change without the consistences and continuity that limits bring to the table. Without them such conversations become full of holes and conflicting truths. Or try to imagine walking down a street where you appear in a different place depending on which direction you arrive from. The Universe as we know it is a very tricky place to navigate without limits.

In particular there are two points on our paths that limits are helpful for describing because we often can't actually go there, only approach them; zero (also known as nothing, nil or null) and infinity (the number bigger than the biggest number). In Mathematics these concepts are represented by the symbols “0” and “∞” respectively and they come into play when dealing with very small and very large scale ideas alike. We consistently consider what happens as we get near these places when we are talking about how things are changing at an exact moment in time, or when accumulating the results of continued forces over hundreds of thousands of years.

An xkcd comic uses limits in a punchline in a joke about an infinite number of little pigs in a children's story.

An xkcd comic uses limits in a punchline in a joke about an infinite number of little pigs in a children's story.

The basic concept of limits is just the first step into this toolkit for analysis and expression. When we're talking about information, ideas or even feelings that have this special kind of consistency to them we can use the language and logic that comes with Calculus to reveal deeper meaning and understanding of how things change. In this sense, limits aren't points that can't be crossed, they the interesting places that we can visit, observe from every angle and then move on from. The more limits the better!

However, like any translation, this description is only part of the picture. To get the most insight, humor and beauty out of limits you may need to learn some of the native tongue they are written in. If you remember a little bit of algebra, one place to start is the Khan Academy's introduction to differential calculus which as you would expect, beings with the concept of limits. History and linguistic buffs may prefer to get more of a background of how the of language Calculus has roots that date back thousands of years to the ancient Egyptians and Greeks. Or for a laugh, Look Around You takes a look at Maths.

ps. Thanks to Lisa Rogers Neal for suggested edits and notes.

Continued Histories of the Future (or why we still need the year 2000)

One of my favorite random coffee table books is “The History of the Future,” an illustrated guide to how the people of the past imagined our future, written and curated by Christophe Canto and Odile Faliu in 1993. Flipping through the pages usually leaves me with a two thoughts – what new futures have we created since the year 2000, and what is our next new reference point for the distant but still conceivable future?

It's a wonderful read (if you can find it) and I've included a short photo set of some of my favorite images from the collection to give you an idea of whats inside,

The year 2000 provided an ideal reassurance for the aspirations of the new middle class; they would have appreciated having somebody else to take care of the important things in life, orchestrating their fate and leaving them to worry only about the incidentals.
— The History of the Future

The first chapter of the book is titled “An Ordinary Day In The Year 2000,” which is aptly named – for over a hundred years the year 2000 was somewhat of a classic milestone for predictors of the future and a relevant reference point for the setting of dystopian fictions that extrapolated the knowledge of the time into the foreseeable future.

As the date approached in the late 1990's, we actually began to muse less about how life would be different after we crossed the threshold. That is unless one of those doomsday predictions like Y2K was going to take down all of society overnight. Note in the chart below the drop in “the year 2000” ngram score slightly before the year 2000 actually arrived, and its obvious decline after,

ngram of "the future" tappers off after the year 2000

A Post-2000 Addendum To The History of The Future

As we edged closer to the year 2000 and then into the new millennium, our visions of the future began to course correct from the predictions of the past and looked towards new milestones. To extend and add contrast to the image set from the 20th century looking to the 21st, I've begun to put together a new collection of our more recent fictions of the future. Like their predecessors they are often also a commentary on some aspect of our current age.

Note: There are some spoilers here...

The near future (2015 - 2099)

Many of the fictions for the near future deal with how our society will deal with the unintended repercussions of our newest and soon to be invented technologies..

In Black Mirror, brain implants are coming very soon,

they will change how and who can access our memories.

Minority Report, in 2054 precognition will arrest murderers before they kill.

Cyber brain implants and physical upgrades are a common theme in the not to distant future. Extrapolating on our now constant connectivity to the net, wearable technology like Google Glass, advancements in the fields of robotics and limb prostheses, the merging of human and machine is already happening and may be norm within a generation,

Ghost in the Shell SAC: By 2030 full cyborg bodies are commonplace,

immersive virtual spaces and artificial intelligence will be reality.

This will come along with new forms of addiction and illness.

In order to survive through the end of this century we'll likely need to overcome a few huge hurtles, such as pollution, disease, limited resources and access to the energy needed to power the technology of the future. If we do succeed, we may see the colonization of other planets in our solar system over the next 30-80 years,

Excavations of the Moon could provide the limitless clean HE3 energy we need in 2035..

but this process is only made cost effective by using a duped human clone workforce.

The alternative may be an unending scarcity and conflict aboard the Snowpiercer on an uninhabitable Earth by 2041.

Or maybe we'll be nearly extinct by incurable disease in 2027 like Children of Men.

Assuming we do make it, there's also the question of how technology and scarcity of resources will change the world politically and economically,

When it gets down to it — talking trade balances here — once we’ve brain-drained all our technology into other countries, once things have evened out, they’re making cars in Bolivia and microwave ovens in Tadzhikistan and selling them here — once our edge in natural resources has been made irrelevant by giant Hong Kong ships and dirigibles that can ship North Dakota all the way to New Zealand for a nickel — once the Invisible Hand has taken all those historical inequities and smeared them out into a broad global layer of what a Pakistani brickmaker would consider to be prosperity — y’know what? There’s only four things we do better than anyone else

music, movies, microcode (software), high-speed pizza delivery.

The Deliverator used to make software. Still does, sometimes.
— Neal Stephenson, Snow Crash (approx 2017)
How far into the future will we have pizza delivery?

How far into the future will we have pizza delivery?

2071 - Odd jobs and adventures for profit around the solar system in Cowboy Bebop.

In Star Trek we ditch greed and war only after making first contact in 2063.

Refuge aliens landing in District 9 may not be greeted with the same hospitality as Vulcans..

This century will also likely see the birth of new forms of life of our own creation, both biological and mechanical. Bio engineering is already a reality, though in its infancy and the long term reproductions have yet to play out in the mainstream. True artificial intelligence seems a little further off, however there is a lot of money going into it, along with the robotics to build the bodies for them to live in. Our dealings with new forms of sentient life who have vastly different conceptions of reality may not come from space but will be equally as foreign to our understanding. Will our robot children be slaves or equals in the future?

Humanoid AI made to look like and serve us may be second class citizens with limited rights.

Over 50 years after Asimov wrote I, Robot we still aren't sure if we can make AI that won't just kill us.

Humanity is the ultimate dream of the accidentally sentient Bicentennial Man.

The fuuuuuuuture (2100 and beyond)

Nothing will change in the future: technology will advance, but we will still have the same political, moral, and ethical problems as today.
— Joss Whedon

What lies ahead in the next century and through to the end of this millennium for humanity? Like with illustrators and writers of the early 1900's, it can be difficult for modern day artists and scientists alike to accurately guess the specifics of the future so far out will be like without using current technology and society as a reference. What we can say over this long of a time period is that things are going to either get better, worse or stay somewhat the same.

In the category of things going horribly wrong we still typically have stories that focus on the potential longterm results of some continued destructive behavior such as anti-intellectualism or wastefulness,

The world of Idiocracy has a stupid 2505 civilization about die off.

In Wall-e the Earth is abandoned i2105 due to overwhelming trash..

and by 2805 the human population is much rounder.

A total collapse of civilization is only one we're going out. In other scenarios we survive with our technology but get stuck losing our humanity along the way, becoming unrecognizable versions of ourselves, or enslaved by it,

The Hunger Games, the year 2108s favorite televised blood sport.

In 2167 Fringe science will remove emotion from our genes in favor of increased intelligence.

Hey these year 2199 (or later) tentacle creepers from The Matrix look familiar..

The world of the future might be a lot like the world today only more futuristic and possibly in space. Even as we replace parts of our body and surroundings with increased augmentations or travel the stars, our basic human needs, fears and pleasures will remain the same.. only in more futuristic and therefore totally strange ways..

A modern North Korean architect imagines a future that looks remarkably similar to our own future histories.

The Fututrama year 3000 has room for both one eyed mutants and Oktoberfest.

The Fututrama year 3000 has room for both one eyed mutants and Oktoberfest.

If the human race makes it to the year 3000 there's a good chance it's because we found a way to get off of this rock called Earth and into deep space colonization. However until we figure out a way to travel much faster than the speed of light, the practicality of sending anything other than generation ships or intelligent robots on these unfathomably long trips beyond our solar system seems impossible.

Slower than light generation ships from an overcrowded Earth carried pioneers to distant stars in FireFly by 2517.

In 2014 the new EmDrive could make these slow but very long journeys possible.

Whereas in The Stories of Ibis androids and AI become our longterm space legacy.

The Eve Online MMORPG takes place over 21,000 years in the future, far from Earth.

Dr Who and Rose visit the eventual natural end of the Earth in the year 5.5/apple/26.

The Next Milestones

The AFI 2008 top sci-fi movies, all made before 1992.

To try to find a starting point with my second question, I used ngrams with other noteworthy dates as charted below. If any, the year 2050 seemed to stand out as a next milestone, but even at its peak score it ranks way lower than Y2K. More remarkably I noticed that the ngram score of many of these also began to drastically drop off just before the year 2000 instead of increasing and continue to do so – are we writing less about the future than we did in the past?

After the year 2000 we see a drop in references to the future.

What are some possible reasons for this? One might be that we are busy trying to catch up to our own version of future. Just because the things imagined are now technically possible doesn't mean that it isn't a lot of work to actually make them a mass market reality. In the last 14 years we've had proliferation of access to some of the technological dreams predicted over the last 100 years including 3D printing, smart phones, tablets, household robotics and advances in medicine. Perhaps the ability to collect and parse huge amounts of data in real-time has drawn our attention to short term prediction instead. Along with this we've had major growing pains as a society and not just from the technology itself. Recent wars, terrorism and financial recession have a way of changing the topic to more immediate concerns. On the other hand we could just be spending too much of our time on the infinite cat project.

The LongNow project focuses on the next 10,000 years of human history.

Despite this trend of less writing about the next upcoming milestones, we're still working to guess what the world will be like when we get there. For more on this subject, over at FutureTimeline.net you can read a detailed, part fact, part fiction account of the first 200 years of this millennium and beyond based on their research and analysis of long term trends. Then for further inspiration visit the LongNow project, a organization founded in 01996 with the goal of encouraging very long term thinking for the survival of the human race over the next 10,000 years. And of course there's a reddit for Futurology if you want to discuss the topic further.

One Year Back In Brooklyn

It took me about 12 months of living in Brooklyn before I decided that I needed my own website. That isn't to say that I don't have several other websites already and that other people hadn't given me this suggestion numerous times over the years, but now its finally happened. With this in mind, for the first entry, a quick recap of a bunch of things I would have written, belated mini-blog posts for in the last year if I had been keeping up with it...

View from my apartment. Just happily resigned my lease for another year.

View from my apartment. Just happily resigned my lease for another year.

SXSW 2014

I went to SXSW 2014 with Game of Phones to be part of the AT&T / Vice Mobile Movement pavilion. We built a Telephonegram booth that played a series of continuous daily rounds for anyone passing by.

On the trip I also got a chance to visit with a bunch of VJs at a VDMX meet up. One of those opportunities to put faces to people that I've emailed with for years.

Actually managed to go out to concerts every night while there but sadly didn't get to make it to any of the interactive or film events.

Lastly, as a bonus got to shoot a slow motion video of my friend Juliette showing Lady Gaga how to control the stage visuals from an iPad.

 

NY Mappathon at Mapping Festival 2014

In late May I went to Geneva with my friends Chika, Anton and Tom for the 10th annual Mapping Festival. We ran a 3-day VJ workshop with a focus on projection mapping with students from around the world. There was also plenty of time to check out some of the events, presentations and performances over the weeklong festival, but the big highlight was just spending time talking with other visual artists in the beautiful city of Geneva.

For the festival I also got to do a short VJ set along with vade, 35 seconds of which you can watch in the included video and like a proper tourist I have a short slideshow of pictures from the trip.

 

Eyeo Festival 2014

Photo credit: Blair Neal

Shortly after the trip to Geneva it was off to Minneapolis for Eyeo Festival. This trip was a real treat, the presenters were all amazing and it was probably the friendliest, most excited cross section of the arts tech scene that I've come across so far. Frieder Nake's “Yes, Of Course” keynote speech was extremely fitting for the crowd, we all left the room carrying that spirit with us the rest of the week and then back home.

I also got an opportunity to do a 5 minute Ignite Talk on Game of Phones, the slides of which are available online (the video will be available soon). The best personal moment related to this was getting a high five from Nicholas Feltron and then finding out that I know the guy who does the visuals for his girlfriend.

Too many things to point out, all of the talks I went to were great, but in particular I was looking forward to seeing the ones from Jessica Hagy and Claire Evans who did not disappoint.

PS. There is a special Thank You round of Game of Phones for the Eyeo Festival organizers!

 

I helped out with a bunch of random events in NYC

... and David Lublin.

... and David Lublin.

Since my return a bunch creative agencies, organizations and friends have asked me to help out with projects they are working on. Usually my involvement is minimal, called in to assist with a few technical details at the last minute, but nonetheless they were a lot of fun to be part of and worth mentioning a few particularly cool ones in a single block..

  • I did some of the setup at Tao Downtown with the Rockwell Group. Also performed the visuals at the opening night event, ended up in this random celebrity slide show photo.
  • My friend Nick Gould called me in to do some production and on site work for goodco / i-D Think Aloud: Future Fashion Show. This involved a building a “Peppers Ghost” style holographic projection system along with green screen footage.
  • Filling in as a teacher on visual performance at dubspot.
  • Ran the visuals at one of the venues for the 2013 Brooklyn Electronic Music Festival using material prepared by Benton-C Bainbridge.
 

Brooklyn!

How far / long did you travel to be here in Brooklyn?

Even though the last year has included a lot of travel I'm most excited just to be back in my hometown. It's really my favorite place (so far) in the world and I'm not sure how I stayed away for so long. It probably had something to do with having such amazing friends up in Troy, who are all now encouraged to visit or move here to be closer to me.

I won't hang on this so just to throw out a few highlights in the BK, reconnecting with old friends, making lots of new friends, not missing family events, reconnecting with bagels, getting to go to lots of concerts, getting around by subway.. and I'm also a fan of having a skyline view. There's still a lot of readjustment to do and it feels like I still need to move faster to keep up.

Readily available weird ice cream flavors

And that's a little of what I've been up to over the last year. More to follow as it happens.