Visuals

From Things and Stuff Wiki
Revision as of 17:46, 20 September 2020 by Milk (talk | contribs) (→‎Synthesis)
Jump to navigation Jump to search



General

See also Video, Creative coding, Networked media

post consolidation, to rework

  • https://en.wikipedia.org/wiki/VJing - a broad designation for realtime visual performance. Characteristics of VJing are the creation or manipulation of imagery in realtime through technological mediation and for an audience, in synchronization to music. VJing often takes place at events such as concerts, nightclubs, music festivals and sometimes in combination with other performative arts. This results in a live multimedia performance that can include music, actors and dancers. The term VJing became popular in its association with MTV's Video Jockey but its origins date back to the New York club scene of the 70s. In both situations VJing is the manipulation or selection of visuals, the same way DJing is a selection and manipulation of audio.One of the key elements in the practice of VJing is the realtime mix of content from a "library of media", on storage media such as VHS tapes or DVDs, video and still image files on computer hard drives, live camera input, or from computer generated visuals. In addition to the selection of media, VJing mostly implies realtime processing of the visual material. The term is also used to describe the performative use of generative software, although the word "becomes dubious (...) since no video is being mixed".






  • https://en.wikipedia.org/wiki/Video_art - an art form which relies on using video technology as a visual and audio medium. Video art emerged during the late 1960s as new consumer video technology such as video tape recorders became available outside corporate broadcasting. Video art can take many forms: recordings that are broadcast; installations viewed in galleries or museums; works streamed online, distributed as video tapes, or DVDs; and performances which may incorporate one or more television sets, video monitors, and projections, displaying live or recorded images and sounds.



  • Visual Music Archive - a non-institutional and highly subjective collection of inspirational works from the ever expanding field of Visual Music. This is a non-profit online art archive of links more than films. There is no commerce going on here, no ads, nothing and we will gladly remove anything you ask us to.


  • Drawing Sound - Postgraduate course in Interactive Music Systems Design (CDSIM), Pompeu Fabra University


Synthesis




http://i.m.klif.tv/category/projects/

Vector synthesis

  • macumbista.net » Vector Synthesis - allows the creation and manipulation of 2D and 3D vector shapes, Lissajous figures, and scan processed image and video inputs using audio signals sent directly to oscilloscopes, hacked CRT monitors, Vectrex game consoles, ILDA laser displays, or oscilloscope emulation softwares using the Pure Data programming environment. Audio waveforms control the vertical and horizontal movements as well as the brightness of a single beam of light, tracing shapes, points and curves with a direct relationship between sound and image. The technique is based on the well-known principle of Lissajous figures, which are a mathematical representation of complex harmonic motion. Originally displayed by reflecting light between mirrors attached to a pair of vibrating tuning forks, we are most used to seeing them on the screen of an oscilloscope, where they can be produced using pairs of electronic oscillators tuned to specific ratios. There is a wealth of such experiments from the 1950s onward by major figure such as Mary Ellen Bute, John Whitney, Larry Cuba, Manfred Mohr, Nam June Paik, Ben Laposky, Bill Etra, and Steina & Woody Vasulka, which were all highly inspiration to the development of this library.





Hardware


  • https://github.com/Ttreintaysiete/chaves - Cheap Hacky Audio/Video - Glitch VGA synth based on jonasbers.com/chav/This is a WIP, and PCBs have not fully tested, if you want to contribute send a build to your favorite pcb provider and give us any feedback.





Web

  • https://github.com/morukutsu/popart-io - a WebGL project inspired by hardware video synthesis. Using basic waveforms (sin, square, tri or saw) and advanced modulation, it is possible to combine different effects to create unique video sequences.Each build block has a set of parameters which can be tweaked to reveal new shapes and patterns.

Visualisation

ffmpeg

ffmpeg/ffmplay 


--lavfi-complex=<string>

Set a "complex" libavfilter filter, which means a single filter graph can take input from multiple source audio and video tracks. The graph can result in a single audio or video output (or both). Currently, the filter graph labels are used to select the participating input tracks and audio/video output. The following rules apply: - A label of the form ``aidN`` selects audio track N as input (e.g. aid1). - A label of the form ``vidN`` selects video track N as input. - A label named ``ao`` will be connected to the audio output. - A label named ``vo`` will be connected to the video output.

Each label can be used only once. If you want to use e.g. an audio stream for multiple filters, you need to use the ``asplit`` filter. Multiple video or audio outputs are not possible, but you can use filters to merge them into one. It's not possible to change the tracks connected to the filter at runtime, unless you explicitly change the ``lavfi-complex`` property and set new track assignments. When the graph is changed, the track selection is changed according to the used labels as well. Other tracks, as long as they're not connected to the filter, and the corresponding output is not connected to the filter, can still be freely changed with the normal methods. Note that the normal filter chains (``--af``, ``--vf``) are applied between the complex graphs (e.g. ``ao`` label) and the actual output. 

- ``--lavfi-complex='[aid1] [aid2] amix [ao]'``
   Play audio track 1 and 2 at the same time.
- ``--lavfi-complex='[vid1] [vid2] vstack [vo]'``
   Stack video track 1 and 2 and play them at the same time. Note that both tracks need to have the same width, or filter initialization will fail (you can add ``scale`` filters before the ``vstack`` filter to fix the size). To load a video track from another file, you can use --external-file=other.mk.
- ``--lavfi-complex='[aid1] asplit [t1] [ao] ; [t1] showvolume [t2] ; [vid1] [t2] overlay [vo]'``
         Play audio track 1, and overlay the measured volume for each speaker over video track 1.
- ``null:// --lavfi-complex='life [vo]'``
         A libavfilter source-only filter (Conways' Life Game).



mpv '--lavfi-complex=[aid1]asplit[ao][a];[a]showvolume[vo]' --alpha=no yourfile.wav





MilkDrop / ProjectM

  • MilkDrop - a music visualizer - a "plug-in" to the Winamp music player. As you listen your music in Winamp, MilkDrop takes you flying through the actual soundwaves you're hearing, and uses beat detection to trigger myriad psychedelic effects, creating a rich visual journey through sound. MilkDrop can also be driven by a live audio feed (microphone or line-in) - see the documentation for details. MilkDrop 2 is now here, rocking the pixel shaders.
    • https://sourceforge.net/projects/milkdrop2
    • http://en.wikipedia.org/wiki/MilkDrop - a hardware-accelerated music visualization plugin for Winamp, which was originally developed by Ryan Geiss in 2001. It uses DirectX and intelligent beat detection to render iterated images which blend seamlessly. MilkDrop uses a complex system of interpolation to transition between presets gradually through time, creating a constantly changing visual experience.





Presets

VSXu

  • VSXu - (VSX Ultra) is an OpenGL-based (hardware-accelerated), modular programming environment with its main purpose to visualize music and create graphic effects in real-time.

Cthugha

  • Cthugha Development Page - a program that listens to sound and displays graphics. It is sort of like an oscilliscope, but a lot cooler; the graphics Cthugha displays have flame effects, changing colors, and an alomst endless combination of attributes. Its sort of an interactive multimedia thing, at least it is interacting with the music. Originally Cthugha was a PC-DOS program, but the new version is designed to take advangaes in Windows graphics. This new version is still being developed, but that shouldn't stop you from checking it out! Cthugha is a native Win95 application. Cthugha should support all sound cards and graphics cards. High Res (640x480) and possibly high-color. Midi support (tired of CD's ??). New modes. Smooth palette changes

bomb

FLAM3

  • FLAM3 - The Flame Algorithm. Flames are algorithmically generated images and animations. The software was originally written in 1992 and released as open source, aka free software. Over the years it has been greatly expanded, and is now widely used to create art and special effects. The shape and color of each image is specified by a long string of numbers - a genetic code of sorts.

Flickernoise

  • Flickernoise - our VJ application that is made for our Milkymist One device. It has got a user-friendly interface that should make it very easy to use for anyone that wants to interact or control the Milkymist One. Our VJ application is optimized to be as easy to use as possible. So even if you do not know a lot of computer languages or programming, you should still be able to make use of our devices, through the Flickernoise application, rather easily. You would find that our Flickernoise application can render visual effects without a problem. And that you are also able to use it with no trouble as well. The application itself can render visual effects through hardware acceleration. The effects that are rendered through hardware-accelerated graphics are very similar to that of MilkDrop 1.x.
  • https://github.com/m-labs/flickernoise

radiance

Goom

audio-visualizer-python

  • https://github.com/djfun/audio-visualizer-python - a little GUI tool which creates an audio visualization video from an input audio. You can also give it a background image and set a title text. tested on Linux (Ubuntu 16.04) and Windows (Windows 7), it should also work on Mac OS X.

Kubus

  • Kubus is a minimalist audio visualizer, written in C/C++ using OpenGL and RtAudio. The aesthetic of Kubus draws inspiration from artists like Ryoji Ikeda and Viznut. Audio input is buffered into Kubus 1024 samples at a time, where it is sequentially mapped onto a 32 x 32 grid of squares. The amplitude of each particular sample controls the square color's lightness at that point in time. If FFT mode is enabled, the FFT bins are mapped onto the grid as well, controlling the saturation level of the squares.


Teardrop

  • https://github.com/jlblatt/Teardrop - using the Web Audio API and a more traditional method of visualizing an audio signal. Teardrop implements time & frequency domain analyses, as well as beat detection, and uses WebGL to render 3D graphics inside of the browser.


Oscilloscope

butterchurnviz

Plane9

  • Plane9 - a 3d visualizer where you never have to settle for just one view ever again. It features over 250 predefined scenes to choose from. But it doesn't end there since the scenes can be combined with one another to form a near endless supply of new views to experience. 39 transition are used to form a continous experience when moving from scene to scene. The visualizer can be used either as standalone window, screensaver, oculus rift or HTC Vive VR visualizer. It is sound sensitive and reacts to what your currently listening to, be it from spotify, iTunes or any another sound source, it can even react to what you record from a microphone or other input. Windows 7/8/8.1/10 (32 or 64 bit), Graphics card with OpenGL 3.3 support (Make sure you have the latest drivers for your card)

GLava

  • https://github.com/wacossusca34/glava - an OpenGL audio spectrum visualizer. Its primary use case is for desktop windows or backgrounds. Development is active, and reporting issues is encouranged.

KeyboardVisualizer

music_visualizer

Heilan X3D

  • Heilan X3D Browser - a cross-platform OpenGL X3D browser written in C++ and designed for audiovisual performance. Specifically, it forms the environment within which I developed an audiovisual instrument (Ashitaka) for my PhD.

Hydra

  • Hydra - a platform for live coding visuals, in which each connected browser window can be used as a node of a modular and distributed video synthesizer.

audioMotion.js

Magic Carpet

  • https://github.com/PaulSlocum/magic-carpet - A generative audiovisual artwork that produces imagery using rapidly spinning photos of rugs and quilts. Magic Carpet is written in cross-platform C++ using SDL with minimal dependencies. It currently builds for Raspberry Pi and iOS, and it should be possible to add builds for Android and Windows. This code was ported from my original Magic Carpet iOS Objective C app.

Lasp

  • https://github.com/keijiro/Lasp - a Unity plugin providing low-latency, high-performance and easy-to-use audio input functionality that is useful for creating audio reactive visuals.

Neural network

Generative


Mixing

FreeJ

  • FreeJ - a vision mixer: a digital instrument for realtime videomanipulation used in the fields of dance teather, veejaying, onlinestreaming, medical visualisation and TV.It runs a video engine in which multiple layers can be filtered thrueffect chains and then mixed together with images, movies, livecameras, particle generators, text scrollers and vector graphics. Allthe resulting video mix can be shown on a screen, encoded into a movieand streamed live to the internet.FreeJ can be controlled locally or remotely, also from multiple placesat the same time, using its ascii console interface; operations can bescripted in javascript and triggered live via keyboard, mouse, MIDIcontrollers, Joysticks, OSC clients, Wiimotes and more devices. FreeJ's sourcecode is written in portable C and C++ and it works onmost platforms supported by the GNU C compiler, including 32bit and64bit processors, PowerPC and various ARM flavours.


Freeseer

  • Freeseer - Designed for capturing presentations at conferences. Pre-fill a list of talks to record, record them, and upload them to YouTube with our YouTube Uploader.
  • Kaliscope download | SourceForge.net - a set of tools and plugins to create a professional pipeline for cinema works. It can also be used to create a telecinema that convert silver films into numeric movies. The goal is to produce reliable sophisticated tools to help artists in their analog/numeric film projects by respecting the quality of their precious images.

GLMixer

  • GLMixer - performs real time graphical blending of several movie clips and of computer generated graphics.Drop video files in the mixing workspace and place them in a circular area to change their opacity ; if you selects two videos, moving them together performs a fading transition. This principle generalizes to a large number of videos. Direct interaction with the video allows to be fast and reactive, and to move and deform them on screen.The output of your operations is shown in the output window, typically displayed in full-screen on an external monitor or a projector. But the output can also be saved as a video file.Control GLMixer through network using OpenSoundControl, and generate graphics with ShaderToy GLSL code.


MultiLayeredVideoSequencer


freemix

  • freemix - a limitless, open to creativity project, in beta stage. The aim is to develop a tool that any videoartist can adapt to it's style, taking advantage of it being free (as in speach) to modification or extension. Later reverting these back to the project. Feel free to drop ideas, comments or even cool code.

vusion

  • vusion - a video mixer that supports input from either an image, MPEG-1 encoded video or from a Video4Linux device. Vusion was written in python and uses the pygame library for mixing.It's designed to be controlled via a MIDI controller rather than a standard mouse or keyboard.

Snowmix

  • Snowmix - a Swiss army knife tool for mixing live and recorded video and audio feeds. It supports 2D and 3D clipping, scaling and transparent overlay of video, png graphics and text. It supports animation of video, images and texts through native commands changing scale, placement, transparency and rotation. Animation and actions can also be controlled through native scripting and an embedded Tcl interpreter. Snowmix is designed for control over low bandwidth links and can work as a standalone CLI based program. Control over both CLI and a TCP connections.Input and outputs can be done through GStreamer pipelines or the GStreamer shmsrc/shmsink API.

Kaliscope

  • Kaliscope - a set of tools and plugins to create a professional pipeline for cinema works. It can also be used to create a telecinema that convert silver films into numeric movies. The goal is to produce reliable sophisticated tools to help artists in their analog/numeric film projects by respecting the quality of their precious images.

Nageru

  • Nageru - a live video mixer. It takes in inputs from one or more video cards (any DeckLink PCI card via Blackmagic's drivers, and Intensity Shuttle USB3 and UltraStudio SDI USB3 cards via bmusb), mixes them together based on the operator's desire and a theme written in Lua, and outputs a high-quality H.264 stream over TCP suitable for further transcoding and/or distribution. Nageru is free software, licensed under the GNU General Public License, version 3 or later. Nageru aims to produce high-quality output, both in terms of audio and video, while still running on modest hardware. The reference system for two 720p60 inputs is a ThinkPad X240, ie. an ultraportable dual-core with a not-very-fast GPU. Nageru's performance scales almost linearly with the available GPU power; e.g., if you have a GPU that's twice as fast as mine (which is not hard to find at all these days; desktop GPUs are frequently more like 10x), going to 1080p60 will only cost you about 10% more CPU power.



to sort




  • FLxER is a audio video mixing software based on Adobe Flash™ tecnology born in 2000. FLxER is free. FLxER application is about 80kb and is available for Windows, MAC, Linux, PocketPC, iPhone, iPad and all the other platform with Flash Player, there is also a fully functionality web version that can be used from every computer connected to internet.
  • FreeJ is a video mixer: an instrument for realtime video manipulation used in the fields of dance theater, veejaying, medical visualization and TV. Its development started in 2001 and continued actively for many years.


  • Videopong is a community dedicated to expanding and connecting the world of VJs and video art. It is a hub to share not only video clips, but everything behind the visuals: source code, interactive patches, executables, and more.


  • Open Sound Control (OSC) is a protocol for communication among computers, sound synthesizers, and other multimedia devices that is optimized for modern networking technology. Bringing the benefits of modern networking technology to the world of electronic musical instruments, OSC's advantages include interoperability, accuracy, flexibility, and enhanced organization and documentation.






  • https://github.com/K0F/revolver - a digital video synthetiser. It inputs valid c++ code snippet and turns it into a values/pixels. Something similar as GLSL is doing to your screen on GPU but computed and dumped onto disk by CPU.


  • https://github.com/vooku/skinny - Mix video loops and apply simple effects triggered by MIDI events.Using openFrameworks with additional plugins ofxMidi, ofxArgs and ofxDatGui.Originally prototyped at FEL CTU.


Linux

Luz
v4l2loopback
  • v4l2loopback - a kernel module to create V4L2 loopback devices. this module allows you to create "virtual video devices". normal (v4l2) applications will read these devices as if they were ordinary video devices, but the video will not be read from e.g. a capture card but instead it is generated by another application. this allows you for instance to apply apply some nifty video effects on your Skype video... it also allows some more serious things (e.g. I've been using it to add streaming capabilities to an application by the means of hooking GStreamer into the loopback devices).
Qeve
  • Qeve - a vj software for Gnu/linux. It was born as a prototype for the performance of Qè and after the economic support received by the catalan association Telenoika it has been published as a public vj linux program. The project has evolved and remain just a couples of blogs reviews for historical reason.
GemQ
  • GemQ - a vj software developed primary for GNU/Linux Operative System and working on all platforms (with little effort). GemQ is based on Pure Data framework, a real-time programming language for multimedia. GemQ is the new version of Qeve, the first version of this project developed with the economic help of catalan association Telenoika and Hangar centre de Arts. GemQ uses Gem Video Library, a porting of opengl library for Pure Data, many externals built-in in the pd-extended version and objects written through python bridge (pyext). The aim of this project is to have an easy to use program for make visuals and video editing in real-time primary on linux platform.
PantaliQa
  • PantaliQa is the evolution of GemQ, that it was a vjing program developed in puredata language. PantaliQa is more mature, the code is cleaner and some how it rappresents all the things I have been learning in the last year in programming. The aim of the project is to develop a live video editor tool for real time perfomance with a focus on educational environments.
freej
Veejay
  • Veejay - a visual instrument and realtime video sampler. With veejay, you can play the video like you would play a piano. While playing, you can record the resulting video directly to disk (video sampling), all effects are realtime and optimized for use on modern processors, Veejay likes the sound of your video's as much as their images: sound is kept in sync ( pitched when needed - trickplay) and delivered to JACK for possible further processing. You can cluster to allow a number of machines to work together over the network (uncompressed streaming, veejay chaining) And much more... The engine is historically based upon mjpegtools's lavplay and processes all video in YUV planar It performs at its best, currently with MJPEG AVI (through ffmpeg) or one of veejay's internal formats. Veejay is built upon a servent architecture.
LPMT
  • LPMT - a little projection-mapping tool for use in our office, it is developed in C++ using OpenFrameworks. It’s based on simple quad warping paradigm, and, though rather simple, can be used to achieve complex and professional projection-mapping sets.
delvj
Radiance
  • https://github.com/zbanks/radiance - video art software for VJs. It supports beat detection, animated GIFs, YouTube video, OpenGL shader effects. It is designed for live performance and runs on Linux and MacOS.
vDome
  • vDome is an application designed to calibrate multiple projectors on a hemispherical dome surface and display a domemaster formatted video, image, or interactive application. vDome is generally used in two ways: 1) as a media player and 2) as a background process that listens to software/hardware input streams.
VFX
  • VFX - a software system for manipulating, triggering and mixing digital video clips and loops. There are two main parts to it: the Performer for playing, mixing and manipulating video in real time (not unlike VJamm or Arkaos) and the Compositor for rendering video effects for later use.
VjPirate
voctomix
Le Biniou
IQSynth
3DAudioVisualizers
GoldMosh

PocketVJ

Windows

Windows/macOS

Pure Data

See also Audio#Pure Data

  • Pure Data (aka Pd) is an open source visual programming language. Pd enables musicians, visual artists, performers, researchers, and developers to create software graphically, without writing lines of code. Pd is used to process and generate sound, video, 2D/3D graphics, and interface sensors, input devices, and MIDI. Pd can easily work over local and remote networks to integrate wearable technology, motor systems, lighting rigs, and other equipment. Pd is suitable for learning basic multimedia processing and visual programming methods as well as for realizing complex systems for large-scale projects.
Retrieved from "https://wiki.thingsandstuff.org/index.php?title=Visuals&oldid=56132"