The Déjà Vu Upmix Algorithm
Frank Foti has been at the forefront of audio processing for over 40 years. Recently, he has developed an algorithm that will permit listeners to hear aspects of some musical works as never before. Rolf Taylor had a conversation with Frank regarding his latest algorithm: Déjà Vu Upmix.
Rolf Taylor: Thanks for talking with me Frank. Why not start with a quick summary of your career and interests.
Frank Foti: My background spans broadcast engineering, where I had the pleasure of working at some legendary radio stations, among them, WMMS in Cleveland, and WHTZ (Z100) in New York City – the latter I built from the ground up in 1983.
While working in radio, my interest was always in audio processing. New York City demanded a great on-air signature sound, and I was responsible for that. This was also where I developed my first audio processor, The Vigilante, which was the heart of the Z100 audio chain.
From there, I set foot on my own, in 1988, and launched a company that was known as “Cutting Edge Technologies.” We specialized in on-air processing for radio. A few years later, I merged Cutting Edge with Telos Systems. Steve Church, founder of Telos, and I were both great friends and colleagues. So, we decided to put our collective ventures under one roof.
RT: I know that your years at WHTZ in New York City were an important part of your background. There was a documentary about this recently released was there not?
FF: Yes! Thanks for asking! In early February, the film “From Worst To First, The Story of Z100, New York” was released. It is available on just about all VOD services. It has received rave reviews!
FROM STEREO TO SURROUND SOUND
RT: Tell me a bit about your first encounter with surround sound, and your impression at the time?
FF: My first encounter with surround was back in the 1970’s when the fad of Quad was brewing.
My high school buddy, Bob Klass, was really into it. He eventually had an SQ setup, but honestly I was not impressed. Those old matrix systems were pretty lame. The CD-4 system was interesting, but the true discrete performance would only last a few plays of the vinyl record, as after a few plays, you lost much of the added carrier, which supported the surround audio.
ADDING A WOW FACTOR
RT: And at that time, you and your partner Steve Church were quite involved in promoting some surround technology for potential use on HD FM radio, correct?
FF: Yes. Both Steve and I were thinking about a way to add a “wow factor” to the HD Radio FM System.
As it is, there is not a noticeable difference, on FM, when the channel switches from the conventional signal to the HD counterpart. We felt that by offering discrete 5.1 surround, over-the-air, this would be a truly “WOW!” experience.
Our friends at the Fraunhaufer Institute (FhG) had just developed MPEG Surround, and this new tech (at the time, early 2003) would enable us to transmit discrete 5.1 surround over-the-air. It is perfect for the automobile environment.
RT: Back when I worked for you I recall early on you were experimenting with various ways to create upmixes – making more channels out of basic stereo tracks – back around 2005, maybe even earlier. Have you been playing with upmix technology ever since?
FF: Yes, I became enamored with the SACD/DVD-A 5.1 Surround formats. So, I went out and bought anything I could find.
When the music industry pulled the plug on promoting those formats, it felt like being a junky, and needing a fix. Steve and I had heard a few upmixer offerings, and nothing truly blew us away. So, I would noodle around with various upmixer ideas of my own. It was one of those occasional rainy day efforts, as most of my work was focused on dynamics signal processing.
RT: You told me previously that your success with Déjà Vu came after you changed your focus. Tell the readers a bit about this.
FF: Well, I had this idea of truly isolating the center channel, as compared to having a phantom center Channel, and adding a Low Frequency Enhancement (sub-woofer) channel.
The challenge for me was how best to do this. Eventually, the light bulb went on and the details fell into place. The original concept worked, as imagined. It was getting it from imagination to reality. That is what consumed the majority of time.
RT: Sounds like when Nicola Tesla envisioned the AC power system!
Could you share some of the insights and adventures you had over the past couple of years demoing Déjà Vu to various well-known professionals at various locations?
FF: Upon getting the system operating, I had colleagues, at Telos, listen. They loved it. Friends loved it. I loved it. As a developer, you sort of live as the oxymoron because while you want to love it, you are also wondering if it is truly done, or is it the best it can be?
Fortunately, I became connected with Gary Katz, the legendary producer of Steely Dan. I am sure everyone knows how “The Dan” were regarded for having great technical quality to their records, as well as production.
I gave Gary a listen, and he was blown away. Although he claimed to not be as ‘technical’ as he thought I was looking for. So, he offered up a few ‘others’ who could fit that bill: Hugh Padgham, Giles Martin, and Frank Filipetti, among others. All well-known award winning engineers/producers. In order to visit with Hugh and Giles, it required a visit to Abbey Road, in London. Yes, that Abbey Road!
I spent a wonderful afternoon, with Gary and Hugh, at Abbey Road, where they were blown away by the performance of Déjà Vu. This was after Hugh telling me, the night before at dinner, that he thought every upmixer he had heard was rubbish.
RT: That is impressive. Can you summarize the comments and feedback your received?
FF: To date, in addition to those already mentioned, Déjà Vu has been auditioned for Jean-Michele Jarre, Greg Hawkes (The Cars), Elliot Randall (Steely Dan), Andrew Scheps. All of them have given me the proverbial “thumbs up!”
RT: Any suggestions?
FF: To get it out there!
GETTING IT OUT THERE
RT: I know you have become one of the founders of the Syndicate of Sounds. Is there anything more you can reveal about the status of the algorithm as far as licensing and deployment?
FF: Right now we are working with a company, Angry Audio, to produce a plug-in, as well as a hardware box.
RT: My experience with surround music, both native mixes and some up-mixed material, is that by simply spreading the content across multiple speakers makes it possible to hear elements that had been “lost” in the original stereo mix. Listening to material though your upmixer definitely un-masks elements of the recording that could not be heard before, like hearing it for the first time. Care to comment?
FF: That is why I named it Déjà Vu.
THE LOST SOUND IS FOUND
RT: Anyone that has done even the most basic mixing of any band more complex than one or two instruments and singer, is familiar with this “lost in the mix” phenomenon during mix down. But just because it cannot be heard in the resulting mix does not mean it is not there, does it?
FF: In most cases yes. By example, the group Chicago had a huge hit with the song “Does Anyone Really Know What Time It Is.” In a later verse, there is some spoken word under the lead vocal. When listening in stereo, it has always been really difficult to know what the spoken parts are being said. Through Déjà Vu, you can hear the spoken word segment clear as a bell, out of the right-rear channel. That sort of highlights your point.
RT: I have always had 4.1 and 5.1 surround setups that follow the ITU-T recommended speaker positions (rear speakers at roughly 4 o’clock and 8 o’clock). Many home theater enthusiasts are using the Dolby 7.1 speaker positioning where the side speakers are at 3 and 9 o’clock, and rear speakers at roughly the 5 and 7 o’clock positions. Those with the Dolby scheme would, of course, use the front, Center and Side positions only for 5.1 music. Any thoughts on the 5.1 music experience with ITU versus the side speaker schemes?
FF: I have become accustomed to the ITU-T setup. For me, 5.1 is plenty.
It is like diminishing returns when adding more channels. The audio starts to become jumbled together. Maybe it works for the video experience, but with music, 5.1 is plenty.
PUTTING SOUND WHERE IT BELONGS
RT: A question specific to how your Déjà Vu algorithm works.
Am I correct that there are no “virtual’ side channels? My listening environment is not ideal, and a few times it appears I was hearing content between the right front and right rear channels, for example, like the stereo virtual center channel. I am guessing that the material is coming out of only one of those speakers, not both.
FF: It all depends on the original stereo mix. Déjà Vu will put the sound field, where it was originally intended. There are some instances, when signals will be isolated to the individual channels, and others where there can be signal that appears a bit in more than one channel.
RT: OK, so I really did hear the cow bell on “Don’t Fear the Reaper” coming out of the cabinet on the side of the room. It was a very stable virtual channel.
Obviously room reflections and psychoacoustics play a role here. I have definitely noticed the well known psychoacoustic principle coming into play when I am much closer to the front speakers – all content collapses and appears to come from those three speakers – just like in stereo where if you stand close to one speaker all content appears to come from that one.
But in this case, I still hear un-masked content as compared to stereo. My general experience is that being in the sweet spot is far less important for surround music than for stereo music. With film soundtracks being in the sweet spot seems to be more important.
FF: The ear has a natural level drop for signals which emanate from the rear. There are some surround enthusiasts who will boost the output of the rear channels by 2.5 to 3 dB. It is a matter of preference.
RT: AV Receivers include adjustable delay settings for situations where the speaker positions are asymmetrical, or the listening position is not dead center. I suspect some purists would consider that unnecessary for music. What are your thoughts?
FF: I have never employed the adjustable delay functions. I set them all to zero.
RT: Of course it goes without saying that all 5 speakers should be identical if possible. If the center channel is different, it needs to be closely matched.
FF: For music, I suggest that all five surround speakers be of the same style and vintage.
RT: Care to share any thoughts on the future direction for Syndicate of Sounds?
I know your Déjà Vu surround album listening parties at the Music Box in Cleveland have been a big success. I came to the debut Steely Dan Aja party – as expected the Déjà Vu rendition was a remarkable improvement to the original stereo. That is one of the few albums that have literally sent chills down my spine, so it an improvement was not easy.
FF: Well, the buzzword right now is “immersive.” I am working on some ideas for headphone listening. That too is exciting work.
RT: Well I have to congratulate you on this excellent algorithm Frank. I know that I, for one, will buy it the minute it is available as a hardware device.
– – –
The Déjà Vu upmix algorithm is now available for both the PC and Mac platforms as nCIRCL Music. To learn more about Déjà Vu visit: https://syndicateofsounds.com/ And to learn more about the Upmixer from Angry Audio: https://angryaudio.com/ncircl/
– – –
Rolf Taylor has over 30 years experience in dealing with audio issues. He currently is at Rocket Engineering and Consulting in Springfield, VA. You can connect with Rolf at firstname.lastname@example.org
– – –
Would you like to know when more articles like this are published? It will take only 30 seconds to
click here and add your name to our secure one-time-a-week Newsletter list.
Your address is never given out to anyone.