{"id":4235,"date":"2020-05-05T07:05:15","date_gmt":"2020-05-05T11:05:15","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=4235"},"modified":"2021-04-06T15:48:34","modified_gmt":"2021-04-06T19:48:34","slug":"optimization-of-hrtf-models","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2020\/05\/05\/optimization-of-hrtf-models\/","title":{"rendered":"Optimization of HRTF Models with Deep Learning"},"content":{"rendered":"Today\u2019s post is from Sunil Bharitkar, \u00a0who leads audio\/speech research in the Artificial Intelligence & Emerging Compute Lab (AIECL) within HP Labs. He will discuss his research using deep learning to model and synthesize head-related transfer functions (HRTF) using MATLAB. This work has been published in an IEEE paper, linked at the bottom of the post.<\/em>\r\n
<\/h6>\r\nToday I\u2019d like to discuss my research, which focuses on a new way to model how to synthesize sound from any direction at all angles using deep learning.\r\n
<\/h6>\r\n

A crash course in Spatial Audio<\/h2>\r\n

<\/h6>\r\nOne important aspect of this research is in the localization of sound. This is studied quite broadly for audio applications and relates to how we as humans hear and understand where sound is coming from. This will vary person to person, but it generally has to do with the delay relative to each ear (below ~800 Hz) and spectral details at the individual ears at frequencies greater than ~ 800 Hz. These are primarily the cues that we use to localize sound on a daily basis (for example see the cocktail party effect<\/a><\/span>).\r\n
<\/h6>\r\nBelow are figures displaying the Head-Related impulse response<\/strong> measured in an anechoic chamber at the entrance of both ear-canals of a human subject (left figure), and the Fourier-domain representation i.e., the Head-related Transfer function<\/strong> (right figure), which shows how a human hears sound at both ears at a certain location (e.g., 45 degrees to the left and front and 0 degrees elevation) in the audible frequencies.\r\n
<\/h6>\r\n\r\n\r\n\r\n
\"\"<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
<\/h6>\r\nIf we look at this plot, you can see that when a sound is played at a direction of 45 degrees from the center, the sound at the left ear is higher in amplitude at this angle than the right. Also embedded in this plot is the difference in arrival time between the left and right ear, where only a few milliseconds in difference can have an important impact on where we perceive sound. Subconsciously, we interpret the location of the sound source based on this difference in spectra and delay.\r\n
<\/h6>\r\nCompare this with a sound coming 180 degrees behind a human:\r\n
\"\"<\/h6>\r\nThe spectral detail at the left and right ear is nearly identical at all frequencies, since the sound source is substantially equidistant from both ears. The difference in arrival time will be insignificant for the sounds source at 180 degrees. These discrepancies (or lack thereof) is what helps us determine where sounds is coming from.\r\n
<\/h6>\r\nWe are very good at localizing sound at certain frequencies, and not as good at others*. This depends on the frequency and the location of the sound.\r\n
<\/h6>\r\n*It\u2019s interesting to note that humans aren\u2019t very good at determining whether sound in certain angles (e.g., in the cone of confusion). The best way we can help to localize if we are confused is to move our head around to try to optimize the discrepancies between left and right ear. I\u2019m sure you\u2019re now curious to try this experiment informally at home with your next beeping fire alarm.<\/em>\r\n
<\/h6>\r\nThis research has many applications where localization of sound is critical. One example is in video game design, or virtual reality, where the sound must match the video for a truly immersive experience. For sound to match video, we must match the expected cues for both ears at desired locations around the user.\r\n
<\/h6>\r\nThere are many aspects of this research which makes this a challenging problem to solve:\r\n
<\/h6>\r\n