AIR Lab logo Audio Information Research Lab Electrical and Computer Engineering · University of Rochester

Welcome to AIR!

At the Audio Information Research (AIR) Lab, we conduct research in the emerging field of computer audition: designing computational systems that are able to analyze and understand sounds including music, speech, and environmental sounds.

We address fundamental issues such as parsing polyphonic auditory scenes (the cocktail party effect), as well as designing novel applications such as sound retrieval and music information retrieval. We also combine sound analysis with the analysis of other signal modalities such as text and video towards multi-modal scene analysis. Various projects that we have been working on include audio source separation, automatic music transcription, audio-score alignment, speech enhancement, speech diarization and emotion recognition, sound retrieval, sound event detection, and audio-visual scene understanding.

News

  • 2026.10Congrats to Christos on his successful PhD defense. Well done, Dr. Benetatos!
  • 2026.09Congrats to Zhiyao on his new position as the Director of NYS Center of Excellence in Data Science and Artificial Intelligence (CoE-DSAI)!
  • 2026.09Neil will join the ADASP Group at Telecom Paris as an Assistant Professor. Very proud of him!
  • 2026.06Baotong and Xingjian are out for summer internships at Adobe Research and Meta Superintelligence Labs (MSL), respectively.
  • 2026.04Congrats to Neil on his successful PhD defense! Rising Star, Dr. Zhang!
  • 2025.11Congrats to Neil on his Best Student Paper Award at WASPAA 2025!
  • 2025.08Congrats on Yujia's successful PhD defense! Well done, Dr. Yujia Yan!
  • 2025.06Congrats on Ge's impressive PhD defense! Well deserved, Dr. Ge Zhu!
  • 2025.06Xingjian, Huiran, Christos and Melissa are out for summer internships at MERL, Bosch, Smule, and Meta!
  • 2025.06Congrats on Yujia's full-time researcher position at Cartesia!
Earlier news
  • 2025.06Congrats on Neil's full-time senior researcher position at Dolby!
  • 2025.05Congrats on the graduation to Kyungbok Lee (BS in CS), Paul Berggren (BS in ECE), and Brynn Lee (MS in Data Science)!
  • 2025.05Zhiyao will be promoted to Full Professor on July 1, 2025! A Big Thank You to all collaborators!
  • 2025.03Congrats on Moji's full-time research engineer position at Apple!
  • 2025.02Zhiyao is invited to give a talk at AAAI 2025 Workshop on Artificial Intelligence for Music.
  • 2024.11Congratulations to Yujia for his paper on piano transcription receiving a best paper nomination at ISMIR 2024! Try his awesome model on any piano recording you can find!
  • 2024.11Congrats to Zhiyao and Stella on playing a violin-piano duet at the conference jam session!
  • 2024.11Congratulations to Neil on receiving a 2024 IEEE SPS Scholarship among 45 recipients worldwide!
  • 2024.10Zhiyao is invited to give a talk at SANE 2025 on speech synthesis and another talk at Boston Music AI Meetup on AI-Powered Music Production.
  • 2024.10We are proud to receive an unrestricted gift and the Audiobox license from Meta. Thank you Meta Audiobox!
  • 2024.09We are grateful to be on a team with Honeywell, UT Dallas, and Texas A&M to work on the IARPA Anonymous Real-Time Speech (ARTS) program. UR had a news release.
  • 2024.09Congratulations to our undergraduate student, Kyungbok Lee (CS'25), on his first first-authored peer-reviewed paper at IEEE MMSP! He also won a student travel award from this workshop and the University.
  • 2024.09Welcome two new PhD students, Stella Wong and Baotong Tian! Welcome back current PhD students, Moji, Neil and Melissa, from internships and visits at Apple, Meta and Trinity College Dublin!
  • 2024.08Congratulations to our undergraduate student Paul Berggren (ECE'25) on receiving the Tau Beta Pi engineering honor society award (see news here)!
  • 2024.05Congratulations to our new graduate, Yutong Wen (AME'24), who will pursue a PhD at UIUC this fall! Also Congratulations to last year's MS graduate, Qiaoyu Yang, who will pursue a PhD at Georgia Tech this fall!
  • 2024.03Neil, Yongyi and Zhiyao are proud to co-organize the Singing Voice Deepfake Detection (SVDD) challenge at IEEE Spoken Language Technology Workshop (SLT) 2024, together with Jiatong Shi from CMU and Ryuichi Yamamoto and Prof. Tomoki Toda from Nagoya University.
  • 2024.01Zhiyao became the President of the International Society for Music Information Retrieval (ISMIR) for the term of 2024-2025.
  • 2023.11Christos, Zhiyao and Prof. Philippe Pasquier from Simon Fraser University delivered a tutorial on Computer-Assisted Music-Making Systems: Taxonomy, Review, and Live Coding. In addition to a comprehensive review, it also features a live coding session on building a real-time musical agent using Euterpe, a prototyping framework for creating music interactions on the web.
  • 2023.11Neil's NIJ fellowship and research was covered by News10NBC (Rochester local TV station) and a nice article!
  • 2023.10Congratulations to Melissa for winning an NSF NRT travel award for a three-month visit to Trinity College Dublin in Summer 2024!
  • 2023.10Neil Zhang received a 2023 National Institute of Justice (NIJ) Graduate Research Fellowship among a total of 24 awardees nationwide. Congratulations, Neil!
  • 2023.09Yutong Wen received a 2023 University of Rochester Undergraduate Research Presentation Award and a WASPAA 2023 travel grant! These awards will support his travel to WASPAA 2023 to present his first-authored paper.
  • 2023.07Undergrads in AIR lab are doing awesome research! Yongyi and Yutong had their first publication as the first author at Interspeech 2023 and WASPAA 2023, respectively. They were both co-mentored by Neil!
  • 2023.06Our paper co-authored by You (Neil) Zhang, Yuxiang Wang, and Zhiyao Duan was recognized as among the top 3% of all accepted papers at ICASSP 2023! Neil was also selected as one of the 24 presenters at the inaugural "Rising Star Program in Signal Processing" program. Zhiyao received an outstanding reviewer award.
  • 2023.06Several AIR lab PhD students will go for summer internships at Adobe, Meta, Microsoft, and Sony.
  • 2023.05Congratulations to our MS graduates, Qiaoyu Yang (ECE) and Zehua Li (TEAM), and undergraduate graduates, Yongyi Zang (AME) and Enting Zhou (CS)! All the Best!!
  • 2023.05Yongyi Zang received a 2023 University of Rochester Undergraduate Research Presentation Award! This award will support his travel to Interspeech 2023 to present his first-authored paper.
  • 2023.04Yongyi Zang presented an oral presentation and a poster at the Undergraduate Research Exposition about "Euterpe". Yiyang Wang and Neil Zhang presented at the Graduate Research Symposium.
  • 2023.03Zhiyao Duan and our collaboration with industry partner IngenID are featured on this article.
  • 2023.02Four papers from AIR lab are accepted by ICASSP 2023. Congrats to all students and collaborators!
  • 2023.01Zhiyao Duan was on WXXI Connections radio program together with Raffaella Borasi and Blaire Koerner, discussing how artificial intelligence may affect the music industry. This is an interview with host Mona Seghatoleslami about our NSF project on "Toward an Ecosystem of Artificial Intelligence-Powered Music Production (TEAMuP)". Listen to the recording here.
  • 2022.12AIR lab is awarded a New York State Center of Excellence in Data Science grant to develop and deploy spoofing aware speaker verification systems with IngenID. Thank you, NYS!
  • 2022.11AIR lab is awarded a seed funding from the University of Rochester Goergen Institute for Data Science to investigate Personalized Immersive Spatial Audio with Physics Informed Neural Field.
  • 2022.09NSF grants $1.8M to a fantastic and diverse team of researchers between UofR and Northwestern to build foundations for AI-powered music production ecosystems! AIR lab and Interactive Audio Lab at Northwestern will co-lead the technical component of this project. Thank you, NSF!
  • 2022.04Several AIR lab members will go for summer internships at ByteDance, Yousician and Tencent.
  • 2022.04We are happy to release the web-based version of BachDuet. Everyone with a web browser can now improvise with AI in real time in the style of Bach counterpoint! Great work, Yongyi, Christos and Tianyu!
  • 2021.10AIR lab is awarded a New York State Center of Excellence in Data Science grant and a University of Rochester Goergen Institute for Data Science grant. Thank you, NYS and UR!
  • 2021.05Congratulations to Bochen Li on winning a 2021 Outstanding PhD Dissertation Award at the University of Rochester! Well deserved!
  • 2021.05Several AIR lab members are going for a summer internship this year, at Adobe, ByteDance, Chordify, Tencent, and Pandora.
  • 2020.08Check out AIR lab's YouTube channel!
  • 2019.12Our Vroom! search engine for sounds using vocal imitation as queries is online!
  • 2019.10Check out our demo video of BachDuet, a system for real-time interactive duet counterpoint improvisation between human and machine in the Bach chorale style. A brief description of the system is here.
  • 2019.10Check out the AIR lab production for the ISMIR2019 Call for Music - Variations on ISMIR: some funny reflections on AI.

Join Us

We are looking for highly motivated students to join the AIR lab. Students are expected to have a solid background in mathematics, programming, and academic writing. Experience in music activities is a plus. Most importantly, students should be passionate about research in computer audition, music information retrieval, and multimodal learning.

Prospective Ph.D. students: please apply to the ECE Ph.D. program or the CS Ph.D. program and mention Prof. Zhiyao Duan in your application.

UR master's and undergraduate students: if you want to do a project or thesis in the AIR lab, please email Prof. Duan or stop by his office, Room 720, Computer Studies Building.

Funding

Our work is funded by the National Science Foundation under the following grants:

Our work is also funded by the National Institutes of Health (NIH), the National Institute of Justice (NIJ), the Intelligence Advanced Research Projects Activity (IARPA), the National Artificial Intelligence Research Resource (NAIRR) Pilot, the New York State Center of Excellence (CoE) in Data Science, University of Rochester internal awards on AR/VR, health analytics, data science, and integrated research computing, as well as gifts from Adobe, ByteDance, Kwai, IngenID, Meta, and Microsoft. We are very grateful for their support!

National Science Foundation National Institutes of Health National Institute of Justice IARPA New York State Adobe ByteDance Kwai IngenID Meta Microsoft