Heitor R. Guimarães

📍 Montréal, QC, Canada

I am a Research Scientist at RBC Borealis. I obtained my PhD from the University of Quebec (INRS–EMT), where I worked with Prof. Tiago H. Falk and Prof. Anderson Avila.

I am broadly interested in developing ML models that learn useful abstractions from heterogeneous data and generalize across domains to support agent perception, planning, decision-making, and interaction in simulated and real-world environments.

Some of my current research interests include:

Structured and Tabular Foundation Models: developing models for heterogeneous, tabular, and relational data, with applications across scientific, financial, behavioral, and enterprise-scale prediction problems.
Multimodal Perception and Representation Learning: learning robust and transferable representations from high-dimensional signals (e.g., speech and vision) to support perception and interaction.
Robustness and Real-World Deployment: developing models that represent the world, generalize under distribution shifts, and remain reliable under resource constraints, adversarial conditions, and deployment requirements.

During my PhD, I worked on audio intelligence and machine perception, focusing on how foundation models can be adapted to real-world speech and bioacoustic applications. My work studied efficiency and robustness to distribution shifts and adversarial attacks. I was also fortunate to intern at Adobe Research and Meta, where I worked on generative speech enhancement for full-bandwidth studio-quality audio and small-footprint speech enhancement methods for smart glasses, respectively.

I am always open to collaborations and to learning about new ideas along these areas. Please feel free to reach out!

news

Mar 2026	I have joined RBC Borealis to work on Foundation models for credit modeling!
Feb 2026	I have officially defended my Ph.D.!

selected publications

IEEE ICASSP

DiTSE: High-Fidelity Generative Speech Enhancement via Latent Diffusion Transformers

Heitor R. Guimarães, Jiaqi Su, Rithesh Kumar, and 2 more authors

In IEEE ICASSP, 2026

Abs DOI

Real-world speech recordings suffer from degradations such as background noise and reverberation. Speech enhancement aims to mitigate these issues by generating clean high-fidelity signals. While recent generative approaches for speech enhancement have shown promising results, they still face two major challenges: (1) content hallucination, where plausible phonemes generated differ from the original utterance; and (2) inconsistency, failing to preserve speaker’s identity and paralinguistic features from the input speech. In this work, we introduce DiTSE (Diffusion Transformer for Speech Enhancement), which addresses quality issues of degraded speech in full bandwidth. Our approach employs a latent diffusion transformer model together with robust conditioning features, effectively addressing these challenges while remaining computationally efficient. Experimental results from both subjective and objective evaluations demonstrate that DiTSE achieves state-of-the-art audio quality that, for the first time, matches real studio-quality audio from the DAPS dataset. Furthermore, DiTSE significantly improves the preservation of speaker identity and content fidelity, reducing hallucinations across datasets compared to state-of-the-art enhancers. Audio samples are available at: http://hguimaraes.me/DiTSE
IEEE ASRU

Improving Resource-Efficient Speech Enhancement via Neural Differentiable DSP Vocoder Refinement

Heitor R. Guimarães, Ke Tan, Juan Azcarreta, and 4 more authors

In IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2025

Abs DOI

Deploying speech enhancement (SE) systems in wearable devices, such as smart glasses, is challenging due to the limited computational resources on the device. Although deep learning methods have achieved high-quality results, their computational cost limits their feasibility on embedded platforms. This work presents an efficient end-to-end SE framework that leverages a Differentiable Digital Signal Processing (DDSP) vocoder for high-quality speech synthesis. First, a compact neural network predicts enhanced acoustic features from noisy speech: spectral envelope, fundamental frequency (F0), and periodicity. These features are fed into the DDSP vocoder to synthesize the enhanced waveform. The system is trained end-to-end with STFT and adversarial losses, enabling direct optimization at the feature and waveform levels. Experimental results show that our method improves intelligibility and quality by 4% (STOI) and 19% (DNSMOS) over strong baselines without significantly increasing computation, making it well-suited for real-time applications.
IEEE MetroAgriFor

Adapting Self-Supervised Features for Background Speech Detection in Beehive Audio Recordings

Heitor R. Guimarães, Mahsa Abdollahi, Yi Zhu, and 4 more authors

In IEEE MetroAgriFor, 2023

Best Paper Award Abs DOI

Best Paper Award presented by a Young Researcher

The decline in honey bee populations and the labor-intensive nature of manual hive inspection have sparked interest in automating beehive monitoring. Acoustic monitoring has emerged as a versatile modality for this purpose. Beehive audio has proven useful in detecting events such as swarming, queen absence, parasite infestation, and assessing hive strength. However, external factors such as rain, wind, traffic noise, and background voices of beekeepers can degrade the quality of recorded bee audio and impact the effectiveness of beehive monitoring. In this study, we focus on the detection of background beekeepers speech, which has shown to significantly degrade audio-based hive monitoring performance. In particular, we propose the use of universal audio representations extracted from self-supervised models to predict whether a given audio frame contains speech or only bee sounds. Experimental results on the NU-Hive dataset show that the proposed methodology applied on top of the WavLM Base+ model can outperform state-of-the-art voice activity detectors by a relative improvement of 19% and 16% in the ROC-AUC and F1-score metrics, respectively. Furthermore, we show that audio representations can be reliably used for unsupervised anomaly detection.