Md Rifat Arefin

About Me

I am a Ph.D. student at Mila and University of Montreal supervised by Professor Irina Rish. In the past, I had worked with Yoshua Bengio and Yann LeCun.

My research focuses on representation learning, and generalization in large-scale systems. My goal is to understand how large-scale deep learning systems are very useful in compressing data and learning useful abstract representations. I am currently studying how architectural and optimization-induced inductive bias (Simplicity Bias) in transformers shape the geometry of representations, aiming to improve generalization and multi-step reasoning. I am also explroing how transformers learn hierarchichal memory using compression.

Long before, I worked on alternative algebra like complex numbers and quaternion to build neural networks. In industry, I’ve interned at Amazon, Samsung, ServiceNow, Recursion and am currently interning at Meta.

Research Interests

• Representation Learning, Hierarchichal Memory Learning, Architecture, Optimization, Random Matrix Theory, Information Theory, Tensor Product Represention Theory

News

• [Augut 2025] Started Internship at Meta.
• [May 2025] Our paper “Layer by Layer: Uncovering Hidden Representations in Language Models” is accepted with Spotlight Oral @ ICML, 2025.
• [April 2025] Attended ICLR 2025 and presented our poster on Seq-VCR.
• [February 2025] Started Research Internship at Samsung AI Lab, Montreal, Canada.
• [January 2025] 1 paper submitted to ICML 2025.
• [January 2025] Our paper “Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning” is accepted @ ICLR, 2025.
• [January 2025] Finished Applied Scientist Research Internship at Amazon, Seattle, USA.
• [October 2024] Started Applied Scientist Research Internship at Amazon, Seattle, USA.
• [October 2024] Our paper on representation compression in LLMs is accepted to Compression Workshop @ NeurIPS, 2024.
• [October 2024] 1 paper submitted to ICLR 2025.
• [July 2024] Attended ICML 2024 and presented our poster.
• [June 2024] 1 paper accepted at ICML 2024 Workshop on Foundation Models in the Wild.
• [May 2024] Started as Visiting Researcher at ServiceNow Research.
• [May 2024] Our paper on Spurious Correlation is accepted at ICML, 2024.
• [February 2024] 1 paper submitted to ICML, 2024.
• [January 2024] Passed PhD Candidacy Exam.
• [May 2023] Started Internship at Recursion Pharmaceuticals.

Services

Conference Reviewers

• International Conference on Machine Learning (ICML) 2025
• International Conference on Learning Representations (ICLR) 2025
• Conference on Neural Information Processing Systems (NeurIPS)2024, 2025
• International Conference on Artificial Intelligence and Statistics (AISTATS) 2024

Scolarships, Grants and Awards

• Notable Reviewer of ICLR 2025
• IVADO PhD Excellence Scholarship (September 2021 - August 2024)
• 1st Runner up in PROBA-V Super Resolution Competition organized by European Space Agency, Team: "rarefin"
• Deep and Reinforcement Learning Summer School 2019 Travel Grant
• Obtained President Scholarship during the study of Masters
• IEEE BDS best paper award in International conference on Computer and information Technology 2015

Publications

Powered by Jekyll and Minimal Light theme.