Jaisidh Singh

I am a master’s student at the University of Tübingen studying machine learning and a fellow at Zuse School ELIZA. I’m also a guest researcher at the Max Planck Institute for Intelligent Systems Tübingen advised by Antonio Orvieto. Currently, I’m writing my master’s thesis on the scaling behaviour of LLMs with hybrid attention with . Previously, I was an intern at Bosch Research India during my undergrad at IIT Jodhpur.

I am currently working on scaling behaviour of hybrid-attention LLMs for my master’s thesis with Dr. Aaron Klein from OpenEuroLLM. Particularly, I’m analysing how various ratios of linear-to-dense attention influences hyper-parameters across different scales. Pretraining and scaling behaviour influencing downstream choices for data mixtures reflect my primary interests.

Coming from an engineering background, I like to harmonise theory with practice and move quickly but surely. Challenging assumptions before accepting them is important in my approach to ML. For instance, my MSc thesis exists simply because frontier technical reports never showed explicit scaling studies for different hybridisation strategies, and so I took it up.

If you’re interested in collaborating or chatting about these topics, reach out to me via email. I like getting messages! For more information about my publications or work experience, you can check my resume.