Jaisidh Singh

I am a master’s student at the University of Tübingen studying machine learning and a fellow at Zuse School ELIZA. I’m also a guest researcher at the Max Planck Institute for Intelligent Systems Tübingen advised by Antonio Orvieto. Currently, I’m writing my master’s thesis on the scaling behaviour of LLMs with hybrid attention with . Previously, I was an intern at Bosch Research India during my undergrad at IIT Jodhpur.

I am currently working on scaling behaviour of hybrid-attention LLMs for my master’s thesis with Dr. Aaron Klein from OpenEuroLLM. Particularly, I’m analysing how various ratios of linear-to-dense attention influences hyper-parameters across different scales. Pretraining and scaling behaviour influencing downstream choices for data mixtures reflect my primary interests.

If you’re interested in collaborating or chatting about these topics, reach out to me via email. I like getting messages! For more information about my publications or work experience, you can check my resume.