Hi, I am Xinyu Shi, a last-year Ph.D. candidate at University of Waterloo in Canada. I am advised by Prof. Jian Zhao in WVisdom Research Group, affiliated with the WaterlooHCI Lab. During my PhD, I have spent multiple wonderful summers at Adobe Research (twice) and Microsoft Research as a research intern. My work has been recognized with a Best Paper Award and an Honorable Mention Award at CHI.

I work on Human–Computer Interaction (HCI) and Human-Centered AI. My research improves human–AI alignment in open-ended creative domains, enabling AI systems to better serve individual users’ goals and values. I pursue this through two complementary threads — alignment through interaction and alignment through training: (1) designing new human–AI interaction paradigms — constructing common ground between humans and AI at use time to make ambiguous intent explicit and model behavior inspectable; and (2) integrating human insights into AI models — eliciting the tacit knowledge, constraints, and judgment of domain experts and translating them into computational representations for model training and fine-tuning.

Outside of research, I love skiing, playing badminton, rock climbing (only belay, no bouldering), and travelling. I am the mom of two cats (a boy and a girl), they are super sweet!

Recent News
  • 2026/04: Attending CHI'26 in Barcelona and presenting our award-wining paper Notational Animating!
  • 2026/02: Our paper Notational Animating received Best Paper Honorable Mention Award at CHI 2026!
  • 2025/11: Gave an invited talk at UCSD Design Lab (Title: Interactive Visual Abstractions for Steerable Creativity)
  • 2025/11: Gave an invited talk at UofT iSchool (Title: Interactive Visual Abstractions for Steerable Creativity)
  • 2025/06: Interning at Adobe Research with Rubaiat, Li-Yi, and Cherry this summer!
  • 2025/01: Our paper about design intent expression, Brickify, is accepted by CHI'25!
  • 2024/05: Interning at Adobe Research (San Jose) with Shunan, Jane, Gromit, and Eunyee this summer.
  • 2024/04: Super excited that Piet won Best Paper Award at CHI24!
  • 2024/01: Two papers are accepted by CHI'24!
  • 2023/11: Passed my Comprehensive Exam II, I am a Ph.D. candidate now!
  • 2023/05: Interning at Microsoft Research Asia (Beijing).
  • 2023/04: Attentding CHI 2023 to present De-Stijl at Hamburg, Germany!
  • 2022/06: FeMaSR is accepted by ACM MM'22 and selected as Oral Presentation.
  • 2022/05: Transferred from M.Math to Ph.D. program.
  • 2021/09: Fortunate to receive the 2021 Vector Scholarship from Vector Institute.
  • 2021/09: Started my journey at University of Waterloo!

PUBLICATIONS

  • Notational Animating: An Interactive Approach to Creating and Editing Animation Keyframes

    Xinyu Shi, Li-Yi Wei, Nanxuan Zhao, Jian Zhao, Rubaiat Habib Kazi

    CHI 2026 In Proc. of ACM CHI Conference on Human Factors in Computing Systems.

    Best Paper Honorable Mention

    PDF Project Page Video DOI
  • BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent
    BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent

    Zijian Chen*, Xueguang Ma*, Shengyao Zhuang*, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Sahel Sharifymoghaddam, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, Jimmy Lin

    ACL 2026 In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL).

    NeurIPS Workshop NeurIPS 2025 Workshop on Multi-Turn Interactions in Large Language Models.

    Spotlight at NeurIPS MTI-LLM Workshop

    Code Paper
  • Brickify: Enabling Expressive Design Intent Specification through Direct Manipulation on Design Tokens

    Xinyu Shi, Yinghou Wang, Ryan Rossi, Jian Zhao

    CHI 2025 In Proc. of ACM CHI Conference on Human Factors in Computing Systems.

    PDF Video 30s Preview DOI
  • Exploring Interactive Color Palettes for Abstraction-Driven Exploratory Image Colorization

    Xinyu Shi, Mingyu Liu, Ziqi Zhou, Ali Neshati, Ryan Rossi, Jian Zhao

    CHI 2024 In Proc. of ACM CHI Conference on Human Factors in Computing Systems.

    PDF Video 30s Preview DOI
  • De-Stijl: Facilitating Graphics Design with Interactive 2D Color Palette Recommendation

    Xinyu Shi, Ziqi Zhou, Julia Zhang, Ali Neshati, Anjul Tyagi, Ryan Rossi, Shunan Guo, Fan Du, Jian Zhao

    CHI 2023 In Proc. of ACM CHI Conference on Human Factors in Computing Systems.

    Featured in Adobe News | Waterloo News | TechTimes | GraphicSpeak

    PDF Code Video 30s Preview Dataset (3.4G) DOI
  • Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors
    Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors

    Chaofeng Chen*, Xinyu Shi*, Yipeng Qin, Xiaoming Li, Xiaoguang Han, Tao Yang, Shihui Guo

    MM 2022 In Proc. of ACM International Conference on Multimedia.

    Oral Presentation (Top 5%)

    PDF Code DOI
  • Teaching American Sign Language in Mixed Reality
    Teaching American Sign Language in Mixed Reality

    Qijia Shao, Amy Sniffen, Julien Blanchet, Megan E. Hillis, Xinyu Shi, Themistoklis K. Haris, Jason Liu, Melissa Malzkuhn, Lorna C. Quandt, James Mahoney, David J.M. Kraemer, Xia Zhou, Devin Balkcom

    UbiComp 2021 / IMWUT 2020 Proc. of ACM Interactive, Mobile, Wearable and Ubiquitous Technologies.

    PDF Video DOI
  • Accurate and Fast Classification of Foot Gestures for Virtual Locomotion
    Accurate and Fast Classification of Foot Gestures for Virtual Locomotion

    Xinyu Shi, Junjun Pan, Zeyong Hu, Juncong Lin, Shihui Guo, Minghong Liao, Ye Pan, Ligang Liu

    ISMAR 2019 In Proc. of IEEE International Symposium on Mixed and Augmented Reality.

    PDF Video DOI

EXTENDED ABSTRACTS, WORKSHOPS

  • Comprehensive Sketching: Exploring Infographic Design Alternatives in Parallel

    Xinyu Shi, Shunan Guo, Jane Hoffswell, Gromit Yeuk-Yin Chan, Victor S. Bursztyn, Jian Zhao, Eunyee Koh

    CHI LBW 2025 In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems.

    PDF Video DOI