Hi, I am Xinyu Shi, a final-year Ph.D. candidate at University of Waterloo working with Prof. Jian Zhao. During my PhD, I have spent multiple wonderful summers at Adobe Research (twice) and Microsoft Research as a research intern. My work has been recognized with several awards including Best Paper Award (CHI 2024), Best Paper Honorable Mention Award (CHI 2026), Oral (MM 2022), and Spotlight (MTI-LLM@NeurIPS 2025).

Research Interest

I work on Human–Computer Interaction (HCI) and Human-Centered AI. My research aims to improve human–AI alignment in open-ended creative domains by helping people think through complex problems, communicate their intentions, and collaborate with AI:

  1. Think

    I build cognitive instruments that make complex information visible and manipulable, helping people understand and reason through problems that are difficult to hold in mind. (MondrianPiet )

  2. Communicate

    I develop expressive visual languages that give form to ambiguous and evolving intentions through composable structures for communicating with AI. (BrickifyDiagrammatic Sketching)

  3. Collaborate

    I design shared representations that align AI’s interpretations with human intent as collaboration unfolds. My recent research also evaluate agents’ ability to collaborate with human and explore how they can learn from human interactions. (Notational Animating Hawkeye)

Recent News

Pinned On the 2026–2027 job market. I am seeking positions in both academia and industry. Feel free to reach out if you think I’d be a good fit!

  • 2026/09: Excited to serve as an Associate Chair for CHI 2027!
  • 2026/04: Attending CHI'26 in Barcelona and presenting our award-wining paper Notational Animating!
  • 2025/11: Gave an invited talk at UCSD Design Lab (Title: Interactive Visual Abstractions for Steerable Creativity)
  • 2025/11: Gave an invited talk at UofT iSchool (Title: Interactive Visual Abstractions for Steerable Creativity)
  • 2025/06: Interning at Adobe Research with Rubaiat, Li-Yi, and Cherry this summer!
  • 2025/01: Our paper about design intent expression, Brickify, is accepted by CHI'25!
  • 2024/05: Interning at Adobe Research (San Jose) with Shunan, Jane, Gromit, and Eunyee this summer.
  • 2024/04: Super excited that Piet won Best Paper Award at CHI24!
  • 2024/01: Two papers are accepted by CHI'24!
  • 2023/11: Passed my Comprehensive Exam II, I am a Ph.D. candidate now!
  • 2023/05: Interning at Microsoft Research Asia (Beijing).
  • 2023/04: Attentding CHI 2023 to present De-Stijl at Hamburg, Germany!
  • 2022/06: FeMaSR is accepted by ACM MM'22 and selected as Oral Presentation.
  • 2022/05: Transferred from M.Math to Ph.D. program.
  • 2021/09: Fortunate to receive the 2021 Vector Scholarship from Vector Institute.
  • 2021/09: Started my journey at University of Waterloo!

PUBLICATIONS

  • Diagrammatic Sketching: Exploring Infographic Design Alternatives in Parallel
    Diagrammatic Sketching: Exploring Infographic Design Alternatives in Parallel

    Xinyu Shi, Chun Ye, Shunan Guo, Jane Hoffswell, Gromit Yeuk-Yin Chan, Jian Zhao, Eunyee Koh

    IUI 2027 In Proceedings of the Annual ACM Conference on Intelligent User Interfaces.

    Conditionally Accept (comming soon)
  • Notational Animating: An Interactive Approach to Creating and Editing Animation Keyframes
    Notational Animating: An Interactive Approach to Creating and Editing Animation Keyframes

    Xinyu Shi, Li-Yi Wei, Nanxuan Zhao, Jian Zhao, Rubaiat Habib Kazi

    CHI 2026 In Proc. of ACM CHI Conference on Human Factors in Computing Systems.

    Best Paper Honorable Mention

    PDF Project Page Video DOI
  • Hawkeye: Inspecting Agentic Search Trajectories
    Hawkeye: Inspecting Agentic Search Trajectories

    Crystina Zhang, Zigeng Xu, Xinyu Shi, Raphael Tang, Nandan Thakur, Xueguang Ma, Yao Lu, Jimmy Lin

    SIGIR-AP 2026 In Proc. of the ACM SIGIR Conference on Information Retrieval in the Asia Pacific.

    PDF
  • BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent
    BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent

    Zijian Chen*, Xueguang Ma*, Shengyao Zhuang*, Ping Nie, Kai Zou, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Sahel Sharifymoghaddam, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, Jimmy Lin

    ACL 2026 In Proc. of the Annual Meeting of the Association for Computational Linguistics (ACL).

    Spotlight at NeurIPS MTI-LLM Workshop

    Featured in Anthropic Claude Opus 4.5 launch, also adopted by Microsoft, Databricks, IBM Research, etc.

    Code Paper
  • Brickify: Enabling Expressive Design Intent Specification through Direct Manipulation on Design Tokens
    Brickify: Enabling Expressive Design Intent Specification through Direct Manipulation on Design Tokens

    Xinyu Shi, Yinghou Wang, Ryan Rossi, Jian Zhao

    CHI 2025 In Proc. of ACM CHI Conference on Human Factors in Computing Systems.

    PDF Video 30s Preview DOI
  • Exploring Interactive Color Palettes for Abstraction-Driven Exploratory Image Colorization
    Exploring Interactive Color Palettes for Abstraction-Driven Exploratory Image Colorization

    Xinyu Shi, Mingyu Liu, Ziqi Zhou, Ali Neshati, Ryan Rossi, Jian Zhao

    CHI 2024 In Proc. of ACM CHI Conference on Human Factors in Computing Systems.

    PDF Video 30s Preview DOI
  • De-Stijl: Facilitating Graphics Design with Interactive 2D Color Palette Recommendation
    De-Stijl: Facilitating Graphics Design with Interactive 2D Color Palette Recommendation

    Xinyu Shi, Ziqi Zhou, Julia Zhang, Ali Neshati, Anjul Tyagi, Ryan Rossi, Shunan Guo, Fan Du, Jian Zhao

    CHI 2023 In Proc. of ACM CHI Conference on Human Factors in Computing Systems.

    Featured in Adobe News | Waterloo News | TechTimes | GraphicSpeak

    PDF Code Video 30s Preview Dataset (3.4G) DOI
  • Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors
    Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors

    Chaofeng Chen*, Xinyu Shi*, Yipeng Qin, Xiaoming Li, Xiaoguang Han, Tao Yang, Shihui Guo

    MM 2022 In Proc. of ACM International Conference on Multimedia.

    Oral Presentation (Top 5%)

    PDF Code DOI
  • Teaching American Sign Language in Mixed Reality
    Teaching American Sign Language in Mixed Reality

    Qijia Shao, Amy Sniffen, Julien Blanchet, Megan E. Hillis, Xinyu Shi, Themistoklis K. Haris, Jason Liu, Melissa Malzkuhn, Lorna C. Quandt, James Mahoney, David J.M. Kraemer, Xia Zhou, Devin Balkcom

    UbiComp 2021 / IMWUT 2020 Proc. of ACM Interactive, Mobile, Wearable and Ubiquitous Technologies.

    PDF Video DOI
  • Accurate and Fast Classification of Foot Gestures for Virtual Locomotion
    Accurate and Fast Classification of Foot Gestures for Virtual Locomotion

    Xinyu Shi, Junjun Pan, Zeyong Hu, Juncong Lin, Shihui Guo, Minghong Liao, Ye Pan, Ligang Liu

    ISMAR 2019 In Proc. of IEEE International Symposium on Mixed and Augmented Reality.

    PDF Video DOI