Takashi Ishida

English | 日本語

I am a Research Scientist at RIKEN AIP, an Associate Professor at The University of Tokyo, and a Research Scientist (part time) at Sakana AI. I work on statistical machine learning, with particular interest in model evaluation and alignment for reliable and safe AI.

Previously, I was a Lecturer at the University of Tokyo. I did my PhD at the University of Tokyo, advised by Prof. Masashi Sugiyama. During my PhD, I was an Applied Scientist intern at Amazon.com, a Google PhD Fellow, and a Research Fellow at JSPS (DC2). Prior to that, I was an Assistant Manager at Sumitomo Mitsui DS Asset Management. I received the MSc from the University of Tokyo and the Bachelor of Economics from Keio University.

Preprints

• S. Ono, J. Ackermann, S. Nishimori, T. Ishida, M. Sugiyama. Mitigating Reward Hacking in RLHF via Advantage Sign Robustness. arXiv preprint arXiv:2604.02986, 2026. [arXiv]
  

• J. Ackermann, M. Noukhovitch, T. Ishida, M. Sugiyama. Gradient Regularization Prevents Reward Hacking in Reinforcement Learning from Human Feedback and Verifiable Rewards. arXiv preprint arXiv:2602.18037, 2026. [arXiv] [code]
  

• C.-K. Chiang, T. Ishida, M. Sugiyama. LLM Routing with Dueling Feedback. arXiv preprint arXiv:2510.00841, 2025. [arXiv]
  

• T. Ishida, T. Lodkaew, I. Yamane. How Can I Publish My LLM Benchmark Without Giving the True Answers Away? arXiv preprint arXiv:2505.18102, 2025. [arXiv] [slides] [video] [toolkit] [dataset] [blog] Accepted as an oral presentation at the MemFM @ ICML 2025 workshop!
  

Papers (peer reviewed)

• R. Yin, T. Ishida, M. Sugiyama. Towards Scalable Oversight via Partitioned Human Supervision. In Proceedings of Fourteenth International Conference on Learning Representations (ICLR2026), to appear. [arXiv] [code]
  

• I. Sugiura, T. Ishida, T. Makino, C. Tazuke, T. Nakagawa, K. Nakago, D. Ha. EDINET-Bench: Evaluating LLMs on Complex Financial Tasks using Japanese Financial Statements. In Proceedings of Fourteenth International Conference on Learning Representations (ICLR2026), to appear. [arXiv] [blog] [bench code] [tool code] [dataset]
  

• R. Ushio, T. Ishida, M. Sugiyama. Practical estimation of the optimal classification error with soft labels and calibration. In Proceedings of Fourteenth International Conference on Learning Representations (ICLR2026), to appear. [arXiv] [code]
  

• T. Lodkaew, T. Fang, T. Ishida, M. Sugiyama. Importance Weighting for Aligning Language Models under Deployment Distribution Shift. Transactions on Machine Learning Research (TMLR), 2025. [OpenReview] [code] [poster] Selected for expert certification!
  

• J. Ackermann, T. Ishida, M. Sugiyama. Off-Policy Corrected Reward Modeling for Reinforcement Learning from Human Feedback. In Conference on Language Modeling (COLM2025). [arXiv] [code]
  

• W. Wang, T. Ishida, Y.-J. Zhang, G. Niu, M. Sugiyama. Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical. In Proceedings of the 41st International Conference on Machine Learning (ICML2024). [arXiv] [PMLR] [code]
  

• T. Ishida, I. Yamane, N. Charoenphakdee, G. Niu, M. Sugiyama. Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classification. In Proceedings of Eleventh International Conference on Learning Representations (ICLR2023). [arXiv] [OpenReview] [code] [Fashion-MNIST-H (Papers with Code)] [Video] [nnabla ディープラーニングチャンネル] Selected for oral (notable-top-5%) presentation!
  

• I. Yamane, Y. Chevaleyre, T. Ishida, F. Yger. Mediated Uncoupled Learning and Validation with Bregman Divergences: Loss Family with Maximal Generality. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS2023). [paper] [code] [video]
  

• Z. Lu, C. Xu, B. Du, T. Ishida, L. Zhang, & M. Sugiyama. LocalDrop: A hybrid regularization for deep neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.44, No.7, pp.3590-3601, 2022. [paper]
  

• H. Ishiguro, T. Ishida, & M. Sugiyama. Learning from Noisy Complementary Labels with Robust Loss Functions. IEICE Transactions on Information and Systems, Vol.E105-D, No.2, pp.364-376, Feb. 2022. [paper]
  

• T. Ishida, I. Yamane, T. Sakai, G. Niu, M. Sugiyama. Do We Need Zero Training Loss After Achieving Zero Training Error? In Proceedings of Thirty-seventh International Conference on Machine Learning (ICML2020). [paper] [code] [video] [@hardmaru's post]
  

• T. Ishida, G. Niu, A. K. Menon, and M. Sugiyama. Complementary-label learning for arbitrary losses and models. In Proceedings of Thirty-sixth International Conference on Machine Learning (ICML2019). [paper] [poster] [slides] [video] [code]
  

• T. Ishida, G. Niu, and M. Sugiyama. Binary classification from positive-confidence data. In Advances in Neural Information Processing Systems 31 (NeurIPS2018). [paper] [poster] [slides] [video] [code] [Press Release] [ScienceDaily] [PHYS.ORG] [ASIAN SCIENTISTS] [ISE Magazine] [RIKEN RESEARCH] [日刊工業新聞] [ITmedia] Selected for spotlight presentation!
  

• T. Ishida, G. Niu, W. Hu, and M. Sugiyama. Learning from complementary labels. In Advances in Neural Information Processing Systems 30 (NeurIPS2017). [paper] [code] [日刊工業新聞]

• T. Ishida. Forecasting Nikkei 225 Returns By Using Internet Search Frequency Data. In Securities Analysts Journal, Vol.52, No.6, pp.83-93, 2014. Selected as Research Notes.

Books

• M. Sugiyama, H. Bao, T. Ishida, N. Lu, T. Sakai, & G. Niu. Machine Learning from Weak Supervision: An Empirical Risk Minimization Approach. Adaptive Computation and Machine Learning series, The MIT Press, 2022. [link]
  

Recent Talks

• Apr, 2026. Reliable Model Evaluation Without Full Ground Truth. Statistical Safeguarding Workshop 2026, Tokyo. [workshop page]
  

• Mar, 2026. Reliable Model Evaluation Without Full Ground Truth. The 2nd RIKEN AIP - IIT Hyderabad Joint Workshop (online).
  

• Aug, 2025. Ensuring Contamination-Robustness in LLM Benchmark Design. Department of Computer Science, Hong Kong Baptist University, Hong Kong. [slides] [event page]
  

• Aug, 2025. EDINET-Bench: Evaluating LLMs on Complex Financial Tasks using Japanese Financial Statements (in Japanese). Weights & Biases Japan's Monthly Meetup, Tokyo. [event page]
  

• July, 2025. How Can I Publish My LLM Benchmark Without Giving the True Answers Away? MemFM@ICML2025 workshop, Vancouver, Canada. [slides] [poster] [video] [workshop page]
  

• July, 2025. Data-Centric Machine Learning via Bayes-accuracy Estimation. Sydney - RIKEN AIP Joint AI Workshop, Sydney, Australia. [slides] [workshop page]
  

• July, 2025. Ensuring Contamination-Robustness in LLM Benchmark Design. School of Computer Science, University of Sydney, Sydney, Australia.
  

• May, 2025. How Can I Publish My LLM Benchmark Without Giving the True Answers Away? International Symposium on Trustworthy Foundation Models, MBZUAI, Abu Dhabi, UAE. [symposium page]
  

• Nov, 2024. The evolving role of human supervision in modern AI systems. International Workshop on Weakly Supervised Learning 2024, University of Queensland, Brisbane, Australia. [workshop page]
  

• Oct, 2024. Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classification. Tokyo-Oulu Joint Symposium on Human-AI Interaction, Tokyo. [symposium page]
  

• July, 2024. How and how far can we supervise AI systems? ENSAI/CREST, Renne, France.
  

Grants and Donations

• ASPIRE 2025 Japan-UK joint call for collaborations in Advancing Human-Centered AI, JST, 2025--2031 [press release] [project page]

• Google Research Grant 2025

• Grant-in-Aid for Scientific Research (B), JSPS, 2022--2025 (Co-Investigator)

• Grant-in-Aid for Early-Career Scientists, JSPS, 2022--2026

• Frontier of mathematics and information science, ACT-X, JST, 2020--2023 Selected as one of the achievements (JP/EN) at JST in FY2024!

• Grant-in-Aid for JSPS Fellows, JSPS, 2020--2021

Awards & Achievements

• Asian Trustworthy Machine Learning (ATML) Fellowship, 2025

• Expert Reviewer, Transactions on Machine Learning Research (received in 2024)

• Achievements, Information and Communications Technology (JP/EN/PDF) at JST, 2024

• Funai Information Technology Award for Young Researchers, 2022 (received in 2023)

• IEICE TC-IBISML Research Award Finalist, 2020 (received in 2021)

• Dean's Award for Outstanding Achievement, Graduate School of Frontier Sciences 2021

• Toyota/Dwango AI Scholarship, 2020 – 2021

• Award Finalist, IBIS2020

• Top 10% reviewer, NeurIPS 2020

• JSPS Research Fellowship for Young Scientists (DC2), 2020 – 2021

• Top 50% reviewer, NeurIPS 2019

• Google PhD Fellowship 2019

• IEICE TC-IBISML Research Award, 2017 (received in 2018)

Service

• Inaugural Organizing Committee Member: Google Developer Group AI for Science Japan (2025 – present)

• Committee member: FY2022 -- FY2025 IEICE, Information-Based Induction Sciences and Machine Learning (IBISML) Technical Group

• Workshop organizer: PC member, IBIS2023. Executive Group, TrustML Young Scientist Seminars. Organizer, NeurIPS Meetup Japan 2021.

• Area chair: [2026] ICLR, ICML, NeurIPS [2025] ICLR, ICML, ACML

• Conference PC/reviewer: [2025] NeurIPS [2024] ICLR, AISTATS, ICML, ACML, NeurIPS [2023] ICLR [2022] ICLR, AISTATS, ICML, NeurIPS, [2021] NeurIPS, ACML, ICLR, UAI, ICML, [2020] NeurIPS (top 10% reviewer), ICML, ICLR, AAAI, AISTATS, UAI, ACML, [2019] NeurIPS (top 50% reviewer), ICML, AAAI, AISTATS, UAI, ACML

• Journal action editor: Transactions on Machine Learning Research (2024 -- 2026)

• Journal reviewer: IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Transactions on Image Processing, Journal of Information Processing, Machine Learning, Artificial Intelligence (AIJ), Transactions on Machine Learning Research (TMLR; Expert Reviewer in 2024)

• Workshop reviewer: 3rd edition of Reproducibility Challenge @ NeurIPS 2019, IJCAI 2021 Workshop on Weakly Supervised Representation Learning, ICML 2025 Workshop on The Impact of Memorization on Trustworthy Foundation Models

Courses at UTokyo

• Advanced Data Analysis (Graduate, S1S2, English) with Prof. Masashi Sugiyama: 2021, 2023, 2025

• Statistical Machine Learning (Undergraduate, S1S2, Japanese) with Prof. Issei Sato and Prof. Masashi Sugiyama: 2021-present

• Statistics and Optimization (Undergraduate, A1A2, Japanese) with Prof. Issei Sato and Prof. Masashi Sugiyama: 2021-present

• Intelligent Systems (Undergraduate, A1A2, Japanese) with Prof. Issei Sato, Prof. Masashi Sugiyama, and Prof. Yusuke Miyao: 2021-present

• Machine Learning (UTokyo Extension, Japanese): 2024 Spring

Tools

• notations-cli: Extract notation tables from arXiv papers using LLMs (Mar 2026)

• bibfixer: Standardize BibTeX entries using LLMs and web search (Sep 2025)

• arxiv-latex-mcp: MCP server that fetches and processes arXiv LaTeX sources (Apr 2025)

• arxiv-to-prompt: Transform arXiv papers into a single LaTeX usable as a prompt (Feb 2025)

• paper2slides: Transform arXiv papers into slides using LLMs (Jul 2024)

Join our team

At UTokyo, I am co-running Machine Learning and Statistical Data Analysis Lab (Sugiyama-Yokoya-Ishida Lab). I belong to the Department of Complexity Science and Engineering, the Department of Computer Science, and the Department of Information Science. If you are interested in joining our team, see here.