I am a Research Scientist at RIKEN AIP, an Associate Professor at The University of Tokyo, and a Research Scientist (part time) at Sakana AI.
I work on statistical machine learning, with particular interest in model evaluation, alignment, and safety.
Previously, I was a Lecturer at the University of Tokyo. I did my PhD at the University of Tokyo, advised by Prof. Masashi Sugiyama.
During my PhD, I was an Applied Scientist intern at Amazon.com, a Google PhD Fellow, and a Research Fellow at JSPS (DC2).
Prior to that, I was an Assistant Manager at Sumitomo Mitsui DS Asset Management.
I received the MSc from the University of Tokyo and the Bachelor of Economics from Keio University.
Preprints
I. Sugiura, D. Hattori, K. Araragi, K. Ogawa, S. Onose, T. Makino, T. Usuki, T. Ishida. CoffeeBench: Benchmarking Long-Horizon LLM Agents in Heterogeneous Multi-Agent Economies. arXiv preprint arXiv:2606.16613, 2026. Accepted at the FAGEN @ ICML 2026 workshop.
T. Lodkaew, J. Ackermann, S. Nishimori, N. Charoenphakdee, M. Sugiyama, T. Ishida. Do Coding Agents Deceive Us? Detecting and Preventing Cheating via Capped Evaluation with Randomized Tests. arXiv preprint arXiv:2606.07379, 2026.
Accepted at the AgenticUQ @ ICML 2026 workshop.
F. Futami, T. Ishida. Unified Approach for Weakly Supervised Multicalibration. arXiv preprint arXiv:2605.09857, 2026.
S. Ono, J. Ackermann, S. Nishimori, T. Ishida, M. Sugiyama. Mitigating Reward Hacking in RLHF via Advantage Sign Robustness. arXiv preprint arXiv:2604.02986, 2026. Accepted at the EIML @ ICML 2026 workshop.
C.-K. Chiang, T. Ishida, M. Sugiyama. LLM Routing with Dueling Feedback. arXiv preprint arXiv:2510.00841, 2025.
Papers (peer reviewed)
J. Ackermann, M. Noukhovitch, T. Ishida, M. Sugiyama. Gradient Regularization Prevents Reward Hacking in Reinforcement Learning from Human Feedback and Verifiable Rewards. In Proceedings of the 43rd International Conference on Machine Learning (ICML2026), to appear.
T. Ishida, T. Lodkaew, I. Yamane. CapBencher: Give Your LLM Benchmark a Built-in Alarm for Test-Set Overfitting In Proceedings of the 43rd International Conference on Machine Learning (ICML2026), to appear.
An earlier version of this paper, titled "How Can I Publish My LLM Benchmark Without Giving the True Answers Away?", was selected as an oral at the MemFM @ ICML 2025 workshop.
R. Yin, T. Ishida, M. Sugiyama. Towards Scalable Oversight via Partitioned Human Supervision. In Proceedings of Fourteenth International Conference on Learning Representations (ICLR2026).
I. Sugiura, T. Ishida, T. Makino, C. Tazuke, T. Nakagawa, K. Nakago, D. Ha. EDINET-Bench: Evaluating LLMs on Complex Financial Tasks using Japanese Financial Statements. In Proceedings of Fourteenth International Conference on Learning Representations (ICLR2026).
R. Ushio, T. Ishida, M. Sugiyama. Practical estimation of the optimal classification error with soft labels and calibration. In Proceedings of Fourteenth International Conference on Learning Representations (ICLR2026).
T. Lodkaew, T. Fang, T. Ishida, M. Sugiyama. Importance Weighting for Aligning Language Models under Deployment Distribution Shift. Transactions on Machine Learning Research (TMLR), 2025. Selected for expert certification!
J. Ackermann, T. Ishida, M. Sugiyama. Off-Policy Corrected Reward Modeling for Reinforcement Learning from Human Feedback. In Conference on Language Modeling (COLM2025).
W. Wang, T. Ishida, Y.-J. Zhang, G. Niu, M. Sugiyama. Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical. In Proceedings of the 41st International Conference on Machine Learning (ICML2024).
T. Ishida, I. Yamane, N. Charoenphakdee, G. Niu, M. Sugiyama. Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classification. In Proceedings of Eleventh International Conference on Learning Representations (ICLR2023). Selected for oral (notable-top-5%) presentation!
I. Yamane, Y. Chevaleyre, T. Ishida, F. Yger. Mediated Uncoupled Learning and Validation with Bregman Divergences: Loss Family with Maximal Generality. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS2023).
Z. Lu, C. Xu, B. Du, T. Ishida, L. Zhang, & M. Sugiyama. LocalDrop: A hybrid regularization for deep neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.44, No.7, pp.3590-3601, 2022.
H. Ishiguro, T. Ishida, & M. Sugiyama. Learning from Noisy Complementary Labels with Robust Loss Functions. IEICE Transactions on Information and Systems, Vol.E105-D, No.2, pp.364-376, Feb. 2022.
T. Ishida, I. Yamane, T. Sakai, G. Niu, M. Sugiyama. Do We Need Zero Training Loss After Achieving Zero Training Error? In Proceedings of Thirty-seventh International Conference on Machine Learning (ICML2020). (@hardmaru)
T. Ishida, G. Niu, A. K. Menon, and M. Sugiyama. Complementary-label learning for arbitrary losses and models. In Proceedings of Thirty-sixth International Conference on Machine Learning (ICML2019).
T. Ishida, G. Niu, and M. Sugiyama. Binary classification from positive-confidence data. In Advances in Neural Information Processing Systems 31 (NeurIPS2018). [Press Release] [ScienceDaily] [PHYS.ORG] [ASIAN SCIENTISTS] [ISE Magazine] [RIKEN RESEARCH] [日刊工業新聞] [ITmedia] Selected for spotlight presentation!
T. Ishida, G. Niu, W. Hu, and M. Sugiyama. Learning from complementary labels. In Advances in Neural Information Processing Systems 30 (NeurIPS2017). [日刊工業新聞]
T. Ishida. Forecasting Nikkei 225 Returns By Using Internet Search Frequency Data. In Securities Analysts Journal, Vol.52, No.6, pp.83-93, 2014. Selected as Research Notes.
Books
M. Sugiyama, H. Bao, T. Ishida, N. Lu, T. Sakai, & G. Niu. Machine Learning from Weak Supervision: An Empirical Risk Minimization Approach. Adaptive Computation and Machine Learning series, The MIT Press, 2022. [link]
Join our team
At UTokyo, I am co-running Machine Learning and Statistical Data Analysis Lab (Sugiyama-Yokoya-Ishida Lab). I belong to the Department of Complexity Science and Engineering, the Department of Computer Science, and the Department of Information Science. If you are interested in joining our team, see here.