Hiroyasu Akada (朱田 浩康)

Hi, I am Hiroyasu Akada from Japan .

I am a Ph.D. student in the Visual Computing and Artificial Intelligence Department at Max Planck Institute for Informatics with my supervisor Dr. Vladislav Golyanik and Prof. Christian Theobalt.

Previously, I obtained my MS and BS at Keio University, with Prof. Masaki Takahashi. During my MS, I had a great opportunity to work with Prof. Peter Wonka at KAUST.
Also, I went to University of California, Berkeley to study Business Administration and spent a great time at Tencent in California, USA as an intern.

E-mail: hakada@mpi-inf.mpg.de
Google Scholar  /  Twitter /  Linkedin /  Github

[May 2024] Two papers were invited to a poster session in EgoVis workshop at CVPR 2024.
[Feb 2024] Two papers got accepted to CVPR 2024 main conference (Highlight and Poster).
[Oct 2022] Paper "UnrealEgo" was invited to a spotlight session in Ego4D workshop and a poster session in HBHA workshop at ECCV 2022.
[Aug 2022] I'm pleased to announce that I received the Nakajima Foundation Scholarship with financial support for my PhD study (5 years), including tuition fees, living expenses, monthly stipends, etc.
[Jul 2022] Paper "UnrealEgo" got accepted to ECCV 2022 main conference.
[Oct 2021] Paper Self-Supervised Learning of Domain Invariant Features for Depth Estimation got accepted to WACV 2022.
[May 2021] Paper Dynamic Object Removal from Unpaired Images for Autonomous Agricultural Robots got accepted to IAS-16.

My research focus lies at the intersection of Computer Vision and Deep Learning. In particular, I am interested in the following topics:

• Egocentric 3D Vision
• Human Pose Estimation
• Human Motion Generation (Diffusion Model)
• Human Body Reconstruction with Self-Supervised Personalization from Images and Speech (LLM)
• Synthetic-to-Real Domain Adaptation

Egocentric 3D full-body tracking has been studied using cameras installed in front of a head-mounted device (HMD). While frontal placement is the optimal and the only option for some tasks, such as hand tracking, it remains unclear if the same holds for full-body tracking. Notably, even the state-of-the-art methods often fail to estimate accurate 3D poses in many scenarios, such as when HMD users tilt their heads upward---a common motion in human activities. Hence, this paper investigates the usefulness of rear cameras in the HMD design for full-body tracking. We show that simply adding rear views to the frontal inputs is not optimal for existing methods and propose a new method that refines 2D joint heatmap estimation, thereby significantly improving 3D pose tracking (>10% on MPJPE). Furthermore, we introduce two new large-scale datasets, Ego4View-Syn and Ego4View-RW, for a rear-view evaluation.

This paper is an extension of our previous work "EventEgo3D (CVPR 2024)" (See below). We tackle a new problem, i.e. 3D human motion capture from an egocentric monocular event camera with a fisheye lens. "EventEgo3D (CVPR 2024)" proposed the EE3D framework that is specifically tailored for learning with event streams in the LNES representation, enabling high 3D reconstruction accuracy. We upgrade this framework to a new EE3D++ framework for further performance improvement. We also introduce a new dataset, EE3D-W, in addition to EE3D-S and EE3D-R from "EventEgo3D (CVPR 2024)".

In this work, we propose a new transformer-based framework to improve egocentric stereo 3D human pose estimation, which leverages the scene information and temporal context of egocentric stereo videos. Furthermore, we introduce two new benchmark datasets, i.e., UnrealEgo2 and UnrealEgo-RW (RealWorld). Our extensive experiments show that the proposed approach significantly outperforms previous methods.

We tackle a new problem, i.e. 3D human motion capture from an egocentric monocular event camera with a fisheye lens. Event streams have high temporal resolution and could provide reliable cues for 3D human motion capture under high-speed human motions and rapidly changing illumination. We leverage these characteristics and propose the first approach for event-based 3D human pose estimation, EventEgo3D (EE3D). The proposed EE3D framework is specifically tailored for learning with event streams in the LNES representation, enabling high 3D reconstruction accuracy. We also provide two new datasets, EE3D-S and EE3D-R.

We present UnrealEgo, i.e. a new large-scale naturalistic dataset for egocentric 3D human pose estimation. UnrealEgo is based on an advanced concept of eyeglasses equipped with two fisheye cameras that can be used in unconstrained environments. UnrealEgo is the first dataset to provide in-the-wild stereo images with the largest variety of motions among existing egocentric datasets. We also propose a new benchmark method that achieves the state-of-the-art results on UnrealEgo.

We tackle the problem of unsupervised synthetic-to-real domain adaptation for single image depth estimation. An essential building block of single image depth estimation is an encoder-decoder task network that takes RGB images as input and produces depth maps as output. In this paper, we propose a novel training strategy to force the task network to learn domain invariant representations in a self-supervised manner.

We developed a GAN-based stem that remove dynamic objects in images. The system can be trained without using paired images with/without the dynamic objects.

[Sep 2022 - Present] Ph.D. student, Max Planck Institute for Informatics (Saarbrucken, Germany)
[July 2021 - Aug 2022] Visiting researcher, Max Planck Institute for Informatics (Saarbrucken, Germany)
[Apr 2018 - Aug 2022] Master student, Keio University (Kanagawa, Japan)
[Sep 2020 - June 2021] Internship, KAUST (Saudi Arabia, remote)
[July 2019 - Aug 2019] Internship, Tencent (Palo Alto, CA, USA)
[Aug 2018 - May 2019] Student, University of California, Berkeley (Berkeley, CA, USA)
[Apr 2014 - Mar 2018] Bachelor student, Keio University (Kanagawa, Japan)

• Nakajima Foundation (中島記念国際交流財団), 2022-2027
  Scholarship for my PhD study for 5 years, including tuition fees, living expenses, stipends, etc.
• CREST, Japan Science and Technology Agency, 2021-2022
  Financial support for my research outside of Japan for 1 year.
• TOBITATE, Ministry of Education, Culture, Sports, Science and Technology, Japan 2018-2019
  Scholarship for my study outside of Japan for 1 year.

Lab Visits:

• 2021-2022: Prof. Christian Theobalt @MPI-INF
• 2020-2021: Prof. Peter Wonka @KAUST

Conference Participation:
  CVPR 2024 (Seattle, USA), ECCV 2022 (Tel Aviv, Israel), WACV 2022 (Hawaii, USA), IAS (Singapore, online)

© Hiroyasu Akada 2022 / Design: jonbarron Icons: Freepik - Flaticon