NEWS お知らせ

（特別講演）最先端マルチメディアAI技術の異分野融合研究への発展的導入

https://ai.med.hokudai.ac.jp/news/1763/

ゴム材料解析に関する研究成果がIEEE Accessに採録されました。本研究は，住友ゴム工業株式会社との共同研究による研究成果となります．

Rintaro Yanagi, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Material compound-property retrieval using electron microscope images for rubber material development,” IEEE Access, 2023. (Accepted)

Yusuke Akamatsu, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Zero-shot Neural Decoding with Semi-supervised Multi-view Embedding,” Sensors (Accepted for publication)

2023年7月25日（火）〜7月28日（金）の日程で開催された国内最大規模の画像認識系の国内会議、画像の認識・理解シンポジウムMIRU2023にて当研究室から6件の発表を行いました！

【ロングオーラル】

・ OS5A-L1：Yingrui Ye, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama (Hokkaido Univ.)，Zero-shot Visual Sentiment Prediction with Cross-domain Sentiments Using Knowledge Distillation

【ショートオーラル】

・ OS1B-S1：Guang Li, Ren Togo, Takahiro Ogawa, Miki Haseyama (Hokkaido Univ.)，Dataset Distillation via Self-Adaptive Parameter Matching

・ OS1B-S9：Zongyao Li, Ren Togo, Takahiro Ogawa, Miki Haseyama (Hokkaido Univ.)，Source-data-free Domain-adaptive Semantic Segmentation with Inter-domain and Intra-domain Style Transfer

・ OS3B-S1：Yuto Watanabe, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama (Hokkaido Univ.)，Text-guided Image Manipulation Tolerant to Real-world Image

【インタラクティブ】

・ IS2-100：Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama (Hokkaido Univ.)，Human Emotion Recognition While Viewing Images Based on Multi-view Variational Recurrent Neural Network

・ IS2-102：Kyohei Kamikawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama (Hokkaido Univ.)，Feature integration introducing back-projection based on ordering in labels for rating prediction

http://cvim.ipsj.or.jp/MIRU2023/

Yuya Moroto※, Rintaro Yanagi※, Naoki Ogawa, Kyohei Kamikawa, Keigo Sakurai, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Personalized Content Recommender System via Non-verbal Interaction Using Face Mesh and Facial Expression”

※ equal contribution

https://www.acmmm2023.org/

[1] Keigo Sakurai, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Bundle Detection Based on Graph Convolutional Network Considering Categorical Hierarchical Relationship”

[2] Tatsuki Seino, Naoki Saito, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama, “Skill Level Classification Using Motion Data via Spatial Temporal Graph Convolutional Network”

[3] Masaki Kashiwagi, Keisuke Maeda, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Deterioration Level Estimation for Infrastructures Considering Noisy Labels via DivideMix”

[4] Masaya Sato, Keisuke Maeda, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Caption Unification for Multiple Viewpoint Lifelogging Images and Its Verification”

[5] Haruka Matsuda, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Text-To-Image Diffusion Model Suppressing Catastrophic Forgetting via Elastic Weight Consolidation”

[6] Taro Togo, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Novel Feature Extraction for Classification of Auditory-Visual Stimuli From fNIRS Signals”

[7] Xiang Li, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Improving Visual Counterfactual Explanation Models for Image Classification via CLIP”

[8] Haoyang Wang, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “A Controllable Recoloring Method for Novel Views Using Segment Anything Model”

https://www.ieee-gcce.org/2023/

コンシューマエレクトロニクスに関する国際会議IEEE ICCE-TW 2023 (2023 IEEE International Conference on Consumer Electronics – Taiwan)に当研究室より７件の発表を行いました。

また、[5]の発表がBest Paper Award (Honorable Mention)を受賞しました！おめでとうございます！

[1] Masaki Yoshida, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Binaural Audio Generation with Data Augmentation from 360° Videos”

[2] Tsubasa Kunieda, Ren Togo, Noriko Nishioka, Yukie Shimizu, Shiro Watanabe, Kenji Hirata, Keisuke Maeda, Takahiro Ogawa, Kohsuke Kudo, Miki Haseyama, “Estimation of Amyloid-β Positivity Using QSM Images Considering Age Information”

[3] Huaying Zhang, Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Parameter-efficient Tuning of a Pre-trained Model via Prompt Learning in Cross-modal Retrieval”

[4] He Zhu, Ren Togo, Takahiro Ogawa, Miki Haseyama, “A Medical Domain Visual Question Generation Model via Large Language Model”

[5] Ryota Goka, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Shoot Event Prediction in Soccer Considering Expected Goals Based on Players’ Positions”

[6] Ryota Goka, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Prediction of Shoot Events by Considering Spatio-temporal Relations of Multimodal Features”

[7] Tatsuki Seino, Naoki Saito, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama, “Proficiency-level Estimation Using Heterogeneous Features via Label Dequantized CCA”

Koshi Watanabe, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Gaussian Process Decoder with Spectral Mixtures and Locally Estimated Manifold for Data Visualization,” Applied Sciences (Accepted for publication)

Naoki Saito, Keisuke Maeda, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama, “Visual emotion recognition through multimodal cyclic-label dequantized Gaussian process latent variable model,” Journal of Robotics and Mechatronics (Accepted for publication)

世界トップの画像処理系国際会議2023 IEEE International Conference on Image Processing (IEEE ICIP 2023、開催地：Malaysia Kuala Lumpur、採択率：45%)に当研究室から投稿した以下の5件の論文が採択されました。
①
Yuto Watanabe, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “TEXT-GUIDED FACIAL IMAGE MANIPULATION FOR WILD IMAGES VIA MANIPULATION DIRECTION-BASED LOSS”
②
Yuki Era, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “VIDEO-MUSIC RETRIEVAL WITH FINE-GRAINED CROSS-MODAL ALIGNMENT”
③
HE ZHU, Ren Togo, Takahiro Ogawa, Miki Haseyama, “INTERPRETABLE VISUAL QUESTION ANSWERING REFERRING TO OUTSIDE KNOWLEDGE”
④
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “MULTI-VIEW VARIATIONAL RECURRENT NEURAL NETWORK FOR HUMAN EMOTION RECOGNITION USING MULTI-MODAL BIOLOGICAL SIGNALS”
⑤
Kyohei Kamikawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “FEATURE INTEGRATION VIA BACK-PROJECTION ORDERING MULTI-MODAL GAUSSIAN PROCESS LATENT VARIABLE MODEL FOR RATING PREDICTION”

現地ギリシャ ロードス島で発表を行いました！

(1) Koshi Watanabe, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Learning graph Laplacian from intrinsic patterns via Gaussian process”

(2) Hiroki Okamura, Keisuke Maeda, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Improving dropout in graph convolutional networks for recommendation via contrastive loss”

(3) Ryo Shichida, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Estimation of visual contents from human brain signals via VQA based on brain-specific attention”

(4) Ryosuke Sawata, Takahiro Ogawa, Miki Haseyama, “Class-aware shared Gaussian process dynamic model”

(5) Jiahuan Zhang, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Defense against black-box adversarial attacks via heterogeneous fusion features”

(6) Masaki Yoshida, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Binauralization Robust to Camera Rotation Using 360°Videos”

(7) Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Recallable question answering-based re-ranking considering semantic region for cross-modal retrieval” (OJSP Review Track枠)

１．東孝明，小川 直輝，前田 圭介，小川 貴弘，長谷山 美紀，”データ横断型対照学習を用いた道路構造物における変状画像の劣化レベル分類”

２．李宗曜，前田 圭介，藤後 廉，小川 貴弘，長谷山 美紀，”Developing technologies for the practical application of deep learning-based distress segmentation in subway tunnel images”

「１」の論文は東日本高速道路株式会社との共同研究，「２」の論文は東京地下鉄株式会社との共同研究の成果になります．

—-

本シンポジウムにて発表した論文は，「AI・データサイエンス論文集」として，J-STAGEに掲載されています．

2023年7月25日（火）～28日（金）に浜松で開催されるMIRU2023に、以下の4件の口頭発表論文(ロングオーラル：1件、ショートオーラル：3件）が採択されました。

【ロングオーラル発表(約16%)】

(1) Zero-shot Visual Sentiment Prediction with Cross-domain Sentiments Using Knowledge Distillation

【ショートオーラル発表(約30%)】

(2) Dataset Distillation via Self-Adaptive Parameter Matching

(3) Source-data-free Domain-adaptive Semantic Segmentation with Inter-domain and Intra-domain Style Transfer

(4) Text-guided Image Manipulation Tolerant to Real-world Image

http://cvim.ipsj.or.jp/MIRU2023/

本研究では，属性情報の階層構造に注目した，新しい属性分類アプローチを実現しました．

Ziwen Lan, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Multi-label Classification in Anime Illustration Based on Hierarchical Relationships of Attributes,” Sensors, (Accepted for publication)

※Figure taken from our accepted paper

質問応答AIに関するElectronics採録されました．外部知識の参照により画像に関する質問応答の精度向上を実現しました．

He Zhu, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Multimodal Natural Language Explanation Generation for Visual

Question Answering based on Multiple Reference Information,” Electronics, 2023. (Accepted)

※Figure taken from our accepted paper

本研究では，視線情報から高度な意味理解を可能とし，新しい画像再検索アプローチを実現しました．

Yuhu Feng, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Gaze-dependent Image Re-ranking Technique for Enhancing Content-based Image Retrieval,” Applied Sciences (Accepted for publication)

※Figure taken from our accepted paper

本研究では、オーディオビジュアル事前学習に基づく新しい画面外音源分離法を実現しました。

Masaki Yoshida, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Off-screen sound separation based on audio-visual pre-training using binaural audio,” Sensors (Accepted for publication)

※Figure taken from our accepted paper

プレイヤーの時空間的な関係と予測の不確実性を効果的に利用し、より高い精度と頑健性をもってシュートイベントの発生を予測する新しい手法を実現しています。
本研究では、北海道コンサドーレ札幌から提供を受けた映像を利用しています。
Ryota Goka, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Prediction of shooting events in soccer videos using complete bipartite graphs and players’ spatial-temporal relations,” Sensors (Accepted for publication)
※Figure taken from our accepted paper

2023年7月17～19日に台湾にて開催予定のコンシューマエレクトロニクスに関する国際会議に当研究室から投稿していた以下の7件の論文が採択されました。

[1] Masaki Yoshida, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Binaural Audio Generation with Data Augmentation from 360° Videos”

[2] Tsubasa Kunieda, Ren Togo, Noriko Nishioka, Yukie Shimizu, Shiro Watanabe, Kenji Hirata, Keisuke Maeda, Takahiro Ogawa, Kohsuke Kudo, Miki Haseyama, “Estimation of Amyloid-β Positivity Using QSM Images Considering Age Information”

[3] Huaying Zhang, Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Parameter-efficient Tuning of a Pre-trained Model via Prompt Learning in Cross-modal Retrieval”

[4] He Zhu, Ren Togo, Takahiro Ogawa, Miki Haseyama, “A Medical Domain Visual Question Generation Model via Large Language Model”

[5] Ryota Goka, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Shoot Event Prediction in Soccer Considering Expected Goals Based on Players’ Positions”

[6] Ryota Goka, Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama, “Prediction of Shoot Events by Considering Spatio-temporal Relations of Multimodal Features”

[7] Tatsuki Seino, Naoki Saito, Takahiro Ogawa, Satoshi Asamizu, Miki Haseyama, “Proficiency-level Estimation Using Heterogeneous Features via Label Dequantized CCA”

http://www.icce-tw.org/

ICLR2023 に採択された研究成果のコードを公開しました！

5/1-5/5に開催予定のAI・機械学習分野のトップカンファレンスInternational Conference on Learning Representation (ICLR)に採択された表現学習に関する研究成果のコードを公開しました．以下のリンクからアクセス可能です．

https://github.com/ganmodokix/gwae

Nao Nakagawa, Ren Togo, Takahiro Ogawa, Miki Haseyama, “Gromov-Wasserstein Autoencoders,” ICLR, 2023. (Accepted)

https://openreview.net/forum?id=sbS10BCtc7

本研究成果は，中国のAI関連大手メディアにも紹介されています．

https://mp.weixin.qq.com/s/Gj8en7StSXOQ5HBQr-5Yhw

※ICLRは，google scholar metricにおいて全学術分野の論文・会議において9位，人工知能分野において1位にランクされる世界最高峰の国際会議です．