Lin Chen (陈林)

Greetings! I'm currently a PhD student in School of Automation, University of Science and Technology of China, advised by Prof. Feng Zhao. I got a B.E. degree at Anhui University in 2020 and join the USTC-BIVLab. And I serve as a research intern in Shanghai AI Laboratory now, supervised by Dr. Xiaoyi Dong, Dr. Pan Zhang and Dr. Jiaqi Wang.

My research interest includes:

• transfer learning
• parameter-efficient fine-tuning
• large vision-language models

Email  /  Google Scholar  /  Github  /  HuggingFace  /  Twitter

[2024.9] Thrilled that ShareGPT4V ranks in Top 5 of the Most Influential ECCV 2024 Papers.

[2024.9] Delighted to announce that MMStar, Prism and ShareGPT4Video were accepted by NeurIPS 2024!

[2024.8] Thrilled that I've won third place on the July 2024 Most Impactful Users leaderboard on HuggingFace.

[2024.7] Happy to announce that ShareGPT4V was accepted by ECCV 2024!

[2024.6] 🔥 We release ShareGPT4Video, comprising 40K GPT4V-generated captions, 4.8M high-quality captions, a general video captioner, and a superior large multi-modal model, ShareGPT4Video-8B

[2024.5] We release Open-LLaVA-Next, an open-source implementation of LLaVA-NeXT series for facilitating the large multi-modal model community. All training data and checkpoints at each stage are open-sourced, and friendly for research usage.

[2024.4] We release MMStar, an elite vision-indispensable multi-modal benchmark.

[2024.3] Two papers Rein and FreeDrag were accepted in CVPR 2024!

[2023.11] 🔥 We release the ShareGPT4V project, comprising 100K GPT4-Vision-generated captions, 1.2M high-quality captions, a general image captioner, and a superior large multi-modal model, ShareGPT4V-7B

[2023.7] DTP is accepted in ICCV 2023 and achieves SOTA in night-time and full-time semantic segmentation!

[2023.7] We release the FreeDrag framework for more superior and stable "drag" editing!

[2022.10] Our DDB receives the Spotlight Award in NeurIPS 2022!

[2022.9] DDB is accepted in NeurIPS 2022 and achieves SOTA with ResNet counterparts on the single-source, multi-source, and multi-target domain-adaptive semantic segmentation tasks!

[2022.3] A discriminator-free adversarial domain adaptation framework DALN is accepted in CVPR 2022!

[2022-07 ~ Now] Research Intern, Open Algorithm group of Shanghai AI Laboratory.

[2022-03 ~ 2022-06] Computer Vision Intern, MMSegmentation team in OpenMMLab group of Shanghai AI Laboratory.

* indicates the equal contribution.

We identify two primary issues in existing evaluation studies for large vision-language models. We further develop an elite vision-indispensable multi-modal benchmark and two novel metrics to measure data leakage and actual performance gain in multi-modal training.

A versatile large-vision language model that supports long-contextual input and output. IXC-2.5 excels in various text-image comprehension and composition applications, achieving GPT-4V level capabilities with merely 7B LLM backend.

We propose the ShareGPT4V project, comprising 100K GPT4-Vision-generated captions, 1.2M high-quality captions, a general image captioner, and a superior large multi-modal model, ShareGPT4V-7B

We present Prism, an innovative framework designed to disentangle the perception and reasoning processes involved in visual question solving.

A large-scale highly descriptive video-text dataset, with 40K captions annotated by GPT4V and 4.8M captions annotated by our ShareCaptioner-Video. The total videos last with 300 hours and 3000 hours separately!

We propose the Rein framework, which efficiently fine-tunes vision foundation models for the domain generalized semantic segmentation (DGSS) task with just 1% trainable parameters, surprisingly surpassing full parameter fine-tuning. And Reins builds a new SOTA in various DGSS benchmarks.

We propose a novel "drag" editing framework called FreeDrag free of the burden of erroneous point tracking and enables achieving stable point-based editing in challenging scenarios with similar structures, fine details, or under multi-point targets.

We propose a novel nigh-time semantic segmentation paradigm, i.e., disentangle then parse (DTP), which explicitly disentangles night-time images into light-invariant reflectance and light-specific illumination components and then recognizes semantics based on their adaptive fusion.

We leverage the complementary characteristics of the coarse-wise and fine-wise data mixing techniques to progressively transfer the knowledge from the source to the target domain.

We reuse the category classifier as a discriminator to form a discriminator-free adversarial learning framework.

• Open-LLaVA-NeXT: an open-source implementation of LLaVA-NeXT
• VLMEvalKit: open-source evaluation toolkit of large vision-language models
• InternLM-XComposer: a series of large multi-modal models in the InternLM group
• MMSegmentation: an open source semantic segmentation toolbox based on PyTorch

• 800+ Citations on Google Scholar

• The Third Place on the July 2024 Most Impactful Users of HuggingFace

• 14M+ Model Downloads on HuggingFace

• National Scholarship Award, PRC, 2022.