Lin Chen (陈林)

Greetings! I'm currently a PhD student in School of Automation, University of Science and Technology of China, advised by Prof. Feng Zhao. I got a B.E. degree at Anhui University in 2020 and join the USTC-BIVLab. Now I'm serving as a LVLM research intern in the Seed team of ByteDance.

My research interest includes:

• transfer learning
• large vision-language models

Email  /  Google Scholar  /  Github  /  HuggingFace  / 

[2025.3] Excited that MMStar ranks in Top 10 of the Most Influential NeurIPS 2024 Papers.

[2024.9] Thrilled that ShareGPT4V ranks in Top 5 of the Most Influential ECCV 2024 Papers.

[2024.9] Delighted to announce that MMStar, Prism and ShareGPT4Video were accepted by NeurIPS 2024!

[2024.8] Thrilled that I've won third place on the July 2024 Most Impactful Users leaderboard on HuggingFace.

[2024.7] Happy to announce that ShareGPT4V was accepted by ECCV 2024!

[2024.6] 🔥 We release ShareGPT4Video, comprising 40K GPT4V-generated captions, 4.8M high-quality captions, a general video captioner, and a superior large multi-modal model, ShareGPT4Video-8B

[2024.5] We release Open-LLaVA-Next, an open-source implementation of LLaVA-NeXT series for facilitating the large multi-modal model community. All training data and checkpoints at each stage are open-sourced, and friendly for research usage.

[2024.4] We release MMStar, an elite vision-indispensable multi-modal benchmark.

[2024.3] Two papers Rein and FreeDrag were accepted in CVPR 2024!

[2023.11] 🔥 We release the ShareGPT4V project, comprising 100K GPT4-Vision-generated captions, 1.2M high-quality captions, a general image captioner, and a superior large multi-modal model, ShareGPT4V-7B

[2025-01 ~ Now] LVLM Research Intern, Seed team of ByteDance.

[2022-07 ~ 2024-12] Research Intern, Open Algorithm group of Shanghai AI Laboratory.

[2022-03 ~ 2022-06] Computer Vision Intern, MMSegmentation team in OpenMMLab group of Shanghai AI Laboratory.

* indicates the equal contribution.

Seed1.5-VL, a powerful and efficient vision-language foundation model designed for advanced general-purpose multimodal understanding and reasoning, achieves state-of-the-art performance on 38 out of 60 public benchmarks.

We identify two primary issues in existing evaluation studies for large vision-language models. We further develop an elite vision-indispensable multi-modal benchmark and two novel metrics to measure data leakage and actual performance gain in multi-modal training.

A versatile large-vision language model that supports long-contextual input and output. IXC-2.5 excels in various text-image comprehension and composition applications, achieving GPT-4V level capabilities with merely 7B LLM backend.

We propose the ShareGPT4V project, comprising 100K GPT4-Vision-generated captions, 1.2M high-quality captions, a general image captioner, and a superior large multi-modal model, ShareGPT4V-7B

A large-scale highly descriptive video-text dataset, with 40K captions annotated by GPT4V and 4.8M captions annotated by our ShareCaptioner-Video. The total videos last with 300 hours and 3000 hours separately!

We propose the Rein framework, which efficiently fine-tunes vision foundation models for the domain generalized semantic segmentation (DGSS) task with just 1% trainable parameters, surprisingly surpassing full parameter fine-tuning. And Reins builds a new SOTA in various DGSS benchmarks.

We leverage the complementary characteristics of the coarse-wise and fine-wise data mixing techniques to progressively transfer the knowledge from the source to the target domain.

We reuse the category classifier as a discriminator to form a discriminator-free adversarial learning framework.

• Open-LLaVA-NeXT: an open-source implementation of LLaVA-NeXT
• VLMEvalKit: open-source evaluation toolkit of large vision-language models
• InternLM-XComposer: a series of large multi-modal models in the InternLM group
• MMSegmentation: an open source semantic segmentation toolbox based on PyTorch

• 3K+ Citations on Google Scholar

• The Third Place on the July 2024 Most Impactful Users of HuggingFace

• 14M+ Model Downloads on HuggingFace

• National Scholarship Award, PRC, 2022.