[2310.09199] PaLI-3 Vision Language Models: Smaller, Faster, Stronger

The authors improve PaLI-based vision language model (VLM) by replacing the vision encoder from classification pre-trained ViT to a contrastive-pretrained ViT, the SigLIP. By introducing the SigLIP as the vision encoder, the authors show that the proposed PaLI-3 performs comparably on vision-language tasks to recent models with 10$\times$ larger parameters.

Diagram of the PaLI-3 VLM

• A Picture is Worth a Thousand Words: Principled Recaptioning Improves Image Generation
• Knowledge-Augmented Language Model Verification
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
• Video Language Planning
• Large Language Models Are Zero-Shot Time Series Forecasters