[2310.06825] Mistral 7B

Improving the performance of an LLM without scaling up the model size is important for not only training efficiency but also for real-world deployment scenarios. The authors propose simple and intuitive tricks, including Sliding Window Attention, Roling Buffer Cache, and Pre-fill and Chunking for to outperfrom LLaMa2 13B on all benchmarks with a 7B model.

Mitral 7B outperforms LLaMa2 13B on all benchmarks tested

The proposed Mitral 7B can be instruction-tuned, and is readily available on huggingface, AWS, GCP, and Azure ML.

• Knowledge-Augmented Language Model Verification
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
• Video Language Planning
• PaLI-3 Vision Language Models: Smaller, Faster, Stronger
• Large Language Models Are Zero-Shot Time Series Forecasters