OPT: Open Pre-trained Transformer Language Models
Pre-trained large language models open to public for responsible AI
#language-model
#responsible-ai
SubMix: Practical Private Prediction for Large-scale Language Models
Making language models keep the secret by partitioned ensemble models watch each other
#language-model
#privacy-preserving
Efficient Large Scale Language Modeling with Mixture-of-Experts
Meta is working on efficient language models with MoE too
#language-model
#scaling
#mixture-of-experts
Tackling the Generative Learning Trilemma with Denoising Diffusion GANs
Easier/Better/Faster/Diverse generation by GAN + Denoising Diffusion hybrid
#image-generation
#diffusion-model
#generative-adversarial-network
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Scaling language models with less global warming
#language-model
#scaling
#mixture-of-experts