OPT: Open Pre-trained Transformer Language Models

Pre-trained large language models open to public for responsible AI

#language-model #responsible-ai

SubMix: Practical Private Prediction for Large-scale Language Models

Making language models keep the secret by partitioned ensemble models watch each other

#language-model #privacy-preserving

Efficient Large Scale Language Modeling with Mixture-of-Experts

Meta is working on efficient language models with MoE too

#language-model #scaling #mixture-of-experts

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs

Easier/Better/Faster/Diverse generation by GAN + Denoising Diffusion hybrid

#image-generation #diffusion-model #generative-adversarial-network

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

Scaling language models with less global warming

#language-model #scaling #mixture-of-experts

#language-model #responsible-ai #privacy-preserving #scaling #mixture-of-experts #image-generation #diffusion-model #generative-adversarial-network #speech-model #multi-modal #contrastive-learning #self-supervised #image-representation #image-processing #object-detection #pseudo-labeling #scene-text-detection #neural-architecture-search #notice #data-sampling #long-tail #graph-neural-network #graph-representation #zero-shot #metric-learning #federated-learning #weight-matrix #low-rank #vision-transformer #computer-vision #normalizing-flow #invertible-neural-network #super-resolution #image-manipulation #thread-summarization #natural-language-processing #domain-adaptation #knowledge-distillation #scene-text #model-compression #semantic-segmentation #instance-segmentation #video-understanding #code-generation #graph-generation #image-translation #data-augmentation #model-pruning #signal-processing #text-generation #text-classification #music-representation #transfer-learning #link-prediction #counterfactual-learning #medical-imaging #acceleration #transformer #style-transfer #novel-view-synthesis #point-cloud #spiking-neural-network #optimization #multi-layer-perceptron #adversarial-training #visual-search #image-retrieval #negative-sampling #action-localization #weakly-supervised #data-compression #hypergraph #adversarial-attack #submodularity #active-learning #deblurring #object-tracking #pyramid-structure #loss-function #gradient-descent #generalization #bug-fix #orthogonality #explainability #saliency-mapping #information-theory #question-answering #knowledge-graph #robustness #limited-data #recommender-system #anomaly-detection #gaussian-discriminant-analysis #molecular-graph #video-processing