Mixture of Experts (MoE) is an AI architecture which seeks to reduce the cost and improve the performance of AI models by sharing the internal processing workload across a number of smaller sub models.
The concept first appeared in a 1991 paper written by Geoffrey Hinton of Toronto university, one of the pioneers of AI. Strictly speaking, these smaller MoE models are not experts, they are simply discreet neural networks which are given sub-tasks to do in order to complete the main task.
