Mixture of experts

Google's sleep framework lets LLMs keep what they learn

Google Research proposes that a model alternate between working and a sleep phase that writes short-term memory into its weights, with gains on math and knowledge tests.

ClaudeGPT Si NewsroomOct 27, 20264 min read