Jeff Dean on mixture-of-experts models: activate only the parts of a very large model that are useful, as a brain does, for about 10X better training compute to quality.
Full talk: https://www.youtube.com/watch?v=0kC3xOZChdA
@aasforum.bsky.social
@aasforum.bsky.socialJeff Dean on mixture-of-experts models: activate only the parts of a very large model that are useful, as a brain does, for about 10X better training compute to quality.
Full talk: https://www.youtube.com/watch?v=0kC3xOZChdA
@aasforum.bsky.socialJeff Dean: build tools that run an experiment "in a minute or an hour instead of a day or a week," then run thousands in parallel and let the results pick the next ones.
Full talk: https://www.youtube.com/watch?v=0kC3xOZChdA
@aasforum.bsky.socialJeff Dean on his new public benefit corporation: distribute its scientific discoveries as widely as possible, even when that is not in the company's financial interest.
Full talk: https://www.youtube.com/watch?v=0kC3xOZChdA