🔥 Search Hot Tweets

Very late to the party. When Thinking Machines Lab released their blog on On Policy Distillation, my first reaction was that it should be just like DAGGER from 15 years ago: https://t.co/qh1NdMObBm. I finally had time to read the blog today and sure enough, they mentioned DAGGER. Actually, if you have access to an expert, there are quite a few on-policy imitation learning approaches you can try. Here’s one incremental idea that really works from an obscure paper we published at 2018. It is basically an on policy multi-teacher imitation learning method that chooses the best teacher for each state based on their value functions. We found that it’s possible to imitate multiple imperfect teachers simultaneously, and eventually outperform all of them. Naturally, the more diverse those teachers are, the better. Would be interesting to reproduce this in the LLM era: https://t.co/xctBqEbPXb