Scale and Time Cost RT-1: 17 months, 13 robots, 130k Episodes over 700 tasks PaLM model has 780B tokens. Assuming 100 robots that produce one action token at 30 Hz for each robot, then the 100 ro...

Good Enough Prior Goal: achieve baby-level of control of the body and is capable of doing fundamental instruction following. This could significantly narrow down the search space for the follo...

Imitation Learning BC-Z Gato Did not evaluate generalization to new tasks. Limited real-world data and evaluation. Robocat RT-1 It has its own action tokenization. It discretized eac...

Requirements Functional Requirements What do we want to achieve? Non-Functional Requirements Scale, qps Read heav...

What does Pytorch register_buffer do? register_buffer saves the parameters in state_dict, but no gradients will be computed (hence no updates). model.parameters() will not return the ...

It has been ~4 years since I read through Sutton’s Reinforcement Learning textbook. It is time to review it especially in this post-ChatGPT eura, and see there are any “温故而知新” moments. $k$-arm Ban...

Intro Here is a blog to summarize all the deep learning training tricks I encountered these days. AdamW Optimizer “W” means weight decay, which is the same concept of weight decay in L2 regulari...