Tree of Thoughts - A Review
This is my take on the paper Tree of Thoughts: Deliberate Problem Solving with Large Language Models by researchers from DeepMind and Princeton University (Yao et al., 2023). It’s based on my understanding, and corrections and discussions would be welcome! In my analysis, the ‘Tree of Thoughts’ framework effectively identifies the problem with simple left-to-right decoding in LMs in tasks like lexical reasoning, and combines the generation and evaluation capabilities of LMs with search over a ‘thought tree’. In a way, the CoT framework can be seen as ‘augmenting’ the input with the intermediate steps, and ToT further ‘navigates’ the best of these augmentations. The paper is comprehensive in the sense that it properly sets-up the IO prompting and CoT baselines for the proposed tasks and studies ablations, which gives an idea of performance of the individual components like pruning, evaluator etc. ...