New trends in GPT models: Reflection, derived models, and autonomous AI

These last seven days have seen more monumental change in AI than probably in the last 5-10 years. It’s not the release of GPT-4 and how it’s being used – it’s the release of hundreds of papers and products being driven by GPT-4 that are now out-performing GPT-4. Here are a few of the more exciting things happening with links below.

One of the biggest trends are models doing self-improvement. Getting GPT-4 to do “reflection.” Models are being built by different groups that out-perform GPT-4 by getting them to reflect and without specific prompting on how to do it - to reflect and improve themselves. Also, another paper came out this past week showing a system doing self-improvement by pairing up a “Decider” agent with a “Researcher” agent. The one example they give is a model using this process and coming up with a healthcare plan for a patient and then improving it.

Another trend is a GPT-4 “controller” model being hooked up to use other language models, APIs, and to even other non-AI systems to meet goals. This is very loosely being described as giving AI the tools to create “meta-cognition” by people like noted researcher Andrej Karpathy.

In a similar vein in this post-GPT-4 world, AI models are being used to do reinforcement learning to train smaller models from larger models that are as good as the larger models but can be done quicker, cheaper and can even be run on local machines (Llama, Alpaca, and Vicuna as examples).

Finally, the ultimate result of this are people now creating “autonomous AI.” An AI that can access the internet and without intervention do general goal seeking. Yes, that’s happening now.

Then beyond all this, there’s the potential release of GPT-5 at some point and the resulting models built on top of that and I think also most likely an improvement of the modeling to not be primarily based on auto-regressive transformers.

As disorienting as this rate of change can be and not to totally ignore the widely discussed possible negative impacts, there are tremendous positive outcomes on bringing these ideas together, aligning them, and executing really well on them.

https://nanothoughts.substack.com/p/reflecting-on-reflexion
https://arxiv.org/pdf/2303.17071v1.pdf
https://github.com/nomic-ai/gpt4all
https://arxiv.org/pdf/2303.17491.pdf
https://arxiv.org/pdf/2303.17580.pdf
https://arxiv.org/pdf/2303.16434.pdf
https://arxiv.org/pdf/2210.11610.pdf
https://github.com/Significant-Gravitas/Auto-GPT