AI Weekly Digest #17: Top News Shaping AI in 2023 - Part 1 LLMs!

Hello, tech enthusiasts! This is Wassim Jouini and Welcome to my AI newsletter, where I bring you the latest advancements in Artificial Intelligence without the unnecessary hype. You can also find me on medium covering AI related topics.

LLMs: Revolutionizing Conversations and Shaping the Future of AI!

Context: LLMs (Large Language Models) have become essential in various AI developments. Serving as catalysts for content generation, NLP tasks, AI agents, smart robots, video games, and even prompt-based image generation. Thus emerging as true enablers, revolutionizing the field of artificial intelligence.

Evolution or revolution? While language models have existed for some time, it was the remarkable breakthrough of GPT2 in February 2019 that set the stage for a transformative revolution. Despite being relatively small with 1.5B parameters, GPT2 demonstrated the ability to generate coherent text over longer contexts, surpassing the state of the art at that time. Microsoft's early investment of $1B in 2019 showcased their belief in the potential of these models.

In April 2022, Google's PaLM emerged as the best-performing model, surpassing even the remarkable GPT3 (175B parameters) in most, if not all, benchmarks, thanks to its staggering 540B parameters. However, PaLM remained behind closed doors, offering only glimpses of its immense capabilities and limiting its availability to the wider community. Meanwhile, OpenAI took a different approach. By opening its models early, sharing them as open source or via APIs, it allowed the community to contribute, by providing the model with feedback and data ; helping fine-tuning these models, and aligning them with humans’ expectations! This strategic decision made all the difference, as the community had already started building on top of OpenAI's APIs since early 2022!

Tipping point: OpenAI's paid access to their APIs, provided a glimpse into the capabilities of its models (i.e., Davinci-003 model), but it was still cumbersome to use, mostly designed for developers and … not free. The landscape changed dramatically with the official release of ChatGPT on November 30th, 2022. Unlike the previous API-centric approach, ChatGPT broke free from those constraints, offering a seamless and free to use chat interface!

And the rest is History! A million users in 5 days, then a $10B investment from Microsoft, and GPT-4’s integration in Bing! All in a matter of weeks!.

A new Productivity tool: What’s so amazing about it? Well … LLMs can compete with the average human on a wide range of tasks! And studies have consistently shown that humans leveraging these AI tools outperform both humans working alone and AI systems operating independently, making AI models a great asset to supercharging humans productivity!

AI won’t replace humans ; humans might be replaced by AI supercharged humans though…

News Shaping AI in 2023 - Part 1 LLMs!

What to remember about the past 6 months?

#1 ChatGPT’s Release and Adoption

ChatGPT has made a remarkable entry by reaching 100 million users within just two months. However, it's essential to note that we are still in the nascent stage of this AI revolution and widespread adoption will take time. As a matter of fact daily usage remains fairly low, and business integration of these tools is almost inexistant!

Furthermore, when it comes to AI adoption, age, education and wealth matter with mostly younger, better-educated, and wealthier demographics being the early adopters.

Microsoft on the other hand, swiftly launched its Azure OpenAI service helping companies develop, and scale, fast their LLM based systems. I was recently quoted by Microsoft, along side the startup Expensya, as one of the early architects on its platform.

#2 Open Source Models Making Strides: The Alpaca Revolution

When asked about GPT-4, Sam Altman, OpenAI’s CEO stated that it took more than $100M dollars to develop the model. But with a cheaper compute, quality datasets and new optimization tricks, the open source community came with surprisingly good models in a few months!

In a nutshell, here’s what you need to remember:

• Fully Open Source models are getting smaller and better: Falcon 40B sits today at the top of the “Open LLM leaderboard”

• Commerciale Open Source models are now available with the needed training and inference infrastructure. The release of MPT-30B by MosaicML opens the way to custom GPT-3 like models but 15 times cheaper (read all about it here)

• A framework to build on top of these models thanks to Stanford’s Alpaca experiment, showing that with a few hundred dollars, anyone can build a ChatGPT like model on top of an opensource large language models (aka Falcon 40B or MPT-30B for instance). Read this short twitter thread about how this new experiment unleashed Open Source models.

#3 GPU shortage slowing down AI development

Ever wonder why you have a limited access to GPT4 or why its multi-modal version is still closed? Well the answer is “because there isn’t enough compute for everyone”.

As a matter of fact, scaling AI models (both for training and consumer access) requires a huge amount of GPUs…. leading to a GPU shortage. As a consequence:

• Nvidia’s stocks have soared, turning it into a trillion-dollar company, and consolidating its position as a market leader.

• Microsoft is taking matters into its own hands by developing its own chips.

• AMD is also gearing up for competition, announcing the launch of the most powerful GPU, set to be released in Q3.

#4 Google still shy, but building its AI muscles

Google is discreetly making substantial advancements in AI technology and expanding its global presence.

• Bard, Google's ChatGPT, is now publicly available (yet still limited in Europe).

• Google has introduced PaLM2, a public preview of its API aimed at rivaling OpenAI’s APIs.

• Vertex AI, Google’s AI platform, has launched Imagen, an image generation tool that can compete with Midjourney and Stable Diffusion.

• Google has begun releasing AI capabilities into Google Workspace products such as Gmail and Google Sheets.

   

#5 Safety and Regulations

Last but not least: Safety and Regulation!

Will AI be the end of us? The debate surrounding AI safety is intensifying, and while some fear its implications, others view it as a non-issue, a natural process that will be dealt with incrementally.

One thing is for sure: AI companies won't slow down their efforts to dominate the AI space! There is too much at stake and the competition is raging! Let’s remember that a few months ago Microsoft invested $10B in OpenAI!

Meanwhile, regulations are already taking shape: As of mid-June 2023, the EU AI Act has come into effect. These new regulations will have a similar impact to GDPR, and anyone venturing into the European market will need to comply with them.

Conclusion and Part-2!

In this exciting chapter, we're witnessing the beginnings of a raging competition and a steady march towards smarter, safer, and more user-friendly AI systems.

• ChatGPT is already turning heads,

• the open-source community is stepping up their game!

• At the same time, Nvidia and AMD are competing towards faster and more efficient AI compute.

Google, although reserved in recent months, maintains a steady pace in AI developments. It's prudent to anticipate considered moves from Google as it continues to build its capabilities.

Furthermore, it’s important to recognize that this is the commencement of a new phase. Adoption will be slow and governed by AI regulations!

In Part 2, we will meticulously examine the ramifications of these developments, especially on AI agents, robotics, and multi-modal models: this will dominate the next phase of AI developments!

Subscribe and stay tuned!

That’s it for today! If you made it this far, I’d appreciate a quick feedback 😋! I know there’s room for improvement! So don’t hesitate to share with me the things you liked and those that you didn’t.

Have a great Sunday and may AI always be on your side!