Multimodal chatbots are coming - H+ Weekly - Issue #433

This week - the release of DALL·E 3 and Microsoft Copilot; Neuralink received approval for clinical trials; and more!

Sep 22, 2023

The next few months are going to be interesting as the race to unveil the next generation of multimodal chatbots is heating up.

Gemini, Google’s response to GPT-4, is rumoured to be very close to being released. About two months ago, The Wall Street Journal revealed that Demis Hassabis said at a company-wide meeting that Gemini is going to be released by the end of the year. Recently, it has been reported that some of Google’s partners have received access to an early version of Gemini.

We already have a rough idea of what Gemini will look like. In the interview with Time, Hassabis said that we “should think of Gemini as a series of models rather than a single model”. This statement hints at Gemini being not a single large model but a collection of smaller models working together (this architecture is known as a mixture of experts and it is almost certain that GPT-4 also uses a similar architecture). What Google is adding is a “combination of scale and innovation,” as Hassabis put it. We can also reasonably expect Gemini to be a multimodal model, being not only able to operate on text but also with images. At the Google I/O 2023 conference, Sundar Pichai hinted that Gemini will have memory and be capable of planning and reasoning, which Hassabis stated are in the “earliest exploratory stages” in the interview with Time.

The upcoming release of Gemini is putting some pressure on OpenAI. There are rumours that OpenAI is rushing to be the first to release a multimodal model to the public. When GPT-4 was released in March 2023, OpenAI presented the multimodal capabilities of the model but never released them to the public. However, some multimodality is coming soon to ChatGPT, thanks to the recently announced DALL·E 3 and its integration with ChatGPT, which will enable ChatGPT to produce images from a text prompt. But ChatGPT's ability to take in images as input and process them is still missing.

There was also a mention of something called GPT-V spotted in recent OpenAI job listings. The “V” in GPT-V might mean two things. It could be a Roman numeral, or as some speculate, it can stand for “vision”. Given DALL·E 3 being integrated into ChatGPT, the latter is more plausible.

Earlier this year, some people spotted OpenAI filling a trademark for GPT-5. Some interpreted that as GPT-5 is coming soon, but it can also be OpenAI taking steps in advance to protect their intellectual property.

In any case, multimodal chatbots are coming and we can expect them to arrive in the coming months.

Becoming a paid subscriber is the best way to support the newsletter.

Become a paid subscriber

If you enjoy and find value in my writing, please hit the like button and share your thoughts in the comments. Share the newsletter with someone who will enjoy it, too.

You can also buy me a coffee if you enjoy my work.

🦾 More than a human

Neuralink’s First-in-Human Clinical Trial is Open for Recruitment
Neuralink has received approval to start clinical trials in humans and they are looking for patients willing to take part in trials. The study aims to evaluate the safety of their implant N1 and surgical robot R1, and assess the initial functionality of their BCI for enabling people with paralysis to control external devices with thoughts.

Brain implant lets cancer patients try 20 different drugs at a time
Researchers have created a brain implant that injects tiny doses of up to 20 different drugs into brain tumours to compare how well they work. This new approach allows doctors to rapidly screen lots of treatments to find the drug most likely to help and give it to the patients quicker.

🧠 Artificial Intelligence

DALL·E 3
OpenAI released DALL·E 3, the new version of their text-to-image generator. The new model is capable of generating text in the images and better understands spatial relationships. DALL·E 3 includes new safety measures, such as declining requests to generate violent, adult, or hateful content as well as declining requests to generate images of a public figure by name, or those that ask for images in the style of a living artist. DALL·E 3 is also integrated with ChatGPT which will be assisting in creating prompts used by DALL·E 3 to generate images. DALL·E 3 is now in research preview and will be available to ChatGPT Plus and Enterprise users in October, via the API and in Labs later this fall.

Announcing Microsoft Copilot
Microsoft announced the release of Microsoft Copilot, an AI assistant built into Windows. Copilot will be available as a free upgrade for Windows 11 from September 26th and across Bing, Edge, and Microsoft 365 Copilot this fall.

DeepMind is using AI to pinpoint the causes of genetic disease
Google DeepMind released AlphaMissense, the new model able to predict which misspellings found in human DNA are likely to cause disease. DeepMind says it has provided free access to all its predictions for human genes, as well as all the details needed to fully replicate the work, including computer code. However, DeepMind has opted not to release the complete model due to concerns about potential biosecurity risks if it is applied to analyse genes in species other than humans.

US rejects AI copyright for famous state fair-winning Midjourney art
Last year, an AI-generated art piece titled "Theatre D'opera Spatial" won a Colorado State Fair art contest. Its creator filed a copyright for it but it was denied by the US Copyright Office Review Board, which argued that the image lacks the human authorship required for granting a copyright.

Nearly half of CEOs believe AI could replace their own jobs, says new poll—and 47% say that’s a good thing
49% of CEOs say AI could effectively replace 'most,' or even 'all,' of their own roles, and 47% say it might even be a good thing, according to a survey from the online education platform edX. Moreover, the C-suite executives surveyed by edX said 49% of the skills that exist in their current workforce won't be relevant by 2025, and 47% of their workers aren't prepared for the future of work. This is in sharp contrast to responses from non-executive workers who believe only 20% of AI could replace 'most' or 'all' of their jobs.

🤖 Robotics

Agility Robotics opening RoboFab to manufacture humanoid robots
Agility Robotics has announced the opening of RoboFab, a robot manufacturing facility in Salem, Oregon with the capability to produce more than 10,000 robots per year. Founded in 2016, Agility Robotics is best known for their ostrich-like robot Cassie and a humanoid robot Digit, the only humanoid bipedal robot that is currently commercially available and in production.

▶️ The Robots That Swim Like Fish (16:22)
This video documents our efforts in creating robots that look and swim like fish, from figuring out the mechanics of fish motion to replicating them in various robotic bodies. Compared to underwater robots using propellers, a fish-like robot can be twice as energy efficient and more manoeuvrable.

🧬 Biotechnology

Insiders Reveal Major Problems at Lab-Grown-Meat Startup Upside Foods
During the tours around their factory, Upside Foods, one of the leading companies in the lab-grown meat industry, shows rows of steel bioreactors in which the meat is grown without killing any animals. However, according to former and current employees who have blown the whistle, the company has faced technical setbacks while projecting an image of having successfully solved the key scientific challenge of scaling up the production of whole cuts of meat. These revelations raise questions about the extent of progress cultured meat companies have made after nearly $3 billion in investment over the past seven years. They also prompt questions about whether certain types of cultivated meat products can ever become commercially viable.

💡 Tangents

▶️ Intel: We're Replacing PCBs with Glass Core (8:03)
In pursuit of more powerful CPUs, Intel is proposing to replace the current substrates (that green thing on which chips are placed) with new Glass Core Substrates. According to Intel, Glass Core can support bigger chips and higher signal speeds. They can also be better tuned for the specific product and offer better thermal stability. Intel intends to release the first CPUs using Glass Core by 2030.

H+ Weekly sheds light on the bleeding edge of technology and how advancements in AI, robotics, and biotech can usher in abundance, expand humanity's horizons, and redefine what it means to be human.

A big thank you to my paid subscribers, to my Patrons: whmr, Florian, dux, Eric, Preppikoma and Andrew, and to everyone who supports my work on Ko-Fi. Thank you for the support!

You can follow H+ Weekly on Twitter and on LinkedIn.

Thank you for reading and see you next Friday!