The immense potential and challenges of multimodal AI

Recommendable! One of the latest trends in AI: Combining text, video, audio, and images etc.

"... While systems capable of making these multimodal inferences remain beyond reach, there’s been progress. New research over the past year has advanced the state-of-the-art in multimodal learning, particularly in the subfield of visual question answering (VQA), a computer vision task where a system is given a text-based question about an image and must infer the answer. As it turns out, multimodal learning can carry complementary information or trends, which often only become evident when they’re all included in the learning process. ...
In multimodal systems, computer vision and natural language processing models are trained together on datasets to learn a combined embedding space, or a space occupied by variables representing specific features of the images, text, and other media. ..."

The immense potential and challenges of multimodal AI | VentureBeat