Multimodal
멀티모달
The ability to understand and create not only text but also images, sound and video together.
Overview
Early LLMs only read text and only wrote text. A multimodal model can look at a photo and describe it, listen to a voice and reply, and read a whole document that contains tables and charts.
It is a trend of separate models for text, images and sound merging into one. This is also what made AI possible that talks with you out loud and looks at your screen with you.
An everyday example
Taking a photo of the inside of your fridge and asking "what can I make with this?" is an example of using multimodal AI.
Why it matters
Most real-world information is not text but images, sound and video. As AI has learned to handle them together, the places it can be used have grown a lot.
Also calledmultimodal AI