Recently Google released a large-language model for high-quality video generation called “Video Poet”, captioned it under “LLM model for Zero-Shot video generation”.

“Zero-Shot” - That’s a catchy part and made me to explore further.

Zero-shot learning (ZSL) in ML is a model’s ability to detect classes never seen during training. i.e recognise objects from classes they have never earlier.

It is very helpful in autonomous object discovery system where it can identify and categorize new objects on their own. This is widely studied in computer vision, natural language processing, and machine perception.

Earlier days researchers at Meta AI trained this model, to identify more than 600 classes of birds across two databases containing more than 60,000 images.

It was then given web articles and asked to use the information there to identify birds it had not seen before.

The model extracted seven key visual features from the text, created synthetic visualisations of these features, and used those features to identify the correct class of bird.