What are Zero-Shot Learning and its applications in Computer Vision?

Instruction: Explain the principle of zero-shot learning and discuss its potential applications in computer vision tasks.

Context: This question assesses the candidate's understanding of zero-shot learning—a technique for recognizing objects without having seen examples during training—and its innovative applications.

Updated

Example Answer

Zero-shot classification predicts target classes without task-specific labelled training examples for those classes, often by using attributes or text descriptions. With a model such as CLIP, I can compare image and text representations for candidate labels. That does not mean the model has never encountered related images or words during pretraining, and it does not discover arbitrary unknown categories without a useful representation and label space. I would evaluate prompts, confusing classes and unfamiliar inputs explicitly.

Make it your own

Illustrative check: compare several label descriptions on a held-out dataset and include images outside the candidate categories. Define whether the system can abstain rather than always choosing a label.

Why this works

The training boundary is clearer than claiming the model has never seen a concept. Follow-up: why can a highly confident forced-choice output still be wrong?

Sources: CLIP research.

Related Questions