Використання CNN для багатокласової класифікації та класифікації з багатьма мітками зображень їжі

Abstract

The article analyzes the problem of the "information bottleneck" in hybrid modular generative architectures for the image-to-recipe generation task. The analysis focuses on the theoretical flaws of a "naïve" modular approach, which, due to the high intra-class variability of the food domain, fails to provide the generative component with sufficient details, leading to the generation of template-based, non-relevant recipes. To solve this problem, an advanced architecture is proposed that implements the "Explicit Semantic Supervision" method. This approach is based on combining multiclass classification (CNN-1, identifying the dish class) and multilabel classification (CNN-2, identifying ingredients), implemented via parallel "heads" with different loss functions (CCE and BCE). The analysis confirms that this dual-component approach, which acts as an ensemble method, provides the generative component with visually-grounded details (class + ingredients) and allows for the generation of relevant descriptions, thus overcoming the bottleneck.