flava-full

モデル紹介

FLAVA model was developed by the researchers at FAIR to understand if a single model can work across different modalities with a unified architecture. The model was pretrained solely using publicly available multimodal datasets containing 70M image-text pairs in total and thus fully reproducible. Unimodal datasets ImageNet and BookCorpus + CCNews were also used to provide unimodal data to the model. The model (i) similar to CLIP can be used for arbitrary image classification tasks in a zero-shot manner (ii) used for image or text retrieval in a zero-shot manner (iii) can also be fine-tuned for natural language understanding (NLU) tasks such as GLUE and vision-and-language reasoning tasks such as VQA v2. The model is able to use the data available as images, text corpus and image-text pairs. In the original paper, the authors evaluate FLAVA on 32 tasks from computer vision, NLU and vision-and-language domains and show impressive performance across the board scoring higher micro-average than CLIP while being open. The FLAVA model uses a ViT-B/32 transformer for both image encoder and text encoder. FLAVA also employs a multimodal encoder on top for multimodal tasks such as vision-and-language tasks (VQA) which is a 6-layer encoder. Each component of FLAVA model can be loaded individually from `facebook/flava-full` checkpoint. If you need complete heads used for pretraining, please use `FlavaForPreTraining` model class otherwise `FlavaModel` should suffice for most use case. This repository also contains code to pretrain the FLAVA model from scratch. FLAVA model supports vision, language and multimodal inputs. You can pass inputs corresponding to the domain you are concerned with to get losses and outputs related to that domain.

主な特徴

特点摘要待依据来源补齐，不凭模型名称推断能力。

制約と注意点

具体的な制約を判断する出典が不足しています。実際の性能、遅延、データ方針、ライセンス、提供期間を確認してください。

出典別料金と費用分析

料金はチャネル・地域・設定によって異なります。各チャネルの料金は個別に表示し、提供元の統一公式料金とは見なしません。

公開評価と実行設定

暂无已采集的可核验公开评分。模型未上榜不代表能力较低。

よくある質問

出典と追跡記録

许可证：bsd-3-clause。模型介绍与第三方评测均以原来源的具体版本和方法为准。