THE AI MODEL DIRECTORY 모델 탐색을 시작하세요

나에게 맞는 AI 모델을 찾아보세요.

언어부터 이미지, 오픈 웨이트부터 API까지. 기능과 실제 비용을 비교하세요.
다음 아이디어에 맞는 AI를 찾아보세요.

AI 모델 탐색 3

제공사 입력 출력 라이선스

주목할 모델을 찾아보세요

bridgetower-base

BridgeTower

The BridgeTower model was proposed in "BridgeTower: Building Bridges Between Encoders in Vision-Language Representative Learning" by Xiao Xu, Chenfei Wu, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan. The model was pretrained on English language using masked language modeling (MLM) and image text matching (ITM)objectives. It was introduced in this paper and first released in this repository. The abstract from the paper is the following: Vision-Language (VL) models with the Two-Tower architecture have dominated visual-language representation learning in recent years. Current VL models either use lightweight uni-modal encoders and learn to extract, align and fuse both modalities simultaneously in a deep cross-modal encoder, or feed the last-layer uni-modal representations from the deep pre-trained uni-modal encoders into the top cross-modal encoder. Both approaches potentially restrict vision-language representation learning and limit model performance. In this paper, we propose BridgeTower, which introduces multiple bridge layers that build a connection between the top layers of uni-modal encoders and each layer of the cross-modal encoder. This enables effective bottom-up cross-modal alignment and fusion between visual and textual representations of different semantic levels of pre-trained uni-modal encoders in the cross-modal encoder. Pre-trained with only 4M images, BridgeTower achieves state-of-the-art performance on various downstream vision-language tasks. In particular, on the VQAv2 test-std set, BridgeTower achieves an accuracy of 78.73%, outperforming the previous state-of-the-art model METER by 1.09% with the same pre-training data and almost negligible additional parameters and computational costs. Notably, when further scaling the model, BridgeTower achieves an accuracy of 81.15%, surpassing models that are pre-trained on orders-of-magnitude larger datasets. You can use the raw model for masked language modeling, but it's mostly intended to be fine-tuned on a downstream task. See the model hub to look for fine-tuned versions on a task that interests you.

번역 대기 중 · 원문 표시

분류 준비 중

입력 미확인→ 출력 미확인

요금 미확인멀티모달 및 채널 요금 별도 확인

자세히 보기 ↗

비교 추가출처 / 문서 ↗

bridgetower-base-itm-mlm

BridgeTower

基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。

기본 정보 등록 · 상세 정보 보완 예정

분류 준비 중

입력 미확인→ 출력 미확인

요금 미확인멀티모달 및 채널 요금 별도 확인

자세히 보기 ↗

비교 추가출처 / 문서 ↗

bridgetower-large-itm-mlm-itc

BridgeTower

The BridgeTower model was proposed in "BridgeTower: Building Bridges Between Encoders in Vision-Language Representative Learning" by Xiao Xu, Chenfei Wu, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan. The model was pretrained on English language using masked language modeling (MLM) and image text matching (ITM)objectives. It was introduced in this paper and first released in this repository. The abstract from the paper is the following: Vision-Language (VL) models with the Two-Tower architecture have dominated visual-language representation learning in recent years. Current VL models either use lightweight uni-modal encoders and learn to extract, align and fuse both modalities simultaneously in a deep cross-modal encoder, or feed the last-layer uni-modal representations from the deep pre-trained uni-modal encoders into the top cross-modal encoder. Both approaches potentially restrict vision-language representation learning and limit model performance. In this paper, we propose BridgeTower, which introduces multiple bridge layers that build a connection between the top layers of uni-modal encoders and each layer of the cross-modal encoder. This enables effective bottom-up cross-modal alignment and fusion between visual and textual representations of different semantic levels of pre-trained uni-modal encoders in the cross-modal encoder. Pre-trained with only 4M images, BridgeTower achieves state-of-the-art performance on various downstream vision-language tasks. In particular, on the VQAv2 test-std set, BridgeTower achieves an accuracy of 78.73%, outperforming the previous state-of-the-art model METER by 1.09% with the same pre-training data and almost negligible additional parameters and computational costs. Notably, when further scaling the model, BridgeTower achieves an accuracy of 81.15%, surpassing models that are pre-trained on orders-of-magnitude larger datasets. Here is how to use this model to perform contrastive learning between image and text pairs:

번역 대기 중 · 원문 표시

분류 준비 중

입력 미확인→ 출력 미확인

요금 미확인멀티모달 및 채널 요금 별도 확인

자세히 보기 ↗

비교 추가출처 / 문서 ↗

COST EXPLORER

좋은 모델, 명확한 비용.

비용 분석

월간 토큰 사용량으로 예상 API 비용을 비교하세요.

월간 입력 토큰 10 M월간 출력 토큰 2 M1 M = 100만 토큰 · USD
동일 채널의 기본 텍스트 토큰 요금 비교

수집된 백만 토큰당 USD 요금입니다. 캐시 할인, 구간 요금, 추가 추론 출력, 세금은 제외됩니다. 최신 요금은 출처에서 확인하세요.

MODEL HANDBOOK

이해하고 선택하세요

선택 가이드 및 FAQ

API 요금은 어떻게 비교하나요?+

입력, 출력, 캐시 요금을 구분하고 동일 작업량으로 비교하세요. 이미지, 음성, 영상은 개수, 시간, 해상도로 과금될 수 있습니다.

오픈 웨이트는 무료 상업 이용을 뜻하나요?+

반드시 그렇지는 않습니다. 버전별 라이선스, 상업 이용 제한, 정책을 확인하세요. 자체 호스팅에는 연산 및 유지 비용도 듭니다.

1위 모델이 나에게 가장 적합한가요?+

순위는 작업, 버전, 평가 방식에 따라 다릅니다. 실제 업무로 정확도, 지연, 안정성, 비용을 비교하세요.

나에게 맞는 AI 모델을 찾아보세요.

AI 모델 탐색 3

bridgetower-base

bridgetower-base-itm-mlm

bridgetower-large-itm-mlm-itc

일치하는 모델 없음

좋은 모델, 명확한 비용.

이해하고 선택하세요

뉴스로 흐름을, 도구로 효율을, 모델로 가능성을.