AI NEWS.ccAI正在发生 未来正在更新
♡ 我的收藏 0
AI NEWS · 模型情报站发现能力边界，让每一次选择更有依据 ↗基础档案先收录 · 缺失资料持续补齐
THE AI MODEL DIRECTORY 模型探索，从这里开始

找到适合你的 AI 模型。

从语言到影像，从开源到 API。读懂模型能力、比较真实成本，
为你的下一个想法，找到合适的智能引擎。

热门探索
84,991 款模型档案13202 家厂商 / 发布者45 个能力分类465 款有可比报价 · 30 款有公开评测
T文本生成▧图像生成▷视频生成⌘代码生成◖语音合成♫音乐生成⬡3D 建模▤文档分析
DISCOVER & COMPARE

探索 AI 模型 3

为你发现值得关注的模型

Nyströmformer model for masked language modeling (MLM) pretrained on BookCorpus and English Wikipedia for sequence length 512. The Nyströmformer model was proposed in Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention by Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, and Vikas Singh. Transformers have emerged as a powerful tool for a broad range of natural language processing tasks. A key component that drives the impressive performance of Transformers is the self-attention mechanism that encodes the influence or dependence of other tokens on each specific token. While beneficial, the quadratic complexity of self-attention on the input sequence length has limited its application to longer sequences — a topic being actively studied in the community. To address this limitation, we propose Nyströmformer — a model that exhibits favorable scalability as a function of sequence length. Our idea is based on adapting the Nyström method to approximate standard self-attention with O(n) complexity. The scalability of Nyströmformer enables application to longer sequences with thousands of tokens. We perform evaluations on multiple downstream tasks on the GLUE benchmark and IMDB reviews with standard sequence length, and find that our Nyströmformer performs comparably, or in a few cases, even slightly better, than standard self-attention. On longer sequence tasks in the Long Range Arena (LRA) benchmark, Nyströmformer performs favorably relative to other efficient self-attention methods. Our code is available at this https URL.

译文排队中 · 当前显示来源原文

填空与掩码
输入 文本→ 输出 文本
价格待核验多模态与渠道费用须分别核对
查看详情 ↗
来源 / 文档 ↗

yoso-4096

uw-madison

The YOSO model was proposed in You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling by Zhanpeng Zeng, Yunyang Xiong, Sathya N. Ravi, Shailesh Acharya, Glenn Fung, Vikas Singh. Transformer-based models are widely used in natural language processing (NLP). Central to the transformer model is the self-attention mechanism, which captures the interactions of token pairs in the input sequences and depends quadratically on the sequence length. Training such models on longer sequences is expensive. In this paper, we show that a Bernoulli sampling attention mechanism based on Locality Sensitive Hashing (LSH), decreases the quadratic complexity of such models to linear. We bypass the quadratic cost by considering self-attention as a sum of individual tokens associated with Bernoulli random variables that can, in principle, be sampled at once by a single hash (although in practice, this number may be a small constant). This leads to an efficient sampling scheme to estimate self-attention which relies on specific modifications of LSH (to enable deployment on GPU architectures). We evaluate our algorithm on the GLUE benchmark with standard 512 sequence length where we see favorable performance relative to a standard pretrained Transformer. On the Long Range Arena (LRA) benchmark, for evaluating performance on long sequences, our method achieves results consistent with softmax self-attention but with sizable speed-ups and memory savings and often outperforms other efficient self-attention methods. Our code is available at this https URL

译文排队中 · 当前显示来源原文

填空与掩码
输入 文本→ 输出 文本
价格待核验多模态与渠道费用须分别核对
查看详情 ↗
来源 / 文档 ↗
COST EXPLORER

好模型，也要算好成本。

费用分析

调整月度 Token 用量，直观比较不同模型的预估 API 开支。

1 M = 100 万 Token · 美元计价
仅比较同一渠道的基础文本 Token 报价

使用已采集且单位可比的 USD / 百万 Token 渠道报价。报价为采集时快照；不计缓存优惠、阶梯价、推理额外输出与税费，请以来源的当前报价为准。

MODEL HANDBOOK

读懂模型，再做选择

选型与常见问题
API 价格应该怎么比较？+

先区分输入、输出和缓存价格，再使用同一任务和用量估算总费用。图像、音频和视频可能按张、时长或分辨率计费，不能直接与 Token 单价比较。

开放权重，就代表可以免费商用吗？+

不一定。必须逐一核对具体模型版本的许可证、商用限制及使用政策。自己部署还需要考虑显存、推理资源和运维费用。

排行榜第一，就是最适合我的模型吗？+

榜单取决于测试任务、版本与评测方法。建议用实际业务样本比较准确性、时延、稳定性和成本，再选择模型。

AI NEWS · 从信息到选择

资讯看趋势，工具提效率，模型筑可能。

探索 AI 资讯 ↗发现 AI 工具 ↗