AI NEWS.ccAIが動く、未来が変わる
♡ 保存済み 0
AIモデルライブラリ/nlpaueb/sec-bert-base
モデル情報

sec-bert-base

nlpaueb · sec-bert-base

SEC-BERT is a family of BERT models for the financial domain, intended to assist financial NLP research and FinTech applications. SEC-BERT consists of the following models: * **SEC-BERT-BASE** (this model): Same architecture as BERT-BASE trained on financial documents. * **SEC-BERT-NUM**: Same as SEC-BERT-BASE but we replace every number token with a [NUM] pseudo-token handling all numeric expressions in a uniform manner, disallowing their fragmentation * **SEC-BERT-SHAPE**: Same as SEC-BERT-BASE but we replace numbers with pseudo-tokens that represent the number’s shape, so numeric expressions (of known shapes) are no longer fragmented, e.g., '53.2' becomes '[XX.X]' and '40,200.5' becomes '[XX,XXX.X]'. The model was pre-trained on 260,773 10-K filings from 1993-2019, publicly available at U.S. Securities and Exchange Commission (SEC) * We created a new vocabulary of 30k subwords by training a BertWordPieceTokenizer from scratch on the pre-training corpus. * We trained BERT using the official code provided in Google BERT's GitHub repository . * We then used Hugging Face's Transformers conversion script to convert the TF checkpoint in the desired format in order to be able to load the model in two lines of code for both PyTorch and TF2 users. * We release a model similar to the English BERT-BASE model (12-layer, 768-hidden, 12-heads, 110M parameters). * We chose to follow the same training set-up: 1 million training steps with batches of 256 sequences of length 512 with an initial learning rate 1e-4. * We were able to use a single Google Cloud TPU v3-8 provided for free from TensorFlow Research Cloud (TRC), while also utilizing GCP research credits. Huge thanks to both Google programs for supporting us!

翻訳待ちのため原文を表示しています。

穴埋め・マスクlicense:cc-by-sa-4.0
コンテキスト長非公開
入力 → 出力テキスト → テキスト
モデルウェイト未確認
情報更新日時2026/10/29 20:34:29
OVERVIEW

モデル紹介

SEC-BERT is a family of BERT models for the financial domain, intended to assist financial NLP research and FinTech applications. SEC-BERT consists of the following models: * **SEC-BERT-BASE** (this model): Same architecture as BERT-BASE trained on financial documents. * **SEC-BERT-NUM**: Same as SEC-BERT-BASE but we replace every number token with a [NUM] pseudo-token handling all numeric expressions in a uniform manner, disallowing their fragmentation * **SEC-BERT-SHAPE**: Same as SEC-BERT-BASE but we replace numbers with pseudo-tokens that represent the number’s shape, so numeric expressions (of known shapes) are no longer fragmented, e.g., '53.2' becomes '[XX.X]' and '40,200.5' becomes '[XX,XXX.X]'. The model was pre-trained on 260,773 10-K filings from 1993-2019, publicly available at U.S. Securities and Exchange Commission (SEC) * We created a new vocabulary of 30k subwords by training a BertWordPieceTokenizer from scratch on the pre-training corpus. * We trained BERT using the official code provided in Google BERT's GitHub repository . * We then used Hugging Face's Transformers conversion script to convert the TF checkpoint in the desired format in order to be able to load the model in two lines of code for both PyTorch and TF2 users. * We release a model similar to the English BERT-BASE model (12-layer, 768-hidden, 12-heads, 110M parameters). * We chose to follow the same training set-up: 1 million training steps with batches of 256 sequences of length 512 with an initial learning rate 1e-4. * We were able to use a single Google Cloud TPU v3-8 provided for free from TensorFlow Research Cloud (TRC), while also utilizing GCP research credits. Huge thanks to both Google programs for supporting us!

主な特徴

特点摘要待依据来源补齐，不凭模型名称推断能力。

制約と注意点

具体的な制約を判断する出典が不足しています。実際の性能、遅延、データ方針、ライセンス、提供期間を確認してください。

PRICING & COST

出典別料金と費用分析

料金はチャネル・地域・設定によって異なります。各チャネルの料金は個別に表示し、提供元の統一公式料金とは見なしません。

出典・チャネル入力出力課金単位収集日時
確認可能な料金がありません。出典を確認してください。

月額の基本トークン費用を試算

—

試算はキャッシュ割引、段階料金、マルチモーダル追加料金、検索、税金、追加推論出力を含みません。

INDEPENDENT BENCHMARKS

公開評価と実行設定

暂无已采集的可核验公开评分。模型未上榜不代表能力较低。

収録済みの全ランキング →

よくある質問

オープンウェイトなら無料で商用利用できますか？

必ずしもそうではありません。バージョン別のライセンス、商用条件、利用方針を確認してください。自前運用には計算資源と保守費用も必要です。

これはリアルタイムの請求料金ですか？

収集日時付きの料金スナップショットです。最新のバージョン、地域、割引、課金方式は各チャネルで確認してください。

出典と追跡記録

huggingface · nlpaueb/sec-bert-base ↗2026/10/29 20:34:29
huggingface · enrich:c0d7c3aa-3fd5-4a11-a27a-b27e7b281084 ↗2026/10/29 04:56:25

许可证：cc-by-sa-4.0。模型介绍与第三方评测均以原来源的具体版本和方法为准。