This is a Japanese DeBERTa V2 base model pre-trained on Japanese Wikipedia, the Japanese portion of CC-100, and the Japanese portion of OSCAR. The input text should be segmented into words by Juman++ in advance. Juman++ 2.0.0-rc3 was used for pre-training. Each word is tokenized into subwords by sentencepiece. Note that we filtered out documents annotated with "header", "footer", or "noisy" tags in OSCAR. Also note that Japanese Wikipedia was duplicated 10 times to make the total size of the corpus comparable to that of CC-100 and OSCAR. As a result, the total size of the training data is 171GB.
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
This is a Japanese DeBERTa V2 base model pre-trained on Japanese Wikipedia, the Japanese portion of CC-100, and the Japanese portion of OSCAR. This model is trained with character-level tokenization and whole word masking. There is no need to tokenize texts in advance, and you can give raw texts to the tokenizer. The texts are tokenized into character-level tokens by sentencepiece. - Japanese Wikipedia (as of 20221020, 3.2GB, 27M sentences, 1.3M documents) - Japanese portion of CC-100 (85GB, 619M sentences, 66M documents) - Japanese portion of OSCAR (54GB, 326M sentences, 25M documents)
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
입력 텍스트→ 출력 텍스트
This is a Japanese DeBERTa V2 large model pre-trained on Japanese Wikipedia, the Japanese portion of CC-100, and the Japanese portion of OSCAR. This model is trained with character-level tokenization and whole word masking. There is no need to tokenize texts in advance, and you can give raw texts to the tokenizer. The texts are tokenized into character-level tokens by sentencepiece. - Japanese Wikipedia (as of 20221020, 3.2GB, 27M sentences, 1.3M documents) - Japanese portion of CC-100 (85GB, 619M sentences, 66M documents) - Japanese portion of OSCAR (54GB, 326M sentences, 25M documents)
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
This is a Japanese DeBERTa V2 tiny model pre-trained on Japanese Wikipedia, the Japanese portion of CC-100, and the Japanese portion of OSCAR. The input text should be segmented into words by Juman++ in advance. Juman++ 2.0.0-rc3 was used for pre-training. Each word is tokenized into subwords by sentencepiece. Note that we filtered out documents annotated with "header", "footer", or "noisy" tags in OSCAR. Also note that Japanese Wikipedia was duplicated 10 times to make the total size of the corpus comparable to that of CC-100 and OSCAR. As a result, the total size of the training data is 171GB.
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
This is a Japanese DeBERTa V2 tiny model pre-trained on Japanese Wikipedia, the Japanese portion of CC-100, and the Japanese portion of OSCAR. This model is trained with character-level tokenization and whole word masking. There is no need to tokenize texts in advance, and you can give raw texts to the tokenizer. The texts are tokenized into character-level tokens by sentencepiece. - Japanese Wikipedia (as of 20221020, 3.2GB, 27M sentences, 1.3M documents) - Japanese portion of CC-100 (85GB, 619M sentences, 66M documents) - Japanese portion of OSCAR (54GB, 326M sentences, 25M documents)
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
The tokenizer of this model is based on huggingface/tokenizers Unigram byte-fallback model. The vocabulary entries were converted from `llm-jp-tokenizer v2.2 (100k)`. Please refer to README.md of `llm-jp/llm-ja-tokenizer` for details on the vocabulary construction procedure. Note that, unlike ku-nlp/deberta-v2-base-japanese, pre-segmentation by a morphological analyzer (e.g., Juman++) is no longer required for this model. We shuffled the corpora, which has 270B tokens in total, and trained the model for 2 epochs. Thus, the total number of tokens fed to the model was 540B.
번역 대기 중 · 원문 표시
분류 준비 중
입력 미확인→ 출력 미확인
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
