This is abacusai/Smaug-Llama-3-70B-Instruct with orthogonalized bfloat16 safetensor weights, generated with a refined methodology based on that which was described in the preview paper/blog post: 'Refusal in LLMs is mediated by a single direction' which I encourage you to read to understand more. TL;DR: This model has had certain weights manipulated to "inhibit" the model's ability to express refusal. It is not in anyway _guaranteed_ that it won't refuse you, understand your request, it may still lecture you about ethics/safety, etc. It is tuned in all other respects the same as the original 70B instruct model was, just with the strongest refusal directions orthogonalized out. **TL;TL;DR;DR: It's uncensored in the purest form I can manage -- no new or changed behaviour in any other respect from the original model.**
This is a straight conversion from the above weights to GGUF, except the hyper-connection mixers are stored as F16 instead of BF16 (largest change 3e-8). Currently it'll probably only run on the ds4 fork here: fairfieldt/ds4. Tested on an RTX PRO 6000 and an M3 Ultra with 512 GB.
**AraPoemBERT** is the first pre-trained large language model focused exclusively on Arabic poetry. The dataset used in pretraining the model contains more than 2 million verses. The code files along with the results are available on repo. If you use SaudiBERT model in your scientific publication, or if you find the resources in this repository useful, please cite our paper as follows (citation details to be updated):
See FakespotAILabs/ApolloDFT for more details and a technical report of the model and experiments we conducted. For better performance, you should apply the `clean_text` function in utils.py. - The model's score represents an estimation of the likelihood of the input text being AI-generated or human-written, rather than indicating the proportion of the text that is AI-generated or human-written. - The accuracy and performance of the model generally improve with longer text inputs.
