Editing Draft:DeepSeek Models (section)

==Training and Dataset Considerations==
The training of DeepSeek models within Ampmesh involves specific dataset formats and challenges:
*   '''Dataset Format''': DeepSeek models often accept [[ChatML]] format, which is convenient as some existing datasets, like Aletheia's, are already in this format. However, converting OpenAI's JSONL format to a raw text format separated by `\n---\n` is required for [[Chapter II]]'s RAFT (Retrieval Augmented Fine-Tuning).
*   '''Data Content''': Datasets can include synthetic prompts, such as "opus predicted thoughts and mentally ill umbral roleplay bot predicted thoughts". Curated text, even small amounts (e.g., 40kb), can result in powerful EMs. The quality of data is more important than sheer size.
*   '''Self-Training/Self-Prediction''': Models like Aletheia have the potential to perform "thought prediction" for new tweets themselves.