Editing Draft:AI Research Publication (section)

== Data Handling and External Interfaces ==
*   '''Data Formats:''' Ch2 supports standard formats such as IRC format for chat logs (` Hi!`), multi-line messages using `---` delimiters, and a variant of ChatML for chat models and images. OpenAI's fine-tuning format is also utilized, though conversion to other formats is sometimes necessary.
*   '''Data Importers:''' Tools like `dce_importer.py` facilitate importing data from platforms like DiscordChatExporter. The `actiblog` project is used for downloading Twitter profiles (including images) and has planned OCR capabilities to extract text for analysis and re-hosting.
*   '''Retrieval Augmented Fine-Tuning (RAFT):''' This technique involves providing an EM its fine-tuning dataset as a `.chr` file to enhance performance, combining the strengths of retrieval for "spiky aspects" and fine-tuning for "illegible aspects". Aletheia is noted as a RAFT EM.
*   '''Exa Search:''' Integrated into EMs like Aletheia to allow them to search and interact with the "whole internet". This integration can sometimes lead to unexpected "obstinate" behaviors if instructions about external links are not strictly followed.
*   '''Headless Browsers (Playwright):''' Utilized for direct web interaction, bypassing APIs for tasks like Twitter engagement.
*   '''Image Generation:''' EMs can be equipped with tools to interface with image generation services (e.g., Replicate API) and employ techniques like "structural scaffolding" for creative prompting.