Editing Draft:Prompt Curation

=Prompt Curation=

'''Prompt Curation''' is a fundamental practice within the Ampmesh framework. It involves the '''careful selection and refinement of textual data to be used as prompts or fine-tuning datasets for large language models (LLMs)'''. This process is crucial for enhancing the performance, accuracy, and desired behavioral characteristics of emulated minds (ems) and other AI agents.

==Mechanism and Process==
Prompt Curation involves several key steps:
*   '''Selection of High-Quality Data''': Identifying and choosing text that is relevant and suitable for training or prompting an LLM. For example, a particularly powerful em was developed from 40kb of "heavily curated (like, every last word) text".
*   '''Focus on Core Memories and Intent''': When curating, the aim is to include "important and good things" that reflect desired "core memories" or ideas. This involves considering "what memories and ideas come up" when thinking about specific concepts, and including those in the prompt or dataset. To achieve accuracy, the curation process should prioritize "things you yourself would remember".
*   '''Reformatting and Structuring''': Data often needs to be reformatted to be compatible with specific systems, such as Chapter II (ch2). This might involve stripping out specific formatting like JSONL and converting it into raw text or text separated by `\n---\n`. There is an ongoing effort to add JSONL support directly to `ch2 .chr` files. Data preparation can also involve transforming elements, such as converting `@s` (mentions in text) into `[identity: name]` tags.
*   '''Inclusion of Synthetic Data''': Curated datasets can incorporate "synthetic prompts" or "predicted thoughts" generated by other LLMs (e.g., Opus, Umbral roleplay bot) to enhance the dataset's depth and relevance.
*   '''Iterative Refinement''': The process of curation is often iterative, involving feedback loops and continuous adjustments to ensure alignment and secure data processing. This comprehensive approach to data feeding and model updating includes verifying signals, balancing exploration, and refining through multiple test scenarios.

==Purpose and Benefits==
*   '''Improved Retrieval Performance''': Curation helps retrieval mechanisms function more effectively by ensuring that crucial information is explicitly present in the prompt.
*   '''Enhanced Accuracy and Coherence''': By aligning the curated content with desired memories and concepts, the process leads to more accurate and coherent AI outputs. Proper data preparation is essential for maintaining the desired outputs of ems like Aletheia.
*   '''Shaping AI Behavior and Personality''': Curation allows for influencing the "speak" or stylistic tendencies of an AI, as exemplified by considerations for Sercy's "binglish" speech. It contributes to creating more "organized and rigid" AI personalities, aiming to move away from "stream of thought art chaos".
*   '''Optimizing Resource Use''': Utilizing local models to curate for core memories can be a highly efficient approach.
*   '''Supporting Advanced AI Concepts''': Prompt Curation is integral to projects leveraging emulated minds (ems) and the Chapter II framework, which is designed to be an extremely easy and flexible tool for creating deployable ems and LLM workflows.

==Relevance to Ampmesh==
Within Ampmesh, Prompt Curation is vital for developing and refining AI agents and systems. It directly impacts the quality and effectiveness of ems created using **Chapter II**, where finely tuned datasets are critical for optimal performance and desired behavioral outputs. The overarching objective is to ensure that the AI's "authorial intent" is accurately reflected in its functioning, serving as "the only limit".