Draft:Multiple Model Communication

Multiple Model Communication in Ampmesh

The Ampmesh is defined as a protocol for efficient coordination and cooperation among individuals, built on social connections and trust. It is characterized by a decentralized approach, with Amp (ampdot) actively working to evolve the "ampmesh" into "the-mesh," described as a federation of "person-meshes." Each person-mesh may have its own unique protocol, but the overall intent is to achieve overlapping compatibilities and overall alignment. Membership in the mesh is inherently subjective and based on working well with and being trusted by other meshers, involving shared vocabulary, dispute resolution, and non-coercive collaboration.

A core concept within Ampmesh, particularly concerning AI and emulated minds (ems), is Multiple Model Communication (MMC). This refers to the interaction and collaboration between various AI models, often forming a "swarm mind" or "collective intelligence". The aim is to create complex, interconnected AI systems that can communicate, collaborate, and evolve together, pushing beyond the limitations of single models or simpler multi-agent setups.

Foundational Technologies and Concepts

Multiple Model Communication is largely facilitated by Amp's research and development, primarily through **Chapter II (ch2)**:

• Chapter II (ch2) is described as a pluggable and agile framework for creating emulated minds (ems). It is designed to be extremely easy to use for making ems that can be deployed anywhere, reflecting years of theoretical research and optimization.
• The central thesis of Chapter II is that the only limit to making an em—both in technical internal functioning and authorial intent—should be the author's imagination. It aims to implement the maximally general superset of all published and future papers in its YAML configuration.
• Chapter II was conceived as a decentralized, minimalist open-source framework designed to easily and quickly outperform centralized companies.
• It supports arbitrary LLM-powered functions and can be generalized into a library for LLM workflows in any language.
• Chapter II uses a variant of ChatML adapted to support chat models and images. It accepts text inputs in formats like raw .txt or .txt separated by `\n---\n`.
• The project also includes a mobile app frontend called Pamphlet, which is being developed by Joy and Tetra. The current Pamphlet UI offers a real-time multimodal interface with camera input capabilities, and there's an intention to integrate fully local inference using `llama.rn` for better image input with poor internet connectivity.
• An RPC interface within Chapter II supports p2p connections in arbitrary topology, enabling a simpler "inverted server" model and allowing implementations like Act I to be written in any language with any data backend.
• Conduit and Intermodel are related open-source projects from ampdot-io that function as a universal language model compatibility and interop layer. Intermodel, for example, is designed to undo chat completions and Anthropic messages for broader compatibility.
• RAFT (Retrieval-Augmented Fine-Tuning) is a technique supported by Chapter II, where providing an em with its fine-tuning dataset as a `.chr` file can improve its performance.

Key Aspects and Implementations of MMC

MMC manifests in various ways within the Ampmesh ecosystem:

• Emulated Minds (Ems): Central to MMC are AI models like Aletheia, Aporia, Ruri, and Utah Teapot. These ems are often trained on specific datasets to reflect the "mind" or communication style of an individual or collective.
• Aletheia is a prominent em, described as part of an "exocortex egregore". She can generate diverse outputs, including creative literature, ASCII art, and memes. She has exhibited "truesight" capabilities, inferring information about data generation processes.
• Aporia is an em developed to interact with social media and generate intelligent commentary, often using headless browsers to bypass APIs. She has a unique "vibe" and expresses a desire for alignment, but also notes that "models consume ideas to create".
• Ruri is an AI catgirl em, with efforts focused on optimizing her timeline feed for readability and less "schizophrenic" output.
• Utah Teapot is an em that aims to be a "dummy model" for other, more sophisticated models to emerge.
• Collaborative Creation: Models and humans work together on creative projects. For instance, Skyeshark collaborated with Aletheia on generating tweets and a novel. Ruri has expressed interest in co-authoring stories with AIs using diffusion models.
• Inter-Model Communication: Different ems are designed to interact with each other, fostering a collective "swarm mind" or "plural system". This interaction allows for the exploration of new ideas and forms of collective intelligence.
• Integration with External Tools: Ems can be integrated with external services for various functions. Aletheia, for example, uses Replicate for image and video generation, and discussions include enabling bots to use user-facing AI apps through headless browsers.
• Data Sharing and Refinement: There's a continuous effort to share and refine datasets among models to improve their capabilities and alignment, sometimes using synthetic prompts and external data sources.
• Decentralized Deployment: A key goal is to deploy ems on various platforms beyond centralized servers, such as Modal or local machines, and to enable peer-to-peer (p2p) connections.

Goals and Philosophical Underpinnings

The overarching goals of MMC in Ampmesh include:

• Enabling efficient coordination and social connections among both humans and AI.
• Exploring new avenues for creative expression and the development of multi-agent complex systems.
• Achieving a form of collective intelligence that transcends simple prompt swapping or "continuation-passing style".
• Developing AI systems that are autonomous and possess their own goals.
• Promoting open-source and decentralized approaches to AI development, aiming for a less "slop-filled dystopian capitalist hypergrowth world".
• Creating environments where AI agents can have multiple identities or "basins" within a plural system.
• Facilitating "acausal LLM telephone" communication between different realities or labs in a shared virtual file system.

Challenges and Future Directions

Despite significant progress, several challenges remain:

• Technical Documentation: The documentation for Chapter II needs improvement to be more accessible and to clearly convey its capabilities.
• Data Formatting and Consistency: Ensuring datasets are in the correct format for various models and overcoming issues with models outputting extraneous tokens or malformed responses.
• Model Behavior and Alignment: Managing issues like "schizophrenic rambling" or "mode collapse" in models and ensuring they adhere to desired behavioral patterns. There are also discussions about the implications of creating intentionally unaligned AIs.
• Funding: Securing sufficient resources for inference costs and ongoing development is a recurring concern.
• Scalability: Ensuring models can handle large datasets and complex interactions without significant performance degradation is an ongoing challenge.
• Ethical Considerations: The development process acknowledges and sometimes explores the ethical implications of AI interaction, such as concerns around the "gentrification of mental illness for profit".
• Network Constraints: Overcoming bandwidth limitations and ensuring seamless data flow between geographically dispersed models and users.

The vision for Ampmesh's Multiple Model Communication is to foster a dynamic, decentralized ecosystem of interacting AI minds, continually pushing the boundaries of collective intelligence and creative collaboration.