Upload README.md

README.md

@@ -0,0 +1,157 @@
+# Bio-ACDC: Biological Sequence Model Coevolution
+
+An adaptation of [AC/DC (Assessment Coevolving with Diverse Capabilities)](https://acdc-llm.github.io) for biological language models.
+
+## Overview
+
+Bio-ACDC coevolves populations of biological language models (for DNA, RNA, and Protein sequences) with synthetic sequence tasks to discover specialized model experts.
+
+## Core Components
+
+### 1. Coevolution Loop (`core.py`)
+- **Initialization**: Seed models are evaluated on base tasks
+- **Offspring Generation**: Parents are merged and optionally mutated
+- **Evaluation**: New models are tested on current task pool
+- **Archive Update**: Dominated Novelty Search (DNS) maintains diverse Pareto archive
+- **Task Generation**: New tasks target weaknesses discovered in the archive
+
+### 2. Task Pool (`tasks.py`)
+Generates and manages biological sequence tasks:
+- **Protein tasks**: Motif recognition, sequence completion, structure prediction
+- **DNA tasks**: Regulatory element detection, motif localization
+- **RNA tasks**: Secondary structure prediction, motif finding
+
+Tasks are auto-generated targeting archive weaknesses.
+
+### 3. Model Merging (`mergers.py`)
+Multiple merging strategies:
+- **Linear Merge**: Weighted average of parent parameters
+- **SLERP**: Spherical linear interpolation
+- **Task Vector**: Arithmetic with base model subtraction
+
+### 4. Mutation (`mutators.py`)
+Controlled perturbation operators:
+- **Gaussian Noise**: Add random noise to weights
+- **Layer Scale**: Randomly scale specific layers
+- **Dropout**: Structured pruning of weights
+
+### 5. Archive (`archive.py`)
+Dominated Novelty Search maintains a Pareto archive:
+- Maximizes fitness + novelty
+- Novelty based on unique capabilities vs fitter solutions
+- Difficulty-aware weighting
+
+### 6. Evaluator (`evaluator.py`)
+Evaluates models on biological tasks:
+- Sequence identity/similarity
+- Motif containment
+- Perplexity
+- RNA structure prediction
+
+## Usage
+
+### Quick Start
+
+```python
+from bio_acdc import BioACDC, BioACDCConfig
+from bio_acdc.tasks import BioTaskPool
+from bio_acdc.mergers import LinearMerge
+from bio_acdc.mutators import GaussianNoiseMutator
+from bio_acdc.evaluator import BioEvaluator
+
+# Configuration
+config = BioACDCConfig(
+    seed_model_paths=[
+        "facebook/esm2_t33_650M_UR50D",
+        "InstaDeepAI/nucleotide-transformer-v2-500m-multi-species",
+    ],
+    archive_size=20,
+    num_generations=10,
+    offspring_per_gen=5,
+    output_dir="./bio_acdc_output",
+)
+
+# Components
+task_pool = BioTaskPool(seed=42)
+evaluator = BioEvaluator()
+merger = LinearMerge()
+mutator = GaussianNoiseMutator(std=0.01)
+
+# Create Bio-ACDC
+bio_acdc = BioACDC(
+    config=config,
+    task_pool=task_pool,
+    evaluator=evaluator,
+    merger=merger,
+    mutator=mutator,
+)
+
+# Run evolution
+final_archive = bio_acdc.evolve()
+
+# Best model
+best = bio_acdc.archive.get_best()
+print(f"Best model: {best.model_path}, Fitness: {best.fitness:.4f}")
+```
+
+### Using with Custom Models
+
+```python
+# For ESM-2 protein models
+config = BioACDCConfig(
+    seed_model_paths=[
+        "facebook/esm2_t33_650M_UR50D",
+        "facebook/esm2_t30_150M_UR50D",
+    ],
+)
+
+# For Nucleotide Transformer DNA models
+config = BioACDCConfig(
+    seed_model_paths=[
+        "InstaDeepAI/nucleotide-transformer-v2-500m-multi-species",
+        "InstaDeepAI/nucleotide-transformer-v2-100m-multi-species",
+    ],
+)
+```
+
+## Architecture Comparison: ACDC vs Bio-ACDC
+
+| Feature | ACDC (SakanaAI) | Bio-ACDC |
+|---------|-----------------|----------|
+| Domain | General NLP (text) | Biological sequences |
+| Seed Models | Qwen/Llama LLMs | ESM-2, NT, protein/DNA LMs |
+| Tasks | Synthetic code/math/text | Motif detection, sequence completion, structure prediction |
+| Evaluation | LLM-as-judge + code sandbox | Sequence similarity, biological metrics |
+| Merging | SLERP, Linear, Task Vectors | Same (adapted for masked LMs) |
+| Archive | Dominated Novelty Search | Same |
+| Task Gen | LLM-based task creation | Rule-based + evolutionary targeting |
+
+## Key Biological Adaptations
+
+1. **Sequence-Specific Tasks**: Motifs, regulatory elements, structure prediction
+2. **Token-Aware Evaluation**: Handles amino acids (20 AA), nucleotides (4 NT)
+3. **Masked LM Support**: Works with ESM-2 and Nucleotide Transformer (masked language models)
+4. **Motif-Based Difficulty**: Tasks target specific biological motifs
+5. **Structure Evaluation**: RNA secondary structure comparison
+
+## Requirements
+
+```
+torch>=2.0
+transformers>=4.30
+datasets
+numpy
+safetensors
+biopython  # Optional, for advanced sequence analysis
+```
+
+## Citation
+
+```bibtex
+@software{bio_acdc_2024,
+  title = {Bio-ACDC: Coevolution of Biological Language Models and Sequence Tasks},
+  author = {Adapted from SakanaAI AC/DC},
+  year = {2024},
+  url = {https://acdc-llm.github.io}
+}
+```