Explode Codex - Modularize Your Codex Files

Extract children from codex files into separate files with include directives

Last updated: 2025-11-17

Explode Codex - Modularize Your Codex Files

The Explode Codex tool automatically extracts direct children from a codex file based on entity type, saves each as a standalone V1.0 codex file, and replaces them with include directives in the parent file.

This enables true modularity, team collaboration, and Git-friendly workflows.

What Does It Do?

Before Explosion:

# story.codex.yaml - Single large file
children:
  - id: "char-hero-001"
    type: character
    name: "Hero"
    # ... full character data ...

  - id: "char-villain-001"
    type: character
    name: "Villain"
    # ... full character data ...

  - id: "loc-castle-001"
    type: location
    name: "Castle"
    # ... full location data ...

After Explosion:

# story.codex.yaml - Now uses includes
children:
  - include: /characters/Hero.codex.yaml
  - include: /characters/Villain.codex.yaml
  - include: /locations/Castle.codex.yaml

Extracted Files: - characters/Hero.codex.yaml - Full standalone V1.0 codex - characters/Villain.codex.yaml - Full standalone V1.0 codex - locations/Castle.codex.yaml - Full standalone V1.0 codex

Each extracted file is a complete, valid V1.0 codex with proper metadata.

Benefits

🧩 True Modularity

Each entity lives in its own file, making projects easier to navigate and maintain.

👥 Team Collaboration

Multiple people can work on different entities simultaneously without merge conflicts.

📝 Git-Friendly

Smaller files lead to clearer diffs, better conflict resolution, and more meaningful commits.

🗂️ Organized Structure

Entities are automatically organized by type into appropriate directories.

🔧 Auto-Fixed

Extracted files are automatically run through the auto-fixer to ensure V1.0 compliance.

Installation & Usage

Command-Line Usage

The tool is located at codex/services/explode_codex.py and can be run as a standalone script:

python codex/services/explode_codex.py <input_file> [options]

Basic Examples

Extract Specific Types

Extract only characters and locations:

python codex/services/explode_codex.py story.codex.yaml --types character,location

Extract All Direct Children

Extract all direct children regardless of type:

python codex/services/explode_codex.py story.codex.yaml

Custom Output Pattern

Control where extracted files are saved:

python codex/services/explode_codex.py story.codex.yaml \
  --types character \
  --output-pattern "./entities/{type}/{name}.codex.yaml"

Dry Run Preview

See what would be extracted without making changes:

python codex/services/explode_codex.py story.codex.yaml \
  --types character,location \
  --dry-run

Disable Auto-Fix

Skip the automatic fixing step:

python codex/services/explode_codex.py story.codex.yaml \
  --types character \
  --no-auto-fix

Command-Line Options

Option	Description	Default
`--types`	Comma-separated entity types to extract (e.g., `character,location`)	All types
`--output-pattern`	Path pattern with placeholders: `{type}`, `{name}`, `{id}`, `{index}`	`./{type}s/{name}.codex.yaml`
`--format`	Output format: `yaml` or `json`	`yaml`
`--dry-run`	Preview without making changes	`false`
`--no-backup`	Don't create backup of original file	`false` (backup enabled)
`--no-auto-fix`	Don't run auto-fixer on extracted files	`false` (auto-fix enabled)
`--force`	Overwrite existing files	`false`
`-v, --verbose`	Show detailed logging	`false`

Output Pattern Placeholders

The --output-pattern option supports these placeholders:

{type} - Entity type (e.g., character, location)
{name} - Entity name (sanitized for filenames)
{id} - Entity ID
{index} - Zero-based index in extraction order

Examples:

# Group by type, then name
--output-pattern "./{type}s/{name}.codex.yaml"
# Result: ./characters/Hero.codex.yaml

# Flat structure with type prefix
--output-pattern "./{type}-{name}.codex.yaml"
# Result: ./character-Hero.codex.yaml

# Organized by ID
--output-pattern "./entities/{id}.codex.yaml"
# Result: ./entities/char-hero-001.codex.yaml

# Numbered sequence
--output-pattern "./extracted/{index}-{name}.codex.yaml"
# Result: ./extracted/0-Hero.codex.yaml

How It Works

1. Read & Validate

Reads input codex file (YAML or JSON)
Validates V1.0 format
Checks for children array

2. Extract Children

Filters direct children by specified types
Only processes first-level children (not recursive)
Preserves original child order

3. Create Standalone Files

Each extracted child becomes a full V1.0 codex:

metadata:
  formatVersion: "1.0"
  documentVersion: "1.0.0"
  created: "2025-11-17T10:00:00Z"
  extractedFrom: "/path/to/parent.codex.yaml"
  author: "Inherited from parent"
  license: "Inherited from parent"

id: "char-hero-001"
type: "character"
name: "Hero"
# ... all child data preserved ...

4. Update Parent File

Replaces extracted children with include directives
Keeps non-extracted children in place
Updates metadata:
metadata.updated - Timestamp
metadata.exploded - Extraction details
Creates backup (.backup extension)

5. Auto-Fix (Optional)

Runs auto-fixer on all extracted files
Ensures V1.0 compliance
Fixes common issues:
Missing IDs
Invalid UUIDs
Empty names
Attribute structure
String formatting

Library Usage

You can also use Explode Codex programmatically:

from codex.services.explode_codex import CodexExploder

exploder = CodexExploder()
result = exploder.explode(
    input_path="story.codex.yaml",
    types=["character", "location"],
    output_pattern="./{type}s/{name}.codex.yaml",
    options={
        "dry_run": False,
        "backup": True,
        "format": "yaml",
        "auto_fix": True,
        "force": False,
        "verbose": False
    }
)

# Check results
if result['success']:
    print(f"Extracted {result['extracted_count']} entities")
    for file_path in result['extracted_files']:
        print(f"  - {file_path}")

    # Auto-fix stats
    auto_fix = result['auto_fix_results']
    print(f"Fixed {auto_fix['fixed_files']} files")
    print(f"Total fixes: {auto_fix['total_fixes']}")
else:
    print("Errors:")
    for error in result['errors']:
        print(f"  - {error}")

Workflow Integration

1. Initial Project Setup

Start with a monolithic codex file:

# Create your initial codex
vim my-story.codex.yaml

# Add all entities directly as children

2. Explode When Ready

When your project grows, modularize it:

# Extract all characters and locations
python codex/services/explode_codex.py my-story.codex.yaml \
  --types character,location

3. Work on Individual Files

Edit extracted files independently:

vim characters/Hero.codex.yaml
vim locations/Castle.codex.yaml

4. Add More Entities

Add new extracted files and reference them:

# Create new character file
vim characters/NewCharacter.codex.yaml

# Add include to parent
# children:
#   - include: /characters/NewCharacter.codex.yaml

5. Version Control

Commit your modular structure:

git add my-story.codex.yaml characters/ locations/
git commit -m "Modularized project structure"

Best Practices

When to Explode

✅ Good use cases: - Projects with many entities (10+ characters, locations, etc.) - Team collaboration on large projects - When entities need independent editing - When you want better Git diffs

❌ When to keep monolithic: - Small projects (< 10 entities) - Single-author projects - Tightly coupled entities - Rapid prototyping

Naming Conventions

Use consistent naming patterns:

# Good: Clear type-based organization
characters/Hero.codex.yaml
characters/Villain.codex.yaml
locations/Castle.codex.yaml
locations/Forest.codex.yaml

# Also good: Type prefixes
character-Hero.codex.yaml
character-Villain.codex.yaml
location-Castle.codex.yaml
location-Forest.codex.yaml

Directory Structure

Recommended structure:

my-story/
├── index.codex.yaml          # Main file with includes
├── characters/
│   ├── Hero.codex.yaml
│   ├── Villain.codex.yaml
│   └── Sidekick.codex.yaml
├── locations/
│   ├── Castle.codex.yaml
│   └── Forest.codex.yaml
└── factions/
    └── Knights.codex.yaml

Git Workflow

# 1. Explode the codex
python codex/services/explode_codex.py story.codex.yaml --types character,location

# 2. Review changes
git status
git diff story.codex.yaml

# 3. Commit with clear message
git add .
git commit -m "Modularized characters and locations

- Extracted 5 characters to characters/
- Extracted 3 locations to locations/
- Updated main file with include directives"

# 4. Push changes
git push

Troubleshooting

"No children matched the specified types"

Problem: The specified types don't match any children.

Solution: Check the type field in your children. Types are case-insensitive.

# List types in dry-run mode first
python codex/services/explode_codex.py story.codex.yaml --dry-run

"File already exists"

Problem: Output file already exists.

Solutions: 1. Use --force to overwrite 2. Delete existing files manually 3. Use a different output pattern

# Overwrite existing files
python codex/services/explode_codex.py story.codex.yaml --types character --force

"Unsupported file format"

Problem: Input file has unsupported extension.

Solution: Ensure file ends with .yaml, .yml, or .json:

# Rename file if needed
mv story.codex story.codex.yaml

Include Paths Not Resolving

Problem: Included files can't be found when viewing parent.

Solution: Ensure include paths are relative to the project root:

# Good: Relative to project root
children:
  - include: /characters/Hero.codex.yaml

# Bad: Absolute system path
children:
  - include: /Users/me/project/characters/Hero.codex.yaml

Auto-Fix Failures

Problem: Some files fail auto-fix step.

Solution: Check the error logs and fix manually, or disable auto-fix:

# Skip auto-fix if it's causing issues
python codex/services/explode_codex.py story.codex.yaml --no-auto-fix

Then manually run auto-fixer on specific files:

python codex/services/auto_fixer.py characters/Hero.codex.yaml

Advanced Usage

Selective Extraction

Extract only specific entity types in phases:

# Phase 1: Extract characters
python codex/services/explode_codex.py story.codex.yaml --types character

# Phase 2: Extract locations  
python codex/services/explode_codex.py story.codex.yaml --types location

# Phase 3: Extract factions
python codex/services/explode_codex.py story.codex.yaml --types faction

Custom Output Patterns

Create complex directory structures:

# Organize by first letter of name
--output-pattern "./{type}s/{name:0}/{name}.codex.yaml"
# Result: ./characters/H/Hero.codex.yaml

# Add type subdirectories
--output-pattern "./entities/{type}/{name}.codex.yaml"
# Result: ./entities/character/Hero.codex.yaml

# Use IDs for technical projects
--output-pattern "./codex-entities/{id}.codex.yaml"
# Result: ./codex-entities/char-hero-001.codex.yaml

Combining with Auto-Fixer

The explode tool automatically runs auto-fixer, but you can also:

# Explode with auto-fix enabled (default)
python codex/services/explode_codex.py story.codex.yaml --types character

# Later, re-run auto-fixer with specific options
python codex/services/auto_fixer.py characters/ --recursive

JSON Output

Extract to JSON instead of YAML:

python codex/services/explode_codex.py story.codex.yaml \
  --types character \
  --format json \
  --output-pattern "./{type}s/{name}.codex.json"