ruflo/docs/batch-initialization.md

# Batch Initialization Features

The Claude-Flow batch initialization system provides powerful capabilities for initializing multiple projects simultaneously with parallel processing, templates, and comprehensive monitoring.

## Overview

Batch initialization enables you to:
- Initialize multiple projects simultaneously
- Use project templates for consistency
- Set up different environments (dev, staging, prod)
- Leverage parallel processing for speed
- Monitor performance and resource usage
- Validate configurations before execution

## Core Features

### 1. Parallel Processing
- **Concurrent Operations**: Up to 20 simultaneous project initializations
- **Resource Management**: Automatic memory and CPU usage monitoring
- **Smart Queuing**: Intelligent task scheduling to prevent overload
- **Progress Tracking**: Real-time progress bars and statistics

### 2. Project Templates
- **Web API**: Express.js-based REST API with TypeScript support
- **React App**: Modern React application with TypeScript and build tools
- **Microservice**: Containerized microservice with Docker and Kubernetes configs
- **CLI Tool**: Command-line application with argument parsing and packaging

### 3. Environment Management
- **Development**: Debug-enabled, hot-reload, verbose logging
- **Staging**: Testing-focused, monitoring, preview features
- **Production**: Optimized, secure, minimal logging, monitoring

### 4. Performance Monitoring
- **Real-time Metrics**: Memory usage, operation count, timing
- **Resource Thresholds**: Automatic warnings for high resource usage  
- **Performance Reports**: Detailed completion statistics
- **Optimization Recommendations**: Smart suggestions for better performance

## Usage Examples

### Basic Batch Initialization
```bash
# Initialize multiple projects with default settings
claude-flow init --batch-init project1,project2,project3

# With SPARC development environment
claude-flow init --batch-init api-v1,api-v2 --sparc
```

### Template-Based Initialization
```bash
# Web API template
claude-flow init --batch-init api-service,user-service --template web-api

# React application template
claude-flow init --batch-init admin-portal,customer-portal --template react-app

# Microservice template with SPARC
claude-flow init --batch-init auth-service,payment-service --template microservice --sparc
```

### Multi-Environment Setup
```bash
# Create dev, staging, and prod versions
claude-flow init --batch-init myapp --environments dev,staging,prod

# Multiple projects across environments
claude-flow init --batch-init api,web --environments dev,staging --template web-api
```

### Configuration File Mode
```bash
# Use JSON configuration file
claude-flow init --config batch-config.json

# With custom concurrency
claude-flow init --config enterprise-setup.json --max-concurrent 8
```

### Performance Optimization
```bash
# Disable parallel processing
claude-flow init --batch-init projects --no-parallel

# Custom concurrency limit
claude-flow init --batch-init projects --max-concurrent 3

# Force overwrite with minimal setup
claude-flow init --batch-init projects --force --minimal
```

## Configuration Files

### Simple Configuration
```json
{
  "projects": ["api-service", "web-frontend", "admin-dashboard"],
  "baseOptions": {
    "sparc": true,
    "parallel": true,
    "maxConcurrency": 3,
    "template": "web-api",
    "environments": ["dev", "staging"]
  }
}
```

### Advanced Configuration
```json
{
  "baseOptions": {
    "sparc": true,
    "parallel": true,
    "maxConcurrency": 4,
    "force": true
  },
  "projectConfigs": {
    "user-api": {
      "template": "web-api",
      "environment": "dev",
      "customConfig": {
        "database": "postgresql",
        "auth": "jwt"
      }
    },
    "admin-portal": {
      "template": "react-app",
      "environment": "dev",
      "customConfig": {
        "ui": "material-ui",
        "state": "redux"
      }
    }
  }
}
```

## Batch Manager Utilities

### Create Configuration
```bash
# Basic template
claude-flow batch create-config my-batch.json

# Interactive mode
claude-flow batch create-config --interactive
```

### Validate Configuration
```bash
# Validate before running
claude-flow batch validate-config my-batch.json
```

### List Available Options
```bash
# Show all project templates
claude-flow batch list-templates

# Show all environment configurations
claude-flow batch list-environments
```

### Estimate Operations
```bash
# Estimate time and resources
claude-flow batch estimate my-batch.json
```

## Project Templates

### Web API Template
- Express.js server with TypeScript
- CORS and security middleware
- Environment configuration
- Package.json with development scripts
- Basic REST endpoints

**Files Created:**
- `package.json` - Dependencies and scripts
- `src/index.js` - Main server file
- `src/controllers/` - API controllers
- `src/routes/` - Route definitions
- `src/models/` - Data models
- `tests/` - Test files

### React App Template
- Modern React with TypeScript
- Create React App toolchain
- Development and build scripts
- Component structure
- Testing setup

**Files Created:**
- `package.json` - React dependencies
- `tsconfig.json` - TypeScript configuration
- `src/components/` - React components
- `src/hooks/` - Custom hooks
- `src/services/` - API services
- `public/` - Static assets

### Microservice Template
- Containerized Node.js service
- Docker and Docker Compose configs
- Kubernetes deployment files
- Health checks and monitoring
- Environment-based configuration

**Files Created:**
- `Dockerfile` - Container configuration
- `docker-compose.yml` - Local development
- `k8s/` - Kubernetes manifests
- `config/` - Configuration files
- `scripts/` - Deployment scripts

### CLI Tool Template
- Commander.js for argument parsing
- NPM package configuration
- Cross-platform executable
- Help system and documentation

**Files Created:**
- `package.json` - CLI package config
- `src/cli.js` - Main CLI file
- `src/commands/` - Command implementations
- `src/utils/` - Utility functions
- `bin/` - Executable files

## Environment Configurations

### Development Environment
- `NODE_ENV=development`
- `DEBUG=true`
- `LOG_LEVEL=debug`
- Hot reload enabled
- Verbose error reporting

### Staging Environment
- `NODE_ENV=staging`
- `DEBUG=false`
- `LOG_LEVEL=info`
- Testing tools enabled
- Preview mode features

### Production Environment
- `NODE_ENV=production`
- `DEBUG=false`
- `LOG_LEVEL=error`
- Performance optimizations
- Security hardening
- Minimal logging

## Performance Features

### Parallel Processing
- **Automatic Concurrency**: Smart defaults based on system resources
- **Resource Limits**: Memory and CPU usage monitoring
- **Queue Management**: Intelligent task scheduling
- **Error Isolation**: Failed projects don't affect others

### Progress Tracking
- **Real-time Updates**: Live progress bars with statistics
- **Project Status**: Individual project completion tracking
- **Time Estimates**: Remaining time calculations
- **Success Rates**: Success/failure statistics

### Resource Management
- **Memory Monitoring**: Automatic memory usage tracking
- **Threshold Warnings**: Alerts for high resource usage
- **Optimization Suggestions**: Smart recommendations
- **System Adaptation**: Automatic adjustment based on available resources

### Performance Reports
```
📊 Performance Report
====================
Duration: 45.32s
Operations: 12
Operations/sec: 0.26
Peak Memory: 1247.3MB
Average Memory: 892.1MB
Memory Efficiency: good

💡 Recommendations:
  • Consider using parallel processing for better performance
  • Use configuration files for better organization
```

## Error Handling

### Validation
- Pre-flight configuration validation
- Template and environment verification
- Resource availability checks
- Directory permission validation

### Recovery
- Continue processing on individual failures
- Detailed error reporting
- Retry mechanisms for transient failures
- Graceful degradation

### Monitoring
- Real-time error tracking
- Warning system for potential issues
- Performance threshold monitoring
- Resource usage alerts

## Integration with SPARC

All batch operations support SPARC development environment:

### SPARC Features
- **Slash Commands**: Automatic creation of Claude Code commands
- **Development Modes**: 17+ specialized SPARC modes
- **TDD Workflows**: Test-driven development support
- **Architecture Tools**: System design and documentation
- **Code Generation**: Automated code scaffolding

### SPARC Batch Benefits
- **Consistent Setup**: All projects use same SPARC configuration
- **Team Collaboration**: Standardized development environment
- **AI Integration**: Optimized for Claude Code workflows
- **Quality Assurance**: Built-in testing and validation

## Best Practices

### Configuration Management
1. **Use Templates**: Ensure consistency across projects
2. **Validate First**: Always validate configurations before execution
3. **Environment Separation**: Clear separation between dev/staging/prod
4. **Version Control**: Store batch configurations in version control

### Performance Optimization
1. **Right-size Concurrency**: Use optimal concurrency for your system
2. **Monitor Resources**: Watch memory and CPU usage during execution
3. **Batch Size**: Break large operations into smaller batches
4. **Template Selection**: Choose appropriate templates for your needs

### Error Management
1. **Incremental Approach**: Start with small batches to test
2. **Backup Strategy**: Ensure you can recover from failures
3. **Monitoring**: Set up alerts for long-running operations
4. **Documentation**: Document your batch configurations

## Troubleshooting

### Common Issues

**High Memory Usage**
- Reduce max concurrency
- Use minimal template options
- Monitor system resources

**Slow Performance**
- Enable parallel processing
- Check disk speed (SSD vs HDD)
- Optimize template selection

**Configuration Errors**
- Validate before execution
- Check template availability
- Verify environment names

**Permission Issues**
- Ensure write permissions
- Check directory access
- Validate user permissions

### Debug Options
```bash
# Verbose output
claude-flow init --batch-init projects --verbose

# Disable parallel processing for debugging
claude-flow init --batch-init projects --no-parallel

# Force operation
claude-flow init --batch-init projects --force
```

## Command Reference

### Init Command Options
```bash
claude-flow init [options]

Options:
  --batch-init <projects>     Initialize multiple projects
  --parallel                  Enable parallel processing (default: true)
  --template <name>          Use project template
  --environments <envs>      Comma-separated environments
  --config <file>           Load batch configuration
  --max-concurrent <n>      Maximum concurrent operations
  --force                   Overwrite existing files
  --minimal                 Create minimal configuration
  --sparc                   Enable SPARC development environment
```

### Batch Manager Commands
```bash
claude-flow batch <command> [options]

Commands:
  create-config [file]      Create batch configuration template
  validate-config <file>    Validate batch configuration
  list-templates           Show available project templates
  list-environments        Show available environment configs
  estimate <config>        Estimate time and resources
  help                     Show help message
```

## API Reference

For programmatic usage, the batch initialization system provides JavaScript APIs:

```javascript
import { 
  batchInitCommand,
  PROJECT_TEMPLATES,
  ENVIRONMENT_CONFIGS
} from './src/cli/simple-commands/init/batch-init.js';

// Initialize projects programmatically
const results = await batchInitCommand(['project1', 'project2'], {
  template: 'web-api',
  parallel: true,
  maxConcurrency: 5
});
```

## Conclusion

The Claude-Flow batch initialization system provides enterprise-grade capabilities for managing multiple project setups efficiently. With parallel processing, comprehensive monitoring, and extensive customization options, it scales from simple multi-project setups to complex enterprise deployments.

For additional support and examples, see the `/examples` directory and visit the [Claude-Flow documentation](https://github.com/ruvnet/claude-code-flow).