HostFactory Integration Guide¶

This guide covers the integration of the Open Resource Broker with IBM Spectrum Symphony Host Factory, including configuration, deployment, and operational considerations.

Overview¶

The Open Resource Broker integrates with IBM Spectrum Symphony Host Factory through shell script interfaces that conform to the HostFactory API specification. The plugin acts as a bridge between HostFactory and cloud providers, enabling dynamic provisioning of compute resources.

Integration Architecture¶

Component Interaction¶

IBM Spectrum Symphony Host Factory
           |
           | (Shell Script Calls)
           v
Open Resource Broker Shell Scripts
           |
           | (Python Execution)
           v
Open Resource Broker Core
           |
           | (API Calls)
           v
Cloud Provider (AWS)

Shell Script Interface¶

The plugin provides four main shell scripts that implement the HostFactory API:

getAvailableTemplates.sh - Lists available machine templates
requestMachines.sh - Requests provisioning of machines
requestReturnMachines.sh - Requests termination of machines
getRequestStatus.sh - Checks status of provisioning requests

HostFactory Configuration¶

Basic HostFactory Setup¶

Add the plugin to your HostFactory configuration file:

<HostProvider name="aws-cloud-provider">
  <Script>
    <GetAvailableTemplates>/opt/orb/scripts/getAvailableTemplates.sh</GetAvailableTemplates>
    <RequestMachines>/opt/orb/scripts/requestMachines.sh</RequestMachines>
    <RequestReturnMachines>/opt/orb/scripts/requestReturnMachines.sh</RequestReturnMachines>
    <GetRequestStatus>/opt/orb/scripts/getRequestStatus.sh</GetRequestStatus>
  </Script>
  <Parameters>
    <Parameter name="CONFIG_PATH">/opt/orb/config/config.json</Parameter>
    <Parameter name="LOG_LEVEL">INFO</Parameter>
  </Parameters>
</HostProvider>

Advanced HostFactory Configuration¶

For production environments with multiple providers:

<HostProviders>
  <HostProvider name="aws-us-east-1">
    <Script>
      <GetAvailableTemplates>/opt/orb/scripts/getAvailableTemplates.sh</GetAvailableTemplates>
      <RequestMachines>/opt/orb/scripts/requestMachines.sh</RequestMachines>
      <RequestReturnMachines>/opt/orb/scripts/requestReturnMachines.sh</RequestReturnMachines>
      <GetRequestStatus>/opt/orb/scripts/getRequestStatus.sh</GetRequestStatus>
    </Script>
    <Parameters>
      <Parameter name="CONFIG_PATH">/opt/orb/config/aws-us-east-1.yml</Parameter>
      <Parameter name="AWS_REGION">us-east-1</Parameter>
      <Parameter name="PROVIDER_NAME">aws-us-east-1</Parameter>
    </Parameters>
  </HostProvider>

  <HostProvider name="aws-us-west-2">
    <Script>
      <GetAvailableTemplates>/opt/orb/scripts/getAvailableTemplates.sh</GetAvailableTemplates>
      <RequestMachines>/opt/orb/scripts/requestMachines.sh</RequestMachines>
      <RequestReturnMachines>/opt/orb/scripts/requestReturnMachines.sh</RequestReturnMachines>
      <GetRequestStatus>/opt/orb/scripts/getRequestStatus.sh</GetRequestStatus>
    </Script>
    <Parameters>
      <Parameter name="CONFIG_PATH">/opt/hostfactory-plugin/config/aws-us-west-2.yml</Parameter>
      <Parameter name="AWS_REGION">us-west-2</Parameter>
      <Parameter name="PROVIDER_NAME">aws-us-west-2</Parameter>
    </Parameters>
  </HostProvider>
</HostProviders>

API Compliance¶

getAvailableTemplates¶

HostFactory Call:

/path/to/getAvailableTemplates.sh

Expected Output Format:

{
  "templates": [
    {
      "templateId": "basic-template",
      "maxNumber": 10,
      "attributes": {
        "type": ["String", "X86_64"],
        "ncpus": ["Numeric", "2"],
        "nram": ["Numeric", "4096"]
      }
    }
  ]
}

requestMachines¶

HostFactory Call:

/path/to/requestMachines.sh -f /tmp/input_file

Input Format:

{
  "templateId": "basic-template",
  "maxNumber": 5,
  "attributes": {
    "priority": "high",
    "environment": "production"
  }
}

Expected Output Format:

{
  "requestId": "req-12345",
  "machines": [
    {
      "machineId": "i-0123456789abcdef0",
      "status": "pending",
      "attributes": {
        "type": ["String", "X86_64"],
        "ncpus": ["Numeric", "2"],
        "nram": ["Numeric", "4096"]
      }
    }
  ]
}

requestReturnMachines¶

HostFactory Call:

/path/to/requestReturnMachines.sh -f /tmp/input_file

Input Format:

{
  "requestId": "req-12345"
}

Expected Output Format:

{
  "requestId": "req-12345",
  "status": "terminating",
  "machines": [
    {
      "machineId": "i-0123456789abcdef0",
      "status": "terminating"
    }
  ]
}

getRequestStatus¶

HostFactory Call:

/path/to/getRequestStatus.sh -f /tmp/input_file

Input Format:

{
  "requestId": "req-12345"
}

Expected Output Format:

{
  "requestId": "req-12345",
  "status": "completed",
  "machines": [
    {
      "machineId": "i-0123456789abcdef0",
      "status": "running",
      "ipAddress": "10.0.1.100",
      "attributes": {
        "type": ["String", "X86_64"],
        "ncpus": ["Numeric", "2"],
        "nram": ["Numeric", "4096"]
      }
    }
  ]
}

Deployment Scenarios¶

Scenario 1: Single Region Deployment¶

For simple deployments with one AWS region:

# config/config.yml
provider:
  type: aws
  aws:
    region: us-east-1
    profile: default

storage:
  type: dynamodb
  dynamodb:
    table_prefix: hostfactory
    region: us-east-1

templates:
  - template_id: basic-template
    max_number: 10
    attributes:
      vm_type: t3.medium
      image_id: ami-0abcdef1234567890

Scenario 2: Multi-Region Deployment¶

For deployments across multiple AWS regions:

# config/aws-us-east-1.yml
provider:
  type: aws
  aws:
    region: us-east-1
    profile: default

storage:
  type: dynamodb
  dynamodb:
    table_prefix: hostfactory-east
    region: us-east-1

# config/aws-us-west-2.yml
provider:
  type: aws
  aws:
    region: us-west-2
    profile: default

storage:
  type: dynamodb
  dynamodb:
    table_prefix: hostfactory-west
    region: us-west-2

Scenario 3: High Availability Deployment¶

For production environments requiring high availability:

# config/production.yml
provider:
  type: aws
  aws:
    region: us-east-1
    profile: production
    max_retries: 5
    timeout: 60

storage:
  type: dynamodb
  dynamodb:
    table_prefix: hostfactory-prod
    region: us-east-1
    backup_enabled: true

logging:
  level: INFO
  file_path: /var/log/hostfactory-plugin/app.log
  max_size: 100MB
  backup_count: 10

monitoring:
  enabled: true
  metrics_endpoint: http://monitoring.internal:8080/metrics

Security Configuration¶

IAM Permissions¶

The plugin requires specific IAM permissions to function properly:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:RunInstances",
        "ec2:TerminateInstances",
        "ec2:DescribeInstances",
        "ec2:DescribeImages",
        "ec2:DescribeSubnets",
        "ec2:DescribeSecurityGroups",
        "ec2:CreateTags"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "autoscaling:CreateAutoScalingGroup",
        "autoscaling:DeleteAutoScalingGroup",
        "autoscaling:DescribeAutoScalingGroups",
        "autoscaling:UpdateAutoScalingGroup"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:UpdateItem",
        "dynamodb:DeleteItem",
        "dynamodb:Query",
        "dynamodb:Scan"
      ],
      "Resource": "arn:aws:dynamodb:*:*:table/hostfactory-*"
    }
  ]
}

Network Security¶

Configure security groups for provisioned instances:

# In template configuration
templates:
  - template_id: secure-template
    attributes:
      security_group_ids:
        - sg-12345678  # Allow SSH from management network
        - sg-87654321  # Allow application traffic
      subnet_ids:
        - subnet-abcdef12  # Private subnet

Monitoring and Logging¶

HostFactory Integration Monitoring¶

Monitor the integration through HostFactory logs and plugin logs:

# HostFactory logs
tail -f /opt/symphony/logs/hostfactory.log

# Plugin logs
tail -f /var/log/hostfactory-plugin/app.log

# AWS CloudWatch logs (if configured)
aws logs tail /aws/lambda/hostfactory-plugin --follow

Health Checks¶

Implement health checks for the integration:

# Test plugin health
/opt/hostfactory-plugin/scripts/getAvailableTemplates.sh

# Test AWS connectivity
aws sts get-caller-identity

# Test DynamoDB connectivity
aws dynamodb describe-table --table-name hostfactory-requests

Troubleshooting¶

Common Integration Issues¶

Script Execution Permissions¶

# Ensure scripts are executable
chmod +x /opt/hostfactory-plugin/scripts/*.sh

Configuration Path Issues¶

# Verify configuration file exists and is readable
ls -la /opt/hostfactory-plugin/config/config.yml

AWS Credentials Issues¶

# Test AWS credentials
aws sts get-caller-identity

Debugging HostFactory Calls¶

Enable debug logging to troubleshoot integration issues:

# config/config.yml
logging:
  level: DEBUG
  console_enabled: true
  file_path: /var/log/hostfactory-plugin/debug.log

Error Handling¶

The plugin provides structured error responses for HostFactory:

{
  "error": {
    "code": "TEMPLATE_NOT_FOUND",
    "message": "Template 'invalid-template' not found",
    "details": {
      "available_templates": ["basic-template", "advanced-template"]
    }
  }
}

Performance Optimization¶

Caching¶

Enable template and configuration caching:

# config/config.yml
caching:
  enabled: true
  template_cache_ttl: 300  # 5 minutes
  config_cache_ttl: 600    # 10 minutes

Connection Pooling¶

Configure AWS connection pooling:

# config/config.yml
provider:
  aws:
    connection_pool_size: 10
    max_retries: 3
    timeout: 30

Best Practices¶

Configuration Management¶

Use separate configuration files for different environments
Store sensitive information in environment variables or AWS Secrets Manager
Validate configuration before deployment

Monitoring¶

Monitor HostFactory integration logs
Set up CloudWatch alarms for AWS resource usage
Track plugin performance metrics

Security¶

Use IAM roles instead of access keys when possible
Regularly rotate credentials
Implement least privilege access policies

Maintenance¶

Regularly update the plugin to the latest version
Monitor AWS service limits and quotas
Implement automated backup and recovery procedures

This integration guide provides comprehensive coverage of integrating the Open Resource Broker with IBM Spectrum Symphony Host Factory. For specific deployment scenarios or troubleshooting, refer to the related documentation sections.

invoke_provider.sh Wiring¶

ORB ships an invoke_provider.sh script that HostFactory calls for each provisioning operation. After orb init, the script is copied to ORB_SCRIPTS_DIR.

Environment variables to set in HostFactory¶

Configure these in your HostFactory provider definition or the environment where the HF daemon runs:

Variable	Required	Description
`ORB_CONFIG_DIR`	Yes	Points HF to the ORB config directory.
`ORB_WORK_DIR`	Recommended	Separates ORB working data from HF working data.
`ORB_LOG_DIR`	Recommended	ORB process logs (distinct from `HF_LOGDIR`).
`ORB_VENV_PATH`	If using venv	Path to the venv containing `orb`. The script activates it automatically.
`HF_LOGDIR`	Set by HF	HostFactory log directory — `invoke_provider.sh` appends to `$HF_LOGDIR/scripts.log`. Do not set this yourself.
`HF_LOGGING_CONSOLE_ENABLED`	No	Set to `false` (default) to suppress ORB console output in HF script context.
`USE_LOCAL_DEV`	Dev only	Set to `true` to run ORB from source instead of the installed package.

Where scripts go¶

After orb init, provider scripts are placed in ORB_SCRIPTS_DIR (default: $ORB_ROOT_DIR/scripts). Point HostFactory's providerCommandPath at this directory:

{
  "providerCommandPath": "/opt/myapp/orb/scripts"
}

The key script is invoke_provider.sh. It:

Sources $ORB_VENV_PATH/bin/activate if ORB_VENV_PATH is set.
Locates the orb command (installed package or local dev mode).
Passes all HF arguments through verbatim to orb.
Appends stdout/stderr to $HF_LOGDIR/scripts.log.

Minimal HF provider config example¶

export ORB_CONFIG_DIR=/opt/myapp/orb/config
export ORB_WORK_DIR=/opt/myapp/orb/work
export ORB_LOG_DIR=/opt/myapp/orb/logs
export ORB_VENV_PATH=/opt/myapp/.venv

Then in your HostFactory hostProviders config:

{
  "name": "orb-provider",
  "providerCommandPath": "/opt/myapp/orb/scripts",
  "providerCommand": "invoke_provider.sh"
}