Optimizing Performance: Condor Workflows in Intergraph 2013

Condor Integration with Intergraph 2013: Best Practices and Tips

Integrating Condor (HTCondor) with Intergraph 2013 can significantly improve batch processing, simulation throughput, and workload management for engineering and GIS workloads. This guide covers planning, installation, configuration, performance tuning, security, and common troubleshooting tips to help you deploy a stable, efficient integration.

1. Plan your deployment

• Assess workloads: Inventory the Intergraph modules you’ll run (CAD/CAM tasks, GIS analysis, rendering, batch conversions) and estimate CPU, memory, disk I/O, and GPU needs per job.
• Scale by job type: Use small, dense worker nodes for CPU-bound batch jobs and larger nodes with more memory or GPUs for rendering or memory-intensive analyses.
• Network and storage: Ensure low-latency connectivity between Intergraph application servers, Condor submit nodes, and shared storage. Prefer 10 Gbps or at least gigabit with proper NAS/scale-out filesystem tuning.
• Licensing and compliance: Confirm Intergraph licensing allows distributed/automated execution and plan license servers or token management accordingly.

2. Prepare the environment

• Choose submit and execution hosts: Designate a reliable submit host (often the Intergraph application server or a dedicated gateway) and multiple execution nodes. Keep the submit host highly available if workflows depend on it.
• Shared storage configuration: Use a shared filesystem (NFS, SMB, or a parallel filesystem) for job input/output, ensuring consistent path mounts and permissions across nodes.
• User accounts and permissions: Create consistent usernames/UIDs across nodes or configure Condor’s file transfer to stage files. Limit privileges on execution nodes to reduce risk.
• Time synchronization: Ensure all nodes use NTP to avoid job scheduling and file timestamp issues.

3. Install and configure Condor

• Version compatibility: Use a stable HTCondor release compatible with your OS. Apply vendor-recommended patches and keep Condor updated for security fixes.
• Basic configuration:
  • Set the central manager (CONDOR_HOST) on a reliable server.
  • Configure collectors and negotiators to match cluster size.
  • Define sensible default STARTD and SCHEDD settings for resource advertisements and job limits.
• File transfer vs. shared FS: If using shared storage, disable unnecessary file transfers to reduce overhead. If not, enable Condor’s file transfer mechanisms and test thoroughly.
• Account mapping: If Intergraph jobs run under specific user accounts, map Condor’s execution to those accounts using condor_config settings like ALLOW_READ, UID_DOMAIN, and SCHEDD_controls.

4. Integrate with Intergraph workflows

• Submission method: Use scripted submission of Condor submit files from Intergraph job queues or via a small middleware service. Wrap Intergraph commands in robust shell or Python scripts that capture exit codes and write logs to shared storage.
• Job templates: Create standardized Condor submit templates for common Intergraph job types with resource requests, retry logic, and sensible timeouts.
• License handling: If Intergraph requires floating licenses, add license-check steps