Abstract

A method to create heterogeneous software build farm(environment) that reduces the build time, load on Configuration Management system, using less CM licenses, and all platform builds would be made available at almost the same time due to parallel compilation on all platforms


Problem Statement

Now-a-days most of the enterprise products support more than single OS/HW platforms. Therefore, the product builds needs to happen on various build machines (with different operating systems, hardware, compilers etc). Typically the build process starts with labeling the sources, creating a build workspace, compilation, collection of binaries, packaging and finally pushing the build packages to a user availability location. This cycle is repetitive until the product founds to be acceptable to reach the customers.

 

Imagine the level of complexity if the product supports about 20 different platforms and doing this on daily basis.

 

Solution

This white paper aims to address a method which would simplify the above build related steps, by virtue of it, it would save time, load on CM system, CM licensing costs, builds on unsupported CM platforms, parallel builds on multiple platforms, fault-tolerance etc.,

This technique can also be applied for other automations.

 

Below picture depicts a central build machine connected through NFS with various other machines in the build farm. This machine must have the CM client installed, so that the source labeling can be done.

 

The build workspace is prepared on the central build machine and the same is disk is mounted to other build machines in the build farm through NFS. Build is triggered from the central build machine and the compilation would happen on the other machines with their native compilers. This is achieved by using remsh or SSH. These two utilities have the capability to invoke the necessary commands in the remote machines that are connected with NFS and NIS.

 

The binaries built are collected in the same shared disk. Therefore, all the produced binaries are available in a singe disk; this will enable to take the back up, to take care of fault-tolerance etc. Due this framework, the build workspace is created only once, thus causes less load on CM systems. All the binaries are available in one disk so that the backup is easy.

 

 

build farm<!

Implementation

 

Prepare central build machine:

      • A decently powerful machine with enough RAM
      • A hard disk with enough memory depending on the product size
      • Install CM client (to create build workspace)
      • Build user account (managing through NIS)
      • NFS client configuration
      • Remsh, SSH and other utilities like make, Perl etc

 

Configure NIS server:

>NIS would enable the $HOME directory access from NIS server. Therefore, any user can access their $HOME directories from any of the build systems, provided that login is registered in NIS server

>Now make all build systems as NIS clients so that build user can access $HOME directory from all build systems.

 

Configure NFS server:

>NFS would enable the sharing of disk space and build workspace with all build systems. Therefore, all the built binaries are collected back in to the same disk.

 

Mount the build workspace on the build machines:

>Make an entry in /etc/exports (in central build machine) for all build machines.

>Mount the build workspace on each of the build machine. This is one time activity. However, it’s recommended to place a mount command in .profile for each build machine.

 

NFS client configuration (in main build machine):

>Build user home – the directory which users can access from all build systems.

>View – shared build work space.

>Build staging – where all binaries are collected after compilation.

 

Define build user $HOME:

>Platforms file – Contains mapped host names to all build systems.

>Configuration file. The example of Platforms file is shown below:

 

Prerequisites for Other Build Systems:

      • Hardware
      • System with latest patches and compiler
      • CM client if possible
      • Build user account
      • Perl
      • Enabling remsh access from view server

 

Advantages

  • Saves time as compared to manual compilation on each platform.
  • Reduces complexity and the build setup is simple to use. So that the builds can be triggered from a single machine.
  • A single location to back-up to implement fault-tolerance.
  • Reduces the cost of CM licenses and the load on CM server