Different Architectures for Clusters

Different Architectures for Clusters

Different Architectures

Clusters follow general guidelines for different applications. 

Database Replication - MySQL database replication cluster as a backend for a website.
Batch Processing - When many tasks need scheduling.
Render Farm - Distributed rendering cluster configuration.
Software Development Architecture - Compile Farm - Clusters for software development.
Message Passing Architectures - Essentially Supercomputers for true high performance applications.

Database Replication Clusters

Database replication is a necessary and useful application of clusters. Many databases are read intensive with many more read requests made than write requests. By replicating the data across a community of nodes it is possible to scale the number of read requests which can be dealt with per second in a linear fashion.

A website has large amounts of content all stored in a MySQL database. The web servers (which are also probably clustered), make read requests from the replication nodes through a load balancer. Write requests are sent to the master node.

Multi master configurations are also common. In situations with high levels of writes it is necessary to be creative with the architecture of the database to allow replication between masters or partition the database such that there are essentially two separate databases for different queries. For example one database for searches and another for user data.

Batch Processing

Batch processing systems are key in the banking industry. Good scheduling and rapid response are important if for example we are not to be kept waiting at the cash point as our bank checks we have the money in our account we just asked for.

Sometimes referred to as compute farm, the key part of batch processing systems is maximising up time and maintaining performance at peak load while minimising cost levels.

In this sort of situation it can be wise to save money by reducing capcity during low demand by shutting down some nodes and bringing them back up when demand will be high. To maximise efficiency a intelligent Workload Management System (WMS) should be implemented.

Render Farms

Render farms are a special form of batch processing clusters, with less of an emphasis on responsiveness - most of the processing jobs will take more than a minute. Low cost hardware and the quantity of available processing power is most important. Rendering is used in the visual effects, computer modelling and CGI industries and refers to the process of creating an image from what are essentially mathematical formulae. Rendering engines provide numerous different features, which in combination can produce a scene with the desired effects.

Where is Rendering Found

§  Computer Aided Design (CAD), in engineering and design.

§  Visual Effects (VFX), for film, television and advertising.

§  Architectural Rendering, for visualising buildings, rooms and landscapes.

One of the most intensive aspects of these processes is in producing photo realistic scenes and animations. This is where clusters come into their own.