Cluster Setup for an Autosurf Website
Over the years, mutual traffic exchange programs have been gaining popularity as a medium to promote websites. Most of these programs/websites use autosurf technogies to generate hits for their advitiser or member’s websites. Various HYIP and Paid traffic exchange programs using autosurf scripts as their base exist on the internet. The cluster setup we did was for a site which was purely designed for mutual traffic exchange with no paid services. The member sites needed to add banners and ads from the parent website in lue of the traffic exchanged via the site.
This site gained a huge membership base in just a few months and had traffic which could not be manged by a single dedicated server. The site involved an autosurf bar which checked for the banner/ads existence on the member site and rotated it amongst the members every few minutes. The amount of hits the member’s website would receive is dependent on the amount of sites he would surf via his own autosurf area.The cluster we setup had the structure as below:
WebServers: The webservers were the most busy nodes of this cluster. Their site was all about generating http traffic and their members were on the site almost all the time. The autosurf structure made sure that no member was idle and so the http hits were continous and enormous in number. We had to use 3 webservers to be able to manage the amount of traffic this site was getting. Each of the webserver nodes was setup with Lighttpd + PHP as Fastcgi for faster site performance and dynamic page generation.
The configuration for the webservers we used was:
- Intel Pentium Dual Xeon 2.8 Processor
- 2GB DDR2 RAM / 160GB SATA HDD Drive
- CentOS 4.6
- 1000Mbps Uplink
- Private LAN Connections to other Nodes on the Cluster
Master Mysql Server: All the three webservers connected internally via private switch to the master Mysql server. All the site’s membership data and hits stats were being stored in the database and database traffic was huge as well as each hit was being logged into mysql. The client needed managed mysql backups and redundancy and so mysql replication was used for the cluster. Due to the amount of mysql activity on the master mysql server, generation of backups from this node was almost impossible. We used replication to sync the db data on the slave server and used the slave server for generating mysql backups. The configuration we used for the Mysql Server is as below:
- Intel Pentium Quad Core Q6600 Processor
- 8GB DDR2 RAM/ 1×300GB SCSI 10k RPM Drive
- CentOS 4.6
- 100Mbps Uplink
- Private LAN Connections to other Nodes on the Cluster
The slave Mysql server also acts as the mail server for the setup. The mailing activity for the site is moderate and so only one mail server is being used.
- Intel Pentium Quad Core Q6600 Processor
- 4GB DDR2 RAM/ 1×300GB SCSI 10k RPM Drive
- CentOS 4.6
- 100Mbps Uplink
- Private LAN Connections to other Nodes on the Cluster
This setup distributes the traffic for the site to the three nodes using round robin DNS which is managed by the TTL. Comments are welcome as always.
Corporate Clustered Mail Server Setup
Organizations with huge staff or even medium sized organizations with heavy mailing activity usually use a clustered setup for handling their mails. Proper internal and client Communication is the core need for any such company and so mail server uptime is of utmost importance for them. I am going to present a case study of the architecture of an actual mail server cluster setup we designed for one of our clients.
Environment:
- The organization is a medium enterprise with work area related to Engineering drawing and CAD designing.
- The employee strength is around 250.
- They mailing activity is heavy as the huge design files are being passed along internally within the organization and even to their end clients mostly using mails and heavy attachments.
- The staff is located in two different countries with almost the same strength at each office.
Our Task and Resources:
Our primary task to build a robust and fail over enabled cluster for their mailing activity. The volume of mails being sent via this cluster was substantial due to the fact that mails were the most important aspect of their daily chores. We were approved to use a six node cluster for their mails. Even though this was a corporate setup, we had to use open source softwares as the client wanted to keep it cost effective. We used a qmail, vpopmail etc for their cluster preventing the use of any paid softwares.
The Setup:
As we were allowed to use six nodes for the cluster, we decided to use two nodes for outbound mails ,two nodes for inbound mails and one node for a pop3/imap connector and one as a mail gateway.
The hardware configurations for all these nodes was as below:
- Intel Pentium Dual Xeon 2.8Ghz
- 2GB DDR2 RAM, 2×500GB SATA Drives
- Fedora Linux Base OS, 100Mbps Uplink on each node
The setup flow was as below:
The Actual Functionality:
As you can see above, the domain was setup to use seperate nodes to send and receive the mails. A brief explanation of how it all worked together and each node’s function is below:
MX Records: The domain had two MX records. One was the primary MX which accepted the mails for the domain and prepared them to be fetched by the pop3 connector. The second backup MX record’s node was a failover to the primary MX and stored the inbound mails while the primary MX was down.
SMTP Servers: The company had a very high volume of outbound mails and so two smtp servers needed to be used. As they had two offices, each of them got a dedicated SMTP server to use.
Mail Gateway: Both the SMTP servers were connected to a single Qmail based mail gateway. The mail gateway provided functions like archiving mails to monitor employee activity, filtering content and proper monitoring of the outbound mail stats for the entire company.
Pop3 connector: The pop3 connector retrieved and redistributed the mails received by the primary MX server for the domain to the user mailboxes which were made available via webmail or regular POP3 as well as IMAP based mail clients to the employees.
This particular mail setup did not require use of round robin DNS as failover functionality was more important than distribution of traffic. This type of setup also gives us the scope of expansion if there is a surge in the mail activity for the client. Another SMTP server for outbound mails or another node to act as the primary mail server can always be added due to the architecture used. I hope you find this case study useful for deploying mail server clusters. Comments are always welcome 
GeoLocation - Explained
Geolocation refers to the method of identifying the actual Geographical location of a Computer on the internet or a visitor on your site or even the a mobile device. With reference to Hosting Clusters, we mostly trace the geographical location of the website visitors. There can be various procedures used to track down the geographical location of the visitors some of which are:
Internet Protocol Address (IP Address) Lookup:
This is the most common procedure we use on our setups. The visitors are traced on the basis of the whois lookups done to their current IP address. These lookups are automated, real-time and are implemented using special server side modules or even third party lookup services. These lookups can provide the most accurate locations of the visitors as these IP whois databases can always have updates.
Media Access Control Address (MAC Address):
Media Access Control or MAC addresses is an Unique Identifier assigned to most network adapters or Ethernet Cards by their Manufacturer. It contains a combination of the manufacturer’s unique Identification number and some other codes to generate an unique MAC address for the physical device. The location of this device can be traced based on the Sales data for the manufacturer or some other similar methods.
Radio Frequency Identification (RFID):
Radio Frequency Identification or RFID uses Radio waves to transmit data with the help of devices like Transponders. RFID does not have much applications on internet related geolocation but it has wide uses in corporate and enterprise supply chain management systems for inventory tracking and other applications.
Global Positioning System (GPS):
Global Positioning System or GPS uses different Artificial Satellites to interpret the microwave signals obtained from certain GPS enabled mobile devices to calculate the exact geographical location of the device. The device receives this interpreted data which it can use to locate itself and obtain navigation instructions for a particular destination from its current location.
The IP whois lookup method is the most commonly used method for clusters. We will study the exact implementation of this technique in the actual case studies we will be releasing soon.
Translator
By N2H -
-
