Connect & Share

Menu

Archives

March 2010
M T W T F S S
« Feb    
1234567
891011121314
15161718192021
22232425262728
293031  


Blog Archives

Clustered Shared Hosting Demo available at downlods.info

Thursday, December 24th, 2009

Hello People,

We have had an enormous response to our services ever since we have launched them commercially. We appreciate all your queries and we will always try our level best to satisfy them all. We have noticed a common trend with most of the clients from whom we received information or quote requests. They asked us for example sites where in the round robin DNS has been implemented for them to review the speed, functionality and benefits of these services. We have now setup a demo site ourselves which shows the entire range of features we can offer with this service. The demo site’s URL is http://www.downlods.info and we will be explain more about the unique setup step by step below:

1. The domain hosts a wordpress site which has almost 1700+ random articles from different RSS feeds found on the web. We are still populating the site with more and more content and we intend to put in a few thousand pages/posts on it so that we can show how a resource intensive can work on the clusters. The first thing you should notice is that the site uses round robin DNS and so it has two A records.

[root@server ~]# host downlods.info
downlods.info has address 69.10.60.149
downlods.info has address 69.10.48.100

Thus the domain runs from two different IPs to balance out the traffic on the domain using round robin DNS configuration.

2. These two IPs run lighttpd installations with PHP configured on FastCGI mode. This can be seen as below:

[root@server ~]# HEAD 69.10.60.149
200 OK
Connection: close
Date: Thu, 24 Dec 2009 09:29:42 GMT
Server: lighttpd/1.4.23
Content-Type: text/html
Client-Date: Thu, 24 Dec 2009 09:27:25 GMT
Client-Peer: 69.10.60.149:80
Client-Response-Num: 1
X-Powered-By: PHP/5.2.10
 
[root@server ~]# HEAD 69.10.48.100
200 OK
Connection: close
Date: Thu, 24 Dec 2009 09:29:46 GMT
Server: lighttpd/1.4.23
Content-Type: text/html
Client-Date: Thu, 24 Dec 2009 09:27:29 GMT
Client-Peer: 69.10.48.100:80
Client-Response-Num: 1
X-Powered-By: PHP/5.2.10

Lighttpd has been custom tweaked on these servers to handle scripts like wordpress and acts as a proxy to pass http requests to much lighter Nginx instances. You can notice the actual site is served from Nginx as below :

[root@server ~]# HEAD http://www.downlods.info
200 OK
Connection: close
Date: Thu, 24 Dec 2009 09:28:25 GMT
Server: nginx/0.8.7
Content-Type: text/html; charset=UTF-8
Client-Date: Thu, 24 Dec 2009 09:26:09 GMT
Client-Peer: 69.10.48.100:80
Client-Response-Num: 1
Set-Cookie: wassup=MDE5Y2Y5N2QzYjNkMjRiNWZlYTM1NDg5ZDVmNDBlMzQ6OjEyNjE2NDk2MDU6Ojo6NjguMTY4LjIwOC4xMDo6c2VydmVyLmRvd25sb2RzLmluZm8%253D; expires=Thu, 24-Dec-2009 10:18:25 GMT; path=/
Set-Cookie: wpgb_visit_last_php-default=1261646905; expires=Fri, 24-Dec-2010 09:28:25 GMT; path=/
X-Pingback: http://www.downlods.info/xmlrpc.php
X-Powered-By: PHP/5.2.10

Traffic is distributed using round robin balancing which can be seen as per the logs below:

2009-12-24 04:33:36: (mod_proxy.c.1105) proxy - start
2009-12-24 04:33:36: (mod_proxy.c.1143) proxy - ext found
2009-12-24 04:33:36: (mod_proxy.c.1214) proxy - used round-robin balancing
2009-12-24 04:33:36: (mod_proxy.c.1277) proxy - found a host 66.45.229.83 80
2009-12-24 04:33:36: (mod_proxy.c.392) connect delayed: 15
2009-12-24 04:33:36: (mod_proxy.c.1017) proxy: fdevent-out 1
2009-12-24 04:33:36: (mod_proxy.c.828) proxy - connect - delayed success
2009-12-24 04:33:36: (mod_proxy.c.1105) proxy - start
2009-12-24 04:33:36: (mod_proxy.c.1143) proxy - ext found
2009-12-24 04:33:36: (mod_proxy.c.1214) proxy - used round-robin balancing
2009-12-24 04:33:36: (mod_proxy.c.1277) proxy - found a host 69.10.60.149 8080
2009-12-24 04:33:36: (mod_proxy.c.392) connect delayed: 17
2009-12-24 04:33:36: (mod_proxy.c.1017) proxy: fdevent-out 1
2009-12-24 04:33:36: (mod_proxy.c.828) proxy - connect - delayed success

PHP is configured to run in FastCGI mode across the clustered setup and thus gives amazing speed to the blog site. You can use various third party tools to confirm the site speed even when the blog has more than 1700 posts and runs numerous ads as well. Here are some of the readily available tests we have already done.

The various results can be found at the links below:

Pingdom Results :

Website information

Total loading time: 4.9 seconds
Total objects: 29 (373.9 KB)
External objects: 5 (120.8 KB)
HTML (X)HTML: 1 (41.6KB)
RSS RSS/XML: 0
CSS CSS: 6 (19.3KB)
Scripts Scripts: 4 (144KB)
Images Images: 18 (169KB)
Plugins Plugins: 0
Other Other: 0
Redirected Redirected: 0
SiteUptime Results:

Quick check for: www.downlods.info (http)

Date City Country Time (s) Result
December 24, 2009 10:59:18 New York United States 0.184 Ok
December 24, 2009 10:59:19 London United Kingdom 0.190 Ok
December 24, 2009 10:59:20 Hong Kong China 0.460 Ok

Average Response Time: 0.278 seconds

HyperSpin Results:

Location
Status
Time
Traceroute
1. Seattle, Washington, USA (WA)
OK
 2346ms do traceroute
2. San Jose, California, USA (CA)
OK
 1274ms do traceroute
3. Dallas, Texas, USA (TX)
OK
 834ms do traceroute
4. Ashburn, Virginia, USA (VA) Test temporarily unvailable
5. Chicago, Illinois, USA (IL) Test temporarily unvailable
6. Montreal, Quebec, Canada (QC)
OK
 1680ms do traceroute
7. Maidenhead, Berkshire, UK (UK)
OK
 3160ms do traceroute
8. Amsterdam, Netherlands (NL)
OK
 2728ms do traceroute
9. Madrid, Spain (ES)
OK
 3788ms do traceroute
10. Milan, Italy (IT)
OK
 4692ms do traceroute
11. Frankfurt, Hesse, Germany (DE)
OK
 4144ms do traceroute
12. Singapore (SG)
OK
 4973ms do traceroute
13. Zhongzheng, Taipei, Taiwan (TP) Test temporarily unvailable
14. Neihu, Taipei, Taiwan (NH) Test temporarily unvailable
15. Brisbane, Queensland, Australia (AU)
OK
 6406ms do traceroute

The domains we host on our shared clustered servers as well as on our custom built dedicated clustered have similar configuration and speed like this test domain. They infact have higher performance as each cluster is deployed with tweaks needed from a pre-hosting professional analysis of the sites we do. If you have further questions with the site setups on clusters, please feel free to contact us for clarification.

Tags: , , , , , , , , ,
Posted in Case Studies | No Comments »

Cluster Setup for an Autosurf Website

Thursday, August 21st, 2008

Over the years, mutual traffic exchange programs have been gaining popularity as a medium to promote websites. Most of these programs/websites use autosurf technogies to generate hits for their advitiser or member’s websites. Various HYIP and Paid traffic exchange programs using autosurf scripts as their base exist on the internet. The cluster setup we did was for a site which was purely designed for mutual traffic exchange with no paid services. The member sites needed to add banners and ads from the parent website in lue of the traffic exchanged via the site.

1480_420_280_crop_548af.jpg

This site gained a huge membership base in just a few months and had traffic which could not be manged by a single dedicated server. The site involved an autosurf bar which checked for the banner/ads existence on the member site and rotated it amongst the members every few minutes. The amount of hits the member’s website would receive is dependent on the amount of sites he would surf via his own autosurf area.The cluster we setup had the structure as below:

autosurf.jpg

WebServers: The webservers were the most busy nodes of this cluster. Their site was all about generating http traffic and their members were on the site almost all the time. The autosurf structure made sure that no member was idle and so the http hits were continous and enormous in number. We had to use 3 webservers to be able to manage the amount of traffic this site was getting. Each of the webserver nodes was setup with Lighttpd + PHP as Fastcgi for faster site performance and dynamic page generation.

The configuration for the webservers we used was:

  • Intel Pentium Dual Xeon 2.8 Processor
  • 2GB DDR2 RAM / 160GB SATA HDD Drive
  • CentOS 4.6
  • 1000Mbps Uplink
  • Private LAN Connections to other Nodes on the Cluster

Master Mysql Server: All the three webservers connected internally via private switch to the master Mysql server. All the site’s membership data and hits stats were being stored in the database and database traffic was huge as well as each hit was being logged into mysql. The client needed managed mysql backups and redundancy and so mysql replication was used for the cluster. Due to the amount of mysql activity on the master mysql server, generation of backups from this node was almost impossible. We used replication to sync the db data on the slave server and used the slave server for generating mysql backups. The configuration we used for the Mysql Server is as below:

  • Intel Pentium Quad Core Q6600 Processor
  • 8GB DDR2 RAM/ 1×300GB SCSI 10k RPM Drive
  • CentOS 4.6
  • 100Mbps Uplink
  • Private LAN Connections to other Nodes on the Cluster

The slave Mysql server also acts as the mail server for the setup. The mailing activity for the site is moderate and so only one mail server is being used.

  • Intel Pentium Quad Core Q6600 Processor
  • 4GB DDR2 RAM/ 1×300GB SCSI 10k RPM Drive
  • CentOS 4.6
  • 100Mbps Uplink
  • Private LAN Connections to other Nodes on the Cluster

This setup distributes the traffic for the site to the three nodes using round robin DNS which is managed by the TTL. Comments are welcome as always.

Tags:
Posted in Case Studies | No Comments »

Corporate Clustered Mail Server Setup

Wednesday, August 13th, 2008

Organizations with huge staff or even medium sized organizations with heavy mailing activity usually use a clustered setup for handling their mails. Proper internal and client Communication is the core need for any such company and so mail server uptime is of utmost importance for them. I am going to present a case study of the architecture of an actual mail server cluster setup we designed for one of our clients.

Environment:

  • The organization is a medium enterprise with work area related to Engineering drawing and CAD designing.
  • The employee strength is around 250.
  • Their mailing activity is heavy as the huge design files are being passed along internally within the organization and even to their end clients mostly using mails and heavy attachments.
  • The staff is located in two different countries with almost the same strength at each office.

Our Task and Resources:

Our primary task to build a robust and fail over enabled cluster for their mailing activity. The volume of mails being sent via this cluster was substantial due to the fact that mails were the most important aspect of their daily chores. We were approved to use a six node cluster for their mails. Even though this was a corporate setup, we had to use open source softwares as the client wanted to keep it cost effective.  We used a qmail, vpopmail etc for their cluster preventing the use of any paid softwares.

hosting.jpg

The Setup:

As we were allowed to use six nodes for the cluster, we decided to use two nodes for outbound mails ,two nodes for inbound mails and one node for a pop3/imap connector and one as a mail gateway.

The hardware configurations for all these nodes was as below:

  • Intel Pentium Dual Xeon 2.8Ghz
  • 2GB DDR2 RAM, 2×500GB SATA Drives
  • Fedora Linux Base OS, 100Mbps Uplink on each node

The setup flow was as below:

mail_cluster.jpg

The Actual Functionality:

As you can see above, the domain was setup to use seperate nodes to send and receive the mails. A brief explanation of how it all worked together and each node’s function is below:

MX Records: The domain had two MX records. One was the primary MX which accepted the mails for the domain and prepared them to be fetched by the pop3 connector. The second backup MX record’s node was a failover to the primary MX and stored the inbound mails while the primary MX was down.

SMTP Servers: The company had a very high volume of outbound mails and so two smtp servers needed to be used. As they had two offices, each of them got a dedicated SMTP server to use.

Mail Gateway: Both the SMTP servers were connected to a single Qmail based mail gateway. The mail gateway provided functions like archiving mails to monitor employee activity, filtering content and proper monitoring of the outbound mail stats for the entire company.

Pop3 connector: The pop3 connector retrieved and redistributed the mails received by the primary MX server for the domain to the user mailboxes which were made available via webmail or regular POP3 as well as IMAP based mail clients to the employees.

This particular mail setup did not require use of round robin DNS as failover functionality was more important than distribution of traffic. This type of setup also gives us the scope of expansion if there is a surge in the mail activity for the client. Another SMTP server for outbound mails or another node to act as the primary mail server can always be added due to the architecture used. I hope you find this case study useful for deploying mail server clusters. Comments are always welcome :)

Tags: , ,
Posted in Case Studies | No Comments »