Inside Indiana University's $40 million Data Center
If Bloomington gets hit by a nuclear strike, Oncourse will still be running after the fallout clears. At least, that was the impression I got when I toured the IU Data Center earlier this week. Located on the UITS campus at 10th St and the 45/46 bypass, IU's IT infrastructure hub is located in a beige, single-story facility with only a single set of windows on the whole building. The subdued appearance of the building hides the fact that it has 18-inch thick reinforced concrete walls and can withstand a hit from an F5 tornado. And once you go past the card-access front door, there's some very impressive IT and network engineering to be seen inside.
I was honestly really excited for my tour when I received an invite about a week ago. I'm going out on a limb here, but I'm guessing that being excited about a brand new data center is a phenomenon strictly limited to IT professionals. Seeing endless rows of server racks, perfectly routed bundles of fiber optic cables, and all the high performance electrical and plumbing work that goes into keeping the cogs of the Information Age spinning is just damn impressive, especially when you understand the work that goes into keeping it all online.
So that's why I took a few pictures, asked a few questions, and put together an article highlighting this 90,000 sq ft facility that handles IU's Netflix traffic, serves up their free software downloads, and does a few other things too.
Click on any of the pictures below to view them full-size:
<img class="size-large wp-image-449" alt="Here's one of the neat data visualization tools they had in the main lobby, showing the Internet2 academic fiber backbone. The colors represent percentages of each "pipe" that is being used. There's also a weather overlay displayed." src="http://mattvukas.com/wp-content/uploads/2014/02/3-1024x782.jpg" width="625" height="477" />
Here's one of the neat data visualization tools they had in the main lobby, showing the Internet2 academic fiber backbone. The colors represent percentages of each "pipe" that is being used. There's also a weather overlay displayed.
Here is a row of servers provided by clients which are colocated to the Data Center. There servers contain sensitive data, so they can only be accessed by the client using the RFID locks on each rack (the red lights).
In the back of the Data Center are two 16-cylinder, 2,200HP Cummins diesel generators. The Data Center has about 10,000 gallons of diesel fuel on hand at all times, meaning that they can run critical systems off of a generator for 7 days without refueling.
Here's a look into the flywheel room. The flywheels in these boxes spin 24x7 whenever the main power is on. If the power suddenly gets cut, they'll continue to spin a generator, providing power to the servers for about 20 seconds. This gives the generators time to start up, since they take about 9 seconds to get going.
This room contains a lot of lead acid backup batteries, as you can see. While the critical systems would switch to the generators in a time of emergency, non-critical servers would switch over to these. This then gives the IT staff time to save everything and shut things down properly.
Here's where the magic behind that 100Mbps IU Internet happens. The yellow wires you see there are fiber optic cables. Each pair of cables can transmit about 10Gbps of data, and it's all plugged in to some very expensive network equipment. A few of these wires run all the way from Bloomington to Indianapolis, through which lasers transmit data at distances of 50+ miles.
It was a lot of fun seeing all this expensive hardware up close, and it makes you appreciate all the amazing, 21st-century engineering that goes into making everyday technology work. Hopefully you enjoyed this virtual tour as much as I enjoyed the real thing.
Level 3 Communications and the perpetual game of ISP "Chicken"
Breaking - Mt. Gox is dead, long live Mt. Gox