Home
Join

15 Replies

  • Good questions.

    I can tell you that when we built out our data center, it came down to the fact that the CRAC tonnage available sizing made it clear that we didnt need to measure too carefully, because there was no perfect CRAC for just your load. You get the bigger one, and thn we were N+1, or maybe N+2, or N+3 if we open up the doors and use fans.

    Most of our draw is using 208V from 3 phase UPS. When you increase the voltage, you decrease the amps and increase the efficiency of the power supplies, so you generate less waste heat. We also have 110V for devices that doen't accept the higher voltage.

    I am guessing that perhaps you are looking at 10 racks, each with 2 circuits?

    We went with Starline Track Busway, which allows you to deliver 208V and 110V without having to reconfigure circuits at the electrical panel. Not that we have ever had to change it since we went in 6 years ago.

    https://www.starlinepower.com/busway/

    It looks like you have enough load to justify 2 large UPS to power the entire room.

    I use two different color power cords so we know that each device is plugged into 2 different PDU, and that each PDU is fed by a separate UPS. All cords on a PDU should be a single color. Each device has 2 different color cords. Very easy to see visually if things are wrong.

    Spice (4) flagReport
    1 found this helpful thumb_up thumb_down
  • The largest draw for electronic equipment (servers, switches, SANs & routers etc.) is on powerup.  So that's why the equipment is rated at one rating and then generally uses much less.  For question 1, you need service for 40KW.

    Was this post helpful? thumb_up thumb_down
  • You can go with a statistical approach because not all units will be taking their maximum power at the same time. This  is root of the mean square calculation (RMS). For example if you have 3 units rated at 400W, 1200W and 100W then instead of adding up all the power loads to 1700W you would add up 160000, 1440000 and 10000 to 1610000 and then take the square root of that which is 1269W. That answer is less than the 1700W. Realistically you could take a percentage of the of the power rating and then do the RMS calculation such as 80% load and then the power rating is 1015W. 

    Of course with your measurements you are seeing a power load of much less than what you could get with my calculation. If you monitor the 10 kW load to see what peak it reaches then you can add 50% and you should be fine.

    Was this post helpful? thumb_up thumb_down
  • With your question on 3 phase. If you have only one phase of the 120V then the voltage from a line to neutral (about earth potential) is that 120V. However, if you have 3 phase then the voltage between any two of the 3 phase lines is 1.732 (square root of 3) times the 120 which is 208V.  

    Was this post helpful? thumb_up thumb_down
  • I recall this being answered here years ago.

    0 of 1 found this helpful thumb_up thumb_down
  • And BiscuitKing​ had a few general loads listed here:

    https://community.spiceworks.com/topic/1980560-trying-to-approximate-heat-load-for-a-server-room

    Was this post helpful? thumb_up thumb_down
  • kevinmhsieh wrote:

    Good questions.

    ...

    Most of our draw is using 208V from 3 phase UPS. When you increase the voltage, you decrease the amps and increase the efficiency of the power supplies, so you generate less waste heat. We also have 110V for devices that doen't accept the higher voltage.

    ...

    Don't worry too much about amps because power is measured in watts.  The units draw whatever watts they need, regardless of what voltage they are set to use.  You take average wattage from the product cut sheets.  And peak wattage is 2x average wattage, in case you need to know.

    Was this post helpful? thumb_up thumb_down
  • WhenInDoubtRestart wrote:

    I'm planning a new, large server room for a new building we hope to move into in a year. We had a rather unfruitful meeting with the electrical & mechanical (HAVC) engineers, in that they were unable to answer a few key questions I need answered to answer theirs.

    The basic 101 rule to IT cooling & electric is, "add up your equipment's power loads." Insert record scratch sound here. I've done that... and as computers are wont to be, the different between the rated loads of their PSUs, and what your equipment actually uses day to day, is drastic

    From my sporadic measurement, if I sample actual draw, it's often a quarter of the total rated power. We're talking 5-figures ramifications for cooling tonnage, more if we need to modify parts of the roof, etc.

    1. If e.g. our equipment's total is 40KW rated but typically draws 10KW, do you order electric service for 40KW, or less? Do you plan cooling for how many KW?

    2. Yes, 1KW = 0.284345 cooling ton, but what does it actually mean? E.g. is there an assumption of 50% HVAC duty cycle for optimal performance? So would 6 tons (50%) be plenty, and for the theoretical moments of 100% draw the HVAC could run continuously? Even if money was no issue (ha), short-cycling your A/C isn't good either.

    And we haven't even gotten into N+1 territory...

    3. Another (US) high-school physics question I couldn't get the electrical engineer to answer -- to supply 40 x 20A 120V circuit breakers, do you need 800A or 400A service? Because 3-phase is 240V, no?

    We do have an engineering consultant we're getting onboard, but doesn't hurt to ask the Spiceheads. Seems like pretty basic questions to me.

    Thanks!

    Re: 1 & 2 there is some nuance around floor space, projected growth, room layout, other heat sources, etc. Some good answers coming in on the topic already that help address it.

    Re: 3, one of the ways that helps me think about it:
    Remember that you can't directly add amps from system A (single phase 120 in this case) to get amps in system B (3 phase 208 for example) for panel sizing. Calculate your loads in kVA/VA and then size your panel to support the maximum loads (plus any additional considerations due to code, future growth, etc). The Mike Holt forums had a decent thread on this topic:
    https://forums.mikeholt.com/threads/questions-on-120v-loads-on-a-120-208-3-phase-panel-with-no-3-pha... 

    Spice (1) flagReport
    Was this post helpful? thumb_up thumb_down
  • One other option is to looking into a consulting company that specializes in data center builds. We we build a North American headquarters the consulting company worked with us to make sure we had the correct CRAC sizes, and UPS sized. By no means was it a large data center, we only had 4 racks. But with the company they corrinated with the rack vendor for the install of the rack and CRAC units, Worked with the electrical contractor to wire everything in, and the fire extinguishing system installers as well. For us it was money well spent. Then they had to work with the electrical contractor again when they wired the CRAC units in behind the UPSs.

    For most general electrical contractors they don't know about wiring in UPSs and then there is A/B power as well. The consulting company was able to supply electrical diagrams for the electrical contractors to install the UPSs.

    So depending on how large/complex this new data center will be and if there is room in the budget then this might be something worth looking into. 

    Was this post helpful? thumb_up thumb_down
  • 3 phase 120 V RMS = 340 volts peak to peak

    If you take any two of the three phases from that 3 phase input, you end up with an instantaneous difference of always...208VAC (This conveniently allows electrical engineers to shift load balance toward whichever pair of phases are loaded somewhat less, since it's often only very large fans and certain large power supplies that require 3 phase input....

    As voltage is now higher, current is now lower (as to compared to a circuit designed for 115 VAC RMS, allowing smaller diameter wires and smaller breakers.

    I know from studying various electrical loading charts/phase distribution loading on airplanes that typically, a particular device will receive a breaker very close to approximately twice what it's *nominal* load is...

    You'd never want to run any office /data-center running at any where near 100% of it's rated capacity...(50% seems a common goal, so, when/if power were applied all at once, the start-up surge could easily be much closer to 30-40% over it's nominal load...

    B Ut...

    Consult with an electrical engineer to confirm these 'postulates', I only know what I've observed in aircraft power distribution/wiring for the past 40 years...

    Was this post helpful? thumb_up thumb_down
  • " Another (US) high-school physics question I couldn't get the electrical engineer to answer -- to supply 40 x 20A 120V circuit breakers, do you need 800A or 400A service? Because 3-phase is 240V, no?"

    Your total of all circuit breakers used does not/will  need to add up to the total supply of voltage/amps 'supplied' in service. (Remember, most devices draw only 40-50% of that for it's 'nominal/typical' power draw, and, each breaker is easily twice above that)

    Was this post helpful? thumb_up thumb_down
  • I always over spec requirements to at least N+2. What you have today wont necessarily be what you have in the future. Allow for server estate growth. Also if possible can you get multiple power circuits into your server room

    Was this post helpful? thumb_up thumb_down
  • Phase and voltage have little to do with each other. Three phase power is more efficient for large loads (industrial heaters, welding, motors, etc) while most other electrical equipment is single phase. But you can have any voltage regardless of single or three phase power. If you don't have three phase gear, you just get single phase installed.

    Loads are calculated in VA (volt-amps) rather than watts because almost all loads are inductive, where voltage and current are not in phase.

    Breakers are fed for loads that don't exceed the maximum breaker rating. I'm going to have less than 20A on each 20A breaker. If the load draws more than that, the breaker trips.

    Typically you'll have sustained load and peak load. A motor might have a high current draw when first started (which can be dealt with by using a motor controller) but much lower sustained load.

    This is why you have an electrical firm that does commercial work help you with the circuit design.

    Was this post helpful? thumb_up thumb_down
  • 1. Best practice is to size it off the sum of max input for in rush current for the entire load on day 1 plus expected growth for day 2.

    2. Similar to power, and also within your given budget, base your cooling capacity on expected future growth and your organization's redundancy requirements or downtime tolerance.

    3. You'll need to size a UPS based on your total load calculation. The UPS will then be either single phase or 3 phase depending on size of the load. If it's a 3phase UPS, any UPS manufacturer can provide upstream circuit breaker and wire sizing information.

    Shoot me a message if you'd like and we'll be happy to have one of our power and cooling experts come on site to provide a free on-site power assessment.

    Spice (1) flagReport
    1 found this helpful thumb_up thumb_down
  • You will want to look at BTU output of your equipment, not kW.  These figures are available at the OEM site and represent cooling load while kW does not all convert to heat and it becomes a matter of energy efficiency vs loss to heat.

    Like kW, however, the BTUs can vary widely.  Obviously every piece of equipment won't be outputting it's maximum BTUs, but you should have upside capacity depending on how densely you plan on running your servers and disk storage.  By densely I mean how many drive bays, memory slots and CPUs are in it.

    12000 BTUs = 1 ton cooling

    For power, 208V and 240V are both based on two-phase power. It's a matter of wiring.  A deeper description can be found in this video:  If you take the phases 120° out of phase you get 208V when you add the sine waves.

    https://www.youtube.com/watch?v=r3hSaiIt8-Y

    Was this post helpful? thumb_up thumb_down

Read these next...