Case Study: Resolving a BMS Alert
Overview
in Fall 2021, a Content Delivery Provider faced issues with PDU equipment failures at one of their data centers housed within a colocation facility. Despite the colocation provider's Building Management System (BMS) indicating normal conditions, technicians on-site reported excessive heat. To address these discrepancies and pinpoint the root cause, the client used Purkay Lab’s Rental to perform an independent thermal survey. The results from AUDIT-BUDDY revealed significant temperature variances, confirming the technician's observations and prompting necessary corrective measures.
The Project
The facility is a contained space with 20 cabinets with 5kW load each (100kW Total IT load). Cold air is supplied via the ceiling from three roof mounted units (N+1) and return ducts in the hot aisle. A ceiling partition separated the hot and cold aisles, and blanking plates were installed on every rack to prevent air mixing. Power is fed from the A & B busway overhead in the hot aisle. The BMS room sensors were placed on the cold aisle wall about 48” from the front of the racks. The Client used the AUDIT-BUDDY to take 1-minute readings across 20 racks in the hot and cold aisle.
The Results
Measurement #1 - The Baseline
The live reporting showed (Figure 1 AUDIT-MATE Dashboard ) that 8 racks had inlet temperatures above the SLA limit of 80.6°F / 27 °C. This confirmed that the BMS reports were incorrect.
Figure 1 AUDIT-MATE Dashboard
Measurement #2 - Trending
Using this initial data, The Colocation Provider launched an investigation. First, they checked the HVAC set points but found no anomalies. Then, they checked the BMS and found that an offset value for the deployed sensor was incorrect. This caused the reported temperatures to be different from the actual temperature of the air delivered to the room, causing the server inlet temperatures to reach critical levels.
After the Colocation Provider corrected the offset values, the Client placed AUDIT-BUDDY next to the BMS sensors to confirm the reported temperatures were now correct. The Client collected information for 1 hour, as seen in Figure 2 below.
Figure 2: Time-Trend Plot
Measurement #3- Before and After View
Once the room stabilized, the Client repeated the initial 1 minute per rack data collection and found that the hot spots had been eliminated. The two before and after Static Heat Maps (Figure 3 and Figure 4) show a drastic reduction in room temperatures. The change in temperature allowed the Client and the Provider to avoid the risk of equipment failure and downtime.
When you compare the static heat maps, before and after the intervention, demonstrated a drastic reduction in room temperatures, ensuring the facility's cooling system was now operating within safe limits and preventing potential equipment failure and downtime.
The resolution of this issue not only ensured the physical safety of the IT equipment but also enhanced operational reliability and efficiency. The conclusive data from Purkay Labs also supported a decision by the client’s senior management to purchase an AUDIT-BUDDY system for ongoing benchmarking and validation within their colocation facilities.
Summary
This case study illustrates the critical role of independent temperature monitoring in validating and calibrating BMS readings in data center environments. Purkay Labs' AUDIT-BUDDY system proved instrumental in identifying and resolving significant discrepancies in reported and actual temperature conditions, thus safeguarding equipment and ensuring compliance with SLAs. The intervention not only rectified immediate cooling issues but also provided a robust framework for future temperature management and dispute resolution in the colocation space.
About Purkay Labs
Purkay Labs specializes in portable, easy-to-deploy environmental monitoring systems that provide reliable and independent data. Our products are designed to support data center operators in managing airflow, reducing emissions, and optimizing cooling efficiency without the need for complex installations.
You can follow us @purkaylabs or visit our website: www.purkaylabs.com
About the Thermal Survey Service
The Thermal Survey Service from Purkay Labs offers a comprehensive assessment of environmental conditions within data centers, employing our AUDIT-BUDDY systems to capture precise temperature, humidity, dew point, and delta-T measurements across multiple locations. Our service ensures minimal disruption while delivering data crucial for operational management and decision-making.