ATLAS e-News

The Inner Detector Evaporative Heater Problems

7 April 2008

Valery Akhnazarov (left) and Kirill Egorov, two of the many people who have contributed to the repair of the heater system, standing in front of the far heater areas.

Over the past 18 months the heaters used in the evaporative cooling system have caused the Inner Detector community countless sleepless nights. The strange thing is that these heaters are very simple in concept construction, and should be easy to operate. If only it had been that easy!

The function of the heater is easy enough to understand. As the fluid used to cool the silicon sensors leaves the Inner Detector, it is very cold (-25°C). Cold enough, in fact, to form ice on the exhaust pipes as they go into the ATLAS cavern. That could potentially cause  a lot of damage, especially to the sensitive electronics. So before the cooling fluid  goes into the cavern, we put a small coiled heating element into the pipe to heat up and boil the fluid. This is quite similar to what is put inside an electric kettle and has about the same power, but in our case we have more than 200 of them inside specialized stainless steel tubes about 70 cm long.

The first real problems were found in July 2006. At the time,  we only had four days to operate just a few cooling circuits of the evaporative cooling system inside the empty cryostat before the end-caps were closed and the magnetic field mapping started. The last set of cables we needed to start the tests was installed on the Monday and we ran our tests from Tuesday afternoon until the morning of Friday the 14th of July. As the Barrel SCT had not been installed, we were running with the inlet connected to the exhaust with a heater in between. At the time of the tests, things seemed to go quite well. I went on vacation on the Saturday feeling very pleased with what we had done. But looking at the results later, it was clear that something was not quite right and with hindsight something was very wrong. The next scheduled chance to run the cooling again was October of the same year. What we found was that some of the heaters seemed to need enormous amounts of electrical power to boil just small amounts of fluid.  At first we thought it was a problem of the electronics reading the temperature or computer control, but it turned out to be that, for  a small number of heaters under circumstances that we still do not fully understand, we get cold spots near the temperature sensors that were used to control the heater. No matter how much power we pumped in, we were  unable to raise the temperature near those sensors. We spent the rest of that year and the first few months of the next fixing this and other associated problems that had appeared by moving the sensors down-stream of the heater in a way that was never foreseen in the original layout of the experiment.

The next time we got a chance to operate the cooling system was with the SCT barrel in February 2007. Part way through some operational tests, one of the heaters developed a short circuit to ground. One of the connectors that take the power to the heater had failed in a spectacular way causing a small explosion. During our problems in October of the previous year, we had noticed that on some heaters the degree to which the heating element is electrically separated from electrical ground (the  isolation resistance) was not as good as it should have been. At the time, we had talked to the supplier and agreed on a way to recover a good value of the isolation resistance that we knew was due to the presence of very small amounts of water in the connector and cable. The heater that had failed had been one with a very bad isolation resistance value that had been repaired but it still failed. At first we thought it might be our handling of the heaters that caused problems but when we checked the heaters that were unused (and still in their delivery box), we found that some of them had the same problems. Within a few days our attention was focused on one particular geometrical region of the connector that had failed. We could tell where it was going wrong. We were concerned about some of the materials used in the construction and we were beginning to piece together how the failure could happen. Working with the supplier, we devised a repair scenario for the connector which would prevent moisture from getting into the connector. We dried out the heaters and retrofitted about 500 sleeves over the connectors and prepared to re-install. 

Unfortunately after a few weeks the connectors failed again just before the ID end-caps were due to be installed. This revealed another much bigger headache as confidence in the heater was at an all time low. If we had proceeded with the installation of the ID EndCaps as originally foreseen we would have lost  access to the Barrel Heaters for 10 years and would have had no chance to repair them. In addition if we had installed the EndCap heaters in their original design location, we would have lost access to them when finalizing  the pixel installation. A decision was taken to move the location of nearly 120 heaters and their associated services to the end flange of the calorimeter, an area with very little space. So now we were fighting two battles: First we needed to find a solution to the heater connector problem with the supplier and second, we needed to re-engineer new pipe work in an already overcrowded area at the end of the detector. Between July and December 2007 both of these tasks were achieved with an enormous amount of effort by a lot of people inside and outside of the Inner Detector community. The design of the connectors was changed and new exhaust circuits were installed to connect and reuse the old ones inside the detector. Since then we have signed off the SCT Barrels and End-Caps with the new heaters and run all of the pixel circuits in test mode accumulating nearly 12,000 hrs of heater operation all without problems. 

In the mean time we have also started to look at new ways to solve this problem in case it comes back again in another form. One is a new kind of heater that is not a coiled wire but a printed circuit placed in the flow. The other is a passive system that uses a hot liquid to heat up the cold exhaust liquid from the detector. Work on these will start when the installation of the Inner Detector is finished in a few weeks. Let’s hope we never need them.

Stephen Mcmahon Rutherford Appleton Laboratory