The ATLAS data acquisition system

13 October 2008


The first five minutes of continuous beam through ATLAS. The measured rate of about 58 Hz is driven by the trigger of the beam pick-up detectors firing at every LHC turn and pre-scaled by a factor of 200.



If ATLAS in P1 was compared to a country with villages called Tile, Pixel or TGC and large cities called SFOs (Sub-Farm Outputs), then the Data AcQuisition (DAQ) could be compared to the national railway system (well… except that data in ATLAS travel mostly in one direction…). 

It has taken over 15 years to build it. One of the first and biggest challenges was the correct dimensioning of the infrastructure in order to allow citizens (the data) to travel around, taking into account costs as well as the estimates of population evolution, and leaving enough headroom to absorb traffic peaks due to seasonal effects or high interest events. Another interesting problem to solve was how to layout the rail network in order to allow the peripheral areas, our villages, to be connected via hub-like train stations (ever heard of the Read Out System or ROS?) to the main rail lines (the large Data Collection switches for the Level-2 trigger and the Event Building).

Different designs were prototyped and tested and, while some groups were concentrating on this aspect, other experts studied the best trains to use so as to offer the best service to our citizens, the event data (in the TDAQ: which network layer and which communication protocols?). It was also apparent from the beginning that it was essential to put substantial effort into tools to control the traffic (the Run Control), setup the correct timetables (the Configuration database) and monitor what was happening around the country (the Monitoring and Diagnostic systems). Also this part required quite some thinking, exploring, trying and restarting from scratch…

When in 2006 the first villages, Tile and later LAr, were becoming lively and started being populated by globe-trotters (cosmic, calibration data, etc), the first parts of our railway system was made operational. The ROS stations were installed and opened to the public, and a few of the backbone rail lines were activated.

Towards the end of 2006, we started having trial weeks where brave passengers from more and more ATLAS towns could test out our achievements and provide feedback. We had 8 such weeks, called Milestone weeks, interleaved with so called Technical Runs, in which we loaded the railway system with empty trains and ensured that not only the connectivity was correct, but also that the high speed trains could circulate safely.

By the summer of 2008, we had transported travelers from all villages, including the little Lucid and BCM towns. The tourist’s flow to the main cities was quite substantial and we realized that we had to provide accommodation facilities to international voyagers, since (as the Swiss very well know) connections to neighboring countries, such as Tier0, might be interrupted at times, due to strikes or technical problems.

We also put in place a number of redundant routes in order to guarantee that, once a passenger had joined the main railway system via one of the ROS stations, he had a very good chance of reaching his destination, irrespective of locally flooded areas in the HLT (High Level Trigger) or EB (Event Builder) regions. Finally, we put in place something called “stop-less recovery”: the aim of it was to ensure that even in case of earthquakes in some parts of the country the rest of the railway system could continue transporting passengers with minimum disruption.

On the 10th of September, we were ready to greet the first delegations of VIP citizens on our trains. It was a day of particular stress but also great satisfaction. And when our main press agencies (Atlantis, VP1) showed the first pictures of happy customers (the splash events) the relief was incredible! Nevertheless, we did not allow the attention level to decrease and one day later our monitoring tools were regularly counting travelers and could spot the first five minutes of continuous traffic induced by the LHC shown on the figure at the top.

The design and construction of our railway system has been quite a challenge and definitely a lot of fun. There are still plenty of aspects to tune and improve but we have already demonstrated that we are capable of welcoming all types of voyagers, large (13 MBytes sometimes…) or small, VIP or common, and that we are ready to do so seven days a week, day and night.

Giovanna Lehmann Miotto

CERN