Diary of a Shifter

10 November 2008


Shifters at work


7:00 a.m.: On a morning shift, this is the best time to be in the control room. If I were on night-shift, 2:00 a.m. would be the best. Why? Because at 7:00 a.m., none of the detector experts have arrived yet (or at 2:00 a.m. they have all finally gone home) and these are the few hours in the day where I can have this multi-million dollar detector all to myself.

I guess it is not entirely ‘all to myself’. Even in the early hours of the morning, the ATLAS control room is busy. At any given time, there are more than 20 people on shift in the main control room; four for the muons, three for the calorimeters, four for the inner detector, three for the trigger and DAQ, five for more global operations, such as shift leader, overall safety (SLIMOS), and detector data quality shifts.

The advantage to having so many different types of shifts is that there is a shift type that suits every personality on ATLAS. As I work on the operations of the Tile Calorimeter, I like to ‘escape’ by taking shift leader, data quality or run control shifts.

The shift leader is the floor boss and has the job of executing the daily run plan as determined by the ATLAS run coordinators. But the shift leader also has to be able to make some tough decisions, such as whether or not to temporarily prescale a trigger due a very high level‐one rate,  that is temporarily reject a fraction of the events normally retained until the situation returns to normal. When the detector is running smoothly, the data quality shift is the best way to get a feel for the whole of ATLAS and monitor more physics‐based quantities. In contrast, the run control shift is the most exciting when things are going horribly wrong. This shifter controls the main DAQ and therefore must deal with all problems that prevent the run from going. But today I am on Tile shift, which is rare but still enjoyable. It’s like having coffee with an old friend.

8:00 a.m.: Slowly people start trickling in. In an hour the control room will be swarming with people just like a bazaar. Armed with the morning’s first cup of coffee, they will bounce from desk to desk, gathering all the news from last night’s runs. Soon my quiet time with the detector will come to an end.

While I look forward to time with the detector, I know that many people consider taking shift a burden. Personally, I have never understood that mentality. All the physics that ATLAS can ever hope to discover starts here. If the data is junk, the physics is junk. The person on shift is the first person to determine if the data is good and the best person to correct the situation if the data is bad. It’s the kind of responsibility I really enjoy having.

9:00 a.m.: The ATLAS bazaar is in full swing. The morning meeting starts at half past nine but already the subsystems are bartering with each other for use of the central trigger or use of the RPC trigger for specialised timing studies. Animated discussions are breaking out in all corners of the control room about the number of cosmic muon tracks seen in the inner detectors, or potential calorimeter ‘hot regions’ seen by the high‐level trigger. Slowly the conversations make their way to the conference room upstairs where the morning run meeting is held.

10:00 a.m.: The shift leader returns from the morning meeting and starts explaining to each shifter the plan for the day. “Smooth running for the entire day,” he tells me, “but try to investigate any difference in trigger timing between the different Level-one triggers. This feedback will be used to determine if the global trigger timing needs to be adjusted.” As Tile has a plethora of monitoring histograms devoted to understanding the timing of different triggers, I start to make use of the four screens at my desk and bring up tons of plots in order to try to answer this question.

1:00 p.m.: Tile is running very smoothly. I think I will take this opportunity to check my email. Suddenly I have this uncanny feeling that many pairs of eyes are staring at me. There can be only one reason for this; Tile has gone BUSY thus bringing the data taking to a complete halt and clearly everyone in the control room has realised this except for me – the unsuspecting Tile shifter who is obviously checking her email. The shift leader approaches and utters the words most feared by all shifters everywhere: “Your system is killing the run”. I diligently get to work debugging the problem and soon, with the help of the run controller, the run is going again.

3:00 p.m.: My shift has officially ended. After summarising my shift with the next Tile shifter, I head for the door.

5:00 p.m.: I finally make it out the door! Leaving the control room is a very hard thing to do. It is impossible to go five metres without bumping into someone whom you need to talk to. As I finally close the door blocking out the roar of conversations, it is clear that the control room will be a main hub of ATLAS activity for many years to come.


Monica Dunford

University of Chicago