Information about http://www-bdnew.fnal.gov/tevatron/converted/TM-1961.pdf

Fermi National Accelerator Laboratory …

Tags: accelerator, accuracy, annala, batavia illinois, collider, completeness, department of energy, endorsement, fermi national accelerator, fermi national accelerator laboratory, legal liability, national accelerator laboratory, observation, speci, trademark manufacturer, united states department, united states department of energy, united states government, universities research association, warranty,
Pages: 16
Language: english
Created: Thu Mar 28 12:04:54 2002

Display cached document

Page 1

Page 2

Page 3

Page 4

Page 5

Page 6

Page 7

Page 8

Page 9

Page 10

Page 11

Page 12

Page 13

Page 14

Page 15

Page 16

Fermi National Accelerator Laboratory

FERMILAB-TM-1961

Collider Shot Setup for Run II
Observation and Suggestions

Jerry Annala and Bob Joshel
Fermi National Accelerator Laboratory
P.O. Box 500, Batavia, Illinois 60510

March 1996

Operated by Universities Research Association Inc. under Contract No. DE-AC02-76CHO3000 with the United States Department of Energy
Disclaimer
This report was prepared as an account of work sponsored by an agency of the United States
Government. Neither the United States Government nor any agency thereof, nor any of
their employees, makes any warranty, expressed or implied, or assumes any legal liability or
responsibility for the accuracy, completeness, or usefulness of any information, apparatus,
product, or process disclosed, or represents that its use would not infringe privately owned
rights. Reference herein to any speci c commercial product, process, or service by trade
name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its
endorsement, recommendation, or favoring by the United States Government or any agency
thereof. The views and opinions of authors expressed herein do not necessarily state or re ect
those of the United States Government or any agency thereof.
FERMILAB-TM-1961

Collider Shot Setup for Run II
Observations and Suggestions

31 January 1996
Jerry Annala and Bob Joshel

INTRODUCTION
This note is intended to provoke discussion on Collider Run II shot setup. We hope this
is a start of activities that will converge on a functional description of what is needed
for shot setups in Collider Run II. We will draw on observations of the present shot
setup to raise questions and make suggestions for the next Collider run. It is assumed
that the reader has some familiarity with the Collider operational issues. Shot setup is
defined to be the time between the end of a store and the time the Main Control Room
declares colliding beams. This is the time between Tevatron clock events $CE and
$CB. This definition does not consider the time experiments use to turn on their
detectors. This analysis was suggested by David Finley.

The operational scenarios for Run II will require higher levels of reliability and speed
for shot setup. See Appendix I & II. For example, we estimate that a loss of 3 pb-
1/week (with 8 hour stores) will occur if shot setups take 90 minutes instead of 30
minutes. In other words: If you do l2 shots for one week and accept an added delay of
one minute in each shot, you will loose more than 60 nb-l for that week alone (based
on a normal shot setup of 30 minutes). These demands should lead us to be much more
pedantic about all the factors that affect shot setups.

Shot setup will be viewed as a distinct process that is composed of several inter-
dependent 'components': procedures, hardware, controls, and sociology. These
components don't directly align with the different Accelerator Division departments,
but are topical groupings of the needed accelerator functions. Defining these
components, and categorizing our suggestions within them, are part of the goal of this
document. Of course, some suggestions span several of these components.

1
PROCEDURES
Procedures are the written descriptions of what is done in the control room to run the
Collider. The operating scenarios, described in design reportsl,2 and past experience
make up the procedures. In Run I these procedures were translated into Sequencer
language. This was very effective and should be a goal for Run II; even if 'Sequencer'
takes on a new meaning. This leads to deterministic operation of the Collider.

The overriding operational scenario for Run II is the use of the Main Injector (MI) and
eventual use of the Recycler. We are directed to assume that the Collider will be
commissioned without the Recycler initially available3. Another assumption is: protons
destined for colliding bunches will not be cooled in the Recycler. Two key factors will
be important in Run II: the need for fast shot setups and dealing with much greater
operational complexity. See Appendix III for a draft outline of the shot setup's time
evolution.

In order to have a fast shot setup we will have to deal with the following procedural
issues:

We will need tools to analyze shots using standardized measurements. As an
example, look at Appendix IV. It summarizes elapsed times for Collider shot
setups during run 1B. These kinds of tools--and the use of them--will be
needed.

Moving the Tevatron injection lambertsons is slow. Operational tactics may
reduce the need to move them. We also need to modernize the motion control
system--see the hardware section for more details.

Scraping the colliding beams in the Tevatron takes from 15 to 30 minutes, on
average. This is a manual activity with a great deal of Operator involvement.
This process should be much more automated. The highest luminosity is lost
during scraping. With use of the Recycler, we estimate an average of 4 pb-
l/week would be lost if scraping takes 30 minutes. We may consider a more
exact (re)definition of scraping requisites. This definition may allow some of the
more subjective judgments to be quantified. For example, could we declare data
taking started, even though there continues to be some low-level reduction of
losses?
2
Presently we have a 30 minute quiet time--no beam in Tevatron enclosure
between a store and the next shot. Do we expect to maintain this policy? Or, can
we schedule quite time to take place once a week, or during accelerator down-
time?

Between a store and the next shot we ramp the Tevatron 6 times to reset
'remnant' fields. The optimal requirements to reset fields needs to be described.
Related questions are: use of shorter super-cycles, requirements for resetting the
low beta magnets versus the ring magnets, description of needs for resetting
fields under different failure modes (e.g. quenches).

Instrumentation data return-rates will need to increase, given the desired speed
of shot setup. This will prevent having to pause shot setup while acquiring
important data. Specification of these data return rates is recommended.

To assist with dealing with the operational complexity we will have to deal with the
following issues:

Under use of the Recycler we will have to remove protons from the Tevatron
before recycling the antiprotons. The technique to do this (in a timely fashion) is
yet to be designed. Included in this issue is how we will handle the beam at low
beta: will we un-squeeze before deceleration prior to transferring antiprotons to
MI?

Will the present controls infrastructure be sufficient to handle the new levels of
complexity? In particular, do we need any structural change to the Sequencer,
like: conditional invocation of aggregate commands, ability to branch in
sequences, triggering aggregate commands on states or events rather than
sequential, manual invocation.

There are detailed designs for the clock events scenarios4 We will need higher
level tools to deal with the configuration of the clock. We suggest significant
changes to the present Time Line Generator (TLG) system--see the Controls
section for more details.

3
HARDWARE
Hardware includes all the physical components of the accelerator complex, including
instrumentation. The following items are suggested hardware improvements:

We need a more modern motion control system. The present CAMAC (C057)
system demands feedback through the highest level of the control system to
accurately place motordriven hardware. We suggest a newer system that consists
of a smart module--in the field--that can accept a desired position request and
handle the feedback to carry out the request. This may also include feedback
based on signals other than position (e.g. losses). This will allow for more
independent and dependable use of motor-driven hardware.

The Beam Position Monitors should be able to measure all beams with the bunch
structure that will exist. For example, we should be able to measure coalesced
beam in MI and easily measure antiproton orbits in the Tevatron and MI.

The importance of prompt and reliable passive measurement of transverse beam
size is apparent. The present systems should be reviewed to assure their
combined results are sufficient. This includes: flying wires, ion profile monitors,
synch-light monitors...etc.

We have seen a benefit in reducing the variety of ramp-cards used. It simplifies
the number of objects that Operators must master to diagnose problems. We
suggest to continue further with this activity.

Presently C160 modules are the interface to Tevatron dipole correctors. We
should review if there would be a benefit to using more modern ramp-cards.

TECAR will need to support the new set of operational scenarios for the
Tevatron. An explicit review of the needs for TECAR may uncover any
enhancements that are needed.

With the number of operating scenarios, a comprehensive system to do 'particle
accounting' would be useful. Do we have enough beam current toroids in place?
These should be fed into a system that would account for all the beam on a given
Linac pulse or beam transfer.
4
Reliability becomes more important under Run II operating scenarios. A
comprehensive review of hardware failures and plans to improve reliability would be
time well spent. At present there are steps in shot setup that exist only to check, or
allow, for hardware problems (e.g. checking for bad VFCs in the Tevatron). These
steps should move from shot setup to routine subsystem maintenance.

The safety system should be configured to ease changes in operational scenarios. MI
design specifies that the Tevatron radiation permit be up to allow stacking1. It would
be worth investigating breaking up the Tevatron safety enclosure into several sub-
enclosures. We may then allow access to experimental areas, or other areas of the
Tevatron, during stacking. It would be an advantage to be able to access any region of
the Tevatron tunnel while allowing a Recycler store. A general review of safety system
constraints on Operational scenarios may uncover some desired changes to safety
system configurations.

CONTROLS
Controls includes the computers, architecture, and software to allow for control of the
accelerator complex.

There are a class of activities that are mundane and well described but still carried out
manually. The following should be considered for some form of automation or
refinement:

Provide tuning of closed orbits and injection closure in all machines. Provide
steering (and beta-matching) in transport lines between machines.

Automate applications programs that are always used by carrying out the same
sequence of operations (e.g. create more program scripts).

Consolidation of applications would be an advantage. For example, it would be
efficient to have just one program for tuning of beam lines. This would reduce
the user-interfaces that Operators will need to learn. Operators need to be
included in the user-interface design of applications.

Replace manually loaded VCR tape machines with a more modern system. This
would provide for large (indefinite) amount of storage.
5
Recovery from power-outages and enclosure accesses could be more automated.

More un-interruptible power supplies will help; for example, in the MAC room
where the clock system resides.

There will be a need to manipulate clock event profiles at a more complex level then
present. The TLG and related console application should be upgraded with the
following features:

Allowing storage of pre-defined clock tables, in the TLG module, would allow
for more flexibility in switching time-lines. Time-line switching would then be
done by changing a table pointer rather than having to down-load the whole
clock table. We estimate the TLG needs to hold at least 50 tables; with super-
cycle support in the range of 1 to 200 seconds. Flags to specify time-line
repetition and one-shot playback will be needed. The TLG application will need
to support higher level views of the clock scenarios. This will assist with
managing the more complex operational scenarios that are expected. Any new
design should continue to support beam power warnings for Operators. In
addition, programs that affect the beam power (e.g. Booster bunches and turns)
should be included in the checks.

The general approach of looking at trends in shots is not formalized. There are several
sources of saved data: SDA, SRSAVE, lumberjack, Settings...etc. These may be
correlated and more used, if accessed with a common method. Then data across
different sources could be analyzed. Some other related suggestions:

Provide methods to access archived data into commercial products (e.g. Excel
spreadsheet). As a corollary to this, we suggest the placement of several Personal
Computers in the Main Control room. This will invite the use of powerful tools
for analysis and other activities that are 'off-line' from the accelerator controls.
These are not intended to replace the control system consoles.

Examine and evaluate the need for upgrades to SDA. Formalize shot analysis,
and trending, through a console application and personal computing tools.

6
SOCIOLOGY
Sociology is the interface between humans and all the other components of shot setup.
This includes administrative 'forces'. Suggested changes follow:

The structure and definition of roles of Collider and shot coordinators, machine
specialists, operators, Operations Specialists, crew chiefs...etc. should be
examined. Is there a way to streamline the organization and optimize its
functionality? We recommend a review of the daily '9:00 meeting'. We may
consider a focused scheduling session and a distinct session to review problems.

In a similar manner, the physical and administrative configuration of the Main
Control Room may be made to encourage more efficiency. Experts should be
encouraged to participate but at the same time we need a method to prevent
confusion in the Main Control Room.

There is strong element of 'organic' use of Sequencer modes. The use of several
modes began in Run I. We may benefit by reviewing this and possibly unifying
the style and integration of the modes.

We have seen a trend to automate mundane tasks. This should continue as it frees
Operators to work on more complex tasks. Machine experiments (more
commonly called 'studies') should be better planned and more automated.

As the subsystems become more complex having systems people attend to
problems and maintenance is important. Operators no longer can, nor should,
know the all details of the subsystems. Shot setup should not be made longer by
Operator attention to these subsystems. We may consider having technicians also
on shifts (e.g. power supply and electronics techs).

There is a longer-term effect that feeds back into individual shots: outstanding
problems fold back into a shot setup. A method for tracking and inviting closure
to reported problems would lessen this effect. An electronic fix-list should be
used. It would greatly increase the accountability of getting problems resolved.

There are several accelerator groups that use electronic log-books. We may
consider this become a standard.
7
CONCLUSION
The above suggestions vary in priority and detail. It is our hope that part of the next
step would be for the involved parties to resolve what should be considered further,
and which items are more immediate.

Given the constraints on future shot setups, we need to design accelerator subsystems
with the forethought of how they will integrate into the shot setup; rather than having
to fit them into shot setup as a last step of their design.

We hope this document will be picked apart, augmented, and refined. We suggest a
working group be formed to do so. This may present a common understanding of what
is needed for an effective Collider Run II.

8
APPENDIX I Reliability Constraints

The following argument assumes the use of the Recycler. Without the Recycler, the
described results are diminished but still significant.

Refer to table 2.1.2, in the Recycler Technical Design Report Draft, to approximate
the impact of a lost store (note that a six hour store length is used for this table). If the
antiprotons are lost from the Tevatron at the scheduled time for the store to end there
will be only 43% of normal stack to shoot from for the following store. This will
correspond to a luminosity of less than half of the nominal value for the next store.
The effect of the lost antiprotons will also trickle down to stores in the future--but we
will ignore this for now. The scenario of losing antiprotons at the end of the store is
the Tevatron failure with the least impact on integrated luminosity. If a store drops out
earlier, time will be spent stacking just to reach a usable stack size. At the very least
the integrated luminosity lost will be 50% of the normal integrated luminosity for a
store or 1.5 pb-l. Also, if a store is lost, chances are high that there will be some
recovery time for the Tevatron to prepare it for the next store. Without downtime, the
design calls for the integrated luminosity to average 0.4 pb-l/hr; so the cost of
recovery time is also quite high. It seems a conservative estimate to say that an average
Tevatron failure will result in a loss of more than 2 pbl.

9
APPENDIX II Shot Setup Time Constraints

To understand the importance of efficient shot setups look at the Recycler Technical
Design Report Draft. The Operational overview of the Recycler shows a plot average
luminosity vs. store duration for various shot setup times. We use this plot to calculate
the amount of integrated luminosity missed while in shot setup. We will normalize the
numbers to a 92 store hour week; which produces the design value of 40 pb-l /week.
The plot then shows the amount of integrated luminosity missed because of being in
shot setup for various shot setup times vs. store duration.

If we use an estimate of an 8 hour store length, we see that the difference between a 30
minute shot setup and a 90 minute shot setup equates to a difference of over 3 pb-l
/week. This is more luminosity than we were able to reliably produce in a week during
Run 1B.

10
APPENDIX III Shot Setup's Time Evolution

Run II time line with the Recycler:

The diagram below shows a likely time line that will occur with the use of the three
pivotal machines used during shot setup, under the Recycler scenario. The vertical
lines represent breaks in activities for the machines whose time lines are intersected.
The narrow black time lines show blocks of time that should be fairly well understood.
The thicker gray lines indicate blocks of time that are harder to predict, due to either a
lack of procedure or the hope of an improved method to be developed.

The length of the Recycler block, labeled antiproton cooling and partitioning, depends
on how well electron cooling performs. In the initial operation, however, the recycled
antiprotons will be kept separate from the cool stack and the partitioning will easily be
performed in the time the Tevatron is preparing for transfers.

The more defined blocks are shown with a time estimate. It is not clear how long the
MI deceleration cycle will be, so a range of 1 minute to 5 minutes is used for an
estimate of this time block. It is quite practical to believe that all parts of shot setup
denoted by the narrow black lines can be completed in less than 30 minutes. The Quiet
Time block takes 29 minutes to complete in Run 1B. Also, scraping the protons away
at the end of a store can take 30 minutes with only 1.2 e12 Protons in the Tevatron.
During the November 1995 multi-bunch studies Operators were unable to scrape away
the Protons at the end of a store without quenching. Leaving the Quiet Time blocks and
the Remove Proton blocks at 30 minutes each changes the shot setup from less than 30
minutes to 90 minutes or more. This difference equates to over 3-1pb/week.

11
The other block, denoted by the thick gray line, is the scrape at the beginning of the
store. During run 1B this step took between 10 and 30 minutes. If scraping continues
to take up to 30 minutes, the luminosity lost to the experiments is 10% of that
delivered for the 8 hour store. This equates to an additional 4-1pb/week lost.

Run II time line without the Recycler:

The explanation of the diagram is the same, but with this operating scenario the
removal of protons goes away. The poorly defined block, labeled quiet time, is the
dominant time block of the shot setup. This block could change the shot setup time
from around 20 minutes to nearly 50 minutes. The description of scraping is the same
as in the Recycler scenario, but with the longer stores, the percent of integrated
luminosity lost during this period would be slightly less.

12
APPENDIX IV
The diagram below summarizes data from the last 40 shots. It shows times in minutes;
derived from Collider Sequencer logged data. Resolution is limited by a 1 minute
logging rate. Average and standard deviations are shown. 'Time halted' includes
elapsed time at the completion of an aggregate, before continuing on to the next
aggregate. This analysis was provided by Keith Engell, Accelerator Operations .

# aggregate name < run time > sigma < time halted > sigma
(run time) (time halted)
--------------------------------------------------------------------------------------------------------
12 recover from lowBeta 6 3 0.1 0.6

l0 recover from store 66 63 3.6 7.0
(includes 'quite time')

1 stop at 150 Gev 4 1.7 0 0.1

2 inject protons for tuneup 10 8.7 0.5 0.9

13 reverse injection 26 14.6 0.4 0.9

3 set up proton injection 2 1.4 0.0 0.0

18 proton pilot shot 5 5.7 0.4 0.9

20 MR or pbar curve loading 15 17.9 0.4 1.4

4 inject protons (P1 - P6) 15 10.4 0.7 2.1

5 set up pbar injection 4 3.0 0.2 0

7 inject pbars (Al - A6) 9 3.6 0.3 1.0

8 prepare to ramp 2 1.0 0.0 0.0

9 ramp to flattop 4 2.6 0.0 0.0

1l turn on lowBeta then collide 34 45.2 1.5 2.2
--------------------------------------------------------------------------------------------------------