Resources: What’s available?
Today’s scientific experiments place substantial demands on scientific computing facilities in order to perform their science. Many of these experiments have the ability to collect huge amounts of data. Substantial computational resources are needed to process and analyze those data sets as well as to generate the complementary computer simulations necessary for a physics result. Fermilab plays a leading role in collecting, managing and analyzing these torrents of data, as well as developing specialized tools and software for handling data and presenting them to the public.
Fermilab provides a variety of computing and storage resources optimized for different phases of an experiment’s lifecycle. In addition, Fermilab provides tools for users to process and analyze their data, manage, protect and control their instrument and refined data. High-speed networks enable access to data within the Fermilab facility and into and out of the Fermilab site. Experts in computation, simulation, and data storage systems are available to assist researchers in the efficient use of computational and storage solutions.
The Computational Science and AI Directorate maintains a digital data archive facility for experiments associated with the facility as well as experiments where Fermilab is the lead laboratory. This archive facility has robotic tape storage systems intended for long-term archival storage and backup. It also has available significant disk resources for caching data that is being actively processed and analyzed.
In addition, Fermilab has a variety of GRID, CLOUD and high performance computing resources that are established for its user community, including disk resources and networking to make effective use of these facilities.
Who can use the resources?
Approved experiments of the Fermilab Accelerator Complex, approved scientific programs where Fermilab is the lead, or is involved, and HEP community members developing scientific proposals. Furthermore, Scientific endeavors not associated with Fermilab can petition through a proposal process for use of the lab’s scientific computing and archival facilities. Such proposals are expected to come with funding to cover its costs.
Fermilab Practices:
Fermilab establishes allocations for time on its computing facilities, disk and tape storage and data retention terms, and backup service to meet the needs of the majority of our researches and their experiments. These default allocations avoid oversubscription, and place practical limits on the costs of computing and storage provided. Fermilab is willing to work with individual researchers, experiments and projects to accommodate special needs on a case-by-case basis. Exceptional requests may require alternative funding.
The Fermilab digital data archival facility will migrate its archived data to more modern media, as it deems necessary in order to keep its technologies current.
User responsibilities:
The Fermilab computer network is an open research network, intended primarily for fundamental research. The facility does not guarantee complete confidentiality for data that resides within the facility. It is the responsibility of the user to set access controls, provided and supported by the facility, appropriate for individual/experiments needs.
Policy: The Computational Science and AI Directorate provides the means for researchers to share data. However, the decision as to who has access to the data is the responsibility of each experiment or project’s leadership.
It is Fermilab’s policy that the primary copy of the raw data collected at its facility be stored in its data archive and kept for a minimum of 3 years after project/experiments completion. The overall period for which data remains in the active archive beyond the minimal amount will be determined on a case-by-case basis depending on the scientific relevance of the data and the cost required to maintain it, and documented in individual agreements between the experiment/project and the laboratory.