LOFAR is preparing the upgrade to the
LOFAR2.0 system, which will offer simultaneous low- and high-band observing,
increased field-of-view, and various other improvements to the sensitivity and
operation of the telescope. The new system will generate considerably more data
than the current one. The need for processing and storage of this data represents
a significant challenge, as it outstrips current affordable solutions. To
realize the goals of the new Observatory, the ILT board has approved a data
life cycle that optimises
science data processing capacity and efficiency with respect to affordable data
storage. This includes data retirement. The ILT
anticipates that (final) science data products, generated through standard
pipelines (such as image cubes) can be kept available indefinitely (under open
access policies), but that raw/intermediate products may only be retained for
further processing within a restricted dwell time.
To prepare the ILT infrastructure for the LOFAR2.0
data challenges, a first step needs to be taken now that involves the retirement
of early Cycle data from the LOFAR Long-Term Archive (LTA). After careful
analysis, it has been concluded that data from Cycle 0 up to and including
Cycle 6 is either low in metadata quality or already valorized and published (https://science.astron.nl/telescopes/lofar/science-with-lofar/lofar-papers/)
or, based on the download statistics, of little demonstrable interest to the
current community. The ILT has therefore approved that mid-2024 this data will
be deleted from all LTA sites.
Through this notification, we give the
opportunity to any users (i.e. not only the projects’ PI’s) who may need to
retrieve data from these early LOFAR Cycles, to signal this to the Observatory to
coordinate the data download. The requests will be evaluated by the Observatory
for technical feasibility and reviewed by the ILT-board for approval. We
emphasize that all data concerned is public data.
Please, refer to the overview distributed via e-mail to understand which data is selected for retirement. To avoid excessive load of the data access services at the LTA sites, the Observatory needs to be informed of your data download plans and needs to evaluate them in relation to data access requirements for other users and purposes. For this, we ask you to submit a formal request for data download by using this Google form.