|
| | | | |
| | | |
| Stat |
Members: 3645
Articles: 2'501'711
Articles rated: 2609
20 April 2024 |
|
| | | | |
|
Article overview
| |
|
|
A Comprehensive Perspective on the Pilot-Job Abstraction | | Matteo Turilli
; Mark Santcroos
; Shantenu Jha
; | | Date: |
18 Aug 2015 | | Abstract: | This paper offers a comprehensive analysis of the Pilot-Job abstraction
assessing its evolution, properties, and implementation as multiple Pilot-Job
software systems. Pilot-Job systems play an important role in supporting
distributed scientific computing. They are used to consume more than 700
million CPU hours a year by the Open Science Grid communities, and by
processing up to 5 million jobs a week for the ATLAS experiment on the
Worldwide LHC Computing Grid. With the increasing importance of task-level
parallelism in high-performance computing, Pilot-Job systems are also
witnessing an adoption beyond traditional domains. Notwithstanding the growing
impact on scientific research, there is no agreement upon a definition of
Pilot-Job system and no clear understanding of the underlying Pilot abstraction
and paradigm. This lack of foundational understanding has lead to a
proliferation of unsustainable Pilot-Job implementations with no shared best
practices or interoperability, ultimately hindering a realization of the full
impact of Pilot-Jobs. This paper offers the conceptual tools to promote this
fundamental understanding while critically reviewing the state of the art of
Pilot-Job implementations. The five main contributions of this paper are: (i)
an analysis of the motivations and evolution of the Pilot-Job abstraction; (ii)
an outline of the minimal set of distinguishing functionalities; (iii) the
definition of a core vocabulary to reason consistently about Pilot-Jobs; (iv)
the description of core and auxiliary properties of Pilot-Job systems; and (v)
a critical review of the current state of the art of their implementations.
These contributions are brought together to illustrate the generality of the
Pilot-Job paradigm, to discuss some challenges in distributed computing that it
addresses and future opportunities. | | Source: | arXiv, 1508.4180 | | Services: | Forum | Review | PDF | Favorites | |
|
|
No review found.
Did you like this article?
Note: answers to reviews or questions about the article must be posted in the forum section.
Authors are not allowed to review their own article. They can use the forum section.
browser Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
|
| |
|
|
|
|
News, job offers and information for researchers and scientists:
| |
REGISTER