Comunicação em grupo para projetos de computação em grande escala

Abstract

In volunteer computing systems based on the master-workers model of communication, client machines receive jobs from the project’s servers, execute them, and then return the respective results, all the while being unaware of other nodes executing simultaneously. This kind of architecture is adequate for massively parallel applications, but it raises problems for processes that would benefit from communication in parallel or replicated execution, as we would have to rely on the servers for client coordination, thus creating a server-side bottleneck. In this research, we propose a solution that allows parallel applications to work in standard networked environments, via the implementation of a library, IGCL (Internet Group Communication Library), for inter-node communication and distributed task execution. This library provides clients with peer-to-peer capabilities inside configurable groups of nodes and allows them to communicate during execution in a server-independent way. Exchange of data is done via basic send/receive methods, n-buffering, and several common communication patterns. To ascertain what kinds of applications are adequate for Internet-scale communication, we implement a set of example algorithms and show that it is plausible to use IGCL for such ends when applications are below certain communication requirements. We also demonstrate that the library has performance comparable to MPI when running in local groups of machines.