On the EDGE On the EDGE

Highlights from the 2012 Summer School in Budapest

The joint Summer School on Workflows and Gateways for Grids and Clouds was co-organised by three current European Union projects: SCI-BUS, SHIWA and EDGI, to show various aspects of cloud and grid computing. The school has attracted a total of 51 participants from 25 institutes. The tutors and students came to Budapest from 16 different countries.

The school provided answers to how to exploit grid and cloud infrastructures in Europe, how to port and develop applications for these infrastructures and how to extend their user communities. Every morning session was dedicated to presentations to introduce the technologies and to provide theoretical background for the afternoon tutorial sessions where students could try out the mentioned infrastructures. Related partner projects were invited to explain their technologies, talks from Wf4ever, MAPPER, IGE, EGI-InSPIRE and more widened the range of topics in the programme.

During the week, the participants could explore Budapest and a trip was organised to a nearby village where they could taste a set of traditional Hungarian food and see a horse show and a Hungarian folklore dance performance.

Based on the received feedback forms, the school can be considered to be very successful and most likely a summer school next year will be organised in Budapest again.

Kitti Varga

XtremWeb-HEP 8.1.2 released

EDGI project partner CNRS from the Laboratoire de l'Accélérateur Linéaire in Orsay, France has released the XtremWeb-HEP 8.1.2 version.

Major points in this version include:

  • Correction: inter node communication thanks to Jason Maassen from the Netherlands eScience Center in Amsterdam
  • Correction: REST interface
  • New feature: the server embeds Web service available through HTTPS

A. Corrections:

  • A.1 on server side, the scheduling improved: hosts.incomingconnection is now taken into account to ensure a server like job is not sent to a volunteer resource where incoming connections are not allowed
  • A.2 on server side, the scheduling improved: restoring a feature from XtremWeb 1.8.0 by INRIA: "expected host" so that the end user can specify at submission time the expected volunteer resource to run the job. Note that if the expected host does not request a job or does not match the requirements (OS, CPU, ...), the job is never computed.
  • A.3 the DG to SG bridge is now compatible with shared applications so that one can use VirtualBox on EGI resources
  • A.4 some bugs corrected on client side regarding application registration and task management
  • A.5 a bug corrected on temporary file usage: they are now cleaned at software exit
  • A.6 the REST interface has been completely rewritten. The <xwhep></xwhep> root element MUST be used. Ex:

a/ to send a new application: https://server/sendapp/?XWPARAM=<xwhep version="8.0.2-flyinggrid"><app name="pouet" /></xwhep>

b/ to retrieved registered applications: https://server/getapps

  • A.7 a bug corrected in HTTP data download
  • A.8 SmartSocket usage corrected; inter node communications fully functional now

B. New features:

  • B.1 the creation of new Live CD can be customized
  • this is done by providing several optional files:
  • > a text file 'user.packages' may be provided to install custom packages: this file must contain a space separated packages list
  • > a text file 'user.hostname' may be provided to customize the LiveCD hostname: this file must contain the host name only
  • > any package files are installed on the fly (*.rpm or *.deb, depending on the LiveCD OS)
  • B.2 the client can now be used to create SmartSockets end point, independently of any task
  • this may be useful to create SmartSockets tunnel from a running VM to the client PC
  • > e.g. mount the client FS inside the running VM
  • B.3 a new Mac OS X package "xwhep.vworker" to deploy the middleware inside a VM
  • B.4 the server contains an embedded Web service interface available through HTTPS

C. Known Bugs:

  • C.1 xtremweb.gmond.pl does not scale
  • C.2 the scheduler is not fair
  • C.3 there may be some concurrent access problems leading to some errors

There may be two concurrent access problems

  • on server side, DB access may lead to inconsistency between works and tasks tables

This could certainly be solved using SQL transactions

But we would then need MySQL >=5

There are sime issues with transactions with hsqldb

  • on worker side, file access if a worker is configured to run jobs in parallel (default one thread per CPU core)

CNRS has observed that these concurrent access problems may lead to up to 6% erroneous jobs. Until further notification, CNRS considers this rate acceptable.

More information is available at http://www.xtremweb-hep.org/spip.php?rubrique15

EDGI in European Cloud Research BoF session at ISC12

The EDGI project participated in ISC12 in Hamburg, explaining Desktop Grids and integration with EGI, Unicore, and KnowARC to a HPC focused audience. A video from the Birds-of-a-Feather Session is now available at the Vimeo channel.

EDGI in European Cloud Research BoF session at ISC12 from Desktop Grid Federation on Vimeo.

Showing 1 - 3 of 49 results.
Items per Page 3
of 17

EDGI EDGI

EDGI will develop middleware that consolidates the results achieved in the EDGeS project concerning the extension of Service Grids with Desktop Grids (DGs) in order to support European Grid Initiative (EGI) and National Grid Initiative user communities that are heavy users of Distributed Computing Infrastructures (DCIs) and require an extremely large number of CPUs and cores. EDGI will go beyond existing DCIs that are typically cluster Grids and supercomputer Grids, and will extend them with public and institutional Desktop Grids and Clouds. EDGI will integrate software components of ARC, gLite, Unicore, BOINC, XWHEP, 3G Bridge, and Cloud middleware such as OpenNebula and Eucalyptus into SG-DG-Cloud platforms for service provision and as a result EDGI will extend ARC, gLite and Unicore Grids with volunteer and institutional DG systems.

EDGI will develop DG-Cloud bridge middleware with the goal to get instantly available additional resources for DG systems if the application has some QoS requirements that could not be satisfied by the available resources of the DG system. EDGI will improve Desktop Grid middleware (BOINC and XtremWebHEP-E) in order to handle QoS requirements and the SG-DG bridge middleware in order to support data-intensive applications. EDGI will deploy a production infrastructure that integrates ARC-, gLite- and Unicore-based Grids with Desktop Grids based on the bridge middleware developed in EDGI.

The production EDGI infrastructure will also enable the dynamic, on-demand extensions of the connected Desktop Grids with Cloud resources. As such EDGI users can benefit of the versatile and flexible eco-system provided by EDGI. The EDGI production infrastructure will be offered as service for EGI and NGI user communities. It will also serve as a demonstration for NGIs to extend their eco-system with Desktop Grids and Clouds. EDGI will establish a European Desktop Grid federation to coordinate DG-related activities in Europe both for solving technical issues as well as to attract volunteer DG resource donors by disseminating results of the EDGI and EGI-related projects. The European Grid Federation and EDGI will work in strong collaboration with EGI, EMI, NorduGrid, Unicore Forum and interested NGIs.
 

International Desktop Grid Federation International Desktop Grid Federation

EDGI supports the http://desktopgridfederation.org/. The International Desktop Grid Federation brings together Desktop Grid operators, developers, organisations and companies working with Desktop Grids and intergration with other infrastructures through EDGeS Bridge Technology.

About EDGI About EDGI

EDGI is a project managed by the EDGI consortium.

EDGI is supported by the FP7 Capacities Programme under grant agreement nr RI-261556.

                                                                                                                                  

© EDGI Website and EDGI: EDGI Consortium 2010 - 2011 - 2012.