Compute Clusters Integration for Debian Development and Building – Report 4

Hello, this is the fourth report for the project. First, I’d like to
apologize for being late, I took a short trip over the weekend and
missed the deadline. This time I’ve good news for my project. As
expected I’ve made the necesary changes to have Eucalyptus support ARM
images. The approach was extending and exposing an ‘arch’ field that
will be used by libvirt’s XML domain file. I’m working currently on
having the involved functions working with the new extension and I
expect to have this done over the middle of the next week.

What was done so far
——————–
– Extended the ncInstance_t struct on util/data.h adding the archId
parameter. This struct keeps the information regarding a node (nc),
say RAM, Kernel, etc. Currently it has no way to set an arch in order
to be used later.

typdef struct ncInstance_t{
...
char reservationId[CHAR_BUFFER_SIZE];
char userId[CHAR_BUFFER_SIZE];
char archId[CHAR_BUFFER_SIZE]; // arch field added
int retries;
...
}

The Eucalyptus team was OK with this approach that I’ve proposed few
weeks ago. For all my changes the approach is maintain compatibility
with existing features.
We still have to define if we are setting a default value, say amd64,
or keep it unset until the user sets one. For now we will be supporting
amd64 and arm as valid fields.

– Modified the allocate_intance() function to support the archId
field. This involved adding a new parameter and detecting whether its
present and setting de value accordingly. This function is later used
by the doRunInstance() function in node/handlers_kvm.c

– Modified the involved functions that are called by the KVM handlers
and result in passing parameters to the XML file generator for being
used by libvirt’s and KVM and have the image running under
Eucalyptus. The new parameters and field are not required and only
being used if they are set with a value. The archId parameter has been
added as the last one, so existing function calls can work. This could
be rearranged later, but will involve a more extensive revision with
Eucalyptus developers.

– I’ve set a git local repo in order to keep track of the changes to
Eucalyptus. Since this part is a sort of a Eucalyptus branch I took
this approach. Steffen and I are discussing whether to release this as
a pkg-eucalyptus branch (as dpatch, using existing packaging). Either
way I’m putting the diff files form my git repo over the wiki and will
put them on my site too. I plan to use git-svn in order to send them
later to the pkg-eucalyptus repo once we are settled with this.

– Updated the project’s wiki[1] page with information regarding all that’s
involved on my work, so it can be ‘re-played’ or resumed later.

1- http://wiki.debian.org/SummerOfCode2011/BuildWithEucalyptus/ProjectLog

What I plan to to over the next weeks
————————————-
– Finish the modifications to the relevant functions. Expected dealine:
next week.
– Do integration tests:
– Write a test program, there’s an example under node/ that could be
helpful.
– Generate the XML domain definition from Eucalyptus functions. The
full cycle: handler_kvm -> allocate_instance -> get_instance_xml ->
have the XML file properly generated -> qemu-kvm runs the instance
using the proper emulator (arm in our case).
– Test runnning the image I’ve worked on the project’s begining.
– Fixing bits and cooperate with the pkg-eucalyptus team on the
packaging bits.
– Talk to Eucalyptus so they adopt our patches.
– Rock and roll :)

What would be nice to have after the project ends
————————————————-
– The Debian packaging *needs* work. I’ve already discussed this on the
pkg-eucalyptus mailing list. I’ve also committed myself to help on
that side even after the SoC ends.
– Talk with upstream and Ubuntu to coordinate the packaging efforts and
modifications to the software.
– Get adopters to use our work :)

By now, few weeks about to finish the SoC, I can say that it’s been a
good experience and as I stated I’m planning to keep maintaing/working
on the software after that. I’ve already joined the pkg-eucalyptus team
and plan to contribute actively. I’m sad I didn’t managed to get Debconf
this year, time was a constraint again, hopefully we can meet in 2012! I
guess that all by now!

Best regards,
Rud