Ismael Alonso
Problem: inner-system load balancing is an important part of computing in situations where which machines the clients connect to matters but the service should not degrade when load spikes. In these circumstances the system should effectively reallocate tasks to complete them in a timely manner.
Approach: centralized schemes don't scale well. Fully distributed schemes arrive at poor conclusions or take longer to get there [1] since, by nature, all parts of the system have to be synchronized. My solution was a hybrid: Hierarchical Load Banancing; a compromise between the two. In hierarchical there is delegation, but the decisions are centralized.
Most of the work I did was based in ref #1. Whereas the paper was quite non-specific, there were a number of ideas that helped me throughout the development of the project. Some of these include the way the hierarchy is built and the way the nodes interact with each other in it. The authors provide hints on how load data is collected and how job information is passed around during balancing events, but they do leave a lot to the imagination. In part, I had to think about how to structure the data so that no relevant information was lost at any step while still aggregating it. They never talked about all the metadata nodes had to keep to make deaggregation, a vital part to make the system work, possible... although they did perform a memory footprint analysis. Finally, they mentioned jobs aren't passed around during balancing, which makes sense if the inputs get large. They talk about how job transfers are somehow represented, but they don't say what that representaiton was. In short, the story of this project was one of discovery; realizing the need to go back to the drawing board because unforeseen problems present themselves.
Results: the outcome of this project is a product that demonstrates hierarchical load balancing in a single machine. The product is 90% ready to actually be deployed to a cluster, as it uses sockets for communication and controllers can spawn as many nodes as desired locally.
Future work: