Odessa: Enabling Interactive Perception Applications on Mobile Devices

The use of interactive perception applications over the mobile devices have increased. Unique set of requirements for these applications:

1. crisp response
2. continuous processing of high data rate sensors
3. running compute intensive algorithms whose performance is highly variable 
4. performance dependent on the content of the data. 

As the mobile devices are not sufficient to fulfill these requirements, techniques using offloading of application components and parallelism have been a point of research. This paper presents Odessa that is agile to changes in network, device and input variability. Odessa automatically and adaptively determines how best to use aforesaid techniques dynamically at runtime to improve makespan (measure of responsiveness) and throughput of applications (measure of accuracy).

Design

Odessa is built on Sprout framework (a distributed system supporting continuous processing of high rate streaming data) which provides mechanisms to dynamically adjust running applications, change the degree of parallelism and migrate processing stages between machines.

Components:

1. Lightweight Application Profiler - maintains the complete application performance profile. Following information is collected for each frame processed: execution time of each stage, wait time on every connector, volume of data transferred on each connector, and the transfer time across the network connector edges.

2. Decision engine - functionality split in 2 threads which make use of application profile data to make their decisions. First thread manages the data parallelism and stage offloading. Second thread manages the pipeline parallelism.

Odessa uses a greedy algorithm that periodically acquires information from a lightweight application profiler to estimate the bottleneck in the current configuration. Then, the decision engine estimates whether offloading or increasing the level of parallelism of the bottleneck stage would improve performance at a negligible overhead cost.

Strengths:

1. Odessa improves application performance across different mobile platforms when the mobile is disconnected from the server.
2. Dynamically decision making at runtime and hence no requirement of any prior performance information.
3. No requirement of a set of feasible partitions like in CloneCloud.
4. Applications chosen for experimentation are both compute and data intensive. Also, detailed study of the performance of applications is done on the basis of effect by different factors.
5. Adapting quickly to changes in scene complexity, compute resource availability, and network bandwidth with negligible overhead.
6. Self-correcting mechanism to maintain stability in case if the decision taken by Odessa needs to be reversed.

Weaknesses and discussion points:

1. Current implementation only acts when the resulting estimate improves both makespan and throughput. There is a need to handle the cases when one metric is improving.
2. Decision engine should take application fidelity into account when deciding the level of data-parallelism.
3. As future work, authors want to extend Odessa to take advantage of public cloud. I am not sure if this will improve the system much.
4. System need to be tested for a wider range of applications.
5. Offloading and data-parallelism decisions may need to be reversed resulting in a stage bouncing between mobile and server.