3 Apt and Repeatable Research
One of Apt’s key goals is to enable repeatable research—
Virtual machines in Apt do not provide strong resource guarantees or performance isolation from other concurrent users. Therefore, they are suitable primarily under the following conditions:
During initial exploration, development, gathering of preliminary performance results, etc.
When it is the output of the software, rather than the software’s performance, that is of interest.
We therefore recommend that researchers who want to provide a repeatable environment do the following:
Conduct initial development and gather initial numbers using virtual machines. Because much of the time in this phase is often spent on debugging, development, etc., using a full physical machine is often an inefficient use of resources.
Switch to physical machines to collect numbers of publication. This ensures that published numbers are not affected by interference from other users of Apt.
Similarly, for those who are repeating or building on the work of others, we recommend:
During the initial, exploratory phase of figuring out how to run the code, examining its configuration and parameters, etc., use virtual machines.
Switch to physical machines when it’s time to do real experiments, compare performance with published results, etc.
There are planned features relating to this section: see "Simpler Virtual/Physical Profile Switching" for more details.
Because the disk images that Apt provides boot on both virtual and physical machines, in most cases, switching means simply modifying one’s profile to request the other type of resource.