2020-07-03 (71fe26c)

3 Apt and Repeatable Research

One of Apt’s key goals is to enable repeatable researchwe aim to make it easier for researchers to get the same software and hardware environment so that they can repeat or build upon each others’ work.

Virtual machines in Apt do not provide strong resource guarantees or performance isolation from other concurrent users. Therefore, they are suitable primarily under the following conditions:

We therefore recommend that researchers who want to provide a repeatable environment do the following:

  1. Conduct initial development and gather initial numbers using virtual machines. Because much of the time in this phase is often spent on debugging, development, etc., using a full physical machine is often an inefficient use of resources.

  2. Switch to physical machines to collect numbers of publication. This ensures that published numbers are not affected by interference from other users of Apt.

Similarly, for those who are repeating or building on the work of others, we recommend:

  1. During the initial, exploratory phase of figuring out how to run the code, examining its configuration and parameters, etc., use virtual machines.

  2. Switch to physical machines when it’s time to do real experiments, compare performance with published results, etc.

There are planned features relating to this section: see "Simpler Virtual/Physical Profile Switching" for more details.

Because the disk images that Apt provides boot on both virtual and physical machines, in most cases, switching means simply modifying one’s profile to request the other type of resource.