Some people think a mobile experience can only be properly evaluated in the field, whereas others prefer a conventional lab test for better control of the study conditions.
Xu Sun and Andrew May, of Nothingham University (UK) compared the benefits of these two methods. Field evaluations took place in a soccer stadium, whereas laboratory tests recreated an athletic event. Results revealed interesting differences between the two approaches.

The laboratory test identified problems related to details of the interface such as colours and icons.  The test seemed less immersive and interesting to participants because it lacked the stadium atmosphere.  The participants were more critical than in the field study.
The field study brought out problems related to the validity and precision of data during a game.  It also brought out problems of use rather than pure usability issues  — this came out spontaneously in the language linked to the situation (a football stadium).  The participants were also more likely to express that they felt in less of a position of being evaluated.The field study however was more difficult to conduct (due to weather and ambient noise) and certain participants were distracted from the objective of the task.  In addition certain data could not be collected.

The laboratory seems better embedded in a “test” process whereas the field evaluation was conducted more with the goal of understanding an experience. The laboratory is therefore recommended when the objective of the study is to target the interface in itself and the usability problems that are related to it. The field study is more adapted for trying to understand the factors that influence users’ adoption of a mobile interface from the point of view of the proposed functionalities and the device’s possible context of use.