The popularization of smartphones and tablets and their increasing usage turns them into an attractive target for cyber attackers. This, together with the fact that many organizations employ a Bring Your Own Device (BYOD) policy, make assessing and comparing the security of mobile devices a key issue, in particular in what regards user-defined configurations as these have a huge impact on the potential attack surface. In order to allow characterizing the relative security level of mobile devices with respect to user configurations, this paper proposes a research towards a benchmarking approach to compare and rank mobile devices, with respect to their security. Preliminary results show that Android users neglect important security recommendations while configuring their devices and that benchmarking is indeed a good way to identify the most secure user-defined configuration Ver menos