Pascal Duvier said he was not part of the pop star’s security team following allegations from Brazilian soccer player Jorginho