Only get the groups list if calc_perms() is called

Only get the groups list if calc_perms() is called as there's no point
grabbing the list otherwise.

Signed-off-by: David Howells <dhowells@redhat.com>