Opening a site within a group and then clicking back to exit might display the site's own group and not the default one

I think one rationale of grouping sites  is to visit them one by one, picking up the most interesting in a given moment. Exiting from a site within a specific group by clicking Page Back and then returning to the Default speed dial is very inconvenient in this case. There might be an option to select between Default and Current Group