Unfortunately that is a open item on the wish list for many users. It has been requested multiple times in the past. It currently is not possible.
Maybe the 3D Dimension Lines plugin can do something like that but I never used that plugin. Even then it will be to individual objects, not a group.