There are a few things that you need to consider if you are attempting to add an AD Security group to a workflow in GP.
First, the same limitation exists for groups that exist for users in that you need to have a 'Display Name' configured for the group. If you do not you will not be able to successfully add the group to the Workflow. I setup an AD Security Group called 'WrkFlow' and am trying to add it to my Workflow in GP 2015. You'll notice that the group comes up with just the (Domain\Name) when I look it up.
You need to go into 'Active Directory Users and Computers' and give the group a 'displayName' in the Attribute Editor. You will need to set the window up to view Advanced Features to accomplish this.
In the 'Active Directory Users and Computers' window go to 'View' and make sure the 'Advanced Features' option is checked.
You will then be able to 'Right-Click' your security group and go to 'Properties'. In the 'Properties' window you will now have an 'Attribute Editor' tab available at the top. In the 'Attribute Editor' tab you'll be able to select the 'displayName' attribute and click the edit button to give it a name.
Once you've given your security group a 'displayName' it will now show up in Workflow like the following:
Notice that it is no longer showing JUST the (domain\group) name, it will now also show the display name you gave it. This is required for wokflow to successfully add a user or a group.
There is also another issue that can arise here were you will find the group but it will not add it. When you create a security group in Active Directory it is assigned a 'Group name (pre-Windows 2000):' name. This will default to whatever you name the actual group. We insert this name into SQL with a character limit of 21 or less. So if you have security group name that exceeds 21 characters in length you need to manually change the 'pre-Windows 2000' name to be less than 21 characters in length. You can find this setting in 'Active Directory Users and Computers' by right-clicking the group and going into properties and looking at the 'General' tab.
These are the only 2 factors that I am currently aware of that will prevent you from adding an AD security group to Workflow 2.0. The group must be a security group and not a distribution group as well. I hope this information helps you out if you're in the process of implementing Workflow 2.0 in your environment.