vCloud Director doesn't have a way to reconcile this automatically.
I assume you are using allocation pools (since that is the only Elastic setting I can think of)?
If so, you should be able to delete the non-Primary Compute Resource from the provider itself.
Manage & Monitor > Provider vDCs > Double Click the Provider > Resource Pools > Right Click > Disable and then Detach any not listed as Primary. The pool must be empty as per vCloud Director.