Hello tjoren,
In WebDir 4.0, we removed the Add and Remove buttons that displayed next to the directReports window of the Manager's property sheet. Microsoft does not recommend using this method to add or remove direct reports. The proper way to remove the direct report would be to open the properties sheet of the direct report you plan to remove and clear the "Manager" field.