A historic Forest hospital which served generations of patients could be demolished and replaced with new homes.