No such conveyance or charge shall be deemed fraudulent, in favor of a subsequent purchaser, who shall have actual or legal notice of the prior conveyance or charge, at the time of his purchase, unless it shall appear that the grantee in such prior conveyance, or person to be benefited by such charge, was privy to the fraud intended.