Traditional U.S. investors who once flocked to the crypto market have begun heading for the exit.