I'm not saying that trying to secure an alliance with the Iron Isles was a bad idea, but sending Theon certainly was. Aside from sending someone with a clear conflict of interest, Theon's whole purpose at court was to serve as a hostage. You don't send a hostage home to negotiate on your behalf. It's that simple.