I'm pretty sure, though not positive, that quest makes you join the Mordinos or something. They only say "don't try anything funny" when you are Hated. Joining an NPC factions sounds like the only explanation for your sudden low reputation.
Type the command "~factioninfo" into your client to see what faction you are a part of. If you aren't a part of any, I don't know, you probably have nothing to worry about except reputation. Also, the bad reputation will go back down to Neutral after a few days or a week or two at most. Until you reach Neutral reputation, Unity will attack you in encounters, and no, it was not because you got too close to them; that doesn't make rep drop.