It will be totally unefficient. 6 AG is really bad idea.
I suggest you to forget about toughness, and go for 1 LK.
Put SPECIAL points into max agility, END 8, and one more to INT. Buff a lil bit FA and DOC skill.
Instead of BRD and toughness, choose quick recovery, and some support perks (maybe bonus move if you play TB).
3 CH is useless, just take mentats and you can handle anything.
But please, note: this build will still suck in PvP (same as yours), but at least, you can do PvE without any problems.