I think it would be a good idea to boost HtH somehow. Maybe add AC bonus if both item slots empty, plus maybe reduce AP required to use secondary HtH attacks (Aimed piercing kick to head = 11AP without bonus HtH attacks).
I loved playing FO1&2 with bare fists (and feet), and somehow playing HtH (especially in RT) is a pain in the ass. I know it's very unlikely to take down someone with automatic weapon with just you fists, but I think unarmed skill is a bit useless as it is.