back down (v.)
in figurative sense of "withdraw a charge," 1859, American English, from notion of descending a ladder, etc.; from back (v.) + down (adv.).