Don't you have to be a Rain Man to count cards?
Short answer: no.
You're not trying to remember every card during play. You do have to memorize a basic strategy table and some variation index numbers, but you can do that long before sitting down at the table. The mathematical skills used to count cards during a game aren't at all difficult. They involve addition, subtraction, and a little bit of division or multiplication - the sort of math most people learn in grade school.
Playing blackjack well enough to have a small edge over the house is actually pretty easy. A "Rain Man" could learn it, but so can you.