There's something you should know...I don't drive

We live in a completely car-centric society and I don't know what's more difficult: telling people I have a form of epilepsy - or instead, breaking the alien, freakish, horribly unusual news to them that I don't drive...
Met a couple of women at the weekend and arranged to see them this week - and then the horrible happened. (Bear in mind that ep is a hidden condition and they have absolutely no idea that I have it.)
Woman Number 1 gives me driving instructions to her house.
Me: "I don't drive. Are you near a train station or on a bus route?"
WN1: "You don't drive?"
Me: "No, but I'll get to you by public transport."
WN1: "How?"
Me:" I don't exactly know yet but if you're not near a train station I can look up bus routes."
Woman Number 1 looked at Woman Number 2 with a 'that's odd, what-have-we-got-here expression'.
WN2 said: "How did you get here?"
Me: "Walked."
WNs1&2: "Walked!"
I was beginning to find this amusing but also a bit awkward.
Me: "Yes, walked."
They thought about that for a minute...legs as a form of transport. Then -
WN2: "If you don't drive, how do you do your food shopping. I couldn't carry mine."
Me: "Online. It gets delivered."
WN2: "I prefer to do it myself - you can never be sure what you get."
Me: "Yes, I'd prefer to do it myself too but I don't drive so I have to work around it."
Now they have me cornered. Either I let them believe that I choose not to drive because I'm lazy and haven't bothered to learn or I'm banned because of an offence. I could make up a story about being kind to the environment but I'm not going to lie to help them out with their prejudice. So I tell them that the DVLA wouldn't give me a licence because I have a condition where I have a type of seizure - sometimes.
This somehow satisfies the two women. WN1 tells me the numbers of buses that go near her home and WN2 offers, not only a lift home, but also invites me to join her on a her next weekly shop!
I think the car thing is quite funny. People will spend hours bemoaning their weight (too much); diet (when they're going to start one); the gym (when they're going to go now they've bought membership). Then they jump in their car to drive a few yards down the road to buy a bunch of something they shouldn't be eating.
People ask me if my medication helps keep my weight off. Now I think about it, I don't know any fat people with epilepsy. That maybe a gross (pun intended) generalisation. It's not because we pop pills though. It's because, to us, walking isn't a huge exercise programme: it's a way of life! And I don't think we deserve medals for it. We were already born with our reward - feet - it's just that car-drivers rarely use theirs.