1) A guy's silence doesn't mean he's dumb. In fact, the more talkative you are, the more dumb you get.
2) Not being able to "sakay" in few topics of discussion does not mean the guy is dumb. Some guys might find talking about religion or politics on a date awkward. But when he only wants to talk about things that he's interested in, that also doesn't necessarily mean he's dumb. He's actually egocentric.
3) If he can't get any of your jokes- not one at all, then you're probably right about him being a dumbass.
4) Check his grades. If he's a 25 year old still in high school, then you shouldn't be dating him.