A book by Charles Pierce called Idiot America that among other things describes a tour of Ken Ham's Creation Museum might help explain why it does often matter.
Pierce opens with a tour of the Creation Museum in Kentucky, where dinosaurs are depicted, living Flintstone style (some of them have saddles!) in Eden, and being taken on Noah's Ark, two by two, T-Rex and Raptor and the rest, with the other animals and Noah's family.
Dinosaurs with Saddles? Bananas as proof of a creator? If you choose to completely ignore science then you end up getting the Ken Ham's and Ray Comfort's of the world getting picked to head agencies like the Texas state Board of Education or oilmen like George W Bush and Dick Cheney determining environmental policy.
Science is far from perfect but it certainly beats wingnuttery. That is why it matters.