Tax deductions for charity give the IRS the power to decide what qualifies as a charity and what doesn't. The IRS can abuse this power, as it has for example to disqualify Tea Party organizations.

In many issues there are charities on both sides, fighting a zero sum game, for example pro-choice and pro-life charities. If they both have less money the end result in policies being enacted probably won't be any different.

It gives disproportionate power to the few rich to decide which causes deserve funding and it decreases state tax revenue that can be more democratically allocated.

Tax exemption implicitly subsidises the (sometimes foolish) donations of (mostly rich) private donors

Tax exemptions create legal complexity and distort incentives (e.g. big 4 have tax minimisation service lines which is a waste of money and talent). Getting rid of all tax exemptions is a good idea, and removing the charity exemption is a step in this direction.