Smith defends using taxpayer dollars on travelling to U.S. to appear alongside controversial podcaster