Jason Miller, who President-elect Donald Trump announced this week would serve as his White House communications director, said on Saturday he would not take the job.

Axar.az reports that, Miller, the communications director for Trump’s transition team, said in a statement that his family needed to take priority over career moves. He said he and his wife were expecting their second daughter in January. Trump takes office on Jan. 20.

“This is not the right time to start a new job as demanding as White House communications director,” Miller said.

He said his duties would instead be handled by Sean Spicer, who Trump named his press secretary on Thursday, the same day Miller was announced as communications director. Trump also said in that announcement that spokeswoman Hope Hicks would be director of strategic communications.