Republicans struggling to find a replacement for outgoing House Speaker John Boehner have their eyes set on Paul Ryan. But if the Wisconsin congressman and 2012 vice presidential nominee can't be convinced to take the job, the race could turn into a free-for-all.

Ryan has hunkered down in his Janesville, Wisconsin home and is reluctantly weighing a possible run, CNN reported. Common wisdom has described the 45-year-old as the perfect candidate to bridge the gap between mainstream Republicans and the party's conservative wing in the House, but far-right media figures are now questioning Ryan's credentials, The New York Times noted.

"Tryouts for speaker continue," Phyllis Schlafly, the founder and chairwoman of the conservative Eagle Forum, said in a statement on Friday. "The kingmakers are so desperate for someone to carry their liberal priorities that they are trying to force Congressman Paul Ryan into a job he does not want."

But Oklahoma Rep. Tom Cole, a member of the House's powerful Appropriations Committee, vigorously defended his Wisconsin colleague, The New York Times noted.

"Anyone who attacks Paul Ryan as being insufficiently conservative is either woefully misinformed or maliciously destructive," Cole said. "Paul Ryan has played a major role in advancing the conservative cause and creating the Republican House majority. His critics are not true conservatives. They are radical populists who neither understand nor accept the institutions, procedures and traditions that are the basis of constitutional governance."

The only official candidates in the race so far, meanwhile, are Reps. Jason Chaffetz of Utah, Daniel Webster of Florida, and Bill Flores of Texas. But unlike House Majority Leader Kevin McCarthy, who dropped his bid for the speakership last week, none of the current contenders represents a heavily Hispanic district, Think Progress pointed out.

House Democrats, meanwhile, used the GOP struggle to score political points on social media, urging Republicans to "end your Game of Thrones so we can get to work on problems facing Americans."