President Barack Obama confronted the immigration system and plans for executive action to Latino attendees at the annual Congressional Hispanic Caucus Institute Gala on Thursday.

On immigration, Obama acknowledged the "deep frustration" many communities are feeling. Obama said he understand the sentiments of families torn apart due to the "broken" immigration system but reassured he will not give up on fixing immigration.

A heckler interrupted Obama's speech regarding the growing reputation of the president's high record of deportations during his administration. Obama told the heckler he was going to address the topic of deportations. Although he didn't immediately address deportation, he spoke on the Senate's bipartisan comprehensive immigration reform legislation of 2013 and how Attorney General Eric Holder sued stated such as Alabama and Arizona for implementing the "harshest" immigration laws.

While the Senate passed the immigration reform legislation, the House of Representatives have yet to debate the bill. Obama referred to the House as the "most uncooperative" in history.

"If House Republicans brought the Senate bill up for a vote today, it would pass today. I would sign it today. And they know it," Obama said to applause by attendees. "But instead, they've been sitting on it for more than a year. They voted to strip DREAMers of new protections and make them eligible for deportation -- not once, but twice they voted that way."

With the influx of unaccompanied children entering the U.S. during the summer, Obama said his administration countered with a "firm message" that resulted in fewer parents sending the children to the U.S. He also mentioned plans to give more children the opportunity to apply for asylum from their native countries and therefore avoid the journey to the U.S.

Obama also name-dropped Speaker of the House John Boehner, R-Ohio, who said he would block a vote on immigration reform for the remainder of 2014. The crowd booed.

"I've said before that if Congress failed to live up to its responsibilities to solve this problem, I would act to fix as much of our immigration system as I can on my own, and I meant what I said. So this is not a question of if, but when," Obama said.

The president said potential immigration executive action could occur between the November midterm elections and the end of the year despite the "scare tactics" by reform opponents.

"They'll use whatever excuse they have to try to block any attempt at immigration reform at all," Obama said. "And we have to be realistic: For any action to last, for it to be effective and extend beyond my administration -- because I'm only here two more years -- we're going to have to build more support of the American people so that it is sustainable and lasting."

Obama said he'll spend the next six to eight weeks discussing the positives of immigration reform and relay its importance for the U.S. economy. Obama called for immigration reform supporters to pressure Congress to pass the Senate legislation since "anything [he] can do can be reversed by the next president."

"To move beyond what I can do in a limited way, we are going to need legislation. And if we want that legislation to happen sooner rather than later, then there's one more thing I need you to do -- I've got to have you talk to your constituents and your communities, and you've got to get them out to vote," Obama said.

Obama then spoke in Spanish, stating, "Si, se puede, si votamos. Yes we can, if we vote."

President Barack Obama's Congressional Hispanic Caucus Institute Gala Speech Transcript: