After getting caught using deceptive practices and charging customers more than it should, U.S. Bank has been forced to pay $57 million in consumer relief and penalties.

On Thursday, federal regulators announced that the Minneapolis-based bank is on the hook to pay $9 million in penalties and refund $48 million to the more than 420,000 customers who paid for identity protection and credit-monitoring services that they never received.

According to the Consumer Financial Protection Bureau, U.S. bank sold the "Privacy Guard" and "Identity Secure" services as add-on products for credit cards and other bank products starting around 2003 up until 2012.

"The programs, administered by the bank's third-party vendor, Affinion, required written authorization from customers before the services could begin. Although that authorization in many cases was delayed or not processed properly, the CFPB said customers were billed for the protection as soon as they enrolled," reports USA Today.

As a result, customers acquired fees for services that they never received. In some cases, the charges caused customers to exceed their credit card account limits, and pay additional fees. Meanwhile, other customers were misled into believing that their credit was being monitored for identity theft and fraud.

"We have consistently warned companies about practices related to add-on products, and we will do what is necessary to prevent further harm to consumers," CFPB Director Richard Cordray said.

Due to the CFPB's crackdown, the bank, which is the fifth largest in the nation, was ordered to pay a $5 million penalty fee to the CFPB and a $4 million penalty to the Treasury Department.

The bank released a statement expressing regret for the errors involving Affinion.

"We regret that errors occurred," said a U.S. Bank spokesman, according to CNN Money. "We will be compensating customers who did not receive full services from Affinion, and providing our apology," he said.

The bank also explained that the fraudulent services were sold "as soon as we became aware of the issues with Affinion, we took swift action to protect our customers and ultimately discontinued our relationship with Affinion approximately two years ago."

U.S. Bank's customers who were unfairly charged for these services will be compensated with either a credit to their bank account or a check in the mail.