The Financial Ombudsman Service (FOS) has reviewed 185 phone scam cases and found that 80 per cent of the victims were over 55-years-old.
One in five people lost between £20,000 to £49,999 and some victims had lost more than £100,000 through a scam.
In most cases, banks are not liable to replace these financial losses.
Older people are being tricked by scammers pretending to be from their bank or the police who tell them their account is at risk.
Often, they stay on the line and when the victim tries to phone their bank they are still speaking to the scammer.
Sometimes people are tricked into giving away their Pin number and others give their card to a ‘courier’ who takes it to the scammer.
The FSO is advising people to check who they are dealing with and not to respond to any calls, texts or e-mails they are not expecting.
Caroline Abrahams, charity director at Age UK, said: “That anyone would target an older person to defraud them in the first place is abhorrent, but we know that older people are being deliberately targets and can be especially at risk if they are living with dementia and/or cognitive decline.
“Some older people are more vulnerable to fraud because they live along or in isolation, but fraud if something that can happen to any of us.”