Adlington Retirement Living was today granted approval for the scheme which it said would become "unviable" if reduced in size.
It follows complaints from residents and councillors over the size of the building, parking, traffic and the risk of flooding at the Wetherby Road site.
However, no objections were raised by statutory consultees.
At a meeting of Harrogate Borough Council's planning committee today, Conservative Robert Windass was one of eight councillors who voted in favour of the plans, while three voted against.
Councillor Windass said: "The applicant has made significant moves to improve this scheme.
"There are no objections from the clinical commissioning group, Environment Agency or highways authority - it is about time we passed this."
Adlington Retirement Living - which is part of the Gladman group - first submitted its plans in March 2020. Since then, it has had one application refused and been told to rethink another twice.
The developers had agreed to remove a fourth storey of the building and also pay for a new footpath on Grimbald Bridge.
It will also pay £40,000 to the NHS North Yorkshire Clinical Commissioning Group (CCC) which previously raised concerns that another care facility in the area could “overload” local health services. The CCG later withdrew its objection once the payment had been agreed.
Despite these changes and agreements, residents had continued to push for the plans to be turned down.
Speaking at today's meeting, resident Steve Benn said: "This development is twice the height of any current building on Wetherby Road. The developer claims it has been to a minimum of 60 apartments to be viable, yet its flagship scheme in Otley is only 48.
"Knaresborough town centre and local facilities are a mile uphill from the site, and residents will be unable, without personal transport, to make it to churches and community events."
The risk of flooding was another concern raised by Mr Benn, as well as councillor Pat Marsh, leader of the opposition Liberal Democrat group, who previously asked for the building to be moved further away from the River Nidd.
Adlington Retirement Living refused to do this after including a flood storage basement in its plans. The company also previously said there is “no record” of the site flooding.
However, residents claim the River Nidd has overflowed at the site as recently as February.
Councillor Marsh also told today's meeting: "I sent every member of this planning committee a photograph of a building literally just down the river from this site which has been badly flooded for the last five years. We know this area floods."
She added: "This scheme is an overdevelopment and I still have very serious concerns about it.
"The developer hasn't done anything to allay those concerns."
By Jacob Webster, Local Democracy Reporter