TORONTO, Canada — Hudson's Bay Co.'s real estate strategy is leaving some investors frustrated.

At its annual meeting in Toronto Tuesday, the owner of Saks Fifth Avenue heard from an exasperated shareholder wondering why the retailer’s not doing more to cash in on hot property markets such as Toronto while it can.

“Are you waiting for us to go into a recession before you sell some of your real estate, or do something with it?" asked Heino Ader, an individual investor who’s held the stock for about two years. He had a suggestion: build condominiums on top of stores, “a cash cow just waiting there for you.”

Hudson’s Bay shares jumped 3.6 percent to C$11.67 in Toronto Tuesday, though they’ve still lost about 22 percent over the past two years. Condo prices are rising in many Canadian cities even as the overall market shows signs of cooling off.

Real estate is already part of the struggling retailer’s turnaround plan. Hudson’s Bay is in talks to sell its prime location in Vancouver and agreed to sell Lord & Taylor’s flagship store in Manhattan for $850 million last year. It’s also cutting other losses, selling flash-sale website Gilt earlier this month to buyer Rue La La.

Asked about real estate initiatives by another investor, Chairman Richard Baker also cited partnerships in some cities with retailers such as TopShop or office-sharing firm WeWork Cos., which is taking over some stores' top floors. Baker came under fire from some investors, including the Ontario Teachers' Pension Plan, over his pay package that totals almost C$55 million ($42 million) including performance-based awards.

It’s not the first time Hudson’s Bay has faced pressure on its real-estate portfolio. It was the recurring criticism of activist investor Land & Buildings Investment Management LLC, until a six-month truce was agreed to last year.

Comments by chief executive Helena Foulkes that Hudson’s Bay is “looking strategically at selling certain properties” failed to convince Ader.

“I want them to do something, not just look at it,” he told reporters after the meeting. “Real estate will not stay hot forever.”

By Sandrine Rastello and Maciej Onoszko, with assistance from Natalie Wong; editors: Crayton Harrison, David Scanlan and Christine Maurus.