Shoppers are scanning Manhattan’s brick-and-mortar stores again, in search of wardrobe upgrades for the return to offices and in-person gatherings. Madison Avenue is one retail hub getting left out of the rebound.Foot traffic on the stretch of Madison from 57th to 72nd streets was at just 71 percent of 2019 levels the week of Aug. 8, according to estimates by Orbital Insight. That’s lagging behind Upper Fifth Avenue just a block away, and Soho, which is seeing more shoppers than before the pandemic.Historically, all three districts have beckoned consumers with a taste for luxury, their designer boutiques and flagship stores attracting New Yorkers and global travelers alike. While nearly every retail strip has suffered from the rise of e-commerce over the years, Madison has been especially decimated. Covid-19 restrictions only accelerated the slide, forcing ever more stores to shutter for good.The avenue where now-shuttered Barneys New York was a swanky mainstay for decades now has Manhattan’s highest rate of available retail space, giving shoppers less motivation to stroll by.One reason the area has struggled is that it has little appeal for young people, who “want to be where it’s hip,” according to Ruth Colp-Haber, who runs Wharton Property Advisors Inc.“You’re more likely to meet your friend down in Soho to go to brunch on the weekend than you are to go to a museum on Madison Avenue,” she said. “They don’t want to go uptown — that’s where their parents and grandparents are living.”While streets across the city may be quieter than usual these days, some areas are bouncing back stronger than Madison. On similarly posh Upper Fifth Avenue, from 49th to 60th streets, shopper traffic has recovered to 92 percent of 2019 levels, according to Orbital, which tracks pedestrian activity through mobile phone data and satellites. (Orbital has received funding in the past from Bloomberg Beta, a venture-capital unit of Bloomberg LP.)Downtown in Soho, foot traffic has been hovering around 110 percent of 2019 levels since early July. On a recent Saturday, shoppers swarmed Lululemon, the Apple Store and Uniqlo, and cashier lines at fast-fashion chain Zara’s Broadway store stretched almost to the door. Storefronts in the arty neighborhood have been filling up, thanks in part to large rent discounts.On Madison, meanwhile, the availability rate for retail space was 39 percent in the second quarter, the highest in Manhattan, according to Cushman & Wakefield.All along the avenue, storefronts are advertising closings or relocations. On one stretch, between 66th and 67th streets, nearly every other address is empty. “We’ve loved being part of your community and bringing a smile to Madison Avenue,” reads one sign at Anya Hindmarch, a luxury handbag retailer. “It’s time to move on.”Rents on Madison haven’t declined as much as they have in Soho, which may account for the difficulty in hanging onto tenants and filling empty storefronts.“The reality is that landlords need to recognize that brick-and-mortar retailers were already in trouble before Covid,” said Hoai Ngo, a retail analyst at Bloomberg Intelligence. “Retailers are renegotiating the leases, but I think landlords need to be more realistic about what they can get.”For Alan Sandler, who walks along a stretch of Madison every day on his commute home, the neighborhood is the quietest it’s been in the 40 years he’s lived there.One of the things that bothers him the most: all the smaller, mid-range retailers “that are closed now,” said Sandler, 78. That’s led to small inconveniences, like a lack of affordable places to buy luggage.He gestured to his satchel: “It took me forever to find the one that’s on my shoulder.”By Maria Heeter and Natalie WongLearn more:Saving America’s Most Illustrious Stretch of Retail Real EstateMadison Avenue used to be one of the most popular places to shop in the world. Now, after brands such as Calvin Klein and Proenza Schouler have exited, landlords are struggling to find tenants to replace them.