The Rialto Center for the Arts was boiling over with anticipation on Tuesday night (Oct. 24, 2023). Atlanta became just the 7th United States destination for Michelin Guide, and many of our top restaurants and chefs were recognized with a coveted Michelin Star and other banner distinctions. Congratulations to the 5 restaurants that each received one star: Atlas, Hayakawa, Lazy Betty, Mujo and our long time client, Bacchanalia. I also want to recognize a few other clients and former clients of The Shumacher Group who earned distinction. These include Bib Gourmand recipients, Antico Pizza Napoletana, Fishmonger, and Little Bear; Michelin Recommended Restaurant, Delbar; and Jarrett Steiber, owner and chef at Little Bear, who won the Michelin Young Chef Award. Jarrett once told me that great food should not be inaccessible. These Michelin distinctions are a stamp of recognition for what he has been doing for a long time, providing first-class meals at affordable prices.

Jarrett Steiber, owner and chef at Little Bear – Photo: Daniel Varnado/AJC

How Does “Atlanta” Being in the Michelin Guide Warn Us About Problems with Data Analytics?

While traveling back from our annual pilgrimage to Lake Eden in North Carolina last week, my wife, as she so often does, announced within about 30 minutes of our 3.5-hour drive, that it was time for her to go to sleep. The kids were in the back making themselves carsick, but quiet and entertained, so it was time for me to find a mindless podcast. A careful search through my go-to selections came up short, but one I had never listened to kept popping up as suggested so I gave it a go. Today, Explained is a daily news explainer podcast and this particular episode, “America’s Most Successful Downtown?,” seemed like it was somewhat up my alley. I live in a city. I often question Atlanta’s success. I like to be in the know. Let’s learn about another city who must be doing it right. 

Within the first few minutes the show’s hosts begin talking about cell phone data and how a group of researchers at the University of Toronto are using this data to judge the recovery of Downtowns across North America. We all have colleagues and friends who are office brokers and are continuing to have to adapt, pivot, and sometimes change careers because of what the pandemic and other societal changes have done to the office sector in CRE, especially in cities where most large office buildings are sitting well below capacity compared to pre-Covid days. The hosts of the show said something fascinating that really stood out to me: Salt Lake City, Utah is at 139% of its mobile device activity compared to 2019 (counting the number of unique visitors within a given time frame within a given geographical area). Now, Salt Lake City has been growing fairly rapidly for a number of years as they have attracted outdoor adventurers who can work from home, retirees, and large businesses have set up homebase. But 139%? That is amazing, especially when compared to New York City’s recovery rate at 67%, Phoenix at 62%, Nashville- 66%, and Atlanta (it really never seems like we stop growing in population) at a staggering 49%. The podcast goes on to explain what is happening in Salt Lake and my brain just can’t get past the numbers. 

*in this graph and the one below, a score of 100% reflects the city is at the same level as it was in 2019

OK, so Atlanta is just at 49%. I get it. They probably just measured our Downtown, the ever-promised next frontier of development and destination for affordable housing, entertainment, and mecca of sports and culture that will soon define our great city. Unfortunately, we continue to get excited and then our optimism turns to pessimism… they aren’t looking at Midtown or Buckhead, right? 

Here comes the issues with data and data analysis… Ashley Carse, tenured Associate Professor of Human and Organizational Development at Vanderbilt University, says that “it’s important, I think, to distinguish data and analysis, both can have issues. So we might have census data and raise questions about how it was collected. What was asked? What categories were given as choices, etc.” My curiosity got the better of me so I looked up the study. In the case of the data and analysis that I understood to be the premise for the entire podcast episode, what came next really bothered me. 

Suddenly the walls of this incredible story of the most amazing post-pandemic recovery for a city came crashing down. Salt Lake City is nowhere near the top of the graph! Las Vegas is sitting up there and is the only city, at 103%, that has recovered to 2019 numbers. The next is El Paso at 97%; and Phoenix, now at 89%, Nashville- 88%, and Atlanta, at 85% suddenly did not look so bad. Salt Lake, on the other hand, dropped from a 139% recovery rate and #1 of 66 cites to a respectable, but much lower 82%, 23rd out of 66. The podcast first aired on Apple Podcasts on October 13, 2023. The new data from University of Toronto was released just 3 days later on October 16th. What changed? Are the hosts of the podcast going to re-record their show? Should there at least be a correction added to the beginning? Or, for years to come, are people going to listen to this episode and forever repeat the amazing fact they learned over and over at dinner parties, boring friend after friend, not realizing they are regurgitating a number the analysts no longer stand by? (am I the only one that does that?) Maybe I am overreacting (the statistics say that I overreact 173.6% of the time, by the way), but how can we look at a data and know that what we are looking at has been analyzed truthfully? How can we know that the data was appropriately collected? “And then there is how that data is used to answer specific questions. That’s analysis,” Carse continues, “and the analyst has to make choices about a lot of stuff, like scale of analysis or what kinds of information represents some other phenomenon. I think there could be problems at any point along this chain, some unintentional some intentionally misleading.” In this particular case, the analysts at University of Toronto realized that their definition of downtown did not necessarily reflect the actual geographies of cities’ downtowns, so they changed their definition to identify concentrations of jobs and then drew polygons rather than use zip codes and census boundaries- a prime example of how the analyst’s choices plays a role. 

For the record, I don’t think at all that the hosts of the podcast intentionally based their show on incorrect data. Rather, the analysts in Toronto realized a flaw in their methodology and made changes. These changes are reflected in the outcome which are no longer  in tune with the idea that Salt Lake City is some superb outlier, and that we all need to look to their planning department to base our own cities’ recovery.

So why is this obscure data error in a random NPR podcast so interesting?

I use data to make decisions multiple times a day, every day. New software and tools for data collection and analysis have become obtainable for smaller, boutique groups like The Shumacher Group, which allows us to keep up with the CBREs and SRSs of the world. You no longer have to have millions to spend on research and data to provide your client with what they want. 

With newer companies like Placer.AI and Alphamaps battling for customer share within our industry, the cost to companies is slowly dropping as the competition rages on. But how many different services can we all afford, which ones do we have to have, and which ones are on the way out the door? I am not making a grand prediction by saying that in the next couple of years we will see several smaller data companies that we have used for years go the way of the floppy disk.  We all need to subscribe to enough data to attract and retain clients, but now we are faced with understanding what it is that makes our clients believe in a site, market, etc., as more data becomes available and companies require that data to make decisions. We need to collect data and then we need to analyze it in such a way that does not include biases that ultimately lead to the outcome that we prefer. Brokers and business folks are not necessarily statistically literate and data analysts are not necessarily business professionals.

Kelly Gray, Director of Partner Management at InComm Payments, considers the multiple layers that exist when working with clients to make decisions, “[Our Client] is always trying to base everything off quantifiable data which is not an accurate way of looking at things as so much other nuanced data, such as feel, needs to be taken into consideration.”

Feel is interesting. How can you analyze something that can be defined so differently for so many? Your experience in certain situations is certainly different than mine, and both may lead us to a different conclusion. Is it okay to value one person’s experience greater than another’s? Be careful answering that question. In all fairness, though, yes, it is when you are talking about putting a particular business in a particular area if one person’s experiences includes a better understanding of what characteristics will ultimately lead to success. 

As different types of data become available and the use more prevalent, it is important for Commercial Brokers to educate ourselves in how to read, analyze, and use this data to help our clients make the best possible decisions when looking at sites, understanding costs, and forecasting success. It is easy to misinterpret and misuse data and doing so will not only harm our clients, but also harm our reputations. As a matter of fact, I was very ready to lambast all my friends who live outside of I-285, pointing out that they had ZERO, zip, zilch, not one restaurant recognized by the Michelin Guide. ALL of the greatest restaurants in the world are ITP! I can now justify paying so much money in property tax! Oh, wait. Luckily, I held back and read a couple more articles. This go around, Michelin only looked at restaurants within I-285, so we have a lot more restaurants to be considered next time, and I didn’t look too much like a fool.

Understand your data, folks!