(Update 04-05-2024 BleepingComputer) Panera Bread week-long IT outage caused by ransomware attack
(Update 03-27-2024) Both the app and the website are back, fully functional as of now (this morning). Btw, I cleaned up a few CC in my account (from 6 to 3); also enabled 2 factor authentication for MyPanera account (via SMS, not perfect, but better than nothing). Honestly I still don’t know what happened to Panera’s system. One thing impressed me a bit is their mobile app works out of the box – it’s logged in this morning when I opened the app. I used to work on an app when I was working for the Mastercard, and that feature alone (make sure the user logged in if he/she was logged in before the app update) is a difficult task to do.
(Update 03-26-24) It looks like the website is coming back, but I cannot log in using my old credential either. Forgot password via email or #MyPanera number don’t work either 🙁 Thanks to the Twitter user who reported the website coming back. || A small benefit of this thing is it made me to look up the good old MyPanera number. Another side benefit is for me to rethink the practice of saving credit cards under my Panera account (going forward will probably only do Apple Pay). Also refer to our friend’s new article at SVDaily.
Yesterday morning (3/24), I went to the the new U City Panera Bread as usual, and noticed system is down nationwide: the mobile app and the kiosk didn’t work. But the register still works. Note the new U City is an all digital store and it’s almost full outage – the manager said they have a computer for employee order and I assume he can ring a bit regular customer sales as needed (maybe it’s the 3rd party app such as Uber Eats order instead?). WiFi is down too so I am using iPhone as hotspot. At this time it seems 3rd party apps (e.g., Uber Eats) still work.
I posted it on Twitter and also posted a survey yesterday evening.
This sadly could have more impact to Panera compared to say, companies such as Garmin. Some of my readers may know I am a loyal Panera bread customer, and I regularly went to the Old Olive store (sometimes walk) since year 2005; sometimes I work there using laptop (remote work). So it’s almost 20 years since I regularly go there. I also talked about them quite extensively on my blog.
(In Chinese) 原来Panera网站炸了. Note Panera is popular in Chinese American customers too.
In the News
This incident was not widely reported in the news, due to various reasons: it happened on a Saturday; and Panera is not McDonald’s or Starbucks.
Panera Bread digital channels suffer national outage (restaurantbusinessonline, 03/25/24): quote – But it appeared third-party delivery orders were still coming in to units.
Panera Bread Hit By Massive National Outage (SVDaily, 03/24/24, also the Reddit thread around the outage). I noticed the discussion around Disaster Recovery, and business continuity plan are valid. Back in the days at Mastercard we do those once every year, and it was fairly rigorous. But I don’t see many other organizations do that kind of exercise.
Panera Bread Hit by Nationwide Outage: Cyberattack Rumors Swirl as Digital Ordering Crashes (the Cyber Express).
Note the issue surfaced on Saturday 03/23/24.
Previous Security Issue
Btw, it’s the 1st time Panera Bread website/app/kiosk underwent an incident like this. A few years ago, I do recall Panera Bread had a security related issue, and was remediated quickly. Below are some articles around the issue.
No, Panera Bread Doesn’t Take Security Seriously;
Panera Bread’s half-baked security;
Panerabread.com Leaks Millions of Customer Records
Btw, some reddit discussions on KrebsOnSecurity (Brian Krebs: Wikipedia); again Reddit: What do you think about Brian Krebs as security professional?
Digital Transformation
Panera Bread, here in the St. Louis area they are called St. Louis Bread co., or breadCo as some people call it, is early in its digital transformation effort. The in store kiosks and the online ordering (web and mobile app). Personally I liked the convenience of it, although I do miss the “first name basis” at the counter when I started going to Bread Co almost 20 years ago.
Ownership Change
The company was public, until it was took private in year 2017 (Wikipedia, scroll down a bit, you will notice the 03/24 Computer System issue as well)
Sipper Club (loyalty program) (and in year 2011, I interviewed there once for a dev position related to that, no offer)
I like that too. I recall it was initially called “Coffee+” or something like that, and it was limited free coffee refill. Later it was expanded to soda and other drinks. Note there were controversy around that too. I mean the charged lemonade. Initially I didn’t pay close attention to the caffeine content, until the news break. My wife was not happy that I gave our daughters that drink once or twice.
Pandemic (during pandemic in year 2020?, I interviewed there again for a software architect position, no offer)
I recall in the early days of pandemic, probably in May 2020, we cannot go in the store. The first time I ordered and picked up the food there, when I saw the young girl (masked) brought out the food, I was almost in tears.
Menu Revamp
They are in the process of big revamp of the menu items in the coming April. Then this happened. I still think it’s unlikely menu revamp will cause this. Menu revamp is nothing new at Panera and at other restaurants, Panera has done many times during my 20 years observations with them. I recall once they tried pizza: it didn’t work out very well (this was before the most recent pizza effort). Once they removed the famous Frontega Chicken sandwich, and some fans revolted, and they brought it back. I am a fan of Frontega, btw.
Key People
Ronald M. Shaich: he is the founder of Panera, and I believe he engineered Panera’s buying St. Louis Bread Co., rebranding, and digital transformation and so on. I saw him in person once at the SunsetHills Panera. It seems he is friendly with the rank and file Panera IT people (I saw a guy who interviewed me in year 2011, quite a few years after that).
I also liked Niren Chaudhary, although his tenure is not as long as Ron, once he replied me to my comment in LinkedIn. It was before Pandemic, I commented on the closing of the store in ForestPark way at Central West End, near BJC/Wash U medical school campus.
CIOs, from LinkedIn (former; current/interim).
Panera Bread on LinkedIn