Smoking, Gen AI Pilots & Change Management

If you’ve been following AI news, a recent "State Of AI In Business 2025" report from MIT's Project NANDA made a stir last month by claiming that 95% of generative AI pilots are failing. Headlines everywhere from Fortune to Forbes picked this up citing the failing as an explanation for the market's unease regarding AI stocks and discussions of the "AI bubble".

Now these headlines are great at generating clicks and making noise but, like leaving your keys in the ignition or hearing a smoke detector chirp for a new battery, the annoying beeping noise stealing your attention doesn’t actually explain the problem. Similarly, the noise about “95% of gen AI pilots are failing” is a signal for a larger problem.

For the September Swisspreneur newsletter, I went meta and turned MIT’s research project into a little project of my own. In the next 7 minutes, I’m going to outline MIT’s key findings, connect their data to some interesting theory from various streams of psychology and, draw some connections to other real-world problems like the somewhat-audacious reference to smoking in the title.

BUT FIRST, it’s time to point out the obvious that apparently isn’t that obvious…

Pilots are experiments! We expect pilots to fail and if they didn’t fail, we’d probably question whether or not they were really innovative enough. So, to say “pilots are failing” is kind of like saying ,“rain is coming out of the rain clouds”; it’s in the name— pilots are solutions in test phases and failure is one of the possible outcomes of tests. Now, if you’re thinking “maybe, but a great product wouldn’t fail”….remember that some of the most successful products, solutions and businesses were failures first.

Colonel Sanders of the mega franchise KFC famously faced 1,009 rejections before finding a restaurant to accept his fried chicken recipe. Penicillin and Post-it notes (along with countless other epic inventions) were created by mistake. Maybe there hasn’t really been an increase in failure but rather our resistance to failure as inherent to good entrepreneurship?

Okay, so now that the obvious-not-so-obvious has been stated, let’s get to the meat of the story:

Finding # 1 Context NOT Cause

If you retained anything from high school science or intro to statistics (I know, I know it was ages ago), you know that that correlation doesn’t equal causation. For example:

Causation looks like: A → B Smoking causes cancer

Correlation looks like: A ↑ B ↑ As temperatures rise, ice cream sales increase

Simply put, while causation is really effective at pointing the finger at a culprit, saying something is correlated opens up the possibility of multiple factors influencing the relationship between variables and events. In a weird way, correlation gives us a fuller picture of “causation” because it challenges us to acknowledge multiple possibilities.

Now, AI pilots and pilots’ failure are undoubtedly related but the market and media have broadly interpreted the report's 95% failure rate as an indictment of AI technology itself as if A-pilots cause → B-pilot failure. In reality, the report's own conclusions strongly support an alternative interpretation, explicitly stating that the "GenAI Divide does not seem to be driven by model quality or regulation, but seems to be determined by approach".

The report went on to explain that the failure is primarily a reflection of underlying organizational and implementation challenges. So although the headlines tell one story of A → B, the real relationships could be better represented like:

A-pilots ↑ increase X-change pressures in organizations ****which in turn ↓ B-decreases adoption.

Finding # 2 Key Power Groups Are …Key!

Next, like with adopting ANY new technology, or like with quitting smoking, the success of adaption comes down to change behaviour management. In organizations, this means unless an organization takes control of how a new product gets deployed, with things otherwise running smoothly, integration of the pilot will not succeed.

In fact, the article states, the top barriers were overwhelmingly organizational: "unwillingness to adopt new tools" (rated 9), "lack of executive sponsorship" (6.5), and "challenging change management" (6.5). These findings underscore that pilot failures link to organizational design flaws, resistance to change, and learning gaps, rather than fundamental limitations of the AI technology itself.

Now if we take these insights and turn to some organizational theory like David A. Nadler’s (1997) twelve action steps for change we can make further connections. The first step according to Nadler (1997) is to get the support of key power groups; step two is get leaders to model change behaviour. Do you see any connections between the model and the lack of executive sponsorship cited above? Same — These pilots simply never stood a chance of getting to the reinforcement phases of change!

Finding # 3 Barriers OUTWEIGH Drivers Another way to think of change is as a product of a force field between factors contributing to change (drivers) and barriers to changes.

In the end, we are stuck with a classic entrepreneurial dilemma: no matter how innovative and clever a product is, it comes down to the outcomes and experiences it provides to the people who need to use it, the real challenges they face AND importantly, how they feel about using it (are they motivated, frustrated, fearful, neutral)! Or, as UX Geek Sean Gerety said: "The technology you use impresses no one. The experience you create with it is everything”.

[OR as a recent McKinsey article stated: Piloting gen AI is easy, but creating value is hard.]

Maybe you’re reading this thinking “duh, creating value is entrepreneurship 101”. That’s true, but I’m going to challenge you… value alone is not enough! Changing behaviour is HARD and workflow integration needs to go two steps further though #1 reducing barriers and #2 empowering users (similar, related, but not exactly the same).

Why? People know smoking kills them and yet in 2025, the revenue in the Tobacco Products market in Europe still amounts to a whopping US$282.7bn. The top three barriers according to one study? 1 -Craving (translated to a desire to maintain positive experiences with existing ways of work), 2- tried quitting before and it was too hard (installing/downloading, learning and integrating new things takes effort), 3- concerned that I will feel worse not smoking (what if your pilot is worse than the product I already use?).

If people will continue to smoke and kill themselves to avoid doing something hard, imagine how hard it’s going to be to get them to change from their free ChatGPT account that is “good enough” to whatever new shit you’re bringing in?

The Take Away

According to the MIT study and the McKinsey insights BOTH founders and their customers are underestimating the importance of change management when solving problems through products or services; the focus is too often on tools instead of outcomes and integration.

So, how can pilots tip the scales towards success?

Finding 1: failure isn’t explained through direct causation but rather: A-pilots ↑increase X-change pressures in organizations which in turn ↓ B-decreases adoption. Founders, like any good pirates, need to find, understand and intercept at “X”.

Finding 2: driving the product continues after the sale is made, keep users engaged over time OR if you’re an organization invested in using the product, identify the key power groups who will champion change (including you).

Finding 3: Perhaps the alleged 5% of pilots who didn’t fail are the ones that are focused on segments where frustration with the status quo is significantly higher than average (IOW motivation is high) and have built-in ways to reinforce change behaviour through ease of use, creating a positive experience or rewarding user engagement.

What do you think? What factors correlate to the success of GEN AI pilots in the Swiss ecosystem?

Now for the spoiler: Smoking does not cause GEN AI Pilot failure but they are related in such that behaviour change management is a barrier to both smoking cessation and successful adoption of pilots. Ultimately, you can have amazing drivers like: not dying, but if the force of cons outweigh the pros you’re out of luck. It’s all just game of value adds versus effort required to gain that added value.

PS in case you were curious, steps 3-12 of Nadler’s (1997) model are − use symbols and language, − define areas of stability, − surface dissatisfaction with the present conditions, − promote participation in change, − reward behaviours that support change, − disengage from the old, − develop and clearly communicate an image of the future, − use multiple leverage points, − develop transition management arrangements, − create feedback.

And if you love that, check out ADKAR model (coined by Jeff Hiatt): Awareness, Desire, Knowledge, Ability, Reinforcement.

Just a fascinating table outlining some drivers and barriers of smoking cessation from Joshi V, Suchin V, Lim J. (2010) Smoking Cessation: Barriers, Motivators and the Role of Physicians — A Survey of Physicians and Patients.

References

Hiatt, J. (2006). ADKAR: a model for change in business, government, and our community. Prosci.

Joshi V, Suchin V, Lim J. Smoking Cessation: Barriers, Motivators and the Role of Physicians — A Survey of Physicians and Patients. Proceedings of Singapore Healthcare. 2010;19(2):145-153. doi:10.1177/201010581001900209

Nadler, D. A., & Tushman, M. L. (1997). Implementing new designs: managing organizational change. Managing strategic innovation and change, 595, 606.

Written by Helena Hlasova