Reducing external risk in product development by using Chat GPT

Where I’m ranting on corporate PM, while suggesting ways to reduce market risk by using raw GPT and some ideas as to go along from there.

The Legends-

Steve Jobs used to say – “You’ve got to start with the customer experience and work back toward the technology – not the other way around.”

Steve’s biography tells the real story of a tyrant who listened to none and single handedly created a market from imaginary needs in his head, not from non-existent customer experience.

While Eylon Musk says – “Any product that needs a manual to work is broken.”

Yeah, sure, I’m confident SpaceX is not using manuals…

Mind the Gap-

The point taken is that there is a huge gap between what is perceived to be the customer, and what is the “real need”, as well as taking CEOs’ personal marketing hype with a grain of salt.

Creating a product, especially an engineering one, is a creative endeavor, and the hard part is doing the creative process in an organization. There is another gap between the lonely artist paradigm painting in his attic, and pushing technological innovation inside an organization in an open space environment .

This is why large organizations are continuously buying new product companies, and have investment bodies in the startup ecosystem.

Searching Under the Lamp-

The issue is that corporate thinking is very structured, and HR are thinking in terms of Experience, Roles, Responsibilities and Skills (Sometimes even accounting with personal traits, like leadership?), while the product’s dimensions are measured in the knowledge domain for which its being developed (technology, market constraints, etc.) as well as the clients’ expected needs.

This dimensionality mapping is the cause that if once there was a classical product manager, or project manager if the task was more bespoke and time dependent. Now we have:

1. Product manager that is outbound (marketing dimension)
2. Product owner that is inbound oriented – Focused more on the how.
3. Product success manager – that is, well, like a product manager but more oriented on implementation and receiving of the product on the client side.
4. A PMO that is responsible for the non-functional management tools of the product/project within the organization.
5. A Project Manager that is actually doing planning and hands-on monitoring of the activities.
6. Oh, and then you have the Scrum Master, and whatnot, if you’re doing Agile. 

I’m not promoting maverick operations, but Fred Brooks’ Mythical Man Month book pointed to the diminishing returns of adding resources to a software operation in 1975, so what is this pattern of multiple managers with reduced responsibilities?

I think it all originates from two reasons: 

The first is that it’s hard to get one person equipped with all that is needed to manage all the product aspects. And if you find such a rare person, he’s probably doing something better… 

The second reason is that products are getting more and more complex, despite the famous KISS principles, (Keep It Simple Stupid). This deviation from the KISS principle is partly due to the tendency of systems to get more convoluted with time. Partly due to the second law of thermodynamics and partly because people are not perfect.

So, traditionally one can divide roles to functional and nonfunctional. Functional are all the activities that are related directly to the product’s features, and nonfunctional activities are all the activities that are not directly related, but more like second derivatives, like how to get the right resources, contractual constraints, security policies and so on. 

But this is just an internal mapping between what to do and how to do it. It reduced some of the internal risks involved, yet it still leaves the gap between the alluring market fitment and what you can supply. 

Using ML and statistical simulation to assess market response-

Closing this gap is what it is all about, and the customary MRD to MVP (requirements to most important features) is a step in the right direction that is based on cyclic experimentation and corrections with the costs involved in making errors. But what if we can emulate this stage and thus reduce external risks?

Well there is actually a way – By prompting ChatGPT, which is trained on the “market” data, you can actually get a basic understanding of the market needs, and very soon LLM models will include multimedia referencing. Since LLMs are built with wide factual knowledge, they can emulate a customer, you integrate Monte-Carlo simulation with GPT API, you create a simulated crowd to “test” your product ideas.   

I’m inserting here the dialog between me and GPT as a starting point on a common household refrigerator architure which I know nothing about, as you can see it suggested decision criterions as well as average preferences results. And this is even before the simulation and integration with Wolfram Alpha

My dialog with Chat GPT:

Me: imagine that you’re a customer of a home refrigerator

GPT: As an AI language model, I don’t have personal experiences or physical form, so I can’t be a customer of a home refrigerator. However, I can assist you in understanding the features, benefits, and common considerations when purchasing a refrigerator. Feel free to ask any questions you have, and I’ll be glad to help!

Me: What is your preference regarding doors organization, side by side, freezer on top, or freezer on bottom?

GPT: As an AI language model, I don’t have personal preferences. However, I can provide information on the different door organization options for refrigerators: side by side, freezer on top, and freezer on bottom. Each configuration has its own advantages and considerations, so let’s explore them:

1. Side by Side: In this design, the refrigerator is divided into two vertical compartments, with the freezer on one side and the fresh food section on the other. Side-by-side refrigerators offer a wide space for both the refrigerator and freezer sections. They are suitable for kitchens with narrow spaces or tight layouts. However, the vertical space in each compartment may be limited, making it challenging to store large items like pizza boxes or tall containers.

2. Freezer on Top (Top Mount): This traditional design features the freezer compartment located on the top and the fresh food section at the bottom. Top-mount refrigerators generally provide more usable storage space in both the refrigerator and freezer sections compared to other designs. They are often more energy-efficient and less expensive than other configurations. However, you may need to bend down frequently to access items in the lower refrigerator compartment.

3. Freezer on Bottom (Bottom Mount): This design places the freezer at the bottom and the refrigerator on top. With the freezer located closer to the ground, you can access commonly used refrigerated items without bending down. The refrigerator section is usually more spacious and offers better visibility for frequently accessed items. However, the freezer compartment may require some bending to access frozen items. Bottom-mount refrigerators can be a good option if you use the refrigerator section more frequently than the freezer.

Ultimately, the choice depends on your personal preferences, kitchen layout, and usage patterns. Consider factors such as storage needs, accessibility, and the layout of your kitchen when deciding which door organization is best for you.

Me: So you defined several decision parameters – Space, ease of access, energy consumption, cost, frequency of usage please create a table and score those parameters according to population preferences

GPT: Certainly! Here’s a table that provides a general overview of the parameters you mentioned and assigns scores based on population preferences:

Note: The scoring is based on a scale of 1 to 10, with 1 being the lowest and 10 being the highest preference.

Please keep in mind that these scores are subjective and represent a general population preference. Individual preferences may vary based on specific needs and priorities. It’s always a good idea to consider your own requirements and preferences when choosing a refrigerator configuration.

Me: please add an average score row

GPT: Certainly! Here’s the updated table with an additional row for the average scores:

The average score is calculated by summing up the individual scores for each parameter and dividing it by the number of parameters considered. Please note that these scores are representative and can vary based on individual preferences and priorities.