How I Built an AI Product as a Designer

AI is playing an increasing role in my design work through the tools and processes that make me more efficient and productive.

But I’m also thinking about how a deeper understanding of AI’s capabilities and mechanisms can help me be a better designer of useful, human-centred products.

I’ve learned a huge amount about AI from reading and exploring, but nothing beats getting hands on and learning by doing.

So, I set myself the challenge of building a useful and fully functioning AI-powered product, even though my coding skills are at a basic level (to be kind).

Here’s what I learned.

AI First Thinking

Just as we moved towards a Mobile First mindset when mobile took off years ago, I believe that we need to be thinking about how we can better use AI to solve human problems and meet human needs in effective ways.

This is more than just improving the UI and usability of the interfaces we use with AI tools (although that’s important too) – it’s about having a deep understanding of how AI can be used to help users accomplish their goals.

Defining the AI Product

Of course, all good design is centred on evidence-based human needs and goals.

But my aim in this mini side project was to create something that gave me direct, hands-on experience with a set of AI methods and technologies, including:

Prompt engineering
Data analysis
Combining data sources
Using LLMs in a product
Text to speech
Text to image
Image manipulation

So I thought about product ideas that would let me explore these things. I’ve learned a lot about the above from reading and experimenting with new tools. But I wanted to get more hands-on.

Gaining more knowledge is also a way to be a better collaborator in multi-disciplinary teams.

An Automated Daily Personal Budget Podcast

I use the great personal budgeting platform PocketSmith to stay on track with my spending and financial goals.

I regularly check in on my monthly spending, scanning through each category against my spending limits. I wondered if there might bean easier way to do this.

I noticed that PocketSmith has an API. Could I do something with this?

Could I use Open AI to look at the budget and give me an analysis of my spending so far? Perhaps make some recommendations? Tell me whereI’m about to go over budget, or have done so? Could it also give me some tips and encouragement along the way?

How about if that was in spoken form? Could it be a daily podcast? One that’s automatically created and uploaded to a private feed in my podcast player?

I had my idea.

A phone showing a daily budget podcast — My Daily Podcast in Apple Podcasts

How My AI Product Works

I thought about the different AI capabilities that I would need to make the product work. Breaking the product down like this helped me identify the different AI tech and services that would give me what I needed.

Here’s how I brought things together:

Connecting to the PocketSmith API to grab this month’s budget
Sending the budget to Open AI asking it to analyse the data and produce a two minute podcast script
Converting the script to speech using Open AI and generating a podcast mp3 file
Creating unique cover art for each episode usingStability AI.
Upsizing the image for use as podcast cover art using Stability AI.
Updating the podcast RSS feed using Python
Uploading the updated feed, episode and cover art to a web server using Python
Adding the RSS to my podcast player as a private podcast

*How I combined AI technologies to produce the podcast*

Removing Barriers

With little coding knowledge, there’s no way I would have even started this project before AI and related tools.

Building this product with AI removed many blockers that would have previously stopped me in my tracks.

I used ChatGPT 4 to generate the code that I needed, just by describing to ChatGPT what I wanted to do.

But I didn’t just describe the entire product idea toChatGPT and ask it to write all the code for me.

I broke it down into the things I needed it to do. This allowed me to run and tests things at every stage along the way. For example, just connecting to the budget API. Then running an analysis on the data with OpenAI, then creating the image, then updating the feed etc. etc.

Using Replit as a platform to build the product was a huge boost. This abstracted a way so much of the complexity that can come with coding such as setting up dependencies correctly.

The Outcome

I did it! I now have a private podcast that is automatically published to my podcast player each morning. It gives me a two-minute overview of my spending in various categories, some points of advice and encouragement.

An iPhone showing a daily budget podcast — The podcast is published automatically each morning based on live data

What I Learned

Just being hands-on and having a product-focus has given a massive boost to my learning and exploration of how AI technologies work and what they can do.

I was able to apply and deepen my understanding of:

Prompt engineering: This is a big topic and there’s a lot to learn here, not all of it relevant to the design workflow. But creating this product allowed me to apply some of the core principles of writing effective, robust and repeatable prompts.
Combining AI technologies: Just thinking about solving product problems with multiple technologies gave me insights into how to effectively integrate and combine them. Such as how the AI analysis of the budget can both write the podcast script and write a suitable prompt for the daily cover art.
Working directly with AI: By building a product and not just using public-facing UIs, my understanding of the power and flexibility of technologies such as text-to-image has really grown.

For me, it’s also shown how easy it can be to be prototyping and building using AI in a way that wasn’t possible before. To explore different ways of using these new technologies to solve problems and meet human needs.