Generative AI Workshop: Using ChatGPT and Stable Diffusion to Create Comic Strips

Generative AI is becoming a hot topic in recent years, thanks to advancements in machine learning and natural language processing. The technology opens up new possibilities in the creation of chatbots, language models, and even entire comic strips.

DagsHub recently hosted a workshop in collaboration with FourthBrain. The event showcases how to create a comic using ChatGPT to write the strip and Stable Diffusion to generate two panels. Hosts Greg Loughnane and Ali Kadhim, expert MLOps instructors from FourthBrain, walk you through their experiments and emphasize the importance of collaboration and sharing resources in the field of machine learning.

The workshop is divided into three sections, each of which progresses through Ali’s journey of using generative AI to create a comic strip.

Initial Attempt

The first section begins by creating a proof-of-concept. Using standard Stable Diffusion models Ali creates a baseline, from which to compare future iterations.

While the approach doesn’t work particularly well, it’s an important step in the process. It allows him to draw conclusions from his future experiments to determine when he’s on the right track. The Google Colab for this section can be found here.

Fine-Tuning Stable Diffusion

In the second section, Ali iterates on fine-tuning Stable Diffusion using various datasets he creates from existing comic strips. He uses a data-centric approach and makes heavy use of data and model versioning tools such as DVC and MLflow, both of which are integrated with DagsHub.

Fine-tuning Stable Diffusion gets Ali significantly closer to his goal. The Google Colab that shows how he generated this comic strip can be found here.

Deploying a Generative AI App

The third and final section discusses deploying a generative AI app to Hugging Face Spaces. Ali and Greg walk you through setting up a model card, the basic UI for the application, and connecting everything together.

The workshop concludes with the speakers discussing the ongoing work needed to improve the model's ability to replicate the characters consistently and the importance of automating experimentation in a streamlined way. The event demonstrates the power of generative AI and how the appropriate use of existing MLOps tools can help to turn an idea into a working application.

Ali's repo for this workshop is hosted on DagsHub. Feel free to poke around and play with it!