In anticipation of Ralf D. Müller's session, "Using AI in Software Design: How ChatGPT Can Help With Creating a Solution Architecture", we asked him a number of questions, including his experience with ChatGPT as a tool for solving specific sub-problems in the iSAQB Advanced Level Mock Exam and the role of ChatGPT as a knowledge repository in software architecture tasks.
Taking on the iSAQB Advanced Level Mock Exam using ChatGPT sounds intriguing. Did you simply forward the entire exam task to ChatGPT or did you use ChatGPT more for support with specific sub-problems?
Great question. I didn't simply forward the entire iSAQB Advanced Level Mock Exam to ChatGPT because ChatGPT has a limited context that it can handle, that's not even practical. Instead, I used the model as a support tool for tackling specific sub-problems within the exam. The idea was to utilize ChatGPT as a sparring partner to help brainstorm solutions and validate architectural decisions. Often it just helps to get a starting point via ChatGPT. By breaking down the exam into smaller, more manageable tasks, I was able to generate robust prompts that led to more insightful and targeted responses from the model.
Can you outline a specific scenario where you demonstrate how ChatGPT assists in addressing a challenge from the exam?
I had an enlightening experience when I came across the domain class model in the exam. Since ChatGPT can't interpret graphics, I decided to leverage its text-based capabilities. I fed it the tabular description of the model and asked it to generate a PlantUML diagram. The initial output deviated significantly from the original diagram, leading me to discover that the tabular description itself was incomplete.
While this didn't directly answer an exam question, it revealed a new problem-solving approach. It showed me that ChatGPT could be used not just for direct solutions but also as a tool for uncovering gaps or inconsistencies in existing information.
What were the key insights gained from this interaction and what help can the software architecture expect from using ChatGPT as a sparring partner?
One of the key insights gained from this interaction was the value of iterative problem-solving. ChatGPT excels at providing quick feedback, which allows me to rapidly iterate through different architectural designs and approaches. This iterative process helped me refine my solutions more efficiently than if I were working alone.
Another insight was the model's ability to serve as a "knowledge repository." While it's not a substitute for deep expertise in software architecture, ChatGPT can quickly provide information on a wide range of topics, from design patterns to best practices, thereby filling in gaps in one's knowledge or even offering a fresh perspective.
Conversational interactions with AI can sometimes lead to unexpected or inaccurate responses. What safeguards or practices do you suggest to ensure the accuracy and reliability of the information and suggestions provided by ChatGPT in software architecture tasks?
To make sure the information from ChatGPT is accurate, I treat it like I would any advice from a human expert. I use my knowledge to quickly check if what it's saying makes sense. If I need to be extra sure, I'll look up official documents or other trusted sources to double-check the facts.
It's also worth mentioning that ChatGPT is getting better all the time. A while back, people thought it could only do simple math. But now, if you ask it to calculate something, it will actually show you the right formulas before giving you the correct answer.
So, the main idea is to use chatGPT as a helpful tool but always double-check the information to make sure it's right.
How can the examiner ensure that ChatGPT did not solve the iSAQB real exam task, but the examinee? Will exam questions have to be set differently in the future?
First, I don't think we'll reach a point where AI-generated text can be automatically identified or watermarked. When it comes to the Turing Test, which involves interactive chat rather than a written assignment, machines like ChatGPT are now more likely to be recognized not for their errors, but for their lack of them.
There's an ethical angle to this, of course. AI is a powerful tool, and the question isn't just whether we can use it, but how we should use it responsibly.
This leads us to the core issue: how the role of the software architect will evolve in the future given these advancements in AI.
To address the original question about certification, it's worth noting that the process involves both a written assignment and an oral exam. If I were to rely too heavily on AI for the written part and didn't fully understand the content, that would become glaringly obvious during the oral exam.
This dual-layered approach acts as a safeguard, ensuring that the expertise being certified is genuinely that of the examinee and not the result of AI assistance.
The question that keeps coming up is how Large Language Models will change our professional lives. What do you think, what impact will ChatGPT and related tools have on the role of the software architect in the future?
While there's a common notion that AI will eventually render humans irrelevant, I don't subscribe to this view. This idea has surfaced in the IT world before, and it's proven to be more of a myth than a reality. I see AI as a tool that will enable us to work in new and more efficient ways. It will empower us to build better systems more quickly and to manage increasingly complex architectures.
However, the essence of the software architect's role won't change. Our analytical skills will remain indispensable, as will our ability to communicate effectively and keep the customer's needs at the forefront. In essence, AI will augment our capabilities, not replace them.
Sponsoring a conference is a terrific way to support and connect with our global community of software architects.
