Performance Test Ponderings

What fresh hell is this? Yet another opinion piece on ChatGPT? Well yes and no. This is a little more targeted than that. I’m aiming to focus on the infinitely interesting topic of Performance Testing.

Cutting a long story short, I am curious about the use of AI generally and ChatGPT specifically in Performance Testing. After asking around a bit, I seemed to be hitting more theory than practice. So… in the spirit of DIY (punk that is, not shelving, which I am notoriously bad at) I thought I would try it myself and share with the world. Hence this blog – you’re welcome.

I’m really curious as to how AI can work in Performance Testing. I’ve been lured by the promise of being able to do more with less. I also quite like the tech utopian fantasy of spending happy days funded by UBI whilst painting/cooking/playing sports. Of course ChatGPT is the easiest entry point for seeing how we get on, before I book that “painting for imbeciles” course.

I have a little doubt scratching in my head though. That’s borne out of 30 (yikes) years spent in IT and seeing many, many “revolutionary” technologies fail to really “disrupt” the world as much as they promised. It’s also influenced by my early experience with ChatGPT where it told me that U2’s “The Unforgettable Fire” was one of the best jazz albums of the 80s. It wasn’t a vintage decade for sure, but I think there are better candidates in that category. Actual jazz albums, for example.

So with that all said, how should we proceed? I thought I would ask ChatGPT itself what it could offer. This is ChatGPT 4.0 by the way – I shelled out 20 dollars a month for the premium product. So feel free to buy me a beer or a coffee when you see me.

Anyway, begging aside, this is what ChatGPT said

Here's how I can assist with performance testing:

1. Concepts & Basics: I can explain the fundamentals of performance testing, its types (like load, stress, endurance, spike, volume, and scalability testing), and the reasons for conducting them.
2. Tools & Technologies: I can provide insights and overviews of various performance testing tools such as JMeter, LoadRunner, Gatling, locust.io, and others.
3. Scripting Assistance: If you're writing scripts for a specific performance testing tool, I can help with syntax, best practices, and troubleshooting.
4. Strategy & Planning: I can guide you in defining your performance goals, creating test scenarios, and identifying the metrics that are crucial for your application.
5. Result Analysis: Once you've conducted the tests, I can help interpret the results, understand bottlenecks, and recommend potential solutions.
6. Optimization Tips: Based on your results, I can suggest best practices to optimize server settings, database queries, or application code.
7. Resource Recommendations: If you need deeper insights, I can recommend books, tutorials, and online resources related to performance testing.
8. Terminology Clarification: If you come across any terms or concepts you're unfamiliar with, I can explain them.
9. Integration with CI/CD: I can guide you on how to integrate performance tests into your Continuous Integration/Continuous Deployment (CI/CD) pipeline.
10. Updates: If you're looking for the latest trends, methodologies, or tools in the performance testing domain, I can provide information up to my last training cut-off in January 2022.

Well, I’m so pleased with that I’m already half way to the art supply shop, beret on my head. A couple of things strike me as interesting.

From Concepts and Basis we have this I can explain … types (like load, stress, endurance, spike, volume, and scalability testing). If you ever find yourself having an evening out with a bunch of performance testers and the evening is a bit dull (how could that be?) I suggest spicing things up by asking “what is a stress test?” Just make sure there are no sharp objects or projectile weapons in the room; things can get pretty heated!

The second thing is all this results analysis and optimisation business. My personal experience (and you can read more in my bio, if you’re really bored) is that this kind of analysis needs a collection of multi-disciplinary experts, takes a few weeks, costs a small fortune and can end up with “a broken cable in the SAN” (true story). So I’m really intrigued by that.

Anyway, we will being in earnest next week. I’ll ask ChatGPT to guide me and in the event it runs us into a dead end, I’ll take control. Along the way I shall act like a backseat driver and offer critiques, thoughts and point out where we should have gone left two streets back.

Til then, happy system crashing!

PS The image at the top is from Dall-E 3 with the prompt “a group of performance testers enjoying a night out”. Clearly no-one has mentioned stress testing. Or shaving.