ChatGPT: The Good, The Bad and The Hyperbole

Part 2: ChatGPT and Written Content

By BC, Chief Technologist | Published: February 15, 2023 in Blog

In our previous blog, we broached the topic of ChatGPT and its use for general search needs. We also referred to BuzzFeed’s decision to use ML tools like ChatGPT to author more of its articles. In this blog we’ll talk a bit more about ML-authored content.

ChatGPT and Written Content 

As technology consultants, we produce many types of written content. Like many other content providers, we’re looking at the BuzzFeed experience and wondering if we should also look to ChatGPT to author such material.

Putting aside all of the implications this means for the career of being a writer, as well as the thorny concerns about the copyright of the material that contributed to the training set of ChatGPT, I can understand BuzzFeed’s choice as a viable strategy for creating easy-to-digest, but relatively milquetoast, content. If your strategy is to create a small amount of thought-provoking content, and supplement that with a larger amount of technically banal articles, using a mix of ML-generated and human-generated content is a viable business strategy. 

Having said that, I think that it’ll be a while before ChatGPT can author content that shows real technical leadership (say, of the sort that Martin Fowler writes). But I don’t discount the idea that it will be able to do so eventually.

Sam Altman, the CEO of OpenAI, argues that the biggest mistake people will make about AI over the next decade is to underestimate its ability to create net new knowledge, rather than just synthesize and regurgitate existing content. I think most people will immediately think about that claim in the context of scientific research, but we should also consider what that means in the context of technology, solution architectures and application development. 

But BuzzFeed is not Martin Fowler, and doesn’t aspire to be Martin Fowler. 

I’m wary about over-reliance on ML-generated content. Google, for example, has already signaled that they do not consider ML-generated content as equivalent to human-generated text and they intend to rank pages with ML-generated content in a way that reflects that. That consideration feels important.

On the other hand, we aren’t as concerned with search rankings for some types of content. For example, if we build an application, we might be asked to provide a FAQ. It’s quite possible that the number of people who end up reading that FAQ will be small, and that we don’t expect people to discover our app because the FAQ shows up in the search results of a particular Google query. So, then what is the value of having humans author that content when ChatGPT can produce an entirely acceptable first draft of such a FAQ, given a reasonable prompt? 

ChatGPT as an inspirational aid and copyeditor

There are other ways to use ChatGPT for content that don’t just reduce to “write this for me.” ChatGPT can be used as an inspirational aid: we can ask ChatGPT to provide, for example, a list of 50 long-tail keywords about Platform Modernization to help shape custom content on that topic.  

ChatGPT can also act as a copyeditor: as technology consultants, we are frequently called upon to author technical content: proposals, solution architectures, and so forth. And it’s also true that the technical leaders who are often essential to create that kind of content are not professional-quality writers.  

For years, large consulting firms have used document polishers—often as an outsourced service in places like India—on important sales proposals and the like (apparently business development folks are also, frequently, not professional writers). But ChatGPT can also provide that service, currently for free.  

I can give these types of prompts to ChatGPT: Edit these five paragraphs to remove the passive voice and make the text punchier”, or Can you make this text more concise?”, or Can you rephrase this in a more tenacious way?” Personally, I find that opportunity really exciting, although it’s true that I have a particular hatred of the passive voice and want to see it die. 

Another use of ChatGPT-generated content involves overcoming the it’s hard to start” problem: sometimes certain tasks lag because the people responsible for doing them waffle about how to begin.  

An agile wisdom that has informed a lot of how we work at Intelliware is that people are often better at reacting to something that they are shown rather than coming up with something ab initio. This can be true of UI pages, screen text or even requirements expressed as user stories. When a domain expert is sluggish about articulating requirements for a particular function, it often produces a faster result to propose a first draft and solicit feedback. One of our analysts refers to this as a strategic use of Cunningham’s Law”: the adage that it’s easier to get the right information, not by asking a question, but by presenting the wrong information. 

I think ChatGPT (as well as other ML tools such as DALL-E and Midjourney) can be productive in these instances. If I need a paragraph of text preceding a form on a web page, I think that a ChatGPT-produced stand-in will produce faster and more specific responses from the domain client than some canned Lorem ipsum” text. A ChatGPT-produced first draft of a user story can produce a speedier and more specific response from the domain expert, than to hector them to cough up the requirements. 

Up Next:

In our next blog, we will talk about how ChatGPT can play a role in coding activities.